Improving Transportation
Construction Project Performance: Development of a Model to Support the Decision- Making Process for
Incentive/ Disincentive Construction Projects
MTI Report 09- 07
Improving Transportation Construction Project Performance
The Norman Y. Mineta International Institute for Surface Transportation Policy Studies ( MTI) was established by Congress as part of the Intermodal Surface Transportation Efficiency Act of 1991. Reauthorized in 1998, MTI was selected by the U. S. Department of Transportation through a competitive process in 2002 as a national “ Center of Excellence.” The Institute is funded by Con­gress
through the United States Department of Transportation’s Research and Innovative Technology Administration, the Califor­nia
Legislature through the Department of Transportation ( Caltrans), and by private grants and donations.
The Institute receives oversight from an internationally respected Board of Trustees whose members represent all major surface transportation modes. MTI’s focus on policy and management resulted from a Board assessment of the industry’s unmet needs and led directly to the choice of the San José State University College of Business as the Institute’s home. The Board provides policy direction, assists with needs assessment, and connects the Institute and its programs with the international transportation community.
MTI’s transportation policy work is centered on three primary responsibilities:
MINETA TRANSPORTATION INSTITUTE
Research
MTI works to provide policy- oriented research for all levels of government and the private sector to foster the development of optimum surface transportation systems. Research areas include: transportation security; planning and policy develop­ment;
interrelationships among transportation, land use, and the environment; transportation finance; and collaborative labor- management relations. Certified Research Associates conduct the research. Certification requires an advanced degree, gener­ally
a Ph. D., a record of academic publications, and professional references. Research projects culminate in a peer- reviewed publication, available both in hardcopy and on TransWeb, the MTI website ( http:// transweb. sjsu. edu).
Education
The educational goal of the Institute is to provide graduate- level education to students seeking a career in the development and operation of surface transportation programs. MTI, through San José State University, offers an AACSB- accredited Master of Sci­ence
in Transportation Management and a graduate Certificate in Transportation Management that serve to prepare the nation’s transportation managers for the 21st century. The master’s de­gree
is the highest conferred by the California State University system. With the active assistance of the California Department of Transportation, MTI delivers its classes over a state- of- the- art videoconference network throughout the state of California and via webcasting beyond, allowing working transportation professionals to pursue an advanced degree regardless of their location. To meet the needs of employ­ers
seeking a diverse workforce, MTI’s education program promotes enrollment to under- represented groups.
Information and Technology Transfer
MTI promotes the availability of completed research to professional organizations and journals and works to integrate the research findings into the graduate education program. In addition to publishing the studies, the Institute also sponsors symposia to disseminate research results to transportation professionals and encourages Research As­sociates
to present their findings at conferences. The World in Motion, MTI’s quarterly newsletter, covers innovation in the Institute’s research and education programs. MTI’s extensive collection of transportation- related publications is integrated into San José State University’s world- class Martin Luther King, Jr. Library.
The contents of this report reflect the views of the authors, who are responsible for the facts and accuracy of the information presented here­in.
This document is disseminated under the sponsorship of the U. S. Department of Transportation, University Transportation Centers Program and the California Department of Transportation, in the interest of information exchange. This report does not necessarily reflect the official views or policies of the U. S. government, State of California, or the Mineta Transportation Institute, who assume no liability for the contents or use thereof. This report does not constitute a standard specification, design standard, or regulation.
DISCLAIMER
MTI Report 09- 07
IMPROVING TRANSPORTATION CONSTRUCTION PROJECT PERFORMANCE: DEVELOPMENT OF A MODEL TO SUPPORT THE DECISION- MAKING PROCESS FOR INCENTIVE/ DISINCENTIVE CONSTRUCTION PROJECTS
March 2010
Jae H. Pyeon, Ph. D.
Taeho Park, Ph. D.
a publication of the
Mineta Transportation Institute
College of Business
San José State University
San José, CA 95192- 0219
Created by Congress in 1991 Technical Report Documentationocumentationocumentation Page
Report No. 1.
CA- MTI- 10-- 2801
Government Accession No. 2.
Recipients Catalog No. 3.
Title and Subtitle4.
Improving Transportation Construction Project Performance: Development of a Model to Support the Decision- Making Process for Incentive/ Disincentive Construction Projects
Report Date5.
March 2010
Performing Organization Code6.
Authors 7.
Jae H. Pyeon, Ph. D.
Taeho Park, Ph. D.
Performing Organization Report No. 8.
MTI Report 09- 07
Performing Organization Name and Address9.
Mineta Transportation Institute
College of Business
San José State University
San José, CA 95192- 0219
Work Unit No. 10.
Contract or Grant No. 11.
DTRT 07- G- 0054
Sponsoring Agency Name and Address
12.
Type of Report and Period Covered13.
Final Report
Sponsoring Agency Code14.
California Department of Transportation
Sacramento, CA 94273- 0001
U. S. Department of Transportation
Office of Research— MS42 Research & Special Programs Administration
P. O. Box 942873 400 7th Street, SW
Washington DC 20590- 0001
Supplementary Notes15.
Abstract16.
This research presents a project time and cost performance simulation model to assist project planners and managers by providing a complete picture during the Incentive/ Disincentive ( I/ D) contracting decision- making process of possible performance outcomes with probabilities based on historical data. This study was performed by collecting transportation construction project data. The collected project data from the Florida Department of Transportation were evaluated using time and cost performance indices and then statistical data analysis was performed to identify important factors that influence construction project time performance. Using Monte Carlo simulation procedures, this study demonstrated a methodology for developing an I/ D project time and cost performance prediction model. User- friendly visual interfaces were developed to perform the simulation and report results using Visual Basic Application programming. The developed model was validated using additional cases of transportation construction projects.
Based on statistical analysis, this research found that several project factors influence I/ D contracting performance. The important factors that had significant impacts on project performance were the effects of contract type, project type, district, project size, project length, maximum incentive amount, and daily I/ D amount. In conclusion, the developed model applied to I/ D contracting projects will be a useful tool to assist the project planners and managers during the decision- making process and will promote the efficient use of I/ D contracting, which will benefit the traveling public by saving their travel time from construction delays. With additional project data, the developed model can be updated easily and the more data used for the model, the better the accuracy of prediction that can be expected.
Key Words17.
Contracting; Decision support systems; Highway construction; Performance evaluations; Statistical analysis
Distribution Statement18.
No restrictions. This document is available to the public through
The National Technical Information Service, Springfield, VA 22161
Security Classif. ( of this report) 19.
Unclassified
Security Classifi. ( of 20. this page)
Unclassified
No. of 21. Pages
82
Price22.
$ 15.00
Form DOT F 1700.7 ( 8- 72)
Copyright © 2010
by Mineta Transportation Institute
All rights reserved
Library of Congress Catalog Card Number: 2009943713
To order this publication, please contact the following:
Mineta Transportation Institute
College of Business
San José State University
San José, CA 95192- 0219
Tel ( 408) 924- 7560
Fax ( 408) 924- 7565
email: mti@ mti. sjsu. edu
http:// transweb. sjsu. edu Acknowledgments
The authors would like to express their sincere gratitude to the Mineta Transportation Institute for the financial and administrative support that made this research possible. The authors are especially grateful to Sorawut Srisakorn, Research Assistant, for his constructive assistance. Finally, the authors would like to thank the California Department of Transportation ( Caltrans) and the Florida Department of Transportation for providing valuable inputs for this research.
The authors also thank MTI staff including Research Director Karen Philbrick, Ph. D., Director of Communications and Special Projects Donna Maurillo, Research Support Manager Meg Fitts, Student Research Support Assistant Chris O’Dell, Student Publications Assistant Sahil Rahimi, Student Graphic Artists JP Flores and Vince Alindogan, and Student Webmaster Ruchi Arya.
Additional editing and publication productions services were performed by Editorial Associate Catherine Frazier.
Mineta Transportation Institute
i
Table of Contents
EXECUTIVE SUMMARY 1
Background and Objective 1
Overview of Methodology 1
Research Outcomes 1
INTRODUCTION 3
Research Background 3
Research Objective and Scope 4
Research Methodology 4
LITERATURE REVIEW 7
I/ D Project Selection 7
I/ D Contracting Evaluation 14
Summary of Literature Review 18
DATA COLLECTION 21
I/ D Project Data 21
I/ D Contracting Database Construction for Analysis 21
DATA ANALYSIS 25
Statistical Analysis Process 25
Evaluation of Project Performance 26
Factors Influencing Project Performance 27
Summary of Data Analysis 38
DECISION SUPPORT MODEL DEVELOPMENT 41
Database Update Module 42
Performance Simulation Module 43
MODEL VALIDATION 53
Project Data for Validation 53
Validation Method and Results 53 Mineta Transportation Institute
Table of Contents
ii
CONCLUSIONS AND RECOMMENDATIONS 65
Conclusions 65
Recommendations and Limitations 66
Appendix A: Dataataata Classificationlassification and CodinG Tables 67
APPENDiX B: BetaBeta Distribution Parameters 69
ABBREVIATIONS AND ACRONYMS 71
Bi
bliography liography 73
About the Authors 77
Peer Review 79 Mineta Transportation Institute
iii
List Of Figures
Model Development Process Flowchart 51.
Selection Factors of Five Most Frequently Used ACMS 92.
I/ D Implementation Flowchart 103.
A + B Average Time Savings 154.
Oregon DOT I/ D Project Size by Date 175.
Box Plot of Contract Type Variables 286.
Box Plot of Project Type Variables 307.
Box Plot of District Variables 318.
Box Plot of Project Size Variables 339.
Box Plot of Project Length Variables 3410.
Box Plot of Maximum Incentive Amount Variables 3611.
Box Plot of Daily I/ D Amount Variables 3812.
Flow Chart of I/ D Performance Simulation Model Development Process 4113.
Flowchart of Monte Carlo Simulation Procedures 4514.
Main Page of I/ D Contracting Decision Support Model 4715.
Project Variable Selection Dialog Box for Project FIN 412481 4816.
Performance Index Selection Dialog Box 4817.
Report of Project Performance Simulation Results for Project No. 412481 4918.
Histogram of OTPI Simulation Results for Project No. 412481 5019.
Cumulative Curve of OTPI Simulation Results for Project No. 412481 5120.
Tornado Graph of 21. OTPI Simulation Results for Project No. 412481 51
OTPI Simulation Case Study Results 56 22.
PTPI Simulation Case Study Results 5823.
OCPI Simulation Case Study Results 6024.
PCPI Simulation Case Study Results 6225. Mineta Transportation Institute
List of Figures
iv Mineta Transportation Institute
v
List of Tables
1. Most Frequently Cited Influencing Parameters for Selection of ACMs 8
2. Advantages and Disadvantages for I/ D Contracting 11
3. Categorized Project Candidates Used for I/ D Project Selection in Minnesota 12
4. Project Sizes and Types Recommended by Ohio DOT 13
5. I/ D Contracting Methods with Recommended Project Situation in South Dakota 14
6. Average Time Savings/ Overruns by States: A+ B and A+ B with I/ D 14
7. Summary of I/ D Project Selection Criteria for Good Candidates 19
8. FDOT I/ D Contracting Project Data Sample 22
9. Summary of Construction Projects by Contract Types 23
Summary of Construction Projects by Project Types 210. 4
ANOVA and Tukey Test Results of Contract Type Variables 211. 9
ANOVA and Tukey Test Results of Project Type Variables 312. 0
ANOVA and Tukey Test Results of District Variables 313. 2
ANOVA and Tukey Test Results of Project Size Variables 14. 33
Two Sample t- Test Results of Project Length Variables 315. 5
ANOVA and Tukey Test Results of 16. Maximum I/ D Amount Variables 36
ANOVA and Tukey Test Results of Daily I/ D Amount Variables 317. 8
Summary of Significant ( S) or Non- significant ( NS) Factors by Indices 318. 9
Project Performance Summary by Contract Types and Project Types 419. 0
Input Data Used in OPTI Simulation 520. 4
I/ D Amount Achieved by Contract Types 521. 5
OTPI Simulation Results 522. 7
PTPI Simulation Results 523. 9
OCPI Simulation Results 624. 1
PCPI Simulation Results 625. 3
Work Type Codes 26. 67
Work Mix Classification and Coding 27. 67 Mineta Transportation Institute
List of Tables
vi
Performance Index Sample Data28. 69
Parameters and Weightings of Selected Project Variables 7029. Mineta Transportation Institute
1
EXECUTIVE SUMMARY
Background
And Objective
Incentive/ Disincentive ( I/ D) contracting, a well- known transportation construction contracting method, is designed to minimize the disruption of traffic flow in highway construction projects. Construction project planners and managers have used I/ D contracting as one of their management tools to achieve their projects’ objectives. As a result, I/ D contracting has played an important role in improving project time performance. More than 35 state transportation agencies ( STAs) have implemented I/ D contracting to improve contractors’ project time performance in transportation construction. Incentives have been used specifically to encourage the early completion of highway construction projects.
I/ D contracting experiences in many states have been evaluated in terms of time and cost performance. It has been found that there were substantial project time savings from many project cases. However, it has also been reported that there have been many inefficient cases using I/ D contracting for various transportation construction projects. These inefficiencies can often be attributed to a poor understanding of the factors that affect the suitability of using I/ D contracts. Therefore, a better understanding of the relationships among such factors as contract types, project types, project sizes, project locations, incentive amounts, and other similar factors is key to providing clear guidance for the better use of I/ D contracting.
The purpose of this research project is to develop a model to enhance the decision- making process for the selection of I/ D projects. The proposed decision- making model would be a useful tool to efficiently assist transportation construction project planners and managers to become more knowledgeable and effective in their I/ D contracting decision- making process. Eventually, the efficient use of I/ D contracting will benefit the traveling public by saving their travel time and money from construction delays.
Overviewverview of Methodology
This research was performed by collecting transportation construction project data. The collected project data from the Florida Department of Transportation ( FDOT) were evaluated using time and cost performance indices and then statistical data analysis was performed to identify important factors that influence construction project time performance. Using beta distributions of the input variables for the key factors, a decision support model was developed for prediction of I/ D project time and cost performance. Finally, a new set of I/ D contracting project cases was used to validate the developed decision support model.
Research Outcomes
This research investigated I/ D contracting projects in transportation construction and developed a project performance decision support model to assist project planners and managers during the decision- making process by providing a complete picture of possible performance outcomes with probability based on historical data. Although 100% accurate Mineta Transportation Institute
Executive Summary
2
prediction cannot be guaranteed, the outcome of this research will at least provide the decision makers with better understanding of project factors that influence I/ D contracting project time and cost performance as well as systematic tools that allow them to learn lessons from their previous I/ D contracting experience.
Outcomes of individual projects are affected by various factors. Based on statistical analysis, this research has found several project factors influencing I/ D contracting project performance as follows:
The important factors that had significant impacts on project time performance are • contract type, project type, district, project size, and daily I/ D amount.
The important factors that had significant impacts on project cost performance include • contract type, district, project size, project length, maximum incentive amount, and daily I/ D amount.
This study demonstrated a methodology for developing an I/ D project time and cost performance prediction model using Monte Carlo simulation. User- friendly visual interfaces were developed to perform the simulation and report results using VBA programming. The developed model was validated using 30 additional project cases of transportation construction. In summary, more than 93% of cases were fallen within the predicted performance range. In comparison to the broad range of the historical performance index data set, the performance prediction range of simulation results showed much narrower range ( i. e. 15 to 49% of the historical data range) in order to predict the actual value for each case.
In conclusion, the developed model applied to I/ D contracting projects will become a useful tool to assist the project planners during the decision- making process and will promote the efficient use of I/ D contracting, which will benefit the public by saving their travel time from construction delays. With additional project data, the developed model can be updated easily and the more data used for the model, the better the accuracy of prediction that can be expected. Mineta Transportation Institute
3
INTRODUCTION
Research Background
Transportation construction activities frequently require a reduction in road capacity, so motorists as well as adjacent businesses must endure the delays, costs, and inconveniences associated with transportation construction. Road congestion caused by construction increases travel time, vehicle operating costs, road accidents and air pollution. Recognizing the problems that construction can produce, the Federal Highway Administration ( FHWA) has continuously sought ways to minimize the negative impacts from construction operations. One key aspect has been to seek improvements in construction project performance and, more specifically, to accelerate project completion whenever possible.
Incentive/ Disincentive ( I/ D) contracting, a well- known transportation construction contracting method, is designed to minimize the disruption of traffic flow in highway construction projects. Construction project planners and managers have used I/ D contracting as one of their management tools to achieve their projects’ objectives. As a result, I/ D contracting has played an important role in improving project time performance. More than 35 State Transportation Agencies ( STAs) have implemented I/ D contracting to improve contractors’ project time performances in transportation construction. Incentives have been used specifically to encourage the early completion of highway construction projects.
I/ D contracting experiences in many states have been evaluated in terms of time and cost performance ( Herbsman 1995, PinnacleOne 2004, MnDOT 2005, Ellis and Pyeon 2005, AASHTO 2006, Ellis et. al. 2007). It has been found that there were substantial project time savings from many project cases. However, it has also been reported that there have been many inefficient cases using I/ D contracting for various transportation construction projects. For instance, many contractors were able to achieve maximum incentives without reducing the original contract time since the incentives were generally paid based on the extended contract duration, which included time extensions, supplemental agreement days, and weather days. These inefficiencies can often be attributed to a poor understanding of the factors that affect the suitability of using I/ D contracts. Therefore, a better understanding of the relationships among such factors as contract types, project types, project sizes, project locations, incentive amounts, and other similar factors is key to providing clear guidance for the better use of incentive contracting ( Pyeon 2005).
I/ D for Early Completion
Until the mid- 1980s, the FHWA had a firm policy based on the belief that “ the FHWA should not have to pay ‘ extra’ just to have a project completed early” ( FHWA 1989). However, the new policy which allows participation in “ bonus payments for early completion” was established in the late- 1980s. This policy was partially based on the evaluation outcome of National Experimental and Evaluation Program Project # 24 showing that I/ D provisions are an important cost- effective management tool for a construction project. The FHWA published a technical advisory report titled Incentive/ Disincentive for Early Completion in 1989 for providing “ guidance for the development and administration of I/ D provisions for early completion on highway construction projects or designated phase( s).” Mineta Transportation Institute
Introduction
4
The FHWA advisory defined the I/ D provision as “ a contract provision which compensates the contractor a certain amount of money for each day identified critical work is completed ahead of schedule and assesses a deduction for each day the contractor overruns the I/ D time.” It was also recommended that the use of I/ D provisions be limited to “ those critical projects where traffic inconvenience and delays are to be held to a minimum.” With regard to the I/ D dollar amounts, it was recommended that the amounts be based upon cost estimates of the following factors: traffic safety, traffic maintenance, and road user delay costs.
A clear distinction between I/ D provisions and liquidated damages was mentioned in the FHWA’s Contract Administration Core Curriculum Participant’s Manual and Reference Guide ( FHWA 2008). The functioning mechanisms of I/ D provisions and liquidated damages are similar in that a penalty is charged when the contractor fails to complete the project on time. However, the purpose of each is different in that liquidated damages are designed to recover the STA’s construction oversight costs but I/ D provisions are designed to recover damage costs to the road users for delayed completion. In addition, I/ D provisions are intended to motivate the contractor to complete the work on time, or earlier, by proposing incentives.
Research Objective and Scope
The purpose of this research project is to develop a model to enhance the decision- making process for the selection of I/ D projects. The proposed decision- making model would be a useful tool to effectively and efficiently assist state and federal construction project planners and managers to become more knowledgeable and effective in their decision- making. Eventually, the efficient use of I/ D contracting will benefit the traveling public by saving their travel time and money from construction delays.
In order to achieve the objectives of this research, this study aims to accomplish the following tasks:
To collect I/ D transportation construction project data; 1.
To evaluate project performance for each collected project; 2.
To perform data analysis to identify important factors that influence I/ D project 3. performance;
To develop a model to support decision- making process for the selection of I/ D 4. projects;
To validate that model. 5.
Research Methodology
In this section, a methodology is described for developing a decision support model for selection of I/ D contracting to assist project planners and managers. First, research was performed by collecting transportation construction project data. Second, collected project data were evaluated using time and cost performance indices and then statistical data analysis was performed to identify important factors that influence construction project time performance. Third, using beta distributions of the input variables for the Mineta Transportation Institute
Introduction
5
key factors, a decision support model was developed for prediction of I/ D project time and cost performance. Finally, additional 30 I/ D contracting project cases were studied using the developed decision support model and the results of the case studies were compared with actual performance results to validate the model. The cross- functional flowchart below ( Figure 1) briefly illustrates the model development process.
Model Development Process FlowchartFigure 1 Mineta Transportation Institute
6 Introduction
Mineta Transportation Institute
7
LITERATURE REVIEW
There have been various incentive plans used for transportation construction projects. They can be categorized into three groups: time- based incentives, cost- based incentives, and performance- based incentives. Christiansen ( 1987) recommended that financial incentive plans are more effective than non- financial incentive plans. Abu- Hijileh and Ibbs ( 1989) informed that the use of bonus- only incentives was more effective than the use of penalty- only. The design and implementation of the time- based incentive plans are relatively simple and economical. ( Abu- Hijileh and Ibbs 1989) Therefore, the time- based incentive contracting for early completion of work has been most frequently used in highway construction. In this research, only I/ D contracting for early completion was studied.
In this chapter, issues regarding guidance for I/ D project selection and evaluation for I/ D project performance have been reviewed and summarized. The literature review was performed by searching published papers, manuals, and reports on I/ D contracting processes and evaluations. State- of- the- art information on I/ D contracting from several STAs was obtained and then useful information for selection and evaluation of I/ D contracting was summarized by states.
I/ D Project Selection
The FHWA encouraged STAs to develop their own I/ D project selection criteria for the effective implementation of I/ D provisions. Many STAs developed general guidelines for their states based on the FHWA’s I/ D project selection guidelines. The selection criteria for I/ D contracting obtained from major STAs which frequently used I/ D contracting has been summarized in this section.
According to the FHWA technical advisory, it was recommended that the use of I/ D provisions should not be used routinely and should be limited to “ the projects that severely disrupt highway traffic or highway services, significantly increase road user costs, have a significant impact on adjacent neighborhoods or businesses, or close a gap, thereby providing a major improvement in the highway system.” During early project development, it is important to select I/ D projects as early as possible. In order to guide STAs in identifying I/ D projects early, the characteristics related to projects appropriate for the use of I/ D provisions were suggested in the FHWA advisory report as follows ( FHWA 1989):
High traffic volume projects, generally in urban areas;•
Projects that will complete a gap in the highway system;•
Major reconstruction or rehabilitation on an existing facility that will severely disrupt • traffic;
Major bridges out of service; or•
Projects with lengthy detours.•
The most recent research regarding selection of alternative contracting methods ( ACM) including I/ D was performed by Anderson and Damnjanovic ( 2008). They summarized the up- to- date practice of selecting I/ D contracting in the NCHRP synthesis 379 report Mineta Transportation Institute
Litereature Review
8
entitled Selection and Evaluation of Alternative Contracting Methods to Accelerate Project Completion. The authors performed an online survey to the members of the AASHTO Subcommittee on Construction and reported that thirty agencies responding to the survey had used I/ D contracting. According to the survey results, I/ D contracting played a positive role to improve project time performance. However, the results indicated that project costs might be increased by using incentives. The authors explained that the project cost increase might be tolerable “ if accompanied by a reduction in road user cost ( RUC) as a result of early project completion” ( Anderson and Damnjanovic 2008).
With regard to the perceptions about I/ D contracting among the respondents, they summarized the survey responses based on the respondents’ own opinions and the STAs’ experiences. The most important advantage of I/ D contracting was early or on- time project completion. However, many respondents cited several major disadvantages ( Anderson and Damnjanovic 2008): 1) construction cost increase when incentives were used, 2) the potential for reduced quality by accelerating construction process, 3) problems regarding utility conflicts, and 4) potential increase in contractor disputes for change orders.
In addition, Anderson and Damnjanovic ( 2008) used surveys to investigate influencing factors for selection of ACM including I/ D contracting. Initially, they summarized the four most commonly named influencing factors then asked each respondent to choose and/ or add one or more of governing factors for selection of each ACM. Influencing factors named most frequently for selection of ACMs including I/ D contracting methods were listed with descriptions in Table 1.
Most Frequently Cited Influencing Parameters for Selection of ACMsTable 1
( Source: Anderson and Damjanovic 2008)
Influencing Factors
Descriptions
Project Size
Typically assessed in terms of the estimated cost of a project in dollars
Project Type
Typically assessed in terms of preservation ( seal coats, thin overlays), rehabilitation ( thick overlays), reconstruction projects ( full replacement), and new construction
Project Complexity
Typically assessed in terms of project location, such as urban or suburban, in combination with a number of different components that defines project complexity, such as a combination of pavement and structures construction, utility conflicts, railroad crossings, significant traffic control requirements, and so forth
Critical Completion Date
Typically assessed in terms of requirements to complete a project faster as influenced by issues such as level of traffic disruption or meeting a target date ( e. g., completion before a holiday or within one construction season)
The authors reported the survey results based on “ the percentage of respondents citing the factor” in Figure 2. As shown in Figure 2, approximately 90% of respondents answered critical completion date as the most dominant factor in selecting I/ D contracting. Approximately 52% identified project complexity as the driving factor for selection of I/ D Mineta Transportation Institute
Litereature Review 9
contracting. Project type ( app. 38%) was ranked third followed by project size ( app. 27%) and other factors ( app. 13%).
Figure 2 Selection Factors of Five Most Frequently Used ACMs
( Source: Anderson and Damnjanovic 2008)
Another comprehensive research for I/ D contracting experience among various STAs was performed by Sillars and Leray ( 2007) and a summary process for executing I/ D contracting in construction was proposed. They explained that the proposed model was similar in format to a model developed by Anderson and Russell ( 2001) as guidelines for warranty, multi- parameter, and best value contracting in the NCHRP Report 451. The proposed model included the different phases of the project life cycle and showed the stepwise procedures of I/ D contracting implementation for STAs. The model for I/ D contracting implementation is illustrated in Figure 3. Mineta Transportation Institute
10 Literature Review
Figure 3 I/ D Implementation Flowchart
( Source: Sillars and Leray 2007)
Since the FHWA provided the general I/ D guidance for STAs in 1989, many agencies have developed their own guidelines for selection of I/ D projects. Some of them have made up their own selection criteria and contracting manuals. Others developed their I/ D contracting guiding principles by expanding the original FHWA guidance. In the following section, useful information for selection of I/ D contracting was summarized by states.
California
California’s Department of Transportation, Caltrans, recommended that I/ D provisions be applied only for projects with a larger RUC than $ 5,000 per day in a manual entitled Project Delivery Acceleration Tool Box: Improvements to the Project Delivery Process ( Caltrans 2006). In terms of the minimum RUC recommendation for selection of I/ D projects, it was found that several states required a minimum RUC ( Caputo and Scott 1996): $ 1,500 for South Dakota, $ 2,000 for North Carolina, and $ 3,000 for New York.
According to Caltrans’ Innovative Procurement Practices prepared by Trauner Consulting Services, project characteristics suitable for I/ D contracting were described as follows ( Trauner 2007): Mineta Transportation Institute
Literature Review 11
Projects requiring traffic restrictions, lane closures, or detours that would otherwise • result in high user impacts ( e. g., construction on major roadways, bridges, or interchanges having a high ADT; projects involving temporary lane, ramp, or bridge closures; emergency repair work).
The project is relatively free of third party coordination concerns ( e. g., utility, railroad, • environmental issues, public opposition) that could affect the bid letting date or the project schedule.
The I/ D amount results in a favorable cost/ benefit ratio to the traveling public ( i. e., the • benefit to the highway user exceeds the I/ D amount, and this amount is high enough to motivate a contractor to accelerate).
The agency has the ability to estimate the I/ D time based on expedited production rates • for similar work, historical records, or CPM scheduling.
Emergency contracts.•
In addition to the above guidelines, Trauner identified a qualitative evaluation of advantages and disadvantages for I/ D contracting as shown in Table 2.
Advantages and Disadvantages for I/ D ContractingTable 2
( Source: Trauner 2007)
Advantages
Disadvantages
Significantly reduces project time1.
Encourages contractors to use 2. time- saving means and methods to accelerate construction
Minimizes cost and time impacts 3. to the traveling public for projects having high ADT
Shifts more risk to the contractor for 4. providing the optimum combination of time, cost, and efficient planning and management of the work
Higher bid costs and project costs1.
Acceleration may over- extend agency and 2. contractor personnel ( however, the associated costs may be offset by the overall shorter construction duration).
3. Acceleration could compromise project quality. However, I/ D projects may also motivate contractors to perform work correctly the first time to avoid time- consuming rework efforts.
4. The agency bears the risk of accurately estimating the critical I/ D time and not delaying the I/ D date. Agencies have reported that contractors may complete the I/ D work and earn an incentive without expending extra effort and that contractors have earned incentives even when the project has been delayed.
5. Agencies have reported that disincentive payments are difficult to recover.
Florida
Florida Department of Transportation outlined the I/ D contract selection in the document entitled Alternative Contracting User’s Guide. In Florida, I/ D contracting may be a stand- alone method, or may be applied to other alternative contracting techniques such as A+ B, No Excuse Bonuses, Liquidated Savings, Lane Rental, Design- Build or any combination ( FDOT 1997). For selection of I/ D projects, urban reconstruction and bridge type projects Mineta Transportation Institute
12 Literature Review
were recommended as good candidates. However, it was not limited to the application of only those projects, but recommended to be applied for any projects that need to meet a specific completion date ( FDOT 2000).
Minnesota
Minnesota Department of Transportation ( MnDOT) developed innovative contracting guidelines in selecting I/ D contracting projects. The selection criteria for I/ D contracting were detailed by recommending good candidates and poor candidates to be considered early in the I/ D selection process. The categorized candidates with project descriptions were listed in Table 3.
Categorized Project Candidates Used for I/ D Project Selection in
Table 3Minnesota
( Source: MnDOT 2005)
Category
Project Descriptions
Good Candidates
Projects with high road- user or business impacts•
Bridge replacement projects•
Detour projects•
Unban pavement rehabilitation projects•
Interstate ( high volume) projects with major traffic impacts•
A+ B projects•
Bridge rehabilitation projects•
Projects with commitments to open a roadway as quickly as possible•
Poor Candidates
New construction projects with minimal impacts to road users•
Projects where right- of way or utilities are not clearly identified•
Traffic Management System•
Steel fabrication•
Landscaping•
Ohio
The Ohio DOT’s Innovative Contracting Manual published in 2006 provides general guidelines for selection of I/ D projects. It recommends that the major consideration for selecting I/ D contracting be based on the project, or a portion of the project, causing a significant delay or impact to the road users ( Ohio DOT 2006). Ohio DOT not only took project types into consideration but also project size as important factors for selecting I/ D projects. All time- sensitive projects and interstate lane closure projects are typical I/ D projects at all project sizes.
Ohio DOT further provided various project types in detail for the purpose of I/ D project selection requiring the district to execute some vital studies to verify “ if a potential innovative contracting method is truly appropriate for the specific project” ( Ohio DOT 2006). Table 4 shows project sizes and types recommended by Ohio DOT. The following criteria are used for I/ D selection guidance in Ohio ( Ohio DOT 2006):
The project or a portion of the project results in a significant delay or impact to the road • Mineta Transportation Institute
Literature Review 13
users.
The Department must have a good understanding of the construction time needed to • complete the Incentive/ Disincentive portion of the project.
Project Sizes and Types Recommended by Ohio DOTTable 4
Project Size
Recommended Project Type
Small Projects
Bridge projects or bituminous resurfacing
Mid- Level Projects
Interstate resurfacing, or minor rehabilitation
Mega Projects
Corridor reconstruction or interstate rehabilitation
All Project Sizes
Time- sensitive projects:
New Construction – Relocation•
Major Reconstruction •
Major Widening •
Minor Widening •
New Bridge/ Bridge Replacement •
Four- Lane Resurfacing & Overlays•
Bridge Rehabilitation, Repair & Widening •
Bridge Painting •
Culvert Construction, Reconstruction or Repair•
New Interchange •
Intersection Upgrade•
South Dakota
In order to identify a candidate project for early completion during or immediately after the preliminary design, Caputo and Scott ( 1996) recommended the following project selection criteria for implementing time- based innovative contracting methods such as I/ D, Cost plus Time ( A+ B) , A+ B with I/ D, and Lane Rental in South Dakota:
High traffic volumes, with traffic restrictions, or lane closures resulting in road user cost • estimates in excess of the liquidated damages for the project;
Long detours causing delay in excess of 10 minutes;•
High accident rates or safety concerns during construction;•
Potentially significant impacts to the local community or economy; or•
Projects coordinated with special events.•
After identifying candidate projects and estimating road user costs, the recommended procedures for selecting innovative contracting were to identify potential impacts, re- evaluate project by finalizing RUC, estimate time, choose a contract method, and develop special provisions. In case of no severe impact on the bidding date or the critical schedule, they recommended an innovative contracting method for more detailed project situations shown in Table 5. Mineta Transportation Institute
14 Literature Review
I/ D Contracting Methods with Recommended Project Situation Table 5
in South Dakota
( Source: Caputo and Scott 1996)
Contracting Methods
Recommended Conditions
I/ D
RUC is high, and the monetary benefit equals or exceeds the incentives paid to the contractor to finish early;
It is in the public interest to complete the project as soon as possible, or by a specific completion date; and
The Department can estimate contract time based on similar projects or CPM scheduling.
A+ B with I/ D
RUC is high, and the monetary benefit equals or exceeds the incentives paid to the contractor to finish early;
It is in the public interest to complete the project as soon as possible; and
The Department seeks contractor expertise to estimate contract time.
A+ B
The project does not require to be completed by a specific completion date;
RUC is relatively low but other factors warrant expediting the project; and
The Department seeks contractor expertise to estimate contract time.
I/ D Contracting Evaluationvaluationvaluation
With the help of FHWA, Herbsman ( 1995) collected highway construction project data using A+ B and A+ B with I/ D contracting from 15 states. Of a total of 101 project data collected, 41 completed projects used I/ D provisions. He also conducted interviews with practitioners, contractors, and others involved in the innovative contracting process. During quantitative data analysis, he measured project time and cost performance for each project and analyzed the project performance by states and project types. Average time savings/ overruns of the top five states that completed 10 projects or more per state were summarized in Table 6.
Average Time Savings/ Overruns by States: A+ B and A+ B with I/ DTable 6
( Source: Herbsman 1995)
States
Number of Projects Completed
Percent Average Time Savings (+) / Overruns (-)
Maryland
28
13.37
North Carolina
13
27.73
Missouri
13
- 4.54
New York
12
18.89
California
10
14.43
Average time savings from four states showed 18.6% and an average time overrun from one state for 13 projects was 4.54%. These results indicated that there could be some project factors that affect project performance. Herbsman ( 1995) further investigated a few case studies and concluded that “ motivated contractors can reduce construction time with more accurate scheduling, more efficient managing of the project, and better use Mineta Transportation Institute
Literature Review 15
of their own resources.” In the following section, useful information for evaluation of I/ D contracting was summarized by states.
California
In California, project time and cost performance comparisons between 28 A+ B projects ( with or without I/ D provisions) and 28 non- A+ B projects were performed. In a report entitled Summary Level Study of A+ B Bidding, it was found that A+ B contracting showed positive impacts on time savings at the beginning of the projects and no significant time or cost overruns were found after construction began. ( PinnacleOne 2004) Average time savings of 27% was reported as shown in Figure 4. Average cost growth amount on A+ B projects ($ 4.6M) was greater than non- A+ B projects ($ 3.8M). In addition, it was reported that the average claim amounts of the A+ B projects ($ 0.85M) were approximately half that of the representative non- A+ B ($ 1.72M).
Figure 4 A + B Average Time Savings
( Source: PinnacleOne 2004)
Florida
With regard to evaluation of FDOT alternative contracting techniques including I/ D contracting, Ellis et al. ( 2007) performed a comprehensive quantitative evaluation on FDOT construction projects as well as interviews with FDOT district engineers. The quantitative project cost and time evaluation results showed that total cost growth and time growth of the alternative contracting projects, including I/ D, were lower than the traditional design- Mineta Transportation Institute
16 Literature Review
bid- build projects during construction. They concluded that the choice of contracting method did not seem to have an effect on project quality by investigating contractor past performance rating scores. Regarding FDOT I/ D contracting practice, 144 projects were evaluated. Comparing to traditional design- bid- build contracting practice during the same research period, I/ D projects showed average time savings of 16.5% but average cost overruns of 3.3%. These results indicated that there was a trade- off effect between project cost and time. It was also reported that “ contractors achieved full or partial incentives approximately 51% of the time for I/ D contracting projects” ( Ellis et al. 2007).
Ellis et al. ( 2007) also performed interviews with FDOT district engineers regarding project selection of I/ D contracting and reported the following findings:
Project type, project cost, project duration, project location, and time of year were • important factors when considering the use of I/ D contract.
Projects over $ 10 million, projects of longer duration and interstate projects were • recommenced by applying I/ D provision.
Rural projects were only recommended, if having a high traffic volume.•
Using I/ D contracts near hurricane season, caution was recommended.•
I/ D contracting seems to work best when applied on large, interstate, or high- volume • rural projects.
With regard to I/ D contracting time performance evaluation, Pyeon ( 2005) further investigated incentive contracting techniques in Florida by analyzing various project factors. He found many significant factors that affect construction time performance using statistical analyses and developed a simulation model to predict project time performance as a framework. In this model, many processes, including categorization of variables, were functioned manually. More importantly, project cost performance was not considered in this model.
Michigan
The Michigan DOT evaluated 26 I/ D projects let and completed in 1998 and 1999. Michigan DOT’s project time and cost evaluation results were briefly summarized in a report of the Contract Administration Section of the AASHTO Subcommittee on Construction. According to the report entitled Primer on Contracting for the Twenty- first Century, project time and cost performance were found as follows ( AASHTO 2006):
65% of I/ D projects were completed early.•
12% were completed on time.•
23% were completed late.•
Average I/ D rate for all projects was $ 18,500.•
Average project user delay savings were $ 610,500.•
The use of I/ D provisions indicated an average increase of 1.5% of the contract • amount. Mineta Transportation Institute
Literature Review 17
Oregon
Oregon DOT has used I/ D provisions in two different forms: I/ D only and A+ B with I/ D. Sillars ( 2007) pointed out that Oregon DOT like many other DOTs had limited experience and only a few people with I/ D experience made decisions for the development of I/ D contracting on an ad- hoc basis. On the other hand, he emphasized that developing standardized methods for the use of I/ D contracting would benefit Oregon DOT by encouraging more frequent and effective use of I/ D contracts, as well as many others by providing useful lessons learned from Oregon.
Sillars ( 2007) evaluated Oregon DOT’s I/ D contracting experience for 18 I/ D contracting projects started between 1996 and 2005. Project values were varied ranging from $ 300,000 up to $ 65,200,000. From a frequency analysis of I/ D projects, it was found that a maximum number of four I/ D projects per year were released and reported that I/ D contracting remained a somewhat uncommon practice in Oregon. However, as more I/ D projects were practiced, he addressed “ the need of better documentation and more consistent techniques” ( Sillars 2007). An approximate value of each I/ D project was categorized by year and illustrated in Figure 5.
Figure 5 Oregon DOT I/ D Project Size by Date
( Source: Sillars 2007) Mineta Transportation Institute
18 Literature Review
Summaryummary of Literatureiterature Review
Selection of I/ D contracting guidelines by agencies are summarized in Table 7. The selection criteria for each STA listed in Table 7 were found in the following literature: FHWA 1989, Plummer 1992, Caputo and Scott 1996, FDOT 1997, MnDOT 2005, and Ohio DOT 2006. Many STAs developed their own selection criteria based on FHWA’s guidelines. Although there were many similarities on the I/ D selection criteria among STAs, it was also found there were many differences regarding the use of I/ D contracting. It indicated that there were different levels of I/ D contracting experience and preference based on their previous experience.
Through the literature review, it was found that there were many general guidelines developed by STAs, with many similarities and differences among their I/ D contracting selection criteria. Some STAs performed qualitative evaluation of their I/ D contracting practices and identified advantages and disadvantages for I/ D contracting methods. In addition, several STAs performed quantitative evaluations of I/ D contracting and reported project time and/ or cost performances comparing with other contracting methods. However, no STAs have implemented a certain type of decision support system for selection of I/ D contracting based on quantitative data analysis of the previous I/ D contracting practices. It is important for STAs to learn from their previous I/ D contracting experiences in order to improve I/ D project performance and refine I/ D usage. Therefore, it is recommended that more research efforts should be made to identify I/ D contracting project factors influencing project performance and develop a decision support system using the influencing factors to assist project planners and managers for selection of I/ D contracting.
Mineta Transportation Institute
Literature Review 19
Summary of I/ D Project Selection Criteria for Good CandidatesTable 7
Agencies ( Year)
Traffic and Business Impacts
Bridge
Roadway
Others
FHWA
( 1989)
High volume; High road- user cost or business impacts
Major bridge out of service
Major projects which severely disrupt traffic
Lengthy detour
Illinois DOT
( 1992)
Project type consideration ( even with low volume): Road, River Structure
River structures involving economic impacts or next to central business district
Roadway projects involving economic impacts
Night time construction on urban freeway
Maryland DOT
( 1992)
High volume
N/ A
N/ A
Impairment of emergency service; Elimination of hazardous condition; Safety of traveler & contractor employee
SD DOT
( 1996)
Interstate lane closure and restriction; High road- user cost or business impacts; Long off- site detour (> 10 min. delay)
Bridge closure with long off- site detour (> 10 min. delay)
Signalized intersection reconstruction
Two- way traffic disruption for long period
Project’s impacts on public, pedestrian or work
FDOT
( 1997)
High road- user cost or business impacts
Yes
Reconstruction in urban area
 
MnDOT
( 2005)
High road- user cost or business impacts; Interstate projects with major traffic impacts
Bridge rehab. & replacement involving high road- user or business impacts
Pavement rehabilitation in urban area with high road- user or business impacts
Commitment to open a roadway as soon as possible
Ohio DOT
( 2006)
All time- sensitive project; Interstate Lane Closure
Small project
Small bituminous project; Mid- Level projects ( interstate resurfacing and minor rehabilitation); Mega projects ( corridor reconstruction and Interstate rehabilitation)
 
N/ A
Caltrans
( 2007)
Required traffic restriction ( lane closure or detour on major roadway)
Bridge or interchange with a high ADT ( temporary lane, ramp, bridge closures; emergency repair)
Temporary Lane on major roadway ( High ADT)
Emergency contracts; I/ D time; I/ D amount ( Favorable cost/ benefit ratio and high enough); Relatively free of third party coordination concerns Mineta Transportation Institute
20 Literature Review
In summary, there are many unanswered questions regarding I/ D contracting project selection and evaluation. In order to enhance the decision- making process for the selection of I/ D projects, the following questions should be addressed:
How effective were I/ D contacting for given project situations in improving project time 1. and cost performance?
Which variables are the important factors that affect project time and cost performance 2. for an I/ D project?
What levels of project time and cost performance can the project planner expect for 3. an I/ D project?
Better understanding of the answers to these questions will make state and federal transportation project planners and managers more knowledgeable and effective in their decision- making so that I/ D contacting techniques may be applied in a more efficient way for transportation construction projects. Mineta Transportation Institute
21
DATA COLLECTION
From previous research experience, the research team found that most DOTs did not have construction project information in a database or easily accessible elsewhere. ( Pyeon 2005) When representatives of the DOTs were asked to provide construction project data, they responded that providing the project information would require considerable time and effort, and some project information was generally not tracked. For these reasons, project data collection is one of the most challenging tasks of this kind of research.
FDOT is the most active STA that has implemented I/ D contracting in their transportation construction projects. The required project information for this study is located in several different systems within FDOT. From previous research experience, the research team has already obtained part of the required project data by contacting the FDOT construction database engineer. However, the project data does not include the most recent practices, which need to be updated in a construction project database.
In this study, the research team collected recent I/ D contracting project information from FDOT. Due to time and resource limitations, I/ D project data from other states were not collected. In the following sections, the project data collection process and I/ D contracting project database construction procedures are described.
I/ D Project Dataataata
Transportation construction project data were obtained from the FDOT main office and district offices. Relevant project data, such as contract type, project type, duration, cost, location, length, maximum I/ D dollar amount, daily I/ D dollar amount, etc., were collected. FDOT I/ D contracting project data in transportation construction were obtained from several sources, such as Construction Time and Cost Quarterly Reports, Time and Cost Analysis of Passed Alternative Contracts Reports, and FDOT WebFocus database. A total of 295 I/ D contracting projects from the fiscal years 1998 through 2008 were utilized. Four different I/ D contracting types were identified: 1) I/ D only, 2) A+ B with I/ D, and 3) A+ B Bonus with I/ D. An example of I/ D project sample data obtained from FDOT is shown in Table 8.
I/ D Contracting Dataataatabase Construction for Analysisnalysis
Although the FDOT construction time and cost quarterly reports were obtained electronically, they needed to be joined to create a single database. An Excel spreadsheet of Time and Cost Analysis of Passed Alternative Contracts Reports collected from a district office was then merged into the time and cost report database. Finally, Excel spreadsheets of roadway contract data and historical contract data obtained from FDOT WebFocus database were joined with the time and cost report database. A total of 295 I/ D contracting project data were listed in the database. Relevant project data like contract type, project type, duration, cost, location, length, maximum I/ D dollar amount, and daily I/ D dollar amount were included in the I/ D project database for analysis. The project data collected for analysis and included in model development is summarized in Tables 9 and 10. Mineta Transportation Institute
Data Collection
22
FDOT I/ D Contracting Project Data SampleTable 8
Column Name
Data
Column Name
Data
Project ID
410678
Contract type
I/ D
District
06
Roadway ID
87060000
County
Miami- Dade
Transportation system
Non- intrastate
Work mix
Bridge - painting
Location
SR A1A
/ Mcarthur CSWY
Let date
5/ 22/ 02
Project manager
Luis Amigo
Award date
6/ 19/ 02
Contractor
Mayo Contracting
Execution date
7/ 03/ 02
Project length
0.399 miles
Notice to proceed
8/ 2/ 02
Number of lanes
0
Work begin date
2/ 16/ 03
Number of lanes added
0
Final acceptance date
9/ 26/ 03
DOT original estimate
$ 1,501,000
DOT time estimate
240
Original contract amount
$ 1,976,732
Incentive days
239
Present contract amount
$ 2,083,065
Original contract days
240
Total amount paid
$ 1,979,886
Present contract days
267
Actual expenditure
$ 1,945,886
Days used
222
Actual Incentive paid
$ 34,000
Days suspended
0
Daily incentive amount
$ 2,000
Weather days
27
Max. incentive proposed
$ 105,000
Total work order TE
0
Total SA amount
$ 106,333
Total SA days
0
Production rate
$ 8,100
Number of SAs
2
Incentive production rate
$ 10,400
Incentive time maximum
188
Historical production rate
$ 7,700 Mineta Transportation Institute
Data Collection 23
Summary of Construction Projects by Contract TypesTable 9
District
Contract Type
Number of Projects
Total Contract Amount
1
A+ B with I/ D
11
$ 101,234,088
 
I/ D
22
$ 203,299,659
District 1 Total
33
$ 304,533,747
2
A+ B with I/ D
23
$ 134,369,850
 
I/ D
2
$ 3,853,518
District 2 Total
25
$ 138,223,368
3
A+ B with I/ D
19
$ 243,325,709
 
I/ D
8
$ 45,733,389
District 3 Total
27
$ 289,059,098
4
A+ B with I/ D
9
$ 116,752,055
 
A+ B Bonus with I/ D
4
$ 199,693,064
 
I/ D
31
$ 226,169,502
District 4 Total
44
$ 542,614,621
5
A+ B with I/ D
15
$ 237,207,911
 
I/ D
13
$ 102,124,145
District 5 Total
28
$ 339,332,056
6
A+ B with I/ D
8
$ 35,029,381
 
A+ B Bonus with I/ D
26
$ 345,650,232
 
I/ D
62
$ 83,698,282
District 6 Total
96
$ 464,377,895
7
A+ B with I/ D
9
$ 113,845,418
 
A+ B I/ D Bonus
1
$ 7,861,142
 
I/ D
14
$ 92,001,259
District 7 Total
24
$ 213,707,819
8
A+ B with I/ D
6
$ 119,281,020
 
A+ B Bonus with I/ D
1
$ 3,721,761
 
I/ D
11
$ 169,181,846
District 8 Total
18
$ 292,184,627
Grand Total
295
$ 2,584,033,231 Mineta Transportation Institute
24 Data Collection
Summary of Construction Projects by Project TypesTable 10
Project Work Type
Number of Projects
Total Construction Duration ( Days)
Total Contract Amount
Access improvement
2
375
$ 4,750,119
Add lanes & reconstruction
66
38,610
$ 957,745,630
Add lanes & rehabilitate pavement
16
8,957
$ 252,154,000
Add right turn lane( s)
2
210
$ 436,396
Add thru lane( s)
1
130
$ 1,330,442
Add turn lane( s)
7
830
$ 4,234,520
Bridge— painting
2
440
$ 3,138,951
Bridge/ culvert replacement
2
500
$ 4,741,346
Bridge- rehab and add lanes
1
925
$ 32,859,777
Bridge- repair/ rehabilitation
14
2,612
$ 31,805,272
Construct bridge— low level
4
1,525
$ 17,509,373
Construct bridge— movable span
1
576
$ 23,445,002
Construct bridge— high level
1
500
$ 18,486,091
Construct/ reconstruct median
1
120
$ 593,653
Federal aid resurface/ repave
1
120
$ 2,944,870
Fender work
1
390
$ 2,284,662
Fixed guideway improvements
1
500
$ 3,494,000
Flexible pavement reconstruction
5
1,510
$ 24,633,355
Guardrail
5
1,156
$ 44,472,567
Highway- enhancement
1
152
$ 3,607,477
Interchange ( major)
6
4,885
$ 233,479,355
Intersection ( major)
2
1,345
$ 36,624,974
Intersection ( minor)
7
640
$ 3,017,766
Landscaping
1
150
$ 2,212,452
Mill and resurface
1
150
$ 4,229,690
Miscellaneous construction
4
1,039
$ 10,730,812
Miscellaneous structure
1
525
$ 37,935,485
New road construction
6
3,185
$ 132,177,053
Replace low level bridge
19
6,194
$ 103,284,848
Replace medium level bridge
6
3,876
$ 74,358,292
Replace movable span bridge
4
3,485
$ 171,273,445
Resurfacing
79
18,034
$ 253,119,539
Rigid pavement reconstruction
2
1,082
$ 32,286,750
Rigid pavement rehabilitation
1
280
$ 6,630,067
Safety project
7
1,163
$ 9,759,660
Sidewalk
1
100
$ 420,608
Traffic signals
6
670
$ 1,978,393
Widen bridge
3
1,260
$ 18,062,628
Widen/ resurface exist lanes
5
806
$ 17,783,911
Grand total
295
109,007
$ 2,584,033,231 Mineta Transportation Institute
25
DATA ANALYSIS
The purpose of the data analysis in this study was to identify important factors that influence construction project time and cost performance. The obtained I/ D project data were evaluated using time and cost performance indices. Four performance indices were developed and used for analysis: ( 1) Time performance index based on original contract duration ( OTPI); ( 2) Time performance index based on present contract duration ( PTPI); ( 3) Cost performance index based on original contract cost ( OCPI); and ( 4) Cost performance index based on present contract cost ( PCPI). Next, statistical analyses were performed to identify any differences on project performance among project variables. Finally, significant factors that influence project performance were identified and summarized.
Statisticaltatisticaltatistical Analysisnalysis Process
The construction project data used for this study consist of quantitative variables such as project length, cost, duration, and maximum or daily I/ D dollar amounts, and qualitative variables such as project type, contract type, and project location. For the quantitative variables, correlation analysis was performed to identify potential key factors that might influence project performance. In the next step, factors selected for further analysis were classified using an appropriate categorization process. Finally, statistical analyses were performed to identify any differences among project variables.
Numerous statistical analyses were performed to investigate the possible differences on project performance among project factors. The following statistical analysis tests were used in this study: ( 1) the two- sample t- test was used to determine whether there was a significant difference between the means of the two groups, ( 2) the analysis of variance ( ANOVA) test was performed to test the null hypothesis that all population means are equal, and ( 3) the multiple comparison test was performed to determine which means are different from which others whenever the ANOVA test is significant. Since each project was completed at a different location and in a different time, each project was assumed to be independent. In probability theory, a sufficiently large sample of independent random variables is approximately normally distributed. Since the central limit theorem justifies the approximation of large- sample statistics to the normal distribution, it is practical to assume that variables in this study with a large sample size are normally distributed. Therefore, it is reasonable to perform the hypothesis tests to identify factors that influence project performance among project variables.
For qualitative variables already categorized in several groups, an ANOVA test was performed to test the null hypothesis that all population means for the groups are equal. Sometimes, it was necessary that an appropriate grouping process be performed prior to the ANOVA test for qualitative variables with many different categories. For instance, each project has a major work type description ( i. e., FDOT Work Mix), which briefly describes project characteristics. According to the major work type, projects were put into similar groups such as bridge rehabilitation/ reconstruction, roadway rehabilitation/ reconstruction, roadway resurfacing/ paving, and others. Then, an ANOVA test was performed to test the null hypothesis that all population means for the major work type categories are equal. A multiple comparison procedure was performed whenever the F- test for the effect was Mineta Transportation Institute
Data Analysis
26
significant in the ANOVA table to determine which means were different from which others.
Evaluationvaluationvaluation of Project Performance
Project performance was measured using two key parameters: time and cost. Using the time parameter, a project time performance index ( TPI) for each project was determined based on the following formula:
TPI = Final Duration – Contract Duration ,
Contract Duration
( 1)
where a negative value of TPI means time savings and a positive value of TPI means time overruns. For example, a value of TPI = - 0.05 indicates a 5% project time savings, while a value of TPI = + 0.05 means a 5% time overrun.
The TPI was refined using details such as a time performance index based on original contract duration ( OTPI) and a time performance index based on present contract duration ( PTPI), which included time extensions and supplemental agreement days. However, the total number of days granted as weather days in accordance with specifications was not included when calculating both indices. Thus, OTPI and PTPI indices were calculated as:
OTPI = Final Duration – Original Contract Duration ,
Original Contract Duration
( 2)
PTPI = Final Duration – Present Contract Duration ,
Present Contract Duration
( 3)
Using the cost parameter, a project cost performance index ( CPI) for each project was determined as follows:
CPI = Final Cost – Contract Cost ,
Contract Cost
( 4)
where a negative value of CPI means cost savings and a positive value of CPI means cost overruns. For example, a value of CPI = - 0.05 means project cost savings of 5%, while a value of CPI = + 0.05 means a 5% cost overrun.
The CPI was also refined using details such as a cost performance index based on original contract cost ( OCPI) and a cost performance index based on present contract cost ( PCPI), which included total work order amount, supplemental agreement amount, incentives paid, and other contract adjustments. These indices were calculated as: Mineta Transportation Institute
Data Analysis
27
OCPI = Final Cost – Original Contract Cost ,
Original Contract Cost
( 5)
PCPI = Financial Cost – Present Contract Cost ,
Present Contract Cost
( 6)
Factorsactors Influencing Project Performance
In order to identify important factors that influence construction project time and cost performance based on original contract and present contract, many project factors were studied. Although not presented in detail here, many variables were tested to identify key factors. The tested variables are listed below:
Contract type1.
Project location: district and county2.
Project type: work mix3.
Project length: number of lanes4.
DOT time estimate5.
Original contract duration6.
Days suspended7.
Weather days8.
( Weather days)/( Original contract duration) 9.
( Days between let date and work begin date)/( Original contract duration) 10.
( Total work order time extension)/( Original contract duration) 11.
( Supplemental agreement days)/( Original contract duration) 12.
DOT original cost estimate13.
Original contract cost14.
Daily incentive amount15.
Maximum incentive proposed16.
( Original contract cost)/( Original contract duration) 17.
( Total supplemental agreement amount)/( Original contract cost) 18.
( Total supplemental agreement amount)/( DOT’s actual expenditure) 19.
( Innovative contract adjustments amount)/( Original contract cost) 20.
( Innovative contract adjustments amount)/( DOT’s actual expenditure) 21.
This section only describes statistically significant factors among all tested variables. Through statistical analysis, the significant factors were determined to be project size, contract type, project type, project length, maximum incentive proposed, daily incentive amount and district.
Factor 1: Contract Type
The I/ D contracting technique has been used as a stand- alone method or with a combination of other contracting methods such as A+ B and/ or Bonus. Construction project data collected were categorized by three I/ D contracting types: ( 1) I/ D, ( 2) A+ B with I/ D, and ( 3) Mineta Transportation Institute
28 Data Analysis
A+ B with I/ D and Bonus. The contract type variables as qualitative variables were already categorized by three I/ D contracting types. With 295 observations ( I/ D: 163, A+ B I/ D: 100, and A+ B I/ D Bonus: 32), the boxplots, used for descriptive statistics, graphically depict the five- number summary of a data set consisting of the minimum, the lower quartile ( the lowest 25% of the data), the median, the upper quartile ( the highest 25% of the data), and the maximum. Results of box- and- whiskers plot comparison of time and cost performance of each contract type variable are illustrated in Figure 6.
Figure 6 Box Plot of Contract Type Variables
For contract type variables of each project performance index, an ANOVA test was performed to test the null hypothesis that all three population means for the groups are equal. The F- test results are shown in Table 11. The statistical significance of the variables is given by the probability value ( p- value) defined in this study to be significant when it is smaller than 0.05. Since the p- value is smaller than 0.05, it was concluded from this test that the effect of contract type is significant.
Further analysis was therefore needed to test which means are different from which others. The Tukey test was performed for multiple comparisons. The Tukey test results are shown in Table 11. Three possible cases investigated were: ( 1) I/ D vs. A+ B I/ D, ( 2) I/ D vs. A+ B I/ D Bonus, and ( 3) A+ B I/ D vs. A+ B I/ D Bonus. Although it was not found that there is any difference among contract type variables in the case of OCPI, the test results showed that the differences among contract type variables are significant in the case of OTPI, PTPI, and PCPI. It indicates that contract type variables have an influence on project performance. Mineta Transportation Institute
Data Analysis
29
ANOVA and Tukey Test Results of Contract Type VariablesTable 11
Contract Type
Variables
F- value
p- value
Significant Tukey Tests
( 0.05 Level)
OTPI
9.623
< 0.001
A+ B I/ D – A+ B I/ D Bonus
I/ D – A+ B I/ D
PTPI
5.644
0.0039
I/ D – A+ B I/ D
OCPI
0.445
0.6412
N/ A
PCPI
4.586
0.0109
A+ B I/ D – A+ B I/ D Bonus
I/ D – A+ B I/ D Bonus
Factor 2: Project Type
Considering the variety of project situations, there are numerous work types in highway construction. Typically, each project consists of one major work type, which briefly describes project characteristics, and several other minor work types. Projects were grouped according to major work description for a further analysis to test the effect of project type. Major work types used in this study are listed in Appendix A. The project type variable classifications are also shown in the table in Appendix A. Four levels of project type variables used in this study were: ( 1) Bridge Rehabilitation/ Reconstruction ( BRR), ( 2) Roadway Rehabilitation/ Reconstruction ( RRR), ( 3) Roadway Resurfacing/ Paving ( RRP), and ( 4) Others. The box- and- whiskers plot of time performance of each project type variable is shown in Figure 7.
After categorizing project work types, an ANOVA test was performed to test the null hypothesis that all four population means for the groups are equal. The F- test results are shown in Table 12. Since the p- value is smaller than 0.05, it was concluded from this test that the effect of project type is significant. Thus, further analysis was needed to test which means are different from which others. The Tukey test was performed for multiple comparisons to test six possible cases: ( 1) BRR vs. RRR, ( 2) BRR vs. RRP, ( 3) BRR vs. Others, ( 4) RRR vs. RRP, ( 5) RRR vs. Others, and ( 6) RRP vs. Others. All cases were tested and only conclusive cases are summarized in Table 12. Although it was not found that there is any difference among contract type variables in the case of OCPI and PCPI, the test results showed that the differences among contract type variables are significant in the case of OTPI and PTPI. This indicates that contract type variables have an influence on project time performance. Mineta Transportation Institute
30 Data Analysis
Figure 7 Box Plot of Project Type Variables
ANOVA and Tukey Test Results of Project Type VariablesTable 12
Project Type Variables
F- value
p- value
Significant Tukey Tests
( 0.05 Level)
OTPI
6.545
0.0003
BRR – Others
RRR – Others
RRP – Others
PTPI
6.212
0.0004
BRR – Others
RRR – Others
OCPI
1.582
0.1938
N/ A
PCPI
0.634
0.5936
N/ A
Factor 3: District
There are eight transportation districts in Florida, including the turnpike district. Although each district generally has similar major divisions, the FDOT allows districts flexibility to manage their businesses using systems with which they feel most comfortable. Consequently, the organizational structure of each district varies. Since different district management systems may influence project performance before or during construction, the district variable was investigated. The levels of the district variable studied were as follows: ( 1) District 1, ( 2) District 2, ( 3) District 3, ( 4) District 4, ( 5) District 5, ( 6) District 6, ( 7) District 7, and ( 8) District 8. As a descriptive statistical summary, the box- and- whiskers plots of time performance of each district are illustrated in Figure 8. Mineta Transportation Institute
Data Analysis
31
Next, an ANOVA test was performed to test the null hypothesis that all eight population means for the groups are equal. The F- test results are shown in Table 13. Since the p- value is smaller than 0.05, it was concluded from this test that the effect of district is significant. As a result, further analysis was needed to test which means are different from which others. The Tukey test was performed for multiple comparisons to test all possible cases. In summary, only conclusive cases are included in Table 13. The test results showed that the differences among district variables are significant in all cases, OTPI, PTPI, OCPI and PCPI. This indicates that district variables have an influence on project time performance.
Figure 8 Box Plot of District Variables Mineta Transportation Institute
32 Data Analysis
ANOVA and Tukey Test Results of District VariablesTable 13
District Variables
F- value
p- value
Significant Tukey Tests
( 0.05 Level)
OTPI
7.579
< 0.0001
District 1 – District 6
District 3 – District 6
District 4 – District 6
District 5 – District 6
PTPI
2.487
0.0171
District 1 – District 6
OCPI
6.735
< 0.0001
District 4 – District 6
District 6 – District 8
PCPI
4.460
< 0.0001
District 1 – District 8
District 2 – District 8
District 3 – District 8
District 4 – District 8
District 5 – District 8
District 6 – District 8
District 7 – District 8
Factor 4: Project Size
The original contract cost for each project is a quantitative variable. The contract amounts of the projects studied ranged from $ 114,185 to $ 99,537,000. The project size variable used in this study is the daily project cost, which can be calculated using the following formula:
Daily Project Cost = Original Contract Cost ,
Original Contract Duration
( 7)
Daily project cost, also a quantitative variable, ranged from $ 1,014 to $ 96,638. Correlation analysis between daily project cost and performance indices was performed and the result showed a positive relationship with each index. Next, the categorization process, using quartiles of a distribution box- and- whiskers plot analysis, was performed. The distribution of data was divided using the inter- quartile range ( IQR), which is the distance between the lower quartile ( Q1) and the upper quartile ( Q3). Daily project costs of Q1 and Q3 were $ 9,152 and $ 24,450, respectively, with IQR = $ 15,298. The groups of daily project cost variables were: ( 1) project size small ( PSS; <$ 9,152), ( 2) project size medium ( PSM; $ 9,152-$ 24,450), and ( 3) project size large ( PSL; >$ 24,450). Results of the box- and- whiskers plot comparison of time and cost performance of each project size variable are illustrated in Figure 9. Mineta Transportation Institute
Data Analysis
33
Figure 9 Box Plot of Project Size Variables
Next, an ANOVA test was performed to test the null hypothesis that all three population means for the groups are equal. The F- test results are shown in Table 14. Since the p- value is smaller than 0.05, it was concluded from this test that the effect of project size is significant. Thus, further analysis was needed to test which means are different from which others. Tukey tests were performed for multiple comparisons. The Tukey test results are shown in Table 14. Two out of three possible cases were significant. They were: ( 1) PSS vs. PSM and ( 2) PSS vs. PSL. Although it was not found that there is any difference among project size variables in the case of PTPI, the test results showed that the differences among project size variables are significant in the case of OTPI, OCPI, and PCPI. It indicates that project size variables have an influence on project performance.
ANOVA and Tukey Test Results of Project Size VariablesTable 14
Project Size Variables
F- value
p- value
Significant Tukey Tests
( 0.05 Level)
OTPI
7.186
0.0009
PSS – PSM
PSS – PSL
PTPI
1.945
0.1448
N/ A
OCPI
16.788
< 0.001
PSS – PSM
PSS – PSL
PCPI
15.877
< 0.001
PSS – PSM
PSS – PSL Mineta Transportation Institute
34 Data Analysis
Factor 5: Project Length
Project length data collected from 136 projects were used for analysis. Project lengths, a quantitative variable, ranged from 0.001 to 23.5 miles. Typically, project lengths of roadway resurfacing/ paving type projects were longer than any project types with an average of 4.23 miles. On the other hand, projects types like low level bridge construction, movable span bridge replacement, safety, traffic signals, minor intersection, and add turn lane( s) had relatively short project length than other projects.
Initially, correlation analyses between the project length and performance indices were performed. Test results showed a small positive relationship with OCPI and PCPI and a small negative relationship with OTPI and PTPI between two variables. For further analysis, a categorization process was followed. Considering the distribution of the dataset, project length data was divided by the mean value of total project length ( 2.8 miles). The two groups of project length variables were: ( 1) project length below average ( PLBA; < 2.8 miles) and ( 2) project length above average ( PLAA; > 2.8 miles). As a descriptive statistical summary, box- and- whiskers plots of time and cost performance of each project length variable are illustrated in Figure 10.
Figure 10 Box Plot of Project Length Variables
After categorizing project length variables, statistical significance tests were performed to determine the possible differences in project performance between project length variables. The two- sample t- test was used to determine whether there is a significant difference between the means of the two groups, PLBA and PLAA. In this statistical Mineta Transportation Institute
Data Analysis
35
analysis, 68 observations from each variable were compared. Summary statistics of project length variables and the t- test results with p- value and significance are shown in Table 15. Although the t- test for project time performance was not significant, the t- test for project cost performance in the case of PCPI was significant at the 0.05 confidence level. The t- test result showed sufficient evidence that the average project cost performance from the two groups, PLBA and PLAA, are not the same. It indicates that project length variables have an influence on project cost performance.
Two Sample t- Test Results of Project Length VariablesTable 15
Project Length Variables
t- Test Statistics
p- value
Significant Tests
( 0.05 Level)
OTPI
0.358
0.7213
N/ A
PTPI
0.516
0.6064
N/ A
OCPI
- 0.695
0.4888
N/ A
PCPI
- 2.743
0.0070
PLBA – PLAA
Factor 6: Maximum Incentive Amount
The maximum incentive amount proposed for each project is a quantitative variable. The various amounts ranged from $ 3,000 to $ 2,643,559 and the average incentive proposed amount was $ 370,548 per project. Initially, correlation analysis between maximum incentive amounts and performance indices was performed and the result showed a positive relationship with each index. Next, the categorization process, using quartiles of a distribution a box- and- whiskers plot analysis, was performed. The distribution of data was divided using the IQR. The maximum incentives of Q1 and Q3 were $ 45,000 and $ 450,000, respectively, with IQR = $ 405,000. The groups of maximum incentive amount variables were: ( 1) maximum incentive proposed small ( MIS; <$ 45,000), ( 2) maximum incentive proposed medium ( MIM; $ 45,000-$ 450,000), and ( 3) maximum incentive proposed large ( MIL; >$ 450,000). As a descriptive statistical summary, box- and- whiskers plots on time and cost performance of maximum incentive variables are illustrated in Figure 11.
After categorizing maximum incentive amount variables, an ANOVA test was performed to test the null hypothesis that all three population means for the groups are equal. The F- test results are shown in Table 16. Since the p- value is smaller than 0.05, it was concluded from this test that the effect of maximum incentive amount is significant. Thus, further analysis was needed to test which means are different from which others. Tukey tests were performed for multiple comparisons. The Tukey test results are shown in Table 16. Three possible cases were tested: ( 1) MIS vs. MIM, ( 2) MIS vs. MIL, and ( 3) MIM vs. MIL. Mineta Transportation Institute
36 Data Analysis
Figure 11 Box Plot of Maximum Incentive Amount Variables
With regard to project time performance, no test was significant to conclude that there is any difference among maximum incentive amount variables. However, the tests were significant in both cases of OCPI and PCPI regarding project cost performance. The test results showed that there are significant differences among maximum incentive amount variables. This indicates that maximum incentive amount variables have an influence on project cost performance.
ANOVA and Tukey Test Results of Maximum I/ D Amount VariablesTable 16
Maximum I/ D Amount
Variables
F- value
p- value
Significant Tukey Tests
( 0.05 Level)
OTPI
2.335
0.1016
N/ A
PTPI
1.849
0.1622
N/ A
OCPI
11.611
< 0.001
MIS – MIM
MIS – MIL
PCPI
18.065
< 0.001
MIS – MIM
MIS – MIL
MIM – MIL
Factor 7: Daily I/ D Amount
The daily I/ D amount for each project is a quantitative variable. The various I/ D amounts ranged from $ 600 to $ 10,000 and the average daily I/ D amount was $ 3,390 per project. Initially, correlation analysis between daily I/ D amounts and performance indices was Mineta Transportation Institute
Data Analysis
37
performed and the result showed a positive relationship with each index. Next, the categorization process, using quartiles of a distribution a box- and- whiskers plot analysis, was performed. The distribution of data was divided using the IQR. Daily I/ D amounts of Q1 and Q3 were $ 2,000 and $ 4,000, respectively, with IQR = $ 2,000. The groups of daily I/ D amount variables were: ( 1) daily I/ D amount small ( DIS; <$ 2,000), ( 2) daily I/ D amount medium ( DIM; $ 2,000-$ 4,000), and ( 3) daily I/ D amount large ( DIL; >$ 4,000). As a descriptive statistical summary, box- and- whiskers plots of time and cost performance of daily I/ D amount variables are illustrated in Figure 12.
After categorizing daily I/ D amount variables, an ANOVA test was performed to test the null hypothesis that all three population means for the groups are equal. The F- test results are shown in Table 17. Since the p- value is smaller than 0.05, it was concluded from this test that the effect of daily I/ D amount is significant. Thus, further analysis was needed to test which means are different from which others. Tukey tests were performed for multiple comparisons. The Tukey test results are shown in Table 17. Three possible cases tested were as follows: ( 1) DIS vs. DIM, ( 2) DIS vs. DIL, and ( 3) DIM vs. DIL.
With regard to project time performance, the tests were not significant to conclude that there is any difference among daily I/ D amount variables in the case of PTPI. However, a comparison between DIS and DIL was significant in the case of OTPI. The result showed that there is a significant difference between daily I/ D amount variables. This indicates that daily I/ D amount variables have an influence on project time performance. With regard to project cost performance, the tests were significant in both cases of OCPI and PCPI. The test results showed that there are significant differences among daily I/ D amount variables. This indicates that daily I/ D amount variables have an influence on project cost performance. Mineta Transportation Institute
38 Data Analysis
Figure 12 Box Plot of Daily I/ D Amount Variables
ANOVA and Tukey Test Results of Daily I/ D Amount VariablesTable 17
Daily I/ D Amount Variables
F- value
p- value
Significant Tukey Tests
( 0.05 Level)
OTPI
4.699
0.0112
DIS – DIL
PTPI
2.989
0.0549
N/ A
OCPI
13.298
< 0.001
DIS – DIM
DIS – DIL
PCPI
17.247
< 0.001
DIS – DIM
DIS – DIL
Summaryummary of Dataataata Analysisnalysis
Outcomes of individual projects are affected by various factors. This research has found several project factors influencing I/ D contracting performance based on statistical analysis as follows:
The important factors that had significant impacts on project time performance were • the effects of contract type, project type, district, project size, and daily I/ D amount.
The important factors that had significant impacts on project cost performance were • the effects of contract type, district, project size, project length, maximum incentive amount, and daily I/ D amount. Mineta Transportation Institute
Data Analysis
39
The results of project data analysis will help decision makers understand project key factors that affect project time and cost performance. The important findings from data analysis are summarized as follows:
A+ B Bonus with I/ D contracting was most effective to improve original project time • performance.
Project type “ Others” showed better project time performance compared to roadway or • bridge project types. It is important for decision makers to understand that higher traffic impact is generally expected for any construction projects of roadway or bridge types during construction.
Project time performance of I/ D contracting projects completed in District 6 were • significantly better than any other districts.
Project contract amount was not an important factor that influences project performance. • However, daily project cost ( also know as project size) had an influence on project performance. For instance, the smaller projects in terms of daily cost tended to be more efficient to improve original project time and cost performance.
In summary, significant/ non- significant factors at the 0.05 level based on statistical analysis are shown in Table 18. Project time and cost performances grouped by contract types and categorized by project types are shown in Table 19.
Summary of Significant ( S) or Non- significant ( NS) Factors by IndicesTable 18
Variables
OTPI
PTPI
OCPI
PCPI
Contract Type
S
S
NS
S
Project Type
S
S
NS
NS
District
S
S
S
S
Project Size
S
NS
S
S
Project Length
NS
NS
NS
S
Max. Incentive Amount
NS
NS
S
S
Daily I/ D Amount
S
NS
S
S Mineta Transportation Institute
40 Data Analysis
Project Performance Summary by Contract Types and Project TypesTable 19
Contract Type
Project Type Category
Number of
Projects
Average
OTPI
PTPI
OCPI
PCPI
I/ D
Bridge Rehabilitation/ Recon­struction
29
0.022
- 0.126
0.054
0.005
Roadway Rehabilitation/ Re­construction
51
0.102
- 0.038
0.086
- 0.001
Roadway Resurfacing/ Paving
59
- 0.005
- 0.102
0.046
- 0.005
Others
24
- 0.184
- 0.188
0.037
0.015
I/ D Total
163
0.007
- 0.099
0.059
0.001
A+ B I/ D
Bridge Rehabilitation/ Recon­struction
25
0.167
- 0.006
0.060
- 0.008
Roadway Rehabilitation/ Re­construction
52
0.197
- 0.007
0.075
0.014
Roadway Resurfacing/ Paving
20
0.160
- 0.059
0.049
0.004
Others
3
- 0.020
- 0.164
0.061
0.012
A+ B I/ D Total
100
0.176
- 0.022
0.066
0.007
A+ B I/ D Bonus
Bridge Rehabilitation/ Recon­struction
5
0.128
0.004
0.105
0.028
Roadway Rehabilitation/ Re­construction
18
- 0.025
- 0.071
0.086
0.053
Roadway Resurfacing/ Paving
9
- 0.085
- 0.093
0.057
0.037
A+ B I/ D Bonus Total
32
- 0.018
- 0.065
0.081
0.045
Grand Total
295
0.061
- 0.069
0.063
0.008 Mineta Transportation Institute
41
DECISION SUPPORT MODEL DEVELOPMENT
In this chapter, a model to support the decision- making process for I/ D construction projects is presented. A project performance prediction model using Monte Carlo simulation was developed. The development process is described in detail. To predict project time and cost performance, Monte Carlo simulation procedures were adopted for the development of a spreadsheet- based decision support model. The factors that affect I/ D project performance were employed as input variables. In the modeling process, beta distributions were selected as the theoretical distribution of the input variables used for the Monte Carlo simulation. For this study, the @ Risk Version 5.5 add- in for Microsoft Excel was implemented to perform the Monte Carlo simulation procedures. Graphic User Interfaces were designed using Visual Basic Application programming. The entire development process of the decision support model is illustrated in Figure 13.
Figure 13 Flow Chart of I/ D Performance Simulation Model Development Process Mineta Transportation Institute
Decision Support Model Development
42
The decision support model consists of two modules: ( 1) a database update module, and ( 2) a performance simulation module. The database update module includes the “ Classification and Performance Evaluation” process. During this process, each project in the initial construction project database was automatically classified and its time and cost performance was automatically evaluated as well. As an outcome of this process, a modified project database was generated to be used as inputs of the performance simulation module.
There are three parts in the performance simulation module: ( 1) Selection of project variables and performance index as simulation inputs, ( 2) Monte Carlo simulation procedures, and ( 3) Graphs and reports of simulation output results, including distributions of possible results, frequency distributions of possible output values, cumulative probability curves, and regression sensitivity analysis displayed as a bar chart.
Dataataatabase Updatepdate Module
The database update module is designed to provide inputs for performance simulation as well as update the construction project database in the future. This module consists of three parts: ( 1) Initial construction project database, including all raw project data, ( 2) Classification and performance evaluation process categorizing project data into similar groups and evaluating each project with four performance indices, OTPI, PTPI, OCPI and PCPI, calculated using Eq. ( 1, 2, 3, and 4), respectively, and ( 3) Modified project database including input variables of the performance simulation module as an outcome of the classification and performance evaluation process. All variables and selection criteria used for performance simulation are listed as follows:
1. Contract type variables are categorized into three groups.
1.1. A+ B
1.2. A+ B Bonus
1.3. I/ D
2. Project work type variables are grouped into four categories using work- mix classification shown in Appendix A.
2.1. Bridge Rehabilitation/ Reconstruction
2.2. Roadway Rehabilitation/ Reconstruction
2.3. Roadway Resurfacing/ Paving
2.4. Others
3. District variables include all eight districts.
3.1. District 01
3.2. District 02
3.3. District 03
3.4. District 04
3.5. District 05
3.6. District 06
3.7. District 07
3.8. District 08 Mineta Transportation Institute
Decision Support Model Development
43
4. Project size variables are grouped into three levels. In case project size data are not available, an “ N/ A” option is given to the user.
4.1. Small: < $ 9,152 ( 25th Percentile)
4.2. Medium: $ 9,152–$ 24,450
4.3 Large: > $ 24,450 ( 75th Percentile)
4.4. N/ A
5. Project length variables are categorized into two groups. In case project length data are not available, an “ N/ A” option is given to the user.
5.1. Below Average: < 2.8 Miles ( Mean Value)
5.2. Above Average: ≥ 2.8 Miles
5.3. N/ A
6. Maximum incentive proposed amount variables are grouped into three levels. In case maximum incentive amount data are not available, an “ N/ A” option is given to the user.
6.1. Small: < $ 45,000 ( 25th Percentile)
6.2. Medium: $ 45,000–$ 450,000
6.3. Large: > $ 450,000 ( 75th Percentile)
6.4. N/ A
7. Daily I/ D amount variables are grouped into three levels. In case daily I/ D amount data are not available, an “ N/ A” option is given to the user.
7.1. Small: < $ 2,000 ( 25th Percentile)
7.2. Medium: $ 2,000–$ 4,000
7.3. Large: > $ 4,000 ( 75th Percentile)
N/ A
The selection criteria of all variables are determined based on the existing project database. Once the initial project database is updated, then the selection criteria will be automatically recalculated and stored in the modified database. In addition, it will automatically update drop down boxes for selecting project inputs in the performance simulation module.
Performance Simulationimulation Module
The I/ D project performance simulation module is designed to select project variables and performance index as simulation inputs, perform Monte Carlo simulation procedures, and generate user- friendly simulation results. During selection of input variables and performance index, the system retrieves the selected project performance indices which belong to the selected input variables from the modified database in the database update module.
In order to perform Monte Carlo simulation, the modeling procedure used herein is based on the flexibility of beta distributions that provides various shapes of probability distribution. A beta probability density function can be formulated using shape parameters and the lower boundary ( a) and the upper boundary ( b) of the distribution: Mineta Transportation Institute
44 Decision Support Model Development
f ( x) = ( x – a) p– 1 ( b – x) q– 1 ,
B( p, q)( b – a) p+ q– 1
( 8)
where a ≤ x ≤ b, p and q represent shape parameters, and B( p, q) represents a beta function. Beta functions used in Eq. ( 8) are defined as:
B( p, q) = ∫ 01xp– 1( 1 – x) q– 1dx,
( 9)
where
p = {( x – a)/( b – a)}[{( x – a)/( b – a)} { 1 – ( x – a) /( b – a)} – 1],
S2 / ( b – a) 2
( 10)
q = [ 1 – {( x – a)/( b – a)}][{( x – a)/( b – a)} { 1 – ( x – a)/( b – a)} – 1],
S2 / ( b – a) 2
( 11)
where x represents the sample mean and S2 represents the sample variance.
The shape parameters and the lower and upper boundaries were determined from a dataset of each input variable. Using the beta distribution given in Eq. ( 8), such data of each variable were fitted into its own shape. An example of generating parameters of beta distribution is shown in Appendix B.
Monte Carlo Simulation Procedures
The Monte Carlo simulation method, a stochastic analysis, is a well known method for handling uncertainty and has been widely used as an aid in decision- making processes ( Guyonnet et al. 1999 and Schuyler 2001). This approach was used to estimate potential project time and cost performance in this study. Figure 14 illustrates the Monte Carlo simulation procedures for an example of OTPI simulation. Mineta Transportation Institute
Decision Support Model Development
45
Figure 14 Flowchart of Monte Carlo Simulation Procedures
The following five steps describe the Monte Carlo simulation procedures for an example of OTPI simulation shown in Figure 14.
Step 1: A beta probability density function for each variable was determined computing the parameters, p, q, a, and b in Eq. ( 10) and ( 11).
Step 2: Considering the probability density of each input variable, an OTPI value was randomly generated from the distribution of each input variable.
Step 3: An OTPIN value was computed using the following formula: Mineta Transportation Institute
46 Decision Support Model Development
n
OTPIN = Σ OTPIfi X Wi ,
f = 1
( 12)
where the OTPIN represents an OTPI value generated from each iteration process. The N represents the number of iterations, usually N = 1000. The OTPIfi represents an OTPI value generated from the input variables. The subscript fi stands for the ith factor selected in simulations. The n represents the number of input variables considered in this study, n = 7. The Wi represents the weight of each input variable.
The variance of each input variable was used to assign weights to input variables. The assigned weights were calculated using the following formula:
Wi = ( n wi ) ,
Σ wi
i = 1
( 13)
n
where wi = ( 1 ) and Σ wi = 1.
Si2 i = 1
The weighting process considered the impact of input variables. Since smaller variance is more desirable for developing a prediction model, the process assigned more weight to the variables that have smaller variance. Thus, each simulation included not only the most dominant variable but also the least dominant variable among input variables.
Step 4: The iteration process was performed N times. A value of OTPIN was computed and stored iteration by iteration. The process stopped when the number of iterations reached the desired level.
Step 5: A cumulative frequency curve and a histogram of all OTPINs were plotted and the summary statistics of simulation results were reported. A tornado graph was plotted to determine what factors had the most influence on the success of the project. Regression sensitivity for OTPI was reported.
Tools and Programming for Simulation
In this study, the @ Risk Version 5.5 add- in for Microsoft Excel was implemented to perform Monte Carlo simulation procedures. The @ Risk functions and types are accessible to programmers of Excel Visual Basic for Applications ( VBA) and allow them to automate the process of editing @ Risk settings using code, as well as starting and controlling an @ Risk simulation to obtain simulation results (@ Risk 2009 and Kimmel 2003). Graphic User Interfaces were developed using VBA programming. Input forms as data entry screens were created in the Visual Basic Editor.
Figure 15 shows a screen snapshot of the main page of I/ D contracting decision support Mineta Transportation Institute
Decision Support Model Development
47
model. A dialog box of project variable selection for a roadway resurfacing project is shown in Figure 16. The input dialog box includes seven options of project variable selections. Each drop down box has two to eight levels of the variable with “ N/ A” as one of the options.
When the “ NEXT” button is clicked in the project variable selection dialog box, the dialog box of project performance selection, shown in Figure 17, pops up. The user then selects one of the performance indices. When the “ START” button is clicked in the form displayed in Figure 17, a report of simulation results is generated and displayed, as shown in Figure 18.
Figure 15 Main Page of I/ D Contracting Decision Support Model Mineta Transportation Institute
48 Decision Support Model Development
Figure 16 Project Variable Selection Dialog Box for Project FIN 412481
Figure 17 Performance Index Selection Dialog Box Mineta Transportation Institute
Decision Support Model Development
49
Figure 18 Report of Project Performance Simulation Results for
Project No. 412481
Interpretation of Simulation Results
A probability distribution is well known as a device for presenting the quantified risk for a variable. The simulation result is also easy to understand since the output probability Mineta Transportation Institute
50 Decision Support Model Development
distribution graphically displays the probabilities and users can get a feel for the risks involved. Since the output probability distribution describes a range of possible values and their likelihood of occurrence, the decision- maker can easily recognize that some outcomes are more likely to occur than others.
A histogram of all OTPINs and a cumulative frequency curve of all OTPINs are shown in Figure 19 and 20, respectively. The interpretation of the histogram and cumulative curve can answer the following questions from the project planners:
What is the most likely 1. OTPI value of the simulation result?
What is the probability that the actual project time performance will be ahead of 2. schedule or on time?
What is the probability that the actual project cost will not exceed project contracting 3. amount?
What is the project planner’s certainty that the project performance index will be higher 4. than a specific level?
A tornado graph that demonstrates what factors have the most influence on the success of the project is shown in Figure 21. In this example case, the most dominant factor was the maximum incentive amount while the least dominant factor was daily I/ D amount. The probability that the actual project time performance will be ahead of schedule or on time is approximately 70%.
Figure 19 Histogram of OTPI Simulation Results for Project No. 412481 Mineta Transportation Institute
Decision Support Model Development
51
Figure 20 Cumulative Curve of OTPI Simulation Results for Project No. 412481
Figure 21 Tornado Graph of OTPI Simulation Results for Project No. 412481 Mineta Transportation Institute
52 Decision Support Model Development
Mineta Transportation Institute
53
MODEL VALIDATION
Unlike a regression prediction model, the developed simulation model is not designed to predict a specific value but instead is designed to predict a range of values with probability. It is also possible that an actual value falls out of a prediction range of the simulation model because the prediction results are based on the performance of historical projects. However, the accuracy of the performance prediction range is important to ensure the project planners can use the developed model with confidence. As a result, the developed simulation model needs to be validated through project case studies.
Project Dataataata for Validationalidation
A total of 30 additional FDOT construction projects not included in developing the proposed model were used to investigate the prediction accuracy of the simulation model. All projects were completed in Florida and accepted in fiscal year 2007 to 2008. All three contract types were used for 16 different project work types. There were ten resurfacing projects completed and eight add lane or turn lane projects using I/ D, A+ B I/ D, or A+ B Bonus I/ D.
Project duration varied from 50 to 1200 days and original contract amounts ranged from $ 513,256 to $ 80,159,992. The daily I/ D amounts varied from $ 2,000 to $ 10,000 and the maximum incentive amount proposed ranged from $ 50,000 to $ 4,600,000. Twenty- one contractors completed 30 projects and each contractor finished up to three projects during the case study period. The input data of the 30 cases used in the simulation are shown in Table 20.
Of the 30 I/ D projects, contractors were able to achieve incentives from 21 projects and the overall incentive achievement rate was approximately 70%. Total incentive amount paid was $ 9,993,235 and the incentive amounts achieved varied from $ 9,900 to $ 4,600,000 with an average of $ 326,708. During the case study period, one contractor was charged with a disincentive of $ 192,000 from a resurfacing project. Approximately 27% of the time, contractors were not able to achieve incentives or were not charged with any disincentives. Table 21 shows the number of projects and dollar amounts paid for incentives by contract types as well as by project types during the case study period.
Validationalidation Method and Resultsesults
For the model validation purpose, an analysis of project performance prediction range was used to test whether an actual performance value falls within the expected boundary of the minimum and the maximum of simulation values. Four simulations were run for each project case and a total of 120 simulations were performed in the cases of OTPI, PTPI, OCPI, and PCPI. Mineta Transportation Institute
Model Validation
54
Input Data Used in OPTI SimulationTable 20
Case
Contract Type
Project Type
District
Project Size
Project Length*
Max.
Incentive*
Daily I/ D Amount*
1
A+ B I/ D
RRR
03
PSL
N/ A
N/ A
N/ A
2
A+ B I/ D
RRR
05
PSM
PLAA
N/ A
N/ A
3
A+ B Bonus I/ D
RRR
06
PSL
PLBA
MIL
N/ A
4
A+ B Bonus I/ D
RRR
06
PSL
N/ A
MIL
DIL
5
A+ B I/ D..............
RRR
05
PSL
PLAA
N/ A
N/ A
6
A+ B I/ D
RRR
05
PSL
PLAA
N/ A
N/ A
7
I/ D
RRR
06
PSM
N/ A
N/ A
N/ A
8
I/ D
RRR
06
PSS
N/ A
N/ A
N/ A
9
I/ D
BRR
02
PSL
N/ A
N/ A
N/ A
10
I/ D
Others
04
PSM
N/ A
N/ A
N/ A
11
I/ D
RRP
06
PSM
N/ A
N/ A
N/ A
12
A+ B I/ D
RRR
05
PSL
PLBA
N/ A
N/ A
13
I/ D
RRR
08
PSM
N/ A
N/ A
N/ A
14
A+ B I/ D
RRR
05
PSL
PLAA
N/ A
N/ A
15
I/ D
RRR
06
PSS
N/ A
N/ A
N/ A
16
A+ B I/ D
RRR
01
PSL
PLAA
N/ A
N/ A
17
I/ D
Others
06
PSS
N/ A
MIS
DIM
18
I/ D
RRP
04
PSM
N/ A
N/ A
N/ A
19
I/ D
RRP
04
PSM
N/ A
N/ A
N/ A
20
I/ D
RRP
04
PSM
N/ A
N/ A
N/ A
21
I/ D
RRP
06
PSM
PLAA
MIM
DIL
22
I/ D
RRP
04
PSM
N/ A
N/ A
N/ A
23
I/ D
RRP
06
PSS
N/ A
MIS
DIS
24
I/ D
RRP
06
PSM
PLAA
MIM
DIL
25
I/ D
RRP
06
PSM
N/ A
MIM
DIL
26
A+ B I/ D
RRP
05
PSL
PLBA
N/ A
N/ A
27
I/ D
RRP
06
PSS
N/ A
MIM
DIM
28
I/ D
RRR
06
PSM
PLAA
N/ A
N/ A
29
I/ D
Others
06
PSS
N/ A
N/ A
N/ A
30
I/ D
Others
06
PSS
N/ A
N/ A
N/ A
* Note that not all project data were available. Mineta Transportation Institute
Model Validation
55
I/ D Amount Achieved by Contract TypesTable 21
Contract Type
Project Work Description
No. of Projects
Incentive
Paid(+) / Disincentive Charged(-)
I/ D
Add turn lane( s)
2
$ 280,000
Bridge- repair/ rehabilitation
1
$ 500,000
Drainage improvements
1
$ 73,000
Highway access improvement
1
$ 0
Interchange ( major)
1
$ 0
Intersection ( minor)
1
$ 28,000
Pedestrian safety improvement
1
$ 34,000
Resurfacing
9
$ 1,060,135 / -$ 192,000
Rigid pavement reconstruction
1
$ 200,000
Safety improvement
1
$ 40,000
Sidewalk
1
$ 9,900
I/ D Total
20
$ 2,033,035
A+ B I/ D
Add lanes & reconstruct
2
$ 406,000
Add lanes & rehabilitate pave­ment
2
$ 392,200
Interchange ( major)
2
$ 798,000
New road construction
1
$ 372,000
Resurfacing
1
$ 0
A+ B I/ D Total
8
$ 1,968,200
A+ B Bonus I/ D
Add lanes & reconstruct
2
$ 5,800,000
A+ B Bonus I/ D Total
2
$ 5,800,000
Grand Total
30
$ 9,801,235 Mineta Transportation Institute
56 Model Validation
OTPI Simulation Case Study Results
An analysis of the prediction range of each simulation was performed in order to evaluate whether the actual OTPI value falls within the expected boundaries of the minimum and the maximum. Of the 30 project cases studied, the actual OTPI values of two projects fell outside this expected maximum boundary. Two projects exceeded the expected maximum by 0.222 and 0.258, respectively. They are an average of 31% greater than the expected range ( 35% of historical OTPI dataset). However, in most cases, the actual OTPI values fell within the limits, as shown in Figure 22.
The mean value of historical OTPI data used in this model was 0.062 and the minimum and maximum OTPIs were - 0.710 ( i. e. 71% time savings) and 1.567 ( i. e. 156.7% time overruns). It was calculated that the range of the historical data set is 2.277 ( i. e. 227.7%). In comparison to this broad range, the time performance prediction range of OTPI simulation results showed much narrower range ( i. e. 18% to 49% of the historical data range) in order to predict the actual OTPI for each case. Considering these circumstances, the prediction range of actual OTPI was reasonably accurate in that approximately 93% of cases were within the predicted range. The simulation results for OTPI are shown in Table 22.
Figure 22 OTPI Simulation Case Study Results Mineta Transportation Institute
Model Validation
57
OTPI Simulation ResultsTable 22
Case
Project FIN
Expected minimum
Expected maximum
Expected mean
Actual OTPI
Most Dominant Factor
Correlation
1
21972215201
- 0.279
0.569
0.129
- 0.179
District
0.625
2
23876215201
- 0.271
0.676
0.114
- 0.043
Contract Type
0.472
3
24964815201
- 0.300
0.324
- 0.015
- 0.078
Contract Type
0.526
4
24965315201
- 0.336
0.298
- 0.015
0.189
Contract Type
0.545
5
23842115201
- 0.307
0.609
0.127
- 0.177
Contract Type
0.459
6
24271615201
- 0.307
0.609
0.127
0.197
Contract Type
0.459
7
24961455201
- 0.363
0.516
0.000
- 0.214
District
0.632
8
41642345201
- 0.427
0.563
- 0.027
- 0.221
District
0.622
9
20961655201
- 0.431
0.694
0.082
0.033
District
0.557
10
40653615201
- 0.380
0.443
- 0.034
- 0.276
Project Type
0.635
11
41275425201
- 0.390
0.590
- 0.016
0.848
District
0.601
12
24270225201
- 0.232
0.632
0.130
- 0.183
Project Length
0.482
13
40611215201
- 0.374
0.565
0.064
0.102
District
0.576
14
24253115201
- 0.246
0.599
0.127
0.189
Contract Type
0.457
15
41642325201
- 0.427
0.563
- 0.027
- 0.120
District
0.622
16
42064715201
- 0.281
0.643
0.131
- 0.229
Contract Type
0.468
17
41823615201
- 0.363
0.054
- 0.157
- 0.120
Max Incentive
0.776
18
22807315201
- 0.373
0.536
0.055
0.027
Project Type
0.566
19
22862315201
- 0.373
0.536
0.055
- 0.198
Project Type
0.566
20
22974915201
- 0.373
0.536
0.055
- 0.174
Project Type
0.566
21
40763315201
- 0.334
0.272
- 0.039
0.494
Max Incentive
0.549
22
41143815201
- 0.373
0.536
0.055
- 0.135
District
0.566
23
41247615201
- 0.348
0.083
- 0.153
0.000
Max Incentive
0.805
24
41248115201
- 0.334
0.272
- 0.039
- 0.089
Max Incentive
0.549
25
41248415201
- 0.362
0.332
- 0.050
- 0.115
Max Incentive
0.556
26
41552715201
- 0.240
0.492
0.102
0.020
Project Type
0.533
27
41791415201
- 0.341
0.310
- 0.066
0.064
Max Incentive
0.545
28
25166235201
- 0.350
0.540
0.010
0.048
District
0.587
29
25166235201
- 0.450
0.327
- 0.102
- 0.161
Project Type
0.595
30
41823635201
- 0.450
0.327
- 0.102
- 0.121
Project Type
0.595
PTPI Simulation Case Study Results
An analysis of prediction range was performed in order to evaluate whether the actual PTPI values fall within the expected boundaries of the minimum and the maximum. Of the 30 project cases, the actual PTPI values of only one project fell outside of the expected maximum boundary. It was close to the expected upper boundary but exceeded the expected maximum by 0.057, which is 17% greater than the expected range ( 15% of Mineta Transportation Institute
58 Model Validation
historical PTPI dataset). However, in all other cases, the actual PTPI values fell within the limits, as shown in Figure 23.
The mean value of historical PTPI data used in this model was - 0.069 and the minimum and maximum PTPIs were - 0.717 ( i. e. 71.7% time savings) and 1.567 ( i. e. 156.7% time overruns). Therefore, the range of the historical PTPI data set was 2.284 ( i. e. 228.4%). In comparison to this broad range, the time performance prediction range of PTPI simulation results showed much narrower range ( i. e. 15 to 30% of the historical data range) in order to predict the actual PTPI for each case. Considering these circumstances, the prediction range of actual PTPI was quite accurate in that approximately 97% of cases were within the predicted range. The simulation results for PTPI are shown in Table 23.
Figure 23 PTPI Simulation Case Study Results Mineta Transportation Institute
Model Validation
59
PTPI Simulation ResultsTable 23
Case
Project FIN
Expected Minimum
Expected Mean
Actual PTPI
Most Dominant Factor
Correlation
1
21972215201
- 0.302
- 0.031
- 0.179
District
0.557
2
23876215201
- 0.256
- 0.052
- 0.079
Contract Type
0.479
3
24964815201
- 0.244
- 0.075
- 0.075
Contract Type
0.592
4
24965315201
- 0.234
- 0.073
0.000
Contract Type
0.521
5
23842115201
- 0.264
- 0.048
- 0.179
Contract Type
0.486
6
24271615201
- 0.264
- 0.048
0.000
Contract Type
0.486
7
24961455201
- 0.349
- 0.075
- 0.214
Project Size
0.616
8
41642345201
- 0.383
- 0.086
- 0.221
Contract Type
0.523
9
20961655201
- 0.352
- 0.068
0.000
Project Size
0.522
10
40653615201
- 0.375
- 0.102
- 0.287
District
0.575
11
41275425201
- 0.338
- 0.089
0.000
Project Type
0.581
12
24270225201
- 0.286
- 0.047
- 0.183
Project Length
0.531
13
40611215201
- 0.340
- 0.067
- 0.002
District
0.646
14
24253115201
- 0.282
- 0.048
0.000
Contract Type
0.523
15
41642325201
- 0.383
- 0.086
- 0.120
Contract Type
0.523
16
42064715201
- 0.250
- 0.045
- 0.253
Contract Type
0.493
17
41823615201
- 0.332
- 0.155
- 0.158
Max Incentive
0.627
18
22807315201
- 0.334
- 0.078
- 0.019
District
0.538
19
22862315201
- 0.334
- 0.078
- 0.198
District
0.538
20
22974915201
- 0.334
- 0.078
- 0.197
District
0.538
21
40763315201
- 0.291
- 0.099
0.101
Daily I/ D Amount
0.499
22
41143815201
- 0.334
- 0.078
- 0.172
District
0.538
23
41247615201
- 0.349
- 0.152
0.000
Max Incentive
0.664
24
41248115201
- 0.291
- 0.099
- 0.104
Daily I/ D Amount
0.499
25
41248415201
- 0.282
- 0.100
- 0.137
Daily I/ D Amount
0.571
26
41552715201
- 0.232
- 0.058
0.000
Project Length
0.490
27
41791415201
- 0.343
- 0.120
0.000
Daily I/ D Amount
0.487
28
25166235201
- 0.319
- 0.080
- 0.141
Project Length
0.514
29
25166235201
- 0.437
- 0.138
- 0.188
Project Type
0.614
30
41823635201
- 0.437
- 0.138
- 0.201
Project Type
0.614
OCPI Simulation Case Study Results
An analysis of prediction range was performed in order to evaluate whether the actual OCPI values fall within the expected boundaries of the minimum and the maximum. Of the 30 project cases, the actual OCPI values of two projects fell outside of the expected maximum or minimum boundaries. One was very close to the expected lower boundary, Mineta Transportation Institute
60 Model Validation
but exceeded the expected minimum by - 0.011, which is 4% smaller than the expected range ( 25% of historical OCPI dataset). The other project case exceeded the expected maximum by 0.105, which is 35% greater than the expected range ( 26% of historical OCPI dataset). However, in all other cases, the actual OCPI values fell within the limits, as shown in Figure 24.
The mean value of historical OCPI data used in this model was 0.063 and the minimum and maximum OCPIs were - 0.345 ( i. e. 34.5% cost savings) and 0.763 ( i. e. 76.3% cost overruns). It was calculated that the range of the historical OCPI data set is 1.107 ( i. e. 110.7%). In comparison to this relatively broad range, the cost performance prediction range of OCPI simulation results showed much narrower range ( i. e. 20 to 43% of the historical data range) in order to predict the actual OCPI for each case. Considering these circumstances, the prediction range of actual OCPI was reasonably accurate in that approximately 93% of cases were within the predicted range. The simulation results for OCPI are shown in Table 24.
Figure 24 OCPI Simulation Case Study Results Mineta Transportation Institute
Model Validation
61
OCPI Simulation ResultsTable 24
Case
Project FIN
Expected Minimum
Expected Maximum
Expected Mean
Actual OCPI
Most Dominant Factor
Correlation
1
21972215201
- 0.073
0.256
0.070
0.036
District
0.550
2
23876215201
- 0.073
0.199
0.066
- 0.007
District
0.553
3
24964815201
- 0.047
0.214
0.076
0.106
Contract Type
0.636
4
24965315201
- 0.033
0.194
0.079
0.036
Contract Type
0.567
5
23842115201
- 0.049
0.184
0.068
0.047
District
0.614
6
24271615201
- 0.049
0.184
0.068
0.134
District
0.614
7
24961455201
- 0.129
0.222
0.054
0.017
District
0.575
8
41642345201
- 0.146
0.212
0.032
- 0.120
District
0.544
9
20961655201
- 0.078
0.281
0.076
0.112
District
0.549
10
40653615201
- 0.143
0.333
0.079
- 0.063
Project Size
0.585
11
41275425201
- 0.131
0.292
0.045
0.111
District
0.586
12
24270225201
- 0.045
0.224
0.066
0.061
District
0.636
13
40611215201
- 0.091
0.353
0.094
0.064
Project Type
0.531
14
24253115201
- 0.069
0.195
0.068
0.061
District
0.613
15
41642325201
- 0.146
0.212
0.032
0.087
District
0.544
16
42064715201
- 0.051
0.249
0.076
0.108
Project Length
0.543
17
41823615201
- 0.108
0.148
0.023
- 0.063
Daily I/ D Amount
0.543
18
22807315201
- 0.115
0.333
0.075
0.007
Project Type
0.518
19
22862315201
- 0.115
0.333
0.075
0.032
Project Type
0.518
20
22974915201
- 0.115
0.333
0.075
0.054
Project Type
0.518
21
40763315201
- 0.076
0.229
0.063
- 0.071
Project Length
0.419
22
41143815201
- 0.115
0.333
0.075
- 0.019
Project Type
0.518
23
41247615201
- 0.120
0.124
0.002
- 0.027
Daily I/ D Amount
0.493
24
41248115201
- 0.076
0.229
0.063
- 0.045
Project Length
0.419
25
41248415201
- 0.073
0.199
0.061
0.112
Daily I/ D Amount
0.464
26
41552715201
- 0.057
0.227
0.062
- 0.068
District
0.667
27
41791415201
- 0.077
0.195
0.042
0.008
Daily I/ D Amount
0.545
28
25166235201
- 0.078
0.223
0.058
0.328
Project Length
0.560
29
25166235201
- 0.165
0.199
0.019
- 0.072
District
0.582
30
41823635201
- 0.165
0.199
0.019
- 0.134
District
0.582 Mineta Transportation Institute
62 Model Validation
PCPI Simulation Case Study Results
An analysis of prediction range was performed in order to evaluate whether the actual PCPI values fall within the expected boundaries of the minimum and the maximum. Of the 30 project cases, the actual PCPI values of only one project fell outside of the expected minimum boundary. It exceeded the expected minimum by - 0.039, which is 26% greater than the expected range ( 18% of historical PCPI dataset). However, in all other cases, the actual PCPI values fell within the limits, as shown in Figure 25.
The mean value of historical PCPI data used in this model was 0.008 ( i. e. 0.8% cost overruns) and the minimum and maximum PCPIs were - 0.345 ( i. e. 34.5% cost savings) and 0.511 ( i. e. 51.1% cost overruns). The range of the historical PCPI data set was 0.855 ( i. e. 85.5%). In comparison to this relatively broad range, the cost performance prediction range of PCPI simulation results showed much narrower range ( i. e. 15 to 33% of the historical data range) in order to predict the actual PCPI for each case. Considering these circumstances, the prediction range of actual PCPI was quite accurate in that approximately 97% of cases were within the predicted range. The simulation results for PCPI are shown in Table 25.
Figure 25 PCPI Simulation Case Study Results Mineta Transportation Institute
Model Validation
63
PCPI Simulation ResultsTable 25
Case
Project FIN
Expected Minimum
Expected Maximum
Expected Mean
Actual PCPI
Most Dominant Factor
Correlation
1
21972215201
- 0.084
0.104
0.017
0.034
Contract Type
0.602
2
23876215201
- 0.060
0.080
0.013
- 0.018
Contract Type
0.472
3
24964815201
- 0.046
0.102
0.028
0.065
Contract Type
0.616
4
24965315201
- 0.037
0.105
0.033
0.019
Contract Type
0.553
5
23842115201
- 0.052
0.080
0.014
0.026
Contract Type
0.490
6
24271615201
- 0.052
0.080
0.014
0.052
Contract Type
0.490
7
24961455201
- 0.096
0.140
0.010
0.008
Project Type
0.569
8
41642345201
- 0.133
0.114
- 0.001
- 0.120
Project Type
0.593
9
20961655201
- 0.081
0.080
0.010
0.027
District
0.638
10
40653615201
- 0.101
0.136
0.015
- 0.081
District
0.684
11
41275425201
- 0.117
0.111
0.005
0.020
Project Type
0.562
12
24270225201
- 0.070
0.092
0.010
0.043
Contract Type
0.514
13
40611215201
- 0.088
0.198
0.021
0.006
Project Type
0.635
14
24253115201
- 0.068
0.091
0.014
0.033
Contract Type
0.478
15
41642325201
- 0.133
0.114
- 0.001
0.087
Project Type
0.593
16
42064715201
- 0.058
0.075
0.012
0.008
District
0.477
17
41823615201
- 0.116
0.071
- 0.003
- 0.063
Daily I/ D Amount
0.604
18
22807315201
- 0.089
0.117
0.011
- 0.003
Daily I/ D Amount
0.604
19
22862315201
- 0.089
0.117
0.011
0.021
Daily I/ D Amount
0.604
20
22974915201
- 0.089
0.117
0.011
0.002
Daily I/ D Amount
0.604
21
40763315201
- 0.055
0.095
0.020
- 0.094
Max Incentive
0.509
22
41143815201
- 0.089
0.117
0.011
- 0.028
Daily I/ D Amount
0.604
23
41247615201
- 0.118
0.061
- 0.019
- 0.027
Project Type
0.509
24
41248115201
- 0.055
0.095
0.020
- 0.045
Max Incentive
0.509
25
41248415201
- 0.058
0.098
0.020
0.077
Max Incentive
0.588
26
41552715201
- 0.079
0.076
0.008
- 0.068
Contract Type
0.513
27
41791415201
- 0.061
0.098
0.013
- 0.045
Max Incentive
0.554
28
25166235201
- 0.082
0.112
0.013
- 0.016
Project Length
0.580
29
25166235201
- 0.159
0.127
- 0.010