Appropriate Project Characteristics for Public
Sector Design-Build Projects

Owner Survey Results



Introduction
This document summarizes the results of a design-build questionnaire sent to public sector owners during June and July of 1995. This questionnaire is an initial step in a three-year National Science Foundation funded research effort at the University of Colorado at Boulder. The goal of this research is to develop an automated selection tool for public sector design-build projects.

Summary
The questionnaire reached a wide cross section of the public sector design-build population. It confirmed that all of the project characteristics impact project success because they all had high ratings from at least a few respondents. There are six project characteristics which are crucial to project success. The results also showed that there are three critical criteria by which public sector owners judge design- build success and one main reason why public sector owners choose design-build. The results are summarized in Table 1.

Table 1: Results Summary

Critical Project Characteristics for Sector Public Design-Build	Critical Success Criteria for Public Sector Design-Build	Critical Selection Factor for Public Sector Design-Build
1. Well Defined Scope	1. On Budget	1. Shorten Duration
2. Shared Understanding of Scope	2. Conforms to User’s Expectation
3. Adequate Owner Staffing	3. On Schedule
4. Owner’s Construction Sophistication
5. Established Budget
6. Established Completion Date

Definitions for all of the project characteristics, success criteria, and selection factors are given in Tables 8 - 10.

Population Characteristics

The questionnaire was prepared and initially sent to 180 public sector owners. The recipients were chosen from a list of public sector owners requesting information from the Design-Build Institute of America (DBIA) and a list of attendees of the second annual Design-Build Conference. Also, these recipients were asked to forward the questionnaire to others in their agency with design-build experience. The survey was then sent a second time to owners who did not return the first mailing and also to new contacts found through the first mailing.

The final total of surveys sent is 211. Of the 211 owners contacted, 88 completed the survey and 49 replied that they did not have proper experience to answer the survey or could not complete the survey at this time. This represents a 65% response to the survey. Agencies represented in the survey are listed in Table 2.

Table 2: Agencies Represented in the Survey

Alaska Department of Transportation & Public Facilities	U.S. Army Community & Family Support Center
Baltimore County Board of Education, Maryland	U.S. Army Corps of Engineers
City of College Park, Georgia	U.S. Army Corps of Engineers - Albuquerque
City of Laramie, Wyoming	U.S. Army Corps of Engineers - Military Programs HQ
City of Midland, Michigan	U.S. Army Corps of Engineers - Mobile District
City of Renton, Washington	U.S. Army Corps of Engineers - Pacific Ocean
City of San Diego, California	U.S. Army Corps of Engineers - Trans Atlantic Division
Dade County Public Schools, Florida	U.S. Department of Commerce
East Bay Municipal Utility District - Oakland, California	U.S. Department of Defense
Georgia State Financing and Investment Commission	U.S. Department of Navy
Greenville Water System, South Carolina	U.S. Department of State
Harford Community College, Maryland	U.S. Department of Transportation
Los Alamos National Laboratories, New Mexico	U.S. Department of Veterans Affairs
Mesa County, Colorado	U.S. Environmental Protection Agency
Metropolitan Sewer District - Louisville, Kentucky	U.S. Forest Service
National Aeronautics and Space Administration	U.S. General Services Administration
Nassau County, New York	U.S. Naval Facilities Engineering Command
New Jersey Transit Authority - Engineering & Construction	U.S. Naval Facilities Engineering Command - Atlantic Division
Orange County Public Schools, California	U.S. Naval Facilities Engineering Command - Contracts
Palm Beach County Capital Improvements Division	U.S. Naval Facilities Engineering Command - Southern Division
Smithsonian Institution	U.S. Navy Public Works Center
South Carolina, State Budget & Control Board, Office of General Services	U.S. Postal Service
State of Florida - Department of Transportation	University of Maryland System Administration
Tennessee Valley Authority	University of Medicine and Dentistry, New Jersey
U.S. Air Force	University of Missouri
U.S. Air Force Center for Environmental Excellence	Washington State Parks and Recreation Commission

The agencies surveyed represent federal, state, and local governments. Figure 1 illustrates the distribution of these of agencies surveyed. Also, these agencies construct various types of projects. Respondents were asked to classify their primary type of construction projects into the following categories: building, heavy and highway, industrial, or other. The results of the construction type represented in the survey are illustrated in Figures 2.



Figure 1: Types of Agency Surveyed


Figure 2: Types of Construction Represented in Survey


Respondents were also asked to quantify the number of design-build projects they have been associated with and the total dollar amount of these projects. The respondents have individual experience with approximately 763 design-build projects totaling over $7.6 billion of construction. It should be noted that since there are some agencies represented by more than one respondent, there may be redundancy in the project totals noted. Disregarding the redundancy, the totals represent the cumulative volume with which the individuals have experience.

Survey Results
The survey was divided into three sections. The primary section asked respondents to rate the fifteen project characteristics in terms of importance towards a successful design-build project. The next section on design-build selection factors asked respondents to rank the factors as motivation for choosing design-build. The final section on design-build success criteria asked respondents to rank criteria for design-build success. A discussion of the survey results follows.

Project Characteristic Results
The project characteristic portion of the survey asked the owners to rate each characteristic’s impact on project success (1-6, 6 being highest). The results are summarized in Table 3.

Table 3: Project Charateristic Results

Characteristic Description	Mean	Rank	Median	Rank	Min	Max
Well Defined Scope	5.41	1	6	1	1	6
Shared Understanding of Scope	5.22	2	6	1	1	6
Owner's Construction Sophistication	4.62	3	5	3	1	6
Adequate Owner Staffing	4.57	4	5	3	1	6
Established Budget	4.35	5	4	5	1	6
Established Completion Date	4.16	6	4	5	1	6
Availability of Design-builders	3.79	7	4	5	1	6
Willingness to Forgo Design Input	3.78	8	4	5	1	6
Owner's Risk Aversion	3.72	9	4	5	1	6
Standard Design Specifications	3.56	10	4	5	1	6
Technologically Advanced	3.20	11	3	11	1	6
Type of Contract	3.17	12	3	11	1	6
Size of Project	3.16	12	3	11	1	6
Current State of the Market	2.97	14	3	11	1	6
Alternative Financing Options	1.78	15	1	15	1	6

The project characteristics are sorted by mean score. Rankings for median score are shown as well. While mean and median rankings agree, the mean score offers more insight into the relationship of the ranking. For example, there is only one number 5 ranking by mean score, but there are six number 5 rankings by median. The minimum and maximum also offer some insight into the range of rankings returned, but offer no insight into the frequency of these rankings. Frequency tables and histograms which follow will offer better insight into the rankings.

There were three goals to the owner survey: validate, order, and find additional characteristics. The fact that all characteristics scored at least one maximum rating (6) validates the fact that they all are important. This is especially true since the characteristics are rated against one another and would stand out even more against arbitrary project characteristics.

The second goal of ordering the characteristics was achieved through the mean score rankings. The characteristics can be ranked 1 through 15 and a relative weighting can be assigned to each. A graphical representation of the ranking through mean scores is shown in Figure 3.



Figure 3: Project Characteristics Ranking by Mean Score

As clearly seen in the mean score graph, the two characteristics which have the highest impact on project success are Well Defined Scope and Shared Understanding of Scope. The next plateau displayed on the graph is with the characteristics of Adequate Owner Staffing and Owner’s Construction Sophistication which have a mean score approximately 13% lower than the first two. The next two project characteristics, Established Budget and Established Completion Date, drop approximately 6% and 10% from the third and fourth. These top six characteristics are the ones which owners feel most strongly impact success. The owners also feel strongly about one which least impacts project success the least, Alternative Financing Options, as seen in the 40% drop from the 14th ranked characteristic.

The dominance of the top six characteristics is illustrated more clearly in the form of frequency histograms. Shown in Figure 4 are the frequency histograms for the 1st, 9th, and 15th characteristics.



Figure 4: Frequency Histograms for 1st, 9th, and 15th Project Characteristics


A substantial positive skew is seen in the 1st histogram and a substantial negative skew is shown in the 15th. The histogram for the 9th histogram has no strong positive or negative skew. The prominent skewness represents that the majority of owners rated the characteristic high or low. An absence of skewness represents that the owners have differing views as to impact on project success.

From the information obtained through the mean scores and frequency histograms, it is apparent that owners think the top six rated characteristics have a high impact on design-build project success. The 7th through 14th have relatively less impact on success and the 15th has relatively no impact on project success when compared to the other characteristics.

The final goal of the project characteristic portion of the survey was to find any additional project characteristics which may have been overlooked. This was accomplished through the comments portion of the survey. One additional characteristic found in the owner comments related to repetitive nature of work. A similar characteristic (Repetitiveness of Elements) was found in the literature review but disregarded during the unstructured interview portion of the research. The unstructured interviews concluded that a project which has repetitive elements is encompassed in the project having Standard Design Specifications. In retrospect, this characteristic should have probably remained in the final list of appropriate project characteristics.

Project Success Criteria Results
The project success criteria portion of the survey asked owners to rank the six success criteria (1 being most important). The results are summarized in Table 4.

Table 4: Success Criteria Results

Success Criteria Description	Mean	Rank	Median	Rank	Min	Max
On Budget	2.20	1	2	1	1	6
Conform to User's Expectaions	2.67	2	3	3	1	6
On Schedule	2.91	3	3	3	1	6
Meets Specifications	3.65	4	4	4	1	6
High Quality of Workmanship	4.38	5	5	5	2	6
Minimizes Construction Aggravation	5.08	6	6	6	1	6

The success criteria are sorted by mean score. Rankings for median score are shown as well. While mean and median rankings agree, the mean score offers more insight into the relationship of the ranking. For example, there is only one number 3 ranking by mean score, but there are two number 3 rankings by median score. The minimum and maximum scores are also shown as well.

The individual rankings of the six success criteria yield a mean score which can be used to achieve an overall ranking. This overall ranking is shown graphically in Figure 5.



Figure 5: Success Criteria Ranking by Mean Score


As graphically displayed in Figure 5, owners strongly feel the On Budget is the most important criteria for project success. The next two criteria, Conforms to User’s Expectations and On Schedule, are ranked very closely. The next three criteria are less important, and the last criteria is ranked extremely low.

The three highest ranked criteria show a dominance over the lowest three. This dominance is displayed in the frequency with which the criteria scored in the top three rankings. The frequencies are shown in Table 5 as well as in the cumulative frequency chart in Figure 6. Frequency charts for the individual success criteria are included in Appendix V - Owner Survey Results.

Table 5: Success Criteria Frequency for Top Three Rankings

Success Criteria Description	#1 Rankings	#2 or > Rankings	#3 or > Rankings
On Budget	20	43	58
Conform to User's Expectaions	21	31	49
On Schedule	17	33	42
Meets Specifications	4	11	27
High Quality of Workmanship	0	7	13
Minimizes Construction Aggravation	5	7	9

Figure 6: Success Criteria Cumulative Frequency Chart


As displayed in the table and chart above, the top three criteria are shown to be the most important. They are rated 1, 2, or 3 much more frequently than the bottom three criteria. The concavity downward on the cumulative frequency versus the concavity upward also displays a dominance. The data shows that On Budget, Conforms to User’s Expectations, and On Schedule are the most important criteria by which public sector owners judge success design-build project success.

Selection Factor Results

The design-build selection factor portion of the survey asked owners to rank the seven factors (1 being most important). The results are summarized in Table 6.

Table 6: Design-Build Selection Factor Results

Selection Factor Description	Mean	Rank	Median	Rank	Min	Max
Shorten Duration	2.48	1	2	1	1	7
Establish Cost	3.52	2	4	2	1	7
Reduce Cost	3.68	3	4	3	1	7
Constructability Innovation	3.82	4	4	3	1	7
Reduce Claims	4.08	5	4	3	1	7
Establish Schedule	4.32	6	4	3	1	7
Large Project Size/Complexity	6.11	7	7	7	1	7

The success criteria are sorted by mean score. Rankings for median score are shown as well. While mean and median rankings agree, the mean score offers more insight into the relationship of the ranking. For example, there is only one number 3 ranking by mean score, but there are four number 3 rankings by median score. The minimum and maximum scores are also shown.

The individual rankings of the seven success criteria yield a mean score which can be used to achieve an overall ranking. This overall ranking is shown graphically in Figure 7.


Figure 7: Selection Factor Ranking by Mean Score


As displayed by Figure 7, there are only two selection issues which the owners feel strongly about. The number one reason why owners choose design-build is to Shorten Duration, and they do not feel strongly inclined to choose design-build due to having Large Project Size/Complexity. These strong inclinations are shown graphically in the large difference between the first, last, and middle five selection factors.

As with the project characteristics and success criteria, the dominance of these two factors over the middle five is displayed in the frequency histograms and cumulative frequency charts. The frequency histograms in Appendix V - Owner Survey Results, show a strong positive and negative skew, while the middle five selection factors display an almost uniform distribution. The dominance is also apparent in Table 7 and Figure 8.

Table 7: Selection Factor Frequency for Top Three Rankings

Selection Factor Description	#1 Rankings	#2 or > Rankings	#3 or > Rankings
Shorten Duration	28	41	49
Establish Cost	4	21	30
Reduce Cost	11	21	32
Constructability Innovation	10	19	32
Reduce Claims	6	17	25
Establish Schedule	5	9	24
Large Project Size/Complexity	2	4	5

Figure 8: Selection Factor Cumulative Frequency Chart


As displayed Table 7 and Figure 8 above, the only factor truly displaying a dominance is Shorten Duration. It scored the most #1 rankings as well as the most Cumulative #2 or Greater and #3 or Greater rankings. The middle five selection factors are almost evenly distributed at the #3 or Greater rankings. The lowest ranked factor, Large Project Size/Complexity, has by far the least #3 or Greater rankings. The ordering and uniformity of the middle five factors is graphically displayed in the cumulative frequency chart.

Conclusions
Overall, we are very pleased with the response to the questionnaire. The primary goal of this questionnaire was to answer the question: “What are the appropriate project characteristics for a successful public sector design-build project?” Of the fifteen project characteristics, six are determined to be critical: Well Defined Scope, Shared Understanding of Scope, Owner Sophistication, Adequate Owner Staffing, Established Budget, and Established Completion Date. All owners surveyed, as well as the experts interviewed, concur that these six characteristics are critical in all projects. The remaining nine project characteristics do impact success, but on a project-by-project rather than global basis.

This questionnaire also answers the question: “By what criteria do public sector owners judge project success?” The criteria which are most important are: On Budget, Conforms to User’s Expectations, and On Schedule. The number one criteria is On Budget which is notably more important than both Conforms to User’s Expectations and On Schedule, but all three are significant.

The final question this questionnaire answers is: “What factors are important to public sector owners when choosing design-build?” Overwhelmingly, owners responded that the number one reason they choose design build is to Shorten Duration. Owners believe that the primary advantage of design-build procurement is to shorten duration.

An interesting note is that owners choose design-build to shorten duration, but judge its success more so in terms of budget than schedule. The two items are not necessarily contradictory. Budget is most likely a criteria for success with all types of procurement (i.e. traditional, construction management, etc.) while shortening duration is an advantage of design-build in particular. Also, owners do judge schedule as a significant criteria for success, but not as important as budget.

As a result of this questionnaire, research efforts toward developing a systematic design-build project selection model have been refined. This research has established critical project characteristics, success criteria, and design-build selection factors which will be the basis for building a formalized selection model. The findings of this research are sound building blocks for further research. The items which should be concentrated on for the selection model are in the areas of scope, owner, schedule, and budget. These areas were proven to be critical.

Another area which was discovered late in the research and not included in the findings concerns the type of design-build form used. There are numerous different types of design-build forms available to public sector owners (i.e. One-Step, Two-Step, Source Selection, etc.). These forms differ in two primary areas: the level of design complete before design-builder selection and the method of design-builder selection. The experts interviewed throughout noted that different project types have a better chance of success with different design-build forms. It will be important to determine how project characteristics correlate with design-build types.

This questionnaire has yielded very encouraging results. Design-build will continue to be used in the pubic sector and the findings from this research will make the process more successful.

Table 8: Definitions of Appropriate Design-Build Project Characteristics

PROJECT CHARACTERISTIC
PROJECT
A. Well Defined Scope - The owner has a precise understanding of the project scope before it is submitted to the design-build team.
B. Established Budget - The project has a fixed cost before the it is submitted to the design-build team.
C. Established Completion Date - The project has a fixed schedule or finish date before it is submitted to the design-build team.
D. Standard Design Specifications - The project can utilize design specifications similar to existing projects.
E. Technologically Advanced - The project uses unique or specialized building techniques (e.g., a sewage treatment plant or industrial production plant).
OWNER
F. Owner’s Construction Sophistication - The owner has the ability to precisely define the project scope, either with in-house staff or with a preconstruction consultant.
G. Adequate Owner Staffing - The owner has a project manager or staff that can be dedicated to this specific design-build project.
H. Owner’s Risk Aversion - The owner prefers to shift some of the traditional risks (e.g., design errors and omissions) to the design-builder.
I. Owner’s Willingness to Forgo Design Input - The owner is willing to give up a large amount of design input after design-builder selection.
MARKET
J. Current State of the Market - The amount of work available in the area and the bidding climate therein.
K. Availability of Design-builders - The number of local designers, contractors, and design-build firms with experience.
L. Size of Project - The size and dollar amount of a project as compared to others available for design- builders.
RELATIONSHIP
M. Type of Contract - Whether the project is being awarded as lump sum, unit price, cost-plus, guaranteed maximum price, fixed fee, or other.
N. Shared Understanding of Scope - The owner and design-builder share a clear understanding of functional and technical performance required in the finished project.
O. Alternative Financing Options - The project is using or can utilize third-party financing (e.g., build/operate/transfer).

Table 9: Definitions for Design-Build Selection Factors

Selection Factor	Definition
Establish Cost	Secure a project cost before the start of detailed design.
Reduce Cost	Decrease the overall project cost as compared to other procurement methods (design-bid-build, construction management, etc.).
Establish Schedule	Secure a project schedule before the start of detailed design.
Shorten Duration	Decrease the overall project completion time as compared to other procurement methods (design-bid-build, construction management, etc.).
Reduce Claims	Decrease litigation due to separate design and construction entities.
Large Project Size/ Complexity	The project’s shear magnitude is too complex to be managed through multiple contracts.
Constructability/ Innovation	Introduce construction knowledge into design early in the process.

Table 10: Design-Build Success Criteria

Success Criteria	Definition
On Budget	The project is completed at or under the contracted cost.
On Schedule	The project is completed on or before the contracted finish date.
Meets Specifications	The completed project meets or exceeds all technical performance specifications provided by the owner.
Conforms to User’s Expectations	The completed project meets or exceeds the user’s envisioned functional goals (fitness for purpose).
High Quality of Workmanship	The completed project meets or exceeds the accepted standards of workmanship in all areas.
Minimizes Construction Aggravation	The construction process does not unduly burden the owner’s project management staff.

---

© Keith Molenaar; All Rights Reserved.