II. Project Risks
III. Models
IV. Results
VI. Conclusions
VII. Recommendations
VIII. Appendix A - Base Cost Estimate and Description of Project Scope
IX. Appendix B - Model Schematics
X. Appendix C - Description of DPL Influences
XI. Appendix D - Model Output
XII. Appendix E - Sensitivity Analysis Output
This project was begun in January 1993 following the U. S. Department of Energy's approval of a proposed study to investigate the applicability of cost risk analysis methods to environmental restoration projects. Specifically, the intent of this project is to use Monte Carlo analyses and influence diagramming techniques to model the uncertainties surrounding DOE hazardous waste remediation sites. Software packages being employed on this project include Crystal Ball by Decisioneering, Inc., DPL by ADA Decision Systems, and DynRisk by TerraMar Associates. The procedural and computational efficacy of these packages is being evaluated based on data obtained from case studies of various DOE sites. In addition, the availability of necessary input data as well as the utility and plausibility of model output are being examined. The goal of this research is to analyze the risks associated with past project expenditures in order to derive ranges for total project costs which reflect their inherent uncertainties. From these results, decisions will be made regarding the usefulness of such risk analysis techniques in conjunction with standard cost estimating procedures on future DOE remediation projects.
Remedial projects can involve a great deal of risk that can take many forms. One of the primary forms is the risk of cost overruns. Cost estimates at hazardous waste sites have been known to be subject to a wide range of variability. It is not uncommon to substantially exceed the original cost estimate by the time the cleanup is complete. Difficulty in estimating the costs on a hazardous waste project comes from the many sources of uncertainty present on these types of projects. A formal approach to identifying, classifying, and incorporating risk into the cost estimate is needed to successfully account for the potential cost drivers on these projects.
The fifth case study undertaken as part of this research was the Uranium Mill Tailings Remedial Action (UMTRA) Project in Durango, Colorado. The UMTRA Project involves the cleanup and control of tailings from inactive uranium processing sites to eliminate potential environmental and health hazards, as well as the remediation of groundwater on those sites to the levels established by the EPA. The 24 sites are located in 10 states, primarily encompassing the four corners region of Colorado, New Mexico, Arizona, and Utah. The main objectives of this project are to stabilize the 24 designated sites and clean up the estimated 5,192 "vicinity properties", areas around the sites that have been contaminated due to past uranium mining activities.
This case study examines the incorporation of risk into the cost estimate for the Durango, Colorado UMTRA Project. The Durango Project consists of site work related to the removal of contaminated residual radioactive materials from the abandoned uranium mill site at the Durango processing site and disposal of these materials in an embankment with a protective cover at the Bodo Canyon disposal site. Also included in the scope of this project is the demolition and disposal of existing structures on the site. The base cost estimate for this case study, contained in Appendix A with a description of the scope of work, can be divided into costs for mobilization and site preparation, testing and monitoring, liner installation, tailings piles, cover, erosion protection, decontamination, and site restoration. Throughout the project, unexpected difficulties were encountered which increased the cost of the project from the estimated $14.5 million to $27.3 million, an 88% increase.
In an attempt to represent the risks in this estimate, cost models were created using Crystal Ball and DPL (Decision Programming Language) risk analysis software. A model was also created by combining the powers of the DPL and Crystal Ball softwares. Due to the lack of explicit functional relationships among many project uncertainties, the DynRisk software was not applied to this case study. Models for the Durango Project are centered around the original cost estimate which was modified to include all the risks that could occur during the course of the project. Crystal Ball uses Microsoft Excel spreadsheets and incorporates Monte Carlo probabilistic sampling techniques to analyze risk. Output for Monte Carlo models in this case study consists of distributions and associated probabilities for total project cost and elements of project cost. The independent Crystal Ball model developed here is labeled as the Evocative model. It provides a means of formally addressing risk in the cost estimating procedure and was found to give an adequate, yet rough approximation of total cost as impacted by potential project risks.
The DPL program uses influence diagrams to graphically portray and analyze the risks involved in a project. Nodes are drawn to represent the risks, and arrows are drawn from the risks to the costs which they influence. This provides a graphical representation of the costs and the risks which affect them. Probability trees are used to quantitatively analyze the effects of risk on project costs. As with Crystal Ball, the model output for DPL is a distribution of total project cost. Two models, one with input values generated by approximation and the other with input derived by a Crystal Ball analysis, were developed using the DPL software package. Several other DPL models providing more complex analysis were investigated and will be discussed in the Model Description portion of this paper.
In previous case studies, the implementation of the Crystal Ball software as an independent risk analysis tool has been one focus of investigation. Spreadsheet models that utilized correlation coefficients and conditional probability statements were considered adequate ways of representing risks surrounding project expenditures. However, in this case study, emphasis is placed on the use of influence diagrams, with the Crystal Ball models serving as a support for DPL. Although one Crystal Ball model was developed to perform an independent analysis of risk, the primary purpose of the Monte Carlo approach in this case study is that of an enhancement to the DPL models.
All of the models are modifications of the original cost estimate which has been adjusted to incorporate the risks that could occur during the project. An initial set of risks were identified through meetings with project management personnel and review of project records. The risks were found to be both technical, internal uncertainties and speculative, externally driven uncertainties. From the extensive documentation of contract modifications, the risks which impacted project costs were well-defined. The final list of risks used in the modeling process was as follows:
1. Community Involvement
2. Regulatory/Administrative Change
3. Vicinity Properties
4. Additional Site Preparation/Upgrades
5. Schedule
6. Demolition
7. Weather
8. Additional Waste Quantity
9. Design Change
10. Additional Testing Monitoring
11. Scope Change
12. Additional Decontamination
All these risks could have been reasonably foreseen at the beginning of the project. For the purpose of modeling, it is important that all factors effecting the original estimate be foreseeable. For example, that the occurrence of additional vicinity properties could impact project schedule is certainly a foreseeable occurrence. From a practical standpoint, if the risks can not be foreseen, they can not be included in the prospective model.
The identification of risks would normally be achieved through
an iterative simplification of the events that may increase costs
on the project. On past projects, the initial set of risks consisted
of the final list plus several additional risks which were later
deemed insignificant. For the Durango project, however, the risks
were evident and extensively documented, thereby increasing the
accuracy of risk identification. The end result was a list of
the most influential risks and external factors which combined
to increase project costs. The identification of key elements
in this manner produces manageable models which retain the necessary
level of detail.
After completion of the risk identification phase, relationships were developed between the uncertainties and the cost elements of the project. These relationships define the effects of project risks on one another, as well as their effects on project costs. There are various methods of showing these interrelationships. One method, the influence diagram, is illustrated in Figure 1 below:
The nodes represent project uncertainties and project costs while the arrows represent the relationships between nodes. For example, it is noted that a change in regulatory requirements could cause a necessary revision to the design of the cell, which would impact costs associated with the cover. This is the basis for the modeling procedure. After determining what the risks are, their interrelationships and their impacts on project costs must be assessed. The computer then determines the collective impact of the risks on total project cost. Developing realistic relationships between risks and project costs while retaining an adequate level of detail is fundamental to the creation of an accurate model.
The final outcome of the modeling
process was two Influence Diagram models and a Monte Carlo model
labeled as the Evocative model. All probabilistic distributions
in the Evocative model were given triangular shapes for simplicity.
In most cases, the mode of each distribution is the cost designated
in the project manager's estimate. Each of the models has its
own advantages and drawbacks, but all seem to adequately represent
the impact of risk on the cost estimate, depending on the accuracy
required of the model. In addition to the following descriptions,
Appendix B contains schematic representations of each model type.
Evocative Model
The Evocative model, as developed using the Crystal Ball software, is the simplest of the model formats. It is essentially a range estimate of the project in which costs for individual detailed bid items are represented by probabilistic distributions which account for the risks internally. These bid items are summed to forecast a probabilistic distribution for the cost groupings of the estimate (e.g., cover costs, decontamination costs, etc.), which are then summed to derive an estimated distribution for total project cost. To create this model, the identified risks are listed next to the base estimate and are used to "evoke" distributions for the costs in the detailed estimate. Distributions are derived by considering a combination of the inherent uncertainties in the elements of the estimate and risks that may influence those elements. For example, without considering any risk in the project, the "Cover Costs" may be estimated to be as low as $900,000, as high as $5,000,000, and most likely $2,000,000. However, if the estimator considers the chance of a necessary design change, the estimate may be a given a low value of $1,100,000, a high value of $8,000,000, and a most likely value of $3,000,000. Here the consideration of a potential uncertainty increases the mode and range of values in the distribution, making the task more risky. The final cost distribution is a sum of these bid items.
The primary advantage of this model is its simplicity. There
are no extensive calculations to be performed, as the distributions
are derived from available data and an estimator's interpretation
of the risks involved. Depending on the experience of the estimator
and the desired accuracy, this may be an adequate way to incorporate
risk into the estimate. The Evocative model provides a rough
estimate of the distributions for cost groupings and for total
project cost. Instead of creating cost ranges for groupings in
the estimate, it also allows for approximation of more detailed
items, which is often a more simple estimation. For example,
it is much easier to develop a cost range for a single lab test
than for all costs associated with testing and monitoring. Also,
if the modeler wanted to vary certain bid item quantities instead
of costs, the spreadsheet model would carry this variation through
to the cost of the item. A disadvantage may be that the risk
is not divided into small enough elements. For example, if there
is one primary risk that influences a cost, it may be relatively
simple to consider this risk when developing a cost distribution
for that bid item. However, if there are several risks that influence
a cost, it may be more difficult to think of their collective
influence on that particular bid item. In this circumstance the
risks may need to be considered on a more individual basis, analyzing
the effect of each risk on all relevant bid items. Another disadvantage
is that the "evocative" estimation of the impacts of
risks on project costs is difficult and is a very subjective
quantification.
A formalized process for quantifying the effects of risk would
result in more accurate input parameters.
DPL Influence Diagram Models
The Influence Diagram models consist of multiple chance nodes representing risks and elements of project cost, with the influences among nodes indicated by connecting arrows. For example, an arrow from the "community involvement" node to the "vicinity properties" node indicates that additional vicinity properties may need to be included in the remedial action if active members of the community claim for previously unaccounted damages caused by past mining operations. This graphical display of the risks and their impacts facilitates visualization of the interrelationships between project risks and project costs. A description of the relationships represented by each individual link in the diagram can be found in Appendix C.
Risk quantification using DPL is accomplished by building a probability tree to analyze the effects of risk on project costs. The tree consists of chance nodes connected by branches representing their paths of influence. These nodes are defined by several state names (e.g. "few", "many") and the associated probabilities of each state. No values were given to the risk nodes, only probabilities of being in one state or another. All cost nodes, with the exception of the Tailings Pile Costs and Cover Costs nodes, were given three states: minimum, most likely, and maximum. Since the Tailings Pile and Cover Costs represented large portions of project cost with large variances, they were divided into five states so as to create a less discretized distribution of costs. Each of these states was assigned a unique probability and a corresponding value. The values associated with each state remained constant while the probabilities of achieving these values varied dependent on influencing factors of risk. The cost nodes lead directly into a value node labeled "total cost". An analysis was then performed to generate a distribution for total project cost.
For this case study, several variations of the DPL Influence Diagram model were analyzed for their utility and accuracy. One of the model formats involved an automatic link between the DPL Influence Diagram and the Evocative model in Crystal Ball. This would allow modelers to use Crystal Ball to estimate the ranges for the detailed bid items (e.g., lab tests) and calculate the probability distributions for the larger cost groupings (e.g., Testing/Monitoring Costs). The characteristics of the calculated distributions (e.g., mean, standard deviation, etc.) would then be transferred automatically to the cost nodes in the DPL model where the distributions would be discretized and the global effects of the risks on project costs would be determined, resulting in a distribution for total cost. The problem encountered is that automated sharing of data between DPL and Crystal Ball is not yet technically possible. DPL and Excel can share data and Excel can link to Crystal Ball, but an automated link between Crystal Ball and DPL is not yet available. However, there will soon be an enhancement to Crystal Ball called Automated Macro Interface (AMI) which will allow the Crystal Ball software to "talk" to other applications through the development of a macro. This option may be worth investigating when the technology becomes available.
Since automated sharing of data was deemed impossible, a manual link between DPL and Crystal Ball was investigated. Values would be calculated by Monte Carlo analysis and manually introduced into the DPL model. With this came the question of whether a more detailed DPL model alone might provide the benefits expected to be obtained from the integration with Crystal Ball. Two models with increased detail were examined. The first model, which provided the greatest detail, influenced the cost groupings by their component bid items, which were influenced by nodes for quantity and cost per unit. The risks would then be linked to the quantity and price per unit for the costs which they effected. A sample of this scheme with only a detailed node for Tailings Pile Cost is shown in Figure 2 below:
As can be noted from the diagram, this model is far too complicated. The Tailings Pile Cost node has only three components, each with an associated quantity and price per unit, and the diagram is already unnecessarily complex. If other cost nodes, some of which have up to fifteen components, were broken down this way the diagram would become too confusing to be useful.
The obvious next step would be to explore the possibility of creating a model which divided the cost nodes into their components only, without the presence of the nodes for quantity and price per unit. A sample from this type of model can be seen in Figure 3:
Although less complicated, it can still be seen that detailing the cost nodes into their components creates an overly complicated model, thus taking away from the benefit of graphical representation of risk and simplicity provided by the DPL program.
It was therefore determined that a model which left Total Cost as a function of its divisions only would be the best alternative. Values for these cost nodes could be estimated by either large scale approximation or through the implementation of Crystal Ball. Both of these alternatives were addressed and compared.
Generating values by approximation is somewhat self explanatory. Simply, each cost node was given a value as estimated in the original estimate for its most likely value, and the minimum and maximum costs were estimated. A DPL analysis was performed using this input data and is discussed in the Results section.
Values were also generated using the Crystal Ball software. Probabilistic distributions were given to individual bid items (e.g., lab tests), and distributions for the cost groupings (e.g., Testing/Monitoring) were forecasted, along with forecasts for total project cost. The minimum, maximum, and mode values that were calculated in the forecasts were manually input into the cost nodes in the DPL model and a risk analysis was performed.
One main advantage of the DPL models is the graphical representation of risks and their influences on elements of project cost. This allows simple illustration of the potential impacts of uncertain events on a project's outcome. Also, this software allows for "collapsible" diagrams. An entire influence diagram and its data can be collapsed into one node on another influence diagram, allowing modelers to break projects down into a series of subprojects.
Another advantage of the DPL models is that they provide the benefits of a probability tree while avoiding the difficulties of creating the conditional (IF-THEN) statements that would be required in spreadsheet models. Although there is an exponential increase in the number of branches with each additional contributing risk, the relationships are fairly easy to represent. However, a large number of relationships between nodes can still make for a cumbersome model.
The primary disadvantage of the DPL Influence Diagram models is the amount and detail of input data required to quantitatively describe model relationships. Values are held constant for each cost node, but conditional probabilities must be specified for each branch of the probability tree, requiring simultaneous assessment of multiple risks. Estimating conditional probabilities becomes a problem in the DPL format when there are several risks influencing one node, since the number of probabilities to be assessed also increases exponentially with each additional contributing risk. This amount of data is often difficult to think about, even for seasoned project managers. A second disadvantage associated with this model can be the computational requirements needed to perform an analysis. Depending on the size of the model and power of the hardware, excessive computer time may be needed to create a distribution for total cost. In these models all risks were limited to two states, thus minimizing the number of possible outcomes and decreasing the amount of computations to be performed.
In the DPL models, cost elements are grouped into categories
to simplify the arrangement of nodes and their relationships,
creating a less specific representation of uncertainty. However,
utilizing a link with the Crystal Ball software, numerous detailed
estimates can be combined to reasonably forecast distributions
for the encompassing cost nodes. Rather than having individual
line items of the estimate related to the risks, as in the more
detailed DPL models explored, the detailed estimate can be used
to generate more accurate distributions for the general groupings
of cost elements to be implemented in an influence diagram.
For each model, the final output was
a distribution of values for the total project cost. Below is
a summary of the output from the three different models:
All numbers are in millions of dollars except "CV" and "% Certainty", which are non-dimensional values. "Type" refers to the model classification type, and "Min", "Max", and "Mean" refer to the minimum, maximum, and mean values of the total cost distribution. "St Dev" is the standard deviation of the distribution, and "CV" is the coefficient of variability. The coefficient of variability is a measure of uncertainty obtained by dividing the standard deviation by the mean, with a high coefficient indicating a "risky" distribution. "% Certainty" indicates where the final project cost falls within the range of the distribution. For example, the DPL-Crystal Ball Link model predicts a 67.5% chance that the project cost should be less than or equal to the actual project cost of $26.57 million.
As can be noted, the Evocative model provides a high % certainty, low standard deviation, and low CV relative to the other two models. This indicates that the Evocative model accounts for a much smaller impact of the risks on project costs. The two DPL models, although developed from different input values, have similar means and standard deviations. The average mean value for the three models is $22.0 million. None of the three mean project costs vary from this average value by more than 7%, thus providing some confidence in the relative consistency of the models. The coefficients of variability reflect the greatest amount of uncertainty for the two DPL models, thus providing the most conservative estimates of total project cost. A complete record of the output for each of the three models is included in Appendix D.
Examination of the percentiles for the entire range of total direct cost reveals that the actual project cost was well above the predicted median values for all three model types. Final project cost fell above the ninetieth percentile for the Evocative model, and values near the seventieth and eightieth percentiles were seen in the DPL Influence Diagram models. Such high values indicate that either the riskiness of the project was understated during modeling or that nearly all the risks which were foreseeable by project personnel actually occurred during the course of the project.
Based on project records and interviews with the project manager,
it has been concluded that cost increases due to additional waste
quantities coupled with an expensive design change in the cell
cover were responsible for the majority of cost growth seen on
this project. While these risks were likely foreseeable, the
magnitude of their impact was significantly underestimated by
project engineers and managers. Because the models described
herein rely on the base estimate for modal values of line item
cost distributions, it is likely that the probability distributions
developed for the models were somewhat conservative. In short,
it is believed that a conservative base cost estimate led to conservative
assumptions during the modeling process for the Durango project.
It was also determined in the Lowman case study that conservative
estimates influenced the results of the models, and it is possible
that this tendency was also a factor in previous case studies,
although its effect was less significant. While perhaps somewhat
conservative, the final results of these models are still considered
to be adequate and appropriate predictors of total direct project
cost, although some may be more rough approximations than others.
Sensitivity analyses were performed on the DPL Influence Diagram models, and the results are illustrated in Appendix E. Sensitivity analysis is a method of determining which risks have the greatest potential to impact project costs. This type of analysis was not performed for the Evocative model because the relationships between project risks and costs are implicit in the range estimates. The absence of more formal relationships make a sensitivity analysis impractical for this model.
The DPL software provides a means of directly performing sensitivity
analyses for influence diagram models. Using the "event
sensitivity comparison" command, the model was run once for
each chance event while all other chance nodes were held at their
expected values. The results are displayed in the form of a tornado
diagram which graphically portrays the relative impact of each
chance node on total project cost (see Figure 4 below).
The impact of each chance node is measured in terms of the range of total project costs resulting from its variability. For example, when "additional waste quantity" is at its "low risk" state and all other chance nodes are at their expected values, total project cost is approximately $18.0 million. When at its "high risk" state, however, "additional waste quantity" increases total project cost to nearly $21.75 million. In short, the DPL sensitivity analysis focuses on the potential ranges of project costs resulting from the variability of individual chance events, giving a comprehensive analysis of the impacts of each identified risk.
As indicated in the tornado diagram, "additional waste
quantity" is the primary cost driver on the project. The
risks of "design change", "demolition",
"schedule",
and "regulatory/administrative change" compose the next
tier of influential risks, but they do not result in as much variability
as does "additional waste quantity". Also, these risks
have almost identical impacts on project cost. Although the remainder
of the identified risks influence the project cost in some manner,
their significance is not as great as that of these primary risks.
Intuitively, the "additional waste quantity" risk was
considered to be one of the most important at the beginning of
the modeling process and based upon the sensitivity analyses,
it was found to have, by far, the greatest influence.
As stated in previous sections of this report, each of the three proposed models has its unique advantages and disadvantages. Although all three models provide plausible distributions of total project cost, some are more realistic than others. The Evocative model produced a cost range which is relatively narrow. This model fails to directly relate the risky items in the estimate to each other and this independence leads to a less widely distributed range of costs. Also, the Evocative model relies most heavily on the ability of the estimator to quantify the impact of multiple risks on individual bid items. This may lead to an oversimplification of project risk. Accounting for all types of risk within a single estimated distribution makes this model relatively subjective and hence less formal. This model provides a relatively quick method for determining the approximate cost distribution associated with the project. It allows for the more simple development of distributions for detailed bid items and gives a rough estimate of project costs.
The DPL Influence Diagram models are also believed to be appropriate representations of total project cost. Generating cost values by approximation, this model combines a somewhat subjective quantification of uncertainty with a formal means of addressing the implications of qualitative or speculative project risks. The generation of values for cost nodes using the Crystal Ball program retains the formal representation of project risks and adds to it an easier and more precise means of estimation for elements of project cost. This type of model benefits from the positive attributes of both programs. Monte Carlo sampling of a detailed estimate gives a more accurate approximation of cost groupings that appear in the DPL model. DPL's simplified graphical portrayal of project uncertainties combined with the ability to quantify their impact on project cost allow modelers to examine the inherent and global risks surrounding the project.
As the DPL Influence Diagram models rely on the concept of conditional probabilities to quantify uncertainty, a frequency distribution for project cost is generated by following each branch of the underlying probability tree and recording its respective value. The DPL models produce the widest range of total project costs and have the highest coefficient of variability. Since DPL models examine every combination of costs, while Crystal Ball models rely on sampling techniques which don't necessarily explore every combination, the distributions of the estimated costs tend to be wider for DPL models. This provides a more conservative representation of uncertainty than found in the Crystal Ball models. This is found to be true when comparing the estimation of total project cost produced by the Evocative model with that of the DPL-Crystal Ball Link model. Both models have the same input parameters, yet the distribution created by the DPL model is more variable because every cost combination is examined.
The nature of all the models and their input data is somewhat
qualitative, as historical data was supplemented by expert judgment.
Project staff was consulted to provide insight into the logistics
of various activities. Although historical data provided the
majority of model input, reliance upon expert judgment was necessary
for explanation of project documents and description of project
events. This combination of project documentation and expert
confirmation provided sufficient detail for modeling purposes.
Since there are three different models,
each with unique advantages and disadvantages, formalized criteria
were needed for the selection of appropriate model formats. The
selection criteria were based on model attributes which are most
relevant to the research being performed. A list was developed
of ten model criteria, each having a possible score ranging from
one ("poor") to five ("excellent"). Scores
for the first three criteria were selected based on the reasonableness
of values obtained after running the models. The remaining seven
criteria were used to evaluate important characteristics of the
model formats as well as the various software packages. A maximum
score of fifty points was possible for each model. The selection
criteria are listed below:
1. Certainty Level of Projected Project Cost
2. Mean Value of Cost Distribution
3. Coefficient of Variability of Cost Distribution
4. Availability of Input Data
5. Facility of Model Conceptualization
6. Facility of Model Generation
7. Accuracy of Model Format
8. Formality of Modeling Process
9. Utility of Model Output
10. Utility of Sensitivity Analysis
The certainty level describes where within the distribution of total cost the final projected cost falls. Scores were assigned based on the relative consistency of this value among the three models. The mean values of the cost distributions were scored in a similar manner. The coefficient of variability, a measure of the riskiness of each cost distribution, was scored based on an assumed optimal range of 0.15 to 0.25. Scores for the availability of input data were assigned to reflect the ease of acquisition of the necessary data. Facility of model conceptualization measures the simplicity with which the relationships among project uncertainties can be described and quantified. Similarly, scores for the facility of model generation indicate the ease with which the format of each model type was developed. The accuracy of model format was scored based on the ability of each particular model to realistically portray the impacts of project cost uncertainties. Models exhibiting explicit relationships between risks and cost elements were given higher scores for the formality of the modeling process, while those relying on more implicit relationships received lower scores. The extent and practicality of the presentation of results by each type of modeling software were measured by the utility of model output. Finally, utility of sensitivity analysis was scored based on the method of measuring uncertainty as well as the graphical portrayal of results.
Each of the three models was given a score for the above ten
criteria. The total scores achieved by the models are listed
in the table below:
| Criteria | ||||||||||||
The above scores indicate that the DPL Influence Diagram with input values generated by Crystal Ball gave the most appropriate results for this case study. None of the models were considered to be poor representations of project cost, as all achieved scores within a fourteen point (28%) range. However, it was determined that the Evocative model serves best as a tool for rough approximation of total project cost. It does not incorporate enough detailed input to stand alone as a sole source for risk analysis. The DPL model based on approximated values for project cost divisions instills a greater amount of detail into the modeling process, but the estimation of such large portions of project cost is difficult to perform accurately. It was determined that the influence diagram model which derives its cost distributions from a detailed Crystal Ball analysis is most accurate. It allows estimators to quantify smaller portions of project cost, relying on computer simulations to develop distributions for the larger cost groupings. These distributions can then be implemented in the DPL model where project risks are considered.
The most simple way to develop a model of the DPL-Crystal Ball Link nature is to first create an influence diagram in the DPL Draw mode. No quantification needs to occur during this step. From the graphical representation of the risks, an Evocative model is quickly formed since the influence diagram serves as a map, connecting risks to the costs which they impact. Detailed portions of the estimate can be given distributions and the simulation of the Evocative model will forecast distributions for the larger cost groupings utilized in the DPL model. Performing a DPL analysis with the values calculated in Crystal Ball will result in a cost distribution which provides considerable reliability.
Based on these findings, it has been concluded that all model formats are viable means of quantifying project uncertainty to some degree and should be applied to other case studies in further research, along with the spreadsheet models developed in previous case studies.