Definitive Scenario Standard Edition Release 2.0
A Decision Analysis Tool for the Masses
By Abel A. Fernandez
Definitive Scenario is the first product offering by Definitive Software, Inc., a recently formed software and consulting firm headquartered in Broomfield, Colo. The company develops decision analysis tools for business managers. Its aim is to provide non-technical specialists with capabilities restricted to operations research and decision science analysis.
Definitive Scenario, the Standard Edition and the more powerful Select Edition version allows business managers to examine problems in which uncertainty is a major element. The program uses influence diagrams within a simple and visually oriented framework that allows generalists to quickly become self-reliant in the creation and analysis of decision models. This focus toward the non-specialist user may, however, leave the professional analyst feeling constrained by the absence of detailed data and other features needed for more in-depth study.
Definitive Scenario is conceptually based on influence diagrams a method of presenting and evaluating decisions through simple graphical representations (Shachter, 1986). Influence diagrams allow decisions to be structured using three decision elements or nodes: decision nodes (wherein a choice must be made from mutually exclusive alternatives); chance nodes (representing various states of nature with their respective probability of occurrence); and calculation nodes (representing fixed values or mathematical calculations).
Directed arcs connect nodes so as to create graphs. The power of influence diagrams is in their capability to effectively and simply communicate the structure of a decision situation. Principal factors and their inter-relationships are presented in a format easily understood by decision-makers [Clemen, 1996]. Although influence diagrams are easily understood by business managers, the available software tools to create influence diagrams require the knowledge and skill of a technical expert. Hence, the market niche that Definitive Software seeks to fill: a software tool that allows the general business manager to create and analyze influence diagram-based decision models.
Although the company describes Definitive Scenario as an influence diagram modeling tool, the software does not provide the full capabilities typically associated with influence diagrams. Models under Definitive Scenario use only calculation nodes and connecting arcs (edges in the Definitive Scenario nomenclature) decision nodes and chance nodes are not used. This absence does not detract from the software's intended purpose, but does mean that someone expecting complete influence diagram capability will not find it. However, Definitive Software enhances the definition of calculation nodes and edges to provide additional modeling power. Under the software, calculation nodes may either represent variables (constants or random variables) or mathematical functions. Edges not only denote relationships between nodes (i.e., the input to a particular node) but may also define mathematical transforms performed on the input to a node. These enhanced characteristics provide powerful processing capability while retaining the communications effectiveness of influence diagrams.
Figure 1 is an example of a small Definitive Scenario model for a family's monthly budget. Note that the graphical representation of the model clearly identifies the variables and their relationships. Variables may be defined as either constants, random variables (taken from a set of seven standard probability distribution functions) or as complex functions of other variables (these may be other variables within the model or imported from external Microsoft Excel spreadsheet models). Monte Carlo simulation is used to evaluate the impact of random variables upon model outputs. Definitive Scenario thus blends elements from influence diagrams and spreadsheet simulation packages such as @Risk [Palisade, 1997] or Crystal Ball [Decisioneering, 1997]. The result is a non-intimidating tool allowing business-oriented managers to create and analyze models incorporating uncertainty.
Definitive Scenario Standard Edition is a Windows-based, stand-alone decision analysis program that provides the user with all the necessary tools to become familiar with the software, build and test models, run simulations, compile output data, and perform sensitivity analysis. The overarching design criterion is easy to use and easy to comprehend by a business manager, or the intended user. Avoidance of intimidating technical jargon, emphasis on visual representation of data, detailed explanations within the user interface and other similar hand-holding is evident throughout this product. This is very effective for the first-time user or otherwise technically challenged manager but may become pedantic for advanced or more sophisticated individuals.
The Definitive Scenario software may be grouped into three functional areas: aids for learning how to use the system, features for model creation and tools for model testing and analysis. The software includes the customary user's manual and on-line help, but these are complemented with several alternative learning assistance mechanisms. The most notable of these is Model Builder, a creative interface to a set of sample models included in the system. Model Builder guides the user in selecting and customizing a model template to fit a particular application. Models may be created using either Model Builder or from scratch through a straightforward drag-and-drop interface. The software also provides features for testing models and analyzing results. Model testing is done through a debugging feature that allows the user to compare node values in the as-built model against expected values a good reasonability check for model verification. A set of standard displays (which when printed become reports) and a sensitivity analysis feature are the principal tools for output analysis.
The design emphasis on user friendliness has been well implemented in Definitive Scenario's human interface. From creating a model to producing and analyzing resultant output, all interfaces have been simplified to facilitate use. This emphasis on user friendliness, however, comes at a cost: The software comes with relatively few user configurable options. The more advanced or sophisticated user may feel constrained by the relative lack of flexibility for addressing special requirements or performing more advanced analysis.
The Definitive Scenario software assists the non-technical user in all phases of the problem-solving process, from model formulation to output analysis. The following sections describe the principal functional areas of the software.
Tools for learning to use the system
Definitive Scenario includes a user's manual, a "Help" feature, an on-line tutorial and a Web site for real-time product announcements and E-mail technical support. The first-time user, regardless of technical sophistication, can quickly become proficient in the use of the software.
The User's Guide, included with the Standard Edition, provides a concise and well-written description of the software features and its application. The manual could perhaps be expanded to provide more detailed explanations on particularly important technical aspects (after all, non-technical managers are the target audience), but it is, in general, well written and error free. One notable exception is in the description of variance found on page 175 of the User's Guide:
Variance and Standard Deviation Š are independent of the forecast units, which implies that we can use the Variance or the Standard Deviation to compare the variability of two or more forecasts, even when the forecast scale differs (capitalized in the original).
This clear error highlights the potential for misuse of this software by relatively inexperienced users. Future versions should correct all errors, and should also include further explanation of technical areas and a list of references for additional reading. An electronic version of the User's Guide, enhanced by an extensive and useful set of hyperlinks, serves as the on-line help for the software. Although practical, it lacks help features taken for granted on other modern application programs (e.g., a search capability).
The Web site for Definitive Software (www.definitivesoftware.com) is a well-designed complement to the User's Guide. The site offers general information on the product line, update releases, additional sample models and a direct link to the Definitive Software technical support staff.
Definitive Scenario provides two methods of developing a decision model: direct construction using a modern desktop environment and assisted construction using the Model Builder module. The overall design of the human interface can be characterized as effective and well thought out. The display area is Windows based (i.e., incorporates the familiar pull-down menu area and toolbars) with the remaining sections divided between program icons, a workspace area (for display of special icons and functions) and a modeling area. All commands are executed through a combination of pull-down menus, icons, and drag-and-drop actions. The human interface exemplifies the effort expended to make this product extremely easy to use. Anyone with a general familiarity of Windows-based applications software can easily create and solve models within an hour of installing the software. As with any modeling software, understanding the subtleties and mastering the full power of the software requires much study and insight only through actual model building.
Definitive Scenario models are comprised of calculation nodes representing variables and directed arcs (edges) representing node inter-relationships. Nodes are placed into the modeling area by either direct command through pull-down menus or by icon drag-and-drop actions. Selecting a node or a group of nodes and dragging it to the target node creates an edge between the initial set of nodes and the target node. Models are, to some degree, self-documenting as variables and their inter-relationships are clearly presented (see Figure 1). This characteristic of influence diagrams not only facilitates an understanding of a completed model but also simplifies the model-building process. Model creation is, at least in simpler instances, intuitive since variables and their inter-relationships may be easily created and visualized. The simplicity of the software allows the model creation process to focus on the problem without getting bogged down on software mechanics.
Nodes may either be independent variables or functions of predecessor nodes (i.e., they input values from other nodes and perform a defined mathematical function to produce an output). Independent nodes may be defined as constants or random variables. The Standard Edition of Definitive Scenario allows random variables to be modeled using seven probability distribution functions: normal, truncated normal, lognormal, binary, uniform, triangular and exponential. A descriptive dialog box guides the user through the steps necessary to completely define a node and its associated probability distribution function.
One unusual and somewhat confusing aspect of Definitive Scenario is the manner by which the parameters of the probability distribution functions are defined. The designers have chosen to use values for the 10th, 50th and 90th percentiles to define the parameters of the six continuous distribution functions. Thus, for example, the exponential distribution is defined by user entered values for these three percentiles rather than the more usual single estimate for the average rate, b. The User's Manual does not explain the rationale for this unusual approach, but somewhat cryptically states, "The program may temporarily adjust these numbers further to make them fit an Exponential distribution" (Definitive Scenario User's Guide, page 72). It should also be noted that all random variables are assumed to be independent; it is impossible to specify a correlation between random variables. This limitation may have significant impact in more complex modeling situations.
Definitive Scenario's ease of use and straightforward modeling approach could lead one to overlook its potential application for solving extremely complex problems. The full potential power of Definitive Scenario is realized through its hierarchical modeling feature, its Excel nodes and its edge function capabilities. A drawback of influence diagrams is that large models may be difficult to comprehend simply because of the large number of nodes and edges in the model. Definitive Scenario provides a straightforward (here again, user friendly is the guiding design principle) method for organizing models into easily-created and understood hierarchies. Consider a problem in which system costs are to be examined under high and low cost scenarios. This problem lends itself to the creation of two submodels, one for each of the problem scenarios.
Figure 2 depicts a Definitive Scenario model for this problem. This first level provides an overview of the entire model: the submodels (the three folders in the upper section), the principal model outputs (the four calculation nodes), the special graphs to be output (the four output nodes) and a link to a text document describing the model. Double clicking on any of the folders displays the respective submodel.
For example, the low-cost submodel (Figure 3) can be accessed by simply double clicking the corresponding folder in the display shown as Figure 2. The hierarchical depth of the model is limited only by the available memory. For ease of use, the entire modeling structure can be viewed and controlled through a hierarchical tree structure diagram. This modeling flexibility significantly extends the reach of Definitive Scenario to more sophisticated users and complex applications.
An Excel node is a special kind of node that links data between a Definitive Scenario model and an Excel spreadsheet. During a simulation, these nodes can send and receive values from the respective spreadsheets. This capability extends the features and calculating power of Definitive Scenario, allowing the user to make use of Excel as a supplement to the software.
The other feature of particular utility to advanced users is edge processing. In the simplest case, edges merely define relationships between nodes: The output of a predecessor node becomes the input of the successor node. Definitive Scenario also provides edge transformations in which the output value of a predecessor node is transformed by a user-defined formula and then input to the successor node. Transformations may be defined using a set of 70 standard formulas very similar to the special functions available in Excel. The fundamental conceptual simplicity of influence diagrams is thus leveraged to allow modeling of complex problems.
An interesting standard feature for assisting the user in the application of the software is the Model Builder. Model Builder serves as an interface between the user and the suite of 30 sample programs provided with the Definitive Scenario. This interface provides step-by-step guidance in selecting an appropriate sample model template and customizing it to a particular application. This is a useful tool for gaining insights in the construction of models using Definitive Scenario, but may be of little practical use to the experienced modeler and perhaps even dangerous in the hands of a neophyte modeler. The experienced modeler may find it time consuming and frustrating to customize a canned model to fit a particular problem. This class of users may find it more effective to simply create a model from scratch. The neophyte modeler, on the other hand, may be tempted to "fit" the problem to one of the canned models provided in the software a potentially dangerous possibility.
Running a simulation and producing output
As with model building, Definitive Scenario makes it very simple to test and run completed models. Model debugging is done through a "Node Test" feature that displays the input and output from a selected node or set of nodes. The user performs a reasonability check on values to confirm that node outputs are as expected. This is a very simple yet useful verification of the model.
Once completed, the user may run a Monte Carlo simulation to examine the impact of uncertainty upon output nodes (nodes of interest). Simulations are invoked through either a pull-down menu or icon. Note that the only configurable simulation parameter is the number of iterations performed during the simulation; there are no other configurable options associated with the simulation. Furthermore, the documentation does not provide any detail on the simulation methodology used, (i.e., the method used to generate random numbers.)
Simulation results are output through what is termed an "Analysis Notebook." This set of displays contains two standard tables containing simulation statistics and a set of user-defined plots of simulation results. The first standard table, shown in Figure 4, shows the mean, standard deviation and range for each variable in the model. The second table (not shown) displays the mode and the 10th, 50th and 90th percentile value of each variable. These tables are not configurable (with respect to either their content or format) and are read only. Although the data may be copied and pasted to other application programs, the tabular output capabilities of Definitive Scenario are very limited.
One would expect, for example, access to the data sets generated during the simulation (a standard feature of @Risk and Crystal Ball). Definitive Scenario allows the user to define output plots for any node in the model. There are a total of 17 different output plots, classified into three categories: simple plots, time series plots and x-y plots. These plots include cumulative distribution functions (S curves), inverse distribution functions (inverse S curves), histograms, scatter plots, pie charts and others. This comprehensive set of displays provides a strong visual statement of results, but nevertheless does not make up for access to more complete simulation output data and results. The advanced user will feel constrained by the inability to perform more in-depth analysis and generate customized output.
Definitive Scenario also provides a sensitivity analysis capability, termed "What If" in its nomenclature. Invoked through either an icon or pull down menu, the "What-If" feature allows one to modify the base value of any model variable by a user-defined percentage. The software then performs simulation runs using both the original and modified values and presents both sets of results.
An important step in the analysis of any problem is documenting the solution. Definitive Scenario allows the user to produce hard copy output of the model, the simulation statistics (in the form of the two standard tables providing key statistics) and graphical plots. These are strictly hard copies of displays and simulation results (tables and plots). The user is not allowed to configure or modify any of these, but must resort to copy and paste to other application programs for customization. The design emphasis of the software is on "any color as long as it is black."
An overall product assessment of Definitive Scenario Standard Edition should keep in mind the product's intended purpose. The developers of Definitive Scenario have specifically targeted this product to the non-technical business manager (although this target market should perhaps also include students and technical specialists that are but occasional users). For these types of individuals, Definitive Scenario is a good product for addressing analytical problems where uncertainty is important. The software blends features of spreadsheet-based simulation packages (i.e., Crystal Ball by Decisioneering or @Risk by Palisade Corporation) and influence diagram based software products (i.e., Analytica by Decisioneering or DPL by ADA Decision Systems). The intended user will discover new modeling possibilities heretofore available only through difficult-to-use (again keeping in mind the type of user) software products. Without a product like Definitive Scenario, this type of user will have likely had to hand the work off to a technical specialist (or simply not included uncertainty in their analysis) or spent considerable effort in coming up to speed with the more difficult-to-understand alternative products in the marketplace.
Definitive Scenario is the right product for a segment of the modeling community, but it is an inappropriate product for the more advanced modeler. This class of user will likely feel frustrated by the absence of features likely important for advanced analysis and found in more sophisticated products. Some examples of desirable features for these advanced users include: a richer pallet of probability distribution functions, the capability of accessing a fuller set of simulation data, the capability of editing hard copy results, the capability of including decision nodes and a find feature in the on-line help. As previously described, there is potential for enhancing Definitive Scenario for more sophisticated usage. The existing hierarchical modeling capability, the edge functions and the capability of passing data to and from Excel are powerful modeling features that may provide the foundation for future product enhancement.
Definitive Scenario is a technically-sound product perfectly suited for its defined target market (business managers across a wide range of organizations). It may open up the modeling world to individuals with limited firsthand experience in modeling; individuals who, in the past, may have not done their own analysis. Models (and implicitly the tools used to create them) must ultimately be assessed relative to their intended purpose. In this regard, Definitive Scenario has achieved the stated objective of its creators. There is, however, the lingering question of whether relatively inexperienced modelers can effectively create and analyze models of this type.
1. @Risk (1997), Palisade Corporation, Newfield, NY, www.palisade.com.
2. Analytica (1997), Decisioneering, Denver, www.decisioneering.com.
3. Clemen, R. (1996), "Making Hard Decisions," Duxbury Press, Belmont, CA.
4. Crystal Ball (1997), Decisioneering, Denver, www.decisioneering.com.
5. DPL (1997), ADA Decision Systems, Menlo Park, Calif., www.dpl.adainc.com.
6. Shachter, R. (1986), "Evaluating Influence Diagrams," Operations Research, Vol. 34, pages 871-882.
Abel A. Fernandez, Ph.D., P.E. is Assistant Professor in the Department of Engineering Management at Old Dominion University. His areas of teaching and research interests include project management, decision analysis and risk analysis.
OR/MS Today copyright © 1998 by the Institute for Operations Research and the Management Sciences. All rights reserved.
Lionheart Publishing, Inc.
506 Roswell Street, Suite 220, Marietta, GA 30060, USA
Phone: 770-431-0867 | Fax: 770-432-6969
Web Site © Copyright 1998 by Lionheart Publishing, Inc. All rights reserved.