August 1996 Volume 23 Number 4
Pro Version 9.0 for Windows confirms product as an outstanding choice
for addressing complex, multicriteria problems
By Abel A. Fernandez
Expert Choice Pro Version 9.0 for Windows is a multiattribute decision support
software tool based on the analytic hierarchy process (AHP) methodology
developed by Thomas L. Saaty. This product is the flagship of the decision
support products available through Expert Choice Inc. The Expert Choice
product has been available for more than 12 years. This latest release provides
several new features and is designed as a Windows application. Version 8.0
of Expert Choice, a DOS based product, was reviewed in the June 1993 issue
of OR/MS Today [Bahouth, 1993].
Expert Choice Pro helps a decision maker examine and resolve problems involving
multiple evaluation criteria. The software uses the AHP methodology to model
a decision problem and evaluate the relative desirability of alternatives.
Dr. Saaty (developer of AHP) is co-founder of Expert Choice Inc. and still
active within the firm, so there is full confidence in the technical soundness
of Expert Choice Pro with respect to the AHP methodology.
The active research on many aspects of this technique aside, AHP has become
increasingly popular among practitioners for its comprehensible and insightful
analysis of complex problems. The academic and practitioner literature cites
hundreds (if not thousands) of research and application papers involving
AHP. Reported applications include problems in public policy, marketing,
procurement, health care, corporate planning, transportation planning and
many other areas. A recent Interfaces article by Thomas Saaty [Saaty, 1994]
provides an excellent overview of the AHP and its applicability to complex
multicriteria problems. Due to its relatively widespread familiarity within
the OR/MS community, the theory, history and present state of the AHP will
not be discussed in this review.
Expert Choice Pro is a Windows application requiring a minimum of 8 MB of
RAM and 10 MB of disk space. As part of this review, the program was run
under both Windows and Windows 95 without encountering problems under either
environment. Version 9.0 migrates the previous DOS release to Windows and,
according to the product literature, incorporates 15 new capabilities and
features, including enhanced interface capabilities and the new Structuring
The Expert Choice Pro software is grouped into three principal software
modules: the Online Tutorial, the Structuring module, and the Evaluation
and Choice module. The Online Tutorial is a well-designed software version
of the hard copy tutorial manual provided with the product. The Structuring
module is a feature designed to assist users in creating an AHP model of
the decision problem. It is a self-help tool for problem understanding and
hierarchical modeling. Although models can be directly created within the
Evaluation and Choice module, the Structuring module provides a mechanism
for facilitating problem understanding.
The Evaluation and Choice module is the principal component of Expert Choice
Pro. This module is used for creating a model (alternatively a model can
be imported from a previous session with the Structuring module or imported
from a library of standard model formats), eliciting expert comparison assessments,
solving a model, performing sensitivity analysis, and generating reports.
Expert Choice Pro decision models follow the standard AHP format, a functional
hierarchy with the broad overall objective (or goal) at the highest level
(known as level zero). Lower levels correspond to the criteria and respective
subcriteria used to choose among alternatives.
At the lowest level of the hierarchy are the alternatives to be evaluated.
Figure 1 illustrates a small model for selecting a site location for an
ice cream store. The goal is to select a site from three possible alternatives:
a shopping center, a downtown location and a mall location. The four criteria
under the goal represent the principal considerations for making the selection.
Note that this particular hierarchy is not symmetrical since the Cost criterion
is further divided into the subcriteria of initial and monthly costs. The
three alternatives are at the lowest level of the hierarchy.
Very large AHP models can be created using Expert Choice Pro, since Version
9.0 allows up to nine children nodes for each parent node. Even larger models
can be accommodated by a technique of clustering and linking between nodes.
Very large models, however, impose significant effort in eliciting pairwise
comparison assessments. For these situations, Expert Choice Pro provides
a Ratings capability in which alternatives are not compared against each
other but are compared against standards or norms.
The Expert Choice Pro software assists the user in all phases of the problem
solving process, from model formulation to final report output. The following
sections describe the three principal modules of the software.
The Online Tutorial
Supplementing the hard copy documentation provided with the software (a
User's Guide, quick reference card and Tutorial Manual), the Online Tutorial
provides an interactive overview of the product's principal features. A
first-time user can quickly become familiar with the software through the
Online Tutorial (which takes less than one hour to complete). There are
three particularly good features of the Online Tutorial: 1. the user learns
by example; 2. procedures are demonstrated using Lotus Screen Cam; and 3.
the user can easily switch between the tutorial and the actual program.
Learning by example concerns the method of instruction adopted by the Online
Tutorial: product features are illustrated by focusing on a single, simple
application. The user follows the entire problem-solving process for this
Lessons consist of hypertext descriptions complemented by Lotus Screen Cam
animated demonstrations. This multimedia approach is well conceived and
implemented. Furthermore, at the completion of each lesson the Online Tutorial
encourages the user to practice the concepts on the actual program. This
encouragement is an effective reinforcement of the lessons learned throughout
each of the tutorial steps.
The Online Tutorial is accompanied by a tutorial manual. Although the manual
is comprehensive and useful, it contains a number of typographical and other
editorial errors which detract from the professional impact of the product.
The Structuring Module
Expert Choice Pro provides two methods of developing a decision model: direct
construction using the Evaluation and Choice module, and assisted construction
using the Structuring module. (The Expert Choice Pro documentation states
that one may also "use an existing model from the model library,"
however without detailed descriptions these models were not very useful
starting points for specific applications.)
The Structuring module provides an interface mechanism for deriving criteria
and subcriteria. It is a framework for collecting ideas and transforming
these into an AHP model, a facilitating mechanism for constructing models.
In a bottom-up approach, criteria are developed by listing pros and cons
of individual alternatives, grouping these into clusters and converting
each cluster into a generic objective (criteria). The criteria are then
organized into the hierarchy of an AHP model. Models can also be built using
a top-down approach, in which objectives (criteria) are defined and then
organized into an AHP hierarchy. The Structuring module provides user interfaces
for creating a hierarchy under either a top-down or bottom-up approach,
and hooks for linking the model with the Evaluation and Choice module.
The technique of identifying pros and cons of alternatives and transforming
these into criteria is very useful for conceptualizing an AHP model. However,
although practical for small problems, the Structuring module's interface
quickly became cumbersome for moderately sized problems. This was principally
due to the physical limitations associated with a 13-inch monitor.
The Evaluation and Choice Module
The Evaluation and Choice module is the principal component of Expert Choice
Pro. This module provides the facilities for model creation, pairwise comparisons,
solution synthesis, sensitivity analysis and report generation. Version
9.0 for Windows includes many interface enhancements over the previous release,
including toolbars, more aesthetically pleasing window format, folder and
tab style presentation and other user-oriented features. In general, the
design is effective and simple to learn and use. Special symbols and icons
are kept to a minimum, and where used they are logical and easy to interpret.
In lieu of using the Structuring module, models can be directly input using
the Evaluation and Choice module. The facilities for entering and modifying
model data are generally very good. Models can be easily created using straightforward
commands and icons. Restricting names to eight characters, however, became
a minor annoyance.
The next step after constructing the model is to perform comparison assessments
leading to the AHP solution synthesis. Expert Choice Pro provides excellent
facilities for performing these comparisons using either of two modes. The
pairwise comparison mode is recommended for problems with small numbers
of alternatives. In this mode, elements are pairwise compared against each
other with respect to a higher level element.
The required comparisons rapidly become impractical in problems with large
numbers of alternatives, therefore Expert Choice Pro also provides a ratings
comparison mode. In the ratings mode, alternatives are compared not against
specific criteria but against norms (standards). This approach reduces the
number of required comparisons, and makes it possible to apply the AHP technique
to problems with large numbers (potentially in the hundreds) of alternatives.
Expert Choice Pro provides excellent facilities for eliciting expert assessments.
In order to accommodate different types of models (having different classes
of criteria), pairwise comparisons can be made using combinations of different
comparison types and modes. The three different comparison types are importance
, preference, and likelihood. For any of these types, assessments may be
made using either a verbal mode (comparisons are based on English language
descriptors), a graphical mode (comparisons are based on visual expressions
of preference), or a numerical mode (comparisons are based on numerical
The numerical mode is particularly useful for eliciting expert opinion from
third parties since hard-copy questionnaires can be produced for distribution
(a standard feature). The numerical questionnaire for the first level of
a decision model for selecting a retail store site is shown in Figure 2.
A shortcoming of this feature is that questionnaires can only be output
on a node-by-node basis. This presents a severe time imposition for large
problems, since printing all questionnaires for an entire model (a typical
need) requires a slow serial node-by-node manual process. During the product
testing, it took more than 15 minutes to produce all questionnaires for
a mid-sized model. (Note: As of April 1996, a change was made in the
software that allows questionnaires and reports to be generated for any
plex or the entire model with one command.)
Inconsistent expert judgment can be a factor when using the pairwise comparison
method. The Evaluation and Choice module calculates and displays the Inconsistency
Ratio (IR) of the AHP technique. The IR provides a measure of the logical
rationality of the pairwise comparisons: a value less than 0.10 is generally
considered acceptable. The Evaluation and Choice module calculates the IR,
identifies the nine greatest inconsistent judgments (in ranked order) and
recommends more consistent values.
The latter two features are under user control. Although very useful for
detecting inconsistencies, these features should be used with caution, as
the objective is to create a model of a physical process and not to create
a perfectly consistent model. Moreover, during the product review, the system
once recommended a value of 27.6 to improve consistency, well outside the
1-9 range of the pairwise comparison scale. (Note: The pairwise
comparison mode can be extended from 1-99 in the numerical mode and in the
graphical mode. The system is not recommending changing the value, but just
reporting what value would provide the best fit.)
For some classes of criteria it may be more appropriate to directly enter
data values. For example, assuming cost is a criterion then the actual values
associated with each alternative are the best measure of each alternative's
respective priority. As a standard feature, Expert Choice Pro allows direct
data entry of values such as probabilities, sizes, dollars, lengths or any
The ratings comparison mode is useful for applications involving large numbers
of alternatives. Rather than comparing each alternative against specific
criteria, standards are established for each criteria and prioritized (based
on pairwise comparison between the standards). Alternatives are then compared
against the standards of each criterion, e.g., excellent, above average,
average and poor. The number of necessary pairwise comparison is thus a
function of the number of criteria and not of alternatives. The ratings
mode uses a spreadsheet format to facilitate data entry and review (data
can also be imported or exported to a spreadsheet program).
After the decision model is completely defined (expert judgments made for
all nodes and alternatives), the Evaluation and Choice module can be used
to synthesize a solution to the model. The final result is an overall weight
(or total score if the rating comparison mode is used) for each alternative.
The results are presented in bar graph form, as shown in Figure 3.
A controversial aspect of the AHP technique is the possibility of rank reversal
after modification to a model [Saaty, 1994; Canada and Sullivan, 1989].
Expert Choice Pro addresses this issue by providing two methods for synthesizing
final results: a distributive method and an ideal method. The distributive
method distributes the weights of the criteria among the alternatives at
the particular level. Rank reversal can occur when using this method. Note,
however, that some theorists consider rank reversals not only possible,
but sometimes desirable.
The ideal method does not normalize the weights of the criteria, but assigns
all the weight to the most preferred alternative. Rank reversal thus cannot
occur since the most preferred alternative receives the entire priority,
i.e., the ranks of the alternatives do not depend on each other. The software
and the documentation provide guidance in the use of these methods, but
care is needed to appropriately apply and interpret the results from these
The Evaluation and Choice module provides six standard reports on the principal
aspects of a decision problem. These reports produce hard-copy output of
the model hierarchy, expert assessments, synthesis results and user-entered
comments. Reports may be copied to the Clipboard and imported into Windows
applications as bit map pictures. Although the standard formats provide
comprehensive coverage, report contents or formats are not user configurable.
Sensitivity analysis examines the sensitivity of the results to changes
in the priorities of the criteria. This is a particularly important aspect
of an AHP problem analysis, since results are based on subjective expert
assessments. Version 9.0 builds upon the excellent interactive sensitivity
analysis tools of the previous release [Bahouth, 1993]. Sensitivity analysis
can be performed from any level in the hierarchy; the software displays
the sensitivity of alternatives to priority changes of the criteria immediately
below a user selected node. This flexibility is very useful for fine tuning
the sensitivity analysis.
The Evaluation and Choice module provides five different graphical modes
for performing sensitivity analysis: 1. performance; 2. dynamic; 3. gradient;
4. 2-dimensional plot; and 5. differences.
Each of these graphical modes provide a different viewpoint to a sensitivity
analysis. Under any of these five modes, the user can easily manipulate
criterion priorities and immediately see the impact of the change (as reflected
in the ranking of alternatives). Figure 4 illustrates the graphical interface
of the performance sensitivity analysis as applied to a site selection problem.
Here the sensitivity analysis is performed with respect to the goal node,
The terms VISIBLE, COMPET'N, CUST FIT and COST refer to the four nodes immediately
below the goal. The left y-axis represents the relative priority of each
criterion (as synthesized from the expert pairwise comparisons). The right
y-axis represents the overall priority of each alternative (with the OVERALL
axis showing the overall priority of each alternative). The vertical bars
represent the derived relative priorities of each criterion.
Dragging any of the vertical bars causes immediate change in the priority
of each alternative. According to the User's Manual, "The lines for
the alternatives in between the vertical criterion lines have no meaning"
[Expert Choice Pro, 1995, page 169]. The outputs for any combination of
modes can be tiled so that they may be viewed simultaneously.
A variety of insightful sensitivity analysis studies can be performed using
the standard features. It was difficult, however, to interpret the output
of the 2-dimensional mode. The other four modes, by contrast, presented
very useful information in a readily understood format.
As part of this software review, two groups of users were asked to apply
Expert Choice Pro to actual multicriteria decision problems. The first application
was a public policy problem for a city in southeastern Virginia. The problem
was to decide among a number of alternative sports complexes in order to
provide sufficient playing capacity over a 10-year planning horizon. A team
of graduate students in the Department of Engineering Management at Old
Dominion University was responsible for developing a model for the problem.
The second application was a personal decision problem: selecting among
different school choices for our son. In this problem, my spouse and I developed
the model for the problem. Prior to the start of the respective projects,
the group of students and my spouse were familiar with decision analysis
but had limited exposure to AHP, and no familiarity with Expert Choice Pro.
The combination of a well-designed human interface and the Online Tutorial
allowed the users to quickly start using the software. Questions about specific
details were easily answered through the Online HELP or the User's Manual.
One student commented that the Online Tutorial was also useful as a periodic
refresher (valuable for the occasional user). They both agreed that the
judgment elicitation facilities and model creation tools were outstanding.
Asked to provide an overall assessment of the applicability of the tool
and the ease of use of Expert Choice Pro, everyone agreed that it is an
excellent product for addressing complex problems. However, I noted that
although by the end of their work all participants were very comfortable
with the fundamentals of the technique, there were uncertainties regarding
some of the finer details of Expert Choice Pro. These first-time users were
unsure regarding the suitability and interpretation of features such as
the different assessment methods, the ideal and distributive synthesis modes,
and the 2-dimensional plot sensitivity analysis. These uncertainties are
understandable given that, in all cases, this was their first time use of
the AHP. However, it does highlight the need for careful use of this powerful
Expert Choice Professional Version 9.0 for Windows is an excellent product
for addressing complex, multicriteria problems. Its design will satisfy
both seasoned and relatively inexperienced analysts. Furthermore, Dr. Saaty's
personal involvement with Expert Choice Pro ensures its theoretical soundness
(with respect to the state of the art concerning AHP) and breadth of features.
Version 9.0 includes a revised User's Manual (with condensed quick reference
card) and a new Online Tutorial (with accompanying hard copy manual). These
provide generally very good descriptions of the software and its applicability
to multicriteria problem solving. The Online Tutorial is particularly noteworthy
for its effectiveness.
Any relatively minor shortcomings do not detract from the outstanding capabilities
and features of Expert Choice Professional Version 9.0 for Windows.
1. Bahouth, S.B. (1993), "Expert Choice: Version 8 of Decision-making
Software is Easier to Learn," OR/MS Today, Vol. 20, No. 3, pp.
2. Canada, J.R. and Sullivan W.G. (1989), "Economic and Multiattribute
Evaluation of Advanced Manufacturing Systems," Prentice-Hall, Englewood
3. Expert Choice Professional Version 9.0 for Windows, "User's Manual,"
4. Saaty, T.L. (1994), "How to Make a Decision: The Analytic Hierarchy
Process," Interfaces, Vol. 24, No. 6, pp. 19-43.
Abel A. Fernandez is an assistant professor in the Department of Engineering
Management at Old Dominion University. His areas of teaching and research
interests include project management, decision analysis and risk analysis.
For more information about this article, input the number
9 in the appropriate space on the Reader
Vendor & Product Information
Expert Choice Pro is produced and marketed by: Expert Choice Inc., 5001
Baum Boulevard, Suite 650, Pittsburgh, PA 15213; Phone: (412) 682-3844;
Fax: (412) 682-7008.
The present pricing of Expert Choice Pro Version 9.0 for Windows is:
Full version (Commercial use), $595
Full version (Academic use), $416.50
Trial version, $20
Student version, $70
The trial and student versions have limited features and come with limited
documentation. Special pricing is also available for volume orders, upgrades
from previous releases and for site licenses.
Expert Choice Inc. also provides a number of other decision support software
products, publications and seminars on decision analysis, and consulting
services. Their home page on the World Wide Web provides up-to-date information
on their products and services, location: http://www.expertchoice.com.
E-mail to the Editorial Department of OR/MS Today: firstname.lastname@example.org
OR/MS Today copyright © 1997, 1998 by the Institute for Operations Research and the Management Sciences. All rights reserved.
Lionheart Publishing, Inc.
2555 Cumberland Parkway, Suite 299, Atlanta, GA 30339 USA
Phone: 770-431-0867 | Fax: 770-432-6969
Web Site © Copyright 1997, 1998 by Lionheart Publishing, Inc. All rights reserved.
Web Design by Premier Web Designs, e-mail email@example.com