United States
Environmental Protection
Agency
Hazardous Waste Engineering
Research Laboratory
Cincinnati OH 45268
Research and Development
EPA/600/S2-88/038 Aug. 1988
&EPA Project Summary
d-SSYS, A Computer
Model for the Evaluation of
Competing Alternatives
Albert J. Klee
This study was instigated to
develop a computer model that (a)
quantitatively evaluates competing
research and development projects,
and (b) assists in prioritizing such
projects when resources are not
sufficient to conduct all of them. An
evaluation model was developed,
based upon existing multiattribute
utility theory but with some
modification and innovation. The
model, with user input, helps
determine the relative weights of the
factors or criteria used to evaluate
the projects under consideration,
and, again with user input,
determines the utility function for
each of the attributes. A computer
program was written to implement
the model. A unique feature of this
model is that it incorporates
uncertainties of three types: (1)
those dealing with the factor weights,
(2) those dealing with the worth of
each project with respect to each
factor, and (3) those dealing with the
utilities of the attributes. The model
is adapted to run on personal
computers as well as on larger ones,
although the distribution version
available is designed for personal
computers. No special knowledge of
the basic theory involved is required
to exercise the computer model.
Although the study was designed
with the objective of dealing with
competing research and devel-
opment projects, the model is
sufficiently general so that it may be
applied to any problem of competing
alternatives. Thus, it has wide
application in the health, engi-
neering, environmental, and decision
sciences.
This Project Summary was devel-
oped by ERA'S Hazardous Waste
Engineering Research Laboratory,
Cincinnati, OH, to announce key
findings of the research project that Is
fully documented in a separate report
of the same title (see Project Report
ordering information at back).
Introduction
The Office of Research and
Development of the United States
Environmental Protection Agency, the
firms under contract to it, and the
recipients of the grants and cooperative
agreements that it administers, are
presently engaged in a broad program of
research studies in a campaign to attack
the pressing problems of hazardous
waste minimization, containment,
disposal, detoxification, and destruction.
Resources, however, are limited and it
not possible to fund every research study
proposed. Furthermore, it is not
uncommon for an individual research
study to identify more than one solution
to a particular environmental problem.
Either situation involves the selection of a
subset of alternatives from a larger set. In
the first instance, the winning subset of
alternatives consists of those studies that
will be funded; in the latter, the winning
subset consists of those solutions that
will be recommended or implemented as
circumstances warrant.
The problem of determinating such
winning subsets can be referred to
simply as the "Evaluation of Competing
Alternatives," or EGA for short. There
-------
are, however, two general types of EGA
problems: (1) Best Subset and (2)
Ranking. EGA best subset problems
seek to select some subset of the
complete set of alternatives that
maximizes the worth of the subset while
meeting one or more constraints placed
upon the selection, whereas EGA ranking
problems seek to determine an order of
the alternatives according to worth. In
ranking, worth can be measured on
ordinal, interval or ratio scales, while the
determination of the best subset requires
a ratio scale.
Examples
As an example of a best subset
problem, consider the set of three
alternatives shown in Table 1 and
assume that the goal is to fund as many
of these projects as possible with a
budget of $30,000. The solution, clearly,
is to fund Alternatives 1 and 2. The
problem could have been made more
complicated by imposing an additional
constraint, e.g., one involving time.
Problems of this sort are generally
solved using a technique known as
"mathematical programming." However,
with substantive problems, the technique
is by no means trivial.
As an example of a ranking problem,
suppose both cost and time are
important and we have to pick a single
project to fund. Given the data shown in
Table 1, the solution is not clear. If 3-
year completion time is too long for the
results of the project to be useful, then
perhaps only Alternative 2 should be
funded. To clarify the decision-making
for Criteria Set II, the two attributes, cost
and time, must somehow be combined.
This, however, involves an additional
concept, i.e., that of the utility of money
and the utility of time.
Table 1. A Comparison of Three Alter-
natives
Alternative
Cost
Time
1
2
3
$10,000
$20,000
$40,000
3 years
2 years
1 year
If we could combine the cost and time
attributes of Table 1 into amalgamated
units that express the collective worth of
each alternative (these worth units will be
called "Utility units"), we might arrive at
the situation shown in Table 2. This is
fine with regard to the ranking problem
since Alternative 3 is now clearly the
winner. Unfortunately, selecting a best
subset using this new scale is not
possible. We are not dealing with
absolute worth (which requires a ratio
scale) in Table 2 but with relative worth
defined on an interval scale. Interval
scales have certain properties (see
Section B.1 of the full report). Adding the
utilities of Alternatives 1 and 2 in Table 1,
for example, produces the number 15
but it is not the same number as the 15
of Alternative 3. (Consider three
employees with IQ's of 75, 75 and 150,
respectively. Assigning the first two to a
particular job may be a vastly different
proposition from assigning only the
third!) Thus, adding the utilities of
alternatives under a utility budget
constraint figure is logically flawed. In
practice, however, this may not be a
serious problem. Often, best subset
problems are constrained solely by
budget. One could simply select projects
according to their ranking until the
budgetary constraint was exceeded. This
might leave some funds left over but
often additional funds can be found to
fund the next project or the remaining
funds can be used profitably for some
other purpose. The practical solution to
EGA problems, therefore, is to consider
them all simply as ranking problems.
Table 2. A Comparison of Three Alterna-
tives, the Criteria Converted to
Utilities
Table 3. A Fallacious ECA Model
Alternative
Alternative
1
2
3
Utility Units
5
10
15
Common Fallacies
Consider the following EGA model:
(a) Define factors or criteria;
(b) Weight the factors;
(c) Determine the value of each factor
for each;
(d) Compute a score for each alternative
by multiplying the factor score for
each alternative by its corresponding
factor weight.
As an example, consider the two-
alternative, two-factor problem shown in
Table 3. The Alternative scores would be
calculated as follows:
Alternative 1: (0.2)(30) + (0.8)(20) = 22
Alternative 2: (0.2)(5) + (0.8)(25) = 21
Alternative 1, therefore, is superior to
Alternative 2.
Factor
1
2
1
30
20
2
5
25
Factor
Weight
0.2
0.8
Suppose, however, that Factor 2 w
measured in dollars. We now conv
Factor 2 into some other unit of curren
say pesos, by multiplying the Facto
row by 100. (The other factors
measured on different scales so tt
remain unchanged). The Alternat
scores would become:
Alternative 1: (0.2)(30) + (0.8)(200) = 1
Alternative 2: (0.2)(5) + (0.8)(250) = 2
and Alternative 2 is now superior
Alternative 1. This is illogical, howe>
since it shouldn't make any differer
whether the scoring is done using doll
or pesos. This is a common fallacy
EGA models.
Results
The objective of this study was
develop a cmoputerized model i
procedure for the ranking of alternativ
the requirements made on the mo
were as follows. The model must:
(a) Produce a ranking of alternatives
at least an interval scale;
(b) Be based upon sound decis
theory;
(c) Be easy to understand and to use;
(d) Generate easily understood output
(e) Be usable on a wide variety
computers, including perso
computers; and
(f) Be able to deal with the uncertain)
of its inputs.
The computerized model (called
SSYS" for 'Decision Support Syster
that resulted from this study meets al
these requirements. It is a true decis
support system, i.e., an interact
system that provides the user with ei
access to decision models and data
order to support semistructur
decision-making tasks. Such a mo
perforce must deal with subject
judgments and often are criticized on I
account. The truth of the matter is tha
any decision-making process, a crit
assessment of worth must be ma
Even if we ensure that these judgme
are made by those with a large store
knowledge and a refined sensitivity to
-------
;elevance, ultimately judgments must
orm part of the process. What is
'equired of any model, however, is that it
nave a logically consistent assessment
structure, hence the judgments made will
ae consistent. Furthermore, such models
lave the advantage of making explicit:
a) what factors were used to evaluate
he alternatives, (b) the evaluator's view
Df the relative importance of the factors,
and (c) the worth of each alternative with
respect to each factor. A major fault of
most conventional decision-making is
that these matters are seldom made
clear
The major testing of d-SSYS involved
an actual application. The application
concerned the evaluation of processes
that have been proposed to deal with the
problem of PCB-contammated sedi-
ments. Eleven processes wre evaluated
based upon six different classes of
technologies, one on low-temperature
oxidation, two on chlorine removal, one
on pyrolysis, three on removing and
concentrating, one on vitrification, and
three on microorganisms. Seven factors
or criteria were used to evaluate the 11
processes studied. The weights of these
factors were assigned to emphasize the
extent of decontamination, the estimated
cost of treatment, and the versatility of
the process The application of d-SSYS
in this instance proved to be highly
successful and was instrumental in the
determination of the final recom-
mendations of the PCB study.
-------
The EPA author Albert J. K/ee (also the EPA Project Officer, see below) is with
the Hazardous Waste Engineering Research Laboratory, Cincinnati, OH
45268.
The complete report, entitled "d-SSYS, A Computer Model for the Evaluation
of Competing Alternatives," (Order No. PB 88-234 1821 AS; Cost: $14.95,
subject to change) will be available only from:
National Technical Information Service
5285 Port Royal Road
Springfield, VA 22161
Telephone: 703-487-4650
The EPA Project Officer can be contacted at:
Hazardous Waste Engineering Research Laboratory
U.S. Environmental Protection Agency
Cincinnati, OH 45268
United States
Environmental Protection
Agency
Center for Environmental Research
Information
Cincinnati OH 45268
PULK RATE
POS'TAGfi & FEES PAID
'< •""•' !' EPA' , „ ,.
PERMIT No: 6-33
Official Business
Penalty for Private Use $300
EPA/600/S2-88/038
OOOG329 PS
U S SHVIR PROTECTION ASiNCY
REGION 5 tIBRART
230 S DIAtBOR* STREET
CHICftGO I*- 60604
------- |