ENVIRONMENTAL RISK AND DECISION-MAKING:
RISK ASSESSMENT/RISK MANAGEMENT
CASE STUDY
STUDENT'S
MANUAL
LOCATION
DATE
-------
Environmental Policy Analysis Course Case Study
Introduction
Decisions about environmental policy are complex and incorporate technical information
about health risk and cost with less technical information about political, economic, and social
factors. While it is diff-cult to identify a "typical" environmental policy decision, the analyses of
these decisions have many common elements.
The purpose of this course is to acquaint you with the principles of environmental policy
analysis as currently carried out by the US EPA. The course introduces methods of conducting
policy analysis in lectures and discussions and includes this case study whieh applies these
methods to a hypothetical site with environmental "probtems.
Throughout the course we will be discussing issues and asking questions that are difficult
and do not have clear correct answers. We recognize that you might not encounter some of these
questions and issues in your daily activities and that you might wonder why you should worry
about them here. We hope that by helping you understand the issues and questions surrounding
policy analysis and addressing the same uncertainties as an analyst, you can increase your
appreciation for the use, interpretation, and limitations of these analyses.
Structure Of The Case
Following a chapter providing background information on the hypothetical case study.
the case begins with several chapters on problem identification. This part of the case is organized
around a risk assessment of the area near the city of River City. The risk assessment is a
systematic approach for evaluating and describing, in qualitative and quantitative terms, the
hazards posed by pollutants in the area. Chapters II - V discuss the steps to be taken in a human
health risk assessment including:
• Hazard Identification — a review of the types of injury or disease that a particular
chemical could cause
• Dose-Response Evaluation- an analysis of the potency of the various chemicals present in
the case to help determine how dangerous they are
• Exposure Assessment— an evaluation of the degree to which humans might come in
contact with these harmful emissions
• Risk Characterization— an attempt to summarize information about the risk in a manner
that will facilitate understanding of the problem and the identification of response
alternatives
-------
Each chapter includes a description of the principles underlying that stage of the health
risk assessment, specific information about the chemicals present in the case study, some
discussion questions, and some possible conclusions that can be drawn about the situation in the
city of River City. You will be asked to determine which of the conclusions presented (or an
alternative one) best applies to River City.
Chapter VI concludes the discussion of problem identification with a discussion of
ecological and welfare impacts that these pollutants have on the area around River City.
In Chapters VII through IX we discuss issues relating to problem solutions: options for
addressing the pollution; economic, social, and political factors that may affect your decision;
and scenarios for possible solutions.
Your Role
At the conclusion of the case, you will be asked to make decisions about
• What should be done to address environmental problems in River City?
• Who should pay?
• What is the appropriate level of government to be involved in the implementation and
enforcement of the decision?
• How should the public be involved?
On the last day, each small group will participate in an exercise in which each group will
make an appeal to the World Bank.
Structure Of The Course
We will be working both as an entire group and in smaller groups of about 10 people
each. In the small groups, you will review the case, discuss issues, and reach conclusions about
the situation with the two hypothetical facilities. We will frequently re-convene as a large group
for lectures on additional topics and to listen to reports from the small groups. Each small group
will have a facilitator (or two) to help move the discussion forward and answer any questions that
arise.
-------
I. Background on the Case Study
The case study is based on two hypothetical industrial facilities located near a large city.
The first facility is a metal smelter which produces metal ingot and fabricated products from
metal ore. The second facility is a chemical plant which produces several different chemical
products. The background discussion provides a general description of each facility, its location.
and pollution problems associated with the site. The case is simplified for this course, so some
other releases from the plants are not considered. Those discussed, however, are believed to pose
the greatest danger to human health and the environment.
White River Smelter
The Facility
The White River Enterprise owns and operates a smelter near the city of River City. The
smelter produces about 50,000 tons of metal each year, most of which is converted to finished
products for construction. The metal is used for castings in engines, and is formed into sheet,
pipes, and other construction products. Demand for the products from the smelter depends
heavily on the strength of the country's economy; little of the plant's output is exported. Because
the products are used as inputs to other finished goods, the demand for new construction and new
consumer goods controls demand for the smelter's output. The ore used at the smelter is mined
elsewhere in the country and is of poor quality. Therefore the yield of metal from the ore is low
compared to similar smelters in other countries.
About 2,500 workers are employed at the smelter. The jobs at the Enterprise pay average
wages, but the 50-year-old smelter is in weak financial condition. With sales in 199 of 1.5 billion
unit of currency, the smelter was just able to cover the cost of wages, raw materials, and other
expenses with net income of 10 million U.C. left. The status of privatization of the smelter is
uncertain at this time.
Metal smelters like the one near River City produce metal from metal ore by heating the
ore and removing the metal. During this process, impurities are dispersed into the air or are
collected, in solid or Liquid form, at the bottom of the smelter. The metal is cooled and then
transported to a mill at the Enterprise for forming into final products.
Location
The White River Smelter is located 10 kilometers west of the city of River City along the
White River (see Figure 1-1). The area around the plant contains many apartment buildings,
many of which house smelter workers. However, the plant lies well outside the more densely
settled city neighborhoods. Because the plant is located in a river valley most of the air emissions
from the plant move east along the river. However, the wind is often calm in the valley causing
the air emissions to stay in the valley near the plant resulting in some visibility problems in the
area.
1-1
-------
i j
LOCATION OF CASE STUDY
FACILITIES
Figure 1-1
White River Smelter
Northern Chemical
City Center
-------
Environmental Problems
The Ministry of the Environment, regional authorities, and residents of the area around
River city have expressed concern over the air emissions from the smelter. The processes used at
the facility occasionally result in visible releases of air pollutants which call attention to the
problem.
Emissions from the smelter occur during the heating of the ore as gases and particulates
escape from the-smelting unit into the plant and are vented to the outside at ground level (so-
called "fugitive" emissions). These fugitive emissions cause most of the problem near the plant.
Stacks also vent the smelting unit and its fuel source contributing to air emissions away from the
plant. Both the stack and fugitive emissions are of concern to the^Ministry and residents.
Smoke from the plant is most noticeable when the wind is calm and the smoke reduces
visibility near the plant. When the wind blows in the valley, however, the particles clear away
quickly and visibility improves. Other gases released from the plant behave the same way,
sometimes falling out over the surrounding neighborhoods and other times being carried away by
the wind.
The substances of most concern in the emissions are Metal Oxide and Tri-Oxy-Xray.
Metal Oxide or MO is formed during the reduction of the ore in the smelter. Because of the low
quality of the ore, relatively high levels of MO are released in the smelter. Higher quality ores
produce significantly less MO releases. MO adsorbs onto particles that are also released from the
smelter so MO concentrations are highest when the paniculate pollution is at its worst. MO is a
heavy compound, however, so most MO falls out of the air near the plant. However, soil
contamination does not appear to be a significant problem.
Tri-Oxy-Xray or TOX is a hydrocarbon found in virtually all metal ore. As with the MO.
the TOX is released with particulates from the smelter. TOX is a lighter compound than MO so
deposits of TOX are found throughout the River City area.
Northern Chemical
The Faclity
The Northern Chemical Company is a large chemical plant producing fertilizers.
adhesives, rubber, and several organic and inorganic products for use in other industries.
Annually the firm produces 1.2 billion U.C. worth of chemical products, one-third of which is
exported. Demand for the plant's products is quite strong, both in the country and from foreign
buyers. The export share of the plant's production has been growing steadily.
Northern Chemical employs 3,500 workers. Workers at Northern Chemical earn
1-3
-------
higher wages than workers at the smelter and the company is in better financial condition. In
1991 the firm earned a profit of 60 million U.C. The f~rm is a joint venture having been parr of
the first privatization wave with local residents holding a 20 percent share of the company.
The Northern Chemical plant houses several different production processes which
produce the diverse range of materials sold by the company. Many chemical raw materials are
produced at the plant and then used in other processes to make final products for sale.
Location
The Northern Chemical plant is located between the White River Smelter and River City
about 5 kilometers from the city center (Figure 1-1). Many more people live-near the chemical
plant than the smelter and Northern Chemical is much closer to the thickly settled city center.
Although Northern Chemical is also on the White River, the valley is less deep near the city so
the air emissions from the chemical plant tend to dissipate quickly.
Environmental Problems
Northern Chemical releases a number of pollutants to the air and water around River
City. Two compounds have been identified as posing the greatest danger to humans and the
environment based on limited monitoring and analysis of similar plants in other countries.
One of the most commonly used chemicals in the plant is ABC (amino-bicuspid). ABC is
used in two ways at the plant: (1) As an input to rubber and adhesive production processes and
(2) as a solvent for organic materials. As a result, large quantities of ABC are found throughout
the plant. Releases of ABC at the chemical plant are much different than the smelter emissions
because of how the chemical is used. About 75 percent of the ABC used to produce rubber and
adhesives is incorporated into the product and is not released to the environment. About 10
percent of the ABC in these processes is vented to the air along with other by-products of the
production process. Another 15 percent of ABC is mixed with wastewater and discharged to the
White River.
As a solvent, ABC is used in large quantities for cleaning parts and equipment. ABC is
quite volatile, so it will evaporate quickly if left uncovered. Because these cleaning operations
often occur in open tanks, a significant amount of ABC is released into the air in the plants and is
vented to the outside air. ABC is also stored in covered drums around the plant for small
cleaning and decreasing operations. Most of the ABC used for this purpose is released to the air
during its use or from evaporation while the storage drums are open for filling. Unlike the
particulates that mark the release of gases from the smelter, ABC releases cause no visibility
problems.
The ABC released to the White River enters the river at the plant's outfall pipe. The
wastewater undergoes limited treatment (settling and screening out solids) before discharge.
Downstream of the plant, an intake collects river water for use as drinking water for most of
1-4
-------
the area's residents. The presence of ABC in the drinking water has focused attention on the
Chemical plant's discharge practices.
A second compound of concern is Nitro-10 or N-10. N-10 is produced at Northern
Chemical for use as a catalyst in the adhesive production process. All of the N-10 produced is
used for adhesive production in the plant; none is sold to other companies. The equipment used
to produce the adhesive is relatively old, so a substantial fraction of the N-10 used in the process
is released as fugitive air emissions into the plant and surrounding area. Figure 1-2 summarizes
the releases of all of the compounds from both plants.
1-5
-------
RELEASES OF CONCERN FROM
CASE STUDY FACILITIES
Figure 1-2
V^
White River Smelter
Northern Chemical
White River
-------
1-7
-------
II. Hazard Identification
The first step in the risk assessment process is a review of available toxicological data on
the types of health injury or disease that the chemicals in question can produce, and a description
of the quality and strength of the evidence that supports the conclusions reached about these
health concerns. This review and description is called the hazard identification phase of risk
assessment and this chapter will provide you with an understanding of the type and quality of
information used to determine the scope of the hazard posed by the chemical of concern. The
chapter will review the animal study data available to evaluate ABC only. To simplify the case.
data on the hazard posed by other compounds will be summarized in the next chapter.
Principles for Hazard Identification
1. Epidemiological studies in exposed human populations are generally considered the best
source of information for hazard identification; however, these studies are rarely available
2. Studies with experimental animals usually provide the most useful information for hazard
identification. These studies can be controlled and can therefore demonstrate cause and
effect more easily. Biological data do support the proposition that, in general, responses
in experimental animals should be similar to those in humans. All known human
carcinogens are also carcinogenic in one or more species of experimental animals.
Animal studies provide us with all our information about ABC and we will then have to
extrapolate the results to estimate possible human impacts.
3. In general, a varied response in experimental animals ~ effects at more than one site or
more than one species, with increasing response at increasing exposure - provides more
convincing evidence of a possible human effect than does a response limited to a single
species or sex.
4. In general, results obtained by administering a substance by the same route of exposure
experienced by humans (i.e. ingestion or inhalation) are considered more predictive of
likely human effects than if a different route of exposure is used. If tumors form at
internal body sites, however, the route of exposure may not be important. The specific
sites of tumor formation may differ from those observed in experimental animals.
ll-l
-------
Background on ABC Toxicity - Carcinogenicity
The Dvorak Study
In 1986, an article entitled "Chronic Toxicity of ABC in Rats" (Dvorak, A., Journal of
Environmental Toxicology') presented data on the potential carcinogenic effects of lifetime
exposures of male and female rats to ABC. The study design and results are shown in Tables II-
1 and II-2.
The study was carefully conducted and there does not appear to be any reason to doubt
the accuracy of the results. In the study, rats were given increasing doses of ABC and their
responses were evaluated. The chemical was administered both by controlling the concentration
of ABC in the air breathed by the rats (inhalation) and by- application of the chemical through a
stomach tube (gavage).
As can be seen in Table II-2, not all animals receiving ABC developed tumors, but the
chemical did increase the incidence of tumors in certain animals. Be sure to note that Table 11-2
only shows the results in which a statistically significant increase in tumors was found. Rats
developed spleen cancer after both inhalation and gavage exposures. The data in Table II-2 can
be interpreted in the following manner: the lifetime risk of spleen cancer in male rats exposed by
gavage to the high dose of ABC for their lifetime is 0.33. Lung tumors were produced only by
inhalation and stomach tumors were produced only by gavage.
It is also interesting to note that the stomach tumors appeared at the point where ABC
contacted the stomach by stomach tube. This is the rodent forestomach, an anatomical feature
that is not present in humans. There was evidence of severe irritation in the area of the rodents'
stomachs that were exposed to high doses of ABC and Dr. Dvorak believes that the stomach
tumors arose because of the severe insult caused by the direct stomach exposure.
Epidemiological Data
Recently two manufacturers of ABC issued reports on their own internal evaluations into
the health effects of ABC. Their reports noted that investigations of their employees (350 in the
study) had shown no statistically significant increase in lung or stomach cancer.
11-2
-------
Design of the Dvorak Study
. •-.,'•• ----- ; Number of Animals
Specln ami. Route ;:''•*< .Gfloupa, MB* F«raaie: Amount of Duniionof
' • ol Expoatif*:- ;' * :'': v Fteinrvfeift ABC • » . •>:•• .. „ ABC Received Exposure
•••*•*'*•:?"••'••«» <.ir** -. -=-... Each I
>".:-* '* i,. ';:•*
.. .. , s . * .
Pat. inhalation
Rat. gavage1
1 Mtcgiams at ABC par hlugnm Ol
eanvwtBd B a unit o> wdgni to «
i ' Apprnoim laiMwnt «i •« •nn
» * ;• 1 Y •
Control
Low dose
Hign dosa
Contal
Lowoosa
High dose
r arwnart body w«gn
on M eomMrvd wiv*. t
k^^ i^^^^iv laluwno>wu >^
•» wiDVi wowvy DB
nlMM ay mwra Of 1 1
- ! .j
60
69-
60
60
60
50
' " '
60
• 60
60
60
60
60
n« urau m t» g*¥ag» ittidir.
ndiBont.
nomaen&B*.
Ong*8)'
0
30
60
0
50
100
104
104
104
104
10*
104
in t» inhBUaon •xwnnwnt l-«i !)«••<
II-3
-------
Taote H-2
Significant Findings from the Dvorak Study
TVw following are trie snty groups in which a statistically significant excess ct tumors
was fcund. Nearly 40 oossftte sites of tumor tomaticn were examined in each sen of
totn species.
Percentage or Antnato wttti Tumors
:'--*< ,". . / : : *•
' ' * — . * . ' * * _f % • » .
Pat. inhalation
Rat. inhalation
Rat, gavage
Rat. gavage
• * stnriciiljr vgnfenlaa
in turner madanca bttman
c3nnl and TTCBO animals *
Tumeri wara found « offiv
UCMS.
*"* ri ^ * •«„
;..-;-••«• ••- •
" " • .. • •- » ; ..
•• -• ;•" . .. . . J - • T I .
Male
Male
Female
Male
m of unori magm D unM
fv vwBid ind DonBvi mmh
wis >w pnmnea of ASC. a n
BOH in Don cewrei ana m*ur
" ~ ' '-, ','"'•'*'* ^*
lenceratej
Twawi .. Oontffii Uw Oose High Dose
Jdwd-.; ;;;..r
.* f- ' -•- :. '
*'!*.»* A •. J
Lung 3
Spleen 0
Stomach 0
Spleen o
M oonnl ••null. T)w nwani H II
a dM 10 chanei. Btcaiia* tfia only
Italy tat »o OXSMS (unwr incdsnci
tint annuls. BUI no odw tumon MO
5 25'
2 18'
0 30*
i compound.
MT9d in ittbDolly i^Tiiflca/v
1-4
-------
EPA's Risk Assessment Guidelines
To characterize the strength of the evidence about a particular substance. EPA has
developed guidelines for carcinogenic risk assessment that classify a chemical into one of five
groups:
Group A -- Human Carcinogen
Group B -- Probable Human Carcinogen
Group C -- Possible Human Carcinogen
Group D -- Not classified as to human carcinogenicity
Group E ~ Evidence of noncarcinogenicity to humans
Using data available from human and animal studies, EPA developed a way to categorize
substances as to their potential human carcinogenicity, as shown on Table 11 -3 The strength of
evidence includes "no data" and "no evidence." "No data" means that no data are available
indicating whether a substance is or is not carcinogenic. "No evidence" means, for humans, no
association was found between exposure and increased risk of cancer in well conducted, well-
designed epidemiological studies. Sufficient evidence is definitive enough co determine that the
adverse effect is caused by the agent in question, while limited evidence suggests that the agent
may be causing an effect, but this suggestion is not strong enough co be considered fact.
raft* it-3
Categorization of tr» W»lght ol Evidence of
Carcinoganlclty Baaad on Animal and Human Data
rr^r
Jafl^attt ••^
LJRttd
inadtqw*
No data
NetvU
A
B
B
B
B
A
B
C
C
C
A
B
D
D
D
11-5
-------
Questions for Discussion
1. How would you describe the animal evidence? Is it proof of animal carcinogenicity?
2. Should the information from the manufacturers of ABC affect conclusions about the
animal data? Do the data prove that ABC is or is not carcinogenic in humans?
3. Do you believe ABC causes cancer in humans? How would you describe the hazard ?
Your group should agree on one of the following:
a. ABC is a human carcinogen (Group A)
•
b. ABC is a probable human carcinogen (Croup B). There is sufficient animal
evidence as demonstrated by the increased incidence of tumors involving different
routes of exposure (inhalation and gavage). with increased activity at increasing
doses
c. ABC is a possible human carcinogen (Group C). The use of the stomach tube
caused some of the observed cancers and the animal data are only limited.
d. ABC is not classifiable as a human carcinogen (Group D). There is no evidence
that ABC is a possible human carcinogen.
e. Propose your own conclusion.
II-6
-------
Non-Carcinogenic Toxic Effects of ABC
Toxic effects other than cancer can result from exposure to a chemical such as ABC.
These may range from death to more subtle biochemical, physiological, or pathological changes.
Chemicals that cause toxic effects other than cancer or gene mutation are often called "systemic
toxicants" because they affect the function of various organ systems. As part of the risk
assessment, these toxic effects, as well as potential carcinogenicity, would be evaluated.
The four steps followed for assessing a carcinogen are followed for systemic toxicants as
well. The hazard identification component of the risk assessment of a toxic compound is
conducted to determine a no adverse effect level or NOAEL. This represents the dose below
which no response (an increase in the risk of the toxic effect) was observed™ animal studies,
based on an equivalent animal dose. A chemical can cause more than one toxic effect and show
different NOAELs for the different effects. In general, the lowest dose that represents a NOAEL
for any effect would be chosen as the NOAEL for a particular substance. The underlying
assumption is that if an effect is prevented at this dose then all toxic effects are prevented.
The Mozart Study
Only one chronic study of inhalation exposure to ABC was found in the literature.
Conducted by Dr. W. Mozart, the experiment evaluated the effects of ABC on rats. The study
was carefully conducted and there is no reason to doubt the accuracy of the reported data. The
major results of the study are summarized below:
"A total of 200 rats (100 males and 100 females) were exposed to air containing
either 0 )Ug/m3,100 ^g/m3,250 ^g/m3, 500 A
-------
Questions for Discussion
1. What, if anything, can you say about the systemic toxicity of ABC?
Choose one or create one of your own:
a. It is a human toxicant. Animal study evidence clearly shows that ABC exposure
will cause kidney and liver damage in humans.
b. It is highly likely that ABC is a human toxicant. There is sufficient evidence of
toxicity as shown in rat studies to conclude that it would likely have the same or
similar impacts on humans.
c. It is a potential human toxicant. The available data is too limited to show clearly
that ABC is a toxicant to rats.
d. Other
2. There is no reported evidence of increased liver and kidney problems among the residents
of River City or residents near the chemical plant. Does this affect your view of the
toxicity of ABC to humans?
3. Should policy makers take the time to understand the "strength of evidence" concerning
the toxicity of a chemical? Why?
11-8
-------
HI. Dose-Response Evaluation
In this chapter we present additional information on the potentially harmful effects of toxic
substances. Once the hazard identification is complete, the next issue for the risk assessor is to
understand the relationship between the dose of the chemical to which individuals are exposed and
the likely response to that exposure. This is called the dose response relationship. Because of the
relative complexity of dose-response evaluation, the following discussion is substituted for a
statement of key principles.
Carcinogens
Most hazard data from animal studies, like that presented in Chapter IL come from very high
doses ~ higher than we would predict from human exposure ^see Chapter IV). The question
remaining then is how to extrapolate down to make risk predictions in the range of human exposures.
Several general approaches to this problem have been proposed by various experts:
Approach 1
Based on general theories of how carcinogens act to produce cancer (largely derived from
experimental studies and epidemiological data), EPA assumes for potential carcinogens that there
is no threshold below which the risk of cancer is zero. All finite exposure levels will produce a finite
risk and the magnitude of the risk will decline as the magnitude of exposure declines. This does not
mean that all finite exposures will cause cancer, rather, it means that all finite exposures will increase
the probability (risk) that cancer will occur.
Various models are applied to project the risk from the range of observed responses from
high doses in animals to the range of expected human exposure. (A model is a mathematical formula
that describes the relationships between various factors.) While models cannot predict the impact
of lower doses precisely, they do predict a plausible estimate of human risk with sufficient accuracy
to be used as a guide in making risk decisions. The model used by EPA estimates a plausible upper
limit on human risk. Actual human risk is not likely to exceed the upper limit, and it may be less or
even zero.
Approach 2
An alternative view is based on the theory that the quantitative relationships between high-
exposure and low-exposure risks in rodents and between rodent and human risk are not known with
sufficient reliability to be used in risk assessment. This approach assumes that carcinogenicity, like
other toxic effects, will not be initiated within an individual until a minimum threshold of exposure
is exceeded. EPA is currently reviewing the behavior of some chemicals (e.g., dioxin) to see if this
approach is appropriate and if there is, in fact, a threshold for some carcinogens. This threshold for
carcinogens is similar to the NOAEL described for systemic toxicants in Chapter II, and the risk
posed by the chemicals is often reported as a margin-of-exposure (MOE) by which humans are
III-l
-------
protected. MOE is the NOAEL divided by the actual amount of human exposure.
Estimates of Cancer Risk
For this exercise, we will estimate low-exposure risks using mathematical models following
approach 1, currently used by EPA. Two models are needed to predict low exposure risks:
A high to low exposure extrapolation model is needed to predict low-exposure risks to
rodents from the measured high-exposure, high-risk data. EPA currently uses a "linearized
multistage model" for this purpose. Application of the model to the rodent exposure risk data
produces an estimate of the lifetime risk for each unit of exposure in the low-exposure region. This
is called the unit cancer risk. The linearized model is used to ensure that the unit cancer risk is an
upper-bound estimate of risk.
An interspecies extrapolation model is used to extrapolate from rodent unit risks to human
unit risks. Several methods have been developed to do this by applying scaling factors between
animals and humans. Comparisons may be made based on the milligrams of carcinogen per body
surface area or per body weight.
EPA has chosen models that it believes yield the most supportable upper-bound estimate of
risk that tend to "err on the side of safety". Alternative models are available for both these forms of
extrapolation, and several are equally plausible to those chosen by EPA. In most cases, but not
always, use of alternative models will yield lower estimates of risk than those predicted by the two
models described above. Differences can sometimes be very large, but are generally relatively small
when the models are limited to those that are linear at low exposures.
Application of the EPA models for high-to-low-dose and interspecies extrapolation to the
rat cancer results for chemical ABC shown in Table II-2 yields the results shown in Table m-1. The
result of such extrapolations is the unit cancer risk, a measure of the potency of the chemical in
question. As can be seen, different unit cancer risks are determined for different cancer endpoints.
differing routes of exposure, and animals of different sexes. In this exercise, one unit of exposure
is equal to one milligram (mg) of the carcinogen per kilogram (kg) body weight of the animal (or
human) per day throughout the lifetime of the animal (or human). For example, Table III-l shows
that based on male rat data on spleen cancer from gavage exposure, the risk of developing cancer
from 1 mg/kg/day of lifetime exposure to ABC is 2.28 x 10 2 (a probability of about 2 in 100).
This relationship between dose and potential carcinogenic response is presented graphically
in Figure m-1. The horizontal axis is the dose of ABC measured in mg of ABC per kg of body weight
per day of exposure (mg/kg/day). The vertical axis is the response and is expressed as a probability.
The unit cancer risk is the slope of the function that relates dose and response.
III-2
-------
Table !l!-i
Upper-bound Estimates of Lifetime Unit Cancer Risks
Predicted by Applying EPA's Preferrec Models to ABC Tumor
Data
Unit
Tumor Sta
(ponney)'
mgftgMay
Rat. mala
Rat, male
Rat. female
Rat, male
Inhalation
Inhalation
Gavage
Gavage
Lung
Spleen
Stomach
Spleen
0.0188
0.0125
0.0054
0.022B
' Ri«h tar an •varaao dariy aneaura of on» un*. Unto ara Vw sam* a» \
•arhar tar aMenfemg >M arwmmi •9Oiura (Tab* 11-1). tueti otai ona um • on*
jnribflfun par Mtoofwn booy WOIQM par day. Riih IB oatainad by lYiuteptymQ 9ia unit
n«k by ft* actual numftar of una of human axpotu* For a gican axpoaura. ttw
htghar no um n»< 9m Mofiar tfia risk.
Flipirc IIM
DOSE-RESPONSE RELATIONSHIP FOR ABC
0.0226
(unrt canotf Rik)
(moVkg/dey)
III-3
-------
To obtain the risk from other levels of exposure, the unit cancer risk is multiplied by the
number of units of exposure. (The results of this multiplication for exposures to ABC are described
later in Chapter V.) The underlying assumption is that at low doses the relationship between dose
and response is linear. Thus, if a person were exposed to 0.003 units of ABC throughout his lifetime
(0.003 mg/kg/day), the risk would be 0.003 units times .0228/unit, or 0.000068 (6.8 x 10 5). a
probability of 7 cases arising in 100,000 individuals exposed at this level. The greater the slope of
the line, the more potent the carcinogen.
Questions for Discussion
1. What is the appropriate unit cancer risk to use for ABC? Why?
Unit Cancer Risks for Other Case Study*Polfti'iants
Because of time constraints, we have discussed the hazard identification and dose response
relationship only for ABC. One other carcinogen that is described in the case is Metal Oxide or MO
from smelter air emissions. The unit cancer risk for this pollutant is described next. Cancer risks
from both pollutants are summarized in Table III-2.
Studies of metal ore and smelter workers have shown that Metal Oxide increased risk of
cancer from the lung, trachea, and bronchus; cancer of the kidney; cancer of the prostate; and cancer
at all sites combined. In animals, MO has been found to be carcinogenic in both inhalation studies
and skin-painting biopsy. Due to this strong evidence and the epidemiological studies, MO has been
classified as a known human carcinogen (Group A). The unit cancer risk is 0.29 (mg/kg/day)'1.
Questions for Discussion
1. Which of these carcinogens is more dangerous?
2. What other factors would you want to consider before answering that question?
III-4
-------
\\\-2
Summary of Carcinogens in Case Study
Substance-
Category .
(Weight of Evidence)
Route of
Expoeura
Una Canar Rbk
(potency}1
mg&g/day
ABC
MO
B
A
inhalation.
ingesfion
inhalation4
228x10'!
2.9 x 10'1
' Risk tor an average daJy lilMim •xpowrt of 3"» unit Units are 9-a same as *e» tuad »ad «r tar
deacfiOfiQ ft* animal «xpowr» (T*b)» U-1). such *« ana urat • one mili^nffl per wgiam bod/ wwcpt
p«r day. Risk i abtarart by muMolying OM unit rwk by tf)« acnal nunbv of jniis ot .'uman axcciur»
For a given aocpowiiv, *»• high" IM unit n»k. ffia higher to n«k.
III-5
-------
Non-Carcinogenic Effects
As discussed earlier, "systemic" effects are toxic effects other than carcinogenicity or
mutagenicity, such as liver or kidney damage, induced by a chemical. The assumption for
carcinogens is that there is no level of exposure that does not pose a small, finite probability of
generating a carcinogenic response. Based on EPA's understanding of the mechanisms of action.
systemic toxicity is treated as if there is an identifiable exposure threshold (both for the individual
and for the population), below which effects are not observable. The individual threshold hypothesis
assumes that a range of exposures from zero to some finite value can be tolerated by an individual
with essentially no chance of experiencing the toxic effect.
Reference Concentration
To evaluate systemic effects, the EPA has developed the concept of a reference level. The
level is an estimate of a daily exposure to the human population (including sensitive subpopulations)
that is unlikely to result in risk of deleterious systemic effects during a lifetime. This level is
sometimes referred to as an acceptable daily intake or ADI. For ingestion, EPA defines the level as
a reference dose (RfD) and for inhalation, it defines the level as a reference concentration (RfC).
The RfC is derived from the NOAEL identified during the hazard identification stage of the
risk assessment (see discussion in Chapter II). Once defined, the NOAEL is then reduced by
application of uncertainty factors (UFs) that reflect various types of data and a modifying factor
(MF) that is based on professional judgement of the entire chemical database:
RfC = NOAEL / (UF x MF)
The uncertainty factors account for variation in sensitivity among the members of the human
population; uncertainty in extrapolating animal data to the case of humans; uncertainty in
extrapolating from data obtained in a study that is of less-than-lifetime exposure; and uncertainty in
using data where a NOAEL was not identified. Each level of uncertainty usually adds a factor often.
Thus, if the RfC for ABC were to be based on the animal study conducted by Dr. Mozart, we would
have two factors of ten or an uncertainty factor of 10 x 10 = 100. The first factor of ten accounts for
sensitivity among the members of the human population, and the second factor of ten is based on
the extrapolation from animal data to the case of humans. The Mozart study was a chronic study and
a NOAEL was identified, so no other factors apply.
The modifying factor is greater than zero and less than or equal to ten, and reflects qualitative
professional judgements about scientific uncertainties not covered under the standard uncertainty
factors. These include such considerations as the completeness of the database and the number of
species and animals tested. The usual modifying factor is one.
The concept of an RfC is presented graphically in Figure III-2.
III-6
-------
Figure 111-2
REFERENCE CONCENTRATION
Response
RfC
NOAEL
Concentration
(no/in3)
-------
The RfC is useful as a reference point for gauging the potential effects of various
concentrations. Concentrations less than the RfC are not likely to be associated with any systemic
health risks and are therefore less likely to be of regulatory concern. However, as the frequency and
magnitude of exposure exceeding the RfC increase, the probability increases that adverse effects may
be observed in a human population. Nonetheless, a clear conclusion cannot be categorically drawn
that all concentrations above the RfC are unacceptable.
An alternative measure that is sometimes useful is the individual exposure ratio (IER) which
defines the relationship between the human exposure and the RfC.
ER = Concentration to Which Humans are Exposed
RfC
The operating assumption is that in situations that ER < 1, there is little or no worry about toxic
effects from that source. A summary of the reference concentration information for ABC is presented
below:
Critical effect: Liver and kidney pathology
NOAEL: 250
UF: 100
MF: 1
RfC: 2.5
Questions for Discussion
1. What is the relationship between the NOAEL and the RfC? Do you think the reference level
approach adequately accounts for the uncertainties associated with NOAELs?
2. Should systemic health effects or carcinogenic effects from exposure to ABC be of greater
concern? Why? Do we have enough information at this time to decide?
III-8
-------
IV. Evaluating Human Exposure
Exposure assessment is the third step of a risk assessment. It is the process of measuring or
estimating the magnitude, frequency, and duration of human exposure to a compound in the
environment. The evaluation determines whether humans could be subjected to the potentially
harmful influence of these releases.
The output of the exposure assessment is a numerical estimate of exposure or dose that can
be used in the risk characterization step to quantify risk to human health. The doses, its duration and
timing, and the nature and size of the population receiving it are the critical measures of exposure
for risk characterization. In addition, a thorough exposure assessment also includes a description of
the uncertainties in all estimates.
Be aware that there is no risk if there is no exposure; this is true no matter how toxic the
Contaminant being assessed
In some cases, it is possible to measure human exposure directly, either by measuring levels
of the suspect chemical in the environment or by using personal monitors. In most cases, however,
human exposures must be estimated using measured concentrations in the environment, models of
chemical fate and transport in the environment, and models of human activity patterns. The
following information is required for this type of exposure assessment:
• Rate and quantity of any releases of the compound into the environment, including location
and timing
• Fate of the compound in the environment after release, including its movement, persistence.
and degradation (degradation products may be more or less toxic than the original agent)
• Information on human intake of contaminated media (e.g., water ingestion rate, dermal-soil
contact rate), including activities that facilitate or interfere with contact. This may also
include information on the size and distribution of the human population exposed to the
chemical.
The available information for exposure assessment varies from case-to-case. Some risk
assessments require extensive data collection while for others detailed information on releases into
the environment already exists. For many compounds there is limited knowledge about fate and
transport in the environmentafter release. Measurements of transport and degradation in the complex
natural environment are often difficult, time consuming and expensive, so it is common to rely on
mathematical models of fate and transport to predict environment exposures.
The field of exposure assessment is still at an early stage of development. Except in rare
circumstances where the behavior of a compound in the environment is unusually simple, significant
uncertainties arise in exposure assessments; these can be as large or larger than uncertainties in the
assessment of carcinogenicity.
-------
Principles for Exposure Assessment
1. The purpose of exposure evaluation is to identify the magnitude of human exposure to the
case study pollutants, the duration of that exposure, and the routes of exposure. It may also
be useful to identify the number of exposed people and certain characteristics of that
population (e.g.. age, sex).
2. Exposure may be based on measurement of the amount of each pollutant in various media
and information on the amount of human intake of these media per unit of time (usually per
day).
3. Some individuals may be exposed by contact with several media. It is important to consider
total intake from all media in such situations. Note that in this case we have limited the scope
of the study to air and surface water ic'.eases only.
4. Sometimes concentrations of pollutants can be estimated using mathematical models.
Although some of these models are known to predict concentrations accurately in many
cases, they are not thought to be reliable in all cases.
5. Standard average and high end values for human intake of various media (e.g., air breathed.
water consumed) are available, are commonly accepted, and are generally used unless site-
specific or compound-specific data are available.
Description of the Area
The two facilities are located west of the city of River city along the White River valley. The
White River Smelter is located 10 kilometers from the city and is next to the river in a deep section
of the river valley. While many apartment buildings are located very close to the facility, the
population living near the plant is relatively small. Very few people live in the mountains around the
river.
Northern Chemical is only 5 kilometers from the city and is located in the center of a heavily
populated residential area. Also the chemical plant is in a flat section of the river valley so that air
emissions from the plant are dispersed more widely around the area. Again, most of the area's
population lives within 2 or 3 kilometers of the river. Northern Chemical s wastewater is discharged
to the White River about 4 kilometers upstream of the intake for the area's drinking water system.
This system provides the residents of the entire study area with drinking water.
The area around River City has been divided into 3 grids for study purposes. Each grid is 5
kilometers wide and 5 kilometers long. The site map is shown in Figure IV-1 with the study grids
shown. The smelter is near the center of grid 1, Northern Chemical is near the center of grid 2, and
the most densely settled parts of the city are included in grid 3.
-------
Concentration Data
No air sampling has been conducted in the valley to measure concentrations of any of the
constituents. In place of actual sampling data, modeling has been used to predict concentrations of
the emissions in each of the three grids of the study area. Concentrations of ABC in water have been
measured at three points: at the outfall pipe into the White River, at the intake pipe into the drinking
water system, and in the drinking water.
The air modeling required measurement of the total emissions of the pollutants from the two
facilities. At the smelter, the generation of MO and TOX were based on the total consumption of ore
and the presence of MO and TOXJn the .ore used at the plant. The releases of ABC have been
measured using a mass balance approach. The total quantity~of ABC purchased annually is known
at the plant. ABC that becomes part of other products, that is-^isposed-of, or that is sill in inventory
at the end of the year is not released to the environment around the plant. The remaining ABC was
assumed to- be released to the air or water. The wastewater loadings were computed based on the
level of .adhesive production and the rest was allocated to the air. Finally, N-10 emissions were
computed based on the known quantity of N-10 produced at the plant and the quantity incorporated
into products. The remainder was released to the air.
In a real situation, the risk assessor would look at both mean levels of concentration data and
high end estimates (such as the concentration level that represents the 95% upper confidence limit
on the mean concentration). These differing levels could be used to assess the central tendency or
high end exposures discussed later. However, for purposes of this case we will be using one set of
data derived from the air modeling and the monitoring of ABC in the White River.
Given the mass loading of pollutants and data on the weather and topography of the area.
average annual air concentrations of the four pollutants were estimated using air dispersion models.
The results of the modeling are shown in Table IV-1. Water concentrations measured in the treated
drinking water are also shown in the table (note that they are constant for the entire study area).
-------
LOCATION OF GRIDS FOR AIR
EMISSION MODELING
Figure IV-1
White River Smelter
Northern Chemical
-------
Table IV- 1
Air" and Drinking Water" Concentrations of Pollutants in the Study Area
•''•-••9tii^^'^'^.-w^-. w«- ••
-------
Types of Exposure Estimates
For any specific site or chemical there is a range of exposures actually experienced by
individuals coming into contact with contaminated media. Exposure for some people will be higher
because they are in contact with highly contaminated media for long periods of time. Others will
have a much lower total exposure because they come into contact with less contaminated media for
shorter periods (e.g., people using a recreational site far removed from the source). Risk assessments
should present information on a range of exposures derived from differing exposure scenarios and
using various human uptake assumptions, in the U.S., two primary types of exposure cases are
evaluated to address the range of possible exposures: Central Tendency and High End.
Central Tendency is an estimate of the average exposure or dose experienced by the affected
population. It is based on mean or median values for chenaicaLr oncentralions in contaminated media
and for exposure frequency and duration parameters. The Central Tendency exposure is used to
estimate average risks experienced by the affected population and can also be used to estimate the
potential number of cancer cases to which the affected population is at risk over the period of
exposure.
High End is an estimate of the highest exposure or dose actually experienced by some
individuals. It should estimate exposures for individuals above about the 90th percentile exposure
category but not be higher than the individual who has the highest exposure. The High End exposure
is used to estimate the highest risks experienced by individuals in the affected population. It is
important to understand that the goal of the High End case is a realistic estimate of maximal or near-
maximal exposures. It is not meant to be an unrealistic case which would exceed the highest real
exposure derived simply by combining maximal values for all exposure parameters.
Exposure Assumptions
Ideally, it is best to collect data on exposure parameters specif-c to each risk assessment.
Although this is usually done for the contaminant concentration, it is often not feasible for the other
terms (i.e., those that relate to human activity patterns). For this reason EPA has developed standard
or default exposure assumptions for risk assessment. Table IV-2 lists some of EPA's default
exposure assumptions.
-------
Table IV-
Sample EPA Residential
Air inhalation (cubic meters/day)
Drinking Water Ingestlon (liters/day)
Child
Adult
Exposure Frequency (days/year)
Exposure Duration (years)
Body Weight (kilograms)
Child
Adult
*EPA Convention
2
Exposure Factors
Central
Tendency
20-
1
1.4
350
9
15
70
Hlgn
End
30
1
2'
350
30
15
70
-------
Calculations of Human Exposure
To estimate human exposure or the dose of the contaminants, certain data and assumptions
were applied. For this case, we have assumed that the exposure factors noted below relate to the
citizens of River city and the surrounding area. These data and assumptions relate to the extent and
& frequency of human contact with the media and the degree of absorption of the chemicals (for
example, when a contaminant is inhaled a certain fraction will be absorbed by the lungs).1 Different
assumptions are made for different media.
Air
The assumptions used in this analysis are
• An adult inhales 20 cubic meters of air per day
• The body weight of an adult is 70 kilograms
• The inhalation absorption factor for all of these compounds is 1.0
• The adult lives in the same home for 75 percent of his lifetime [used only for carcinogens]
In a more complete assessment as noted, a range of exposure and intake assumptions should
be evaluated.
Using the average ABC concentration for Grid 1, the calculations are
0.61 ug (ayerageABC air concentration in study area) x 20 m} (inhalation)
m3 day
x 1 (body weight) x 1 me (unit correction)
70kg l,OOOA
-------
Exposure levels are calculated differently for the assessment of systemic toxicity. as opposed
to the probability of inducing carcinogenicity. For systemic toxicity, the daily dose is averaged over
the period of exposure, not over a lifetime. Therefore the exposure duration term drops out.
Therefore, for the average ABC air concentration in Grid 1 the average human exposure for the
consideration of systemic toxic effects is 1.3 x 10'4/.75 = 1.7 x 10"*
Water
The assumptions used in this analysis are
• An adult consumes 2 liters of water per day
• The body weight of an adult is 70 kilograms '
• The adult is exposed for 75 percent of his lifetime
As with the air assumptions, a more complete analysis would include a range of exposure and intake
assumptions emphasizing, for example, sensitive populations.
Using the ABC concentration in drinking water the calculations are
1.443 ug (ABC concentration in drinking water) x 2 liters (ingestion)
liter day
x 1 (body weight) x 1 mg (unit correction) x .75 (exposure duration
70kg 1,000 jug factor)
= 3.1 x 10'2 mg/kg/day (concentration to which people in River City are exposed)
Table IV-3 provides a summary of the exposure calculations used to compute doses in the
study area. The table also shows that nearly 138,000 people live in this area. It is important to
recognize uncertainty in the various assumptions and to focus attention on those factors having the
greatest impact on the results.
-------
Table IV-3
Summary of Results of Exposure Calculations
(doses in mg/kg/day)
• -..i^Maii^v-^yl •' •
. ;.' ?r\:- .'r'/.'f ; '.^j?- ?£:v1 .'*;* *-tj ;.'-»
ABC (Air)
Carcinogen
Toxicant
N-10
MO
TOX
ABC (Water)
Population of Study Area
(Total - 137,889)
H:
1.3x10^
1.7x10^
5.4 x 10s
1.8 x 10 '
8.4x106
3.1 x 102
4,322
;;f:;(iwNfn^
4.9 x 10"
6.5 x 104
1.8 x 102
2.3 x 10 3
4.1 x 10'
3.1 x 10*
43,559
GrW3
-------
Questions for Discussion
1. Does the use of modeled, not measured, air concentrations in the study grids affect the
validity of the analysis? What would you do if you didn't have either type of information?
2. Are the assumptions about human intake and exposure valid? What uncertainties surround
these assumptions or what alternative assumptions could be made? Should populations other
than "average" adults be considered?
3. What would be the effect of using concentrations and doses for each individual grid versus
average concentrations and doses for the entire study area?
-------
V. Risk Characterization
In the last step of human health risk assessment, the information collected and analyzed thus
far is integrated to characterize the excess risk to humans. Consistent with the alternative approaches
for describing dose-response relationships, at least three approaches to this step can be taken:
1. Provide an explicit numerical estimate of excess lifetime cancer risk for each population
group (e.g., each grid) for each pollutant. This estimate is derived by multiplying the unit
cancer risk (Chapter III) by the units of exposure or dose (Chapter IV):
Excess individual lifetime risk = (unit cancer risk) x (dose)
In this equation, excess risk is unitless - it is a probability.
2. Compare the exposure (dose or concentration, Chapter IV) for each population group and
each constituent to the NOAEL to compute the margin of exposure (MOE) (Approach 2 -
Chapter m) and the individual exposure ratio (IER) (Chapter III)
IER = (concentration) / (RfC)
MOE = NOAEL / exposure or dose
3. Describe risks qualitatively for each population group and each constituent
A complete risk characterization would normally include some combination of all three
approaches, along with a description of their relative merits. It is also essential that the statistical and
biological uncertainties in estimating the extent of health effects be described in this step.
Results
Cancer Risks
Table V-l presents the excess lifetime cancer risks for individuals and for the population of
the study area. The risks are computed separately for the two pollutants, by media, and for the three
grids of the study area and are then combined.
Individual risks are based on the unit cancer risks shown in Table III-2. [f unit risks derived
from other dose response models had been used, the excess risks shown would be lower. Risks are
also based on the dose or exposure information provided in Table IV-3.
V-l
-------
Tabto v-i
Estimated Excess Individual Lifetime Cancer Risk and Excess Annual Cancer Cases
from Exposure to Carcinogens In the Study Area
MC'Mr
ABC-Wafer
MO. Air
LoMllMi
AopuMten IMBM EMM* CKMM BJUMM EKMM EKMM
, UfeunM Ctno* UMta* Once* LMkM
Rtek'
Tottto
EXOMC
MMAwl
ExoMt
Anmiri
<
K>
GIKJ 1 • Smeler
OIHJ 2 Northern Chemical
Gnd 3 • Cily Ceniei
Averages
Tolab
4.322 3.01 10* 1.81 10* 70xl04 4.4 x 101 5.3x10* 33x10*
43.569 1.1 « 10* 6.9x10* 70xl04 4.4 « 101 6.6x10* 4.1x10'
90.008 1.7x10* 2.2x10* 7.0x10* 9.1 « 10* 16« 10" 2.1 i 10'
6.3x10* 33x10*
1.4x10' 8.6 > 10'
88 M 10* 11i 10*
18-10'
6 3 i 10*
' Confuted by nwMplylng «w «M COTCBT nik« m Teble III 2 by tw tfoM> In Tebfe IV 2.
1 Computed by dMttng tie tfokm* cancer nth by en expected 70 you llMme and muttplying tie mutt by Iw popub
' Computed by eddMig mhs bom eech I
' Computed by etMng oaiee ken«
-------
The excess risk to individuals is the product of the unit cancer risk and the dose. Using ABC
emissions in grid 1 as an example:
(2.28 x lO'2) x (1.3 x 10-4) = 3.0 x 10'6
(unit cancer risk) x (dose) = individual lifetime cancer risk
The number of excess cancer cases per year is computed by dividing individual lifetime risk
by the life expectancy of the population1 and multiplying the result by the population of the
appropriate grid. Again, based on ABC emissions in grid 1:
(3.0 xlO'6) / 70 x (4,322) = l.gxlO4.
(individual life- / (life expectancy) x (population) = excess cancer cases
time cancer risk) per year
Chronic Systemic Health Effects
The results of the risk assessment for non-cancer health effects are shown in Table V-2. The
dose information using air concentrations is included from Table IV-1. These concentrations are
compared with the reference concentration described for the pollutants in Table III-3. Note that
exposure is measured using concentration rather than dose as in the cancer example; the cancer
example was developed to show the human intake assumptions explicitly.
The individual exposure ratio is computed by dividing the concentration to which various
populations are exposed by the reference concentration. Based on ABC emissions in grid 1:
(6.1 x 10-') / (2.5x10°) = 0.24
(concentration) / (reference concentration) = Individual Exposure Ratio (IER)
Summary of Results by Facility
Table V-3 summarizes risk data separately for the two case study facilities. The table
aggregates the total individual cancer risk and number of cases for carcinogens and summarizes
average and maximum ERs for the non-carcinogens.
'Additional uncertainty results from use of annual cases since life expectancies vary.
V-3
-------
Table V 2
Estimated Individual Risks from Exposure lo Non-Carcinogens In the Study Area
ABC N-10
Location Poputatton Concentration1 CR' Conoenirittoo1 EH*
Grid 1 • Smeler 4.322 6.1x10' 0.24 2.5x10' 028
Grid 2 • Northern Chemical 43.559 2.3 x 10° 0.91 82x10' 92
Grid 3 • City Center 90.008 35x10° 1.4 10x10* 1.1
Averages 21x10* 0.84 28x10' 31
Totals 137.889 1.2 29.8
1 Fran Ubto IV 1
' Computed by dhrtng in cononnidon by t» nttonct cont«n>iign «t 0.9 |iy«*»c muM
' Computed by (Mdng ta concantoten by tw Mtcranc* eonunkaton ol 0 < uycubc M»ttr
TOX
Concentration1 IER*
39x10* 010
1.9x10' 005
MxlO1 0004
20x10' 0.05
002
-------
Table V 3
Summary of Risk Data
NoithMn CMinlMi
bMMAnnu*
IER1
UuimumlER
ABC Air
ABC -Water
Tola!
10 » 10*
70i104
7.1x10'
29x10'
I4«IO°
14x10°
ABC -Air
N 10
WMMRIvtrSmll*
084
31
14
92
IbmMAnnual
Hon-
UMUBWCMOtr
ER1
MuknuiniER
MO
1.8x10'
3.9x10°
TOX
005
' AuMaytd KfOM UuJf «u
1 Summed wiMa iMf *»•»
010
-------
Questions for Discussion
1. Which facility poses the greatest human health risk?
2. Which chemical poses the greatest human health risk?
3. Risks are presented for individual pollutants and combined and for individual grids and
combined. Is this appropriate? What are the advantages and disadvantages of various
presentation formats? If risks are averaged across grids, how would this affect your decision?
4. The reference dose is exceeded, for example, by exposure to ABC in the city grid (#3). How
do the risks from cancer compare with the risks from systemic health effects? Are they
comparable? How does the large population of the city affect your comparison?
5. How would you summarize the health risk situation in River City?
a. Excess cancer risks to humans exposed to MO and ABC are adequately presented in
Table V-l. Although risks obtained from the use of other models maybe lower, the
risks could be as high as those in the table. The risk of systemic health effects are
properly represented by the RfC and ER in Table V-2.
b. ABC is a probable human carcinogen based on observations of carcinogenicity in
experimental animals. MO is a known carcinogen based on epidemiological data.
Humans in the study area are exposed to both pollutants in the air. A relatively small
population is exposed to very high levels of MO causing significant individual risk
and several excess cancer cases (over 3 per year). The remaining population is
exposed to a slightly lower level of risk leading to less than one excess cancer case
annually. The entire population of the area is exposed to a moderately low individual
risk in air, and a slightly higher risk in drinking water. This risk leads to just over one
excess cancer case per year.
A significant fraction of the population is exposed to ABC and N-10 doses that
exceed the reference dose. There is, as a result, a significant potential for systemic
health effects from ABC and N-10 air exposure for most residents of the study area
c. No epidemiologicalevidence is available suggesting that ABC causes cancer. Animal
studies cannot be linked to human health risks so the results should not be used for
policy making. MO cancer risks should be used for policy making.
Similarly, ABC and N-10 studies of systemic effects were animal based and should
not be used to make policy decisions. The TOX studies on workers do provide a
reasonable basis for regulatory decisions.
d. Other (formulate your own)
V-6
-------
VI. Effects of Pollutants on the Environment and Welfare
The risk assessment portion of the case concludes with a discussion of ecological and welfare
damages. While the technical approach for analyzing these risks is similar to that used for human
health risks, data quantifying these non-health risks may be much more difficult to obtain. At a
minimum, a qualitative discussion of these risks should be part of the characterization of the
problem. If these risks appear significant, additional time and resources may be justified to collect
the information necessary to quantify some of the damages and potential benefits.
Background on River City Area
Several characteristics of the area around River -City are Important for understanding
potential impacts of pollution on the environment and welfare or quality of life. Before describing
the impact of the four pollutants on these factors we have summarized the resources of concern.
These factors are summarized on Figure VI-1.
• The White River, while already quite polluted, contains substantial fish populations which
may be affected by ABC.
• Agricultural areas downstream from River City use water from the river for irrigation and
are downwind of the two facilities
• Forests on the slopes of the river valley are exposed to pollutants in the air. Recreation areas
in these forest are also threatened by the pollution.
• Structures near the smelter are exposed to high levels of corrosive Metal Oxide.
Impacts of Hypothetical Facilities
Northern Chemical
ABC discharges to the stream are usually near the averages reported earlier in the case.
During the summer or droughts, the ABC loadings produce a higher concentration of ABC in the
river at the outfall from the plant. During these times, fish kills have occurred, resulting in strong
odors and dead fish washing up on the shore in River City which is just downstream from the plant.
A report from the Netherlands has linked ABC vapors to nervous system damage in insects,
especially bees, in several agricultural areas in North-Central Europe. Based on exposure to ABC
in air at approximately the same concentration measured at Northern Chemical, a large population
of the bee population was either killed or injured endangering the productivity of the nearby crops.
VI-1
-------
White River Smelter
Metal Oxide is corrosive, damaging buildings and other structures over an extended period
of exposure. Stone and brick buildings near the smelter have been discolored and, in a few cases near
the plant, structural damage has begun to appear. This damage has already required replacement of
stonework, brick, and mortar in several buildings.
Studies of smelters elsewhere in Europe have also linked MO emissions to leaf damage and
eventual death of deciduous trees. In fact, the forests on the hillsides nearest ~o the smelter have lost
many deciduous trees over the past 40 or 50 years. Recreation areas in the forest have lost many
trees, causing soil run-off and .discouraging local residents from using these areas.
Issues
1. Which of these problems are the most serious? Why?
2. How would you compare these damages to the human health risks characterized previously?
How would you rank health risks, welfare damages, and ecological risks in order of severity?
3. How would you incorporate these issues into your description of the severity of the problems
posed by the two plants?
4. Do you think that River City residents are more aware of these ecological risks and quality
of life issues or of human health risks? How might that affect your description of the
problem?
VI-2
-------
SUMMARY OF ENVIRONMENTAL
IMPACTS OF CASE STUDY
POLLUTANTS
Figure VI-1
-
White Rlwe Smeller
%rth.rn Chemical
Insect
Damage
ABC
Forest Damage - MO
Structural Damage -
MO
Fish Kills - ABC
City Center
-------
VII. Options for Addressing Industrial Pollution in River City
Review of Sources and Routes of Releases
Northern Chemical
• Releases of ABC to the air result primarily from open tank degreasing operations. These
tanks contain large quantities of ABC and are in use throughout the work day. Other solvent
uses release small quantities of the material throughout the plant, mostly due to evaporation
from drums.
• N-10 is produced at the plant, but air releases result from its use as a catalyst in adhesive
manufacturing.
• ABC is released to the White River as well. ABC is discharged in wastewater from the
rubber and adhesive manufacturing operations. Existing settling basins at the plant do little
to reduce the concentration of ABC before it enters the river.
White River Smelter
• Metal Oxide and TOX are both released during the smelting of metal ore to make metal. The
compounds are released from stacks and at the ground-level from the smelting units. Use
of poor quality ore increases the amount of MO released during smelting.
Options to Reduce Releases
Staff from the Regional Inspectorate and the Ministry have collaborated to develop a series
of technical options specific to the plants and pollutants near River City. The options vary in terms
of their level of sophistication, their effectiveness, and their cost. Option costs were developed by
engineers with experience controlling emissions from various types of industrial facilities in the US
and Europe. Staff review has already eliminated several options that were technically feasible, but
inefficient. That is, the option was less effective and more expensive than another option.
It is important to note that these options may be relevant to a range of approaches for
controlling emissions from these plants. In order to develop an implementation plan, it is important
that you first determine an overall goal or strategy for managing the risks of these facilities, then use
these options to help you implement the strategy. This is simply a menu of possible options from
which you can choose. Your task is to decide on the best approach: you are free to develop additional
options, combine options, or choose no action.
VIM
-------
Implementation Issues
This chapter includes primarily the technical specifications for options such as their potential
for reducing releases of each pollutant at each facility. At the end of the review, a summary table lists
all of the options, their costs, and their effects on pollutant levels. These options were developed
without consideration of how easy they might be to implement, enforce, or afford. It is important that
all of these factors enter into the choice of a regulatory approach, but these factors are not explicitly
discussed in this chapter.
ABC Air Releases
Option 1: Vapor Recovery System
Because most releases come from the open-tank degreasing operations, this option provides
for a system to exhaust the ABC-contaminated air. filter out the ABC, return the recovered
solvent to the tanks, and vent the cleaned air to the outside.
• High capital costs to construct the system, moderate operation and maintenance costs
(O&M) for power to run the system and maintaining the filters.
• Savings on the cost of new ABC will result from reuse of old solvent
• ABC concentrations in the air outside the plant will decline by 99 percent. If the
previous concentration was 100 ng/m\ the new concentration would be 1 jug/m3.
Option 2: Improved Covers for Tanks and Drums Containing ABC
Modified covers for ABC containers can reduce evaporation during use and transfer of the material
from one container to another. These covers would not affect the productivity of the workers, but
would eliminate releases from all sources in the plant.
• Low capital costs for the covers and very low cost to maintain them.
• Some savings will result from preventing evaporation
• ABC concentrations in the air outside the plant will decline by 25 percent. While this
option affects all sources to ABC to the air, it is less effective than Option 1 for the
degreasing tanks.
VII-2
-------
Nitro-10 Air Releases
Option 1: Discontinue N-10 Production at Plant: Substitute New Catalyst
Northern's adhesive productioncould continue with a cleaner catalyst that would not produce
dangerous emissions. The change would close down part of Northern's operation that
currently manufacturers N-10 for this purpose.
• Low capital costs for process modifications, but annual cost is much higher because
the replacement catalyst is much more expensive
• N-10 releases would be eliminated by this option
Option 2: N-10 Recovery System
The adhesive production process could be modified to enable plant engineers to draw off by-
product gases, force them through a carbon or other adsorbent to extract the 5110, and
recover most of the N-10.
• This system would be very expensive with very high capital costs required to
purchase and install the recovery equipment. Annual O&M costs would be moderate
to high for power to operate the equipment and for replacing the carbon filters in the
system.
• Concentrations of N-10 in the air outside the plant would decrease by 99 percent.
ABC Water Releases
Option 1: Aerate Wastewater to Remove ABC
Because ABC is volatile, a relatively simple aerator will drive off most of the ABC in the
wastewater. With this technology, the ABC would be released to the air.
• Low capital costs and low O&M costs
• Concentrations in water at the plant outflow would be 90 percent lower after aeration
• The mass of ABC released to the air would be about 50 percent of the amount
released from other sources (i.e., risk from air emissions would increase 50 percent).
VII-3
-------
Option 2: Air Strip Wastewater and Absorb ABC from Air Stripper
This more complex wastewater treatment technology volatilizes ABC from the water but
instead of venting it to the air, it captures the ABC in a carbon filter.
• Moderate capital costs are required along with low to moderate operating costs.
• Concentrations in water at the plant outflow would be 99.8 percent less that current
concentrations. If the current concentration was 1,000 Mg/m3, the new concentration
would be 2 /zg/m3.
• No measurable increase in ABC concentrations in air would result from this option.
Metal Oxide and TOX Air Releases
Option 1: Baghouse on Stack
MO and TOX are released both from the stack and at ground level from the smelter. This
option focuses on the stack releases and provides for a baghouse which captures particles
released from the stack. MO and TOX both adsorb onto particulates, so stack emissions are
controlled effectively with this option, but fugitive emissions are not affected.
• Moderate (in relation to the other alternatives available to deal with MO and TOX)
capital costs are required with low annual operating costs including disposal of
particulates captured in the filters.
• Total emissions of both MO and TOX are reduced by 50 percent. Control of stack
emissions is much more efficient, but fugitive emissions are not affected.
• Risks from disposing of the particulates captured in the filter are unknown.
Option 2: Dust Recovery Hoods
Hoods could be added to the smelter to capture particulates and gases as they are released
from the ore. This more complex technology would reduce emissions from both stack and
fugitive sources.
• High capital costs and moderate to high O&M costs. These costs include disposal of
collected residue
• Total emissions of both pollutants would be 90 percent lower.
VII-4
-------
• Risks from disposing of the particulates captured are unknown.
Option 3: Purchase Ore From Alternative Source
The level of MO released from the smelter is high because of the poor quality of ore used at
the smelter. Changing sources would mean switching from a mining operation to one in
another country.
• Very low capital costs for plant modifications, but very high annual costs for more
expensive, higher quality ore.
• Emissions and concentrations of MO would decrease 99 percent while TOX
concentrations would decline by 50 percent.-
Summary of Option Costs and Effectiveness
Table VII-1 summarizes the options for the plants. The cash flow of the costs is shown along
with the present value of costs, the annualized present value, and the effect of the options on
pollutant releases. In all cases, the cost of capital was assumed to be 15 percent and an operating life
of 30 years was assumed for all investments. Therefore, all costs are shown for a comparable period
of time. An appendix to the chapter reviews sample calculations of present value and annualized
present value based on the cash flows.
The engineers responsible for developing the options and estimating their impact on
emissions have also reported that it is reasonable to expect that emissions reductions to air and
surface water will translate into parallel reductions in concentrations of the pollutants. For example.
if ABC releases to the air are cut by 90 percent, the average annual concentrations of ABC in the
study areas will also fall by 90 percent.
Issues for Discussion
1. What are the key components of each option and how do the options differ?
2. How would you compare the level of uncertainty in the risk assessment results with
uncertainty in cost estimates? Which would you be more comfortable with?
3. Are there any options you would reject/accept right now, with no additional information?
Why?
4. How would you compare the enforce ability of the various options? How important is this
factor?
5. Should you involve the public or NGOs in the review and evaluation of options? If so. how?
VII-5
-------
Northern Chemical
ABC • Air
Summary
Table vu • i
of Option Costs and Effectiveness
Amuafead
CMfe Rew Present Value Present Vaha»
(000 UJC.) (OOOU.C.) (009U.C.)
1 . Vapor Recovery System
Capital
OAM
Solvent Savings
2. Cover*
S<
£10
1. Replace Catalyst
Capital
OAM
Hvtflt SeiviflQi
Capital
OAM
30.000 41.490 6.319
2.000 ....
(250)
3.000 3.328 507
60
(10)
1.500 165.649 25.228
25.000
•
Effect on ;
Pollutant
Concentration* :
i
. Reducss 3S% ;
Reducss 25%
Eliminates N-iO
2. N-iO Recovery System
ABC • Water
i. Step Aeration
2. AJr Strip/Carbon
^nfnMel RffWC SfffefMM
1. Bagheuea
2. Heeds
3. Change ere
•
Capital
OAM
Capital
OAM
Filter
Captal
OAM
Disposal
r - MO and TOX
Captal
OAM
Captal
OAM
Capital
OAM
200.000 265.660 40.460
10.000
4.000 13.849 2.109
1.500
18.000 28.834 4.391
1.500
150
50.000 82.830 12.615
5.000
240.000 436.979 66.552
30.000
300 394.259 60.046
60.000
Reduces 39% ;
Reduces 90%
Reduces 99 3%
Reduces 50%
Reduces 90%
Reduces MO 99%
Reduces TOx 50%
VII-6
-------
Chapter VII Appendix
Sample Calculations of Present Value and Annualized Present Value
Present Value Example: ABC - Air Option 1
Cash Flows: Capital Cost 30 million Dollars (one-time cost)
Operating Cost 2 million Dollars (each year for 30 years)
Operating Savings 0.25 million Dollars (each year for 30 years)
Computing Present Value: 30 year operating life and 15 percent interest rate
Year 0 30 million [initial cost]
Year 1 1.75 million / (1 + 0.15)' [cost in first year is discounted using interest
rate and year]
Year 2 1.75 million/(I+0.15)2
Year 30 1.75 million / (1 + 0.15)30
Sum = Present Value = 41.5 million Dollars
Annualized Present Value Example: ABC - Air Option 1
Present Value = 41.5 million Dollars
Annualized Present Value = A level stream of payments each year which, given the interest rate
and payment period, would equal the present value
Annualized Present Value = Present Value x Annualization Factor
r (I + r)'
Annualization Factor = where r is the interest rate and t is
(I + r)1 - 1 the time period over which
payments are made
Annualization Factor = 0.15 (1+ 0.15)JO
(1 + 0.15)30- 1
Annualization Factor = 0.152 given 30 years and 15 percent interest rare
Annualized Present Value of Example = 41.5 million Dollars x 0.152 = 6.3 million Dollars
-------
VIII. Economic Factors Of Option Selection
This chapter summarizes economic information about the two companies that may influence
the options selected for addressing the pollution in the River City area. River City is experiencing
substantial unemployment and many in the community hope to retain the current industries in River
City and thereby preserve the jobs.
Northern Chemical
Northern Chemical sold 1.2 billion Dollars worth of chemical products in 1991 and exported
400 million of the total (mostly to the Middle East). After wages, raw material costs, and other
expenses, the net income (profit) of the company was 60 million Dollars last year. Growth is forecast
for the next several years, especially in the export area. Trie company is now a joint venture, with
20 percent ownership in the hands of public investors.
The company employs 3,500 workers from the River City area. About 5 percent of those (175
workers) produce Nitro-10 for use in the adhesive production process. If option I is chosen
(eliminating N-10 production), the 175 workers would lose their jobs and unemployment in River
City is already quite high.
The company has made few investments to control releases of ABC and N-10 in the past. The
wastewater treatment settling basins were designed to control solids and conventional pollutants and
they have very little impact on the ABC concentration in the wastewater.
Options under consideration to alleviate ABC and N-10 emissions have annualized costs of
between 500,000 Dollars and 40,000,000 Dollars This represents between 0.04 percent and 3.4
percent of the company's annual sales or between 0.1 percent and 67 percent of prof-ts.
White River Smelter
The metal smelter is a larger company, but in worse financial condition than Northern
Chemical. The metal sold in 1991 accounted for 1.5 billion Dollars, with 100 million Dollars of the
total in exports to former Soviet Republics. The profit of the enterprise was 10 million Dollars in
1991 and forecasts indicate that revenues may decline in 1992.
The 2,500 workers at the plant are less well paid than those at the chemical plant. The
workers, trade unions, and residents of the area around the smelter are staunch defenders of the plant
and its jobs. The plant output dropped significantly between 1990 and 1991 because much of the
metal was shipped to the former Soviet Union and those markets have now disappeared. At the time.
the unions and residents fought hard against any layoffs at the plant.
VIII-1
-------
The smelter recently made a substantial investment in a scrubber for the stack venting the
power plant at the facility. This scrubber has significantly improved the paniculate and SO: pollution
levels in the river valley. The smelter itself, however, has not been fitted with new emission control
equipment in 20 years. At that time, minimal controls were installed to improve conditions for the
plant workers.
The options under consideration to reduce TOX and MO emissions have annualized costs
ranging from 13,000,000 Dollars to 67,000,000 Dollars, or between 0.9 percent and 4.5 percent of
annual revenues. The cost of each option exceeds the profits earned at the plant in 1991.
If the country's ore were no longer used at the smelter (Option 3), approximately 750 miners
would lose their jobs, since the new ore would come from outside the country.
Issues for Discussion
1. Does the economic information affect your preferences for the various options? Would you
eliminate some options based on the economic impacts?
2. Is it appropriate to consider the costs of various options and the economic impacts they
would have in deciding what to do at these facilities?
3. What level of government (national, regional, local, other) and what departments or agencies
should evaluate these economic impacts? Is it the same level and agency that determines
environmental goals and standards?
4. What information would you provide to the public about these impacts?
VIII-2
-------
IX. Effectiveness and Selection of Options
The final chapter of the case summarizes how the options would improve the
environment around River City. You will then be asked to choose a strategy for addressing the
health and environmental risks in the area.
Effectiveness of the Options
Adopting any of the options identified in Chapter VII would reduce the exposure of
individuals to the pollutants in the area and would therefore reduce the risk. As you have seen,
however, the options vary with regard to the extent to which they control emissions, cost,
economic impacts, and ease of implementation and enforcement.
Table IX-1 summarizes the options for addressing each pollutant and indicates how
human health risk would change as a result of choosing each option. Because of incomplete data
characterizing ecological risk and welfare effects, it is difficult to measure improvements in the
environment and quality of life. It is clear, however, that any reduction in concentration will
reduce the severity and frequency offish kills, materials damage, deforestation, and insect
damage.
The human health risk measures shown on the table include
• Average individual lifetime cancer risk for the entire study area
• Highest average individual risk from the three study grids
• Total additional annual cancer cases predicted for the entire study area resulting from
exposure to these carcinogens
• Average individual exposure ratios (ERs) for the entire study area
• Highest ER from the three study grids
For comparison, the table also includes the annualized cost of each option, and one
possible comparison between risk reduction and cost: the annual cost per annual case avoided.
This ratio simply compares the annual cost of each option with the reduction in predicted annual
excess cancer cases. If an option has a higher ratio than the others, then it provides a relatively
more expensive way to reduce predicted excess cancer cases. Many other measures like this
could be computed as well.
IX-1
-------
Selection of Options
While the ultimate decisions about the area depend on your choice about the relative
importance of the various risks, your overall objectives for controlling pollution in the area, and
many other factors, several possible approaches have been suggested. These are meant to
illustrate possible approaches, but do not exhaust all of the possibilities; you are free to develop
your own decision and rationale for it. None of these scenarios resolve important questions about
what level of government should set and enforce standards.
Scenario 1: Best Technology
The technical staff working on the options have identified several feasibJe approaches for
each problem. In each case, however, one option can be idenxified as the best technological fix
for the problem. These options represent the technologies that are the most effective at reducing
the emissions of the targeted pollutants.
• ABC recovery from air (Option 1)
• N-10 recovery (Option 2)
• ABC air stripping from water (Option 2)
• MO and TOX recovery using hoods (Option 2)
The policy decision represented by this approach is that all facilities should employ the best
available technology to control emissions of pollutants that pose threats to human health
For the carcinogens in the study, the average individual lifetime risks would be reduced
by 90 percent (MO), 99 percent (ABC in air), and 99.8 percent (ABC in water). The number of
excess annual cancer cases would fall from over 5 to 0.4. The average concentration of non-
carcinogens in the study area would decline below an IER of I for !x'-l( (ABC and TOX were
below that threshold in the baseline). Maximum ERs for ABC and 1-110 would also fall below
the 1.0 threshold. This scenario provides significant reductions in pollutants linked to ecological
and welfare damages as well.
The annual cost of this approach would be 51.2 million Dollars at Northern Chemical and
66.6 million Dollars at White River Smelter for a total cost of nearly 118 million Dollars
Scenario 2: Setting Health Standards
Several policy maters have suggested setting an average individual lifetime cancer risk
level which should be enforced throughout the country. A risk level of 10- has been suggested as
IX-2
-------
an appropriate standard.1
This approach is very different from the technology based scenario because it dictates
only the end result, rather than the means of achieving that end result. The facilities would be
free to reach the resulting standard in the most efficient way possible.
Although these options are not mandated, if we use the case study options as possible
strategies for meeting the standard for carcinogens:
• No controls are necessary on ABC air emissions because the baseline average individual
lifetime risks are 1x10-5
• Baseline risks for ABC in water exceed the standard. Option 1, aeration, lowers the risk
below the threshold and would be acceptable under this scenario. (Remember that the
aeration option transfers the ABC to the air directly.)
• None of the options reduce MO risks below the standard, although Option 3, replace ore,
lowers the average individual lifetime cancer risk to 2 x 10 4. In this case, the smelter
might have to scale back production to reduce emissions of MO.
The cost of this scenario would be 2.1 million Dollars for Northern Chemical and an uncertain
cost for the smelter.
Note that a similar standard could be developed for non-carcinogens based on the average
or maximum ER. Also note that progress in reducing ecological risks may be less dramatic under
this scenario.
Scenario 3: Setting Emission Standards
An alternative standard-setting approach would be to set emission or concentration
standards for individual pollutants. These standards would again set the end result and leave
facilities flexibility in how to achieve that result. Rather than developing the entire scenario, we
have included a description of how to "back-calculate" from a standard that protects human
health adequately (i.e., a certain risk level) to ambient air standards.
Risk Standard for Carcinogens
If a human health cancer risk standard is set at 10'4. an ambient air standard can be
calculated to keep cancer risk below that level. First, the dose that would lead to the risk must be
'This standard implies an excess individual lifetime cancer risk of 1 in 10.000. For
companies with the risk analysis results, any risk at or below 1 x 10"4 is assumed to be acceptable.
IX-3
-------
computed:
Individual lifetime cancer risk = Unit cancer risk x Dose
1x104 = 2.28 xlO'2 x Dose
(standard) (Chapter III)
Dose = 4.4 x 10° mg/kg/day
Then the ambient air concentration that results iri this dose can be computed.
Dose = Concentration x Intake assumption
4.4x10° = Concentration x 20m3/day x 170kg x lmg/l,000yug xO.75
(inhalation) (body (unit (exposure
weight) correction) duration
facror)
Concentration = 21 Mg/rn3
If the ambient air standard for ABC were set at 21 Mg/m3, individual lifetime risks for
individuals exposed to ABC would not exceed the desired cancer risk level. Note that exposure
to other carcinogens in the air would add to this risk level.
Risk Standard for Non-Carcinogens
One possible standard for systemic toxicants would be an ER of 1.0 for the pollutant
Using ABC as an example, the standard would be defined by the reference dose as follows:
Individual exposure ratio = Concentration / Reference concentration
1.0 . = Concentration/ 2.5
Concentration =
This ambient air concentration standard for ABC would lead to an IER of I or less.
This exercise illustrates how concentration standards may be set based on health or other
goals using information developed in a risk assessment.
IX-4
-------
Your Role
Your group should develop an approach for dealing with the environmental problems in
River city. You know you will have to compete with many other demands for resources and
cannot, therefore, just choose the most protective measures. Many other scenarios for action in
River city could be developed. In reaching your final decision about your plan for the area, you
may consider any combinations of options you wish. There is no single correct approach for
addressing the pollution problem in the city and you must balance many factors in your decision.
Be sure that you consider
• Benefits: human health benefits, ecological benefits, welfare benefits, better quality of
life
• Costs: direct cost of controls, costs to the companies or the government, employment
effects, government costs to set and enforce standards
• Implementation and enforcement: what level of government should make-these
decisions, is the option enforceable, who will implement and enforce the decision?
• Public opinion: how are the citizens of River City affected and how do they feel about
the problems and solutions?
After you have reached a final decision, your group will report the conclusion in a roleplaying
exercise during which you will make an appeal for financial support from the World Bank. You
should be prepared to discuss the rationale for your group's decision including each category
listed above. Have fun!
IX-5
-------
X
o\
Jaotoor-i
Risk Reduction and Effectiveness of Options
! .. {
.' ' ' '• •
-::- -.-'!.' •
'•':: , tf •
M.CIwmlc«l: ABC-Alt
QaMlin*
Opt I: Recovwy
Opl2: Cov*n
N.ChMiiteal:M-10
Baaelin*
Opl.1: Replace catty*1
Opl.2 Recovery
N.ChMnicil: ABC-Water
BasBlmB
Opll: AoiaU
Opl.2: Air nilp
3m.nw;MOandTOX
Basel**
Op! 1. Baghoua*
Opt 2 Hoods
Opl3 Changs oro
MIMOttO^
naitimlnn
.» ffw1?T*w'lp*1
•
99%
26%
100%
99%
•
80%
99.8%
60%
90%
99% MO
50%TOX
HlghMl
Avangc AVMVM . AimuaJ
H..CIV ILCH1 CMM
1.0 HO1 17x10* 2.9x10*
1.0«10' 1.7x10' 2.9 x 10"
7.6 x 10* 13-10* 22x10*
•
7.0 «I04 7.0x10" 1.4x10*
7.0x10* 7.0x10* 14x10'
1.4 x 10* 1.4 x 10* 2.8 x 10 '
(M0|
1.8i10' 6.3x10' 3.9x10*
9.01 10' 2.6 x 10' 2.0 x 10*
1.8 1 10' 6.3x10' 39x10'
1 Hi Hi' 53 x 10* 3.9 x 10*
Amngc Mwdmurn
Btf! icn*
O.B4 1.4
0.0084 0.014
0.63 1.1
31 92
0 0
031 0.92
rroxj
005 01
0.025 O.OS
0.005 001
0 025 0.05
AnnuailMd
CAM
(000 U.C.)
0
6.300
507
0
25.200
40.500
2.100
4.400
12.600
66.600
60.000
Annual
Cott/CsM
Avoidt*
(000 U.C.)
-
219.000
70.000
I./OO
3.100
6.500
19.000
ts.&oo |
II
' IndiviJjal Uskmk cancer n«k Compuwd by muluptyn^ will idULiN ink by itosv II
' Indrjuijal «>potuia talio Compjlud by UinJMtj uMLanUdliciii (
-------
Instructions for Role Play: Presentation to the World Bank
On the final day of the course, your group will make a presentation to the World Bank (a
group consisting of the course facilitators) during which you will explain the strategy you
developed for addressing the environmental problems in River City and you will make an appeal
for funding from the World Bank to assist in your efforts. Your presentation should include a
discussion of how your plan supports your policy goals, how it will reduce the risks posed to the
citizens of River City and why you chose the specific course of action.
You know that the World Bank will provide no more than 50 million Dollars and you
must indicate to the World Bank how you hope to accomplish other portions of your plan that
can not be met by this level of support.
To prepare for your presentation (which cannot exceed 15 minutes), you will need to:
(1) Identify individuals from your group will make the presentation
(2) Carefully plan your presentation and be sure to include the points you consider
important and which demonstrate the completeness of your strategy
(3) Identify questions the World Bank is likely to ask and prepare your responses
(4) Prepare any visual aids you want to use to make your points
Good luck and have fun.
-------
ENVIRONMENTAL POLICY COURSE
GLOSSARY
Absorbed dose. The amount of a substance penetrating the exchange boundaries of an organism
after contact. Calculated from the intake and the absorption efficiency. Usually expressed as
mass of a substance absorbed into the body per unit body weight per unit time (e.g. mg/kg-day)
Acceptable daily intake (ADI). Estimate of the largest amount of chemical to which a person
can be exposed daily that is not anticipated to result in adverse effects (usually expressed in
mg/kg/day).
Carcinogen. A substance that increases the risk of cancer.
Cost/benefit analysis. An evaluation of the quantifiable costs (such as cash flows, economic
impacts, and welfare effects) and benefits (improvements in human health, environmental
conditions, efficiency) of proposed actions. Both costs and benefits should be evaluated in
marginal terms - the incremental effects compared to the current situation.
Cost-effectiveness. The goal of a cost-effectiveness study is to identify a solution that provides
the greatest benefit (lives saved, for example) for the lowest unit cost.
Dose. Measurement of the amount of a substance received by the subject, whether human or
animal.
Dose-response evaluation. A component of risk assessment that describes the quantitative
relationship between the amount of exposure to a substance and the extent of toxic injury or
disease.
Dose-response relationship. The quantitative relationship between the amount of exposure to a
substance and the extent of toxic injury produced.
Ecological risk. The likelihood that adverse ecological effects will result from exposure to
chemicals or other stressors.
Epidemiological study. Study of human populations to identify causes of disease. Such studies
often compare the health status of a group of persons who have been exposed to a suspect agent
with that of a comparable unexposed group.
Excess lifetime risk. The additional or extra risk incurred over the lifetime of an individual by
exposure to a toxic substance.
-------
Exposure. The accessibility of an organism to contact with a specific chemical or chemical
agent.
Exposure Route. The way a chemical or physical agent comes in contact with an organism (e.g.
by ingestion. inhalation, or dermal contact)
Extrapolation. The estimation of a value beyond its known or identified range on the basis of
certain variables within the known range. An extrapolation model is used to estimate the effects
of a chemical on humans (unknown) from its observed effects on animals (known range).
Gavage. Type of exposure in which a substance is administered to an animal through a stomach
tube.
Hazard identification. A component of risk assessment that involves gathering and evacuating
data on the types of health injury or disease (e.g.. cancer) that may be produced by a chemical
and on the conditions of exposure under which injury or disease is produced.
Hi~h-to-low-dose extrapolation. The process of predicting low-exposure risks to animals from
the measured high-exposure, high-risk data.
Human equivalent dose. The human dose of an agent that is believed to induce the same
magnitude of toxic effect as that which the known animal dose has induced.
Human exposure evaluation. A component of risk assessment that involves describing the nature
and size of the population exposed to a substance and the magnitude and duration of exposure.
The evaluation could concern past exposures, current exposures, or anticipated exposures.
Human health risk. The likelihood (or probability) that a given exposure or series of exposures
may have or will damage the health of individuals experiencing the exposures.
IER (Individual Exposure Ratio). The relationship between actual human exposure and the
reference concentration. ER = concentration to which humans are exposed/RfC.
Incidence of tumors. Percentage of animals with tumors.
Intake assumptions. Assumptions about humans used to convert concentrations of chemicals in
the air, in food, or in water into exposure concentrations or doses.
Interspecies extrapolation model. Model used to extrapolate from results observed in laboratory
animals to humans.
Individual lifetime cancer risk (ILCR). Calculation of expected risk to a given individual
-------
exposure to a chemical. Calculated as the product of the unit cancer risk and the dose.
Limited evidence. According to the U.S. EPA's Guidelines for Carcinogen Risk Assessment.
limited evidence is a collection of facts and accepted scientific inferences which suggests that the
agent may be causing an effect, but this suggestion is not strong enough to be considered
established fact.
Linearized multistage model. Derivation of the multistage model, where the data are assumed to
be linear at low doses. The model used by EPA to estimate human health risks from carcinogens.
This model yields high risk estimates with a 95% level of confidence that the actual risk will not
exceed the estimated risk.
Margin of exposure (MOE1. Maximum amount of exposure producing no measurable adverse
effect in animals (or studied humans) divided by the actual amount of human exposure in a
population. Previously this was called the margin of safety.
Microeram (u.z). One-millionth of a gram (1 ,ug = 3.5 x 10-8 oz. = 0.000000035 oz.).
Milliaram (mg). One-thousandth of a gram (1 mg = 3.5 x 10 5 oz. = 0.000035 oz.).
Model. A mathematical function with parameters which can be adjusted so that the function
closely describes a set of empirical data. A "mathematical" or "mechanistic" model is usually
based on biological or physical mechanisms, and has model parameters that have real world
interpretation. In contrast, "statistical" or "empirical" models are curve-fitting to data where the
math function used is selected for its numerical properties. Extrapolation from mechanistic
models (e.g.. pharmacokinetic equations) usually carries higher confidence than extrapolation
using empirical models (e.g., logit).
Modifying factor. An uncertainty factor, greater than zero and less than or equal to ten, used to
evaluate toxicity data. Its magnitude reflects professional judgment regarding aspects of the data
used from the assessment; e.g., the number of species tested and the completeness of the overall
database.
No observed adverse effect level (NOAEU. The highest experimental dose at which there was
no statistically or biologically significant increase in a lexicologically significant endpoint. Some
effects may be produced at this level, but they are not considered to be adverse, nor precursors to
specific adverse effects.
One-hit model. Mathematical model based on the biological theory that a single "hit" of some
minimum critical amount of a carcinogen at a cellular target-namely DNA~can initiate an
irreversible series of events, eventually leading to a tumor.
Population cancer risk. Individual lifetime cancer risk adjusted for sensitive portions of the
-------
population.
Potency. Amount of material necessary to produce a given level of a deleterious effect.
PPb. Parts per billion.
ppm. Parts per million.
Public participation. Involving the public (workers, town residents, other government officials)
in the development of a management response.
Reference concentration (RfC). An estimate with uncertainty spanning perhaps an order of
magnitude (or greater) of the daily exposure to the human population (including sensitive
subpopulations) that is likely to be without appreciable risk of deleterious effects during a
lifetime. The RfC is used in risk assessments when exposure is through inhalation and is
expressed in units of Mg/m3/day.
Reference dose (RfD). An estimate with uncertainty spanning perhaps an order of magnitude (or
greater) of the daily exposure to the human population (including sensitive subpopulations) that
is likely to be without appreciable risk of deleterious effects during a lifetime. The RfD is, used
in risk assessment when exposure is by ingestion and is appropriately expressed in units of
mg/kg/day.
Risk. The probability of injury, disease, or death under specific circumstances. In quantitative
terms, risk is expressed in values ranging from zero (representing the certainty that harm will not
occur) to one (representing the certainty that harm will occur). The following are examples
showing the manner in which risk may be expressed: E-4 or 10"* = a risk of 1/10,000; E-5 = a
risk of 1/100,000; E-6 = a risk of 1/1,000,000. Similarly, 1.3E-3 = a risk of 1.3/1000 = 1/770;
8E-3 = a risk of 1/125; and 1.2E-5 = a risk of 1/83,000.
Risk assessment. The scientific activity of evaluating the toxic or other damaging properties of a
chemical or human activity and the conditions of exposure to it. Risk assessment is designed to
ascertain the likelihood the effects will occur and to characterize the nature of the effects on
human health, the environment, or on quality of life issues.
Risk characterization. Final component of risk assessment that involves integration of the data
and analysis involved in hazard evaluation, dose-response evaluation, and human exposure
evaluation to determine the likelihood that humans will experience any of the various forms of
toxicity associated with a substance.
Risk management. Decisions using information from human, ecological, and welfare risk
characterizations as well as other non-risk information about the appropriate means for control of
a risk. These other issues can include economic, legal, and technical considerations.
-------
Route of exposure. Method by which the chemical is introduced into the biological organism.
Safe. Condition of exposure under which there is a "practical certainty" that no harm will result
in exposed individuals.
Statistically significant. The difference in tumor incidence between the treated and control
animals that is probably not due to chance.
Sufficient evidence. According to the U.S. EPA's Guidelines for Carcinogen Risk Assessment,
sufficient evidence is a collection of facts and scientific references which is definitive enough to
establish that the adverse effect is caused by the agent in question.
Systemic toxicants. Chemicals that cause toxic effects, other than cuiker, at sites distant from
the entry point of a chemical.
Threshold dose. The dose or exposure below which a significant adverse effect is not expected.
Carcinogens are thought by EPA to be non-threshold chemicals, to which no exposure can be
presumed to be without some risk of adverse effect.
Total dose. Sum of doses received by all routes of exposure.
Uncertainty factor. Factors used in operationally deriving the RfD or RfC from experimental
data. These factors are intended to account for (a) the variation in sensitivity among the members
of the human population; (b) the uncertainty in extrapolating animal data to the case of humans;
(c) the uncertainty in extrapolating from data obtained in a study that is of less-than-lifetime
exposure; and (d) the uncertainty in using experiments from which no NOAEL was established
for the chemical. Usually these factors are each set equal to ten.
Unit cancer risk. The increased likelihood of an individual developing cancer from exposure to
one unit of a substance over a lifetime.
Upper-bound estimate. Estimate not likely to be lower than the true risk.
Weight of evidence classification. A system for characterizing the extent to which the available
data indicate that an agent has a certain effect (e.g., that it is a human carcinogen).
Welfare risk. Likelihood that due to exposure to chemicals or from other stressor (such as
deforestation) a community will suffer economic loss or degradation of quality of life issues such
as aesthetics or recreation.
-------
RISK ASSESSMENT AND RISK MANAGEMENT: A BRIEF OVERVIEW
As defined by the National Academy of Sciences, risk assessment, for human health
impacts, involves the scientific evaluation of the toxic properties of a chemical and the likely
human exposure routes to it. The assessment consists of four basic elements, as shown in Figure
1:
• Hazard Identification ~ a review of the types of injury or disease that a particular
chemical could cause;
• Dose-Response Evaluation— an analysis of the "strength" of the various chemicals
present in the case to help determine how toxic they are;
• Exposure Assessment- an evaluation of the degree to which humans might be subject to
these harmful emissions; and
• Risk Characterization- an attempt to describe the risk in a manner that will facilitate
understanding of the problem and the identification of response alternatives.
The risk characterization identifies the likelihood that humans will be affected by the chemical in
question and the nature of the effects that they might experience. The risk characterization, then,
captures the two elements necessary to define a risk; the probability or likelihood that an injury
or disease will occur and a description of the nature of that effect. In addition, the
characterization should address uncertainties in the data and assumptions made in arriving at the
final conclusion about the risk.
Risk management is a term that encompasses a range of regulatory activities that uses the
risk characterization as well as non-risk data including technical, economic, legal, and regulatory
considerations. The relationship between risk assessment and risk management, with risk
characterization as the link between the two, is shown in Figure 2. Information from the risk
assessment can also be used to compare the relative risks posed by different facilities or different
media, to help set regulatory priorities, and to set standards.
While the risk assessment is a scientific process, it is not an exact one. At each of the
four stages scientists must review evidence from a number of experiments and determine what
conclusions are best supported by the available data. There is, by the nature of the process.
uncertainty at each step. This uncertainty does not preclude the use of the information but does
require that the manager fully understand the conditions underlying the assessment. It is not
enough for the manager to be told that a certain chemical is a probable human carcinogen and
that the risk posed by a given facility is that there is a I in 100,000 chance of exposure
-------
Figure 1
RISK ASSESSMENT
Dose-Response
Assessment
Risk
Characterization
Hazard
Identification
Exposure
Assessment
-------
Kigure 2
RISK ASSESSMENT
RISK MANAGEMENT
Dose-Response
Assessment
Risk
Characterizati
Hazard
Identification
&
Risk
mmunicftion
Regulatory
Decision
Exposure
Assessment
Economics
Other Non-risk
Factors
-------
causing an excess cancer among the affected population. Rather, the manager must understand
the strength of evidence and uncertainties surrounding the risk characterization in order to make
an informed decision. Accordingly, the manager must understand the steps involved in a risk
assessment, even if he/she would never actually do the analytical work of the assessment.
HAZARD IDENTIFICATION
Hazard identification is a qualitative assessment of data to determine if a chemical is
capable of causing deleterious effects. The experimental data is usually from animal testing. The
animals (usually rats or mice) are given very large doses of the agent to determine if an effect
(e.g., an increase in the incidence of cancer) can be induced. Assessors will wan- to know if the
effects were seen in more than one species or more than one sex of the same species.
EPA has developed guidelines for carcinogenic risk assessment that review the evidence
about a particular substance and classify it into one of five groups:
Group A—Human Carcinogen
Group B—Probable Human Carcinogen
Group C—Possible Human Carcinogen
Group D~Not classified as to human carcinogenicity
Group E—Evidence of noncarcinogenicity to humans
Using data available from human and animal studies, EPA developed a way to categorize
substances as to their potential human carcinogenicity, as shown below:
Categorization of tha Walght of EvManca of CarclnoganlcKy
Baaad on Animal and Human Data
«.
.'... ' /'!••,
• • - jw g^^^ JT'.T'T* J
nif'O "—''•
".. .J ^ mm m •
A
• !-.«. J
Sufficient
United
inadequate
No data
No evidence
A
B1
82
B2
B2
A
81
C
C
C
BtfcanA Ciafaa^aaM
rHuH CwlDHal
A
B1
D
D
0
•r
Ite OMB
A
B1
0
0
0
HO
I.VIIMIH-V |
A
81
0
E
E
-------
Again, it should be pointed out that the assignment of a chemical to a particular category is often
an uncertain one and the assessor should be prepared to explain why he/she made the
determination.
DOSE- RESPONSE
The same experiments that are used to determine whether or not a particular substance
causes a deleterious effect are used to attempt to define a relationship between the size of the
dose to which individuals are exposed and the likely response (the dose-response relationship).
Using the animal studies to estimate this relationship in humans poses two major analytical
problems: 1) the need to extrapolate from the very large doses given to animals in order to elicit
statistically significant responses in a short period of time and 2}1o extrapolate from the animal
responses to likely human responses. The issue is further complicated by different assumptions
that are applied to chemicals that cause cancer (carcinogens) and those that cause other toxic
effects (toxicants).
While EPA assumes that there is no threshold for carcinogens, there are researchers who
disagree and contend that carcinogenicity will not be initiated within an individual until a
minimum threshold of exposure is exceeded. EPA believes that its approach is more protective of
public health and therefore the Agency feels comfortable applying the no threshold concept to
carcinogens. With regard to toxicants, however, EPA does believe that there are threshold
exposures that have to be exceeded before any effect can be seen. The distinction for the two
approaches stems from a difference in the way EPA thinks that the two types of chemicals
behave in causing cancer or a different toxic effect.
Carcinogens
EPA assumes that for potential carcinogens, there is no dose that does not elicit a
response. In other words, exposure to carcinogens has no safe threshold below which there is no
increase in the risk of cancer. This does not mean that any exposure to a carcinogen will cause
cancer, only that any exposure is presumed to cause an increase in the probability of contracting
cancer. This is graphically displayed in Figure 3 which illustrates the estimated dose response
curve for two different carcinogens. The graph results from the application of a model to the data
points resulting from the animal studies. Data points from these studies occur in the high dose
range and EPA then uses a model to extrapolate down to the very low doses to which humans are
usually exposed.
The model chosen by EPA predicts a plausible upper bound limit (95% confidence level)
on human risk. The actual risk faced is likely to be less than that predicted and may even be
zero. As seen in Figure 4, choosing a different model could elicit very different results.
-------
Figure 3
DOSE-RESPONSE ASSESSMENT
CARCINOGENS
Response
(unit cancer risk)
Linearized Multistage Model
(95% upper confidence lirnii)
Dose
(mg/kg/day)
Models like (he Linearized Multistage Model transform high dose data into low dose estimates
-------
Figure 4
DOSE-RESPONSE CURVE
R
e
s
P
o
n
s
e
Observable
Range
Range of
Inference
Dose
-------
The slope of the line resulting from application of the model is referred to as the unit
cancer risk, an estimate of the lifetime risk for each unit of exposure in the low-exposure range.
Several methods have been developed to extrapolate from animal to human exposures.
These methods, referred to as scaling factors, assume that animals and humans are at equal risk at
the same exposure measured as a factor of body weight (milligrams of carcinogen per kilogram
of body weight per day) or of surface area (milligrams of carcinogen per square meter of body
surface area per day).
Non-carcinogenic Toxic Effects
The threshold concept is illustrated in Figure 5 which shc^vs a dose, known as the No
Adverse Effect Level (NOAEL), below which no response (increase in the risk of the toxic
effect) was observed in animal studies, based on the equivalent animal dose. To ensure that it is
regulating at a protective level. EPA applies some uncertainty factors (based on the quality of
data used in determining the NOAEL) to reduce the allowable dosage to a level below the
NOAEL. This protective level is called a Reference Dose or RfD.
EXPOSURE ASSESSMENT
Assessment of human exposure requires an estimation of the number of people exposed
and the magnitude and duration of the exposure. Information on the quantities of the chemical
that are released, the media to which it is released, and factors controlling human exposure to the
chemical and uptake of it are necessary components of an exposure assessment. Ambient
monitoring data are also very valuable but often are not available so the assessor is forced to rely
on modeled estimates of pollutant concentrations in the environment and human uptake. The
exposure assessment should include a number of scenarios including that for the population as a
whole, for high risk individuals, and other important subgroups.
RISK CHARACTERIZATION
In the risk characterization, results from the hazard identification, dose-response, and
exposure assessments are integrated into an assessment of the human health risk. With
carcinogens, for example, the exposure value is multiplied by the unit cancer risk, as described
above, to determine the risk. With non-carcinogenic toxicants the human exposure (as either a
dose or concentration) is then compared to the reference dose (or concentration).
The risk characterization should include a discussion of the uncertainties associated with
the final risk conclusion and the steps leading up to it as well as a discussion of the strength of
evidence in support of the conclusion. For example, were the exposure estimates based on
monitored or modeled results and what was the strength of evidence supporting the hazard
identification? The characterization should also include information on the risks from
8
-------
Figure 5
DOSE-RESPONSE ASSESSMENT
NONCARCINOGENS
Response
RID
NOAEL
Dose
(mg/kg/day)
A series of decreasing doses finally elicits no adverse effects. A safely
margin is applied to this adverse effects level.
UF=Uncertainty Factor
-------
the range of exposures investigated, including that of the population as a whole and important
subgroups. The key feature of the risk characterization is to relate all the relevant information
about the assessment for use by decision makers in a risk management decision as part of a
comparative risk analysis, or for some other purpose.
10
-------
SCHEDULE: EPA ENVIRONMENTAL RISK AND DECISION MAKING COURSE
US EPA Office of Inspector General
March 20-21,2001
Day 1
8:30 - 9:15 Introduction to Course
and Introduction for Participants
9:15-10:30 Principles of Risk Assessment
Problem Formulation
Hazard Identification and Dose Response
Uncertainty
10:30-10:45 Break
10:45 - 12:00 Principles of Risk Assessment (Continued)
Exposure and Risk Characterization
Characterization of non-Health Impacts
12:00-1:00 Lunch
1:00- 1:30 . Overview of Case Study
1:30- 3:00 Breakout Groups - facilitated risk assessment case study exercise
Problem Identification and Data Needs
Hazard Identification and Dose Response
3:00-3:15 Break
3:15 - 4:45 Breakout Groups - facilitated risk assessment case study exercise
. Exposure and Risk Characterization
Non-Health Impacts
4:45 - 5:00 Summary Report-Out by Groups on Risk Assessment Outcomes
***
Day 2
9:00- 9:45 Introduction to Decision Making Exercise
Economics/ Option Development
9:45 - 11:00 Breakout Groups - facilitated case study exercise
-------
Option Identification and Data Needs
Costs and Benefits
11:00 - 12:00 Breakout Groups - case study decision/ policy development
12:00 - 1:00 Working Lunch -Breakout Groups - case study decision/ policy development
(continued)
1:00 - 2:00 Breakout Group Exercise — prepare final decisions for report-out
2:00 - 3:00 Breakout Group Presentation of Case Study Risk Assessment
Risks
Uncertainties
3:00 - 4:00 Breakout Group Presentations -- Case Study Decisions/Policy
Options, Process and Decisions
4:00 - 5:00 General Discussion -- Risk and Decision Making
Case Study
Application of Principles to Current Issues?
Course Wrap up and Evaluation
-------
OFFICE OF INSPECTOR GENERAL
Office of Mission Systems - Human Resources Divsion
EVALUATION
COURSE TITLE: Environmental Risk & Decision ftfaTHrtq DATE; Marr^h 20-21. 2000
NAME: (Optional) OFFICE:
INSTRUCTOR:
Please take a few minutes to provide us with your views on this training. It
will assist us in measuring the effectiveness of . the training and making
•i n« >r» imamai^fcg , manBPT.TIiJB yCP"" TftHXUiSeS anH OCM^^^T ^^erro Jnoropr Jate :
1. The length of the training was appropriate.
a. Strongly Agree b. Agree c. Disagree d. Strongly Disagree
2 . The training will be useful in my work.
a. Strongly Agree b. Agree c. Disagree d. Strongly Disagree
3. The training was well organized and easy to follow.
a. Strongly Agree b. Agree c. Disagree d. Strongly Disagree
4. The objectives were accomplished.
a. Strongly Agree b. Agree c. Disagree d. Strongly Disagree
5. The instructor/facilitator was well prepared.
a. Strongly Agree b. Agree c. Disagree d. Strongly Disagree
6. The instructor/facilitator answered and explained the participants'
questions adequately.
a. Strongly Agree b. Agree c. Disagree d. Strongly Disagree
-------
7. Which parts of the training did you find most beneficial?
8. Which parts of the training did you find least beneficial?
9. List additional training topics you would like presented.
10. List other suggestions for improvements.
Thank you.
Dr. Dana Ashley Sharon
OI6 Office of Mission Systems
Human Resources Division
Room 3304 - VC 2241 - sharon.dana@epa.gov
202-260-6271
------- |