Lay Views on Uncertainty in Health Risk Assessment:
A Report on Phase II Research
Branden B. Johnson
New Jersey Department of Environmental Protection
• - , x
Paul Slovic.
Decision Research :
Eugene, Oregon .
Final Report
Prepared for:
U.S. Environmental Protection Agency
Office of Policy, Planning, and Evaluation
Risk Communication Project
Cooperative Agreement No. CR-820522
Lynn Desautels, EPA Project Officer
August 5, 1996
-------
-------
Contents
Executive Summary 1
Background 3
Method 6
interviews • • 7
Focus Groups , v ' 8
Open-Ended Questionnaires' . ' 9
Closed-Ended Questionnaire - >/ 10
Results 13
General Responses to Environment^ Health Risk Uncertainty ' 13
Responses to Range Bounds , .. . '. l -17
Reasons for Uncertainty. ' • 22
Uncertainty and Risk Management - . 3'5
Understanding of Science and Risk Assessment 38
Potentially Mediating Factors '. " ,42
Multivariate Analyses - • , 47
Importance of the Independent Variables •' . 56
Conclusions 58
Practical Implications 64
Research Implications 68
References . 70
Appendix I: Fourteen Items Used in Qualitative Data
Collection ,
Appendix II: Presenting Uncertainty In Health Risk
Assessment: Initial Studies Of Its Effects On, Risk
Perception And Trust
-------
Acknowledgements
This research was funded by Cooperative Agreement No. CR-820522 with the U.S.
Environmental Protection Agency. We would like to thank Steve Johnson for conducting the
focus groups, Donald MacGregor for consulting on the layout of the closed-ended questionnaire,
and C.K. Mertz for statistical analysis. Drs. Alan Stern and Robert Hazen of the New Jersey
Department of Environmental Protection were consulted on the technical accuracy of some
interview stimuli. The -authors, however, take full responsibility for the data collection materials
used. The views herein do not necessarily represent the views of either the U.S. Environmental
Protection Agency or the New Jersey Department of Environmental Protection.
-------
Lay Views on Uncertainty in Health Risk Assessment •- "'•''.. ;page {'
Executive Summary
It will be a challenge to present uncertainty iri environmental health risk estimates to the
public in ways that inform, rather than outrage,- this important audience. Although such
presentations may increase citizens' risk knowledge and trust in the honesty and competence of
institutions providing risk estimates, careless communication could have undesirable results.
In Phase I of this research, people had no response to certain experimental presentations of a
range ofrisk estimates; in other experiments, two-thirds of the sample saw the range estimates as
a signal of government honesty and competence. We needed mo^e study of how laypeople
thought about uncertainty in risk assessment, and its implications for risk management, to
untangle these mixed findings, • '
Phase ,11 research included intensive interviews and open-ended ^questionnaires in New
Jersey and Pennsylvania, focus groups in Oregon, and a long closed-ended questionnaire given to
280 Eugene, Oregon, residents (largely college students). We presented uncertainty in the form
of a range of risk estimates, primarily in a hypothetical case of a chemical in drinking water.
The following responses appeared among our participants (Eugene sample, unless'otherwise
specified): . .- •
• About 45% doubted the honesty of the government in presenting ranges ofrisk estimates,
''a higher proportion than in the earlier research. '
'• About a third disagreed that government was less competent for discussing ranges ofrisk
estimates. , . ' •: ;
.' • On average, people supported presentation of uncertainty information. A third of our
sample strongly desired certainty, rejecting risk ranges or quantitative risk estimates of
any kind. • .
-*• , ' ' - - '
' ' • Zero as a lower bound for a range of risk estimates was seen on average as an indicator of
' either incompetence or dishonesty, whereas a very small positive lower bound (e.g:, 1 in
10 million) had much less effect, even though a majority saw no difference between zero
and a 1 in 10 million lower bound. Those who did see a difference seemed to doubt that
the government would discuss risks if they were truly zero.
-------
Lay Vie\vs on Uncertainty in Health Risk Assessment . • page 2
• On average, people tended to see the high end of a risk range as being more likely, even
with a lower risk being labeled as "most likely."
• Some, but not all, interviewees and focus group members volunteered reasons for
uncertainty, such as differences in exposure and personal susceptibility. However,
explanations of uncertainty as due to extrapolation from high-dose data (e.g., from
workers, or due to an accident) or from animal experiments confused, irritated, or
disturbed them. Eugene responses to animal and high-dose extrapolation explanations
were no different; an overwhelming-majority agreed with scientist-offered reasons for.
uncertainty, but this seemed to be a rationalization of distrust rather than true agreement.
• People said they would doubt a lower risk estimate from a second study more than they
would a higher risk estimate, but most felt more confident about the safety of drinking
water after reading a scenario about a lower risk estimate.
• When faced with hypothetically overlapping risk estimates from government and
environmentalists, people did not observe the overlap or interpret it as meaning anything;
however, in the abstract they said agreement among these two groups (or of academics
with government) would make them suspect that the environmentalists arid academics,
had been corrupted.
• People in interviews and focus groups did not see environmental health uncertainties as
having any relationship to financial, recreational, or transportation (mortality)
uncertainties.
• Trust in officials was the most common predictor of people's reactions to government
risk ranges. •, . . ,.
• Although a substantial minority of our sample had difficulties with numbers,
mathematical literacy played a very small role in their reaction to risk uncertainties.
-------
Lay Views on Uncertainty in-Health R'isk'Assessment '" • page j
Background
; Several government, industry, and academic commentators have urged communication of
uncertainties in environmental health risk estimates to public and other, lay audiences, as we
noted in our final report for Phase I (Johnson & Sloyic, 1994a) and a subsequent published
article (Johnson & Slovic, 1995). Such arguments continue to be made. Communication of
uncertainties is needed, it is said, to avoid conveying information that "is incomplete and may be
misleading" (Carpenter, 1995, p. 127). Narrative statements of "statistical reliability" should be
made even if it "is not easy for scientists to make" them (Carpenter, 1995, p. 132). A recent
National Research Council report (1994) advised U.S. Environmental Protection Agency (EPA)
to communicate exposure or variability information in a three^part format: the estimated risk, the
level of confidence that the risk is no greater than this estimate, and"the subpopulation to which
the estimate applies. Administrator Browner's memorandum on EPA's Risk Characterization
Program (1995) said that "a balanced discussion of reasonable conclusions and related
uncertainties enhances, rather than detracts, from the overall credibility of each assessment."
Since our earlier report, we have found a few dissenting voices on the benefits of conveying
uncertainty. Arguing that discussing uncertainly in risk estimates would give homeowners an
excuse to, avoid testing for natural radon in their homes, academic researchers, state and federal
officials", and nonprofit groups have urged that radon communications present "an unambiguous
'united front'" (Garrison, 1991; USEPA, 1992, ch. 6, p. 6; Weinstein, Sandman, & Roberts,
1989, p. 18). We cannot definitively refute the argument that communicating uncertainty about
radon risks would have made the testing rate worse. However, with about 5% of U.S. homes
having been tested, the unambiguous and forceful U.S. media campaign does not appear to have
had much better results than lower-key campaigns in Sweden (4%) and Finland (2%) (Cole,
1993, pp. 72, 179, 181-182). Further, we used a radon analogue ("zydin") in a Phase I ' . .
experiment on uncertainty. Zydin elicited significantly lower ratings of risk and worry than a
Superfund site chemical at equal levels of estimated cancer risk. Yet intentions to get the
problem solved did not vary, by either hazard or the level of uncertainty in the risk estimate. If
these results can be generalized, they imply that communicating uncertainty in radon risk
estimates would not have depressed radon testing frequency any further. .
-------
Ltiy Vieva on Uncertainty in Health Risk Assessment ,. • page 4
Various other criticisms of communicating uncertainty have been made. The President's
Commission on Risk Assessment and Risk Management sent a letter to House leaders December
14, 1994, that raised concerns about H.R. 9, the Job Creation and Wage Enhancement Act.
Among these concerns was that "providing for 'best estimates' of risk along with plausible
upper- and lower-bounds on those estimates may not be much improvement on current methods"
(Riskpolicy report, 1995, pp. 12-13). A prominent risk assessor (and member of the President's
Commission) independently assailed mandated communication of risk uncertainty, arguing that a
range of risk estimates might be misinterpreted as implying equal likelihood of all estimates,
calculating such ranges in a consensual way would be very difficult and time-consuming, and no
other policy-related numbers—including cost-benefit analyses being considered for a mandate
under the same legislation mandating risk characterization, as well as other economic estimates.
(e.g., of gross domestic product or health care reform costs)—are expected to come with
uncertainty estimates (Goldstein, 1995, pp. 1602-1603,1609). In less explicit language,
MacGregor, Slovic, and Morgan (1994, p. 827) suggested that "even careful presentation of the
current 'complicated' ... state of affairs [concerning scientific knowledge about electromagnetic
fields] will increase people's concerns."
This continuing debate triggered our initial studies, and led us to conclude that more
research on public response to uncertainty should be conducted. The findings of our earlier
research (Johnson & Slovic, 1994a, 1995) were that:
• People are unfamiliar with the notion of uncertainty in risk assessment, and with
uncertainty in science generally.
• People may recognize uncertainty (i.e., a range of risk estimates) when it is presented
simply.
• Graphical presentation produced mixed results in communicating uncertainty, making a
, range, ofestimates more obvious, but causing the information to seem less trustworthy.
• People's views on the environmental cases presented in our experimental stories may
have been influenced less by variations in uncertainty than by personal attitudes toward
risk, government and authority.
• Agency discussion of uncertainty in risk estimates appears to signal agency honesty.
-------
Lay Views on Uncertainty in Health Risk Assessment . ' page 5
• Agency discussion of uncertainty in risk estimates may be a signal for some people that
the agency is incompetent. ,
• Low risk levels were deemed to be "preliminary" estimates by laypeople, possibly
' s ' ;'-"-•
implying distrust. • ..
Our Phase I studies used several experiments to see how people would respond to
uncertainty in the form of a range of risk estimates: two hazards (natural radiation and a chemical
in drinking water from a hazardous waste site), various upper- and lower-bounds, and graphics.
Two focus groups also directly compared their reactions to point and range estimates. In pur
conclusions, we pointed out that other presentations of uncertainty may have' stronger, or
different, effects on perceived risk, honesty, and competence. For example, only one source of
uncertainty (the difficulty of extrapolating from animal .data) was described in one of our
experiments; more focus on explanation might affect lay responses.
However, we concluded that it would be premature to test such variations without a better
understanding of how citizens conceive of uncertainty in environmental health risk assessment.
Rather than employing experiments, where one infers thoughts from behaviors, our present goal
was to begin with qualitative methods (interviews, open-ended questionnaires, focus groups) to
-- . ' • : 'i i ,'-•-.,. - ' - „ ' -
see the world from the lay participant's viewpoint. Once we can describe how.citizens might
interpret a description of uncertainty, we might be able to suggest useful contextual and focal.
information for use by organizations wishing to convey uncertainty in health risk assessment to
their audiences.
-------
Lay Views on Uncertainty in Health Risk Assessment ' page 6
Method
Our earlier findings suggested that uncertainty in risk assessment was a novel concept for
laypeople. This implied that the approach to revealing "mental models" of,risk-related topics
pursued by Bostrom, Fischhoff, and Morgan (1992) would not work. In that approach one asks
people to "tell me" about the topic (e.g., radon) and uses follow-up questions to elicit details of
the person's cognitions that aren't volunteered initially. Asking people to tell us about
"uncertainty in risk assessment" did not seem likely to be fruitful.
We, therefore, provided stimulus items that people could read and react to (see Appendix I
for 14 examples). These items used one to six sentences to briefly introduce a range of risk
estimates, or to explain how risk assessors identify a "safe" level for human exposure to a
noncarcinogen. In contrast to the simulated news stories of our earlier work,! these items
included only information that identified the hazard and described 'or explained uncertainty as a
range of risk estimates. Once a stimulus was read, we began by asking "what thoughts,
questions, and so forth, come to mind when you read this?" Only after allowing this open-ended
response, and asking (in interviews) for more details about this response, were more focused
questions asked. The intent was to minimize biasing responses through interviewer behavior "or
survey construction. .
We collected information through interviews, focus groups, open-ended questionnaires, and
two versions of a closed-ended questionnaire. All stimulus items used in focus groups and
questionnaires dealt with a government announcement about a chemical in the person's drinking
water (whether tap or bottled water). Another seven items used only in interviews probed
reactions when the same risk information concerned contaminated soil or air in people's
neighborhood, or automobile, or airplane crashes.
1 Experiments in Phase I research (Johnson & Slovic, 1994a, 1995—Appendix has copy of latter) used simulated
news stories about environmental problems, with headlines, quotations from government spokespersons and local
citizens, and so forth. The content of these stories varied. The problem'might be chemicals from an abandoned
waste site or natural radiation; a point estimate or a range of estimates of risk might appear; and so on. Each
participant read one of these stories. By comparing their responses across different stories, we could see whether
hazard type, mention of uncertainty, and so forth, made a difference.
-------
Lay Views on Uncertainty in Health Risk Assessment . page 7
Interviews j
Thirteen people were interviewed in twelve interviews (one interview included both
members of a cpuple, at their request) in January 1995. The median interview took 45 minutes
(range 30-120 minutes) and involved responses to seven items (range 4-11). Nine interviewees
were women; four interviewees were in their 20s, five in their 30s, and the rest up to age 79. Two
interviewees were black, with the rest white; five were single, two widowed, and the rest
married. Four interviews were conducted with nonscientist employees of the New Jersey
Department of Environmental Protection'in Trenton, the rest with'residents of an apartment
complex in a Philadelphia suburb. Interviewees' occupations included personnel administrator,
secretary, paralegal, florist, nurse, university administrator, home health care administrator,
dentist, entertainer, and risk communication specialist; two retirees had been in business and
engineering. ' / ' ' ' :
A brief introduction included a request to read and answer each stimulus item as if it
concerned the water that people actually drank, including bottled water for the two people who
relied on it exclusively (see Exhibit 1 for examples; others appear in Appendix I). Each
interviewee was then handed a sheet of paper on which the initial item was printed and asked to
read it and respond with any thoughts, questions, or comments that occurred. Probes were used to
elicit further comments, or to clarify earlier comments. When the interviewee indicated that he or
she had no more thoughts to offer on an item, that sheet of paper was retrieved and the next one
proffered. At the end of 30 minutes—the time span promised when people were first contacted—
, \ * • . . - • '
the interviewer asked interviewees whether they wished to continue. That only .two people chose
to stop at this point, and that four took an hour or more, suggests that the topic—despite
repetitiveness in the stimuli—captured people's attention. '_
f' ' ' ' • . ' ...
' No interviewee read every stimulus item. The reasons for this were that there were too many
items to be covered in even the longest interview, and there was no need to ask the same
' i - ' • " ' , ' • . , . ~
questions if, after several interviews, no new responses were being heard. Thus each item was
presented to a median of five interviewees (the range was 1-7, out of 12 interviews). Overall,
there was no attempt to present items in the' same order, because this (given time constraints)
would have prevented some items from ever being presented. ,
-------
Lay Views on Uncertainty in Health Risk Assessment
EXHIBIT 1. Examples of Interview Stimulus Items •
Item 2a
The government announces that the water you drink contains a level of
chemical X that poses an extra health risk of getting nonfatal kidney damage
of 1 in 1,000,000 (one in a million) over a lifetime of drinking that water.
Item 11 a .
The government has found a chemical in your drinking water that can cause
nonfatal kidney damage in laboratory rats. The level of the chemical which
the government considers safe for humans to be exposed to is 1 part per
100 million. The level of the chemical in your drinking water is smaller than 1
part per 100 million.
Focus Groups
Two focus groups—each comprised of three women and one man—were conducted on
February 21 and March 7,1995, in Eugene, Oregon. The first group included employees and
volunteers of United Way of Lane County, while the second comprised clients and staffers of a
job retraining program jointly run by the county and private industry, The aim was to reduce the
educational level of members, relative to those in our earlier focus group research. Interview
results were used to design the focus group protocol.
Each member of the focus groups read three items on risk uncertainty (see Exhibit'2), and
answered a questionnaire about them, before coming to the focus group session. This approach
allowed us to'tap their individual views as'well as those they would express in a group
environment; overall these views did not differ from those in interviews or (for Item 3 only)
open-ended questionnaires. Besides reviewing these responses, issues raised by the focus group
facilitator included (a) the effect of zero versus positive lower bounds; (b) use of similar or
different denominators in the probabilities presented; (c) extrapolation from animal data; (d)
extrapolation from high-dose human data; (e) exposure to average concentrations below-standard
versus slightly above-standard; (f) explanation of Reference Concentration calculations;. (g)
i ' . * . - -
reaction to a graphic (see Figure 1) on confidence bounds in excess risk relative to varying
' l
exposure; and (h) how reaction to uncertainty in environmental risks compares to uncertainty in
financial or safety risks.
-------
Lay, Views, on Uncertainty in Health Risk Assessment _ page 9
, . " \ ••'•.•' • • • • .
EXHIBIT 2. Focus Group Questionnaire Items • . ' , , , .
items " ../• .'. ••;.., '.".".•,"•.'..'.•''•' •• '' ;. .''"'' -:..-''.•
The government announces that the water you drink contains a levelof. ,
chemicalX- that poses an extra health risk of getting cancer in 1 in 1,000,000
(one in one million) over a lifetime of drinking that water. The government
says that is the most likely risk, but it says the true risk could be as low as
'zero or as high as tin 100,000.
Item 5 . . '; ..'.,.
The government arinounces that there is a 5% chance that the extra level of
risk from drinking chemical X in the water you drink for your entire lifetime is
above 1 in 100,000 (one in one hundred thousand), and a 5%'chance that ' . , .
the risk is zero. The government says this means there is a 90% chance that
the true risk is between zero and tin 100,000. "•'..-
Item 8 , . . ,
The government announces that its scientists have calculated the risk of
getting cancer from drinking the water you drink, which has a small amount
of chemical X in it, for an entire lifetime. It is 95% certain that at least 90% of . ,
the population has an extra risk of no more than 1 in 1,000,000 (one in one'~
million). A small proportion of people who are more likely than others, when
exposed to a'cause of cancer, to get cancer may have a risk as high as 10 ' '
in 1,000,000 (ten in one million) if they drink this water for their entire lives. .
Open-Ended Questionnaires
" - A wider variety of items (12, rather than 3), but similar to those given to focus group ;
members, were distributed to volunteers from the jury pool for Essex County, New Jersey. Four
versions of the. questionnaire, each including three unique questions, were created; 20 copies of
each version were distributed. Of the 80 questionnaires distributed, only 29 were returned, and
only 18 included substantive responses. The "nonsubstantive" responses included answering
only the sociodemographic questions, providing one or two answers and then apparently losing
interest, and providing answers whose content (e.g., "none," "good thing") seemed to indicate a
superficial or irrelevant response to the questions.
Most respondents (24 of 29) gave enough sociodemographic information to compare those
who gave substantive responses to those who did not. Overall, men and women were equally
numerous (11 to 13), although women dominated in the substantive group (10 to 6). Median age
was nearly identical (45 and 44) and ranged from 20 to 67. The sample of 17 whites,-4 blacks,
and 1 Hispanic was equally divided among substantive and peripheral responses (79% vs. 75%
white). The main difference between the two groups was education: 75% of substantive
respondents had a college degree or better, while five of the peripheral respondents had a high
-------
Lay Views on Uncertainty in Health Risk Assessment
page !-0
.J2
tn
§
1 in 100
1 in 1,000
1 in 10,000
1 in 100,000
1 in 10 million
1 in 100 million
50% ,
Half of the population has higher
exposure, but half has lower exposure
Percent of exposure to risk
, U (95%certain)
B
L (5% certain)
39.9%
90% 99%
We are 90% certain (95% minus 5%) that the risk to a person with
high (90%) exposure is between 1 in 1 million and tin 333.
Figure 1. Graphic presenting variability in exposure, shown to focus groups; adapted from NRC
(1994, pp. 10-25).
school diploma or less, and the other two members in this category reported their educational
background as "some college." .
Closed-Ended Questionnaire
Results from the preceding data collection were used to develop a questionnaire that could
give'us a broad view of lay beliefs and attitudes about uncertainty in environmental health risk
estimates. Material was added from focus groups in our earlier research, scientific literature on
uncertainty,'and hypotheses stimulated by our earlier research. Statements were also drawn from
the literature and elsewhere that might measure some potentially mediating variables (e.g.,
attitudes to uncertainty in daily life, ideology, behavior regarding drinking water). No attempt
-------
, Lay Views on Uncertainty in Health Risk Assessment • ' ' ' ' • page
was made to be comprehensive or rigorous with such mediating variables, because our primary
goal was to understand lay concepts about uncertainty in environmental health risk assessment,
'not to explain differences in these concepts. Most statements were designed with a four-step,
strongly disagree-strongly agree Likert response format. The sequence of items in thfe closed-
ended questionnaire included the following: ,
• mediating factors ' ,
,• questions about their drinking,water
• a scenario on a range of drinking water risk estimates and an explanation for the range
(extrapolation from either animal or high-dose data) .,
. v responses to the scenario
• attitudes toward receiving no risk estimate (e.g., being told they are safe or unsafe), a
point estimate, or a range of estimates .
/ ' ' • ',-"" ' ' ' • '
• miscellaneous uncertainty statements , ' ' .
•, a scenario on two estimates of risk over time, and responses to'it . , "
• sociodemographic information ,
After pretesting, the questionnaire was answered in late November of 1995 by 280
• •_-,-. . . "' . I • ' ' ,
, respondents to an advertisement in the University of Oregon newspaper; This group was 50.7%
male, with a median age of 20 (range 17-36). Three-quarters (74.6%) had some college, with
14.7% having a bachelor's degree or better. Most .(79.3%) were white, with 8.6% labeling
themselves as Asians or Pacific Islanders, 2.9% as Native American, and 1.4% as black. Only
2,5% were of Hispanic origin, although 5.4% did not answer this question. About 14% had one
or .more children living at home, and 13% were affiliated with an environmental group. Split
samples read animal (49.6%) or high-dose (50.4%) extrapolation explanations of the scenarios'
range of health risk estimates. For 127 questions these two groups (readers of animal or high-
dose, scenariosjliad only three significant differences (p< .05). Given that this small proportion
of across-scenario differences could have arisen by chance, and occurred only once for each of
three different topics, these differences are probably not reliable. Thus we combined all 280
' >
responses for analysis.
-------
Lay Views on Uncertainty in Health Risk Assessment page (1
Reviewers of our earlier research suggested that use of mostly student respondents in
Eugene, Oregon, limited generalization of our results. Phase II interviews and open-ended
questionnaires used a more diverse, New Jersey/Pennsylvania population, and focus group
members who were less educated than in earlier research, but these were few in number. Non-
Eugene, nonstudent responses did not differ qualitatively from those of Eugene students. It is
unlikely a less educated audience would grapple more easily with the concept of uncertainty in '
risk estimates or science, or have a more positive reaction to it, although generalization from
responses to the closed-ended questionnaire should be done cautiously. People with lessxthan a
college education were very unlikely to answer our open-ended questionnaire; whether due to
incomprehension or disinterest in the topic, or less socialization to answer questionnaires, this
may imply that we would fail to get adequate answers from a more diverse group. However, a
much shorter instrument based on our closed-ended questionnaire might be used with a more
I
diverse population (like our New Jersey jury pool members) to test these hypotheses.
-------
Lay Views on Uncertainty in Health Risk Assessment > . •. , page 13
Results
In the discussion below we group qualitative and quantitative (closed-ended questionnaire)
results by topic. Lay-proffered concepts are distinguished from those derived from, the literature
on uncertainty and earlier hypotheses of ours. In contrast to sketchy evidence (from this study
and others) that citizens have a clear (if incomplete) concept of environmental problems and
government management of those problems^ they seem to lack a structured view of uncertainty
about risk estimates. We thus will not offer a detailed lay "mental model" of uncertainty like
that suggested for radon (Bostrom et al., 1992). , '•
Results are, discussed in the following order, to facilitate understanding: general responses
to environmental health risk uncertainly; responses to range bounds; reasons for uncertainty;
uncertainty, and risk management; understanding of science and risk assessment; potentially. .
mediating factors; and multivariate analysis. ,
General Responses to Environmental Health Risk
Uncertainty
Here we discuss responses to the relation of health risk uncertainty to other kinds of'risk;
willingness to see risk ranges; and reactions to a drinking water scenario involving a range of risk
estimates. •. , ; • . : ; , '
Uncertainties in Environmental Health Risks Seen as
Dissimilar to Uncertainties for Other Risks
It has been suggested that the value for improved decision making of assessing risk
uncertainty can be demonstrated with the example of investment uncertainties (Finkel, 1994).
The author's goal seemed as much to educate risk policymakers as citizens, and he did not make
explicit the implication that investment and health risks (and thus their uncertainties) are similar,
at least in their value for decision making. We thought a brief (because this was not our central
goal) .exploration in interviews and focus groups of whether citizens see such analogies as valid
might show'whether they could be used to educate people about uncertainty in health risk
assessment; , •
-------
Lay Fiews on Uncertainty in Health Risk Assessment - . page 14
First, we examined in interviews whether reaction to uncertainty was.common across
environmental health risks. Some interviewees were asked whether their reactions to the risk
estimate ranges would change if the hazard was not a chemical in drinking water, but instead a
chemical in neighborhood soil or in the local air from a nearby factory. Some people felt water
offered more personal control, and thus less personal worry, because there were more feasible
protective actions (e.g., switch to bottled water vs. move out of the neighborhood). Others said .
that one could move away from polluted soil, while water was needed for survival. The air-,
factory scenario elicited particular outrage, as implying immoral behavior on the part of business.
These immediate reactions seemed to be based on factors unrelated to uncertainty, suggesting
that such variations might occur across environmental hazards regardless of whether point or
range estimates of risk were used. /
Second, parallels between responses to uncertainty in environmental and nonenvironmental
risks were rejected by both interviewees and focus group members. A comparison of the drinking
water scenario with the same range of risk estimates for airplane crashes was dismissed by
interviewees. Their grounds were that air travel was occasional, familiar, and (if frequent) a
probable requirement of one's job, while drinking contaminated water was unfamiliar and a
matter of daily exposure; similar responses occurred for uncertainty in automobile risks. Focus
group members said recreational risks were chosen,(on the basis of joy garnered from the
activity), and financial risks (in investing or job security) were not irrevocable, whereas'
environmental health risks were imposed by others and death permanent. Thus uncertain
recreational and financial risks were acceptable, usually without formal or (allegedly) conscious
weighing of risks and benefits; uncertainty in environmental health was not. These'reactions
suggest, but do not prove, that using a stock market investment example to show the superiority
of choices made with understanding of uncertainty (Finkel, 1994) may not work with lay
audiences facing^incertain environmental risks and estimates.
i . "<-•-'
Some Willingness to Receive Risk Estimate Ranges -
Interviews yielded several statements that people would prefer only a single risk estimate
rather than a range. For example, two people offered diametrically opposed views: "Government
should announce its preliminary calculations for an environmental health risk, no matter how
-------
Lay Views on Uncertainty in Health Risk Assessment ' . • ' - page .'3
uncertain these numbers are" and "Government should never announce a risk number about •
anything until they know for sure exactly how risky it is." We included 13 such statements (six
from interviews, the rest from the literature) about preferences for single risk numbers or for no
numbers at all in our closed-ended questionnaire. Table 1 shows those questionnaire statements
that loaded high on a single dimension in factor analysis, and thus were used to produce a scale
for multivariate analysis (a = .80). As can be seen, our hypothesis that aversion to numbers (of
any kind) and aversion'to ranges of risk estimates would be distinct concepts was rejected.
, Further, a "desire for certainty" was not dominant in this sample (Weinstein, 1987); about 35%
rated at or above the middle score on this index. Strong desire for certainty in this minority may
explain the slightly positive net response to statement Id, because otherwise this sample seemed
distrustful of experts and officials (see below). '
TABLE 1. Desire for Certainty , . ;•.'''.. __.
t Disagree Agree Don't know
1 a I prefer being told that a situation is safe or unsafe, rather than 53.2% , 42.1% 4.6%
hearing>risk numbers, such as "a health risk of getting cancer of 1 ,
in 1,000,000," , ' -
1b If the government is having difficulty in. determining how much of 18.9 74.3 6.8
a risk an environmental condition poses to me, I prefer they give
me a range of numbers rather than give me a single risk number.' - ' , • .-
(R) ' ' '.. • '. '-..., - ' .'.-..-- :. ':
1c When a chemical is discovered in my drinking water, I don't want 47.1 48.9 , 3.9
to hear statistics, I just want to know if my water is safe. ,
1d I am more comfortable with an expert's opinion about whether or 42.9 53.2 3.9
not my water is safe than with a range of risk numbers from which '
I must draw my own conclusions.* ,
1 e I would prefer that government tell me that they're just not sure 51.4 43.9 " 4.6
about the size of an environmental health risk, if that's the case, . -
rather than giving me a range of risk numbers.*
1f I'd prefer a single, concrete risk number rather than a range of 61.4 30.7 7.9
numbers for the environmental health risks I face. ,
Note. * Item (quotation or paraphrase) derived from Phase II interview; data from Eugene, Oregon, questionnaire.
-. - (R) Reversed scoring for construction of CERTAIN index. Cronbach's alpha = .80.
Mixed Reactions to the Uncertain Risks Scenario
Respondents to the closed-ended questionnaire were asked several questions after reading
one of the initial scenarios (see figures 2 and 3), with a caution to assume that '"the term 'water'.
-------
qt Views on Uncertainty in Health Risk Assessment
page 16
refers to the water you drink, whether it is bottled or tap water" (during interviews bottled water
drinkers tended to assume "pollution" scenarios only concerned tap water). Table 2 shows the
answers, including indices created from statements loading high on the same dimensions.
Responses to items in the NO WORRY index (items 2h, 2i, and 2j) indicates that people felt the
risk was low and they would continue to drink the water. However, the CONCERN index (items
2c and 2d) showed that they were concerned about the effects of drinking this water and
committed to getting it cleaned up.
Table 2 also,suggests mixed reaction to the government agency in the scenario
(GOVHONEST index, items 2e, 2f, and 2g). Nearly a third of the respondents could not decide
whether the government was honest, and a plurality doubted it was "telling the truth." Iii Phase I
research, about two-thirds of our Study 2 sample felt discussion of a range of risk estimates
signaled government honesty. In the study reported here, equal numbers agreed and disagreed
with this proposition. However, as in the earlier research, only about a third felt such discussion
made the government seem less competent and 45.7% disagreed that the discussion made the
agency seem more honest.
FIGURE 2. Initial Scenario, Version 1
Assessing the Risks of Water Quality
Next consider risks from the water you drink, whether it is bottled or tap water.
Suppose that a government studyfound that the water you drink in your home
contains a level of a particular chemical that poses a health risk of getting cancer of 1
in 1,000,000 (one in a million) over a lifetime of drinking that water.
Most likely risk
1 in 1,000,000
' Lifetime extra cancer risk
The government study indicates that this is the most likely
level of risk, but the study also finds that the true risk could be
as low as zero or as high as 10 in 1,000,000.
Highest risk
10 in 1,000,000
The government says that the reason for this range of risk numbers is that the only scientific studies of risks from
this chemical involved laboratory tests with animals. Scientists disagree about whether the way an animal reacts to
a chemical will reliably predict how a human would react to the same chemical. This is because different
assumptions about how to predict human risks from animal reactions to the chemical result in the range of risk
numbers shown above. -
Read each of the items below and respond by circling the appropriate number or checking the appropriate box.
Assume that the term "water" refers to the water you drink, whether it is bottled or tap water.
-------
Lay Views on Uncertainty in Health Risk Assessment
page {7.
FIGURE 3. Initial scenario, Version 2, New Last Paragraph
The government says that the reason for this range of risk numbers is that the only scientific studies/of risks from
this chemical involved cases where people were exposed to much larger amounts of the chemical in their drinking
water than appear in your drinking water (at the time they were exposed, no one knew this chemical could affect
health). It is not clear that the cancer-causing-effects in these people of high levels of the chemical reliably predict
how a human would react to the much lower levels of the same chemical in your water. This is because different
assumptions about how to predict risks from low levels of the chemical, when only information about risks from high
levels is available, result in different risk numbers. •"'.'.
TABLE 2. Scenario Responses Statement
' . - - - l' ' /
2a The information in the scenario above is not understandable.
2b Government knows exactly what the level of risk is.*
CONCERN index (a = .63) -
2c I would actively work to get my water supply cleaned up.
2d 1 would be concerned about the effects of drinking this water.
GOVHONEST index (a = .74) .
2e The government is telling the truth. .
2f The government's discussion of the range of possible risk levels
makes the government seem less competent. .(R)
2g The government's discussion of the range of possible risk levels
makes the government seem more honest. ,
NOWORRY index (a = .74)
2h This water poses a serious health risk. (R)
2i 1 would continue drinking this water. •
Somewhat or
' considerably
. • - - . ' higher
Disagree
73.6%
62.5
31.4
26.8
40.4
56.8
45.7
68.2
21.4
About the
same as
other health
risks
Agree
20.4%
26.8
61.8
70.7
29.3
• 32.5
44,6
26.8
73.9
Somewhat or
considerably
lower >
Don't know
' 6.1%
10.7
6.8,
2.5
30.4
10.7
9.6
5.0
4.6
Don't know
• 2j Compared to other health risks in my life, the
risk from drinking the water discussed in the
scenario on the previous page is;..
8.6%
19.6%
.71.4%
0.4%
Note. * Item (quotation or paraphrase) derived from Phase II interview; R = item reversed in index; Cronbach's
alpha for indices in parentheses; data from Eugene, Oregon, questionnaire.
-^W ' • • , - . • , . • •
Responses to Range Bounds ,
Presenting uncertainty through ranges of risk estimates by definition requires upper and
lower bounds to the range. This raises questions about how people might react to these bounds
-------
Lay I'iews on Uncertainty in Health Risk Assessment . page !R
(e.g., by anchoring to one or the other extreme, and basing their response on that number rather
than on the range). We present results here suggesting an upward bias among some people and a
seemingly distrustful view of zero as the lower bound of a range of risk estimates.
Upward Bias in Assessing Ranges of Estimates .
Earlier experiments had people pick a level of known risk that left them indifferent between
living in the area subject to that risk and in another area of ambiguous risks. Some respondents
showed a strong bias toward anchoring on the upper bound of a range of risk numbers (Viscusi,
Magat, & Huber, 1991). We tested this bias hypothesis in two ways.
First, we asked about agreement with the statement "If a range 'of environmental health risk
numbers is given, I would believe that the highest risk number is the, correct one," which had
been volunteered by a Phase II interviewee. Half (46.4%) of our respondents said they agreed
with it, although nearly as many (43.2%) disagreed (10.4% did not know). If we take agreement
with this statement as a valid measure of "upward bias," Viscusi etal. (1991,p. 163) founda
much smaller bias than we did. For a range of disease risk between 110 and 240 cases per
million, 13 of 58 (22.4%) picked the highest number of the range. We do not know whether the
wider spread in their range (130 in 1,000,000) than in ours (10 in 1,000,000) affected the relative
upward bias. We should note that an even larger upward bias occurred in response to scientific
conflict (see "Disagreement Among Groups and Scientists").
Our second means of testing the Viscusi et al. result produced less dramatic results. We
asked readers of our initial scenario about cancer risk from drinking water to judge the size of
this risk in two ways, for the community and for themselves. The scenario (presented in figures 2
and 3) said that the "most likely risk" was 1 in 1,000,000, but that "the true risk could be as low
as zero or as high as 10 in 1,000,000." Table 3 shows the proportion of sample members who
estimated the cosnmunity and personal risks from our scenario at various levels of risk. When we
i " .*''*.
add the proportion who assessed the risk as equal to the upper bound in the scenario (10 in
1,000,000) and the proportion who assessed the risk as higher than this level, we see that 14.3%
(7.5% plus 6.8%) judged the community risk at or above the upper bound of the government risk
estimate. Similarly, 10% (4.3% plus 5.?%) judged the personal risk at or above the government's
upper bound estimate. We would expect only 5% (rather than 10-15%) of our respondents to
-------
Lay Views'on Uncertainty in Health Risk Assessment . ; . .pa?e 19
estimate risks as this high if the government's" upper bound is a 95% confidence limit, and there
was the same distribution of estimates among citizens as in the government analysis. These
results also suggest a tendency among our respondents to see larger risk estimates as more likely.
However^ the discrepancy between this finding and the proportion who claimed they assumed
higher risk numbers were more correct prevents us from estimating the prevalence of this upward
bias in the population. ' ,
TABLE 3. Respondent-Estimated Risks of Initial Scenario
Risk Estimate
Zero [lower bound of scenario]
Between zero and 1 in 1 ,000,000
' 1 in 1 ,000',000 ["most likely risk" in scenario]
Between 1 in 1,000,000 and 10 in 1,000,000 .
1 0 in 1 ,000,000 [upper bound of scenario]
Higher than 10 in 1,000,000
Don't know .
Community
1.1%
11.1,
12.5
52.5
' 7.5
6.8
8.6
Personal
6.8%
15.4
17.5
40.7
- 4.3
5.7
9.6
Note. Respectively, questions were "The risk to the community from the scenario
on the previous page is.'.." and "My personal risk from the scenario...is most
likely to. be...." These formed ESTRISK index, a =.73, with data from Eugene
sample. ' • -
•These results raise questions about the hypothesis that "those not familiar with quantitative
methodology" would misconstrue each number within a range of risk estimates as "equally
probable" (Goldstein, 1995, p. 1602). Some 89% of the Eugene sample whose answers appear in
Table 3 have at least some college education. Although some are less comfortable with
mathematics than this level of education might imply (see later discussion), these results do not
seem to reflect a widespread equal-probability view. Because the questions were phrased as "The
risk to the community from the scenario .-.. is" and "My personal risk from the scenario ... is
most likely to be," we were encouraging participants to pick only one of the options listed in
Table 3. If they expected each risk number to be equally likely, we would expect them to have
selected "I don't know," because we did not offer an "equally probable" choice. Clearly, most
people-did not make this selection. Our results may have differed if we had had access to a less
-------
•j
Ley Hews on Uncertainty in Health Risk Assessment , . page 2Ci
educated sample, but at this point we would lean to ward rejecting the equal-probability
hypothesis. , ,
Zero as a Lower Bound Risk Estimate Created
Problems
A National Research Council (NRC) committee noted (1994, pp. 9-23) that USEPA has :
typically used the standard phrase that a risk "could be as low as zero." The committee said this '
might be misleading in a given case, and current information allowed for more accurate (in many .
cases nonzero) lower bound estimates. A reviewer of our previous article on public responses to
uncertainty felt our findings might have been affected by use of the same 'terminology in our
scenarios. People might think "zero" indicated risk assessors' ignorance, and he felt a small but
f.
positive lower bound might yield different results. On the other hand, experiments with college
students suggest that people treat small incidence numbers (e.g., less than .000003) as essentially
zero (Stone, Yates, & Parker, 1994). One of our earlier studies did include positive (one in ten
thousand and one in one million, bracketing the Stone et al. figure) and zero lower bounds,
without prompting significant differences in response.
We presented interviewees, focus group members, and jury pool members with alternative
lower bounds. Among their open-ended responses were the following:
• government never announces an environmental issue if there truly could be zero risk
involved, even if they say the risk could be zero . • ,
• when government says about environmental issues that "the true risk could be as low as
zero," they are really saying "we could be wrong" -• ;
• if the risk of getting an environmental health problem could be zero, one doesn't worry
about it
• if one is told that an environmental risk could be zero or could be higher, one tends to
1 assume that the risk is zero
• the fact that government says the risk could be zero means to me that it's probably not
zero.
Clearly different people offered different, and therefore sometimes contradictory,
comments. They agreed that it would make no difference whether the lower bound of a range of
-------
Lay Views on Uncertainty iri'Health Risk Assessment • ' • • page?f
risk estimates-was zero, or 1 in 10,000,000, but disagreed on the reasoning. Some,saw the two
•numbers as essentially the same, others saw the latter as a risk that should be eliminated.
To understand the scope of these contradictions and' disagreements, we turned four of these
volunteered phrases (shown in Table 4),into statements for our closed-ended questionnaire. Each
statement occurred with two different numbers—zero and 1 in 10,000,000;—with otherwise
identical language. Depending on the statement, 56-72% of participants gave the same response
to both items in each pair, thus apparently treating these alternate lower bounds as identical. The
majority, treating very low risks as effectively equal to zero, confirmed.the findings of Stone et
al. (1994), .
TABLE 4. Responses to Lower Bounds
4a The fact that government says the risk could be
means to me that it's probably not . *
(p < .0001)
4b When government says about environmental
problems that "the true risk could be as low as
," they are really saying "We could be wrong." *
(p<.0002)
4c If the risk for humans of getting an environmental
health problem could be , I don't worry about it.
* ' - -
i .
(p<.0001) • ,. , .
4d If I'm told that an environmental health risk could .
be __ or could be higher, I tend to assume that the-
risk is higher than . * , «. •
(p<.0001)
'Risk
Z
T
Z
: T
Z
T
Z
T
Disagree
; 29.3%
50.7
' 28.6
36.4
53.2
37.9 '.
9.3
26.1
Agree
55.7%
34.6
58.9
53.6
41.4
58.9 .
87.5
62.5
Don't know
15.0%'
14.6
: 12.5
10.0
5.4
3.2
3.2
11.4 •'.
A/ofe. Risk: Z = zero; T = 1 in 10 million. * Item (quotation or paraphrase) derived from Phase II interview;
data from Eugene, Oregon, questionnaire.
. However, close examination of Table 4' shows that not everyone reacted to the two numbers
in the same way^More people agreed that zero, versus 1 in 10 million, as a lower bound meant
that government could be wrong (item 4b), the true risk was not at that level (item 4a), and the
true risk must be higher (item 4d). Thus use of zero as a lower bound 'seemed to evoke doubt
about the risk estimate, and an upward bias jn perception of the risks.
-------
Lay Views on Uncertainty in Health Risk Assessment page 22
Statement 4c (Table 4) presents an apparent exception. Respondents were less likely to ,
worry if the lower bound was 1 in 10 million than if it was zero. Because a zero risk is
objectively lower than a 1 in 10 million risk, there should be less worry about zero risk. These
results are thus consistent with those discussed above, concerning doubt about the government's
risk estimates if they include a lower bound of zero. Given this doubt, the higher risk number
may seem more trustworthy, and thus less worrying. Phase I research (Johnson & Slovic, 1994a,
1995) found people rating lower risk estimates (1'in 1,000,000) as "preliminary" information
significantly more often than higher risk estimates (1 in 1,000). Together these -findings suggest •
that, everything else being equal, the public assumes that higher risk numbers are either more
honest or more competent (e.g., have been based on a more thorough, less "preliminary,"
.analysis).
Overall, these results show that use of the term "zero" as a lower bound had a significant
impact on the Eugene sample's responses to a range of risk estimates. The National Research
Council (1994) recommended that government agencies avoid the default language of "as low as
zero" in reporting ranges of risk estimates, unless the data did not allow alternatives. The NRC
committee making this recommendation felt that agencies making full use of available
information would rarely have to resort to zero as a lower bound.2 We believe our results suggest
that communication, as well as risk analysis, would benefit from avoiding zero as a lower bound
on risk estimate ranges.
Reasons for Uncertainty
General unfamiliarity with the concept of uncertainty in health risk assessment makes
laypeople's understanding of reasons for such uncertainty difficult to plumb. However, because -
anyone wishing to communicate about uncertainty would want to take audiences' understanding
of these reasons into account, we did some exploration of this topic. First, we review some
evidence on people's reaction to explanations of uncertainty that concerned extrapolation from
* Other technical reasons exist for avoiding ranges of risk estimates that include zero. For example, suppose there, is
uncertainty about whether an animal carcinogen is also a human carcinogen. Two risk estimates for human health
are then possible: zero if the chemical is not a human carcinogen, or a range of risk estimates whose lower bound is
a small positive number., According to this analysis, which was provided to us by Dr. Alan Stem of the New Jersey
Department of Environmental Protection, it is improper to have a range of risk estimates that includes zero.
-------
, Lay Views on Uncertainty in Hetilth Risk Assessment •.,-.'.' page .23
either animal data or human data involving high doses, of a chemical. Then we discuss how they
respond to shifting risk estimates o'ver time, disagreement over risk estimates among groups
(e.g., government vs. environmentalists) or scientists, and expert-proffered reasons for scientific
uncertainty about environmental conditions (e.g., in measurement, due to complexity, or about
past or future risks). -"•' . '< ". .'-•'..' . ." "-•'.•
Little Impact of Explanations of Uncertainty
If ranges of risk estimates are offered by institutions, as many risk assessors'suggest,
explanations of why there are ranges (i.e., uncertainty) would seem necessary for lay . ..
understanding and acceptance. Except in Study 4 (where explanation of extrapolation from
animal data had no observable impact), such explanations were largely absent from Phase I
- experiments. We decided to explore the impact of several different explanations on people's
response to uncertainty in this phase of research.
The bulk of our effort focused on explanations concerning extrapolation from animal and
high-dose data. The versions used in interviews, focus groups, and open-ended questionnaires
appear in Exhibit 3. Reading these items did not seem to help people either understand or accept
the reasons for having a range of risk estimates. .Most who read them rated the explanations as
.- . • .- i
irrelevant to the main issues of risk management (e.g.,, cleaning up the pollution), confusing, and
too wordy. The few people \yho reacted positively seemed to be more satisfied with the
- * , ', * .. ' • ' .
extrapolation-from-animal-data explanation than by the extrapolation-from-high-dose
explanation. •.-"'-'' .•
The interviewees who were less appreciative of the high-dose than the animal extrapolation
explanation could not explain their feelings. However, anecdotes from sources outside this study ,
raised the possibility that.pepple might believe that risks to workers, the population which often
provides the high-dose data from which risk assessors extrapolate to lower doses, are not
properly related to risks to,the general public. We asked our Eugene sample, to indicate their level
of agreement with the following statement: "Suppose the government uses the effects on humans
of high levels of a chemical (for example, among factory workers) tq predict the effects of lower
levels of the phemical oh members of the public. The government is competent when it uses this
' method .to determine the size of an environmental chemical risk for humans." Although
-------
Lay Views on Uncertainty in Health Risk Assessment , page 24
EXHIBIT 3. Examples of Interview Stimulus Items
ItemS
The government announces that the water you drink contains a level of
chemical X that poses an extra health risk of getting cancer of 1 in 1,000,000
(one in one million) over a lifetime of drinking that water. The government
says that is the most likely risk, but the true risk could be as low as zero or
as high as 1 in 100,000. The government says that the reason for this range
of risk estimates is that the only scientific studies of this chemical's effects
on cancer risks involved laboratory tests with animals. It is not clear that the
way an animal reacts to a chemical will reliably predict how a human would
react to the same chemical. Different assumptions about how to predict
human risks from animal reactions to the chemical result in different risk
estimates.
Item 7
The government announces that the water you drink contains a level of
chemical X that poses an extra health risk of getting cancer of 1 in 1,000,000
(one in one million) over a lifetime of drinking that water. The government
says that is the most likely risk, but the true risk could be as low as 0.01 in
1,000,000, or as high as 10 in 1,000,000. The government says that the
reason for this range of risk estimates is that the only scientific studies of this
chemical's effects on cancer risks involved cases where people were
exposed to much larger amounts of the chemical in their drinking water than
appear in your drinking water (at the time they were exposed, no one knew
this chemical could affect health). Itis not clear that the cancer-causing
effects in these people of high levels of the chemical reliably predict how a
human would react to the much lower levels of the same chemical in ypur
water. Different assumptions about how to predict risks from low levels of the
chemical, when only information about risks from high levels is available,
result in different risk estimates.
this phrasing does not directly ask people about the propriety of high-dose extrapolation from
workers, it provides an indirect measure of this concern. Slightly more people disagreed (4L4%)
than agreed (33.9%) with this statement; mariy.(24.6%) said they did not know. Given these
answers, we can only conclude that the impropriety of extrapolation from high doses to workers
to low doses to members of the general public may be a factor in the qualitative negative
reception for our high-dose explanation. , .
' During early interviews, requests were made for an explanation of the high doses
"elsewhere" from which scenario risk estimates were supposedly extrapolated. Item 7 (see
Exhibit 3) about high-dose extrapolation was thus,modified to say that "an accident" had caused
the high doses. It was assumed that an accident would have less moral taint than legal (i.e.,
permitted) emissions, and thus would not distract people from the explanation of the range of
-------
Lay Views on Uncertainty in Health Risk Assessment ''.'••• -' " • • page 25
risk estimates. However, the 'few interviewees who had a chance to comment upon this
explanation after the "accident" modification seemed to take it as a reason to be more, rather
than less, afraid of the consequences of their own exposure to lower concentrations. This reaction
•• • ^
may reflect a long-hypothesized (and debated) aversion to "chance" as a reason for troubles that
people like oneself suffer, since that seems to make one more vulnerable: ho.w can! guard
against chance making me a future victim (Shaver, 1970)? Interesting in this regard is the
response to the following statement in the Eugene questionnaire: "Suppose scientists use the
health effects of a high dose of a chemical on people exposed elsewhere by chance to predict the
effects of a much lower dose of the same chemical found in my water. That would make me feel
~ • = . r ' -. >
more at risk than if,they got the same numbers for predicted health effects from just studying the
chemical's effect on animals." Somewhat more people agreed (40.4%) than disagreed (32.5%)
with this statement, which had been volunteered by an interviewee, and a substantial number
• • . ' ' . f - ' v
(27.1%) could not answer it either way. This is not proof that "chance" is an upsetting
explanation of risk alone, much less of uncertainty, but it raises questions for future practice and
research.- ; ' •
Explanations about extrapolation from animal and high-dose data also appeared in the
closed-ended questionnaire; half of our Eugene sample read one explanation, half read the other
(see Figures 2 and 3). We did not provide a "control" scenario (that is, one without,any
explanatory paragraph). O.ur focus here was on seeing-.whether these two. explanations had
different effects on responses to the scenario. Out of 127 comparisons of the answers given by
our split sample, only three significant differences were found (the questions on which these
differences appeared are in Exhibit 4). , .
-------
Lay Views bn Uncertainty in Health Risk Assessment , page 26
EXHIBIT 4. Items Eliciting Significantly Different Responses From Readers of "Anifnal"
and "High-Dose" Extrapolation Explanations (from the close-ended questionnaire)
• I prefer being told that a situation is safe or unsafe, rather than hearing ' ,
risk numbers, such as "a health risk of getting cancer of 1 in 1,000,000."
• Government risk estimates tend to indicate the risk that an environmental
pollutant poses for the average person.
• The second study came up with a lower risk number because better
scientific knowledge was available.
If the proportion of comparisons finding a difference that is statistically significant does not
exceed the criterion of significance (p < .05 in this study), a rule of thumb is that these
differences might have arisen by chance and should be treated as suspect. Because three '
significant differences out of 127 comparisons is less than the.criterion of 5%, we do not believe
the animal and high-dose explanations for uncertainty evoked substantively different responses.
Our confidence in this conclusion is heightened by the fact that the significant differences were
found for three different topics, and each of these topics also appeared in one or more other
statements for which significant differences in the two split samples were not found. If a
substantive difference in response to the explanations was occurring, but not meeting statistical
significance, we would expect it to be concentrated in one topic (if only three differences occur).
A final explanation of uncertainty was offered during qualitative data collection, concerning
concentrations of a noncarcinogen in drinking water (the stimulus items concerned appear in
Exhibit 5). Each person read them in the same sequence: first the item with concentrations
slightly below the government-calculated "safe" level, then the item with concentrations slightly
above this level, and then the lengthy explanation. As can be seen, the latter concerns calculation
of the "reference concentration" for the noncarcinogen, extrapolated from animal data. The,
drinking water in this item also contained concentrations above "safe" concentrations, to see if
the e,\planation~would offset the discomfort we expected respondents to have with these abo;ve
standard concentrations. With one exception, every person who read this explanation deemed it
irrelevant, confusing, overly complex, or arbitrary ("why divide by, 10?," as one interviewee put
l
it). The one exception was a woman who was very distrustful and upset'over the above safe
concentration; this explanation changed her attitude to one of feeling very safe and informed..
-------
Lay Views on Uncertainty, in Health Risk Assessment . • , page 27
EXHIBIT 5. Exploring the Impact of an Uncertaiqty Explanation in the Qualitative
Study. . ' __ \ _. "
Item 11'a ' . • .
The government has found a chemical in your drinking water that can cause
a nonfatal kidney damage in laboratory rats. The level of the chemical which '
the government considers safe for humans to be exposed to is 1 part per j
100 million. The level of the chemical in your drinking water is smaller than 1
part per 100 million. . '. . / '
IternUb ' . ..'•.."..-... '.':.'. .•_'.,•.-..:.
The government has found a chemical in your drinking water that can cause
nonfatal kidney damage in laboratory rats, The level of the chemical which
the government considers safe for humans to be exposed to is 1 part per
10O million: The level of the chemical in your drinking water is 1.2 parts per
100 million. . ' v . ^
• : . , • _ ' , • ' '
Item 11c • :'-.'•'
The government has found a chemical in your drinking water that can cause .
nonfatal kidney damage in laboratory rats. However, this chemical did not ' i
cause kidney damage when fed to laboratory rats in doses of 1,000 parts , . ,
per 100 million or less. Government scientists took a cautious approach to
calculating a safe dose for humans, although it is possible 1,000 parts per
100 million would be safe for them too. They divided this number by 10 to, .
account for the fact that the rats were only fed the chemical for a short time, ,, ,
'and humans (at worst) might be exposed for an entire lifetime; divided by 10
again to account for the possibility that humans might be mpre sensitive to >.
the chemical than are rats; and divided ,by 10 again to account for humans ;
who may be more sensitive to the chemical than the average human. Thus,
the level of the chemical which the government considers safe for humans to
be exposed to is 1 part per 100 million. The level of the chemical in your
drinking water is 1.2 parts per million.
Shifting Risk Estimates Over Time
, Item 4 in the qualitative study (see Exhibit 6) dealt with changing estimates over time as
new scientific information became available. In general, temporal changes did not seem to alter
interviewees' concerns that there was any risk at all, although there seemed more distrust of
government reducing its estimates than increasing them over time. This reaction is consistent
with Weinstein's (1987) finding that government is more credible when it says that a situation is
unsafe than when it reassures people. , . . •' . .
-------
Lay
Uncertainty in Health Risk Assessment
page 28
EXHIBIT 6. Example of Shifting Risk Item in the Qualitative Study
Item 4
A government study was done of the water you drink a few years ago, which
contained (and still contains) small amounts of chemical X. It found that the
most likely extra level of risk of getting cancer from drinking this water for
your entire lifetime was 1 in 100,000 (one in one hundred thousand). This
was below the drinking water standard for this chemical at the time, so no
action was taken and the amount of the chemical in the water has stayed the
same. The government did another study recently, using new scientific' .
information about the chemical's effects, and concluded that the most likely
level of risk was 1 in 1,000,000 (one in a million). .
The closed-ended questionnaire included a scenario (Figure 4) of a follow-up study to the
"original" risk estimate for health risk of cancer from drinking water (most likely 1 in
1,000,000; range of zero to 10 in 1,000,000). The scenario said that the government was trying
"to obtain a more accurate number." The result was a most likely estimate of one in a billion,
with a range from zero to 10 in a billion. The fact that the second study produced a lower risk
estimate was specified in the scenario, to avoid any mathematical confusion. Answers to
questions about this scenario appear in Table 5.
FIGURE 4. Second Study Scenario
You will recall that the government study described earlier in this survey found that the water you drink in your
home contains a particular chemical that poses a health risk of getting cancerpf 1 in 1,000,000 (one in a million)
over a lifetime of drinking that water. Though this is the most likely level of risk, the.study also found that the true
risk could be as low as zero or as high as 10 in 1,000,000.
Imagine that the government has now conducted a second study to try to obtain a more accurate number for the
actual health risk you face. This second study finds that the health risk of getting cancer is most-likely 1 in
1,000,000,000 (one in a billion), though the study says it could be as low as zero or as high as 10 in 1,000,000,000
(ten in a billion). Thus the second study finds a lower risk than the first study.
Risk Study One:
Risk Study Two:
Most likely risk
1 In 1,000,000
Lifetime extra carfcer risk
Lowest risk
0
Highest risk
10 in 1,000,000
Most likely risk
1 in 1,000,000,000
Lifetime extra cancer risk
Lowest risk
0
.<->
Highest risk
10 in 1,000,000,000
-------
Lay Views on Uncertainty in Health Risk Assessment -....- . • • page 19
TABLE 5. Responses to Lower Risk Estimate From Follow-Up Study
5a The second study was. more competently done than the first. ."
5b 1 am more confident about the safety of my water after seeing the
results of the second study.
, 5c The two studies are essentially in agreement about the level of,
cancer risk from the chemical in the water.
5d Study One, with the higher risk number, was less competently .
done. • .
5e The difference in level of risk between the two-studies is probably
due to chance. .. . •
5f If the government did a third study it would find a risk level even
closer to zero than the second study.
5g Taken together, the results of the two studies provide a more
accurate assessment of the level of risk than either study by
itself.* ,
5h the second study came up with a lower risk number because
better scientific knowledge was available.
Disagree
29.3%
35.4
60.0
41.8
40.4
31.8
23.9
27.5
Agree
20.0%
54.6
31.8
12.9
25.4
15.7
59.6
2,0.7
Don't know
50.7% '
10.0
8.2
45.4
34.3 '
52.5
16.4 :
51.8
Note. These statements followed the scenario in Figure 3. * Item (quotation or paraphrase) derived from Phase II
interview; data from Eugene, Oregon, questionnaire.
A majority felt more confident of their water's 'safety after the second study (5b),
presumably because it found a lower risk. But in the context of the above-mentioned findings of
greater government credibility with higher risk estimates, this is surprising. However, a full third
did not feel better even with a maximum, risk of 10"8, perhaps due to distrust, About half of the
sample .refused to rate the relative competence of the two.studies (5a), to predict what risk level a
third study might find (5f), or to judge that the second (lower) estimate was based on better
knowledge (5h). Perhaps this was due to respondents feeling such opinions were not supportable
by the available information. • ; '.'.-' r "•
Several interviewees indicated that conducting more studies of environmental problems is a
signal that officials respond to public concerns. An alternate hypothesis is that multiple studies
reduce uncertainty. The fact that a majority agreed with an earlier interviewee that the studies -
together gave a more accurate assessment of the risk than either one. alone (5g) could be
interpreted as confirming the second hypothesis. This judgment would be due to the two
estimates confirming roughly the same risk level, with the second range of risk entirely
-------
Lay Views on Uncertainty in Health Risk Assessment . page 30
subsumed within the first, and both levels being quite small. Yet, contrary to the uncertainty-
reduction hypothesis, most people disagreed that the two studies had found the same level of risk
(5c), and only a quarter thought the difference was probably due to chance (5e). This suggests,
though it does not prove, that the response-to-public-concerns hypothesis is a better explanation
of the positive response to having more than one risk,study. , ' ,
Disagreement Among Groups and Scientists
We expected to find that people had differing views of the credibility of risk estimates from
various types of organizations, and this was so. Environmentalists and university scientists were
seen by interviewees and open-ended questionnaire respondents as somewhat more trustworthy
in their risk estimates than government or business. However, there was surprisingly little
unconditional trust even in the first two groups. People volunteered the possibility of academic .
bias due to funding sources for research and were as likely to be suspicious if environmentalists
agreed with institutional risk assessments as to see this as a signal of the latter's accuracy. Item
18 offered in interviews (see Exhibit 7) had government estimates lower than, but overlapping*
with, environmentalists' estimates (both in ranges). Environmentalists were universally favored
over government in risk assessment, and it was very difficult in either interviews or the open-
ended questionnaire to get people to recognize that the two groups' ranges overlapped.
EXHIBIT 7. Overlapping Estimates of Government and an Environmental Group
Item 18 " .
The government announces that the water you drink contains a level of
chemical X that poses an extra health risk of getting cancer of 1 in 1,000,000
(one in one million) over a lifetime of drinking water. The government says " •,
that is the most likely risk, but it says the true risk could be as low as 0.01 in
a million or as high as 10 in 1,000,000 (ten in one million). An
environmentalist group responds by saying that the most likely risk is ,
probably 10 in 1,000,000, and it says the true risk could be as high as 100 in
1,000,000 (one hundred in one million).
i
Disagreement among scientists is a form of uncertainty (as well as a reflection of
uncertainty) probably more familiar to most people than arguments about dose-response curves.
Item 10 (see Exhibit 8) had a majority of scientists disagreeing with a minority. The fact that
"most" scientists agreed with the government estimate was not persuasive to interviewees and
-------
Lay Views on Uncertainty in Health Risk Assessment ' . , ' , page 3)
jury pool members; distrust of government seemed to carry over to scientists, who might be on
its payroll. Our qualitative respondents were prone to attribute conflict among scientists to self-
• ".... : -\ .
interest cif incompetence. One interviewee suggested that she would trust the view of whoever
had been right in the past, rather than by how many scientists shared that view, -but was unable to
articulate criteria for such predictive success. Two-thirds (65.4%) of our Eugene sample agreed
with the interviewee-volunteered statement: "When scientists disagree over the size of an
environmental health risk, I assume the worst case is true, just in case." This reaction also
appears to mirror the Viscusi etal. (1991) finding of an upward bias in people's response to a
L \
range of risk estimates. However, it may represent a prudent rule of thumb as well as a personal
bias in response to risk ranges. In our earlier discussion of upward bias findings, people had no
external guidance for their answers when we asked them, for example, what the community and
personal risks were of drinking the chemical-contaminated water. The fact that, given only the
numbers in the risk scenario, as a group they produced numbers somewhat higher on average
seems fairly direct evidence of an upward bias. Similarly, nearly Half agreed—without external
signals—with the statement that they would tend to take the highest number in a range of risk
• „. • - ' ' i, -
estimates to be correct. In the case of the statement quoted above, however, they have external
guidance on the risks: the scientists' opinions. And even the scientists cannot agree on the size of
1 the^ risk! Why should laypeople attempt to judge which scientist is correct? In this case it would
seem prudent, even,without a personal bias toward higher risk numbers, to assume the worst is
true. This may explain why the proportion assuming the worst rises from 46.4% without
, scientists being mentioned to 65.4% when scientists disagree. One of our interviewees had said
that, "When scientists disagree over the size of an environmental health risk, it must not be a
serious problem." However, most Eugene respondents (84.3%) rejected this proposition. This
response may mean that they presume one side must be right, and—given an upward bias (see
"Upward Bias" discussion)—would assume that danger is more likely than no danger. ;
-------
Lay Views on Uncertainty in Health Risk Assessment ' ' page 32
EXHIBIT 8. Disagreement Among Scientists' Estimates
Item 10
The government announces that the water you drink contains a level of .
chemical X that poses an extra health risk of getting cancer of 1 in 1,000,000
(one in one million) over a lifetime of drinking that water. Most scientists
agree with this number, and many think that the true risk could be as low as
zero. However, a few scientists believe that the true risk could be as high as
100 in 1,000,000. Both groups have reputations as being competent
scientists. ,
Reasons for Scientific Uncertainty About Environmental -f
Conditions .
Given the clear ignorance of, and doubt about, risk estimate uncertainty among our
participants, we did not seek to explore whether people had their own explanations for such
uncertainty. However, a few spontaneous responses (e.g., in interviews) suggested some such
ideas among the public. For example, in interviews and open-ended questionnaires a few people
volunteered that point estimates of risk did not account for varying exposure (e.g., amount of
water drunk or food eaten) or susceptibility (e.g., children or elderly vs. healthy adults), or that
scientists might be biased by who paid them. We used statements developed from Rowe (1994),
on various potential reasons for scientific uncertainty, in our closed-ended questionnaire, to see
how this sample (N = 280) would react to general reasons for scientific uncertainty. These
statements appear in tables 6 to 9, with the responses of our Eugene respondents.
People agreed with all of these reasons for scientific uncertainty in the areas of
measurement, complexity, past risks and future risks; two-thirds of the reasons (16 of 24)
garnered 80% or more agreement. (These topics formed coherent dimensions in factor analysis,
and created scales for use as dependent variables in multivariate analysis.) The most consensus
(81% - 91% agreement) occurred for reasons concerning complexity in the processes producing
environmental health risks, with the least agreement (55% - 88%) on measurement problems.
Such consensus may be due to agreement with positively framed statements on topics unfamiliar
or not salient to respondents, or to a fair understanding of the challenges that risk assessors face,
perhaps due more to inference from daily experience or distrust of experts than to education or "
experience with risk assessment. Only further study can test these hypotheses, although this
group of people could have signaled lack of salience or lack of a firm position by answering
-------
Lay Views on Uncertainty in Health Risk Assessment • '•; page 33
TABLE 6. Measurement Reasons for Uncertainty.
. Each question began: "Scientists can be uncertain about their
measurements of environmental and health conditions because"
6a ...of the inherent randomness of nature.
6b ...experts can have biases, conscious or unconscious, in how
•.- ' they make measurements.
6e ...experts can have biases, conscious or unconscious,, in their
interpretation of the results. , .
6d ...they can't get enough measurements to be sure that they've
measured accurately. .
6e ...it can be difficult to decide which facts to include and which can,
be safely ignored.
f =•
6f ...the act of measuring itself can change the conditions being .
measured. '
6g ...the available instruments may not use units of measurement
small enough to tell one condition from another.
Disagree
.14.6%
11,1
8.2
34.3
18.2
11.8 '.,
21.4
Agree
79.6%
83.9
87.5
55.0
75.4
77.9 ;
60.4
Don't know
5.7%
5.0
4.3
10:7.
6.4
10.4
" 18.2
Note. Statements were abstracted from the discussion in Rowe (1994).
TABLE 7. Complexity Reasons for Uncertainty - .
Each question began: "Scientists can be uncertain about
environmental health risks because..."
7a ...they can't be sure they have identified all factors that affect
such risks. . . :
7b... the factors that affect risks may interact with each other in •
• unknown ways.
7c ....their models of how the environment works may be
oversimplified^
7d . . .there may be more than one believable model of how the
environment works, with no evidence to tell which one is more
correct. • - ' . • ' .. • . •
Disagree
5.4%
3.9
10.0
6.8 .
Agree
91.1%
.91.1 -
80.7
85.4
Don't know
3.6% .
5.0.'
9.3
7.9
Note. Statements were abstracted from the discussion in Rowe (1994).
-------
Lay Views on Uncertainty in Health Risk Assessment
TABLE 8. Reasons for Uncertainty About Past Risks
page 34
Each question began: "Scientists can be uncertain abgut past
environmental health risks because..."
...they don't have complete historical information.
...conditions now are different from what they used to be.
...there are conflicting reports about what happened in the past.
...the conditions measured in the past may not provide information
suitable for identifying past risks.
...historical information may be biased.
...they can't do experiments to check that past measurements were
accurate.
...they can't be sure today's interpretations of past information are
correct, even if hindsight seems 20/20.
Disagree
16.4%
5.0
10.0
8,2
5.0
18.9
8.6
Agree
77.1%
90.0
82.9
82.9
87.5
70.4
80.7
Don't know
6.4%
5.0
7.1
8.9
7.5
10.9
10.7
Note, Statements were abstracted from the discussion in Rowe (1994). ,
TABLE 9. Reasons for Uncertainty About Future Risks
Each question began: "Scientists can be uncertain about future
environmental health risks because..."
...of the inherent randomness of nature.
...unusual combinations of outcomes can occur.
...humans often do the unexpected.
...they haven't studied the environment long enough to separate
long-term trends from short-term changes.
...rare events could occur that would make a big difference.
...the same health problem can have nonenvironmental causes, too,
and it's hard to tell which cause is responsible for a particular
person's (or group of people's) health problems.
Disagree
11.4% -
6.4
11.1
21.4
11.8
8.2
Agree
83.6%
89.3,
81.4
66.1
82.5
80.7
Don't know
5.0%
4.3
7.5
12.5
5.7
11.1
Note. Statements were abstracted from the discussion in Rowe (1994).
"don't know/no opinion." They used this choice often elsewhere (above 10%, up to more than
50%, for 47 ofTt)3 other nondemographic statements), but much less here (7 of 24 reasons
' ' -
evoked "don't know" answers from 10% or more, to a maximum of 18.2% "don't know" '
responses).
Lack of understanding of environmental science may underlie some of the stronger
disagreements with these statements. For example, a full third (34.3%) disagreed (6d) that
-------
Lay Views on Uncertainty in Health Risk Assessment • - .- page 3 5
measurement uncertainty is due to scientists being unable to "get enough measurements"
implying ignorance of resource constraints on field collection of data or an assumption that
reliable risk estimates require very few measurements. Other major disagreements suggest belief
in the obviousness of environmental problems and data (6e), unfamiliarity with the concept of
limits to measurement precision (6g), confusion over the nature of experimentation or "past
data" (8f), and belief that any change in the environment would be long-term, making
discrimination of short-term changes unnecessary (9d). The impact of level of formal-education
on these misconceptions may be small. A survey of Americans (Miller, 1993) found poor
conceptions of science: 2% saw it as involving the development and testing of theory, 13% as
experimentation, 25% as rigorous comparison or precise measurement, and 60% volunteered no
idea about the nature of science. Those with graduate and professional degrees were not much
' '". •",,-" • - r v' . , . . .
more accurate: 9% saw science as dealing with theory and 38% with experimentation:
Uncertainty and Risk Management
Although uncertainty is usually-vie wed as a technical issue in risk assessment, it may also ,
have implications for risk management. Here we report the only two such implications raised in
Phase II research: the relative perceived value of uncertainty and risk estimates in risk
management, and the signals uncertainty might give to citizens about agency honesty and . .
competence. - ^ ' '
Neither Uncertainty Nor Risk Estimates Were Very Important
Questions asked about stimulus scenarios by nearly every interviewee, focus group member,
and respondent to the open-ended questionnaire were:
• Why .did this chemical get into the water, and why wasn't this prevented?
• What is the government doing to get the chemical out of the water, and why hasn't it
• done so already? •
• What can I do to protect myself? • . .
Other questions were raised as well (e.g., about the source, effects, and naturalness of the '
chemical), but qualitative respondents returned again and again to these three issues. They
seemed to be driven'by two assumptions: that any exposure and any risk was too large, and that
-------
Lay Views on Uncertainty In Health Risk Assessment ' page 36
government had a responsibility to ensure that no environmental health risk was present in their
water, soil, or air. However, we do not believe that this expectation of government responsibility
can be easily equated with a desire for, or expectation of, "zero" risk in their lives. As noted
earlier, interviewees and focus group members were willing to accept recreational, financial, and
even airplane risks, but not environmental health risks. Furthermore, the Eugene sample's
reactions to the initial scenario were not solely concern about effects of drinking the
contaminated water and commitment to getting it cleaned up. Simultaneously, they disagreed
that this was "a serious health risk" and said they intended to "continue drinking this water." If
they really wanted zero risk, these would'not be their answers. After all, they could switch water
supplies (e.g., by moving from tap water to bottled water, or from one brand of bottled water to
another).
Doubts About Government Honesty and Competence
In Phase I we found evidence that providing ranges of risk estimates might, at least for a
* • "
minority, undermine perceptions of government honesty and competence. Our Phase I measures
for these responses were few, and our understanding of the reasoning of respondents on this point
limited. For example, Phase I (Study 2) focus group members said that government never offers
any information about environmental problems unless it is forced to, releases information about
environmental problems only so that people can't say that they were never told about them, will
tell you there is a big risk if it wants credit for cleaning up an environmental problem, and will
tell you the risk is zero if it can't or won't, clean up an environmental problem. These responses
did not clarify what people thought in detail (if anything) about government honesty and
competence with regard to risk assessment, much less whether'disclosures of uncertainty or more
general judgments about government drove their answers, so we chose to study this further in
Phase II.
' We found in qualitative work that people would often explicitly point to the range of risk
estimates (or to related issues, such as statistical "confidence"—see discussion below) as a • »
rationale for their conclusions that government was dishonest or incompetent. For example, one
comment was that a government that was honest about the size of an environmental health risk
would give only one estimate. Another was that government is just guessing if it gives a range of
-------
LayViews on Uncertainty in Health Risk Assessment ' - ; , ' ' page 37
\ ' ' •-'''' -..-••.'''.• • ,
risk estimates for an; environmental problem. Someone who said that the government calculates
environmental risks without taking into account the inevitability of unexpected events was
clearly using her awareness of uncertainty to judge government competence as low. Some people
said that judgments of honesty depend on explanations of how the research was done on which •
government's risk number is based, or whether government is willing to show how they
calculated a "safe" risk number. , ,
Questions about whether a given item's risk estimate was honest or competent seemed to be
the most challenging ones for interviewees and focus group members to 'answer. Although many
people seemed to answer these based o'n their prior views of government, many others said that
they couldn't answer them, didn't have enough information, or had no idea how to evaluate
honesty or competence in risk assessment. Although-we gained some insights on the cues, people
used (or would like to use) to mak'e such decisions—the Agent Orange controversy, background
research studies' design or number, whether environmentalists agreed with the government—it is
clear, that we do not fully understand.how people judge honesty and competence in risk
assessment.. We have no evidence that people have explicit, a priori criteria for such judgments;
, if this is true, it would mak? future research on this issue particularly challenging and valuable.
Despite this caveat, it seemed worthwhile to -identify the distribution of opinion about
government honesty and competence in risk assessment in our Eugene, Oregon, sample. As ,- -.
noted in Table 2 earlier, there'were mixed feelings about government honesty and'competence
regarding the scenario of drinking water contamination they read. More generally (see Table 10),
' - s • . ' = ,--•• t- • • ' • • ... •, - %
three-quarters disagreed that government provision of a range of risk numbers was less honest,
and a slight majority said it was more competent, than providing a single risk number. (Items 1Ob
and 1 Oc were deliberately phrased to mirror each other, to test the reliability of competence
judgments; these were clearly reliable.) These responses clash somewhat with the response to
statement lOd, a"view expressed by one interviewee. Roughly half of ounsample said that
government talks about environmental issues only when they pose a high risk. This opinion may
explain why the reaction to lower bounds of ze.rcf(see above) is apparently so distrustful, and •
people split over whether government discussion of ranges is honest. Risk estimate ranges in
general may be welcomed as honest and possibly competent, but particular ranges may seem
-------
Lay View on Uncertainty in Health Risk Assessment . page 38
dishonest if they can include zero when "obviously" the problem must be high-risk for .
government even-to discuss it publicly. By contrast, the disagreement with statement lOe may
reflect skepticism over the validity of extrapolation from animal data (see below) rather than a
judgment of government itself. ,
* : ,
TABLE 10. General Views of Government Risk Assessment . -
, Disagree Agree Don't know
10a If government was being honest about the size of an 75.0% 15.4% 9.6%
environmental health risk it would give only one number, rather •
than a range of numbers. , .
10b I would feel more confident that government knows how to 59.6 26.1 14.3
determine the size of environmental health risks if they give a .
single risk number for an environmental problem.
10c I would feel more confident that government knows how to 29.3 53.6 17.1
determine the size of environmental health risks if they give a
range of risk numbers for an environmental problem. • ,
10d Government only makes an announcement about an . 39.3 48.6 12.1
environmentalissue when there is a high risk involved. . . '
10e The government is competent in estimating the size of an 50.4 35.4 14.3
environmental chemical risk for humans when it takes into
account the chemical's effects on animals. '
Understanding of Science and Risk Assessment
Whether people are familiar with how science and risk assessment operate may have an
impact on their responses to uncertainty in risk estimates. We discuss here awareness of
uncertainty in science and risk assessment, and beliefs about risk assessment concepts (e.g.,
animal data extrapolation) and government practice.
Familiarity With Uncertainty in Risk Assessment and
Science
Qualitative data supported our earlier conclusion that uncertainty in risk assessment was
unfamiliar. Occasionally people noted that they "had heard" statements like the ones we offered,
but usually in reaction to such items (see Exhibit 9), which to our knowledge have not been
publicized in any form. We suspect this sense of familiarity was due more to an overall
impression of numbers and phrases (e.g., "risk of getting cancer") likely to accompany
environment-related information, rather than an actual recognition of uncertainty. This does not
-------
Lay Views on Uncertainty in Health Risk Assessment ., . • . . . page 39
mean that interviewees and others were entirely unable to recognize uncertainty as an issue in
risk assessment. For example, some people mentioned differences in apparent vulnerability (e.g.,
. children, elderly, the sick) or exposure (e.g;, "How much water are we supposed to drink to get
this risk?"). Focus group members felt there were "too many numbers" and not enough
"personalized information" about risk factors or information.on what protective measures were
being or could be taken. The comments about "personalized", information also imply an
awareness of variability. r . .
EXHIBIT 9. Uncertainty Scenario: Qualitative Study •
Item 8 •'...'"'• -.-..' ~ * .
The,government announces that its scientists have calculated the risk of
getting cancer from drinking the water you drink, which as a small amount of
chemical X in it, for an entire lifetime. It is 95% certain that at least 90% pf
the population has an extra risk of no more than 1 in 1,000,000 (one in one
milliori). A small proportion of people who are more likely than others, when
exposed to a cause of cancer, to get cancer may have a risk as high as 10
in 1 ;000,000 (ten in one million) if they drink this water for their entire lives.
The closed-ended questionnaire had only one direct measure of whether people expect risk
assessment-to be certain (Table 11). Most disagreed with statement 1 la that a single number
could describe .an environmental health risk. This may be why they felt (above) that presentation
of ranges of risk estimates was potentially honest. The difference between the qualitative and
quantitative results may be due to different samples. The quantitative group had more education,
and were largely still in college, where they might have been recently exposed to similar
concepts. There was no correlation between education and answers to this question, although
education did not vary much in the Eugene sample. .Statements lib and lie discuss the abstract
fact that "different scientific ideas" about how to extrapolate from high human doses or animal
data "can.be equally valid" in estimating human risks at.low doses. About half of our sample
agreed with these indirect measures of familiarity with uncertainty in risk assessment.. Similarly,
the interviewee-volunteered idea that scientific certainty must accompany a "real"
environmental concern (1 Id) was rejected by a scant majority.
-------
Lay Views on Uncertainty in Health Risk Assessment page 40
TABLE 11. Expectations of Uncertainty in Risk Assessment and Science
' Disagree Agree Don't know
11 a No environmental health risk can be described by a single 17.1% 69.6% 13>2%
number.* -
11b When experts only have information on the effects of much 28.6 50.0 s 21.4
higher levels of the chemical, different scientific ideas of how
to determine human health risks from lower levels of that
chemical can be equally valid.
11c When there is only information on animal reactions to a 38.2 51.8 10.0
chemical, different scientific ideas of how to determine human
health risks from that chemical can be equally valid. • .
11d If an environmental problem was a real concern, scientists 54.3 35.0 -10.7
wouldn't say that there "could be" health risks.*
11e A competent scientist gives a single, definitive answer to a 80.4 12.5 7.1
question.
11f It is typical of good science that.the most likely value for what 11.1 77.9 11.1
is being measured has a range of uncertainty around it., •
Note. * Item (quotation or paraphrase) derived from Phase II interview; data from Eugene, Oregon questionnaire.
As for uncertainty in science, few agreed that' scientific competence requires definitive
answers (statements lie and 1 If), or even said that they did not know. Most members of this
highly educated group (most of whom had at least some college education) thus seem somewhat
familiar with the concept. While education did not correlate with "competent scientist"
answers—note that variability in education was low in this sample—better educated respondents
were more likely (r = .21, p < .0007) to agree that good science could be uncertain. Thus the
substantial minority of this sample who doubted science's uncertainty may better represent the
view of the general, less educated public. Statements 1 le (reversed) and 1 If formed an index
(a = .41) on "Science Uncertainty" for multivariate analysis.
Knowledge of Risk Assessment ,
.DifferentiaTknowledge has been hypothesized as a cause of different responses to risk in
general and is worth exploring as a factor in reactions to uncertainty. One of many ways to •
measure the elusive concept of''knowledge" is to see whether experts and laypeople agree on
propositions about concepts relevant for risk assessment. Table 12 compares agreement with
items on dose-response relationships and animal data among three groups: our Eugene sample,
-------
Lay Views, on Uncertainty in Health Risk Assessment ; . page 4}
and samples from the Portland, Oregon, public, and the Society of Toxicology in earlier research
(Kraus, Malmfdrs, & Slovic, 1992). For about half of the items, people in our study were even
farther from expert views than the earlier, public sample, while for the other half they were closer
to the expert views. Overall, however, experts and both public samples are farther apart—with
citizens more likely to see any exposure as harmful. As noted in the table, we used two of these
items to create an "Animal Studies" variable for multivariate analysis (a = .48);' about a third of
our respondents believed in the extrapolation value of such studies. -
TABLE 12. Concepts of Risk Exposure
12a
12b
12c
i2d
There is no safe level of exposure to a cancer-
causing agent, , • ,
For pesticides, it's not how much of the
chemical you are exposed to that should worry
you, but whether or not you are exposed to it at
..all. '.".'".-'
The way .that an animal reacts to a chemical is
a reliable predictor of how a human would react
to the same chemical.
If a scientific study produces evidence that a
chemical causes cancer in animals, then we
can be reasonably sure that tfie chemical will,
Sample*
E :-
P ' '
T
••' E
P
T
' E
P
T
E
P
T
Disagree
38.6%
34.7
74.7.
47.9
59.2
94.6
52.9
45.7
40.8
29.6
24.8'
57.6
Agree
52.5%
53.9
18.7
43.6
36.1
4.2
34.3 •
43.7
55.4
55.4
69.4
40.6
Don't know
8.9%
11.3
6.6
8.6
4.6
1.2
12.9
. 10.6
3.8
15'.0
5.8 '
1.8
cause cancer in humans. . ".".-.'
12e If a person is exposed to a chemical that can E 62.1 20.0 17.9 .
cause cancer, then that person will probably , .
get cancer some day.' ' " - ,, ' .
If you are exposed to a carcinogen, then you P' 47.7 34.4 17.2
are likely to get cancer. > T 88.0 8.4 3.6
Note. * E = Eugene, Oregon, sample in this study; P and T are the samples of the Portland, Oregon, public and
Society of Toxicology members, respectively, in Kraus et al. (1992). Statements 12c and 12d formed the index
ANIMAL, = .48. • • • •
• .Another Way to assess knowledge is to examine notions of standard practice in government
risk assessment (see Table 13), although experts could dispute what that is. The many "don't
knows" indicate that people feel they don't know much about this topic. Statements 13e and 13f,
asked, at different points of the questionnaire, imply the belief that average exposures are used in
government risk estimates, but contradict statement 13a (maxima). Both are contradicted by
-------
Lay Views on Uncertainty in Health Risk Assessment . page 42
statement 13b on actual measures. Whatever government risk assessors do, our sample clearly
has no firm concept of that practice; these statements did not load together on any dimension
when data were factor analyzed. . ,
TABLE 13. Beliefs About Government Risk Assessment . ; ,
Disagree Agree Don't know
13a Government risk numbers tend to be based in part on the 20.0% 47,5% 32.5%
maximum amount of a pollutant that any person could be
exposed to.
13b Government risk numbers are based on actual measurements 39.3 32.5 28.2
of how much of a pollutant people are exposed to, not just on
assumptions about human exposures.
13c Government tends to take a cautious approach in calculating 45.4 33.6 21.1
the size of environmental health risks, and assumes the worst •
case about how people might be exposed or react to a
pollutant. -.
13d Government risk numbers tend to indicate the risk that an 38.6 32.5 . 28.9
environmental pollutant poses for the person at greatest risk. • - •
13e Government risk numbers tend to be based in part on the 17.1 60i4 22.5
amount of a pollutant that the average person is exposed to. •••
13f Government risk numbers tend to indicate the risk that an 19.6 59.3 21.1
environmental pollutant poses for the average person.
Potentially Mediating Factors
Beliefs and stances unrelated to the environment or risk or risk assessment may affect how
people react to the issue of uncertainty in human health risk estimates. It is thus important for us
to account for at least some of these potentially mediating factors in this study, particularly in
multivariate analysis. Here we discuss our findings with regard to mathematical prowess,
personal vulnerability, uncertainty in daily life, and worldviews,
Numbers, Uncertain or Not, Are Hard to Grasp
1 Difficulty with numbers may be related to education, mathematical or statistical knowledge,
and/or personal comfort with numeration. Education varied little in our closed-ended
questionnaire sample (75% with some college, 4% with a high school degree or less), although-
we used it as an independent variable in multivariate analysis. As noted under "Methods," most
people with a high school degree or less failed to give substantive responses to the open-ended
-------
Views on Uncertainty in Health Risk Assessment • page 43
questionnaire. Perhaps education makes it easier to cope with risk numbers, including uncertain
ones; perhaps people with more education were more' accustomed to answering questionnaires..
Several interviewees had problems comparing different risk numbers (e.gv two "most
likely," estimates, or "lowest" vs. "highest" risk numbers). For example, one person expected to
* > - , - • *
"feel safer about drinking water" if estimated risks changed from 1 in 1,000,000 to 1 in 100,000.
A retired engineer had trouble with Item 1 Ic (see Appendix I), because it required a confusing
triple-division by 10, in the process of applying "uncertainty factors" to a NOAEL to derive a
Reference .Concentration for a noncarcinogen, (Another reaction to that item—"there was no
justification for dividing by 10 rather than 20 or 5"—was a policy or communication conflict
. rather than a signal of difficulty with numbers;) •
Because of these earlier reactions, we decided to put some crude measures of mathematical
skill into the closed-ended questionnaire. We asked for the degree of agreement with two
.statements that compared two risk numbers: "You are less likely to get a disease if the risk of
getting it is 10 in 1,000,000 than if the risk is 1 in 1,000,000" and "You are more .likely to get a
disease if the risk of getting it is -7 in 1,000,000 than if the risk is 1 in 100,000." The first
. statement used a common denominator and numerators one magnitude apart; the second
statement used different denominators-and an odd difference (7-fold) in'numerators. Their
difficulty varied as one might expect: only 13.6% of our respondents agreed (incorrectly) with
the first statement, while 31.4% agreed (also incorrectly) with the second statement.
» - ...
Soine focus group members in Phase I research had suggested that using a common
denominator to compare numbers (e.g., 0.1 in 1,000,000, 1 in 1,000,000 and 10 in 1,000,000)
would make this task easier than the usual common-numerator format (1 in 10,000,000, 1 in
1,000,000, and 1 in 100,000). Most.people shown these variants in Phase II qualitative research
felt .the first format was more confusing than helpful; some people given items in this format
actually translated them into the common-numerator format to answer the open-ended
' - - \
questionnaire, implying the latter is more familiar. However, in our closed-ended questionnaire .
there was very little error, with the cornmon-denominator format. We also used this common-
. denominator format in.our scenarios for the closed-ended questionnaire (e.g., a most likely risk
of 1 in 1,000,000, with a range extending up.to 10 in 1,000,000). About a fifth (20.4%) of the .
-------
Lay Kr'eivjr on Uncertainty in Health Risk Assessment ' page 44
sample agreed that the information in the initial scenario was "not understandable," but this
could have been due as much to the presence of numbers alone, or to the other information in the
scenario, as to the use of this numerical format.
A third measure of mathematical skill was self-reported: agreement with the statement "I
feel very comfortable dealing with numbers and calculations." About a third (32.9%) of our
Oregon questionnaire respondents—equal to the proportion erring on the more challenging
numerical comparison cited above—disagreed with this statement.
Responses to the two comparisons and one self-report were poorly correlated (e.g., r = .03
among the comparisons; highest correlation was r = .15 [p > .01] between, self-reported comfort
with numbers and the common-denominator statement), and Cronbach's alpha was extremely
low (a = .19). We thus used these three items as separate independent variables in subsequent
multivariate analyses.
Statistical wording created other problems for respondents. For example, use of the phrase
"95% confidence" in some early interviews evoked outrage about the government's "lack of
confidence" and lack of explanation as to why it wasn't surer; explanations of what was meant
by statistical confidence were not persuasive. Changing this term to "95% certain" did not
reduce confusion and anger much either; people seemed to interpret this primarily as government
not knowing what the risk was, or "there is too much of a risk," although one or two said this
was about the same as being certain. Item 5 (see Exhibit 10) said there was a 5% chance of risk
above 1 in 100,000, and a 5% chance that the risk was zero, and thus a 90% chance of the true
risk being between zero and 1 in 100,000. It failedto help interviewees understand, the concept of
confidence limits. • ./.".,
EXHIBIT 10. Confidence Limits: Qualitative Study .
-j»
ltem'5
The government announces that there is a 5% chance that the extra level of .
risk from drinking chemical X in the water you drink for your entire lifetime is
above 1 !n 100,000 (one in one hundred thousand), and a 5% chance that
the risk Is zero. The government says this means there is a 90% chance that
the true risk is between zero and 1 in 100,000.
-------
Lay Views on Uncertainty in Health Risk Assessment , . • : ' •. • page 45
Confusion became even worse in response to a phrase recommended by the National
Research Council (1994, pp. 10-24) for communicating about variability in susceptibility. Our
version (item 8; see Exhibit 9) said that "It is.95% certain that at least 90% of the population has
an extra risk of no more than ,1 in 1,000,000." Focus group members (in questionnaires answered
before the group met) said, among other responses, that this meant:
• "They are not 100% sure. They need to be 100% sure of what's goin'g to happen."
• "What are they really, saying.... This info is truly frustrating."
• "These numbers are self-explanatory and give people the information they need."
• "A guess ..; could be a fair guess." ,
• "Most confusing to me is: how do'they get these numbers?"
The focus groups also saw a graphic proposed by a National Research Council committee
(1994, pp. 10-25), slightly modified for readability, for communicating variability in exposure-
and confidence limits on the resulting risk estimate (see Figure 1). This was also greeted with
confusion by focus group members, despite an explanation by the focus group facilitator, who
teaches statistics professionally. For example, they could not by themselves find on the graphic
the risk of median expo.sure.
Personal Vulnerability . . •".>'.'
We seemed to confirm earlier conclusions about an "optimism" bias, in which people see
their own risks as being less than those of society or other people (Weinstein, 1980). In response
to the initial scenario (Figure 2), as many as 39.7% saw their personal risk as at or below the
"most likely" (1 in 1,000^000) level calculated by government, while only 24.7% saw the same
level of risk as likely for the community (see Table 3). These two answers formed an'index of
ESTRISK (a = .73) used in later multivariate analysis. Over two-thirds (71.4%) rated the risk
from drinking trie water discussed in the scenario as "somewhat" or "considerably" lower than
"other health risks in my life." Others saw equal (19.6%) or higher (8.6%) risks from drinking
this water than from other health risks they faced. .
' More generally, most respondents felt their chances of being exposed to an environmental
pollutant were "about the same as" (78.2%) or "less likely than" (14.3%) chances of exposure
-------
Lay Views on Uncertainty in Health Risk Assessment - page 4 6
for the average person. Similar responses (78.9% and 15.0%, respectively) were obtained on
judgments of their chances of getting sick from environmental pollution, compared to the
average person. About 13 agreed with an interviewee who said she assumes she is likely to be the
person who suffers a 1-in-1,000,000 environmental health risk. These findings together suggest
that a conscious (or at least admissible) sense of personal vulnerability to environmental
problems is not common; we formed an index from these statements (a = .57) to use in
multivariate analysis.
However, we also found nearly half our sample (43.2%) agreeing that "There are serious
environmental health problems where I live" (45% disagreed). They saw moderate to 'high risk
from drinking tap water in their homes as more likely (35.4%) than for bottled water (7.1 %), •
although only 4.3% drank bottled water every day. The distinction between those who. saw tap
water risks as zero or low (62.5%) and those who saw these risks as high might ha.ve affected
responses to uncertainty in a scenario about drinking water contamination. Each of these
questions (except the question on perceived risks of bottled water, which had too little variance)
was an independent variable in subsequent multivariate analyses.
Life Uncertainty
We also expected that personal stances to uncertainty in general might affect reactions to
uncertainty in health risk assessment. Over half of our Eugene sample (56.1%) agreed that "I try
to avoid uncertainty in my life as much as possible," but disagreed (62.9%) that "To a great
extent my life is controlled by accidental happenings." These items did not form a viable index,
so they were entered as separate independent variables in the multivariate analyses.
Worldviews
As mentioned in our Phase I report and article (Johnson & Slovic, 1994a, 1995), we had
found evidence that general worldviews might affect response to uncertainty. Because of
analytical constraints on the earlier data set, however, our findings were not conclusive, and we
believed that some further analysis was warranted. Thus several questions that appeared to elicit
salient worldviews were asked in our Eugene questionnaire. Table 14 shows those that formed
-------
Lay Views on Uncertainty in Health Risk Assessment ' , . page 4 7
reasonably coherent indices, which were entered in multivariate analyses. (Cronbach's alpha is
listed in parentheses next to each index title.) •
One apparent dimension we labeled "antiegalitarian," because most of its component
statements concerned support for inequality (e.g., beliefs that the "equal rights" movement has
gone too far, or more pay should go to those with greater ability). Measures of egalitarianism
seemed to be associated with beliefs about dishonesty or incompetence in ranges of risk
estimates in our Phase I research. They also have been associated with risk beliefs in some other
studies (e.g., Dake, 1991; Peters & Slovic, in press; Rayner & Cantor, 1987). Earle and .
Cvetkovich (1995) have argued that values~such as those that appear to underlie our ANTIEGAL
index drive trust, and thus in turn affect perceptions of risk, and so forth. The fact that our factor
analysis found TRUST to be separate from ANTIEGAL raises questions about this hypothesis.
Statements that .together seemed to represent support for "environmentalism" formed another
-index. This viewpoint has been associated with egalitarianism in some of the research cited
above, but it formed a separate dimension here; A "trust in authority" index, although weaker
(see the low Cronbach's alpha of .54), also emerged from factor analysis. Phase I research found
trust linked to judgments of honesty and competence in government discussion of risk
uncertainty. Finally,.a,very weak "fatalism" index appeared as well, exemplifying a perceived
lack of control over one's life. .
Multivariate Analyses .
Although our sample was not randomly selected, we felt it would be useful to compare the
impact of some potential influences on responses to the risk uncertainty scenarios (figures 1 to 3)
through multiple regression analyses. Tables 15 and 16 show the dependent and independent
variables, respectively, indicating (where appropriate) Cronbach's alpha for indices and the text
location of the statements making up these indices (table number and/or statement number within
a table are provided), as well as the number of respondents providing each item.
-------
Lay Mews on Uncertainty in Health Risk Assessment
page -18
TABLE 14. Worldview Statements and Indices
-
Antiegalitarian Index (a = .69)
We have gone too far in pushing equal rights in this
country.
In a fair system people with more ability should earn
more.
If people in this country were treated equally, we
would have fewer problems. (R)
What this world needs is a more equal distribution of
wealth. (R)
When the risk is very small, it is OK for society to
impose that risk on individuals without their consent.
1 am in favor of capital punishment.
Environmentalism Index (a = .70)
All species, including humans, have an equal right to
co-exist on the planet.
1 would be willing to sacrifice much of my current
standard of living to insure that nature is not harmed.
1 am attracted to the spiritual qualities inherent in the
natural world.
1 know a lot about environmental health issues.
Trust in Authority Index (a = .54)
Decisions about health risks should be left to the
experts.
The police should have the right to listen to private
phone calls to investigate a crime.
When there is a really serious health problem, then
public health officials will take care of it. Until they
alert me about a specific problem, 1 don't really have
to worry.
Those in power often withhold information about
things that are harmful to us. (R)
Government has no right to regulate people's
personal risk-taking activities such as smoking,
mountain climbing? hang gliding, etc. (R)
1
Fatalism Index (a = .43)
I have very little control over risks to my health.
It's no use worrying about public affairs; 1 can't do
anything about them anyway.
/Vote. R = reversed scoring for construction of indices;
1 Disagree
83.2%
23.9
18.9
21.1
81.8
37.5
12.1
35.0
19.6
45.0
•
71.8-
76.4
78.2
17.1
24.6
70.7
84.6
Cronbach's alpha
Agree
15.4%
65.0
74.6
75.0
12.1
52.1
85.0
„ /
56.1
71.1
48.9
20.4
17.9
16.4
69.6
68.9
23.6
13.6
indices in
Don't know
1.4%
11.1
6.4 .
3.9
• 6.1
'10.4 .
2.9
8.9
9.3
6.1
' 7'9
5.7
5.4-
13.2
6.4
5.7
1.8
parentheses.
-------
Lay'Views- on Uncertainty in Health Risk Assessment
page 4.9
TABLE 15. Multiple Regression Dependent Variables
Variable name
CERTAINTY
WRONG
WORST
NOWORRY
CONCERN
ESTRISK
GOVHONEST
GOVKNOWS
2NDE'ST
. 2BETTER
MEASURE
COMPLEX
PAST
FUTURE
TAPWATER
SERIOUS
Variable description
desire for certainty
government_could be wrong . .
assumes worst given risk range
not worried by drinking water scenario
concerned about water in drinking water scenario
estimated risk of drinking water scenario
government honest fn drinking water scenario
government knows exactly what risk is in drinking
water, scenario
second study more competent in two-study scenario
two studies better than one in two-study scenario
.measurement reasons for uncertainty
complexity reasons for uncertainty ' '
past risk.reasons for uncertainty'
future risk reasons for uncertainty
perceived risks of home tapwater
serious environmental health problem where 1 live
Cronbach's
alpha
.80
.74
.38
.74
.63
.73
.74
-^
.74
.49
.73
.80 ,
.84
,.83
—
' — •
Text
location
' Table 1
Table 4b
Table 4d
Table 2
Table 2
Tables
• Table 2
Table 2b
Table
5a,d,h
• Table 5b,g
Table 6
Table 7
Tables
Table 9
> Page 47
Page 47 '
A/
273
229 .
244
256
254
245
247
280
146
222
' 259
266
261
,: 265
274
247
A/ofe..Shading separates variable types; respectively, reaction to uncertainty in general; initial scenario;
government risk estimates; the second study scenario; reasons for scientific uncertainty; local risks.
, The shading in Table 15 separates dependent variables that cover reaction to: uncertainty in
general, the initial scenario and government risk estimates, a second study with lower risk
ranges, reasons for scientific uncertainty, and local risks. Table 16 is shaded to distinguish
various categories of independent variables, dealing in turn with worldviews, personal stances
toward uncertainty, knowledge, and sociodemographic characteristics (three dependent
i " ' '• * * ' ..•,-'•""
variables—CERTAINTY, WRONG and WORST—also became independent variables for some
multiple regressions). , ; v •
-------
Lay Views on Uncertainty in Health Risk Assessment
TABLE 16. Multiple Regression Independent Variables
page 50
Variable name
ANTIEGAL
ENVIRON
TRUST
FATAL
AVOIDUNC
LIFEACC
P_VULNER
TAPWATER
BOTTLED
SERIOUS
MATHCOMF
EASYODDS
HARDODDS
SCIUNCER
ANIMAL
SEX
AGE
BAEDUC
WHITE
CHILDREN
ENVGRP
CERTAINTY
WRONG
WORST
Variable description
antiegalitarian
environmentalism
trust in authority
fatalism
avoids uncertainty in one's life
life controlled by accidental happenings
feels personally vulnerable
high health risks from drinking tap water in own home
frequency of drinking bottled water
serious environmental health problem where I live
comfortable with numbers and calculations
1 0 in 1 ,000,000 risk less than 1 in 1 ,000,000 risk
7 in 1,000,000 risk less than 1 in 100,000 risk
science is uncertain
belief in extrapolation from animal studies
sex
age in years
education of bachelor's degree or higher
race
children living in household
affiliated with environmental group
desire for certainty .
government could be wrong
assumes worst given risk range
Cronbach's Text
alpha location
' .69
.70
.54
.43
. —
—
.57
—
—
—
. —
—
— ' ./ " '
...41 11
.48 12
— — '
—
— .
—
— ,
—
.80
•74
.38
N
277
267
273
260
256
253
260
274
279
247
275
263
261
235
214
278
277
. 276
276
280
280
273
229
244
Note. Shading separates variable types; respectively, worldviews; personal stances toward uncertainty
(personal vulnerability, life uncertainty); knowledge, sociodemographic; reactions to uncertainty in general. •
'To increase sample sizes, we included respondents who had only one missing response for
indices consisting of three to four items, and two missing responses for indices of five to six ,
items. No missing responses were allowed for two-item indices, or where indices mixed
statements with differing numbers of response options. This resulted in Ate of 128-145 for
-------
Lay Views'on Uncertainty in'Health Risk Assessment . . ' page 5 J
various regression analyses, except that for the dependent variable of considering the second
study to be more competent (JV-88).
, • Note that Cronbach's alpha for most index variables were' modest to weak, although far
better for dependent than independent variables, (Ten of .13 dependent variable indices had
alphas above .70; only one exclusively independent variable reached that level, and three of the
seven independent indices had alphas under .50.) Although this means that generalizing the
results of multivariate analysis must be done cautiously, we think it is appropriate in an
exploratory study like this one to relax statistical constraints. Our primary aim for Phase II
research was to understand the concepts, if any, that laypeople have of uncertainty in human
health risk assessment, which-the univariate analyses discussed earlier portray. The purpose of
the multiple regression analyses discussed below is not to describe uncertainty concepts, but
instead to provide preliminary examination of reasons for these concepts. For such a purpose the
current data are sufficient. .
Reactions to Uncertainty ~
The multiple regressions are divided into two'groups. Table 17 concerns three generic
reactions to uncertainty—desire for certainty (CERTAINTY), belief that risk ranges mean the
government could be wrong (WRONG), and tendency to assume the worst if a range is given
(WORST)—and several reactions to the initial scenario (NOWORRY, CONCERN, ESTRISK,
GOVHONEST, GOVKNOWS). Table 18 includes regressions concerning reactions to the
scenario of a second risk estimate (2NDEST, 2BETTER), reasons for scientific uncertainty
" (MEASURE, COMPLEX; PAST, FUTURE), and risk beliefs (TAPWATER, SERIOUS).
As shown in Table 17, six of the eight regressions conducted to predict reactions to
uncertainty were statistically significant (with a.p of at least .05 or greater). The two regressions
that'were not significant were those predicting the belief that risk ranges mean government could.
, . • - - ..... '. - - . i
be wrong (WRONG) and the tendency to assume the worst if a range is given (WORST). For
five the six significant models the variance explained ranged from 22% to 35%. The variance •
explained in the regression predicting belief in government's honesty was only 11% and none of
the regression coefficients were significant,
-------
Lay Views on Uncertainty in Health Risk Assessment . page 52
TABLE 17. Multiple Regressions: Reactions to Uncertainty and Initial Scenario
Variables
antiegal
environ
trust
fatal
avoidunc
lifeacc
p_vulner
tapwater
bottled
serious
mathcomf
easyodds
hardodds
sciuncer
animal
sex
age
baeduc
white
children
envgrp
wrong
worst
certainty
adjusted r2
f value
f value probability.
N
• p < .05
"p < .01
*"p < .001
••••p<.0001
certainty
-.03
.01
-.09
.18
.08
.04
.04
-.01
.04
-.25*
-.10
.06
-.07
-.37***
.09
-.02
.06
-.09
-.03
.18
.01
.11
.05
—
.29
3.00
.0001
117
Note. Cell
wrong
.26*
.16
-.24
-.17
.05
-.07
-.07
.02
.00
-.00
-.07
.17
-.03
-.16
.09
.07
.11
.01
-.09
.05
.00
—
—
.20
.10
1.60
.06
122
entries
worst
-.08
.17
-.07
-.14
.02
-.05
.12
-.01
.17
.08
.04
-.16
-.03
-.10
.06
-.02
-.09
.14
.05
-.02
.04
—
— '
.07
.05
1.30
.21
124
noworry
-.01
.10
.04
-.05
. .08
-.06
-.24*
-.28**
.02
-.11
-.05
-.11
-.09
.25* •
-.14
-.14
-.05
-.19*
-.00
-.16
-.04
— ,
. —
-.06
.27
3.10
.0001
125
concern
-.32**
.02
-.23**
-.07
.06
-.04
.05
.19*
.05
-.08
-.06
.10
-.01
.18
.22*
.06
-.01
.06
. --02
.10
.02
—
— '
.05
.27
3.20
.0001
129
estrisk
-.11
.14
-.10
-.16
.14
-.04
.03
.15
-.10
.10
.08
-.13
.04
.05
.10
-.08
-.02
.08
.14
-.02
-.16
—
_ —
.03
.11
1.70
.04
122
are standardized regression coefficients
govhonest
.09
-.05
.35****
.01
-.03
.07
-.03
-.09 .'
-.04 .
.06
.07
-.27**
-.06
.07
-.07
.09
-.02
.09
.11
-.04
-.01
"— •
—
-.33***
.35
4.10 '
.0001
124
(J3 weights).
govknows
.01
.-io
. .22*
.10 "
.15 -
.04
.11
,05 -
• 14
-.13
-.10
-.05
.02,.
-.17
.19*
.10
-.10
, .06
.04
-.04
.06
—
—
.20
.22
2.50
.001
• 119
-------
Lay Views on Uncertainty in Health Risk Assessment
page j 3
TABLE 18. Multiple Regressions: Reactions to Second Study, Reasons for Uncertainty and Risk Beliefs'
Variables
antiegal
environ
trust
fatal
avoidunc
lifeacc :
p_vulner ••
tapwater
bottled
•serious •-
mathcomf
easyodds
hardodds
sciuncer
animal
t
sex
age
baeduc
white
children
envgrp
certainty
adjusted r2
f value
f value
probability
N. ,
*p<.05
**p < .01
***p < .001
****p<.0001
2ndest
-.06
-.17
-.28*
,-. .39**
. -.00
-.10
.01
.01
.01
.01
.16
'. -.02
-.11
-.05
.00
:02
;• .04
-.01
.05
.30*
-.02
.36*
.20
1.90
.03
^77
Note. Cell
2better
.11
.02
-.06
-.00 ''.'
.08
, .01
.06
.07
-.08
-.10
.10'
-.11
-.03
-.05
.07
-.11
-.00
.05
.13
. -.06
-.13
.13
-.03
0.90
.64
,116
entries are
measure
.04
-12
-.13
-.08
.05 ,
.34***'
-.01
-.06
-.00
.11
.03
, .06
-.18
.06
-.00
.25*
.04
.06
-.02
,.13
-.04
-.07
.07
, 1.40
•11
127
\
complex
-.07
-12
-.32***
-.00
.02
.13 '
.05 ,
-.09
-.12 ,
-.05
.11 ,
-.12
-.08
.09
.12
. .27**
.17
.08
• -.01
.03.
.05 '
-.10
.21
2.50
.0009
127
standardized regression
past
-.05 ;
-.19
-.19*
-.21
.01
.18*
-.04
-.10
.04
-'•12
.17
.00
-•I'1 ,
.07 '
.06
.24*
.16 , ,
.03
-;12
.15 •
-.10
-1.2
.14
2.00
.01.
128
future
.09
-.17 '
-.27**
^.03
.08
.17
-.09
-.07
-.06
-.07
.12 :
.-.09
,.17
.10
-.04
.23*
.08
-.01
.01
.03
-.02
'.05 :
' .13 '
1.90
.02
1.27
tapwater :
.02
.10
-.04
• .07 •
, .02
-.07 ... •
.06
• —
—
.41****
-.04
.11
s-.04
.08
.10
.12
.23*
-.07
-.21*
'.17* '
-.05
-.02 •'-
•21
2.80
.0004
131
serious
.01 .
.20*
-.15
-.06
-.06
.06
.20*'
.36****
--.04
—
-.14
-.06
-.03,
-.06
-.01
-.12
-.12
.16
.09
.00
.06
-.24**
.32
4.00
.0001 .
131
coefficients (fl weights).
-------
Lay Views on Uncertainty in Health Risk Assessment page 54
CERTAINTY. Two independent variables were significant predictors (SCIUNCER and
SERIOUS) of desire for certainty (CERTAINTY). Respondents who disagreed that good science
has a range of uncertainty (SCIUNCER) expressed a greater desire for certainty. Those who
disagreed that there were serious environmental health problems where they live (SERIOUS)
were more likely to have a greater desire for certainty.
NOWORRY. Four variables were significant predictors of worry (TAP WATER,
SCIUNCER, P_VULNER, and BAEDUC). Respondents with higher perceived risks of home
tapwater were more likely to worry about the water risks presented in scenario one. The belief
that good science has a range of uncertainty was positively correlated with not worrying.
Respondents who feel greater personal vulnerability were more likely to worry about the water.
risks in the initial scenario. Respondents with a bachelors degree were more likely to worry about
the water risks.
CONCERN. Four of the independent variables significantly predicted concern about the
water in the drinking water scenario (ANTIEGAL, TRUST, ANIMAL, and TAPWATER).
Respondents holding egalitarian worldviews and having less trust in authority were more likely
to be concerned about the water. Those with a greater belief in the reliability of animal studies
were more likely to be concerned about the water. Higher risk perceptions of home tapwater was
positively correlated with greater concern about the water.
GOVHONEST. Three variables were significant predictors of the belief that the
government is honest in drinking water scenario (TRUST, CERTAINTY, and EASYODDS).
Greater trust in authority was positively correlated with, the belief that, the government is honest.
Respondents with a greater desire more certainty were less likely to believe that the government
was honest in the drinking water scenario. Those who understood that "you are less likely to get
a disease if the risk of getting it is 10 in 1,000,000 than if the risk is 1 in 1,000,000" .
(EASYODDS) were more likely to believe in the government's honesty.
GOVKNOWS. Two variables were significant predictors of the belief that the government
knows exactly what the risk is in the drinking water scenario (TRUST and ANIMAL). '•
Respondents with higher levels of trust in government/authority were more likely to believe that
-------
Lay Views on Uncertainty in Health Risk Assessment
page 55
the government knows what the risk* is in the scenario. People with a greater belief in the,
reliability of animal studies were also more-likely to think that the government knows what the
risk is in the scenario. • -•• ; ,
Regressions To Predict Reactions to the Second Risk
, Estimate, Reasons for Uncertainty,; and Risk Beliefs
. Eight regressions were conducted to predict reactions to second'study (and second risk
estimate), reasons for uncertainty and risk beliefs. Overall regressions predicting six of the
dependent variables were statistically significant (p<.05). .
. 2NDEST. Three variables were significant in predicting the belief that the second study
was more competent' in the two-study scenario (FATAL, CERTAINTY, and CHILDREN).
Respondents holding fatalistic worldviews, with a greater desire for certainty, and with children
living in the household were more likely to believe that the second study (with the lower risk
estimate) was more competent, • : , , ~
COMPLEX. The two variables significant in predicting complexity as reasons for scientific
uncertainty were trust in authority (TRUST) and genderc(SEX). Women and people with less
trust in authority were more likely to see complexity as reasons for/scientific uncertainty.
PAST. Three variables were significant in predicting the belief that problems with past
information about risks were reasons for scientific uncertainty (TRUST, LIFEACC, and SEX).
Respondents lacking trust in authority, women, and those with a belief that "my life is controlled
by accidental happenings" are more likely to view these problems from the past as reasons for
scientific uncertainty. .. .
FUTURE. Only two variables were significant in predicting the belief that, problems with •
forecasting the-future were reasons for scientific uncertainty (TRUST and SEX). .Women and
, respondents with less trust in authority were more likely to see these.problems as reasons for
scientific uncertainty. ,
: TAPWA TER. Three, variables were significant in predicting perceived risks of home
tapwater (SERIOUS, AGE, and WHITE). Respondents who agreed that there were serious
-------
Lay Views on Uncertainty in Health Risk Assessment page 56
environmental health problems where they live were more likely to have higher risk perceptions
of home tap water as were older and non-white respondents.
SERIOUS. Two variables were significant predictors of the belief that there are serious
environmental health risks were one lives (TAPWATER/and CERTAINTY). Respondents with
higher perceived risks of home tapwater, environmentalists, and those with less desire for
certainty were more likely to believe that there were serious environmental problems where they
live. . .
Importance of the Independent Variables
TRUST (i.e., trust in officials) was the most common predictor of the various dependent
variables, reaching statistical significance in 6 of the 17 multiple regressions run. Those who
trusted officials were much more likely to see government's discussion of a range of risk
estimates as implying honesty and competence (GOVHONEST). This finding confirms our
Phase I regression findings. To a lesser degree, TRUST also predicted the belief that government
knew exactly what the risk was for the initial drinking water scenario. Earlier we had
hypothesized that this belief might indicate distrust, in the sense that government had a correct
point estimate of the risk, but only announced a range of risk estimates. But if it is the trusting
ones who are more likely to hold this belief, this hypothesis seems less tenable. Possibly the
belief that government knows the risk is another signal of a belief in its honesty and competence,
but if so this statement should have loaded on the same dimension under factor analysis.
By contrast, those who trusted (TRUST) were /ess likely than others to agree with three of
the four sets of reasons for uncertainty (COMPLEX, PAST, and FUTURE). This may mean that
people who tended to distrust government on principle seized upon these reasons to justify their
v •
suspicions, on the grounds that all these things that could "go wrong" showed that government
risk estimates could not be trusted. Earlier we had wondered whether the overwhelming
« _ , .,
agreement with these sets of reasons for uncertainty truly represented the Eugene sample's -
beliefs about uncertainty. This regression result suggests that that doubt may be justified.
TRUST was inversely related to CONCERN. Those who trusted were less concerned about
the drinking water contamination than others. This is consistent with other findings (e.g., Bord &
-------
•> • - • ' • • ' ' '.' .''•-..•'".. •
•f Lay Views on Uncertainty in Health Risk Assessment , . page 57"
O'Connor, 1992) that trust seems to reduce perceived risk. However, no significant relation
•.-';- , between TRUST and two other measures of perceived risk—NO WORRY and ESTRISK—
appeared, undermining our confidence in a general relationship between trust and perceived risk.
The three worldview variables other than TRUST had less of an impact on the dependent
; variables in our regressions than expected, haying among them significant effects in only four of
the 17 regressions. As noted earlier, beliefs like those included in our ANTIEGAL
(anfiegalitarianism) have been associated with risk beliefs in some other studies. It was the
strongest predictor of CONCERN, but—as also happened with TRUST (above)—it had no
significant impact on NO WORRY and ESTRISK. ENVIRON (environmentalism) had a
significant impact only on the belief that there are serious environmental health risks in one's
community (SERIOUS). . / , . ' ' '
The final worldview variable, FATAL (fatalism), was the strongest predictor of 2NDEST,
the belief, that the second risk study—which produced a lower risk estimate—was more
competently done arid based on better scientific knowledge. It is not clear what this means, .given
the much more common view that lower risk estimates are less trustworthy, and majority
disagreement (or inability to decide) about the second study's competence. Possibly fatalists
were more likely to see competence in the second, lower-risk estimate because this allowed them
to deny a risk that made them feel vulnerable (i.e., they might be "fated" to suffer health
consequences of the contamination). The fact that FATAL did not combine with seemingly
related indices of vulnerability (e.g., AVOIDUNC, LIFEACC, P_VULNER) raises doubts about
this hypothesis. On the other hand, the other significant predictors of 2NDEST—CHILDREN
(having children in the household) and CERTAINTY (desire for certainty)—are certainly
• potential motives for denial of risk, iri the sense of accepting the lower risk estimate as more
competent rather than continuing to doubt its validity. • •
-------
Lay Views on Uncertainty in Health Risk Assessment , page :8
Conclusions
As a consequence of Phase II research on public responses to uncertainty in environmental
health risk assessment, we have a much better-grasp of the content and limits of lay conceptions
of risk uncertainty and its implications for hazard management. Certainly there are gaps
remaining in our understanding, which will be outlined in detail under "Research Implications,"
but these have more to do with generalizing our results to other populations, risk types, arid so
forth, than with introducing entirely new concepts. Our findings are as follows.
People Support Ranges of'Risk Estimates in General
For the above-average-education sample in our Eugene study, the presentation of
uncertainty in the form of a range of risk estimates was deemed honest and competent in general,
although (as in Phase I studies) we continued to find a substantial minority who .questioned the
presentation of uncertainty on both grounds. This support for receiving a range of risk estimates
seemed to have been enhanced by a college-derived familiarity with the concept that uncertainty
is common in good science (regression analysis found an inverse relation between desire for
certainty and belief that science is inherently uncertain). However, they were not any more
familiar with concepts of dose-response relationships than earlier public samples, and had no
coherent idea of standard practice in government risk assessment. To the extent that such support
is knowledge-based, we would not expect it to be more common among populations with
average education (i.e., less than in our sample). .
Yet Uncertainty Makes Many People Uneasy
.' About a third of our Eugene sample rejected ranges and even risk estimates, saying they
preferred to be told simply that an environmental condition was safe or unsafe. As we noted
above, others doubt risk information that does not incorporate ranges; together these beliefs
potentially put purveyors of risk estimates in a quandary, chancing alienation of some individuals
whatever they do. Multiple regression analysis suggested that desire for certainty was associated
with beliefs that good science is certain and that there are serious local environmental health
problems.
-------
Lay Flews on Uncertainty in Health Risk Assessment ., ' , page)?
Uncertainty in Government Risk Estimates Is
Particularly Suspect .
/ While a majority of our respondents, as in Phase I, found government discussion of a range
of risk estimates to be competent, they were.more dubious about whether this indicated honesty.
We have suggested that this reaction is partly related to two other findings; that zero as a lower
bound on a range of risk estimates, rather than a small positive bound, seems to raise doubts
. . l " •••-'. • "v - -
about honesty and competence all by itself, and that people believe government discusses
publicly only high risk items. The presence of zero led people on average to feel that the risk was
. probably higher, to believe that the government could be wrong, and to worry significantly more
often than when a small positive lower bound was used. Citizens who see zero risk as unlikely or
impossible, and high-risk information as both technically more complete (see Phase.! results) and
politically more likely to appear in-public discussion, would be suspicious of a government risk
estimate ranging from zero to some higher number, even if they supported risk ranges in general.
If a risk exists at all—and by this definition it must, because government announces it—then it
. must be a high risk, and the range should include only risks higher than zero. On this
interpretation our scenarios stimulated general distrust of government risk ranges because they
included the lower bound of zero, which the National Research Council has characterized as EPA
"boilerplate."' Distrust for some people also seemed to be ideologically-based, or derived from
beliefs about how government risk estimates were developed (in technical or political terms).
Belief in government honesty and competence was related to low desire for certainty, high trust
' in officials' (a combination of expectations of honesty and deference, to authority), and one of our
measures of difficulty with arithmetic. ' ; ; '.
Environmental Health Risk Uncertainties Are Seen as
Special
The exact same range of risk estimates that elicited concern when it applied to ,
environmental health elicited little or no concern when applied to transportation (mortality), ,
financial or recreation risks. People offered contextual information (work requirements,
. -familiarity, offsetting joys) for their differing reactions to the latter uncertainties. Because the ,
transportation risks (airplane and automobile crashes) involved death, we cannot explain this
reaction solely on the grounds that the stakes with environmental health 'are different (although
-------
Lay Views on Uncertainty in Health Risk Assessment page 60
that was the reasoning offered by focus group members about the financial and recreation cases).
Varying reactions within the environmental health category (drinking water vs. soil vs. air
contamination) seemed driven more by criteria of morality or protective opportunities than by
uncertainty considerations (as our comparison of "natural" to "human" sources of risk in Study
1, Phase I, suggested as well).
All Range Numbers Are Not Interpreted Equally .
People do not seem to treat every number within a range as .equally probable, despite the
only signal of varying probabilities across risk estimates being a brief statement that one risk .
level is "most likely." Instead, on average they appear to treat higher risk numbers as more
probable (compare to the Phase I finding that lower risk numbers were seen as "preliminary"
estimates), confirming an earlier hypothesis (Viscusi et al., 1991) of an "upward bias" in
response to ranges. By contrast, use of zero as a lower bound seemed to be particularly disturbing
(see above), although a majority responded to questions using this bound no differently than they
responded to a lower bound of 1 in 10 million.
Explanations of Uncertainties Are Confusing, Irrelevant, .
or Troubling
Explanations of extrapolation from animal or high-dose human data did not differ in their
impact, and qualitative reactions suggest people largely found them confusing or irrelevant.
Earlier research (Kraus et al., 1992), confirmed by this study, found laypeople and experts,
divided over the validity of using animal data.. This may partly explain why explaining
uncertainty in risk estimates as due to extrapolation from animal data was seemingly without
effect ("seemingly" because we attempted no experimental test in this study, although a Phase I
experiment found no evidence of impact). Poor responses to explanation of high-dose
extrapolation in interviews seemed partly due to use of the word "chance" to describe how the
high* dose was received by the other population, perhaps stimulating aversion to a risk that thus
seemed more likely to affect the interviewee (on the grounds that chance cannot be avoided).
Extrapolation from factory workers to members of the general population also seemed troubling
for a plurality of our Eugene sample. Explanation of how a safe reference concentration is
calculated for a noncarcinogen also confused or upset almost all interviewees and respondents to
-------
Lay Views on Uncertainty in Health Risk Assessment • .'.''.'•'. page 61
open-ended questionnaires. By. contrast, the Eugene sample overwhelmingly agreed with reasons
for risk uncertainty stemming from measurement or complexity, or problems associated with past
or future risks, derived from the risk literature (Rowe, 1994). On methodological grounds alone
we raised the question of whether these are their own views; the regressions showing distrust
significantly related to agreement suggest that people were finding these reasons justifying prior
distrust. '
These findings do not necessarily mean explanations of uncertainty are useless. A minority
(- of our qualitative respondents found them enlightening, even enthralling; some respondents
volunteered reasons (varying exposures or susceptibility across subpopulations) why they found
ranges of risk estimates preferable to point estimates, or said they would distrust risk estimates
without explanations of how they were derived. However, any effort at explanation must deal
with the challenges of salience, trust and scientific understanding. Some of the lack of impact
among our respondents may be due to our using hypothetical cases of drinking water
contamination that did'hot motivate people to seek and process information; yet people who
believe they face a real life-and-death situation may put a low priority on understanding
; , ' ' \ f • . '
uncertainty as opposed to protecting themselves. As our regression results imply, people who are
disposed to distrust officials anyway may seize explanations of uncertainty as reasons for such
distrust: Finally, many citizens do not understand how science works or have difficulties with
mathematical information. Despite the general poor role of measures of mathematical
competence in this study, successful explanations of uncertainty will probably require a basis in
lay cognitive models of both science in general and what scientists are doing (and why) when
they, for example, extrapolate from animal data to human risks.
Shifting or Disputed Risk Estimates Do Not Reassure
Changes in risk estimates do not seem to 'affect public beliefs about risks strongly,, except
possibly to make them more suspicious if the estimates decrease. Although a majority in our
Eugene sample did feel more confident in their drinking water's safety after such a decrease, our
scenario concerned a deliberately unlikely change from a "most likely" risk of 10"6 to 10~9, and
even, then a full third of respondents were not reassured. The basis for the latter reaction is
unclear: the only variables predicting a belief that a lower follow-up risk estimate was more
-------
Lay Views on Uncertainty in Health Risk Assessment p
-------
Lay Views on Uncertainty in Health Risk Assessment . ' '• • > \ ,'pape 53
(except for "trust in officials") were dominant explanations of these responses among-those we
were able to test. ' : r . • -
- ' " " - " ' - - " ' \
Risk Estimates and Their Uncertainties Are Not Critical . ,' ',
to Most People's Views of Environmental Problems
Willing though they were, by and large, to answer our questions about uncertainty in
environmental health risk assessment, interviewees and other qualitative respondents made quite
clear that the central issue for them was prevention and cleanup of environmental pollution. They.
made it clear that for them risk numbers were generally beside the point. Although we did not
give Eugene respondents to the closed-ended questionnaire a chance to express this view
directly, the third of the sample who did not want to see ranges of risk estimates plausibly fall
into the same category. Nor can we assume that the rest of the sample welcome such ranges as
much more .than yet another piece of information to be weighed for its credibility.
We think this is the key message to be taken from our study. Risk assessment will clearly
remain an important component of environmental problems; making risk estimates available to
the public will not'only continue, but is part of their right to be informed and participate in
decision making (Johnson, and Sldvic, 1994b). However the debate over the role of uncertainty in
risk assessment and risk management is resolved, at some point it is likely (even if only
qualitatively) that such uncertainty will be discussed publicly. Such discussion will undermine
the credibility arid effectiveness of government if it is made the centerpiece of attention, rather
than prevention and cleanup of pollution. Uncertainty is a fact of life, but life goes on; similarly,
citizens expect government action on pollution in spite of uncertainty, and may suspect
uncertainty, is being raised merely to justify inaction. Attention to uncertainty can be critical to
deciding which action to take, but this public discussion must take due account of the wider
context to ensure a successful and informed outcome..
-------
Lay Views on Uncertainty In Health Risk Assessment page 64
Practical Implications
As we have noted several times, generalization of our results should be done cautiously.
However, it is unlikely that any substantial knowledge-based differences in response to
uncertainty will be found from applying our methods to populations less educated than pur
college sample. If a group relatively knowledgeable about uncertainty in science is troubled by
uncertainty in risk assessment, there seems little hope of finding a less concerned public
audience. Ideologically-based differences may vary more widely among the general public than
in our Eugene sample, of which 67% was "environmentalist," 79% "egalitarian," 88%
distrustful of authority, and 92% not fatalist, as measured by worldview statements in the ,
questionnaire. Nevertheless, we believe there are reasonably strong grounds for making
recommendations to government (and other institutions) about practical implications of our
findings.
Our interviewees and focus groups in particular stressed their interest in having
environmental problems prevented, and cleaned up swiftly if not prevented; getting risk
estimates was of far less interest, even for those who preferred a range of risk estimates to simply
being told that conditions are safe. As noted earlier, we conclude from this that communication of
risk estimates, whether uncertainty is included or not, should be secondary to dealing with the
risk management issues posed by our respondents. This may seem like stating the obvious, but
official emphasis on communicating "the risk numbers" is still common enough to make
repeating this argument worthwhile. This does not mean that communication of risk estimates is
useless or unnecessary,, but it conveys our feeling that an emphasis solely or primarily 'on. such
communication will be interpreted by most citizens as an attempt to hide incompetence or the
"true" risks. This interpretation would help neither agencies nor the public.
Some of our recommendations apply to forms of communicating uncertainty:
i - -
• Use of "zero" as a lower bound in a range of risk estimates should be used only when the
available data clearly show this is a likely "true" level of risk. The doubts that this term
raised among our respondents about government honesty and competence, plus the
technical arguments about its validity referred to earlier (NRC, 1994), do not justify its
-------
.. . _. - .
Lay Views on Uncertainty in Health Risk Assessment ' _ ' page^J
use as boilerplate. Very small positive lower bounds may be much more 'acceptable: a :
majority of-our respondents saw no difference between zero and 1 in 10 million as a
.' • ' ' ' , ', , \ ' - . '
,' . lower bound, and yet those greatly disturbed by zero were far less affected by use of the
latter figure as a lower bound. If there is no alternative to using the term "zero" in a risk
estimate, clear explanation of the reasons for its presence should be included. Although
(as mentioned below) the explanations for uncertainty we provided rarely "worked" for -
our respondents, in this case omitting an explanation only means that audiences will
substitute their,own explanations (of dishonesty or.incompetence or both).
. • About two-thirds of our respondents "assumed the worst" (thought the high end of a
range of risk estimates to be more likely); to the extent that this contradicts the beliefs of
officials and risk assessors, the discrepancy should be pointed out. 'On average people's
estimates of risk to themselves of to their community based on our scenario were not
. greatly skewed upwards; on the other hand, simply saying that one intermediate, estimate
was "the most likely risk" did not prevent an upward bias, even if it seemed to avoid
projections that every risk level within the range was equally probable. This suggests that
a more explicit specification of the reasons for the (presumably) low probability of a
range's upper boun4 might help some members of public audiences put the range into
perspective. However, as with many of our other recommendations, these explanations
should be, brief (with fuller details for those who wish them available separately) and
'''. undue faith in their effectiveness should be avoided. ''-•"' ,
• Phrases like "95% confidence" or "95% certain" aroused suspicion among our
- • • . . -. - ' . ' ' ' '' " • •''.•'
respondents, and we suggest they be used very cautiously if at all. One option is to simply
announce upper and lower bounds to risk ranges, without specifying their link to
confidence limits except in the more detailed information available to those who request •
it. What that more detailed explanation should be, however, we cannot say on the basis of
**m • . - ' ' • '
. ' our current findings. . "
• Our Phase If research added little to what we had learned from Phase I.about the
helpfulness of graphics: just as the earlier research suggested a simple line graphic made
the presence of uncertainty itself more obvious (but raised doubts about the truth of the
. risk information), the graphics appearing in figures 1 to'3 apparently conveyed
-------
>
i-
Lay Views on Uncertainty in Health Risk Assessment page 66
uncertainty without implying equal probability of all risk levels within the range.
However, a more complex (and more accurate and potentially informative) graphic
recommended by a National Research Council body (1994) was quite confusing. We
suggest that simpler graphics be used, if at all, until further research can. specify what
kinds, if any, graphics would be most helpful to citizens.
Some recommendations relate to use of explanations of uncertainty:
• Attempts to explain in detail why a range of risk estimates has been presented should be
directed primarily at audiences that express an interest in understanding the reasons. .
Otherwise, explanations will be seen as irrelevant or distracting from central issues (e.g.,
risk management). A simple statement may be enough in most cases, such as: "We are
providing a range of risk estimates because [for example] we want to account for the fact
that different people may have different levels of exposure to the problem; if you have a •
special interest in information on this uncertainty, we have a handout." Some cases (e.g., v
those where zero is a lower bound) may require explanation anyway, to avoid even more
serious communication problems, but these explanations should be few and brief.
• Responses to our explanations of uncertainties based on extrapolation from animal data or
high-dose human data did not vary significantly. Combined with our Phase I finding that
adding such an explanation to a hypothetical news story did not produce different
responses than a story without explanation of uncertainty, this might imply that
explanations are useless and should not be attempted. "We do not recommend offering no
explanation of uncertainty. Our Phase II finding only suggests that animal and high-dose .
explanations are equally good or poor; the Phase I finding may have been due to
information overload (the "explanation" story included much other information) rather
than irrelevance of the explanation; other phrasings than the ones we used might work
, better. ~*
• We do not at this time recommend using analogies to uncertainty in financial or other ..
nonhazard areas to explain environmental health uncertainties. The aim of such
explanation might be to clarify how uncertainty works, or how it.can help one make
better decisions. We are concerned that the public may interpret this instead as an
-------
Lay Views on Uncertainty in HedthRisk Assessment • •' ' page 67
.argument that they should accept risk and uncertainty for environmental health if they ,-
accept it in investments or recreation or transportation. Our Phase II study suggests they
• are likely to strongly resent such an argument. Because explaining the difference between
their inference and the intention will be challenging, using the analogy does not seem
worth the dangers. ' ' \' ' ' • ' . '
• Extrapolation from animal data faces skepticism from both citizens and experts (Kraus et
al., 1992; Slovic, in press; Slovic etal., 1995), and so will explanations of its role in risk
uncertainty. Direct acknowledgment of this skepticism, that such data are the only ones
available for this pollutant, and that having some rough.idea of the risk is better than
having no idea at all (the only alternative in this case), may be the best options for
explaining this kind of uncertainty. We do not guarantee that this explanation will satisfy
all or even most audiences. \ '
• Extrapolation from high-dose data resulting from occupational exposure or accidents
• .' seems to evoke doubt as well, perhaps due to concern that occupational data are not
relevant (or, in the example we offered in interviews, that others' exposure to a pollutant
by accident or "chance" may make people feel more vulnerable than otherwise). The
recommendations (and caveats) for explaining animal data apply here as well. ' > •
Administrator Browner, in her memorandum on EPA's Risk Characterization Program
(1995), said that while "the final risk assessment'document... is available to the public, the risk
communication process may be better served by separate risk information documents designed
for particular audiences." We endorse this statement wholeheartedly, in the sense that details
should be reserved for "backup" documents whose existence would'be publicized.
-------
Lay Views on Uncertainty in Health Risk Assessment page 68
Research Implications
Our study raised many questions for future research. Those that concern communication
about uncertainty in risk assessment include: .
• Our research examined lay response only to uncertainties in risk assessments based upon
toxicological studies. Although we have no reason to believe that the general reaction to
epidemiological uncertainties would be much different, research on this topic would
allow testing of reaction to'particular kinds of such uncertainties and to alternative
explanations of them. ,
• Phase II research focused on drinking water scenarios. In a few interviews reactions to
other environmental health scenarios (e.g., air pollution from a local factory; soil
contamination in the neighborhood) varied from those given by .the same person to the
' - , '•
drinking water scenario. These variations, however, were not the same across
interviewees, and did not seem to be based on uncertainty as much as other factors (e.g.,
perceived immorality of the factory's emissions; perceived availability of protective
options). But we did not test more widely for the presence of similarities or differences in
* reactions to uncertainty across hazards (one study in Phase I found differing reactions,
unrelated to uncertainty, between natural radiation and a hazardous-.waste-site chemical).
Although we are confident that reactions of uncertainty will not vary widely across
hazard types, research testing this hypothesis .would be valuable. •
• The potential role of graphics in communicating risk uncertainty has been little explored
(Ibrekk & Morgan, 1987), and our results make the issue seem even more complex. We
wonder whether the graphic proposed by Ibrekk and Morgan (a cumulative distribution
function above its equivalent probability distribution) can be explained simply and
briefly. Certainly the graphic proposed by the National Research Council that we showed
' focus groups elicited nothing but confusion. Our simpler graphics, although they did not
seem to make people think that risk estimates within the range were equally probable,
may not be as helpful to the public as agency officials might like. Research focusing on
these and alternatives seems valuable.
• More extensive testing of reactions to various explanations of uncertainty is needed. Our
explanations of animal-data extrapolation, high-dose extrapolation (including citing
-------
Lay Views on Uncertainty'in Health Risk Assessment , , " page (J1?
chance or workers as the source of the data), and construction of a reference
• concentration were largely taken'to be confusing, irrelevant, or upsetting. Because we
expect that some such explanations will be used when agencies and others present
uncertainty, more study (including development of alternate versions, in language and for
: /different audiences) is necessary to provide agencies some confidence that they will not
make matters worse with explanations. As part of this research, we need to understand '
how. widely explanations might be treated as reasons for distrust, rather than .as .
enlightening. . .
• Our findings that zero as a lower bound of a range elicited strong negative reactions are
striking, but further experimental work is needed to generalize these results (which, after
all, did not appear in Phase I, Study 1). Among other things, we need to test whether the
alternative suggested earlier makes a difference. This concerned having uncertainty about
whether a chemical is a human carcinogen represented as "a risk of zero, if it is not a
carcinogen, or a risk ranging from [a low positive lower bound] to [a higher upper
bound]," rather than as "could be as low as zero or as high as...," which is misleading.
• Several other research topics are less directly focused on uncertainty, but are important:
• Understanding of lay cognitive models of science and risk assessment-including, but not
stressing, uncertainty—-would help usDesign communications that better explain these
critical factors in hazard management. -'• ' • ' . - ;
• • Trust in officials was the most powerful, although by no means universal, predictor of
responses to government risk ranges; although "trust" has elicited much attention from
^ - ' , " . • ' i
researchers in the last several years,/little is yet known about its antecedents. Further
research, perhaps using think-aloud protocols while respondents read a risk estimate
announcement (with or without uncertainty), would be valuable. .
, • Similarly, we were unable to clarify how people judge the honesty and competence of
officials with regard to risk estimates (much less on other risk management tasks). Some
trust researchers suggest such judgments stem from the perception that officials share the
observer's values, but others believe these judgments reflect other factors.
-------
Lay Views on Uncertainty in Health Risk Assessment • , page ~0
References
Bord, R. J., & O'Connor, R. E. (1992). Determinants of risk perceptions of a hazardous waste
site. Risk Analysis, 12,411-416.
Bostrom, A., Fischhoff, B., & Morgan, M. G. (1992). Characterizing mental models of hazardous
processes: A methodology and an application to radon. Journal of Social Issues, 48, 85-
100. :. •
Browner, C. M. (1995, March 21). EPA risk characterization program (memorandum).,
Washington, DC: U.S. Environmental Protection Agency. '
Carpenter, R. A. (1995). Communicating environmental science uncertainties. Environmental
Professional, 17,127-136.
Cole, L. A. (1993). Elements of risk: The politics of radon. New York: Oxford University Press.
Dake, K. (1991). Orienting dispositions in the perception of risk: An analysis of contemporary
worldviews and cultural biases. Journal of Cross-Cultural Psychology, 22, 61-82.
Earle, T.C., & Cvetkovich, G. T. (1995). Social trust: Toward a cosmopolitan society. Westport,
CN: Praeger.
Finkel, A. (1994). Stepping out of your own shadow: A didactic example of how facing
uncertainty can improve decision-making. Risk Analysis, 14,751 -761.
Garrison, J. R. (1991, April). Keynote Address. Presented at the International Radon Symposium
on Radon and Radon Reduction Technology, Philadelphia, PA.
Goldstein, B. D. (1995). Risk management will not be improved by mandating numerical
uncertainty analysis for risk assessment. University of Cincinnati Law Review, 63(4), 1599-
,1610. - . .' -
Ibrekk, H., & Morgan, M. G. (1987). Graphical communication of uncertain quantities to non-
technical people. Risk Analysis, 7, 519-529.
-------
Lay Views on Uncertainty in Health Risk Assessment ,: • ; page7!
Johnson, B. B., & Slovic, P. (I994a). Explaining uncertainty in health risk assessment: Effects
on, risk perception and trust (Phase 1 final progress report to U.S. Environmental Protection
Agency as part of cooperative agreement no. CR820522). Eugene, OR: Decision Research.,'
Johnson, B. B., & Slovic, P. (1994b). 'Improving' risk communication and risk management:
' Legislated solutions or legislated disasters? Risk Analysis, 14, 905-906.
Johnson, B. B., & Slovic, P. (1995). Explaining uncertainty in health risk assessment: Initial
; '•;.'' • -
studies of its effects on risk perception and trust. Risk Analysis, 15, 485-494.
Kraus, N., Malmfors, T., & Slovic, P. (1992). Intuitive toxicology: Expert and lay judgments of
chemical risks. Risk Analysis, 12(2), 215-232. '
MacGregor, D. G., Slovic, P., & Morgan, M. G. (1994). Perception of risks from electromagnetic
fields: A psychometric evaluation of a risk-communication approach. Risk Analysis, 14(5),
815-828. ' '
Miller, J. D. (1993, November 22). Scientific literacy: An updated conceptual and empirical
review. Paper presented at the European Community Conference on the Future of Scientific
Culture, Lisbon. -
National Research Council (NRC). (1994). Science and judgment in risk assessment.
Washington, DC: National Academy Press. ,
Peters, E., & Slovic, P. (in press). The role of affect and worldviews as orienting dispositions in
the perception and acceptance of nuclear power. Journal of Applied Social Psychology.
Rayner, S., & Cantor, R. (19,87). How fair is safe enough? The cultural approach to societal
technology choice. Risk Analysis, 7, 3-10. .
; : - - • - •
Riskpolicy report.^1995, January 20). pp. 12-13.. ,
i . •' • .' , - .
Rowe, W. D. (1994). Understanding uncertainty. Risk Analysis, 14(5), 743-750.
Shaver, K. (197.0). Defensive attribution: Effects of severity and relevance on the responsibility
-',"-•. - ' ' ' v,
, assigned for an accident. Journal of Personality Social Psychology, 14,101-113.
-------
Lay \'ie\vs on Uncertainty in Health Risk Assessment page ~1
Slovic, P. (in press). Trust, emotion, sex, politics, and science: Surveying the risk-assessment
battlefield. In M. Bazerman, D. Messick, A. Tenbrunsel, & K. Wade-Benzoni (Eds.),
Psychological perspectives to environment and ethics in management. San Francisco:.
Jossey-Bass.
Slovic, P., Malmfors, T., Krewski, D., Mertz, C. K., Neil, N., & Bartlett, S. (1995). Intuitive
toxicology II: Expert and lay judgments of chemical risks in Canada. Risk Analysis, 15(6),
- ' ^
661-675.
Stone, E. R., Yates, J. F., & Parker, A. M. (1994). Risk communication: Absolute versus relative
expressions of low-probability risks. Risk Analysis, 60, 387-408.
United States Environmental Protection Agency (USEPA), Radon Division, Office of Radiation
Programs. (1992, May). Technical support document for the 1992 citizen's guide to radon.
Washington, DC: U.S. Environmental Protection Agency. ,
Viscusi, W. K., Magat, W. A., & Huber, J. (1991). Communication of ambiguous risk
information. Theory and Decision, 31,159-173.
Weinstein, N. D. (1980). Unrealistic optimism about future life events. Journal of Personality
and Social Psychology, 39, 806-820.
Weinstein, N. D. (1987). Public perception of environmental hazards: State-wide poll of
environmental perceptions. (Final report to New Jersey Department of Environmental
Protection). New Brunswick, NJ: Rutgers University.
Weinstein, N. D., Sandman, P. M., & Roberts, N. E. (1989). Public response to the risk from
radon, 1988-1989 (Research contract C29418). Trenton, NT: Division of Environmental'
Quality, New Jersey Department of Environmental Protection.
-------
Appendix I
Fourteen Items Used in Qualitative Data Collection
NOTE: ftem numbers are those identifying items developed during interviews, which were retained in focus groups and
the open-ended questionnaire (some with slight wording changes) to aid comparing responses across the various data
collection methods. .
' ' ' ' f '- • ',..'.•
' . ' ITEM2a '
The government announces that the water you drink contains a level of chemical X that poses an extra health risk of
getting.non-fatal kidney damage of 1 in 1,000,000 (one in one million) over a lifetime of drinking that water.
;, _.',., ITEM 3 [focus group] • • .
the government announces that the water you drink contains a level of chemical X that poses an extra health risk of
getting cancer of 1 in 1,000,000 (one in one million) over a lifetime of drinking that water. The government says that
is the most likely risk, but it says the true risk could be as low as zero or as high as 1' in 100,000.
,. ' ITEM 4 ;. ' -..'.'•••'.. '
A government study was dohe-of the water you drink a few years ago, which contained (and still contains) small amounts
• of chemical X. It found that the most" likely extra level of risk of getting cancer from.drinking this water for your entire
lifetime was 1 in 100,000 (one in one hundred thousand). This was below the drinking water standard for this chemical
at the time, so no action was taken and the amount of the chemical'in the water has stayed the same. The. government
did another study recently, using new scientific information about the chemical's effects, and concluded that the most
likely level of risk was 1 in 1,000,000 (one in a million). • ,
ITEM 5 [both focus'group and questionnaire]
The government announces that there is a 5% chance that the extra level of risk from drinking chemical X in the water
you drink for your entire lifetime is above 1 in 100,000 (pne in one hundred thousand), and a 5% chance that the risk
is zero. The government says this means there is a 90% chance that the true risk is between zero and 1 in-100,000.
"'•".,• • . . . /ITEM 6 ' ' * '•- ' '
The government announces that the water you drink'contains a level of chemical X that poses an extra health risk of
getting cancer of 1 in 1,000,000 (one in one million) over a lifetime of drinking that water. The government says that'
is the most likely risk, but the true risk could be as low as zero or as high as 1 in 100,000. The government says that the
reason for this range of risk estimates is that the only scientific studies of this chemical's effects on cancer risks involved
' ' . ' ', 5,1 .
-------
laboratory tests with animals. It is not clear that the way an animal reacts to a chemical will reliably predict how a human
would react to the same chemical. Different assumptions about how to predict human risks from animal reactions to the
chemical result in different risk estimates. ,
ITEM?
The government announces that the water you drink contains a level of chemical X that poses an extra health risk of
getting cancer of I in 1,000,000 (one in one million) over a lifetime of drinking that water. The government says that
is the most likely risk, but the true risk could be as low as 0.01 in 1,000,000, or as high as 10 in 1,000,000. The
government says that the reason for this range of risk estimates is that the only scientific studies of this chemical's effects
on cancer risks involved cases where people were exposed to much larger amounts of the chemical in their drinking water
than appear in your drinking water (at the time they were exposed, no one knew this chemical could affect health). It
is not clear that the cancer-causing effects in these people of high levels of the chemical reliably predict how a human
would react to the much lower levels of the same chemical in your water. Different assumptions about how to predict
risks from low levels of the chemical, when only information about risks from high levels is available, result in different
risk estimates. .
ITEM 8 [focus group] , '
The government announces that its scientists have calculated the risk of getting cancer from drinking the water you drink,
which has a small amount of chemical X in it, for an entire lifetime. It is 95% certain that at least 90% of the population
has an extra risk of no more than 1 in 1,000,000 (one in one million). A small proportion of people who are more likely
than others, when exposed to a cause of cancer, to get cancer may have a risk as high as 10 in 1,000,000 (ten in one
million) if they drink this water for their entire lives.
ITEM 10 '
The government announces that the water you drink contains a level of chemical X that poses an extra health risk of
getting cancer of 1 in 1,000,000 (one in one million) over a lifetime of drinking that water. Most scientists agree with
this number, and many think that the true risk could be as low as zero. However, a few scientists believe that the true
risk could be as high-as 100 in 1,000,000. Both groups have reputations as being competent scientists.
ITEM 1 la
The government has found a chemical in your drinking water that can cause non-fatal kidney damage in laboratory rats.
The level of the chemical which the government considers safe for humans to be exposed to is 1 part per 100 million.
The level of the chemical in your drinking water is smaller than 1 part per 100 million.
ITEM lib .''.-•;
The government has found a chemical in your drinking water that can cause non-fatal kidney damage in laboratory rats.
The level of the chemical which the government considers safe for humans to be exposed to is 1 part per 100 million.
The level of the chemical in your drinking water is 1.2 parts per 100 million.
52
-------
; ' ' . , ITEM lie
The government has'found a chemical in your drinking water that can cause non-fatal kidney damage in laboratory rats.
However, this chemical did noj cause kidney damage when fed to laboratory rats in doses of 1000 parts per 100 million
or. less. Government scientists took a cautious approach to calculating a safe dose for humans, although it is possible
1000 parts per 100 million would be safe for them top. They divided this number by 10 to account for the fact that the
rats were only fed the chemical/or a short time, and humans (at worst) might be exposed for an entire lifetime; divided
by 10 again to account for the possibility that humans might be more sensitive to the chemical than are rats; and divided
by 10 again to account for humans who may be more sensitive to the chemical than, the average human. Thus the level
of the chemical which the government considers safe for humans to be exposed to is .1 part per 100 million. The level
'of the chemical in your drinking water is 1.2 parts per 100 million.
. ' 'ITEM 13 . . . , '
Chemical X occurs naturally in drinking water in this part of the country. The government standard for a safe level of
this chemical, assuming that a person drinks this level for an entire lifetime, is" 1 part per 100 million. The government'
has found that the level in your drinking water system averages 0.9 parts per lOO.million. Due to random changes in the
natural level of the chemical in the water source, the government says the level of chemical X in the water you drink at
any particular time and place can range from undetectable amounts to 1.2 parts per 100 million.
53
-------
ITEM 17
The government announces that the water you drink contains a level of chemical X that poses an extra health, risk of
getting non-fatal kidney damage of 1 in 1,000,000. (one in one million) over a lifetime of drinking that water. The
government says that is the most likely risk, but it says the true riskcould.be as low as zero or as high as 1 in 100,000
(one in one hundred thousand).
ITEM 18
The government announces that the water you drink contains a level of chemical X that poses an extra health risk of
getting cancer of 1 in 1,000,000 (one in one million) over a lifetime of drinking that water. The government says that
is the most likely risk, but it says the true risk could be as low as 0.01 in a million or as high as 10 in 1,000,000 (ten in
one million). An environmentalist group responds by saying that the most likely risk is probably 10 in 1,000,000, and
it says the true risk could be as high as 100 in 1,000,000 (one hundred in one million). ' .
54
-------
' ' Risk Analysis,- Vol. 15, No. 4, 1995 , " . . •
Appendix H
Presenting Uncertainty in Health Risk Assessment: Initial
Studies of Its Effects on Risk Perception and Trust
Branden B. Johnson1-3 and Paul Slovic2
Received August 29, 1994; revised March 20, 1995
Some analysts suggest'that discussing uncertainties in health risk'assessments might reduce citi-
zens' perceptions of risk and increase their respect for the risk-assessing agency. We tested this
assumption with simulated news'stories varying simple displays of uncertainty (e.g., a range of
risk estimates, with and without graphics). Subjects from Eugene, Oregon,.read one story each,
and then answered a questionnaire. Three studies tested between 180 and 272 subjects each. Two
focus groups obtained more detailed responses to these stories. The results suggested that (1) people
are unfamiliar with uncertainty in risk assessments and in science; (2) people may recognize un-
certainty when it is presented simply; (3) graphics may help people recognize uncertainty; (4)
reactions to the environmental problems in the stories, seemed affected less by presentation of
uncertainty than by general risk attitudes and perceptions; (5) agency'discussion of uncertainty in
risk estimates may signal agency honesty and agency incompetence for some people; and (6) people
seem to see lower risk estimates (10~6,-as opposed to 10~3) as less credible. These findings, if
confirmed, would have important implications for risk communication.
KEY WORDS: Risk perception; risk communication; risk assessment; uncertainty.
1. INTRODUCTION ... , addressed in thepresent studies. Here we use "uncer-
'.";.• tainty" to mean a risk assessment presented in terms of
•An abiding issue in risk communication is how best a range of risk estimates, rather than as a point estimate.
to convey, risk information from scientists and officials
to citizens. A key element of such technical information
. is uncertainty, yet neither for this topic nor for risk char- 2. BACKGROUND .. '
acterization materials in general has there been "system- • ' ' ' .
atic study of.. .their comprehensibility and usefulness There have been numerous claims that explicit dis-
to various types of users"") (p. 14). In particular, no one' cussion of uncertainties in risk estimates would, among
knows how presentations of uncertainty in risk estimates other benefits, improve'the public's views of environ-
affect'public risk perceptions or citizens' trust in risk ; mental hazard management. One alleged effect is that be-
managers. — • ing open about uncertainty will enhance credibility and
We report Tiere the results of four preliminary stud-. 'trustworthiness*" presenting important uncertainties in a
ies of this topic, funded by the U.S. Environmental Pro- . risk assessment will improve' "public confidence in the
tection Agency^' Defining and calculating uncertainty quality Of our scientific output..'."'5' (p. 1, emphasis
itself has been the subject of entire books/3) and was not ^ added). Another argument is that citizens will' make
' , • '*- '-more informed'choices when the range of risks [from a
^sk Communication Unit, Division of Science and Research,: New iyen ^^ ^ which Qne % expQ^ ^ considered"(«
Jersey Department of Environmental Protection, CN409, Trenton, . ° J r
New Jersey 08625. '' ' , (P- °7)- , , - ' . '• . ,
2 Decision Research, 1201 Oak Street, Eugene, Oregon 97401. Few. Studies of public response to risk uncertainty
.3To whom all correspondence should be addressed. ' have been done. Risk perceptions of a hypothetical haz-
'.,..-",'... '.„•';• .485' •'•'- • • '•''.'
. . 0272-4332/95/0800-0485S07.50/I O 1995 Society for Risk Analysis
-------
-486
Johnson and Slovic
ardous waste case were unaffected by ,a caution about
the uncertainty of risk estimates in general/7' Views on
global warming effects were unaltered by large varia-
tions in the timing and magnitude of scientific predic-
tions about such warming.(8) Another study asked people
to name the certain risk magnitude in one area that
would make them indifferent to living there or in an area
with uncertain risk. The wider the range of risk for the
second area, the more risk averse people became; a risk
figure received more weight if it came from a "later
study." Risk aversion was attenuated if the lowest risk •
estimate was well below average, and intensified if the
highest estimate was well above average.'9' In another
study, people's desire to reduce catastrophic risks in-
creased with uncertainty: risk aversion was greater for
an accident with an equal chance of yielding zero or 100
deaths than for an accident that would certainly yield 50
fatalities.00' Only one case study cited an uncertainty-
trust relation, implying that lack of discussion of uncer-
tainty may be a problem. British sheep farmers distrusted
official statements on post-Chernobyl radiation because
these ignored uncertainty, while their farming experience
exemplified uncertainty.*1" Thus most previous studies
did not test hypotheses about uncertainty effects on pub-
lic trust or manipulate uncertainty directly.
It is unclear that explaining uncertainty (at least by
itself, and in the one-shot state of most environmental
communication) will increase trust and public confi-
dence. First, uncertainty may disturb people; they want
assurances of their safety and may prefer being told that
a situation is safe or unsafe to receiving formal risk es-
timates/12-'3' Descriptions of uncertainty in risk estimates
may undercut any illusion of safety. Second, -technical
risk information, including information on uncertainty,
may affect public response to risk and government less
than other factors; stressing uncertainty might confuse
people, or even cause outrage/14' One study found that
government actions to address public concerns and share
information early sharply reduced perceived risk and im-
proved judgments of agency performance for a hypothet-
ical chemical spill. By contrast, details on health effects
and exposure pathways had no apparent effect/15-16' In
another study, conaem for a hypothetical hazardous
waste'site was significantly affected by trust in industry
and government, perceived health threats to oneself and
family, and the sense that hazardous waste risks were
controllable. However, knowledge about chemical risks
and a generic warning about the uncertainty of risk es-
timates were not related to concern/"
In short, presenting uncertainty in risk estimates
may create, rather than remove, public confusion or con-
flict. Yet uncertainty, inherent in risk assessment, must
be part of accurate communication about risk. Research
is needed to help us determine how best to communicate
uncertainty to the public. The studies reported here rep-
resent initial steps_ toward that goal.
3, RESEARCH DESIGN
Our subjects read simulated newspaper stories
about a hypothetical USEPA risk estimate for a poten-
tially hazardous case. Stimuli were varied to identify the
separate effect each of several design variables made on
risk and other judgments. Use of a mock newspaper
story reflects the press's role as a major channel for cit-
izens' receipt of risk information. Use of an official or
simulated agency fact sheet instead might restrict exper-'
imental variation due to limits on what the agency can
say about risks.
Table I shows the major manipulations'deployed in
the four studies reported Here; Table II shows excerpts
of three stories. Study 1 used four levels of uncertainty,
described as percentages of the (highest) point estimate;
Studies 2 and 4 used only two treatments, a point esti-
mate and a range of estimates. Risk estimate ranges in
Studies 1 and 4 extended from zero to an order of mag-
nitude above the point estimate of risk; Study 1's range
extended only downward from the point estimate. Study
3 (two focus groups with Eugene, Oregon, residents)
used three Study 2 stories without alteration. In Study
4,-information about the range of probabilities was sup-
plemented by translating these probabilities to additional
cases of cancer that would be expected in a city the size
of Eugene, Oregon, over 70 years. Study 4 also a'dded, -
due to comments from the focus £roup participants, a
statement about the typicality of uncertainty in science,
and an explanation of the uncertainty (only animal tox-
icity data were available, and extrapolation from animals
to humans is inevitably uncertain). This issue of anirrial-
to-human extrapolation is one of the most debated issues
in toxicology for both citizens and experts/17''
We used fake names for the hazard stimuli: "bu-
tydin" for a chemical from an abandoned hazardous
waste site (used in all four studies) -and "zydin" for
radioactive gas entering homes from a natural source
(Study 1 only). These names reduced biases (e.g., dread
of dioxin or apathy about radon) from prior beliefs about
real chemicals or radioactivity/18' Two risk magnitudes
(point estimates)—one in a thousand and one in a mil-
lion—Were used as stimuli in all four studies. Graphics
were added in Study 2 (see examples in Figure, 1) to see
whether respondents would take more notice of a visual-
-------
Uncertainty in Health Risk Assessment
487
Table I. Research Design: Variables Used in. Four' Studies"
Study
No. of slories,
(no. of subjects)
Story designs
Uncertainty variations (examples)*
16
(272)
(180)
3
. (13, in
two focus
groups)
'•4
(217)
2 Hazards * 2 Risk
Magnitudes * 4
Uncertainty Conditions
2 Risk Magnitudes * 2
Uncertainty Conditions
2 Graphic Conditions
1:1,000 stories from Study 2:
Certainty without Graphic,
Certainty with Graphic,
Uncertainty with Graphic
2 Risk Magnitudes * 2
Uncertainty-Plus-Graphic
Conditions
1) No mention of uncertainty, e.g.: "EPA scientists estimate that the additional
risk of getting cancer over a lifetime of living in a home that might be contam-
inated by zydin at the levels seen in local houses is one in a thousand."
2) "EPA announced the highest risk estimate produced by their risk model; the
true risk could be as low as 10% of the current EPA estimate, or one in ten
thousand." , ',
3) ",. .as low as 0.1% of the current EPA estimate, or one in one million." "
4) "... as low as zero."
I)'No mention of uncertainty: "EPA scientists stated that the additional risk of
getting cancer over a lifetime of drinking water that might be contaminated by
butydin at the levels seen at the Lancaster site is one in a thousand."
2) "EPA announced the most likely risk estimate. However; the true risk could
be as low as zero, or as high as. one in a hundred."
;\ ' •
as in Study 2 , • . ' >
J) No mention of uncertainty: "If butydin gets into the water supply, EPA sci- -
enlists calculate a person who drinks this butydin-contamiriated water for 70 years
would have one additional chance in a thousand of getting cancer.... The one-
in-a-thousand calculation is the equivalent, in Eugene's population of about-
100,000, of 100 extra cases of cancer if .all city residents drank water with this
level of .butydin in it for'their entire lives. This compares to the average Amer-
ican's one chance in four of getting cancer from any cause (or an average of
25,000 cases of cancer over 70 years in a city like Eugene)."
2) "The EPA spokesperson said -that one additional chance of cancer in a thou-
sand is the most likely risk level, but added that the true risk could be as low as
zero, 'or as high as one in a hundred (1,000 extra cases of cancer if all city
residents drank water with this level of butydin in it for their entire lives)."
"Hazards: Imaginary names were used-for the chemical ("butydin") and radiation ("zydin"). Risk magnitudes: One in a thousand (1:1,000) and
one in 'a million'(1:1,000,000). Graphic conditions: Study 2 stories appeared with or without a graphic, for both certainty and,uncertainty conditions;
all Study 4 stories included a graphic, for both certainty and uncertainty conditions (see Fig. 1 for examples).
'All examples are based upon a point risk estimate of 1:1,000; stories using a point estimate of 1:1,000,000 were altered accordingly-(e.g., for
Study 1, "as low as 0.1% of the current EPA estimate, or one in one billion").
plus-written, than of a written only, presentation of un-
certainty. Each story in ,Study 4 had a graphic.
Simulated news stories included a headline, date-
line, and a column format, as in real news stories. All
stories in each study cited (1) the ,risk estimate source
(USEPA); (2) the weight offfvidence (possible cause of
cancer); (3) the effect of estimate uncertainty (more
study needed); and (4) a risk comparison ("For com-
parison, the risk of getting cancer from exposure to all
possible causes of cancer is about one in four for an
American"). Study 4'^s risk comparison (and graphic)
also listed the cases of cancer the estimated risk would
entail in a city the size of Eugene,- Oregon.
All four studies took place during 1993. Subjects
for Studies 1, 2, and .4, mostly college students, were
recruited through an advertisement in the University of
Oregon newspaper and were paid a nominal fee.
Subjects in Studies 1, 2, and 4 answered a ques-
tionnaire after reading one story. They could refer to the
story when answering the questions. Some questions
measured how well the manipulation worked (e.g., did
people reading stories with ranges of estimates see more
uncertainty 'than those reading stories with point esti-
mates?). Other questions assessed dependent or con-
founding variables, such as perceived risk, agency
honesty and technical competence, and- general attitudes
toward risk, government, and authority.09* The question-
naire also asked respondents to, indicate their sex and to
rate how well adjectives from a scale devised by Bem(20)
to measure masculinity and femininity described them-
-------
488
Johnson and Slovic
Table II. Excerpts from Simulated News Stories
Study 1. EPA scientists estimate that the additional risk of getting
cancer over a lifetime of drinking water that might be contaminate'd
by butydin at the leyels seen at the Lancaster site is about one in a
thousand.
For comparison, the risk of getting cancer from exposure to all
possible causes of cancer is about one in four for an American
EPA announced the highest risk estimate produced by their risk
model; the true risk could be as low as 10% of the current EPA es-
timate, or one in ten thousand.
Study 2. EPA scientists estimate that the additional risk of getting
cancer over a lifetime of drinking water that might be contaminated
by butydin at the levels seen at the Lancaster site is one in a thousand.'
For comparison, the risk of getting cancer from exposure to all
possible causes of cancer is about one in four for an American.
EPA announced the most likely risk estimate. However, the true
risk could be as low as zero, or as high as one in a hundred.
Study 4. If butydin gets into the water supply, EPA scientists esti-
mate a person who drinks this butydin-contaminated water for 70 years
would have one additional chance in a thousand of getting cancer....
The onc-in-a-thousand estimate is the equivalent, in Eugene's
population of about 100,000, of 100 extra cases of cancer if all city
residents drank water with this level of butydin in it for their entire
lives. This compares to the average American's one chance in four of
getting cancer from any cause (or an average of 25,000 cases of cancer
over 70 years in a city like Eugene).
The EPA spokesperson said that one additional chance of cancer
in a thousand is the most likely risk level, but added that the true risk
could be as low as zero, or as high as one in a hundred (1,000 extra
cases of cancer if all city residents drank water with this level of
butydin in it for their entire lives).
(a)
1 additional chance In 1,000
(EPA's Risk Calculation)
(b) Rangn of Risk Estimates for Lindley
I— 1 iddRlonil ch»nc
-------
Uncertainty in Health Risk Assessment
489
Table III. Effects of Experimental Manipulations
•Table IV. Findings of Experimental Studies
. - ' ' Study
Findings- " 1
Range, story read-
. er's saw more
range in story No
Percent of sub-
jects who saw , '
risk mentioned
in story . 84% •
Percent of (
subjects who
correctly
recognized 71-83%
range of risk (across the •
estimates in . three range
range-story . stories read)
Percent of
subjects who -
correctly ,
recognized sin- •
gle risk esti-
mate in point- "> . .;
story . 41.5%
.Effect of graphic Not
applicable
.
: 2 - 4
Yes ' • No
-
r .
78%. 91%
, ' • • ' •
j'
80% , " 83%
,
52% . ;46.5%
Raised perceived . Not
range of risk in appli-
story; lowered cable
trustworthiness
of story infor-
mation; did not
raise fraction
seeing risk
number in story
whether a risk number appears in the story, not whether
the agency or the reporter gave an accurate number.").
This last wording change reduced to 9.7% the percent-
age who saw no risk mention in the story. Perhaps there <
was too much information, even in the briefest experi-
mental stories, for these respondents to notice a risk
number written as a word, as in "one in a thousand."
Study 1 and 2 questions also did not ask about a "risk
number." However, the same-risk estimate appearing as
a number in an accompanying graphic did not increase
the proportion in Study 2 who said risk was mentioned
in the story, compared to those who read stories lacking
the graphic. . ,
Recognition of Ranges. Those who said a risk was
mentioned in the story were then asked whether the gov-
ernment gave "a single number for the risk or ... a
range within which the -risk might lie." In,all three.ex-
periments (Studies 1, 2, and 4) those who read a story
Experimental
questions
Study
1
Change in risk
perceptions
No Range—more risk,
effect worry (but not
with attitudes
. constant) .• •
No
effect
Agree agency is
more honest,
disagree that it's
less competent
Effect of agency
discussion of
how much risk
might vary on
• views of agency
' honesty and . ,
competence , -
Effect of attitudes No .Attitudes predict
toward risk, effect ratings of risk,
government, worry, experts'
and authority knowledge of
hazardous
• - waste risks
Range—more risk
(with attitudes
constant)
No
effect
Attitudes predict
ratings of risk,
. worry, experts'
knowledge of
hazardous
waste risks,
EPA honesty on
size of risks,
EPA compe-
tence at risk es-
timation, EPA
competence ^
dealing with en-
vironmental
problems
in which no intended uncertainty appeared were signif-
icantly more likely to answer "a single number" than
were readers of stories mentioning uncertainty. How-
,ever, many readers of point-estimate stories said a range
'of risks was mentioned (58.5% in.Study 1, 48% in Study
2, 53.5% in Study 4). A lower but still striking fraction
.of range-story readers (from 17% to 29% for the three
uncertainty conditions in Study 1; 20% in Study 2 and
16.7% in' Study 4) reported that only one number .ap-
peared in the story. •'."-,
These results may be due to a combination of con-
. .fusion and lack of comprehension.'All stories, including
the single-number stories, also contained a risk compar-
ison (see earlier language). Readers of single-number
stories may have inferred from remembering two num-
bers—for example, the 1:1,000 point estimate of risk and
the 1:4 general cancer risk—that these comprised a
range. The rarer error of classifying range-stories as con-
taining a single number is more likely due to incompre-
hension, because both the text and the graphic portrayed
a range. The consistency of these error rates across three
-------
490
Johnson and Slovic
studies (1, 2, and 4) suggests that, except for removal
of the risk comparison, further revision of the stories and
questionnaire would not greatly reduce the error rate for
well-educated respondents for whom the issue is not im-
mediately salient.
These results do not mean people could not rec-
ognize the experimental manipulations of uncertainty.
Study 2 (although not Studies 1 or 4) found that readers
of stories with ranges of risk estimates were more likely
than those reading single-estimate stories (86% vs. 42%;
p < .001) to say that a "very great*' or "moderate"
range of risk was reported in the story. The "range"
groups were also more likely to rate the story's risk in-
formation as uncertain (5-7 on a seven-point scale), by
54% to 28% (p < .02).
Uncertainty Effects on Perceived Risk. Study 1
found no statistically significant differences in perceived
risk across the four uncertainty levels (whether point es-
timate, or a range of—for example—1:1,000,000 to 1:
1,000). Readers of Study 2 range-stories were more
likely than point-story readers (34% vs. 14%) to rate the
butydin risk as high (5-7 on a seven-point scale; p <
.05). We think this was due to a higher top risk in range-
stories: a maximum of 1:100 for the point estimate of 1:
1,000, and of 1:100,000 for the other point estimate of
1:1,000,000. If one focuses only on .the range maxima
it is reasonable to see the range-story risk estimates as
higher than in the point stories.19' This idea was con-
firmed by the reasoning of one participant in Study 3:
I ignored the fact that there was zero risk because they
wouldn't have reported it if'there was zero risk ... for some
reason this graph looked more government-like, and so I im-
mediately went to worst case scenario
Another participant said, "if it is zero, they are just re-
ally saying, 'We could be wrong.' " (Note that Study 1
found similar risk perceptions with a lower-bound esti-
mate of zero and one that is a small positive number;
Table 1 has examples of these variations.)
Uncertainty was also more worrisome in Study 2:
73% of the "range" readers, versus 58% of the "point"
readers, were "somewhat" or "very worried" (p <
.01). About 71%-Of the "range" group in Study 2
agreed that the agency's discussion of uncertainties
would have made them more concerned about the risk
had they lived in this imaginary town. Somewhat more
than half of all Study 3 members, who read both stories,
were more concerned about the risks described in the
range-story than those in the point-story. (However, be-
cause the range included zero risk, one person noted that
"people who don't want to worry about this are going
to find plenty of support for not worrying about it.")
Studies 1 and 4 found no differences in worry by un-
certainty condition.
In Study 4, whether one received uncertainty infor-'
mation significantly predicted responses only to a ques-
tion about the perceived risk to Eugene residents. On a
scale from 1 (very low risk) to 7 (very high risk), those
reading range-stories had a mean rating of 3.14, to 2.58
for readers of point-stories (p <.01).
Effect of Uncertainty on Perceptions of the Agency
That Provided the Risk Assessment. Participants who
read Study 1 and Study 4 stories with varied levels of
uncertainty did not differ in their views of USEPA:
Within the Study 2 group that read a range-story and
recognized that it contained a range of estimates (TV = -
56), 66% agreed' that the agency's discussion of how
much the risk might vary made it seem more honest;
29% disagreed. Some 34% agreed that this discussion
made the agency seem less competent; 59% disagreed
with the statement. However, these findings need not,
contradict the findings of Studies 1 and 4: Study 2 judg-
ments of agency honesty and competence correlated
with attitudes about risk, government and authority, im-
plying that people were not reacting to uncertainty in-
formation (see discussion of regression analyses, below).
In Study 3, most participants felt that providing a
.range was more honest. For example,
• [The range approach] tends to see the public as competent,
educated citizens, who are going to have more information,
who are going to have to make up their own minds, which
I think is a good first step for the government to do. It hasn't
done it in the past most of the time.
• I assumed vast uncertainty even when'it was presented-as
• an absolute fact, so ... I guess it is more encouraging to
see it [in a range].
• I think it's a little more honest when there might not be any
risk and there might be a high risk or a higher risk.... I feel
more comfortable with something like this than I do with
... a number like a risk 1 in 1,000, or whatever. Just like a
definitive number where ... I can't even imagine how they
come up with something like that.
Yet several people in Study 3 said they would not
be upset by agency silence about a range if they did not
know such a range existed (most were unaware of un-
certainty in risk estimates). For some, however, the
range evoked doubt about agency trustworthiness: '
• They may give you the idea that there is zero risk, if they
don't have any way to clean it up. ... If ... they got the
money, and they are going to clean it up, and they want to
look good, and they want to do PR and stuff, they are going
to tell you there was this huge risk and we're going to take
care of it. And I just think it's all so politically motivated
.that it doesn't really mean anything anyway. " • s
-------
Uncertainty in Health Risk Assessment
491
•Despite-the general sense of honesty evoked by
ranges of risk estimates, several Study, 3 members felt
that USEPA's report of a range meant that "they don't
have a clue." Among other comments:
.• It bothers me when there are a lot of maybes and who'
knows. '.-'.''
* I didn't think much of their ability to be precise....
• Their preliminary results were too preliminary.
Even if reporting uncertainty seemed to convey
agency honesty, this did,'not seem to offset concerns
about the agency's competence:
• I kind of assume that the government doesn't know what
they are doing most of the time .At least they are finally'
admitting that they don't know what is up.
• [Person 1:] How would you feel if the government... told
you that 'they have no idea whether this is going to pose a
risk of cancer to you or not. And they really just are having
a hard time with studies determining it. [Person 2:j Yeah,
thank you for being honest.
• [about the story's line that "the true risk could be as low as
zero":] How come you can't even figure out if there is a •
risk or'not? You say it causes cancer. Well, is there a risk
. or is there not a risk? I don't know, it just bothered me.
• The honest imbeciles: The EPA.
In Study 4, exposure to uncertainty information
predicted (in multiple regression analyses) lower per-
ceived competence of USEPA at risk assessment and
environmental management, and mare apparent truthful-
ness in news stories, than for those reading point-stories.
Despite their very weak statistical significance (p <£ .10
or• p < .20), these findings' direction fits with the ' 'hon-
est but ignorant" theme of >the focus group discussions.
Nor is this inconsistent: Honesty and competence are
logically independent-attributes.
Uncertainty in Science. Study 4 subjects were asked
whether they agreed that "It is typical of good science
that the most likely estimate of what is being measured
has a range :of uncertainty around it." Item intercorre-
lations were' analyzed for the 56 subjects who read the
range stories and correctly-reported the story as contain-
ing a range of agency risk numbers. These correlations
showed that those agreeing with this statement were
more likely to'find the risk information in the story un-
derstandable (r = .36,'.p <'.01), certain (r = .27, p
<.05), and scientifically valid (r = .39, p < .001). They
were less likely to think that the agency's discussion of
uncertainty indicated incompetence (r = — .25,p < .05);
and less likely to be concerned because of that discus-
sion (r = -.34, p <.01).
Some "focus group members (Study-3) thought that
reminding people about science's imprecision would
.make ranges of risk estimates more credible:
Sometimes a" little disclaimer that reminds people that no matter
how many tests you do you can never be positively sure will
remind people that ... they are doing the best they can. And
I would think that that would help me assess that at least they
are being honest about the fact that they'are really not sure
what risk this poses.
However, the presence of such a caveat (with other
"clarifying" information) in Study 4's uncertainty sto-
ries did not evoke significantly higher ratings of EPA's
honesty than in the same, study's point estimate stories.
Graphics. For all conditions in Study 2, use of a
graphic significantly increased the perceived range of
risk—perceived uncertainty—described in ' the stories
(3.06 for graphic, 2.8 without, on four-point scale from
"no" to "very great range," p < .04). It also decreased
the perceived trustworthiness of story information (mean
of 331 for graphic, 3.84 without, on seven-point scale
from "not trustworthy" to "trustworthy,"p <..01). Yet
reaction to graphics in the Study. 3 focus groups was
generally positive: For example," the graphic with the
certainty story (see Figure la) was said to make the story
clearer and more salient. •
Natural vs. Technological Hazards. Study 1 stories
about zydin (natural radiation) elicited significant rank-
ings of lower risk, less wprry, more understandable and
honest information, and a more honest agency than were
"elicited by stories about butydin, the imaginary chemical
from an abandoned hazardous waste site, despite iden-
tical risk estimates for the two hazards. This finding is
consistent with previous studies that have found lower
perceived risk from natural hazards than for technolog-
ical hazards/22' . ,
Risk Magnitude. Significantly different reactions
occurred to differing risk estimates. In Study 1, risk es-
timates of 1:1,000 drew higher ratings of honesty, for
both the news story information and the agency, than
did 1:1,000,000 estimates. This parallels an earlier find-
ing that people would believe a government agency
more if it said there was an environmental problem than
if it-said there was no problem/13' Study 2 did not rep-
licate this finding about perceived honesty. However, the
higher probability (1:1,000) in the story did evoke more
perceived risk and more worry, and (while'not quite sig-
nificant atp < .05) greater expressed intention of getting
the site cleaned up. Lower probability (i.e.; 1:1,000,000)
was seen a's preliminary rather than complete informa-
•tion. Study 4 subjects who read the 1:1,000 range story
were significantly more likely than thqse reading the.
1:1,000,000 range story to say that the agency's discus-
sion of uncertainty made them more concerned. They
also rated the risks as better known to the government
.than did-readers of the lower risk range story/These
-------
492
Johnson and Slovic
results, with earlier research,*13' suggest that citizens see
lower risk numbers as either less accurate or less honest.
However, Study 4 found no significant differences
across risk magnitude conditions in ratings of agency
honesty, agency competence in risk estimation, or
agency competence in managing environmental prob-
lems.
Attitudes. The results reported so far were based on
direct manipulation of the presentation of uncertainty in
Studies 1, 2 and 4, and qualitative analysis of focus
group discussions in Study 3. We also conducted step-
wise regression analyses to examine effects of the un-
certainty manipulations relative to the effects of attitudes
about risk and agency honesty and competence. These
analyses used responses from the 100 (Study 2) and 135
(Study 4) respondents who correctly answered questions
1 and 2, thus indicating they recognized the experimen-
tal conditions (single risk estimate vs. range of risk es-
timates). After respondents with missing data were
deleted, effective sample sizes were 37-38 for Study 2
and 62-67 for Study 4. These samples were small; each
of nine regression analyses found different independent
variables to be significant; and these analyses combined
experimental (uncertainty) with nonexperimental (atti-
tude) data. For these reasons, we will not discuss these
results in detail, but we believe they carry important im-
plications worth further study.
Six dependent variables were used: perceived risk,
worry, experts' ability to correctly estimate risk, EPA's
honesty about the size of risks, and EPA's competence
at estimating risk magnitudes and managing environ-
mental problems (the last three for Study 4 only). In-
dependent variables were (a) the respondent's
uncertainty condition (0 = single number; 1 = range);
(b) various attitudes toward risk, government, and au-
thority; (c) sex; (d) gender-related adjectives respondents
used to describe themselves™; and (e) being active in
various environmental activities.
Within the context of the regression analyses, ex-
posure to uncertainty stories significantly predicted only
one variable: In Study 4 it increased perceived risk to
Eugene residents. Responses to other dependent varia-
bles were predictable from attitudes toward risk, gov-
emmeht, and authority, and gender-related adjectives
(sex itself was not a significant predictor). For example,
those prone to judge EPA as honest in reporting the size
of environmental risks did not see risks as serious in
their home community, trusted the government, disa-
greed that more equality would solve social problems,
and described themselves as gentle. Respondents who
judged the EPA competent to deal with environmental
problems were less likely to see local environmental
problems as serious, more likely to trust the government
to manage risks, and more likely to describe themselves
as gentle leaders without strong personalities who were
not active in environmental groups. The best predictor,
entering first in five of the nine regressions, was the
belief that "There are serious environmental health
problems where I live."
If supported by further research (one study found
similar trust, equality, and risk attitudes linked to low
risk ratings among white men, compared to others),'21'
these findings would indicate that uncertainty is less im-
portanf than general attitudes about risk and .authority in
shaping views of risk and agency performance.
4. DISCUSSION
Results of these initial studies of public response to
uncertainty in risk assessments raise more questions than
they answer. However, some tentative conclusions can
be reached:
• People are unfamiliar with uncertainty in risk as-
sessment, and with uncertainty in science gener-
ally. Responses by respondents in Studies 3 and 4
• point to unfamiliarity with uncertainty in science. Up
to 20% of respondents in, Studies 1, 2, and 4 had
difficulty even recognizing the presentation of uncer-
tainty in the form of a range of risk estimates.
• People may recognize uncertainty (i.e., a range of
risk estimates) when it is presented simply. Moving
from Study 1's percentage-based, one-tailed presen-
tation of uncertainty to Study 2's probability-based,
two-tailed presentation did produce some responses to
uncertainty manipulations. Yet many participants in -
both studies cpuld not correctly categorize risk esti-
mates (as a single number or a range). Moreover,
Study 4, intended to enhance Study 2's uncertainty
effects, failed to show any statistically significant ef-
fects of uncertainty except in a'-regression analysis for
perceived risk to people in Eugene, Oregon. Because
our exploratory regressions implied a greater impact
of attitudes about risk and government on views of
risk and agency performance, the univariate analyses
for Study 2 may overstate the effects of uncertainty.
• Graphics had mixed results in communicating un-
certainty. Those used in Study 2 made the range of
estimates more obvious, but made the story informa-
tion seem less trustworthy. Because the latter finding
was experimental—people reading a story with a
graphic rated its trustworthiness lower than did people
reading the same story without a graphic—we cannot
say whether people consciously saw the graphic as
being (or as ,a sign of the story being) untrustworthy.
-------
Uncertainty in Health Risk Assessment
493
Focus group members found the same graphics useful,
but their comments indicated that crafting clear, help-
ful graphics would not be easy (Ibrekk and Morgan^
argued that no graphical display of uncertainty will
be clear to everyone). Visual displays of the relative
probabilities of various risk estimates being correct
may have different impacts than our graphics, which ,
merely stressed that,there was 'a range of estimates.
• People's views on the environmental cases pre-
sented in the stories may have been influenced less
by uncertainty manipulations than by attitudes to-
ward risk, government, andf authority. Trust in
government and attitudes toward authority have been
identified in the research literature as important, if not
dominant, factors in perceived risk. With the meth-
1 odological caveats noted earlier, this view seems to
be supported by findings from exploratory regression
analyses using data from Studies 2 and 4. Such atti-
tudes as personal risk aversion,. trust in government
risk management, and .support for equality strongly
affected reactions to the stories, while uncertainty had .
a significant effect in only one of the nine regressions.
• Agency discussion of uncertainty in risk estimates
1 , appears to signal agency honesty. Effects of the ex-
perimental manipulations (not regressions) in Studies
2 and 4, and comments in Study 3, support this find-,
ing. The focus groups' reactions combined surprise
• that government would offer any unsought informa-
tion, belief that all information is desirable, and sus-
, picion (among a", few, anyway) that precise risk
estimates cannot be believed. Yet several comments
•about possible cover-ups suggest .some people may
find declarations about uncertainty a signal of dishon-
esty. Past experience (direct or through the mass me-
dia) with agencies actually or seemingly using risk
assessment to delay pollution cleanups may fuel this .
suspicion. Or reactions to uncertainty may be shaped
more by prior views of agency honesty. ..' - .
• Agency discussion of uncertainty in risk estimates
may be a signal of incompetence for some people.
One-third of Study 2 range-story readers said the
agency seemed less compfetent when discussing un-
certainty, this may be due to unfamiliarity with sci-
ence: If science is deemed certain, uncertain risk
estimates must come from' incompetent scientists.
Comments in Study 3 about uncertainty being ex-
• pected (and acceptable)'only for "preliminary" risk
.estimates also suggest citizens find it hard to fathom
. that competence and uncertainty can co-exist. Again,
prior trust or distrust of, agency competence may
shape reactions to uncertainty rather than the reverse.
• Estimated risk levels may affect views of expert
•, knowledge.That "low" estimates were deemed more
"preliminary," whether uncertainty was mentioned or
, not, implies that risk assessors may have difficulty
'communicating their estimates even if citizens deem
these to be honest. This response may be due to dis-
trust (no government estimate that low could be true).
Or it may stem from feeling that a 1:1,000 risk esti-
mate must be based on high, and thus tangible, inputs
(e.g., for exposure), while a l:.l,000,000 risk estimate
is based more on guesswork.
Our studies used words and graphics to present a
range of risk estimates, in simulated news stories, to
communicate uncertainty about two hazards (natural ra-
diation and a chemical in drinking water from a hazard-
ous waste site). Study 1 used an upper-bound point
estimate, and both zero and positive lower-bounds; the
other studies' ranges went from zero to an order of mag-
nitude above the point estimate. Experimental respondents
read only one story, while Study 3 participants directly
compared point and range stories. Although they pre-
ferred the latter, this may not have been a good measure
of point- and; range-estimates' respective ability to meet
public needs and skills. Only one source of uncertainty
(extrapolating from animal data) was described, in Study
4. , -. - . .', '.. \ . "' ".
Despite the various tests we used, other presenta-
tions may have, stronger, or different, effects on. per-
ceived risk, hdnesty, and competence. A short list might,
include other descriptions or sources of .uncertainty;
graphics conveying the relative probabilities of a given
risk estimate (e.g., a probability density function plotted
directly, below its cumulative distribution function, with
the mean clearly marked on each curve'23');4 the source
of the risk estimate;, the source of the hazard; who is
threatened; presentations of variability; interactions of
uncertainty with agency actions affecting trust, or con-
flict among environmental policy actors over risk esti-
mates and uncertainties. We expect to address some of
these issues in future research.
Communicating about uncertainty is necessary, be-
cause it is a reality of risk assessment and risk manage-
•ment, and-all parties—including citizens—deserve access
to full information. Our results should not be taken as
proof that iaypeople do not grasp uncertainty. They may
not understand it in the way scientists do; they may also
not have understood our specific presentations of uncer-
••The value of testing the hypothesis that these graphics provide useful
risk^ communication information would depend in part on the likeli-
: hood that rislc assessors could routinely provide the information
needed to generate these graphics.
-------
494
Johnson and Slovic
tainty. These hypotheses do not mean they lack experi-
ence with uncertainty, in their own lives' or in
environmental matters. Yet no one should assume that
"educating" citizens on scientific uncertainty will be
simple; given the above-average education of our sam-
ples, their members may have given the most
sophisticated responses to our simulated news stories
that one could expect.
Given our initial results, we advise caution in as-
suming that explaining uncertainties will improve public
trust or knowledge. In the long run, such explanations
may make citizens' expectations of government, risk as-
sessors, and scientists more realistic.5 Yet overall public
trust and knowledge on risk issues may have to be built
with methods more direct and difficult than uncertainty
explanations.
ACKNOWLEDGMENTS
Research funding was provided under Cooperative
Agreement No. CR820522 with the U.S. Environmental
Protection Agency. We are grateful for the assistance of
Dr. Lynn Desautels, USEPA project officer. Draft news
stories for Study 1 were reviewed by Dr. Adam Finkel,
then of the Center for Risk Management, Resources for
the Future, although he is not responsible for the stories
actually used. Stephen Johnson and C.K. Mertz provided
valuable assistance with data collection and analysis.
REFERENCES
1. American Industrial Health Council, Improving Risk Characteri-
zation (American Industrial Health Council, Washington, D.C.,
1992).
* Among options suggested by anonymous reviewers:,agency reporting
of uncertainty may force reporters to be more probing and balanced
even if they do not report risk estimates; citizen outrage and alien- ,
atton over reporting of risk ranges may ultimately yield more respect
for science's limits and reduce undue expectations of risk assessment
and government; uncertainty analysis may replace single point esti-
mates with a variety of point estimates chosen to illustrate (when
lakcn together) the underlying distribution, so that only experts need
deal with ranges, probability distributions, and the like; and a more
extended discussion of uncertainty with citizens—unlike the "snap-
shot" of lay reactions in these studies—may correct their assump-
tions that science is certain. These proposals are thoughtful, perhaps
even right. Yet they imply that positive effects will come only from
discussing uncertainty publicly—in general and for specific cases—
for prolonged periods (years?). Because uncertainty is hardly men-
tioned now in government announcements about risk, and long-term
dialogues on scientific uncertainty are infeasible in the way that
government and citizens currently interact (e.g., in one-time public
hearings), it is unclear how these outcomes will occur without so-
cietal reforms that go far beyond simply discussing uncertainty.
2. B. B. Johnson and P. Slovic, Explaining Uncertainty in Health
Risk Assessment: Effects on Risk Perception and Trust (Phase 1
Final Progress Report, Cooperative Agreement No. CR820522,
U.S. Environmental Protection Agency, 1994).
3. A. M. Finkel, Confronting Uncertainty in Risk Management: A
Guide for Risk Management (Resources for the Future, Washington,
D.C., 1990); S. 0. Funtowicz and J. R. Ravetz, Uncertainty and
Quality in Science for Policy (Kluwer, Dordrecht, 1990); M. G.
Morgan and M. Henrion, Uncertainty: A Guide to Dealing with
Uncertainty in Quantitative Risk and Policy Analysis (Cambridge
University Press, New York, 1990). . . " .
4. B. J. Hance, C. Chess, and P. M. Sandman, Improving Dialogue
with Communities: A Risk Communication Manual for Govern-
ment (New Jersey Department of Environmental Protection, Tren-
ton, 1988), pp. 69-73, 83.
5. F. H. Habicht, Guidance on Risk Characterization for Risk Man-
agers and Risk Assessors (Office of the Administrator, U.S. En-
vironmental Protection Agency, Washington, D.C., 1992).
6. Risk and the Environment (Carnegie Commission on Science,
Technology, and Government, New York, 1993).'
7. R. J. Bord and R. E. O'Connor, "Determinants of Risk Percep-
tions of a Hazardous Waste Site," Risk Anal., 12,411-416 (1992).
8. R. J. Bord, R. E. O'Connor, and D. J. Epp, Communicating Cu-
mulative Long Term Risks (Report to U.S. Environmental Protec-
tion Agency CR816305, Pennsylvania State University, University
Park, 1992).
9. W. K. Viscusi, W. A. Magat, and J. Huber, "Communication of
Ambiguous Risk Information," Theory Dec.,,31, 159-173 (1991).
10. P. Slovic, S. Lichtenstein, and B. Fischhoff, "Modeling the So-
, cietal Impact of Fatal Accidents," Mgmt Sci., 30,464r474 (1984).
11. B. Wynne, "Sheepfarming After Chernobyl: A Case Study in
Communicating Scientific Information," Environment, 31, 10-15,
33^tO (1989),
12. P. Slovic, B. Fischhoff, and S. Lichtenstein, "Response Mode,
Framing, and Information-Processing Effects .in Risk Assess-
ment," in Hogarth, R. (ed.), New Directions for Methodology of
Social and Behavioral Science: Question Framing and Response.
Consistency (Jossey-Bass, San Francisco, 1982), pp. 21-36.
13. N. D. Weinstein, Public Perception of Environmental Hazards:
Statewide Poll of Environmental Perceptions (Final Report to the
New Jersey Department of Environmental Protection, Rutgers •
University, New Brunswick, NJ, 1987).
14. P. Slovic, "Perceived Risk, Trust, and Democracy: A Systems
Perspective," Risk Anal., 13, 675-682 (1993).
15. B. B. Johnson, P. M. Sandman, and P. Miller, "Testing the Role
of Technical Information in Public Risk Perception," RISK: Issues
Health Safety, 3, 341-364 (1992).
16. P. M. Sandman, P. Miller, B. B. Johnson, and'N. Weinstein,
"Agency Communication, Community Outrage, and Perception of
Risk," Risk Anal., 13, 589-602(1993). ' .'
17. N. Kraus, T. Malmfors, and P.'Slovic, "Intuitive Toxicology:
Expert and Lay Judgments of Chemical Risks,',' Risk Anal., 12,
215-232 (1992).
18. G. L. Carlo, N. L. Lee, K. G. Sund, and S. O. Pettygrove, "The
Interplay of Science, Values, and Experiences Among Scientists
Asked to Evaluate the Hazards of Dioxin, Radon, and Environ-
mental Tobacco Smoke," Risk Anal., 12, 37-43 (1992).
19. A. Wildavsky and K. Dake, "Theories of Risk Perception: Who
Fears What and Why?," Daedalus, 41-60 (1990).
20. S. L. Bern, "Sex.Role Adaptability: One Consequence of Psycho-
logical Androgyny," J. Person. Soc. Psycho!., 31, 634-643 (1975).
21. J. Flynn, P. Slovic, and C. K. Mertz, "Gender, Race, and Perception
of Environmental Health Risks," Risk Anal., 14, 1101-1108 (1994).
22. A. Baum, R. Fleming, and L. M. Davidson, "Natural Hazards and
Technological Catastrophe," Environment and Behavior, 15, 333—
354(1983).
23. H. Ibrekk and M. G. Morgan, "Graphical Communication of Un-
certain Quantities to Non-Technical People," Risk Anal., 7, 519-
529 (1987).
------- |