Lay Views on Uncertainty in Health Risk Assessment:
              A Report on Phase II  Research
                           Branden B. Johnson
               New Jersey Department of Environmental Protection

                     •  -   ,   x
                              Paul Slovic.
                           Decision Research          :
                            Eugene,  Oregon .
                             Final Report
                             Prepared for:
                    U.S. Environmental Protection Agency
                   Office of Policy, Planning, and Evaluation
                       Risk Communication Project
                    Cooperative Agreement No. CR-820522
                     Lynn Desautels, EPA Project Officer
                            August 5, 1996

-------

-------
                                  Contents

Executive Summary                                                            1

Background                                                                    3

Method                                                                         6
  interviews                                •                    •                  7
  Focus Groups            , v                                  '                   8
  Open-Ended Questionnaires'                 .                      '                 9
  Closed-Ended Questionnaire             - >/                                       10

Results                                                                        13
  General Responses to Environment^ Health Risk Uncertainty                   '            13
  Responses to Range Bounds   ,     ..                      .         '.           l  -17
  Reasons for Uncertainty.    '                                                 •      22
  Uncertainty and Risk Management -                        .                           3'5
  Understanding of Science and Risk Assessment                                         38
  Potentially Mediating Factors                   '.   "                                ,42
  Multivariate Analyses    - •                           ,                              47
  Importance of the Independent Variables                    •'            .              56

Conclusions                                                                   58

Practical Implications                                                         64

Research Implications                                                         68

References                                .                                    70

Appendix I: Fourteen Items Used in Qualitative Data
Collection                        ,

Appendix II: Presenting Uncertainty In Health Risk
Assessment: Initial Studies Of Its Effects On, Risk
Perception And Trust

-------
Acknowledgements
    This research was funded by Cooperative Agreement No. CR-820522 with the U.S.
Environmental Protection Agency. We would like to thank Steve Johnson for conducting the
focus groups, Donald MacGregor for consulting on the layout of the closed-ended questionnaire,
and C.K. Mertz for statistical analysis. Drs. Alan Stern and Robert Hazen of the New Jersey
Department of Environmental Protection were consulted on the technical accuracy of some
interview stimuli. The -authors, however, take full responsibility for the data collection materials
used. The views herein do not necessarily represent the views of either the U.S. Environmental
Protection Agency or the New Jersey Department of Environmental Protection.

-------
Lay Views on Uncertainty in Health Risk Assessment   •-   "'•''..                           ;page {'

                            Executive Summary
     It will be a challenge to present uncertainty iri environmental health risk estimates to the
public in ways that inform, rather than outrage,- this important audience. Although such
presentations may increase citizens' risk knowledge and trust in the honesty and competence of
institutions providing risk estimates, careless communication could have undesirable results.
     In Phase I of this research, people had no response to certain experimental presentations of a
range ofrisk estimates; in other experiments, two-thirds of the sample saw the range estimates as
a signal of government honesty and competence. We needed mo^e study of how laypeople
thought about uncertainty in risk assessment, and its implications for risk management, to
untangle these mixed findings,                •                               '
     Phase ,11 research included intensive interviews and open-ended ^questionnaires in New
Jersey and Pennsylvania, focus groups in Oregon, and a long closed-ended questionnaire given to
280 Eugene, Oregon, residents (largely college students). We presented uncertainty in the form
of a range of risk estimates, primarily in a hypothetical case of a chemical in drinking water.
     The following responses appeared among our participants (Eugene sample, unless'otherwise
specified):                                          . .-   •
     •  About 45% doubted the honesty of the government in presenting ranges ofrisk estimates,
      ''a higher proportion than in the earlier research.    '
     '•  About a third disagreed that government was less competent for discussing ranges ofrisk
       estimates.                  , .  ' •:                                ;
  .'   •  On average, people supported presentation of uncertainty information. A third of our
       sample strongly desired certainty, rejecting risk ranges or quantitative risk estimates of
       any kind.                         •                      .
             -*•          ,        '    ' -                   -         '
  ' ' •  Zero as a lower bound for a range of risk estimates was seen on average as an indicator of
    '   either incompetence or dishonesty, whereas a very small positive lower bound (e.g:, 1  in
       10 million) had much less effect, even though a majority saw no difference between zero
       and a 1 in 10 million lower bound. Those who did see a difference seemed to doubt that
       the government would discuss risks if they were truly zero.

-------
Lay Vie\vs on Uncertainty in Health Risk Assessment                           .   •             page 2

     •  On average, people tended to see the high end of a risk range as being more likely, even
       with a lower risk being labeled as "most likely."
     •  Some, but not all, interviewees and focus group members volunteered reasons for
       uncertainty, such as differences in exposure and personal susceptibility. However,
       explanations of uncertainty as due to extrapolation from high-dose data (e.g., from
       workers, or due to an accident) or from animal experiments confused, irritated, or
       disturbed them. Eugene responses to animal and high-dose extrapolation explanations
       were no different; an overwhelming-majority agreed with scientist-offered reasons for.
       uncertainty, but this seemed to be a rationalization of distrust rather than true agreement.
     •  People said they would doubt a lower risk estimate from a second study more than they
       would a higher risk estimate, but most felt more confident about the safety of drinking
       water after reading a scenario about a lower risk estimate.
     •  When faced with hypothetically overlapping risk estimates from government and
       environmentalists, people did not observe the overlap or interpret it as meaning anything;
       however, in the abstract they said agreement among these two groups (or of academics
       with government) would make them suspect that the environmentalists arid academics,
       had been corrupted.
     •  People in interviews and focus groups did not see environmental health uncertainties as
       having any relationship to financial, recreational, or transportation (mortality)
       uncertainties.
     •  Trust in officials was the most common predictor of people's reactions to government
       risk ranges.              •,           .          .  ,.
     •  Although a substantial minority of our sample had difficulties with numbers,
       mathematical  literacy played a very small role in their reaction to risk uncertainties.

-------
Lay Views on Uncertainty in-Health R'isk'Assessment '"                            •           page j


                                  Background
  ;  Several government, industry, and academic commentators have urged communication of
uncertainties in environmental health risk estimates to public and other, lay audiences, as we
noted in our final report for Phase I (Johnson & Sloyic, 1994a) and a subsequent published
article (Johnson & Slovic, 1995). Such arguments continue to be made. Communication of
uncertainties is needed, it is said, to avoid conveying information that "is incomplete and may be
misleading" (Carpenter, 1995, p. 127). Narrative statements of "statistical reliability" should be
made even if it "is not easy for scientists to make" them (Carpenter, 1995, p. 132). A recent
National Research Council report (1994) advised U.S. Environmental Protection Agency (EPA)
to communicate exposure or variability information in a three^part format: the estimated risk, the
level of confidence that the risk is no greater than this estimate, and"the subpopulation to which
the estimate applies. Administrator Browner's memorandum on EPA's Risk Characterization
Program (1995) said that "a balanced discussion of reasonable conclusions and related
uncertainties enhances, rather than detracts, from the overall credibility of each assessment."
     Since our earlier report, we have found a few dissenting voices on the benefits of conveying
uncertainty. Arguing that discussing uncertainly in risk estimates would give homeowners an
excuse to, avoid testing for natural radon in their homes, academic researchers, state and federal
officials", and nonprofit groups have urged that radon communications present  "an unambiguous
'united front'" (Garrison, 1991; USEPA, 1992, ch. 6, p. 6; Weinstein, Sandman, & Roberts,
1989, p. 18). We cannot definitively refute the argument that communicating uncertainty about
radon risks would have made the testing rate worse. However, with about 5% of U.S. homes
having been tested, the unambiguous and forceful U.S. media campaign does not appear to have
had much  better results than lower-key campaigns in Sweden (4%) and Finland (2%) (Cole,
1993, pp. 72, 179, 181-182). Further, we used a radon analogue ("zydin") in a Phase I ' .  .
experiment on uncertainty. Zydin elicited significantly lower ratings of risk and worry than a
Superfund site chemical at equal levels of estimated cancer risk.  Yet intentions to get the
problem solved did not vary, by either hazard or the level of uncertainty in the risk estimate. If
these results can be generalized, they imply that communicating uncertainty in radon risk
estimates would not have depressed radon testing frequency any further.            .

-------
Ltiy Vieva on Uncertainty in Health Risk Assessment                  ,.        •                page 4

     Various other criticisms of communicating uncertainty have been made. The President's
Commission on Risk Assessment and Risk Management sent a letter to House leaders December
14, 1994, that raised concerns about H.R. 9, the Job Creation and Wage Enhancement Act.
Among these concerns was that "providing for 'best estimates' of risk along with plausible
upper- and lower-bounds on those estimates may not be much improvement on current methods"
(Riskpolicy report, 1995, pp.  12-13). A prominent risk assessor (and member of the President's
Commission) independently assailed mandated communication of risk uncertainty, arguing that a
range of risk estimates might be misinterpreted as implying equal likelihood of all estimates,
calculating such ranges in a consensual way would be very difficult and time-consuming, and no
other policy-related numbers—including cost-benefit analyses being considered for a mandate
under the same legislation mandating risk characterization, as well as other economic estimates.
(e.g., of gross domestic product or health care reform costs)—are expected to come with
uncertainty estimates (Goldstein, 1995, pp. 1602-1603,1609). In less explicit language,
MacGregor, Slovic, and Morgan (1994, p. 827) suggested that "even careful presentation of the
current 'complicated' ... state of affairs [concerning scientific knowledge about electromagnetic
fields] will increase people's concerns."
     This continuing debate triggered our initial studies, and led us to conclude that more
research on public response to uncertainty should be conducted. The findings of our earlier
research (Johnson & Slovic, 1994a, 1995) were that:
     •  People are unfamiliar with the notion of uncertainty in risk assessment, and with
        uncertainty in science generally.
     •  People may recognize uncertainty (i.e., a range of risk estimates) when it is presented
        simply.
     •  Graphical presentation produced mixed results in communicating uncertainty, making a
     ,   range, ofestimates more obvious, but causing the information to seem less trustworthy.
     •  People's views on the environmental cases presented in our experimental stories may
        have been influenced less by variations in uncertainty than by personal attitudes toward
        risk, government and authority.
     •  Agency discussion of uncertainty in risk estimates appears to signal agency honesty.

-------
Lay Views on Uncertainty in Health Risk Assessment     .  '                                     page 5





     •  Agency discussion of uncertainty in risk estimates may be a signal for some people that


       the agency is incompetent.                 ,


     •  Low risk levels were deemed to be "preliminary" estimates by laypeople, possibly

               '                                  s        '       ;'-"-•
       implying distrust.    •                                                    ..



     Our Phase I studies used several experiments to see how people would respond to


uncertainty in the form of a range of risk estimates: two hazards (natural radiation and a chemical


in drinking water from  a hazardous waste  site), various upper- and lower-bounds, and graphics.


Two focus groups also  directly compared  their reactions to point and range estimates. In pur


conclusions, we pointed out that other presentations of uncertainty may have' stronger, or


different, effects on perceived risk,  honesty, and competence. For example, only one  source of


uncertainty (the difficulty of extrapolating from animal .data) was described in one of our


experiments; more focus on explanation might affect lay responses.



     However, we concluded that it would be premature to test such variations without a better


understanding of how citizens conceive of uncertainty in environmental health risk assessment.


Rather than employing experiments, where one infers thoughts from behaviors, our present goal


was to begin with qualitative methods (interviews, open-ended questionnaires, focus  groups) to
    --   .         '     •      : 'i                  i   ,'-•-.,.    -     '      - „ '   -

see the world from the lay participant's viewpoint. Once we can describe how.citizens might


interpret a description of uncertainty, we might be able to suggest useful contextual and focal.


information for use by organizations wishing to convey uncertainty in health risk assessment to


their audiences.

-------
Lay Views on Uncertainty in Health Risk Assessment                                       '     page 6


                                        Method
     Our earlier findings suggested that uncertainty in risk assessment was a novel concept for
laypeople. This implied that the approach to revealing "mental models" of,risk-related topics
pursued by Bostrom, Fischhoff, and Morgan (1992) would not work. In that approach one asks
people to  "tell me" about the topic (e.g., radon) and uses follow-up questions to elicit details of
the person's cognitions that aren't volunteered initially. Asking people to tell us about
"uncertainty in risk assessment" did not seem likely to be fruitful.
     We, therefore, provided stimulus items that people could read and react to (see Appendix I
for 14 examples). These items used one to six sentences to briefly introduce a range of risk
estimates, or to explain how risk assessors identify a "safe" level for human exposure to a
noncarcinogen. In contrast to the simulated news stories of our earlier work,! these items
included only information that identified the hazard and described 'or explained uncertainty as a
range of risk estimates. Once a stimulus was read, we began by asking "what thoughts,
questions, and so forth, come to mind when you read this?" Only after allowing this open-ended
response,  and asking (in interviews) for more details about this response, were more focused
questions asked. The intent was to minimize biasing responses through interviewer behavior "or
survey construction.                                                             .
     We collected information through interviews, focus groups, open-ended questionnaires, and
two versions of a closed-ended questionnaire. All stimulus items used in focus groups and
questionnaires dealt  with a government announcement about a chemical in the person's drinking
water (whether tap or bottled water). Another seven items used only in interviews probed
reactions when the same risk information concerned contaminated soil or air in people's
neighborhood, or automobile, or airplane crashes.
1 Experiments in Phase I research (Johnson & Slovic, 1994a, 1995—Appendix has copy of latter) used simulated
news stories about environmental problems, with headlines, quotations from government spokespersons and local
citizens, and so forth. The content of these stories varied. The problem'might be chemicals from an abandoned
waste site or natural radiation; a point estimate or a range of estimates of risk might appear; and so on. Each
participant read one of these stories. By comparing their responses across different stories, we could see whether
hazard type, mention of uncertainty, and so forth, made a difference.

-------
Lay Views on Uncertainty in Health Risk Assessment                            .               page 7

Interviews                                                                         j
     Thirteen people were interviewed in twelve interviews (one interview included both
members of a cpuple, at their request) in January 1995. The median interview took 45 minutes
(range 30-120 minutes) and involved responses to seven items (range 4-11). Nine interviewees
were women; four interviewees were in their 20s, five in their 30s, and the rest up to age 79. Two
interviewees were black, with the rest white; five were single, two widowed, and the rest
married. Four interviews were conducted with nonscientist employees of the New Jersey
Department of Environmental Protection'in Trenton, the rest with'residents of an apartment
complex in a Philadelphia suburb. Interviewees' occupations included personnel administrator,
secretary, paralegal, florist,  nurse, university administrator, home health care administrator,
dentist, entertainer, and risk communication specialist; two retirees had been in business and
engineering.  '      /                 '                        '  '           :
     A brief introduction included a request to read and answer each stimulus item as if it
concerned the water that people actually drank, including bottled water for the two people who
relied on it exclusively (see Exhibit 1 for examples; others appear in Appendix I). Each
interviewee was then handed a sheet of paper on which the initial item was printed and asked to
read it and respond with any thoughts, questions, or comments that occurred. Probes were used to
elicit further comments, or to clarify earlier comments. When the interviewee indicated that he or
she had  no more thoughts to offer on an item, that sheet of paper was retrieved and the next one
proffered. At the end of 30 minutes—the time span promised when people were first contacted—
     , \        * •   .        .         -                           •                        '
the interviewer asked interviewees whether they wished to continue. That only .two people chose
to stop at this point, and that four took an hour or more, suggests that the topic—despite
repetitiveness in the stimuli—captured people's attention.  '_
           f'              '             '               ' •       .    '            ...
  '  No interviewee read every stimulus item. The reasons for this were that there were too many
items to be covered in even the longest interview, and there was no need to ask the same
    ' i          -       '              •        "      '         , '   •  .   ,  . ~
questions if, after several  interviews, no new responses were being heard. Thus each item  was
presented to a median of five interviewees (the range was 1-7, out of 12 interviews). Overall,
there was no attempt to present items in the' same order, because this (given time constraints)
would have prevented some items from ever being presented.                      ,

-------
Lay Views on Uncertainty in Health Risk Assessment
EXHIBIT 1. Examples of Interview Stimulus Items  	•	
 Item 2a
 The government announces that the water you drink contains a level of
 chemical X that poses an extra health risk of getting nonfatal kidney damage
 of 1 in 1,000,000 (one in a million) over a lifetime of drinking that water.
 Item 11 a               .
 The government has found a chemical in your drinking water that can cause
 nonfatal kidney damage in laboratory rats. The level of the chemical which
 the government considers safe for humans to be exposed to is 1 part per
 100 million. The level of the chemical in your drinking water is smaller than 1
 part per 100 million.
Focus Groups
     Two focus groups—each comprised of three women and one man—were conducted on
February 21 and March 7,1995, in Eugene, Oregon. The first group included employees and
volunteers of United Way of Lane County, while the second comprised clients and staffers of a
job retraining program jointly run by the county and private industry, The aim was to reduce the
educational level of members, relative to those in our earlier focus group research. Interview
results were used to design the focus group protocol.
     Each member of the focus groups read three items on risk uncertainty (see Exhibit'2), and
answered a questionnaire about them, before coming to the focus group session. This approach
allowed us to'tap their individual views as'well as those they  would express in a group
environment; overall these views did not differ from those in interviews or (for Item 3 only)
open-ended questionnaires. Besides  reviewing these responses, issues raised by the focus group
facilitator included (a) the effect of zero versus positive lower bounds; (b) use of similar or
different denominators in the probabilities presented; (c) extrapolation from animal data; (d)
extrapolation from high-dose human data; (e) exposure to average concentrations below-standard
versus slightly above-standard; (f) explanation of Reference Concentration calculations;. (g)
    i                                              '                     .   *       .    -    -
reaction to a graphic (see Figure 1) on confidence bounds in excess risk relative to varying
                                               ' l
exposure; and (h) how reaction to uncertainty in environmental risks compares to uncertainty in
financial or safety risks.

-------
  Lay, Views, on Uncertainty in Health Risk Assessment                                 _            page 9
     ,   .        "                    \ ••'•.•'     •        •        •       •         .

  EXHIBIT 2.  Focus Group Questionnaire Items         	   • .   '  ,     ,                       ,  .

   items   "   ../•  .'.    ••;..,    '.".".•,"•.'..'.•''•'     ••   ''  ;.      .''"''   -:..-''.•
   The government announces that the water you drink contains a levelof.                                 ,
   chemicalX- that poses an extra health risk of getting cancer in 1 in 1,000,000
   (one in one million) over a lifetime of drinking that water. The government
   says that is the most likely risk, but it says the true risk could be as low as
   'zero or as  high as tin 100,000.

   Item 5         .                       .    ';                   ..'.,.
   The government arinounces that there is a 5% chance that the extra level of
   risk from drinking chemical X in the water you drink for your entire lifetime is
   above 1 in 100,000 (one in one hundred thousand), and a 5%'chance that   '      .        ,   .
   the risk is zero. The government says this means there is a 90% chance that
   the true risk is between zero and tin 100,000.                   "•'..-

   Item 8     ,  .         .      ,
   The government announces that its scientists have calculated the risk of
   getting cancer from drinking the water you drink, which has a small amount
   of chemical X in it, for an entire lifetime. It is 95% certain that at least 90% of                       . ,
   the population has an extra risk of no more than 1 in 1,000,000 (one in one'~
   million). A small proportion of people who are more likely than others, when
   exposed to a'cause of cancer, to get cancer may have a risk as high as 10                            '     '
   in 1,000,000 (ten in one million) if they drink this water for their entire lives.    .
  Open-Ended Questionnaires
"   -    A wider variety of items (12, rather than 3), but similar to those given to focus group ;

  members, were distributed to volunteers from the jury pool for Essex County, New Jersey. Four

  versions of the. questionnaire, each including three unique questions, were created; 20 copies of

  each version were distributed. Of the 80 questionnaires distributed, only 29 were returned, and

  only 18 included substantive responses. The "nonsubstantive" responses included answering

  only the sociodemographic questions, providing one or two answers and then apparently losing

  interest, and providing answers whose content (e.g., "none," "good thing") seemed to indicate a

  superficial or irrelevant response to the questions.

       Most respondents (24 of 29) gave enough sociodemographic information to compare those

  who gave substantive responses to those who did not.  Overall, men and women were equally

  numerous (11 to 13), although women dominated in the substantive group (10 to 6). Median age

  was nearly identical (45 and 44) and ranged from 20 to 67. The sample of 17 whites,-4 blacks,

  and 1 Hispanic was equally divided among substantive and peripheral responses (79% vs. 75%

  white).  The main difference between the two groups was education: 75% of substantive

  respondents had a college degree or better, while five  of the peripheral respondents had a high

-------
Lay Views on Uncertainty in Health Risk Assessment
                                                                                   page !-0
.J2
tn
§

     1 in 100

    1 in 1,000

   1 in 10,000

  1 in 100,000

 1 in 10 million

1 in 100 million
                                       50%     ,
                             Half of the population has higher
                          exposure, but half has lower exposure
                             Percent of exposure to risk
                                                                , U (95%certain)
                                                                  B
                                                                  L (5% certain)
                                                                 39.9%
                                                           90% 99%
                We are 90% certain (95% minus 5%) that the risk to a person with
                high (90%) exposure is between 1 in 1 million and tin 333.
Figure 1. Graphic presenting variability in exposure, shown to focus groups; adapted from NRC
(1994, pp. 10-25).

school diploma or less, and the other two members in this category reported their educational
background as "some college."                              .

Closed-Ended Questionnaire
     Results from the preceding data collection were used to develop a questionnaire that could
give'us a broad view of lay beliefs and attitudes about uncertainty in environmental health risk
estimates. Material was added from focus groups in our earlier research, scientific literature on
uncertainty,'and hypotheses stimulated by our earlier research. Statements were also drawn from
the literature and elsewhere that might measure some potentially mediating variables (e.g.,
attitudes to uncertainty in daily life, ideology, behavior regarding  drinking water). No attempt

-------
,   Lay Views on Uncertainty in Health Risk Assessment          •               '     ' '       '  • page

   was made to be comprehensive or rigorous with such mediating variables, because our primary
   goal was to understand lay concepts about uncertainty in environmental health risk assessment,
  'not to explain differences in these concepts. Most statements were designed with a four-step,
   strongly disagree-strongly agree Likert response format. The sequence of items in thfe closed-
   ended questionnaire included the following:                             ,
       •  mediating factors   '                                                    ,
      ,•  questions about their drinking,water
       •  a scenario on a range of drinking water risk estimates and an explanation for the range
         (extrapolation from either animal or high-dose data)                  .,
      . v responses to the scenario
       •  attitudes toward receiving no risk estimate (e.g., being told they are safe or unsafe), a
         point estimate, or a range of estimates                                        .
       /        '       '         •                           ',-""        ' '      '      •     '
       •  miscellaneous uncertainty statements                        ,         '  '  .
       •,  a scenario on two estimates of risk over time, and responses to'it         .  ,  "
       •  sociodemographic information         ,
      After pretesting, the questionnaire was answered in late November of 1995 by 280
    • •_-,-.                                          .         . "'    .  I            •  ' '  ,
, respondents to an advertisement in the University of Oregon newspaper; This group was 50.7%
  male, with a median age of 20 (range 17-36). Three-quarters (74.6%) had some college, with
   14.7% having a bachelor's degree or better. Most .(79.3%) were white, with 8.6% labeling
  themselves as Asians or Pacific Islanders, 2.9% as Native American, and 1.4% as black. Only
  2,5% were of Hispanic origin, although 5.4% did not answer this question. About 14% had one
  or .more children living at home, and 13% were affiliated with an environmental group. Split
  samples read animal (49.6%) or high-dose (50.4%) extrapolation explanations of the scenarios'
  range of health risk estimates. For 127 questions these two groups (readers of animal or high-
  dose, scenariosjliad only three significant differences (p< .05). Given that this small proportion
  of across-scenario differences  could have arisen by chance, and occurred only once for each of
  three different topics, these differences are probably not reliable. Thus we combined all 280
                                                                  ' >
  responses for analysis.

-------
Lay Views on Uncertainty in Health Risk Assessment                                          page (1

     Reviewers of our earlier research suggested that use of mostly student respondents in
Eugene, Oregon, limited generalization of our results. Phase II interviews and open-ended
questionnaires used a more diverse, New Jersey/Pennsylvania population, and focus group
members who were less educated than in earlier research, but these were few in number. Non-
Eugene, nonstudent responses did not differ qualitatively from those of Eugene students. It is
unlikely a less educated audience would grapple more easily with the concept of uncertainty in '
risk estimates or science, or have a more positive reaction to it, although generalization from
responses to the closed-ended questionnaire should be done cautiously. People with lessxthan a
college education were very unlikely to answer our open-ended questionnaire; whether due to
incomprehension  or disinterest in the topic, or less socialization to answer questionnaires, this
may imply that we would fail to get adequate answers from a more diverse  group. However, a
much shorter instrument based on our closed-ended questionnaire might be used with a more
                                                                          I
diverse population (like our New Jersey jury pool members) to test these hypotheses.

-------
Lay Views on Uncertainty in Health Risk Assessment        >      .    •.    ,                    page 13


                                       Results
     In the discussion below we group qualitative and quantitative (closed-ended questionnaire)
results by topic. Lay-proffered concepts are distinguished from those derived from, the literature
on uncertainty and earlier hypotheses of ours. In contrast to sketchy evidence (from this study
and others) that citizens have a clear (if incomplete) concept of environmental problems and
government management of those problems^ they seem to lack a structured view of uncertainty
about risk estimates. We thus will not offer a detailed lay "mental model" of uncertainty like
that suggested for radon (Bostrom et al., 1992).   ,                                         '•
     Results are, discussed in the following order, to facilitate understanding: general responses
to environmental health risk uncertainly; responses to range bounds; reasons for uncertainty;
uncertainty, and risk management; understanding of science and risk assessment; potentially. .
mediating factors; and multivariate analysis.              ,

General Responses to Environmental Health Risk
Uncertainty
     Here we discuss responses to the relation of health risk uncertainty to other kinds of'risk;
willingness to see risk ranges; and reactions to a drinking water scenario involving a range of risk
estimates. •.                  ,  ; •            .    :     ;           ,               '

Uncertainties  in Environmental Health Risks Seen as
Dissimilar to Uncertainties for Other Risks
     It has been suggested that the value for improved  decision making of assessing risk
uncertainty can be demonstrated with the example of investment uncertainties (Finkel, 1994).
The author's goal seemed as much to educate risk policymakers as citizens, and he did not make
explicit the implication that investment and health risks (and thus their uncertainties) are similar,
at least in their  value for decision making. We thought a brief (because this was not our central
goal) .exploration in interviews and focus groups of whether citizens see such analogies as valid
might show'whether they could be used to educate people about uncertainty in health  risk
assessment;         ,                            •

-------
Lay Fiews on Uncertainty in Health Risk Assessment          -                  .              page 14

     First, we examined in interviews whether reaction to uncertainty was.common across
environmental health risks. Some interviewees were asked whether their reactions to the risk
estimate ranges would change if the hazard was not a chemical in drinking water, but instead a
chemical in neighborhood soil or in the local air from a nearby factory. Some people felt water
offered more personal control, and thus less personal worry, because there were more feasible
protective actions (e.g., switch to bottled water vs. move out of the neighborhood). Others said  .
that one could move away from polluted soil, while water was needed for survival. The air-,
factory scenario elicited particular outrage, as implying immoral behavior on the part of business.
These immediate reactions seemed  to be based on factors unrelated to uncertainty, suggesting
that such variations might occur across environmental hazards regardless of whether point or
range estimates of risk were used.                                                    /
     Second, parallels between responses to uncertainty in environmental and nonenvironmental
risks were rejected by both interviewees and focus group members. A comparison of the drinking
water scenario with the same range of risk estimates for airplane crashes was dismissed by
interviewees. Their grounds were that air travel was occasional, familiar, and (if frequent) a
probable requirement of one's job, while drinking contaminated water was unfamiliar and a
matter of daily exposure;  similar responses occurred for uncertainty in automobile risks. Focus
group members said recreational risks were chosen,(on the basis of joy garnered from the
activity), and financial risks (in investing or job security) were not irrevocable,  whereas'
environmental health risks were imposed by others and death permanent. Thus  uncertain
recreational and financial risks were acceptable, usually without formal or (allegedly) conscious
weighing of risks and benefits; uncertainty in environmental health was not. These'reactions
suggest, but do not prove, that using a stock market investment example to show the superiority
of choices made with understanding of uncertainty  (Finkel, 1994) may not work with lay
audiences facing^incertain environmental risks and estimates.
    i                    .                          "<-•-'
Some Willingness to  Receive Risk Estimate Ranges                      -
     Interviews yielded several statements that people would prefer only a single risk estimate
rather than a range. For example, two people offered diametrically opposed views: "Government
should announce its preliminary calculations for an environmental health risk, no matter how

-------
  Lay Views on Uncertainty in Health Risk Assessment  '     .          •               '       -      page .'3



  uncertain these numbers are" and "Government should never announce a risk number about  •

  anything until they know for sure exactly how risky it is." We included 13 such statements (six

  from interviews, the rest from the literature) about preferences for single risk numbers or for no

  numbers at all in our closed-ended questionnaire. Table 1 shows those questionnaire statements

  that loaded high on a single dimension in factor analysis, and thus were used to produce a scale

  for multivariate analysis (a = .80). As can be seen, our hypothesis that aversion to numbers (of

  any  kind) and aversion'to ranges of risk estimates would be distinct concepts was rejected.

 , Further, a "desire for certainty" was not dominant in this sample (Weinstein, 1987); about 35%

  rated at or above the middle score on this index. Strong desire for certainty in this minority may

  explain the slightly positive net response to statement Id, because otherwise this sample seemed

  distrustful of experts and officials (see below).                                               '


  TABLE 1. Desire for Certainty      	,	.    	;•.'''..   	__.

                                                           t    Disagree      Agree     Don't know

   1 a I prefer being told that a situation is safe or unsafe, rather than         53.2%    ,    42.1%         4.6%
      hearing>risk numbers, such  as "a health risk of getting cancer of 1 ,
      in 1,000,000,"                                             ,           '                        -

   1b If the government is having  difficulty in. determining how much of       18.9         74.3          6.8
      a risk an environmental condition poses to me, I prefer they give
      me a range of numbers rather than give me a single risk number.'       - '  ,                  •   .-
      (R)   '             ' '..   •    '.      '-...,          - '     .'.-..--    :.  ':

   1c When a chemical is discovered in my drinking water, I don't want      47.1         48.9    ,     3.9
     to hear statistics, I just want to know if my water is safe.                                     ,

   1d I am more comfortable with  an expert's opinion about whether or       42.9         53.2          3.9
      not my water is safe than with a range of risk numbers from which    '
      I must draw my own conclusions.*                                              ,

   1 e I would prefer that government tell me that they're just not sure        51.4         43.9   "      4.6
     about the size of an environmental health risk, if that's the case,       .             -
     rather than giving me a range of risk numbers.*

   1f I'd prefer a single, concrete risk number rather than a range of         61.4         30.7          7.9
     numbers for the environmental health risks I face. ,

   Note. * Item (quotation  or paraphrase) derived from Phase II interview; data from Eugene, Oregon, questionnaire.
-. - (R)  Reversed scoring for construction of CERTAIN index. Cronbach's alpha = .80.
  Mixed Reactions to the Uncertain Risks Scenario
       Respondents to the closed-ended questionnaire were asked several questions after reading

  one of the initial scenarios (see figures 2 and 3), with a caution to assume that '"the term 'water'.

-------
  qt Views on Uncertainty in Health Risk Assessment
               page 16
refers to the water you drink, whether it is bottled or tap water" (during interviews bottled water
drinkers tended to assume "pollution" scenarios only concerned tap water). Table 2 shows the
answers, including indices created from statements loading high on the same dimensions.
Responses to items in the NO WORRY index (items 2h, 2i, and 2j) indicates that people felt the
risk was low and they would continue to drink the water. However, the CONCERN index (items
2c and 2d) showed that they were concerned about the effects of drinking this water and
committed to getting it cleaned up.
     Table 2 also,suggests mixed reaction to the government agency in the scenario
(GOVHONEST index, items 2e, 2f, and 2g). Nearly a third of the respondents could not decide
whether the government was honest, and a plurality doubted it was "telling the truth." Iii Phase I
research, about two-thirds of our Study 2 sample felt discussion of a range of risk estimates
signaled government honesty. In the study reported here, equal numbers agreed and disagreed
with this proposition. However, as in the earlier research, only about a third felt such discussion
made the government seem less competent and 45.7%  disagreed that the discussion made the
agency seem more honest.
FIGURE 2. Initial Scenario, Version 1
Assessing the Risks of Water Quality
 Next consider risks from the water you drink, whether it is bottled or tap water.
 Suppose that a government studyfound that the water you drink in your home
 contains a level of a particular chemical that poses a health risk of getting cancer of 1
 in 1,000,000 (one in a million) over a lifetime of drinking that water.
     Most likely risk
     1 in 1,000,000
' Lifetime extra cancer risk
 The government study indicates that this is the most likely
 level of risk, but the study also finds that the true risk could be
 as low as zero or as high as 10 in 1,000,000.
        Highest risk
      10 in 1,000,000
 The government says that the reason for this range of risk numbers is that the only scientific studies of risks from
 this chemical involved laboratory tests with animals. Scientists disagree about whether the way an animal reacts to
 a chemical will reliably predict how a human would react to the same chemical. This is because different
 assumptions about how to predict human risks from animal reactions to the chemical result in the range of risk
 numbers shown above.                                                       -
 Read each of the items below and respond by circling the appropriate number or checking the appropriate box.
 Assume that the term "water" refers to the water you drink, whether it is bottled or tap water.

-------
Lay Views on Uncertainty in Health Risk Assessment
                                            page {7.
FIGURE 3. Initial scenario, Version 2, New Last Paragraph
 The government says that the reason for this range of risk numbers is that the only scientific studies/of risks from
 this chemical involved cases where people were exposed to much larger amounts of the chemical in their drinking
 water than appear in your drinking water (at the time they were exposed, no one knew this chemical could affect
 health). It is not clear that the cancer-causing-effects in these people of high levels of the chemical reliably predict
 how a human would react to the much lower levels of the same chemical in your water. This is because different
 assumptions about how to predict risks from low levels of the chemical, when only information about risks from high
 levels is available, result in different risk numbers.                                •"'.'.
TABLE 2. Scenario Responses Statement
' . - - - l' ' /
2a The information in the scenario above is not understandable.
2b Government knows exactly what the level of risk is.*
CONCERN index (a = .63) -
2c I would actively work to get my water supply cleaned up.
2d 1 would be concerned about the effects of drinking this water.
GOVHONEST index (a = .74) .
2e The government is telling the truth. .
2f The government's discussion of the range of possible risk levels
makes the government seem less competent. .(R)
2g The government's discussion of the range of possible risk levels
makes the government seem more honest. ,
NOWORRY index (a = .74)
2h This water poses a serious health risk. (R)
2i 1 would continue drinking this water. •
Somewhat or
' considerably
. • - - . ' higher
Disagree
73.6%
62.5

31.4
26.8

40.4
56.8
45.7

68.2
21.4
About the
same as
other health
risks
Agree
20.4%
26.8

61.8
70.7

29.3
• 32.5
44,6

26.8
73.9
Somewhat or
considerably
lower >
Don't know
' 6.1%
10.7

6.8,
2.5

30.4
10.7
9.6

5.0
4.6
Don't know
• 2j Compared to other health risks in my life, the
    risk from drinking the water discussed in the
    scenario on the previous page is;..
8.6%
19.6%
.71.4%
0.4%
 Note. * Item (quotation or paraphrase) derived from Phase II interview; R = item reversed in index; Cronbach's
 alpha for indices in parentheses; data from Eugene, Oregon, questionnaire.
               -^W    '               •  •       ,     -   .        • ,                         . • •
Responses to Range Bounds         ,
     Presenting uncertainty through ranges of risk estimates by definition requires upper and

lower bounds to the range. This raises questions about how people might react to these bounds

-------
Lay I'iews on Uncertainty in Health Risk Assessment                                    .     page !R

(e.g., by anchoring to one or the other extreme, and basing their response on that number rather
than on the range). We present results here suggesting an upward bias among some people and a
seemingly distrustful view of zero as the lower bound of a range of risk estimates.

Upward Bias in Assessing Ranges of Estimates      .
    Earlier experiments had people pick a level of known risk that left them indifferent between
living in the area subject to that risk and in another area of ambiguous risks. Some respondents
showed a strong bias toward anchoring on the upper bound of a range of risk numbers (Viscusi,
Magat, & Huber, 1991). We tested this bias hypothesis in two ways.
    First, we asked about agreement with the statement "If a range 'of environmental health risk
numbers is given, I would believe that the highest risk number is the, correct one," which had
been volunteered by a Phase II interviewee. Half (46.4%)  of our respondents said they agreed
with it, although nearly as many (43.2%) disagreed (10.4% did not know). If we take agreement
with this statement as a valid measure of "upward bias," Viscusi etal. (1991,p. 163) founda
much smaller bias than we did. For a range of disease risk between 110 and 240 cases per
million, 13 of 58 (22.4%) picked the highest number of the range. We do not know whether the
wider spread in their range (130 in 1,000,000) than in ours (10 in 1,000,000) affected the relative
upward bias. We should note that an even larger upward bias  occurred in response to scientific
conflict (see "Disagreement Among Groups and Scientists").
    Our second means of testing the Viscusi et al. result produced less dramatic results. We
asked readers of our initial scenario  about cancer risk from drinking water to judge the size of
this risk in two ways, for the community and for themselves.  The scenario (presented in figures 2
and 3) said that the "most likely risk" was 1 in 1,000,000, but that "the true risk could be as low
as zero or as high as 10 in 1,000,000." Table 3 shows the  proportion of sample members who
estimated the cosnmunity and personal risks from our scenario at various levels of risk. When we
    i                                         "                       .*''*.
add the proportion who assessed the risk as equal to the upper bound  in the scenario (10 in
1,000,000) and the proportion who assessed the risk as higher than this level, we see that 14.3%
(7.5% plus 6.8%) judged the community risk at or above the upper bound of the government risk
estimate. Similarly,  10% (4.3% plus 5.?%) judged the personal risk at or above the government's
upper bound estimate. We would  expect only 5% (rather than 10-15%) of our respondents to

-------
Lay Views'on Uncertainty in Health Risk Assessment        .         ;                 .        .pa?e 19

estimate risks as this high if the government's" upper bound is a 95% confidence limit, and there
was the same distribution of estimates among citizens as in the government analysis. These
results also suggest a tendency among our respondents to see larger risk estimates as more likely.
However^ the discrepancy between this finding and the proportion who claimed they assumed
higher risk numbers were more correct prevents us from estimating the prevalence of this upward
bias in the population.                                        '               ,
TABLE 3. Respondent-Estimated Risks of Initial Scenario
Risk Estimate
Zero [lower bound of scenario]
Between zero and 1 in 1 ,000,000
' 1 in 1 ,000',000 ["most likely risk" in scenario]
Between 1 in 1,000,000 and 10 in 1,000,000 .
1 0 in 1 ,000,000 [upper bound of scenario]
Higher than 10 in 1,000,000
Don't know .
Community
1.1%
11.1,
12.5
52.5
' 7.5
6.8
8.6
Personal
6.8%
15.4
17.5
40.7
- 4.3
5.7
9.6
 Note. Respectively, questions were "The risk to the community from the scenario
 on the previous page is.'.." and "My personal risk from the scenario...is most
 likely to. be...." These formed ESTRISK index, a =.73, with data from Eugene
 sample.              '         •     -
    •These results raise questions about the hypothesis that "those not familiar with quantitative
methodology" would misconstrue each number within a range of risk estimates as "equally
probable" (Goldstein, 1995, p. 1602). Some 89% of the Eugene sample whose answers appear in
Table 3 have at least some college education. Although some are less comfortable with
mathematics than this level of education might imply (see later discussion), these results do not
seem to reflect a widespread equal-probability view. Because the questions were phrased as "The
risk to the community from the scenario .-.. is" and "My personal risk from the scenario ... is
most likely to be," we were encouraging participants to pick only one of the options listed in
Table 3. If they expected each risk number to be equally likely, we would expect them to have
selected "I don't know," because we did not offer an "equally probable" choice. Clearly, most
people-did not make this selection. Our results may have differed if we had had access to a less

-------
                                                                                            •j
Ley Hews on Uncertainty in Health Risk Assessment                               ,         . page 2Ci

educated sample, but at this point we would lean to ward rejecting the equal-probability
hypothesis.                                                    ,                     ,

Zero as a Lower Bound Risk Estimate Created
Problems
     A National Research Council (NRC) committee noted (1994, pp. 9-23) that USEPA has     :
typically used the standard phrase that a risk "could be as low as zero." The committee said this '
might be misleading in a given case, and current information allowed for more accurate (in many .
cases nonzero) lower bound estimates. A reviewer of our previous article on public responses to
uncertainty felt our findings might have been affected by use of the same 'terminology in our
scenarios. People might think "zero" indicated risk assessors' ignorance, and he felt a small but
                                       f.
positive lower bound might yield different results. On the other hand, experiments with college
students suggest that people treat small incidence numbers (e.g., less than .000003) as essentially
zero (Stone, Yates, & Parker, 1994). One of our earlier studies did include positive (one in ten
thousand and one in one million, bracketing the Stone et al. figure) and zero lower bounds,
without prompting significant differences in response.
     We presented interviewees, focus group members, and jury pool members with alternative
lower bounds. Among their open-ended responses were the following:
     •  government never announces an environmental issue if there truly could be zero risk
       involved, even if they say the risk could be zero    .                                •   ,
     •  when government says about environmental issues that "the true risk could be as low as
       zero," they are really saying "we could be wrong"             -•                       ;
     •  if the risk of getting an environmental health problem could be zero, one doesn't worry
       about it
     •  if one is told that an environmental risk could be zero or could be higher, one tends to
    1  assume that the risk is zero
     •  the fact that government says the risk could be zero means to me that it's probably not
       zero.
     Clearly different people offered different, and therefore sometimes contradictory,
comments. They agreed that it would make no difference whether the lower bound of a range of

-------
Lay Views on Uncertainty iri'Health Risk Assessment                       •          '   •    •  page?f

risk estimates-was zero, or 1 in 10,000,000, but disagreed on the reasoning. Some,saw the two
•numbers as essentially the same, others saw the latter as a risk that should be eliminated.
     To understand the scope of these contradictions and' disagreements, we turned four of these
volunteered phrases (shown in Table 4),into statements for our closed-ended questionnaire. Each
statement occurred with two different numbers—zero and 1  in 10,000,000;—with otherwise
identical language. Depending on the statement, 56-72% of participants gave the same response
to both items in each pair, thus apparently treating these alternate lower bounds as  identical. The
majority, treating very low risks as effectively equal to zero, confirmed.the findings of Stone et
al. (1994),  .
TABLE 4. Responses to Lower Bounds

4a The fact that government says the risk could be
means to me that it's probably not 	 . *

(p < .0001)
4b When government says about environmental
problems that "the true risk could be as low as
," they are really saying "We could be wrong." *
(p<.0002)
4c If the risk for humans of getting an environmental
health problem could be 	 , I don't worry about it.
* ' - -
i .
(p<.0001) • ,. , .
4d If I'm told that an environmental health risk could .
be __ or could be higher, I tend to assume that the-
risk is higher than 	 . * , «. •
(p<.0001)
'Risk
Z
T

Z
: T

Z
T

Z
T

Disagree
; 29.3%
50.7

' 28.6
36.4

53.2
37.9 '.

9.3
26.1

Agree
55.7%
34.6

58.9
53.6

41.4
58.9 .

87.5
62.5

Don't know
15.0%'
14.6

: 12.5
10.0

5.4
3.2

3.2
11.4 •'.

 A/ofe. Risk: Z = zero; T = 1 in 10 million. * Item (quotation or paraphrase) derived from Phase II interview;
 data from Eugene, Oregon, questionnaire.
   .  However, close examination of Table 4' shows that not everyone reacted to the two numbers
in the same way^More people agreed that zero, versus 1 in 10 million, as a lower bound meant
that government could be wrong (item 4b), the true risk was not at that level (item 4a), and the
true risk must be higher (item 4d). Thus use of zero as a lower bound 'seemed to evoke doubt
about the risk estimate, and an upward bias jn perception of the risks.

-------
 Lay Views on Uncertainty in Health Risk Assessment                                          page 22

     Statement 4c (Table 4) presents an apparent exception. Respondents were less likely to  ,
 worry if the lower bound was 1 in 10 million than if it was zero. Because a zero risk is
 objectively lower than a 1 in 10 million risk, there should be less worry about zero risk. These
 results are thus consistent with those discussed above, concerning doubt about the government's
 risk estimates if they include a lower bound of zero. Given this doubt, the higher risk number
 may seem more trustworthy, and thus less worrying. Phase I research (Johnson &  Slovic, 1994a,
 1995) found people rating lower risk estimates (1'in 1,000,000) as "preliminary" information
 significantly more often than higher risk estimates (1  in 1,000). Together these -findings suggest •
 that, everything else being equal, the public assumes that higher risk numbers are either more
 honest or more competent (e.g., have been based on a more thorough, less "preliminary,"
.analysis).
     Overall, these results show that use of the term "zero" as a lower bound had  a significant
 impact on the Eugene sample's responses to a range of risk estimates. The National Research
 Council (1994) recommended that government agencies avoid the default language of "as low as
 zero" in reporting ranges of risk estimates, unless the data did not allow alternatives. The NRC
 committee making this recommendation felt that agencies making full use of available
 information would rarely have to resort to zero as a lower bound.2 We believe our results suggest
 that communication,  as well as risk analysis, would benefit from avoiding zero as  a lower bound
 on risk estimate ranges.

 Reasons for Uncertainty
     General unfamiliarity with the concept of uncertainty in health risk assessment makes
 laypeople's understanding of reasons for such uncertainty difficult to plumb. However, because -
 anyone wishing to communicate about uncertainty would want to take audiences'  understanding
 of these reasons into  account, we did some exploration of this topic. First, we review some
 evidence on people's reaction to explanations of uncertainty that concerned extrapolation from
 * Other technical reasons exist for avoiding ranges of risk estimates that include zero. For example, suppose there, is
 uncertainty about whether an animal carcinogen is also a human carcinogen. Two risk estimates for human health
 are then possible: zero if the chemical is not a human carcinogen, or a range of risk estimates whose  lower bound is
 a small positive number., According to this analysis, which was provided to us by Dr. Alan Stem of the New Jersey
 Department of Environmental Protection, it is improper to have a range of risk estimates that includes zero.

-------
, Lay Views on Uncertainty in Hetilth Risk Assessment  •.,-.'.'                       page .23

 either animal data or human data involving high doses, of a chemical. Then we discuss how they
 respond to shifting risk estimates o'ver time, disagreement over risk estimates among groups
 (e.g., government vs. environmentalists) or scientists, and expert-proffered reasons for scientific
 uncertainty about environmental conditions (e.g., in measurement, due to complexity, or about
 past or future risks).   -"•'   .   '<      ".       .'-•'..'   .             ."       "-•'.•

 Little Impact of Explanations of Uncertainty
      If ranges of risk estimates are offered by institutions, as many risk assessors'suggest,
 explanations of why there are ranges (i.e., uncertainty) would seem necessary for lay       .  ..
 understanding and acceptance. Except in Study 4 (where explanation of extrapolation from
 animal data had no observable impact), such explanations were largely absent from Phase I
- experiments. We decided to explore the impact of several different explanations on people's
 response to uncertainty in this phase of research.
      The bulk of our effort focused on explanations concerning extrapolation from animal and
 high-dose data. The versions used in interviews,  focus groups, and open-ended questionnaires
 appear in Exhibit 3. Reading these items did not  seem to help people either understand or accept
 the reasons for having a range of risk estimates. .Most who read them rated the explanations as
                        .-             . •                  .-  i
 irrelevant to the main issues of risk management (e.g.,, cleaning up the pollution), confusing, and
 too wordy. The few people \yho reacted positively seemed to be more satisfied with the
                                             - * ,   ',           * ..  '              • '  .
 extrapolation-from-animal-data explanation than by the extrapolation-from-high-dose
 explanation.         •.-"'-''           .•
      The interviewees who were less appreciative of the high-dose than the animal extrapolation
 explanation could not explain their feelings. However, anecdotes from sources outside this study  ,
 raised the possibility that.pepple might believe that risks to workers,  the population which often
 provides the high-dose data from which risk assessors extrapolate to lower doses, are not
 properly related to risks to,the general public. We asked our Eugene sample, to indicate their level
 of agreement with the following statement: "Suppose the government uses the effects on humans
 of high levels of a chemical (for example, among factory workers) tq predict the effects of lower
 levels of the phemical oh members of the public. The government is competent when it uses this
' method .to determine the size of an environmental chemical risk for humans." Although

-------
Lay Views on Uncertainty in Health Risk Assessment      ,                                        page 24
EXHIBIT 3. Examples of Interview Stimulus Items
 ItemS
 The government announces that the water you drink contains a level of
 chemical X that poses an extra health risk of getting cancer of 1 in 1,000,000
 (one in one million) over a lifetime of drinking that water. The government
 says that is the most likely risk, but the true risk could be as low as zero or
 as high as 1 in 100,000. The government says that the reason for this range
 of risk estimates is that the only scientific studies of this chemical's effects
 on cancer risks involved laboratory tests with animals. It is not clear that the
 way an animal reacts to a chemical will reliably predict how a human would
 react to the same  chemical. Different assumptions about how to predict
 human risks from  animal reactions to the chemical result in different risk
 estimates.

 Item 7
 The government announces that the water you drink contains a level of
 chemical X that poses an extra health risk of getting cancer of 1 in 1,000,000
 (one in one million) over a lifetime of drinking that water. The government
 says that is the most likely risk, but the true risk could be as low as 0.01 in
 1,000,000, or as high as 10 in 1,000,000. The government says that the
 reason for this range of risk estimates is that the only scientific studies of this
 chemical's effects on  cancer risks involved cases where people were
 exposed to much larger amounts of the chemical in their drinking water than
 appear in your drinking water (at the time they were exposed, no one knew
 this chemical could affect health). Itis not clear that the cancer-causing
 effects in these people of high levels of the chemical reliably predict how a
 human would react to the much lower levels of the same chemical in ypur
 water. Different assumptions about how to predict risks from low levels of the
 chemical, when only information about risks from high levels is available,
 result in different risk estimates.
this phrasing does not directly ask people about the propriety of high-dose extrapolation from

workers, it provides an indirect measure of this concern. Slightly more people disagreed (4L4%)

than agreed (33.9%) with this statement; mariy.(24.6%) said they did not know. Given these

answers, we can only conclude that the impropriety of extrapolation from high doses to workers

to low doses to members of the general public may be a factor in the qualitative negative

reception for our high-dose explanation.            ,                          .

    ' During early interviews, requests were made for an explanation of the high doses

"elsewhere" from which scenario risk estimates were supposedly extrapolated. Item 7 (see

Exhibit 3) about high-dose extrapolation was thus,modified to say that "an accident" had caused

the high doses. It was assumed that an accident would have less moral taint than legal (i.e.,

permitted) emissions, and thus would not distract people from the explanation of the range of

-------
Lay Views on Uncertainty in Health Risk Assessment   ''.'•••    -'  "          •        •    page 25

risk estimates. However, the 'few interviewees who had a chance to comment upon this
explanation after the "accident" modification seemed to take it as a reason to be more, rather
than less, afraid of the consequences of their own exposure to lower concentrations. This reaction
                            ••                     •                                 ^
may reflect a long-hypothesized (and debated) aversion to "chance" as a reason for troubles that
people like oneself suffer, since that seems to make one more vulnerable: ho.w can! guard
against chance making me a future victim (Shaver, 1970)? Interesting in this regard is the
response to the following statement in the Eugene questionnaire: "Suppose scientists use the
health effects of a high dose of a chemical on people exposed elsewhere by chance to predict the
effects of a much lower dose of the same chemical found in my water. That would make me feel
               ~ •    =   .                 r     '       -. >
more at risk than if,they got the same numbers for predicted health effects from just studying the
chemical's effect on animals." Somewhat more people agreed (40.4%) than disagreed (32.5%)
with this statement, which had been volunteered by an interviewee, and a substantial number
       • •               .   '       '           .                           f -            '  v
(27.1%) could not answer it either way. This is not proof that "chance" is an upsetting
explanation of risk alone, much less of uncertainty, but it raises questions for future practice and
research.-     ;                  '             •
     Explanations about extrapolation from animal and high-dose data also appeared in the
closed-ended questionnaire; half of our Eugene sample read one explanation, half read the other
(see Figures 2 and 3). We did not provide a "control" scenario (that is, one without,any
explanatory paragraph). O.ur focus here was on seeing-.whether these two. explanations had
different effects on responses to the scenario.  Out of 127 comparisons of the  answers given by
our split sample, only three significant differences were found (the questions on which these
differences appeared are in Exhibit 4).                  ,   .

-------
Lay Views bn Uncertainty in Health Risk Assessment     ,                                      page 26

EXHIBIT 4. Items Eliciting Significantly Different Responses From Readers of "Anifnal"
and "High-Dose" Extrapolation Explanations (from the close-ended questionnaire)
 • I prefer being told that a situation is safe or unsafe, rather than hearing  '  ,
   risk numbers, such as "a health risk of getting cancer of 1 in 1,000,000."
 • Government risk estimates tend to indicate the risk that an environmental
   pollutant poses for the average person.
 • The second study came up with a lower risk number because better
   scientific knowledge was available.
     If the proportion of comparisons finding a difference that is statistically significant does not
exceed the criterion of significance (p < .05 in this study), a rule of thumb is that these
differences might have arisen by chance and should be treated as suspect. Because three  '
significant differences out of 127 comparisons is less than the.criterion of 5%, we do not believe
the animal and high-dose explanations for uncertainty evoked substantively different responses.
Our confidence in this conclusion is heightened by the fact that the significant differences were
found for three different topics, and each of these topics also appeared in one or more other
statements for which significant differences in the two split samples were not found. If a
substantive difference in response to the explanations was occurring, but not meeting statistical
significance, we would expect it to be concentrated in one topic (if only three differences occur).
     A final explanation of uncertainty was offered during qualitative data collection, concerning
concentrations of a noncarcinogen in drinking water (the stimulus items concerned appear in
Exhibit 5). Each person read them in the same sequence: first the item with concentrations
slightly below the government-calculated "safe" level, then the item with concentrations slightly
above this level, and then the lengthy explanation. As can be seen, the latter concerns calculation
of the "reference concentration" for the noncarcinogen, extrapolated from animal data. The,
drinking water in this item also contained concentrations above  "safe" concentrations, to see if
the e,\planation~would offset the discomfort we expected respondents to have with these abo;ve
standard concentrations. With one exception, every person who read this explanation deemed it
irrelevant, confusing, overly complex, or arbitrary ("why divide by,  10?," as one interviewee put
                                                                                        l
it). The one exception was a woman who was very distrustful and upset'over the above safe
concentration; this explanation changed her attitude to one of feeling very safe and informed..

-------
Lay Views on Uncertainty, in Health Risk Assessment               .       •                  ,       page 27


EXHIBIT 5. Exploring the Impact of an Uncertaiqty Explanation in the Qualitative
Study.	.	'      	__	\	_.	"

 Item 11'a        '                                                  . •          .
 The government has found a chemical in your drinking water that can cause
 a nonfatal kidney damage in laboratory rats. The level of the chemical which        '
 the government considers safe for humans to be exposed to is 1 part per                       j
 100 million. The level of the chemical in your drinking water is smaller than 1
 part per  100 million.  .     '.     .    /     '

 IternUb '                  .            ..'•.."..-...              '.':.'.  .•_'.,•.-..:.
 The government has found a chemical in your drinking water that can cause
 nonfatal  kidney damage in laboratory rats, The level of the chemical which
 the government considers safe for humans to be exposed to is 1 part per
 10O million: The level of the chemical in your drinking water is 1.2 parts per
 100 million.   .                '    v                       .          ^
                                  • :         .  , •       _            ' ,  •    ' '
 Item 11c                                            •       :'-.'•'
 The government has found a chemical in your drinking water that can cause      .
 nonfatal  kidney damage in laboratory rats. However, this chemical did not          '          i
 cause kidney damage when fed to laboratory rats in doses of 1,000 parts          ,     .     ,
 per 100 million or less. Government scientists took a cautious approach to
 calculating a safe dose for humans, although it is possible  1,000 parts per
 100 million would be safe for them too. They divided this number by 10 to,                                 .
 account  for the fact that the rats were only fed the chemical for a short time,          ,,                     ,
 'and humans (at worst) might be exposed for an entire lifetime; divided by 10
 again to  account for the possibility that humans might be mpre sensitive to                     >.
 the chemical than are rats; and divided ,by 10 again to account for humans               ;
 who may be more sensitive to the chemical than the average human. Thus,
 the level of the chemical which the government considers safe for humans to
 be exposed to is 1 part per 100 million. The level of the chemical in your
 drinking  water is 1.2 parts per million.
Shifting Risk Estimates Over Time
 ,    Item 4 in the qualitative study (see Exhibit 6) dealt with changing estimates over time as

new scientific information became available. In general, temporal changes did not seem to alter

interviewees' concerns that there was any risk at all, although there seemed more distrust of

government reducing its estimates than increasing them over time. This reaction is consistent

with Weinstein's (1987) finding that government is more credible when it says that a situation is

unsafe than when it reassures people.          ,  .   .   •'                            .  .

-------
Lay
            Uncertainty in Health Risk Assessment
page 28
EXHIBIT 6. Example of Shifting Risk Item in the Qualitative Study	

 Item 4
 A government study was done of the water you drink a few years ago, which
 contained (and still contains) small amounts of chemical X. It found that the
 most likely extra level of risk of getting cancer from drinking this water for
 your entire lifetime was 1 in 100,000 (one in one hundred thousand). This
 was below the drinking water standard for this chemical at the time, so no
 action was taken and the amount of the chemical in the water has stayed the
 same. The government did another study recently, using new scientific'  .
 information about the chemical's effects, and concluded that the most likely
 level of risk was 1 in  1,000,000 (one in a million).             .
     The closed-ended questionnaire included a scenario (Figure 4) of a follow-up study to the

"original" risk estimate for health risk of cancer from drinking water (most likely 1 in

1,000,000; range of zero to  10 in 1,000,000). The scenario said that the government was trying

"to obtain a more accurate number." The result was a most likely estimate of one in a billion,

with a range from zero to  10 in a billion. The fact that the second study produced a lower risk

estimate was specified in the scenario, to avoid any mathematical confusion. Answers to

questions about this scenario appear in Table 5.
FIGURE 4. Second Study Scenario
 You will recall that the government study described earlier in this survey found that the water you drink in your
 home contains a particular chemical that poses a health risk of getting cancerpf 1 in 1,000,000 (one in a million)
 over a lifetime of drinking that water. Though this is the most likely level of risk, the.study also found that the true
 risk could be as low as zero or as high as 10 in 1,000,000.

 Imagine that the government has now conducted a second study to try to obtain a more accurate number for the
 actual health risk you face. This second study finds that the health risk of getting cancer is most-likely 1  in
 1,000,000,000 (one in a billion), though the study says it could be as low as zero or as high as 10 in 1,000,000,000
 (ten in a billion). Thus the second study finds a lower risk than the first study.
 Risk Study One:
                                                   Risk Study Two:
      Most likely risk

      1  In 1,000,000

  Lifetime extra carfcer risk
  Lowest risk

      0
                            Highest risk

                          10 in 1,000,000
Most likely risk
1 in 1,000,000,000
Lifetime extra cancer risk

Lowest risk
0
.<->


Highest risk
10 in 1,000,000,000

-------

Lay Views on Uncertainty in Health Risk Assessment   -....-   .   •  •                      page 19

TABLE 5. Responses to Lower Risk Estimate From Follow-Up Study

5a The second study was. more competently done than the first. ."
5b 1 am more confident about the safety of my water after seeing the
results of the second study.
, 5c The two studies are essentially in agreement about the level of,
cancer risk from the chemical in the water.
5d Study One, with the higher risk number, was less competently .
done. • .
5e The difference in level of risk between the two-studies is probably
due to chance. .. . •
5f If the government did a third study it would find a risk level even
closer to zero than the second study.
5g Taken together, the results of the two studies provide a more
accurate assessment of the level of risk than either study by
itself.* ,
5h the second study came up with a lower risk number because
better scientific knowledge was available.
Disagree
29.3%
35.4
60.0
41.8
40.4
31.8
23.9
27.5
Agree
20.0%
54.6
31.8
12.9
25.4
15.7
59.6
2,0.7
Don't know
50.7% '
10.0
8.2
45.4
34.3 '
52.5
16.4 :
51.8
 Note. These statements followed the scenario in Figure 3. * Item (quotation or paraphrase) derived from Phase II
 interview; data from Eugene, Oregon, questionnaire.
     A majority felt more confident of their water's 'safety after the second study (5b),
presumably because it found a lower risk. But in the context of the above-mentioned findings of
greater government credibility with higher risk estimates, this is surprising. However, a full third
did not feel better even with a maximum, risk of 10"8, perhaps due to distrust, About half of the
sample .refused to rate the relative competence of the two.studies (5a), to predict what risk level a
third study might find (5f), or to judge that the second (lower) estimate was based on better
knowledge (5h). Perhaps this was due to respondents feeling such opinions were not supportable
by the available information.                     •   ;          '.'.-'  r  "•
     Several interviewees indicated that conducting more studies of environmental problems is a
signal that officials respond to public concerns. An alternate hypothesis is that multiple studies
reduce uncertainty. The fact that a majority agreed with an earlier interviewee that the studies  -
together gave a more  accurate assessment of the risk than either one. alone (5g) could be
interpreted as confirming the second hypothesis. This judgment would be due to the two
estimates confirming  roughly the same risk level, with the second range of risk entirely

-------
Lay Views on Uncertainty in Health Risk Assessment              .                              page 30

subsumed within the first, and both levels being quite small. Yet, contrary to the uncertainty-
reduction hypothesis, most people disagreed that the two studies had found the same level of risk
(5c), and only a quarter thought the difference was probably due to chance (5e). This suggests,
though it does not prove, that the response-to-public-concerns hypothesis is a better explanation
of the positive response to having more than one risk,study.        ,              '    ,

Disagreement Among Groups and Scientists
     We expected to find that people had differing views of the credibility of risk estimates from
various types of organizations, and this  was so. Environmentalists and university scientists were
seen by interviewees and open-ended questionnaire respondents as somewhat more trustworthy
in their risk estimates than government  or business. However, there was surprisingly little
unconditional trust even in the first two groups. People volunteered the possibility of academic   .
bias due to funding sources for research and were as likely to be suspicious if environmentalists
agreed with institutional risk assessments  as to see this as a signal of the latter's accuracy. Item
18 offered in interviews (see Exhibit 7)  had government estimates lower than, but overlapping*
with, environmentalists' estimates (both in ranges). Environmentalists were universally favored
over government in risk assessment, and it was very difficult in either interviews or the open-
ended questionnaire to get people to recognize that the two groups' ranges overlapped.

EXHIBIT 7. Overlapping Estimates of Government and an Environmental Group
 Item 18                                                                 "          .
 The government announces that the water you drink  contains a level of
 chemical X that poses an extra health risk of getting cancer of 1 in 1,000,000
 (one in one million) over a lifetime of drinking water. The government says                        "  •,
 that is the most likely risk, but it says the true risk could be as low as 0.01 in
 a million or as high  as 10 in 1,000,000 (ten in one million). An
 environmentalist group responds by saying that the most likely risk is                                ,
 probably 10 in 1,000,000, and it says the true  risk could be as high as 100 in
 1,000,000 (one hundred in one million).
    i
     Disagreement among scientists is a form of uncertainty (as well as a reflection of
uncertainty) probably more familiar to most people than arguments about dose-response curves.
Item 10 (see Exhibit 8) had a majority of scientists disagreeing with a minority. The fact that
"most" scientists agreed with the government estimate was not persuasive to interviewees and

-------
 Lay Views on Uncertainty in Health Risk Assessment       '          .     ,      ' ,             page 3)

 jury pool members; distrust of government seemed to carry over to scientists, who might be on
 its payroll. Our qualitative respondents were prone to attribute conflict among scientists to self-
       • "....       :                   -\    .
 interest cif incompetence. One interviewee suggested that she would trust the view of whoever
 had been right in the past, rather than by how many scientists shared that view, -but was unable to
 articulate criteria for such predictive success. Two-thirds (65.4%) of our Eugene sample agreed
 with the interviewee-volunteered statement: "When scientists disagree over the size of an
 environmental health risk, I assume the worst case is true, just in case." This reaction also
 appears to mirror the Viscusi etal. (1991) finding of an upward bias in people's response to a
                                     L                          \
 range of risk estimates. However, it may represent a prudent rule of thumb as well as a personal
 bias in response to risk ranges. In our earlier discussion of upward bias findings, people had no
 external guidance for their answers when we asked them, for example, what the community and
 personal risks were of drinking the chemical-contaminated water. The fact that, given only the
 numbers in the risk scenario, as a group they produced numbers somewhat higher on average
 seems fairly direct evidence of an upward bias. Similarly, nearly Half agreed—without external
 signals—with the statement that they would tend to take the highest number in a range of risk
  • „.            • -     '   '       i,            -
 estimates to be correct. In the case of the statement quoted above, however, they have external
 guidance on the risks: the scientists' opinions. And even the scientists cannot agree on the size of
1 the^ risk! Why should laypeople attempt to judge which scientist is correct? In this case it would
 seem prudent,  even,without a personal bias toward higher risk numbers, to assume the worst is
 true. This may explain why the proportion assuming the worst rises from 46.4% without
, scientists being mentioned to 65.4% when scientists disagree. One of our interviewees had said
 that, "When scientists disagree over the size of an environmental health risk, it must not be a
 serious problem." However, most Eugene respondents (84.3%) rejected this proposition. This
 response may mean that they presume one side must be right, and—given an upward bias (see
 "Upward Bias" discussion)—would assume that danger is more likely than no danger.  ;

-------
Lay Views on Uncertainty in Health Risk Assessment  '                                  '       page 32

EXHIBIT 8. Disagreement Among Scientists' Estimates	
 Item 10
 The government announces that the water you drink contains a level of           .
 chemical X that poses an extra health risk of getting cancer of 1 in 1,000,000
 (one in one  million) over a lifetime of drinking that water. Most scientists
 agree with this number, and many think that the true risk could be as low as
 zero. However, a few scientists believe that the true risk could be as high as
 100 in 1,000,000. Both groups have reputations as being competent
 scientists.                                                                          ,
Reasons for Scientific Uncertainty About Environmental                   -f
Conditions                                    .
     Given the clear ignorance of, and doubt about, risk estimate uncertainty among our
participants, we did not seek to explore whether people had their own explanations for such
uncertainty. However, a few spontaneous responses (e.g., in interviews) suggested some such
ideas among the public. For example, in interviews and open-ended questionnaires a few people
volunteered that point estimates of risk did not account for varying exposure (e.g., amount of
water drunk or food eaten) or susceptibility (e.g., children or elderly vs. healthy adults), or that
scientists might be biased by who paid them. We used statements developed from Rowe (1994),
on various potential reasons for scientific uncertainty, in our closed-ended questionnaire, to see
how this sample (N = 280) would react to general reasons for scientific uncertainty. These
statements appear in tables 6 to 9, with the responses of our Eugene respondents.
     People agreed with all of these reasons for scientific uncertainty in the areas of
measurement, complexity, past risks and future risks; two-thirds of the reasons (16 of 24)
garnered 80% or more agreement. (These topics formed coherent dimensions in factor analysis,
and created scales for use as dependent variables in multivariate analysis.) The most consensus
(81% - 91% agreement) occurred for reasons concerning complexity in the processes producing
environmental health risks, with the least agreement (55% - 88%) on measurement problems.
Such consensus may be due to agreement with positively framed statements on topics unfamiliar
or not salient to respondents, or to a fair understanding of the challenges that risk assessors face,
perhaps due more to inference from daily experience or distrust of experts than to education or "
experience with risk assessment. Only further study can test these hypotheses, although this
group of people could have  signaled lack of salience or lack of a firm position by answering

-------
Lay Views on Uncertainty in Health Risk Assessment   •   '•;                                        page 33
TABLE 6. Measurement Reasons for Uncertainty.
. Each question began: "Scientists can be uncertain about their
measurements of environmental and health conditions because"
6a ...of the inherent randomness of nature.
6b ...experts can have biases, conscious or unconscious, in how
•.- ' they make measurements.
6e ...experts can have biases, conscious or unconscious,, in their
interpretation of the results. , .
6d ...they can't get enough measurements to be sure that they've
measured accurately. .
6e ...it can be difficult to decide which facts to include and which can,
be safely ignored.
f =•
6f ...the act of measuring itself can change the conditions being .
measured. '
6g ...the available instruments may not use units of measurement
small enough to tell one condition from another.
Disagree
.14.6%
11,1
8.2
34.3
18.2
11.8 '.,
21.4
Agree
79.6%
83.9
87.5
55.0
75.4
77.9 ;
60.4
Don't know
5.7%
5.0
4.3
10:7.
6.4
10.4
" 18.2
Note. Statements were abstracted from the discussion in Rowe (1994).
TABLE 7. Complexity Reasons for Uncertainty - .
Each question began: "Scientists can be uncertain about
environmental health risks because..."
7a ...they can't be sure they have identified all factors that affect
such risks. . . :
7b... the factors that affect risks may interact with each other in •
• unknown ways.
7c ....their models of how the environment works may be
oversimplified^
7d . . .there may be more than one believable model of how the
environment works, with no evidence to tell which one is more
correct. • - ' . • ' .. • . •
Disagree
5.4%
3.9
10.0
6.8 .
Agree
91.1%
.91.1 -
80.7
85.4
Don't know
3.6% .
5.0.'
9.3
7.9
 Note. Statements were abstracted from the discussion in Rowe (1994).

-------
Lay Views on Uncertainty in Health Risk Assessment
TABLE 8. Reasons for Uncertainty About Past Risks
page 34
Each question began: "Scientists can be uncertain abgut past
environmental health risks because..."
...they don't have complete historical information.
...conditions now are different from what they used to be.
...there are conflicting reports about what happened in the past.
...the conditions measured in the past may not provide information
suitable for identifying past risks.
...historical information may be biased.
...they can't do experiments to check that past measurements were
accurate.
...they can't be sure today's interpretations of past information are
correct, even if hindsight seems 20/20.
Disagree
16.4%
5.0
10.0
8,2
5.0
18.9
8.6
Agree
77.1%
90.0
82.9
82.9
87.5
70.4
80.7
Don't know
6.4%
5.0
7.1
8.9
7.5
10.9
10.7
Note, Statements were abstracted from the discussion in Rowe (1994). ,
TABLE 9. Reasons for Uncertainty About Future Risks
Each question began: "Scientists can be uncertain about future
environmental health risks because..."
...of the inherent randomness of nature.
...unusual combinations of outcomes can occur.
...humans often do the unexpected.
...they haven't studied the environment long enough to separate
long-term trends from short-term changes.
...rare events could occur that would make a big difference.
...the same health problem can have nonenvironmental causes, too,
and it's hard to tell which cause is responsible for a particular
person's (or group of people's) health problems.
Disagree
11.4% -
6.4
11.1
21.4
11.8
8.2
Agree
83.6%
89.3,
81.4
66.1
82.5
80.7
Don't know
5.0%
4.3
7.5
12.5
5.7
11.1
Note. Statements were abstracted from the discussion in Rowe (1994).
"don't know/no opinion." They used this choice often elsewhere (above 10%, up to more than

50%, for 47 ofTt)3 other nondemographic statements), but much less here (7 of 24 reasons
                                                                    ' '  -
evoked "don't know" answers from 10% or more, to a maximum of 18.2% "don't know" '

responses).


     Lack of understanding of environmental science may underlie some of the stronger

disagreements with these statements. For example, a full third (34.3%) disagreed (6d) that

-------
 Lay Views on Uncertainty in Health Risk Assessment           •                  -    .-       page 3 5

 measurement uncertainty is due to scientists being unable to "get enough measurements"
 implying ignorance of resource constraints on field collection of data or an assumption that
 reliable risk estimates require very few measurements. Other major disagreements suggest belief
 in the obviousness of environmental problems and data (6e), unfamiliarity with the concept of
 limits to measurement precision (6g), confusion over the nature of experimentation or "past
 data" (8f), and belief that any change in the environment would be long-term, making
 discrimination of short-term changes unnecessary (9d). The impact of level of formal-education
 on these misconceptions may be small. A survey of Americans (Miller, 1993) found poor
 conceptions of science: 2% saw it as involving the development and testing of theory, 13% as
 experimentation, 25% as rigorous comparison or precise measurement, and 60% volunteered no
 idea about the nature of science. Those with graduate and professional degrees were not much
' '".     •",,-"        •         -             r  v'   .       ,     .     . .
 more accurate: 9% saw science as dealing with theory and 38% with experimentation:

 Uncertainty  and Risk Management
     Although uncertainty is usually-vie wed as a technical issue in risk assessment, it may also ,
 have implications for risk management. Here we report the only two such implications raised in
 Phase II research: the relative perceived value of uncertainty and risk estimates in risk
 management, and the signals uncertainty might give to citizens about agency honesty and . .
 competence. -                ^               '        '

 Neither Uncertainty Nor Risk Estimates Were Very Important
     Questions asked about stimulus scenarios by nearly every interviewee, focus group member,
 and respondent to the open-ended questionnaire were:
     • Why .did this chemical get into the water,  and why wasn't this prevented?
     • What is the government doing to get the chemical out of the water, and why hasn't it
    •   done so already?   •
     • What can I do to protect myself?                   •      .             .
     Other questions were raised  as well (e.g., about the source, effects, and naturalness of the '
 chemical), but qualitative respondents returned again and again to these three issues. They
 seemed to be driven'by two assumptions: that any exposure and any risk was too large, and that

-------
Lay Views on Uncertainty In Health Risk Assessment            '                             page 36

government had a responsibility to ensure that no environmental health risk was present in their
water, soil, or air. However, we do not believe that this expectation of government responsibility
can be easily equated with a desire for, or expectation of, "zero" risk in their lives. As noted
earlier, interviewees and focus group members were willing to accept recreational, financial, and
even airplane risks, but not environmental health risks. Furthermore, the Eugene sample's
reactions to the initial scenario were not solely concern about effects of drinking the
contaminated water and commitment to  getting it cleaned up. Simultaneously, they disagreed
that this was "a serious health risk" and said they intended to "continue drinking this water." If
they really wanted zero risk, these would'not be their answers. After all, they could switch water
supplies (e.g., by moving from tap water to bottled water, or from one brand of bottled water to
another).

Doubts About Government Honesty  and Competence
    In Phase I we found evidence that providing ranges of risk estimates might, at least for a
                                 *                  •                       "
minority, undermine perceptions of government honesty and competence. Our Phase I measures
for these responses were few, and our understanding of the reasoning of respondents on this point
limited. For example, Phase I (Study 2) focus group members said that government never offers
any information about environmental problems unless it is forced to, releases information about
environmental problems only so that people can't say that they were never told about them, will
tell you there is a big risk if it wants credit for cleaning up an environmental problem, and will
tell you the risk is zero if it can't or won't, clean up an environmental problem. These responses
did not clarify what people thought in detail (if anything) about government honesty and
competence with regard to risk assessment, much less whether'disclosures of uncertainty or more
general judgments about government drove their answers, so we chose to study this further in
Phase II.
    ' We found in qualitative work that people would  often explicitly point to the range of risk
estimates (or to related issues, such as statistical "confidence"—see discussion below) as a •  »
rationale for their conclusions that government was dishonest or incompetent. For example, one
comment was that a government that was honest about the size  of an environmental health risk
would give only one estimate. Another was that government is just guessing if it gives a range of

-------
  LayViews on Uncertainty in Health Risk Assessment   ' -        ;      ,    '  '                page 37
       \            '  '     •-''''         -..-••.'''.•       •     ,
  risk estimates for an; environmental problem. Someone who said that the government calculates
  environmental risks without taking into account the inevitability of unexpected events was
  clearly using her awareness of uncertainty to judge government competence as low. Some people
  said that judgments of honesty depend on explanations of how the research was done on which •
  government's risk number is based, or whether government is willing to show how they
  calculated a "safe" risk number.                                                   ,  ,
      Questions about whether a given item's risk estimate was honest or competent seemed to be
  the most challenging ones for interviewees and focus group members to 'answer. Although many
  people seemed to  answer these based o'n their prior views of government, many others said that
  they couldn't answer them, didn't have enough information, or had no idea how to evaluate
  honesty or competence in risk assessment. Although-we gained some insights on the cues, people
  used (or would like to  use) to mak'e such decisions—the Agent Orange controversy, background
  research studies' design or number, whether environmentalists agreed with the government—it is
  clear, that we do not fully understand.how people judge honesty and competence in risk
  assessment.. We have no evidence that people have explicit, a priori criteria for such judgments;
,  if this is true, it would  mak? future research on this issue particularly challenging and valuable.
      Despite this caveat, it seemed worthwhile to -identify the distribution of opinion about
 government honesty and competence in risk assessment in our Eugene, Oregon, sample. As ,- -.
 noted in Table 2 earlier, there'were mixed feelings about government honesty and'competence
 regarding the scenario  of drinking water contamination they read. More generally (see Table 10),
   ' - s  •       .        ' =  ,--••      t-     •        •         '        •  •     ...     •, -        %
 three-quarters disagreed that government provision of a range of risk numbers was less honest,
 and a slight majority said it was more competent, than providing a single risk number. (Items 1Ob
 and 1 Oc were deliberately phrased to mirror each other, to test the reliability of competence
 judgments; these were  clearly reliable.) These responses  clash somewhat with the response to
 statement lOd, a"view expressed by one interviewee. Roughly half of ounsample said that
 government talks about environmental issues only when they pose a high risk. This opinion may
 explain why the reaction to lower bounds of ze.rcf(see above) is apparently so distrustful, and  •
 people split over whether government discussion of ranges is honest. Risk estimate ranges in
 general may be welcomed as honest and possibly competent, but particular ranges may seem

-------
Lay View on Uncertainty in Health Risk Assessment                              .             page 38

dishonest if they can include zero when "obviously" the problem must be high-risk for .
government even-to discuss it publicly. By contrast, the disagreement with statement lOe may
reflect skepticism over the validity of extrapolation from animal data (see below) rather than a
judgment of government itself.                                ,
                        *                       :                       ,
TABLE 10. General Views of Government Risk Assessment	.	-	
                                     ,                     Disagree      Agree     Don't know
  10a  If government was being honest about the size of an             75.0%      15.4%        9.6%
      environmental health risk it would give only one number, rather                            •
      than a range of numbers.                                                      ,   .
  10b  I would feel more confident that government knows how to         59.6        26.1         14.3
      determine the size of environmental health risks if they give a                        .
      single risk number for an environmental problem.
  10c  I would feel more confident that government knows how to         29.3        53.6         17.1
      determine the size of environmental health risks if they give a
      range of risk numbers for an environmental problem.                                        •  ,
  10d  Government only makes an announcement about an .            39.3        48.6         12.1
      environmentalissue when there is a high risk involved.        .   .                '
  10e  The government is competent in estimating the size of an          50.4        35.4         14.3
      environmental chemical risk for humans when it takes into
      account the chemical's effects on animals.               '
 Understanding of Science and Risk Assessment
     Whether people are familiar with how science and risk assessment operate may have an
 impact on their responses to uncertainty in risk estimates. We discuss here awareness of
 uncertainty in science and risk assessment, and beliefs about risk assessment concepts (e.g.,
 animal data extrapolation) and government practice.

 Familiarity With Uncertainty in Risk Assessment and
 Science
     Qualitative data supported our earlier conclusion that uncertainty in risk assessment was
 unfamiliar. Occasionally people noted that they "had heard" statements like the ones we offered,
 but usually in reaction to such items (see Exhibit 9), which to our knowledge have not been
 publicized in any form. We suspect this sense of familiarity was due more to an overall
 impression of numbers and phrases (e.g., "risk of getting cancer") likely to accompany
 environment-related information, rather than an actual recognition of uncertainty. This does not

-------
  Lay Views on Uncertainty in Health Risk Assessment    .,               .           • .       .  .   page 39

  mean that interviewees and others were entirely unable to recognize uncertainty as an issue in
  risk assessment. For example, some people mentioned differences in apparent vulnerability (e.g.,
.  children, elderly, the sick) or exposure (e.g;, "How much water are we supposed to drink to get
  this risk?"). Focus group members felt there were "too many numbers" and not enough
  "personalized information" about risk factors or information.on what protective measures were
  being or could be taken. The comments about "personalized", information also imply an
  awareness of variability.          r                      .                 .

  EXHIBIT 9. Uncertainty Scenario: Qualitative Study	•	
   Item 8      •'...'"'•      -.-..'   ~ * .
   The,government announces that its scientists have calculated the risk of
   getting cancer from drinking the water you drink, which as a small amount of
   chemical X in it, for an entire lifetime. It is 95% certain that at least 90% pf
   the population has an extra risk of no more than 1 in 1,000,000 (one in one
   milliori). A small proportion of people who are more likely than others, when
   exposed to a cause of cancer, to get cancer may have a risk as high as 10
   in 1 ;000,000 (ten in one million) if they drink this water for their entire lives.
      The closed-ended questionnaire had only one direct measure of whether people expect risk
  assessment-to be certain (Table 11). Most disagreed with statement 1 la that a single number
  could describe .an environmental health risk. This may be why they felt (above) that presentation
  of ranges of risk estimates was potentially honest. The difference between the qualitative and
  quantitative results may be due to different samples. The quantitative group had more education,
  and were largely still in college, where they might have been recently exposed to similar
  concepts. There was no correlation between education and answers to this question, although
  education did not vary much in the Eugene sample. .Statements lib and lie discuss the abstract
  fact that "different scientific ideas" about how to extrapolate from high human doses or animal
  data "can.be equally valid" in estimating human risks at.low doses. About half of our sample
  agreed with these indirect measures of familiarity with uncertainty in risk assessment.. Similarly,
  the interviewee-volunteered idea that scientific certainty must accompany a "real"
  environmental concern (1 Id) was rejected by a scant majority.

-------
Lay Views on Uncertainty in Health Risk Assessment                                            page 40

TABLE 11. Expectations of Uncertainty in Risk Assessment and Science	
                                       '                     Disagree     Agree     Don't know
 11 a  No environmental health risk can be described by a single         17.1%       69.6%       13>2%
      number.*                                   -
 11b  When experts only have information on the effects of much        28.6        50.0     s    21.4
      higher levels of the chemical, different scientific ideas of how
      to determine human health risks from lower levels of that
      chemical can be equally valid.
 11c  When there is only information on animal reactions to a           38.2        51.8         10.0
      chemical, different scientific ideas of how to determine human
      health risks from that chemical can be equally valid.         •     .
 11d  If an environmental problem was a real concern, scientists         54.3        35.0        -10.7
      wouldn't say that there "could be" health risks.*
 11e  A competent scientist gives a single, definitive answer to a         80.4        12.5          7.1
      question.
 11f  It is typical of good science that.the most likely value for what      11.1        77.9         11.1
      is being measured has a range of uncertainty around it.,                                          •
 Note. * Item (quotation or paraphrase) derived from Phase II interview; data from Eugene, Oregon questionnaire.

     As for uncertainty in science, few agreed that' scientific competence requires definitive
answers (statements lie and 1 If), or even said that they did not know. Most members of this
highly educated group  (most of whom had at least some college education) thus seem somewhat
familiar with the concept. While education did not correlate with "competent scientist"
answers—note that variability in education was low in this sample—better educated respondents
were more likely (r = .21, p < .0007) to agree that good science could be uncertain. Thus the
substantial minority of this sample who doubted science's uncertainty may better represent the
view of the general, less educated public. Statements 1 le (reversed) and 1 If formed  an index
(a = .41) on "Science Uncertainty" for multivariate analysis.

Knowledge of Risk Assessment                            ,
    .DifferentiaTknowledge has been hypothesized as a cause of different responses to risk in
general and is worth exploring as a factor in reactions to uncertainty. One of many ways to  •
measure the elusive concept of''knowledge" is to see whether experts and laypeople agree on
propositions about concepts relevant for risk assessment. Table 12 compares agreement with
items on dose-response relationships and animal data among three groups: our Eugene sample,

-------
Lay Views, on Uncertainty in Health Risk Assessment             ;              .                page 4}

and samples from the Portland, Oregon, public, and the Society of Toxicology in earlier research
(Kraus, Malmfdrs, & Slovic, 1992). For about half of the items, people in our study were even
farther from expert views than the earlier, public sample, while for the other half they were closer
to the expert views. Overall, however, experts and both public samples are farther apart—with
citizens more likely to see any exposure as harmful. As noted in the table, we used two of these
items to create an "Animal Studies" variable for multivariate analysis (a = .48);' about a third of
our respondents believed in the extrapolation value of such studies.  -

TABLE 12. Concepts of Risk Exposure

12a


12b



12c


i2d



There is no safe level of exposure to a cancer-
causing agent, , • ,

For pesticides, it's not how much of the
chemical you are exposed to that should worry
you, but whether or not you are exposed to it at
..all. '.".'".-'
The way .that an animal reacts to a chemical is
a reliable predictor of how a human would react
to the same chemical.
If a scientific study produces evidence that a
chemical causes cancer in animals, then we
can be reasonably sure that tfie chemical will,
Sample*
E :-
P ' '
T
••' E
P
T

' E
P
T
E
P
T
Disagree
38.6%
34.7
74.7.
47.9
59.2
94.6

52.9
45.7
40.8
29.6
24.8'
57.6
Agree
52.5%
53.9
18.7
43.6
36.1
4.2

34.3 •
43.7
55.4
55.4
69.4
40.6
Don't know
8.9%
11.3
6.6
8.6
4.6
1.2

12.9
. 10.6
3.8
15'.0
5.8 '
1.8
      cause cancer in humans. .                  ".".-.'
 12e  If a person is exposed to a chemical that can        E         62.1         20.0        17.9 .
      cause cancer, then that person will probably                       ,  .
      get cancer some day.' '               "   -                 ,,   '  .
      If you are exposed to a carcinogen, then you        P'        47.7        34.4        17.2
      are likely to get cancer.                  >       T         88.0         8.4         3.6
 Note. * E = Eugene, Oregon, sample in this study; P and T are the samples of the Portland, Oregon, public and
 Society of Toxicology members, respectively, in Kraus et al. (1992). Statements 12c and 12d formed the index
 ANIMAL, = .48.                                          •     •                  •  •
  •  .Another Way to assess knowledge is to examine notions of standard practice in government
risk assessment (see Table 13), although experts could dispute what that is. The many "don't
knows" indicate that people feel they don't know much about this topic. Statements  13e and 13f,
asked, at different points of the questionnaire, imply the belief that average exposures are used in
government risk estimates, but contradict statement 13a (maxima). Both are contradicted by

-------
Lay Views on Uncertainty in Health Risk Assessment          .                                 page 42


statement 13b on actual measures. Whatever government risk assessors do, our sample clearly

has no firm concept of that practice; these statements did not load together on any dimension

when data were factor analyzed.       .           ,

TABLE 13. Beliefs About Government Risk Assessment     	.	;	,    	

                                                           Disagree      Agree     Don't know

 13a  Government risk numbers tend to be based in part on the         20.0%      47,5%       32.5%
      maximum amount of a pollutant that any person could be
      exposed to.
 13b  Government risk numbers are based on actual measurements      39.3        32.5        28.2
      of how much of a pollutant people are exposed to, not just on
      assumptions about human exposures.

 13c  Government tends to take a cautious approach in calculating      45.4        33.6        21.1
      the size of environmental health risks, and assumes the worst                                •
      case about how people might be exposed or react to a
      pollutant.                                                               -.
 13d  Government risk numbers tend to indicate the risk that an         38.6        32.5   .     28.9
      environmental pollutant poses for the person at greatest risk.               •    -  •

 13e  Government risk numbers tend to be based in part on the         17.1        60i4        22.5
      amount of a pollutant that the average person is exposed to.                          •••

 13f  Government risk numbers tend to indicate the risk that an         19.6        59.3        21.1
      environmental pollutant poses for the average person.
Potentially Mediating Factors
     Beliefs and stances unrelated to the environment or risk or risk assessment may affect how

people react to the issue of uncertainty in human health risk estimates. It is thus important for us

to account for at least some of these potentially mediating factors in this study, particularly in

multivariate analysis. Here we discuss our findings with regard to mathematical prowess,

personal vulnerability, uncertainty in daily life, and worldviews,


Numbers, Uncertain or Not, Are Hard to Grasp
    1 Difficulty with numbers may be related to education, mathematical or statistical knowledge,

and/or personal comfort with numeration. Education varied little in our closed-ended

questionnaire sample (75% with some college, 4% with a high school degree or less), although-

we used it as an independent variable in multivariate analysis. As noted under "Methods," most

people with a high school degree or less failed to give substantive responses to the open-ended

-------
      Views on Uncertainty in Health Risk Assessment                                 •         page 43

   questionnaire. Perhaps education makes it easier to cope with risk numbers, including uncertain
   ones; perhaps people with more education were more' accustomed to answering questionnaires..
        Several interviewees had problems comparing different risk numbers (e.gv two "most
   likely," estimates, or "lowest" vs. "highest" risk numbers). For example, one person expected to
                        *         > -          ,      -          •                 *
   "feel safer about drinking water" if estimated risks changed from 1 in 1,000,000 to 1 in 100,000.
   A retired engineer had trouble with Item 1 Ic (see Appendix I), because it required a confusing
   triple-division by 10, in the process of applying  "uncertainty factors" to a NOAEL to derive a
   Reference .Concentration for a noncarcinogen, (Another reaction to that item—"there was no
   justification for dividing by 10 rather than 20 or 5"—was a policy or communication conflict
.   rather than a signal of difficulty with numbers;)  •
       Because of these earlier reactions, we decided to put some crude measures of mathematical
   skill into the closed-ended questionnaire. We asked for the degree of agreement with two
   .statements that compared two risk numbers: "You are less likely to get a disease if the risk of
   getting it is  10 in 1,000,000 than if the risk is 1 in 1,000,000" and "You are more .likely to get a
   disease if the risk of getting it is -7 in 1,000,000 than if the risk is 1 in 100,000." The first
 .  statement used a common denominator and numerators one magnitude apart; the second
   statement used different denominators-and an odd difference (7-fold) in'numerators. Their
   difficulty varied as one might expect: only 13.6% of our respondents agreed (incorrectly) with
   the first statement, while 31.4% agreed (also  incorrectly) with the second statement.
                                            »                    -         ...
       Soine focus  group members in Phase I research had suggested that using a common
   denominator to compare numbers (e.g., 0.1 in 1,000,000, 1 in 1,000,000 and 10 in 1,000,000)
   would make this task easier than the usual common-numerator format (1 in 10,000,000, 1 in
   1,000,000, and 1  in 100,000). Most.people shown these variants in Phase II qualitative research
   felt .the first format was more confusing than  helpful; some people given items in this format
   actually translated them into the common-numerator format to answer the open-ended
                               ' - -           \
   questionnaire, implying the latter is more familiar. However, in our closed-ended questionnaire  .
   there was very little error, with the cornmon-denominator format. We also used this common-
.  denominator format in.our scenarios for the closed-ended questionnaire (e.g., a most likely risk
   of 1 in 1,000,000, with a range extending up.to 10 in  1,000,000). About a fifth (20.4%) of the  .

-------
Lay Kr'eivjr on Uncertainty in Health Risk Assessment                                   '       page 44

sample agreed that the information in the initial scenario was "not understandable," but this
could have been due as much to the presence of numbers alone, or to the other information in the
scenario, as to the use of this numerical format.
     A third measure of mathematical skill was self-reported: agreement with the statement "I
feel very comfortable dealing with numbers and calculations." About a third (32.9%) of our
Oregon questionnaire respondents—equal to the proportion erring on the more challenging
numerical comparison cited above—disagreed with this statement.
     Responses to the two comparisons and one self-report were poorly correlated (e.g., r = .03
among the comparisons; highest correlation was r = .15 [p > .01] between, self-reported comfort
with numbers and the common-denominator statement), and Cronbach's alpha was extremely
low (a = .19). We thus used these three items as separate independent variables in subsequent
multivariate analyses.
     Statistical wording created other problems for respondents. For example, use of the phrase
"95% confidence" in some early interviews evoked outrage about the government's "lack of
confidence" and lack of explanation as to why it wasn't surer;  explanations of what was meant
by statistical confidence were not persuasive. Changing this term to "95% certain" did not
reduce confusion and anger much either; people seemed to interpret this primarily as government
not knowing what the risk was, or "there is too much of a risk," although one or two said this
was about the same as being certain. Item 5 (see Exhibit 10) said there was a 5% chance of risk
above 1 in 100,000, and a 5% chance that the risk was zero, and thus a 90% chance of the true
risk being between zero and 1 in 100,000. It failedto help interviewees understand, the concept of
confidence limits.                                              •       ./.".,

EXHIBIT 10. Confidence Limits: Qualitative Study    	.
              -j»
 ltem'5
 The government announces that there is a 5% chance that the extra level of                             .
 risk from drinking chemical X in the water you drink for your entire lifetime is
 above 1 !n 100,000 (one in one hundred thousand), and a 5% chance that
 the risk Is zero. The government says this means there is a 90% chance that
 the true risk is between zero and 1 in 100,000.

-------
 Lay Views on Uncertainty in Health Risk Assessment     , .              • :         '   •. •       page 45

     Confusion became even worse in response to a phrase recommended by the National
 Research Council (1994, pp. 10-24) for communicating about variability in susceptibility. Our
 version (item 8; see Exhibit 9) said that "It is.95% certain that at least 90% of the population has
 an extra risk of no more than ,1 in 1,000,000." Focus group members (in questionnaires answered
 before the group met) said, among other responses, that this meant:
     •  "They are not 100% sure. They need to be 100% sure of what's goin'g to happen."
     •  "What are they really, saying.... This info is truly frustrating."
     •  "These numbers are self-explanatory and give people the information they need."
     •  "A guess ..; could be a fair guess."                     ,
     •  "Most confusing to me is: how do'they get these numbers?"
     The focus groups also saw a graphic proposed by a National Research Council committee
 (1994, pp. 10-25), slightly modified for readability, for communicating variability in exposure-
 and confidence limits on the  resulting risk estimate (see Figure 1). This was also greeted with
 confusion by focus group members, despite an explanation by the focus group facilitator, who
 teaches statistics professionally. For example, they could not by themselves find on the graphic
 the risk of median expo.sure.

 Personal Vulnerability     .      .                           •".>'.'
     We seemed to confirm earlier conclusions about an "optimism" bias, in which people see
 their own risks as being less than those of society or other people (Weinstein, 1980). In response
 to the initial scenario (Figure 2), as many as 39.7% saw their personal risk as at or below the
 "most likely" (1 in 1,000^000) level calculated by government, while only 24.7% saw the same
 level of risk as likely for the community (see Table 3). These two answers formed an'index of
 ESTRISK (a = .73) used in later multivariate analysis. Over two-thirds (71.4%) rated the risk
 from drinking trie water discussed in the scenario as "somewhat" or "considerably" lower than
 "other health risks in my life." Others saw equal (19.6%) or higher (8.6%) risks from drinking
this water than from other health risks they faced.                        .
   '  More generally, most respondents felt their chances of being exposed to an environmental
pollutant were "about the same as" (78.2%) or "less likely than" (14.3%) chances of exposure

-------
Lay Views on Uncertainty in Health Risk Assessment                   -                       page 4 6

for the average person.  Similar responses (78.9% and 15.0%, respectively) were obtained on
judgments of their chances of getting sick from environmental pollution, compared to the
average person. About  13 agreed with an interviewee who said she assumes she is likely to be the
person who suffers a 1-in-1,000,000 environmental health risk. These findings together suggest
that a conscious (or at least admissible) sense of personal vulnerability to environmental
problems is not common; we formed an index from these statements (a = .57) to use in
multivariate analysis.
     However, we also found nearly half our sample (43.2%) agreeing that "There are serious
environmental health problems where I live" (45% disagreed). They saw moderate to 'high risk
from drinking tap water in their homes as more likely (35.4%) than for bottled water (7.1 %),  •
although only  4.3% drank bottled water every day. The distinction between those who. saw tap
water risks as  zero or low (62.5%) and those who saw these risks as high might ha.ve affected
responses to uncertainty in a scenario about drinking water contamination. Each of these
questions (except the question on perceived risks of bottled water, which had too little variance)
was an independent variable in subsequent multivariate analyses.

Life Uncertainty
     We also expected that personal stances to uncertainty in general might affect reactions to
uncertainty in health risk assessment. Over half of our Eugene sample (56.1%)  agreed that "I try
to avoid uncertainty in my life as much as possible,"  but disagreed (62.9%) that "To a great
extent my life is controlled by accidental happenings."  These items did not form a viable index,
so they were entered as separate independent variables in the multivariate analyses.

Worldviews
     As mentioned in our Phase I report and article (Johnson & Slovic, 1994a,  1995), we had
found evidence that general worldviews might affect response to uncertainty. Because of
analytical constraints on the earlier data set, however, our findings were not conclusive, and we
believed that some further analysis was warranted. Thus several questions that appeared to elicit
salient worldviews were asked in our Eugene questionnaire. Table 14 shows those that formed

-------
 Lay Views on Uncertainty in Health Risk Assessment                         '   ,   .          page 4 7

 reasonably coherent indices, which were entered in multivariate analyses. (Cronbach's alpha is
 listed in parentheses next to each index title.)  •
     One apparent dimension we labeled "antiegalitarian," because most of its component
 statements concerned support for inequality (e.g., beliefs that the "equal rights" movement has
 gone too far, or more pay should go to those with greater ability). Measures of egalitarianism
 seemed to be associated with beliefs about dishonesty or incompetence in ranges of risk
 estimates in our Phase I research. They also have been associated with risk beliefs in some other
 studies (e.g., Dake, 1991; Peters & Slovic, in press; Rayner & Cantor, 1987). Earle and .
 Cvetkovich (1995) have argued that values~such as those that appear to underlie our ANTIEGAL
 index drive trust, and thus in turn affect perceptions of risk, and so forth. The fact that our factor
 analysis found TRUST to be separate from ANTIEGAL raises questions about this hypothesis.
 Statements that .together seemed to represent support for "environmentalism" formed another
-index. This viewpoint has been associated with egalitarianism in some of the research cited
 above, but it formed a separate dimension here; A "trust in authority" index, although weaker
 (see the low Cronbach's alpha of .54), also emerged from factor analysis. Phase I research found
 trust linked to judgments of honesty and competence in government discussion of risk
 uncertainty. Finally,.a,very weak "fatalism" index appeared as well, exemplifying a perceived
 lack of control over one's life. .

 Multivariate Analyses                              .
     Although our sample was not randomly selected, we felt it would be useful to compare the
 impact of some potential influences on responses to the risk uncertainty scenarios (figures 1  to 3)
 through multiple regression analyses. Tables 15 and 16 show the dependent and independent
 variables, respectively, indicating (where appropriate) Cronbach's alpha for indices and the text
 location of the statements making up these indices (table number and/or statement number within
 a table are provided), as well as the number of respondents providing each item.

-------
Lay Mews on Uncertainty in Health Risk Assessment
page -18
TABLE 14. Worldview Statements and Indices
-
Antiegalitarian Index (a = .69)
We have gone too far in pushing equal rights in this
country.
In a fair system people with more ability should earn
more.
If people in this country were treated equally, we
would have fewer problems. (R)
What this world needs is a more equal distribution of
wealth. (R)
When the risk is very small, it is OK for society to
impose that risk on individuals without their consent.
1 am in favor of capital punishment.
Environmentalism Index (a = .70)
All species, including humans, have an equal right to
co-exist on the planet.
1 would be willing to sacrifice much of my current
standard of living to insure that nature is not harmed.
1 am attracted to the spiritual qualities inherent in the
natural world.
1 know a lot about environmental health issues.
Trust in Authority Index (a = .54)
Decisions about health risks should be left to the
experts.
The police should have the right to listen to private
phone calls to investigate a crime.
When there is a really serious health problem, then
public health officials will take care of it. Until they
alert me about a specific problem, 1 don't really have
to worry.
Those in power often withhold information about
things that are harmful to us. (R)
Government has no right to regulate people's
personal risk-taking activities such as smoking,
mountain climbing? hang gliding, etc. (R)
1
Fatalism Index (a = .43)
I have very little control over risks to my health.
It's no use worrying about public affairs; 1 can't do
anything about them anyway.
/Vote. R = reversed scoring for construction of indices;
1 Disagree

83.2%

23.9

18.9

21.1

81.8

37.5

12.1

35.0

19.6

45.0
•
71.8-

76.4

78.2



17.1

24.6




70.7
84.6

Cronbach's alpha
Agree

15.4%

65.0

74.6

75.0

12.1

52.1

85.0
„ 	 /
56.1

71.1

48.9

20.4

17.9

16.4



69.6

68.9




23.6
13.6

indices in
Don't know

1.4%

11.1

6.4 .

3.9

• 6.1

'10.4 .

2.9

8.9

9.3

6.1

' 7'9

5.7

5.4-



13.2

6.4




5.7
1.8

parentheses.

-------
Lay'Views- on Uncertainty in Health Risk Assessment
page 4.9
TABLE 15. Multiple Regression Dependent Variables
Variable name
CERTAINTY
WRONG
WORST
NOWORRY
CONCERN
ESTRISK
GOVHONEST
GOVKNOWS
2NDE'ST
. 2BETTER
MEASURE
COMPLEX
PAST
FUTURE
TAPWATER
SERIOUS
Variable description
desire for certainty
government_could be wrong . .
assumes worst given risk range
not worried by drinking water scenario
concerned about water in drinking water scenario
estimated risk of drinking water scenario
government honest fn drinking water scenario
government knows exactly what risk is in drinking
water, scenario
second study more competent in two-study scenario
two studies better than one in two-study scenario
.measurement reasons for uncertainty
complexity reasons for uncertainty ' '
past risk.reasons for uncertainty'
future risk reasons for uncertainty
perceived risks of home tapwater
serious environmental health problem where 1 live
Cronbach's
alpha
.80
.74
.38
.74
.63
.73
.74
-^
.74
.49
.73
.80 ,
.84
,.83
—
' — •
Text
location
' Table 1
Table 4b
Table 4d
Table 2
Table 2
Tables
• Table 2
Table 2b
Table
5a,d,h
• Table 5b,g
Table 6
Table 7
Tables
Table 9
> Page 47
Page 47 '
A/
273
229 .
244
256
254
245
247
280
146
222
' 259
266
261
,: 265
274
247
 A/ofe..Shading separates variable types; respectively, reaction to uncertainty in general; initial scenario;
 government risk estimates; the second study scenario; reasons for scientific uncertainty; local risks.
   ,  The shading in Table 15 separates dependent variables that cover reaction to: uncertainty in
general, the initial scenario and government risk estimates, a second study with lower risk
ranges, reasons for scientific uncertainty, and local risks. Table 16 is shaded to distinguish
various categories of independent variables, dealing in turn with worldviews, personal stances
toward uncertainty, knowledge, and sociodemographic characteristics (three dependent
    i   "  ' '•       *                       *         '                       ..•,-'•""
variables—CERTAINTY, WRONG and WORST—also became independent variables for some
multiple regressions).           ,            ;                    v •

-------
 Lay Views on Uncertainty in Health Risk Assessment
TABLE 16. Multiple Regression Independent Variables
page 50
Variable name
ANTIEGAL
ENVIRON
TRUST
FATAL
AVOIDUNC
LIFEACC
P_VULNER
TAPWATER
BOTTLED
SERIOUS
MATHCOMF
EASYODDS
HARDODDS
SCIUNCER
ANIMAL
SEX
AGE
BAEDUC
WHITE
CHILDREN
ENVGRP
CERTAINTY
WRONG
WORST
Variable description
antiegalitarian
environmentalism
trust in authority
fatalism
avoids uncertainty in one's life
life controlled by accidental happenings
feels personally vulnerable
high health risks from drinking tap water in own home
frequency of drinking bottled water
serious environmental health problem where I live
comfortable with numbers and calculations
1 0 in 1 ,000,000 risk less than 1 in 1 ,000,000 risk
7 in 1,000,000 risk less than 1 in 100,000 risk
science is uncertain
belief in extrapolation from animal studies
sex
age in years
education of bachelor's degree or higher
race
children living in household
affiliated with environmental group
desire for certainty .
government could be wrong
assumes worst given risk range
Cronbach's Text
alpha location
' .69
.70
.54
.43
. —
—
.57
—
—
—
. —
—
— ' ./ " '
...41 11
.48 12
— — '
—
— .
—
— ,
—
.80
•74
.38
N
277
267
273
260
256
253
260
274
279
247
275
263
261
235
214
278
277
. 276
276
280
280
273
229
244
 Note. Shading separates variable types; respectively, worldviews; personal stances toward uncertainty
 (personal vulnerability, life uncertainty); knowledge, sociodemographic; reactions to uncertainty in general. •
    'To increase sample sizes, we included respondents who had only one missing response for
indices consisting of three to four items, and two missing responses for indices of five to six   ,
items. No missing responses were allowed for two-item indices, or where indices mixed
statements with differing numbers of response options. This resulted in Ate of 128-145 for

-------
 Lay Views'on Uncertainty in'Health Risk Assessment        .                    .      '       page 5 J

 various regression analyses, except that for the dependent variable of considering the second
 study to be more competent (JV-88).
    , • Note that Cronbach's alpha for most index variables were' modest to weak, although far
 better for dependent than independent variables, (Ten of .13 dependent variable indices had
 alphas above .70; only one exclusively independent variable reached that level, and three of the
 seven independent indices had alphas under .50.) Although this means that generalizing the
 results of multivariate analysis must be done cautiously, we think it is appropriate in an
 exploratory study like this one to relax statistical constraints. Our primary aim for Phase II
 research was to understand the concepts, if any, that laypeople have of uncertainty in human
 health risk assessment, which-the univariate analyses discussed earlier portray. The purpose of
 the multiple regression analyses discussed below is not to describe uncertainty concepts, but
 instead to provide preliminary examination of reasons for these concepts. For such a purpose the
 current data are sufficient.                            .

 Reactions to Uncertainty  ~
     The multiple regressions are divided into two'groups. Table 17 concerns three generic
 reactions to uncertainty—desire for certainty (CERTAINTY), belief that risk ranges mean the
 government could be wrong (WRONG), and tendency to assume the worst if a range  is given
 (WORST)—and  several reactions to the initial scenario (NOWORRY, CONCERN, ESTRISK,
 GOVHONEST, GOVKNOWS). Table 18 includes regressions concerning reactions to the
 scenario of a second risk estimate (2NDEST, 2BETTER), reasons for scientific uncertainty
" (MEASURE, COMPLEX; PAST, FUTURE), and risk beliefs (TAPWATER, SERIOUS).
     As shown in Table 17, six of the eight regressions conducted to predict reactions to
 uncertainty were  statistically significant (with a.p of at least .05 or greater). The two regressions
 that'were not significant were those predicting the belief that risk ranges mean government could.
     ,     .       •        -    -   .....       '.  - -                   .                 i
 be wrong (WRONG) and the tendency to assume the worst if a range is given (WORST). For
 five the six significant models the variance explained ranged from 22% to 35%. The variance •
 explained in the regression predicting belief in government's honesty was only 11% and none of
 the regression coefficients were significant,

-------
Lay Views on Uncertainty in Health Risk Assessment                                .                 page 52
TABLE 17. Multiple Regressions: Reactions to Uncertainty and Initial Scenario
Variables
antiegal
environ
trust
fatal
avoidunc
lifeacc
p_vulner
tapwater
bottled
serious
mathcomf
easyodds
hardodds
sciuncer
animal
sex
age
baeduc
white
children
envgrp
wrong
worst
certainty
adjusted r2
f value
f value probability.
N
• p < .05
"p < .01
*"p < .001
••••p<.0001
certainty
-.03
.01
-.09
.18
.08
.04
.04
-.01
.04
-.25*
-.10
.06
-.07
-.37***
.09
-.02
.06
-.09
-.03
.18
.01
.11
.05
—
.29
3.00
.0001
117
Note. Cell
wrong
.26*
.16
-.24
-.17
.05
-.07
-.07
.02
.00
-.00
-.07
.17
-.03
-.16
.09
.07
.11
.01
-.09
.05
.00
—
—
.20
.10
1.60
.06
122
entries
worst
-.08
.17
-.07
-.14
.02
-.05
.12
-.01
.17
.08
.04
-.16
-.03
-.10
.06
-.02
-.09
.14
.05
-.02
.04
—
— '
.07
.05
1.30
.21
124
noworry
-.01
.10
.04
-.05
. .08
-.06
-.24*
-.28**
.02
-.11
-.05
-.11
-.09
.25* •
-.14
-.14
-.05
-.19*
-.00
-.16
-.04
— ,
. —
-.06
.27
3.10
.0001
125
concern
-.32**
.02
-.23**
-.07
.06
-.04
.05
.19*
.05
-.08
-.06
.10
-.01
.18
.22*
.06
-.01
.06
. --02
.10
.02
—
— '
.05
.27
3.20
.0001
129
estrisk
-.11
.14
-.10
-.16
.14
-.04
.03
.15
-.10
.10
.08
-.13
.04
.05
.10
-.08
-.02
.08
.14
-.02
-.16
—
_ —
.03
.11
1.70
.04
122
are standardized regression coefficients
govhonest
.09
-.05
.35****
.01
-.03
.07
-.03
-.09 .'
-.04 .
.06
.07
-.27**
-.06
.07
-.07
.09
-.02
.09
.11
-.04
-.01
"— •
—
-.33***
.35
4.10 '
.0001
124
(J3 weights).
govknows
.01
.-io
. .22*
.10 "
.15 -
.04
.11
,05 -
• 14
-.13
-.10
-.05
.02,.
-.17
.19*
.10
-.10
, .06
.04
-.04
.06
—
—
.20
.22
2.50
.001
• 119


-------
Lay Views on Uncertainty in Health Risk Assessment
                                                                                                page j 3
TABLE 18. Multiple Regressions: Reactions to Second Study, Reasons for Uncertainty and Risk Beliefs'
Variables
antiegal
environ
trust
fatal
avoidunc
lifeacc :
p_vulner ••
tapwater
bottled
•serious •-
mathcomf
easyodds
hardodds
sciuncer
animal
t
sex
age
baeduc
white
children
envgrp
certainty
adjusted r2
f value
f value
probability
N. ,
*p<.05
**p < .01
***p < .001
****p<.0001
2ndest
-.06
-.17
-.28*
,-. .39**
. -.00
-.10
.01
.01
.01
.01
.16
'. -.02
-.11
-.05
.00
:02
;• .04
-.01
.05
.30*
-.02
.36*
.20
1.90
.03
^77
Note. Cell
2better
.11
.02
-.06
-.00 ''.'
.08
, .01
.06
.07
-.08
-.10
.10'
-.11
-.03
-.05
.07
-.11
-.00
.05
.13
. -.06
-.13
.13
-.03
0.90
.64
,116
entries are
measure
.04
-12
-.13
-.08
.05 ,
.34***'
-.01
-.06
-.00
.11
.03
, .06
-.18
.06
-.00
.25*
.04
.06
-.02
,.13
-.04
-.07
.07
, 1.40
•11
127
\
complex
-.07
-12
-.32***
-.00
.02
.13 '
.05 ,
-.09
-.12 ,
-.05
.11 ,
-.12
-.08
.09
.12
. .27**
.17
.08
• -.01
.03.
.05 '
-.10
.21
2.50
.0009
127
standardized regression
past
-.05 ;
-.19
-.19*
-.21
.01
.18*
-.04
-.10
.04
-'•12
.17
.00
-•I'1 ,
.07 '
.06
.24*
.16 , ,
.03
-;12
.15 •
-.10
-1.2
.14
2.00
.01.
128
future
.09
-.17 '
-.27**
^.03
.08
.17
-.09
-.07
-.06
-.07
.12 :
.-.09
,.17
.10
-.04
.23*
.08
-.01
.01
.03
-.02
'.05 :
' .13 '
1.90
.02
1.27
tapwater :
.02
.10
-.04
• .07 •
, .02
-.07 ... •
.06
• —
—
.41****
-.04
.11
s-.04
.08
.10
.12
.23*
-.07
-.21*
'.17* '
-.05
-.02 •'-
•21
2.80
.0004
131
serious
.01 .
.20*
-.15
-.06
-.06
.06
.20*'
.36****
--.04
—
-.14
-.06
-.03,
-.06
-.01
-.12
-.12
.16
.09
.00
.06
-.24**
.32
4.00
.0001 .
131
coefficients (fl weights).

-------
Lay Views on Uncertainty in Health Risk Assessment                                        page 54

     CERTAINTY. Two independent variables were significant predictors (SCIUNCER and
SERIOUS) of desire for certainty (CERTAINTY). Respondents who disagreed that good science
has a range of uncertainty (SCIUNCER) expressed a greater desire for certainty. Those who
disagreed that there were serious environmental health problems where they live (SERIOUS)
were more likely to have a greater desire for certainty.
     NOWORRY. Four variables were significant predictors of worry (TAP WATER,
SCIUNCER, P_VULNER, and BAEDUC). Respondents with higher perceived risks of home
tapwater were more likely to worry about the water risks presented in scenario one. The belief
that good science has a range of uncertainty was positively correlated with not worrying.
Respondents who feel greater personal vulnerability were more likely to worry about the water.
risks in the initial scenario. Respondents with a bachelors degree were more likely to worry about
the water risks.
     CONCERN. Four of the independent variables significantly predicted concern about the
water in the drinking water scenario (ANTIEGAL, TRUST, ANIMAL, and TAPWATER).
Respondents holding egalitarian worldviews and having less trust in authority were more likely
to be concerned about the water. Those with a greater belief in the reliability of animal studies
were more likely to be concerned about the water. Higher risk perceptions of home tapwater was
positively correlated with greater concern about the water.
     GOVHONEST. Three variables were significant predictors of the belief that the
government is honest in drinking water scenario (TRUST, CERTAINTY, and EASYODDS).
Greater trust in authority was positively correlated with, the belief that, the government is honest.
Respondents with a greater desire more certainty were less likely to believe that the  government
was honest in the drinking water scenario. Those who understood that "you are less likely to get
a disease if the risk of getting it is 10 in 1,000,000 than if the risk is 1 in 1,000,000"   .
(EASYODDS) were more likely to believe in the government's honesty.
     GOVKNOWS. Two variables were significant predictors of the belief that the government
knows exactly what the risk is in the drinking water scenario (TRUST and ANIMAL).   '•
Respondents with higher levels of trust in government/authority were more likely to believe that

-------
Lay Views on Uncertainty in Health Risk Assessment
                                                                                   page 55
  the government knows what the risk* is in the scenario. People with a greater belief in the,
  reliability of animal studies were also more-likely to think that the government knows what the
  risk is in the scenario.        • -••             ;             ,

  Regressions To Predict Reactions to the Second Risk
 , Estimate, Reasons for Uncertainty,; and Risk Beliefs
     . Eight regressions were conducted to predict reactions to second'study (and second risk
  estimate), reasons for uncertainty and risk beliefs. Overall regressions predicting six of the
  dependent variables were statistically significant (p<.05).                .
    .  2NDEST.  Three variables were significant in predicting the belief that the second study
  was more competent' in the two-study scenario (FATAL, CERTAINTY, and CHILDREN).
  Respondents holding fatalistic worldviews, with a greater desire for certainty, and with children
  living in the household were more likely to believe that the second study (with the lower risk
  estimate) was more competent,  •     :                 ,           ,   ~
      COMPLEX. The two variables significant in predicting complexity as reasons for scientific
  uncertainty were trust in authority (TRUST) and genderc(SEX). Women and people with less
  trust in authority were more likely to see complexity as reasons for/scientific uncertainty.
      PAST. Three variables were significant in predicting the belief that problems with past
  information about risks were reasons for scientific uncertainty (TRUST, LIFEACC, and SEX).
  Respondents lacking trust in authority, women, and those with a belief that "my life is controlled
  by accidental happenings" are more likely to view these problems from the past as reasons for
  scientific uncertainty.                                             ..    .
      FUTURE.  Only two variables were significant in predicting the belief that, problems with •
  forecasting the-future were reasons for scientific uncertainty (TRUST and SEX). .Women and
,  respondents with less trust in authority were more likely to see these.problems as reasons for
  scientific uncertainty.       ,
   :   TAPWA TER. Three, variables were significant in predicting perceived risks of home
  tapwater (SERIOUS, AGE, and WHITE). Respondents who agreed that there were serious

-------
Lay Views on Uncertainty in Health Risk Assessment                                         page 56

environmental health problems where they live were more likely to have higher risk perceptions
of home tap water as were older and non-white respondents.
     SERIOUS. Two variables were significant predictors of the belief that there are serious
environmental health risks were one lives (TAPWATER/and CERTAINTY). Respondents with
higher perceived risks of home tapwater, environmentalists, and those with less desire for
certainty were more likely to believe that there were serious environmental problems where they
live.                                                        .                        .

Importance of the Independent Variables
     TRUST (i.e., trust in officials) was the most common predictor of the various dependent
variables, reaching statistical significance in 6 of the 17 multiple regressions run. Those who
trusted officials were much more likely to see government's discussion of a range of risk
estimates as implying honesty and  competence (GOVHONEST). This finding confirms our
Phase I regression findings. To a lesser degree, TRUST also predicted the belief that government
knew exactly what the risk was for the initial drinking water scenario. Earlier we had
hypothesized that this belief might indicate distrust, in the sense that government had a correct
point estimate of the risk, but only announced a range of risk estimates. But if it is the trusting
ones who are more likely to hold this belief, this hypothesis seems less tenable. Possibly the
belief that government knows the risk is another signal of a belief in its honesty and competence,
but if so this statement should have loaded on the same dimension under factor analysis.
     By contrast, those who trusted (TRUST) were /ess likely than others to agree with three of
the four sets of reasons for uncertainty (COMPLEX, PAST, and FUTURE). This may mean that
people who tended to distrust government on principle seized upon these reasons to justify their
                                                                v •
suspicions, on the grounds that all these things that could  "go wrong" showed that government
risk estimates could not be trusted. Earlier we had wondered whether the overwhelming
    «                                     _                     ,   .,
agreement with these sets of reasons for uncertainty truly  represented the Eugene sample's   -
beliefs about uncertainty. This regression result suggests that that doubt may be justified.
     TRUST was inversely related to CONCERN. Those  who trusted were less  concerned about
the drinking water contamination than others. This is consistent with other findings (e.g., Bord &

-------
  •>   •  -            •  '  • •    '          '     '.'      .''•-..•'"..          •
•f        Lay Views on Uncertainty in Health Risk Assessment                                     , .  page 57"

         O'Connor, 1992) that trust seems to reduce perceived risk. However, no significant relation
•.-';-   ,  between TRUST and two other measures of perceived risk—NO WORRY and ESTRISK—
         appeared,  undermining our confidence in a general relationship between trust and perceived risk.
              The three worldview variables other than TRUST had less of an impact on the dependent
;         variables in our regressions than expected, haying among them significant effects in only four of
         the 17 regressions. As noted earlier, beliefs like those included in our ANTIEGAL
         (anfiegalitarianism) have been associated with risk beliefs in some other studies. It was the
         strongest predictor of CONCERN, but—as also happened with TRUST (above)—it had no
         significant impact on NO WORRY and ESTRISK. ENVIRON (environmentalism) had a
         significant impact only on the belief that there are serious environmental health risks in one's
         community (SERIOUS).  .   /     ,            .                  '  '       '
              The final worldview variable, FATAL (fatalism), was the strongest predictor of 2NDEST,
         the belief, that the second risk study—which produced a lower risk estimate—was more
         competently done arid based on better scientific knowledge. It is not clear what this means, .given
         the much more common view that lower risk estimates are less trustworthy,  and majority
         disagreement (or inability to decide) about the second study's competence. Possibly fatalists
         were more likely to see competence in the second, lower-risk estimate because this allowed them
         to deny a risk that made them feel vulnerable (i.e., they might be "fated" to  suffer health
         consequences of the contamination). The fact that FATAL did not combine with seemingly
         related indices of vulnerability (e.g., AVOIDUNC, LIFEACC, P_VULNER) raises doubts about
         this hypothesis. On the other hand, the other significant predictors of 2NDEST—CHILDREN
         (having children in the household) and CERTAINTY (desire for certainty)—are certainly
       •  potential motives for denial of risk, iri the sense of accepting the lower risk estimate as more
         competent rather than continuing to doubt its validity.      •                     •

-------
Lay Views on Uncertainty in Health Risk Assessment            ,                             page :8


                                  Conclusions
    As a consequence of Phase II research on public responses to uncertainty in environmental
health risk assessment, we have a much better-grasp of the content and limits of lay conceptions
of risk uncertainty and its implications for hazard management. Certainly there are gaps
remaining in our understanding, which will be outlined in detail under "Research Implications,"
but these have more to do with generalizing our results to other populations, risk types, arid so
forth, than with introducing entirely new concepts. Our findings are as follows.

People Support Ranges of'Risk Estimates in General
    For the above-average-education sample in our Eugene study, the presentation of
uncertainty in the form of a range of risk estimates was deemed honest and competent in general,
although (as in Phase I studies) we continued to find a substantial minority who .questioned the
presentation of uncertainty on both grounds. This support for receiving a range of risk estimates
seemed to have been enhanced by a college-derived familiarity with the concept that uncertainty
is common in good science (regression analysis found an inverse relation between desire for
certainty and belief that science is inherently uncertain). However, they were not any more
familiar with concepts of dose-response relationships than earlier public samples, and had no
coherent idea of standard practice in government risk assessment. To the extent that such support
is knowledge-based, we would  not expect it to be more common among populations with
average education (i.e., less than in our sample).                                         .

Yet Uncertainty Makes Many People  Uneasy
  .'   About a third of our Eugene sample  rejected ranges and even risk estimates, saying they
preferred to be told simply that an environmental condition was safe or unsafe. As we noted
above, others doubt risk information that  does not incorporate ranges; together these beliefs
potentially put purveyors of risk estimates in a quandary, chancing alienation of some individuals
whatever they do. Multiple regression analysis suggested that desire for certainty was associated
with beliefs that good science is certain and that there are serious local environmental health
problems.

-------
  Lay Flews on Uncertainty in Health Risk Assessment                .,               '     ,     page)?

  Uncertainty in Government Risk Estimates Is
  Particularly Suspect                             .
 /      While a majority of our respondents, as in Phase I, found government discussion of a range
  of risk estimates to be competent, they were.more dubious about whether this indicated honesty.
  We have suggested that this reaction is partly related to two other findings; that zero as a lower
  bound on a range of risk estimates, rather than a small positive bound, seems to raise doubts
              .            .             l "        •••-'.           • "v    -      -
  about honesty and competence all by itself, and that people believe government discusses
  publicly only high risk items. The presence of zero led people on average to feel that the risk was
.  probably higher, to believe that the government could be wrong, and to worry significantly more
  often than when a small positive lower bound was used. Citizens who see zero risk as unlikely or
  impossible, and high-risk information as both technically more complete (see Phase.! results) and
  politically more likely to appear in-public discussion, would be suspicious of a government risk
  estimate ranging from zero to some higher number, even if they supported risk ranges in general.
  If a risk exists at all—and by this definition it must, because government announces  it—then it
 . must be a high risk, and the range should include only risks higher than zero. On this
  interpretation our scenarios stimulated general distrust of government risk ranges because they
  included the lower bound of zero, which the National Research Council has characterized as EPA
  "boilerplate."' Distrust for some people also seemed to be ideologically-based, or derived  from
  beliefs about how government risk estimates were developed (in technical or political terms).
  Belief in government honesty and competence was related to low desire for certainty, high trust
 ' in officials' (a combination of expectations of honesty and deference, to authority), and one of our
  measures of difficulty with arithmetic.                  '                             ; ;   '.

  Environmental Health Risk Uncertainties  Are Seen as
  Special
       The exact same range of risk estimates that elicited concern when it applied to ,
  environmental health elicited little or no concern when applied to transportation (mortality),   ,
  financial or recreation risks. People offered contextual information (work requirements,
.  -familiarity, offsetting joys) for their differing reactions to the latter uncertainties. Because the  ,
  transportation risks (airplane and automobile  crashes) involved death, we cannot explain this
  reaction solely on the grounds that the stakes  with environmental health 'are different (although

-------
Lay Views on Uncertainty in Health Risk Assessment                                         page 60

that was the reasoning offered by focus group members about the financial and recreation cases).
Varying reactions within the environmental health category (drinking water vs. soil vs. air
contamination) seemed driven more by criteria of morality or protective opportunities than by
uncertainty considerations (as our comparison of "natural" to "human" sources of risk in Study
1, Phase I, suggested as well).

All  Range Numbers Are Not Interpreted Equally                            .
     People do not seem to treat every number within a range as .equally probable, despite the
only signal of varying probabilities across risk estimates being a brief statement that one risk   .
level is "most likely." Instead, on average they appear to treat higher risk numbers as more
probable (compare to the Phase I finding that lower risk numbers were seen as "preliminary"
estimates), confirming an earlier hypothesis (Viscusi et al., 1991) of an "upward bias" in
response to ranges. By contrast, use of zero as a lower bound seemed to be particularly disturbing
(see above), although a majority responded to questions using this bound no differently than they
responded to a lower bound of 1 in 10 million.

Explanations of Uncertainties Are Confusing, Irrelevant,    .
or Troubling
     Explanations of extrapolation from animal or high-dose human data did not differ in their
impact, and qualitative reactions suggest people largely found them confusing or irrelevant.
Earlier research (Kraus et al., 1992), confirmed by this study, found laypeople and experts,
divided over the validity of using animal data.. This may partly explain why explaining
uncertainty in risk estimates as due to extrapolation from animal data was seemingly without
effect ("seemingly" because we attempted no experimental test in this study, although a Phase I
experiment found no evidence of impact). Poor responses to explanation of high-dose
extrapolation in interviews seemed partly  due to use of the word "chance" to describe how the
high* dose was received by the other population, perhaps stimulating aversion to a risk that thus
seemed more likely to affect the interviewee (on the grounds that chance cannot be avoided).
Extrapolation from factory workers to members of the general population also seemed troubling
for a plurality of our Eugene sample. Explanation of how a safe reference concentration is
calculated for a noncarcinogen also confused or upset almost all interviewees and respondents to

-------
   Lay Views on Uncertainty in Health Risk Assessment            •      .'.''.'•'.    page 61

   open-ended questionnaires. By. contrast, the Eugene sample overwhelmingly agreed with reasons
   for risk uncertainty stemming from measurement or complexity, or problems associated with past
   or future risks, derived from the risk literature (Rowe, 1994). On methodological grounds alone
   we raised the question of whether these are their own views; the regressions showing distrust
   significantly related to agreement suggest that people were finding these reasons justifying prior
   distrust.                                                        '
       These findings do not necessarily mean explanations of uncertainty are useless. A minority
(-  of our qualitative respondents found them enlightening, even enthralling; some respondents
   volunteered reasons (varying exposures or susceptibility across subpopulations) why they found
   ranges of risk estimates preferable to point estimates, or said they would distrust risk estimates
   without explanations of how they were derived. However, any effort at explanation must deal
   with the challenges of salience, trust and scientific understanding. Some of the lack of impact
   among our respondents may be due to our using hypothetical cases of drinking water
   contamination that did'hot motivate people to seek and process information; yet people who
   believe they face a real life-and-death situation may put a low priority on understanding
      ; , '           '  \     f    •   . '
   uncertainty as opposed to protecting themselves. As  our regression results imply, people who are
   disposed to distrust officials anyway may seize explanations of uncertainty as reasons for such
   distrust: Finally, many citizens do not understand how science works or have difficulties with
   mathematical information. Despite the general poor role of measures of mathematical
   competence in this study, successful explanations of uncertainty will probably require a basis in
   lay cognitive models of both science in general and what scientists are doing (and why) when
   they, for example, extrapolate from animal data to human risks.

   Shifting or Disputed Risk Estimates Do Not Reassure
       Changes in risk estimates do not seem to 'affect public beliefs about risks strongly,, except
   possibly to make them more suspicious if the estimates decrease. Although a majority in our
   Eugene sample did feel more confident in their drinking water's safety after such a decrease, our
   scenario concerned a deliberately unlikely change from a "most likely" risk of 10"6 to 10~9, and
   even, then a full third of respondents were not reassured. The basis for the latter reaction is
   unclear: the only variables predicting a belief that a lower follow-up risk estimate was more

-------
Lay Views on Uncertainty in Health Risk Assessment                                          p
-------
 Lay Views on Uncertainty in Health Risk Assessment          .     '     '•  •              >   \ ,'pape 53

 (except for "trust in officials") were dominant explanations of these responses among-those we
 were able to test.   '              :   r             .  •    -
                    -             '     "          "    -       " '      -        - "   '         \
 Risk Estimates and Their Uncertainties Are Not Critical    . ,'                      ',
 to Most People's Views of Environmental Problems
     Willing though they were, by and large, to answer our questions about uncertainty in
 environmental health risk assessment, interviewees and other qualitative respondents made quite
 clear that the central issue for them was prevention and cleanup of environmental pollution. They.
 made it clear that for them risk numbers were generally beside the point. Although we did not
 give Eugene respondents to the closed-ended questionnaire a chance to express this view
 directly, the third of the sample who did not want to see ranges of risk estimates plausibly fall
 into the same category. Nor can we assume that the rest of the sample welcome such ranges as
 much more .than yet another piece of information to be weighed for its credibility.
     We think this is the key message to be taken from our study. Risk assessment will clearly
 remain an important component of environmental problems; making risk estimates available to
 the public will not'only continue, but is part of their right to be informed and participate in
 decision making (Johnson, and Sldvic, 1994b). However the debate over the role of uncertainty in
 risk assessment and risk management is resolved, at some point it is likely (even if only
 qualitatively) that such uncertainty will be discussed publicly. Such discussion will undermine
 the credibility arid effectiveness of government if it is made the centerpiece of attention, rather
than prevention and cleanup of pollution. Uncertainty is a fact of life, but life goes on; similarly,
citizens expect government action on pollution in spite of uncertainty, and may suspect
uncertainty, is being raised merely to justify inaction. Attention to uncertainty can be critical to
deciding which action to take, but this public discussion must take due account of the wider
context to ensure a successful and informed outcome..

-------
Lay Views on Uncertainty In Health Risk Assessment                                         page 64


                           Practical Implications
    As we have noted several times, generalization of our results should be done cautiously.
However, it is unlikely that any substantial knowledge-based differences in response to
uncertainty will be found from applying our methods to populations less educated than pur
college sample. If a group relatively knowledgeable about uncertainty in science is troubled by
uncertainty in risk assessment, there seems little hope of finding a less concerned public
audience. Ideologically-based differences may vary more widely among the general public than
in our Eugene sample, of which 67% was "environmentalist," 79% "egalitarian," 88%
distrustful of authority, and 92% not fatalist, as measured by worldview statements in the  ,
questionnaire. Nevertheless, we believe there are reasonably strong grounds for making
recommendations to government (and other institutions) about practical implications of our
findings.
     Our interviewees and focus groups in particular stressed their interest in having
environmental problems prevented, and cleaned up swiftly if not prevented; getting risk
estimates was of far less interest, even for those who preferred a range of risk estimates to simply
being told that conditions are safe. As noted earlier, we conclude from this that communication of
risk estimates, whether uncertainty is included or not, should be secondary to dealing with the
risk management issues posed by our respondents. This may seem like stating the obvious, but
official emphasis on communicating "the risk numbers" is still common enough to make
repeating this argument worthwhile. This does not mean that communication of risk estimates is
useless or unnecessary,, but it conveys our feeling that an emphasis solely or primarily 'on. such
communication will be interpreted by most citizens as an attempt to hide incompetence or the
"true" risks. This interpretation would help neither agencies nor the public.
     Some of our recommendations apply to forms of communicating uncertainty:
    i                                           - -
     •  Use of "zero" as a lower bound in a range of risk estimates should be used only when the
        available data clearly show this is a likely "true" level of risk. The doubts that this term
        raised among our respondents about government honesty and competence, plus the
        technical arguments about its validity referred to earlier (NRC, 1994), do not justify its

-------
                    ..             .                _.            -    .
 Lay Views on Uncertainty in Health Risk Assessment          '                        _    '    page^J

        use as boilerplate. Very small positive lower bounds may be much more 'acceptable: a  :
        majority of-our respondents saw no difference between zero and 1 in 10 million as a
           .' •       '       '     '              , ',                  ,      \     ' -       . '
   ,' .   lower bound, and yet those greatly disturbed by zero were far less affected by use of the
        latter figure as a lower bound. If there is no alternative to using the term "zero" in a risk
        estimate, clear explanation of the reasons for its presence should be included. Although
        (as mentioned below) the explanations for uncertainty we provided rarely "worked" for -
        our respondents, in this case omitting an explanation only means that audiences will
        substitute their,own explanations (of dishonesty or.incompetence or both).
    . •  About two-thirds of our respondents "assumed the worst" (thought the high end of a
        range of risk estimates to be more likely); to the extent that this contradicts the beliefs of
        officials and risk assessors, the discrepancy should be pointed out. 'On average people's
        estimates of risk to themselves of to their community based on our scenario were not
   .     greatly skewed upwards; on the other hand, simply saying that one intermediate, estimate
        was "the most likely risk" did not prevent an upward bias, even if it seemed to avoid
        projections that every risk level within the range was equally probable. This suggests that
        a more explicit specification of the reasons for the (presumably) low probability of a
        range's upper boun4 might help some members of public audiences put the range into
        perspective. However, as with many of our other recommendations, these explanations
        should be, brief (with fuller details for those who wish them available separately) and
'''.       undue faith in their effectiveness should be avoided.      ''-•"'            ,
     •  Phrases like "95% confidence" or "95% certain" aroused suspicion among our
 -       •             • .       .         -.      -        '   . '   '   '    '' "        •      •''.•'
        respondents, and we suggest they be used very  cautiously if at all. One option is to simply
        announce upper and lower bounds to  risk ranges, without specifying their link to
        confidence limits except in the more detailed information available to those who request •
        it. What that more detailed explanation should be, however, we cannot say on the basis of
              **m            •            .  -         '              '                 •    '
   . '    our current findings.  .                                             "
     •  Our Phase If research added little to what we had learned from Phase I.about the
        helpfulness of graphics: just as the earlier research suggested a simple line graphic made
        the presence of uncertainty itself more obvious (but raised doubts about the truth of the
      .  risk information), the graphics appearing in figures 1 to'3 apparently conveyed

-------
                                                                                             >
                                                                                               i-
Lay Views on Uncertainty in Health Risk Assessment                                          page 66

       uncertainty without implying equal probability of all risk levels within the range.
       However, a more complex (and more accurate and potentially informative) graphic
       recommended by a National Research Council body (1994) was quite confusing. We
       suggest that simpler graphics be used, if at all, until further research can. specify what
       kinds, if any, graphics would be most helpful to citizens.
     Some recommendations relate to use of explanations of uncertainty:
     •  Attempts to explain in detail why a range of risk estimates has been presented should be
       directed primarily at audiences that express an interest in understanding the reasons.       .
       Otherwise, explanations will be seen as irrelevant or distracting from central issues (e.g.,
       risk management). A simple statement may be enough in most cases, such as: "We are
       providing a range of risk estimates because [for example] we want to account for the fact
       that different people may have different levels of exposure to the problem; if you have a •
       special interest in information on this uncertainty, we have a handout." Some cases (e.g., v
       those where zero is a lower bound) may require explanation anyway, to avoid even more
       serious communication problems, but these explanations should be few and brief.
     •  Responses to  our explanations of uncertainties based on extrapolation from animal data or
       high-dose human data did not vary significantly.  Combined with our Phase I  finding that
       adding such an explanation to a hypothetical news story did not produce different
       responses than a story without explanation of uncertainty, this might imply that
       explanations are useless and should not be attempted. "We do not recommend offering no
       explanation of uncertainty. Our Phase II finding only suggests that animal and high-dose  .
       explanations are equally good or poor; the Phase I finding may have been due to
       information overload (the "explanation" story included much other information) rather
       than irrelevance of the explanation; other phrasings than the ones we used might work
     ,  better. ~*
     •  We do not at this time recommend using analogies to uncertainty in financial or other   ..
       nonhazard areas to explain environmental health uncertainties. The aim of such
       explanation might be to clarify how uncertainty works, or how it.can help one make
       better decisions. We are concerned that the public may interpret this instead as an

-------
Lay Views on Uncertainty in HedthRisk Assessment •      •'           '                       page 67

       .argument that they should accept risk and uncertainty for environmental health if they ,-
       accept it in investments or recreation or transportation. Our Phase II study suggests they
      • are likely to strongly resent such an argument. Because explaining the difference between
       their inference and the intention will be challenging, using the analogy does not seem
       worth the dangers.                     '  ' \' ' '  •   '  .                  '
     • Extrapolation from animal data faces skepticism from both citizens and experts (Kraus et
       al., 1992; Slovic, in press; Slovic etal., 1995), and so will explanations of its role in risk
       uncertainty. Direct acknowledgment of this skepticism, that such data are the only ones
       available for this pollutant, and that having some rough.idea of the risk is better than
       having no idea at all (the only alternative in this case), may be the best options for
       explaining this kind of uncertainty. We do not guarantee that this explanation will satisfy
       all or even most audiences.                                          \              '
     • Extrapolation from high-dose data resulting from occupational exposure or accidents
   •  .' seems to evoke doubt as well, perhaps due to concern that occupational data are not
       relevant (or, in the example we offered in interviews, that others' exposure to a pollutant
       by accident or "chance" may make people feel more vulnerable than otherwise). The
       recommendations (and caveats) for explaining animal data apply here as well.     ' >  •
     Administrator Browner,  in her memorandum on EPA's Risk Characterization Program
(1995), said that while "the final risk assessment'document... is available to the public, the risk
communication process may be better served by separate risk information documents designed
for particular audiences." We endorse this statement wholeheartedly, in the sense that details
should be reserved for "backup" documents whose existence would'be publicized.

-------
 Lay Views on Uncertainty in Health Risk Assessment                                          page 68


                           Research  Implications
     Our study raised many questions for future research. Those that concern communication
 about uncertainty in risk assessment include:                                  .
     •  Our research examined lay response only to uncertainties in risk assessments based upon
        toxicological studies. Although we have no reason to believe that the general reaction to
        epidemiological uncertainties would be much different, research on this topic would
        allow testing of reaction to'particular kinds of such uncertainties and to alternative
        explanations of them.                                         ,
     •  Phase II research focused on drinking water scenarios. In a few interviews reactions to
        other environmental health scenarios (e.g., air pollution from a local factory; soil
        contamination in the neighborhood) varied from those given by .the same person to the
                                          '  -                        , '•
        drinking water scenario. These variations, however, were not the same across
        interviewees, and did not seem to be based on uncertainty as much as other factors (e.g.,
        perceived immorality of the factory's emissions; perceived availability of protective
        options). But we did not test more widely for the presence of similarities or differences in
*       reactions to uncertainty across hazards (one study in Phase I found differing reactions,
        unrelated to uncertainty, between natural radiation and a hazardous-.waste-site chemical).
        Although we are confident that reactions of uncertainty will not vary widely across
        hazard types, research testing this hypothesis .would be valuable.                •
     •  The potential role of graphics in communicating risk uncertainty has been little explored
        (Ibrekk  & Morgan, 1987), and our results make the issue seem even more complex. We
        wonder whether the graphic proposed by Ibrekk and Morgan (a cumulative distribution
        function above its equivalent probability distribution) can be explained simply and
        briefly.  Certainly the graphic proposed by the National Research Council that we showed
     '  focus groups elicited nothing but confusion. Our simpler graphics, although they did not
        seem to make people think that risk estimates within the range were equally probable,
        may not be as helpful to the public as agency officials might like. Research focusing on
        these and alternatives seems valuable.
      •  More extensive testing of reactions to various explanations of uncertainty is needed. Our
        explanations of animal-data extrapolation, high-dose extrapolation (including citing

-------
Lay Views on Uncertainty'in Health Risk Assessment    ,         ,                  "          page (J1?

       chance or workers as the source of the data), and construction of a reference
    •   concentration were largely taken'to be confusing, irrelevant, or upsetting. Because we
       expect that some such explanations will be used when agencies and others present
       uncertainty, more study (including development of alternate versions, in language and for
   :   /different audiences) is necessary to provide agencies some confidence that they will not
       make matters worse with explanations. As part of this research, we need to understand  '
       how. widely explanations might be treated as reasons for distrust, rather than .as   .
       enlightening.     .                                      .
    •  Our findings that zero as a lower bound of a range elicited strong negative reactions are
       striking, but further experimental work is needed to generalize these results (which, after
       all, did not appear in Phase I, Study 1). Among other things, we need to test whether the
       alternative suggested earlier makes a difference. This concerned having uncertainty about
       whether a chemical is a human carcinogen represented as "a risk of zero, if it is not a
       carcinogen, or a risk ranging from [a low positive lower bound] to [a higher upper
       bound]," rather than as "could be as low as zero or as high as...," which is misleading.
  •  Several other research topics are less directly focused on uncertainty, but are important:
    •  Understanding of lay cognitive models of science and risk assessment-including, but not
       stressing, uncertainty—-would help usDesign communications that better explain these
       critical factors in hazard management.             -'•         '   •      '    . -  ;
  •  •  Trust in officials was the most powerful, although by no means universal, predictor of
       responses to government risk ranges; although "trust" has elicited much attention from
                                         ^       - ' ,      "      .                    • '    i
       researchers in the last several years,/little is yet known about its antecedents. Further
       research, perhaps using think-aloud protocols while respondents read a risk estimate
       announcement (with or without uncertainty), would be valuable.   .
    , •  Similarly, we were unable to clarify how people judge the honesty and competence of
       officials with regard to risk estimates (much less on other risk management tasks). Some
       trust researchers suggest such judgments stem from the perception that officials share the
       observer's values, but others believe these judgments reflect other factors.

-------
Lay Views on Uncertainty in Health Risk Assessment    •    ,                                page ~0


                                  References
Bord, R. J., & O'Connor, R. E. (1992). Determinants of risk perceptions of a hazardous waste
    site. Risk Analysis, 12,411-416.
Bostrom, A., Fischhoff, B., & Morgan, M. G. (1992). Characterizing mental models of hazardous
    processes: A methodology and an application to radon. Journal of Social Issues, 48, 85-
    100.                                                                         :.   •
Browner, C. M. (1995, March 21). EPA risk characterization program (memorandum).,
    Washington, DC: U.S. Environmental Protection Agency.              '
Carpenter, R. A. (1995). Communicating environmental science uncertainties. Environmental
    Professional, 17,127-136.
Cole,  L. A. (1993). Elements of risk: The politics of radon. New York: Oxford University Press.
Dake, K. (1991). Orienting dispositions in the perception of risk: An analysis of contemporary
    worldviews and cultural biases. Journal of Cross-Cultural Psychology, 22, 61-82.
Earle, T.C., & Cvetkovich, G. T. (1995). Social trust: Toward a cosmopolitan society. Westport,
    CN: Praeger.
Finkel, A. (1994). Stepping out of your own shadow: A didactic example of how facing
    uncertainty can improve decision-making. Risk Analysis, 14,751 -761.
Garrison, J. R. (1991, April). Keynote Address. Presented at the International Radon Symposium
    on Radon and Radon Reduction Technology, Philadelphia, PA.
Goldstein, B. D. (1995). Risk management will not be improved by mandating numerical
    uncertainty analysis for risk assessment. University of Cincinnati Law Review, 63(4), 1599-
    ,1610.    - .                                               .'      -
Ibrekk, H., & Morgan, M. G. (1987). Graphical communication of uncertain quantities to non-
    technical people. Risk Analysis, 7, 519-529.

-------
Lay Views on Uncertainty in Health Risk Assessment            ,:       •        ;            page7!

Johnson, B. B., & Slovic, P. (I994a). Explaining uncertainty in health risk assessment: Effects
     on, risk perception and trust (Phase 1 final progress report to U.S. Environmental Protection
     Agency as part of cooperative agreement no. CR820522). Eugene, OR: Decision Research.,'
Johnson, B. B., & Slovic, P. (1994b). 'Improving' risk communication and risk management:
   '  Legislated solutions or legislated disasters? Risk Analysis, 14, 905-906.
Johnson, B. B., & Slovic, P. (1995). Explaining uncertainty in health risk assessment: Initial
                                                       ;      '•;.''        •   -
     studies of its effects on risk perception and trust. Risk Analysis, 15, 485-494.
Kraus, N., Malmfors, T., & Slovic, P. (1992). Intuitive toxicology: Expert and lay judgments of
     chemical risks. Risk Analysis,  12(2), 215-232.           '
MacGregor, D. G., Slovic, P., & Morgan, M. G. (1994). Perception of risks from electromagnetic
     fields: A psychometric evaluation of a risk-communication approach. Risk Analysis, 14(5),
     815-828.                     '                 '
Miller, J. D. (1993, November 22). Scientific literacy: An updated conceptual and empirical
     review. Paper presented at the European Community Conference on the Future of Scientific
     Culture, Lisbon. -
National Research Council (NRC). (1994). Science and judgment in risk assessment.
     Washington, DC: National Academy Press.  ,
Peters, E., & Slovic, P. (in press). The role of affect and worldviews as orienting dispositions in
     the perception and acceptance of nuclear power. Journal of Applied Social Psychology.
Rayner, S., & Cantor, R. (19,87). How fair is safe enough? The cultural approach to societal
     technology choice. Risk Analysis, 7, 3-10.    .
         ;    : -     -       •             -              •
Riskpolicy report.^1995, January 20). pp. 12-13..                             ,
     i          .      •'     •             .'               , -  .
Rowe, W. D. (1994). Understanding uncertainty. Risk Analysis, 14(5), 743-750.
Shaver, K. (197.0). Defensive attribution: Effects of severity and relevance on the responsibility
          -',"-•.        -                       '          '           '    v,
  ,   assigned for an accident. Journal of Personality Social Psychology, 14,101-113.

-------
Lay \'ie\vs on Uncertainty in Health Risk Assessment                                         page ~1

Slovic, P. (in press). Trust, emotion, sex, politics, and science: Surveying the risk-assessment
    battlefield. In M. Bazerman, D. Messick, A. Tenbrunsel, & K. Wade-Benzoni (Eds.),
    Psychological perspectives to environment and ethics in management. San Francisco:.
    Jossey-Bass.
Slovic, P., Malmfors, T., Krewski, D., Mertz, C. K., Neil, N., & Bartlett, S. (1995). Intuitive
    toxicology II: Expert and lay judgments of chemical risks in Canada. Risk Analysis, 15(6),
                                     -                            '                   ^
    661-675.
Stone, E. R., Yates, J. F., & Parker, A. M. (1994). Risk communication: Absolute versus relative
    expressions of low-probability risks. Risk Analysis, 60, 387-408.
United States Environmental Protection Agency (USEPA), Radon Division, Office of Radiation
    Programs. (1992, May). Technical support document for the 1992 citizen's guide to radon.
    Washington, DC: U.S. Environmental Protection Agency.              ,
Viscusi, W. K., Magat, W. A., & Huber, J. (1991). Communication of ambiguous risk
    information. Theory and Decision,  31,159-173.
Weinstein, N. D. (1980). Unrealistic optimism about future life events. Journal of Personality
    and Social Psychology, 39, 806-820.
Weinstein, N. D. (1987). Public perception of environmental hazards: State-wide poll of
    environmental perceptions. (Final report to New Jersey Department of Environmental
    Protection). New Brunswick, NJ: Rutgers University.
Weinstein, N. D., Sandman, P. M., & Roberts, N. E. (1989). Public response to the risk from
    radon, 1988-1989 (Research contract C29418). Trenton, NT: Division of Environmental'
    Quality, New Jersey Department of Environmental Protection.

-------
                                                Appendix I



                                Fourteen Items Used in Qualitative Data Collection


  NOTE: ftem numbers are those identifying items developed during interviews, which were retained in focus groups and
  the open-ended questionnaire (some with slight wording changes) to aid comparing responses across the various data
  collection methods.    .

                               ' '       '      '                          f '-                 •      ',..'.•

                       '           .    '          ITEM2a                           '


  The government announces that the water you drink contains a level of chemical X that poses an extra health risk of
  getting.non-fatal kidney damage  of 1 in 1,000,000 (one in one million) over a lifetime of drinking that water.


;,                     _.',.,         ITEM 3 [focus group]         •                      •     .


  the government announces that the water you drink contains a level of chemical X that poses an extra health risk of
  getting cancer of 1  in 1,000,000 (one in one million) over a lifetime of drinking that water.  The government says that
  is the most likely risk, but it says  the true risk could be as low as zero or as high as 1' in 100,000.


                         ,.                 '        ITEM 4    ;.      '           -..'.'•••'..      '


  A government study was dohe-of the water you drink a few years ago, which contained (and still contains) small amounts
•  of chemical X.  It found that the most" likely extra level of risk of getting cancer from.drinking this water for your entire
  lifetime was 1 in 100,000 (one in one hundred thousand). This was below the drinking water standard for this chemical
  at the time, so no action was taken and the amount of the chemical'in the water has stayed the same. The. government
  did another study recently, using new scientific information about the chemical's effects, and concluded that the most
  likely level of risk was 1 in  1,000,000 (one in a million).                            •         ,


                                 ITEM 5 [both focus'group and questionnaire]


 The government announces that there is a 5%  chance that the extra level of risk from drinking chemical X in the water
 you drink for your entire lifetime is above 1 in 100,000 (pne in one hundred thousand), and a 5% chance that the risk
 is zero.  The government says this means there is a 90% chance that the  true risk is between zero and 1 in-100,000.


  "'•".,•   •          .           .   .           /ITEM 6                          '    '    * '•-  '    '


 The government announces  that the water you drink'contains  a level of chemical X that poses an  extra health risk of
 getting cancer of 1 in 1,000,000 (one in one million) over a lifetime of drinking that water. The government says that'
 is the most likely risk, but the true risk could be as low as zero or as high as 1 in 100,000. The government says that the
 reason for this range of risk estimates is that the only scientific studies of this chemical's effects on cancer risks involved

          '       '                    .      '     ', 5,1 .

-------
 laboratory tests with animals. It is not clear that the way an animal reacts to a chemical will reliably predict how a human
 would react to the same chemical. Different assumptions about how to predict human risks from animal reactions to the
 chemical result in different risk estimates.                ,
                                                  ITEM?


 The government announces that the water you drink contains a level of chemical X that poses an extra health risk of
 getting cancer of I in 1,000,000 (one in one million) over a lifetime of drinking that water.  The government says that
 is the most likely risk, but the true risk could be as low as 0.01 in 1,000,000, or as high as 10 in 1,000,000. The
 government says that the reason for this range of risk estimates is that the only scientific studies of this chemical's effects
 on cancer risks involved cases where people were exposed to much larger amounts of the chemical in their drinking water
 than appear in your drinking water (at the time they were exposed, no one knew this chemical could affect health). It
 is not clear that the cancer-causing effects in these people of high levels of the chemical reliably predict how a human
 would react to the much lower levels of the same chemical in your water.  Different assumptions about how to predict
 risks from low levels of the chemical, when only information about risks from high levels is available, result in different
 risk estimates.                                                                .
                                            ITEM 8 [focus group]                                    ,       '


 The government announces that its scientists have calculated the risk of getting cancer from drinking the water you drink,
 which has a small amount of chemical X in it, for an entire lifetime. It is 95% certain that at least 90% of the population
 has an extra risk of no more than 1 in 1,000,000 (one in one million). A small proportion of people who are more likely
 than others, when exposed to a cause of cancer, to get cancer may have a risk as high as 10 in 1,000,000 (ten in one
 million) if they drink this water for their entire lives.


                                                 ITEM 10                                    '


 The government announces that the water you drink contains a level of chemical X that poses an extra health risk of
 getting cancer of 1 in 1,000,000 (one in one million) over a lifetime of drinking that water. Most  scientists agree with
 this number, and many think that the true risk could be as low as zero. However, a few scientists believe that the true
 risk could be as high-as  100 in 1,000,000.  Both groups have reputations as being competent scientists.


                                                ITEM 1 la


 The government has found a chemical in your drinking water that can cause non-fatal kidney damage in laboratory rats.
 The level of the chemical which the government considers safe for humans to be exposed to is 1  part per  100 million.
 The level of the chemical in your drinking water is smaller than 1 part per 100 million.


                                                ITEM lib                    .''.-•;


The government has found a chemical in your drinking water that can cause non-fatal kidney damage in laboratory rats.
The level of the chemical which the government considers safe for humans to be exposed to is 1  part per  100 million.
The level of the chemical in your drinking water is 1.2 parts per 100 million.

                                                   52

-------
          ;     '         '   .       ,               ITEM lie


 The government has'found a chemical in your drinking water that can cause non-fatal kidney damage in laboratory rats.
 However, this chemical did noj cause kidney damage when fed to laboratory rats in doses of 1000 parts per 100 million
 or. less. Government scientists took a cautious approach to calculating a safe dose for humans, although it is possible
 1000 parts per 100 million would be safe for them top. They divided this number by 10 to account for the fact that the
 rats were only fed the chemical/or a short time, and humans (at worst) might be exposed for an entire lifetime; divided
 by 10 again to account for the possibility that humans might be more sensitive to the chemical than are rats; and divided
 by 10 again to account for humans who may be more sensitive to the chemical than, the average human. Thus the level
 of the chemical which the government considers safe for humans to be exposed to is .1 part per 100 million.  The level
'of the chemical in your drinking water is 1.2 parts per 100 million.


                           .       '              'ITEM 13       .    .      .     ,   '


 Chemical X occurs naturally in drinking water in this part of the country. The government standard for a safe level of
 this chemical, assuming that a person drinks this level for an entire lifetime, is" 1  part per  100 million. The government'
 has found that the level in your drinking water system averages 0.9 parts per lOO.million.  Due to random changes in the
 natural level of the chemical in  the water source, the government says the level of chemical X in the water you drink at
 any particular time and place can range from undetectable amounts to 1.2 parts per 100 million.
                                                   53

-------
                                                 ITEM 17
The government announces that the water you drink contains a level of chemical X that poses an extra health, risk of
getting non-fatal kidney damage of 1 in  1,000,000. (one in one million) over a lifetime of drinking that water. The
government says that is the most likely risk, but it says the true riskcould.be as low as zero or as high as 1 in 100,000
(one in one hundred thousand).
                                                ITEM 18
The government announces that the water you drink contains a level of chemical X that poses an extra health risk of
getting cancer of 1 in 1,000,000 (one in one million) over a lifetime of drinking that water.  The government says that
is the most likely risk, but it says the true risk could be as low as 0.01 in a million or as high as 10 in 1,000,000 (ten in
one million). An environmentalist group responds by saying that the most likely risk is probably 10 in  1,000,000, and
it says the true risk could be as high as 100 in 1,000,000 (one hundred in one million).               '    .
                                                    54

-------
' ' Risk Analysis,- Vol. 15, No. 4, 1995    , "            .    .      •
                                                  Appendix H
  Presenting Uncertainty in  Health  Risk Assessment:  Initial
  Studies of Its  Effects on  Risk Perception  and  Trust
  Branden B. Johnson1-3 and Paul Slovic2
                             Received August 29, 1994; revised March 20, 1995
                             Some analysts suggest'that discussing uncertainties in health risk'assessments might reduce citi-
                             zens' perceptions of risk and increase their respect for the risk-assessing agency. We tested this
                             assumption with simulated news'stories varying simple displays of uncertainty (e.g., a range of
                             risk estimates, with and without graphics).  Subjects from Eugene, Oregon,.read one story each,
                             and then answered a questionnaire. Three studies tested between 180 and 272 subjects each. Two
                             focus groups obtained more detailed responses to these stories. The results suggested that (1) people
                             are unfamiliar with uncertainty in  risk assessments and in science; (2) people may recognize un-
                             certainty when it is presented simply; (3) graphics  may help people recognize uncertainty;  (4)
                             reactions to  the environmental problems in the  stories, seemed affected less by presentation  of
                             uncertainty than by general risk attitudes and perceptions; (5) agency'discussion of uncertainty in
                             risk estimates may signal agency honesty and agency incompetence for some people; and (6) people
                             seem to see  lower risk estimates (10~6,-as opposed  to 10~3) as less credible. These findings, if
                             confirmed, would have important implications for risk communication.


                             KEY WORDS: Risk perception; risk communication; risk assessment; uncertainty.
  1. INTRODUCTION    ...                ,            addressed in thepresent studies. Here we use "uncer-
                    '.";.•                                  tainty" to mean a risk assessment presented in terms of
      •An abiding issue in risk communication is how best     a range of risk estimates, rather than as a point estimate.
  to convey, risk information  from scientists and officials
  to citizens. A key element of such technical information
 . is uncertainty, yet neither  for this topic nor for risk char-     2. BACKGROUND    ..      '
  acterization materials in general has there been "system-         •            '                  '  '  .
  atic study of.. .their comprehensibility and usefulness         There have been numerous claims that explicit dis-
  to various types of users"")  (p. 14). In particular, no one'     cussion of uncertainties in risk estimates would, among
  knows how presentations of uncertainty in risk estimates     other  benefits, improve'the public's views of environ-
  affect'public risk perceptions or citizens' trust in risk  ;   mental hazard management. One alleged effect is that be-
  managers.                —          •                 ing open about uncertainty will enhance credibility and
      We report Tiere the results of four preliminary stud-.     'trustworthiness*" presenting important uncertainties in a
  ies of this topic, funded by the U.S. Environmental Pro-   .  risk assessment will improve' "public confidence  in the
  tection Agency^' Defining  and calculating uncertainty     quality Of our  scientific  output..'."'5' (p. 1,  emphasis
  itself has been the subject of entire books/3) and was not ^    added).  Another argument is that  citizens will' make
                           '     ,   •                  '*-   '-more informed'choices when the range of risks [from a
  ^sk Communication Unit, Division of Science and Research,: New      iyen ^^ ^ which Qne % expQ^ ^ considered"(«
   Jersey Department of Environmental  Protection, CN409, Trenton,   .  °          J                  r
   New Jersey 08625.   ''       '                          ,    (P- °7)-    ,  ,    -                '    . '•  .  ,
  2 Decision Research, 1201 Oak Street, Eugene, Oregon 97401.              Few. Studies of public response to risk uncertainty
  .3To whom all correspondence should be addressed.       '           have been done. Risk perceptions of a hypothetical haz-
               '.,..-",'...    '.„•';•  .485'      •'•'-       •       •            '•''.'
                                                            .    .      0272-4332/95/0800-0485S07.50/I O 1995 Society for Risk Analysis

-------
-486
                                                                                         Johnson and Slovic
ardous waste case were unaffected by ,a caution about
the uncertainty of risk estimates in general/7' Views on
global warming  effects  were unaltered by large varia-
tions  in the timing and magnitude of scientific predic-
tions about such warming.(8) Another study asked people
to name the certain  risk magnitude  in one area that
would make them indifferent to living there or in an area
with uncertain risk. The wider the range of risk for the
second area, the more risk averse people became; a risk
figure received more weight if it came from a "later
study." Risk aversion was attenuated if the lowest risk •
estimate was well below average, and intensified if the
highest estimate was well  above average.'9' In another
study, people's  desire to reduce catastrophic risks in-
creased with uncertainty: risk aversion was greater for
an accident with an equal chance of yielding zero or 100
deaths than for an accident that would certainly yield 50
fatalities.00' Only one case study cited an uncertainty-
trust relation, implying that lack of discussion of uncer-
tainty may be a problem. British sheep farmers distrusted
official statements on post-Chernobyl radiation because
these ignored uncertainty, while their farming experience
exemplified uncertainty.*1" Thus  most previous studies
did not test hypotheses about uncertainty effects on pub-
lic trust or manipulate uncertainty directly.
     It is unclear that explaining uncertainty (at least by
itself, and in the one-shot state of most environmental
communication)  will increase trust and public confi-
dence. First, uncertainty may  disturb people; they want
assurances of their safety and may prefer being told that
a situation is safe or unsafe to receiving formal risk es-
timates/12-'3' Descriptions of uncertainty in risk estimates
may  undercut any illusion of safety.  Second, -technical
risk information, including information on uncertainty,
may  affect public response to risk and government less
than  other factors; stressing uncertainty might confuse
people, or even cause outrage/14' One study found that
government actions to address public concerns and share
information early sharply reduced perceived risk and im-
proved judgments of agency performance for a hypothet-
ical chemical spill. By contrast, details on health effects
and exposure pathways had  no apparent effect/15-16' In
another study,  conaem for a  hypothetical hazardous
waste'site was significantly affected by trust in industry
and government, perceived health threats to oneself and
family, and the  sense that hazardous waste risks were
controllable. However,  knowledge about chemical risks
and a generic warning about the uncertainty of risk es-
timates were not related to concern/"
      In short, presenting  uncertainty  in risk estimates
may  create, rather than remove, public confusion or con-
 flict. Yet uncertainty, inherent in risk assessment, must
be part of accurate communication about risk. Research
is needed to help us determine how best to communicate
uncertainty to the public. The studies reported here rep-
resent initial steps_ toward that goal.
3, RESEARCH DESIGN

     Our  subjects read  simulated newspaper stories
about a hypothetical USEPA risk estimate for a poten-
tially hazardous case. Stimuli were varied to identify the
separate effect each of several design variables made on
risk and other judgments.  Use of a mock newspaper
story reflects the press's role as a  major channel for cit-
izens' receipt of risk information. Use  of an official or
simulated agency fact sheet instead might restrict exper-'
imental variation due to limits on what the agency can
say about risks.
     Table I shows the major manipulations'deployed in
the four studies reported Here; Table II shows excerpts
of three stories. Study 1 used four levels of uncertainty,
described as percentages of the (highest) point estimate;
Studies 2 and 4 used only  two treatments, a point  esti-
mate and a range of estimates. Risk estimate ranges in
Studies 1 and 4 extended from zero to an order of mag-
nitude above the  point estimate of risk; Study  1's range
extended only downward from the point estimate. Study
3 (two focus groups with Eugene, Oregon, residents)
used three Study 2 stories without alteration. In Study
4,-information about the range of probabilities was  sup-
plemented by translating these probabilities to additional
cases of cancer that would be expected in a city the size
of Eugene, Oregon, over 70 years. Study 4 also a'dded, -
due to  comments from the focus £roup participants, a
statement about the typicality of uncertainty in science,
and an explanation of the uncertainty (only animal tox-
icity data were available, and extrapolation from animals
to humans is inevitably  uncertain). This issue of anirrial-
to-human extrapolation is one of the most debated issues
in toxicology for both citizens and experts/17''
     We used fake names  for the hazard stimuli:  "bu-
tydin" for a chemical from  an  abandoned hazardous
waste  site (used in all four studies) -and "zydin" for
radioactive gas  entering homes from  a natural  source
(Study 1 only). These names reduced biases (e.g., dread
of dioxin or apathy about radon) from prior beliefs about
real chemicals or radioactivity/18' Two risk magnitudes
 (point estimates)—one  in a thousand and  one in a mil-
lion—Were used as stimuli in all four  studies. Graphics
were added in Study 2 (see examples in Figure, 1) to see
 whether respondents would take more notice of a visual-

-------
 Uncertainty in Health Risk Assessment
                                                                                                          487
                                    Table I. Research Design: Variables Used in. Four' Studies"
   Study
 No. of slories,
(no. of subjects)
Story designs
Uncertainty variations (examples)*
                  16
                (272)
                (180)
                  3
              .  (13, in
              two focus
               groups)
              '•4
                (217)
                2 Hazards * 2 Risk
                Magnitudes * 4
                Uncertainty Conditions
                2 Risk Magnitudes * 2
                Uncertainty Conditions
                2 Graphic Conditions
                1:1,000 stories from Study 2:
                Certainty without Graphic,
                Certainty with Graphic,
                Uncertainty with Graphic
                2 Risk Magnitudes * 2
                Uncertainty-Plus-Graphic
                Conditions
                     1) No mention of uncertainty, e.g.: "EPA scientists estimate that the additional
                     risk of getting cancer over a lifetime of living in a home that might be contam-
                     inated by zydin at the levels seen in local houses is one in a thousand."
                     2) "EPA announced the highest risk estimate produced by their risk model; the
                     true risk could be as low as 10%  of the  current EPA estimate, or one in ten
                     thousand."                    ,                  ',
                     3) ",. .as low as 0.1% of the current EPA estimate, or one in one million." "
                     4) "... as low as zero."

                     I)'No mention of uncertainty: "EPA scientists stated that the additional risk of
                     getting cancer over a lifetime of drinking water that might be  contaminated by
                     butydin at the levels seen at the Lancaster site is one in a thousand."
                     2) "EPA announced the most likely risk estimate. However; the true risk could
                     be as  low as zero, or as high as. one in a hundred."
                        ;\                     '                  •
                     as in Study 2           , •    .          '                   >
                     J) No mention of uncertainty: "If butydin gets into the water supply, EPA sci- -
                     enlists calculate a person who drinks this butydin-contamiriated water for 70 years
                     would have one additional chance in a thousand of getting cancer.... The one-
                     in-a-thousand  calculation is the  equivalent,  in  Eugene's population of  about-
                     100,000, of 100 extra cases of cancer if .all city residents drank water with this
                     level of .butydin in it for'their entire lives. This compares to  the average Amer-
                     ican's one chance in four of getting cancer from any cause (or an average of
                     25,000 cases of cancer over 70 years in a city like Eugene)."
                     2) "The EPA spokesperson said -that one additional chance of cancer in a thou-
                     sand is the most likely risk level,  but added that the true risk  could be as low as
                     zero, 'or as high as one in a hundred (1,000 extra cases of cancer if all city
                     residents drank water with this level of butydin in it for their entire lives)."
"Hazards: Imaginary names were used-for the chemical ("butydin") and radiation ("zydin"). Risk magnitudes: One in a thousand (1:1,000) and
 one in 'a million'(1:1,000,000). Graphic conditions: Study 2 stories appeared with or without a graphic, for both certainty and,uncertainty conditions;
 all Study 4 stories included a graphic, for both certainty and uncertainty conditions (see Fig. 1 for examples).
'All examples are based upon a point risk estimate of 1:1,000; stories using a point estimate of 1:1,000,000 were altered accordingly-(e.g.,  for
 Study 1, "as low as 0.1% of the current EPA estimate, or one in one billion").
plus-written, than of a written only, presentation of un-
certainty. Each story in ,Study 4 had a graphic.
     Simulated news  stories included a headline, date-
line, and a column format, as in real news stories. All
stories in each study  cited (1) the ,risk estimate source
(USEPA); (2) the weight offfvidence (possible cause of
cancer); (3)  the effect  of estimate  uncertainty (more
study  needed); and (4) a  risk comparison ("For com-
parison, the risk of getting cancer from exposure to all
possible causes of cancer is  about  one in four for an
American"). Study 4'^s  risk comparison (and graphic)
also listed the cases of cancer the estimated risk would
entail  in a city the size of Eugene,- Oregon.
     All four studies  took place during 1993. Subjects
for Studies  1, 2, and .4, mostly college students, were
                                                    recruited through an advertisement in the University of
                                                    Oregon newspaper and were paid a nominal fee.
                                                         Subjects in Studies 1, 2,  and  4 answered  a  ques-
                                                    tionnaire after reading one story. They could refer to the
                                                    story when  answering  the questions.  Some  questions
                                                    measured how  well the manipulation worked (e.g., did
                                                    people reading  stories with ranges of estimates see more
                                                    uncertainty 'than those reading stories  with point esti-
                                                    mates?). Other questions  assessed  dependent or  con-
                                                    founding variables,  such  as  perceived  risk,  agency
                                                    honesty and technical competence, and- general attitudes
                                                    toward risk,  government, and authority.09* The question-
                                                    naire also asked respondents to, indicate their sex and to
                                                    rate how well adjectives from a scale devised by Bem(20)
                                                    to measure masculinity  and femininity  described them-

-------
488
                                                                                                 Johnson and Slovic
        Table II. Excerpts from Simulated News Stories

Study 1.  EPA scientists estimate that the additional risk of getting
cancer over a lifetime of drinking water that might be contaminate'd
by butydin at the leyels seen at the Lancaster site is about one in a
thousand.
    For comparison, the risk of getting  cancer from exposure to all
possible causes of cancer is about one in four for an American	
    EPA announced the highest risk estimate produced by their risk
model;  the true risk could be as low as  10% of the current EPA es-
timate, or one in ten thousand.

Study 2.  EPA scientists estimate that the additional risk of getting
cancer over a lifetime of drinking water that might be contaminated
by butydin at the levels seen at the Lancaster site is one in a thousand.'
    For comparison, the risk of getting  cancer from exposure to all
possible causes of cancer is about one in four for an American.
    EPA announced the most likely risk estimate. However, the true
risk could be as low as zero, or as high as one in a hundred.

Study 4.  If butydin gets into the water supply, EPA scientists esti-
mate a person who drinks this butydin-contaminated water for 70 years
would have one additional chance in a thousand of getting cancer....
    The onc-in-a-thousand estimate is  the equivalent, in Eugene's
population of about 100,000, of 100 extra cases of cancer if all city
residents drank water with this level of  butydin in  it for their entire
lives. This compares to the average American's one  chance in four of
getting cancer from any cause (or an average of 25,000 cases of cancer
over 70 years in a city like Eugene).
    The EPA spokesperson said that one additional chance of cancer
in a thousand is the most likely risk level, but added that the true risk
could be as low as zero, or as high as one in a hundred (1,000 extra
cases of cancer if all city residents drank water with this level of
butydin in it for their entire lives).      	
(a)
   1 additional chance In 1,000
    (EPA's Risk Calculation)
(b) Rangn of Risk Estimates for Lindley
      I— 1 iddRlonil ch»nc
-------
Uncertainty in Health Risk Assessment
                                                                                                           489
       Table III. Effects of Experimental Manipulations
          •Table IV. Findings of Experimental Studies
. - ' ' Study
Findings- " 1
Range, story read-
. er's saw more
range in story No
Percent of sub-
jects who saw , '
risk mentioned
in story . 84% •
Percent of (
subjects who
correctly
recognized 71-83%
range of risk (across the •
estimates in . three range
range-story . stories read)
Percent of
subjects who -
correctly ,
recognized sin- •
gle risk esti-
mate in point- "> . .;
story . 41.5%
.Effect of graphic Not
applicable


.




: 2 - 4


Yes ' • No
-
r .

78%. 91%
, ' • • ' •



j'

80% , " 83%
,





52% . ;46.5%
Raised perceived . Not
range of risk in appli-
story; lowered cable
trustworthiness
of story infor-
mation; did not
raise fraction
seeing risk
number in story
whether a risk number appears in the story, not whether
the agency or the reporter gave an accurate number.").
This last wording change reduced to 9.7% the percent-
age who saw no risk mention in the story. Perhaps there <
was too much information, even in the briefest experi-
mental  stories, for these respondents to notice a risk
number written as a word, as in "one in  a thousand."
Study 1 and 2 questions also did not ask about a "risk
number." However, the same-risk estimate appearing as
a number in an accompanying graphic did not increase
the proportion in Study 2 who said risk  was mentioned
in the story, compared to those who read stories lacking
the graphic.    .    ,
     Recognition  of Ranges. Those who  said a risk was
mentioned in the  story were then asked whether the gov-
ernment gave "a single number  for the risk or ...  a
range within which the -risk might lie."  In,all three.ex-
periments (Studies 1, 2,  and 4) those who read a story
                                                          Experimental
                                                          questions
                                                                                         Study
                  1
                                                          Change in risk
                                                            perceptions
                 No  Range—more risk,
                effect   worry (but not
                       with attitudes
                     .  constant)  .•  •
                                                                          No
                                                                         effect
                     Agree agency is
                       more honest,
                       disagree that it's
                       less competent
Effect of agency
  discussion of
  how much risk
  might vary on
•  views of agency
'  honesty and               .  ,
  competence       ,           -

Effect of attitudes  No  .Attitudes predict
  toward risk,    effect    ratings of risk,
  government,            worry, experts'
  and authority           knowledge of
                       hazardous
 •           -           waste risks
Range—more risk
  (with attitudes
  constant)


      No
     effect
                                                                                                Attitudes predict
                                                                                                  ratings of risk,
                                                                                                 . worry, experts'
                                                                                                  knowledge of
                                                                                                  hazardous
                                                                                                  waste risks,
                                                                                                  EPA honesty on
                                                                                                  size of risks,
                                                                                                  EPA compe-
                                                                                                  tence at risk es-
                                                                                                  timation, EPA
                                                                                                  competence ^
                                                                                                  dealing with en-
                                                                                                  vironmental
                                                                                                  problems
 in which no intended uncertainty appeared were signif-
 icantly more likely to answer "a single number" than
 were readers of stories mentioning uncertainty.  How-
,ever, many readers of point-estimate stories said a range
'of risks was mentioned (58.5% in.Study  1, 48% in Study
 2, 53.5% in Study 4). A lower but still  striking fraction
.of range-story readers (from  17% to 29% for the three
 uncertainty conditions in Study 1; 20% in  Study 2 and
 16.7%  in' Study 4) reported  that only one number .ap-
 peared in the story.   •'."-,
     These results may be due to a combination of con-
. .fusion  and lack of comprehension.'All stories, including
 the  single-number stories, also contained a risk compar-
 ison (see earlier  language).  Readers of single-number
 stories may have inferred from remembering two num-
 bers—for example, the 1:1,000 point estimate of risk and
 the  1:4  general  cancer risk—that these comprised a
 range. The rarer error of classifying range-stories as con-
 taining a single number is more likely due to incompre-
 hension, because both the text and the graphic portrayed
 a range. The consistency of these error rates across three

-------
 490
                                 Johnson and Slovic
studies (1, 2, and 4) suggests that, except for removal
of the risk comparison, further revision of the stories and
questionnaire would not greatly reduce the error rate for
well-educated respondents for whom the issue is not im-
mediately salient.
     These results  do not mean people could not  rec-
ognize the experimental manipulations of uncertainty.
Study 2 (although not Studies 1 or 4) found that readers
of stories with ranges of risk estimates were more likely
than those reading single-estimate stories (86% vs. 42%;
p  <  .001) to say that a "very great*' or "moderate"
range of risk was  reported in the story. The  "range"
groups were  also more likely to rate the story's risk in-
formation as uncertain (5-7 on a seven-point scale), by
54%  to 28% (p <  .02).
     Uncertainty Effects on Perceived Risk.  Study  1
found no  statistically significant differences in perceived
risk across the four uncertainty levels (whether point es-
timate, or a range  of—for example—1:1,000,000 to 1:
1,000). Readers  of Study 2 range-stories were more
likely than point-story readers (34% vs. 14%) to rate the
butydin risk  as  high (5-7  on a seven-point scale; p <
.05). We think this was due to a higher top risk in range-
stories: a  maximum of 1:100 for the point estimate of 1:
1,000, and of 1:100,000 for the other point estimate of
1:1,000,000.  If one focuses only on .the range maxima
it  is reasonable to see the range-story risk estimates as
higher than in the point stories.19' This idea was con-
firmed by the reasoning  of one participant in Study  3:

   I ignored the  fact  that there was zero risk  because they
   wouldn't have reported  it if'there was zero risk ... for some
   reason this graph looked more government-like, and so I im-
   mediately went to worst case scenario	

Another participant said, "if it is zero, they are just re-
ally saying, 'We could be  wrong.' " (Note that Study 1
found similar risk perceptions with a lower-bound esti-
mate  of zero and one that is a small positive number;
Table 1 has examples  of these variations.)
     Uncertainty was also  more worrisome in  Study 2:
73% of the "range" readers, versus 58% of the "point"
readers, were "somewhat"  or "very worried"  (p  <
.01).  About  71%-Of  the  "range" group in  Study 2
agreed  that  the  agency's discussion of uncertainties
would have made them  more concerned about the risk
had they lived in this imaginary town. Somewhat more
than half of all Study 3 members, who read both stories,
were  more concerned about the risks described in  the
range-story than those in the point-story. (However,  be-
cause the  range included zero risk, one person noted that
"people who don't want to worry about this are going
to  find plenty of support for not  worrying about it.")
 Studies  1 and 4 found no differences in worry by un-
 certainty condition.
     In Study 4, whether one received uncertainty infor-'
 mation significantly predicted responses only to a ques-
 tion about the perceived risk to Eugene residents. On a
 scale from 1 (very low risk) to 7 (very  high risk), those
 reading range-stories had a mean rating of 3.14, to  2.58
 for readers of point-stories (p <.01).
     Effect of Uncertainty on Perceptions of the Agency
 That Provided the  Risk Assessment. Participants  who
 read Study  1  and Study 4 stories with varied levels of
 uncertainty  did not  differ  in  their views  of USEPA:
 Within the  Study 2 group that read a  range-story and
 recognized that it contained a range of estimates (TV = -
 56), 66% agreed' that the  agency's discussion of  how
 much  the risk might vary made it seem more  honest;
 29% disagreed. Some 34% agreed that this discussion
 made  the agency  seem less competent; 59% disagreed
 with the statement.  However,  these findings need not,
 contradict the findings of Studies 1  and 4: Study 2 judg-
 ments  of agency honesty  and competence correlated
 with attitudes about  risk, government and authority, im-
 plying that people were not reacting to uncertainty in-
 formation (see discussion of regression analyses, below).
     In Study 3, most participants  felt that providing a
.range was more honest. For example,

   • [The range approach] tends to see the public as competent,
     educated citizens, who are going to have more information,
     who are going to have to make up their own minds, which
     I think is a good first step for the government to do. It hasn't
     done it in the past  most of the time.
   • I  assumed  vast uncertainty even when'it was presented-as
   •  an absolute fact, so ... I guess it is more encouraging to
     see it [in a range].
   • I think it's  a little more honest when there might not be any
     risk and there might be a high risk or a higher risk.... I feel
     more comfortable  with something like this than I do with
     ... a number like a risk 1 in 1,000, or whatever. Just like a
     definitive number where ... I can't even imagine how they
     come up with something like that.

     Yet several people in  Study 3 said they would not
 be upset by agency silence about a range if they did not
 know such a range existed (most  were unaware of un-
 certainty in risk  estimates). For  some, however,  the
 range evoked  doubt  about agency trustworthiness:     '

   • They may give you the idea that there is zero risk, if they
     don't have any way to clean it up. ... If ... they got the
     money, and they are going to clean it up, and they want to
     look good,  and they want to do PR and stuff, they are going
     to tell you there was this huge risk and we're going to take
     care of it. And I just think it's all so politically motivated
     .that it doesn't really mean anything anyway. "            • s

-------
Uncertainty in Health Risk Assessment
                                                  491
    •Despite-the general sense of honesty  evoked  by
ranges of risk estimates, several Study, 3 members felt
that USEPA's report of a range meant that "they don't
have a clue." Among other comments:

   .• It bothers me when  there are a lot of maybes and who'
     knows.    '.-'.''
   * I didn't think much of their ability to be precise....
   • Their preliminary results were too preliminary.

     Even  if reporting uncertainty  seemed to convey
agency honesty, this did,'not seem to offset  concerns
about the agency's competence:

   • I kind of assume that the government doesn't know what
     they are doing most of the time	.At least they are finally'
     admitting that they don't know what is up.
   • [Person  1:] How would you feel if the government... told
     you that 'they have no idea whether this  is going to  pose a
     risk of cancer to you or not. And they really just are having
     a hard time with studies determining it. [Person 2:j Yeah,
     thank you for being honest.
   • [about the story's line that "the true risk could be as low as
     zero":] How come you can't even figure out if there is a  •
     risk or'not? You say it causes cancer. Well, is there a risk
    . or is there not a risk? I don't know, it just bothered me.
   • The honest imbeciles: The EPA.

     In Study  4,  exposure to  uncertainty information
predicted (in  multiple  regression analyses)  lower per-
ceived competence of USEPA at risk assessment and
environmental management, and mare apparent truthful-
ness in news stories, than for those reading point-stories.
Despite their very weak statistical significance (p <£ .10
or• p < .20), these findings' direction fits with the ' 'hon-
est but ignorant" theme of >the focus group discussions.
Nor is  this inconsistent: Honesty and competence are
logically independent-attributes.
     Uncertainty in Science. Study 4 subjects were asked
whether they agreed that "It is typical of good science
that the most likely estimate of what is being measured
has a range :of  uncertainty around it." Item intercorre-
lations were' analyzed for the 56 subjects who  read the
range stories and correctly-reported the story as contain-
ing a range of agency risk numbers. These correlations
showed that those  agreeing with this statement  were
more likely to'find the risk information in the  story un-
derstandable (r =  .36,'.p <'.01), certain (r = .27, p
<.05), and scientifically valid (r = .39, p < .001). They
were less likely to think that the agency's  discussion of
uncertainty  indicated incompetence (r  = — .25,p < .05);
and less likely  to be concerned because of that discus-
sion (r = -.34, p  <.01).
     Some "focus group members (Study-3) thought that
reminding  people  about science's  imprecision would
.make ranges of risk estimates more credible:
   Sometimes a" little disclaimer that reminds people that no matter
   how many tests you do you can never be positively sure will
   remind people that ... they are doing the best they can. And
   I would think that that would help me assess that at least they
   are being honest about the fact that  they'are really not sure
   what risk this poses.

 However,  the presence  of  such a caveat (with other
 "clarifying" information) in Study 4's uncertainty sto-
 ries did  not evoke significantly higher ratings of EPA's
 honesty  than in the same, study's point estimate stories.
      Graphics. For all conditions in Study 2, use of a
 graphic  significantly increased the perceived range  of
 risk—perceived  uncertainty—described  in ' the  stories
 (3.06 for graphic, 2.8  without, on four-point scale from
 "no" to "very great range," p < .04). It also decreased
 the perceived trustworthiness of story information (mean
 of 331  for graphic, 3.84 without, on seven-point scale
 from "not trustworthy" to "trustworthy,"p <..01). Yet
 reaction to graphics in the  Study. 3 focus groups was
 generally  positive:  For example," the  graphic with  the
 certainty story (see Figure la) was said to make the story
 clearer and more salient.            •
     Natural vs.  Technological Hazards. Study 1 stories
 about zydin (natural radiation) elicited significant rank-
 ings of lower risk, less wprry, more understandable and
 honest information, and a more honest agency than were
"elicited by stories about butydin, the imaginary chemical
 from  an abandoned hazardous waste site,  despite iden-
 tical risk estimates for the two hazards. This finding is
 consistent with previous studies  that have found  lower
 perceived risk from natural  hazards than for technolog-
 ical hazards/22'                            .     ,
     Risk  Magnitude.  Significantly different reactions
 occurred to differing risk estimates. In Study 1,  risk es-
 timates  of 1:1,000  drew higher ratings of honesty, for
 both the news story information and  the  agency, than
 did  1:1,000,000 estimates. This parallels an earlier find-
 ing  that people would  believe  a  government  agency
 more if  it  said there was an  environmental  problem than
 if it-said there was no problem/13' Study 2 did not rep-
 licate this  finding about perceived honesty.  However, the
 higher probability (1:1,000)  in the story did evoke more
 perceived risk and more worry, and (while'not quite sig-
 nificant  atp < .05) greater expressed intention of getting
 the  site  cleaned up. Lower probability (i.e.; 1:1,000,000)
 was seen  a's preliminary rather than complete informa-
 •tion. Study 4 subjects who read the 1:1,000 range story
 were  significantly  more likely than thqse reading  the.
 1:1,000,000 range story to say that the agency's discus-
 sion of uncertainty made them more concerned. They
 also rated the risks as better known to the government
 .than did-readers of the lower risk range story/These

-------
492
                                                                                        Johnson and Slovic
results, with earlier research,*13' suggest that citizens see
lower risk numbers as either less accurate or less honest.
However, Study  4  found no  significant differences
across risk magnitude conditions in ratings of agency
honesty,  agency  competence in  risk estimation,  or
agency competence in managing environmental prob-
lems.
     Attitudes. The results reported so far were based on
direct manipulation of the presentation of uncertainty in
Studies  1, 2 and 4, and qualitative analysis of focus
group discussions in Study 3. We also conducted step-
wise regression analyses to examine effects of the un-
certainty manipulations relative to the effects of attitudes
about risk and agency honesty and competence.  These
analyses  used responses from  the 100 (Study 2) and 135
(Study 4) respondents who correctly answered questions
1 and 2,  thus indicating they recognized the experimen-
tal conditions (single risk estimate vs. range of risk es-
timates). After respondents  with missing  data  were
deleted, effective sample sizes were 37-38 for Study 2
and 62-67 for Study 4. These samples were small; each
of nine regression analyses found different independent
variables to  be significant; and these analyses combined
experimental (uncertainty)  with  nonexperimental (atti-
tude) data. For these reasons, we will not  discuss these
results in detail, but we believe they carry important im-
plications worth further study.
     Six  dependent variables  were used: perceived risk,
worry, experts' ability to correctly estimate risk, EPA's
honesty about the size of risks, and EPA's competence
at estimating risk magnitudes and managing  environ-
mental problems  (the last three  for Study 4 only).  In-
dependent   variables   were  (a)   the   respondent's
uncertainty condition (0 = single number; 1 = range);
(b) various  attitudes toward  risk, government, and  au-
thority; (c) sex; (d) gender-related adjectives respondents
used to describe themselves™; and (e) being active in
various environmental activities.
     Within the context of the regression  analyses,  ex-
posure to uncertainty stories significantly predicted only
one  variable: In Study 4 it increased perceived risk to
Eugene residents. Responses  to  other dependent varia-
bles were predictable  from attitudes toward risk, gov-
emmeht, and authority, and gender-related adjectives
(sex itself was not a significant predictor). For example,
those prone to judge EPA as honest in reporting the size
of environmental  risks did not see risks  as serious in
their home  community, trusted the government, disa-
greed that more equality would solve social problems,
and  described themselves  as gentle. Respondents who
judged the EPA competent to deal with environmental
problems were less likely to see local  environmental
problems as serious, more likely to trust the government
to manage risks, and more likely to describe themselves
as gentle leaders without strong personalities who were
not active in environmental groups. The best predictor,
entering first in five of the nine  regressions, was  the
belief that "There are serious environmental health
problems where I live."
     If supported by further research (one study found
similar trust, equality, and risk attitudes linked to low
risk ratings among white men,  compared to others),'21'
these findings would indicate that uncertainty is less im-
portanf than general attitudes about risk and .authority in
shaping views of risk and  agency performance.

4. DISCUSSION

     Results of these initial studies of public response to
uncertainty in risk assessments raise more questions than
they answer. However, some tentative  conclusions  can
be reached:
• People are  unfamiliar  with uncertainty in risk as-
   sessment, and with uncertainty in science gener-
   ally.  Responses by respondents in  Studies 3 and  4
  • point to unfamiliarity with uncertainty in science. Up
   to 20%  of  respondents in, Studies  1, 2, and 4  had
   difficulty even recognizing the presentation of uncer-
   tainty in the form of a range of risk estimates.
• People may recognize  uncertainty (i.e., a range of
   risk estimates) when it is presented simply. Moving
   from Study  1's percentage-based, one-tailed presen-
   tation of uncertainty to Study 2's probability-based,
   two-tailed presentation did produce some responses to
   uncertainty  manipulations. Yet many participants in -
   both  studies cpuld not correctly categorize risk esti-
   mates (as a single number or a range).  Moreover,
   Study 4, intended to enhance Study  2's uncertainty
   effects, failed to show any statistically significant ef-
   fects of uncertainty except in a'-regression analysis for
   perceived risk to people in Eugene, Oregon. Because
   our exploratory regressions implied a greater impact
   of attitudes about risk and government on views of
   risk and agency performance, the univariate analyses
   for Study 2 may overstate the effects of uncertainty.
• Graphics had mixed results in communicating un-
   certainty. Those used in  Study 2 made the range of
   estimates more obvious, but made the story informa-
   tion seem less trustworthy. Because the latter finding
   was  experimental—people  reading  a story  with a
   graphic rated its trustworthiness lower than did people
   reading the same story without a graphic—we cannot
   say whether people consciously saw the graphic as
   being (or as ,a sign of the story being) untrustworthy.

-------
Uncertainty in Health Risk Assessment
                                                                                                         493
   Focus group members found the same graphics useful,
   but their comments indicated that crafting clear, help-
   ful graphics would not be easy (Ibrekk and Morgan^
   argued  that no graphical display of uncertainty will
   be clear to everyone).  Visual displays of the relative
   probabilities of various  risk estimates being  correct
   may have different impacts than our graphics, which   ,
   merely  stressed that,there was 'a range of estimates.
•  People's views on the  environmental cases  pre-
   sented  in the stories may have been influenced less
   by uncertainty manipulations than by attitudes to-
   ward risk,  government, andf authority.  Trust  in
   government and attitudes toward  authority have been
   identified in the research literature as important, if not
   dominant,  factors in perceived risk. With the meth-
 1  odological caveats noted earlier, this view  seems to
   be supported by findings from  exploratory regression
   analyses using data from Studies 2 and 4.  Such atti-
   tudes as personal risk aversion,. trust in government
   risk management, and .support for equality strongly
   affected reactions to the stories, while uncertainty had .
   a significant effect in only one of the nine regressions.
•  Agency discussion of uncertainty in risk estimates
1  , appears to signal agency honesty. Effects of the ex-
   perimental manipulations (not regressions) in  Studies
   2 and 4, and comments in Study 3, support this find-,
   ing.  The focus groups'  reactions combined surprise
•   that government would  offer any unsought informa-
   tion, belief that all information is desirable, and sus-
  , picion  (among a", few,  anyway)  that  precise risk
   estimates cannot be believed. Yet  several  comments
   •about possible cover-ups suggest .some people may
   find declarations about uncertainty a signal of dishon-
   esty. Past experience (direct or through the  mass me-
   dia)  with  agencies actually or seemingly  using risk
   assessment to delay pollution cleanups may fuel this   .
   suspicion. Or reactions to uncertainty may be shaped
   more by prior views of agency honesty.       ..' - .
 • Agency discussion of uncertainty in risk  estimates
   may be a signal  of incompetence for  some  people.
   One-third of Study 2 range-story readers said the
   agency seemed less compfetent when discussing un-
   certainty, this may be due to  unfamiliarity with sci-
   ence: If science  is  deemed certain, uncertain risk
   estimates  must  come from' incompetent  scientists.
   Comments in Study  3  about  uncertainty being ex-
 •  pected (and acceptable)'only for "preliminary" risk
  .estimates also suggest citizens find it hard  to fathom
   . that  competence and uncertainty can co-exist. Again,
   prior trust or distrust  of, agency competence may
   shape reactions to uncertainty rather than the  reverse.
• Estimated risk  levels may affect views  of expert
•,  knowledge.That "low" estimates were deemed more
   "preliminary," whether uncertainty was mentioned or
,   not, implies that risk assessors may have  difficulty
   'communicating their estimates even if citizens deem
   these to be honest. This response may be due to dis-
   trust (no government estimate that low could be true).
   Or it may stem from feeling that a 1:1,000 risk esti-
   mate must be based on high, and thus tangible, inputs
   (e.g., for exposure), while a l:.l,000,000 risk estimate
   is based more on guesswork.
     Our  studies used words and graphics to  present a
range  of  risk estimates, in simulated news  stories, to
communicate uncertainty about two hazards (natural ra-
diation and a chemical in drinking water from a hazard-
ous waste site). Study  1  used an  upper-bound point
estimate,  and both zero  and  positive lower-bounds; the
other studies' ranges went from zero to an order of mag-
nitude above the point estimate. Experimental respondents
read only one story, while Study 3 participants  directly
compared point  and range stories. Although  they pre-
ferred the latter, this may not have been a good measure
of point-  and; range-estimates' respective ability to meet
public needs and skills.  Only one source  of uncertainty
(extrapolating from animal data) was described, in Study
4.   ,   -.   -  . .',       '..  \    .  "'  ".
     Despite the various tests we used, other presenta-
tions may have, stronger,  or different, effects on. per-
ceived risk, hdnesty, and competence. A short list might,
include other descriptions  or sources of .uncertainty;
graphics  conveying the relative  probabilities of a given
risk estimate (e.g., a probability density function plotted
directly, below its cumulative distribution  function, with
the mean clearly marked on each curve'23');4 the source
 of the risk estimate;, the source of  the hazard; who  is
threatened; presentations of variability; interactions of
uncertainty with agency actions affecting trust,  or con-
 flict among  environmental policy actors over risk esti-
 mates and uncertainties. We expect  to address some of
 these issues  in future research.
     Communicating about uncertainty is  necessary, be-
 cause it is a reality of risk assessment and risk manage-
•ment, and-all parties—including citizens—deserve access
 to full information. Our results should not be taken as
 proof that iaypeople do not grasp uncertainty. They may
 not understand it in the way  scientists do; they may also
 not have understood our specific presentations of uncer-

 ••The value of testing the hypothesis that these graphics provide useful
  risk^ communication information would depend in part on the likeli-
:  hood that rislc assessors could routinely provide the information
  needed to generate these graphics.

-------
494
                                      Johnson and Slovic
tainty. These hypotheses do not mean they lack experi-
ence  with  uncertainty,  in   their  own  lives'  or   in
environmental matters. Yet no one should assume that
"educating" citizens  on scientific  uncertainty will  be
simple; given the above-average education of our sam-
ples,   their   members  may  have   given   the  most
sophisticated responses to  our  simulated news  stories
that one could expect.
     Given our initial results, we advise caution in as-
suming that explaining uncertainties will improve public
trust or knowledge. In the  long run,  such explanations
may make citizens' expectations of government, risk as-
sessors, and scientists more realistic.5  Yet overall public
trust and knowledge on risk issues may have to be built
with methods more direct and difficult than uncertainty
explanations.

ACKNOWLEDGMENTS


     Research funding was provided under Cooperative
Agreement No. CR820522 with  the U.S. Environmental
Protection Agency. We are grateful for the assistance of
Dr. Lynn Desautels, USEPA project officer. Draft news
stories for Study  1 were reviewed by  Dr. Adam Finkel,
then of the Center for Risk Management, Resources for
the Future, although he is not responsible for the  stories
actually used. Stephen Johnson and C.K. Mertz provided
valuable assistance with data  collection and analysis.


REFERENCES

 1. American Industrial Health Council, Improving Risk Characteri-
   zation (American Industrial Health Council, Washington, D.C.,
    1992).

* Among options suggested  by anonymous reviewers:,agency reporting
 of uncertainty may force reporters to be more probing and balanced
 even if they do not  report risk estimates; citizen outrage and alien- ,
 atton over reporting  of risk ranges may ultimately yield more respect
 for science's limits and reduce undue expectations of risk assessment
 and government; uncertainty analysis may replace single point esti-
 mates with a variety of point estimates chosen to illustrate (when
 lakcn together) the underlying distribution, so  that only experts need
 deal with ranges, probability distributions, and the like; and a more
 extended discussion of uncertainty with citizens—unlike the "snap-
 shot"  of lay reactions in  these studies—may correct their assump-
 tions that science is  certain. These proposals are thoughtful, perhaps
 even right. Yet they imply that positive effects will come only from
 discussing uncertainty publicly—in general and for specific cases—
 for prolonged periods (years?). Because uncertainty is hardly men-
 tioned now in government announcements about risk, and long-term
 dialogues on scientific uncertainty are infeasible in the way  that
 government and citizens currently interact (e.g., in one-time public
 hearings), it is unclear how these outcomes will occur without so-
 cietal reforms that go far beyond simply discussing uncertainty.
  2. B. B. Johnson and P. Slovic, Explaining Uncertainty in Health
    Risk Assessment: Effects on Risk Perception and Trust (Phase 1
    Final  Progress Report,  Cooperative Agreement No. CR820522,
    U.S. Environmental Protection Agency, 1994).
  3. A. M. Finkel, Confronting Uncertainty in Risk Management: A
    Guide for Risk Management (Resources for the Future, Washington,
    D.C.,  1990); S. 0. Funtowicz and J. R. Ravetz, Uncertainty and
    Quality in  Science for Policy (Kluwer, Dordrecht, 1990); M. G.
    Morgan and M. Henrion, Uncertainty: A Guide to Dealing with
    Uncertainty in Quantitative Risk and Policy Analysis (Cambridge
    University  Press, New York, 1990).         .   .        "  .
  4. B. J. Hance, C. Chess, and P. M. Sandman, Improving Dialogue
    with Communities: A Risk Communication Manual for Govern-
    ment (New Jersey Department of Environmental Protection, Tren-
    ton, 1988), pp. 69-73, 83.
  5. F. H. Habicht, Guidance on Risk Characterization for Risk Man-
    agers  and Risk Assessors (Office of the Administrator, U.S. En-
    vironmental Protection Agency, Washington, D.C., 1992).
  6. Risk and the Environment (Carnegie Commission on Science,
    Technology, and Government, New York,  1993).'
  7. R. J. Bord and R. E. O'Connor, "Determinants of Risk Percep-
    tions of a Hazardous Waste Site," Risk Anal., 12,411-416 (1992).
  8. R. J. Bord, R. E. O'Connor, and D. J. Epp, Communicating Cu-
    mulative Long Term Risks (Report to U.S.  Environmental Protec-
    tion Agency CR816305, Pennsylvania State University, University
    Park, 1992).
  9. W. K. Viscusi, W. A. Magat, and J. Huber, "Communication of
    Ambiguous Risk Information," Theory Dec.,,31, 159-173 (1991).
10. P. Slovic, S. Lichtenstein, and B. Fischhoff,  "Modeling the So-
  ,  cietal Impact of Fatal Accidents," Mgmt Sci., 30,464r474 (1984).
11. B. Wynne, "Sheepfarming After  Chernobyl: A  Case Study in
    Communicating Scientific Information," Environment, 31,  10-15,
    33^tO (1989),
12. P. Slovic, B. Fischhoff,  and S. Lichtenstein, "Response  Mode,
    Framing, and  Information-Processing Effects .in  Risk  Assess-
    ment," in Hogarth, R. (ed.), New Directions for Methodology of
    Social and Behavioral Science: Question Framing and Response.
    Consistency (Jossey-Bass, San Francisco, 1982), pp. 21-36.
13. N. D.  Weinstein, Public Perception of Environmental Hazards:
    Statewide Poll of Environmental Perceptions (Final Report to the
    New Jersey Department of Environmental Protection, Rutgers •
    University, New Brunswick, NJ, 1987).
14. P. Slovic, "Perceived Risk, Trust,  and Democracy:  A Systems
    Perspective," Risk Anal., 13, 675-682 (1993).
15. B. B. Johnson, P. M.  Sandman, and P. Miller, "Testing the Role
    of Technical Information in Public Risk Perception," RISK: Issues
    Health Safety, 3, 341-364 (1992).
16. P. M. Sandman, P.  Miller, B. B.  Johnson,  and'N. Weinstein,
    "Agency Communication, Community Outrage, and Perception of
    Risk," Risk Anal., 13, 589-602(1993). '    .'
17. N. Kraus, T. Malmfors, and P.'Slovic, "Intuitive Toxicology:
    Expert and Lay Judgments of Chemical Risks,',' Risk Anal., 12,
    215-232 (1992).
18. G. L. Carlo, N. L. Lee,  K. G. Sund, and S. O. Pettygrove, "The
    Interplay of Science,  Values, and Experiences Among Scientists
    Asked to Evaluate the Hazards of Dioxin, Radon, and Environ-
    mental Tobacco Smoke," Risk Anal., 12, 37-43 (1992).
19. A. Wildavsky and K. Dake, "Theories of Risk Perception: Who
    Fears What and Why?," Daedalus, 41-60  (1990).
20. S. L. Bern,  "Sex.Role Adaptability:  One Consequence of Psycho-
    logical Androgyny," J. Person. Soc. Psycho!., 31, 634-643 (1975).
21. J. Flynn, P. Slovic, and C. K. Mertz, "Gender, Race, and Perception
    of Environmental Health Risks," Risk Anal., 14, 1101-1108 (1994).
22. A. Baum, R. Fleming, and L. M. Davidson, "Natural Hazards and
    Technological Catastrophe," Environment and Behavior, 15, 333—
    354(1983).
23. H. Ibrekk and M. G. Morgan, "Graphical Communication of Un-
    certain Quantities to Non-Technical People," Risk Anal., 7, 519-
    529 (1987).

-------