EPA/SAB/80/003
Approaches to Health Risk Assessment for
Alternative National Ambient Air Quality Standards
A Seport of the Subcommittee on
Health Risk Assessment
December 1980
Science Advisory Board
U.S. Environmental Protection Agency
Washington, D.C, 20460
-------
EPA NOTICE
This report has been written as part oŁ the activities of
the Science Advisory Board, m public advisory group providing
extramural scientific information to the Administrator and
other officials of the Environmental Protection Agency. The
Board is structured to provide a balanced expert assessment
of the scientific matters related to problems facing the
Agency, This report has not been reviewed for approval
by the Agency, hence its contents do not necessarily
•represent the views and policies of the Environmental
Protection Agency, nor does mention of trade names or
commercial products constitute endorsement or recommendation
for use*
-------
September 1980
SCIENCE ADVISORY BOARD
SUBCOMMITTEE ON HEALTH RISK ASSESSMENT
Chairman:
Dr* Anita S. Curran, Commissioner, Department of Health, Westchester
County* County Office Building 2, 150 Grand Street, White
Plains/ New York 10601
Members;
Dr. Bernard Altshuler, Institute of Environmental Medicine, New
York University Medical Center, S50 First Avenue, New York,
New York 10016
Dr. Judy A. Bean, College of Medicine, Department of Preventive
Medicine and Environmental Health, University of Iowa, Iowa
City, Iowa 52242
Dr. Granger Morgan, Department of Engineering and Public Policy,
Carnegie-Mellon university, 5000 Forbes Avenue, Pittsburgh,
Pennsylvania 15213
Dr. D. Warner North, Decision Focus Inc., 5 Palo Alto Square,
Suite 410, 2alo Alto, Califoraia 94304
Dr. Geoffrey Watson, Professor and Director of Graduate Studies,
Department of Statistics, Fine Sail, Princeton University,
Princeton, New Jersey 08544
SAB Staff Officer;
Mr* Ernst Linde, Scientist Administrator, science Advisory Board
A-101, U.S. Environmental Protection Agency, Washington,
O.C, 20460 Phones {202} 472-9444
-------
Approaches to Health Risk Assessment for
Alternative National Ambient Air Quality Standards
BACKGROUND
In 1979, the Subcommittee on Health Risk Assessment of the
Science Advisory Board reviewed a methodology proposed by EPA's
Office of Air Quality Planning and Standards (OAQPS) for
assessing health risks associated with alternative ambient air
quality standards for ozone.
The Subcommittee stated its comments, conclusions, and
recommendations in a report dated September 1979«^/
Briefly, the Subcommittee, among other comments and
suggestions,
- commended the Office of Air Quality Planning and
Standards for recognizing the need for and undertaking
the development of quantitative subjective decision aids
for use in the assessment of the health risks of air
pollutants;
- found that the methodology developed by OAQPS **/
was not ready for application in the process of
establishing national ambient air quality
standards;
- urged EPA to expand its efforts to develop and
evaluate the use of judgmental probabilities in
the process of establishing national ambient air
quality standards;
- warned that under no circumstances should the
use of such techniques become a replacement for
the research needed to establish an objective
data base for assessing health risks; and
Review of "A Method of Assessing the Health Risks Associated
With Alternative Air Quality Standards for Ozone" (Draft dated
July 1978, A Report of the Subcommittee on Health Risk
Assessment, EPA/SAB/79/001, September 1979)
Subsequently referred to as the P/B Risk Assessment Method
after Thomas B, Feagans, OAQPS Analyst, and Dr, William P.
Biller, Consultant.
-------
-2-
- recommended that OAQPS should formulate a plan
outlining how the Agency will (a) develop the
proposed methodology, including standards and
protocols for application, (b) consider
alternative approaches, and (c) select and
establish the credibility of the best
methodology, ( .
Consistent with these recommendations, the Office of Air
Quality Planning and Standards developed a "Risk Assessment
Program Plan" and, with a view toward considering alternative
approaches to health risk assessment, contracted with six
research teams, each one to develop an integrated conceptual
risk assessment procedure. This particular effort resulted in
six documents which the Subcommittee reviewed in a public
meeting on September 15 and 16, 1980.
SCOPE
The six documents which the Subcommittee reviewed and which are
the subject of this report, are
- "Reporting of Uncertainties in Risk Analysis" by
Howard Raiffa and Richard Zeckhauser, First
Draft, September 1980;
- Assessing Health Risks Associated ffith Ambient Air
Quality Standards" by Barueh Fischhoff and Chris
Whipple, July 1980;
- "A Procedure Based on Decision Analysis for
Assessing the Health Risk Ass«ciatud With
Alternative Ambient Air Quality Standards'* by
M.W. Merkhofer, Preliminary Draft, July 1980;
- "Estimation of Risk of Adverse Health Effects
Associated With Air Quality Standards for
Pollutants^ by If.O. Hartley, FC.fi. Manton, and
M.A. Woodbury, Undated;
- "A Risk Assessment Methodology for Environmental
Pollutants" by Robert L, Winkler and Rakesh K.
Sarin, Draft of Final Report, July 7, 1980; and
- "A Conceptual Risk Assessment Procedure" by
Richard de Neufville and Marie-Elisabeth Pate,
First. Revision, August 25, 1980.
Also at the September 15 and 18, 1380 meeting, the
Subcommittee heard update reports on the Risk Assessment Method
developed by Thomas B. Foagans and William F. Biller (the P/B
method) and on contemplated next steps in the OAQPS "Risk
Assessment Program Plan."
-------
-3-
OAQPS requested that the Subcommittee, in its report, discuss
each one of the suggested approaches, focus on its strengths
and weaknesses, and comment on Agency plans for the further
pursuit of alternative approaches.
I '
COMMENTS AND FINDINGS
"Reporting of Uncertainties in Risk Analysis" by Howard
Raiffa and Richard Zeekhauser, First Draft, September 1980
Dr. Ratffa's paper is not intended as a proposal for a
specific analytical technique but rather as a general
philosophical framework. It contains important insights on
probabilistic methods of assessing environmental risks and.
provides a useful discussion of both advantages and dangers of
using quantitative techniques for risk assessment in the process
of regulatory decisionmaklng.
The paper contains much valuable material particularly
concerning the process of assessing uncertainties when working
with committees of scientific experts. The paper emphasizes that
communication between scientists holding disparate views and
policymakers is, at best, a difficult process. It does not
suggest specific mathematical techniques for aggregating
scientific judgment, but rather emphasises the need for
experience and- ingenuity on the part of "synthesizers" who can
work with diverse experts to facilitate open, honest, and
effective communication. This emphasis on communication by one of
the foremost authorities on decision analysis should be carefully
considered by OAQPS. It addresses implicitly a major weakness of
the Feagans-Biller methodology: its focus on mathematical
technique at the expense of facilitating communication.
Raiffa's argument for occasional nonseparation of risk
assessment and risK evaluation also bears careful consideration.
The paper suffers somewhat from its lack of organization. It
is not specific to the isssues of criteria air pollutants facing
OAQPS,
Dr. Raiffa has indicated that he does not wish to be
considered for subsequent OAQPS contracts relating to this
particular effort.
-------
"Assessing Health Risks Associated With Ambient Air Quality
Standard's" by Barueh Fischhoff and Chris fhipple, July 1980
The paper by Fischhoff and Whipple provides a general
philosophical framework for risk assessment and a broad set of
desiderata that are useful background for developing appropriate
methodology. The paper presents interesting insights on the
psychology of probability assessment and on the difficulties of
establishing air quality standards under the dean Air Act* It
presents virtually no analytical specifics nor an approach suited
to the needs of OAQPS.
The paper's principal contributions are an emphasis upon the
differing requirements and objectives of the various actors in
the standard setting process and an emphasis on the psychological
limitations and cognitive biases which are likely to enter the
process of attempting to estimate uncertain coefficients in the
form of subjective probability density function. Unfortunately,
no real recommendations are provided on specifically how to
incorporate these important insights into one or several
quantitative tools.
OAQPS would be well advised to give these insights careful
consideration as specific analytical techniques are developed for
their use.
"A Procedure Based on Decision Analysis for Assessing
Health Risk Associated With Alternative Ambient Air Quality
Standards" by M.W. Merkhofer, Prelininary Draft, July 1980
In the view of the Subcommittee, the paper by Merkhofer
clearly comes closest to providing a workable framework for risk
assessment to be used In the process of setting ambient air
quality standards. The technical approach is well developed,
with appropriate logic and mathematics used to incorporate
relevant informati6n. Assumptions are clear and explicit,
making the methodology readily comprehensible for public review.
Available data on ambient levels and dose-response relationships
are utilised, and uncertainties are included explicitly as their
importance is assessed* The paper shows sensitivity to the
specific restrictions imposed on the problem of risk assessment
by the Clean Air Act and both willingness and ability to modify
traditional decision analytic techniques to meet the needs of
EPA.
There appears to be too much emphasis on predictive air
quality models and not enough emphasis on the use of historical
air quality data.
One member1 questioned the desirability of significance
weighting, i.e., introducing value judgments which are
implicit in significance weighting.
-------
-5-
One member found the emphasis on sensitivity studies to
explore the importance of alternative sources of information and
the Implication of varying expert opinion to be particularly
appealing. The member felt that while somewhat complicated, the
proposed adaptation of the ideas of value of information to this
problem is useful and worthy of further exploration.
The suggested approach warrants further support for
refinement of techniques and trial applications. Close
collaboration with experts in air pollution other than sulfur
pollutants and with experts in health, atmospheric chemistry and
meteorology will be necessary and should be encouraged.
"Estimation of Risk of Adverse Health Effects Associated
With Air Quality Standards for Pollutants" by i.O. Hartley,
K.G. Manton, and M.A. Woodbury, Undated
The paper by Hartley, Manton, and Woodbury starts with the
commendable position that risk assessment, to the extent possible,
should be based directly upon health iaspact data and that
subjective judgments should enter only to the extent necessary
to interpret and compare these data* While this major emphasis
on data base is absolutely correct, the proposed implementation
is not satisfactory.
It is clear from the procedures proposed that the authors
are thinking in terms of carcinogens and statistical studies in
the context of rather large volumes of data. They propose the
use of a model that emphasizes the "time of onset" of adverse
chronic health effects. No evidence is presented that this
model is useful or appropriate when little is known about the
relation of chronic health impairment to past ambient levels or
when acute or transient effects are the health impairment of
concern.
The problems of obtaining epidemiological data for the
proposed model are formidable. In the absence of
epidemiological data, clinical and animal studies assume great
importance. There is no indication how the proposed approach
would make use of such data.
The Subcommittee feels that the types of techniques
proposed are inappropriate for the assessment of criteria air
pollutants mainly for two reasons: first, because the health
effects evidence typically available for criteria air pollutants
is not amenable to the kinds of statistical manipulations
proposed; and, second, because typically the processes by which
health effects from criteria air pollutants occur would not be
well characterized by the kinds of effects model proposed, j..,e. ,
a model based on "time of onset" of a given condition or effect.
-------
-6-
"A Risk Assesment Methodology for Environmental Pollutants"
by Robert L. Winkler and Rakesh K, Sarin, Draft of Final
Report, July 7» 1980
The Winkler and Sarin approach, while not as completely
developed as Merkhofer's, impressed the Subcommittee as a clear
exposition of decision analysis, a promising start tailored to
the specific problem of assessing the health risks of a criteria
pollutant. The approach appears well suited to making
assumptions and judgments explicit, thus facilitating review by
the public,
The technical approach appears generally adequate, although
much less detail is presented on how air chemistry and
meteorological information will be considered. The description
of health effects is overly simplistic. The complexity of dose-
response modeling is not fully recognized.
Some caution should be exercised regarding the use of
second order probabilities and the regression approach to
pollutant synergism, but the ideas expressed merit some further
investigation. No procedures should be developed which would
require experts to provide rather large numbers of subjective
probability density functions (as might happen in the treatment
of second order probabilities) or to answer questions which pose
severe cognitive difficulties (as might happen with the
multivariate regression procedure).
The discussion, in the paper, of aggregation of experts'
judgments highlights the need to examine the basis for differing
probability assessments. The authors might do well to explore
procedures for displaying the sensitivity of risk assessment
results to alternative expert opinions without necessarily
requiring a procedure for combining those opinions.
To be implemented effectively, the tinkler and Sarin
approach will require extensive familiarity with air quality
models and knowledge of the details of health impacts of
criteria pollutants. To be successful, air pollution and health
effects experts will have to be closely involved in the further
development of this approach.
-------
-7-
HA Conceptual Risk Assessment Procedure" by Richard de
Neufvill'e and Marie-Elisabeth Pate, First Revision,
August 25, 1980
The paper by de Neufville and Pate provides useful general
comments on the use of risk analysis. The general philosophical
position, as outlined in the main body of the paper, places heavy
emphasis upon simple models and on the use of iterative
procedures to reach agreenent on appropriate output measures
among the various parties and is excellent. OAQPS would be well
advised to keep this in mind as specific analytical techniques are
developed for their use.
Unfortunately, the paper is not sucessful in developing a
specific analytical framework. An exploratory research proposal
titled, "Development of a Bisk Indicator for Health Effects of Air
Pollution," which is included in the paper as Appendix A, while
demonstrating analytical ability, suffers from a lack of familiarity
with air quality literature and, in particular, the types and
quality of data available,
The Subcommittee is aware of the preliminary nature of the
paper under review.
"F/B Risk Assessment Method" by Thomas B. Feagans and
William F. Biller
Mr. Feagans deserves much credit for his role in initiating
OAQPS efforts toward formal analytic risk assessment procedures.
It is with great regret, therefore, that the Subcommittee finds
that the presentation at the September meeting was disappointing
and did not reflect significant progress in the development of
this method since the Subcommittee reviewed it and discussed its
deficiencies in public session in April 1979. The Subcommittee
feels strongly that the lack of progress in improving this particular
approach, thought of as an "in-house" methodology, may tend to
discredit the entire Risk Assessment Program and obscure the
significant progress made in developing suitable methodology
elsewhere in the program.
Next Steps in Risk Assessment Program Development
OAQPS indicated that the goal is, in due course, to select
one or more methods for assessing health risks associated with
national ambient air quality standards and to use the method or
methods selected in the next scheduled review of national
ambient air quality standards in 1985* As an immediate next
-------
-8-
step toward that goal, OAQPS plans to select two or perhaps
three of the methods presented for further development and
sample application. The final step would be a test case where
the method or methods selected would be implemented and
evaluated by an independent third party.
The Subcommittee is in general agreement with this approach
but warns against prematurely turning over a method or methods to
a third party for trial testing. The Subcommittee feels that
the party which developed a method which has been selected for
further development and trial application should' be given ample
opportunity to refine and test try that method before it is
turned over to a third party,
SUMMARY AND CONCLUSIONS
The Subcommittee commends the Office of Air Quality Planning
and Standards on the conduct of its Risk Assessment Program. la
the view of the Subcommittee, the program has produced clearly
two and possibly three promising approaches to health risk
assessment for use in developing alternative national ambient
air quality standards. Regrettably, information presented on
the "F/B Risk Assessment Method," in the view of the
Subcommittee, did not reflect significant improvements since it
was first reviewed by the Subcommittee. f_/
It should be noted that the Subcommittee has been advised
that, following the Subcommittee's recommendations, OAQPS
has selected two approaches as alternatives for further
development: the approach presented by M.W. Merfchofer and
the one by Robert L. Winkler and Rakesh K. Sarin* With
respect to the Feagans/Biller (F/B) method, the
Subcommittee has been advised that OAQPS plans to complete
a full report on this method within the next few weeks and
then to conduct^extensive in-house and external reviews of
the comprehensive report. The Subcommittee will be asked
to participate in that review. If the peer review
indicates that further development of the F/B method is
warranted, OAQPS plans to hold it in abeyance until
alternative methodology can be brought into a parallel
state of development. If the peer review indicates that
further OAQPS support of the F/B approach is not warranted,
all work on that method will be terminated except for the
generally-applicable exposure analysis module. Resources
designated for development of the F/B method would be
applied to the alternative approach(es).
------- |