EPA/S&B/81/QQ2
Review of
"A General Method for Assessing Health Risks Associated
With Primary National Ambient Air Quality Standards"
(Draft dated April 1981}
A Report of the Subcommittee on Health Risk Assessment
September -1981
Science Advisory Board
U.S. Environmental Protection Agency
Washington, D.C. 20460
-------
^
EPA NOTICE
This report has been written as" part of the activities of
the Science Advisory Board, a public advisory group providing
extramural scientific • information to the Administrator and
other officials of the Environmental Protection Agency. The
Board is structured to provide a balanced expert assessment
of the scientific matters related to problems facing the
Agency. This report has not been reviewed for approval
by the Agency, hence its contents do not necessarily
represent the views'an4 "policies of the Environmental
Protection Agency» nor does mention of trade names or
commercial products constitute, endorsement or recommendation
for use, '•••••'•'•
-------
fable of Contents
'•'• :; ••'..-. . Page
Background V* *«...,,,*,«*.........„..*..,,**«** I
Comments *..........,,,,.,. ,.. ....»,...„ 4
Advisory Statement ,»»'„»»,.„ *.*..,..„..,....,..., 9
Conclusions .............., *...;...<,*.* + ...*.,,.... 9
Appendices • . ';'•
Appendix A,
Subcowmittet Roster .*.*.....*.....*.*«.. A-l
Appendix B,
Cornnte^ts by Geoffrey S. Watson on Appendix Gr
Mathematical freatment of Dncertainty in Pol-
lutant Concentrations of MA General Method for
Assessing Health 'Risks Associated With Primary
National Ambient Air Quality Standards" (Draft
dated April 1981) *......*.*........ B~l
-------
Background ; •• •;-.;i-:.-^':
This is the th*rd;: report of the Subcommittee on Health
Risk Assessment of' he ; Science Advisory Boarderitictuing.
approaches to health risk' assessment for alternative national
ambient air quality standards developed under the auspices of
FPA's Office of Air Quality ^lannina and Standards CO&QPSK
In 1979, the Subdomait-fcee reviewed a methodology for
assessing health risks associated with alternative ambient
air auality standards forVosone* The Subcommittee stated
its continentsr conclusions, and recommendations in a report
dated September 1§79» ^/
Among other comments and suggestions, the Subcommittee
- found that the methodology developed by QAQPf? was
not ready>for application in the process. of estab-
lishing national arijjbi'ient air quality standards; and
- recommended that; Q&QPS should formulate, a plan
outlining how the ';,ftgency will (a) develop the
proposed methodology?: including standards and
protocols for application, (fa) consider alter-
native apprr .iches, and.Cc) select and establish
the credibility of life.e best methodology,
. $''
Consistent witr the Subcommittee's recommendations, the
Office of Air Quality Planning and Standards developed a "Risk
Assessment Program Plan1* and contracted with si* research teams
to develop integrated conceptual risk assessment procedures.
This effort resulted in aix 'documents which the Subcommittee
reviewed at a public meeting in September 1980. Also at that
weetino, the Subcommittee; lizard update reports on the F/B Risk
Assessment Method (named'.after its authors, Thomas B* Peagans
and William P. Billert and on contemplated next steps in the
OAOPS "Risk Assessment Program Plan** The Subcommittee's
comments, findings, and conclusions are contained in a report
dated Deeeisiber 1980,**/
Review of "A Method of Assessing the Health Risks Associated
with Alternative Air Quality Standards for Ozone*1 (Draft
dated July 1978)„ A Report of the Subcommittee on Health
Risk Assessment,, EPA/SAB/79/001, September 1979. iSie method
is referred to as the F/B Risk Assessment Wethod or the
P/B Hethod after its authorsf Itiomas B. Peagans, OAQPS
Analyst, and Dr» William F« Biller, Consultant.
Approaches to Health Risk Assessment for Alternative
National Ambient Air CMality Standards, A Bepott of the
Subcommittee on Health Risk Assessment, JSPA/SAB/8Q/003r
December 1980.
-------
Briefly, the Subcommittee concluded that
' ' : " ''•/. ' \
- the program has produced two and possibly three
promising approaches,to health risk assessment for
use in developing" alternative national ambient air
quality standards; and
- information presented'on the F/B Risk Assessment
Method did not refljjsct^'significant improvements since
the Subcommittee filttte;". re viewed it and discmssed its
deficiencies in public*session in April 1979*
Following the Subcommittee's recommendations, GAQPS
selected two approaches as•alternatives for further
development; the approach presented by M.W, Merkhofer and
the one by Bobert L+ Winkler and Rakesh K. Sarin* As regards the
F/B Method, QAQPS advised the Subcommittee that the method will
be presented in a comprehensive report and will be subject to
extensive in-house and external reviews including another review
by this Subcommittee, OAQPS then planned to proceed as follows.
If the reviews indicate that further development of the F/B
Method is warranted? OAQPS, will hold it in abeyance until
alternative methodologies can be brought into a parallel state of
development. If the reviews indicate that further OAQPS support
of the F/B approach is not.warranted, all work, on that method
will be terminated except "for the generally-applicable exposure
analysis module. Resources currently designated for further
development of the F/B Method would be applied to the alternative
approach{e s}. V1
A comprehensive report, describing the F/B approach was issued
as a draft ancf became available for 'Subcommittee review in
mid-April* **/
*7 Letter from' Joseph Padgettr Director, Strategies and Mr
~ standards Division, OAQPS, to Dr. Anita S. Curran,
Subcommittee Chairman, dated December 17, 1980.
**/ "A General Method for Assessing Health Risks Associated with
Primary National Ambient Mr Quality Standards" (Draft dated
April 1981) by fhoroas B. Ftagans and William F, Biller
.- 2 -
-------
OH.QPS also arranged for six experts to provide in-depth
technical reviews of the P/i report which became available
shortly before or at the'Meeting,
~1he six experts selected:.'by OAQPS to provide technical
reviews were* ; • ;v,';>
Dr. DaviS S. Bell, Har1iit|f«5 -University Business' School
Or, C*S« Burton, .Syst$»4" ^applications, Smn Rafael,
California ' - .'^\i
Dr. Richard C. Jeffrey^'lDepartment of Linguistics and
Philosophy, Maasachiii^tts Institute of f^chnolofy? after
June 15, 1931s Department of Philosophy, Princeton
University . • .;!. -
Dr. Isaac Levi, Departaiertt of Philosophy, Columbia
University • :'.'.' :
Dr. John H. Seinftld, .Department of Chemical Engineering,
California Institute'of Technology
Or. Andrew J. Van Horn, Energy an<3 Environmental Analysis
Division, Teknetron itesearch, Qrinda, California
— 3 v
-------
Comments _ ..,•.-.•••
We again wish to conmend and compliment the Office of Air
Quality Planning and Standards (OAQFS) on their « Cforts in
developing health risk assessment and evaluation 'methodologies
and recommend that they continue these efforts.
The contracts comX$$$.oned over this past year and a half
have produced 'valuable -t^iarch on the application of decision
analysis principles and'-t^cfeiiques in the development "of-health
risk assessment and evaluation methodologies to aid in setting
ambient air quality standards. But we can see little or no impact
from this research or fraa other sources in remedying the
deficiencies this Subcommittee has perceived with the F/B
approach. It remains unnecessarily complex, continues to be
poorly communicated despite-repeated attempts at clarification,
contains concepts that have not been fully worked out* and in
some areas contains unclear mathematics. {See Appendix Br Dr.
Watson's critique of Appendix G of the F/B Report,}
Our perception is tits'fc . EPA * s goal is to have an impiemen table
methodology within the ine'act several years for assessing health
risks to aid the Administrator in setting ambient air quality
standards. While the F/B approach may have commendable aspects
as a research effort, it is not, in its present ^orm, an
implementable tool for'pifoiic policy decision ma. ing, an<3» we
believe, will not be accepted by a large part of., the scientific
community or the public* .- J^s we have repeatedly 'uggested,
Peagans and Biller should publish their work in peer-reviewed
journals* and the professional community shoulcl 3udge the
merits of their viewpoint* It is our belief -that even if the
F/B viewpoint were to become widely accepted, this would not
occur in the foreseeable future,
The scientific data and reasoning should be emphasizedf not
the mathematical complexities of processing probabilities. There
is a lack of emphasis in 'the F/B approach on what constitutes an
adverse health effect, what constitutes the most sensitive
group, and which health effects should be most important in
establishing ambient standards. The discussion of responses of
concern, sensitive populations, seriousness of effect*
uncertainty about causality (pages 7-2 to 7-4 of the F/B Report)
is unsatisfactory and superficial. Agreement on basic
definitions and measurement-criteria is essential if comparisons
between experts1 probabilities are to have any meaning.
— 4 —
-------
Another area of difficulty is that of working through the
mathematics of a complex'.:;model using upper and lower
probabilities, in this situation, a probability density function
does not exist, and integration over the range of an uncertain
quantity becomes much more:'difficult. In complex parts of their
analysis, such as the dose—response discussion in Appendix R,
Feaqans and Biller go back to usinq singIs-valued probabilities
and density functions, Peagans and Biller state (p. E-2)»
"With care, it is relatively straightforward to recast the
results and the supporting, treatment in the form of upper and
lower probabilities." ;We are not convinced that the extensive
development of pages E~'2 to S-29 and Appendix F can be carried -
through with upper and lower probabilities without encountering
some intractable or severely restrictive mathematical difficulties.
Only the most diligent and determined probability expert could
understand the mathematical development of Appendices fi and F,
given that it is carried;out using upper and lower probabilities.
A third area of di.£;|4calty was pointed out by Professor Levi.
It is not clear, how the filler and lower probabilities are to be
used to arrive at decisicmp, *?fte risk ribbon approach advocated
by Feaqans and Biller isT«3;,hoe and unsatisfactory. Professor Levi
notes that approaches to^tf ision theory using tapper and lower
probabilities have been '«l$e;v;eloped by himself and others and published
in the literature, :but..th*t«* approaches have not been incorporated
into the P/B approach,
i
The Subcoiroittte sucrqwvts that all material relating to upper
and lower probability,'the "theoretical foundations of probability?
and risk ribbons should be:considered-basic research and
separated from the approaeto. For an implementable tool,>we
suggest the analysis shooltf. be based on standard decision analysis
or Bayesian methods, using single-valued probability
distributions* Bach such distribution would represent the
judgment of one expert or a consensus among a group of experts-for
each uncertain input to the' risk analysis/ such as the percentage
of a well-defined sensitive group suffering a health effect from a
given pollutant concentration- "Hiese distributions would be
combined with meteorology and human activity models to produce
probability distributions on threshold exceedances anc! the number
of adverse health effects for alternative standards. Where
experts disagree on a probability distribution that is used as
input, EP& should carry out a sensitivity analysis to determine
the impact of the disagreement. Where the disagreement is
significant to the decision on choice of standard, each of the
disagreeing experts should be asked to communicate the rationale
for his or her probability assessments in such a form that this
rationale can be reviewed by scientific peers and presented to
EPA's senior management •.
-------
Any analysis which'-ls^to be successfully osed and defended
by EPA in the standardHSH&'tlting process must be as simple as-is
consistent with a sattsfftci^pry solution. Its developers and users
must b€ able to describe;;*t!i
-------
Rereading the comments of the six reviewers, as Regards the
merits of the P/i approach, it seems to us that they'were mainly
addressing the questions Is the F/B approach sufficiently
interesting research thatpli;deserves continued support? This,
in our opinion, is very different front the question of whether
the P/B approach could or should be implemented for an ambient
air quality standard in th'^;:,»ext several years. 'We read most of
the reviews to state or iiJply that the 'approach and the
communication of it had.'Significant problems yet to be resolved*
We find Bell's comments o?i'; Efficiency and the caveat in Burton's
review particularly important in relation to the feasibility of
successfully implementing the P/B approach.
Bell statess
"It is well known that there are decreasing
returns to scale with the complexity of a model to
the point where you can -end up worse off than no
model at all. I thiiiJt: the models here are overly'
complex for the current state of applied art. I
agree with the approach' but I believe it is a
little too much all at ones. I would be happier
seeing more modest-goals at this point. If the
report is only intended'-to be a look at the future
or as a research document as opposed to a draft of
an EPA manual then I'fn content. I don't believe
it's realistic to expert? a methodology such as this
to be performed witn'':iitoch creeden.ce Esic] given to
it in the next 10 yewta." (p.3)
Burton adds* :.:;;
"The concepts and iiethodology embodied in the
approach appear; sound/^4 merit (immediate) further
development towards ;tpi$,%oal of incorporating the
techniques into, the-''si&&dar3 setting process*
However, while I support'completely the approach, •'
it is apparent that, .neither the style nor the tone
of the report will facilitate acceptance of the
approach by the technical, regulating,•or regulated-
community. To taker as,the authors do, such an
unyielding position regarding the approach of
eliciting probability-'.estimates, and to use lan-
guage that to many, ii.-not most, will be perceived
as offensive, appears to be both unnecessary and
counterproductive. This statement should not be
construed to mean that the authors should not be
bold and clear in stating their views? it {the
statement) is meant to suggest, however, that there
. 7 -
-------
exist less polarislitq iseans of conmainicating their
views, 'fie authors,*, should be satisfied, that both
their peers and time 'will be the final 1udge. In
short why make the .Introduction and use of this new
methodology into thi (f«cision-€isking process more
cliff ieult .than
The comments of th'e^ir^iewers reflect in general many of the
sentiments expressed . hy-;..^jiljj, Subcommittee 2% years ago in that
they endorse the use of''.'ji^p€ntal probabilities "as a 'useful
additional tool in the ;.flifi|ard-settin« process r while raising a
variety of specific conc^ferii^ with the methodology. Ihough the
overall cciranents • appear. fc£;Se favorable, it must be recognized
that this is a first view
-------
Advisory Statement ^J vv.•':%";•••
•Hie Subcommittee again .wishes to commend and compliment the
Office of Air Quality Plafwting and standards (QAQPS) on their
efforts in- developing health risk assessment and evaluation
methodologies and recommends that these efforts be continued.
But we see no significant progress.in the P/B Beport. The
comments of the reviewers^reflect, in general, many of the
sentiments expressed by.-,tfcls Subcommittee 2 1/2 years ago in- that
they endorse the use of \J:t3-pE|mental probabilities as a useful
additional .tool in ; the:'sl'iti«3ar«3-setting process while raising a
variety of specific coocetriii.
We recognize that this is a first view of the P/B approach
by these commentators,, but we have had the advantage of watching
the development of the process over time, and our opinion is that
there has been a lack of' significant progress toward developing
an implementable tool. We'-see no reason to believe this
situation will change* •; -'_,; ••'
Despite our conc«rtt&.''about this specific piece of analysis,
we believe that the overall program of health risk assessment in
which Q&QFS is now ' enga$&& is well conceived and making excellent
progress. We believe that ••'•'the OAQPS is the appropriate unit
within EFA to pursue this work, and we recommend that they
continue to develop the alternative approaches now under
consideration, :
Conclusions
In the view of the .Sufocojpiittee, the P/B method is not a
promising approach for developing an iinplementable tool in the
foreseeable future.
The Subcommittee recommends that OAQPS continue to develop
the two alternative approaches.
VThis is a slightly edited version of a statement, reflecting
a consensus of Subcommittee members, read into the record by
the Subcommittee Chairman, Dr. Curran, at,the May 13, 1§81
meeting of the Subcommittee*
- 9 -
-------
Appendix A
SCIENCE ADVISORY BOARD
SUBCOMMITTEE'::;0W HEALTH RISK ASSESSMENT
Chairman:
Dr. Anita S. Curran, Commissioner, Department of Health,
Westchester County, County Office Building 2,
150 Grand Street, white Plains, New York 10601
Members:
Dr. Bernard Altshuier, Director of Biomathematics laboratory,
Institute of Envirowueiital Medicine, New fork University
Medical Center, 550 First Avenae, New York, New York 10016
Dr. Judy A. Bean, Associate 'Professor, College of Medicine,
Department of Preventive ledieine ana Environmental Health,
University of Iowa, ^ Iowa i, City, Iowa 52242
Dr. Granger Morgan, Head, • Etepartraent of Engineering ani3 Poblic
Policy, Carnegie-Mellon University, 5000 Forbes Avenue,
Pittsburgh f Pennsylvejtia 15213
Dr, D, Warner North, Principal, Decision Focus Inc*, 5 Palo Alto
Square, Suite 410, Palo'Mto, California S4304
Dr. Geoffrey Watson, Professor and Director of Graduate Studies?
Department of statistics, Fine Hall, Princeton University:,
Princeton, Slew Jersey 08544
SAB Staff Officer?
Mr, Ernst Linde? Scientist Administrator, Science' Advisory
Board, A-lOlKjr U,S, ^'Environmental Protection Agency,
Washington, B.C.- 204'«o:"'"Phones (202) 472-0444
A-l
-------
Appendix B
Comments by Geoffrey S»..Wat.son on Appendix G, Mathematical
Treatment of Un,eertaintyVi*C;:;P611utanit Concentrations of BA
General Method for Assess-J^f Health Risks Associated with •
Primary National Ambient'Air Quality Standards* (Craft dated
April 1981} " "'t'^i '
Appendix G seeks the probability distribution of the
n largest, in a series of la values, ISiree years ago I
pointed out the need to consider dependent values, ^he authors
now recognize this bat 'laalie.': totally incorrect statements, The 1
lines giving the theory for/- the independence case are marred by
five typos i one leading to 'nonsense. Hie discussion of this
topic is at best opaque.. ;.jf.
If the 15 values or<3'eri&C:|fem largest to 'smallest are
If G(x) :;».^i;|t:' i>x) , for any
(as in (Ol) } t the c&aiieelithat exactly v of the m values
will be greater than "C^IisVr from the binomial distribution,
, ™™ipnnmiN^^^^HtHHnHiiMWM ^ ^ %,J ^ ^rf £ ^ ^
Now x,
-------
particular case,,of the Weibull distribution (first
paragraph on p. CS-3) 'co^feetly sets
; • . G(C) :*;''ejcp - (C/5)k . (G-6)
fhe expected number .ofAeitceeclaiices E of a standard '$S£$ in
iS ' . v''i^';^ • ."• • ' - :.^::.- /"':•: '" • ' ''
; To proceed-one iBu^t$pii6w what: is meant by, .the P^
This is apparently 'defin%'d'r;in Section 8.2, but'."this.section is
absolutely baffling. '."•':.-;:--. .
From Section'8.1, ••ifc;:^«ems clear that if the standard
requires E in (i) to he less than unity., then cst£ could be
determined from (i) if --'^-•and ic are known,
Returning to Section'8.2 the notation and definition of P^
seems to change from line to line. Hence I cannot say whether (G-7)
and the paragraph below it. 'are right or wrong,
The remainder of Appendix G deals with the problems caused by
hour*by-ho«r correlations and non-'stationarity. The latter means
daily and seasonal variations. For daily masisBSBD the authors refer
to a paper by Sorowits and Sarakatz (which I have not been able to
see) who use the representation , •
log CCtj) m o(ti) -f ab1i...,ib_ (ii)
I put a finite number of b*s because that is presumably intended,
supposed to be •uneorrel&ted and
that the t^ are equally spaced, Sie condition {ii} no doubt is
B-2
-------
supposed to be necessary, and sufficient to guarantee stationarity
i
It is NEITHER as the trivial example
! Mt) » *(t-l) + a(t-2) + a0(t)
j
should make clear to anyone knowing this theory*
| Further, Peagans and 'Biller assert that Horowitsj 'and Barakatz
have derived the distribution of the maximum (presumably over a
long period like a year) from their model* Uiis is a major achievement
because mathematicians have been trying fruitlessly to solve this
problem for at least 30 years (see, e.g., Watson/ Ann. Math. Stat. 1§54)
Finally go one can guess these results. I have done many
simulations an<3 yet am totally unable to*guess the results when I
i
vary the parameters.
B-3
------- |