NEW YORK UNIVERSITY MEDICAL CENTER
A private university in the public service
Institute of Environmental Medicine
.iSO f IRST AVENUE, NEW YORK, N.Y, 10016
AREA 212 340.7300. Ew. 8SS-
ANTHONY 1. LANZA RIS6ARCH LABORATORIES AT UNIVERSITY VALLEY
LONG MEADOW ROAD, SWUNG FOR6ST. TUXEOO, N.V, 109i7
MAil AND TilJPHONE ADDRESS: SSOftRST AVENUE, NEW YORK. N.Y. 10016
July 31, 1985
Honorable Lse M. Thomas
Administrator
U. S. Environmental Protection Agency
401 M Street, S. W.
Washington, D, C. 20460
Dear Mr. Thomas:
The Clean Air Scientific Advisory Committee (CASAC) has completed
a report on various ways to improve the process for developing National
Ambient Mr Quality Standards, As you know,. CASAC has been a direct
participant in setting NAAQS through its reviews of their technical
bases. In general, the Committee believes that the Agency has made great
strides in incorporating scientific issues into NMOS development and, as
a result, both it and many members of the public believe that EPA's process
for selecting levels for standards is both more widely understood and
more scientifically defensible.
With this in mind, CASAC seeks, in this current report, to identify
areas to further improve the standard setting process. Among the issues
examined include 1) streamlining and expediting the preparation and
review of air quality criteria documents and staff papers; 2} the need to
develop procedures to evaluate the scientific quality of benefits analyses
whose preparation is required under Executive Order 12291; and 3} improving
the nature and timeliness, of CASAC1s advice to the Agency.
The CASAC appreciates the cooperation and input it has received frcra
EPA staff during the preparation of this'report. 1 look forward to
learning which of the CASAC's reccmmendations will be accepted by EPA.
,, Sincerely,..
, "] /} f
/ty«?TW
I '7'
Morton Lippmann,. Chairman
Clean Air Scientific Advisory Committee
Science Advisory Board
cc: A. James Barnes
Charles Elkins
Bernard Goldstein
Terry F. Yosie
-------
REPORT OP THE CIEAN AIR
SCIENTIFIC AWISORf CCMMIITEE (CASAC)
ON
IMPROVING THE PROCESS FOR SETTING
NATIONAL AMBIENT AIR QUALITY STANDEES: AN tJPEATE
Science Advisory Board
[J. S. Environmental Protection Agency
July 1985
-------
I. HSJlROmCTION
Cne of the major responsibilities of the Clean Mr Scientific Advisory
Gcnroittee (CASAC), as established by the 1977 Clean Air Act Amendments, is
to undertake a number of review functions associated with EPA's development,
promulgation,- and implementation of National Ambient Air Quality Standards
(NAAQS). lb this end, CASAC issued a report to the Administrator and to
Congress in September 1981 entitled "Setting Ambient Air Quality Standards;
Improving the Process", EPA has implemented many of CASAC1s recommendations
in this report. As a result, both the Cotmuttee and many members of the
general public believe that the process for developing NAAQS is more widely
understood and more scientifically defensible, CASAC, with four more
years of experience with the process of setting MACS has examined several
additional issues to further improve this process. These include ways to
Streanline and expedite the preparation and review of air quality criteria
documents and staff papers, and to develop procedures for evaluating the
scientific adequacy of benefits analysis. In addition, CASAC makes
several recommendations for improving the quality and timeliness of its
own advice to EPA,
During the preparation of this report, CASAC benefited from information
gathering interviews with EPA staff as well as members of the public. In
particular, the Committee wishes to express its appreciation to Alex
Cristofaro, Office of Policy, Planning and Evaluation; Bruce Jordan and
Harvey Richmond, Office of Air Quality Planning and Standards; les Grant,
Environmental Criteria and Assessment Office? Jack Hidinger, Office of Air
and Radiation; and Richard H. Etowd, R. M» Cowd, Associates,
II. CASAC FINDINGS AND RECCMMENCATIONS FCR PREPARATION OF NAflQS AIR
QUALITY CRITERIA DOCUMENTS
Objective
Ib facilitate preparation by EPA of air quality criteria documents which
present concise summaries and interpretive discussions of scientific and
technological data relevant to the establishment of HAAQSs.
Background _, ... .. . ... . , _;-.- •.-• '
EPA has greatly improved the process by which criteria documents are
prepared. This has resulted in generally high quality documents that are
submitted to CASAC for review and ultimately published by the Agency.
The current process starts with development of a document preparation
plan by the Environmental Criteria and Assessment Office in consultation with
internal EPA task force representatives that include knowledgeable scientists
from CRD research laboratories, program office (O&R, OAQPS) staff members,
and staff from other interested Agency offices. Authors of criteria docu-
ment chapters are selected based upon task force discussions, and the
entire development plan, including selection of authors, is received and
commented on by CASAC at a public meeting. After preparation of initial
draft criteria document chapters, RCAO convenes expert workshops to review
-------
-2-
their scope and quality. The criteria document development process
includes the use of preliminary peer review workshops wherein EPA and
non-EPJ\ scientific experts discuss early drafts of criteria document
materials, which are then revised before release for public comment and
CASAC review. As a result, current criteria documents achieve a high
standard for objective reporting and summarization of the pertinent
scientific data bases and contain valuable critical analyses of the
strengths and weaknesses of key studies leading to bottom line conclusions
regarding lowest-observed effect levels and dose-response relationships*
Defects inthe Current Process for Preparation of Criteria Documents
Existing criteria documents have sane serious limitations which
ultimately affect their value to key Agency decision makers. One problem
yet to be resolved is a definition of a mechanism by which to encourage
chapter authors to be brief and to focus upon key scientific studies.
Difficulties are encountered, for example, in omitting or not emphasizing
the work of colleagues who may be present at the workshop or on CAS&C, who
may be offended to find that their studies were deemed not critical to
standards setting. Mso, workshops held to review comprehensive draft CD
chapters somtimes end up devoting too much time to editorial issues and
the merits of studies that are not critical to standards setting. Finally,
a workshop is an inefficient forum for adding well developed interpretive
discussion if it is not already present in the draft chapter.
Current criteria documents are too encyclopedic in describing the data
base, often making it difficult to identify and synthesize the data most
critical to standards setting. At the same time, they have too little
interpretive discussion of the critical data and their implications for
standard setting purposes, e.g. explicitly identifying all low level effects
that may qualify as adverse bicmedical effects, fin addition, the criteria
documents often take too long to prepare and consume large amounts of EPA's
resources.
CASAC's Proposal for an Improved Criteria Document Development Process
1. Ihe criteria document needs to be planned better at the outset and
more clearly take into account inputs from the program (CBB, C&OPS) and
policy offices,, as well as- ECA0--and-other ORD components. .This should
include the defining of critical issues in standards setting involving;
a) scientific data, its evaluation and interpretation; b) use of models
in risk assessments (including exposure assessments)? c) appropriate
pollutant indices, averaging times, etc. and determination of which of
these issues should be addressed in the criteria document and which in
other standard setting materials, e.g. the OADPS staff .paper.
2. BCAO should prepare guidance to chapter managers on critical issues
or topics to be addressed in criteria document chapters, and select criteria
document chapter managers who are thoroughly familiar with their topics
and the relevant literature. The major focus of the criteria document
should be on the evaluation and interpretation of health and welfare
effects data. Other background information (such as a discussion of
-------
3 -
physico-chemical properties, sources and anissions, measurement raethologies,
transport and transformation and environmental concentrations) helpful in
placing the effects data in perspective should be concisely summarized in the
criteria document and reference made to more detailed discussions of these
issues contained in other published materials or separate background documents
prepared by ECAQ.
3, BCAQ should direct chapter managers and other authors, to prepare a
comprehensive annotated bibliography for their chapters or sections of
chapters. These could follow the form of some of the summary tables in
recent criteria documents or, perhaps, computerized storage files (which
ECAO is now establishing) containing brief summaries and critiques of
studies.
4. ECAO should distribute bibliographies to anticipated workshop partici-
pants prior to the workshop and request that each expert provide to ECAO and
the criteria document authors feedback on which studies are pertinent for
standard setting purposes. Given the number of possible studies, ECAO may
want to consider making assignments to reviewers to ensure that all of the
articles are critically evaluated. This might be accomplished by means of a
rating sheet on which each workshop participant would enter a score for
each study listed in the bibliography. Each study would receive a score as
follows j
1) highly relevant to standards setting
2) possibly relevant to standards setting
3) of little or no apparent relevance to standards setting
4) critically defective - should not be cited, except as to note
why it should not be used,
5. Criteria document authors should draft their chapters by evaluating
those studies most clearly identified as important for standard-setting and
by presenting the principal conclusions to be drawn from those studies* The
workshop convened to review the draft criteria document chapters would
focus initially on the selection of pertinent studies included in the draft
chapter and resolution of":whe~ther' any additional studies need- to be'-added' -
to the chapter. Studies not selected for inclusion in the criteria document
Chapter would appear only in the backup annotated bibliographies cited in
the criteria document as being publically available upon request (and which
would be placed in the appropriate Agency docket files). Advocates of
including studies with a majority of "3" or "4" scores or for excluding
studies with a majority of "1" scores would have an opportunity to challenge
the majority view at the workshop, along with discussion of which "2" score
studies should be included in the chapter. Cnce the study selection issue
is resolved early in the workshop discussion of the given criteria document
chapter, then attention should focus on the evaluative/interpretive aspects
of the draft chapter materials and what changes need to be made to reflect
workshop deliberations*
-------
CASAC understands that new studies will continue to appear in the
scientific literature as the criteria document development process
proceeds. Thus, the final list of studies included in the criteria
document will differ somewhat from the original list,
6. The criteria document chapter managers would subsequently redraft
the narrative chaptersr being sure to include concise descriptions of all
of the selected relevant studies but devoting most of their efforts to
sharpening the interpretive discussions of the significance of the studies
to the setting of MAAQS.
7. The revised draft chapters would be sent to the workshop partici-
pants for their preparation of written reviews and coranents and, if found
to be necessary, ECAQ could convene a second workshop to resolve any
significant remaining controversies concerning discussions of individual
studies or overall interpretation of their collective results.
8. ECAG staff would make final editorial changes, add an executive
summary incorporating overall interpretive discussions, and distribute
the resulting materials as the first external review draft for public
cerement and CftSAC review,
9. Based on public ccnwients and CASAC review and recommendations,
ECAO would then carry out, if needed, any further revisions in completing
a final version of the document ready for use by CAQPS and other Agency
offices (e.g. GPPE) in developing other materials used in the standard
setting process.
III. CASAC REODMMEM3ATIONS FOR PLANNING THE 0£QPS STAFF PAPER
Objective
To achieve early identification of critical data needs to support the
selection of index pollutantsr adverse health effects, sensitive populations,
margins of safety, most sensitive welfare effects, influence of concentration
and averaging times on effects and alternate-ferns for standards.
Background
- The OR.QPS-staff-paper".for'-NftAQS has- proven to be a'-- highly'Successful - •• -
effort in bridging the gap between the scientific studies contained in the
criteria documents and the public health policy judgments required of the
Administrator in setting NAAQS, In particular, it has enabled both the
scientific comrunity and the general public to examine how EPA staff
integrate a host of technical data into a rationale for choosing among
various policy options.
The implementation of CASAC's recommendations for preparing criteria
documents and the anticipated compression of the time frame may not,
however, leave adegaate time for C&QPS to plan the development of the
-------
-5-
staff paper. Earlier and more focussed planning may be heeded to permit
the issuance of a staff paper sooner than it has in the past.
CASAC's Proposal for Staff Paper Planning
Q&OPS is in the process of incorporating additional exposure analyses
into its staff paper. CASAC has, for example, previously reviewed the
exposure analyses for carbon monoxide and sulfur dioxide, and it encourages
the development of exposure profiles and analyses for all NAAQS, Ihese
analyses will prove especially helpful as C&OBS introduces more formal
methods of risk assessment and benefits analysis into NMQS development.
By incorporating exposure assessment, risk assessmsnt and benefits
analysis, the staff' paper planning process would address the following
issues (most of which are already included in the existing process):
1. identification of the sensitive individuals or population groups
expected to be at risk frcm both adute and ohronic exposures to a particular
criteria pollutant.
2, Identification of the important receptors (atmospheric, aquatic and
terrestrial) to evaluate and protect against welfare affects.
3. Articulation of the options from vfaich Agency staff select the index
pollutants and the averaging times for short-term and longer-term standards.
4. Evaluation of the critical effects that influence the selection of
the form and the levels of primary and secondary standards. ^
5. Identification of the factors affecting the selection of margins of
safety.
6. Assessment of factors determining the reliability of various models
used in the staff paper's development. In particular, the staff paper
should discuss the use of and the rationale for choosing specific rnodels,
the uncertainties associated with the chosen model, and a comparison of the
results of the selected model with other models that may apply.
7. Statement of the ranges of interest within which OAOPS staff recornniend
standard levels to the Administrator,'as supported by-the available'scientific
data."
Q&QES should, to the extent practicable, develop exposure assessments
for the populations at risk and the welfare receptors which the primary and
secondary standards, respectively, are designed to protect.
In addition to these issues, OAQPS is applying methods for risk assess-
ment and benefits analysis into the MAOS development process. The next
section of this report will address CASAC's views on these issues.
-------
-6-
IV. THE USE OF RISK ASSESSMENT AND BENEFITS ANALYSES IN SETTING MMD3
Objective
To establish procedures for the evaluation, inclusion and peer review
of Agency risk assessments and benefits analyses during the development
of NMOS,
Background
Various EPA program offices -make extensive use of risk assessment as
a basis for deciding among regulatory options. A risk assessment is
generally defined as including hazard identification, dose-response
assessment, exposure assessment and risk and uncertainty characterization.
GAQPS is seeking to apply risk assessment methodologies to the setting of
NAAQS, ' -
In parallel with this effort is the requirement under Executive Order
12291 for EPA to prepare Regulatory Impact Analyses (FIA) of the costs and
benefits of significant regulatory activities. Ml E>syCS are defined as
significant actions under this Order. GAQPS currently prepares benefits
analyses for each standard but, to date, it has not utilized them as part
of the scientific basis to set standard levels.
CASAC Recctgnendations
Since both risk assessment and benefits analysis involve the use of
scientific data, EPA should attempt to systematically evaluate and integrate
them into the existing process for preparing and reviewing air quality
criteria documents and staff papers. EPA should develop procedures to
ensure their peer review which should,, at a minimum, include opportunities
for CASAC and public consents, these should be no less rigorous than the
procedures already in place for the review of the data bases on pollutant
sources, transport, transformation, environmental concentrations and
health and welfare effects. An example of peer review that SPA should
consider holding from-tine to tirre is to convene scientific workshops.
The objective of such peer review, which could be conducted in concert
with workshops that EPA already sponsors on NAAOS related documents,
is to determine which of the risk assessments and benefits analyses are
scientifically adequate for inclusion in the criteria documents and and
staff papers," - -.. "- " •'•'• ' :'; : "; "• ~":- !'
To the extent that risk assessment and benefits analysis become part
of the scientific basis for setting jMA&OS, RPA should submit them for
public and CASAC review. Simultaneously, CASAC should supplement its
current roster to ensure that it has the appropriate-scientific expertise
to conduct the reviews.
V. CASAC*S R0t£ IN THE REACTS rEVELOPHEMT PROCESS
CASAC's participation in the standard-setting process has evolved
since its creation by the passage of the 1977 Clean Air Act Amendments.
-------
-7-
At the present time CASAC provides its scientific advice through five
distinct channels. These include 1) review of air gaality criteria
documents? 2) review of QAQPS staff papers; 3) review of the standards
proposed in the Federal Register; 4 ) recommendation , on an annual
basis, of the needed research to further improve the Agency's and the
public's understanding of criteria pollutant mechanisms and effects; and
5) periodic reports on ways to improve the NMQS development process.
The Ccnnnittee believes, on the basis of discussions with Agency and
Congressional staff and with representatives of industrial and environmental
groups , that its role , in general , has wide acceptance . • Like the other
elements o£ the standard- sett ing process discussed previously , there is a
need for CASAC to update and fine tune the way in which it prepares its
scientific advice-. There are several steps that CASAC plans to implement
to further enhance its advisory role. These include:
* CASAC is sometimes requested by Agency staff to ccrnnent on issues
which have policy as well as scientific content. One example is CASAC "s
advice on the scientific adequacy of the ranges of interest expressed in
the staff paper which include judgments on margin of safety as well as
other issues. Trying to totally separate the risk assessment and risk
management components of the ranges of interest is likely to prove futile
using the presently available analytical tools. Since the ranges of
interest are useful to both scientists and policy makers, CASAC recommends
that C&QPS retain than as a means of expressing the uncertainties in
developing NAAQS, At the same time, CASAC plans to further clarify which
aspects of its advice are primarily scientific versus that which is primarily
policy related.
* During the course of its scientific reviews for the Agency,
has continually updated its roster to adjust to the charging needs for
expertise. At present, the Committee is particularly strong in the
atmospheric and health sciences disciplines, and within the past year the
Administrator has also appointed a biostatistician. As it forecasts its
future review responsibilities, CASAC recontiends that additional expertise
be acquired in the fields of risk assessment, environmental effects
assessment and economic analysis.
- « CftSAC'S recaranendatiorts'-' for1 streamlining various, aspects of* the "
NAAQS developaent process also apply to itself. The Ccranittee believes
that it has the capability to provide its written advice to the Pqercy
more expeditiously and will strive to do so. In particular, the Coimittee
Chair will transmit a letter summarizing its major conclusions and
reccrcroerdations within thirty to sixty days of its initial review of a
criteria document or staff paper. CASAC will prepare its final report on
the scientific adequacy of such documents within ninety days.
-------
January 1985
Clean Air Scientific Advisory Committee
Science Advisory Board
Subcommittee on NAAQS Process
Chairman
Executive Secretary
Er, Morton Lippmann
institute of Environmental
Medicine
Lanza Laboratory
tong Meadow Road
New York University
Tuxedo, New York 10987
(914) 351-5277
Mr* Robert Flaak
Science Advisory Board
U.S. Environmental Protection
Agency (A-101P)
401 M Street, S.W.
Washington,, D.C. 20460
(202) 382-2552
Members/Consultants
Dr. Mary Arodur
Department of Nutrition and
Food Science
MIT
Cambridge, Massachusetts 02139
(617) 253-3111
Dr. Robert Frank
Department of Environmental and
Health Sciences
Johns Hopkins School of Hygiene
and Public Health
615 North Wolfe Street
Baltimore, Maryland 21205 • .
(301) 955-3720
Or. Warren Johnson
Director, Atmospheric Science Center
Advanced Development Division
SRI International
333 Ravenswood Avenue
Menlo Park, California 94025
(415) 859^1755
Dr. Timothy larson
Department of Civil Engineering
Mail Stop FC-05
University of Washington
Seattle, Washington 98195
(206) 543-6A15
Dr. Sheldon Priedlander
parsons Professor of Chemical
Engineerirg
Department of Chemical Engineering
University of California, tos Angeles
lea Angeles, California 90024
(213) 825-2206
Dr. James Ware
Department of Biostatistics
Harvard School of Public Health
677 Huntington Avenue
Boston, Massachusetts 02115
(617) 732-1056
* Member of CASAC
------- |