NEW YORK UNIVERSITY MEDICAL CENTER A private university in the public service Institute of Environmental Medicine .iSO f IRST AVENUE, NEW YORK, N.Y, 10016 AREA 212 340.7300. Ew. 8SS- ANTHONY 1. LANZA RIS6ARCH LABORATORIES AT UNIVERSITY VALLEY LONG MEADOW ROAD, SWUNG FOR6ST. TUXEOO, N.V, 109i7 MAil AND TilJPHONE ADDRESS: SSOftRST AVENUE, NEW YORK. N.Y. 10016 July 31, 1985 Honorable Lse M. Thomas Administrator U. S. Environmental Protection Agency 401 M Street, S. W. Washington, D, C. 20460 Dear Mr. Thomas: The Clean Air Scientific Advisory Committee (CASAC) has completed a report on various ways to improve the process for developing National Ambient Mr Quality Standards, As you know,. CASAC has been a direct participant in setting NAAQS through its reviews of their technical bases. In general, the Committee believes that the Agency has made great strides in incorporating scientific issues into NMOS development and, as a result, both it and many members of the public believe that EPA's process for selecting levels for standards is both more widely understood and more scientifically defensible. With this in mind, CASAC seeks, in this current report, to identify areas to further improve the standard setting process. Among the issues examined include 1) streamlining and expediting the preparation and review of air quality criteria documents and staff papers; 2} the need to develop procedures to evaluate the scientific quality of benefits analyses whose preparation is required under Executive Order 12291; and 3} improving the nature and timeliness, of CASAC1s advice to the Agency. The CASAC appreciates the cooperation and input it has received frcra EPA staff during the preparation of this'report. 1 look forward to learning which of the CASAC's reccmmendations will be accepted by EPA. ,, Sincerely,.. , "] /} f /ty«?TW I '7' Morton Lippmann,. Chairman Clean Air Scientific Advisory Committee Science Advisory Board cc: A. James Barnes Charles Elkins Bernard Goldstein Terry F. Yosie ------- REPORT OP THE CIEAN AIR SCIENTIFIC AWISORf CCMMIITEE (CASAC) ON IMPROVING THE PROCESS FOR SETTING NATIONAL AMBIENT AIR QUALITY STANDEES: AN tJPEATE Science Advisory Board [J. S. Environmental Protection Agency July 1985 ------- I. HSJlROmCTION Cne of the major responsibilities of the Clean Mr Scientific Advisory Gcnroittee (CASAC), as established by the 1977 Clean Air Act Amendments, is to undertake a number of review functions associated with EPA's development, promulgation,- and implementation of National Ambient Air Quality Standards (NAAQS). lb this end, CASAC issued a report to the Administrator and to Congress in September 1981 entitled "Setting Ambient Air Quality Standards; Improving the Process", EPA has implemented many of CASAC1s recommendations in this report. As a result, both the Cotmuttee and many members of the general public believe that the process for developing NAAQS is more widely understood and more scientifically defensible, CASAC, with four more years of experience with the process of setting MACS has examined several additional issues to further improve this process. These include ways to Streanline and expedite the preparation and review of air quality criteria documents and staff papers, and to develop procedures for evaluating the scientific adequacy of benefits analysis. In addition, CASAC makes several recommendations for improving the quality and timeliness of its own advice to EPA, During the preparation of this report, CASAC benefited from information gathering interviews with EPA staff as well as members of the public. In particular, the Committee wishes to express its appreciation to Alex Cristofaro, Office of Policy, Planning and Evaluation; Bruce Jordan and Harvey Richmond, Office of Air Quality Planning and Standards; les Grant, Environmental Criteria and Assessment Office? Jack Hidinger, Office of Air and Radiation; and Richard H. Etowd, R. M» Cowd, Associates, II. CASAC FINDINGS AND RECCMMENCATIONS FCR PREPARATION OF NAflQS AIR QUALITY CRITERIA DOCUMENTS Objective Ib facilitate preparation by EPA of air quality criteria documents which present concise summaries and interpretive discussions of scientific and technological data relevant to the establishment of HAAQSs. Background _, ... .. . ... . , _;-.- •.-• ' EPA has greatly improved the process by which criteria documents are prepared. This has resulted in generally high quality documents that are submitted to CASAC for review and ultimately published by the Agency. The current process starts with development of a document preparation plan by the Environmental Criteria and Assessment Office in consultation with internal EPA task force representatives that include knowledgeable scientists from CRD research laboratories, program office (O&R, OAQPS) staff members, and staff from other interested Agency offices. Authors of criteria docu- ment chapters are selected based upon task force discussions, and the entire development plan, including selection of authors, is received and commented on by CASAC at a public meeting. After preparation of initial draft criteria document chapters, RCAO convenes expert workshops to review ------- -2- their scope and quality. The criteria document development process includes the use of preliminary peer review workshops wherein EPA and non-EPJ\ scientific experts discuss early drafts of criteria document materials, which are then revised before release for public comment and CASAC review. As a result, current criteria documents achieve a high standard for objective reporting and summarization of the pertinent scientific data bases and contain valuable critical analyses of the strengths and weaknesses of key studies leading to bottom line conclusions regarding lowest-observed effect levels and dose-response relationships* Defects inthe Current Process for Preparation of Criteria Documents Existing criteria documents have sane serious limitations which ultimately affect their value to key Agency decision makers. One problem yet to be resolved is a definition of a mechanism by which to encourage chapter authors to be brief and to focus upon key scientific studies. Difficulties are encountered, for example, in omitting or not emphasizing the work of colleagues who may be present at the workshop or on CAS&C, who may be offended to find that their studies were deemed not critical to standards setting. Mso, workshops held to review comprehensive draft CD chapters somtimes end up devoting too much time to editorial issues and the merits of studies that are not critical to standards setting. Finally, a workshop is an inefficient forum for adding well developed interpretive discussion if it is not already present in the draft chapter. Current criteria documents are too encyclopedic in describing the data base, often making it difficult to identify and synthesize the data most critical to standards setting. At the same time, they have too little interpretive discussion of the critical data and their implications for standard setting purposes, e.g. explicitly identifying all low level effects that may qualify as adverse bicmedical effects, fin addition, the criteria documents often take too long to prepare and consume large amounts of EPA's resources. CASAC's Proposal for an Improved Criteria Document Development Process 1. Ihe criteria document needs to be planned better at the outset and more clearly take into account inputs from the program (CBB, C&OPS) and policy offices,, as well as- ECA0--and-other ORD components. .This should include the defining of critical issues in standards setting involving; a) scientific data, its evaluation and interpretation; b) use of models in risk assessments (including exposure assessments)? c) appropriate pollutant indices, averaging times, etc. and determination of which of these issues should be addressed in the criteria document and which in other standard setting materials, e.g. the OADPS staff .paper. 2. BCAO should prepare guidance to chapter managers on critical issues or topics to be addressed in criteria document chapters, and select criteria document chapter managers who are thoroughly familiar with their topics and the relevant literature. The major focus of the criteria document should be on the evaluation and interpretation of health and welfare effects data. Other background information (such as a discussion of ------- 3 - physico-chemical properties, sources and anissions, measurement raethologies, transport and transformation and environmental concentrations) helpful in placing the effects data in perspective should be concisely summarized in the criteria document and reference made to more detailed discussions of these issues contained in other published materials or separate background documents prepared by ECAQ. 3, BCAQ should direct chapter managers and other authors, to prepare a comprehensive annotated bibliography for their chapters or sections of chapters. These could follow the form of some of the summary tables in recent criteria documents or, perhaps, computerized storage files (which ECAO is now establishing) containing brief summaries and critiques of studies. 4. ECAO should distribute bibliographies to anticipated workshop partici- pants prior to the workshop and request that each expert provide to ECAO and the criteria document authors feedback on which studies are pertinent for standard setting purposes. Given the number of possible studies, ECAO may want to consider making assignments to reviewers to ensure that all of the articles are critically evaluated. This might be accomplished by means of a rating sheet on which each workshop participant would enter a score for each study listed in the bibliography. Each study would receive a score as follows j 1) highly relevant to standards setting 2) possibly relevant to standards setting 3) of little or no apparent relevance to standards setting 4) critically defective - should not be cited, except as to note why it should not be used, 5. Criteria document authors should draft their chapters by evaluating those studies most clearly identified as important for standard-setting and by presenting the principal conclusions to be drawn from those studies* The workshop convened to review the draft criteria document chapters would focus initially on the selection of pertinent studies included in the draft chapter and resolution of":whe~ther' any additional studies need- to be'-added' - to the chapter. Studies not selected for inclusion in the criteria document Chapter would appear only in the backup annotated bibliographies cited in the criteria document as being publically available upon request (and which would be placed in the appropriate Agency docket files). Advocates of including studies with a majority of "3" or "4" scores or for excluding studies with a majority of "1" scores would have an opportunity to challenge the majority view at the workshop, along with discussion of which "2" score studies should be included in the chapter. Cnce the study selection issue is resolved early in the workshop discussion of the given criteria document chapter, then attention should focus on the evaluative/interpretive aspects of the draft chapter materials and what changes need to be made to reflect workshop deliberations* ------- CASAC understands that new studies will continue to appear in the scientific literature as the criteria document development process proceeds. Thus, the final list of studies included in the criteria document will differ somewhat from the original list, 6. The criteria document chapter managers would subsequently redraft the narrative chaptersr being sure to include concise descriptions of all of the selected relevant studies but devoting most of their efforts to sharpening the interpretive discussions of the significance of the studies to the setting of MAAQS. 7. The revised draft chapters would be sent to the workshop partici- pants for their preparation of written reviews and coranents and, if found to be necessary, ECAQ could convene a second workshop to resolve any significant remaining controversies concerning discussions of individual studies or overall interpretation of their collective results. 8. ECAG staff would make final editorial changes, add an executive summary incorporating overall interpretive discussions, and distribute the resulting materials as the first external review draft for public cerement and CftSAC review, 9. Based on public ccnwients and CASAC review and recommendations, ECAO would then carry out, if needed, any further revisions in completing a final version of the document ready for use by CAQPS and other Agency offices (e.g. GPPE) in developing other materials used in the standard setting process. III. CASAC REODMMEM3ATIONS FOR PLANNING THE 0£QPS STAFF PAPER Objective To achieve early identification of critical data needs to support the selection of index pollutantsr adverse health effects, sensitive populations, margins of safety, most sensitive welfare effects, influence of concentration and averaging times on effects and alternate-ferns for standards. Background - The OR.QPS-staff-paper".for'-NftAQS has- proven to be a'-- highly'Successful - •• - effort in bridging the gap between the scientific studies contained in the criteria documents and the public health policy judgments required of the Administrator in setting NAAQS, In particular, it has enabled both the scientific comrunity and the general public to examine how EPA staff integrate a host of technical data into a rationale for choosing among various policy options. The implementation of CASAC's recommendations for preparing criteria documents and the anticipated compression of the time frame may not, however, leave adegaate time for C&QPS to plan the development of the ------- -5- staff paper. Earlier and more focussed planning may be heeded to permit the issuance of a staff paper sooner than it has in the past. CASAC's Proposal for Staff Paper Planning Q&OPS is in the process of incorporating additional exposure analyses into its staff paper. CASAC has, for example, previously reviewed the exposure analyses for carbon monoxide and sulfur dioxide, and it encourages the development of exposure profiles and analyses for all NAAQS, Ihese analyses will prove especially helpful as C&OBS introduces more formal methods of risk assessment and benefits analysis into NMQS development. By incorporating exposure assessment, risk assessmsnt and benefits analysis, the staff' paper planning process would address the following issues (most of which are already included in the existing process): 1. identification of the sensitive individuals or population groups expected to be at risk frcm both adute and ohronic exposures to a particular criteria pollutant. 2, Identification of the important receptors (atmospheric, aquatic and terrestrial) to evaluate and protect against welfare affects. 3. Articulation of the options from vfaich Agency staff select the index pollutants and the averaging times for short-term and longer-term standards. 4. Evaluation of the critical effects that influence the selection of the form and the levels of primary and secondary standards. ^ 5. Identification of the factors affecting the selection of margins of safety. 6. Assessment of factors determining the reliability of various models used in the staff paper's development. In particular, the staff paper should discuss the use of and the rationale for choosing specific rnodels, the uncertainties associated with the chosen model, and a comparison of the results of the selected model with other models that may apply. 7. Statement of the ranges of interest within which OAOPS staff recornniend standard levels to the Administrator,'as supported by-the available'scientific data." Q&QES should, to the extent practicable, develop exposure assessments for the populations at risk and the welfare receptors which the primary and secondary standards, respectively, are designed to protect. In addition to these issues, OAQPS is applying methods for risk assess- ment and benefits analysis into the MAOS development process. The next section of this report will address CASAC's views on these issues. ------- -6- IV. THE USE OF RISK ASSESSMENT AND BENEFITS ANALYSES IN SETTING MMD3 Objective To establish procedures for the evaluation, inclusion and peer review of Agency risk assessments and benefits analyses during the development of NMOS, Background Various EPA program offices -make extensive use of risk assessment as a basis for deciding among regulatory options. A risk assessment is generally defined as including hazard identification, dose-response assessment, exposure assessment and risk and uncertainty characterization. GAQPS is seeking to apply risk assessment methodologies to the setting of NAAQS, ' - In parallel with this effort is the requirement under Executive Order 12291 for EPA to prepare Regulatory Impact Analyses (FIA) of the costs and benefits of significant regulatory activities. Ml E>syCS are defined as significant actions under this Order. GAQPS currently prepares benefits analyses for each standard but, to date, it has not utilized them as part of the scientific basis to set standard levels. CASAC Recctgnendations Since both risk assessment and benefits analysis involve the use of scientific data, EPA should attempt to systematically evaluate and integrate them into the existing process for preparing and reviewing air quality criteria documents and staff papers. EPA should develop procedures to ensure their peer review which should,, at a minimum, include opportunities for CASAC and public consents, these should be no less rigorous than the procedures already in place for the review of the data bases on pollutant sources, transport, transformation, environmental concentrations and health and welfare effects. An example of peer review that SPA should consider holding from-tine to tirre is to convene scientific workshops. The objective of such peer review, which could be conducted in concert with workshops that EPA already sponsors on NAAOS related documents, is to determine which of the risk assessments and benefits analyses are scientifically adequate for inclusion in the criteria documents and and staff papers," - -.. "- " •'•'• ' :'; : "; "• ~":- !' To the extent that risk assessment and benefits analysis become part of the scientific basis for setting jMA&OS, RPA should submit them for public and CASAC review. Simultaneously, CASAC should supplement its current roster to ensure that it has the appropriate-scientific expertise to conduct the reviews. V. CASAC*S R0t£ IN THE REACTS rEVELOPHEMT PROCESS CASAC's participation in the standard-setting process has evolved since its creation by the passage of the 1977 Clean Air Act Amendments. ------- -7- At the present time CASAC provides its scientific advice through five distinct channels. These include 1) review of air gaality criteria documents? 2) review of QAQPS staff papers; 3) review of the standards proposed in the Federal Register; 4 ) recommendation , on an annual basis, of the needed research to further improve the Agency's and the public's understanding of criteria pollutant mechanisms and effects; and 5) periodic reports on ways to improve the NMQS development process. The Ccnnnittee believes, on the basis of discussions with Agency and Congressional staff and with representatives of industrial and environmental groups , that its role , in general , has wide acceptance . • Like the other elements o£ the standard- sett ing process discussed previously , there is a need for CASAC to update and fine tune the way in which it prepares its scientific advice-. There are several steps that CASAC plans to implement to further enhance its advisory role. These include: * CASAC is sometimes requested by Agency staff to ccrnnent on issues which have policy as well as scientific content. One example is CASAC "s advice on the scientific adequacy of the ranges of interest expressed in the staff paper which include judgments on margin of safety as well as other issues. Trying to totally separate the risk assessment and risk management components of the ranges of interest is likely to prove futile using the presently available analytical tools. Since the ranges of interest are useful to both scientists and policy makers, CASAC recommends that C&QPS retain than as a means of expressing the uncertainties in developing NAAQS, At the same time, CASAC plans to further clarify which aspects of its advice are primarily scientific versus that which is primarily policy related. * During the course of its scientific reviews for the Agency, has continually updated its roster to adjust to the charging needs for expertise. At present, the Committee is particularly strong in the atmospheric and health sciences disciplines, and within the past year the Administrator has also appointed a biostatistician. As it forecasts its future review responsibilities, CASAC recontiends that additional expertise be acquired in the fields of risk assessment, environmental effects assessment and economic analysis. - « CftSAC'S recaranendatiorts'-' for1 streamlining various, aspects of* the " NAAQS developaent process also apply to itself. The Ccranittee believes that it has the capability to provide its written advice to the Pqercy more expeditiously and will strive to do so. In particular, the Coimittee Chair will transmit a letter summarizing its major conclusions and reccrcroerdations within thirty to sixty days of its initial review of a criteria document or staff paper. CASAC will prepare its final report on the scientific adequacy of such documents within ninety days. ------- January 1985 Clean Air Scientific Advisory Committee Science Advisory Board Subcommittee on NAAQS Process Chairman Executive Secretary Er, Morton Lippmann institute of Environmental Medicine Lanza Laboratory tong Meadow Road New York University Tuxedo, New York 10987 (914) 351-5277 Mr* Robert Flaak Science Advisory Board U.S. Environmental Protection Agency (A-101P) 401 M Street, S.W. Washington,, D.C. 20460 (202) 382-2552 Members/Consultants Dr. Mary Arodur Department of Nutrition and Food Science MIT Cambridge, Massachusetts 02139 (617) 253-3111 Dr. Robert Frank Department of Environmental and Health Sciences Johns Hopkins School of Hygiene and Public Health 615 North Wolfe Street Baltimore, Maryland 21205 • . (301) 955-3720 Or. Warren Johnson Director, Atmospheric Science Center Advanced Development Division SRI International 333 Ravenswood Avenue Menlo Park, California 94025 (415) 859^1755 Dr. Timothy larson Department of Civil Engineering Mail Stop FC-05 University of Washington Seattle, Washington 98195 (206) 543-6A15 Dr. Sheldon Priedlander parsons Professor of Chemical Engineerirg Department of Chemical Engineering University of California, tos Angeles lea Angeles, California 90024 (213) 825-2206 Dr. James Ware Department of Biostatistics Harvard School of Public Health 677 Huntington Avenue Boston, Massachusetts 02115 (617) 732-1056 * Member of CASAC ------- |