UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON D.C. 20460
OFFICE OF THE ADMINISTRATOR
SCIENCE ADVISORY BOARD
October 25, 2005
EPA-CASAC-CON-06-002
Honorable Stephen L. Johnson
Administrator
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, NW
Washington, DC 20460
Subject: Clean Air Scientific Advisory Committee (CASAC) Ozone Review Panel's
Consultation on EPA's Draft Ozone Environmental Assessment Plan: Scope
and Methods for Exposure, Risk and Benefits Assessment (August 2005)
Dear Administrator Johnson:
EPA's Clean Air Scientific Advisory Committee (CASAC), supplemented by subject-
matter-expert Panelists — collectively referred to as the CASAC Ozone Review Panel ("Panel")
— met via a public teleconference on October 3, 2005, to conduct a consultation with staff from
EPA's Office of Air Quality Planning and Standards (OAQPS), within the Office of Air and
Radiation (OAR), on the Agency's draft Ozone Environmental Assessment Plan: Scope and
Methods for Exposure, Risk and Benefits Assessment (August 2005). The current CASAC roster
is attached as Appendix A of this letter, and the CASAC Ozone Review Panel roster is found in
Appendix B. EPA's charge to the Panel is contained in Appendix C to this letter, and Panelists'
individual written comments are provided in Appendix D.
The SAB Staff Office has developed the consultation as a mechanism to advise EPA on
technical issues that should be considered in the development of regulations, guidelines, or
technical guidance before the Agency has taken a position. A consultation is conducted under
the normal requirements of the Federal Advisory Committee Act (FACA), as amended (5 U.S.C.,
App.), which include advance notice of the public meeting in the Federal Register.
-------
As is our customary practice, there will be no formal report from the CASAC or the SAB
as a result of this consultation, nor does the Committee expect any formal response from the
Agency.
Sincerely,
/signed/
Dr. Rogene Henderson, Chair
Clean Air Scientific Advisory Committee
Appendix A - Roster of the Clean Air Scientific Advisory Committee
Appendix B - Roster of the CASAC Ozone Matter Review Panel
Appendix C - Charge to the CASAC Ozone Review Panel
Appendix D - Comments from Individual CASAC Ozone Review Panelists
-------
Appendix A - Roster of the Clean Air Scientific Advisory Committee
U.S. Environmental Protection Agency
Science Advisory Board (SAB) Staff Office
Clean Air Scientific Advisory Committee (CASAC)
CHAIR
Dr. Rogene Henderson, Scientist Emeritus, Lovelace Respiratory Research Institute,
Albuquerque, NM
MEMBERS
Dr. Ellis Cowling, University Distinguished Professor-at-Large, North Carolina State
University, Colleges of Natural Resources and Agriculture and Life Sciences, North Carolina
State University, Raleigh, NC
Dr. James D. Crapo, Professor, Department of Medicine, National Jewish Medical and
Research Center, Denver, CO
Dr. Philip Hopke, Bayard D. Clarkson Distinguished Professor, Department of Chemical
Engineering, Clarkson University, Potsdam, NY
Dr. Frederick J. Miller, Consultant, Gary, NC
Mr. Richard L. Poirot, Environmental Analyst, Air Pollution Control Division, Department of
Environmental Conservation, Vermont Agency of Natural Resources, Waterbury, VT
Dr. Frank Speizer, Edward Kass Professor of Medicine, Channing Laboratory, Harvard
Medical School, Boston, MA
Dr. Barbara Zielinska, Research Professor, Division of Atmospheric Science, Desert Research
Institute, Reno, NV
SCIENCE ADVISORY BOARD STAFF
Mr. Fred Butterfield, CASAC Designated Federal Officer, 1200 Pennsylvania Avenue, N.W.,
Washington, DC, 20460, Phone: 202-343-9994, Fax: 202-233-0643 (butterfield.fred@epa.gov)
(Physical/Courier/FedEx Address: Fred A. Butterfield, III, EPA Science Advisory Board Staff
Office (Mail Code 1400F), Woodies Building, 1025 F Street, N.W., Room 3604, Washington,
DC 20004, Telephone: 202-343-9994)
A-l
-------
Appendix B - Roster of the CASAC Ozone Review Panel
U.S. Environmental Protection Agency
Science Advisory Board (SAB) Staff Office
Clean Air Scientific Advisory Committee (CASAC)
CASAC Ozone Review Panel
CHAIR
Dr. Rogene Henderson*, Scientist Emeritus, Lovelace Respiratory Research Institute,
Albuquerque, NM
MEMBERS
Dr. John Balmes, Professor, Department of Medicine, University of California San Francisco,
University of California - San Francisco, San Francisco, California
Dr. Ellis Cowling*, University Distinguished Professor-at-Large, North Carolina State
University, Colleges of Natural Resources and Agriculture and Life Sciences, North Carolina
State University, Raleigh, NC
Dr. James D. Crapo*, Professor, Department of Medicine, Biomedical Research and Patient
Care, National Jewish Medical and Research Center, Denver, CO
Dr. William (Jim) Gauderman, Associate Professor, Preventive Medicine, Medicine,
University of Southern California, Los Angeles, CA
Dr. Henry Gong, Professor of Medicine and Preventive Medicine, Medicine and Preventive
Medicine, Keck School of Medicine, University of Southern California, Downey, CA
Dr. Paul J. Hanson, Senior Research and Development Scientist, Environmental Sciences
Division, Oak Ridge National Laboratory (ORNL), Oak Ridge, TN
Dr. Jack Harkema, Professor, Department of Pathobiology, College of Veterinary Medicine,
Michigan State University, East Lansing, MI
Dr. Philip Hopke, Bayard D. Clarkson Distinguished Professor, Department of Chemical
Engineering, Clarkson University, Potsdam, NY
Dr. Michael T. Kleinman, Professor, Department of Community & Environmental Medicine,
University of California - Irvine, Irvine, CA
Dr. Allan Legge, President, Biosphere Solutions, Calgary, Alberta, Canada
B-l
-------
Dr. Morton Lippmann, Professor, Nelson Institute of Environmental Medicine, New York
University School of Medicine, Tuxedo, NY
Dr. Frederick J. Miller*, Consultant, Cary, NC
Dr. Maria Morandi, Assistant Professor of Environmental Science & Occupational Health,
Department of Environmental Sciences, School of Public Health, University of Texas - Houston
Health Science Center, Houston, TX
Dr. Charles Plopper, Professor, Department of Anatomy, Physiology and Cell Biology, School
of Veterinary Medicine, University of California - Davis, Davis, California
Mr. Richard L. Poirot*, Environmental Analyst, Air Pollution Control Division, Department of
Environmental Conservation, Vermont Agency of Natural Resources, Waterbury, VT
Dr. Armistead (Ted) Russell, Georgia Power Distinguished Professor of Environmental
Engineering, Environmental Engineering Group, School of Civil and Environmental
Engineering, Georgia Institute of Technology, Atlanta, GA
Dr. Elizabeth A. (Lianne) Sheppard, Research Associate Professor, Biostatistics and
Environmental & Occupational Health Sciences, Public Health and Community Medicine,
University of Washington, Seattle, WA
Dr. Frank Speizer*, Edward Kass Professor of Medicine, Channing Laboratory, Harvard
Medical School, Boston, MA
Dr. James Ultman, Professor, Chemical Engineering, Bioengineering Program, Pennsylvania
State University, University Park, PA
Dr. Sverre Vedal, Professor of Medicine, Department of Environmental and Occupational
Health Sciences, School of Public Health and Community Medicine, University of Washington,
Seattle, WA
Dr. James (Jim) Zidek, Professor, Statistics, Science, University of British Columbia,
Vancouver, BC, Canada
Dr. Barbara Zielinska*, Research Professor, Division of Atmospheric Science, Desert Research
Institute, Reno, NV
SCIENCE ADVISORY BOARD STAFF
Mr. Fred Butterfield, CASAC Designated Federal Officer
* Members of the statutory Clean Air Scientific Advisory Committee (CASAC) appointed by the EPA
Administrator
B-2
-------
Appendix C - Charge to the CASAC Ozone Review Panel
Within the main sections of the draft Environmental Assessment Plan, questions that we
ask the Panel members to focus on in their review include the following:
Overview of Planned Assessment
1. Do Panel members have any comments on the major components of the planned
environmental assessment as depicted in Figure 1?
National Air Quality Analysis
1. The importance of characterizing Ch exposure of vegetation in non-monitored areas is
described in section 3 of the draft plan. What are the Panel members' views on staffs
primary approach to create a National Ozone Exposure Surface (NOES) using interpolated
monitored data with spatial scaling from Community Multiscale Air Quality (CMAQ) model
outputs?
2. Staff plans to characterize air quality in terms of the 12-hr SUM06 and current 8-hr average
indices. Do Panel members have suggestions of other indices that the staff should consider?
Crop Exposure, Risk and Economic Benefits Analyses
1. Staff plans to use concentration-response (C-R) functions from the National Crop Loss
Assessment Network (NCLAN) to estimate crop yield losses related to Os exposures in the
U.S. What are the Panel members' views on staffs continued reliance on these C-R
functions?
2. Do Panel members have any comments on the overall approach for updating the benefits
analysis for crops, including using the Agricultural Simulation Model (AGSIM®)?
3. Staff believes it is important to compare study results obtained using the open top chamber
(OTC) exposure methodology with those obtained using the alternative "free air" exposure
methodology. Do Panel members have any comments on staffs planned approach for
comparing these two exposure methods using soybean yield loss data, as available (as
described in section 4.5)?
Tree Exposure, Risk and Economic Benefits Analysis
1. What are the Panel members' views on staffs continued use of National Health and
Environmental Effects Research Laboratory-Western Ecology Division (NHEERL-WED)
OTC C-R functions to characterize the risk of tree seedling biomass loss from Os-related
exposures in the U.S.?
C-l
-------
2. Staff is interested in assessing Ch exposure-related effects on trees beyond the seedling stage.
To accomplish this, staff is considering using the linked tree growth (TREGRO) and stand
growth (ZELIG) model system to evaluate how tree or forest growth will respond to Ch air
quality under "as is" and just meeting alternative standard scenarios (Section 5.4). Staff
plans to apply this method to ponderosa pine in the San Bernardino Mountains.
a. What are the Panel members' views on the appropriateness of using the linked TREGRO
and ZELIG modeling system to assess the impacts of Ch air quality on forest growth
under current and alternative standards?
b. What are the Panel members' views on using the USDA Forest Service's Timber
Assessment Market Model (TAMM) to quantify the economic impact of growth rate
changes, modeled by TREGRO/ZELIG, for the different air quality scenarios?
c. What are the Panel members' views on the utility of applying this model system, given
staffs plans to focus on a single species?
d. Can the Panel members suggest other approaches for quantifying the long-term impact of
Ch exposure on mature tree and/or forest growth?
3. What are the Panel members' views on the staffs approach using NHEERL-WED C-R
functions to predict aspen seedling biomass loss in the Aspen FACE study (described in
section 5.5)?
4. Staff is also interested in assessing Ch effects on vegetation in natural settings. One approach
is to make use of the visible foliar injury data within the large bio-monitoring database
maintained by the USDA Forest Service Forest Inventory and Analysis (FIA).
a. What are the Panel members' views on using this database to evaluate the degree of co-
occurrence of visible foliar injury and areas of high estimated Os exposure as indicated
by the NOES (outlined in section 5.6)?
b. Do Panel members have other suggestions on how to analyze this bio-monitoring
database or more broadly, to assess Os impacts to vegetation in natural settings?
C-2
-------
Appendix D - Comments from Individual CASAC Ozone Review Panelists
This appendix contains the preliminary and/or final written comments of the
individual members of the Clean Air Scientific Advisory Committee (CASAC) Ozone
Review Panel who submitted such comments electronically. The comments are included
here to provide both a full perspective and a range of individual views expressed by
Panel members during the consultation process. These comments do not represent the
views of the CASAC Ozone Review Panel, the CASAC, the EPA Science Advisory
Board, or the EPA itself. Panelists providing written comments are listed on the next
page, and their individual comments follow.
D-l
-------
Panelist Page#
Dr. Ellis Cowling D-3
Dr. James D. Crapo D-9
Dr. Henry Gong D-10
Dr. Paul J. Hanson D-ll
Dr. Philip K. Hopke D-16
Dr. Michael T. Kleinman D-17
Dr. Allan Legge D-18
Dr. Frederick J. Miller D-21
Dr. Maria Morandi D-22
Dr. James Ultman D-23
Dr. James (Jim) Zidek D-24
D-2
-------
Dr. Ellis Cowling
Dr. Ellis Cowling
North Carolina State University
October 20, 2005
Review of the Ozone Environmental Assessment Plan: Scope and Methods for
Exposure, Risk, and Benefit Assessment
General Comments on the Assessment Plan
One of the most interesting parts of the August 17, 2005 "Ozone Environmental
Assessment Plan" is the following statement [see indent paragraphs below] which outlines more
thoroughly than I have ever seen before, the details of how the Administrator of EPA did in fact
propose in 1997 to "replace the existing 1-hr Os secondary [welfare based] NAAQS with ... a
new seasonal standard expressed as a sum of hourly concentrations greater than or equal to 0.06
ppm, cumulated over 12 hours per day during the maximum 3-month period during the Os
monitoring season (SUM06), set at a level of 25 ppm-hr ... or alternatively ... a [secondary]
standard identical to the proposed 0.08 ppm, 8-hr primary [public health based] standard."
"In her final decision, the Administrator ... decided that it was not appropriate at that
time [emphasis by me] to establish a new separate seasonal secondary standard ... [but that]
setting the secondary standard equal to the primary standard would allow EPA the opportunity to
evaluate more specifically the improvement in rural air quality and in Os-related vegetation
effects resulting from measures designed to attain the new primary standard. This information
would allow for better evaluation of the incremental need for a distinct seasonal secondary
standard in the next review of the Os criteria and standards (62 FR 38877-78, July 18, 1997)."
Based on the scientific evidence contained in the 2005 Criteria Document for Ozone, and
the approaches outlined in the present Ozone Environmental Assessment Plan, I am even more
convinced that EPA Staff should to do all within their powers of persuasion (after further
analysis of more currently available scientific evidence as proposed in this Plan!) to
convince the Administrator of EPA that a secondary (public welfare-based) standard for
ozone that is different in form from the primary standard should be promulgated and
implemented by the Agency in 2007.
Based on a thorough review of the latest scientific information available in 1996, as
described in the 1996 Os AQCD, on vegetation effects associated with exposure to ambient
levels of Os, as well as (1) staff assessments of the policy-relevant information in the 1996 Os
AQCD and staff analyses of air quality, vegetation exposure and risk, and economic values
presented in the 1996 Os Staff Paper; (2) consideration of the degree of protection to vegetation
potentially afforded by the proposed 0.08 ppm, 8-hr primary standard; (3) CASAC advice and
recommendations; and (4) public comments, the Administrator proposed to replace the existing
1-hr Os secondary NAAQS with one of two alternative new standards: a standard identical to the
proposed 0.08 ppm, 8-hr primary standard, or alternatively, a new seasonal standard expressed as
a sum of hourly concentrations greater than or equal to 0.06 ppm, cumulated over 12 hours per
D-3
-------
day during the maximum 3-month period during the Cb monitoring season (SUM06), set at a
level of 25 ppm-hr (61 FR 65716, December 13, 1996).
In her final decision, the Administrator determined that replacing the then current
secondary standard with an 8-hr standard, set at a level of 0.08 ppm, identical in all respects to
the new primary standard, would provide adequate protection to vegetation. The Administrator
judged that this standard would provide substantially improved protection for vegetation from
Cb-related adverse effects as compared to that provided by the then current secondary standard,
while allowing time for additional research and the development of a more complete rural
monitoring network and air quality database from which to evaluate the elements of an
appropriate seasonal secondary standard. The decision not to set a cumulative seasonal
secondary standard at that time was based in large part on the Administrator's recognition that
the exposure, risk, and monetized valuation analyses presented in the proposal contain
substantial uncertainties, resulting in only rough estimates of the increased public welfare
protection likely to be afforded by each of the proposed alternative standards. In light of these
uncertainties, the Administrator decided that it was not appropriate at that time to establish a new
separate seasonal secondary standard given the potentially small incremental degree of public
welfare protection that such a standard might afford.
The Administrator further concluded that continued research on the effects of Cb on
vegetation under field conditions and on better characterizing the relationship between Cb
exposure dynamics and plant response would be important in the next review because:
• The available biological database highlighted the importance of cumulative, seasonal
exposures as a primary determinant of plant responses.
• The association between daily maximum 8-hr Cb concentrations and plant responses
had not been specifically examined in field tests.
• The impacts of attaining an 8-hr, 0.08 ppm primary standard in upwind urban areas on
rural air quality distributions could not be characterized with confidence due to limited
monitoring data and air quality modeling in rural and remote areas.
It was determined that setting the secondary standard equal to the primary standard
would allow EPA the opportunity to evaluate more specifically the improvement in rural air
quality and in Cb-related vegetation effects resulting from measures designed to attain the new
primary standard. This information would allow for better evaluation of the incremental need for
a distinct seasonal secondary standard in the next review of the Cb criteria and standards (62 FR
38877-78, July 18, 1997).
Specific Comments in Response to the Charge Questions:
The following comments are focused on the specific CASAC "Charge Questions" raised in
Karen Martin's letter to Fred Butterfield dated August 17, 2005. Please note in the paragraphs
below, that my suggested responses to Karen Martin's questions are written in indented
paragraphs, whereas the background statements by Karen Martin and her specific Charge
Questions are written in non-indented paragraphs.
Overview of Planned Assessment
1. Do Panel members have any comments on the major components of the planned
environmental assessment as depicted in Figure 1?
D-4
-------
My major reservation about the categories of risks and benefits depicted in Figure 1, is
the apparent restriction of the "categories of risks and benefits" now being planned for
assessment to yield of saleable commodities (such as Crop yield, Seedling growth, Foliar
injury, Crop loss, and Timber Loss) rather than also including the much more appropriate
and broad array of non-commodity-based "ecosystem services" provided by both natural
and managed ecosystems (such as clean water, clean air, carbon sequestration, aesthetic
values of scenic vistas — not only in wilderness areas but also in urban and suburban
areas, etc., etc.). Assessment of ozone-induced changes in "ecosystem services" would
be much more appropriate and beneficial addition to an up-to-date assessment of the need
for a secondary standard for ozone that is different in form from the present 8-hour
primary standard.
National Air Quality Analysis
1. The importance of characterizing O3 exposure of vegetation in non-monitored areas is
described in section 3 of the draft plan. What are the Panel members' views on staffs
primary approach to create a National Ozone Exposure Surface (NOES) using
interpolated monitored data with spatial scaling from Community Multiscale Air
Quality (CMAQ) model outputs?
In the absence of substantial and well-distributed sets of ozone monitors in near-urban,
rural, and remote wilderness areas in various parts of the US where most agricultural
crops, natural grasslands, and forest lands are concentrated, there appears to be no choice
but to try to infer what these exposures are likely to be by using the best available air
quality models. I agree that CMAQ is presently regarded as the best model to use in
creating a National Ozone Exposure Surface.
2. Staff plans to characterize air quality in terms of the 12-hr SUM06 and current 8-hr average
indices. Do Panel members have suggestions of other indices that the staff should consider?
Some crop and forest air pollution scientists believe that the W126 index is also worth
considering. But I believe that most ecologically focused air quality scientist believe that
a 12-hour SUM06 index would be: 1) more generally applicable to crops, forests, and
natural ecosystems, 2) much easier for air quality managers to implement, and 3) useful
in educating the general public about why this index is necessary and appropriate.
Crop Exposure, Risk and Economic Benefits Analyses
1. Staff plans to use concentration-response (C-R) functions from the National Crop Loss
Assessment Network (NCLAN) to estimate crop yield losses related to O3 exposures in the U.S.
What are the Panel members' views on staffs continued reliance on these C-R functions?
NCLAN is still the most comprehensive and thoroughly researched yield-loss assessment
data base available anywhere in the world. The re-analysis approach called for in Section
4.2 of the Assessment Plan seems very reasonable to me.
D-5
-------
2. Do Panel members have any comments on the overall approach for updating the benefits
analysis for crops, including using the Agricultural Simulation Model (AGSIM©)?
The general approach described in Section 4.3 for estimation of benefits from decreasing
ozone exposures for crops and forests appears appropriate. I have no experience with the
AGSIM model and thus can not offer a specific comment on its usefulness as a tool of
analysis.
3. Staff believes it is important to compare study results obtained using the open top chamber
(OTC) exposure methodology with those obtained using the alternative "free air" exposure
methodology. Do Panel members have any comments on staffs planned approach for
comparing these two exposure methods using soybean yield loss data, as available (as described
in section 4.5)?
My impression from the evidence presented in the Ozone Criteria Document is that most
of the largely theoretical limitations of open top chamber methods for analysis of the
impact of ozone on plants have proven to be of little importance in drawing inferences
from comparisons based on "free air" exposure methods and open top chamber studies.
Thus, I can't help but wonder if this suggested comparison of these two types of methods
is of very high priority?
Tree Exposure, Risk and Economic Benefits Analysis
1. What are the Panel members' views on staffs continued use of the National Health and
Environmental Effects Research Laboratory-Western Ecology Division (NHEERL-WED) OTC
C-R functions to characterize the risk of tree seedling biomass loss from O3-related exposures in
the U.S.?
I have had no personal experience with NHEERL-WED OTC C-R functions and thus
have no basis for providing useful comments with regard to this question.
2. Staff is interested in assessing O3 exposure-related effects on trees beyond the seedling stage.
To accomplish this, staff is considering using the linked tree growth (TREGRO) and stand
growth (ZELIG) model system to evaluate how tree or forest growth will respond to O3 air
quality under 'as is' and just meeting alternative standard scenarios (Section 5.4). Staff plans to
apply this method to ponderosa pine in the San Bernardino Mountains.
a. What are the Panel members' views on the appropriateness of using the linked TREGRO and
ZELIG modeling system to assess the impacts of O3 air quality on forest growth under current
and alternative standards?
In this case also, I have had no personal experience with the TREGRO and ZELIG
modeling systems and thus have no basis for providing useful comments with regard to
this question.
D-6
-------
b. What are the Panel members' views on using the USDA Forest Service's Timber Assessment
Market Model (TAMM) to quantify the economic impact of growth rate changes, modeled by
TREGRO/ZELIG, for the different air quality scenarios?
c. What are the Panel members' views on the utility of applying this model system, given
staffs plans to focus on a single species?
Extrapolations of findings from analyses of any set of single tree species analyses are
always subject to question. Are funds not adequate to maybe do a less-detailed analysis
but do it on more than just one tree species?
d. Can the Panel members suggest other approaches for quantifying the long-term impact of O3
exposure on mature tree and/or forest growth?
My only suggestion would be to see if there is a correlation between the National Ozone
Exposure Surface (NOES) and the US Forest Service's Forest Inventory and Analysis
(FIA) data on: 1) age-class distribution in forest stands, 2) their disease and insect
incidence data, and 3) general rate of growth data for various species of trees in different
parts of the US.
3. What are the Panel members' views on the staffs approach using NHEERL-WED C-R
functions to predict aspen seedling biomass loss in the Aspen FACE study (described in section
5.5)?
Once again, I must confess having had no experience with the NHEERL-WED C-R
functions. Thus, I have no basis for providing useful comments with regard to this
question.
4. Staff is also interested in assessing O3 effects on vegetation in natural settings. One approach
is to make use of the visible foliar injury data within the large bio-monitoring database
maintained by the USDA Forest Service Forest Inventory and Analysis (FIA).
Please note the comment in response to Question d (above).
a. What are the Panel members' views on using this database to evaluate the degree of co-
occurrence of visible foliar injury and areas of high estimated O3 exposure as indicated by the
NOES (outlined in section 5.6)?
I think these are promising possibilities and encourage further exploration of these ideas
with US Forest Service FIA personnel.
b. Do Panel members have other suggestions on how to analyze this bio-monitoring database
or more broadly, to assess O3 impacts to vegetation in natural settings?
D-7
-------
My only suggestion is to see if there are natural grasslands inventory data that could be
used in a manner similar to the possibilities suggested earlier for collaborative analysis
like those discussed earlier with the US Forest Service FIA units across the country.
D-8
-------
Dr. James D. Crapo
Consultation on Draft Ozone Environmental Assessment Plan
(Teleconference October 3, 2005)
Comments of Dr. James D. Crapo
October 5, 2005
I agree with the general approach being proposed by OAQPS under their draft Ozone
Environmental Assessment Plan. The proposal to develop a national ozone exposure surface
map for the United States should provide a useful mechanism to look at existing air quality
conditions. The limitations of this model will need to be clearly identified in terms of
extrapolations to areas where little or no monitoring data exists. Rigorous testing of the model
against existing monitoring data will be essential, particularly to understand regions where the
model does not fit observed monitoring data.
Plans to characterize air quality in terms of the 12 hour SUM06 and current 8 hour average
indices are appropriate. It will be necessary to validate the 12 hour SUM06 model and defend
why this model is better than other possibilities such as AOT06 and W126. A clear discussion of
the validation of each of the models to be selected for the analysis needs to be done. This should
validation of the data from NCLAN and NHEEL-WED programs. The limitations and
assumptions inherent in these data sets need to be carefully identified.
There is a need to consider interactions of ozone with other environmental factors, or the
independent impact of other factors, when estimating the impact of ozone on vegetation. Actual
effects are almost always overestimated when only one factor is considered. For example, is the
net impact of variations in ozone levels altered in the presence of draught conditions, etc. Again,
the assumptions inherent in and limitations of models involving crop loss, vegetation loss, and
economic benefits analysis will be an important part of the final report.
D-9
-------
Dr. Henry Gong
Consultation on Draft Ozone Environmental Assessment Plan
(Teleconference October 3, 2005)
Comments of Dr. Henry Gong, Jr.
October 10, 2005
My initial reading of the OAQPS Assessment Plan (version 8/05) left me concerned about the
many assumptions implicit in the foundation of the plan (e.g., in Figure 1) and references that
were either not yet peer-reviewed or part of previous EPA monographs (which I am admittedly
not an authority). I surmised that there were still significant knowledge gaps that are, in part,
related to the lack of specific research (or published research) and inadequate monitoring
network, and that we might be attempting to essentially fill the "round" gaps with "square"
solutions.
I can now better appreciate why I was unclear about some aspects of the document and
assessment plan. The discussion during the teleconference was enlightening for me. I not only
agree with the insightful comments by my CASAC colleagues but they also seem to confirm my
initial uncertainties. This is not to say that we cannot attempt to develop plans and models, but
we must definitely define their boundaries and acknowledge any substantial limitations of the
assumptions. If this is done, we may end up deleting certain initial models and parts of the plan
(Figure 1), as was frequently pointed out during the teleconference. We should also be able to
honestly admit that we do not currently have sufficient tools or results to go further. The final
product may be smaller in depth and breadth but it will at least be as accurate and complete as
possible.
D-10
-------
Dr. Paul J. Hanson
Draft Ozone Environmental Assessment Plan: Scope and Methods for Exposure, Risk and
Benefits Assessment
October 3, 2005
Written Comments from Paul J. Hanson
1. Do Panel members have any comments on the major components of the planned
environmental assessment as depicted in Figure 1?
The diagram in Figure 1 appears to give equal weight to crop yield, tree seeding exposure and
foliar injury data. I don't believe this is appropriate. While visible injury data are related to ozone
exposure, the connection between ozone foliar injury data and productivity measures is not
strong. How will OAQPS weigh foliar injury data against experimentally derived growth
observations?
The diagram and text suggests that Soy and Aspen FACE data will be evaluated for their
comparability to chamber-based exposure response data. This is a reasonable idea, however,
such data are not yet peer-reviewed. Are there adequate quality assured ozone concentrations
within the FACE exposure rings to support the proposed assessment?
Figure 1 provides a misleading impression that tree seedling exposure response data are used to
inform the TREGRO/ZELIG model simulations. It is my understanding that TREGRO plant
responses to ozone would be driven by the response of photosynthesis to hourly ozone uptake
rather than the empirical seedling exposure/response relationships. The TREGRO/ZELIG
simulations should be shown as parallel (not interconnected) operational pathways.
National Air Quality Analysis
1. The importance of characterizing O3 exposure of vegetation in non-monitored areas is
described in section 3 of the draft plan. What are the Panel members' views on staffs
primary approach to create a National Ozone Exposure Surface (NOES) using interpolated
monitored data with spatial scaling from Community Multiscale Air Quality (CMAQ)
model outputs?
I agree with the plan to develop an interpolated national ozone surface, but feel that it should be
done for hourly data. The national ozone surface database for 2100 should have the flexibility
to address multiple ozone exposure indices.
2. Staff plans to characterize air quality in terms of the 12-hr SUM06 and current 8-hr
average indices. Do Panel members have suggestions of other indices that the staff should
consider?
D-ll
-------
These indices are acceptable, but others might be calculated to allow alternate and independent
assessments. In any case, the exposure response indices previously used within published crop
and tree seedling concentration response studies must be reproduced from the national ozone
concentration surface to facilitate appropriate extrapolations.
Crop Exposure, Risk and Economic Benefits Analyses
1. Staff plans to use concentration-response (C-R) functions from the National Crop Loss
Assessment Network (NCLAN) to estimate crop yield losses related to O3 exposures in the
U.S. What are the Panel members' views on staffs continued reliance on these C-R
functions?
Do the crop varieties used in the NCLAN exposure-response studies of the 1970s still represent
those widely planted in the US today? As the only viable alternative data source, the current
assessment will need to use NCLAN results for their risk analysis activities. If, however, the
current varieties of plants used throughout the US differ from those studied during the NCLAN
program, the final report must clearly state that the assessment analyses could be biased. As
discussed during the consultation, continued development of crop varieties over the past 20 to 30
years since the completion of the NCLAN work may have produced less sensitive crop varieties.
Few research studies on plant response to ozone exposure have been conducted since the 1996
OAQPS staff paper review of the NAAQS for ozone. The paucity of new information results
from a lack of research support for empirical and mechanistic studies. Funding agencies must
realize that changing ambient exposures and agricultural practices over time can lead to the use
of new crop varieties. New varieties may exhibit different ozone response characteristics, and a
research program to periodically re-evaluate plant ozone responses is needed to capture the true
nature of crop ozone responses as new varieties come into general use.
2. Do Panel members have any comments on the overall approach for updating the benefits
analysis for crops, including using the Agricultural Simulation Model (AGSIM©)?
No comment.
3. Staff believes it is important to compare study results obtained using the open top
chamber (OTC) exposure methodology with those obtained using the alternative "free air"
exposure methodology. Do Panel members have any comments on staffs planned approach
for comparing these two exposure methods using soybean yield loss data, as available (as
described in section 4.5)?
This is an excellent research goal and could be included in the current assessment activity if it is
supported by adequate peer-reviewed data. Are adequate archived data on the ozone
concentrations throughout experimental FACE rings available? FACE exposure technology
depends on feedback controls referenced to target concentrations at the center of the exposure
ring. Gradients of gas concentrations exist across the FACE rings from the source to the
measurement location at the center of the ring. Plants located closer to the ozone source pipes
(around the ring edges) receive higher ozone exposures than plants further from the vent pipes.
D-12
-------
Such variation across the ring must be considered when exposure vs. growth data from FACE
studies are compared to data from the open top chamber studies with relatively well-mixed ozone
exposures.
Tree Exposure, Risk and Economic Benefits Analysis
1. What are the Panel members' views on staffs continued use of National Health and
Environmental Effects Research Laboratory-Western Ecology Division (NHEERL-WED)
OTC C-R functions to characterize the risk of tree seedling biomass loss from OS-related
exposures in the U.S.?
This is acceptable but insufficient to capture the range of responses of important species of the
United States. Other ozone response data are available (see especially Table 9-18 and 9-19 of
the first review draft of the ozone criteria document). These new research results should not be
ignored.
2. Staff is interested in assessing O3 exposure-related effects on trees beyond the seedling
stage. To accomplish this, staff is considering using the linked tree growth (TREGRO) and
stand growth (ZELIG) model system to evaluate how tree or forest growth will respond to
O3 air quality under 'as is' and just meeting alternative standard scenarios (Section 5.4).
Staff plans to apply this method to ponderosa pine in the San Bernardino Mountains.
a. What are the Panel members' views on the appropriateness of using the linked
TREGRO and ZELIG modeling system to assess the impacts of O3 air quality on
forest growth under current and alternative standards?
The TREGRO/ZELIG model structure is appropriate, but other methods are available. The
OAQPS staff should include alternate modeling methods or include a discussion within the staff
document suggesting why other methods were considered but rejected.
b. What are the Panel members' views on using the USDA Forest Service's Timber
Assessment Market Model (TAMM) to quantify the economic impact of growth
rate changes, modeled by TREGRO/ZELIG, for the different air quality
scenarios?
No comment.
c. What are the Panel members views on the utility of applying this model system,
given staffs plans to focus on a single species?
The single focus on ponderosa pine is too limiting. TREGRO has been successfully applied to
many other species (see list on page 16 of the assessment document), and the staffs' analyses
should reflect the anticipated responses for both sensitive and insensitive species for various
regions of the United States. Ponderosa pine simulations conducted for southern California air
environments are not representative of the potential responses for other areas of the country. In
particular, the staff paper should include data that reflect the mean sensitivity of trees of the
D-13
-------
eastern United States (see also Chappelk and Samuelson 1998 New Phytologist 139:91-108).
d. Can the Panel members suggest other approaches for quantifying the long-term impact
of O3 exposure on mature tree and/or forest growth?
The following papers include alternate modeling forms (PnET and TEM, respectively) with the
capacity to handle plant ozone responses:
• Felzer B, et al. (2004) Effects of ozone on net primary production and carbon
sequestration in the conterminous United States using a biogeochemistry model. Tellus
568:230-248.
• Ollinger SV, et al. (2002) Interactive effects of nitrogen deposition, tropospheric ozone,
elevated CO2 and land use history on the carbon dynamics of northern hardwood forests.
Global Change Biology 8:545-562.
These models represent other possible means of assessing plant or ecosystem responses to ozone
at regional scales. I recognize that these models are not set up to allow extrapolation of
productivity data directly to products of commercial value, but their strength would be their
application to regional analyses.
3. What are the Panel members' views on the staffs approach using NHEERL-WED C-R
functions to predict aspen seedling biomass loss in the Aspen FACE study (described in
section 5.5)?
Were NHEERL-WED C-R functions developed for the same clones being used in the Aspen
FACE study? If the same clones were not used inter-comparisons may not be appropriate.
Appropriate data for a FACE OTC comparison may be available from the following papers:
Karnosky, D. F.; Gagnon, Z. E.; Dickson, R. E.; Coleman, M. D.; Lee, E. H.; Isebrands, J. G.
(1996) Changes in growth, leaf abscission, biomass associated with seasonal tropospheric ozone
exposures of Populus tremuloides clones and seedlings. Can. J. For. Res. 26: 23-37.
Karnosky DF et al. (2005) Scaling ozone responses of forest trees to the ecosystem level in a
changing climate. Plant, Cell and Environment 28:965-981.
Aspen ozone sensitivity is specific to individual clones. OTC vs. FACE comparisons must
include the evaluation of the same Aspen clones.
4. Staff is also interested in assessing O3 effects on vegetation in natural settings. One
approach is to make use of the visible foliar injury data within the large bio-monitoring
database maintained by the USDA Forest Service Forest Inventory and Analysis (FIA).
a. What are the Panel members' views on using this database to evaluate the degree of co-
occurrence of visible foliar injury and areas of high estimated O3 exposure as indicated by
the NOES (outlined in section 5.6)?
D-14
-------
Foliar injury data might be used to evaluate the sensitivity of natural ecosystems to ambient
ozone exposure, but the ability to extrapolate such data to meaningful economic loss needs
further justification. Foliar injury is not necessarily an indication of lost productivity.
b. Do Panel members have other suggestions on how to analyze this bio-monitoring
database or more broadly, to assess O3 impacts to vegetation in natural settings?
Can long-term data on visible ozone foliar injury collected by the USFS FIA program be used to
evaluate the efficacy of CMAQ-based predictions of a national ozone surface?
D-15
-------
Dr. Philip K. Hopke
I will raise the issue of the lack of valuation of ecosystems as part of the plan. Right now they
think of trees only in terms of lumber and not valuating the ecosystem function and services. I
think this is a serious deficiency. There is an SAB committee working on methods for valuing
ecosystems and they are fairly far along. They will be holding a workshop a week after we
review the revised CD in December. I would like to press OAQPS to be much more creative in
their assessment of the benefits of controlling ozone in terms of protecting ecosystem function
and services. I urge OAQPS to contact Angela Nugent at the SAB office to start a liaison with
the ecosystem valuation committee and see if we cannot greatly improve our
ENVIRONMENTAL PROTECTION functions in the Environmental Protection Agency.
D-16
-------
Dr. Michael T. Kleinman
Comments on "Draft Environmental Assessment Plan: Scope and Methods for Exposure,
Risk and Benefits Assessment."
Michael Kleinman
Department of Community and Environmental Medicine
University of California, Irvine
Overall this plan addresses many of the key points raised by the last assessment. Below I
address some comments in response to the charge questions provided.
1. Comments on major components of plan (Fig. 1.) - Could the plan more clearly articulate
an assessment of uncertainties, at least in general terms? A more specific plan could be
helpful in any decision-making later.
2. Use of interpolated monitored data to create NOES - This seems like a valid approach.
Are there any data sets that would be amenable to use as test data to examine the efficacy
of the approach in at least a limited setting? Would the data collected in Alpine CA be
useable as test site data? The report should stress improvements of this approach to the
potential exposure surface (PES) approach used in the previous analysis.
3. Comparison of OTC to FACE - What is the scheduled availability of the SOYFACE
data? If it is not available it would be useful to include a backup set of non-US data that
could be used, along with a discussion of pros- and cons of using the non-US data.
4. The use of FIA data - Assessing Os effects on vegetation in natural settings using the FIA
database might be compared to an epidemiological study. The approach could be subject
to various confounding effects from various factors (e.g. meteorology) and co-factors
(e.g. plant diseases and insect infestations). There might be recursive influences that
should be examined in this respect. If this analysis is to be conducted, a comprehensive,
a priori, approach to identifying potential confounders and controlling for them would be
very helpful.
D-17
-------
Dr. Allan Legge
REVIEW COMMENTS: Allan H. Legge
Ozone Environmental Assessment Plan: Scope and Methods for Exposure, Risk and Benefits
Assessment (Draft, August, 2005)
Prepared by:
Office of Air Quality Planning and Standards
U.S. Environmental Protection Agency
Research Triangle Park, North Carolina
Overall Comments:
The Staff at OAQPS are to be commended for trying to put together an ozone environmental
assessment plan. Unfortunately, the 'draft' document makes it clear that there has been little
change in scientific knowledge and understanding of the effects of ozone on plants since the last
review of the ozone air quality standard. As a result, it is proposed by OAQPS that the current
'Ozone Environmental Assessment Plan' again use agricultural crop growth and yield data
obtained from the NCLAN Program (National Crop Loss Assessment Network Program) and
tree seedling biomass data obtained from the research conducted by the National Health and
Environmental Effects Research Laboratory-Western Ecology Division (NHEERL-WED) using
the open-top chamber (OTC) ozone exposure methodology. The fundamental problem with this
approach is that essentially the same scientific uncertainties which were evident in the biological
data during the last review are still evident today.
This does not mean to suggest that the experimental data produced using OTCs by the NCLAN
Program and NHEERL-WED were not good science but rather that the extrapolation of these
data and the resulting concentration-response (C-R) functions to plant response in the ambient
environment is still highly uncertain. It is also highly questionable to suggest that ozone
exposure in the ambient environment can be used as a surrogate for plant uptake. This
assumption, however, is fundamental to the proposed growing season exposure indices to be
used in the 'draft plan' such as SUM06 or AOT06. At the very least, one would need to convert
the hourly ozone concentrations at air quality monitoring trailer measurement height to plant
height (refer to Griinhage et al. [1999] "The European critical levels for ozone: improving their
usage", Environmental Pollution 105: 163-173). That being done, one is still left with the
uncertainty of assuming that this modified ozone exposure concentration can be used as a
surrogate for ozone plant uptake over the growing season.
NOTE: AOT is defined in the 'draft plan' in the footnote on the bottom of page 3 as "the areas
over the threshold". This should read "the accumulated exposure over a threshold".
OAQPS Staff does not seem to appreciate the uncertainties associated with data obtained from
OTCs. The 'draft plan' understates this uncertainty in paragraph 1 on page 3 where it is stated
that "An evaluation of whether the OTC itself might influence the Os exposure-plant response
relationship found no results to suggest a difference in plant response to Os when grown in
D-18
-------
chambered and non-chambered plots (Heagle et al., 1988)." The same Heagle et al. (1988)
reference, however, states the following on page 174 in paragraph 2 in Section 7.5 Research
Needs: " Open-top chambers cause major changes in wind velocity, light profile, air velocity
profiles, and O3 concentration-fluctuation compared to ambient. Chambers can affect plant
growth and yield. Thus there is a lingering question as to whether plant response to 63 in open-
top chambers is similar to that in commercial fields."
It is noted in the 'draft plan' that during the NCLAN Program that a total of 15 agricultural crop
species were evaluated for their growth and yield responses to experimental ozone exposures in
OTCs and that these 15 crop plant "species accounted for greater than 85% of the U.S.
agricultural acreage planted at that time." While it is recognized that every effort was made
during the NCLAN Program from 1980 - 1986 to test the most economically important and most
widely grown cultivars within a given region such as the use of nine cultivars of soybean, it is
important for OAQPS Staff to be aware that for the majority of the NCLAN crop species tested
that only one cultivar was evaluated. Further, since almost 20 years have elapsed since the
completion of the NCLAN Program research, one must question the advisability of applying C-R
functions based upon the responses of cultivars developed more than 20 years ago. It is highly
unlikely that these same crop species cultivars are in use today. Plant breeders have been
working on improving growth and yield as well as disease resistance during the intervening
years. It is also highly likely that the sensitivity of the modified cultivars to ozone uptake will
have changed over time. The use of the term 'NCLAN To carry out an assessment today, one
would need to have a more current measure of the ozone uptake responses of the most
economically important crop species and most widely grown cultivars. The problem facing
OAQPS, however, is that these data are not available. The reason these as well as other plant
response data are not available today is that the research funding which was required to obtain
these data was not made available by the EPA after the 1996 scientific review of the ozone
standard as recommended. The research funding that was available during this time period was
directed at issues related to 'particulate matter'. This was most unfortunate.
Specific Comments Selected 'Charge Questions':
Consideration should be give to the use of the available passive ozone monitoring data and the
foliar injury surveys which have been carried out by the park service.
The idea of creating a National Ozone Exposure Surface (NOES) using interpolated data has
merit. One of the important uncertainties will be the adequacy in the number of the rural/remote
air quality monitoring stations. This was a major problem during the last review of the ozone
standard. The extent to which the air quality models better address this problem must be clearly
evaluated.
There is merit in comparing the output from the OTC exposure methodology with the 'free air'
exposure methodology such as ASPEN FACE and SOY FACE. Each of these exposure
methodologies have their advantages and disadvantages and resulting uncertainties. It is vital that
these be clearly understood before any conclusions are drawn.
The use and continued reliance by OAQPS Staff on the OTC C-R functions derived from the
D-19
-------
NCLAN Program is not recommended. One important reason is that the cultivars on which the
C-R functions are based are no longer in use.
The use and continued reliance by OAQPS Staff on the OTC C-R functions derived from
NHEERL-WED research is not recommended. This work was done with tree seedlings in pots.
Further, there is still much debate in the scientific community about the responses a seedlings
and mature tree being different.
D-20
-------
Dr. Frederick J. Miller
Consultation on Draft Ozone Environmental Assessment Plan
(Teleconference October 3, 2005)
Comments of Dr. Fred J. Miller
October 4, 2005
Overall, the draft environmental assessment plan developed by OAQPS staff contains all of the
key elements. This is a situation, however, where 'the devil is in the details". In that regard,
other CASAC panel members are in a better position to provide feedback. There are, however,
some general comments that pertain to the document. These are:
• There are a number of models that will be used by OAQPS in the vegetation and tree
biomass analyses. Examining the reference list, one is left with the impression that
many of these models have not been published in the peer reviewed literature. To the
extent that they have been so published, the assessment plan should note this. Results
from peer reviewed models carry a far greater weight than those that arise from non peer
reviewed models because the assumptions, governing equations, etc. are usually spelled
out and defended in peer reviewed modeling publications. Book chapters are not
considered as peer reviewed publications. Thus, reference to the Taylor et al chapter as
representing a peer reviewed model is incorrect.
• Given staff conclude that there is little new information since the 1996 Ozone Criteria
Document review, careful attention should be placed on evaluating the desirability and
usefulness of completing all of the analyses referred to in the draft assessment plan.
Resources are limited and the analyses to be conducted should be prioritized.
• Many of the proposed analyses will utilize results from fixed site monitors that measure
ozone concentrations at a height different than the height at which a particular crop plant
is exposed. Independent of the indicator variable (SUM06, W126, AOT06, etc.), a
defensible method must be established by which the monitored values can be adjusted or
extrapolated to estimates of the ozone concentration the plants are most likely to have
seen. Without this correction, there will almost certainly always be an overestimation of
the economic benefits from further reductions in ozone levels.
D-21
-------
Dr. Maria Morandi
Consultation on Draft Ozone Environmental Assessment Plan
(Teleconference October 3, 2005)
Comments of Dr. Maria T. Morandi
October 10, 2005
My impression after reading the Draft Plan -confirmed during the teleconference discussions of
October 3 - is that there have not been sufficient additional and significant research findings to
provide appreciable reductions in uncertainties since the 1996 assessment. Consequently, and as
other members of the panel indicated, the Agency needs to consider carefully if the time and
human resources needed to undertake the work proposed in the Draft Plan are warranted by the
likelihood of very modest, if any, improvements over the last assessment.
Data appear to be insufficient across the concentration-effect continuum. Statistics can
compensate for some lack of data and less than complete understanding, only up to a point. The
overwhelming majority of available ozone concentration data centers in urban environments,
with sparse monitoring data in rural areas. It is not apparent that uncertainties in CMAQ
estimates of ozone concentrations in remote rural areas are known or can be evaluated (since
there are few monitors in such areas). The concentration indices proposed for evaluation of plant
effects have been driven by human health concerns and these metrics may or may not be
appropriate for vegetation effects. Research for relating concentration at various heights to
exposure at the point of contact with vegetation appears to be scarce and too limited for the
breath of potentially affected cultivars. The relationship between exposure (i.e., concentration at
the point of contact as defined for human exposure) and biologically-relevant uptake for the
variety of vegetation likely to be affected under the breath of environmental conditions that can
alter the uptake and/or the effect is not sufficiently known. Information on the effects themselves
at various concentrations is also less than the minimally necessary. In addition, experimental
approaches used to collect some of these data may have introduced biases so that findings cannot
be applied directly to the natural environment. In my opinion, these limitations are hard to
overcome without additional information. The outcomes from the new assessment will nor be
robust from a scientific standpoint and thus will not afford a reliable instrument for policy
makers.
At this time, engaging in this assessment appears not to be a fruitful endeavor. Instead, time and
personnel resources could be redirected to formulate a prioritized list of critical uncertainties and
data gaps, and a plan to meet those critical needs so that the next ecological assessment clearly
represents an improvement over the prior one.
D-22
-------
Dr. James Ultman
Consultation on Draft Ozone Environmental Assessment Plan
(Teleconference October 3, 2005)
Comments of James Ultman
October 11, 2005
The Draft Ozone Environmental Assessment Plan lays the groundwork and provides many of the
details for providing a sound, scientifically-based risk assessment. As I understand it, the
conceptual approach and indeed many of the specifics of the proposed assessment follow closely
from the assessment that was described in the previous staff paper in 1996. The newest aspect of
the plan appears to be the National Air Quality portion of the plan, where the CMAQ model will
be used in conjunction with spatial interpolation with the BenMAP model in order to obtain a
surface map of ozone concentrations. I commend staff on the construction of the workable
framework presented in figure 1. However, given the fact that key information, including data
as well as model validations, has not been peer-reviewed or may not be available at all, I
question the wisdom of using this framework at the present time to update the risk assessment
presented in the 1996 review.
As other panelists have also mentioned, I urge staff to use the current assessment process to
provide information on two important points that will improve future assessments of crop and
vegetation damage: 1) the uncertainties and limitations in each stage of the assessment and their
effects on the overall predictions of the process should be described and, as much as possible,
quantified; and 2) critical gaps in research should be identified.
There are two items that I find are particularly problematic about the current plan. The first is
the "quadratic rollback" method used to extrapolate "as is" to "what if air quality. This
approach is neither explained nor justified in the assessment plan. Second, the use of ozone
concentration monitored at or extrapolated to values that actually occur well above the level of
the vegetation itself. Some attempt should be made at correcting for this effect.
D-23
-------
Dr. James Zidek
Comments on "Draft Ozone Environmental Assessment Plan"
Prepared by Jim Zidek
September 29, 2005
National Air Quality Analysis
Ql.
• Data Quality. It is not clear to what extent existing ozone monitoring networks provide an
unbiased picture of the ozone field given their primary role of detecting non-compliance with
standards. Thus data from some network sites may well overestimate overall ozone
concentrations.
• CMAQ vs. the Ozone Field. CMAQ outputs predictions on a mesoscale while monitors
output measurements of ozone fields on a microscale. Thus there is a fundamental difference
between these two outputs and comparing them directly may not be meaningful. As noted by
Fuentes and Raftery (2004),
"Statistical assessment is tricky in this case, because the model predictions and the
observations do not refer to the same spatial locations, and indeed are on different spatial
scales. The fact that they are on different spatial scales is called the 'change of support'
problem."
• Comparison of the outputs have in fact been made according to the EPA's web page for
CMAQ that states:
"Initial results from the test simulations indicated that the new version of the CMAQ
model reproduced patterns of major pollutants during this time period reasonably well."
This assessment seems more optimistic than that of Fuentes and Raftery (2004) who find
CMAQ outputs for 862 at least to be both additively and multiplicatively biased relative to
CASTNET monitoring data. In fact they find the additive bias to be a polynomial function of
location co-ordinates while the constant multiplicative bias is 0.5 plus or minus 0.5. Roughly
speaking the latter means the CMAQ outputs underestimate the true point SC>2 values. For
hourly 63 concentrations I, in unpublished work with a PhD student, found similar biases in
MAQSIP model outputs relative to AIRS data, with MAQSIP tending to overestimate point
concentration values during the day and underestimating them during the night. The use of
block kriging, one of the options in BenMAP, would help to align the supports but that leads
to other technical challenges described below.
• The Plan rightly recognizes that in standards setting the spatial fields for various indices like
4th highest average, not the ozone field itself that are relevant. Moreover, it proposes (p 12) to
"characterize" monitoring and CMAQ outputs in terms of these indices. However the Plan
gives no indication of how this is to be done. In fact, some very challenging issues will arise.
D-24
-------
For one thing, intersite correlations between site pairs will likely decline, in some cases a
great deal in passing from the hourly ozone field to the index field. That could mean far
greater reliance on CMAQ outputs than would be anticipated by looking at the hourly ozone
concentration field itself.
• For another the probability distribution of these indices will be much further from Gaussian
or even the multivariate t than the hourly ozone concentration field itself (which needs to be
square root transformed to get a symmetric distribution). Yet such a distributional
assumption must be met to justify the proposed use of kriging as a linear prediction
methodology. A logarithmic transformation might well make that assumption hold at least
approximately. However new challenges then arise in block kriging since the logarithm, a
nonlinear transformation cannot be passed through the surface integral used to construct the
"block" that would make the support of the process line up with the CMAQ output.
• A further difficulty with kriging arises from its assumption of an isotropic covariance field.
This difficulty might be partially addressed by partitioning the USA into homogeneous sub-
regions and interpolating the processes separately within each region. However, some
difficulties will likely remain since things like topography can sometimes make the intersite
correlation between two distant sites larger than that between two nearby sites in the same
regions. The failure of the assumed covariance model will not likely affect point estimates so
much as it will their predictive intervals: 95% predictive intervals may well cover far less
than 95% of the predicted values. [This and other issues have been addressed in research over
the last few years but the Plan makes no reference to such work.] In any case, spatial
predictors under consideration should be "road tested" say by cross validation before they are
implemented.
• Kriging assumes the covariance to be known when the optimal spatial predictor is found.
Subsequently, that covariance is estimated to get round that assumption but the additional
uncertainty thereby introduced is ignored. That can lead to serious undercoverage of the
95% and the other predictive intervals, thereby giving a misleadingly favorable impression of
the predictor's accuracy. Methods have been proposed for correcting that deficiency and
these may be seen in Cressie (1993).
Crop Exposure, Risk and Economic Benefits Analysis
Ql. This answer to this question depends on how representative of the current population of
crops, the sampled responses collected a decade ago might be. That sample was used then to
estimate the various sub-population C-R functions. The quality of those estimates would have
depended on the representativeness of the sample. That in turn would have depended on such
things as the variability within regions and between regions. I would have thought that the
population would have evolved since then and wonder how good the C-R estimate would be for
today's population, concern about its previous quality notwithstanding. A similar comment could
be made about the relevance of the original survey of trees.
Q3. How were/are comparisons of the sampling methods made? Certainly these different
experiment approaches (FACE etc) would yield responses with different characteristics. A
D-25
-------
sufficiently large sample would lead to the rejection of the hypothesis of no difference, an unduly
small one to non-rejection. The issue that needs addressing concerns the degree of the
differences in the populations of responses that would matter for regulatory/scientific purposes.
If there are significant differences among the possible response populations, which one would
most closely resemble what would actually occur under non-experimental conditions? (This
issue also relates to Sections 4.5 and 5.5.)
References:
Cressie N (1993). Statistics for spatial data. New York: Wiley.
Fuentes, M and Raftery, AE (2005). Model evaluation and spatial interpolation by Bayesian
combination of observations with outputs from numerical models. Biometrics, 61, 36-45.
D-26
-------
NOTICE
This letter has been written as part of the activities of the U.S. Environmental
Protection Agency's (EPA) Clean Air Scientific Advisory Committee (CASAC), a
Federal advisory committee administratively located under the EPA Science Advisory
Board (SAB) Staff Office that is chartered to provide extramural scientific information
and advice to the Administrator and other officials of the EPA. The CAS AC is
structured to provide balanced, expert assessment of scientific matters related to issue
and problems facing the Agency. This letter has not been reviewed for approval by the
Agency and, hence, the contents of this report do not necessarily represent the views and
policies of the EPA, nor of other agencies in the Executive Branch of the Federal
government, nor does mention of trade names or commercial products constitute a
recommendation for use. CASAC letter and reports are posted on the SAB Web site at:
http://www.epa.gov/sab.
D-27
------- |