UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
\ WASHINGTON D.C. 20460
OFFICE OF THE ADMINISTRATOR
SCIENCE ADVISORY BOARD
October 28, 2008
EPA-CASAC-09-001
The Honorable Stephen L. Johnson
Administrator
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, N.W.
Washington, D.C. 20460
Subject: Clean Air Scientific Advisory Committee's (CASAC) Peer Review of
Draft Chapter 8 of EP A's Risk and Exposure Assessment to Support the
Review of the NO2 Primary National Ambient Air Quality Standard
Dear Administrator Johnson:
The Clean Air Scientific Advisory Committee (CASAC), augmented by subject-
matter-experts to form the CASAC Oxides of Nitrogen Primary National Ambient Air
Quality Standards (NAAQS) Review Panel (hereafter referred to as the panel, roster
provided in Enclosure A) held a public teleconference on October 22, 2008 to review
draft chapter 8 of EPA's Risk and Exposure Assessment (REA) to Support the Review of
the NO2 Primary National Ambient Air Quality Standard. Chapter 8 is entitled "Exposure
Assessment and Health Risk Characterization." Additionally, CASAC offers comments
on the schedule for NAAQS revision, prompted by the Staffs presentation of the
proposed outline for Chapter 10 of the REA.
Chapter 8 is critical to the REA because it provides a characterization of health
risk from NO2 exposure based on the estimated number of exceedances obtained from a
comprehensive personal exposure model for asthmatics in Atlanta. The CASAC panel
was generally impressed with the careful work done over a short time with available tools
and resources, but it has concerns about the implications of certain model inputs and
assumptions and about the general presentation of the approach and findings. The panel
asks for changes in two main areas: in the interpretive presentation and in
characterization of potential biases. The CASAC plans to review the completed REA at
its December 5, 2008 public teleconference and will provide recommendations for EPA's
consideration in developing the Advance Notice of Proposed Rulemaking (ANPR) for
NO2 during and immediately after that teleconference.
-------
Response to EPA Charge Questions
EPA requested CASAC's responses to four charge questions related to the
exposure assessment and health risk characterization of the draft chapter 8 provided for
review. The charge questions and responses follow immediately below.
1. To what extent is the assessment, interpretation, and presentation of the results of
the exposure analysis technically sound, clearly communicated, and appropriately
characterized?
Staff has produced a solid application of the APEX model. While recognizing
that there have been significant improvements since the first draft of the REA and the
update at the September CASAC meeting, the panel still has concerns about the
application of AERMOD. Staff should give serious consideration to improving
AERMOD model performance in comparison to the concentrations measured by the
monitors in Atlanta. For example, AERMOD output could be adjusted to better reflect
observed peak values and diurnal profiles. Many factors influence the AERMOD
predictions and the panel is concerned that the overall uncertainty, reflective of the
uncertainties from each of these factors, is not appropriately characterized. In particular,
the tails of the exposure distribution could be strongly biased, notably with respect to on-
road and near-road exposures. These exposures are particularly important, since they are
the major contributors to the benchmark exceedances. On-road exposures may be biased
high because receptors are in the middle of the road; the exhaust is not diluted much; and
observed data suggest that the on-road vs. background ratios are much more narrowly
distributed than the AERMOD predictions. Near-road exposures may similarly be
affected by biases in the on-road estimates. In addition, near-road exposures may not be
appropriately weighted to the Atlanta population because a greater proportion of residents
may live closer to major roads than is represented by the assigned locations of model
receptors at the census block centroids used in the AERMOD model.
In evaluating population exposure under alternative standards, Staff elected to roll
back health benchmark values as a time-saving substitute for rolling up ambient
concentrations. In view of the potentially strong influence of this decision on the
numerical determination of benchmark exceedances, the chapter needs to provide a
clearer justification for this approach and consideration needs to be given to validating
the procedure.
2. The draft risk and exposure assessment document evaluates exposures in Atlanta.
What are the views of the Panel on the approach taken and on the interpretation
of the results of this analysis?
The chapter needs to better evaluate the generalizability of findings for Atlanta.
In particular, three aspects of the selection of Atlanta should be addressed more carefully.
First, the selection of Atlanta should be reviewed with consideration of the extent to
which the city represents exposures nationally, since the model results are being used as
the basis for setting a national ambient air quality standard. What are typical features of
the population behaviors, building use (e.g., air conditioning), housing stock, and
residential patterns that suggest similarities and differences with other major cities,
-------
particularly with respect to anticipated high-end exposures to NC>2? As a minimum, the
panel suggests that a table be added to describe variation of key parameters across cities.
Ideally, a sensitivity analysis that assessed the importance of such parameters would also
be included. Second, is the model applied to Atlanta adequately reflecting the local
population? In particular, in assigning model receptors to census tract centroids, is the
model systematically missing the fraction of the population who lives closest to roads and
is consequently regularly exposed to high near-road exposures? The fraction of the
Atlanta population that lives within 50 meters of a major roadway could be much higher
than the 1% figure represented by the census tract centroids. We encourage EPA to seek
an independent estimate of this important quantity. Third, the application of AERMOD
to Atlanta suggests a significant potential to predict concentrations that are biased
upwards at the high end of the distribution. This bias should be more thoroughly
discussed, along with likely implications.
3. What are the views of the Panel regarding the adequacy of the assessment of
uncertainty and variability with respect to characterization of exposures and
health risks associated with those exposures?
A complete analysis of uncertainty is an essential component of a thorough
exposure and risk assessment. The panel is concerned that there is the potential for more
bias and uncertainty in the results than has been characterized in the current uncertainty
and variability assessment and that no rationale has been provided for assigning weight or
priority to identified uncertainties. A better characterization of uncertainty is needed,
with attention to prioritization of the multiple sources of uncertainty, and where possible,
quantification of their impact. Qualitative approaches that might be followed for this
purpose can be found in other reports1
In addition, previous suggestions made by individual panel members on this topic
have not been explicitly considered; the concerns raised should be addressed in the final
report. More thorough characterization of the assumptions and model features that most
strongly influence the estimated number of exceedances is needed. Most notably, the on-
road and near-road exposures are important determinants of the number of exceedances
of specified concentration levels experienced by the population. In addition to on- and
near-road characterization, and locations of receptors, assumptions about penetration into
various microenvironments, particularly transportation environments, should be
discussed. The implications of the significant biases discussed in the application of
AERMOD need to be covered in this section. Value-laden statements in Section 8.4.8
about the acceptability of AERMOD over-predictions should be relocated to Section 8.10
and discussed in the context of variability and uncertainty.
4. To what extent is the assessment, interpretation, and presentation of health risk
characterization included in Chapter 8 technically sound, clearly communicated,
and appropriately characterized?
1 See, for example, U.S. EPA. Limitations in the 199 6 National-Scale Air Toxics Assessment 1996 [cited
10/23/08. Available from http://www.epa.gov/ttn/atw/nata/natsalim2.html] and World Health Organization
International Programme on Chemical Safety. Part 1: guidance document on characterizing and
Communicating uncertainty in exposure assessment. World Health Organization In Press, [cited 10/23/08.
Available from http://www.who.int/ipcs/methods/harmonization/areas/uncertainty%20.pdf].
3
-------
The overall presentation of the chapter needs to be improved in order to help
readers to understand the broad framework of the modeling and to give a context for the
health risk characterization. In addition to expanding the overview section, several
figures and tables should be added to show the relationships among the various models
and to summarize the data inputs. Furthermore, this chapter needs to note that the health
risk characterization focuses exclusively on exposure benchmark comparisons through
additional discussion and/or cross-referencing to other sections of the REA. EPA should
add a concluding section that summarizes the primary results and implications from the
large number of models and analyses presented in the chapter.
Comments on the Schedule for NAAQS Revision
Staff provided an overview of plans for Chapter 10 of the REA, which will be
critical for considering options for the NAAQS for NC>2. The timeframe of the
teleconference permitted only a brief exchange around the general approach that will be
taken. The panel was told that time might not permit the EPA to respond to the CASAC
recommendations concerning Chapter 10 and the implications of that summary chapter
for developing the ANPR, following the CASAC public teleconference planned for
December 5, 2008. The REA and the ANPR are critical parts of the new review process
and the panel would find it unacceptable to be unable to provide advice on these
documents in an effective manner. Time for these essential CASAC review steps should
be included in any revisions to the timetable for completion of the NOX review.
This is only one example of the implications of the rapid schedule for completion
of the NAAQS revision process. EPA is producing and revising analyses and documents
at a pace that is compromising CASAC's review function and consequently the quality of
the scientific foundation for revising the NAAQS. CASAC recognizes the drivers of the
schedule for NAAQS revision but cautions that there are now warnings that CASAC's
review role, mandated by the Clean Air Act, is likely to be diminished by the compressed
schedule.
In closing, the panel noted the substantial progress in the development of this
chapter of the Risk and Exposure Assessment. We look forward to reviewing the final
version of the entire document later this year.
Sincerely,
/Signed/
Dr. Jonathan M. Samet, Chair
Clean Air Scientific Advisory Committee
cc: Marcus Peacock, Deputy Administrator
-------
NOTICE
This report has been written as part of the activities of the EPA's Clean Air
Scientific Advisory Committee (CASAC), a Federal advisory committee
independently chartered to provide extramural scientific information and advice to the
Administrator and other officials of the EPA. The CASAC provides balanced, expert
assessment of scientific matters related to issues and problems facing the Agency.
This report has not been reviewed for approval by the Agency and, hence, the
contents of this report do not necessarily represent the views and policies of the EPA,
nor of other agencies within the Executive Branch of the Federal government. In
addition, any mention of trade names or commercial products does not constitute a
recommendation for use. CASAC reports are posted on the EPA Web site at:
http://www.epa.gov/casac.
-------
Enclosure A
U.S. Environmental Protection Agency
Clean Air Scientific Advisory Committee
Oxides of Nitrogen Primary NAAQS Review Panel
CHAIR
Dr. Jonathan M. Samet, Professor and Chair of the Department of Epidemiology, Bloomberg
School of Public Health, Johns Hopkins University, Baltimore, MD
CASAC MEMBERS
Dr. Joseph Brain, Philip Drinker Professor of Environmental Physiology, Department of
Environmental Health, Harvard School of Public Health, Harvard University, Boston, MA
Dr. Ellis B. Cowling, University Distinguished Professor At-Large, Emeritus, Colleges of
Natural Resources and Agriculture and Life Sciences, North Carolina State University, Raleigh,
NC
Dr. James Crapo, Professor of Medicine, Department of Medicine, National Jewish Medical
and Research Center, Denver, CO
Dr. H. Christopher Frey, Professor, Department of Civil, Construction and Environmental
Engineering, College of Engineering, North Carolina State University, Raleigh, NC, USA
Dr. Donna Kenski, Data Analyst, Lake Michigan Air Directors Consortium, Des Plaines, IL
Dr. Armistead (Ted) Russell, Professor, Department of Civil and Environmental Engineering,
Georgia Institute of Technology, Atlanta, GA
CONSULTANTS
Professor Ed Avol, Professor, Preventive Medicine, Keck School of Medicine, University of
Southern California, Los Angeles, CA
Dr. John R. Balmes, Professor, Department of Medicine, Division of Occupational and
Environmental Medicine, University of California, San Francisco, CA
Dr. Douglas Crawford-Brown, Emeritus Professor and Director, Department of Environmental
Sciences and Engineering, University of North Carolina at Chapel Hill, Chapel Hill, NC
Dr. Terry Gordon, Professor, Environmental Medicine, NYU School of Medicine, Tuxedo, NY
Dr. Dale Hattis, Research Professor, Center for Technology, Environment, and Development,
George Perkins Marsh Institute, Clark University, Worcester, MA
-------
Dr. Rogene Henderson, Scientist Emeritus, Lovelace Respiratory Research Institute,
Albuquerque, NM
Dr. Patrick Kinney, Associate Professor, Department of Environmental Health Sciences,
Mailman School of Public Health , Columbia University, New York, NY
Dr. Steven Kleeberger, Professor, Lab Chief, Laboratory of Respiratory Biology, National
Institute of Environmental Health Sciences, National Institutes of Health, Research Triangle
Park, NC
Dr. Timothy V. Larson, Professor, Department of Civil and Environmental Engineering,
University of Washington, Seattle, WA, USA
Dr. Kent Pinkerton, Professor, Regents of the University of California, Center for Health and
the Environment, University of California, Davis, CA
Dr. Edward Postlethwait, Professor and Chair, Department of Environmental Health Sciences,
School of Public Health, University of Alabama at Birmingham, Birmingham, AL
Dr. Richard Schlesinger, Associate Dean, Department of Biology, Dyson College, Pace
University, New York, NY
Dr. Christian Seigneur, Director, Atmospheric Environment Center, Universite Paris-Est,
Champs-sur-Marne, France
Dr. Elizabeth A. (Lianne) Sheppard, Research Professor, Biostatistics and Environmental &
Occupational Health Sciences, Public Health and Community Medicine, University of
Washington, Seattle, WA
Dr. Frank Speizer, Edward Kass Professor of Medicine, Channing Laboratory, Harvard
Medical School, Boston, MA
Dr. George Thurston, Professor, Environmental Medicine, NYU School of Medicine, New
York University, Tuxedo, NY
Dr. James Ultman, Professor, Chemical Engineering, Bioengineering Program, Pennsylvania
State University, University Park, PA
Dr. Ronald Wyzga, Technical Executive, Air Quality Health and Risk, Electric Power Research
Institute, Palo Alto, CA
SCIENCE ADVISORY BOARD STAFF
Dr. Angela Nugent, Designated Federal Officer
-------
Enclosure B: Compilation of Individual Panel Member Comments on Chapter 8 of
EPA's Risk and Exposure Assessment (REA) to Support the Review of the NO2
Primary National Ambient Air Quality Standard
This enclosure contains final written comments of individual members of the Clean Air
Scientific Advisory Committee (CASAC) Oxides of Nitrogen Primary National Ambient
Air Quality Standards (NAAQS) Review Panel. The comments are included here to
provide both a full perspective and a range of individual views expressed by panel
members during the review process. These comments do not represent the views of the
CASAC or the CASAC Panel.
Comments Received:
Comments from Prof. Ed Avol 9
Comments from Dr. JohnBalmes 11
Comments from Dr. Joseph Brain 13
Comments from Dr. James Crapo 14
Comments from Dr. Douglas Crawford-Brown 15
Comments from Dr.H. Christopher Frey 19
Comments from Dr. Terry Gordon 20
Comments from Dr. Dale Hattis 22
Comments from Dr. Rogene Henderson 26
Comments from Dr. Timothy Larson 27
Comments from Dr. Kent Pinkerton 29
Comments from Dr. Armistead Russell 31
Comments from Dr. Jonathan Samet 33
Comments from Dr. Richard Schlesinger 34
Comments from Dr. Christian Seigneur 35
Comments from Dr. Elizabeth "Lianne" Sheppard 36
Comments from Dr. Frank Speizer 38
Comments from Dr. George Thurston 40
Comments from Dr. James Ultman 42
-------
Comments from Prof. Ed Avol
Comments on Second Draft NOx REA, Chapter 8
Ed Avol
General Comments:
My expertise is not in modeling, so I am not familiar with AERMOD and will have
to defer to others regarding the appropriate application of it here. I did appreciate
that there was a considerable effort undertaken to lay out the modeling approach,
and Staff should be commended for those efforts. Although a great deal of detail
was provided in the chapter, I did not feel it was properly synthesized or
summarized for the reader. A final few paragraphs asserting what it all means
would have been useful. There was considerable discussion about uncertainty,
but it, too, seemed almost philosophical in nature, rather than specific - can
anything focused to the effort at hand be said regarding the magnitude or degree
of uncertainty related to this modeling exercise?
Specific Comments:
Table of Contents, Section 8.8 title, (typographical error), check font type size
P4, Section 8.3.1, line 21 -the statement that "...Atlanta was selected as the
second-case study location..." is made, with no reference, comment, or
explanation about the first study case (Philadelphia), the reasons for going to a
second, the generalizability of either of these selections, or the selection
process...something needs to be said as to how these choices were made, even
if it is to a discussion deferred to the appendix (although the appropriateness of
this selection would seem to be an issue to be addressed in the main body of the
document).
P5, Section 8.3.3, lines 5-7 - the assertion is made that 3 years of met data
(2001-2003) are used to achieve a degree of stability in dispersion and exposure
model estimates. Given recent fluctuations in meteorological patterns, how
"stable" or "usual" are the 2001-2003 weather patterns for the area? Are they, for
example, unusual drought years, wet years, hot years, or cold years? Is it
reasonable (and credible) to use them to generalize for the air quality modeling?
P6, Section 8.3.4, lines 7 and 8 - specify units (years)
P6, Section 8.4.1, line 23 - change "are" to "were"
P6, Section 8.4.1, line 27 - typo at end of word "included"
P7, Section 8.4.2.1, Iine21 - insert "airport" in Atlanta Hartsfield name.
P8, Section 8.4.2.4, line 16 - insert "was" after "10%"
-------
P11, Table 8.2 - Comment - the HDV fractions seem high (10%-25%) for a
metropolitan area with commuters (most of whom will use LDV for
transportation)...?
P19, Section 8.4.7, line 14-15 - how does assignment of receptors to a height of
0 meters make the model more efficient (let alone realistic)?
P21, Section 8.4.8.1, lines 15-17 - does that observation that there were only
two predicted exceedances of 150ppb N02 at one monitor and no estimated
N02 concentrations above 200ppb for 1hour say something about the modeling
results, or the location of the samplers?
P29, Section 8.5, line 4 - 64,000 asthmatics/500,000 children is about 13%; is
this a high value or in conformance with other estimates for the region?
P33, Section 8.7.2.1, lines 5-18-this discussion exclusively focuses on gas
cooking as the only indoor source of N02 interest. What about un-vented or
poorly-vented room or wall heaters or fireplaces?
P37, line 8 - change "an" to "and"
P38, Section 8.9.1, lines 22-23 - This statement is not necessarily true, so it
would be better to just say the exposure results focused on asthmatics.
P39, Section 8.9.2, line 13 - change "individual" to "individuals"
P39, Section 8.9.2, lines 18-21 -this is poorly phrased and confusing, since
these are two different thoughts. Suggest breaking in to two sentences: "...with
most containing less than 30ppb." "About 5% of these receptors..."
P53, Figure 8-17 - Why are there three Figure 8-17a illustrations? How can the
first Figure 8-17a (>100ppb) be so different from the rest?
10
-------
Comments from Dr. John B alines
Characterization of Exposure and Health Risks Associated with Exposure (Chapter 8 and
Appendix B): John Balmes' comments
GENERAL COMMENTS
The agency staff members responsible for the work presented in Chapter and Appendix B
are to be commended for their hard work over a short timeframe.
1. To what extent is the assessment, interpretation, and presentation of the results of
the exposure analysis technically sound, clearly communicated, and appropriately
characterized?
I do not have modeling expertise, but I found the logic and results of the modeling efforts
conducted to be presented clearly. The discussion of assumptions and uncertainties was
also clear. A general concern that I have is that the overall modeling exercise is complex
and dependent on many sources of data with varying quality and uncertainties. Multiple
assumptions are required. Is it necessary to support the air quality standard setting with
such a complex model. Put another way, could a simpler model with less assumptions
and less data inputs be built that provides reasonable bounds to the likely population
exposures to NO2 above the benchmark values, but that is also easier for non-modelers to
understand? A specific concern that I have is that APEX apparently uses only one
temperature value per day (p. 63, sub-section 8.10.3). Given the overall complexity of
the exposure model, one would think that temperature could be better parameterized.
2. The draft risk and exposure assessment document evaluates exposures in Atlanta.
What are the views of the Panel on the approach taken and on the interpretation of
the results of this analysis?
Atlanta is a large metropolitan area that has both relatively good ambient air quality and
health effects data. Given limitations of time and staff resources, I think the decision to
focus Atlanta is reasonable. My major concern about the decision is that because of its
climate, the penetration of air conditioning is high (apparently almost 100%). This
makes modeled NO2 exposures in Atlanta less representative of such exposures in areas
with more temperate climates.
3. What are the views of the Panel regarding the adequacy of the assessment of
uncertainty and variability with respect to characterization of exposures and
health risks associated with those exposures?
The variability and uncertainty section of the chapter (section 8.10) reads well, but there
is no overall assessment of the impact of the sources of uncertainty on the exposure
estimates presented in section 8.9.
4. To what extent is the assessment, interpretation, and presentation of health risk
characterization included in Chapter 8 technically sound, clearly communicated,
11
-------
and appropriately characterized?
There is no presentation of health risk characterization. In my view, presentation of the
number of potential exposures to NO2 of the total population or persons with asthma
does not constitute sufficient health risk characterization. The impact of the estimated
exposures on the health of the total and asthmatic populations should be discussed in the
chapter.
SPECIFIC COMMENTS
p. 17, line 5 For what the abbreviation, SCC, stands should be spelled out the first time
it is used..
p. 33, line 11 American Housing Survey should be spelled out with its first use.
p. 33, lines 17-18 Should be ".. .CHAD diaries^stratified by hour of the day and
normalized to the expected value of daily food preparation events^ 1 .4 (Table 8-11).
p. 39, line 29 Persons do not "contain" exposures to ambient exposures.
p. 41, lines 6-8 Ibid.
p. 42, line 12 Should be ".. .cooking fuel as an influential variable..."
p. 42, line 13 Should be "...of whom 13 had personal..."
p. 42, line 17 Should be ".. .an exposure distribution was constructed for each
individual,..."
p. 43, line 3 Should be "1999-2000".
p. 51, line 7 Should be "...the times spent..."
p. 51, line 12 Should be "... regarding the effect of indoor sources..."
p. 51, line 13 ".. .microenvironments., however., changes..."
p. 51, line 16 Delete "a" between "to" and "comprising".
p. 56, lines 27-28 "Results for asthmatics when exposures were estimated..."
p. 66, line 15, line 26 For what the abbreviations, CMS A and CSA stand should be
spelled out the first time they are used.
12
-------
Comments from Dr. Joseph Brain
Chapter 8 and other parts of the REA document rely heavily on analyses of
Atlanta data. An essential question which deserves much more attention is the extent to
which these findings can be generalized. In terms of NOx levels and other aspects of
exposure assessment, we need to know how Atlanta compares to other population centers
throughout the United States, such as Los Angeles, Houston, or Chicago. What is the
range of NOx exposure in these other cities? Other critical parameters should also be
compared? What is the prevalence of air conditioning? What do we know about human
activity in different cities, e.g. time spent indoors vs. outdoors, and what about relevant
confounders such as temperature and humidity?
I am sensitive to the limited time available to address their issues. There may not
be time to carry out what should be done - a sensitivity analysis. To what extent do the
predictions and models change when we substitute data from other cities? One could,
however, even with the existing Atlanta model, change various parameters, and see what
happens to the outcomes. Another approach would be to create a table which lists these
parameters for selected US cities. That would reinforce the point that safety factors need
to be built in because of uncertainty about the extent to which Atlanta is typical.
13
-------
Comments from Dr. James Crapo
Comments on Chapter 8, REA N02 Document, dated October 2008
In general the EPA staff have done an excellent job in compiling a very
complex model to predict the number of individuals in Atlanta who would
be exposed to N02 levels above various potential Health Effect Benchmark
Levels. The models are appropriate and the descriptions of how the data
was used in the models, including the assumptions involved, are clearly
described. This will provide a reasonable basis on which to estimate
the population impact of conformance to various N0x NAAQS. My concerns
are:
1. The model is extremely complex with a large number of assumptions at
almost every aspect of the model. In most cases it is not known how
these assumptions may aggregate to create a significant positive or
negative bias in the results. Would a much simpler model provide
estimates of populations at risk that would be of similar - or better
final accuracy? This complex model may create a false sense of
accuracy. My point is that this model needs to be validated with
experimental data. The model should be used to predict testable
outcomes such as ER admissions for asthma, etc., which can then be
compared to actual observations. This could be done in a prospective
fashion or by applying the model to prior time periods not used to
create the model, but where outcome data is already known.
2. The overview for the chapter states that the intent is to document
the methodology and data used in the inhalation exposure assessment and
in the associated health risk characterization for N02. The chapter
only models the temporally and spatially variable N02 concentrations and
simulates the predicted human contact with those modeled N02
concentrations. The chapter needs to go on to use these simulations to
predict health risk in terms of testable outcomes - i.e.: mortality,
hospital admissions, ER admissions for asthma, exacerbation of
respiratory symptoms, etc. This is the ultimate purpose in creating
these models and will provide a means to eventually validate the utility
of the models.
14
-------
Comments from Dr. Douglas Crawford-Brown
Comments on Risk and Exposure Assessment to Support the Review of the NO2
Primary National Ambient Air Quality Standard: Chapter 8 and Appendix
Doug Crawford-Brown
April 22, 2008-04-27
These comments focus on Chapter 8 and the appendix of the Risk and Exposure Draft.
My review is organized around the four charge questions.
1. To what extent is the assessment, interpretation, and presentation of the results of
the exposure analysis technically sound, clearly communicated, and appropriately
characterized?
I was in general quite pleased with by the analysis and the way it was described both in
the text and in the appendix (which was, by the way, well written). There are a number of
important issues from the previous review that have now been addressed adequately, and
there has been an impressive amount of work done in short order to provide these
improvements. I found it easy to understand the analytic steps, and the justification for
formulating them as the staff did.
The characterization of AERMOD could use a bit more work. My sense is that it biases
results upwards, which would then bias upwards the predicted number of effects. This
aspect of the model is not adequately described, nor is it reflected in the uncertainty
section. I also was never completely clear as to how the percentiles of ambient
concentrations were being used in the Atlanta and Philadelphia examples, or whether
there even was consistency between the two case studies in regards to percentiles used.
There needs to be a concise paragraph or two explaining more clearly how the upper tails
of distributions were used and why these particular percentiles were selected. Otherwise,
it strikes me - if I were a first time reader - that there is a lot of conservatism built into
15
-------
the analysis, especially using parts of a distribution (the upper tails) where I am less
confident in the results..
The discussion of receptor locations in Section 8.4.7 could use improvement. I was never
clear in reading that section how the receptor locations were related to the APEX
modelling, which can follow people through an actual exposure field rather than through
representative locations of exposure. As I read the section, it sounded to me as if a few
receptor locations were selected as representative of ambient concentrations associated
with a given activity, and I was not sure how that fit into the APEX approach based on
census tracts. I can imagine how it is done, but the section didn't give me enough
description to understand the issue fully.
I repeat here a point I raised in previous drafts. I generally approve the proportional roll-
up or roll-down methods based on current maximum concentration at a specific site. And
I agree that the adjustment of the benchmarks produces the same result mathematically.
But it continues to make no sense scientifically, and. the savings in processing time don't
appear to me sufficient to justify a method that people will fail to understand as
mathematically equivalent.
2. The draft risk and exposure assessment document evaluates exposures in Atlanta.
What are the views of the Panel on the approach taken and on the interpretation of
the results of this analysis?
I agree with the approach taken, subject to the comments in Charge Question 1. It seemed
to me, however, that the analyses for Atlanta and Philadelphia were not quite the same,
making comparisons difficult. I particularly liked the generation of results on person-days
of effects in Atlanta, and wish the same had been done for Philadelphia.
I think the interpretation of results for Atlanta is appropriate, given the assumptions
made. However, as mentioned earlier, AERMOD appears to bias results high, or at least
above the monitoring results. This should factor into the interpretation, or at least into the
uncertainty analysis, and I didn't find that to be the case.
16
-------
I also didn't know how the staff were interpreting the results that divide effects between
categories of with or without indoor sources included. In some cases, the large majority
of effects appear to be from indoor (and hence uncontrolled) sources, while the results as
the ambient levels rise show the dominance of the ambient exposures. There should be a
better description of how the staff believe these results should be interpreted. However, I
do agree with the results themselves, again given the assumptions and models employed.
The development of the longitudinal activity sequences is a sophisticated piece of work,
and is described adequately. The one comment I have here is that at one point, the staff
had to decide whether to use correlated daily activities or uncorrelated. They used a
cluster approach, which I support. However, it was a bit difficult, reading the text alone,
to understand the cluster approach selected or why the authors believe it produces results
intermediate between the other two results. It might have been useful to present a
representative set of results under each of the three approaches and see how much
variation it produces (a kind of sensitivity analysis).
3. What are the views of the Panel regarding the adequacy of the assessment of
uncertainty and variability with respect to characterization of exposures and
health risks associated with those exposures?
The uncertainty analysis remains inadequate to produce any sort of confidence bounds on
the analytic results, although it improves with each iteration we have seen. And it now
gives the reader a sense of the contribution to variance from each component. I am not
pushing for a formal nested variability-uncertainty analysis, but it does seem to me that
something akin to what went into NATA might be appropriate, where at least the overall
uncertainty was categorized. And there remains the issue of the influence of the
conservatism of AERMOD on whether final results should be seen as simply uncertain,
or biased upwards.
17
-------
On a minor point, there is an odd part of Page 67 where the authors talk of a negative
value for the GSD and for the GM. I must be reading it incorrectly (since these negative
values are not possible), but I cannot understand these sentences.
4. To what extent is the assessment, interpretation, and presentation of health risk
characterization included in Chapter 8 technically sound, clearly communicated,
and appropriately characterized?
My comments are the same as in Charge Question 1, although I note here that the
presentation of health effects results was quite effective in this document. The analysis of
repeat effects is interesting scientifically, but the policy implications are not clear. I
would suspect that the person-days above the benchmark is more likely to inform any
final decision. Some clarification on the significance of this analysis (of multiple effects
for an individual) would be useful
I am not convinced by the argument that the daily activity patterns for asthmatics and
non-asthmatics are the same, especially on high ozone days where an alert has been
issued. There is a passing comment about a study suggesting otherwise, but I don't
believe that study examined the effect of air quality indices being published. I know RTI
were doing such a study at one time (Carol Mansfield was heading it), so perhaps the
staff might find the results of that study and see if the conclusions are changed.
I end with a comment I have made in almost all similar cases reviewed by CAS AC. The
modelling performed here is impressive and represents state-of-the-science. But I worry
that it may be too elaborate for the purposes of establishing a NAAQS. There are many,
many assumptions built into the assessment, from air modelling to activity patterns. And
all of these rest on spatial resolution of exposure fields and receptor locations. The
science is great, but I feel the current analysis is pushing the resolution of the results just
a bit. Some comments in the document mentioning this issue, and why the results still are
useful for a NAAQS determination, would be good.
18
-------
Comments from Dr.H. Christopher Frey
The following guidance document should be referenced and cited in Chapter 8:
"GUIDANCE DOCUMENT ON CHARACTERIZING AND COMMUNICATING
UNCERTAINTY IN
EXPOSURE ASSESSMENT"
http://www.who.int/ipcs/methods/harmonization/areas/exposure/en/index.html
Also,, the assumption of 10% NO2 versus total NO for source strength may be
conservative for existing mobile and stationary sources. However, for diesel vehicles
equipped with particle traps and other postcombustion controls, there is evidence that the
NO2/NOX ratio is higher than this. As such vehicles penetrate the onroad fleet, this may
have some effect on the overall ration of NO2/NOX.
I recommend that EPA consider that one of the useful outputs of this study will be a
prioritized list of key uncertainties, as discussed today. Such a list could be developed
from qualitative weight of evidence approaches, and communicated in broad categories
(e.g., low, medium, high uncertainty) or could be developed quantitatively (preferred
where possible). One benefit of this information is that it could help prioritize data
collection or research to improve the state of knowledge for future reviews and revisions
oftheNO2NAAQS.
However, my concern is that we don't want to imply that what can be done in a short
amount of time under a court-order is what SHOULD be done for the best science. The
qualitative approach is a first step, but not (in my view) sufficient. Along these lines,
EPA should acknowledge any unresolved issues, possible uncertainties that are not fully
characterized, and better methodologies for dealing with these issues, in order to guide
future work.
19
-------
Comments from Dr. Terry Gordon
1. To what extent is the assessment, interpretation, and presentation of the results of
the exposure analysis technically sound, clearly communicated, and appropriately
characterized?
Although my expertise does not allow me to evaluate whether the exposure analysis is
technically sound, the analysis was clearly communicated and characterized.
Explanations as to why certain decisions were made for the chosen methods were
particularly clear. One minor communication suggestion would be to expand the figure
legends and labeling for the figures.
2. The draft risk and exposure assessment document evaluates exposures in Atlanta.
What are the views of the Panel on the approach taken and on the interpretation of
the results of this analysis?
The approach and interpretation were very good, but there still exists the potential
problem or bias associated with using an analysis of one city in the Southeast (and
Philidelphia) to encompass a risk analysis for the entire country. In particular, it appears
that 12 urban centers were used to estimate population exposures for ozone (stated in
Section 8.2, page 1, line 26). Perhaps this will be better addressed in the final REA
chapter.
3. What are the views of the Panel regarding the adequacy of the assessment of
uncertainty and variability with respect to characterization of exposures and
health risks associated with those exposures?
The assessment of uncertainty and variability factors is very clear although a table
delineating the direction of bias that each factor might introduce would be a good
addition either in Chapter 8 or 9 (where I believe there may already be one).
4. To what extent is the assessment, interpretation, and presentation of health risk
characterization included in Chapter 8 technically sound, clearly communicated,
and appropriately characterized?
EPA did a great job under tight time constraints to finish this health risk characterization.
It is technically sound and well communicated and characterized, although a conclusion
section summarizing the relevance of the characterized exceedances/exposed asthmatics
would solidify this chapter (rather than waiting until the final chapter).
Minor comments:
1. Table 8-1 - It would help the reader to define Fringe, Rural, Arterial, Freeway,
etc. below the table.
2. Table 8-5 - Again, please define FIPS, Major and Minor Links, and % Minor.
3. page 15, line 9 - Define NEI here and in Table 8-6.
4. page 26, line 8 - typo = Icocated.
20
-------
5. page 27, Table 8-7 - Define pO to pi00. Including 'N' as the first row is
confusing as it's not really On-Road Hourly NO2 (ppb) or Number of hours
greater than xOO ppb.
6. page 29, line 2 - Given the use of 4 central counties, is it correct to look at the
total population of Atlanta?
7. page 34, line 10 - typo = 'rate and the cooking...'.
8. page 68 - Should there be a discussion of the uncertainty in applying 1 city (or 2)
to the rest of the U.S.?
21
-------
Comments from Dr. Dale Hattis
Comments on the Treatment of Uncertainty and Variability in the Revised Chapter
8 Exposure Analysis
Dale Hattis, Clark University
Over the course of reviewing the APEX and air quality
modeling for past meetings, I and others have offered
quite a few constructive suggestions for improving the
modeling, the distributional assumptions (on such
parameters as indoor air elimination rates and on
road/off-road adjustment factors) and other issues.
These include the following comments on the first draft
REA reviewed in the May 2008 meeting:
"Air exchange distributions contingent on temperature and presence or absence of
air conditioning. Overall the panel does not have any objection to the idea of
using lognormal distributions with very broad limits (.1 and 10 air changes/hr).
However the detailed results seem to show different patterns with temperature
arbitrarily blocked into a few ranges. There does not appear to be any great
consistency or overall theory for this analysis. A better description of the data as
a whole might be produced by a more extensive regression study using
temperature or some transform of temperature as a continuous variable and either
fixed-effect or mixed effects modeling of differences among cities and for the air
conditioner presence variable.
NO2 removal rate distribution—p. 101. At least one panelist expressed an
objection to the narrow fixed limits used for the removal rate distribution based
on six values from Spicer et al (1993). The abstract to the Spicer paper makes it
clear that all six observations were made in a single house, and that there are
additional complications from the presence of HONO, an apparently longer-lived
NOx species:
p. 101—The same panelist also objected to the fixed limits used for the removal
rate distribution based on six values from Spicer et al (1993). The abstract reads.
Transformations, lifetimes, and sources of NO2, HONO, and HNO3 in
indoor environments.
Spicer CW, Kenny DV, Ward GF, Billick IH.
Air Waste. 1993 Nov;43(ll): 1479-85.
Battelle, Columbus, OH 43201-2693.
Recent research has demonstrated that nitrogen oxides are transformed to
nitrogen acids in indoor environments, and that significant concentrations
22
-------
of nitrous acid are present in indoor air. The purpose of the study reported
in this paper has been to investigate the sources, chemical transformations
and lifetimes of nitrogen oxides and nitrogen acids under the conditions
existing in buildings. An unoccupied single family residence was
instrumented for monitoring of NO, NO2, NOy, HONO, HNO3, CO,
temperature, relative humidity, and air exchange rate. For some
experiments, NO2 and HONO were injected into the house to determine
their removal rates and lifetimes. Other experiments investigated the
emissions and transformations of nitrogen species from unvented natural
gas appliances. We determined that HONO is formed by both direct
emissions from combustion processes and reaction of NO2 with surfaces
present indoors. Equilibrium considerations influence the relative
contributions of these two sources to the indoor burden of HONO. We
determined that the lifetimes of trace nitrogen species varied in the order
NO approximately HONO > NO2 > HNO3. The lifetimes with respect to
reactive processes are on the order of hours for NO and HONO, about an
hour for NO2, and 30 minutes or less for HNO3. The rapid removal of
NO2 and long lifetime of HONO suggest that HONO may represent a
significant fraction of the oxidized nitrogen burden in indoor air.
The uniform distribution with its fixed boundaries (0% probability assumed for
values outside of the defined limits) is particularly inappropriate when the data are
limited, as in this case. Use of the uniform distribution artificially reduces the
likelihood of more extreme values of the modeled parameter than happen to be
present in the limited available data. This in turn limits the model-predicted
variability of NO2 concentrations, which critically determines the number of
exceedances of the high hourly NO2 levels that are the focus of the risk
assessment modeling. It would likely be far better to use a lognormal here as an
initial hypothesis, but in the light of the fact that different houses with different
internal materials might well destroy NO2 at different rates, expert judgment
might well be needed to expand the likely distribution beyond what can be
derived from a simple data fit.
The same panelist also strongly objected to the use of uniform distribution of
concentrations of NO2 from use of gas stoves (p. 101). The very breadth of the
bounds derived (4 - 188) ppb argues against a uniform distribution and in favor of
something more skewed, such as a lognormal. The lognormal guarantees a
positive contribution, and doesn't have the unfortunate property of implying zero
chance that the indoor contribution will be above the derived maximum.
Moreover, if a mass balance approach is being used to model indoor NO2, then
the input per cooking event should be in terms of mass units of NO2, not
concentration. Concentration will depend on house- and temperature specific
factors such as air exchange rates, NO2 removal rates and residual contributions
from HONO, among other things. Because these observations were from a single
house in California, there must be extra allowance for variability and uncertainty
in these estimates that must clearly extend beyond the mass equivalent of the
concentration range quoted.
23
-------
Finally the assumption that all cooking events contributing to indoor NO2 last
exactly one hour also artificially limits the variability in NO2 inputs and therefore
exposures represented in the model."
In addition to these suggestions from the last meeting,
earlier suggestions on the planning document for the
REA included explicit modeling to deal with the fact
that the air quality monitors are generally greater
than people's breathing zones, leading to a systematic
underestimation of air concentrations to which people
are exposed in the air quality characterizations.
My reading of the current chapter 8 is that the EPA
authors have completely blown off all of these
suggestions for revision of the uncertainty/variability
analysis using the APEX modeling tool and the air
quality modeling system. Not only that, the most
recent version of the REA appears to omit sections of
the previous exposure analysis draft that gave the
reader a clue that improved quantitative
characterization of the uncertainty and variability was
possible. By not acknowledging these sources of
uncertainty, and discussing the previous suggestions
for improvement it seems to me they have not just been
neglectful. It is fine if EPA wants to disagree with
my/our previous review of their analysis on technical
grounds. It is also fine if they want to say that they
cannot make the improvements in their analyses that we
suggested because of time and resource constraints.
But to completely omit discussion of the previous
critiques and suggestions for improvement of the
analysis is unacceptable and could be interpreted as
actively misleading. At the very minimum, the REA
authors need to provide quantitative documentation and
discussion of:
• The height distributions for the air quality
monitors, both nationally (as in earlier drafts of
the REA and/or ISA appendices) and specifically
for the case study cities (Philadelphia and
Atlanta),
24
-------
• The 30-odd values of the on-road/off-road ratios
that the authors used for their empirical
distribution of this key parameter,
• The six values for the indoor N02 removal rate
distribution that underlies the calculations of
indoor air concentrations.
Documentation of the individual data are needed for
these parameters in order for the committee to
adequately review the conclusions of the final REA and
the eventual comparative evaluations of alternative
standards. The "margin of safety" language of the
Clean Air Act, it seems to me, requires the
Administrator of EPA to consider in some detail the
degree of confidence that can be achieved in
conclusions that alternative NAAQS adequately protect
public health. Therefore, absent a fair analysis and
discussion of the likely quantitative effects of
uncertainty and variability of these parameters on the
estimates of population exposure under different
regulatory scenarios, I would have the committee advise
the Administrator that the current document is so
deficient that it cannot be a reasonable basis for
informed decision-making.
25
-------
Comments from Dr. Rogene Henderson
Preliminary Comments on Chapter 8 of the NOx REA
Submitted by Rogene Henderson, PhD
October 16, 2008
My comments are given from the viewpoint a non-modeler.
l.The details of the models for exposure were fully described and discussed. I think it is
important to compare the modeling values with actual measurements to see how well the
models reflect reality. This was described well in Chapter 8. I am concerned that the
diurnal comparisons showed the models overestimated NO2 exposures at the beginning
and ending of the day. Models can be useful when they are wrong because they tell us
that we are not taking something(s) into account that influences, in this case, the diurnal
variation in NO2. Has any thought been given to what that might be?
2. Using only one city as the basis for the exposure analysis has the potential for not
being representative of the total population of the US. However, from a practical
viewpoint, limited resources prevent one from analyzing more cities in depth. I think the
document describes the limitations of the approach well and I agree with it.
3.The limitations of the models for exposure are discussed in great detail.
4.1 think of "health risk characterization" as comparing what we know about the toxicity
or hazard potential of the pollutant with what we know about exposures to come up with
the potential health risk. But in this chapter I only see the exposure portion of the
characterization discussed. Can you add a discussion of the health effects you might
expect from the described exposures?
26
-------
Comments from Dr. Timothy Larson
Comments by Tim Larson on Chapter 8 of the NC>2 REA
General Comments
EPA is to be commended for the framework outlined in Chapter 8. The inclusion of
deterministic approaches to spatial extrapolation of measurements from sparse network
for purposes of exposure estimation is a welcome addition. It is an improvement on
simple interpolation schemes that ignore the spatial distribution of important sources.
My main concern with this analysis centers around the assumption that the census block
centroids describe the location of the exposed population. My reading of the chapter says
that only 1% of the exposed population is assumed to be living within 100 meters of
relatively small roadways (>15,000 vpd). If this is the underlying assumption in the final
assignment of ambient NO2 by AERMOD, then it does not reflect the reality in most
urban areas. In fact, if this assumption is what was made, the exposed population in the
Atlanta assessment could underestimated that in most urban areas by at least an order of
magnitude. While I agree that the AERMOD model probably overpredicts the on-road
exposures by up to a factor of 2, this is outweighed by the exposure location assignment
bias.
Something should also be mentioned in the uncertainty section regarding the roll-up
methods. There is an apparent correlation between the average NO2 concentrations at
EPA sites and the peak to mean ratio. In the roll-up method used by EPA, the
assumption is that these two quantities are not correlated and therefore the peak to mean
ratio remains constant during roll-up. The data suggest otherwise. Therefore the peak
values may be overpredicted for a scenario just meeting the current annual standard. This
is important when drawing conclusions about how well the current standard protects
against the exceedance of peak hourly benchmark values. The analysis presented by
Paine in the September 26 public comments suggests that the EPA approach is very
conservative. There needs to be acknowledgement of this issue and/or a rebuttal
argument.
Specific Comments
P6 line 19 Strictly speaking, AERMOD is a bi-Gaussian model
P 7 line 8 clarify wording
P23 what about smaller NOx point soujrces with short stacks? Is emission rate the only
criterion?
P25 only 1% near roadways?
Fig 8-7 The overprediction of AERMOD could be the effect of ignoring vehicle induced
turbulence as well as including too much fresh ozone. Comparison of NOx predictions
would be useful in assessing the pure dispersion estimates.
27
-------
P31 locally high values? Do you mean within 100 meters of the road?
Table 8-8 How do you get asthma prevalence rate for a 0-1 year old?
P53 line 16 should read "spend more time"
Fig 8-16 the in-transit estimates are probably generous given the assumption of pen=l.
Also, given that people spend most time in their homes, even if the pen=0.5 for homes
near roads, this could be important. In Atlanta, only 1% of the population lives within 50
m of a major road??
P55 line 2 should read figure 8-19
P61 line 13 in travel exposures may be underestimated due to ignoring the communting
route on-road levels
P63 line 24 a bit optimistic- e.g. compare with line 20 on p64
P64 line 27 the vehicle wake effect is most pronounced for wind directions parallel to the
road, where the wakes of multiple vehicles can reduced on-road concentrations more than
a simple one-time adjustment factor (e.g. 1.7) applied to all wind directions (including
cross-winds).
P66-67 discussion of between city variation in APEX results for ozone not necessarily
applicable to NO2, given the proximity to road effect for NO2.
Other comments (including material in Appendix B)
Discuss whether there any visual check on the conflation of TDM links with the actual
street map. Using an algorithm that is based on closest road to the TDM node will not
always put the link on the correct road (case of nearly parallel roads and also underpass
links).
Add a few sentences about the use of segmented lengths- discuss how much mismatch
there is with actual road locations. Could be large for curved roads and may affect both
on-road and off-road estimates from AERMOD.
Initial NO2 to NOx ratio from tailpipe emissions- include a few sentences about the
effect of catalytic converters on newer heavy duty vehicles.
Add to the discussion of vehicle induced turbulence- initial sigma z values are not
adjusted for wind path length along the roadway, an effect most pronounced for near
parallel wind directions.
Discuss the uncertainties of assuming a superimposed, steady state area source plume
model for aggregated small roadway area sources- if travel times are greater than one
hour, then the steady state assumption breaks down.
28
-------
Comments from Dr. Kent Pinkerton
COMMENTS: Kent E. Pinkerton, University of California, Davis
CHARGE QUESTION #1: To what extent is the assessment, interpretation, and
presentation of the results of the exposure analysis technically sound, clearly
communicated, and appropriately characterized?
REPLY: The authors of chapter 8 have done an incredible job to organize this chapter.
The presentation of information within the chapter is extremely thorough and carefully
described. The methodology and data used in inhalation exposure assessment and health
risk characterization for NO2 as a criteria pollutant is up to date and highly innovative in
its application as presented in Chapter 8. The estimation of temporal and spatial variation
of NO2 concentrations appears to be a highly complicated and complex process, but the
authors have done an reasonable job to provide a sound description and rationale. The
authors describe approaches taken using the Air Pollutants Exposure Model (APEX) to
estimate human population exposure, while AERMOD is used to characterize ambient air
quality. The APEX model simulates exposure using 1) characterization of the study area,
2) generation of simulated individuals, 3) construction of a sequence of activities, 4)
calculation of hourly concentrations in mcroenvironments and 5) estimation of exposure.
Each step in Chapter 8 is systematically described with potential concerns for potential
error described. It is my evaluation that the assessment, interpretation, and presentation
of the results of the exposure analysis in Chapter 8 have been done in a technically sound,
clearly communicated, and appropriately characterized fashion.
CHARGE QUESTION #2: The draft risk and exposure assessment document evaluates
exposures in Atlanta. What are the views of the Panel on the approach taken and on the
interpretation of the results of this analysis?
REPLY: Atlanta appears to be an ideal location for crafting this risk and exposure
assessment, based on the historical richness for NO2 concentrations in this region and the
availability of an excellent database of information to use in creating this model. The
inclusion of four counties covering much of urban and suburban Atlanta seems to provide
a reasonable base to model and to make predictions for risk and exposure assessment.
However, it is critical that the authors fully justify the selection of a single city or region
for making predictions for the entire nation in the assessment of NO2 for the REA for
NOx.
CHARGE QUESTION #3: What are the views of the Panel regarding the adequacy of
the assessment of uncertainty and variability with respect to characterization of exposures
and health risks associated with those exposures?
REPLY: The authors have carefully laid out the parameters used for the modeling to
estimate NO2 exposure risk to individuals. The authors acknowledge the limitations of
their model that must use assumptions, but have applied APEX which from the authors'
point of view provides the most power and flexibility to estimate exposure assessment
and risk characterization. The authors acknowledge CHAD (Consolidated Human
Activity Database) used in their model is likely to produce the greatest uncertainty. This
29
-------
concern is based on 1) the variability of human activity patterns and 2) much of the
information used is over 20 years old. The authors have attempted to correct for some of
these uncertainties, but there appears to still remain potential errors and/or factors that
may not be able to be accounted for and/or are unknown. Population and commuting
data, meterological data, and air quality data are only a few of the many parameters that
must be factored into this model. Based on these parameters as well as indoor and
outdoor factors, air exchange rates and others, it is impressive that the authors can make
any predictions at all with any degree of certainty!
CHARGE QUESTION #4: To what extent is the assessment, interpretation, and
presentation of health risk characterization included in Chapter 8 technically sound,
clearly communicated, and appropriately characterized?
REPLY: The implementation of the exposure modeling and health risk characterization
is very thorough, well documented and reasonably explained. However, concern remains
when multiple assumptions must be made in a model where minor errors or large
variations in values are possible, can easily be compounded, thus leading to a highly
misleading exposure model or health risk characterization. Again, the authors simply
need to make these limitations of the model clearly evident for the reader.
30
-------
Comments from Dr. Armi stead Russell
Review of Chapter 8:
EPA Staff and their contractor are to be complimented on the amount of work conducted,
and how quickly, to address concerns discussed at the last NOx-primary meeting. While
they have reduced some of the largest differences between the modeled and observed
NO2, some remain. Those, for the most part, can be explained by some of the
assumptions made:
1. 10%NO2 in exhaust. This could be lower, and would, in particular, lower
the early morning NO2 peak. It was chosen to be conservative. This is
fine, and should be recognized as leading to a possible bias, and the issue
observed.
2. The temporal NOx emissions pattern would show less hourly variation
than is likely, and, in particular, the smoothing used will put more
emissions before 6 am than is likely the case. I drive here. There are few
on the road before 6 am, and it is ghostish before 5 am. They would have
done better to use the hourly temporal distributions available in SMOKE.
This, alone, might get rid of the aberrant peak at 5 am.
3. They could choose greater initial mixing values, though this would be
small except for on-road estimates.
On the other hand, I would actually think the way they did the NOx emissions estimation
would be more accurate than the NEI, and the 28% is too big of an expected change (and
would go in the other direction).
There is a major bias in the AERMOD results that can not be readily dismissed, that
being that the on-road/non-road ratio gets much higher for AERMOD. There is a
consistent tendency, in all of the approaches, for AERMOD to have higher on-road than
might be indicated by the observations. This is discussed, but should be highlighted
more, particularly later on when interpreting the APEX results. As you will note, most of
the exposures to high levels come from on- and near-road activities.
The application of APEX appears solid, though the evaluation and interpretation is
flawed. While I am glad to see the evaluation performed, it is misleading. In particular,
Fig. 8-11 should show maximums, not P97.5 for comparison with Fig. 8-10. Indeed, I
suspect we will see that the maximums predicted by APEX (which drive the later
analysis) are much higher than observed. I would actually like to see a figure where they
bin the APEX results in deciles (e.g., cumulative %ile deciles), and plot the min, median,
97.5%ile and max. They also might estimate the 97.5%ile of the observations from Suh
et al., for comparison. Call Helen.
The reason all of the above is important (at least to me) is that we are compounding some
apparently large biases at the upper end of the distribution to the point that I think we are
well overestimating the population that is being exposed to greater than the benchmarks
chosen. Remember, there has been one observation of over 200 ppb of NO2 over the
three year period. What was the second high? Does that support the distribution found?
-------
In the end, I am uncomfortable with the compounding of biases for factors leading to the
upper tail of the NO2 exposure distribution. While they may have to work with this, they
should be extremely cautious about how the results are communicated.
32
-------
Comments from Dr. Jonathan Satnet
Comments: Draft Chapter 8 of EPA's Risk and Exposure Assessment to Support the
Review of the NO2 Primary National Ambient Air Quality Standard
General Comments:
The draft Chapter 8 on Exposure Assessment and Health Risk Characterization is
improved and now complete. The Staff made substantial progress in developing the
chapter and it will prove useful as the basis for health risk characterization. As far as
major concerns, I offer the following:
The document is detailed and lengthy, and does not offer a sufficiently
informative overview of the approach that would be taken. It does have an
introductory section, but the description is inadequate. I recommend the
incorporation of appropriate diagrams that would lay out the various
models that would be used and the data inputs for the modeling. This
diagram would be useful for describing what has been done and also for
describing the uncertainties that arise.
At a number of points, judgments are made with regard to what evidence
shows and what determinations can be made. However, I find that the
criteria for these determinations are lacking. For example, on page 27,
lines 7-9, the conclusion is reached that adjustment of model
concentrations to the ambient monitors was not necessary. I cannot find a
clear rationale. Similarly, on page 24, lines 1-3, "general agreement" is
said to be found between observed and modeled values. Again, what is
"general agreement?"
33
-------
Comments from Dr. Richard Schlesinger
I do not have expertise in the area of exposure assessment. However, that said, the
revised chapter did present material in a coherent and understandable manner. As I have
noted in prior types of analyses such as these, I am concerned about having one site
become the model site for all of the analyses. Thus, there should be a more
comprehensive justification for use of this one site.
34
-------
Comments from Dr. Christian Seigneur
Comments on Chapter 8 of the 2nd draft of the REA
Christian Seigneur
CERE A, Universite Paris-Est
OAQPS has accomplished a large amount of work in a short time. Their overall
methodology to calculate exposure and risk for Atlanta seems sound. However, I have
some questions regarding the results of the air quality modeling.
Section 8.4.2.4: It seems that the ozone limitation techniques were applied to AERMOD
by source category ((mobile sources, airport, etc.). Thus, if two source categories were to
impact significantly a same location, would it be possible to double-count the ozone
available for oxidizing NO to NO2?
Section 8.4.8: The report presents a thorough comparison of modeled and measured NO2
concentrations. Both the cumulative distributions and the diurnal profiles show that
AERMOD tends to overestimate the measured concentrations. Such results suggest that
some adjustment of the modeled values is warranted to better reflect the actual
concentrations. However, the document states that "it was determined that adjustment of
the modeled air quality based on the three monitors was not necessary". Such a statement
could apply if the model showed error with no or negligible bias. However, the model
shows a systematic bias (overestimation) for concentrations above the 60th percentile of
the observed values (i.e., those concentrations of most interest for health effects).
Therefore, it seems that some adjustment is warranted to scale the modeled values down
toward the values observed at the monitors.
Section 8.10.4: The discussion of the uncertainties associated with the air quality values
does not highlight the significant bias that was shown in the AERMOD evaluation
section.
35
-------
Comments from Dr. Elizabeth "Lianne" Sheppard
General comments:
I wish to complement EPA staff for completing so much work in a short time. The
APEX model for NO2 is complex and requires more complex inputs than some previous
APEX applications, particularly given the 1-hour averaging time and the need to capture
strong spatial gradients in the ambient exposure. Adequate spatio-temporal modeling of
ambient pollutants is a new focus area for the ERA, and a strong effort has been made to
apply available tools appropriately. While further improvements to the modeling and its
assessment are desirable (and possibly necessary), significant revisions since the first
draft REA have resulted in 1) better predictions and 2) a model assessment that is much
better aligned with the modeling goals.
Important issues that need additional attention before the REA is finalized:
• Philadelphia vs. Atlanta. Make sure to bring out in several places in the report (intro
to Appendix B (B-l), Appendix B case study descriptions (B-3 and B-4), intro to
Chapter 8, and as added comments in conjunction with text references to the
Philadelphia analysis (e.g. p 4, line 21)) the fact that these two modeling efforts were
sequential and the improvements in Atlanta were not applied to Philadelphia. While
it is reasonable to include the Philadelphia analysis in the appendix, the presence of
the two will mean some parties will make an effort to compare the results. Since the
modeling efforts were significantly different, this is not appropriate. Readers need to
be told this.
• AERMOD application:
o On-road model assumptions: The strong discrepancy between the data and
the model predictions plotted in Figure 8-7 suggest much more attention is
needed to this model feature, particularly since it is an important driver of
exceedances. Do the existing published data provide an adequate foundation
for this direct comparison? Is the comparison of ratios of AERMOD receptors
fair (since there may be a large amount of variability in locations of
AERMOD receptors relative to the empirical studies)?
o Census block centroids and the consequent impact on near-road exposures:
The assumption that the Atlanta population resides at census block centroids
must be evaluated more carefully since this could have a huge impact on
estimation of exceedances. Exposure to high pollution falls off rapidly in the
first 100 m from a road. The data show 1% of the simulated population
(located at block centriods) is within 50m of a road. Since census block
boundaries may not cross busy roads, the fraction of population living near
busy roads as defined by census block centroids is most likely underestimated.
For comparison, here are data from the 6 cities that are a part of the MESA air
study. The percentages of 6,014 MESA Air study subjects living within 50
meters of a busy road (census class A1-A3) are approximately 16% (Winston-
Salem), 18% (Baltimore, Los Angeles, Minneapolis-St Paul), 26% (Chicago),
and 48% (New York City).
• Discussion and interpretation: Additional text is needed to discuss the results,
particularly in terms of the large fraction of the simuated population experiencing
exceedances and the dominant locations/activities where exceedances occur (which
36
-------
appear to me to be predominantly driven by near-road and on-road exposures). The
main locations for exceedances are not only important from a policy perspective, but
should also guide the discussion of uncertainty and variability.
• Uncertainty and variability: The key questions to address with respect to uncertainty
and variability are 1) what features are most influential in modeling the tails of the
distributions, and 2) are there major sources of bias and uncertainty in these features.
Both aspects need more attention. Make sure concerns about on-road and near-road
exposures are addressed. One approach for structuring a revision may be to list every
single assumption and model input (e.g. receptor height of 0 m) and discuss its
possible contribution to bias and uncertainly. If more extensive analytical work were
to be done in support of this REA, better quantification of bias and variability is most
likely where the bulk of the attention should be focused.
Specific comments:
• Appendix B-4.14: It is good to summarize the receptor location distribution. This
should be compared to information about the Atlanta population residence
distribution.
• p 1 1 7: insert "ambient" before NO2.
• p 21 119-21: Fix wording
• p 25 1 5: Replace "the same seasonal pattern" with "similar seasonal relationships".
• p 25 1 16-19: This statement needs more analysis to support it.
• Section 8.4.8.3: The value-laden statements in this section should be moved to the
uncertainty and variability section.
• p 37 1 11 and others following: Improve notation. Why define b if b=l-a? Include
indexes, at least for individuals and time, to reflect that the predictions are across
individuals and hours.
• p 47, 52: The fraction of the population exposed to exceedances is huge and deserves
further comment (most likely in the discussion section to be added).
• p 49: The evaluation of microenvironments to reflect locations of exceedances is
valuable.
37
-------
Comments from Dr. Frank Speizer
Comments on Chapter 8, REA NO2 Document, dated October 2008
Submitted by : Frank E. Speizer, MD
Pre Conference Call submission. 10/15/08
In general I thought this chapter reads rather well. Most of my comments are minor.
Charge Question 1: The first part of chapter 8 does an excellent job of describing what
was done to make the exposure estimates. The only question would be whether it would
be worth reviewing and expanding on both the selection of Atlanta and the
reasonableness of Atlanta as a place to derive estimates for the whole country. As I recall
some of this is discussed in previous chapter and there may be no need to repeat here
except to reinforce the selection.
Page 10-11, Tables 8-1 and 8-2: It is not clear how Central Business District (CBD)
differs from Urban. May need an expanded footnote.
Charge Question 2: The logic used in developing the assessment is well laid out. In table
8.12 it might be worth constructing an addition set of figures for the 24 hour measure (or
8 hour), since in the end we will need to advise on one number and it would be good to
be able to justify how that one number differs from or covers the same at risk groupings
as other time periods. At lOOppb there appears to be little difference in the 98th vs 99th
percentiles. Only be dropping the level to SOppb does the form appear to have an effect.
Without additional calculations, it would appear that this argues strongly for a margin of
safety for safety sake rather than as an estimate of uncertainty (see below).
Page 54, Figure 8-17: the labels on the figures need to be corrected, (a-a-a, should be a-
b-c).
Charge Question 3: Although there is an excellent description of the uncertainties that
are associated with the various analyses presented, there appears to be no comment in
either direction of the magnitudes of what these uncertainties do to the estimates
provided. It seems clear that in some cases they would actually reduce the variance of
the estimates and in other cases may increase it. As indicated on the top of page 66, by
analogy with O3 may not be adequate, and therefore some sensitivity analyses with NO2
need to be performed. By example, the sensitivity test performed that generally showed
low variation helped to reinforce the findings as presented, and thus reduce the potential
importance of some of the issues mentioned.
Page 68, section 8.10.8 is useful but raises an issue that is generic to the consideration of
Atlanta. This section states that almost 100 % households are air conditioned. Thus
understanding both penetration from outdoors as well as indoor source contribution,
although understood for this region, may not be adequate for generalizing beyond the
Atlanta region, particularly to regions that are not so saturated by air conditioning.
Charge Question 4: Although the chapter does a reasonable job in characterizing the
exposure of the at risk population, it seems to fall short in estimation of outcomes from
38
-------
those exposures. For example it never goes on to provide estimates of asthma
exacerbations, or hospitalizations based on the estimates of exposure. This is particularly
troublesome in that the lowest estimated exposure result in substantial numbers of at risk
subjects. It would therefore be useful to indicate what those at risk subject will suffer
from, and the degree that true health impacts will occur from those exposures. I would
think this next step is necessary and look forward to hearing it presented in the
conference call (or seeing a few charts or tables before the conference call).
Section B-4.2.6. Supplemental Exposure Results
It appears that there is little impact on the exposure results until the Air Quality
Adjustment gets to 50 ppb for the form (98 vs 99 percentile) to make any difference.
This would suggest that estimates based on a lower bound of lOOppb are too high, and
that they should be recalculated and presented as low as 50 ppb, if not lower.
In reviewing the rest of Appendix B it is clear that there is a repeat of the material
from the first draft on Philadelphia, but because of the way that analysis was done and the
way the current analysis of Atlanta was changed, they really are not comparable. I
therefore am concerned that these two very different urban centers (eg. One warm and
almost 100% air conditioned, and the other more typical of urban Northeast) can't be
judged side by side and therefore cannot figure out how comfortable to be with just using
Atlanta. Right now it looks as if the estimates for Phili are done to a level of 200 ppb
whereas for Atlanta they go down to 50 ppb. I would like to see similar analysis of what
would be predicted exposures in Phili at the same levels seen for example in table B-50
on page B-l 14. (There may be a reason why this can't be done—as long as it is not that
there is just not enough time to do it, and if so, so be it.)
39
-------
Comments from Dr. George Thurston
Prof. George Thurston's Initial Comments for CAS AC on Chpt. 8 of the Second Draft of:
Risk and Exposure Assessment to Support the Review of the NO2 Primary National
Ambient Air Quality Standard
1. To what extent is the assessment, interpretation, and presentation of the results of the
exposure analysis technically sound, clearly communicated, and appropriately
characterized?
The EPA staff should be congratulated for their arduous effort at a formidable
task to reach the ultimate goal of the chapter: to conduct a case study of an exposure and
health risk characterization for various alternative NO2 standards. While the approach is
basically technically sound, it involved numerous challenging modeling steps, each with
their own uncertainties and possible biases that are potentially compounded from step to
step. The populations considered and the 3 receptor classifications considered in these
analyses seem very appropriate, and the modeling applied is state-of-the art. One major
concern I have with regard to the modeling, though, is the continuing overestimation of
peak values, as displayed (for example) in Figures 8-5 and 8-6. I wonder if it is the fact
that the roadway emissions are reportedly modeled here using the Ozone Limiting
Method (as noted on page 8, lines 10-12), an approach which I believe tends to give
higher (more conservative) NO2 level estimates, since it is my understanding that this
model assumes all available ozone is reacted with NO to form NO2, which will not likely
be met at peak times of the day, seemingly consistent with the results in Figure 8-6. Is
this the cause of these overestimates? If so, this assumption is fine for screening analysis,
but may not be satisfactory for this application.
2. The draft risk and exposure assessment document evaluates exposures in Atlanta.
What are the views of the Panel on the approach taken and on the interpretation of the
results of this analysis?
The choice of Atlanta seems appropriate, given the existing pollution conditions and
availability of modeling input information, but more discussion as to how these results
might vary in other cities and regions would seem appropriate. Section 8.10.7.1 needs
much further elaboration (e.g., sensitivity analysis of potentially varying conditions such
as %AC, and discussion of the potential health effects/policy implications of this
variability).
3. What are the views of the Panel regarding the adequacy of the assessment of
uncertainty and variability with respect to characterization of exposures and health risks
associated with those exposures?
To me, this is the greatest weakness of the chapter as it is now written: no
quantitative or qualitative propagation of errors by the sequential modeling steps is
provided in Section 8.10. It is my belief that such a comprehensive uncertainty analysis
would indicate large uncertainties in this approach to achieve the final estimates. A
40
-------
consideration of these uncertainties would, I think, indicate that, while a useful exercise,
such a human exposure-health effects analysis is not sufficient to alone assess the health
implications of alternative standards, and that a consideration of the results of an
application of the NO2-health effects epidemiology are also required in each such REA
(as is the case for this particular document) to obtain a fuller appreciation of the health
implications of various alternative standards.
4. To what extent is the assessment, interpretation, and presentation of health risk
characterization included in Chapter 8 technically sound, clearly communicated, and
appropriately characterized?
Very little is provided in the way of an interpretation of the results (e.g., putting
the numbers into a context useful for regulatory assessment). I suggest an additional
Section 8.11 to provide just such information.
41
-------
Comments from Dr. James Ulttnan
Comments on Second Draft NOx REA, Chapter 8
James Ultman
General Comments
I commend staff on completing an extensive body of work in such a short time. The
document is clearly written. It conveys the methodology (including the description of
computer models, sources of input data and assumptions) in as detailed a manner as is
possible within the confines of a single chapter. The results are also summarized in a
logical fashion.
I do, however, feel somewhat overwhelmed by the complexity that emerges from the
layering of different models, input data sets and accompanying assumptions that I feel are
not always justified or validated. Moreover, lacking an integrated summary at the end of
the chapter, it is not clear what key conclusions should be drawn.
Specific Comments
I am skeptical that rolling back the health benchmark values in the exposure analysis is
equivalent to rolling up of ambient concentrations (and visa versa). The document does
not present clear evidence—either numerical or mathematical—that this is the case.
Moreover, the equation that is used to justify a parallel roll-back in indoor source
concentration (pg 38, line 5) fails to recognize that Cind00r is determined, in part, by
Cambient- In particular, if Cambient is rolled up by a factor f (pg. 38, line 3), Cind00r must also
increase. Thus, its original value cannot be used to compute Cthresh (see equation on pg.
38, line 3). Moreover, the relationship between Cind00r and Cambient is algebraically non-
linear when the mass balance model is used to determine Cind00r.
Given the importance of on-road exposures, I am also concerned with the discrepancy
between the AERMOD-based and data-based determination of the on-road/non-road
concentration ratio. On the other hand, this is not surprising given that the data was
measured in a number of locales where meteorology and emission patterns are different
than in Atlanta.
Finally, I believe that the document provides an extensive discussion of areas of
uncertainty, but it does not attempt to quantify the uncertainty. In retrospect, I feel that it
would have been wiser to spend less time in the modeling/simulation effort and more
time in analyzing those uncertainties that are judged to be the most important.
42
------- |