UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON D.C. 20460
OFFICE OF THE ADMINISTRATOR
SCIENCE ADVISORY BOARD
July 14, 2009
EPA-CASAC-09-012
The Honorable Lisa P. Jackson
Administrator
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, N.W.
Washington, D.C. 20460
Subject: Consultation on EPA's Carbon Monoxide National Ambient Air Quality
Standards: Scope and Methods Plan for Health Risk and Exposure Assessment
Dear Administrator Jackson:
The Clean Air Scientific Advisory Committee (CASAC) Carbon Monoxide (CO) Review Panel
(see Enclosure A) met on May 13, 2009, to conduct a consultation on EPA's Carbon Monoxide National
Ambient Air Quality Standards: Scope and Methods Plan for Health Risk and Exposure Assessment
(April 2009). The CASAC uses consultation as a mechanism for individual technical experts to provide
comments to guide the Agency on issues early in the development of a document, before the first draft is
ready for peer review. Written comments provided by the individual members in response to the
Agency's charge questions are provided in Enclosure B. In general, the written comments focused on
exposure characterization. Given that EPA has not yet fully assessed the health issues we did not have
many comments to offer on health effects for risk assessment. A consultation is conducted under the
normal requirements of the Federal Advisory Committee Act, which include advance notice of the public
meeting in the Federal Register. No request for public participation was received.
As this is a consultation, we do not expect a formal response from the Agency. We thank the
Agency for the opportunity to provide advice early in the NAAQS review process, and look forward to
the review of EPA's First Draft Risk and Exposure Assessment on CO.
Sincerely,
/Signed/
Dr. Joseph Brain
Clean Air Scientific Advisory Committee
Carbon Monoxide Review Panel
Enclosures
-------
NOTICE
This report has been written as part of the activities of the EPA's Clean Air Scientific Advisory
Committee (CASAC), a federal advisory committee independently chartered to provide
extramural scientific information and advice to the Administrator and other officials of the EPA.
CASAC provides balanced, expert assessment of scientific matters related to issues and
problems facing the Agency. This report has not been reviewed for approval by the Agency and,
hence, the contents of this report do not necessarily represent the views and policies of the EPA,
nor of other agencies within the Executive Branch of the federal government. In addition, any
mention of trade names of commercial products does not constitute a recommendation for use.
CASAC reports are posted on the EPA website at http://www.epa.gov/CASAC.
-------
Enclosure A
ROSTER
U.S. Environmental Agency
Clean Air Scientific Advisory Committee
Carbon Monoxide Review Panel
CASAC MEMBERS
Dr. Joseph D. Brain, (Chair) Cecil K. and Philip Drinker Professor of Environmental
Physiology, Department of Environmental Health, Harvard School of Public Health, Harvard
University, Boston, MA
Dr. H. Christopher Frey, Professor, Department of Civil, Construction and Environmental
Engineering, College of Engineering, North Carolina State University, Raleigh, NC
Dr. Armistead (Ted) Russell, Professor, Department of Civil and Environmental Engineering,
Georgia Institute of Technology, Atlanta, GA
PANEL MEMBERS
Dr. Thomas Dahms, Professor and Director, Anesthesiology Research, School of Medicine, St.
Louis University, St. Louis, MO
Dr. Russell R. Dickerson, Professor and Chair, Department of Meteorology, The University of
Maryland, College Park, MD
Dr. Laurence Fechter, Senior Career Research Scientist, Department of Veterans Affairs ,
Research Service (151), Loma Linda VA Medical Center, Loma Linda , CA
Dr. Milan Hazucha, Professor, Department of Medicine, Center for Environmental Medicine,
Asthma and Lung Biology, University of North Carolina - Chapel Hill, Chapel Hill, NC
Dr. Michael T. Kleinman, Professor, Department of Medicine, Division of Occupational and
Environmental Medicine, University of California, Irvine, Irvine, CA
Dr. Arthur Penn, Professor LSU School of Veterinary Medicine, Department of Comparative
Biomedical Sciences, LSU SVM - Room 2425, Louisiana State University, Baton Rouge, LA
Dr. Beate Ritz, Associate Professor, Epidemiology, School of Public Health, University of
California at Los Angeles, Los Angeles, CA
Dr. Paul Roberts, Executive Vice President, Sonoma Technology, Inc., Petaluma, CA
Dr. Stephen R. Thorn, Professor, Institute for Environmental Medicine, 1 John Morgan
Building, University of Pennsylvania, Philadelphia, PA
-------
SCIENCE ADVISORY BOARD STAFF
Dr. Ellen Rubin, Designated Federal Officer, 1200 Pennsylvania Avenue, NW, Washington,
DC, Phone: 202-343-9975, Fax: 202-233-0643, (rubin.ellen@epa.gov)
-------
Enclosure B
Compendium of Comments
CASAC Carbon Monoxide Review Panel on
CO NAAQS Scope and Methods Plan for Health Risk and Exposure
Assessment (April 2009)
Dr. Milan Hazucha 6
Dr. Michael Kleinman 7
Dr. Stephen R. Thorn, 8
Dr. H. Christopher Frey 9
Dr. Russell R. Dickerson 17
Dr. Paul T. Roberts 18
Dr. Armistead (Ted) Russell 20
-------
Dr. Milan Hazucha
This chapter outlines carefully prepared and reasoned approach to Health Effects and Risk
Characterization. The supporting evidence will be based on controlled human exposure studies
(unfortunately almost all of them decades old), epidemiologic, and toxicologic studies. The
toxicologic studies should be limited to exposure under SOOppm CO.
The outlined plan for "integrative synthesis" seems to depend heavily on 1991 and 2000 ACQD
evidence and conclusions (p. 8). As for the integration of new epidemiologic studies published
since 2000 the Staff paper acknowledges that "it is difficult to determine from this groups of
studies the extent to which CO is independently associated with cardiovascular disease outcomes
or if CO is a marker for the effects of another traffic-related pollutant or mix of pollutants" and
that "this complicates the effort to disentangle specific CO-related health effects". Indeed, these
concerns raise the key questions that have to be specifically answered before any
recommendations and conclusions can be made (see Fig. 5-5, page 5-43 ISA 2009). The Staff
nevertheless concludes that a robust CO association in co-pollutant models and the endpoint
coherency with evidence from human studies "support a direct effect of short-term CO exposure
on cardiovascular morbidity at ambient concentrations below the current NAAQS level. "
However, the robust association with co-pollutants and the coherence with experimental
evidence do not necessarily provide supporting evidence for the above statement particularly
when many target physiologic and clinical endpoints for CO and co-pollutant may be the same.
Particularly the interpretation of health effects of mixed pollutants, e.g., CO and PMi0 should be
an integrated assessment based on evaluation and conclusions reached in respective ISA
documents (CO and PM) so that the outcome of the same study reached in PM ISA is not
interpreted differently in CO ISA. For the key studies discussed in CO ISA I suggest that the
reference is made to the PM ISA section where the same study is discussed and the key
conclusions are included in the CO ISA.
The Risk Characterization section (2.2) appropriately suggests more cautious approach in studies
interpretation and notes that currently we do not have enough data to "conduct a quantitative risk
assessment for this health endpoint", i.e., the quantitative dose-response relationship. Therefore,
the risk characterization will be based solely on the controlled human exposure literature with the
proposed benchmark COHb levels at 2.0, 2.5 and 3.0%. However, the conditions under which
these levels will be reached are not given (at rest, exercise, static CO levels, etc. ?). As
subsequently stated the calculation of dose will be based on the "well-established" CFK equation
from 1965.1 am almost sure that it is not the original CFKE (1965) but an enhanced model
which is being utilized. This should be properly referenced here and in other sections as well
when the applied model is cited, e.g., p. 15. Some multicompartment models that claim to be
even more precise particularly under dynamic conditions of exposure should be considered as
well.
The Staff has appropriately expressed a number of concerns regarding Risk Characterization for
cardiovascular effects in epidemiologic studies. As suggested the issues of concern need to be
discussed a depth at this or subsequent meetings.
-------
Dr. Michael Kleinman
General Comments:
The plan is well written and comprehensive.
Specific Comments:
Pg. 7. Although it is true that most healthy individuals can physiologically compensate for CO-
induced reductions in tissue oxygen (O2) levels (e.g. through increased blood flow, blood vessel
dilation), some susceptible individuals may not be able to benefit from compensatory responses
because they have arteriosclerosis or impaired cardiac output (either from vascular or cardiac
tissue inflammation or damage).
Pg 9. Would it be appropriate to use meta-analytical methods to establish exposure-response
relationships for the endpoints used in the controlled human exposure studies?
Pg 10. It would be appropriate to show the results of the zero CO data for these studies (i.e.
extend the X-Axis).
Pg 11. It would be appropriate to draw a distinction between precision and accuracy with respect
to the CO-oximeter responses. The word "variable" is used in this document with multiple
meanings.
Pg 18. Does the dose algorithm not take breathing rate into consideration? The diagram shows
that parameter as a separate unincluded track.
Pg 19. In discussion of the probabilistic sampling approach, has the degree to which the
precision of estimate will depend on the number of individuals being simulated been
determined? This could be considered to be in the nature of a power calculation.
Pg 23. The terms in the column labeled "Method" need to be better defined.
Pg 25 and 26. Rewrite the equation, "d" appears to be used for 2 parameters; a monitor index
and "district d".
Pg 26. Some explanation of the normalized variable Lmd and its SD would be helpful.
Pg 27. The number of significant figures in Table 3-2 could be reduced. Is the value for Vehicle
exposure realistic, given the much higher values cited in the ISA for measured in vehicle
exposures.
Pg 28. "Uncertainty" is used in this document and the ISA with different connotations. It would
be better to define uncertainty in terms of precision, accuracy, bias etc.
Pg 29. Is the commute distance is incorporated into the census tract data?
Pg 30. What is a 'tornado' graph?
Pg 31. The discussion of sensitivity could be improved by including some concrete examples.
-------
Dr. Stephen R. Thorn,
We were asked to submit written comments on the Carbon monoxide national ambient air
quality standards: Scope and Methods Plan for Health Risk and Exposure Assessment. My
comments are merely a recapitulation of ideas I stated at the CASAC meeting in Chapel Hill.
The reasoning and rationale for the Health Effects and Approach to Risk Characterization plans
are quite clear. It is also understandable why the plan uses carboxyhemoglobin (COHb) as the
biomarkers for human exposure. I think a rationale justification for using COHb is that it is the
only quantifiable assessment of environmental exposure currently available, rather than stating
that the health effect of greatest concern from CO exposure is hypoxia caused by elevated COHb
(opening line of section 2.1). Clearly, the COHb levels shown to hasten onset of angina in at-risk
individuals cannot be explained by mere hypoxia. That is, I suggest the EPA might consider
COHb as a surrogate for magnitude of CO exposure and leave the question of CO
pathophysiology to further study.
Responses to questions pertaining to chapter 2:
1 a) Overall planned approach is reasonable and valid.
Ib) The range of potential health effect benchmark COHb levels should include 1.5 % COHb
based on the multicenter study of Allred, et al.
2a) It is reasonable, based on current data, to use CO as a surrogate for multi-component air
pollution exposure. It is not scientifically valid to interpret health risks identified as a
quantitative assessment solely for CO. That is, pathophysiological effects are likely to be due to
the combined presence of agents and may not be reproducible with exposures to any single
agent.
2b) Results from co-pollutant models provide a qualitative - not quantitative - assessment of CO
risk.
-------
Dr. H. Christopher Frey
The comments provided here focus on Chapter 3 and the charge questions related to Chapter 3.
Chapter 3 - Scope and Approach for Population Exposure/Dose Analysis
1. We plan to build upon the basic structure and design of the exposure assessment
conducted in the previous review. Since that time there have been major
improvements in the exposure model and in the data for input to the model. Are the
Panel members aware of information sources that would help inform further
improvements that would be worth considering in the current review?
The 1st draft of the ISA provides a reasonable overview of the state-of-science pertaining to
exposure assessment, particularly in the context of CO. The use of APEX as a basis for
estimating exposure is reasonable. The document appropriately identifies variability,
uncertainty, and limitations associated with the general approach.
The comments here for Charge Question 1 pertain to material contained in the scope and
methods document. During the discussion at the May 13, 2009 CASAC CO Panel meeting to
review this document, EPA indicated that they would not be using the approach indicated in
Equation (1) of page 25. EPA staff did not provide detail on the approach that will be used;
therefore, it is not possible at this time to provide comment on the scope and method of the
proposed approach. Nonetheless, I choose to keep the comments made prior to the meeting just
to illustrate the kinds of issues that might need to be addressed for other modeling approaches.
P 25. What is the validity or evaluation of Equation (1)? The reference cited, Johnson et al.
(2000) is available in a 1999 draft word perfect document at
http://www.epa.gov/ttn/caaa/tl/meta/m2154.html. I could not readily locate the final version. It
appears that Section 2.4.1 of this document presents an approach that is similar to Equation (1)
but not identical. In particular, I am looking for an explanation of how and why the factors M, L,
and T were chosen and why C
-------
Why is the time of day multiplier the same for all microenvironments and districts? Wouldn't
the temporal profile be different for near-roadway versus a location distant from a roadway?
The statement in the middle of Page 26 that comparisons were made with hourly CO
concentrations measured simultaneously at the nearest fixed-site monitor raise some questions.
Fixed site monitors are located for different purposes, as described in the ISA, with some
intended to represent near-roadway conditions and others to be more representative of area-wide
conditions. This would imply the need to stratify the analysis with respect to different purposes
or objectives of fixed monitors.
How is Equation (1) compatible with the latest available information regarding near-roadway CO
concentrations that are influenced by local traffic? How does the performance of Equation (1)
for near roadway applications compare to other approaches, such as estimating incremental local
near-roadway concentrations using a model such as CALINE4 or AERMOD (using emission
factors either from Mobile6 or Draft MOVES 2009)?
2. One of the main issues in this analysis is how to estimate ambient CO concentrations
on and near roadways, which can be significant contributors to ambient CO
exposures. The relationship between CO levels measured at ambient fixed site
monitors is highly variable due to the spatial and temporal variability of on- and
near-roadway CO concentrations. In the previous review, proximity factors were
used to adjust the concentrations measured at monitors to estimate roadway-related
concentrations of CO. We plan to conduct a review of the literature and draw upon
the results of near-road studies to update the proximity factor distributions. Do the
Panel members have recommendations for improvements or alternatives to this
approach?
There is typically very little correlation between the ambient concentration measured at a fixed
site monitor and the concentration of the same pollutant if measured immediately outside of a
vehicle. The latter is a more useful basis for estimating in-vehicle concentration. Thus, the use
of a methodology that attempts to estimate concentration immediately outside of a vehicle is
worthy of serious consideration.
The recent literature indicates that a mass balance approach to modeling in-vehicle concentration
merits consideration. There is significant variability in in-vehicle concentration attributable to
factors such as: (a) status of windows (open, partially open, closed); (b) status of the heating,
ventilation, and air conditioning (HVAC) system (on, off, recirculation mode, outside air intake
mode); (c) vehicle speed; (d) in-vehicle emission sources (e.g., smoking); (e) filter efficiency; (f)
deposition; and (g) concentration of the pollutant immediately outside of the vehicle. Factors
such as (a) through (c) influence the air exchange rate.
The concentration of a pollutant immediately outside of a vehicle that is operating on a roadway
can be conceptualized as having at least two main components: (1) an area-wide concentration
such as obtained from a community-based monitor; and (2) an incremental increase in
concentration when comparing on-road versus area-wide concentration. The latter can be
conceptualized as influenced solely by the local contribution of vehicle emissions and their
dispersion over the roadway. However, there is not as yet an accepted methodology for
predicting onroad concentrations. Existing modeling tools that are used to estimate near-
roadway concentrations, such as CALINE4, model a mixing zone over the road way that extends
several meters to either side of the road. In practice, CALINE4 is typically not used for
receptors any closer than several meters from the edge of the roadway. AERMOD could also be
10
-------
used to estimate near-roadway concentrations. The estimation of the incremental concentration
attributable to traffic flow would be sensitive to factors such as: (a) roadway geometry (number
of lanes, road width); (b) vehicle emissions; (c) atmospheric stability; (d) wind speed and
direction; and (e) terrain or features near the roadway, such as sound barriers or vegetation.
Vehicle emissions depend on factors such as: (a) traffic volume; (b) vehicle fleet distribution
among vehicle sizes and fuels; (c) vehicle speeds and accelerations; (d) potential existence of
cold start (depending on proximity to trip origins) or fuel enrichment effects; (e) road grade; and
(f) ambient temperature.
It is tempting to conclude that given so many sources of variability it is not worth trying to
implement an approach for estimating in-vehicle concentration using vehicle emissions and near
roadway dispersion models. However, not all factors are equally important, and some factors are
likely to emerge as the most important ones that should be the focus of attempts to develop
accurate characterizations. Consideration should be given to whether the use of an improved
methodology would lead to more accurate estimates, even if random components of variability in
the estimates are not perfectly captured. Alternatively, consideration should be give to whether
there is a more reliable theoretical and empirical basis for a more robust fundamentals-based
approach than the use of empirical factors that may be inapplicable to situations other than for
the calibration data from which they were derived.
As an example, we have recently been exploring a methodology for estimating in-vehicle
exposure to PM2.5. This methodology is described in a paper that has been accepted for
presentation at the Annual Meeting of the Air & Waste Management Association (AWMA) in
June 2009:
Liu, Z., H.C. Frey, Y. Cao, and B. Deshpande, "Modeling of In-vehicle PM2.5 Exposure
Using the Stochastic Human Exposure and Dose Simulation Model," Paper 2009-A-238-
AWMA, Proceedings, 102nd Annual Conference and Exhibition, Air & Waste Management
Association, Detroit, Michigan, June 16-19, 2009.
This paper demonstrates the use of a mass balance approach for modeling in-vehicle
concentrations, based on an approach described by Abadie et al. (2006) and Ott et al. (2007)
combined with the use of CALINE4 to estimate near roadway concentration increments. The
methodology is briefly illustrated based on variations in atmospheric stability, windspeed, and
roadway type. A more extensive treatment of this topic is in progress.
It may be reasonable to create somewhat stylized but representative default assumptions for the
typical combination of roadway types that comprise typical commuting trips based on
transportation data. For example, currently, models such as APEX make stylized assumptions
regarding commuting between census tracts without attempting to account for site-specific
characteristics of the commuting. The use of a default-based approach to estimating near-
roadway and in-vehicle concentrations from a more fundamental basis could be a significant
improvement, even though it may not be perfect. Furthermore, sensitivity analysis can be used
to explore the factors that are estimated to most affect in-vehicle concentrations and exposures,
and efforts to improve the modeling framework by developing distributions of variability in such
factors could be prioritized. Of course, there would be a need for evaluation of any new
modeling approach.
Abadie, M., Limam, K.; Builly J.; Genin, D.; Atmospheric Environment, 2006, 38, 2017-2027
Ott, W.; Klepeis, N.; Switzer, P.; Journal of Exposure Analysis and Environmental
Epidemiology, 2007, 18, 312-325
11
-------
3. The planned approach for addressing uncertainty is primarily qualitative with a
focus on sensitivity analysis and a limited quantitative analysis for those variables
determined to be most influential with respect to exposure and/or dose estimation
and where supporting data are available.
a. What are the Panel members' views concerning this general approach?
b. Spatial and temporal gradients in ambient CO relative to CO concentrations
measured at fixed-site monitors are potentially a major source of uncertainty
in the exposure and dose estimates. Do the Panel members have suggestions
for how best to characterize the uncertainties in this relationship?
EPA is correct to state that the APEX exposure model is designed to explicitly account,
quantitatively, for many sources of variability. However, as noted in the comments above, there
may be additional sources of variability that need to be quantified, such as variability in the ratio
M (if the approach of Equation (1) is to be used).
Regarding the general approach proposed to dealing with uncertainty, conceptually it is fine. It
is plausible to deal with some uncertainties qualitatively, some via sensitivity analysis, and some
via probabilistic analysis. The key to how successfully this is implemented is the degree to
which each method is used. Wherever possible, a quantitative approach using probabilistic
methods is preferable if it serves a decision making purpose. Sensitivity analysis is a second-
best alternative to probabilistic analysis. Qualitative analysis is the least preferred among the
three, because ultimately it is the least informative or most subject to ambiguity in interpretation.
There may be situations in which the qualitative approach is the best approach. However, it
should not be the default if there is a need for more information about uncertainty that can be
conveyed quantitatively using plausible input assumptions. Since the NAAQS must be set taking
into account a margin of safety, it should not be presumed that quantitative acknowledgement of
uncertainty would undermine the decision making process. If anything, it could enhance the
decision making process.
For consistency, some comments are repeated here that were offered as part of my review of the
SOx REA, with some minor revisions to be applicable to the CO 1st draft ISA. The weight of
evidence and the uncertainties associated with the state-of-science have implications for the
decision making process. Weight of evidence involves a qualitative determination of causality
and supports conclusions regarding relationships between air quality, exposure, and adverse
effects. Uncertainty implies that scientists are not entirely sure of the numerical values that
precisely and accurately quantify these relationships. However, in many cases these quantities
can be bounded or described using plausible ranges, based either on analysis of empirical data or
encoding of expert judgment. Based on quantitative analysis and reasonable and informed expert
judgments, information regarding uncertainty can be used to inform explicit or implicit choices
of the margin of safety with which to develop a standard that protects public health.
The ISA refers to WHO (2008) as the basis for the qualitative uncertainty analysis approach that
is used by EPA. However, EPA should explain why it chose a qualitative approach rather than a
more quantitative approach, and it should be careful to distinguish or justify situations in which
the qualitative approach is appropriate versus when quantitative methods can be used instead.
Generally, from a scientific perspective, it is preferred to quantify uncertainties wherever
possible. As WHO (2008) explains (p. 31):
Determination of an appropriate level of sophistication required from a
particular uncertainty analysis depends on the intended purpose and scope of a
12
-------
given assessment. Most often tiered assessments are explicitly incorporated within
regulatory and environmental risk management decision strategies. The level of
detail in the quantification of assessment uncertainties, however, should match
the degree of refinement in the underlying exposure or risk analysis. Where
appropriate to an assessment objective, exposure assessments should be
iteratively refined over time to incorporate new data, information and methods to
reduce uncertainty and improve the characterization of variability. Lowest-tier
analyses are often performed in screening-level regulatory and preliminary
research applications. Intermediate tier analyses are often considered during
regulatory evaluations when screening-level analysis either indicates a level of
potential concern or is not suited for the case at hand. The highest tier analyses
are often performed in response to regulatory compliance needs or for informing
risk management decisions on suitable alternatives or trade-offs.
Hence, the Tier 1 (Qualitative) approach is not a default. It should be a justified choice that is
consistent with the purpose and scope of the assessment.
WHO specifies a structured approach to qualitative assessment of uncertainty that includes
1) qualitatively evaluate the level of uncertainty of each specified source;
2) define the major sources of uncertainty;
3) qualitatively evaluate the appraisal of the knowledge base of each major source;
4) determine the controversial sources of uncertainty;
5) qualitatively evaluate the subjectivity of choices of each controversial source; and
6) reiterate this methodology until the output satisfies stakeholders
Hence, there are three dimensions to the qualitative approach, as depicted in Figure 6 of WHO
(2008). In other documents recently reviewed by CAS AC, such as EPA's draft REA for SOx,
EPA seems to have created a different approach in which the level of uncertainty and the
appraisal of the knowledge base are combined, and it is less clear as to the role of subjectivity of
choice in the framework. Given the significance of the ISA and the apparent differences in
approach from that in the WHO Guidelines, further explanation is needed, or the WHO approach
should be adopted with less modification.
The use of low, medium, and high categories of uncertainty can be problematic in that the
interpretation of these is vague and thus may be made differently by different readers or
stakeholders.
In developing the uncertainty analysis, a clear definition of uncertainty and its components (i.e.
bias, imprecision) should be provided.
Sensitivity analysis is a quantitative technique, and there are many variations of sensitivity
analysis. There are recent evaluations of sensitivity analysis with specific focus on their
applicability to exposure assessment models, which build upon some concepts offered by Saltelli
and coworkers, but with more specific application to exposure modeling:
Frey, H.C., and S.R. Patil, "Identification and Review of Sensitivity Analysis Methods,"
Risk Analysis, 22(3):553-578 (June 2002).
Patil, S.R., and H.C. Frey, "Comparison of Sensitivity Analysis Methods Based Upon
Applications to a Food Safety Risk Model," Risk Analysis, 23(3):573-585 (June 2004).
13
-------
Mokhtari, A., and H.C. Frey, "Recommended Practice Regarding Selection of Sensitivity
Analysis Methods Applied to Microbial Food Safety Process Risk Models," Human and
Ecological Risk Assessment, 11(3):591-605 (2005).
Mokhtari, A., H.C. Frey, and J. Zheng, "Evaluation and recommendation of sensitivity
analysis methods for application to Stochastic Human Exposure and Dose Simulation
(SHEDS) models," Journal of Exposure Science and Environmental Epidemiology,
16(6):491-506 (Nov 2006).
Mokhtari, A., and H.C. Frey, "Sensitivity Analysis of a Two-Dimensional Probabilistic
Risk Assessment Model Using Analysis of Variance," Risk Analysis, 25(6):1511-1529
(2005).
Mokhtari, A., and H.C. Frey, "Evaluation of Sampling-Based Methods for Sensitivity
Analysis: Case Study for the E. coll Food Safety Process Risk Model," Human and
Ecological Risk Assessment, 12(6): 1128-1152 (Dec 2006).
EPA appears to be proposing a variation on nominal range sensitivity analysis (NRSA) or
differential sensitivity analysis (DSA). However, by proposing to vary each selected input by
plus or minus 5 percent, EPA is apparently opting for something more like DSA. If only an
arbitrary range is to be used, it may be better to vary each input by plus or minus 1 percent or to
use an elasticity metric. However, NRSA is often more informative. In NRSA, each selected
input is varied over a plausible range of values, with the range differing from input to input. The
insights obtained from NRSA can be quite different than those obtained from DSA, as noted in
many of the references listed above. Both of these are local sensitivity analysis methods, that
assess the response of the model based on perturbations around a specific point.
Even more informative than NRSA are global sensitivity analysis methods, in which many inputs
are varied simultaneously over plausible ranges. Global sensitivity analysis can be conducted in
combination with probabilistic simulation methods such as Monte Carlo simulation, using
techniques such as correlation coefficients, regression analysis, Analysis of Variance (ANOVA),
or categorical and regression trees (CART). Global sensitivity analysis methods take into
account nonlinearity and interaction among inputs. There are some advanced methods, such as
Sobol's method and the Fourier Amplitude Sensitivity Test (FAST) that enable characterization
not only of the first order sensitivities (i.e. the variation in the output directly attributable to
variation in an input) but also interaction effects in which the co-variation of two or more inputs
contributes to variation in an output. Furthermore, if the focus of sensitivity analysis is to assess
inputs and their ranges that lead to high exposure outcomes, then a method such as CART might
be more useful.
In the discussion of the use of global sensitivity analysis as described on the bottom of page 30
and top of page 31, appendix A is cited. However, Appendix A does not appear to define global
sensitivity analysis nor offer a practical approach to conduct it. The references listed above, as
well as the works of Andrea Saltelli, should be consulted for additional methodological
approaches. Jon Helton has also written extensively on the use of sensitivity analysis methods in
conjunction with Monte Carlo simulation and Latin Hypercube Sampling.
On Page 30, sensitivity analysis is implied for many parameters of variability distributions. EPA
may find it helpful to look at the following articles, which explore some of the statistical
properties for characterizing uncertainty based on sampling distributions for parameters of
variability distributions, taken into account sample sizes and variability in the available data:
14
-------
Frey, H.C., and D.S. Rhodes, "Characterizing, Simulating, and Analyzing Variability and
Uncertainty: An Illustration of Methods Using an Air Toxics Emissions Example," Human
and Ecological Risk Assessment: an Inter nationalJournal, 2(4):762-797 (December 1996).
Frey, H.C., and D.S. Rhodes, "Characterization and Simulation of Uncertain Frequency
Distributions: Effects of Distribution Choice, Variability, Uncertainty, and Parameter
Dependence," Human and Ecological Risk Assessment: an Inter nationalJournal, 4(2):423-
468 (April 1998).
Frey, H.C., and S. Li, "Quantification of Variability and Uncertainty in AP-42 Emission
Factors: Case Studies for Natural Gas-Fueled Engines," Journal of the Air & Waste
Management Association, 53(12): 1436-1447 (December 2003).
Zheng, J., and H.C. Frey, "Quantification of Variability and Uncertainty Using Mixture
Distributions: Evaluation of Sample Size, Mixing Weights and Separation between
Components," Risk Analysis, 24(3):553-571 (June 2004).
Zhao, Y., and H.C. Frey, "Quantification of Variability and Uncertainty for Censored Data
Sets and Application to Air Toxic Emission Factors," Risk Analysis, 24(3): 1019-1034 (2004).
Zheng, J., and H.C. Frey, "Quantitative Analysis of Variability and Uncertainty with Known
Measurement Error: Methodology and Case Study," Risk Analysis, 25(3):663-676 (2005).
A key point in many of these papers is that the parameter sampling distributions for some
variability distributions can be treated as statistically independent, but for some others they
cannot be. However, it is relatively straightforward to use bootstrap simulation to estimate the
sampling distributions of parameters and the dependency between them. A prototype software
tool developed for EPA, AuvTool, can be used to assist with this. A version of this, with its
documentation, is publicly available at http://www.foodrisk.org/exclusives/AuvTool/. AuvTool
was developed to support the developing of distributions for uncertainty in the parameters of
distributions of variability for input to exposure models such as SHEDS.
In recent years, EPA has been exploring and applying methods for encoding expert judgments
regarding uncertainties. An example is an expert elicitation for concentration-response functions
related to PM2.5. An EPA guidance document on expert elicitation has recently been reviewed
by an SAB panel. Based on publicly available information, this guidance document has received
strong support from the SAB panel (see
http://vosemite.epa.gov/sab/SABPRODUCT.NSF/81e39f4c09954fcb85256ead006be86e/f4ace05
d0975f8c68525719200598bc7!OpenDocument&TableRow=2.2#2.). The final report from the
panel will need to be reviewed and approved by the SAB before final transmittal to EPA.
However, the purpose of mentioning the positive reaction to the guidance document is to point
out that expert elicitation is a viable technique that can be used along with statistical analysis of
empirical data to develop input assumptions for quantitative analyses of uncertainty. This is a
methodology that OAQPS should consider applying, when appropriate, for other applications.
The bottom of page 31 appropriately mentioned that activity data in CHAD may not be
representative of Denver or Los Angeles. However, in conducting the REA, EPA should attempt
to identify, characterize, and where possible quantify, the ways in which lack of
representativeness, if significant, might lead to biases in results. Furthermore, EPA could
consider making adjustments to attempt to correct for biases. For example, if there are biases in
the CHAD commuting activity patterns that might not reflect city-specific behaviors, it could be
appropriate to make adjustments. Any such adjustments should be documented to facilitate peer
review.
Other Comments
15
-------
Page 19 - have improvements been made to the characterization of near-roadway and in-vehicle
CO concentrations? Are improvements planned?
Page 20 - later mention is made that the CHAD data might not be representative of activity
patterns specifically in Denver and Los Angeles. In what significant ways are CHAD data not
representative of these cities, and how will this affect the results?
Page 21. How are commuting times estimated, and are they internally consistent with the
commuting distances implied by the distance between home and work census tracts? How are
transportation modes selected? Is this an example of where the CHAD data might not be
representative of the study locations?
Page 24, Table 3-1. For the indoor microenvironments, will any distinctions be made in the
estimated outdoor concentration adjacent to the indoor microenvironment taking into account
proximity to roadways? Will it be possible to use site-specific data for LA and Denver regarding
the proximity to actual schools to roadways, and so on? Does the in vehicle- heavy truck
category imply occupational exposure?
16
-------
Dr. Russell R. Dickerson
Comments on the "Plan for Health Risk"
The section on microenvironments in could be more specific on how the modeled concentrations
of CO compare to observations. The uncertainty/sensitivity measures (Appendix A) look good,
but how accurate are personal monitors and what are their detection limits; are they adequate?
At least refer to section 3.6.3.2 of the ISA. Do wood burning stoves of fireplaces (popular
around Denver) contribute to personal exposure?
17
-------
Dr. Paul T. Roberts
I focused my review on Chapter 3 and the presentation made to the Panel on May 13, 2009.
As mentioned during the presentations and following discussions, much of what was
written in Chapter 3 of this Plan on the exposure modeling will not be followed and a
different approach will be used (especially subsections 3.3.1 through 3.3.4. We were told
that they would use spatial interpolation using central-site data with relationships between
near-road (and in-vehicle) concentrations with central-site data. This approach is not
discussed in the ISA, nor in the Scope and Methods Plan - thus it is not possible to evaluate
the approach and determine if the proposed approach is supported by the scientific
literature. In fact, as I mentioned in my comments on the ISA (copied below), the science
discussion in the ISA, Chapter 3.6.7 in particular, is not sufficient.
PTR comments on the ISA, Chapter 3.6.7, which also apply to the Scope and Methods Plan:
"Chapter 367: In light of the currently-planned method for preparing the CO concentration
fields for the exposure model (as discussed at May 13 panel meeting and in slide 13 of the
presentation), the discussion and references cited on pages 3-70 and 3-71 (in Chapter 3.6.7) are not
sufficient to support the methods plan and should be significantly expanded. There is only one
reference cited for concentration surfaces, which will be a major tool in the analysis; many more
are needed. A few additional references that I can easily find are listed at the bottom of my
comments. Note that many of these references are for pollutants other than CO, since few studies
are currently being done on CO; however, the methods can be reviewed and used as guidance for
similar applications for CO. In addition, I think that the exposure modeling Chapter ( 3.6.7)
should include much more specifically about the methods that will be used to address in-vehicle
and near-road exposures. A recent HEI report is now available on the web at:
http://pubs.healtheffects.org/view.php?id=306; this report has an excellent summary of the
current literature and thinking on near-roadway exposures and a good reference list.
Regarding the Chapter 3.7 conclusion that central-site monitor concentrations is generally
a good indicator for the ambient component of personal CO exposure: Total personal
exposure to CO is the time weighted sum of exposure to all microenvironments including
multiple outdoor environments (not just multiple indoor environments). Therefore the central-
site monitor concentration is not viewed as 'a good general indicator for the ambient component
of personal CO exposure'. Equation 3.4 should be reformulated to include multiple outdoor
microenvironments, including at least near roadway exposures (ref section 3.5.1.3 and Figure 3-
34). Equation 3.4 should also distribute the concentration term to both outdoor and indoor
microenvironments as a concentration within both the sum of the indoor components and the
sum of the outdoor components (into a new summation term) specifically as the concentration in
each microenvironment, Ci for both indoor and outdoor. This will also require that the following
sections (and any others) be modified to reflect that more-complex exposure: Lines 30-31, page
3-57; lines 7-10, page 3-65 and page 3-74 lines 10-11."
18
-------
In addition, there are many new references on near-road and in-vehicle pollutant concentrations,
including those in the HEI report referenced above and the recent paper by Barzyk and others at
EPA (Atmospheric Environment 43 (2009) 787-797 and references therein).
The exposure modeling to be performed will need to result in significantly higher CO
concentrations in various near-road and other microenvironments, as illustrated in the M factors
in Table 3-2 of the Scope and Methods Plan.
Regarding the planned approach for addressing uncertainty, I suggest that actual realistic ranges
be used rather than a standard plus/minus 5% type of approach. Using realistic ranges for the
important variables will result in an evaluation that will be closer to the types of situations that
occur in the real world.
In summary, I look forward to reviewing an expanded literature review and state of the science
on exposure estimation in the 2nd draft of the ISA and as part of the 1st draft of the Risk and
Exposure document.
Selected, easy for me to find, references for spatial mapping (see above discussion for Chapter 3.6.7):
Gauderman, Avol, Lurmann, Kuenzli, Filliland, Peters, and McConnell "Childhood Asthma and
Exposure to Traffic and Nitrogen Dioxide, Epidemiology 2005; 16, 737-743.
Ross, Jerrett, Ito, Tempalski, and Thurston "A land use Regression for predicting fine particulate
matter concentrations in the New York City region", Atmospheric Environment 41 (2007) 2255-
2269.
Hoek, Beelen, Hoogh, Vienneau, Gulliver, Fischer, and Briggs "A review of land-use regression
models to assess spatial variation of outdoor air pollution" Atmospheric Environment 42 (2008)
7561-7578.
Henderson, Beckerman, Jerrett, and Brauer "Application of Land Use Regression to Estimate
Long-Term Concentrations of Traffic-Related Nitrogen Oxides and Fine Particulate Matter
ES&T 2007, 41, 2422-2428.
Molitor, Jerrett, Chang, Molitor, Gauderman, Berhane, McConnel, Lurmann, Wu, Winer, and
Thomas "Assessing Uncertainty in Spatial Exposure Models for Air Pollution Health Effects
Assessment EHP vol 115,no 8, August 2007.
Popawski, Gould, Setton, Allen, Su, Larson, Henderson, Brauer, Hystad, LIghtowlers, Keller,
Cohen, Silva, and Buzzelli "Intercity transferability of land use regression models for estimating
ambient concentrations of nitrogen dioxide" J Exposure Science & Environmental Epidemiology
(2008), 1-11
19
-------
Dr. Armistead (Ted) Russell
Given that the Scope and Methods (SM) document describes the approach to be taken mainly in
general terms, it succeeds in laying out a reasonable plan to achieve the objective of providing a
quantitative assessment of exposure to CO using various target levels. The use of Los Angeles
and Denver make sense, though Fairbanks or Anchorage would be of interest as well given the
extreme conditions found there. The use of APEX makes sense, though this again argues for a
more robust evaluation of APEX across pollutants.
A second concern I have is that it is not apparent how the results from LA and Denver will be
generalized to a national scale. In the end, I suspect the panel will want to know what the
resulting exposures will be at a national level to various level/forms of the standard.
In replying to the given questions:
1. As discussed in the Plan, at this time there does not appear to be sufficient controlled
human exposure data to support development of quantitative dose-response relationships
for the health effects reported in subjects with angina. Following the same overall
approach used in prior CO NAAQS reviews, the planned approach is to characterize risks
associated with these effects by estimating exposures and resulting dose (i.e., COHb
levels) and estimating the number and frequency of occurrences over several potential
health effect benchmark levels for the cardiovascular disease population. The potential
health effect benchmark levels are expressed in terms of COHb levels and are based on
the evaluation of the controlled human exposure studies in the draft ISA. With regard to
this planned approach for risk characterization for cardiovascular related health effects
reported in controlled human exposure studies reporting decreased time to onset of
angina, what are the Panel members' views on:
a. The overall planned approach, which is to estimate the number and percent of the
population with cardiovascular disease that would exceed potential health effect
benchmark levels upon just meeting various CO air quality scenarios;
Answer: This is appropriate if done at the national scale, or a strong linkage can be made to
what is found for Denver and LA with the rest of the country.
b. The range of potential health effect benchmark COHb levels (i.e., 2.0, 2.5, and 3.0
percent COHb) that staff plans to use to characterize these health risks.
2. While the first draft ISA reaches the conclusion that the overall health effects evidence
supports the judgment that ambient CO concentrations are likely causal for
cardiovascular morbidity as a category, the document recognizes the uncertainties that
exist with respect to evaluating studies of the association between emergency room visit
and hospital admissions, respectively, for cardiovascular effects and ambient CO
concentrations. In particular, the ISA raises the question of whether ambient CO levels
are serving as a surrogate for one or more elements of the traffic-related air pollution mix.
With regard to the approach for risk characterization, the Plan raises several study-related
20
-------
issues affecting judgments about whether the evidence is supportive of developing
quantitative risk estimates for emergency department visits and hospital admissions for
cardiovascular effects related to ambient CO concentrations.
a. What are the Panel members' views on whether the concerns raised about ambient
CO levels potentially serving as a surrogate for one or more components of the
overall traffic-related air pollutant mixture limit the utility of a quantitative risk
assessment for these health endpoints?
Answer: I do view this as a serious limitation, and while the ISA does bring up the co-exposures
to other pollutants, I do not view that the ISA has gone as deeply as it should. I think this
uncertainty does make a quantitative risk assessment of less value.
b. Given the potential for CO at ambient levels to act as a marker for the effects of
another traffic-related pollutant or mix of pollutants, what are the Panel members'
views on whether or not the results of co-pollutant models provide sufficient
evidence to support a quantitative risk assessment for CO effects at ambient
levels?
Answer: I do not see that there are sufficient studies that have investigated the range of co-
pollutants to adequately model the impacts of these co-pollutants, and thus, I would be hesitant
to conduct a quantitative risk assessment.
Chapter 3 - Scope and Approach for Population Exposure/Dose Analysis
4. We plan to build upon the basic structure and design of the exposure assessment
conducted in the previous review. Since that time there have been major improvements
in the exposure model and in the data for input to the model. Are the Panel members
aware of information sources that would help inform further improvements that would be
worth considering in the current review?
Answer: No.
5. One of the main issues in this analysis is how to estimate ambient CO concentrations on
and near roadways, which can be significant contributors to ambient CO exposures. The
relationship between CO levels measured at ambient fixed site monitors is highly variable
due to the spatial and temporal variability of on- and near-roadway CO concentrations.
In the previous review, proximity factors were used to adjust the concentrations measured
at monitors to estimate roadway-related concentrations of CO. We plan to conduct a
review of the literature and draw upon the results of near-road studies to update the
proximity factor distributions. Do the Panel members have recommendations for
improvements or alternatives to this approach?
Answer: It would appear that, if you have the time and resources, using a more detailed
Gaussian model would be good, or possibly a land use based model. In the end, resource
constraints will dominate, and while I think a proximity-based modeling approach is not
scientifically the best, it likely produces results that will get over the bar. Such a proximity
model should take in to account traffic intensity and fleet mix, not just distance to road.
21
-------
6. The planned approach for addressing uncertainty is primarily qualitative with a focus on
sensitivity analysis and a limited quantitative analysis for those variables determined to
be most influential with respect to exposure and/or dose estimation and where supporting
data are available.
a. What are the Panel members' views concerning this general approach?
Answer: I think that staff should be as quantitative as possible right up front, and provide best
estimates of the uncertainties. These estimates can be caveated and discussed, but without going
through this exercise, it is possible to mis-label an uncertainty as to be low, medium or high.
b. Spatial and temporal gradients in ambient CO relative to CO concentrations
measured at fixed-site monitors are potentially a major source of uncertainty in
the exposure and dose estimates. Do the Panel members have suggestions for
how best to characterize the uncertainties in this relationship?
I would use the results of the Gaussian modeling and the various detailed monitoring
experiments that have been conducted. Some sort of synthesis of the two would be of interest.
22
------- |