^tos%
g? 4% %
ISSK? g
PRO^
April 23, 2015
EPA-SAB-15-008
The Honorable Gina McCarthy
Administrator
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, N.W.
Washington, D.C. 20460
Subject: SAB Review of the EPA's Draft Technical Guidance for Assessing Environmental
Justice in Regulatory Analysis
Dear Administrator McCarthy:
The EPA's Office of Policy (OP) and Office of Environmental Justice (OEJ) requested that the Science
Advisory Board (SAB) review the draft report titled Technical Guidance for Assessing Environmental Justice
in Regulatory Analysis ("EJTG"). The EJTG provides information to assist EPA analysts, including risk
assessors, economists, and other analytic staff, in evaluating potential environmental justice (EJ) concerns
in the context of rule development (i.e., regulatory actions). The EJTG presents the analytic expectations
for EJ analyses to help ensure that potential EJ concerns are appropriately considered.
In response to the EPA's request, the SAB convened an expert panel to review the EJTG. The SAB was
asked to comment on: the clarity and technical accuracy of the guidance; the inclusion of the most
relevant peer-reviewed literature; appropriateness and sufficiency of the six analytic recommendations
listed in the EJTG to ensure consistency, rigor and quality across assessments; the clarity and accuracy
of the guidance on when and how to conduct an analysis of the distribution of costs; and key
methodological or data gaps specific to considering EJ in regulatory analysis. The enclosed report
provides the SAB's consensus advice and recommendations. This letter briefly conveys the major
findings.
The SAB commends the agency for undertaking the very important and complex task of addressing
environmental justice in regulatory decision-making. Overall, the EJTG is a comprehensive presentation
of EJ concerns and of the complex issues, processes and methods associated with EJ analyses. The SAB
would like to offer several recommendations for improving the clarity and rigor of the guidance for
conducting EJ analyses.
To increase the document's clarity, the EJTG needs to include better definitions for key terms (e.g., EJ
populations, susceptibility and vulnerability). The SAB recommends that the EJTG direct analysts to
existing agency guidance documents and focus on providing guidance on elements that are specific for
and can add value to an EJ analysis, thereby reducing redundancy and inconsistency. The EJTG should
provide specific, clear options and examples of best practices for consideration by analysts. Decision
trees, diagrams, checklists and other means may be helpful to summarize key guidance and to steer the
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON D.C. 20460
OFFICE OF THE ADMINISTRATOR
SCIENCE ADVISORY BOARD
-------
analyst to those areas where consistency is essential. The SAB recommends that the EJTG emphasize
the importance of involving communities when conducting an EJ analysis. The EJTG should reference
relevant reports from the EPA National Environmental Justice Advisory Council (NEJAC) and other
published studies that provide recommendations on how to ensure more effective public participation.
The SAB understands the need for the EJTG to remain flexible but is concerned that the EJTG may be
less effective without further specificity. Phrases such as "if feasible" or "when possible" convey a lack
of commitment and may lead to inconsistency in addressing EJ concerns. To ensure consistency and
transparency, the SAB recommends that the EJTG should instead ask analysts to provide an explanation
for why an analysis was not conducted or specific recommendations in the guidance were not followed.
Additionally, the EJTG should not favor quantitative over qualitative analyses, since both are important
and useful. The best and most relevant data should be included in the analyses, not just the most recent.
The terms "differential" and "disproportionate impacts" should be described earlier in the document
where the purpose of the guidance is spelled out. The discussion of these impacts should be made clearer
and less detailed and complex. Conceptual maps may help to highlight the contributors and drivers of EJ
and thereby make them easier to communicate. The SAB recommends that the EJTG direct the analyst
to be transparent in the EJ analysis about how differences across groups are identified for the potential
scenarios that may result from a regulatory action. The EJTG should also encourage the inclusion of
stakeholders early in the analytical process to determine the most relevant metric(s) or analysts should
conduct sensitivity analyses across alternative metrics.
The SAB notes that the use of the standard risk assessment model is emphasized in the guidance as the
primary means to quantify adverse health impacts due to exposure to chemicals in the environment. The
EJTG does not, however, indicate how cumulative impacts should be evaluated, quantified or otherwise
considered in an EJ analysis. The SAB encourages the EPA to develop guidance on how to incorporate
and evaluate cumulative impacts quantitatively and/or qualitatively. In the meanwhile, the EJTG should
consider adopting a more holistic approach to assessing risk and cumulative impacts. In addition, a more
expansive discussion of the limitations of the information used to complete an EJ analysis will add value
by identifying the sources and potential impacts of uncertainties on the effected populations.
The SAB found that there was a lack of sufficient guidance on when and how to conduct an analysis of
the distribution of costs. If EPA documents already exist that provide the needed guidance, analysts
should be directed to them. The issues that are unique to EJ analyses should be highlighted in the EJTG.
The SAB recommends that the EJTG should clarify when cost analyses are appropriate and analysts
should be required to document the basis for any exclusion. Clearer guidance also is needed regarding
the time frame that should be used in a cost analysis. Furthermore, the EJTG should provide direction on
how to characterize the uncertainty inherent in cost estimates.
The SAB agrees with the research gaps and priorities identified by the EPA and the public commenters,
including the need for better distribution of air monitoring locations, use of cumulative impact
assessments, use of appropriate data sources and maintenance of privacy, more complete demographic
information, identification of non-chemical stressors, and the appropriate use of qualitative data. The
SAB recommends that the EPA address these issues systematically by undertaking a strategic planning
exercise to better focus short-term needs versus long-term priorities. To address staffing needs, the SAB
suggests recruitment of appropriately trained postdoctoral researchers, temporary inter-agency transfers,
community-based participatory researchers, and creative use of the agency's STAR research program.
-------
The SAB appreciates this opportunity to review this important EJ guidance and looks forward to the
EPA's response to these recommendations.
Sincerely,
/s/ /s/
Dr. Peter S. Thorne, Chair Dr. H. Keith Moo-Young, Chair
Science Advisory Board SAB Environmental Justice Technical Guidance
Review Panel
Enclosure
-------
1
NOTICE
This report has been written as part of the activities of the EPA Science Advisory Board, a public
advisory group providing extramural scientific information and advice to the Administrator and other
officials of the Environmental Protection Agency. The Board is structured to provide balanced, expert
assessment of scientific matters related to the problems facing the agency. This report has not been
reviewed for approval by the agency and, hence, the contents of this report do not represent the views
and policies of the Environmental Protection Agency, nor of other agencies in the Executive Branch of
the Federal government, nor does mention of trade names or commercial products constitute a
recommendation for use. Reports of the EPA Science Advisory Board are posted on the EPA website at
http://www.epa.gov/sab.
1
-------
U.S. Environmental Protection Agency
Science Advisory Board
Environmental Justice Technical Guidance Review Panel
CHAIR
Dr. H. Keith Moo-Young, Chancellor, Office of Chancellor, Washington State University, Tri-Cities,
Richland, WA
MEMBERS
Dr. Troy D. Abel, Academic Program Director, Associate Professor of Environmental Policy, Western
Washington University, Huxley on the Peninsulas Program, Poulsbo, WA
Dr. Gary Adamkiewicz, Assistant Professor of Environmental Health and Exposure Disparities,
Department of Environmental Health, Harvard School of Public Health, Boston, MA
Dr. Sue Briggum, VP Federal Public Affairs, Public Affairs, Waste Management, Washington, DC
Dr. Linda Bui, Associate Professor, Department of Economics, Brandeis University, Waltham, MA
Dr. Elena Craft, Health Scientist, Environmental Defense Fund, Austin, TX
Dr. Michael DiBartolomeis, Lead, California Environmental Contaminant Biomonitoring Program,
Chief, Exposure Assessment Section, California Department of Public Health, Richmond, CA
Dr. Neeraja Erraguntla, Senior Toxicologist, Toxicology Division, Texas Commission on
Environmental Quality, Austin, TX
Dr. Richard David Schulterbrandt Gragg, Associate Professor, Environmental Science and Policy,
School of the Environment, Florida A&M University, Tallahassee, FL, United States
Dr. Michael Greenberg, Professor, Edward J. Bloustein School of Planning and Public Policy, Rutgers
University, New Brunswick, NJ
Dr. James K. Hammitt, Professor, Center for Risk Analysis, Harvard University, Boston, MA
Dr. Barbara L. Harper, Risk Assessor and Environmental-Public Health Toxicologist, and Division
Leader, Hanford Projects, and Program Manager, Environmental Health, Department of Science and
Engineering, Confederated Tribes of the Umatilla Indian Reservation (CTUIR), West Richland, WA
Dr. Cecilia Martinez, Director of Research Programs, Center for Earth, Energy and Democracy,
Minneapolis, MN
Dr. Eileen McGurty, Director, Graduate Programs in Environmental Studies, Kreiger School of Arts
and Sciences, Johns Hopkins University, Washington, DC
Dr. Douglas Noonan, Associate Professor, School of Public and Environmental Affairs, Indiana
University-Purdue University Indianapolis, Indianapolis, IN
11
-------
Dr. James Sadd, Professor, Environmental Science, Occidental College, Los Angeles, CA
Dr. Thomas L. Theis, Director, Institute for Environmental Science and Policy, University of Illinois at
Chicago, Chicago, IL
Dr. Randall Walsh, Associate Professor, Department of Economics, School of Arts and Sciences,
University of Pittsburgh, Pittsburgh, PA
SCIENCE ADVISORY BOARD STAFF
Dr. Suhair Shallal, Designated Federal Officer, U.S. Environmental Protection Agency, Science
Advisory Board (1400R), 1200 Pennsylvania Avenue, NW, Washington, DC
111
-------
U.S. Environmental Protection Agency
Science Advisory Board
BOARD
CHAIR
Dr. Peter S. Thorne, Professor and Head, Department of Occupational & Environmental Health,
University of Iowa, Iowa City, IA
MEMBERS
Dr. George Alexeeff, Director, Office of Environmental Health Hazard Assessment, California
Environmental Protection Agency, Oakland, CA
Dr. Joseph Arvai, Professor and Svare Chair in Applied Decision Research, Department of Geography,
University of Calgary, Calgary, Alberta, Canada
Dr. Sylvie M. Brouder, Professor and Wickersham Chair of Excellence in Agricultural Research,
Department of Agronomy, Purdue University, West Lafayette, IN
Dr. Thomas Burbacher, Professor, Department of Environmental and Occupational Health Sciences,
School of Public Health, University of Washington, Seattle, WA
Dr. Ingrid Burke, Director and Wyoming Excellence Chair, Haub School and Ruckelshaus Institute of
Environment and Natural Resources, University of Wyoming, Laramie, WY
Dr. George Daston, Victor Mills Society Research Fellow, Global Product Stewardship, The Procter &
Gamble Company, Mason, OH
Dr. Costel Denson, Managing Member, Costech Technologies, LLC, Hockessin, DE
Dr. Michael Dourson, President, Toxicology Excellence for Risk Assessment, Cincinnati, OH
Dr. Joel Ducoste, Professor, Department of Civil, Construction, and Environmental Engineering,
College of Engineering, North Carolina State University, Raleigh, NC
Dr. David A. Dzombak, Hamerschlag University Professor and Department Head, Department of Civil
and Environmental Engineering, College of Engineering, Carnegie Mellon University, Pittsburgh, PA
Dr. Elaine M. Faustman, Professor and Director, Environmental and Occupational Health Sciences,
University of Washington, Seattle, WA
Dr. R. William Field, Professor, Department of Occupational and Environmental Health, and
Department of Epidemiology, College of Public Health, University of Iowa, Iowa City, IA
Dr. H. Christopher Frey, Distinguished University Professor, Department of Civil, Construction and
Environmental Engineering, College of Engineering, North Carolina State University, Raleigh, NC
iv
-------
Dr. Steven Hamburg, Chief Scientist, Environmental Defense Fund, Boston, MA
Dr. Cynthia M. Harris, Director and Professor, Institute of Public Health, Florida A&M University,
Tallahassee, FL
Dr. Robert J. Johnston, Director of the George Perkins Marsh Institute and Professor, Economics,
Clark University, Worcester, MA
Dr. Kimberly L. Jones, Professor and Chair, Department of Civil and Environmental Engineering,
Howard University, Washington, DC
Dr. Catherine Karr, Associate Professor - Pediatrics and Environmental and Occupational Health
Sciences and Director - NW Pediatric Environmental Health Specialty Unit, University of Washington,
Seattle, WA
Dr. Madhu Khanna, ACES Distinguished Professor in Environmental Economics, Department of
Agricultural and Consumer Economics, University of Illinois at Urbana-Champaign, Urbana, IL
Dr. Nancy K. Kim, Independent Consultant, Albany, NY
Dr. Francine Laden, Mark and Catherine Winkler Associate Professor of Environmental
Epidemiology, Harvard School of Public Health, and Channing Division of Network Medicine, Brigham
and Women's Hospital and Harvard Medical School, Boston, MA
Dr. Lois Lehman-McKeeman, Distinguished Research Fellow, Discovery Toxicology, Bristol-Myers
Squibb, Princeton, NJ
Dr. Cecil Lue-Hing, President, Cecil Lue-Hing & Assoc. Inc., Burr Ridge, IL
Dr. Elizabeth Matsui, Associate Professor, Pediatrics, School of Medicine, Johns Hopkins University,
Baltimore, MD
Dr. Denise Mauzerall, Professor, Woodrow Wilson School of Public and International Affairs, and
Department of Civil and Environmental Engineering, Princeton University, Princeton, NJ
Dr. Kristina D. Mena, Associate Professor, Epidemiology, Human Genetics, and Environmental
Sciences, School of Public Health, University of Texas Health Science Center at Houston, El Paso, TX
Dr. Surabi Menon, Director of Research, ClimateWorks Foundation, San Francisco, CA
Dr. James R. Mihelcic, Professor, Civil and Environmental Engineering, University of South Florida,
Tampa, FL
Dr. H. Keith Moo-Young, Chancellor, Office of Chancellor, Washington State University, Tri-Cities,
Richland, WA
Dr. Eileen Murphy, Director of Research Development, Office of Research and Economic
Development, Rutgers University, Piscataway, NJ
v
-------
Dr. James Opaluch, Professor and Chair, Department of Environmental and Natural Resource
Economics, College of the Environment and Life Sciences, University of Rhode Island, Kingston, RI
Dr. Martin Philbert, Dean and Professor, Environmental Health Sciences, School of Public Health,
University of Michigan, Ann Arbor, MI
Mr. Richard L. Poirot, Air Quality Planning Chief, Air Quality and Climate Division, Vermont
Department of Environmental Conservation, Montpelier, VT
Dr. Stephen Polasky, Fesler-Lampert Professor of Ecological/Environmental Economics, Department
of Applied Economics, University of Minnesota, St. Paul, MN
Dr. David B. Richardson, Associate Professor, Department of Epidemiology, School of Public Health,
University of North Carolina, Chapel Hill, NC
Dr. Amanda D. Rodewald, Director of Conservation Science, Cornell Lab of Ornithology and
Associate Professor, Department of Natural Resources, Cornell University, Ithaca, NY
Dr. William Schlesinger, President Emeritus, Cary Institute of Ecosystem Studies, Millbrook, NY
Dr. Gina Solomon, Deputy Secretary for Science and Health, Office of the Secretary, California
Environmental Protection Agency, Sacramento, CA
Dr. Daniel O. Stram, Professor, Department of Preventive Medicine, Division of Biostatistics,
University of Southern California, Los Angeles, CA
Dr. Paige Tolbert, Professor and Chair, Department of Environmental Health, Rollins School of Public
Health, Emory University, Atlanta, GA
Dr. Jeanne VanBriesen, Professor, Department of Civil and Environmental Engineering, Carnegie
Mellon University, Pittsburgh, PA
Dr. John Vena, Professor and Founding Chair, Department of Public Health Sciences, Medical
University of South Carolina, Charleston, SC
Dr. Elke Weber, Jerome A. Chazen Professor of International Business, Columbia Business School,
New York, NY
Dr. Charles Werth, Professor and Bettie Margaret Smith Chair in Environmental Health Engineering,
Department of Civil, Architectural and Environmental Engineering, Cockrell School of Engineering,
University of Texas at Austin, Austin, TX
Dr. Peter J. Wilcoxen, Associate Professor, Economics and Public Administration, The Maxwell
School, Syracuse University, Syracuse, NY
Dr. Dawn J. Wright, Chief Scientist, Environmental Systems Research Institute (Esri), Redlands, CA
vi
-------
SCIENCE ADVISORY BOARD STAFF
Mr. Thomas Carpenter, Designated Federal Officer, U.S. Environmental Protection Agency, Science
Advisory Board (1400R), 1200 Pennsylvania Avenue, NW, Washington, DC
vii
-------
TABLE OF CONTENTS
1. EXECUTIVE SUMMARY 9
2. INTRODUCTION 15
2.1. Background 15
2.2. Charge to the SAB 15
3. RESPONSE TO CHARGE QUESTIONS 16
3.1. Overall Impressions 16
3.2. Quantitative risk and benefit analysis 21
3.3. Key questions for analysts 22
3.4. EJTG Key Recommendations (Section 1.2) 23
3.5. Differential versus Disproportionate Impacts (Section 2) 27
3.6. Contributors and Drivers of Environmental Justice (Section 3) 30
3.7. Human Health Risk Assessments (Section 4) 32
3.8. Methods for Considering Environmental Justice (Section 5) 36
3.9. Analytical considerations 39
3.10. Analysis of the Distribution of Costs 43
3.11. Key Methodological or Research Gaps 46
REFERENCES 51
APPENDIX A. Charge to the SAB A-l
APPENDIX B. Select Evidence of Federal Actions' Unequal Impacts B-l
APPENDIX C. Additional Recommended Edits C-l
viii
-------
1. EXECUTIVE SUMMARY
The Science Advisory Board was asked by the EPA to review the agency's Draft Technical Guidance
for Assessing Environmental Justice in Regulatory Analysis (May 1, 2013 Draft) (also referred to as the
EJTG). The purpose of the EJTG is to provide technical direction to EPA staff and managers to aid them
in incorporating environmental justice into the development of risk assessment, economic analysis and
other scientific input and policy choices as an integral part of the agency rulemaking process. The EJTG
contains guidance on how to assess disproportionate environmental and public health impacts of
proposed rules and actions on minority, low income and indigenous populations in a variety of
regulatory contexts. The charge to the SAB included questions on the following topics: overall
impressions, key questions for analysts, key recommendations, differences and disproportionate impacts,
contributors and drivers, human health risk assessment, suite of methods, distribution of costs analysis
and research gaps. The SAB's response to the questions under each topic are summarized below, with
further discussion of the issues and recommendations contained in the body of the report. In addition, a
major concept in the context of environmental justice - that of public involvement - is highlighted here
and emphasized within the body of the report.
Overall Impressions
The SAB commends the agency for developing the Draft Technical Guidance for Assessing
Environmental Justice in Regulatory Analysis (May 1, 2013 Draft) (also referred to as the EJTG) for
incorporating environmental justice principles into regulatory analyses. In general, the EJTG is a
comprehensive compilation and presentation of environmental justice (EJ) concerns and the complex
issues, factors, parameters, processes and methods. It also presents examples of the necessary elements
to conduct a rigorous, credible and meaningful assessment of environmental justice during the
development of a regulatory action. The EJTG will be useful for understanding EJ issues and will
improve the process for including EJ concerns in rulemaking. It will also be an important resource for
use by other agencies. The SAB offers recommendations and advice on how to improve the clarity,
transparency and utility of the guidance.
The SAB recommends that further guidance be included in the EJTG to assist analysts with
understanding how to conduct an EJ analysis. By doing so, the SAB does not mean to make the EJTG an
all-encompassing document; rather by limiting its scope and not repeating existing guidance, the EJTG
can reduce redundancy and the risk of providing conflicting instructions. To increase the guidance
document's clarity, the EJTG needs to include better definitions for the terms that are used (e.g.,
cumulative risk, co-factors, susceptibility, vulnerability, EJ populations and communities). In addition,
the SAB strongly recommends the use of detailed examples to guide the analyst through conducting the
EJ analysis for regulatory action. The EJTG should provide specific, clear options and examples of best
practices for consideration by analysts. The EJTG should emphasize the role of the analyst while
devoting only a minimum amount of text to explaining the role of the decision/policy-makers in the
same context. The SAB also notes that guidance for EJ methodologies should encourage the use of state,
local, and community level data and assistance that are essential for an accurate national EJ analysis.
Key Questions and Recommendations for Analysts
Flexibility and Feasibility
By attempting to provide flexibility for analysts through ensuring that guidance is not "overly
prescriptive," the recommendations in the EJTG are too broad; hence, the SAB recommends that the
EJTG provide more specificity. One solution is to be more prescriptive regarding when the use of
9
-------
different types of data is appropriate, while leaving flexibility for the application of qualitative
information where applicable and available. Analysts should be provided with guidance on how to
account for uncertainties due to limitations of available data and gaps in knowledge. Analysts may be
directed to existing agency guidance while the EJTG recommendations focus on EJ-specific issues that
should be considered in the analysis. It would be helpful to include a table in the EJTG that presents
alternative analytical methods along with examples (citations) of where they have been applied
effectively, key assumptions embedded in the approaches, and evaluations of their strengths and
weaknesses. Decision trees, diagrams, checklists, and other means to summarize key guidance may be
helpful to steer the analyst to the most important elements of the guidance and those areas where
consistency is essential. To further ensure consistency and transparency, the recommendations presented
in the EJTG should instruct analysts to declare under which conditions specific recommendations were
not followed. There should also be a clear statement or process for determining "feasibility," as
instructed in the guidance, and documenting it as part of the EJ analysis so that these decisions can be
readily understood. This could take the form of a protocol or checklist that outlines how specific
recommendations in this guidance are addressed, or the reasons why they are not addressed. Such a
checklist should also include a statement that addresses the issue of qualitative information in the EJ
analysis or analytical design.
Qualitative Versus Quantitative Data
The EJTG should reinforce the concept that the use of good data, either quantitative or qualitative, is
important. The quality of the data can be measured by the metrics that are used in the sciences, such as
rigor of the study design, sample size, corroboration, universality, proximity, relevance and cohesion.
The "highest quality and most relevant" data should be explicitly favored rather than "newest" data.
Moreover, the EPA should provide more guidance about incorporating qualitative data in EJ analyses,
including how the information should be integrated and what weight it should be given in decision-
making.
Differential and Disproportionate Impacts
Regarding differential and disproportionate impacts, the SAB finds the text in the EJTG to be overly
complex and too detailed to be of practical use to an analyst. In addition, the terms differential and
disproportionate impacts should be described earlier in the document where the purpose of the guidance
is spelled out. While these terms have been defined, what constitutes disproportionality in an EJ context
is not. More discussion is needed on how to analyze and present the data, along with the uncertainties,
that may lead to a determination of a disproportionate impact.
Contributors and Drivers of EJ
The SAB recommends that the concepts of "contributors" and "drivers" be clarified. The section
describing the contributors and drivers of EJ should include a critique of the EPA's traditional risk
assessment and its potential role in contributing to environmental injustice as documented in reports by
the National Research Council (NRC) and the published literature. The EPA should make clear
distinctions between the uses of contributors in analyzing place-based versus health assessment
rulemakings. In rulemakings where there are disproportionate impacts on vulnerable populations (not
limited to specific locations), the contributors described in this section will be important features in
recognizing and addressing the concerns for these populations. Additionally, the SAB notes the
omission of any simplified framework or graphical representation of contributors and drivers to
environmental injustice commonly found in the social determinants of health literature. Conceptual
maps would be a particularly effective heuristic for this section.
10
-------
Human Health Risk Assessment
Cumulative Risk Assessment
The lack of EPA guidance on cumulative risk assessment, dose-response assessment for chemicals in a
mixture and exposure assessment are the primary technical challenges for the EJTG. The SAB
encourages the EPA to update its guidance on cumulative risk assessment so that analysts will have the
tools they need to estimate the toxicity of individual chemicals and chemical mixtures and the impacts
from non-chemical stressors. Currently, the EJTG does not indicate how cumulative impacts should be
evaluated, quantified or otherwise considered in an EJ analysis. More guidance is needed on how to
incorporate and evaluate cumulative impacts for chemical and non-chemical stressors, quantitatively
and/or qualitatively. The SAB recommends that the EJTG consider adopting a Health Impact
Assessment (HIA) approach - a more holistic approach to assessing risk and cumulative impacts. Given
the lack of data or information that might be available when doing such an EJ analysis (for instance,
information on the toxicity of specific compounds and on the cumulative effects of mixtures or multiple
exposures), the assessment should serve as a way to highlight data gaps or lack of available information.
For example, if a more expansive discussion is included of the limitations of the information used to
complete the EJ analysis, the value of the assessment may increase.
Risk Assessment Model
The SAB notes that the current, standard risk assessment model is emphasized in the guidance as the
primary means to quantify adverse health impact from chemicals in the environment. However, the
current risk assessment approach has limitations (both from the technical standpoint and in terms of
communicating with impacted communities) and may not be suitable for assessing complex
environmental justice concerns. If risk assessment continues to be the model of choice for the EPA, then
there should be a subsection in the EJTG to present the difficulties associated with risk assessment and
chemical regulation; the technical limitations and gaps; the lack of mechanisms to incorporate most
qualitative data, in particular social welfare considerations; an inability to incorporate cumulative
impacts of multiple, dissimilar stressors; the lack of effective public involvement inherent in the model
and its application; and complexity that leads to a lack of transparency and accountability. The SAB
also cautions that the use of uncertainty factors in developing dose-response assessments for an
individual chemical might address the general population as a whole, but does not specifically address
differential or disproportionate vulnerability of an environmental justice community. This is especially
true when multiple stressors, factors, and conditions exist to increase the vulnerability of a
subpopulation to a far greater extent than what would be expected in the general population when
exposed to a single stressor, which is how risk assessment is used. Additional uncertainty factors may
not be appropriate as they may become the focus of the assessment and lead to inaction. Instead, it may
be more beneficial to transparently discuss the sources and potential impacts of uncertainties on the
affected populations rather than simply presenting the uncertainties.
Suite of Methods
Literature review
The SAB found this section on the literature review to be an admirable attempt at summarizing an
immense body of research; however, it could be improved. Since this section presented background
information, it warrants an earlier location in the EJTG. While additional references will better reflect
the state of the literature to the benefit of EPA analysts, this section should provide pathways to the
literature instead of a comprehensive literature review. In addition, the social science literature review
should be improved. The EJTG should include narratives and references to health disparities as drivers
11
-------
and contributors, as well as relevant reports from the EPA's National Environmental Justice Advisory
Council (NEJAC). Further, the EJTG should be updated to direct analysts to the most recent guidance on
conducting risk assessment so that these are readily available.
Analytical Considerations
Best practices
The EJTG (Section 5.4 in particular) could benefit from a table or matrix of "best practices." This
should also include and identify some advantages and disadvantages of each concept/method/practice
along with providing information about prior use or noting where their application is most appropriate.
Likewise, Section 5 would be more useful if the key research design elements in EJ analyses were
clarified.
Although the scoping questions in the EJTG are a good starting point, the SAB recommends that the
scoping questions for each EJ analysis should be guided by the circumstance of the assessment and
developed in consultation with the affected populations and stakeholder workgroups. Conducting an
empirical, prospective EJ analysis of EPA rules inevitably entails several major components: (1)
defining the "metric of interest" or dependent variable, (2) defining the comparison group, (3)
identifying the counterfactual distributions, (4) defining the scope of the analysis, and (5) spatially
identifying and aggregating effects. Section 5.4 discusses only (2), (4), and (5), and its discussion of the
scope (Section 5.4.2) is limited. Section 5.4.2 should also be expanded to explicitly address temporal
scope.
The EJTG does not provide clear guidance to analysts with regard to resolving differences in spatial
resolution between two or more geospatial datasets. A list of best practices for geospatial data should be
added to the EJTG to provide guidance on these issues or refer analysts to other EPA documents that
discuss them, if they exist. To enhance the consistency and rigor of EJ data analyses, the SAB envisions
a set of training videos for analysts on topics like exposure, epidemiology, resilience, Geographic
Information Systems (GIS), sample size, and many others. Moreover, the EJTG needs to enforce the
concept that analyses and decisions must be transparent and readily understandable by the public.
The SAB recommends that the EJTG provide better guidance on the selection of a baseline. The EJTG
should provide guidance on identifying and characterizing "hot spots" in the most meaningful context,
as well as resources and examples (in an appendix) to illustrate approaches and best practices. The SAB
also recommends that the EPA examine whether there are any lessons learned or valuable information
that can be gleaned from previous assessments to serve as a guide for future assessments. For instance,
the EPA's Office of Environmental Justice (OEJ) may have data or information on EJ populations that
can be used to assist in the evaluation of potential EJ concerns. A "data repository" may be created for
this purpose.
Transparency and consistency
The EJTG should promote more transparency and consistency in all aspects of an EJ analysis. Clearer
instructions to analysts should be provided when faced with choices over which control variables to
employ, implicitly defining the comparison population. The SAB recommends that working models
should be included in the EJTG until there are better methods developed in the future. These working
models should include clear guidance on what variables to control for when selecting comparison
populations; how to incorporate quantitative and qualitative differences when selecting comparison
populations; demographic versus geographical considerations; national versus state versus local data;
and the level of refinement needed. The EJTG should be clear and consistent in its use of the terms
12
-------
susceptibility and vulnerability when referring to population and individual differences. It is important to
understand that the characteristics defining the population as having an EJ concern are not necessarily
the characteristics that make individuals more susceptible to the hazard. Rather, an EJ population is
regarded as more vulnerable due to its potential increased exposure to hazards and ensuing health
effects.
Distribution of Costs
It is not clear whether the EPA is considering the costs of implementing a regulatory option from the
perspective of individual well-being where costs such as changes in prices and workers' wages are
relevant. Executive Order (E.O.) 12898 refers to costs in terms of disproportionate impacts to health or
exposure. It is plausible that a rule could provide a net reduction in population risk, but an exacerbation
of differential risk to a particular populations. The SAB therefore recommends that the EJTG should
either (a) guide against inclusion of costs in the scope of EJ analyses, or (b) provide much more
guidance on the key issues for addressing costs. Here again, analysts may be directed to use existing
agency guidance but additional information on the issues that are specific to EJ concerns should be
highlighted, such as those discussed below.
The EJTG states that in order to assess the "differences in the baseline incidence [of environmental
harms or risks] and determine if the distribution increases or decreases differences" some information is
required. That information should include the pre-regulation environmental conditions, the projected
environmental conditions without regulation, and the projected post-regulation environmental conditions
for the EJ group and for a comparison group. In practice, even if other regulatory analyses for the rule
define these scenarios, the EJTG should direct the analyst to be transparent about how the differences
across groups are identified for each scenario in the EJ analysis. If the EJTG is not meant to implicitly
define what "justice" looks like through its prescriptions for analysts, then the technical guidance should
encourage sensitivity analyses across alternative metrics or inclusion of stakeholders early in the
analytical process to determine the most relevant metric(s). If there is not clear guidance from the rule as
to scope, then sensitivity analysis would be appropriate to identify the impact on the results of any
environmental outcome or effect.
Public Involvement
Although the EJTG describes public involvement as an essential element of achieving environmental
justice, there is no mechanism specified for ensuring that the public is involved in an environmental
justice analysis. Instead, words like "if feasible" or "ifpossible" are used to guide the analyst on
considering public involvement; such phrases may suggest to impacted communities that the EPA lacks
a commitment for incorporating public involvement and EJ concerns into the risk assessment process.
The EJTG has not adequately addressed or emphasized the need for a more effective means of ensuring
public involvement in risk assessment. This is a major concern and will not address one of the principles
of environmental justice, that is, public involvement should be integrated into the process of risk
assessment from start to finish (including decision-making). Public involvement must be more inclusive
than reaching out to general stakeholders who will not be experiencing first-hand the potential impacts
of a rule or regulation in a community. The SAB recommends that the EJTG reference relevant reports
from the EPA National Environmental Justice Advisory Council (NEJAC) and other published studies
that provide recommendations for analysts on how to ensure public participation when conducting an EJ
analysis.
13
-------
Key methodological and conceptual omissions
Additionally, there are some key methodological and conceptual omissions and ambiguity in the EJTG,
as follows. The EJTG should:
Define and describe methods to identify a comparative control population for evaluating
differential impacts.
Define the distinction between differential impacts and disproportionate impacts.
Define what is "normal" for the sake of establishing a baseline of acceptable risk.
Clarify the concepts of sustainability and prevention.
Explain how disproportionate environmental (ecological) impacts of a rule or regulation should
be factored into an overall, multi-stressor analysis.
Provide guidance for including transparency and accountability to the public.
Define the range of endpoints needed for a holistic or integrated equity analysis (dose,
physiological health, ecological and environmental health, socio-cultural and economic health,
and so on).
14
-------
2. INTRODUCTION
2.1. Background
In July 2010, the Environmental Protection Agency (EPA) released the Interim Guidance on
Considering Environmental Justice During the Development of an Action. This guidance provided
agency analysts and decision-makers with information on when to consider environmental justice in rule
making. As a complement to this document, the Office of Policy, Office of Enforcement and
Compliance Assurance, and the Office of Research and Development led an effort to develop the Draft
Technical Guidance for Assessing Environmental Justice in Regulatory Analysis (May 1, 2013 Draft)
(also referred to as the EJTG). The purpose of the EJTG is to provide technical direction to EPA staff
and managers on incorporating environmental justice into the development of risk assessment, economic
analysis and other scientific input and policy choices as an integral part of the agency rulemaking
process. The EJTG contains guidance on how to assess disproportionate environmental and public health
impacts of proposed rules and actions on minority, low income and indigenous populations in a variety
of regulatory contexts.
2.2. Charge to the SAB
The EPA asked the SAB to conduct a review of the EJTG to assess the appropriateness and scientific
soundness of the technical guidance. The EPA charge (see Appendix A) included questions on: the
clarity and technical accuracy of the guidance; the inclusion of the most relevant peer reviewed
literature; appropriateness and sufficiency of the six analytic recommendations listed in the EJTG to
ensure consistency, rigor and quality across assessments; the clarity and accuracy of the guidance on
when and how to conduct an analysis of the distribution of costs; and key methodological or data gaps
specific to considering EJ in regulatory analysis. In response to EPA's request, the SAB convened an
expert panel to conduct the review. The panel held two public face-to-face meetings (June 19-20, 2013
and January 30-31, 2014) to deliberate on the charge questions and consider public comments and held a
public teleconference (July 22, 2014) to discuss its draft report. The SAB panel's draft report was
considered by the chartered SAB on January 23, 2015 and approved pending some edits. Oral and
written public comments were considered throughout the advisory process.
15
-------
3. RESPONSE TO CHARGE QUESTIONS
3.1. Overall Impressions
Charge Question 1. Please provide your overall impressions of the clarity and technical accuracy of
the EJTG for analyzing and presenting quantitative or qualitative information on potential
environmental justice concerns in the development of EPA regulations.
3.1.1. Clarity
The SAB commends the EPA for developing the Technical Guidance for Assessing Environmental
Justice in Regulatory Analysis (hereafter referred to as the EJTG). The document is thoughtful in
providing guidance for analysts. Moreover, the EJTG represents major philosophical and
communication steps for the agency and EJ communities with a major goal of the guidance being to
incorporate EJ analysis into the framework of regulatory analysis.
In general, the EJTG is a comprehensive presentation of EJ concerns and the complex issues, processes
and methods associated with EJ analyses. It also presents examples of the necessary elements to conduct
a rigorous, credible, and meaningful assessment of environmental justice during the development of a
regulatory action. Appendices can enhance the organization of the EJTG and provide case studies with
greater detail; this organization of the material will allow the main document to focus on the key
technical elements with reference to more detailed information in the appendices.
The EJTG can be improved by reducing redundancy. To strengthen the EJTG, the SAB recommends
that the document leverage the information in existing guidance documents on risk assessment for
regulatory analysis. Indeed, the EJTG will be easier to use if it does not try to repeat general risk
analysis guidance. Limiting the scope of this guidance also reduces the risk of providing conflicting
instructions. Whenever possible, analysts should be directed to relevant agency guidance documents and
the EJTG should focus on additional guidance for EJ specific considerations.
More specific guidance on what to do and how to do it - for example, by identifying decision points and
key methods to use (including what data to consider) - also will assist the analyst. To increase the
document's clarity, the EJTG needs to include better definitions of key terms (e.g., of cumulative risk,
co-factors, susceptibility, EJ populations and communities). Furthermore, a more complete glossary of
terms would improve the EJTG and provide analysts with a consistent definition of the terms used
throughout the document. The SAB also suggests that the terms quantitative, qualitative, analyst,
decision maker, and policy decision be defined in the EJTG narrative or glossary to increase the clarity,
technical accuracy and meaningful community involvement.
Recommendation:
The SAB strongly recommends the use of detailed examples to guide the analyst through conducting an
EJ analysis for regulatory action.
3.1.2. Use of Qualitative Data
The SAB notes that the lack of definitions for quantitative and qualitative data leads to confusion in the
examples provided in Section 5 of the EJTG to illustrate for the use of the proposed methods. The EJTG
appears to erroneously equate qualitative data with anecdotal evidence. The EJTG does not clearly
describe how to analyze and present quantitative or qualitative information about potential
environmental justice concerns during EPA rulemaking. Throughout the EJTG, there are references to
quantitative and qualitative methods, often including comparisons that suggest a hierarchy of methods,
16
-------
with quantitative methods being universally preferred. The text should make clear that both approaches
can be used with success and that in some cases qualitative methods can be the best analytical tool (see
Berg and Lee, 2012). The method selected should be based on context, scope and scale of analysis, and
appropriateness of a given method for the questions posed by the analyst. In some cases, a combination
of qualitative and quantitative methods may be the best approach.
The draft EJTG makes clear that qualitative information should be considered but it does not sufficiently
describe how qualitative information should be integrated and considered in decision-making. This
guidance is especially critical in instances where qualitative data are the only information available.
Clear guidance should be provided on how to account for uncertainties due to limitations of available
data and gaps in knowledge. Data used in risk analyses may include interval, ordinal, and nominal data,
including single words and/or lengthy descriptions. Analysts are able to convert interval data into
ordinal or nominal, and nominal data can now be scanned with computerized tools to convert the data
into quantitative forms. The real issue is not the form of the data, but rather the quality of the data.
Quality is measured by number of samples, reproducibility, and rigorous practices in gathering the data.
The bottom line is how certain can the analyst be in the quality of the data.
Recommendation:
The SAB strongly recommends that EPA provide clear guidance about how qualitative data can be used
in EJ analyses.
3.1.3. Examples, Case Studies and Best Practices
Overall, the guidance should be more specific. It makes sense for the EJTG to be brief in providing a
roadmap for the analyst, without being overly (and unhelpfully) prescriptive. However, the EJTG would
benefit from the inclusion of brief text on additional case studies, best practices, guiding principles, and
definitions for key terms and concepts. In many places, the EJTG advises the analyst to do what is
appropriate and relevant but the guidance should do more to help the analyst determine what factors are
appropriate and relevant, and what factors should be considered when making judgments about this. In
addition, the EJTG needs to provide guidance on how to select key elements of an EJ analysis that must
be part of the analysis and provide specific instructions or choices for an analyst or manager on how to
proceed. The EPA may consider integrating the principles and practices of the health impact assessment
model, including going beyond single chemical exposure risk assessments and considering a more
holistic approach that incorporates stressors other than chemicals and economic burden (Hicken et al.,
2011; Schwartz et al., 2011).
Providing the analyst with a range of best practices will facilitate making appropriate choices, which in
turn can promote consistency among evaluations conducted by different analysts. The document also
could include a section for frequently asked questioned or an overview.
Recommendation:
The SAB recommends that the EJTG be improved by providing specific and clear options and examples
of best practices.
3.1.4. Limitations of the Risk Assessment Model
The EJTG emphasizes the risk assessment model as the primary means to quantify adverse health
impacts from chemicals in the environment. This focus is understandable, given that the agency has
invested decades and countless resources to develop regulations based on risk assessment. However,
some SAB panel members suggest that EPA's current approach to risk assessment may be incompatible
17
-------
with assessments for environmental justice. The EPA may want to consider the availability and
feasibility of alternative approaches that may provide a richer basis for decision-making.
If risk assessment will continue to be the model of choice for the EPA, then there should be a subsection
in the EJTG devoted to the weaknesses and disadvantages associated with risk assessment and chemical
regulation. This discussion should summarize the technical limitations and gaps; the lack of mechanisms
to incorporate most qualitative data (e.g., in particular social welfare considerations); the lack of
methods for incorporating cumulative impacts of multiple dissimilar stressors; the lack of effective
public involvement inherent in the model and its application; and the lack of transparency and
accountability.
The SAB has concerns about the use of the health risk assessment model as the basis for assessing
multiple stressors and impacts unrelated to an individual exposure to a single chemical. The EJTG
presents the environmental justice analyses as being integrated alongside risk assessment and cost-
benefit assessment. Risk assessment requires a highly quantitative relationship between the "cause" and
the "effect" variables and generally uses dose-response models; whereas, the EJTG does not contend
that the parameters important for EJ analyses (e.g. socioeconomic factors, nutritional status and other
susceptibilities) can be modeled that way. The effects of cumulative exposures and cumulative impacts
are mentioned as important considerations when assessing the presence of disproportionate impacts in a
subpopulation. However, there is no further elaboration in the EJTG as to how cumulative impacts
should be evaluated, quantified, or otherwise considered in the EJ analysis. The lack of guidance on
cumulative risk assessment, dose-response assessment, and exposure assessment are the primary
technical challenges for the EJTG. These concerns could be addressed by adopting a Health Impact
Assessment (HIA) approach or another more holistic approach to assessing risk (NRC, 201 la). An
example of this type of assessment is the Duwamish Valley Cumulative Health Impacts Analysis:
Seattle, Washington (Gould et al., 2013). Another opportunity to be more responsive to this concern is in
the EPA's call for planning, scoping and other activities that are consistent with EJ evaluation described
in the agency's "Framework for Human Health Risk Assessment to Inform Decision-Making" (U.S.
EPA, 2014a).
Recommendation:
The SAB recommends that the agency consider adopting a Health Impact Assessment (HIA) approach
or another more holistic approach to conducting an EJ analysis.
3.1.5. Tools for Describing Cumulative Impacts
The EJTG needs to speak in a single voice and incorporate tools - such as a graphic roadmap, flow
charts, decision trees, or checklists - to facilitate use of the material by the reader. Additionally, the
guidance is not clear on when the EJTG is to be used. The problem formulation step should articulate the
reason for conducting an EJ analysis and explain if the human health standard in question is not health
protective from an EJ perspective. The guidance should include a flow chart that can help the EPA
analysts with decision making responsibility to consider EJ issues. A flow chart or roadmap with "Yes"
and "No" paths will help document the various reasons for either conducting or not conducting an EJ
assessment. Also, making use of decision trees, diagrams, checklists, and other means to summarize key
guidance might be helpful to steer the analyst to the most important elements of the guidance and those
areas where consistency is essential. Clear criteria should be included for any inclusion or exclusion of
data for EJ analyses.
When multiple chemical exposures are of concern, the EJ analysis should consider the modes of action
of the individual chemicals, if known, to help determine possible interactions of the chemicals at the
18
-------
cellular and subcellular level and to better describe cumulative impacts. The EJTG should recommend
that analysts include a discussion on the mode of action of the chemical and systematically evaluate all
the information using an Evidence Integration process (Rooney, 2014; NRC, 2014).
The EJTG should acknowledge that some regulatory actions might experience differential compliance
related to some of the same drivers that shape EJ concerns. In these situations, it is plausible that a rule
could provide a net reduction in population risk, but an exacerbation of differential risk. As a simple
example scenario, it is plausible that compliance with the Lead Renovation, Remodeling, and Painting
Final Rule (U.S. EPA, 2008) could vary by housing type, neighborhood, household attributes and other
factors strongly tied to race/ethnicity. The differential exposure of families living in multi-unit properties
owned by non-compliant landlords could potentially increase, relative to renters in a higher
socioeconomic status. To be clear, in most cases, these effects would not be dominant, but it is
worthwhile acknowledging that these dynamics are relevant when thinking about the net impacts of EPA
rules. Further, there is value in acknowledging these effects to highlight the importance of considering
compliance issues in rulemaking and subsequent enforcement.
Recommendation:
The SAB recommends that the EJTG make use of decision trees, diagrams, checklists, and other means
to summarize key guidance to steer the analyst to the most important elements and areas where
consistency is essential.
3.1.6. Inclusion of Updated References
The EJTG refers to some EPA documents but omits many other relevant EPA documents and key
references (e.g. EPA 2014a). The EJTG should be updated to include many new references for
conducting risk assessment (e.g., EPA 2014b). Since these references have not been provided in the
EJTG, it is not clear if the EPA analyst will refer to the latest references. EPA's guidance on cumulative
risk assessment needs to be updated so that analysts would be able to estimate not just the toxicity of an
individual chemical but chemical mixtures and non-chemical stressors. In addition, the social science
literature review should be improved. The EJTG should also include narratives and references to health
disparities as drivers and contributors as well as relevant reports to the Administrator from the EPA's
National Environmental Justice Advisory Council (NEJAC) (e.g., NEJAC 2004).
Recommendation:
The SAB recommends that the references be updated to include additional relevant documents.
3.1.7. Revisions for Key Methodological or Conceptual Issues
There are some additional key methodological or conceptual omissions or ambiguity in the EJTG. These
include:
Defining and identifying a comparative control population for evaluating differential impacts;
Making a clearer distinction between differential impacts and disproportionate impacts;
Characterizing what is "normal" for the sake of establishing a baseline of acceptable risk;
Clarifying the concepts of sustainability and prevention;
Determining how disproportionate environmental (ecological) impacts of a rule or regulation should
be factored into an overall, multi-stressor analysis;
Providing guidance for incorporating transparency and accountability to the public; and
Identifying unique considerations for subsistence populations.
19
-------
3.1.8. Improving Data Quality
To improve the data quality of EJ analyses, the SAB suggests that the EJTG provide guidance on how to
identify an appropriate control population for comparison to a potential environmental justice
community. This extremely important element of an impact assessment is likely to be inconsistent from
one analysis to another and may contribute to extremely flawed analyses of disproportionate risk. There
are several factors that need to be considered, for example, what variables to control for when selecting
comparison populations, how to incorporate quantitative and qualitative differences when selecting a
control population, demographic versus geographical considerations, national versus state versus local
data and level of refinement, and so forth. This might be a consideration for future research but it is such
a critical element to the EJ analysis that at least a working model with clear guidance needs to be
included in the EJTG until better methods are developed in the future.
Recommendation:
The SAB suggests that the EJTG provide guidance on how to identify an appropriate control population
for comparison to a potential environmental justice population.
3.1.9. Accountability and Public Involvement
Moreover, the EJTG needs to reinforce the concept that analyses and decisions must be transparent and
readily understandable by the public. Transparency will be enhanced by discussion of the data with its
limitations and the assumptions. To achieve this, the analyst and manager should provide a detailed
account of decisions on which data to include in the analysis, including a systematic review of the data
considered, with clear inclusion and exclusion criteria. Often, accountability and transparency are
linked; therefore, requiring this level of documentation would go a long way to achieving consistency
and defensible decisions. Better data collection may be possible with new technologies that track human
behavior. However, this is a future research need which can improve and enhance EJ analysis in the
future.
Although the EJTG describes public involvement as an essential element to achieve environmental
justice, there is no mechanism specified in the guidance for ensuring that the public is involved in an
environmental justice analysis.1 Public involvement in the agency's EJ analyses could be enhanced in
two ways. First, the EPA should consider preparing a public version of the EJTG that provides an
accessible summary for the public. Second, analysts should be required to seek input from impacted
communities or citizens (at a minimum public comment) for unique exposure pathways, end points of
concern, and data sources to consider in the analysis (see Berg and Lee, 2012). Additionally, since the
EJTG follows the "2010 Interim Guidance on Considering Environmental Justice During the
Development of an Action", it should be made clear in the EJTG that it is an extension- a further
development and expansion of the 2010 Interim EJ Guidance via the EJ Plan 2014. The utilization and
integration of the 2010 Interim Guidance may facilitate meaningful public involvement along with the
very relevant, thorough and important considerations, questions, and recommendations provided in the
EJTG public comments.
1 The following are three reports from EPA's National Environmental Justice Advisory Committee (NEJAC) regarding public
participation. This is NEJAC's 2013 update to its earlier Model Guidelines for Public Participation. It is current and represents the
consensus of a broad base of stakeholders, http://www.epa.gov/compliance/ei/resources/publications/neiac/recommendations-model-guide-
pp-2013.pdf
This is an older report that takes a broad perspective on ways to solicit stakeholder involvement that might be useful as a secondary
reference, http://www.epa.gov/compliance/ei/resources/publications/neiac/stakeholder-involv-9-27-06.pdf
This URL provides information on a number of initiatives that EPA is undertaking to expand public involvement in the rulemaking
process, http://www2.epa.gov/open/expanding-public-awareness-and-involvement-development-rules-and-regulations
20
-------
Recommendation:
The SAB suggests that the development of EJTG within the framework of the 2010 Interim Guidance be
used as a starting point, essentially modifying and expanding the approach in the 2010 document to
articulate and demonstrate the critical analytical methods and tools necessary to engage stakeholders and
conduct a meaningful assessment of EJ during the Action Development Process (ADP).
3.2. Quantitative risk and benefit analysis
Charge Question 2. The EJTG suggests that if quantitative risk and benefit analysis is done in
support of the rule, analysts should rely on these data to do a quantitative EJ assessment when
feasible. The level of quantitative analysis is expected to vary by regulation and be affected by data,
analytic, or other constraints. If quantified benefit or risk information is not available then a
qualitative EJ analysis is still expected.
(a) Are these directions appropriate? Do they strike the right balance between developing
information that is useful to the decision making process and the cost (time, resources, data
constraints) of doing quantitative EJ assessments.
(b) Please provide advice on methods and best practices for conducting rigorous, high-quality EJ
analyses, both quantitative and qualitative, that may be conducted in support of a national rule
(including data needs or other issues associated with such assessments).
The SAB found the EJTG to be too long for a general public audience but too limited for an analyst
without substantial experience. Those with experience in conducting risk assessments and risk
management projects understand that without much firmer guidance than is currently provided in the
EJTG, the task of doing EJ assessments remains daunting. For example, an economist without extensive
experience would have great difficulty understanding risk assessment, epidemiology, exposure, and
human health data. In contrast, someone from the biological sciences, chemistry and other lab sciences
without experience would find it difficult to address the risk management issues.
There are several options available to address these concerns. One is to have a group of interdisciplinary
analysts work on each assessment and divide their responsibilities according to their experience and
academic background. This method is used when developing environmental impact assessments (EIA)
under the National Environmental Policy Act (NEPA). Second, case studies could be added to the EJTG
as appendices. The small text boxes currently in the document do not suffice, unless there is a direct link
to an example. Third, it would be advisable to hold continuing education seminars from experts in the
elements of the EJ analysis process. Fourth, time and resources have increasingly prohibited the use of
long training sessions. Agencies have responded by developing shorter 15-25 minute training modules.
The SAB envisions a set of training videos aimed at topics like exposure, epidemiology, resilience, GIS,
sample size, and many others. Absent this kind of information, the analysts are expected to make
important recommendations with highly inconsistent backgrounds and without proper support. A final
option would be for the EJTG to provide very detailed instructions on how to do the analysis. Ideally,
that could be done, but it would take a great deal of time to compile and test such instructions.
As noted in the previous section, the SAB is concerned about the stated bias in the EJTG toward
quantitative data and analysis. The key consideration should be the quality of the data, rather than
whether the data are quantitative or qualitative. Moreover, the quality can be measured by the metrics
that are used in the sciences, such as rigor of the study design, sample size, corroboration, universality,
proximity, relevance and cohesion. In some situations, high quality qualitative data are more certain and
21
-------
available, and hence, more reliable than a less rigorous quantitative database. Ideally, the analysts will
gather all the data, assess the quality of all data and then use the best decision-relevant data rather than
focus exclusively on quantitative/numerical data.
Since Question 2b was a later addition to the charge, the SAB panel had already addressed the issues
regarding methods and best practices that may be used in conducting a high-quality EJ analysis in the
response to Questions 9 and 10 as they pertain to Section 5 of the EJTG. The SAB recommendations
have not been repeated here to avoid redundancy and to provide a clearer and better organized narrative.
Recommendations:
The SAB recommends that the EJTG provide more guidance/training to assist analysts so that they can
develop information that is useful to the decision-making process. This training can be in several forms:
utilize the expertise of senior risk assessors to assist new analysts, provide more case studies, and/or
develop training modules.
The SAB suggests that the emphasis on quantitative assessments be decreased and the use of all
relevant, high-quality data be utilized.
3.3. Key questions for analysts
Charge Question 3. Section 1.1 presents 5 key questions analysts should address when analyzing the
environmental justice considerations during the development of a regulation. Are these questions
clear and appropriate for considering EJ during the development of a regulation?
Section 1.1 of the EJTG poses the following three questions and describes five steps that the analyst
should take.
Questions:
1) How did your public participation process provide transparency and meaningful participation for
minority, low-income and indigenous populations, and Tribes?;
2) How did you identify and address existing and new disproportionate environmental and public
health impacts on minority, low-income and indigenous populations during the rulemaking
process? ; and
3) How did actions taken under #1 and #2 impact the outcome or final decision?
Steps:
1) Assess exposures, relevant health and environmental outcomes, and other relevant effects by
population group in the baseline;
2) Assess differences in these exposures, relevant health and environmental outcomes, and other
relevant effects across population groups in the baseline;
3) Assess exposures, relevant health and environmental outcomes, and other relevant effects by
population group for each option;
4) Assess differences in these exposures, relevant health and environmental outcomes, and other
relevant effects across population groups for each option; and
5) Assess how estimated differences in these exposures, relevant health and environmental
outcomes, and other relevant effects across population groups increase or decrease as a result of
each option compared to the baseline.
22
-------
From the risk analysis perspective, the five steps make sense. The first four steps are part of risk
assessment. The fifth step asks the analyst to integrate across risk assessment to assist in risk
management decisions. In the first question, analysts are asked how the public participation process
provided transparency and meaningful participation for the EJ population at risk. The extent of public
participation is not clearly delineated for analysts in the EJTG. Convening a single public meeting may
not be sufficient and many times 3-4 public meetings are needed during which people's assessment of
the decision evolves. A great deal of literature addresses public participation practices and lessons from
this literature could be incorporated into the EJTG (e.g., McComas et al., 2003).
The second question asks the analyst to make a judgment about "disproportionate environmental and
public health impacts." Disproportionality, however, is not defined in terms of magnitude of the impact.
In other words, it is not clear whether an impact requires a 5%, 50%, or one or two standard deviations
of difference to be considered disproportionate. An analyst may be able to estimate differential impacts
and should indicate, as best he or she can, the uncertainty associated with the findings; however, in order
to answer question 2, the analyst would have to make a priori decisions about the level of
disproportionality that requires action. It is more useful that the analyst report the data and the
uncertainty associated with it and leave the determination of disproportionality to the policy and
decision-maker.
The third question relates to and is tied to the answers to the first two questions. Finally, the first
question requires reconsideration in light of the outcomes or final decision. The SAB considers the third
question to be appropriate and consistent with the analysis.
The EJTG describes five steps (listed above) to ascertain the extent to which a potential EJ concern is
associated with the affected environmental stressors prior to the rulemaking; the analyst is instructed to
follow the steps "when feasible." The SAB recommends that this statement be revised to state that the
analysts should follow these steps or document why they could not.
Recommendations:
The SAB recommends that the EJTG include lessons from the literature on public participation
practices.
The SAB recommends that rather than providing a singular assessment, the analyst should instead focus
on the relevant data and other information and seek to offer a variety of options clearly presenting the
uncertainty associated with the analysis.
3.4. EJTG Key Recommendations (Section 1.2)
3.4.1. Comprehensiveness and flexibility of key recommendations
Charge Question 4. The EJTG makes six recommendations to ensure consistency, rigor and quality
across assessments. Are the six analytic recommendations listed in Section 1.2 appropriate and
comprehensive? Are they consistent with the state of the literature while providing flexibility to EPA
program offices in the analytic consideration of EJ in the development of a regulation?
The six recommendations in the EJTG are as follows:
1) For regulatory actions where impacts or benefits will be quantified, some level of quantitative
analysis for EJ is recommended.
23
-------
When feasible, analysts should present information on estimated health and environmental
risks, exposures, outcomes, benefits and other relevant effects disaggregated by
race/ethnicity and income.
When such data are not available, it may still be possible to evaluate risk or exposure using
other metrics (e.g., prevalence of affected facilities as a function of race/ethnicity or income,
evidence of unique or unusual (i.e., atypical) consumption patterns or contact rates).
2) When impacts or benefits will not be quantified or disaggregated by race/ethnicity or income,
analysts should present information that is insightful with regard to potential EJ concerns (e.g.,
basic demographic information, and evidence of differential exposure).
Analysts should use their best professional judgment to determine what combination of
quantitative and qualitative analysis is possible.
3) Analysts should integrate applicable scoping questions during the planning stages of a risk
assessment when one is being conducted for the regulatory action.
4) Analyses should use the same baseline and regulatory option scenarios as other types of
regulatory analyses (e.g., benefit-cost and economic impact analyses) conducted in support of the
rulemaking.
5) Analysts should follow identified best practices when feasible and applicable. Text Box 1.1
outlines current best practices that may be helpful for evaluating potential EJ concerns.
6) Analysts should consider the distribution of costs associated with implementing a regulatory
option from an EJ perspective when appropriate and relevant.
The SAB considers the six recommendations to be generally appropriate and reasonable, but the
guidance on when to implement the recommendations is too broad. More specific guidance should be
provided in terms of both analytical approach and information sources, as described below.
While understanding the EPA's reasons for wanting the technical guidance not to be "overly
prescriptive," the SAB recommends that it should be more so. There are instances in the EJTG key
recommendations, and elsewhere in the EJTG, where the analyst is advised to conduct some analysis
"when feasible and applicable" (e.g., Recommendation 5) or when "appropriate and relevant" (e.g.,
Recommendation 6). The document lacks guidance to assist the analyst in determining the conditions
under which an analysis is applicable, appropriate, or relevant. This overly flexible approach may lead to
a lack of consistency and rigor in the agency's EJ analyses. Therefore, a more specific and prescriptive
guidance would likely be welcomed by analysts and could save time and resources during analytical
design.
A clear statement or process for determining "feasibility," and documenting it, should be part of the EJ
analysis so that these decisions can be readily understood. Allowing analysts too much latitude to define
what is "feasible," "applicable" or "relevant" may not always address EJ concerns adequately, and in
some cases may introduce error or bias to the analysis itself. The stated goals and key priorities for the
EJTG include having a more consistent analytical approach and standardization of metrics. For these
reasons, and also for appropriate transparency, the SAB recommends that the EJTG provide clear,
specific guidance on analytical approaches and standards.
To ensure consistency and transparency, the EJTG recommendations should also include a description
of the conditions under which the six EJTG specific recommendations are not followed. This could take
the form of a protocol or checklist that outlines how specific recommendations in this guidance are
addressed, or the reasons why they are not addressed. Such a checklist should also include a statement
that addresses the issue of qualitative information in the EJ analysis or analytical design. For example,
the guidance could state that "Qualitative data may be considered in addressing potential EJ concerns
24
-------
provided that the information is determined to be valid and reliable" with some explanation of how the
qualities of validity and reliability were evaluated. Other approaches to ensure consistency and
transparency could include a list of "best practices" for specific types of analyses (e.g., selecting and
aggregating geospatial data, proximity analysis, when to use sensitivity analysis, selecting acceptable
statistical techniques appropriate to the data characteristics). The best practices could be illustrated by
carefully selected examples from peer-reviewed research literature. Such examples might better serve
analysts than some of the summaries in boxes now in the draft. Other approaches may include a separate
section on research design with examples, and a matrix of methods that summarizes the strengths and
weaknesses of each method, as well as its implicit assumptions.
The SAB recommends replacing references about using the "most recent data" with the reference
"highest quality data" since, in some cases, the most recent may not be the highest quality. For example,
using the most recent single-year U.S. Census Bureau American Community Survey (ACS) estimates
will introduce greater error into an analysis than using the most recent 5-year rollup simply because of
sample size. This emphasis on data quality is consistent with the language under Section 1.2 of the
EJTG, which states "Rather, they encourage analysts to conduct the highest quality analysis feasible,
recognizing that data limitations, time and resource constraints, and analytic challenges will vary across
media and with the specific regulatory context." One solution is to provide more prescriptive guidance
regarding the use of some types of data, where it is appropriate to do so, while leaving flexibility for the
use of non-quantitative information in cases where it is the highest quality available.
As an example, the SAB recommends the following edits (italicized and strikeout text) be made to the
EJTG's Recommendation 1:
For regulatory actions where impacts or benefits will be quantified, some level of quantitative
analysis for EJ is recommended (see Section 5.1).
When feasible Analysts should present the highest quality, most current and complete information
available on estimated health and environmental risks, exposures, outcomes, benefits and other
relevant effects, disaggregated by race/ethnicity and income if possible.
When such data are not available, it may still be possible to evaluate risk or exposure using other
metrics (e.g., prevalence of affected facilities as a function of race/ethnicity or income, evidence of
unique or unusual (i.e., atypical) consumption patterns or contact rates).
In all cases, analysts should include a discussion of the quality and limits of these data (e.g.,
completeness, accuracy, and validation). It is also advisable to discuss data gaps and suggest
analyses that could provide more definitive answers to key EJ questions if that data were available.
Regarding the guidance for comparison of scenarios, the SAB generally agrees that the present wording
is strong and clear, and that it is important to guide analysts to design these comparisons with specific
relevance to regulatory actions. EJ analyses need not include or repeat specific approaches for
quantitative analysis of risk that are already conducted by the agency and described elsewhere. Rather,
the guidance should refer to standard analytical practice for estimating risk currently used by the EPA,
thereby eliminating any confusion regarding analytical procedures, and avoiding any tendency for non-
technical readers to conclude that risk analysis included as part of an EJ analysis is done differently.
Some panel members and public comments raised issues regarding the need to define control
populations and establish a baseline for statistical power used in data comparisons. These elements are
referenced in Text Box 1.1 of the EJTG, but could be further refined to add more guidance.
Recommendations:
The SAB recommends that analysts be provided with clearer, more prescriptive guidance.
25
-------
The SAB recommends that analysts be instructed to document why an EJ analysis is not feasible or
appropriate or relevant.
The SAB recommends that analysts be instructed to use the highest quality data which may not
necessarily be the most recent.
3.4.2. Additional analytical recommendations
Charge Question 5. Are there any analytic recommendations that should be added? Any that should
be removed?
An additional recommendation on geographic assessment could be included to the EJTG, either as a
separate recommendation or by providing better guidance on the selection of a baseline. Many
communities, such as those that are located around ports, for instance, have similar environmental
exposures as well as potential EJ concerns. It would be worthwhile to examine whether there are any
lessons learned from previous assessments that could serve as a guide for future assessments. To
facilitate this effort, it would be helpful to maintain a list of sources that might be accessed in
completing an assessment. For example, the EPA's Office of Environmental Justice (OEJ) may have
data or information on EJ populations that could be used to assist in identifying a baseline and in
evaluating the potential EJ concerns. A data repository could also serve as an authoritative and easy-to-
access source of publicly available data used in EJ analyses.
Additionally, given the acknowledgment of the lack of data or information that might be available when
doing an assessment, the assessment should serve to highlight data gaps. For example, if a more
expansive discussion of the limitations of the information used to complete the EJ analysis was included,
the value of the analysis may increase. Another example of how such an assessment might be valuable is
by investigating what can be learned using information that is available from other regulatory bodies.
For example, the California EPA's Office of Environmental Human Health Assessment (California
OEHHA, 2002) used a state-approved Inventory Update Reporting (IUR) estimate for diesel particulates
to calculate an estimated lifetime cancer risk for diesel exposure. Although the U.S. EPA does not have
an IUR for diesel, OEHHA has derived a potency estimate for this mixture of compounds and has
classified it as a carcinogen under California law (Proposition 65). Ultimately, this type of information
may advise the agency in future work and highlight data gaps.
Some reviewers expressed concerns that Recommendation 6 in the EJTG was not sufficiently clear and
unambiguous on the subject of costs, as they can be defined differently, depending on context. It is not
clear whether EPA is considering the costs of implementing a regulatory option from the perspective of
individual well-being, where costs such as changes in prices and workers' wages are relevant, or are
limited to the wording of Executive Order (E.O.) 12898, which refers to disproportionate impacts to
health or exposure. If the goal of considering EJ in rulemaking is to ensure that everyone experiences
some minimum level of health or clean environment, then economic costs should, perhaps, not be
included in the analysis. If such costs are included, it will be difficult to describe their distributional
effects in many cases, because the distribution may depend on general-equilibrium effects in the
economy (i.e., national, open to foreign trade) that arise as consumers, industries, and others react to
changes in prices. In the interest of transparency and appropriate guidance to analysts, EPA should
provide clearer guidance on this question. It was also suggested that Recommendation 6 be omitted
entirely, or the role of costs be amplified throughout the guidance.
26
-------
Recommendations:
The SAB recommends better guidance on the selection of a baseline be added.
The SAB recommends that analysts be instructed to provide a discussion of the limitations of the
information used in the EJ analysis.
The SAB recommends that Recommendation 6 in the EJTG be clarified or omitted.
3.5. Differential versus Disproportionate Impacts (Section 2)
Charge Question 6. The EJTG distinguishes between analytically defined differences in impacts and
making a determination of disproportionate impacts. It also suggests six types of information that
may be useful to the decision maker for determining whether differences are disproportionate and
may warrant Agency action (Section 2.4). Is the description of differences in impacts and
disproportionate impacts clear and do reviewers agree with this distinction? Are the types of data
listed to aid the decision maker helpful? Are there other categories of data or information that
should be added to this list?
3.5.1. Description of impacts
The SAB agrees that there is a clear distinction between differential impacts and disproportionate impacts as
EPA defines them but the text on this topic in the EJTG is overly complex and too detailed to be of
practical use to an analyst While the distinction between these impacts is evident, the current definitions,
including the use of the word "substantial," could be reworked to be clearer and more effective.
Providing a brief definition or description of the terms "differential" and "disproportionate" impact,
including how they are evaluated and by whom, is appropriate to retain because analysts will be required
to provide relevant information to decision-makers. However, further detailed discussion and reference
to disproportionate impact should be removed from the EJTG to avoid confusion. In addition, these
terms should be described earlier in the document where the purpose of the guidance is spelled out.
As stated earlier in the response to Charge Question #3, the extent of disproportionality is not defined in
the EJTG. The text should clearly explain that determining whether there is a disproportionate impact-
"that may warrant Agency action" -is a policy judgment made by the decision-makers and informed by
the analysis. Further, the finding of a disproportionate impact is neither necessary nor sufficient for the
EPA to address adverse "differential" impacts; the two issues are separate and distinct.
Recommendations:
The SAB recommends that the terms "differential impacts" and "disproportionate impacts" should be
introduced earlier in the document where the purpose of the guidance is presented.
The SAB recommends that the process for making a determination of disproportionality be clarified.
3.5.2. Types of data and other information to aid the decision
The SAB recommends that the emphasis of Section 2.4 of the EJTG should be to provide clear and
complete guidance to the analyst on what to consider when assessing differential impacts. The section
should be revised to provide more detail and examples of how to present information to decision
makers. The EJTG, on page 11, presents examples (six bullets) of the kinds of information that may be
useful to provide to decision makers. They are:
27
-------
The severity and nature (i.e., biological significance) of the health consequences for which
differences between population groups have been estimated.
The magnitude of the estimated differences in impacts between population groups of concern
and the appropriately defined comparison group (e.g., a measure of statistical significance when
relevant and appropriate).
Mean or median exposures or risks to relevant population groups (or acceptable surrogates when
such data are not available).
Distributions of exposure or risk to relevant population groups - while average exposure or risk
estimates are helpful, it may be the case that differences between population groups only occur in
the tail of the distribution.
Characterization of the uncertainty surrounding various aspects of the analysis.
A discussion of factors that may make population groups of concern more vulnerable to exposure
(e.g., unique pathways, cumulative exposure, behavioral or biological factors).
As currently written, these six bullets are superficial and mostly subjective and thus would provide only
limited guidance to an analyst. In addition, the SAB recommends that any examples provided be drawn
from actual instances or case study examples where an authoritative entity (e.g., federal or state
government, a significant municipality, court case) found impact(s) that were deemed disproportionate
to the degree that corrective actions were taken or penalties imposed.
Recommendations:
The SAB has the following specific comments, concerns, or recommendations related to types of data or
terms used in describing EJ analyses for decision-makers:
a) Make elements of EJ assessments as straightforward and easy for the public to understand as
possible. It is equally important to disclose clearly any elements of uncertainty in the analysis
(e.g., sample size, potentially incorrect assumptions like using proximity as a surrogate for
exposure).
b) With the exception of the last two bullets, the list of information useful to decision-makers
requires or involves quantification. EPA should consider adding an additional statement
reinforcing the concept that the use of good data, either quantitative or qualitative, is important.
c) The fifth bullet recommends the inclusion of an uncertainty analysis. However, this guidance is
too vague to effectively assist analysts in incorporating an uncertainty analysis in their
assessment and presenting useful information to decision-makers. This is an example of a topic
where more detail and clear examples need to be provided in the EJTG.
d) The SAB recommends the EJTG should be clear and consistent in its use of the terms
susceptibility and vulnerability when referring to population and individual differences. These
are not interchangeable terms. Although the terms are defined in the glossary it would be helpful
to also include an example (real or hypothetical) on how the terms should be used in an
environmental justice analysis. For example, according to the EPA Framework on Cumulative
Risk Assessment (U.S. EPA 2003; NEJAC 2004), a subpopulation is vulnerable if it is more
likely to be adversely affected by a stressor than the general population. There are four basic
ways in which a population can be vulnerable: susceptibility/sensitivity, differential exposure,
differential preparedness, and differential ability to recover.
e) Defining "biological significance" has recently been the subject of a National Academy of
Sciences publication (NRC, 2007).While some subtle biochemical change(s) may not be or result
in an adverse effect(s) that is/are biologically significant, many upstream and seemingly benign
28
-------
changes in certain biological responses may result in a significant adverse health outcome
downstream; therefore, it would be helpful to cite this publication in the EJTG and to provide
examples for analysts.
f) As noted in the EJTG, a critical piece of information for decision-makers is the inclusion of
information about cumulative impacts in the assessment. However no definition, method, or
approach is provided in the EJTG to guide analysts about how to include cumulative impact
analysis in their assessments. The SAB emphasizes the importance of including cumulative
impacts from multiple stressors (chemical and non-chemical) and conditions and urges the
agency to provide clearer guidance, both in Section 2.4 and elsewhere in the document, on this
topic for analysts. This concern was echoed in public comments to the SAB for this review.
3.5.3. Other categories of data or information
In addition to the types of data and information discussed in the EJTG, the following clarifications may
also be helpful for analysts conducting EJ analyses:
a) It is difficult to ascertain when a qualitative vs. quantitative analysis is recommended or needed.
The SAB recommends presenting quantitative data and qualitative data separately, with
examples and more detailed guidance.
b) It is not clear if the last bullet in the list from p. 11 of the EJTG (see above) includes exposures
from using consumer products and from occupational exposure. Decision-makers should know
the extent to which both sources influence the overall analysis of impact.
c) In some situations, a hot spot analysis could be useful. While the term "hot spot" can be used in
several different ways in spatial analysis, the hot spots of most concern for EJ will be those
specific locations with multiple risks. Rather than analyzing large geographic areas for specific
risks, an analyst might analyze a few specific locations for multiple risks. The bullets listed on
page 11 would hold but it does imply a broad spatial analysis instead of a hot spot analysis.
Perhaps it would be helpful to indicate that both could be useful, depending on the situation.
d) Census block demographics could be helpful to the analysis (and decision-maker), as well as
information on locations, numbers and types of facilities and their distances from the center of
the census block group within 1 and 3 kilometer radii. This information is part of the needed
spatiotemporal baseline for environmental stressors, i.e., what is happening on the ground.
e) Subsistence populations and unique exposure pathways should be more fully discussed. While
these are mentioned in the EJTG, additional guidance on how to recognize potentially
differential degrees of exposure, even in populated areas, would be useful to the analysts.
3.5.4. Use of exposure assessment statistics
The SAB also noted that there is some inconsistency with regard to the use of exposure assessment
statistics in section 2.4 compared to other sections in the EJTG. Whereas median and geometric mean
can tell part of the story, a distribution of exposures around the mean tells a more complete story. To
maintain both rigor and consistency, the EJTG should provide specific guidance or cite EPA guidance
documents where this information on what values to select when evaluating exposures can be found.
The issue of "disproportionate" exposures is related to how to disaggregate the analytical data and how
fine a scale is intended. There is always a high-end tail of exposure and sensitivity, and with enough
disaggregation it is possible to determine who is in that tail. Sometimes the highly exposed populations
might be clustered in an ethnic or low-income group. In other cases this group might include children,
29
-------
the elderly, disabled or the sick regardless of ethnicity or income. The uneven distribution of stressors
does not always sort along the lines of race or income.
Recommendation:
The SAB recommends that the analyst should describe the characteristics of the population in the higher
percentiles.
3.6. Contributors and Drivers of Environmental Justice (Section 3)
Charge Question 7. Section 3 provides a brief overview of the contributors and drivers of
Environmental Justice. This overview is intended to provide analysts with some considerations that
might drive the analytical decisions used when examining environmental justice for a regulatory
decision.
Does the discussion of contributors and drivers adequately reflect the state of the literature? Is it
clear and technically accurate? Are there any additional factors that should be included in the
discussion?
3.6.1. Reflecting the state of the literature
While the SAB found this section to be an admirable attempt at providing a literature review of an
immense body of research, it could be improved in the following ways.
Since this section presented background information, it warrants an earlier location in the EJTG.
A paragraph on the "Contributors and Drivers" topic should be added early in Section One.
Section Three should become Section Two with an additional paragraph on Environmental
Injustice Contributors and Drivers added early in Section One.
The SAB notes the omission of any simplified framework or graphical illustration of contributors and
drivers to environmental injustice commonly found in the social determinants of health literature.
Concept maps would be a particularly effective heuristic for this section. For example, in a 2002
Environmental Health Perspectives article, Morello-Frosch et al. (2002) proposed a political economy
and social inequality framework for future research. Likewise, Krieger (2001) described the "social
production of disease" or a "political economy of health" perspective. The SAB suggests that while such
additions will better reflect the state of the literature to the benefit of EPA analysts, this section could
provide pathways to the literature, describing a variety of perspectives instead of a comprehensive
literature review. The SAB also recommends that the agency consider the conceptual map discussed in a
2004 NEJAC report (2004, p. 28).
Recommendation:
The SAB recommends the inclusion of a graphical illustration or conceptual map of the contributors and
drivers of environmental injustice.
3.6.2. Clear and technically accurate
The SAB recommends that the EJTG should clarify the concepts of "contributors" and "drivers" of
health disparities in the context of environmental justice. This section should address the concept of
"hotspots," exposure to them, and the drivers of differential susceptibility to hotspots (like residential
sorting behaviors and housing discrimination). Section 3 is repetitive in some places, and redundant
elsewhere which weakens this section's attempt to reflect the literature. A clearer discussion of
contributors and drivers existed in the 2010 Action Development Process- Interim Guidance on
30
-------
Considering Environmental Justice During the Development of an Action (U.S. EPA, 2010). In addition,
the synthesis of studies' findings should be restated. Instead of: "For example, many studies have
established that sources of environmental hazards tend to be located and concentrated in areas that
are dominated by minority, low-income, or indigenous populations", it would more completely reflect
the literature if it were replaced by: "For example, many studies have established that due to both
disproportionate siting and economic and discriminatory factors that push minority, low-income, or
indigenous populations into polluted communities, these groups tend to be located and concentrated in
areas that are dominated by sources of environmental hazards". This should lead into a description of
"the links between the residential and environmental hazard stratifications for these
populations." Additionally, given extant evidence that over the long-run low-income and/or minority
households tend to sort into lower-priced/environmentally degraded neighborhoods (Banzhaf and
Walsh, 2008; Depro et al., 2012), it is important to note that any rule that increases the prevalence of
environmental hotspots raises the potential for long-run EJ concerns.
The EJTG omits key aspects of the historical role that EPA's implementation and enforcement of
regulations may have played in socioeconomic disparities. The agency's use of risk assessment rather
than applying more holistic approaches in regulatory decisions may also fail to provide comparable
protection of environmental justice communities. There is an extensive academic literature on this
perspective and reflects the consensus among a number of risk assessment critics. For instance, a 2002
Environmental Management article noted the following:
While risk assessment continues to drive most environmental management decision-making, its
methods and assumptions have been criticized for, among other things, perpetuating
environmental injustice. The justice challenges to risk assessment claim that the process ignores
the unique and multiple hazards facing low-income and people of color communities and
simultaneously excludes the local, non-expert knowledge which could help capture these unique
hazards from the assessment discourse. . . traditional models of risk characterization will
continue to ignore the environmental justice challenges until cumulative hazards and local
knowledge are meaningfully brought into the assessment process. (Corburn, 2002)
Similar concerns were raised in: (1) a National Research Council document entitled Understanding Risk:
Informing Decisions in a Democratic Society (Stern and Fineberg, 1996); (2) an SAB report entitled
Integrated Environmental Decision-Making in the Twenty-First Century (U.S.EPA SAB 1999); and (3)
a National Research Council document entitled Sustainability and the U.S. EPA (NRC, 201 lb/ None of
these major reports are cited in the EJTG, which reinforces the SAB recommendation that the
Contributors and Drivers section of the EJTG should include a discussion of traditional risk assessment
and its potential role in contributing to environmental injustice.
The EJTG should make clear distinctions between the uses of contributors in analyzing place-based
versus health assessment rulemakings. In rulemakings where there are disproportionate impacts on
vulnerable populations (not limited to specific locations), the contributors described in this section will
be important features in recognizing and addressing the concerns for these populations. In setting a new
contaminant health standard, for example, genetic factors, nutrition and access to healthcare among
subpopulations may lead to the conclusion that a health standard is safe for some categories of
individuals, but not for others. In these cases, the analyst would be greatly helped by cumulative risk
protocols under development at the EPA that would identify reliable data sets and give guidance on how
to characterize the confounding effects of multiple stressors and conditions. Until the EPA's anticipated
cumulative risk guidance becomes available to assure methodical and consistent approaches, the EPA's
analysts will have a particular burden to be transparent about what data they relied upon, its quality and
31
-------
scope, and how they computed cumulative risk. These explanations will need to be sufficient to provide
the public with the assurance that rulemakings are approached with roughly equivalent rigor, and
therefore each rulemaking has thoroughly examined environmental justice impacts.
With regard to place-based rulemakings (e.g., new standards for particular kinds of facilities), there is a
history in the early years of environmental justice advocacy where factors like nutrition or quality of
neighborhood were used to minimize the link between environmental releases and the impacts on
minority communities. In years past, when the issue of disproportionate impacts on communities of
color and low-income communities was raised, poor nutrition or crime were blamed as the source of a
community's health problems. These non-chemical and income-related contributors were used as a
justification to inquire no further into impacts from exposure to local environmental hazards. It was
because of this history that EPA's NEJAC emphasized that additional contributors should not obscure
the need for regulatory action to reduce environmental burdens in communities of color and low income
communities.2 This concern is particularly acute when the scope of potential contributors is expanded to
factors where the EPA will not have uniform sets of data. Data that are insufficiently representative or
factors that distract from identifying disproportionate impacts may obscure the circumstances where
communities of color and low income communities will be disadvantaged by a rule.
Recommendations:
The SAB recommends that the EJTG instruct analysts to transparently present the data and assumptions
used in deriving risk estimates.
The SAB recommends that the EJTG should provide guidance to address the concern that non-chemical
and income-related contributors may inappropriately be used as justification to obscure the need for
regulatory action in placed-based rulemakings.
3.6.3. Additional factors
Section 2 and 3 should be revised to address different social contexts that are relevant for EJ analyses,
such as, occupational and tribal considerations and their differing contributors and drivers to
environmental injustice. The new Section 2 should also include a graphical figure that represents a
prominent conceptual framework from the literature on the contributors and drivers (social and
biological) of health disparities (e.g., see NEJAC, 2004).
Recommendation:
The SAB recommends a number of edits to Section 2 of the EJTG (see Appendix C).
3.7. Human Health Risk Assessments (Section 4)
Charge Question 8. The Guidance directs analysts to use a series of scoping questions at the
planning stages of a human health risk assessment to integrate EJ into analyses conductedfor the
rulemaking. Is section 4 clear and technically accurate? Are the scoping questions outlined in
Section 4.3.2.1 appropriate? Do the scoping questions adequately identify opportunities for
incorporating environmental justice into a human health risk assessment?
2 See, e.g., NEJAC (2010), Nationally Consistent Environmental Justice Screening Approaches ("Moreover, we believe that race is an
appropriate factor in EJSEAT, and currently its relevance may be unintentionally diluted in the EJSEAT methodology by including the
compliance and health variables."), p.13. Note that the NEJAC report was specifically addressing place-based impacts.
32
-------
3.7.1. General concerns about risk assessment methodologies
Overall, the SAB agrees that EJ concerns can be considered within the framework of human health risk
assessment (HHRA) with respect to sensitive and vulnerable populations, the subsistence exposure
pathways, and any group of people that is identified as potentially having disproportionate exposure
and/or disproportionate vulnerability. However, acknowledging that the EJTG emphasizes use of the
risk assessment model as the primary means to quantify adverse health impact from chemicals in the
environment, there are some limitations of the HHRA for EJ analyses and these limitations could be
discussed in the EJTG. Some SAB panel members suggested that a framework based on the risk
assessment model may be difficult for both its technical limitations as well as its reputation for being
difficult to understand, and potentially unfair to impacted communities with multiple sources of stressors
(Coburn, 2002). Section 3 of the EJTG could provide a brief summary of the difficulties historically
associated with risk assessment and chemical regulation, including: technical limitations and gaps; the
lack of mechanisms to incorporate most qualitative data, in particular social welfare considerations; an
inability to incorporate cumulative impacts of multiple, dissimilar stressors; the lack of effective public
involvement inherent in the model and its application; and the lack of transparency and accountability.
The California Comparative Risk Project (1994) and other comparative risk projects are recommended
as references to provide a historical perspective and critique of the risk assessment model. In addition,
the SAB recommends that the EPA consider integrating the principles and practices of the health impact
assessment model, including deviating from single chemical exposure risk assessment and considering a
more holistic approach that incorporates stressors other than chemicals and economic burden (Hicken et
al., 2012). Risk assessment could be more broadly defined as opposed to focusing solely on
conventional human health concerns. EPA's Comparative Risk method was mentioned as an example to
address everything that is "at risk" including quality of life and well-being. The EJTG should direct
analysts to broaden risk assessment beyond health and economics, if this is a goal.
The SAB raises some general concerns regarding the use of a status quo risk assessment as a model
rather than tailoring it to address specific environmental justice concerns. Four key elements identified
as missing or not adequately incorporated into the risk assessment guidance are:
Public Involvement. The EJTG should emphasize the importance of including more effective
means of public involvement in risk assessment. Words like "if feasible" or "if possible" were used
to guide the analyst on considering public involvement. This is a major concern and will not address
one of the principles of environmental justice, that is, public involvement is inviolate and should be
integrated into the process of risk assessment from start to finish (including decision-making). In this
case, public involvement must be more inclusive than reaching out to general stakeholders and
instead to include those who are experiencing first-hand the impact of a rule or regulation in a
community.
Cumulative Impacts. Some advances have been made for evaluating cumulative impacts
quantitatively when numerical data are available and qualitatively when not. The EPA needs to
develop guidance on how to incorporate qualitative data, with sufficient specificity to address how
the information should be integrated in EJ analyses and what weight it should be given in decision
making. Guidance is also needed on how to account for uncertainties due to limitations of available
data and gaps in knowledge if qualitative data is the only information available. This guidance will
advance the agency's ability to conduct cumulative assessments and is especially needed for EJ
analyses.
Hot Spots. Identification and characterization of "hot spots" should be included in the analysis. The
SAB recommends that the EJTG should define the term "hot spots" in its most meaningful context
33
-------
and provide resources and examples (in an appendix) illustrating approaches and best practices. The
agency also could examine whether there are any lessons learned from previous assessments to serve
as a guide for future assessments; for instance, EPA's Office of Environmental Justice (OEJ) may
have data or information on EJ populations that could be used to assist in the evaluation of potential
EJ concerns. This idea was captured in the concept of a "data repository".
Uncertainty Factors. The use of uncertainty factors in developing dose-response assessments for an
individual chemical might address risks to the general population as a whole, but does not
specifically address disproportionate vulnerability of an environmental justice community. This is
especially true when multiple stressors, factors, and conditions exist to increase the vulnerability and
sensitivity of that subpopulation to a far greater extent than would be expected in the general
population exposed to a single stressor, which is how risk assessment is most commonly used.
Recommendations:
The SAB suggests that the EJTG consider integrating the principles and practices of the health impact
assessment model, including deviating from single chemical exposure risk assessment and using a more
holistic approach that incorporates stressors other than chemicals and economic burden.
The SAB recommends that the EJTG be revised to address the four elements presented above.
3.7.2. Clarity and technical accuracy
As noted previously, the SAB recommends that statements such as "whenfeasible " and "ifpossible "
with respect to public involvement be changed or deleted. Use of such language may suggest to
impacted communities that EPA lacks a commitment to incorporating public involvement into the risk
assessment process. A specific example of language that may be considered by some groups to be
inflammatory and was recommended for revision includes the statement on page 23, the last sentence in
section 4.3.1, "The scope of the HHRA also will be affected by ... limitations in time and resources. "
An EJ community is not likely to find comfort in statements that EPA does not have the time or
resources to help them.
In order to clarify section 4, the agency should revise the first paragraph of Section 4.3. The text seems
misleading, since racially/culturally diverse (minority), low-income or indigenous populations are of EJ
concern by definition. For example, the text could be changed from "it is important that HHRAs
conducted in support of regulatory actions explicitly consider health risks that may disproportionately
accrue within minority, low-income or indigenous populations since these demographic attributes may
reflect underlying vulnerability and susceptibility to environmental stressors " to " ...define
subpopulations of concern for environmental justice. "
The 3rd paragraph of Section 4.3.2.1 also needs clarification. The language in the guidance, "Similarly,
communities with potential EJ concerns may experience differential risks due to higher susceptibility
(e.g., due to lifestage or pre-existing health conditions) to the stressor being regulated" raises a
conceptual issue that should be clarified. If everyone at some life stage (e.g., prenatal) is more
susceptible to a particular stressor that may be regulated, then if one income/racial/ethnic group has
more children than the majority, does that fact by itself make the stressor an issue for EJ? A suggested
clarification for that language is that the stressor is a concern for people at the relevant life stage. It is
important to understand that the characteristics defining the population as an EJ concern are not
necessarily the characteristics that make individuals more susceptible to the hazard. Rather, an EJ
population is regarded as more vulnerable due to their potential increased exposure to hazards and
ensuing health effects.
34
-------
The effects of cumulative exposures should be highlighted when assessing the presence of
disproportionate impacts in a subpopulation. The EJTG does not indicate how cumulative exposures
should be evaluated, quantified, or otherwise considered in an EJ analysis. For example, Sections 4.3.2
(Planning and Scoping) and 4.3.3 (Problem Formulation) are sections where a discussion of cumulative
exposures could be included. The SAB recommends that the gui dance include a consideration of the
cumulative environmental health risks faced by low-income and minority populations or, at the very
least, provide a detailed explanation for its decision to exclude consideration of cumulative risks.
Moreover, the EJTG should explicitly refer users to any EPA cumulative risk assessment (CRA)
guidance it develops. Cumulative HI IRA should not be limited to the mode of action and target organ
interactions; assessments should evaluate multiple chemicals of concern and multiple exposure pathways
and media.
Affected Resources
Identify what is "At Risk"
-Natural Resources & Eco-cultural Systems at risk
- Human systems and uses at risk
-Existing Stressors
Hazard Identification
- Probability
- Severity
- Contaminants
- Migration
Socio-cultural
'Exposure'
Co-risks &
vulnerability
Socio-cultural &
Socio-economic
Impact
Characterization
Human
Exposure
Cumulative Impacts
to the Affected People and
their Eco-Cultural Systems
and Service Flows
Human Risk
Characterization
with vulnerability &
susceptibility
Human toxicity and
sensitivity
Ecological toxicity
and sensitivity
Ecorisk
Characterization
To Species &
Ecosystems
Ecological
Exposure
Figure 1. A broader view of risk assessment including elements of the overall eco-cultural system: human
health, ecological health, and socio-cultural/socioeconomic health (Adapted from Harper et al., 2007)
The willingness to include quality of life or well-being, as well as the use of a term like HIA in order to
force some thinking outside the conventional box is encouraged. In addition, incorporating co-stressors
and the broader identification of what is "at risk" in a community should be considered. To further
illustrate this idea, the SAB suggests that a figure be added (see Figure 1 above) to introduce a step to
identify "affected resources" prior to the "hazard identification" and "exposure assessment" steps, and a
"cumulative impacts" step after human and ecological risks are evaluated. This new figure could be
blended with Figure 4.2 in the EJTG in order to demonstrate how FtHRA can include co-stressors or co-
risk factors.
35
-------
3.7.3. Appropriateness of questions
Getting EJ issues recognized can be difficult, even when minority/ethnic groups are vocal and well-
recognized. Specifically, flaws in the community participation process impact participation from Tribes
and other rural EJ populations. Initial demographic and income screens are not adequate in determining
whether an EJ concern exists; for instance, these screens are not appropriate for Tribes and other groups
who may experience increased risk due to exposures from subsistence pathways. Assessors should be
required to find out who uses natural resources within the impact area, thus giving more emphasis on
pathways of exposure earlier in the screening assessment, independent of a formal delineation of the EJ
community. This is especially needed when the EJ community is dispersed or represents only a stratum
of the overall population. Also, a mention of the proportion of an affected population, and not just the
absolute numbers of affected people, should be part of the analyst's report. For example, 20 percent of a
rural population (or tribe) might be fewer people than 2 percent of an urban population, but the risk
manager might need to know this. If an important tribal resource-gathering area is affected, 100 percent
of the tribe is affected even if it is a small area, or somewhat remote from a population center.
3.7.4. Prioritizing scoping questions
The SAB recommends that the scoping questions be guided by the circumstance of the assessment and
determined in consultation with the affected populations and stakeholder workgroups. Each HHRA is
unique based on the situation being assessed, the regulatory action being considered, the resources and
EPA office conducting it and therefore may call for different priorities in assessing risk.
The SAB finds that the EJTG does not adequately address the exposure assessment, which is a critical
(and difficult) step in the risk assessment process. Exposure assessment is the one part of the risk
assessment model that may identify (or miss) disproportionate impacts of a stressor depending on the
available data, the experience of the analyst, and/or the proper use of tools and methods available to
assess exposure. The EJTG should provide additional guidance to the analyst on methods and sources of
information.
The EJTG also lacks guidance for identifying an appropriate control population for comparison to a
potential environmental justice community. This will likely lead to inconsistent analyses and result in
flawed assessments of disproportionate risk. The identification of an appropriate control population is a
critical element to the EJ analysis.
Recommendation:
The SAB recommends that the EJTG include at least a working model with clear guidance (e.g.,
including what variables to control for when selecting comparison populations, how to incorporate
quantitative and qualitative differences when selecting control populations, demographic versus
geographical considerations, national versus state versus local data and the level of refinement needed)
until there are better methods developed in the future.
3.8. Methods for Considering Environmental Justice (Section 5)
Charge Question 9. Does Section 5 provide a clear overview of the methods that could be usedfor
considering environmental justice? Are there other methods that should be added to the discussion?
3.8.1. Clarity and other methods
For the most part, Section 5 provides a clear overview of some methods for use in analyzing EJ, but
lacks sufficient detail about the full suite of methods that can and, more importantly, should be used.
Other methods that could be used but are not mentioned, include HIA and other social science methods
36
-------
(e.g., mixed methods, approaches using qualitative data). Ideally, the research design could incorporate
both qualitative and quantitative approaches, possibly giving analysts feedback on their investigations
from the people who are potentially impacted by the rule. A mixed methods approach can promote a
critical aspect of EJ, that of ensuring meaningful involvement.
Additional methods to be added are the use of EJ or "cumulative impacts" screening tools or methods.
One such tool, EJScreen, under development by EPA, is mentioned once in the EJTG but there are many
other efforts that represent the varying approaches, data types, analytical methods, and
scoring/weighting systems and rationale that could be useful to analysts. The public comments also
mentioned other screening approaches that should be included in the EJTG (e.g., Cal-EPA's
EnviroScreen).
It also would be helpful to include a table that presents alternative analytical methods along with
examples (citations) of where they have been applied effectively, key assumptions embedded in the
approaches, and evaluations of their strengths and weaknesses. Appendix B provides a summary table
and a list of peer-reviewed, empirical EJ studies of agency actions. Some of these studies might be
useful examples of approaches to be included in the EJTG. There is a very limited literature that
accomplishes what EJTG directs analysts to do which can be expanded by including examples from state
and local rules.
Recommendation:
The SAB recommends that the EJTG expand its presentation of the available methods to include
Enviroscreen and reference the list of state and local rules to provide further examples of other
approaches.
3.8.2. Data considerations
The SAB notes that the narrative and glossary in the EJTG lack definitions for quantitative and
qualitative data, which leads to confusion in the examples in Section 5 for the use of the proposed
methods. The EJTG appears to erroneously equate qualitative data with anecdotal evidence. Examples of
when an analyst would use qualitative data to answer the research question should be given. Qualitative
data likely will be used when EJ analyses seek to describe processes or to understand people's values,
behaviors, motivations, or culturesalthough social science and ethnographic methods can yield
numerical data about people's values etc. An outline of the diversity of qualitative data analytic methods
would also be useful (e.g., see Tesch, 2013, pp.72-73).
Where restrictions outside the scope of the EJTG constrain the selection of data or methods, these
limitations should be made explicit and the rationale for selecting a particular type of data should be
included. Otherwise, "highest quality and most relevant" data ought to be explicitly favored in all
instances rather than the "latest" data (pp.4, 44). In principle, qualitative methods should not be favored
differently than quantitative methods. The EPA should not assume that numerical or statistical data are
always the highest quality and preferred data - they can be precise but inaccurate. At the other end,
qualitative data can be imprecise but correct or accurate. Analysts should be instructed to justify their
choice of data and analytical methods. Unless other rules or feasibility (time, resource constraints within
EPA) dictate, the EJTG should not pre-judge the intrinsic superiority of either quantitative or qualitative
approaches.3
3 Whether it is OMB stipulations or other concerns about validity, reliability, and generalizability, qualitative data analysis can meet
high quality standards. If done correctly, qualitative approaches can be generalized to a national level or at least transferred to other
contexts. As is true across all inferential methods, purposeful sampling for cases would be key to the findings being useful in other
37
-------
It is generally recognized that it is important to evaluate data quality and risk of bias (ROB) in risk
assessments. However, it has been difficult to define a single set of rules for evaluating data quality
because risk assessments often include diverse data streams (i.e., animal studies, human chamber
studies, and epidemiological studies). There are several publications that provided best practices and
frameworks to assess data quality and ROB. For example, Klimisch et al. (1997) proposed a systematic
approach for evaluating the quality of experimental toxicological and ecotoxicology data. In their
approach, Klimisch et al. (1997) provided three categories (Reliability, Relevance, and Adequacy) for
evaluating data quality in animal studies. The three categories are described as follows:
Reliability evaluating the inherent quality of a test report or publication relating to preferably
standardized methodology and the way that the experimental procedure and results are described to
give evidence of the clarity and plausibility of findings.
Relevance covering the extent to which data and/or tests are appropriate for a particular
hazard identification or risk characterization.
Adequacy defining the usefulness of data for risk assessment purposes. When there is more
than one set of data for each effect, the greatest weight is attached to the most reliable and
relevant.
The National Research Council provided a table that discusses the strengths and weaknesses of human
studies and animal studies and can be a good reference for the EPA analyst (NRC 2014). Human studies
are generally preferred over animal studies because they do not require animal-to-human extrapolation.
Human studies are mainly of two types: human chamber studies and epidemiological studies. Human
chamber studies are very useful but are limited in that they often have very limited sample sizes.
Epidemiological studies are also useful but often are limited because of poor exposure data. Rooney et
al. (2014) provided a very good summary on ROB as well as a comprehensive set of questions to discuss
ROB. Lavelle et al. (2012) and Money et al. (2013) provide frameworks for systematically integrating
human and animal evidence and evaluating and scoring human data, respectively. Rhomberg et al.
(2013) not only provide best practices for conducting weight of evidence analysis but also a critical
review of the available frameworks.
More broadly in Section 5, there are important gaps and confusion about evaluating feasibility and
presenting information. For example, the introduction (p.36 in the EJTG) identifies what the analyst
should do "when feasible," which suggests that the EPA is using a screening process to determine
feasibility of conducting an EJ analysis. The process and the criteria for feasibility are absent in Section
5 of the EJTG. Footnote 51 (p.42 in the EJTG) references a "screening analysis" without a full
discussion. In addition, Section 5.1 does not discuss how to evaluate the feasibility of doing an analysis.
For a section titled "Evaluating the Feasibility...," the text should avoid the use of "when feasible" and
instead focus on explaining the criteria and process for determining feasibility. Alternatively, the section
could be retitled to accurately reflect its contents (e.g., "Data and Methodological Considerations in
Assessing Potential EJ Concerns"). The SAB urges the former because this section of the guidance is an
appropriate place to better address several related concerns expressed by the SAB, such as: evaluating
feasibility, articulating the research design, and selecting among alternative data sources and analytic
methods.
or broader contexts. Three strategies are employed to assist with transferability: thick description, purposeful sampling and
triangulation. Thick description paints a highly detailed picture of the context and boundaries so that the key issues can be
discerned for other contexts. Purposeful sampling refers to the many ways of designing a research study with qualitative data,
depending on the purpose of the study and the guiding questions. Triangulation is the use of multiple data points to draw
conclusions (Popay et al., 1998; Fossey et al., 2002).
38
-------
Section 5.2 stresses the need for detailed information about the baseline distribution and the projected
distribution of outcomes (or at least the distribution of regulatory impacts). Most of the bulleted list in
Section 5.3 (p.41) refers to information about the baseline. Parallel bullet points about information
reporting expectations related to the projected distribution of outcomes should be added. (Additional
discussion of this issue appears in response to charge Question 10.)
Recommendation:
The SAB recommends that, in Section 5, the EJTG clarify elements of EJ analysis such as, evaluating
feasibility, articulating the research design, and selecting among alternative data sources and analytic
methods.
3.9. Analytical considerations
Charge Question 10. Section 5.4 discusses analytical considerations that may have a significant
impact on results. Are these considerations appropriate for assessing EJ in the context of a
regulation? Are there considerations that should be added/removedfrom the discussion?
The SAB agrees that the analytical considerations in Section 5.4 of the EJTG are relevant for conducting
an EJ analysis, but recommends that additional important considerations be added. Section 5.4 in
particular could benefit from a table or matrix of "best practices," to include information about prior use
and identify some advantages and disadvantages of each or note where their application is most
appropriate. Likewise, the whole of Section 5 would be more useful if the key research design elements
in EJ analyses were clarified. Conducting an empirical, prospective EJ analysis of EPA rules inevitably
entails several major components, including: (1) defining the "metric of interest" or dependent variable,
(2) defining the comparison group, (3) identifying the counterfactual distributions, (4) defining the scope
of the analysis, and (5) spatially identifying and aggregating effects. Section 5.4 discusses only (2), (4),
and (5), and its discussion of the scope (Section 5.4.2) is limited, as noted below. In addition, the EJTG
could benefit from a richer, more detailed and more prescriptive discussion of these crucial points in
order to better guide analysts. Each of these important topics is discussed further below.
3.9.1. Defining metrics of interest
Selecting the metrics to assess EJ concerns is a critical component of any EJ analysis. Section 5.2 makes
two bold and restrictive statements in this regard. These statements are hidden in an overall confusing
explanation in Section 5.2. First, it notes that analysts need to characterize both the pre- and the post-
regulation distribution of environmental quality (or, equivalently, a baseline distribution and a
distribution of changes in environmental quality). The argument is that knowing just the distribution of
the change in environmental quality (AE) owing to the rule is insufficient for an EJ analysis. The EJTG
can be improved by stating that a useful EJ analysis could still be done if only the distribution of AE is
known. It might not be ideal, but reasonable quantitative and qualitative EJ analyses have been based on
just AE before, and it would be unfortunate if the absence of baseline distributional information is used
to prevent an analysis of the distribution of AE.
The reality is that the EJ discourse has not settled on a single metric (e.g., distribution of AE or change
in distribution of E). While an EJTG that prescribes a single conceptual measure takes away discretion
from future analysts, it also implies potentially objectionable policy priorities by any such measure.4
4 For instance, the example on p.40 seems to imply that a policy that had only a 5% reduction in asthma cases for minorities and
a 10% reduction for others might not appear unjust if the baseline incidence rate for minorities was more than double that of
others. Regardless, a metric consistent with dispersing new pollution sources would not be seen as "just" by some.
39
-------
The choice of the metric might implicitly target policy to equalize pollution levels or environmental
risks across groups or to equalize gross or relative environmental improvements across groups. Noonan
(2008) argues for less ambiguity in defining the metric of interest. The EJTG's assertion that EJ analyses
should assess convergence in the distribution of environmental quality or stressors rather than equity in
the distribution of AE goes a long (and controversial) way to taking a policy stand. More prescription
about measuring environmental impacts, in particular whether analysts should be measuring in relative
(as rates or per capita) or gross terms, would help.5 The results of an EJ analysis can differ significantly
depending on the use of a maximum individual risk (MIR) or a population risk measure (Turaga et al.,
2011).
Recommendation:
The SAB recommends that the EJTG should encourage sensitivity analyses across alternative metrics or
inclusion of stakeholders early in the analytical process to determine the most relevant metric(s).
3.9.2. Defining comparison groups
The description of comparison groups (Section 5.4.2) should be clarified. If the objective of EJ analysis
is to compare environmental conditions (exposure, risk, etc.) for EJ groups identified on the basis of
income, race/ethnicity or other factors, then the relevant comparison group for each EJ group would be
the population that is as similar as possible, but lacking the characteristic defining the group as of EJ
concern.
An important question is on what variables this similarity is based. Clearer and better guidance in
Section 5.4.2 is needed. Race/ethnicity, family income, and other (permanent) characteristics can affect
individuals over their entire lifetime, contributing in various ways to their current situation. One
possibility is to select a population 'as similar as possible' before the birth lottery is resolved (i.e., the
uncertainty about who one's parents will be), in which case the comparator would be the general
population excluding those that are of EJ interest. Other approaches - especially when the defining EJ
characteristic is something that individuals have some discretion over (e.g., region of residence, religion,
education, etc.)6- might imply many other controls or alternative research (e.g., quasi-experimental)
designs in order to identify the proper comparison group.
Recognizing alternative explanations for unequal baseline (and potentially future) distributions, the use
of multivariate statistical analysis to control for these factors offers the analyst considerable latitude to
implicitly define a comparison group (insofar as the findings are then conditional on the covariates).
This is particularly important in many EJ analyses, where common EJ group characteristics like race and
income or subsistence lifestyles are highly correlated. EJ studies in the literature employ an inconsistent
variety of conditioning variables, sometimes including both race and income. These various controls not
only affect the findings, they often implicitly define the comparison groups. The EJTG should promote
more transparency and consistency by providing clearer instruction to analysts faced with choices over
which control variables to employ that implicitly define the control group.
5 The language on p.40 should be revisited to ensure consistency with evaluating the rate of incidence rather than number of
cases.
6Tribes will argue that their birth into a tribe determines location and religion tribal members cannot move from their
homeland, and dislocation causes immense harm (e.g., Trail of Tears). Religion is also often seen as immutable. Education might
seem to be a matter of choice, but not so much in poor communities. For example, the Creator gave tribes particular natural
resources in their home regions, and partaking of those foods is a requirement of natural law (religion). So, tribal members may
knowingly eat contaminated fish because that is the food the Creator gave them. They do not see that they have a choice,
although they might reduce the amount of fish they feed their children. Someone has to continue the First Foods consumption,
and adults may accept the burden of eating contaminated foods on behalf of the tribe.
40
-------
Section 5.4.2 also presents two perspectives by Bowen (2001) and Rinquist (2005) on the selection of an
appropriate comparison group. Bowen suggesting to restrict the comparison group to a sub-national
level and Ringquist contending that placing restrictions on comparison groups may bias the results
against finding disproportionate impacts. These perspectives have been discussed elsewhere in the EJ
research literature. The EJTG recommends that the analyst conduct a sensitivity analysis regarding how
the comparison group is defined, as if marginal differences in geographic extent are a principal
determinant in error or bias for the results. In work subsequent to the articles cited in the EJTG, this
argument has been resolved using a better analytical approach and should be cited instead of those
detailing the competing points of view, and would serve as a much better source of information for
analysts seeking appropriate methodological direction on defining a comparison group. The two papers
that best detail this approach are Mohai and Saha (2006; 2007).
The SAB also recommends that the EPA consider both urban and rural examples in the application of a
proposed rule. This might be a research question, or the EPA may already have examples it can give. In
rural or western United States/Alaska areas, population densities are lower, so census tracts are larger, and
the proximity rule might need to be larger. The EJ population identification might differ as well, especially if
income and race are the primary filters. The definition of hotspot might differ in urban and rural settings,
which is important because the intent of EJ is not to simply shift new source permits to low-population
rural areas.
Recommendations:
The SAB suggests that the definition of a comparison group be clarified.
The SAB recommends that the EJTG instruct analysts to be transparent about the choices they make
when deciding which control variables to employ that implicitly define the control group.
The SAB recommends that the discussion on the selection of a comparison group in Section 5.4.2 of the
EJTG be updated to include the approaches described in papers by Mohai and Saha.
The SAB recommends that both rural and urban examples be considered in the application of proposed
rules.
3.9.3. Identifying counterfactual distributions
As presented on page 40 of the EJTG, in order to assess the "differences in the baseline incidence [of
environmental harms or risks] and determine if the distribution increases or decreases the differences"
some information is required about:
the baseline (pre-regulation) environmental conditions for the EJ group and for a comparison
group;
the counterfactual (projected-yet-absent regulation) environmental conditions for the EJ group
and for a comparison group; and
the projected (post-regulation) environmental conditions for the EJ group and for a comparison
group.
In principle, this presents no less than six different distributions. The counterfactual and the projected
distributions need to be known or assumed in order to identify the rule's impact. The baseline
distribution (per p.40 of the EJTG) needs to be known to fully assess the justice of that impact. EJ
assessments will typically require comparing distributions between at least two groups for each of those
scenarios (baseline, counterfactual, projected). In practice, the analysis may be much simpler, perhaps
41
-------
because the baseline and counterfactual distributions are assumed to be the same. In addition, the
"baseline" definition in the Glossary (p.54) confuses matters by defining the baseline as both the status
quo and as the counterfactual.
Recommendation:
The SAB recommends that the EJTG direct the analyst to be transparent about how the differences
across groups are identified for each scenario in the EJ analysis.
3.9.4. Defining the scope of analysis
This section simply mentions the possibility that rules may require EJ analyses at a sub-national level.
Because the result of an analysis of difference in impact is significantly affected by the selection of
geographic extent (e.g., Baden et al., 2007), the analyst should make certain that the specific scope
selected for analysis is policy-relevant or rule-relevant. If there is no clear guidance as to scope from the
rule, sensitivity analysis would be appropriate to identify the impact on the results of any "boundary
effect" - the SAB recommends that this be also discussed in 5.4.2 and added as a recommendation.
Further, the SAB recommends that Section 5.4.2 be also expanded to explicitly address temporal scope.
This is partly wrapped up in decisions about identifying counterfactuals (how things would be in the
absence of the rule) and how far into the future to project post-rule. Regardless, analysts should have
clear guidance on where to place temporal bounds in the analysis. Such bounding implicitly defines
what sort of behavioral responses to regulations are included in the analysis (e.g., reducing emissions,
switching technologies, averting behavior and defensive investments, relocation of polluting activities or
receptors/residences), which are likely consequential for results of EJ analyses. EJ analysts should be
given more guidance and less discretion here. Insofar as a rule's Regulatory Impact Analysis (RIA)
prescribes the temporal boundaries for analysis (per Section 5.2, p.40), EJ analysts should be invited to
report on the likely implications of using these boundaries.
Recommendation:
The SAB recommends that analysts be instructed to transparently present results of sensitivity analyses
to identify the impact of geographic and temporal boundary choices.
3.9.5. Spatially identifying and aggregating effects
The wording of this section does not provide clear guidance to analysts with regard to resolving
differences in spatial resolution between two or more geospatial datasets, in particular on how to avoid
two classic "bad geospatial practices" in this regard: ecological fallacy (the impact of spatial resolution
on conclusions one can accurately draw) and the modifiable areal unit problem (the source of bias that
can impact statistical tests if data are aggregated incorrectly). The SAB suggests that a list of best
geospatial practices be added to the EJTG to provide guidance on these issues.
The SAB also recommends that the EJTG provide useful guidance on data sources by expressing a
preference for certain types of data - notably individual-level data (rather than spatial aggregates) and
exposure data (rather than crude proxies and buffers-around-sources). Some of these issues appear in
Text Box 5.3, where a presumption of aggregated data remains even in the "data rich" context, but
guidance on ecological fallacy and aggregating effects belongs in Section 5.4.3.7
7 The preference for finer-scale data (p.43) should be tempered and guided by context. Finer resolution allows detection of more
and smaller hotspots, but does not imply better measures of exposure. Highly resolved environmental quality data puts more
pressure on data describing where and when receptors (people) are. Eventually groupings can get small enough that inequities in
risk become inevitable.
42
-------
This section of the EJTG also suggests that analysts use buffer circles in a GIS to select and aggregate
census-related data, often called a "cross-walking" procedure, but does not include the various ways that
this can be done. These include: selecting tracts that intersect the buffer circle, selecting tracts with
centroids (or geographic center, depending on which is used by the analyst) captured by the buffer
circle, or using the geo-processing capability of the GIS to actually intersect the buffer circle with the
tract polygon, and transferring attributes from tracts to the buffer area using area-weighting or
population-weighting. All of these methods have been used in the EJ research literature, and all carry
assumptions that need to be acknowledged by the analyst and the specific method selected that is most
appropriate to the analysis that is being conducted.8
Recommendations:
The SAB recommends that the EJTG provide a list of "best geospatial practices" as guidance for
analysts.
The SAB recommends that the EJTG provide a list of GIS data sources for analysts to use.
The SAB recommends that the EJTG provide clearer guidance on the methods used to select and
aggregate census data in EJ analyses.
3.9.6. Interpreting geographic patterns
Section 5.4.4 describes instances where interpreting a geographic pattern can be difficult because many
metrics are correlated, and the relative role or strength of various determinants is not known. The EJTG
notes "regression techniques are able to partially control for these factors," but offers the analyst no
specific direction. The EJTG could usefully cite research here, which the analyst could use to examine
how other researchers have approached this problem using various multivariate techniques; for example,
Boer et al. (1997); Sadd et al. (1999); Pastor et al. (2001, 2004a, 2004b, 2005, 2006); and references
therein.
Recommendation:
The SAB recommends that the EJTG provide additional citations that can assist analysts in interpreting
geographical patterns using various multivariate techniques.
3.10. Analysis of the Distribution of Costs
Charge Question 11. Is there sufficient guidance on when and how to conduct an analysis of the
distribution of costs? Is the guidance associated with distribution of costs appropriate?
The SAB does not agree that there is sufficient or appropriate guidance on when and how to conduct an
analysis of the distribution of costs. The solution to the problem of inadequate guidance on costs is to be
clearer about the conscribed nature of the EJTG and point to other sources/parts of the rule-making
process for a discussion on costs.
8 As the GIS of choice for federal agencies is the ESRI, Inc. software suite, a good summary review can be found in the ArcGIS
help files: http: / / resources.esri.com/help / 9.3 / a regis engine /java/gp toolref/geoprocessing/proximity analysis.htm
43
-------
If EJ is about disproportionate impacts to health (or exposure to environmental stressors), and not about
total well-being,9 then this interpretation can be justified by the view that health (or a clean
environment) is a merited outcome and everyone should experience at least some minimum benefit.
Under this definition, it may be inappropriate to consider costs. SAB panel members raised concerns
that consideration of costs could be used as an argument against protecting the health of particular EJ
communities. One dimension of this concern is the potential that in traditional cost-benefit approaches,
measures of benefits utilizing a willingness-to-pay metric will lead to low benefit measures due to the
lower ability to pay in EJ communities. The discussions regarding how to fix the guidance on costs
highlighted the complexities involved in providing guidance on costs. Some of these issues are
addressed below.
3.10.1. Sufficiency
One potential response to these concerns would be a determination that it is beyond the scope of the
EJTGto adequately address these concerns. The second approach, and one proposed by the SAB, is to
expand the treatment of costs in the EJTG to provide sufficient guidance on when and how to conduct an
analysis of the distribution of costs. Section 5.5.1 states that the need to undertake an exploration of the
distribution of costs should be assessed on a "case by case" basis, and then proceeds to give examples of
when such an analysis is warranted based on characteristics of the case or assumptions about the effect
of a rule. This section also states that "Data or methods may not exist to fully examine the distributional
implications of costs across population groups of concern." In this second case, the reader gets the
impression that the notion of "difficult to perform an analysis" is the sufficient condition for
"unnecessary to perform an analysis."
While it is entirely possible that data constraints may prevent a serious analysis of cost distributions in
many instances, feasibility is a different rationale than relevance or appropriateness. The sentence in the
middle of page 51 of the EJTG that states cost analyses are not always necessary, combined with its
footnote (58), misleads the analyst because it confuses necessity with difficulty in measurement and
infeasibility. Adding further confusion, the following sentence implies that considering cost distributions
might not be necessary because they are evenly distributed. An analyst would get better guidance if the
entire paragraph up to the word "Whether" were deleted. Better still, the EJTG should more clearly
identify when cost analyses are appropriate.
Recommendations:
The SAB recommends that the EJTG be revised to include the following specific suggestions regarding
the question of when to conduct a cost analysis:
Where analyses are omitted, the analyst should be required to document the basis for the
exclusion. Specifically, was the exclusion the result of a qualitative or quantitative determination
based on attributes of the rule or its assumed effects, or was it based on the lack of relevant data
or methods?
The EJTG lists examples of cases where consideration of the distribution of costs is warranted,
including: ".. .costs to consumers may be concentrated among particular types of households
such that they impose a noticeable burden...; identifiable plant closures or relocation of
facilities; or behavioral changes in response to a rule or regulation." This list is helpful, but it
9 In indigenous communities, personal and community health are inseparable from environmental quality, culture, and many
other factors (Donatuto et al., 2011)
44
-------
exemplifies the need to have more examples and case studies available for analysts. The list does
not provide enough context/rationale to guide (potentially) similar analyses.
Under Section 5.5.2 (Other Impacts), in providing guidance on estimating non-health endpoints
the analyst is instructed: "When data are available, analysts should use them in the evaluation."
This is another example where if the analyst can propose scenarios where non-health endpoints
may be important but data availability prevents or limits analysis, the analyst should be
instructed to note this limitation.
3.10.2. Clarity
With regard to whether the guidance associated with distribution of costs is appropriate, the SAB
identified several areas that need further clarification. One key set of concerns relates to issues of scope
and the types of responses and/or adjustments that will be accounted for in the analysis. The SAB
recommends that these issues which can be categorized into two main areas, short run versus long run
analysis and general versus partial equilibrium analysis, be further clarified as follows.
Short Run versus Long Run Analysis
What time frame should be used in cost analyses? This is important because the distributional effects
can change over time. As an example, consider the distribution of costs associated with the requirement
for additional pollution controls on automobiles. Such regulatory changes cause the cost of cars to go up.
This burden initially falls on higher income individuals (who buy cars more rapidly) over time, lower
income people will possibly buy new cars, or experience a cost, in the long-run, as the price impacts in
the new car market spill over into the used car market. Thus, these controls may become more regressive
over time.
Conversely, what about regulations that impose upfront costs on consumers that are "paid back" over
time? An example is the requirement to purchase Low Carbon/High Efficiency appliances. There would
potentially be large upfront costs, but likely long-run savings. The time component and personal
behavior/choice are important here.
General versus Partial Equilibrium Analysis
In general, an accurate accounting of the distributional cost impacts will require a general equilibrium
analysis. The EJTG should provide guidance on if and when a partial versus general equilibrium
analysis will be required. Another way to look at this issue is in terms of what set of costs should be
considered. Should only first order costs be considered? Should second order costs also be considered,
and the costs to whom? If there are guidance documents that currently exist which answer these
questions, analysts should be instructed to use them.
The SAB recommends that this section of the EJTG should highlight what other considerations are
important and specific to EJ analyses (e.g., those particularly likely to arise in assessing cost) and to
define the scope or put some bounds on what the cost analyses should consider. One possible suggestion
is that direct consumer costs would be appropriate for EJ analyses but the general equilibrium or second-
order cost effects would not be expected to be covered by an EJ analysis. Factors such as compliance,
averting behavior and precision/range of estimates also need to be addressed. For example, should
analysts assume complete compliance? If not, how should this be reflected in the analysis? How should
analysts address the potential role of averting behavior? If a policy induces a change in behavior, where
and how does that get taken into account? A related issue is the treatment of adjustment costs. How
45
-------
should they be accounted? Furthermore, the EJTG should provide guidance on how to characterize the
uncertainty inherent in cost estimates. While guidance on how to address these considerations may be
available in other agency documents; issues that are specific to and may differ for EJ analyses should be
highlighted and examples provided in the EJTG.
Recommendations:
The SAB recommends that the EJTG should provide guidance on the time frame that should be used in
cost analyses as well as guidance on if and when a partial versus general equilibrium analysis will be
required.
The SAB recommends that the EJTG should provide guidance on how to characterize the uncertainty
inherent in cost estimates.
3.11. Key Methodological or Research Gaps
Charge Question 12. What are the key methodological or data gaps specific to considering EJ in
regulatory analysis? Which factors should be prioritized in the near-term to improve how EPA
considers potential EJ concerns in regulatory analyses?
The EJTG core writing team led by the Office of Environmental Justice, Office of Policy, and Office of
Research and Development has done an excellent job surveying and querying EPA personnel and the
interested public on methodological and research gaps and needs in the EJ area. The team identified long
and short-term research priorities among various EPA offices and regions that they presented in Tables 1
and 2, respectively (shown below). The SAB has identified additional research planning, staffing needs,
data gaps, and methodological needs which can strengthen the EJTG.
3.11.1. Research Planning
To a significant degree, gaps noted in public comments reflect the research gaps and priorities expressed
by agency personnel. These include better distribution of air monitoring locations, use of cumulative
impact assessments, use of appropriate data sources and maintenance of privacy, more complete
demographic information, identification of non-chemical stressors, and the use of qualitative data in an
appropriate manner.
In examining these gaps, the SAB noted that the short-term and long-term needs expressed are quite
similar, suggesting the need for a greater degree of strategic thinking on longer-term priorities. There is
a danger that without careful alignment of immediate needs and longer-term aims, there may be
considerable misdirection in research that may require frequent readjustment of objectives and scope.
One approach, common in the EPA's Office of Research and Development (ORD), is to differentiate
between short-term "outputs" and longer-term "outcomes," the latter providing guidance as the results of
short-term projects become available. Only the "framework for using available data" in Table 12-2
appears to address this need (without further explanation), while other long-term priorities mimic those
presented as short-term. Longer-term priorities that could emerge might come about through work with
other agencies that are concerned with demographic and behavioral trends (e.g., the Departments of
Labor, Commerce, and Homeland Security), and agencies that address long-term human and ecological
health needs, for example, the Department of Health and Human Services' National Institutes of Health
(NIH), Food and Drug Administration (FDA), and Centers for Disease Control and Prevention (CDC).
For example, the FDA's Office of New Drugs has a wealth of experience with drawing conclusions
from limited sample size and quantifying differing reactions to an agent according to race or ethnicity. A
"brainstorming" session with appropriate staff members in these agencies might reveal protocols,
46
-------
practices and reference materials valuable to the ORD/risk assessment staff developing cumulative risk
and impact guidance. Similarly, CDC expertise in biomonitoring might be used to better assess
community exposures.
Recommendation:
The SAB recommends a greater degree of strategic thinking on longer-term priorities since the short-
term and long-term needs expressed are quite similar.
Table 1. Short-term Research Priorities for EJ Identified by the EPA
GENERAL RESEARCH PRIORITIES-TOP 5 SHORT
TERM PRIORITIES (DRAFT)
Offices identifying priority
Analysis: chemical and non-chemical stressors.
Cumulative effects, behavioral effects, costs, health impacts
OAR; OCHP; OCSPP; OP.
OW; Regions
Data gaps: chemical and non-chemical stressors, cultural,
product use, workplace characteristics, finer resolution air
quality data
OAR; OCHP; OCSPP; OEJ;
OP; OW; REGIONS
Review of criteria used to characterize EJ communities
OAR; OCSPP; OW
Methodology: distribution of risk, receptor approach
different types of rules, and validity of assumptions in BCA
OAR; OCSPP; OEJ;
OSWER; OW
Improve tools: behavioral responses, combined risk
including non-chemical stressors, IRIS for system specific
endpoints.
ORD; OSWER
Table 2. Long-term Research Priorities for EJ Identified by the EPA
GENERAL RESEARCH PRIORITIES - TOP 5 LONG
TERM PRIORITIES (DRAFT)
Offices identifying priority
Data gaps: chemical, non-chemical, cultural, product use,
demographic characteristics, health outcomes, group dose
response, workplace characteristics, finer resolution air
quality data, new enviromnental burdens
OAR; OCHP; OCSPP; OEJ;
OW
Analysis: consistent analytical approach other routes of
exposure, health indicators
OAR; OCSPP; OW;
REGIONS
Framework, guidelines for using available data
OCSPP
Methodology: standardization of metrics, differential burdens,
disaggregating BCA for EJ analysis
OAR; OCHP; OEJ; OP;
ORD; OW; REGIONS
Improve tools: for policy makers, vulnerability by life stage,
characterizing vulnerable communities.
OAR; ORD
47
-------
3.11.2. Staffing Needs
The SAB is concerned that the EPA may not have the full complement of expertise among its research
scientists to fully address EJ needs and priorities. To a significant extent the agency relies on its cadre of
economists (behavioral economists) to discern complex human behaviors. While this is certainly a valid
approach, insights from the learning, sociological, anthropological and psychological science
communities might also provide complimentary expertise that could result in new methods of data
management and interpretation, and more robust ways of treating uncertainty. This is especially the case
as long-term trends are considered. Recruitment of appropriately trained postdoctoral researchers,
temporary inter-agency transfers, community-based participatory researchers, and creative use of the
STAR research program would assist in meeting short-term personnel needs. Regarding the latter, the
SAB suggests a well-structured set of EJ-focused research areas that might bring new thinking into the
agency, and result in a greater array of information with which to work.
3.11.3. Data Needs
During the SAB panel meetings, there were frequent concerns expressed by agency personnel that data
extent, quality, and availability were often insufficient to carry out the EJ mission; i.e., that the agency is
"data poor." While this may be the case in some instances, for example, with respect to the locations
and numbers of air quality monitoring stations or low dose exposures and their impact as contributors
and drivers of riskthe SAB is concerned that other data sources and methods of analysis are being
overlooked or not fully integrated into analyses. Quantitative structure-activity relationships (QSAR),
Relative Potency Factor, and EPA's NexGen methods can all be used to fill data gaps. In addition, the
investigation of EJ issues requires examining the problem both from the behavioral and social equity
perspectives and the environmental risk perspective. Making use of surrogate and metadata, and the
application of advanced methods of cyber-analysis (data mining, ontological matching, and
disambiguation) to build more robust and useful data sets are ways of transforming a "data poor"
problem into one which is "data rich."
The SAB agrees that ensuring data sufficiency, accuracy, and appropriateness is essential for EJ
analyses, particularly for the detection of EJ-relevant "hotspots." An important aspect of this involves
using more, optimized monitoring locations, models that better incorporate the implications of
monitoring locations, and techniques for better integrating dispersion models for areas not directly
monitored. Better (and more accessible) techniques for including model errors into EJ analyses (which
typically use questionable proxies for environmental quality or risks at a particular location without
formally accounting for error in that measurement) would help. Even basic data indicating the spatial
extent of regulated hazards could bring substantial improvements.
Beyond better characterizations of pollutant levels in space and time, a concomitant concern is better
characterization of pollutant receptors in space and time. The convenience of census data, and the new
limitations brought on by the shift to American Community Survey (ACS) data with less granular
geographic range, bring with it important limitations in assessing actual exposure. The SAB notes that
improving the spatial precision in measuring risks should not come at the expense of improved temporal
precision and a better understanding of how this contributes to the overall goal of improving the
characterization of exposure. Accordingly, the EPA should invest in research to better understand actual
exposure, i.e. reflective of how receptors actually behave, rather than reliance on standard models of
fixed behavior. Shifting empirical analyses to aggregate units (e.g., census tracts) and relying on location
of residence (rather than time spent outdoors or using indoor and workplace risks) departs from unbiased
estimates of actual exposure and adds to the uncertainty associated with the analysis. Technologies, such
as the incorporation of cell phone tracer data (anonymously), could add an element of mobility to risk
48
-------
analyses that is currently lacking. Similarly, the Longitudinal Employer-Household Dynamics10 program
holds great promise for improving spatial and temporal precision in measuring receptors' location and
travel habits. Efforts to make use of this information - and to promote more researchers' access to the
data for these purposes should be encouraged. Additionally, while better use of geospatial and spatial
econometric models is needed, the EPA might facilitate their use by analysts by making spatio-temporal
models easier to access, especially those with limited dependent variables.
The SAB also recommends that group-specific estimates of dose-response relationships, as well as
estimates of workplace and indoor exposures, be improved. The rigor and quality of EJ analyses will be
enhanced by standardization of the use of the block group level census for demographics and as the
spatial unit of analysis, as well as the comparison of environmental stressors and their impacts at 1 and 3
kilometer radii for proximity analyses. TRI data, commonly used in EJ analyses, could be released in
ways that include more information about the data itself (e.g., when releases are estimated or measured)
and in ways to allow more "accurate" use of the data. The ubiquity and ease-of-use of TRI data also
leads to misuse, and the EPA can do more to improve how these data are used. Efforts to better model
"hotspots of pollutant receptors" should parallel better models of pollutant hotspots (i.e., Does more
precise identification of pollutant hotspots increase or decrease the bias in estimated exposure?).
Because the field of EJ is fairly dynamic, it would be useful to develop a repository of relevant empirical
methods and analytical toolkits as well as geospatial/temporal data, including environmental and
psychosocial stressors, facilities, and demographics relative to impacts. Such a repository should include
those which have been created and maintained by other agencies or stakeholders. The repository would
be expected to grow with the field and facilitate more and better analyses, including external review of
the EPA's actions. Each EJ analysis can also be used to identify data needs for this repository, if analysts
are required to document the data gaps and uncertainties which shaped their analysis.
3.11.4. Methodological Needs
The SAB strongly encourages the EPA to work toward the incorporation of cumulative impacts and
multiple facility proximity in its analyses of its proposed rules and regulations as they pertain to
environmental justice and identification of disproportionate impact. The SAB understands the challenges
posed by cumulative assessments, and acknowledges that, practically speaking, a complete and robust
assessment might not be feasible until further methods and tools are developed and data become
available. However, examples of cumulative health impact assessments, an alternative to traditional risk
assessments that take into account both quantitative and qualitative data, continue to be documented
(Dannenberg et al., 2008). Further, with current knowledge and methods available, a trained practitioner
should be able to identify limitations in their analysis and consider characterizing (both quantitatively
and qualitatively) the degree of uncertainty introduced short of a complete impact assessment.
While understanding the need for national guidance for EJ methodologies, the SAB notes that state,
local, and community level data and assistance are essential for an accurate EJ analysis. In addition, it
often will be necessary to engage community leaders, EPA regional offices, and others in dialogue to
fully understand what information they can contribute to the analysis and what exposure or other
concerns a community might have. For example, it is not clear that proposed methodologies are
adequate for evaluating subsistence communities (whether tribal or other rural populations) where
dietary needs are met largely through hunting and fishing. To this end, the SAB recommends adopting
one of the suggestions made in the public comments regarding funding pilot projects with states, local
governments, and communities to develop and test mechanisms for sharing data and information and
10 See http://lehd.ces.census.gov/
49
-------
engaging communities in order to inform an EJ analysis. Section 4.3.2.3 of the EJTG should include a
specific recommendation of early, thorough, and culturally and linguistically competent community
involvement in order to identify and address relevant data gaps.
50
-------
REFERENCES
Baden, B.M., Noonan, D.S., and Turaga, R.M. (2007). Scales of Justice: Is there a Geographic Bias in
Environmental Equity Analysis? Journal of Environmental Planning and Management 50(2):
163-185.
Banzhaf, H.S., Walsh, R.P. (2008). Do people vote with their feet? An empirical test of Tiebout's
mechanism. The American Economic Review 98(3): 843-863.
Berg, B.L. and Lee, H. (2012). Qualitative Research Methods for the Social Sciences. 8th Edition,
Pearson. ISBN 9780205234967.
Boer, J., Pastor, M., Sadd, J., and Snyder, L. (1997). Is There Environmental Racism? The
Demographics of Hazardous Waste in Los Angeles County. Social Science Quarterly 78(4): 793-
810.
Bowen, W. (2001). Environmental Justice through Research-Based Decision Making. New York:
Garland.
California Comparative Risk Project (1994). Toward the 21st Century: Planning for the Protection of
California's Environment. Office of Environmental Health Hazard Assessment, Berkeley, CA.
Available at: http://oehha.ca.gov/multimedia/pdf/compriskl994.pdf
California Office of Environmental Health Hazard Assessment (2002). Hot Spot Unit Risk and Potency
Values. Available at: http://www.oehha.ca.gov/air/hot spots/pdf/TSDlookup2002.pdf
Corburn, J. (2002). Environmental justice, local knowledge, and risk: the discourse of a community-
based cumulative exposure assessment. Environmental Management 29(4): 451-466.
Dannenberg, A.L., Bhatia, R., Cole, B.L., Heaton, S.K., Feldman, J.D., and Rutt, C.D. (2008). Use of
Health Impact Assessment in the U.S. 27 Case Studies, 1999-2007, Am JPrevMed34(3): 241-
256.
Depro, B.M., Timmins, C., O'Neil, M. (2012). Meeting Urban Housing Needs: Do People Really Come
to the Nuisance? NBER Working Paper No. 18109.
Donatuto, J.L., Satterfield, T.A. and Gregory, R. (2011). Poisoning the body to nourish the soul:
Prioritizing health risks and impacts in a Native American community, Health, Risk & Society
13(2): 103-127.
Fossey, E., Harvey, C., McDermott, F., and Davidson, L. (2002). Understanding and evaluating
qualitative research. Australian and New Zealand Journal of Psychiatry 36(6): 717-732.
Gould, L., Cummings, B.J. (2013) Duwamish Valley Cumulative Health Impacts Analysis. Seattle, WA:
Just Health Action and Duwamish River Cleanup Coalition/Technical Advisory Group.
Greenberg, M., Haas, C., Cox, Jr, A., Lowrie, K., McComas, K. and North, W. (2012). Ten Most
Important Accomplishments in Risk Analysis, 1980-2010. Risk Analysis, An International
Journal 32(5): 771-781.
51
-------
Harper, B., Harding, A., Waterhous, T., and Harris S. (2007). Traditional Tribal Subsistence Exposure
Scenario and Risk Assessment Guidance Manual. Corvallis: Oregon State University.
http:// superfund.oregonstate.edu/outreachevents;
and https://app.box. com/shared/70r3 5 79u5 gh7 vsdugfv7
Hicken, M., Gragg, R., and Hu, H. (2011). How Cumulative Risks Warrant A Shift In Our Approach To
Racial Health Disparities: The Case of Lead, Stress, and Hypertension. Health Affairs 30(10):
1895-1901.
Klimisch, H. J., Andreae, M. et al. (1997). A Systematic Approach for Evaluating the Quality of
Experimental Toxicological and Ecotoxicological Data. Regulatory Toxicology and
Pharmacology 25(1): 1-5.
Krieger, N. (2001). Theories for social epidemiology in the 21st century: an ecosocial perspective.
International Journal of Epidemiology 30(4): 668-677.
Lavelle, K. S., Schnatter, A.R. et al. (2012). "Framework for integrating human and animal data in
chemical risk assessment." Regulatory Toxicology and Pharmacology 62(2): 302-312.
McComas, K. (2003). Public meetings and risk amplification: a longitudinal study. Risk Analysis 23:
1257-1270.
Mohai, P. and Saha, R. (2006). Reassessing Racial and Socioeconomic Disparities in Environmental
Justice Research. Demography 43(2): 383-399.
Mohai, P. and Saha, R. (2007). Racial Inequality in the Distribution of Hazardous Waste: A National-
Level Reassessment. Social Problems 54(3): 343-370.
Money, C. D., Tomenson, J.A. et al. (2013). A systematic approach for evaluating and scoring human
data. Regulatory Toxicology and Pharmacology 66(2): 241-247.
Morello-Frosch, R., Pastor Jr., M., Porras, C. and Sadd, J. (2002). Environmental justice and regional
inequality in southern California: implications for future research. Environmental Health
Perspectives 110(S2): 149-154.
National Research Council (2007). Toxicity Testing in the 21st Century: A Vision and A Strategy.
Washington, D.C: The National Academies Press.
National Research Council (201 la). Improving Health in the United States: The Role of Health Impact
Assessment. Washington, DC: The National Academies Press.
National Research Council (201 lb). Sustainability and the U.S. EPA. Washington, DC: The National
Academies Press.
National Research Council (2014). Review of EPA's Integrated Risk Information System (IRIS) Process.
Washington, DC: The National Academies Press.
52
-------
NEJAC (2004) Ensuring Risk Reduction in Communities with Multiple Stressors: Environmental Justice
and Cumulative Risk/Impacts. Available online at:
http://www.epa.gov/environmentaliustice/resources/publications/neiac/neiac-cum-risk-rpt-
122104.pdf
NEJAC (2010). Nationally Consistent Environmental Justice Screening Approaches. A Report of
Advice and Recommendations of the National Environmental Justice Advisory Council.
Available online at:
http://www.epa.gov/compliance/environmentaliustice/resources/publications/neiac/ei-screening-
approaches-rpt-2010.pdf
NEJAC (2014) Recommendations for Integrating Environmental Justice into the EPA's Research
Enterprise. Available online at:
http://vvvvvv.epa.gov/compliance/environmentaliustice/resources/publications/neiac/neiac-
research-recommendations-2014. pdf
Noonan, D.S. (2008). Evidence of Environmental Justice: A Critical Perspective on the Practice of EJ
Research and Lessons for Policy Design. Social Science Quarterly 89(5): 1154-1174.
Pastor, M., Morello-Frosch, R., Sadd, J. (2006) Breathless: Pollution, Schools, and Environmental
Justice. California. Policy Studies Journal 34 (3): 337-362.
http://citc.ucsc.edu/docs/Breathless 03 with figures unlinked.pdf
Pastor, M., Sadd, J., and Hipp, J. (2001). Which Came First? Toxic Facilities, Minority Move-in, and
Environmental Justice. Journal of Urban Affairs 23(1): 1-21.
Pastor, M., Sadd, J., and Morello-Frosch, R. (2004a). Waiting to Inhale: The Demographics of Toxic Air
Releases in 21st Century California. Social Science Quarterly 85(2): 420-440.
Pastor, M., Sadd, J., and Morello-Frosch, R. (2004b). Reading, Writing, and Toxics: Children's Health,
Academic Performance, and Environmental Justice in Los Angeles. Environment and Planning
C: Government and Policy 12 (1): 271-290.
Pastor, M., Sadd, J., and Morello-Frosch, R. (2005). The Air is Always Cleaner on the Other Side: Race,
Space, and Air Toxics Exposures in California. Journal of Urban Affairs 27(2): 127-148.
Popay, J., Rogers, A., and Williams, G. (1998). Rationale and standards for the systematic review of
qualitative literature in health services research. Qualitative Health Research 5(3): 341-35 1.
Rhomberg, L.R., J.E. Goodman, et al. (2013). A survey of frameworks for best practices in weight-of-
evidence analyses. CritRev Toxicology 43(9): 753-84.
Rinquist, E. (2005). Assessing Evidence of Environmental Inequities: A Meta-Analysis. Journal of
Policy Analysis and Management 24(2): 223-247.
Rooney, A.A., Boyles, A.L., Wolfe, M.S., Bucher, J.R., and Thayer, K.A. (2014). Systematic Review
and Evidence Integration for Literature-Based Environmental Health Science Assessments.
Environmental Health Perspectives 2014 Apr 22. [Epub ahead of print]
53
-------
Sadd, J., Pastor, M., Boer, J., and Snyder, L. (1999). Every Breath You Take The Demographics of
Toxic Air Releases in Southern California. Economic Development Quarterly 13(2): 107-123.
Schwartz, J., Glass, T., and Bellinger, D. (2011). Exploring potential sources of differential vulnerability
and susceptibility in risk from environmental hazards to expand the scope of risk assessment.
American Journal of Public Health 101(1): S94-S101.
Sexton, K. and Linder, S. (2011). Cumulative risk assessment for combined health effects from chemical
and nonchemical stressors. American Journal of Public Health 101(1): S81-S88.
Stern, P.C. andFineberg, H.V. eds. (1996). Understanding risk: Informing decisions in a democratic
society. Washington, DC: National Academies Press.
Tesch, R. (2013). Qualitative Research. Routledge Falmer. New York, New York.
Turaga, R.M.R., Noonan, D., and Bostrom, A. (2011). Hot Spots Regulation and Environmental Justice,
Ecological Economics 70(7): 1395-1405.
U.S. EPA (2003). Framework for Cumulative Risk Assessment. EPA/100/R-14/001. Available online at:
http://www.epa.gov/raf/publications/pdfs/frmwrk cum risk assmnt.pdf
U.S. EPA (2008). Lead Renovation, Repair and Painting Rule, Final. Available online at:
http://www.gpo.gOv/fdsys/pkg/FR-2008-04-22/html/E8-8141.htm
U.S. EPA (2010) Action Development Process- Interim Guidance on Considering Environmental Justice
During the Development of an Action. Available online at:
http://www.epa.gov/environmentaliustice/resources/policv/considering-ei-in-rulemaking-guide-
07-2010.pdf
U.S. EPA (2013). Next Generation Risk Assessment: Incorporation of Recent Advances in Molecular,
Computational, and Systems Biology. External review draft. Available online at:
http://ofmpub.epa.gov/eims/eimscomm.getfile7p download id=515861
U.S. EPA (July 2014a). EPA Policy on Environmental Justice for Working with Federally Recognized
Tribes and Indigenous Peoples. Available online at:
http://www.epa.gov/environmentaliustice/resources/policv/indigenous/ei-indigenous-policv.pdf
U.S. EPA (2014b). Framework for Human Health Risk Assessment to Inform Decision-Making.
Available online at: http://www.epa.gov/raf/files/hhra-framework-final-2014.pdf
U.S. EPA SAB (Science Advisory Board) (1999). Integrated Environmental Decision-Making in the
Twenty-First Century. EPA-SAB-EC-99-018.
http://vosemite. epa.gov/sab/sabproduct.nsf/492FElA83 86469798525719300559027/$File/ec990
18.pdf
54
-------
APPENDIX A. Charge to the SAB
Revised11 Charge Questions for the SAB review of
EPA's Technical Guidance for Assessing Environmental Justice in Regulatory Analysis
Overall Impressions
The Technical Guidance for Assessing Environmental Justice in Regulatory Analysis (EJTG) provides
EPA economists, risk assessors and other analysts with information on how to assess potential
environmental justice (EJ) concerns during the development of a regulatory action. It is intended to
introduce consistency and rigor to the analytic consideration of EJ, while maintaining flexibility in how
analysts implement the guidance.
L Please provide your overall impressions of the clarity and technical accuracy of the EJTG for
analyzing and presenting quantitative or qualitative information on potential environmental
justice concerns in the development of EPA regulations.
Key Questions for Analysts
The EJTG suggests that if quantitative risk and benefit analysis is done in support of the rule, analysts
should rely on these data to do a quantitative EJ assessment when feasible. The level of quantitative
analysis is expected to vary by regulation and be affected by data, analytic, or other constraints. If
quantified benefit or risk information is not available then a qualitative EJ analysis is still expected.
2. a- Are these directions appropriate? Do they strike the right balance between developing
information that is useful to the decision making process and the cost (time, resources, data
constraints) of doing quantitative EJ assessments?
b- Please provide advice on methods and best practices for conducting rigorous, high-quality EJ
analyses, both quantitative and qualitative, that may be conducted in support of a national rule
(including data needs or other issues associated with such assessments).
3. Section 1.1 presents 5 key questions analysts should address when analyzing the environmental
justice considerations during the development of a regulation. Are these questions clear and
appropriate for considering EJ during the development of a regulation?
Key Recommendations (Section 1.2)
The EJTG makes six recommendations to ensure consistency, rigor and quality across assessments.
4. Are the six analytic recommendations listed in Section 1.2 appropriate and comprehensive? Are
they consistent with the state of the literature while providing flexibility to EPA program offices
in the analytic consideration ofEJin the development of a regulation?
5. Are there any analytic recommendations that should be added? Any that should be removed?
^ ^ The EPA released for public comment its Draft Technical Guidance for Assessing Environmental Justice in Regulatory Analysis on May
9, 2013 (see httvs://www.federalreeister.eov/articles/2013/05/09/2013-11165/technicaleuidance-for-assessine-environmentaliustice-in-
reeulatorv-analvsis). As a result, EPA received a number of comments (see docket # EPA-HQ-OA-2013-0320 at
http://www.regulations. gov-). After considering these comments, the EPA Office of Policy has revised the charge questions posed to the
SAB panel to include an additional question (#2b).
A-l
-------
Differences and Disproportionate (Section 2)
The EJTG distinguishes between analytically defined differences in impacts and making a determination
of disproportionate impacts. It also suggests 6 types of information that may be useful to the decision
maker for determining whether differences are disproportionate and may warrant Agency action
(Section 2.4).
6. Is the description of differences in impacts and disproportionate impacts clear and do reviewers
agree with this distinction? Are the types of data listed to aid the decision maker helpful? Are
there other categories of data or information that should be added to this list?
Section 3
Section 3 provides a brief overview of the contributors and drivers of Environmental Justice. This
overview is intended to provide analysts with some considerations that might drive the analytical
decisions used when examining environmental justice for a regulatory decision.
7. Does the discussion of contributors and drivers adequately reflect the state of the literature? Is it
clear and technically accurate? Are there any additional factors that should be included in the
discussion?
Section 4
The Guidance directs analysts to use a series of scoping questions at the planning stages of a human
health risk assessment to integrate EJ into analyses conducted for the rulemaking.
8. Is section 4 clear and technically accurate? Are the scoping questions outlined in Section
4.3.2.1 appropriate? Do the scoping questions adequately identify opportunities for
incorporating environmental justice into a human health risk assessment? Should certain
scoping questions be prioritized at various stages of the risk assessment (e.g. exposure, dose-
response)?
Section 5
This section provides a suite of methods that can be used to assess EJ in the context of a regulation.
9. Does Section 5 provide a clear overview of the methods that could be usedfor considering
environmental justice? Are there other methods that should be added to the discussion?
10. Section 5.4 discusses analytical considerations that may have a significant impact on results. Are
these considerations appropriate for assessing EJ in the context of a regulation? Are there
considerations that should be added/removedfrom the discussion?
Program Offices are advised to consider the distribution of costs associated with implementing a
regulatory option from an EJ perspective when appropriate.
11 .Is there sufficient guidance on when and how to conduct an analysis of the distribution of costs?
Is the guidance associated with distribution of costs appropriate?
A-2
-------
Research Gaps
The EJTG acknowledges that analysis of potential EJ concerns in regulatory analysis is an ongoing and
evolving area and that EPA needs additional research to develop better EJ assessment tools and
methodologies. In answering this question, we ask that you think less about general data or methodology
gaps for conducting quantitative risk or benefits analysis, and instead focus on research gaps that are
specific to evaluating potential EJ concerns.
12. What are the key methodological or data gaps specific to considering EJ in regulatory analysis?
Which factors should be prioritized in the near-term to improve how EPA considers potential EJ
concerns in regulatory analyses?
A-3
-------
APPENDIX B. Select Evidence of Federal Actions' Unequal Impacts
Authors
Regulatory/
issue context
Research question
Finding
Gianessi et al.
Clean Air Act
Do uniform CAA standards yield uniform
No. The poor appear to gain the most.
(1979)
results?
Hird (1990)
CERCLA
Is the cleanup pace or spending at NPL sites
correlated with neighborhood income?
Neither.
Hamilton (1993)
hazardous waste
processing facilities
Did the post-CERCLA regulatory regime
change siting of hazardous waste facilities
No longer drawn to counties with more
minorities; collective action explained more
Gupta et al. (1996)
CERCLA cleanup
decisions
Do demographics affect EPA remedial
decisions?
No. Permanent remedies were not favored
different in minority or poor areas.
Sigman (2001)
CERCLA
Do demographics affect listing, cleanup pace?
Somewhat. Community income affects pace;
progress is faster with more Hispanics.
O'Neil (2007)
CERCLA (listing on
NPL)
Do neighborhood demographics predict the
likelihood of a proposed site getting listed to
the NPL? Did EO 12898 increase equitability
of Superfund program?
Proposed sites in poor and minority tracts are
less likely to be listed.
Post-EO, sites in minority tracts are even less
likely to be listed.
Daley (2007)
CERCLA cleanup
decisions
Does EPA supporting local citizen groups
affect remedial decisions?
Yes. Forming CAGs and TAGs leads to more
health-protective clean-up approaches.
Noonan (2008)
CERCLA cleanup
progress
Does neighborhood race or income predict
NPL deletions?
Deletions uncorrelated with race, less likely in
wealthier areas.
Shimshack and
Ward (2010)
mercury advisories
in fish
Did advisories alter consumption? Differently
for different groups?
Fish consumption fell, even for groups not at risk
Baryshnikova
(2010)
air emissions at pulp
& paper mills
Does regulatory pressure yield inequitable
impacts on plant abatement
Children and high-school dropouts enjoy less
abatement; no difference for minorities and poor
Ringquist (2011)
Clean Air Act
Does the SO2 trading regime transfer pollution
to minority communities?
No. Minority communities received fewer
imports.
Bibliography
Baryshnikova, Nadezhda V. 2010. "Pollution abatement and environmental equity: A dynamic study." Journal of Urban Economics; 68:
183-190.
B-l
-------
Daley, Dorothy. 2007. "Citizen Groups and Scientific Decision-making: Does Public Participation Influence Environmental Outcomes?"
Journal of Policy Analysis and Management, 26(2): 349-368.
Gianessi, Leonard P., Henry M. Peskin, and Edward Wolff. 1979. "The Distributional Effects of Uniform Air Pollution Policy in the United
States." Quarterly Journal of Economics 91: 654-74.
Gupta, Shreekant, George Van Houtven, and Maureen L. Cropper. 1996. "Paying for Permanence: An Economic Analysis of EPA's Cleanup
Decisions at Superfund Sites." Rand Journal of Economics 27(3): 563-82.
Hamilton, James T. 1993. "Politics and Social Costs: Estimating the Impact of Collective Action on Hazardous Waste Facilities." Rand
Journal of Economics 24/1: 101-125.
Hird, John A. 1990. "Superfund expenditures and cleanup priorities: distributive politics or the public interest?" Journal of Policy Analysis
and Management 9: 455-483.
Noonan, Douglas S. 2008. "Evidence of Environmental Justice: A Critical Perspective on the Practice of EJ Research and Lessons for Policy
Design." Social Science Quarterly; 89(5): 1154 - 1174.
O'Neil, Sandra George. 2007. "Superfund: Evaluating the Impact of Executive Order 12898." Environmental Health Perspectives 115: 1087-
1093.
Ringquist, Evan J. 2011. "Trading Equity for Efficiency in Environmental Protection? Environmental Justice Effects from the S02 Allowance
Trading Program." Social Science Quarterly 92(2): 297-323.
Shimshack, Jay P. and Michael B. Ward. 2010. "Mercury advisories and household health trade-offs." Journal of Health Economics 29: 674-
685.
Sigman, Hilary. 2001. "The Pace of Progress at Superfund Sites: Policy Goals and Interest Group Influence." Journal of Law and Economics
44: 315-344.
B-2
-------
APPENDIX C. Additional Recommended Edits
The SAB panel suggests the following edits of the five steps in section 1 (suggested changes are
italicized):
"Assess exposure, relevant health and environmental outcomes, and other relevant effects
separately by population group and within each population group in the baseline, including the
extent of uncertainty in the data and how that uncertainty impacts the results.
Assess differences in these exposures, relevant health and environmental outcomes, and other
relevant effects separately by population group and within each population group in the baseline
for the most recent decade and in the local community (e.g., 1, 3 and 5 mile radius) at highest
risk. Include an assessment of the quality of the data, and uncertainties that impact the results.
Assess exposure, relevant health and environmental outcomes, and other relevant effects
separately by population group for each option, including the extent of uncertainty in the data
and how that uncertainty impacts the results.
Assess differences in these exposures, relevant health and environmental outcomes, and other
relevant effects across population groups for each option, /or the next decade and in the local
community (e.g., 1, 3 and 5 mile radius) at highest risk. Include an assessment of the quality of
the data, and uncertainties that impact the results.
Assess how estimated differences in these exposures, relevant health and environmental outcome
and other relevant effects across population groups increase or decrease as a result of each option
compared to the baseline. Include an assessment of the key variables that account for these
differences and an assessment of the quality of the data for these key drivers. If these drivers are
pronounced in particular types ofplaces, indicate the kinds of locations, and if they appear at
varying times, indicate how soon we might expect to see evidence of their effects. "
The SAB panel suggests the following edits to improve section 2:
Change the title to "Defining Differential Impacts."
Para 1, Cut after .. .policies, programs and activities."
Cut Text Box 2.2
Para 2, Cut after ... implement the Executive Order and also the three bullet points.
Para 3, Cut first sentence.
Para 3, Change sentence 2 to .. .whether there are differential impacts.
Para 3, before "Examples of the kinds..." insert "The decision makers will use this to determine
if the differences are disproportionate and require agency action.
Cut the last two paragraphs.
Points of clarification for section 5
Text box 5.1 is not a good example of qualitative analysis. It is essentially an example of using
secondary data. It is unclear to the reader of what this is an example.
Change the word "statistics" to "data" on p.42.
A new passage in Section 5.3.1 on presenting qualitative data summaries should be added.
The emphasis on statistical significance (p.48 and Section 5.4.4) directs attention to analytical
precision without sufficient attention to accuracy and bias. This should be emphasized more. The
third and fourth concerns in Section 5.4.4 (i.e., non-socioeconomic factors that may have
C-l
-------
influenced stressor source location and using distance as a proxy for exposure), is simply too
limited in appreciating sources of bias and in understanding the complexity using control
variables in multivariate analyses in implicitly defining counterfactuals. Incorporation of co-
stressor information should be encouraged.
The italicized recommendation (p.44 and p.4) should instead say that 'Analysts should follow
best practices appropriate to the question at hand. If infeasible, explain.'
As Section 5.3 indicates baseline health data, the EJTG should also incorporate other triggers
(e.g., for asthma) and co-stressors. This includes incorporating background pollutant
concentrations and other potential confounders like indoor pollution concentrations (especially
insofar as they co-vary with expected regulatory impacts).
The EJTG should instruct analysts to provide qualitative and quantitative characterizations of the
data used in the analysis, including how pollutants effects arise and margins of safety. Analysts
should be advised to provide qualitative or quantitative characterizations of the (differential)
effect sizes identified in their analysis.
The title for Section 5.3 refers to methods to assess EJ concerns, yet frequently refers to
presenting information (which is not the same as assessing). Semantic clarity here would help.
Further, Section 5.3.2.1 poses Visual Displays as an analytic method, yet it is not. This important
subsection should be relocated.
The issue of spatial autocorrelation in inferential statistics is common and important to EJ
analyses, and it should be elaborated upon in Section 5.4.4 (rather that relegated to footnote 56).
The EJTG should include more complete explanations and guidance on how to test for spatial
autocorrelation, as well as guidance on how to properly work with spatially auto-correlated data
to accomplish reliable statistical measures.
Sensitivity analyses should be emphasized more. They should be done for all key assumptions.
(The "when feasible" qualification is not needed on p.4 and p.44.) This is true generally, and not
just a matter of Summary Statistics (p.44) and should not be limited to demographic data
resolution (p.44) or comparison group definitions (p.49). For instance, distances and buffers for
proximity-based analyses typically merit sensitivity analyses consistent with underlying
uncertainty in the model. Analysts should document why sensitivity analyses were not
performed.
Analysts should be guided to characterize uncertainties, especially sampling and modeling
uncertainties that might affect findings. EJ analyses should not portray exposures or population
data as known with certainty when substantial uncertainty exists.
An analytical consideration worth mentioning in Section 5.4 is non-environmental and non-
health related impacts of EPA rules. This could include accounting for impacts on cultural
practices or resources with particularly high value.
More examples would help, as well as mentioning how the examples described in the EJTG
could have been improved by adhering to the guidelines.
Time-activity information, especially as it differs across comparison groups, should be
incorporated into the analysis. The same is true of differential consumption of local natural
resources, whether wild or domesticated. Both types of information are required for exposure
analysis.
The EJTG should include the latest references to conducting risk assessment (e.g., EPA 2013,
National Research Council 2007).
The sentence on page 50 that reads: "Analysts will need to examine what the coefficient [sic]
estimate implies (e.g., how different is poverty across these geographic areas)." This could be
improved to read "Analysts will need to examine what the coefficient estimate implies (e.g., how
C-2
-------
different is poverty across these geographic areas), and summarize and report those differences
in a manner appropriate for policy relevance."
The word "probably" should be struck from Section 5.4.
C-3
------- |