United States Science Advisory EPA-SAB-IHEC-00-018
Environmental Board (1400A) September 2000
Protection Agency Washington DC ww.epa.gov/sab
AN SAB REPORT: THE
DRAFT STRATEGIC PLAN
FOR THE ANALYSIS OF
NATIONAL HUMAN
EXPOSURE ASSESSMENT
SURVEY (NHEXAS) PILOT
STUDY DATA
A REVIEW BY THE INTEGRATED
HUMAN EXPOSURE COMMITTEE
(IHEC) OF THE SCIENCE
ADVISORY BOARD (SAB)
-------
September 29, 2000
EP A-S AB-fflEC-00-018
Honorable Carol Browner
Administrator
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, NW
Washington, DC 20460
Subject: Review of the Draft Strategic Plan for the Analysis of National Human
Exposure Assessment Survey (NHEXAS) Pilot Study Data
Dear Ms. Browner:
EPA's Office of Research and Development has carried out a series of pilot studies known
collectively as the National Human Exposure Assessment Survey (NHEXAS). The NHEXAS studies
tested protocols for acquiring population distributions of exposure measurements and developed
exposure databases for use in exposure models, exposure assessment, and risk assessment. These
unique databases focus on the exposure of people to environmental pollutants during their daily lives.
Some 550 volunteer participants were selected randomly from several areas of the country to obtain a
population-based probability sample. NHEXAS project scientists measured the levels of a suite of
chemicals to which participants were exposed in the air they breathe, in the foods and beverages they
consume, in the water they drink, and in the soil and dust around their homes.
The actual data collection was accomplished by three consortia (the University of
Arizona/Battelle Memorial Institute/Illinois Institute of Technology; the Research Triangle
Institute/Environmental and Occupational Health Sciences Institute; and Harvard/Emory/Johns
Hopkins/Southwest Research Institute/Westat). All three consortia used the same basic set of
questionnaires. However, by utilizing three consortia, alternative and innovative variations on the theme
of multimedia measurements to estimate total human exposure were possible.
The ultimate goal of the NHEXAS is to allow the Agency to estimate more accurately the
number of people exposed to the pollutants, as well as the magnitude and duration of the exposure.
Generally, estimates of exposure are based on "default assumptions," such as emissions or
environmental concentration data, rather than on actual measures of human exposure to contaminants.
-------
In 1998, the SAB's Integrated Human Exposure Committee (fflEC) reviewed the NHEXAS
effort. Along with other suggestions for improvement, the IHEC recommended that EPA develop a
strategic plan for the analysis of the NHEXAS Pilot Study Data (EPA-SAB-fflEC-ADV-99-004).
The Agency developed such a plan and requested that the IHEC review it. To this end, the IHEC met
on July 10-11, 2000, in Research Triangle Park, North Carolina.
The Charge for this meeting asked the IHEC to comment on four major areas (the complete
Charge is provided in section 2.2 of the enclosed report):
a) Does the Strategy encompass all the significant needed analysis projects?
b) Are the projects strategically presented and prioritized?
c) Is the Strategy likely to be useful to ORD management for resource allocation?
d) Does the Strategy provide adequate guidance to scientists for developing the most
useful analysis tasks?
Looking at the effort globally, the Committee concurred that the NHEXAS strategic plan
represented a remarkable effort and that its authors should be congratulated. The Committee also
supported the efforts already under way to create a public use data set and develop and support Web-
based access. The Committee suggests that EPA carefully review the long-term support needs of such
an endeavor and ensure that the necessary resources will be available to maintain this very publically
visible effort.
Addressing the specific Charge issues, the Committee felt that EPA did a commendable job of
setting priorities, and that the draft Strategy was logical, well thought out, and consistent with the
recommendations of the IHEC in its 1998 review of NHEXAS. The Committee was also impressed
with the scope and analysis of the proposed projects. The Committee was more concerned that too
much was being proposed then it was about any significant needed analyses were being omitted. The
list of proposed analyses looks overly optimistic when viewed in relation to the funding likely to be
available for the program. Also, some effort should be made to include some geographic information
which will permit analysis of the data in geographic information systems.
The Committee agreed that EPA had done a very good job developing and presenting the
NHEXAS data analysis strategy. This praise notwithstanding, the Committee did identify several
possibilities for improving this aspect of the draft document:
a) Promote a significantly greater degree of multi-disciplinary integration, especially
utilizing these integrated exposure data with health outcome modeling risk assessments.
-------
b) Emphasize work that does a better job of predicting exposure by quantitatively
elucidating multiple sources of exposure.
c) Consider setting priorities across topic areas as well as within them, and assess whether
they address EPA management policy priorities.
d) Divide very large projects into smaller efforts.
e) Compare the different sampling methods and survey tools used by the three consortia,
as well as the varying measurement results in order to assess the feasibility of
integrating the components of the data sets.
f) Carefully review previously published work (in order to avoid redundancy) and then
prioritize current Strategy implementation efforts to enhance support for the allocation
of research resources.
The Committee felt that the draft Strategy's prioritization of projects within given topic areas
was carefully thought out and well executed, and that it provided valuable information for use when
deciding how best to allocate limited resources. Several recommendations to make the Strategy even
more useful for resource allocation were noted, including:
a) Include an estimate of the hours (perhaps broken down by skill level) required to
complete the tasks.
b) Include (at least) qualitative cost-estimates to facilitate deciding among projects that
might otherwise be equally compelling
c) Provide comparisons and prioritization of highly ranked projects from different areas.
d) Consider the time-to-completion of the proj ects and whether they are likely to
contribute to attaining policy goal needs in a timely fashion.
e) Promote awareness of the data to encourage other EPA offices and other federal
agencies to use them and to assist in raising necessary resources to fully implement the
strategy.
f) Develop a five to seven year operational plan which has a merged list of individual
projects prioritized across the six descriptive classification categories (Descriptive
Statistics; Predictors of Exposure; Spatial and Temporal Variability; Aggregate
Exposure, Pathway Analysis and Cumulative Risk; Evaluation/Refinement of Exposure
Models and Assessment; and Designing Exposure Studies). The operational plan
-------
should be refined periodically as new information becomes available from the ongoing
research.
Addressing the issue of how well the strategy provided guidance to scientists for developing the
most "useful" analysis tasks, the Committee first noted that the answer to this question is not
straightforward, since the definition of "useful" can be interpreted in different ways by the many diverse
communities of scientists who could be potential users of the data. With that caveat, the Committee
agreed that the draft Strategy provides adequate guidance to scientists both inside and outside EPA
who are already familiar with the exposure assessment field, the NHEXAS effort, and the needs of the
Agency. With respect to usefulness, the Committee offered the caveat that the strategy could benefit
from additional guidance on applying the four basic criteria to potential projects that cut across the
different topic areas presented.
The Committee expressed concern however, that the draft document's "Timing" criterion may
tend to narrow the focus towards "short-term" studies, because these projects are likely to have the
most immediate impact. The Agency should elaborate the draft Strategy further, to make sure that
longer term projects that might impact policy decisions in the future are carried out, and do not
disappear in the year-to-year budgetary allocation process.
The Committee also recommended that EPA:
a) Expand the universe of researchers outside the Agency who could respond to
NHEXAS-related proposals.
b) Make a strong effort to ensure quality as much as possible before the data is posted.
c) Consider how to distribute the NHEXAS information to the public so as to be most
accessible and useable.
We appreciate the opportunity to review these issues, and look forward to the response of the
Assistant Administrator for Research and Development.
Sincerely,
/S/
Dr. Morton Lippmann, Interim Chair
Science Advisory Board
/S/
Dr. Henry Anderson, Chair
Integrated Human Exposure Committee
Science Advisory Board
-------
NOTICE
This report has been written as part of the activities of the Science Advisory Board, a public
advisory group providing extramural scientific information and advice to the Administrator and other
officials of the Environmental Protection Agency. The Board is structured to provide balanced, expert
assessment of scientific matters related to problems facing the Agency. This report has not been
reviewed for approval by the Agency and, hence, the contents of this report do not necessarily
represent the views and policies of the Environmental Protection Agency, nor of other agencies in the
Executive Branch of the Federal government, nor does mention of trade names or commercial products
constitute a recommendation for use.
-------
Distribution and Availability: This Science Advisory Board report is provided to the EPA
Administrator, senior Agency management, appropriate program staff, interested members of the
public, and is posted on the SAB website (www.epa.gov/sab). Information on its availability is also
provided in the SAB's monthly newsletter (Happenings at the Science Advisory Board). Additional
copies and further information are available from the SAB Staff.
-------
ABSTRACT
EPA's Office of Research and Development has carried out a series of pilot studies known
collectively as the National Human Exposure Assessment Survey (NHEXAS). The NHEXAS studies
tested protocols for acquiring population distributions of exposure measurements and developed
exposure databases for use in exposure models, exposure assessment, and risk assessment. The actual
data collection was accomplished by three consortia, employing the same basic set of questionnaires,
but using some different methodologies.
In 1998, the SAB's Integrated Human Exposure Committee (IHEC) recommended that EPA
develop a strategic plan for the analysis of the NHEXAS Pilot Study Data. EPA developed such a
plan and the IHEC met on July 10-11, 2000, in Research Triangle Park, North Carolina to review it by
discussing the Strategy's completeness/inclusiveness; its strategic presentation/prioritization of projects;
its usefulness for resource allocation; and its utility for providing guidance for developing useful analysis
tasks.
Looking at the effort globally, the Committee concurred that the NHEXAS strategic plan
represented a remarkable effort and that its authors should be congratulated. The Committee
suggested that EPA review long-term support needs to ensure that the necessary resources will be
available. Addressing the specific issues, the Committee felt that EPA set priorities well, but was
concerned that too much was being proposed. Also, some effort should be made to include some
geographic information which will permit analysis of the data in geographic information systems.
The Committee agreed the data analysis strategy was well done and well presented, but also
recommended that EPA: promote greater multi-disciplinary integration, including linking exposure data
with health risk assessments; emphasize work predicting exposure; set priorities across topic areas, and
assess whether they address EPA management policy priorities; subdivide large projects; integrate the
data collected by the three consortia into a single comparable database; and review previously
published work to avoid redundancy.
The Committee felt that the draft Strategy's prioritization of projects was well executed,
Recommendations advised EPA to: provide estimates of time and cost for projects; prioritize highly
ranked projects from different areas; consider the timing of the projects vs. the schedule for attaining
various policy goal; "market" the data to other EPA offices and other agencies; and develop a five to
seven year operational plan.
Addressing the issue of how well the strategy provided guidance to scientists for developing the
most "useful" analysis tasks, the Committee first noted that the answer to this question is not
straightforward, since the definition of "useful" can be interpreted in different ways by the many diverse
communities of scientists who could be potential users of the data. With that caveat, the Committee
111
-------
agreed that the draft Strategy provides adequate guidance to scientists both inside and outside EPA
who are already familiar with the exposure assessment field, the NHEXAS effort, and the needs of the
Agency. With respect to usefulness, the Committee offered the caveat that the strategy could benefit
from additional guidance on applying the four basic criteria to potential projects that cut across the
different topic areas presented.
The Committee also recommended that EPA: expand the universe of researchers who could
respond to NHEXAS-related proposals; ensure quality as much as possible before the data is posted;
and consider how to distribute the NHEXAS information to the public.
KEYWORDS: NHEXAS; Exposure; Risk assessment; Health effects; Population distributions
IV
-------
U.S. Environmental Protection Agency
Science Advisory Board
Integrated Human Exposure Committee
National Human Exposure Assessment Survey Advisory Panel
July 10-11,2000
CHAIR
Dr. Henry A. Anderson, Chief Medical Officer, Wisconsin Bureau of Public Health, Madison, WI
MEMBERS
Dr. Annette Guiseppi-Elie, Senior Consultant, Dupont Spruance Plant, Richmond, VA
Dr. Michael Jayjock, Senior Research Fellow, Rohm and Haas Co., Spring House, PA
Dr. Kai-Shen Liu, California Department of Health Services, Berkeley, CA
Dr. Thomas E. McKone, School of Public Health, University of California, Berkeley, CA
Dr. Jerome O. Nriagu, The University of Michigan, School of Public Health, Department of
Environmental and Industrial Health, Ann Arbor, MI
Dr. Barbara Petersen, Novigen Sciences, Inc., Washington, DC
Dr. David Wallinga, Institute for Agriculture and Trade Policy, Minneapolis, MN
Dr. Charles Wechsler, Telcordia Technologies, Red Bank, NJ
CONSULTANTS
Dr. Leyla McCurdy, Director, Indoor Air Programs, American Lung Association, Washington, DC
Dr. Maria Morandi, University of Texas Health Science Center at Houston, School of Public Health,
Houston, TX
SCIENCE ADVISORY BOARD STAFF
Mr. Samuel Rondberg, Designated Federal OfficER, U.S. Environmental Protection Agency,
Science Advisory Board (1400A), Washington, DC
Ms. Wanda R. Fields, Management Assistant, U.S. Environmental Protection Agency, Science
Advisory Board (1400A), Washington, DC
-------
TABLE OF CONTENTS
1 EXECUTIVE SUMMARY 1
2 INTRODUCTION 5
2.1 Background 5
2.2 Charge 6
3 DETAILED RESPONSE 7
3.1 Composition of the Project List (Issue a) 7
3.2 Presentation and Prioritization of the Projects (Issue b) 10
3.3 Utility of the Strategy for Resource Allocation (Issue c) 14
3.4 Utility of the Strategy as Guidance for Developing Analysis tasks (Issue d) 16
REFERENCES R-l
VI
-------
1. EXECUTIVE SUMMARY
EPA's Office of Research and Development has carried out a series of pilot studies known
collectively as the National Human Exposure Assessment Survey (NHEXAS). The NHEXAS studies
tested protocols for acquiring population distributions of exposure measurements and developed
exposure databases for use in exposure models, exposure assessment, and risk assessment. These
unique databases focus on the exposure of people to environmental pollutants during their daily lives.
Some 550 volunteer participants were selected randomly from several areas of the country to obtain a
population-based probability sample. NHEXAS project scientists measured the levels of a suite of
chemicals to which participants were exposed in the air they breathe, in the foods and beverages they
consume, in the water they drink, and in the soil and dust around their homes. Measurements were
made of chemicals or their metabolites in biological samples (including blood and urine) taken from the
participants. In addition, participants completed questionnaires to help identify possible sources of
exposure to chemicals and to characterize major activity patterns and conditions of the home
environment.
The actual data collection was accomplished by three consortia (the University of
Arizona/Battelle Memorial Institute/Illinois Institute of Technology; the Research Triangle
Institute/Environmental and Occupational Health Sciences Institute; and Harvard/Emory/Johns
Hopkins/Southwest Research Institute/Westat). All three consortia used the same basic set of
questionnaires. Within chemical classes selected by the consortia, each consortium analyzed for a basic
set of chemicals. However, by utilizing three consortia, alternative and innovative variations on the
theme of multimedia measurements to estimate total human exposure were possible. For example, each
consortium was able to target some specific concerns or opportunities. Two of the consortia focused
on measuring potential exposures of each participant once; one consortia studied fewer people but
repeated the measurements several times over the year to enable estimates of temporal variability for
the exposures and activities of interest.
The Committee's discussion of the strategic presentation and prioritization of the projects
resulted in strong consensus that the NHEXAS strategic plan represented a remarkable effort and that
its authors should be congratulated. The Committee also supported the efforts already under way to
create a public use data set and develop and support Web-based access. Although the specifics of the
Web page were not discussed, the Committee was impressed with the concept and cautioned that such
efforts, while critically important, are time and resource intensive and that before opening the site EPA
carefully review the long-term support needs of such an endeavor and ensure that the necessary
resources will be available to maintain this very publically visible effort.
Addressing the specific issues, the Committee noted that EPA did a commendable job of
setting priorities, and that the draft Strategy was logical, well thought out, and consistent with the
recommendations of the IHEC in its 1998 review of NHEXAS. It was clear that the periodic
interaction with the IHEC had a positive synergistic impact upon the program. The Committee was
1
-------
also impressed with the scope and analysis of the proposed projects, but was more concerned that too
much was being proposed then it was about any significant needed analyses being omitted. The list of
proposed analyses looks overly optimistic when viewed in relation to the funding likely to be available
for the program. The Committee feels that there should be some caution included in discussing the
applicability of the data and the goals of the analyses. Reconciliation of the differences in the data
resulting from different methodologies employed by the three consortia should be accomplished, and
the limitations of the entire NHEXAS database determined, before starting detailed analyses and model
development. Also, some effort should be made to include some geographic information which will
permit analysis of the data in geographic information systems. Spatially resolved data should be useful
especially in comparing population risks and for some applications of the data such as environmental
justice issues. Agency staff pointed out the privacy issues raised by geo-coding, but the added value of
such information would be great, and a solution should be sought. In considering whether analysis
projects could be added or deleted, the Committee identified some additional projects, needs for better
linking projects, and ways to leverage the value of the proposed projects. The Committee did not
identify any projects to be deleted. The Committee also determined that there are potential
opportunities to further aggregate and link the analysis projects and discussed some examples.
The Committee agreed that EPA had done a very good job developing the NHEXAS data
analysis strategy. This praise notwithstanding, the Committee did identify several possibilities for
improving the draft document:
a) Promote a significantly greater degree of multi-disciplinary integration, especially
utilizing these integrated exposure data with health outcome modeling risk assessments.
b) Emphasize work that does a better job of predicting exposure by quantitatively
elucidating multiple sources of exposure.
c) Consider setting priorities across topic areas as well as within them, and assess whether
they address EPA management policy priorities.
d) Divide very large projects into smaller efforts.
e) Compare the different sampling methods and survey tools used by the three consortia,
as well as the varying measurement results in order to assess the feasibility of
integrating the components of the data sets.
f) Carefully review previously published work (in order to avoid redundancy) and then
prioritize current Strategy implementation efforts to enhance support for the allocation
of research resources.
-------
The Committee felt that the draft Strategy's prioritization of projects within given topic areas
was carefully thought out and well executed, and that it provided valuable information for
use when deciding how best to allocate limited resources. Several recommendations to make the
Strategy even more useful for resource allocation were noted:
a) Include an estimate of the hours (perhaps broken down by skill level) required to
complete the tasks.
b) Include (at least) qualitative cost-estimates to facilitate deciding among projects that
might otherwise be equally compelling
c) Provide comparisons and prioritization of highly ranked projects from different areas.
d) Consider the time-to-completion of the proj ects and whether they are likely to
contribute to attaining policy goal needs in a timely fashion.
e) Promote awareness of the data to encourage other EPA offices and other federal
agencies to use them and to assist in raising necessary resources to fully implement the
strategy.
f) Develop a five to seven year operational plan which has a merged list of individual
projects prioritized across the six descriptive classification categories (Descriptive
Statistics; Predictors of Exposure; Spatial and Temporal Variability; Aggregate
Exposure, Pathway Analysis and Cumulative Risk; Evaluation/Refinement of Exposure
Models and Assessment; and Designing Exposure Studies). The operational plan
should be refined periodically as new information becomes available from the ongoing
research.
Addressing the issue of how well the strategy provided guidance to scientists for developing the
most "useful" analysis tasks, the Committee first noted that the answer to this question is not
straightforward, since the definition of "useful" can be interpreted in different ways and there are many
diverse communities of scientists who could be potential users of the data. With that caveat, the
Committee agreed that the draft Strategy provides adequate guidance to scientists both inside and
outside EPA who are already familiar with the exposure assessment field, the NHEXAS effort, and the
needs of the Agency. With respect to usefulness, the Committee offered the caveat that the strategy
could benefit from additional guidance on applying the four basic criteria to potential projects that cut
across the different topic areas presented.
The Committee expressed concern however, that the draft document's "Timing" criterion may
tend to narrow the focus towards "short-term" studies, because these projects are likely to have the
-------
most immediate impact. The Agency should elaborate the draft Strategy further, to make sure that
longer term projects that might impact decisions in the future are carried out, and do not disappear in
the year-to-year budgetary allocation process.
The Committee also recommended that EPA:
a) Expand the universe of researchers outside the Agency who could respond to
NHEXAS-related proposals (to include those from other federal agencies, state and
local health and environmental departments, and the academic community) beyond that
of exposure assessors, and scientists working with environmental and citizen groups, for
example, by adding web links to seminal publications describing the conceptual
framework of NHEXAS.
b) Make a strong effort to ensure quality as much as possible before the data are posted.
In this context (but understanding that the Agency does not have a mission, or the
resources, to provide extensive and continued support for addressing technical
questions about the database use), it is advisable that an "expert" on the NHEXAS
studies and the database contents be designated to address content and data quality
issues that might arise after the database release
c) Consider how to distribute the NHEXAS information to the public so as to be most
accessible and useable (The Agency might consider setting-up a separate project on
how to communicate NHEXAS data to the public in an appropriate manner)
-------
2. INTRODUCTION
2.1 Background
To evaluate the risks posed by chemical pollutants in the environment, EPA must be able to
estimate the number of people exposed to the pollutants as well as the magnitude and duration of the
exposure (i.e., the distribution of human exposures). Until recently, estimates of exposure have been
based on "default assumptions," extending simple measurements of emissions or environmental
concentration data to estimation of human exposure, rather than on actual measures of human exposure
to contaminants. Without measurements of human exposure, these default assumptions are of limited
value because they do not reflect actual patterns (distributions) of human exposure to chemicals in the
environment.
Increasingly, EPA's scientific advisors are concerned about reliance on these default
assumptions - particularly when evaluating the risks from exposure to environmental contaminants or
when estimating the benefits that may be obtained from managing these risks. Addressing these
concerns is a vital link in reducing the scientific uncertainty in health risk assessment and in regulatory
decision making.
To respond to these concerns, EPA's Office of Research and Development sponsored three
related pilot studies known as National Human Exposure Assessment Survey (NHEXAS). The
NHEXAS studies respond to these concerns by testing protocols for acquiring population distributions
of exposure measurements and by developing exposure databases for use in exposure models,
exposure assessment, and risk assessment. The principle objectives of the NHEXAS pilot studies are
to (1) evaluate the feasibility of NHEXAS concepts, methods, and approaches for the conduct of future
population-based exposure studies; (2) evaluate the utility of NHEXAS data for improved risk
assessment and management decisions; (3) test the hypothesis that the distributions of exposure given
by modeling and extant data do not differ from the measurement-based distributions of exposure; (4)
define the distribution of multi pathway human exposures for a relatively large geographic area; and (5)
stimulate exposure research and forge strong working relationships between government and non-
government scientists.
The strategy is intended to provide broad guidance to EPA decision makers on resources and
to those who would undertake analyses. The investigators would use this foundation to bring their
expertise and a more exact examination of the relevant portions of the NHEXAS database to the
development of proposals for specific analysis tasks. Essentially, there are two parts to the draft
Strategy: 1) a description of the analyses/efforts that have been completed or are already funded and
underway (a brief summary and appendix) and 2) the main portion of the document containing ORD's
strategic criteria and recommendations for future analyses.
-------
The Strategic Plan for the Analysis of the National Human Exposure Assessment Survey
(NHEXAS) Pilot Study Data was developed in response to a recommendation by the IHEC in their
September, 1998 advisory on the NHEXAS (EPA-SAB-fflEC-ADV-99-004). Given the vast array
of data and analysis possibilities, the IHEC and ORD believed that a strategy was essential to ensure
that the most important analyses were identified and completed in a timely way. To these ends, ORD
convened an NHEXAS analysis workshop in July of 1999, with participants from EPA Program
Offices, EPA Regions, other federal agencies, state agencies, academia, and the private sector. These
experts, with varying perspectives, discussed various options for projects that might be conducted.
They then were asked to select those with the greatest scientific value, and jointly developed project
descriptions which are described in the Proceedings of the NHEXAS Data Analysis Workshop
(www.epa.gov/NERL/nhexas.htm). These projects served as a key input to ORD in developing the
analysis strategy which is the subject of the IHEC review.
2.2 Charge
To improve the draft, ORD seeks the advice of the IHEC, on the following four specific
questions:
a) Does the Strategy encompass all the significant needed analysis projects? If not, which
should be added or deleted? (e.g., is the list of projects good?)
b) Even if all the projects are optimal, are they strategically presented and prioritized?
Would alternative strategic criteria be useful? (e.g., is the prioritization good?)
c) Is the Strategy likely to be useful to ORD management for resource allocation (e.g., is it
of sufficient quality for managerial use)?
d) Does the Strategy provide adequate guidance to scientists for developing the most
useful analysis tasks?
-------
3. DETAILED RESPONSE
3.1 Composition of the Project List (Issue a)
The initial issue asked the Committee to determine if the draft strategy encompassed all the
priority analysis project, i.e., is the project listing "good?" In addressing this issue, the Committee
noted that the NHEXAS data are unique in that they provide the first opportunity to consider
concentrations along the pathway from emissions to dose. There are extant measurements of human
tissue concentrations (biomarkers) and measures of emissions and ambient concentrations in air, water,
food, and soil. But the ability to attribute observed tissue concentrations to specific sources has
remained elusive. NHEXAS now provides the first opportunity to explore this frontier. Because the
term "good" is open ended and defining a "good" list is subjective, the Committee attempted first to
provide its interpretation of what constitutes a "good" list. There is consensus that a good list is
a) Useful to EPA and its regulatory programs
b) Useful to other state and federal health and environmental programs
c) Scientifically feasible
d) Prioritized by multiple objectives
e) Comprehensive
The Committee observed that the EPA did a commendable job of accounting for these factors
in establishing priorities among the various projects. The process for setting priorities was logical, well
thought out, and consistent with the recommendations of IHEC in its 1998 review of NHEXAS. In
order to develop and prioritize a list of analysis projects, the EPA set up a workshop, selected broad
input from the scientific community, and used this process to develop a large set of recommendations
and then organized this set into a smaller set. These efforts are particularly noteworthy. This is a useful
and effective approach. The experts who participated provided a good representative cross-section of
those who will use, evaluate, and collect NHEXAS-like data. This effective use of broad input from
outside practitioners should be continued.
The Committee was also impressed with the scope and analysis of the proposed projects. The
Committee was more concerned that too much was being proposed then it was about any significant
needed analyses being omitted. The list of proposed analyses looks much like a wish list when viewed
in relation to the funding likely to be available for the program. Although each "project" has been
ranked, the probability of having any projects, especially those that fall in the intermediate time frame,
funded is not clear. Timing will be critical in determining whether a particular analysis gets done, since
priorities change, funding can dry up, and the science and database become outmoded. Although the
report is clearly inclusive in terms of what should be done, it is less specific with regards to what will
actually be done with available resources.
-------
There should be some caution included in discussing the applicability of the data and the goals
of the analyses. NHEXAS data were collected for specific purposes (as a demonstration project in
some cases), and may be unsuitable for some of the analyses being proposed. For instance, the three
consortia (the University of Arizona/Battelle Memorial Institute/Illinois Institute of Technology; the
Research Triangle Institute/Environmental and Occupational Health Sciences Institute; and
Harvard/Emory/Johns Hopkins/Southwest Research Institute/Westat) actually collecting the data
sometimes used differing measurement instruments with different sensitivities (to show that it could be
done), and the resulting differences in the database need to be better understood before the strategic
plan is implemented. Consequently, the limitations of the entire NHEXAS database should be carefully
examined before the NHEXAS data are actually subjected to detailed statistical analyses or used for
model development. Once the data are posted on the Internet, users will be tempted to merge the
databases from the consortia, which could lead to erroneous conclusions unless these differences are
well documented.
Some effort should be made to include a geographic unit of analysis (zip code, census tract, etc)
which will permit analysis of the data by spatial statistics and/or geographic information systems. This
should facilitate comparison of exposures (and health outcomes) for the NHEXAS areas with those for
other parts of the country. Spatially resolved data should be useful especially in comparing population
risks and for some applications of the data such as environmental justice issues. We understand, and
are sensitive to, the privacy issues raised by geo-coding, but the added value of such information would
be great, and a solution should be sought.
The proposed projects are relatively well defined and are based on current science. Some
thought should be given to devoting a small portion of the resources to "exploratory" projects where
scientists will be encouraged to come forward with new and innovative methods for analyzing and
applying the NHEXAS data. The projects have been prioritized on the basis of timing, feasibility,
applicability and demand. There is a need for including scientific innovation within the prioritizing and
resource allocation framework.
Carefully defined process evaluation should be included as part of any program evaluation.
Knowing the key elements of why the projects worked or did not work is important in planning any
national survey based on the NHEXAS experience.
In considering whether analysis projects could be added or deleted, the Committee identified
some additional projects, needs for better linking projects, and ways to leverage the value of the
proposed projects. The Committee did not identify any projects to be deleted.
One issue that was missing or not well articulated is some way to indicate where integration
among several projects is feasible and would offer benefits that are greater than the sum of the
individual projects. The Committee determined that there are potential opportunities to further
-------
aggregate and link the analysis projects. Some examples recommended by the committee are provided
below:
a) Form an NHEXAS Model Evaluation Group: One clear example of an opportunity
to link analysis projects is the integration of the prediction studies with the model
assessment studies. To address this concern, the Committee suggests that EPA
establish a NHEXAS Model Evaluation group addressing the use of the NHEXAS
data. This could be modeled after the Biosphere Model Validations Study
(BIOMOVS) or the Validation of Environmental Model Predictions (VAMP)
programs that have been organized by the International Atomic Energy Agency. It can
be coordinated with other on-going, agency-wide model evaluation efforts. But the
unique information on exposure pathways contained in NHEXAS makes these data of
particular value for evaluating exposure models. The idea of these model-evaluation
working groups is to make optimum use of data and models. Those who collect data
set up problems for those who make models. Modelers run the results and compare
among themselves how well their models work. But importantly, they not only consider
whether their models match or don't match the data, but why they don't.
b) Conduct Data Simulation: Data simulation is intermediate between descriptive
statistics/statistical prediction on one pole and process model predictions at the other
pole. In this case a model could be developed to simulate exposure data, not
necessarily exposure. It would provide a tool for evaluating patterns within a set of
data.
c) Integrate Value of Information concepts into the Strategy: During the EPA
presentation at the public meeting, the Agency staff discussed the use of value of
information (VOI) studies with the existing NHEXAS data. The Committee found this
proposal of particular interest and encourages the use of VOI concepts as a means of
integration among the six topic areas. The usefulness to EPA of NHEXAS data
depends on the possible impact it has on decision making and specifically on achieving
articulated agency policy goals. Information that cannot influence policy or regulatory
decisions or inform models and advance measurement strategies is not useful.
Information that only adds marginally to what is already known is not useful in the
decision-making context. Too much information can be a nuisance and an obstacle to
good decision-making. These general ideas led to an exploration of the concept of
VOI (see, e.g., Clemen, 1990, and von Winterfeldt and Edwards, 1986). These
examples give rise to two key questions: 1) How well can decisions be made without
NHEXAS-type data?; and 2) How much can decisions be improved by collecting
more NHEXAS-type data? The use of existing NHEXAS data, together with a
Bayesian iterative process, can be used to address these questions. VOI is an
appropriate measure of the usefulness of information, but the measurement methods that
-------
provide the information also have other attributes, including cost, sensitivity,
temporal/spatial coverage, etc. To evaluate new NHEXAS-type studies, it is therefore
useful to determine how the technologies stack up on these other attributes.
d) Other Recommended Modifications to the Proposed Projects: The Committee
discussed other potentially useful modifications to the analysis projects based on
the NHEXAS data. The data provide an opportunity to validate exposure transfer
factors, such as house dust to skin surfaces, tap water to ingestion, or soil to food.
The way NHEXAS data will be used and interpreted will be impacted by issues such
as how non-detects (situations in which an agent may be present, but at a level below
which the monitoring/analytical approach can detect) are quantitated, by weighting
factors, and by the monitoring calendar dates. The reporting and evaluation of non-
detects can have a significant impact on the characterization of source/exposure
relationships. It is important not to obscure any decision about the use of non-detects.
To avoid or limit this problem, it is important to include both the limit of detection and
the decision algorithms used to interpret non-detects along with reported
concentrations. Similarly, any weighting factors used to combine exposure data among
populations and regions should be clearly stated. Calendar dates associated with data
collection should be included with the data. Including the above information will make
more transparent any decisions that are made to interpret or record exposure data.
The proposed projects should be modified to make possible comparison and
integration with data collected in other studies. For example the Office of Air Quality
Planning and Standards, in its National Air Toxics Assessments program, will be
collecting a great deal of region-specific air data. It would be used to integrate the
NHEXAS data with this type of study. Similar integration with water and food
monitoring programs should also be considered. At the other end of the monitoring
spectrum, there is value in linking NHEXAS with efforts to collect biomarker data at
organizations such as the Center for Disease Control. That is, will the time and spatial
resolution of the data sets be sufficiently matched for the sets to be truly linked?
3.2 Presentation and Prioritization of the Projects (Issue b)
The second element of the Charge asked the Committee to assess how well the projects were
strategically presented and prioritized.
The Committee's overall thinking is that the NHEXAS strategic plan represents a remarkable
effort and its authors are to be congratulated on a job well done. This is an outstanding effort. Clearly
a significant amount of work and thought has gone into its presentation and prioritization. The reasoning
and the explanation for the chosen topics and their prioritization are excellent. Also the plan has been
10
-------
quite responsive to the previous comments from the IHEC. Indeed, if left alone the current plan would
render significant value and accomplish most if not all of the objectives laid out for it. Our task,
however, as reviewers was to seek improvement and to advise in line with that enhancement.
Consequently, we wish to advance the following suggestions:
Cause & Effect Continuum and the
Boundary of Exposure Assessment
Source >
Transport >
Contact >|
Intake >
Absorption>
Transport >
Reaction>
EFFECT
Outside
Inside
HUMAN
Figure 1
a) We recommend that the plan contain a significantly greater degree of multi-
disciplinary integration with the inclusion of scientists in the areas of
physical/chemical modeling, toxicology and health outcomes.
Our sense is that the plan would benefit if it were significantly broader in scope,
especially related to the critical topics of model development and the integration of
lexicological dose-response information.
11
-------
If one imagines a continuum of cause and effect for exposure and resulting adverse
health effect as a line of events that proceeds normally on a time-line from left to right
(see Figure 1, above), then a natural priority for work could evolve. That is, there
should be a concentration of effort for work that does a better job of predicting
exposure by elucidating the early causes for the exposure coming from the "left." This
would include first-principle physical/chemical descriptors or causes of exposure.
As expected, there are likely to be competing needs, and, as these play out, the need to
prioritize between categories becomes obvious. The need for integrated multi-media
models from a number of regulatory programs would suggest that this area should be
given some priority. The creation of a model evaluation workgroup, as discussed
earlier in our report, also responds to this issue.
Assuming we can learn about exposure from the "left," we then at some point need to
put that exposure into context by understanding the ultimate effect on humans from the
exposure. In other words, it would benefit us to look further to the "right" on the cause
and effect continuum in order to help us appropriately prioritize which exposures and
exposure sources are most important from the perspective of projected health impact.
Priority should be reserved for models looking to the "left" because they will do the
most to develop the hypotheses and thus generalize and grow the science of exposure
assessment.
In a practical sense, given the fact that NHEXAS was solely a multi-source exposure
assessment, a prominent but slightly lower priority in the Strategy should be given to
how the data can be utilized to advance our understanding of exposure-response. The
point here is that this critical piece which, via modeling, would link the exposure data
with health outcomes should be a more prominent, defined and identified part of the
strategic plan. NHEXAS was designed as an "exposure" project and it may be quite
difficult to integrate the considerations of dose-response and risk into it; however, the
need to include this critical element in future work is clear.
In future work, the scientific areas to the "left" on the chart should perhaps continue to
receive the highest priority, the but priority for lexicological and health outcome
considerations should also be relatively high. The real contribution of the toxicology
would come from putting any estimated exposure into context by providing the specific
dose-response and toxic potency which then would give the exposures and sources
their public health importance. Indeed, exposure intensity and the meaning of low,
medium and high (L, M, H) exposure only can be meaningfully addressed with this
information.
12
-------
We note that these comments are not to imply that the integration of physical/chemical
models and exposure-response data are completely missing in the current plan. Indeed
there are indications that these elements exist; however, to at least some readers their
prominence appears to be muted or be considerably understated. As critical elements
our suggestion is that they be brought forward and enhanced in the current plan to the
extent feasible.
b) Priorities should be considered across topic areas as well as within them.
There are six topic areas:
1) Descriptive Statistics
2) Predictors of Exposure
3) Spatial and Temporal Variability
4) Aggregate Exposure, Pathway Analysis and Cumulative Risk
5) Evaluation/Refinement of Exposure Models and Assessment
6) Designing Exposure Studies
Priorities within each of these 6 topic areas of the current plan appear to be reasonably well
described and justified, even though there appears to be a relatively low level of multi-
disciplinary integration. Taking into account the vast number of potential projects that could be
developed, the approach taken to prioritizing those taken appears to be appropriate. Choosing
the six general categories and then prioritizing within these groups makes good sense. Although
the evaluation was performed in a hierarchical manner, the criteria used for prioritization within
the groups (i.e., timing, feasibility, applicability, demand) should result in the most relevant
projects being advanced (because all factors are taken into account).
We believe this serves the project well as a tactical plan but may cause it some problems
strategically. A potential area for improvement would be a natural prioritization among topics.
In line with the above advice, it is suggested that these be viewed with an increased emphasis
on the ideal of enhancing an integrated multi disciplinary approach as mentioned above. In this
situation and given this perspective, topic 5 would receive greater attention and development.
It could rise to natural prominence among the areas since models arguably represent the basic
business of the science of exposure assessment.
The priority of the proposed projects was ranked first by timing, and then by feasibility,
broad application and demand. It would appear that how urgent the information is needed
and how widely it can be used are more important than how soon and how easy the information
can be delivered. Demand and broad applicability should be higher considerations than
timing and degrees of feasibility for feasible projects.
13
-------
The idea of then allowing projects (using the funding mechanisms described at the public
meeting (i.e., internal EPA, leveraging with other agencies/organizations) to proceed based on
the prioritization within the groups should work. However, it is important to ensure that all
projects that relate to developing the data and assuring its quality (reducing uncertainty) should
take precedent (these projects provide the data that are the foundation of all other projects).
In the current strategic scheme, demand and applicability are somewhat bundled together. We
suggest that the applicability criteria be split out and separated as an individual entity. Further,
the applicability category itself combines a number of somewhat dissimilar goals including the
Government Performance Results Act, NHEXAS goals, and previous IHEC recommendations.
The framework's criteria, while well thought out, appear to focus too little attention on whether
the proposed projects will help illuminate exposures and risks to the most highly exposed
populations. Thus it is suggested that explicit criteria focused on public health protection be
added to the prioritization in order to focus the direction of future work.
c) Consider dividing very large projects into smaller lots
Some projects are huge jobs that will require extraordinary efforts to carry out, e.g., modeling
work of 30-40 chemicals. To make them manageable, some consideration should be given to
dividing such large projects into several small projects by groups of chemicals. The groups of
chemicals can be further prioritized based on the demand and their broad applicability within
the agency and ability to address policy goals.
Further, specific projects that are developed from these areas will also need to be prioritized.
As far as possible, projects should be hypothesis-driven. Great care would be needed to
ensure that there is adequate peer review and control over what is funded.
d) Compare sampling methods, survey tools and measurement results across the
three consortia
Comparisons of sampling methods, survey tools, and measurements results across the three
consortia might be considered as important as data management procedures in designing future
exposure studies. Although different aspects of this issue have been touched in various
projects, no specific project deals with it in a systematic and exclusive way.
e) Coordinate and prioritize current efforts after careful review of previously
published work
Some of the projects listed in the strategic plan have already been published . To avoid
redundancy and inconsistency, it is suggested that proposed projects should take into
consideration the published work, articles in press or in progress by the principal investigators
14
-------
of the three consortia. In published articles, the three consortia presented their data in different
formats, which makes cross-study comparison and the explanation of differences in the data
very difficult. Coordination should take place across three consortia so that data and models
can be easily compared and checked.
3.3 Utility of the Strategy for Resource Allocation (Issue c)
The third element of the Charge asked the Committee to assess the utility of the Strategy to
support the allocation of research resources.
The Strategy has classified the various analysis projects into six distinct topic areas (see section
3.2, preceding). The projects are then evaluated using clearly stated ranking criteria that are applied to
each of the projects. The final result is a prioritization of projects within a given topic area. We believe
that these step s have been carefully thought out and executed.
Tables 5 -10 in the review document's text (and collected and reproduced in its Appendix D),
are very useful summaries for managerial use; they provide a ready comparison among projects within a
particular topic area. The draft Strategy's Appendix E contains more detailed summaries for each
project and is also quite useful. Within each of the tables in Appendix E, the rows headed "Suggested
Approach," "Data or Input Needs," and "Feasibility," are especially valuable in trying to decide how
best to allocate limited resources.
Despite the many "pluses" noted above and evident in the document, there are several areas
where additional information would make the Strategy even more useful for resource allocation:
a) In addition to estimated dollar costs, it would be valuable if each of the analysis projects
included an estimate of the hours (perhaps broken down by skill level) required to
complete the task. This would be especially valuable for EPA programs that wish to
allocate existing staff to projects. As with cost estimates, these estimates could be done
in a qualitative fashion indicated by "stick figures" or some sort of icon representing
ranges of person-hours (recognizing that these are, per force, "soft" estimates, rather
than documented accounting data). Thus, a project that did not require many hours
might be assigned one stick-figure, while a project that required a very large number of
hours might be assigned four stick-figures.
b) Some of the projects will be much more expensive to complete then others. The costs
are not necessarily just "person-hours." Hence this is information a step beyond the
"hours required to complete the task." Cost-estimates would facilitate deciding among
projects that might otherwise be equally compelling. If there is reluctance to place cost
estimates on projects, expense projections for each project could also, as noted above,
be indicated by an icon. The relative cost- estimates for each project could be
15
-------
qualitatively indicated by an icon with each icon indicating a range of cost (i.e., $ =
$10,000 - 100,000; $$ = $100,000 - $500,000 etc)
c) Although the projects are rank ordered within each topic area, there is no attempt to
compare the priorities of projects from different topic areas. There will likely be
insufficient resources to execute the highest ranked projects in each of the six topic
areas. Hence, we recommend comparisons and prioritization of highly ranked projects
from different areas and specific attention be paid to projects that will address pressing
policy goals.
d) The life-expectancy of the projects should be considered. For example, some projects
may become less relevant as time passes - national diets change, concentrations of
airborne contaminants change, etc. This relates to projects that address fundamental
issues versus projects that address immediate statutory and regulatory needs.
An additional point was made regarding the importance of external marketing. Making other
federal and state agencies aware of the data can to lead to "buy-in" and may assist in raising necessary
resources.
In conclusion, the Office of Research and Development should develop a five to seven year
operational plan which has a merged list of projects prioritized across the six categories. Each of the
projects should have human resource needs, cost and a time line attached. The manner in which
various parts of this plan will be implemented (in-house versus Request For Proposals (RFP), etc..)
should be spelled out. The operational plan should be refined periodically as new information becomes
available from the ongoing research.
3.4 Utility of the Strategy as Guidance for Developing Analysis tasks (Issue d)
The last Charge element asked if the strategy provided adequate guidance to scientists for
developing the most useful analysis tasks.
The answer to this question is not straightforward, since the definition of "useful" can be
interpreted in different ways and there are many diverse communities of scientists who could be
potential users of the data.
The Agency's definition of "useful" employs the criteria of Timing, Feasibility, Broad
Applicability and Demand (Urgency). Within these criteria, the draft Strategy provides a good
roadmap and adequate guidance to scientists both inside and outside EPA who work in, or are familiar
with, the field of exposure assessment and who understand the Agency's needs. A caveat to this
finding, however, is that because the strategy applies criteria only within topic areas, it could benefit
from additional guidance on potential projects that cut across topic areas (For example, model
16
-------
evaluation can be linked with current plans to design future measurement efforts or with exposure
prediction.). With that caveat, the guidance provided by the draft Strategy is adequate for informing the
interested scientist on which areas of research involving NHEXAS data are the most important to the
Agency.
The Strategy is also sufficiently flexible in how a specific project may address each type of
analysis. A concern is that the "Timing" criterion may tend to narrow the focus towards "short-term"
studies, because these projects are likely to have the most immediate impact. The Agency should
elaborate the draft Strategy further, to make sure that longer term projects that might impact decisions
in the future are carried out, and do not disappear in the year-to-year budgetary allocation process.
In its broadest sense, "useful" should refer to the wide range of exposure questions that might
be addressable by analysis of the NHEXAS data, but may not necessarily have an apparent
programmatic or regulatory interest at the present time. These questions may go beyond the Agency
priorities at this time; many are summarized in the Workshop report, but others will be originated by
individual researchers as they explore the database. In either case, the quality and nature of the
database are of paramount importance, although the "usefulness" of the Strategy is less relevant than
the "usefulness" of the database for issues not included into the Strategy analysis plan. Both the quality
of the data and the back-up information available in the database are key issues. From a scientific
standpoint, there has to be enough information in the metadata to be able to permit the individual
researcher to judge their adequacy for addressing specific questions that may or not be presented in the
current draft Strategy.
The potential users of the data are diverse, including scientists in the EPA, other Federal
agencies, state and local health or environmental departments, the academic community, and scientists
working with environmental and citizens groups. As noted, the document provides adequate guidance
for scientists at EPA and for those in the academic community who are familiar with the exposure
assessment field, the NHEXAS effort, and the needs of the Agency. In order to expand the universe of
researchers outside the Agency that could be responsive to NHEXAS- related Request For
Proposals, it might be useful to add web links to the seminal publications describing the conceptual
framework of NHEXAS. Also, because of the breath of many of the projects described in the
Strategy, a further prioritization within these large projects might be needed for targeting the specific
RFPs.
As indicated above, the key issue for the academic community is the quality of the database and
the supporting information. EPA should make a strong effort to ensure quality as much as possible
before the data are posted. As with any other undertaking of this size, problems will be found by the
users of the data, but these problems should not be so extensive or of a nature that will impact its
credibility. In this context (but understanding that the Agency does not have a mission, or the
resources, to provide extensive and continued support for addressing technical questions about the
17
-------
database use), it is advisable that an "expert" on the NHEXAS studies and the database contents be
designated to address content and data quality issues that might arise after the database release.
To the extent that the Strategy may not have captured all questions of interest, it is important
that RFPs not be so restrictive as to discourage the possibility of new avenues of research that might
address a question relevant to a specific topic, but not fitting exactly the letter of the request. Further
guidance is needed for issues that cross the six areas of the draft Strategy. There is also some danger
that the broad academic community may be discouraged from applying to RFPs because of the
perception that the scientists in the existing consortia have an advantage due to intimate knowledge of
the data and the overall study. RFPs should include wording that encourages all qualified applicants to
participate.
Finally, the academic community currently engaged in exposure studies or planning them has a
special interest in projects DES-01 through DES-08. Although the Strategy has given several of these
projects a high priority, there is a risk that the data analysis projects might use most of the available
resources because of the Agency's internal pressure for information needed to address regulatory
program issues. It is very important that the methodology questions be addressed early-on, so mistakes
can be avoided by outside scientists engaged in exposure research. In addition, as the National Health
Exposure Monitoring Survey develops, the lessons learned from NHEXAS will have to be
incorporated and the outside community of scientists could provide alternatives or refinements for
approaches that did not work or were not as satisfactory as originally believed.
The state and local agency scientists with responsibilities in the environmental health area will be
consumers of these data and constitute an important base of support for a future national exposure
study. The level of knowledge and expertise with regards to exposure issues is highly variable among
these scientists, so the Strategy may not provide sufficient guidance. The links to NHEXAS papers
described above will help to provide some background information, but there might be a need to
develop a tutorial/case study program or some similar other mechanism that provides basic hands-on
training on the use of the data and the relevant (for the local agencies/states) information it might provide
RFPs can also encourage participation of scientists in the state/local agencies specifically, and in
collaboration with academic researchers.
Community-based scientists will also need some further guidance since they typically are not
familiar with the technical aspects and complexity of data analyses of this type. A tutorial (with
appropriate case/studies) could be helpful as well as issuing RFPs (where appropriate) that encourage
collaboration between academic and community investigators.
The EPA also should consider how to distribute the NHEXAS information to the public so as
to be most accessible and useable. While not necessarily related to the research process presented in
the Strategy, this latter question is important to ensure the broadest possible impact of NHEXAS. The
Agency might consider setting-up a separate project on how to communicate NHEXAS data to the
18
-------
public in an appropriate manner. A separate project will ensure that this issue does not become
"buried" by the major data analyses projects.
19
-------
REFERENCES
Clemen, R. 1990. Making hard decisions. New York, N.Y. P.W.T. Kent.
von Winterfeldt, D. and W. Edwards. 1986. Decision analysis and behavioral research. New York.
Cambridge University Press.
R-l
------- |