United States Environmental Protection Jf lkAgency EPA/600/R-23/061 March 2023 www.epa.gov/isa Integrated Science Assessment for Lead Appendix 12: The Process for Developing the Pb Integrated Science Assessment External Review Draft March 2023 Health and Environmental Effects Assessment Division Center for Public Health and Environmental Assessment Office of Research and Development U.S. Environmental Protection Agency ------- DISCLAIMER 1 This document is an external review draft for peer review purposes only. This information is 2 distributed solely for the purpose of predissemination peer review under applicable information quality 3 guidelines. It has not been formally disseminated by the Environmental Protection Agency. It does not 4 represent and should not be construed to represent any agency determination or policy. Mention of trade 5 names or commercial products does not constitute endorsement or recommendation for use. 6 External Review Draft 12-ii DRAFT: Do not cite or quote ------- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 DOCUMENT GUIDE This Document Guide is intended to orient readers to the organization of the Lead (Pb) Integrated Science Assessment (ISA) in its entirety and to the sub-section of the ISA at hand (indicated in bold). The ISA consists of the Front Matter (list of authors, contributors, reviewers, and acronyms), Executive Summary, Integrated Synthesis, and 12 appendices, which can all be found at https://cfpub.epa.gov/ncea/isa/recordisplav.cfm?deid=357282. Front Matter Executive Summary Integrative Synthesis Appendix 1. Lead Source to Concentration Appendix 2. Exposure, Toxicokinetics, and Biomarkers Appendix 3. Nervous System Effects Appendix 4. Cardiovascular Effects Appendix 5. Renal Effects Appendix 6. Immune System Effects Appendix 7. Hematological Effects Appendix 8. Reproductive and Developmental Effects Appendix 9. Effects on Other Organ Systems and Total (non-Accidental) Mortality Appendix 10. Cancer Appendix 11. Effects of Lead in Terrestrial and Aquatic Ecosystems Appendix 12. Process for Developing the Pb Integrated Science Assessment External Review Draft 12-iii DRAFT: Do not cite or quote ------- CONTENTS APPENDIX 12 THE PROCESS FOR DEVELOPING THE INTEGRATED SCIENCE ASSESSMENT FOR LEAD 12-1 12.1 Introduction 12-2 12.2 Documentation 12-2 12.2.1. Literature Database: Health and Environmental Research Online 12-2 12.2.2. Study Quality Documentation: Health Assessment Workspace Collaborative 12-3 12.3 Overview of the Process Steps for Developing Integrated Science Assessments 12-3 12.4 Relevance and Scope 12-5 12.4.1. Atmospheric Sciences 12-5 12.4.2. Exposure, Toxicokinetics, and Biomarkers 12-6 12.4.3. Health 12-6 12.4.4. Welfare—Effects on Terrestrial and Aquatic Ecosystems 12-10 12.5 Literature Search and Study Selection 12-13 12.5.1. Title and Abstract Screening 12-16 12.6 Study Selection: Full-Text Evaluation of Studies 12-18 12.6.1. Individual Study Quality 12-18 12.7 Peer Review and Public Participation 12-26 12.7.1. Request for Information 12-26 12.7.2. Integrated Review Plan 12-27 12.7.3. Peer Input 12-27 12.7.4. Internal Technical Review 12-28 12.7.5. Clean Air Scientific Advisory Committee Peer Review 12-28 12.8 Quality Assurance 12-29 12.9 Conclusion 12-29 12.10 References 12-30 External Review Draft 12-iv DRAFT: Do not cite or quote ------- LIST OF TABLES Table 12-1 Table 12-2 Table 12-3 Table 12-4 Table 12-5 Population, Intervention, Comparison, Outcome, and Context statement to define the parameters and provide a framework for identifying relevant atmospheric science studies. 12-5 Population, Exposure, Comparison, Outcome, and Study design statement to define the parameters and provide a framework for identifying relevant experimental studies. 12-7 Population, Exposure, Comparison, Outcome, and Study design statement to define the parameters and provide a framework for identifying relevant epidemiologic studies. 12-9 Level of Biological Organization, Exposure, Comparison, Endpoint, and Study design statement to define the parameters and provide a framework for identifying relevant ecological studies. 12-11 Scientific considerations for evaluating the strength of inference from studies on the health effects of Pb. 12-22 External Review Draft 12-v DRAFT: Do not cite or quote ------- LIST OF FIGURES Figure 12-1 General process for developing Integrated Science Assessments. 12-4 Figure 12-2 Literature flow diagram for the Pb Integrated Science Assessment. 12-15 External Review Draft 12-vi DRAFT: Do not cite or quote ------- ACRONYMS AND ABBREVIATIONS AQCD Air Quality Criteria Document BLL blood lead level CASAC Clean Air Scientific Advisory Committee EPA Environmental Protection Agency FRN Federal Register Notice GFR glomerular filtration rate HAWC Health Assessment Workspace Collaborative HERO Health and Environmental Research Online IQ intelligence quotient IRP Integrated Review Plan ISA Integrated Science Assessment LECES Level of Biological Organization, Exposure, Comparison, Endpoint, and Study Design NAAQS National Ambient Air Quality Standards NASGLP North American Soil Geochemical Landscapes Project NHANES National Health and Nutrition Examination Survey ORD Office of Research and Development PECOS Population, Exposure, Comparison, Outcome, and Study Design PICOC Population, Intervention, Comparison, Outcome, and Context PQAPP Program-Level Quality Assurance Project Plan QA quality assurance QAPP Quality Assurance Project Plan RBC red blood cell External Review Draft 12-vii DRAFT: Do not cite or quote ------- APPENDIX 12 THE PROCESS FOR DEVELOPING THE INTEGRATED SCIENCE ASSESSMENT FOR LEAD i Summary of Public Resources for the Pb ISA This appendix describes the process for developing the Pb ISA, including literature search and screen methods; peer input and peer review; and public participation. This table summarizes the publicly available resources related to this ISA and its development. Readers looking for Federal Register Notices (FRNs) mav search http://www.reaulations.aov bv either the document citation number (the reference number to the specific FRN) or the Docket ID number (reference number for the overall docket that may house multiple FRNs, as well as public comments in response to those FRNs). Pb ISA External Review Draft httDs://cfDub.eDa.aov/ncea/isa/recordisDlav.cfm?deid= 357282 Federal Register Notices http://www.requlations.qov Request for Information Document Citation: 85 FR 40641 Docket ID: EPA-HQ-OAR-2020-0312-0001 Integrated Review Plan, Document Citation: 87 FR 13732 Volume 2 Docket ID: EPA-HQ-OAR-2020-0312-0010 Peer Input Workshop Document Citation: 87 FR 27147 Docket ID: EPA-HQ-ORD-2020-0701-0001 Integrated Review Plan https://www.epa.qov/naaqs/lead-pb-standards- planninq-documents-current-review Peer Input Workshop https://cfpub.epa.qov/ncea/isa/recordisplav.cfm?deid= 354420 Literature https://hero.epa.qov/hero/index.cfm/proiect/paqe/proie ct id/4081 Study Quality Evaluations https://hawc.epa.qov/assessment/100500318/ ISA Preamble https://cfpub.epa.qov/ncea/isa/recordisplav.cfm?deid= 310244 External Review Draft 12-1 DRAFT: Do not cite or quote ------- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 12.1 Introduction Integrated Science Assessments (ISAs) provide the scientific foundation for the review of the primary (health-based) and secondary (welfare1-based) National Ambient Air Quality Standards (NAAQS). ISAs contain a synthesis and evaluation of the most policy-relevant science using methods and approaches described in the Preamble to the Integrated Science Assessments (U.S. EPA. 2015b). hereafter "Preamble," which provides an overview of the ISA development process. This ISA for Pb builds upon the 2013 ISA (U.S. EPA. 2013a) and prior Air Quality Criteria Documents (AQCDs) for Pb from 1977 (U.S. EPA. 1977). 1986 (U.S. EPA. 1986). and 2006 (U.S. EPA. 2006). and includes literature published since September 2011, the literature cutoff date of the previous Pb ISA. In March 2022, the U.S. Environmental Protection Agency (EPA) released the first two volumes of the Integrated Review Plan (IRP) for the Pb NAAQS review. Volume 2 of the IRP (U.S. EPA. 2022) identifies policy-relevant issues (i.e., those intended to frame the review and focus it on the critical scientific and policy questions related to the adequacy of the standards) and describes key considerations in EPA's development of the Pb ISA. Volume 2 was made available for public comment and a consultation with by the Clean Air Scientific Advisory Committee (CASAC) Pb Review Panel at a public meeting on April 8. 2022. This ISA has been developed by U.S. EPA scientists in the Office of Research and Development (ORD), other U.S. EPA scientists with relevant experience, and external authors from ICF, an EPA contractor. The general ISA development steps are presented in Figure 12-1, though particular details can vary across assessments. This appendix supplements the 2015 ISA Preamble (U.S. EPA. 2015b) and Volume 2 of the IRP (U.S. EPA. 2022). and further describes the process of developing this ISA for Pb, including methods for documentation, literature review, study quality evaluation, public engagement, and quality assurance (QA). 12.2 Documentation 12.2.1. Literature Database: Health and Environmental Research Online To improve transparency, studies considered in the development of the ISAs are documented in the U.S. EPA Health and Environmental Research Online (HERO) database. The publicly accessible HERO project page for this ISA contains the references that were considered for inclusion and provides bibliographic information and abstracts. Within HERO, each reference has a unique HERO ID number. 1 Under The Clean Air Act section 302(h) (42 U.S.C. § 7602(h)), effects on welfare include "effects on soils, water, crops, vegetation, manmade materials, animals, wildlife, weather, visibility, and climate, damage to and deterioration of property, and hazards to transportation, as well as effects on economic values and on personal comfort and well-being." External Review Draft 12-2 DRAFT: Do not cite or quote ------- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 References can be viewed individually or filtered by appendix, discipline, or the draft in which they are referenced. Inclusion and exclusion decisions for references at each stage of screening are recorded by a tagging system and are documented in the HERO database. A two-step screening process (title and abstract screening and full-text screening) was used for this ISA; subsequent sections of this appendix discuss the screening process in greater detail. References that passed through title and abstract screening are tagged in HERO as "Title-Abstract Screening Included." Inclusion and exclusion decisions from full- text screening of references passing through title and abstract screening are tagged in HERO as "Full-Text Screening Included." References identified from sources other than literature searches were also screened using the same discipline-specific criteria, and inclusion and exclusion decisions for these references are also documented in HERO. Specific data about concentrations, experimental design, and results are reported within the appendices. 12.2.2. Study Quality Documentation: Health Assessment Workspace Collaborative Reference-specific information about study quality is documented in the U.S. EPA Health Assessment Workspace Collaborative (HAWC) for select health studies and can be accessed through the HAWC project page for this ISA. All decisions about full-text screening, including both relevance and quality, are additionally documented in the HERO database and on the publicly available HERO project page for this ISA. See Section 12.6 for a more detailed discussion about study quality. 12.3 Overview of the Process Steps for Developing Integrated Science Assessments As described in the Preamble and shown in Figure 12-1, developing an ISA consists of the following steps: literature search and study selection; evaluating study quality; developing initial draft materials for peer-input consultation; evaluating, synthesizing, and integrating evidence; and developing scientific conclusions and causality determinations (U.S. EPA. 2015b). External Review Draft 12-3 DRAFT: Do not cite or quote ------- Literature Search and Study Selection * Evaluation of Individual Study Quality After study selection, the quality of individual studies is evaluated by EPA or outside experts in the fields of atmospheric science, exposure assessment, dosimetry, animal toxicology, controlled human exposure studies, epidemiology, ecology, and otherwelfare effects, considering the design, methods, conduct, and documentation of each study. Strengths and limitations of individual studies that may affect the interpretation of the study are considered. * Develop Initial Sections Review and summarize new study results as well as findings and conclusions from previous assessments by category of outcome/effectand by discipline, e.g., toxicological studiesoflung function. Peer Input Consultation Review of initial draft materials by scientists from both outside and within EPA in public meeting orpublic teleconference. * Evaluation, Synthesis, and Integration of Evidence Integrate evidence from scientific disciplines - for example, toxicological, controlled human exposure, and epidemiologic study findings for a particular health outcome. Evaluate evidence for related groups of endpoints or outcomes to draw conclusions regarding health orwelfare effect categories, integrating health orwelfare effects evidence with information on mode of action and exposure assessment. Development of Scientific Conclusions and Causal Determinations Characterize weight of evidence and develop judgments regarding causality for health or welfare effect categories. Develop conclusions regarding concentration- or dose-response relationships, potentially at-risk populations, lifestages, or ecosystems. it Draft Integrated Science Assessment Evaluation and integration of newly published studies after each draft. Clean Air Scientific Advisory Committee Independent review of draft documents for scientific quality and sound implementation of causal framework; anticipated review of two drafts of ISA in public meetings. Public Comments Comments on draft ISA solicited by EPA Final Integrated Science Assessment Source: Modified from Figure II of the Preamble to the Integrated Science Assessment (U.S. EPA. 2015b). Figure 12-1 General process for developing Integrated Science Assessments. External Review Draft 12-4 DRAFT: Do not cite or quote ------- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 12.4 Relevance and Scope As a synthesis and evaluation of the most policy-relevant science, the Pb ISA includes information on atmospheric science, exposure assessment, experimental health studies, epidemiologic health studies, and studies of effects on terrestrial and aquatic ecosystems. For this ISA, "policy-relevant" science is described in Volume 2 of the IRP (U.S. EPA. 2022) as referring to "scientific information and analyses intended to address key questions related to the adequacy of the standards." Those "key questions" are also laid out in Volume 2 of the IRP. As stated in the Preamble (U.S. EPA. 2015b). "The key policy-relevant questions included in the IRP serve to clarify and focus the NAAQS review on the critical scientific and policy issues, including addressing uncertainties discussed during the previous review and newly emerging literature." The sections below describe the approaches and scoping statements used to identify relevant studies in each discipline. The use of scoping statements to define study relevance is consistent with recommendations by the National Academies of Sciences, Engineering, and Medicine for improving the design of risk assessment through planning, scoping, and problem formulation to better meet the needs of decision makers (NASEM. 2018). 12.4.1. Atmospheric Sciences Studies were considered relevant for inclusion in this ISA if they were judged to provide original data and to substantially advance our understanding of Pb emission sources; atmospheric and environmental processes (including chemistry and transport); measurement and estimation methods; or recent concentrations and trends. This approach to determining study relevance required judgments about whether a subject area of the research had the potential to inform policy specific to the NAAQS, and whether a paper published in the area provided sufficiently original results to add to the existing body of knowledge. Tablel2-1 shows the relevance criteria used for broadly identifying recent environmental research advances and knowledge gaps. These criteria are based on the approach described by Mengist et al. (2020). who formulated a Population, Intervention, Comparison, Outcome, and Context (PICOC) statement that designated the population as the population of scientific research work itself and the outcome as the assessment of its knowledge and gaps. Table 12-1 Population, Intervention, Comparison, Outcome, and Context statement to define the parameters and provide a framework for identifying relevant atmospheric science studies. Concept Application Population Include policy-relevant scientific research on Pb source emissions, environmental processes (including chemistry and transport), measurement and estimation methods, and concentration and trends. Intervention Assess policy-relevant scientific advances and knowledge gaps. External Review Draft 12-5 DRAFT: Do not cite or quote ------- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Comparison Evaluate emissions, concentrations, and their rates of change across sources, atmospheric and environmental processes, measurement and estimation methods, long-term temporal scales, seasons, diurnal cycles, geographic regions, and urban and neighborhood spatial scales. Outcome Identify policy-relevant scientific advances and knowledge gaps. Context Focus on policy-relevant research performed in the U.S. or Canada; for some topics, research performed outside of the U.S. or Canada can be excluded if sources or concentrations are not relevant to the U.S. or if the body of research is very large; for other topics, if source and concentration differences are not relevant to the topic or the number of publications is very small, non-U.S. research can be included. Pb = lead. 12.4.2. Exposure, Toxicokinetics, and Biomarkers The following guidelines were used to judge the relevance of studies examining Pb exposures, toxicokinetics, and biomarkers. Studies were included if they provided original data and substantially advanced understanding of Pb exposure through environmental media and other pathways; Pb toxicokinetics including uptake, distribution, metabolism, and elimination from the body; Pb biomarker measurement techniques; Pb biomarker concentration trends; and the relationships between Pb in environmental media and Pb biomarker concentrations, including biokinetic and empirical modeling of those relationships. Exposure studies pertaining to the U.S. population and U.S.-based Pb sources were preferred. Studies were included from outside the United States if these studies were judged to have important findings, with a focus on studies from Canada, western Europe, and Australia (i.e., areas with study populations and air quality characteristics most similar to the United States). If it was deemed that studies from the United States, Canada, western Europe, or Australia were not adequate (i.e., little to no information that advanced understanding of a particular topic was found), then it was necessary to consider all studies regardless of geographic location. For Pb toxicokinetics and biomarker measurement techniques, studies, regardless of geographic location, were considered since the physical location in which a study took place may have less bearing on results. Finally, although exposures in relation to Pb in ambient air and originating from air-related sources are the focus of the appendix, studies containing Pb concentrations in other media (soil, dietary sources, consumer products, occupational sources, and ammunition) were included because cumulative body burden can occur as a result of contributions from multiple exposure pathways (i.e., ingestion of Pb-containing soil by children) and the origin of Pb can be difficult to determine as stemming from an air-related source. 12.4.3. Health Relevance for studies that evaluate the relationship between Pb exposure and health effects was assessed using scoping statements that define the relevant Population, Exposure, Comparison, Outcome, External Review Draft 12-6 DRAFT: Do not cite or quote ------- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 and Study design (PECOS). Discipline-specific PECOS statements for epidemiologic and experimental studies were developed to establish inclusion criteria based on the objectives of the review, thereby facilitating identification of the most relevant literature to inform the Pb ISA (Table 12-2 and Table 12-3). In some cases, PECOS statements differ by health outcome depending on well-established areas of research; gaps in the literature; and inherent uncertainties in specific populations, exposure metrics, comparison groups, and study designs identified in the 2013 Pb ISA. Additionally, some epidemiologic PECOS statements were further refined to emphasize the strongest recent epidemiologic studies that address key uncertainties from the previous review; these PECOS refinements are identified and described in detail in the relevant appendices. The use of PECOS statements is widely accepted and often applied in the health disciplines for systematic review in risk assessment. PECOS statements for this ISA can also be found in each health effects appendix. 12.4.3.1. Experimental Studies For experimental studies (specifically animal exposure studies), the relevance evaluation focused on studies with appropriate study designs and relevant exposure concentrations (Table 12-2). The scope of the experimental evidence used for this ISA encompassed studies of nonhuman mammalian animal species with exposures that are relevant to the range of human exposures (blood Pb levels [BLLs] up to 30 (ig/dL, which is about one order of magnitude above the 95th percentile of the 2011-2016 National Health and Nutrition Examination Survey [NHANES] distribution of BLLs in children) (Egan ct al.. 2021). Table 12-2 Population, Exposure, Comparison, Outcome, and Study design statement to define the parameters and provide a framework for identifying relevant experimental studies. Concept Application Population Laboratory nonhuman mammalian animal species (i.e., mouse, rat, Guinea pig, minipig, rabbit, cat, dog; whole organism) at any lifestage (including preconception, in utero, lactation, peripubertal, and adult stages). Exposure Oral, inhalation, or intravenous routes administered to a whole animal (in vivo) that results in a BLLof30 |jg/dLor below.ab Comparison A concurrent control group exposed to vehicle-only treatment or untreated control. Outcome Cancer and noncancer health outcomes including cardiovascular, dermal, developmental, endocrine system, gastrointestinal, hematological, hepatic, immunological, metabolic syndrome, musculoskeletal, neurological, ocular, renal, reproductive, or respiratory effects. Study Controlled exposure studies of animals in vivo. Design BLL = blood lead level; Pb = lead. aPb mixture studies are included if they employ an experimental arm that involves exposure to Pb alone. This level is approximately an order of magnitude above the upper end of the distribution of U.S. young children's BLLs. The 95th percentile of the 2011-2016 NHANES distribution of BLL in children (1-5 years; n=2,321) is 2.66 |jg/dL (Eganet al.. 2021). and the External Review Draft 12-7 DRAFT: Do not cite or quote ------- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 proportion of individuals with BLLs that exceed this concentration varies depending on factors including housing age, geographic region, and a child's age, sex, and nutritional status. 12.4.3.2. Epidemiologic Studies To identify the most relevant epidemiologic literature, the body of evidence from the 2013 Pb ISA was considered in the development of the PECOS statements. Specifically, the scope of the current assessment is informed by well-established areas of research, gaps in the literature, inherent uncertainties in specific populations, exposure metrics, comparison groups, and study designs identified in the 2013 Pb ISA. The evaluation of epidemiologic studies focused on the association between exposure to Pb (as indicated by Pb levels in blood, bone, and teeth; validated environmental indicators of Pb exposure; or intervention groups in randomized trials and quasi-experimental studies) and an ensemble of health effects, including effects on the nervous system, cardiovascular effects, and reproductive and developmental outcomes (Table 12-3). Emphasis was placed on studies conducted in non-occupationally exposed populations, but recent longitudinal studies of occupational exposure to Pb published since the literature cutoff date for the 2013 Pb IS A were considered insofar as they addressed a topic that was of particular relevance to the NAAQS review (e.g., longitudinal studies designed to examine recent versus historical Pb exposure). Additionally, the following types of epidemiologic studies are generally considered to fall outside the scope and are not included in the ISA: review articles (which typically present summaries or interpretations of existing studies rather than bringing forward new information in the form of original research or new analyses); Pb poisoning studies or clinical reports (e.g., involving accidental exposures to very high amounts of Pb described in clinical reports that may be extremely unlikely to be experienced under ambient air exposure conditions); and risk or benefit analyses (e.g., that apply existing concentration-response functions or effect estimates to exposure estimates for differing cases). For some health outcomes for which the evidence assessed in the 2013 Pb ISA supported a "causal" relationship, the epidemiologic PECOS statements were refined in order to further emphasize the strongest recent epidemiologic studies that address the key uncertainties from the previous review and the scientific questions in Volume 2 of the IRP (U.S. EPA. 2022). These PECOS refinements, which are identified and described in detail in the relevant appendices, generally focus on the most informative study designs and relevant BLLs, and emphasize control for important potential confounders that were identified in the 2013 ISA. Studies that met the broader PECOS criteria in Table 12-3 but were no longer relevant under the refined criteria were still included in evidence inventories that summarize key study details, including study population, exposure assessment, confounders, and select results. External Review Draft 12-8 DRAFT: Do not cite or quote ------- 1 Table 12-3 Population, Exposure, Comparison, Outcome, and Study design statement to define the parameters 2 and provide a framework for identifying relevant epidemiologic studies. Population: Any human population, including specific populations or lifestages that might be at increased risk of a health effect. Exposure: Exposure to Pba as indicated by biological measurements of Pb in the body, with a specific focus on Pb in blood, bone, and teeth; validated environmental indicators of Pb exposure, or intervention groups in randomized trials and quasi-experimental studies. Comparison: Populations, population subgroups, or individuals with relatively higher versus lower levels of the exposure metric (e.g., per unit or log unit increase in the exposure metric, or categorical comparisons between different exposure metric quantiles). Outcome Nervous System Cardiovascular Renal Immune Hematological Reproductive Developmental Cancer Other Nervous system Cardiovascular Renal Immune system Hematological Developmental Reproductive Cancer Effects on the effects including effects including effects effects including effects including effects including effects, including incidence, hepatic system, cognitive function coronary heart including immunotoxicity, disruption of adverse altered age of mortality, or gastrointestinal (e.g., IQ decrement), disease, elevated systemic heme synthesis pregnancy puberty onset, related system, externalizing and hypertension serum inflammation, and and RBC outcomes (e.g., reduced fertility, biomarkers. endocrine internalizing and increased creatinine immune-based function. reduced fetal poor semen system, bone behaviors, blood pressure, levels and diseases. growth, preterm quality or and teeth, ocular psychopathological and lower birth, small for motility, and health, and effects, sensory cardiovascular- GFR. gestational age, miscarriage. respiratory organ function, motor related mortality. birth defects), as system. function, and well as postnatal neurodegenerative developmental diseases. effects. Study Design: Epidemiologic studies consisting of longitudinal and retrospective cohort studies, case-control studies, cross-sectional studies with appropriate timing of exposure for the health endpoint of interest, randomized trials, and quasi-experimental studies examining interventions to reduce exposures. Pb = lead, IQ = Intelligence quotient, GFR = glomerular filtration rate, RBC = red blood cell. aThe focus was on populations with nonoccupational Pb exposures, though recent longitudinal studies of occupational exposure to Pb were considered insofar as they addressed a topic that was of particular relevance to the NAAQS review (e.g., longitudinal studies designed to examine recent versus historical Pb exposure). bStudies that estimate Pb exposure by measuring Pb concentrations in particulate matter with a nominal mean aerodynamic diameter less than or equal to 10 |jm3 (PM10) and particulate matter with a nominal mean aerodynamic diameter less than or equal to 2.5 |jm3 (PM2.5) ambient air samples are only considered for inclusion if they also include a relevant biomarker of exposure. Given that size distribution data for Pb-PM are fairly limited, it is difficult to assess the representativeness of these concentrations to population exposure [Section 2.5.3 (U.S. EPA. 2013a)l. Moreover, data illustrating the relationships of Pb-PM-io and Pb-PIVhs with blood Pb levels are lacking. External Review Draft 12-9 DRAFT: Do not cite or quote ------- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 12.4.4. Welfare—Effects on Terrestrial and Aquatic Ecosystems For welfare effects (effects on terrestrial and aquatic ecosystems), scoping statements defining the Level of Biological Organization, Exposure, Comparison, Endpoint, and Study design (LECES) were used. EPA developed the LECES based on the PECOS with some concepts substituted to provide a better fit with ecological science. In the LECES, "population" (PECOS) is replaced with "level of biological organization" (LECES) and "outcome" (PECOS) is replaced with "endpoint" (LECES). A LECES statement was developed for terrestrial and aquatic ecosystems. For research evaluating ecological effects, emphasis was placed on recent studies published since the literature cutoff date of the 2013 Pb ISA that: (1) evaluated effects at concentrations at or near current environmental concentrations of Pb in soil, water, and sediment and (2) investigated effects on species, subspecies, or study populations of algae and plants, microbes, invertebrates, or vertebrates at any lifestage or in any biological community or ecosystem. Exposure concentrations, endpoints, and study types considered for this ISA that inform understanding of the ecological effects of Pb in terrestrial and aquatic systems are summarized further in the LECES statement (Table 12-4). In addition to the biological effects described in the LECES statement, other topics within scope included how chemical and biological modifying factors affect bioavailability in terrestrial, freshwater, and saltwater environments, as well as studies that address key uncertainties and limitations in the evidence identified in the 2013 ISA. Site-specific studies in non-U.S. locations that do not contribute to novel insights into Pb biogeochemistry or effects are considered outside of the scope of this ISA. Generally, studies on mine tailings, biochar, industrial effluent, sewage, ship breaking, bioremediation of highly contaminated sites, and ingestion of Pb shot, fishing tackle, or pellets are also not within the scope of the ISA due to the high concentration of Pb and lack of a connection to an air-related source or process. External Review Draft 12-10 DRAFT: Do not cite or quote ------- Table 12-4 Level of Biological Organization, Exposure, Comparison, Endpoint, and Study design statement to define the parameters and provide a framework for identifying relevant ecological studies. Level of Biological Organization: Species or subspecies, study populations of vegetation, microbes, invertebrates, or vertebrates, at any lifestage, or any biological community or ecosystem in terrestrial environments present in the United States or similar to those in the United States. Exposure: Short or long-term Pb concentrations in exposure media (e.g., soil or diet) that are most relevant to environmental concentrations of Pb in the United States.3 For soil, the cutoff value for screening of terrestrial studies of Pb exposure and effects was defined as a concentration of approximately 230 mg Pb/kg,b with higher concentrations considered if the study elucidates a mechanism or is an acute exposure and at least one concentration in the test series is in the range described above. Analytically verified exposure concentrations preferred; nominal concentrations considered. Terrestrial Comparison: A comparison to an unexposed laboratory control, a reference population, or site with no detectable exposure or with lower Pb Endpoint: Species or population effects including effects on growth, reproduction or development, neurobehavioral effects, reduced survival or fitness, carbon fixation and photosynthesis. At higher levels of biological organization endpoints include changes in community composition, altered ecosystem processes and functions, such as productivity, community composition, or shifts in genotypes or species, species extirpation, declines in total number of species or biomass, or decreased species richness. Study Design: Laboratory, mesocosm, observational or experimental field or gradient studies, or mechanistic modeling studies that estimate the effect of Pb on an organism, biological population, community, or ecosystem whose processes may be represented quantitatively (e.g., in a dynamic or steady state). Level of Biological Organization: Species and subspecies, study populations of vegetation, microbes, invertebrates, or vertebrates, at any lifestage, or any biological community or ecosystem in freshwater or saltwater environments and transition zones present in the United States, or similar to those in the United States, excluding the open ocean. Exposure: Short or long-term Pb concentrations in exposure media (e.g., water, sediment, or diet) that are most relevant to environmental concentrations of Pb in the United States.3 For freshwater or saltwater, the cutoff value for screening of Pb exposure and effects was defined as a concentration of approximately 10 |jg Pb/Lc with higher concentrations considered if the study elucidates a mechanism plausibly relevant at lower Aquatic concentrations. For sediments, exposure concentration of approximately 300 mg Pb/kg, dry weight.d For dietary pathways, at least one experimental group (prey) exposed to approximately 10 |jg Pb/L (aqueous cutoff value for screening) prior to a feeding study. If a study provides toxicity data on a previously untested organism grouping (such as Class, Order, Family) or for lower concentration studies of an organism with a protected status, studies were included even if concentrations exceeded cutoffs. Analytically verified exposure concentrations preferred; nominal concentrations considered. Comparison: A comparison to an unexposed laboratory control, a reference population, or site with no detectable exposure or with lower Pb exposure. External Review Draft 12-11 DRAFT: Do not cite or quote ------- Endpoint: Species or population effects including effects on growth, reproduction or development, neurobehavioral effects, reduced survival or fitness, carbon fixation and photosynthesis. At higher levels of biological organization endpoints include changes in community composition, altered ecosystem processes and functions, such as productivity, or shifts in genotypes or species, species extirpation, declines in total number of species or biomass, or decreased species richness. Study Design: Laboratory, mesocosm, observational or experimental field or gradient studies or mechanistic modeling studies that estimate the effect of Pb on an organism, biological population, community, or ecosystem whose processes may be represented quantitatively (e.g., in a dynamic or steady state). Pb = lead. Generally, studies on mine tailings, industrial effluent, land-applied sewage sludge, ship breaking, bioremediation of highly contaminated sites, and ingestion of Pb shot or pellets are not within the scope of the ISA due to a high concentration of Pb or lack of a connection to an air-related source or process; however, exceptions include studies conducted in biological systems with insufficient information on routes of Pb exposure. Generally excluded are studies of metal mixtures for which a specific effect of Pb was not separated unless conducted in biological systems with limited experimental evidence. Lastly, most site-specific studies conducted outside of North America that do not contribute novel insights on Pb biogeochemistry or effects are excluded. bThe cutoff value for screening of terrestrial studies of Pb exposure and effects is based on the values reported for soils of the conterminous United States in the 2013 United States Geological Survey report. "Geocnemical ana mineralogical data for soils of the conterminous United States" (Smith et al., 2013). This survey was conducted between 2007 and 2013 and sampled three soil horizons (surface, A, and C) at 4,857 nonurban, non-near-road sites. The Q1, median, mean, and Q3 values in surface soil (0-5 cm) for 4841 locations for which Pb data was available in North American Soil Geochemical Landscapes Project (NASGLP) were 13.5, 18.1, 25.8, and 23.9 mg Pb/kg soil. The Q1, median, mean, and Q3 values in the A horizon (relevant for plants, invertebrates, and microorganisms as well as burrowing mammals and reptiles) for 4841 locations for which Pb data was available in NASGLP were 13.2, 17.8, 22.2, and 23.2 mg Pb/kg soil. The 230 mg Pb/kg soil concentration cutoff is approximately one order of magnitude higher than the Q3 values from the survey. The cutoff value for screening of Pb concentration in water is based on United States Geological Survey National Water Quality Assessment sampling for which the 2006 Pb AQCD reported summary statistics as of the time (U.S. EPA. 2006). The 99th and 95th percentile dissolved Pb values were 5.44 |jg/L and 1.1 |jg/L, respectively (see Table 6-2 in the 2013 ISA) (U.S. EPA. 2013b). A more relevant upper bound value for dissolved Pb would be closer to 1 |jg/L, and 10 |jg/L is one order of magnitude above that value. As dissolved Pb concentrations in saltwater would be expected to be no higher—and generally, lower—than concentrations in freshwater (due to odds of greater proximity of freshwaters to anthropogenic sources and less access to mixing), an upper bound for saltwater would reasonably be expected to be lower than that for freshwater concentrations. dThe cutoff value for Pb screening in sediment is based on an older survey of urban and reference lake sediments across the U.S. (Mahler et al.. 2006) and further supported by evidence from more recent regional survey data. A median 1990s concentration for 35 U.S. sites (Table 2 of (Mahleret al.. 2006)) ot 16 mg HD/Kg was reported and the paper concluded that Pb had decreased since 1970s, with the 1990s median being 40% lower than the 1970s median. For saltwater, Kim et al. (2004) reported samples in a lower Delaware coastal saltmarsh that would be expected to have much less historic and non-air contamination. The concentrations for the upper depths (0 to 5 cm), dated to reflect the 90s through the early 2000s, range from 20 to 30 mg/kg. Thus, 30 mg/kg appears to be a more appropriate upper bound value for freshwater and saltwater sediments, and 300 mg Pb/kg is one order of magnitude above that value. External Review Draft 12-12 DRAFT: Do not cite or quote ------- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 12.5 Literature Search and Study Selection EPA uses a structured approach to identify relevant studies for consideration and inclusion in the ISAs. The search for relevant literature in this review began with publishing a Request for Information FRN (July 7, 2020, 85 FR 40641). This FRN announced the initiation of this Pb NAAQS review and invited the public to submit relevant research studies and data that have been published, accepted for publication, or presented at a public scientific meeting since January 1, 2011, providing some overlap with the previous Pb ISA wherein the literature considered extended to September 2011. Literature submitted by the public in response to this FRN can be viewed in EPA's HERO database . EPA reviewed these studies for relevance following the literature screening process described in this appendix. In addition to the Request for Information FRN, EPA applied systematic review methodologies to identify peer-reviewed scientific literature relevant to this ISA. The literature searching and screening methodology used for this ISA generally followed the process depicted in Figure 12-2. The process began with a combination of keyword searches and citation network searches to find relevant literature in PubMed and Web of Science published between September 2011 and December 2020. This literature search strategy was designed to maximize precision1 and recall2 for each discipline (i.e., health, welfare effects, atmospheric sciences, and exposure). The literature then went through two levels of screening to identify relevant studies: (1) title and abstract screening using SWIFT-Active Screener (SWIFT-AS), and (2) full-text screening if the peer-reviewed paper was deemed potentially relevant after initial title and abstract screening. Keyword searches were developed for each appendix using strings of relevant search terms to capture literature relevant to Pb and the topics in each appendix. For human health search results, automatic topic classification, a process that uses machine learning to classify references based on a set of already identified relevant papers, was then used to separate epidemiologic references from experimental references. In addition to keyword searches, topic-specific citation network searches for all disciplines were used to identify publications that have cited references included in the 2013 Pb ISA. This approach allows for relevance ranking based on the number of references in a bibliography that match references in the seed set. In addition, a small number of references were also identified for consideration in this ISA through identification of relevant literature by U.S. EPA expert scientists; recommendations received in response to the Request for Information and the Peer Input Workshop; and by review of citations included in previous assessments or in newly identified literature. Reviewers during the Peer Input Workshop were 1 Precision is the proportion of relevant references relative to all references retrieved in a literature search. 2 Recall is the proportion of relevant references identified by screening, relative to the total number of relevant references that exist. External Review Draft 12-13 DRAFT: Do not cite or quote ------- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 asked to provide a list of additional references (if any) that EPA should consider for the ISA, including those published since the initial literature search. Following the 2022 Peer Input Workshops and prior to the release of the External Review Draft, EPA updated the initial literature searches. These searches were conducted in response to comments received on the IRP Volume 2 from the CASAC consultation, and feedback received during the Peer Input Workshops. The updated literature searches targeted key, policy-relevant topics (i.e., "scientific information and analyses that address key questions related to the adequacy of the standards" (U.S. EPA. 2022)) most informative to reviewing the Pb NAAQS to ensure that literature published since the cutoff date of the initial literature searches was captured. For the selected health effects (nervous system, cardiovascular, and reproductive and developmental health effects) the updated literature search captured literature (both epidemiologic and experimental studies) published between December 2020 and June 2022. For effects of Pb in terrestrial and aquatic ecosystems, the updated literature search included the date range of August 2020 to June 2022 and focused on studies reporting effects on growth, reproduction, and development or survival. For atmospheric sciences, the same search strings used for the original search were applied to the date range of August 2020 to June 2022. Title and abstract and full-text screening steps were similarly applied as described for the initial literature searches. External Review Draft 12-14 DRAFT: Do not cite or quote ------- Keyword Search (164,814) Citation Network Search (131,199) r Literature Search Results (deduplicated) (277,817) References from Other Sources I (1,393) J f \ Level 1 Title-Abstract Screening Included (12,703) V J r Level 2 Full-Text Screening Included (2,828) Level 1 Title-Abstract Screening Excluded (265,122) _y Level 2 Full-Text Screening Excluded (11,268) Included in Pb Integrated Science Assessment (2,828) Pb = lead. Figure 12-2 Literature flow diagram for the Pb Integrated Science Assessment. External Review Draft 12-15 DRAFT: Do not cite or quote ------- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 12.5.1. Title and Abstract Screening Consistent with the 2020 Ozone ISA (U.S. EPA. 2020b). EPA used SWIFT-AS to perform the first-level screening of the search results for relevance, based on the title and abstract. SWIFT-AS is a web-based literature screening software application that uses machine learning to allow screeners to efficiently screen literature for relevance. It ranks search results by descending likely relevance using a bag-of-words approach and Latent Dirichlet Allocation, trained by both the screener's inclusion and exclusion decisions and a positive training set, when supplied (Howard et al.. 2016). EPA used such a set of "seed references" (references known to be relevant from the previous Pb ISA). As references are screened and tagged as relevant or not relevant, the ranking model is further trained to sort the remaining literature, pushing predicted relevant literature to the top of the queue. EPA screened literature until SWIFT-AS estimated that 95% of relevant literature was included, a threshold considered comparable to human error rates (Howard et al.. 2020; Cohen et al.. 2006). 12.5.1.1. Atmospheric Science Initial literature related to air quality, atmospheric chemistry, fate, and transport discussed in Appendix 1 of this ISA, Lead Source to Concentration, was identified using a strategy consistent with the approach described in Volume 2 of the IRP (U.S. EPA. 2022). The search involved both a citation network search and a keyword search component. For all air sections (Appendix 1. Sections 1.2. 1.3.1. 1.3.4. 1.4. and 1.5). the citation network search identified all publications that cited any references from the 2013 Pb ISA chapter, Ambient Lead: Source to Concentration, and a keyword search was developed to capture additional relevant publications in the Web of Science database that did not cite any 2013 Pb ISA references. The search string was tested to confirm it would achieve greater than 99% recall when applied to the 2013 Pb ISA chapter references. Literature for the fate and transport sections on soil and water (Appendix 1. Sections 1.3.2 and 1.3.3) was obtained in a similar manner, using the citation network and keyword searches used for terrestrial and aquatic ecosystems (Section 12.5.1.4). SWIFT-AS was used for title and abstract screening with seed references from the 2013 Pb ISA. Decisions about inclusion or exclusion were guided by the PICOC statement (Table 12-1). After the Peer Input Workshop (Section 12.7.3), the literature search was updated using the same two search strings originally applied to the Web of Science database for references published after the original cutoff date. Consistent with the initial literature search, EPA screened these additional studies for relevance using SWIFT-AS; decisions about relevance were guided by the PICOC statement. External Review Draft 12-16 DRAFT: Do not cite or quote ------- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 12.5.1.2. Exposure Assessment Initial literature related to ambient Pb exposure, toxicokinetics, and biomarkers discussed in Appendix 2 of this ISA, Exposure, Toxicokinetics, and Biomarkers, were identified using a keyword search strategy consistent with the approach described in Volume 2 of the IRP (U.S. EPA. 2022). This search involved both a citation network search and a keyword search component. The citation network search was designed to identify all publications that cited any references from Chapter 3: Exposure, Toxicokinetics, and Biomarkers of the 2013 Pb ISA (U.S. EPA. 2013b). Two separate keyword searches were developed to capture additional relevant publications that did not cite any 2013 Pb ISA references from the Web of Science and PubMed databases, respectively. The inclusion and exclusion terms used for each search were developed independent of one another to maximize the relevance for each database. Given the extensive overlap between publications that contained information on Pb exposure, biomarkers, and toxicokinetics, both keyword searches were performed on all topics in the appendix. Results from both searches were combined and literature was de- duplicated. SWIFT-AS was used for title and abstract screening. The SWIFT-AS algorithm was initially trained using references from Chapter 3: Exposure, Toxicokinetics, and Biomarkers of the 2013 Pb ISA as seed references. Literature tags were developed to organize results by subsection. Judgments of inclusion and exclusion were based on guidelines described in the Relevance and Scope section above (Section 12.4.2). Following the 2022 Peer Input Workshop, peer input reviewers determined that EPA had identified most of the relevant literature. Suggested additions were screened for relevance and judgments of inclusion and exclusion were, again, based on guidelines described in the Relevance and Scope section above (Section 12.4.2). 12.5.1.3. Health Epidemiologic and experimental studies (i.e., animal toxicology studies) examining health effects from Pb exposure were targeted using a broad keyword search and citation network search strategy consistent with Volume 2 of the IRP (U.S. EPA. 2022). EPA screened the identified literature for relevance against PECOS statements for each health endpoint (see Section 12.4.3), using SWIFT-AS. The SWIFT-AS algorithm was trained initially using seed references from the 2013 Pb ISA (U.S. EPA. 2013b). During this first phase of screening, EPA tagged experimental studies reporting health outcome- related literature that potentially informs the biological or chemical events associated with phenotypic effects, including in vitro, in vivo (by various routes of exposure), ex vivo, and in silico studies. Although External Review Draft 12-17 DRAFT: Do not cite or quote ------- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 these studies do not necessarily meet PECOS criteria, they were tracked as a supplemental evidence stream to inform biological plausibility. Following the 2022 Peer Input Workshop, EPA updated the literature search for the following health outcome categories using the same keyword and citation network search strategy: nervous system effects (Appendix 3); cardiovascular effects (Appendix 4); and reproductive and developmental effects (Appendix 8). The updated literature search focused on key, policy-relevant health outcomes for which a substantial body of recent literature conducted at relevant Pb biomarker levels was expected, as suggested by results from the initial search. Consistent with the initial literature search, EPA screened these additional studies for relevance using SWIFT-AS and the PECOS statements. 12.5.1.4. Welfare—Effects on Terrestrial and Aquatic Ecosystems Studies potentially relevant to Pb effects in terrestrial or aquatic ecosystems (freshwater and saltwater) were identified using a broad keyword search and citation network search strategy consistent with the approach described in Volume 2 of the IRP (U.S. EPA. 2022). EPA screened the identified literature for relevance against LECES statements using SWIFT-AS (Table 12-4). The SWIFT-AS algorithm was trained initially using seed references from the 2013 Pb ISA (U.S. EPA. 2013b). Studies that were not within the scope of the ISA or that did not meet the criteria for inclusion based on title and abstract screening (Section 12.4.4 and Table 12-4) were excluded from further consideration. Following the 2022 Peer Input Workshop, EPA updated the literature search and screened additional studies in SWIFT-AS for relevance using the LECES statements. 12.6 Study Selection: Full-Text Evaluation of Studies EPA performed a second level of screening based on assessment of the full text of the references remaining after the first-level screening (title and abstract). EPA continued to use relevance criteria outlined in Section 12.4 during full-text screening. 12.6.1. Individual Study Quality After selecting studies for inclusion based on relevance, individual study quality was evaluated by considering the design, methods, conduct, and documentation of each study, but not the study results. For ISAs, the overall individual study quality evaluation process is described in the Preamble (U.S. EPA. 2015b). which outlines a base set of questions for consideration when evaluating the scientific quality of studies, intended for use in both human health and ecological studies: External Review Draft 12-18 DRAFT: Do not cite or quote ------- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 • Were the study designs, study groups, methods, data, and results clearly presented in relation to the study objectives to allow for study evaluation? Were limitations and any underlying assumptions of the design and other aspects of the study stated? • Were the ecosystems, study site(s), study populations, subjects, or organism models adequately selected, and are they adequately defined to allow for meaningful comparisons between study or exposure groups? • Are the air quality, exposure, or dose metrics of adequate quality and are they sufficiently representative of or pertinent to ambient air? • Are the welfare effect measurements meaningful, valid, and reliable? • Were likely covariates or modifying factors adequately controlled or taken into account in the study design and statistical analysis? • Do the analytical methods provide adequate sensitivity and precision to support conclusions? • Were the statistical analyses appropriate, properly performed, and properly interpreted? Worldwide, formal methods for individual study quality evaluation are much better developed for human health research than for ecological, atmospheric, and exposure studies. The study quality approach for health and welfare are described further below. For this ISA, atmospheric and exposure studies were considered acceptable if they were published in a peer-reviewed journal, though further scrutiny was applied during full-text screening of exposure studies to identify whether the exposure assessment methods were clearly described; the selected exposure assessment methods were appropriate for the research question evaluated; the assumptions of the method(s) were clearly stated; the uncertainties and limitations of the methods were clearly stated; and QA testing had been performed. No studies in the atmospheric or exposure, toxicokinetics, and biomarkers appendices were deemed to have unacceptable study quality. Study quality was a final step in full-text screening to decide whether to include a study in the ISA. Any references that did not pass the study quality review and deemed uninformative for the purposes of this assessment were excluded from the ISA. Studies that passed both the relevance screening and the study quality evaluation were included in the ISA. The combination of approaches described in this section are intended to produce a comprehensive collection of pertinent studies needed to address the key scientific issues that are examined in the ISA. 12.6.1.1. Health As described in the Preamble, causality determinations are informed by integrating evidence across scientific disciplines (e.g., exposure, animal toxicology, epidemiology) and related outcomes, and by judgments of the strength of inference in individual studies. For health outcomes, study quality is evaluated using a uniform approach that considers study strengths and limitations, including the possible roles of chance, confounding, and other biases that may influence results. The process for individual study quality evaluation has been refined by discipline with each successive ISA based on input and feedback External Review Draft 12-19 DRAFT: Do not cite or quote ------- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 from numerous reviews by CASAC. Recent ISAs have developed study quality criteria tables to provide clarity on important aspects of study quality for health outcomes and serve as the foundation for the review of individual health studies (U.S. EPA. 2020b. 2019a. 2017. 2016). These aspects describe the characteristics of study elements (e.g., study design, exposure assessment, potential confounding factors) that can increase or decrease confidence in the study results. Where possible, study elements, such as exposure assessment and confounding (i.e., bias due to a relationship with the outcome and correlation with exposures to Pb) are tailored to address factors specific to health studies of Pb exposure. Thus, judgments on the ability of a study to inform the relationship between an air pollutant and health vary depending on the specific pollutant being assessed. Table 12-5 describes the aspects considered in evaluating study quality of animal toxicological and epidemiologic studies considered for inclusion in this ISA. The specific aspects of each domain listed in Table 12-5 are consistent with current best practices for reporting or evaluating health science data.1 Additionally, the aspects are compatible with published U.S. EPA guidelines related to cancer, neurotoxicity, reproductive toxicity, and developmental toxicity (U.S. EPA. 2005. 1998. 1996. 1991). These aspects were not used as a checklist to determine if a study should be included or excluded; the presence or absence of particular features in a study did not necessarily lead to the conclusion that a study was less informative or should be excluded from consideration in the ISA. Instead, reviewers considered each element of a study and made a final binary judgment (include or exclude) based on overall study quality. Study quality considerations for individual studies may be discussed within the health appendices of this ISA in instances when specific aspects affect the interpretation of a study, either increasing or decreasing confidence in study results. Importantly, judgments were made without considering the outcome of a study (e.g., whether an adverse health outcome was observed), and these aspects were not used as criteria for determining the causal relationship between Pb exposure and health effects. As described in the Preamble, causality determinations were based on judgments of the overall strengths and limitations of the collective body of available studies and the coherence of evidence across scientific disciplines. Table 12-5 is not intended to be a complete list of aspects that define a study's ability to inform the relationship between Pb and health effects, but it describes the major aspects considered in this ISA to evaluate studies. A limited number of studies have been excluded based on consideration of the study quality aspects described in Table 12-5. For example, specific epidemiologic studies have been excluded due to the evaluation (solely) of univariate models; lack of statistical power to detect an association; and inadequate or missing description of methods. In addition, specific toxicological studies were excluded from consideration because observed effects could not be reliably attributed to Pb exposure; application of an experimental model that was not intended for use with animals; reporting data that directly conflict with results of different experiments described in the same publication without explanation, along with 1 For example, NTP OHAT approach (Roonev et al.. 20141. IRIS Preamble (U.S. EPA. 2013c). ToxRTool (Klimisch elal.. 1997). STROBE guidelines (von Elm et al.. 20071. and ARRIVE guidelines (Kilkenny et al.. 20101. External Review Draft 12-20 DRAFT: Do not cite or quote ------- 1 2 3 4 5 6 7 8 9 10 11 12 mislabeled figures, which together reduce confidence the conclusions of the study; and for conducting experiments performed in animals that were not approved by an institutional animal care and use committee. To document the study quality evaluation for a subset of the most policy-relevant health studies, a narrative approach was used to provide nuanced and transparent documentation of the strengths and limitations that support expert judgment for individual studies. Narrative reviews were completed for epidemiologic studies of Pb exposure and full-scale IQ in children, which played a significant role in the development of the Policy Assessment in the 2016 Pb NAAQS review. The study quality tables (Table 12-5) were used to develop prompting questions for each study domain designed to assist in the narrative documentation of study quality, ensuring the inclusion of consistent information across reviewers. The narrative reviews, along with the prompting questions, were recorded in HAWC and can be accessed on the HAWC project page. External Review Draft 12-21 DRAFT: Do not cite or quote ------- Table 12-5 Scientific considerations for evaluating the strength of inference from studies on the health effects of Pb. Study Design Epidemiology Inference is stronger for studies that clearly describe the primary and any secondary aims of the study, or specific hypotheses being tested. Information including the age of the population studied, study period, and study location is used to aid in the interpretation of findings because Pb exposure has declined over time and exposures vary depending on proximity to Pb sources. For observational studies of Pb exposure and health outcomes, inference is considered to be stronger for prospective cohort studies and case control studies nested within a cohort (e.g., for rare diseases) than other case control, cross sectional, or ecologic studies. Cohort studies can better inform the temporality of exposure and effect. Other designs can have uncertainty related to the appropriateness of the control group or validity of inference about individuals from group level data. Study design limitations can bias health effect associations in either direction. Animal Toxicology The primary and any secondary objectives of the study, or specific hypotheses being tested should be clearly described. Studies should include appropriately matched control exposures (e.g., to clean filtered air, time matched). Studies should use experimental conditions that provoke little concern for concern for uncontrolled variables or different practices across groups. Groups should be subjected to identical experimental procedures, conditions, and animal care (e.g., housing and husbandry). Study Population/Test Model Epidemiology There is greater confidence in results for study populations that are recruited from and representative of the target population. Studies with high participation and low dropout over time that is not dependent on exposure or health status are considered to have low potential for selection bias. Clearly specified criteria for including and excluding subjects, and the reporting of baseline information on participants that are lost to follow up can aid assessment of selection bias. For populations with an underlying health condition, independent, clinical assessment of the health condition is valuable, but self-report of physician diagnosis generally is considered to be reliable for respiratory and cardiovascular diseases.15 Comparisons of groups with and without an underlying health condition are more informative if groups are from the same source population. Selection bias can influence results in either direction or may not affect the validity of results but rather reduce the generalizability of findings to the target population. Animal Toxicology The animal species and strain used for toxicology investigations must be appropriate for the study goals and have relevance to a corresponding outcome in humans. Ideally, studies should report species, strain, substrain, genetic background, age, sex, and weight. Where applicable, approval of study protocols by appropriate institutional animal care and use committees must be obtained. Unless data indicate otherwise, PECOS-relevant laboratory nonhuman mammalian species and strains are considered appropriate for evaluating effects of Pb exposure. It is preferred that the authors test for effects in both sexes across multiple lifestages and report the result for each group separately. External Review Draft 12-22 DRAFT: Do not cite or quote ------- Pollutant Epidemiology The focus is on studies evaluating Pb exposure. Animal Toxicology Studies should focus on the effects of Pb exposure on health outcomes; however, information from mixture studies in which Pb is a component may be informative if the study employs a Pb-only treatment arm with appropriate control group. Ideally, studies should report the source, purity, and form of Pb (e.g., lead acetate) used. Exposure Assessment or Assignment Epidemiology General population studies using Pb biomarkers (e.g., blood, bone, or tooth Pb concentrations) are emphasized. The most useful biomarker of exposure is one that reflects the exposure timing and duration that is appropriate to the underlying pathogenetic processes (e.g., recent, cumulative over lifetime, or cumulative over a developmental^ sensitive window). Blood Pb (PbB) is typically measured in venous or capillary blood specimens using a variety of laboratory analytical techniques. Validated analytical methods with lower LODs, such as inductively coupled plasma mass spectrometry or graphite furnace atomic absorption spectrometry, are preferred. Capillary blood Pb determinations have greater potential for contamination during collection, resulting in greater measurement error, particularly at concentrations approaching the LOD. While PbB is most commonly measured in samples of whole blood, the small fraction of Pb in plasma (<1%) is the more toxicologically active fraction of the circulating Pb. Bone Pb is most commonly measured in the tibia, calcaneus, patella, or finger bone via x-ray fluorescence (XRF). Recent studies favor measurement of the patella for estimating trabecular bone Pb, because it has more bone mass and may afford better measurement precision than the calcaneus. Bone measurements are typically expressed in units of |jg Pb per g bone mineral. This convention may potentially introduce variability into the bone Pb measurements related to variation in bone density. Notably, lower bone mineral density is associated with greater measurement uncertainty in bone Pb, which can have important implications for studies in populations for whom low bone mineral density is more common (e.g., older women). Measurements of Pb in hair, saliva, nails, urine, and feces suffer from high interlaboratory variability, low reproducibility, and a lack of reliable reference values. A more detailed discussion of exposure biomarkers can be found in Appendix 2. Animal Toxicology For this assessment, the administration of Pb by oral, inhalation, or intravenous routes are considered relevant. Studies that resulted in measured blood Pb levels <30 |ig/dl_ will be used in the health section narrativesd. Studies should characterize Pb concentration, environmental temperature and relative humidity, and/or have measures in place to adequately control the exposure conditions. All studies should include exposure control groups (e.g., dosing vehicle, or no Pb treatment) that are appropriate to the route, duration of exposure, and study design. Studies should randomize assignment to exposure groups and, where possible, conceal allocation to research personnel. Blinding of research personnel to study group may not be possible due to animal welfare and experimental considerations; however, differences in the monitoring or handling of animals in all groups by research personnel should be minimized. External Review Draft 12-23 DRAFT: Do not cite or quote ------- Outcome Assessment Epidemiology Inference is stronger when outcomes are assessed or reported without knowledge of exposure status. Knowledge of exposure status could produce artifactual associations. Confidence is greater when outcomes assessed by interview, self-report, clinical examination, or analysis of biological indicators are defined by consistent criteria and collected by validated, reliable methods. Independent, clinical assessment is valuable for incidence of disease, but report of physician diagnosis has shown good reliability.15 Validated questionnaires for subjective outcomes such as symptoms are regarded to be reliable,0 particularly when collected frequently and not subject to long recall. For biological samples, the stability of the compound of interest and the sensitivity and precision of the analytical method is considered. If not based on knowledge of exposure status, errors in outcome assessment tend to bias results toward the null. Animal Toxicology Endpoints should be assessed in the same manner for control and exposure groups (e.g., time after exposure, evaluation methods/procedures, endpoint evaluation) using valid, reliable methods. Wherever possible, the limit of detection for quantitative assays should be given. For each experiment and each experimental group, including controls, precise details of all procedures carried out should be provided. Time of the endpoint evaluations is a key consideration that will vary depending on endpoint evaluated. Endpoints should be assessed at time points that are appropriate for the research questions. Additionally, in order to preclude reporting bias, studies should report results for all experimental procedures conducted. All animals used in a study should be accounted for, and rationale for exclusion of animals (e.g., attrition) or data should be specified and reasonable given the study design. Other Potential Confounding Factors'" Epidemiology Factors are considered to be potential confounders if demonstrated in the scientific literature to be related to health effects and correlated with Pb. Not accounting for confounders can produce artifactual associations; thus, studies that statistically adjust for multiple factors or control for them in the study design are emphasized. Less weight is placed on studies that adjust for factors that mediate the relationship between Pb and health effects, which can bias results toward the null. Confounders vary according to study design and health effect of interest, and may include, but are not limited to the following: socioeconomic status, parental caregiving, race, age, medication use, smoking status, noise, urbanicity, and environmental and/or occupational exposures. Animal Toxicology Preference is given to studies using experimental and control groups that are matched for individual level characteristics (e.g., strain, sex, body weight, litter size, and food and water consumption) and time varying factors (e.g., seasonal and diurnal patterns). External Review Draft 12-24 DRAFT: Do not cite or quote ------- Statistical Methodology Epidemiology Multivariable regression models that include potential confounding factors are emphasized. However, multipollutant/mixtures models (i.e., models that include more than two pollutants/metals) are considered to produce too much uncertainty due to copollutant collinearity to be informative. Models with interaction terms aid in the evaluation of potential confounding as well as effect modification. Sensitivity analyses with alternate specifications for potential confounding inform the stability of findings and aid in judgments of the strength of inference from results. In the case of multiple comparisons, consistency in the pattern of association can increase confidence that associations were not found by chance alone. Statistical methods that are appropriate for the power of the study carry greater weight. For example, categorical analyses with small sample sizes can be prone to bias results toward or away from the null. Statistical tests such as correlation coefficients, f-tests, and chi-squared tests are not considered sensitive enough for adequate inferences regarding Pb-health effect associations. For all methods, the effect estimate and precision of the estimate (i.e., width of 95% CI) are important considerations rather than statistical significance. Animal Toxicology Statistical methods should be clearly described and appropriate for the study design and research question (e.g., correction for multiple comparisons). Specific sample sizes are not criteria for inclusion or exclusion; ideally, the sample size should provide adequate power to detect hypothesized effects. Because statistical tests have limitations, consideration is given to both trends in data and reproducibility of results. Results should be presented quantitatively in the appropriate format for the data (e.g., continuous data ideally should not be presented as categorical or dichotomized) and separately by sex and cohort. a(U.S. 2008). bMuraia et al. (2014): Weakley et al. (2013): Yang et al. (2011): Heckbert et al. (2004): Barr et al. (2002): Muhaiarine et al. (1997): Toren et al. (1993). cBurnev et al. (1989). dStudies not including a blood lead biomarker were tracked during study screening but were not included/evaluated in the health section narratives. eMany factors evaluated as potential confounders can be effect measure modifiers (e.g., season, comorbid health condition) or mediators of health effects related to Pb (comorbid health condition). External Review Draft 12-25 DRAFT: Do not cite or quote ------- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 12.6.1.2. Welfare—Effects on Terrestrial and Aquatic Ecosystems Generally, the field of study quality evaluation is much more robust for human health research than for ecological research. Study quality is still very important for ecological research, and U.S. EPA staff have relied on the criteria listed in the Preamble as criteria for reviewing the quality of individual studies within this ISA. A limited number of studies were excluded based on consideration of these study quality questions and application of the LECES statement. The main reasons studies were eliminated were: exposure concentrations that exceeded concentration cutoff, as specified in the LECES; no report of Pb concentration; Pb was part of a mixture of metals with no testing of the independent effect of Pb; a lack of statistical testing for endpoints of interest; inadequate or missing description of methods; or inadequate study design. 12.7 Peer Review and Public Participation Peer review is an important component of any scientific assessment, as formalized in the guidance found in the Peer Review Handbook (U.S. EPA. 2015a'). This ISA follows all the policies and procedures identified therein. Additionally, this ISA follows the Information Quality Guidelines (U.S. EPA. 2002V EPA has designated this ISA as a Highly Influential Scientific Assessment, which is defined by The Office of Management and Budget's Final Information Quality Bulletin for Peer Review (hereafter, "Peer Review Bulletin") as: A subset of Influential Scientific Information that is a scientific assessment (i.e., an evaluation of a body of scientific or technical knowledge, which typically synthesizes multiple factual inputs, data, models, and assumptions and applies the best professional judgment to bridge uncertainties in the available information) that "could have a potential impact of more than $500 million in any year on either the public or private sector" or "is novel, controversial, or precedent-setting, or has significant interagency interest." (https://obamawhitehouse.archives.gov/omb/memoranda 1V2005 m05-03/). As such, there are additional review and transparency steps required in the release of this information. These steps are described below in Section 12.7.1. CASAC also plays an important role in reviewing this ISA (see Section 12 7 5). 12.7.1. Request for Information Consistent with the Preamble, a Request for Information was published in the Federal Register on July 7, 2020 (85 FR 40641). The purpose of this Request for Information was announcing the beginning of the review cycle of the air quality criteria and the Pb NAAQS and inviting the public to submit relevant External Review Draft 12-26 DRAFT: Do not cite or quote ------- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 research studies and data that had been published, accepted for publication, or presented at a public scientific meeting since January 1, 2011. The public was given 60 days to respond to this FRN; EPA received eight comments via the Federal eRulemaking Portal (http: //www. regulations .gov. Docket ID: EPA-HQ-OAR-2020-0312). Literature submitted by the public in response to this FRN can be viewed in EPA's HERO database. 12.7.2. Integrated Review Plan Following the Request for Information, EPA prepared a multi-volume IRP: Volume 1 provides background information on the air quality criteria and standards for Pb; Volume 2 addresses the general approach for the review and planning of the ISA; and Volume 3 is the planning document for quantitative analyses considered in the policy assessment. Volume 2 of the IRP (U.S. EPA. 2022). which describes the plan for developing the ISA, was discussed by CASAC at a public meeting on April 8. 2022. Availability of Volume 2 of the IRP for public comment was announced in the Federal Register on March 10, 2022 (87 FR 13732). The public was given the opportunity to respond, and U.S. EPA received one public comment via the Federal eRulemaking Portal (http://www.regulations.gov. Docket ID: EPA-HQ-OAR- 2020-0312-0010). Following the CASAC public meeting, documentation of the meeting and written comments from individual CASAC members were sent to the U.S. EPA Administrator in a letter dated April 22, 2022 (https://casac.epa.gov/ords/sab/f?p=l 13:12:17516491975646::: 12::). 12.7.3. Peer Input The role of peer input is described in the Preamble, as well as the Peer Review Handbook (U.S. EPA. 2015a. b). After a thorough literature search and screening process, EPA developed preliminary draft appendices for initial peer input. Causality determinations had yet to be developed. Peer input is a process that allows EPA to gather early-in-the-process feedback from subject-matter experts, internal and external to EPA, to ensure that the ISA captures relevant new literature and is focused on the most policy- relevant findings. Peer input serves as a supplement to other peer-review mechanisms and does not replace a thorough external peer review completed by CASAC. Peer input for this ISA occurred as a series of four webinar workshops, which EPA announced in an FRN on May 6, 2022 (87 FR 27147, Docket ID: EPA-HQ-ORD-2020-0701). The four workshops were organized by subject: Effects of Pb in Terrestrial and Aquatic Ecosystems; Epidemiologic and Toxicological Evidence for Health Effects of Pb Exposure; Ambient Pb: Source to Concentration; and Exposure, Toxicokinetic, and Pb Biomarkers. Workshops were facilitated by EPA's contractor, ICF. Peer input reviewers were selected by ICF, with input from EPA, in accordance with EPA's Peer Review Handbook (U.S. EPA. 2015a). External Review Draft 12-27 DRAFT: Do not cite or quote ------- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Peer input reviewers were given the following charges: • Correct technical errors and identify critical gaps. • Consider how clearly and logically the appendices and content within the sections are organized. • Indicate how accurately scientific information is characterized, whether advances in knowledge in the recent literature have been adequately highlighted, and whether emphasis has been placed on the most informative, policy-relevant literature. • Identify any key studies missing, (including those published after the early 2021 literature search dates for the draft materials), especially any associated with the effects of Pb from ambient air. Provide full citations for suggested references. • Indicate any specific issues that should be considered or highlighted that will be important for integrating evidence across disciplines. There were additional topic-specific charge questions. Peer input reviewers were not asked to correct typos or grammatical errors. During the workshops, peer input reviewers affirmed that EPA included the relevant literature, though some additional studies were identified for EPA's consideration. Following the workshop, EPA considered comments and incorporated revisions based on the reviewers' feedback. Suggested studies were screened for relevance as described for the initial literature searches and incorporated if they met the inclusion criteria (see Sections 12.4 and 12.5.1). 12.7.4. Internal Technical Review The U.S. EPA ORD guidelines require an internal technical review process prior to any external dissemination of scientific information. Consistent with this policy, the draft ISA was reviewed by EPA subject-matter experts. Following the technical review, EPA used the reviewers' comments to revise the document. 12.7.5. Clean Air Scientific Advisory Committee Peer Review Two sections of the Clean Air Act, Sections 108 and 109 [42 U.S.C. 7408 and 7409], govern the periodic review and establishment of the NAAQS (2020a). With respect to CASAC, section 109(d)(2) addresses the appointment and advisory functions of an independent scientific review committee. Section 109(d)(2)(A) requires the Administrator to appoint this committee, which is to be composed of "seven members including at least one member of the National Academy of Sciences, one physician, and one person representing State air pollution control agencies." Section 109(d)(2)(B) states that the independent scientific review committee periodically "shall complete a review of the criteria... and the national primary and secondary ambient air quality standards... and shall recommend to the External Review Draft 12-28 DRAFT: Do not cite or quote ------- 1 Administrator any new... standards and revisions of existing criteria and standards as may be 2 appropriate..." Since the early 1980s, this independent review function has been performed by CASAC. 3 CASAC serves as the official peer review mechanism for this ISA. As a Highly Influential 4 Scientific Assessment, the review process is also governed by the Peer Review Bulletin. All requirements 5 in the Peer Review Bulletin regarding the selection of reviewers, information access, opportunity for 6 public participation, transparency, and management of the peer-review process and reviewer selection 7 have been met for the CASAC review of this ISA. 12.8 Quality Assurance 8 QA helps ensure that U.S. EPA conducts high-quality science that can be used to inform 9 policymakers, industry, and the public. Agency-wide, the U.S. EPA Quality System provides the 10 framework for planning, implementing, documenting, and assessing work performed by the Agency, and 11 for carrying out required quality assurance and quality control (QA/QC) activities. Additionally, the 12 Quality System covers the implementation of the U.S. EPA Information Quality Guidelines (U.S. EPA. 13 2002). This ISA follows all Agency guidelines to ensure a high-quality document. 14 Within EPA, Quality Assurance Project Plans (QAPPs) are developed to ensure that all Agency 15 materials meet a high standard for quality. EPA has developed a Program-Level QAPP (PQAPP) for the 16 ISA Program to describe the technical approach and associated QA/QC procedures associated with the 17 ISA Program. All QA objectives and measurement criteria detailed in the PQAPP have been employed in 18 developing this ISA. 19 QA checks were conducted on numerical entries throughout the ISA. At a minimum, numerical 20 values from every fifth citation were verified for accuracy by an independent EPA scientist against the 21 original source, and any errors were subsequently corrected. Furthermore, publicly available databases 22 (e.g., HERO) have their own QA processes. 23 A Technical Systems Audit of this ISA occurred in July 2022 by an independent contractor, 24 Neptune and Company, Inc. The auditor verified that QA procedures were adequately performed and 25 documented. 12.9 Conclusion 26 This appendix describes the overall process of developing the Pb ISA: literature search and 27 screening methods; study quality evaluation; peer input and peer review; documentation; and QA. 28 Overall, EPA has a robust set of policies and procedures in place to ensure the highest quality products. In 29 developing this ISA, EPA has followed all the appropriate processes and endeavored to add additional External Review Draft 12-29 DRAFT: Do not cite or quote ------- 1 steps as practicable and needed (e.g., use of SWIFT-AS, scoping statements, and documentation of 2 individual study quality). 12.10 References Barr. RG; Herbstman. J; Speizer. FE; Camargo. CA. Jr. (2002). Validation of self-reported chronic obstructive pulmonary disease in a cohort study of nurses. Am J Epidemiol 155: 965-971. http://dx.doi.org/10.1093/aie/155.lQ.965. Burnev. PG; Laitinen. LA: Perdrizet. S; Huckauf. H: Tattersfield. AE: Chinn. S; Poisson. N; Heeren. A: Britton. JR; Jones. T. (1989). Validity and repeatability of the IUATLD (1984) Bronchial Symptoms Questionnaire: an international comparison. Eur Respir J 2: 940-945. Cohen. AM: Hersh. WR: Peterson. K: Yen. PY. (2006). Reducing workload in systematic review preparation using automated citation classification. J Am Med Inform Assoc 13: 206-219. http://dx.doi.org/10.1197/iamia.M1929. Egan. KB: Cornwell. CR; Courtney. JG: Ettinger. AS. (2021). Blood lead levels in U.S. children ages 1- 11 years, 1976-2016. Environ Health Perspect 129: 37003. http://dx.doi.org/10.1289/EHP7932. Heckbert. SR; Kooperberg. C: Safford. MM: Psatv. BM; Hsia. J: McTiernan. A: Gaziano. JM; Frishman. WH: Curb. JD. (2004). Comparison of self-report, hospital discharge codes, and adjudication of cardiovascular events in the Women's Health Initiative. Am J Epidemiol 160: 1152-1158. http://dx.doi.org/10.1093/aie/kwh314. Howard. BE: Phillips. J: Miller. K; Tandon. A: Mav. D; Shah. MR: Holmgren. S: Pelch. KE; Walker. V: Roonev. AA: Macleod. M: Shah. RR: Thaver. K. (2016). SWIFT-Review: A text-mining workbench for systematic review. Syst Rev 5: 87. http://dx.doi.Org/10.l 186/sl3643-016-0263-z. Howard. BE: Phillips. J: Tandon. A: Maharana. A: Elmore. R; Mav. D; Sedvkh. A: Thaver. K; Merrick. BA; Walker. V: Roonev. A: Shah. RR. (2020). SWIFT-Active Screener: Accelerated document screening through active learning and integrated recall estimation. Environ Int 138: 105623. http://dx.doi.Org/10.1016/i.envint.2020.105623. Kilkenny. C: Browne. WJ: Cuthill. IC: Emerson. M; Altman. DG. (2010). Improving bioscience research reporting: The ARRIVE guidelines for reporting animal research [Review]. PLoS Biol 8: el000412. http://dx.doi.org/10.1371/iournal.pbio. 1000412. Kim. G: Alleman. LY; Church. TM. (2004). Accumulation records of radionuclides and trace metals in two contrasting Delaware salt marshes. Mar Chem 87: 87-96. http://dx.doi.Org/10.1016/i.marchem.2004.02.002. Klimisch. HJ; Andreae. M; Tillmann. U. (1997). A systematic approach for evaluating the quality of experimental toxicological and ecotoxicological data. Regul Toxicol Pharmacol 25: 1-5. http://dx.doi.org/10.1006/rtph.1996.1076. Mahler. BJ: van Metre. PC: Callender. E. (2006). Trends in metals in urban and reference lake sediments across the United States, 1970 to 2001. Environ Toxicol Chem 25: 1698-1709. http://dx.doi.org/10.1897/05-459Rl. External Review Draft 12-30 DRAFT: Do not cite or quote ------- Mcngist. W; Soromessa. T; Lcgcsc. G. (2020). Method for conducting systematic literature review and meta-analysis for environmental science research. MethodsX 7: 100777. http://dx.doi.Org/10.1016/i.mex.2019.100777. Muhaiarine. N; Mustard. C; Roos. LL; Young. TK; Gelskev. DE. (1997). Comparison of survey and physician claims data for detecting hypertension. J Clin Epidemiol 50: 711-718. http://dx.doi.org/10.1016/S0895-4356(97)00019-X. Murgia. N; Brisman. J; Claesson. A; Muzi. G; Olin. AC; Toren. K. (2014). Validity of a questionnaire- based diagnosis of chronic obstructive pulmonary disease in a general population-based study. BMC Pulm Med 14: 49. http://dx.doi.org/10.1186/1471-2466-14-49. NASEM (National Academies of Sciences. Engineering, and Medicine). (2018). Progress toward transforming the Integrated Risk Information System (IRIS) program: A 2018 evaluation. Washington, DC: National Academies Press, http://dx.doi.org/10.17226/25086. Roonev. AA; Bovles. AL; Wolfe. MS: Bucher. JR; Thaver. KA. (2014). Systematic review and evidence integration for literature-based environmental health science assessments. Environ Health Perspect 122: 711-718. http://dx.doi.org/10.1289/ehp.1307972. Smith. DB: Cannon. WF: Woodruff. LG: Solano. F: Kilburn. JE: Fev. PL. (2013). Geochemical and mineralogical data for soils of the conterminous United States. (Data Series 801). Reston, VA: U.S. Department of the Interior, U.S. Geological Survey, http://pubs.usgs.gov/ds/801/. Toren. K: Brisman. J: Jarvholm. B. (1993). Asthma and asthma-like symptoms in adults assessed by questionnaires: A literature review [Review]. Chest 104: 600-608. http://dx.doi.Org/10.1378/chest.104.2.600. U.S. EPA. (2008). Integrated science assessment for sulfur oxides: Health criteria [EPA Report]. (EPA/600/R-08/047F). Research Triangle Park, NC: U.S. Environmental Protection Agency, Office of Research and Development, National Center for Environmental Assessment- RTP. http://cfpub.epa.gov/ncea/cfm/recordisplav.cfm?deid=198843. U.S. EPA (U.S. Environmental Protection Agency). (1977). Air quality criteria for lead [EPA Report]. (EPA-600/8-77-017). Washington, DC. http://nepis.epa.gov/Exe/ZvPURL.cgi?Dockev=20013GWR.txt. U.S. EPA (U.S. Environmental Protection Agency). (1986). Air quality criteria for lead: Volume I of IV [EPA Report]. (EPA-600/8-83/028aF). Research Triangle Park, NC. http://cfpub.epa.gov/ncea/cfm/recordisplav.cfm?deid=32647. U.S. EPA (U.S. Environmental Protection Agency). (1991). Guidelines for developmental toxicity risk assessment. Fed Reg 56: 63798-63826. U.S. EPA (U.S. Environmental Protection Agency). (1996). Guidelines for reproductive toxicity risk assessment [EPA Report]. (EPA/630/R-96/009). Washington, DC: U.S. Environmental Protection Agency, Risk Assessment Forum. https://nepis.epa.gov/Exe/ZvPURL.cgi?Dockev=30004YQB.txt. U.S. EPA (U.S. Environmental Protection Agency). (1998). Guidelines for neurotoxicity risk assessment [EPA Report]. (EPA/630/R-95/00IF). Washington, DC: U.S. Environmental Protection Agency, Risk Assessment Forum, http://www.epa.gov/risk/guidelines-neurotoxicitv-risk-assessment. U.S. EPA (U.S. Environmental Protection Agency). (2002). Guidelines for ensuring and maximizing the quality, objectivity, utility, and integrity of information disseminated by the Environmental Protection Agency [EPA Report]. (EPA/260R-02-008). Washington, DC: U.S. Environmental Protection Agency, Office of Environmental Information. https://www.epa.gov/sites/production/files/2017-03/documents/epa-info-qualitv-guidelines.pdf. External Review Draft 12-31 DRAFT: Do not cite or quote ------- U.S. EPA (U.S. Environmental Protection Agency). (2005). Notice of availability; Documents entitled: Guidelines for carcinogen risk assessment and supplemental guidance for assessing susceptibility from early-life exposure to carcinogens. Fed Reg 70: 17765-17817. U.S. EPA (U.S. Environmental Protection Agency). (2006). Air quality criteria for lead (Final report, 2006): Volume I of II [EPA Report], (EPA/600/R-05/144aF). Washington, DC. http: //cfpub. epa.gov/ncea/ CFM/recordisplav. cfm ?deid= 158823. U.S. EPA (U.S. Environmental Protection Agency). (2013a). Integrated science assessment for lead [EPA Report], (EPA/600/R-10/075F). Washington, DC. https://nepis.epa.gov/Exe/ZvPURL.cgi?Dockev=P100K82L.txt. U.S. EPA (U.S. Environmental Protection Agency). (2013b). Integrated science assessment for lead (contains errata sheet created 5/12/2014) [EPA Report]. (EPA/600/R-10/075F). Washington, DC. https://nepis.epa.gov/exe/ZvPURL.cgi?Dockev=P100K82L.txt. U.S. EPA (U.S. Environmental Protection Agency). (2013c). Toxicological review of trimethylbenzenes (CASRN 25551-13-7, 95-63-6, 526-73-8, and 108-67-8) in support of summary information on the Integrated Risk Information System (IRIS): revised external review draft [EPA Report]. (EPA/635/R-13/171a). Washington, D.C.: U.S. Environmental Protection Agency, National Center for Environmental Assessment. http://vosemite.epa.gov/sab/SABPRODUCT.NSF/b5d8alce9b07293485257375007Q12b7/eele28 0e77586de985257b65005d37e7!OpenDocument. U.S. EPA (U.S. Environmental Protection Agency). (2015a). Peer review handbook [EPA Report] (4th ed.). (EPA/100/B-15/001). Washington, DC: U.S. Environmental Protection Agency, Science Policy Council, https://www.epa.gov/osa/peer-review-handbook-4th-edition-2015. U.S. EPA (U.S. Environmental Protection Agency). (2015b). Preamble to the Integrated Science Assessments [EPA Report]. (EPA/600/R-15/067). Research Triangle Park, NC: U.S. Environmental Protection Agency, Office of Research and Development, National Center for Environmental Assessment, RTP Division. https://cfpub.epa.gov/ncea/isa/recordisplav.cfm?deid=310244. U.S. EPA (U.S. Environmental Protection Agency). (2016). Integrated science assessment (ISA) for oxides of nitrogen: Health criteria (Final report, Jan 2016) [EPA Report]. (EPA/600/R-15/068). Washington, DC. https://cfpub.epa.gov/ncea/isa/recordisplav.cfm?deid=310879. U.S. EPA (U.S. Environmental Protection Agency). (2017). Integrated science assessment for sulfur oxides: Health criteria [EPA Report]. (EPA/600/R-17/451). Washington, DC. https://www.epa.gov/isa/integrated-science-assessment-isa-sulfur-oxides-health-criteria. U.S. EPA (U.S. Environmental Protection Agency). (2019a). Integrated Science Assessment (ISA) for particulate matter (final report, Dec 2019) [EPA Report]. (EPA/600/R-19/188). Washington, DC. https://cfpub. epa.gov/ncea/isa/recordisplav. cfm?deid=347534. U.S. EPA (U.S. Environmental Protection Agency). (2019b). Ozone ISA Study Quality Evaluations - Health. U.S. EPA. Definitions. CAA § 302: 42 USC § 7602 (2020a) U.S. EPA (U.S. Environmental Protection Agency). (2020b). Integrated Science Assessment (ISA) for ozone and related photochemical oxidants (final report, Apr 2020) [EPA Report]. (EPA/600/R- 20/012). Washington, DC. https://nepis.epa.gov/Exe/ZvPURL.cgi?Dockev=P1011 lKI.txt. External Review Draft 12-32 DRAFT: Do not cite or quote ------- U.S. EPA (U.S. Environmental Protection Agency). (2022). Integrated review plan for the National Ambient Air Quality Standards for lead. Volume 2: Planning for the review and the Integrated Science Assessment [EPA Report]. (EPA-452/R-22-003b). Research Triangle Park, NC: U.S. Environmental Protection Agency, Office of Air Quality Planning and Standards and Office of Research and Development. https://nepis.epa.gOv/Exe/Z vPURL.cgi?Dockev=P10148PX.txt. von Elm. E; Altman. DG; Egger. M; Pocock. SJ; Gotzschc. PC; Vandenbroucke. JP. (2007). The strengthening the reporting of observational studies in epidemiology (strobe) statement: guidelines for reporting observational studies [Review]. PLoS Med 4: e296. http://dx.doi.org/10.1371/iournal.pmed.004Q296. Weakley. J: Webber. MP; Ye. F; Zeig-Owens. R: Cohen. HW; Hall. CB; Kelly. K; Prezant. DJ. (2013). Agreement between obstructive airways disease diagnoses from self-report questionnaires and medical records. Prev Med 57: 38-42. http://dx.doi.Org/10.1016/i.vpmed.2013.04.001. Yang. CL; To. T; Fotv. RG; Stieb. DM; Dell. SD. (2011). Verifying a questionnaire diagnosis of asthma in children using health claims data. BMC Pulm Med 11. http://dx.doi.Org/10.l 186/1471-2466- 11-52. External Review Draft 12-33 DRAFT: Do not cite or quote ------- |