G V Vv vC ^ PRO^ OFFICE OF INSPECTOR GENERAL Catalyst for Improving the Environment Evaluation Report The Effectiveness of the Office of Children's Health Protection Cannot Yet Be Determined Quantitatively Report No. 2004- P- 00016 May 17, 2004 ------- Report Contributors: Jerri Dorsey Laurie Adams Jeffrey Harris Abbreviations EPA Environmental Protection Agency OCHP Office of Children's Health Protection OIG Office of Inspector General ORD Office of Research and Development ------- I '% UNITED STATES ENVIRONMENTAL PROTECTION AGENCY % ,"^^7 a WASHINGTON, D.C. 20460 % T % ^ \ pr0^° OFFICE OF INSPECTOR GENERAL May 17, 2004 MEMORANDUM SUBJECT: The Effectiveness of the Office of Children's Health Protection Cannot Yet Be Determined Quantitatively Report No. 2004-P-00016 FROM: Kwai Chan /<*/ Assistant Inspector General for Program Evaluation TO: Stephen Johnson Acting Deputy Administrator Bill Saunders Acting Director, Office of Children's Health Protection This is a final report on the subject review conducted by the Office of Inspector General (OIG) of the U.S. Environmental Protection Agency (EPA). This report contains findings that describe the problems the OIG identified and corrective actions the OIG recommends. This report represents the opinion of the OIG and the findings contained in this report do not necessarily represent the final EPA position. Final determinations on matters in the report will be made by EPA managers in accordance with established resolution procedures. Action Required In accordance with EPA Manual 2750, you are required to provide a written response to this report within 90 days of the date of this report. You should include a corrective actions plan for agreed upon actions, including milestone dates. We have no objections to the further release of this report to the public. For your convenience, this report will be available at http://www.epa.gov/oig/. If you or your staff have any questions, please contact me at (202) 566-0827 or Jeffrey Harris, Director for Program Evaluation, Cross-Media Issues, at (202) 566-0831. ------- ------- Executive Summary Children are more vulnerable to environmental health hazards than the general population. Recent Environmental Protection Agency (EPA) reports have indicated that children face significant and unique threats from such hazards. These hazards range from air pollution and lead-based paint in older homes to treatment-resistant microbes in drinking water and industrial chemicals. During this review, we therefore sought to determine: What is EPA's agenda for fostering children's health and how does the Office of Children's Health Protection (OCHP) ensure its achievement? What are the impediments to OCHP ensuring the achievement of the Agency's National Agenda to Protect Children's Health From Environmental Threats? How well does OCHP plan and coordinate children's health activities within the Agency? OCHP was established in 1997 to formalize and integrate EPA's efforts on behalf of children, to coordinate those efforts with other government agencies, and to carry out Executive Order 13045 ("Protection of Children From Environmental Health Risks and Safety Risks," issued April 27, 1997) and implement the Agency's "National Agenda to Protect Children's Health From Environmental Threats," announced September 1996. In 2002, EPA announced its Aging Initiative to promote environmental health protection to older persons and named OCHP as the Agency's lead for that Initiative as well. While OCHP is responsible for implementing the National Agenda, it is not directly responsible for many of the goals outlined in the National Agenda. OCHP's mission is not a task that can be undertaken in isolation, and a full and active partnership with stakeholders must be established and fostered. However, we found that there was no overall, coordinated strategy integrating children's environmental health efforts into the Agency as a whole and no active communication process in place among the program offices and OCHP. Further, OCHP has not had a permanent Director since April 2002. Recognizing that OCHP cannot implement the National Agenda without collaborating and partnering with stakeholders, the lack of a permanent director could have a negative impact on such efforts. We found that OCHP has no formal mechanism in place to ensure performance results or assess the relationships between program costs, activities, and results. OCHP has a strategic planning process that includes an annual office planning i ------- meeting followed by a mid-year meeting to assess the status on the current year's plan and to begin consideration of potential focus areas for the following year. However, despite these meetings, there is no formal tracking to ensure that goals are met. More specifically, data and information systems are not available to measure, analyze, and demonstrate overall performance specific to National Agenda on a continuing basis. We recommended that the Acting Deputy Administrator: expedite the appointment of a permanent Director for OCHP, establish an official children's health contact within each media program office to improve coordination and communication, and direct OCHP to make improvements to its annual planning process to include a methodology to set priorities to ensure resources are being allocated to those problems that pose the greatest environmental risks to children, as well as periodic meetings with the program offices. The Agency concurred with most of our recommendations and positions, and agreed to explore options for improving coordination within EPA and ways to better measure progress and results. However, the Agency expressed concern that the report did not recognize the role of OCHP and the accomplishments made to protect the Nation's children. We had noted in the report that many of EPA's program and regional offices have projects focused on children's environmental health protection; our concern was that OCHP's role was not clear. Also, while the Agency commented that it was not beneficial to define parameters for aging, we believe it is necessary for the Agency to develop a clear and concise definition of the target audience before it can take sufficient action on issues regarding that audience. We summarized the comments and provided our evaluations at the end of each chapter. The full text of EPA's memorandum and comments are in Appendix A, while additional attachments to that memorandum are available at http://www.epa.gov.oig/. ------- Table of C Executive Summary i Chapters 1 Introduction 1 2 Impediments to OCHP Effectively Implementing the National Agenda 7 3 Strengths, Weaknesses in OCHP Implementation Efforts Noted 15 Appendices A Auditee's Response 19 B Offices Visited and Contacted 25 C Distribution 27 ------- ------- Chapter 1 Introduction Purpose The Office of Inspector General (OIG) initiated a review to determine how the Environmental Protection Agency (EPA) planned, prioritized, coordinated, implemented, and evaluated activities related to children's health protection and the environment. This is the second in a series of planned reports1 on EPA children's health activities. During this review, we sought to determine: What is EPA's agenda for fostering children's health and how does the Office of Children's Health Protection (OCHP) ensure its achievement? What are the impediments to OCHP ensuring the achievement of the Agency's National Agenda to Protect Children's Health From Environmental Threats? How well does OCHP plan and coordinate children's health activities within the Agency? Background Children's Unique Health Threats and Susceptibilities According to recent EPA reports, children face significant and unique threats from environmental hazards and industrial chemicals. These hazards range from air pollution and lead-based paint in older homes to treatment-resistant microbes in drinking water. It has been documented that children are more vulnerable to environmental hazards for many reasons. For example, "The EPA Children's Environmental Health Yearbook" of June 1998 noted that: Children spend more time close to the ground and engage in more hand-to-mouth activities; and In proportion to body weight, children breathe more air, drink more water, and consume more food than adults, thus receiving higher doses of contaminants. Further, it has been documented that children are more susceptible because of the immaturity of their biochemical and physiological functions. For example, air pollutants that would produce only slight breathing difficulties in adults may 1 The first was Report Number 2003 -M-000017, Selected Children's Health Annual Performance Measures for Goal 8, Sound Science, issued September 26, 2003. ------- contribute to a more serious breathing problem in young children because children have smaller airways. Evolution of National and EPA Children's Health Policy A 1993 National Academies' of Science report, "Pesticides in the Diets of Infants and Children," noted that the Federal government needed to improve some of its scientific and regulatory procedures to afford children greater protection from possible adverse health effects of pesticides in their diets. On April 21, 1997, the Presidential Executive Order 13045, "Protection of Children From Environmental Health Risks and Safety Risks," was issued directing Federal agencies to place a high priority on protecting children from environmental and safety risks and ensure that their policies, programs, activities, and standards address disproportionate risks to children. Additionally, the executive order established the President's Task Force on Environmental Health Risks and Safety Risks to Children. In October 2001, Executive Order 13229 was signed extending the mandates of the original order, and the Task Force until April 2003. In April 2003, the original executive order was amended extending the Task Force for an additional two years. During the 1990s, EPA took steps to identify environmental risks unique to children. An outline of key actions taken by EPA during the 1990s is in Table 1.1. Table 1.1: Key Actions Taken by EPA 1995: EPA Issued Agency-Wide Policy to ensure that EPA consistently and explicitly evaluates environmental health risks of infants and children. 1996: EPA Announced National Agenda to Protect Children's Health from Environmental Threats, in which the Agency called for a national commitment to ensure a healthy future for children. 1997: Office of Children's Health Protection Established by EPA to formalize and integrate the Agency's efforts on behalf of children. 1998: Children's Health Research Program Initiated by the Office of Research and Development (ORD) to conduct research and provide methods to reduce uncertainties in EPA risk assessments for children. These steps included the issuing of a report in 1996, "Environmental Health Threats to Children," that set an agenda for a national commitment to ensure a healthy future for children. The National Agenda calls for the Agency to ensure that all standards set by EPA protect children, there is a scientific research 2 ------- strategy that focuses on gaps related to children, new policies are developed as needed, communities' right to know is expanded, educational efforts are expanded, and needed funding is provided. Creation of OCHP OCHP was established in 1997 to formalize and integrate the Agency's efforts on behalf of children, to coordinate those efforts with other government agencies, to carry out Executive Order 13045, and to implement the Agency's Agenda. OCHP's original mission was to "make the protection of children's health a fundamental goal of public health and environmental protection in the United States." The mission statement was revised to incorporate the Aging Initiative announced in 2002 (see Chapter 2). The current mission statement is to "make the health protection of children and the aging a fundamental goal of public health and environmental protection in the United States and around the world." OCHP's overall goal is to ensure that every individual, community, organization, corporation, and government agency understands the link between children's health and the environment and takes positive action to improve children's environmental health. To accomplish its overall mission and goal, OCHP focuses its efforts on two broad categories: (1) building infrastructure and capacity; and (2) "getting the word out." Building Infrastructure and Capacity: OCHP's plans include investing efforts to institutionalize children's environmental health protection within Federal agencies, States, and the private sector, as well as within EPA. Due to the small size of the office, OCHP considers EPA program offices and regions to be its partners, as well as external interest groups. "Getting the Word Out": OCHP's outreach plans include making communities, parents, care-givers, school systems, etc., aware of environmental risks to children and providing these groups with information and/or resources that might enable them to take their own future actions. OCHP's annual budget is approximately $6 million, which includes an estimated $4 million in extramural resources (grants and contracts). The office as of our review consisted of 14 staff members, divided into the following four speciality area teams: 3 ------- Table 1.2: OCHP's Four Specialty Area Teams Team Description Outreach and Partnerships This team helps EPA program and regional offices identify and promote children's health issues in public information, education, and training, and to identify successful outreach efforts for use by others. Regulations, Economics, Data Analysis This team conducts a retrospective review of EPA's existing standards to select standards that should be protective of children. Science This team assesses current research science agenda and has developed an integrated cross-Agency agenda of children's environmental health issues by working with ORD. Aging Initiative In October of 2002, EPA's Administrator announced the EPA Aging Initiative. This team is charged to develop a national agenda for the Agency in protecting the elderly from environmental health threats. OCHP is currently acting as the lead office for this Initiative. Internal Stakeholders According to the organizational proposal creating OCHP, their internal stakeholders include each of the media program offices at EPA, as well as the Agency's research, administrative, and policy programs. The Office of Enforcement and Compliance Assurance, especially its Office of Environmental Justice, also plays a key role in implementing the goals and objectives of OCHP. At the front lines of the Agency's efforts to protect children's health are EPA's regional offices. Each EPA region has a Children's Health Coordinator whose primary role is to ensure that all regional programs consider children within their on-going work. During Fiscal Year 2001, the Agency's budget for issues related to children's health peaked at approximately $71 million. As shown in the following chart, the budget for children health protection activities declined to approximately $62 million in Fiscal Year 2003. 4 ------- Children's Health Funding 80,000,000 67,634,100 70,000,000 60,000,000 50,000,000 40,000,000 30,000,000 20,000,000 10,000,000 2000 2002 71,176,200 32,419,900 67,580,100 62,390,100 1999 2001 2003 Fiscal Year Scope and Methodology In determining how the Agency plans, prioritizes, implements, and coordinates children's environmental health protection, we interviewed a number of Agency officials. We gathered information from OCHP and EPA program offices, as well as EPA Regional Children's Health Coordinators, to determine activities planned and implemented regarding children's environmental health protection, to identify the various roles, and to obtain documentation. We also interviewed outside stakeholders and representatives from other Federal agencies to document the Agency's coordination efforts with these entities regarding children's environmental health protection issues. A list of the offices visited and contacted is in Appendix B. We evaluated OCHP practices against criteria that the EPA OIG has adopted in assessing systems/organizations. The EPA OIG developed "Assessing Organizational Systems: A User's Guide" (November 2002) as a tool in assessing organizations. This guide's components are based on key program management practices consistent with the President's Management Agenda, the Government Performance and Results Act, and the Office of Management and Budget's efforts to assess effectiveness endorsed by the Agency. We performed our evaluation in accordance with Government Auditing Standards, issued by the Comptroller General of the United States. We performed our field work from November 18, 2002, through October 31, 2003. 5 ------- 6 ------- Chapter 2 Impediments to OCHP Effectively Implementing the National Agenda During our review, we identified several impediments to OCHP's overall effectiveness. While OCHP is responsible for implementing the National Agenda, it is not directly responsible for many of the goals outlined in the Agency's National Agenda. Additionally, according to OCHP, its mission is not a task that can be undertaken in isolation, and a full and active partnership with stakeholders must be established and fostered. However, we found that there was no overall, coordinated strategy and no active communication process in place among the program offices and OCHP. The Agency needs to develop a coordinated strategy with milestones and targets, establish a Children's Health contact within each media program office, and aid in coordinating children's health issues within the Agency. OCHP has not had a permanent Director since April 2002. Recognizing that OCHP cannot implement the National Agenda without collaborating and partnering with stakeholders, the lack of a permanent director could have a negative impact on such efforts. Further, EPA named the OCHP as the lead for the Agency's Aging Initiative following the announcement of the Initiative in 2002 which increased OCHP's workload and areas of responsibility. The Agency needs to determine whether that Initiative should be lead by the same office responsible for children's environmental health and if so, provide it with additional resources to adequately protect both populations. EPA Lacks Overall Children's Health Strategy OCHP is responsible for integrating the Agency's efforts on behalf of children. According to OCHP, its mission is not a task that can be undertaken in isolation, and a full and active partnership with stakeholders must be established and fostered. While many EPA program and regional offices had projects focused on children's environmental health, we found there was no overall, coordinated strategy to establish and foster a full and active partnership with stakeholders. While OCHP is responsible for implementing the National Agenda, it is not directly responsible for many of the goals outlined in the National Agenda. See table 2.1 that follows for further details. 7 ------- Table 2.1: National Agenda to Protect Children's Health from Environmental Threats Seven Steps of The National Agenda Ensure that all standards set by EPA are protective of any heightened risk faced by children; Develop a scientific research strategy focus on the gaps in knowledge regarding child-specific susceptibility and exposure to environmental pollutants; Develop new, comprehensive policies to address cumulative and simultaneous exposures faced by children; Expand community right to know allowing families to make informed choices concerning environmental exposures to their children; Provide parents with basic information so they can take individual responsibility for protecting their children from environmental health threats in homes, schools, and communities; Encourage and expand educational efforts with health care providers and environmental professionals so they can identify, prevent, and reduce environmental health threats to children; and Provide the necessary funding to address children's environmental health as a top priority among relative health risks. For example, the National Agenda specifically states that the Agency is to ensure that all standards set by EPA are protective of any heightened risk faced by children; however OCHP does not set standards within EPA. Additionally, OCHP is not responsible for setting comprehensive policies to address cumulative and simultaneous exposures to children. While OCHP does not have direct control over many of the goals outlined in the National Agenda, we found the Agency has taken several steps towards meeting this Agenda. For example, ORD coordinated with OCHP as well as with other program offices to develop "The Strategy for Research on Environmental Risks to Children." Coordination Improvements Needed OCHP has taken actions to coordinate its efforts with EPA program offices and regions, but has not been consistent with program office coordination efforts. Because children's environmental health efforts were not coordinated, there was inadequate assurance that limited resources were being used to the maximum extent possible to protect children from risk. Coordination Efforts with Program Offices Need Improvement OCHP has not been effective in coordinating and/or communicating with EPA's media program offices. OCHP officials recognized that their role differs among 8 ------- program offices and their relationship with some program offices has not been as strong or consistent as with others. We also found that OCHP does not conduct standard annual meetings or monthly conference calls with EPA's media program offices like it does with the regions. Further, unlike the regions, the program offices do not have designated children's health coordinators. We also found that OCHP is not a part of the program office's annual or strategic planning process, unless the program office requests their participation, and OCHP officials indicated that usually does not occur. The program offices had varying perceptions of the role and impact of OCHP. OCHP officials indicated they had limited resources for labor intensive projects. According to the organizational proposal creating OCHP, this office was to chair an EPA Board on Children's Environmental Health. The Board's primary charge was to ensure the integration of Agency activities effecting children and to serve as a catalyst for actions that will be protective of children. Among its specific responsibilities, the Board was to consider developing a comprehensive strategy and action plan with milestones for Agency-wide activities related to protecting children's health. This Board was to include the Deputy Office Director level and higher, including the Assistant Administrators from ORD and Office of Prevention, Pesticides and Toxic Substances. We spoke with OCHP officials about the status of this board and they advised us that the Board has not been active since 2000. Coordination Efforts with Regional Offices and ORD Beneficial In contrast to the communication efforts with the Program Offices, we found that OCHP communicates with the Agency's regions through the use of Regional Children's Health Coordinators. These coordinators hold annual meetings and conduct monthly conference calls with OCHP. These interactions provide the opportunity for the regions and OCHP officials to discuss on-going relevant issues and network amongst themselves, and facilitates coordination efforts and information sharing. While it varied among regions, we found OCHP had input in regional planning efforts related to children's health activities. The Regional Coordinators said that OCHP provides key areas of emphasis during their annual meetings to be incorporated within the regions' annual planning and strategic processes. They also indicated that OCHP serves as a focal point at Headquarters for children's issues, and provides important information and funding. In contrast to OCHP's efforts with the media program offices, we found that ORD has a process to coordinate with OCHP and program offices in planning EPA's children's health research through the use of Research Coordination Teams for each media office. These teams, which meet throughout the year, allow each program office to be involved in the planning for research in their media area. 9 ------- Lack of A Permanent Director OCHP has not had a permanent Director since April 2002 - over 2 years. Recognizing that OCHP cannot implement the National Agenda without collaborating and partnering with stakeholders, the lack of a permanent director could have a negative impact on such efforts. Various EPA regional coordinators, the former OCHP Director, the Chair of the Children's Health Protection Advisory Committee, and the Director of the Children's Environmental Health Network (a national children's environmental health advocate) expressed concern that the lack of a permanent Director may have a negative impact on the longevity and importance of the children's environmental health program within EPA. Specifically, the Director of the Children's Environmental Health Network said that it is difficult to view EPA as having a unified children's health program without the presence of a permanent Director. Additionally, the Children's Health Protection Advisory Committee, in a memo to the Administrator dated December 16, 2002, wrote that OCHP could not continue to play a key role within EPA, and across the nation without permanent leadership. It is imperative that all internal and external stakeholders know that children's health protection is a priority within EPA. The Administrator responded to the Children's Health Protection Advisory Committee on May 2, 2003, that selecting a permanent director for OCHP was a high priority for EPA. OCHP officials indicated an announcement for the Director position was closed in May 2003, but as of May 2004 a final decision had still not been made on a permanent OCHP Director. OCHP Named EPA Lead for Aging Initiative The United States is undergoing a demographic transformation with respect to the elderly. By 2030, the U.S. population over the age of 65 is expected to double to an estimated 70 million. As people age they become more susceptible to environmental hazards, such as: microorganisms in drinking water; second-hand smoke and carbon monoxide in indoor air pollutants; ozone and particulate matter in ambient air; volatile organic compounds; radon; temperature extremes; and such neurotoxins as lead and mercury. Additionally, as people age, they have accumulated a lifetime of exposures, and their immune systems diminish. In October 2002, the EPA Administrator announced the EPA Aging Initiative, making protecting the health of older persons a new priority of the Agency. The goal of the Initiative is that every citizen, community, organization, corporation, and particularly the Federal government, including EPA, is encouraged to understand the link between the environment and the health of older Americans; and take appropriate action to identify and reduce environmental threats to protect the health of older Americans. 10 ------- OCHP was named the Aging Initiative's lead and revised its mission statement, accordingly. Children and the elderly are both considered susceptible populations. To ensure adequate coverage under this new area, OCHP reorganized, adding a fourth team ("Aging Initiative Team") and hired one additional full-time staff member. OCHP officials said they received $300,000 from the Agency in their 2003 operating plan/budget to fund the Aging Initiative. However, to ensure adequate coverage within this new area, OCHP estimated that it spent an additional $300,000 from its children's environmental health annual budget toward the Aging Initiative. Further, these estimates did not take into consideration the time spent by OCHP officials other than the one hired for the Aging Initiative. A May 2003 report issued by the Children's Environmental Health Network, a national children's environmental health advocate, raised concerns about OCHP's mission being "severely compromised" due to such decisions as placing the Aging Initiative within OCHP without sufficient additional resources being provided. The report, "Are Children Left Behind?: Children's Environmental Health Under the Bush Administration," noted that by doing this EPA was doing a disservice to both the children and the Aging Initiative. Additionally, the Children's Health Protection Advisory Committee2 raised concerns that expanding the mission of OCHP to include the aging would diminish OCHP's effectiveness with children's environmental health. The decision to make OCHP the lead of the Aging Initiative would greatly increase OCHP's workload and areas of responsibility. Therefore, EPA needs to carefully consider whether making the same office responsible for protecting the environmental health of both the children and the aging is appropriate and, if so, provide it with additional resources to adequately protect both populations. Also, we noted that the term "aging" had not clearly been defined by OCHP. According to OCHP, different Federal agencies have different age parameters in defining "aging." An unclear definition could cause OCHP problems in implementing programs. 2The Children's Health Protection Advisory Committee is a body of researchers, academicians, health care providers, environmentalists, children's advocates, professionals, government employees, and members of the public who advise EPA on regulations, research, and communication issues relevant to children. 11 ------- Recommendations We recommend that the EPA Acting Deputy Administrator: 2-1. Develop a coordinated strategy that ties specific measures, milestones, and targets toward meeting the Agency's National Agenda to Protect Children's Health from Environmental Threats. 2-2. Reevaluate the role of OCHP and redefine its function and areas of responsibility within the Agency. 2-3. Establish an official children's health contact within each program office. 2-4. Expedite the decision making process in appointing a Director for the OCHP. 2-5. Ensure that OCHP is given resources to effectively implement the Aging Initiative. 2-6. Clearly define the parameters for "Aging" as part of the Aging Initiative. Agency Response and OIG Evaluation The Acting Deputy Administrator concurred with all recommendations and positions except Recommendation 2.6, which related to the aging parameters under the Aging Initiative. According to the Acting Deputy Administrator, it is not beneficial to define parameters for aging. The response further added that it is important to recognize that aging is a process that is not tied to a specific age. We recognize that while aging is a process, we believe it is important for the lead of the Aging Initiative to establish a clear and concise definition of what constitutes aging under the Aging Initiative to ensure consistency within EPA program offices. The Older Americans Act of 19653, for example, defines "older individuals" as individuals who are 60 years of age or older. Recognizing that the National Agenda for the Aging will consist of three parts, including identifying research gaps in environmental health and encouraging older adults to volunteer to address similar concerns, it would be prudent for EPA to clearly define the target audience. In response to our other recommendations, the Agency agreed to explore other options for improving coordination within the Agency and ways to better measure their progress and results. The Agency cited as an example a plan to work with the Office of the Chief Financial Officer to explore options to improve the way 3 Older Americans Act of 1965, Public Law 89-73, was enacted to provide assistance in the development of new or improved programs to help older persons. 12 ------- children's health is coordinated and measured in the Agency's strategic and annual budget planning process. Additionally, the Agency agreed to reiterate the role of OCHP to EPA senior managers to ensure OCHP can be an effective participant with EPA's programs that affect children. Although the Agency concurred with most of the recommendations and positions of the report, the Agency expressed a concern that the OIG did not recognize the role of the OCHP and the accomplishments that OCHP and EPA have made to protect the Nation's children. They recognized that there is always room for improvement, but they believe the focus of the OIG report did not recognize the substantial and significant progress that the EPA and OCHP - working with internal and external collaborators - have made in a relatively short period of time. According to the response, the Agency has made significant progress in developing science related to children's health and regulatory policy, as well as developing tools and information for communicating children's health issues. In addition, the response added that the Agency has taken significant actions in recent years to protect children's health from pollution in the air, water and land. A snapshot of some of the examples of EPA's accomplishments were included in the response and a more detailed summary of accomplishments was provided as an attachment. As cited in the body of our report, we recognize that many of EPA's program and regional offices have projects that focused on children's environmental health protection. However, the role OCHP had in developing and implementing these programs is not clear. OCHP identified several EPA accomplishments in their response memorandum, but support was not provided to illustrate the role of OCHP in these projects. OCHP also provided an attachment in their response that related to overall EPA accomplishments and, again, it is not clear as to the role and impact of OCHP on these projects. We recognize that OCHP has taken actions in several areas related to children's health protection, but OCHP needs to focus on results rather than activity. The Acting Deputy Administrator expressed concern regarding the title of the report. Consequently, we modified to title to more clearly articulate the findings in the report. Clarification was also provided for a date referenced in the report pertaining to the lack of a permanent director. This date was modified based on information provided by OCHP. 13 ------- 14 ------- Chapter 3 Strengths, Weaknesses in OCHP Implementation Efforts Noted We found that OCHP has no formal mechanism in place to ensure performance results or assess relationships between program costs, activities, and results. OCHP has a strategic planning process that includes an annual office planning meeting followed by a mid-year meeting to assess the status on the current year's plan and to begin consideration of potential focus areas for the following year. However, despite these meetings, there is no formal tracking to ensure that goals are met. More specifically, data and information systems are not available to measure, analyze, and demonstrate overall performance specific to the National Agenda on a continuing basis. Improved Focus Needed To evaluate OCHP's efforts in integrating children's health protection throughout the Agency's operations, we used the OIG's "Assessing Organizational Systems: A User's Guide," issued in November 2002. This tool was designed to assess organizational "Systems" by providing managers with seven key areas that establish the foundation for an organization or program to be successfully implemented. We assessed OCHP's planning process and operations in implementing children's health protection against these key areas. While OCHP has taken actions in different areas related to children's health protection, OCHP needs to focus on results. OCHP should have a methodology in place to set priorities to ensure resources are being allocated to those problems that pose the greatest environmental risks to children, and needs to efficiently measure those results to ensure sufficient strides are being made in improving children's environmental health. OCHP's broad overall mission, limited resources, and unclear role contributed to the shortcomings found. We believe an OCHP annual internal review of results or outcomes versus planned activities would be beneficial. This should become an annual assessment or review of OCHP's work. The results should be incorporated into OCHP's annual strategic planning meeting, as well as in its mid-year meeting. This review should verify performance results and analyze more accurately the relationship among costs, activities, and results. The results of our evaluation for each of the seven key areas are in Table 3.1. 15 ------- Table 3.1: Seven Vital Areas Seven Vital Areas Strengths Identified Inadequacies Strategic Planning OCHP developed an action plan No clear linkage between resources and planned activities The action plan does not describe how resources, activities, and outputs are linked to achieve the objectives and environmental results Leadership OCHP has defined a vision and mission statement Leadership has not ensured understanding throughout the Agency of the vision and mission of OCHP OCHP has not had a permanent Director since April 2002 Customer/ Stakeholder and Market Focus Customer/stakeholders and partners have been identified Monthly meetings are held with Regional Children's Health Coordinators OCHP collaborates with partners on various projects OCHP held public listening sessions on the Aging Initiative There are no formal contacts within EPA program offices Information and Analysis OCHP produces episodic summary products that illustrate Agency activities in children's health OCHP developed a book on trend analysis Data and information systems are not available to measure, analyze, and demonstrate overall performance specific to National Agenda on a continuing basis OCHP's objectives and goals are not specific and clearly measurable Human Capital Staff is divided into speciality area teams Recent Aging Initiative was added to OCHP's responsibilities, which adds additional duties to existing staff Process Management OCHP maintains a website There is no documentation or explanation of deviation from action plan There is no formal followup to ensure completeness of projects Performance Results Mid-year planning meetings are conducted to discuss projects' status OCHP does not evaluate products to ensure usefulness 16 ------- Recommendations We recommend that the Office of Children's Health Protection: 3-1. Make improvements to its annual planning process to include, at a minimum: a. more complete and measurable details on how planned activities are projected to be completed, identifying quantifiable attainable goals and targets; b. a plan of what potential resources, either internally within OCHP or externally within EPA or other stakeholders, are to be used; c. projected allocations of these resources and what the end products will be; and d. a methodology to set priorities to ensure resources are being allocated to those problems that pose the greatest environmental risks to children, recognizing that OCHP has limited resources. 3-2. Perform an annual internal review of results or outcomes versus planned activities. 3-3. Conduct periodic meetings with program office officials to discuss upcoming children's health projects, coordination efforts, and future areas of focus. Agency Response and OIG Evaluation OCHP agreed with our recommendations and positions outlined in this chapter. In response to our recommendations, OCHP is planning to develop quantifiable measures and develop long term objectives and strategic targets. Additionally, in future years, they plan to incorporate a review and analysis of the strategic targets and projects to determine if the results and outcomes identified were met. 17 ------- 18 ------- Appendix A Auditee's Response MEMORANDUM SUBJECT: Draft Evaluation Report: Effectiveness of the Office of Children's Health Protection Cannot Be Determined TO: Kwai Chan Assistant Inspector General for Program Evaluation We welcome the OIG report and concur with some of the observations and recommendations. However, we are concerned that the OIG did not appear to recognize the role of the Office of Children's Health Protection (OCHP) and the enormous accomplishments that OCHP and EPA have made to protect the Nation's children. We appreciate and highly value having an independent, unbiased review and evaluation of our efforts in this area. Protecting children from environmental hazards remains a very high priority for EPA. We also continue to view the role of the Office of Children's Health Protection (OCHP) as critical to the Agency's perseverance in moving forward on children's environmental health issues. We believe that it is very important to understand the role of OCHP within the Agency. OCHP was established to raise awareness and facilitate change in the way EPA does business so that protecting children's health becomes an integral part of all of the Agency's work from research to regulatory activities. While OCHP does not conduct research or set standards, they do work with Agency offices in those areas to insure that risks to children are considered. They are also responsible for working outside the Agency to increase efforts focused on children's environmental health (i.e., with other federal agencies, States, healthcare providers, and the private sector). We recognize the difficulty in assessing the full scope of work for any program being evaluated. As you point out in the report, implementing the National Agenda to Protect Children From Environmental Health Threats is the responsibility of the Agency as whole, not just OCHP. While we recognize that there is always room for improvement, we believe that the focus of the IG report does not recognize the substantial and significant progress that the EPA and OCHP - working with internal and external collaborators - have made in a relatively short period of time. We do agree with the OIG that we do not yet have the tools to quantitatively evaluate environmental health results of the program, a finding not inconsistent with many of our environmental programs as we all seek improvement in our ability to measure environmental and public health outcomes. We are committed to doing what is needed to measure the results of all of EPA's programs. However, we think that the report does not adequately recognize the qualitative advances made by the Agency on children's health and presents an unbalanced picture of the overall value of the program. The Agency has made significant progress in developing science related to children's health and regulatory policy, as well as developing tools and information for communicating children's health issues. The Agency has taken significant actions in recent years to protect children's health from pollution in the air, water and land. A snapshot of some of the examples of EPA's accomplishments include: the phaseout of chromated copper arsenate (CCA) which will significantly lower potential risks to children; establishment of Centers of Excellence in Children's Environmental Health Research dedicated solely to the study of 19 ------- children's environmental health; cancellation or use restriction of pesticides affecting children's health including methyl parathion, azinphos methyl, chlorpyrifos, and diazinon; the first-ever collaboration of State environmental and health officials to reduce childhood asthma; the National Children's Study, a longitudinal cohort study to investigate the relationship of the environment to children's health; the diesel rule which will reduce emissions of NOx by 2.6 million tons each year when the program is fully implemented; the Toxic Release Inventory (TRI) lead standard which requires more reporting of environmental releases of lead; the off-road diesel rule that will reduce emissions by more than 90 percent; the Clean School Bus U.S.A. initiative which will reduce both children's exposure to diesel exhaust and the amount of air pollution created by diesel school buses; and the Voluntary Children's Chemical Evaluation Program which will evaluate 22 chemicals for potential risks to children. In addition, efforts with the States, healthcare providers, and the international community have resulted in significant actions on children's environmental health where there was virtually no activity prior to EPA's involvement. A more detailed summary of accomplishments is in attachment 2. We hope that you will consider recognizing these accomplishments in your final report to present a more balanced evaluation of the program. Further, we believe that the current title of the report does not accurately reflect the actual findings of the report. We believe that" Improvements Are Needed in Efforts to Measure the Effectiveness of EPA's Efforts to Protect Children" or "Effectiveness of the Office of Children's Health Protection Cannot Yet Be Quantitatively Determined" would better describe the reports findings. Please refer to attachment 1 for a detailed response to the recommendations. In addition to the summary of OCHP accomplishments mentioned above (attachment 2), a September 2003 report "Evaluation of EPA Activities Related to Children's Health Protection" is also attached for your consideration in finalizing the report. Stephen L. Johnson Acting Deputy Administrator Nikki Tinsley Jeffrey Harris Jerri Dorsey Laurie Adams Tom Gibson Jeff Holmstead Paul Gilman Marianne Horinko Benjamin Grumbles Dona DeLeon Natalie Gochnour Rich McKeown William Sanders Joanne Rodman Elizabeth Blackburn c: Attachments 20 ------- OCHP's Response to the IG Report: Effectiveness of the Office of Children's Health Protection Cannot Be Determined General Comments and Corrections: We believe that the report does not adequately credit the great strides that the Agency has made in the area of Children's Environmental Health Protection. For example, prior to the efforts of OCHP and other EPA programs there was little or no activity on children's environmental health in the States or in the health care provider community. Now we have significant programs in both areas. The same is true in the area of science and economics. We refer you to the September 2003 report "Evaluation of EPA Activities Related to Children's Health Protection"(attached), which evaluated EPA's progress in implementing the EPA National Agenda to Protect Children From Environmental Health Threats. We have also attached an appendix to this response which highlights the children's environmental health (CEH) accomplishments over the last several years. We hope that the OIG will be able to use these documents to include examples of the significant progress that has been made in the final report. We believe that the current title of the report does not accurately reflect the actual findings of the report. We believe that "Improvements Are Needed in Efforts to Measure the Effectiveness of EPA's Efforts to Protect Children" or "Effectiveness of the Office of Children's Health Protection Cannot Yet Be Quantitatively Determined" would better describe the reports findings. The report states that the office has been without a permanent director since April 2001- nearly 3 years. The correct date is April 2002 - nearly 2 years. Response to Recommendations for the EPA Acting Deputy Administrator: 2-1. Develop a coordinated strategy that ties specific measures, milestones, and targets toward meeting the Agency's National Agenda to Protect Children's Health from Environmental Threats. We agree that it is important that EPA efforts focus on children's environmental health be coordinated and that we need to do a better job measuring results in this area. An attempt to do this is reflected in the September 2003 report "Evaluation of EPA Activities Related to Children's Health Protection". We recognize that, while useful, that report fell short of quantitatively measuring health outcomes, but it did look at progress made related to the National Agenda. OCHP and the Deputy Administrator will explore other options for improving coordination in the Agency and ways to better measure our progress and results. For example we will work with the Office of the Chief Financial Officer to explore options to improve the way children's health is coordinated and measured in the Agency's strategic and annual budget planning process. 2-2. Reevaluate the role of OCHP and redefine its function and areas of responsibility within the Agency We agree with this recommendation, with modification. While OCHP's mandate is to facilitate the implementation of the National Agenda to Protect Children's Health from Environmental Threats, it was never intended that the office would have responsibility to implement most areas in the National Agenda. For example, although OCHP is an active voice for 21 ------- children's health in the Agency standards and regulation development process, the office does not develop standards. Developing standards is the responsibility of the program offices. Protecting children from environmental hazards remains a very high priority for EPA and the role of OCHP is critical to the Agency's continuing progress. OCHP was established to raise awareness and facilitate change both inside and outside EPA. With support from the Agency's leadership, they have been tasked with raising awareness and facilitating change in the way EPA does business so that protecting children's health becomes an integral part of all of the Agency's work from research to regulatory activities. While OCHP does not conduct research or set standards, they do work with Agency offices in those areas to insure that risks to children are considered. They are also responsible for working outside the Agency to increase efforts focused on children's environmental health (i.e., with other federal agencies, States, healthcare providers, and the private sector). EPA leadership will reiterate the role of OCHP to EPA senior managers so that OCHP can continue to be an effective participant in all of the Agency's programs that affect children. 2-3: Establish an official children's health contact within each program office. We agree with this recommendation. We believe it is very important to have a management-designated CEH contact in each program office to facilitate our collaborative efforts implementing the National Agenda and other issues relating to CEH. Many programs have designated CEH contacts already, and OCHP currently works closely with these contacts throughout the Agency. OCHP has been very effective at leveraging resources within the Agency to ensure that the directives of the National Agenda are met and has drawn upon ongoing efforts throughout EPA to develop collaborative relationships with regional and program offices. OCHP and the DA are exploring ways to assure that the current CEH contacts represent their offices and ways to increase management involvement in CEH throughout the Agency. 2-4: Expedite the decision making process in appointing a Director for OCHP. We agree with this recommendation. The Agency is in the process of recruiting nationally for a permanent director. In the interim an SES level manager, the Deputy Assistant Administrator for Prevention, Pesticides and Toxic Substances, has been appointed as Acting Director for OCHP. He has extensive experience with children's issues both in Region 5, where he dealt with the impact of lead poisoning on minority children in the Midwest, and in Headquarters where he dealt with implementation of the lead program, including development of regulations to put into place a protective infrastructure for kids. 2-5: Provide a formal decision on what office should be given the lead for implementing the Aging Initiative, and ensure that whichever office is designated is given resources to effectively implement the program. We agree with this recommendation with modification. The Agency has made the decision to have OCHP take the lead for the Aging Initiative. Recognizing OCHP's experience and success in working within EPA as well as with outside stakeholders on issues relating to susceptible populations, we believe that they are the appropriate office to manage the initiative. We also agree that, within budget constraints, appropriate resources are needed to implement the initiative. It is important to acknowledge that the Aging Initiative is still at the early vetting stage. More discussion 22 ------- and deliberation is required with outside stakeholders and with EPA programs to determine whether or not additional work in this area- beyond existing efforts in science, outreach and communications- might be a beneficial and productive use of agency resources. 2-6: Clearly define the parameters for "aging" as part of the Aging Initiative. We disagree with this recommendation. It is not beneficial to define parameters for aging. It is important to recognize that aging is a process that is not tied to a specific age. EPA's Aging Initiative was established to protect the environmental health of older persons, independent of specific chronological age. For example, outreach and educational efforts are targeted to older people with chronic conditions that can be exacerbated by the environment. This approach is consistent with professionals working in this field. The aging population in the United States is a heterogeneous group reflecting a broad diversity in health status, behavioral patterns, social and cultural influences and environmental conditions.[U.S. Census, 65+ in the United States (1996)]. As we age, we may become more susceptible to environmental hazards. Aging is a natural progressive decline in organ function and compensatory reserves, accompanied by a reduced capacity to detoxify and eliminate toxicants. Health effects resulting from a lifetime of environmental or occupational exposures to persistent agents may also manifest as one grows older [National Academy of Sciences Workshop on the "Differential Susceptibility of Older Persons to Environmental Hazards," December 2002]. Understanding the biology underlying differing age-related responses can inform a scientific rationale for decisions on how to appropriately incorporate the differential sensitivity of aging adults into environmental risk assessment, decisions and actions. Response to Recommendations to the Office of Children's Health Protection: 3-1: Make improvements to its annual planning process to include, at a minimum: 1.1 More complete and measurable details on how planned activities are projected to be completed. identifvin2 quantifiable attainable goals and targets: 1.2 A plan of what potential resources, either internally within OCHP or externally within EPA or other stakeholders, are to be used: 1.3 Projected allocations of these resources and what the end products will be: and 1.4 A methodology to set priorities to ensure resources are being allocated to those problems that pose the greatest environmental risks to children, recognizing that OCHP has limited resources. We agree with this recommendation. OCHP is committed to continuing to make improvements to ensure that the directives of the national Agenda are met. As pointed out in the IG report, OCHP has a vision, mission and goals to guide the office's decision-making. OCHP supports and facilitates Agency efforts in three primary areas: 1) regulations and standards, 2) science and risk assessment, and 3) public awareness, community-based programs, and education. In addition, OCHP develops an action plan annually based on the goals and 23 ------- identifying areas where there appear to be gaps and needs. These annual decisions are revisited several times throughout the year. In September of FY 2003, OCHP began to reevaluate its goals and to develop long-term objectives, sub-objectives and strategic targets to assure that we can move forward in achieving our goals. OCHP has set aside funds in the office's FY 2004 budget and has acquired a knowledgeable detailee for six months that will spend full time managing a project to establish quantifiable measures for the office. The results of both activities above will be used to inform decisions and set priorities at OCHP's annual planning meeting. 3-2: Perform an annual internal review of results or outcomes versus planned activities. We agree with this recommendation In future years OCHP's annual planning process will include a review and analysis of the strategic targets and projects to determine if the results and outcomes identified have been met, and if not, why not, lessons learned, and whether additional efforts are required to meet them. All proposed projects will include clearly defined expected results and the methodology that will be used to measure the results. 3-3: Conduct periodic meetings with the program office officials to discuss upcoming children's health projects, coordination efforts, and future areas of focus. We agree with this recommendation. OCHP intends, with the Deputy Administrator's help, to make the relationship with the program offices more "official" at a management level. However, as mentioned under recommendation 2-3, OCHP has consistently worked closely with dedicated CEH contacts in all of the Agency program offices and regions, and could not have the same level of success without their assistance. 24 ------- Appendix B Offices Visited and Contacted EPA Office of Air and Radiation: Indoor Environments Division Office of Prevention, Pesticides, and Toxic Substances: Office of Science Coordination and Policy Office of Pollution Prevention and Toxics Office of Pesticide Programs Office of Solid Waste and Emergency Response Office of Water Office of Children's Health Protection Office of the Chief Financial Officer: Office of Planning and Budget Analysis Office of Research and Development Regional Children's Environmental Health Coordinators (Regions 1-10) Other Federal Agencies Agency for Toxic Substances and Disease Registry National Institute of Environmental Health Sciences Outside Stakeholders Children's Environmental Health Network National Resources Defense Council 25 ------- 26 ------- Appendix C Distribution Acting Deputy Administrator Acting Director, Office of Children's Health Protection Agency Followup Official (the CFO) (271 OA) Agency Audit Followup Coordinator (2724A) Audit Followup Coordinator, Office of Children's Health Protection Acting Associate Administrator, Office of Public Affairs (1101 A) Acting Associate Administrator for Congressional and Intergovernmental Relations (1301 A) Acting Director, Office of Regional Operations Assistant Administrator, Office of Research and Development Assistant Administrator, Office of Air and Radiation Assistant Administrator, Office of Prevention, Pesticides, and Toxic Substances Assistant Administrator, Office of Solid Waste and Emergency Response Assistant Administrator, Office of Water Audit Liaison, Office of the Administrator Audit Followup Coordinator, Office of Air and Radiation Audit Followup Coordinator, Office of Prevention, Pesticides, and Toxic Substances Audit Followup Coordinator, Office of Solid Waste and Emergency Response Audit Followup Coordinator, Office of Water Audit Followup Coordinator, Office of Research and Development Inspector General (2410) 27 ------- |