oEPA [ntemational .- Environmental I'ii.i[c.vni'>r, A" cue-. Best Practices For EPA's International Capacity-Building Programs Report of EPA Task Force ------- TABLE OF CONTENTS EXECUTIVE SUMMARY Page 01 INTRODUCTION Page 05 UNDERLYING PHILOSOPHY Page 07 ACHIEVING EPA'S MISSION Reducing Global and Transboundary Environmental Risks Page 09 Achieving Environmental Results Worldwide Page 10 Strengthening Environmental Protection in the U.S Page 13 Serving Broader National Objectives Page 14 BEST PRACTICES FOR ENVIRONMENTAL RESULTS Factors for Success Page 15 Project Guidelines Page 19 MEASURING RESULTS UNDER GPRA Establishing Performance Goals and Measures Page 23 Establishing Baseline Conditions Page 28 Other Measurement Challenges Page 30 FUTURE MANAGEMENT CHALLENGES Page 33 APPENDICES Task Force Members Page 35 Overview of Existing Capacity-Building Programs Page 37 Acknowledgments Page 53 Reference Materials Page 55 List of Acronyms Page 59 ENDNOTES Page 61 ------- EXECUTIVE SUMMARY EPA's international capacity-building programs play a critical role in achieving the Agency's mission. Weak managerial, technical, financial, scientific, and/or institutional capabilities have often served as the major stumbling blocks to developing-country action on behalf of the environment, including progress in addressing global and transboundary environmental problems that directly affect health and the environment in the United States. Building on its reputation as one of the world's most effective environmental organizations, and using expertise and experience developed over the last thirty years, EPA works with foreign partners to achieve the capabilities needed to address environmental challenges on a long-term and sustainable basis. Programs already underway at the Agency are helping to reduce global environmental risks; strengthen environmental protection efforts worldwide; lower the cost of environmental protection in the United States; and serve broader foreign policy, economic and other national objectives. In 1999, EPA's Office of International Activities convened a cross-Agency group of experts to develop a framework for improving the way the Agency designs, implements, and measures the effectiveness of these programs, consistent with the Government Performance and Results Act (GPRA) of 1992. Based on a comprehensive review of existing and proposed programs, the task force drafted a set of best-practice guidelines related to the design and implementation of international capacity-building programs. It developed a methodology, consistent with GPRA, for measuring and evaluating the effectiveness of these programs. Finally, the task force identified a set of issues for future consideration by senior management. After a discussion of underlying philosophy, the task force emphasized the importance of five "factors for success" in considering whether the Agency should undertake a particular program. Recognizing that no program or project would be able to satisfy every criterion, the task force laid out these factors for reference by EPA managers and staff in program conceptualization and implementation. All five factors apply to both EPA and the partner country. (1) Political support, based on a clear understanding of mutual interests, to assure long- term dedication of the necessary staff and resources; (2) The existence (or development, if necessary) of the underlying legal, policy, technical, and administrative frameworks to serve as the foundation for sustained and effective results; (3) The mobilization or leveraging of adequate financial resources to support EPA participation as well as concrete programs in the partner country; (4) The dedication of appropriate management and technical personnel to assure proper selection, design, implementation, and evaluation of projects; and -1- ------- (5) Public participation and the involvement of all relevant stakeholders to help inform and build support for program objectives. The task force also laid out a set of best-practice guidelines related to each stage of program implementation: project selection, design, implementation, monitoring and evaluation. The task force consulted work already carried out by the United States Agency for International Development (USAID), the World Bank, and the Organization for Economic Cooperation and Development (OECD) on environmental indicators and other measurement techniques. It developed a hierarchy, consistent with the broader EPA measurement methodology under GPRA, for better establishing longer-term environmental goals against which progress can be measured. At the top tier of the hierarchy (see below) is the long-term environmental result: healthy GOAL HIERARCHY FOR ERA'S INTERNATIONAL CAPACITY-BUILDING PROGRAMS 1. Healthy People and Ecosystems 2. Ambiert Air, Water and Sofl Quality 3. Pollution Prevention, Control and Remediation *. Laws and Policies, Institutions and Public/Private Funding people and ecosystems. This should be the ultimate objective for any EPA program or project regardless of the initial impetus for EPA engagement (e.g., Congressional or Administration mandate). The lower three tiers all support this goal. The second level reflects the state of ambient air, water, and soil quality upon which healthy people and ecosystems depend. This would imply, for example, air and water quality that would meet established international standards for breathing, drinking, etc. At the third level of the hierarchy—directly contributing to the level above—is action by the foreign government to prevent pollution in the first place, to control the pollution that cannot be avoided, and to remediate pollution problems which already exist. -2- ------- At the bottom level of the hierarchy is action by the foreign government to establish the necessary underlying conditions that will support the environmental outcomes at the higher levels. This includes the establishment of effective laws and policies, the strengthening of institutions at the national, municipal, and local levels, and the mobilization of funding for environmentally beneficial purposes from both the public and private sector. Absent these conditions, environmental outcomes will not be sustainable, even if achievable in the short term. EPA's international capacity-building programs are largely directed at the third and fourth levels of the hierarchy. This focus is premised on the assumption that work at this level is the most effective way to achieve long-term and sustainable environmental results. The level at which the work is directed has significant implications for the way results can and should be measured. The task force laid out two sets of sample performance goals and measures for use by EPA managers and staff in gauging progress under GPRA. One set, directed largely at the third level of the hierarchy, includes environmental indicators related to air, water, waste, and natural resources. The second set, targeted at the bottom level of the hierarchy, provides sample programmatic indicators related to laws, policies, institutions, human resources, information, financing, and public participation. The task force affirmed the need to establish baseline conditions - or reference values - against which to measure the effectiveness of programs. This capability will enable EPA and its partner organizations to set priorities, gauge progress, and assure mid-term corrections. The task force described many of the challenges facing the Agency in this area, including the lack of data on existing environmental conditions in many developing countries and the problems in ensuring accuracy, uniformity and timeliness. The task force provided a number of recommendations for overcoming challenges related to baseline conditions. It also described challenges related to quantifying environmental outcomes, establishing linear relationships, and other measurement issues. The task force reaffirmed the importance of international capacity building to EPA's overall mission. It concluded with a brief discussion of future management challenges for future consideration by senior management, including the need to: (1) improve intra-agency coordination, (2) promote the link between capacity-building programs and broader Agency policies and programs internationally, and (3) improve the way EPA works with USAID, the World Bank and other international financial and development organizations. The appendices provide more detailed information on the Agency's existing and proposed international capacity-building programs. -3- ------- -4- ------- INTRODUCTION In a December 1998 meeting with the EPA Team on Reducing Global and Cross-Border Environmental Risk, then-Acting Deputy Administrator Peter Robertson emphasized the importance of strengthening the Agency's international capacity-building programs. To that end, Peter Robertson directed the Office of International Activities (OIA) to examine these programs across EPA and to develop a framework for improving the way the Agency designs, implements, and measures the effectiveness of these programs, consistent with the Government Performance and Results Act (GPRA) of 1992. Responding to the Acting Deputy Administrator's mandate, OIA convened a cross-Agency task force composed of staff with experience hi designing and implementing capacity-building programs. Chaired by Jamison Koehler and Joan Fidler, the task force included representation from 10 offices and 8 regions. A full list of participants is provided on page 36. The task force: (1) reviewed the rationale and scale of existing programs, (2) drafted a set of "best practices" for the design and implementation of international capacity-building programs, (3) developed a methodology for improving the way the Agency measures and evaluates the effectiveness of these programs consistent with the GPRA, and (4) laid out a set of future management challenges for senior management consideration. This report provides a summary of the task force's conclusions in each of these areas. It includes examples from ongoing programs to illustrate particular points contained in the report. -5- ------- -6- ------- UNDERLYING PHILOSOPHY The underlying philosophy ofEPA's international capacity-building programs is perhaps best summarized by the often-quoted proverb: If you give a man a fish, he will eat for a day. If you teach a man to fish, he will eat for a lifetime. It is based on this philosophy that the Agency focuses on establishing the legal, policy, institutional, financial, and scientific frameworks in the partner country from which meaningful and sustainable environmental results will follow. EPA often supplements such broad, institution-based work with shorter-term technical assistance, training or demonstration projects which not only achieve a specific environmental outcome, but which can also be adapted and replicated for environmental results elsewhere. Consistent with this philosophy is EPA's adherence to the tenet, as stated by the Chief Economist at the World Bank, that: "Foreign aid is as much about knowledge as it is about money."1 Although the Agency supplements its appropriated funds with additional support secured through cooperative agreements with the U.S. Agency for International Development (USAID), multilateral development banks and other development institutions, EPA's financial resources for international capacity-building programs are modest, representing about one-half of one percent of the Agency's overall budget for FY1999. The Agency's major contribution is the knowledge, expertise, creativity, and energy of its 18,000 employees, based on over 30 years of experience implementing a domestic environmental program in the United States. WORKING DEFINITION OF "CAPACITY BUILDING'' EPA's international capacity-building programs provide technical assistance, training, information exchange and other forms of cooperation to enhance the capabilities of governments and other stakeholders to protect human health and the -7- ------- -8- ------- ACHIEVING EPA'S MISSION International capacity-building programs play a critical role in helping EPA achieve its mission while helping partner countries address priority environmental challenges. Weak managerial, technical, financial, scientific and/or institutional capabilities have often served as stumbling blocks to developing country action to protect the environment, including progress in addressing the global and transboundary environmental problems that directly impact health and environment in the United States. Building on its reputation as one of the world's most effective environmental organizations, and using expertise and experience developed over the last thirty years, EPA works with its foreign partners in developing the capabilities needed to address these environmental challenges on a long-term and sustainable basis. As described in greater detail below, EPA's international capacity-building work to date already has achieved substantial results in four major areas: (1) reducing global and transboundary environmental threats that directly affect U.S. public health and the environment; (2) achieving environmental results worldwide; (3) strengthening environmental protection in the U.S.; and (4) promoting broader U.S. foreign policy, economic, humanitarian and national security interests. REDUCING GLOBAL ENVIRONMENTAL RISKS Pollution originating in other countries can have a direct impact on the United States. Transboundary air pollution, ozone depletion, the transport and deposition of persistent bioaccumulative toxics and other international environmental problems directly affect public health and the environment in this country. The Agency's international capacity-building programs have made significant progress in addressing such problems. For example, despite growing awareness of stratospheric ozone depletion in the late 1980s, only a handful of developing countries participated in the original negotiations of the Montreal Protocol on Substances that Deplete the Ozone Layer. This lack of developing country participation undermined the Protocol's initial effectiveness. Many developing countries, led by China and India, not only questioned their responsibility for helping to solve the problem, but their technical and -9- ------- financial ability to do so as well. EPA's efforts helped to reverse this situation. The Agency was an active participant in international assessment efforts which laid the basis for strengthening the terms of the Protocol through the London Amendments of 1990. A scientific assessment helped convince reluctant developing countries of the seriousness of the threat. Economic and technical assessments alleviated many concerns about the economic and technical viability of non ozone-depleting chemicals and processes. Similarly, our bilateral "EPA is the nation's chief technical and regulatory agency for environmental matters. As such, it plays a major role not only in domestic environmental protection activities but in international environmental programs and activities as well...EPA's international programs also serve important U.S. economic, foreign policy, and security interests." nfr. capacity-building work in preparing national General Accounting Office . .J . ° . * . .?..,.. emission inventories and m identifying alternative chemicals and processes helped encourage greater developing-country participation in the Protocol. Partly as a result of EPA's efforts, over 100 developing countries are implementing the provisions of the Protocol, worldwide production and consumption of ozone-depleting chemicals have been substantially curtailed, and the ozone layer has begun the first steps toward recovery. Capacity building will play an equally important role with respect to the U.N. Framework Convention on Climate Change and ongoing negotiations on an international convention on persistent organic pollutants. More details on ongoing and proposed programs in each of these areas are provided in Appendix B. ACHIEVING ENVIRONMENTAL RESULTS WORLDWIDE In addition to helping developing countries accede to multilateral environmental treaties, the EPA maintains bilateral capacity-building programs which help many of these countries address their pressing environmental problems. The paragraphs below provide highlights from these programs. Appendix B contains a more detailed description of each program. Mexico — Strengthening the environmental management capacity of organizations on both sides of the border has been a centerpiece of EPA's cooperative programs with Mexico. With work carried out under the La Paz Agreement in 1983 — and through implementing programs such as Border XXI - training, technology exchange, and increased access to information have resulted in significant environmental gains. The construction of wastewater treatment facilities in Mexico and the U.S. is helping solve decades-old environmental health problems. Joint air pollution efforts are helping reduce respiratory and other health-related problems in cities on both sides of the border. Coordinated enforcement efforts are reducing illegal waste dumping and other pollution problems. The two nations have expanded tracking oftransborder shipments ofhazardous and toxic substances, and have improved binational emergency response procedures. In addition to bilateral work, the U.S. and Mexico are also working with Canada to enhance capacity related to the sound management of chemicals, including DDT, chlordane, and other pesticides. -10- ------- Central America—EPA's programs in Central America, funded through US AID, are helping to improve food quality for fresh produce imported from Central America; reduce the inventory of obsolete pesticides stockpiled throughout the region; replicate pilot projects on municipal wastewater treatment and integrated solid waste management; introduce cleaner production practices by private firms; and establish regional networks of environmental lawyers, technical experts, and environmental engineers. ENVIRONMENTAL RESULTS IN THE CZECH REPUBLIC Funded in large part by USAID, EPA's capacity- building programs in the Czech Republic have helped: reduce cancer risk 5-fold for 330,000 residents in the industrial city of Ostrava through technical upgrades and closures of four polluting coke ovens; breathe new life into the rehabilitation of abandoned brownfield sites through the mobilization of $43 million in Czech funds for remediation and redevelopment in downtown Ostrava; reduce health threats from air pollution in Northern Bohemia through the commitment of $240 million in Czech funds to convert home heating from coal to natural gas. Central and Eastern Europe — The Agency's program in Central and Eastern Europe, also funded by USAID, enabled seven countries to develop and approve national plans to phase out leaded gasoline. EPA's work in Poland has improved water quality for 400,000 residents in the City of Krakow through the provision of state-of- the-art U.S. disinfection and analytical equipment. It has also reduced agricultural run-off in Poland via improved farm management practices, with the World Bank and European Union financing the replication of innovative waste management technologies in other areas. EPA's successes in the Czech Republic are described in the text box on the right. Russia/NIS- EPA's assistance to Russia in upgrading and expanding low-level radioactive waste at Murmansk is helping to eliminate the only known source of nuclear contamination in the Arctic. Agency efforts through US AID-funded programs in Russia and the New Independent States have resulted in: - a 25,000 ton reduction in annual CO2 emissions at the Mosenergo Elektrostal Power Plant near Moscow; - a 95% reduction hi nitrates, a 70% reduction in total suspended particles, and a 85% reduction in biological oxygen demand at the Gagarin wastewater treatment plant through improved operations and maintenance; - a 60% reduction in annual nitrogen oxide emissions at the Ladyzhin coal-fired power plant in the Ukraine; - a 7% reduction in stationary source particulate emissions in Volgograd; -11- ------- — a 75% reduction in bronchial asthma among children and improved drinking water for 10,000 people in Nizhnii Tagil through the installation of 65 drinking water filters at hospitals and schools; - provision of safer drinking water and hot water service for 125,000 residents of the Dzerzhinsk District; and - achievement of 500,000 ton reduction in air emissions from mine waste at the Chermshansk site; hi addition to these efforts, EPA's successful pilot project on improving municipal wastewater treatment plant performance in 3 cities near Moscow is being sustained through the establishment of a professional wastewater treatment plant training center in Smolensk. The center has helped disseminate best practice information to more than 700 treatment plant operators in the region. Asia - EPA interaction with China (along with the work of the World Bank and other partners) on public health, technical and policy issues surrounding the use of leaded gasoline resulted in a national ban on leaded gasoline. This ban should be fully implemented by early 2000. With private vehicle use in China becoming the largest contributor to air pollution in many cities of China, EPA is now working with the Chinese national and local governments on implementation of vehicle emissions standards and on inspection and maintenance programs. EPA expects to continue its capacity-building efforts with China on a range of sustainable transportation issues. EPA is also testing a number of innovative air pollution control technologies with Chinese coals. While the main objective of the project is to find sulfur removal technologies suitable for China, the technologies could be configured to reduce emissions of NOx, mercury and other toxic substances. EPA is also working with other Asian nations in managing the environmental degradation resulting from rapid urbanization and industrial expansion. Working primarily through US AID's U.S.-Asia Environmental Partnership (USAEP), EPA has focused on strengthening environmental regulatory systems and performance-based strategies, encouraging the use of market mechanisms, and promoting access to environmental information. Projects implemented since 1998 have ranged from technical assistance with haze mitigation and monitoring in Indonesia and Malaysia to environmental enforcement /compliance training in Thailand. In FY 2000, EPA will begin new projects in Asia which focus on toxics, urban air quality, waste management and global climate change. Capacity-building Tools - EPA has developed a set of generic tools for use in conducting capacity-building programs worldwide. Recognizing the similarities in many environmental challenges facing developing countries, these tools combine the Agency's best information and expertise into a single integrated package on the most commonly requested topics, thereby assuring quality control and avoiding duplication of effort. Information on EPA's work related to training, information, environmental education, and assistance in securing financing is provided in Appendix B. -12- ------- STRENGTHENING ENVIRONMENTAL PROTECTION IN THE U.S. EPA often can achieve the most cost-effective environmental results domestically through cooperation with another country. Many U.S. cities along the Mexico border- El Paso, for example - have had difficulty attaining ambient air quality standards under the Clean Air Act because of shared airsheds with Mexico. Work with Mexico hi identifying and reducing these emissions is far more cost-effective - and also fairer to the U.S. border communities - than undertaking more drastic measures in the U.S. Similarly, in the context of global issues, the international community has recognized that many of the most cost- effective opportunities for reducing carbon dioxide emissions — which impact the global atmosphere regardless of origin — can be found in developing countries. BENEFITS TO EPA STAFF EPA staff, and the Agency as a whole, benefit from working with partners outside of the U.S. International capacity-building programs expose staff to alternative approaches to environmental management and, in turn, help inject creativity and innovation into our domestic programs. To cite one example, private sector participation in drinking water and wastewater services is proceeding at a faster pace outside of the U.S. than inside; however, regulatory development which ensures that the public is well-served by privatization efforts tends to be more advanced in the U.S. EPA staff assist with regulatory capacity building on water privatization issues in Asia, Eastern Europe and Latin America and, in turn, have the opportunity to learn how multinational water companies are approaching privatization, lessons which can prove useful as the U.S. moves away from public management of utilities. EPA's international capacity-building programs have often introduced new or unproved environmental technologies or management techniques to the U.S. An innovative technology - first piloted through a demonstration project in Poland - using sewage sludge to stabilize and revegetate land contaminated by mining waste is now being applied at two contaminated sites hi the U.S. Similarly, experience with biological wastewater treatment methods in tropical countries can support the introduction of treatment methods that are less energy- and chemical-dependent in southern parts of the U.S. In another example, EPA used data from the demonstration of a natural gas reburn system at the Ladyzhin thermal power plant in Ukraine to set nitrogen oxide emissions limits on cyclone and wet-bottom utility boilers in the U.S. Because the Ukraine demonstration cost less than $500,000 (as compared to the $10 million cost of a similar demonstration at the Niles facility in Ohio), the demonstration made the data available to EPA at 95% less cost than a U.S.-based demonstration. -13- ------- SERVING BROADER NATIONAL OBJECTIVES As recognized by the General Accounting Office,2 the Agency's international capacity- building work also promotes broader U.S. economic, humanitarian, national security, and other foreign policy interests. With respect to U. S. economic interests, for example, the Environmental Technologies Trade Advisory Committee has emphasized the important role which EPA's international technology and capacity-building programs play in opening up markets for U.S. exporters of environmental technologies and services (see text box). Both Congress and the Administration have urged EPA to play a larger role in this area.3 "Widely recognized as the leading source of environmental regulatory and management expertise worldwide, EPA is frequently approached by foreign governments and organizations for assistance in establishing the regulatory and management capabilities that can drive the demand for U.S. technologies." Environmental Technologies Trade Advisory Committee Assistance to Mexico has made a positive contribution to a relationship often fraught with political disagreements on immigration, trade, and narcotics. Capacity-building programs have also helped bolster U.S. foreign policy initiatives with Russia, China, South Africa, Ukraine, Egypt, and other strategically important nations and regions. By encouraging nations with diverse political interests to work together in addressing shared environmental challenges, for example, the Agency's capacity-building programs can play an important role in U.S. diplomatic efforts in the Middle East. EPA and the Department of Energy are cooperating with Israelis, Palestinians and Jordanians to demonstrate how treated domestic waste water can be reused for irrigation in West Bank villages. If successful, this demonstration project will show that the application is viable for other West Bank communities as well as for similar communities in Israel and Jordan. In addition to the potential economic and environmental benefits of this effort, a success will serve to show that scientists, engineers and government officials from all three sides can work together to address common problems. -14- ------- BEST PRACTICES FOR ENVIRONMENTAL RESULTS The task force reviewed the Agency's ongoing international capacity-building programs, and discussed in detail which elements of the programs have worked and, perhaps more importantly, which have not. Based on the review, the task force developed a set of best practice guidelines for future international capacity-building programs Focusing on both EPA and host country considerations, the task force divided these guidelines into two sets: 1) factors to consider in determining the value of EPA participation in an overall program; and, 2) implementation guidelines at the project level (i.e., how to design and implement a project once political agreement on an overall program has been reached).* A third section deals with measuring the results of EPA's international capacity-building programs under the GPRA. FACTORS FOR SUCCESS The task force concluded that there are five factors for success that should be considered up front in determining whether or not the Agency should undertake a particular program. The task force noted that few programs will be able to satisfy each factor under this idealized framework. Instead, it proposed these factors as a checklist for EPA program managers to use during program design and implementation. Below is detailed information on each of the factors, illustrated through examples from existing programs. All five factors apply to both EPA and the partner country.4 Political Support High-level political support from both EPA and the host country is the first factor in assuring a successful capacity-building program. Effective and sustainable programs require the support of political leadership willing to devote the necessary staff and resources to the effort. Such support requires that the benefits of working together are immediately apparent to both sides. Critical on the U.S. side for major program development will be support from the Administrator, Deputy Administrator, and key Assistant and Regional Administrators and, more * He report addresses both programs and projects. Generally, international capacity-building programs establish broad goals which EPA and one or more partner countries will work toward. Projects are discrete activities (e.g., training courses, technical assistance) designed to achieve the goals set forth in a program. -15- ------- broadly, from Congress and the Administration. In some cases, the involvement of other federal agencies will be important for technical, programmatic, and/or financial reasons-. While environmental results are EPA's primary objective, the need for political support also gives weight to other considerations, such as the extent to which a program also promotes U.S. foreign policy, economic, humanitarian or national security interests. EPA's international capacity-building work has, for example, been an important component of the Gore Commissions with Russia, Ukraine, South Africa, and the Middle East. It also has helped to respond to Congressional interests in such areas as environmental technology exports, Central American hurricane relief, and democratization of Central and Eastern Europe and the former Soviet Union. On the partner country side, the programs will need the support of the political officials at all appropriate levels — national, state/provincial, municipal and local - who will be responsible for ultimate follow-through. EPA should monitor in-country political events closely to assure that activities are not initiated during times of significant political transition. EPA should also work with USAID, Embassy and other U.S. officials to understand the political context in countries where the Agency contemplates establishing programs (to determine, for example, that it is working with the right ministry or organization). MICROBIOLOGICALLY SAFE DRINKING WATER FOR CHILDREN'S HEALTH EPA's international initiative on microbiologically safe drinking water, focusing initially on Central America, provides a good example of the need for political support in both the U.S. and host country. Administrator Browner enlisted the interest of her G-8 counterparts in the protection of children's health (including a focus on microbiological contaminants in drinking water) as a high priority at the G-8 Environmental Summit in May 1997. Funding for the initiative was then included in the President's budget request and passed by Congress. Congress provided additional funding for the program through a FY 1999 supplemental appropriations bill to, among other things, assist Central America with hurricane relief. EPA's international partners have identified this issue as one of their top environmental, social and political priorities. El Salvador, for example, is now working to incorporate laboratory capacity and treatment plant improvement into its national sector plan. Legal, Policy, and Administrative Framework The second major factor to consider in ensuring effective and sustainable results is the existence (or development, if necessary) of the legal, policy, and administrative framework necessary to facilitate the achievement of environmental results. In the U.S., where the policy and administrative framework is already strong, the important consideration will be the extent to which EPA's statutory authority permits the Agency to undertake -16- ------- a given program. While most of EPA's operating program statutes provide for the transfer of technologies and management strategies to other countries on a media-specific basis, a specific statutory requirement or a high-level agreement of cooperation facilitates the effectiveness of international partnerships. The Support for Eastern European Democracy (SEED) Act of 1989, for example, explicitly mandated that EPA carry out specific programs in Poland and Hungary, resulting in the establishment of the Regional Environment Center in Hungary and the creation of air and water monitoring networks in Krakow, Poland. The explicit authority also put EPA in a stronger position to negotiate involvement in other countries of the region not explicitly covered by the SEED Act. In partner countries, the existence or development of an appropriate legal or administrative framework will be critical to the sustainability of programs or projects. Enforcement training, for instance, would be of little consequence in a country which does not have sufficiently strong environmental regulations or which has not dedicated sufficient funding to maintain a cadre of qualified inspection and enforcement personnel. EPA's capacity-building efforts are often directed at strengthening these frameworks as a prelude to other environmental improvements. In Central America, for example, EPA has worked with the Central American Commission for Environment and Development (CCAD) to develop compatible systems for environmental legislation, regulations and standards among the seven countries which participate in the CCAD. Financial Resources The third factor - related to the first two - is the commitment of adequate funding to support the program through completion. On the U.S. side, adequate funding includes the monies necessary to support EPA participation: travel and per diem of EPA personnel, contracts and grants and, in some cases, salary funds for field personnel. On the host country side, it includes multilateral grants and loans, the commitment of public funds (national, municipal and/or local), and the leveraging of private sector investment to the greatest extent possible. EPA and the host country should work together to identify and mobilize additional funding through all available mechanisms. EPA's work with Poland in improving drinking water disinfection, wastewater treatment, and water monitoring equipment for Krakow provides a good example of the importance of obtaining adequate financial resources. Garnering matching funds to EPA's initial budget of $4 million, Polish authorities provided buildings to house EPA-provided equipment as well as power and piping to fully complete all planned facilities. They also built long-term operation and maintenance costs into the core budget for the program. Funding facilitated institutional support for the watershed management and environmental education programs needed to sustain the infrastructure investments. In some cases, political and social changes may necessitate significant re-budgeting during the course of a program. A major watershed initiative in Russia, for example, included the demonstration of improved agriculture and water management methods. Midway through the three- year effort, the initial investments in demonstration site design and field implementation were progressing, but the Russian partner (a swine operation) went bankrupt due to vast institutional -17- ------- constraints and shifts in the agricultural sector. Because project components on environmental education and citizen participation were proceeding much more successfully than expected, EPA and its Russian counterparts shifted the funds to these elements in order to maximize project results. Personnel The fourth factor is the commitment of appropriate management and technical personnel to select, design, implement and evaluate projects under the program. The EPA should seek out personnel with strong and current technical skills, international experience and cross-cultural sensitivity, flexibility, and a commitment to partnership. It is also important that participating EPA personnel enjoy the full support of their management at all levels. On the host country side, effectiveness will require the identification (or development, if necessary) of committed and results- oriented staff with the necessary management and technical skills. Kaniv Reservoir Water Quality Project EPA's support (from 1994 - 1998) for assessing the water quality of the Kaniv Reservoir in Ukraine offers an example of committing appropriate staff to a program before beginning a project The Kaniv Reservoir is intensively used for recreation, fishing, and some water supply, yet its position downstream from the Chernobyl nuclear power plant accident has been a serious health concern. The assessment was carried out by a broad, multi-disciplinary partnership between EPA Region 4 (Atlanta), the Office of Water, and the Office of Research and Development on the U.S. side and a team from Ukraine. The EPA team included experts on international water program design, field monitoring, water quality assessment, water modeling, laboratory analysis, fisheries, ecology, compliance and enforcement, and other fields. The involvement of EPA specialists was fully endorsed at the Assistant Administrator and Regional Administrator levels, and was incorporated into staff Performance agreements. EPA specialists worked with counterparts from the Ukrainian Ministry of Environmental Protection and Nuclear Safety, the Ukranian Environmental Inspectorate (field and laboratory staff), and several specialist technical institutes. The level of active project participation ranged from the Assistant Administrator-equivalent to field technicians. Ukraine's endorsement of the Kaniv program as one of the two most important efforts under U.S.-Ukrainian environmental collaboration facilitated the assignment of key staff from both partners, which set the foundation for a successful assessment Public Participation and Involvement of Stakeholders The World Bank's Assessing Aid report found that participation by beneficiaries is crucial to the success of traditional rural water supply projects. Low participation by the beneficiaries on projects was associated with a project success rate of only 14%, in contrast to the 68% success rate for high participation projects. Similarly, EPA's capacity-building efforts can only succeed when there is a sincere commitment to participation and change by partners. EPA's domestic and international experience has clearly demonstrated the importance of public participation and the involvement of stakeholders in assuring meaningful and sustainable -18- ------- results. A knowledgeable and involved public will usually build and maintain support for the overall program. Non-governmental organizations (NGOs), both national and international, industry coalitions, affected ministries and other state and local groups often possess the most detailed understanding of existing conditions and needs, and can also be expected to contribute additional field-based people and resources. INVOLVEMENT OF STAKEHOLDERS IN THE BUTUANON RIVER PROJECT EPA's efforts (under the U.S.-Asia Environmental Partnership) to help clean up the Butuanon River in the Philippines provides a good example of the importance of stakeholder involvement to the sustainability of aprogram. Most of the stakeholder groups in the Butuanon River project were identified at the outset of the program by a local proponent, die Mayor of Mandaue City. While some of these stakeholders, particularly the NGOs, had voiced concerns for many years about increasing pollution in the Butuanon River, many felt powerless to become involved. One of EPA's critical first efforts was to unite the various stakeholder groups into an organization with the power to make decisions. Doing so allowed these groups to develop a shared understanding of the problems and of possible solutions; it also nurtured a sense of ownership. Buy-in was also critical to enlisting the cooperation of members of the industrial community, who, though partially motivated by the threat of regulatory action, came to understand the significance of their pollution. Consequently, industrial partners in the organization continued to construct pollution control facilities even through the Asian financial crisis. EPA experts provided support to this group in overseeing the project, including assistance in conducting the initial watershed environmental assessment and in developing and implementing a management plan. PROJECT GUIDELINES Once the Agency has decided to undertake a particular program, there are five basic stages of implementation: project selection, design, implementation, monitoring, and evaluation. Many of the guidelines described earlier for programs will also apply to specific projects carried out under the program. Below is additional information on each element of the project cycle, with examples from past and ongoing capacity-building programs used for illustration. Project Selection Selecting the right projects is the first step toward successful implementation of a capacity- building program. A preliminary assessment carried out jointly by EPA and the host country in consultation with other organizations should identify a number of potential projects. Drawing on information contained earlier in the 'Tactors for Success" section, potential projects also should be evaluated and ranked according to the extent to which: — the project addresses high priority issues for both EPA and the host country; -19- ------- - available funding is commensurate with project objectives; - information on baseline conditions is available to measure the success of the project; - the host country's legal, policy and scientific/technological framework will ensure effective implementation and sustainability; — project results can be adapted for replication in other parts of the country or region; — additional funding can be leveraged from outside sources (e.g., public and private investments, multilateral development banks); - EPA (or other agencies and the host country) are able to dedicate the appropriate personnel and/or contractors to the project; and — the project involves the appropriate stakeholders, with particular emphasis on the presence of a local champion to take ownership for the project and ensure long-term results. Project Design Major factors to be considered during project design include mutual agreement on: — goals, objectives, and performance measures that can be tied to both EPA's strategic plan and the host country's priorities; — respective roles and responsibilities for proj ect implementation, including staff-time, financial contributions, travel expenses, and information dissemination; - budget and time frame, including milestones and deadlines; and - identification of baseline conditions against which the results of the project will be measured. These elements should be addressed hi writing as part of a project work plan prior to initiating any activity. As described in greater detail later in this report under "Measuring Results," an understanding of baseline conditions will enable project managers to set priorities, gauge progress and assure mid-term corrections, and evaluate the ultimate effectiveness of the project. Project Implementation Properly selected and designed projects will greatly facilitate project implementation. -20- ------- Because even a well-designed proj ect can encounter unexpected obstacles, it is wise to revisit project design elements on a regular basis and adjust milestones, budget estimates, and other plans to account for changing conditions. In the event minor adjustments are not sufficient, it might be necessary to cancel or suspend a project. Political upheaval in Indonesia, for example, caused EPA to suspend two of its train-the-trainer courses there. As the country returns to political stability, the Agency will resume these course. Flexibility and careful monitoring are key to successful implementation. EPA staff have found that projects which include the provision of equipment or training and technical assistance should receive close attention during the implementation stage since other capacity-building efforts frequently rely upon the equipment provided or upon the skills transferred through training or assistance. The paragraphs below discuss these two elements in greater detail: Providing Equipment: In cases where equipment transfer is set forth in the project design, EPA should ensure that the host is prepared and trained to house, use and maintain the equipment properly prior to the transfer. Shipping equipment without making such a determination is inadvisable, especially in cases where other components of the capacity-building program are predicated upon the availability of working equipment. For example, the air monitoring and water treatment equipment provided by EPA under the provisions of the original SEED Act have been fully incorporated into the environmental program of the Krakow region in Poland, and the local authorities have committed portions of their own budgets to ensure that adequate staff are assigned to use and maintain the equipment as required. Without an appropriate commitment of staff and maintenance funds on the part of the host country, investments in hardware and software will not be sustainable. In Bulgaria, EPA provided several Geographic Information System (GIS) workstations and software to the Ministry of Environment and trained staff to use the equipment. However, because mid-level management at the Ministry did not commit its own resources to staff and maintain the equipment over time, the equipment was not used to its maximum potential, and it eventually fell idle for extended periods of time. By the time management changes took effect at the Ministry and a good-faith effort was made to refurbish and use the equipment, the technology had already become obsolete. Training and Technical Assistance: In the case of training and technical assistance projects, host countries should take the lead in ensuring that EPA's assistance is targeted at the appropriate audience. EPA, in turn, should ensure that participants in training courses have the ability to use acquired skills. Delivery of EPA's training module on compliance and enforcement in the Philippines, for example, benefitted from the participation of the Secretary of the Committee on Ecology, a permanent committee of the Philippine Congress tasked with drafting key environmental laws. The Secretary used skills learned through EPA training in drafting new legislation on clean air in the Philippines. EPA should also ensure that training and technical assistance are carried out within a broader country framework and are tied to specific environmental outcomes. For example, EPA's training in Poland on negotiating compliance agreements with industry was integrated into a broader effort -21- ------- to enhance the Polish environmental enforcement system. As a result of the training, four pilot agreements with industrial facilities paved the way for their provisional removal from the list of the 80 worst polluters in Poland. Other industrial facilities are now requesting similar training opportunities and additional courses are being offered. Project Monitoring Joint development of performance indicators and baseline data will enable project managers to gauge progress in achieving project objectives and facilitate agreement on any mid-term corrections which might prove necessary. The section of this report on measuring results under the GPRA provides more detailed information on both the selection of indicators and the establishment of baseline conditions. Project Evaluation Project evaluation should be built into every EPA capacity-building program. The evaluation should cover such issues as the following: — How successfully were project objectives accomplished? — What did work and why? - What didn't work and why? - Were there any unexpected results (positive or negative)? - What are the lessons for future applications in other settings? — How were data collected and used to ensure project effectiveness? EPA and the host country should also develop a strategy for disseminating project results - along with recommendations on project improvement—for use in adapting or replicating the project in other parts of the country or region. More detailed information on measuring and evaluating the results of programs can be found in the next section. -22- ------- MEASURING RESULTS UNDER GPRA A critically important factor in the success of EPA's international programs will be the Agency's ability to measure the effectiveness of its programs under the GPRA. Going beyond the project design, selection, implementation, monitoring, and evaluation considerations discussed above (activities which are carried out jointly with the partner country), the ability to gauge progress will better enable the Agency to set priorities, manage limited financial and human resources, and undertake corrective action as needed. It will also enable the Agency to better justify the results of its programs to Congress, the Administration, and the public. The task force consulted work carried out by US AID, the World Bank, and the Organization for Economic Cooperation and Development (OECD) in developing performance indicators and other forms of measurement, particularly as they relate to international development work. It developed a framework, consistent with the broader EPA measurement methodology under GPRA, for better measuring the effectiveness of the Agency's international capacity-building programs. ESTABLISHING PERFORMANCE GOALS AND MEASURES The first step in evaluating the effectiveness of a program is to establish the long-term goals against which progress can be measured. The illustration on the next page provides a conceptual hierarchy for establishing goals and measuring the effectiveness of EPA's international capacity- building programs. As discussed previously, there may be many reasons for EPA to become involved in a capacity-building program. The Agency might find, for example, that action in a developing country could achieve the same emission reductions at far lower cost than similar action in the United States. The Agency might also respond to foreign policy, economic, or other imperatives set by Congress and the Administration. Regardless of the initial impetus, once EPA is engaged in the work, the work itself is necessarily of an environmental nature, and the long-term result of the program should be healthy people and ecosystems, either in the United States or in the host country in which activities are carried out. This goal can be found at the top of the hierarchy. The second level reflects the state of ambient air, water, and soil quality upon which healthy people and ecosystems depend. This would imply, for example, air and water quality that would meet established international standards for breathing, drinking, etc. -23- ------- GOAL HIERARCHY FOR ERA'S INTERNATIONAL CAPACITY-BUILDING PROGRAMS 1. Healthy People and Ecosystems 2. Ambient Air, Water and So3 Quality 3. Pollution Prevention, Control and Remediation 1. Laws and Policies, Institutions and Public/Private Funding At the third level of the hierarchy - directly contributing to the level above - is action by the foreign government to prevent pollution in the first place, to control the pollution that cannot be avoided, and to remediate pollution problems that already exist. At the bottom of the hierarchy is action by the foreign government to establish underlying conditions which will support the environmental outcomes at the hierarchy's higher levels. This level includes the establishment of effective laws and policies, the strengthening of national, municipal and local institutions, and the mobilization of funding for environmentally beneficial purposes from both the public and private sectors. Absent these conditions, environmental outcomes will not be sustainable, even if achievable in the short-term. The Agency's shorter-term technical assistance, training, information exchange, and demonstration projects are directed largely at the third level. In this case, the Agency works directly with host country officials in taking specific action to prevent, control or remediate pollution problems. As such, these programs should be measured in environmental terms. Table 1 on "environmental indicators" lays out a set of generic measures that can be considered by EPA program managers in developing annual performance goals (APGs) and annual performance measures (APMs) under GPRA. In general, EPA's international capacity-building programs include work directed at both the third (pollution prevention, control and remediation) and fourth (laws and policies, institutions and funding) levels of the hierarchy. The level at which the work is directed has significant implications -24- ------- for the way the results can and should be measured. TABLE 1: ENVIRONMENTAL INDICATORS CATEGORY Air Water PERFORMANCE GOAL decrease air-related disease occurrence in children, other sensitive populations, and general populations reduce ambient concentrations of priority air pollutants reduce emissions of priority ah- pollutants reduce greenhouse gas emissions reduce emissions of stratospheric-ozone depleting compounds decrease water-related disease occurrence hi children, other sensitive populations, and general populations improve drinking water quality increase access to drinking water/sanitation preserve surface water and groundwater quality preserve marine environments SAMPLE MEASURE (in %or#) reduction of respiratory diseases hi targeted population reduction hi ambient concentrations of SO2, NOx, and other targeted pollutants reduction hi stack emissions reduction hi pollution per vehicle miles traveled reduction hi CO2, NOX, and other GHGs reduction of CFCs and other ozone- depleting chemicals reduction hi gastrointestinal and other waterbome illnesses hi targeted population increase of drinking water plants that meet standards increase of households with direct access to drinking water increase of households with access to sanitation increase of protected watersheds increase hi surface water bodies meeting national/international standards reduction hi contamination of water by pesticides from agricultural sources increase hi wastewater treatment reduction hi pollutant inputs/loadings health of selected biological indicators (e.g., coral reefs, mussels, oysters) water quality trends (where available) -25- ------- Waste source reduction recycling of waste sound management/disposal of waste reduction in generation of solid & hazardous wastes increase in recovered waste increase in waste transported, treated, stored and/or disposed according to established standards; decrease in sites contaminated with waste. Natural Resources preserve natural resources and biological diversity increase in alternative use of renewable energy sources (e.g., solar, wind, geo thermal) increase in protected species, ecosystems, and habitats (e.g., forests, wetlands, coral reefs) Taking into account the over-arching goal of sustainability, there are strong arguments in favor of focusing the Agency's international programs on the fourth level of the hierarchy - namely, the establishment of the necessary legal, policy, institutional, and financial framework - that will ensure broader-scale and more sustainable results over the long-term. Doing so, however, means that the Agency will need to measure the success of such programs in qualitative, programmatic terms. Table 2 on "programmatic indicators" lays out generic performance goals and measures in this area. TABLE 2: PROGRAMMATIC INDICATORS CATEGORY PERFORMANCE GOAL SAMPLE MEASURE Laws Effective and enforceable environmental laws at national, provincial, municipal, and local levels Adoption of new or strengthened environmental laws, regulations, codes, etc. Establishment of judicial due process, dispute resolution, and environmental liability procedures Adherence to international agreements Policies Selection and use of most effective policy instruments at national, provincial, municipal, and local levels Successful identification and implementation of effective policy tools, including single-source (e.g., harm-based or design standards), multi-source (e.g., integrated permitting, tradeable emissions), and other (e.g., pollution charges, liability) -26- ------- Institutions Effective, responsive and accountable institutions at national, provincial, municipal, and local levels Enhanced environmental analysis capabilities (e.g., sampling and laboratory equipment) Enhanced environmental monitoring capabilities Enhanced enforcement capabilities Establishment of mechanisms to transfer knowledge and experience among institutions Establishment of emergency preparedness and response capabilities Human Resources Enhanced expertise, experience, and/or credentials of policy-makers, managers, and technical personnel Establishment/strengthening of environmental management and science programs (e.g., through universities) Establishment/strengthening of professional associations and accreditation programs Establishment and replication of training programs, fellowships, exchanges, study tours, etc. Information Access to environmental information Enhanced capabilities to generate, store, retrieve and analyze environmental data and other information Improved vehicles for disseminating environmental information (e.g, through Internet) Financing Enhanced public investment % increase in public investment in environmental infrastructure and environmentally beneficial programs Enhanced private investment % increase in environmentally beneficial investment Enhanced trade in environmental technologies and services % increase in sales of environmental or environmentally sound technologies and services -27- ------- Public Participation Participation of active and informed citizenry in all aspects of environmental decision-making and implementation % increase in number of environmental education programs at primary, secondary, and graduate-school levels % increase in number of environmental NGOs per thousand population % increase in financially self-sustaining NGOs To the extent to which EPA's international capacity-building programs straddle the third and fourth levels of the hierarchy, EPA program managers should draw on a mix of environmental and programmatic indicators. In the case of EPA's international initiative on microbiologically safe drinking water, for example, program managers might use a mix of both environmental indicators (e.g., a quantitative decrease in the number of children exposed to contaminated drinking water) and programmatic indicators (e.g., a qualitative increase in laboratory capacity for analyzing the microbiological quality of drinking water). ESTABLISHING BASELINE CONDITIONS The establishment of baseline conditions — or reference values - against which to measure a program's effectiveness will be among the primary challenges facing EPA hi implementing its international capacity-building programs. Data on existing environmental conditions in developing countries are often non-existent or incomplete; where the data do exist, there are often serious problems with consistency and compatibility. At the same time, EPA's ability to measure the effectiveness of its programs in any meaningful way depends upon such data. The lack of adequate and consistent data has affected EPA's most successful international effort on this issue to date, the U.S.-Mexico environmental indicators program (see text box on next page). The approach taken there might contain insights for international capacity-building programs more generally. Data are readily available for many of the areas covered by the program. For example, the Border program selected indicators for ambient air quality and emissions for which much of the necessary baseline information is already available in both the U.S. (e.g, through EPA's Airometric Information Retrieval System) and Mexico (e.g, the Mexican National Institute for Ecology).5 However, lack of consistent or compatible data has prevented the establishment - at least temporarily—of quantifiable indicators in other areas (e.g., hazardous and solid waste),6 though the workgroups have pledged to narrow these data gaps hi future work. The North American Commission on Environmental Cooperation (NACEC) has launched a trilateral effort to identify and eliminate data gaps related to the safe management of chemicals. -28- ------- ENVIRONMENTAL INDICATORS FOR U.S.- MEXICO BORDER EPA's work with Mexico in developing environmental indicators for binational work along the border is probably the Agency's most comprehensive effort to date in developing environmental indicators specific to international cooperation. Prepared jointly by binational working groups, the report includes indicators for each focal area covered by a working d ^ chemicals covered by the treaty. group: water, air, hazardous and solid waste, pollution r ° J J prevention, contingency planning and emergency response, cooperative enforcement and compliance, environmental information resources, natural resources and environmental health. indicators covering the full-range of water quality measurements, including the percentage of population being served potable water; population provided wastewater sewer services; wastewater collected receiving wastewater treatment; and total volume of Another example of a capacity- building program for which data are more readily available is the international effort to protect the stratospheric ozone layer. In this case, reporting provisions under the Montreal Protocol require all participating nations to provide the United Nations Environment Program (UNEP) with annual reports on the production and consumption of ozone- The Montreal Protocol Fund also pays for projects in developing nations which gather baseline data and build the administrative capacity to report the data. The resulting data, _, t . •_ , j i j which date back to 1986, provide a useful The water workgroup, for example, developed r vehicle for gauging a country's progress in implementing the Protocol. What should the Agency do when baseline data is not reliable or easily drinking water being disinfected prior to delivery. .... , _ _ _ , . accessible, though? The preferred option is to build the establishment of baseline conditions into the project itself as part of the design phase. Although doing so will likely raise the costs of the overall project, common sense suggests that the ability to set priorities, gauge progress and assure mid-term corrections — all based on a thorough understanding of baseline conditions — will be essential to a project's overall success. Conversely, goals might need to be adjusted to reflect the reality of data gaps. Data most closely tied to the project's activities will clearly be the most valuable source of baseline information. In the case of EPA's small community drinking water demonstrations in Mexico, Ecuador, and China, EPA experts visited local health clinics in communities to be served by the demonstrations. These visits allowed EPA to determine the number of cases of diarrhea and other intestinal diseases reported prior to the demonstrations, which became a baseline for judging water quality improvements. National health and environmental ministries often compile the same information on a broader scale, and may be consulted for data on national and regional conditions and trends. However, EPA and its partners will need to review the quality of such data to confirm that it can serve as a meaningful baseline for measurements. There are also a number of international organizations which provide information on a national and international scale. UNEP manages the Global Environmental Monitoring System, which contains worldwide information and data on the environment. The World Health Organization (WHO) collects, analyses, and disseminates a broad range of health-based data in almost 200 member nations. Similarly, a general database managed by the United Nations Industrial Development Organization (UNIDO) collects data on 13 industry- -29- ------- related indicators, providing information on pollution production status for the 96 countries contributing data. Unfortunately, significant gaps in critical environmental data persist worldwide. Until the international community - and nations individually - do a better job of collecting and integrating information, any solution will be imperfect at best. The emphasis should be on generating, analyzing, and reporting data which meet a number of criteria, including completeness, accuracy, clarity, uniformity and timeliness. CRITERIA FOR ESTABLISHING "IDEAL" INDICATORS The OECD has developed three criteria for use in establishing "ideal" indicators: (1) Policy Relevance and Utility for Users: An environmental indicator should provide a representative picture of environmental conditions, pressures on the environment or society's responses; be simple, easy to interpret and able to show trends over time; be responsive to changes in the environment and related human activities; provide a basis for international comparisons; be either national in scope or applicable to regional environmental issues of national significance; and have a threshold or reference value against which to compare it, so that user are able to assess the significance of the values associated with it. (2) Analytical Soundness: An environmental indicator should be theoretically well founded in technical and scientific terms; be based on international standards and international consensus about it validity; and lend itself to being linked to economic models, forecasting and information systems. (3) Measurability: The data required to support the indicator should be readily available or made available at a reasonable cost-benefit ratio; adequately documented and of known quality; and updated at regular intervals in accordance with reliable procedures. OTHER MEASUREMENT CHALLENGES EPA will also encounter a number of other challenges in measuring results which, while not unique to capacity-building, deserve mention. Quantifying Environmental Outcomes The Agency's clear preference is to use quantifiable environmental outcomes wherever possible. At the same time, EPA may find that in some cases qualitative measures are preferable to quantifiable ones. The delivery of a few top-quality technical assistance or training projects is better than delivering five or ten projects of lesser quality. -30- ------- Establishing Linear Relationships Establishing linear (i.e., "cause-and-effect") relationships between the work EPA carries out in cooperation with its partners and environmental outcomes presents another challenge. EPA's assistance might play an instrumental role in securing a country's commitment and ability to undertake action on a particular issue. At the same time, there will clearly be many other factors (such as political considerations) beyond EPA's influence. How can these other factors be incorporated into honest and credible performance measures which accurately reflect the MEASURING RESULTS OF INTERNATIONAL TRAINING With enormous demand for environmental training worldwide, EPA's international "train-tile-trainer" modules are generally recognized as among the Agency's most effective tools for international capacity-building. The Agency has used the modules to train over 5000 people representing over 20 countries since 1990, and EPA continues to update and add new modules in response to increased demand. The Agency is seeking to modify module format to benefit from the lessons learned over recent years and, by adding more country- or region- specific case studies, to adapt the modules more specifically to the needs of its participants. It has also improved the way it plans and evaluates the training programs carried out At the same time, training by its very nature poses a special challenge to measurement efforts, and there is still considerable work for EPA to do in this area. The Agency can currently quantify the number of individuals who have received the training, either directly from EPA trainers or through EPA-trained facilitators from the host country. But the Agency cannot accurately measure the quality of the training provided; nor can it gauge — beyond anecdotal information — the full extent to which imparted expertise is being applied on behalf of specific environmental outcomes. EPA might be successful, for example, in providing the highest quality training to officials with the commitment and ability to exact an environmental solution. However, exogenous factors outside of EPA's influence (e.g., political considerations) might make it difficult to establish a linear relationship between EPA's training and environmental results downstream. contribution of EPA's work? One objective under EPA's Strategic Plan is to help "at least 75 developing countries comply with their obligations under the Montreal Protocol to achieve a 50% reduction in their production and consumption of CFCs" by 2005. The objective is an outcome: achieving it will lead directly to environmental results (i.e., decreased emissions of ozone-depleting chemicals). EPA's technical and financial assistance to developing countries under the Protocol has undoubtedly played a significant role in securing developing country participation. Yet, there are other factors beyond EPA's influence that may have far more effect on a country's ultimate ability to comply. How, then, are these influences reflected in the reporting of EPA's progress? The interim solution in this case was through the development of an annual performance goal for FY1999 within -31- ------- the Strategic Plan stating that "[t]hrough our contribution to the Multilateral Fund, assistance will be provided to at least 50 countries working toward achieving the Montreal Protocol." By specifying the number of countries to which assistance will be provided (50), the performance goal becomes quantifiable. The problem, though, is that it measures process (i.e., assistance) as opposed to environmental outcomes (e.g., CFC emission reductions). Measuring "Results-Avoidance" Activities Another challenge - again not unique to capacity-building programs—will be measuring the effectiveness of "results-avoidance" activities and other intangibles. Many of EPA's capacity- building programs are designed to prevent a negative environmental result (e.g., emphasize pollution prevention as the preferred management approach). How should the Agency quantify pollution that is not generated as a result of these activities? How can the Agency measure "credible" enforcement programs and then: "deterrent" effect? Partial solutions to both of these issues can be found in the Border XXI Indicators Project. The binational workgroup on pollution prevention is developing indicators for pollution prevention which will rely on "data that are normalized for production, a calculation that distinguishes actual reductions in waste generation from normal changes in production."7 The workgroup has also begun collecting data from site assistance visits, from workshops, and from industry reports on water usage in industrial processes so that it may demonstrate "before-and-after" results. With respect to enforcement and compliance, the binational workgroup covering this area decided to use "output" measures (such as the number of inspections conducted along the border and the number of enforcement actions undertaken and penalty fees collected) as an indirect - or surrogate - indicator for environmental outcomes. Again, however, this approach leads to the use of process and output activities as opposed to the preferred use of environmental outcomes. -32- ------- FUTURE MANAGEMENT CHALLENGES EPA's international capacity-building programs are achieving substantial and quantifiable environmental results worldwide. This report lays out a number of ways for improving the effectiveness of these programs. Careful deliberation of the requirements for success before launching a major capacity-building program, consistency with EPA's domestic mission and strategic priorities, consideration of "best practices" in carrying out programs, and an ability to measure results under GPRA will each play an important role in helping to assure that the Agency's limited resources are committed strategically and effectively. Particularly important in this regard, as already discussed in the report, will be the Agency's willingness and ability to fund the establishment of baseline conditions as an integral component of every capacity building program or project. In the program management environment created by GPRA, the key to measuring the success of a program depends on setting clear goals and objectives during the design phase. Whether the objectives are environmental results or programmatic outcomes, the Agency and its partner organizations will need to rely upon valid, defensible baseline data as markers against which the program's success or failure can be measured. 4 Baseline information for environmental and human health conditions in developing countries can be difficult to obtain. In many cases, EPA might need to gather the baseline data itself prior to initiating a program. This undertaking could be both expensive and time-consuming, thereby limiting the scope of the Agency's international capacity-building efforts. Should EPA decline to implement otherwise attractive and useful capacity-building programs if they require a substantial outlay of funds to establish a baseline or to measure change against that baseline? How should EPA handle programs which arise out of Presidential initiatives but which do not supply sufficient funding for baselines/measurements? The Agency will also face a number of the challenges in the area of international capacity- building which, going beyond the mandate of the task force, have not been addressed hi this report. The task force flags these issues now for future consideration by senior EPA management. 1. How can EPA improve cross-Agency management and coordination of various international capacity-building programs underway throughout the Agency? Virtually every EPA office or region is engaged in the Agency's international capacity- building work. Broad participation strengthens the programs, offers a broader range of EPA staff the opportunity to learn about international environmental issues, and ensures that no single office bears the weight of starring these often complex programs. Effective cross-Agency coordination facilitates the transfer of technical tools and "lessons learned" among the programs and helps ensure -33- ------- that all programs further the goals set forth in EPA's Strategic Plan. The Goal 6 Team under EPA's Strategic Plan, led by OIA, plays an important role in coordinating overall Agency programs in the international arena. The Agency might want to consider a more formal role for this Team in coordinating programs specific to international capacity-building. 2. How can the Agency improve the link between its international capacity-building programs and broader EPA and U.S. policies and programs? The success of EPA's work on stratospheric ozone depletion, climate change, and other global and transboundary issues require developing-country participation in resolving problems that the U.S. cannot solve unilaterally. International capacity-building programs on a particular global or regional environmental issue should be carefully integrated with, and support, U.S. policy in international negotiations. Programs will be most effective when carried out within the broadest possible framework that meets the particular needs of the partner country while also furthering the Agency's objectives. A cross-Agency coordination effort, as outlined above, would prove useful in assuring this integration. 3. How can the Agency improve the way it works with the U.S. Agency for International Development, the multilateral development banks, and other international financial and development organizations? EPA has worked closely with USAID, multilateral development banks, and other international financial and development organizations in mobilizing substantial resources in support of environmental objectives worldwide. Because such externally funded programs need to reflect the interests of both EPA and the sponsoring organizations, these programs frequently involve extensive negotiations between the two parties. It is critical in representing Agency interests at these negotiations that EPA has fully defined its priorities in this area and is aware of ongoing programs. The Agency should also consider the implications of its reliance on external funding sources for international capacity building programs. -34- ------- APPENDIX A: TASK FORCE MEMBERS Jamison Koehler (Co-Chair), Office of International Activities Joan Fidler (Co-Chair), Office of International Activities Vickie Booths, Office of the Chief Financial Officer Michael Brody, Office of the Chief Financial Officer Keith Chanon, Office of Pesticide Programs Dennis Cunningham, Office of International Activities Orlando Gonzalez, Office of International Activities Katherine Dawes, Office of the Administrator Francesco DiCosmo, Region ffl (Philadelphia) Alan Farmer, Region IV (Atlanta) Brenda Groskinsky, Region YE (Kansas City) John Haederle, Region I (Boston) Brian Hirsch, Office of International Activities Ron Hoffer, Office of Water Melissa Jaeger, Region n (New York) Patricia Koshel, Office of International Activities Anne Lesperance, Region X (Seattle) Daniel Magraw, Office of General Counsel James Morant, Office of the Administrator Ari Nathan, Office of Policy Lee Otis, Region X (Seattle) Anna Phillips, Office of International Activities Vacys Saulys, Region V (Chicago) Ellen Spitalnik, Office of Solid Waste and Emergency Response Doug Steele, Office of Research and Development David H. Strother, Office of Pollution Prevention and Toxics Jan Tarpey, Region VII (Kansas City) Beverly Updike, Office of Enforcement and Compliance Assurance Gina Weber, Region VI (Dallas) ------- APPENDIX B: OVERVIEW OF EXISTING PROGRAMS Capacity-building is a critical component of virtually all of the Agency's international programs. Ongoing capacity-building programs at EPA include treaty-related efforts on stratospheric ozone depletion, climate change, and persistent organic pollutants; AID-funded assistance programs in Central and Eastern Europe, the New Independent States, Central America and Southeast Asia; bilateral programs with Mexico, China and other priority countries; and the Agency's new initiatives related to persistent bioaccumulative toxics (PBTs), and microbiologically safe drinking water. The Agency has also developed a set of generic capacity-building tools — covering the areas of training, information, education and financing - that can be applied universally. Provided below is additional information on each of the Agency's on-going programs and capacity-building tools. GLOBAL AND OTHER ENVIRONMENTAL ISSUES Stratospheric Ozone Depletion Despite growing awareness of stratospheric ozone depletion in the late 1980s, only a handful of developing countries participated in the original negotiations of .the Montreal Protocol on Substances that Deplete the Ozone Layer, and the Protocol's initial effectiveness was undermined by the lack of developing country participation. Many developing countries, led by China and India, not only questioned their responsibility for helping to solve the problem but, importantly, their ability to do so as well. EPA was an active participant in the international assessments that laid the basis for strengthening the terms of the Protocol through the London Amendments of 1990: a scientific assessment which helped convince reluctant developing countries of the seriousness of the threat; and economic and technical assessments which alleviated many concerns about the economic and technical viability of alternative chemicals and processes. EPA's bilateral assistance to developing countries in preparing national emission inventories and in identifying alternative chemicals and processes - as well as EPA's leadership in establishing the Montreal Protocol Multilateral Fund - also played an influential role in achieving greater developing country participation. Over 100 developing nations and econornies-in-transition are implementing the provisions of the Protocol, worldwide production and consumption of ozone-depleting chemicals have been substantially curtailed, and the ozone layer has begun the first steps toward recovery. Climate Change Developing country participation will be critical to global efforts to stabilize and reduce -37- ------- greenhouse gas emissions. Toward this end, EPA is working closely with USAID, the Department of State, the Department of Energy, and other U.S. agencies to enlist the participation of key developing countries. The overall objectives of this work are to: (1) provide technical and financial assistance through cooperative bilateral efforts in key countries, such as Argentina, who demonstrate willingness to assume commitments; (2) promote capacity of key developing countries and economies in transition in support of U.S. climate policy positions and strategies to conduct economic and policy analysis; improve data collection, emission measurement, emission inventories, and projects; develop and conduct mitigation and adaptation strategies; provide training for use of market mechanisms (including programs for criteria pollutants); develop and conduct co-control approaches; and encourage technology transfer and innovative funding mechanisms; and (3) share effective non-regulatory program models—such as the ENERGY STAR branding strategy for energy efficiency - to increase understanding of the potential of market-based approaches to reduce greenhouse gas emissions in the largest emitting countries. Persistent Organic Pollutants Capacity-building will be equally important to the success of ongoing negotiations on an international convention to control persistent organic pollutants (POPs), such as DDT and polychlorinated biphenols (PCBs). The Intergovernmental Negotiating Committee, the multi- national body negotiating the convention, has dedicated significant attention to the role of capacity building in reducing POPs. Major capacity-building components under consideration for the convention include the development of POPs inventories; identification of emission sources and control approaches (e.g., for dioxins and furans); development of phase-out, management and disposal plans (e.g., for PCBs); management and disposal of stockpiled pesticides; and identification of safe and effective alternatives to POPs pesticides and PCBs. In support of these negotiations, the Agency has already begun working with a number of developing countries on plans to phase out production and use of selected POPs. With funding from the State Department, for example, EPA is planning a workshop for early 2000 to assist nations of the South African Development Community strengthen then* understanding of POPs-related issues in support of the convention. The Agency is also planning a workshop on dioxins and furans with the Egyptian Environmental Affairs Agency and, in cooperation with Japan and other partners, a similar workshop on dioxins and furans for the Asia region. There is little reliable data on the presence and effects of many of the persistent toxics in most countries, especially in the developing world. It will take a good deal of time to collect statistically-reliable, representative information on POPs production, use and release levels in these countries and on the long-range transport of persistent toxics. It also will take time to link exposures and their effects more definitively. It could be some time before EPA can move beyond process results (e.g., conclusion of POPs treaty, the establishment of monitoring stations in a country) and outputs (e.g., establishment of a Pollutant Release and Transfer Register which captures key persistent toxics) to outcomes (e.g., reduction of adverse health impacts linked to earlier exposures -38- ------- to one or more persistent toxics). Microbiologically Safe Drinking Water EPA launched a new program in February 1999 to enhance the capabilities of targeted developing countries to provide rnicrobiologically safe drinking water, a major factor in the protection of children's health. Initial implementation of the program in Central America has been facilitated through a multi-organizational framework led by the Pan American Health Organization. EPA leveraged initial funds included in EPA's FY 1999 appropriations bill with additional funds secured from USAID through the FY 1999 supplemental appropriations bill for Central American hurricane relief. EPA assessment teams have defined the following objectives for work in three Central American countries (El Salvador, Nicaragua, and Honduras): (1) enhancement of laboratory capacities for microbial and metal contaminants, (2) source water protection, (3) treatment plant optimization, and (4) general training and assistance on drinking water. In late 1999, EPA teams will begin working with our partners hi Central America to address each of these objectives. Because the initiative is still in the early stages of planning and implementation, the Agency can assure that meaningful performance goals andmeasures are incorporated into theprojectup front, incorporating many of the lessons of the past. Moreover, responding to the two-year time frame for demonstrating results under the AID-leveraged portion of the initiative, the initiative is geared for quick results. The Agency is also looking to extend the drinking-water initiative to Africa, depending on the availability of funding in FY 2000. GEOGRAPHIC-BASED PROGRAMS With the important exception of Mexico, for which funds are appropriated directly to EPA by Congress, many of EPA's geographic-based capacity-building programs are funded through USAID. As such, these programs reflect the intersection between EPA and USAID for environmental development assistance in targeted regions of the world. Mexico Strengthening the capacity of organizations on both sides of the border is a central focus of EPA's cooperative programs with Mexico. Growing concentrations of people and industry are causing increased stresses on the environment. In some high population localities, centralized wastewater collection and treatment systems have never been built. In other places, residents live in homes without plumbing or potable water. The U.S. and Mexico began working together on a formal basis to address environmental issues along their shared border in 1983 when the two governments signed the Agreement for the Protection and Improvement of the Environment in the Border Area (commonly known as the La -39- ------- Paz Agreement). In 1992, the environmental authorities from the two governments released the Integrated Environmental Plan for the Mexican-U.S. Border Area (IBEP), which the Border XXI Program (1996) helped implement. Environmental side-agreements under the North American Free Trade Agreement (NAFTA) also facilitated cooperation between the countries. One such agreement created the Border Environment Cooperation Commission (BECC) and the North American Development Bank (NADBank) to help develop, certify, and finance environmental infrastructure projects in the border area. EPA and its Mexican and Canadian counterparts also established a NAFTA technical working group on pesticides to address capacity-building efforts related to regulatory processes. In addition to bilateral work, the U.S. and Mexico are also working with Canada - through the Working Group on the Sound Management of Chemicals (SMOC) under the North American Commission on Environmental Cooperation (NACEC) - to enhance capacity related to the management of pesticides, including DDT and chlordane. Enhanced capacity through technology and information exchange and training has yielded substantial environmental results. The construction of wastewater treatment facilities in Mexico and the U.S. is helping solve decades-old environmental health problems. Joint air pollution efforts under the La Paz Agreement are helping reduce respiratory and other health related problems in cities along the border. Coordinated enforcement efforts are reducing illegal waste dumping and other pollution on both sides of the border. Other objectives of these joint cooperation and capacity- building programs include expanded tracking of transborder shipments of hazardous and toxic substances and improved binational emergency response procedures. The development of "Sister City Emergency Plans" between cities along the U.S.-Mexico Border seeks to coordinate response efforts to emergencies involving hazardous substances. These plans, with resource and technical assistance from EPA, represent the first step toward developing an efficient, coordinated standard emergency response procedure. El Paso-Ciudad Juarez-Dona Ana County Air Quality Improvement Basin Joint Advisory Committee (JAC) was created as a binational advisory group to EPA and Mexico's Secretariat for Environment, Natural Resources and Fishery (SEMARNAP) on air issues in the El Paso- Ciudad Juarez- Dona Ana County area. The JAC's primary function is to recommend to EPA and SEMARNAP air quality improvement measures from the community. The JAC also encourages cooperation in monitoring networks, compliance activities, economic incentive programs, public outreach and technology transfer. Progress continues to be made in border water and wastewater infrastructure projects. The BECC received $10 million in grant funds from the EPA's Border Environmental Infrastructure Fund. These funds have been used to enhance the BECC's technical assistance program in relation to water and waste water projects. An additional $10 million from EPA to the BECC will be used for final design projects, as well as for other eligible project components. Through this program totaling $20 million, the BECC has approved $11.1 million to assist 62 communities with 58 -40- ------- projects located in the U.S. or Mexico. Similarly, the BECC has financed 16 projects for five communities with $0.7 million of its own funds. In addition, BECC's Capacity Building Needs and Solid Waste Technical Assistance Program, supported through its own funds, has assisted 19 communities with $1.4 million for 15 solid waste projects. Technical assistance is also provided by the International Boundary and Water Commission (IBWC). The U.S. and Mexico sections of the IBWC agreed to assist border communities in developing facilities plans for wastewater infrastructure projects. These projects are developed with EPA funds and are administered by the U.S. Section of the IBWC. Consultants from both sides of the border assist development of these projects, and all projects must comply with BECC certification. The Pollution Prevention program has an extensive technical assistance and capacity- building program with the private sector along the U.S.-Mexico border. This program includes technical conferences and industry-specific manuals. Following assistance from this EPA program, participating maquiladoras in Mexico reported annual reductions of 4.5 million gallons of waste water, 7,001 tons of hazardous waste, and 47,789 tons of non-hazardous waste. The maquiladoras also reduced electricity usage by 8.6 million kilowatt hours per year. Participating plants saved a total of over $5 million annually through avoided disposal costs, reduced input costs, and other savings due to implementation of suggestions made by EPA and PROFEPA, Mexico's environmental enforcement agency. Other pollution prevention programs along the border which have enhanced local capacity involve the implementation of binational seminars for transporters and importers of maquiladora hazardous waste. The seminars (a joint EPA/PROFEPA effort) are part of a EPA's compliance assistance outreach program designed to provide the border maquiladora industry with an understanding of requirements associated with the return of hazardous waste to the U.S. The seminars included discussions on enforcement policies, voluntary compliance incentives, Federal and state requirements, as well as case studies and the latest innovative technologies for pollution prevention. EPA's National Environmental Policy Act (NEPA) team has prepared environmental assessment (EA)/Finding of No Significant Impact (FNSI) documents on six border projects receiving Border Environmental Infrastructure Fund (BEIF) grants. The NEPA Team has coordinated the preparation of the documents with BECC staff, and has provided guidance and training to NADBank, BECC and Mexican consultants on the NEPA process, historic preservation, wetlands, flood plains, and other cross-cutting law requirements. Eleven other projects are currently in-house for review and assessment, and EPA anticipates receiving 20 - 30 additional projects for review and assessment. * The NEPA team has also participated in the development of the international training courses Principles of Environmental Impact Assessment (PEIA) and Principles for Review ofEnvironmental Impact Assessment (PREIA) prepared for presentation in the various countries around the world including Latin America. Countries that have benefitted from the training include Mexico, Chile, -41- ------- Peru, Ecuador, Bolivia, Argentina, Costa Rica and presently, Guatemala. The Agency's programs in Mexico have also focused on capacity building for- chemical emergency preparedness and prevention. EPA has developed the Computer-Aided Management of Emergency Operations (CAMEO) in Spanish, and has provided Mexican officials with several training opportunities on using CAMEO, including lectures, discussion sessions, and hands-on computer training. These efforts have been effective means of transferring emergency preparedness and pollution prevention technologies to Mexico. Small grants have also been made available by EPA in an effort to provide local border communities with funds to implement their own capacity activities. EPA grants to address sustainable development, environmental education, pollution prevention, and Border XXI activities have been disbursed along the border. Central and Eastern Europe Neglect, mismanagement, and overuse of the environment and natural resources were among the many unfortunate by-products of the centrally-planned economies in Central and Eastern Europe prior to 1989. Unregulated and intensive industrial activity, especially in the post-World War II period, took a heavy toll on both the region's population and its environment. However, the fall of the Berlin Wall and the subsequent opening of the Iron Curtain provided a unique opportunity for the United States and other nations to work with the people of Central and Eastern Europe (CEE) to promote democracy and address their environmental problems. The U.S. government responded by including funding for environmental programs in the Support for East European Democracy (SEED) Act of 1989, which specifically directed EPA to assist in the environmental reconstruction of the region. With significant financial support from the U.S. Agency for International Development (US AID) and in partnership with other U.S. agencies, EPA has conducted cooperative environmental programs in Bulgaria, the Czech Republic, Estonia, Hungary, Latvia, Lithuania, Poland, Romania, and Slovakia. EPA also promoted regional solutions to transboundary problems through the creation of the Regional Environmental Center (REC) for Central and Eastern Europe, located in Szentendre, Hungary. In terms of program design, EPA's approach in Central and Eastern Europe has attempted to maintain a balance between helping to alleviate some acute problems in the near term, while addressing the long-term need to strengthen management capacity and foster effective environmental institutions. Through provision of training and technical assistance, this work has been carried out in two major areas: institution-building and demonstration projects. Institution-building activities strengthen public and non-governmental institutions by providing information on legislative/regulatory frameworks, various environmental assessment tools, and pollution mitigation and prevention hi order to encourage the development of sound environmental policies, hi particular, EPA developed a series of environmental management -42- ------- training modules within the framework of the CEE program to enhance the professional credentials of an already trained workforce, and to build capacity for improved environmental management in the region. Demonstration projects address high priority environmental issues in key geographic areas or "hot spots" where large populations are at risk from exposure to numerous pollutants, or where unique natural resources are threatened. Such projects demonstrate innovative, cost-effective, and appropriate technologies or are focused on pollution or environmental problems common to the region. These projects also are designed with future replicability in mind, so that successful approaches, once adapted to local conditions, can be spread more easily and efficiently throughout neighboring areas. In an effort to coordinate and maximize benefit from various environmental efforts in the region, EPA programs adhere to the principles of the Environmental Action Program (EAP) for Central and Eastern Europe which was endorsed at the 1993 Environment Ministerial in Lucerne, Switzerland. The EAP is an international framework which advocates the use of risk and cost information to establish environmental policy priorities for CEE countries, and promotes the integration of environmental and economic considerations toward sustainable development. Further details on EPA programs in Central and Eastern Europe can be found in the EPA publication, Dark Past, Bright Future? Central America The Central American integration process has spanned many decades and distinct periods in the history of the region. In recent years, this integration has been interrupted — and the economy of the region has been severely shaken - by natural disasters and by military and political conflicts. But with the end to the wars and the beginnings of political democracies in the region, the governments of Central America are concerned with perfecting democratic institutions and achieving sustainable development as well as an economy that can compete in today's global market. Under the rubric of a 1994 agreement between the Central American Alii ance for Sustainable Development and the U.S. Government (Coordination Conjunta Centroamericana - USA, CONCAUSA), EPA participates in the Central American Regional Environmental Technical Assistance Program. This is one of three components of USAID's Regional Environmental Project for Central America undertaken with the Central American Commission for Environment and Development (CCAD). The central organizing principle for the EPA technical assistance program is the development of compatible systems for environmental legislation, regulations and standards among the seven Central American countries which participate in CCAD. The program's primary goal of this project is to protect public health from the risks posed by environmental contamination. A secondary goal is to promote the conservation of natural resources. Three inter-related project areas comprise EPA's Central America program: regulatory and -43- ------- compliance capacity building; technical assistance; and information and training. A Regional Comparative Risk Assessment study helped to identify priority areas for EPA technical assistance which would complement and implement the EPA legal and regulatory assistance. This assessment, concluded in June 1996, identified three such priority areas: safe pesticide use, solid waste management, and waste water treatment. EPA prepared a strategy for providing technical assistance in these areas through on-site training and demonstration projects at the local/municipal level. By linking locally-based technical assistance training and demonstration activities with on-going national efforts, EPA aimed to improve the legislative and regulatory legal regimes that provide compatible, high levels of environmental protection throughout the region. At the same time, EPA experts on environmental legislation and regulatory enforcement worked with their Central American counterparts at the national level: - To develop national environmental protection laws for each Central American country; - To set common standards for individual media (i.e. pesticides, water, air); and - To develop cost-effective and locally-sustainable training programs for conducting and reviewing environmental impact assessments, multi-media field inspections, and training prosecutors and judges on the enforcement of environmental laws. The principal anticipated results of EPA's work in Central America include the following: - Improved food quality for fresh produce imported from Central America, through best management practices for the importation, use, storage and disposal of agro-chemicals; - Reduction in the inventory of obsolete pesticides which are stockpiled throughout Central America, and which pose a risk to public health and to aquatic ecosystems (in the event of accidental spills or natural disasters); — Replication in other Central American cities of a pilot project in municipal wastewater treatment that is being field tested in Puerto Barrios, Guatemala; - Replication in other Central American cities of a pilot project in integrated solid waste management that will be field tested in Usulutan, El Salvador; - Collaboration with private sector firms on the implementation of a pilot project on cleaner production, including the introduction of economic incentives to encourage improved environmental performance in the manufacturing and food processing industries; Establishment of regional networks of environmental law experts, technical experts, and environmental engineers to facilitate information exchange. -44- ------- Asia Rates of industrial and urban growth in Asia have surpassed growth rates in all other regions of the world during the past three decades. Accompanying these dramatic growth rates, however, has been an accelerated level of environmental degradation, which also ranks as among the world's most rapid. Notably, industrial and urban development in the region is still in its infancy: many Southeast Asian countries have yet to install 80% of the industrial capacity that they will have by the year 2010. EPA, working through the U.S.-Asia Environmental Partnership (US-AEP) at USAID, has been assisting Asian nations in strengthening the environmental management capabilities needed to address this urgent problem. EPA's capacity-building program in the region emphasizes the development and cost- effective implementation of environmental regulatory systems which protect public health and the environment and which promote new trends towards performance-based strategies, the use of market mechanisms, and information-based approaches to informing and engaging civil societies. The program builds capacity in several ways: through training and technical workshops; through the provision of targeted technical assistance, and through the transfer of innovative technologies. Examples of recent projects under the program include: Haze Mitigation - In response to the 1997-98 forest fires in Indonesia, EPA air experts monitored the resulting haze over Indonesia and Malaysia, determined its effects on human health, and helped to develop a set of recommendations for improved monitoring and for . lowering the risk of adverse health effects from haze. Following these recommendations, EPA worked with the Singapore Ministry of the Environment to co-sponsor a regional workshop on fine particulate matter (PM 2.5), a major component of the haze. As a result of this workshop, the Government of the Philippines decided to expand its air monitoring network using U.S. monitoring technologies. Environmental Enforcement in Thailand- EPA trained 300 Thai officials on the principles of environmental enforcement and compliance. As a result of this training effort, the Thai Bangkok Metropolitan Authority is currently developing a municipal enforcement program which will help the city of Bangkok reduce air pollution from mobile sources. Public Communications in Thailand - EPA worked closely with Thai municipal officials, community leaders, university faculty and students from the city of Hat Yai to develop a community-wide environmental campaign. The EPA course "Communications for Environmental Results" served as the basis of the program. Based on the course, the Thais have developed two outreach campaigns thus far: one with the goal of educating the public about the need for increased wastewater treatment facilities and one with the goal of motivating residents to sort their garbage as away of extending the life of the area's sanitary landfill. EPA plans to continue its Asia program in FY 2000, with projects which focus on toxics, -45- ------- urban air quality, waste management and global climate change. In support of the Philippines' new Clean Air Act, EPA will help that country' s government organize a workshop using the recent EP A- USAUD guidance document "Implementer's Guide to Phasing Out Lead in Gasoline" as a text. Vietnam and Indonesia have also expressed interest in working with EPA on lead phase-out projects. In Bangkok, Thailand, EPA will help establish a "Green Fleets" program designed to reduce air emissions from municipal and private fleets of automobiles and trucks. In Taiwan, the Agency will organize a workshop on eco-efficiency benchmarking. This effort will promote unproved environmental performance in selected industry sectors through the use of voluntary self-assessment benchmarking tools. In these and other projects, EPA will seek to leverage the resources and expertise of non-governmental organizations, multilateral development banks and other groups. South Africa In addition to the social and economic legacies of apartheid, there is also an environmental legacy. Generally, the environmental infrastructure built for the minority white population does not extend to the majority black population of South Africa. The formerly centralized system of governance in South Africa did not allow local communities or provinces to develop skills in planning for and managing their environmental problems. Under the auspices of the U.S.- South Africa Binational Commission (often called the "Gore-Mbeki Commission"), EPA is working with the Department of Environment and Tourism (DEAT) as it develops a more decentralized system of protecting human health and the environment. In addition, EPA is working on pilot projects with selected communities in South Africa to develop the capacity to deal with real environmental problems at the local level. As DEAT reshapes itself to more appropriately reflect the demographics of South Africa and as it begins to delegate new environmental authorities to the provincial level, capacity building has become an especially important priority. EPA is working with DEAT to provide training and technical assistance in several areas. For example, EPA is helping DEAT develop a training center which will replicate many of EPA's training modules. Courses currently in the process of being delivered and transferred include Environmental Impact Assessment, Environmental Enforcement, Principles of Solid Waste Management, and Principles ofPollution Prevention. Intensive study tours to the U.S. have been organized on Mining Waste Management and Environmental Impact Assessment. Over the past two years, DEAT has emphasized the development and implementation of a national waste management strategy. EPA waste experts have provided technical analysis of this strategy and have made recommendations on the implementation of this national program. EPA helped organize an international Pollutant Release and Transfer Register (PRTR) workshop for South Africa which resulted hi South Africa committing to providing public access to environmental information through the development of a PRTR. On the local level, EPA is working with several target communities as they organize to address environmental issues. For example, EPA worked with the township of Duncan Village and the larger East London community in developing a grassroots Recycling Forum which had the aims of dealing with a growing waste problem and creating local employment. EPA experts and the U.S. Environmental Training Institute provided initial training for the program. The National Association -46- ------- of State Development Agencies provided U.S. expertise to work with local institutions to complete a waste characterization and market analysis for the community. The Recycling Forum has now established itself as a community action group which meets to coordinate community-wide waste issues. EPA also developed a modest community grants program to help disadvantaged communities which are organizing around an environmental issue for the first time. These grants help communities cover the basic expenses (e.g., copying and telephone charges) of running an organization. Fledgling organizations are given assistance in setting up bank accounts and developing basic administrative procedures. EPA's Green Communities program is providing tools for the communities to use in incorporating environmental considerations into their development planning. While it is possible to measure the number of people trained and the number of times the Agency has provided technical analysis and recommendations hi South Africa, the real measure of success will be in replication of environmental programs in South Africa and other countries in the region. Environmental outcomes will also depend on the degree to which these programs can be assimilated to South African conditions and implemented. Middle East EPA has been an active participant in the Middle East Peace Process, led for the U.S. by the Department of State. During a regional workshop on environmental management training, held in Jordan in November 1998, EPA helped facilitate discussions among Israeli, Jordanian, and Palestinian officials in developing a cooperative training program hi seven priorities areas: environmental economics, solid waste management, environmental law/regulation/enforcement, agro-ecology, hazardous waste management, land use, and emergency response. EPA delivered its first environmental training course in June 1999 on regional solid waste management. EPA is now developing a hazardous waste management course for expected delivery in late 1999/early 2000, and will participate in Japanese-led course on laws, regulations and enforcement. CAPACITY-BUILDING TOOLS In addition to the geographic- and issue-based capacity-building programs described above, EPA has developed a set of generic tools for use in conducting priority capacity-building programs worldwide. Recognizing the similarities in many environmental challenges facing developing countries and economies-in-transition, these tools combine the Agency's best information and expertise into a single integrated package on the most commonly requested topics, thereby assuring quality control and avoiding duplication of effort. Below is additional information on the Agency's work related to training, information, environmental education, and assistance in securing financing. -47- ------- International Training Modules With enormous demand for environmental training worldwide, EPA's international training modules are generally recognized as among the Agency's most effective tools for international capacity-building. First developed in the 1990s in response to the enormous need for environmental training in Central and Eastern Europe, the modules use a facilitated, train-the-trainer approach to impart information on key environmental management techniques, including risk assessment/management, pollution prevention, environmental impact assessment, enforcement and compliance, risk communication, and management principles related to air, water, chemicals, and waste. The standard module uses a three-step process and a combination of case studies, exercises and other interactive methods to teach the fundamentals of a specific management technique. Drawn from a broad range of organizations within the host country, both public and private, participants typically include representatives from national, state and local governments and universities, trade associations, and other non-governmental organizations. A team of three EPA trainers typically facilitates the first delivery of the module, which consists of a three- to five-day course supported by student and instructor manuals, slides, handouts, and other written manuals. Where possible or necessary, materials are translated and adapted to meet the particular needs of the country or region in which the course is being delivered. A smaller team of EPA trainers returns for the second delivery of the module to assist host country individuals selected during the first delivery in developing their skills as facilitators, and EPA and the newly trained facilitators deliver the course jointly to a new group of participants. Finally, during the third delivery, a single EPA trainer works with the new facilitators in delivering the module to yet another group of targeted individuals. The expectation is that an ever widening pool of trained facilitators will help disseminate the necessary skills throughout the country or region long after EPA's participation has ended. EPA also follows up with host country officials at three-month, six-month and one-year intervals to determine the extent to which skills acquired during the training have been used to achieve environmental objectives in that country. EPA has used the modules to train over 5000 people representing over 20 countries since 1990. The Agency continues to update and add new modules to respond to increased demand. The Agency is seeking to modify the module format to incorporate lessons learned over recent years and, by adding more country- or region-specific case studies, to adapt the modules more specifically to the needs of participants. It has also improved the way it plans and evaluates these training programs. In 1996, for example, a group led by the Office of Federal Activities laid out a protocol for improving the effectiveness of the training module on the "Principles of Environmental Impact Assessment (EIA)."9 Based on a statement of the philosophy underpinning EPA's international engagement on this issue, the protocol lays out generic objectives, desired outcomes, and criteria for -48- ------- success. It includes guidelines on the selection of U.S. trainers and host country participants and provides a checklist of necessary activities and materials. Training by its very nature poses a special challenge to measurement efforts, and there is still considerable work for EPA to do in this area. The Agency can currently quantify the number of individuals who have received the training, either directly from EPA trainers or through EP A-trained facilitators from the host country. These are in fact the measures currently included as annual performance goals and measures under EPA's Strategic Plan. But the Agency cannot accurately measure the quality of the training provided, nor can it gauge - beyond anecdotal information - the full extent to which imparted expertise is being applied on behalf of specific environmental outcomes. EPA might be successful, for example, in providing the highest quality training to officials with the commitment and ability to exact an environmental solution. At the same time, exogenous factors outside of EPA's influence (e.g., political considerations) might make it difficult to establish a linear relationship between EPA's training and environmental results downstream. International Information Dissemination The EPA has a substantial repository of technical information which it makes available to the general public. This repository includes over 7,000 publications which can be viewed on the World Wide Web through the Agency's National Environmental Publications Information System (NEPIS). The electronic versions of these documents are, in turn, part of over 250,000 pages of searchable information available through the Agency's web site. As one component of its international capacity-building efforts, EPA has bundled selections from these holdings, along with some materials from outside of EPA, into environmental information products designed with international users in mind. These products attempt to put an ever-growing supply of environmental information in context, emphasize priority issues (such as cleaner production) and present this information in a format which complements other generic capacity-building tools. With access to Internet and other technologies expanding rapidly throughout the world, EPA is emphasizing the use of electronic media which allow for frequent updates and broad distribution at a low cost. The Technical Information Packages (TTPs), the oldest of the capacity-building information products, began in 1991 as a set of portable libraries, with each TIP consisting of an introductory document and one or more boxes of associated EPA technical publications. The TIPs covered topics such as Small Community Wastewater Systems, Hazardous Waste Management and Air Quality Management, and often constituted the most complete technical libraries available to our partners in economically-developing countries. In 1997, EPA began to update the TIPs, post the revised versions on the EPA web site and create a CD-ROM version for use in countries where Internet access is still unreliable. In addition to the TIPs, EPA offers two key information products for the international community as part of its popular Enviro$en$e web site. The Virtual Sector Guides (developed in cooperation with the U.S.-Asia Environmental Partnership at AID) provide on-line access to technical, policy and programmatic resources on specific industry sectors such as electronics and food processing. Each guide includes chapters on: industry overview, pollution prevention strategies -49- ------- and innovative technologies, environmental management practices, recycling and waste management, and regulations and compliance. Through these guides, EPA is able to provide web-based technology transfer and promote, at the international level, improved environmental performance in key sectors. Also on Enviro$en$e is the International Cleaner Production Cooperative, an resource for cleaner production/pollution prevention (CP/P2) information from around the world. The Cooperative consists of over 40 member web sites from the U.S. and abroad, each site emphasizing CP/P2 information. Through a sophisticated indexing function, the Cooperative allows any user to search across all member sites either through a simple text search or by taking advantage of canned searches based on geographic region, industry sector or topic (such as environmental accounting). Through its ability to pull information from a variety of sites, the Cooperative facilitales the transfer and comparison of pollution prevention technologies and strategies within the international community. Along with on-line resources, EPA continues to produce other significant technical documents for international users. The recent Implementer 's Guide to Phasing Out Lead in Gasoline is intended to support worldwide phaseout of lead in gasoline by providing a checklist and guidance for government officials tasked with developing and implementing a lead phaseout policy, and by assembling the data and resources these officials need in order to carry out their task. Similarly, the 1998 publication Guidance for Landfilling Waste in Economically Developing Countries provides guidance on all aspects of the siting, design and operations of landfills in economically developing countries. Both publications are designed to build the capacity of our partner nations to address threats to human health and the environment. Though we can measure outputs associated with these information products (e.g., number of hard copies distributed; number of additional copies requested; number of page requests on various web sites), measuring outcomes proves challenging. Unlike technical assistance projects, technology demonstrations or training programs, information products are widely available, maybe accessed by anyone with an Internet connection, and exist in the public domain — all of which make then" use and influence difficult to track. International Environmental Education EPA's international environmental education programs are intended to foster a greater sense of awareness and understanding about environmental problems, improve decision-making skills, and disseminate environmental education techniques more broadly. Environmental education, for example, has been an important component of EPA's assistance to the City of Nizhnii Tagil hi Russia, where broad, long-term community support, particularly among young people, is needed to protect public health and the environment. Through cooperative programs with the Institute for Sustainable Communities (ISC), the North American Association for Environmental Education (NAAEE), and other organizations, EPA is using technical assistance, training, and information dissemination to support the development of -50- ------- environmental education programs in Central and Eastern Europe, Central and South America, and, more recently, in parts of Africa. Under the "Blue Thumb" program in Poland, for example, EPA worked with Polish authorities in spreading the lessons of environmental stewardship-and water protection from a pilot project in Krakow to thousands of school children in many cities across Poland and Ukraine. Many of the materials originally developed for use in Central and Eastern Europe are now being translated into Spanish for use in Latin America. The NAAEE has taken the first steps toward developing a "strategic framework" for establishing goals, obj ectives and strategies along with performance measures. Reflecting challenges similar to those facing the EPA training modules, performance measures currently quantify: the number of professionals incorporating environmental education principles into their work; the level of government funds and in-kind support devoted to environmental education; and the number of participants in workshops and other information exchange activities. EPA has also provided the opportunity for international scientists and students to work in the Agency's research labs as postdoctoral fellows, with EPA staff contributing to the development of tailored training courses. Through an interagency agreement with the Fogarty International Center at the National Institutes of Health and through a cooperative agreement with the International Agency for Research on Cancer, EPA has provided training in toxicology test methods and epidemiology for scientists and students in Egypt, Argentina, Mexico, China, Africa and Brazil. -51- ------- -52- ------- APPENDIX C: ACKNOWLEDGMENTS The task force co-chairs would like to thank Dennis Cunningham for coordinating the work of the task force: organizing meetings, drafting large portions of the report, and overseeing the overall review process. The co-chairs of the report would also like to acknowledge the following individuals who, though not serving as formal members of the task force, contributed to the report: Stephanie Adrian, Paulo Almeida, Marianne Bailey, Carolyn Barley, Inga Bamett, Paul Cough, Thomas DeKay, Martin Dieu, Jon Grand, Dona Harris, Sarita Hoyt, Mark Kasman, Al Korgi, Harley Laing, Cristina Mercuric, Ted MacDonald, Brian Muehling, Lee Pasarew, Sarah Sowell, Pam Teel, Cheryl Wassennan, and Bryan Wood-Thomas. -53- ------- APPENDIX D: REFERENCE MATERIALS Commission for Environmental Cooperation (U.S.-Mexico-Canada), Assessing Environmental Effects of the North American Free Trade Agreement: An Analytic Framework (Phase II) and Issue Studies, ISBN 2-922305-27-9, Montreal, Canada, 1999 Commission for Environmental Cooperation (U.S.-Mexico-Canada), "Sound Management of Chemicals: Capacity Building Strategic Plan 1999-2001 (Draft 2)," Report of the Capacity Building Task Force, April 28,1999 Environmental Technologies Trade Advisory Committee (ETTAC), Recommendations Report, Washington, DC, June 1996 French Institute for the Environment (IFEN), Environmental Performance Indicators in France, 1996-1997 Organization for Economic Cooperation and Development (OECD), Environmental Indicators, Paris, France, 1994 Organization for Economic Cooperation and Development (OECD), Environmental Performance in OECD Countries: Progress in the 1990s, Paris, France, 1996 Organization for Economic Cooperation and Development (OECD), OECD Environmental Data: Compendium 1997, Paris, France, 1997 United Nations Conference on Environment and Development, Agenda 21, Chapter 37, Rio de Janeiro, 1992 U.S. Agency for International Development (USAID), "Measuring the Environmental Transition in Eastern Europe and the Newly Independent States," Bureau for Europe and Newly Independent States, Prepared by the Harvard Institute for International Development and the International Resources Group, Washington, DC, January 1,1998 U.S. Agency for International Development, "Performance Monitoring and Evaluation TIPs," Center for Development Information and Evaluation, Numbers 1-12,1996 U.S. Congress, Office of Technology Assessment (OTA), Environmental Policy Tools: A User's Guide, OTA-ENV-634, U.S. Government Printing Office, Washington, DC, September 1995 -55- ------- U.S. Environmental Protection Agency, Dark Past, Bright Future: Environmental Cooperation in Central and Eastern Europe and the New Independent States, Office of International Activities, EPA/160-K-98-002, Washington, DC, June 1998 U.S. Environmental Protection Agency, Emerging Global Environmental Issues, Office of International Activities, EPA/160-K-97-001, Washington, DC, January 1997 U.S. Environmental Protection Agency, Environmental Training Modules: International Catalogue, Office of International Activities, EPA/160-B-98-001, Washington, DC, January 1999 U.S. Environmental Protection Agency, EPA's International Technology Programs, Office of International Activities, EPA/160-R-96-002, Washington, DC, September 1996 U.S. Environmental Protection Agency, EPA Strategic Plan, Office of the Chief Financial Officer, EPA/190-R-97-002, Washington, DC, September 1997 U.S. Environmental Protection Agency, EPA Strategy for Promoting U.S. Environmental Exports: A Report to Congress, Office of International Activities, Washington, DC, October 1998 U.S. Environmental Protection Agency, Implementer's Guide to Phasing Out Lead in Gasoline, Office of Technology Cooperation and Assistance, Office of International Activities, EPA/160-B-99- 001, Washington, DC, March 1999 U.S. Environmental Protection Agency, Partners in Capacity-Building for Environmental Impact Assessment: Protocol for Capacity Building Engagements, Office of Enforcement and Compliance Assurance, Washington, DC, July 11,1997 U.S. Environmental Protection Agency, "Principles for Annual Performance Goals and Annual Performance Measures (Draft)," Office of the Chief Financial Officer, Washington, DC, July 20, 1998 U.S. Environmental Protection Agency, Providing Solutions for a Better Tomorrow: A Progress Report on U.S. EPA's Drinking Water Treatment Technology Demonstrations in Ecuador, Mexico and China, Office of Research and Development, EPA/600-F-98-008, Cincinnati, Ohio, October 1998 U.S. Environmental Protection Agency, Reducing Health Risks Worldwide: EPA's International Lead Risk Reduction Program, Office of International Activities, EPA/160-K-98-001, Washington, DC, June 1998 U.S. Environmental Protection Agency, Strengthening EPA's International Programs: A Report to Congress, Office of International Activities, Washington, DC, March 1997 U.S. General Accounting Office, International Environment: Literature on the Effectiveness of -56- ------- International Environmental Agreements, Staff Study, GAO/RCED-99-148, May 1999 U.S. General Accounting Office, International Environment: U.S. Funding of Environmental Programs and Activities, Report to the Chairman, Committee on Foreign Relations, U.S. Senate, GAO/RCED-96-234, Washington, DC, September 1996 U.S.-Mexico Border XXI Program, United States-Mexico Border Environmental Indicators 1997, EPA/909-R-98-001 U. S. Senate, Departments of Veteran Affairs and Housing and Urban Development and Independent Agencies Appropriations Bill, 1998, 105th Congress, 1st Session, Report 105-175, July 8,1997 World Bank, Assessing Aid: What Works, What Doesn 't, and Why, A World Bank Policy Research Report, Oxford University Press, 1998 -57- ------- -58- ------- APPENDIX E: ACRONYMS USED IN THIS REPORT APGs — annual performance goals APMs — annual performance measures BECC - Border Environment Cooperation Commission BEIF - Border Environmental Infrastructure Fund CAMEO — Computer-Aided Management of Emergency Operations CCAD - Central American Commission for Environment and Development CEE - Central and Eastern Europe CEPPO - Chemical Emergency Preparedness and Prevention Office CFC - chlorofluorocarbon DEAT - Department of Environment and Tourism EA — environmental assessments EAP - Environmental Action Program EIA - environmental impact assessment FNSI - finding of no significant impact FY - fiscal year GHG - greenhouse gas GIS — geographic information systems GPRA - Government Performance and Results Act IBEP - Integrated Border Environmental Plan IBWC - International Boundary and Water Commission ISC - Institute for Sustainable Communities JAC - Joint Advisory Committee NAAEE - North American Association for Environmental Education NACEC — North American Commission on Environmental Cooperation NADBank - North American Development Bank NAFTA - North American Free Trade Agreement NEPA - National Environmental Policy Act NGOs - non-governmental organizations OECD - Organization for Economic Cooperation & Development OIA - Office of International Activities ORD - Office of Research and Development PCBs - polychlorinated biphenyls PEIA - Principles of Environmental Impact Assessment POPs - persistent organic pollutants PREIA - Principles for Review of Environmental Impact Assessment REC - Regional Environmental Center SEED - Support for Eastern European Democracy [Act] -59- ------- SMOC - Sound Management of Chemicals TIPs - Technical Information Packages UNEP - United Nations Environment Program UNIDO - United Nations Industrial Development Organization USAEP - United States Asia Environmental Partnership USAID - United States Agency for International Development USG - United States Government WHO - World Health Organization -60- ------- ENDNOTES 1. World Bank, Assessing Aid: What Works, What Doesn 't, and Why, Forward by Joseph Stiglitz, A World Bank Policy Research Report, Oxford University Press, 1998, Page ix 2. U.S. General Accounting Office, International Environment: U.S. Funding of Environmental Programs and Activities, Report to the Chairman, Committee on Foreign Relations, U.S. Senate, GAO/RCED-96-234, Washington, DC, September 1996, Page 22 3. U.S. Senate, Departments of Veteran Affairs and Housing and Urban Development and Independent Agencies Appropriations Bill, 1998,105th Congress, 1st Session, Report 105- 175, July 8,1997, Page 60 4. These findings are consistent with other analyses of international capacity-building programs. One of the most influential reports in this field is the 1998 World Bank Policy Research Report entitled "Assessing Aid." While the World Bank's loan programs include a much higher ratio of infrastructure projects as compared to capacity-building efforts, and while their efforts are broad enough to facilitate country-wide economic and social change, the factors behind "success" were found to be similar to EPA's. The task force also consulted studies of similar programs by the Organization for Economic Cooperation and Development (OECD) and USAID. 5. U.S.-Mexico Border XXI Program, Indicators 1997, Page 6 6. U.S.-Mexico Border XXI Program, Indicators 1997, Page 26 7. U.S.-Mexico Border XXI Program, Indicators 1997, Page 32 8. U.S. Environmental Protection Agency, Dark Past, Bright Future: Environmental Cooperation in Central and Eastern Europe and the New Independent States, Office of International Activities, EPA/160-K-98-002, Washington, DC, June 1998 9. U.S. Environmental Protection Agency, Partners in Capacity-Building for Environmental Impact Assessment: Protocol for Capacity Building Engagements, Office of Federal Activities, Office of Enforcement and Compliance Assurance, Washington, DC, July 11, 1997 ------- |