"
OFFICE OF INSPECTOR GENERAL
Evaluation Report
EPA Needs a Better Strategy
to Measure Changes in the
Security of the Nation's Water
Infrastructure
Report No. 2003-M-00016
September 11,2003
-------
Report Contributors:
Abbreviations
DHS
EPA
ISAC
VA
Erin Barnes-Weaver
Eric Hanger
Jeffrey Harris
Fredrick Light
Ricardo Martinez
Erin Mastrangelo
Department of Homeland Security
Environmental Protection Agency
Information Sharing and Analysis Center
Vulnerability Assessment
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
THE INSPECTOR GENERAL
MEMORANDUM
SUBJECT: Final Evaluation Report: EPA Needs a Better Strategy to Measure Changes in the
Security of the Nation's Water Infrastructure
Report No. 2003-M-00016
FROM: NikkiL.Tinsley U.UU,
TO: G Tracy Mehan, III *
Assistant Administrator for Office of Water
As part of our ongoing evaluation of Environmental Protection Agency (EPA) activities to
enhance the security of the Nation's water supply, we noted an issue that requires your attention.
Specifically, we suggest that EPA develop specific measurable performance indicators of water
security activities. We propose this action because, during our preliminary research,1 we
obtained information that indicates EPA has neither;
• Articulated measurable goals for EPA's water security efforts; nor
• Obtained or analyzed data to develop a baseline for water security.
It is important for EPA to develop measures to monitor the security of our Nation's water supply
and to ensure Federal funds are not spent without clear goals or expectations. To effectively
perform its lead agency responsibilities, EPA needs to collect and analyze data that depicts the
changes in security levels at water utilities.
Our observations and suggestions are based on information obtained from our interviews with
water security experts, water utility officials, and EPA headquarters and regional representatives;
attendance at water vulnerability assessment training; and a review of EPA's Strategic Plan for
Homeland Security, dated September 2002. We are performing our evaluation in accordance
with Government Auditing Standards, issued by the Comptroller General of the United States.
EPA Designated as Lead Agency for Water Security
The Nation's water supply is one of our most vital natural resources. Potential threats to this
resource include contamination with biological, chemical, or radiological agents, or destruction
of physical infrastructure. The water supply is also dependent on other critical infrastructures,
'The EPA Office of Inspector General is conducting preliminary research on an evaluation of water system
security activities in support of the Agency's Strategic Plan for Homeland Security.
-------
such as energy and transportation. Presidential Decision Directive 63, issued in May 1998,
designated EPA as the lead agency responsible for the water sector and for accomplishing the
following functions:
• Establish and maintain channels of communication with all private and public entities having
an infrastructure assurance interest in the sector;
• Facilitate Ihe selection of a S ector Infrastructure Assurance Coordinator;
• Assist the Sector Coordinator in establishing and operating an effective information-sharing
program;
• Draft new legislation and regulations, as required, and propose the use of Federal incentives
to facilitate private investment in assurance programs, if appropriate;
* Promote infrastructure assurance education and training, to include advocating use of best
practices, within the sector;
• Assist in developing plans for prevention (long-term reduction of vulnerabilities and short-
term defensive actions), mitigation, restoration, and reconstitution; and
• Coordinate, in support of the Federal Response Plan, as amended, management of the
consequences of a successful infrastructure attack and prepare for various contingent attacks
through participation in training and exercise programs.
EPA's Efforts to Improve Water Security
To better execute its responsibilities after the terrorist attacks on September 11, 2001, EPA
developed a strategy for improving the security of water utilities. EPA's Strategic Plan focuses
on preparedness and prevention, assisting those responsible for critical infrastructures in
assessing and reducing vulnerabilities and maximizing their response capabilities. Also, EPA
intends to develop technologies to improve the Nation's critical infrastructure and key
responders' abilities to detect and monitor environmental threats. Through this work, EPA plans
to "significantly improve the Nation's overall capacity to protect critical infrastructure from
terrorist attacks." EPA will rely on relationships with water utilities, water-related governmental
entities, and associations to assist utilities, and the Agency has already taken the following steps:
• Facilitated the development of vulnerability assessment methodology and training;
• Provided threat guidance to utilities to help them conduct vulnerability assessments and
identify possible threats to critical assets;
• Provided financial assistance to large drinking water systems to conduct one-time
vulnerability assessments;
• Funded research on technology development and verification; and
• Facilitated the development of a secure Information Sharing and Analysis Center for the
water utility sector (Water-ISAC) to exchange threat/incident information.
EPA's Strategic Plan is organized into four mission-critical areas, the first being "Critical
Infrastructure Protection." This area states that EPA will work with the States, tribes, drinking
water utilities, and other partners to enhance the security of water utilities. The Strategic Plan
articulates both the tactics to execute the Plan as well as the anticipated results. The following
table illustrates selected tactics and results.
-------
iiiii^
Technical Assistance and Grants
EPA will provide tools, training, and technical assistance
to assist water utilities in conducting vulnerability
assessments, implementing security improvements, and
effectively responding to terrorist events.
EPA provided grants to large drinking water utilities for
vulnerability assessments, security enhancement
designs, and/or emergency response plans.
By the end of fiscal 2003, all water utility
managers wil! have access to basic
information to understand potential water
threats, and basic tools to identify security
needs. By 2005, unacceptable security
risks at water utilities across the country
will be significantly reduced through
completion of appropriate vulnerability
assessments, design of security
enhancement plans, development of
emergency response plans, and
implementation of security enhancements.
Terrorism Methods and Prevention Techniques
EPA will work with the Department of Homeland Security
(DHS), other Federal agencies, universities, and the
private sector to:
- Solicit and review methods to prevent, detect, and
respond to chemical, biological, and radiological
contaminants that could be intentionally introduced in
drinking water systems;
- Review methods and means by which terrorists could
disrupt the supply of safe drinking water or take other
intentional actions against water collection, pretreatment,
treatment, storage, and distribution facilities; and
- Review methods and means by which alternative
supplies of drinking water could be provided in the event
of a disruption.
Starting in fiscal 2003, water utilities, key
response agencies, and policymakers will
have improved information and knowledge
to make timely and effective analytical and
technological decisions to enhance
security, detect contamination, and
respond to incidents.
Security Practices
EPA will work to implement water security practices in
ongoing water utility operations. EPA will also work to
build security concerns into ongoing review systems
(e.g., sanitary survey, capacity development, operator
certification, and treatment optimization program for
drinking water systems).
Beginning in fiscal 2003, water utilities will
incorporate security measures as a
standard aspect of day-to-day operations
and EPA, States, and tribes will review
security measures at water utilities on a
continuous basis. Through ongoing
practice and review, water utility managers
and employees will optimize security
measures.
-------
ifactics;!
Communication with Utilities
EPA will work with other government agencies, utility
organizations, and water utilities to establish formal
communication mechanisms to facilitate the timely and
effective exchange of information on water utility security
threats and incidents.
Starting in fiscal 2003, water utilities, law
enforcement agencies, and State and
Federal response and prevention
programs will have timely and accurate
security threat information and incident
analysis to make effective decisions for
water security preparedness and
response.
Coordination with First Responders
EPA will work in coordination with DHS to foster
coordination among Federal, State, tribal, and local
emergency responders, health agencies, environmental
and health laboratories, the medical community, and the
law enforcement community at all levels (Federal, State,
and local) concerning response to potential terrorist
actions against water utilities. This will be achieved
through training and support of simulations and
emergency response exercises.
In the majority of water security incident
responses and exercises, the decision-
making and communication structures of
response agencies will function smoothly
(without critical errors).
Coordination with Other Critical Infrastructures
EPA will work with other critical infrastructure sectors to
further understand and reduce the impact to water
utilities of terrorist attacks on related infrastructures as
well as the impacts of attacks on water utilities to other
critical infrastructures.
Water sector vulnerabilities and impacts
resulting from attacks on other critical
infrastructure sectors will be reduced and
vice versa.
EPA Has Not Articulated Measurable Goals for Water Security
The Office of Water has not outlined how resources, activities, and outputs will achieve the
water security program's goals. EPA's Strategic Plan lacks fundamental components, such as
measurable performance results and information and analysis, to ensure the greatest practicable
reductions in risks to the critical water sector infrastructure. We based our observations on key
program management practices consistent with the President's Management Agenda and the
Government Performance and Results Act.2
T"he EPA Office of Inspector General has compiled these program management principles in Assessing
Organizational Systems: A User's Guide, OA/OPE-5, November 5, 2002. The Guide identifies seven areas
requiring management attention for a successful program. The seven areas consist of Leadership, Strategic Planning,
Customer/Stakeholder and Market Focus, Information and Analysis, Human Capital, Process Management, and
Performance Results.
-------
Because EPA lacks the indicators that define the baseline for water security, EPA cannot monitor
program performance against goals. Without this baseline, EPA cannot determine whether its
strategy resulted in improved water security. Officials we interviewed at the Office of
Management and Budget and DHS also endorsed the need for EPA to establish performance
indicators to determine the effectiveness of its water security activities.
In the absence of specific measurable water security goals, the Agency has focused on
compliance with legislative requirements. For example, EPA states, "By 2005, unacceptable
security risks at water utilities across the country will be significantly reduced through
completion of appropriate vulnerability assessments; design of security enhancement plans;
development of emergency response plans; and implementation of security enhancements."
The completion of these documents, however, does not equate to the outcome of reducing
unacceptable security risks.
EPA Needs to Obtain and Analyze Information to Measure Changes in Security
For EPA to develop performance indicators that measure changes in security, EPA needs to
collect and analyze information from water utilities. However, the Agency believes that it:
(1) lacks the authority to ask for information directly from water utilities or utilize information in
the Water-ISAC that would show changes in security levels, and (2) cannot analyze information
in vulnerability assessments because that would violate the Public Health Security and
Bioterrorism Preparedness and Response Act (Bioterrorism Act).
However, we believe that EPA does in fact have the authority, as well as the responsibility, to
collect and analyze necessary information from these sources. Information obtained
anonymously, through the Water-ISAC, or through coordination with DHS, would provide EPA
with data to benchmark changes in security levels. EPA needs to review the vulnerability
assessments to identify and prioritize threats to water utilities. Subsection 6(A) of the
Bioterrorism Act appears to contemplate review and analysis of vulnerability assessments by
duly authorized Agency officials. Further, additional support for vulnerability assessment
analysis is found in the legislative history of the Bioterrorism Act. Specifically, congressional
members showed bipartisan support for EPA's review of the vulnerability assessments to
develop plans to protect drinking water supplies, and noted the Agency's discretion to review the
assessments and make recommendations to improve water security.3
• Representative Frank Pall one, Jr. (D-NJ) stated that EPA's review of the findings of the
vulnerability assessments would "help the government understand the threats to our water
systems and develop plans to protect our safe drinking water supply."
• Representative Michael Bilirakis (R-FL) stated that EPA could use the assessments
"to address the threat of terrorism and for any other lawful purpose."
Floor Action on H.R. 3448, the Public Health Security and Bioterrorism Preparedness and Response Act
of 2002, in the Senate on May 23, 2002, and in the House on May 22, 2002.
-------
• Senator James Jeffords (I-VT) stated that "there is not a restriction on EPA's discussing the
content of the assessments with persons who may benefit from information about the security
of our Nation's water supply, such as state and local officials, nor is there restriction intended
by mis bill upon a water system's voluntarily sharing information with other systems,
emergency responders or communities. Our attempt to provide a safeguard against broad
disclosure of sensitive information does not lead us to conclude that our citizens should not
have the information they need to protect and inform themselves."
EPA could also utilize information from DHS to develop performance indicators that illustrate
changes to security at critical infrastructures. An official at DHS stated that the Department is
developing performance indicators for other critical infrastructures, such as the chemical
industry. According to The National Strategy for the Physical Protection of Critical
Infrastructure and Key Assets (February 2003). DHS coordinates with Federal agencies to assess
threats to critical infrastructure and evaluate preparedness. The National Strategy places
responsibility with DHS for gathering threat and vulnerability information. However, EPA has
only recently allowed DHS to review drinking water utilities' vulnerability assessments.
Suggestions
We suggest that EPA:
(1) Develop specific, measurable goals, objectives, and performance indicators for its water
security programs; and
(2) Utilize available sources of information to collect and analyze data to develop a
baseline for water security.
Agency Comments and Office of Inspector General Evaluation
In its response to our draft report, EPA agreed with our assessment that the Agency's Strategic
Plan for Homeland Security lacks clearly defined performance measures for critical water
infrastructure protection activities. EPA indicated that it will be actively involved in developing
more outcome-focused performance indicators in its revised strategic plan.
EPA also agreed with our assessment that it needs to develop a baseline for water security. EPA
said it will analyze a sample of vulnerability assessments to develop the baseline. EPA also
plans to use this information to assist in identifying tools that need to be developed, and to
research priorities and appropriate security enhancements for water utilities. EPA indicated it
will share this analysis with Congress and the Department of Homeland Security.
We commend EPA's commitment to create outcome-focused performance measures and a
baseline to measure and monitor changes in water security. However, the necessary information
required for these initiatives may not reside solely in the water system vulnerability assessments.
As the lead agency for water security, we suggest that EPA partner with the utilities and collect
and analyze information from additional sources such as the Water-ISAC and through
-------
coordination with DHS so that the appropriate actions could be taken to enhance the security of
the Nation's water infrastructure.
The full Agency response is provided in Appendix A.
If you or your staff have any questions regarding this report, please call me at (202) 566-0847 or
Kwai Chan, Assistant Inspector General for Program Evaluation at (202) 566-0828.
-------
Appendix A
Agency Response
August 22, 2003
MEMORANDUM
SUBJECT: Response to OIG Concerns regarding "EPA Needs a Better Strategy to Measure
Changes in Utilities Water Security"
DRAFT: Report No. 2003-M-00016
FROM: G. Tracy Mehan, III /signed/
Assistant Administrator
TO: Jeffrey K. Harris
Director for Program Evaluation, Cross-Media Issues
Office of Inspector General
I am responding to the two principal issues and concerns that were stated in your July 23,
2003, memorandum/report to me. Your preliminary research on performance measures of water
security activities indicates that EPA has neither:
• articulated measurable goals for EPA's water security efforts; nor
• obtained or analyzed data to develop a baseline for water security.
I agree with your assessment that the Agency's Strategic Plan for Homeland Security,
dated September 2002, lacks clearly defined performance measures for critical water
infrastructure protection activities. I believe this document should be considered a blueprint of
near-term (2002 and 2003) homeland security activities rather than a strategic plan in the strict
sense of mat term. Attachment 1 highlights our significant accomplishments under this near-
term plan. I am a strong proponent of EPA's Office of Homeland Security's recently-announced
initiative to develop a revised strategic plan. This exercise may serve as the opportunity to
identify longer-term goals, objectives, and measurable performance indicators for all EPA
programs involved in high priority activities to protect public health and the environment from
terrorist and other intentional acts. I assure you that my staff will be very active participants in
this endeavor and will devote significant attention to the challenge of developing outcome-
focused performance measures. For the FY 2005-2010 Agency-wide strategic plan, I assigned
my Senior Advisor for Water Policy to work with all water programs to shift from a
predominance of output-oriented performance measures to ones that will demonstrate more direct
results and benefits to public health and environmental protection. We made considerable
-------
progress and will continue to make improvements in all water programs including critical water
infrastructure protection.
In addition to the limited performance measures in the 2002 Strategic Plan for Homeland
Security, the Agency's FYs 03 and 04 annual performance plans have contained annual
performance goals and measures. (Attachment 2) Because these measures are indeed outputs not
indicators of outcomes, I and my staff would welcome any specific recommendations and direct
assistance from you as we proceed to formulate water security program goals and appropriate
performance measures.
With respect to your second issue/concern on obtaining and analyzing data to develop a
baseline for water security, I want to clarify the Office of Water's position on examining
vulnerability assessments (VAs) submitted by some 9,000 community water systems as required
by the Bioterrorism Act of 2002. I personally identified to the few members of the Water
Protection Task Force who have been designated by me to have access to the VAs three specific
purposes for reviewing and analyzing them. The first is that all VAs should be reviewed to
determine both compliance with the statutory requirements for submission/ certification of
completion and necessary enforcement action. Second, a representative sample (based on the
total number of systems in each size of systems as stipulated in the statute) of VAs should be
examined for: a) compliance (and subsequent enforcement, if any) with the statutory requirement
that the VA address all applicable parts of a system (e.g., pipes, physical barriers, treatment, etc.)
and b) determination that drinking water systems used a "reasoned process" (e.g., a tool like
RAM-W) to evaluate their vulnerabilities. Third, aggregated data are to be compiled and
analyzed to assist in identifying tools that need to be developed, research priorities and
appropriate security enhancements. In addition, this aggregated information will help us develop
a baseline for water security, which is consistent with your suggestion as well as OW's emphasis
on improving performance indicators and measures for all its programs. Besides using this
aggregated data to inform and guide our future actions, we intend to share this information
(mindful of the restrictions imposed by the Bioterrorism Act of 2002) with Congress and other
decision makers, e.g., the Department of Homeland Security.
I appreciate the opportunity to respond to your draft report. Should you have any
questions or need additional information, please contact Michael Mason, the Office of Water's
liaison to OIG, on 564-0572.
Attachments
-------
Attachment 1
ACCOMPLISHMENTS OF THE WATER PROTECTION TASK FORCE
UNDER EPA'S HOMELAND SECURITY STRATEGIC PLAN
7/24/03
EPA has and is continuing to support a number of activities to improve security of both drinking
water and wastewater utilities using approximately $90 million and $23 million appropriated in
Fiscal Years '02 and '03, respectively. Examples of these activities include:
* Developed vulnerability assessment tools for both drinking water and wastewater utilities and
supported extensive training for thousands of utility operators.
* Provided more than $50 million in grants to more than 400 of the Nation's largest drinking
water systems to undertake vulnerability assessments and do related security planning.
Visited 30 of the largest cities to discuss water security. Received 463 of the 466
vulnerability assessments required to be submitted to EPA in March 2003.
* Provided more than $24 million to the States and non-profit organizations to provide training
and technical assistance to small and medium water utilities on vulnerability assessments,
>• Developed and distributed Baseline Threat Informal ion for Vulnerability Assessments of
Community Water Systems.
*• Met statutory deadline to implement protocol to protect vulnerability assessments.
>• Supported establishment of a state-of-the-art, secure information sharing system (the
WaterlSAC) to share up-to-date threat and incident information between the intelligence
community and the water sector. Provided several water-specific advisories.
>• Developed guidelines on what actions utilities should take under DHS-specified threat levels.
*• Developed and began testing/distribution ofRiverspill and Pipeline Net models to determine
fate and transport of contaminants in both source water and drinking water systems.
*• Supported establishment of a water security emphasis for the Environmental Technology
Verification Program and the WATERS test site to evaluate water security technologies.
*• Collaborated with ORD in development of the Water Security Research and Technology
Development Action Plan, currently under Review by the National Academy of Science.
* Developed initial guidance on emergency response notification protocols. Currently
developing detailed guidance on revising emergency response plans to meet Bioterrorism Act
requirements, as well as a protocol to respond to a drinking water contamination event.
*• More information on EPA's Water Protection Task Force water security program can be
found at: www.epa.gov/safewater/sec.uri.ty
10
-------
Attachment 2
Annual Performance Goals and Measures for Critical Water Infrastructure Protection
(Drinking Water and Wastewater Utilities)
"l
Drinking Water
FY03:
Annual Performance Goal
Enhance public health protection by securing the Nation's critical water infrastructure through
support for counter-terrorism preparedness.
Percent of the population and the number of community water systems - - serving 100,000 or
more people - - that have certified the completion of their vulnerability assessment and
submitted a copy to EPA.
Percent of the population and the number of community water systems - - serving 100,000 or
more people - ~ that have certified the completion of the preparation or revision of their
emergency response plan..
FY04; '
Annual Performance Goal
Enhance public health protection by securing the Nation's critical water infrastructure through
support for counter-terrorism preparedness. L
Annual Performance Measures
Percent of the population and the number of community water systems - - serving more than
50,000 but less than 100,000 people ~ - that have certified the completion of their vulnerability
assessment and submitted a copy to EPA.
Percent of the population and the number of community water systems - - serving more than
50,000 but less than 100,000 people - - that have certified the completion of the preparation or
revision of their emergency response plan.
Percent of the population and the number of community water systems - - serving more than
3,300 but less than 50,000 people - - that have certified the completion of their vulnerability
assessment and submitted a copy to EPA.
11
-------
Annual Performance Goals and Measures for Critical Water Infrastructure Protection
Wastewater
FY03;
Annual Performance Goal
Enhance public health and environmental protection by securing the Nation's critical water
infrastructure through support for counter-terrorism preparedness including system operator
training.
Annual Performance Measure
Percent of the population and the number of large and medium size (10,001 and larger) of
Publicly Owned Treatment Works (POTWs) that have been taken for homeland security
preparedness.
03:
Annual Performance Goal
Enhance public health and environmental protection by securing the Nation's critical water
infrastructure through support for counter-terrorism preparedness including system operator
training,
Annual Performance Measure
Percent of the population and the number of large and medium size (10,001 and larger) of
Publicly Owned Treatment Works (POTWs) that have been taken for homeland security
preparedness.
12
-------
Appendix B
Distribution
Acting Administrator
Associate Administrator for Congressional and Intergovernmental Relations
Acting Associate Administrator, Office of Public Affairs
Assistant Administrator, Office of Water
Audit Followup Coordinator, Office of Water
Director, Office of Ground Water and Drinking Water
Chair, Water Protection Task Force
General Counsel
Director, Office of Homeland Security
13
-------
------- |