\ ^EDSX Q pro"^0 O z UJ o T U.S. ENVIRONMENTAL PROTECTION AGENCY OFFICE OF INSPECTOR GENERAL Catalyst for Improving the Environment EPA Has Not Fully Implemented a National Emergency Response Equipment Tracking System Report No. 11-P-0616 September 13, 2011 ------- Report Contributors: Richard Eyermann Mike Davis Jennifer Hutkoff Heather Layne Gul Sharma Abbreviations CPIC Capital Planning and Investment Control EMP Emergency Management Portal EPA U.S. Environmental Protection Agency ERT Environmental Response Team NDT National Decontamination Team NEMS National Equipment Management System OEM Office of Emergency Management OIG Office of Inspector General OMB Office of Management and Budget OSWER Office of Solid Waste and Emergency Response SLCM System Life Cycle Management Hotline To report fraud, waste, or abuse, contact us through one of the following methods: e-mail: OIG Hotline@epa.gov write: EPA Inspector General Hotline phone: 1-888-546-8740 1200 Pennsylvania Avenue NW fax: 703-347-8330 Mailcode 8431P (Room N-4330) online: http://www.epa.qov/oiq/hotline.htm Washington, DC 20460 ------- £ 33 \ PRO^° o in (J T U.S. Environmental Protection Agency Office of Inspector General At a Glance 11-P-0616 September 13, 2011 Why We Did This Review We initiated this audit based on a Hotline complaint related to the Emergency Management Portal (EMP) equipment tracking module. Our objectives were to determine the extent to which the U.S. Environmental Protection Agency (EPA) implemented the EMP equipment tracking software, what efforts EPA has made to assess functionality and cost effectiveness, and how the EMP equipment module compared to the previous interim system. Background Since September 11, 2001, EPA's emergency response focus has expanded to better coincide with its new role in homeland security. In May 2002, EPA determined that it needed to create a national equipment tracking system to be better prepared for terrorist acts and nationally significant incidents. For further information, contact our Office of Congressional, Public Affairs and Management at (202) 566-2391. The full report is at: www.epa.qov/oiq/reports/2011/ 20110913-11-P-0616.pdf Catalyst for Improving the Environment EPA Has Not Fully Implemented a National Emergency Response Equipment Tracking System What We Found Although EPA spent $2.8 million as of October 2010 to develop and implement an EMP emergency equipment tracking module, EPA has not fully implemented the module, and the module suffers from operational issues. Our review of allegations in a Hotline complaint found that: EPA does not fully use the EMP equipment tracking module because no EPA office with overall authority has mandated its use. EPA has made no formal effort to assess functionality and cost effectiveness due to its decision to perform such assessments only after fully implementing the EMP equipment module. The EMP equipment module is cumbersome and slow, and may not be the most efficient and effective emergency equipment tracking alternative. EPA has guidance and policies that require the Agency to develop and implement a plan for a national equipment tracking system. Both the Office of Management and Budget and EPA require performance measurement of such systems. However, EPA has not fulfilled this requirement. In addition to the $2.8 million it has already spent, EPA plans to spend another $5.5 million over the next 15 years on the EMP equipment module's maintenance. Further, the regions that are using the module continue to maintain their own tracking systems, resulting in wasted resources. Because EPA has not fully implemented the EMP equipment module and the module is cumbersome and slow, EPA's ability to protect public health and the environment in the event of a nationally significant incident may be impaired. What We Recommend We recommend that the Assistant Administrator for Solid Waste and Emergency Response ensure that only essential equipment tracking data are required to be recorded and determine whether the EMP equipment module is the most cost- efficient alternative. We also recommend that the EPA Deputy Administrator mandate that regions and emergency response teams employ the national tracking system that EPA decides to use for emergency response equipment. The Agency concurred with the findings and recommendations, but did not provide a corrective action date for the first recommendation. The Assistant Administrator for Solid Waste and Emergency Response will hire an outside contractor to conduct an alternative analysis to determine the most efficient and effective national emergency response equipment tracking alternative. The Deputy Administrator also plans to issue a memo requiring the use of the EMP equipment module for tracking equipment. ------- ^tDsrx Q v S S UNITED STATES ENVIRONMENTAL PROTECTION AGENCY % VK# * WASHINGTON, D.C. 20460 THE INSPECTOR GENERAL September 13, 2011 MEMORANDUM SUBJECT: EPA Has Not Fully Implemented a National Emergency Response Equipment Tracking System Report No. ll-P-0616 FROM: Arthur A. Elkins, Jr. Inspector General TO: Bob Perciasepe Deputy Administrator Mathy Stanislaus Assistant Administrator for Solid Waste and Emergency Response This is our report on the subject audit conducted by the Office of Inspector General (OIG) of the U.S. Environmental Protection Agency (EPA). This report contains findings that describe the problems the OIG has identified and corrective actions the OIG recommends. This report represents the opinion of the OIG and does not necessarily represent the final EPA position. Final determinations on matters in this report will be made by EPA managers in accordance with established audit resolution procedures. The estimated direct labor and travel costs for this report are $197,352. Action Required In accordance with EPA Manual 2750, you are required to provide a written response to this report within 90 calendar days. You should include a corrective action plan for agreed-upon actions, including milestone dates. Your response will be posted on the OIG's public website, along with our memorandum commenting on your response. Your response should be provided as an Adobe PDF file that complies with the accessibility requirements of Section 508 of the Rehabilitation Act of 1973, as amended. The final response should not contain data that you do not want to be released to the public; if your response contains such data, you should identify the data for redaction or removal. We have no objections to the further release of this report to the public. We will post this report to our website at http://www.epa.gov/oig. ------- If you or your staff have any questions regarding this report, please contact Melissa Heist, Assistant Inspector General for Audit, at (202) 566-0899 or heist.melissa@epa.gov; or Richard Eyermann at (202) 566-0565 or evermann.richard@epa.gov. ------- EPA Has Not Fully Implemented a National Emergency Response Equipment Tracking System 11-P-0616 Table of Contents Purpose 1 Background 1 National Equipment Management System 1 Emergency Management Portal 2 Scope and Methodology 2 Prior OIG Audits 3 Results of Review 4 EPA Directives and Policy Require Implementation and Formal Review of a National Emergency Equipment Tracking System 4 EPA Has Not Performed Formal Functionality and Cost Effectiveness Assessments of the EMP Equipment Module 5 Equipment Module Not Fully Used Because No EPA Office With Overall Authority Has Mandated Use 6 The EMP Equipment Module Is Cumbersome and Slow Compared With NEMS 8 Conclusion 9 Recommendations 10 Agency Comments and OIG Evaluation 10 Status of Recommendations and Potential Monetary Benefits 12 Appendices A EPA Response to Draft Report 13 B Distribution 17 ------- Purpose The U.S. Environmental Protection Agency (EPA), Office of Inspector General (OIG), initiated this audit based on a Hotline complaint. The Hotline complaint alleged that the regions and environmental response and radiological emergency response teams were not using the Emergency Management Portal (EMP) equipment module, and that the module was ineffective and costly. The Hotline complaint also alleged that a significantly cheaper, previously developed interim system, the National Equipment Management System (NEMS), was able to accomplish the same overall objective of nationally tracking emergency response equipment. Finally, the Hotline complaint alleged that, as of February 2010, contractor costs in excess of $8 million had been expended for the replacement EMP system, but that EPA had not fully implemented it. Based on the allegations, our objectives were to answer the following questions: To what extent has EPA implemented the EMP software for tracking emergency response equipment? What efforts has EPA made to assess the functionality and cost effectiveness of its EMP system for emergency response equipment? How does the cost, efficiency, and effectiveness of the EMP equipment tracking capability compare to NEMS? Background Since September 11, 2001, EPA's counterterrorism and emergency response responsibilities expanded to better coincide with its new role in homeland security. EPA determined, in part due to these activities, that to be better prepared for terrorist acts and nationally significant incidents, it needed to purchase more emergency response equipment, establish maintenance contracts, and create a national equipment tracking system. Implementation of a national equipment tracking system has been a long-standing gap in EPA's emergency support capabilities. The Office of Solid Waste and Emergency Response's (OSWER's) May 2002 60-Day Task Force Report on the EPA's Emergency Response Contracting Network initially identified the need to track emergency response equipment on a national level. National Equipment Management System EPA's Edison, New Jersey, Environmental Response Team (ERT) developed NEMS in 2005 at a cost of $300,000. NEMS was an interim system that provided a national listing of emergency response equipment. Regions and response teams maintained their own equipment management systems, but uploaded equipment data through a Web-based service to NEMS. NEMS allowed regions and response 11-P-0616 1 ------- teams to identify the availability and location of emergency response equipment stored throughout the nation. EPA never fully implemented NEMS, and it is not currently in use. Emergency Management Portal EPA's Office of Emergency Management (OEM), part of OSWER, designed the EMP to tie together prevention, preparedness, and response information that EPA's emergency management community needs to plan for and act on in emergency situations. OEM began developing the EMP equipment module in 2004 and released it for use in 2007. The EMP equipment module is a comprehensive equipment system that provides warehouse management functions that include equipment identification, location, and availability, along with additional maintenance, repair, and expense records. The EMP also includes removal and emergency response data on site cleanup, personnel readiness (training and experience), and technical information in support of field personnel. OEM developed the EMP equipment module to manage emergency response equipment throughout the Agency. The primary objective of the module is to provide information on the availability and location of emergency response equipment. The module also includes information to assist warehouse managers in managing and recording calibrations, maintenance, and repairs of their equipment. Scope and Methodology We conducted this audit from September 2010 to June 2011 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. We visited OEM at headquarters to obtain an understanding of how it uses the EMP to manage emergency response equipment, conducted interviews with the EMP project manager, and participated in a walk-through demonstration of the EMP equipment module. We determined the extent to which EPA has implemented the EMP equipment module through an analysis of documents and interviews with headquarters, regional, and response team personnel. To determine the extent to which EPA has implemented the module nationwide, we reviewed OEM's September 2010 assessment of regional and response team use of the equipment module, and an OEM PowerPoint presentation provided to us in October 2010. We reviewed the following documents to understand the requirements for assessing the EMP equipment module's functionality and cost effectiveness: 11-P-0616 2 ------- EPA System Life Cycle Management (SLCM) policy and procedure documents EPA's Exhibit 300 Guide for Capital Planning and Investment Control (CPIC) Budget Year 2012 Office of Management and Budget (OMB) Circular A-130 EPA's CPIC for Management of Information Technology Investments To compare the cost, efficiency, and effectiveness of the EMP equipment module to that of the previous tracking system, NEMS, we interviewed personnel in Regions 2, 4, 5, and 6; ERTs in New Jersey and Las Vegas; the Radiological Emergency Response Team in Las Vegas; and the National Decontamination Team (NDT). Regions 4, 5, and 6 had recent major responses to nationally significant incidents in April and July 2010 for which we could evaluate the EMP equipment module. Region 2 had no equipment tracking software system prior to EMP and therefore served as a pure baseline region. We also reviewed the NEMS Systems Management Plan, the EMP Alternatives Analysis, and Exhibit 300: Capital Asset Plan and Business Case Summary for the EMP (part of CPIC). We reviewed fiscal years 2008, 2009, and 2010 Federal Managers Financial Integrity Act Annual Assurance Letters for OSWER and for the Office of Environmental Information to determine whether they identified any weaknesses related to the EMP equipment module. The letters did not identify any such weaknesses. We reviewed internal controls related to the processes for planning and implementing information technology and management systems. We also reviewed controls for measuring, reporting, and monitoring system performance. Our scope was limited to evaluating EPA activities related to the equipment module portion of the EMP. We did not evaluate activities related to development and implementation of the EMP as a whole. Prior OlG Audits OIG reports in 2004, 2006, and 2009 cited EPA's lack of a nationwide tracking system for emergency response equipment: Report No. 2004-P-00011, EPA Needs to Better Manage Counter Terrorism/Emergency Response Equipment, issued March 29, 2004 Report No. 2006-P-00022, EPA Needs to Better Implement Plan for Protecting Critical Infrastructure and Key Resources Used to Respond to Terrorist Attacks and Disasters, issued April 26, 2006 Report No. 09-P-0087, EPA Plans for Managing Counter Terrorism/Emergency Response Equipment and Protecting Critical Assets Not Fully Implemented, issued January 27, 2009 The 2004 report recommended that the Agency establish an aggressive timetable to (1) determine what emergency response equipment and characteristics 11-P-0616 3 ------- (e.g., location and condition) EPA was to track, and (2) develop and implement a plan, with points of accountability, for a national tracking system. The Agency's response to this report set up the initial September 2004 deadline for establishing a national emergency response equipment tacking system. The 2006 and 2009 OIG reports disclosed that EPA had not implemented a national system for tracking emergency response equipment and recommended that EPA implement such a system. Results of Review As of October 2010, EPA had spent $2.8 million on the EMP emergency equipment tracking module, which has not been fully implemented. The system suffers from operational issues. In our review of allegations in the Hotline complaint, we found that: EPA does not fully use the EMP equipment tracking module because no EPA office with overall authority has mandated its use. EPA has made no formal effort to assess functionality and cost effectiveness due to its decision to perform such an assessment only after fully implementing the EMP equipment module. The EMP equipment module is cumbersome and slow and may not be the most efficient and effective emergency equipment tracking alternative. EPA has guidance and policies that require the Agency to develop and implement a plan for a national tracking system, and both OMB and EPA require performance measurement of such systems. However, EPA has not fulfilled these requirements. EPA has spent $2.8 million as of October 2010 on the EMP equipment module, and it plans to spend another $5.5 million over the next 15 years on maintenance. In addition, regions and response teams that are also using the module continue to maintain their own tracking systems, resulting in wasted resources. Because EPA has not fully implemented the EMP equipment module and the module is cumbersome and slow, EPA's ability to protect public health and the environment in the event of a nationally significant incident may be impaired. EPA Directives and Policy Require Implementation and Formal Review of a National Emergency Equipment Tracking System In implementing computer software systems like EMP, EPA offices must follow the SLCM policy and procedure. EPA's SLCM policy promotes effective and efficient solutions for designing and operating information systems. The policy mandates a series of reviews and approvals, and integration with the Agency's information technology security, enterprise architecture, and investment management processes for information technology, while allowing flexibility to accommodate varying developmental approaches. The SLCM policy states that 11-P-0616 4 ------- advancement from one phase to the next entails an Information Technology Investment Management Board review. EPA designates these reviews as Agency- level control gates and management must ensure that they take place. When a control gate review is required, management should not advance a system without documented, written approval resulting from that review. EPA's SLCM procedure mandates the measurement of asset performance and the alignment of costs to projected resources required within cost-benefit analyses. EPA's Exhibit 300 Guide for CPIC FY 2012 quotes OMB's Capital Asset Plan and Business Case Summary Exhibit 300 when it states that agencies should be measuring the performance of assets against the baseline. EPA's SLCM procedure states that EPA should conduct a postimplementation review to ensure that the system functions as planned and to verify that the system cost is within the estimated amount determined by the cost-benefit analysis. The SLCM procedure directs EPA personnel to use technical management practices in the planning, acquisition, operation, maintenance, and termination of information technology systems. The SLCM procedure outlines various documents and planning activities that EPA must conduct during different phases of projects. Two key documents and activities are: The Business CaseThe OMB CPIC submission requires that current business processes be described using activity and data models. Current costs and performance are also associated with the models. The submission must also (1) identify and analyze gaps between current and desired outcomes, and (2) develop and evaluate alternatives for improving the business based on readily available information. Requirements SubphaseThis phase emphasizes determining what functions must be performed rather than how to perform those functions. To do this, the project team defines functional and system requirements that are not easily expressed in data and process models. Functional and system requirements also include the requirements of the business process, the user requirements, and operational requirements. EPA Has Not Performed Formal Functionality and Cost Effectiveness Assessments of the EMP Equipment Module While EPA advised in an October 2008 OSWER memorandum that the equipment module was in production and that the regions and special teams were required to use the equipment module, it has not formally assessed operational functionality or cost effectiveness. The Agency has committed to addressing functionality issues within the EMP equipment module as they arise rather than assessing usability and benefits against the baseline established to measure performance of the equipment module. EPA plans to assess usability and benefits against the baseline only when all regions and response teams are using the EMP equipment module to track emergency response equipment. Thus, EPA has spent 11-P-0616 5 ------- $2.8 million as of October 2010 on the equipment module without formally verifying that the system cost is within the estimated amount determined by the cost-benefit analysis, and without measuring the impact on the success, increased functionality, and use of the module. Equipment Module Not Fully Used Because No EPA Office With Overall Authority Has Mandated Use Despite an initial deadline of September 2004 for implementing a nationwide tracking system, some regions and response teams are not using the EMP equipment module because no EPA office with overall authority has mandated that offices with an emergency response equipment managing role use it. EPA has deployed the national system, but the Agency is not fully using the EMP equipment module nationwide. In October 2008, the Director of OEM issued a memorandum on the EMP equipment portal, stating that the equipment module was in production and ready for use for all EPA equipment warehouses. OEM sent the memorandum to Regional Removal Managers; the Directors of the Office of Radiation and Indoor Air and the Office of Superfund Remediation and Technology Innovation; and the Deputy Director, Office of Criminal Enforcement, Forensics and Training. The memorandum also stated, "All regions and special teams are required to use the new system for tracking equipment that is used in support of emergency response." According to OEM, as of October 2010, 5 of the 10 regions and 3 of the 4 response teams were not fully using EPA's EMP equipment module to track and maintain emergency response equipment (table 1). OEM determined whether regions and response teams were using the equipment module by office participation in Web conferences and training, calls for support, assistance on using the system, and special reports requested. OEM considered an office to be "fully implemented" and using the system if it was using the system to determine availability of equipment. 11-P-0616 6 ------- Table 1: EMP equipment module implementation status by region and response team Region and response team Implementation status Region 3 Region 5 Region 6 Region 7 Region 9 ERT Las Vegas Fully implemented Region 1 Region 8 Region 10 ERT Kentucky Radiological ERT Las Vegas Nearly implemented Region 2 Region 4 ERT New Jersey Not implemented Source: October 2010 OEM PowerPoint presentation. While OEM has required the use of the EMP equipment module, OEM does not have the direct authority to require that regions and response teams use the EMP equipment module. Regions are under the authority of Regional Administrators, and response teams are under the authority of the Office of Superfund Remediation and Technology Innovation. OEM has not elevated the issue of regions and response teams not using the system. Instead, OEM has delegated the implementation of the equipment module to its NDT. NDT's role is to: Train the regional warehouse managers Support warehouse managers in fully utilizing the equipment module Assist the warehouse managers in developing equipment-related reports Assist headquarters in developing and prioritizing new features and fixes for a future version of the equipment module As a subordinate group to OEM, NDT also cannot require compliance by the regions and response teams. NDT staff responsible for the implementation of the equipment module stated that regions and response teams are resistant to change, and regions and response teams found their own tracking systems to be more efficient and easier to use. 11-P-0616 7 ------- Regions and response teams had various reasons for not using the EMP equipment module: They are comfortable working with the warehouse systems they currently have. The EMP equipment module is not easy to use, and it is slow due to additional required fields that users must complete. The EMP equipment module is missing functions, such as the capability to load large amounts of data simultaneously. The EMP Equipment Module Is Cumbersome and Slow Compared With NEMS EMP equipment module user feedback casts doubt on it being the most efficient and effective emergency equipment tracking alternative. The EMP equipment module replaced the interim system, NEMS, because OEM determined that the Agency needed a more robust national equipment management system based out of headquarters. However, NEMS met the emergency equipment tracking needs of the regions according to the New Jersey ERT and Regions 4 and 6. NEMS provided a national view of equipment, including identification, location, and availability. NEMS collected data from regional and response team systems to display each region's individual categorization of equipment and equipment location. EPA did not intend for NEMS to replace local systems. EPA designed the EMP equipment module to collect data from user input or local systems to track equipment and compile additional information such as cost and schedule of equipment purchases and maintenance actions. Some regional and response team personnel have deemed the EMP equipment module to be "cumbersome" and slow because of the number of extra fields that users are required to fill out to support the module's increased functionality. Regional and response team personnel stated that some EMP equipment module functionality is beyond what a national tracking system needs, and that the functionality is not an improvement on previous regional warehouse management systems. The number of extra fields that users are required to fill out to support the module's increased functionality adversely affects the EMP equipment module's efficiency. Specific comments include the following: Region 4 staff stated that they do not use the EMP equipment module because it is very slow. They tested the system by checking out (assigning equipment to a first responder in the system) a piece of equipment both in the EMP equipment module and in their regional system. To check out the same piece of equipment, the EMP equipment module took 48 minutes, and Region 4's system took 11 minutes. Region 4 staff also stated that they need basic equipment tracking information, which is the location and the availability of equipment on a 11-P-0616 8 ------- nationwide basis, but nationally they do not need to know information on other regions' maintenance of equipment and information on parts and supplies. This information is not important in an emergency. Region 5 staff stated that the EMP required that they enter more data into the system (into the fields), so it takes more time and effort to complete functions (e.g., entering costs). Region 6 staff said that the EMP equipment module was more cumbersome than the region's previous system. Using the EMP required 20-30 percent more effort (due to the extensive number of fields in EMP that must be filled out), but had mostly the same functionality as the region's Response Manager System. OEM created the check-in/check-out part of the module with many extraneous features and steps. NDT staff provided us with information compiled by Region 7. Region 7 performed a comparison of the EMP equipment module to its own system and determined that it took 5 minutes and 9 seconds to perform multiple tasks including: check back in, inventory event, and in-house repair tasks in its own system, compared to 16 minutes and 2 seconds in the EMP equipment module. In a New Jersey ERT contractor comparison of speed in getting equipment through the EMP equipment module versus the legacy system, the total check-out and check-in time with the legacy system was 13 minutes, compared to 54 minutes with the EMP equipment module. ERT-Las Vegas staff stated that if they used the EMP, they would duplicate some of their efforts. For example, their maintenance records are already located in their own maintenance log books. Conclusion EPA has implemented neither the EMP equipment module nor NEMS as EPA's national equipment tracking system. The development costs of the EMP equipment module were significantly higher than the costs of NEMS. The cost of developing the EMP equipment module accounted for $2.2 million of the $2.8 million spent on the module through October 2010. Estimated development costs for NEMS were $300,000. Therefore, EPA has spent millions of dollars on a system that the regions and response teams are not fully using as intended, and plans to spend $5.5 million more on maintenance over the next 15 years. 11-P-0616 9 ------- Recommendations We recommend that the Assistant Administrator for Solid Waste and Emergency Response: 1. Ensure that only data essential to tracking emergency response equipment are required in the EPA national emergency response tracking system. 2. Determine whether the EMP equipment module is the most cost-efficient and functional national equipment tracking alternative. We recommend that the EPA Deputy Administrator: 3. Mandate that regions and response teams employ the national tracking system EPA decides to use for emergency response equipment. Agency Comments and OIG Evaluation EPA concurred with the findings and recommendations. Originally, the first recommendation directed that only data essential to tracking emergency equipment be included in the EPA national emergency response tracking system. In its response, EPA explained that the equipment module is already only tracking essential equipment, but went on to say that the equipment module contained data elements beyond the essential needs to help regions and special teams manage their equipment. The OIG agreed to change the first recommendation from ensuring only essential tracking emergency response equipment data be "included" in the system to ensuring that only essential tracking emergency response equipment data are "required" in the system, and the Agency concurred with the revised recommendation. OSWER stated that the users need more training to better familiarize them with the system. OEM plans to send a memo in September 2011 promoting all available training opportunities. While training may assist users of the equipment module in understanding data applications beyond their essential needs, it does not meet the intent of the recommendation to require only essential equipment tracking data. To be responsive to this revised recommendation, OSWER must assure that the system requires the data to track equipment for emergency response and does not require data for other purposes, such as equipment management. OSWER's 90-day response should provide a completion plan and date for complying with the first recommendation. OSWER is also planning to optimize the data entry screens to improve the responsiveness of the equipment module and to allow users to enter essential tracking data quickly and efficiently. However, OSWER did not provide a completion date for optimizing the data entry screens and should include one in its 90-day response to the final report. The Agency agreed with recommendation 2 and stated that it would hire an outside contractor to conduct an alternative analysis and determine the most efficient and effective national emergency response equipment tracking 11-P-0616 10 ------- alternative by spring 2012. We agree with this corrective action plan for recommendation 2. The Agency agreed with recommendation 3 and stated that it would prepare a memo for the Deputy Administrator's signature by fall 2011 requiring the regions and special teams to use the equipment module. The Deputy Administrator needs to establish plans, with dates, for when the Agency will send an updated memo following the alternative analysis scheduled for 2012, to be completed in response to recommendation 2. We agree with the Agency corrective action plan for recommendation 2 and request that additional information on action plans and completion dates for recommendations 1 and 3 be added to its corrective action plans when responding to the final report. The Agency's full response is in appendix A. 11-P-0616 11 ------- Status of Recommendations and Potential Monetary Benefits RECOMMENDATIONS POTENTIAL MONETARY BENEFITS (In $000s) Page Rec. No. No. Subject Status1 Action Official Planned Completion Date 1 10 Ensure that only data essential to tracking emergency response equipment are required in the EPA national emergency response tracking system. 2 10 Determine whether the EMP equipment module is the most cost-efficient and functional national equipment tracking alternative. 3 10 Mandate that regions and response teams employ the national tracking system EPA decides to use for emergency response equipment. Deputy Administrator Claimed Amount Ag reed-To Amount Assistant Administrator for Solid Waste and Emergency Response 0 Assistant Administrator for 06/20/12 Solid Waste and Emergency Response 12/22/11 1 0 = recommendation is open with agreed-to corrective actions pending C = recommendation is closed with all agreed-to actions completed U = recommendation is unresolved with resolution efforts in progress 11-P-0616 12 ------- Appendix A EPA Response to Draft Report UNITED STATES ENVIRONMENTAL PROTECTION AGENCY WASHINGTON, D.C. 20460 July 27, 20 ll MEMORANDUM SUBJECT: Comments on Draft Report: EPA Has Not Fully implemented a National Emergency Response Equipment Tracking System. Project No. OA-FY10-0210 Thank you for the opportunity to review and comment on the Draft OIG Evaluation Report "EPA Has Not Fully Implemented a National Emergency Response Equipment Tracking System" Project No. OA-FY10-0210. We have completed our review of the report and proposed recommendations. We have outlined in the attachment our response to the recommendations and our proposed actions going forward. We agree that the Emergency Management Portal (EMP) - Equipment Module ("Equipment Module") has not been fully implemented. However, regions are making continual progress toward using the Equipment Module as their day-to-day operational database. Furthermore, as discussed in the attachment, we have several initiatives underway to ensure the success of the Equipment Module. Our comments on the draft OIG report are attached. Please contact Johnsie Webster at 202-566- 1912 if you have further questions or concerns. FROM: Mathy Stanislaus, Assistant Administrator /Signed/ Office of Solid Waste and Emergency Response THRU: Bob Perciasepe Deputy Administrator TO: Melissa M. Heist Assistant Inspector General for Audit Attachment 11-P-0616 13 ------- ATTACHMENT Response to OIG Recommendations Ensure that only essential "equipment tracking data" needs are included in the EPA national emergency response equipment tracking system. Response & Actions: The Equipment Module is already designed to track equipment using only "essential tracking data." To track equipment checked out for field use, the module requires six pieces of "essential tracking data": unique barcode number, response site name, name of the EPA responsible party, name of the person that actually picked up the equipment from the warehouse (which may be the same as the EPA responsible party), the check-out date, and the check-out type (field use, maintenance, demonstration/training). If a warehouse enters this information, its equipment can be adequately located per the purpose of the Equipment Module. Additional information (maintenance, parts & supplies, etc.) can be entered and managed within the Equipment Module. This information can provide data on the condition and operability of the equipment, which is important information during a response. In addition, the Equipment Module allows the warehouse to track all equipment-related information in one application, rather than managing tracking data in one application and maintenance data in another, or entering some of the same data in two applications. In response to the comment that there are too many required fields and the database is too cumbersome, our National Decontamination Team (NDT) provides ongoing support and educational opportunities to the users of the Equipment Module through various means. Contractor support is available on a daily basis via email, a toll free help line and via direct cell phone line. User support requests are typically handled immediately over the phone or by conducting an impromptu web conference if a more extensive response is needed. In addition to the help desk requests, routine webinars are scheduled to introduce and review some the newer functions (e.g. bulk data uploads) that continue to be added to the system. A regularly scheduled quarterly Equipment Module User's Group web conference has also been held for the past few quarters. These web conferences are used to inform the users of upcoming changes and to gather their input on areas for possible improvements to the system. Several refresher training courses have been conducted at the request of the Regions. These courses provide a means to train new staff and review existing and new functions for all the staff. However, in order to better inform our users, OEM will send a memo promoting all training opportunities to the Regions and Special Teams by September 2011. In response to earlier concerns that the Equipment Module is cumbersome and slow, we have completed a pilot study on the use of bar code scanners. The results of this study showed that by using barcode scanners, the warehouse managers will be able to easily record the barcode numbers of equipment scheduled for check out or transfer to another warehouse and then have that information transferred to the Equipment Module, using either a direct interface, or by using a batch process with a spreadsheet or text file 11-P-0616 14 ------- containing the data. The scanners can be used for recording maintenance activities, annual warehouse inventories and other routine equipment inventory reports, although this use is not required. Most importantly during a large scale response when various equipment is borrowed or rented, the barcode scanners can be utilized as a standalone unit, with the Equipment Module updated on a periodic basis. We are currently working on improving the responsiveness by optimizing the data entry screens so that users can enter the essential tracking data quickly and efficiently. We believe the steps we are taking to improve the application will alleviate the problems we have had in the past and allow all of the warehouses to use the application more efficiently and therefore no corrective action date is needed. Determine whether the EMP equipment module is the most cost-efficient and functional national emergency response equipment tracking alternative. Response and Actions: We plan to conduct a thorough alternatives analysis by Spring 2012, utilizing an outside firm that specializes in this analysis. This analysis will consider several products, including commercial and government off-the-shelf software. Once complete, we will share the results with the OIG for review. While we are conducting this analysis, we will still move forward with the bar code scanning functionality. If the alternatives analysis shows that we should move to another solution, the bar code scanners can be integrated into that tool. In response to the comment "The EMP Equipment Module is Cumbersome and Slow Compared to NEMS," the EMP Equipment application cannot be easily compared with the NEMS system. The Equipment Module is a web-based application that was developed to ensure nationally consistent terminology and tracking. The NEMS system was an interim solution that connected to regional databases through web services. The NEMS system did not promote nationally consistent equipment terminology and was not meant to be an operational, day-to-day equipment management system. If the NEMS system were implemented as a national tool, each region would need to maintain its own database and ensure that it could successfully share data with NEMS utilizing the correct, consistent terminology. Recently the National Decontamination Team (NDT) conducted a review of internet speeds at each of the EPA warehouses. Many of the issues that the regions have experienced with the EMP Equipment application is due to slow internet speeds. We are developing a minimum internet speed standard. Once the warehouses have bar code scanners and acceptable internet capability, usability should improve. In addition, the Equipment Module already includes a local version that can be installed on the warehouse local machines and that does not require constant access to the internet. The data within the local version can then be uploaded to the central system on a regular basis, usually during downtime of the warehouse. This mitigates problems with 11-P-0616 15 ------- suboptimal warehouse internet connections. We recommend this solution to until the bar code scanners and improved internet capabilities can be implemented fully. For the past two years NDT has been gathering input from the regions on improvements to the Equipment Module and to date, over 100 improvements have been implemented, which have greatly improved usability. Mandate that regions and response teams employ the national tracking system EPA decides to use for emergency response equipment. Response and Actions: We agree with this recommendation and OEM will prepare a memo for the Deputy Administrator's signature by Fall 2011 requiring the Regions and Special Teams to use the Equipment Module. For the purpose of background, the majority of regions and special teams use the EMP Equipment application, many regions have existing contracts in place that do not require the use of the EMP Equipment application. As these contracts turnover, more warehouses will be required to use the application as part of their contract renewals or contract competitions. Additionally, OEM conducts annual evaluations with each region called Core National Approach to Response (Core NAR). Core NAR sets standards to ensure that each region works toward improving and maintaining an excellent response program. For the past two years, the Core NAR evaluation has included a criterion which we use to determine whether or not the regions and special teams are complying with the 2008 memo and using the Equipment Module. OEM will continue to include this criterion in the Core NAR to further evaluate the increased usage of the Equipment Module by EPA regions and special teams. 11-P-0616 16 ------- Distribution Office of the Administrator Deputy Administrator Assistant Administrator for Solid Waste and Emergency Response Agency Followup Official (the CFO) Agency Followup Coordinator General Counsel Associate Administrator for Congressional and Intergovernmental Relations Associate Administrator for External Affairs and Environmental Education Audit Followup Coordinator, Office of Solid Waste and Emergency Response 11-P-0616 ------- |