U.S. ENVIRONMENTAL PROTECTION AGENCY OFFICE OF INSPECTOR GENERAL U.S. Chemical Safety and Hazard Investigation Board Needs to Complete More Timely Investigations Report No. 13-P-0337 July 30, 2013 ------- Report Contributors: Anthony Grear Marcia Hirt-Reigeluth Denise Darasaw Gloria Taylor-Upshaw Michael Davis Abbreviations CFR Code of Federal Regulations CSB U.S. Chemical Safety and Hazard Investigation Board EPA U.S. Environmental Protection Agency FY Fiscal year GAO U.S. Government Accountability Office GPRA 2010 Government Performance and Results Act Modernization Act of 2010 OIG Office of Inspector General OMB Office of Management and Budget TRIM Total Records and Information Management Hotline To report fraud, waste, or abuse, contact us through one of the following methods: email: OIG Hotline@epa.gov write: EPA Inspector General Hotline phone: 1-888-546-8740 1200 Pennsylvania Avenue, NW fax: 202-566-2599 Mailcode 2431T online: http://www.epa.gov/oiq/hotline.htm Washington, D.C. 20460 ------- x-^eD sT/if. •*. U.S. Environmental Protection Agency 13-P-0337 | \ Office of Inspector General July 30 2013 a % ? At a Glance Why We Did This Review The U.S. Environmental Protection Agency Office of Inspector General initiated this audit to determine whether the U.S. Chemical Safety and Hazard Investigation Board has an effective system for managing its investigative process. CSB is an independent federal agency, authorized by the Clean Air Act Amendments of 1990 to investigate, determine and report to the public in writing the facts, conditions and circumstances and the cause or probable cause of any accidental release resulting in a fatality, serious injury or substantial property damages. This report addresses the following CSB goal: • Conduct incident investigations and safety studies concerning releases of hazardous chemical substances. U.S. Chemical Safety and Hazard Investigation Board Needs to Complete More Timely Investigations For further information, contact our Office of Congressional and Public Affairs at (202) 566-2391. The full report is at: www.epa.qov/oiq/reports/2013/ 20130730-13-P-0337.pdf What We Found CSB does not have an effective management system to meet its established performance goal to "[cjonduct incident investigations and safety studies concerning releases of hazardous chemical substances." CSB has not fully accomplished its related strategic objective to "[cjomplete timely, high quality investigations that examine the technical, management systems, organizational, and regulatory causes of chemical incidents." We identified five reasons why CSB did not meet its objective to timely complete investigations: • A lack of defined performance indicators in CSB's annual performance plan, which are necessary to assess the efficiency of its investigations process. • A backlog of open investigations without documented plans for resolution. • An average investigative staff turnover rate of 15 percent. • Non-collocation of files and incorrectly classified or coded investigation files. • A need for updated policies over current investigative procedures and a policy that defines final investigative products. By completing investigations more timely, CSB can better fulfill its mission and improve its ability to ensure that it provides the community and other stakeholders with findings and recommendations to help reduce the occurrence of similar incidents, which would protect human health and the environment. Recommendations and CSB Planned Corrective Actions We made nine recommendations, including that the CSB chairperson: • Develop and implement performance indicators. • Revise and publish annual and individual action plans. • Review investigations open for over 3 years and develop a close-out plan. • Review investigation files for each ongoing investigation to ensure it contains all the supporting documents related to the investigation. • Implement and update the management policy for investigative records. CSB agreed with six of nine recommendations and responded in detail to each in appendix A. CSB plans to complete proposed corrective actions by December 31, 2013. CSB disagreed with the remaining three recommendations and resolution efforts are in progress. Noteworthy Achievements CSB has made progress in providing investigators with training and identifying operational tools for the investigative process. CSB also instituted the use of scoping documents and recommendation briefings to help improve the process. ------- UNITED STATES ENVIRONMENTAL PROTECTION AGENCY WASHINGTON, D.C. 20460 THE INSPECTOR GENERAL July 30, 2013 The Honorable Rafael Moure-Eraso, Ph.D. Chairperson and Chief Executive Officer U.S. Chemical Safety and Hazard Investigation Board 2175 K Street, NW, Suite 400 Washington, D.C. 20037-1809 Dear Dr. Moure-Eraso: This is our report on the U.S. Chemical Safety and Hazard Investigation Board's need to complete investigations more timely. This report contains findings the Office of Inspector General has identified and corrective actions the OIG recommends. This report represents the opinion of the OIG and does not necessarily represent the final CSB position on the subjects reported. CSB managers will make final determination on matters in this report in accordance with established audit resolution procedures. CSB disagreed with recommendations 2, 4 and 6 from our draft report. Resolution efforts are in progress on these recommendations. Please provide a written response to this final report, including proposed corrective actions, within 60 calendar days of the report date. The response will be posted on the OIG's public website, along with our memorandum commenting on the response. The response should be provided as an Adobe PDF file that complies with the accessibility requirements of Section 508 of the Rehabilitation Act of 1973, as amended. The final response should not contain data that should not be released to the public; if the response contains such data, the data for redaction or removal should be identified. We will post this report on our website at http://www.epa.gov/oig. If you or your staff has any questions regarding this report, please contact Richard Eyermann, acting assistant inspector general for audit, at (202) 566-0565 or eyermann.richard@epa.gov: or Michael Davis, product line director, at (513) 487-2363 or davis.michaeld@epa.gov. Sincerely, Arthur A. Elkins Jr. ------- U.S. Chemical Safety and Hazard Investigation Board Needs to Complete Investigations More Timely 13-P-0337 Table of C Chapters 1 Introduction 1 Purpose 1 Background 1 Noteworthy Achievements 2 Scope and Methodology 2 2 CSB Needs to Improve in Meeting Its Performance Objective to Complete More Timely Investigations 4 CSB Not Meeting Its Goal Related to Timely Investigations 4 Federal Laws and Policies Address How to Manage and Monitor Performance Objectives 6 CSB Has Guidance for Managing Processes 7 Reasons Why CSB Did Not Meet Its Performance Objective 8 Conclusion 14 Recommendations 14 CSB Comments and OIG Evaluation 15 Status of Recommendations and Potential Monetary Benefits 17 Appendices A CSB Response to Draft Report 19 B CSB Board Member Response to Draft Report 28 C OIG Comparison of CSB Responses to the Draft Report 30 D Distribution 38 ------- Chapter 1 Introduction Purpose The U.S. Environmental Protection Agency Office of Inspector General initiated this audit to determine whether the U.S. Chemical Safety and Hazard Investigation Board has an effective system for managing its investigative process. The project contributes to CSB's Strategic Goal 1, "[cjonduct incident investigations and safety studies concerning releases of hazardous chemical substances." Background CSB is an independent federal agency authorized by the Clean Air Act Amendments of 1990. The act directs CSB to (1) investigate and report on the cause or probable cause of any accidental release resulting in a fatality, serious injury or substantial property damage; (2) make safety recommendations to reduce the likelihood or consequences of accidental chemical releases and propose corrective measures; and (3) establish by regulation requirements binding on persons for reporting accidental releases into the ambient air subject to the Board's investigatory jurisdiction. The purpose of a CSB incident investigation is to determine the causes of an incident and whether those causes were the result of a violation of any current and enforceable requirement. The CSB investigative staff includes chemical and mechanical engineers, industrial safety experts, and other specialists with experience in the private and public sectors. After a CSB team reaches a chemical incident site, investigators conduct detailed interviews of witnesses such as plant employees, managers and neighbors. Chemical samples and equipment obtained from accident sites are sent to independent laboratories for testing. Company safety records, inventories and operating procedures are examined as investigators seek an understanding of the circumstances of the accident. CSB is located in Washington, D.C., with an investigation field office in Denver, Colorado. CSB's authorizing statute provides for five board members, including a chairperson, all appointed by the President and confirmed by the United States Senate. As of April 2013, there were three appointed board members, including the chairperson, and a staff of 39 that includes 18 who are involved in investigations. 13-P-0337 1 ------- Noteworthy Achievements CSB has made progress in providing investigators with training and identifying operational tools for the investigative process. In August 2012, CSB began conducting semiannual investigator training conferences that included topics on investigation method discussions and exercises defining the differences between investigation products. CSB also instituted the use of scoping documents and recommendation briefings to help improve the investigation process. CSB explained that scoping documents provide staff with a list of personnel who are involved in an investigation and the percentage of time they plan to devote to an investigation, and hopes to refine further this process to include the total number of staff hours estimated for each investigation. Scope and Methodology We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions and that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objective. We performed our audit work from June 2012 through May 2013. Our objective was to determine whether CSB has an effective system for managing its investigative process. The scope of the audit included investigations from October 1, 2006, through May 31, 2012. We judgmentally selected six investigations based on length of time the investigation was open and were still open or recently closed. The cost of the investigation and the location of the lead investigator were also considered in our selection. Our sample included four investigations that remained open from 2008, 2009 or 2010 and two that were closed in fiscal year 2012. Three had a lead investigator from headquarters and remaining three had a lead investigator from CSB's Denver office. We reviewed the sample investigative files for documented support. We interviewed CSB's management, senior investigators and investigative staff to identify and discuss the investigative process. We obtained and reviewed CSB's board orders (policies and procedures) that govern the investigative process. We reviewed the status of investigations and CSB's performance goals. We performed a site visit to CSB's Denver office. Prior CSB-Related Audits The EPA OIG assumed oversight responsibility for CSB in FY 2004. Previously, the Federal Emergency Management Agency and U.S. Department of Homeland Security OIGs had performed OIG oversight for CSB. We issued two prior reports to CSB relating to its implementation of prior audit recommendations and CSB's safety recommendation process. 13-P-0337 2 ------- In EPA OIG Report No. 1 l-P-0115, Chemical Safety and Hazard Investigation Board Did Not Take Effective Corrective Actions on Prior Audit Recommendations, issued February 15, 2011, we sought to determine whether CSB had implemented audit recommendations made by those three OIGs and the U.S. Government Accountability Office. We found that CSB did not take timely corrective actions to address audit recommendations. Also, CSB had not established and implemented a management control program to evaluate and report on the effectiveness of controls related to its program operations. In the FY 2011 report, we recommended that CSB develop and implement a management control plan that documents and addresses internal control standards in accordance with Office of Management and Budget Circular A-123, Management's Responsibility for Internal Control, and the U.S. Government Accountability Office's Standards for Internal Control in the Federal Government. We recommended that the management control plan include procedures for conducting periodic internal control reviews and properly documenting those reviews, including verifying and ensuring that audit recommendations are resolved promptly. CSB noted it is in the process of creating a management control plan. During this audit, we identified an additional concern with the lack of documented internal reviews. We requested documented support for CSB's FY 2011 assurance statement on the effectiveness of its operational program's internal controls. CSB did not provide documented support as required by OMB Circular A-123, Section 6. CSB has certified its compliance with FY 2012 management controls based on internal and external evaluations and knowledge gained from daily operations. In EPA OIG Report No. 12-P-0724, U.S. Chemical Safety and Hazard Investigation Board Should Improve Its Recommendations Process to Further Its Goal of Chemical Accident Prevention, issued August 22, 2012, we sought to determine what factors impede implementation of CSB safety recommendations. We found that CSB did not consistently achieve its goals and standards, as outlined in its current strategic plan, for timely implementation of its safety recommendations. Although CSB does not have enforcement authority, and implementation of some of its recommendations may face lengthy regulatory processes, CSB has not established or maintained sufficient internal controls and processes for safety recommendations. 13-P-0337 3 ------- Chapter 2 CSB Needs to Improve in Meeting Its Performance Objective to Complete More Timely Investigations CSB does not have an effective management system to meet its established performance goal to "[cjonduct incident investigations and safety studies concerning releases of hazardous chemical substances." Specifically, CSB has not fully accomplished its related strategic objective to "[cjomplete timely, high quality investigations that examine the technical, management systems, organizational, and regulatory causes of chemical incidents." Various federal laws and policies address how federal agencies should manage and monitor their performance. We identified five reasons why CSB did not meet its objective to complete timely investigations: • A lack of defined performance indicators in CSB's annual performance plan, which are necessary to assess the efficiency of its investigations process. • A backlog of open investigations without documented plans for resolution. • An average investigative staff turnover rate of 15 percent. • Non-collocation of files and incorrectly classified or coded investigation files. • A need for updated policies over current investigative procedures and a policy that defines final investigative products. By completing more timely investigations, CSB can better fulfill its mission and improve its ability to ensure that it provides the community and other stakeholders with findings and recommendations to help reduce the occurrence of similar incidents, which would protect human health and the environment. CSB Not Meeting Its Goal Related to Timely Investigations CSB's 2012-2016 strategic plan, issued in June 2012, is a revision to its prior 2007-2012 plan. The new strategic plan replaces the FYs 2007-2012 plan's five goals with three goals. In addition, the new plan adds 13 "outcome oriented" strategic objectives based on the three goals, and the objectives measure the effectiveness or quality of their performance and the public benefit derived. CSB notes that the new plan places "emphasis on conducting investigations, securing implementation of recommendations, and disseminating CSB findings." In the revised plan, the first goal is to "[cjonduct incident investigations and safety studies concerning releases of hazardous chemical substances." CSB states that this goal drives the core mission to ensure that CSB selects and 13-P-0337 4 ------- completes incident investigations that have the potential to generate thorough recommendations with high preventive impact. This goal is a change from CSB's prior first goal to "[sjelect and complete accident investigations and recommend actions with a high potential for protecting workers, the public, and the environment." An objective for the first goal is to "[cjomplete timely, high quality investigations that examine the technical, management systems, organizational, and regulatory causes of chemical incidents." This objective relates to its prior key result to "[rjeduce the time to complete investigation products." CSB's revised plan notes that it "endeavors to complete its thorough investigations as quickly as possible and, in many cases, incident investigations are completed in less than a year." CSB's revised strategic plan identifies two performance measures for its objective to complete timely investigations. For FY 2012, one measure was to complete seven investigations and another was to establish a baseline for the average time to complete an investigation. We reviewed and compared CSB's accomplishments reported in its FY 2007 through 2012 performance accountability reports and the status of current and completed investigations. We found that over the 6-year period, CSB has steadily fallen behind in accomplishing its objective related to timeliness, as described in table 1. Table 1: CSB's planned and actual accomplishments for goal 1 FY Investigations planned to complete Investigations actually completed Percent actually completed 2007 10 10 100.00% 2008 6 6 100.00% 2009 6 4 66.67% 2010 8 4 50.00% 2011 15 5 33.33% 2012 8 2 25.00% Totals 53 31 (58%) Sources: CSB's Performance and Accountability Reports for FYs 2007-2012. From comparing this information to data on the status of investigations provided to us during our audit and on the website, we identified an additional seven investigations reported as completed in FYs 2007, 2010 and 2012. Of the seven, only two are included on the CSB website. The remaining five were listed in a safety bulletin as examples for "hot works" demonstrations, which include burning, welding or similar operations that are capable of initiating fires or explosions. We could not clearly confirm whether these investigations were actually completed or just reported in the safety bulletin. CSB is not consistent in accomplishing its objective to complete timely investigations. Other than stating that it endeavors to complete an 13-P-0337 5 ------- investigation as soon as possible, CSB's strategic plan does not clearly define a "timely completed investigation." If CSB clarifies what is "timely" and uses that definition in its measures or indicators for setting its performance goals and objectives, CSB could be more consistent and efficient in completing investigations. Federal Laws and Policies Address How to Manage and Monitor Performance Objectives Several federal laws and policies address how agencies should manage and monitor to achieve their performance goals and objectives. The Government Performance and Results Act Modernization Act of 2010 (GPRA 2010), requires that agencies annually establish agency performance plans to include "a balanced set of performance indicators to be used in measuring or assessing progress toward each performance goal." GPRA 2010 also stipulates that agencies are to provide in plans "a description of how the performance goals are to be achieved, including ... the operation processes, training, skills and technology, and the human, capital, information, and other resources and strategies required to meet those performance goals." GPRA 2010 states in section 3 that agency performance plans are to be made available annually on an agency public website. GAO's Standards for Internal Control in the Federal Government addresses establishing internal controls for the monitoring of performance. An example of a control activity category common to all agencies is the establishment and review of performance measures and indicators because "[control] [activities need to be established to monitor performance measures and indicators. These controls could call for comparisons and assessments relating different sets of data to one another so that analyses of the relationships can be made and appropriate actions taken. Controls should also be aimed at validating the propriety and integrity of both organizational and individual performance measures and indicators." OMB Circular A-123 clarifies in Section 3 that effective internal control is a key factor in achieving agency missions and program results through improved accountability. Section 4 states that agency managers should continuously monitor and improve the effectiveness of internal control associated with their programs. This continuous monitoring, and other periodic assessments, should provide the basis for the agency head's annual assessment of and report on internal control in its annual performance plans. CSB's annual performance plans and reports are sources of information CSB can use to assess its internal controls. The Code of Federal Regulations, in 36 CFR 1236.26, provides steps that agencies are required to take to maintain electronic information systems. Specifically, the regulation states, "(a)Agencies must maintain inventories of 13-P-0337 6 ------- electronic information systems and review the systems periodically for conformance to established agency procedures, standards, and policies as part of the periodic reviews required by 44 U.S.C. 3506. The review should determine if the records have been properly identified and described, and if the schedule descriptions and retention periods reflect the current informational content and use." CSB Has Guidance for Managing Processes CSB has issued its own specific guidance for its investigative process and management of CSB records: Board Order 040, Investigation Protocol, March 2006, as amended December 2011; and Board Order 019, Records Management Program, March 2001, amended May 2003. These policies, if followed, would provide CSB with the procedures needed to ensure investigations and supporting investigative records are managed to achieve CSB's performance objectives. Board Order 040 outlines several procedures for initiating, managing and organizing, interviewing, and gathering evidence on investigations. Procedure A outlines the investigator in charge responsibilities in conducting an incident investigation. Procedures B and C describe the planning in preparation for deployment and activities of the investigative team between the time a decision is made to deploy, their arrival at the incident scene, and initial field activities. Procedure D provides guidance on how to organize various types of investigation data. Section 2 of this procedure requires the investigator in charge to appoint a records liaison coordinator from the investigation team to label, file and track all investigation materials. The liaison is responsible for creating and maintaining the electronic records management files system for the duration of the investigation and once it is closed. Appendix D1 provides the codes by category that investigators should use for electronic evidence files. Board Order 019 provides the detail needed to implement and operate CSB's records management program to ensure that CSB meets recordkeeping requirements. The order states in Appendix A, Section 2 that records include: ... all books, papers, maps, photographs, machine readable materials, or other documentary materials, regardless of physical form or characteristics, made or received by an agency of the United States Government under Federal law or in connection with the transaction of public business and preserved or appropriate for preservation by that agency or its legitimate successor as evidence of the organization, functions, policies, decisions, procedures, operations, or other activities of the Government or because of the informational value of the data in them (44 U.S.C. 3301). 13-P-0337 7 ------- Appendix A, Section 4 outlines CSB's responsibilities for the records management program, explaining that"... each employee must [ojrganize files according to their functional area's standard file plan and based on guidance from their Record's Officer; [ m]ake sure that folders, disks and other filing media are properly labeled and have some method for identifying the full contents and date ranges of the records contain within." This section also explains that employees must "[cjreate and maintain indexes and logs to facilitate the retrieval of files as needed." The board order states that appropriate management of records will be a standard performance element in all employee performance plans. Reasons Why CSB Did Not Meet Its Performance Objective CSB did not meet its objective for timely completing investigations due to: • A lack of performance indicators. • A backlog of open investigations. • An investigative staff turnover rate averaging 15 percent. • Files not collocated and incorrectly classified or coded investigation files. • A need for updated policies to address current investigative procedures and define final investigative products. Lack of Defined Performance Indicators CSB does not have specific performance indicators to measure the efficiency of its investigative process. In addition, CSB does not have a current annual action plan describing these indicators for the key phases and periods of the investigative process that would help ensure it is completing timely investigations. In its FY 2012 performance accountability report, CSB noted several performance measures for its first goal, to conduct timely incident investigations and safety studies that involve accidental releases or potential releases of hazardous chemical substances. Those measures were: • Completed investigations. • Average time to complete an investigation. • Implementation of lessons learned process. • Complete priority sections of investigation protocol. • Cost per project versus historical cost per report format. CSB's overall status for all the measures is "ongoing," with the exception of the lessons learned measure, which CSB noted as complete for two investigations. As of December 2012, CSB has not defined the related performance indicators necessary in key phases of the investigative process to assess the efficiency and, ultimately, the achievement of its goals. CSB lacked specific performance indicators that define the amount of time necessary to 13-P-0337 8 ------- perform/witness interviews, prepare scoping documents, and write the report and/or case study. As a result, CSB did not have information to identify areas for improvement, plan for future investigations and allocate resources. CSB stated that it is very difficult to estimate the appropriate amount of time it takes on average to conduct witness interviews, design laboratory testing (where no protocols exist) and write reports. However, CSB could gain further efficiencies through a detailed analysis of its investigative process for common performance indicators. We believe that indicators could help CSB measure the cost effectiveness and timeliness of each phase in the investigation process. By not identifying performance indicators for key phases in the investigations cycle, CSB has less capability to hold staff accountable for efficient completion of chemical accident investigations. In addition to the performance indicators, CSB does not have a published FY 2011 or 2012 agency performance plan (CSB refers to it as an annual action plan) as required by GPRA 2010, and its individual performance plans (plans used to monitor staff performance) do not define timeliness requirements for key phases of the investigation process. According to C SB's 2012-2016 Strategic Plan, CSB develops an annual action plan that includes many additional performance measures that correspond to a specific fiscal year. CSB suspended the board vote on its draft 2011 action plan on January 17, 2011, and, as of February 2013, does not have an action plan in place. The strategic plan also states that it, along with the annual action plan and individual performance plans, constitute the foundation of the organization's performance management framework. CSB stated it is in the process of developing an annual action plan for FY 2013. Backlog of Open Investigations With No Closeout Plan CSB needs to address its backlog of six investigations that have been open for over 3 years (table 2). Table 2: CSB's six investigations open for more than 3 years Investigation Date of accident Fatalities Injuries to workers or contractors *BP America Refinery Ultracracker Explosion January 14, 2008 1 0 *Packaging Corporation Storage Tank Explosion July 29, 2008 3 1 Silver Eagle Refinery Flash Fire and Explosion and Catastrophic Pipe Explosion January 12, 2009 0 4 *CITGO Refinery Hydrofluoric Acid Release and Fire July 19, 2009 0 1 *Caribbean Petroleum Refining Tank Explosion and Fire October 23, 2009 0 0 NDK America Inc. Explosion with Offsite Fatality December 7, 2009 1 2 Source: CSB website www.csb.aov. * Investigation was in our sample. 13-P-0337 9 ------- Closing its backlog would allow CSB to provide to the community and other stakeholders its findings and recommendations, which would help reduce the occurrence of similar incidents and result in the protection of human health and the environment. Closure would also provide more time for investigative management to focus on recent and new incidents. In our sample of four investigations open for over 3 years, two involved fatalities. One of those open investigations is the BP America Refinery Ultracracker Explosion, in Texas City, Texas. On January 14, 2008, a worker died when the top of large steel filter housing suddenly blew off in the refinery's ultracracker unit. The second open investigation involves the Packaging Corporation Storage Tank Explosion in Tomahawk, Wisconsin. On July 29, 2008, three workers died and a fourth was injured when an explosion occurred inside an 80-foot-tall storage tank at the corrugated cardboard mill; the workers were on a catwalk above the tank, performing hot work (welding), when the explosion occurred. CSB placed these investigations on hold due to their BP Deepwater Horizon investigation in April 2010 related to the explosion and subsequent oil spill in the Gulf of Mexico. However, both investigations had already identified safety deficiencies that CSB could have formally addressed. CSB said the two investigations had been left open from a prior administration and CSB has been faced with challenges from the company and courts in completing the Deepwater Horizon investigation. We also noted that CSB's board orders do not identify a specific period for the completion of investigations even though its strategic plan notes CSB's endeavors to complete thorough investigations as quickly as possible. One manager noted, "... there is no formal schedule for the development and planning of an investigation. Investigative teams often set the investigative plan, but that is often interrupted with new deployments that bring new investigations.... Investigators are often deployed to multiple incidents often at the request of Congress or other external stakeholders, within a 6 month range that often delays the progress of their investigation." A close-out plan to address investigations open for more than 3 years would enable CSB to focus on completion. Turnover of Investigative Staff CSB's staff average turnover rate of 15 percent for the past 5 years has an effect on its mission to conduct and complete timely investigations (table 3). According to a Board Member, the turnover includes "... senior investigative staff. Staff with investigative experience, particularly experience with CSB['s] approach to investigations, [which] is essential in improving the timeliness and maintaining the quality of the CSB reports." 13-P-0337 10 ------- Table 3: CSB's turnover percentages FY Separations On board "Turnover percentage 2008 3 16 19% 2009 4 20 20% 2010 1 21 5% 2011 4 21 19% 2012 3 22 14% 5-year average 15% Source: OIG analysis. Turnover was calculated using separations and on-board figures for each year. Although senior management believes it has a much more gifted and committed investigative staff in place now than at any prior time, management stated "[t]he primary barrier [to completing investigations] is the lack of resources available to the agency as a whole, and the need to constantly reshuffle investigators among multiple new projects." In response to questions regarding the investigation process, one investigator manager stated that"... CSB needs a retention policy. Investigators are often leaving due to the inconsistent travel demands of the job and lack of support.... There is also an overreliance on employees voluntarily working after hours to complete investigations. Better management can reduce this burden and improve the work-life balance necessary to retain competent staff." We reviewed the Office of Personnel Management's 2012 Federal Employee Viewpoint Survey Results to look at CSB employees' perceptions about how effectively CSB is managing its workforce. Of the 35 employees asked to take the survey, 33 completed the survey, for a response rate of 94.3 percent. The results show that the staff have a positive view of their mission despite the fact that they believe CSB does not have sufficient resources (people, material, budget) to accomplish the mission. The areas identified for improvement primarily related to CSB's board and senior leadership. Specifically, 59 percent of CSB employees surveyed expressed dissatisfaction in senior leadership areas relating to satisfaction with policies and practices of senior leaders. Further, 53 percent of respondents disagreed that leaders generate high levels of motivation and commitment to the workforce. CSB rated below government averages in the senior leadership areas, but has told us the agency has considerably improved compared to its 2011 survey results. Involving staff investigators in planning process is a best practice. In A Model of Strategic Human Capital Management exposure draft from March 2002, GAO states, "[t]he involvement of employees both directly and through employee organizations will be crucial to success," and notes involving employees can "... increase employees' understanding and acceptance of organizational goals and objectives and improve motivation and morale." 13-P-0337 11 ------- Senior management stated that developing a formal career ladder for investigator development, along with expanded investigator training programs, will also aid in investigator retention and development of lead investigator candidates. As CSB's board and senior leadership continue their commitment to its workforce, discussing and addressing employee concerns may result in a change in the turnover rate and a more efficient investigative process. Investigative Files Not Collocated and Incorrectly Classified or Coded Our review of CSB's electronic investigation files showed that files were not in one location and the electronic records were incorrectly classified or coded. Staff did not properly classify or code multiple electronic investigation records in accordance with records management or investigation protocol policies. CSB does not use its electronic investigation files to store project plans, milestones or decision documents. CSB investigators use several in-house documents that support its investigation file, such as work plans, scoping documents and recommendation briefs. These documents provide an ongoing investigation status summary of what happened, how it happened, what has been done, and what remains to be done with the timelines and preliminary recommendations to correct the problems found. CSB does not maintain these documents in its electronic investigation file. We could not locate these documents to support the six sample investigations in our audit. CSB explained that it primarily uses its electronic system to store investigative evidence and not project management documents. Investigation files located together provide a complete review of the investigation status at any given time. Our review also found that CSB staffs are not classifying or coding multiple records in their electronic investigation files in accordance with records management or investigation protocol policies. Our review of six sample investigations found that the majority of the 21,316 documents had classification and document identification numbers. However, we determined that document identification numbers were missing from 28 percent of the documents in three of the investigations, and classification numbers were missing from 19 percent of the documents in two of these three investigations. Further, 13 percent of all documents had new coding inconsistent with the category codes listed in Board Order 040, Appendix Dl, Evidence Log Filing System. Records should clearly identify the investigation process and support investigation results. We were informed that there are no routine reviews or assessments, but the investigation close-out process includes a certification that confirms the reliability of the data in the file. CSB noted that when an investigator separates from the agency, CSB's most important task is first to make sure everything investigation-related on the person's computer is uploaded to the electronic file and then, second, to perform manually an edit of the information. However, CSB noted it often does 13-P-0337 12 ------- not have the time to perform the edits. CSB stated it"... encourages investigators to register evidentiary records in the system as quickly as possible so that record fidelity and access is controlled." Ensuring investigation files are collocated and correctly classified or coded would enable any investigator to easily identify the status of an investigation and compare results from completed investigations. Investigators could more efficiently complete or close out investigations in a timely manner, particularly cases open for more than 3 years. Lack of Investigative Policies That Address Current Procedures and Define Final Products CSB needs updated policies that address its current investigative procedures and define its final products. CSB staffs follow procedures that are not listed in Board Order 040, CSB's policy for its investigation process. As noted above, CSB did not keep investigation files collocated, and did not correctly classify or code investigations so that staff could efficiently identify whether the team followed outlined procedures in Board Order 040. From our interviews, we found that CSB has implemented changes to the investigation process but has not updated its internal policies. CSB teams started using scoping documents and recommendation briefings for its investigations in FY 2010 without a policy to define the use of the documents. Staff investigators designed these procedures to keep management and team members abreast of a recent investigation to be placed on hold due to the BP Deepwater Horizon investigation. Since the introduction of these documents, CSB has started to use these procedures on newer investigations, but without direction. A manager stated that the investigation scoping process assists the board in determining the most pertinent issues to develop in an investigation, the process is new, and each investigative team views the scoping process with varying levels of importance. Additionally, there has been no formal training on this new scoping process. Board Order 040 does not cover these documents and does not define when the investigators should use scoping documents. Further, CSB's investigative process would be more efficient if it implemented procedure changes more consistently and updated policies to provide clear and complete direction. CSB does not determine the type of final product it will produce when it begins an investigation. Having some guidelines in performing the investigation for specific types of final products—such as case studies, safety studies or investigation reports—would help staff be more efficient in completing investigations. CSB generally makes a determination on whether to produce an investigation report or a case study after investigators present incident information to the board on a case-by-case basis. One staff investigator stated that it "may be months into a case before the team has been informed as to ... the finished product... [e]ach type of product has a 13-P-0337 13 ------- different degree of thoroughness for completion." We agree with staff who told us that CSB should have a policy document with a product definition to help teams' better scope what needs to occur earlier in the investigation so that the team can react accordingly. Conclusion CSB needs to ensure its investigative process supports the accomplishment of its goals and objectives to complete timely investigations that examine the causes of chemical incidents. By addressing the issues noted, CSB would improve the investigation process and help meet its goals and objectives to complete timely investigations and, ultimately, provide communities with the help needed to reduce the occurrence of incidents and thus protect human health and the environment. Recommendations We recommend that the chairperson, U.S. Chemical Safety and Hazard Investigation Board: 1. Develop and implement performance indicators related to its first strategic performance goal and objective to complete timely investigations. Indicators should track and measure the efficiency of key phases of the investigation process and clarify the definition of a "timely" completed investigation. Also, address the indicators in the investigation protocol policy. 2. Revise and publish an annual action plan to comply with GPRA 2010 and update related individual performance plans to ensure that performance indicators are addressed and investigative staff are held accountable for performing key phases in the investigation process. 3. Review investigations open for more than 3 years and develop a plan to close out those investigations. 4. Develop and implement a succession or retention policy to help with any future effects of the turnover rate on CSB's mission. 5. As a best practice, involve staff in the planning process of an investigation. Hold meetings between senior management and staff to address any concerns with the investigation process. 6. Review and collocate investigation files for each ongoing investigation to ensure that they contain all the supporting documents related to the investigation. At a minimum, ensure files have proper classifications, project plans, scoping documents and board decisions. 13-P-0337 14 ------- 7. Implement and update the records management policy to ensure that the classification of electronic investigation files agrees with the investigation protocol policy and staffs perform internal reviews of records as required by the policy. 8. Update the investigation protocol policy for all current investigation procedures to include scoping documents and recommendation briefs. Provide formal training to the investigative staff on changes and updates to the investigative process. 9. Provide guidelines for staff to determine the type of final product in the beginning of the investigation process to help staff be more efficient in completing investigations. CSB Comments and OIG Evaluation The OIG evaluation considers the response from the CSB chairperson as the official response in appendix A. We also attached a separate response to the report submitted by one of CSB's other board members in appendix B. We compared the responses for information purposes only in appendix C. CSB agreed with six recommendations (1, 3, 5, 7, 8 and 9) and responded in detail to each. CSB plans to complete corrective actions by December 31, 2013. CSB did not clearly address recommendation 2 and stated, "CSB has published an up-to-date Strategic Plan, annual performance-based budgets, and annual performance reports," which it believes meets the requirements of GPRA 2010. GPRA 2010 requires that agencies annually establish agency performance plans to include a balanced set of performance indicators to be used in measuring or assessing progress toward each performance goal. GPRA 2010 also stipulates that agencies are to provide in plans a description of how the performance goals are to be achieved, including the operation processes, training, skills and technology, and the human, capital, information, and other resources and strategies required to meet those performance goals. CSB's strategic plan states that it, along with the annual action plan and individual performance plans, constitute the foundation of the organization's performance management framework. CSB previously stated that it is in the process of developing an annual action plan for FY 2013. In response to our draft report, CSB considers its "action plans" as internal, evergreen documents that are developed annually and updated periodically through the year to track initiatives as they strive to accomplish the goals set in our Strategic Plan. We maintain that a published agency performance plan (annual action plan) with performance indicators as prescribed by GPRA 2010 is needed along with updated individual performance plans to ensure that investigative staff are held accountable for performing key phases in the investigation process. We request that CSB reconsider this recommendation and include a plan to publish an agency performance plan 13-P-0337 15 ------- (annual action plan) with performance indicators as well as updated individual performance plans in its 60-day response to our final report. CSB did not address the need for a policy in recommendation 4 and stated it is a very small agency, and it is indisputable that unexpected turnover can have negative impacts on the completion of specific projects where individual investigators may have developed unique or unusual knowledge. We agree with CSB's size and the impact that turnover would have on the agency. We calculated an average turnover rate of 15 percent for the 5-year period 2008 through 2012, which reflects those who no longer work at CSB for numerous reasons. The average percent remains even though CSB lists various reasons for the different types of turnover in its response to our draft report. We have not verified those reasons and therefore cannot place the information in our report. Our report notes that turnover is just one of several reasons why CSB has not timely completed investigations. Additionally, we acknowledge CSB has informed us of its succession and retention initiatives and noted there have been zero investigative turnovers during the past 8 months. We believe the list of implemented initiatives should be described in an official CSB policy to provide transparency for current and future staff. We request that CSB reconsider this recommendation and include a plan to issue a policy in its 60-day response to our final report. CSB disagreed with recommendation 6 and stated the OIG's assertions concerning the lack of co-location of files and incorrectly classified or coded investigation files have nothing to do with the completion of final written products. During our audit, for completed investigations as well as those ongoing, we could not locate the investigation status summary or what remains to be done on an investigation for the six sample investigations in our audit. We were informed of the purpose of the TRIM (Total Records and Information Management) system, as we explained in our report that it is used to store investigative evidence and not project management documents. Additional information related to investigation files—such as work plans, scoping documents and recommendation briefs—provide an ongoing investigation status summary of what happened, how it happened, what has been done, and what remains to be done with the timelines and preliminary recommendations to correct the problems found. Having all files relating to an investigation be located together provides for a complete review of the investigation status at any given time. In addition, correctly classified and coded investigative documents help ensure investigations are in accordance with records management or investigation protocol policies. We request that CSB reconsider this recommendation and provide a corrective action plan to collocate investigative files and correctly classify/code investigative documents in its 60-day response to our final report. 13-P-0337 16 ------- Status of Recommendations and Potential Monetary Benefits RECOMMENDATIONS POTENTIAL MONETARY BENEFITS (In $000s) Rec. No. Page No. Subject Status1 Planned Completion Action Official Date Claimed Amount Agreed-To Amount 14 Develop and implement performance indicators related to its first strategic performance goal and objective to complete timely investigations. Indicators should track and measure the efficiency of key phases of the investigation process and clarify the definition of a "timely" completed investigation. Also, address the indicators in the investigation protocol policy. 14 Revise and publish an annual action plan to comply with GPRA 2010 and update related individual performance plans to ensure that performance indicators are addressed and investigative staff are held accountable for performing key phases in the investigation process. 14 Review investigations open for more than 3 years and develop a plan to close out those investigations. 14 Develop and implement a succession or retention policy to help with any future effects of the turnover rate on CSB's mission. 14 As a best practice, involve staff in the planning process of an investigation. Hold meetings between senior management and staff to address any concerns with the investigation process. 14 Review and collocate investigation files for each ongoing investigation to ensure that they contain all the supporting documents related to the investigation. At a minimum, ensure files have proper classifications, project plans, scoping documents and board decisions. 15 Implement and update the records management policy to ensure that the classification of electronic investigation files agrees with the investigation protocol policy and staffs perform internal reviews of records as required by the policy. Chairperson, U.S. Chemical Safety and Hazard Investigation Board Chairperson, U.S. Chemical Safety and Hazard Investigation Board Chairperson, U.S. Chemical Safety and Hazard Investigation Board Chairperson, U.S. Chemical Safety and Hazard Investigation Board Chairperson, U.S. Chemical Safety and Hazard Investigation Board Chairperson, U.S. Chemical Safety and Hazard Investigation Board Chairperson, U.S. Chemical Safety and Hazard Investigation Board December 2013 December 2013 December 2013 December 2013 15 Update the investigation protocol policy for all current investigation procedures to include scoping documents and recommendation briefs. Provide formal training to the investigative staff on changes and updates to the investigative process. Chairperson, U.S. Chemical Safety and Hazard Investigation Board December 2013 13-P-0337 17 ------- RECOMMENDATIONS POTENTIAL MONETARY BENEFITS (In $000s) Rec. No. Page No. Subject Status1 Planned Completion Action Official Date 15 Provide guidelines for staff to determine the type of final product in the beginning of the investigation process to help staff be more efficient in completing investigations. Chairperson, U.S. Chemical Safety and Hazard Investigation Board December 2013 Claimed Amount Agreed-To Amount 1 0 = recommendation is open with agreed-to corrective actions pending C = recommendation is closed with all agreed-to actions completed U = recommendation is unresolved with resolution efforts in progress 13-P-0337 18 ------- U.S. Chemical Safety and Hazard Investigation Board 2175 K Street, NW • Suite 650 • Washington, DC 20037-1809 Phone: (202) 261-7600 • Fax: (202) 261-7650 www.csb.gov Appendix A June 21, 2013 Honorable Arthur A. Elkins, Jr. Inspector General U.S. Environmental Protection Agency Washington, DC 20460 Re: U.S. Chemical Safety and Hazard Investigation Board (CSB) Response to EPA OIG Report OA-FY12-0513 Dear Mr. Elkins: Thank you for the opportunity to review and comment on the draft OIG Report OA-FY12- 0513, "U.S. Chemical Safety and Hazard Investigation Board Needs to Complete Investigations More Timely," dated May 8, 2013. The CSB has completed its review and offers the following comments. In general terms, we agree with the majority of the recommendations, and note that much of what has been recommended here reflects work that is already in progress at the agency. Generally, we believe that the overwhelming factor in how quickly investigations can be completed is the agency's staffing level and the constraints on the agency's budget, which following sequestration is less than $10.6 million and has remained largely stagnant, after adjusting for inflation, for more than a decade. This major issue, which has weighed upon every CSB chair since the agency opened its doors in 1998, is barely acknowledged in the current draft, which focuses instead on what we regard as peripheral issues such as turnover rates and case file organization. Staffing and budget limitations mean that in response to new and unforeseen chemical disasters, our very tiny staff of about 20 investigators is constantly being pulled off existing projects and redeployed to new cases. This unrelenting process of redeployment is inefficient but inevitable. Currently at least half of the agency's workload - including the largest cases such as Deepwater Horizon, West Fertilizer, Chevron, and Tesoro - has been dictated either by the statutory mandate to investigate all public fatalities or serious injuries, and/or by requests from Congressional members and committees. We continue to have concerns about the draft's assertions concerning investigator turnover, and its potential impact on timeliness of investigations. In a very small agency, it is indisputable that unexpected turnover can have negative impacts on the completion of 13-P-0337 19 ------- U.S. Chemical Safety and Hazard investigation Board specific projects where individual investigators may have developed unique or unusual knowledge. For that reason and others, I am fully supportive of developing and retaining our highly talented and committed investigative staff, and am continually seeking to minimize voluntary turnover. I believe we have the best staff in place today of any time in the agency's history, and I want to keep these professionals. However. I have significant concerns about how the draft report seeks to portray the amount of investigative turnover as well as the reasons behind it. The draft report continues to allege that the CSB has "an average investigative staff turnover rate of 15 percent,"" A disturbing example - which my staff has pointed out repeatedly to DIG personnel, apparently to no avail - is that the draft continues to count a CSB investigator who died suddenly of natural causes in 2009 as part of its "turnover" statistics. Yet this tragic and unforeseeable death remains part of the OKI's tally. The OlG's calculations do not follow common HR practices for determining turnover and its effect on an organization. There are three main types of turnover relevant here: involuntary, voluntary, and unavoidable. Involuntary turnover occurs when an employee leaves the agency due to poor performance or fit with the agency. Voluntary turnover means the agency is losing an employee whose performance, skills and qualifications are valuable resources. Unavoidable turnover is where there is a change of life situation which the agency has no control over that results in a loss of an employee; examples include death or major life changes (e.g. pregnancy/child rearing decisions, enrollment in higher education), These critical distinctions are absent from the OIG report. Although turnover often has a negative connotation, turnover is not always a negative event. Involuntary turnover is one example. This turnover event opens a spot for a potential employee whose performance meets or exceeds expectations. In this case, involuntary turnover is desirable because poor job performance, poor teamwork, or deficient conduct are very costly, especially in a small organization - thus, replacing a poor performer with an employee who does his/her job can improve the agency's performance dramatically. Desirable involuntary turnover also has the added benefit of infusing new talent and skills into the agency workforce with replacement personnel, providing opportunities for the introduction of new ideas and approaches. These important benefits must be weighed against the costs of losing any existing project knowledge. 13-P-0337 20 ------- U.S. Chemical Safety and Hazard investigation Board Therefore the industry standard way to measure the agency's turnover is represented in following chart: FY Vol In Vol Una void On Board ! Vol % In Vol % Unavoid% 2008 3 0 0 16 | 19% 0% 0% 2009 1 ~i 1 20 5% 10% 5% 2010 0 0 I 21 0% 0% 5% 2011 3 0 1 21 14% 0% 5% 2012 2 1 0 22 9% 5% j 0% 5 Year Avg 9% 3% | 3% Instead of a live year average turnover rate of 15% as claimed by the OIG, the agency in reality experienced a 9% voluntary turnover rate for our investigators and the rate has been declining. Over the past eight months, there has been no investigative turnover whatsoever, which I believe is a tribute to agency managers' efforts at training, development, and retention. Comparing the CSB's turnover percentages to turnover data obtained from the Society for Human Resources indicates the agency's live-year average from 2008-2012 remains below the industry average for both voluntary (10% vs. 9% CSB) and involuntary turnover (7% vs. 3% CSB). Based on this information the CSB requests that this section of the draft report be removed, because there is no factual basis for asserting that (a) the CSB's level of turnover is unusually high, or (b) a high turnover level is a significant factor in the overall length of time for completing reports. At a minimum, the OIG should use the chart above, which is at least a factually accurate statement of how much turnover has occurred, and its type. The CSB offers the following responses to nine recommendations listed in the report: Recommendation 1 - Develop and implement performance indicators related to its first strategic performance goal and objective to complete timely investigation (sic). Indicators should track and measure the efficiency of ke> phases of the investigation process and clarify the definition of a "timely" completed investigation. Also, address the indicators in the im estigation protocol policy The CSB agrees with this recommendation and will review the live objectives from the 2012-2016 Strategic Plan related to Strategic Goal 1 to develop and implement performance indicators. In addition the CSB will also take a look at the remaining eight objectives relating to Goals 2 and 3 to develop performance indicators for those goals as well. The CSB continually strives to complete its open investigations in a timely manner. However, the CSB's definition of "timely" completed investigation is subject to change based on the deployment to a new incident. With our limited staff, all investigators are juggling multiple cases simultaneously. When new incidents occur, this results in shitting investigators" focus onto new cases and slowing the completion of existing ones. 13-P-0337 21 ------- U.S. Chemical Safety and Hazard Investigation Board In addition, the various investigations span a broad range of industries and hazards; while the proximate cause of one incident may become clear within days, others may require many months of complex study. In addition, cooperation by other parties is highly variable and has a dramatic effect on the speed of completion. The CSB currently estimates completing an investigation as follows: Major Investigation: 1.5-3 years Case Study: 1-1.5 years Safety Bulletin: 1 year The CSB will review past history and on average the length of time it takes to complete various phases of an investigation (i.e., deployment, writing, and review process) and will further consider how best to define "timely" investigations - bearing in mind that there are vast differences in complexity and effort among different cases. The CSB is analyzing key investigation metrics such as investigator hours, costs and elapsed days lo develop performance indicators for various investigation product types. These indicators will be incorporated in the Investigation Product Development and Review procedure of the investigation protocol, which will provide timelines for key milestones. Given staff resources and the investigation workload we expect to provide the Board with a draft Investigation Product Development and Review procedure for consideration and approval by December 31,2013. Recommendation 2 - Revise and publish an annual action plan to comply with GPRA 2010 and update related individual performance plans to ensure that performance indicators are addressed and investigative staff are held accountable for performing ke\ phases in the investigation process. From our review the only reports required for publishing under GPRA are Strategic Plans, Annual Performance Plans and Annual Performance Reports. The CSB has published an up-to-date Strategic Plan, annual performance-based budgets, and annual performance reports, which we believe meet the requirements of GPRA. The CSB considers its "action plans" as internal,, evergreen documents that are developed annually and updated periodically through the year to track initiatives as we strive to accomplish the goals set in our Strategic Plan. We consider action plans to be living documents that must be changed based on inherently unforeseeable incident deployments. The FY2014 action plan will be developed and updated during the upcoming fiscal year. Recommendation 3 - Review investigations open for more than 3 years and develop a plan to close out those investigations. The CSB agrees with the recommendation and provides the following update. Of the six open investigations listed in the 10 report, the status of each investigation is as follows: NDK - Final report draft in review cycle Caribbean Petroleum - Final report draft in review cycle 13-P-0337 22 ------- U.S. Chemical Safety and Hazard Investigation Board Silver Eagle - Close out plan was developed and will have to be revised in light of major West Fertilizer and Williams Olefins explosions Packaging Corporation of America - Hot work safety bulletin incorporating findings issued in March 2010; remaining case proposed for termination Cil'GO -- Urgent recommendations issued in December 2009; remaining case proposed for termination BP America Refinery Ultraeraeker Explosion - Proposed for termination Terminating certain older cases that have been idle for several years also responds to a stakeholder request from January 2013, allowing materials thai were gathered by the Board on these cases to become more readily available for external safety use through the Freedom of Information Act, in addition, the CSB has prepared detailed scoping documents for the completion of all currently open cases initiated after 2010 and selected prior cases, including the West Fertilizer, US Ink, Chevron, Deepwater Horizon/'Macondo, and Tesoro investigations. Other cases, such as the Horsehead Holding Co- investigation, will likely be proposed for termination. The CSB will agree to update the OIG on the status and plans for closure for all investigations at the end of calendar 2013, Recommendation 4 - Develop and implement a succession or retention policy to help with any future effects of the turnover rate on ( SB's mission. As discussed above, the CSB's turnover rates, when accurately and fairly calculated, are equal to or less than sector averages. The agency has implemented the following succession/retention initiatives and they have had a positive effect with our workforce, as demonstrated by the fact there has been zero investigative turnover during the past eight months: 1. Student loan payment program - the agency has implemented OPM "s program to assist employees paying back student loans. The employees participating in this program are required to commit to three years of service in exchange for the loan payments. 2. Supervisor development - the agency sends managers to at least one management/leadership development class a year in order to ensure they are up to date on the latest techniques for managing and developing our workforce, 3. Employee development - the agency has set up twice-a-year week long training sessions for our investigation and recommendations teams to continue the development of their skill sets. 4. Improved agency communication - the agency holds quarterly all hands meetings to ensure that the entire agency understands the developments of the previous quarter and the challenges for the next quarter. 13-P-0337 23 ------- U.S. Chemical Safety and Hazard Investigation Board 5. Workplace flexibility - the CSB has expanded opportunities for tclework, work from home, remote duty locations, and temporary leave during parenting to ensure the retention of key positions In addition, the agency is developing 01* considering the following initiatives that should have a positive effect on both succession and retention: a) Career ladder - the agency is creating a ladder for the development and advancement of investigation employees, from entry level to lead investigator (or other senior positions) b) New hire orientation - the agency is developing an orientation process for new investigators to ensure proper integration into the agency. e) Advance hiring - the agency will use historical vacancy and turnover data to project the number and location of new hires needed for each fiscal year. Recommendation 5 - As a best practice, imoh e staff in the planning process of an investigation. Hold meetings between senior management and staff to address any concerns with the investigation process. Investigation staff are already heavily involved and chiefly responsible for the planning of each investigation; this has always been the case. In the last two years this responsibility has been enhanced with the development of formal scoping documents for each active or new case, as well as recommendations briefs for major proposed recommendations. All these documents are then reviewed by the Board, In any event, the CSB agrees with this recommendation and has already undertaken a number of steps to address the issue. The protocol development team will hold three meetings with investigators and senior staff to identify any concerns with the investigation process. In addition, investigation teams will continue to hold Lessons Learned meetings after returning from a deployment and upon completion of a CSB written product and/or investigation case closure. The CSB has identified key investigation milestones as a first step in preparing the Investigation Product Development and Review procedure of the investigation protocol. Early milestones include scoping documents and project work plans, which will be developed by the investigation learn and reviewed by management and the Board: this has already been practiced for some time and has increased staff involvement in the planning process. Key investigation milestones also include report outlines, recommendations briefs, and interim public meetings that should provide opportunities to address concerns with the investigative process. We will provide the Board with a proposed scoping document template for consideration by August 31, 2013. This will be incorporated in the draft Investigation Product Development and Review procedure that, as stated above in our comments on recommendation 1, we expect to provide to the Board for consideration and approval by December 31, 2013. 13-P-0337 24 ------- U.S. Chemical Safety and Hazard Investigation Board Recommendation 6 - Km iew and collocate (sic) investigation files for each ongoing investigation to ensure that they contain all the supporting documents related to the investigation. At a minimum, ensure files have proper classifications, project plans, scoping documents and hoard decisions. The CSB disagrees with this recommendation. The OIG's assertions concerning the lack of co-location of files and incorrectly classified or coded investigation files have nothing to do with the completion of final written products. As we have stated on several occasions, the purpose of the TRIM electronic records management system is to hold evidentiary records for investigation team members while they conduct their investigation; TRIM provides word-search capabilities that streamline and enhance investigators1 abilities to review and analyze evidentiary materials. We use TRIM as a tool for searching/reviewing evidence during a case and as a system of archiving once the case is complete. The evergreen planning and scoping documents developed internally by CSB staff would not be included in such a system, Such evergreen documents are often not included in TRIM specifically because the team members do not need such records when reviewing evidence. Scoping/planning documents are management tools used by the team leads, supervisors, and agency leadership to keep abreast of the case; those sorts of evergreen records do not hinder or help the team in their review of evidence and, thus, any argument that their lack of placement in TRIM is unrelated to the team's ability to complete the investigations. Also, and very importantly, in the last few years, we have been using Optical Character Recognition (OCR) for most records; once documents are OCR scanned, they become fully word-searchable. The classification ID numbers are not needed if the files are OCR scanned and word searchable. There are many fields that can be used to find documents - - the ID field was an old field needed before docs could be OCR scanned. Thus, once OCR scanned, any categorization of the files beyond identification of the investigation the document pertains to become irrelevant, as the investigator can search/find the records needed using a variety of search data fields. We recommend corrections be made to page 17 of the draft report to reflect these facts. The OIG displays a critical misunderstanding of the TRIM system by the statements it makes in the final paragraph of the Investigation Files section: "Ensuring investigation files are collocated (sic) and correctly classified or coded would enable any investigator to easily identify the status of an investigation and compare results from completed investigations. Investigators could more efficiently complete or close out investigations in a timely manner, particularly cases open for more than 3 years." In no way would "'correctly classified or coded records" within TRIM enable anyone to easily identify the status of an investigation and thus allow individuals to close out cases in a more timely manner; it would merely provide an additional means to locate a specific evidence record within the TRIM evidence database. The coding of evidentiary materials and the need for project management of ongoing cases are two entirely separate issues. The OIG 13-P-0337 25 ------- U.S. Chemical Safety and Hazard Investigation Board incorrectly assumes that the coding of evidentiary materials somehow relates to the agency's ability to manage its investigative products. This correlation is not correct. Recommendation 7 - Implement and update the records management policy to ensure that I he classification of electronic investigation files agrees with the investigation protocol policy and staffs (sic) perform internal reviews of records as required by the policy. The CSB agrees with this recommendation and will review its Records Management policy and update it by December 31. 2013, to reflect the need for a full OCR scan of the evidentiary case file for each investigation as well as completeness of the case file for official closeout/archiving. Recommendation 8 - Update the investigation protocol policy for all current investigation procedures to include scoping documents and recommendation briefs. Provide formal training to the investigative staff on changes and updates to the investigative process. The CSB has been using formal scoping documents for investigations for the past two years and last year began developing selected recommendations briefs for major recommendations. All these tools have proven useful in completing cases that describe an agreed set of issues, The CSB agrees that a more formal policy and standard templates should be developed for these tools. We expect to provide the Board with a proposed scoping document template for consideration and approval by August 31, 2013. 1'his will be incorporated in the Investigation Product Development and Review procedure of the investigation protocol along with a template for recommendations briefs. As previously stated, we expect to provide the Board with a draft of this procedure for consideration and approval by December 31, 2013. We will then train investigative staff on the procedures within 90 days of Board approval Recommendation 9 - Provide guidelines for staff to determine the type of final product in the beginning of the investigation process to help staff tie more efficient in completing investigations. The Investigation Product Development and Review procedure of the investigation protocol will identify CSB investigation product types and their attributes. The procedure will also require that the scoping document include a recommended product type and required resources so that the investigation team, management, and the Board have the same expectations and goals for each investigation. Again, we expect to provide the Board with a proposed scoping document template by August 31, 2013, and draft of the Investigation Product Development and Review procedure for consideration and approval by December 31, 2013. 13-P-0337 26 ------- U.S. Chemical Safety and Hazard Investigation Board Thank you again for your courtesy in providing the draft report. Should you have questions, please contact our audit liaison, Anna Brown, at (202) 261-7639. Sincerely, f y t .* y - - ^ ^ ^ O—t- l^<_/ Rafael Moure-Eraso, Ph.D. Chairperson 13-P-0337 ------- U.S. Chemical Safety and Hazard Investigation Board 2175 K Street, NW • Suite 650 • Washington, DC 20037-1809 Phone: (202) 261-7600 • Fax: (202) 261-7650 www.csb.gov Mark A. Griffon Board Member Appendix B Honorable Arthur A. Elkins, Jr. Inspector General U.S. Environmental Protection Agency Washington, DC 20460 Re: U.S. Chemical Safety and Hazard Investigation Board (CSB) EPA OIG Report OA-FY12-0513 Dear Mr. Elkins: Thank you for the opportunity to review and comment on the OIG Report OA-FY12- 0513, "U.S. Chemical Safety and Hazard Investigation Board Needs to Complete Investigations More Timely," dated May 8, 2013. I believe that two factors contributing to the CSBs delay in completing investigations are staffing levels and budget constraints. The CSB budget has remained essentially level for the last decade and the agency continues to have to make difficult choices on whether to deploy to serious incidents that clearly warrant our attention. Notwithstanding the CSBs budget constraints, this report points out weaknesses that I believe can and should be improved upon. I appreciate the insights offered regarding improving the efficiency of the CSBs investigative process and I believe, on the whole, the recommendations were appropriate and I plan to work with agency leadership to ensure implementation. I would offer the following specific comments on your recommendations: 1. Recommendation 1: I agree with your recommendation to develop indicators that will track and measure the efficiency of the CSB investigations. My understanding from senior staff is that they plan to address this within a revision of the investigative protocol and I will urge that protocol be completed and adopted by the Board by the end of this calendar year. 2. Recommendation 2: I agree with the recommendation and will encourage the staff to revise and publish a Board approved Annual Action Plan in accordance with GPRA 2010 (for 2013 and future years). 3. Recommendation 3: I agree with this recommendation and I would recommend that agency management develop an overall agency plan for managing all open investigations for the Board's consideration and approval. 13-P-0337 28 ------- U.S. Chemical Safety and Hazard Investigation Board This plan should be continually updated and approved by the Board at least on an annual basis. 4. Recommendation 4: I am very concerned about the significant turnover of senior investigative staff. Staff with investigative experience, particularly experience with CSB approach to investigations, is essential in improving the timeliness and maintaining the quality of the CSB reports. To the extent that your team has further information as to the root cause of the turnover problem I believe it would be helpful in making improvements in this area. 5. Recommendations 5-9: I believe that the revision of the investigative protocol, mentioned above, should address most of the issues identified in these recommendations. Specifically, including a formal process for scoping investigations, defining investigative products and incorporating the use of a Lessons Learned process will greatly improve the efficiency and quality of the CSB investigations. Again, thank you for the opportunity to comment on your report and please contact me if you have any questions. Sincerely, Mark A. Griffon Board Member 13-P-0337 29 ------- Appendix C OIG Comparison of CSB Responses to the Draft Report OIG Recommendation CSB Chair, Moure-Eraso Response In Appendix A CSB Board Member Griffon Response In Appendix B Recommendation 1: Develoi) and implement performance indicators related to its first strategic performance goal and objective to complete timely investigation. Indicators should track and measure the efficiency of key phases of the investigation process and clarify the definition of a "timely" completed investigation. Also, address the indicators in the investigation protocol policy. The CSB agrees with this recommendation and will review the five objectives from the 2012-2016 Strategic Plan related to Strategic Goal 1 to develop and implement performance indicators. In addition the CSB will also take a look at the remaining eight objectives relating to Goals 2 and 3 to develop performance indicators for those goals as well. The CSB continually strives to complete its open investigations in a timely manner. However, the CSB's definition of "timely" completed investigation is subject to change based on the deployment to a new incident. With our limited staff, all investigators are juggling multiple cases simultaneously. When new incidents occur, this results in shifting investigators' focus onto new cases and slowing the completion of existing ones. In addition, the various investigations span a broad range of industries and hazards; while the proximate cause of one incident may become clear within days, others may require many months of complex study. In addition, cooperation by other parties is highly variable and has a dramatic effect on the speed of completion. The CSB currently estimates completing an investigation as follows: Major Investigation: 1.5-3 years Case Study: 1-1.5 years Safety Bulletin: 1 year The CSB will review past history and on average the length of time it takes to complete various phases of an I agree with your recommendation to develop indicators that will track and measure the efficiency of the CSB investigations. My understanding from senior staff is that they plan to address this within a revision of the investigative protocol and I will urge that protocol be completed and adopted by the Board by the end of the calendar year. 13-P-0337 30 ------- investigation (i.e., deployment, writing, and review process) and will further consider how best to define "timely" investigations -bearing in mind that there arc vast differences in complexity and effort among different cases. The GSB is analyzing key investigation metrics such as investigator hours, costs and elapsed days to develop performance indicators for various investigation product types. These indicators will be incorporated in the Investigation Product Development and Review procedure of the investigation protocol, which will provide timelines for key milestones. Given staff resources and the investigation workload we expect to provide the Board with a draft Investigation Product Development and Review procedure for consideration and approval by December 31, 2013. Recommendation 2: Revise and publish an annual action plan to comply with GPRA 2010 and update related individual performance plans to ensure that performance indicators are addressed and investigative staff are held accountable for performing key phases in the investigation process. From our review the only reports required for publishing under GPRA are Strategic Plans, Annual Performance Plans and Annual Performance Reports. The CSB has published an up-to-date Strategic Plan, annual performance-based budgets, and annual performance reports, which we believe meet the requirements of GPRA. The CSB considers its "action plans" as internal, evergreen documents that arc developed annually and updated periodically through the year to track initiatives as we strive to accomplish the goals set in our Strategic Plan. We consider action plans to be living documents that must be changed based on inherently unforeseeable incident deployments. The FY2014 action plan will be developed and updated during the upcoming fiscal year. I agree with the recommendation and will encourage the staff to revise and publish a Board approved Annual Action Plan in accordance with GPRA 2010 (for 2013 and future years). Recommendation 3: Review investigations open for more than 3 years and develop a plan to close out those investigations The CSB agrees with the recommendation and provides the following update. Of the six open investigations listed in the IG report, the status of each investigation is as follows: NDK - Final report draft in review cycle Caribbean Petroleum - Final report draft in review cycle I agree with this recommendation and I would recommend that agency management develop an overall agency plan for managing all open investigations for the Board's consideration and approval. This plan should be continually updated and approved by the Board at least on an annual basis 13-P-0337 31 ------- Silver Eagle- Close out plan was developed and will have to be revised in light of major West Fertilizer and Williams Olefins explosions Packaging Corporation of America - Hot work safety bulletin incorporating findings issued in March 201 0; remaining case proposed for termination CrrGO - Urgent recommendations issued in December 2009; remaining case proposed for termination BP America Refinery Ultracracker Explosion - Proposed for termination Terminating certain older cases that have been idle for several years also responds to a stakeholder request from January 2013, allowing materials that were gathered by the Board on these cases to become more readily available for external safety use through the Freedom of Information Act. In addition, the CSB has prepared detailed seeping documents for the completion of all currently open cases initiated after 2010 and selected prior cases, including the West Fertilizer, US Ink, Chevron, Deepwater Hori/on/Macondo, and Tesoro investigations. Other cases, such as the Horsehead Holding Co. investigation, will likely be proposed for termination. The CSB will agree to update the OIG on the status and plans for closure for all investigations at the end of calendar 20 13. Recommendation 4: DeveloD and implement a succession or retention policy to help with any future effects of the turnover rate on CSB's mission. As discussed above, [Appendix A] the CSB's turnover rates, when accurately and fairly calculated, are equal to or less than sector averages. The agency has implemented the following succession/retention initiatives and they have had a positive effect with our workforce, as demonstrated by the fact there has been zero investigative turnover during the past eight months: I am very concerned about the significant turnover of senior investigative staff. Staff with investigative experience, particularly experience with CSB approach to investigations, is essential in improving the timeliness and maintaining the quality of the CSB reports. To the extent that your team has further information as to the root cause of the turnover 13-P-0337 32 ------- 1. Student loan payment program - the agency has implemented OPM's program to assist employees paying back student loans. The employees participating in this program are required to commit to three years of service in exchange for the loan payments. 2. Supervisor development - the agency sends managers to at least one management/leadership development class a year in order to ensure they are up to date on the latest techniques for managing and developing our workforce. 3. Employee development- the agency has set up twice-a- year weeklong training sessions for our investigation and recommendations teams to continue the development of their skill sets. 4. Improved agency communication- the agency holds quarterly all hands meetings to ensure that the entire agency understands the developments of the previous quarter and the challenges for the next quarter. 5. Workplace flexibility - the CSB has expanded opportunities for telework, work from home, remote duty locations, and temporary leave during parenting to ensure the retention of key positions In addition, the agency is developing or considering the following initiatives that should have a positive effect on both succession and retention: a) Career ladder - the agency is creating a ladder for the development and advancement of investigation employees, from entry level to lead investigator (or other senior positions) b) New hire orientation- the agency is developing an orientation process for new investigators to ensure proper integration into the agency. 13-P-0337 problem I believe it would be helpful in making improvements in this area. 33 ------- c) Advance hiring - the agency will use historical vacancy and turnover data to project the number and location of new hires needed for each fiscal year. Recommendation 5: As a best practice, involve staff in the planning process of an investigation. Hold meetings between senior management and staff to address any concerns with the investigation process. Investigation staff are already heavily involved and chiefly responsible for the planning of each investigation; this has always been the case. In the last two years this responsibility has been enhanced with the development of formal scoping documents for each active or new case, as well as recommendations briefs for major proposed recommendations. All these documents are then reviewed by the Board. In any event, the CSB agrees with this recommendation and has already undertaken a number of steps to address the issue. The protocol development team will hold three meetings with investigators and senior staff to identify any concerns with the investigation process. In addition, investigation teams will continue to hold Lessons Learned meetings after returning from a deployment and upon completion of a CSB written product and/or investigation case closure. The CSB has identified key investigation milestones as a first step in preparing the Investigation Product Development and Review procedure of the investigation protocol. Early milestones include scoping documents and project work plans which will be developed by the investigation team and reviewed by management and the Board; this has already been practiced for some time and has increased staff involvement in the planning process. Key investigation milestones also include report outlines, recommendations briefs, and interim public meetings that should provide opportunities to address concerns with the investigative process. We will provide the Board with a proposed scoping document template for consideration by August 31, 2013. This will be incorporated in the draft Investigation Product Development and Review procedure that, as stated above in our comments on recommendation 1, we expect to provide to the Board for consideration and approval by December 31, 2013. I believe that the revision of the investigative protocol, mentioned above, should address most of the issues identified in these recommendations. Specifically, including a formal process for scoping investigations, defining investigative products and incorporating the use of a Lessons Learned process will greatly improve the efficiency and quality of the CSB investigations. 13-P-0337 34 ------- Recommendation 6: Review and collocate investigation files for each ongoing investigation to ensure that they contain all the supporting documents related to the investigation. At a minimum, ensure files have proper classifications, project plans, scoping documents and board decisions. The CSB disagrees with this recommendation. The OIG's assertions concerning the lack of co-location of files and incorrectly classified or coded investigation files have nothing to do with the completion of final written products. As we have stated on several occasions, the purpose of the TRIM electronic records management system is to hold evidentiary records for investigation team members while they conduct their investigation; TRIM provides word-search capabilities that streamline and enhance investigators' abilities to review and analyze evidentiary materials. We use TRIM as a tool for searching/reviewing evidence during a case and as a system of archiving once the case is complete. The evergreen planning and scoping documents developed internally by CSB staff would not be included in such a system. Such evergreen documents are often not included in TRIM specifically because the team members do not need such records when reviewing evidence. Scoping/planning documents arc management tools used by the team leads, supervisors, and agency leadership to keep abreast of the case; those sorts of evergreen records do not hinder or help the team in their review of evidence and. thus, any argument that their lack of placement in TRIM is unrelated to the team's ability to complete the investigations. Also, and veiy importantly, in the last few years, we have been using Optical Character Recognition (OCR) for most records; once documents are OCR scanned, they become fully word-searchable. The classification ID numbers are not needed if the files are OCR scanned and word searchable. There are many fields that can be used to find documents ™ the ID field was an old field needed before docs could be OCR scanned. Thus, once OCR scanned, any categorization of the files beyond identification of the investigation the document pertains to become irrelevant, as the investigator can search/find the records needed using a variety of search data fields. We recommend corrections be made to page 17 of the draft report to reflect these facts. 13-P-0337 I believe that the revision of the investigative protocol, mentioned above, should address most of the issues identified in these recommendations. Specifically, including a formal process for scoping investigations, defining investigative products and incorporating the use of a Lessons Learned process will greatly improve the efficiency and quality of the CSB investigations. 35 ------- The OIG displays a critical misunderstanding of the TRIM system by the statements it makes in the final paragraph of the Investigation Files section: "Ensuring investigation files are collocated (sic) and correctly classified or coded would enable any investigator to easily identify the status of an investigation and compare results from completed investigations. Investigators could more efficiently complete or close out investigations in a timely manner, particularly cases open for more than 3 years." In no way would "correctly classified or coded records" within TRIM enable anyone to easily identify the status of an investigation and thus allow individuals to close out cases in a more timely manner; it would merely provide an additional means to locate a specific evidence record within the TRIM evidence database. The coding of evidentiary materials and the need for project management of ongoing cases are two entirely separate issues. The OIG incorrectly assumes that the coding of evidentiary materials somehow relates to the agency's ability to manage its investigative products. This correlation is not correct. Recommendation 7: I mi)lenient and update the records management policy to ensure that the classification of electronic investigation files agrees with the investigation protocol policy and staffs perform internal reviews of records as required by the policy. The CSB agrees with this recommendation and will review its Records Management policy and update it by December 31, 2013, to reflect the need for a full OCR scan of the evidentiary case file for each investigation as well as completeness of the case file for official closeout/archiving. I believe that the revision of the investigative protocol, mentioned above, should address most of the issues identified in these recommendations. Specifically, including a formal process for scoping investigations, defining investigative products and incorporating the use of a Lessons Learned process will greatly improve the efficiency and quality of the CSB investigations. Recommendation 8: IIndate the investigation protocol policy for all current investigation procedures to include scoping documents and recommendation briefs. Provide formal training to the investigative staff on changes and updates to the investigative process. The CSB has been using formal scoping documents for investigations for the past two years and last year began developing selected recommendations briefs for major recommendations. All these tools have proven useful in completing cases that describe an agreed set of issues. The CSB agrees that a more formal policy and standard templates should be developed for these tools. We expect to provide the Board with a proposed scoping document template for consideration and approval by August 31, I believe that the revision of the investigative protocol, mentioned above, should address most of the issues identified in these recommendations. Specifically, including a formal process for scoping investigations, defining investigative products and incorporating the use of a Lessons Learned process will greatly improve the efficiency and quality of the CSB investigations. 13-P-0337 36 ------- 2013.This will be incorporated in the Investigation Product Development and Review procedure of the investigation protocol along with a template for recommendations briefs. As previously stated, we expect to provide the Board with a draft of this procedure for consideration and approval by December 31, 2013. We will then train investigative staff on the procedures within 90 days of Board approval. Recommendation 9: Provide guidelines for staff to determine the type of final product in the beginning of the investigation process to help staff be more efficient in completing investigations. The Investigation Product Development and Review procedure of the investigation protocol will identify CSB investigation product types and their attributes. The procedure will also require that the scoping document include a recommended product type and required resources so that the investigation team, management, and the Board have the same expectations and goals for each investigation. Again, we expect to provide the Board with a proposed scoping document template by August 31, 2013, and draft of the Investigation Product Development and Review procedure for consideration and approval by December 31. 2013. I believe that the revision of the investigative protocol, mentioned above, should address most of the issues identified in these recommendations. Specifically, including a formal process for scoping investigations, defining investigative products and incorporating the use of a Lessons Learned process will greatly improve the efficiency and quality of the CSB investigations. 13-P-0337 37 ------- Appendix D Distribution Chairperson and Chief Executive Officer, U.S. Chemical Safety and Hazard Investigation Board Board Members, U.S. Chemical Safety and Hazard Investigation Board Managing Director, U.S. Chemical Safety and Hazard Investigation Board Deputy Managing Director, U.S. Chemical Safety and Hazard Investigation Board General Counsel, U.S. Chemical Safety and Hazard Investigation Board Senior Counselor to the Chair, U.S. Chemical Safety and Hazard Investigation Board Director of Administration and Audit Liaison, U.S. Chemical Safety and Hazard Investigation Board Communications Manager, U.S. Chemical Safety and Hazard Investigation Board 13-P-0337 38 ------- |