* Ck \
I SKI
V
-------
Report Contributors: Richard Eyermann
Michael Davis
Gail Saunders
Gloria Taylor-Upshaw
Anthony Grear
Vanessa Rodriguez-Moya
Myka Sparrow
Abbreviations
CSB
U.S. Chemical Safety and Hazard Investigation Board
EPA
U.S. Environmental Protection Agency
FY
Fiscal year
GPRA
Government Performance and Results Act of 1993
NTSB
National Transportation Safety Board
OIG
Office of Inspector General
OMB
Office of Management and Budget
OSHA
Occupational Safety and Health Administration
TRIM
Total Records and Information Management
Cover photo: Damage resulting from a runaway chemical reaction at the Bayer Crop Science
facility in Institute, West Virginia. (CSB photo)
Hotline
To report fraud, waste, or abuse, contact us through one of the following methods:
e-mail: OIG Hotiirie@epa.aov write: EPA Inspector General Hotline
phone: 1-888-546-8740 1200 Pennsylvania Avenue NW
fax: 202-566-2599 Mailcode 2431T
online: http://www.epa.gov/oig/hotline.htm Washington, DC 20460
-------
^tDsrx
* O \
\jSkj
U.S. Environmental Protection Agency
Office of Inspector General
At a Glance
12-P-0724
August 22, 2012
Why We Did This Review
We initiated this audit to
determine what factors impede
implementation of U.S. Chemical
Safety and Hazard Investigation
Board (CSB) safety
recommendations. CSB was
created under the Clean Air Act
Amendments of 1990 to
investigate industrial chemical
accidents. CSB issues
recommendation reports to
government agencies,
companies, trade associations,
labor unions, and other groups.
The reports contain specific,
measurable safety
recommendations designed to
prevent future accidents. In 2004,
CSB created the Office of
Recommendations to work with
recipients to pursue closure of
safety recommendations by
recipients' taking acceptable
actions.
Furthering CSB's Goals
• Improve safety and
environmental protection by
ensuring that CSB's
recommendations are
implemented and by broadly
disseminating CSB's findings
through advocacy and
outreach.
For further information, contact
our Office of Congressional and
Public Affairs at (202) 566-2391.
The full report is at:
www.epa.gov/oiq/reports/2012/
20120822-12-P-0724.pdf
U.S. Chemical Safety and Hazard Investigation
Board Should Improve Its Recommendations
Process to Further Its Goal of Chemical
Accident Prevention
What We Found
CSB did not consistently achieve its goals and standards, as outlined in its
current strategic plan, for timely implementation of its safety recommendations.
As of December 2010, CSB had issued 588 safety recommendations, of which
218 (37 percent) were open while actions were in progress to resolve them. Of
the 218 recommendations, 54 (nearly 25 percent) were open for more than 5
years. Although CSB does not have enforcement authority, and implementation
of some of its recommendations may face lengthy regulatory processes, CSB
has not established or maintained sufficient internal controls and processes for
safety recommendations. Such internal controls could include updating board
orders and making full use of CSB's information management system. Without
effective controls and efficient processes, there is an increased likelihood that
recipients will not timely implement CSB safety recommendations and chemical
accidents may not be prevented to the greatest extent possible.
Recommendations and Planned Agency Corrective Actions
We recommend that the CSB Chairperson update board orders that establish
policies for the Recommendation Program, timeliness of board votes, and
coordination between Offices of Investigation and Recommendation. We also
recommend that the Chairperson make full use of the TRIM (Total Records and
Information Management) system and implement a formal advocacy program for
safety recommendation implementation.
CSB concurred with all our recommendations except one involving calendaring
notation items, and we consider that recommendation unresolved and are
working toward a resolution. CSB has redrafted Board Order 022, to improve the
data quality of its recommendation information. CSB plans to update Board Order
040 to enhance collaboration between investigations and recommendations
personnel.
Noteworthy Achievements
CSB established Board Order 046 to develop a most-wanted program to
advocate implementation of its safety recommendations. Also, CSB implemented
TRIM, a commercial electronic data storage and management system, to enable
CSB to track workflow activities related to recommendations as well as other
organizational activities.
-------
^£Dsr%
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
THE INSPECTOR GENERAL
August 22, 2012
MEMORANDUM
SUBJECT: U.S. Chemical Safety and Hazard Investigation Board Should Improve Its
Recommendations Process to Further Its Goal of Chemical Accident Prevention
This is our report on the U.S. Chemical Safety and Hazard Investigation Board's (CSB's)
recommendation process conducted by the Office of Inspector General (OIG) of the U.S.
Environmental Protection Agency. This report contains findings the OIG has identified and
corrective actions the OIG recommends. This report represents the opinion of the OIG and does
not necessarily represent the final CSB position on the subjects reported. CSB managers will
make final determination on matters in this report in accordance with established audit resolution
procedures.
Action Required
CSB disagreed with recommendation lb, which is unresolved with resolution efforts in progress.
CSB agreed with recommendation la, but we request that CSB confirm the date the Board
approved revisions to Board Order 022. CSB provided an acceptable corrective action plan for
the remaining recommendations, which are in an open status. Therefore, please provide a written
response to recommendation 1, including a proposed corrective action plan, within 90 calendar
days of the report date. In addition, in your 90-day response, you may update the OIG on the
implementation status of the agreed-to corrective actions for the other recommendations. The
response will be posted on the OIG's public website, along with our memorandum commenting
on the response.
Report No. 12-P-0724
FROM: Arthur A. Elkins, Jr.
TO:
The Honorable Rafael Moure-Eraso, Ph.D.
Chairperson and Chief Executive Officer
U.S. Chemical Safety and Hazard Investigation Board
The response should be provided as an Adobe PDF file that complies with the accessibility
requirements of Section 508 of the Rehabilitation Act of 1973, as amended. The final response
-------
should not contain data that should not be released to the public; if the response contains such
data, the data for redaction or removal should be identified. We have no objections to the further
release of this report to the public. We will post this report on our website at
http: //www. ep a. go v/o i g.
If you or your staff have any questions regarding this report, please contact Michael Davis,
Product Line Director, at (513) 487-2363 or davis.michaeld@epa. gov: or Gloria Taylor-Upshaw,
Project Manager, at (404) 562-9842 or tavlor-upshaw.gloria@epa.gov.
-------
U.S. Chemical Safety and Hazard Investigation Board
Should Improve Its Recommendations Process to
Further Its Goal of Chemical Accident Prevention
12-P-0724
Table of Contents
Chapters
1 Introduction 1
Purpose 1
Background 1
Noteworthy Achievements 2
Scope and Methodology 3
2 CSB Should Improve Internal Controls and
Recommendations Processes 5
Laws, Guidance, and Policy Require Timely Implementation of
Safety Recommendations 5
CSB Recommendations Not Always Closed Within Prescribed Time 7
CSB Needs Effective and Efficient Internal Controls 8
Other Impediments to Timely Implementation 13
Conclusion 13
Recommendations 14
CSB Comments and OIG Evaluation 15
3 Status of Recommendations and Potential Monetary Benefits 17
Appendices
A Details on Scope and Methodology 18
B CSB Response to Draft Report 20
C Distribution 34
-------
Chapter 1
Introduction
Purpose
We initiated this audit to determine what factors impede implementation of
the U.S. Chemical Safety and Hazard Investigation Board's (CSB's) safety
recommendations. The U.S. Environmental Protection Agency's (EPA's)
Office of Inspector General (OIG) is the Inspector General for CSB.
Background
CSB is an independent federal agency, authorized by the Clean Air Act
Amendments of 1990. The Act directs CSB to (1) investigate and report
on the cause or probable cause of any accidental chemical release resulting
in a fatality, serious injury, or substantial property damage; (2) make
safety recommendations to reduce the likelihood or consequences of
accidental chemical releases and propose corrective measures; and
(3) establish regulations for reporting accidental releases. CSB became
operational in January 1998.
Congress directed that CSB's investigative function be completely
independent of the rulemaking, inspection, and enforcement authorities of
EPA and the Occupational Safety and Health Administration (OSHA).
CSB states on its website, from Legislative History - Senate Report No.
101-228, that:
. . . the investigations conducted by agencies with dual
responsibilities tend to focus on violations of existing rules
as the cause of the accident almost to the exclusion of other
contributing factors for which no enforcement or
compliance actions can be taken. The purpose of an
accident investigation (as authorized here) is to determine
the cause or causes of an accident whether or not those
causes were in violation of any current or enforceable
requirement.
CSB's authorizing statute provides for five board members, including a
chairperson, all appointed by the President of the United States. As of
December 2010, there were five appointed board members, including the
chairperson, and a professional staff of 39. As of December 2011,
professional staff was at the same level, and the board had three appointed
members.
12-P-0724
1
-------
CSB Investigates Chemical Accidents and Issues
Recommendations
CSB investigations consider all aspects of chemical accidents, including
physical causes such as equipment failures, inadequacies in regulations,
industry standards, and safety management systems. CSB's written
products contain specific, measurable safety recommendations designed to
prevent future accidents. Recommendations are suggestions for actions
made to specific parties, and they are based on the lessons derived from
each investigation or study. The board makes safety recommendations to
government agencies, companies, trade associations, labor unions, and
other groups.
CSB does not have authority to enforce implementation of its safety
recommendations. According to the Clean Air Act Amendments of 1990,
one of CSB's roles is "recommending measures to reduce the likelihood or
the consequences of accidental releases and proposing corrective steps to
make chemical production, processing, handling and storage as safe and
free from risk of injury as is possible." CSB employs various strategies to
secure implementation of its safety recommendations but does not have a
formal advocacy program, and information technology is not used to its
maximum extent to improve operations. CSB closes its safety
recommendations for various reasons, such as when recipients take
acceptable action.
CSB Created an Office of Recommendations
CSB's Board Order 022, CSB Recommendation Program, was adopted in
December 2001. According to CSB, recommendations activities were
secondary to investigative work for CSB's first several years, with very
limited resources devoted to recommendation tracking, advocacy, and
closure. In 2004, CSB established a permanent Office of Recommendations,
and staffed the office from mid-2004 to 2005. The priorities in the first
years were to follow up and close a backlog of open recommendations. CSB
noted that it began using an electronic database for investigation records in
2005, and beginning in 2006 existing recommendations records were
scanned into the database. As of December 2011, the office had four staff
members, representing approximately 10 percent of the agency's staff. One
of the staff members dedicates 20 percent of his time to functions other than
those of the Office of Recommendations.
Noteworthy Achievements
Despite not having enforcement authority, CSB has been relatively
successful in encouraging implementation of its recommendations. CSB
met with the National Transportation Safety Board (NTSB), another
12-P-0724
2
-------
agency that makes recommendations to industries and regulatory agencies,
to gain an understanding of NTSB's advocacy program. NTSB's advocacy
program includes a "most-wanted list" that highlights safety issues
identified from accident investigations. NTSB uses the list to increase
industry, congressional, and public awareness about these priority issues
and recommended safety solutions. As a result of meeting with NTSB,
CSB began developing, but has not yet implemented, a most-wanted list
program to advocate for safety recommendation implementation. CSB
noted, in its response to our draft report, the approval of Board Order 046,
Most Wanted Chemical Safety Improvements Program, and indicated it
plans to select the most wanted issues in July 2012.
CSB implemented TRIM (Total Records and Information Management),
a commercial electronic data storage and management system, to enable
CSB to track workflow activities related to recommendations as well as
other organizational activities.
Scope and Methodology
We conducted this performance audit in accordance with generally
accepted government auditing standards. Those standards require that we
plan and perform the audit to obtain sufficient, appropriate evidence to
provide a reasonable basis for our findings and conclusions based on our
audit objective. We believe that the evidence obtained provides a
reasonable basis for our findings and conclusions based on our audit
objective.
We performed our audit work from January 2011 through May 2012.
The scope of the audit included all safety recommendations issued from
September 1998 to December 2010.
We analyzed CSB's recommendations data to determine whether data
quality and internal controls may have an impact on CSB's ability to get
recommendations implemented. We found that CSB has not established
sufficient internal controls to promote effective and efficient program
operations. We tested and assessed CSB's internal control structure related
to ensuring effective and efficient operations and compliance with
applicable laws and regulations.
We interviewed CSB's managing director, office directors, and Office of
Recommendations staff members to identify and discuss the factors that
have hindered implementation of CSB's safety recommendations. We
obtained and reviewed CSB's board orders and standard operating
procedures that govern and affect safety recommendations. We assessed
their efficiency and effectiveness. Board orders are CSB's policies and
procedures.
12-P-0724
3
-------
We developed 13 questions that we provided to CSB recommendation
recipients with open recommendations to ascertain why they have not
implemented the safety recommendations. We conducted a benchmark
analysis with NTSB to identify best practices that could assist CSB in
gaining implementation of safety recommendations. We also attended, as
observers, a meeting between CSB and OSHA to gain an understanding of
OSHA's rulemaking and standards development process.
For additional details on our scope and methodology, see appendix A.
Prior EPA OIG Audit
The EPA OIG assumed OIG oversight responsibility for CSB in fiscal
year (FY) 2004. Previously, the Federal Emergency Management Agency
and U.S. Department of Homeland Security OIGs had performed OIG
oversight for CSB. In EPA OIG Report No. ll-P-0115, Chemical Safety
and Hazard Investigation Board Did Not Take Effective Corrective
Actions on Prior Audit Recommendations, issued February 15, 2011, we
sought to determine whether CSB had implemented audit
recommendations made by those three OIGs and the U.S. Government
Accountability Office. We found that CSB did not take timely corrective
actions to address audit recommendations. Also, CSB had not established
and implemented a management control program to evaluate and report on
the effectiveness of controls related to its program operations.
12-P-0724
4
-------
Chapter 2
CSB Should Improve Internal Controls and
Recommendations Processes
CSB did not consistently achieve its goals and standards, as outlined in its
current strategic plan, for timely implementation of its safety
recommendations. As of December 2010, CSB had issued 588 safety
recommendations, of which 218 (37 percent) were open while efforts were
in progress to resolve them. Of the 218 recommendations, 54
(nearly 25 percent) were open for more than 5 years. The Government
Performance and Results Act of 1993 (GPRA) requires federal agencies to
have strategic plans, and Office of Management and Budget (OMB)
Circular A-123, Management's Responsibility for Internal Controls,
requires policies and procedures to ensure effective and efficient internal
controls. Although CSB does not have enforcement authority, and
implementation of some of its recommendations may face lengthy
regulatory processes, CSB has not established or maintained sufficient
internal controls. Such internal control activities could include updating
board orders and processes related to safety recommendations. Without
effective controls and efficient processes, there is an increased likelihood
that recipients will not timely implement CSB safety recommendations
and chemical accidents may not be prevented to the greatest extent
possible.
Laws, Guidance, and Policy Require Timely Implementation of
Safety Recommendations
Federal laws, including the Clean Air Act Amendments of 1990 under
which CSB was created, GPRA, OMB Circular A-123, and OMB
Memorandum M-10-24, require that CSB be effective in achieving results.
Section 2 of GPRA includes, as purposes of the Act, to improve:
• Federal program effectiveness and public accountability by
promoting a new focus on results, service quality, and customer
satisfaction
• Internal management of the federal government
GPRA requires that federal agencies' strategic plans include a description
of how goals and objectives are to be achieved, including a description of
the operational processes, information, and other resources required to
achieve those goals and objectives.
12-P-0724
5
-------
OMB guidance and memoranda outline approaches for meeting the intent
of GPRA. OMB Circular A-123, Management's Responsibility for
Internal Controls, issued December 2004, identifies management as being
responsible for establishing and maintaining internal controls to achieve
the objective of effective and efficient operations. OMB Memorandum
M-10-24, Performance Improvement Guidance: Management
Responsibilities and Government Performance and Results Act
Documents, issued June 25, 2010, explains that managers should analyze
performance and measure progress in achieving desired results, and
requires management to identify more effective and efficient approaches
when needed.
According to the Clean Air Act Amendments of 1990, one of CSB's roles
is to recommend measures to reduce the likelihood or the consequences of
accidental chemical releases and propose corrective steps to make
chemical production, processing, handling and storage as safe and free
from risk of injury as is possible. CSB established policy in the form of a
board order to achieve this purpose. CSB developed Board Order 022,
CSB Recommendation Program, in December 2001, to establish
procedures for the development, issuance, follow-up, and closing of
CSB's recommendations. CSB's 2007-2012 strategic plan explains that
safety recommendations are designed to prevent future accidents and to
accomplish its mission. In the strategic plan, CSB's goal #3 is to
"Reduce the likelihood of similar accidents in the future by securing
implementation of CSB safety recommendations." The strategic plan lists
a number of key strategies to accomplish this goal:
(1) Maintain an efficient system for rapidly communicating
with recommendations recipients, tracking the status of all
open safety recommendations, and evaluating the adequacy
of recipient actions; (2) Publicize up-to-date status
information on all safety recommendations through the
CSB website and other public communication channels,
with an emphasis on most-wanted safety; (3) Conduct
focused advocacy programs for significant, challenging
safety recommendations through ongoing dialogue with
relevant government and other stakeholders, testimony, and
other public communications; and (4) Publicly recognize
recommendations recipients that implement significant
safety actions based on CSB safety recommendations.
12-P-0724
6
-------
CSB Recommendations Not Always Closed Within Prescribed Time
Not all safety recommendations are implemented timely. As of
December 31, 2010, CSB had issued 588 recommendations, of which 218
were still open (table 1). The likelihood that a similar accident may occur
in the future increases if a recommendation remains open.
Table 1: CSB recommendations by category as of December 31, 2010
Recommendation
categories
Number of
recommendations
Universe
percentage
Category explanation
Ope n—Acce pta b le
Response or Alternate
Response
62
10.5%
Response from recipient indicates a
planned action that would satisfy the
objective of the recommendation when
implemented.
Open—Awaiting
Response or
Evaluation/Approval of
Response
152
25.9%
Recipient has not submitted a substantive
response, evaluation of response by CSB
staff is pending, or the board has not yet
acted on staff recommendation.
Open—Unacceptable
Response
4
0.7%
Recipient response disagrees with the need
outlined in the recommendation; the board
believes there is enough supporting
evidence to ask the recipient to reconsider.
Subtotal open
recommendations
218
37.1%
C losed—Acce pta bl e
Action
320
54.4%
Recipient's completed action on the
recommendation meets the board's
objectives.
C losed—Ex ceed s
Recommended Action
8
1.4%
Action on the recommendation meets and
surpasses the board's objectives.
C losed—U n acce pta b le
Action/No Response
Received
6
1.0%
Recipient response disagrees with the need
outlined in the recommendation; the board
concludes that further effort would not
change the recipient's position.
Closed—No Longer
Applicable
30
5.1%
Due to subsequent events, the
recommendation action no longer applies.
Closed—
Reconsidered/
Superseded
6
1.0%
Recipient rejects the recommendation and
supports the rejection with a rationale with
which the board concurs (e.g., the concerns
expressed in the recommendation were
actually addressed prior to the incident or a
recommendation is superseded by a new,
more appropriate recommendation).
Subtotal closed
recommendations
370
62.9%
Total
recommendations
588
100%
Source: Data provided by CSB as of December 31, 2010. Categories defined by CSB in Board Order 022.
Board Order 022 requires that final action on recommendations, other than
those that do not require urgent attention, should generally be completed
as soon as possible, but not later than 3 to 5 years after issuance of the
recommendation. According to the board order, only in rare cases will
recommendations be carried past the 5-year period. However, of the 218
12-P-0724
7
-------
open recommendations, 54, or nearly 25 percent, were more than 5 years
old (table 2).
Table 2: Status of CSB open safety recommendations more than 5 years old
Recommendation category status
Number of
recommendations
Open—Awaiting Response or Evaluation/Approval of
Response
35
Open—Acceptable Response or Alternate Response
17
Open—Unacceptable Response
2
Total
54
Source: CSB TRIM system as of December 31, 2010.
CSB Needs Effective and Efficient Internal Controls
CSB's internal controls related to safety recommendations do not promote
effective and efficient operations. Weaknesses in control activities—such
as those that govern updating and consistently implementing policies and
procedures, promote adherence to board orders, and ensure data quality—
have negatively affected CSB's recommendations program. Further, CSB
could be more effective if it took advantage of its information technology
tools and created a formal advocacy program. Internal controls are a major
part of managing an organization and are necessary for CSB to achieve its
mission and program results. According to OMB Memorandum M-l 1-17,
Delivering on the Accountable Government Initiative and Implementing
the GPRA Modernization Act of 2010, "[a]gencies measure, analyze, and
communicate performance information to identify successful practices to
spread and problematic practices to prevent or correct." CSB's weak
internal control over the recommendations program activities impedes its
effectiveness in securing recommendation closure.
CSB Should Update Board Orders
CSB has not updated or assessed policies and procedures established in its
board orders that govern or affect its recommendations program. In OIG
Report No. 1 l-P-0115, Chemical Safety and Hazard Investigation Board
Did Not Take Effective Corrective Actions on Prior Audit
Recommendations, February 15, 2011, we identified outdated board orders
and recommended that CSB strengthen its control activities.
In this current audit, we identified three board orders in need of updates for
which two are a repeat from the previous audit mentioned above. The updates
will assist CSB's efforts to secure safety recommendation implementation:
• Board Order 022, CSB Recommendation Program
• Board Order 001, Board Quorum and Voting
• Board Order 040, Investigation Protocol
12-P-0724
8
-------
Board Order 022, CSB Recommendation Program
Board Order 022 does not require data quality reviews for the
recommendations data entered in TRIM or analysis of key milestones
concerning the recommendations processes. We recommended in our
prior audit report that CSB update Board Order 022 to include new
practices for following up on safety recommendations. The
recommendation called for a quality review program to ensure timely
follow-up on safety recommendations. CSB concurred with the
recommendation and responded that Board Order 022 was currently
under review and that CSB would consider including a quality review
program to ensure timely follow-up on safety recommendations. CSB
noted in its response to our draft report that Board Order 22 has been
redrafted and is expected to be voted on by the board by the end of
June 2012.
At the start of our review, we found that CSB did not have updated
e-mail or telephone contact information for 11 of the 92 recipients
(nearly 12 percent) of the 218 open recommendations. However, by
June 20, 2011, CSB provided us with contact information for all
92 recipients. CSB indicated that its staff search for the last
correspondence documented in TRIM when they need to send
follow-up correspondence. According to a recommendations specialist,
CSB does perform a data quality review of TRIM folders for
completion and accuracy as they conduct follow-up activities.
However, there is no formal process in place.
We could not determine the average time it takes CSB to respond to
recommendation recipients regarding their proposed corrective action
plans due to CSB's inconsistent data entry in TRIM. However,
recipients' responses to our survey identified concern with the
timeliness of CSB's reply to its proposed corrective action plans.
Seven recommendation recipients that responded to our survey noted
that CSB did not provide timely feedback on proposed corrective
actions. In addition, one respondent noted that CSB does not
communicate whether planned actions meet their expectations.
In response to the 13 questions we sent to 79 recipients of
recommendations that were open as of December 2010, we received
responses from 34 recipients. Recipients provided some favorable
responses. For example, 75 percent stated they believe CSB's
recommendations will improve the safety of similar chemical
facilities. Also, a few responses indicated areas where CSB could
improve.
12-P-0724
9
-------
Board Order 001, Board Quorum and Voting
Office of Recommendations' efforts to obtain timely implementation
of safety recommendations could be affected by the practice of
"calendaring" notation items. A notation item consists of a transmittal
memorandum, the draft document proposed for adoption, and pertinent
attachments necessary for a full understanding of the document. Board
members are empowered to calendar votes on recommendations,
which means they can postpone decision making until they receive
more information. Neither Board Order 022 nor Board Order 001
contains specific guidelines regarding the length of time that a
recommendation may remain suspended before a vote must be taken.
Indefinite calendaring can affect the adoption of investigation reports
and their recommendations as well as prevent the occurrence of
chemical accidents to the greatest extent possible. An analysis that we
completed in July 2011 identified 11 investigation reports with
37 recommendations that had been calendared.
Other notation items, such as the adoption of board orders, can also be
calendared. CSB developed Board Order 04 6, Board Members Roles
and Responsibilities, to strengthen advocacy efforts and define roles and
responsibilities. However, the vote to approve and implement this board
order was calendared in November 2011; as of May 2012, this notation
item remained unresolved. CSB noted in its response to our draft report
that it approved Board Order 046 as the Most Wanted Chemical Safety
Improvements Program. CSB noted during our exit conference that the
Board's roles and responsibilities are an administrative issue and some
responsibilities are in other board order documents.
Board Order 040, Investigation Protocol
Parameters regarding collaboration between CSB's Office of
Investigations and the Office of Recommendations are unclear. Board
Order 040 does not outline procedures concerning the involvement of
the Office of Recommendations or recommendation recipients during
the investigation and the recommendations-development process.
According to the U.S. Government Accountability Office Standards
for Internal Control in the Federal Government, "[f]or an entity to run
and control its operations, it must have relevant, reliable, and timely
communications relating to internal as well as external events."
Board Order 040 does mention that the investigation manager is
responsible for "[coordinating discussions with CSB [Office of]
Recommendations [;]" however, the order does not describe the details
and timing of such discussions. Further, Board Order 040 does not
12-P-0724
10
-------
specify any recipient involvement during the investigation or the
development of safety recommendations.
Six of the seven CSB staff members we interviewed during the audit
stated that it is not clear whether the Office of Investigations or the
Office of Recommendations is responsible for identifying the
recommendation recipient within an organization. As a result,
sometimes recipients could be identified too late, causing issues with the
way recommendations are written and to whom
they are directed. Nearly 60 percent of the
recommendations recipients who responded to
our 13 questions claimed they had no input or
involvement in the development of
recommendations affecting them.
The director of the Western Regional Office of
Investigations stated that he fully supports
integration of the recommendations and
investigations teams. He believes that it is
valuable to have a team member from the Office
of Recommendations involved in the
investigation process early. He stated that Office
of Recommendations staff could more effectively
advocate or track recommendations when they
have had prior involvement in the case.
CSB Should Use Information
Technology to Track Key Milestones
Although CSB has developed and
implemented TRIM for tracking safety
recommendations and related activities, the
Office of Recommendations is not optimally
using information technology to improve
operations. CSB's key strategies state they will
"[m]aintain an efficient system for rapidly
communicating with recommendations
recipients, track the status of all open safety
recommendations, and evaluate the adequacy
of recipient actions." Maximizing the use of
TRIM and ensuring that the data therein are
complete and accurate will help CSB
communicate with recommendation recipients,
track the status of all open safety
recommendations, and evaluate the adequacy
of recipients' actions.
When follow-up on unimplemented
recommendations is inconsistent, CSB is
not doing all it can to prevent accidents
with the same root causes from
occurring. Examples of investigations
with recommendations that identified
root causes of serious, and even fatal,
accidents that lacked consistent follow-
up and were unimplemented for 5 or
more years include the following:
Uncontrolled Chemical Reactions:
In 2002, CSB issued a safety study that
identified 167 serious accidents from
January 1980 to June 2001 that were
caused from poorly-understood and
uncontrolled chemical reactions. Forty-
eight of these incidents resulted in 108
fatalities. CSB found that, during the
period under review, an average of six
injuries and five fatalities occurred
annually as a result of uncontrolled
chemical reactions.
Inadequate safety inspections:
CSB identified a refinery that failed to
perform safety inspections. A corroded
sulfuric acid tank at the plant exploded
when repair work was performed over
the tank. The accident resulted in one
fatality and eight injuries. A significant
volume of sulfuric acid was released into
the environment.
Deficient management protocols:
CSB identified a chemical plant that
experienced three accidents within a
1-month period due to deficient
management protocols. The accidents
injured workers and triggered a shelter-
in-place advisory to community
residents.
12-P-0724
11
-------
As of December 2010, the four staff members of the Office of
Recommendations had 218 open recommendations to track and follow up
on without the benefit of data that have undergone data quality review and
automated key milestones reminders. A federal government entity
responding to our 13 questions commented that CSB's follow-up on
recommendations has often been slow. Although CSB's Chief Information
Officer stated that TRIM could be programmed to notify recommendation
specialists that follow-up dates are approaching, this TRIM function was
not programmed and utilized during our audit.
We found that 31 of the 41 (over 75 percent) open recommendations in
our sample1 did not show follow-up activity at least every 6 months. Of
those 31, 13 (42 percent) were open for more than 5 years. The Office of
Recommendations stated that higher priorities and staff changes kept it
from following up on some of the older unimplemented recommendations.
Untimely follow-up can lead to untimely implementation, which could
prevent the occurrence of similar chemical accidents causing injuries and
fatalities.
Further, in reviewing three case files for reports with recommendations in
a "Closed—No Longer Applicable" status, we found that all were missing
some form of documentation. For example, one of the files documented in
TRIM contained no information for the recipient's initial response to the
recommendations, no recommendation response evaluation form, no
documentation to indicate that CSB followed up with the recipient, and no
notations as to whether the recipient's responses were ever received or
follow-up was necessary.
CSB Has Not Implemented a Formal Advocacy Program for
Safety Recommendation Implementation
Although CSB recognizes that advocacy techniques, when employed, have
successfully contributed to its ability to secure implementation of its safety
recommendations, it does not have a formal advocacy program. However,
CSB's strategic plan for 2007-2012 calls for CSB to "Publicize up-to-date
status information on all safety recommendations through the CSB
website and other public communication channels, with an emphasis on
most-wanted safety actions," and state its intention to "Conduct focused
advocacy programs for significant, challenging safety recommendations."
The Office of Recommendations stated that it would need additional staff
to handle advocacy responsibilities. Meeting its strategic plan strategies
related to advocacy would increase the likelihood that CSB safety
recommendations will be implemented.
1 As explained in appendix A, we sampled 44 of 218 open recommendations from the data provided by
CSB as of December 31, 2010, but 3 of the 44 were less than 6 months old.
12-P-0724
12
-------
Although it does not have a formal program, CSB has used advocacy
techniques in certain instances. CSB explained that success in gaining
implementation could be attributed to advocacy techniques, including:
• Strong media coverage, including public meetings on issues.
• Direct advocacy by board members.
• Keeping issues before the community and, in some cases,
Congress.
• Building relationships with influential people in recipients' or
other pertinent organizations.
The goal of NTSB's advocacy program is to implement safety
recommendations. According to testimony given by NTSB's former acting
chairperson in 2005, prior to having an advocacy program in place, NTSB
lost significant time engaging only through the written process with
recipients to arrive at a mutual understanding and agreement on
recommendations. Among a number of advocacy techniques, NTSB
established a most-wanted list in 1990 to increase public awareness of, and
support for, recommendations with the greatest potential to prevent
accidents and save lives.
A formal advocacy program with required efforts established through
board order or other policy framework could mitigate external forces such
as CSB's lack of enforcement authority and the lengthy regulatory
process. CSB's strategic plan acknowledges that time-consuming
advocacy efforts are required to secure implementation of significant
recommendations.
Other Impediments to Timely Implementation
CSB has external impediments in addition to its internal control and
procedural deficiencies that delay implementation of its safety
recommendations. CSB has no legal authority to force federal agencies,
states, or industries to implement its recommendations. The long process
that regulatory agencies must go through to implement safety
recommendations is another external impediment over which CSB has no
control. For example, an OSHA representative explained that the OSHA
rulemaking process could take 10 years for adequate research, including
risk assessments, and another 2.5 years to go from a proposal to
publication of a final rule.
Conclusion
Without effective and efficient internal controls and processes, CSB's
Office of Recommendations is not timely in carrying out its functions to
encourage the timely implementation of its recommendations and achieve
12-P-0724
13
-------
its goal of chemical accident prevention. Although we cannot say with
certainty that untimely implementation has caused similar accidents, we
believe that untimely implementation increases the risk of accidents that
harm the environment, workers, businesses, and communities.
Recommendations
We recommend that the Chairperson, U.S. Chemical and Hazard Safety
Investigation Board:
1. Update board orders to ensure that CSB achieves its mission of
chemical accident prevention through improved recommendations
processes, to include:
a. Board Order 022, CSB Recommendation Program,
i. To establish and implement data quality reviews to
verify the accuracy, completeness, and reliability of
recommendations data entered in TRIM, such as error
checks and inclusion of required supporting
documentation.
ii. To require that the Office of Recommendations director
periodically analyze and assess the recommendations
process to identify potential process improvements.
b. Board Order 001, Board Quorum and Voting, to establish and
implement guidelines that define the length of time notation
items can be calendared before a vote must be taken.
c. Board Order 040, Investigation Protocol, to clearly outline
roles and responsibilities of the Office of Recommendations
and Office of Investigations with respect to the
recommendations process, including a requirement that Office
of Recommendations staff participate in accident
investigations, and identification of the office responsible for
identifying potential recommendation recipients.
2. Make full use of TRIM'S capabilities, to include:
a. Incorporating formal scheduling components in TRIM to track
the recommendations process and alert staff to impending
milestones to ensure timely follow-up.
b. Highlighting the absence of required supporting
documentation.
12-P-0724
14
-------
3. Implement a formal advocacy program to advocate for safety
recommendation implementation, to include adoption of a most-
wanted list of safety actions.
CSB Comments and OIG Evaluation
CSB agreed with recommendations la, lc, 2, and 3. CSB noted the
following corrective actions in its response to our draft report:
• Board Order 022 has been redrafted and is being reviewed by the
Board. We request CSB confirm the date the Board approved
revisions to Board Order 022 and provide a copy of the completed
corrective action with its 90 day response. (Recommendation la)
• Updates to Board Order 040 by the end of 2012.
(Recommendation lc)
• The Board approved and issued Board Order 046, Most Wanted
Chemical Safety Improvements Program on June 12, 2012. The
selection of the first two most wanted issues is expected by the end
of July 2012. (Recommendation 3)
In response to recommendation 2, CSB implemented a tagging system for
identifying missing key documents in TRIM. CSB had a contractor
program TRIM to send automatic reminders to staff to help ensure timely
follow-up.
CSB disagreed with recommendation lb. CSB noted the following in its
response.
The OIG findings underlying this recommendation are
erroneous. Moreover this recommendation contains a
flawed conclusion with which we strongly disagree - that
calendaring of votes "... can contribute to the occurrence of
similar chemical accidents." [Draft report at p. 9] Finally,
this recommendation simply misses the purpose for
establishment of a collegial voting body like the Board.
Forcing votes when there is no consensus is a poor practice.
The draft report fails to recognize that this is not an
operational issue. Rather, the voting process, whether it
applies to CSB reports or board orders, is an iterative
process that relies on consensus building and ultimately on
policy judgments. Forcing votes to occur in the manner
suggested and imposing "deadlines" for policy decisions is
12-P-0724
15
-------
neither productive nor consensus-oriented, which are
necessary for the effective operation of a collegial body.
The OIG's approach would apply a simplistic deadline to
difficult problems and attempt to force votes upon the
Board, when the Board has not been able to reach a
consensus. While consensus cannot always be achieved,
this is our goal, and we believe the OIG's recommended
approach would not serve this goal. Ultimately, consensus
is best achieved when board members have the inclination
and desire to resolve issues, rather than when an issue is
forced upon them because of the passage of an arbitrary
period of time.
The OIG agrees that CSB should not force votes. However, we reaffirm
that CSB should develop and implement internal controls to ensure that
the calendaring process does not impede a resolution in the interest of
public health and safety. We suggest that CSB identify internal controls to
timely move items from the calendared status. Further, CSB's Board
Order 001 refers frequently and consistently to the adoption or disapproval
of an item by a "majority" vote. There is no mention of a requirement for a
"consensus."
We made wording changes in response to CSB's concerns with our draft
report regarding tone and accuracy. We acknowledge that CSB created a
baseline for future data quality reviews and believe that documenting
signatures, finding missing records, and updating the electronic system are
all important controls to data quality. We support CSB's commitment to
maintaining reliable data as an internal control that promotes accurate
reporting, timely follow-up, and informed decision making.
CSB's complete response to our draft report is in appendix B.
12-P-0724
16
-------
Status of Recommendations and
Potential Monetary Benefits
POTENTIAL MONETARY
RECOMMENDATIONS BENEFITS (in $000s)
Rec.
No.
Page
No.
Subject
Status1 Action Official
Planned
Completion
Date
Claimed
Amount
Agreed-To
Amount
1
14
Update board orders to ensure that CSB
achieves its mission of chemical accident
prevention through improved recommendations
processes, to include:
a. Board Order 022, CSB Recommendation
Program,
Chairperson, U.S.
Chemical Safety and
Hazard Investigation Board
U
i. To establish and implement data
quality reviews to verify the accuracy,
completeness, and reliability of
recommendations data entered in
TRIM, such as error checks and
inclusion of required supporting
documentation.
ii. To require that the Office of
Recommendations director
periodically analyze and assess the
recommendations process to identify
potential process improvements.
b. Board Order 001, Board Quorum and U
Voting, to establish and implement
guidelines that define the length of time
notation items can be calendared before a
vote must be taken.
c. Board Order 040, Investigation Protocol, to
clearly cutline roles and responsibilities of
the Office of Recommendations and Office
of Investigations with respect to the
recommendations process, including a
requirement that Office of
Recommendations staff participate in
accident investigations, and identification
of the office responsible for identifying
potential recommendation recipients.
2 14 Make full use ofTRIM's capabilities, to include: Chairperson, U.S. March 2013
a. Incorporating formal scheduling O Chemical Safety and
components in TRIM to track the Hazard Investigation Board
recommendations process and alert staff to
impending milestones to ensure timely
follow-up.
b. Highlighting the absence of required O March 2013
supporting documentation.
3 15 Implement a formal advocacy program to O Chairperson, U.S. March 2013
advocate for safety recommendation Chemical Safety and
implementation, to include adoption of a most- Hazard Investigation Board
wanted list of safety actions.
December
2012
1 O = recommendation is open with agreed-to corrective actions pending
C = recommendation is closed with all agreed-to actions completed
U = recommendation is unresolved with resolution efforts in progress
12-P-0724
17
-------
Appendix A
Details on Scope and Methodology
We performed our audit work from January 2011 through May 2012. Our initial objective
was to determine what factors impede implementation of CSB's open safety
recommendations. Following preliminary research, we broadened the scope of the review
to include a review of all CSB safety recommendations. We reviewed recommendations
from September 1998 to December 2010. We analyzed CSB's recommendations data to
determine whether data quality and other internal controls may have affected the ability
of CSB to get its recommendations implemented.
Preliminary Research
During our preliminary research phase, CSB provided us data on the universe of all
safety recommendations, which was retrieved from TRIM on December 17, 2010. On
January 7, 2011, CSB provided updated data on the status of open recommendations as of
December 31, 2010. We reconciled the updated data to the universe received from TRIM.
The universe was atotal of 588 recommendations as of December 31, 2010, which
included 218 open and 370 closed recommendations.
Using IDEA®-Data Analysis Software, we randomly selected 7 (3 percent) of the 218
open recommendations. We used those seven to review the accuracy and reliability of
data maintained in TRIM.
In a separate exercise, we judgmentally selected 11 (5 percent) of the 218 open
recommendations. For those 11, we reviewed whether follow-up occurred in accordance
with Board Order 022, and whether correspondence, communication, and other relevant
information pertaining to recommendations and responses were documented in TRIM.
We summarized and documented our observations and compiled interview questions for
CSB staff and supervisors relative to our findings and for validation purposes.
Field Work
During field work, we used IDEA® to select a random sample that incorporated a
population of all recipient types (government-federal, industry, government-state,
standards development organization, industry corporate, trade association,
academia/training institution, and professional organization). The sample plan included
44 (20 percent) of the 218 open recommendations. We tested CSB's internal controls and
the quality of recommendations data maintained in TRIM and identified potential
opportunities for improvement in the recommendations processes.
12-P-0724
18
-------
We employed various methodologies to select our samples from the universe of 370
closed recommendations as of December 31, 2010, to assess the data quality and internal
controls over CSB's closed recommendations:
• For Closed—No Longer Applicable recommendations, we selected a non-
statistical sample to include three investigation cases—Sierra (1998),
Hayes-Lemmerz (2004), and ASCO (2006)—and their 20 corresponding
recommendations.
• For Closed—Acceptable Action and Closed—Exceeds Recommended Action
recommendations, we first eliminated all closed recommendations older than
3 years and arrived at a population of 108 recommendations closed in the last
3 years. From the 108, we extracted a random sample of 22 (20 percent) using
IDEA®.
• For Closed—Unacceptable Action/No Response Received and Closed—
Reconsidered/Superseded recommendations, we reviewed nearly 92 percent
(11 of the 12). This group included 12 recommendations, 2 percent of the entire
safety recommendations universe of 588.
Thirteen Questions Provided to Certain Recipients of CSB
Recommendations
We developed 13 questions that we provided for response to CSB recommendation
recipients with open recommendations to determine the factors that hindered recipients
of CSB investigative reports from implementing the recommended safety
recommendations. Although we planned to provide these questions to all 92 recipients of
the 218 open recommendations, some of the e-mail contact information for the
92 recipients was duplicative. We were able to discern 79 unique e-mail addresses, and
we provided the questions to those 79 CSB safety recommendation recipients on June 23,
2011. A total of 34 (43 percent) of the 79 recipients responded to the questions.
2 Zoomerang survey software was used to provide the questions and receive responses.
12-P-0724
19
-------
Appendix B
CSB Response to Draft Report
U.S. Chemical Safety and
Hazard Investigation Board
21 75 K Street, NW • Suite 650 •
Phone: (202) 261-7600 • Fax: (202)
, DC 20037-
261-7650
Office of the Chair
June 13,2012
Ms. Melissa Heist
Assistant Inspector General for Audit
Office of the Inspector General
U.S. Environmental Protection Agency
Washington, DC 20460
Dear Ms. Heist:
Thank you for the opportunity to comment on the draft report entitled, "CSB Should
Improve Recommendations Process to Further its Goal of Accident Prevention." We
continue to appreciate the professionalism and courtesy of the Office of Inspector
General (OIG) staff who were involved in the evaluation.
As noted below, the CSB has serious concerns about the accuracy and the tone of the
draft report. However, the CSB agrees with all but one of the proposed
recommendations, and has already taken a variety of measures to implement them.
These comments are in addition to the concerns raised in my letter to Inspector General
Elkins dated May 21, 2012. In particular, the CSB believes that the audit report should
make clear that the most important obstacle to the implementation of CSB
recommendations is the reluctance of recipients to implement them in certain cases -
especially because of the lack of any enforcement power on the part of the CSB. We do
not believe the OIG demonstrated that any weaknesses within the CSB's four-person
Recommendations Office contributed significantly to the failure of any recipient to
implement CSB recommendations.
12-P-0724
20
-------
SUMMARY OF RESPONSE:
The CSB agrees with all but one of the recommendations in the draft audit report, and the
agency has already, or will soon complete implementation of these recommendations. A
revised Board Order 22 (OIG Recommendation la) is expected to be considered by the
Board before the end of June. The CSB disagrees with OIG Recommendation lb,
regarding calendaring of notation votes, as explained more fully below. Updates to
Board Order 40, Investigation Protocol, are expected to be completed and approved by
the Board by the end of 2012, including those issues addressed in OIG Recommendation
lc. Regarding the fuller use of our current electronic records database (TRIM), a tagging
system for identifying missing key documents is already in operation and a contractor is
programming the system to send automatic reminders to staff to ensure follow-up every
six months. The latter function is expected to be fully tested and operational by the end of
2012. The Board just approved Board Order 46, Most Wanted Chemical Safety
Improvements Program, and will be selecting the first two most wanted issues by July
2012, consistent with OIG Recommendation 3.
The draft report gives an overall impression that there are widespread data quality
problems with the recommendations data. Therefore, CSB recommendations staff
conducted a baseline data quality review to better quantify and understand the concerns
of the OIG. Our findings indicate that the data identified as "missing" by the OIG do not
materially affect the agency's efficiency or effectiveness in its pursuit of
recommendations implementation. In summary, our review found that only about 6% of
a total of more than 600 recommendations were missing one of three key documents.
More importantly, more than 60% of these "missing" documents were not missing at all -
rather they were digital copies of the letters in the database that were only missing
physical signatures. The majority involved recommendations that are already closed, and
most also date back to the period before 2004-05, when the present Recommendations
Office was established. These missing data do not materially impede our growing efforts
to follow-up and advocate for the implementation of recommendations. Our internal
review also located about one fifth of the "missing" data, as well as numerous other
follow-up related documents for many investigations, especially older ones. Lastly, this
internal review found that for all investigations completed since 2006, our electronic
database is practically 100% complete, which indicates that the quality of
recommendations-related records has substantially improved.
BACKGROUND:
While the draft report notes that the CSB established a dedicated Recommendations
Office in 2004, an expanded history and evolution of the Recommendations Office, as
well as its primary responsibilities, would be helpful to put our comments in their proper
context. We suggest that a similar summary be a part of the audit report itself. The
agency was first funded in 1998, and Recommendations Board Order 22 was adopted in
December 2001. Recommendations activities were secondary to investigative work for
the first several years, with very limited resources devoted to recommendation tracking,
advocacy, and closure. A Recommendations Office was not permanently created until
12-P-0724
21
-------
2004 and not staffed until mid-2004 to 2005. The overwhelming priorities during the
first years of the new office were to follow-up and close a growing backlog of open
recommendations. CSB began using an electronic database for investigation records in
2005, and then existing and somewhat disorganized recommendations records were
scanned into the database beginning in 2006.
The combination of establishing a Recommendations Office and using an electronic
database for recommendations records has been very successful. As illustrated in the
graph below, the percentage of CSB recommendations that were closed in 2004, when
the office was created, stood at 24%. That figure now stands at 71% (mid-2012), and we
are systematically pursuing the remaining open recommendations, as well as the new
ones that are issued Our electronic database is well in place and our essential records are
nearly 100% complete for investigations since 2006. The office is every day more
effectively contributing to the development of recommendations, a long-sought CSB
goal, and the evaluation of responses has consistently become more thorough and
comprehensive. As the remainder of this response indicates, we anticipate operating the
office under a revised Board Order 22, and we expect to have a functioning Most Wanted
Chemical Safety Improvements Program by the end of 2012.
1998 1999 2000 2001 2002 2003 2004 200* 200C 2007 2008 2009 2010 20 U
RESPONSES TO OIG RECOMMENDATIONS
OIG Recommendation 1(a): Update board orders to ensure that CSB achieves its
mission of chemical accident prevention through improved recommendations processes,
to include:
Percent of CSB Recommendations Closed by FY
TRIM
implemented
Older 22iwued
Office OiKXtor Hired
12-P-0724
22
-------
• Board Order 022, CSB Recommendation Program
(i) To establish and implement data quality reviews to verify the accuracy,
completeness, and reliability of recommendations data entered in TRIM, such as
error checks and inclusion of required supporting documentation.
(ii) To require that the Office of Recommendations director periodically analyze
and assess the recommendations process to identify potential process
improvements.
CSB Response:
The CSB agrees with this recommendation. Board Order 22 has been redrafted and is
expected to be voted on by the Board by the end of June 2012. Among many other
changes and improvements, the revised draft includes two new sections that specifically
address the two items in the above-referenced OIG recommendation.
The revisions also address a broad range of other issues that will specifically improve the
development, follow-up and closure of recommendations, including improved
collaboration between investigative and recommendations staff.
With regard to data quality, the Recommendations Office recently carried out a
comprehensive baseline data quality review, the results of which are described in
Attachment A. In summary, the review established that any recordkeeping quality issues
were relatively minor and mostly dated from the period prior to the use of an electronic
database. This finding is in sharp contrast with the impression of widespread data quality
problems conveyed by the OIG audit report.
OIG Recommendation 1(b): Update board orders to ensure that CSB achieves its
mission of chemical accident prevention through improved recommendations processes,
to include:
Board Order 01, Board Quorum and Voting, to establish and implement
guidelines that define the length of time notation items can be calendared before a
vote must be taken.
CSB Response:
The CSB disagrees with this recommendation. The OIG findings underlying this
recommendation are erroneous. Moreover this recommendation contains a flawed
conclusion with which we strongly disagree - that calendaring of votes "... can
contribute to the occurrence of similar chemical accidents." [Draft report at p. 9] Finally,
this recommendation simply misses the purpose for establishment of a collegial voting
body like the Board. Forcing votes when there is no consensus is a poor practice. The
draft report fails to recognize that this is not an operational issue. Rather, the voting
process, whether it is applies to CSB reports or board orders, is an iterative process that
12-P-0724
23
-------
relies on consensus building and ultimately on policy judgments. Forcing votes to occur
in the manner suggested and imposing "deadlines" for policy decisions is neither
productive nor consensus-oriented, which are necessary for the effective operation of a
collegial body.
The specific cited example of a "calendared" investigation reported in the draft OIG
report, which allegedly led to a nearly five-month delay in issuance, is in serious error.
The facts are that this notation item (NI 846) - Adoption of an Investigation Report and
Recommendations for an Incident Occurring at the DuPont Plant in Belle, WV. - was
circulated on April 27, 2011. It was calendared by two board members on April 28 and
May 9, respectively. Subsequent to this calendaring, notation item 852 was circulated on
May 20, 2011, to address some ofthe issues underlying the calendaring ofNI 846. This
NI was disapproved by the Board on June 1, 2011. It was not calendared as stated in the
draft OIG report. Subsequent to this disapproval, NI 862 was circulated on June 8, 2011,
and approved by the full Board on June 22, 2011 - less than two months after the very
first vote on the issue (and arguably just as quickly as if a public meeting had been called
to consider the report). This NI approved public issuance of the draft report and
recommendations and provided a 45-day comment period for receipt of public input. The
comment period ran from July 7 - August 22, 2011, and resulted in over 13 separate
public comments received totaling 43 pages, many of which were very useful to the
Board. Following completion of the comment period and a staff analysis of the
comments received, NI 880 was circulated on September 7, 2011, and approved on
September 20, 2011, leading to the report's issuance. Most of these key facts are omitted
from the draft report.
Contrary to the draft OIG report's characterization, the DuPont Belle Investigation was
not subject to a long calendaring process. Rather, several different votes were taken
(only one of which resulted in a short period of calendaring), and in the end, both public
input and a superior report resulted from the Board's actions.
The other example cited in the draft OIG report relates to the calendaring of proposed
Board Order 46 entitled "Board Members Roles and Responsibilities," which is outside
the scope of an evaluation of CSB recommendations activities. While it is true, as
reported by the OIG, that this item has remained calendared since November 2011, the
CSB disagrees that a document involving changes in board member roles should be
subject to an arbitrary deadline for issuance. Obviously, some board orders and notation
items involve sensitive topics requiring the development of consensus among members.
This is not always easily achieved as members represent different points of view and
bring different experiences to bear upon the issues faced by the Board.
The OIG's approach would apply a simplistic deadline to difficult problems and attempt
to force votes upon the Board, when the Board has not been able to reach a consensus.
While consensus cannot always be achieved, this is our goal, and we believe the OIG's
recommended approach would not serve this goal. Ultimately, consensus is best
achieved when board members have the inclination and desire to resolve issues, rather
12-P-0724
24
-------
than when an issue is forced upon them because of the passage of an arbitrary period of
time.
OIG Recommendation 1(c): Update board orders to ensure that CSB achieves its
mission of chemical accident prevention through improved recommendations processes,
to include:
• Board Order 040, Investigation Protocol, to clearly outline roles and
responsibilities of the Office of Recommendations and Office of Investigations
with respect to the recommendations process, including a requirement that Office
of Recommendations staffparticipate in accident investigations, and
identification of the office responsible for identifying potential recommendation
recipients.
CSB Response:
The CSB agrees with this recommendation. The CSB is revising Board Order 40,
addressing a broad range of enhancements for collaboration among the Investigative and
Recommendations Offices, including the two specific areas mentioned in the
recommendation. The revision includes a provision that the Office of Recommendations
staff participate in accident investigations and a clear identification of the office
responsible for identifying recommendation recipients. The CSB is continually revising
Board Order 40, a complex order that encompasses the Agency's entire investigative
protocol. However, the CSB will focus on completing sections related to the OIG's
recommendations and expects they will be ready for a Board vote by the end of 2012.
Of particular relevance to this recommendation is the ongoing work of a staff task force
from the Offices of Investigations, Recommendations, and Administration to develop
three new or revised chapters of the protocols entitled "Causal Analysis," "Product
Development and Review," and "Recommendations." The section on Causal Analysis
has already been approved by the Board.
OIG Recommendation 2a: Make full use of TRIM'S capabilities, to include:
• Incorporating formal scheduling components in TRIM to track the
recommendations process and alert staff to impending milestones to ensure timely
follow-up.
OIG Recommendation 2b: Make full use of TRIM'S capabilities, to include:
• Highlighting the absence of required supporting documentation.
12-P-0724
25
-------
CSB Response:
The CSB agrees with these recommendations. The CSB's chief information officer
has contracted with an expert to program TRIM to:
• Automatically prompt assigned recommendations specialists to review each open
recommendation file at least once every six months. We expect this system of
reminders to be available for testing by the end of FY 2012, and fully operational
in the first quarter of FY13.
In addition, TRIM has been modified to include a tagging system for the three key
document types that must be associated with recommendations: notification letters,
board action reports/recommendation response evaluations, and status change letters. All
existing documentation and all incoming documentation will be marked with these tags
as appropriate, to ensure that all documents are present and to alert staff if they are
absent. CSB recommendations staff also have logged all Board actions relevant to a
recommendation on the "Action Summary" tab associated with each recommendation.
This now permits staff to use the "Action Summary" page to identify the documentation
that should be in any investigation folder and fill gaps as appropriate.
Some additional comments are in order regarding TRIM:
1. TRIM is a commercially available electronic data storage and management
system, which is used by the entire agency, not only the Recommendations
Office. It may be replaced by another software system with similar or improved
functions in the future. We suggest this be made clear in the final OIG report.
2. A statement on page 8 of the draft report states incorrectly that CSB staff "did not
use the contact field in the TRIM database." We request that it be corrected. The
TRIM database did not have a field for a recipient point of contact. The CSB's
chief information officer created this field in the database when IG audit staff
requested contact information for all recommendation recipients in order to
conduct their planned survey. Recommendations staff then reviewed all open
recommendations and made sure that an updated point of contact was listed.
Contacts are also now updated routinely, as needed, whenever follow-up occurs
for any recommendation.
3. A statement on page 11 of the draft is incorrect and we request that it be
corrected. The statement asserts that "Although CSB's Chief Information Officer
stated that TRIM can be programmed to notify recommendations specialists that
follow-up dates are approaching, this TRIM function is not being utilized." It is
inaccurate to state that this functionality was "not being utilized," because it does
not yet exist in the software. The contractor is now programming it to
automatically send reminders to staff.
12-P-0724
26
-------
4. A second assertion on page 11 is also incorrect, and we request that it be
corrected. The second paragraph incorrectly states that 13 recommendations in
the OIG's sample of 44 "showed no follow-up activity for more than 5 years." A
review of these files by CSB staff found that eight of them (61%) had not gone
five years without follow-up. Indeed, many of them had repeated follow-up
activity, some in the recent past. Moreover, five of those eight (63%) were closed
between May 2011 and April 2012, and nearly all the others have now received
active follow-up.
OIG Recommendation 3: Implement a formal advocacy program to advocate for safety
recommendation implementation, to include adoption of most-wanted safety actions.
CSB Response:
The CSB agrees with this recommendation. The Board just approved Board Order 46,
Most Wanted Chemical Safety Improvements Program. A copy of the Board Order can
be found in Attachment B. In addition, the Board is now considering two initial "most
wanted" issues, and is expected to select them officially by the end of July 2012. The
new program is projected to be put in place during the last quarter of FY 2012 and to
become fully operational in the first quarter of FY 2013.
Thank you again for the opportunity to comment on the draft evaluation report. If you or
your staff have any questions regarding the CSB's comments and concerns, please
contact Daniel Horowitz, Managing Director, CSB, at 260-7613
(Daniel.Horowitz@csb.gov')
Sincerely,
Rafael Moure-Eraso, Ph.D., CIH
Chairperson
Attachments
12-P-0724
27
-------
ATTACHMENT A. Baseline Data Quality Review
The Office of Recommendations conducted a systematic, comprehensive review of its current
electronic database records. This review will serve as a baseline for future data quality reviews.
The review findings indicate that the number of missing data points is small, and will not
materially affect the CSB's closure of recommendations. In summary, most of the missing data
occurred in the early years of the agency's existence, before an electronic database management
tool was utilized. The majority of missing records are signed copies of letters (the files contain
copies of letters without signatures); most involve recommendations that are already closed, so
their absence is irrelevant. Lastly, the review was able to identify and locate approximately one-
fifth of the missing records the OIG highlighted, as well as many other paper records concerning
follow-up that were available in hard copy, but had not been scanned into the electronic database
at the time of the OIG's audit.
Staff reviewed the files for each of the agency's 641 recommendations to ensure the presence of
the following essential documents:
• An initial notification letter informing the recipient of the issuance of the CSB's
recommendation(s)
• Board action report(s) and associated recommendation response evaluation(s) for all
board votes regarding recommendation status designation(s).
• Status change letter(s) informing the recipient of board actions relevant to CSB's
recommendations (e.g., advancement, closure)
When these documents could not be located in the electronic database, staff searched its office
files as well as those of the Office of General Counsel. Numerous missing items were found and
filed; when a document could not be located, staff so noted it in the electronic database. In
addition to closing gaps related to these basic records, our review found numerous other records
related to follow-up that were scanned and entered into the electronic database. These additional
data largely reflected documents from before 2005-06, when CSB began using the electronic
database for recommendations records.
Our review revealed that: (1) the missing records constituted only a small proportion of the
overall records; (2) many of them actually existed but were missing only signatures; and (3)
most of the incomplete records date to before mid-2005, when the recommendations office was
created and no electronic record-keeping tool was available. The actual review results present a
more accurate picture of the missing data than is suggested in the OIG draft audit report. Of an
estimated total of 2,000 records (the three key records above multiplied by a total of 641
recommendations), approximately 114 records (6%) were found to be missing or partly missing.
Twenty-five (22%) of these were found in our review. Of the remaining 89 incomplete records,
approximately 63% were only missing a signature on a document but were otherwise accurate.
Moreover, approximately 66% of the records containing gaps involved investigations completed
prior to 2004, before the recommendations office formally existed, and before electronic records
management was implemented. Finally, our review indicates that nearly 100% of records since
2006 have the three basic documents listed above.
12-P-0724
28
-------
ATTACHMENT B.
BOARD ORDER 046
U Chaaileal Safety ail lazart
lavastlgatlaB Baari
l€jt
] SUBJECT: Most Wanted Chemical Safety Improvements Program
CONTENTS
1. Purpose 2
2. Effective Date 2
3. Background and Authority 2
4. Policy 2
5. Responsibilities 3
6. Advocacy 5
7. Review and Update 5
12-P-0724 29
-------
1. PURPOSE. This Order establishes a "Most Wanted Chemical Safety Improvements
Program" ("Most Wanted Program" or "Program") for the Chemical Safety and Hazard
Investigation Board (CSB) to:
• identify the most important chemical safety improvement goals of the CSB in the form of
a "Most Wanted List" of Chemical Safety Improvements, which will be based on
recommendations resulting from CSB investigations, studies, hearings, and similar fact-
finding activities;
• make efficient use of limited resources to pursue implementation of changes that are most
likely to achieve important national-level safety improvements;
• focus special advocacy efforts by Board Members and staff on the Most Wanted List;
• inform the CSB's deployment decisions and strategic allocation of staff resources; and
• issue periodic reports of the activities of the Program.
This Order provides policy guidance for the conduct of the Program, including the periodic
selection and advocacy of the Most Wanted List, issuance of periodic reports on the Program,
guidelines for advocacy by Board Members and staff, and a summary of the key roles of staff
offices in the Program. As described in this Order, the extent of advocacy activities
surrounding the Most Wanted List will depend on the resources available to the CSB.
2. EFFECTIVE DATE. This Order is effective upon passage by the Board.
3. BACKGROUND AND AUTHORITY. The CSB authorizing statute, 42 U.S.C. §
7412(r)(6)(C)(ii) assigns the CSB the responsibility to "issue periodic reports to the
Congress, Federal, State and local agencies ... concerned with the safety of chemical
production, processing, handling and storage, and other interested persons, recommending
measures to reduce the likelihood or the consequences of accidental releases and proposing
corrective steps to make chemical production, processing, handling and storage as safe and
free from risk of injury as is possible ...."
The CSB conducts investigations, studies, and research concerning chemical incidents and
hazards (or potential hazards), and the Board issues preventive or corrective
recommendations to various parties based upon the findings of those activities to prevent
similar incidents in the future. These recommendations serve as the basis for the Most
Wanted List.
4. POLICY. The Most Wanted List will be a group of critical safety improvements selected by
the Board for intensive follow-up and heightened awareness. The list will be based primarily
on CSB recommendations and their potential to enhance chemical safety at the national level
when implemented, but may also involve broader issues drawn from other fact-finding
activities of the CSB. The selection of the Most Wanted List will consider and assess risk,
through information such as:
12-P-0724
30
-------
a. the nature of the risk and estimated extent of exposure to workers and offsite populations;
b. previous loss and potential for future loss of life or property, illnesses, and environmental
damage;
c. very high risks disproportionately affecting discrete but highly identifiable groups of
individuals (e.g., a narrow sector of industry or certain specialized workers facing a very
high probability of risk);
d. strong concern of important sectors of a community, civic leaders or the like; and
e. The possibility that advocacy will help bring about change.
At least once per year, the Board shall vote to select or revise the Most Wanted List based on
factors such as a review of the advocacy activities of the previous year, staff advice, arising
opportunities for impact, and others.
The list shall be published on the CSB webpage and publicized through other means.
The Most Wanted List does not preclude advocacy for all other recommendations issued by
the CSB, and may be modified as the status of issues change during the year (e.g.,
recommended actions are implemented by recipients, new opportunities arise).
5. RESPONSIBILITIES.
a. Board Members.
(1) Periodically vote to select the "Most Wanted List" and monitor the operation of the
Program.
(2) Consider additions or other changes to the Most Wanted List during the year as
necessary.
(3) Select and individually take the lead in advocating specific "Most Wanted" issue
areas through speeches, editorials, scientific and lay articles, interviews, contacts with
potentially influential stakeholders, press conferences, videos, and similar activities.
b. Role of Staff Offices in the Program.
(1) Office of Recommendations.
(a) Act as the primary staff office responsible for coordinating the implementation of
the Program, including the contributions of other offices.
(b) Assign a specific staff member as the lead person accountable for coordinating the
Most Wanted Program.
(c) Annually develop and recommend a draft Most Wanted List to the Board for
consideration.
12-P-0724
31
-------
(d) Develop and deliver periodic progress reports, at least annually, on the Program to
the Board and others.
(e) Identify, recommend, and participate in advocacy activities, consistent with
resources.
(f) Recommend additions or other changes to the Most Wanted List during the year
as necessary.
(g) Collaborate with the Office of Congressional, Public, and Board Affairs as
described in 5.d., below.
c. Office of Congressional, Public, and Board Affairs.
(1) Prepare and distribute written, audio-visual, and other relevant materials to encourage
coverage of the Most Wanted List and its issues.
(2) Contact media sources to encourage coverage of the Most Wanted List.
(3) Post the Most Wanted List and related information on the CSB website, and update as
needed.
(4) Collaborate with the Office of Recommendations on the items listed in 5.d., below.
d. The two above departments will collaborate in:
(1) helping draft speeches, articles, editorials, and similar written pieces to submit to
print or electronic publications on behalf of Board or staff members;
(2) assisting the Board Members in identifying and participating in events, speaking
engagements, and similar activities likely to contribute to the advocacy of the Most
Wanted List;
(3) assisting the Board Members in preparation of slides, presentations, handouts, and
other materials related to advocacy of the Most Wanted List;
(4) helping identify venues for Board Member and staff presentations that present
opportunities for advancing the Most Wanted List, and encourage Board and staff
members to present; and
(5) preparing web, printed, and audio-visual materials for advocacy activities.
e. Office of Investigations.
(1) Support advocacy activities through testimony, hearings, contacts with recipients and
potential supporters, public presentations, interviews, and similar activities.
(2) Recommend changes to the Most Wanted List during periodic reviews or at other
times, as necessary.
P-0724
32
-------
6. ADVOCACY. Board Members and staff will advocate for the items on the Most Wanted
List through these and similar methods:
a. publicizing the Most Wanted List Report;
b. conducting public education through the CSB webpage;
c. using web announcements, press releases, press conferences, interviews, editorials, and
similar written pieces;
d. providing testimony at federal, state, or local legislative, administrative, and rulemaking
hearings, and similar venues;
e. featuring the Most Wanted List in Board and staff speeches, especially before groups that
may advance the implementation of issues on the list; and
f. engaging in written and verbal communication with recommendation recipients, other
interested parties, and important stakeholders on behalf of items on the Most Wanted
List.
7. REVIEW AND UPDATE. The Managing Director with the support of the Office of
Recommendations and other staff shall be responsible for reviewing this Order on a periodic
basis and for proposing revisions to the Board.
U.S. CHEMICAL SAFETY AND HAZARD INVESTIGATION BOARD
June 12, 2012.
12-P-0724
33
-------
Appendix C
Distribution
Chairperson and Chief Executive Officer, U.S. Chemical Safety and Hazard Investigation Board
Managing Director, U.S. Chemical Safety and Hazard Investigation Board
Deputy Managing Director, U.S. Chemical Safety and Hazard Investigation Board
Counselor to the Chair, U.S. Chemical Safety and Hazard Investigation Board
Director of Administration and Audit Liaison, U.S. Chemical Safety and Hazard
Investigation Board
General Counsel, Office of General Counsel, U.S. Chemical Safety and Hazard
Investigation Board
Director, Office of Recommendations, U.S. Chemical Safety and Hazard Investigation Board
Communications Manager, Office of Congressional, Public, and Board Affairs,
U.S. Chemical Safety and Hazard Investigation Board
12-P-0724
34
------- |