-------
Report Contributors:
Bakari Baker
Heather Drayton
Jeffrey Harris
Lauretta Joseph
Kalpana Ramakrishnan
Abbreviations
EPA
U.S. Environmental Protection Agency
FY
Fiscal Year
IRIS
Integrated Risk Information System
OEI
Office of Environmental Information
OIG
Office of Inspector General
OPPT
Office of Pollution Prevention and Toxics
QAC
Quality Assurance Coordinator
QAM
Quality Assurance Manager
RAD
Risk Assessment Division
TSCA
Toxic Substances Control Act
Cover photo: Elements of the RAD Quality Management Plan. (EPA OIG image—
colors chosen at random)
Are you aware of fraud, waste or abuse in an
EPA program?
EPA Inspector General Hotline
1200 Pennsylvania Avenue, NW (2431T)
Washington, DC 20460
(888) 546-8740
(202) 566-2599 (fax)
OIG Hotline@epa.gov
More information at www.epa.gov/oiq/hotline.html.
EPA Office of Inspector General
1200 Pennsylvania Avenue, NW (241OT)
Washington, DC 20460
(202) 566-2391
www.epa.gov/oig
Subscribe to our Email Updates
Follow us on Twitter @EPAoig
Send us your Project Suggestions
-------
September 10, 2014
x-^tD sr^v^
* • U.S. Environmental Protection Agency 14-P-0350
mm "z Office of Inspector General
® I
At a Glance
Why We Did This Review
Our objective was to determine
to what extent the
U.S. Environmental Protection
Agency (EPA) Office of
Pollution Prevention and
Toxics' (OPPT's) Risk
Assessment Division (RAD)
uses and implements quality
management policies during
chemical risk assessments.
The goal of the quality
management system is to
provide a foundation to "ensure
that environmental data are of
sufficient quantity and quality to
support the data's intended
use." Each EPA office that
collects, evaluates and uses
environmental data is required
to develop a Quality
Management Plan. That plan
defines an organization's
quality-related policies,
procedures, roles,
responsibilities and authorities.
This report addresses the
following EPA goal or
cross-agency strategy:
• Ensuring the safety of
chemicals and
preventing pollution.
Send all inquiries to our public
affairs office at (202) 566-2391
or visit www.epa.gov/oia.
The full report is at:
www.epa.gov/oig/reports/2014/
20140910-14-P-0350.pdf
EPA's Risk Assessment Division Has Not
Fuiiy Adhered to its Quality Management Pian
What We Found
RAD has adhered to some but not all aspects of its
current Quality Management Plan. RAD has not
fully implemented key aspects of its plan related to
training, internal audits and plan revisions.
Additionally, unlike other agency offices, RAD does
not post its Quality Management Plan online as a
good business practice.
Without a robust quality
management system,
RAD risks making
environmental and
human health policy
decisions that rest on a
faulty foundation.
RAD is not ensuring managers and staff take in-house formal quality assurance
training. RAD, instead, relies on branch chiefs and project managers to informally
train staff through mentoring. OPPT is aware of the need to provide formal quality
assurance training to its staff. The Quality Assurance Manager acknowledged the
quality assurance training gap and plans to develop specific quality assurance
training. Additionally, RAD did not conduct a formal quality assurance training
needs assessment.
The RAD Quality Assurance Coordinator has not conducted internal audits of
quality assurance programs. Moreover, RAD has not revised its Quality
Assurance Annual Report and Work Plan or Quality Management Plan when
changes occurred to its program activities that involve major risk assessment
responsibilities. Lastly, RAD does not post its Quality Management Plan on its
Intranet, which can facilitate internal sharing and ease staff access. The EPA
needs to have accurate, reliable and relevant Quality Management Plans
because they are an essential part of valid and reliable decisions. Chemical risk
assessments using high-quality data are critical to maintaining public trust in the
EPA.
Recommendations and Planned Agency Corrective Actions
We recommend that the Assistant Administrator for Chemical Safety and
Pollution Prevention develop formal quality assurance training, direct RAD to
conduct internal quality assurance audits and training needs assessments, and
ensure that relevant RAD Quality Management Plans are updated when changes
to quality assurance activities occur. In addition, the Office of Chemical Safety
and Pollution Prevention needs to provide RAD's Quality Management Plan on
the OPPT Intranet and conduct a quality assurance analysis of OPPT to
determine whether all divisions have fully implemented their quality management
plans. The EPA agreed with our recommendations and has proposed acceptable
corrective actions. All recommendations are resolved.
-------
^£DSX
I \ UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
i S WASHINGTON, D.C. 20460
PRO"^
K THE INSPECTOR GENERAL
September 10, 2014
MEMORANDUM
SUBJECT: EPA's Risk Assessment Division Has Not Fully Adhered to Its
Quality Management Plan
Report No. 14-P-0350
FROM: Arthur A. Elkins Jr.
TO: Jim Jones, Assistant Administrator
Office of Chemical Safety and Pollution Prevention
This is our report on the subject evaluation conducted by the Office of Inspector General (OIG) of the
U.S. Environmental Protection Agency (EPA). This report contains findings that describe problems the
OIG has identified and corrective actions the OIG recommends. This report represents the opinion of the
OIG and does not necessarily represent the final EPA position. Final determinations on matters in this
report will be made by EPA managers in accordance with established audit resolution procedures.
The offices with primary jurisdiction over the issues evaluated in this report are the Office of Chemical
Safety and Pollution Prevention's Office of Pollution Prevention and Toxics, and the Office of Pollution
Prevention and Toxics' Risk Assessment Division.
Action Required
You are not required to provide a written response to this final report because you provided agreed-to
corrective actions and planned completion dates for the report recommendations. The OIG may make
periodic inquiries on your progress in implementing these corrective actions. Should you choose to
provide a final response, we will post your response on the OIG's public website, along with our
memorandum commenting on your response. You should provide your response as an Adobe PDF file
that complies with the accessibility requirements of Section 508 of the Rehabilitation Act of 1973, as
amended.
We will post this report to our website at http://www.epa.gov/oig.
-------
EPA's Risk Assessment Division Has Not Fully
Adhered to Its Quality Management Plan
14-P-0350
Table of C
Purpose 1
Background 1
Responsible Offices 4
Scope and Methodology 4
Results of Review 4
Conclusions 7
Recommendations 8
Agency Comments and OIG Evaluation 8
Status of Recommendations and Potential Monetary Benefits 9
Appendices
A Agency Response to Draft Report and OIG Comments 10
B Distribution 15
-------
Purpose
Our objective was to determine to what extent the U.S. Environmental Protection
Agency's (EPA's) Office of Pollution Prevention and Toxics' (OPPT's) Risk
Assessment Division (RAD) uses and implements quality management policies
during chemical risk assessments.
Background
Since 1979, the EPA has required all organizations1 supporting environmental
programs to participate in the agencywide quality system. The EPA's Office of
Environmental Information (OEI) is responsible for developing quality assurance
and quality control requirements2 and overseeing the agencywide quality system.
The goal of the quality management system is to "ensure that environmental
data. . . are of sufficient quantity and adequate quality to support the data's
intended use."3 A consistent quality management system, when implemented, will
provide the EPA the needed technical practices and management to assure that the
agency decisions are supported by adequate environmental data.
Each EPA office that collects, evaluates and uses environmental data is required
to develop a Quality Management Plan. The Quality Management Plan defines an
organization's quality-related policies, procedures, roles, responsibilities and
authorities. According to the EPA Quality Manual for Environmental Programs,4
the OEI reviews and approves an office's Quality Management Plan, which is
then valid for 5 years.
To assess the effectiveness of the approved quality management plan, program
offices are required to submit a Quality Assurance Annual Report and Work Plan
to the OEI. The OEI also conducts periodic assessments of the EPA's
environmental programs to determine the effectiveness of their mandatory quality
systems and recommend corrective actions.
1 EPA Policy CIO 2105.0 (EPA Order 2105), Policy and Program Requirements for the Mandatory Agency-Wide
Quality System, defines an organization as an office, region, national center or laboratory that collects or uses
environmental data.
2 Quality assurance is an integrated system of management activities to ensure high-quality processes and products.
Quality control, in contrast, focuses on the quality of technical and operational activities. We focused on quality
assurance and related management activities in this report.
3 EPA Order 2105 provides minimum quality system requirements for EPA organizations that collect or use
environmental data. EPA Policy CIO 2106.0, EPA Quality Program Policy, "provides the structure and procedures
to ensure and enhance the effectiveness of the quality program and its application to agency products and services."
In addition, the EPA issued its Information Quality Act guidelines in 2002.
4 The EPA Quality Manual for Environmental Programs provides the minimum program requirements for
implementing the mandatory Quality System defined in EPA CIO 2105-P-01-0.
14-P-0350
1
-------
OPPT and Its Quality Management Plan
OPPT manages programs under the Toxic Substances Control Act (TSCA). The
objective of TSCA is to "allow the EPA to regulate new commercial chemicals
before they enter the market, to regulate existing chemicals when they pose an
unreasonable risk to human health or the environment, and to regulate their
distribution and use."5 OPPT gathers data to provide the EPA information that can
be used and analyzed to reduce the risk of chemical exposure. OPPT helps to
ensure that chemicals used and sold in the United States do not harm human
health and the environment. OPPT has six divisions to address the production,
importation, use and disposal of both existing and new chemicals (see figure 1).
Figure 1. OPPT organizational chart
Office of Pollution
Prevention and
Toxics
I
T
1
Chemical
Control
Division
Chemist
ry,
Economic
Sustains
ble
Strategies D
vision
Information Environmental
Management Assistance
Division Division
Source: OPPT.
OPPT's Director is responsible for the overall quality of OPPT's programs, in
accordance with their quality management plan. The OPPT Director delegates
management of quality assurance activities to the Quality Assurance Manager
(QAM). The QAM has full responsibility and authority to implement OPPT's
quality assurance activities with respect to the agency's quality management
systems. The QAM is the official point of contact for all quality assurance matters
and is the liaison between OPPT and OEI. The QAM is responsible for reviewing
and approving each OPPT division's Quality Management Plan and quality
assurance project plans. Each OPPT Division Deputy Director works as the
Quality Assurance Coordinator (QAC) and oversees the division's quality
assurance activities.
OPPT began an office reorganization in the fall of 2013 which coincided with
OPPT's 5-year Quality Management Plan renewal deadline. OPPT requested an
extension to submit the renewed Quality Management Plans to OEI and OEI
approved the extension. OEI may grant an extension if it recently conducted a
quality systems assessment, and OEI had conducted a quality systems assessment
5 15 U.S.C. §2601.
14-P-0350
2
-------
of OPPT in November 2012. OEI's 2012 quality systems assessment found that
OPPT's quality systems were in compliance with their Quality Management Plan.
OPPT must submit an updated Quality Management Plan to OEI by September
2014. Although OPPT has its own Quality Management Plan, each of its six
divisions are responsible for implementing its own divisional Quality
Management Plan.
RAD and Its Quality Management Plan
RAD consists of four branch offices (see figure 2) and is responsible for
numerous activities assessing the health and environmental hazards and risks of
new and existing chemicals and microorganisms for both regulatory and
non-regulatory programs. The division's responsibilities include assessing the
hazards of high-production-volume chemicals. The main components of these
assessments are reviewing and evaluating test data submitted under TSC A for
environmental and health effects. RAD also manages the Structure-Activity
Team, which is responsible for initial assessment of fate and effects of chemicals.
Figure 2. RAD organizational chart
Risk Assessment
Division
Existing Chemicals
Assessment Branch
T
High Production
Volume Chemicals
Branch
T
Scientific
Support Branch
~l
New Chemicals
Screening
Assessment Branch
Source: RAD.
According to RAD's Quality Management Plan, it uses numerous tools to ensure
data quality.6 RAD develops Quality Assurance Project Plans and uses standard
operating procedures and other guidance documents to prepare work products and
risk assessments. RAD uses external peer reviews, such as the agency's Science
Advisory Board; journal article and paper peer reviews; and, occasionally, the
Office of Pesticide Program's Science Advisory Panel. The RAD QAC is
assigned the responsibility to assess quality assurance training needs and conducts
annual internal audits of its quality assurance program. The branch chiefs are
responsible for ensuring that needed training is scheduled, funded and taken. OEI
guidance provides information on minor and major revisions. Minor revisions to
the Quality Management Plan must be documented in the annual quality
assurance report. Major revisions or reissuance of the Quality Management Plan
are required when there is a significant mission change or major reorganizations.
Similar to OPPT, RAD has committed to submitting an updated Quality
Management Plan for approval to OEI by September 2014.
6 The RAD "Quality Management Plan exists in the context of the OPPT Quality Management Plan," which is under
the control of the OPPT QAM.
14-P-0350
3
-------
Responsible Offices
The offices responsible for the issues evaluated in this report are the Office of
Chemical Safety and Pollution Prevention's OPPT and OPPT's RAD.
Scope and Methodology
We conducted this performance audit in accordance with generally accepted
government auditing standards. Those standards require that we plan and perform
the audit to obtain sufficient, appropriate evidence to provide a reasonable basis
for our findings and conclusions based on our objectives. We believe that the
evidence obtained provides a reasonable basis for our results based upon our
objectives. We conducted this audit from January to July 2014.
We reviewed relevant documents, including laws, regulations, policies and
procedures. We focused on management activities listed in EPA Order 2105 and
the EPA Quality Manual for Environmental Programs. We also reviewed quality
guidance documents, including the current OPPT and RAD Quality Management
Plans.7
Our review of the current RAD Quality Management Plan focused on quality
assurance training, internal audits and Quality Management Plan revisions. We
interviewed agency officials, including OEI's Director of Quality Staff, OPPT's
Deputy Director of Programs, OPPT's QAM, RAD's Division Director and
Deputy Division Director/QAC, and other RAD personnel. We reviewed the
RAD's Quality Assurance Project Plans, standard operating procedures, and the
fiscal years (FYs) 2009/2010, 2012/2013 and 2013/2014 annual quality assurance
reports and work plans. We obtained information regarding quality assurance
training and types of RAD work products. We reviewed quality management
plans from the Office of Pesticide Programs and the Office of Research and
Development. We also reviewed annual quality assurance reports from the Office
of Resource Conservation and Recovery and the Office of Pesticide Program's
Biological and Economic Analysis Division.
Results of Review
According to the RAD's Quality Management Plan, the division uses the elements
shown in table 1 to ensure data quality. During our review, we found that RAD
has adhered to some but not all aspects of its current Quality Management Plan.
RAD did not fully implement key aspects of its Quality Management Plan related
to training, internal audits and plan revisions. Additionally, we found that, unlike
other agency offices business practices, RAD does not post its Quality
Management Plan online.
7 For both OPPT and RAD, the 2008 Quality Management Plan is the most current version.
14-P-0350
4
-------
Table 1: RAD adherence to quality assurance activities
Quality assurance requirement
Adherence
RAD QAC assigned.
Yes
Management review of quality assurance project plans and
standard operating procedures.
Yes
Identification of quality assurance training needs.
No
Provide in-house quality assurance training.
No
Contribute to the OPPT annual report.
Yes
Revisions and updates to Quality Management Plan.
No
Yearly internal audits of divisional quality assurance program.
No
Source: OIG analysis of quality management plan requirements.
Quality Assurance Training
Within its Quality Management Plan, RAD lists "providing in-house formal
training classes" as a training mechanism. Yet RAD is not ensuring managers and
staff take in-house formal quality assurance training—a key aspect of its current
Quality Management Plan. RAD training mechanisms include: attending
scientific and professional society meetings and workshops, attending seminars
and training classes offered by various software and vendors, and offering
recertification classes and informal peer-to-peer training. The Office of Water
offered quality assurance training in FY 2013 that only seven RAD staff members
attended; however, although applicable to the environmental data RAD works
with, the training was not RAD-specific.
RAD senior management stated that RAD does not have formal quality assurance
training.8 One of RAD's four branch chiefs commented that he could not
remember taking any quality assurance training since becoming a branch chief.
The RAD division director informed the Office of Inspector General (OIG) that
their training process is informal. In FY 2012/2013, informal training was
achieved through a technical discussion of topics presented to a team of peers.
RAD also relies on branch chiefs and project managers to train staff through
mentoring. However, without formal quality assurance training, RAD will be
unable to ensure that the information passed from mentoring is consistent or the
best quality assurance information is available. Additionally, without formal
training in place, institutional knowledge can be lost with employee attrition.
8 OEI does not provide a formal overall quality assurance training program.
14-P-0350
5
-------
OPPT is aware of the need to provide quality assurance training to its staff. The
QAM acknowledged the quality assurance training gap and plans to develop
specific quality assurance training for OPPT, which will include RAD. The QAM
stated that there are plans to design training with other agency QAMs, which
would potentially establish a more solid program for the entire OPPT quality
assurance system. Additionally, a meeting was held in March 2014 with the EPA
quality assurance community to discuss ways to better serve their respective
program offices.
In addition to requiring quality assurance training, RAD's current Quality
Management Plan states that the QAC is responsible for conducting formal
quality assurance training needs assessments. Training needs assessments are
necessary to ensure that quality assurance-related training needs are addressed and
the resources are available to complete the training. However, according to
OPPT's quality assurance annual reports (FYs 2009, 2012 and 2013), RAD did
not conduct a formal quality assurance training needs assessment.
Internal Quality Assurance Audits
The RAD QAC has not conducted internal audits of their quality assurance
programs. A quality audit is defined as "a systematic and independent
examination to determine whether quality activities and related results comply
with planned arrangements and whether these arrangements are implemented
effectively and are suitable to achieve objectives."9 As stated earlier, the role of
the QAC is to ensure quality assurance of their division. According to the current
RAD Quality Management Plan, the division's QAC is required to conduct annual
internal quality assurance audits of their programs. According to OPPT's quality
assurance annual reports (FYs 2009, 2012 and 2013), RAD did conduct routine
reviews of work products by staff and the assigned management teams. RAD also
had external peer reviews of scientific data. However, these activities do not
qualify as annual internal quality audits by RAD QAC. Moreover, during
interviews, RAD senior management acknowledged that their division has not
conducted any internal quality assurance audits. Routine internal quality
assurance audits help identify potential deficiencies in data quality and
management controls.
Quality Management Plan Revisions
RAD has not revised its Quality Management Plan when changes occurred to its
program activities, including major risk assessment responsibilities. The EPA
Quality Manual for Environmental Programs requires that the Quality
Management Plan be reviewed annually and updated or reissued as needed.
Necessary updates include major revisions to program roles and responsibilities.
The current RAD Quality Management Plan specified that they develop and
prepare Integrated Risk Information System (IRIS) assessments. In contrast, RAD
9 Defined by the EPA Quality Manual for Environmental Programs, CIO 2105-P-01-0, May 2000.
14-P-0350
6
-------
stopped producing IRIS assessments in 2011 and a subsequent Federal Register
notice announced the elimination of RAD's role in the IRIS preparation.
However, the revisions to their quality assurance activities regarding IRIS
assessments have not been documented in either the quality assurance annual
report and work plan or a revised Quality Management Plan. In addition, the
current RAD Quality Management Plan included information on the Chemical
Assessment and Management Program, which is a program that ceased in 2009.
Although this change was contained in the FY 2009 annual quality assurance
report, it was a major shift in quality assurance activities for RAD and, therefore,
should have resulted in a revision or reissuance of their Quality Management Plan
in accordance with the EPA and RAD quality assurance guidance.
Quality Management Plan Accessibility
RAD does not post its Quality Management Plan on its Intranet. Posting the plan
would increase transparency and enhance availability to the EPA staff. As a good
business practice, we found that other agency offices make their quality
management plans available to all by posting it on the EPA's Intranet. Quality
management plans, annual quality assurance reports, internal audits, and revisions
to the plans that are posted to an intranet can be easily found by managers and
staff. For instance, the Office of Pesticide Programs has its Quality Management
Plan online. Its Quality Management Plan includes training requirements for its
QAM, quality assurance officers and contracting officer representatives. It goes
further to list the specific quality assurance training that these individuals must
take. The Office of Research and Development includes revisions or updates to its
Quality Management Plan on the Office of Research and Development's Intranet.
Conclusions
The lack of adherence to several aspects of RAD's Quality Management Plan
creates the risk that RAD's quality system cannot meet the EPA's quality
management system's goal. Without a robust quality management system, RAD
risks having environmental and human health policy decisions that rest on a faulty
foundation. To enhance its quality management system, RAD needs to implement
quality assurance training and conduct training needs assessments to ensure that
managers and staff obtain relevant quality assurance knowledge. RAD's QAC
needs to conduct annual independent reviews of the division's quality assurance
processes to assure the quality of work products. The EPA needs to have accurate,
reliable and relevant Quality Management Plans because they are an essential part
of valid and reliable decisions. Chemical risk assessments using high-quality data
are critical to maintaining public trust in the EPA.
14-P-0350
7
-------
Recommendations
We recommend that the Assistant Administrator for Chemical Safety and
Pollution Prevention:
1. Develop formal quality assurance training that enhances awareness and
understanding of relevant quality management policies and requirements
for managers and staff to prevent loss of institutional knowledge and
comply with their Quality Management Plan.
2. Direct RAD's QAC to conduct annual internal quality assurance audits in
accordance with RAD's Quality Management Plan.
3. Direct RAD to identify and document individual staff training needs to
ensure that RAD addresses quality assurance-related training gaps.
4. Ensure that RAD's Quality Management Plan and/or OPPT's quality
assurance annual report and work plan are updated accordingly when
minor and major changes to RAD's quality assurance activities are made.
5. Provide internal online access to RAD's Quality Management Plan to
increase staff accessibility to relevant quality assurance activities.
6. Conduct a quality assurance analysis of OPPT to determine whether all
divisions have fully implemented their Quality Management Plans.
Agency Comments and OIG Evaluation
The agency agreed with our recommendations, and provided corrective actions
and estimated completion dates that meet the intent of the recommendations.
Based on the agency's written response, we have determined that the
recommendations are resolved and open with corrective actions ongoing. One
recommendation is closed due to being implemented prior to the final report
issuance. No further response to this report is required. The agency's detailed
response is in appendix A and our response to the agency is embedded in
appendix A. The agency also provided technical comments on the draft report,
which we have incorporated into our report as appropriate.
14-P-0350
8
-------
Status of Recommendations and
Potential Monetary Benefits
RECOMMENDATIONS
POTENTIAL MONETARY
BENEFITS (In $000s)
Rec.
No.
No.
Subject
Status1
Action Official
Planned
Completion
Date
Claimed
Amount
Ag reed-To
Amount
Develop formal quality assurance training that
enhances awareness and understanding of
relevant quality management policies and
requirements for managers and staff to prevent
loss of institutional knowledge and comply with
their Quality Management Plan.
Direct RAD's QAC to conduct annual internal
quality assurance audits in accordance with RAD's
Quality Management Plan.
Direct RAD to identify and document individual staff
training needs to ensure that RAD addresses
quality assurance-related training gaps.
Ensure that RAD's Quality Management Plan
and/or OPPT's quality assurance annual report and
work plan are updated accordingly when minor and
major changes to RAD's quality assurance
activities are made.
Provide internal online access to RAD's Quality
Management Plan to increase staff accessibility to
relevant quality assurance activities.
Conduct a quality assurance analysis of OPPT to
determine whether all divisions have fully
implemented their Quality Management Plans.
Assistant Administrator for 12/31/14
Chemical Safety and
Pollution Prevention
Assistant Administrator for 9/30/15
Chemical Safety and
Pollution Prevention
Assistant Administrator for 9/30/15
Chemical Safety and
Pollution Prevention
Assistant Administrator for 9/30/15
Chemical Safety and
Pollution Prevention
Assistant Administrator for 7/29/14
Chemical Safety and
Pollution Prevention
Assistant Administrator for 9/30/15
Chemical Safety and
Pollution Prevention
1 0 = Recommendation is open with agreed-to corrective actions pending.
C = Recommendation is closed with all agreed-to actions completed.
U = Recommendation is unresolved with resolution efforts in progress.
14-P-0350
9
-------
Appendix A
Agency Response to Draft Report and OIG Comments
July 30, 2014
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
MEMORANDUM
SUBJECT: Response to Office of Inspector General Draft Report No. OPE-FY14-0012
"EPA's Risk Assessment Division has Not Fully Adhered to Its Quality
management Plan," dated June 30, 2014
Thank you for the opportunity to respond to the subject audit report. This memorandum provides
the Agency's response to OIG's recommendations, and identifies corrective actions the Agency
will be taking in response (Attachment A). In addition, the Agency has suggested technical
corrections to improve the accuracy of the report (Attachment B).
Every day the Office of Pollution Prevention and Toxics (OPPT) produces and uses scientific
and technical work products that inform Agency decision making. We make approximately
1,000 new chemicals decisions a year based on assessments that are unchallenged by industry
and other stakeholders. Our tools and models are used around the world for chemicals-related
decision making. OPPT scientists play leading roles internationally in such bodies as the OECD
to develop the next-generation of chemical assessment guidelines and risk assessment
approaches. OPPT's Work Plan risk assessment program for existing chemicals is producing
assessments that are subjected to independent, expert review that surpasses in rigor federal and
Agency guidelines for peer review. It is notable that the external peer reviews of all the Work
Plan assessments we have completed thus far have supported our overall assessment
methodologies and conclusions. OPPT's record of strong and sound science is consistent with
OIG Response: The scope of our evaluation did not include a review of the quality of
OPPT's scientific and technical work products.
14-P-0350 10
FROM: James J. Jones
Assistant Administrator for Chemical Safety and Pollution Prevention
TO:
Arthur A. Elkins, Jr.
Inspector General (OIG)
-------
the fact that your office's review of our quality system found no concerns with the quality of
OPPT's scientific and technical work products.
The administrative recommendations that have resulted from your audit are very useful, and we
look forward to implementing them. The report contains a total of six recommendations:
Recommendation 1. Develop formal quality training that enhances awareness and understanding
of relevant quality management policies and requirements for managers and staff to prevent loss
of institutional knowledge and comply with their quality management plan.
Recommendation 2. Direct RAD's QAC to conduct annual internal quality assurance audits in
accordance with RAD's Quality Management Plan.
Recommendation 3. Direct RAD to identify and document individual staff training needs to
ensure that RAD addresses quality assurance-related training gaps.
Recommendation 4. Ensure that RAD's Quality Management Plan and/or OPPT's quality
assurance annual report and work plan updated accordingly when minor and major changes to
RAD's quality assurance activities are made.
Recommendation 5. Provide internal online access to RAD's Quality Management Plan to increase
staff accessibility to relevant quality assurance activities.
Recommendation 6. Conduct a quality assurance analysis of OPPT to determine whether all divisions
have fully implemented their Quality Management Plans.
The Agency concurs with all of the report recommendations and will begin steps to implement
the corrective actions identified in Attachment A.
We do take issue with one finding. The draft report states (p. 7) that OPPT's changes in program
activities since 2009 have led to a major shift in quality assurance activities, and implies that its
current Quality Management Plan is not accurate, reliable, or relevant. The final report should
take care not to convey this implication. Our QMP is relevant to and supportive of all OPPT
scientific and technical activities, including our new and existing chemical risk assessments.
While the policy goals and programmatic focus for conducting data reviews and evaluations
within OPPT have changed over the past 5 years, the sources and types of data, the evaluation
approaches and tools, and the QA/QC activities associated with using this data have not changed
significantly or substantively. This is largely due to OPPT's adherence to Agency assessment
guidelines and contract management processes and procedures. We did not make changes to the
QMP because changes in program activities were explained in the Quality Assurance Annual
Report and Work Plan (QAARWP), and the QA/QC activities did not change in any substantive
way. Each QAARWP has a section for addressing changes to the QMP. In each of the
QAARWPs for 2008-2013, OPPT noted that no changes to the QMP were warranted; the
QAARWPs were submitted to and accepted by the Agency's Office of Environmental
Information.
14-P-0350
11
-------
OIG Response: The OIG did not state that RAD's current QMP is not accurate, reliable or
relevant. We specifically state that RAD has adhered to some but not all aspects of its
current quality management plan. OEI's guidance was used to determine RAD's use and
implementation of its QMP. Our review of OEI guidance and RAD's activities indicated
that certain RAD program changes were major activities and thereby should have resulted
in an updated QMP. OIG suggests that OPPT contact OEI to obtain more information or
request updated guidance that further clarifies how to determine a major activity.
If you have questions or need further information about this response, please contact Deborah
Hartman, OCSPP's Audit Liaison at (202) 564-1488.
14-P-0350
12
-------
Attachment A: Corrective Action Plan
EPA's Risk Assessment Division Has Not Fully Adhered to Its Quality Management Plan
Report No. OPE-FY14-0012 (June 30, 2014)
Recommendation
Corrective Action
Target Date
Recommendations to OCSPP Assistant Administrator
1. Develop formal quality
training that enhances
awareness and
understanding of
relevant quality
management policies
and requirements for
managers and staff to
prevent loss of
institutional knowledge
and comply with their
quality management
plan.
OCSPP accepts the
OIG recommendation.
OPPT will develop a
formal quality training
program.
FY15; Q1
2. Direct RAD's QAC to
conduct annual internal
quality assurance audits
in accordance with
RAD's Quality
Management Plan.
OCSPP accepts the
OIG recommendation.
The OPPT-RAD QAC
will conduct internal
quality assurance
audits.
Annually, beginning in FY15.
3. Direct RAD to identify
and document
individual staff training
needs to ensure that
RAD addresses quality
assurance-related
training gaps.
OCSPP accepts the
OIG recommendation.
OPPT-RAD will
identify and document
Division staff training
needs in the annual
QA/AC report.
Individual staff
training needs will be
addressed in
Individual
Development Plans
Annually, beginning in FY15.
4. Ensure that RAD's
Quality Management
Plan and/or OPPT's
quality assurance
annual report and work
OCSPP accepts the
OIG recommendation.
OPPT-RAD will more
explicitly identify and
Annually, beginning in FY15 or when a
major change in QA/QC activity occurs.
14-P-0350
13
-------
plan updated
accordingly when
minor and major
changes to RAD's
quality assurance
activities are made.
document, in the
annual QA/QC report,
when changes to
quality assurance
activities are made.
5. Provide internal online
access to RAD's Quality
Management Plan to
increase staff
accessibility to relevant
quality assurance
activities.
OCSPP accepts the
OIG recommendation.
OPPT has posted its
QMP on the intranet.
Completed
July 29, 2014
httD ://intranet. eoa. gov/ODDtwork/aualitv-
management/
6. Conduct a quality
assurance analysis of
OPPT to determine
whether all divisions
have fully implemented
their Quality
Management Plans.
OCSPP accepts the
OIG recommendation.
FY15
14-P-0350
14
-------
Appendix B
Distribution
Office of the Administrator
Assistant Administrator for Chemical Safety and Pollution Prevention
Agency Follow-Up Official (the CFO)
Agency Follow-Up Coordinator
General Counsel
Associate Administrator for Congressional and Intergovernmental Relations
Associate Administrator for External Affairs and Environmental Education
Deputy Assistant Administrator for Chemical Safety and Pollution Prevention
Director, Office of Pollution Prevention and Toxics, Office of Chemical Safety and
Pollution Prevention
Director, Risk Assessment Division, Office of Pollution Prevention and Toxics,
Office of Chemical Safety and Pollution Prevention
Audit Follow-Up Coordinator, Office of Chemical Safety and Pollution Prevention
Audit Follow-Up Coordinator, Office of Pollution Prevention and Toxics,
Office of Chemical Safety and Pollution Prevention
14-P-0350
15
-------