EPA Completed OIG Recommendations for the Presidential Green Chemistry Challenge Awards Program but Lacks Controls over Use of Unverified Results


^tDsr^
$ O
® J
U.S. ENVIRONMENTAL PROTECTION AGENCY
OFFICE OF INSPECTOR GENERAL
Operating efficiently and effectively
EPA Completed OIG
Recommendations for the
Presidential Green Chemistry
Challenge Awards Program but
Lacks Controls over Use of
Unverified Results
Report No. 18-P-0222
July 20, 2018

-------
Report Contributors:
Jee W. Kim
Eric Lewis
Andre von Hoyer II
Abbreviations
AFC	Audit Follow-Up Coordinator
EPA	U.S. Environmental Protection Agency
FY	Fiscal Year
MATS	Management Audit Tracking System
OCSPP	Office of Chemical Safety and Pollution Prevention
OIG	Office of Inspector General
OPPT	Office of Pollution Prevention and Toxics
P2	Pollution Prevention
PGCCA	Presidential Green Chemistry Challenge Awards
Cover Photo: Presidential Green Chemistry Challenge Awards on display. (EPA photo)
Are you aware of fraud, waste or abuse in an
EPA program?
EPA Inspector General Hotline
1200 Pennsylvania Avenue, NW (2431T)
Washington, DC 20460
(888) 546-8740
(202) 566-2599 (fax)
OIG Hotline@epa.gov
Learn more about our OIG Hotline.
EPA Office of Inspector General
1200 Pennsylvania Avenue, NW (2410T)
Washington, DC 20460
(202) 566-2391
www.epa.gov/oiq
Subscribe to our Email Updates
Follow us on Twitter @EPAoig
Send us your Project Suggestions

-------
^£DS7X
• JL v
I®/
U.S. Environmental Protection Agency
Office of Inspector General
At a Glance
18-P-0222
July 20, 2018
Why We Did This Project
We conducted this audit to
accomplish two objectives:
•	Identify the controls put in
place by the U.S.
Environmental Protection
Agency (EPA) for use during
the fiscal year 2017
Presidential Green Chemistry
Challenge Awards (PGCCA)
to prevent the use of
unverified, self-reported
results for agency reporting
purposes.
•	Assess the EPA's actions to
address recommendations in
Office of Inspector General
(OIG) Report No. 15-P-0279.
EPA's Presidential Green
Chemistry Challenge Awards
Program Lacks Adequate
Support and Transparency
and Should Be Assessed for
Continuation, issued
September 2015.
The PGCCA Program is
sponsored by the EPA's Office of
Chemical Safety and Pollution
Prevention to promote the
environmental and economic
benefits of developing and using
green chemistry by recognizing
industry innovations.
This report addresses the
following:
•	Operating efficiently and
effectively.
Send all inquiries to our public
affairs office at (202) 566-2391
or visit www.epa.gov/oia.
EPA Completed OIG Recommendations for the
Presidential Green Chemistry Challenge Awards Program
but Lacks Controls over Use of Unverified Results
What We Found
In 2015, we reported that award
results submitted to the EPA's
Pollution Prevention Program from
PGCCA recipients were not
adequately supported or
transparent. Specifically, we found
that PGCCA results were self-reported by award recipients, were not verified or
validated by the agency, and were included as metrics in agency reports
detailing the Pollution Prevention Program's performance and goals.
The OIG's 2015 report contained nine recommendations. During this current
audit, we verified that the EPA completed the agreed-upon corrective actions
for seven of these recommendations. The remaining two recommendations
were not considered in this current audit; the EPA completed one before we
issued our 2015 report, and the EPA and OIG later agreed that the other
recommendation was not applicable.
By completing these outstanding recommendations, the EPA discontinued the
use of unverified PGCCA results in its Pollution Prevention Program
performance metrics, developed a program logic model that communicates how
PGCCA contribute to Pollution Prevention Program goals, and established
internal metrics that track program outputs and provide future direction for the
PGCCA Program. However, we found that the EPA lacked documented internal
controls to prevent the use of PGCCA results in agency performance metrics.
Without documented controls, there is a risk that unverified PGCCA results
could be used as part of future agency metrics (for example, if and when new
staff become involved with the PGCCA Program).
We also found that the Office of Chemical Safety and Pollution Prevention
disagreed with the OIG about the requirements regarding supporting
documentation for completed corrective actions.
Recommendation and Planned Agency Corrective Action
We recommend that the Assistant Administrator for Chemical Safety and
Pollution Prevention establish and document internal controls to prevent the
use of PGCCA results in Pollution Prevention Program performance metrics.
The EPA agreed with our recommendation and provided an acceptable
planned corrective action and completion date.
The EPA discontinued the use of
unverified PGCCA results in agency
performance metrics, but a lack of
documented controls presents risk that
these data may be used in the future.
Listing of OIG reports.

-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
THE INSPECTOR GENERAL
July 20, 2018
MEMORANDUM
SUBJECT: EPA Completed OIG Recommendations for the Presidential Green Chemistry Challenge
Awards Program but Lacks Controls over Use of Unverified Results
TJpnnrtNn 1 8-P-D99?
This is our report on the subject audit conducted by the Office of Inspector General (OIG) of the
U.S. Environmental Protection Agency (EPA). The project number for this audit was OPE-FY18-0003.
This report contains findings that describe the problems the OIG has identified and the corrective action
the OIG recommends. This report represents the opinion of the OIG and does not necessarily represent the
final EPA position. Final determinations on matters in this report will be made by EPA managers in
accordance with established audit resolution procedures.
In accordance with EPA Manual 2750, your office provided an acceptable corrective action and
milestone date in response to the OIG recommendation. The recommendation is resolved and no final
response to this report is required. However, if you submit a response, it will be posted on the OIG's
website, along with our memorandum commenting on your response. Your response should be provided
as an Adobe PDF file that complies with the accessibility requirements of Section 508 of the
Rehabilitation Act of 1973, as amended. The final response should not contain data that you do not want
to be released to the public; if your response contains such data, you should identify the data for
redaction or removal along with corresponding justification.
We will post this report to our website at www.epa.gov/oig.
FROM:
Arthur A. Elkins Jr.
TO:
Charlotte Bertrand, Acting Principal Deputy Assistant Administrator
Office of Chemical Safety and Pollution Prevention

-------
EPA Completed OIG Recommendations for the
Presidential Green Chemistry Challenge Awards Program
but Lacks Controls over Use of Unverified Results
18-P-0222
Table of C
Purpose		1
Background		1
Responsible Office		1
Scope and Methodology		2
Results		2
Conclusion		5
Recommendation		5
Agency Response and OIG Evaluation		5
Status of Recommendations and Potential Monetary Benefits		6
Appendices
A Agency Response to Draft Report	 7
B Distribution	 9

-------
Purpose
The U.S. Environmental Protection Agency's (EPA's) Office of Inspector
General (OIG) conducted this audit to address two objectives:
•	Identify the internal controls put in place by the EPA for use during the
fiscal year (FY) 2017 Presidential Green Chemistry Challenge Awards
(PGCCA) to prevent the use of unverified, self-reported results for agency
reporting purposes, as related to EPA performance measures and goals.
•	Verify that the EPA implemented the corrective actions documented in the
agency's Management Audit Tracking System (MATS) in response to our
recommendations in OIG Report No. 15-P-0279. EPA's Presidential
Green Chemistry Challenge Awards Program Lacks Adequate Support
and Transparency and Should Be Assessedfor Continuation, issued
September 15, 2015.
Background
According to the 2015 OIG report, the PGCCA Program is part of the EPA's
Pollution Prevention (P2) Program. The P2 Program's mission is to prevent
pollution at the source, promote the use of greener substances and conserve
natural resources. Through the P2 Program, the EPA encourages and supports
innovative changes in industrial production and the use of raw materials. The
PGCCA Program promotes the environmental and economic benefits of
developing and using green chemistry by recognizing industry innovations.
The OlG's 2015 Report
In OIG Report No. 15-P-0279. the OIG found that all PGCCA results were self-
reported by award recipients. The EPA did not verify or validate award results,
and recipients were not required to conduct quality assurance certifications on the
results they reported. Moreover, we found that the EPA used these data as part of
its P2 Program performance metrics. The OIG made nine recommendations,
including that the EPA discontinue using data from the PGCCA in its P2 Program
performance metrics until data quality controls were in place. The OIG also
recommended that the EPA assess the need for and value of the PGCCA Program
to support agency goals. Based on these 2015 findings, the agency committed to
discontinuing the use of PGCCA Program results in P2 Program performance
metrics by September 30, 2015.
Responsible Office
The EPA's Office of Pollution Prevention and Toxics (OPPT), within the Office
of Chemical Safety and Pollution Prevention (OCSPP), is responsible for the
issues discussed in this report.
18-P-0222
1

-------
Scope and Methodology
We conducted this performance audit from January through May 2018 in
accordance with generally accepted government auditing standards. Those
standards require that we plan and perform the audit to obtain sufficient,
appropriate evidence to provide a reasonable basis for our findings and conclusions
based on our audit objectives. We believe that the evidence obtained provides a
reasonable basis for our findings and conclusions based on our audit objectives.
To answer our first objective, we interviewed key agency management and staff
within the OCSPP and OPPT. For our second objective, we assessed the status of
the seven outstanding recommendations from the 2015 OIG report. We
interviewed agency staff and the prior OIG audit team project manager. We
verified whether the information recorded in MATS matched documentation
provided to us by the EPA. We also reviewed other pertinent documents about the
corrective actions. We did not review Recommendations 2.c and 3 from the 2015
report. Recommendation 2c was deemed "not applicable" by the agency, and the
OIG concurred.1 Recommendation 3 was completed before the OIG issued its
2015 report.
Results
The OCSPP completed all seven corrective actions that remained outstanding
from the OIG's 2015 report. The data in MATS matched the agency's
certification memorandum detailing these corrective actions. However, we found
that the OCSPP lacked suitable controls to prevent the use of PGCCA data for
agency reporting purposes in the future. The OCSPP also disagreed with the OIG
regarding the documentation required by EPA Manual 27502 to adequately
support the completion of corrective actions.
EPA Completed All Outstanding Corrective Actions
The EPA reported in MATS that it completed all outstanding corrective actions
for the 2015 report recommendations, and the OCSPP provided us with a
certification memorandum and other supporting documentation regarding these
corrective actions. Our current 2018 audit verified that the information reported in
MATS was consistent with the supporting documentation, as shown in Table 1.
1 In its response to the OIG's final 2015 report, the EPA said that Recommendation 2.c was "not applicable" because
PGCCA data would be used for "information outreach and educational purposes only." Therefore, the agency said
the segregation of data to show domestic versus international benefits was unnecessary. The OIG concurred.
2 EPA Manual 2750, Audit Management Procedures, March 28, 2017.
18-P-0222
2

-------
Table 1: EPA corrective actions for outstanding 2015 report recommendations
Outstanding
recommendations
Status of recommendations
No.
Actions recommended
1
Discontinue the use of Green
Chemistry Awards data in
EPA pollution prevention
performance metrics until
controls over data quality are
implemented.
EPA corrective action taken
The OCSPP discontinued providing PGCCA results to the P2 Program in FY 2015.
OIG analysis of corrective action
We reviewed the agency's Annual Performance Measures documents and
confirmed that the EPA did not use PGCCA results within its P2 Program
performance metrics in FYs 2015 and 2016. Due to the annual "data lag" in
reporting performance measures, we were not able to confirm this corrective
action for FY 2017.
2.a
Implement a system to track
and analyze data and
environmental results
collected by the program.
EPA corrective action taken
The OCSPP completed entering backlogged PGCCA data into the existing
tracking system (the Green Chemistry Matrix System).
OIG analysis of corrective action
We reviewed the OCSPP's Green Chemistry Matrix System and verified
completed data entries for the 11 awardees in FYs 2015 and 2016.
2.b
Develop a program feedback
system that includes a
process for gathering
information on the
subsequent impact(s) of
projects that have received
awards, and includes tracking
data to evidence the long-
term benefits of green
chemistry innovations.
EPA corrective action taken
The OCSPP arranged numerous site visits with award winners, as well as
participated in several meetings and conferences attended by award winners and
stakeholders. The OCSPP was able to obtain updates on green chemistry
technologies and gather additional information on their impacts.
OIG analysis of corrective action
We reviewed numerous examples of site visits and meetings/conferences and
verified that the OCSPP gathered information related to specific award recipients
and the impact of the technologies.
2.d
Develop program-specific
goals, objectives and
measures.
EPA corrective action taken
The OCSPP updated its existing P2 Program logic model to include PGCCA-
specific outputs, goals and measures.
OIG analysis of corrective action
We reviewed the P2 Program logic model and verified that the model included
PGCCA-specific goals, objectives and measures.
2.e
Link the program's activities
to EPA and Office of
Chemical Safety and
Pollution Prevention strategic
plan goals and performance
measures.
EPA corrective action taken
The OCSPP updated its existing P2 Program logic model to link its activities to
EPA and OCSPP strategic plan goals and measures.
OIG analysis of corrective action
We reviewed the P2 Program logic model and verified that the model included
links to EPA and OCSPP strategic plan goals and measures.
2.f
Create a program-specific
logic model that reflects
outputs and short-,
intermediate- and long-term
outcomes of the program.
EPA corrective action taken
The OCSPP updated its existing P2 Program logic model to include outputs and
outcomes of the PGCAA Program.
OIG analysis of corrective action
We reviewed the P2 Program logic model and verified that the model included
PGCCA Program outputs and outcomes.
2.g
Periodically review the
program to evaluate results
and to assess progress in
achieving goals.
EPA corrective action taken
The OCSPP conducted an FY 2016 review of the PGCCA Program.
OIG analysis of corrective action
We reviewed the OCSPP FY 2016 PGCCA review report and verified that the
OCSPP completed its FY 2016 PGCCA Program review.
Source: EPA MATS, OIG Report No. 15-P-0279. information provided by agency personnel, and OIG analysis.
18-P-0222

-------
EPA Lacks Internal Controls to Prevent Future Misuse of
PGCCA Results as Agency Performance Metrics
As a result of the 2015 OIG report, the agency committed to discontinuing the use
of unverified, self-reported PGCCA results for agency reporting purposes, as
related to EPA performance measures and goals, by September 30, 2015.
However, EPA managers said that the OPPT has established no documented
internal controls to prevent such use. EPA managers said that only a small number
of staff work on the PGCCA and that these staff know not to use PGCCA results
for agency performance metrics.
A small staff may not be an effective control. Specifically, departures of key
personnel may prevent this institutional knowledge from being retained. A lack of
documented procedures could mean that new staff are never informed that they
should not include the PGCCA results as part of agency performance metrics.
While we did verify that PGCCA results were not incorporated into EPA
performance measures and goals for FYs 2015 and 2016 (see Table 1,
Recommendation 1), there are no internal controls to prevent PGCCA results
from being included in agency performance measures in the future.
OCSPP Disagrees with OIG About Documentation Requirements
EPA Manual 2750 requires that the audit follow-up coordinator (AFC) for each
performance audit maintain "[documentation supporting completion of corrective
actions" taken, as well as audit files that include "proof of completed corrective
actions" and "[a]ll pertinent documentation of corrective actions taken." In
support of this current audit, the OCSPP AFC provided a certification
memorandum signed by the then-OCSPP Assistant Administrator and internal
emails from OCSPP staff confirming that the corrective actions for the
2015 recommendations were completed. The OCSPP AFC stated that the
certification memorandum and internal emails meet the documentation
requirements of EPA Manual 2750.
EPA Manual 2750 is an agency document that is subject to agency interpretation;
however, the OCSPP interpretation does not provide the proof of completion that
the OIG requires. The certification memorandum and emails were inadequate
because they did not provide evidence that the actions occurred; rather, they
documented managerial statements that the actions occurred. After we informed
the OCSPP and OPPT AFCs that the information provided was inadequate, the
OCSPP required several weeks to obtain and provide us with adequate
documentation.
The OIG notes that in previous follow-up audits the agency did not solely rely on
a certification memorandum as "proof' of completion of corrective actions. To
address this issue, the OIG included a discussion regarding "supporting
18-P-0222
4

-------
documentation" requirements for follow-up audits in a previously planned
meeting with the agency AFCs on March 8, 2018. In this meeting, which included
agency AFCs and the OIG's Office of Audit and Evaluation senior management,
the OIG clarified that it adheres to the generally accepted government auditing
standards.3 OIG management further explained that the OIG expectation is for
AFCs to maintain audit files that contain direct evidence of the completion of
corrective actions (e.g., actual products that evidence corrective action
completion). This OIG expectation exists regardless of any agency interpretation
of EPA Manual 2750.
Conclusion
The EPA's corrective actions satisfy all outstanding recommendations from the
OIG's 2015 report. As a result, the agency did not include unverified, self-
reported PGCCA results in its FYs 2015 and 2016 reporting of P2 Program
accomplishments. In addition, the agency developed a program model that
communicates how the PGCAA Program contributes to P2 Program goals. The
agency also established internal metrics that track program outputs and provide
future direction for the awards. However, without documented internal controls to
prevent the inclusion of PGCCA results in agency performance metrics, there is
potential for such application in the future.
Recommendation
We recommend that the Assistant Administrator for Chemical Safety and
Pollution Prevention:
1. Establish and document internal controls to prevent the use of Presidential
Green Chemistry Challenge Awards results in Pollution Prevention
Program performance metrics.
Agency Response and OIG Evaluation
The agency concurred with the recommendation and provided an acceptable
planned corrective action. The OIG considers this recommendation resolved with
the corrective action pending.
In addition to a response to our recommendation, the agency provided technical
comments on the draft report. Based on the technical comments received, we made
revisions to the report where appropriate. Appendix A contains the agency's
response to the draft report.
3 These standards require that the auditor obtain sufficient evidence to support findings, conclusions and
recommendations.
18-P-0222
5

-------
Status of Recommendations and
Potential Monetary Benefits
RECOMMENDATIONS
Rec.
No.
Page
No. Subject
Status1 Action Official
Planned
Completion
Date
Potential
Monetary
Benefits
(in $000s)
1
5 Establish and document internal controls to prevent the use of
Presidential Green Chemistry Challenge Awards results in
Pollution Prevention Program performance metrics.
R Assistant Administrator for
Chemical Safety and
Pollution Prevention
12/31/18

1 C = Corrective action completed.
R = Recommendation resolved with corrective action pending.
U = Recommendation unresolved with resolution efforts in progress.
18-P-0222
6

-------
Appendix A
Agency Response to Draft Report
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D C 20480
"JUL' -3 2018
Of-HC r Ol CtlLMk. V 5V H\
a\,i pu' un on i .s vt shiin
MEMORANDUM
SUBJECT: Response to Draft Report entitled "MiPA Completed DIG Recommendations for
the Presidential (keen Chemistry Challenge Awards Program but Lucks Controls
Over Use of Results."
This memorandum is in response to the Office of Inspector General's (OIG's) May 29, 2018,
Draft Report entitled "EPA Completed OICi Recommendations for the Presidential Green
Chemistry Challenge Awards Program hut Lacks Controls Over Use of Results," Project No.
OAti-FYl 8-0003, "
The Office of Chemical Safety and Pollution Prevention (OCSPP) appreciates the OIG's effort in
evaluating the following;
•	Whether internal controls put in place by the I*PA for use during the fiscal year (FY)
2017 Presidential Green Chemistry Challenge Awards (PCiCCA) prevent the use of
unverified, sell-reported results for agency reporting purposes.
•	Whether EPA has sufficient veri fieation of its implementation of the corrective actions
associated with the OIG's September 15, 2015 report entitled. "FPA's Presidential Green
Chemistry Challenge Awards Program Lacks Adequate Support and Transparency and
Should Be Assessed for Continuation," Report No. ' P tC
OCSPP's Response to the Recommendation:
Recommendation 1: Fstablish and document internal controls to prevent the use of Presidential
Green Chemistry Challenge Awards results in Pollution Prevention Program performance
metrics.
•	OCSPP Response: OCSPP values internal controls as a critical management tool, and
agrees that internal controls are important to ensure that externally-reported performance
metrics are calculated properly. In our comments on the Discussion Document, OCSPP
FROM: Charlotte Bert
Acting Principal Depuij Assistant Administrator
TO:
Arthur A. Elkitis Jr.
Inspector General
1
18-P-0222
7

-------
outlined the actions the Office of Pollution Prevention and Toxics (OPPT) had taken to
inform its program staff that Green Chemistry data were not to be included in the
determination of performance results for any P2 Program outcome measures, OCSPP is
pleased that the OIG's analysis continued that OPPT had indeed fully implemented the
recommendations of the OIG's 2015 Report.
The Draft Report recommended that OCSPP establish additional internal controls to
ensure that OPPT program staff, including new hires, are made aware of this policy. In
response to this recommendation in OIG's Draft Report. OCSPP has developed
moditieations to the Data Quality Record (DQR) documents associated with the four P2
Program Strategic/Annual Budget outcome measures directing that Green Chemistry
data are not to be included in the determination of performance results for those
measures until appropriate \erification procedures are implemented. DQRs are the
agency's official repository for till information needed to understand and implement
official performance measures. Existing and new stalT and program managers refer to
the DQRs as they develop targets and associated results (actuals).
OCSPP will work with the Office of the Chief Financial Officer fOCFO) to formally
implement these proposed DQR modifications within six months of the date of this
memorandum. The four P2 Program Outcome Measures may be (bund at the following
locations in the attached DQR Document:
•	Page 46: MEASURE CODE/ 297 - METRIC TONS OF CARBON DIOXIDE
EQUIVALENT (MTC02HQ) REDUCED OR OFFSET THROUGH
POLLUTION PREVENTION
•	Page 66: MEASURE CODE: 264 - POUNDS OF IIAZARDOUS MATERIALS
REDUCED THROUGH POLLU HON PREVENTION
•	Page 74: MEASURE CODE; 2t>3 - BUSINESS, INST11 UT10NAL AND
GOVERNMENT COSTS REDUCED IHROUGH POLLUTION PREVENTION
•	Page 82: ME AS I IRE CODE: 262 - GALLONS OF WATER REDUCED
THROUGH POLLUTION PREVENTION
Proposed Corrective \cfion and Timeline for Completion: B> December 3 L 20IS
OCSPP will work with OCFO to implement the attached proposed amendments to the
Data Quality Records (DQRs) for the four P2 Program Strategic/Annual Budget outcome
measures directing that Green Chemistry data are not to be included in the determination
of performance results for those measures until appropriate verilieation procedures are
implemented.

-------
Appendix B
Distribution
The Administrator
Deputy Administrator
Chief of Staff
Chief of Operations
Special Advisor, Office of the Administrator
Assistant Administrator for Chemical Safety and Pollution Prevention
Agency Follow-Up Official (the CFO)
Agency Follow-Up Coordinator
General Counsel
Associate Administrator for Congressional and Intergovernmental Relations
Associate Administrator for Public Affairs
Principal Deputy Assistant Administrator for Chemical Safety and Pollution Prevention
Deputy Assistant Administrator for Chemical Safety and Pollution Prevention
Director, Office of Continuous Improvement, Office of the Administrator
Director, Office of Pollution Prevention and Toxics, Office of Chemical Safety and
Pollution Prevention
Audit Follow-Up Coordinator, Office of the Administrator
Audit Follow-Up Coordinator, Office of Chemical Safety and Pollution Prevention
Audit Follow-Up Coordinator, Office of Pollution Prevention and Toxics, Office of Chemical
Safety and Pollution Prevention
18-P-0222
9

-------