£ < 30 O % &z. % pro^ o U.S. ENVIRONMENTAL PROTECTION AGENCY OFFICE OF INSPECTOR GENERAL EPA Needs to Improve STAR Grant Oversight Report No. 13-P-0361 August 27, 2013 Scan this mobile code to learn more about the EPA OIG. ------- Report Contributors: Eileen Collins Kimberly Crilly Janet Kasper Madeline Mullen Shannon Schofield LaTanya Scott Abbreviations CFR Code of Federal Regulations EPA U.S. Environmental Protection Agency FY Fiscal year IGMS Integrated Grants Management System NCER National Center for Environmental Research OARM Office of Administration and Resources Management OGD Office of Grants and Debarment OIG Office of Inspector General ORD Office of Research and Development STAR Science to Achieve Results Cover photo: A mass spectrometer in a research laboratory is an example of laboratory equipment that can be used to analyze samples during STAR grant research (EPA OIG photo). Hotline To report fraud, waste, or abuse, contact us through one of the following methods: email: OIG Hotline@epa.gov write: EPA Inspector General Hotline phone: 1-888-546-8740 1200 Pennsylvania Avenue. NW fax: (202) 566-2599 Mailcode 2431T online: http://www.epa.gov/oiq/hotline.htm Washington. DC 20460 ------- ^tDST^ U.S. Environmental Protection Agency 13 P 0361 ¦ \ Office of Inspector General August 27,2013 ^ At a Glance z uj PRO^ Why We Did This Review EPA Needs to Improve STAR Grant Oversight The U.S. Environmental Protection Agency, Office of Inspector General, conducted this audit to answer the questions: 1. Is EPA effectively monitoring Science to Achieve Results grant recipient activities? 2. Does EPA verify that STAR grant recipients take action to ensure that research misconduct is not associated with grant activities? The STAR competitive grant program is the primary vehicle through which the EPA funds research at universities and nonprofit groups. From fiscal years 2010 through 2012, the EPA funded 220 projects totaling $150,043,796 through the STAR grant program. This report addresses the following EPA Goal or Cross-Cutting Strategy: • Advancing science, research and technological innovation. For further information, contact our Office of Congressional and Public Affairs at (202) 566-2391. The full report is at: www.epa.aov/oia/reports/20131 20130827-13-P-0361.pdf What We Found We found that the EPA's project officers did not monitor STAR grant recipients in a manner consistent with the agency's policy and guidance. For example, project officers did not take action when recipients submitted annual reports late, did not follow baseline monitoring guidance, and did not routinely follow up when disclaimers about EPA's endorsement were not included in published articles. By not following policy, project officers increased the risk that issues would not be corrected in a timely manner and that projects might not meet specified goals. During administrative advanced-monitoring reviews, the EPA did not ensure costs were allocable to the grant and did not request certified effort reports. We reviewed drawdowns totaling $639,045 and found $53,854 in costs that were not allowable. We also found that one grant recipient's certified effort reports did not comply with the recipient's own internal policies. Project officers did not actively monitor STAR grant recipients for potential research misconduct, or review recipients for compliance with research misconduct terms and conditions. When the EPA does not monitor research misconduct, the agency puts grant funds at risk. Recommendations and Planned Agency Corrective Actions We recommend that the EPA provide mandatory training to STAR grant project officers. The training should include baseline monitoring performance and instructions to ensure baseline monitoring reports are accurately completed. For incrementally funded grants, the EPA should enforce the terms and conditions that allow withholding of funds for late or missing reports, and amend these requirements for all awards so that payment will be withheld when progress reports are missing or late. We recommend that the EPA require grant recipients to submit corrections to publications when acknowledgement of EPA funding and disclaimers of EPA endorsement are missing from articles. To address unallowable costs, we recommend that the EPA follow up on the unallowable costs that we identified and issue guidance to grant specialists for improving administrative advanced-monitoring reviews. We also recommend that the EPA require project officers to verify grant recipients' awareness of research misconduct reporting requirements. The EPA's completed and planned corrective actions address all of the OIG's recommendations. Noteworthy Achievements The EPA immediately corrected a website containing outdated terms and conditions after we informed the agency about the problem. Grant recipients are now directed to the correct website for grant terms and conditions. ------- UNITED STATES ENVIRONMENTAL PROTECTION AGENCY WASHINGTON, D.C. 20460 THE INSPECTOR GENERAL August 27, 2013 MEMORANDUM SUBJECT: EPA Needs to Improve STAR Grant Oversight Report No. 13-P-0361 FROM: Arthur A. Elkins Jr. TO: Lek Kadeli, Principal Deputy Assistant Administrator Office of Research and Development Craig Hooks, Assistant Administrator Office of Administration and Resources Management This is our report on the subject audit conducted by the Office of Inspector General of the U.S. Environmental Protection Agency. This report contains findings that describe the problems the OIG has identified and corrective actions the OIG recommends. Action Required In responding to the draft report, the agency provided a corrective action plan for addressing the recommendations with milestone dates. Therefore, a response to the final report is not required. The agency should track corrective actions not implemented in the Management Audit Tracking System. This report will be available at http://www.epa.gov/oig. If you or your staff have any questions regarding this report, please contact Richard Eyermann, acting assistant inspector general for the Office of Audit, at (202) 566-0565 or eyermann.richard@epa. gov; or Janet Kasper, product line director, at (312) 886-3059 or kasper.ianet@epa.gov. ------- EPA Needs to Improve STAR Grant Oversight 13-P-0361 Table of Contents Chapters 1 Introduction 1 Purpose 1 Background 1 Noteworthy Achievements 2 Scope and Methodology 2 2 Project Officers Did Not Follow EPA Guidance 4 STAR Grants Must Comply With Federal and EPA Requirements 4 Annual Reports Were Frequently Late 5 Project Officers Did Not Follow Baseline Monitoring Guidance 6 Published Articles Were Missing Required Disclaimers 7 Recommendations 8 Agency Comments and OIG Evaluation 8 3 Drawdown Reviews Need Improvement 10 Federal Requirements Define Allowable Costs 10 EPA Reimbursed Unallowable Costs 10 Administrative Advanced-Monitoring Review Needs Improvement 12 Increased Risk of Reimbursing Unallowable Costs 12 Recommendations 13 Agency Comments and OIG Evaluation 13 4 Research Misconduct Terms and Conditions Were Not Monitored 14 EPA's Terms and Conditions Define Research Misconduct and Reporting Requirements 14 Project Officers Did Not Monitor for Potential Research Misconduct 15 Project Officers Did Not Ask About Research Misconduct 17 Risk of Research Misconduct Not Addressed 17 Recommendation 17 Agency Comments and OIG Evaluation 18 Status of Recommendations and Potential Monetary Benefits 19 Appendices A STAR Grants Reviewed by the EPA OIG 20 B Agency's Response to Draft Report 21 C Distribution 26 ------- Chapter 1 Introduction Purpose The U.S. Environmental Protection Agency, Office of Inspector General, conducted an audit of the Science to Achieve Results grant program to answer the questions: 1. Is EPA effectively monitoring Science to Achieve Results grant recipient activities? 2. Does EPA verify that STAR grant recipients take action to ensure that research misconduct is not associated with grant activities? Background The STAR program began in 1995 and was assigned to the EPA's National Center for Environmental Research and Quality Assurance—currently known as the National Center for Environmental Research. The program's research focus was developed specifically to meet the research needs of the EPA. The program funds targeted research grants in numerous environmental science and engineering disciplines. The STAR competitive grant program is the primary vehicle through which the EPA funds research at universities and nonprofit groups. From fiscal years 2010 through 2012, the EPA funded 220 projects totaling $150,043,796 through the STAR grant program. Research efforts funded by these grants are aligned with the EPA's strategic goals, including the cross-cutting strategic goal of advancing science and research. The public can access progress reports about these research projects by visiting the NCER website at http://cfpub.epa.gov/ncer abstracts/index.cfm/fuseaction/search.welcome. Although the NCER is responsible for administering the programmatic aspects of STAR grants, the Office of Grants and Debarment is responsible for the financial management of the STAR program. The NCER and OGD each perform baseline and advanced-monitoring reviews of grant recipients. The EPA includes terms and conditions in the grants that it awards. The terms and conditions cover topics such as financial and program management of the grants, timing and content of annual and final reports, and reporting requirements for research misconduct. According to the Code of Federal Regulations through 2 CFR §215.61, grant recipients failing to comply with the terms and conditions of an award can have their awards terminated. 13-P-0361 1 ------- Noteworthy Achievements The EPA ensured that an outdated website was corrected so that the terms and conditions included in the awards would no longer direct grant recipients to outdated agency-specific information. We found that the EPA's STAR grant agreements did not always include the correct website that contained the applicable agency-specific terms and conditions. When we informed EPA staff about this, they took immediate action to correct the problem. Scope and Methodology We conducted this performance audit from July 2012 through June 2013, in accordance with generally accepted Government Auditing Standards (Yellow Book) issued by the Comptroller General of the United States. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objective. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objective. The 12 STAR grants that we reviewed had a total funding amount of $10,385,348. The audit team initially made a judgmental selection of two grants with a total obligation amount of $4,680,608. After reviewing these two grants to gain an understanding of the STAR grant process and documents, the team randomly sampled 10 additional grants. We identified 84 STAR grants with project start dates after October 1, 2008, and project end dates prior to September 30, 2013. The total funding amount for these grants was $49,798,525. The team generated a random sample of 10 grants with a total obligation amount of $5,704,740. To determine whether there was evidence of monitoring and whether the monitoring was conducted in accordance with applicable policies and procedures, we obtained annual progress reports from the selected grant recipients' project officers. We reviewed the annual progress reports to determine if the reports were submitted on time and whether they indicated progress commensurate with the outputs and outcomes identified in the work plan. We also obtained programmatic and administrative reports from the EPA's Integrated Grants Management System. The IGMS reports were completed by project officers and grant specialists. To determine whether grant recipients were complying with the terms and conditions of their grants, we obtained the following documents from selected grant recipients and from websites where scientific articles are posted. • Supporting financial documentation for three selected drawdowns for each of the five judgmentally selected grant recipients to determine if claimed costs were allowable. We selected the five grants based on 13-P-0361 2 ------- information noted on the grant application indicating the recipient had received funding for similar grants from EPA or other federal agencies. • Six published articles to determine whether the articles contained an acknowledgment of EPA funding and a disclaimer stating the EPA does not endorse the article. At the time of the audit, only six of the grant recipients had published articles. • Scientific integrity policies of nine grant recipients to determine whether their policies were consistent with EPA requirements for reporting research misconduct, as outlined in the agency's terms and conditions. We reviewed the scientific integrity policies for the grant recipients that we were also reviewing for financial documentation and published articles. We limited our review to nine grant recipients to comply with requirements of the Paperwork Reduction Act. We interviewed the OGD and Office of Research and Development staff (including project officers), and discussed their procedures for monitoring grant recipients and ensuring scientific integrity. We also interviewed the grant recipients to discuss applicable terms and conditions, reporting, and interactions with the EPA. The grants reviewed are listed in appendix A. There were no prior audits impacting the objectives of this assignment. 13-P-0361 3 ------- Chapter 2 Project Officers Did Not Follow EPA Guidance Project officers did not monitor STAR grant progress in a manner consistent with the EPA's policy and guidance. Project officers also allowed grant recipients to submit annual reports late. STAR grant recipients must comply with numerous federal and EPA requirements. The project officers emphasized overall progress rather than compliance with specific requirements. This introduced risks that could lead to the project officers not detecting issues needing corrective action and that might impact the project meeting its goals. In addition, missing acknowledgements and disclaimers in published research put the EPA's transparency at risk and could give an appearance of the agency endorsing the research. STAR Grants Must Comply With Federal and EPA Requirements The EPA's Assistance Administration Manual (EPA Order 5700) requires agency project officers to conduct baseline monitoring of all assistance agreements. Monitoring is supposed to include a review of: • Compliance with programmatic terms and conditions. • Drawdowns and budget expenditures. • Recipient progress, including a comparison of funds available to project progress or time remaining on the project. The EPA's I)olicy on Compliance, Review, and Monitoring (EPA Order 5700.6) requires project officers to conduct baseline monitoring at least annually, and that baseline monitoring should include a review of the grant recipient's annual report. Terms and conditions for EPA grants require recipients to submit annual reports on grant progress. Depending on when the grant was awarded, the due date for the annual report varies. Table 1: Annual report due date Terms and conditions issuance date1 Annual report due date July 2008 30 days prior to the end of the reporting period June 2009 30 days prior to the end of the reporting period November 2010 90 days after the end of the reporting period Source: EPA grant terms and conditions. 1 Terms and conditions apply to all grants awarded after the date the terms and conditions were issued. 13-P-0361 4 ------- If the recipient does not submit the annual report by the due date, the terms and conditions for partially funded grants allow the EPA not to provide further funding until the recipient has submitted the required annual progress report. The EPA encourages independent publication of results from its extramural research in appropriate scientific journals. When doing so, the terms and conditions require that there be an acknowledgement of EPA funding. The exact wording of the acknowledgement depends on when the grant was awarded. The FY 2008 terms and conditions required grant recipients to include the following statement in all journal articles publishing work funded by an EPA grant: Although the research described in this article has been funded wholly or in part by the United States Environmental Protection Agency through grant/cooperative agreement (number) to (name), it has not been subjected to the Agency's required peer and policy review and therefore does not necessarily reflect the views of the Agency and no official endorsement should be inferred. The FY 2009 and FY 2010 terms and conditions revised the statement as follows: This publication was made possible by USEPA grant (number). Its contents are solely the responsibility of the grantee and do not necessarily represent the official views of the USEPA. Further, USEPA does not endorse the purchase of any commercial products or services mentioned in the publication. Annual Reports Were Frequently Late Of the 12 grants that we reviewed, 10 grant recipients submitted late annual reports for 2 years or more during the life of the projects (see table 2): Table 2: Annual progress reports that were late or not submitted Grant number 2007 2008 2009 2010 2011 2012 83337401 273 109 Not submitted 231 Not submitted 83409301 Not submitted 140 289 Not submitted 83389201 98 123 225 83435901 277 237 141 83428501 122 139 135 83428201 123 139 Not submitted 83438601 150 211 83459601 322 161 83399002 454 434 168 83337302 144 1 Source: OIG analysis of recipient annual reports for days late. (Shaded cells indicate reports were not required that year.) 13-P-0361 5 ------- The EPA's project officers did not enforce the specific annual reporting time frames listed in the terms and conditions. In many cases, the project officer did not identify the correct annual report due dates in the terms and conditions, because award documents directed the reader to an outdated website that contained incorrect information. Project officers also did not conduct monitoring in compliance with EPA Order 5700.6, and one project officer chose to allow a grant recipient to submit late annual reports despite the due dates that were required by the terms and conditions. The often substantial tardiness of annual reports introduced an unnecessary risk that could lead project officers not to detect issues needing corrective action and might impact the project not meeting its goals. For grants that were awarded in full, there were no consequences for late or missing reports. For grants that received incremental funding, the terms and conditions gave the EPA the option of not providing further funding until grant recipients submitted annual progress reports. Four of the grants we reviewed received incremental funding, and three of those four grant recipients consistently submitted annual reports late. In fact, further funding had already been provided for two of the recipients before the first progress report was due, and one recipient was fully funded after the first progress report was submitted late. As a result, the EPA could not exercise this option. Project Officers Did Not Follow Baseline Monitoring Guidance Of the eight EPA project officers that we interviewed, four did not conduct monitoring in compliance with EPA Order 5700.6 requirements. Specifically: • Two project officers did not complete programmatic baseline monitoring reports for 1 year or more. • One project officer conducted baseline monitoring without reviewing the recipient's annual progress report. • One proj ect officer did not review the proj ect's progress against the timeline in the work plan, because the timeline was missing. The project officers are supposed to conduct baseline monitoring annually and document the monitoring in the IGMS. Baseline monitoring includes a review of the grant recipients' annual progress reports. The 2007 baseline monitoring checklist guidance did not require project officers to note whether annual reports were late. This introduced a risk that the EPA could not detect issues needing corrective action and that projects might not meet their goals. The EPA updated that checklist in 2012 to include a specific question concerning whether progress reports were submitted as required. When project officers do not follow baseline monitoring guidance, the EPA's oversight of grants is hindered. The terms and conditions require grant recipients to report expenditures to date, along with providing a comparison of the percentage of the project completed 13-P-0361 6 ------- according to the project schedule. However, one project officer did not think that a timeline was necessary to review the project progress. Because the grant recipient's work plan did not include a timeline, the project officer was unable to monitor sufficient progress within the agreed-upon project period. Subsequently, he approved a no-cost time extension to perform the work and/or to use the unexpended funds for additional work. When project officers allow STAR grant recipients to rely upon no-cost time extensions to complete work that the recipients had agreed to finish in a shorter time period, the integrity of the STAR grant award process is called into question. Published Articles Were Missing Required Disclaimers Of the six published articles reviewed, three did not contain the disclaimer required by grant terms and conditions noting the EPA does not endorse the article (see table 3). Table 3: Published articles reviewed Grant number Article title Disclaimer was not included 83409301 UV Irradiation and Humic Acid Mediate Aggregation of Aqueous Fullerene (nC60) Nanoparticles X 83428501 Economically Consistent Long-Term Scenarios for Air Pollutant Emissions X 83459601 Exposure to Traffic-related Air Pollution During Pregnancy and Term Low Birth Weight: Estimation of Causal Associations in a Semiparametric Model X Source: EPA OIG analysis of select published articles. Project officers learned about the inclusion of the required acknowledgment and disclaimer in published articles only after the actual publication; therefore, any corrective action by the agency was after the fact. Some project officers said they checked and found missing information, but only one acknowledged following up with the grant recipient. Follow-up was not consistent. During the audit, NCER management said actions could be taken to improve compliance with this requirement. Without the required disclaimer, people may presume that the EPA approves of the information presented in the articles. This is especially true when EPA funding is acknowledged. Public confidence in the EPA's mission could be diminished if the research is contradicted in the future. 13-P-0361 7 ------- Recommendations We recommend that the Assistant Administrator for Research and Development: 1. Require all project officers to attend training on how to perform baseline monitoring. Include a discussion concerning actions to be taken when publications do not include required acknowledgements and disclaimers. 2. Conduct a review of baseline monitoring reports to ensure they are accurately completed. 3. Enforce the terms and conditions that allow funds to be withheld from incrementally funded grants, if reports are missing or late. 4. Amend terms and conditions so that all awards are subject to a condition that payments will be withheld when reports are missing or late. 5. Require the grant recipient to submit corrections to publications when project officers identify missing acknowledgements and disclaimers. Require immediate correction for online articles and at the next publication date for printed articles. 6. Require grant recipients to submit corrections to publications for the articles that the EPA OIG identified as missing acknowledgments and disclaimers. Agency Comments and OIG Evaluation The EPA agreed with the recommendations and stated that the ORD will provide all STAR grant project officers with training on the performance of baseline monitoring. The ORD has drafted standard operating procedures for steps to be taken when annual progress reports are not received in a timely manner, including when to pursue withholding funds from incrementally funded grants due to failure to comply. The ORD is revising its guidance pertaining to annual report reviews, to include actions to be taken when publications do not have the required acknowledgements and disclaimers. The ORD is adding a requirement for grant recipients to ensure that publications have EPA acknowledgements and disclaimers. The ORD is also revising its terms and conditions to include a condition that payments may be withheld when reports are missing or late. This will be applied to all new STAR grants. If acknowledgements and disclaimers are identified as missing, principal investigators will be instructed to submit corrections. When implemented, the agency's actions should address recommendations 1, 3, 4 and 5. 13-P-0361 8 ------- The EPA reported that corrective actions for two of the recommendations have already been completed. The NCER completed a review of project officer file records and modified its multiyear review plan to include an annual review of STAR grants, including baseline monitoring reports. Also, the ORD received copies of three correction requests that journal editors made to add the required disclaimers. The EPA's corrective actions address recommendations 2 and 6. 13-P-0361 9 ------- Chapter 3 Drawdown Reviews Need Improvement During administrative advanced-monitoring reviews, the EPA does not ensure that costs are allocable to the grant and does not request certified effort reports. We reviewed drawdowns totaling $639,045 and found $53,854 in costs that were not allowable because the costs were either unallocable or unsupported. We also found that one grant recipient's certified effort reports did not comply with the recipient's own internal policies. EPA Order 5700.6 states that the purpose of the advanced monitoring is to conduct in-depth assessments, including administrative and financial progress. Federal regulations state costs are allowable only if they are allocable to the grant agreement. The lack of detailed cost reviews increases the risk that more unallowable costs will be reimbursed in the future. Federal Requirements Define Allowable Costs An allowable cost is defined in 2 CFR Part 220, Appendix A. A cost is allowable if it is reasonable, allocable to the sponsored agreements, and given consistent treatment. A cost is allocable to a particular cost objective if it is incurred solely to advance the work and benefits both the sponsored agreement and other work of the institution. According to 2 CFR Part 220, Appendix A, grant recipients are also required to confirm the activity of personnel who charge time to the grant through what is referred to as "certified effort reports." These signed reports verify the work was performed by employees and can be signed by either the employee or a responsible official. The effort reports verify that salaries or wages were correctly charged against the grant. EPA Order 5700.6 requires the EPA to evaluate the allowability of costs as a part of advanced monitoring. The order states that the purpose of administrative advanced monitoring is to conduct in-depth assessments of a recipient's administrative and financial progress. Additional EPA guidance for administrative advanced monitoring requires the testing of transactions to determine whether claimed costs are allowable, allocable, and reasonable under the assistance agreement. EPA Reimbursed Unallowable Costs The EPA incorrectly reimbursed a total of $53,854 (see table 4). Of the $6,906,269 drawn down by the five grant recipients, we reviewed $639,045. 13-P-0361 10 ------- The costs were not allowable because they were either not allocable to the grant or were not supported. While we only looked at a sample of the drawdowns, the sample may be an indicator that other unallowable costs exist. Table 4: Summary of unallowable costs Type of cost Unallowable costs Note Personnel $44,886 1 Travel 426 2 Other 292 3 Indirect costs 8,250 4 Total $53,854 Source: OIG analysis of recipient drawdown records. Note 1-Personnel Costs One grant recipient claimed $44,886 of personnel costs that were not allowable. The grant recipient did not provide certified effort reports as required by 2 CFR Part 220, Appendix A to confirm that distribution of labor is accurate. Recipient effort reports did not always meet federal regulations or the grant recipient's own internal policies. For example, one grant recipient provided incorrect effort reports: • Three effort reports were certified by individuals who were not authorized to certify effort reports, according to the grant recipient's own internal policy. • Nine effort reports did not identify the certifier by name, as required by 2 CFR Part 220, Appendix A. Note 2-Travel Costs Travel costs totaling $426 were not allowable: • Travel expenses totaling $224 were not allocable. o The travel expense report showed a $200 expense for a personal side trip to a different location. The costs claimed for the return flight were not reduced to account for the expense associated with the personal side trip, o Internal audit fees related to two travel expense reports and totaling $24 were included in the draw. However, the purpose of the travel associated with these expense reports was not entirely related to the EPA grant. 13-P-0361 11 ------- • Travel expenses totaling $202 were not supported. o A $198 hotel expense was not supported with a hotel receipt, o A $4 local travel expense was not supported because $18 was claimed, but only $14 was supported. Note 3-Other Costs The grant recipient claimed $292 in other costs that were not allowable: • Computer services costs totaling $211 were not supported. • Communication costs totaling $81 were charged to the STAR grant when the cost should have been shared with another non-EPA funded project. Note 4-Indirect Costs Because personnel, travel and other costs identified above were not allowable, associated indirect costs totaling $8,250 were also not allowable. Administrative Advanced-Monitoring Review Needs Improvement One of the grants in our sample had an administrative advanced-monitoring desk review, where we found: • The grant specialist traced payroll costs to the payroll register, but not to the certified effort reports. According to OGD staff, the grant specialist does not request effort reports during an administrative advanced- monitoring review. • The grant specialist traced travel costs to the supporting documents, but did not determine whether the travel was applicable to the grant. According to OGD staff, the grant specialist would not know whether the travel was applicable to the grant, because the specialists do not have the same technical knowledge about the project as the project officers. The EPA's grants staff do not collaborate with program staff on cost reviews. The agency's policy requires grant specialists to review drawdowns as a part of the administrative advanced monitoring. However, according to EPA staff, drawdown reviews by grant specialists did not include verification or consultation with the project officer about cost allocability. increased Risk of Reimbursing Unallowable Costs Federal funds drawn for unallowable costs could have been put to better use. When the OGD and project officers do not collaborate on reviewing costs during administrative advanced monitoring, there is a risk that unallowable costs will be 13-P-0361 12 ------- reimbursed. Also, not reviewing effort reports increases the risk that unallowable personnel costs will be reimbursed. Recommendations We recommend that the Assistant Administrator for Administration and Resources Management: 7. Follow up on unallowable costs identified in the finding. If grant recipients cannot provide documentation, require repayment of the funds. 8. Issue guidance to grant specialists, and remind them that during transaction testing they are required to: a. Trace costs to source documents, including a review of certified effort reports. b. Make a determination as to whether costs are related to the activities funded by the grant. Where grant specialists cannot determine allocability, they should work with the project officer to verify costs associated with the grant. Agency Comments and OIG Evaluation The EPA agreed with the recommendations and stated that the Office of Administration and Resources Management will follow up on unallowable costs and require repayment as appropriate. When implemented, the agency's actions should address recommendation 7. Also, the OARM reminded grant specialists to follow advanced-monitoring guidance provisions to trace grant costs to source documentation and determine whether costs are related to grant-funded activities. If there are allocability questions, grant specialists will work with the project officer to verify costs. The agency's action addresses recommendation 8. 13-P-0361 13 ------- Chapter 4 Research Misconduct Terms and Conditions Were Not Monitored Project officers did not actively monitor STAR grants for potential research misconduct or assess compliance with research misconduct terms and conditions. The grant recipients' scientific integrity policies do not always reflect an understanding of this requirement and can conflict with agency policy. EPA Order 5700.6 requires project officers to conduct baseline monitoring of all assistance agreements, including a review of compliance with programmatic terms and conditions. The EPA's terms and conditions require grant recipients to notify the agency when the recipients discover research misconduct. Project officers assumed grant recipients would report research misconduct. The project officers also assumed the grant pre-award peer-review process and the published article peer-review process would lessen the likelihood of research misconduct. However, grant recipients may not understand the requirements for identifying and reporting research misconduct. The pre-award peer-review process only searches for one type of research misconduct—a conflict of interest; while during the publication peer-review process, the NCER does not receive any information. In both instances, the lack of monitoring increases the risk that federal funds will be misused. EPA's Terms and Conditions Define Research Misconduct and Reporting Requirements The EPA's terms and conditions for research misconduct state that in accordance with 40 CFR 30.51(f), the recipient agrees to notify the EPA project officer in writing about research misconduct involving research activities that are supported in whole or in part with agency funds. The EPA defines research misconduct as fabrication, falsification, or plagiarism in proposing, performing, or reviewing research or in reporting research results; or ordering, advising or suggesting that subordinates engage in research misconduct. Terms and conditions specified in the "EPA Research and Related Agency Specific Requirements" call for STAR grant recipients to: (1) immediately notify the EPA Project Officer who will then inform the EPA Office of Inspector General (OIG) if, at any time, an allegation of research misconduct falls into one of the categories listed below: A. Public health or safety is at risk. B. Agency resources or interests are threatened. 13-P-0361 14 ------- C. Circumstances where research activities should be suspended. D. There is a reasonable indication of possible violations of civil or criminal law. E. Federal action is required to protect the interests of those involved in the investigation. F. The research entity believes that the inquiry or investigation may be made public prematurely so that appropriate steps can be taken to safeguard evidence and protect the rights of those involved. G. Circumstances where the research community or public should be informed. (2) report other allegations to the OIG when they have conducted an inquiry and determined that there is sufficient evidence to proceed with an investigation. EPA Order 5700.6 requires project officers to conduct baseline monitoring of all assistance agreements, including a review of compliance with programmatic terms and conditions. The EPA's programmatic baseline monitoring template includes the following question: Is the recipient complying with all applicable programmatic terms and conditions? The 2012 baseline monitoring template provides additional guidance and states that the project officer has to confirm that all terms and conditions are being met to the maximum extent practicable. Project Officers Did Not Monitor for Potential Research Misconduct Project officers are not monitoring grant recipient compliance with research misconduct terms and conditions. Project officers said it is the grant recipient's responsibility to report research misconduct, and the officers assume the recipients will do so if misconduct occurs. The project officers also did not review grant recipients' scientific integrity policies to determine whether the recipients understood the EPA's research misconduct definitions or the agency's notification requirements. We reviewed scientific integrity policies for nine grant recipients (see table 5) and found that the policies did not include: • A complete definition of research misconduct. • A statement that the sponsoring agency would be notified immediately upon discovery of research misconduct. 13-P-0361 15 ------- Table 5: Scientific integrity policies not consistent with terms and conditions Grant number Missing notification requirement Incomplete definition of research misconduct 83337401 X X 83329201 X X 83409301 X X 83438601 X X 83428501 X X 83428201 X X 83459601 X X 83399002 X X 83337302 X Source: OIG analysis of grant recipients' scientific integrity policies. Notification Requirement The EPA's terms and conditions require grant recipients to immediately notify the project officer if, at any time, an allegation of research misconduct falls into one of seven specified categories. However we found that eight grant recipients' policies contained reporting language that was inconsistent with the terms and conditions. For example: • One policy stated that the dean shall take whatever action he or she considers appropriate and would provide a copy of the proceedings to the EPA when required. There is a risk that all applicable allegations of research misconduct would not be reported unless the dean took action on the allegations first. • Another policy said the grant recipient would only notify the federal sponsor within 24 hours if there is a reasonable indication of a possible federal criminal and civil violation. It did not state that the sponsor would be immediately notified for other conditions and did not include circumstances where research activities should be suspended. There is a risk that all applicable allegations of research misconduct would not be reported. Definition of Research Misconduct All of the nine grant recipients' policies defined research misconduct as "fabrication, falsification or plagiarism, in proposing, performing, or reviewing research, or in reporting research results," as included in the EPA's terms and conditions. However, none of the nine grant recipients included "ordering, advising or suggesting that subordinates engage in research misconduct" in their definition—terms that are found in the EPA's definition of research misconduct. 13-P-0361 16 ------- Project Officers Did Not Ask About Research Misconduct The EPA's guidance does not include specific procedures for project officers to follow when monitoring grant recipients for potential research misconduct. Project officers relied on the grant recipients' experience and expertise and assumed recipients would report research misconduct when it occurred. Project officers made the following assumptions: • Grant recipients could be relied on to exercise scientific integrity, and that it would not be necessary to review the policies to ensure that grant recipients understood the research misconduct reporting requirements in the terms and conditions. • The prestige of the universities and the professional reputation of the principal investigators lessened the likelihood of research misconduct. • The pre-award process and the peer-review process for articles published in reputable journals lessened the likelihood of research misconduct. Risk of Research Misconduct Not Addressed Because grant recipients' policies do not specifically address the reporting of research misconduct allegations to the EPA, there is a higher risk that the agency may not be notified when grant recipients become aware of actual or alleged research misconduct. An article appearing m Nature, an international journal of science, described a survey of scientists early in their career and in mid-career, and noted that 33 percent of the scientists admitted to engaging in at least one research misconduct activity during the previous 3 years. If the EPA is not notified when research misconduct is discovered, this could prevent the EPA from taking corrective action or determining whether grant funds were misused. The STAR program funds millions of dollars in research each year, and research misconduct puts those federal dollars at risk. Recommendation We recommend that the Assistant Administrator for Research and Development: 9. Require that during annual baseline monitoring, project officers verify that grant recipients fully understand research misconduct reporting requirements. 13-P-0361 17 ------- Agency Comments and OIG Evaluation The EPA agreed with the recommendation and stated that the NCER will modify its terms and conditions for annual progress reports to include an assertion of compliance with the research misconduct terms and conditions. When implemented, the agency's action should address recommendation 9. 13-P-0361 18 ------- Status of Recommendations and Potential Monetary Benefits RECOMMENDATIONS POTENTIAL MONETARY BENEFITS (In $000s) Rec. No. No. Subject Status1 Action Official Require all project officers to attend training on how to perform baseline monitoring. Include a discussion concerning actions to be taken when publications do not include required acknowledgements and disclaimers. Planned Completion Date Claimed Amount Agreed-To Amount Assistant Administrator for 12/31 /13 Research and Development 8 Conduct a review of baseline monitoring reports to ensure they are accurately completed. 8 Enforce the terms and conditions that allow funds to be withheld from incrementally funded grants, if reports are missing or late. 8 Amend terms and conditions so that all awards are subject to a condition that payments will be withheld when reports are missing or late. 8 Require the grant recipient to submit corrections to publications when project officers identify missing acknowledgements and disclaimers. Require immediate correction for online articles and at the next publication date for print articles. 8 Require grant recipients to submit corrections to publications for the articles that the EPA OIG identified as missing acknowledgements and disclaimers 13 Follow up on unallowable costs identified in the finding. If grant recipients cannot provide documentation, require repayment of the funds. 13 Issue guidance to grant specialists, and remind them that during transaction testing they are required to: a. Trace costs to source documents, including a review of certified effort reports. b. Make a determination as to whether costs are related to the activities funded by the grant. Where grant specialists cannot determine allocability, they should work with the project officer to verify costs associated with the grant. 17 Requirethat during annual baseline monitoring, project officers verify that grant recipients fully understand research misconduct reporting requirements, Assistant Administrator for 7/19/13 Research and Development Assistant Administrator for 9/30/13 Research and Development Assistant Administrator for 9/30/13 Research and Development Assistant Administrator for 9/30/13 Research and Development Assistant Administrator for 7/19/13 Research and Development Assistant Administrator for 12/31 /13 Administration and Resources Management Assistant Administrator for 08/12/13 Administration and Resources Management $54 $54 Assistant Administrator for 12/31 /13 Research and Development 1 0 = Recommendation is open with agreed-to corrective actions pending. C = Recommendation is closed with all agreed-to actions completed. U = Recommendation is unresolved with resolution efforts in progress. 13-P-0361 19 ------- Appendix A STAR Grants Reviewed by the EPA OIG Grant number Project title Grant period EPA funding Grant randomly selected Principal investigators interviewed Scientific integrity policy reviewed Published articles reviewed Drawdown support reviewed 83337401 Changes in Climate, Pollutant Emissions, and United States Air Quality 03/01/07- 02/28/12 $896,597 X X X X 83329201 Center for Children's Environmental Health 11/01/06 - 10/31/13 $3,784,011 X X X X 83389201 Platinum-Containing Nanomaterials: Sources, Speciation, and Transformation in the Environment 02/01/09- 01/31/13 $399,406 X 83409301 Natural Organic Matter/C60 Fullerene Interactions 12/01/08 — 11/30/12 $399,996 X X X X 83428501 Analysis of Co-benefits of GHG Abatement 09/01/09- 08/31/13 $300,000 X X X X 83399002 Novel Markers of Air Pollution Vascular Toxicity 11/01/10 — 10/31/12 $432,664 X X X X 83459601 Children's Environmental Health Center 05/07/10- 05/06/13 $1,091,783 X X X X X 83428201 Effects of Changing Climate on Fires & Air Quality 11/01/09- 10/31/13 $599,366 X X X X 83438601 C02 Sequestration Modeling to Protect Drinking Water Sources 12/01/09 — 11/30/13 $899,567 X X X X 83337302 Impacts of Global Climate and Emissions Changes on U.S. Air Quality Zones 01/04/11 - 04/14/13 $259,778 X X X X 83482401 Comparison of Mice for Allergenicity Testing 09/15/10 — 09/14/13 $423,546 X 83435901 Predicting Regional Allergy Hotspots - Future Climates 09/01/09- 08/31/13 $898,634 X Total $10,385,348 10 9 9 7 5 Source: Project officers interviewed, file documents, and annual reports reviewed. 13-P-0361 20 ------- Appendix B Agency's Response to Draft Report July 19, 2013 MEMORANDUM SUBJECT: Response to the Office of Inspector General Draft Report titled, The EPA Needs to Improve STAR Grant Oversight Project No. OA-FY12-0606, Dated June 18, 2013 FROM: Craig E. Hooks, Assistant Administrator Office of Administration and Resources Management (OARM) Lek G. Kadeli, Principal Deputy Assistant Administrator Office of Research and Development (ORD) TO: Arthur A. Elkins, Jr., Inspector General Office of Inspector General (OIG) Thank you for the opportunity to respond to the OIG draft report titled, The EPA Needs to Improve STAR Grant Oversight. EPA is committed to keeping the Science to Achieve Results (STAR) grant program as a highly competitive, rigorously peer-reviewed, well-managed extramural research grants program. To further the cross-cutting Agency goal of enhancing science and research while upholding the integrity of the way in which the Agency utilizes its resources, the EPA's ORD and the OARM will renew our commitment to collaborate closely to that end. The STAR grant program helps maximize the Agency's resources by complementing and supplementing the research of our in-house laboratories that otherwise would not benefit from the increased interagency collaboration and greater interactions with academic and non- profit institutions on the forefront of innovative scientific research. We appreciate the OIG's work to assess EPA's monitoring of STAR grants and proactive efforts to ensure that research misconduct is not associated with grant activities. Most notably, your team's efforts to communicate with the Agency on a regular basis about potential findings and recommendations have allowed us to take some immediate corrective actions during the course of the review. In addition, these communications have provided us an opportunity to clarify issues relating to certain findings and recommendations. In short, and as discussed during our communications with your team, EPA generally agrees with the OIG's recommendations regarding STAR grants and has provided the corrective actions shown below. 13-P-0361 21 ------- AGENCY'S RESPONSE TO REPORT RECOMMENDATIONS No. Recommendation Responsible Office Corrective Action (s) Estimated Completion Date 1 Require all project officers to attend training on how to perform baseline monitoring. Include a discussion concerning actions to be taken when publications do not include required acknowledgements and disclaimers. ORD ORD will provide training on performance of baseline monitoring to all STAR grant project officers. In addition, we are revising the annual report review guidance to include information on actions to be taken when publications do not include required acknowledgements and disclaimers. 1st Quarter FY 2014 2 Conduct a review of baseline monitoring reports to ensure they are accurately completed. ORD In January 2013, NCER completed a review of official file records of STAR grants project officers, which included a review of baseline monitoring reports. As part of ORD's FY 2013 Annual Assurance Letter, NCER modified its 2013-2017 Multiyear Review Plan to include an annual review of STAR grants, including a review of baseline monitoring reports. Documentation that completes this corrective action has been provided to ORD's Audit Liaison. Completed 3 Enforce the terms and conditions that allow funds to be withheld from incrementally funded grants, if reports are missing or late. ORD ORD has drafted Standard Operating Procedures (SOP), which will be finalized with assistance from OARM, to address steps to be taken when annual progress reports for STAR grants are not received in a timely manner. The SOP includes information on when to pursue withholding funds from incrementally funded grants due to failure to comply with this term and condition. 4th Quarter FY 2013 13-P-0361 22 ------- No. Recommendation Responsible Office Corrective Action (s) Estimated Completion Date 4 Amend terms and conditions so that all awards are subject to a condition that payments will be withheld when reports are missing or late. ORD ORD is revising the terms and conditions for its STAR grant program to including a condition that payments may be withheld when reports are missing or late. The revised terms and conditions will apply to all new STAR grants. 4th Quarter FY 2013 5 Require the grant recipient to submit corrections to publications when project officers identify missing acknowledgements and disclaimers. Require immediate correction for online articles and at the next publication date for printed articles. ORD ORD is revising its SOP for STAR grants annual progress report review to include a requirement that grant recipients ensure their publications have appropriate EPA acknowledgements and disclaimers. If acknowledgements and disclaimers are identified as missing, the principal investigator will be instructed to submit corrections to the publications requesting corrections. 4th Quarter FY 2013 6 Require grant recipients to submit corrections to publications for the articles that the EPA OIG identified as missing acknowledgments and disclaimers. ORD ORD has received copies of three correction requests made of the journal editors and determined the fourth publication was submitted in error. That publication has been removed from the grants publication list. Documentation that completes this corrective action has been provided to ORD's Audit Liaison. Completed 7 Follow up on unallowable costs identified in the finding. If grant recipients cannot provide documentation, require repayment of the funds. OARM OARM will follow up on unallowable costs and require repayment of costs as appropriate. 1st Quarter of FY 2014. 13-P-0361 23 ------- No. Recommendation Responsible Office Corrective Action (s) Estimated Completion Date 8 Issue guidance to grant specialists, and remind them that during transaction testing they are required to: a. Trace costs to source documents, including a review of certified effort reports. b. Make a determination as to whether costs are related to the activities funded by the grant. Where grant specialists cannot determine allocability, they should work with the project officer to verify costs associated with the grant. OARM a. OARM has guidance that relates to this finding which can be found on the OARM/OGD G share at G:\GIAMD\Advanced Monitoring Guidance. For example, the Standard Operating Procedures for Grants Management Office Desk and Onsite Reviews, Section III, Attachment 1 C (e), covers the review of supporting documentation related to payroll and other costs. OARM will remind grant specialists to follow the provisions in the guidance on tracing grant costs to source documentation. b. OARM guidance related to this issue can be found in the Standard Operating Procedures for Grants Management Office Desk and Onsite Reviews, Section III, Attachment 6. OARM will remind grant specialists that during transaction testing they are required to determine whether costs are related to grant funded activities and if there are allocability questions, to work with the project officer to verify costs. 4th Quarter FY 2013 9 Require that during annual baseline monitoring, project officers verify that grant recipients fully understand research misconduct reporting requirements. ORD NCER will modify the term and condition for annual progress reports to include an assertion of compliance with the research misconduct term and condition. 4th Quarter FY 2013 Should you or your staff have any questions related to EPA's Office of Research and Development's responsibilities for recommendations 1-6 and 9, please contact Deborah Heckman at (202) 564-7274. For questions related to EPA's Office of Administration and Resources Management responsibilities for recommendations 7 and 8, please contact Jennifer Hublar at (202) 564-5294. cc: Ramona Trovato Bob Kavlock Nanci Gelb 13-P-0361 24 ------- Howard Corcoran Jim Johnson Bruce Binder Darrell Winner 13-P-0361 ------- Appendix C Distribution Office of the Administrator Principal Deputy Assistant Administrator, Office of Research and Development Assistant Administrator, Office of Administration and Resources Management Agency Follow-Up Official (the CFO) Agency Follow-Up Coordinator General Counsel Associate Administrator for Congressional and Intergovernmental Relations Associate Administrator for External Affairs and Environmental Education Principal Deputy Assistant Administrator, Office of Administration and Resources Management Director, Grants and Interagency Agreements Management Division, Office of Administration and Resources Management Director, Office of Policy and Resource Management, Office of Administration and Resources Management Deputy Director, Office of Policy and Resource Management, Office of Administration and Resources Management Audit Follow-Up Coordinator, Office of Research and Development Audit Follow-Up Coordinator, Office of Administration and Resources Management Audit Follow-Up Coordinator, Office of Policy and Resource Management, Office of Administration and Resources Management 13-P-0361 26 ------- |