^tDsr-% • B \ U.S. Environmental Protection Agency Office of Inspector General At a Glance 16-N-0317 September 21, 2016 Why We Did This Review We reviewed the Council of the Inspectors General on Integrity and Efficiency's (CIGIE's) Inspection and Evaluation (l&E) pilot External Peer Review (EPR) process. Specifically, we evaluated whether the EPR pilot process (1) provided a basis to ensure CIGIE Quality Standards for Inspection and Evaluation are achieved; and (2) identified lessons learned and best practices, and whether recommendations were incorporated into subsequent training and guidance. CIGIE is establishing an external peer review process to provide assurance that Offices of Inspector General adhere to the professional standards established in the 2012 CIGIE Quality Standards for Inspection and Evaluation, also called the "Blue Book." These standards were developed to help ensure a level of quality, objectivity and independence in the work of offices that adopt the standards. This report addresses the following U.S. Environmental Protection Agency Office of Inspector General goal: • Be the best in public service. Send all inquiries to our public affairs office at (202) 566-2391 or visit www.epa.gov/oiq. Listing of OIG reports. Examination of Pilot Peer Review Process for Inspectors General That Follow "Blue Book" Quality Standards for Inspection and Evaluation What We Found The EPR process provides a basis for determining participants' adherence to seven of 14 Blue Book standards. All 14 quality standards in the EPR process are needed to provide a reasonable basis that Inspectors General who adopt the Blue Book quality standards are being adequately evaluated. A peer review process that measures adherence to all the quality standards established for federal Inspector General Inspection and Evaluation offices provides assurance that participating offices are being adequately evaluated. We identified the following best practices for conducting peer reviews of l&E units: • Continue using EPR teams comprised of staff from multiple l&E units. • Continue to support time management by encouraging offsite access to documents needed to conduct the EPR. We found opportunities for EPR improvement, including the following: • Develop a policy to conduct periodic reviews of the EPR process. • Develop a policy to require that corrective actions are implemented if an l&E unit is found deficient in adhering to all quality standards. • Update the EPR guidance document to include a Blue Book standards reference guide to promote consistent analysis of quality standards and language to require that l&E units being reviewed have no input in the EPR report selection process. • Retain documents created to support peer review findings following the completion of the EPR. • Develop a pre-planning tool to assure that review team members possess the prerequisites, such as security clearance or specific software skills, required by the l&E unit being reviewed. Suggestions We suggest that all quality standards in the CIGIE peer review guidance be evaluated, or seek a legal opinion concerning the appropriateness of a peer review process that reviews adherence to only seven of 14 standards; develop a policy to conduct periodic reviews of the effectiveness of the EPR process; require that corrective actions are implemented if an l&E unit is found deficient in adhering to all quality standards; update EPR guidance documents with best practices for peer reviews; and develop a pre-EPR planning tool. The CIGIE l&E Roundtable agreed with all the suggestions and has modified its draft Guide for Conducting Peer Reviews. The Roundtable indicated it will recommend that the CIGIE l&E Committee present these recommendations to the full CIGIE for consideration. ------- |