$ < 73 O \ ^£OSX sty & ro -Z Ui o PRO"* U.S. ENVIRONMENTAL PROTECTION AGENCY OFFICE OF INSPECTOR GENERAL EPA Data Standards Plan Completed But Additional Steps Are Needed Report No. 12-P-0519 June 5, 2012 Scan this mobile code to I earn more about the EPA OIG ------- Report Contributors: Rudolph M. Brevard Warren Brooks Teresa Richardson Rashad Holloway Abbreviations CIO Chief Information Officer CPIC Capital Planning and Investment Control DERS Data Element Registry Service EPA U.S. Environmental Protection Agency OEI Office of Environmental Information OIG Office of Inspector General READ Registry of EPA Applications and Databases Hotline To report fraud, waste, or abuse, contact us through one of the following methods: e-mail: OIG Hotiirie@epa.aov write: EPA Inspector General Hotline phone: 1-888-546-8740 1200 Pennsylvania Avenue NW fax: 202-566-2599 Mailcode 2431T online: http://www.epa.aov/oia/hotline.htm Washington, DC 20460 ------- STA?. . *. U.S. Environmental Protection Agency 12-P-0519 I' IfcJl \ Office of Inspector General June 5>2012 p 4* 22 At a Glance Why We Did This Review We sought to determine what progress the U.S. Environmental Protection Agency (EPA) made in completing its corrective action plan to close out the Agency- level weakness on data standards. Background In fiscal year 2005, EPA recognized data standards as an Agency-level weakness, and the Office of Environmental Information developed a corrective action plan to address this weakness. EPA's corrective action plan was based on completing three concurrent courses of action. These actions include (1) communicating with EPA program offices on the need to implement data standards more fully, (2) tracking program implementation of data standards, and (3) verifying progress in implementing data standards. EPA Data Standards Plan Completed But Additional Steps Are Needed For further information, contact our Office of Congressional and Public Affairs at (202) 566-2391. The full report is at: www.epa.gov/oiq/reports/2012! 20120605-12-P-0519.pdf What We Found Although EPA completed the steps listed in its corrective action plan to close out the Agency-level weakness on data standards, the actions taken were either incomplete or lacked steps to help management determine the overall effectiveness of the Agency's implementation of data standards. In particular, we determined that EPA: • Developed a data standards training program. However, management took no steps to identify who needed the training, track whether the appropriate personnel took the training, or obtain feedback from staff on the training to ascertain the training's effectiveness. • Created data standards report cards. However, these report cards are inaccurate because EPA offices did not update the system used to create the report cards. Also, the report card format is such that management could not clearly see whether individual offices were in compliance with data standards. • Completed two conformance reviews to determine system compliance with the data standards. However, management made no plans to conduct additional reviews. What We Recommend Based on our on-going discussions with EPA officials, we modified the draft report recommendations to be consistent with the new strategy EPA plans to take to implement data standards. As such, among other recommendations, we recommend that the Assistant Administrator for Environmental Information: • Update the data standards guidance available to EPA offices. • Implement a new data standards communication plan. • Provide specific instructions to EPA offices for updating the Registry of EPA Applications and Databases. • Create a high-level data standards report card for senior executives. • Develop a new strategy for ensuring compliance with data standards. Planned Agency Corrective Actions The Agency concurred with the recommendations and provided a complete corrective action plan to address our revised recommendations. ------- UNITED STATES ENVIRONMENTAL PROTECTION AGENCY WASHINGTON, D.C. 20460 THE INSPECTOR GENERAL June 5, 2012 MEMORANDUM SUBJECT: EPA Data Standards Plan Completed But Additional Steps Are Needed Report No. 12-P-0519 FROM: Arthur A. Elkins, Jr. / / TO: Malcolm D. Jackson Assistant Administrator for Environmental Information and Chief Information Officer This is our report on the subject audit conducted by the Office of Inspector General (OIG) of the U.S. Environmental Protection Agency (EPA). This report contains findings that describe the problems the OIG has identified and corrective actions the OIG recommends. This report represents the opinion of the OIG and does not necessarily represent the final EPA position. Final determinations on matters in this report will be made by EPA managers in accordance with established audit resolution procedures. Action Required We have closed this report in our audit tracking system based on your response to the draft report. We believe the proposed actions, when implemented, will adequately address the report's findings and recommendations. Please provide updated information in EPA's Management Audit Tracking System as you complete each planned corrective action or revise any corrective actions and/or milestone dates. If you are unable to meet your planned milestones, or believe other corrective actions are warranted, please send us a memorandum stating why you are revising the milestones or why you are proposing alternative corrective actions, as required by EPA Manual If you or your staff have any questions regarding this report, please contact Rudolph M. Brevard, Director, Information Resources Management Assessments, at (202) 566-0893 or brevard.rudv@,epa. gov: or Warren M. Brooks, Project Manager, at (202) 566-2467 or brooks .warren@,epa. gov. 2750. ------- EPA Data Standards Plan Completed 12-P-0519 But Additional Steps Are Needed Table of Contents Purpose 1 Background 1 Scope and Methodology 2 Results of Review 2 Online Training May Not Ensure Working Knowledge of Data Standards Implementation 3 Data Standards Report Cards Are Inaccurate and Cumbersome 4 Data Standards Conformance Reviews Are Not Performed or Planned 5 Conclusion 7 Recommendations 7 Agency Comments and OIG Evaluation 8 Status of Recommendations and Potential Monetary Benefits 9 Appendices A Agency Corrective Action Plan (as of February 27, 2006) 10 B Agency Response to Draft Report 12 C Subsequent Agency Response to Draft Report 16 D Distribution 19 ------- Purpose We sought to determine what progress the U.S. Environmental Protection Agency (EPA) has made in completing its corrective action plan to correct the Agency- level weakness on data standards. Background The Reinventing Environmental Information initiative of 1995 directed EPA to develop data standards used in core business areas and implement data standards in all of EPA's national information systems. As defined in the Agency's Data Standards Policy, "data standards are documented consensus-based agreements on the format and definition of common data." The policy states: The use of common terminology and common data element definitions enables the integration of databases, and promotes more efficient and effective use of data by users of commonly defined data from disparate sources. By using data standards, EPA managers and the public can more quickly and accurately conduct environmental assessment and analyze environmental data, maximize use of resources, and be assured of greater data integrity. The goal of the program is to ensure all Agency information systems and data exchange efforts use approved data standards. In fiscal year 2005, EPA first recognized data standards as an Agency-level weakness. The Office of Environmental Information (OEI) developed a comprehensive action plan to address this weakness. The plan indicated that the Agency would undertake three concurrent courses of action: 1. Communicate with EPA program offices on the need to implement data standards more fully 2. Track program implementation of data standards 3. Verify progress in implementing data standards OEI developed its Data Standards Policy in March 2007. The policy applies to all Agency systems that exchange data within and outside EPA, or between EPA and its business partners. The Data Standards Policy "establishes principles, responsibilities, and requirements for the development, maintenance, and implementation of data standards," and reflects "EPA's commitments to improve data quality and promote data interoperability, exchanges, sharing and the ability to effectively use the data in situations other than originally intended." The Federal Managers' Financial Integrity Act requires agencies to establish internal controls to provide reasonable assurance that the integrity of agency programs and resources are protected from fraud, waste, abuse, and misappropriation. Likewise, Office of Management and Budget Circular A-123, 12-P-0519 1 ------- Management's Responsibility for Internal Control, provides guidance for managers responsible for internal control. As such, Agency managers are responsible for taking timely and effective action to correct deficiencies. Managers should determine that a reportable condition has been corrected only when sufficient corrective actions have been taken and the desired results achieved, to include keeping detailed and organized documentation that contains sufficient information to support management's assertions. Scope and Methodology We evaluated EPA's implementation of data standards based on its three courses of action outlined in the corrective action plan dated February 27, 2006. Appendix A contains the complete corrective action plan. We focused on the Agency's data standards communication plan, the data standards online training process, and the mechanism for reporting compliance with the data standards (i.e., data standards report cards). We also reviewed the Agency's process for choosing systems that require compliance with data standards. We conducted this audit from April 2010 through December 2011 at EPA headquarters in Washington, DC. During our audit, we interviewed Agency personnel within the OEI Data Standards Branch. We requested and reviewed supporting documentation for completed corrective actions. We analyzed EPA's communication plan to promote the implementation of data standards to the Agency. In addition, we compared the Agency's classroom data standards training to the current web-based training modules to assess the effectiveness of the current system. To evaluate EPA's data standards compliance reporting method, we reviewed the most recent data standards report cards for an understanding of what the report cards capture and convey. Further, we interviewed program office system administrators to understand their processes for updating their systems' data standards compliance status. We analyzed the data standards report cards for updating and reporting accuracy. We reviewed the Data Standards Policy to determine whether all systems meet the Agency's criteria for compliance with data standards. We analyzed information technology investment documentation for 21 systems not included in the Agency's inventory. We conducted this audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions on our audit objectives. Results of Review Our audit determined that steps the Agency has taken to improve data standards did not meet the intended goal. Management cannot be assured that the time, 12-P-0519 2 ------- money, and effort spent on implementing data standards achieved the desired effects. EPA completed the data standards corrective action plan, as well as provided guidance for the implementation. However, the actions taken were incomplete or lacked steps to help management determine the overall effectiveness of the Agency's implementation of data standards. Management did not periodically assess Agency progress against the corrective action plan and adjust the plan accordingly. According to Office of Management and Budget Circular A-123, documentation should be detailed and organized, and contain enough information to support management's assertion regarding the sufficiency of implemented internal controls. Documentation should include evidence from personnel tasked with monitoring, improving, and assessing internal controls. Data standards affect data quality at each major step of the information collection, storage, and retrieval process. Until the Agency fully implements data standards, the accuracy of Agency data cannot be assured. Online Training May Not Ensure Working Knowledge of Data Standards Implementation The Agency cannot determine whether system owners and developers have the basic knowledge to implement data standards. Although required by Office of Management and Budget Circular A-123, the Agency did not document the assessment process over the internal control structure to monitor, improve, and assess the implemented training. For example, management had not made the online data standards implementation training mandatory for system owners and developers prior to beginning system development. In addition, EPA has no tracking or recordkeeping method in place to document which individuals have completed the training. We believe these key records would have helped to support management's assertion that the training achieved its desired results. In its corrective action plan, the Agency stated that it would develop and conduct training on standards implementation for developers who support EPA program offices. In 2006, OEI developed classroom training to provide systems developers with a working knowledge of all approved data standards, which is needed to achieve conformance with approved standards. OEI geared this classroom training toward individuals managing EPA information systems, contractors and grantees developing information systems for the Agency, and any parties involved in environmental data exchange with the Agency. In December 2009, OEI transitioned from the traditional classroom training to a web-based training approach. Although the training module is accessible to its target audience, the training is not mandatory, and the Agency cannot determine which individuals have completed the training. Further, in contrast to the classroom training, course evaluation or feedback forms are not issued to participants to assess the overall quality of the online data standards implementation training. If the Agency does not make the training mandatory and put proper tracking and quality assessment 12-P-0519 3 ------- methods in place, it has limited assurance that personnel have the expertise to support EPA program offices in conforming with all approved data standards. Subsequent to the issuance of our draft report, we met with Agency officials to discuss the draft report recommendations and their concerns. EPA officials indicated that the Agency no longer uses the online training as the primary method for informing system owners about data standards. EPA officials outlined plans for a more structured communication strategy, which included keeping data standards information up to date and developing implementation guidance. While we are encouraged by the Agency's planned actions, our review of the current data standards website disclosed that it has not been updated recently, contains outdated information about training opportunities, and contains no information regarding standards implementation. Further, during our discussions, we noted Agency officials had not formally developed a new communication strategy and had not identified all methods they planned to use to further raise awareness about data standards. Data Standards Report Cards Are Inaccurate and Cumbersome EPA's data standards report cards are inaccurate because program system administrators did not update READ (Registry of EPA Applications and Databases) with the status of their systems' compliance with data standards. READ is the authoritative source of information about EPA information resources and contains information about an office's compliance with data standards. Additionally, the data standards report cards are cumbersome and do not readily convey the Agency's progress on data standards compliance. Report cards are 24 pages long and do not succinctly and conclusively express the program offices' status regarding compliance with data standards. In the Agency's data standards corrective action plan, the Agency outlined steps to (1) continue to track implementation of data standards, and (2) create a method for reporting compliance with data standards. This method would involve automating the process in READ and issuing a semiannual data standards report card to the Chief Information Officer (CIO) and senior program management. To address this corrective action, the Data Standards Branch personnel, along with the Enterprise Architecture team, issued an annual data call. In that data call, program offices were asked to update their systems information. The program system administrators we interviewed stated that they updated their system data standards compliance status during the Capital Planning and Investment Control (CPIC) process. This update is done by self-reporting data standards compliance information on the appropriate screen in READ. Our analysis of the data standards report card disclosed that several program office systems have not met the completion dates for compliance with applicable data standards. We found that some program offices' systems have been noncompliant for at least 2 years. We found that some program offices did not 12-P-0519 4 ------- update the database as part of the CPIC process with the current data standards compliance information. As the Agency strives to meet the intent of the data standards corrective action plan, it needs to re-enforce the importance of data standards compliance, which begins with accurate and timely data standards compliance information. Without accurate information, the CIO and senior management cannot rely on the data standards report cards and begin remediation to improve compliance with data standards. Data Standards Branch personnel stated that management does not use the report cards to make decisions. CIO and senior management may not use the report cards because they are not easy to understand and are unclear about data standards status. If Agency executives are to be held accountable for ensuring that program office systems comply with data standards, they must have a report card that easily conveys progress against compliance for the data standards applicable to each system. Subsequent to our review, the OIG met with the Agency regarding the report recommendations. The Agency did not agree with our recommendation to develop a policy that requires program offices to update READ. The Agency felt that providing more information within the annual data call was sufficient to inform offices about their responsibility for updating READ. We informed EPA officials that our analysis disclosed that not all offices were aware of what steps to take when they received the annual data call. Additionally, not all EPA offices received the data call. EPA officials acknowledged that providing more details within the data call and making a wider distribution of the annual guidance should result in more complete data in READ. Lastly, EPA officials indicated that READ has the capability to create high-level data standards report cards for senior Agency officials' use. However, our audit disclosed that these reports were generated for OEI management and not being used to make program decisions. EPA officials acknowledged that more could be done to create a better report and to make it more accessible. Data Standards Conformance Reviews Are Not Performed or Planned The Agency cannot verify that all EPA systems conform to established data standards. The Agency no longer performs conformance reviews for systems required to comply with the Data Standards Policy. Without conformance reviews or some type of measurable plan to continue verification of data standards, management has limited assurance that common data elements are consistent across all EPA systems. During our review, the Agency provided a list of exchange network systems with the most recent data standards validation reviews. However, the Agency could not provide documentation for other systems that do not utilize the exchange network or central data exchange. The Agency has not included all applicable systems in its data standards conformance review process. Data standards conformance 12-P-0519 5 ------- reviews are focused on exchange network systems only, which is not in line with the Data Standards Policy. We analyzed information technology investment documentation for 21 Agency systems to determine whether there are other systems that should comply with data standards. As of December 2011, 12 of 21 major information technology investments (57 percent) share data with two or more EPA programs. We asked Agency management whether these systems meet the criteria for data standards compliance. Management stated that the systems data are not significant enough to comply with data standards. We do not agree, because this view is not in accordance with the purpose of submitting funding request documentation for information technology investments. According to federal requirements, agencies should submit investment documentation for systems that support an Agency's core mission functions. Thus, these 12 systems, and by extension, their data, merit conformance reviews to determine whether the data conform with established data standards and meet the Agency's needs in terms of core mission responsibilities. Conformance reviews would further facilitate data sharing across EPA and partner systems. Further, according to EPA's Data Standards Policy, these 12 non-exchange network systems should either be in compliance with data standards or have a data standards waiver on file with Agency officials. We requested that OEI provide the CIO-approved data standards waivers or documentation that excludes these systems form the data standards policy. The Agency was unable to provide such documentation relevant to the 12 non-exchange network systems. Based on our review of the 21 Agency systems, EPA made little progress to ensure that non-exchange network systems met data standards. We met with OEI personnel to discuss our concerns with the data standards corrective action plan. They shared a concept to develop data dictionaries for the Data Element Registry Service (DERS)—EPA's registry of data dictionaries and their data elements. Management stated that it was actively working with program offices to develop usable data dictionaries that would be registered in DERS. As data dictionaries are registered, OEI plans to work with program offices to determine how data elements in the data dictionary map to EPA's standardized data. This mapping will identify how systems conform to data standards. Unlike conformance reviews, OEI plans to register data dictionaries and crosswalk- related data elements with standards data elements in DERS, which will be a more automated method of determining data conformance. OEI confirmed that developing data dictionaries will be an ongoing and iterative process. Management plans to prioritize data dictionary development for those systems that are subject to CPIC and those that exchange information with other systems. While OEI will work with all program offices, focus will be on those systems that exchange information with EPA's partners, including states, tribes, and industry. This effort will become EPA's method for reviewing offices' efforts 12-P-0519 6 ------- to implement data standards. OEI expects the data dictionary development for systems and data sets to take at least 5 years. The plan OEI outlined during our meeting lacks a concrete and measurable strategy with timelines and milestone dates. We met with EPA officials subsequent to the issuance of our draft report. EPA officials indicated that data standards conformance reviews were too resource intensive and the Agency is no longer using this approach to ensure compliance with data standards. Management indicated that helping system owners develop useable data dictionaries was the prevailing methodology the Agency is using to ensure conformance with data standards. We inquired about the strategy's objectives, planned actions, and major milestones and how the Agency would measure its progress for achieving this new strategy. EPA officials disclosed that while they are currently working with some offices to develop data dictionaries, no formal strategy has been developed. Furthermore, EPA acknowledged that more could be done to ensure data standards waiver requests were available and that the Agency would take steps to ensure the waivers are kept within the official record keeping system. Conclusion Although EPA took steps to address its corrective action plan, actions taken were either incomplete or lacked steps to help management determine the overall effectiveness of actions taken. EPA should take additional steps to ensure that it fully implements data standards in existing and new systems. Such steps would include working with program offices to develop a plan for putting data standards in place. Recommendations We recommend that the Assistant Administrator for Environmental Information and Chief Information Officer: 1. Conduct and document a review of the data standards website to identify areas where EPA could provide updated guidance on data standards. Develop an action plan to produce needed information and implement a process to annually review and update the data standards website. 2. Develop and implement a new data standards communication plan that outlines a structured approach for marketing and raising awareness of EPA's data standards and identifies key Agency opportunities for promoting data standards. 3. Add new language to the annual data standards data call memorandum with specific guidance that directs EPA offices to update the required data standards fields in READ. 12-P-0519 7 ------- 4. Increase the distribution of the annual data standards data call memorandum to all system owners. 5. Develop and distribute a high-level data standards report card that informs EPA regional and program office executives about their offices' progress in complying with EPA data standards. 6. Develop and implement a process to maintain data standards waivers within EPA's official record keeping system. 7. Develop a formal plan that outlines EPA's use of data dictionaries as the methodology for facilitating compliance with data standards. The plan should include, at a minimum, the strategy's broad objectives, major action items, and major milestone dates. Agency Comments and OIG Evaluation In its February 9, 2012, response to the draft report, EPA did not agree or disagree with the draft report's recommendations to address the findings. However, EPA cited actions already underway, as well as new actions that it believes would help the Agency continue to move toward greater conformance with data standards. The Agency indicated that it would pursue data standards outreach and communications using methods other than training. Additionally, the Agency indicated it would disseminate the data call to all READ stewards to ensure awareness of the requirement to update READ, and ensure the PDFs of the current waivers are archived in the Records Repository. We met with the Agency prior to receiving the response to the draft report to get a better understanding of the actions management is taking to address the reported findings and recommendations. During our discussion, the Agency provided an outline of actions they were taking to address the report findings. We accepted many of the proposed actions and sent the Agency revised draft report recommendations. Subsequent to receiving the Agency's response to the draft report, the Agency notified the OIG via electronic mail that it concurred with the recommendations. The Agency also provided a complete corrective action plan to address our revised recommendations. We consider these recommendations open until the Agency completes its corrective action plan. Appendix B provides the full text of the Agency's response to the draft report. Appendix C provides an excerpt of the Agency's subsequent response to the draft report. 12-P-0519 8 ------- Status of Recommendations and Potential Monetary Benefits POTENTIAL MONETARY RECOMMENDATIONS BENEFITS (in $000s) Rec. No. Page No. Subject Status1 Action Official Planned Completion Date Claimed Amount Agreed-To Amount 1 7 Conduct and document a review of the data standards website to identify areas where EPA could provide updated guidance on data standards. Develop an action plan to produce needed information and implement a process to annually review and update the data standards website. 0 Assistant Administrator for Environmental Information and Chief Information Officer 2 7 Develop and implement a new data standards communication plan that outlines a structured approach for marketing and raising awareness of EPA's data standards and identifies key Agency opportunities for promoting data standards. 0 Assistant Administrator for Environmental Information and Chief Information Officer 3 7 Add new language to the annual data standards data call memorandum with specific guidance that directs EPA offices to update the required data standards fields in READ. 0 Assistant Administrator for Environmental Information and Chief Information Officer 4 8 Increase the distribution of the annual data standards data call memorandum to all system owners. 0 Assistant Administrator for Environmental Information and Chief Information Officer 5 8 Develop and distribute a high-level data standards report card that informs EPA regional and program office executives about their offices' progress in complying with EPA data standards. 0 Assistant Administrator for Environmental Information and Chief Information Officer 6 8 Develop and implement a process to maintain data standards waivers within EPA's official record keeping system. 0 Assistant Administrator for Environmental Information and Chief Information Officer 7 8 Develop a formal plan that outlines EPA's use of data dictionaries as the methodology for facilitating compliance with data standards. The plan should include, at a minimum, the strategy's broad 0 Assistant Administrator for Environmental Information and Chief Information Officer objectives, major action items, and major milestone dates. 1 0 = recommendation is open with agreed-to corrective actions pending C = recommendation is closed with all agreed-to actions completed U = recommendation is unresolved with resolution efforts in progress 12-P-0519 9 ------- Appendix A Agency Corrective Action Plan (as of February 27, 2006) Weakness: While EPA has a substantial effort in place to develop data standards and provide guidance for their implementation, the Agency needs to establish a process for ensuring that each data standard adopted by the Agency is fully implemented in a cost-effective and timely manner. To address this weakness, the Office of Environmental Information (OEI) is implementing three concurrent course of action: Continue to communicate with EPA program offices on need to implement data standards more fully; Continue to track program implementation of data standards; and Continue to verify progress in implementing data standards. Executive Summary: While the Agency has a substantive effort in place to develop data standards and provide guidance for their implementation, incorporation of data standards in information collections from initial plans to obtaining the data for analysis is just becoming a routine activity in all programs. Using data standards affects data quality at each major step of the information collection, storage and retrieval process and is a factor in the usability of data sets. Furthermore, if applying data standards causes a program to modify its business practices, the implementation may take several years to conduct because of modifications to data gathering, database design and development, and interpretation of results. CORRECTIVE ACTION MILESTONES OEI CONTACT STATUS/ COMMENTS DUE DATE 1. Continue to Communicate with EPA Program Offices on Need to Implement Data Standards More Fully 1 A. Develop a Communication Plan to: 1) address implementation of 15 upcoming standards (the Institutional Controls Standard and various other standards related to Environmental Sampling, Analysis and Results), 2) promote awareness of implementation documentation including the Implementation Strategy and various procedures, and 3) promote use of best practices in the implementation of specific standards. Dawn Banks 202-566-0625 03/2006 12-P-0519 10 ------- CORRECTIVE ACTION MILESTONES OEI CONTACT STATUS/ COMMENTS DUE DATE 1. Continue to Communicate with EPA Program Offices on Need to Implement Data Standards More Fully IB. Hold a System of Registry Users Conference to include promoting awareness and use of existing data standards Dawn Banks 202-566-0625 12/2006 and annually 1C. Develop and Conduct Training on standards implementation for developers supporting EPA program offices Dawn Banks 202-566-0625 10/2006 and ongoing 2. Continue to track program implementation of data standards 2A. Make Tracking Report on Data Standards Implementation available in Registry of EPA Applications and Databases (READ) Dawn Banks 202-566-0625 06/2006 and ongoing 2B.Continue to issue semi-annual Data Standards "Report Card" Dawn Banks 202-566-0625 March and September annually 3. Continue to verify progress in implementing data standards 3 A. Report on use of data standards validation processes in EPA Central Data Exchange Dawn Banks 202-566-0625 04/2006 3B. Conduct Conformance Review of Estuary Program Dawn Banks 202-566-0625 05/2006 3C. Conduct Conformance Review "To Be Determined by Office of Environmental Information Big Decisions Initiative and Subject to Available Related Funding". Dawn Banks 202-566-0625 TBD 12-P-0519 11 ------- Appendix B Agency Response to Draft Report MEMORANDUM SUBJECT: Response to OIG Draft "EPA Data Standards Plan Completed but Additional Steps Are Needed" FROM: Malcolm D. Jackson Assistant Administrator and Chief Information Officer TO: Rudolph M. Brevard Director, Information Resources Management Assessments Office of Mission Systems Office of Inspector General The Office of Environmental Information (OEI) appreciates the interest and emphasis placed on data standards conformance by the Office of the Inspector General (OIG). In its draft "EPA Data Standards Plan Completed but Additional Steps Are Needed," dated January 4, 2012, OIG identifies six recommendations to enhance compliance with data standards by EPA system owners. In response to a meeting between the two offices, OEI outlines below the actions already underway, as well as new actions, which will help the Agency continue to move toward greater conformance with data standards. The OIG recommendations were in addition to steps OEI had already taken to meet its commitments in response to "Implementation of Data Standards" under the Federal Managers Financial Integrity Act (FMFIA). From 2005 to 2010, OEI introduced considerable changes, including a data standards screen in EPA's Registry of EPA Applications and Databases (READ) that allows system owners to self-report their data standards conformance, and a report function in READ to track the status by office and by data standard. OEI also developed online training modules for any individual who wishes to learn about data standards implementation. OEI continues to support data standards. The draft "System Lifecycle Management (SLCM) Procedure" includes a requirement for systems to implement data standards. There also is a draft "Data Resources Policy" and its "Registration and Documenting Data Resources Procedure" that require data be compliant with data standards. In 2011, OEI put a registry for data dictionaries into production that will support data standards implementation. The Data Element Registry Services (DERS) will enable OEI to map data elements in system data dictionaries with the data elements in EPA's data standards. The crosswalk can be kept current by updating DERS with any updates to the system data dictionaries. 12-P-0519 12 ------- OEI is working with program offices to develop comprehensive data dictionaries to be loaded into DERS. Developing robust data dictionaries for the more than 120 Capital Planning and Investment Control (CPIC) systems will take several years. As data dictionaries are registered, OEI will work with the program offices to determine which data elements in a specific data dictionary map to EPA's standardized data elements. Below are the six recommendations by OIG, and OEI's responses and completion dates. OIG Recommendation 1: Require system owners and developers to complete the data standards implementation training. OEI Response: As agreed between OEI and OIG, Recommendations 1 and 2 are not necessary since OEI is already pursuing data standards outreach and communications using other methods. The Information Exchange and Services Division (IESD) within OEI has identified increased communications as a priority and has recently established a communications team to raise awareness of data standards as well as other IESD services. Specifically for data standards, OEI is developing implementation guidance for each data standard that will be posted to the data standards Webpage fwww.epa.gov/datastandardsY To ensure currency of information, OEI also will review this site at least annually. OEI also will discuss data standards at the upcoming Vendor Day in Spring 2012. Completion Dates: Vendor Day: Spring 2012 Implementation Guidance: 2013 Communications: Ongoing OIG Recommendation 2: Develop and employ a mechanism to record and track those individuals who have completed the training. OEI Response: Please see the response to OIG Recommendation 1. OIG Recommendation 3: Develop a policy to update READ during the CPIC review process. OEI Response: The annual READ Data Call already requires that systems subject to the CPIC process must complete their READ records, including the data standards screen. OEI's CPIC team provides a further check by mandating that systems subject to CPIC have fully populated READ records. The data call goes out to a wide audience, including Senior Information Officials (SIOs) and Information Management Officers (IMOs). Going forward, OEI will also disseminate the data call to all READ stewards to ensure awareness of this requirement. OEI also is developing two EPA policies and accompanying procedures, the System Lifecycle Management Procedure under the System Lifecycle Policy and the Data Resources Policy, which stipulate conformance to data standards. 12-P-0519 13 ------- Completion Date: System Life Cycle Management Policy and Procedure: 2012 Data Resources Policy and Procedure: 2012 READ Data Call: June 2012 OIG Recommendation 4: Simplify the data standards report card for usability and ensure that it is distributed to all relevant program office personnel. OEI Response: OEI will distribute, in conjunction with the data call, office-specific Data Standards Report Cards that highlight those systems that have indicated their status for certain data standards as "in process". This will be an opportunity to raise awareness among senior management of the importance of data standards. Completion Date: READ Data Call: June 2012 OIG Recommendation 5: Develop and execute a plan to verify system data conformance with data standards policies for all EPA systems, including timelines and milestone dates. OEI Response: OEI is working with program offices to develop data dictionaries. This effort will take many years as developing a robust data dictionary is time consuming and requires significant communication with system owners. Additionally, data dictionaries vary widely in size and complexity. For these reasons, OEI has an internal target of seven data dictionaries per year in lieu of a specific plan identifying systems and milestones. OEI also is formalizing its standard operating procedures for the DERS, which will include steps for obtaining and developing data dictionaries. Completion Date: Data Dictionary Development: Seven data dictionaries per year. DERS SOP: August 2012 OIG Recommendation 6: Document and maintain data standards waivers for all EPA national systems to comply with EPA's Data Standards Policy. OEI Response: There is a process for documenting and maintaining data standards waivers. The READ Report Card provides information about systems that have requested waivers and the status of the waivers. OEI will ensure the PDFs of the current waivers are archived in the Records Repository. Completion Date: Waivers archived in Records Schedule: February 2012 Waivers recorded in READ: Done 12-P-0519 14 ------- In conclusion, OEI promotes and facilitates data standards conformance and will continue to do so. We believe we have an effective approach to managing data standards conformance while providing service to the program offices. Thank you for bringing ideas for how to improve outreach and value to this program through this assessment. For any questions, please do not hesitate to contact John Harman at (202) 566-0748. cc: Renee Wynn Andrew Battin Connie Dwyer 12-P-0519 15 ------- Appendix C Subsequent Agency Response to Draft Report To: Rudy Brevard/OIG/USEPA/US@EPA From: Jeffrey Wells/DC/USEPA/US Date: 05/22/2012 07:19PM Subject: Re: OIG Recommendations on Data Standards Conformance Hi Rudy, We concur with OIG's recommendations. Thank you very much. Jeff Jeff Wells Acting Deputy Director Office of Information Collection Office of Environmental Information U.S. Environmental Protection Agency From: Jeffrey Wells/DC/USEPA/US To; Rudy Brevard/OIG/USEPA/US@EPA Cc: Date: 05/21/2012 06:12 PM Subject: OIG Recommendations on Data Standards Conformance Hi Rudy, I'm responding for Andy Battin and following up on an email you sent him asking for OEI's response to the seven recommendations on our Data Standards program that your group identified to us. Here is the response from our office as requested. It is my understanding that our team has previewed these responses with you and your team already - but that you were looking for an email from us to codify them formally in response to the recommendations. I understand that the next step is for OIG to issue the final report, which will include the seven recommendations listed below. OEI will then respond within 90 days with the responses you see below in italics along with implementation dates (see below) that address these recommendations. That, as I 12-P-0519 16 ------- understand it, is the last step in the process. Here, below are the seven OIG recommendations along with our proposed responses and dates for completion. OIG Recommendations and OEI Responses 1. Conduct and document a review of the Data Standards website to identify areas where EPA could provide updated guidance on Data Standards. Develop an action plan to produce needed information and implement a process to annually review and update the Data Standards website. OEI will include the actions to review the Data Standards website annually in the Data Standards Communications plan; July 2012 2. Develop and implement a new Data Standards communication plan that outlines a structured approach for marketing and raising awareness of EPA's data standards and identifies key Agency opportunities for promoting data standards. OEI will update its data standards communications plan, which will identify steps for creating increased awareness of EPA data standards; July 2012 3. Add new language into the annual Data Standards data call memorandum with specific guidance that directs EPA offices to update the required data standards fields in READ. OEI will include new language to the READ Data Call that directs EPA offices to update data standards conformance status on the data standards screen in READ; June 2012 4. Increase the distribution of the annual Data Standards data call memorandum to all system owners. OEI will disseminate the READ Data Call to all Information Management Officers for distribution to system owners; June 2012 5. Develop and distribute a high-level data standards report card that informs EPA regional and program office executives about their office's progress in complying with EPA data standards. OEI will issue the high-level data standards report card as part of the READ Data Call; June 2012 6. Develop a formal plan that outlines EPA's use of data dictionaries as the methodology for facilitating compliance with data standards. The plan should include at a minimum the strategy's broad objectives, major action items, and major milestone dates. OEI will develop a Standard Operating Procedure (SOP) for developing data dictionaries. It will indicate that OEI intends to develop at least seven data dictionaries per year - focusing on systems that are subject to Capital Planning and Investment Control (CPIC); September 2012 12-P-0519 17 ------- 7. Develop and implement a process to maintain data standards waivers within EPA's official record keeping system. OEI will maintain all waivers in READ and register all accepted waivers in the EPA Records Schedule. OEI will add a sentence to the Data Standards screen in READ that reads: "If an EPA Data Standard is not listed, the system owners have made the determination that the standard is not applicable to his /her system August 2012 Source: Wells, Jeffrey. Acting Deputy Director of the Office of Information Collection, Office of Environmental Information, responding for Andrew Battin, Director of the Office of Information Collection. Excerpt from electronic correspondence. Subject, "OIG Recommendations on Data Standards Conformance," May 21, 2012 and May 22, 2012 12-P-0519 18 ------- Appendix D Distribution Office of the Administrator Assistant Administrator for Environmental Information and Chief Information Officer Agency Follow-Up Official (the CFO) Agency Follow-Up Coordinator General Counsel Associate Administrator for Congressional and Intergovernmental Relations Associate Administrator for External Affairs and Environmental Education Director, Office of Information Collection, Office of Environmental Information Audit Follow-Up Coordinator, Office of Environmental Information 12-P-0519 19 ------- |