$
<
73
O
\
^£OSX
sty
&
ro
-Z
Ui
o
PRO"*
U.S. ENVIRONMENTAL PROTECTION AGENCY
OFFICE OF INSPECTOR GENERAL
EPA Data Standards Plan
Completed But Additional
Steps Are Needed
Report No. 12-P-0519
June 5, 2012
Scan this mobile
code to I earn more
about the EPA OIG

-------
Report Contributors:
Rudolph M. Brevard
Warren Brooks
Teresa Richardson
Rashad Holloway
Abbreviations
CIO
Chief Information Officer
CPIC
Capital Planning and Investment Control
DERS
Data Element Registry Service
EPA
U.S. Environmental Protection Agency
OEI
Office of Environmental Information
OIG
Office of Inspector General
READ
Registry of EPA Applications and Databases
Hotline
To report fraud, waste, or abuse, contact us through one of the following methods:
e-mail:	OIG Hotiirie@epa.aov	write: EPA Inspector General Hotline
phone:	1-888-546-8740	1200 Pennsylvania Avenue NW
fax:	202-566-2599	Mailcode 2431T
online:	http://www.epa.aov/oia/hotline.htm	Washington, DC 20460

-------
STA?.
. *. U.S. Environmental Protection Agency	12-P-0519
I' IfcJl \ Office of Inspector General	June 5>2012
p
4*
22
At a Glance
Why We Did This Review
We sought to determine
what progress the U.S.
Environmental Protection
Agency (EPA) made in
completing its corrective action
plan to close out the Agency-
level weakness on data
standards.
Background
In fiscal year 2005, EPA
recognized data standards as an
Agency-level weakness, and
the Office of Environmental
Information developed a
corrective action plan to
address this weakness. EPA's
corrective action plan was
based on completing three
concurrent courses of action.
These actions include
(1) communicating with EPA
program offices on the need to
implement data standards more
fully, (2) tracking program
implementation of data
standards, and (3) verifying
progress in implementing data
standards.
EPA Data Standards Plan Completed
But Additional Steps Are Needed
For further information, contact
our Office of Congressional and
Public Affairs at (202) 566-2391.
The full report is at:
www.epa.gov/oiq/reports/2012!
20120605-12-P-0519.pdf
What We Found
Although EPA completed the steps listed in its corrective action plan to close out
the Agency-level weakness on data standards, the actions taken were either
incomplete or lacked steps to help management determine the overall
effectiveness of the Agency's implementation of data standards. In particular, we
determined that EPA:
•	Developed a data standards training program. However, management
took no steps to identify who needed the training, track whether the
appropriate personnel took the training, or obtain feedback from staff on
the training to ascertain the training's effectiveness.
•	Created data standards report cards. However, these report cards are
inaccurate because EPA offices did not update the system used to create
the report cards. Also, the report card format is such that management
could not clearly see whether individual offices were in compliance with
data standards.
•	Completed two conformance reviews to determine system compliance
with the data standards. However, management made no plans to conduct
additional reviews.
What We Recommend
Based on our on-going discussions with EPA officials, we modified the draft
report recommendations to be consistent with the new strategy EPA plans to take
to implement data standards. As such, among other recommendations, we
recommend that the Assistant Administrator for Environmental Information:
•	Update the data standards guidance available to EPA offices.
•	Implement a new data standards communication plan.
•	Provide specific instructions to EPA offices for updating the Registry of
EPA Applications and Databases.
•	Create a high-level data standards report card for senior executives.
•	Develop a new strategy for ensuring compliance with data standards.
Planned Agency Corrective Actions
The Agency concurred with the recommendations and provided a complete
corrective action plan to address our revised recommendations.

-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
THE INSPECTOR GENERAL
June 5, 2012
MEMORANDUM
SUBJECT: EPA Data Standards Plan Completed But Additional Steps Are Needed
Report No. 12-P-0519
FROM: Arthur A. Elkins, Jr.


/
/
TO:
Malcolm D. Jackson
Assistant Administrator for Environmental Information and
Chief Information Officer
This is our report on the subject audit conducted by the Office of Inspector General (OIG) of the
U.S. Environmental Protection Agency (EPA). This report contains findings that describe the
problems the OIG has identified and corrective actions the OIG recommends. This report
represents the opinion of the OIG and does not necessarily represent the final EPA position.
Final determinations on matters in this report will be made by EPA managers in accordance with
established audit resolution procedures.
Action Required
We have closed this report in our audit tracking system based on your response to the draft
report. We believe the proposed actions, when implemented, will adequately address the report's
findings and recommendations. Please provide updated information in EPA's Management Audit
Tracking System as you complete each planned corrective action or revise any corrective actions
and/or milestone dates. If you are unable to meet your planned milestones, or believe other
corrective actions are warranted, please send us a memorandum stating why you are revising the
milestones or why you are proposing alternative corrective actions, as required by EPA Manual
If you or your staff have any questions regarding this report, please contact Rudolph M. Brevard,
Director, Information Resources Management Assessments, at (202) 566-0893 or
brevard.rudv@,epa. gov: or Warren M. Brooks, Project Manager, at (202) 566-2467 or
brooks .warren@,epa. gov.
2750.

-------
EPA Data Standards Plan Completed	12-P-0519
But Additional Steps Are Needed
	Table of Contents	
Purpose		1
Background 		1
Scope and Methodology		2
Results of Review		2
Online Training May Not Ensure Working Knowledge of
Data Standards Implementation		3
Data Standards Report Cards Are Inaccurate and Cumbersome 		4
Data Standards Conformance Reviews Are Not Performed or Planned		5
Conclusion		7
Recommendations 		7
Agency Comments and OIG Evaluation 		8
Status of Recommendations and Potential Monetary Benefits		9
Appendices
A Agency Corrective Action Plan (as of February 27, 2006)		10
B Agency Response to Draft Report		12
C Subsequent Agency Response to Draft Report 		16
D Distribution 		19

-------
Purpose
We sought to determine what progress the U.S. Environmental Protection Agency
(EPA) has made in completing its corrective action plan to correct the Agency-
level weakness on data standards.
Background
The Reinventing Environmental Information initiative of 1995 directed EPA to
develop data standards used in core business areas and implement data standards
in all of EPA's national information systems. As defined in the Agency's Data
Standards Policy, "data standards are documented consensus-based agreements
on the format and definition of common data." The policy states:
The use of common terminology and common data element
definitions enables the integration of databases, and promotes more
efficient and effective use of data by users of commonly defined
data from disparate sources. By using data standards, EPA
managers and the public can more quickly and accurately conduct
environmental assessment and analyze environmental data,
maximize use of resources, and be assured of greater data integrity.
The goal of the program is to ensure all Agency information
systems and data exchange efforts use approved data standards.
In fiscal year 2005, EPA first recognized data standards as an Agency-level
weakness. The Office of Environmental Information (OEI) developed a
comprehensive action plan to address this weakness. The plan indicated that the
Agency would undertake three concurrent courses of action:
1.	Communicate with EPA program offices on the need to implement data
standards more fully
2.	Track program implementation of data standards
3.	Verify progress in implementing data standards
OEI developed its Data Standards Policy in March 2007. The policy applies to all
Agency systems that exchange data within and outside EPA, or between EPA and
its business partners. The Data Standards Policy "establishes principles,
responsibilities, and requirements for the development, maintenance, and
implementation of data standards," and reflects "EPA's commitments to improve
data quality and promote data interoperability, exchanges, sharing and the ability
to effectively use the data in situations other than originally intended."
The Federal Managers' Financial Integrity Act requires agencies to establish
internal controls to provide reasonable assurance that the integrity of agency
programs and resources are protected from fraud, waste, abuse, and
misappropriation. Likewise, Office of Management and Budget Circular A-123,
12-P-0519
1

-------
Management's Responsibility for Internal Control, provides guidance for
managers responsible for internal control. As such, Agency managers are
responsible for taking timely and effective action to correct deficiencies.
Managers should determine that a reportable condition has been corrected only
when sufficient corrective actions have been taken and the desired results
achieved, to include keeping detailed and organized documentation that contains
sufficient information to support management's assertions.
Scope and Methodology
We evaluated EPA's implementation of data standards based on its three courses
of action outlined in the corrective action plan dated February 27, 2006.
Appendix A contains the complete corrective action plan. We focused on the
Agency's data standards communication plan, the data standards online training
process, and the mechanism for reporting compliance with the data standards
(i.e., data standards report cards). We also reviewed the Agency's process for
choosing systems that require compliance with data standards.
We conducted this audit from April 2010 through December 2011 at EPA
headquarters in Washington, DC. During our audit, we interviewed Agency
personnel within the OEI Data Standards Branch. We requested and reviewed
supporting documentation for completed corrective actions. We analyzed EPA's
communication plan to promote the implementation of data standards to the
Agency. In addition, we compared the Agency's classroom data standards training
to the current web-based training modules to assess the effectiveness of the
current system. To evaluate EPA's data standards compliance reporting method,
we reviewed the most recent data standards report cards for an understanding of
what the report cards capture and convey. Further, we interviewed program office
system administrators to understand their processes for updating their systems'
data standards compliance status. We analyzed the data standards report cards for
updating and reporting accuracy. We reviewed the Data Standards Policy to
determine whether all systems meet the Agency's criteria for compliance with
data standards. We analyzed information technology investment documentation
for 21 systems not included in the Agency's inventory.
We conducted this audit in accordance with generally accepted government
auditing standards. Those standards require that we plan and perform the audit to
obtain sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that the
evidence obtained provides a reasonable basis for our findings and conclusions on
our audit objectives.
Results of Review
Our audit determined that steps the Agency has taken to improve data standards
did not meet the intended goal. Management cannot be assured that the time,
12-P-0519
2

-------
money, and effort spent on implementing data standards achieved the desired
effects. EPA completed the data standards corrective action plan, as well as
provided guidance for the implementation. However, the actions taken were
incomplete or lacked steps to help management determine the overall
effectiveness of the Agency's implementation of data standards. Management did
not periodically assess Agency progress against the corrective action plan and
adjust the plan accordingly. According to Office of Management and Budget
Circular A-123, documentation should be detailed and organized, and contain
enough information to support management's assertion regarding the sufficiency
of implemented internal controls. Documentation should include evidence from
personnel tasked with monitoring, improving, and assessing internal controls.
Data standards affect data quality at each major step of the information collection,
storage, and retrieval process. Until the Agency fully implements data standards,
the accuracy of Agency data cannot be assured.
Online Training May Not Ensure Working Knowledge of Data
Standards Implementation
The Agency cannot determine whether system owners and developers have the
basic knowledge to implement data standards. Although required by Office of
Management and Budget Circular A-123, the Agency did not document the
assessment process over the internal control structure to monitor, improve, and
assess the implemented training. For example, management had not made the
online data standards implementation training mandatory for system owners and
developers prior to beginning system development. In addition, EPA has no
tracking or recordkeeping method in place to document which individuals have
completed the training. We believe these key records would have helped to
support management's assertion that the training achieved its desired results.
In its corrective action plan, the Agency stated that it would develop and conduct
training on standards implementation for developers who support EPA program
offices. In 2006, OEI developed classroom training to provide systems developers
with a working knowledge of all approved data standards, which is needed to
achieve conformance with approved standards. OEI geared this classroom training
toward individuals managing EPA information systems, contractors and grantees
developing information systems for the Agency, and any parties involved in
environmental data exchange with the Agency. In December 2009, OEI
transitioned from the traditional classroom training to a web-based training
approach.
Although the training module is accessible to its target audience, the training is
not mandatory, and the Agency cannot determine which individuals have
completed the training. Further, in contrast to the classroom training, course
evaluation or feedback forms are not issued to participants to assess the overall
quality of the online data standards implementation training. If the Agency does
not make the training mandatory and put proper tracking and quality assessment
12-P-0519
3

-------
methods in place, it has limited assurance that personnel have the expertise to
support EPA program offices in conforming with all approved data standards.
Subsequent to the issuance of our draft report, we met with Agency officials to
discuss the draft report recommendations and their concerns. EPA officials
indicated that the Agency no longer uses the online training as the primary
method for informing system owners about data standards. EPA officials outlined
plans for a more structured communication strategy, which included keeping data
standards information up to date and developing implementation guidance. While
we are encouraged by the Agency's planned actions, our review of the current
data standards website disclosed that it has not been updated recently, contains
outdated information about training opportunities, and contains no information
regarding standards implementation. Further, during our discussions, we noted
Agency officials had not formally developed a new communication strategy and
had not identified all methods they planned to use to further raise awareness about
data standards.
Data Standards Report Cards Are Inaccurate and Cumbersome
EPA's data standards report cards are inaccurate because program system
administrators did not update READ (Registry of EPA Applications and
Databases) with the status of their systems' compliance with data standards.
READ is the authoritative source of information about EPA information resources
and contains information about an office's compliance with data standards.
Additionally, the data standards report cards are cumbersome and do not readily
convey the Agency's progress on data standards compliance. Report cards are 24
pages long and do not succinctly and conclusively express the program offices'
status regarding compliance with data standards.
In the Agency's data standards corrective action plan, the Agency outlined steps
to (1) continue to track implementation of data standards, and (2) create a method
for reporting compliance with data standards. This method would involve
automating the process in READ and issuing a semiannual data standards report
card to the Chief Information Officer (CIO) and senior program management. To
address this corrective action, the Data Standards Branch personnel, along with
the Enterprise Architecture team, issued an annual data call. In that data call,
program offices were asked to update their systems information. The program
system administrators we interviewed stated that they updated their system data
standards compliance status during the Capital Planning and Investment Control
(CPIC) process. This update is done by self-reporting data standards compliance
information on the appropriate screen in READ.
Our analysis of the data standards report card disclosed that several program
office systems have not met the completion dates for compliance with applicable
data standards. We found that some program offices' systems have been
noncompliant for at least 2 years. We found that some program offices did not
12-P-0519
4

-------
update the database as part of the CPIC process with the current data standards
compliance information. As the Agency strives to meet the intent of the data
standards corrective action plan, it needs to re-enforce the importance of data
standards compliance, which begins with accurate and timely data standards
compliance information. Without accurate information, the CIO and senior
management cannot rely on the data standards report cards and begin remediation
to improve compliance with data standards.
Data Standards Branch personnel stated that management does not use the report
cards to make decisions. CIO and senior management may not use the report
cards because they are not easy to understand and are unclear about data standards
status. If Agency executives are to be held accountable for ensuring that program
office systems comply with data standards, they must have a report card that
easily conveys progress against compliance for the data standards applicable to
each system.
Subsequent to our review, the OIG met with the Agency regarding the report
recommendations. The Agency did not agree with our recommendation to
develop a policy that requires program offices to update READ. The Agency felt
that providing more information within the annual data call was sufficient to
inform offices about their responsibility for updating READ. We informed EPA
officials that our analysis disclosed that not all offices were aware of what steps to
take when they received the annual data call. Additionally, not all EPA offices
received the data call. EPA officials acknowledged that providing more details
within the data call and making a wider distribution of the annual guidance should
result in more complete data in READ. Lastly, EPA officials indicated that READ
has the capability to create high-level data standards report cards for senior
Agency officials' use. However, our audit disclosed that these reports were
generated for OEI management and not being used to make program decisions.
EPA officials acknowledged that more could be done to create a better report and
to make it more accessible.
Data Standards Conformance Reviews Are Not Performed or Planned
The Agency cannot verify that all EPA systems conform to established data
standards. The Agency no longer performs conformance reviews for systems
required to comply with the Data Standards Policy. Without conformance
reviews or some type of measurable plan to continue verification of data
standards, management has limited assurance that common data elements are
consistent across all EPA systems.
During our review, the Agency provided a list of exchange network systems with
the most recent data standards validation reviews. However, the Agency could not
provide documentation for other systems that do not utilize the exchange network
or central data exchange. The Agency has not included all applicable systems in
its data standards conformance review process. Data standards conformance
12-P-0519
5

-------
reviews are focused on exchange network systems only, which is not in line with
the Data Standards Policy.
We analyzed information technology investment documentation for 21 Agency
systems to determine whether there are other systems that should comply with
data standards. As of December 2011, 12 of 21 major information technology
investments (57 percent) share data with two or more EPA programs. We asked
Agency management whether these systems meet the criteria for data standards
compliance. Management stated that the systems data are not significant enough
to comply with data standards. We do not agree, because this view is not in
accordance with the purpose of submitting funding request documentation for
information technology investments. According to federal requirements, agencies
should submit investment documentation for systems that support an Agency's
core mission functions. Thus, these 12 systems, and by extension, their data, merit
conformance reviews to determine whether the data conform with established data
standards and meet the Agency's needs in terms of core mission responsibilities.
Conformance reviews would further facilitate data sharing across EPA and
partner systems.
Further, according to EPA's Data Standards Policy, these 12 non-exchange
network systems should either be in compliance with data standards or have a
data standards waiver on file with Agency officials. We requested that OEI
provide the CIO-approved data standards waivers or documentation that excludes
these systems form the data standards policy. The Agency was unable to provide
such documentation relevant to the 12 non-exchange network systems. Based on
our review of the 21 Agency systems, EPA made little progress to ensure that
non-exchange network systems met data standards.
We met with OEI personnel to discuss our concerns with the data standards
corrective action plan. They shared a concept to develop data dictionaries for the
Data Element Registry Service (DERS)—EPA's registry of data dictionaries and
their data elements. Management stated that it was actively working with program
offices to develop usable data dictionaries that would be registered in DERS.
As data dictionaries are registered, OEI plans to work with program offices to
determine how data elements in the data dictionary map to EPA's standardized
data. This mapping will identify how systems conform to data standards. Unlike
conformance reviews, OEI plans to register data dictionaries and crosswalk-
related data elements with standards data elements in DERS, which will be a more
automated method of determining data conformance.
OEI confirmed that developing data dictionaries will be an ongoing and iterative
process. Management plans to prioritize data dictionary development for those
systems that are subject to CPIC and those that exchange information with other
systems. While OEI will work with all program offices, focus will be on those
systems that exchange information with EPA's partners, including states, tribes,
and industry. This effort will become EPA's method for reviewing offices' efforts
12-P-0519
6

-------
to implement data standards. OEI expects the data dictionary development for
systems and data sets to take at least 5 years. The plan OEI outlined during our
meeting lacks a concrete and measurable strategy with timelines and milestone
dates.
We met with EPA officials subsequent to the issuance of our draft report. EPA
officials indicated that data standards conformance reviews were too resource
intensive and the Agency is no longer using this approach to ensure compliance
with data standards. Management indicated that helping system owners develop
useable data dictionaries was the prevailing methodology the Agency is using to
ensure conformance with data standards. We inquired about the strategy's
objectives, planned actions, and major milestones and how the Agency would
measure its progress for achieving this new strategy. EPA officials disclosed that
while they are currently working with some offices to develop data dictionaries,
no formal strategy has been developed. Furthermore, EPA acknowledged that
more could be done to ensure data standards waiver requests were available and
that the Agency would take steps to ensure the waivers are kept within the official
record keeping system.
Conclusion
Although EPA took steps to address its corrective action plan, actions taken were
either incomplete or lacked steps to help management determine the overall
effectiveness of actions taken. EPA should take additional steps to ensure that it
fully implements data standards in existing and new systems. Such steps would
include working with program offices to develop a plan for putting data standards
in place.
Recommendations
We recommend that the Assistant Administrator for Environmental Information
and Chief Information Officer:
1.	Conduct and document a review of the data standards website to identify
areas where EPA could provide updated guidance on data standards.
Develop an action plan to produce needed information and implement a
process to annually review and update the data standards website.
2.	Develop and implement a new data standards communication plan that
outlines a structured approach for marketing and raising awareness of
EPA's data standards and identifies key Agency opportunities for
promoting data standards.
3.	Add new language to the annual data standards data call memorandum
with specific guidance that directs EPA offices to update the required data
standards fields in READ.
12-P-0519
7

-------
4.	Increase the distribution of the annual data standards data call
memorandum to all system owners.
5.	Develop and distribute a high-level data standards report card that informs
EPA regional and program office executives about their offices' progress
in complying with EPA data standards.
6.	Develop and implement a process to maintain data standards waivers
within EPA's official record keeping system.
7.	Develop a formal plan that outlines EPA's use of data dictionaries as the
methodology for facilitating compliance with data standards. The plan
should include, at a minimum, the strategy's broad objectives, major
action items, and major milestone dates.
Agency Comments and OIG Evaluation
In its February 9, 2012, response to the draft report, EPA did not agree or disagree
with the draft report's recommendations to address the findings. However, EPA
cited actions already underway, as well as new actions that it believes would help
the Agency continue to move toward greater conformance with data standards.
The Agency indicated that it would pursue data standards outreach and
communications using methods other than training. Additionally, the Agency
indicated it would disseminate the data call to all READ stewards to ensure
awareness of the requirement to update READ, and ensure the PDFs of the
current waivers are archived in the Records Repository.
We met with the Agency prior to receiving the response to the draft report to get a
better understanding of the actions management is taking to address the reported
findings and recommendations. During our discussion, the Agency provided an
outline of actions they were taking to address the report findings. We accepted
many of the proposed actions and sent the Agency revised draft report
recommendations. Subsequent to receiving the Agency's response to the draft
report, the Agency notified the OIG via electronic mail that it concurred with the
recommendations. The Agency also provided a complete corrective action plan to
address our revised recommendations. We consider these recommendations open
until the Agency completes its corrective action plan. Appendix B provides the
full text of the Agency's response to the draft report. Appendix C provides an
excerpt of the Agency's subsequent response to the draft report.
12-P-0519
8

-------
Status of Recommendations and
Potential Monetary Benefits
POTENTIAL MONETARY
RECOMMENDATIONS	BENEFITS (in $000s)
Rec.
No.
Page
No.
Subject
Status1
Action Official
Planned
Completion
Date
Claimed
Amount
Agreed-To
Amount
1
7
Conduct and document a review of the data
standards website to identify areas where EPA
could provide updated guidance on data standards.
Develop an action plan to produce needed
information and implement a process to annually
review and update the data standards website.
0
Assistant Administrator for
Environmental Information
and Chief Information Officer



2
7
Develop and implement a new data standards
communication plan that outlines a structured
approach for marketing and raising awareness of
EPA's data standards and identifies key Agency
opportunities for promoting data standards.
0
Assistant Administrator for
Environmental Information
and Chief Information Officer



3
7
Add new language to the annual data standards
data call memorandum with specific guidance that
directs EPA offices to update the required data
standards fields in READ.
0
Assistant Administrator for
Environmental Information
and Chief Information Officer



4
8
Increase the distribution of the annual data
standards data call memorandum to all system
owners.
0
Assistant Administrator for
Environmental Information
and Chief Information Officer



5
8
Develop and distribute a high-level data standards
report card that informs EPA regional and program
office executives about their offices' progress in
complying with EPA data standards.
0
Assistant Administrator for
Environmental Information
and Chief Information Officer



6
8
Develop and implement a process to maintain data
standards waivers within EPA's official record
keeping system.
0
Assistant Administrator for
Environmental Information
and Chief Information Officer



7
8
Develop a formal plan that outlines EPA's use of
data dictionaries as the methodology for facilitating
compliance with data standards. The plan should
include, at a minimum, the strategy's broad
0
Assistant Administrator for
Environmental Information
and Chief Information Officer



objectives, major action items, and major milestone
dates.
1 0 = recommendation is open with agreed-to corrective actions pending
C = recommendation is closed with all agreed-to actions completed
U = recommendation is unresolved with resolution efforts in progress
12-P-0519
9

-------
Appendix A
Agency Corrective Action Plan
(as of February 27, 2006)
Weakness: While EPA has a substantial effort in place to develop data standards and provide
guidance for their implementation, the Agency needs to establish a process for ensuring that each
data standard adopted by the Agency is fully implemented in a cost-effective and timely manner.
To address this weakness, the Office of Environmental Information (OEI) is implementing three
concurrent course of action:
Continue to communicate with EPA program offices on need to implement data standards more fully;
Continue to track program implementation of data standards; and
Continue to verify progress in implementing data standards.
Executive Summary: While the Agency has a substantive effort in place to develop data
standards and provide guidance for their implementation, incorporation of data standards in
information collections from initial plans to obtaining the data for analysis is just becoming a
routine activity in all programs. Using data standards affects data quality at each major step of
the information collection, storage and retrieval process and is a factor in the usability of data
sets. Furthermore, if applying data standards causes a program to modify its business practices,
the implementation may take several years to conduct because of modifications to data gathering,
database design and development, and interpretation of results.
CORRECTIVE ACTION MILESTONES
OEI
CONTACT
STATUS/
COMMENTS
DUE DATE
1. Continue to Communicate with EPA Program Offices on Need to Implement Data
Standards More Fully
1 A. Develop a Communication Plan to: 1)
address implementation of 15 upcoming
standards (the Institutional Controls
Standard and various other standards related
to Environmental Sampling, Analysis and
Results), 2) promote awareness of
implementation documentation including
the Implementation Strategy and various
procedures, and 3) promote use of best
practices in the implementation of specific
standards.
Dawn Banks
202-566-0625

03/2006
12-P-0519
10

-------
CORRECTIVE ACTION MILESTONES
OEI
CONTACT
STATUS/
COMMENTS
DUE DATE
1. Continue to Communicate with EPA Program Offices on Need to Implement Data
Standards More Fully
IB. Hold a System of Registry Users
Conference to include promoting awareness
and use of existing data standards
Dawn Banks
202-566-0625

12/2006 and
annually
1C. Develop and Conduct Training
on standards implementation for developers
supporting EPA program offices
Dawn Banks
202-566-0625

10/2006 and
ongoing
2. Continue to track program implementation of data standards
2A. Make Tracking Report on Data
Standards Implementation available in
Registry of EPA Applications and
Databases (READ)
Dawn Banks
202-566-0625

06/2006 and
ongoing
2B.Continue to issue semi-annual Data
Standards "Report Card"
Dawn Banks
202-566-0625

March and
September
annually
3. Continue to verify progress in implementing data standards
3 A. Report on use of data standards
validation processes in EPA Central Data
Exchange
Dawn Banks
202-566-0625

04/2006
3B. Conduct Conformance Review of
Estuary Program
Dawn Banks
202-566-0625

05/2006
3C. Conduct Conformance Review "To Be
Determined by Office of Environmental
Information Big Decisions Initiative and
Subject to Available Related Funding".
Dawn Banks
202-566-0625

TBD




12-P-0519
11

-------
Appendix B
Agency Response to Draft Report
MEMORANDUM
SUBJECT: Response to OIG Draft "EPA Data Standards Plan Completed but Additional
Steps Are Needed"
FROM: Malcolm D. Jackson
Assistant Administrator and Chief Information Officer
TO:	Rudolph M. Brevard
Director, Information Resources Management Assessments
Office of Mission Systems
Office of Inspector General
The Office of Environmental Information (OEI) appreciates the interest and emphasis placed on
data standards conformance by the Office of the Inspector General (OIG). In its draft "EPA Data
Standards Plan Completed but Additional Steps Are Needed," dated January 4, 2012, OIG
identifies six recommendations to enhance compliance with data standards by EPA system
owners. In response to a meeting between the two offices, OEI outlines below the actions already
underway, as well as new actions, which will help the Agency continue to move toward greater
conformance with data standards.
The OIG recommendations were in addition to steps OEI had already taken to meet its
commitments in response to "Implementation of Data Standards" under the Federal Managers
Financial Integrity Act (FMFIA). From 2005 to 2010, OEI introduced considerable changes,
including a data standards screen in EPA's Registry of EPA Applications and Databases (READ)
that allows system owners to self-report their data standards conformance, and a report function
in READ to track the status by office and by data standard. OEI also developed online training
modules for any individual who wishes to learn about data standards implementation.
OEI continues to support data standards. The draft "System Lifecycle Management (SLCM)
Procedure" includes a requirement for systems to implement data standards. There also is a draft
"Data Resources Policy" and its "Registration and Documenting Data Resources Procedure" that
require data be compliant with data standards.
In 2011, OEI put a registry for data dictionaries into production that will support data standards
implementation. The Data Element Registry Services (DERS) will enable OEI to map data
elements in system data dictionaries with the data elements in EPA's data standards. The
crosswalk can be kept current by updating DERS with any updates to the system data
dictionaries.
12-P-0519
12

-------
OEI is working with program offices to develop comprehensive data dictionaries to be loaded
into DERS. Developing robust data dictionaries for the more than 120 Capital Planning and
Investment Control (CPIC) systems will take several years. As data dictionaries are registered,
OEI will work with the program offices to determine which data elements in a specific data
dictionary map to EPA's standardized data elements.
Below are the six recommendations by OIG, and OEI's responses and completion dates.
OIG Recommendation 1: Require system owners and developers to complete the data standards
implementation training.
OEI Response: As agreed between OEI and OIG, Recommendations 1 and 2 are not
necessary since OEI is already pursuing data standards outreach and communications using
other methods. The Information Exchange and Services Division (IESD) within OEI has
identified increased communications as a priority and has recently established a
communications team to raise awareness of data standards as well as other IESD services.
Specifically for data standards, OEI is developing implementation guidance for each data
standard that will be posted to the data standards Webpage fwww.epa.gov/datastandardsY To
ensure currency of information, OEI also will review this site at least annually. OEI also will
discuss data standards at the upcoming Vendor Day in Spring 2012.
Completion Dates:
Vendor Day: Spring 2012
Implementation Guidance: 2013
Communications: Ongoing
OIG Recommendation 2: Develop and employ a mechanism to record and track those
individuals who have completed the training.
OEI Response: Please see the response to OIG Recommendation 1.
OIG Recommendation 3: Develop a policy to update READ during the CPIC review process.
OEI Response: The annual READ Data Call already requires that systems subject to the
CPIC process must complete their READ records, including the data standards screen. OEI's
CPIC team provides a further check by mandating that systems subject to CPIC have fully
populated READ records. The data call goes out to a wide audience, including Senior
Information Officials (SIOs) and Information Management Officers (IMOs). Going forward,
OEI will also disseminate the data call to all READ stewards to ensure awareness of this
requirement.
OEI also is developing two EPA policies and accompanying procedures, the System
Lifecycle Management Procedure under the System Lifecycle Policy and the Data Resources
Policy, which stipulate conformance to data standards.
12-P-0519
13

-------
Completion Date:
System Life Cycle Management Policy and Procedure: 2012
Data Resources Policy and Procedure: 2012
READ Data Call: June 2012
OIG Recommendation 4: Simplify the data standards report card for usability and ensure that it
is distributed to all relevant program office personnel.
OEI Response: OEI will distribute, in conjunction with the data call, office-specific Data
Standards Report Cards that highlight those systems that have indicated their status for
certain data standards as "in process". This will be an opportunity to raise awareness among
senior management of the importance of data standards.
Completion Date:
READ Data Call: June 2012
OIG Recommendation 5: Develop and execute a plan to verify system data conformance with
data standards policies for all EPA systems, including timelines and milestone dates.
OEI Response: OEI is working with program offices to develop data dictionaries. This effort
will take many years as developing a robust data dictionary is time consuming and requires
significant communication with system owners. Additionally, data dictionaries vary widely
in size and complexity. For these reasons, OEI has an internal target of seven data
dictionaries per year in lieu of a specific plan identifying systems and milestones.
OEI also is formalizing its standard operating procedures for the DERS, which will include
steps for obtaining and developing data dictionaries.
Completion Date:
Data Dictionary Development: Seven data dictionaries per year.
DERS SOP: August 2012
OIG Recommendation 6: Document and maintain data standards waivers for all EPA national
systems to comply with EPA's Data Standards Policy.
OEI Response: There is a process for documenting and maintaining data standards waivers.
The READ Report Card provides information about systems that have requested waivers and
the status of the waivers. OEI will ensure the PDFs of the current waivers are archived in the
Records Repository.
Completion Date:
Waivers archived in Records Schedule: February 2012
Waivers recorded in READ: Done
12-P-0519
14

-------
In conclusion, OEI promotes and facilitates data standards conformance and will continue to do
so. We believe we have an effective approach to managing data standards conformance while
providing service to the program offices. Thank you for bringing ideas for how to improve
outreach and value to this program through this assessment. For any questions, please do not
hesitate to contact John Harman at (202) 566-0748.
cc: Renee Wynn
Andrew Battin
Connie Dwyer
12-P-0519
15

-------
Appendix C
Subsequent Agency Response to Draft Report
To: Rudy Brevard/OIG/USEPA/US@EPA
From: Jeffrey Wells/DC/USEPA/US
Date: 05/22/2012 07:19PM
Subject: Re: OIG Recommendations on Data Standards Conformance
Hi Rudy,
We concur with OIG's recommendations.
Thank you very much.
Jeff
Jeff Wells
Acting Deputy Director
Office of Information Collection
Office of Environmental Information
U.S. Environmental Protection Agency
From:	Jeffrey Wells/DC/USEPA/US
To;	Rudy Brevard/OIG/USEPA/US@EPA
Cc:
Date:	05/21/2012 06:12 PM
Subject:	OIG Recommendations on Data Standards Conformance
Hi Rudy,
I'm responding for Andy Battin and following up on an email you sent him asking for OEI's
response to the seven recommendations on our Data Standards program that your group
identified to us.
Here is the response from our office as requested. It is my understanding that our team has
previewed these responses with you and your team already - but that you were looking for an
email from us to codify them formally in response to the recommendations. I understand that the
next step is for OIG to issue the final report, which will include the seven recommendations
listed below. OEI will then respond within 90 days with the responses you see below in italics
along with implementation dates (see below) that address these recommendations. That, as I
12-P-0519
16

-------
understand it, is the last step in the process.
Here, below are the seven OIG recommendations along with our proposed responses and
dates for completion.
OIG Recommendations and OEI Responses
1.	Conduct and document a review of the Data Standards website to identify areas where EPA
could provide updated guidance on Data Standards. Develop an action plan to produce needed
information and implement a process to annually review and update the Data Standards website.
OEI will include the actions to review the Data Standards website annually in the Data
Standards Communications plan; July 2012
2.	Develop and implement a new Data Standards communication plan that outlines a structured
approach for marketing and raising awareness of EPA's data standards and identifies key
Agency opportunities for promoting data standards.
OEI will update its data standards communications plan, which will identify steps for creating
increased awareness of EPA data standards; July 2012
3.	Add new language into the annual Data Standards data call memorandum with specific
guidance that directs EPA offices to update the required data standards fields in READ.
OEI will include new language to the READ Data Call that directs EPA offices to update data
standards conformance status on the data standards screen in READ; June 2012
4.	Increase the distribution of the annual Data Standards data call memorandum to all system
owners.
OEI will disseminate the READ Data Call to all Information Management Officers for
distribution to system owners; June 2012
5.	Develop and distribute a high-level data standards report card that informs EPA regional and
program office executives about their office's progress in complying with EPA data standards.
OEI will issue the high-level data standards report card as part of the READ Data Call; June
2012
6.	Develop a formal plan that outlines EPA's use of data dictionaries as the methodology for
facilitating compliance with data standards. The plan should include at a minimum the strategy's
broad objectives, major action items, and major milestone dates.
OEI will develop a Standard Operating Procedure (SOP) for developing data dictionaries. It
will indicate that OEI intends to develop at least seven data dictionaries per year - focusing on
systems that are subject to Capital Planning and Investment Control (CPIC); September 2012
12-P-0519
17

-------
7. Develop and implement a process to maintain data standards waivers within EPA's official
record keeping system.
OEI will maintain all waivers in READ and register all accepted waivers in the EPA Records
Schedule. OEI will add a sentence to the Data Standards screen in READ that reads: "If an
EPA Data Standard is not listed, the system owners have made the determination that the
standard is not applicable to his /her system August 2012	
Source: Wells, Jeffrey. Acting Deputy Director of the Office of Information Collection, Office of
Environmental Information, responding for Andrew Battin, Director of the Office of Information
Collection. Excerpt from electronic correspondence. Subject, "OIG Recommendations on Data
Standards Conformance," May 21, 2012 and May 22, 2012
12-P-0519
18

-------
Appendix D
Distribution
Office of the Administrator
Assistant Administrator for Environmental Information and Chief Information Officer
Agency Follow-Up Official (the CFO)
Agency Follow-Up Coordinator
General Counsel
Associate Administrator for Congressional and Intergovernmental Relations
Associate Administrator for External Affairs and Environmental Education
Director, Office of Information Collection, Office of Environmental Information
Audit Follow-Up Coordinator, Office of Environmental Information
12-P-0519
19

-------