*«*
.Oox
oovovicnouoo.ouooio.
3VOioooiooiouooiooio.io..
U.S. Environmental Protection Agency
Open Government Data Quality Plan 1.0
^N
EPA@40: Healthier Families, Cleaner Communities, A Stronger America
May 18, 2010
-------
v\\
Open Government Plan \V
\ \
EPA Open Government Data Quality Plan
Table of Contents
I. Introduction 1
II. Implementation of the Data Quality Framework 2
A. Governance Structure 2
1. Open Government Work Group 2
2. Open Government Management Work Group 3
3. Open Government Project Teams 3
4. Open Government Spending Information Quality Work Group 4
B. Risk Assessment 4
C. General Governing Principles & Control Activities 6
D. Com m unications 7
E. Monitoring 8
III. USASpending.gov Data 12
A. Com pilation 12
B. Steps for compiling and reporting the data 12
C. Time elapsed 13
D. Review 14
E. EPA's process to ensure consistency of Federal Spending Data 14
F. EPA's process to ensure completeness of the Federal spending information.... 15
G. Monitoring 15
1. EPA's Internal Metrics 15
2. Deficiencies Identified 16
IV. Glossary 18
V. Appendix 1: Data Quality Assurance Check-List Instructions 20
VI. Appendix 2: Data Quality Assurance Check-List 21
VII. Appendix 3: FPDS Data 22
Nothing in this plan shall be construed to supersede existing requirements for review and
clearance of pre-decisional information by the Director of the Office of Management and Budget,
relating to legislative, budgetary, administrative, and regulatory materials. Moreover, nothing in
this plan shall be construed to suggest that the presumption of openness precludes the
legitimate protection of information whose release would threaten national security, invade
personal privacy, breach confidentiality, or damage other genuinely compelling interests.
-------
II
Open Government Plan
\\\
\
i.
Introduction
The Administration's Open Government Directive directs Federal agencies to promote
transparency, participation, and collaboration. While the Environmental Protection Agency
(EPA) has embraced these tenets since our creation 40 years ago, EPA's recent efforts have
resulted in landmark progress. The EPA Open Government Team has taken numerous steps to
promote Open Government as well as solicit valuable feedback from a variety of stakeholders
using technological as well as traditional feedback tools. On April 7, 2010, EPA released our
Open Government Plan 1.0 to emphasize and expand EPA's culture of openness with a
mission-focused framework for engaging our stakeholders and the public. The Plan, which is a
dynamic document to be updated every 180 days, also presents EPA's Community
Engagement Flagship initiative, a broad spectrum of activities intended to promote a greater
understanding of, participation in, and contribution by individuals regarding the environmental
issues that affect or interest them most.
Enhanced transparency requires making more data available to our stakeholders and the public.
However, more information is not helpful if the data is not current or of high quality. Toward that
end, EPA collects and shares reliable, quality data to help carry out the Agency's mission. This
commitment is not new, and its scope continues to grow over time. In 1979, Administrator
Costle initiated the first Agency- wide quality management system for data and information.
Furthermore, EPA's leadership, in implementing the Information Quality Act of 2001, issued the
Agency's Quality Policy (CIO Policy 2106) to expand the quality management of data,
information, products, and services to meet the challenges of openness and transparency in
agency operations. EPA's ongoing commitment to sharpening our focus on public access to
high-value data is reflected in this Plan.
Sound spending data is necessary to inform sound spending decisions. The Open Government
Directive emphasizes the specific goal of ensuring that objective and high-quality financial
spending information is available, particularly as the Administration focuses on economic growth
and reinvestment resources. While EPA's Financial Statements have received clean audits
annually for over a decade, our commitment does not end there. We are committed to
proactively enhancing the quality of the Agency's financial spending information by leaning on
our management integrity experience to provide context for ensuring the quality of EPA
spending information. This plan establishes the framework to assess the risks to, and control
environment over, Agency data submitted to USASpending.gov, and recommend improvements
as necessary.
EPA's Chief Financial Officer serves as the Agency's Senior Accountable Official (SAO) to
ensure spending data integrity. EPA established an Open Government Spending Information
Quality Work Group consisting of representatives from the Office of Administration and
Resource Management (OARM), Office of Environmental Information (OEI), and Office of the
Chief Financial Officer (OCFO). Led by OCFO, the work group prepared this plan, which is a
companion document to the Agency's overarching Open Government Plan 1.0. While the SAO
is accountable for spending data quality and the CIO is accountable for general data quality,
both will continue to work in parallel to ensure the quality of all high-value Agency data.
-------
Implementation of the Data Quality Framework
A. Governance Structure
Figure 1
EPA Administrator and
Deputy Administrator
Open Government
Work Group
Open Government
Management Work Group
Open Government
Project Teams
(Website restructuring,
High-value datasets.
Open Government plan)
Open Government
Spending Information
Quality Work Group
The Agency has created an overall Governance Structure (Figure 1) for the President's Open
Government Directive. An executive level work group serves as a steering committee to guide
the development of the two prongs of the Agency's Open Government Plan: the overall Open
Government Plan, and the Spending Information Quality Plan. An Open Government Spending
Information Quality Work Group (SIQ Work Group) has been established to develop the
Spending Information Quality Plan required by OMB's December 8, 2009 guidance to ensure
the quality of Agency spending information.
1. Open Government Work Group
The Open Government Work Group is made up of Senior Executive Service (SES) employees
from key responsible offices within the Agency. This group is charged with setting policy and
direction while meeting legal requirements. The group also works to secure resources in support
of the Agency's plan and associated projects. The following offices are represented on the
committee:
• Office of the Administrator (OA).
Office of Public Affairs (OPA) (Allyn Brooks-LaSure, Principal Deputy Associate
Administrator).
Office of Policy, Economics and Innovation (OPEI) (Louise Wise, Principal Deputy
Associate Administrator).
• Office of Administration and Resources Management (OARM) (Susan Hazen, Deputy
Assistant Administrator).
-------
• Office of the Chief Financial Officer (OCFO) (Maryann Froehlich, Deputy Chief Financial
Officer).
• Office of General Counsel (OGC) (Pat Hirsch, Associate General Counsel).
• Office of Environmental Information (OEI) (Linda Travers, Acting Chief Information Officer).
The Open Government Work Group regularly reports to the Deputy Administrator, the Deputy
Assistant Administrators and the Deputy Regional Administrators of most offices within the
Agency. This group provides feedback and support to the Open Government Work Group,
ensuring the entire Agency is represented and involved. Representation includes all offices
listed above, as well as the following offices:
• Office of Air and Radiation (OAR).
• Office of Administration and Resources Management.
Office of Grants and Debarment (OGD).
Office of Acquisition Management (OAM).
• Office of Enforcement and Compliance Assurance (OECA).
• Office of Chemical Safety and Pollution Prevention (OCSPP).
• Office of Research and Development (ORD).
• Office of Solid Waste and Emergency Response (OSWER).
• Off ice of Water (OW).
• Regional Offices.
2. Open Government Management Work Group
This work group consists of senior managers and staff from across the Agency, including
regional representation. It meets on a weekly basis to develop actions, provide input and track
progress.
3. Open Government Project Teams
Three project teams currently operate as a subset of the management work group. Each team
has a specific task that supports open government.
• Website Restructuring Team: Charged with restructuring EPA's Web site based largely upon
public feedback.
• High Value Datasets Team: Charged with identifying and prioritizing high-value datasets for
future addition to Data.gov, incorporating public feedback.
• Open Government Plan Team: Charged with coordinating Open Government Plan
development, managing public input to the plan, monitoring plan progress, and coordinating
updates to the plan.
-------
Figure 2
EPA Administrator
and Deputy
Administrator
CIO
(Open Gov Plan)
CFO/SAO
(Spending Info Quality)
Open Gov —
Group
(Executive steering
mmmittee)
ing
Quality Work
Group
Open Gov
Management
Work Group
Open Gov
"oject Teams
4. Open Government Spending Information Quality Work Group
The Open Government Spending Information Quality Work Group is charged with developing
this plan to assure quality and integrity of EPA spending information. The work group will also
work towards integrating the quality plan into a long-term Agency strategy that ensures the
integrity of all federal spending information. The work group is modeling its approach using A-
123 principles.
The SIQ Work Group (Figure 2) is comprised of senior level personnel from the Office of
Financial Management, Office of Environmental Information Quality Staff, the Office of
Acquisition Management, and the Office of Grants and Debarment. The SIQ Work Group
reports to the Chief Financial Officer who serves as the Senior Accountable Official for spending
information. The SIQ Work Group seeks guidance from the executive level Open Government
Work Group.
B. Risk Assessment
The Agency is building its risk assessment plan around the Government Accountability Office's
(GAO) five principles for assessing risk1. Furthermore, we are incorporating risk assessment
practices identified under the Agency's A-123 process. The GAO standards are:
• Identifying threats and the likelihood of those threats materializing.
GAO/AIMD-OO-33, "Information Security Risk Assessment: Practices for Leading Organizations," November 1999.
-------
• Identifying and ranking the value, sensitivity, and criticality of the operations and assets that
could be affected should a threat materialize in order to determine which data sets and
assets are the most important.
• Estimating potential damage.
• Identifying cost effective mitigating controls. These actions can include implementing new
organizational policies and procedures as well as technical or physical controls.
• Documenting assessment findings and developing action plans in an effort to circumvent
future threats.
Because reliable data on probability and costs are not currently available, we have decided to
take a qualitative approach by defining risk in more subjective and general terms such as high,
medium, and low. EPA's qualitative assessments depend on the expertise, experience and
judgment of those conducting the assessment.
Critical Factors for Our Success
• EPA has engaged senior management support and involvement. Briefings have been held
with the Deputy Administrator and Deputy Assistant Administrators in most program offices.
• Designated focal points are the Office of the Chief Financial Officer (OCFO)ADffice of
Financial Management (OFM) and the Office of Environmental Information (OEI).
• Defined procedures:
OCFO /OFM is responsible for initiating and conducting risk assessments over data
sets.
OFM and OEI Quality Staff will participate in the assessments.
A comprehensive check sheet was developed by OFM and OEI Quality Staff
(Appendices 1 and 2) to review spending data to determine if the information is accurate
and sufficient to post. This tool will help ensure a standardized approach throughout the
organization. Meetings conducted during each risk assessment process will include
individuals from the program and administrative offices with expertise in business
operations and processes, security, information resource management, information
technology and operations.
Disagreements, if they occur, will be resolved at the lowest level possible, with a final
determination made by the Senior Accountable Official (SAO) if needed.
All new spending data sets will be certified as accurate and complete to the EPA's SAO
before posting to the Internet.
Current USASpending.gov datasets will be assured on a quarterly basis to be accurate
and complete to the SAO by the Assistant Administrator (AA), OARM. This assurance
will be supported by certifications from the regional and headquarters Grants
Management Officials and Contract Offices to the National Grants and Contracts
Program Managers in OARM who will then assure to the AA, OARM.
All additional datasets sent for posting will have AA assurances from the appropriate
region or program office to the SAO. Data sets that are not static and are updated will
require quarterly certifications from the appropriate AA.
Completed check sheets will be maintained by OCFO/OFM. Risk assessment results will
be documented and maintained so that managers can be held accountable for the
-------
decisions made and a permanent record can be established. In this way, risk
assessment records will be available to serve as the starting point for subsequent risk
assessments and as a ready source of useful information for managers new to the
program. OFM will establish a database where all risk assessments will be stored.
OFM will provide reports on a quarterly basis to the SAO on activity and the results of
any performed risk assessments.
Current Risk
The Information Quality Guidelines (IQG or the Guidelines) contain EPA's policy and procedural
guidance for ensuring and maximizing the quality of information we disseminate. Pre-
dissemination reviews are designed to provide additional assurance that information EPA
disseminates to the public is consistent with the quality principles described in the IQG. Agency
Program Offices, Regions and Labs ("Agency Offices") should develop pre-dissemination review
procedures consistent with the Guidelines and their respective authorities and activities.
Given the wide variety of activities within EPA that generate information, EPA's challenge is to
identify this information as early as possible so that systematic planning can be integrated into
the development process, and quality incorporated during development. When information is
created, its intended use(s) can determine whether the information will be "disseminated" and
therefore should undergo pre-dissemination review. The process of document approval can be
quite complex. Multiple forms and mechanisms for review and approval of Agency documents
exist. Some examples are legal reviews, peer reviews, programmatic reviews, scientific and
technical review clearance processes, stakeholder reviews, and product review in accordance
with the Office of Public Affairs guidelines. Additionally, some approval processes call for the
incorporation of pre-dissemination review from the concept stage, such as Quality Assurance
Project Plan reviews and approvals under the Quality Management Plan.
The two risks we were able to identify were the possibility that information would be used for
purposes it was not designed for and the possibility, although slight that a key control could fail
and information was posted that had not been pre-disseminated reviewed. The implementation
of the Spending Data Quality Plan should help reduce these risk factors.
Benefits of this Plan
• Identifying and properly addressing the greatest levels of risk.
• Increased understanding of risks related to the posting of spending information.
• Promoting consensus building to give all involved parties an understanding of the impact of
faulty data.
• Acquiring support for needed controls and assurances.
• Implementing effective communication tools.
C. General Governing Principles & Control Activities
OEI will provide OCFO/OFM with all new data sets containing spending data. Subsequently,
OFM will perform a risk assessment and data quality/assurance review before posting the
dataset.
-------
Policy
EPA has a strong quality assurance process including EPA CIO Policy 2106.0, Quality Policy.
The scope of this Quality Policy includes Agency products and services that are developed for
external distribution or dissemination. EPA organizations may consider the application of this
Policy to other products and services developed for internal use. Agency products and services
encompassed by this Policy include, but may not be limited to: environmental data and
technology; environmental information; assessments; reports; peer review; scientific research;
regulatory development; and information technology supporting external applications such as
services that collect and provide access to data and information.
While affirming EPA's commitment to quality, this Policy recognizes new and emerging factors
influencing the need for ongoing assurance and continual improvement of the quality of the
Agency's products and services. Moreover, this Policy recognizes that the expanded and
consistent application of such quality management principles and practices in all areas of the
Agency's operations is a desirable business objective.
The accompanying Procedure for Quality Policy supports the implementation of EPA's Quality
Policy2. The Quality Policy establishes management principles and responsibilities for ensuring
that EPA products and services meet Agency quality-related requirements, are of sufficient
quality for their intended use and support EPA's mission to protect human health and the
environment. The Procedure describes Policy implementation processes and the governance
of EPA's Quality Program. It is intended to provide a comprehensive, coordinated approach for
consistent implementation of the Quality Policy to ensure the continual improvement in the
quality of EPA's products and services. The quality tools and processes described in the
Procedure are based on national and international consensus standards.
Additionally, EPA is currently updating the policy on Maintenance of the Public Access Site
(IdeaScale). This update is due at the end of June.
1. Contracts: Office of Acquisition Management
The Office of Acquisition Management's (OAM's) general governing principles and control
activities related to the quality of contract/data information are an integral part of the Agency's
existing acquisition internal control procedures, practices, and oversight program. They can be
found in the Acquisition Handbook (Section 4.2, "Quality Assessment Plans"), Contract
Management Manual, Federal Acquisition Regulation (FAR), EPA Acquisition Regulation
(EPAAR) and various OMB guidance documents. The acquisition oversight program is a three-
tiered approach (see Monitoring below).
2. Grants: Office of Grants and Debarment
The Office of Grants and Debarment's (OGD's) governing principles and control activities are
reflected in operating procedures, system and internal controls, training and performance
requirements and monitoring practices.
D. Communications
In developing our Open Government Data Quality Plan, we actively sought EPA participation.
Updates and changes to the Open Government Data Quality Plan will be communicated via the
Open Government Website. Quarterly metrics on the timeliness, completeness and accuracy of
Procedure for Quality Policy available at http://www.epa.qov/qualitv/qa docs.html#qmanual
-------
EPA's spending data will be made available to the public by the Office of Management and
Budget (OMB) on a public dashboard.
E. Monitoring
1. Contracts
EPA's acquisition oversight program, which includes data quality, is a three-tiered approach:
i. Each of the thirteen major EPA contracting offices is responsible for overseeing the
quality of its acquisition operations through its Quality Assessment Plan (QAP). Each of
the QAPs contain internal control and integrity requirements to ensure that acquisition
rules and regulations are followed, files and records are kept to document the acquisition
processes and procedures, and sufficient efforts are taken to perform pre- and post-
award reviews of transactions to make sure that controls are working as intended3.
Each QAP also directly places the responsibility for data quality and integrity on the
contract specialists (CSs) and contracting officers (COs) within the office. The CSs and
COs are required to perform annual reconciliations of data between EPA's contract-
writing systems. The current systems involved in the procurement process are the
Integrated Contract Management System (ICMS, a contract writing system) and the
Small Purchases Electronic Data Interchange (SPEDI, a contract writing system); the
Integrated Financial Management System (IFMS, the Agency's official accounting
system); and the Government-wide Federal Procurement Data System-Next Generation
(FPDS-NG), which works between ICMS and SPEDI . Please note that
USASpending.gov and Recovery.gov4 both receive EPA contracting data from FPDS-
NG. We do not transmit data directly to either of those systems5.
ii. Employees within OAM's Policy, Training and Oversight Division (PTOD) perform
extensive independent reviews of each contracting office's compliance with its QAP.
QAP Compliance Reviews are performed in each contracting office at least once every
three years, or more frequently in cases where offices have failed to maintain consistent
implementation of their plan. One of the areas covered in the compliance reviews is the
quality and integrity of contracting data, as outlined in each specific QAP.
iii. OAM and contractor support have performed A-123 assessments of various aspects of
acquisition in the OAM Divisions since FY 2006. These reviews include assessments of
the processing, awarding, and obligating of funds via contracting and simplified
acquisition procedures, management of contract property, and the use of Government
purchase cards. The reviews consist of identifying financial-related internal controls and
testing acquisition transactions to determine the efficiency and effectiveness of the
controls. Data accuracy is included in the reviews. OAM has also, for the past two
years, performed entity-level internal control reviews of the Agency's acquisition
program, as defined by OMB guidelines6 and GAO's four cornerstones7. Information
management is one of these cornerstones.
3 The general requirements of a QAP are specified in Section 4.2 of the Agency's Acquisition Handbook
4 ARRA contracting data only
5 Additionally, please note that EPA submitted its FY 2009 FPDS-NG Data Quality Plan to OMB/OFPP on January 8,
2010, and most of this Open Government Data Quality Plan comes directly from our previously-submitted FPDS-NG
Data Quality Plan.
6 Specifically "Conducting Acquisition Assessments Under OMB Circular A-123," dated May 21, 2008
8
-------
In addition to the three-tiered oversight program, OAM also requires that data quality be
included in the Performance Appraisal and Recognition System (PARS) plan for all COs, CSs,
and senior contracting personnel at EPA, under Critical Element 2 - Business Acumen/Process
Management. OAM also distributes quarterly reports on year-to-date acquisition data and
accomplishments to program customers and the Office of Small and Disadvantaged Business
Programs, who review the accuracy of contract data and provide valuable feedback to OAM.
This is another oversight tool OAM uses to ensure data quality.
Additionally, OAM annually performs a verification and validation of its FPDS-NG data. The
results of OAM's latest review were submitted to OMB on January 8, 2010. In support of this
effort, basic FPDS-NG information is populated by EPA's contract writing system, which assures
master contract information forwarded to individual contract actions is accurate and consistent.
Additionally, all FPDS-NG reports are subject to a second level review by the Contracting
Officer. This approach and monitoring enables EPA to have confidence in reported FPDS data.
The review of FY 2009 data just completed is the first time an independent contractor has
performed this exercise. We intend to use the results (Appendix 3) as a baseline for future
accuracy rates of the reviewed data elements. Please note that, while the overall accuracy was
91.5% for the 24 elements reviewed (at a 95% confidence level with a 5% error rate, meaning
that we are 95% confident that the overall accuracy for the 24 reviewed fields is between 86.5%
and 96.5%), the elements containing financial information (Base and All Options Value, Base
and Exercised Options Value, Action Obligation) approach 100% in accuracy.
The only information data sets currently available to the public involving EPA contracting data
are found on FPDS-NG, USASpending.gov, and Recovery.gov8. Please note that the
Recipient-reported ARRA data in Recovery.gov is input by EPA contract recipients and not EPA
employees. EPA/OAM are not aware of any upcoming additional data sets that will be made
available within the next six months.
2. Grants
At EPA, grants management is a shared responsibility between the Program Offices responsible
for programmatic management of grants and the Grants Management Offices responsible for
the administrative management of grants. EPA's quality monitoring program reflects this dual
responsibility.
EPA uses a variety of tools to support an overall quality approach including internal
assessments, staff requirements, audits, and system controls.
i. Monitoring and Oversight Policies
EPA has a comprehensive five-year Grants Management Plan (GMP)
http://www.epa.qov/oqd/EO/finalreport.pdf which establishes, among other things, the
strategic direction for monitoring and oversight. In implementing the GMP, the Agency
follows well-defined policies and guidance for cradle-to-grave grants management to
enhance transparency, accountability and results.
The Agency has established an internal review process to assess compliance with
established policies and processes, including those affecting data quality, in offices that
award and monitor grants. As described below, the reviews are conducted to ensure
7 GAO, Framework for Assessing the Acquisition Function at Federal Agencies." Four cornerstones: (1)
organizational alignment and leadership, (2) policies and processes, (3) human capital, and (4) knowledge and
information management.
8 ARRA contracts only
-------
that offices are following proper grant award and grants management procedures as well
as meeting EPA and federal requirements.
a. Comprehensive Performance Reviews (CPR) are conducted to evaluate compliance
with, and measure the effectiveness of EPA grants management policy, procedures
and other requirements. These reviews are typically coordinated by staff in OGD's
National Policy, Training and Compliance Division (NPTCD). Historically, the
reviews may be broad (covering a range of policies and requirements) or narrow
(covering a specific aspect of grants management). Execution of these reviews may
include the development of a CPR protocol, pulling a statistically valid sample of
files, review of selected files and databases, creation of EPA and office-specific
reports and presentation of findings to EPA staff and management.
Multiple CPRs may take place during a year, based upon the need, commitments,
costs and opportunities presented.
b. Grants Management Self-Assessments (GMSA). This review is conducted by the
Headquarters Program offices and Regional Grants Management Offices to evaluate
their own grants management performance, identify key issues (including data
issues), and develop corrective action plans. Generally, GMSAs are completed
every three (3) years.
NPTCD is responsible for developing and issuing guidance prior to each cycle. This
explains how the self-assessments are to be conducted. In the guidance, there is a
list of review topics that represent issues identified by EPA, the Government
Accountability Office or by other reviews. Also included in the guidance are
recommended protocols, sampling suggestions and additional items necessary to
the effective completion of the GMSA. After the review is complete, the EPA office
submits the report to the Director, NPTCD. The report must come directly from the
Senior Resource Official or equivalent.
c. Post Award Management Plan (PAMP). This review is conducted by the
Headquarters Program Offices and Regional Grants Management Officers to
evaluate their post-award management performance and identify key issues.
Generally, PAMPs are completed annually and list the awards proposed for on-site
and off-site advanced monitoring reviews. Typically, the number of reviews must be
at least ten percent (10%) of active recipients of an office or region. Occasionally,
new programs that require additional stewardship and monitoring (e.g. American
Recovery and Reinvestment Act) will necessitate a deviation from or modification to
the PAMP requirement.
The PAMP includes a review and summary of advanced monitoring activities
including trends, significant findings and corrective actions. It also documents how
details of the assessment will be communicated and shared with EPA grants
management staff. EPA offices submit their plans to the Director of NPTCD. The
plan must come from the Senior Resource Official or equivalent.
EPA offices also conduct two types of in-depth reviews on individual grants, including
baseline monitoring and financial reviews.
d. Baseline Monitoring is done at the level of the individual grant. It is a snapshot of the
project at the time the review is conducted. It follows standard protocols (one
administrative and one programmatic) and must be completed in a central database
by EPA Grant Specialists and Project Officers, respectively. EPA staff performs
baseline monitoring annually for each active award. The reviews include ensuring
10
-------
recipient compliance with applicable terms and conditions and Federal grant
regulations. If issues are identified during the review, EPA staff work with recipients
to resolve the issues and/or refer the findings to the appropriate management official
for further action before the report is closed.
e. Financial Reviews. In partnership with our Las Vegas Finance Center, payment
requests are reviewed for status within project period and for potentially excessive
draw. Spending patterns are reviewed for unusual drawdown activity and grant
payment reports are run to monitor spending percentages by grant as related to
project period. The Finance Center works directly with EPA administrative and
program staff to resolve any issues with our grantees to achieve project goals.
Staff Requirements
OGD annually issues guidance for assessing grants management and the management
of interagency agreements under the Performance Appraisal and Recognition System
(PARS). Performance agreements are also expected to contain appropriate references
to grants and interagency management responsibilities, including compliance with
applicable Agency policies.
Training requirements have been defined for Grants Specialists (GS), Project Officers
and Managers who are responsible for entering grant data in the Integrated Grants
Management System (IGMS) or who approve the documents containing the data.
A-123 Audit of Active Grants
OGD's Grants and Interagency Agreements Management Division (GIAMD) completes
an annual A-123 audit of a statistical sample of active grants in which key controls of the
grants review process are identified and tested. The key controls identify the
responsible source for data input and quality through each phase of the award review.
At the beginning of the award review, IGMS limits access based on assigned job roles
and responsibilities through Access Control Lists and membership in Organization
Control Groups (Key Control #1). Based on IGMS access limits, the Funds Certifying
Officer is the only person that has the ability to certify funds in IGMS (Key Control #1).
Once the programmatic and administrative reviews are complete, the Award Official
approves the grant in IGMS by applying a digital signature (Key Control #2). The Award
Official and their Delegate are the only people who have the ability to digitally sign an
Award in IGMS (Key Control #1). When the award is returned to GIAMD, the Grants
Specialist or the qualified Administrative Assistant enters the affirmation of the award
from the grantee into IGMS (Key Control #3).
During the post-award phase, the Award Official's approval is required in IGMS for
amendments with increases in funding (Key Control #4). At the closeout phase of a
grant, the Award Official or qualified Grants Specialist at the GS-9 level or above is
required to review and sign the closeout package to financially and administratively
closeout a grant (Key Control #5). Finally, the unliquidated balances on active grants
are researched on an annual basis by the Grants Specialist through the review of a
report generated from the Financial Data Warehouse (FDW) in the spring of each year.
The report reflects grants that have had an inactive balance for over 180 days.
Annually, the OGD Office Director signs the Review of Unliquidated Obligations Year-
End OGD Certification and submits it to OCFO/OFM to attest that the unneeded grant
funds identified in the annual review were deobligated in IFMS (Key Control #6).
11
-------
iv. System Controls
The source system for grants data is IGMS. IGMS has edits to ensure that key fields are
completed, multiple choice lists that include valid options (e.g. current Catalog of Federal
Domestic Assistance (CFDA) numbers), and a centralized table of recipients and
address information. In addition to the field level controls, IGMS supports management
routing and review of grant awards and documents and dates their approval.
III. USASpending.gov Data
A. Compilation
EPA has two main subcategories of awards: contracts and grants.
1. Contracts
OAM reports any and all contract actions which are collected by FPDS-NG. These include new
contracts, modifications, delivery or task orders, blanket purchase agreements, and simplified
acquisitions/purchase orders.
2. Grants
EPA awards mandatory grants, discretionary grants, cooperative agreements, and
fellowships. EPA reports on all grants awarded, including foreign awards and grants under
$25,000, except fellowships. Since EPA has been informed that individual awards are to be
transmitted to USASpending.gov, the Agency will begin reporting fellowships in the next
reporting period. EPA collects county data for the address of the individual fellow, however,
EPA does not collect place of performance data for fellowship awards. In most cases, reporting
individual aggregate awards by county will, in reality, be reporting single awards since very few
fellowship awardees are from the same county. EPA will incorporate the reporting of aggregate
individuals in subsequent USASpending.gov data submissions using the fellow's county.
B. Steps for compiling and reporting the data
1. Contracts
For contracts, all EPA reported/USASpending.gov required data is submitted to
USASpending.gov via FPDS-NG. The Treasury Accounting Symbol (TAS) used is the one
associated with the accounting line with the predominant amount of funding. Since FPDS-NG
does not have the capability to capture or collect data for each line of accounting, funding from
the lines of accounting are rolled up and reported as a total for the contract action submitted.
There could be multiple TAS codes involved, but only one may be reported in the FPDS-NG.
EPA's Contract Writing Systems (CWSs), ICMS & SPEDI, report their contract data machine-to-
machine to FPDS-NG. For the required data fields in FPDS-NG that are not filled out by the
CWSs, the User accesses the FPDS-NG web interface to fill in the incomplete fields.
FAR 42.505 states "Government representatives must recognize the lack of privity of contract
between the Government and subcontractors." Accordingly, the Government contracting
community has little interaction with sub-contractors. To the extent the forthcoming OMB
guidance contains a clause requiring prime contractors to report 1st tier subcontract awards,
OAM will include the clause in awarded contracts (as prescribed). However, given the
regulatory guidance regarding contract privity, OAM will not be in a position to independently
12
-------
review, comment, or confirm the quality of any subcontractor data entered by a prime
contractor.
2. Grants
Submission of USASpending.gov grants data is a multi-step process. The first step is the
extraction of a data set from the Grants Data Mart for submission to Dun and Bradstreet for
review. The extract is adjusted to match submission format and the data manually checked for
completeness. The resulting fixed width text file is submitted to Dun and Bradstreet using an
on-line web tool. Upon completion of the review, Dun and Bradstreet notify EPA that the data,
with the required D-U-N-S confidence code, is available for download from the web tool.
Next, the USASpending.gov dataset is generated via a query against the Grants Data Mart. The
data is formatted according to USASpending.gov specifications and manually checked for
completeness and required format. Missing or incorrect data is corrected and entered before
submission. The file is saved as a fixed width text file and submitted via email to
USASpending.gov.
EPA submits TAS information with the grants submission to USASpending.gov. Since
USASpending.gov can only accommodate one TAS code for each transaction, the total funding
for each transaction with multiple TAS codes is associated with the TAS code for the primary
line of accounting for the grant transaction.
C. Time elapsed
1. Contracts
In accordance with the updated requirements of the Federal Funding Accountability and
Transparency Act (FFATA), the TAS information is loaded on the USASpending.gov site on the
5th and the 20th of each month (or the prior work day if it falls on a weekend).
For contracting actions prepared in SPEDI, the associated record in FPDS-NG is finalized
simultaneously with the signature (the action must be logged out/finalized prior to a paper
document being printed for signature. This document is normally signed immediately after
printing.)
For contracting actions prepared in ICMS, the associated record in FPDS-NG is finalized after
signature when the CO goes back into the system (ICMS) and checks-in/finalizes the action.
The time frame for this to happen after signature varies depending on the due diligence of the
CO, but a request has been made for 5 business days. In addition, ICMS has been set up to
send reminder e-mails on the 1st and 15th of each month to the CO and CS of record in the
system for each action that has been reported to FPDS-NG in draft. After the close of the fiscal
year, these reminder e-mails are sent out at a frequency of every 4 calendar days until all the
actions under prior fiscal years are completed.
Once the contracting data has been loaded into FPDS-NG, EPA does not know the transaction
time for data from FPDS-NG to be loaded into USASpending.gov.
2. Grants
EPA grants data is reported monthly to USASpending.gov. The first week of each month the
data from the previous month's grant awards are compiled, submitted for Dun and Bradstreet
review and subsequently, submitted to USASpending.gov. While the actual date of submission
varies depending on Dun and Bradstreet review and other factors, the data is always submitted
to USASpending.gov prior to the 20th of the month.
13
-------
D. Review
1. Contracts
Data is entered into the EPA CWSs and FPDS-NG by either a CS or a CO.
In SPEDI, only a CO with System Rights granted by their Management can log out/finalize an
action (and thus finalize the action in FPDS-NG, as well) prior to signature. If a CS entered the
data, the CO would review the data prior to log out. If a CO entered the data, they would have
knowledge of what they entered and have the authority to log themselves out. This process
allows the CO to ensure that the data entered passes all FPDS-NG validations.
In ICMS, only a warranted CO can check-in/finalize an action (and thus finalize the action in
FPDS-NG, as well) after signature. If a CS entered the data, the CO would review prior to
check in. If a CO entered the data, they would have knowledge of what they entered and have
the authority to check themselves in. As in the case with SPEDI, this process allows the CO to
ensure that the data entered passes all FPDS-NG validations.
FAR 42.505 states "Government representatives must recognize the lack of privity of contract
between the Government and subcontractors." Accordingly, the Government contracting
community has little interaction with sub-contractors. To the extent the forthcoming OMB
guidance contains a clause requiring prime contractors to report 1st tier subcontract awards,
OAM will include the clause in awarded contracts (as prescribed). However, given the
regulatory guidance regarding contract privity, OAM will not be in a position to independently
review, comment, or confirm the quality of any subcontractor data entered by a prime
contractor.
2. Grants
Data is entered in EPA's source grants system, IGMS, by Grants Specialists (GS) with system
rights approved by management. GS personnel use a checklist for evaluating the completeness
and quality of the information in the award. The Award is routed through the management chain
to the Award Official for approval. Managers in the review chain and the Award Official may
return the award for correction in the event an error is identified. All interim Approvers and
Award Officials are required to take training to be certified for their positions. Re-certification is
required every three years.
E. EPA's process to ensure consistency of Federal Spending Data
1. Contracts
USASpending.gov uses data from FPDS-NG, which does not collect data by specific line of
accounting, but by preponderance of funding. For example, each line of accounting has a
specific Allowance Holder/Program Office, and there could be multiple Allowance
Holders/Offices providing funds on a specific contract action. For FPDS-NG purposes,
information is collected for the Office providing the most funds on a single line. While the grand
total of funds obligated will be the same as in EPA's IFMS, the funds will all show up under one
program in the FPDS-NG and USASpending.gov - and this will continue to happen until those
Government-wide systems obtain the capability to capture data by individual accounting line.
OAM's internally run reports (e.g., end of year statistics) from ICMS/SPEDI collect data based
on specific lines of accounting.
14
-------
2. Grants
EPA ensures consistency in its reporting by using a single source, the Grants Data Mart, for all
grants information reporting. Both the USASpending.gov and the Federal Assistance Award
Data System (FAADS) data sets are produced from this source.
EPA's Grants Data Mart includes data from both the grants system and the finance system.
Standard reports monitor the match between grant and financial records, ensuring that records
exist in both systems for every transaction. In addition, periodic exercises are conducted
comparing obligated balances reports produced by the financial system to grants data.
EPA also controls the CFDA field in the source system so that the CFDA programs that can be
selected are those posted currently on cfda.gov.
F. EPA's process to ensure completeness of the Federal spending
information
1. Contracts
A user cannot finalize an action in FPDS-NG unless they populate all required data fields and
pass all validations. Finalized data is then transmitted to USASpending.gov. The supplemental
TAS .txt file submitted on line must pass their file check prior to acceptance. At a later point in
time, the USASpending.gov support team validates the .txt file submission and sends an e-mail
to the Agency to determine if any issues have been found. In addition, persons with
USASpending.gov log-in accounts can check the site to determine the current status of the date
and if it is considered submitted and complete.
2. Grants
The USASpending.gov data is manually checked for completeness, and missing or incorrectly
formatted data is corrected and entered before submission. The total number of records for the
month is cross checked against IGMS, the source system, and an ad hoc query of the Data
Mart is performed to confirm the total count. After submission, the USASpending.gov Support
Team reviews the submission and notifies the Agency of any issues requiring correction.
Registered EPA staff monitors the USASpending.gov site to assess the status of the submission
and the completeness of the posted grant data.
G. Monitoring
1. EPA's Internal Metrics
i. Contracts
OAM monitors USASpending.gov data quality through its oversight efforts devoted to
FPDS-NG data. We directly transmit data to the latter system, which then sends the
contract data to USASpending.gov. FPDS-NG data is monitored through on-line data
entry edits in that system which must be met by ICMS/SPEDI in order for data to be
transmitted, as well as the QAPs and QAP review process, various recurring data
reconciliations between systems, and the annual FPDS-NG verification and validation
exercise. This latter exercise identifies accuracy rates for selected data elements.
Please see Section II above for comments regarding the oversight processes and
procedures which are in place to ensure data accuracy and integrity.
15
-------
ii. Grants
OGD monitors USASpending.gov data quality through its monitoring and control efforts
devoted to the source system for grants data, IGMS, and the reporting database, the
Grants Data Mart. EPA monitors a set of metrics monthly (e.g. closeout, unliquidated
obligations, and advance monitoring) which track the performance of the grants program
including reporting on grant awards. Metrics results are checked for accuracy before
being published.
2. Deficiencies Identified
i. Contracts
Based on results of QAP Compliance Reviews, as well as FPDS-NG verification and
validation exercises, we are aware that deficiencies exist as far as data accuracy on
selected elements. However, EPA is in the process of replacing its legacy automated
acquisition systems, ICMS and SPEDI, with a commercial, off-the-shelf (COTS) product,
Compusearch's PRISM application. The new system, which will be known as the EPA
Acquisition System (EAS), has been in use throughout the Federal Government for
many years, and we are confident that we will be able to greatly improve the accuracy of
our acquisition data and collections, including submissions to the FPDS-NG (and, from
that system, to USASpending.gov), via the new EAS. The new system will provide for
single-point data entry, real-time reporting, and enhanced on-line logic and edit checks
that will allow for more timely correction of any data entry problems or inconsistencies.
Data definitions, sources and controls will be consistent, due to the consolidation of
multiple systems and applications into a single data base structure. Phased-in
deployment is scheduled to begin during the second half of FY 2010, and the system is
scheduled to be fully deployed during the first half of FY 2011.
ii. Grants
EPA has identified two deficiencies with respect to USASpending.gov data. The first is
that USASpending.gov only has the capacity to collect one TAS code per transaction.
For grant transactions with multiple sources of funding, this limitation requires the
Agency to over-report funding to one TAS and under-report to others.
The second data challenge is the place of performance data elements, as well as
congressional districts, city and county codes, and country codes. The types of
deficiencies include missing data, misspelled names, incorrect use of codes, and
mistyped numbers. The Agency is dependent upon the recipient to provide place of
performance data during the application process. This is especially challenging for
continuing environmental program and project grants whose applicability might be "state-
wide" or "Potomac watershed." This also presents a challenge for research grants
where the applicability of results is global.
16
-------
Senior Accountable Official Certification:
"In connection with the plans detailing information disseminated, as required by the Open
Government Directive, the undersigned Barbara J. Bennett, Senior Accountable Official hereby
certifies that the information contained in the attached plan materially represents the identity and
other relevant information over the quality and integrity of EPA's Federal spending information.
Signature:
Date : ____________________ 5/1 7/1 0
17
-------
IV. Glossary
AA
ARRA
CFDA
COTS
CO
CPR
CS
CWS
EAS
FAADS
FDW
FFATA
FPDS-NG
GAO
GIAMD
GMP
GMSA
GS
ICMS
IGMS
IFMS
NPTCD
CAM
OARM
OCFO
OEI
OFM
OGD
OMB
OMB/OFPP
PAMP
Assistant Administrator
American Recovery and Reinvestment Act of 2009 (P.L. 111-5)
Catalog of Federal Domestic Assistance
Commercial, off-the-shelf
Contracting Officer
Comprehensive Performance Reviews
Contracting Specialist
Contract Writing System
EPA Acquisition System
Financial Award Assistance Data System
Financial Data Warehouse
Federal Funding Accountability and Transparency Act
Federal Procurement Data System- Next Generation
Government Accountability Office
Grants and Interagency Agreements Management Division
Grants Management Plan
Grants Management Self-Assessment
Grants Specialist
Integrated Contract Management System
Integrated Grants Management System
Integrated Financial Management System
National Policy, Training and Compliance Division (located in the Office of
Grants & Debarment
Office of Acquisition Management
Office of Administration and Resources Management
Office of the Chief Financial Officer
Office of Environmental Information
Office of Financial Management
Office of Grants and Debarment
Office of Management & Budget
Office of Federal Procurement Policy
Post-Award Management Plan
18
-------
PARS Performance Appraisal and Recognition System
PTOD Policy Training and Oversight Division
QAP Quality Assessment Plan
SAO Senior Accountable Official
SES Senior Executive Service
SIQ Open Government Spending Information Quality Work Group
SPEDI Small Purchases Electronic Data Interchange
TAS Treasury Accounting Symbol
19
-------
V. Appendix 1: Data Quality Assurance Check-List Instructions
The associated attached check-list will be used as a tool in performing risk assessments on
all spending data sets before posting to the web to assure spending data quality and
reliability. All of the procedures must be completed by the data provider and OCFO/OFM
and checked prior to OEI submitting any financial data for posting. The below assurance
must accompany the data set. This will assist in the assurance that spending data can be
used with a high level of confidence in its accuracy reflecting the true economic reality of the
situations and transactions that occurred.
1. Each data set will be reviewed by OCFO/OFM staff who will:
a. Determine the risk level of the data's accuracy and reliability.
b. Based on the risk level, the reviewer will collect a random sample of the data and
pull the supporting documentation large enough to provide statistically significant
assurance of its accuracy.
c. The reviewer will also pull the largest 1% of transactions by dollar value and
verify their accuracy.
d. Perform a test of the internal controls verifying their reliability.
e. Trace the data flow and verify that the procedures and policies were followed as
prescribed.
f. Spot check to assure that all of the data provided is properly labeled in a manner
that is clear and accurate including the date of collection.
g. All data that is being provided has a clean audit opinion if applicable. If there are
any outstanding audit issues with the data, the information will not be posted.
2. Each office requesting posting of a spending data set must be able to certify and trace
the origin of the data.
The Office Director or Designee from the office providing the data set, assures the
information is materially accurate and all relevant policy, procedures and internal controls
were followed in the collection and presentation of the data set.
Signature:
Date:
20
-------
VI. Appendix 2: Data Quality Assurance Check-List
This check-list will be used as a tool in performing risk assessments on all spending data sets before
posting to the web to assure spending data quality and reliability. Please reference Appendix #1 for any
clarifications.
D Determine the risk level of the data's accuracy and reliability. If the risk level is deemed high, the data
should not be released. Risk level can be based on possible users of the data, past problems with the
accuracy of the data or other applicable methods as deemed appropriate.
D Based on the risk level, collect a random sample of data and pull the supporting documentation large
enough to provide statistically significant assurance of its accuracy. To pull the random data, use
either a random number generator or other similar methods. For medium risk data establish a 10% or
less sampling error. This can either be calculated or via the following site:
http://www.dssresearch.com/toolkit/secalc/error.asp
D Enter '50%' as the sampling proportion; then enter the total universe of your data in the population
box. Next enter '95%' as your confidence interval. Based on the size of your data, enter an
appropriate sample size to achieve less than 10% sampling error.
Ex. A sample size of 50 out of a population of 100 will result in a 9.8 % sample error.
Ex. A sample size of 90 out of a population of 1,000 will result in a 9.9 % error rate.
The larger the population, the smaller % of the population you will need to sample.
D Pull the largest 1% of transactions by dollar value and verify their accuracy. This test is performed
independently of other transaction tests. This will also reduce the magnitude of any misstatements.
D If the actual error level in the data of either test is above 5%, do not release the data and consider
corrective actions or further internal controls.
Ex. if your sample size is 100 and you find 10 errors, you would have a 10% error rate.
D Perform a test of the internal controls verifying their reliability. Read through the governing policies
and procedures and verify that they are all properly followed.
D Trace the data flow and verify that the procedures and policies were followed as prescribed for a
random sample of data. This can vary based on the type of data and required internal controls.
D Spot check to assure that all data provided is properly labeled in a clear and accurate manner. Make
sure that the labeling is intuitive to the reader to reduce misinterpretation. Provide a list of acronyms
for the data, what seems obvious to you will not be the same for an outside reader.
D All data has a clean audit opinion if applicable. Data with outstanding audit issues will not be posted.
D Each office requesting posting of a spending data set must be able to certify and trace the origin of
the data. You must be able to audit your financial information and verify the data's accuracy with the
source documentation. Also be mindful of possible data corruption if your data has been migrated
from one database to another.
D Ensure Office Director or Designee's signature is present assuring that all information provided is
materially accurate and all relevant policy, procedures and internal controls were followed in the
collection and presentation of the data.
21
-------
VII. Appendix 3: FPDS Data
Agency Name: U.S. Environmental Protection Agency Fiscal Year of FPDS Data: FY 2009
Percent of Total Procurement Spend Covered by Sample: 3.23%
Accuracy Computation for Key Data Elements
Data Element Name
2A Date Signed
2C Completion Date
2D Est. Ultimate Completion Date
2E Last Date to Order
3A Base and All Options Value
3B Base and Exercised Options Value
3C Action Obligation
4C Funding Agency ID
6A Type of Contract
6F Performance Based Service Acquisition
6M Description of Requirement
SA Product/Service Code
8G Principal NAICS Code
9A DUNS No
9H Place of Manufacture
9K Place of Performance ZIP Code (+4)
10A Extent Competed
10C Reason Not Competed
10D Number of Offers Received***
10N Type of Set Aside
10R Statutory Exception to Fair Opportunity
11A CD's Business S.ze Selection
11B Subcontract Plan
12AIDVType
12B Award Type
Tola; Records Sampled
(Column A)
No. of Records
Reviewed
405
384
373
13
395
389
404
318
314
356
391
388
404
10
372
201
72
NA
359
10
377
62
16
389
6533
(Column B)
No. of Correct
Records
336
367
356
13
382
379
396
131
275
274
315
335
354
374
10
302
171
58
NA
340
33S
59
16
388
5978
(Column A/
Column B as %)
Accuracy Rate
83.0%
95.4%
100%
96.7%
97.4%
98.0%
100%
36 5%
87 3%
88.5%
85.7%
91 2%
92 6%
100%
91 2%
85.1%
80 6%
NA
94.7%
80.0%
95.2%
100%
99.7%
91.5%
Exhibit 1
Attachment
"Overall Accuracy Rate: 91.5 %
Systemic Causes of Invalid Data
(Check all that apply)
User FPDS Other
X
X
X
X
X
X
X
X
Exhibit 1
Attachment - cont.
* The EPA is 95% confident that the overall accuracy rate for the 24 reviewed fields in the enti-e EPA FPDS-NG population is between 86.5% and
96.5%.
•"This data element must be validated beginning with the FY 2010 data, but is not required to be validated for the FY 2009 data.
NOTE: This Attachment provides a standard format for agencies to use in reporting the overall accuracy rate for the data elements being validated
as well as the accuracy rate for each data element. (Note that the data element names are as they appear on the FPDS screens.) Please
summarize the data accuracy results collected from all subordinate offices that validated and certified their own data into this Attachment. Please
also discuss any systemic causes of invalid data in as much detail as you can. with particular attention to errors caused by FPDS or any other
components of the Integrated Acquisition Environment. Use additional pages as needed.
22
------- |