U.S. ENVIRONMENTAL PROTECTION AGENCY
OFFICE OF INSPECTOR GENERAL
Efficiency of EPA's
Rule Development Process
Can Be Better Measured
Through Improved
Management and Information
Report No. 13-P-0167
February 28, 2013

-------
Report Contributors:	Laurie Adams
Daniel Carroll
Carolyn Copper
Jerri Dorsey
Gabby Fekete
Jeffrey Harris
Olga Stein
Abbreviations
ADP	Action Development Process
EPA	U.S. Environmental Protection Agency
OIG	Office of Inspector General
OMB	U.S. Office of Management and Budget
OP	Office of Policy
RAPIDS	Rule and Policy Information and Development System
Hotline
To report fraud, waste, or abuse, contact us through one of the following methods:
e-mail: OIG Hotline@epa.gov	write: EPA Inspector General Hotline
phone: 1-888-546-8740	1200 Pennsylvania Avenue, NW
fax:	202-566-2599	Mailcode 2431T
online:
http://www.epa.gov/oiq/hotline.htm
Washington, DC 20460

-------
^eDsx
' JL \
i®i
U.S. Environmental Protection Agency
Office of Inspector General
At a Glance
13-P-0167
February 28, 2013
Why We Did This Review
The purpose of this review was
to evaluate whether the U.S.
Environmental Protection
Agency's (EPA's) Action
Development Process (ADP)
results in the timeliest, most
efficient, and most effective
method for rule development.
This review was requested by
the Agency. We focused on the
key aspects of efficiency with
which program offices
implement the ADP's guidance
for rule development. EPA's
Office of Policy coordinates the
ADP.
This report addresses the
following EPA Goals or
Cross-Cutting Strategies:
•	Taking action on climate
change and improving
air quality
•	Protecting America's
waters
•	Cleaning up communities
and advancing sustainable
development
•	Ensuring the safety of
chemicals and preventing
pollution
Efficiency of EPA's Rule Development Process
Can Be Better Measured Through Improved
Management and Information
What We Found
Rule development is one of the Agency's principal tasks. EPA develops rules to
carry out the environmental and public health protection laws passed by
Congress. Efficient EPA rulemaking may accelerate the progress of protecting
human health and the environment. However, due to limitations in EPA
rulemaking documentation and guidance, the Agency is unable to evaluate the
efficiency of the rulemaking process or identify potential delays in its rulemaking
activities. For example, EPA has limited information on the time and resources
used to complete the various stages of the rule development process.
The development and implementation of management controls to ensure that the
rulemaking process is progressing efficiently and that resources are accurately
accounted for will enhance EPA's ability to assure efficiency during the
development process and accelerate the progress of protecting human health
and the environment.
Recommendations and Planned Agency Corrective Actions
We recommend that the Associate Administrator for EPA's Office of Policy
establish guidance, maintain database documentation, and track resources, to
enhance the Agency's ability to determine the efficiency of the rulemaking
process. The Associate Administrator concurred with the first and second
recommendations but disagreed with the third recommendation. We consider the
third recommendation unresolved. As required by EPA Manual 2750, the Agency
will need to initiate the audit resolution process.
For further information, contact
our Office of Congressional and
Public Affairs at (202) 566-2391.
The full report is at:
www.epa.qov/oiq/reports/2013/
20130228-13-P-0167.pdf

-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
THE INSPECTOR GENERAL
February 28, 2013
MEMORANDUM
SUBJECT: Efficiency of EPA's Rule Development Process Can Be Better Measured
Through Improved Management and Information
Report No. 13-P-0167
This is a report on the subject evaluation conducted by the Office of Inspector General (OIG) of
the U.S. Environmental Protection Agency (EPA). This report contains findings that describe the
problems the OIG has identified and corrective actions the OIG recommends. This report
represents the opinion of the OIG and does not necessarily represent the final EPA position.
Final determinations on matters in this report will be made by EPA managers in accordance with
established audit resolution procedures.
Action Required
In accordance with EPA Manual 2750, you are required to provide a written response to this
report within 60 calendar days. You are not required to provide a written response to the first
and second recommendations because you provided agreed-to corrective actions and planned
completion dates. The OIG may make periodic inquiries on your progress in implementing
these corrective actions. The OIG and Agency have not agreed on a course of action to address
the third recommendation. We consider that recommendation unresolved. As required by
EPA Manual 2750, the Agency will need to initiate the audit resolution process.
Should you choose to provide a response, we will post your response on the OIG's public
website, along with our memorandum commenting on your response. You should provide your
response as an Adobe PDF file that complies with the accessibility requirements of Section 508
of the Rehabilitation Act of 1973, as amended. The final response should not contain data that
FROM
Arthur A. Elkins Jr.
TO
Michael Goo, Associate Administrator
Office of Policy

-------
you do not want to be released to the public; if your response contains such data, you should
identify the data for redaction or removal along with corresponding justification. We have no
objections to the further release of this report to the public. We will post this report to our
website at http://www.epa.gov/oig.
If you or your staff have any questions regarding this report, please contact Carolyn Copper,
Assistant Inspector General for Program Evaluation, at (202) 566-0829 or
copper.carolvn@epa.gov; or Jerri Dorsey at (919) 541-3601 or dorsev.ierri@epa.gov.

-------
Efficiency of EPA's Rule Development Process Can Be	13-P-0167
Better Measured Through Improved Management and Information
		Table of 	
Chapters
1	Introduction		1
Purpose		1
Background		1
Scope and Methodology		3
2	Additional Controls Needed to Measure Efficiency		5
Enhanced Tracking and Document Retention Needed		5
Rulemaking Costs Are Not Tracked		6
Conclusion		8
Recommendations		8
Agency Comments and OIG Evaluation		8
Status of Recommendations and Potential Monetary Benefits		10
Appendices
A Completed Rules Sampled and Reviewed by OIG		11
B Agency Response and OIG Comments		12
C Distribution		18

-------
Chapter 1
Introduction
Purpose
The purpose of this review was to evaluate whether the U.S. Environmental
Protection Agency's (EPA's) Action Development Process (ADP) ensures the
timeliest, most efficient, and most effective method for rule development.
Specifically, we focused on key aspects of the efficiency1 with which the program
offices implement the Agency's ADP guidance. The initial impetus for this
review was a request by the Agency to improve the process.2
Background
EPA protects public health and the environment in a variety of ways, but one of
the most important ways is through regulation development. EPA regulations
cover a range of environmental and public health protection issues, from setting
standards for clean water and establishing requirements for proper handling and
reductions of toxic wastes to controlling air pollution from industry and other
sources. According to EPA, it is one of the most active regulatory agencies in the
federal government. Each year, the Agency develops a considerable number of
rules which address highly technical, scientific, and complex environmental
problems. On average, EPA issues over 122 Administrator-signed regulations
each year.3 Between 2005 and 2010, EPA published 735 major or Administrator-
signed rules, of which approximately 40 percent were final rules.4
According to the Agency, much of its environmental success and organizational
credibility is directly linked to the quality of its work under the ADP. EPA
designed the ADP to develop rules based on sound scientific, economic, legal,
and policy analyses. The ADP is designed to be a multi-disciplinary,
collaborative, cross-office, and cross-media approach to rule development and
accelerate the progress of protecting human health and the environment.
1	For the purposes of this review, "efficiency" is defined as the resources used to achieve program results. This is the
U.S. Government Accountability Office's Government Auditing Standards definition.
2	OIG management received a suggestion to review the Agency's rule development process in an effort to identify
areas of redundancy, waste, and inefficiency.
3	EPA's Administrator-signed rules are published actions which modify or propose to modify the Code of Federal
Regulations and, therefore, contain rule text. The Code of Federal Regulations is the official record of all regulations
created by the federal government.
4	A final rule is the conclusion of the Agency's regulatory development. A final rule represents a range of
documents that are published in the Rules section of the Federal Register. These include final rules that amend the
Federal Register by adding new text or by revising or removing existing text. These also include other types of final
actions or documents that have no regulatory text and do not amend the Federal Register but affect the Agency, such
as general policy statements, final guidance documents, or petitions for reconsideration.
13-P-0167
1

-------
The ADP serves as a framework to ensure that rules are developed using quality
information and all scientific, economic, and policy issues are addressed during
the appropriate rule development stages. The ADP also provides opportunities for
senior management's early involvement and provides guidance to rulemaking
staff at key points of the rule's development. EPA's Office of Policy (OP) leads
the regulatory development process via its Office of Regulatory Policy and
Management, which provides support and guidance to EPA's program and
regional offices as they develop regulations. EPA's OP coordinates the ADP and
also serves as liaison to other agencies involved in the rule development, such as
the Office of Management and Budget (OMB).
There are five major stages of the ADP:
1.	Tiering the action and obtaining commencement approval.
2.	Developing the proposed rule or draft action.
3.	Requesting OMB review (if needed) for proposed and final actions.
4.	Requesting signatures, publishing an action in the Federal Register, and
soliciting and accepting public comment.
5.	Developing the final action and ensuring congressional review.
Figure 1 illustrates the key steps related to the five stages of the ADP. This figure
represents the development process for all Tier 1 and 2 actions.
Figure 1: Flowchart of the Action Development Process in EPA
1 0»iissfor \

y
2 <
/Ticniiti
LJ»
j *
'iM-rkyrobp
pWte-
Ar.aMic
4 Ear^v
GuiJsmv
X .
¦'%. ,r'

5
Detail! Artal^c S'ueiwit

8,
Mw^ierivnl

1,)uhilertsUsartsk**ar^ \
refjulatervoptoGrts	f

v-
9 fftMgwu?*
pPfientipitarnHe
nils- »wl iufpsitna
dotuBiei.b
"X
X,
1J Fna:
Rfv'fir
¦x.

Source: EPA's Action Development Process training presentation.
Note: Step 7 of the illustration involves the Agency's necessary analyses and an assessment of all
applicable statutes and executive orders, of which there are many.
AA/RA: Assistant Administrator/Regional Administrator
EO: Executive Order
13-P-0167
2

-------
Scope and Methodology
We conducted our work from August 2011 through August 2012 in accordance
with generally accepted government auditing standards. Those standards require
that we plan and perform our work to obtain sufficient, appropriate evidence to
provide a reasonable basis for our findings and conclusions based on our
objectives. We believe that the evidence obtained provides a reasonable basis for
our findings and conclusions based on our objectives.
Our review focused on the key aspects of efficiency with which program offices
implement the ADP's guidance for rule development. Efficiency can refer to not
only the timeliness in producing rules but the management of the resources used
in the rule development process. In an effort to evaluate the efficiency of the
rulemaking process, we identified rules that had final regulatory actions in 2010
and 2011. We reviewed these final rules to determine the adequacy with which
the program offices' staff complied with preparing documents that evidenced the
"products of a quality action."5 We reviewed information in the Rule and Policy
Information and Development System (RAPIDS), EPA's database to manage the
creating, tracking, approving, and reporting of the Agency's rulemaking actions. 6
We also interviewed EPA staff to determine how the Agency tracks the resource
cost for rule development activities.
During field work, we took an initial sample of four completed rulemaking
actions to determine the feasibility of conducting a larger sample of completed
rules.7 We attempted to document progress for each rule throughout the entire
ADP process using a data collection instrument developed by the EPA Office of
Inspector General (OIG). We sought to obtain key documentation and dates both
from RAPIDS and the workgroup chairs. We were eventually able to obtain some
documentation for each rule sampled; however, the documentation was not
always the final signed product. This impeded our ability to track the progress of
the rules though the ADP process in an effort to identify areas to increase
efficiency. After completing the limited sample work, we discovered fundamental
gaps in the ADP tracking process that made further data collection unnecessary.
These gaps are discussed in detail in chapter 2 of this report.
5	"Products of a quality action" are defined by EPA's ADP guidance document as those actions that include the
following elements: Tiering Form; Preliminary Analytic Blueprint; Early Guidance Briefing Package; Early
Guidance Memorandum; Detailed Analytic Blueprint; Options Selection Briefing Package; Draft Action (e.g., rule
text, report to Congress, policy or guidance); Action Memorandum; and Communications Plan, coordinated with the
Office of Public Affairs.
6	During our field work, RAPIDS was the database used at the time to manage all ADP activities and the one to
which we were directed for our evaluation's data collection activities.
7	See appendix A for details on these four actions.
13-P-0167
3

-------
Prior Reports
The U.S. Government Accountability Office reviewed federal rulemaking in two
recent reports.8 One of the reports concluded that agencies had little data on the
time and resources used to comply with regulatory requirements, making it
difficult to evaluate the effects of requirements on rulemaking. In the other report,
EPA was cited for not following key steps in its guidance before finalizing a
major rule. There were no previous EPA OIG reviews of this subject matter.
8 GAO-08-128: EPA Actions Could Reduce Environmental Information Available to Many Communities, November
2007; GAO-09-205: Improvements Needed to Monitoring and Evaluation of Rules Development as Well as to the
Transparency of OMB Regulatory Reviews, April 2009.
13-P-0167
4

-------
Chapter 2
Additional Controls Needed to Measure Efficiency
EPA established the ADP to ensure the quality of actions developed, but ADP
guidance does not contain adequate controls to assure that the process is
completed in the most efficient manner. Inefficiencies in the rule development
process may result in delays that could postpone new or revised rules and impede
progress to protect human health and the environment. To determine the
efficiency of the rulemaking process, the Agency needs adequately maintained
documentation, defined milestone goals that measure and monitor rule
development progress, and cost information. Adequate system controls and the
development of a method to track resource use will provide EPA with the tools
necessary to measure and assure the efficiency of the development process.
Enhanced Tracking and Document Retention Needed
We found that EPA's database for creating and tracking rules was sparsely
populated and did not contain the necessary documents or information to allow us
to complete our review.9 We found program offices were not adequately utilizing
the Agency's database due to a lack of standardized procedures from the OP
describing who was responsible for uploading the developmental documents to
the databases used to manage the ADP process. The development documents
called for in the ADP guidance represent tools for both the rulemaking workgroup
as well as for management's oversight responsibilities. The availability of these
documents and associated milestone completion data would provide management
with the information needed to oversee the rule development process and assure
the rulemaking process progresses in a timely and efficient manner.
In February 2012, EPA implemented a new database, ADP Tracker. However,
there remain some limitations in regards to tracking and documentation which
challenge the Agency's ability to monitor, evaluate, and assure the efficiency of
EPA rulemaking. According to the Agency, the new system will provide
improved capability to track milestones, manage workgroups, and track work
flow, as well as provide better security and access.10 According to OP, the Agency
conducted nearly 20 training sessions for staff throughout the Agency on proper
use of ADP Tracker. However, there are still no standardized procedures
describing who is responsible for uploading the key developmental documents to
the database used to manage the ADP process. OP, the office that manages the
9 Because Agency staff had not populated RAPIDS with the milestone dates and other data on the ADP, efficiently
determining any potential sources of delay in the rulemaking process for our sampled rules was not possible.
111 We were unable to access or conduct a full evaluation of the new system because it was not fully implemented
during our evaluation field work.
13-P-0167
5

-------
ADP, should provide additional instruction to the program offices involved in
rulemaking regarding the roles and responsibilities for uploading the various
documents into the new database. Furthermore, OP should assure that the
program offices comply with the data entry requirements. This will provide OP
with the ability to ensure data reliability and the rulemaking process progresses in
a timely and efficient manner.
Rulemaking Costs Are Not Tracked
The Agency does not maintain cost data associated with action development and
rulemaking. Rule development requires input from various offices in the Agency
with an associated time commitment, and can take several years to complete. The
lack of cost data means that EPA has limited information on the resources used to
complete the ADP process.
Rulemaking Activities Represent Time Commitment Across Agency
EPA prepares and releases hundreds of actions each year that define the technical
and operational details of environmental programs. These actions can be complex.
As illustrated previously in Figure 1, the rulemaking process includes a number of
steps, including internal and external comment and review. According to the
Agency, the development time (i.e., time from initiation of a rulemaking to final
rule stage) for publication of an Administrator-signed rule for the period 2005-
2010 was a total average time of 2 years and 6 months (930 days). Figure 2
illustrates how rule development time varies by program office and by year.11
11 The variation in the development time across the five main offices with published Administrator-signed rules is
greatly influenced by a variety of internal and external factors. These factors include the complexity of the
rulemaking, whether the rulemaking is under a court-schedule or legislative mandate, the interests of other program
offices, and the extent of public involvement during the rule development process.
13-P-0167
6

-------
Figure 2: Administrator-signed rules, development time
Administrator-Signed Rules: Average Development Time (Days)
by EPA Program Offices, 2005-2010
6000
5500
5000
4500
4000
3500
3000
2500
2000
1500
1000
500
0
4,690
1,714
2010
2009
2008
2007
2006
2005
OCSPP OSWER
OW
Source: EPA's RegStat website.
OAR:	Office of Air and Radiation
OEI:	Office of Environmental Information
OCSPP:	Office of Chemical Safety and Pollution Prevention
OSWER:	Office of Solid Waste and Emergency Response
OW:	Office of Water
Rule development requires input and time from offices across the Agency, neither
of which are tracked for budgetary or management purposes. It should be the
Agency's goal to ensure that rulemaking is done in the most cost effective and
efficient manner. Because EPA does not have input measures that track time spent
on the development of individual rules, rulemaking cost information is not
available to EPA management to determine efficiency or return on investment.
OMB recommends using input measures (e.g., time, employee hours, funding,
equipment) to determine the resources used to achieve an output (activity) or
outcome (impact). When compared to the achieved outputs and outcomes, the
information can be used to determine program efficiency and return on
investment.
Resource Data Can Enhance Budgeting and Forecasting of Resources
EPA has limited data on the time and resources used to complete the various
stages of the rule development process. By tracking both costs and milestones,
management can monitor the rulemaking progress and better assess where delays
are occurring and make the necessary corrections. Tracking and documenting
costs would also provide management with historical data that could be used for
13-P-0167
7

-------
budgeting and forecasting resource needs in future rulemaking work. For
budgetary considerations, management can have better control on resource needs
in the future based on the expected workload of new rules and how much of their
budget is allocated to rulemaking activities.
Conclusion
Developing environmental regulations is one of EPA's principal tasks. The ADP
was designed to deliver actions that are based on sound science, promote
economic efficiency, and accelerate the progress of protecting human health and
the environment. EPA should take steps to track and document the actions of the
rule development process. Additionally, the development of a method to track
resource use in key rulemaking activities will provide the Agency with the tools
necessary to assess the key aspects of efficiency in the rule development process.
A commitment to assessing efficiency of the rulemaking process shows the
Agency values accelerating the protection of human health and the environment.
Recommendations
We recommend that the Associate Administrator, Office of Policy:
1.	Establish guidance that clarifies roles and responsibilities in ADP
implementation, including data entry, record keeping, and the status of
action development.
2.	Ensure that ADP Tracker has established clear roles, responsibilities, and
requirements for the program offices to upload development documents;
and that entries are updated in a timely manner, are monitored for data
quality, and all features available are used to evaluate the efficiency of the
rule development process.
3.	Develop and implement a method to track resource use in key rulemaking
activities.
Agency Comments and OIG Evaluation
The Associate Administrator concurred with the first and second
recommendations. The Agency provided high-level corrective actions in response
to these recommendations with an estimated milestone completion date of
June 2013. These recommendations are considered open with agreed-to corrective
actions pending.
OP disagreed with the third recommendation because it does not have the expertise
to develop a method to track resources. The OIG and Agency have not agreed on a
course of action to address this recommendation. We consider this recommendation
unresolved with actions underway as required by EPA Manual 2750.
13-P-0167
8

-------
OP believes the report does not present an accurate picture of rulemaking in the
Agency. Additionally, OP believes that clarification of the purpose of the report is
critical to assessing what appropriate conclusions can be fairly reached regarding
the ADP process. OP also questioned the emphasis on the RAPIDS database,
which was replaced by ADP Tracker in February 2012. We made changes to the
report as appropriate. The Agency's complete response, along with the OIG's
evaluation, is in appendix B.
13-P-0167
9

-------
Status of Recommendations and
Potential Monetary Benefits
RECOMMENDATIONS
POTENTIAL MONETARY
BENEFITS (In $000s)
Rec.
No.
Page
No.
Subject
Status1
Action Official
Establish guidance that clarifies roles and
responsibilities in ADP implementation, including
data entry, record keeping, and the status of action
development.
Ensure that ADP Tracker has established clear
roles, responsibilities, and requirements for the
program offices to upload development documents;
and that entries are updated in a timely manner,
are monitored for data quality, and all features
available are used to evaluate the efficiency of the
rule development process.
Develop and implement a method to track resource
use in key rulemaking activities.
Associate Administrator,
Office of Policy
Planned
Completion
Date
Claimed
Amount
Ag reed-To
Amount
Associate Administrator, June 2013
Office of Policy
Associate Administrator, June 2013
Office of Policy
1 0 = Recommendation is open with agreed-to corrective actions pending.
C = Recommendation is closed with all agreed-to actions completed.
U = Recommendation is unresolved with resolution efforts in progress.
13-P-0167
10

-------
Appendix A
Completed Rules Sampled and Reviewed by OIG
Rule title
Tier
Program office
Transport Rule (CAIR Replacement Rule) - Rule No. 2060-AP50
(SAN: 5336)
Tier 1
Office of Air and
Radiation
Federal Requirements under the Underground Injection Control
(UIC) Program for Carbon Dioxide (C02) Geologic Sequestration
(GS) Wells - Rule No. 2040-AE98 (SAN: 5211)
Tier 2
Office of Water
Mercury; Significant New Use Rule for Elemental Mercury in Flow
Meters, Manometers, and Pyrometers - Rule No. 2070-AJ36 (SAN:
5238)
Tier 3
Office of Chemical
Safety and Pollution
Prevention
Protection of Stratospheric Ozone: Listing of Substitutes for Ozone-
Depleting Substances-Hydrocarbon Refrigerants - Rule No. 2060-
AP54 (SAN: 5339)
Tier 3
Office of Air and
Radiation
Source: OIG analysis.
13-P-0167
11

-------
Appendix B
Agency Response and OIG Comments
October 12, 2012
MEMORANDUM
SUBJECT: Response to Draft Report: Efficiency of EPA's Rule Development Process
Cannot Be Assessed, Project No. OPE-FY 11-0021
FROM: Michael L. Goo /s/
Associate Administrator
TO:	Carolyn Copper
Assistant Inspector General for Program Evaluation
Office of Inspector General
In accordance with U.S. Environmental Protection Agency (EPA) Manual 2750,1 hereby submit
the Office of Policy's (OP) response to the findings and recommendations cited in the Office of
Inspector General's (OIG) draft evaluation report "Efficiency of EPA's Rule Development
Process Cannot Be Assessed, Project No. OPE-FY11-0021."
Thank you for providing the Office of Policy with an opportunity to share our feedback on the
draft evaluation report. We believe the report seeks to examine important features of the EPA
Action Development Process (ADP) and we are eager to engage with the Office of Inspector
General in crafting an accurate and relevant report that fairly characterizes the complete ADP
context. As currently drafted, we believe the report does not present an accurate picture. We
believe that additional data bearing on this issue will be forthcoming in the near future and that
such data will be extremely helpful in accurately describing the situation. We also believe that
clarification of the purpose of the report is critical to assessing what appropriate conclusions can
fairly be reached regarding the ADP process.
OIG Response: As communicated to the Agency, the purpose of this review was to
evaluate whether EPA's ADP resulted in the timeliest, most efficient, and most effective
method for rule development. This basis for our review was a suggestion by the Agency to
look at the process and potential ways to increase efficiency. We focused on the efficiency
with which program offices implemented the ADP guidance for rule development.
13-P-0167
12

-------
Finally, we believe that the draft findings contemplated in the draft report relate largely to a
database that is no longer in use by EPA and therefore such conclusions are of, at best,
diminished relevance to the ongoing work of the Agency. In light of these concerns we would
appreciate the opportunity to consult further with the Office of the Inspector General regarding
this Draft Report prior to its finalization.
OIG Response: The OIG addressed suggested corrections as appropriate.
Nevertheless, OP agrees with the OIG's conclusion that opportunities exist to expand the
usefulness and applicability of the ADP Tracker and concurs with the intent of OIG's second
recommendation to "ensure that ADP Tracker has established clear roles, responsibilities, and
requirements for the program offices to upload development documents; and that the rules are
updated in a timely manner, entries are monitored for data quality, and all features available are
used to evaluate the efficiency of the rule development process."
The Draft report states, under the heading "Why We Did This Review" that:
The purpose of this review was to evaluate whether the U.S. Environmental Protection
Agency's Action Development Process results in the timeliest, most efficient and most
effective method for rule development. The initial impetus for this review was a request
by the Agency to look at the process and potential ways to increase efficiency. We
focused on the efficiency with which program offices implement the ADP's guidance for
rule development.
Except for the description above, the details of the "request by the Agency" remain largely
unclear to the Office of Policy at present. Although the Office of Policy welcomes the
opportunity to explore ways to improve the efficiency of EPA's rulemaking process, the
language above seems to suggest a concept of "efficiency" in which the speed of ADP is the
primary focus of the inquiry. Moreover the concept of "efficiency" as "speed" is then
incorporated into the draft findings which state "the ADP Guidance does not contain controls to
ensure that the process is completed in the most efficient manner. Inefficiencies can result in
delays that postpone new or revised rules." As a result, the report concludes that "The Efficiency
of EPA's Rule Development Process Cannot be Assessed" and that "[The] ADP Does Not
Include Controls to Ensure Efficiency."
OIG Response: The OIG had multiple communications with the Agency regarding the
basis for this evaluation and the details of the request. Our review was requested by
Agency officials. At no time during our review did anyone within the Agency question or
raise concerns with our scope.
As the draft report also acknowledges, "the ADP serves as a framework to ensure that rules are
developed using quality information and that all scientific, economic and policy issues are
addressed during the appropriate rule development stages." Given this purpose for the ADP, it
should be evident that mere hastening of the rulemaking process, in such a way as to produce
rulemaking products in the shortest amount of time, and using the fewest amount of Agency
13-P-0167
13

-------
resources, may not ultimately be consistent with a more comprehensive and appropriate
definition of "efficiency."
OIG Response: OIG's report does not advocate for speed in the rulemaking process over
effectiveness of the process. OIG's report recommends that additional controls are needed
to measure efficiency.
As OP staff conveyed during our May 30, 2012 meeting and in several conference calls, the
purpose of the ADP is to guide the process for developing a wide variety of rulemakings, public
notices, and guidance documents. Each such action type can vary significantly with regard to
Agency priority, level of input required from other EPA offices, need for inter-agency review,
and for technical and/or scientific complexity. The goal is to produce quality actions that are
timely, clear, scientifically sound, technically accurate, effective, and legally defensible. It would
serve no public purpose to produce a rule quickly, using few resources, if it fails to withstand
legal challenge and/or produce social benefits.
OIG Response: The OIG report does not advocate that speed or any other factor is more
or less important than efficiency. The report presents the aspects of efficiency that we were
unable to assess due to limitations in the Agency's data systems.
Perhaps more importantly, the rapid development of rulemaking products in ways which fail to
pass either scientific, legal or policy muster cannot credibly be conceived of as "efficient," since
the end result is to produce products that must either be substantially reworked or possibly even
abandoned. Paradoxically, the consequence of moving potentially flawed products through the
system, with speed as the main goal, may in fact result in greater delay in achieving
environmental benefits, thereby undermining even the relatively narrow value of increased speed
at a particular point in the process.
OIG Response: In this review, we did not assess the effectiveness of the ADP nor the
potential for delayed environmental benefits due to the process.
For example, a rule may be "delayed" at a particular ADP stage while important policy and
analytical issues are resolved. While this may appear to be a "delay," well crafted changes to the
rule can speed later stages of review and successful implementation. And of course, rapid action
can result in an inefficient uses of the agency's resources negatively, impact transparency, and
potentially undermine the Agency's credibility. Thus, the ADP process is designed not only
with timeliness in mind, but perhaps more importantly, with the paramount goal of quality
decision making at its heart. We believe the ADP, as currently configured and implemented,
plays a critical role in achieving that goal that should not be overlooked.
OIG Response: Based on the information available to us during our review, we concluded
that the efficiency of the rule development process could not be assessed. We could not
conduct a detailed review of the Agency's rulemaking process due to the lack of data and
documents in support of the process. We believe that the intent of the ADP is to ensure
effective actions are developed.
13-P-0167
14

-------
Accordingly, any evaluation of the ADP process should begin with an assessment of whether the
goals of clarity, scientific validity, technical accuracy, effectiveness and legal defensibility are
systematically being met and should examine the extent to which ADP process requirements
have contributed to their fulfillment. OP's review of the draft report leads us to conclude that
OIG did not fully consider either the complexity or the needed flexibility of the ADP as it
considered its draft recommendations and conclusions. In the attached redline strikeout of the
draft report we provide suggested edits to areas that we believe reflect misunderstandings about
the ADP.
OIG Response: It is reasonable and appropriate to assess the efficiency of EPA's
rulemaking process. Our report does not address effectiveness issues or advocate that those
factors are more or less important than efficiency concerns.
We also note that the draft report directs much of its attention toward the RAPIDS system, which
is no longer in use. A more recent, system, ADP Tracker is in use and we are beginning to have
data available from that system that could help EPA and the OIG in drafting this report. RAPIDS
was developed over 15 years ago and was designed as a forward-looking management
information system to help Agency and program-level leadership track and schedule decision
meetings. Contrary to the Draft Report's suggestion, RAPIDS was not intended to be an
aggregator for all Agency records relating to action development, nor was it intended to be used
to assess the extent to which the documents supporting EPA's rules are efficiently produced.
These activities are managed within individual EPA programs. In any event, RAPIDS has been
replaced.
OIG Response: When we initiated our field work, RAPIDS was the database used by the
Agency to manage the regulatory development activities. The Agency launched ADP
Tracker in February 2012, approximately a month after we met with the Agency to discuss
our final evaluation objectives. At that meeting we were advised that since our sample was
from 2010 and 2011, RAPIDS would have all the information for our selected rules. The
Agency said that we should not have to get involved with the new database. Consequently,
we based our review on the data available to us in RAPIDS. The issues we uncovered with
RAPIDS were not things that the Agency said would be addressed with ADP Tracker,
particularly issues of responsibility for data entry and attaching documents. Therefore the
first and second OIG recommendations are directed toward strengthening the controls in
place governing ADP Tracker.
The replacement system for RAPIDS is ADP TRACKER. This system does include features
that can be used to track process milestones. For example, ADP TRACKER allows us to track
whether ADP milestones were waived by senior management. It also allows us to track the time
workgroups were given to review milestone material. By early next year, a full year of these data
will be available for review. Even so, it should be noted again that even when fully utilized,
ADP TRACKER will not allow us to measure the "efficiency" of the Agency's rulemaking
process and whether the Agency is allocating resources optimally. Again, the system was not
designed for this purpose. A fair notion of "efficiency" would examine whether the ADP
process has been efficient at enhancing the quality of EPA rulemaking. Such an assessment
cannot credibly be performed by a tracking or database management system alone.
13-P-0167
15

-------
OIG Response: While we did not conduct a detailed review of ADP Tracker, the issues
we uncovered with RAPIDS were not things that the Agency said would be addressed with
ADP Tracker, particularly issues of responsibility for data entry and attaching documents.
During our review we were advised that a subset of the data was moved from RAPIDS to
ADP Tracker. If that is the case, the data in ADP Tracker could have the same problems
that RAPIDS had. The Agency did not disclose what it defines as the "fair notion of
efficiency" in its response.
Apart from these concerns, I agree that opportunities exist to expand the usefulness and
applicability of the ADP TRACKER. Therefore, OP concurs with the draft report's first
recommendation to "establish guidance that clarifies roles and responsibilities in ADP
implementation, including data entry, record keeping, and the status of action development." OP
has addressed many of the concerns with regard to clearly delineated roles in recording data and
key action development documents. Additionally, ADP TRACKER (unlike RAPIDS) utilizes
controls to ensure that program offices enter key fields before updating later milestones. OP
plans to issue the updated ADP TRACKER user guides that will clarify these roles and
responsibilities by June 2013.
OP also concurs with the intent of OIG's second recommendation to "ensure that ADP-Tracker
has established clear roles, responsibilities, and requirements for the program offices to upload
development documents; and that the rules are updated in a timely manner, entries are monitored
for data quality, and all features available are used to evaluate the efficiency of the rule
development process." Along with the updated ADP TRACKER user guides, OP will issue a
memo clearly outlining the responsibility of program offices for uploading development
documents. We will also use this opportunity to ensure that program offices are working to
achieve to a high degree of data and record-keeping integrity in ADP TRACKER. However, we
believe the draft report incorrectly suggests that OP is responsible for maintaining records
associated with each rulemaking either in ADP TRACKER or otherwise, when it is, in fact, the
Agency policy for the originating office to maintain such records. As OIG states on page 6 of the
draft report, "program offices are required to maintain electronic records in the enterprise-wide
electronic content management system to allow for timely access and retrieval." We believe this
distinction concerning recordkeeping should be more clearly and consistently communicated in
OIG's final report and we suggest edits to this end.
OIG Response: The OIG acknowledges the Agency's concurrence with the first and
second recommendation. While it is the program offices' responsibility to maintain
records, the OIG notes that OP still has oversight responsibility over the ADP process and
program office responsibility does not diminish that oversight role.
With regard to OIG's third and final recommendation, OP disagrees with OIG's suggestion that
it "develop and implement a method to track resource use in key rulemaking activities." OP does
not have the expertise to develop or implement such a method. OP's resources and expertise are
best suited to overseeing the regulatory development process to ensure that the Agency produces
effective regulatory actions to protect human health and the environment. Monitoring the use of
13-P-0167
16

-------
internal Agency costs associated with rule development may fall under the purview of the Office
of the Chief Financial Officer or the relevant program office.
OIG Response: The OIG position is that, without the controls outlined in
recommendation 3, the Agency will continue to be challenged in assessing the efficiency
of the rulemaking process and therefore the recommendations stands. We consider this
recommendation unresolved. As required by EPA Manual 2750, the Agency will need to
initiate the audit resolution process.
OP welcomes the opportunity to discuss the comments and suggested revisions detailed here and
in the attached markup. If you or your staffs have questions regarding this response, please feel
free to contact Nathaniel Jutras at (202) 564-0301 or iutras.nathaniel@epa.gov.
Attachment
13-P-0167
17

-------
Distribution
Office of the Administrator
Associate Administrator, Office of Policy
Agency Follow-Up Official (the CFO)
Agency Follow-Up Coordinator
General Counsel
Associate Administrator for Congressional and Intergovernmental Relations
Associate Administrator for External Affairs and Environmental Education
Audit Follow-Up Coordinator, Office of the Administrator
Audit Follow-Up Coordinator, Office of Policy
13-P-0167

-------