United States
Environmental Protection
Agency
Oftica of
Inspector Generii
401 M Street SW
Washington OC 20460
Report of Internal and
Management Audit
Review of the Office of
Research and Development's
Extramural Research Activities
The primary mission of the Environmental Protection Agency's Office of
Research and Development (CRD) is to provide regulatory offices with
scientific information for use in developing and enforcing regulations.
0R0 accomplishes its mission through a combination of 1n-house research
and extramural research procured througn contracts, cooperative agree-
Tents, grants, ana interagency agreements.
The Office of the Inspector General's (OIG* review Indicated that ORO
has been faced with a number of problems aid criticisms over the last
few years and is current'y in the process of Instituting some inportant
operational and or;ani rational changes to effect improvements. Addi-
tional improvements, however, are needed to strengthen administrative
and internal controls and increase the eff1:lency and effect'veness of
the processes governing procurement and management of research projects.
The OIG has made 2 runber of recommendations to Agency management
to ennjnee JRD's operational and administrative effectiveness. Agency
management has generally concurred with these recommendations and have
taken or are in the process of taicing corrective action to address
problems identified.
Audit Report El gB2-l 1 -nm <5-30323
March 31, 1983
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, O.C. 20460
March 31, 1983
OFFICE OF
THe INSPECTOR GENERAL
MEMORANDUM
SUBJECT: Report of Audit of the Office of Research and Development's
Extramural Research Activities, Audit Report .ElgB2-11-0019-30828
We have completed an audit of the Office of Research and Development's .
Extramural Research Activities.
The Acting Assistant Administrator for Research and Development did
not respond to all appropriate recommendations contained In the report.
However, according to ORD senior management, ORD is in full agreement with
all of the report's recommendations and has either taken action to correct
deficiencies noted or is committed to proceed.
Due to mlscharges to the Research and Development and Abatement and
Control Appropriations totaling $1.3 million, we have recommended that
the Agency Comptroller review all contract charges to these appropriations
for fiscal 1980 through 1982, and present a formal written report to you
regarding the propriety of such charges. We are requesting the Office
of Legal and Enforcement Counsel, under a separate memorandum, to provide
us with a formal legal opinion on whether such mlscharges constitute a
violation of 31 U.S.C. 628 and/or 665, and what actions if any, you must
take. The Comptroller proposed to conduct a review of contract charges
for fiscal 1983, but disagreed with our recommendation to review charges *
for fiscal 1980 through 1982.
With the Exception of the Comptroller's position on review of contract
charges, the Agency's comments to the recommendations in our draft report
were responsive.
FROM: Ernest E. Bradley
Assistant Inspectoi
TO: Lee L. Verstandig
Acting Administrator (A-100)
-------
-2-
We request that within 120 days, you provide us information concerning
the actions undertaken to implement each recommendation contained in
this report.
Should your staff have any questions concerning the report, please
contact me at 382-4106.
-------
TABLE OF CONTENTS
Page
ABBREVIATIONS 1v
PART I-DIGEST 1
PURPOSE AND SCOPE OF REVIEW 1
BACKGROUND 4
SUMMARY OF REVIEW RESULTS ' 5
MANAGEMENT RESPONSE 10
Olb COMMENTS II
PART II—REVIEW RESULTS 12
CHAPTER 1 - PROGRESS AND PROBLEMS
IN MANAGING EXTRAMURAL RESEARCH 12
SCOPE AND METHODOLOGY 13
BACKGROUND 13
PROBLEMS AND COMPLEXITIES IN
MANAGING RESEARCH AND DEVELOPMENT ACTIVITIES 15
PROBLEMS AFFECTING ORD'S ABILITY TO PLAN RESEARCH 17
RECENT PLANS AND ACTIONS TO IMPROVE ORD OPERATIONS 19
OTHER IMPORTANT ISSUES TO BE CONSIDERED 24
RECOMMENDATIONS 27
MANAGEMENT RESPONSE 28
OIG COMMENTS 28
CHAPTER 2 - ADMINISTRATION AND CONTROL
OF COOPERATIVE AGREEMENTS 29
SCOPE AND METHODOLOGY 29
BACKGROUND 30
PROJECT OFFICER CONTROL OVER THE REVIEW
AND SELECTION OF COOPERATIVE AGREEMENTS 30
IN-HOUSE REVIEWS PERFORMED
3Y PROJECT OFFICERS 33
\
RESOLUTION OF ISSUES RAISED
BY EXTERNAL PEER REVIEWERS 34
-i -
-------
TABLE OF CONTENTS (Continued)
Page
IMPACT OF LIMITED TRAVEL FUNDS ON
COOPERATIVE AGREEMENT MONITORING 35
CONFLICT-OF-INTEREST STATEMENTS 37
RECOMMENOATIONS 38
MANAGEMENT RESPONSE 39
OIG COMMENTS 39
CHAPTER 3—RESEARCH AND
DEVELOPMENT CONTRACTS 40
SCOPE AND METHODOLOGY 41
BACKGROUND 41
TIMELY RECEIPT OF FINAL
RESEARCH PRODUCTS 42
CONTRACT FUNDING 45
CONTROL OF LEVEL-OF-
EFFORT CONTRACT FUNOING 52
IMPROVEMENTS UNDERWAY FOR PROCUREMENT
AND MANAGEMENT, OF CONTRACTS 53
RECOMMENDATIONS 55
MANAGEMENT RESPONSE 56
OIG COMMENTS 57
CHAPTER 4—MANAGEMENT OF
INTERAGENCY AGREEMENTS 59
SCOPE AND METHOOOLOGY ; 60
BACKGROUND 60
ACTIONS NEEDED TO FINALIZE
AND IMPROVE IAG PROCEDURES i 61
RECEIPT ANO REVIEW OF FINANCIAL
INFORMATION BY PROJECT OFFICERS AND
ACCOUNTING FOR IAG COSTS 53
PROJECT MONITORING AND
ADMINISTRATION 65
RECOMMENDATIONS 68
MANAGEMENT RESPONSE ... .69
-i 1-
-------
TABLE OF CONTENTS (Continued)
Page
016 COMMENTS 70
CHAPTER 5—COMPETITIVE GRANTS
PROCESS AND ACTIVITIES 71
SCOPE AND METHOOOLOGY 72
BACKGROUND 73
GRANT APPLICATION REVIEWS'
AND FUNOING DECISIONS 75
SRA WORKLOAD AND GRANT MONITORING • 76
TRACKING LONG-TERM RESEARCH FUNDING 79
RECOMMENDATIONS 81
MANAGEMENT RESPONSE 82
OIG COMMENTS v 82
APPENDIX A—Office of Research and Development
Organization Chart 83
APPENDIX B—Listing of ORD Research Committees 84
APPENDIX C—Days Required to Complete
Contract Award Process . 85
" APPENDIX D—Amount of Time Taken to
Obtain Final Reports 86
APPENDIX E—Contracts and Amounts Charged to
Object Class 2532 (Research and
Development Contracts) Instead of
to Object Class 25.35 (Program Contracts) 87
APPENDIX F—Illustration of Fiscal 1981
Funding for Subagreement with
Oak Ridge National Laboratory 88
APPENDIX G—Response from the Acting Assistant Administrator
for Research and Development 90
APPENDIX ^--Response from the Assistant Administrator
for Adminl strati on 94
APPENDIX I—Preliminary and Final Responses
from the Comptroller. 100
-111-
-------
ABBREVIATIONS
ORD ' Office of Research and Development
RTP Research Triangle Park, Durham, North Carolina
IAG Interagency Agreements
OER Office of Exploratory Research
6A0 United States General Accounting Office
FMD Financial Management Division
GAD Grants Administration Division
ORDIS Office of Research and Development Information System
IERL Industrial Environmental Research Laboratory
OEPER Office of Environmental Processes and Effects Research
PCMD Procurement and Contracts Management Division
-i v-
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
OFFICC OP
rwe INSPECTOR GENeRAi.
Audit Report ElgB2-11-0019-30828
Report of Review of the
Office of Research and Development's
Extramural Research Activities
PART I—DIGEST
PURPOSE AND SCOPE OF REVIEW
We have completed a' review of the Office of Research and Development's
(ORDs) extramural research activities. The purpose of the review was to:
(1) Identify Important Issues and problems facing ORD, as well as
their historical perspective;
(2) Identify recent key actions taken by ORD to address major
problems and Institute Improvements;
(3) Determine whether the results of research projects were meeting
the high priority needs of the agency; and
(4) Determine the adequacy of administrative and internal controls
and the efficiency and effectiveness of the process governing
the procurement and management of projects financed through
cooperative agreements* contracts, Interagency agreements, and
grants.
Our field work was conducted from March 15, 1982, through May 28, 1982.
We held a preliminary exit conference with the Acting Assistant Administrator
for Research and Development and his office directors 1n June 1982. We
also discussed matters contained in this report with the director of the
Office of Fiscal ana Contracts Management ana his staff. Additional
foilowup work continued at Headquarters through early August 1982.
-------
-2-
Our scope Included:
(1) Interviews with officials and personnel of ORD, Office of
General Counsel, Grants Administration Division, Procurement
and Contracts Management Division, Office of Personnel and
Organization, Financial Management Division, and Headquarters
media program offices;
(2) Review of applicable laws, regulations, policies, procedures,
and other documents;
(3). An examination of files and records pertaining to selected
cooperative agreements, grants, contracts, and Interagency
agreements; and
(4) An examination of budget and financial data, reports, and
studies pertaining, to ORD.
We conducted the review at EPA Headquarters and at the environmental
research centers located at Research Triangle Park (RTP), North Carolina,
and Cincinnati, Ohio. We selected laboratories within these centers
because they represented the largest portion of ORD's budget and had a
large number of research projects in support of several different media
program areas including air, water, and toxic substances.
"Our review comprised two major components: (1) an overview of ORD and
(2) an analysis of controls and the efficiency and effectiveness of 1
the process governing procurement and management of selected extramural
research projects. The former component was basically a broad-based
survey of ORD's progress and problems In planning and managing extramural
research. In contrast, the latter component, which was the thrust of our
review, was broken down Into a more detailed assessment of four areas:
cooperative agreements, contracts, interagency agreements, and grants.
The following is a brief discussion of the scope and methodology for
each major component.
Overview
In performing an overview of ORD, we focused on its current operations,
key issues, and problems, as well as recent plans for improving its
organization and operations. We prepared a series of questionnaires
which were used to conduct extensive interviews with the Acting Assistant
Administrator for Research and Development, ORD Headquarters office
directors and division directors as well as laboratory directors and
their staffs. Information obtained during these inter/lews was to gain
background and persoectlve on ORD, and represented the opinions and
beliefs of those interviewed.
-------
-3-
Review of Extramural Research Projects
Our review of the procurement and management of extramural research was
based on a judgmental sample of cooperative agreements, contracts,
Interagency agreements, and grants (instruments) funded in fiscal 1981
and the first quarter of fiscal 1982. The following is a breakdown
of the universe and sample size for each of the above areas.
UNIVERSE'
Dol lar
Amount
Number (1n mi11 ions)
SAMPLE
Dol lar
Amount
Number Percent (in mi 11ions) Percent
Cooperative
Agreements
486
$ 53.9 1/
. 49
10.0
$ 7.4
13.7
Grants
102
15.6
20
19.6
5.3
33.9
Contracts
487
283.9
85
17.4
71.2
25.0
Interagency
Agreements
155
$ 43.2 U
26
16.7
$32.4
75.0
1/ This includes only fiscal 1981 funding due to
limited funding "in "the first quarter of 1982.
In selecting these samples, we considered the dollar value of the
instruments and the level and type of activities being performed by
various laboratories at the two locations selected.
Although one of our four review objectives was to determine whether
research results had met the h1gh-pr1or1ty needs of the agency, we were
unable to offer an overall opinion on the issue. During the early stages
of the review, we recognized that such an undertaking would be most
difficult, requiring substantial assistance from Independent scientific
and technical staff. Jn addition, ORD's computerized management infor-
mation system did not accumulate the kind of detailed information necessary
to relate the results of specific laboratory research projects back to
ORD's research strategies and plans. Without this Information, 1t wouldv
have been extremely time consuming—essentially involving manual review
of hundreds of projects and tasks, examination of numerous complex planning
documents, and interviews with many different persons inside and outside
of ORD—to assess how research results either related to plans or benefited
EPA's various media programs.
ORD 1s sensitive to this problem, however, and was in the process of
changing Its management information system to better relate researcn
results back to research strategies and plans.. (See page 22 of our
report for a discussion of such changes.)
-------
-4-
BACKGROUND
The primary mission of the ORD is to provide regulatory offices with
scientific information for use in developing and enforcing regulations.
ORD is one of six major components of the Agency, each of which is headed
by an Assistant Administrator reporting directly to the Administrator.
ORD is organized into seven Headquarters offices consisting of five
scientific or technical offices, an administrative office, and an
office of exploratory research; and fourteen geographically dispersed
laboratories,.responsible for conducting intramural research, as well as
administering a large amount of extramural research (see appendix A ).
ORD's research and development activities generally fall into three basic
categories: (1) shorter term (1-2 years) regulatory related research,
which responds to specific regulatory needs; (2) longer term (3-5 years)
regulatory research, which supports planned program or operational require-
ments but does not address immediately planned regulatory actions; and
(3) exploratory research, which 1s conducted primarily to develop fundamental
knowledge and principles to solve current problems or identify or understand
future environmental problems for which no specific regulatory activity
is currently planned.
ORD accomplishes its mission through a combination of in-house research
and extramural research procured through contracts, cooperative agreements,
competitive grants, and interagency agreements. At any given time, ORD
manages over 2,000 research tasks and projects through several Head-
quarters offices, field administrative offices, one research information
center, and its laboratories.
A number of evaluations of ORD's research program have been made over the
last few years. These evaluations were conducted by congressional committees,
the General Accounting Office, the Office of Technology Assessment, and
the National Academy of Sciences, as well as by EPA itself. The problems
identified as a result of these evaluations included:
(1) A perceived lack of responsiveness to EPA's priority research
needs;
(2) Ineffective and unreasonably complex planning processes;
(3) Excessively cumbersome and top heavy management;
(4) A failure to.properly balance long-term research against short-
term, problem-solving scientific and technical activities; ana
(5) A.lack of consistent scientific quality.
-------
-5-
SUMMARY OF REVIEW RESULTS
Overall, our review disclosed that ORD has been faced with a number of
problems and criticisms over the last few years, and is instituting some
major operational and organizational improvements. However, we found
that additional Improvements are still needed to strengthen administrative
and Internal controls and increase the efficiency and effectiveness of
the processes governing procurement and management of research projects.
Our findings are summarized below and presented in detail in Part II of
the report.
Progress and Problems in Managing Extramural Research (See Chapter 1)
The Office of Research and Development is currently making organizational
and operational improvements to address problems concerning the respon-
siveness, timeliness, and credibility of its research activities. Relatively
recent changes 1n progress include implementation of an improved ORD-wide
process for planning research and development, with emphasis on a "top-down"
policy of management accountability and controls; actions to improve
research quality and results by implementing peer review programs and
quality assurance mechanisms; establishment of a new system of management
information and program documentation; and initiation of a wide-ranging
reorganization of both ORD Headquarters offices and laboratory operations.
Our review indicated that, contemplated changes appear to address some of
the major criticisms. We also believe that these changes,appear to be
reasonable and, 1f fully Implemented, offer promise for Improvement.
However, since these changes were not fully implemented, and written
policies and procedures effecting these changes were not finalized during
our review, we were unable to examine such changes in-depth, and therefore,
cannot offer an opinion on them.
Our review also disclosed several issues which we believe warrant
management's consideration in planning and implementing contemplated
changes. These issues include (1) the importance of appointing a
permanent Assistant Administrator for Research and Development; (2) the
need to improve ORD's current system for disseminating policies and
procedures guiding its operations; (3) a need to improve communication
between ORD researchers and media program offices; and (4) concerns
expressed by many ORD officials about_the negative impact that limited
travel funds have had on research project monitoring, training, and
professional development. We believe these issues should be considered
because to be supportive, research must withstand continuous and compre-
hensive scrutiny. Thus, ORD must maintain an organizational climate
which rrot only ensures quality research, but also responds to changing
public concerns and shifting legislative and executive directives.
-------
-6-
We recommend that the Administrator continue efforts to appoint a
permanent Assistant Administrator for Research and Development. We
also recommend that the Acting Assistant Administrator for Research
and Development (1) adopt an effective mechanism to disseminate ORD
policies and procedures;. (2) vigorously pursue ORD's objectives for
implementing Improvements and changes 1n its computerized management
information system; (3) reemphasize the need for effective communication
between ORD and media program offices; and (4) evaluate the adequacy of
ORD's existing and planned travel funds, as well as their allocation, in
meeting its operational needs, Including project monitoring and employee
development and training. (See page 27.)
Administration and Control of Cooperative Agreements (See Chapter 2)
In fiscal 1980, ORD issued interim procedures to control and administer
cooperative agreements. Although these procedures are a further step
in improving controls over cooperative agreements, we believe they still
need to be Improved and existing procedures fully enforced. Our review
of 49 projects (cooperative agreements) and interviews with 44 project
officers disclosed that project officers appeared to have excessive
control over certain aspects of the cooperative agreement system. Project
officers frequently selected external peer reviewers and were allowed to
prepare 1n-house reviews and important decision memorandums. We believe
this gives the appearance of a less-than-objective process and could
provide an opportunity for someone to influence the selection of recipients
for project funding. Similarly,, the General Accounting Office, in an
October 1980 report, recognized that opportunities existed in the cooperative
agreement system for biased judgments. In addition, we found that: (1)
issues raised by external peer reviewers had not been adequately resolved
and documented, (2) some project officers believed monitoring could be
more effective if additional travel funds were available to make research
site visits, and (3) external peer reviewers had not always completed
required conflict-of-interest statements in conjunction with their review
of cooperative agreement applications.
We recommend that the Acting Assistant Administrator for Research and
Development take steps to ensure that: project officers are prevented
from having excessive control over aspects of the cooperative agreement
system; issues raised by external peer reviewers are adequately resolved
and documented; travel funds are sufficient to foster effective monitoring;
and conflict-of-interest statements are obtained from all peer reviewers.
(See pages 38 and 39.)
Research and Development Contracts (See Chapter 3)
Ue reviewed && active and 41 completed research and develODment (R&D)
contracts for fiscal 1981 and the first quarter of 1982. These contracts
were administered by ORD laboratories ac a TP, .North Carolina, and
Cincinnati, Ohio. We found that for the sample we reviewed:
-------
-7-
(1) Contract end products (final reports) were not always received
in a timely manner, and in a significant number of the cases
we reviewed the end products had not been received at all. In
our opinion, the primary reasons for such delays were:
(a) The long lead-time required to complete the procurement
process; and
(b) Failure of Individual contractors to deliver end products
within the time frames prescribed in individual contracts.
(2) In six instances amounting to $3,673,245, it appeared initially
that the R&D appropriation was used to fund contracts or portions
of contracts that provided management and other types of services
which, in our opinion, should have been funded from the S&E
appropriation. The Agency Comptroller agreed that four of
these six contracts amounting to $1,313,339 should have been
charged to the S&E appropriation. (See page 47). Similarly,
1n two other Instances amounting to $10,494,314, the R&D appro-
priation was used to fund contracts which, in our opinion,
provided materials and a multitude of services to operate or
otherwise support government-owned facilities at RTP and Cincinnati.
However, because funding sources were not linked to specific
contract work tasks we were unable to determine the accuracy
of charges to the R&D and other appropriations. The Acting
Assistant Administrator for Research and Development pointed
out that the ambiguities contained 1n the definitions of each
appropriation are such that the above contracts could be charged
to either the R&D or S&E appropriations (See page 46).
(3) In six instances amounting to about $6.8 million, it appeared
that program support contracts or portions of contracts were
incorrectly classified as R&D contracts, which prevented an
accurate comparison of actual expenditures to the amounts
budgeted in the Agency's resource management information system.
(4) Financial controls over level-of-effort contracts funded by
multiple program elements were not sufficient to ensure that
specific tasks performed were related to the program elements
used.to fund them.
We discussed the results of our review with the Acting Assistant Admini-
strator for Research and Development and ORD office director?. They
generally agreed with the results of our review. We also discussed the
results of our '-svisw with the director of the Office of Fiscal and
Contracts Management and senior members of his staff, and discussed the
funding asDects with the Agency Comotroller and senior members of his
staff. Although we rioted areas for improvement, we recognize that in
fiscal 1982, the Agency took a number of positive actions to ensure that
(1) major procurements were better managed, (2) procurements were necessary
to achieve a specific policy objective, and (3) funding authority was
-------
-3-
available and appropriate for the intended procurement. It appeared
that if effectively implemented, these corrective actions should prevent
or reduce future occurrences of most of the problems we noted. Specific
Agency improvements are commented on in various sections of Chapter 3.
We plan to request the Office of Legal and Enforcement Counsel to provide
us with a legal opinion on whether the $1.3 million charged to the wrong
appropriations violated 31 U.S.C 628 and/or 665, and if so, what actions
must be taken by the Agency Administrator. In general, 31 IJ.S.C. 62fl,
requires that appropriated funds can be used only for purposes appropriated,
while 665 provides that obligations cannot be made in excess of amounts
appropriated.
We recommend that the Acting Assistant Administrator for Research and
Development take appropriate action to improve internal controls over
contract procurement, monitoring, and funding. We also recommend that
the Comptroller's office review and report to the Administrator on the
propriety of contract charges for fiscal 198H, 1981, and the first quarter
of 1982, and, adjust Agency obligation records as necessary. Finally,
we recommend that the Assistant Administrator for Administration direct
the Office of Fiscal and Contracts Management to review and strengthen
controls over level-of-effort and other term form contracts to ensure
that work tasks are more directly identifiable ta contract funding (see
pages 55 and 56).
Management of Interagency Agreements (See Chapter 4)
In the past, due to generally limited involvement by EPA in monitoring
interagency agreements (IAGs), and the apparent lack of guidance and
training for ORD project officers, there was little assurance that such
agreements were effectively administered and managed, or always benefited
EPA1s high-priority needs. In the last few years, however, EPA.has
taken some positive actions to improve the overall administration and
management of IAGs. Nevertheless, project management and administrative
controls still need to be improved.
We found that although the Financial Management Division (FMD) had developed
new IAG procedures in draft form, these procedures have not been finalized
and can be further strengthened. The Office of Administration recently
transfered responsibilities for IAG administration from the Financial
Management Division to the Grants Administration Division, which is
charged with issuing final IAG procedures.
-------
-9-
Our review disclosed that project officers, in many instances, did not
require submission of detailed project cost information from other agencies.
We believe this Information would have enabled project officers to (1)
ensure that work was performed 1n accordance with IAG terms and (?.)
determine whether billings were commensurate with the progress of the
work. The absence of this cost information precluded ORG project officers
from providing reasonable assurance that the other Federal agencies
complied with IAG terms and authorized project funding.
We also found that some project officers had not received adequate guidance
and training and consequently were not aware of all their responsibilities
to review financial information and to maintain IAG files properly. We
believe that if IAG project officers are allowed to attend the Office of
Administration's Project Officer Certification Course, they should be
made more fully aware of their responsibilities to manage and administer
their projects effectively. Some ORD officials also cited lack of staff
and travel funds as adversely impacting project management and monitoring.
We discussed these matters with officials of the Office of Administration,
including the director of the Office of Fiscal and Contracts Management,
the director of the Grants Administration Division, and a senior represen-
tative of the FMD. These officials agreed, 1n general, with our review
results.
We recommend that the Acting Assistant Administrator for Research and
Development take action to Improve IAG management and monitoring by:
providing necessary guidance to project officers concerning their responsi-
bilities for receipt and review of detailed project cost information and
file maintenance; ensuring project officers receive adequate traininq;
and evaluating the adequacy of travel funds in ensuring effective project
monitoring. We have also recommended that the Assistant Administrator
for Administration finalize new IAG procedures as expeditiously as possible
to ensure that media program offices and project officers are provided
with necessary guidance regarding their responsibilities for effective
administration of IAGs (see pages 68 and 69).
Competitive Grants Process and Activities fSee Chapter 5)
The Office of Research Grants and Centers' (ORGCs) process for reviewing
and administering competitive grants appeared generally adequate.
Furthermore, our examination of the overall summary scoring, ranking,
and resulting funding decisions of four peer review panels during fiscal
1981 disclosed that the most highly ranked grant applications were
usually the ones funded. However, our review also disclosed that there
were some areas ORD still needed to address ig improve ORGC operations.
ORD should:
(1) Require ORGC Science Review Administrators (SRAs) to maintain
readily available documentation to support grant application
ranking and funding decisions;
-------
-10-
(2) Consider the appropriateness of adopting a formal ORD-wide
mechanism to track, by project, long-term environmental research
and development funding;
(3) Evaluate the existing workload of SRAs and its impact on
their ability to effectively monitor grant projects.
We discussed the above areas with the director of Office of Exploratory
Research (OER). He agreed, 1n" general, that the above improvements were
needed. We also explained to h1m that because SRAs did not maintain
readily available documentation to support grant decisions on funding
applications, we were unable to determine the accuracy or appropriateness
of some decisions. Therefore, in an effort to further assess ORfiCs'
process, we requested that the director of OER review selected grant
applicaton decisions we could not fully verify during our review of
three peer review panels for fiscal 1981. In late October 1982, the
director of OER provided an explanation of funding decisions, along with
additional supporting documentation. Based on our review.of this information
and followup discussions with the director, it appeared that the subject
funding decisions were appropriate.
In addition, we met with the Acting Assistant Administrator for Research
and Development in late August 1982. He informed us that he recognized the
need to evaluate ORGC's current operations and that ORD was conducting a
comprehensive review of ORGC, focusing on such areas as (1) administrative
controls over the peer review process, Including the selection of peer
panel chairpersons; (2) SRAs' responsibilities and the adequacy of controls
governing their responsibilities; (3) effectiveness of grant monitoring;
and (4) adequacy of the ORD office directors' relevancy review. This
review is expected to be completed in March 1983..
We recommend that the Acting Assistant Administrator for Research and
Development (1) ensure that SRAs maintain adequate supporting documentation
regarding peer panel funding decisions; (2) consider establishing a formal
0RD-w1de mechanism to track long-term environmental research and development
funding by project; and ensure that, to the extent possible, projects
are evenly distributed among the SRAs to ensure effective monitoring
(see page 81).
MANAGEMENT RESPONSE
We provided a copy of our draft audit report dated September 9, 1982, to
both the Acting Assistant Administrator for Research and Development and
the Assistant Administrator for Administration for. their detailed review
and consideration. Both of these officials generally agreed with our
findings and recommendations. They stated that actions had been taken or
would be taken to address our recommendations. In addition, they provided
us with important comments and editorial suggestions which were i ncorporated
after pertinent findings 1n the body of the report. A complete copy of
these responses is included in the report as appendixes G and H, respec-
tively.
-------
-11-
We also discussed our conclusions regarding contract funding with the
Agency Comptroller and senior members of his staff and provided him with
a copy of our draft report on March 4, 1983. In his written response to
the report, the Comptroller agreed that four contracts amounting to about
51.3 million had been Incorrectly charged to the R&D appropriation. The
Comptroller, however, disagreed with our recommendation that his office
conduct or direct a review of contracts funded from fiscal 1980 through
the first quarter of 1982, because he believed such a review would not be
fruitful (see page 104). Nevertheless, he agreed that because contract
mischarges may be continuing, the Agency should examine all R&D contracts
for fiscal 1983. The Comptroller also provided us with other consents on
our findings which we incorporated in our final report as necessary. A
complete copy of the Comptroller's written response 1s included as
Appendix I.
PIG COMMENTS
We have considered comments provided by the Comptroller's Office and,
where we considered it appropriate, revised the report to reflect such
comments. However, we strongly disagree with the proposal by the Office
of the Comptroller to review only fiscal 1983 contracts for mischarges.
In our opinion, the fact that over 51 million was charged to the wrong
appropriations, which may be a violation of 31 U.S.C. 628, is sufficient
reason to conduct such a review, regardless of how the amount of such
mischarges relate to the total value of the contracts in our sample.
-------
-12-
PART II— REVIEW RESULTS
CHAPTER 1
PROGRESS AND PROBLEMS IN MANAGING EXTRAMURAL RESEARCH
The Office of Research and Development (ORD) has long been plagued by
problems and criticisms concerning the responsiveness, timeliness, and
credibility of its research activities. In response to past criticisms,
ORD Initiated a number of actions to address the problems. ORD recognized
that key problems still existed and at the time of our review was in the
process of making additional, broad, organizational and operational changes
Intended to address them. Important changes planned or in progress included:
(1) Implementing a new 0RD-w1de process for planning research and
development which emphasized a "top-down" policy of management
accountability and controls to achieve planned objectives, and
was driven by the development of broad research strategies
(see pages 19 and 20);
(2) Implementing actions to Improve research quality and results by
starting peer review programs and quality assurance mechanisms
(see pages 20 and 21);
(3) Establishing a new system of management Information and program
documentation designed to monitor accomplishment of planned
objectives and Improve project accountability (see pages 22 and
23); and
(4) Initiating a wide-ranging reorganization of both ORD Headquarters
offices and laboratory operations (see pages 23 and 24).
Our review also disclosed the following issues which we believe warrant
consideration in planning and implementing contemplated changes:
(a) The importance of appointing a permanent Assistant Administrator
for Research and Development;
(b) The need to establish a more effective system for disseminating
ORD's operating policies and procedures;
(c) The need to Improve communication within ORD and between
ORD and other program offices (such as pesticides, toxic
substances, water, etc.); and
(d) The belief of many ORD officials and employees that limited
travel funds have had a negative impact on monitoring research
projects, training, and professional development.
-------
-13-
Our review disclosed that ORD's proposed changes appeared reasonable
and, if effectively implemented, offered promise for improvement. However,
since these changes had not been fully implemented, and written policies
and procedures reflecting such changes had not been completed during our
review, we did not review them and we are not offering an opinion on
them.
SCOPE AND METHODOLOGY
Our review of EPA's research program focused on operations, areas identified
as problems, and ORD's plans for Improving its organization and programs
to correct these problems. The purpose of this part of the review was
to obtain a broad overview of EPA's progress and problems in planning
and managing extramural research.
In performing this review, we prepared a series of questionnaires which
were used to conduct extensive interviews with the Acting Assistant
Administrator for Research and Development, ORD office directors,
Assistant Administrators, laboratory directors at RTP and Cincinnati,
and other senior ORD officials. We also examined prior studies conducted
by EPA and other organizations, and reviewed policies, procedures, and
other documentation governing ORD's organization and operations.
BACKGROUND
The Environmental Protection Agency's mission is to systematically abate
and control pollution through ah integrated program of research, monitoring,
standard setting, and enforcement activities which 1s aimed at protecting
human health and ensuring environmental quality. The ability of the
Agency to perform these congressionally mandated tasks depends to a
great extent upon the ability of its scientific staff to (1) analyze and
interpret the implications of scientific and technical data, (2) provide
information considered necessary to support the basic decision making
process, and (3) develop and enforce Agency regulations. Because of the
regulatory and consequently sensitive nature of EPA's mission, research,
to be supportive, must withstand continuous and comprehensive scrutiny.
Responsibility for anticipating the need for and providing the wide range
of quality scientific data required to develop and enforce regulations
rests principally with ORD, as well as with program offices.
-------
-14-
Organization, Responsibilities, and Funding
The Office of Research and Development is one of six components of EPA,
each headed by an Assistant Administrator reporting directly to the
Agency Administrator. It performs multidisciplinary research funded by
numerous congressional acts in each area over which EPA has regulatory
authority, (e.g., Clean Air Act, Clean Water Act, Solid'Waste Disposal
Act). In general, ORD conducts research and development to (I) support
decision making, (2) anticipate future problems, and (3) advance basic
environmental science horizons. ORD accomplishes its mission through a
comblnation of both its own in-house research staff and an extensive
program of extramural research obtained by utilizing contracts, cooperative
agreements, competitive grants, and interagency agreements. As of
March 31, 1982, ORD had a total of about 1,563 permanent full-time
employees and 507 temporary employees.
ORD's current organization couples Its disciplinary approach to research
and development with the program offices through a media orientation of
the Agency's research committee system. ORD's present organizational
framework is the product of an evolutionary series of reorganizations
and consolidations designed to improve its management and enhance
responsiveness to changing Agency priorities.
Total funding levels requested in the President's budget for ORD have
decreased significantly in total from fiscal 1981 through 1983. the
largest portion of this.decrease has been in the research and development
appropriation, which provides funds for ORD's extramural research
activities. As shown below, ORD's total funding declined about 44 percent,
from about $373.9 million in fiscal 1981 to $209.5 in fiscal 1983.
Further, the research and development appropriation declined about 60
percent, from $270.4 million in fiscal 1981 to $108.7 million in fiscal
1983. In contrast, however, the salaries and expense appropriation
remained relatively constant during the same period.
Total ORD Funding Levels based on
the President's Budget (thousands)
Percentage
Decrease
FY 1981 FY 1982 FY 1983 l^fil to 1§83
Total Salaries—
and Expense
Appropri ation
$103,553.1 $115,599.9 $100,844.8
2.6
Total Research
and Development
Appropri ation
270,383.4 190,635.0 108,703.8
59.8
Totals
$373,936.5 $306,234.9 $209,548.6
44.0
-------
-15-
PROBLEMS AND COMPLEXITIES IN MANAGING
RESEARCH AND DEVELOPMENT ACTIVITIES""
As early as 1977, EPA Identified Inadequate research planning and manage-
ment systems as the primary reasons for ORD's failure to meet Agency
needs for quality, responsive, and timely research. These problems led
to additional congressional hearings, as well as to reviews by the General
Accounting Office (GAO) and independent studies by the National Academy
of Sciences and the Congressional Office of Technology Assessment. In
addition, the Agency conducted internal reviews and contract studies
which also confirmed such problems. Reports resulting from these reviews
generally repeated major criticisms concerning the quality, responsiveness,
and timeliness of EPA1s research and development. The criticisms were
commonly acknowledged as perennial by several senior ORD officials we
Interviewed. In the past, EPA's response to these criticisms Involved
numerous reorganizations and realignments of the planning and management
process. We found that although positive progress had been made, some
problems still existed.
In part, the problems not only reflected faults of the ORD, but also
reflected the complexities of (1) environmental policy, (2) externally
Imposed constraints, „and (3) expectations over which EPA in general and
ORD-in particular had little or no control.
Envi ronmental Pol 1cy
In the 12 years since Congress created EPA as a regulatory agency with
the mandate to protect human health and the environment, progress has
been made in achieving environmental goals. However, due to the com-
plexities of environmental problems, meeting these goals continues
to be difficult. This 1s further complicated by having to deal with
unanticipated and unpleasant environmental crises and changing public
perceptions of environmental problems.
To the extent that ORD provides the scientific and technical data in
support of these critical decisions, it forms an important part of the
Agency's operations. Any attempt at an objective review of ORD's activities
must acknowledge the changing politics, science, regulations, and public
concern affecting the evolution of environmental policy.
-------
-16-
External Constraints
Numerous externally Imposed constraints limit ORD's planning and managment
of environmental research and development, Including:
(1) The Congressional policy-making process;
(2) The lack of compatibility between the Federal budgetary pro-
cess and the R&D program planning and Implementation process,
resulting because Congress appropriates funds for Agency resources
on an annual cycle while research strategies, plans, programs,
and projects are most often prepared and implemented on a
multiyear basis;
(3) The Federal procurement process; and
(4) C1v1l service regulations which tend to create a relatively
inelastic mix of 0R0 in-house scientific and technical personnel.
Differing Expectations
V
Differing expectations (perspectives, orientations, and interests)
produce chronic tensions and frequent confrontations between researchers
and regulators which are not readily (and in some cases may never be)
overcome. Regulatory personnel are generally results-oriented and possibly
less sensitive to the time demands of research, whereas scientists are
process oriented and possibly less sensitive to providing specific
results to meet regulatory deadlines.
Researchers approach problems from a long-term perspective, and thus
require long lead-times to complete a project. Researchers also tend to
resist stopping or dropping a project to refocus efforts as required by
regulators. In contrast, regulators are frequently faced with immediate
and often inflexible deadlines established by legislation, court decisions,
or unanticipated occurrences, and are interested 1n timeliness and utility.
The inherent tensions 1n this relationship may never be fully resolved.
In our opinion, with proper communication and education, both regulators
and researchers can achieve a greater appreciation of each other's needs
and concerns- and the reasonable balance necessary to best support the
Agency's regulatory mission.
-------
PROBLEMS AFFECTING ORD'S ABILITY TO PLAN RESEARCH
The principal EPA mechanism for planning its research and development
program is the research committee system. £PA has had several years of
experience operating under the research committee system, which 1s pri-
marily responsible for: (1) developing multiyear research strategies;
(2) reviewing research plans; (3) participating in the development of
research budget plans; and (4-)' reviewing ongoing and recently completed
research. As of June 30, 1982, there were 13 research committees in
existence (see appendix 8). Each research committee was cochaired by a
senior manager from ORD and an official from the corresponding program
office. Participants included managers with technical and scientific
background from EPA program offices. A primary output of these research
committees is a strategy document jointly developed by ORD and program
offices which address ORD program goals.
From our interviews with ORD and program officials and staff, we
received numerous comments and concerns about the research committee
system, planning and budget cuts, and other related areas involving
research planning. These are presented in detail below.
Comments on the Research Committee System
Interviews with ORD and program office personnel resulted in diverse
opinions concerning (1) the success of the research committee system
in meeting its stated responsibilities and (2) how the system should
operate to accomplish its responsibilities more effectively-
"Most of those we interviewed believed that by creating the research
committee system, EPA management elevated previously informal communications
between ORD research and program staff to a higher and more formal level.
They generally believed that this helped to eliminate some of the previous
frustrations they experienced due to the lack of assurance that such
interchanges would result in program office needs being recognized and
addressed by the researchers. The system enabled program offices to be
brought into the planning process 1n a more formal manner, resulting in
research strategies and priorities being arrived at on a consensus basis.
In our opinion, the fairly flexible procedures governing the research
committee system also serve to a great extent to Increase trust and
communication between researchers and program personnel by allowing than
to establish their own research strategies and work out their own problems.
A number of program officials told us that because research committees
were only advisory in nature, they could not ensura that all arogram
office priorities and programs would be addressed. However, they agreed
that the top priorities of program offices were being met by ORD through
the research committee system.
-------
-18-
Impact of Funding Cuts on Planning Research
Research committees meet at the beginning of the fiscal year planning
period with an estimate of the resources available for each program.
This estimate is then used to prepare decision units related to the
research strategies which have been developed. According to program
offices and some ORD officials, the process can develop problems, however,
when a higher level (i.e., the Assistant Administrator for Research and
Development, 0MB, or Congress) initiates funding cuts or redirections in
the planned program. Although research committees have input into the
process, the final decisions on the impact of the budgetary change on
the planned program occur at the ORD Headquarters level (office directors,
Assistant Administrator). As a result, there is no assurance that all
research committee program strategies will be accomplished. According
to one senior ORD official, this 1s an example of the "normal" tension
that exists 1n the system. This official stated that the Acting Assistant
Administrator for Research and Development 1s responsible for the research
program, and as such, has an agency-wide perspective and the responsibility
to advise the Administrator on R&D funding shifts. He also stated that
when such shifts are made, specific research committees (or ORD program
managers) whose areas have been cut sometimes believe they have lost
control and have been victimized.
Other Problems in Planning Research
Another problem in planning the research program is the relatively long-
term (typically three to five years) nature of research versus the annual
budget process. There was a consensus among those interviewed that it
was difficult to plan years ahead when the current year (fiscal 1982)
had not been finalized and fiscal 1983 plans were just being initiated.
Other problems which were brought to our attention by ORD research and
program staff included:
(1) A lack of flexibility in allocating funding to cover unanticipated
problems. Thus, when "crises" situations occurred, they tended
to disrupt planned programs. In addition, program office
officials told us they were reluctant to acknowledge international
research commitments (e.g., World Health' Organization, NATO)
for Inclusion in the research program because they believed
such commitments lacked relevance to their own program needs.
(2) The reluctance of program offices to assist in preparing long-
term strategies and needs because of the pressures to respond
zo more immediate problems and the frequent uncertainty as to
the future direction of their program.
(3) The inability of program offices to present consistently a clear
agenda of their needs to ORD. Program officials agreed that
they should work more diligently to present well-thought-out
needs during research planning sessions.
-------
-19-
(4) The assignment of lower level personnel to attend research
committee strategy development sessions. These employees
often did not possess the knowledge or the decision-making
authority to deal with strategies being developed at these
sessions.
(5) The number of research committees and the way they were set
up related to the regulatory programs, but did not complement
the multimedia, interdisciplinary requirements of many of the
research strategies, programs, and projects.
(6) Program offices which wished to control some of the research
program funds for projects they would like to initiate, but
were not Included in ORD's program budget. Thus, program
offices could design the proposal, give 0R0 first refusal
rights, and proceed with the project. Program officials believed
such a system could enhance responsiveness, timeliness, and
program and project trackability and control.
The above concerns were discussed in August 1982 with the director of
the Office of Research Program Management. He stated that lower level
personnel should be all owed.to attend research committee strategy develop-
ment sessions because they may better understand the technical problems
and can give needed perspective. In addition, he stated that 1n 1977 a
representative of an EPA task force proposed a system (similar to 6
above) that would allow program offices to control research program
funds. However, the proposal was rejected because it could have led to
(1) duplication of research; (2) creation of a new duplicative layer of
research managers within the program offices; (3) fragmentation of the
Agency's research program; and (4) lack of credibility of research
sponsored by regulatory offices because of the inherent conflict-of-
interest.
RECENT PLANS AND ACTIONS TO IMPROVE ORD OPERATIONS
In recent years, EPA has taken Important steps to address deficiencies
in Its research and development program. Such changes were primarily
intended to address (1) a perceived general lack of responsiveness to
program offices* regulatory concerns; (2) uneven research quality; and
(3) an in-adequate research planning and management system. According to
ORD and program officials, these problems have been caused by:
(1) Inherent differences 1n perspective between research and program
staff, Inadequate communications between che two groups, ana
lack of an adequate framework within which to assess potential
program needs and to establish relative priorities (research
planning);
(2) Inconsistent use of the peer review mechanism and need to
implement fully a quality assurance program (research quality);
-------
-20-
(3) Lack of an effective management information and control
system; and
(4) The need to change ORD's organizational structure.
A detailed discussion follows of actions taken or being taken by ORD to
improve its research planning and quality, management information, and
organizational structure.
Research Planning
The formal research committee system was established by EPA 1n 1978.
This system comprised common research and program areas around which
research strategies could be developed to relate media program objectives
and priorities to research activities. By creating this permanent forum
for bringing research and program personnel together, EPA anticipated
that more effective communication, monitoring, and decision making would
take place and increase the responsiveness of the research and development
program to users' (program offices, regional offices, state and local
government) needs.
In addition, beginning with the fiscal 1984 planning and budgeting
cycle, 0R0 started developing research strategies on a broad media basis.
The strategies, referred to as "megastrategies," were developed by top
level ORD managers,.with input from program office, Assistant Administrators
and other key agency personnel. These strategies which were reported to
appropriate media program office Assistant Administrators for concurrence,
serve as both programmatic and resource guides to the research committees,
which, in turn, are responsible for developing the more detailed plans
and objectives at the decision unit level.
It was anticipated that this approach could simplify and amplify the .
process of combining program office needs Into a more efficient planning
process. This was an interim measure which, 1f the current proposed 0R0
Headquarters reorganization plan 1s approved by the Administrator,
could lead to this approach being Institutionalized as the principal ORD
planning process. (See pages 23 and 24 for a discussion of ORD's
reorganization plans.)
Research Quality
Peer review and quality assurance are very important aspects of the
operation of a successful research and development program. By Implement-
ing appropriate peer review mechanisms and quality assurance programs,
ORD believed it could further improve its research quality and help elevate'
its credibility and reputation within the scientific community.
-------
Peer Review
A former Acting Assistant Administrator for Research and Development
issued guidance in February 1980, to provide uniform mechanisms of peer
review throughout ORO. The guidance established the scope of the peer
review mechanisms to include: "...(1) the review of research results
for publication in scientific.and technical journals; (2) peer review of
Intramural and extramural research projects; (3} peer review of our
research programs at the laboratory level, Deputy Assistant Administrator
[now office director] level and the Assistant Administrator level;
and (4) appropriate peer review in programs that recognize outstanding "
scientific and technical contributions...." Further, according to this
guidance, laboratories wefe to revise their peer review procedures where
necessary.
In June 1982, the Acting Assistant Administrator for Research and
Development issued draft Instructions to be followed in conducting
future ORD peer reviews of proposed and ongoing laboratory research.
This proposed peer review process focused on review of:
(1) The proposed approach to be used 1n conducting major projects
or programs;
(2) Ongoing major research..projects or programs at major milestones
in the research; and
(3) The ability of laboratories to enter Into new research areas.
These instructions, however, did not address the peer review of
reports that present the results of research. EPA Order 2200.4 dated
December 18, 1981, which established Agency policy on reviewing scientific,
informational or educational materials, addressed that component of peer
review.
Our interviews disclosed some complaints about ORD's peer review process,
but given the recent efforts to strengthen this area it is apparent that
ORD recognizes the Importance of peer review and reasonable steps are
being taken to implement uniform standards and controls to review and .
evaluate research.
Quality Assurance Program
In June 1979, ORD instituted a mandatory quality assurance program designed
to ensure research quality by setting (1) standards for methods and
procedures to be used in conducting research and (2) criteria for accepting
or rejecting research data. This program was the responsibility of ORD's
Office of Monitoring Systems and Quality Assurance and at tne time of
our review had not been fully implemented.
-------
-22-
Management Information Systems
Since December 1978, EPA has been deve-1 oping various subsystems of the
Office of Research and Development Information System (0R0IS). Components
of the system are in various stages of implementation, revision, and
use. During our review of ORD, we noted that an existing project tracking
system (not a part of the current ORDIS), which was primarily intended
to provide project level Information to ORD Headquarters, had essentially
ceased to function. However, according to the Director of the Office of
Research Program Management, the system has been revived and modified,
and will be updated to cover work done since the time it ceased to
function. (Functions of this system will be covered by the new systems
which are discussed below.)
ORD Headquarters staff and laboratory directors and managers have a
critical need for program management information. Headquarters staff
must be able to track the status of the research program, and need timely
information to do so. Laboratory directors and managers have similar
needs, but they require a much higher level of detail. Currently, 14
ORD laboratories have implemented various types of systems, utilizing
different degrees of automation, to meet their individual management
information needs* The laboratory directors and managers we Interviewed
indicated a high degree of satisfaction with the performance of their
own systems, and a great reluctance to become involved in a centralized
system.
Efforts To Improve Management Information
As part of a planned program to develop comprehensive systems, ORD hired
an outside contractor in October 1981, to assist in evaluating the system
and to make recommendations. On April 23, 1982, the contractor provided
a final report to ORD entitled "Laboratory Information Resources for
Research Program Management" (Volumes I and II). The report is based on
the results of a series of Interviews performed by the project staff,
which assessed the research program management Information systems in
use at each of the EPA laboratories. The report states in part:
Of the many possible ways to design a standardized Laboratory
Program Management Information System (PMIS), the following options
appear to be the most feasible within the constraints and requirements
described above:
(1) Each laboratory uses the ORDIS/PMIS directly.
(2) Each laboratory implements locally a standardized
lab/PMIS tthich has been designed to interface
with the ORDIS/PMIS.
(3) Each laboratory chooses either (1) or (2).
-------
-23-
The concepts of an ORDIS program management information system (ORDIS/PMIS)
and a standardized laboratory program management information system
(Lab/PMIS) were the subject of a presentation at an ORD senior management
meeting held in Baltimore, Maryland, on April 23, 1982. The Acting Assistant
Administrator for Research and Development indicated that such a concept
would be Implemented.
The proposed ORDIS/PMIS and Lab/PMIS share many of the same requirements,
but on the functional level, the Lab/PMIS is oriented toward review
and control during the lab planning and implementation process, while
the 0RD/PMIS.1s oriented toward data collection, synthesis, and analysis
on an ORD-wide basis and 1s designed to link together research planning,
implementation and resource utilization.
An ORDIS coordinating committee has been- set up to advise in the
implementation of the PMIS components of ORDIS. The Acting Assistant
Administrator for Research and Development has charged the committee
with performing review, guidance, and coordination functions. Its initial
meeting was held on May 19, 1982. Since that time, several meetings
have been held.
Proposed Reorganization of
ORD Headquarters and Laboratories
The managerial philosophy surrounding the current reorganization of 0R0
will include (1) top-down planning and development of broad strategies
(megastrategies) to ensure, among other things, that the Assistant
Administrator for Research and Development's policy is taken Into
consideration 1n developing the more specific annual research plans and
budgets; (2) linkage of research planning and Implementation; and (3)
clear accountability trails from research objectives down to specific
project plans and outputs at the laboratory level. ORD has planned a
two stage reorganization, the first stage Involving Headquarters and
the second stage the laboratories. The entire reorganization 1s subject
to the Administrator's approval.
As planned, the structure of Headquarters staff offices would be
simplified and reconfigured along both program and discipline lines to
complement the planning and budgeting process. It is anticipated that
one component of ORD will be organized on a program basis and will focus
on determining users' research needs, translating these needs into annual
research plans and the budget, reviewing and monitoring how laboratories
implement che plans, etc. Another component will be organized on a
disciplinary basis and will focus on generally long-term activities and
issues: evaluating che quality of science, determining overall research
program effectiveness, developing and managing the competitive grants
program, etc. The middle layer of Headquarters ,management, which the
laboratories have dealt with on most matters, will be eliminated to
create what ORD officials believe will be a less complex structure more
suitable to guidance and direction from the Acting Assistant Administrator
for Research and Development level and below.
-------
-24-
As of July 1982, ORD had not decided which of five laboratory reorgani-
zation options it would implement. Nevertheless, common to all options
were (1) direct reporting from ORD laboratories to the Acting Assistant
Administrator for Research and Development and (2) greater accountability
and commitment to research results consistent with approved plans. In
the latter area, in line with management's more results-oriented philosophy,
laboratory performance will be monitored using the revamped ORDIS management
information and control system (which was previously discussed).
OTHER IMPORTANT ISSUES TO BE CONSIDERED
As we previously outlined, there are several important issues which
we believe should be considered 1n making contemplated changes: (1) the
importance of appointing a permanent Assistant Administrator for Research,
and Development, (2) the need to improve ORD's system for disseminating
policies and procedures, (3) the need to enhance communications between
ORD researchers and users of research results, and (4) the need to devote
special attention to the impact of limited travel funds on monitoring
and on employee development and training.
Assistant Administrator for
Research and Development
ORD is presently headed by an Acting Assistant Administrator for
Research and Development _(AARD) on an organizational par with the
Assistant Administrators of the various media program offices. The
AARD, among other things, (1) directs and coordinates all ORD research
activities; (2) develops and implements policy guidance and targets for
planning, budgeting, and controlling research; and (3) Interprets the
needs of ORD's clients (program offices, regional offices, State and
local governments), sets broad program goals and objectives to meet
them, and supports these decisions with appropriate resource allocations.
Meeting these objectives requires a careful blend of administrative
expertise, scientific leadership, diplomacy, and political skills.
Administrative expertise is needed to obtain and allocate resources, to
organize people and programs, and to create and maintain an environment
which nurtures quality science. Scientific leadership 1s Important for
evaluating and establishing priorities and judging the scientific merit
of ORD's research. A leader with recognized scientific credentials also
enhances both the credibility and acceptance of the Agency's research by
the scientific community-
The present Acting Assistant Administrator (AARD) is the third in a
series of Acting AARDs since the departure in December 1980 of the last
presidentially appointed AARD. Of those ORD employees we interviewed,
a majority believed the succession of Acting AARDs in this critical
position, has had a negative impact on ORD. They believed that each
Acting AARD had placed a strong personal stamp on the organization and
had thus contributed to instability in EPA's research programs and
hampered ORD's ability to function and achieve its goals and objectives.
-------
-25-
We believe the Administrator should continue her efforts to appoint a
permanent Assistant Administrator for Research and Development. In our
opinion this will help to further stabilize ORD so that it can maintain
an organizational climate that promotes quality and responsive research
and fosters good morale.
Policies and Procedures
We discussed the system for promulgating and issuing policies and procedures
with an ORD administrative management staff official. He indicated that
ORD's present system Involves Issuing policies and procedures by memorandum
on an ad hoc basis. It 1s-left to each office to determine how the
memorandums will be filed since no standard procedure had been adopted.
This official conceded that he could not readily assemble all of the
current ORD policies and procedures due to the manner In which they had
been disseminated.
This official also stated that ORD had established a policy and procedures
manual in 1976. However, updates to the manual could not be made in a
timely manner due to the volume of changes that occurred in the source
materials and the limited staff available to make changes in the manual.
ORD discontinued updating the manual 1n 1978. Nevertheless, our interviews
disclosed that some offices are still using the manual. We did not
determine whether they were aware of how current the information was
that they were using. At the time of our review there were no plans for
revising the updated 1978 policy and procedures manual.
In our opinion, an organization gains a measure of stability and control
when it has a clear, current, and readily available series of documents
setting down the policies and procedures it expects its personnel to
follow in day-to-day administrative and operational matters.
Enhanced Communication
Interviews with both ORD and program personnel emphasized the Importance
of communciatlon 1n arriving at research strategies, conveying research
capabilities and constraints, and disseminating research results. Most
of those we interviewed agreed that the research committee system either
Improved communications between researchers and research users or
Institutionalized previously effective informal communication links.
Sane believed, however, that long-standing communications problems and
misconceptions led to skepticism over whether any research management
system could function properly*
Clear communication is also a crucial element of any effort to .uitigata
differences between ORD and its clientele to avoid the potential for
falling to meet client needs. This failure can occur for a variety
of reasons. The right question may not have been posed or carefully
described. Thus, if the questions are invalid, the answers will also be.
-------
-26-
Program personnel also were critical of ORD's past failure to market and
communicate the results of its research. They stated that good research
which is not effectively communicated to those who need it is scarcely
better than no research at all. In our opinion, where there 1s a failure
to communicate research progress and results to the appropriate audience,
there is a natural inclination to assume that nothing is being done to
address their needs. This is compounded by program office perceptions
that there was no formal mechanism by which an interested program manager
could access an ORD information system and determine what research had
been accomplished as well as its progress. However, according to the
director of the Office of Research Program Management, summary data 1s
available and ORD recently started to develop a mechanism to assure that
program offices receive timely research results.
In our opinion, some of these concerns may be addressed through ORD's
actions (which were previously discussed) to Improve the planning process
by developing research strategies on a broad basis, which will then be
presented to appropriate program office assistant administrators. Never-
theless, ORD should move to assess its present system for communicating
results and other vital Information to program offices.
Limited Travel Funds
Many ORD officials and staff we Interviewed believed limits on travel
funds had an adverse impact on research monitoring, professional staff
development and recognition, and overall employee morale. Those we
interviewed believed there were insufficient funds for project officers
to monitor extramural research and for program personnel to attend
laboratory reviews. They also criticized the negative impact of travel
fund limitations on the opportunities for formal training, professional
development, and recognition of their technical staff. Several observed
that research 1s only as good as the quality and reputation of the
researcher and that the credibility of EPA's research is aided in part
by the professional stature of the EPA scientific staff. They believed
the inability to fund travel for necessary training, to attend professional
conferences, and to maintain contact with the scientific community could
impact on ORD's research credibility.
In an October 1980 report concerning ORD's extramural research program,
the GAO concluded that insufficient travel funds often prevented ORD
project officers from making timely visits to contractors' and grantees'
sites to determine whether or not the research was being conducted to
best meet ZPA's needs. The GAO recommended that ORD (i) determine the
amount of travel funds needed for its laboratory project officers to
adequately monitor research projects, (2) use that information in seeking
additional travel funds in future years from GMB and the Congress, and
(3) justify within EPA a larger allocation of its current travel
appropriations.
-------
-27-
On July 28, 1980, the then Acting Assistant Administrator for Planning
and Management presented EPA's comments on the report. He stated, in
part, that EPA agreed that attention needed to be given to project
officer's workload and travel funds for visits to extramural research
project sites. However, he also added that in the past, EPA had
consistently sought travel funds for this purpose, but the Congress
reduced EPA's travel request by $2 million for fiscal 1980 and the House
of Representatives appropriations committee had proposed a cut of $250,000
for fiscal 1981. In addition, he stated that 0MB imposed both personnel
and travel ceilings. Finally, he stated that EPA disagreed with GAO's ,
recommendation to look for appropriations specifically earmarked for
site visits because it would split a resource already under continuous
scrutiny.
In light of the above concerns, we believe ORD management still needs
to evaluate the adequacy of its existing and planned travel funds in
meeting its operational needs, including project monitoring and employee
development and training. In addition, we believe such an evaluation
should take Into consideration how effectively such funds have been
allocated within ORD.
RECOMMENDATIONS
(1) We recommend that the Administrator continue efforts to
appoint a permanent Assistant Administrator 'for Research and
Development as soon as possible to give necessary stability to
ORD's operations.
(2) We recommend that the Acting Assistant Administrator for Research
and Development:
(a) Adopt Improved uniform, consistent procedures for
disseminating and maintaining policies and procedures
governing ORD operations;
(b) Continue to pursue vigorously the implementation of
improvements and changes in the computerized management
information system (ORDIS) and work to gain the support
of all ORD parties Involved to enhance chances of its
success;
(c) Reemphasize the need for effective communication "among
ORD, program offices, and other ORD clients to further
enhance research planning and management; and
(d) Comprehensively 'evaluate (1) the adequacy of existing
and planned travel funds and (2) the manner in which
such funds are allocated within ORD. The results of
this evaluation should be presented to the Administrator
for budgetary considerations.
-------
-28-
MANAGEMENT RESPONSE
In a November 18, 1982 memorandum, the Acting Assistant Administrator
for Research and Development agreed with the reports' recommendations.
In summary, he stated that ORD is already working to achieve the reports'
recommendations (i.e., ORD's communications, travel analysis) or is
committed to proceed (i.e., policy and procedures dissemination).
Further, he stated that ORD is pleased to note that the report finds
that its proposed reorganization seems reasonable and promises further
improvement. -He stated that 0R0 shares that optimism although it must
agree that final judgment as to the efficacy of the reorganization only
can be delivered in the future. Finally, he commented that at least in
the recent past, ORD has operated in a professional manner and continued
to plan for its future even in the absence of permanent leadership,
which was due to the cooperativeness and professionalism of ORD career
staff, which should be recognized.
The Acting Assistant Administrator for Research and Development stated
that ORD was sensitive to the need for adequate travel funds and was in
the process of reviewing the adequacy of fiscal 1982 travel funds. In
his March 24, 1983 written response, the Agency Comptroller stated that
ORD's total travel funds are adequate. He stated that since 1981 ORD's
travel ceiling Increased from S2..2 million to $2.7 million in 1983 while
ORD's extramural resources have declined by approximately 50 percent.
He stated that 1n fiscal 1982, ORD lapsed $230,000 in travel funds or 9
percent of their celling. In contrast, however, the Comptroller agreed
that the Acting Assistant Administrator for Research and Development
should examine the allocation of travel funds within ORD and 1f
appropriate, re-allocate some to support monitoring of extramural
resources.
PIG COMMENTS
We believe the Comptroller's coiranents on ORH travel funds are valid.
However, in further discussing the matter with the Deputy Director of
the Office of Research Program Management, we were Informed that ORD can
explain why $230,000 in travel funds had lapsed and how their travel
requirements have Increased under Superfund. Nevertheless, we share the
Comptroller's concern that ORD needs to evaluate how effectively travel
funds are being allocated. Me will await ORD's final response regarding
this matter.
-------
-29-
CHAPTER 2
ADMINISTRATION AND CONTROL OF COOPERATIVE AGREEMENTS
-\
The Office of Research and Development issued interim procedures in
fiscal 1980 to control and administer cooperative agreements. In our
opinion, although these procedures have helped to improve the cooperative
agreement system, there are still some Improvements needed. Our review
disclosed that:
(1) Project officers appeared to have excessive control over
cooperative agreements. We found that project officers have
been allowed to (a) select external peer reviewers and (b)
prepare in-house reviews, as well as Important decision
memorandums.
(2) The conclusions of external peer reviews, 1n which Issues
(concerns over cost, time, technical approach, etc.) are raised,
have not always been adequately resolved and documented by
project personnel.
(3) Some project officers believed that due to lack of travel
funds, they were unable to monitor their research projects
effectively. This problem was previously discussed in
Chapter 1 and was cited by ORD officials as having an adverse
impact on their ability to monitor contracts and interagency
agreements (see chapters 3 and 4).
(4) About fil percent of the 44 project officers we Interviewed
advised us that external peer reviewers had not been required
to complete conflict-of-interest statements or they were not
sure 1f such statements had been received.
We discussed the above needed improvements with the Acting Assistant
Administrator for Research and Development and ORD Office 01 rectors.
They agreed, in general, with the need to make such improvements.
SCOPE AND METHODOLOGY
During fiscal 1981, EPA provided about $53.9 mill ion to fund 487 projects
(cooperative agreements and predecessor grants). Of this amount, only
J15.5 million was used to fund about 140 new awards. The balance ($38.4
million) was to fund amendments to existing projects.
Our review was based on a sample af 49 projects administered by 5
Headquarters offices and S-ORO laboratories at RTP and Cincinnati, Ohio,
during fiscal 1981 through tne first quarter of 1982. Our sample was
comprised of newly awarded and completed projects. The following
summarizes our sample size.
-------
-30-
Projects
Administered
From
Headquarters
Cincinnati
RTP
Projects/Agreements Reviewed
Number Amount
8
30
11
19
SI,844,189
4,588,279
965,468
$1,347,$36
We also interviewed a number of ORD officials, including project officers,
on procedures, practices,-and specific aspects of project management.
BACKSROUND
The Federal Grant and Cooperative Agreement Act of 1977 (Public Law 95-224)
authorized Federal agencies to use cooperative agreements. Cooperative
agreements are similar to grants, except that they require substantial
involvement by the awarding agency. The Act states that:
"... substantial involvement 1s anticipated between the executive
agency, acting for the Federal Government and the state or local
government or other recipient during performance of the contem-
plated activity...."
Once it is determined by the Agency that a project will be a cooperative
agreement, the application should be submitted for peer review. EPA
regulations [40 CFR 40.150 (a)3 require that such applications be peer
reviewed for technical merit by one 1n-house (EPA) scientist and two
scientists from outside EPA. In addition, ORD requires four outside
reviewers on any application that has an annual estimated project cost
1n excess of $250,000. Before funding a cooperative agreement, issues
raised by these peer reviewers must be adequately resolved by ORD manage-
ment.
PROJECT OFFICER CONTROL OVER THE REVIEW
ANb SELECTION OF COOPERATIVE AflftEEMEOTs
Although Improved procedures were Issued by ORD in May 1980, we found
that project officers st1U appeared to have excessive control over
cooperative agreements. We found that they frequently selected the
external peer reviewers and have been allowed to prepare in-house reviews,
as well as important decision memorandums. We believe this situation
gives the appearance of a lass-than-objective process and could provide
an opportunity for someone to influence the selection of project recipients.
-------
-31-
In addition, before ORD initiated Its Improved procedures, in an
October 1980 report, the General Accounting Office stated that:
ORD's procedures for review and selection of grant proposals to be
funded were susceptible to bias.
The GAO also stated:
ORO's former proposal review procedures [December 1979] utilized
an ad hoc process which seemed to give the project officer undue
control of the review and selection process. After formal sub-
mission of proposals [applications], reviews and recommendations
for funding were largely made by the same scientists who had
already established interest in the proposals during the pre-
development, negotiation stage. The scientists,-who usually would
later become project officers, also had strong influence 1n
selecting two outside scientists to assist in the review.
Finally, the GAO stated:
The entire process had the appearance of being controlled in-
house by those who had already preselected the proposals for '
funding.... Opportunities existed for biased judgments.
Until the Federal Grant and Cooperative Agreement Act was enacted on
February 3, 1978, 0R0 followed policies and procedures covering grants
which were last revised on November 1, 1976. The Act, as well as
criticisms of EPA's research program, demonstrated a need to develop
new, substantially revised procedures. To minimize subjectivity in the
review process, ORD developed 1n December 1979 a preapproved list of
qualified Individuals (peer reviewers) from which external reviewers
were to be selected. In a memorandum dated December 5, 1979, a former
Acting Assistant Administrator for Research and Development directed
that applications over .150,000 were to be reviewed by someone other than
project officers. In addition, on May 28, 1980, ORD issued interim
operating procedures which were a further step to minimize EPA scientists'
influence over the review and selection of cooperative agreement applications.
Selection Of External Peer Reviewers
We reviewed 49 projects from fiscal 1981 and the first quarter of fiscal
1982. Of the 49 projects, 18 were newly awarded in this time period.
We found that project officers selected external peer reviewers from
sources other than the preapproved list for 14 of the 18 projects (or
about 78 percent). As previously discussed, ORD instituted interim
operating procedures in May 1980 to ensure that the process for selecting
such reviewers was more objective. The procedures require that extramural
reviewers must be selected from the list that was sent to each laboratory.
-------
-32-
Although these procedures allow ORD the flexibility to add reviewers to
the lists, we believe such additions should be controlled to the extent
possible. We recognize that due to employee turnover and other factors,
the list will have to be revised from time to time. However, we believe
ORD should institute necessary safeguards to ensure that the list 1s
controlled and composed of qualified and highly objective reviewers.
To determine how frequently the list was used, we compared the names of
external peers actually selected to perform the reviews to the names of
peers on the latest approved list (dated January 1982) provided to us by
ORD. The comparison only included those projects which were awarded
during fiscal 1981 and the first quarter of fiscal 1982 so that the
results would reflect current practices. The results of our comparision
are detailed below.
Number of External Reviewers ,
Percent of
Location
Number of
Projects
Total
Selected
On
Lt st
Not on
the list
Total Selected
Not On List
Headquarters
5
19
5
14
73.7
Cincinnati
8
19
12
7
36.8
RTP
5
13
0
13
100.0
Totals
IS
TT
TT
3*
As shown above, of 51 external reviewers about 67 percent (or 34 reviewers)
were selected from sources other than the approved 11st. In our opinion,
one possible reason was that project officers had been frequently selecting
external reviewers of their own choice.
We interviewed 44 project officers responsible for the 49 projects.
These interviews disclosed that 30 project officers (or about 68 percent)
selected external peer reviewers for their'projects. One ORD project
officer told us that 1n his opinion, project officers select reviewers
who will give the best comments. In addition, this employee stated
that when project officers work on potential projects for many months
and they believe the projects are worthwhile, they select those peers
who they think will give a favorable review.
In our opinion, ORD should take necessary action to ensure that project
officers are precluded, to the extent pusst-ble, from selecting external
peer reviewers, especially In those cases where project officers nay
have personal interests in the project being reviewed. In addition, we
encourage ORD management to continue maintaining the praapproved list
of qualified external peer reviewers as the only source for selecting
such reviewers. Finally, we believe ORD officials should continue to
provide an effective means to ensure that the list is properly applied
and the selection of names from it is done by ORD officials who have no
direct interest in such a selection.
-------
IN-HOUSE REVIEWS PERFORMED BY PROJECT OFFICERS
ORD procedures prohibit project officers from reviewing cooperative
agreement applications in which they could be involved. We found that
in-house reviews were still being performed by project officers for
applications over $50,000. ORD Interim procedures state:
at least one in-house review (not the prospective project
Director.... officer) will be designated by the appropriate
Laboratory
In May 1979, ORD Initiated a special review of Its grant and cooperative
agreement operations. A primary purpose of the review was to determine if
the award process was objective and free from bias. In regard to in-house
reviews, the special review concluded:
An analysis of current review procedures revealed that 1n a sample
of 119 funded projects, 1\% were reviewed by the project officer
as the in-house reviewer. While not in conflict with Part 40
regulations, the purpose of 1n-house review (or review in general)
is to obtain an impartial opinion on the merits of a proposal.
When the project officer himself gives advice, the practice does
not appear objective.
Our review of 49 projects Indicated that this situation has improved
since 1979. We found that project officers performed the in-house reviews
1n 16 (or about 33 percent) of the 49 projects. However, we believe the
incidence of 1n-house reviews performed by project officers 1s still too
high. We believe project officers should be precluded from performing
in-house reviews. In our opinion, allowing than to do so gives the
appearance of a review process which appears less-than-objectlve.
Preparation Of Decision Memorandums
Our review disclosed that the project officers prepared and signed
decision memorandums Involving 11 or (22 percent) of the 49 projects
reviewed. Oeclsion memorandums essentially constitute the recommendation
to either fund or reject a project. They are also used as the means to
discuss and resolve issues raised by peer reviewers. Consequently, a
decision memorandum represents a key decision making document. Current
interim procedures do not specify who 1s to prepare and sign the decision
memorandum. However, 1n our opinion, the memorandum should not be prepared
or signed by the project officer, who may have personal interests in the
project. Rather, we believe 1t should be prepared and signed by ORD
office directors or other senior ORD staff who would be in a better
position to independently assess individual applications and make a Tjora
objective decision.
-------
-34-
Ouring fiscal 1980, ORD made a concerted effort to develop new procedures
in accordance with the 1977 Act and to address GAO's concerns. These
procedures were, in part, specifically designed to ensure that the
review and award processes could not be challenged for lack of objectivity.
Based on our review, however, ORD cannot be assured the processes are
free from some subjectivity.. Project officers still appear to have
significant control over the review and award of cooperative agreements.
RESOLUTION OF ISSUES RAISED BY EXTERNAL PEER REVIEWERS
Issues raised by external peer reviewers were not always given adequate
consideration or properly resolved. We believe the quality of external
review could possibly deteriorate 1f such reviewers' comments are not
given serious consideration. Failure to consider these issues, in our
opinion, defeats the purpose of the review process and could raise
questions as to Its objectivity and usefulness.
ORD officials Informed us that scientists who are selected to perform
external peer reviews do not receive any compensation for their services.
We were told that remuneration for their participation in the review
process is based on their interest in the state of the art. We believe
that.issues (concerns over costs, time, technical approach, etc.) raised
by such reviewers should be given careful consideration and resolved in a
technically sound manner. Therefore, 1n our opinion, 1t is important to
document fully how such issues were resolved. Our review of 49 agreements
revealed that although many Issues raised by external peer reviewers were
carefully reviewed and resolved, there were 17 agreements (or 35 percent)
in which such Issues did not appear to have been adequately addressed.
The following 1s a brief example of this problem. (The example used does
not identify the project or ORD personnel.)
Example
The following project was reviewed by one in-house scientist and two
external peers. The first external peer stated:
In general, the proposal 1s quite broad and nonspecific - the
research objectives and benefits, and project deliverables,
are not clearly delineated.
He then recommended that the project:
...be funded but at a reduced lave! of effort, with major
re-or1entat1on of the work effort....
Similarly, the other external peer stated:
The two year time span is reasonable but it seems that they
have more people and money included than they really need.
-------
-35-
Basically, the issues raised by these external peers Indicated that the
project may have been too ambitious and too costly. Subsequently,
the project was modified. However, we found that the EPA official who
prepared the decision memorandum did not respond to the observations
made by these Individuals. In the section of the memorandum titled
"Reconciliation of Any Contrary View," the EPA official simply stated:
There were no contrary views on this proposal.
It appeared as 1f no consideration was given to any of these peers
comments even though there appeared to be an agreement that the project
should be scaled down. In any event, the impression conveyed by the
memorandum was that the two external reviewers unequivocally endorsed
the project as proposed.
In our opinion, in the example discussed above, no real consideration
was given to the external reviewers' comments and observations in
regard to costs, period, and technical makeup of the project. However,
a majority of the external reviews Included in our sample of 49 projects
reflected thorouqh and conscientious evaluations of respective applications.
Nevertheless, 1n our opinion it is still important that controls be
introduced in the review and award procedures to ensure that issues
raised by external reviewers are carefully resolved and documented. We
believe these actions are important to ensure the objectivity of such
peer reviews, which is an essential tool in evaluating the merits of
project applications.
IMPACT OF LIMITED TRAVEL FUNDS ON
COOPERATIVE AGREEMENT MONITCTCTT
Our review disclosed that some project officers believed they could have
monitored cooperative agreement projects more effectively if additional
travel funds had been available to make needed visits to research sites.
In an October 1980 report, the GAO concluded that ORD project officers
could not adequately monitor technical progress of extramural research
projects, due, 1n part, to limited travel funds. In its report, GAO
concluded that more travel -funds were needed by EPA to monitor Its
extramural program adequately. GAO reported that over 50 percent of
those project officers interviewed expressed dissatisfaction with their
opportunities to make site visits.
As previously discussed in chapter 1, our Interviews with ORD Headquarters
and various laboratory personnel revealed that the perceived lack of
travel funds is a major concern, especially regarding project monitoring
and employee training and development. Although we recognize that budget
cuts or other forms of travel funding restrictions are not always under
ORO's control, nevertheless, as we previously stated, ORD should evaluate
the adequacy of its existing and planned travel funds and their allocation
in meeting operational needs, including project monitoring and employee
development and training.
-------
-36-
Intervlews with 44 project officers who were responsible for managing the
49 projects disclosed that 10 of them believed their projects were not
monitored and managed as effectively as they could have been due to a
lack of travel funds. However, 3 of the 44 project officers explained
that they did not always consider site visits imperative because their
communication with researchers (telephone, progress reports, etc.) provided
necessary information to evaluate their projects.
The following are two examples of projects that we believe could have
been monitored more effectively.
Example 1
A project application submitted by a minority Institution was rejected
by external peer reviewers due to the lack of technical merit. The
project officer recommended acceptance because he was confident he could
work with the applicant to prepare a revised, acceptable project plan.
A revised application was prepared which was considered acceptable, and
a grant was awarded to the applicant for about $44,000 to cover first-
year funding. The project officer did not visit the recipient during
the first year of work and at the completion of this period he made the
following observation:
During the first year of this work, the investigator has
expended considerable time on method development, or what
I would call verification.
In addition, he stated that much of this effort had been a revaluation
of Information available 1n the literature and probably should have
consumed considerably less time.
The project officer also noted that the recipient's plan for the second-
year funding was vague. Nevertheless, we found that EPA provided funding
for the second year of research. Once again, the project officer did
not visit the research site during this period of the project. After
considerable delay, the recipient submitted the final report. That report
was considered unacceptable by the project officer.
Example 2
The project officer stated that he had not visited the research site due
to travel fund shortages. However, he said that provision for additional
travel funds was made and budgeted under the grant for the recipient to
travel to EPA as a substitute for EPA travel. We believe this is not as
effective as project officer visits because it excludes EPA from direct,
on-site observation of the research and data collection process. We
believe such observations are important further to ensure the progress
and quality of projects.
-------
-37-
As previously discussed, the Federal Grant and Cooperative Agreement Act
provides federal agencies with a new procurement tool, the cooperative
agreement. The cooperative agreement's most distinct characteristic
is the requirement for a high level of involvement by the awarding agency.
The Act states that cooperative agreements are to be used whenever:
...substantial Involvement 1s anticipated between the execu-
tive agency, acting for the Federal Government, and the state
or local government or other recipient during performance of
the contemplated activity-...
With substantial involvement as a key characteristic of cooperative
agreements, we believe 1t 1s imperative that project officers maintain
proper vigilance over projects as well as the necessary level of involvement
to protect the Agency's interest. In our opinion, although there are
various ways to monitor projects, 1n many instances there does not appear
to be an effective substitute for visits to research project sites.
CONFLICT-OF-INTEREST STATEMENTS
External peer reviewers have not always been required to complete required
conflict-of-interest statements 1n conjunction with their reviews of
cooperative agreement applications. Such statements are required by 0R0
interim procedures. We found that in one laboratory, such statements
were not obtained because personnel were not aware of the requirement.
Discussions with project officers also indicated that they were generally
not aware of this requirements
Research disciplines are often so specialized that there may be a limited
number of scientists associated with any given field. Through universities,
seminars, meetings, and other associations, such specialization draws
scientists together* Thus, it is not unusual for close associations to
form among scientists in a particular field of research. Conversely,
adversary situations could exist where peers may be competitors for
research resources or for some other reason could be biased against
certain peer reviewers.
ORO developed procedures to ensure such associations or competition
does not adversely affect the external peer review process covering
cooperative agreement applications. ORD also issued Interim guidelines
to be furnished to external reviewers in conjunction with ORD's request
for their evaluations of cooperative agreement applications. These
guidelines include a preprinted statement that (1f signed by the reviewer)
certifies that the reviewer does not have a conflict-of-interest regarding
his or her review responsibilities.
The statement Includes the following certification:
I certify that I have read the above statement and that no
conflict-of-interest exists with regard to my review and
discussion of EPA assistance applications.
-------
-38-
Our review of 18 projects awarded in fiscal 1981 and 1982 disclosed that
conflict-of-interest statements were obtained from external reviewers
for 10 (or 55 percent) of these projects. However, 30 of the 44 project
officers (or about 68 percent) we Interviewed informed us that such
statements had not been obtained, or they did not know whether such
statements had been obtained.
We discussed external review procedures with a laboratory official who
was responsible for preparing "and sending the requests for reviews to
the external peers. The official told us that conflict-of-interest
statements were not Included 1n the request packages because Vie had not
been informed of this requirement.
We noted one particular project which may not have been awarded as proposed
if the conflict-of-interest requirement had been enforced. This project
(grant) application was initially rejected by the Headquarters Office of
Research Grants and Centers' peer review panel for lack of technical
merit. Subsequently, the same application was submitted for consideration
as a cooperative agreement. Requests for external reviews were sent out
to three external peers. One external reviewer's comments were positive;
however, another reviewer pointed out that he was closely associated
with two of the researchers on the proposed project. Nevertheless, his
review was- used as part of the basis for recommending the award. If the
reviewer with the potential conflict-of-interest had been made aware of
this requirement and furnished a copy of the preprinted statement, as
well as the accompanying instructions, he probably would have disqualified
himself from reviewing the application. This disqualification would
have resulted in a situation where the project application, being endorsed
by only one external reviewer, may not have been awarded without an
additional review.
It 1s our opinion that the conflict-of-interest requirement serves an
Important purpose 1n minimizing subjectivity 1n the process of selecting
cooperative agreements. These requirements are not only a further step
in ensuring objective peer reviews of cooperative agreement applications,
but also a step toward avoiding unnecessary criticisms of possible
favoritism in awards.
RECOMMENDATIONS
We recommend that the Acting Assistant Administrator for Research and
Development:
(1) Fully enforce existing cooperative agreement procedures or
develop new procedures to ensure that project officers are
removed from:
(a) Selecting external reviewers;
(b) Preparing in-house reviews; and
-------
-39-
(c) Preparing decision memorandums covering cooperative
agreement applications with which they have been
involved.
(2) Ensure that external reviewers are selected from the approved
11st, issues raised by external peer reviewers are adequately
resolved, and resolutions are documented.
(3) In conjunction with our recommendation in chapter 1 (see page
27), determine the adequacy of existing and planned travel
funds, and the way they are allocated, to ensure project officers
can effectively monitor cooperative agreement projects.
(4) Enforce requirements -to obtain conflict-of-interest statements
from external peer reviewers.
MANAGEMENT RESPONSE
The Acting Assistant Administrator for Research and Development did not
provide comments on specific recommendations contained in this chapter.
However, according to ORD senior management, ORG agrees with each of the
above recommendations and has either already taken action or plans to
fully implement them.
PIG COMMENTS
ORD agrees with the above recommendations, and indicated that it has
either taken corrective action or plans to take action. Although we
believe senior ORD management has seriously considered these
recommendations, ORD still needs to provide us with specific actions
they are taking to implement each recommendation. These actions should
be contained in their final response to our report.
-------
-40-
CHAPTER 3
RESEARCH AND DEVELOPMENT CONTRACTS
We reviewed a sample consisting of 44 active and 41 completed research
and development (R&D) contracts for fiscal 1981 and the first quarter of
1982. These contracts were administered by the laboratories 1n RTP,
North Carolina, and Cincinnati", Ohio. We found that:
(1) Contract end products were not always received 1n a timely
manner, and in a significant number of cases had not been
received at all.
(2) In six Instances amounting to $3,673,245 the R&D appropriation
was used to fund contracts or portions of contracts that provided
management and other types of services which, in our opinion,
should have been funded from the S&E appropriation. The Agency
Comptroller agreed that four of these six contracts amounting
to SI,313,339 should have been charged to the S&E appropriation.
(See page 47). Similarly, in two other Instances amounting to
$10,494,314, the R&D appropriation was used to fund contracts
which, In our opinion, provided materials and a multitude of
services to operate or otherwise support government-owned
facilities at RTP and Cincinnati. However, because funding
sources were not linked to specific contract work tasks, we were
unable to determine the accuracy of charges to the R&D and
other appropriations.1 The Acting Assistant Administrator for
R&D pointed out that the ambiguities contained in the definitions
of each appropriation are such that the above contracts could
be charged to either the R&D or S&E appropriations (See page 46).
(3) In 6 Instances, amounting to about $6.8 million, it appeared
that program support contracts or portions of contracts were
Incorrectly classified as R&D contracts. This misclassiflcation
prevented an accurate comparison of actual expenditures to
amounts budgeted 1n the Agency's resource management information
system.
(4) Financial controls over level-of-effort contracts funded by
multiple program elements were not sufficient to ensure that
the specific tasks performed were related to the program
elements used to fund the contracts.
We discussed the results of our review with the Acting Assistant
Administrator for Research and Development, ORD Headquarters office
directors, and the director of the Office of Fiscal and Contracts
Management and senior members of his staff. They generally agreed with
the results of our review. In addition, we discussed the funding aspects
with the Agency Comptroller and senior members of his staff.
-------
-41-
As previously discussed, we are requesting the Office of Legal and
Enforcement Counsel to provide us with a formal legal opinion on whether
the SI.3 million of mischarges to the R8D and Abatement and control
appropriations violated 31 U.S.C. 628 and/or 665, and if so, what actions
must be taken by the Agency Administrator. In general, 31 U.S.C. 628
requires that appropriated funds can only be used for the purposes appro-
priated, while 665 provides that obligations cannot be made 1n excess of
the amounts appropriated.
In fiscal 1982, the Agency took a number of pos1tive~actions to ensure
that (1) major procurements were better managed, (2) procurements were
necessary to achieve a specific policy ibjective, and (3) that funding
authority was available and appropriate for the intended procurement. It
appears that if effectively Implemented, these corrective actions should
prevent or reduce future occurrences of most of the problems we noted.
Specific Agency Improvements are commented on 1n the various sections of
this chapter.
SCOPE AND METHODOLOGY
We conducted our review at EPA laboratories and the Procurement and
Contracts Management Division offices located at RTP, and Cincinnati,
Ohio. We Interviewed EPA officials, project officers, and contracting
officers, and we examined contract files at both locations.
We based our review on a sample taken during fiscal 1981 and the first
quarter of 1982. The sample consisted of 44 active contracts, which"
were selected from a universe of 144 active contracts totaling about
$60.3 million, and 41 completed contracts, which were selected from a
universe of 343 completed contracts totaling about S223.6 million.
Our sample represented about 31 percent of the total number of open
contracts and about 12 percent of the completed contracts.
BACKGROUND
The ORD is responsible for administering EPA's research and development
activities to meet the needs of EPA's operating programs. Thus, Its role
is to provide important scientific and technical support 1n (1) developing
effective standards, (2) preventing and abating pollution, and (3) moni-
toring pollution conditions. As previously discussed, ORD provides
assistance for mission-related RSD projects through EPA Headquarters and
its laboratories. A major part of such assistance is provided through
R&D contracts. 0MB Circular A-ll, dated January 15, 1981, broadly defines
R8D activities to essentially entail a systematic or Intensive study
directed towards fuller scientific knowledge or understanding of a subject
area, as well as the use of this knowledge or understanding directed
toward the production of useful outputs (e.g., materials, devices, systems
or methods).
Resources assigned to ORD are from two appropriations: the RSO appro-
priation, which 1s used to fund extramural research activities, and the
S4E appropriation, which is used for intramural research.
-------
-42-
Appendix B.3 of EPA's planning and budgeting manual provides that R&D
contracts are to be funded under the R*D appropriation, object class
25.32. Further, the manual requires that contracts for management and
administrative services and equipment acquisition be funded under the
S&E appropriation.
TIMELY RECEIPT OF FINAL RESEARCH PROOUCTS
In reviewing 41 completed contracts, we found that final products were
not always delivered to EPA 1ri a timely manner and in a significant
number of instances were not received at all. In our opinion, the primary
reasons for such delays were due to the (1) long lead-time required to
complete the procurement process and (2) failure of individual contractors
to deliver end products within the time frame prescribed in individual
contracts.
EPA's failure to obtain timely end products was also pointed out by the
GAO in an October 1980 report, as well as by internal EPA studies.
Failure by contractors to deliver timely products, or at all, was in our
opinion partially attributable to EPA project officers failure to effectively
monitor the projects and alert the contracting officer so that appropriate
action could be taken to protect the Agency's interests.
Timeliness of the Contract Award Process
Long lead-times required to complete the procurement process contributed
to delays 1n receiving final research products. In our opinion, because
contract end products (i.e., reports, data, etc.) are often used by the
Agency 1n making decisions and issuing regulations, such delays have an
adverse Impact on meeting these responsibilities and successful completion
of extramural research projects.
Our review of 37* completed contracts disclosed that about 70 percent
(or 28 contracts) exceeded EPA's acquisition lead-time of 156 days for
competitive contracts. Based on EPA's past experience, this is the
"optimum" amount of time 1t should take to complete the process, and it
is expected that this optimum will be met in most cases. For the 37
contracts, we compared the number of days between the procurement request
date and the contract award date to determine the amount of time it took
EPA to complete the award process. We then compared this amount of time
to the 156-day optimum established by EPA. We found that overall, it
took an average.of 241 days (or 54 percent above the optimum) to complete
the award process. (Appendix C shows the average days required to complete
the process for the 37 contracts.)
Although we did not conduct a detailed analysis of delays, we discussed
the delays with ORO project officers, contracting officers, and contracting
specialists. They provided us with the following reasons for delays.
* We examined 41 contracts included on EPA's contract information system
listing as being closed during fiscal 1981. We subsequently reduced
the sample to 37 because 4 contracts did not have the date of initiation
on the procurement request.
-------
-43-
(1) Procurement requests had not been evenly distributed through-
out the year, resulting in heavy workloads at the end of the
year. This was attributed in part to delays by Congress in
approving the budget and continuous changes in the budget and
annual procurement plan throughout the year.
(2) There had not been enough personnel 1n the Contracts Management
Division to handle the heavy workload.
(3) Project officers have had a heavy workload, which has resulted
in an inability to conduct timely technical evaluations of
contractor proposals.
In addition, 1n an October 1980 report, the GAO concluded that research
projects often have been delayed because contract awards were slow and
generally took much longer than EPA anticipated. The GAO's examination
of 39 competitive and 25 noncompetitive research contracts awarded
during fiscal years 1978 and 1979 disclosed that the 156 day optimum
lead-time was exceeded by 39 percent and the noncompetitive contract
standard of 119 days was exceeded by 17 percent.
In addition, the GAO pointed out that the longest delays occurred at two
stages-rthe presollcitation stage, during which project officers are
responsible for preparing the procurement request package, and the
evaluation stage, where project officers review proposals for technical
merit- GAO believed that project officers caused the greatest delays In
the procurement process because: .
(1) They failed to prepare adequate procurement request packages
within the established time frame. The packages initially
submitted to contracting offices were often vague and general,
requiring substantial revision before requests for proposals
could be Issued to prospective contractors.
(2) They failed to complete technical evaluations of contractors'
proposals within established time frames. Project officers'
busy schedules, along with the need to plan and coordinate
activities, more effectively, were cited as factors causing
untimely evaluations.
We recognize that EPA and ORD have taken and are taking actions to Improve
the procurement process. However, we believe that additional emphasis
should be olaced on ensuring that the contract award process is completed
within the optimum lead-time established by EPA. In our opinion, such
delays contribute to end Droducts not being received in a timely manner,
which in turn has an adverse imoact on EPA's regulatory responsibilities
and deadlines imposed by Congress-
-------
-44-
Another factor affecting EPA's ability to obtain timely research products
was the failure of contractors to deliver final reports within the
contractual time frames.
Timely Receipt of Final Reports
Contractors frequently failed to deliver final reports to EPA within the
contractual time frames, and in a significant number of instances, EPA
had not received final reports. Part of this failure, 1n our opinion,
can be attributed to the lack of effective on-going contract monitoring
and administration. Of the 41 completed contracts Included in our sample,
28 required final reports to be delivered to EPA. We found that only 11
reports (or about 39 percent) were actually received as of April 1982.
For the remaining 17 contracts, reports were overdue from 1 to 37 months,
or an average of 15 months for each contract. Of the 11 reports that
were received, 3 were received late (two months, five months, and nine
months). (See appendix D for a detailed breakdown by contract.)
The following are two examples where contract monitoring was not effective.
Example 1
A cost-plus-fixed-fee (CPFF) contract was awarded to a firm for $663,034
to study carbon absorption technology In removing specific materials
from wastewater. The Initial period of performance was 24 months, which
was extended by modifications to 38 months, with a corresponding contract
increase to $873,809. At the time of our review, 1t was 17 months past
the contract period of performance, and the five final drafts had been
sent back to the contractor over seven months ago, and all were still
outstanding. Of the total contract award of $873,809, there had been
payments totaling $864,113, through January 21, 1981. The project officer
filled out the contractor performance evaluation on January 27, 1982,
and recommended the contractor for consideration 1n future solicitations
even though the final products had not been received.
The contracting officer stated that he normally got Involved only if the
project officer notified him of a problem or 1f the contractor encountered
any problems.
Example 2
A CPFF contract was awarded to a firm for $266,246 to develop a two-volume
publication on energy aspects of municipal wastewater collection and
treatment. The initial period of performance was 9 months, which was
extended twice for an additional 10 months, and the contract was Increased
to $284,527, which included a cost overrun. The project officer did not
require the contractor to produce a final report after he recieved an
unacceptable draft report. However, another contractor was selected by
EPA to produce a final- report on the same subject.
-------
-45-
We recognize that 0R0 and EPA have been—and still—are taking positive
steps to improve the procurement process, Including improving the time-
liness- of research end products. These improvements are discussed in
more detail on pages 53 and 54 of our report.. Nevertheless, we believe .
ORD should take action against those contractors who are delinquent in
providing final reports within the contractual time frames. EPA's ability
to achieve responsive research depends, 1n part, on completing research
as planned and receiving quality results on schedule. In our opinion,
continuous and effective monitoring by both project officers and contracting
officers will aid EPA in obtaining quality results within contractual
time frames.
CONTRACT FUNDING
We reviewed 44 R&D contracts from a universe of 144 contracts which were
active between fiscal year 1981 and the first quarter of 1982-. We selected
these active contracts for review to determine whether procedures and
controls were adequate to ensure that the correct appropriation, object
class and program elements were selected for contract funding. We found
a number of Instances where (1) the R&D appropriation was used to fund
non-R&D contract expenditures, (2) 1t appeared that the wrong object
class was selected, and (3) controls needed to be strengthened to provide
greater assurance that level of effort (LOE) and other term form contract
tasks could be related directly to the specific program elements used to
fund them.
Use of the R&D and Other Appropriations
In six instances amounting to $3,673,245 1t appeared Initially that
the R&D and/or abatement and control (A&C) appropriations were used to
fund contracts or portions of contracts that provided personnel services,
materials, and other forms of support which, in our opinion, should have
been funded from the S&E appropriation. Similarly, 1n two other instances
amounting to $10,494,314, the R&D appropriation was used to fund contracts
with the Northrup Corporation and the University of Cincinnati which, in
our opinion, provided a multitude of personnel services and materials to
operate and/or otherwise support government-owned facilities at RTP and
Cincinnati. These two contracts were also funded with the S&E and A&C
appropriations. However, because specific contract tasks could not be
linked to funding sources, we were unable to determine the propriety of
the amounts allocated to the R&D and these other appropriations. .
The Acting Assistant Administrator for Research and Development initially
agreed that the above contracts should have been charged to the S&E
approorfation, 1f today's standards and definitions for R&D were applied.
He also explained that fiscal year 1980 was the first year ORD had two
appropriations to use; up to that time ORD had only the R&D appropriation
to fund all of ORO's work (see page So). In a January 21, 1983 memorandum,
he clarified his initial opinion on the contracts. He explained that
-------
-46-
the contracts were properly charged and that he. did not mean to imply
that the contracts should have been charged to the S&E appropriation but
rather that they could have been charged. He believed that ambiguities
contained in the definitions of each appropriation are such that the
contracts could have been charged to either the S&E or R&D appropriations.
To resolve these ambiguities, he Indicated that 0R0 staff was working
with the offices of the Comptroller and Administration to put in place
more precise guidance (see appendix G, page 4).
Appendltx B.3 of the EPA Planning and Budgeting Manual provides that R&D
contracts (as defined in section 44.2 of 0MB Circular A-ll) be funded
under the R&D appropriation, object class 25.32. 0MB Circular A-ll,
section 44.2, defines research and development as follows:
. .Research is a systematic, Intensive study directed toward
fuller scientific knowledge or understanding of the subject studied.
Development is systematic use of knowledge and understanding gained,
from research, directed toward the production of useful materials,
devices, systems or methods, including the design and development
of prototypes and processes. Research and development excludes
routine product testing, quality control, mapping, collection of
general purpose statistics, experimental production, routine
evaluation of an operational program, and the training of scientific
and technical personnel. ..."
The Planning and Budgeting Manual requires that contracts for management
or administrative services, and equipment acquisition, be funded under the
S&E appropriation. Further, chapter 5 of the manual requires 1n part, that:
. .Allowance Holders within the Research and Development appro-
priation and within the Research and Development portion of the
Salaries and Expense appropriation may not transfer funds to or
from the operating plan without Office of Policy and Resource
Management's (OPRM) approval because of congressional restrictions
on research and development programs. ..."
Our opinion on the above contracts was based on (1) our reading of the
definitions of R&D contained 1n 0MB circular A-ll and the Planning and
Budgeting Manual, (2) ORO's initial qualified agreement that the contracts
should have been charged to the S&E appropriation, and (3) a preliminary
review by an official of the Comptroller's office who agreed that some of
the contracts appeared to be non R&D activities which, in his opinion,
should have been funded from the S&E appropriation.
In mid-January 1983, we met with the Comptroller and members of his staff
to discuss the contracts in question. During this meeting, the Comptroller
expressed general disagreement with us on the contracts and requested
additional time to review them. We requested the Comptroller to provide
us with a final written opinion on the appropriateness of charges to the
R&D appropriation after he completed his review. To facilitate the
Comptroller's review, we met with his staff, provided them with additional
information, and made our workpapers available.
-------
-47-
On February 1, 1983, the Comptroller provided us with his final written
opinion on the contracts in question, and the criteria followed by the
Agency in charging the R&D appropriation. (See appendix I.) The Comptroller
agreed that four of the six contracts amounting to $1,313,339 (from R&D
and/or A&C appropriations) should have been charged to the S&E appropriation
because they essentially involved automatic data processing, repair and
maintenance of equipment, or equipment purchases. In contrast, the
Comptroller believed that the remaining two contracts were properly
charged. For the other two facility support contracts (Northrup and
University of Cincinnati), the Comptroller believed that although these
contracts involved a multitude of S&E related activities, they were
either predominately or entirely comprised of R&D activities. The Office
of the Comptroller did agree with us, however, that because these two
contracts were also funded from the S&E and A&C appropriations, and
tasks were not linked to funding sources (program elements), the validity
of the amounts charged to the three appropriations could not be determined.
The following Illustrates the Comptroller's agreement or disagreement
with us on the six contracts:
Amount of Agreement Disagreement
Contract R&D Funds with PIG with PIG
68-02-2832 $ 693,579 * X
68-02-3296 254,660 X
68-02-3482 310,000 X
68-02-3199 55,000 X
Subtotal $1,313,333
68-03-2672 $ 793,405 X
68-03-2765 1,566,501 X
Subtotal $2,359,906
Total $3,673,245
* This amount also includes funding from the A&C
appropriation for $416,679.
The following generally describes two of the four contracts which the
Comptroller agreed should have been funded from the S&E appropriation.
Rockwel.l International Corporation (68-02-3482)
A time-and-materfals contract was awarded to a firm for the calibration,
inspection, repair, and fabrication of equipment. The contract was funded
for $310,000 from QRD's R&D appropriation instead of the S&E appropriation.
The project officer indicated that this was dene because funds were not
available from the S&E appropriation.
-------
-48-
SDC Integrated Services, Inc. (68-02-2832)
R&D funds of $277,000 were used to fund a contract for management,
personnel, services, supplies, and equipment necessary to operate and
maintain the national computer center in RTP, North Carolina.
In contrast, the Comptroller believed two of the six contracts were
properly charged because they (a) involved "development activities" as
defined by 0MB Circular A-ll (as previously discussed) and (b) involved
"quality assurance" services which is consistent with the definition of
R&D in the Agency's Planning and Budgeting Manual. The following is a
brief discussion of the basic scope of these two contracts, the Comptroller's
opinion on them and our views on each contract.
Centec Consultants, Inc. (68-03-2672)
This contract was awarded for $468,924 to support the Environmental
Research Information Center, Cincinnati. Contract tasks include, reviewing
research documents, evaluating technical outputs of environmental pollution
control research laboratories, and field stations, preparing, revising
and updating manuals and handbooks for pollution control, conducting
seminars and workshops (developing the program, arranging for space and
equipment, lining up speakers, etc.), assembling, editing and preparing
seminar and workshop publications, producing capsule reports and executive
briefings, organizing national symposia, and preparing newsletters and
audiovisual materials. The contract was funded from the R&D ($793,405),
S&E ($90,579) and A&C ($321,850) appropriations. The Comptroller believed
that the contract was charged correctly to the R&D appropriation because
most of the above tasks were considered by him to be "development activities"
as defined by 0MB Circular A-ll. In contrast, however, we believe most
of these tasks appear to be administrative, technical and management
support which should have been funded from the S&E and/or A&C appropriations.
However, because specific tasks are not linked to funding sources, we
were unable to determine the propriety of charges to the individual
appropriations.
Radian Corporation (68-03-2765)
This contract was awarded to provide repository services to Federal, state
and commercial laboratories engaged 1n water and wastewater analyses.
Tasks include classifying chemicals, replenishing the repository, operating
a program to review chemicals for the repository, and developing a system
for.cataloging, indexing,and compiling information on repository materials.
The contract was funded from the R&D ($1,565,501) and A&C ($90,000)
appropriations. The Comptroller believed the contract was to provide
quality assurance services and as such was properly charged to the R&D
appropriation consistent with the definition of the R&D appropriation in
the °lanning and Budgeting Manual.
-------
-49-
In a February 1, 1983 memorandum to the Inspector General, the Comptroller
pointed out that the Agency's definition of R&D follows 0MB Circular
A-11, but 1s modified by definitions incorporated in the Agency's Planning
and Budgeting Manual and the appendix to the President's budget. The
Comptroller also explained that it is Important to note that R&D activities
as defined by Circular A-11 may be charged to either S&E or R&D appropriations;
if the activities involve in-house staff and Intramural object classes
(PC&B, travel, ADP services, etc.) they should be charged to the S&E
appropriation; and if the activities involve extramural services and
object classes (contracts, grants, etc.) they should be charged to the
R&D appropriation. Regarding the above two contracts the Comptroller
stated that:
". . .Quality assurance 1s a research and development activity as
defined in the Agency's Planning Manual. . .
". . .Monitoring is a research and development activity and is
specifically mentioned in the appendix to the President's budget
as one of the activities for which the funds will be used. Activities
encompass research on. . . the development of new and improved
sampling and analytical methods and instruments for measuring
pollutant. . .. ."
Finally, in discussing development activities 1n the two contracts, the
Comptroller stated that:
"These activities [reviewing research documents, evaluating
technical outputs, preparing manuals and handbooks, conducting
seminars] are appropriately charged to the research and development
"appropriation as defined 1n 0MB Circular A-11 as long as they involve
the systematic use of knowledge gained from research directed toward
the production of useful materials. This 1s the case for contracts
under review. ..."
Initially, we did not consider these two contracts to be R&D, based on
the definition contained in 0MB Circular A-11 (A-11). However, we agree
with the Office of the Comptroller that the definition in A-11 is broad
and somewhat ambiguous, and as the Acting Assistant Administrator for R&D
pointed out, the ambiguities were such that these contracts could have
been charged to either the R&D or S&E appropriation. In addition, as the
Comptroller further pointed out, these activities have historically been
included in the budget and approved by Congress as R&D. In view of this
fact, and because the contracts were. charged in the same manner they were
budgeted, we have concluded that only the four contracts amounting to
$1.3 million were mischarged.
-------
-50-
We believe the Comptroller should conduct or direct a review of the
remaining contracts not included in our review, to determine the full
extent of any Incorrect charges and prepare a formal report to the
Administrator on their findings. Subsequent to the completion of this
review, the Comptroller's office should determine the effect, 1f any,
that correction of any incorrect charges would have on the obligation
balances for the years in question.
Classification of R&D Versus Program Contracts
We found 6 Instances amounting to about $6.8 million where it appeared
contracts, or portions of contracts, were classified as R&D contracts
(object class 25.32) rather than program contracts (object class 25.35;
see appendix E). Improper classification distorts the actual amount
tfeing spent for R&D as defined by 0MB Clrcuar A-ll and precludes a valid
comparison to budgeted amounts 1n the Agency's resource management
information system. The planning and budgeting manual, appendix B.3,
defines program contracts as:
Planned obligations for contracts which support program
operations. Included 1n this category are contracts for
the preparation of Environmental Impact Statements, for
the development of regulations, standards, guidelines and
criteria, for monitoring, for surveillance and analysis,
for pesticides certification, for regional laboratory
analysis, and for special studies and analyses of programs.
Management, administrative, and research contracts are not
included 1n this category.
We recognize that 1n some cases the dividing line between R&D and program
contracts may not be completely clear and that judgment may be Involved
in making the classification. However, it appeared in some cases that
the two were used almost Interchangeably, possibly to avoid submission of
reprogramnrfng requests to the Office of Comptroller if sufficient funding
was not available in an object class. Two examples are presented below
which Illustrate this condition.
Example 1
A $4.1 million contract was awarded to conduct a continuing program of
review, evaluation, and development 1n support of EPA's responslbil1ty
for providing policies, guidelines, and technical support for the quality
assurance program for State and local air monitoring stations (SLAMS).
Tills support was to be provided in six technical service areas. The
areas are listed below along with an estimated percent of the work effort.
-------
-51-
(1) Perfonnance audits of SLAMS—50 percent
(2) Ouallty assurance guideline development—5 percent
(3) Audit workshop development and presentations—5 percent
(4) Audit system verification program—10 percent
(5) Audit materials analysis and traceability service—15 percent
(6) Quality assurance technical assistance—15 percent
Example 2
A contract was initially awarded for $500,000 (and has a current value of
$2.9 million) to conduct a continuing program of review, evaluation, and
development in support of EPA's functional responsibility for providing
EPA policies, guidelines, and technical support for quality assurance
programs to support the Office of A1r Ouallty Planning and Standards,
regional offices, ORD laboratories, and the Office of Enforcement.
Quality assurance technical support was to be provided for environmental
pollutants in air, water, biological tissue, soil, and manufacturing raw
materials and products. An estimated 80 to 90 percent of the technical
support was to be Iff the air program.
t
In commenting on our draft report,' the Comptroller believed the above
examples of m1sclass1f1ed contracts are "Quality Assurance" contracts
which is an R&n activity as defined by the Agency's planning manual, and
thus they were charged to the proper object class. As stated previously,
the dividing line between R&D and program contracts 1s not always clear
and judgement may be Involved 1n making the classification. In our
opinion, both of the examples cited appear to be providing program related
support of EPA's, as well as States quality assurance program for air
monitoring. Furthermore, these contract examples, like other contracts
we believe were misclasslfled, involve a number of technical tasks
which appear to be consistent with the Agency's definition of a program
contract. We were not alone 1n our opinion. The Acting Assistant
Administrator for Research and Development briefly reviewed the above
examples, as well as other contracts we believed were misclasslfled. He
believed the above examples and some of the other contracts or portions
thereof appeared to be program support.
According to a senior official in the Office of Monitoring and Quality
Assurance, the present object class definitions for R&D and program
contracts are too broad to ensure that contracts are properly classified.
He also stated that without more refined guidance, it will continue to be
left up to an individual's judgement as to whether a contract should be
classified as R8D or program.
-------
-52-
CONTROL OF LEVEL-OF-EFFORT CONTRACT FUNDING
Of the 44 active contracts we reviewed, 23 (or 52 percent) were LOE
contracts. We found that these contracts were usually funded on a multi-
media basis. However, funds obligated for these^ contracts were not
identified to specific tasks or work assignments. Thus, we could not
relate the funds to tasks or work assignments to determine if the funds
were being used for the specific program elements from which they were
provided. We believe this lack of control existed because (1) contract
task or work assignments were merely added by project officers when
needed Instead of being made part of the contract by modification; (2)
hours were not assigned directly to the task or work assignments by
modification; and (3) the funding source (program element) was not
identified to tasks or work assignments. Without this information, we
could not determine if the funding under these types of contracts was
being used appropriately.
Program elements represent distinct subsets of budget activities and are
an essential mechanism to ensure proper use of funds consistent with the
Agency's operating plan. However, without being able to Identify such
program elements to specific contract tasks, the control mechanism built
into the process is not effective in ensuring the proper use of funds.
The following are two brief examples of selected contracts where we
could not determine how the funds were used or whether the effort for
a program element was comparable to the amount funded:
Example 1
A LOE contract was awarded on September 30, 1981, for $2,287,757. The
contract had multiple tasks for R&D of remedial measures for uncontrolled
hazardous waste sites. It was funded from various program elements 1n
the superfund, hazardous waste, and water programs. It was not possible
to link the funds supporting this contract by program element to the
work task being performed.
Example 2
Another contract was awarded on September 28, 1979, for $6,284,140.
The contract was for mult1d1sc1plinary technical services in support
of demonstration programs and the Industrial Environmental Research
Laboratory (IERL), RTP. It was funded from various program elements
under the energy, hazardous waste, radiation, air, solid waste,
interdisciplinary, and toxics programs. Similar to the example above,
funds by program element could not be directly related to work tasks.
The above examples, along with some other examples, were discussed
with the Acting Assistant Administrator for Research and Development.
He stated that he generally understood the problems and added that
ORD was presently reviewing such contracts to determine how ORD
would fund them in the future.
-------
-53-
A1 though our review was of RSD contracts, the Agency makes extensive use
of LOE contracting in other programs as well, and controls should be in
place to ensure contracts are funded from the proper program element by
more directly relating the source of funds to the work being performed.
To resolve this problem on an agencywide basis, the Procurement and
Contracts Management Division (PCMD) drafted a procurement information
notice (PIN), "Revised Policy for Cost Reimbursement Term Form Contracts
Utilizing Work Assignments," and requested us on September 22, 1982, to
review and comment upon the draft. In commenting on the draft, we stated
that it appeared to provide further assurance that funding sources
(program elements) were more directly related to specific contract work
tasks. However, in late January 1983, a PCMD official informed us that
the draft PIN was not going to be issued because 1t was not approved by
Agency procurement offices who argued (I) that 1t limited funding on
tasks for each funding source instead of on a contract total, and (2) it
was a program office problem and not a contract problem.
In late January 1983, we met with officials of PCMD, FMD, and the
Comptroller's office to discuss contemplated actions to develop and issue
a new PIN that addresses the problem identified in our report. Although
a number of alternatives were discussed during this and subsequent
meetings, a decision had not been reached. However, we did request that
PCMD provide us with a copy of the revised PIN once it is circulated for
comment.
IMPROVEMENTS UNDERWAY FOR PROCUREMENT
And MANAgEMEnT tiF CbNTftACYS
In fiscal 1982, the Agency took a number of positive actions to (1)
ensure better management of major procurements, (2) avoid unnecessary
procurements, and (3) eliminate potentially wasteful procurements.
Important actions included:
(1) Procurement Information Notice (PIN) No. 82-09, dated
November 11, 1981, Contract Planning and Procurement Request
Approval Requirements. These procedures provide for continuous
management oversight to ensure efficient and economical
procurements. They include provisions for:
(a) Strengthening the contracting planning system;
(b) Approving procurement requests by requiring
specific certifications;
(c) Revalidating procurement requests pr^or to final
award; and
(d) Approving the competitive range on contract awards.
-------
-54-
On March 3, 1982, the Assistant Administrator for Administration
placed more stringent requirements regarding contract signoff.
On March 18, 1982, the Acting Assistant Administrator for
Research and Development, in turn, required that the cert1f1cat1
document Include a statement that:
(a) The procurement is necessary for statutory or
regulatory compliance to achieve a specific policy
objective;
(b) The contract product will be delivered in time to
accomplish its purpose and the product will be used
by a specified program office 1n specific ways;
(c) Adequate staff are available to monitor the contractor
and use the resulting product;.
(d) Work will be completed in a timely manner that will
support, not impede, policy considerations;
(e) Specified sources have been checked to determine
whether the information or resources are available*
and the existing information or resources are adequate
and
(f) Funds proposed to be used are available, committed,
and appropriate (i.e., appropriation and program
element) for this work.
(3) A contract project officer certification course 1s being
initiated by the Office of Administration to ensure that
only qualified program personnel are assigned to perform
as project officers on Agency contracts.
In addition, the Office of Administration initiated a major
study 1n July 1982 to Investigate ways to streamline the
procurement process. This study Included representatives
from almost every Agency program; three individuals were
from ORD.
(4) EPA's Administrator issued a memorandum to Agency personnel
on November 19, 1981, concerning unauthorized raprogramming
of funds between program elements.
(5) The Assistant Administrator for Administration has
undertaken an in-depth analysis of active contracts
in excess of $100,000.
-------
-55-
We believe these actions are appropriate and, if effectively implemented
or complied with, should help further to improve management control over
procurement actions and avoid or reduce future occurrences of the problems
we noted.
RECOMMENDATIONS
(1) We recommend that the Acting Assistant Administrator for
Research and Development:
(a) Direct project officers to alert contracting officers
to initiate action against contractors who are delin-
quent 1n providing final reports;
(b) Coordinate closely with the Procurement and Contracts
Management Division 1n taking necessary action to
prevent Inordinate delays 1n the procurement process;
(c) Issue guidance to ensure that R&D contracts are funded
from the proper appropriation and are properly classified
to describe work efforts accurately;
(d) Ensure proper control over the use of funds under
LOE contracts, including modifications.
(2) We recommend that the..Office of the Comptroller:
(a) Conduct or direct a review of the remaining contracts for
fiscal 1980, 1981, and the first quarter of 1982, which
were not included 1n our sample, to determine whether any
other contracts were Improperly charged.
(b) Conduct a review to determine the effect, if any, that
mischarges would have on the obligation balances for fiscal
1980, 1981, and 1982, and make all appropriate adjustments
of Agency records.
(c) Prepare a report to the Administrator and provide a copy
to the Office of Inspector General on the results of the
reviews conducted under (a) and (b) above.
(d) Coordinate with ORD and issue any necessary additional
guidance to ensure that all contracts are correctly
classified In terms of the correct appropriation and
as either R&D or Program contracts.
-------
-56-
(3) We recommend that the Assistant Administrator for Administration
direct the Office of Fiscal and Contracts Management to review and
strengthen controls over level-of-effort and other term form contracts
to provide a direct link between funding and work tasks performed.
MANAGEMENT RESPONSE
The Acting Assistant Administrator for Research and Development did not
provide comments on specific recommendations contained in this chapter.
However, in a memorandum dated November 5, 1982, he Initially agreed
that by using todays standards and definitions the subject contracts (see
pages 45 through 47) should have been charged to the S&E appropriation.
However, after a further review of the matter, he clarified his initial
opinion by stating that he did not mean to imply that the contracts
should have been charged to the SSE appropriation. He also believed the
ambiguities contained in the definitions of each appropriation are such
that these contracts could have been charged to either the S&E or R&D
appropriations. Finally, he stated that since fiscal 1980, ORD had
gained Increased experience in using the multi-appropriation structure
and that 0R0 has been working with the Office of the Comptroller and the
Financial Management Division to develop guidance and controls to ensure
proper charges in the future.
Tn a memorandum dated March 24, 1983, the Comptroller agreed that there
may be continuing problems with contract funding and that the Agency
should examine all R&D contracts for fiscal 1983 to ensure that there are
no mlscharges. He disagreed, however, with our recommendation that his
office conduct or direct a review of the remaining contracts for fiscal
1980, 1981 and the first quarter of 1982. The Comptroller believed such
a review would not be fruitful for a number of reasons, including: (1)
the magnitude of the mischarging indicated in our report Is not large
enough to warrant such a review; (2) although four contracts were
incorrectly charged, no one Is suggesting that the funds were not.used to
support ORD's major missions; and (3) the Comptroller's office would be
more productive by ensuring that such mischarging does not occur 1n the
future.
The Comptroller also disagreed that there are significant ambiguities 1n
the definitions of the R&D and SSE appropriations. He stated that although
the report does not Indicate what these ambiguities are, the Comptroller's
office will work with ORD and provide additional guidance, if 0R0
believes it is necessary. With respect to misclasslfication of contracts,
the Comptroller believed the two examoles used in our report were prooerly
charged to the R&D object class because these were "quality assurance"
contracts as defined by the Agency's planning manual. Finally, the
Comptroller agreed with our recommendation that the Agency review and
strengthen controls over level-of-effort and other term form contracts to
ensure that funding sources are linked to specific work tasks.
-------
-57-
The Assistant Administrator for Administration, in response to the
recommendation regarding the need to review and strengthen controls
over level-of-effort contracts, stated that the Office of Administration
has been aware of this problem for some time and the Procurement and
Contract Management Division (PCMD) has been working with the Financial
Management Division to devise a solution. He stated that PCMD has developed
a draft procurement information notice (PIN) Incorporating policies and
procedures designed to address, this problem. He also offered some comments
to help clarify findings contained 1n the chapter. He stated that when
a contract is awarded, it contains a schedule of deliverables (generally
final reports) which the program office has determined will meet its
project requlrements. He stated that delay in the receipt of deliverables,
as called for 1n the contract, can certainly adversely impact project
completion (as the report points out happens in too many instances);
however, it 1s unclear how the procurement process leading to the award
of the contract causes this effect. Further, he stated that the report
references lead-times of 156 days for competitive procurements and 119
days for noncompetitive procurement as "standard lead-times." He pointed
out these lead-times have never been established as standards, but rather
are defined as optimum and each individual procurement is analyzed to
determine the lead-time necessary to award that particular contract.
QIG COMMENTS
Although we agree with the Comptroller that emphasis should be placed on
ensuring that such mlscharging does not occur 1n the future, we disagree
that a review of the remaining contracts should not be conducted for
fiscal 1980, 1981, and the first quarter of 1982. We believe that such
a review is necessary because over $1 million was charged to the wrong
appropriations, which 1s potentially a violation of Federal.statutes,
and any other similar mischarges should be identified and Agency records
corrected accordingly. We also agree with the Comptroller that the
Agency should review all R&D contracts for fiscal 1983, to ensure there
are no mischarges.
As we pointed out 1n our report, the Acting Assistant Administrator for
Research and Development believed that ambiguities contained in the
definitions of each appropriation are such that contracts could be charged
to either the S&E or RfiD appropriations. In light of ORD's concerns and
the fact that mischarges to the R8D appropriation have already taken
place, we believe our recommendation to resolve any ambiguities 1s still
valid and agree with the Comptroller that he should work closely with ORD
to resolve ambiguities related to contract funding.
-------
-58-
With respect to the Comptroller's opinion regarding the two contracts used
as examples of misclassified contracts, we believe there is reasonable
doubt that they are classified properly as R&D contracts. As we previously
stated, both of the contracts we used as examples appeared to be
providing program related support of EPA's, as well as States "quality
assurance" programs for air monitoring- Furthermore, the Acting Assistant
Administrator for Research and Development believed portions of these
contract tasks appeared to involve program support.
We agree with action being taken by the Acting Assistant Administrator
for Research and Development to develop guidance and controls to ensure
proper appropriation charges in the future.
We also understand management's comments regarding the timeliness of the
contract award process and its effect on receipt of end products. We
clarified our report to reflect that the 156 day lead-time is the optimum
amount of time to complete the competitive procurement process. Our
position remains, however, that long lead-times required to complete the
procurement process contributed, in part, to delays in receiving end
products. It follows logically that the longer it takes to get a firm
under contract the longer it will take to ultimately start and complete
the work and thus receive an end product.
-------
-59-
CHAPTER 4
MANAGEMENT OF INTERAGENCY AGREEMENTS
In the past, due generally to limited involvement by EPA in monitoring
interagency agreements (IAGs).and the apparent lack of guidance and
training for ORD project officers, there was little assurance that
such agreements were always effectively controlled and thus benefited
EPA's high-priority needs. In the last few years, however, EPA has
taken some positive actions to improve the overall administration and
management of IAGs. Nevertheless, our review disclosed that project
management and administrative controls still need to be improved. We
found that:
(1) Neither existing IAG procedures, nor the Financial Management
Division's draft procedures, were completely sufficient.
"(2) Although Federal agencies are required by the terms of the
IAGs to provide EPA with Itemized costs, by research project,
on reimbursement vouchers (form 1081), some project officers
have not been ensuring compliance with this requirement.
(3) Due to the lack of detailed cost data on the reimbursement
vouchers* EPA's accounting system did not Identify funding for
Individual IAG projects or track project disbursements against
authorized project funding levels.
(4) Some project officers had not received adequate guidance and
training and were unaware of their responsibilities for
administering and monitoring IAG projects. In addition, lack
of staff and travel funds were cited by some ORD officials as
having an adverse impact on project management. The problem
of limited travel funds was pointed out 1n previous chapters
of our report.
We discussed the above matters with officials of the Office of Admini-
stration, including the director of the Office of Fiscal and Contracts
Management, the director of the Grants Administration Division (GAD), and
a senior representative of the Financial Management Division (FMD).
These officials agreed, in general, with our review results and advised
us of some recent actions that had been.taken to improve IAG management,
including a recent transfer of responsibilities for IAG administration
from the FMD to the GAO. We included, where appropriate, a discussion
of these actions and the officials' comments below.
-------
-60-
SCOPE AND METHODOLOGY
During fiscal 1981, ORD funded 155 IAGs for a total of about $43.2
million. We selected 26 of these IAGs valued at over $100,000 each which
totaled about $32.4 million. Of the above 26 IAGs, we reviewed 9 umbrella
agreements and 17 conventional agreements. Unftrella IAGs are agreements
under which EPA is mandated by 0MB or Congress to enter into agreements
with several other Federal agencies. Under conventional agreements, EPA
determines whether another Federal agency should perform research-related
activities and then selects the appropriate agency.
We also reviewed applicable EPA orders, ORD procedures and manuals, and
IAG project files. We interviewed officials at ORD Headquarters and
laboratories at RTP and Cincinnati. We focused our review on administrative
and financial controls. In addition, we reviewed only those IAGs between
EPA and other Federal agencies whereby EPA purchased goods or services
in exchange for monetary reimbursement.
BACKGROUND
An IAG is a written agreement between EPA and other Federal agencies or a
State or local government. Through these agreements, EPA can provide or
receive goods from another agency in exchange for monetary reimbursement;
services can be provided with or without monetary reimbursement.
EPA's statutory authority to enter into such agreements is provided
in the Economy Act of 1932 [31 U.S.C 686(a)]. Further, under individual
congressional acts such as the Clean Air Act and Solid Waste Disposal Act,
EPA 1s authorized to enter into joint projects with other Federal agencies
where the services of private contractors can be obtained by either
agency.
The bulk of total IAG funding was concentrated in a relatively few IAGs.
In fiscal 1981, ORD's Office of Environmental Processes and Effects
Research (0EPER) funded 11 umbrella IAGs amounting to about $26.7 million
(or 62 percent of the previously discussed $43.2 million). Of these 11
IAGs, 10, totaling about $9.6 million, were in support of ORD's Interagency
Energy and Environment Research and Development program (IEEP), and the
remaining one, which totaled about $17.1 million, supported the Department
of Energy (DOE) transfer program.
-------
-61-
IEEP
The IEEP was initiated in 1974 to achieve Independence from foreign oil.
To fund the program, EPA was mandated by both 0MB and Congress to provide
research funds to the participating Federal agencies. The IEEP was
originally designed to allow these agencies the flexibility of being
responsible for their own resources and implementing specific legislative
responsibilities with limited.EPA involvement and control. However,
recognizing that this program was not as responsive to EPA's legislative
mandates as it could have been and that project activities needed more -
effective monitoring, EPA increased Its involvement in fiscal 1979.
The director of the Office of Research Program Management stated that
EPA has funded a total of about $163.6 million from fiscal years 1974
through 1981 for the IEEP. He also stated that ORD expected to fund
$8.7 million and $2.3 million for this program 1n fiscal 1982 and 1983,
respectively.
00E Transfer Program
In 1979 0M8 directed that 00E transfer research projects totaling about
$14 million to EPA. These projects pertained to conventional technology,
health, and environmental effects research. In addition, the 0MB directive
required EPA to transfer to DOE projects pertaining to energy-related
control technology for synthetic fuels. According to ORD officials,
although projects were transferred to DOE, 0MB directed EPA to support
research projects being performed at DOE's national laboratories through
fiscal 1980.
The director of the Program Operations Staff, OEPER, stated that EPA
funded a total of about $44.9 million from fiscal years 1974 through
1981 for the DOE transfer program. She also stated that ORD expected to
fund $9.9 million and $1.6 million for this program 1n fiscal years 1982
and 1983.
ACTIONS NEEDED TO FINALIZE AND IMPROVE IAG PROCEDURES
i
Although 1n draft stages since July 1980, procedures governing the adminis-
tration of IAGs had not been finalized by the Officeof Administration at
the time of our review.. In addition, we found that the draft procedures
could be further strengthened to ensure that IAG projects are more
effectively administered and managed. In our opinion, aggressive action
needs to be taken to improve, finalize, and disseminate these procedures
to all project officers so that they are made fully aware of their IAG
responsibilities.
-------
-62-
The FMD assumed responsibilities for administering IAGs from the Procurement
and Contracts Management Division in April 1980. The Cincinnati Financial
Management Office (CFMO) was designated by FMD to be the lead financial
management center for IAGs and was given responsibility for implementing
IA6 policies and procedures. In July 1980, CFMO prepared the initial
draft of the proposed IAG policies and procedures. Although these draft
procedures were further refined on February 11, 1982, they have not been
finalized.
These procedures were updated in June 1982 to further clarify IAG policies
and procedures. However, our preliminary examination of them revealed
that they were essentially the same as the February 1982 draft. We were
also informed by an official of FMD that responsibilities for administering
IAGs, including finalizing IAG procedures, were transferred to the Grants
Administration Division (GAD) In November 1982. According to a GAD
memorandum dated July 9, 1982, since administering IAGs was quite similar
to administering grants, the transfer was intended to improve the overall
management and control of IAGs. The director of GAD also advised us
that his office was reviewing all existing and proposed IAG procedures
and expected to Issue final IAG procedures by the end of fiscal 1982.
We compared FMD's February 11, 1982, draft procedures with (1) existing
IAG procedures contained 1n the Contracts Management Manual (CMM) dated
February 19, 1980, and (2) EPA Order 1610.1A, Interagency Activities—
Interagency Agreements, dated October 1, 1973. (The EPA order was
superceded by the CMM.) Our comparison revealed that the draft FMD
procedures did not include some important provisions which were Included
1n the CMM and EPA Order 1610.1 A.
We found that the draft FMD procedures require project officers to notify
the servicing Financial Management Office that work has been performed
or services or goods have been received. In contrast, the CMM procedures
[paragraphs 5.c.(3) and 7.b.(2)] are more detailed and require project
officers to review and certify reimbursement vouchers from other Federal
agencies to ensure that work was performed In accordance with the terms
of the agreement. In addition, we noted that neither the draft FMD /
procedures nor the CMM-procedures required that the terms of IAG's v
Include the provision that other Federal agencies must submit such
vouchers with itemized costs by research project. However, EPA Order
1610.1A contained billing Instructions for agreements which were to be
paid through an advance of funds or on a reimbursement basis and required
that all bills be Itemized. Finally, none of these procedures (1)
addressed program office responsibilities for receiving and reviewing
detailed project cost Information,during the course of project performance
or (2) provided guidance to program offices to enable them to provide
EPA accounting offices with sufficient accounting data to enable EPA's
accounting system to Identify and track disbursements for individual
projects.
-------
-63-
RECEIPT AND REVIEW OF FINANCIAL INFORMATION
BY PROJECT OFFICERS AND ACCOUNTING FOR IAG COSTS
IAG terms require that EPA receive detailed cost information by project
(either on vouchers paid on a reimbursement basis or from quarterly
financial status reports for IAGs paid through an advance of funds).
Our review disclosed that (1) ORD project officers had not always been
ensuring that such Information was submitted on reimbursement vouchers
and (2) project officers 1n Cincinnati and Headquarters were not receiving
or reviewing quarterly financial status reports available from the CFMO.
We reviewed vouchers submitted under 13 IAGs and determined they only
provided lump sum amounts without any detailed cost information. During
our discussions with project officers at Headquarters and the laboratories,
we found that project officers certified reimbursement vouchers authorizing
EPA accounting offices to disburse funds even though these vouchers did
not provide detailed cost information by project. As a result of not
receiving detailed cost information, ORD project officers did not have
sufficient financial Information to provide reasonable assurance that
the other agencies complied with IAG terms. Nor were the project officers
able to use EPA's accounting system to identify project funding or track
disbursements against authorized project funding.
At Cincinnati, we found that quarterly financial status reports were
received by the Financial Management Office, but not forwarded to the
project officers for use-in-monitoring. The "quarterly financial status
reports provided detailed project cost information and, at the time of
our review, were required by the terms of IAGs receiving an advance of
funds. Generally, project officers advised us that they were required
to certify only the technical aspects of projects and were not required
to review or certify the financial aspects. The following example
illustrates the need for project officers to more effectively.review
detailed project cost information and for EPA's accounting system to
track disbursements for each project funded.
In fiscal 1981, the OEPER funded an umbrella agreement with the DOE for
about $17.2 million. This amount represented about 40 percent of the
$43.2 million which ORD funded for the above 155 IAGs 1n fiscal 1981.
Article X of this agreement required that requests for reimbursement be
itemized by research project and amounts to be reimbursed for operating
costs and capital equipment be shown separately. The agreement also
required that individual project costs could not exceed authorized funding
by more than 10 percent.
-------
-64-
We selected a subagreement under this IAG with DOE, Oak Ridge National
Laboratory (ORNL), for review. This subagreement funded 17 research
projects valued at $2.9 million, ranging from a low of $52,000 to a high
of $726,000. Our review of ORNL vouchers submitted to EPA disclosed
that these vouchers did not identify the costs of each project for which
reimbursement was requested. At the present time, such vouchers show
only a lump sum amount without itemization of the costs incurred by •
project or by object class (salaries, travel, supplies, equipment, etc.).
However, in late July 1982, we were informed by the Acting Director,
Energy and Air Division, OEPER, that a memorandum was prepared requesting
other Federal agencies to provide itemized information in support of
requests for reimbursements.
The lack of detailed project cost information precluded use of EPA's\/
accounting system to track disbursements for each of the 17 projects
funded under this subagreement. The 17 projects were funded by 5
program elements (a program element is a 6-d1git number which represents
a distinct program activity, including the appropriation, budget activity
and program area) and were identified in the accounting system under
five document control numbers. Although the accounting system did track
disbursements against total authorized subagreement funding, it did
not identify individual project funding (e.g., 5 document control numbers
were identified but the 17 projects being funded were not), or track
disbursements against these authorized projects. At the time of our
review the CFM0 was disbursing funds from each account number until the
total authorized subagreement funding was reached. (Appendix F Illustrates
the funding of projects under this subagreement.) Consequently, project
officers were unable to determine whether reimbursement to ORNL had
exceeded authorized project funding levels by more than the 10 percent
limitation.
The director of CFM0, stated that EPA program offices were responsible
for providing sufficient accounting data to track project costs. However,
he stated that the director of OEPER Informed him that OEPER was only
interested in tracking total subagreement disbursements. A senior repre-
sentative in FMD agreed with the need to track individual project costs
to ensure compliance with the terms of the agreement. He also stated
that the tracking could easily be performed by program offices if Federal
agencies provided detailed cost Information 1n support of vouchers. The"^
acting director of FMD Informed us that it was not a standard practice J
for other agencies to provide detailed cost information on vouchers C
requesting reimbursements because there
-------
-65-
/
We believe that project management and internal controls over expenditure y
of EPA funds could be improved if Federal agencies were required to submit
detailed project cost information. Additionally, we believe EPA's accounting
system should track disbursements for individual projects to provide
project officers and other EPA officials with reasonable assurance that
authorized funding for these projects will not be exceeded. By requiring
full disclosure of project costs expended, the project officer, as well
as other EPA officials, could.compare costs expended against the Itemized
11st of major object classes Included 1n the project budget. In turn,
this comparison could provide EPA officials with reasonable assurance
that the other Federal agencies Incurred costs 1n the same manner as
originally budgeted and complied with IAG terms. Unless detailed project
cost Information is reviewed at the beginning and during the entire
project period, we believe that: (1) excessive funds could be obligated
by EPA, (2) funds authorized for one project could be used on other C
projects and as a result the technical progress of some projects could \
be adversely affected, and (3) EPA could be put in a position of having J
to reimburse another agency for unauthorized work.
PROJECT MONITORING AND ADMINISTRATION
Some project officers we Interviewed were not fully aware of their
responsibilities for monitoring and administering IAG projects. ORD
officials also cited lack of staff and limited travel funds as adversely
affecting project management. In addition, our review of several IAG
project files at ORD Headquarters and laboratories located at RTP and ,
Cincinnati, as well as discussions with personnel at these locations, /
disclosed that the files have not been adequately maintained as required
by EPA procedures.
In our opinion, part of the above problems stem from ORD project
officers not always receiving-written guidance regarding their project
management responsibilities. As discussed in chapter 1 of this report,
ORD needs to Institute a more effective mechanism for disseminating
policies and procedures. In addition, as discussed in some of the
preceding chapters, various ORD officials have expressed concern that
limited travel funds have adversely impacted project monitoring.
In February 1982 the Office of Administration established a program to
ensure that only qualified program personnel are assigned to perform as
project officers under contracts- We believe all project officers,
including those responsible for IAGs, should be required to attend this
course to further ensure that project officers are fully qualified to
effectively discharge their responsibilities.
-------
-66-
Project Monitoring
Three Headquarters project officers responsible for projects under umbrella '
IAGs were uncertain about their responsibilities concerning managing and
monitoring their projects. One of these project officers said that he V
had not received training or guidance as a project officer and was not
aware of all the documentation requirements to support his projects. In
addition, the acting director.of the Energy and Air Division, OEPER,
told us that project officers were not able to conduct intensive project
monitoring due to insufficient staff and file space.
The director of the Office of Research Program Management informed us
that 0MB had approved only eight positions to manage and monitor the
IEEP and the DOE transfer programs. He also Informed us that the lack
of adequate staff positions, coupled with travel fund limitations, resulted
in varying degrees of success in ensuring that all research conducted
under these two programs continued to be relevant to EPA needs. He
further advised us that 1n fiscal 1981 ORD changed these two programs by
(1) transferring most of the day-to-day project management responsibilities
to ORD laboratories and (2) starting a program planning and evaluation
process aimed at redirecting these programs so that they can be more
responsive to EPA's regulatory needs. In his opinion, as a result of
these changes and decisions within EPA and 0MB, these programs were
better focused and were probably more effectively managed and monitored.
As previously discussed, we believe the Office of Administration's project
officer certification course should help to ensure t-hat project officers
more fully understand and effectively discharge their responsibilities
concerning project management and monitoring. Although this course 1s
intended for program personnel assigned as project officers, under
contracts, we believe this course would benefit all project officers
Including those responsible for IAG projects. We recognize that some
persons may be assigned as project officers under contracts as well as
IAGs and therefore will be required to,be certified as project officers.
However, we" believe that ORD should still take the necessary actions to
ensure that all ORD project officers are adequately trained and fully
understand their responsibilities for project monitoring. In this regard,
ORD management should ensure that all of Its project officers attend the
project officer certification course.
Project File Documentation
We found that ORD laboratory project officers generally maintain complete,
specific, and required documentation 1n their project files. In contrast,
project officers in the ORD Energy and Air. Division (OEPER) generally
did not. They cited Uck of staff or Insufficient filing space as the
primary reasons. We believe, files should be adequately maintained by
all ORD project officers to further ensure sound project management and
compliance with IAG terms.
-------
-67-
The CMM places responsibilities on project officers for certifying billings,
negotiating proposed IAGs and maintaining official IAG files. According
to the manual, the EPA originating office responsible for the IAG is to
develop and retain as part of the official IAG file, documentation to
support: (1) the agreement's purpose, justification, and duration; (2)
an estimated project cost, itemized by major cost element, including any
future funding; and (3) justification for selecting the performing agency
including a statement explaining why the work should not be accomplished
with a private firm in accordance with 0MB Circular No. A-76.
In addition, project officers are required by the CMM to retain, as part
of their official IAG files, substantive correspondence and reports or
other project deliverables (e.g., progress reports). However, our
review of files maintained by project officers in OEPER disclosed that
the following information was not always maintained in project files:
0 An itemized 11st of major cost elements by project, such as
direct labor, supplies, materials, equipment, and indirect
expenses.
° A statanent which justifies the selection of the performing
agency and explains why the work should not be accomplished
by direct contract between EPA and a private firm as required
by 0MB Circular No. A-76.
0 Trip reports to document results of project officers'
site visits.
According to the acting director of the OEPER, a project officer's main
responsibility is to ensure, among other things, that another Federal
agency's products are scientifically acceptable and commensurate with
EPA program office needs. He advised us that project officers in his
division usually maintained current IAG files. However, such files were
not centrally located and did not contain all required documentation due
to insufficient staff and office space. He stated, however, that files
generally contained documentation relative to the IAG subagreements,
reports or outputs, and other relevant correspondence. In contrast,
two OEPER project officers told us that due to lack of guidance they
developed their own methodology for filing. One project officer said
he maintained a file that contained the IAG, laboratory operating plans,
and status reports, while another project officer said he did not, because
he believed such documentation should be maintained elsewhere in the
division.
-------
-68-
Flnally, project officers have not always been required to prepare trip
reports after making project site visits. As a result, such reports
have not been available as part of official IAG files. According to one
project officer at the Cincinnati Industrial Environmental Research
Laboratory (IERL), one project officer In the ORD Headquarters Office of
Environmental Engineering and Technology (OEET) and two project officers
1n OEPER,- trip reports are not required after making site visits. These
officials stated that trip reports are prepared only if problems are
noted or the project needs management's attention. In contrast, discussions
with project officers from other offices disclosed that trip reports are
prepared regardless of problems. Although such reports are not required,
ORD management should encourage 0R0 project officers to prepare and
maintain them in project files to further enhance monitoring and
management's ability to evaluate individual IAG projects.
RECOMMENDATIONS
(1) We recommend that the Acting Assistant Administrator for
Research and Development:
(
(a) Disseminate to all project officers and other
officials responsible for project management and
monitoring, guidance regarding receipt and review
of detailed project cost Information submitted by
other Federal agencies~
(b) Ensure that ORD project officers submit sufficient
accounting Information to the Financial Management
Office to enable It to effectively monitor the pro-
jects and to ensure that EPA's accounting system
tracks obligations and disbursements by project.
(c) Initate a review of all ORD-funded IAGs and, where
appropriate, request other Federal agencies to submit
Itemized vouchers broken down by research project
with details of cost expended by object class.
(d) Establish an ORD policy for standardizing official
IAG files, and Issue Instructions to all project
officers on proper file maintenance, Including docu-
mentation.
(e) Ensure that all program personnel assigned as project
officers under IAGs attend the project officer
certification course.
(f) In conjunction with recommendations in chapters 1 and 2,
evaluate the impact that limited travel funds have on the
effectiveness of IAG project monitoring.
-------
-69-
(2) We recommend that the Assistant Administrator for Administration:
(a) Finalize and disseminate new procedures governing the
administration of IAGs as expeditiously as possible.
(b) Incorporate a requirement in draft and final IAG policies
and procedures to ensure that:
0 Project officers review and certify reimburse-
ment vouchers to ensure that there is a proper
expenditure of project funds and that work has
been performed in accordance with the terms of
the agreement;
0 The terms of IAGs include a clear requirement
to provide EPA with detailed project cost
Information;
0 EPA program offices clearly understand their
responsibilities regarding receipt and review
of detailed project cost information submitted
by other Federal agencies; and
° Program offices provide sufficient accounting
information to FMD to enable EPA's accounting
system to track obligations and disbursements
by project.
MANAGEMENT RESPONSE
The Acting Assistant Administrator for Research and Development did not
offer comments on the above recommendations. The Assistant Administrator
for Administration stated that he generally concurred with the recommendations
contained in the chapter. He stated that audit concerns have been addressed
1n the reorganization of the interagency agreement management functions
Into OA's Grants Administration Division. He stated that the Grants
Administration Division will have responsibility for legal and administrative
management of Interagency agreements, including maintenance of official
files, execution of agreements, and management as the action office. In
his opinion, these responsibilities will enhance ORD's use of interagency
agreements and he stated that OA's draft interagency agreement procedures
are being., discussed with ORD to ensure that their respective technical,
legal and administrative management responsibilities are properly Integrated.
He also stated that OA expected to issue final reviewed and approved
procedures by November 1, 1982. However, according to a senior official
of the Grants Administration Division, interim IAG procedures were issued
in January 1983 with finalized IAG procedures expected to be effective on
May 31, 1983.
-------
-70-
Th e Assistant Administrator for Administration also stated that historical
difficulties 1n tracking individual projects have occurred because other
Federal agencies billed EPA in one lump sum and did not identify specific
project costs. Further, he commented that the draft procedures will
require that an interagency agreement be separately prepared for each
specific proposed project. He stated that these individual cost estimates
will be available to project officers, and the project officers will
review and approve vouchers. .He also stated that these changes should
remedy the difficulties highlighted 1n the draft recommendations 2a, b,
and d, and that detailed guidance regarding OA responsibilities will be
made available to all ORD officials who need to be involved inlthe
initiation, negotiation, development, monitoring and determination of
interagency agreements. Finally, he stated that this guidance would be
available before the end of this calendar year, and that general training
can be provided to all lab coordinators and pertinent Headquarters, ORD
staff to ensure that the procedures effected (May 31, 1983), are used
correctly.
016 COMMENTS
We encourage the Assistant Administrator for Administration to finalize
IAG draft procedures as soon as possible. We believe the draft procedures
and other improvements OA has taken are positive steps in Increasing
controls over IAGs. However, we also believe OA must ensure that once
these procedures and changes are in place, that they effectively address
problems we have identified, and are enforced on an agency-wide basis.
-------
-71-
CHAPTER 5
COMPETITIVE GRANTS PROCESS AND ACTIVITIES
The Office of Research Grants and Centers' (ORGC's) process for adminis-
tering competitive research grants generally appeared adequate. Our
examination of the scoring, ranking, and resulting funding decisions of
four peer review panels during fiscal 1981 disclosed that the most highly
ranked grant applications were usually funded. However, we also noted
some areas which we believe ORD needs to address to further improve ORGC
operations:
(1) Requiring ORGC science review administrators (SRAs) should be
required to maintain readily available documentation to support
grant application funding decisions;
(2) Adoption of a formal ORD-wide mechanism should be considered
to track by project, congressionally mandated long-term research
and development funding; and
(3) The existing workload of SRAs and its Impact on their ability
to effectively monitor grant projects should be evaluated.
We discussed the above areas in August 1982 with the director of the
Office of Exploratory Research (OER). He agreed, 1n general, with the
above improvements and advised us that OER was initiating a review of
ORGC and that the results of our review would be helpful- OER's review
is discussed below.
We also explained to the director of OER that because SRAs did not always
maintain readily available documentation to support decisions on funding
grant applications, we were unable to determine the accuracy or appro-
priateness of soire decisions. Therefore, we requested that the director
review selected grant and funding decisions we had noted during our
review of peer panel activities for fiscal 1981. We also requested that
upon completion of this review, the director advise us of the appropriate-
ness of such decisions, along with any available supporting documentation.
In late October 1982, the director of OER provided an explanation of the
above funding decisions along with additional supporting documentation.
Based on our review of this Information and follow-up discussions with
the director of OER, It appeared that these decisions were appropriate.
In addition, as part of our overall objectives, we initially planned to
assess grant results. However, because ORGC had only been in existence
since January 1980, and consequently very few (only four) grants had been
completed during our review, we were unable to assess, overall, whether
researcn administered by ORGC Benefited 0R0 or EPA program offices in
meeting their responsibilities. Projects being funded through these
grants are generally of a long-range nature (generally more than 2 years).
-------
-72-
Furthermore, our review disclosed that there had been great concern
expressed by some EPA officials, including ORD laboratory directors,
in the last couple of years about (1) the perceived lack of relevancy
and usefulness of competitive grants administered by ORGC; (2) the
adequacy of grant solicitation packages; (3) the lack of a defined
research grants program; and (4) weaknesses 1n the ORD office director's
relevancy review. Although we discuss these concerns in general below,
an evaluation of their validity would have been outside the scope of our
review. Some of these concerns were being addressed by the Acting
Assistant Administrator for Research and Development.
/
We met with the Acting Assistant Administrator for Research and
Development in late August 1982. He informed us that he recognized
the need to evaluate ORGC's current operations and that the previously
discussed review by the director of OER would focus on such areas as
administrative controls over the peer review process, including the
selection of peer panel chairpersons; SRA responsibilities and the
adequacy of controls governing their responsibilities; effectiveness
of grant monitoring; and adequacy of the relevancy review by ORD
office directors. According to a senior OER official, this review which
is still on-going is expected to be completed in March 1983.
We believe that the above review is a positive step and we encourge ORD
management to pursue it aggressively. In our opinion, the review should
assist ORD in isolating the causes of concerns brought to our attention
so that effective solutions can be offered. As part of the solutions,
we believe ORD should take steps to correct the deficiencies noted on
page 81.
SCOPE AND METHODOLOGY
We conducted our review primarily at the ORD Headquarters ORGC.
We interviewed various ORD officials including the directors of the
OER and the ORGC.
We reviewed a sample of grant projects out of a universe of 102 new
grants awarded in fiscal 1981 which totaled about $15.6 million. We
selected 20 of these grants which totaled about $5.3 million, or about
34 percent of the dollar value of the universe. We did not select any
completed grants because, as previously stated, ORGC has only been in
existence since January 1980, and only four grants awarded since then
had been completed. We focused our review efforts on the controls and
the administration of the grants process. We examined the scoring and
ranking of the fiscal 1981 grants reviewed by three peer review panels
(1) environmental biology, (2) environmental engineering and pollution
control processes, and (3) health effects.
-------
-73-
We verified the scores received by ORGC from all three reviews (peer,
relevancy, and long-term significance) in our sample to the source
documentation. We also verified to the Grants Information System (GICS)
those grants that were recommended for award. However, we did not inter-
view panel chairpersons or review records which they personally maintained
because these Individuals were at geographically dispersed locations.
BACKGROUND
The ORGC's competitive grants program was established, in part, as a
result of the congressional mandate in November 1977 (Public Law 95-155)
which required EPA to "...establish a separate program to conduct continuing
long-term environmental research and development...." This Act also
required that at least 15 percent of ORD's appropriated research funds
be allocated for long-term environmental research and development.
The ORGC, which became operational in January 1980, is a part of the
OER. These offices were established by the ORD in response to
recommendations made by an internal ORD review group to Improve the
process for evaluating competitive grant applications. The OER's primary
responsibilities include administering the research grants and centers
program, the minority institutions research support program, and the
scientific assessment program.
In general, the objective of ORGC's competitive research grants program
is to encourage highly qualified scientists to initiate research projects
which will provide a long-range scientific base for EPA's regulatory
responsibilities.
Description of the Competitive
Grants Review Process
A grant is awarded when no substantial Involvement 1s anticipated between
EPA and the State or local government or any other recipients during the
performance of the contemplated activity.
Applications received by EPA are considered for a research grant provided:
(1) the project 1s for research (as compared with development, demonstration,
survey, or preparation of material and documents); (2) the project is of
a long-range nature; (3) the project 1s germane to EPA's mission; and
(4) the grantee 1s eligible to apply under the various Federal laws
which authorize EPA to award research grants.
Before a decision is made to fund a grant application', 1t 1s presently
subjected to two types of reviews: a peer review and a relevancy review
by ORD office directors.
-------
-74-
Peer Review
Each grant application received by ORGC is sent to one of five peer
review panels, where at least two panelists perform a detailed review of
the grant application for scientific merit and quality. Each panel
meets three times a year and is headed by a chairperson. These panels
and areas of responsibility include:
(1) Health Research: reviews grant applications that provide a
scientific basis upon which the Agency can make regulatory
decisions concerning the protection of human health from
environmental pollutants.
(2) Environmental Engineering and Pollution Control Processes:
reviews grant applications that supplement ORD's activities
by stimulating scientific and technical research fundamental
to pollution control advances.
(3) Environmental Chemistry and Phys1cs--A1r: reviews grant
applications that develop the scientific tools and information
generally needed to guide EPA and the States in making regulatory
decisions.
(4) Environmental Chemistry and Physics--Water: same as
Environmental Chemistry and Physics—A1r, but the grants relate
"to water.
(5) Environmental Biology: reviews grant applications involving
the examination of ecological effects of pollutants and abatement
practices.
According to ORGC, these panels are composed of leading scientists from
both academla and various EPA research facilities. However, no more
than 20 percent of each panel's membership can be from EPA laboratories.
Each panel chairperson (who is not employed by EPA) 1s nominated by the
Oirector, ORGC, and approved by EPA's Science Advisory Board. Final
selection, however, is made by the Acting Assistant Administrator for
Research and Development.
When the detailed review is completed by the Individual panelists, the
review results are presented to the entire peer review panel for a decision
on the application's merit. In making this decision, the panelist gives
the application a score between 0 and 100. Applications w*th a score of
60 percent or lower are rejected by the panel. Approved applications
are forwarded to the appropriate 0R0 office director for a relevancy
review.
-------
-75-
Qffice Directors' Relevancy Review
ORD Office Directors review grant applications to ensure that they are
relevant to EPA's mission and program area (air, water, toxic substances,
etc.) responsibilities.. Upon completion of their review, the directors
assign each approved grant application a score between 0 and 100. This
score denotes the degree of relevance.
Ranking of Grant Applications
ORGC ranks each application according to the results of the above reviews.
Based on percentage weights, ORGC then gives each application an overall
score not to exceed 100 percent. After being scored, applications were
ranked 1n descending order. Based on their overall numeric ranking,
ORGC then makes a recommendation to the Assistant Administrator for
Research and Development to fund individual grants.
GRANT APPLICATION REVIEWS AND FUNDING DECISIONS
Our review of the results of three peer review panels disclosed that
ORGC SRA's have not always maintained readily available documentation to
support grant funding decisions. However, we found that grant applications
receiving the highest scores from the peer review panels, ORD office
directors, and director of OER were ranked accordingly and usually funded
according to rank. However, several grant applications, although ranked
higher than some other applications which were funded, had to be recompeted
in a subsequent panel cycle due to lack of funds in various program
elements. Because.supporting documentation was not readily available,
we were unable to reach a conclusion on the appropriateness ,of such
decisions. However, as previously discussed, the director of OER has
reviewed certain decisions and provided us with additional supporting
documentation. Based on our review of the information, and followup
discussions with the director of OER, these decisions appeared appropriate.
We examined the scoring, ranking, and resulting funding decisions for
three peer review panels during three cycles 1n fiscal 1981: environmental
biology, environmental engineering and pollution control or processes,
and health effects. The following table describes activities over the
three cycles.
Environmental Biology
A total of 96 grant applications were reviewed by this panel during the
three cycles. Of these, 26 were funded, 42 were recompeted, and 28 were
rejected. Overall, grant applications which ranked the highest were
generally funded 1n all three cycles. We noted, however, that six appli-
cations in cycle 2 and three applications in cycle 3 were either recompeted
in the next peer review panel cycle or were rejected even chough 8 grant
-------
-76-
applications with lower scores were funded. Grant applications were
recompeted generally because there were no funds in various program
elements in this cycle. Except for footnotes to summary ORGC worksheets,
however, there was no readily available documentation to support these
funding decisions.
Environmental Engineering
and Pollution Control Processes
A total of 45 grant applications were reviewed by this panel during the
three cycles. Of these, 19 were funded, 4 were recompeted, and 10 were
rejected. Overall, grant applications which were ranked the highest
were generally funded. However, we noted that one grant application in
cycle 1 was rejected, but had a higher score than 2 other applications
which were funded. Again, other than footnotes on the ORGC summary
worksheets, there was no readily available documentation to support
funding decisions, including the need to recompete applications.
Health Effects
A total of 120 grant applications were reviewed by this panel during the
three cycles. Of these, 52 were funded, 18 were recompeted, and 50 were
rejected. Overall, we found that the highest ranked grant applications
were generally funded. However, we also noted that there were three
applications in cycle 1 that were not funded, although three other
applications with lower rankings than these applications were funded.
Of the three which were not funded initially, one was under the radiation
program, for which no funds were budgeted, one was funded 1n the third
cycle, and the final application was recompeted 1n the second cycle, but
was subsequently rejected due to a lack of funds. In cycle 2, two grant
applications were not funded even though they had higher scores than two
other applications that were funded. Again, there was no readily available
documentation to support these decisions.
We believe that SRAs should maintain readily available documentation to
fully support funding decisions and actions. Although SRA worksheets
sometimes explained through footnotes why grant applications have been
recompeted or rejected, we were unable to fully verify such decisions.
Me believe such documentation will not only assist in an independent
verification of such decisions and provide a necessary "audit trail,"
but also enhance the objectivity of the peer panel process.
SRA WORKLOAD AND GRANT MONITORING
Our review disclosed that some SRAs' project workload appeared to be
too heavy to ensure effective monitoring. However, the Acting
Assistant Administrator for Research and Development planned to transfer
responsibility for monitoring new grant awards to ORD laboratories,
effective October 1, 1982. At the time of our review, this decision was
being held in abeyance pending completion of the previously discussed
review by the director of OER.
-------
-77-
During our review, ORGC had seven SRAs assigned to the five peer review
panels. As part of this responsibility, SRAs serve as project officers
over each grant awarded by the panels. During fiscal 1981, these SRAs
were responsible for about 250 grants. In addition, SRAs assist the
panel chairpersons in operating the panel and are responsible for a
number of other duties, Including: (1) assisting 1n the preparation of
the grant solicitation package; (2) screenl ngand forwarding grant
applications to appropriate peer review panels; (3) monitoring peer
review panels for compliance with policies and procedures established by
ORGC; (4) recommending grant applications for funding (based on peer
review and available funding); (5) monitoring the progress of grants (by
serving as project officers); (6) recommending grants for continued
funding; (7) recommending grants to be closed out; and (8) providing a
means to disseminate grants results.
SRAs are required to monitor the progress of grants as a part of their
project management responsibilities. The Grant and Cooperative Agreement
Act of 1977, section 5(2), provides SRAs with general guidance concerning
their monitoring responsibilities. According to two SRAs, there are no
written instructions regarding their monitoring responsibilities. They
told us that they do, however, ensure compliance with the grant conditions.
Because research grants do not require substantial Involvement on the
part of EPA, such monitoring is normally done by reviewing semiannual
progress reports submitted by grantees as well as through discussions with
principal investigators at periodic ORGC seminars. Based on our August
1982 examination of various documents and worksheets, as well as interviews
with two of these SRAs, we found that SRAs appeared to be adequately
tracking progress reports. However, we noted that the number of grants
being monitored by these four SRAs varied. The following table shows
the number of grants being monitored.
-------
-78-
SRA Panel
Time
Period
Number of
Grants
Monitored
Percent
of Total
1 Health Effects
FY 1980-81
62 1/
34
2 Environmental Engineering and
Pollution Control Processes FY 1980-81
55
29
3 Environmental Chemistry
And Physics -Air
FY 1980-81
29 ,
_42 2/
Total 188
15
4 Environmental Biology
FY 1980
22
Too
Note: Number of Grants monitored
Number of SRA's
188
4~ * 47 Average
1/ As of August 1982, this SRA and his assistant were
monitoring 90 grants.
1/ Information was available for fiscal 1980 only.
As shown 1n the above examples, these four SRAs monitored an average of
47 projects during fiscal 1980 and 1981. The number of grants monitored
by these SRAs ranged from 29 to 62. In addition, one SRA told us that
he was responsible for about 90 projects as of August 1982.,
In a memo dated April 20, 1982, the Acting Assistant Administrator
for Research and Development outlined a decision to transfer responsi-
bilities for monitoring new grant awards to ORD laboratories, effective
October 1, 1982. His reasons were that (1) the process would reduce the
number of people needed in Headquarters to monitor grant research projects,
thus reducing the need to employ Intergovernmental Personnel Act (IPA)
staff; and (2) laboratory project officers would benefit by closer associ-
ation to more basic long-term research as well as to the applied research
which they currently direct-
The Director of ORGC disagreed with this decision. In a memorandum
dated May 11, 1982, he stated that by their nature, grants require minimal
involvement by the Agency during the course of research, and more intimate
association by project officers would be contrary to the intent of the
Grant and Cooperative Agreement Act of 1977. He further stated that,
while all ORGC research is monitored through progress reports, as well
as the principal investigator's attendance at ORGC seminars, the project
officer intervenes only when it 1s apparent that contact is necessary or
the principal investigator needs guidance, possibly through a site visit.
-------
-79-
In late August 1982, the^Acting Assistant Administrator told us that his
decision to transfer new grant monitoring to ORD laboratories was being
held in abeyance pending the results of the previously discussed review
by the director of OER. He also stated that his primary reason for
wanting the laboratories to monitor grants was to get laboratories
involved because of the amount of research being performed at the
laboratory level.
In our opinion, ORD's plan to transfer new grant monitoring responsibilities
to various laboratories is plausible- However, we believe that before a
final decision is made, the concerns raised by the Director of ORGC need
to be effectively addressed. In addition, this decision should ensure
that SRAs will be effectively utilized after the transfer takes place.
TRACKING LONG-TERM RESEARCH FUNDING
The ORD does not have a formal mechanism to track congressionally mandated
long-term environmental research and development funding by project.
However, the director of the ORPM.stated that 1n his opinion such a
mechanism was really not necessary because ORGC accounted for most of
these funds through the grants and centers program.
In a memorandum dated June 2, 1982, the director of the ORPM stated
that "the Office of Exploratory Research Grants and Centers of Excellence
were funded through a percentage set aside (15 percent) as stipulated in
the Environmental Research, Development, and Demonstration Authorization
Act of 1981. The percentage was applied against EPA's extramural research
programs and the funds were used to support the competitive grants program
(universities and colleges). The funds also provided operational support
for eight centers of excellence."
The director of the ORPM also provided us with a breakdown of extramural
funding directed at long-term environmental research and development
activities. The following chart shows funding to meet the congressional
mandate.
Fiscal
Appropriation
(In Millions)
—m—
RSD
Funding for
Long-TernTTTSD 1/
(In Mill ions)
RiD Appropriation
qlZ_D
Percent of-
Funding tip
nr
1981
1982
1933
$234.9 1/
$154.3 If
$108.7 7/
$26.4
$20.3
$13.5
11.3
13.2
12.4
U Based on ORD's final operating plan.
2/ Based on ORD's initial operating plan.
7/ Estimate based on 1983 President's budget.
7/ Extramural research funding - grants and centers.
-------
-80-
The above chart shows that the amount of funds provided in each fiscal
year for long-term environmental research and development was slightly
less than the mandated 15 percent. However, the director of ORPM stated
that the percent of total funding directed at long-term research is
significantly greater than 15 percent, because ORD also provides funding
for in-house long-term research and some cooperative agreements which
are long-term in nature. In support of this, he provided us with estimates
that showed total long-term environmental research (including the extramural
funding 1n the chart above) to be 33.1 percent (or $51.1 million) in fiscal
1982 and 36.4 percent (or $39.6 million) in fiscal 1983.
The Environmental Research, Development, and Demonstration Authorization
Act of 1981 provides that the Administrator must establish a separately
identified program of continuing long-term environmental research and
development for each activity 1n subsection (a) of H.R. bill 3115. The
Act requires that, unless otherwise specified by law, at least 15 percent
of funds appropriated to the Administrator for environmental research
and development for each activity listed 1n this subsection shall be
obligated and expended for such long-term environmental research and
development under this subsection (42 U.S.C* § 4363). Subsection (a)
of H.R. bill 3115 identified the total funds appropriated to the ORD,
Including the extramural funds.
The Director of ORPM told us that he interpreted the congressional
mandate to mean that 15 percent of the extramural funds appropriated to
the Administrator shall be used for long-term environmental research
rather than 15 percent of the amounts identified in subsection (a) of
H.R. bill 3115. He also stated that the ORD's extramural funds were
Identified in the budget as research and development funds. Therefore,
he interpreted the wording of "environmental research and development"
in the mandate to mean extramural funds.
We requested a legal opinion from the Office of General Counsel regarding
whether the 15 percent set aside applied only to extramural research
funds or to both extramural and intramural research. In a memorandum
dated May 11, 1982, the Associate General Counsel stated, that: "Neither
the Acts, nor their legislative histories, place any restrictions on the
use of the 15 percent set-aside other than that 1t 1s to be used exclusively
~f9r long-term environmental research and development. The legislative
histories behind this provision emphasize the importance of long-term
studies, analyses and research which seek to advance the basic understanding
of environmental matters."
We believe that ORD should consider adopting a formal mechanism to track
congressioaally mandated long-term research and development funding by
project. Although we recognize that ORD has the means to estimate long-
term funding, such a "tracking" system in our opinion should be seriously
considered to provide a more reliable source of information on such
funding. We discussed this matter 1n August 1982, with the director of
OER. He stated that such a tracking system was necessary and would
provide a more reliable source of Information to account for the mandated
15 percent funding.
-------
-81-
Concerns over the Quality and
Usefulness of Competitive Grants
During our review, concerns were expressed about the quality and usefulness
of the research grants and centers program. In summary, these concerns
included:
(1) The grant soli cation package was too broad and general. Some
EPA officials believed that a more precisely worded and highly
specific solicitation package should be prepared. They believed
this would result in better prepared packages, which would
lead to more relevant research. .
(2) The purpose, scope, and objectives of the research grants
program were not well defined.
(a) The relevancy review by ORD office directors was weak.
(b) Laboratories and research committees were providing
limited Input to the relevancy review.
As stated previously, we believe these concerns are Important and that
the Acting Assistant Administrator for Research and Development has
taken action to correct some problems. However, we still believe that
such concerns must be comprehensively addressed by ORD management to
ensure that ORGC continues to serve the Agency in an effective manner.
RECOMMENDATIONS
We recommend that the Acting Assistant Administrator for Research and
Development:
(1) Ensure that SRAs maintain adequate supporting documentation
regarding peer panel funding decisions;
(2) Consider adopting a formal mechanism to accurately track,
by project, long-term environmental research and development
funding. (Implementing a system to track grants that have been
awarded may require coordination with the Grants Administration
Division to determine what additional data elements would be
required in the grants information and control system.
(3) Review the project workload of SRAs to ensure that, to the
extent possible, projects are evenly distributed to ensure
proper and continuous monitoring.
(4) Consider all of the concerns and criticisms directed at ORGC
as part of the review being conducted by the director of OER.
-------
-82-
MANAGEMENT RESPONSE
The Acting Assistant Administrator for Research and Development did not
offer comments on the above recommendations. However, the Assistant
Administrator for Administration agreed that post-award project officer
and SRA responsibilities cannot be adequately managed given present
resource allocations. Further, he encouraged the transfer of project
officer monitoring responsibilities after award from ORGC to the labora-
tories. He also stated that with respect to our recommendation regarding
the 15 percent congressional set-aside, the Office of Administration
interpreted this recommendation to mean an accounting system that will
track information on applications prior to award, and assumed that the
report is not proposing financial accounting for these funds outside the
Agency's central financial management (accounting) system. In this
regard, he also stated that pre-award tracking could be accomplished
through modification to ORD's existing system or, once funding decisions
are made, through additional data elements 1n the grants Information and
control system.
PIG COMMENTS
We revised our report to more fully clarify the recommendation regarding
ORD's need to track congresslonally mandated long-term research and
development funding. In this regard, we have not proposed a separate ORD
accounting system, but rather a more formal 0RD-w1de means of tracking
and reporting on such funding by research project. We agree that ORD
should consider developing a system to track projects by amending its
information system. We also agree that ORD may have to coordinate its
efforts with GAD.
-------
6. OFFICE * MHHH
•NVWRMWMM0UM
1MMAIMV
wom^wtw
Mint
CTSSV.S?"0* L
Atmm.a* I
HXMUIM 1
...^ uRA
AJT
aNtOkHMU
now*
MMMVinUMKMai
IAKWAV9M
MUHIllMl
WNWtltMMMK ItttM
iAMMUtOM
—Dirm?
HCAtl l| PJCKAHCM .
mntf _
I moanAnlortnAtioNt |
~1 mm I
API WWW A BAI'IAIWW
HtALIXMKAItCII
0IVUK7N
¦4tMM«»aM«A* Urn' 4
BHWI
^ WWTIH1WW ~
tUMTAKCU MMIW
RCSARGHOIVWOfl
¦MwttftenMT bim
¦nun utttn
Miiiwi
AtMMKM mm I NUM. «e
WMwiMrw mwi
¦unTI
MWttl
tftm)
bfna OF HfAltlf Atffl •
tNvmoNMCNtAl I i
lUUNtniMtMM MM
F-AncMoatM Aunwtwl
onour
WMfltf MtnMMMV U»n *
IwoJOfilKHFffiJIRr
onovr
WKI'WW IWW
HIWWUHIV»l»lCf»
AttCMMCNf onovr
tm»»om« ntnw
RtlUfM muMUrM&m
UWWWMf IWMIt
imMMifiMOMtlMI
ilHW|Wfff»K
CMCifMn«l
-------
APPENDIX B
-84-
RESEARCH COMMITTEES
1. Chemical Testing and Assessment
2. Pesticides
3. Radiation
4. Mobile Sources
5. Oxidants -
6. Gases and Particles
7. Hazardous Air Pollutants
8. Municipal Wastewater
9. Water Quality
10. Drinking Water
11. Solid and Hazardous Waste
12. Energy
13. Hazardous Emergency Response
-------
APPENDIX C
-85-
DAYS REQUIRED TO COMPLETE CONTRACT AWARD PROCESS
Contract
Days Required
Optimum *
or below o
68-02-2137
270
156
+
114
68-02-2631
90
156
_
66
68-02-2296
360
156
+
204
68-02-2703
330
156
+
174
68-02-2811
360
156
+
204
68-02-3101
15
156
141
68-02-3199
60
156
96
68-02-3162
120
156
_
36
68-02-3275
270
156
+
114
68-02-2993
180
156
+
24
68-02-3467
180
156
. +
24
68-02-3461
150
156
6
68-03-2855
82
156
_
74
68-03-2936
187
156
+
31
68-03-6189
202
156
+
46
68-03-2993
142
156
_
14
68-03-2795
360
156
+
204
68-03-3004
127
156
.
29
68-03-2791
345
156
+
189
68-03-2501
375
156
219
68-03-2610
255
156
+
99
68-03-2762
517
156
+
361
68-03-2523
472
156
+
316
68-03-2751
120
156
—
36
68-03-2984
180
156
+
24
68-03-2880
225
156
+
69
68-03-2803
465
156
+
309
68-03-2633
195
156
+
39
68-03-2711
165
156
+
9
68-03-2718
255
156
+
99
68-03-2800
300
156
144
68-03-2686
172
156
+
16
68-03-2578
660
156
+
504
68-03-2587
105
156
m
51
68-03-2945
75
156
81
68-03-2483
217
156
+
61
68-03-2567
360
156
+
204
Total
days 8,943
Average Number Of Days Required
Total Days Reoulred 89^3
Number o'r contracts 3 37 " 241.7 average
* This refers to EPA's optimum acquisition lead time
of 156 days. This is the optimal amount of time it'
should usually take to complete the award process.
-------
-86-
APPENOIX 0
AMOUNT OF TIME TAKEN TO OBTAIN FINAL REPORTS
Contract
Final
Months
Completion
Report
Date
Since
Contract
bate
Received
Received
Completlon
68-02-2137
01-05-80
NO
——
27
68-02-2631
07-26-81
• NO
—
9
68-02-2296
07-17-81
NO
—
9
68-02-3101
06-30-81
YES
06-81
---
68-02-3287
05-18-81
NO .
...
11
68-02-3162
12-05-79
NO
...
28
68-02-3275
11-30-80
YES
08-81
9
68-02-2993
01-29-81
NO
—
15
68-02-3461
06-29-81
YES
01-82
7
68-03-2855
12-21-80
NO
16
68-03-2936
07-23-80
NO
...
21
68-03-2795
05-01-81
NO
...
11
68-03-3004
09-22-81
YES
11-25-81
2
68-03-2501
03-03-79
NO
...
37
68-03-2610
11-30-80
NO
...
17
68-03-2984
09-12-81
NO
m
7
68-03-2880
04-07-82
NO
---
1
68-03-2803
11-14-81
NO
---
• 5
68-03-2633
02-25-82
YES
n/a|/
...
68-03-2711
12-14-81
YES
N/As/
...
68-03-2718
09-30-81
YES
N/A§/
68-03-2800
09-25-81
YES
N/A&
68-03-2686
04-21-80
YES
n/a|>
...
68-03-2578
03-15-81
YES
n/aS/
...
68-03-2587
02-25-81
NO
...
14
68-03-2945
07-31-81
YES
N/A2/
68-03-2483
12-24-80
NO
m w w
16
68-03-2567
04-16-81
NO
...
12
i/ As of April1982.
i/ Date received not available.
Summary
28
11
1-37
256
17 = IS average
Number of contracts requiring final reports
Number of final reports received
Range of months 17 reports not received were overdue
Average number of months 17 reports not received were overdue:
Months since completion
Final reports not recaived
-------
APPENDIX E
-87-
CONTRACTS AND AMOUNTS CHARGED TO
OBJECT CLASS 25.32 (RESEARCH AND DEVELOPMENT CONTRACTS)
INSTEAD OF TO OBJECT CLASS 25.35 (PROGRAM CONTRACTS)
Laboratory Obi 1 gated
Contract Location Amount
68-02-3262 EMSL/RTP $1,666,000
68-02-3405 EMSL/RTP 340,950
68-02-3431 EMSL/RTP 2,296,295
68-02-3487 EMSL/RTP 976,244
68-02-3496 EMSL/RTP 930,140
68-02-3226 HERL/RTP 566,710
Total 26,776,339
-------
-88-
APPENOIX F
Page 1 of 2
ILLUSTRATION OF FISCAL YEAR 1981 FUNDING OF
SUBAGREEMENT WITH OAK RIDGE NATIONAL LABORATORY
UNDER IAG NUMBER EPAIAG0533
Information Contained in EPA
Financial Management System
Document Control
Information Extracted
Files for
by. Auditors From Project
ft
ustration
and Account
Distribution of
Project Funding by
Account Number
Authorized Funding
Amount 1/
Project
2/
Number
Number
Amount
Nunfber
Amount
1
$
94,000
1
$ 94,000
2
83,000
3
90,000
2
83,000
4
69,000
EX4503
17
83,000
3
90,000
1C9J63W001
$419,000
S
419,000
4
108,000
4
$
39,000
5
211,000
EX4303
6
69,000
1C9A63W001
$108,000
$
108,000
6
94,000
EX4204
7
100,000
1C0U63W001
$ 25,000
6
$
25 ,.000
- ¦
-
Of
8
200,000
5
$
211,000
9
94,000
7
100,000
8
200,000
10
125,000
9
94,000
10
125,000
11
200,000
11
200,000
12
92,000
12
200,000
17
102,000
EX3607
13
80,000
1CCK63W001
$1,124,000
$1
,124,000
14
52,000
12
108,000
15
210,000
13
80,000
14
52,000
16
200,000
15
210,000
16
200,000
17
726,000
17
541,000
EX 3 904
1 CCV63W001
$1,191,000
$1,191,000
Totals
$2,867,000
$2,867,000
$2,867,000
-------
-89-
APPENDIX F
Page 2 of 2
1/ As shown, there were five document control and account numbers
~~ which provided a total of $2,867,000 for 17 projects. EPA's
accounting system identified only, the five numbers and not the 17
individual projects under this subagreement. Senior representatives
from the Cincinnati Financial Management Office advised us that,
upon receiving a voucher from Oak Ridge National Laboratory requesting
reimbursement (as previously discussed, these vouchers are based on
lump sum amounts and did. not Identify projects), EPA disbursed
funds from each account number in the order listed until EPA reached
the total subagreement funding of $2,867,000. For example, if EPA
received a voucher requesting $700,000, the disbursement would be
$419,000 from account number 1C9J63W001, $108,000 from account
number 1C9A63W001, etc., until EPA disbursed the full $700,000.
Accordingly, the accounting system did not track disbursements for
each project, and there was no assurance provided by the accounting
system that EPA disbursements did not exceed authorized project
funding.
2/ Each account number provided partial or total funding for numerous
projects. Projects 4, 6, 12, and 17 received partial funding from
two or more program elements.
-------
~90* APPENDIX 1
EXECUTIVE CORRESPONDENCE page i of
-i
| UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
~ -<¦>*
WASHINGTON, D.C. 20460
NOV 13 1282
owcx or
MCSSAftOI ANO OCVfUOMIIMT
SUBJECT: Oraft Audit Report Elgl32-ll-0019
. Report of Review of the Office of Research and Development
(tranu^al Research Activities
lordan
FROM:
VJ
Acting Assistant Administrator
for-Research and Development (RD-572)
TO: Steven- A. McNamara, Chief
Internal Audits Staff
Office of the Inspector General (PM-208)
We have reviewed your draft report evaluating the conduct and management
ofORO's extramural research activities- T have four general observations
.regarding the report.
1) It 1s balanced and fair. The report takes note of both the
strengths and weaknesses of the Office of Research and Development (ORD)
and, where 1t is not possible to offer an opinion at present, it notes
that fact. In short* the report 1s objective.-
2) It 1s well researched and detailed. Perhaps reflecting the
very large dedication of man-power, the report displays ample evidence
of basing Its conclusions-on facts and not Impressions.
Z) It 1s useful and specific- The facts developed are discussed
in a professional manner and comments and suggestions rationally flow
from the discourse. The recommendations are sensible and achievable.
4-) It 1s. somewhat too lone and repetitive. Our only substantial
criticism* is that the report bears evidence of 'stapling together" the
worlc of many Investigators and therefore Its length and repetitive
quality can be disconcerting. Substantial editing- would markedly
improve the quality of the document. We encourage this activity.
The report offers four reccnsnendations to the Assistant Administrator
(AA) for as well as recoinnending that the Administrator appoint a
permanent AA for ORD as soon as possible to give necessary stability...".
Allow me to point out that* at least in the recent past, ORD has operated
-------
Page 2 of
-91-
In a professional manner and continued to plan for its future evenvin
the absence of permanent leadership. This is a tribute to the coopera-
tiveness and professionalism of QRD career staff, and it should be
recognized.
Allow me to offer my contents on the four reconsnendations to the
AA, QRD:
1) "adopt improved uniform, consistent procedures for dissaninating
and maintaining policies and procedures governing QRD operations."
In the past ORD had such a system embodied in a "Policy and
Procedures Manual.u We agree that such a device (or something
similar) is necessary and we will devote attention to preparing
and issuing it.
2) "continue to pursue with: vigor, implementation of improvements,
and changes in the computerized management information system
(OROIS) and work to gain the support, of all ORD parties involved
to enhance chances of its success.u We agree that QRDIS is a
high priority activity and we are expending all efforts to insure
its success.
1
3) "reemphasize the need for effective communication between
QRD and media program- offices,, and other ORD clients to
further enhance research planning and managanent." Effective
conanunication with our clients 1s one of our highest priorities.
We have taken steps this past year to strengthen the dialogue
by starting the planning process with a "Mega-stratagy" signed
by both the AA'for ORD and the client AA. This strategy then
serves as the policy guidance document to the research committees
charged with crafting- the actual budget decision units. We
share your concern and continue to seek new ways to enhance our
-jcanxnunication with clients.
4) "conduct a comprehensive evaluation of (1) the adequacy of
existing and planned travel funds, as well as (2) the manner in
which- such funds are allocated within QRD, and-(3) upon completion,;
present the results of this evaluation to the Administrator for
budgetary'Considerations." We are sensitive to the need for
adequate travel funds and we have already conducted the sort of
analysis you suggest for part of FY-€2. We are now updating
the study to Include all of FY-S2. We will determine if any
reallocations are indicated based on the results of our study.
rn sumnary, either we are already working to achieve the report's
recommendations (i.e., QRDI5,. ccnmunications, travel analysis) or we are
connritted to proceed (i.e., policy and procedures disssnination). Further,
-------
-92-
Page 3 of
we are pleased to note that the report finds that our proposed reorganization
seems- reasonable and promises further improvement. We share that optimism
although we must agree that final judgment as to the efficacy of the'
reorganization only can be delivered in the future. Nevertheless, we
appreciate your confidence, we share your expressed concerns and we
applaud the-balance and objectivity of the report.
Thank you for the opportunity of reviewing the report in draft
and providing these comments.
-------
-93-
APPENDIX G
^ A \ UNITED STATES ENVIRONMENTAL PROTECTION AGENCY page 4 0f 4
i®, I WASHINGTON. D.C. 20460
ar
N pqQl*"
JAN 21 1383
OFFICE OF
RESEARCH ANO DEVELOPMENT
SUBJECT: Contract Appropriation Charges
/?!
FROM? Courtney Riordan /6/
Acting Assistant Administrator
¦ for Research and Development (RD-672)
TO: Matthew N. Novick
Inspector General (A-109)
My previous memo to you regarding the nine questioned contracts which
Mere charged to the Research and Development appropriation advised that
the QRD agreed that the contracts could have been charged to the Salaries
and Expenses appropriation. I did not mean to imply that the charges
should be so charged. The ambiguities contained in the definitions of
each appropriation are such that the contracts could have been charged
to either one of the- two accounts.
The revised audit report just received by my staff has been reviewed
and I cannot agree with your reccmnendations that the charges be moved
to the Salaries and Expenses appropriation „ I wish to go on record as
stating that I consider that the nine referenced contracts charged to
the Research and Development appropriation were, by any reasonable
interpretation, valid and should remain as charged. •
I also want to reiterate that I consider the object class definitions
presently available to be sufficiently ijrprecise so that many contracts
could be correctly charged to either appropriation» My staff is working
with the Comptroller and with the Office of Administration to resolve
these ambiguities and put in place more precise guidance. >fe have provided
draft object class definitions as a basis for discussion to resolve this
issue. We would be pleased to share these with you or have you join us
in seeking a more precise set of definitions to avoid future confusion.
cc: Morgan Kinghorn
-------
/ A i
mj
-94-
UNITED STATES EN V1SONMENTAL. PROTECTION AGENCY
WASHINGTON. D.C. 204SQ
Page 1 of c
t 5 1982
op pica op
AOMIN JSTflATJON
MEMORANOOM
TO
Steven A. McNamara, Chief
Internal Audita Staff
PWDMs
SUBJECT: Comments on Draft Audit Report ElglB2-llr-QQ19
I appreciate the opportunity to review and comment on the
subject report. Many OA offices spent considerable time in pro-
viding the background, information used to prepare the report/ and
I share your disappointment, that the review could not fully answer
all of the questions posed» We have summarized below our ccmments
on the two draft reccm mend at ions addressed to OA* I have also
included, a number of ccmments and editorial suggestions you may
find useful in preparing the final report.
Draft Recommendation t We recommend that the Director of the
Office of Fiscal and Contracts Management review and strengthen
controls over level-of-effort contracts to provide a direct link
between the finding and work tasks perfomed. (P.52 3.)
Commentr Me- have been aware of this problem for some time and
the Procurement and Contracts Management Division (-?CMD} has been
working with the Financial Management Division to devise a solution.
?CMD has developed a draft Procurement Information Notice (PIN)
incorporating policies and procedures designed to address this
problem;. The draft PIN is currently being circulated. fog_canment
and I have attached a copy for your information..
Draft Recommendations we recommend that the Assistant Adminis-
trator £or Administration: 1* finalize and disseminate new proce-
dures-governing the administration of. IGAs as expeditiously as
possible. 2. incorporate a requirement in draft and. final IGA
policies and procedures to ensure that: (a.) project officers review
and certify reimbursement vouchers to ensure, chac there is a proper
expenditure of project funds and that work has 'seen -cerforaed in'
accordance- with the teems of the agreement; (b.) the terms of IGAs
include a clear requirement to provide EPA with, detailed project
cost information; (c.) EPA media program offices clearly understand
their responsibilities regarding receipt and. review of detailed
project cost information submitted, by cither Federal agencies;
-------
-95-
Page 2 of
and that (d.) media program offices provide sufficient accounting
information to PMD-to enable EPA's accounting system to track,
obligations and disbursements by project. (P.63 3.)
Comments: We generally concur with the recommendations but
find that most of these concerns have been addressed in the
reorganization of the interagency agreement management functions
into OA's Grants Administration Division* The Grants Administration
Division will have responsibility for legal and administrative
' management of interagency agreements, including maintance of the
official file, execution of agreements and management as the action
office. These responsibilities will enhance ORD's use of interagency
agreements. Specifically, our 'draft interagency agreement procedures
are being discussed with ORD to ensure that our respective responsi-
bilities with regard to technical, legal and administrative
management responsibilities are properly integrated. OA expects to
issue final reviewed and approved procedures by November 1,*1982.
Historical difficulties in tracking individual project costs have
occurred because other Federal agencies billed EPA in one lump sum
and did not identify specific project costs. The draft procedures
will require that an interagency agreement be separately prepared
for each specific proposed project- These individual cost estimates
will be available- to the- project officers, and the project officer
will review and approve vouchers* These- changes, should remedy the
difficulties highlighted, in the draft recommendations 2a, b, and d*
Detailed guidance-regarding OA. responsibilities will be made avail-
able to all ORD officials who need to be involved in the initiation,'
negotiation, development, monitoring, ,and determination of interagency
agreements. This guidance will be available before the end of this
-calendar year. Additionally, we believe that-general training can
be provided to all the lab coordinators and pertinent Headquarters
ORD staff to ensure that the- procedures effected December 1, 1982
are used correctly.
Generally, we concur with many of the recommendations and
findings appearing elsewhere in the report* Because cooperative
agreements and research grants are viewed as assistance rather than
contractual arrangements, we do suggest, however,, that Chapters- 2
' Vnd 5 appear sequentially in the reports Our comments on'Chapter 1~
are limited to.editorial suggestions to delete excess commas on
line 1 of page 12, line 2 of. paragraph 3 on page 13.and clarify the
reference to EPA Order 220(7.4 on page 20 - We generally concur with
the findings and recommendations addressed to ORD in Chapter 2* We
note that the reference to Office Directors on page 32 should
perhaps be revised. in light of the pending reorganization.
Cur- comments on Chanter 2 primarily concern the dicussion on
pages 41 thrcugh 45. We take issue with the report's apparent
conclusions that alleged delays in the award of procurements are
impacting timely completion of contracted work* When a contract is
awarded, it contains a schedule of deliverables (generally final
reports) which the program office has detenained will meet its
-------
"96"
Page 3 of
project requirements. Delay in the recsipt of deliverables, as
called for in the contract/ can certainly adversely impact project
completion (as the report points out happens in too many instances);
however, it is unclear how the procurement process leading to the
award of the contract causes this effect, we also point out that
the report and Appendix C reference lead times of IS 6 days for caa—
pecitive procurements and 119 days for non-competitive procurement
as "standard leadtimes". These leadtimes have never been established
as standards but rather are defined as optimum. Sach individual
procurement is analyzed to determine the leadtime necessary to
award that particular contract; there is no standard leadtime. We
also note a typo oa the 10th line o£ paragraph 3 on page 41 (last
word). Finally/ the review referenced in point 5 on page SI has
been caapleted.
in addition, to our previous ccmments on the recommendations
contained in Chapter 4, we point out that the term intergovern-
mental agraanent is limited to only one kind of agreement, an
agreement between 2PA and the State or local government authorized
to enter into agreement under the Intergovernmental Cooperation Act.
The report^ in places, inappropriately uses the term, to apply to all
types of agreenents. For the purposes of discussing all types of
agreements/ the tent "interagency agreement" is more appropriate
and preferred*
While Chapter 5 address no recommendations to OA/ we have
several comments on the findings contained therein. We do not believe
that the post-award project, officer and SRA responsibilities can be
adequately managed given present resource allocations. Given resource
constraints/ we would encourage the transfer of project officer moni-
toring responsibilities after award from ORGC to the laboratories.
Finally, we- note the auditor's recommendations that ORD develop and
utilize an accounting system to track the 13% of ORD funds which
Congress mandated for long tern environmental research and develop-
ment » We interpret this recommendation to mean an accounting systeu
that will track info mat ion on applications prior to award/ and
assume that the- report is not proposing financial accounting for
these-, funds outside the Agency's central financial management
(accounting) system. This distinction should perhaps be-clarified
in the report. Preaward tracking could be accomplished through
modifications to OHD's existing system or/ once funding decisions are
made, through additional data elements in the grants information and
control. system. (GICS) -
I suggest yeu contact, the Divisions in OF CM to review any final
changes proposed to the report and to answer any questions that these
ccmments occasion.
Attachment
-------
-97-
I .S. Environmental Protection figenaj I Pa'e 4 of
PROCUREMENT INFORlTlflTlON HOT1CS
— DRAFT
j Revised Policy for Cost Reimbursement Term Form Contracts
Utilizing WorJc Assignments
! taviKa: Chapter 24 of the Contracts Management Manual
tapos* & Scop^: This policy is designed to ensure that the correct
allowance holder and program element appropriations
are charged for work done by voric assignments on any
cost reimbursable tera form, contracts-
Cost reimbursable tera form contracts are being awarded
by this Agency that allow work. assignments to be issued
by any program office having need for the service provided
by the- contract*. The Financial. Management Division has
been experiencing difficulty in determining which
allowance holder account to charge as contractors are
submitting vouchers for the completed work assignments*
This- problem becomes particularly acute when the contract
contains funding by more than one allowance holder and
work assignments, are issued in support of morej than one
program, office.
To obviate this situation the following policy shall be
foJJ.owedr
S» HO FQMt litwi <1-7«
-------
-98-
Page 5 of
1. Program funding oust be obligated in a contract before that
program office say issue work assignments^ against the contract.
2. In contracts that have multiple allowance holder funding, the
program office may issue work assignments only against the funds
In the contract that support the program office requesting the
work assignment*
3. Sach work assignment .requested by a program office shall
identify the program account (by number) whose funds in the
contract, will, be charged for the woric assignment-.
4. For work, assignments that are requested by more than one
program officer the program offices oust determine and state in
the work assignment the percentage of the costs to be charged to
each of their respective accounts.
5. In contracts that contain multiple funding, a statement shall
be included in the Special Instructions of the contract that the
Limitation of Cost, clause applies-to each program line of funding
in the contract against, which work assignments are being issued.
6. The. work assignment:, when issued, to the contractor shall
contain a statement: that: a- separate voucher will be issued for
costs incurred against that, woric assignment* The work assignment
shall contain arid the voucher" shall, identify the account number(s)
to- be charged and.in the case when more than one account is to be
charged, the percentage of the costs to be applied to-each program
account.
The following article ihail be used in all cost reimbursable term
form contracts utilizing work assignments*
•AB2XCL2 - WOES ASSIGNMENTS
(a) The (a portion of the) work, to be performed under this
contract, will, be defined in Work Assignments issued by the
Contracting Offices* The- Woric Assignments - will be within
the ' of the Statement of Work of this contract and
will, be- in writing*- As a~ minimum#, each. Work Assignment will
include: (1) the numerical designation of the Work Assignment;
(2) the Government's estimate of required man hours (and
laboc categories) ? (I) the required period of performance
of the Work Assignment (schedule of deliverables) r (4) the
Statement: of Work of the Work Assignment; (5) the programis)
account to be charged, and (6) the percentage co be used
when aora than one account is tc be charged for a single
Work Assignment*
-------
-99-
Page 6 of
(b) The Contractor shall acknowledge receipt or the ftoric
Assignment by signing the Assignment. The signed Wocic
Assignment. shall be returned to the Contracting Officer'
within (number) working days following the Contractor's
receipt of the Work Assignment.
(c). This Article in no way modifies or affects the provisions
and requirements of the Article entitled (name the applicable
level, of effort Article}, and. the clause entitled "Limitation
o£ Cost." or "Limitation of Funds#' whichever is prevailing.*
2T?SCTT7S DATS;
ACTION OFFXCZXs
This ?IN is effective upon issuance^
Larry Sawler (PM-214)/ telephone 332*3132
Saul. A. Martin.
Acting Director '
Procurement and Contracts
Management Division (PM-214)
-------
J
z
-100-
UNITED STATES ENVIRONMENTAL. PROTECTION AGENCY
. WASHINGTON. D.C. 20460
APPENDIX I
Page 1 of 7
FEB t 1303
OPPICS OP
POLICY ANO RESOURCE MANAGEMENT
for QRD Contracts
To : Matthew N. Novlck
Inspector General (A-109)
As per your request we have reviewed the nine Office of Research and
Development contracts which you have questioned to determine the appropriate
funding source. The results of our analysis are outlined below:
1. Contract No. 68-02-2566, Northrop, $8,966,262
This contract contains both research and development and salaries and
expenses activities, however, the majority of activities are research and
development. The following sections of the contract are primarily research
and development: 2..0 to 3.8.1, 4.3 to 4-.7. The following sections are
primarily salaries and expenses: 3.9.1 to 3.9.3, 4.1, 4.2, 5.0, 5.1.
Your auditors have Informed us that approximately 60S of the contract
was charged to the R & 0 appropriation and 40% to salaries and expenses.
As far as we, or anyone else at this stage can tell, the distribution of
charges was appropriate*
2. Contract Mo. 68-02-2832, SOC Integrated Services, $277,000
This Is an ADP contract as defined by the Agency's Planning Manual
and therefore should have been charged to the salaries and expenses appro-
priation as indicated 1rr Appendix 8 pages 6-8 of the Planning Manual.
3. Contract Mo. 68-02-3296, Systems Research & Development, 525' ,600
This is an ADP contract and should have been charged to the salaries
and expenses appropriation.
4. Contract No. 63-02.-3482, Rockwell International, $310,000
This 1s a contract to repair and maintain equipment and should have
been classified as an-Other Contractual Service contract and charged to
the salaries and expense appropriation 1n accordance with Appendix 8 of
the PIanning Manual.
-------
APPENDIX I
Page 2 of 7
-101-
¦ 5. Contract Mo. 63-02-3199, Spectron Development Laboratories, 555,000
This 1s a contract to purchase scientific equipment and should have
been classified as Equipment and charged to the salaries and expense
appropriation 1n accordance with Appendix B of the Planning Manual.
6. Contract Mo. 68-^03-2672, Centec Consultants, Inc., 5793,405
This 1s a research and development contract and has been charged
appropriately to the R & D appropriation. Items A through H In the
scope of work are "development activities" as defined by 0MB Circular
A-ll.
7. Contract No. 68-03-2913, Mathtech, 5239,108
This 1s a research and development contract and has been charged
appropriately to the R & 0 appropriation. I tens 1 through 3 1n the
scope of work are "development activities" as defined by 0MB Circular
A-ll.
8. Contract No. 68-03-2765, Radian Corp., 51,566,501
This Is a. contract to provide quality assurance services and has
-been charged appropriately to the R 4 D appropriation in accordance with
-the-definition of research and development in the Planning Manual.
9. Contract No» 68-03-2846, University of Cincinnati, 51,522,585
. This is a contract to. assist EPA in the operation and design of
research experiments and has been charged appropriately to the research
and development appropriation consistent with the definition of "research"
provided in QMB Circular A-ll.
-------
-102-
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON. D.C. 20460
FEB t 1983
APPENDIX I
Page 3 of 7
OFPICE OF
poucy ano aesouacE managsmsnt
MEMORANDUM
Subject:
F roatr^ cmo rgan,
Coruptrol
of Rej^rch/and Development Appropriation
nghorn
(PM-225X
To: Matthew M» Novlck
Inspector General (A—109)
In talking to your staff 1t has become clear that there is some
confusion 1n their nrfnds as to what definition of the research and devel-
opment appropriation the Agency Is using 1n budgeting and charging
obligations and expenditures. The definition which the Agency uses for
the research and development appropriation Is the one provided In QMB
Circular/A-ll section 44.2. as modified by the definitions incorporated
in the Agency's Planning Manual and the Appendix to the President's
Budget* I have attached the relevant portions of these-documents.
It is important to note that RSD activities as defined by Circular
A-ll may be charged to either the salaries and expenses or research and
development appropriations. If the activities inv'olve in-house staff and
intramural object classes (P.C.48, travel» ADP contracts, etc.) they
should be charged- to the saTartes and expenses appropriation. If the
activities involve extramural services and object classes (contracts,
grants etc.) they should be charged to the research and development
appropriation.
There were several specific activities Itemized in the contracts
which your auditors felt should not have been charged to the research
and development appropriation. These activities are research and
development as defined by the documents noted above. The areas are as
follows:
Quality Assurance
Quality assurance is a research and development activity as defined
fn the Agency Planning Manual. Tnis appropriation (RSI) provides
standardized methods to treasure and assure quality control,
-------
-103-
APPENDIX I
Page 4 of 7
2. Monitoring
Monitoring 1s a research and development activity and is specifically
mentioned in the Appendix to the President's Budget as one of the activities
for which the funds will be used. " Activities encompass research on
,the development of new and improved sampling and analytical
methods and Instruments for measuring pollutants; "
3. Review of Research Documents, Evaluation erf Technical Outputs,
Preparation or Manual s/Hanopootcs, conduction of seminars/woncsnop?
These activities are appropriately charged to the research and develop-
ment appropriation as defined by 0MB Circular A-ll as long as they Involve
the systematic use of Icnowledge gained from research directed toward the
production of useful materials- This Is the case for the contracts
under review. Circular A-il defines development as the; "systematic use
of the knowledge and understanding gained from research, directed toward
the production of useful materials, devices, systems or methods, including
the design and development of prototypes and processes."
4. ADP Services
ADP services wl thi n-contracts which primarily Involve research
and development activities should be charged to the research and
development appropriation. Contracts which Include only ADP activities
should be charged to the salaries and expenses appropriation. This
information on ADP charging Is provided 1n the Planning Manual.
Attachments
-------
V -104-
^ \ UNITED STATES ENVIRONMENTAL PROTECTION AGENCY APPENDIX I
; Page 5 of 7
WASHINGTON, D.C. 20460 3
MAR 2 4 1983 OFFICE OF
POLICY ANO RESOURCE MANAGEMENT
MEMORANDUM
Subject: Ccnuhe^ts on InspectajOieneral's Audit Report
^SHD's Extradural [Research Activities
L-"' / j l^fa«ii ^ /L—
Fran: MorganIcir^hornl Comptroller
To: Charlesviempsey, Lztirig Inspector General
Office of Inspector General
As per your request outlined below are our comments on the
ORD audit report.
IG Statement: "Conduct or direct a review of the remaining con-
tracts for fiscal 1980, 1981, and the first quarter of 1982,
which were not included in our sample, to determine whether any
other contracts were improperly charged." p. 55.
Comptroller's Comments: We do not feel "that such~a~review would
be fruitful for the following reasons;
o The magnitude of the mischarging indicated in the report is
not large enough to warrant such a review. The IG's sample
of contracts revealed that only 4% or $896,600 out of a total
$20.1 million was incorrectly charged to the Research and
Development appropriation.
o Although the four contracts were incorrectly charged, no one
is suggesting that the funds were not used to support ORD's
major missions.
o We feel that our time would be more productively spent on
ensuring that such mischarging does not occur in the future.
Your report mentions a number of actions which the Agency is
taking to improve the management of extramural research
activities. We have long supported those efforts.
o We do believe EPA should examine all R&D contracts for FY 1983
to ensure there are no mischarges. We agree that there maybe
continuing problems.
OFFICE OF
iPECTOR GENERAC
24 MAS RECD
* 6 - i/lv-"'
-------
-105-
appendtx r
Page 6 of 7
IG Statement? "We believe the Office of the Comptroller, PCMD
and ORD need to effectively resolve these ambiguities (in the
definitions of the R&D and S&E appropriations) and take appropriate
steps to ensure that contracts are correctly classified and charged
to proper appropriations." p. 49.
Comptroller'3 Comments: We do not agree that there are significant
ambiguities in the definitions. The IG's report does not indicate
what these ambiguities are.. However, as the report recommends
on p. 55 we will work with ORD and provide them with additional
guidance if they feel it i3 necessary.
IG Statement: "We found 6 instances amounting to about $6.8 million
where it appeared contracts were classified as R&D contracts
(object class 25.32) rather than program contracts (object class
25.35)."
Comptroller's Comments: The two contracts used as examples of
misclassified contracts are R&D contracts as defined in the
Comptroller's memo to the IG. The two contracts in question are
quality assurance contracts. Quality assurance is a research and
development activity as defined in the Agency Planning Manual.
Thus ORD charged the appropriate object class, "research contracts".
The "program contracts" object class was designed to be used
mainly by the program offices' and OPRM. We would expect most of
ORD's contracts within the R&D appropriation to be R&D contracts.
IG Statement: "Comprehensively evaluate (1) the adequacy of
existing and planned travel funds and (2) the manner in which
such funds are allocated within ORD." p. 27
Comptroller's Comments: The Comptroller's Office feels that ORD's
total travel funds are adequate. Since 1981 their travel ceiling
has increased frcm $2.2 million to $2.7 million in 1983 while
their extramural resources have declined by approximately 50%.
In FY 1982, ORD lapsed $230 thousand in travel funds or 9% of
their ceiling. We agree that the Assistant Administrator for
Research and Development should examine the allocation of travel
funds within ORD and if appropriate, re-allocate some to support
monitoring of extramural resources.
IG Statement: "Contractors frequently failed to deliver final
reports to EPA within the contractual time frames, and in a
significant number of instances, EPA had not received final
reports reports were overdue from 1 to 37 months,
or an average of 15 months for each contract." p. 44.
-------
-106-
APPENDIX I
Page 1 of 1
Comptroller'3 Comments: Given the constant criticism: of the
timeliness of ORD's products we feel that this is a significant
finding . The report should recommend that ORD review the list
of 41 completed contracts and provide an explanation as to why
the final reports were so late.
IG Statements "We recommend that the Assistant Administrator
for Administration direct the Office of Fiscal and Contracts
Management to review and strengthen controls over level-of-
effort and other term form contracts to provide a direct link
between funding and work tasks performed" p. 55.
\
Comptroller's Comments: We agree that the Agency needs to
strengthen its procedures to ensure that funding sources are
•linked to specific work tasks.
------- |