^•^M^« uJ
31
.
\ Office of Inspector General
. Report of Review
SURVEY OF
RISK REDUCTION
ENGINEERING LABORATORY'S
QUALITY ASSURANCE ACTIVITIES
E1JBB5-23-0002-5400099
September 8, 1995
-------
Inspector General Division
Conducting the Audit: Northern Audit Division
Cincinnati, OH
Program Offices Involved: Risk Reduction Engineering
Laboratory, Cincinnati, OH
Office of Research and
Development, Quality Assurance
Management Staff
Washington, D.C.
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
OFFICE OF THE INSPECTOR GENERAL
NORTHERN DIVISION
77 WEST JACKSON BOULEVARD
CHICAGO, IL 60604-3590
September 8, 1995
MEMORANDUM
SUBJECT: Special Report E1JBB5-23-0002-5400099
Survey of Risk Reduction Engineering
Laboratory's Quality Assurance Activities
FROM: Anthony C. Carrollo
Divisional Inspector General for Audits
Northern Division
TO: E. Timothy Oppelt, Director
National Risk Management Reduction Laboratory
Cincinnati, OH
Attached is our final report entitled "Survey of Risk
Reduction Engineering Laboratory's Quality Assurance Activities."
We performed a survey of the Laboratory's quality assurance (QA)
program at your request. Our overall objectives were to
independently evaluate the QA program, assess its successes, and
suggest areas where improvements were needed. We found that the
Risk Reduction Engineering Laboratory (RREL) and its QA office
employed several good management practices related to the
achievement of QA goals and objectives. Most importantly, RREL
management officials were extremely committed to QA and this
attitude, therefore, seemed to permeate the organization.
We appreciate the cooperation and responsiveness we received
from your staff during this review. Your staff's professionalism
and commitment resulted in a cooperative effort to resolve the
two issues we identified in this special report.
This special report represents the opinion of the OIG.
Final determinations on matters in this report will be made by
EPA managers in accordance with established EPA audit resolution
procedures. Accordingly, the findings described in this report
do not necessarily represent the final EPA position.
ACTION REQUIRED
In responding to the draft report, your office took action
to correct the findings presented. As a result, your response to
the report is adequate in accordance with EPA Order 2750.
Therefore we are closing out this report in our tracking system
effective today, September 8, 1995.
Printed on Recycled Paper
-------
We have no objections to the further release of this report
to the public. Should you or your staff have any questions
regarding this report, please contact Leah Nikaidoh, Audit
Manager, Northern Audit Division, Cincinnati Branch, at
(513) 366-4365.
-------
RREL QA ACTIVITIES
EXECUTIVE SUMMARY
PURPOSE AND OBJECTIVE
The Office of Inspector General (OIG) performed a survey of
the Risk Reduction Engineering Laboratory’s (RREL) quality
assurance (QA) program, at the request of RREL management.
RREL officials believed their investment in resources devoted
to QA activities resulted in an effective program. According
to RREL officials, the successful implementation of QA
results in a savings to RREL of over $1 million annually.
Various Environmental Protection Agency (EPA) labs have
recently been criticized for inadequate QA programs and not
having Quality Management Plans documenting QA policies and
procedures. Furthermore, EPA cited environmental data
quality as a Presidential level weakness in its Federal
Managers’ Financial Integrity Act reports for fiscal years
1992 through 1995.
Our overall objective was to independently evaluate RREL’s QA
program, assess its successes, and suggest areas where
improvements were needed. On May 1, 1995, RREL was
consolidated with several other EPA organizations to form the
National Risk Management Research Laboratory (NPJvIRL).
RESULTS- IN-BRIEF
RREL’s QA program was well-managed, and met EPA QA
requirements in an innovative manner. RREL management was
committed to achieving QA goals and objectives, and this
commitment was evident in its QA reporting structure, its
philosophy of continuous improvement, and in the level of in-
house resources RREL management dedicated to carrying out QA
functions.
RREL’s QA policies and procedures (internal controls) were
well-documented in its Quality Management Plan, and
implementation of the Plan was evident. RREL’s QA office
also performed QA review functions--such as the review of QA
project plans and the performance of technical systems
reviews--in a timely and thorough manner.
The QA office also maintained good oversight of its two QA
support contracts. RR.EL had adequate procedures to ensure
that its QA support contractors were not performing
inherently governmental functions. The QA office ensured
1
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
that its staff was the final decision maker regarding the
approval of QA project plans and technical systems review
reports. We found no indication that QA support contractors
performed any inherently governmental functions.
RREL’s QA program also included the development and
implementation of innovative approaches to QA. RREL
officials adapted Agency QA procedures to meet specific RREL
needs. RREL QA officials also structured the QA program to
be proactive rather than reactive, thereby resulting in
dollar savings in some cases.
We found two issues which have potential to impact RREL: (1)
two RREL Technical Project Managers who functioned as Work
Assignment Managers (WAM5) had not taken a required contract-
related training course and (2) additional organizational
conflict of interest (OCI) controls were needed. Efforts
should be made to address these issues. Although we did not
identify any negative impact or effect from these conditions,
there are potential effects that could impede the program’s
progress. Because our survey did not disclose any material
weaknesses in RR.EL’s QA program, we did not proceed into the
audit phase.
LABORATORY COMMENTS AND ACTIONS
The NRMRL Director agreed with the recommendations in this
report. He stated that management will, by November 6, 1995,
review training records of all NRMRL Cincinnati (NRMRL-CI)
staff acting as WAMs and identify those lacking required
contract-related training courses. The Director also stated
that new Office of Acquisition Management procedures, found
in the September 27, 1994 version of the Contracts Management
Manual, help to ensure that WAMs who lack training take
required courses. According to the Director, RR.EL and Office
of Research and Development officials placed a significant
emphasis on training.
In response to the recommendations pertaining to OCIs, the
NRMRL Director stated that the NRMRL-CI QA staff has begun to
implement the following procedures:
(1) Project Officers for QA support contracts will
distribute to the NRI4RL-CI QA staff lists of companies
with which QA support contractors do NRMRL-CI business.
The Director stated QA support contractors will update
these lists quarterly, beginning October 1. The
Director also stated that QA support contractors will
ii
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
notify the appropriate EPA Project Officer when they are
awarded non-QA NRMRL contracts.
(2) The NRMRL-CI QA staff will review the lists described
above when QA support is requested for a contracted
project. If the project contractor is on a QA support
contractor’s list, the NRMRL-CI QA staff will place a
“C” on the project file to indicate that a potential OCI
exists. When an in-house staff member obtains the file,
he or she is immediately alerted that a potential OCI
exists with one or both of the QA support contractors.
(3) If possible, the NRNRL-CI QA staff will provide QA
support in-house or task a QA support contractor from
another EPA laboratory when potential OCI situations
arise. The Director anticipated that in-house staff
would perform nearly all QAPP reviews by fiscal year
1998.
We included a summary of the Director’s comments in
appropriate sections throughout the report. Appendix 1
contains a copy of the Director’s written response to our
draft report.
OIG EVALUATION
The NRNRL-CI staff’s actions, when completed, will address
the recommendations in this report.
iii
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
[ This page intentionally left blank.)
iv
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
TABLE OF CONTENTS
EXECUTIVE SUMMARY ±
CHAPTERS
INTRODUCTION 1
PURPOSE 1
BACKGROUND 1
OTHER RELATED MATTERS 5
SCOPE AND METHODOLOGY 6
2 GOOD MANAGEMENT PRACTICES 9
RREL MANAGEMENT’ S
COMMITMENT TO QA GOALS WAS EVIDENT 9
INTERNAL CONTROLS WERE
ADEQUATE AND APPROPRIATE 11
QA OFFICE’S REVIEWS
WERE TIMELY AND THOROUGH 12
QA SUPPORT CONTRACTORS WERE NOT
PERFORMING INHERENTLY GOVERNMENTAL FUNCTIONS 13
RREL’S QA PROGRAM WAS
INNOVATIVE AND RESULTED IN SAVINGS 14
3 OTHER ISSUES 17
WAMs LACKED REQUIRED
CONTRACT-RELATED TRAINING 17
ADDITIONAL ORGANIZATIONAL
CONFLICT OF INTEREST CONTROLS NEEDED . . . . . . 19
EXHIBIT
EXHIBIT 1
RREL PROJECTS REVIEWED BY OIG 25
V
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
APPENDI CES
APPENDIX 1 AGENCY RESPONSE TO DRAFT REPORT . . . 27
APPENDIX 2 ABBREVIATIONS 31
APPENDIX 3 DISTRIBUTION 33
vi
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
CHAPTER 1
INTRODUCTION
PURPOSE
RREL management requested that the OIG perform a survey of
its quality assurance (QA) program. Our specific objectives
were to:
• determine if RREL’s internal controls related to the
achievement of QA goals were adequate;
• determine if QA project plans (QAPPs) were prepared, as
required, for projects involving environmental data
measurements;
• determine if QAPPs contained information in accordance
with EPA requirements;
• evaluate the appropriateness of RREL’s QA support
contracts; and,
• identify performance measures RREL management used to
assess the success of its QA program.
BACKGROUND
EPA management decisions rest on the quality of environmental
data. Therefore, EPA requires that all routine or planned
scientific projects involving environmentally related data
measurements adhere to certain QA requirements. These
requirements can be found in EPA Order 5360.1, “Policy and
Program Requirements to Implement the Mandatory Quality
Assurance Program.” The primary goal of EPA’s QA program is
to ensure that projects produce data of known quality; that
is, how the data were derived is thoroughly documented,
verified, and defensible. The Quality Assurance Management
Staff (QAMS), within the Office of Research and Development
(ORD) in Washington, D.C., is responsible for general
oversight of QA activities within EPA. RREL’s QA office was
responsible for oversight of all QA activities within RREL
and meeting the requirements in EPA Order 5360.1.
QA activities are crucial in maintaining superior scientific
research. EPA officials are required to prepare QAPPs for
each project that involves data gathering, generation, or
measurement. A Quality Management Plan (QMP) must also be in
1
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
place for each laboratory, with annual review and updating,
as necessary. An annual report and workplan must be prepared
each year to detail accomplishments of the past year and
describe future QA activities. The QA effort is a valuable
internal control technique with strong ties to the goals of
the Federal Managers’ Financial Integrity Act (FMFIA).
EPA created its QA program in 1979. In 1980, EPA issued
guidelines for preparing QMPs and QAPPs, two key processes
for ensuring quality work. In 1983, QA requirements were
incorporated into assistance regulations (see 40 Code of
Federal Regulations (CFR) Parts 30 and 31) and acquisition
regulations (see 48 CFR Part 15). In April 1984, EPA Order
5360.1 was issued to document the Agency’s requirements to
implement its mandatory QA program.
RREL’s Mission and Functions
EPA’s Risk Reduction Engineering Laboratory (RREL) is
headquartered at the Andrew W. Breidenbach Environmental
Research Center in Cincinnati, Ohio. RREL’s primary mission
is to develop and apply engineering approaches to solve or
prevent environmental problems that threaten human health and
the environment. RREL focuses mostly on the following areas:
drinking water, industrial and municipal wastewater,
hazardous waste, site remediation, leaking underground
storage tanks, and oil spills. RREL performs highly
technical work. Included among the many scientific
disciplines they employ are chemistry, engineering,
microbiology, statistics, soil science, geology, and mining
science.
Specifically, RREL staff:
• Compare drinking water treatment systems for
effectiveness and cost, and also study ways to control
disinfection by-products.
• Search for cost effective ways to treat and manage
wastewater, including sludge, urban runoff, and
industrial discharges.
• Develop new disposal technologies for hazardous waste,
including innovative concepts such as bioremediation
which uses safe, natural organisms to degrade pollutants
in water, air, or soil.
• Oversee the operation of the Incineration Research
Facility in Jefferson, Arkansas, where scientists look
2
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
for safe ways to burn hazardous wastes, organic
chemicals, and metals and prevent their release into the
air.
• Support Superfund activities by developing ways to
prevent and contain hazardous releases.
• Manage the Superfund Innovative Technology Evaluation
(SITE) program which is designed to make it easier to
devise and use new, alternative disposal and cleanup
methods.
• Provide on-call assistance for Superfund activities
through RREL’s Superfund Technical Assistance Response
Team (START).
• Conduct research with robots and evaluate protective
clothing and respirators to eliminate exposure of humans
to potential danger.
RREL accomplished most of its work via contractors and
subcontractors. Our report focuses primarily on the work of
RREL’s QA office and its two QA support contractors--Science
Applications International Corporation (SAIC) and S-Cubed
Division (S-Cubed), a subsidiary of Maxwell Corporation.
SAIC and S-Cubed performed basically all QAPP reviews for
RREL, although RREL staff reviewed contractors’ comments and
made final decisions whether to endorse QAPP5. The two
contractors also performed other services. For example, they
conducted technical systems reviews (QA audits) of projects
or experiments to ensure that those responsible for
performing the work followed the EPA-approved QAPPs.
In this report we also discuss issues related to two other
contractors--PRC Environmental Management Incorporated (PRC)
and Radian Corporation--which conducted technical projects
for RREL. PRC was a direct competitor to SAIC for SITE and
START work.
RREL QA office staff responsibilities included:
• Tracking the status of RREL’s projects, to ensure that
QAPPs were prepared, before project execution, for all
projects which involved data measurements.
• Reviewing QAPPs prior to project execution, to ensure
that adequate and sound approaches would be used to
achieve defensible results.
3
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
• Conducting on-site audits at contractors, while projects
were ongoing, to ensure that those who performed the
work followed the procedures previously documented in
the QAPPs.’
• Providing training and written guidance to those who
conducted work to ensure they understood QA requirements
and procedures.
• Coordinating with and overseeing the work of RREL’s QA
support contractors.
• Preparing a QMP, and annual updates, for submission to
QAMS.
• Working with other RREL staff, including technical
project managers (TPMs), to resolve QA problems or
weaknesses and to guide them in executing projects
properly.
RREL QA staff often assisted TPMs, who were responsible for
technical aspects of projects, with QA concerns. RREL
management created the ItTPMII title for its internal use. The
title was not widely known outside of RREL. TPMs often also
functioned as Work Assignment Managers (WAMs).
Organization
RREL was formed in 1988 when the Water Engineering Research
Laboratory and the Hazardous Waste Engineering Research
Laboratory were merged. RREL consisted of an Office of the
Director, Office of Program Operations, and four program
divisions. The four program divisions were: (1) Drinking
Water Research Division; (2) Superfund Technology
Demonstration Division; (3) Waste Minimization, Destruction,
and Disposal Research Division; and, (4) Water and Hazardous
Waste Treatment Research Division. The QA office, under the
Office of the Director, consisted of four full-time staff,
including a QA manager. The QA Manager reported directly to
RREL’s Deputy Director regarding all QA-related matters
relevant to RREL.
Shortly after completion of audit fieldwork, RREL was
consolidated, along with five other organizations, into a new
“megalab” known as the National Risk Management Research
1 0n-site audits almost always involved the review of contractors’
performance, although audits could be conducted on in-house Laboratory projects
or cooperative agreements.
4
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
Laboratory (NRNRL). Consequently, RREL ceased to exist as a
formal organization, and RREL officials became NRMRL
officials. All audit work, however, focused on RREL QA
procedures; for this reason, we refer to RREL in this audit
report rather than to NRMRL. As a result of the
reorganization, the Laboratory’s response to this report
addressed actions the NRMRL Cincinnati (NRMRL-CI) QA staff
will take with respect to our recommendations.
RREL Budget
RREL’s budget for fiscal year 1994 was about $57 million,
with $43 million (75 percent) of that total going for
extramural work. Total FTE positions at RREL for 1994 were
about 230, with RREL’s work in hazardous waste, Superfund,
and drinking water research accounting for 78 percent of the
total.
In addition to the funding required for RREL’s four QA office
positions, RREL spent another $967,000 for QA contract
support during fiscal year 1994. Of this, they spent about
$338,000 for technical systems reviews (audits), $315,000 for
QAPP reviews, $208,000 for Final Report reviews and
miscellaneous support, and $106,000 for project management.
OTHER RELATED MATTERS
As part of our survey, we evaluated factors related
specifically to the inherent risk of RREL’s QA program, as a
comparison to EPA-wide risk in this area. For example, in
EPA’s 1994 Material Weakness Risk Ranking Model, the QAMS
Director rated the Agency’s risk related to environmental
data quality at 105 points. The maximum score possible
(highest level of risk) was 120 points. The QANS Director
rated the following eight risk factors at the highest risk
level possible: (1) human health, (2) environment, (3)
information/data integrity, (4) government resources, (5)
public confidence, (6) EPA-wide impact, (7) duration, and (8)
source identification.
QAMS identified some problems with QA programs at other EPA
labs. The QAMS Director told us that she believed that
RREL’s QA program was superior to other labs’ programs.
However, according to the Director, QANS had not audited RREL
since it became an entity in 1988 because of resource
limitations. She stated that QAMS planned to audit RREL
during the third quarter of fiscal year 1996.
OIG Risk Assessment of RREL
5
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
We believe, based on OIG criteria, the inherent risk of
RREL’s QA program was “moderate”, on a scale of low—moderate-
high. Several attributes inherent to the nature of RREL’s
work carried moderate to high risk: (1) RREL had four
functional divisions and about 230 staff; (2) nearly every
project 2 required the preparation of a QAPP; (3) the RREL QA
office staff was required to monitor each project; (4)
ongoing projects were in various stages of completion,
thereby making them more difficult to monitor; (5) RREL
management, scientists, and engineers relied heavily on
contractors to accomplish RREL’s workload 3 , and (6) QAMS had
not audited RREL.
Although we determined inherent risk of RREL’s program was
“moderate” overall, we believe some aspects of RREL’s QA
program minimized the level of risk: (1) RR.EL QA officials
reported to top-level RREL management; (2) the regulations
and guidelines that applied to QA were easily identifiable,
clear, and not unduly burdensome, and (3) incentives to
circumvent QA requirements were not prevalent and, according
to TPMs and Division Directors, it was reasonable to expect
compliance with QA requirements.
SCOPE AND METHODOLOGY
Our survey focused primarily on the efforts of RREL’s QA
office. To accomplish our work, we:
• Obtained and analyzed EPA and RREL QA requirements and
guidance;
• Reviewed the content of the QA office’s project tracking
database;
• Assessed RREL’s QAPP review process, including the
timeliness of RREL’s (and its support contractors’) QAPP
reviews;
• Reviewed a sample of RREL-endorsed QAPPs for the
sufficiency of information covered;
2 RREL typically begins between 100 and 150 new projects each year.
3 According to the RREL QA Manager, about 60 percent of RREL’s work deals
with SITE projects. Nearly all SITE work is performed by contractors, such as
SAIC and PRC. Such projects, according to RREL’s Deputy Director, are “risky”
because they are politically visible and involve large amounts of Federal funds.
6
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
• Assessed RREL’s QA technical systems review (QA audit)
process and reviewed a sample of its QA audit reports;
• Interviewed RREL QA office staff regarding oversight of
its two QA support contracts and obtained budget data
for those contracts;
• Reviewed contract documentation and other EPA guidance 4
related to RREL’s two QA support contracts and
interviewed contractor officials to determine if
contractors performed any inherently governmental
functions;
• Interviewed RREL Division Directors and TPM5 to obtain
their perspectives as “users” of the QA process;
• Reviewed RREL’s July 1994 QMP;
• Obtained and reviewed records pertaining to training
courses taken by RREL staff; and
• Assessed the inherent risk of RREL’s QA program and
evaluated its internal controls related to the QA
program. We also reviewed ORD FMFIA reports for fiscal
years 1993 and 1994 and interviewed the Director of
QAMS.
We concentrated our work on recently executed RREL projects--
that is, those projects reviewed by the QA office from
October 1993 to January 1995. According to a January 1995
printout from the QA office’s internal database, from the
beginning of fiscal year 1993 to January 1995, RREL managed
and the QA office reviewed about 260 total projects. We
judgmentally selected a sample of 35 projects for review (see
exhibit 1). We selected the 35 projects based on the risk
assigned by RREL officials to the projects, the parties which
conducted the projects, and the type of funds involved. For
example, we focused our efforts on projects that were either
politically or publicly visible, or involved the expenditures
of large amounts of Federal funds. The mix of projects we
selected, which follows, was a good representation of the
work RREL did:
4 We reviewed three key EPA documents related to this issue: (1) EPA Order
1900.2, entitled “contracting at EPA”, (2) ORD Transmittal #92-13, issued
November 1991, and (3) a September 1992 ORD memo regarding the “Use of Non-EPA
Support Personnel for Quality Assurance Activities.”
7
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
Eighteen projects were classified as Category 2 (risky),
13 were Category 3 (somewhat risky), and 4 were Category
4 (least risky). We did not review any Category 1 (most
risky) projects because RREL officials did not conduct
any. According to the QA Manager, it was not the nature
of RREL to perform Category 1 projects.
• Twenty-eight of the 35 projects were performed by
contractors, 4 were cooperative agreements, and 3 were
conducted in-house.
• Twenty-five of the 35 projects were conducted with
Superfund monies. The remaining 10 projects were
executed with other appropriations.
We performed our work from October 21, 1994 to April 13,
1995. We briefed RREL officials on the results of our work
on April 13, 1995. We provided RREL officials with a draft
of this report for discussion purposes on May 30, 1995. We
held an exit conference with the Deputy Director of RREL on
June 29, 1995. In response to his comments, we made
appropriate changes and finalized our report. The Director’s
response is included as appendix 1.
This survey was a short-term study of RREL’s activities. It
was not designed to be a statistical research study or a
detailed audit. Rather, it was an information gathering
study that sought to identify issue areas for management
attention. Thus, it was more limited in scope than an audit
and, as such, did not necessarily encompass all generally
accepted government auditing standards. Alternatively, this
review was conducted in accordance with the provisions of OIG
Manual Chapter 150, Special Reports .
8
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
CHAPTER 2
GOOD MANAGEMENT PRACTICES
RREL’s QA program was well-managed, and met EPA QA
requirements in an innovative manner. RREL management was
committed to achieving QA goals and objectives, and this
commitment was evident in its QA reporting structure, its
philosophy of continuous improvement, and in the level of in-
house resources RREL management dedicated to carrying out QA
functions.
RREL’s QA policies and procedures (internal controls) were
well-documented in its Quality Management Plan, and
implementation of the Plan was evident. RREL’s QA office
also performed QA review functions--such as the review of QA
project plans and the performance of technical systems
reviews--in a timely and thorough manner.
The QA office also maintained good oversight of its two QA
support contracts. RREL had adequate procedures to ensure
that its QA support contractors were not performing
inherently governmental functions. The QA office ensured
that its staff was the final decision maker regarding the
approval of QA project plans and technical systems review
reports. We found no indication that QA support contractors
performed any inherently governmental functions.
RREL’s QA program also included the development and
implementation of innovative approaches to QA. RREL
officials adapted Agency QA procedures to meet specific RREL
needs. RREL QA officials also structured the QA program to
be proactive rather than reactive, thereby resulting in
dollar savings in some cases.
RREL MANAGEMENT’ S
COMMITMENT TO QA GOALS WAS EVIDENT
RREL’s top-level management was committed to achieving QA
goals and objectives. They established a QA reporting
structure which emphasized this commitment. They also
operated under a philosophy of continuous improvement and
expended significant in-house resources to achieve QA goals.
QA Reporting Structure
The Director of RREL was responsible for implementing the QA
program in accordance with the Agency’s QA policies. The
Director delegated direct oversight authority of RREL’s QA
9
Report No. E1 ThB5-23-OOO2-54OOO99
-------
RREL QA ACTIVITIES
program to the Deputy Director. In turn, RREL’s QA manager
was given the authority to direct and manage the QA program.
The QA manager reported directly to RREL’s Deputy Director.
Therefore, the QA manager was organizationally independent of
all RR.EL groups with direct research program responsibility
and was given direct access to RREL’s top-level management.
By nature of this reporting chain, RREL management
established that QA was a high priority.
Philosophy of Continuous Improvement
According to RREL’s QMP, “commitment to quality assurance is
embedded in a continuous process of total quality management
that will be an ever-present part of RREL’s culture.”
According to the QMP, QA is the responsibility of all RREL
personnel.
According to the Deputy Director, all RREL staff are
responsible for helping the organization achieve QA goals.
For example, he stated that, although the QA manager had
direct access to top-level management, he held the QA manager
accountable for maintaining a customer-oriented focus with
those RREL technical staff executing projects. The Deputy
Director stated that, if the QA manager and his staff
continually focused on critiquing without showing the rest of
RREL’s technical staff how to improve upon QA processes and
products, the QA office’s efforts would not be effective.
Resources Dedicated to OA Office
RREL management’s commitment to QA was also exhibited by the
level of intramural resources devoted to its QA office. The
QA office was staffed by a full-time QA manager and three
other full-time employees. RREL officials received approval
in October 1994 to hire three additional full-time QA office
staff members. They hired two of the three individuals in
May 1995, and planned to hire the third by the end of
September 1995. RREL management’s goal is to internalize
most QA activities and minimize the level of extramural
resources used for QA support contracts.
According to the Director of EPA’s QANS, even without the
hiring of three additional QA staff, RREL’s staff of four
full-time employees was the largest in-house QA staff of any
EPA laboratory. During subsequent reviews, we intend to
5 Due to an Agency hiring freeze effective July 11, 1995, RREL officials
indefinitely suspended their plans for hiring the third individual.
10
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
evaluate whether the level of resources spent by RREL for QA
activities is appropriate.
INTERNAL CONTROLS WERE
ADEQUATE AND APPROPRIATE
RREL’s internal controls were detailed, well-documented, and
designed to attain QA goals. RREL, in its 1994 input into
ORD’s FMFIA report, did not report any material weaknesses.
In contrast, ORD cited “environmental data quality” as a
material weakness in its 1993, 1994, and 1995 FMFIA reports
to the President. As of July 1994, only about half of EPA’s
offices that generated and used environmental data had ORD-
approved QMPS, which were required in accordance with Order
5360.1. The EPA Administrator agreed to direct Assistant
Administrators and Regional Administrators to develop and
complete QMPs as a top priority and to name a senior manager
responsible for overseeing the effort.
Policies and Procedures Were Designed
to Ensure That OA Goals Were Attained
RREL’s policies and procedures (internal controls) for
adhering to QA requirements and achieving QA goals were
adequate and appropriate. RREL management’s policies were
laid out in detail in its QMP. Such policies, if followed,
were designed to ensure that QA regulations and requirements
were complied with. In addition, RREL prepared annual
updates to its QMP, in accordance with Order 5360.1, and
submitted them to QAMS for review. The QAMS Director told us
that RREL was about the only EPA lab that regularly prepared
annual updates.
RREL had several controls in place which were designed to:
(1) prevent the unnecessary expenditure of funds and (2)
ensure that QA requirements were addressed on projects, when
applicable. For example, RREL prioritized its scientific
projects, whether executed in-house or by contractors, by a
category system, from 1 to 4 6 This is a good practice that
6 Category 1 projects are the most significant and visible projects
conducted. EPA uses results of Category 1 projects to develop new Agency-wide
regulations. Since 1991, when RREL established the categories, RREL has not
worked on any category 1 projects. RREL managed various Category 2 projects,
many of which were SITE projects. Category 3s and 4s typically are less risky
and are either in-house RREL projects or cooperative agreements with university
professors and graduate students for general research.
1].
Report No. E1J8B5-23-0002-5400099
-------
RREL QA ACTIVITIES
was designed to ensure that staff time and Federal funds were
spent on projects in proportion to the level of risk and
importance of the work.
As another good control, RREL required TPMs to complete a QA
review form 7 for all scientific projects requiring funding
for their execution. The QA manager or QA coordinator
reviewed and signed this form stating whether they agreed
that QA was relevant to a project’s execution and, therefore,
that QA activities, including the development of a QAPP,
needed to be implemented. In addition, the QA office also
tracked RREL research projects when work assignments were
issued, rather than waiting until TPM5 submitted QAPPs for
review, Consequently, QA office staff were able to ?nsure
that all projects for which QA requirements applied had QAPPs
prepared for them. If the QA office staff did not receive a
QAPP within three to four months after issuance of a work
assignment, the staff would notify the appropriate division
director and TPM that one must be prepared. Therefore, it is
not likely that a project could be started, or worked on very
long, without an approved QAPP in place.
QA OFFICE’S REVIEWS
WERE TIMELY AND THOROUGH
RREL’s QA office performed QA review functions--such as the
review of QAPPs and the performance of technical systems
reviews--in a timely and thorough manner. Of the 73 QAPP
submissions we reviewed, RREL met its internal 15-day review
and comment deadline in 57 cases (78 percent). In each of
the 16 remaining cases that RREL did not meet its deadline,
QA support contractors responsible f or providing review
comments to the QA office missed their assigned deadlines.
The content of the QAPP5 and RREL’s QMP we reviewed was
complete. Of the 31 QAPPs we examined that had been endorsed
or conditionally endorsed by the QA office, only one was
missing a required section of information from the QAPP.
RREL conducted 75 technical systems reviews (audits) in 1994.
Based on our review of 13 audited projects, RREL provided
proper feedback on audited projects, as required under its
QMP. We found that written feedback was usually provided to
the auditee the day of the audit. In addition, audit reports
7 Quality Assurance Review for Extramural Projects-Contracts, QA Form QAR-C,
Revision No.1, 1981 and Quality Assurance Certification for Cooperative
Agreements, July 1990.
12
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
were generally completed and provided to TPMs within the 30-
day requirement.
According to the QA office manager, the audit process allows
them to make decisions regarding the quality of results
achieved by the various labs and contractors who manage
projects. For example, RREL determined after one audit that
a contractor was unfit to conduct research projects for them,
and discontinued doing business with that contractor.
QA SUPPORT CONTRACTORS WERE NOT
PERFORMING INHERENTLY GOVERNMENTAL FUNCTIONS
The QA office maintained good oversight of its two QA support
contracts. The QA office ensured that its staff were the
final decision makers regarding the approval of QAPPs and
technical systems review reports. We found no indication
that QA support contractors were performing inherently
governmental functions.
Office of Management and Budget Circular A-76 defines
inherently governmental activities as those “being so
intimately related to the public interest as to mandate
performance only by Federal employees.” EPA Order 1900.2
states, in section 3.a.(4), that if contractors provide
background research or perform analyses,
it must be carefully examined by EPA personnel for
partiality, favoritism, and any contractor conflict
of interest. In all cases, there should be a clear
record that the contractor’s work was reviewed and
that the final decisions were made by Agency
personnel.
EPA Order 1900.2 also states, in section 3.b., that any
support contracted for the in-house evaluation of another
contractor’s performance must be justified. The program
office is to prepare a written justification which describes
the type of work to be performed, why the work cannot be
performed internally, and what will be done to ensure a final
EPA product that is unbiased and represents EPA thinking.
For contracts valued over $25,000, this justification must be
signed by the cognizant Assistant Administrator, Associate
Administrator, or Regional Administrator.
Although some QA functions are considered by EPA to be
inherently governmental, we do not believe that either of
RREL’s QA support cont ractors performed any such activities.
13
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
For example, although RREL relied heavily on its QA support
contractors to review QAPPs and to provide technical review
comments, RREL officials did not simply “rubber stamp” their
approval of those comments. We found documented evidence in
the project files which indicated that QA office staff
closely reviewed support contractors’ comments. Moreover, QA
office staff were knowledgeable when discussing project
details. By closely reviewing contractors’ comments and
recommending other revisions as needed, the QA office acted
as the final decision maker regarding what information QAPPs
needed to address before the QA office would endorse them.
In addition, we found written and approved justifications in
the contract files for having the QA support contracts.
Finally, contractor representatives stated strongly that they
acted only as technical advisors to RREL on QAPP reviews and
technical systems reviews.
RREL’ S OA PROGRAM WAS
INNOVATIVE AND RESULTED IN SAVINGS
RREL’s QA program included innovative approaches to QA. QA
officials adapted Agency QA procedures to meet specific RREL
needs. For example, RREL QA officials modified an existing
QA approach to prioritizing projects by the risk associated
with conducting each project, and published this approach in
a pocket-sized booklet. According to the QA Manager, since
publishing the booklet in 1989, RREL officials had
distributed more than 50,000 copies to various government
organizations, industries, contractors, and universities.
In addition, RREL QA officials improved forms, as needed, to
meet specific RREL needs, and also recently began to track
“customer” satisfaction through the use of staff surveys.
RREL QA officials also developed a new control procedure to
ensure RREL officials prepared QAPPs when required. That is,
QA officials assigned QA identification numbers to projects
as soon as they reviewed procurement packages, rather than
when they received QAPPs. Procurement packages contain
various contract-related documents--including QA review
forms- -and are circulated to appropriate government officials
for their approval as part of the procurement process.
Assigning QA numbers early on in the procurement process
enabled QA officials to track, through their internal
database, projects for which they had not yet received QAPPs.
RREL QA officials adopted a proactive approach to problem-
solving, resulting in savings which the QA Manager quantified
14
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
and documented to illustrate the value of RREL’s QA program.
For example, a 1993 audit of a SITE demonstration project
related to the biogenesis soil washing process resulted in a
cost savings of about $800,000. According to the QA office,
if the audit had not been performed when it was, all data
from the project would have been biased and the entire
project would have had to be repeated.
Another audit in 1992 resulted in savings of about $450,000,
according to the QA office. By identifying sampling concerns
related to a SITE demonstration, corrective actions were able
to be implemented early enough to allow collection of data of
known and adequate quality to meet project needs.
Consequently, the demonstration did not have to be re-run.
On another SITE project in 1992 involving low temperature
thermal treatment, a complete re-run of the project was
avoided, thereby saving about $270,000. The QA office also
documented other cases, without quantifiable dollar savings,
which could have resulted in biased and misleading data had
audits not been conducted.
15
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
[ This page intentionally left blank.]
16
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
CHAPTER 3
OTHER ISSUES
We found two issues which have potential to impact the
management of RREL’s QA program. We found that: (1) two
RREL TPMs who also functioned as WAI 1s had not taken a
required contract-related training course and (2) additional
organizational conflict of interest (OCI) controls were
needed. Efforts should be made to address these issues.
Although we did not identify any negative impact or effect
from these conditions, there are potential effects that could
impede the program’s progress.
WAMs LACKED REQUIRED
CONTRACT - RELATED TRAINING
A total of 22 TPMs managed the 35 projects we reviewed. Some
of the TPM5 also acted as WAMs. Two (9 percent) of the TPMs
who acted as WAMs lacked contract-related training required
for WAMs. 8 The two WAMs had not taken a contract
recertification (“refresher”) course. In accordance with
EPA-wide requirements in Order 1900.2 and EPA’s Contracts
Management Manual, all WAI4s are required to take an 8-hour
recertification (“refresher”) course every three years. This
training was contract-related, not QA-related. Consequently,
although the issue could indirectly impact the RREL QA
program, it pertains to RREL as a whole.
RREL’s Extramural Management Specialist said she believed
contract-related training had been emphasized at RREL.
However, she speculated that reasons WAMs might not have
taken required contract-related courses were: (1) the Office
of Acquisition Management (OAM) did not offer the courses
often enough, in some cases because of a lack of travel funds
to come to Cincinnati or (2) RREL staff were unclear on what
courses were required. For example, she stated that some
staff might not realize they need to take the “refresher”
course every three years. According to RREL’s training
coordinator, it is the responsibility of each employee to
track which courses they must take and to take them. The
training coordinator said she tracks what courses people have
taken and the date when the training was completed in an
internal database.
8 The 2 WAMs who lacked training managed 2 (6 percent) of the 35 projects.
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
The extramural management specialist also stated that, as a
result of the September 1994 revision to Chapter 7 of the
Contracts Management Manual, procedures are in place to
ensure that employees do not act as a WAN or a Project
Officer (P0) unless they have taken relevant courses. EPA
Form 1900-65, used to nominate someone to be a WAN, requires
that completion dates be listed for two contract-related
courses WAMs must take. 9 If the nominee has not completed
the required training courses, a memorandum justifying the
nomination must be prepared by the nominating official,
forwarded to the contracting officer for approval, then
forwarded to the Director of Policy, Training, and Oversight
Division within OAM for approval. According to the Contracts
Management Manual, “A deferment of training will only be
granted under extenuating circumstances and for a limited
period until training can be completed.”
Conclusion and Recommendations
Contract-related training is vital to help ensure that
projects are carried out appropriately and cost-effectively.
We recommend that management (1) review the training records
of all RREL staff acting as WAMs and identify any others who
are lacking training, (2) ensure that those individuals
identified take the required contract-related courses, and
(3) place a greater emphasis on ensuring that its staff is
sufficiently trained.
Laboratory Comments and Actions
The NRMRL Director agreed with the recommendations. He
stated that management will, by November 6, 1995, review
training records of all NRMRL Cincinnati (NRNRL-CI) staff
acting as WAIls and identify those lacking required contract-
related training courses. The Director also stated that new
0AM procedures, found in the September 27, 1994 version of
the Contracts Management Manual, help to ensure that WAMs who
lack training take required courses. According to the
Director, RREL and ORD officials placed a significant
emphasis on training.
OIG Evaluation
The NRNRL-CI staff’s actions, when completed, will address
the recommendations in this section.
9 me two required courses are (1) Contract Administration and (2)
Recertification.
18
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
ADDITIONAL ORGANIZATIONAL
CONFLICT OF INTEREST CONTROLS NEEDED
Although the QA office’s support contractors did not perform
any inherently governmental functions, we believe another
contract-related issue exists that should be addressed--
namely, potential contractor organizational conflicts of
interest (OCIs) during QAPP reviews. In one case, an OCI
occurred because a QA support contractor was the sole
reviewer of a primary competitor’s QAPP. We also identified
two OCI concerns involving subcontractual affiliations
between RREL QA support contractors and RREL contractors that
prepared QAPPs.
Federal Acquisition Regulations (FAR) Subpart 9.501 defines
“organizational conflict of interest” as follows:
Organizational conflict of interest means that
because of other activities or relationships with
other persons, a person is unable or potentially
unable to render impartial assistance or advice to
the Government, or the person’s objectivity in
performing the contract work is or might be
otherwise impaired, or a person has an unfair
competitive advantage.
FAR Subpart 9.505-3 specifically addresses OCIs involving
advisory and assistance services, which would apply to the
review of QAPPs:
Advisory and assistance services shall not
generally be awarded to a contractor that would
evaluate, or advise the Government concerning, its
own products or activities, or those of a
competitor, without proper safeguards to ensure
objectivity and protect the Government’s interests.
EPA Acquisition Regulations Subpart 1509.5 states that it is
Agency policy to avoid, neutralize, or mitigate OCIs.
RREL officials were aware of the potential for OCIs and had
developed some management controls to mitigate or avoid OCIs.
RREL QA officials stated that QAPP review procedures
mitigated potential OCIs. RREL QA officials checked QA
support contractor QAPP reviews for accuracy and consistency.
RREL QA officials also stated that they required QA support
contractors to review QAPPs using standards found in Agency
guidance on the preparation of QAPPs. To avoid OCIs, RREL
policy prohibited QA support contractors from reviewing QAPPs
19
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
prepared by their own companies or by direct competitors.
For example, RREL policy prohibited SAIC--an RREL QA support
contractor--from being the sole reviewer of PRC SITE program
QAPPs because PRC was a direct SAIC competitor for RREL SITE
work.
A QA Support Contractor
Reviewed a Competitor’s OAPP
RREL officials generally followed their policy that
prohibited QA support contractors from reviewing their own QA
products or those of a competitor. However, we identified
one case in which a QA contractor was the sole reviewer of a
QAPP prepared by a direct competitor. SAIC obtained an RREL
Superfund Technology Assistance Response Team (START) program
contract while conducting a review of a PRC START program
QAPP for RREL.’° RREL endorsed this PRC QAPP without
recognizing the OCI. RREL QA officials said they were not
aware at the time of the QAPP review that SAIC had become an
RREL START program contractor.
OCI Concerns Involving
Subcontractual Affiliations
We also identified two OCI concerns involving subcontractual
affiliations between RREL QA support contractors and other
RREL contractors. First, SAIC, on occasion, was the sole
reviewer of QAPPs prepared by a company--Radian Corporation--
with which SAIC subcontracted.” Radian also was a
subcontractor with PRC--an SAIC competitor.’ 2 As such, SAIC
(as a QA support contractor) reviewed and judged the adequacy
of work products prepared by a contractor that both SAIC (as
a non-OA support contractor) and its primary competitor used
as a subcontractor on previous RREL projects.
10 QA identification number 940, according to the QA office’s database. The
QA office endorsed this QAPP on November 15, 1994. The START contract was
awarded to SAIC on November 1, 1994; SAIC delivered the QAPP review to the QA
office on November 2, 1994.
“According to RREL’s database, Radian was a subcontractor for SAIC on RREL
projects 909, 865, 823, and 735. SAIC was the only reviewer of Radian’s QAPP5
for the following RREL projects: 848, 919, and 801.
‘ 2 According to RREL’s database, Radian was a subcontractor for PRC on RREL
projects 903, 867, 829, and 728.
20
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
Secondly, SAIC acquired the services of S-Cubed, a
subcontractor on some SITE projects. 13 Although RREL
appropriately used another QA support contractor (PEI/IT
Corporation) to review those QAPPs, we believe the potential
for OCIs remains between SAIC and S-Cubed. S-Cubed, like
SAIC, is a QA support contractor for RREL. S-Cubed must
review QAPPs for all SAIC-managed projects because SAIC
cannot review its own company’s products. According to
RREL’s database, S-Cubed was the sole reviewer of QAPPs for
at least twelve SAIC projects during 1994. Because S-Cubed
sometimes is a subcontractor to SAIC, S-Cubed QA review
officials might hesitate to issue unfavorable QA reviews of
SAIC QAPPs.
According to the FAR definition cited above, OCIs occur when
objectivity in performing contract work might be impaired.
We believe this could occur because of subcontractual
affiliations. However, the FAR advisory and assistance-
specific OCI reference, as well as ORD and RREL policies, do
not expressly address subcontractual relationships.
According to FAR 9.505, each contract should be examined on
the basis of the nature of the proposed contract, and any
decision to determine whether a significant potential OCI
exists should be based on “common sense, good judgement, and
sound discretion.” According to RREL officials, given the
nature of RREL’s work and the number of contractors qualified
to perform it, subcontractual affiliations between RREL QA
support contractors and RREL contractors that prepared QAPPs
were sometimes unavoidable.
To mitigate the perception that contractors might be
performing inherently governmental functions and to avoid
OCIs, RREL management obtained authorization for more in-
house QA positions to replace contractor positions.
According to RREL’s Deputy Director, RREL hired additional
staff members in 1995 for its QA office. RREL management
plans to eliminate most of its QA contract support by
acquiring additional FTE5. RREL estimated a total savings of
$450,000 in fiscal year 1996 due to the conversion of three
new positions.
‘ 3 For the following RREL projects, S-Cubed was a subcontractor for SAIC:
797, 684, and 619.
21
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
Conclusion and Recommendations
Although we did not identify any negative effects from these
OCI-related conditions, we believe such situations could
impede RREL’s QA program by impairing QA support contractors’
objectivity when reviewing QAPP5. Given this, we recommend
RREL QA officials establish control procedures to identify,
and either avoid, neutralize, or mitigate, potential OCIs
between QA support contractors and other Laboratory
contractors. We also recommend that, if possible, RREL QA
officials conduct QA work in-house or use QA support
contractors that perform no other work for the Laboratory in
these instances.
Laboratory Comments and Actions
The NRNRL Director agreed with the recommendations. He
stated that subcontractual affiliations between QA support
contractors and other Laboratory contractors represented “an
additional potential conflict of interest that was not
specifically addressed previously.” In response to the
recommendations, the NRMRL Director stated that the NRMRL-CI
QA staff has begun to implement the following procedures:
(1) POs for QA support contracts will distribute to the
NRMRL-CI QA staff lists of companies with which QA
support contractors do NRNRL-CI business. The Director
stated QA support contractors will update these lists
quarterly, beginning October 1. The Director also
stated that QA support contractors will notify the
appropriate EPA P0 when they are awarded non-QA NRMRL
contracts.
(2) The NRMRL-CI QA staff will review the lists described
above when QA support is requested for a contracted
project. If the project contractor is on a QA support
contractor’s list, the NRMRL-CI QA staff will place a
“C” on the project file to alert NRMRL-CI QA staff that
a potential OCI exists. When an in-house QA staff
member obtains the file, he or she is immediately
alerted that a potential OCI exists with one or both of
the QA support contractors.
(3) If possible, the NRMRL-CI QA staff will provide QA
support in-house or task a QA support contractor from
another EPA laboratory in potential OCI situations. The
Director anticipated that in-house staff would perform
nearly all QAPP reviews by fiscal year 1998.
22
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
OIG Evaluation
The NRMRL-CI staff’s actions, when completed, will address
the recommendations in this section.
23
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
[ This page intentionally left blank.1
24
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
EXHIB
Page
IT
1
1
of 2
RREL PROJECTS REVIEWED BY OIG
#
QA ID #
Category
Fundg’ 4
Project Executed by
1
976
2
NSF
SAIC
2
955
4
NSF
Southern Methodist
Univ.
3
953
3
SF
IT Corp.
4
950
2
SF
PRC/Unstitut
Fresenius
5
945
3
NSF
Jefferson Parish-LA
6
944
4
NSF
E&E Res. Ctr./Univ.
of ND
7
940
4
SF
PRC
8
934
3
NSF
In-house/RREL
9
931
2
SF
EQMI
10
925
3
SF
IT Corp.
11
924
2
SF
FWEI
12
923
3
NSF
CCP&CT Univ. of TN
13
917
2
SF
SAIC
14
914
3
SF
SAIC/Coast to Coast
15
913
2
SF
PRC/Gen’l Testing
Corp.
16
912
2
SF
PRC/Versar
17
910
2
SF
PRC/Versar
18
909
2
SF
SAIC/Radian
19
903
2
SF
PRC/Radian
20
890
3
SF
R.F. Weston Inc.
‘ 4 NSF refers to Non-Superfund monies; SF refers to Superfund monies.
25
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
EXHIBIT 1
Page 2 of 2
RREL PROJECTS REVIEWED BY OIG (cont.)
#
QA ID
#
Category
Funds
Project Executed
by
21
889
4
NSF
In-house/RREL
22
888
3
SF
General Atomics
23
886
2
SF
PRC/GTC
24
882
3
SF
SAIC
25
880
2
SF
SAIC/EA
26
875
2
SF
Acurex (AR)
27
866
3
NSF
EQMI
28
865
2
SF
SAIC/Radian
29
848
2
NSF
Radian
30
829
2
SF
PRC/IT-Radian
31
827
3
SF
PRC
32
819
3
NSF
In-house/RREL
33
810
2
SF
SAIC/Lockheed
34
797
2
SF
SAIC/S-Cubed
35
790
3
SF
MSE, Inc.
26
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
APPENDIX 1
Page 1 of 3
UNITED STATES ENVIRONMENTAL PROTECTiON AGENCY
NATIONAL RIS ( MANAGEMENT RESEARCH LA8OR TORV
CINCINNATI, OH 45268
OFFIC! OF
R!MAACM AHD O€VELOPMENT
August 22, 1995
MEMORkN
BUEJECT: Respons. to Office of Inspector General Report of
Review: Survey of Risk Reduction Engineering
laboratory’s Quality Assurance Activities
ElJ8flS —23—OOO2-,nne,r xx
FROM:
TO: Anthony C. Carrollo
Divisional Inspector General for Audits
Northern Division
Th• Risk Reduction Engineering Laboratory (RREL) would like
to thank the Off ice of Inspector G.n.ral for its thorough and
helpful review of PRIL’s Quality Assurance (QA) activities. The
following response documents the actions the National Risk
Management Research Laboratory - Cincinnati (into which RREL was
absorbed on May 1, 1995) will take with respect to the two minor
issues idntifi.d during the Office of Inspector General (OIG)
review.
Issue 1: Tvo RREL Technical Prolect Managers (TPMs) who lj
functioned as Work Assiensent Manaaers (WAMs had not tak*j j
reauired contract—related training course .
National Risk Management Research Laboratory — Cincinnati
(NRXRL—Ci) management will review the training records, by
November 6. 1995, of gjJ NRXRL-Ci staff acting as WAlls and
identify those who are lacking training. NP,MRL-Ci believes
that the new procedures established by Office of Acquisition
Management (OAK) in the 9/27/94 version of Chapter 7 of the
Contracts Management Manual will ensure that individuals who
are lacking training take th. required courses. OAII has
established procedures in this chapter which require the
Project Off ic.r (P0) or WAll to indicate the dates of each
training course and the recertification course on EPA Form
1900—65A (Rev. 7-94). OAK policy indicates that Contracting
Officers will not approve the appointment of POs and WAlls
who lack th. necessary training.
It should be noted that NBXRL-Ci and the Office of Research
and Development (ORD) have placed a significant emphasis on
27
Report No. E1JBB5-23-0002 -5400099
-------
RREL QA ACTIVITIES
APPENDIX 1
Page 2 of 3
In order to further clarify NpJ(RL-Ci policy with respect to
aitigatinq the potential organizational conflict of interest
id.ntifi.d by the 010, the NRMRL-Ci staff has also b.gun to
inpls .nt the following procedur es
The P0 for the QA support contracts r.qu.st.d that the
support contractors list those cospanies with which
they do NRXRL-Ci business and to send thee. lists to
her. The P0 has received these Lists and will
distribute thes. lists to all seabers of the WRMRL—Ci
Q& staff. Thes . lists will be reviewed and updated
quarterly by the QA support contractors (October 1,
January 1, April 1, and July 1). The QA support
contractors will also notify the EPA Project Officer
whenever they are awarded a non-QA NRMRL-Ci contract.
• When QA support is requested by IIRMEL-Ci staff on a
contracted project, the lists d.scrib.d above are
reviewed by an in-house NRNRI -Ci QA staff sesber.
Should the project contractor appear on either QA
support contractor list, the lettr C is placed on
the project file. When an in-house QA staff aesber
verifies the contractor deliverable of a project with
the C. designation, that QA staff asaber obtains the
file and is lasediately alerted that a potential
organizational conflict of interest exists with one or
both of the QA support contractors. IlR? L-Ci QA staff
a ’ ers verify all contractor deliverables in order to
ensure accuracy; this includes checking for bias du. to
potential organizational conflicts of interest.
• If possible, the WR)IRL—Ci QA staff will provide the QA
support in-house or task a non-N G L-Ci QA support
contractor as opposed to tasking a NRMRL-Ci QA support
contractor. It is anticipated that QAPP reviews and
Technical Sy.tsas Reviews will be perforsed prisarily
by in—house staff by PT 1998 • Weabers of the )IRI(RL-Ci
QA staff are currently actively participating in both
QAPP reviews and Technical Systeas Reviews.
NRI L aanageaent believes that the actions described above
satisfactorily address th. two ainor issues identified during the
010 review. Again, thank you for your in depth review of the
PREL QA prograa.
cc: Wancy W.ntvorth
candac. Castillo
28
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
APPENDIX 1
Page 3 of 3
txaininq In addition to the WAX r.guir.d training course.,
ORD has instituted a 3-day contraCt sartagea.nt training
course for all WAXs and POs. This course has been presented
OlD—via, over the past 18 sonths. It is .stisatsd that all
NRXRL—Ci and OlD P0. and WAMs will have compl.t.d the course
by the snd of fiscal year 1995.
Issue 2: Additional er anizationa1 conflict of interest (OCfl
controls wars needed .
The OIG report identifies that a potential organizational
conflict of interest exists between a QA support contractor
and a coapany with which it has worked with in the past.
For xaapl., Science Applications International Corporation
(SAIc), a QA support contractor, subcontracted with Radian
Corporation en an RRZL project for the Sup.rfund Innovativ.
Technology Ivaluation (SITt) d.sonstretion prograa. In this
instance, S-Cubed was used to review th. quality assuranos
project plan (QAPP). The 010 sees a potential for an
organizational conflict of interest for SAIC to provide O k
support to IIRXP.L-Ci on a project which involves Radian, even
if Radian and SAIC are not in a contractual relationship on
that project.
)IBXRL—Ci views thi, particular situation as an additional
pot.ntial organizational conflict of interest that was not
specifically addressed previously within the aanag.s.nt
controls for WR) L-Ci QA support contractors. NRMRL—Ci
saintains that while potential organizational conflicts of
interest say be ispossible to avoid, they can be sitigated
by appropriat, contract panagesent controls. As stated in
the OIG report (p.13), “Tb. QA office saintained good
oversight of its two QA support contracts. The QA office
ensured that its staff were the final decision saksrs
regarding the approval of QAPPS and technical systess review
reports.’ The 14RXRL-Ci QA staff believes that this
oversight attigatss potential organizational conflict of
interest situation..
The following procedures are already in place and will
ensure that QA support contractors saintain their
objectivity:
All QAPPs are reviewed against the standards provided
in ‘Preparation Aids for the Developssnt of QA Project
Plans,” (EPA/600/8—91/003 through 006) February 1991.
All reviews p.rtors.d by QA support contractor. are
verified by in-houss personnel for accuracy and
consistency.
29
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
[ This page intentionally left blank.)
30
Report No. E133B5-23-0002-5400099
-------
RREL QA ACTIVITIES
APPENDIX 2
Page 1 of 1
ABBREVIATIONS
CFR Code of Federal Regulations
OCI Organizational Conflict of Interest
EPA Environmental Protection Agency
FAR Federal Acquisition Regulations
FMFIA Federal Managers’ Financial Integrity Act
OAIvl Office of Acquisition Management
OIG Office of Inspector General
ORD Office of Research and Development
PRC PRC Environmental Management Incorporated
QA Quality Assurance
QANS Quality Assurance Management Staff
QAPP Quality Assurance Project Plan
QMP Quality Management Plan
RREL Risk Reduction Engineering Laboratory
SAIC Science Applications International Corporation
SITE Superfund Innovative Technology Evaluation
START Superfund Technology Assistance Response Team
TPM Technical Project Manager
WAN Work Assignment Manager
31
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
[ This page intentionally left blank.]
32
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
APPENDIX 3
Page 1 of 2
DISTRIBUTION
Inspector General (2410)
Assistant Administrator for Research and Development (8101)
Director, Office of Research Program Management (8102)
Director, National Center for Environmental Assessment (8601)
Acting Director, National Center for Environmental Research
and Quality Assurance (8701)
Director, National Exposure Research Laboratory (MD75)
Director, National Health and Environmental Effects Research
Laboratory (MD51)
Director, Office of Resource Management and Administration
(8102)
Director, Quality Assurance Management Staff (8201)
Director, Office of Acquisition Management (3801F)
Regional Administrator, Region 1 (RAA)
Regional Administrator, Region 2
Regional Administrator, Region 3 (3RAOO)
Regional Administrator, Region 4 (4RA)
Regional Administrator, Region S (R-19J)
Regional Administrator, Region 6 (6RA)
Regional Administrator, Region 7 (RGAD)
Regional Administrator, Region 8
Regional Administrator, Region 9 (ORA)
Regional Administrator, Region 10 (SO-141)
Audit Liaison, Office -of Acquisition Management (3801F)
33
Report No. E1JBB5-23-0002-5400099
-------
RREL QA ACTIVITIES
Audit Liaison, Office of Research and Development (8101)
Agency Followup Official (3101)
Attention: Assistant Administrator for
Administration and Resources Management
Agency Followup Coordinator
Associate Administrator for Communications and
Public Affairs (1701)
Associate Administrator for Congressional and
Legislative Affairs (1302)
Headquarters Library (3404)
34
Report No. ELJBB5-23-0002-5400099
------- |