EPA/600/A-95/099
QUALITY ASSURANCE
ORGANIZATIONAL - CATALYTIC - TECHNICAL
by
Guy F. Simes
Quality Assurance Manager
National Risk Management Research Laboratory
Cincinnati, OH 45268
National Risk Management Research Laboratory
Office of Research and Development
U.S. Environmental Protection Agency
Cincinnati, OH 45268
-------
According to recent national studies over two-thirds of all
participants in quality assurance (QA) and quality control (QC)
systems are disappointed in the results. Quality is clearly a
good thing, but quality systems are not when 2 out of 3 lead to
disappointment. What's wrong? Is it possible that dynamic
quality systems (Figure 1) incorporate special additives
(catalysts) that normal quality systems (Figure 2) do not?
In a normal quality system for the monitoring and the remediation
of solid wastes, two distinct levels of QA requirements are
addressed: the organizational (or institutional) level and the
technical (or project) level. Dynamic quality systems, on the
other hand, not only include the normal components but also
integrate catalytic components. These catalytic components are
critical to the foundation supporting the Quality System Pyramid
(QSP); however, they are often overlooked in the design stage of
many quality systems. While this chapter does consider the
characteristics of a normal quality system, an intentional focus
is levied at the seldom explored dimension — catalysts.
ORGANIZATION (OR INSTITUTIONAL) QA
The essential elements of QA management systems have been
discussed widely in the literature. For instance, the 9000-
series documents of the International Organization for
Standardization (ISO) list twenty components of QA management
systems, with emphasis on production and engineering
organizations1. "Guide 25" from the same organization adapts
these guidelines to routine testing laboratories2. More
recently, the US Environmental Protection Agency has developed
general QA management requirements intended for organizations
generating environmental data.3 Other agencies have produced
similar guidelines for their areas of concern4, and private
authors have also presented their views on the essential
elements of general QA management5.
These standards and guidelines, while differing in organization
and emphasis, specify similar elements. In all cases, an
organization's general QA practices must be documented in a QA
management plan that at a minimum describes the organization's
policy and goals; the organizational structure of the QA effort
and its relationship to the larger institute; the authorities
and responsibilities of all parties, including both production
and QA personnel; and the general activities and tools of the QA
group. Depending on the nature of the organization, these
documents also specify that the QA management plan address
document control, flow-through provisions for subcontractors and
vendors, product traceability, calibration requirements for
instrumentation, training requirements, etc. Thus, the QA
management plan provides the basic infrastructure that enables
consistent quality procedures to be applied to similar projects.
1
-------
Figure 1. Dynamic Quality System Pyramid (QSP)
Organizational Technical
(Institutional) (Project)
QA QA/QC
Figure 2. Normal Quality System Triangle
Organizational
Technical
(Institutional) (Project)
QA QA/QC
-------
Nevertheless, it is the experience of this author that a quality
system containing all the expected elements is no guarantee of
success. Much like an Olympic athlete possesses all the same
body parts as the average person, so the quality systems of both
mediocre and superior organizations possess the same QA elements.
For both the athlete and the organization, it is how well these
elements operate together that distinguishes the champion from
the ordinary entity. That is to say, it is not only the
organizational elements, but also the sum of the attitudes and
relationships — the culture of the organization that lead to
success or failure.
CATALYTIC QA
The following text describes those catalytic aspects of QA
management that are essential for an effective QA system. Much
like a catalyst lowers the boundaries between reagents, the
following "catalysts" make all of the organizational elements fit
together to yield a product that is greater than the individual
parts.
Communication. Communication is the glue that holds any
organization together, and its importance to the QA system cannot
be overemphasized. Like actual glue, communication must be
applied consistently, must be accepted by all members, and must
be allowed time to take hold.
What should the QA staff do to foster effective communication?
First, the QA staff needs to take the initiative in identifying
both internal and external customers of QA programs. (Here
external customers are the end users of a company's product, and
internal customers are other groups within the institution who
use QA services.) The QA staff must regularly contact these
persons to establish effective communication. The first meeting
should demonstrate the QA staff's sincere and serious commitment
to a dynamic quality system. It is a confidence builder, plain
and simple! Second. because the customer will likely spend
several meetings venting past dissatisfaction or discussing both
constraints and problems that affect their group, the QA staff
needs to be good listeners. Listening is the least practiced
form of communication. Effective listening will help the QA
staff adjust their speaking and writing to get the QA message
across. Active, attentive listening also communicates one's
desire to understand and to engage in a mutual process. Third.
the QA staff must be prepared to explain what QA can contribute
and what QA requirements need to be satisfied. However, the QA
staff should not plan to present everything at once. A more
effective approach is to provide information when needed, when
the user is likely to be most receptive. Fourth, the QA staff
must be persistent. Developing effective working relationships
requires time even when all parties are highly motivated. A
3
-------
major goal of the QA staff in this regard is to develop shared
goals and effective mechanisms for achieving these goals.
Independence. The QA staff must report directly to top-level
management to make independent judgments without concern for
retribution and without the pressures of day-to-day production.
This arrangement fosters independent thinking by the QA staff,
for the benefit of all parties.
Management Support. If the relationship with management is
strained or not well established, a great deal of resistance to
QA may result. For this reason, the QA manager should report
regularly to management on QA activities, plans, and
accomplishments.
The support of middle management is perhaps the most difficult to
win, but the most important condition necessary for an effective
QA program. Not only do middle managers strongly influence what
resources will be directed towards QA efforts, they also set the
attitudes of much of the technical staff. Management must be
approached on a basis of mutual trust and respect at the starting
point. Comments by the QA staff regarding management or
organizational performance must be non-threatening, lest critical
information be obscured by a defensiveness that leads to attacks
on the QA program. In the long run, relationships with project
managers will become fruitful only when they realize that QA
contributes to the quality of their projects and helps minimize
rework/costs. The relationship with middle management should be
a creative tension—which is to say, productive even if
disagreements occasionally occur.
Employee Involvement. Ideas for improvements can arise from any
level within an organization. All employees should understand
how their work product is used by others, and should be
encouraged to contribute their ideas for product improvement.
Incentive programs can assist in this area.
Full-Time Commitment. To benefit any organization, a QA program
must be an empowered, intrinsic part of that organization, not an
appendage. In establishing a QA program, the first step is
committing full-time resources to QA, with no strings attached
and no other assigned duties. Total commitment is paramount in
reaching ultimate performance.
Substantial Contributions. QA activities must make a difference.
Otherwise, QA will be viewed as an irrelevant nuisance to be
ignored while getting on with the "real work".
Customer Awareness. The QA staff must be aware of the
motivations and pressures on management and technical staff (the
internal customer) in presenting its case. Resistance can be
expected if QA procedures restrict the availability of
4
-------
traditional resources, add constraints, or require an excessive
amount of documentation or labor. Sometimes changes recommended
by QA simply meet institutional inertia, or challenge the egos of
those who have labored long to establish the current procedures.
Technical staff may also be concerned about additional labor
requirements and delays, or may be facing imminent deadlines. If
the project manager is packing for a sampling trip on Tuesday,
then a lengthy QA review should not be planned for Monday!
Legitimate issues must be addressed in a forthright manner;
after all, QA does require additional documentation, and
sometimes QA reviews do unavoidably occur immediately before a
sampling episode. The QA staff must strive to eliminate
unnecessary cost and should avoid inopportune meetings, whenever
possible. Discussions may need to be restricted to the most
pressing issues with remaining topics postponed when deadlines
are imminent. It is also helpful if the QA staff is able to
provide convincing testimony regarding the long-term benefits of
QA.
Since the QA staff can have an impact on the professional staff's
decision to make a change, it is critical that credit and
ownership go to the professional staff and not to the QA staff.
Peak performance within an organization is the natural outgrowth
of a partnership between management, the technical staff, and the
QA staff. From persuasive presentations to painful prodding, the
QA staff must help management and the technical staff achieve
their successes.
Lessons Learned. The QA staff should be the corporate memory
regarding lessons-learned that might be helpful to future
projects. Did some concerns that seemed critical during the
planning stage turn out to be trivial? Were other important
concerns overlooked? Were innovative procedures developed that
might be useful for future projects? The QA staff should not only
keep a log of such case studies as a benefit to future projects
but also ensure dissemination of the information.
Celebrating Success. How did QA benefit each project? Were some
projects saved from failure by early intervention? Were some
projects lost because QA advice was ignored? Were cost savings
realized by a better focussing of effort? Success stories that
embody the value of QA need to be shared with everyone,
especially management.
Measuring Quality. Quality should be measured over time to
objectively assess quality improvement. For instance, a
laboratory might track the percentage of reports delivered within
schedule, or a research group might record the number of times
their work is cited in the literature. These examples suggest,
such measurements must be consumer-oriented and must present data
that allows the organization to take effective action.
5
-------
Back-Up Support. A QA staff cannot possibly have all the
technical expertise to cover every problem that may arise. It
may be necessary to enhance the quality system by judiciously
employing external QA support. This arrangement permits access
to a wide range of practical experience and resources that are
often unavailable within the organization.
Individual Recognition. Undoubtedly the most important factor in
achieving a quality product is the ability and dedication of the
technical staff and management. Managers are acutely aware of
this and spend great effort in recruiting and retaining talented
scientists and engineers. To retain productive staff, the best
performers should be awarded with improved salaries and increased
professional challenges and opportunities. The QA staff can
assist upper management in identifying the best performers by
informing them of any outstanding achievements.
Education and Training. The essence of an effective quality
system is learning, not coercing or controlling. Learning takes
time? it requires real problems to be solved; and it involves
trial and error, experimentation, and tolerance for mistakes.
Training is an important component for any quality system.
Training must be general and specific; appropriate and relevant;
and timely. Quality training must target both specific audiences
and specific skills.
Initial employee training should expose employees to QA concepts
and philosophy. It should focus on general QA principles to make
everyone cognizant of the "who, what, where, when, and why" of
the QA process.
Awareness training, however, is of little benefit if specific
skill training is not provided in a timely manner. Skill
building must answer the "how to" question by introducing the
customer to the tools and mechanisms for pursuing quality.
Finally, training benefits are best recognized when there are
opportunities to apply the new knowledge. Consequently, "just-
in-time" training — giving people the skills they need
immediately before a project — is critically important.
Unfortunately, "just-in-time" training requires a one-on-one
training approach and is often neglected because of the short
term drain on resources. However, those organizations investing
in this approach very quickly realize substantial returns.
Continuous Improvement through "Rapid Inching". All quality
systems must constantly evolve to remain relevant to the
challenges of tomorrow. Take fast but small steps (rapid
inching). Tweak your program and continually seek out small
increments of optimization. Solicit constructive criticism and
be prepared to change.
6
-------
Vision. Envision the QA future in rich detail and then turn
vision into action by practicing the nine E's:
1. Envision the challenge.
2. Entice others to become interested in the challenge.
3. Enable participants through education and understanding.
4. Engage participants to create alliances.
5. Embrace the cause of the team.
6. Empower the team to action.
7. Employ the team in action.
8. Enjoy the rewards.
9. Envision new challenges.
By envisioning the future, one embraces the belief that the
future can be influenced. That belief helps create the fact.
TECHNICAL (PROJECT) QA
The previous section discusses the attitudes and relationships,
the QA catalysts, essential to an effective QA system. How are
these general concepts applied to a specific project? Given the
task of designing and verifying a cleanup action, or the task of
assessing the effectiveness of a new test kit, or that of
establishing the efficacy of a new vaccine; how does the QA staff
apply the aforementioned concepts?
To illustrate some concepts applicable to projects in general,
consider first a specific example, namely, an evaluation of a
nearly mature, pilot-scale environmental control technology at a
specific site. Figure 3 (Typical Flow of Project Activities)
represents the general flow of activities for such a project from
initial planning to release of the final report. Here,
activities led by the QA group are shadowed to distinguish them
from other project activities.
Project activities can be summarized as follows:
a. Complete necessary preliminary studies. For a project of
this maturity, information from treatability studies and
site characterization activities may be available to aid in
experimental design. Further, any questionable measurement
methods will be "debugged" before proceeding with the test.
The extent of prior knowledge may vary significantly for
other projects.
b. Agree to project objectives. Project objectives should be
stated in the most quantitative form possible to aid in the
subsequent design. An effective statement of project
objectives must reflect the goals of all principal
participants to the project, must be practical, and must
define the scope of the investigation. Arriving at a clear
statement of project objectives is perhaps more difficult
but more important as the complexity of the project or the
7
-------
Preliminary Studies
- Site Characterization
- Treatability Studies
- Method Verification
Agree to Project Objectives
- Expected Range of Performance
- ID Positive and Negative Outcomes
»» ««
Prepare Written
Project Plan
Revise Project Plan
A* needed
£
I
¦ Not Approved
REVIEW PROJECT PLAN I ^ *
>
Approved
Audit Sampling Activities
and Recommend Correctivel
Action, ac needed
l
Take Corrective Action
Implement Project Plan
- Plant Operation
• Sample Collection
- Sample Analysis
J
Audit Laboratory Activitlas
and Recommend Correctiv
Action, a* needed
V,
I
J
Take Corrective Action
Evaluate Data
vs Project Objectives
i i Yes
Data Inconclusive?
Collect \
N. More Data /
No i
Data Conclusive
Make Decisions with
Less Confidence
Prepare Final Report
Respond to Review; Modify
Report, if needed
REVIEW FINAL REPORT
V
>
Not Approved
Approved
Release Report for End Use
Figure 3. Typical Flow of Project Activities
8
-------
number of principal participants increases. Less explicit
objectives may be needed for more exploratory projects.
c. Prepare a written Project Plan. Preparation of a written
document relating project objectives to the individual
measurements is central to any complex test. This document
is important for the data producers in that it serves as the
project "Standard Operating Procedure" for those groups
involved in the sample collection, preservation, transport,
custody, storage, preparation, and analysis, quality control
procedures (including corrective action), and data
reduction. This step is common to essentially all projects
although the complexity of the document varies with the
nature of the project and may be handled on a project
category basis*.
d. Review the written Project Plan. The project team, should
be supplemented by technical experts not involved in the
plan preparation who should review this document and
recommend changes, if necessary.
e. Implement the Project Plan. At this stage the test is
actually performed and data are generated. As might be
expected, this is the period of most intense activity.
Ideally all measurements follow the plan, but in practice
unanticipated occurrences often lead to adjustments in
approach, especially for the more exploratory projects.
f. Audit the sampling and laboratory activities. The QA
involvement at this stage is to perform on-site audits of
the sampling and laboratory activities. Concerns are
identified by the QA staff and corrected by project
personnel, as needed.
g. Evaluate the data. Before preparing the final report, data
are evaluated against project objectives. More data may be
collected, if needed. Although collecting more data may be
the norm for exploratory projects, for major studies this is
often impractical or impossible. Sometimes decisions must
be made even when data are incomplete or inconclusive.
h. Prepare and review the final report. No project is complete
until the final report is prepared and reviewed. As shown
in the Figure 3, project personnel prepare the final report,
which is then reviewed by the QA department prior to
release.
This sequence of activities is similar whether one is evaluating
a pollution control technology, assessing the health effects of
radon exposure, or establishing the efficacy of a new vaccine.
9
-------
As suggested by Figure 3, the most intense interactions with the
QA function occur at limited but important junctures, in this
case as reviews of project plans and final reports, and as audits
during data generation. However, an equally important juncture
involves the early interaction between project management, the
customer and the QA staff in developing the blueprint for the
project plan. This early group interaction as well as project
plan reviews, audits and final report reviews are explained in
the following text.
Developing the Blueprint. It is either prudent (low risk
project) or imperative (high risk project) that earnest project
planning activities precede implementation activities. The
blueprint activities, when done correctly, can help ensure that
proper focus is given, adequate resources are provided and
difficult issues are resolved. The process for planning a
project comprises seven steps7:
1. State the problem.
2. Identify the decision.
3. Identify the inputs to the decision.
4. Define the study boundaries.
5. Develop a decision rule.
6. Specify tolerable limits on decision errors.
7. Optimize the design.
Implementation of these 7 steps results in qualitative and
quantitative statements that pinpoint specific study objectives,
define the types of data needed, define the statistical
population the data are considered to represent, and specify the
tolerable risks for false positive and false negative decision
errors.
Initial Inputs (Steps 1-3)
The initial inputs include a concise statement of the
problem that is being addressed, the decision(s) that will be
made based on the results of the study, and all of the critical
parameters that are needed in order to make the decision(s).
Parameter inputs may include decisions such as: list of analytes,
sampling strategies, type of sample containers needed, sample
preservation requirements, analytical methods that can be used,
types of QC samples needed, approaches to statistical
interpretations, etc.
Define the Study Boundaries (Step 4)
This step involves the defining of the physical boundaries
of the site being investigated as well as the boundaries of the
inference space (i.e., defining the conceptual population
represented by the sample data). Defining the boundaries of the
10
-------
study however, goes beyond defining the physical boundaries of
the site. It also includes defining temporal boundaries (i.e.,
considering and addressing the potential impacts of seasonality
or other time-related considerations and how these will be
addressed in the data collection process).
One of the fundamental ideas that must be kept in mind when
defining the boundaries of a study is that the decisions made,
ultimately, rest on inference. Although one talks about
measuring the concentration at a site and basing decisions on the
data, what one actually does is make decisions on the basis of
inferences that are, in turn, based on estimates. The analytical
result on one sample is only one out of a theoretically infinite
number of possible results for a theoretically infinite number of
possible analyses of that sample.
Develop a Decision Rule fStep 5)
The decision rule is a summary statement that defines how a
decision maker expects to use data to make the decision(s)
identified in Step 2. The same way that multiple decisions, for
example, might pertain to multiple areas within a site, there may
be {and often are) multiple decision rules for different areas of
the site or for different pollutants. Development of a decision
rule involves the following check steps:
* Specify the statistical parameter (e.g., mean, 90th percentile,
upper tolerance limit, etc.) that characterizes the population
of interest.
* Specify the action level of the study.
* Develop an "if...then" statement that describes the decision
rule in terms of alternative actions.
Specify Tolerable Limits on Decision Errors fStep 61
As noted in the previous discussion on defining study
boundaries, decisions about a site ultimately rest on estimates
of parameters of statistical populations. The true average
concentration at a site is not known and is not knowable because
it is a mean of an infinite population. Therefore, decisions
based on average site concentration must be made using estimates
of the true site average,developed on the basis of limited
sampling data for an infinite population. This introduces
sampling error into the estimate that is used as the basis for
decision making. These estimates, which are based on measurement
data, also have an inherent uncertainty associated with them
because of random and systematic errors in the measurement
process. These elements of uncertainty reflect measurement
error. Because decisions are based on estimates that contain
inherent uncertainties, there is always some risk of error in the
final decision.
11
-------
Decision errors are commonly divided into false positive errors
and false negative errors. To reduce the risk of decision errors,
the study design must include sufficient data collected in a
statistically sound manner to adequately estimate the population
parameter used as the basis for decision-making. Uncertainty due
to sampling error can be reduced by collecting large number of
samples. Uncertainty due to measurement error can be reduced by
using more precise and accurate analytical methods and by
performing multiple analyses of each sample and averaging the
results. However, reducing uncertainty and associated risk of
decision errors increases the cost of collecting data.
Therefore, one of the most important steps in the initial
planning process is the sixth step, in which the acceptable risks
of the two types of decision error are established.
Optimize the Design (Step 7)
The seventh and last step of the initial planning process is
to develop and optimize the project design. This involves
integrating the outputs of the previous steps into the most
resource-effective data collection design that satisfies the data
user (i.e., customer) needs. The outcome of this last step wil1
provide the necessary information to decide on one of two plans
of action: 1) Given the available resources, the project
objectives can be achieved and, therefore, the project can move
forward; or 2) given the available resources, the project
objectives cannot be achieved and ,therefore, the initial
planning process (steps 1-7) must be repeated with an adjustment
to resources, project objectives or both.
Reviewing the Project Plan. Even if no formal QA review were
planned, preparing a written project plan is an essential step in
any formal investigation. Preparing such a plan has the virtues
of
clarifying the thinking of all involved parties;
integrating the goals and efforts of disparate project
participants into a single document that can be
reviewed by all;
permitting a review by independent experts;
providing clear instructions to data generators; and
finally,
fostering agreement on how data will be interpreted.
To achieve these goals, the project plan must not merely list the
methods to be used, but must demonstrate how the intended
measurements will achieve the project goals. This plan must
provide concrete steps for assuring that the data will be of
12
-------
known and adequate quality, and should provide means for
documenting data quality. The project plan should also assign
responsibilities and establish means for regular communication
among project participants.
The QA staff reviews this document in detail using experts in the
field of interest. The primary question asked by the reviewers
is "Will the planned test achieve the project objectives?"
Questions relating to this central issue include the following:
"Are objectives clearly stated? Are sampling and analytical
methods appropriate, and are the applicable QC methods clear and
adequate? Will data reduction and statistical procedures permit
unbiased statements of overall uncertainty?" In short, the QA
reviewer "thinks through" the entire project to recognize any
problems beforehand, and then writes up observations in a
detailed report that delineates each concern and its potential
implication. This report is then sent to project management.
It is noteworthy that the project plan is "repaired" not by the
QA staff but by project staff. It is thus essential that the
reviewer describe all concerns clearly, emphasizing the effect
that the planned approach will have on the outcome of the
project. Once the cause-and-effect relationships are clear,
project staff normally are anxious to solve the concern before it
can develop into an intractable and perhaps embarrassing problem.
It is also important that the QA reviewer avoid "nit picking" and
concentrate on the most important issues. Consequently QA
reviewers must be aware of the general programmatic setting and
must be experienced and knowledgeable in the technical field of
interest. Often it is necessary to employ multiple reviewers
with complementary expertise to address all aspects of a project.
To maintain the necessary objectivity and to provide a fresh
outlook, it is also best that the QA reviewer not be previously
involved in the project.
Keeping with the philosophy that project management — not the QA
office — is responsible for overall quality, project management
may choose to address all or some of the QA review comments.
However, in the vast majority of cases, all review comments are
satisfied before the project proceeds.
Auditing the Project. The audit can be divided roughly into
three parts—planning, the site visit, and reporting. The
planning stage must begin by defining the goals and scope of the
audit, that is, by defining the standard against which the audit
will be performed. Of course, the auditor must become familiar
with the planned measurements and should attempt to anticipate
likely areas of concern. The auditor must also contact the
principal participants to identify their requirements and to set
up schedules. The site visit is the central activity and
consists largely of personal interviews with technicians and
other data generators. During this stage the auditor must
13
-------
inspect relevant operations, samples, and documentation. Any
apparent concern must be brought up promptly and discussed as
needed. A closeout meeting must be conducted at the end of the
review to inform management of any concerns and to discuss
possible corrective action, if needed. While a formal written
report must be prepared after completion of the review, project
management should not be "surprised" by any major findings in the
follow-up report.
To be useful, the audit must be conducted early in the project
cycle to permit corrective action before irreversible harm has
occurred. Identifying concerns late in a project cycle is
usually not constructive and does little more than build
resentment.
Perhaps more than other QA tasks, the audit reguires verbal and
interpersonal skills. The auditor must take the lead and set the
tone but at the same time must foster the free exchange of ideas.
During an audit most information is obtained from personal
interviews with the individual technicians who handle the
samples, and in this situation the auditor must be patient,
perceptive, and persistent. While the auditor likely arrives
with certain routine questions in mind, it is often the
unexpected finding that leads to the most significant concern.
After all, the technicians have likely been audited before and
have consequently corrected the most common problems. Thus,
flexibility and inquisitiveness are very important during the
audit process.
Sometimes the auditor meets resistance and must persist to fully
uncover an adverse situation. In these cases it is particularly
important to explain the potentially deleterious effect on the
project, and to consider the opinions of all parties. The
auditor must convince project management that the concern truly
matters if corrective action is to occur.
Reviewing the Final Report. The review of the final report is
the last "inspection" before the "product" is released. As in
the case of the project plan, the final report is reviewed by
technical experts who were not involved in the project. The
principal goal here is to assure that the data support the
conclusions, that the major areas of uncertainty are identified,
and that data are developed in a logical and consistent manner.
A written review is prepared identifying all concerns and
providing recommendations, as needed.
RULES OF ENGAGEMENT
The previous section briefly describes the tools available to the
QA staff operating in a project environment. The following
14
-------
discussion presents guidelines to the QA staff regarding how they
can apply these tools effectively.
Do Your Chores. QA staff in a project environment typically find
themselves dealing with a variety of subject matter and
personnel. Indeed, it is this feature that primarily
distinguishes a project management environment from routine
production setting. Nevertheless, the QA staff is expected to
carry out certain routine tasks for nearly every project. For
environmental projects, these tasks include reviews of test plans
and final reports, on-site inspections during testing, and
occasionally assistance to project management. Regardless of the
setting, though, these operations constitute the most tangible
and immediate contribution to project execution, and it is in
this context that interaction with project management most often
occurs. Consequently, such operations must be given first
priority as the mechanism for making significant contributions
and for building rapport with project management.
Require a Written Project Plan. A written plan describing how
project objectives will be achieved and how the needed quality
will be assured is considered central to any investigation. For
this reason groups as diverse as the US Environmental Protection
Agency", the Japanese Ministry of Trade and Industry', and the
Organization for Economic Cooperation and Development10 have for
many years required that projects begin with a written plan.
Subsequent experience has shown that preparation and review of a
written plan clarifies issues and helps avoid errors. The
various benefits of preparing a written plan were discussed
previously.
Document Your Needs. The agencies cited previously provide only
general guidelines for QA compliance. Unless the QA staff
translate these into specific requirements for the projects
typical of their work place, project managers will not know what
is expected. In this situation QA requirements may simply be
ignored, or efforts may be extensive but misguided. It is thus
essential that the QA staff develop written guidelines
translating the general organizational goals into specific, local
requirements.
Be Flexible. Diversity and variety are one hallmark of a project
management setting. It is not unusual for the QA staff to deal
with projects as disparate as basic research, demonstrations of
mature technology, legal investigations, or even epidemiological
studies. Professionals involved may include engineers,
scientists, legal staff, economists, or medical personnel. In a
university research setting, speed of response and flexibility
are of utmost importance, and experiments can simply be repeated
if results are inconclusive. In contrast, a demonstration of
pilot-scale technology or a major epidemiological study may waste
millions of dollars if results are inconclusive. In such
15
-------
situations the QA staff must adapt to the nature of the work and
the potential risks, allowing the basic researcher maximum
freedom while minimizing the risk of costly errors. In part,
this means providing different QA guidelines for different types
of projects.
Set Priorities. To make the most significant contribution, QA
efforts must be weighed against the potential benefit. This
means that QA involvement should be focussed where the cost of
wrong decisions is greatest, or in those areas most prone to
problems. From this perspective, natural areas of focus are
large programs requiring the participation of multiple parties.
Don't Dilute Responsibility. As is suggested by Figure 3, QA is
only a small (but very important) part of the overall project
effort. Here the QA manager may be likened to a visiting uncle
who provides child rearing advice to the parents. No matter how
insightful the advice may be, the uncle's understanding of the
situation is always less than complete, and consequentially the
final authority and responsibility must rest with the parents.
Further, unless such advice is offered with tact, it will be
ignored and perhaps even resented. Similarly, the project
management martials the resources, selects the personnel, makes
the day-to-day decisions, and is fully involved over the entire
duration of the project. As such, the project manager must have
the final authority and responsibility for the quality of the
project. This means that the project manager may reject the
advice of the QA staff, although in practice this is not likely
to occur very often.
It has been said that "quality is everyone's responsibility."
This is true in the sense that everyone needs to look beyond
their limited assignment and take initiative to improve the final
product. This statement means that the QA staff must perform its
tasks competently and always be alert for additional methods of
improving the final product. Nevertheless, the project manager
has special responsibility for directing the effort and for
acting as the "gatekeeper" of the product, not allowing the
release of any product of inadequate quality.
Be on Time. Be Prepared, and Be an Expert. There is a saying
among project managers, "On time, on budget, on spec.—choose
two!" In contrast, the QA staff must deliver in all three
regards to gain the cooperation of project managers. The most
insightful review, delivered when the subject activity is
complete, is worthless. This means that on-site reviews must be
completed early in the project to permit effective corrective
action, if needed. Similarly, efforts must be completed within
budgetary constraints and with minimum disruption to the project.
Finally, it is not enough to be merely an expert in QA; the QA
reviewers must also be technical experts in the subject area. If
16
-------
the QA team does not have the appropriate technical background,
then it becomes necessary to find someone who does.
Be Clear, but not Confrontational. The QA staff should neither
equivocate nor be unnecessarily confrontational. State the
potential concern in terms of its possible impact on the project.
Listen respectfully to possible explanations and suggested
corrective action. Remember, what originally appears to be a
serious shortcoming frequently becomes reasonable once further
explanation is provided. If need be, contact upper management to
resolve the problem. However, almost without exception, concerns
can be resolved by the mutual effort of project participants.
Find the Principals. In a large organization it may be difficult
to identify what projects are underway and who is working on
them. This means that some projects may evade QA oversight
entirely, or that QA documentation may be prepared without the
needed guidance. The QA staff may discover such projects only
when a poorly prepared project plan is delivered for review. To
provide uniform QA oversight in the most cost-effective manner,
the QA staff must implement a mechanism for identifying active
projects and the accountable project managers.
Take the Initiative. The QA staff, because of their exposure to
a wide range of projects, is uniquely situated to identify
frequently occurring problems. For instance, the QA staff may
become aware that several investigators need help in selecting
appropriate statistical procedures in certain recurring
situations, or that an important QA procedure is frequently
overlooked. The QA staff should take the initiative and
recommend solutions. The QA personnel should also be alert to
possible cost savings that might be realized without degrading
quality. Surprisingly, some project planners provide for an
excessive number of expensive measurements, and such situations
should be brought to the attention of project management for cost
reasons, even when there is no adverse effect on quality.
SUMMARY
As depicted in the Quality System Pyramid (Figure 1), a dynamic
quality system consists of three interdependent parts:
Organizational (Institutional) QA; The management structure
that provides the needed organization, tools, and resources.
Catalytic QA: The attitudes and relationships among staff
needed to make all of the organizational elements fit
together to yield a product that is greater than the
individual parts.
Technical (Project) QA: The application of concepts to a
specific project.
17
-------
None of the parts can be successful without the other parts.
While the principles previously described are taken from the
perspective of one working in the environmental field, it seems
likely that they would apply equally well (with some adjustments)
to other project-oriented endeavors such as the health or
forensic sciences.
ACKNOWLEDGEMENTS
The author wishes to thank Dr. John Wallace of Wallace
Technologies, San Diego, CA for his consultation and assistance.
Dr. Wallace consults in the areas of experimental design,
chemical analysis, as well as the technical and management
aspects of quality assurance.
The author would also like to acknowledge the Quality Assurance
Management Staff of the United States Environmental Protection
Agency for providing him with the inspiration to write this QA
chapter.
REFERENCES
1. ISO Central Secretariat, ISO 9000 International Standards for
Quality Management, third edition, International Organization for
Standardization, Geneva, Switzerland.
2. ISO, Guide 25, General Requirements for the Competence of
Calibration and Testing Laboratories, International Organization
for Standardization, Geneva, Switzerland, 1990.
3. USEPA, Quality Assurance Management Staff, EPA Requirements
for Quality Management Plans, EPA QA-R2, August, 1994.
4. US Department of Energy Order DOE 5700.6C, August 21, 1991;
United States Pharmacopeia, Part 210, Current Good Manufacturing
Practice; Code of Federal Regulations, Title 21, part 58, Good
Laboratory Practices.
5. F. M Garfield, Quality Assurance Principles for Analytical
Laboratories, Association of Official Analytical Chemists (AOAC),
1991.
6. USEPA, Risk Reduction Engineering Laboratory, Preparation
Aids for the Development of Category I, II, III, and IV Quality
Assurance Project Plans (EPA/600/8-91/003, 004, 005, and 006),
February 1991.
7. USEPA, Quality Assurance Management Staff, Guidance for the
Data Quality Objective Process, EPA QA/G-4, September, 1994.
18
-------
8. Office of Research and Development, USEPA, Interim Guidelines
and Specifications for Preparing Quality Assurance Project Plans,
QAMS-005/80, December 29, 1980; Quality Assurance Management
Staff, USEPA, EPA Requirements for Quality Assurance Project
Plans for Environmental Data Operations, EPA QA/R-5, 1994.
9. Japanese Ministry of International Trade and Industry, GLP
Standards Applied to Industrial Chemicals and Annexes.
Notification Number 85, March 31, 1984.
10. OECD, The OECD Principles of Good Laboratory Practice,
Environmental Monograph Number 45, Paris, France, OECD/GD(92)32
19
-------
TECHNICAL REPORT DATA
(Please read Instructions on the reverse before complett
1. REPORT NO. 2
EPA/600/A-95/099
3. I
4. TITLE ANO SUBTITLE
QUALITY ASSURANCE
ORGANIZATIONAL - CATALYTIC - TECHNICAL
5. REPORT DATE
6. PERFORMING ORGANIZATION CODE
7. AUTHOR(S)
Guy F. Simes
8. PERFORMING ORGANIZATION REPORT NO.
9. PERFORMING ORGANIZATION NAME ANO ADDRESS
EPA, Office of Research and Development
National Risk Management Research Laboratory
26 W. Martin Luther King Drive
Cincinnati, OH 45268
10. PROGRAM ELEMENT NO.
11. CONTRACT/GRANT NO.
Not Applicable
12. SPONSORING AGENCY NAME ANO ADDRESS
National Risk Manajetient Rasaarch Laboratory
Office of Research and Development
U.S. Environmental Protection Agency
Cincinnatif OH 45268
13. TYPE OF REPORT ANO PERIOO COVERED
Book Chapter
14. SPONSORING AGENCY CODE
EPV&00/14
15. SUPPLEMENTARY NOTES
Project Officer = Guy Simes (513) 569-784b
16. ABSTRACT
In a normal quality system, two distinct levels of quality
assurance (QA) requirements are addressed: the organizational (or
institutional) level and the technical (or project) level.
Dynamic quality systems, on the other hand, not only include the
normal components but also integrate catalytic components. These
catalytic components are critical to the foundation supporting
the Quality System Pyramid (QSP); however, they are often
overlooked in the design stage of many quality systems. While
this chapter does consider the characteristics of a normal
quality system, an intentional focus is levied at the seldom
explored dimension — catalysts.
17. KEY WORDS ANO DOCUMENT ANALYSIS
a. DESCRIPTORS
b. IDENTIFIERS/OPEN ENDED TERMS
c, COSATi Field/Croup
Quality Assurance
Quality
18. DISTRIBUTION STATEMENT
Releasable to public
19 S£CU81TY CLASS I This Report)
Unclassirled
21. NO. of pages
20. SECURITY CLASS (This page)
Unclassified
22. PRICE
EPA Form 2220-1 (R«y. 4-77) previous edition isossolete
------- |