ELAB-9801
July 1998
RECOMMENDATIONS FOR THE USE OF CHECKLISTS
DURING A NELAC ON-SITE ASSESSMENT
A Report Prepared by
The Environmental Laboratory Advisory Board,
An EPA-Sponsored Federal Advisory Committee
JULY 1,1998
-------
FOREWORD
This report was prepared as part of the activities of the Environmental Laboratory Advisory Board
(ELAB), a Federal Advisory Committee sponsored by the U. S. Environmental Protection Agency. This
report has not been reviewed for approval by the U. S. Environmental Protection Agency; hence, the
contents may not necessarily represent the views and policies of the U. S. Environmental Protection
Agency, or of other agencies in the Executive Branch of the federal government. The mention of trade
names or commercial products does not constitute an endorsement or a recommendation for use.
For further information about this report, or other activities of ELAB, please contact the Designated
Federal Official (DFO) for ELAB.
Ms. Elizabeth Dutrow
USEPA/ORD
401 M St. SW (8724R)
Washington, DC 20460
-------
ACKNOWLEDGMENTS
The Environmental Laboratory Advisory Board (ELAB) wishes to recognize the efforts of those
individuals who served as members of the Laboratory Assessment Workgroup and that helped develop
the recommendations in this report.
Laboratory Assessment Workgroup Members
Name
Mr. Wayne Davis
Mr. Randy Dersham
Mr. Ray Frederici
Dr. Jack Hall
Ms. Janet Hall
Dr. Wilson Hershey
Dr. Tom McAnninch
Mr. Paul Mills
Ms. Marlene Moore
Mr. Anand Mudambi
Mr. Jerry Parr
Ms. Ann Rosecrance
Ms. Jackie Sample
Dr. Joe Slayton
Mr. Alfredo Sotomayor
Dr. Allen Verstuyft
Affiliation
South Carolina DEQ
New York DOH
Severn-Trent Laboratories
Quanterra
INDUS International
Lancaster Laboratories
Eastman Chemical
Mentorprises
Advanced Systems
US Corps of Engineers
Catalyst Information Resources
Core Laboratories
US Navy
USEPA Region 3
Wisconsin DNR
Chevron Research Company
-------
EXECUTIVE SUMMARY
The Environmental Laboratory Advisory Board (ELAB) was established on July 31, 1995, in accordance
with the Federal Advisory Committee Act (FACA), 5 U.S.C. App. 2 Section 9 (c). As specified by federal
charter, ELAB provides advice and counsel to the United States Environmental Protection Agency's
(USEPA) Administrator, Deputy Administrator, and Environmental Monitoring Management Council, the
National Environmental Laboratory Accreditation Conference (NELAC) Board of Directors, and other
federal agencies concerning the systems and standards of accreditation for environmental laboratories.
This report presents the recommendations of the Environmental Laboratory Advisory Board (ELAB) on
the use of checklists during laboratory on-site assessments by accrediting authorities recognized by the
National Environmental Laboratory Accreditation Program (NELAP). ELAB recommends that:
The NELAC On-Site Assessment Committee develop checklists to perform Quality Systems and
Technical Systems reviews during laboratory assessments.
Checklists strive for simplicity and brevity, and that their structure parallel the order of events of
on-site assessments.
Method-specific checklists be primarily reserved for assessing performance of method-defined
parameters.
That ensuring consistency of assessments be redirected to efforts to develop sound assessor
training courses.
-------
ELAB RECOMMENDATIONS
1. Checklists are tools to use during laboratory on-site assessments, but should be used in the context of
other tools available to assessors. Checklists should help assessors determine logically the conformance
of a laboratory with its quality system and the adequacy of such a system for its intended use.
Accordingly, NELAC assessors should determine laboratory competence by focusing on reviewing a
laboratory's quality system. Checklists developed for this purpose should:
• Be sufficiently detailed to be useful reminders but should not attempt to be exhaustively
comprehensive.
• Reflect assessor knowledge of the discipline under evaluation.
• Have a structure that corresponds to the flow and order of laboratory processes.
• Strive for simplicity.
• Be logical.
• Be prescriptive but not restrictive.
• Include space to document assessment findings.
• Help laboratories understand why a finding constitutes a deficiency.
• Make reference to pertinent sections of the NELAC standards.
• Be universally available.
• Strive for brevity.
2. Checklists are primarily needed to assess laboratory compliance with the NELAC standards, especially
Chapter 5, Quality Systems.
3. Checklists should accommodate both essential technical requirements in mandated methods and
performance-based measurement systems (PBMS).
4. Checklists for specific methods may be appropriate when methods do not include Quality Control
(QC) information, or when methods define the assayed parameter (e.g., BOD, TCLP). Technical
Systems review checklists, independent of the specific method used by a laboratory, can be used to
evaluate correct execution of method and technology essentials in most assessments.
5. Although checklists can standardize some aspects of laboratory assessments, consistency of on-site
assessments is best accomplished by uniform assessor training on the NELAC Quality Systems Standard,
the NELAC accreditation process, technical competencies, and proper use and interpretation of checklists.
6. Technical Systems checklists should:
• Be organized by analytical technology or instrumentation.
• Segregate preparatory and determinative steps in an analysis.
• Document a review of a laboratory's data collection processes.
• Document a review of a laboratory's data reduction and verification processes such as calculation
checks, raw data conversions, data transfers, and permanence and incorruptibility of electronic files.
• Verify the implementation of a laboratory's Quality System by analysts at the bench level.
-------
7. NELAC should establish a defensible process for developing, publishing and updating on-site
assessment checklists.
BACKGROUND
The 1992 report prepared by the Committee on National Accreditation of Environmental Laboratories
stated that the key elements of a national program for laboratory accreditation would be "on-site audits,
performance evaluation testing, and data audits."
On April 22, 1998, draft method checklists were posted on the NELAC website. Based on a review of
these checklists, ELAB decided to form a workgroup to:
Provide recommendations to ELAB regarding key elements of a laboratory assessment (audit)
reflecting audit consistency, a reasonable level of detail and an eye towards PBMS. Ideally, the
recommendation would include a detailed model checklist that could be used by laboratory
assessors.
The Workgroup members worked on this activity from May through June of 1998 and presented their
findings to ELAB on July 1, 1998. The Workgroup reviewed NELAC Chapters 3 and 5, the Assessor
Training Manual and the draft method checklists posted on the NELAC Website. Workgroup members
shared other audit checklists, related technical information and participated in several conference calls to
develop the findings that were presented to ELAB.
On July 1, 1998, ELAB recommended this report be forwarded to the NELAC On-Site Assessment
Committee for consideration.
A prototype for a Quality Systems checklist is included with this report (Attachment 1). This prototype is
provided to show the level of detail and organization envisioned by ELAB. The checklist would need
further refinement to allow for adequate space to record findings and to improve its utility.
ELAB has also considered future directions based on draft ISO Standard 17025. These comments are
provided in Attachment 2.
-------
Attachment 1. Example of a Quality Systems Checklist
5.4 ORGANIZATION AND MANAGEMENT
5.4.1 Legal Definition of Laboratory
5.4.2 Organization
Pre-Assessm Job Descriptions for All Positions.
ent Clear Description of Lines of Responsibility.
Technical Director (Resp. For Tech. Operation)
OAO (Resp. For QS Implementation & Independent from Lab Operations Oversee QS)
• Perform Internal Audits Yearly & Doc. Corrective Actions
• Maintains Quality Manual
5.6.1 General requirements for laboratory staff
Combination of Exp. & Ed. To Do the Job
All Must Comply with Quality Manual
Define Exp. & Ed Requirements (Proof: PT; IDC; Splits)
Have Training & Training Doc.
Have Read/Understand Quality Documents
Adhere to Sample Acceptance Policy
Comply with Sample Tracking System
Document Quality of All Reported Data
5.6.2a Laboratory Management Responsibilities (defining personnel minimal
qualification and experience)
Quality Manual
Pre-
Analytical
Assessment
5.5.2i (Project Management) Manual specifies mechanisms for ensuring
laboratory reviews all potential new work to ensure it has the appropriate
facilities and resources before commencing such work
Table of Contents
Quality Policy statement
Organizational Structure & Job Descriptions
Record System/Procedures
Approved Signatories
Procedures for Traceability of Results
Listing of Test Methods
Sample handling Procedures
Feedback and Corrective Action
Confidentiality Procedures
Procedures for Audits & Data Review
Procedures to Assure Needed Training/Experience
Reporting Procedures (Analytical Results)
Listing of References/Glossaries/Appendices
5.15 (Procurement) OUTSIDE SUPPORT SERVICES AND SUPPLIES
(Bottles, sample containers, preservatives) Also other procurement practices of
supplies
* Of Adequate Quality to:
Sustain Confidence in the Lab Tests.
* Records Maintained of All Such Suppliers/Vendors.
5.11 SAMPLE HANDLING, SAMPLE ACCEPTANCE POLICY AND
SAMPLE RECEIPT
5.11.1 Sample Tracking
* Unique Number for Each Container
* Linked to Field Id
* Durable Labels
5.11.2 Sample Acceptance Policy Containers/ Holding Time/
-------
Preservation. /Volume/ Records
* If Unacceptable All Data Flagged
5.11.3 Sample Receipt Protocols
* General condition of Samples Noted
* Procedures to Check Preservatives (e.g., pH, Temp.)
* Permanent Chronological Record
* Link Sampling/Field Information & Measurements
* Chain-of-Custody Records Retained (If Needed; see 5.12.4)
5.11.4 Storage Conditions
* Established Protocols Consistent with Regulations
* Acceptance Criteria established: e.g., above freezing to 6C
* Security as Appropriate To Maintain Integrity
Sample Disposal
* SOP for Sample Disposal
5.12.4 Legal or Evidentiary Custody (Complete when required.)
5.12.4.1 Basic Requirements
5.12.4.2 Required Information in Custody Records
5.12.4.3 Controlled Access to Samples
5.12.4.4 Transfer of Samples to Another Party
5.12.4.5 Sample Disposal
5.7 PHYSICAL FACILITIES - ACCOMMODATION AND ENVIRONMENT
5.7.1 Environment
5.7.2 Work Areas
Analytical Do Not Invalidate or Adversely Affect Results:
Sectlon Monitoring of Conditions:
Assessment As Per Test Method Requirement (Example: 9222D MF)
5.9.4.2.1 Analytical Support Equipment (Balances, refrigerators, ovens,
pipetors)
- Calibration
- Control limits
- Corrective Action
- Record Keeping
Acceptance Criteria for Support Equipment:
All Calibrated Yearly vs. NIST Traceable Reference (When Available)
Each Working Day: Balances, Ovens, Ref. Freezers, Incubators, Water baths checked with NIST Traceable
References
Each Week: Liquid Dispensing Devices Checked for Accuracy (Except Class A)
* Example = Media Dispensers
Per Use: Autoclaves Temp.& Pressure Doc. Effective
* Sterility Tests- NOT Heat Tape Alone!
5.8 EQUIPMENT
5.8a Equipment Maintenance
5.8c Out-of-service equipment
5.8d Records
Records of:
* Maintenance Procedures (Clean & Maintained)
* Manufacturer &. Operator Instructions.
* Location; A Use Status (labeled when Defective/Out-Of-Service)
* History of Malfunctions/Repairs/Modifications
5.10.5 Documentation and Labeling of Standards and Reagents
Manufacturer's Certificates of Purity, etc.
Details of Preparation (SOPs)
Labeled w/ Unique Identifier & Expiration Dates
5.9.3 Reference Standards
-------
Reporting
Section
Assessment
Management
Assessment
5.9.4.3 Instrument Calibrations
5.9.4.4 Calibration Verification
5.9.4.4.1 Initial Calibration Verification
5.9.4.4.2 Continuing Calibration Verification
5.9.2 Traceability of Calibration
5.5.4 Essential Quality Control Procedures (use specific checklists developed
for appendix D types of testing)
+/- Controls (Blanks, Reference Materials)
Variability/ Repeatability (Replicates)
Accuracy (Calibration, PTs.)
Method Capability (MDLs, Quant. Limits)
Appropriate Formulae (Calculations)
Reagent/Std. Quality (Reagent Grade)
Established OC Acceptance Criteria (Control Limits)
Sample Acceptance Policy (Criteria to Reject Samples)
All QC listed in Methods Manual (Yearly Audit)
Essential QC for Specific Fields of Testing (Appendix D)
5.12.3.2 (Records) Laboratory Support Activities
5.12.3.3 Analytical Records
5.10 TEST METHODS AND STANDARD OPERATING PROCEDURES
5.10.1.1 Standard Operating Procedures (SOPs)
Up-To-Date, Organized & Readily Available to Staff.
* May Cite Published Methods (By Reference ok!)
* Must Doc. Any Changes to Published Methods
* Organized with Effective Date & Rev. #
Fully Documented and Validated
5.12.3.3 Analytical Records
5.10.4 Data Verification
5.5.3.5 Corrective Actions
5.13 LABORATORY REPORT FORMAT AND CONTENTS
17 Must Items, Including: Out of Control QC;
< 3.18 *MDL Values
Exceptions: Facility Lab Providing Compliance Data Solely for the Facility
Information 1-17 Available for Review By Accrediting Authority
5 . 1 3c LABORATORY REPORT subcontract laboratory data
* Clients Must Be Advised in Writing
* Subcontractor Must Be NELAC Accredited For Tests Performed.
5.10.6 Computers and Electronic Data Related Requirements
Computers: Performing, Manipulation and Reporting of Test Data
Follow: A 2 1 85 Good Automated Laboratory Practices- Section 8.1-8.11"
5.12.2 Records Management and Storage
5.12.1 Record Keeping System and Design
Two Levels:
Sample Tracking
Legal/Evidentiary
Generally Redundant with Previous Sections.... Additions Include:
* Indelible Ink
* No Obliterations- Single Line
-------
* Record Management System (Access Log, Protected from Fire, Theft, Flood, Pests
More Emphasis on Security than TrackingScenario
5.12.3.4 Administrative Records
5.10.2.1 Method Validation/Initial Demonstration of Method Performance
5.6.2b-c (TRAINING and documentation) Laboratory Management
Responsibilities
5.5.3 Audits
5.5.3.1 Internal Audits + Corrective Action
5.5.3.2 Managerial Review + Corrective Action
5.5.3.3 Audit Review + Corrective Action
5.5.3.4 Performance Audits + Corrective Action
5.5.3.5 Corrective Actions (process)
Audits: Self-Assessment Review
Internal Audits: (Yearly QAO - Compliance with QS)
Management Review : (Yearly Review by Management of QS to Assure Continued Suitability
Effectiveness)
Performance Audits'. (Ongoing Checks to Monitor Quality of Analytical Activities)
* QC Procedures & Control Limits
* Qtly QCs 2nd Source vs. Calibration Stds.
* Logic Checks, e.g., P04 < TP
Corrective Actions:
* For Each of the Audit Type
* For Out-of-Control QC (Qualify Data)
EXTERNAL AUDITS: ON-SITE ASSESSMENT @ 2 YRS
5.16 COMPLAINTS + Corrective Action
* Doc. Policy & Procedures for Resolution of Complaints
* Serious Complaints: [Concern Quality of the Calibration or Tests]
= Lab Shall Perform an Internal Audit
-------
Attachment 2. Comments for Future Consideration
ISO Guide 25 is the basis of the lab accreditation system of NELAC, but the Guide is being changed to an
ISO standard, proposed as 17025. The changes (additions) of categories could be used to restructure the
Quality Systems audit checklist and process into 2 parts: Management requirements, and technical
requirements. These are subdivided into the following areas:
Management Requirements
Organization and management
Quality system
Document control
Request, tender and contract review
Sub-contracting of tests and calibrations
Purchasing services and supplies
Service to the client
Complaints
Control of nonconforming testing and/or calibration work
Corrective action
Preventive action
Records
Internal audits
Manage reviews
Technical Requirements
General
Personnel
Accommodation and environmental conditions
Test and calibration methods including sampling
Equipment
Measurement traceability
Sampling
Handling and transportation of test and calibration items
Assuring the quality of test and calibration results
Reporting the results
The addition of sampling will require a written description of sub-sampling and also fits with the
"Measurement System" part of PBMS. The "request, tender and contract review" makes sure the lab
understands and can commit to do the work requested as is called out in 5.5.21, Quality Manual, in the
Quality Systems Checklist. These requirements also cover continuous improvement (preventive action)
and management reviews, in line with the EPA's Executive Order 5360.1. Measurement traceability is
also key, considering lack of available PT samples and certified reference standards in some areas (5.9.3,
Reference Standards). The Guidance and Requirements documents from EPA (G-7 and R-7) covering
assessments could also be considered in the content of the Quality System Checklist.
------- |