Section No.: 1 Revision No.: 1 Date: September 15, 1993 Page: 1 of 21 QUALITY ASSURANCE PROJECT PLAN FOR THE NATIONAL PERFORMANCE AUDIT PROGRAM (NPAP) Approved by: Ronald K. Patterson Quality Assurance Manager, AREAL William J. Mitchell Chief, Quality Assurance Support Branch Elizabeth T. Hunike NPAP Coordinator Kenneth J. Caviston Manager, ManTech Environmental Technology, Inc. ------- Section No.: 2 Revision No.: 1 Date: September 15, 1993 Page: 2 of 21 SECTION 2 TABLE OF CONTENTS Section Page 1 Title Page 1 2 Table of Contents 2 3 Introduction and Project Description 3 4 Project Organization and Responsibilities 4 5 Quality Assurance Objectives 6 6 Participant Selection 8 7 Data Custody 9 8 Calibration Procedures, Sample Material, and Frequency ... 10 9 Analytical Procedures 12 10 Data Processing and Reporting 13 11 Internal Quality Control 14 12 Performance and Systems Audits 16 13 Preventive Maintenance Procedures 18 14 Specific Routine Procedures Used to Select QC Samples and Assess Data Precision, Accuracy, and Completeness 19 15 Corrective Action 20 16 Quality Assurance Reports to Management 21 Appendix A: SOP for Performing the Routine Activities of the AREAL Coordinator of the NPAP Appendix B: SOPs for Computer Data Entry, Report Printing and Maintenance for the NPAP; Data Validation for Data Bases of the NPAP; and Editing NPAP Data Bases Appendix C: SOPs for Calibration of ReF Devices for Surveying Performance of Hi-Vol Sampler Flow Rates; Conducting the Lead NPAP Audit; Calibration of a Pulsed Fluorescent S02 Analyzer; S02 Audit Device Calibration; Conducting the Sulfate-Nitrate NPAP Audit; Analysis of N0/N02/N0x in Gas Cylinders; N02 Audit Device Quality Assurance Operation Checks; Quality Assurance Checks of Dichot (PM-10) Audit Devices; Conducting an Ozone National Performance Audit; Analysis of CO in Gas Cylinders with GFC Analysis Appendix D: SOP for Audit Systems Verification Center Operational Procedures ------- Section No.: 3 Revision No.: 1 Date: September 15, 1993 Page: 3 of 21 SECTION 3 INTRODUCTION AND PROJECT DESCRIPTION 3.1 Introduction The Quality Assurance Project Plan (QAPjP) for the National Performance Audit Program (NPAP) and related Standard Operating Procedures (SOPs) comprise the basic working documentation for all NPAP activities. To ensure this documentation reflects detailed operational procedures while retaining practical usefulness for day to day NPAP activities, redundancy and excess verbiage have been eliminated. To facilitate this goal, the QAPjP refers to specific sections in the SOPs to supply detailed information whenever possible. 3.2 Project Description The NPAP provides EPA a means to assess the proficiency of agencies that are operating monitors in the State and Local Air Monitoring System (SLAMS) network with one aspect of the minimum quality control (QC) program required under Section 2.0 of 40 CFR Part 58, Appendix A. The following criteria air pollutants are audited under NPAP: high-volume/PM-10 (SSI/dichot) (flow only), sulfur dioxide, nitrogen oxide, nitrogen dioxide, carbon monoxide, ozone, and lead (analysis and flow audited separately). Also audited under NPAP is sulfate/nitrate on filter strips. The NPAP Coordinator is responsible for the administration of the NPAP. This includes contacting participants, scheduling audits , registering participants, following up on poor performers, authorizing reaudits and changes to the information in the data base, keeping the EPA Regional Offices informed, and performing systems and performance audits on the on-site contractor. The on-site contractor (ManTech Environmental Technology, Inc.) is responsible for audit material preparation, shipment of audit materials to NPAP participants, data entry, distribution of data results to the audit participants, analysis, procurement, calibration, and repair and maintenance of audit equipment. ------- Section No.: 4 Revision No.: 1 Date: September 15, 1993 Page: 4 of 21 SECTION 4 PROJECT ORGANIZATION AND RESPONSIBILITIES The NPAP is conducted by both the AREAL Quality Assurance Support Branch (QASB) and ManTech Environmental Technology, Inc. The NPAP Coordinator is Elizabeth (Liz) Hunike, who reports to the Chief, QASB. The NPAP Coordinator is responsible for the administration of the NPAP. This includes contacting participants, scheduling audits , registering participants, following up on poor performers, authorizing reaudits and changes to the information in the data base, keeping the EPA Regional Offices informed, and performing systems and performance audits on ManTech. The duties and decisions of the NPAP Coordinator are considered to be within the scope of policy-making and, therefore, are within the sole purview of EPA. The responsibilities of the NPAP Coordinator are described in detail in the SOP for Performing the Routine Activities of the AREAL Coordinator of NPAP (AREAL/RTP-SOP-QAD-553). This SOP is located in Appendix A. The ManTech personnel consists of Kenneth (Ken) Caviston, Manager, and his support group. ManTech is responsible for audit material preparation, shipment of the audit material to NPAP participants, data entry, distribution of data results to the audit participants, analysis, procurement, calibration, and repair and maintenance of audit equipment. Figure 4-1 shows the project structure. ------- Section No.: 4 Revision No.: 1 Date: September Page: 5 15, 1993 of 21 Director Atmospheric Research and Exposure Assessment Laboratory I Director Quality Assurance and Technical Support Division 1 1 1 1 Quality Assurance Manager] Ronald K. Patterson | I Chief Quality Assurance Support Branch —I _L NPAP Coordinator ManTech Liz Hunike Ken Caviston, Manager Ron Bousquet Ron Brande B. J. Carpenter Rita Harrell Shirley Henry John Holland Mark Holland Annette King Matthew Miller Andy Reinhardt Jim Robbins Kerry Rufty I 1 | ManTech Quality | —(Assurance Manager] | Mike Ray | I I Figure 4-1. Project Structure ------- Section No.: 5 Revision No.: 1 Date: September 15, 1993 Page: 6 of 21 SECTION 5 QUALITY ASSURANCE OBJECTIVES NPAP's goal is to provide audit materials and devices that will enable EPA to assess the proficiency of agencies that are operating monitors in the SLAMS network. To accomplish this, NPAP, based on SLAMS requirements, has established acceptable limits or criteria for each of the audit materials and devices provided in the program. Any device or material not meeting these pre-determined criteria are not used in the program. To assure that audit materials and devices meet these criteria, each material batch or device is tested following established SOPs. Quality objectives for each audited parameter are discussed below and in referenced SOPs. 5.1 Audit Devices/Materials All audit devices and materials used in the NPAP will be certified as to their true value and that certification will be traceable to an NIST standard material or device wherever possible. This represents a 100 percent objective for completion. Accuracy and precision will be dependent on the NIST standard material or device but in all cases will be known. Control charts showing the trends of critical parameters will be developed. SOPs for the calibration of all instruments, devices, and the analysis of performance audit samples will be maintained and kept up to date. Section 8 describes the SOPs. The audit materials used in the NPAP will be as representative and comparable as possible to the calibration materials and actual air samples used/collected by the SLAMS network. 5.2 Audits The objectives for the audits are two-fold: (1) to complete at least 95 percent of the scheduled audits by the end of the year and (2) to determine if the participants' performance exceeds the limits shown below. Audit EPA Determined Limits High Volume/PM-10 (SSI) Dichot (PM-10) Sulfate/Nitrate Lead Sulfur Dioxide Nitrogen Dioxide Ozone Carbon Monoxide Mean absolute % difference > 15% Mean absolute % difference > 15% Mean absolute % difference > 15% Mean absolute % difference > 15% % difference > +15% for 1 or more flows % difference > +15% for 1 or more flows % difference > +15% for 1 or more levels % difference > +15% for 1 or more levels ------- Section No.: 5 Revision No.: 1 Date: September 15, 1993 Page: 7 of 21 5.3 Data Base All audit results will be entered into the NPAP data base as received from the participants . Changes to the data base will be made only upon receipt of written confirmation from the EPA NPAP Coordinator. Documentation of the change will be made in a change control logbook. In order to assess data entry accuracy, a data summary will be calculated for each audit and quarter to be reviewed. This summary will consist of the mean percent differences and the standard deviations for each audit concentration level in each U.S. EPA Region. If any mean or standard deviation is 20% or more, all data sets included in that result will be checked by comparing the original data sheets to the values stored in the NPAP data base. Errors found will be corrected and will be noted in the change control logbook. SOPs for data entry and report production will be maintained and kept up to date. Additional details on data entry, data validation, and correction are given in the SOPs in Appendix B. Appendix B contains copies of AREAL-SOP-QAD-546 (Computer Data Entry, Report Printing and Maintenance for the NPAP), AREAL-SOP-QAD-548 (Data Validation for Data Bases of the National Performance Audit Program), and AREAL-SOP-QAD-551 (Editing NPAP Data Bases). ------- Section No.: 6 Revision No.: 0 Date: July 31, 1991 Page: 8 of 21 SECTION 6 PARTICIPANT SELECTION The list of potential participants for the NPAP is derived from the current year's audit participants, requests from the EPA Regional Offices and other governmental agencies and from a comparison of the SLAMS PARS agency listing with the NPAP participation list. Sections 4.1 and 4.2 of AREAL/RTP-SOP-QAD-553 describes in detail procedures for contacting and registering potential NPAP participants. ------- Section No.: 7 Revision No.: 0 Date: July 31, 1991 Page: 9 of 21 SECTION 7 DATA CUSTODY NPAP data custody follows standard QA procedures to ensure all data generated or received is trackable. This includes the tracking of data results from shipment of audit materials/devices to the NPAP audit participants; receiving data results from the participants; validating data results, and storing results in the NPAP data base. Records are also kept on acceptance testing of audit materials and calibration of audit devices. These records allow the tracking of an audit material/device from its acceptance testing or calibration through to the storage of the audit results in the NPAP data base. Additional details on tracking sample data, acceptance test data and calibration results are contained in the SOPs for each audit procedure. Section 5 of AREAL/RTP-SOP-QAD-553 describes in detail the procedures to follow for processing the raw data sheets received from the audit participants. ------- Section No.: 8 Revision No.: 1 Date: September 15, 1993 Page: 10 of 21 SECTION 8 CALIBRATION PROCEDURES, SAMPLE MATERIAL, AND FREQUENCY 8.1 Calibration Procedures The calibration procedures for audit devices/materials used in the NPAP are detailed in the SOPs in Appendix C. The SOPs located in Appendix C are: SOP No. SOP Title 508 Calibration of ReF Devices for Surveying Performance of Hi-Vol Sampler Flow Rates 510Conducting the Lead NPAP Audit 512Calibration of a Pulsed Fluorescent S02 Analyzer 520SO2 Audit Device Calibration 521Conducting the Sulfate-Nitrate NPAP Audit 523Analysis of N0/N02/N0x in Gas Cylinders 542N02 Audit Device Quality Assurance Operation Checks 543Quality Assurance Checks of Dichot (PM-10) Audit Devices 544Conducting an Ozone National Performance Audit 549Analysis of CO in Gas Cylinders with GFC Analysis 8.2 Sample Material Lead. By May 1, the NPAP Coordinator provides ManTech with instructions to prepare filter strips (quantity and target concentrations) for the next calendar year's audits. By September 1, ManTech delivers 20 sets of filter strip samples from each lot to the NPAP Coordinator for acceptance testing by an independent laboratory and/or the AREAL Standards Laboratory. The lead audits ------- Section No.: 8 Revision No.: 1 Date: September 15, 1993 Page: 11 of 21 are offered quarterly and participants may select one or more audits. The lead samples are prepared in accordance with SOP-510. Sulfate and Nitrate. By May 1, the NPAP Coordinator provides ManTech with instructions to prepare filter strips (quantity and target concentrations) for the next calendar year's audits. By September 1, ManTech delivers 20 sets of filter strip samples from each lot to the NPAP Coordinator for acceptance testing by an independent laboratory and/or the AREAL Standards Laboratory. The sulfate and nitrate audits are offered semi-annually and participants may select one or both audits. The sulfate and nitrate samples are prepared in accordance with SOP-521. Sulfur Dioxide, Oxides of Nitrogen (NO and N02), Carbon Monoxide, and Ozone. Audits of these analyzers are performed using dilution devices. The sulfur dioxide, oxides of nitrogen and carbon monoxide audits additionally use one or two compressed gas cylinders. Participants select only 1 audit per year. Use of the audit device is described in the following SOPs: SOP-520 for S02, SOP-544 for 03, SOP-547 for CO (alternate method), and SOP-542 for N02. High Volume/PM-10 (SSI/Dichot) Particulate Collectors. Audits of particu- late collectors are offered quarterly and participants select a maximum of two non-consecutive quarters. Calibration of the reference flow (ReF) device used in the high volume/PM-10 (SSI) audits is described in SOP-508. The procedure for performing operational checks on the dichot audit device is contained in SOP-543. 8.3 Frequency The frequency of calibrations and acceptance testing is detailed in the SOPs and the frequency of the NPAP audits is detailed in Section 8.2 above. NOTE: The dates specified above are open to negotiation between ManTech and EPA when extenuating circumstances exist. ------- Section No.: 9 Revision No.: 0 Date: July 31, 1991 Page: 12 of 21 SECTION 9 ANALYTICAL PROCEDURES Since the NPAP is an auditing program and not a sampling project, there are no analytical procedures for analysis of samples other than those used to certify the quality of the audit materials or devices. These analytical and certification procedures are covered in Section 8. ------- Section No.: 10 Revision No.: 0 Date: July 31, 1991 Page: 13 of 21 SECTION 10 DATA PROCESSING AND REPORTING 10.1 Data Processing Section 5 of SOP-553 (see Appendix A) describes in detail the procedures to follow for receiving data from the audit participants, entering data into the NPAP data base, handling late data and making corrections to the data. 10.2 Reports ManTech mails the data results to the audit participant and gives the NPAP Coordinator two copies. One copy is mailed to the Regional QA Coordinator and one copy is maintained by the NPAP Coordinator. Audit results that are unacceptable are handled as described in Section 5.6 of SOP-553. 10.3 Intended Use of the Data The minimum quality assurance requirements applicable to SLAMS air monitoring data are specified in Section 2.0 of 40 CFR Part 58, Appendix A. One quality assurance function is the assessment of the quality of the monitoring data. As stated in Appendix A, documentation of the quality of the monitoring data is important to data users so that they can consider the impact of the data quality in their specific applications . Qualitative performance in the audits of analytical proficiency is one of the measures of quality assessment of individual stations, along with other quality assurance measures outlined in 40 CFR Part 58. The NPAP also supplies the EPA Office of Air Quality Planning and Standards (OAQPS) with information on the quality of the SLAMS data base. ------- Section No.: 11 Revision No.: 1 Date: September 15, 1993 Page: 14 of 21 SECTION 11 INTERNAL QUALITY CONTROL CHECKS The adequacy of the internal SOPs and adherence to these SOPs will be periodically reviewed by ManTech's Quality Assurance Officer. All audit devices or materials will be checked prior to each use for cleanliness, operational fitness, and calibration. One point calibrations may be used. If the device fails the preliminary test, a 5-point calibration will be done prior to the audit device's use in the NPAP. Checks on calibrations will be performed using alternative materials from a different manufacturer or lot number. Control charts will be prepared for each critical parameter. Initial values will be assigned by performing 10 analyses over 2 days. After 30 analyses have been performed routinely, the new ranges will be established. These ranges will change with new equipment or new control material. Changes in the acceptable range will be documented with the reason for the change in the appropriate laboratory notebook. Specific internal QC guidelines follow. Device/Material Frequency of Checks QC Check Compressed gas cylinders 6 months Laminar flow element Yearly Dichot Prior to shipping + 3% of certified value Certified vs. NIST-certified LFE + 2°C NIST-certified thermometer + .5 mm H20 NIST-certified LFE + 7 mm Hg NIST-certified barom- eter Gas dilution system Prior to shipping + 3% of the calculated value for values 1 & 3 + 5% of the calculated value for values 2 & 3 + 7% of the calculated value for value 3 ReF Prior to shipping + 2% of a slope determined from 6 years of flow data Pb S04 N03 Yearly + 5% RSD from the average of the determined values ------- Section No. : 11 Revision No.: 1 Date: September 15, 1993 Page: 15 of 21 Device/Material Frequency of Checks QC Check Prior to shipping + 4% or 4 ppb of the calculated concentration Rotameter setting tracked and logged for a gauge setting of 6 psi ------- Section No.: 12 Revision No . : 1 Date: September 15, 1993 Page: 16 of 21 SECTION 12 PERFORMANCE AND SYSTEMS AUDITS 12.1 Performance Audits of On-Site Contractor 12.1.1 Sulfur Dioxide/Nitrogen Dioxide/Carbon Monoxide and Ozone. Four to six audit devices will be selected each quarter (one device every two weeks) to verify ManTech's determined values. The NPAP Coordinator will select the audit devices by requesting that ManTech pull the next device that is ready for shipment to the field. The quality assurance audits will be conducted by the AREAL Standards Laboratory. The audit will consist of running one zero point and three upscale points following the procedures in AREAL/RTP-SOP-QAD-004, Audit Systems Verification Center Operational Procedures. This SOP is located in Appendix D. The results of the quality assurance audits will be forwarded in writing within 5 working days to the NPAP Coordinator for comparison of ManTech's determined values and the EPA determined values. The report to the NPAP Coordinator will also contain an assessment of the condition of the equipment (i.e., external appearance). The audit device will be considered acceptable if an agreement of +5 percent (based on the slope of a linear regression determination) is achieved between ManTech's determined values and the EPA determined values. If the results are unacceptable, the AREAL Standards Laboratory will run the audit a second time to eliminate the possibility of human error. The NPAP Coordinator will forward the results of the comparison along with the actual audit results and the audit device to ManTech for review. ManTech will determine the cause of failures, if any, and report to the NPAP Coordinator in writing within 5 working days the cause and corrective action taken. If ManTech is unable to identify the cause, a meeting will be held between the NPAP Coordinator, the ManTech Manager, and a representative from each laboratory to identify an agreed upon, scientifically sound, probable error source. Minutes of this meeting will be kept and a typed, signed copy will be kept on file by the NPAP Coordinator and the ManTech Manager. 12.1.2 High Volume/PM-10 (SSI). Six ReF devices will be selected each quarter following the procedure described in Section 12.1.1 above. The quality assurance audits will be conducted by the AREAL Standards Laboratory. The audit will consist of verifying ManTech's calibration of the ReF device by using an NIST-certified rootsmeter and following the procedures in AREAL/RTP-SOP-QAD-508. The results of the audit will be given in writing within five working days to the NPAP Coordinator for comparison of ManTech's determined values and the EPA determined values. The ReF device is considered acceptable if the air flow measurement generated by ManTech is within +5 percent of the air flow measurement generated by EPA. If the results are unacceptable, the AREAL Standards Section No.: 12 ------- Revision No.: 1 Date: September 15, 1993 Page: 17 of 21 Laboratory will run the audit a second time to eliminate the possiblity of human error. See Section 12.1.1 above for reporting procedures and corrective action. 12.1.3 Dichot (PM-10). A system consisting of an inclined manometer, an altimeter, a small dial thermometer, and the laminar flow element (LFE) will be selected each quarter as described in Section 12.1.1 above. The audit will be conducted by the AREAL Standards Laboratory using the LFE provided as part of ManTech's system and compared to a system consisting of an electronic manometer, barometer and thermometer provided by the AREAL Standards Laboratory. The flows through the common LFE will be determined by utilizing ManTech's system and the AREAL Standards Laboratory's flow audit system. ManTech's system is considered acceptable if the two air flow measurements determined by their system and the AREAL Standards Laboratory system agree within +5 percent. If the results are unacceptable, the AREAL Standards Laboratory will run the audit a second time to eliminate the possibility of human error. See Section 12.1.1 above for reporting procedures and corrective action. 12.2 Systems Audit of On-Site Contractor Each calendar quarter, a systems audit will be performed by the NPAP Coordinator and a member of the AREAL Standards Laboratory to ensure that ManTech is adhering to the SOPs that cover conducting audits, entering data, distributing data, and maintaining files. However, during the quarter that the independent audit of the total NPAP program is done (see Section 12.3), the NPAP Coordinator will not do a separate systems audit on ManTech. The systems audit will follow a set format based on the information contained in ManTech's SOPs. Attachment 13 of AREAL/RTP-SOP-QAD-553 describes the format. The systems audit will be coordinated with the ManTech Manager. The results of the systems audit will be forwarded in writing within 5 working days to the ManTech Manager for review. ManTech will determine the cause of deficiencies, if any, and report to the NPAP Coordinator within 5 working days the cause and corrective action taken. 12.3 Audit of NPAP The NPAP Coordinator will arrange for an independent yearly systems and performance audit of the total NPAP program by the AREAL Quality Assurance Manager. 12.4 Reports The Chief, QASB will receive copies of all systems and performance audit reports and follow-up reports from ManTech, if any. Reports will be made to OAQPS on an as requested basis. ------- Section No. : 13 Revision No.: 0 Date: July 31, 1991 Page: 18 of 21 SECTION 13 PREVENTIVE MAINTENANCE PROCEDURES All instrumentation used to calibrate or analyze NPAP audit devices or material will be maintained in accordance with the manufacturer's guidelines for routine maintenance of that instrument. Preventive maintenance performed on audit devices is addressed in Sections 11 and 12. ------- Section No. : 14 Revision No.: 0 Date: July 31, 1991 Page: 19 of 21 SECTION 14 SPECIFIC ROUTINE PROCEDURES USED TO SELECT QC SAMPLES AND ASSESS DATA PRECISION, ACCURACY, AND COMPLETENESS Sections discusses data 8 and 12 discuss selection of quality control samples, and Section 10 assessment. ------- Section No. : 15 Revision No.: 0 Date: July 31, 1991 Page: 20 of 21 SECTION 15 CORRECTIVE ACTION When results of the internal quality control checks or the external quality assurance audits exceed the limits specified in the Quality Assurance Project Plan or in the individual SOPs, appropriate action will be instituted by the NPAP Coordinator and/or the ManTech Manager. This corrective action will be documented in the summary reports. ------- Section No.: 16 Revision No.: 0 Date: July 31, 1991 Page: 21 of 21 SECTION 16 QUALITY ASSURANCE REPORTS TO MANAGEMENT The NPAP Coordinator and the ManTech Manager will prepare a comprehensive yearly quality assurance summary report by April 30 for the previous calendar year. The report will include the internal quality control reviews and assessments and will incorporate the quality audit reports (discussed in Sections 12.1 and 12.2) and the latest independently performed audit on the entire NPAP system. ------- APPENDIX A SOP-QAD-553 - Performing the Routine Activities of the AREAL Coordinator of the NPAP ------- APPENDIX B SOP-QAD-54 6 - Computer Data Entry, Report Printing and Maintenance for the NPAP SOP-QAD-548 - Data Validation for Data Bases of the NPAP SOP-QAD-551 - Editing NPAP Data Bases ------- APPENDIX C SOP-QAD-508 - Calibration of ReF Devices for Surveying Performance of Hi-Vol Sampler Flow Rates SOP-QAD-510 - Conducting the Lead NPAP Audit SOP-QAD-512 - Calibration of a Pulsed Fluorescent S02 Analyzer SOP-QAD-52 0 - S02 Audit Device Calibration SOP-QAD-521 - Conducting the Sulfate-Nitrate NPAP Audit SOP-QAD-523 - Analysis of N0/N02/N0x in Gas Cylinders SOP-QAD-542 - N02 Audit Device Quality Assurance Operation Checks SOP-QAD-543 - Quality Assurance Checks of Dichot (PM-10) Audit Devices SOP-QAD-544 - Conducting an Ozone National Performance Audit SOP-QAD-547 - Conducting Performance Audits for Carbon Monoxide SOP-QAD-549 - Analysis of CO in Gas Cylinders with GFC Analysis ------- APPENDIX D SOP-QAD-004 - Audit Systems Verification Center Operational Procedures ------- |