DEVELOPMENT OF AGENCY-WIDE QUALITY CONTROL PROGRAM February 1973 U.S. Environmental Protection Agency Office of Monitoring Systems Quality Assurance Division ------- 26804 DEVELOPMENT OF AGENCY-WIDE QUALITY CONTROL PROGRAM February 1973 U.S. Environmental Protection Agency Office of Monitoring Systems Quality Assurance Division ------- TABLE OF CONTENTS Page INTRODUCTION 1-3 ELEMENTS OF A QUALITY ASSURANCE PROGRAM 4-18 Development and Issuance of Procedures .... 4-11 Sampling-Procedures 6-7 Method Selection Procedures 8-10 Laboratory Procedures 10-11 Intra-laboratory Quality Control 11-15 Quality Measurement and Performance Evaluation 11-14 Personnel Staffing and Training 14-15 Inter-laboratory Quality Control 15-16 Management 16-18 Program Evaluation 17 Certification 17-18 Communication 18 CURRENT STATUS AND NEEDS 19-30 Water 22-24 Pesticides 24-27 Air 27-28 Radiation 28-30 IMPLEMENTATION 31-44 Organizational Plan 32-39 Projects and Schedules 39-44 ------- Page QUALITY CONTROL PROGRAM FOR STATE AND LOCAL AGENCIES 45-70 COMMENTS ON PLAN BY HEADQUARTERS' STAFF, REGIONS, AND NERC's 71-80 ------- 1 Executive Summary DEVELOPMENT OF AN AGENCY-WIDE QUALITY CONTROL PROGRAM Introduction The primary goals of a quality control program are to improve and document the accuracy and validity of environmental measurements. In achieving these goals/ quality control must be imposed on nearly all segments of monitoring activities and should cover personnel/ methods selection/ equipment, and data handling procedures. A total quality control program is composed of four major elements: - Development and issuance of procedures - Intra-laboratory quality control - Inter-laboratory quality control Monitoring program evaluation and certification. All these elements are equally essential to a successful quality control program and must be developed and carried out simultaneously. Current Status The on-going EPA quality control activities are quite varied for the different media and categorical programs. Certain groups exist within EPA that presently provide some support for quality control. These are: Air/ NERC-Research Triangle Park; Water/ NERC-Cincinnati and NERC-Corvallis; Pesticides, NERC-Research Triangle Park/ Perrine; and Radiation, Winchester and NERC-Lasi Vegas. The programs for water, pesticides/ and radiation have been operational for several years/ and we estimate that they are providing approximately 50 percent of the support necessary in their area of monitoring. The program in air was just established in January of this year and is the most critical area. A major deficiency of these programs is that they are almost entirely voluntary, i.e. EPA monitoring activities are not required to participate. The Regional quality control programs are lacking in most areas and are not properly supported. ------- 11 Implementation Plan A quality control program can succeed only if it is given authority and made mandatory for all areas supplying environmental data. The plan calls for central authority and for adherence to prescribed guidelines by the operating monitoring programs. It recommends that a uniform control is implemented over all EPA data and that it must be carried out by the operating programs. Although the program (resources) is limited to EPA, it must in the future be extended to include State and local agencies and contracting laboratories. The organizational elements primarily involved in this program are the Office of Monitoring, quality control groups at the NERC's and the Surveillance and Analysis Divisions in the Regional Offices. Other programs with monitoring interests will primarily have review responsibilities. Nine specific project areas have been identified in this plan which are necessary for achieving the goals of the quality control program. These are: Preparation and Issuance of Procedures and Guideline Manuals Repository of Standard Reference Material (SRM) and Samples (SRS) Operation of Intra-laboratory Quality Control - Operation of Inter-laboratory Quality Control Evaluation of Instruments both for Laboratory and Field Use Quality Control - Site and Laboratory Evaluation Development and Management of Certification Program - Communication and Coordination - Quality Control - Training. ------- Ill The plan calls for immediate action, and projects that the total quality control activity will be, to a large extent, operational by July 1973. Completion dates for the various tasks have been established and they are consistent with this goal. Resouj:ce_ The proposed quality control program will require al- location of resources for three organizational elements: - Office of Monitoring to direct and. coordinate the program - NERC's to implement and support the program - Regions to implement the program. Not all of the resources needed for the quality control program are new requirements. ------- INTRODUCTION Measurements are made daily of pollutants present in the environment and being discharged from a multitude of sources with the resultant data being used in all facets of pollution control. These data are used in investigating the effects of pollution on human health, in determining the types of environmental quality standards that should be promulgated, and in providing evidence for enforcement actions. The measurements are made by numerous agencies and private organizations at a large number of field stations and laboratories. Much of the data are stored in the Environmental Protection Agency's (EPA) data banks where it is used to establish nation-wide appraisals of environ- mental quality, and subsequently, to aid in specific decision-making processes. Regardless of the end uses of the data, it is necessary that the accuracy and validity of the data be known in order that the actions taken can be technically supported. The primary goals of the quality control program are to improve and document the accuracy and validity of environ- mental monitoring measurements. In order to achieve these goals, quality control must be imposed on nearly all phases of monitoring activitiesfrom sampling site selection to data formatting. Quality control should include personnel, methods selection, equipment, and data handling procedures. ------- 2 Quality control applies not only to these components indi- vidually, but also to their interrelationships and functioning as a total system. The benefits derived from an Agency-wide quality con- trol program will amply justify the resources expended. In addition to assuring that the accuracy and validity of the data can be verified, an active quality control program, through the use of uniform procedures, will enhance the utilization of personnel, facilities, and equipment. The data gathered by the various laboratories and field stations will become comparable. Once accepted at the Federal and hopefully the non-Federal level, the promulgation of instru- ment specifications will undoubtedly improve the quality of instruments, laboratory apparatus, and commercially pro- duced reagents. Furthermore, the implementation of a qual- ity control program will increase the overall credibility of the Agency, thereby, strengthening its means for enforcement. A quality control program is a continuing activity. The program must be flexible in responding to changes in monitoring requirements for the various media. Quality control practices are inherent in any monitor- ing or laboratory operation--personnel must be trained, instruments must be calibrated, and uniform operating proce- dures must be developed and followed. Thus, the structuring ------- 3 of an Agency-wide quality control program does not imply that all of the activities to be undertaken are new and require new resources. The program implies that activities must be identified and expanded, and if necessary, developed into an Agency-wide quality control framework which is given emphasis and authority for supporting the Agency's requirements. This report presents an implementation plan for an Agency-wide quality control program. Because of the urgent need to upgrade the quality of EPA's monitoring activities, the plan calls for immediate action and projects that the total quality control activity will be, to a large extent, operational by July 1974. In addition, sections are included that describe the various elements of a quality control program and summarize the existing quality control activities. ------- 4 ELEMENTS OF A QUALITY ASSURANCE PROGRAM A total quality control program is composed of four major elements: Development and issuance of procedures. Intra-laboratory quality control program. Inter-laboratory quality control program. Monitoring program evaluation and certification. All these elements are equally essential to a successful quality control program and must be developed and carried out simultaneously. This section of the report describes each element and its subelements (Table 1) necessary to provide a basis for understanding the proposed plan for a total quality assurance program. Development and Issuance of Procedures A basic requirement of a quality control program is a series of manuals describing the procedures to be followed during the course of sampling, analysis, and data handling. It is the use of such prescribed procedures that provide a uniform approach in the various monitoring programs and which allow the evaluation of the validity of data produced. Procedures are needed for all of the facets of a monitoring programfrom criteria to be used in locating stations to the formats to be used in reporting the data. The required procedures may be grouped into three categoriessampling, ------- TABLE I Elements of Quality Control DEVELOPMENT AND ISSUANCE OF PROCEDURES Sampling Procedures Site selection criteria Station design criteria Sampling criteria Calibration procedures Methods Selection Procedures Methods selection criteria Compendium of recommended methods Equivalency determination Laboratory Procedures INTRA-LABORATORY QUALITY CONTROL Staffing control Training programs Use of correct procedures Performance evaluation INTER-LABORATORY QUALITY CONTROL Cross-check sample program Methods evaluation studies MONITORING PROGRAM EVALUATION AND CERTIFICATION Periodic review/evaluation Certification ------- 6 methods selection, and laboratory procedures. A brief description of each of these follows: Sampling Procedures (a) Site selection The selection of sampling/ monitoring sites is the responsibility of each monitoring program, however, guidelines must be established that govern the specific placement of monitors or the exact location where the sample is to be taken. Such rules or guidelines are necessary to ensure that the measurements made or the samples taken are representative and comparable. For example, in the case of air pollution monitoring, criteria must be established specifying the allowable nearness of inlet probes to buildings or, in the case of water monitoring, the depth at which samples are taken. (b) Station or instrument Measurements may be severely affected by the type and configuration of the facil- ities used in collecting the sample. For example, in some situations, variability in voltage, temperature, and humidity can influence the measurements. Consequently, operational parameters must be specified and controlled. Similarly, design characteristics must specify the types of monitors and special equipment which must be accommodated. Adherence to station design criteria will provide optimum use of ------- 7 equipment, ease of operation, minimum maintenance, and reduced data losses. (c) Sampling and preservation criteria -- Procedures must be established that govern the manner in which samples are collected and handled. These procedures should include the following: (1) Use of equipment and materials for collecting, preserving, and transporting the samples. (2) The length of sampling periods. (3) The types of accompanying information needed. In order that meaningful data are obtained, a sound statis- tical basis for determining the frequency and duration of sampling/monitoring must be used in the program design. Such procedures must be provided in the manuals. Specific procedures must also be established that document the chain- of-custody of samples needed for enforcement actions and for samples taken at or near points of suspected violations. (d) Calibration procedures To ensure that data generated by automatic or integrating field sampler-analyzers are valid, procedures for routine field calibration of these instruments must be specified. The calibration fre- quency, as well as the procedures used, should be identified and described. ------- 8 Method Selection Procedures (a) Compendium of recommended methods Many different methods are available for measuring pollutants in environmental media. Some methods have been promulgated by groups within EPA. Other methods have been published by groups such as the American Public Health Association (water), the American Society for Testing and Materials (air/water), and the Intersociety Committee (air). The Environmental Protection Agency has the statutory responsi- bility to promulgate environmental, source, and effluent standards which includes a method of collection and analysis. To ensure the use of standard, reference, or equivalent methods throughout the Agency, an approved com- pendium of standard, reference, or equivalent methods must be developed for all pollutants in all media and published for all monitoring activities. Method selection procedures based on standardization activities - The cornerstone of any quality control program is the uniform use of acceptable methods and procedures. The need for stand- ard methods of sampling, analysis, and data handling is readily apparent, and the use of such methods between cooperating lab- oratories removes methodology as a variable in the comparison or joint use of data. The use of standard or equivalent methods is particularly important when ------- 9 laboratories are providing data to central data banks such as STORET and NADIS. The routine use of non-standard or non- approved methods within an agency raises serious doubts as to the validity of the data reported. Methods selected must be pre- sented in a concise, easy-to- follow format. The format will indicate: principle, appli- cability, range, sensitivity, interferences, accuracy and precision, apparatus, calibra- tions , reagents, procedures for sampling and analysis, calcula- tions, references or bibli- ographies, and special precautions necessary for handling and/or disposing of hazardous materials and reagents. Based on past experience, generally applicable criteria for the selection of acceptable methods for environmental monitoring should be recommended by a panel of experts represent- ing the ultimate users. The measurement technique will exhibit sufficient precision and accuracy to meet the data needs of EPA. Furthermore, the necessary equipment should be available at many laboratories and be adequate for the routine examination of a large number of samples. In addition, the validity of the technique should be tested. (b) Method selection criteria All EPA labora- tories will be encouraged to utilize the same methods for measuring pollutants. The methods which may be selected ------- 10 will be described in an EPA-approved compendium. Where more than one method is available, the selection criteria will be based on the applicability of the method, its comparability with methods used in other laboratories, and its ability to meet requirements of the data user. Specific guidelines for selection of alternative methods and procedures should be included in all manuals. (c) Equivalency determination criteria Individual monitoring activities may decide to utilize methods that differ from the prescribed EPA reference method. It is necessary to establish guidelines to ascertain that the results of the particular monitoring method employed is equivalent to EPA-approved methods and, hence, valid data can be obtained. Laboratory Procedures Bias and variability in data may result from variation in the general design and construction of facili- ties. Differences in the equipment, apparatus, and reagents used by laboratories can also contribute to both the bias and variability of data which decreases the reliability and comparability of results between laboratories. The functional design of a laboratorythe use and maintenance of equipment, standardization of reagents, temperature, and humidityinfluences the reliability of data. Therefore, ------- 11 it is extremely important that quality control procedures concerning standard laboratory requirements and practices be developed, promulgated, and followed. These guide- lines should cover all facets of routine laboratory operations and maintenance of equipment and apparatus. Intra-laboratory Quality Control Quality Measurement and Performance Evaluation To maintain a high level of competence in daily activities, quality control must be implemented in the field and at the bench using a system of checks to determine the accuracy and precision of results and the performance of monitors and analysts. Intra-laboratory quality control is a continuing in-house activity to ensure the output of valid data. The specific objectives of the program are to devise and implement procedures that: Measure and control the precision of procedures and instruments. Measure and control the accuracy of analytical results. Ensure data output is computer compatible. Present data in proper format. Document performance of instru- ments and analysts. Document training needs. Identify weak methodology and consequently research needs. ------- 12 An intra-laboratory quality control program employs several important tools/techniques: Standard Reference Materials (SRM) are substances which qualify as absolute quantities against which other like substances can be calibrated. The SRM, typically produced by organizations like the National Bureau of Standards, is used to prepare standard reference samples (SRS) for lab- oratory application. Standard Reference Samples (SRS) are preparations or known amounts of standard reference materials added to an actual environmental sample which has been previously analyzed. The amount of the sub- stance found in the sample is a "true" indication of the accuracy of the method for a given measure- ment. The use of the standard reference samples measures the extent of interferences which cannot be obviated. Quality control charts should be an integral part of a quality control program. Figure 1, which shows the summation of the dif- ferences squared for replicate samples, is one form of a quality control showing trends. Additional control charts are recommended where standard deviations are (d = V«/«7£ ) for use on a daily basis to establish rapidly if an analysis is out of control on a given day. Once precision and accuracy data are available on the method and the analyst (from the use of SRM/SRS), systematic daily checks are necessary to ensure that valid ------- 13 QUALITY CONTROL CHARTS = The summation of the difference squared for replicate samples, NO APPARENT TREND lid2) SAMPLE SET NO. ANALYSIS IN CONTROL Kd2) SAMPLE SET NO. ANALYSIS OUT OF CONTROL UPPER LIMIT APPARENT TREND EVIDENT ltd2) SAMPLE SET NO. ANALYSIS OUT OF CONTROL LOWER LIMIT I(d2) SAMPLE SET NO. ANALYSIS OUT OF CONTROL UPPER LIMIT FIG. 1 ------- 14 data are being generated. From these daily precision and accuracy data, quality control charts can be constructed and maintained to determine when the method used is producing valid data, when the data are questionable, or when a trend is detected which must be investigated and corrected. Several techniques are available for constructing quality control charts and plotting subsequent data. The two techniques currently used by EPA are the Shewhart technique and the Cumulative- Summation technique. For both techniques, precision control charts are constructed from duplicate sample analyses, and accuracy control charts are con- structed from spiked sample analysis. Personnel Staffing and Training An important part of intra-laboratory quality control is proper staffing and the continuous training of personnel. Guidelines need to be established and made available to all laboratories concerning the qualifications (training and experience) that are necessary to perform the various tasks. For example, the operation of a gas chromatograph will typically require a professional scientist with considerable experience. On the other hand, many routine functions can be adequately performed by entrance grade technicians. Staffing guidelines will assist the laboratories in selecting and deploying personnel. ------- 15 The rapidly changing methodology of environmental monitor- ing requires that the personnel undergo periodic training both formal and on-the-job. Training courses in method- ology as well as quality control practices will need to be incorporated into EPA's formal training program and some requirements will need to be established for all monitoring personnel to participate. Inter-laboratory Quality Control An inter-laboratory quality control program serves to select and evaluate methods, characterize their precision and accuracy, and provide data for evaluating both lab- oratory and analyst performance. This aspect of quality control is referred to as cross-check sample studies or methods evaluation studies. Specific objectives of this program are to: Measure the precision of reproducibility of methods of analysis within various programs. Identify interference in different sampling environments. Measure the precision and accuracy of results between laboratories. Provide a mechanism for evaluation and/or certification of laboratories and analysts. Detect weak, improper, or impractical methodology. ------- 16 Detect training needs and upgrade laboratory performance. Assist laboratories or programs in obtaining new resources. Participating laboratories should be provided with standard reference samples, instructions, and data forms necessary to test methods under certain prescribed condi- tions. The results should be submitted to a coordinating laboratory where they are statistically evaluated to determine the accuracy and precision of the method. More- over, they are also evaluated to determine the general applicability of the method and to rate the performance of the laboratories and analysts. This type of activity involving laboratories in evalua- ting methods is necessary to provide a sound statistical model. Single laboratory tests introduce both method and laboratory bias, and the results obtained may have little relationship to the "true" reliability. All operating quality control programs have used an inter-laboratory quality control program as a mechanism for methods standardization and selection. Management Certain aspects of implementing and maintaining a quality control program require actions and decisions which affect total operation or reflect position/policy of the ------- 17 Agency. These include program evaluation, certification, and communications. Program evaluation Involves periodic review and assessment of all quality control activities as necessary to determine their proper functioning, effective- ness, and reliability. Evaluations are necessary at least quarterly and should be comprehensive enough to include all field and laboratory activities related to monitoring. Special attention should be given to field procedures and calibration, performance of laboratories and analysts, and adequacy of manuals/methods, training, etc. This evaluation should provide the basis for a quarterly quality control status/needs report to Regional Administrators, Center Directors, and Headquarter's elements. Certification This is an important subelement of the management activity which involves formal approval and/or endorsement of acceptable performance by a laboratory or analyst. Certification is directly relatable to the inter-laboratory quality control. Certification implies that performance of analysts and laboratories is acceptable with respect to an established population average. It is important to note that certification does, in both fact and concept, imply endorsement by EPA and, as such, can have serious impact/repercussions on our enforcement actions and ------- 18 our contracting systems. Since EPA's experience in con- ducting certification programs is limited/ it is necessary to review other programs existing outside EPA and select the best available information to develop guidelines for EPA. Certification programs may first be applied to EPA laboratories and then expanded to include States and others on a voluntary basis. A reasonable and valid alternate approach to a certification program would be to. require all laboratory/ field activities supplying monitoring data to EPA to oper- ate within certain prescribed guidelines for intra-laboratory quality control and participate in EPA's inter-laboratory quality control program. This alternative will provide documented evidence of laboratory/analyst performance and, hence, the validity of the data. Communication For a quality control program to succeed, a necessary subelement is an adequate information exchange program. The following types of activities are included: Reports from meetings of a quality assurance committee. Reports from central files of information on request/ use of SRM and SRS. Reports of quality control conference. Quality Control Newsletter. ------- 19 CURRENT STATUS AND NEEDS Many elements of quality control are currently being practiced to some degree at EPA and other environmental monitoring programs. The quality control programs for the different media and categorical programs are quite varied ranging from rather comprehensive activities in some areas to essentially non-existent activities in others. What- ever the case, all of the on-going activities suffer from the lack of established requirements resulting in essen- tially voluntary and partial participation by individual programs and laboratories. The problem is further com- pounded by the decentralization of monitoring activities. The effort thus far, then, has been uncoordinated, fragmented, and operated with minimal resources. The results have been unsatisfactory. Much of the data have been of invalid or questionable quality. For example, more than 20 percent of the air quality data gathered from the State and local agencies for insertion into the National Air Quality Data Bank was not useable. In addition to loss of data, this represents considerable waste of both manpower and resources. Since different methods, equipment, or proce- dures have been used to monitor separate locations, various programs have produced data that are not comparable. In many instances, analyses are being made without proper ------- 20 reagents, without calibration, and by unqualified analysts. This has resulted, for the most part, in a data base that is of questionable validity and, therefore, limited utility. In developing a plan, it is necessary to assess the present status of our quality control programs, evaluate their needs, and provide solutions. Each of the current programs in quality control has a record of accomplishments and capabilities as well as cer- tain shortcomings. Although quality control programs are under way for air, water, radiation, and pesticide monitor- ing, they all are notably lacking in several areas. The most critical deficiencies are lack of established procedures for sampling methodology, field calibration, equivalency determination, and the lack of performance evaluation and certification. Comparable quality control programs for noise or solid waste monitoring are non-existent. The current capabilities for each media's quality control program are the subject of Figure 2. While the estimated degrees of accomplishment are subjective, they indicate relative strengths of the current programs. It is estimated that the quality control programs for water, pesticide, and radia- tion monitoring are about 50 percent complete. However, the quality control activity for.air monitoring which has just begun is inadequate. The major accomplishments and ------- STATUS OF EXISTING QUALITY CONTROL ACTIVITIES JUNE'72 (AS PERCENT TOWARD FULL ACTIVITY) MEDIA 0% 50% 100% AIR WATER PESTICIDES SOLID WASTES RADIATION NOISE FIG. 2 ------- 22 shortcomings of each quality control program are high- lighted below. Water The quality control activities for monitoring water and wastewater are centered at the Analyt- ical Quality Control Laboratory (AQCL) at the National Environmental Research Center (NERC) in Cincinnati. This program has been in existence for more than five years. However, the limited resources available have not allowed for the development of a comprehensive program. The major accomplishments and deficiencies of the water quality control program are: Accomplishments (1) Compendium of Methods (latest issuance in 1971) (2) Intra-laboratory Quality Control Manual (1971) (3) Inter-laboratory Quality Control (maintenance of SRS repository for cross-check sample studies and methods evaluations) (4) Laboratory procedures guide- lines (1971) (5) Quality Control Newsletter (issued periodically) Deficiencies (1) The methods manual does not include instrumental, bio- logical, and microbiological measurements (external methods exist) ------- 23 (2) Technical assistance to field laboratories is provided on an ad hoc basis. (3) The sources of SRM are not well established. (4) The production of SRS fall short of demand. (5) Reports of methods evaluated are not timely. (6) Laboratory manual needs to be updated. (7) The scope of the present program is not geared to meet the expanding needs of the Regions. (8) Participation by laboratories is voluntary. In addition to the activities of the Analytical Quality Control Laboratory, the Water Supply Research Laboratory, NERC-Cincinnati, conducts limited quality con- trol services for State laboratories concerned with drinking water standards. This laboratory, started in 1953 as part of a training program, supplies reference sam- ples and uses and recommends use of the 13th Edition of Standard Methods for Water/Wastewater Analyses. In addition, it prepares a training manual which includes some quality control procedures. Moreover, the Water Supply Research Laboratory participates in a certification program for State bacteriological and chemistry laboratories. ------- 24 Additionally, Consolidated Laboratory Services, NERC-Corvallis, has conducted an intra-laboratory quality control program for water quality data since its inception and has improved to the point of being a near model system. This program has recently extended this activity to its satellite laboratories, and NERC-Corvallis requires that quality control be practiced by all contractors assisting it in acquisition of water quality data. This Laboratory has also developed a Sampling Handling and Verification System (SHAVE) which assures continued integ- rity of the sample from collection to analysis to data reporting. A simplified description is portrayed in Figure 3. It should be noted that NERC-Corvallis works closely with AQCL-Cincinnati by using its methods, supporting lake eutrophication studies, and participating in other special projects. Pesticides The Primate and Pesticide Effects Laboratory under NERC-RTP conducts an inter-laboratory quality control program for pesticides monitoring. This program is almost totally dedicated to supporting the Com- munity Pesticides Studies and the State Services Labora- tories. Under this program, quality control services are available to 18 laboratories which are under contract to EPA. ------- 25 SAMPLE HANDLING AND VERIFICATION SYSTEM SOURCE COLLECTION CODE LOG ANALYSIS DATA (VERIFICATION) STORET USERS FIG. 3 ------- 26 The complexity of pesticide analysis requires extensive quality control activities. Hence, the scope of the services provided by the Primate and Pesticide Effects Laboratory is extensive. Standard reference materials and samples and standardized materials for gas chroma- tographic columns are provided as well as technical assist- ance, training, repair, and maintenance of both gas chromatographs and mass spectrometers. An inter-agency agreement (December 22, 1971) established a repository of standard reference materials from which the Food and Drug Administration (FDA) and State and local enforcement authorities may draw standards. The major accomplishments and deficiencies of the program are highlighted below. Accomplishments (1) Compendium of Methods (2) Inter-laboratory quality control (3) Intra-laboratory quality control (4) Methods evaluation program (5) Technical assistance and training (6) Standard Reference Materials Deficiencies (1) Methods are not recommended for enforcement actions and are not official EPA methods. ------- 27 (2) The cross-check sample program is inadequate to meet needs of the Regions. (3) No quality control manuals are available. (4) The methods evaluation program should be expanded to Regional participation. (5) Technical assistance and train- ing must be expanded and redirected to include Regional needs. (6) Standard reference materials are not analyzed and maintained. Air A formal quality control program for air monitoring activities was initiated in January 1972. It is located in the Quality Assurance and Environmental Monitor- ing Laboratory, NERC-RTP. The slow start in this area is due primarily to lack of resources and from difficulties encountered in developing standard reference materials for gases as well as delivery systems to generate standard reference samples. Although some of these initial diffi- culties have now been resolved, and a quality control program is under way, much remains to be done. The implementation of quality control for air monitoring is the most critical area. The current status of this program is briefly described below. ------- 28 Accomplishments (1) Reference methods have been promulgated with Air Quality and Source Performance Standards. (2) Standard reference materials have been developed for sulfur dioxide, nitrogen dioxide, carbon monoxide, and some hydrocarbons. (3) Methods for suspended particulates, sulfur dioxide, and carbon monoxide have been evaluated and tested. Deficiencies (1) No compendium of EPA-approved methods has been developed. (2) No delivery systems have been developed to generate standard reference samples. (3) No quality control manuals or guidelines have been developed. (4) No cross-check sample program has been initiated. (5) Lack of standard gas mixtures. (6) Lack of source and emission test procedures. Radiation The Analytical Quality Control Service of the Office of Radiation Programs located at Winchester, Massachusetts, which had been providing an inter-laboratory quality control program for radiation measurements since ------- 29 1962, has been relocated to NERC-Las Vegas. It is currently known as the Office of Quality Assurance-Radiation. The present program consists of conducting cross-check sample studies for EPA contract laboratories and utility companies. Standard reference samples for radionuclides are provided to other Government agencies and private organizations as requested. The Office of Quality Assurance-Radiation col- laborates with groups such as the American Society for Testing and Materials, the American Public Health Association, and the Association of Official Analytical Chemists in test- ing radiochemical methods of analysis. Technical assistance in developing quality control programs is provided to EPA and other laboratories on an ad hoc basis. In addition to developing an operational intra- laboratory quality control program, this program conducts quality control activities with States, the International Atomic Energy Agency, the World Health Organization, and the U.S. Atomic Energy Commission laboratories. A functional system for collecting, handling, and analyzing large numbers of samples has been developed. Moreover, data processing and reporting have been computerized. The computer program generates quality control charts of measurement accuracy and precision. The program also conducts a field sampling course and provides for the certification of participants. ------- 30 The current status of this activity is briefly summarized below. Accomplishments (1) A compendium of radiochemical methods of analysis has been published. (2) A cross-check sample studies program has been developed. (3) A methods evaluation program has been developed. Deficiencies (1) A compendium of EPA-approved methods has not been promulgated. (2) A quality control manual has not been developed. (3) Training and technical assist- ance to support Regional needs is inadequate. (4) Methods evaluation studies are not comprehensive. ------- 31 IMPLEMENTATI ON In the development of an Agency-wide quality control strategy, two overriding issues must be considered and resolved: What should be the scope of coverage? (EPA only or include State and local and other programs) What should be the extent of control? Environmental monitoring data are produced by many organizations, both governmental and private. The Environ- mental Protection Agency, State and local agencies, and other Federal agencies are involved in environmental moni- toring. The State and local efforts are especially exten- sive and the data are routinely placed into EPA's data systems. In the private sector, contracting laboratories and private industry are involved in the measurement and analysis of pollution. Because of EPA's reliance on outside organizations for the production of environmental data, the quality assurance program should extend outside EPA. Only in this manner can EPA support the data it uses and, consequently, the actions that are taken. Therefore, at a minimum, the qual- ity assurance program should embrace EPA's own programs, the State and local laboratories, and the contracting laboratories conducting work for EPA. The quality control ------- 32 strategy presented herein is based upon the development of the tools and procedures for a quality control program which can easily be extended from a regional base to State, local, and contract laboratories. A quality control program is doomed to failure if it is not given authority and is not adhered to by the organ- izational elements that it encompasses. Usefulness of the procedures, manuals, and training programs will be limited if the operating programs are not required to incorporate them into their monitoring activities. The successful implementation of the quality assurance program has two requisites. First, the program should be emphasized so that it becomes widely accepted. Secondly, the program should have sufficient authority to support the control activity necessary to produce valid data. This section of the report presents an organizational plan and a listing of projects, tasks, and schedules. Organizational Plan (1) The program must be authoritative/ compulsory. (2) The program must be centrally directed and coordinated, yet allow for individual program implementation. ------- 33 (3) At least initially, the program must utilize existing facilities and areas of competence. (4) The program must provide for centrali- zation of all elements of the program that lend themselves to such organization, as in provision of SRM and prepara- tion of SRS. (5) The program must provide for routine exchange of quality control information. (6) The program must be responsive to all operating monitoring programs. (7) The program must extend outside EPA, both in terms of building the program and providing service. Accordingly, four levels of activities and respon- sibilities are necessary: (1) Program Management, planning, and coordination -- In order to ensure the development of a uniform quality control program that meets the needs of all monitoring activities, it is necessary to centralize general management, planning, and coordination. In this way, each program receives equitable emphasis and is evaluated by a single standard. (2) Program review and concurrence Since monitoring activities within EPA are carried out independ- ently by the various organizational elements (i.e., Regions, ------- 34 Program Offices, Enforcement, and Research), the quality control program should be developed and operated in concert with these offices. The responsibilities of each monitor- ing activity, therefore, is to identify its needs and review and concur on proposed Agency-wide quality control programs. (3) Technical procedures, development, and technical support A sound quality control program can only be developed and operated if the field programs are supported by central quality control groups. It is these groups, located at the NERC's, that will assist in program development, develop and test the procedures before routine field application, provide necessary materials, and render assistance as needed. (4) Program implementation and operation Quality control must be implemented by each of the operating monitoring programs within the guidelines and directives provided. This implies that monitoring programs in each of the Regions and those in Program Offices, Research, and Enforcement will conduct their own quality control programs. To ensure that these programs operate within central directives, close liaison must be maintained between indi- dual monitoring activities and the central quality assurance activity. ------- 35 Specific program functions and responsibilities have been determined. They are designed to provide the Agency with an effective quality control program for all moni- toring activities. The proposed program takes advantage of currently available expertise in order to implement a quality control program responsive to the Agency's needs. Office of Monitoring; The Office of Monitoring will assume overall responsibility for the direction and coordination of both the development and implementation of the quality control program. It will continually review and assess the program's progress and efficacy. These respon- sibilities will be carried out in concert with the appro- priate elements of the NERC's. Specific tasks require the Office of Monitoring to: Issue directives and guidelines for quality control to operating monitoring programs. Prepare and review program plans. Ensure that .technical assistance is provided to Regions and other monitoring programs. Develop and direct a laboratory certification program. Evaluate the effectiveness of continuing quality control programs and provide biannual status reports. Coordinate requisite training programs. ------- 36 Maintain a rapid information exchange system that will notify appropriate organiza- tions of important changes and events relating to quality control. Establish an internal quality assurance committee necessary for program coordination. Provide a central focal point within EPA on all matters pertaining to quality control of monitoring activities. National Environmental Research Centers; The technical support elements of the quality control program will be located at the National Environmental Research Centers. The quality control groups at these Centers will be responsible for providing the necessary tools for operating monitoring programs as required. Responsibilities for particular environmental media and categories are specified as follows: Air NERC-RTP Pesticides NERC-RTP (Primate and Pesticide Effects Laboratory) Radiation NERC-Las Vegas Water NERC-Cincinnati Solid Waste and Noise responsibilities for the quality control activities have not yet been assigned. ------- 37 Responsibilities of these quality control groups will include the following tasks: Develop and provide standard reference materials/samples. Direct and coordinate intra- and inter-laboratory quality control programs. Develop quality control procedures. Provide technical assistance in quality control. Regions: Each Regional Office will implement the quality control program for all appropriate monitoring activities through an officially designated full-time quality control coordinator. Specific Regional tasks include: Centralization of Regional authority for quality control activities. Operation of intra-laboratory quality control programs to assure adherence to EPA procedures and guidelines for all monitoring activities. Provision of technical assist- ance to State and local agencies. « Participation in methods selection and in cross-check sample studies. The review of each program to provide a quarterly status/ needs report to the Office of Monitoring. ------- 38 Participation in the develop- ment and implementation of quality control training activities. Provision of technical support and assistance as required for certification programs. Office of Categorical Programs; Develop require- ments and review quality control plans for the monitoring of hazardous materials, radiation, pesticides, solid waste, and noise. Office of Air and Water Programs: Develop require- ments and review plans for quality control activities of air and water monitoring, both ambient and source. The Office of Air and Water Programs will be responsible for the general management of data acquisition and storage and will review and concur in this aspect of the quality control program. Office of Enforcement and General Counsel; Review and concur in all quality control activities which support enforcement and case preparation. The Office of Enforcement and General Counsel will establish case preparation monitor- ing guidelines and provide manuals to appropriate program managers and operational elements. Office for Regional Liaison; Review and concur in all quality control activities which relate to Regional responsibilities. The Office for Regional Liaison will also ------- 39 actively participate in all communication and coordination activities. Projects and Schedules The implementation of an Agency-wide quality control program will require the accomplishment of a series of projects. In total, nine separate projects with numerous tasks have been identified. The accomplishment of these projects and tasks will require the efforts of many EPA organizational elementsmost notably the Regional Surveil- lance and Analysis Divisions, the quality control groups at the NERC's, and the Office of Monitoring. It must be remembered, however, that each monitoring program will need to expend resources for quality control. A brief description of each project, including the objectives they fulfill, is included below- Project 1: Preparation and Issuance of Procedures and Guidelines Manuals The result of this pro- ject will be "loose-leaf" manuals that will include uniform procedures for the entire monitoring activity. The proce- dures are to be used by all environmental monitoring activi- ties within and outside EPA. The project calls for continual expansion and updating of these manuals as more and better information becomes available. Primarily, the manuals and inserts will be developed by the quality control groups at ------- 40 the NERC's. In some instances, the issuance of procedures and guidelines may involve only review and compilation of existing materials. For others, it may require extensive surveys of actual field and laboratory investigations. Project 2: Provide a Repository of Standard Reference Materials and Samples The result of this pro- ject will be stockpiles of standard reference materials and samples for testing and evaluating all aspects of monitor- ing. This will include ambient, emission, and effluent monitoring as well as fuel additive and pesticide registra- tion. The tasks to be accomplished include identification of sources of standard reference materials and development of standard reference samples including appropriate delivery systems. In some cases, such as hazardous materials, special procedures will be developed for handling, storage, and use. Whenever possible, standard reference materials will be obtained from outside organizations, such as the National Bureau of Standards, and will be produced internally only when not available elsewhere. Project 3: Operation of Intra-laboratory Quality Control This is a continuing project which will be carried out by all on-going monitoring programs. This effort includes resources spent calibrating instrumentation and equipment, conducting laboratory performance checks, ------- 41 maintaining daily control charts, etc. This project also includes technical assistance rendered to other organiza- tions to improve or upgrade their programs. Project 4; Operation of Inter-laboratory Quality Control This project is a continuing activity involving the quality control centers and the operating monitoring programs. Its results will be documentation of laboratory, method, and analyst performance. Performance will be continually evaluated by use of computerized control charts which will show performance levels and data quality. Project 5; Laboratory and Field Evaluation of Instruments -- The result of this project will be a series of reports and papers that will provide a scientific basis for selection of pollution monitors. For the most part, this involves review of operational data from monitors which are currently in the field or are used in laboratories. The data will be evaluated and organized into a user performance report which will be made available to all prospective users. This activity will be primarily carried on by the NERC's. Project 6; Quality Control - Site and Laboratory Evaluation -- This project provides external checks on the performance of on-going monitoring activities. One task is the periodic evaluation of EPA's monitoring programs by the quality control groups of the NERC's and the Office of ------- 42 Monitoring personnel. A second task is the evaluation of State and local monitoring programs by the quality control elements of the Surveillance and Analysis Divisions. These checks may range from site inspections and review to actual field calibrations using mobile units. Project 7; Development and Management of a Certification Program _The result of this project will be to establish and maintain qualified laboratories and analysts for EPA and other monitoring activities. Certification programs will be the culmination of all the quality control activities. Performance levels will be established against which the performance of each laboratory will be judged, and upon approval, certificates will be issued and renewed periodically. The actual mechanisms to be followed and the extent of the programs are yet to be investigated. Alterna- tives must be weighed as to whether the program should be mandatory and whether it should be extended to external lab- oratories. The frequency with which the program should be updated must also be considered. Project 8: Communication and Coordination This project provides the overall management of the quality control program and ensures continuity of all operations. Some of the tasks under this project are concerned with information exchange. The newsletter, the information system, ------- 43 and the conferences will provide a mechanism whereby the people engaged in quality control can share their experiences and resolve problems. Quality control activities within EPA must depend upon the expertise of outside organizations. A task under this project will ensure that the necessary con- tacts are maintained. The development of a successful quality control program involves numerous policy and major technical decisions. It is proposed under this project that an internal quality assurance committee be established. Project 9; Quality Control - Training This project will serve to coordinate and develop training courses and lectures concerning the use of approved methods and quality control practices. The intent of this project will be to work with the formal EPA training programs to ensure that the proper courses are developed and given. No sepa- rate training program is to be developed. In order that all monitoring professionals and technicians may be accommodated, an attempt will be made to make the requisite courses available through selected colleges or junior colleges. Courses on methodology and quality control practices might be incorporated into the chemistry programs of selected schools (primarily evening schools) via the grant mechanism. An alternative is to provide on-the-job training at the Regional or NERC laboratories. An important aspect of this ------- 44 project will be to develop mechanisms for ensuring that all appropriate monitoring personnel take advantage of these courses. ------- 45 QUALITY CONTROL PROGRAM FOR STATE AND LOCAL AGENCIES INTRODUCTION In June of this year, the Office of Monitoring pre- pared and distributed for review a proposal for an Agency- wide quality control program. Comments have now been received from most of the Program and Regional Offices and their suggestions have been incorporated into the plan. The plan discusses and sets forth the organizational strat- egy to be employed and delineates the projects and tasks that must be done. A further analysis is required to focus on the scope of coverage and the extent of control that the EPA quality control program should have. This paper addresses these points and makes recommendations regarding these issues. Thus, the "plan," together with this issue paper, constitutes the strategy for an Agency-wide quality control program. The primary issue is whether the quality control pro- gram should be limited to the internal EPA monitoring activities or be extended to cover State and local agencies, contracting laboratories, EPA grant recipients, and the manner in which it should be implemented. What are the alternatives and their costs? The other major consideration is the extent of control needed to ensure reliable and defensible data. What are the costs associated with the various alternative "control levels?" ------- 46 Should EPA's quality assurance program be extended to external monitoring programs? Although stated as an unresolved issue, the question of extending the EPA quality assurance program to external laboratories, namely State and local agencies, is rather academic. The very nature in which the monitoring functions and responsibilities are carried out by the State and local agencies and the Federal pollution control agencies dictate the need for a broadly based quality assurance program encompassing all these activities. This conclusion is based on the following: (1) EPA depends upon State and local data. (2) EPA has and will continue stipulating requirements for State and local monitoring programs^ (3) EPA as well as State and local data are routinely entered into joint information systems for common useage. The resources available for monitoring activities have always been limited, requiring that the maximum use be made of the "monitoring dollar." Wherever possible, duplication of effort must be avoided and the available data must be used to serve a multitude of purposes and by the various agencies or organizations requiring the data. The data needs by EPA are many and varied ranging from evaluating compliance with standards to determining the health effects that occur at different levels of pollution. The current ------- 47 or projected EPA monitoring capabilities fall far short of the level of effort that is needed. In the future, as in the past, EPA will need to depend to a large extent upon monitoring data produced by the State and local agencies. In fact, the current level of effort by the State and local agencies for monitoring the more common pollutants surpass those of EPA by a factor of nearly 10 in the case of air monitoring and by a factor of more than five for water quality monitoring. Full use should be made of these data. Before EPA can place full reliance upon these data, their accuracy and precision must be known and fall within acceptable limits. The use of data generated by numerous agencies and laboratories for a common purpose (e.g., delin- eation of nation-wide trends) required that these data be of acceptable quality and be comparable. Only through an active quality assurance program can EPA verify the State and local data it uses and upgrade the data where necessary. In many instances, EPA now places monitoring require- ments upon State and local agencies by specifying the pol- lutants that need to be measured, the number of stations that must be installed, types of instrumentation, frequency of sampling, etc. It follows that the requirements for data quality should be made part of these requirements. ------- 48 Many agencies and organizations, governmental and private, use environmental data from the EPA environmental data systems, e.g., SAROAD and NEDS. It is extremely desirable that all of the data entered into these systems be, if not of the same quality, of acceptable and known accuracy and precision. A good quality assurance program can meet this goal. RECOMMENDATION: Because of the above reasons, it is recommended that State and local monitoring programs be covered under the EPA quality assurance program. How should the EPA quality assurance program be operated with respect to the State and locaT monitoring programs? There are several basic options available to EPA in the formulation of the strategy for a quality control program for State and local monitoring activities. They differ in the requirements that EPA wishes to place upon the State and local programs. Each option must be very carefully con- sidered in terms of the improvements in data that would result and in the resources that EPA must expend. Since the number of operating agencies and their laboratories is large, a systematic and coordinated effort is required to implement a successful program. Furthermore, the acceptance of the program by the individual State and local agencies is a necessity because without their willing participation the ------- 49 quality assurance program will fail. A well designed and operated quality assurance program will greatly enhance our future working relationships with the State and local monitoring activities. A poorly operated program can have the opposite effect and must be avoided. In all, three approaches have been identified. They are: (1) mandatory participation; (2) voluntary participation; and, (3) voluntary participation with some requirements as to the data. The first option would require the participation of all State and local monitoring activities that impinge on the mandated EPA responsibilities. Under this option, EPA would require the States to adhere to EPA monitoring guidelines, use EPA designated standard reference materials and samples, and participate in inter-laboratory testing and evaluation. These requirements could be levied upon the States through regulations concerning national environmental standards. The major advantages and disadvantages of this approach are: Advantages: Full participation by all monitoring programs is assured. The "weak" programs can be readily identified and corrective actions can be instituted. Data of better quality would ensue. ------- 50 Uniformity of data would be increased through use of common and approved techniques nationwide. Disadvantages: Another EPA requirement for States could "worsen" relationships. The required EPA resources for immediate implementation of the program are nearly prohib- itive. (The average Region would require about four to five qualified individuals to admin- ister the program. The volume of standard reference materials and samples required would increase tenfold.) The legal issues concerning requirements will delay start of the program. The second option places no requirements on the external programs. The Environmental Protection Agency would make the various aspects of the quality control program available to the State and local agencies. For example, EPA would distribute methods and quality control guidelines and invite the various laboratories to participate in the inter- laboratory testing program. The Environmental Protection Agency would provide standard reference materials and samples as requested, if available. The advantages and disadvantages of this approach are: ------- 51 Advantages; Program can be started immediately and expanded as resources and expertise increase. (For water quality, radiation, and pesticide measurements, this service is now being provided through the NERC's.) An additional requirement is not placed on the States. This may result in a program that may be more acceptable. Disadvantages: It is probable that the programs/ laboratories in most need of quality control would not volun- tarily participate. Data of questionable quality would still be submitted to EPA1s information systems. The implementation of the system may never reach full coverage. The third option would combine certain facets of the previous two approaches. The Environmental Protection Agency would require that the validity of all of the monitoring data submitted to EPA be documented. The Environmental Protection Agency would issue guidelines concerning the information that must accompany the data. Such information could include sample history, analytical methodology/ calibration, and performance checks. In addition, EPA would continue to provide monitoring methods and guidelines, ------- 52 conduct inter-laboratory testing (on a voluntary basis), and make standard reference materials and standard reference samples available. Certification may or may not be part of this program. The pros and cons of this option are: Advantages: The validity of all data submitted to EPA is known. The program can be implemented on a more timely basis and expanded as resources become available. The "weak" programs can be iden- tified and assistance may be offered. The latitude for quality assurance given to the States should make the program more widely acceptable. A less rigid approach to quality control may better serve to advance the state-of-the-art. The quality assurance program will be self regulating. Disadvantages; At least initially, the overall data quality may not be substantially improved. EPA will need to be concerned with a much greater number of methods and procedures. The preparation and issuance of data validity requirements will delay the start of the program. ------- 53 Ideally, the first option (i.e., mandatory State par- ticipation) will provide EPA with the best data base and the most complete coverage. Furthermore, since the States would be required to adhere to EPA guidelines, the pro- liferation of non-uniform methods and procedures would be minimized. This approach, however, does not appear possible at the existing resource levels. Once the requirements are placed upon the States, EPA must be in a position to admin- ister the program, which would include both technical assistance and official evaluation functions. Both of these tasks will require substantial resources by the Regions and the quality control support groups at the NERC's. The second option (voluntary participation) is the easiest for EPA to undertake. In fact, the on-going programs in water, radiation, and pesticides presently operate in this manner. Past experience has shown that these programs are well accepted by the more qualified external programs/ laboratories. However, the major weakness of this approach is that the laboratories in most need of quality control do not generally participate in a voluntary program. It is these programs/laboratories that are of primary concern to the EPA quality assurance program. The third option eliminates the major disadvantages of the other two approaches--the large resource requirements of ------- 54 the mandatory program and the limited coverage of the volun- tary program. The submission of "data quality" information by the State and local agencies will provide a basis upon which to judge the validity of the data EPA receives. The States, although invited to take part in the various ele- ments of the EPA quality assurance program, are not required to participate. They can implement and carry on their own quality control. This approach does not place an over- whelming resource requirement upon EPA immediately, yet pro- vides for a way to review data validity. An important aspect of this option is that it provides a self-regulating mechanism. The need for the State and local agencies to provide certain quality control statistics will encourage them to upgrade their monitoring activities through increased quality control practices. Development of a quality assurance program for State and local agencies The proposed strategy calls for both the stipulation of certain quality control information that must accompany all the data submitted to EPA and the provision of quality con- > trol services to the State and local agencies. Each of these are briefly discussed here as are the various options that are available with respect to implementation of the program. ------- 55 "Quality Control" Information; The data cur- rently submitted to EPA, for the most part, consist only of the results with a minimum of supporting information. This supporting information is limited generally to site location, sampling time and duration, method of analysis used (sometimes), and the agency reporting the data. On the basis of this information, it is impossible to evaluate the validity of the data. With the help of additional statistics (that can be compiled by each agency/laboratory and submitted with the data), EPA could evaluate the data prior to acceptance into the information systems used by EPA. The supporting information needed would consist of: (1) Sample handling and verification methods. (2) Details of the analytical method used. (3) The type and source of standard reference materials and standard reference samples used. (4) Frequency of use of quality control or check samples. (5) Last time the program participated in an inter-laboratory testing program. In addition, to aid in evaluating the validity of data, "baseline" information on each laboratory's capabil- ities would also be compiled and kept up to date. This ------- 56 would contain information on each physical facility, equip- ment, personnel, and the like. This "baseline" information together with the accompanying quality control statistics would provide the basis for evaluating the validity of the data submitted to EPA. There does not appear to be any alternatives to this approach. Therefore, the basic decision is how best to develop and implement a program to gather this information. The implementation strategy must consider the following: (1) The quality control data (including "base- line" facts) should be compatible with EPA's data systems. (The checking of data validity is to be computerized.) (2) The information required must be kept at an absolute minimum. (3) The submission of the accompanying infor- mation must be made an integral part of the data flow from State/local agencies to EPA. Implementing this portion of the program will require the accomplishment of a series of tasks. These tasks together with suggested organizational responsibilities are given below: (1) Compilation of "baseline" information. This information will serve to provide an initial rating of the capability of an individual monitoring program/laboratory ------- 57 that submits data. (In most likelihood, the capabilities will be pollutant-oriented.) It is proposed that a question- naire be developed by the Office of Monitoring (and the NERC's) in concert with the Regional Offices and other appropriate EPA elements (including the information systems). The information would be collected, compiled, evaluated by the Regional Offices, and forwarded to the Office of Monitor- ing through the appropriate information systems. The evaluation or "rating" scheme is to be devised by the pre- viously named EPA organizations. An annual update of this information is envisioned. The following sequence is proposed: Develop a questionnaire and rating scheme. Compile baseline information. Baseline information included in data systems (to be determined). (2) Submission of "quality control" information. This information, to be routinely supplied with all data, will provide a statistically sound basis for evaluating the validity of submitted data. It is proposed that the list of required "quality control" information be developed by the Office of Monitoring (with the NERC's), the Regional Offices, and other appropriate EPA elements. The participation of ------- 58 information systems (SAROAD, NEDS, etc.) is essential, since the routine evaluation or "flagging" of invalid or questionable data is to be computerized. Once developed, the quality control information will become an integral part of all data submitted to EPA. The manner in which the quality control information is to be collected and incorporated into the system will need to be investigated. Quality Control Services; According to the recommended option, EPA will provide certain quality con- trol services to the State and local monitoring programs/ laboratories. The alternative approaches of the EPA involve- ment are best discussed through the consideration of the nine project areas considered in the "plan." The organiza- tional responsibilities for these tasks are given at the end of this discussion. (1) Preparation and issuance of procedures and guidelines. The manuals prepared and issued for EPA monitoring activities (see plan, page 4) will be applicable to State and local agencies. Whether or not included under EPA quality control, the State/local agencies would take advantage of these manuals. The only extra cost (minimal) to EPA would be on publication and distribution. There are, however, two points that should be considered: ------- 59 (a) State/local agency involvement in their preparation and review. (b) Whether reordering of priorities is necessary. In order that these manuals reflect the needs of the States, take advantage of their expertise (in some areas, certain State/local laboratories are very advanced), and receive wider acceptance, a mechanism for their partic- ipation should be developed. This may be either in the form of scheduled advisory group meetings or purely on an ad hoc basis. The preparation of some guidelines may, in fact, be "contracted" to a given State or States. Since the pri- orities for control are dictated by common legislation, the priorities for methods or procedures should not vary between EPA and the States. (2) Provision of standard reference materials and samples. Standard reference materials and samples are needed to calibrate monitoring and laboratory instrumentation, to evaluate proposed new methods, and to maintain a con- tinuous check on the validity of the data that are produced. The success of an inter-laboratory quality assurance program depends on the availability and use of high quality standard reference materials and standard reference samples. Cur- rently, there is little or no control over the use of standard ------- 60 reference materials and standard reference samples (type, quality, frequency of use, etc.). The State/local programs, as well as other monitoring activities, obtain these mate- rials from a wide variety of sourcesEPA laboratories, National Bureau of Standards, private laboratories, and in some cases, the reference samples are being prepared by the user laboratory itself. Increased control on the production and use of standard reference materials and standard refer- ence samples is necessary if the State/local data are to be upgraded and their validity assessed. There are several options available to EPA in making these materials available to State/local programs. These are: (a) EPA purchases the standard refer- ence materials from appropriate sources such as the National Bureau of Standards and private organizations, prepares standard reference samples, and makes these materials avail- able to State/local agencies at no cost. (These repositories could be maintained by the quality control groups at the NERC's, Regional Offices, or both.) (b) EPA identifies and supports (and approves) an external organization(s), such as the National Bureau of Standards and private suppliers, as a central repository for standard reference materials and standard reference samples. (The actual production cost of the ------- 61 materials would be borne by the users. EPA would continue the support of the development of these materials. Although the National Bureau of Standards would be a major source, private suppliers could be designated as approved sources of given materials.) (c) Combination of options 1 and 2. EPA provides priority standard reference materials and stand- ard reference samples as resources allow and identifies com- mercial sources where these and other materials may be obtained. (d) EPA requires that all agencies sub- mitting data to EPA document the source(s) and quality of the standard reference materials/standard reference samples they use. (The agencies would be free to select their own source of these materials.) The first option assures the maximum EPA con- trol over all monitoring data which it receives from the State/local programs. It also assures a much wider use of these materials. However, with present funding restrictions, the logistics and the cost of providing this service may seriously limit the scope of coverage. Option 2 above would reduce the cost to EPA. This alternative requires that external organizations have the necessary expertise and willingness to develop and maintain such a repository. ------- 62 Moreover, a service "charge to the user" could reduce volun- tary participation and delay the response to State/local needs. The third option (e.g., the provision of all standard reference materials and standard reference samples to State and local monitoring programs) would not place an over- whelming requirement upon EPA. It would allow EPA to develop and provide reference materials on a priority basis. The identification of private and other governmental suppliers of reference materials will allow EPA to maintain some con- trol over their use. This option, although requiring a minimum of EPA resources, will not provide performance up- grading in the various laboratories. If it results in proliferation of standard reference materials/standard reference samples of unknown purity, the overall quality of data will not be substantially improved. RECOMMENDATION: For the reasons cited above, option 3 is recommended to provide standard reference materials and standard reference samples. (3) Operation of intra-laboratory quality control. This activity is the responsibility of each program and laboratory. The need to submit quality control infor- mation with the data will encourage the States and local agencies to maintain an adequate quality control program ------- 63 within their laboratories. Guidelines would be provided through Project 1. There would be no extra cost to EPA. (4) Operation of inter-laboratory quality control. An inter-laboratory quality control program provides an evaluation of the performance of individual laboratories in conducting a given analysis or series of analyses. This activity also serves as the mechanism for evaluating new methods. An inter-laboratory quality control program (cross- check sample program) serves the interests of participating laboratories by identification of their shortcomings. It also serves the interests of the data recipients by documenting the validity of the data. As part of the recommended strategy, inter- laboratory quality control would be provided to the State/ local agencies on a voluntary or service basis. This proce- dure, in fact, is currently being followed in the water, pesticide, and radiation monitoring activities. Under this approach, the quality control groups at the NERC's would conduct inter-laboratory tests (issue test procedures and samples and evaluate the results). The State/local agencies would be informed of these tests and invited to participate (by the Regional Offices). This approach raises the question of extent of State participation in a voluntary program. Experience has shown that the more qualified laboratories ------- 64 participate in a voluntary program of this type whereas laboratories/programs most in need of performance improve- ment do not participate. Under the proposed program, this should not happen. The requirement for the State and local monitoring programs to provide quality control infor- mation as part of their data submission to EPA should serve as a self-regulating mechanism. Furthermore, it encourages all programs to participate in inter-laboratory quality control. (5) Laboratory and field evaluation of instruments. This project area is intended to provide guide- lines for selecting and evaluating instruments. The extension of the quality control program to the State/local agencies would not require an additional effort on the part of EPA. The experience of State/local agencies with methods and instruments can provide valuable inputs to guideline formulations. (6) Quality control - Site and laboratory evaluation. This project provides external checks on the per- formance of on-going monitoring activities. These checks may range from site inspections and review to actual field calibration using mobile units. This activity is a necessary consequence of an on-going inter-laboratory program uncover- ing weak or marginal performance of State/local monitoring ------- 65 programs. Also, this project provides assistance, if requested, by State/local programs. This project area will require considerable activity on the part of EPA which would be carried out by the Regional Offices with the assistance of the NERC's as required. (7) Development and management of laboratory certification program. A most important issue in the develop- ment of a strategy for extending the EPA quality control activity to State/local monitoring programs is the question of certification of State and local laboratories, analysts, etc. A certification program requires the establishment of performance levels against which the performance of each laboratory will be judged in terms of facilities, equipment, personnel, participation in inter-laboratory quality control programs, etc. and, upon approval, the issuance and periodic updating of certificates. "Certification" is the culmination of an inter-laboratory quality control program and carries with it an official approval label. The advantages and dis- advantages of a formal certification program are discussed below: Advantages; The evaluation of data validity is improved. Laboratory performance is upgraded. (The need for increased performance ------- 66 levels can be used as a tool for obtaining resources, high level of personnel, etc.) Disadvantages: The current state-of-the-art is not sufficient to provide a strong scientific basis for setting performance levels. (The setting of performance levels assumes the existence of model laboratories against which all others would be judged. Statistical models may or may not be a sufficient basis.) Official certification may "worsen" relationships with the States. (Another EPA require- ment on the States.) The operation of a certification program will require substantial resources that can best be used elsewhere. A certification program may be a deterrent in continual development and improvement for the "certified" programs/laboratories. RECOMMENDATION: For the present, a formal certification program is not proposed. It must be understood, however, that many of the activities proposed as part of the quality control program for States, i.e., data requirements, inter-laboratory tests, etc., provide a mechanism for continuous evaluation of program/laboratory performance. In effect, the "grading" of laboratories and their data ------- 67 (which will be carried out) constitutes a form of certifica- tion, lacking only the formal approval of laboratory performance. Once the quality control program is under way, the question of certification should be reexamined. In the interim, EPA should continue to provide assistance to those States that are developing their own certification programs. (8) Communication and coordination. The extension of quality control services to State/local programs- will require a substantial coordination and management effort by EPA. Information on laboratory activities/capabilities will need to be collected and maintained. Problem areas will need to be identified and addressed. Continual updating of the program will also be necessary. These tasks will primarily fall upon the Regional Offices. (9) Training. Training of quality control specialists and monitoring personnel is an important part of the quality control program. The number of State/local personnel that will need training is large and requires sub- stantial resources. The following approach is recommended: (a) Workshops on quality control manage- ment aspects, proposed for EPA programs, should also include the appropriate State personnel. ------- 68 (2) Specific courses on quality control techniques, i.e., use of quality control charts, split samples, etc., should be provided to State/local personnel through the Regions. (3X Courses in the use of analytical methods should continue to be part of EPA's training program (see plan). RESOURCE REQUIREMENTS The extension of the quality control program to the State and local agencies will require EPA to provide resources for: (1) The initial development and the routine main- tenance and evaluation of quality control information which will accompany all data submitted to EPA; and, (2) the provision of quality control services to the State and local monitoring programs. Whereas the first of these activities is independent of the development of the EPA quality control program, the provision of quality control services is closely allied to it. Any limitations placed upon the EPA program will reduce the services accordingly. Thus, for example, the lack of support for the development and production of standard reference materials and standard reference samples will reduce the ------- 69 number of these materials available for distribution to State and local programs. The development and maintenance of quality control information will require resources for: (1) expansion of the information systems (NADIS, STORET, etc.) to accommodate this information; and, (2) the routine handling and evalua- tion of the information. The cost of reprogramming will need to be determined. The routine evaluation will require one to two individuals to be assigned to each information system for dealing with quality control problems. The provision of quality control services to the State/ local agencies should be primarily the responsibility of the Regional Offices with technical support provided by the quality control groups at the NERC's. The main thrust of this program, as pointed out earlier, will be to supply standard reference materials and standard reference samples, conduct inter-laboratory tests, provide technical assistance, conduct periodic quality control checks, and make training courses available to the appropriate State and local personnel. IMPLEMENTATION SUMMARY The increased dependence by EPA on the State/local monitoring data requires that the quality control program be implemented as soon as possible. In most likelihood, ------- 70 the provision of quality control services will be insuf- ficient at firstbecause of funding limitations and the need for EPA to develop the basic quality control tools. The following major milestones are proposed: Determine the mechanisms to be used for collecting and evaluating quality control information. Issue a list of quality control information needed. Initial "grading" of monitoring programs. States informed of quality control services. Program under way. ------- 71 Comments on "Development of an Agency-wide Quality Control Program" INTRODUCTION Earlier this year, the Office of Monitoring distri- buted for review a proposal for an Agency-wide quality control program. Comments were requested from nearly 30 EPA organizational segments including the NERC's, Regions, and Program Offices. In all, some 20 responses were received covering all of the Regional and many of the other offices. This paper summarizes their comments and the major changes made in the document. First, the major comments and the resulting changes are discussed. This is followed by a summary listing the individual comments submitted. DISCUSSION There was a general consensus throughout EPA that a strong Agency-wide quality control program is urgently needed. Although most of the reviewers agreed with the implementation strategy, there were some different points of view regarding the manner in which the program should be carried out. The technical aspects of the program, i.e., the description of the elements of a quality control program were well received by the reviewers. Some objections were raised with respect to the appraisal of current status and capabilities of the on-going programs. Several reviewers felt that the report was overly critical of some programs. The major issues raised by the review are given below. (1) Varying degrees of quality control; The proposal included a recommendation that the degree of quality control should be varied on the basis of the end uses of data, e.g., data to be used for enforcement would be produced under stricter quality control than, say, data for trend assessment. Several reviewers addressed this issue and strongly recommended that all EPA data be subjected to same control. Their recommendations were based on the argument that EPA shou-ld only produce data of high quality, that the end uses of data are not frequently known, and that strict quality control measures are needed for trend and background monitoring where the concentrations measured are much lower than for enforce- ment. On the basis of their arguments, the proposal was ------- 72 changed and now recommends a uniform degree of quality control for all data. (2) Regional role in the development of guidelines; Comments submitted by the Regional Offices addressed the need for Regional input in the development of the proce- dures and guidelines. In their view a much greater Regional input was called for than outlined in the proposal. It was not intended to exclude the Regions from this very important task. The proposal was modified accordingly. (3) Quality control over sampling: Some reviewers felt that the sampling portion of monitoring was not given sufficient attention in the proposed quality control program. The general consensus was that it is extremely important. Changes were made in the document to give increased emphasis on quality control relating to sampling. (4) Compulsory quality control program; The proposal calls for a very strong and compulsory program. Comments regarding this issue were quite varied. Although several reviewers supported a, compulsory program, some felt that the proposed program was too authoritative and under too much "strict Washington control." If EPA is to establish and implement an adequate program, it must be adhered to by all monitoring programs. No basic change was made. In addition to the above, numerous changes were made in the document in response to the comments. SUMMARY OF COMMENTS REGION I (1) Does not agree that the degree of quality control should be varied according to uses of data. (2) Emphasizes that program must be compulsory- (3) Emphasizes that program must cover all aspects of monitoring. Feels that program is too much laboratory oriented. (4) Does not think that the program can be operational by July 1973. ------- 73 REGION II REGION III REGION IV (1) Feels that the question of how, who, and how many will direct and operate the program is not suf- ficiently described. (2) Comments that changes in design and construction are not applicable to existing facilities. (3) Recommends the use of additional control charts. (4) Recommends that a full-time quality control coordinator be designated in each Region. (5) Recommends a certification program for State and local programs/ laboratories. (6) Recommends that a separate quality control group be set up for marine related monitoring. (1) Agrees with the proposed approach. (2) Recommends extending the program to State and local agencies. (3) The resources programmed for train- ing are underestimated. Should be about two man years and $60,000 per Region. (4) Raises questions concerning the detailed implementation of the program. (1) Agrees that a strong quality con- trol program is needed. (2) Suggests that resources be made available to the Regions for prep- aration of methods manuals and the provision of standard reference materials and standard reference samples. ------- 74 REGION V REGION VI REGION VII (3) Recommends the allocation of 10 man years to Region IV for carrying out a certification program. (4) Does not agree with the statement that much of the existing data are questionable. (5) Recommends that every EPA reference method be interlab- oratory tested. (6) Recommends that EPA do not depend upon external organization for quality control. (1) Agrees with the approach, particu- larly with the recommendation that the program be compulsory. (2) Does not agree that the degree of quality control can be varied. All data should be of same quality. (3) Raises a question on how many laboratories can one quality con- trol office handle. (1) Questions the need for a certifica- tion program. (2) Thinks the schedule of projects and tasks are well laid out. (1) Agrees with the concept of an Agency-wide quality control program. (2) Thinks the program is under too much "strict Washington control." (3) All data must be subjected to same level of quality control. (4) Instructions on sampling site selection and calibration frequency. ------- 75 REGION VIII REGION IX REGION X (5) Laboratory certification is a must. (6) Resources and identity of the program are needed now. (1) Agrees with the concept of the program. (2) Does not agree with the estimated resource requirements for the Regions. Recommends that such additional resources must be made available to the Regions. (3) Agrees that the program should cover State and local efforts, but that its scope should not be disproportionate to the available resources. (1) Regions should be involved in the development of guidelines, train- ing programs, etc. (1) Agrees with the proposed program. (2) Thinks guidelines for sampling may result in being too restrictive, (3) Feels that a formal laboratory certification program is several years away in Region X. (4) The issuance of standard reference materials and standard reference samples should be through Regional coordinators. (5) Suggests that Regions be involved in procedure and guideline development. (6) Recommends the inclusion of Regional coordinators in any Quality Control Intra-Agency Committee. ------- 76 Office of Radiation Programs Office of Categorical Programs (1) Agrees to the need to establish an Agency-wide quality control program. (2) Recommends that the existing radiation quality control group (activity) at Winchester be more recognized and that EPA quality control programs for radiation measurements be under the Office of Radiation Program's control at Winchester. (3) Suggests that quality control for the sampling end of monitor- ing be emphasized more. (4) Thinks that the report is too critical of on-going quality control activities. (5) Suggests an inconsistency between recommendations regarding "an authoritative/compulsory program" and the statement that quality control is the responsibility of the monitoring program which it serves. (6) Suggests that the proposed mile- stones are far too ambitious. (1) States that the report is unclear regarding the development of standard reference materials to support the pesticide registration program. (2) Recommends that guidelines on sampling consider statistical basis for design of sampling programs and schedules. (3) States that there are a number of unresolved issues within EPA to adequately evaluate the proposal with respect to inter-laboratory ------- 77 Office of Water Programs NERC-RTP NERC-LV NERC-Cin quality control, coordination, certification, and training. (4) States that the proposal does not consider the quality control activities of Program laboratories. Suggests that these operate independently with the Agency- wide program as a point of reference. (5) States that the proposed resources are excessive. (1) Comments that the program is realistic. (2) Recommends that quality assurance must also include data storage and validation. (1) Agrees with the proposal. (2) Does not favor a certification program at this time. (1) Agrees with the proposal. (2) Suggests that too much emphasis is placed on standard methods. (3) Suggests the use of duplicate samples for intfa-laboratory quality control. (4) Recommends the shifting of radia- tion quality control from Winchester to NERC-LV. (1) States that the program cannot be centrally imposed, but must be accepted by all elements. (2) The evaluation of current status does not give sufficient credit to existing quality control programs in water and pesticide monitoring. ------- 78 NERC-Cin (3) The suggested sampling guidelines (Cont) are too restrictive. (4) The specific roles of the Office of Monitoring and the NERC's are not adequately defined. (5) The project/task descriptions for water are not accurately described. ------- 79 Development and Review of the Plan Quality Control Workshop An Agency-wide quality control workshop was held on February 16 and 17 to lay the groundwork for the Agency's quality control program. The workshop was attended by approximately 50 individuals represent- ing all Regions, NERC's, and most of the Program Offices. Review of the Quality Control Plan Comments on the plan were submitted by the following individuals: Region I Region II Region III Region IV Region V Region VI Region VII Region VIII Region IX Region X Office of Radiation Programs - E.V. Fitzpatrick,Director Surveillance & Analysis Division R.T. Dewling, Director Surveillance & Analysis Division J.G. Gardner, Deputy Director Surveillance & Analysis Division J.H. Finger, Chief Chemical Services Branch R.J. Bowden, Director Surveillance & Analysis Division E.R. Lozano, Deputy Director Surveillance & Analysis Division G.L. Fish, Director Surveillance & Analysis Division J.A. Green, Administrator B.D. Clark, Director Surveillance & Analysis Division G.L. O'Neal, Director Surveillance & Analysis Division - W.D. Rowe, Deputy Assistant Administrator ------- Office of Categorical Programs Office of Water Programs NERC-RTP NERC-LV NERC-Cin NERC-Corv Office of General Counsel 80 G.R. Cornstock, Program and Management Operations K.M. Mackenthun, Director Applied Technology Division S. Hochheiser, Chief Quality Control Branch M.W. Carter, Director D.G. Ballinger, Director Analytical Quality Control Laboratory D.F. Krawcyzk, Director Consolidated Laboratories B. Anderson, Staff Chemist . GOVERNMENT PRINTING OFFICE: 1974 546-317/297 ------- POINTS OF CONTACT FOR QUALITY CONTROL PROGRAM Media Program Management Contact Mr. Guntis Ozolins Director or Mr. Thomas W. Stanley Chief Location Quality Assurance Division EPA Headquarters Quality Control Branch EPA Headquarters Phone No. 202/755-0646 202/426-2382 Water Mr. Dwight G. Ballinger Director Methods Development and Quality Assurance Research Laboratory NERC-Cincinnati 513/684-2925 Air Dr. S. David Shearer Director Quality Assurance and Environmental Monitoring Laboratory NERC-RTP 919/549-2106 Pesticides Dr. William F. Durham Director Pesticides and Toxic Substances Effects Laboratory NERC-RTP 919/549-2655 Radiation Mr. Richard E. Jaquish Chief Technical Support Laboratory NERC-Las Vegas 702/736-2969 Attachment ------- |