United States Office of Air Quality Environmental Protection Planning and Standards Agency Research Tnangle Pa'k NC 2771 1 EPA-450/3-84-006 March 1984 Air Methods 6 and 7 Quality Assurance and Quality Control Revisions — Summary of Comments and Responses \ ------- EPA-450/3-84-006 Methods 6 and 7 Quality Assurance and Quality Control Revisions — Summary of Comments and Responses Emission Standards and Engineering Division U.S. ENVIRONMENTAL PROTECTION AGENCY Office of Air and Radiation Office of Air Quality Planning and Standards Research Triangle Park, North Carolina 27711 March 1984 ------- This report has been reviewed by the Emission Standards and Engineering Division of the Office of Air Quality Planning and Standards, EPA, and approved for publication. Mention of trade names or commercial products is not intended to constitute endorsement or recommendation for use. Copies of this report are available through the Library Services Office (MD-35), U.S. Environmental Protection Agency, Research Triangle Park, N.C. 27711, or from National Technical Information Services, 5285 Port Royal Road, Springfield, Virginia 22161. ------- TABLE OF CONTENTS Page Chapter 1. INTRODUCTION 1 Chapter 2. SUMMARY OF CHANGES SINCE PROPOSAL 2 Chapter 3. SUMMARY OF PUBLIC COMMENTS AND RESPONSES 3 Table 1. LIST OF COMMENTERS 19 m ------- CHAPTER 1 INTRODUCTION On March 30, 1983, the U.S. Environmental Protection Agency (EPA) published "Quality Assurance and Quality Control Revisions to Methods 6 and 7," in the Federal Register (48 FR 13388). These revisions will require source testers to analyze audit samples when making compliance determi- nations in order to assess their analytical accuracy. These revisions were proposed under the authority of Sections 111, 114, and 301(a) of the Clean Air Act, as amended. Public comments were solicited at the time of proposal. An invitation to request a public hearing was issued to provide interested persons the opportunity for oral presentation of data, views, or arguments concerning the proposed revisions, but no person desired to make an oral presentation. The public comment period was from March 30, 1983, to June 13, 1983. Letters concerning issues relative to the proposed revisions were received from 15 commenters. A detailed discussion of these comments and responses is summarized in this document. The summary of comments and responses serves as the basis for the revisions which have been made between proposal and promulgation. ------- CHAPTER 2 SUMMARY OF CHANGES SINCE PROPOSAL Method 6 1. Section 3.3.6. The audit sample set will not include a known sample. The tester is requested to notify the quality assurance (QA) office or the appropriate enforcement agency at least 30 days prior to the test date to allow sufficient time for sample del i very- 2. Section 4.4. EPA's Source Branch is listed as one source of known quality control samples. A single audit per 30-day period for the same enforcement agency is allowed, and the proposed provision for quarterly audits when samples are analyzed frequently has been dropped. The analyst's name is to be included when reporting audit and compliance results, If a reanalysis is required, the initial and reanalysis audit values are to be reported. 3. Section 6.4. The equation for calculating the relative error for the QA audit samples is included. Method 7 The changes for Method 7 are the same as listed for Method 6 except for Section 6.4. ------- CHAPTER 3 SUMMARY OF COMMENTS AND RESPONSES QUALITY ASSURANCE AND QUALITY CONTROL REVISIONS TO METHODS 6 AND 7 1. D-l, D-5, D-14 Comment: We support the proposed additions which appear to be both adequate and reasonable. Although they may require additional work and expense, the resultant benefits should justify it. The proposed measures will provide greater uniformity and confidence to the analytical results, and the improved scientific data base should also enhance decision making during the development of future standards. Response: No response necessary. 2. D-4 Comment: Some regulating agencies already require audit analysis for compliance testing, monitor certification, and for data to be used for background studies. We suggest that audit analysis be required for compliance determinations and that the Administrator have the option of requiring audits for other tests. Response: Audit analysis as a part of monitor certification and for data to be used for background studies is an allowable option, and in some cases, may be required by the Administrator. The Agency, however, does not have the authority to make this mandatory in all cases. 3. D-4 Comment: It is not clear what criteria should be used for determining if audit samples are required only on a quarterly basis. We have had the experience where testing contractors have assured us ------- that they participate in and pass the quarterly audits yet still do not analyze the audit samples within the specified range on the first try. We do not desire to be overly burdensome on the QA aspect, but we feel that there is a definite need for the audit samples, or a well-defined method of determining the analytical accuracy of the laboratory. Response: The quarterly auditing allowance for analyses performed on a frequent basis has been changed. The revised requirement is to analyze audit samples at least once per month. This discourages the use of 3 month-old analytical reagents as allowed in the quarterly audit and also eliminates the need to perform an audit with each set of compliance samples if samples are analyzed on a. frequent basis. 4. D-ll Comment: In general, we agree with the need for QA in source emission data that are obtained to demonstrate compliance with emission standards. However, we feel the requirements should be based upon sound statistical analysis of available data, and this does not appear to be the case with this proposal. The likely impact of the proposed rule can be determined by examining the 1977 through 1980 Environmental Protection Agency (EPA) inter!aboratory data. If the proposed 5 percent criterion for S02 were used, anywhere from 8 to 42 percent of the laboratories would have had unacceptable performance. If the proposed 10 percent were used for NOX, between 27 and 70 percent of the laboratories would have failed the test. There is also a strong indication that samples of lower concentration were more likely to be found unacceptable than those of higher concentration. ------- It is suggested that, initially, the EPA audit samples be used to identify sources of error in the Methods 6 and 7 analyses and calculations. Once these are identified, a guideline document should be prepared to assist analysts performing these analyses. Then, accuracy criteria for the two methods can be developed and implemented. Response: The rationale and data used to support the proposed rule are listed in the background information report that was announced in the preamble to the regulations. The report is based upon previous EPA inter!aboratory audit surveys. The survey data were studied to determine the sources of tester errors and to find some indications of laboratory capabilities. The sources of correctable errors (e.g., analytical biases, calculation, and reporting errors) were identified and corrected, and the data were reanalyzed. The resultant figures showed that 95 percent of the laboratory tests for S02 were within 5 percent, and 86 percent of the tests for NOX were within 10 percent of the true values. The audit samples will not be prepared in low concentrations. 5. D-6 Comment: The proposed acceptance levels, within 5 percent for S02 and 10 percent for N02, are more restrictive than the respective 7 and 20 percent levels recommended by the EPA QA Handbook for Air Pollution Measurement Systems (Redbook). The Redbook values are based on interlaboratory performance surveys for reference Methods 6 and 7. If a more recent interlaboratory survey has been performed and the ------- results justify the stricter tolerances, the data should be presented for review. If the data are not available, the stricter tolerances are not justifiabl e. Response: See the response to Comment 4. The background information report shows that by eliminating correctable mistakes, much of the error margin is eliminated. 6. D-12 Comment: Notwithstanding brief mention of a background document, the proposal does not indicate the existence of a data base that demonstrates the need for or reasonableness of additional QA requirements. The given justification is that "the current regulation includes only limited QA requirements and as a" result of this proposed regulation, the quality of compliance data will improve." Regulation should not be founded on such a generalized, unsupported statement but upon statistically sound and representative data. Response: See the response to Comment 4. The analytical errors revealed in the inter!aboratory survey indicate the need to have an auditing requirement. 7. 0-2 Comment: The rulemaking ignores the logistic problems that will be created with a one-on-one audit requirement, especially for control agencies conducting numerous tests. a. How will EPA guarantee delivery of the audit samples? b. Will EPA ensure that the concentrations are within the analytical range of the samples? ------- c. Can it ensure a different audit concentration for each request? d. Can EPA guarantee fast delivery of audit samples? Our regulations require analysis within 48 hours of sample collection. Response: EPA will be able to respond if notified soon after a decision has been made to perform a compliance test. The quality assurance revisions will have instructions to notify the EPA Quality Assurance Management Office 30 days prior to the actual test. Since the compliance sample concentrations have to fall within the calibration range of the standards for Method 7 and within the titration volume range for Method 6, the audit sample concentrations will be within these analytical ranges. Coordination within the auditing program will assure that sources receive different audits on different requests. 8. D-7 Comment: Laboratories performing Methods 6 and 7 analyses should be required to participate in the semiannual EPA inter!aboratory surveys to evaluate their analytical procedures and personnel. Perhaps, certain standards of performance on their work should be specified. The survey offers enough time for submittal of results to allow freedom in scheduling the analysis to coincide with compliance tests. Response: The semiannual interlaboratory survey will be modified to place primary emphasis on the new auditing program. Standards of performance are specified in the new requirements which evaluate analytical procedures and personnel. ------- 9. D-9 Comment: The proposed rule apparently ignores the ongoing EPA interlaboratory audit program for Methods 6 and 7 which is done semi annually. Since this program is underway, we feel that it is redundant and unnecessary to add a second program requiring more frequent audits. More frequent audit analyses should only be required in those cases where the ongoing interlaboratory program points out a deficiency in a laboratory's quality control program. Response: See the response to Comment 8. Participation in the interlaboratory auditing program gave laboratories an indication of their relative analytical performance but did not identify or eliminate inaccuracies experienced on an actual test due to using different or inexperienced personnel, mistakes in the standards preparation, calculations, reporting, etc. There is a need to audit each compliance test whenever there is a change in the analyst, analytical reagents, or analytical system to maximize the accuracy of the determined results. A more frequent auditing program than the current EPA interlaboratory survey is needed. 10. D-6 Comment: The EPA Redbook recommends the appropriate enforcement agency should provide the audit samples after notification of intent to demonstrate compliance by the source. This recommendation allows the Agency the option to distribute the audit samples to the sources of unproven analytical abilities and minimize the workload of both the Agency and the source. 8 ------- If QA procedures are to be added, they should allow for the continued discretion of the Agency in requiring the audit in contrast to forcing the source to perform the audit regardless of the known analytical abilities of that source. Response: All testers should be audited to detect possible errors in their analytical technique. Even laboratories of proven ability occasionally experience oversights that affect the results of compliance analyses. 11. D-13 Comment: We suggest that a voluntary audit program be developed that allows interested parties to run compliance samples on a predetermined schedule. Any interested laboratory could be audited by the Agency and be supplied with written confirmation of the audit results. In this manner an owner or operator could know the qualifications of a contractor prior to the issuance of a contract, the Agency would be assured of the laboratory results, and the documented record of the QA program and compliance testing will be preserved. Response: See the response to Comment 9. A voluntary auditing program that would make known the qualifications of contractors for the source operator to evaluate would be weakened and its importance lessened by the number of laboratories that opt to abstain from participation. 12. D-10 Comment: We have very serious reservations regarding both the necessity and practicability for analyzing audit samples at least once each quarter for all compliance determinations. In view of the ------- finite resources of both EPA and industry, it is strongly recommended. that sample auditing for critical or suspect compliance determinations only be considered. Response: See the responses to Comments 5 and 9. In view of the need for improved analytical performance as indicated by past interlaboratory audit surveys, we feel the small increase in resources that will be experienced will be justified by the increased reliability of the compliance results. 13. D-8 Comment: The proposed rules impose an additional level of regulation. The rules state the Administrator may choose to use compliance test results to determine the compliance or noncompliance status of the affected facility regardless of the audit sample results; and he may waive reanalysis if the audit analysis fails. Therefore, the rationale for the need to have QA regulations is missing. Reputable analytical facilities have and continually update their QA procedures. The proposed requirements are limited in nature and probably less comprehensive than existing laboratory QA procedures. Response: Administrator discretion comes into play in only two cases: when the test shows the source is substantially below the standard, but the audit is marginally failed; and when the source is substantially above the standard, but the audit is marginally failed. In each case, the application of a correction factor would not change the compliance or noncompliance status of the source. Acceptance of the test results in each case is allowed because compliance or noncompl iance is clearly shown and would not be affected by the additional error above that normally allowed. 10 ------- 14. D-8 Comment: The proposed rules do not provide recourse if the analyses are not within the specified boundaries. There needs to be a mechanism for a laboratory to resolve any differences in analytical results without necessarily performing a reanalysis. Response: The mechanism for resolving analytical differences will depend upon the source of error. If the standards were incorrectly made, a new set would have to be prepared and the samples reanalyzed. If only a calculation error is made, then data correction without reanalysis would correct the problem. It is recommended that a known quality control sample be analyzed prior to the audit and compliance samples to optimize the system accuracy and precision. 15. D-13 Comment: Administrator discretion in accepting failed audit results is wholly inconsistent with accepted QA practice. It is not known whether the data will be used as submitted or a correction factor developed from the audit results. If the Agency has determined that 5 percent is the maximum allowable error, all associated data with such an error should be invalidated with no exceptions and no corrections. The results should not be left to the arbitrary discretion of an individual. Response: See the response to Comment 13. The data will be used as submitted without applying a correction factor. 16. D-5, D-14 Comment: The proposed audit would not guarantee the accuracy of emission data since the evaluation is on the analysis and not on the 11 ------- sampling portion. Although the proposed additions are a step in the right direction, it would be hetter to have a procedure to evaluate both the sampling and analysis, even though this would not be ideal because of possible matrix or interference effects in the actual source samp! es. Response: We agree. However, the Agency's experience in auditing the sampling procedure (using gas cylinders) has found this approach to be impractical, expensive, and the benefits received would not justify the effort required to perform it. If a simple auditing technique can be developed, EPA will consider such an audit in the future. 17. D-2, D-9 Comment: An audit per set of compliance-samp! es is over- regulation. The proposed quarterly exemption only covers repetitive testing at a given source. Most control agencies will have a central laboratory and will use different personnel and analytical systems. The NOX analyses are very time consuming, and a team of chemists may be routinely assigned to analyze the samples so they can be completed in a timely manner. The requirement that only one person analyze NOX samples (Paragraph 4.4) could delay the preparation of the compliance test report past deadlines now set by the States and EPA. The proposal should be amended to allow more than one person to analyze the samples. Any analytical variations would be checked by quality control procedures currently incorporated in EPA methods (e.g., equipment calibrations, duplicate testing and analysis, and inter!aboratory testing). Furthermore, since the "oneness" rationale is not and should 12 ------- not apply to the sample collection phase, it should not be mandated for the analyst and analytical equipment phase. Response: See the response to Comment 3. The intention of the rule is not to restrict the analysis to one person but to ensure that all parties involved in the analysis of the compliance samples likewise take part in the analysis of the audits. The names of all persons participating in the audit analysis are to be included in the report that is submitted to the appropriate enforcement agency. 18. D-6 Comment: The requirement to analyze two unknown samples as well as a known sample prior to the analysis is redundant. Each component of the analysis is either checked for impurities or standardized against a primary standard. Response: The QA audit is an overall check of the method's quality control. These quality control checks in the methods are beneficial but do not guarantee accuracy. Standardization against primary standards is subject to such inaccuracies as weighing errors, dilution errors, calculation errors, etc.; the QA audit will detect such errors. Analysis of the known sample is optional. 19. D-15 Comment: We oppose the proposed rule because it will increase the cost of conducting a compliance test considerably and will put the smaller stack sampling firms at a competitive disadvantage. These firms may be .forced to discontinue the testing services because they will not be able to absorb the additional cost and remain competitive. In our opinion, the benefit gained by the improved QA does not justify the 13 ------- increased cost. Both methods.currently provide for adequate standardization of reagents and provide detailed analytical procedures that should yield acceptable results with an acceptable degree of accuracy. Response: See the responses to Comments 12 and 18. The increased cost associated with implementing QA requirements should not be substantial enough to give larger firms a competitive advantage. 20. D-8, D-12 Comment: One-concern of the proposed rule is cost. The first time cost to analyze the audit samples would not be significant since analysis would be done with the compliance samples. However, costs may be very significant if reanalysis is necessary due to audit samples of poor quality, failure to acquire audit samples when needed, or a subsequent determination that the specifications are unduly stringent. Moreover, there is no evidence in the proposal that EPA considered compliance retesting costs to any extent in the determination of economic impact. Response: The background information report shows that a very high percentage of representative laboratories are capable of analyzing EPA-prepared audit samples and passing the proposed acceptance levels once correctable mistakes have been eliminated. Past experience also indicates that increased familiarity with the methods (especially Method 7) tends to increase operator accuracy. The known quality control sample will point out deficiencies in the analysis which can be corrected and eliminated for subsequent tests. Laboratories employing qualified personnel and good quality control procedures will incur little additional expense due to reanalysis. 14 ------- 21. D-7 Comment: We are concerned about the delay that would be caused by repeating the entire analysis procedure if audit results fall outside the acceptable range. The delay due to reordering, receiving, and finally reanalyzing the audit samples could impede laboratory routine and delay report submittal which, in some instances, is required within 30 days. The entire procedure is cumbersome and counterproductive to lab efficiency. Response: The entire procedure should increase lab efficiency and accuracy since analysis error will be kept within an acceptable level. Each S02 and NOX audit vial contains enough solution for at least two duplicate tests, so reordering samples for reanalysis should not be necessary. In the event a reanalysis is required, retesting can be avoided by saving the remainder of the samples for reanalysis. 22. D-7 Comment: Rather than performing an analysis on unknown samples, it may be more beneficial to analyze EPA-supplied specimens of known concentration prior to running compliance test samples. If necessary, require that results of the standards analysis be included with the compliance test report. In this way, mistakes can be immediately identified and resolved before continuing further. Response: Known samples are available from EPA and can be used for that purpose. The incentive to maximize accuracy by a tester would be lessened if analysis of the known were required instead of the blind audit. It would be easiest to resolve mistakes by reporting the audit results by telephone. In this manner, mistakes will be immediately 15 ------- known and can be resolved before writing the test report and having to wait for a reply. 23. D-2 Comment: A single audit check per quarter should be adequate. Individual precision checks should be accomplished by requiring that the known concentration be analyzed with each set of compliance samples collected during the quarter. Response: See the responses to Comments 3 and 22. 24. D-8 Comment: The proposed rules do not address the origin, quality, or nature of the audit samples. This is an important concern if a laboratory's performance is being assessed based on these samples. A reference standard for preparation of the audit samples should be cited. Response: The audit samples are to be obtained from EPA as detailed in Sections 3.3.6 for Method 6 and 3.3.9 for Method 7. 25. D-3, D-6 Comment: We object to the requirement that only EPA-derived audit samples be accepted when making compliance analyses. We propose that internally-generated standards be allowed in lieu of the EPA standards. The practice of analyzing standards is routinely followed in good laboratories with good QA procedures. These laboratories should be allowed to continue such practices without the added workload of EPA audit samples. In addition, the potential exists for error in the audit sample preparation or in documenting the actual audit concentrations. 16 ------- Response: See the response to Comment 18. The potential for errors in preparation and documentation of actual concentrations would be compounded if each laboratory were allowed to generate its own audit samples. Any error in an EPA-derived audit would be rapidly detected since many laboratories will be analyzing them. To ensure fairness in meeting the audit accuracy levels, a single source of samples is needed to serve as a reference for all laboratories. 26. D-13 Comment: A serious concern is that the audit sample results can only be obtained if the audit and set of compliance results are telephoned to the Agency, presumably by the contractor. It is unthinkable to use telephone communication for compliance determinations as the Agency proposes. Response: Reporting the audit results by telephone is an option and not a requirement. Submission of a written test report is still mandatory in this case. The allowance for telephone communication enables the tester to determine at an early stage whether the audit is passed and hence the compliance data acceptable, or whether the audit is failed, and a reanalysis is necessary. Determination of the audit status before the test report is written and before the audit and compliance samples have aged substantially would eliminate much of the unnecessary work and expense created if it is determined at a later date that a reanalysis is needed. 17 ------- 27. D-2, D-ll Comment: In Method 6, Section 6.4 is lacking the relative error equation for the QA audits. Response: This addition has been made. 18 ------- Table 1. LIST OF COMMENTERS Docket Number A-80-57 Document Number IV-D-1 IV-D-2 IV-D-3 IV-D-4 IV-D-5 Commenter/Affi 1 i ati on Dick Serdoz, Air Quality Officer State of Nevada Dept. of Conservation and Natural Resources Division of Environmental Protection Carson City, Nevada 89710. Chief Services and Ben A. Brodovicz, Div. of Technical Monitoring Commonwealth of Pennsylvania Dept. of Environmental Resources Post Office Box 2063 Harrisburg, Pennsylvania 17120 Thomas A. Lowe, Manager Analytical Research Kaiser Aluminum and Chemical Center for Technology Post Office Box 877 Pleasanton, California 94566 Corp. Charles S. Krick, Environmental Engineer State of Delaware Dept. of Natural Resources and Environmental Control Division of Environmental Control Edward Tatnall Building Dover, Delaware 19901. Charles D. Mai loch, Director Regulatory Management, Air Environmental Policy Staff Monsanto Company 800 N. Lindbergh Boulevard St. Louis, Missouri 63166. 19 ------- Table 1. LIST OF COMMENTERS Docket Number A-80-57 (Continued) Document Number IV-D-6 IV-D-7 IV-D-8 IV-D-9 IV-D-10 IV-D-11 Commenter/Affi 1 i ati on Douglas G. Mercer, Superintendent Environmental Programs Texas Gulf Chemicals Co. Post Office Box 48 Aurora, North Carolina 27806 J.M. Reinke, Manager Survey and Evaluation Stationary Source Environmental Control Office Ford Motor Company One Parklane Boulevard Dearborn, Michigan 48126 Roger D. Dodds, Air Quality Engineer Environmental Dept. Wisconsin Electric Power Co. 231 W. Michigan Post Office Box 2046 Milwaukee, Wisconsin 53201 Robert L. Pearson, Administrator Environmental Affairs Public Service Company of Colorado Post Office Box 840 Denver, Colorado 80201 V.C. Eissler, Vice President North American Production Conoco, Inc. Post Office Box 2197 Houston, Texas 77252 Ashok K. Jain, Manager Engineering Projects Research National Council of the Paper Industry for Air and Stream Improvement Southern Regional Center Post Office Box 14483 Gainesville. Florida 32604 20 ------- Table 1. LIST OF COMMENTERS Docket Number A-80-57 (Continued) Document Number IV-D-12 IV-D-13 IV-D-14 IV-D-15 Commenter/Affiliation T.C. Owen, Director Corporate Office of Environmental Affairs Union Camp Corporation Post Office Box 1391 Savannah, Georgia 31402 Wayne T. Swallow, Supervisor Regulatory and Environmental Monitoring Public Service Indiana Plainfield, Indiana 46168 Theodore L. Kinne, Director Construction and Operations Interstate Natural Gas Association of America 1660 L Street, N.W. Washington, D.C. 20036 Herman A. Fritschen, General Manager Safety and Environmental Services Cities Service Company Box 300 Tulsa, Oklahoma 74102 21 ------- TECHNICAL REPORT DATA (Please read Instructions on the reverse before completing) 1. REPORT NO. EPA-450/3-34-006 3. RECIPIENT'S ACCESSION NO. 4. TITLE AND SUBTITLE Quality Assurance and Quality Control Revisions to Methods 6 and 7 5. REPORT DATE March 1984 6. PERFORMING ORGANIZATION CODE 7. AUTHOR(S) 8. PERFORMING ORGANIZATION REPORT NO. Emission Standards and Engeneerinq Division 9. PERFORMING ORGANIZATION NAME AND ADDRESS Emission Measurement Branch (MD-19) Emission Standards and Engineering Division U.S. Environmental Protection Agency Research Triangle Park, North Carolina 27711 10. PROGRAM ELEMENT NO. 11, CONTRACT/GRANT NO. rj. SPONSORING AGENCY NAME AND ADDRESS 13. TYPE OF RE PORT AN D PERIOD COVERED DAA for Air Quality Planning and Standards (MD-10) Office of Air, Noise, and Radiation U.S. Environmental Protection Agency Research Triangle Park, North Carolina 27711 14. SPONSORING AGENCY CODE EPA/200/04 .UI-PLEMENTARY NOTES !u ABSTRACT This document addresses the public comments submitted after proposal of the rulemaking in the Federal Register. Changes made to this rulemaking as a result of these comments are included. This document serves as the basis for the revisions which have been made to the rulemaking between proposal and promulgation. KEY WORDS AND DOCUMENT ANALYSIS DESCRIPTORS It! ui:.l HIQUTION STATEMENT Release Unlimited b.lOENTlFlERS/OPEN ENDED TERMS 19. SECURITY CLASS (This Reporr) Unclassified 20. SECURITY CLASS (This page) Unclassified c. COSATi F-'ield/Croup 21. NO. OF PAGES 21 22. PRICE ------- |