2023 & 2024 EPA WOOD BURNING APPLIANCE TEST LABORATORY Proficiency Test Protocol INTRODUCTION: 40 CFR 60, subpart AAA and subpart QQQQ regulate wood burning appliances and include details that provide a pathway for laboratories wishing to conduct compliance testing on affected appliances a pathway to obtaining EPA Approval. INTENT: This document provides a framework, informing a Proficiency Test (PT) Provider how an EPA Approved Wood Burning Appliance Test Lab Proficiency Test program is to be conducted, including detailed direction such that PT providers working independently will provide like data to EPA for evaluation. SCOPE: This document is intended to provide a framework sufficient for construction and delivery of a robust and thorough PT test, whether these tests are contracted through a sole provider or multiple providers. REQUIREMENTS FOR PT PROVIDERS: All entities, both contractor and subcontractor, wishing to provide Proficiency Test services to an EPA Approved Wood Burning Appliance Test Laboratory must register with EPA's Burn Wise Program by sending an e-mail stating that they agree to abide by the terms specified within this protocol and by the requirements specified by US EPA in establishing the PT parameters. Providers must also agree that they will cooperate with EPA's requests for data used to conduct outlier analysis and laboratory comparative analyses. PT providers must provide to EPA, via electronic mail, confirmation of each laboratory successfully completing a PT program. PT providers must provide their contact information and the location (physical address) of their business, must also disclose to the EPA the name and address of any subcontractor used to deliver their PT services, and their degree of the subcontractors involvement in the PT program. ------- SECTION 1: The Wood Burning Appliance Test Laboratory Proficiency Test Protocol consists of the following five elements: 1. PM measurements must be conducted on a pellet burning appliance model common to each laboratory, while burning pellet fuel common to all Wood Burning Appliance Lab PT tests during each PTTest cycle (every two years), and using equipment and procedures specified in ASTM E2779-10 and ASTM E2515-11, with the appliance operation and PM sampling conducted at a prescribed volumetric rate. Specifics regarding the type of common fuel, operational setting of the pellet burning appliance, and sampling rate of the ASTM E2515-11 apparatus will be determined by the EPA for each PT test cycle, and distributed ONLY to the PT Provider(s) (not directly to labs, nor to PT Provider subcontractors) for the expressed purposes of conducting Proficiency Tests. The PT provider(s) will obtain common fuel meeting that specified by US EPA, and distribute to the laboratories that they service. a. Specifics that must be met prior to, or during each PT test are: i. Installation of pellet burning appliance per manufacturer's written operation instructions and in compliance with test requirements outlined in ASTM E2779- 10 and ASTM E2515-11. The flue and dilution tunnel surfaces must be cleaned prior to stove installation. If the stove has been sealed following a previous PT test, the stove must be unsealed in the presence of the PT Provider, however all other installation tasks may be undertaken prior to arrival of the PT provider, if desired. ii. Check the appliance installation for tight flue joints and gaskets, intact and tight door seal, and replace cracked bricks if found. iii. No 60-minute filter information will be collected during any PT test. iv. A room-blank sample must be conducted during the PT test and gravimetric results of that room blank must be reported with the PT data. v. The fuel pot, dilution tunnel and stove flue must be cleaned before the first pre- burn preceding Test Run #1. The fuel pot must be cleaned and area around the fuel pot must be vacuumed following Test Run #1 and Test Run #2, before proceeding to another PT Test Run. vi. Three ~ 200 -minute (targeting a specified sample volume) PM measurements are made by each participating lab. All three measurements must be made at the same operational setting, with the same fuel and consistent sampling parameters. vii. Measurement of carbon monoxide (CO) must be conducted in flue per CSA Method B415.1-10. viii. Each proficiency test must be conducted at the same feed rate setting. This setting must be used at all test labs. b. The PT test sequence shall be: EPA Wood Burning Appliance Test Laboratory Proficiency Test Protocol 2023-2024 ------- i. Review and verification of appropriate equipment necessary to conduct the test. ii. Fuel hopper of the appliance must be filled with the test fuel, and stove weight with full hopper must be noted prior to beginning the test. iii. Ignition and 40 minutes of pre-burn at the designated appliance setting (to be communicated to the lab by the PT test provider on site) using only the test fuel provided by the PT provider. iv. Begin the first PT test pre-burn period of 40 minutes and begin the sampling portion of the first PT test, per instruction from the PT provider, at the end of this 40-minute period. v. Sample continuously at the prescribed rate for the entire duration specified by the PT provider. • After the third PTTest Run, the stove must be cleaned and re-sealed in the presence of the PT Provider and stored until the next round of PT Testing where it will be un-sealed, again in the presence of the PT provider. vi. At the end of the first PM measurement, and prior to the beginning of a second PM measurement, immediately refill the hopper and record-stove weight as described in item (ii), above. vii. Once the fuel hopper is filled, record the stove and fuel weight, clean the fuel pot and the area around the fuel pot must be vacuumed, then begin the second 40-minute pre-burn at the prescribed PT test burn rate. viii. Begin the second test period immediately following the second 40-minute pre- burn period. ix. Again, at the end of the second PM measurement and prior to the beginning of a third PM measurement immediately refill the hopper and record the stove weight as described in item (ii), above. x. Once the fuel hopper is filled, record the stove and fuel weight, clean the fuel pot and the area around the fuel pot must be vacuumed, then begin the third thirty-minute pre-burn at the prescribed PT test burn rate. xi. Begin the third test period immediately following the second thirty-minute pre- burn period. xii. After the third PTTest Run has completed, the cooled stove must be cleaned and sealed in the presence of the PT Provider and stored until a subsequent Proficiency Test requires reinstallation of the stove. c. The PT Provider will be on site to observe: i. Unsealing /sealing of the stove before/after the PT test ii. Setup and recovery of the room blank sampling system iii. Setup and recovery of sampling system blank recovery iv. Set up of all sampling equipment including pre-test leak checks v. Entire PT test series from start of first pre-burn to end of the third PT test vi. Post PT test leak checks vii. Post PT sample recovery and sample transport procedures EPA Wood Burning Appliance Test Laboratory Proficiency Test Protocol 2023-2024 ------- viii. Laboratory sample treatment and gravimetric analysis of audit filter d. PT Provider to receive a copy (printed spreadsheet or .pdf file) of calculated test results for each PT particulate test run, including blank correction, laboratory weighing sheets, and all pm calculation results as well as blank train recovery and room blank data. e. PT Provider is to log observations of the PT particulate testing and note all deficiencies and corrective measures effected during the testing. f. PT Provider may remotely witness (video link observation) follow up or corrective action procedures, where feasible, to avert travel costs for a return site visit, provided stove operation is not required to close out the PT activities. 2. The test laboratory technical staff will perform a flow traverse of the dilution tunnel flow during each of the 40-minute pre-burn periods preceding each test run and use this information to determine the ASTM E2515-11 test probe location during the subsequent test. The dilution tunnel flow velocity will be calculated, and those results included in the data reported to the PT Provider. 3. The PT Provider will present the laboratory with a filter audit samples of NIST traceable known dry mass intended to verify that desiccation procedures and gravimetric analysis of filter media is within acceptable tolerance (± 1.5 mg). These QC samples are commercially available and commonly used in many source particulate measurement compliance programs. An example may be found here: http://www.eraqc.com/Products/catalogid/l-4GR6WY/categorvid/l- 4GSDK2/catalognumber/1150 4. The PT Provider will observe each participating laboratory conducting a sampling train blank analysis, including the setup of a full sampling train (loading filters, connecting probes, equipment, etc.). This sampling system will be fully assembled, two complete leak checks performed, then disassembled, sample recovered, and gravimetric analysis conducted with results presented to the PT provider in printed spreadsheet or .pdf file format. Front and back filters must be weighed in pairs (pre and post-test) to mitigate some gravimetric error propagation. The results of the sample train blank study will be reported. 5. The PT Provider will observe each participating laboratory conduct a room blank sample, per ASTM E2515-11, Sections 9.5.2, 9.8.1 and 9.8.5 then recover the room blank sample and analyze per ASTM E2515-11, Sections 9.10, 10.2 and 11. The results of the room blank sample will be reported to the PT Provider and the PT Provider will conduct an outlier analysis of these data as described below. EPA Wood Burning Appliance Test Laboratory Proficiency Test Protocol 2023-2024 ------- SECTION 2: The Proficiency Test Program will function as follows: 1. Laboratories have purchases and must retain a commercially available pellet stove specifically for the purposes of participating in the PT program. Labs must re-use the identical pellet stove used during the 2017-2018 PT program testing. The PT Provider will verify that the pellet stove in use is identical to the pellet stove previously used at that same laboratory, and has not been used or modified since, through examining the Seal. 2. Laboratories will contract with a PT Provider on terms to deliver elements 1-4 in Section 1 (above). The PT Provider will notify the EPA Burn Wise program quarterly with a list of laboratories having completed a PT during that quarter (if any), and provide results of PM measurement data and sample train blank analysis from numbers 1 and 4, above as well as Verification of Completion of the PT Program for each lab serviced. The verification statement from the PT Provider is only to be provided to EPA after a PT Program for a specific EPA Approved Laboratory is complete and either no issues of concern were noted or that all issues of concern have been resolved through investigation, root cause analysis, or a repeated section of the PT, to the satisfaction of the PT Provider. 3. If multiple PT Providers are used to service the community of Approved Test Laboratories, ALL PT Providers MUST follow this publicly posted protocol and MUST coordinate their efforts through the Measurement Technology Group at EPA. Each round of PT Testing requires standardization of fuel, burn rate and sampling rate of the PM testing, and coordination of test results for the purposes of outlier analysis. A sole PT provider serving all the EPA Approved Labs must also coordinate each round of PT testing with EPA's Measurement Technology Group and provide quarterly updates with respect to completed PT tests and program progress. 4. PT Providers will submit data and results to EPA in de-identified form, but marked such that the PT Provider(s) retain ability to track and maintain records of each Laboratories results and information. For example, the EPA does not need to know which lab has provided data that is an outlier and yet the PT Provider, once informed by EPA that Lab "4" returned results that are identified as an outlier, must be able to work with Lab "4" to resolve that concern. 5. A statistical outlier analysis will be conducted for each full round of PT Testing, on the results of the PM Measurement outlined in Section 1. The means of outlier analysis will be a statistical test using a Dixon test conducted with ProUCL software publicly available here: https://www.epa.gov/land-research/proucl-version-5100-documentation-downloads a. With the ProUCL tool the PT must evaluate the full data set for normal or lognormal distribution. If the data set is lognormally distributed, the data must be lognormally transformed prior to outlier analysis. EPA Wood Burning Appliance Test Laboratory Proficiency Test Protocol 2023-2024 ------- b. The full data set of individual test run values (generally three per lab), normal or log- transformed (depending on the outcome of normal/ lognormal analysis), is then evaluated by the Dixon test using the 5% significance criterion. c. If an outlier is identified, that value must be removed from the data set and the data evaluated once more, using the Dixon test a second time. Repeat this procedure until the output report shows no outliers remaining in the data set. Two or more outlier values representing any single lab result set shall require the PT Provider to investigate and reconcile the issue(s) with the lab according to the PT Provider's procedures. 6. Failed PT Tests (where a PT Provider and laboratory cannot resolve an outlier result or a procedural result) will trigger an ISO 17025 Root Cause evaluation for the EPA Approved Lab involved in that PTTest. ISO 17025 procedures must be followed, with corrective measures identified and applied. 7. PT Providers MUST provide records to EPA upon request, including calibration documentation of all NIST traceable items, documentation relating to but not limited to test results, re-test efforts, problem resolution as well as records of notes and data collected during PT testing should EPA feel it necessary to audit a PT Provider. EPA Wood Burning Appliance Test Laboratory Proficiency Test Protocol 2023-2024 ------- |