2020 & 2021 EPA Wood Burning Appliance Test Laboratory
Proficiency Test Protocol
INTRODUCTION:
40 CFR 60, subpart AAA and subpart QQQQ regulate wood burning appliances and include details that
provide a pathway for laboratories wishing to conduct compliance testing on affected appliances a
pathway to obtaining EPA Approval.
INTENT:
This document provides a framework, informing a Proficiency Test Provider how an EPA
Approved Wood Burning Appliance Test Lab Proficiency Test program is to be conducted,
including detailed direction such that PT providers working independently will provide like data
to EPA for evaluation.
SCOPE:
This document is intended to provide a framework sufficient for constructing and delivery of a
robust and thorough PT test, whether such tests are contracted through a sole provider or
multiple providers.
REQUIREMENTS for PT Providers: All entitles, both contractor and subcontractor, wishing to
provide Proficiency Test services to an EPA Approved Wood Burning Appliance Test Laboratory
must register with EPA's Burn Wise Program by sending an e-mail stating that they agree to
abide by the terms specified within this protocol and by the requirements specified by US EPA in
establishing the PT parameters. Providers must also agree that they will cooperate with EPA's
requests for data used to conduct outlier analysis and laboratory comparative analyses. PT
providers must provide to EPA, via electronic mail, confirmation of each laboratory successfully
completing a PT program. PT providers must provide their contact information and the location
(physical address) of their business, and must also disclose to the EPA the name and address of
any subcontractor used to deliver their PT services, and their degree of the subcontractors
involvement in the PT program.
-------
SECTION 1:
The Wood Burning Appliance Test Laboratory Proficiency Test Protocol consists of the following five
elements:
1. PM measurements must be conducted on a pellet burning appliance model common to each
laboratory, while burning pellet fuel common to all Wood Burning Appliance Lab PT tests during
each PT Test cycle (every two years), and using equipment and procedures specified in ASTM E-
2779-10 and E-2515-11, with the appliance operation and PM sampling conducted at a
prescribed volumetric rate. Specifics regarding the type of common fuel, operational setting of
the pellet burning appliance, and sampling rate of the ASTM E-2515 apparatus will be
determined by the EPA for each PT test cycle, and distributed ONLY to the PT Provider(s) (not
directly to labs, nor to PT Provider subcontractors) for the expressed purposes of conducting
Proficiency Tests. The PT provider(s) will obtain common fuel meeting that specified by US EPA,
and distribute to the laboratories that they service.
a. Specifics that must be met prior to, or during each PT test are:
i. Installation of pellet burning appliance per manufacturer's written operation
instructions and in compliance with test requirements outlined in ASTM E-2779-
10 and ASTM E-2515-11. The flue and dilution tunnel surfaces must be cleaned
prior to stove installation. If the stove has been sealed following a previous PT
test, the stove must be unsealed in the presence of the PT Provider, however all
other installation tasks may be undertaken prior to arrival of the PT provider, if
desired.
ii. Check the appliance installation for tight flue joints and gaskets, intact and tight
door seal, and replace cracked bricks if found.
iii. No 60-minute filter information will be collected during any PT test.
iv. A room-blank sample must be conducted during the PT test and gravimetric
results of that room blank must be reported with the PT data.
v. The fuel pot, dilution tunnel and stove flue must be cleaned before the first pre-
burn preceding Test Run #1. The fuel pot must be cleaned and area around the
fuel pot must be vacuumed following Test Run #1 and Test Run #2, before
proceeding to another PTTest Run.
vi. Three 180-minute PM measurements are made by each participating lab. This is
due to PM carryover from Test Run to Test Run during the initial round of PT
testing. We do not anticipate the need for three measurements in the next
round of PT Testing. All three measurements must be made at the same
operational setting, with the same fuel and sampling parameters.
b. The PT test sequence shall be:
i. Review and verification of appropriate equipment necessary to conduct the test
ii. Fuel hopper of the appliance must be filled with the test fuel, and stove weight
with full hopper must be noted prior to beginning the test.
EPA Wood Burning Appliance Test Laboratory Proficiency Test Protocol 2020-2021
-------
iii. Ignition and 30 minutes of pre-burn at the designated appliance setting (to be
communicated to the lab by the PT test provider on site) using only the test fuel
provided by the PT provider.
iv. Begin the first PT test pre-burn period of thirty minutes and begin the sampling
portion of the first PT test, per instruction from the PT provider, at the end of
this thirty-minute period.
v. Sample continuously at the prescribed rate for the entire duration specified by
the PT provider.
• After the third PT Test Run, the stove must be cleaned and re-sealed in
the presence of the PT Provider and stored until the next round of PT
Testing where it will be un-sealed, again in the presence of the PT
provider.
vi. At the end of the first PM measurement, and prior to the beginning of a second
PM measurement immediately refill the hopper so that the stove weight is +/-
0.1 lb of the initial stove weight recorded in item (ii), above.
i. Once the fuel hopper is filled, record the stove and fuel weight, clean the fuel
pot and the area around the fuel pot must be vacuumed, then begin the
second thirty-minute pre-burn at the prescribed PT test burn rate.
ii. Begin the second test period immediately following the second thirty-minute
pre-burn period.
iii. Again, at the end of the second PM measurement and prior to the beginning
of a third PM measurement immediately refill the hopper so that the stove
weight is +/- 0.1 lb of the initial stove weight recorded in item (ii), above.
iv. Once the fuel hopper is filled, record the stove and fuel weight, clean the fuel
pot and the area around the fuel pot must be vacuumed, then begin the third
thirty-minute pre-burn at the prescribed PT test burn rate.
v. Begin the third test period immediately following the second thirty-minute
pre-burn period.
vi. After the third PT Test Run has completed, the cooled stove must be cleaned
and sealed in the presence of the PT Provider and stored until a subsequent
Proficiency Test requires reinstallation of the stove.
c. The PT Provider will be on site to observe:
i. Unsealing/sealing of the stove before/after the PT test
ii. Setup and recovery of the room blank sampling system
iii. Setup and recovery of sampling system blank recovery
iv. Set up of all sampling equipment including pre-test leak checks
v. Entire PT test series from start of first pre-burn to end of the third PT test
vi. Post PT test leak checks
vii. Post PT sample recovery and sample transport procedures
viii. Laboratory sample treatment and gravimetric analysis of audit filter
EPA Wood Burning Appliance Test Laboratory Proficiency Test Protocol 2020-2021
-------
d.
PT Provider to receive a copy (printed spreadsheet or .pdf file) of calculated test results
for each PT particulate test run, including blank correction, laboratory weighing sheets,
and all pm calculation results as well as blank train recovery and room blank data.
e. PT Provider is to log observations of the PT particulate testing and note all deficiencies
and corrective measures effected during the testing.
f. PT Provider may remotely witness (video link observation) follow up or corrective action
procedures, where feasible, to avert travel costs for a return site visit, provided stove
operation is not required to close out the PT activities.
2. The test laboratory technical staff will perform a flow traverse of the dilution tunnel flow during
each of the 30-minute pre-burn periods preceding each test run and use this information to
determine the ASTM 2515 test probe location during the subsequent test. The dilution tunnel
flow velocity will be calculated, and those results included in the data reported to the PT
Provider.
3. The PT Provider will present the laboratory with a filter audit sample of NIST traceable known
dry mass intended to verify that desiccation procedures and gravimetric analysis of filter media
is within acceptable tolerance {+/-1.5 mg). These QC samples are commercially available and
commonly used in many source particulate measurement compliance programs. An example
may be found here: http://www.eraqc.com/Products/catalogid/l-4GR6WY/categoryid/l-
4GSDK2/catalognumber/1150
4. The PT Provider will observe each participating laboratory conducting a sampling train blank
analysis, including the setup of a full sampling train (loading filters, connecting probes,
equipment, etc.). This sampling system will be fully assembled, two complete leak checks
performed, then disassembled, sample recovered, and gravimetric analysis conducted with
results presented to the PT provider in printed spreadsheet or .pdf file format. The results of
the sample train blank study will be reported to the PT Provider and the PT Provider will conduct
an outlier analysis of these data as described below.
5. The PT Provider will observe each participating laboratory conduct a room blank sample, per
ASTM E-2515-11, Sections 9.5.2, 9.8.1 and 9.8.5 then recover the room blank sample and
analyze per ASTM E-2515-11, Sections 9.10, 10.2 and 11. The results of the room blank sample
will be reported to the PT Provider and the PT Provider will conduct an outlier analysis of these
data as described below.
EPA Wood Burning Appliance Test Laboratory Proficiency Test Protocol 2020-2021
-------
SECTION 2:
The Proficiency Test Program will function as follows:
1. Laboratories have purchases and must retain a commercially available pellet stove specifically
for the purposes of participating in the PT program. Labs must re-use the identical pellet stove
used during the 2017-2018 PT program testing. The PT Provider will verify that the pellet stove
in use is identical to the pellet stove previously used at that same laboratory, and has not been
used or modified since, through examining the Seal.
2. Laboratories will contract with a PT Provider on terms to deliver elements 1-4 in Section 1
(above). The PT Provider will notify the EPA Burn Wise program quarterly with a list of
laboratories having completed a PT during that quarter (if any), and provide results of PM
measurement data and sample train blank analysis from numbers 1 and 4, above as well as
Verification of Completion of the PT Program for each lab serviced. The Verification statement
from the PT Provider is only to be provided to EPA after a PT Program for a specific EPA
Approved Laboratory is complete and either no issues of concern were noted or that all issues of
concern have been resolved through investigation, root cause analysis, or a repeated section of
the PT, to the satisfaction of the PT Provider.
3. If multiple PT Providers are used to service the community of Approved Test Laboratories, ALL
PT Providers MUST follow this publicly posted protocol and MUST coordinate their efforts
through the Measurement Technology Group at EPA. Each round of PT Testing requires
standardization of fuel, burn rate and sampling rate of the PM testing, and coordination of test
results for the purposes of outlier analysis. A sole PT provider serving all the EPA Approved Labs
must also coordinate each round of PT testing with EPA's Measurement Technology Group and
provide quarterly updates with respect to completed PT tests and program progress.
4. PT Providers will submit data and results to EPA in de-identified form, but marked such that the
PT Provider(s) retain ability to track and maintain records of each Laboratories results and
information. For example, the EPA does not need to know which lab has provided data that is
an outlier and yet the PT Provider, once informed by EPA that Lab "4" returned results that are
identified as an outlier, must be able to work with Lab "4" to resolve that concern.
5. A statistical outlier analysis will be conducted for each full round of PT Testing, on the results of
the PM Measurement outlined in Section 1. The means of outlier analysis will be a statistical
test using a Dixon test conducted with ProUCL software publicly available here:
https://www.epa.gov/land-research/proucl-version-5100-documentation-downloads
a) With the ProUCL tool the PT must evaluate the full data set for normal or lognormal
distribution. If the data set is lognormally distributed, the data must be lognormally
transformed prior to outlier analysis.
EPA Wood Burning Appliance Test Laboratory Proficiency Test Protocol 2020-2021
-------
b) The full data set of individual test run values (generally three per lab), normal or log-
transformed (depending on the outcome of normal/ lognormal analysis), is then evaluated
by the Dixon test using the 5% significance criterion.
c) If an outlier is identified, that value must be removed from the data set and the data
evaluated once more, using the Dixon test a second time. Repeat this procedure until the
output report shows no outliers remaining in the data set. Two or more outlier values
representing any single lab result set shall require the PT Provider to investigate and
reconcile the issue(s) with the lab according to the PT Provider's procedures.
6. A statistical outlier analysis will be conducted for each full round of PT Testing, using data from
each laboratory blank sample mass analysis, as described in Section 1, item 4 (above). The
outlier analysis must be conducted per Section 2, item 5 (analysis of results of PM
measurement) but the blank train outlier analysis will use a 10% significance criterion.
7. A statistical outlier analysis will be conducted for each full round of PT Testing, on the results of
the Room Blank measurement outlined in Section 1, item 5 (above). The outlier analysis must
be conducted per Section 2, item 5 (analysis of results of PM measurement but the room blank
outlier analysis will use a 10% significance criterion.
8. Failed PT Tests (where a PT Provider and laboratory cannot resolve an outlier result or a
procedural result) will trigger an ISO 17025 Root Cause evaluation for the EPA Approved Lab
involved in that PTTest. ISO 17025 procedures must be followed, with corrective measures
identified and applied.
9. PT Providers MUST provide records to EPA upon request, including calibration documentation of
all NIST traceable items, documentation relating to but not limited to test results, re-test efforts,
problem resolution as well as records of notes and data collected during PT testing should EPA
feel it necessary to audit a PT Provider.
EPA Wood Burning Appliance Test Laboratory Proficiency Test Protocol 2020-2021
------- |