SRI/USEPA-GHG-QAP-39
December 2004
Test and Quality Assurance
Plan
White Sands, LLC. -
CleanBoost Combustion Catalyst
Diesel Fuel Additive
Prepared by:
Greenhouse Gas Technology Center
Southern Research Institute
Under a Cooperative Agreement With
U.S. Environmental Protection Agency
-------
EPA REVIEW NOTICE
This report has been peer and administratively reviewed by the U.S. Environmental Protection Agency, and
approved for publication. Mention of trade names or commercial products does not constitute endorsement or
recommendation for use.
-------
SRI/USEPA-GHG-QAP-39
December 2004
Greenhouse Gas Technology Center
A U.S. EPA Sponsored Environmental Technology Verification ( jj^) Organization
Test and Quality Assurance Plan
for
White Sands, LLC. -
CleanBoost Combustion Catalyst
Diesel Fuel Additive
Prepared By:
Greenhouse Gas Technology Center
Southern Research Institute
PO Box 13825
Research Triangle Park, NC 27709 USA
Telephone: 919/806-3456
Reviewed By:
White Sands, LLC.
Southwest Research Institute
U.S. EPA Office of Research and Development
^ indicates comments are integrated into Test Plan
-------
(this page intentionally left blank)
-------
Greenhouse Gas Technology Center
f/.5. EPA Sponsored Environmental Technology Verification ( f^jy) Organization
Test and Quality Assurance Plan
for
White Sands, LLC. -
CleanBoost Combustion Catalyst
Diesel Fuel Additive
This Test and Quality Assurance Plan has been reviewed and approved by the Greenhouse Gas
Technology Center Project Manager and Center Director, the U.S. EPA APPCD Project Officer, and the
U.S. EPA APPCD Quality Assurance Manager.
Tim Hansen Date
Deputy Director
Greenhouse Gas Technology Center
Southern Research Institute
David Kirchgessner
APPCD Project Officer
U.S. EPA
Date
William Chatterton Date
Project Manager
Greenhouse Gas Technology Center
Southern Research Institute
Robert Wright Date
APPCD Quality Assurance Manager
U.S. EPA
Richard Adamson Date
Quality Assurance Manager
Greenhouse Gas Technology Center
Southern Research Institute
Test Plan Final: December 2004
-------
(this page intentionally left blank)
-------
Distribution List
White Sands, LLC.
Bret Christiansen
U.S. EPA
David Kirchgessner
Bob Wright
Southern Research Institute
Stephen Piccot
Timothy Hansen
William Chatterton
Richard Adamson
Southwest Research Institute
Robert Fanick
Mike Van Hecke
-------
List of Acronyms and Abbreviations
ADQ
APPCD
Bhp-hr
BSFC
CFR
CH4
CO
C02
cov
CVS
DEER
DQIG
DQO
EPA-ORD
ETV
Fe
FS
FTP
GHG Center
GVP
NIST
NO2
NOX
PEA
PM
PM2.5
ppmv
QA/QC
QMP
QPP
QSM
SOP
SwRI
THC
TQAP
TSA
audit of data quality
Air Pollution Prevention and Control Division
brake horsepower-hour
brake specific fuel consumption
Code of Federal Regulations
methane
carbon monoxide
carbon dioxide
coefficient of variation
constant volume sampling
Department of Engine and Emissions Research
data quality indicator goals
data quality objective
Environmental Protection Agency Office of Research and Development
Environmental Technology Verification
iron
full scale
Federal Test Procedure
Greenhouse Gas Technology Center
ETV Generic Verification Protocol
National Institute of Standards and Technology
nitrogen dioxide
blend of NO, NO2, and other oxides of nitrogen
performance evaluation audit
paniculate matter
paniculate matter with diameter of 2.5 microns or less
parts per million by volume
quality assurance / quality control
quality management plan
quality policy and procedures
quality systems manual
standard operating procedure
Southwest Research Institute
total hydrocarbons (as carbon)
test and quality assurance plan
technical systems audit
-------
TABLE OF CONTENTS
1.0 INTRODUCTION 1
1.1 BACKGROUND 1
1.2 SwRI TESTING QUALIFICATIONS 2
1.3 ORGANIZATION OF THIS TQAP 3
1.4 REFERENCED SwRI QUALITY DOCUMENTS 3
2.0 TEST DESCRIPTION AND TEST OBJECTIVES 5
2.1 TECHNOLOGY DESCRIPTION 5
2.2 TEST DESCRIPTION 5
2.2.1 Overview 5
2.3 TEST ENGINE AND FUEL SELECTION AND SPECIFICATIONS 7
2.3.1 Test Engine 7
2.3.2 Test Fuel 7
2.4 BASELINE ENGINE PREPARATION 8
2.4.1 Engine Oil Change 8
2.5 FUEL MODIFICATION WITH THE CLEANBOOST TECHNOLOGY 8
2.6 ENGINE TESTING PROCEDURES 8
2.6.1 Break-in Periods 8
2.6.2 Engine Mapping. 9
2.6.3 Test Cycle 9
2.6.4 Engine Preconditioning 10
2.6.5 Emissions and Fuel Consumption Testing 10
2.7 TEST ORGANIZATION AND RESPONSIBILITIES 11
2.7.1 EPA 12
2.7.2 Southern Research Institute 13
2.7.3 SwRI 14
2.7.4 White Sands 14
2.8 SCHEDULE AND MILESTONES 15
2.9 DOCUMENTATION AND RECORDS 15
3.0 DATA QUALITY OBJECTIVES 17
3.1 DATA QUALITY OBJECTIVES 17
4.0 SAMPLING AND ANALYTICAL PROCEDURES 21
4.1 EXHAUST GAS SAMPLING SYSTEM 21
4.2 FILTER WEIGHING 22
4.3 GASEOUS ANALYZERS 22
4.4 PM2.5 AND FE DETERMINATIONS 22
5.0 SAMPLE HANDLING AND CUSTODY 23
6.0 DATA QUALITY INDICATOR GOALS AND QA/QC CHECKS 23
7.0 INSTRUMENT CALIBRATION AND FREQUENCY 27
8.0 DATA ACQUISITION AND MANAGEMENT 28
9.0 INTERNAL AND EXTERNAL AUDITS 28
9.1 TECHNICAL SYSTEMS AUDIT 28
-------
9.2 PERFORMANCE EVALUATION AUDITS 29
9.3 AUDIT OF DATA QUALITY 29
9.4 EXTERNAL ASSESSMENTS 29
9.5 INTERNAL ASSESSMENTS 29
10.0 CORRECTIVE ACTION 30
11.0 DATA REDUCTION, REVIEW, VALIDATION, AND REPORTING 30
12.0 REPORTING OF DATA QUALITY INDICATORS 30
13.0 DEVIATIONS FROM GVP 30
14.0 REFERENCED QUALITY DOCUMENTS 31
14.1 EPA-ETV 31
14.2 GHGTC 31
14.3 SOUTHWEST RESEARCH INSTITUTE 32
Reference Lubricant Fuel Economy 8
Fuel Economy Change 9
Appendix A: Test Log Forms and Checklists
Appendix B: Baseline Emissions and Fuel Economy Normalization Procedure
-------
1.0 INTRODUCTION
1.1 BACKGROUND
The U.S. Environmental Protection Agency's Office of Research and Development (EPA-ORD) operates
the Environmental Technology Verification (ETV) program to facilitate the deployment of innovative
technologies through performance verification and information dissemination. The ETV program's goal
is to further environmental protection by substantially accelerating the acceptance and use of improved
and innovative environmental technologies. Congress funds ETV in response to the belief that there are
many viable environmental technologies that are not being used for the lack of credible third-party
performance data. The performance data developed under this program will allow technology buyers,
financiers, and permitters in the United States and abroad to make more informed decisions regarding
environmental technology purchase and use.
The Greenhouse Gas Technology Center (GHG Center) is one of six ETV organizations. EPA's partner
verification organization, Southern Research Institute (Southern), manages the GHG Center. The GHG
Center conducts verification testing of promising GHG mitigation and monitoring technologies. It
develops verification protocols, conducts field tests, collects and interprets field and other data, obtains
independent peer-review input, and reports findings. The GHG Center conducts performance evaluations
according to externally reviewed verification Test and Quality Assurance Plans (TQAPs) and established
protocols for quality assurance (QA).
Volunteer stakeholder groups guide the GHG Center's verification activities. These stakeholders advise
on specific technologies most appropriate for testing, help disseminate results, and review TQAPs and
technology Verification Reports. National and international environmental policy, technology, and
regulatory experts participate in the GHG Center's Executive Stakeholder Group. The group also
includes industry trade organizations, environmental technology finance groups, governmental
organizations, and other interested parties. Industry-specific stakeholders peer-review key documents
prepared by the GHG Center and provide verification testing strategy guidance in those areas related to
their expertise.
One sector of significant interest to GHG Center stakeholders is transportation - particularly technologies
that result in fuel economy improvements. The Department of Energy reports that in 2001, "other trucks"
(all trucks other than light-duty trucks) consuming diesel fuel emitted approximately 72.5 million metric
tons of carbon dioxide (CO2). These emissions increase to 107.5 million metric tons when considering all
diesel vehicles in the transportation sector. Small fuel efficiency or emission rate improvements are
expected to have a significant beneficial impact on nationwide greenhouse gas emissions.
White Sands, LLC. of Bluffdale, Utah markets the CleanBoost combustion catalyst, a fuel additive that
can be used in mid to heavy duty diesel engines as well as various other applications fueled with biodiesel
and heating oil. The CleanBoost additive can act as a detergent in older engines removing carbon
deposits and improving performance, and can catalytically improve fuel combustion in newer engines.
According to White Sands, improved fuel economy and reduced emissions are the primary benefits of
using this technology.
White Sands wishes to verify performance of the CleanBoost technology for reductions in fuel
-------
consumption and emissions on a heavy-duty diesel engine. CleanBoost is a suitable verification
candidate considering its potentially significant beneficial environmental quality impacts and ETV
stakeholder interest in verified transportation sector emission reduction technologies.
This test will be conducted following guidelines provided in a ETV Generic Verification Protocol (GVP)
developed by the Air Pollution Control Technology Verification Center: "Environmental Technology
Verification Protocol - Determination of Emissions Reductions Obtained by Use of Alternative or
Reformulated Liquid Fuels, Fuel Additives, Fuel Emulsions, and Lubricants for Highway and Nonroad
Use Diesel Engines and Light Duty Gasoline Engines and Vehicles ". The GVP makes use of the Federal
Test Procedure (FTP) as listed in 40 CFR Part 86 for highway engines as a standard test protocol to
evaluate fuel modifications (FMs). This verification will include evaluation of the CleanBoost
technology as an immediate-effect FM only and will not include evaluation as a cumulative-effect FM.
Performance will be assessed using the GVP test sequence by comparing the fuel consumption and
emission rates measured on a heavy-duty test engine before and after application of the CleanBoost
additive. Verification testing will be directed by the GHG Center. The tests will take place at Southwest
Research Institute's (SwRI) Department of Engine and Emissions Research (DEER) in San Antonio, TX.
The test program is described in the following sections. Any deviations from the GVP are noted in
Section 13 of this TQAP.
This TQAP specifies verification parameters and the rationale for their selection. It contains the
verification approach, data quality objectives (DQOs), and Quality Assurance/Quality Control (QA/QC)
procedures. It will also guide test implementation, document creation, data analysis, and interpretation.
This TQAP has been reviewed by White Sands, SwRI, and the EPA-ETV QA Manager. The EPA-
APPCD Project Officer provided final approval of the TQAP. The TQAP meets the requirements of the
GHG Center's Quality Management Plan (QMP) once approved and signed by the responsible parties
listed on the front of this document. The TQAP is available on GHG Center internet site at www.sri-
rtp.com and the ETV program site at www.epa.gov/etv.
The GHG Center will prepare a verification report and verification statement upon field test completion.
The same organizations listed above will review the verification report and statement, followed by EPA-
ORD technical review. The GHG Center Director and EPA-ORD Laboratory Director will sign the
verification statement when this review is complete and the GHG Center will post the final documents as
described above.
1.2 SWRI TESTING QUALIFICATIONS
The GHG Center has selected SwRI to conduct the testing for this verification. The following describes
the accreditations and registrations of SwRI relevant to this TQAP.
The SwRI DEER is registered to International Organization for Standardization 9002 "Model for Quality
Assurance in Production and Installation." This independently assessed quality system provides the basis
for quality procedures that are applied to every project conducted in the DEER. DEER is accredited to
ISO/IEC Guide 25 "General Requirements for the Competency of Calibration and Testing Laboratories"
and EN 45001, "General Criteria for the Operation of Test Laboratories." The American Association for
Laboratory Accreditation Certificate Number 0702-01 accredits DEER to perform evaluations of
automotive fluids, fuel emissions, automotive components, engine and power-train performance and
durability using stationary engine dynamometer test stands (light-duty, non-road, and heavy-duty) and
-------
vehicle dynamometer facilities, and automotive fleets (see http://www.a21a2.net/scopepdf/0702-01.pdf).
The certificate accredits DEER to use specific standards and procedures, including dynamometer
procedures for hydrocarbons, carbon monoxide, oxides of nitrogen, and particulate matter. DEER has
also: (1) achieved Ford Tier 1 status for providing engineering services, (2) received the Ford Ql Quality
Award and the Ford Customer-Driven Quality Award, and (3) maintains its status as a Caterpillar-
certified supplier.
For prior ETV tests, EPA has reviewed the DEER quality system and verified that the information
conforms to the specific required elements of the [EPA Requirements for Quality Assurance Project
Plans], the ETV QMP, and the general requirements of the GVP.
1.3 ORGANIZATION OF THIS TQAP
This TQAP addresses ETV technology testing at SwRI under the applicable GVPs. It is deliberately
organized to parallel the structure of EPA QA/R-5. Since all laboratory data will be generated by SwRI,
much of this TQAP also parallels the SwRI Test/QA Plan for the Verification Testing of Diesel Exhaust
Catalysts, Particulate Filters, and Engine Modification Control Technologies for Highway and Nonroad
Use Diesel Engines (Version 1.0, April 8, 2002; SwRI QPP ) which was developed based on the GVPs.
The referenced SwRI QPP was developed for ETV testing under the current GVPs and is posted on the
ETV website. Differences between the SwRI QPP and this TQAP reflect organizational differences and
the specific role of the GHG center as the verification organization on this test. This TQAP also contains
test-specific details of the CleanBoost technology, its implementation, verification parameters, schedule,
and test design. These details are generally inserted in the appropriate sections of the main text rather than
in a test-specific attachment to the existing SwRI QPP.
This TQAP also describes testing under the framework of the GVPs and the relevant FTP (from 40 CFR
86 Subpart N for highway engines), and both documents will be cited as applicable by reference where
such citation is clear. This TQAP also describes how the FTP will be specifically implemented for this
verification.
1.4 REFERENCED SWRI QUALITY DOCUMENTS
Several relevant internal SwRI documents will be incorporated by reference in this TQAP, including the
(1) DEER Quality System Manual (QSM), (2) Quality Policy and Procedures (QPPs), and (3) Standard
Operating Procedures (SOPs). These internal quality documents, unlike the GVP and FTP references, are
considered proprietary to SwRI and are not publicly available. However, they will be made available for
review during the on-site assessment of the DEER technical and quality systems, and for test-specific on-
site audits by the GHG or EPA QA personnel. Several of the referenced SOPs were previously reviewed
by GHG Center staff as part of a previous verification test and found adequate by the GHG Center QA
manager as discussed in the TSA report for that test. Certain sections of this document reference specific
SwRI quality documents that describe DEER's conformance with specific QPP-required elements. These
references do not supersede the applicable GVPs and FTP citations, but are included to document the
specific implementations of these directions by SwRI staff.
-------
-------
2.0 TEST DESCRIPTION AND TEST OBJECTIVES
2.1 TECHNOLOGY DESCRIPTION
The CleanBoost combustion catalyst is a diesel fuel additive that can be used in heavy duty diesel engines
used in the transportation sector, as well as various other applications using diesel, biodiesel, and heating
oil. CleanBoost consists of organo-ferrous compounds in a petroleum solvent (naphtha) base. The
additive performs several functions, according to White Sands. In older engines, the additive can have a
detergent effect, removing carbon soot deposits in the engine, thus improving engine performance. In
well-maintained engines, the additive acts catalytically to improve the fuel combustion. White Sands
claims that the catalytic action helps break down long chain hydrocarbons into smaller, more readily
combustible molecules, lowers the temperature of combustion, provides more complete combustion, and
reduces soot formation and buildup. The additive is utilized at a mixing ratio of 1:3000, and requires a
short break in time to obtain the full effect in most engines due to the detergent action of the additive.
White Sands claims that enhanced fuel economy and reduced emissions are the benefits of using this
technology.
The technology was tested in May 2004 by SwRI using the SAE J1321 fuel consumption test procedure.
Results of the testing indicate an average 3% increase in fuel economy resulting from the use of the
CleanBoost additive on a diesel long-haul truck. Additional testing and case studies indicate emission
reductions from use of CleanBoost on the order of 20% for CO and 14% for hydrocarbons. Reductions in
opacity and particulate emissions have also been observed, with no increases in NOX emissions. Case
studies on use of CleanBoost with biodiesel blends have indicated significant reductions in NOX
emissions.
2.2 TEST DESCRIPTION
2.2.1 Overview
This TQAP describes testing of the CleanBoost technology following the guidelines detailed in the
previously referenced GVPs. Section 5.1 of the fuel additives GVP provides a detailed analysis of test
design and data analysis for fuel modification technologies. In it, the inadequacy of a simple comparison
of a baseline test with tests conducted with treated fuels is described. Specifically, the GVP describes
how an ETV test for fuel modifications must be designed to evaluate emission reductions with a likely
changing baseline emissions profile. To address this, the GVP provides test sequences between base fuels
and treated fuels. The test sequences vary according to fuel additive type and purpose, but in general
require a series of baseline tests, followed by a series of tests with treated fuel, followed by a second
series of baseline tests.
This general approach was used by the GHG Center previously on a similar verification (Test and Quality
Assurance Plan—ConocoPhillips Fuel-Efficient High-Performance SAE 75W90 Rear Axle Gear
Lubricant, SRI/USEPA-GHG-VR-29) and will be followed for this verification. During the previous
-------
verification, baseline fuel economy testing was conducted before and after the testing conducted with the
candidate technology (in that case, an axle lubricant). In general, test results from the before and after
baseline tests were used to develop a normalized baseline fuel economy, which was then compared to fuel
economy achieved with the candidate technology. This approach will be used for this verification to
evaluate changes in engine emissions and fuel economy attributable to CleanBoost. Appendix B provides
a detailed description of this approach and how this analysis was conducted for the previous verification.
During this verification, the exhaust from the engine will be analyzed for emissions of NOX, PM, PM2.5,
THC, CO, CO2, CH4, and Fe (since CleanBoost contains ferrous compounds). Additional measurements
and calculations will be used to determine fuel economy of the engine over a specified test cycle. The test
procedure will consist of the following, at a minimum, based on the requirements of 40 CFR 86 Subpart
N (detailed descriptions of each test phase are provided in Sections 2.2 through 2.4):
1. Install test engine on dynamometer, change engine fluids and stabilize.
2. Flush fuel system and operate the test engine for 25 hours on ULSD reference diesel fuel.
3. Perform engine mapping and preconditioning, followed by an overnight soak.
4. Complete heavy-duty transient FTP cycles consisting of one cold-start and three hot-start tests
each. Sample engine exhaust.
5. Evaluate the engine operational parameters, emissions, and fuel consumption.
6. Complete additional FTP cycles as needed to improve data quality and credibility.
7. Prepare CleanBoost treated reference fuel at recommended dosage rate.
8. Flush fuel system and operate the test engine for 25 hours on treated fuel.
9. Perform engine mapping and preconditioning, followed by an overnight soak.
10. Complete heavy-duty transient FTP cycles consisting of one cold-start and three hot-start tests
each. Sample engine exhaust.
11. Evaluate the engine operational parameters, emissions, and fuel consumption.
12. Complete additional FTP cycles as needed to improve data quality.
13. Change engine fluids and flush fuel system and operate the test engine for 25 hours on baseline
reference fuel.
14. Perform engine mapping and preconditioning, followed by an overnight soak.
15. Complete heavy-duty transient FTP cycles consisting of one cold-start and three hot-start tests
each. Sample engine exhaust.
16. Evaluate the engine operational parameters, emissions, and fuel consumption.
17. Complete additional FTP cycles as needed to improve data quality.
18. Evaluate baseline and treated fuel test results for statistically significant changes in operational
parameters, emission rates, and fuel consumption.
19. Evaluate data quality as specified in this test plan.
The verification test generally requires operation of a test engine on an engine dynamometer. The engine
dynamometer simulates operating conditions of the engine by applying loads to the engine and measuring
the amount of power that the engine can produce against the load. The engine is operated on the
dynamometer over a simulated duty cycle that mimics a typical on-road heavy-duty vehicle. This is the
"transient" cycle heavy-duty FTP specified in 40 CFR 86.1333.
Exhaust emissions from the engine are collected through a constant volume sampling (CVS) system and
then analyzed to determine emission concentrations. An adjustable-speed turbine blower in the CVS
dilutes the exhaust with ambient air while the vehicle operates on the dynamometer. This dilution
prevents the exhaust moisture from condensing and provides controllable sampling conditions. A sample
-------
pump and a control system transfers diluted exhaust to emission analyzers, sample bags, or particulate
sampling systems (filters). Samples are collected at constant sampling rates.
During each test run, the following parameters are measured:
Dynamometer Operations:
• Speed
• Torque (load)
• Test cell temperature, humidity, and pressure
Constant Volume Sampling System Conditions:
• System pressure and temperature
• Volumetric flow rate
Engine exhaust components:
• CO, CO2, NOX, and THC concentrations
• PM, PM2.5, and Fe concentrations
2.3 TEST ENGINE AND FUEL SELECTION AND SPECIFICATIONS
2.3.1 Test Engine
The diesel engine to be used in this test program is a Cummins ISB 305 turbocharged engine
manufactured in 2004. This engine was selected for testing because it represents a large segment of
heavy-duty diesel engines currently on the road for which the CleanBoost technology is intended.
CleanBoost will also be applicable to other types of heavy duty diesel engines. The test engine is located
at the SwRI facility and SwRI has verified that the engine is new and is operating reasonably within
original OEM specifications.
The ISB 305 is a 5.9 liter displacement inline six-cylinder diesel engine. The engine is rated at 305 brake
horsepower (bhp) at 2900 rpm and has a peak torque of 600 Ib-ft at 1600 rpm. Prior to this verification
test, the engine will be used for the CleanBoost fuel additive testing required under Section 21 l(b) of the
Clean Air Act. As part of the requirements of the 21 l(b) test, the engine will be operated for a duration of
125 hours for engine break-in.
2.3.2 Test Fuel
Testing will use certified ultra low sulfur diesel (ULSD) test fuel with sulfur content below 15 ppm. This
reference fuel was selected because, with future ULSD mandates looming, it represents a potential
majority of the intended CleanBoost market. With the exception of low sulfur content, this fuel has the
same properties of EPA standard No.2 diesel. SwRI will provide the ULSD for this test, along with a
certificate of analysis. The GHG Center will review fuel analyses and verify the fuel to be within
specifications before the start of testing.
-------
2.4 BASELINE ENGINE PREPARATION
2.4.1 Engine Oil Change
At the conclusion of the Section 21 l(b) testing, the engine's oil will be changed prior to baseline testing
using the standard manufacturer oil change procedure. This ensures that the engine oil will not impact the
performance of the engine from the baseline to treated fuel test. A suitable grade of engine oil will be
used based on manufacturer specifications.
The technicians performing this maintenance will document the oil changes, including the date and
quantity and type of oil used. Documentation will be signed by the technicians and copies provided to the
field team leader. The same engine oil will be used throughout the initial baseline and treated fuel testing.
Prior to the final baseline testing (after completion of the testing with treated fuel) a second oil change
will be conducted to minimize baseline engine drift.
2.5 FUEL MODIFICATION WITH THE CLEANBOOST TECHNOLOGY
The test fuel will be treated by administering the CleanBoost fuel additive to the baseline ULSD reference
fuel after baseline testing is complete. A White Sands representative will be present to confirm that
proper additive dosing is performed, that the proper break-in is completed, and to provide oversight and
consultation during the administering of the CleanBoost technology. Flushing of the engine with treated
fuel will begin after White Sands approves the CleanBoost dosing. All dosing and additive
administration activities will be decided the GHG Center field team leader.
2.6 ENGINE TESTING PROCEDURES
The test engine will be installed on the engine dynamometer after engine preparations are completed. The
engine test procedure is described in the following sections.
2.6.1 Break-in Periods
The baseline engine will go through a break-in period to ensure proper break-in of the new engine oil and
sufficient flushing with the baseline reference fuel. This allows the engine to stabilize and eliminates any
effects of oil break-in or previous fuel carryover on engine performance. A break-in period of 25 hours is
specified here since only fresh oil is added to the engine and no other mechanical changes will be
performed on the baseline engine.
Break-in is completed by operating the engine at specified conditions for a specified time period. The
cycle operates at various engine conditions, including idle, peak torque, rated speed, and high idles. The
actual break-in time for the baseline tests will be documented by SwRI.
After the baseline testing is completed and the fuel additive is administered, a second 25-hour break-in
period will be conducted to fully flush the baseline fuel from the test engine and to stabilize engine
operation on the treated fuel. The actual break-in time, operating conditions, and test cycle will be
documented by SwRI. The break-in/flushing process will be repeated a third time using reference
-------
baseline fuel after completion of the testing with the treated fuel (before starting the second set of baseline
tests).
2.6.2 Engine Mapping
Engine mapping is completed to generate a torque curve for the test engine by running the engine at full
throttle at increasing engine speed from curb idle through the manufacturer's rated speed. The engine
torque is measured at each speed. The torque curve is subsequently used to generate data for the transient
test cycle for that specific engine. The engine mapping procedure follows the procedure specified at 40
CFR 86 SubpartN, Sections 86.1332 and 86-1333.
Engine mapping will be completed after the break-in procedure is completed for both the baseline and
treated fuels. The baseline map obtained will be compared to the manufacturer-specified engine map.
Significant differences between the two maps will be investigated. Corrective actions will be
implemented once the cause of the discrepancy is identified. The required corrective action will be
completed prior to accepting the engine for further testing. The engine may be labeled as unacceptable
for the test if fundamental problems with the engine are identified based on the engine map. A new test
engine would then be located.
In order to allow a fair comparison of engine performance with the baseline and treated fuels, the torque
curve developed during the baseline mapping will be used to develop the FTP duty cycle for all testing
periods. Mapping results will be reported for both the baseline and treated test fuels so that potential
users can see changes in engine performance using the treated fuel (e.g., power output may be different
with the modified fuel at the same engine speed levels as the baseline fuel).
2.6.3 Test Cycle
The test engine is operated on the dynamometer over the transient heavy-duty FTP driving cycle the
specified in 40 CFR 86.1333 that simulates the operation of a typical on-road heavy-duty vehicle. The
FTP cycle takes into account the operation of a variety of heavy-duty trucks and buses, and includes
simulation of traffic on roads and expressways in and around cities. The average speed is about 30 km/h
and the equivalent distance traveled is 10.3 km. The cycle lasts 1200 s [dieselnet:
http://www.dieselnet.com/standards /cycles/ftp_trans.html].
The test cycle is specified as a normalized cycle. The data points specified in the FTP are the percent of
maximum torque and speed over time. The specific transient cycle for the test engine is calculated based
on these values and the engine mapping values. One complete FTP cycle consists of two test segments.
The first is a "cold-start test" completed after the engine has been "soaked" (not operating) for a specified
time period (overnight). The second period is a "hot-start" test. This is the same cycle as the cold start
test, begun 20 minutes after the completion of the cold-start test, while the engine is still "hot".
The specific FTP cycle used for both the baseline and treated engines will be calculated for this
verification test using the initial baseline engine mapping results. This ensures that identical test cycles
are utilized.
Testing of each engine configuration will consist of a single cold-start test, followed by the required 20-
minute soak period, and a minimum of three subsequent hot-start tests. A 20-minute soak period is
required between each hot-start test.
-------
2.6.4 Engine Preconditioning
The test engine will be preconditioned after engine mapping is completed. Preconditioning is completed
by running the engine through the FTP test cycle that it will be seeing for the actual test procedure. Both
the baseline and treated engine will be preconditioned for this test by running the engine through the
transient FTP cycle three times. The transient cycles, each 20 minutes long, are run concurrently without
any intermittent soak period. Once the preconditioning runs are completed, the engine is turned off and
allowed to "soak" overnight. The length of the soak period between the end of preconditioning and
beginning of test runs will be recorded and will be approximately the same for both the baseline and
treated test engine.
2.6.5 Emissions and Fuel Consumption Testing
The emissions and fuel consumption tests will be completed after the overnight soak following the
preconditioning runs. The test runs will consist of operating the test engine over the specified FTP test
cycle for one cold-start test, and a minimum of three hot-start tests for both the baseline and treated
engine. Additional hot-start tests may be added depending on the data quality of the initial test runs and
upon agreement between all parties and funding agencies involved in the test campaign. Total minimum
test duration is two hours and twenty minutes, consisting of one cold-start test, three hot-start tests, and
three soak periods.
The composite brake-specific fuel consumption (BSFC) evaluated during the test is a measure of engine
efficiency and is a primary verification parameter for this test series. BSFC is the ratio of the engine fuel
consumption to the engine power output and has units of grams of fuel per kilowatt-hour (g/kWh) or
pounds mass of fuel per brake horsepower-hour (Ib/Bhp-hr). The calculation of composite BSFC is
shown at 40 CFR 86.1342-90. The equation and supporting parameters are:
c
BSFC= 17/ ; - - Equation 1
where: BSFC = brake-specific fuel consumption in pounds of fuel per brake horsepower-hour,
Ibs/Bhp-hr
Mc = mass of fuel used by the engine during the cold start test, Ibs
M h = mass of fuel used by the engine during the hot start test, Ibs
Bhp-hrc = total brake horsepower-hours (brake horsepower integrated with respect to
time) for the cold start test
Bhp-hrh = total brake horsepower-hours (brake horsepower integrated with respect to
time) for the hot start test
The Bhp-hr values for each test are calculated using the engine torque and speed data measured on the
dynamometer. The mass of fuel, M, used during each test is calculated via a carbon balance method
using the emission rates and fuel properties determined during testing. These rather complex calculations
are specified in 40 CFR 86.1342-90 and not repeated here. Generally, the calculations rely on the
measured engine exhaust mass emissions of THC, CO, and CO2 and the measured test fuel carbon weight
fraction, specific gravity, and net heating value.
10
-------
These fuel properties are cited on the fuel certificate of analyses and are determined using the following
methods:
• Specific gravity-ASTMD1298
• Carbon weight fraction - ASTM D3343
• Net heating value - ASTM D3348
During the previous ConocoPhilips verification, separate volumetric and gravimetric cross-checks were
conducted on the fuel consumption determinations. Specifically, fuel consumption was determined
volumetrically and gravimetrically during each test for comparison with the carbon balance fuel
consumption determinations. The test and quality assurance plan (SRI/EPA-GHG-QAP-28) specified
that a coefficient of variation (COV) of greater than ±0.3 would indicate a potential bias in the carbon
balance method. Results presented in the verification report (SRI/EPA-GHG-VR-29) showed that the
COVs averaged 0.15 for both the volumetric and gravimetric checks. Both cross-checks had absolute
differences higher than the carbon balance method (average 0.16 and 0.23 mpg higher for the volumetric
and gravimetric checks, respectively), but since both were consistently high and the COVs were
favorable, no further investigations were conducted. Since the same carbon balance procedures and
instrumentation will be used for this verification, these cross-checks will not be repeated here.
Engine exhaust gas will be analyzed during each test to determine mass emissions of NOX, PM, PM2.5,
THC, CO, CO2, CH4, and Fe. Engine and dynamometer operating conditions will be recorded. Sampling
system, emission analyzer, and test cell operations will also be monitored.
Each test run will be followed by evaluation of data quality in accordance with the requirements of
Section 3. Achievement of all data quality indicator goals and FTP requirements will allow the field team
leader to declare a run valid. A test run where required data quality indicator goals are not met will cause
the test run to be invalidated and repeated immediately (if a hot-start).
2.7 TEST ORGANIZATION AND RESPONSIBILITIES
Project management responsibilities are divided among the EPA, Southern, and SwRI staff as shown in
Figure 2-1 and described in the following sections.
11
-------
Robert Wright
US EPA APPCD
QA Manager
David Kirchgessner
US EPA
APPCD Project
Officer
Timothy Hansen
GHG Center
Deputy Director
Bill Chatterton
GHG Center
Project Manager
and Field Team
Leader
Richard Adamson
GHG Center QA
Manager
Bret Christiansen
White Sands
Robert Fanick
SwRI Project
Manager
Mike Van Hecke
SwRI QA Manager
Figure 2-1. Project Organization
2.7.1 EPA
2.7.1.1 Project Management
The EPA Project Manager, David Kirchgessner, has overall EPA responsibility for the GHG Center. He is
responsible for obtaining EPA's final approval of project TQAPs, and the verification statement and
report from the ORD Director and the ETV Program Manager.
2.7.1.2 Quality Manager
The EPA Quality Manager for the GHG Center is Robert Wright of EPA's Air Pollution Prevention and
Control Division (APPCD). His responsibilities include:
• Communicate quality systems requirements, quality procedures, and quality issues to the EPA
Project Manager and the GHG Project Manager;
• Review and approve GHG Center quality systems documents to verify conformance with the
quality provisions of the ETV quality systems documents;
• Conduct performance evaluations and audits of verification tests, as appropriate;
• Provide assistance to GHG Center personnel in resolving QA issues;
• Review and approve this TQAP;
• Review and approve the verification report and statement for each technology tested under this
TQAP; and
12
-------
2.7.2 Southern Research Institute
2.7.2.1 GHG Center Deputy Director
Southern's GHG Center has overall planning responsibility and will ensure successful verification test
implementation. The GHG Center will:
• coordinate all activities;
• develop, monitor, and manage schedules; and
• ensure the achievement of high-quality independent testing and reporting.
Mr. Timothy Hansen is the GHG Center Deputy Director. He will ensure that staff and resources are
sufficient and available to complete this verification. He will review the TQAP to ensure consistency
with ETV operating principles. He will oversee GHG Center staff activities and provide management
support where needed. Mr. Hansen will sign the verification statement along with the EPA-ORD
Laboratory Director.
2.7.2.2 GHG Center Project Manager
Mr. Bill Chatterton will serve as the Project Manager for the GHG Center. His responsibilities include:
• drafting the TQAP and verification report;
• overseeing the field team leader's data collection activities, and
• ensuring data quality objectives (DQOs) are met prior to completion of testing.
The project manager will have full authority to suspend testing should a situation arise that could affect
the health or safety of any personnel. He will also have the authority to suspend testing if the DQIGs
described in Section 3.0 are not being met. He may resume testing when problems are resolved in both
cases. He will be responsible for maintaining communication with White Sands, SwRI, EPA, and
stakeholders.
2.7.2.3 GHG Center Field Team Leader
Mr. Chatterton will also serve as the Field Team Leader. He will supervise all SwRI testing activities to
ensure conformance with the TQAP. Mr. Chatterton will assess test data quality and will have the
authority to repeat tests as determined necessary to ensure achievement of data quality goals. He will
perform on-site activities required for data quality audits under the direction of the GHG Center QA
Manager and perform other QA/QC procedures as described in Section 3.0. He will also communicate
with the SwRI Program and Quality Managers to coordinate the internal audit activities of the SwRI
Quality Manager with those of the GHG Center. Mr. Chatterton will communicate test results to the
deputy director at the completion of each test run. The field team leader and deputy director will then
determine if sufficient test runs have been conducted to report statistically valid fuel economy
improvements.
13
-------
2.7.2.4 GHG Center Quality Manager
Southern's QA Manager, Mr. Richard Adamson, is responsible for ensuring that all verification tests are
performed in compliance with the QA requirements of the GHG Center QMP, GVPs, and TQAP. He has
review this TQAP. He has reviewed the applicable elements of the SwRI Quality System and approved
the quality requirements for implementation by SwRI technical and quality staff on this test. He will also
review the verification test results and ensure that applicable internal assessments are conducted as
described in Section 9.5. He will reconcile the DQOs and MQOs at the conclusion of testing and will
conduct or supervise the ADQ. In addition, the QA manager will review the results of the PEA that is
administered by the field team leader. Mr. Adamson will report all internal reviews, DQO reconciliation,
the ADQ, the PEA, and any corrective action results directly to the GHG Center Deputy Director who
will provide copies to the project manager for corrective action as applicable and citation in the final
verification report. He will review and approve the final verification report and statement. He is
administratively independent from the GHG Center Deputy Director.
2.7.3 SwRI
2.7.3.1 SwRI Program Manager
Mr. Bob Fanick is the SwRI Program Manager for this test program. He will be the primary contact for
SwRI and will be responsible for set-up and testing of the engine. He will also review the TQAP and
verification report.
2.7.3.2 SwRI Quality Manager
Mr. Mike Van Hecke plays a central role in the introduction, implementation, and consistent application
of continuous quality improvement at the DEER. He fulfills the role as quality management
representative for SwRI and conducts audits of all pertinent quality standards to ensure compliance. He is
administratively independent of the unit generating the data and conducts QA activities as specified in
SwRI's internal SOPs. He will conduct these internal QA activities on this test as described in Section 9
and report results to the GHG Center QA Manager. However, these activities do not replace or eliminate
the need for the GHG Center internal QA reviews and activities outlined in Section 2.7.2.4 above.
2.7.3.3 Support Personnel
All persons supporting the project will be qualified as prescribed by SwRI QPP 10 (Training and
Motivation).
2.7.4 White Sands
Mr. Bret Christiansen will serve as White Sands' primary contact person. Mr. Christiansen will provide
technical support for the CleanBoost technology including instructions for product dosing, application,
and break-in. Mr. Christiansen will review the TQAP and verification report and provide written
comments. Mr. Christiansen or a designated White Sands representative will be present during the
verification testing to insure proper application of the CleanBoost additive. White Sands is also
responsible for providing the CleanBoost additive to the test facility in sufficient quantities to complete
the entire verification test.
14
-------
2.8 SCHEDULE AND MILESTONES
The tentative schedule of activities for testing the CleanBoost technology is as follows:
Verification Test Plan Development Dates
GHG Center Internal Draft Development October 15, - November 10, 2004
White Sands Review/Revision November 10 - 17, 2004
EPA TQAP Review and Approval November 17 - December 17, 2004
Final Document Posted December, 2004
Verification Testing and Analysis Dates
Preliminary Teleconference and Project Review Mid-December, 2004
Testing January, 2005 (exact dates to be
determined)
Data Validation and Analysis January, 2005
Verification Report Development Dates
GHG Center Internal Draft Development January 3 - 28, 2005
White Sands Review and Report Revision February 1-15, 2005
EPA and Industry Peer-Review February 18-28, 2005
Final Report Revision and EPA Approval March 2005
Final Report Posted March 31,2005
2.9 DOCUMENTATION AND RECORDS
Test-specific documentation and records generated by SwRI will be processed as specified in:
• SwRI QPP 03 (Document Preparation and Control);
• SwRI QPP 07 (Testing and Sample Analysis); and
• SwRI QPP 14 (Quality Records).
Copies of results and supporting data will be transferred to the GHG Center and managed according to the
GHG Center QMP. See Section 8 for details of test data acquisition and management. SwRI, in
accordance with Part A, Sections 5.1 and 5.3 of EPA's QMP, will retain all test-specific documentation
and records for seven years after the final payment of the agreement between SwRI and the GHG ETV
Center. Southern will retain all verification reports and statements for seven years after final payment of
the agreement between Southern and EPA.
15
-------
16
-------
3.0 DATA QUALITY OBJECTIVES
3.1 DATA QUALITY OBJECTIVES
DQOs are statements about the planned overall accuracy of the verification parameters. Three documents
provide the basis for this subsection: (1) the [GVP], (2) the Test and Quality Assurance Plan—
ConocoPhillips Fuel-Efficient High-Performance SAE 75W90 Rear Axle Gear Lubricant (SRI/USEPA-
GHG-TQAP-28), and (3) the Test and Quality Assurance Plan—Universal Cams, LLC Dynamic Cam
Diesel Engine Retrofit System (SRI/USEPA-GHG-TQAP-31). An abbreviated discussion of DQO
development is presented here.
The primary verification parameter for this technology is reduction in BSFC. Improvement in BSFC will
be expressed as the mean change, or delta (A), between results from the baseline and CleanBoost treated
fuel tests. Based on tests previously conducted by White Sands, an approximate 3 percent decrease in
BSFC is expected. Therefore, the DQO for this parameter is to demonstrate a statistically significant
BSFC delta of 3 percent or greater. This section provides a brief description of the data analysis and
statistical procedures used here to demonstrate if this DQO is met. More detailed presentations of the
statistical analyses that will be used are presented in the reference materials cited above and Appendix B
of this TQAP.
This verification also includes determination of NOX, CO, THC, PM, PM2.5, and Fe emissions as
secondary verification parameters. These emissions tend to be much lower than any applicable standards,
and their higher measurement variability (because of low absolute values) lead to large A determination
errors. Therefore, this verification will not adopt explicit engine emissions DQOs analogous to the BSFC
DQO. The implicit DQOs will be that all emissions tests will conform to the specified reference methods.
Each of the reference methods include numerous QA/QC checks and data quality indicators (DQIs) that,
if met, ensure that the tests were properly performed. Section 6.0 summarizes these checks. Although
explicit DQOs are not specified for these emission parameters, the analysis described in Section 3.1.1 for
determination of statistical significance in changes in BSFC will also be used to evaluate if changes in
emissions are significant.
3.1.1 Determination of Statistical Significance
The mean BSFC delta cannot be deemed statistically significant if it is equal to or smaller than the 95
percent confidence interval. The confidence interval (e) is a function of the sample standard deviation (sn_
i) and the number of test runs conducted during the test campaign. The coefficient of variation (COV), or
the sample standard deviation normalized against the sample mean (for each test condition), combined
with the number of test runs will therefore serve as the DQI that links the width of the confidence interval
with the DQO. The mean delta for BSFC must be greater than e. If it is not, the 95 percent confidence
interval is wider than the change itself, and it cannot be deemed statistically significant.
Data collected during several similar ETV verifications show that, when the BSFC test methods are
properly applied, a COV of 0.7 percent is achievable for BSFC. The data evaluated to develop this COV
includes nine test series for similar diesel engine retrofit technology engine dynamometer tests. Each test
series consisted of three test runs (n=3). By conducting at least three baseline and modified fuel test runs
17
-------
and achieving the 0.7 percent COV, this verification will be able to demonstrate a statistically significant
BSFC delta of 3 percent or greater. If fuel consumption changes are statistically significant, the GHG
Center will calculate the difference's confidence interval. After the 3rd test run, and after each following
run (up to the 6th), analysts will calculate a test statistic, ttest, and compare it with the Student's T
distribution value with (ni + n2 - 2) degrees of freedom as follows:
Equation 2
Equation 3
Where:
Xi = mean fuel economy with baseline fuel
X2 = mean fuel economy with treated fuel
l^i - \\.2 = zero (H0 hypothesizes that there is no difference between the population means)
ni = number of repeated test runs with baseline fuel
n2 = number of repeated test runs with treated fuel
Si2 = sample standard deviation with baseline fuel, squared
s22 = sample standard deviation with treated fuel, squared
sp2 = pooled standard deviation, squared
Selected T-distribution values at a 95-percent confidence coefficient (t0 025, DF) appear in the following
table.
Table 3-1. Selected T-distribution Values
ni
3
4
5
6
n2
3
4
5
6
Degrees of
Freedom,
DF (n!+n2-
2)
4
6
8
10
to.025, DF
2.776
2.447
2.306
2.228
If ttest > to 025.DF, then it is concluded that the data shows a statistically significant difference between the
baseline and treated fuel BSFC. Otherwise, it will be concluded that a significant BSFC difference did
not occur. If significant, the difference and its confidence interval will be reported.
18
-------
Use of equations 2 and 3 requires the assumption that the baseline and treated fuel test run results have
similar variance. The ratio of the sample variances (sample standard deviation squared) between the two
fuel test series is a measure of this similarity. Analysts will calculate an Ftest statistic according to
Equation 4 and compare the results to the values in Table 3-1 to determine the degree of similarity
between the sample variances.
Where:
„
test ~
Equation 4
n2 _
= F-test statistic
= larger of the sample standard deviations, squared
= smaller of the sample standard deviations, squared
Table 3-2 presents selected FQ
acceptable uncertainty (a; 0.05).
distribution values for the expected number of test runs and the
Table 3-2. Selected F0 05 Distribution Values
s min number of
runs
3
4
5
6
s2max number
of runs
Degrees of
Freedom
2
3
4
5
3
2
19.00
9.55
6.94
5.79
4
3
19.16
9.28
6.59
5.41
5
4
19.25
9.12
6.39
5.19
6
5
19.30
9.01
6.26
5.05
If the F-test statistic is less than the corresponding value in Table 3-2, then analysts will conclude that the
sample variances are substantially the same and the statistical significance evaluation and confidence
interval calculations are valid approaches. If a statistically significant difference in BSFC is observed, the
95-percent confidence interval will be calculated. The half width (e) of the 95 percent confidence interval
is:
Equation 5
Reported results for improvement in BSFC will include the 95 percent confidence interval, if the results
are statistically significant.
19
-------
3.1.2 Baseline Emissions and BSFC Normalization Procedure
The CleanBoost fuel treatment is generally regarded as an immediate effect fuel modification and this
verification was designed to evaluate the immediate effect only. Although baseline engine performance
drift is not likely during this verification, baseline testing with reference fuel will be repeated after the
conclusion of testing with the CleanBoost treated fuel to confirm this (as specified in Section 2.2.1). A
statistical analysis of the two baseline test series will be conducted using the procedures detailed in
Appendix B. The GVP and Appendix B contain procedures for baseline normalization should a
statistically significant change in baseline BSFC occur. If the results from the two sets of baseline tests
are not statistically different, then the pooled variance for all of the baseline runs will be used to evaluate
changes to BSFC as outlined above.
20
-------
4.0
SAMPLING AND ANALYTICAL PROCEDURES
4.1 EXHAUST GAS SAMPLING SYSTEM
The exhaust gas constant volume sampling measurement system conforms to 40 CFR 86.1310, 89.308,
and 89.309. The system that will be used at SwRI is depicted in Figure 4-1 below.
Optional for
Paniculate
Background Reading
Zero Air
HC Span Gas
Read Background Bag
Dillution Tunnel
Heated Probe
Paniculate Probe
Mixing Orifice
LEGEND
Flow Control Valve
Selection Valve
Paniculate Filler
Pump
Flowmeter
Pressure Gauge
Recorder
Temperature Sensor
Vehicle Exhaust Inlet
Primary Filter (Phase 1 and 3)
Back-up Filter (Phase 1 and 3)
Note: Three filter holders
(one for each phase)
are also acceptable.
Heated Sample Line
Heat Exchanger
I R I
Integrator
^L
rTo Formaldehyde
Sample Collection
\4-4--f 4.4-
Supply Air
Primary Filter (Phase 2)
Sack-up Fifter(Phase 2)
.To Pump Rotometer
and Gas Meter
as Diagramed
Immediately
Below
Discharge
To Methanol Sample
Collection
Manometer
Revolulion Counter
PickUp
Manometer
Discharge
Figure 4-1. SwRI Gaseous and Particulate Emissions Sampling System (PDP-CVS)
Table 4-1 lists the major equipment to be used during the test campaign, expected values, and instrument
spans. Typical manufacturers and model numbers are listed for reference only and may vary by test cell.
21
-------
Table 4-1. Exhaust Gas Measurement System Specifications
Parameter or
Subsystem
Dynamometer speed
Dynamometer load
CVS pressure
CVS temperature
CVS volumetric
flow rate
CO
C02
CH4
NOX
THC
PM
PM2.5
Fe
Expected
Operating Range
0-2100RPM
0 - 368 hp,
0 - 1350 Ib.ft
950 - 1050 millibar
Otol91°C
2000 ft3 / min
(nominal)
0 - 300 ppmv
0 - 10000 ppmv
0-10 ppmv
0-100 ppm
0-300 ppmv
0-100 ppmv
0-5mg
0-5mg
0-5mg
Manufacturer,
Model / Operating
Principle
Varies with test cell
Varies with test cell
SwRI-built constant
volume sampler
HoribaOPE-135/
NDIR
HoribaOPE-135/
NDIR
GC/FID
Rosemount 955 /
Chemiluminescence
Rosemount 402 /
HFID
Gravimetric
Gravimetric
ICP spectroscopy
Span
Varies with test cell
up to 6000 RPM
Varies; up to 600 hp,
2600 Ib.ft
0 - 1500 millibar
0 - 200 °C
1800-2200 ft3 /min;
Varies with test cell
0 - 1000 ppmv
0 - 10000 ppmv
10 ppmv
100 ppmv
0 - 300 ppmv
0 - 100 ppmv
0 - 100 mg
0 - 100 mg
0 - 100 mg
Measurement
Frequency
10 Hz (10/s)
10 Hz (10/s)
10 Hz (10/s)
1 analysis per bag, 2
bags (1 dilute
exhaust, 1 ambient
air) per each cold-
start. Similar set of
2 bags for each hot-
start
10 Hz (10/s) (Note:
online gas analysis
through sampling
probe)
1 per each cold- and
hot-start
1 per each cold- and
hot-start
1 per each cold- and
hot-start
4.2 FILTER WEIGHING
Particulate filters are stored, conditioned, and weighed in a dedicated facility which conforms to 40 CFR
86.1312. The chamber in which the particulate filters are conditioned and weighed conforms to 40 CFR
86.112 without deviation.
4.3 GASEOUS ANALYZERS
Gaseous analyzers conform to §86.309, §86.1311, and §89, Subpart D, App B, Figure 1. Their operation
is specified in SwRI SOP# 07-009, which conforms to required elements B4 (Analytical Methods), B5
(Quality Control), and B6 (Instrument/Equipment Testing, Inspection, and Maintenance) of EPA QA/R-5.
4.4 PM2.5 AND FE DETERMINATIONS
PM2.5 and Fe measurements will be conducted as secondary verification parameters. These parameters
are not included in the FTP or GVP. A MOUDI Model 110 cascade impactor will be used to determine
emissions of PM2.5. Engine exhaust gases are sampled isokinetically and collected particulate matter is
22
-------
separated into equivalent aerodynamic diameter cut sizes of greater than 10, 5.6, 2.5, 1.8, 1.0, 0.56, 0.32,
0.18, 0.10, and 0.06 micrometers ((im). The mass of particulate matter collected on each stage is
determined gravimetrically using the same procedures as the FTP PM determination. PM2.5 emissions
are reported as the total mass collected on the stages up to 2.5 (im.
Particulate phase Fe emissions will be determined using the PM catches for each test conducted. At the
conclusion of the PM gravimetric analyses, the filters will be digested in solutions of nitric / perchloric
acid and aqua regia. The resulting solution is analyzed for Fe content using ion chromatography/mass
spectroscopy (IC/MS) procedures. Analytical instrumentation will be standardized using NIST traceable
standard reference materials. A blank sample is run to verify zero, and a independent check standard is
run to verify calibration. SwRI internal control limits are 90 -110 percent recovery on the check standard.
5.0 SAMPLE HANDLING AND CUSTODY
Only particulate matter (PM) filter measurements and bag samples involve manual handling, since
gaseous emission measurements are made and recorded by the computer-controlled data system
associated with the continuous sampling system.
The PM filters are prepared and processed according to SwRI SOP# 07-020 which specifies a method of
conditioning and weighing filters used to collect particulate samples during exhaust emission testing. This
SwRI SOP conforms to required element B3 (Sample Handling and Custody) of EPA QA/R-5.
Samples are handled according to SwRI SOP 07-023. This SOP conforms to required element B3
(Sample Handling and Custody) of EPA QA/R-5.
6.0 DATA QUALITY INDICATOR GOALS AND QA/QC CHECKS
Test measurements that contribute to a verification parameter's determination have specific data quality
indicator goals (DQIGs) that, if met, imply achievement of the parameter's DQOs. For this test,
completion of the QA/QC checks and achievement of the DQI goals ensures that the specified test
methods have been completed in accordance with the TQAP and CFRtest method requirements. Based
on historical data, when testing is properly completed, the specified DQOs should be achievable.
Tables 6-1 through 6-5 list the individual analyzer and system DQIGs in terms of accuracy. A variety of
calibrations, QA/QC checks, and other procedures ensure the achievement of each DQIG. The table
summarizes those QA/QC checks for each of the major test systems.
23
-------
Table 6-1. CVS System Data Quality Indicators and QA/QC Checks
Parameter
Pressure
Temperature
Volumetric
flow rate
Data Quality Indicators Goals
Accuracy
+ 2.0% of
reading
+ 2.0% of
reading
+ 0.5% of
reading
How Verified
Calibration of
sensors with
NIST-traceable
standard
Calibration of
sensors with
NIST-traceable
standard
CVS and propane
critical orifice
calibration
Frequency
At initial
installation or
after major
repairs
At initial
installation or
after major
repairs
At initial
installation or
after major
repairs
QA/QC Checks
Description
Inspect
calibration
certificates
Inspect
calibration
certificates
Inspect
calibration data
Propane
composition
verification via
analysis with
FID
Propane
injection check
Sample bag leak
check
Flow rate
verification
Dilution air
temperature
Frequency
Prior to test
Prior to test
Prior to test
Prior to
placing new
propane tank
in service
Weekly
Before each
test run
Before each
test run
During each
test run
Allowable Result
Current calibration
meeting DQI goal
Current calibration
meeting DQI goal
Current calibration
meeting DQI goal
< 0.35 % difference
from previously
used and verified
tank
Difference between
injected and
recovered propane <
+ 2.0%
Maintain 10" Hg for
10 seconds
< + 5 cfm of
nominal test point
Between 20 and 30
°C
24
-------
Table 6-2. Instrumental Analyzers Data Quality Indicators and QA/QC Checks
Parameter
CO
CO2
NOX
THC
CO2 only
NOX only
Data Quality Indicators Goals
Accuracy
+ 1.0%FS
or + 2.0%
for each
calibration
gas
How Verified
11-point
calibration
(including zero)
with gas divider;
protocol
calibration gases
Frequency
Monthly
QA/QC Checks
Description
Review and
verify analyzer
calibration
Gas divider
linearity
verification
Calibration gas
certification or
naming
Zero gas
verification
Analyzer zero
and span
Analyzer drift
Wet C02
interference
check
CO2 Quench
Check
Converter
Efficiency Check
Frequency
Once during
test and upon
completion of
new calibration
monthly
Prior to service
Prior to service
Before and
after each test
run
For each bag
analysis
Monthly
Annually
Monthly
Allowable Result
Current calibration
meeting DQI goal
All points within + 2.0%
of linear fit; FS within +
0.5 % of known value
Average concentration of
three readings must be
within + 1 % for
calibration gas and NIST-
traceable reference
material
THC < 1 ppmv
CO < 1 ppmv
CO2 < 400 ppmv
NOX<0.1 ppmv
O2 between 18 and 21 %
All values within + 2.0 %
of point of +1.0% of FS;
zero point within + 0.2 %
ofFS
Post-test zero or span drift
shall not exceed +2.0%
FS
CO (0 to 300 ppmv)
interference < 3 ppmv;
CO (> 300 ppmv)
interference < 1 % FS
NOX quench < 3.0 %
Converter Efficiency
>90%
25
-------
Table 6-3. PM, PM2.5, and Fe Analysis Data Quality Indicators and QA/QC Checks
Data Quality Indicators Goals
Parmater
PMand
PM2.5
Fe
Accuracy
+ 1.0ug
±10 %
reading
How Verified
NIST-traceable
scale calibration,
weighing room
controls, filter
weight control
NIST-traceable
instrument
calibration
Frequency
Daily
Daily
QA/QC Checks
Description
NIST-traceable
calibration weight cross-
check
Weight room
temperature
Weight room relative
humidity
Reference filter weight
change
Analysis of blank and
check standards
Frequency
Daily
Daily
Daily
Daily
Every 10
analyses
Allowable Result
Weight change <10
ug
Between 19 and 25
°C
Between 35 and
53% RH
Weight change <20
ug
±10 percent of
reading
Table 6-4. Supplementary Instruments and Additional QA/QC Checks
Description
Test cell Wet/dry bulb thermometer
calibration
Test cell Barometer calibration
Test cell temperature
Test fuel analysis
Frequency
Monthly
Weekly
Each test run
Prior to testing
Allowable Result
Within + 1.0 °F NIST-traceable standard
Within + 0.1" Hg of NIST-traceable
standard
Between 68 and 86 °F
Conforms to 40 CFR §86.1313
specifications (See Appendix A-2)
Table 6-5. Dynamometer Data Quality Indicators and QA/QC Checks
Parameter
Speed
Load
(Torque
Sensor)
Data Quality Indicators Goals
Accuracy
+ 2.0%
+0.5%
How Verified
60-tooth wheel
combined with
frequency counter
NIST-traceable
weights and
torque arm
Frequency
At initial
installation or
after major
repairs
Weekly
QA/QC Checks
Description
Inspect
calibration
certificate
Inspect
calibration
certificate
Torque trace
acceptance test
Frequency
Prior to test
Prior to test
and after new
calibration
Each test run
Allowable Result
Current calibration
meeting DQI goal
Current calibration
meeting DQI goal
+ 2. 5 Ib.ft for values
< 550 Ib.ft,
+ 5.0 Ib.ft for values
< 1050 Ib.ft,
+ 10 Ib.ft for values
< 1550 Ib.ft
26
-------
7.0 INSTRUMENT CALIBRATION AND FREQUENCY
The calibration schedule for major instruments is included with other QC activities in Table 6-1 above. 40
CFR 86.1316-86.1326 completely specifies the methods, frequency, and requirements of these
calibrations. Specific instruments and the applicable SOPs for implementation are described below. The
general reference is SwRI QPP 05 - Measurement and Test Equipment. Records of all calibration
activities are retained at SwRI and will be inspected by the GHG Field Team Leader and/or QA Manager
to ensure the TQAP and CFR requirements are met.
27
-------
8.0 DATA ACQUISITION AND MANAGEMENT
This section describes the generation and processing of test data at SwRI and the flow and disposition of
these data from origin to the GHG Center for reporting and archiving. Data acquisition and data
management at SwRI are performed according to SwRI QPP 08 - Data Processing and Reduction, which
conforms to required element BIO (Data Management) of EPA QA/R-5. The SwRI project manager is
operationally responsible for all aspects of a test, and the SwRI QA Manager is operationally responsible
for all data quality aspects of a test.
SwRI will submit copies of initial raw and intermediate data at the end of each test sequence and at test
completion. These data include:
• documents describing the engine inspection and setup activities;
• tracking forms for daily test activities and QC check results;
• external documents such as test fuel lot analyses and NIST-traceable calibration gas certificates;
• test cell data system printouts showing run summary instrument results for test cell system (dyno,
CVS, direct and bag cart analysis instruments, etc.); and
• QC check summary printouts (zero, span drift, etc.).
SwRI will prepare and submit a letter report in printed and electronic format to the GHG Center after
completion of the field activities. The report will describe the test conditions, document all QA/QC
procedures, and summarize intermediate data. The SwRI QAO will also submit a QA report documenting
the internal data assessment activities of the test as described in Section 9.0.
The GHG Center Project Manager will incorporate the SwRI material into the final verification report and
statement and submit for review according to the GHG Center QMP and ETV Program guidance
documents. The GHG Center QA Manager will incorporate the SwRI QA material into the GHG Center's
internal assessment documentation for the test, along with assessment activities of the Center. These will
include the supplemental TSA, performance audit, and ADQ described in Section 14.
9.0 INTERNAL AND EXTERNAL AUDITS
Several assessments are specified for this verification in accordance with the GHG Center QMP and the
ETV Program QMP.
9.1 TECHNICAL SYSTEMS AUDIT
The GHG Center staff has previously conducted a quality and technical systems audits (TSA) of the SwRI
DEER on an earlier ETV test. That TSA addressed major test components including documentation and
adherence to standard procedures for testing, instrument calibration and QC checks, data processing,
audits, and reporting. It also included review of some of the documentation of elements of the
SwRI/DEER quality system. In view of the positive findings of that TSA and the similarity between the
previous verification and the upcoming test, a second TSA on this technology class is not proposed for
the upcoming test.
28
-------
A tracking checklist of calibrations and QC activities, adapted to the experimental details of this test, will
be used to verify that equipment, SOPs, and calibrations are as described in this TQAP. The field team
leader will complete the items on this checklist during his observation of the test and return the form to
the GHG Center QA manager as part of the QC documentation of the test. He will incorporate this
material into the ADQ described below.
9.2 PERFORMANCE EVALUATION AUDITS
The GHG Center specifies internal Performance Evaluation Audits (PEAs), as applicable, on critical
measurements of every verification test. The Center will use the SwRI quality infrastructure for an
internal PEA for this test. SwRI maintains a set of NIST-certified gas standard mixtures in the
concentration ranges applicable to these measurements. The monthly calibration procedure requires that
the DEER challenge the analytical instruments with these standards as a performance check independent
of the calibration gas standards. The GHG Center will use this internal check in lieu of a blind PEA. The
standard mixture challenge from that time will be used as a PEA if a monthly analyzer calibration under
SOP 6-012 has been performed within a week of testing on the test cell used for this study. A separate
challenge, according to the applicable portion of the SOP, will otherwise be conducted during the period
of the test.
9.3 AUDIT OF DATA QUALITY
The GHG Center QA Manager will oversee an audit of data quality (ADQ) of at least 10% of all of the
verification data in accordance with Table 9-1 of the ETV QMP. The ADQ will be conducted in
accordance with EPA's [Guidance on Technical Audits and Related Assessments for Environmental Data
Operations}. The ADQ will include (1) verification of input data and outputs reported by test cell
instrumentation, (2) checks of intermediate calculations, and (3) a review of study statistics. The ADQ
will also draw conclusions about the quality of the data from the project and their fitness for their
intended use. Effort on this audit will be assigned as follows. The SwRI QAO, in this case, will conduct
an internal ADQ of results generated by SwRI covering the areas described above and submit the audit
report to the GHG Center QA Manager. The GHG Center QA Manager will review and incorporate this
into an overall ADQ report, including documentation of subcontractor oversight and review of the final
processing and reporting of the results.
9.4 EXTERNAL ASSESSMENTS
SwRI and GHG Center staff will cooperate with any external assessments by EPA. EPA personnel may
conduct optional external assessments (TSA, PEA, or ADQ) during this or any subsequent test. The
external assessments will be conducted as described in EPA QA/G-7.
9.5 INTERNAL ASSESSMENTS
Internal assessment reports will be reviewed by the SwRI QAO and GHG Center QA Manager. The
written report of the ADQ will be reviewed by the GHG Center QA Manager and submitted as a separate
addendum to the verification report.
29
-------
10.0 CORRECTIVE ACTION
A corrective action must occur when the result of an audit or quality control measurement is shown to be
unsatisfactory as defined by the DQOs or by the measurement objectives for each task. The corrective
action process involves the GHG Center project and QA staff as well as subcontractor personnel. A
written corrective action report is required on major corrective actions that deviate from the TQAP.
Corrective action is performed at SwRI according to QPP 11 - Nonconformance and Corrective Action,
which conforms to required elements B5 (Quality Control) and Cl (Assessments and Response Actions)
of EPA QA/R-5. Situations requiring corrective action will be communicated to the GHG Center field
team leader who will, under direction of the GHG Center project manager, assess the incident and take
and document appropriate action on behalf of the center. The project manager is responsible for and is
authorized to halt work if it is determined that a serious problem exists.
11.0 DATA REDUCTION, REVIEW, VALIDATION, AND REPORTING
The field team leader's primary on-site function will be to monitor SwRI's activities. He will be able to
review, verify, and validate certain data (test cell file data, QA/QC check results) during testing. The
GHG Center project manager will incorporate the SwRI material into the final verification report and
statement and submit this information for review according to the GHG Center QMP and ETV program
guidance documents. The GHG Center QA Manager will incorporate the SwRI QA material into the
GHG Center's internal assessment documentation for the test along with assessment activities of the
Center. These will include the performance audit and ADQ described in Section 9.0.
12.0 REPORTING OF DATA QUALITY INDICATORS
The GHG Center staff will collect and tabulate the DQIG values specified in Table 6-1 as part of the data
processing steps described above. These will be reviewed both internally and by the GHG Center QA
Manager in the preparation of their verification report and assessment reports and to determine
achievement of the DQOs. These reports, as specified in the GHG Center QMP, are submitted to both the
EPA project officer and QA Manager.
13.0 DEVIATIONS FROM GVP
The technical aspects of this TQAP were constructed to be consistent with the technical requirements and
philosophy of the GVP. The only planned deviations from the GVP are the omission of the additional
GVP test runs at maximum power and torque. No other deviations from the GVP or this document are
anticipated. If any such deviations are identified in the course of implementing this test, SwRI staff will
consult with GHG Center staff as soon as possible to resolve the issues. Section 2.7 of EPA/QA R-5
states that the EPA will be notified of any significant deviations and the QAO will revise this document
and submit it to EPA for review and approval.
30
-------
14.0 REFERENCED QUALITY DOCUMENTS
14.1 EPA-ETV
EPA QA/R-5
EPA ETV QMP
EPA QA/G-5
EPA QA/G-7
GVP
EPA Requirements for Quality Assurance Project Plans, EPA QA/R-5, Office of
Environmental Information, U.S. Environmental Protection Agency, EPA/240/B-
01/003, March 2001.
Environmental Technology Verification Program Quality and Management Plan
for the Pilot Period (1995-2000), National Risk Management Research
Laboratory, National Exposure Research Laboratory, Office of Research and
Development, U.S. Environmental Protection Agency, EPA/600/R-98/064, May
1998 (or current version).
Guidance on Quality Assurance Project Plans, EPA QA/G-5, Office of
Environmental Information, U.S. Environmental Protection Agency, EPA/600/R-
98/018, February 1998.
Guidance on Technical Audits and Related Assessments, EPA QA/G-7, Office of
Environmental Information, U.S. Environmental Protection Agency, EPA/600/R-
99/080, January 2000.
Generic Verification Protocol for Diesel Exhaust Catalysts, Particulate Filters,
and Engine Modification Control Technologies for Highway and Nonroad Use
Diesel Engines (Draft), EPA Cooperative Agreement No. CR826152-01-3,
January 2002.
Environmental Technology Verification Protocol - Determination of Emissions
Reductions Obtained by Use of Alternative or Reformulated Liquid Fuels, Fuel
Additives, Fuel Emulsions, and Lubricants for Highway and Nonroad Use Diesel
Engines and Light Duty Gasoline Engines and Vehicles. EPA Cooperative
Agreement No. CR826152-01-3, September 2003.
14.2 GHGTC
GHGTC QMP
SRI/USEPA-GHG-
QAP-28
Greenhouse Gas Technology Center Quality Management Plan, Version 1.4,
March, 2003.
Test and Quality Assurance Plan—ConocoPhillips Fuel-Efficient
High-Performance SAE 75W90 Rear Axle Gear Lubricant, SRI/USEPA-GHG-
QAP-29, March 2003.
SRI/USEPA-GHG- Environmental Technology Verification Report—ConocoPhillips Fuel-Efficient
31
-------
VR-28 High-Performance SAE 75W90 Rear Axle Gear Lubricant, SRI/USEPA-GHG-
VR-28, August 2003.
SRI/USEPA-GHG- Test and Quality Assurance Plan—Universal Cams, LLC Dynamic Cam Diesel
QAP-31 Engine Retrofit System, SRI/USEPA-GHG-QAP-31, April 2004.
14.3 SOUTHWEST RESEARCH INSTITUTE
SwRI QAPP Test/QA Plan for the Verification Testing of Diesel Exhaust Catalysts,
Particulate Filters, and Engine Modification Control Technologies for Highway
and Nonroad Use Diesel Engines (Version 1.0 April 8, 2002).
Quality Policy and Procedures (QPPs)
QSM Quality System Manual - 2000, April 2001
QPP-03 Document Preparation and Control
QPP-05 Measurement and Test Equipment
QPP-07 Testing and Sample Analysis
QPP-07-003 Transient Test for Heavy-Duty Diesel Engines
QPP-08 Data Processing and Reduction
QPP-09 Analysis and Reporting
QPP-10 Training and Motivation
QPP-11 Nonconformance and Corrective Actions
QPP-12 Internal Audits
QPP-14 Quality Records
Standard Operating Procedures (SOPs)
SOP-06-003 Linearity Verification of Gas Dividers
SOP-06-002 NOX Converter Efficiency Determination
SOP-06-012 Monthly Calibration of Analyzers for Continuous Dilute Gaseous Exhaust
SOP-06-016 Wet CO2 Interference Check for CO Analyzers
SOP-06-021 FID Response for Methane
SOP-06-025 NOX Analyzer and System Response Checks
SOP-06-041 NOx Analyzer CO2 Quench Check
SOP-06-044 Hydrocarbon Analyzer Optimization
SOP-07-001 Power Validation for Heavy-Duty Diesel Engines
SOP-07-002 Power Mapping for Heavy-Duty Diesel Engines
SOP-07-009 Emissions Testing During Heavy-Duty Diesel Engine Transient Cycle
SOP-07-020 Particulate Filter Conditioning and Weighing
SOP-07-023 Operation of Bag Cart
SOP-12-001 Quality Audits
32
-------
Appendix A
Test Log Forms and Checklists
33
-------
Appendix A-l. Test Results Summary and DQO Checks
Complete after each hot start test run is complete.
After the third hot start test (and any additional tests), calculate the mean, sample standard deviation, and coefficient of
variation (COV) for each parameter. COV is the sample standard deviation divided by the mean, as a percentage.
Verify that the Data Quality Objectives (DQOs) are met for each parameter.
Signature:
Table A-la: Baseline Test Results & DQO Check
Reported Value,
g/Bhp-hr*
The value is the weighted value of the single cold start FTP test with the hot start FTP test for each run. See the TQAP for detailed calculations.
-------
Table A-lb: Candidate Test Results & DQO Check
-------
Appendix A-2. Test Fuel Verification
Obtain a copy of the test fuel lot analysis.
Review all analysis results and test method documentation.
Properties and test methods must conform to the specifications given in the
following table.
Audit Date:
Fuel Lot ID:
Signature:
Date Received:
Table A-2. Test Fuel Specifications
Description
Cetane Number
Cetane Index
Distillation Range:
IBP
10 % point
50 % point
90 % point
Endpoint
Sulfur
Viscosity
Flashpoint
Hydrocarbons:
Olefms
Aromatics
Specific Gravity
ASTM Test
Method No.
D613
D976
D86
D2622
D445
D93
D1319
D5186
D287
Specified
Value
40-50
40-50
340 - 400 °F
400 - 460 °F
470 - 540 °F
560 - 630 °F
610- 690 °F
0.03 - 0.05 %
2.0-3.2
130 °F min.
Balance
27%
32-37 °API
Analysis
Value
Mfg. Certified
Value
Meets
Spec.?
Notes:
-------
Appendix A-3
QA/QC Checks
Signature:
Table A3-1: QA/QC Checks
QA/QC
Check
Description
Frequency
Allowable Result
Date Check
Completed
(SwRI)
Date Audit
Completed
(GHG
Center)
OK?
Audit Data
Source
Dynamometer
Dynamometer
Calibration
Certificates
Review
Torque trace
acceptance
test
Prior to
test
Each test
run
Sensor accuracies (speed and load)
meet Table 6-1 specifications
+ 2.5 Ib.ft for values < 550 Ib.ft,
+ 5.0 Ib.ft for values < 1050 Ib.ft,
+ 10 Ib.ft for values < 1550 Ib.ft
CVS System
CVS System
Calibration
Certificates
Review
Propane tank
composition
verification
Propane
injection
check
Sample bag
leak check
Flow rate
verification
Dilution air
temperature
verification
Prior to
test
Prior to
placing
new
propane
tank in
service
Weekly
Before
each test
run
Before
each test
run
During
each test
run
Sensor accuracies (P, T, Q) meet
Table 6-1 specifications
< 0.35 % difference from
previously used and verified tank
Difference between injected and
recovered propane < + 2.0 %
Maintain 10" Hg for 10 seconds
< + 5 cfm of nominal test point
Between 20 and 30 °C
Emission Analyzers
Analyzer
calibrations
review
Once
during test
and upon
completion
of new
calibration
All values within + 2.0 % of point
of +1.0% of FS;
-------
Table A3-1: QA/QC Checks
QA/QC
Check
Description
Gas divider
linearity
verification
Calibration
gas
certification or
naming
Zero gas
verification
Analyzer zero
and span
Analyzer drift
Wet CO2
interference
check
CO2 Quench
Check
Converter
Efficiency
Check
Frequency
monthly
Prior to
service
Prior to
service
Before and
after each
test run
For each
bag
analysis
Monthly
Annually
Monthly
Allowable Result
All points within + 2.0 % of linear
fit; FS within + 0.5 % of known
value
Average concentration of three
readings must be within + 1 % for
calibration gas and NIST-traceable
reference material
THC < 1 ppmv
CO < 1 ppmv
CO2 < 400 ppmv
NOX < 0. 1 ppmv
O2 between 18 and 21%
All values within + 2.0 % of point
of + 1.0 % of FS; zero point
within + 0.2% of FS
Post-test zero or span drift shall
not exceed + 2.0 % FS
CO (0 to 300 ppmv) interference <
3 ppmv;
CO (> 300 ppmv) interference < 1
%FS
NOX quench < 3.0%
Converter Efficiency >90 %
Date Check
Completed
(SwRI)
Date Audit
Completed
(GHG
Center)
OK?
Audit Data
Source
Particulate Measurement
NIST-
traceable
calibration
weight cross-
check
Weight room
temperature
Weight room
relative
humidity
Reference
filter weight
change
Daily
Daily
Daily
Daily
Weight change < 10 ug
Between 19 and 25 °C
Between 35 and 53 %RH
Weight change < 20 ug
-------
Table A3-1: QA/QC Checks
QA/QC
Check
Description
Frequency
Allowable Result
Date Check
Completed
(SwRI)
Date Audit
Completed
(GHG
Center)
OK?
Audit Data
Source
Ambient Monitoring
Test cell
Wet/dry bulb
thermometer
calibration
Test cell
Barometer
calibration
Test cell
temperature
Monthly
Weekly
Each test
run
+ 1.0 °F NIST-traceable standard
Within + 0.1" Hg of NIST-
traceable standard
Between 68 and 86 °F
-------
Appendix A-4. Corrective Action Report
Verification Title:
Verification Description:
Description of Problem:
Originator:
Investigation and Results:
Date:
Investigator:
Corrective Action Taken:
Date:
Originator:
Approver:
Carbon copy: GHG Center Project Manager, GHG Center Director,
Date:
Date:
SRI QA Manager, APPCD Project Officer
-------
Appendix B
Baseline Emissions and Fuel Economy Normalization Procedure
Changes to engine emissions or fuel economy resulting from the use of CleanBoost will be calculated by
comparing the test results while using the CleanBoost treated fuel with test results while using the
baseline reference fuel. Oftentimes, there can be an observed change in engine performance, both in
emissions and in fuel economy over time. This is referred to as baseline performance drift. Therefore,
analysts must evaluate engine performance with the baseline fuel before and after testing with the treated
fuel to determine the overall reference fuel mean engine performance for comparison to the treated fuel
engine performance.
There are three ways that the emissions or fuel economy changes caused by the treated fuel (and not the
baseline drift, if any) can be analyzed:
(1) Determine that there is no statistical difference in engine performance with reference fuel from
the initial to final data sets. In this case, all baseline data collected with reference fuel is pooled
together and compared to the treated fuel data;
(2) Compare each individual set of reference fuel data to the treated fuel data to obtain a range of fuel
economy changes based on the two data sets;
(3) Determine that the two reference fuel data sets are statistically different and cannot be directly
pooled. Assume that the change in reference fuel performance from the initial and final baseline
tests is the result of a systematic drift in vehicle performance. In this case, all data can be
normalized to account for such systematic changes. The normalized reference fuel data is then
pooled and compared to the normalized treated fuel data.
The following discussion is an excerpt from a similar verification previously conducted by the GHG
Center. It is presented here to provide a detailed example of this analysis. This test evaluated changes in
vehicle fuel economy as a result of using a candidate axle lubricant (indicated as FEHP). The statistical
analysis and procedural approach shown here will be used on the current verification to evaluate changes
in engine emissions and fuel economy that are a direct result of use of the CleanBoost technology. The
data presented here are used as an example only and are not intended to represent anticipated changes in
fuel economy as a result of CleanBoost.
Reference Lubricant Fuel Economy
Analysts evaluated the two sets of reference lubricant fuel economy data to determine the statistical
significance of the difference in mean fuel economy between the data sets. An F-test was completed on
the two reference lubricant data sets to compare the data variance of the two groups. Table B-l presents
the results of the F-test.
-------
Table B-l: F-test Evaluation of Reference Lubricant
Fuel Economy Data Set Variances
Parameter
Standard Deviation, initial reference lubricant tests (mpg)
Standard Deviation, final reference lubricant tests (mpg)
-T test
FO.OS
F test < F0 05 (variances statistically equivalent)?
Value
0.0408
0.0448
1.207
5.192
Yes
Results of the F-test indicate that the two sets of reference lubricant data have equivalent variances at a 95
percent confidence level. Therefore, analysts applied the t-test to evaluate the statistical significance of
the change in fuel economy between the two reference lubricant data sets. Table B-2 presents the results
of the t-test analysis for the two reference lubricant data sets.
The t-test results indicate that there is a statistically significant difference between the two reference
lubricant fuel economy data sets at a 95 percent confidence level. Based on this analysis and SwRI's
previous experience, it is likely that the change in fuel economy is the result of a systematic drift in
vehicle performance due to mileage effects or other phenomena. Therefore, analysts calculated the fuel
economy improvement using the method discussed in bullet item (3) above.
Table B-2: Statistical Analysis of Reference Lubricant Tests
Fuel Economy Difference
Parameter
Initial Ref. Lubricant Standard Deviation (mpg)
Final Ref. Lubricant Standard Deviation (mpg)
Mean Fuel Economy - Initial Reference Lubricant (mpg)
Mean Fuel Economy - Final Reference Lubricant(mpg)
Change in Fuel Economy (mpg)
Change in Fuel Economy (%)
COV-Initial Reference Lubricant (%)
COV-Final Reference Lubricant (%)
Initial Ref. Lubricant Test count
Final Ref. Lubricant Test count
Total count
Degrees of Freedom
(Pooled std dev) 2
(Pooled std dev)
Critical t distribution value (t 0.025. DF )
Calculated t-test value, ttest
W^ o.o25,DF (Is the change statistically significant)?
Value
0.0408
0.0448
18.021
18.139
0.118
0.655
0.226
0.247
5
6
11
9
0.0019
0.043
2.262
4.525
YES
Fuel Economy Change
The two reference lubricant data sets are statistically independent based on the statistical analysis of the
reference lubricant fuel economy data presented in Table B-2. Analysts must compare the complete
reference lubricant data set and FEHP lubricant test results to determine a representative fuel economy
-------
change resulting from the use of FEHP lubricant. No viable explanation for the shift in reference
lubricant fuel economy was determined after review of test and QA/QC data. SwRI concluded that there
was a "drift" in vehicle performance associated with the mileage accumulation on the test vehicle. The
GHG Center evaluated the test data by making the assumption that, during this test period, vehicle drift
occurred and that the drift follows a linear behavior with fuel economy improving with mileage
accumulation. The fuel economy data for all runs were therefore normalized to remove the effects of the
observed linear vehicle performance drift. Any fuel economy change calculated for the normalized data
set was then attributable solely to the FEHP lubricant and not mileage or other effects.
A linear regression was performed on the reference lubricant data (initial and final) to complete the
normalization This provides the linear drift relationship. Table B-3 presents the results of the linear
regression. Figure B-l presents the fuel economy results vs. vehicle mileage with the linear regression
results.
Table B-3: Reference Lubricant Data Regression Statistics
Parameter
Intercept
Slope
Standard error - intercept
Standard error - slope
R-Square
Regression sum of squares
Residual sum of squares
Observations
Value
17.397
3.86E-05
0.163
9.10E-06
0.6664
0.0364
0.0182
11
10
-------
18.5
18.3
,§ 18.1
I
o
o
u
H! 17.9
0)
3
17.7
17.5
y=0.0000386x+ 17.397
R2 = 0.6664
16000
17000 18000
Odometer (miles)
19000
Figure B-l: Reference Lubricant Fuel Economy Results vs. Mileage
With Drift Regression Line
All test data (reference lubricant and FEHP) was normalized to a common point for comparison based on
the reference lubricant regression. Therefore, the GHG Center normalized the test data to the y-intercept.
Data was normalized using the following equation:
i 7
mx +b
where:
FEN>i
FE;
m
b
Xj
normalized fuel economy for test run i
fuel economy for test run i
slope of "drift" line
intercept of "drift" line
vehicle odometer reading at beginning of test run i
11
-------
Table B-4 presents the results of the normalization procedure.
results as a function of mileage.
Figure B-2 presents the normalized test
Table B-4: Normalized Fuel Economy Test Results
Test Run
ID
Reference Lubricant
Base-1
Base-2
Base-4
Base-6
Base-7
Mean
Standard Deviation
FEEP Lubricant
FEHP-1
FEHP-2-R2
FEHP-3
FEHP-4
FEHP-5
FEHP-6
Mean
Standard Deviation
Reference Lubricant
Post Base- 1R2
Post Base-2
Post Base-3
Post Base-4
Post Base-5
Post Base-6
Mean
Standard Deviation
Composite Fuel Economy
(mpg)
18.070
18.013
17.994
18.055
17.973
18.021
0.0408
18.272
18.272
18.284
18.233
18.263
18.206
18.255
0.0296
18.208
18.111
18.143
18.169
18.121
18.082
18.139
0.0448
Normalized
Fuel Economy (mpg)
17.448
17.392
17.370
17.425
17.345
17.396
0.0414
17.588
17.584
17.594
17.543
17.571
17.515
17.566
0.0307
17.468
17.374
17.402
17.426
17.379
17.340
17.398
0.0447
12
-------
18.0
Q.
>
O
o
u
LU
"3
LL.
1
(0
17.6
17.4
17.2
17.0
• Reference Lubricant Fuel Economy
• FEHP Fuel Economy
y = 0.00x+ 17.397
16000
17000 18000
Odometer Reading (miles)
19000
Figure B-2: Normalized Reference Lubricant and FEHP Fuel Economy Results vs. Mileage
Analysts evaluated the normalized reference lubricant data to determine if the two data sets are from the
same population and can, therefore, be pooled to determine a mean reference fuel economy for
comparison to the normalized FEHP fuel economy. An F-test was initially completed on the two
normalized reference lubricant data sets to compare the data variance of the two groups. Table B-5
presents the results of the F-test.
Table B-5: F-test Evaluation of Reference Lubricant
Fuel Economy Data Set Variances
Parameter
Standard Deviation, initial reference lubricant tests (mpg)
Standard Deviation, final reference lubricant tests (mpg)
r test
FO.OS
F test < F0 .05 (variances equal)?
Value
0.0414
0.0447
1.166
5.192
Yes
Results of the F-test indicate that the two sets of normalized reference lubricant data have equivalent
variances at a 95 percent confidence level. Therefore, analysts applied the t-test to evaluate the statistical
significance of the change in fuel economy between the two normalized reference lubricant data sets.
Table B-6 presents the results of the t-test analysis for the two normalized reference lubricant data sets.
13
-------
Table B-6: Statistical Analysis of Normalized Reference
Lubricant Fuel Economy Difference
Parameter
Initial Ref. Lubricant Standard Deviation (mpg)
Final Reference Lubricant Standard Deviation (mpg)
Mean Fuel Economy - Initial Reference Lubricant (mpg)
Mean Fuel Economy - Final Reference Lubricant (mpg)
Change in Fuel Economy (mpg)
Change in Fuel Economy (%)
COV-Reference Lubricant (%)
COV-FEHP Lubricant (%)
Reference Lubricant Test count
FEHP Test count
Total count
Degrees of Freedom
(Pooled std dev) 2
(Pooled std dev)
Critical t distribution value (t 0.025, DF )
Calculated t-test value, ttest
ttest>t o.o25,DF (Is the change statistically significant)?
Value
0.0414
0.0447
17.396
17.398
0.002
0.011
0.238
0.257
5
6
11
9
0.0019
0.043
2.262
0.076
NO
The t-test results indicate that there is not a statistically significant difference between the two normalized
reference lubricant fuel economy data sets at a 95 percent confidence level. The two data sets have
statistically equivalent means and are from the same population. Therefore, the reference lubricant data
was pooled. Table B-7 presents the results of the pooled reference lubricant data analysis.
Table B-7: Summary of Pooled Normalized Reference Lubricant Data
Parameter
Ref. Lubricant Mean Normalized Fuel Economy (mpg)
Standard Deviation (mpg) - Pooled Normalize Reference Lubricant
COV-Pooled Normalized Reference Lubricant (%)
Value
17.397
0.0411
0.236
The mean pooled, normalized reference lubricant fuel economy is compared to the mean normalized
FEHP fuel economy to determine the change in fuel economy resulting from the use of the FEHP
lubricant. The calculated fuel economy improvement attributable to the use of the FEHP lubricant in the
test vehicle is
A = 17.566 mpg - 17.397 mpg = 0.169 mpg
This represents a 0.97 percent improvement in fuel economy when using the FEHP lubricant when
compared to the reference lubricant fuel economy.
14
------- |