Test Plan to Determine PEMS
Measurement Allowance for the PM
Emissions Regulated under the
Manufacturer-Run Heavy-Duty Diesel
Engine in-use Testing Program
&EPA
United States
Environmental Protection
Agency
-------
Test Plan to Determine PEMS
Measurement Allowance for the PM
Emissions Regulated under the
Manufacturer-Run Heavy-Duty Diesel
Engine in-use Testing Program
Assessment and Standards Division
Office of Transportation and Air Quality
U.S. Environmental Protection Agency
and
California Air Resources Board
and
Engine Manufactures Association
SEPA
United States
Environmental Protection
Agency
EPA-420-B-10-901
August 2010
-------
EXECUTIVE SUMMARY
This test plan sets forth the agreed upon processes and methodologies to be utilized to
develop additive, brake-specific, data-driven measurement allowance for PM emissions
measured by PEMS as required under the HDIUT regulatory program.
As detailed in this test plan, there is a clear consensus on what components of
measurement error are intended to be covered by the measurement allowance. Namely,
the allowance is to be calculated in a manner that subtracts lab error from PEMS error.
Specifically, utilizing Part 1065 compliant emissions measurement systems and
procedures for both the lab and PEMS, the lab error associated with measuring heavy-
duty engine emissions at stabilized steady-state test points within the NTE zone, will be
subtracted from the PEMS error associated with measuring heavy-duty engine emissions
utilizing PEMS over events under a broad range of environmental conditions. This
subtraction will yield "PEMS minus laboratory" measurement allowance. The
experimental methods and procedures specified in this test plan for determining,
modeling, and comparing each of the various components of measurement error are
designed to generate statistically robust data-driven measurement allowance for the PM
emissions.
Successful completion of this test plan is part of the resolution of a 2001 suit filed against
EPA by EMA and a number of individual engine manufacturers. The suit challenged,
among other things, certain supplemental emission requirements referred to as "not-to-
exceed" (NTE) standards. On June 3, 2003, the parties finalized a settlement of their
disputes pertaining to the NTE standards. The parties agreed upon a detailed outline for a
future regulation that would require a manufacturer-run heavy-duty in-use NTE testing
("HDIUT") program for diesel-fueled engines and vehicles. One section of the outline
stated:
"The NTE Threshold will be the NTE standard, including the margins built into the
existing regulations, plus additional margin to account for in-use measurement accuracy.
This additional margin shall be determined by the measurement processes and
methodologies to be developed and approved by EPA/CARB/EMA. This margin will be
structured to encourage instrument manufacturers to develop more and more accurate
instruments in the future."
Given the foregoing, the work to be completed under this test plan is a vital component to
the fulfillment of the settlement agreement, and it is vital to the successful
implementation of a fully-enforceable HDIUT program. Because of this significance, it
is critically important that the work detailed in this test plan be carried out in as thorough,
careful and timely a manner as possible.
-------
TABLE OF CONTENTS
Executive Summary 2
Table of Contents 3
List of Figures 5
List of Tables 5
1 Introduction 6
2 Monte Carlo Error Model and Measurement Allowance 8
2.1 Objective 8
2.2 Background 9
2.3 Methods and Materials 15
2.4 Simulation Procedure 16
2.4.1 Construction of the Error Surface 16
2.5 Model Considerations 24
2.5.1 Convergence 24
2.6 Simulation Output 24
2.6.1 Sensitivity Variation Effect 24
2.6.2 Sensitivity Bias Effect 25
3 Engine Dynamometer Laboratory Tests 26
3.1 Preliminary Audits 26
3.1.1 Objective 26
3.1.2 Background 26
3.1.3 On-site meeting to establish 1065 compliance requirements 26
3.1.4 Methods and Materials 26
3.1.5 Data Analysis 27
3.1.6 PEMS Manufacturer PM PEMS Commissioning 27
3.2 Bias and Precision Errors under steady state engine operation 27
3.2.1 Objective 27
3.2.2 Background 28
3.2.3 Methods and Materials 28
3.2.4 Data Analysis 30
3.3 Precision Errors under transient engine operation (dynamic response) 31
3.3.1 Objective 31
3.3.2 Background 31
3.3.3 Methods and Materials 31
3.3.4 Data Analysis 35
3.4 ECM Torque and BSFC 37
3.4.1 Objective 37
3.4.2 Data Analysis 37
4 Environmental Chamber 37
4.1 Data Analysis for Environmental Tests 38
4.2 PM Generator Commissioning 39
4.3 Baseline 39
4.3.1 Objective 39
4.3.2 Background 39
-------
4.3.3 Methods and Materials 40
4.3.4 Data Analysis 40
4.4 Electromagnetic Radiation 40
4.4.1 Objective 40
4.4.2 Methods and Materials 40
4.5 Atmospheric Pressure 41
4.5.1 Objective 41
4.5.2 Background 41
4.5.3 Methods and Materials 41
4.5.4 Data Analysis 43
4.6 Ambient Temperature and Humidity 44
4.6.1 Objective 44
4.6.2 Background 44
4.6.3 Methods and Materials 45
4.6.4 Data Analysis 46
4.7 Orientation and Vibration 46
4.7.1 Objective 46
5 SwRI CVS and CE-CERT Trailer Correlation 47
5.1 Method and Materials 47
6 Model Validation and Measurement Allowance Determination 48
6.1 Model validation 48
6.1.1 Objective 48
6.2 Measurement Allowance Determination 49
6.2.1 Objective 49
6.2.2 Background 49
6.2.3 Methods and Materials 49
6.2.4 Data Analysis 49
7 Time and Cost 50
7.1 Timeline 50
7.2 Cost 51
8 Abbreviations used in Brake Specific Equations 51
-------
LIST OF FIGURES
Figure 1. Probability Density Functions For Sampling Error Surfaces 9
Figure 2. Brake-Specific PM Emissions Calculation For Method 1 11
Figure 3. Brake-Specific PM Emissions Calculation For Method 2 12
Figure 4. Brake-Specific PM Emissions Calculation For Method 3 13
Figure 5. Error Surface: PEMS Vs. LAB 17
Figure 6. Error Surface: (PEMS-LAB) Vs. LAB 18
Figure 7. Error Surface: Final Version 19
Figure 8. Overview of Monte Carlo Simulation 22
Figure 9. Example of ANTE Cycle 35
Figure 10. Pressure Histogram 41
Figure 11. Pressure-Time Environmental Test Cycle 43
Figure 12. temperature histogram 44
Figure 13. Time Series Chart of Ambient Temperature Test 46
LIST OF TABLES
Table 1. Allowed Modifications 6
Table 2. Example Of Selection Of Measurement Allowance At 0.02 g/hp-hr NTE
Threshold 15
Table 3. Error Surfaces for the BSPM Simulation 23
Table 4. Engine, Exhaust Configuration, And Steady-State Modes 28
Table 5. Example Of SS Error Surface 30
Table 6. NTE Transient Cycle 32
Table 7. Dynamic response inter-NTE Events 34
Table 8. Concentration and Dilution Ratio Schedule with PM Generator 38
Table 9. Atmospheric Pressure Test Sequence 42
Table 10. Ambient Temperature test sequence 45
Table 11. Projected PM-PEMS Timeline 50
Table 12. projected cost estimate 51
-------
1 INTRODUCTION
This test plan will establish a PEMS measurement allowances for PM, as regulated by the
manufacturer-run on-highway heavy-duty diesel engine in-use test program. The
measurement allowance will be established using various laboratory facilities and PEMS.
The measurement allowance will be established in units of brake-specific emissions
(g/hp-hr), and it will be added to the final NTE PM standard, after all the other additive
and multiplicative allowances have been applied. This test plan will establish the PM
measurement allowance.
The PEMS used in this test plan must be standard in-production makes and models that
are for sale as commercially available PEMS. In addition, PEMS and any support
equipment must pass a "red-face" test with respect to being consistent with acceptable
practices for in-use testing. For example, the equipment must meet all safety and
transportation regulations for use on-board heavy-duty vehicles.
Even though the PEMS can not be "prototypes" nor their software "beta" versions, the
steering committee has already agreed that after delivery of PEMS to the contractor, there
may be a few circumstances in which PEMS modifications might be allowed, but these
modifications must meet certain deadlines, plus they are subject to approval by the
steering committee. Also, any implementation of such approved modifications will not
be allowed to delay the test plan, unless the steering committee specifically approves
such a delay. Table 1 summarizes these allowable modifications and their respective
deadlines:
TABLE 1. ALLOWED MODIFICATIONS
Allowed Modifications
Steering committee approved hardware and software modifications
that affect emissions results; including but not limited to fittings,
components, calibrations, compensation algorithms, sampling rates,
recording rates, etc.
Steering committee approved hardware modifications for DOT
approval or any other safety requirement approval
Delivery of any environmental / weather enclosure to contractor
Post-processing software to determine NTE results
DOT approval and documentation
Steering committee approved hardware or software that improves
the contractor's efficiency to conduct testing and data reduction
Before start of. . .
Steady- State
Testing
Environmental
Chamber Testing
Environmental
Chamber Testing
Model Validation
Model Validation
Always Allowed
The steering committee approved three different PEMS that includes the AVL Micro-
Soot Sensor (MSS), the Horiba Transient Particulate Matter (TRPM), and the Sensors
Proportional Particulate Matter Diluter (PPMD). However, because of the different
measurement technologies employed by each of these systems, the three different PEMS
hold slightly different status with respect to determining the PM measurement allowance.
-------
Because inertial microbalances are already approved for PEMS applications in 40 CFR
Part 1065, the Sensors PPMD will be one of the PEMS used to determine the
measurement allowance. And because EPA's PM standard is based upon a gravimetric
filter analysis, the Horiba TRPM will also be used to determine the measurement
allowance. The lowest measurement allowance value between the two will be selected as
the final measurement allowance for PM. If that value does not validate, then the lowest
validated value will be chosen. If the lowest validated value chosen is within 0.0075
g/hp-hr from the lowest non-validated value, then the lowest validated value will be the
measurement allowance. Otherwise, the MASC will spend up to a $100,000 to figure out
a resolution to the problem by generating more data or changing the way the validation
was performed. If that does not lead to a resolution, then Executive Management of EMA
and EPA will have to settle the issue.
Note that at the conclusion of successful testing of the Horiba system in this measurement
allowance program, EPA intends to approve the Horiba system as an alternative for use,
or EPA may elect to amend 40 CFR Parts 86 and/or 1065 to allow the use of the Horiba
TRPM or other PEMS that operate upon similar measurement principles. Because the
AVL system measures only the soot component of PM, the measurement allowance will
not be determined using the AVL results, unless both the Sensors and Horiba systems fail
to complete the measurement allowance program. Note that the steering committee may
determine at the conclusion of the program that the AVL MSS is a viable alternative for
demonstrating compliance. Under such a circumstance EPA may amend the Heavy-Duty
In-Use regulation to allow for its use.
This test plan describes a computer model, a series of experiments that are used to
calibrate the model, and another series of experiments that are used to validate the
calibrated model.
The test plan first describes the computer model. The computer model statistically
combines many sources of PEMS and lab error, which are nearly impossible to capture
simultaneously in a single test. The model will use statistics to apply the errors in a way
that simulates actual running of a PEMS in-use. The model will also consider only the
portion of error that is attributable to PEMS, and it will subtract the error that is already
tolerated in an emissions lab today. The model will also calculate and validate results
according to 40 CFR Part 1065.
The test plan then describes the series of experiments. These tests will characterize the
many sources of PEMS and lab error so that the specific nature of the errors can be
programmed into the computer model. The nature of the error has to do with the way
PEMS and the lab react to certain conditions. For example, under varying environmental
conditions such as temperature or vibration, a PEMS might exhibit signal drift, or it may
record noise that is not a part of the true emissions.
Next, the experimental results will be entered into the computer model, and the
measurement allowances are calculated by the model. The model uses a "reference"
PEMS data set, which will have many "reference NTE events." The model statistically
-------
applies all the errors to the reference data set, calculates results, and saves the results.
Then the model will be run with all errors set to zero to calculate the ideal results of the
reference data set. Each difference between a reference NTE event's result with errors
and its respective ideal result will be a brake-specific difference that is recorded for later
use. Then the process repeats using the same reference data set, to which new,
statistically selected errors are applied, and thus another unique set of differences is
calculated. As the model continues to iterate and generate more and more results,
patterns are expected to appear in the output data. These patterns should be the
distributions of differences, based upon the error that was statistically and repeatedly
applied to the reference data set. Many difference distributions will be determined: for
each reference NTE event, for each of the two brake-specific calculation methods (three
in case of the AVL system only), and for each PEMS. It has been agreed that the 95th
percentile values of these distributions will be taken as reasonable "worst case" results
for each reference NTE event. Details on how all these distributions will be reduced to
determine the PM measurement allowance is given in the "Error Model" section of this
test plan.
Because the calculation based on Method 2 and Method 3 require gas-based fuel flow
calculation based on the measurement of CC>2, CO, and NMHC, a decision was made to
use the gaseous PEMS data for this purpose, without the need to perform gaseous
measurement during the PM-PEMS program.
Finally, the test plan describes how the computer model will be validated against real-
world over-the-road in-use PEMS operation as well as additional lab testing. For the
over-the-road testing, PEMS emissions measurements will be conducted, while at the
same time a reference laboratory will be towed along to measure the same emissions. For
the lab testing, an attempt will be made to simulate real-world engine operation to
"replay" an over-the-road test in the lab. Data from these final experiments will be used
to validate the model, which must be done in order to gain sufficient confidence that the
model did not establish unreasonable measurement allowances.
The following sections of this test plan are written as instructions to the contractor or
contractors who will complete the test plan.
2 MONTE CARLO ERROR MODEL AND MEASUREMENT ALLOWANCE
2.1 Objective
Use Monte Carlo (e.g. random sampling) techniques in an error model to simulate the
combined effects of all the agreed-upon sources of PEMS error incremental to lab error.
Create error "surfaces" for the Monte Carlo simulation to sample, based upon results
from the experiments described in Sections 3 and 4. Exercise the model over a wide
range of NTE events, based on a single, reference data set of at least 150 but no more
than 200 unique NTE events. Determine the pollutant-specific brake-specific additive
measurement allowance for PM.
-------
2.2 Background
The error model uses Monte Carlo techniques to sample error values from "error
surfaces" that are generated from the results of each of the experiments described in
Section 3 on engine dynamometer laboratory tests and Section 4 on environmental
chamber tests. The lab test error surfaces cover the domain of error versus the magnitude
of the signal to which the error is to be applied (i.e. 1st to 99th percentile error vs.
concentration, flow, torque, etc.). This is illustrated later in this section. The
environmental test error surfaces for shock & vibration and electromagnetic & radio
frequency interference (EMI/RFI) cover the same domain as the lab tests. The
environmental test error surfaces for pressure and temperature are characteristically
different because they cover the domain of environmental test cycle time versus the
magnitude of the signal to which the error is to be applied (i.e. error at a selected time vs.
concentration). Details on how each surface is generated are given in each of the
respective sections. These surfaces will already be adjusted to represent PEMS error
incremental to lab error; therefore, these surfaces are sampled directly by the model.
The error model will use two different probability density functions (PDFs) as shown in
Figure 1 to sample the error surfaces, depending upon which experiment the surface
represents. To sample error surfaces that are generated from all the laboratory test results
(Section 3), and the environmental test results for shock & vibration (Section 4), the
model will use a truncated normal PDF because these tests are designed to evenly cover
the full, but finite, range of engine operation and ambient conditions. To sample error
surfaces that are generated from the pressure and temperature environmental test results
(Section 4), the model will use a uniform PDF because these tests are already designed to
cover the typical range and frequency of the respective conditions.
1
Probability Density Functions for Sampling Error Surfaces Once Per NTE Event
Lab Tests, Normal, SD=0.60795, truncate @ -1 & 1
— Environmental Tests, Uniform
Note: A non-truncated normal
distribution with SD=0. 60795 has P
values of 0.01 an
ic=+1, respective
d 0.99 at ic=-1 anc
y.
^
z
V
^^
^^
^^
^>^_
^^
^^
f^*^
^^^
^^^^
432
Relative Probability
- -0.25
- -0.50
- -0.75
0
FIGURE 1. PROBABILITY DENSITY FUNCTIONS FOR SAMPLING ERROR
SURFACES
-------
The random values that are obtained from both distributions are labeled ic in Figure 1 and
range from -1 to 1. Note that for the pressure and temperature environmental tests, a
uniform PDF will be used to sample test time, from which the nearest (in time) calculated
errors are used. The errors from the other tests will be aligned with the truncated normal
PDF such that each of the 50th percentile values at each of the tested signal magnitudes is
centered at the median of the PDF (ic = 0), and the 1st and 99th percentile error values at
each of the tested signal magnitudes will be aligned with the extreme negative (ic = -1)
and positive (ic = +1) edges of the PDF, respectively.
Each error surface will be sampled along its ic axis (y-axis) once per reference NTE event
trial, and it will be sampled along its parameter value axis (x-axis, e.g., concentration
(only for AVL MSS), flow, torque, etc...) once per second, within a given reference NTE
event trial. An error will be determined for a given second and parameter along the error
axis (z-axis) at the intersection of an ic value and a parameter value.
To ensure that the magnitudes of the error surfaces are appropriate, each data point used
to generate the surfaces will be a mean or a weighted mean of 30 seconds of sampling.
Interpolation will be performed by first linearly interpolating error values at each tested
magnitude along the selected line perpendicular to the ic axis. Then from that line of
errors, individual error values will be linearly interpolated at each second-by-second
signal magnitude of the given NTE event in the reference data set.
The reference data set to which all errors will be applied will be a large data set of engine
operation over a wide range of NTE events. This reference data set will be initially
generated from collections of real-world PEMS data sets. The reference data set should
contain at least 150 but no more than 200 unique NTE events. Parameters in the
reference data set may be scaled in order to exercise the model through a more
appropriate range of parameters (i.e. concentrations, flows, ambient conditions, etc.). If
the parameters are scaled, care should be taken to maintain the dynamic characteristics of
the reference data set.
After the errors are applied, NTE brake-specific PM emissions results are calculated,
using each of the three agreed-upon NTE calculation methods. The three different brake-
specific emission calculation methods for PM referred to in this test plan are i) Torque-
Speed method, ii) BSFC method, and iii) ECM-Fuel Specific method, and these are
illustrated in Figure 2, Figure 3, and Figure 4, respectively.
10
-------
For all PM PEMS:
m PM is a flow weighted particulate matter exhaust concentration in g/mol
m
m
ePM(glkW-hr} = -
i=\
. (mol\
n
J V
I
2*3.14159* A/'
60*1000*3600
Where for AVL:
m p,, is computed numerically as follows,
m
'PM
mol
I
•
«..
I * A v
* A?
FIGURE 2. BRAKE-SPECIFIC PM EMISSIONS CALCULATION FOR
METHOD 1
11
-------
For all PM PEMS:
m PM is a flow weighted particulate matter exhaust concentration in g/mol
em(g/kW-hr)=-
4c *V
r~ 2.
Wfuel
vmo/j tr
•
n
'mo/
Where for AVL:
is computed numerically as follows,
PM
mol
rnol
S
•
«.
± A.
* A?
,
4
*icr2
* ,(*]
>6UJ
,^peeti( (?pm) *7^(Ar-m) *2*3. 14159
60*1000*3600
FIGURE 3. BRAKE-SPECIFIC PM EMISSIONS CALCULATION FOR
METHOD 2
12
-------
For AVL Only:
ePM(g/kW-hr) =
( a \ W
\mol) Mc '
!=1
Where:
1/1;
N
F
__i
*^f)
,77/C!(^)*10-+(^(%) + ^2
Speedf (rpm\*Ti (7V-#2)*2*3. 14159* A?
-
60*1000*3600
Wfuel * ^
f a- ^
m g -
mPM\ ,
\rnol )
n;
c
Z-.
¥
fuel *\^
C
(m,PM;(^/mo/))*ml
* A/
V|
s ^ / ^
= Af
JcTT/C, (/?p»i) * 1 0"6 + (jcCO;. (%) + xCO2i (%))*! O"2
• f^l
m/K
\^y
^
JcTT/C, (/?p»i) * 1 0~6 + (xCOt (%) + xCO2j (%))*! O"2
:A?
FIGURE 4. BRAKE-SPECIFIC PM EMISSIONS CALCULATION FOR
METHOD 3
Next, the NTE events are calculated by each of the three calculation methods, but with no
error sampled or applied to the reference data set. These results are considered the
"ideal" results of the reference NTE events. These ideal results are subtracted from each
respective NTE event result 'with errors', and the difference is recorded. Then a new set
of errors are sampled and applied to the reference NTE event, and the NTE results 'with
errors' are calculated again. The ideal results are again subtracted, and the difference is
recorded. This is repeated thousands of times so that the model converges upon
distributions of brake-specific differences for each of the original NTE events in the
reference data set.
Then the 95th percentile difference value is determined for each NTE event distribution of
brake-specific differences for PM for each calculation method. At this point there is one
distribution of 95th percentile differences for PM, where all the NTE events are pooled by
the PM emissions for each of the three different calculation methods. Each of the 95th
percentile distributions represents a range of possible measurement allowance values.
13
-------
From each of these three distributions of possible measurement allowance values, one
measurement allowance per distribution must be determined. First the correlation
between 95th percentile differences versus the ideal PM emission is tested. For each
calculation method, if a least squares linear regression of 95th percentile differences
versus ideal PM emissions has an r2 (squared correlation coefficient) > 0.85 and an SEE
(standard error of the estimate or root-mean-squared-error) < 5 % of the median ideal PM
emission, then that linear regression equation will be used to determine the measurement
allowance for that calculation method at the following NTE threshold:
PM = 0.02 g/hp-hr and 0.03 g/hp-hr
In cases where extrapolation is required to determine the measurement allowance at the
NTE threshold, the measurement allowance will be determined using the linear
regression, but evaluated at the ideal PM emission that is closest to the NTE threshold,
not extrapolated to the NTE threshold itself. If the linear regression does not pass the
aforementioned r2 and SEE criteria, then the median value of the 95th percentile
differences is used as the single measurement allowance for that calculation method.
Next, the calculation method is selected. The above procedure will provide three
measurement allowances, where applicable, one for each of the three different calculation
methods. To make them comparable, the three measurement allowance values will be
normalized by the PM threshold and expressed as a percent. Also, if any measurement
allowance is determined to have a value less than zero, then that measurement allowance
will be set equal to zero. The calculation method with the minimum normalized PM
value will be chosen and the corresponding normalized PM value will be selected as the
best measurement allowance for PM, assuming it validates. If it does not validate, then
the minimum value that validates will be chosen as long as it is within 0.0075 g/hp-hr
from the minimum value that did not validate. If the difference between the minimum
value that validates and the minimum value that did not validate is greater than 0.0075
g/hp-hr, additional investigation with up to a $100,000 will be spent in order to
understand why the minimum value chosen did not validate. If the problem is not
resolved after spending the $100,000, then the matter will be referred to executive
management of EPA and EMA to decide on the PM measurement allowance.
Error! Reference source not found, below illustrates the selection of the calculation
method. The example is based on a hypothetical set of normalized PM measurements for
the three calculation methods. The minimum of these normalized allowances is used to
select the best method (highlighted in blue). In this hypothetical case, the BSFC method
would be selected.
14
-------
TABLE 2. EXAMPLE OF SELECTION OF MEASUREMENT ALLOWANCE AT
0.02 G/HP-HR NTE THRESHOLD
Calc. Method ==>
BSPM
Selected Method==>
Allowance at Respective NTE Threshold (%)
Torque-Speed
38%
BSFC
18%
ECM fuel specific
N/A
BSFC Method
Therefore, 18% would be selected as the best measurement allowance for PM, assuming
it validates. Otherwise, the 38 % will be chosen if it validates. Thus, the additive brake-
specific measurement allowance would be:
PM = 18 % * 0.02 g/hp-hr = 0.0036 g/hp-hr, if it validates, and if not, then:
PM = 38 % *0.02 g/hp-hr = 0.0076 g/hp-hr, if it validates, and if not, then:
spend up to a $100,000 to figure out why it did not validate in the first place, and then
apply the above strategy again, assuming the value now validates. If not, then EPA and
EMA executive management will decide on the PM measurement allowance value.
This PM value would be the value added to the actual brake-specific NTE threshold for a
given engine, based on actual family emissions limit, mileage, model year, etc.
2.3 Methods and Materials
Exercise the model using three different calculation methods: a) Torque-Speed method,
b) BSFC method, and c) ECM-Fuel Specific method (only for AVL MSS). Determine
which calculation method is the most accurate, and use it to estimate the measurement
allowance. Each calculation method is described in Figure 2, 3, and 4.
Prepare an Excel spreadsheet model for use with the Crystal Ball Monte Carlo software
for error analysis of brake specific emissions, BSE, as outlined in section 2.4. Changes to
the model specifications may be requested as agreed upon by the Steering Committee.
Prepare the spreadsheet in a modular structure following the specified model outline, and
make provisions for the identified calculation modules. Additionally, clearly identify and
easily locate input cells to the model to facilitate any revisions that may become
necessary for users who want to exercise the model with other Monte Carlo add-ins such
as @Risk or the newest versions of Crystal Ball. Test the spreadsheet with controlled test
cases of simplified input distributions with the Crystal Ball add-in to confirm correct
model implementation in accordance with this test plan. Run at least one typical analysis
as an additional confirmation.
Deliver the electronic spreadsheet and a brief report describing the model, presenting the
test cases, and describing pertinent information including the Crystal Ball version number,
the Excel version number, the operating system and the computer. Use standard
15
-------
spreadsheet calculations so that no serious difficulties will be anticipated regarding
application in other spreadsheet versions. Use Crystal Ball Version 7 or higher, and
confirm test cases using Excel 2003.
Control revisions of the spreadsheet model using descriptive file names. Extensive
revisions or testing with other software versions beyond that initially proposed may be re-
proposed by the Steering Committee if and when a need for such additional work is
identified.
2.4 Simulation Procedure
For each of the measurement errors in Section 3, create an error surface and sample it
according to the aforementioned PDFs. Each error surface represents an additive error—
or a subtractive error if the sign is negative—relative to the reference value to which it is
applied. Figure 5, Figure 6, and Figure 7 serve as a hypothetical PM example of how
these error surfaces should be created for every error. The plots shown correspond to PM
emissions concentration data representing 1 PEMS, two engines, and three exhaust
configurations each, with all 6 sets of PEMS data pooled together. Note that separate
error surfaces will be constructed for each of the three PEMS units (AVL, Horiba and
Sensors). The example applies to the error module for steady-state (SS) bias and
precision PM concentration errors (Section 3.2). These figures will be referenced by
each "Data Analysis" section for the various errors discussed in this test plan.
Errors from Section 3 (Engine Dynamometer Laboratory tests) and Section 4
(Environmental Chamber Tests) are combined by adding all of the sampled errors once
per NTE event trial. For example, in order to assess the errors in PM concentration for
each NTE event, several modules will be created such that:
PM_with errors = PM _ideal + A(ug/mole)i + A(ug/mole)2 + A(ug/mole)3 + ...
where,
A(ug/mole)i = PM concentration errors due to steady state bias and precision
errors,
A(ug/mole)2 = PM concentration errors due to ambient temperature,
A(ug/mole)3 = PM concentration errors due to ambient pressure,
etc....
2.4.1 Construction of the Error Surface
2.4.1.1 PEMS vs. Lab
Acquire raw data with the PEMS at various average concentration levels as per Section
3.2. Plot the "PEMS" signals versus the corresponding "lab" signals that were measured
using lab equipment. This plot pools all bias and precision errors for one PEMS and for
all data from all engines for all steady-state modes. Shown in Figure 5 are the 5th, 50th
16
-------
and 95th percentiles at the mean PM concentration level from the lab (note that the
distribution of data at each level is not necessarily Gaussian). If the 50th percentile is
different than the line of perfect agreement (diagonal), the data suggests that there is a
bias error between PEMS and Lab. In essence this graph shows the statistical distribution
measured by the PEMS at each average concentration level sampled. The example shows
only 6 discrete PM concentration levels (ranging from 10-60 ug/mole). However, the
actual number of discrete levels will be determined by the total number of operating
conditions actually run for all the tests of all the engines. For example, the SS PM testing
will select 6 modes representing typical operating conditions. Thus, the actual plot for SS
PM will likely have 36 discrete concentration levels (6 modes x 1 PEMS x 2 engines x 3
exhaust configurations).
Error Surface forSS PM Concentration
10 20 30 40 50
PM |jg/mole (lab, mean)
60
70
80
diagonal
50th percentile (median)
•95th percentile
-5th percentile
FIGURES. ERROR SURFACE: PEMS VS. LAB
17
-------
Error Surface forSS PM Concentration
PM |jg/mole (lab,mean)
-95th percentile
50th percentile (median)
•5th percentile
FIGURE 6. ERROR SURFACE: (PEMS-LAB) VS. LAB
18
-------
1.5
0.5
tn
-0.5
-1
-1.5
Error Surface forSS PM Concentration
Error Surface: z-axis = ASS_PM_ug/mole
10.1
A--
10
• -•
-4.1
6.8
--A--
20
-3.1
8.8
•-A--
1.8
6.
-A-
-2.2
7.2
-A--
0.2
PM ug/mole (lab,mean)
-A--- 99th percentile
50th percentile (median)
-•-- - 1st percentile
FIGURE 7. ERROR SURFACE: FINAL VERSION
19
-------
2.4.1.2 (PEMS - Lab) vs. Lab
The plot in Figure 6 basically shows the "additive error band" measured during testing.
The plot is created by first subtracting the "lab" PM value from the corresponding
individual PEMS PM measurement for each test run. This difference is defined as the
'delta' error. Next, the "PEMS - Laboratory" delta errors are pooled at each average lab
PM value to obtain the 95th, 50th, and 5th percentile values, respectively, displayed in
Figure 5. Notice that if lab error exceeds PEMS error at a given percentile, crossover of
values can occur. This is acceptable because the crossover effectively reduces PEMS
error whenever lab error exceeds PEMS error.
In order to obtain estimates of the 1st and 99th percentiles for the delta errors for a given
"lab" PM value, each side of the corresponding error distribution will be assumed to
independently fit a normal distribution. Because of the asymmetry of the data, this
methodology will yield two halves of a normal distribution. The median of each normal
distribution will be the median based on the delta errors given in Figure 6. The 95th
percentile delta error will form the upper boundary of one half of the normal distribution,
and the 5th percentile delta error will form the lower boundary of the other half of the
normal distribution. When each side of the data distribution is fitted to a normal
distribution using the above boundary conditions, one can then expand each half of the
distribution from the error surface to obtain the 1st and 99th percentiles of the data for the
given "lab" PM value.
2.4.1.3 Error Surface
This step normalizes the data in Figure 7 using what is called a "variability index (ic)",
which represents the random sampling by the Monte Carlo technique, in order to select a
given error level. This variability index is allowed to vary from -1 to +1. The likelihood
of ic being any value between -1 through +1 is specified by the PDF assigned to ic. In the
given example, ic is assumed to vary according to a normal distribution during Monte
Carlo calculations. This is because it is believed that the distribution of errors due to
steady-state bias and precision will be centered about the 50th percentile of the full range
of conditions measured according to Section 3.2. The pressure and temperature
environmental error modules use uniform probability density functions for their
respective variability index. Each set of data for each lab set-point mean (i.e., lab
reference value) in Figure 6 is normalized by aligning the 1st percentile error from the
fitted normal distributions with ic = -1, the 50th percentile error with ic = 0, and the 99th
percentile error from the fitted normal distribution with ic= +1.
Error surfaces such as the one presented in Figure 7 are the input modules that the Monte
Carlo simulation program will use during calculations of brake-specific PM emissions.
For example, for a given NTE calculation a random ic value is chosen once per NTE
event trial. Let us assume that the first random sample produced an ic= 0.5. Let us also
20
-------
assume that during this NTE event trial, the reference PM concentration is 10 ug/mole.
In this case,
A(ug/mole)i = (3 + 10.1) / 2 = 6.55 ug/mole.
Also, from Figure 7, for ic = 0.5, the reference PM = 10 ug/mole.
For that step in the calculation, the Monte Carlo approach will add this "delta" to the
reference concentration value of 10 ug/mole (10 ug/mole + 6.55 ug/mole = 16.55
ug/mole) to represent errors in steady-state bias and precision for ic= 0.5, and reference
NTE PM =10 ug/mole. If during the same NTE event in the reference data set, a
reference concentration of 35 ug/mole is read, then,
A(ug/mole)i = ((6 + 8.8) / 2 + (2 + 6.2) / 2) / 2 = 5.75 ug/mole (from
Figure 7)
Note that first the error along the ic line perpendicular to the ic axis (in this case the line
along 0.5) is linearly interpolated at each discrete concentration level. Then those
interpolated values are themselves linearly interpolated to determine the error
corresponding to each reference concentration in the NTE event. Note that the random
selection is once per reference NTE event trial, but the error along that ic line is applied to
every second-by-second value within the given reference NTE event, except for PM
concentration in the case of Horiba and Sensors, where no second-by-second information
are available, but different PM concentration levels may be available for a specific NTE
event.
Now let us assume that the error in PM concentration is composed of only 3 deltas:
A(ug/mole)i, A(ug/mole)2 , and A(ug/mole)3 . And let us assume that for a given
reference NTE event trial we have the following values:
• Reference PM at one second= 30 ug/mole
• A(ug/mole)i = 6 ug/mole
• A(ug/mole)2 = -2 ug/mole
• A(ug/mole)3 = -3 ug/mole.
When the model calculates brake-specific emissions by each of the three calculation
methods, it will use the following PM value, which has all of its error applied:
PM = 30 + 6 -2 - 3 = 31 ug/mole.
The application of error at the first selected ic continues during the entire NTE event
without having to randomly sample again. In other words, ic will not change during that
random trial. For all of the variables except for mPM, the errors may continue to change
during an NTE event on a second-by-second basis if their error surface happens to be a
function of level. For the second randomly selected ic this entire process of determining
21
-------
the Aug/mole errors is repeated. The simulation will continue to randomly selected ic
values for thousands of trials until convergence is met.
For the Horiba and Sensors generated reference NTE events, there is only one flow-
weighted PM value for the entire NTE event. During the simulation for these types of
reference NTEs, the single PM value will be used in the interpolation of the
corresponding PM error surfaces (i.e., steady-state PM, transient PM) at all seconds of
the reference NTE event. Since the PM value will not vary from second-to-second, the
only interpolation will occur according to the ic value at each of the simulation trials.
The same second-by-second sampling and interpolation approach would be used for other
deltas such as ambient temp, ambient pressure, shock and vibration, BSFC interpolation,
torque, exhaust flow rate, etc. An overview of the Monte Carlo simulation for PM is
detailed in Figure 8.
Reference NTE
Monte-Carlo Simulation
PM
CO%
NMHC
(ppm)
Exhfiow
(scfm)
Torque
(N-m)
Speed
(rprn)
Fuel
Rate
(L/sec)
CO.%
95th percentile
BSPM
differences
4
Torque +
ATorque
c_Exhflow
*c_Torque
1c_Speed
Fuel Rate +
AFuel Rate
(1) BSPM = f (PM, Exhflow, Torque, Speed)
(2) BSPM = f (PM, Exhflow, BSFCECM)
(3) BSPM = f (PM, CO2, CO, THC, Torque,
Fuel RateECM, Speed)
* Differences = BSPM "with errors" - "Ideal" BSPM
FIGURE 8. OVERVIEW OF MONTE CARLO SIMULATION
Table 3 lists the error surfaces that will be created for use in simulating the BSPM error
differences.
22
-------
TABLE 3. ERROR SURFACES FOR THE BSPM SIMULATION
Calculation
Component
Delta PM
Delta PM
Delta PM
Delta PM
Delta PM
Delta PM
Delta CO
Delta CO
Delta CO
Delta CO
Delta CO2
Delta CO2
Delta CO2
Delta NMHC
Delta NMHC
Delta NMHC
Delta NMHC
Delta NMHC
Delta Exhaust Flow
Delta Exhaust Flow
Delta Exhaust Flow
Delta Exhaust Flow
Delta Exhaust Flow
Delta Exhaust Flow
Delta Exhaust Flow
Delta Torque
Delta Torque
Delta Torque
Delta Torque
Delta Torque
Delta Torque
Delta Speed
Delta Fuel Rate
Test Source
Engine Dyno
Engine Dyno
Environ
Environ
Environ
Environ
Engine Dyno
Environ
Environ
Engine Dyno
Engine Dyno
Engine Dyno
Environ
Engine Dyno
Engine Dyno
Environ
Environ
Environ
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Environ
Environ
Environ
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Engine Manuf
Engine Dyno
Engine Dyno
Error Surface
Delta PM SS
Delta PM Transient
Delta PM Ambient Temperature
Delta PM EMI/RFI
Delta PM Atmospheric Pressure
Delta PM Vibration
Delta CO SS
Delta CO Atmospheric Pressure
Delta CO Ambient Temperature
Delta CO Time Alignment
Delta CO2 SS
Delta CO2 Transient
Delta CO2 Ambient Temperature
Delta NMHC SS
Delta NMHC Transient
Delta NMHC Atmospheric Pressure
Delta NMHC Ambient Temperature
Delta Ambient NMHC
Delta Exhaust Flow SS
Delta Exhaust Flow Transient
Delta Exhaust Flow Pulsation
Delta Exhaust Flow Swirl
Delta Exhaust EMI/RFI
Delta Exhaust Temperature
Delta Exhaust Pressure
Delta Dynamic Torque
Delta Torque DOE Testing
Delta Torque Warm-up
Delta Torque Humidity /Fuel
Delta Torque Interpolation
Delta Torque Engine Manuf
Delta Dynamic Speed
Delta Dynamic Fuel Rate
23
-------
2.5 Model Considerations
2.5.1 Convergence
The main goal of the convergence criteria is to define how many simulation trials at a
given reference NTE event are required to estimate the 95th percentile BSPM emission
differences with a given precision. The convergence method to be used is based on a
nonparametric statistical technique3 which defines a 90% confidence interval for the 95th
percentile of the BSPM emissions differences for an individual reference NTE
simulation. If the width of the 90% confidence interval is less than 1% of the BSPM
emissions threshold, then convergence is met. The following steps define the
convergence method:
1. Run the Monte Carlo simulation for TV trials for a single reference NTE event.
2. Order the BSPM emissions differences from smallest to largest.
3. Identify the trial number at the lower end of the 90% confidence interval
niower = 0.95 * N -1.645-VO.95 * 0.05 * N
4. Identify the trial number at the upper end of the 90% confidence interval
= 0.95 * N + 1.645V0.95 * 0.05 * N
5. Compute (BSPM difference value at nupper)- (BSPM difference value at niower).
6. If the result in (5) < 1% of the BSPM emissions NTE threshold (0.02 g/hp-hr) then
convergence is met.
2.6 Simulation Output
It is important to understand and identify what error surfaces have the most influence
(i.e., sensitivity) on the BSPM emissions 'with errors' and, thus, the resulting BS
emissions differences. Contributions to sensitivity can be attributable to changes in
variance and/or bias.
2.6.1 Sensitivity Variation Effect
During the Monte Carlo simulation for each reference NTE event, sensitivity charts
produced by Crystal Ball will be generated and stored in output REPORT files. Crystal
Ball calculates sensitivity by computing the rank correlation coefficient between every
assumption (error surface) and forecast value (delta BS emissions) while the simulation is
running. Positive rank correlations indicate that an increase in the assumption is
associated with an increase in the forecast. The larger the absolute value of the rank
correlation the stronger the relationship.
Sensitivity charts in Crystal Ball provide a means to determine how the variances of the
error surfaces affect the variance in the forecast values. Hence, the sensitivity charts
developed during a simulation are displayed as "Contribution to Variance" charts which
24
-------
are calculated by squaring the rank correlation coefficients for all assumptions used in a
particular forecast and then normalizing them to 100%. The assumption (error surface)
with the highest contribution to variance (in absolute value of the percent) is listed first in
the sensitivity chart.
Simulation results from all reference NTE events will produce sensitivity values for the
95th percentile delta PM emissions by all three calculation methods.
2.6.2 Sensitivity Bias Effect
Another type of sensitivity to be examined in this study is concerned with the effects of
potential "bias" in error surfaces and their effects on the forecast values. In order to
study these effects a new error surface assumption will be added to the simulation model
for each of the original error surfaces.
This assumption will be sampled as a discrete binary distribution (i.e., on or off) during
the simulation. For each trial of the simulation, the original error surfaces and 'on/off
error surfaces will be sampled according to their defined sample distribution. If the
'on/off error surface produces an 'off condition, the delta emissions from that particular
error surface will not be added to the BSPM emissions computations for the BSPM
emissions 'with errors'. Similarly, if the 'on/off error surface produces an 'on'
condition, the delta emissions from that particular error surface will be added to the
BSPM emissions calculations.
During every trial of the simulation, the exclusions due to the 'off conditions will result
in various combinations of the error surface delta emissions being added to the BSPM
emissions 'with errors' computations. Over the course of a simulation with thousands of
trials, the sensitivity of a particular error either 'on' or 'off will be assessed by
examining the change in the forecast delta emission. Therefore, in a single Monte Carlo
simulation of a reference NTE event sensitivities due to variance and/or bias will be
explored.
25
-------
3 ENGINE DYNAMOMETER LABORATORY TESTS
Utilize engine dynamometer laboratory testing to establish the difference between PM
PEMS and PM based on laboratory measurement in accordance with Part 1065. Also
establish how well ECM parameters can be used to estimate torque and BSFC.
First, however, audit all the PEMS and lab equipment to ensure that they are operating
properly, according to 40 CFR Part 1065, Subpart D. Next, conduct steady-state engine
dynamometer tests to establish PEMS steady-state bias and precision relative to the lab.
Then, conduct transient engine dynamometer testing to determine PEMS transient
precision by repeating transient NTE events. Finally, compare ECM derived torque and
BSFC to laboratory measured torque and BSFC.
3.1 Preliminary Audits
3.1.1 Objective
Conduct 40 CFR Part 1065, Subpart D audits of all engine dynamometer laboratory
systems and all PEMS.
3.1.2 Background
Because the overall purpose of this entire test plan is to establish measurement allowance
that account for the incremental difference in the performance of PEMS versus engine
dynamometer laboratory systems, the first task is to audit all of the measurement systems
to ensure that the specific systems used for testing meet EPA's minimum performance
requirements. The audits also help to minimize bias errors between PEMS and lab
systems measurements. However, in case a specific PM-PEMS does not meet the
specifics of Part 1065 requirement, the MASC will decide on how to move forward by
perhaps allowing some flexibility in passing Part 1065 audit, in situations where it might
be needed, especially if the performance of a system is within the expectation of the
manufacturer.
3.1.3 On-site meeting to establish 1065 compliance requirements
In order to clarify what are all the requirements expected from the lab-grade
instrumentation and PEMS equipment, with respect to 1065 compliance, a meeting will
be held between the test plan steering committee and the contractor at the contractor site
to provide the contractor with guidance regarding which specific sections of Part 1065
Subpart D are required and which are optional. In case Part 1065 requirement is
demonstrated to be too stringent or impractical, the contractor may seek approval from
the MASC to lessen the stringency of Part 1065 in relation to the PEMS.
3.1.4 Methods and Materials
Use the methods and materials described in 40 CFR Part 1065, Subpart D to conduct
audits of all lab and PEMS measurement systems. Even if lab systems and PEMS pass
initial Subpart D audits, allow lab operators and PEMS manufacturers to make on-site
26
-------
adjustments to improve the performance of their systems prior to engine testing. Allow
adjustments to be based on recalibrations with reference signals that are allowed in 40
CFR Part 1065. The steering committee may direct the contractor to calibrate or adjust
the laboratory sampling system based on audit results. The steering committee may also
suggest that a PEMS manufacturer calibrate or adjust one or more PEMS based on lab
audits.
3.1.5 Data Analysis
Use the data analyses described in CFR Part 1065 Subparts D, J and G. For all
subsequent testing, use only those measurement systems that pass the minimum
performance criteria in Subpart D, unless a deficiency is deemed acceptable in writing by
all parties including PEMS manufacturers. Provide a list and brief description of all the
audits conducted for each PEMS manufacturer type. EPA would likely use this list as a
template for the data requirements in the PM portion of the HDIU testing program.
3.1.6 PEMS Manufacturer PM PEMS Commissioning
Notify PEMS manufacturers when the 1065 audits are complete and the first set of PM
PEMS are completely installed in the engine dynamometer test cell—in preparation for
emissions testing. Schedule dates and times that are prior to the start of emissions testing
for each PEMS manufacturer to conduct a final commissioning of all their PEMS that are
on site, including those PEMS that are not installed in the test cell. PEMS manufacturers
may inspect their PEMS and make any final adjustments to their respective PEMS in
order for the PEMS to meet their specifications. Allow PEMS manufacturers to inspect
the installation of their PEMS in the test cell. If PEMS manufacturers take exception to
any portion of the installation or on-site configuration, attempt to resolve any such
installation issues. If such issues are not easily resolvable, notify the steering committee,
who will determine a course of action. Once PEMS manufacturers have completed their
commissioning, notify the steering committee. From this point any further modifications
to the PEMS may only be made according to Table 1 of this test plan.
3.2 Bias and Precision Errors under steady state engine operation
3.2.1 Objective
Evaluate the bias and precision using one engine and one exhaust configuration, shown in
Table 4, and 10 repeats of steady-state modes, and three sets of PEMS units, each set
including the MSS, TRPM, and PPMD. Thus, the total number of NTE steady-state
points required to conduct the steady-state experiments is 30. This constitutes six steady-
state modes of engine operation (6), 10 repeats (10), one exhaust configuration, one
engine (1), and three different PEMS units (3), 6x10x1x1x3= 180.
Determine the AsswPM (-Jjl surface plots for the error model based upon all data pooled.
Note that each brand of PEMS will have its own AsswPM error surface generated for use
27
-------
in both calculation methods 1 and 2. For calculation method 3, the AVL brand PM
PEMS will have a unique AssmPM calculated according to Figure 4 of this test plan.
Recommend six steady-state points based on the PM measurement, using the AVL MSS,
of 80 SS points of the Cummins cycle that is typically used to generate ECM torque and
BSFC errors versus laboratory. The MASC will accept the six steady-state points or
choose alternative points for each exhaust configuration. The objective for the MASC
will be to select steady-state points within a given exhaust configuration that provides a
nominal spread of concentrations within that configuration's target brake-specific levels.
Note that to achieve the brake-specific targets under steady-state conditions, the bypass
might have to be opened further, relative to the transient NTE bypass settings.
TABLE 4. ENGINE, EXHAUST CONFIGURATION, AND STEADY-STATE
MODES
07 Engine 1
No. of Steady-State
Modes for Bypass
Setting 1 (BSPM and
PM Concentration,
representative of PM
threshold of 0.025
g/hp-hr under NTE
Transient Operation)
SSI, SS2, SS3, SS4,
SS5, SS6
PM-PEMS Units
Three Sets of (MSS,
TRPM, and PPMD)
Number of Repeats
10 per Mode per PM-
PEMS Set
3.2.2 Background
Testing will be conducted to capture bias and precision errors in PEMS' emissions
instruments versus the laboratory filter-based method. The tests will be steady-state only.
Note: Section 3.3 (next section) will evaluate precision errors (not bias) due to the
dynamic response of the PEMS instrumentation. The precision error captured during
steady state testing (section 3.2) will have to be subtracted from the overall precision
error captured in section 3.3 in order not to double-count the steady state precision errors
of PEMS instrumentation. This process is detailed in Section 3.3.
3.2.3 Methods and Materials
Use the following systems:
a) One model year 2007 heavy duty diesel engines, equipped with a DPF in the exhaust
(Mack MP9)
b) Nine PM PEMS (3 Sensors PPMD, 3 AVL MSS, 3 Horiba TRPM)
c) One PEMS exhaust flow-meter from Sensors, Inc., and one and from Horiba,
applicable to the engine to be tested
28
-------
d) DPF with Bypass Setting 1 for SS testing, representing a threshold level of about
0.025 g/hp-hr under NTE transient testing
Use the following overall guidelines:
e) Measure PM via the CVS, Part 1065 Lab Method (most recent publication)
f) Measure engine inlet airflow through use of LFE or equivalent
g) Use a series of six steady-state modes, and set each mode time to collect a CVS filter
mass of at least 75 microgram per mode, simultaneously with other PM-PEMS
h) Regenerate DPF system prior to each series of steady-state tests
i) Capture ECM broadcast channels and other common diagnostic channels, as
recommended by engine manufacturer(s), to ensure proper engine operation
j) Do not measure gaseous species by the PEMS
k) Stabilization time =180 seconds, with a different running time per mode to achieve a
75 microgram or higher of PM on the CVS filter
1) Always power off PEMS equipment at end of each day, according to PEMS
manufacturer instructions. Re-start start-up process every day according to PEMS
manufacturer instructions and Part 1065, Subpart J.
m) Whenever PEMS are exchanged, swap the order of the Horiba and Sensors
flowmeters, if the steup allows for it.
6 point steady-state repeat-testing, evaluate bias and precision errors:
a) The MASC will select 6 SS operating conditions for repeat testing from a matrix of
80 SS points containing information on PM emissions using the AVL MSS
b) Randomize the order of the six modes
c) Repeat each six steady-state cycle two or three times, prior to DPF regeneration
d) Each test will use three PEMS (Sensors, AVL, and Horiba) at a time, to measure PM
emissions concentration and exhaust flow rate.
e) Expected test duration is 5 days per PEMS set, with a total of 15 days for all three
sets.
Bypass Setting:
a) Run NTE transient cycle using the CVS filter-based method
b) Set bypass to produce CVS filter-based average brake-specific of about 0.025 g/hp-hr
c) Determine the average PM mass concentration
d) Run the 80 SS Cummins cycle to capture PM concentration at each mode using the
AVL MSS
e) Check the PM concentration levels and select the six-steady state modes from the 80
point matrix. As a first order, check the concentration at the pre-selected steady-state
modes to see if they spread within reason around the concentration produced for the
NTE transient cycle. If not, adjust the bypass as needed to establish the right spread in
brake-specific emissions and concentration for the six steady-state modes
29
-------
f) Make sure that the points selected spread around a brake specific level and
concentration level of a threshold of 0.025 g/hp-hr, and concentration range of 4 to 15
milligram per cubic meter.
3.2.4 Data Analysis
Use the acquired data to create the "error surfaces" to be used by the Monte Carlo
simulation. An example of the steady-state error surface determination is shown in Table
5 forPM.
TABLE 5. EXAMPLE OF SS ERROR SURFACE
Error Surface for SS PM Concentration
Figure 5
x-axis
y-axis
PM ug/mole (lab mean at setpoint)
PM ug/mole (PEMS)
Figure 6
x-axis
y-axis 5th percentile
y-axis 50th percentile
y-axis 95th percentile
PM ug/mole (lab mean at setpoint)
5th [PM ug/mole (PEMS) - PM ug/mole (|ab)]
50th [PM ug/mole (PEMS) - PM ug/mole (|ab)]
95th [PM ug/mole (PEMS) - PM ug/mole (|ab)]
77?e 5th, 50th and 95th percentiles from the (PEMS - lab) delta data will be used to
estimate the 1st and 99th percentiles from assumed Gaussian distributions.
Figure 7
x-axis
y-axis
Z-axiS = Ass_PM_|jg/mole
ic sample frequency
ic sample distribution
PM ug/mole (lab mean at setpoint)
ic ss PM
1st Percentile from Gaussian distribution based on 5th
and 50th [PM ug/mole (PEMS) - PM ug/mole (|ab)]
deltas.
99th Percentile from Gaussian distribution based on 50th
and 95th [PM ug/mole (PEMS) - PM ug/mole (|ab)]
deltas.
50th Percentile based on [PM ug/mole (PEMS) - PM
ug/mole (lab)] deltas.
once per NTE event trial
Gaussian (normal distribution)
30
-------
3.3 Precision Errors under transient engine operation (dynamic response)
3.3.1 Objective
The objective of this portion of the work is to determine the precision error, ATRwPM,
with each PM-PEMS under NTE transient engine operation. This will be achieved by
creating a 20 to 25-minute transient NTE cycle where the PEMS measure in each NTE.
3.3.2 Background
PEMS are expected to operate in a repeatable manner over NTE events as short as 30
seconds. Two sources of PEMS precision error are hypothesized: 1) dynamic response to
rapidly changing signals, and 2) susceptibility to "history" effects. Dynamic response
error includes error due to measurement signal time alignment, and the dissimilarity of
the dynamic response and aliasing of signals; including those signals used to determine
entry into and exit from the NTE zone. History effects include the effects of previously
measured quantities on currently measured quantities. For example, this may be caused
by ineffective sample exchange in the PM emissions sampling volumes, or it may be
caused by one or more sensors' characteristic rise time or fall time. To account for any
dynamic response precision error, the increase in precision error incremental to the
steady-state emissions measurement precision will be incorporated into the overall error
model.
Selection of short NTE cycles (each 32 seconds) maximizes the sensitivity of this test to
effects of dynamic response. Thirty-two seconds was chosen as the minimum instead of
thirty seconds, which is the shortest NTE event time, to ensure that 1 Hz ECM updating
of torque and speed values would be unlikely to interfere with capturing NTE events. For
each repeat of the test cycle, the order of the 30 different NTE events will be the same.
In addition the 29 different intervals separating each NTE event from the next will have a
range of durations and these will be randomly arranged in each test cycle as well. Fixed
arrangement of the NTE events and the inter-NTE events will maximize the sensitivity of
this test to dynamic response and history effects, and make the DPF and bypass operation
very consistent.
The total length of the NTE transient cycle will assume that only 5 quartz crystal of the
Sensors PPMD are working, and it takes five minutes of stabilization time for reusing a
crystal after PM collection. Thus, the same NTE transient cycle used in the gaseous
PEMS program will be used here, except for changes in the inter-NTE times to
accommodate the Sensors PPMD.
3.3.3 Methods and Materials
a. Use a transient engine dynamometer emissions laboratory.
-------
b. Use a laboratory that can accommodate at least three PEMS, their power supplies,
the PEMS flow meters, cables and lines.
c. Use same overall guidelines described in section 3.2, but applied to transient
engine testing.
d. Record the EEPS' total mass signal during transient testing.
Challenge PEMS to 30 different 32-second NTE events, shown in Table 5, over about 23
minute test cycle, or whatever needed to accommodate the need for five crystals of the
PPMD to be operational. Randomize the NTE events shown in Table 6 once, scale up
every fifth inter-NTE time, shown in Table 7, to accommodate the PPMD, and use the
same order for repeat testing. Repeat the test cycle 10 times for each set of three PEMS.
Note that for any torque command that is less than zero, command closed throttle (i.e.
zero or minimum fuel command), and motor the engine at the commanded speed for that
data point. An example of an NTE transient cycle is shown in Figure 9.
Based on 10 repeats with each set of PEMS, the total number of repeats will be 30 cycles,
assuming 1 NTE cycle x 10 repeats x one exhaust configuration x 3 sets of PEMS x one
engine (1x10x1x3x1 = 30). Assuming a 25 minutes of NTE with 30 minutes of forced
regeneration and preparation for the second repeat, the total number of days for NTE
transient testing is 10 days (8 hours per day). This time includes PEMS and engine setup,
PEMS warm up, and daily checks.
TABLE 6. NTE TRANSIENT CYCLE
NTE
Event
NTE1
NTE2
NTE3
NTE4
NTE5
NTE6
NTE7
NTE8
NTE9
NTE10
1 Speed %
Range
17%
59%
Governor line
17%
59%
Governor line
17%
59%
100%
Lower third
Torque %
Range
J32%
332%
J32%
66%
66%
66%
100%
100%
100%
332% - 100%
Description
Steady speed and torque; lower left of
NTE
Steady speed and torque; lower center of
NTE
Steady speed and torque; lower right of
NTE
Steady speed and torque; middle left of
NTE
Steady speed and torque; middle center of
NTE
Steady speed and torque; middle right of
NTE
Steady speed and torque; upper left of
NTE
Steady speed and torque; upper center of
NTE
Steady speed and torque; upper right of
NTE
Highly transient torque; moderate transient
32
-------
NTE11
NTE12
NTE13
NTE14
NTE15
NTE16
NTE17
NTE18
NTE19
NTE20
NTE21
NTE22
NTE23
NTE24
NTE25
NTE26
NTE27
NTE28
NTE29
NTE30
Upper third
Middle third
17% - governed
17% - governed
17% - governed
332% - 100%
332% - 100%
Lower third
Upper third
Middle third
Lower right diagonal
Upper left diagonal
Full diagonal; lower left to upper
right
Lower left diagonal
Upper right diagonal
Full diagonal; lower right to upper
left
Third light — heavy-duty NTE
event from International, Inc. data
set
Cruise; ~ 50 mph
Cruise; ~ 75 mph
Small bulldozer
Large bulldozer
Second of three
FTP
NTE events in
Third light — heavy-duty NTE
event from International, Inc. data
set
First of two NTE
First of two NTE
events in NRTC
events in NRTC
speed
Highly transient torque; moderate transient
speed
Highly transient torque; moderate transient
speed
Highly transient speed; moderate transient
torque
Highly transient speed; moderate transient
torque
Highly transient speed; moderate transient
torque
Transient; speed increases as torque
increases
Transient; speed increases as torque
increases
Transient; speed increases as torque
increases
Transient; speed decreases as torque
increases
Transient; speed decreases as torque
increases
Transient; speed decreases as torque
increases
Sample from LHDE
Sample from HDDE
Sample from HDDE
Sample from NRDE
Sample from NRDE
Seconds used from FTP: 714-725, 729-
743,751-755
Sample from LHDE
Seconds used from NRTC: 423-430, 444,
448-450, 462-481, increased 464 speed
from 40% to 42%
Seconds used from NRTC: 627-629, 657-
664, 685-696, 714-722
1 Speed (rpm) = Curb Idle + (Speed % * (MTS - Curb Idle)
2 Torque (Ibf-ft) = Torque % * Maximum Torque At Speed (i.e. lug curve torque at
speed)
3 Torque (Ibf-ft) = Maximum of (32 % * peak torque) and the torque at speed that
produces (32 % * peak power)
33
-------
TABLE 7. DYNAMIC RESPONSE INTER-NTE EVENTS
INT
Event1
INT1
INT2-6
INT7-10
INT11-14
INT15-18
INT 19-21
INT22
INT23
INT24
INT25
INT26
INT27
INT28
INT29
INT30
INT31
Duration
(s)
10
2
3
4
5
6
7
8
9
11
13
17
22
27
35
5
Frequency
1
5
4
4
4
O
1
1
1
1
1
1
1
1
1
1
Description
Initiation of cycle; INT1 is always first
Shortest and most frequent inter-NTE events
Short and frequent inter-NTE events
Short and frequent inter-NTE events
Short and frequent inter-NTE events
Short and frequent inter-NTE events
Medium inter-NTE event
Medium inter-NTE event
Medium inter-NTE event
Medium inter-NTE event
Long inter-NTE event
Long inter-NTE event
Long inter-NTE event
Long inter-NTE event
Longest inter-NTE event
Termination of cycle; INT31* is always last
Interval speeds and torques are not identical, but they are clustered around zero torque
and the speed at which 15% of peak power and 15% of peak torque are output.
34
-------
Torque-Speed Domain
700
600
500
_ 400
-------
maximum median lab values. Note that some MADSsi values might be zero because the
lab data for that median failed the F-test in the previous section.
For any MADxni-ssi less than zero, set that MADxni-ssi equal to zero.
Create a transient error surfaces using all of the MADTRi_ssi. Be sure to include any
MADTRi-ssi data points that are equal to zero because they will affect the 1st and 99th
percentile values.
36
-------
3.4 ECM Torque and BSFC
3.4.1 Objective
Compare the ECM-based torque and fuel rate with that of the laboratory-based
measurement using the Cummins 80 SS mode cycles. For the laboratory purposes, use
the gas-based fuel flow values instead of the measured fuel flow. Repeat the Cummins 80
SS cycle three times, and use the average values produced.
Use at least six engines for these experiments that include the one engine to be used in
the PM PEMS program and Engine B, C, and D of the ACES program.
3.4.2 Data Analysis
Use the acquired data pooled and normalized to % of max torque and % of maximum fuel
rate to replace the manufacturer submitted error surfaces that were previously used in the
gaseous portion of the Monte Carlo simulation. Refer to section 2.4 for description and
example of an error surface. Include any bias error, unless there is an assignable cause
that would not occur in-use and the steering committee approves to eliminate such bias
error.
4 ENVIRONMENTAL CHAMBER
The environmental chamber tests challenge PEMS to a variety of environmental
disturbances, namely electromagnetic interference, atmospheric pressure, ambient
temperature and humidity, and shock and vibration.
During each of the tests, plus a baseline test, the PEMS will cycle through sampling four
different dilution preparations of aerosol particles that contain volatile hydrocarbon and
elemental carbon using a particle generator that mimics the formation of diesel particles.
The OC/EC will be used to determine the concentration levels needed for the PM
generator. Essentially, after determining the steady-state points to run on the engine, the
OC/EC semi-continuous instrument will be used along with the filter-based method. Then,
for the concentration levels to be used with the PM generator, the OC/EC instrument will
be used to set the PM generator to produce the desired composition and concentration
levels, similar to those encountered under steady-state. Three particle concentration
levels of 5, 10, and 15 mg/m3, as shown in Table 8, will be generated by the particle
generator. Each concentration will be fed to the PEMS after applying dilution ratios of 6,
12, 20, and 30. For each concentration and dilution ratio combination, the PM generator
will be stabilized for 4.5 minutes, and data will be collected by the PEMS for 30 seconds.
The test will continue for a period of 8 hours. The first six cycles of every test will serve
to be the baseline before any environmental change is made.
37
-------
The temperature/humidity and pressure tests are designed to mimic real-world
environmental disturbances with the magnitude and frequency of the disturbances
adjusted to real-world conditions. Randomly sample a uniform distribution of probability
for their /c , from any minute of the test. By randomly sampling from the minutes of
these tests the magnitude and frequency of the real-world error will be built into the error
model, which is described in Section 2. The other environmental tests represent the full
range of possible conditions. For these tests, randomly sample the normal distribution in
Figure 1 for their ic.
For EMI/RFI and vibration, the instruments will be subjected to screening tests with
HEP A filtered air to detect if there any changes in the response of the instruments. Based
on these results, the MASC will decide if the particle generator will need to be used with
these tests.
For the vibration screening test, in order to avoid damage to the instruments, a frequency
sweep will be used at low amplitude. The idea here is to detect the frequency that may
trigger a response by the instrument, without doing any damage due to high amplitude.
TABLE 8. CONCENTRATION AND DILUTION RATIO SCHEDULE WITH PM
GENERATOR
Raw PM Concentration,
u,g/m3
5000
10000
15000
Dilution Ratio
DR1
6
DR2
12
DR3
20
DR4
30
Concentration at Above Dilution Ratio
833.3
1666.7
2500.0
416.7
833.3
1250.0
250.0
500.0
750.0
166.7
333.3
500.0
4.1 Data A nalysis for Environmental Tests
Reduce data by first calculating means for each 30-second period of stabilized
measurements. Subtract from each mean the respective baseline concentration. The
results are errors or "deltas". Correct each of these error distributions by removing their
respective baseline variances, which were determined by quantifying PM Generator
output with no environmental perturbations. Calculate the variance of each of the
distributions. Subtract the respective baseline variance from each calculated variance.
Use the resulting difference in variance as the target variance for adjusting the error
distributions. If the target variance is zero or negative, leave all error values of the
distribution as is and do not proceed to the next step. If the target variance is positive,
iteratively solve to find a single numerical value that can be used to divide each error in a
given distribution such that the resulting distribution has a variance equal to the target
variance. Now each of the errors is corrected for baseline variance.
38
-------
Then, calculate the NTE result with all errors, including torque and flow errors set to zero.
This is the true value. Then subtract the true NTE value from the result with all errors
and record this difference in one of the 7 measurement allowance distributions: mPM
times three calculation methods (torque-speed, fuel-specific * BSFC, ECM fuel flow)
times three PEMS manufacturers, except Sensors and Horiba can not use the ECM fuel
flow calculation method. Then proceed to the next NTE event in the nominal data set.
Repeat the entire nominal data set over and over until all 7 measurement allowance
distributions converge. Follow the data reduction steps set out in Section 2 to select the
final measurement allowance.
4.2 PM Generator Commissioning
The PM generator is developed by EPA. The PM generator can create various
hydrocarbon mixtures along with solid particle generation using carbon rods arcing. The
PM generator is also equipped with a micro-proportional diluter, and is intended to
simulate diesel exhaust particle phase compounds.
EPA will ship the PM generator to SwRI. EPA (Matt Spears) will train SwRI staff on
using it. In addition, SwRI together with EPA may incorporate to it a soot particle
generation mechanism that is different than the carbon rod arcing, using instead a
propane flame mini-CAST technology.
The PM generator will be used during atmospheric chamber testing, temperature and
humidity testing, and may be used during EMI/RFI and vibration experiments.
4.3 Baseline
4.3.1 Objective
The baseline variance will be established using an 8 hour baseline test in which the PM
generator cycles through the same compositions and concentrations of PM used during
the actual environmental tests. Mean values will be determined from the first five cycles
through the PM concentrations. Deviations (deltas) from these mean values during
subsequent cycles through the concentrations will be used to determine the baseline
variance. This variance will be subtracted from the environmental test results.
4.3.2 Background
All of the other environmental tests inherently incorporate the baseline bias variance of
the PEMS. Because the Monte Carlo simulation model adds all the errors determined
from the various environmental tests, it would add the baseline variance of PEMS to the
model too many times. In order to compensate for this in the model, the baseline
variance of PEMS is determined and subtracted from each of the environmental tests'
results.
Note that the baseline variance of PEMS is measured and modeled (i.e. added) once as
part of the steady-state engine dynamometer laboratory experiment.
39
-------
4.3.3 Methods and Materials
For this experiment use a well ventilated EMI/RFI shielded room capable of maintaining
reasonably constant temperature and pressure. Use a room that can house one of each
PEMS, their power supplies, the PEMS flow meters, cables and lines.
Prior to executing the baseline test, setup each PEMS and stabilize the PEMS in the room.
Perform PEMS setup according to 40 CFR Part 1065 Subpart J and PEMS manufacturer
instructions, including any warm-up time, and audit. Then supply the PEMS' sample
ports with the sequence of PM from the PM generator as described at the beginning of
Section 4.
At each PM concentration, flow PM long enough so that stable readings of the PEMS can
be recorded. When the OC/EC analyzer is used to spot-check the output of the PM
generator, ensure that enough time has elapsed to achieve an accurate OC/EC analysis.
Position PEMS and configure PM transport tubing to minimize transport delays and PM
losses.
Test at least one PEMS from each PEMS manufacturer.
4.3.4 Data Analysis
Reduce the baseline data for each PM PEMS, using artificial NTE sampling event times.
Subtract from each mPMthe mean mPM from the initial (short) baseline test of six cycles
through the PM concentrations, which were conducted at the beginning of the test. The
results are errors or "deltas". Calculate the variance of these values, and use them for
baseline variance correction in the data reduction of the remaining environmental tests.
4.4 Electromagnetic Radiation
4.4.1 Objective
Evaluate the effect of Electromagnetic Interference (EMI) and Radio frequency
Interference (RFI) on the performance of the PEMS and determine error factors for the
PEMS due to these effects. First, a screening test on each instrument will be performed
with HEPA filtered air to determine if the EMI/RFI affects the instrument response. If it
does, the MASC will decide on the test matrix required for this evaluation.
4.4.2 Methods and Materials
Use an EMI test facility capable of running the SAE tests listed above. This would
include: Signal generators, Power amplifiers, Transmit antennas, Electric Field Sensors,
Measurement Receiver, Data recording device, LISNs (Line Impedance Stabilization
Networks) and shielded enclosure.
40
-------
4.5 Atmospheric Pressure
4.5.1 Objective
Evaluate the effects of ambient pressure on PEMS PM concentration outputs.
4.5.2 Background
PEMS are expected to operate over ranges of ambient pressures. It is hypothesized that
some of the errors of the PEMS concentration outputs may be a function of ambient
pressure. Therefore, this experiment will change the ambient pressure surrounding
PEMS to evaluate its effects on PEMS measured concentrations and flow meter
transducer outputs. As with all of the environmental tests, the test cycle for this test is
based on the best-known distribution of real world conditions. For this test, the test cycle
pressure distribution was matched to the county-by-county annual average atmospheric
pressure distribution in EPA's 2002 National Emissions Inventory (NEI) model. Figure
10 depicts the NEI data distribution (based on 3149 data points) and the test cycle
pressure distribution.
•inn"/
OfiO/
yu/o
RIW
"5T
~ 70%
"S Rn%
•5
« cn%
>
O A(\o/
0
O" ^fiO/
0 oU/o
LL
ono/
•ino/
0%
Pressure Histograms
2002 NEI Pressure Distribution <5500 ft
Test Cycle Pressure
^M
• •
_ _ • m
84.1 86.8 89.6 92.3
Pressure (kPa)
- • •
95.0 97.8 100.5
FIGURE 10. PRESSURE HISTOGRAM
4.5.3 Methods and Materials
Use a barometric chamber that can be well ventilated and capable of controlling a wide
range of pressure changes (82.74 to 101.87 kPa). Use a chamber that can house at least
41
-------
three PEMS at a time, one of each PEMS manufacturer, their power supplies, the PEMS
flow meters, cables and lines, plus the PM generator.
Follow a pattern of first soaking the PEMS at a constant pressure, then ramp the pressure
to a new pressure, soak the PEMS at that new pressure, and then ramp to another pressure.
Use the sequence of pressures and times, as shown in Table 9, to simulate a typical
distribution of real-world pressures and changes in pressure, which are believed to be
dominated by changes in altitude during driving in the United States.
TABLE 9. ATMOSPHERIC PRESSURE TEST SEQUENCE
Atmospheric Pressure Test Sequence
Phase
1 Soak
2 Ramp
3 Soak
4 Ramp
5 Soak
6 Ramp
7 Soak
8 Ramp
9 Soak
10 Ramp
11 Soak
12 Ramp
13 Soak
14 Ramp
15 Soak
16 Ramp
17 Soak
18 Ramp
19 Soak
20 Ramp
21 Soak
22 Ramp
23 Soak
Pressure
kPa
101
101-97
97
97-101.87
101.87
101.87-101
101
101-97
97
97-96.6
96.6
96.6-82.74
82.74
82.74-96.8
96.8
96.8-90
90
90-96.8
96.8
96.8-99.2
99.2
99.2-101
101
Alt. ft.
89
89-1203
1203
1203- -148
-148
-148-89
89
89-1203
1203
1203-1316
1316
1316-5501
5501
5501-1259
1259
1259-3244
3244
3244-1259
1259
1259-586
586
586-89
89
Time
min
10
20
20
60
20
20
20
20
25
20
20
20
20
30
20
15
10
20
20
20
20
10
20
Rate
ft/min
0
56
0
-23
0
12
0
56
0
6
0
209
0
-141
0
132
0
-99
0
-34
0
-50
0
Comments
Flat near sea-level
Moderate hill climb from sea level
Flat at moderate elevation
Moderate descent to below sea
level
Flat at extreme low elevation
Moderate hill climb to near sea
level
Flat near sea level
Moderate hill climb from sea level
Flat at moderate elevation
Slow climb from moderate
elevation
Flat at moderate elevation
Rapid climb to NTE limit
Flat at NTE limit
Rapid descent from NTE limit
Flat at moderate elevation
Rapid hill climb to mid elevation
Flat at mid elevation
Rapid descent within middle of
NTE
Flat at moderate elevation
Moderate descent to lower
elevation
Flat at lower elevation
Moderate decent to near sea-level
Flat near sea-level
42
-------
Pressure-Time Environmental Test Cycle
Altitude (ft)
1/2-hr moving avg dA/dt (ft/hr)
Vertical gridlines = hours
Vertical gridlines = 7-min gas
cylinder cycle times
-9000
01234
Time (hr)
FIGURE 11. PRESSURE-TIME ENVIRONMENTAL TEST CYCLE
Prior to executing this pressure sequence, setup each PEMS and stabilize the PEMS in
the chamber's first pressure. Perform PEMS setup according to 40 CFR Part 1065
Subpart J and PEMS manufacturer instructions, including any warm-up time, zero-span-
audits of the analyzers and the setup of all accessories including flow meters, ECM
interpreters, etc. Then supply the PM PEMS' sample port with the sequence of PM from
the PM generator as described at the beginning of Section 4.
Flow each generated PM sample long enough so that at least 30 seconds of stable
readings are recorded for the slowest responding gas concentration output of all the
PEMS. Position PEMS and configure gas transport tubing to minimize transport delays.
Target to sample about 30 seconds. Repeat this cycle over the 8-hr test cycle, by cycling
through the concentration shown in Table 8, which represents one hour of testing, using a
4.5 minutes of stabilization and 30 seconds of sampling at each condition.
Perform this test once for one set of PEMS with as many PEMS tested at once.
4.5.4 Data Analysis
Perform data analysis according to Section 4.1.
43
-------
4.6 Ambient Temperature and Humidity
4.6.1 Objective
Evaluate the effects of ambient temperature and humidity on PEMS PM concentration
outputs. The histogram in Figure 12, along with Table 10 and Figure 13, will be updated
by a new temperature profile that takes into consideration the data generated by CE-
CERT.
4.6.2 Background
PEMS are expected to operate over a wide range of changing ambient temperatures. It is
hypothesized that some of the errors of the PEMS outputs may be a function of changes
in ambient temperature. Therefore, this experiment will change the ambient temperature
surrounding PEMS to evaluate its effects on PEMS measured concentrations and flow
meter transducer outputs. As with all of the environmental tests, the test cycle for this
test is based on the best-known distribution of real world conditions. For this test, the test
cycle temperature distribution was matched to the hour-by hour county-by-county
average atmospheric temperature distribution, weighted by vehicle miles traveled
according to EPA's 2002 National Emissions Inventory (NEI) model. Figure 12 depicts
the NEI data distribution (based on over 900,000 temperatures and over 270 trillion
vehicle miles) and the test cycle temperature distribution.
•i nno/ __
ono/
ono/
E 70,%
~ /U/0
"n fin.%
•5
_ cno/
>
O Af\0/
0)
O" 'ino/
ono/
10%
0%
Temperature Histograms
Real World Temperature 23 -100 F
Test Cycle Temperature
28.5 39.5 50.5 61.5 72.5 83.5 94.5
Temperature (F)
FIGURE 12. TEMPERATURE HISTOGRAM
44
-------
4.6.3 Methods and Materials
Use a well ventilated room capable of controlling a wide range of temperature changes (-
23 to 100 °F). Use a room that can house at least six PEMS, their power supplies, the
PEMS flow meters, cables and lines, plus seven different zero, audit, and span gas
cylinders, and a gas switching system.
Follow a pattern of first soaking the PEMS at a constant room temperature, then ramping
the room temperature to a new temperature, soaking the PEMS at that new temperature,
and then ramping to another temperature. Use the following sequence of temperatures,
shown in Table 10, and times to simulate the range of real-world temperatures and
changes in temperature:
TABLE 10. AMBIENT TEMPERATURE TEST SEQUENCE
Ambient Temperature Test Sequence
Phase
1 Soak
2
Ramp
3 Soak
4
Ramp
5 Soak
6
Ramp
7 Soak
8
Ramp
9 Soak
10
Ramp
1 1 Soak
12
Ramp
13 Soak
Temperature
°C
13.89
13.89-5.00
-5.00
-5.00-
12.78
12.78
12.78-
28.33
28.33
28.33-
37.78
37.78
37.78-
22.22
22.22
22.22-
13.89
13.89
°F
57
57-23
23
23-55
55
55-83
83
83-100
100
100-72
72
72-57
57
Tim
e
min
10
5
5
145
40
5
52
5
8
100
60
5
40
Rate
°C/mi
n
0.00
-3.78
0.00
0.12
0.00
3.11
0.00
1.89
0.00
-0.16
0.00
-1.67
0.00
Comments
Cool in-garage pre-test PEMS
operations
Leaving cool garage into cold ambient
Operating at cold temperature outside
of vehicle
Diurnal warming during cool day
Steady cool temperature during testing
Return to hot garage on a cool day
Hot in-garage pre- post- test PEMS
operations
Leaving ho garage into hot ambient
Operating at hot temperature outside
of vehicle
Diurnal cooling during hot day
Steady moderate temperature during
testing
Return to cool garage on a moderate
day
Cool in-garage post-test PEMS
operations
45
-------
Temperature-Time Environmental Test Cycle
emperature (C)
J -^ -^ N
3 O O O C
1- -tu
o.n
-4U
en
T_
i r
^
\
i r
_ ^_^^^
^**—
^"~ — * .
0 1 2
H
-I
s I
I I
J\
— 1 \
r\
/ \
— j \
i
i
i
i
V
r
/
^
\
; ;
_ j
Temperature
1/2-hr moving average dT/dt (C/hr)
Vertical gridlines = hours
Vertical gridlines = 7-min gas cylinder cycle times
34567
Time (hr)
116
92
80
DO
56 _
44 ~
20 1
8 E
-4 i—
-16
-28
-4U
-52
-64
8
FIGURE 13. TIME SERIES CHART OF AMBIENT TEMPERATURE TEST
Prior to executing this temperature sequence, setup each PEMS and stabilize the PEMS in
the chamber's first temperature. Perform PEMS setup according to 40 CFR Part 1065
Subpart J and PEMS manufacturer instructions, including any warm-up time, zero-span-
audits of the analyzers and the setup of all accessories including flow meters, ECM
interpreters, etc.
Run the 8-hour cycle test by stepping through the concentration and dilution ratio shown
in Table 8.
4.6.4 Data Analysis
Perform data analysis according to Section 4.1.
4.7 Orientation and Vibration
4.7.1 Objective
Evaluate the effect of vehicle vibration on the performance of the PEMS and determine
error factors for the PEMS due to these effects. Prior to doing extensive vibration work,
perform a screening using HEPA filtered air sampling at a sweep of different frequencies
with low amplitude. If any of the PEMS shows a response to a particular frequency,
propose a frequency test and submit it for the MASC for approval.
46
-------
5 SWRI CVS AND CE-CERT TRAILER CORRELATION
Prior to performing the in-use work with the PM-PEMS, it is important to establish
the degree of correlation between SwRI CVS-based PM measurement and CE-CERT
CVS-based PM measurement. For this purpose, the CE-CERT trailer will move to
SwRI facilities and PM measurement will be conducted on the engine used for the
PM-PEMS program.
5.1 Method and Materials
Below is a list of a step by step approach for the correlation between SwRI and CE-
CERT
1. Perform a propane check on SwRI CVS and 47 mm filter and CE-CERT
CVS and 47 mm filter. Both systems should pass Part 1065 on propane.
However, even if they pass, note any difference between the two.
2. Set the CVS flow rate to be the same on both systems
3. Set the filter face temperature and velocity to be the same on both systems
4. Set the secondary dilution ratio to be the same on both systems.
5. Use Whatman PTFE membrane filters (7592-104), and filter screens that
meet the latest Part 1065.
6. Modify the exhaust path to SwRI CVS to be comparable with that for the CE-
CERT Trailer
7. Pre-condition the SwRI CVS tunnel and the CE-CERT trailer CVS tunnel for
a period of 10 hours at engine rated power using exhaust configuration with
DPF without bypass. The conditioning time may include active DPF
regenerations.
8. Run a total of 12 repeats of the NTE transient cycle using DPF with Bypass
Level at 0.025 g/hp-hr emission level, over a period of three days. Four
repeats per day with the CE-CERT followed by four repeats with SwRI CVS
and then alternate. Prior to each set of four repeats manually regenerate the
DPF.
9. Use SwRI DMM-230 and CE-CERT DMM-230 to make sure that the engine
PM source is not shifting and being consistent.
10. SwRI should handle and weigh all the filters for both SwRI and CE-CERT in
accordance with their protocol.
11. The CE-CERT trailer is needed at SwRI for at least two weeks per engine.
One week for setup and two weeks of testing assuming the above schedule.
12. In a separate task, EPA will equilibrate and pre-weigh 20 filters using EPA's
weighing protocol. EPA will then ship them to SwRI for repeat preweighing
using their protocol. SwRI will then ship the same filters to EPA for
reweighing. After reweighing at EPA, EPA will ship the filters to CE-CERT
for weighing using CE-CERT's weighing protocol. Finally, CE-CERT will
47
-------
ship the filters to EPA for reweighing. Results will be reported by EPA for
MASC discussion. No threshold for acceptance has been established at the
time of this testplan writing.
13. The target for correlation at the 0.025 g/hp-hr level is CE-CERT's mean of
12 repeats being within +/-10% of the mean value reported by SwRI.
6 MODEL VALIDATION AND MEASUREMENT ALLOWANCE
DETERMINATION
6.1 Model validation
6.1.1 Objective
Validate the Monte Carlo model by
1. Testing the PEMS in parallel with the CE-CERT trailer
2. Checking the data to see if it fits the model predicted based on the laboratory
efforts
6.1.1.1 CE-CERT Validation
The difference between the PEMS results and the CE-CERT trailer results will be
compared to the error predicted by the Monte Carlo model. To validate the Monte Carlo
model, data must be run through the model and the model results must predict the actual
test results within a reasonable level of accuracy.
Validation will be based on the following procedure. For each reference NTE event, the
Monte Carlo model will be used to generate the 5th and 95th percentiles of the simulated
distribution of the brake-specific PM emission differences. In order to obtain simulations
representing similar conditions to those obtained on-road, some error surfaces may need
to be suppressed in the simulations since not all of them may be applicable to the on-road
conditions. The choice of which error surfaces to suppress would need to be made by the
Steering Committee.
Next, the 5th and 95th delta percentiles obtained from the above simulations will be
separately fit to a line or curve using two chosen methods: a linear regression procedure
and a local regression (loess) technique1. Depending on which of the resulting two fits is
best for each set of data (i.e., either for the 5th percentile deltas or the 95l percentile
deltas), the resulting line or curve will be used as one of the lower or upper limits for the
on-road data.
To determine the best fit for a given set of delta percentiles (i.e., 5th or 95th), a simple
regression line initially will be fit to the data. If a least squares linear regression of the 5th
or 95th percentile deltas versus the ideal PM emission has an r2>0.85 and an SEE < 5 % of
the median ideal PM emissions, then the regression line will be used. If this set of
criteria is not met, then a loess fit will be used. Since a loess regression requires the
48
-------
selection of a smoothing parameter2 to smooth the data, the chosen smoothness parameter
should balance the residual sum of squares against the smoothness of the fit.
The on-road delta errors, obtained from the results of collecting data on several NTE
events during on-road operations, will be plotted on a graph containing the 5th and 95th
percentile delta limits determined from the regression fits chosen above. The graph will
consist of a plot of delta PM versus ideal PM. The number of on-road points outside
these limits will be determined and expressed as a percentage of the total number on on-
road data points. If this number does not exceeds 10% of the total number of on-road data,
the simulation data will be considered to be valid.
6.2 Measurement Allowance Determination
6.2.1 Objective
Use the Monte Carlo simulation program developed with data from sections 2, 3 and 5,
and validated with section 5.1 to determine the measurement allowance for all regulated
emissions, at 2007 emissions standards.
6.2.2 Background
After the Monte Carlo model has been validated and confidence in its ability to predict
errors from PEMS instrumentation, the last step in this program will be to actually
calculate a single set of measurement allowance for PM.
6.2.3 Methods and Materials
Using the criteria explained in section 2.2 calculate the various levels of measurement
accuracy corresponding to the three PEMS manufacturers and the brake specific PM
emissions calculations. Use all the various error surfaces developed during this test
program, including those provided by engine manufacturers to the EPA and ARB.
6.2.4 Data Analysis
Use the methodology explained in section 2.2, and Table 2.2 to arrive at the final
measurement allowance.
49
-------
7 TIME AND COST
7.1 Timeline
Table is a tentative timeline projecting the major tasks to be accomplished during this program. The additional work if needed option
is the work that may need to be done if the model did not validate. Otherwise, the final report will be submitted by September 30,
2009.
TABLE 11. PROJECTED PM-PEMS TIMELINE
Tentative Timeline
April May June July August SeptembelOctoberNovembe Decembe
January February March April May June July August Septembei Octobe
PEMS Audit and Engine 1 Setu
S3 Experiments, Eng.1
Transient Experiments. Eng. 1
Data Analysis and Reporting
Environmental Chamber, etc.
Data Preparation for Mode
Model Processing
SwRI CVS/CE-CERTCVS Corr.
CE-CERTIn-Use Testing
Data Processing and Model validation
Additional work if needed
Draft Final Report
Review of Final Report
-------
7.2 Cost
The rough estimated cost is shown in Table 12. Based on the current estimate, a $125,000
of the $200,000 is needed to complete the project.
TABLE 12. PROJECTED COST ESTIMATE
PEMS Training, Setup, Audit, and Debug
Steady-State and Transient Experiments
SwRI and CE-CERT Correlation (1 engine)
PM Generator and Environmental Testing Activities
Modeling Activities (Including CC^)
Data and analysis, reporting, and final report
Contingency if validation fails
General Contingency
Grand Total
Grand Total Without General Contingency
Grand Total without General Contingency and
Contingency if Validation Fails
$660,000
$190,000.00
$75,000.00
$200,000.00
$225,000.00
$150,000.00
$100,000.00
$200,000.00
$1,800,000.00
$1,600,000.00
$1,500,000.00
8 ABBREVIATIONS USED IN BRAKE SPECIFIC EQUATIONS
Method 1:
ePM = brake-specific emission, PM (g/hp-hr)
N = total number (of time intervals) in series
x = amount of substance fraction (mol PM/mol exhaust; note that l|j,mol (emission
constituent)/mol (exhaust) = Ippm (part per million)
n = amount of substance rate (mol/sec, in this case, mol (exhaust)/sec
t = time interval (sec)
fn = rotational frequency (shaft), rev/min
T = torque (N-m)
NOTE: The units of the numerator work out to gemission as is. However, using the
units given for the denominator (RPM * N-m * s), you would still need to divide by
1.978 to get to hp-hr (using RPM * N-m = kW * 9550, 1 hour = 3600 sec, and kW =
hp*0.7457)
-------
Method 2:
ePM = brake-specific emission, PM (g/hp-hr)
MNO2 = Molecular weight, NO2 (-46 g/mol)
N = total number (of time intervals) in series
x = amount of substance fraction (mol PM/mol exhaust; note that l|j,mol (emission
constituent)/mol (exhaust) = Ippm (part per million)
n= amount of substance rate (mol/sec, in this case, mol (exhaust)/sec) that is linearly
proportional to n (Note: this is a proportional sample, which means that you may use a
flow meter that has a span error, as long as its calibration is linear)
t = time interval (sec)
MC = Atomic weight of carbon (-12 g/mol)
wfuel = g (carbon)/g (fuel); Note fuel is roughly 86% carbon by mass
xCproddry = amount of carbon products on a Cl basis per dry mol of measured flow
(exhaust), mol/mol, solved iteratively per 1065.655
xH2O = amount of water in measured flow, mol/mol (see 1065.645 for calculations)
efuel = brake-specific fuel consumption (g (fuel)/hp-hr)
Methods:
ePM = brake-specific emission, PM (g/hp-hr)
MNO2 = Molecular weight, NO2 (-46 g/mol)
wfuel = g (carbon)/g (fuel); Note fuel is roughly 86% carbon by mass
MC = Atomic weight of carbon (-12 g/mol)
N = total number (of time intervals) in series
x = amount of substance fraction (mol PM/mol exhaust; note that Ijjmol (emission
constituent)/mol (exhaust) = Ippm (part per million)
ntjuei = mass rate of fuel (g/sec)
xH2O = amount of water in measured flow, mol/mol (see 1065.645 for calculations)
xCproddry = amount of carbon products on a Cl basis per dry mol of measured flow
(exhaust), mol/mol
t = time interval (sec)
fn = rotational frequency (shaft), rev/min
T = torque (N-m)
t = time interval (sec)
NOTE: The units of the numerator work out to gemission as is. However, using the
units given for the denominator (RPM * N-m * s), you would still need to divide by
1.978 to get to hp-hr (using RPM * N-m = kW * 9550, 1 hour = 3600 sec, and kW =
hp*0.7457)
52
------- |