Determination of PEMS Measurement
Allowances for Gaseous Emissions
Regulated Under the Heavy-Duty Diesel
Engine In-Use Testing Program
Revised Final Report
United States
Environmental Protection
Agency
-------
Determination of PEMS Measurement
Allowances for Gaseous Emissions
Regulated Under the Heavy-Duty Diesel
Engine In-Use Testing Program
Revised Final Report
v>EPA
Assessment and Standards Division
Office of Transportation and Air Quality
U.S. Environmental Protection Agency
Prepared for EPA by
Southwest Research Institute
EPA Contract No. EP-C-05-018
Work Assignment No. 0-6,1 -6,2-6
NOTICE
This technical report does not necessarily represent final EPA decisions or
positions. It is intended to present technical analysis of issues using data
generated in the associated test program. The purpose in the release of such
reports is to facilitate the exchange of technical information and to inform the
public of technical developments which may form the basis for a final EPA
decision, position, or regulatory action.
United States EPA420-R-08-005
Environmental Protection r , „„„
Agency Feburary2008
-------
EXECUTIVE SUMMARY
This report documents a program conducted by Southwest Research Institute® (SwRI®),
on behalf of the U.S. Environmental Protection Agency (EPA), the objective of which was to
determine a set of brake-specific measurement allowances for the gaseous pollutants regulated
under the Heavy-Duty In-Use Testing (HDIUT) program. Those pollutants are non-methane
hydrocarbons (NMHC), carbon monoxide (CO), and oxides or nitrogen (NOX). Each
measurement allowance represents the incremental error between measuring emissions under
controlled conditions in a laboratory with lab-grade equipment, and measuring emissions in the
field using Portable Emissions Measurement Systems (PEMS).
The completion of this program was part of the resolution of a 2001 legal suit filed
against EPA by the Engine Manufacturer's Association (EMA) and several individual engine
manufacturers regarding certain portions of the Not-to-Exceed (NTE) standards. This dispute
was settled on June 3, 2003. A portion of the settlement documents stated:
"The NTE Threshold will be the NTE standard, including the margins built into the existing regulations, plus
additional margin to account for in-use measurement accuracy. This additional margin shall be determined by the
measurement processes and methodologies to be developed and approved by EPA/CARB/EMA. This margin will
be structured to encourage instrument manufacturers to develop more and more accurate instruments in the future."
The program detailed in this report is the result of the aforementioned statement.
Therefore, while this program was contracted through EPA, it represented a joint effort between
EPA, EMA, and the California Air Resources Board (CARB). The Memorandum of Agreement
(MOA) that was part of the settlement documents outlined a process during which a Test Plan
would be jointly developed by EPA, EMA, and the California Air Resources Board (CARB).
SwRI was chosen as the contractor to carry out this Test Plan. All efforts during the program
were conducted under the direction of a joint body, the HDIUT Measurement Allowance
Steering Committee, referred to in this report simply as the Steering Committee. This group was
composed of representatives of EPA, CARB, EMA, and various individual EMA member
companies. The Steering Committee reviewed all decisions and results during this program, and
any changes made to the Test Plan were subject to Steering Committee review and approval
before being executed.
The measurement allowances determined in this program were meant to be assessed in
comparison to certain NTE compliance threshold values. For this program, a single set of NTE
threshold values was determined by the Steering Committee. These values served as the basis
for calculating the final measurement allowances, as well as for the scaling of various other
parameters during this program. The NTE threshold values used for the program are given in
Table 1.
This revised version of the final report contains a number of changes made following
EPA's peer review of the original final report. None of the results or conclusions of the original
report were affected as part of the revision. The changes made to the report primarily involved
additional clarifying language in areas were the peer review process indicated that the original
report was unclear or vague.
SwRI Report 03.12024.06
-------
TABLE 1. NTE THRESHOLD VALUES USED FOR MEASUREMENT ALLOWANCE
PROGRAM
Pollutant
NMHC
NOX
CO
NTE Threshold,
g/hp-hr
0.21
2.0
19.4
The final Measurement Allowance values determined at the conclusion of this program
are summarized in Table 2. These values were unanimously approved by the Steering
Committee, and will be the values published by EPA for use during the HDIUT program. The
effective date for these values was be March 1, 2007.
TABLE 2. FINAL MEASUREMENT ALLOWANCES
Pollutant
NMHC
NOX
CO
Measurement Allowance,
g/hp-hr
0.02
0.45
0.5
The remainder of this report details the process used to determine the values reported
above.
Acknowledgements
This program could not have taken place without significant contributions from a wide
variety of participants. SwRI would like to acknowledge EPA, CARB, and the various
participating EMA member companies for jointly funding this effort. Numerous other
contributions were made as well. Test engines were supplied by Daimler Chrysler, Caterpillar,
and International. In addition Caterpillar also supplied the test truck used for validation testing,
and International contributed the diesel particulate filters (DPFs) used on that truck. A
considerable quantity of PEMS equipment was provided by EPA in order to facilitate the timely
completion of this program. Significant support in terms of training, diagnosis, and technical
support was also provided by Sensors Inc. as the primary PEMS manufacturer participating in
the program. SwRI would like to thank all of the companies who made these contributions.
SwRI would also like to acknowledge the efforts of CE-CERT in completing the on-road
validation tests that were a key portion of this program.
SwRI would also like to extend a special thanks to the various individual representatives
who made up the Steering Committee. Participation in the Committee required a significant
amount of time and energy on the part of these individuals, and SwRI would like to acknowledge
the commitment of the Committee members to the successful completion of this program.
SwRI would also like to thank Mr. Ed Oelkers for his efforts in developing and executing
the Crystal Ball Monte Carlo simulation model.
SwRI Report 03.12024.06
11
-------
Program Methodology
Statistical Simulation Approach (Monte Carlo Model)
During the Test Plan development, it was understood that it would not be feasible to
conduct enough representative experiments to directly quantify the measurement allowances.
Therefore, the Steering Committee chose a methodology which involved the construction of a
statistical Model of the measurement errors. The statistical Model incorporated Monte Carlo
(random sampling) methodology to simulate the variation in errors over repeat measurements.
The Model was then run thousands of times to generate a large data set to allow determination of
a robust set of measurement allowances.
The Model incorporates a variety of error components, each of which represents a
different source of potential error between the laboratory and the PEMS. Each of these error
components was associated with a laboratory experiment designed to characterize and quantify
the effect of a potential error source. The result of each experiment was an empirical model,
often visualized as a three-dimensional surface, which related the chosen test conditions to the
error between a laboratory reference measurement and a PEMS measurement. These empirical
models are thus referred to as "error surfaces" in this report. The individual errors are generally
referred to as "deltas", and are typically characterized as the PEMS measurement value minus
the laboratory reference value. A positive delta indicates a PEMS measurement higher than the
reference, while a negative delta would indicate a PEMS value below that of the laboratory.
A total of 37 error surfaces were incorporated into the Model. The individual error
surfaces encompassed a wide variety of error sources. A number of additional potential error
sources were investigated during the program beyond those which ultimately resulted in error
surfaces. However, in those cases, upon reviewing the experimental data, the Steering
Committee deemed that the errors from those sources were not significant; therefore, inclusion in
the final Model was not warranted. A wide variety of experiments were conducted to examine
the various error terms, but they can be grouped into several major categories.
1. Steady-State error surfaces. These error terms characterized precision and bias
errors over repeated steady-state measurements. The errors were characterized
via steady-state testing in an engine dynamometer test cell. The Model
incorporates steady-state surfaces for each gaseous pollutant and exhaust flow
rate.
2. Transient error surfaces. These error terms characterized precision errors of
repeated measurement of 30-second NTE events. Note that bias errors were
specifically not included in the transient error surfaces due to concerns about the
ability of the reference laboratory methods to accurately quantify emissions over
30-second events. These errors were characterized by repeat transient testing.
The transient cycles were composed of a series of 30-second NTE events whose
order was randomized for each repeat. This testing was also run in an engine
dynamometer test cell. The Model incorporates transient error surfaces for each
SwRI Report 03.12024.06 iii
-------
gaseous pollutant and exhaust flow rate. Transient error surfaces also were
incorporated to look at dynamic errors in the ECM CAN broadcast signals.
3. Torque and BSFC error surfaces. These error terms were included to quantify
the ability of the engine ECM to accurately predict and broadcast torque and
brake-specific fuel consumption (BSFC) in a wide variety of conditions. These
experiments involved subjecting the test engines to changes in a variety of
different conditions (altitude, temperature, fuel, etc.), and comparing laboratory
reference measurements to values generated using parameters broadcast from the
engine ECM via the CAN bus. All of these experiments used steady-state tests
conducted in an engine dynamometer test cell which was capable of simulating a
wide variety of ambient conditions.
4. Exhaust Flow Measurement error surfaces. These error terms characterized
the effect of various installation/measurement conditions (wind, pipe bends, etc.)
on the PEMS exhaust flow meter measurements. These experiments were
conducted in an engine dynamometer test cell, with PEMS measurements
compared to the laboratory reference flow meters, again using steady-state testing.
5. Environmental Testing error surfaces. These error terms were designed to
model the effects of various ambient conditions on the PEMS. The conditions
examined included ambient temperature and pressure, vibration, electromagnetic
interference (EMI), etc. These experiments were conducted using a variety of
environmental test facilities at SwRI, each of which was designed to simulate a
wide variety of change to a given environmental parameter (such as altitude or
EMI interference). In these cases, PEMS were set up to measure standard
reference gases during testing, while the environmental conditions were varied
according to the design of each experiment. The deltas generated for these tests
were between the PEMS measurement and the known reference concentrations.
6. Miscellaneous error surfaces. Several additional error surfaces were
incorporated in the Model to account for diverse error terms, such as time
alignment of data and engine production variability. These error terms involved
computational exercises made using data from some of the aforementioned
experiments, and in some cases, data supplied by participating engine
manufacturers.
Test Methods and Equipment
Engine dynamometer tests were conducted using three different test engines, one Heavy
Heavy-Duty (HHD) engine, one Medium Heavy-Duty (MHD) engine, and one Light Heavy-
Duty (LHD) engine. These engines were contributed to the program by the engine
manufacturers, along with all support needed to insure successful engine operation. These
engines were generally model year 2005 or 2006 engines. In order to simulate a post-2007 test
environment, SwRI procured several diesel particulate filters (DPFs) which were installed in the
exhaust of the various engines during all testing. It should be noted that the filters selected were
SwRI Report 03.12024.06 iv
-------
designed to regenerate primarily via passive regeneration, as active regeneration systems were
not available at the time of this program.
The original intention of the program was to examine PEMS from more than one
manufacturer. However, at the time of this program only one equipment manufacturer, Sensors
Incorporated, was able to supply commercially available PEMS units to the program. Therefore,
the measurement allowance values are based only on measurements made using the Sensors Inc.
SEMTECH-DS instrument. Multiple examples of that instrument were used during the program,
often in parallel with each other, in order to encompass instrument-to-instrument variation errors.
Late in the program, several Horiba OBS-2200 units became available. These units were
incorporated into the program as time and resources allowed. However, all Horiba PEMS
measurements were performed for information purposes only, and no Horiba PEMS data was
used in the generation of the measurement allowances.
The primary engine laboratory reference measurements used for this program were made
using a transient capable engine dynamometer test cell, which incorporated a full-flow CVS
dilution tunnel. The test cell was capable of simulating a wide variety of ambient conditions in
order to facilitate some of the Torque and BSFC error experiments. All of the emission
concentration deltas that went into the Model were generated using the dilute laboratory
measurements as the reference value. However, raw exhaust laboratory measurements were also
conducted during this program for quality assurance purposes, and as an additional check on the
primary reference. The laboratory reference values are summarized in Table 3. All calculations
were made using methods detailed in 40 CFR Part 1065. Unless otherwise stated, all engine tests
were run using U.S. EPA certification grade ultra-low sulfur 2-D diesel fuel.
TABLE 3. LABORATORY REFERENCE METHODS
PEMS Measurement
Gaseous Analyzers - engine
testing
Raw Exhaust Flow
Predicted Torque (from CAN)
Predicted BSFC (from CAN)
Gaseous Analyzers -
environmental chamber testing
Laboratory Reference
Dilute Emission Analyzers1
Measured Intake Air Flow and
Fuel Flow
Measured Torque
Measured Fuel flow and
Power
Standard reference gas
concentrations
Reference Method
Dilute mass calculated using
CVS flow, then raw
concentrations back-calculated
using laboratory raw exhaust
flow
Air Flow measured using
Laminar Flow Element (LFE).
Fuel Flow measured using
coriolis type meter.
Shaft mounted in-line torque
meter
see above notes
Reference values validated on all
bottles at SwRI
Reference NMHC levels were based on laboratory raw measurements due to very low levels.
SwRI Report 03.12024.06
-------
PEMS and Laboratory Audits
In order to insure that all measurements were conducted at the highest quality level, a full
audit of the reference laboratory was conducted prior to the start of testing. In addition, the audit
was performed to verify that all requirements given in 40 CFR Part 1065 were met, and that any
recommended practices were followed to the extent possible. Quality assurance procedures were
in place to insure that all test equipment was maintained within the requirements of Part 1065
throughout the program.
Audits based on 40 CFR Part 1065 were also conducted on all PEMS equipment used in
the program. In addition, PEMS equipment was re-audited whenever equipment failures resulted
in major repairs to one or more PEMS. This occurred on numerous occasions throughout the
program. In general, the PEMS passed the requirements in 40 CFR Part 1065, but there were
exceptions. In cases where the requirements were not initially met, the PEMS manufacturer was
offered an opportunity to correct the problem. However, in cases where no correction was
available, the Steering Committee had the option to approve the deficiency and continue testing.
Individual audit results for each PEMS are detailed in Section 2 of this report. However,
there were several general issues which arose during the audits which are summarized here.
Gaseous Analyzer Linearity
Numerous gas analyzers on the various PEMS units failed to meet the 1065 Subpart D
linearity criteria during the program. Nearly all of these failures were in the regression line
intercept criteria outlined in 1065.307 Table 1, which specify a tolerance on the intercept of 0.5%
of the maximum value expected during testing. Because this value was not known at the time of
the audits, this maximum value was interpreted as the span gas value used for the instrument. It
should be noted that this interpretation resulted in a relatively loose tolerance for this particular
check, which the PEMS still failed periodically. A number of the gas analyzers in various
PEMS, particularly the NDUV analyzers used for NO and NC>2 measurement tended to fail this
requirement high. Certain units passed all linearity criteria.
Sensors Inc. initially re-calibrated one unit as a result of this failure, but numerous other
units were deemed by Sensors Inc. to be operating correctly, and Sensors Inc. indicated that they
felt there were issues with the linearity procedure as written in 40 CFR Part 1065. Due to the
difficulty in continually re-calibrating these units and comments from Sensors Inc., the Steering
Committee ultimately elected to allow testing to continue with PEMS units that failed the 1065
linearity verification. However, this remains an issue to be addressed during in-use testing,
wherein manufacturers will be legally bound to use equipment that meets all 1065 specifications.
It should be noted that the Horiba PEMS passed all 1065 linearity checks.
NOi Penetration Checks and NOi Measurement
Initially all of the PEMS failed the NC>2 penetration check in 1065.376 due to issues
within the sample handling systems of the SEMTECH DS units. This issue turned out to have a
measurable effect on NOX emissions results, due to the relatively high tailpipe NC>2 fractions
resulting from the use of the catalyst-based DPFs. The result was a significant low bias in NOX
SwRI Report 03.12024.06 vi
-------
measurements from the PEMS units as initially configured. As a result, it became necessary for
Sensors Inc. to modify the test equipment during the course of the program. Although such a
modification was initially not allowed in the Test Plan, the Steering Committee approved a
retrofit in order to address this significant measurement error. This modification was
successfully accomplished, and is now commercially available on all new SEMTECH DS units
and as a retrofit for existing units. Following this retrofit, all PEMS passed the NC>2 penetration
check. All of the data that was used in the Monte Carlo Model reflects the use of this retrofit.
However, the issue resulted in significant program schedule delays as the retrofit was designed,
tested, and implemented.
Exhaust Flow Meter Linearity and Calibration
The exhaust flow meter linearity checks required considerable effort on the part of both
SwRI and Sensors Inc., and may ultimately have been a source of some of the bias errors
observed during later testing as well. The issue was directly linked to the size (diameter) of the
Sensors Inc. exhaust flow meter (EFM). The 5-inch flow meters had little difficulty with the
linearity check at SwRI, with only one failing unit which was re-calibrated by Sensors Inc. and
then passed linearity at SwRI. However, the initial linearity checks showed slope problems with
all the smaller flow meters, some low and some high. All of the 3-inch and 4-inch meters were
sent to Sensors Inc. for re-calibration. Linearity checks on the newly calibrated meters indicated
low slopes for the 4-inch flow meters, and even lower on average for the 3-inch flow meters.
Considerable effort was directed into determining the root cause for these discrepancies.
A possible cause for the linearity failures was a design difference between the flow stands
used by SwRI and Sensors Inc. for calibration and linearity checks. The arrangement of the
SwRI flow stand is test flow meter followed by reference flow meter followed by pump, while
the Sensors Inc. flow stand uses the reverse order. Thus the Sensors Inc. calibrations were
performed with the EFM under a slight positive pressure, while the SwRI linearity checks were
performed with the EFM under a slight negative pressure. According to the static pressure
measurement in the PEMS EFMs, the 5-inch meters, which showed minimal error, were under a
vacuum of about 2 kPa, while the 4-inch and 3-inch meters both experienced slightly higher
vacuum levels of about 2.5 kPa.
The Steering Committee ultimately decided to authorize recalibration of the Sensors Inc.
flow meters using the SwRI linearity data. The average slope adjustment was on the order of a 4
percent positive offset for both the 3-inch and 4-inch flow meters.
Engine Dynamometer Results
Detailed information on the results of all of the engine dynamometer laboratory
experiments performed to generate individual error surfaces is given in Section 3 of this report.
However, there were several overall trends which affected the results of several of the
dynamometer experiments, which are discussed below.
SwRI Report 03.12024.06 vii
-------
NMHC Measurement and Low NMHC Emissions
Although the engine experiments were designed to quantify errors in NMHC, CO, and
NOX, the NMHC measurements presented a particular problem due to the very low levels of
NMHC observed during testing. This was due to a combination of relatively low engine-out
NMHC levels and the use of catalyst-based DPFs. In this program, the DPFs used were CRTs
procured from Johnson Matthey. In addition, the DPFs were sized to be larger than normal for
these engines in order to keep engine backpressure levels low, because none of these pre-2007
test engines were originally designed to operate with DPFs installed. The resulting DPF-out
NMHC levels were at or near zero all of the time.
An attempt was made to address this issue by using a 2007 production-style DPF, which
likely had lower precious metal loadings, on one of the test engines. However, NMHC was still
at a level of less than 10 percent of the 2007 standard level of 0.14 g/hp-hr. The levels observed
were often below the noise limitations of the laboratory reference method for all test engines,
thus complicating the generation of meaningful deltas. As a result, the data analysis methods
and resulting error surfaces for NMHC were modified considerably from the Test Plan.
An additional issue with NMHC derived from the fact that no commercially viable
method of in-field NMHC measurement existed at the time of the experiments. All available
PEMS measured only total hydrocarbons (THC). Therefore, the PEMS NMHC measurements
were THC measurements multiplied by a factor of 0.98 as allowed in CFR 40 Part 1065.
Engine-PEMS Installation Variability
The Test Plan was designed to capture PEMS variability from unit-to-unit and engine-to-
engine. However, it was anticipated that there would be a certain amount of uniformity in the
measurement error trends and the response of the PEMS observed from one engine to the next,
despite the different installations, and in some cases different measurement equipment (such as
different sized exhaust flow meters). When the results of experiments on all three engines were
compared, it was apparent that reproducibility from engine installation to engine installation was
a more important variable than expected. This resulted in the need to modify some of the
initially planned data analysis methods to account for the unexpectedly large source of variation.
CO Measurement
CO measurements throughout the program were generally affected by the relatively poor
resolution of the NDIR detector used for CO measurement in the SEMTECH DS. Tailpipe CO
levels during this program were orders of magnitude below the CO NTE threshold levels (due to
the catalyst-based DPFs). The NDIR detector in the SEMTECH DS uses the same percent scale
resolution for CO that is used for CO2. As a result, the minimum resolution is 0.001 %, or 10
ppm. In addition, it was found that simply switching from the calibration gas port to sample line
generally resulted in a reading of roughly 20 to 60 ppm, even when reading zero gas through the
sample line. However, this lack of accuracy at low levels is not likely to be a compliance issue,
as the tailpipe levels observed with the SEMTECH-DS CO even with resolution and bias issues
were still orders of magnitude below the NTE threshold. Therefore, no particular modifications
were made to the Test Plan to account for this issue.
SwRI Report 03.12024.06 viii
-------
Individual Error Surfaces
The following is a brief overview of the results for each of the various engine
dynamometer experiments. This summary gives a broad overview of the general magnitude of
each error term, as well as any major issue or findings associated with a given experiment.
Detailed information for each experiment is given later in the report.
Steady-State Error Surfaces
Considerable effort was expended in the generation of the steady-state data sets, as the
data from these experiments was also used in the analysis of data from most of the other engine
experiments. These experiments were run on all three engines, with three PEMS run
simultaneously for all of these experiments. Due to various equipment failures, the same three
PEMS were not used on all three engines. The steady-state error surfaces deal with both bias and
precision errors. Steady-state deltas were generated by comparing PEMS measurements to
laboratory measurements for each individual data point, and the data were then pooled to
generate the error surface. Because these matched pairs of PEMS-laboratory data were used, the
steady-state error surfaces were not affected by variability of the test article itself.
The final NMHC error surface incorporated only data from Engine 2, which was the only
engine showing a significant number of non-zero PEMS THC readings. The magnitudes of the
error deltas for the steady-state error surfaces are summarized in Table 4. For errors which were
not level dependent, the size of the error is shown as a percentage of the average value at the
appropriate NTE threshold. It should be noted that these average concentration values at the
thresholds are only estimates which were calculated by examining NTE data supplied by various
engine manufacturers, and that these calculations assume certain average power levels and flow
rates. These values are used only as a means to portray the magnitude of the steady-state errors.
TABLE 4. MAGNITUDE OF ERROR TERMS FOR STEADY-STATE ERROR
SURFACES
Percentile
5th
50th
95th
Error Magnitudes
NMHC,
% threshold1
0%
1%
7%
CO,
% threshold1
0.3%
1.1%
2.0%
NOX,
% threshold1
-5%3
0
5%
C02,
% threshold1
0.3%
0.4%
0.8%
EFM,
% max2
-1%
5%
11%
1 %threshold = percent of average concentration at NTE threshold, or for CO2 average value
during "typical" NTE event (NMHC = 60 ppm, CO = 4450 ppm, NOX = 290 ppm, CO2 = 8 %)
2 %max = percent of maximum value, varies by flow meter size
3 Above 400ppm, NOX 5th percentile appeared level dependent at -14% of point
The NOX error surface was complicated by the engine-to-engine variability issues
described earlier. Steady-state NOX errors were generally independent of level. However, at
SwRI Report 03.12024.06
IX
-------
NOX concentrations above 400 ppm, larger negative errors were observed. These errors showed
a dependency on level; generally at about -14% of point (positive errors remained unchanged).
The NOX errors were due to negative biases observed for some of the PEMS during tests on
Engine 3. The reason for this negative bias is not fully understood, as the PEMS passed all NOX
related Part 1065 QA checks during this time.
The steady-state exhaust flow meter error surfaces were also complicated by large
variations in observed errors engine and test installation to another. A different size flow meter
was used for each of the three test engines, and each size flow meter appeared to have different
magnitudes of error. In general, a net positive bias was observed with the PEMS EFMs as
compared to the laboratory, with larger biases for the smaller diameter EFMs. Some of this error
may have been the result of calibration method differences between Sensors Inc. and SwRI, as
discussed earlier, but the calibration differences were not large enough to account for all of the
positive bias observed.
Transient Error Surfaces
The transient error surface experiments were also run on all three test engines. This data
set was used not only to generate the transient error surfaces, but also to generate other error
surfaces dealing with dynamic and time alignment errors that are described later. As with the
steady-state experiments, three PEMS were run in parallel for the transient experiments, although
the same three PEMS were not used for all three engines.
The transient error surfaces deal with precision errors that result from transient operation.
Although bias errors could have been quantified, the Test Plan specifically excluded bias error
from the transient error surfaces. As a result, PEMS variability was characterized with respect to
the median PEMS value for a given NTE event, without direct reference to the transient
laboratory data. A secondary task for this experiment was to provide an initial assessment of the
laboratory's ability to repeat such short transient measurements.
The transient error surface data analysis was complicated by the desire to correct for
precision errors already characterized by the steady-state measurements, so that the transient
surfaces would characterize only the incremental error due to transient operation. The method
given in the Test Plan called for the variability of the steady-state measurements to be subtracted
from the variability observed for the transient experiments, on an engine-by-engine basis.
However, because both of these variability terms are evaluated across all the repeats for a given
engine as a pooled data set, the analytical method was particularly vulnerable to issues related to
variability in the test article itself (i.e. variability in the pooled NOX level during steady-state or
transient testing).
This vulnerability manifested on several occasions throughout the transient error surface
experiments. In some cases, it was addressed by removing selected outliers where the engine did
not repeat from the pooled data sets for both steady-state and transient experiments. In other
cases, however, this approach was not adequate to address variability problems. For Engine 3,
steady-state variability was intermittently higher than transient variability for many concentration
levels. As a result, the transient data analysis methodology was modified considerably from the
one originally designed in the Test Plan. Because bias errors were not included, all transient
SwRI Report 03.12024.06
-------
error surfaces have 50th percentile error values of zero. A summary of the magnitude of the
transient error surfaces is given in Table 5. The gaseous emissions errors showed a dependency
on level, and are therefore given as percent of point values. The EFM transient errors were not
as level dependant and are given is a percent of maximum flow.
TABLE 5. MAGNITUDE OF ERROR TERMS FOR TRANSIENT ERROR SURFACES -
GASEOUS ANALYZERS AND EXHAUST FLOW
Percentile
5th
50th
95th
Error Magnitudes
NMHC,
% point
-0.03%
0%
3%
CO,
% point
0%
0%
0%
NOX,
% point
-2.5%
0%
2.5%
COi,
% point
1%
0%
1%
EFM,
% max1
-0.7%2
0%
0.6%2
1 %max = percent of maximum flow rate, varies by flow meter size
2 Values represent average across range of range of flow, individual 5th and 95th percentile values
varied
Another set of transient error surfaces were generated to capture the effects of transient
operations on ECM broadcast signals that are used to predict torque and BSFC. These error
surfaces were again designed only to capture precision errors, and therefore the PEMS deltas for
each repeat were generated with respect to the median PEMS value for a given event. The
magnitude of these ECM-related transient error surfaces is summarized in Table 6.
TABLE 6. MAGNITUDE OF TRANSIENT ERROR TERMS FOR ECM VARIABLES
Percentile
rth3
50th3
95th3
Error Magnitudes
CAN- Speed,
% point
-0.2%
0%
0.2%
CAN-Fuel Rate,
% max1
-0.8%
0%
0.6%
Interpolated
Torque,
% max1
-0.9%
0%
0.7%
Interpolated
BSFC,
% average2
-0.2%
0%
0.2%
% max = percent of maximum engine torque or fuel rate
2 % average = percent of average BSFC over a "typical" NTE event = 245 g/kW-hr
3 Values represent average across range of measurement, individual 5th and 95th
percentile values varied and may be as large is 2-3 times averages
Torque and BSFC Error Surfaces
There were a number of engine experiments associated with various sources of error in
torque and BSFC estimation. Some of these were run only on selected engines, as noted below
for each error surface. The PEMS values for these surfaces were not broadcast directly from the
engine ECMs. Rather ECM CAN broadcast speed and CAN broadcast fuel rate were recorded
during these experiments. For each test engine, a 40-point steady-state map was run to
interpolate torque and/or BSFC from CAN-speed and CAN-fuel rate. The recorded values were
SwRI Report 03.12024.06
XI
-------
post-processed via interpolation to provide the resulting PEMS values for comparison to the
laboratory reference values. A summary of the magnitude of each of the torque and BSFC error
surfaces is given in Table 7. Each of the error surface types is summarized briefly below.
TABLE 7. MAGNITUDE OF TORQUE AND BSFC ERROR SURFACES
Error Surface
Torque
Interacting Parameters - DOE2
Interacting Parameters - Warm-up
Independent Parameters
Interpolation
BSFC
Interacting Parameters - DOE
Interacting Parameters - Warm-up
Independent Parameters
Interpolation
Error Magnitudes
5th
Percentile
50th 95th
Percentile Percentile
% max1
-0.5%
-5.9%
-1.0%
-0.9%
0.6%
0%
0%
0.06%
2.3%
5.9%
1.8%
1.6%
% average3
-4.2%
-3.6%
-1.8%
1.0%
-1.5%
0%
0.2%
0.3%
0.8%
3.6%
1.2%
3.7%
1 % max = percent of maximum engine torque
2 DOE percentiles are average percentiles for whole load range, values varied somewhat by level
3 % average = percent of average BSFC during "typical" NTE event = 245 g/kW-hr
Interacting Parameters - Design of Experiment
The Design of Experiment (DOE) experiment was designed to characterize errors in
predicted torque and BSFC based on a variety of operating and environmental conditions. The
conditions included barometric pressure, manifold temperature, exhaust restriction, and inlet
restriction. These parameters were all varied according to the DOE test matrix. Using steady-
state testing, this experiment was run on two of the three test engines. The data was all pooled
together to form a single error surface. A "baseline" set of tests were used to remove
interpolation errors from the data set, because those errors are already accounted for elsewhere in
the Model.
Interacting Parameters - Warm-up Experiment
The warm-up experiment was designed to capture errors in predicted torque and BSFC
related to variations in engine fluid properties and operating temperatures, including viscosity
effects. An exhaustive test matrix of these parameters could not readily be conducted; therefore,
these errors were dealt with collectively using a relatively simple cold-start warm-up experiment.
This experiment was run on all three engines. However, two of the three engines (both of which
were EGR equipped) were started from low room temperature condition (roughly 15°C), while a
third engine (non-EGR equipped) was soaked to a temperature near 0°C prior to engine start.
The error surface values were characterized by finding the maximum error observed during the
experiment after the point in time where all engine temperatures had reached the entry point of
SwRI Report 03.12024.06
xil
-------
the NTE zone, as defined in CFR 40 Part 86. The data from all tests was pooled together to
generate a final error surface.
Independent Parameters
The independent parameters experiment characterized errors in predicted torque and
BSFC caused by changes in fuel or ambient humidity levels. Three different ULSD fuels were
tested which spanned a wide range of properties including aromatic content, density, and cetane
number. Three humidity levels were also run from near zero humidity to levels near 28 g/kg. A
full nine point test matrix was run testing all nine combinations of these parameters. A clear
trend was observed for fuel changes, while humidity changes did not demonstrate an obvious
trend. All of the data for all test points was collected into a single error surface. This experiment
was run only on the MHD engine.
Interpolation Torque and BSFC Errors
During the design of the Test Plan, it was determined that a 40-point speed-load matrix
would be used to define an interpolation grid for predicted torque and BSFC from CAN-speed
and CAN-fuel rate. While this matrix served the needs of the program, it was felt that in real-
world testing, the test matrix was too dense, placing an excessive mapping burden on individual
engine manufacturers. The Steering Committee determined that a 20-point speed-load matrix
would be a more acceptable level of effort. However, the less dense grid would likely lead to
more interpolation errors in use.
The interpolation error surfaces were designed to capture the incremental error involved
in dropping from a 40-point matrix to a 20-point matrix. The generation of this surface was a
computational exercise carried out using the initial 40-point steady-state map data generated for
each engine. The Steering Committee down-selected 20 points from those 40 to generate the
coarser grid. A matrix of several thousand CAN-Speed and CAN-Fuel Rate combinations was
run using both 40-point and 20-point grids, and these data sets were compared to generate the
final deltas, 20-point values minus the 40-point values. Percentile values from this data set were
averaged for all three engines to derive the final error surface.
Exhaust Flow Meter Error Surfaces
There were three exhaust flow meter installation experiments, each of which dealt with a
different potential error source. All of these experiments were conducted only using Engine 1,
which used a 5-inch diameter Sensors Inc. exhaust flow meter. The first dealt with errors due to
pulsations in the exhaust. For this experiment, the DPFs were removed from the exhaust and the
flow meter was relocated to a position relatively close to the turbocharger outlet. The second
experiment dealt with non-uniform velocity profiles in the EFM introduced by pipe bends
upstream of the flow meter. This second experiment was referred to as swirl error. The
magnitude of these error terms is summarized below in Table 8.
SwRI Report 03.12024.06 xiii
-------
TABLE 8. MAGNITUDE OF ERROR TERMS FOR EXHAUST FLOW ERROR
SURFACES
Percentile
50th
Total spread 5th to 95th 2
Pulsation
Errors,
%max1
0.5% to 2%
0.2% to 0.8%
"Swirl"
Errors,
%max1
0.1% to 0.9%
0.1% to 0.5%
%max = percent of maximum flow rate, varies by flow
meter size
2 This value is the total width of the error band between the
5th and 95th percentile boundaries.
A third experiment was conducted to examine the possible effects of air currents up to 60
mph across the outlet of the EFM in various directions. The wind experiments resulted in no
significant errors; therefore this error surface was removed from the Model.
Miscellaneous Error Surfaces
There were several error surfaces which either did not fit under the above categories, or
were based on data taken outside this program. These error surfaces are described below.
OEM Torque and BSFC Error Surfaces
The OEM torque and BSFC error surfaces were generated based on data supplied by the
various engine manufacturers directly to EPA. The intention for these error surfaces was to
characterize errors based on a variety of terms chosen by joint agreement of the Steering
Committee members. Some of these error sources include production variability, the action of
various AECDs, etc. The data was combined and analyzed by EPA. Discussions were held
between EPA and individual engine manufacturers, due to the confidential nature of much of the
information being disclosed. At the end of this process, EPA submitted a single set of error
surfaces, which was approved by the Steering Committee for inclusion in the Model. The
magnitude of these errors is described in Table 9.
TABLE 9. MAGNITUDE OF OEM ERROR SURFACES
Percentile
5th
50th
95th
Error Magnitudes
Torque,
% point
-5.9%
0%
5.9%
BSFC,
% point
-6.5%
0%
6.5%
SwRI Report 03.12024.06
xiv
-------
Time Alignment Errors
The time alignment error surface captured the effect of errors in time alignment of the
various continuous PEMS data sources on the final brake-specific emission results. This error
source was not originally included in the Test Plan, and no experiment had been designed to
examine it. However, during various Steering Committee discussions over the course of the
program, it was decided that time alignment was a potentially significant source of error, and that
it should be incorporated into the Model. This proved difficult because unlike many of the other
terms which dealt with a single measurement term, time alignment is associated with the
collection of the various data streams into the final result. Therefore, a single additive delta
could not easily be generated.
Ultimately, the Steering Committee settled on a multiplicative adjustment factor which
would be applied after all other error terms had been added and the final brake-specific result had
been determined. A separate factor was developed for each pollutant, and for each of the three
calculation methods allowed in the HDIUT program. The error values were generated using a
set of transient data from each engine. Time alignment of three data streams; the gaseous
analyzers, the exhaust flow meter, and the ECM vehicle interface data stream, were perturbed
relative to one another by increments of 0.5 and 1 second alignment errors in various
combinations forward and backward. The brake-specific emission levels for all 30 NTE events
in the cycle were calculated for each misaligned data set, and were compared to values calculated
using the nominal time alignment values. The errors were pooled across all three engines to
arrive at a final set of error terms. Time alignment values were only generated for NOX and CO,
because NMHC values were too low and stable to see any discernible trends in NMHC due to
time alignment. The final time alignment values are given in Table 10 below.
TABLE 10. MAGNITUDE OF TIME ALIGNMENT ERROR SURFACES
Calculation
Method
1
2
3
Percentile
5th
50th
95th
5th
50th
95th
5th
50th
95th
Error Values, % point (BS emission level)
CO
-7.5%
0.0%
4.6%
-5.4%
0.0%
5.1%
-5.2%
0.0%
12.3%
NOX
-3.2%
-0.1%
1.5%
-1.3%
0.0%
1.5%
-1.4%
0.0%
2.9%
The use of the time alignment error term was not universally accepted by all of the
Steering Committee members, due to concerns over the method by which it was applied, and the
potential magnitude of its effects compared to all other error terms. However, the majority vote
of the Steering Committee was to include this error surface in the final Model.
SwRI Report 03.12024.06
xv
-------
Environmental Chamber Results
Detailed information on the results of all of the environmental chamber experiments
performed to generate individual error surfaces is given in Section 4 of this report. However,
several general observations can be made regarding the environmental chamber test results.
These tests were different from the engine dynamometer tests in that they did not involve
the sampling of engine exhaust. Rather PEMS errors during environmental chamber testing were
conducted while sampling reference gases at various concentrations over an automated sequence.
These gases were sampled continuously while the various environmental disturbances were
applied to the PEMS. Environmental error sources that were examined included the effects of
ambient temperature, ambient pressure (i.e., altitude), vibration, and electromagnetic interference
(EMI/RFI). In addition, because the PEMS HC instrument used ambient air for the FID burner
air supply, the effect of ambient HC variation was examined on the NMHC measurement.
In most cases, the observed effect of most of the environmental disturbances was
relatively small, as compared to other error sources examined during this program. During the
course of environmental testing, it was often noted that the PEMS exhibited similar variations on
analyzer response whether the environmental disturbances were applied or not. The exception to
this general trend was NMHC, which demonstrated considerable variation as a result of both
temperature and ambient HC variation. This is despite the fact that a fairly broad range
disturbance was applied for each potential error source. It should be noted that in some cases,
particularly for the vibration and EMI/RFI experiments, the range of environmental disturbances
was actually sufficient to cause occasional functional failures of the PEMS.
The relative magnitude of the environmental error surfaces is given in Table 11. In
general, these error surfaces were centered around a zero error, with the table value showing a
typical maximum range of the error surface values, as a percentage of average value at the
appropriate NTE threshold. Generally, this error could be either positive or negative.
TABLE 11. MAGNITUDE OF ERROR TERMS FOR ENVIRONMENTAL ERROR
SURFACES
Error Surface
Temperature
Pressure
EMI/RFI
Vibration
Ambient HC
Maximum Error Magnitudes
NMHC,
% threshold
7.5 %
2.5 %
2
2
10%
CO,
% threshold
0.1%
0.7 %
2
2
n/a
NOX,
% threshold
2.8 %
2
2
2
n/a
C02,
% threshold
0.5 %
2
2
2
n/a
EFM,
% max1
0.2 %
0.5 %
0.3 %
2
n/a
1 %max = percent of maximum value, varies by flow meter size
2 a dash (-) indicates that error effect was not deemed significant enough to justify inclusion in
the model
SwRI Report 03.12024.06
xvi
-------
It should be noted that the effect of the environmental surfaces was limited in the overall
model by design. This was done in an attempt to simulate the effect of the drift check criteria in
40 CFR Part 1065. The effect of this drift check was simulated in the model by comparing the
brake-specific results of each model run, both with and without the environmental errors applied.
If these two results diverged by more than the tolerance allowed in 40 CFR Part 1065, generally
4 percent, the result of that particular model run was discarded for the pollutant in question as
having failed the drift check. Therefore, the maximum potential effect of the environmental
error surfaces was limited.
Given the magnitude of the environmental error surfaces for NMHC, it was recognized
that a large number of model runs were likely to fail this drift check at a 4 percent tolerance.
Therefore, EPA agreed to widen the drift check tolerance for NMHC for in-use testing from 4
percent to 10 percent. This change will be applied to 40 CFR Part 1065 Subpart J.
Monte Carlo Model Results and Validation
The results of the Monte Carlo simulation run, as well as the details of procedures used to
validate these results are given in Section 5 of this report. A summary of these is given below,
including a brief description of the selection of the final Measurement Allowances which were
given at the front of the Executive Summary.
The final model run to generate the measurement allowance values using all of the data
described above was a significant investment of time and resources. A data set of 195 "reference
NTE events" was used to conduct the model run. Each event was run through the model at least
10,000 times and in some cases many more times. During each repeat, all of the error surfaces
described above were randomly sampled, and the resulting errors were applied to appropriate
terms (i.e., concentrations, exhaust flow, etc.). The model would then calculate several sets of
brake-specific results. An "ideal" result would be calculated from the un-perturbed reference
data for the event in question. Then a set of perturbated results would be calculated from the
data set after all errors had been applied. Calculation of the perturbated results was done using
each of the three brake-specific calculations allowed by 40 CFR Part 1065 Subpart J for in-use
testing. The perturbated results were then compared to the ideal result to generate a delta. The
details of each of the three calculation methods are described in the background information
given in Section 1 of this report.
For each of the 195 reference events, the resulting deltas for all of the 10,000 or more
repeats were pooled together, and a 95th percentile delta was determined for each pollutant by
each of the three calculation methods. These 95th percentile deltas were pooled together for all
195 reference events in order to generate a final potential measurement allowance value. A set
of 9 candidate measurement allowance values was determined, three values for each pollutant
(NMHC, CO, NOX), one for each calculation method. The candidate values determined by the
model run are given in Table 12. The results of model validation are also shown in this table.
SwRI Report 03.12024.06 xvii
-------
TABLE 12. MODEL RESULTS AND VALIDATION
Measurement Errors (%) at Respective NTE Threshold
Emission
BSNOx
BSNMHC
BSCO
Method 1
"Torque-Speed"
22.30
10.08
2.58
Method 2
"BSFC"
4.45
8.03
1.99
Method 3
"ECM Fuel Specific"
6.61
8.44
2.11
Note: values in white cells were validated successfully, while values shown in gray
cells were not validated.
As is the case with all simulation results, no model values should be used until those
results are somehow validated against a set of real test data. The generation of a validation data
set was a considerable challenge, as it required in-use testing with PEMS to be performed, while
at the same time requiring comparison to some acceptable form of reference measurement. This
was required because the output of the model is a set of deltas between a PEMS measurement
and a laboratory reference measurement.
Two methods of generating validation data were used in this program. The primary
method involved on-road field testing using one of the PEMS that was examined during this
program. The reference for this on-road validation testing was the CE-CERT Mobile Emission
Laboratory, which is operated by the University of California-
Riverside. This unique facility incorporates a full-flow CVS dilution tunnel and measurement
system into a trailer which can be pulled behind a Class 8 heavy-duty truck. During validation
testing, truck exhaust was sampled simultaneously by the PEMS and the mobile laboratory, in
order to generate deltas. As an added quality assurance measure, the mobile laboratory was
correlated to the SwRI reference laboratory test cell, in order to eliminate any potential effect of
biases between the two facilities on the model validation effort.
A secondary validation data set was generated in the SwRI dynamometer laboratory
reference test cell. This was done because the on-road validation could not incorporate any form
of reference torque and fuel flow measurements, to allow validation of torque and BSFC error
terms. Therefore, selected portions of the on-road testing operation were re-played in the
dynamometer laboratory, in order to try to validate the errors predicted by the model for torque
and BSFC sources.
The final result of these validation exercises is depicted in Table 12. As noted in that
table, the model result for NOX was validated only for calculation Method 1. The model result
for NMHC was validated for all three calculation methods, while the CO results did not validate
for any of the three calculation methods. As has been noted earlier, the test engines used during
this program generated very low levels of CO, orders of magnitude below the NTE thresholds,
and therefore the model result based on that data was not likely to be a good predictor of actual
measurement errors at the CO compliance threshold. However, the Steering Committee noted
SwRI Report 03.12024.06
xvin
-------
that actual engines which would be evaluated during the HDIUT program are also likely to be
several orders of magnitude below the NTE threshold, and therefore the lack of validation of CO
was not deemed to be a significant problem.
Efforts were made to examine the reasons for the lack of validation of NOX for
calculations Methods 2 and 3. These involved examination of both the model results and the
validation data sets to determine if any errors were made or issues could be resolved. It should
be noted that these two methods predicted considerably smaller overall measurement allowances,
as compared to Method 1. However, the CE-CERT on-road validation deltas for those same
methods were larger, resulting in a lack of validation. As a result, after considerable Steering
Committee discussion, the values for Methods 2 and 3 were deemed not usable as candidates for
measurement allowance generation.
The methodology for selecting measurement allowance values from among the three
calculation methods called for a single method to be chosen for all three pollutants. With CO not
considered relevant for this purpose, only Method 1 contained validated values for the other
pollutants. Therefore, the candidate measurement allowance values for Method 1 were adopted
as the basis for calculating the final measurement allowances. These percentage values were
applied to the appropriate NTE thresholds, as given in Table 1, in order to generate the final
allowances which were given earlier in Table 2.
It should be noted, however, that as part of the final agreement reached by the Steering
Committee on the Method 1 values, EPA indicated its desire to continue to examine the possible
reasons for lack of validation, as well as the potential to modify the model and the error surfaces
in order to correct the issues. If upon further examination, this path appeared promising in terms
of being able to achieve validation of all three calculation methods, then a further cooperative
program would be initiated to revise the model result. However, any revised measurement
allowance values which were generated as a result of such a future program would not take effect
before the 2010 model year.
SwRI Report 03.12024.06 xix
-------
LIST OF ACRONYMS
Auxiliary Emission Control Device AECD
American Society for Testing and Materials ASTM
Brake-Specific BS
Brake-Specific Fuel Consumption BSFC
Bulk Current Injection BCI
California Air Resources Board CARB
Center for Environmental Research
& Technology CE-CERT
Code of Federal Regulations CFR
Constant Volume Sampling CVS
Controller Area Network CAN
Design of Experiment DOE
Diesel Particulate Filter DPF
Electromagnetic Interference EMI
Electronic Flow Meter EFM
Electrostatic Discharge ESD
Empirical Distribution Function EDF
Engine Coolant Temperature ECT
Engine Control Module ECM
Engine Manufacturer's Association EMA
Environmental Protection Agency EPA
Heavy Duty In-Use Testing HDIUT
Heavy Heavy Duty HHD
Intake Manifold Temperature IMT
Laminar Flow Element LFE
Light Heavy Duty LHD
Median Absolute Deviation MAD
Mobile Emissions Laboratory MEL
Medium Heavy Duty MHD
Memorandum of Agreement MO A
Non-Dispersive Ultraviolet NDUV
Non-Dispersive Infrared NDIR
Nonmethane Cutter NMC
Nonmethane Hydrocarbon NMHC
Not To Exceed NTE
Portable Emission Measurement System PEMS
Power Spectral Density PSD
Radio Frequency Interference RFI
Root Mean Square RMS
SEMTECH-DS SN G05-SDS04 PEMS 1
SEMTECH-DS SN G05-SDS02 PEMS 2
SEMTECH-DS SN G05-SDS03 PEMS 3
SEMTECH-DS SN G05-SDS01 PEMS 4
SwRI Report 03.12024.06
xx
-------
SEMTECH-DS SN D06-SDS01 PEMS 5
SEMTECH-DS SN D06-SDS06 PEMS 6
SEMTECH-DS SN F06-SDS02 PEMS 7
Society of Automotive Engineers SAE
Southwest Research Institute SwRI
Ultra-Low Sulfur Diesel ULSD
Wide Open Throttle WOT
SwRI Report 03.12024.06 xxi
-------
TABLE OF CONTENTS
Page
EXECUTIVE SUMMARY i
LIST OF ACRONYMS xx
LIST OF FIGURES xxvi
LIST OF TABLES xxxv
1.0 INTRODUCTION 1
1.1 Objective 1
1.2 Background 1
1.2.1 Measurement Allowance Program Test Plan 1
1.2.2 PEMS Steering Committee 2
1.2.3 Portable Emission Measurement Systems (PEMS) Description and Function 2
1.2.4 PEMS Operations at SwRI 5
1.2.5 Emission Calculation Methods for In-Use Testing 6
1.2.5.1 Calculation Method 1 - "Torque" Method 6
1.2.5.2 Calculation Method 2-"BSFC" Method 7
1.2.5.3 Calculation Method 3 - "Fuel Specific" Method 7
1.3 Monte Carlo Model Simulation 8
1.4 1065 PEMS and Laboratory Audit 9
1.5 Engine Dynamometer Laboratory Testing 9
1.6 Environmental Chamber Testing 10
1.7 Exhaust Flow Meter Testing 10
1.8 Model Validation 11
1.9 Measurement Allowance Generation 12
2.0 MONTE CARLO MODEL 14
2.1 Model Background 14
2.1.1 Reference NTE Events 14
2.1.2 Error Surfaces 19
2.1.3 Error Surface Sampling and Interpolation 25
2.1.4 Brake-Specific Emissions Calculations 27
2.1.5 Periodic Drift Check 29
2.1.6 Time Alignment for NOX and CO 30
2.1.7 Convergence and Number of Trials 31
2.1.8 Simulation Output 32
2.1.9 Step-by-Step Simulation Example 33
2.1.10 Measurement Allowance 36
2.1.11 Validation 37
3.0 1065 PEMS AND LABORATORY AUDIT 45
3.1 Audit Objective 45
3.2 Overview of 1065 Audit Activities 45
3.2.1 Laboratory Audits 46
3.2.2 PEMS Audits 47
3.3 Gas Analyzer Linearity Verifications 47
3.4 1065 Gas Analyzer Verifications 59
SwRI Report 03.12024.06 xxii
-------
3.4.1 1065.350 H2O Interference for CO2NDIR 63
3.4.2 1065.355 H2O and CO2 Interference for CO NDIR 63
3.4.3 1065.360 FID Optimization Methane Response 64
3.4.4 1065.362 Non-stoichiometric Raw FID O2 Interference 64
3.4.5 1065.365 Nonmethane Cutter Penetration Fractions 64
3.4.6 1065.370 CO2 and H2O Quench Verification for NOX CLD 65
3.4.7 1065.372 HC and H2O Interference for NOXNDUV 65
3.4.8 1065.376 Chiller NO2 Penetration 66
3.4.9 1065.378 NO2-to-NO Converter Conversion 69
3.5 1065 Exhaust Flow Meter Linearity Verification 70
3.5.1 Five-Inch Exhaust Flow Meter Linearity 71
3.5.2 Four-Inch Exhaust Flow Meter Linearity 74
3.5.3 Three-Inch Exhaust Flow Meter Linearity 76
4.0 ENGINE DYNAMOMETER LABORATORY TESTING 79
4.1 Engine Testing Objectives 79
4.2 Test Engines and Dynamometer Laboratory 79
4.3 40-Point Torque andBSFCMap Generation and Error Surface 82
4.4 Steady-State Repeat Engine Testing and Error Surfaces 85
4.4.1 Engine 1 Detroit Diesel Series 60 Steady-State 86
4.4.2 Engine 2 Caterpillar C9 Steady-State 92
4.4.3 Engine 3 International VT365 Steady-State 98
4.4.4 Steady-State Concentration Error Surface Generation 104
4.5 Transient Engine Testing and Error Surfaces 108
4.5.1 Engine 1 Detroit Diesel Series 60 Transient 110
4.5.2 Engine2 Caterpillar C9 Transient 116
4.5.3 Engine 3 International VT365 Transient 119
4.5.4 Transient Concentration Error Surface Generation 122
4.5.5 Transient Flow Meter Error Surface Generation 129
4.5.6 Transient Dynamic Error Surface Generation 132
4.6 Interacting Parameters - Warm-Up Test Error Surface 136
4.6.1 Interacting Parameters - Warm-Up Test Procedure 136
4.6.2 Interacting Parameters - Warm-Up Data Analysis 138
4.6.3 Interacting Parameters - Warm-Up Error Surface Generation 141
4.7 Torque and BSFC Interacting Parameters - Design of Experiment 144
4.7.1 Interacting Parameters - DOE Data Analysis 146
4.7.2 Engine 1 Detroit Diesel Series 60 DOE 146
4.7.3 Engines International VT3 65 DOE 148
4.7.4 Interacting Parameters - DOE Error Surface Generation 150
4.8 Torque and BSFC Independent Parameters Sensitivity Analysis 155
4.8.1 Independent Parameters Data Analysis 158
4.8.2 Independent Parameters Error Surface Generation 159
4.9 Torque and BSFC Interpolation Errors 161
4.9.1 Interpolation Error Surface Generation 164
4.10 Exhaust Flow Meter Testing 166
4.10.1 Pulsation Test 166
4.10.2 Pulsation Error Surface Generation 168
SwRI Report 03.12024.06 xxiii
-------
4.10.3 Non-Uniform Velocity Profile Swirl Test 169
4.10.4 Swirl Error Surface Generation 171
4.10.5 Tailpipe Wind Test 172
4.11 Torque and BSFC - OEM Supplied Error Surfaces 176
4.12 Time Alignment Error Surfaces 177
5.0 ENVIRONMENTAL CHAMBER TESTING 181
5.1 Environmental Testing Objective 181
5.2 Environmental Testing Procedure 181
5.3 Baseline Testing 185
5.4 Temperature Chamber Testing 193
5.4.1 Temperature Error Surface Generation 200
5.5 Pressure Chamber Testing 207
5.5.1 Pressure Error Surface Generation 217
5.6 Radiation Chamber Testing 220
5.6.1 Bulk Current Injection 220
5.6.2 Radiated Immunity 227
5.6.3 Electrostatic Discharge 233
5.6.4 Conducted Transients 238
5.6.5 Radiation Error Surface Generation 245
5.7 Vibration Table Testing 250
5.8 Ambient Hydrocarbon Testing 257
5.8.1 Ambient Hydrocarbon Error Surface Generation 263
6.0 MODEL RESULTS AND VALIDATION 266
6.1 Model Results 266
6.2 Results of Drift Correction 266
6.3 Convergence Results from MC Runs 270
6.4 Delta BS Emissions Plots for 95th Percentiles 277
6.5 Sensitivity Based on Variance 282
6.6 Sensitivity Based on Bias and Variance 295
6.7 CE-CERT Mobil Emission Laboratory Correlation 307
6.8 CE-CERT On Road Validation Testing 313
6.9 Laboratory Replay Testing 314
6.10 Validation Results 340
7.0 MEASUREMENT ALLOWANCE GENERATION AND CONCLUSIONS 352
7.1 Measurement Error Allowance Results 352
7.2 Conclusions 364
7.2.1 Engine-Installation-PEMS Variability 365
7.2.2 PEMS 1065 Audit Failures 365
7.2.3 Method 2 and Method 3 Validation for NOX 366
7.2.4 PEMS Sampling Handling System Issues and Overflow Checks 369
7.2.5 Lessons Learned for Future Programs 370
SwRI Report 03.12024.06 xxiv
-------
Appendix No. of Pages
A PEMS Operation Log 11
B Brake-Specific Emission Calculations for NOX, CO, andNMHC 5
C Crystal Ball Output File Description 15
D Monte Carlo Spreadsheet Computations 19
E 40-Point Torque and BSFC Map Data 6
F Steady-State Error Surface Data 30
G Transient Error Surface Data 19
H Interacting Parameters -DOE Error Surface Data 9
I Interpolation Torque andBSFC Error Maps 3
J Evaluation of Manufacturer Supplied Error Surfaces 4
K Environmental Chamber Testing Results and Error Surfaces 39
L Measurement Allowance Test Plan -Final Version 75
M CE-CERT On-road Validation Testing Report 93
SwRI Report 03.12024.06 xxv
-------
LIST OF FIGURES
1 Sensors Inc. SEMTECH-DS Portable Emissions Analyzer 3
2 SEMTECH EFM2 Exhaust Flow Meter and Control Unit 5
3 Method 1 BSNOX Values for Reference NTE Events 17
4 Method 1 BSCO Values for Reference NTE Events 17
5 Method 1 BSNMHC Values for Reference NTE Events 18
6 Error Surface Construction: PEMS vs. Laboratory Results 23
7 Error Surface Construction: (PEMS - Lab) vs. Laboratory Results 24
8 Error Surface Construction: Error at Variability Index vs. Laboratory Results .. 25
9 Truncated Standard Normal and Uniform Probability Density Functions 26
10 Steady-State NOX Error Surface with Example Sampling for a Reference NTE
Event 27
11 Periodic Drift Check Flowchart 30
12 Overview of Monte Carlo Simulation for BSNOX 33
13 Error Surfaces Included in Monte Carlo Simulation 35
14 Plot of Model-Generated Empirical Distribution Functions for Two Percentiles 40
15 Plot of On-Road and Model-Generated Empirical Distribution Functions 41
16 STEC Inc. Model SGD-710C Gas Divider with SEMTECH-DS PEMS 48
17 SwRI Gas Humidification and Blending Cart 60
18 NO2 Chiller Penetration Audit Results 67
19 NOX Concentration Pooled Delta Data From Engine 1 Steady-State Repeat
Testing Prior to Chiller NO2 Penetration Retrofit Installation 68
20 NOX Concentration Pooled Delta Data From Engine 1 Steady-State Repeat
Testing After Chiller NO2 Penetration Retrofit Installation 69
21 SwRI LFE Flow Stand Manometers and Reference LFEs 70
22 Sensors Inc. EFM Mounted on the SwRI LFE Flow Stand 71
23 5-Inch Sensors EFM Exhaust Flow Pooled Delta Data From Engine 1 Steady-
State Repeat Testing 73
24 5-Inch Horiba Exhaust Flow Meter Pooled Delta Data From Engine 3 Steady-
State Repeat Testing 74
25 4-Inch EFM Exhaust Flow Pooled Delta Data From Engine 2 Steady-State Repeat
Testing 76
26 3-Inch EFM Exhaust Flow Pooled Delta Data From Engine 3 Steady-State Repeat
Testing 78
27 PEMS Instrumentation Setup in Dynamometer Test Cell 79
28 Engine 1 (HHD) - 14L DDC Series 60 80
29 Engine 2 (MHD) - Caterpillar C9 80
30 Engine 3 (LHD) - International VT 365 81
31 Test Cell Exhaust System showing PEMS Flowmeters and Sampling Points 82
32 Engine 1 - Detroit Diesel Series 60 Lug Curve and 40-Point Map 83
33 Engine 2 - Caterpillar C9 Lug Curve and 40-Point Map 84
34 EngineS -International VT365 Lug Curve and 40-Point Map 84
SwRI Report 03.12024.06 xxvi
-------
35 NOX Concentration Pooled Deltas for repeat steady-state testing on engine 1 after
NO2 Penetration Upgrade 88
36 CO Concentration Pooled Deltas for Repeat Steady-State Testing on Engine 1.. 89
37 Mean Raw HC Concentrations for Engine 1 Steady-State Testing 90
3 8 PEMS NMHC Concentrations for Engine 1 Steady-State Testing 91
39 Pooled EFM Deltas for Engine 1 Steady-State Testing - 5-inch Flow Meter 92
40 Corroded RH Sensor (Left) Compared to a New RH Sensor (Right) 93
41 Disassembled RH Sensor Manifold with RH Sensor 94
42 Engine 2 Initial Steady-State Repeat NOX Results Showing Time Dependent
Concentration Shift 95
43 Raw Hydrocarbon Levels for Engine 2 Steady-State Testing 96
44 PEMS NMHC Concentrations for Engine 2 Steady-State Testing 96
45 Pooled EFM Deltas for Engine 2 Steady-State Testing - 4 Inch Flow Meter 98
46 PEMS NOX Concentration versus Mean Laboratory Reference for Engine 3
Steady-State Testing 99
47 International VT365 Point 35 NOX Concentrations During Steady-State Repeat
Testing 100
48 International VT365 Point 30 NOX Concentrations During Steady-State Repeat
Testing 101
49 Pooled EFM DelTas for Engine 3 Steady-State Testing - 3-Inch Flow Meter.. 102
50 Pooled Horiba OBS-2200 Exhaust Flow Rate Deltas For Engine 3 Steady-State
Testing 102
51 Pooled NOX Deltas for the Horiba OBS-2200 During Engine 3 Steady-State
Testing 103
52 Pooled THC Measurements for the Horiba OBS-2200 During Engine 3 Steady-
State Testing 104
53 Combined Error Surface for Steady-State NOX Concentration 105
54 Final Error Surface for Steady-State NOx Concentration 106
55 Combined Error Surface for Steady-State Exhaust Flow Rate 107
56 Final Error Surface for Steady-State Exhaust Flow Rate 108
57 Engine 1 Example Speed Traces During Transient Testing Ill
58 Engine 1 Example Torque Traces During Transient Testing 112
59 Engine 1 NOX Concentration Traces During Transient Testing Showing Outlying
Events 113
60 Engine 1 Pooled Transient Test NTE Brake-Specific NOX Results 114
61 Engine 1 Pooled Transient Test NTE Brake-Specific NOX MAD Results 114
62 Engine 1 Pooled PEMS NTE NOX Concentration Data Versus the Laboratory
Mean 115
63 Engine 1 Pooled PEMS NTE Exhaust Flow Rate Data Versus the Laboratory
Mean 116
64 Engine 2 Pooled Transient Test NTE Brake-Specific NOX Results 117
65 Engine 2 Pooled Transient Test NTE Brake-Specific NOX MAD Results 117
66 Engine 2 Pooled PEMS NTE NOX Concentration Data Versus the Laboratory
Mean 118
67 Engine 2 Pooled PEMS NTE Exhaust Flow Rate Data Versus the Laboratory
Mean 119
SwRI Report 03.12024.06 xxvii
-------
68 Engine 3 Pooled Transient Test NTE Brake-Specific NOX Results 120
69 Engine 1 Pooled Transient Test NTE Brake-Specific NOX MAD Results 120
70 Engine 3 Pooled PEMS NTE NOX Concentration Data Versus the Laboratory
Mean 121
71 Engine 3 Pooled PEMS NTE Exhaust Flow Rate Data Versus the Laboratory
Mean 122
72 Engine 1 Uncorrected Transient Flow-Weighted NOX Concentration Errors .... 123
73 Engine 1 Transient and Interpolated Steady-State MAD Values with Resulting
Scaling Factor 124
74 Error Surface for Engine 1 Transient Flow-Weighted NOX Concentration 125
75 Engine 3 Transient and Interpolated Steady-State MAD Values with Resulting
Scaling Factor 126
76 Error Surface for Engine 3 Transient Flow-Weighted NOX Concentration 127
77 Final Error Surface for Transient Flow-Weighted NOX Concentration 128
78 Error Surface for Engine 1 Transient Exhaust Flow Rate 130
79 Error Surface for Engine 2 Transient Exhaust Flow Rate 131
80 Error Surface for Engine 3 Transient Exhaust Flow Rate 131
81 Final Error Surface for Transient Exhaust Flow Rate 132
82 Final Error Surface for Dynamic ECM Fuel Rate 133
83 Final Error Surface for Dynamic ECM Speed 134
84 Final Error Surface for Dynamic Interpolated Torque 135
85 Final Error Surface for Dynamic Interpolated BSFC 136
86 Caterpillar C9 Engine Partially Enclosed in the Insulating Box Prior to the Warm-
UpTest 138
87 Caterpillar C9 Engine Fully Enclosed in the Insulating Box Prior to the Warm-Up
Test 138
88 Example of Warm-up Test Bias Correction, DDC Engine Part Load Test 140
89 Example of Determination of NTE Zone Entry for Warmup Test, DDC Engine
Part Load Point 141
90 Error Surface for Interacting parameters - Warm-Up Delta Torque 143
91 Error Surface for Interacting parameters Warm-Up Delta B SFC 143
92 Error Surface for Interacting Parameters - DOE Engine 1 Delta Torque 151
93 Error Surface for Interacting Parameters - DOE Engine 1 Delta B SFC 152
94 Error Surface for Interacting Parameters - DOE Engine 3 Delta Torque 153
95 Error Surface for Interacting Parameters - DOE Engine 3 Delta B SFC 153
96 Error Surface for Interacting Parameters - DOE Delta Torque Final 154
97 Error Surface for Interacting Parameters - DOE Delta B SFC Final 155
98 Intake Air Low Humidity Control SYSTEM 158
99 Error Surface for Independent Parameters Delta Torque 160
100 Error Surface for Independent Parameters Delta BSFC 161
101 Detroit Diesel Series 60 Down Selected 20-Point Map 162
102 Caterpillar C9 Down Selected 20-Point Map 162
103 International VT365 Down Selected 20-Point Map 163
104 Interpolated Torque Error (% Peak Torque) by Speed (rpm) and Fuel Rate (g/s)
for Engine #1 164
105 Torque Interpolation Error Surface 165
SwRI Report 03.12024.06 xxviii
-------
106 BSFC Interpolation Error Surface 166
107 Pulsation Test SEMTECH-DS Exhaust Flow Rate Deltas - Raw Data 167
108 Pulsation Test SEMTECH-DS Exhaust Flow Rate Delta - Corrected for Steady-
State Bias 168
109 Error Surface for Pulsation Exhaust Flow Rate 169
110 Swirl Test SEMTECH-DS Exhaust Flow Rate Deltas - Raw Data 170
111 Swirl Test SEMTECH-DS Exhaust Flow Rate Deltas - Corrected for Steady-State
Bias 171
112 Error Surface for Swirl Exhaust Flow Rate 172
113 High Velocity Blower System 173
114 EFM Wind Test Flow Schematic 173
115 Swirl Test SEMTECH-DS Exhaust Flow Rate Deltas - Raw Data 174
116 Wind Test SEMTECH-DS Exhaust Flow Rate Deltas - Corrected for Steady-
State Bias 175
117 Wind Test Mean Delta Values With 95 % Confidence Level Bars 176
118 Pooled and Weighted Brake-Specific Time Alignment Error Data for NOX and
CO 180
119 Systematic High CC>2 Bias During Zero Air Reference Gas Measurement 184
120 CC>2 Concentration Decay During Zero Air Measurement Following the Quad
Blend Span Reference Gas 184
121 PEMS NC>2 Delta Data During Initial Environmental Baseline Testing 187
122 PEMS 2 NO and NO2 Response During Hour 1 of Initial Environmental Baseline
Testing 188
123 PEMS 2 NO2 and NO Response With Thermoelectric Chiller Bypassed and
Reconnected 189
124 PEMS 2 Environmental Baseline Zero Delta Measurements 190
125 PEMS 2 Environmental Baseline Audit Delta Measurements 191
126 PEMS 2 Environmental Baseline Span Delta Measurements 192
127 5-Inch EFM Environmental Baseline Zero Delta Measurements 193
128 Temperature Histograms for NEI Model and Test Profile 194
129 Temperature Test Profile and Moving Average 195
130 Thermotron SM-32 Temperature Control Chamber with Supplemental Liquid
Nitrogen Cylinder 196
131 PEMS 2 Environmental Temperature Zero Delta Measurements 197
132 PEMS 2 Environmental Temperature Audit Delta Measurements 198
133 PEMS 2 Environmental Temperature Span Delta Measurements 199
134 5-Inch EFM Environmental Temperature Zero Delta Measurements 200
135 Error Surface for Environmental Temperature NOX Concentration Zero Delta
Measurements 202
136 Error Surface for Environmental Temperature NOX Concentration Audit Delta
Measurements 202
137 Error Surface for Environmental Temperature NOX Concentration Span Delta
Measurements 203
138 Final Error Surface for Environmental Temperature NOX Concentration 204
139 Corrected CO Deltas Measured During Environmental Temperature Testing... 205
140 Final Error Surface for Environmental Temperature CO Concentration 206
SwRI Report 03.12024.06 xxix
-------
141 Error Surface for Environmental Temperature Exhaust Flow Rate Delta
Measurements 207
142 Pressure Histograms for NEI Model and Test Profile 208
143 Pressure Test Profile and Moving Average 210
144 Altitude Chamber Top -Removed From Base 211
145 PEMS Equipment on Altitude Chamber Base 212
146 Altitude Chamber and Pressure Control Equipment During Testing 213
147 PEMS 2 Environmental Pressure Zero Delta Measurements 214
148 PEMS 2 Environmental Pressure Audit Delta Measurements 215
149 PEMS 2 Environmental Pressure Span Delta Measurements 216
150 5-Inch EFM Environmental Pressure Zero Delta Measurements 217
151 NMHC Corrected Delta Data for the Environmental Pressure Testing 218
152 Final Error Surface for Environmental Pressure NMHC Concentration 219
153 Error Surface for Environmental Pressure Exhaust Flow Rate 220
154 Bulk Current Injection Probe 221
155 Calibration Device for the Bulk Current Injection Probe 222
156 PEMS 7 in the Radiation Chamber Undergoing Bulk Current Injection Testing223
157 PEMS 7 Environmental Radiation BCI Zero Delta Measurements 224
158 PEMS 7 Environmental Radiation BCI Audit Delta Measurements 225
159 PEMS 7 Environmental Radiation BCI Span Delta Measurements 226
160 5-Inch EFM Environmental Radiation BCI Zero Delta Measurements 227
161 PEMS 7 and Radiation Antenna in the Absorber-Lined Radiation Chamber
During Radiated Immunity Testing 228
162 Signal Generators, Amplifiers, Oscilloscopes and Other Electronics Used to
Perform Radiation Testing 229
163 PEMS 7 Environmental Radiation Radiated Immunity Zero Delta Measurements
231
164 PEMS 7 Environmental Radiation Radiated Immunity Audit Delta Measurements
232
165 PEMS 7 Environmental Radiation Radiated Immunity Span Delta Measurements
232
166 5-Inch EFM Environmental Radiation Radiated Immunity Zero Delta
Measurements 233
167 Electrostatic Discharge Simulator Used During Electrostatic Discharge Testing
234
168 Electrostatic Voltmeter Used to Calibrate the Electrostatic Discharge Simulator
235
169 PEMS 7 Environmental Radiation Electrostatic Discharge Zero Delta
Measurements 236
170 PEMS 7 Environmental Radiation Electrostatic Discharge Audit Delta
Measurements 237
171 PEMS 7 Environmental Radiation Electrostatic Discharge Span Delta
Measurements 237
172 5-Inch EFM Environmental Radiation Electrostatic Discharge Zero Delta
Measurements 238
173 Schaffner NSG 5200 Automotive Electronics Test System 239
SwRI Report 03.12024.06 xxx
-------
174 Example Voltage Trace During a Voltage Spike with Slow Recovery 240
175 Example Voltage Trace During a Voltage Spike with Quick Recovery 241
176 Example Voltage Trace During a Voltage Burst 242
177 PEMS 7 Environmental Radiation Conducted Transient Zero Delta Measurements
243
178 PEMS 7 Environmental Radiation Conducted Transient Audit Delta
Measurements 244
179 PEMS 7 Environmental Radiation Conducted Transient Span Delta
Measurements 244
180 5-Inch EFM Environmental Radiation Conducted Transient Zero Delta
Measurements 245
181 Combined Radiation Chamber Zero Delta Test Results 246
182 Combined Radiation Chamber Audit Delta Test Results 247
183 Combined Radiation Chamber Span Delta Test Results 247
184 Pooled Exhaust Flow Rate Zero Deltas Measured During Environmental
Radiation Testing 249
185 Error Surface for Environmental Radiation Exhaust Flow Rate Delta
Measurements 250
186 PEMS 3 in an Environmental Enclosure During Vibration Testing Using an
Unholtz-Dickie Shaker System 251
187 Army M915 A2 Semi-Tractor Used to Generate Vibration Spectra for Vibration
Testing 252
188 Power Spectral Densities Evaluated For Vibration Testing 253
189 PEMS 3 Environmental Vibration Zero Delta Measurements 254
190 PEMS 3 Environmental Vibration THC Zero Delta Measurements 255
191 PEMS 3 Environmental Vibration Audit Delta Measurements 256
192 PEMS 3 Environmental Vibration Span Delta Measurements 256
193 PEMS 3 Environmental Vibration NOX Span Delta Measurements 257
194 THC Measurements for Test 1 of the Ambient Hydrocarbon Test Sequence.... 260
195 THC Measurements for Test 2 of the Ambient Hydrocarbon Test Sequence.... 262
196 THC Measurements for Test 6 of the Ambient Hydrocarbon Test Sequence.... 263
197 PEMS 3 NMHC Response Versus FID Air Methane Contamination Measured
During Ambient Hydrocarbon Testing 264
198 Error Surface for Ambient Hydrocarbon Testing 265
199 Percent of Trials Deleted for Each Reference NTE Event Due to Periodic Drift
Check for BSNOx Method 1 267
200 Percent of Trials Deleted for Each Reference NTE Event Due to Periodic Drift
Check for BSNOx Method 2 267
201 Percent of Trials Deleted for Each Reference NTE Event Due to Periodic Drift
Check for BSNOX Method 3 268
202 Percent of Trials Deleted for Each Reference NTE Event Due to Periodic Drift
Check for BSNMHC Method 1 269
203 Percent of Trials Deleted for Each Reference NTE Event Due to Periodic Drift
Check for BSNMHC Method 2 269
204 Percent of Trials Deleted for Each Reference NTE Event Due to Periodic Drift
Check for BSNMHC Method 3 270
SwRI Report 03.12024.06 xxxi
-------
205 Convergence Interval Width as a Percent of Threshold for BSNOX Method 1 ..271
206 Convergence Interval Width as a Percent of Threshold for BSNOX Method 2 .. 272
207 Convergence Interval Width as a Percent of Threshold for BSNOX Method 3 .. 272
208 Convergence Interval Width as a Percent of Threshold for BSNMHC Method 1
273
209 Convergence Interval Width as a Percent of threshold for BSNMHC Method 1274
210 Convergence Interval Width as a Percent of Threshold for BSNMHC Method 3
274
211 Convergence Interval Width as a Percent of Threshold for BSCO Method 1.... 275
212 Convergence Interval Width as a Percent of Threshold for BSCO Method 2.... 276
213 Convergence Interval Width as a Percent of Threshold for BSCO Method 3.... 276
214 Box Plot of 95th Percentile Delta BSNOX for Three Methods from 195 Reference
NTE Events 278
215 Box Plot for 95th Percentile Delta BSCO for Three Methods from 195 Reference
NTE Events 279
216 Box Plot for 95th Percentile Delta BSCO for Three Methods for 195 Reference
NTE Events 280
217 Comparison of 95th Percentile Delta BSNOX for Methods 1, 2, and 3 for 195
Reference NTE Events 281
218 Comparison of 95th Percentile Delta BSNMHC for Methods 1, 2, and 3 from 195
Reference NTE Events 281
219 Comparison of 95th Percentile Delta BSCO for Methods 1, 2, and 3 from 195
Reference NTE Events 282
220 Box Plot of Error Surface Sensitivity Based on Variance for BSNOX Method 1287
221 Box Plot of Error Surface Sensitivity Based on Variance for BSNOX Method 2288
222 Box Plot of Error Surface Sensitivity Based on Variance for BSNOX Method 3289
223 Box Plot of Error Surface Sensitivity Based on Variance for BSNMHC Method 1
290
224 Box Plot of Error Surface Sensitivity Based on Variance for BSNMHC Method 2
291
225 Box Plot of Error Surface Sensitivity Based on Variance for BSNMHC Method 3
292
226 Box Plot of Error Surface Sensitivity Based on Variance for BSCO Method 1. 293
227 Box Plot of Error Surface Sensitivity Based on Variance for BSCO Method 2. 294
228 Box Plot of Error Surface Sensitivity Based on Variance for BSCO Method 3.295
229 Distribution of Ideal BSNOX for 13 Reference NTE Events 297
230 Box Plot of error Surface Sensitivity Based on Bias and Variance for BSNOX
Method 1 301
231 Box Plot of Error Surface Sensitivity Based on Bias and Variance for BSNOX
Method 2 302
232 Box Plot of Error Surface Sensitivity Based on Bias and Variance for BSNOX
Methods 303
233 Comparison of Typical CONTINUOUS NOx Mass Rate Data over RMC Cycle -
CE-CERT versus SwRI 312
234 Engine Speed CAN Data Comparison for SwRI Laboratory Testing and CE-
CERT On-Road Validation Testing (Route 2) 315
SwRI Report 03.12024.06 xxxii
-------
235 Fuel Rate CAN Data Comparison for SwRI Laboratory Testing and CE-CERT
On-Road Validation Testing (Route 2) 316
236 Boost Pressure CAN Data Comparison for SwRI Laboratory Testing and CE-
CERT On-Road Validation Testing (Route 2) 317
237 Injection Timing CAN Data Comparison for SwRI Laboratory Testing and CE-
CERT On-Road Validation Testing (Route 2) 318
238 Intake Manifold Temperature CAN Data Comparison for SwRI Laboratory
Testing and CE-CERT On-Road Validation Testing (Route 2) 319
239 PEMS 5 Wet NOX Comparison for SwRI Laboratory Testing and CE-CERT On-
Road Validation Testing (Route 2) 320
240 PEMS 5 Exhaust Flow Rate Comparison for SwRI Laboratory Testing and CE-
CERT On-Road Validation Testing (Route 2) 321
241 PEMS 5 NOX Mass Flow Rate Comparison for SwRI Laboratory Testing and CE-
CERT On-Road Validation Testing (Route 2) 321
242 Brake-Specific NOX Emission Deltas for PEMS 5 Method 1 Calculation Versus
the Laboratory Reference (Laboratory Torque and BSFC) 323
243 Brake-Specific NOX Emission Deltas for PEMS 5 Method 1 Calculation Versus
the Laboratory Method 1 Calculation (ECM Torque and BSFC) 324
244 Incremental Brake-Specific NOX Emission Deltas Due to ECM Torque and BSFC
Errors for Calculation Method 1 325
245 Incremental Brake-Specific NOX Deltas Compared to the Incremental Torque and
BSFC Errors Predicted by the Model for Calculation Method 1 326
246 Brake-Specific NOX Emission Deltas for PEMS 5 Method 2 Calculation Versus
the Laboratory Reference (Laboratory Torque and BSFC) 327
247 Brake-Specific NOX Emission Deltas for PEMS 5 Method 2 Calculation Versus
the Laboratory Method 2 Calculation (ECM Torque and BSFC) 328
248 Incremental Brake-Specific NOX Emission Deltas Due to ECM Torque and BSFC
Errors for Calculation Method 2 329
249 Incremental Brake-Specific NOX Deltas Compared to the Incremental Torque and
BSFC Errors Predicted by the Model for Calculation Method 2 330
250 Brake-Specific NOX Emission Deltas for PEMS 5 Method 3 Calculation Versus
the Laboratory Reference (Laboratory Torque and BSFC) 331
251 Brake-Specific NOX Emission Deltas for PEMS 5 Method 3 Calculation Versus
the Laboratory Method 3 Calculation (ECM Torque and BSFC) 332
252 Incremental Brake-Specific NOX Emission Deltas Due to ECM Torque and BSFC
Errors for Calculation Method 3 333
253 Incremental Brake-Specific NOX Deltas Compared to the Incremental Torque and
BSFC Errors Predicted by the Model for Calculation Method 3 334
254 ECM Broadcast Torque Errors Measured During Replay Validation Testing with
a Caterpillar Cl5 Engine 335
255 ECM Broadcast Fuel Flow Rate Errors Measured During Replay Validation
Testing with a Caterpillar C15 Engine 336
256 ECM-Based BSFC Errors Measured During Replay Validation Testing with a
Caterpillar C15 Engine 337
257 ECM Broadcast Torque Errors Measured During 40-Point Map Generation with a
Detroit Diesel Series 60 Engine 338
SwRI Report 03.12024.06 xxxiii
-------
258 ECM Broadcast Torque Errors Measured During 40-Point Map Generation with a
Caterpillar C9 Engine 339
259 ECM Broadcast Torque Errors Measured During 40-Point Map Generation with
an International VT365 Engine 340
260 Validation On-Road and Model Generated Empirical Distribution Functions for
BSNOX Method 1 342
261 Validation On-Road and Model Generated Empirical Distribution Functions for
BSNOX Method 2 343
262 Validation On-Road and Model Generated Empirical Distribution Functions for
BSNOX Method 3 344
263 Validation On-Road and Model Generated Empirical Distribution Functions for
BSNMHC Method 1 345
264 Validation On-Road and Model Generated Empirical Distribution Functions for
BSNMHC Method 2 346
265 Validation On-Road and Model Generated Empirical Distribution Functions for
BSNMHC Method 3 347
266 Validation On-Road and Model Generated Empirical Distribution Functions for
BSCO Method 1 348
267 Validation On-Road and Model Generated Empirical Distribution Functions for
BSCO Method 2 349
268 Validation On-Road and Model Generated Empirical Distribution Functions for
BSCO Method 3 350
269 Regression Plot of 95th Percentile Delta BSNOX versus Ideal BSNOX for Method 1
353
270 Regression Plot of 95th Percentile Delta BSNOX versus Ideal BSNOX for Method 2
354
271 Regression Plot of 95th Percentile Delta BSNOX versus Ideal BSNOX for Method 3
355
272 Regression Plot of 95th Percentile Delta BSNMHC versus Ideal BSNMHC for
Method 1 356
273 Regression Plot of 95th Percentile Delta BSNMHC versus Ideal BSNMHC for
Method 2 358
274 Regression Plot of 95th Percentile Delta BSNMHC versus Ideal BSNMHC for
Methods 359
275 Regression Plot for 95th Percentile Delta BSCO versus Ideal BSCO for Method 1
360
276 Regression Plot of 95th Percentile Delta BSCO versus Ideal BSCO for Method 2
361
277 Regression Plot of 95th Percentile Delta BSCO versus Ideal BSCO for Method 3
363
278 Variation in Zero Calibration of PEMS During On-Road Validation Testing... 368
279 PEMS Zero Calibration Variations during Laboratory Testing 369
SwRI Report 03.12024.06 xxxiv
-------
LIST OF TABLES
Table Page
1 NTE Threshold Values Used for Measurement Allowance Program ii
2 Final Measurement Allowances ii
3 Laboratory Reference Methods v
4 Magnitude of Error Terms for Steady-State Error Surfaces ix
5 Magnitude of error terms for Transient Error Surfaces - gaseous analyzers and
exhaust flow xi
6 Magnitude of Transient Error TERMS for ECM Variables xi
7 Magnitude of Torque and BSFC Error surfaces xii
8 Magnitude of Error Terms for Exhaust Flow Error Surfaces xiv
9 Magnitude of OEM Error Surfaces xiv
10 Magnitude of Time Alignment Error Surfaces xv
11 Magnitude of Error Terms for Environmental Error Surfaces xvi
12 Model Results and Validation xviii
13 NTE Thresholds for Measurement Allowance Program 12
14 Example of MEASUREMENT Allowance Determination from Test Plan 13
15 Reference NTE Events and Method 1 BS Emissions 15
16 Descriptive Statistics for BS Emissions for Reference NTE Events 18
17 Input Parameters for Reference NTE Events 19
18 Error Surfaces for Monte Carlo Simulation 21
19 Error Surfaces Used for Computing Brake-Specific Emissions by Three
Calculation Methods 29
20 Example of Selection of the Measurement Error 37
21 1065 Audits and Performance Checks Required for the Measurement Allowance
Program 46
22 Span Concentrations used for the SEMTECH-DS and Laboratory Analyzers 48
23 Dilute MEXA 7200D and Horiba CH4 Bench 1065 Linearity Verification
Summary 49
24 Raw MEXA 7200D and Horiba CH4 Bench 1065 Linearity Verification Summary
49
25 PEMS 1 1065 Linearity Verification Summary 51
26 PEMS 2 1065 Linearity Verification Summary 52
27 PEMS 3 1065 Linearity Verification Summary 53
28 PEMS 4 1065 Linearity Verification Summary 54
29 PEMS 4 1065 Linearity Verification Summary Continued 55
30 PEMS 5 1065 Linearity Verification Summary 56
31 PEMS 6 1065 Linearity Verification Summary 57
32 PEMS 7 1065 Linearity Verification Summary 58
33 Horiba OBS-2200 1065 Linearity Verification Summary 58
34 Dilute MEXA 7200D and Horiba NMHC Bench 1065 Analyzer Verification
Summary 60
SwRI Report 03.12024.06 xxxv
-------
3 5 Raw MEXA 7200D and Horiba NMHC Bench 1065 Analyzer Verification
Summary 61
36 PEMS 1 1065 Analyzer Verification Summary 61
37 PEMS 2 1065 Analyzer Verification Summary 61
38 PEMS 3 1065 Analyzer Verification Summary 61
39 PEMS 4 1065 Analyzer Verification Summary 62
40 PEMS 5 1065 Analyzer Verification Summary 62
41 PEMS 6 1065 Analyzer Verification Summary 62
42 PEMS 7 1065 Analyzer Verification Summary 62
43 Horiba OB S-2200 1065 Analyzer Verification Summary 63
44 5-Inch Sensors Inc. EFM Linearity Results 72
45 5-Inch Horiba Exhaust Flow Meter Linearity Results 72
46 4-Inch Sensors Inc. EFM Linearity Results 75
47 3-Inch Sensors Inc. EFM Linearity Results 77
48 NTE Event Descriptions from the Test Plan 109
49 NTE Transition Descriptions from the Test Plan 110
50 Final Gaseous Transient Error Surface Deltas 129
51 nlo and nhi Speed Definitions for Engines 1, 2, and 3 133
52 Warm-Up Test Torque Errors Summary 142
53 Warm-Up Test BSFC Errors Summary 142
54 Interacting Parameters - DOE Adjustment guidance 144
55 Interacting Parameters - DOE Test Matrix 145
56 Test Plan DOE Engine Operating Conditions 145
57 Example of DOE Baseline Correction for Engine 1 146
58 Interacting Parameters - DOE Speed and Torque Steady-State Mode Definition
for Engine 1 147
59 Interacting Parameters - DOE Test Matrix for Engine 1 147
60 Interacting Parameters - DOE Engine 1 Bias Corrected Torque Deltas 148
61 Interacting Parameters - DOE Engine 1 Bias Corrected BSFC Deltas 148
62 Interacting Parameters - DOE Speed and Torque Steady-State Mode Definition
for Engine 3 149
63 Interacting Parameters - DOE Test Matrix for Engine 3 149
64 Interacting Parameters - DOE Engine 3 Bias Corrected Torque Deltas 150
65 Interacting Parameters -DOE Engine 3 Bias Corrected BSFC Deltas 150
66 Example DOE Steady-State Variance Correction using the Mean SS MAD 151
67 Independent Parameters adjustment guidance from the Test Plan 155
68 Independent Parameters Test Modes from the Test Plan 156
69 Independent Parameters Speed and Torque Steady-State Mode Definitions 156
70 Selected Fuel Properties for INDEPENDENT Parameters Testing 157
71 Independent Parameters Bias Corrected Torque Deltas 159
72 Independent Parameters Bias Corrected BSFC Deltas 159
73 Torque Interpolation Error Surface Values 165
74 BSFC interpolation error Surface Values 165
75 OEM Error Surface Deltas for Torque and BSFC 177
76 EFM and Vehicle Interface Adjustment and Weighting Factors Used for Time
Alignment Error Generation 178
SwRI Report 03.12024.06 xxxvi
-------
77 Brake-Specific Time Alignment Error Data forNOx 179
78 Brake-Specific Time Alignment Error Data for CO 179
79 Reference Gases and Typical Concentrations Used During Environmental
Chamber Testing 182
80 Zero, Audit, and Span Delta Recording Strategy Used During Environmental
Chamber Testing 183
81 Temperature Test Profile Definition 195
82 Median Environmental Baseline NOX Concentrations and Environmental
Temperature Scaling Factors 201
83 Pressure Test Profile Definition 209
84 Hexane and Methane Contamination Combinations Used During Ambient
Hydrocarbon Testing 258
85 Ambient Hydrocarbon Test Sequence 259
86 THC Measurements for Test 1 of the Ambient Hydrocarbon Test Sequence.... 261
87 THC Measurements for Test 2 of the Ambient Hydrocarbon Test Sequence.... 262
88 Summary of the Trials Deleted Due to Periodic Drift Check for BSNOX 268
89 Summary of Trials Deleted for Each Reference NTE Event Due to Periodic Drift
Check for BSNMHC Method 3 270
90 Summary of BSNOX Convergence Interval Width as a Function of Threshold for
195 Reference NTE Events 273
91 Summary of BSNMHC Convergence Interval Width as a Function of Threshold
for 195 Reference NTE Events 275
92 Summary of BSCO Convergence Interval Width as a Function of Threshold for
195 Reference NTE Events 277
93 Error Surface Sensitivity to Variance for 195 Reference NTE Events for BSNOX
Method 1 283
94 Error Surface Sensitivity to Variance for 195 Reference NTE Events for BSNOX
Method 2 283
95 Error Surface Sensitivity to Variance for 195 Reference NTE Events for BSNOX
Methods 284
96 Error Surface Sensitivity to Variance for 195 Reference NTE Events for
BSNMHC Method 1 284
97 Error Surface Sensitivity to Variance for 195 Reference NTE Events for
BSNMHC Method 2 284
98 Error Surface Sensitivity to Variance for 195 Reference NTE Events for
BSNMHC Method 3 285
99 Error Surface Sensitivity to Variance for 195 Reference NTE Events for BSCO
Method 1 285
100 Error Surface Sensitivity to Variance for 195 Reference NTE Events for BSCO
Method 2 285
101 Error Surface Sensitivity to Variance for 195 Reference NTE Events for BSCO
Methods 286
102 Ideal BSNOX Values for 13 Reference NTE Events 296
103 Error Surface Sensitivity to Bias and Variance for 13 Reference NTE Events for
BSNOX Method 1 298
SwRI Report 03.12024.06 xxxvii
-------
104 Error Surface Sensitivity to Bias and Variance for 13 Reference NTE Events for
BSNOxMethod2 298
105 Error Surface Sensitivity to Bias and Variance for 13 Reference NTE Events for
BSNOX Method 3 298
106 Error Surface Sensitivity to Bias and Variance for 13 Reference NTE Events for
BSNMHC Method 1 299
107 Error Surface Sensitivity to Bias and Variance for 13 Reference NTE Events for
BSNMHC Method 2 299
108 Error Surface Sensitivity to Bias and Variance for 13 Reference NTE Events for
BSNMHC Method 3 299
109 Error Surface Sensitivity to Bias and Variance for 13 Reference NTE Events for
BSCO Method 1 300
110 Error Surface Sensitivity to Bias and Variance for 13 Reference NTE Events for
BSCO Method 2 300
111 Error Surface Sensitivity to Bias and Variance for 13 Reference NTE Events for
BSCO Method 3 300
112 Summary of Error Surface Sensitive to Bias and Variance for BSNOX Method 1
304
113 Summary of Error Surface Sensitive to Bias and Variance for BSNOX Method 2
304
114 Summary of Error Surface Sensitive to Bias and Variance for BSNOX Method 3
305
115 Summary of Error Surface Sensitive to Bias and Variance for BSNMHC Method
1 305
116 Summary of Error Surface Sensitive to Bias and Variance for BSNMHC Method
2 305
117 Summary of Error Surface Sensitive to Bias and Variance for BSNMHC Method
3 306
118 Summary of Error Surface Sensitive to Bias and Variance for BSCO Method 1
306
119 Summary of Error Surface Sensitive to Bias and Variance for BSCO Method 2
306
120 Summary of Error Surface Sensitive to Bias and Variance for BSCO Method 3
307
121 Correlation Test Matrix 308
122 Correlation Testing Results for NTE Transient Cycle 310
123 Correlation Test Results for RMC 13-Mode SET Cycle 311
124 Summary of Model Validation Results 351
125 Measurement Error at Threshold for BSNOX Using Regression and Median
Methods for Method 1 353
126 Measurement Error at Threshold for BSNOX Using Regression and Median
Methods for Method 2 354
127 Measurement Error at Threshold for BSNOX Using Regression and Median
Methods for Method 3 355
128 Measurement Error at Threshold for BSNMHC Using Regression and Median
Methods for Method 1 357
SwRI Report 03.12024.06 xxxviii
-------
129 Measurement Error at Threshold for BSNMHC Using Regression and Median
Methods for Method 2 358
130 Measurement Error at Threshold for BSNMHC Using Regression and Median
Methods for Method 3 359
131 Measurement Error at Threshold for B SCO Using Regression and Median
Methods for Method 1 360
132 Measurement Error at Threshold for B SCO Using Regression and Median
Methods for Method 2 362
133 Measurement Error at Threshold for B SCO Using Regression and Median
Methods for Method 3 363
134 Measurement Error in Percent of NTE Threshold by Emissions and Calculation
Method 364
135 Measurement Allowance at NTE Threshold by Emissions for Method 1 364
136 Lessons Learned During Gaseous Measurement Allowance Program 370
SwRI Report 03.12024.06 xxxix
-------
1.0 INTRODUCTION
The intent of this section of this report is to provide an overview of the program objectives,
background material on the test plan, test methods, and equipment, and to briefly discuss the
rationale behind each of the major components of the measurement allowance program which
will be discussed in detail in later sections of the report.
This revised version of the final report contains a number of changes made following
EPA's peer review of the original final report. None of the results or conclusions of the original
report were affected as part of the revision. The changes made to the report primarily involved
additional clarifying language in areas were the peer review process indicated that the original
report was unclear or vague.
1.1 Objective
The objective of this program was to determine a set of brake-specific measurement
allowances for the gaseous pollutants regulated under the Heavy-Duty In-Use Testing (HDIUT)
program. These measurement allowances are intended to represent the incremental error
between measuring emissions under controlled conditions in a laboratory with lab-grade
equipment, and measuring emissions in the field using Portable Emissions Measurement Systems
(PEMS). Measurement allowance values were generated for non-methane hydrocarbons
(NMHC), carbon monoxide (CO), and oxide of nitrogen (NOX).
The measurement allowances are fixed brake-specific values, which are intended to be
added to a given NTE threshold in order to provide an additional compliance margin which
accounts for the relative error between laboratory and field measurements.
The completion of this program was part of the resolution of a 2001 legal suit filed against
EPA by the Engine Manufacturer's Association (EMA) and several individual engine
manufacturers regarding certain portions of the Not-to-Exceed (NTE) standards. This dispute
was settled on June 3, 2003. As such, this program represents a cooperative effort between EPA,
EMA, and the California Air Resources Board (CARB). The program was jointly funded by all
three organizations, and was conducted under the direction of a Steering Committee composed of
representatives of all three organizations, as well as representatives of a number of individual
engine manufacturers which are EMA members.
1.2 Background
1.2.1 Measurement Allowance Program Test Plan
The measurement allowance program was conducted according to procedures and
guidelines which were laid out in a detailed test plan document titled Test Plan to Determine
PEMS Measurement Allowances for Gaseous Emissions Regulated under the Manufacturer-Run
Heavy-Duty Diesel Engine In-Use Testing Program. The final version of this document, which
forms the basis of the program, is dated October 24, 2005. This document will be referred to as
SwRI Report 03.12024.06 Page 1 of 371
-------
the Test Plan throughout the remainder of the report. This final version was modified from the
initial version, dated May 20, 2005, which was distributed publicly by EPA and is available via
the Internet at http://www.epa.gov/otaq/regs/hd-hwy/inuse/testplan.pdf The two documents are
identical in terms of overall methodology and scope, and differ primarily in certain details
pertaining to either test execution or data analysis.
The Test Plan was developed as a collaborative effort by the Steering Committee and all
modifications made to the Test Plan were discussed and approved by the Steering Committee
prior to being performed. Throughout the program, every effort was made to adhere to the
procedures given in the Test Plan. However, on numerous occasions, these procedures had to be
modified in response to unexpected occurrences during testing, or as a result of test data
generated during the program. All such modifications that were not captured in the final Test
Plan document are included in this report. When such changes are noted in the report, the
original Test Plan procedure is given, along with the rationale for any changes, and the date at
which these modifications were approved by the Steering Committee.
1.2.2 PEMS Steering Committee
The PEMS Steering Committee was composed primarily of representatives from EPA,
EMA, CARB, and the following engine manufacturers: Cummins Engine Company, Detroit
Diesel Corporation, Volvo Powertrain, Caterpillar Inc., International Engine Company, and
Isuzu. Representatives of other engine manufacturers were also present for some of the
Committee meetings. PEMS Steering Committee meetings were convened on an as needed basis
by agreement of the Committee members. Generally, these meetings were held on a monthly
basis, although bi-monthly meetings were held late in the program as key decisions were
required. During the majority of the program, weekly teleconferences were held to update the
group on progress and to provide feedback to SwRI. It should be noted that this required a
considerable time and travel commitment on the part of Steering Committee members. SwRI
would like to acknowledge this contribution, and thank the Committee members for their efforts.
In general, efforts were made to achieve unanimity among all Steering Committee
members before deciding on a course of action. On the occasions that a unanimous opinion
could not be formed, a majority vote of Committee members was required to decide a given
issue. In such cases, which were generally rare, dissenting votes were noted for the record as
desired by those in dissent.
1.2.3 Portable Emission Measurement Systems (PEMS) Description and Function
The focus of this program was the evaluation of Portable Emission Measurement Systems,
which are referred to by the acronym PEMS throughout this report. A key provision of the Test
Plan was that the PEMS to be evaluated had to represent commercially available hardware. The
intent of this provision is captured in the following language taken from the Test Plan:
"The PEMS used in this test plan must be standard in-production makes and models that are for sale as
commercially available PEMS. In addition, PEMS and any support equipment must pass a "red-face" test with
respect to being consistent with acceptable practices for in-use testing. For example, use of large gas bottles that can
SwRI Report 03.12024.06 Page 2 of 371
-------
not be utilized by the EPA/ARB/EMA HDIU enforceable program is unacceptable. Furthermore, the equipment
must meet all safety and transportation regulations for use on-board heavy-duty vehicles."
The original intent of the program was to evaluate PEMS from two suppliers, Sensors
Incorporated and Horiba Instruments. However, at the time of the start of this program, the
Horiba PEMS was still in the final stages of development, therefore Horiba was not able to
supply a commercially available unit. As a result, the program was conducted primarily with the
Sensors Inc. SEMTECH-DS hardware. Horiba was able to supply examples of its OBS-2200
PEMS hardware in the later stages of the program, but this was evaluated only for purposes of
supplemental information as time permitted. The measurement allowance values were generated
using data from only the Sensors SEMTECH-DS PEMS hardware. The Test Plan called for
three different PEMS units from each Manufacturer to be examined. Ultimately, due to various
scheduling and hardware issues, a total of seven SEMTECH-DS units were evaluated during the
program. However, all seven PEMS were not evaluated for every error source, and no more than
three PEMS were used during any given error test.
The SEMTECH-DS PEMS included several major components. The first of these is the
SEMTECH-DS portable gaseous emission analyzer unit. This unit housed the gaseous emission
analyzers, the sampling system, and the sampling conditioning system. The unit also contained
electronics for analyzer functions, interaction with the other system components, as well as for
communication with the user. User interface was accomplished using a remote interface
program running on a laptop computer, which was connected to the SEMTECH-DS via an
Ethernet cable or using wireless communication. The front of a SEMTECH-DS PEMS is
pictured in Figure 1, showing the connection points for various other components.
•
SEmTECH-
sensors, inc.
.FSSCOMMUNPCArtDWS
in .-WP!U
7-3.BBA
ZERO
H. . 4 1^
HEATED FILTER HEATEP l.irst
• •
moBiLE Emission ARALVZER
READY
RD FLAME
ALBtT
ZERO
OtlSTAlUG
DATA LOG
••
AUX-2 ANALOG TO
• •
FIGURE 1. SENSORS INC. SEMTECH-DS PORTABLE EMISSIONS ANALYZER
SwRI Report 03.12024.06
Page3 of 371
-------
The SEMTECH DS uses a variety of different analyzers to measure various gaseous
emissions. Total hydrocarbons (HC) are measured using a heated flame ionization detector
(HFID). Carbon monoxide (CO) and carbon dioxide (CC^) are measured using a non-dispersive
infrared (NDIR) instrument. Oxides of nitrogen (NOX) are measured using a non-dispersive
ultraviolet (NDUV) instrument, in which NO and NO2 are measured separately and combined
mathematically to produce a final NOX value.
The SEMTECH DS units were initially supplied along with an add-on FID analyzer for
methane measurement to allow for the determination of non-methane hydrocarbons (NMHC).
However, only two of these units were supplied, which was not enough for all the PEMS used in
the program. In addition, upon evaluation, the Steering Committee determined that these
methane analyzers did not pass the red-face requirement outlined in the Test Plan and were not
suitable for field use. Therefore, the methane analyzers were not used in the program, and
NMHC for the PEMS was determined as 0.98 times THC, as allowed under CFR Title 40 Part
1065.
A second key component of the SEMTECH-DS PEMS is the SEMTECH EFM2 exhaust
flow meter. This unit is a pitot-tube based exhaust flow measurement meter which is design to
be attached to the end of a vehicle tailpipe for direct measurement of exhaust flow over a wide
dynamic range. The control box contains a set of pressure transducers for differential and static
pressure measurement. The EFM control unit is connected to the main SEMTECH-DS unit via a
digital interface cable, and flow data is recorded along with gaseous emissions data and other
parameters in a single data file. An example of the SEMTECH EFM2 flow meter is shown in
Figure 2. This flow meter also incorporates the sampling probe through which the SEMTECH-
DS emission analyzer samples exhaust for delivery to the gaseous analyzers. This probe is
connected to the main SEMTECH-DS unit via a heated sampling line which is controlled to a
temperature of 19PC, in accordance with CFR Title 40 Part 1065.
The third key component of the SEMTECH-DS is the vehicle interface. This interface is
used to read engine variables broadcast from the ECM digitally via CAN. Variables are read
according to either the SAE J-1939 or SAE J-1708 protocol, depending on what is available from
a given engine. These ECM broadcast variables are required for estimation of torque and fuel
consumption during in-use testing, as well as to determine entry into or exit from the NTE zone.
A fourth component of the SEMTECH-DS is a temperature and humidity probe. This is
used by the SEMTECH-DS to monitor and record ambient temperature and humidity during in-
use testing. This probe is plugged into the main SEMTECH-DS unit, which takes the raw sensor
data from the probe and converts it to temperature and humidity measurement values.
Data from all of these components is generally recorded simultaneously, and stored in a
single data file for each test run on a memory card in the main SEMTECH-DS unit. This data
can later be retrieved via a laptop computer either over a wireless connection or via a cabled
Ethernet link. The laptop software interface also provides a means of user interface for manual
operations, diagnostics, and monitoring of the SEMTECH-DS during testing. The data recorded
by the SEMTECH-DS is then post-processed to determine emission values and to review quality
assurance parameters.
SwRI Report 03.12024.06 Page 4 of 371
-------
FIGURE 2. SEMTECH EFM2 EXHAUST FLOW METER AND CONTROL UNIT
1.2.4 PEMS Operations at SwRI
Once the PEMS hardware was delivered to SwRI for this program, modifications could
only be conducted in accordance with strict guidelines given in the Test Plan. In general,
modifications could only be conducted following approval from the Steering Committee. In
addition, PEMS operations were conducted only by SwRI staff in accordance with procedures
given in the standard documentation available for the PEMS. SwRI staff members were trained
by PEMS manufacturer representatives prior to the start of the program. PEMS representatives
were not allowed to be present during actual test operations. SwRI technicians Billy Valuk and
Richard Mendez were the PEMS operators during the program.
In general, PEMS manufacturers were allowed access to the hardware during this program
under only two conditions. The first was the failure of a 1065 audit performance check, in which
case, the PEMS manufacturer was offered an opportunity to correct the problem. The second
condition was in the event of an equipment malfunction which could not normally be repaired by
an end user. In the event of such repairs, appropriate 1065 audits were repeated to validate the
operation of the repaired systems before testing continued.
Throughout the course of the program, there were a variety of instances of both audit
failures and equipment malfunctions. An operating log of all of these occurrences was
maintained by SwRI throughout the course of the program. The complete log is included in
Appendix A of this report. For each incident, the log includes the date of occurrence, observed
failure symptoms, diagnostic steps, root cause analysis (if known), and corrective actions taken.
This log represents the collective PEMS operation experience with seven sets of PEMS hardware
over the course of roughly one year.
SwRI Report 03.12024.06
Page 5 of 371
-------
1.2.5 Emission Calculation Methods for In- Use Testing
Once a set of data has been recorded using PEMS hardware, calculations must be
performed to determine brake-specific emission values in accordance with methods outlined in
40 CFR Part 1065 Subparts G and J. The symbolic notation given in the formulas shown later in
this section is fully described in 40 CFR Part 1065 Subpart K.
40CFR Part 1065 allows for the use of any of three different calculation methods in order
to determine brake-specific emission values from in-use test data. The basic calculation of
brake-specific emissions requires three main inputs as follows:
nr^ . . Mass Concentration x Flow Rate
BShmission = =
Work Power
The three calculation methods vary somewhat in the means used to determine either the
Flow component or the Work component of this calculation. Each of the three methods is
summarized below. Because each method relies on different inputs, it is possible that each
method of calculation will react differently to various measurement errors. Therefore,
measurement allowances must be examined independently for each method. However,
according to the Test Plan methodology, only one of the three calculation methods would be
selected to generate the final measurement allowances. The selection methodology is outlined
later in this introduction under the Measurement Allowance Generation section.
1.2.5.1 Calculation Method 1 - "Torque" Method
Calculation Method 1 is analogous to the method used by most dynamometer laboratories,
and relies on direct input of both exhaust flow and torque. In the case of exhaust flow, this is the
flow rate measured by the same form of exhaust flow meter. In the case of the Sensors Inc.
PEMS, this is the value measured by the SEMTECH EFM2 exhaust flow meter. Work is not
measured directly, but is instead calculated using ECM broadcast engine speed and ECM
broadcast engine torque. While engine speed is directly measured by the engine ECM, ECM
broadcast torque is an estimate based on a variety of other parameters, therefore, torque cannot
be directly verified during in-use testing. A simplified formula for this method is:
Methodl = -
The more complete formula used for Method 1, using NOX as an example, is as follows:
eNOx
(glkW
hr}-
N
M * V
1V± N02 / ,
i=\
N
s
z=l
Speedt
(«vo,(
(rpm)*
ppm)
Tt(N
)*1
m}
60*1000*
0~6 *ri
*2*3
3600
( m
{ .
°0 * A "
' J \
14159*Ar"
SwRI Report 03.12024.06 Page 6 of 371
-------
It should be noted that calculation Method 1 is directly dependent on the accuracy of both
the exhaust flow meter and the torque estimation, as well as on the measurement of gaseous
concentration. This formula is applied similarly for CO and HC by replacing the measured
concentration and molecular weight values for NOX with those for the pollutant being calculated.
1.2.5.2 Calculation Method 2 - "BSFC" Method
This calculation is designated solely for in-use testing, and is designed to minimize the
effect of errors related to the accuracy of the exhaust flow measurement. Calculation Method 2
relies on flow weighting of individual readings during a test event. This means that although the
flow meter must be linear, it does not necessarily have to be accurate. In addition, Method 2
uses a carbon balance method to predict the fuel consumption rate, and a brake-specific fuel
consumption (BSFC) value to determine a final work term for the calculation. The BSFC value
is generally calculated using ECM broadcast values for fuel rate and for torque. A simplified
version of this method can be expressed as:
Method 2= -
CO 2 fuel
— x Work
ECMfuel
The more complete formula for Method 2, again using NOX as an example, is:
mol\ . .
(xNOXt (ppm)) * 1 (T6 * »,.
'
«,• [ — I * [xHCi (pPm)* 1 °~6 + (xCOt (%) + xCO2i (%))* 1 (T2 ]* At
BSFC
kW-hr
As mentioned earlier, Method 2 is not subject to accuracy errors for the exhaust flow
measurement, although that measurement must still be linear for the method to function properly.
Application of this formula to HC and CO is the same as what was outlined for Method 1.
L 2.5.3 Calculation Method 3 - "Fuel Specific " Method
Method 3 does not use direct measurement of exhaust flow, but relies on a carbon balance
and ECM broadcast fuel rate to determine mass. The work term for Method 3 is determined
identically to the work term for Method 1; using the ECM broadcast values for engine speed and
torque to calculate work. Method 3 entirely circumvents the use of an exhaust flow meter, but
for the HDIUT program, EPA must approve the use of Method 3 for a given test and
manufacturer. A simplified version of Method 1 may be expressed as:
SwRI Report 03.12024.06
Page 7 of 371
-------
I
Method 3 = •
ECU fuel
ex
CO 2 fuel
The more complete formula for Method 3, using NOX as an example is:
M
*w N
O2 fuel % ^-i
Mc ^
N
v
/_J
2=1
(*A^
xHCj^ppm)*!®
Speed ' i(rpm}*Ti
60*
/• \
v-v . #A/
"6 + (xCO,. (%) + xCO2j (%)) * 1 0~2
(jV-y»)*2*3.14159*A/"
1000*3600
It should be noted that Method 3 is not subject to exhaust flow measurement accuracy
errors, but also that this method is wholly dependent on ECM broadcast values for both mass and
work determination. Application of this formula to HC and CO is similar to that described for
Method 1.
1.3 Monte Carlo Model Simulation
The desire for this program was to generate measurement allowances based on rigorous
statistical methods applied to a large body of data. At the same time, it was desirable to exclude
outlier data caused by extreme measurement errors which were not considered representative of
normal in-use operations. A direct approach could have been to test PEMS against some kind of
mobile laboratory reference (such as the CE-CERT Mobile Emission Laboratory) on a large
number of vehicles, and quantify errors directly. However, such an approach would have been
prohibitively expensive in terms of both time and funding. In addition, the desired laboratory
reference point for error comparison was certification testing, which is normally conducted in a
dynamometer laboratory facility.
Given these factors, the Steering Committee ultimately elected to use a simulation
approach in order to generate the measurement allowances. In this approach, the Steering
Committee would define all of the expected sources of PEMS measurement errors, based on
existing in-use testing expertise and understanding of how the PEMS functioned. Each of these
errors would be quantified using a series of controlled laboratory experiments, each designed to
isolate errors related to a single error source. The results of each experiment would essentially
be an empirical model of a given source of measurement error. In this report, these error models
are referred to as error surfaces. It is important to note that each of these error surfaces
represents an incremental error of PEMS measurement, as compared to an associated laboratory
reference measurement.
All of these error surfaces were programmed into a computer model, which employed
Monte Carlo random sampling methods to simulate the combined effects of all of these sources
of error on the final measured brake-specific value. An ideal data set for a given test event was
SwRI Report 03.12024.06 Page 8 of 371
-------
run through the Model, and all the various errors were applied to that data set in a randomly
chosen manner. Brake-specific emission values were then calculated for both the ideal and
error-applied data sets, which were compared to yield a final measurement error. The process
was repeated thousands of times, with many different ideal data sets, to generate a large, robust
data set which was evaluated to determine a final set of combined measurement errors. These
final errors, referred to in this report as deltas, were generated for each pollutant and for each
calculation method, for a final set of nine deltas; three for each pollutant. A complete description
of the Monte Carlo methodology and of the model is given in Section 2 of this report.
1.4 1065 PEMS and Laboratory Audit
A key provision of both certification testing and compliance testing under the HDIUT
program is that manufacturers must use measurement equipment which meets the requirements
outlined in 40 CFR Part 1065. In particular 40 CFR Part 1065 Subpart D outlines a set of
performance checks which a measurement system must pass to insure the accuracy and
reliability of the instruments.
In light of these requirements, the Test Plan outlined a process wherein both the SwRI
reference laboratory and the PEMS would be audited prior to the start of testing, in accordance
with the procedures outlined in 40 CFR Part 1065 Subpart D. The audit was conducted on all
PEMS and laboratory instrumentation. In addition, a similar audit was also conducted on the
CE-CERT Mobile Emission Laboratory (MEL), which was later used during the validation
process outlined in Section 1.8. The performance checks were regularly repeated for both the
PEMS and the reference laboratory, in accordance with the requirements given in Subpart D. In
the event that a given PEMS failed a given Subpart D performance check during the initial audit,
the PEMS manufacturer was given an opportunity to correct the issue prior to the start of actual
testing, subject to the approval of the Steering Committee. The 1065 audit process and results
for the SwRI laboratory and the individual PEMS are described fully in Section 3 of this report.
1.5 Engine Dynamometer Laboratory Testing
A substantial number of the individual error experiments were conducted in an engine
dynamometer test cell located in the Department of Engine and Emissions Research at SwRI.
The test cell used for this program was Heavy Duty Transient Test Cell 27. This particular test
cell at SwRI is fully compliant with the procedures and methods of 40 CFR Part 1065. In
addition, the test cell incorporates additional equipment that can be used to simulate operation at
high altitudes, and also to simulate a wide range of ambient conditions in the intake air supply of
the engine. These expanded test cell capabilities were required for the proper conduct of some of
the experiments outlined in the Test Plan. SwRI technicians Gabriel Hernandez and Brian
Moczygemba were the engine operators during the program. Billy Valuk was the Test Cell 27
emissions cart operator during the program.
In general, the tests conducted at this location involved simultaneous measurements made
by both PEMS and the Laboratory on running engines. The engines were all equipped with
diesel paniculate filters (DPFs), in order to simulate the exhaust conditions of a 2007 or later
model year engine. Because the engines that were tested were not 2007 model year engines, the
SwRI Report 03.12024.06 Page 9 of 371
-------
NOX levels were roughly twice as high as those expected for such an engine. Three different test
engines were supplied to SwRI by participating engine manufacturers. These engines were a
heavy heavy duty (HHD) engine supplied by Daimler Chrysler, a medium heavy duty (MHD)
engine supplied by Caterpillar, and a light heavy duty (LHD) engine supplied by International.
Tests conducted in the dynamometer test cell included both steady-state and transient
exhaust emission measurements. In addition, a wide variety of experiments were conducted to
quantify errors in ECM broadcast torque and fuel rate, as compared to Reference Laboratory
measured values. Full details of all of these experiments, and their results are given in Section 4
of the report.
1.6 Environmental Chamber Testing
Another major portion of the Test Plan was devoted to characterizing PEMS measurement
errors related to varying environmental conditions that might be experienced in the field during
in-use testing. These tests were performed at a variety of facilities which are part of the
Mechanical and Material Engineering Division at SwRI. Environmental factors included in
these experiments included temperature, altitude, vibration, and electromagnetic interference.
PEMS were installed in specialized test facilities designed to simulate a wide variety of
conditions for each of these factors. In addition, testing was also performed to examine the
effect of ambient hydrocarbon levels on the PEMS HC measurement.
No engines were involved in the environmental tests. Instead, standard reference gases
were sampled by the PEMS during these tests. Therefore, the errors were determined by
comparing PEMS analyzer responses to the known, and verified, concentrations of the reference
gases. The exhaust flow meter was included in some of these tests, but because no exhaust was
flowing through the meter during environmental testing, only zero errors were examined for
exhaust flow during these tests. Full details of environmental testing and test results are given in
Section 5 of this report.
1.7 Exhaust Flow Meter Testing
A small set of experiments was specified in the Test Plan to evaluate the effect of various
installation and operation conditions on the exhaust flow meter. These conditions included
exhaust flow pulsations, non-uniform velocity profiles (possibly caused by pipe bends location
upstream of the flow meter), and the effect of wind across the open end of the exhaust flow
meter. These experiments were also conducted in the dynamometer test cell described in Section
1.5. Special exhaust systems and test rigs were set up for each of these experiments. The PEMS
exhaust flow meter measurements were compared to the Laboratory Reference raw exhaust flow
measurement during these experiments. Exhaust flow meter experiments and the results of those
tests are described in a portion of Section 4 of this report.
SwRI Report 03.12024.06 Page 10 of 371
-------
1.8 Model Validation
For reasons discussed earlier, the measurement allowances were generated using a Monte
Carlo computer model. As with all simulations, it is vital that such a model be validated through
comparison with real experimental data. In this case, the Measurement Allowance model needed
to be validated against a data set generated through actual in-use field testing. Because the
model generates an incremental error in comparison to a Laboratory Reference, a suitable in-use
reference measurement was needed for comparison to the PEMS measurements. The Steering
Committee determined that the CE-CERT Mobile Emission Laboratory, operated by the
University of California-Riverside, would be an appropriate reference for validation of the
model-based in-field testing.
In order to insure that the validation was not disturbed by some inherent bias between the
SwRI Reference Laboratory and the CE-CERT MEL validation reference, a correlation exercise
was performed between the two laboratories, prior to the start of on-road validation efforts. The
CE-CERT MEL was brought to SwRI's laboratory facilities in San Antonio, Texas, and a side-
by-side correlation test was run. During this test, exhaust from the same test engine was
alternately routed to the measurements systems of both SwRI and CE-CERT. This was done
repeatedly over the course of three days of testing. The data was then supplied to the Steering
Committee, in order to allow for a determination to be made that correlation between the
facilities was acceptable for the purposes of validation of the model.
After the correlation exercise was completed, a test truck was supplied to CE-CERT by
Caterpillar for use in this validation exercise. In addition, one of the audited PEMS used at
SwRI during the program was also delivered to CE-CERT. CE-CERT then conducted a series of
on-road test runs over various driving routes in California, which were designed to take the test
truck through a wide range of environmental and ambient conditions. During these tests,
simultaneous measurements were made with the PEMS and the MEL in order to generate a
validation data set. This formed the primary validation set for the model.
Because the CE-CERT MEL does not readily incorporate a means of direct torque
measurement on a vehicle, the on-road validation data set could not be used to validate model
errors associated with broadcast torque and derived BSFC. Therefore, an additional validation
exercise was conducted at SwRI. This involved removal of the engine from the test truck used
by CE-CERT, and installation of that engine in the SwRI dynamometer test cell. Selected
portions of the CE-CERT on-road tests were then simulated in the laboratory, to the extent
possible. Simultaneous laboratory and PEMS measurements were again taken during this
"replay" validation exercise. However, because the laboratory incorporates actual torque
measurement, it was possible to use this "replay" data set to validate the portions of the model
associated with torque and BSFC measurements.
Validation of the model was assessed independently for each of the three pollutants
(NMHC, CO, and NOX), and for each of the three calculation methods. A full description of the
validation efforts, including the data analysis methodology and the results of validation for each
pollutant by all three calculation methods is given in Section 6, with the exception of the CE-
CERT on road validation testing. This effort is described fully in a separate report, titled
SwRI Report 03.12024.06 Page 11 of 371
-------
Measurement Allowance On-Road Validation Project Report dated March 2007. The contents of
that report are incorporated herein by reference.
1.9 Measurement Allowance Generation
The generation of a set of measurement allowances represented the final outcome of this
program. The Test Plan provided a methodology by which all of the data from the millions of
Model simulation runs would be collected and analyzed statistically, in order to generate a set of
three potential measurement allowances for each pollutant, one for each of the three calculation
methods. The Test Plan then outlined a specific method by which the final set of allowances
would be chosen from among deltas generated for each of the three calculation methods. The
assumption made by the Test Plan, was that the final outcome of all previous efforts would be a
set of three validated potential measurement allowance values for each pollutant, NMHC, CO,
and NOX. Each potential allowance was expressed as a percentage of its associated NTE
threshold.
The NTE thresholds used for this program are given in Table 13. These NTE thresholds
were determined by EPA and approved by the Steering Committee during the generation of the
Test Plan. The Test Plan values were supplied in g/hp-hr as shown and calculated values in
g/kW-hr are also given for reference.
TABLE 13. NTE THRESHOLDS FOR MEASUREMENT ALLOWANCE PROGRAM
Pollutant
NMHC
CO
NOX
NTE Threshold
g/hp-hr
0.21
19.4
2.0
g/kW-hr
0.2816
26.02
2.682
These threshold values are of critical importance to the program, as they provide the basis
for the scaling of measurement allowances, the assessment of model convergence, and a variety
of other calculations performed during this program. The general philosophy of the Test Plan
was to determine measurement allowances based on errors at these emission levels, especially in
the case of any errors that scaled with emission level.
The anticipated outcome from the model runs, analysis, and validation efforts can be
represented as a table similar to the one shown in Table 14, which is repeated herein from the
Test Plan. The table illustrates both the model outcome, and the process for selecting the final
measurement allowance values.
SwRI Report 03.12024.06
Page 12 of 371
-------
TABLE 14. EXAMPLE OF MEASUREMENT ALLOWANCE DETERMINATION
FROM TEST PLAN
Calc. Method ==>
BSNOx
BSNMHC
BSCO
Measurement Errors at respective NTE threshold (%)
Method 1
Torque-Speed
18%
19%
3%
Method 2
BSFC
18%
17%
2%
Method 3
ECM fuel specific
20%
14%
1 %
max error ==>
min of max ==>
selected method==>
19%
18%
18%
20%
"BSFC" method
The intent of the final selection process was to first determine the largest percentage error
from among the three pollutants for each calculation method. These three largest errors would
then be compared with each other, and the method which produced the smallest of these three
would be chosen for calculation of the final measurement allowances. At that point, the
percentages given for the chosen calculation method would be applied to the NTE threshold
values given in Table 13, in order to generate the final additive, brake-specific measurement
allowances for each pollutant.
An implicit assumption of the process, as described in the Test Plan, was that the values
produced by the model for all three pollutants and all three calculation methods would be
successfully validated. In the event that this did not occur, it would be necessary for the Steering
Committee to determine a valid alternate course of action, in order to determine the final
measurement allowance values.
The final model run and the selection and generation of measurement allowances are
described fully in Section 7 of this report, including the final allowances approved by the
Steering Committee.
SwRI Report 03.12024.06
Page 13 of 371
-------
2.0 MONTE CARLO MODEL
2.1 Model Background
The main objective of this portion of the project was to use Monte Carlo techniques (e.g.
random sampling) in an error model to simulate the combined effects of all the agreed-upon
sources of PEMS error incremental to lab error on the components of the brake-specific (BS)
emissions. This was accomplished by creating "error surfaces" for the Monte Carlo simulation
to sample, based upon the results of a variety of lab experiments. The constructed model was
simulated for thousands of trials (i.e., iterations) using data taken from a reference data set of 195
unique NTE events. The model results were used to determine the brake-specific additive
measurement allowances for NOX, NMHC, and CO by three different calculation methods.
The error surfaces were generated from the results of each of the engine dynamometer and
environmental chamber laboratory tests described in Sections 4 and 5, respectively. The engine-
lab-test error surfaces covered the domain of error versus the magnitude of the signal to which
the error was to be applied (i.e., 5th to 95th percentile error vs. concentration, flow, torque, etc.).
The environmental-test error surfaces for shock and vibration, and electromagnetic and radio
frequency interference (EMI/RFI) covered the same domain as the engine tests, but only for
concentration. The environmental test for ambient hydrocarbons was similar, but the error
surface did not change as a function of concentration. The environmental test error surfaces for
pressure and temperature were characteristically different because they covered the domain of
the environmental-test cycle time versus the magnitude of the signal to which the error was to be
applied (i.e., error at a selected time vs. concentration). Details on how each surface was
generated are given in Sections 4 and 5 of this report. Since these surfaces are populated with
data representing the incremental errors between PEMS measurements and laboratory
measurements, they were sampled directly by the model.
2.1.1 Reference NTE Events
The reference data set to which all the simulated errors were applied represented engine
operations over a wide range of NTE events. This reference data set was generated from
collections of real-world PEMS data sets. Parameters in the reference data set were scaled in
order to exercise the model through a more appropriate range of parameters (i.e. concentrations,
flows, ambient conditions, etc.). In this scaling process, care was taken to maintain the dynamic
characteristics of the reference data set.
The Monte Carlo simulation model was run on a set of 195 reference NTE events
collected from a number of sources. Five engine manufacturers provided a total of 97 events; 10
reference NTE events came from each of the three engines tested in the lab during the transient
testing; 54 reference NTE events were created by adjusting the engine transient tests to cover a
larger spread of the emissions; and 14 events came from the pre-pilot CE-CERT data. Before
and after errors were applied in the Monte Carlo simulation for each of these reference NTE
events.
SwRI Report 03.12024.06 Page 14 of 371
-------
For all the reference data that was supplied by engine manufacturers, and the CE-CERT
data, it is understood that the NTE event data was based on actual field testing results. This was
done to insure that the NTE reference events would be representative of in-field operation in the
NTE zone. The data from the engine manufacturers was supplied directly to EPA individually.
The data was reviewed at EPA and then transmitted to SwRI. No additional information
regarding procedures used to generate this data was supplied to SwRI along with the reference
NTE events.
NTE brake-specific emissions results were calculated for NOX, CO and NMHC, using
each of the three agreed-upon NTE calculation methods. The three different BS emissions
calculation methods referred to in this test plan are:
1. Method #1: Torque-Speed Method
2. Method #2: BSFC Method
3. Method #3: Fuel Specific Method
The formulas and input constants for these three methods for each of the three emissions types
are provided in Appendix B.
Table 15 lists the number of NTE events obtained from each data source and the three
corresponding BS emissions calculated using Method 1. These emissions have been computed
with no error values added to the input parameters. For this report, emissions with no errors
added will be labeled the "ideal" emissions. In contrast, the emissions with errors added through
the Monte Carlo simulation will be labeled emissions "with errors".
TABLE 15. REFERENCE NTE EVENTS AND METHOD 1 BS EMISSIONS
Source
International
DDC
Caterpillar
Cummins
Volvo
Engine #1
Engine #2
Engine #3
Pre-Pilot
Number
of NTE
Events
19
18
20
20
20
28
28
28
14
BSNOx
g/kW-hr
Min
1.858
3.148
0.025
2.667
1.396
0.844
1.815
1.586
5.328
Max
5.446
6.012
5.865
6.687
2.457
5.799
3.397
3.467
7.193
BSCO
g/kW-hr
Min
0.520
0.221
0.000
5.995
1.159
0.145
0.150
0.261
0.110
Max
1.3563
1.888
1.361
0.232
0.266
0.496
0.511
0.530
0.341
BSNMHC
g/kW-hr
Min
0.073
0.002
0.000
0.006
0.004
0.000
0.000
0.000
0.000
Max
0.276
0.087
0.059
0.426
0.014
0.000
0.004
0.004
0.000
SwRI Report 03.12024.06
Page 15 of 371
-------
When the ideal brake-specific emission values were calculated for the various reference
NTE events, it was noted that these ideal emission values were frequently different from one
calculation method to another. While it was recognized that this was a realistic outcome, the
Steering Committee was concerned that these discrepancies might introduce an unintended bias
into the results of the Monte Carlo simulation. Therefore, the Steering Committee directed SwRI
to adjust the NTE reference event data in order to align the brake-specific emission levels from
all the calculation methods. In general, the values for Methods 2 and 3 tended to be very close to
each other, while the Method 1 value would be farther from the other two.
The adjustment was performed by first assuming that the Method 1 result was the desired
value, and that the other two calculation methods would be aligned to that result. This meant
that torque, speed, and exhaust flow values were not changed. The next step of the alignment
process was to adjust CC>2 values for the NTE event, in order to line up the Method 2 NOX result
with the Method 1 value. This was done by using a single multiplier on all CC>2 values for the
NTE event in question. Finally the fuel rate values were adjusted slightly in order to bring
Method 3 in line with Method 2. This second adjustment was generally on the order of 2 percent
or less, because Methods 2 and 3 were normally fairly close to each other.
The alignment was performed in order to get the NOX emission levels from all three
methods to line up precisely. It was initially assumed that CO and NMHC would also line up,
once the NOX values were aligned. In general, that is what happened; however, selected events
still demonstrated a misalignment of CO or NMHC once NOX was aligned. The Steering
Committee ultimately elected to accept small discrepancies in CO and NMHC between the
calculation methods as long as the magnitude of the differences were less than 1% of the NTE
threshold for CO and 2% of the NTE threshold for NMHC. Events which demonstrated larger
misalignment were removed from the reference data set. This review resulted in the removal of
four of the original events from the reference data set.
The distribution of the BS emissions data for the 195 reference NTE events to be
simulated in the Monte Carlo model are depicted in Figure 3 through Figure 5 for NOX, CO and
NMHC, respectively. Note that each emission has data values spread above and below the
corresponding NTE threshold. The NTE thresholds used in this analysis were:
• BSNOX 2.0g/hp-hr or 2.68204 g/kW-hr
• BSNMHC 0.21 g/hp-hr or 0.28161 g/kW-hr
• BSCO 19.4 g/hp-hr or 26.0150 g/kW-hr
SwRI Report 03.12024.06 Page 16 of 371
-------
Method 1 Ideal NOx (g/kW-hr) for 195 Ref NTE Events
50
40
30
20
10
0
0
2468
NOx(g/kW-hr)
FIGURE 3. METHOD 1 BSNOX VALUES FOR REFERENCE NTE EVENTS
Method 1 Ideal CO (g/kW-hr) for 195 Ref MTE Events
150 P
246
CO (g/kW-hr)
8
FIGURE 4. METHOD 1 BSCO VALUES FOR REFERENCE NTE EVENTS
SwRI Report 03.12024.06
Page 17 of 371
-------
Method 1 Ideal NMHC (g/kW-hr) for 195 Ref NTE Events
OJ
Z!
QJ
100 P
80
60
40
20
0
0
0.1 0.2 0.3
NMHC (g/kW-hr)
0.4
0.5
FIGURE 5. METHOD 1 BSNMHC VALUES FOR REFERENCE NTE EVENTS
Table 16 provides a summary of some descriptive statistics for the reference NTE data set
for each of the three BS emissions.
TABLE 16. DESCRIPTIVE STATISTICS FOR BS EMISSIONS FOR REFERENCE
NTE EVENTS
Descriptive
Statistic
Minimum
Maximum
Mean
Median
Standard Deviation
BSNOX
g/kW-hr
0.0249
7.1927
3.0071
2.6033
1.4807
BSCO
g/kW-hr
0.0000
5.9949
0.5936
0.3836
0.7129
BSNMHC
g/kW-hr
0.0000
0.4258
0.0287
0.0021
0.0591
The parameter data provided in each reference NTE event was on a second-by-second
basis with a minimum of 30 seconds and a maximum of 300 seconds. The input parameters
required for the BS emissions calculation methods and the Monte Carlo simulation are listed in
Table 17. An Excel file with a specific input format structure was used to standardize the format
of the input files. Since the total hydrocarbons (THC) was selected as an input parameter,
NMHC was computed as THC*0.98.
SwRI Report 03.12024.06
Page 18 of 371
-------
TABLE 17. INPUT PARAMETERS FOR REFERENCE NTE EVENTS
Variable
Number
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
Input Variable
NTE Event Number
NTE Source
Engine Make
Engine Model
Engine Displacement
Date
Time Stamp
Wet CO2
Wet CO
Wet kNO
Wet kNO2
Wet THC
Exhaust Flow Rate
Flowmeter Diameter
Speed
Low Speed, nlo
High Speed, nhi
Fuel Rate
Max Fuel Rate
Derived Torque
Peak Torque
BSFC
Units
integer
alphanumeric
alphanumeric
alphanumeric
L
mm/dd/yyyy
hh:mm:ss.s
%
%
ppm
ppm
ppm
scfm
3, 4, or 5 (inches)
rpm
rpm
rpm
L/sec
L/sec
N-m
N-m
g/kW-hr
Description
All reference NTE events must be identified by an NTE number (e.g.,
001).
The source of the NTE event is the company, organization and/or lab
that created the event data.
Engine Make
Engine Model
Engine Displacement (L)
The day the NTE event data was created (mm/dd/yyyy).
Time in seconds. Each reference NTE must contain second-by-
second data only.
CO2 (%)
CO (%)
NO (ppm) with intake air-humidity correction
NO2 (ppm) with intake air-humidity correction
THC (ppm)
Exhaust flow rate (scfm)
To compute the % of PEMS flowmeter maximum flowrate we will need
to know what size flowmeter was used for each NTE event.
Enter either 3, 4, or 5 to represent the following flowmeters and
maximum flow rates:
3 = 3 inch EFM with maximum flow rate = 600 scfm
4 = 4 inch EFM with maximum flow rate = 1 100 scfm
5 = 5 inch EFM with maximum flow rate = 1700 scfm
Engine speed (rpm)
To compute the % of normalized speed we will need nlo and nhi for the
engine computed as follows:
nlo (rpm) = lowest speed below max power at which 50% max power
occurs
nhi (rpm) = highest speed above max power at which 70% max power
occurs
Fuel rate (L/hr))
To compute the % of maximum fuel rate we will need the max fuel rate
of the engine for each NTE event.
Max fuel rate (L/hr)
Torque (N-m)
To compute the % of maximum torque we will need the peak torque of
the engine for each NTE event
Peak torque (N-m)
BSFC (g/kW-hr), enter this based upon interpolating your own BSFC
table or use the calculation in this spreadsheet, which uses fuel rate,
torque, and speed to calculate BSFC, & spgr=0.85, use appropriate
conversion factors and spgr.
2.1.2 Error Surfaces
During the initial review of the Test Plan and from discussions held at several Steering
Committee meetings, 52 error surfaces were initially identified and considered for inclusion in
the Monte Carlo simulation model. These individual error surfaces encompassed a wide variety
of error sources, and each of them was investigated in a specific experiment, as detailed later. In
some cases, upon reviewing the experimental data, the Steering Committee deemed that the
errors from certain sources were not significant; therefore, inclusion in the final Model was not
warranted. The details regarding which errors were not included in the model are given later
SwRI Report 03.12024.06
Page 19 of 371
-------
under the description of the individual error experiments in Sections 4 and 5 of the report. A
final total of 35 error surfaces were incorporated into the Model. Two additional errors terms
were also included for time alignment as detailed later, bringing the total number of error terms
incorporated in the model to 37.
Table 18 lists the error surfaces examined during the study with the surfaces excluded by
the Steering Committee designated in italics. All remaining ones were implemented in the
simulation model. Each error surface was assigned a number for easy identification.
Additionally, two error surfaces relating to the time alignment adjustment for NOX and CO (i.e.,
see Section on Time Alignment for NOX and CO} were also included.
SwRI Report 03.12024.06 Page 20 of 371
-------
TABLE 18. ERROR SURFACES FOR MONTE CARLO SIMULATION
Measurement Error Surfaces and Deltas Used in BS Emissions Calculations
Component
1 . Delta NOx
2. Delta CO
3. Delta NMHC
NMHC = 0.98THC
4. Delta Exhaust Flow
5. Delta Torque
6. Delta BSFC
7. Delta Speed
8. Delta Fuel Rate
9. Delta CO2
#
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
Test Source
Engine Dyno
Engine Dyno
Environ
Environ
Environ
Environ
Engine Dyno
Engine Dyno
Environ
Environ
Environ
Environ
Engine Dyno
Engine Dyno
Environ
Environ
Environ
Environ
Environ
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Environ
Environ
Environ
Environ
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Engine Manuf
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Engine Manuf
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Environ
Environ
Environ
Environ
Error Surface
Delta NOx SS
Delta NOx Transient
Delta NOx EMI/RFI
Delta NOx Atmospheric Pressure
Delta NOx Ambient Temperature
Delta NOx Vibration
Delta CO SS
Delta CO Transient
Delta CO EMI/RFI
Delta CO Atmospheric Pressure
Delta CO Ambient Temperature
Delta CO Vibration
Delta NMHC SS
Delta NMHC Transient
Delta NMHC EMI/RFI
Delta NMHC Atmospheric Pressure
Delta NMHC Ambient Temperature
Delta NMHC Vibration
Delta Ambient NMHC
Delta Exhaust Flow SS
Delta Exhaust Flow Transient
Delta Exhaust Flow Pulsation
Delta Exhaust Flow Swirl
Delta Exhaust Flow Wind
Delta Exhaust EMI/RFI
Delta Exhaust Vibration
Delta Exhaust Temperature
Delta Exhaust Pressure
Delta Dynamic Torque
Delta Torque DOE Testing
Delta Torque Warm-up
Delta Torque Humidity/Fuel
Delta Torque Fuel
Delta Torque Interpolation
Delta Torque Engine Manufacturers
Delta Dynamic BSFC
Delta BSFC DOE Testing
Delta BSFC Warm-up
Delta BSFC Humidity/Fuel
Delta BSFC Fuel
Delta BSFC Interpolation
Delta BSFC Engine Manufacturers
Delta Dynamic Speed |
Delta Dynamic Fuel Rate
Delta CO2 SS
Delta CO2 Transient
Delta CO2 EMI/RFI
Delta CO2 Atmospheric Pressure
Delta CO2 Ambient Temperature
Delta CO2 Vibration
Committee Action
Deleted by Steering Committee
Deleted by Steering Committee
Deleted by Steering Committee
Deleted by Steering Committee
Deleted by Steering Committee
Deleted by Steering Committee
Deleted by Steering Committee
Deleted by Steering Committee
Deleted by Steering Committee
Deleted by Steering Committee
Combined with #32
Combined with #39
Deleted by Steering Committee
Deleted by Steering Committee
Deleted by Steering Committee
For each of the measurement errors defined in Sections 4 and 5, an error surface was
created and used in the Monte Carlo simulation. Each error surface represented an additive
error—or a subtractive error if the sign was negative—relative to the reference parameter value
to which it was applied. Figure 6 through Figure 8 serve as a hypothetical example of how these
error surfaces were created for every measurement error. Details on the construction of each
SwRI Report 03.12024.06
Page 21 of 371
-------
error surface used in the simulation are provided in Sections 4 and 5. The example illustrated in
Figure 6 through Figure 8 represent the error surface for steady-state bias and precision NOX
concentration errors (Section on Steady-State Concentration error Surface Generation). The
plots shown correspond to hypothetical NOX emissions concentration data acquired in the
laboratory with three PEMS and three engines, with all nine sets of PEMS data pooled together.
PEMS vs. Laboratory Nominal Results
Figure 6 was constructed from raw data acquired from steady-state engine lab tests with
the PEMS at repeat testing at various concentration levels (NOX ppm). The plot pools all bias
and precision errors for all three PEMS and for all data from all three engines for all steady-state
modes. Twenty repeat measurements of NOX signals were taken for each of three PEMS yielding
60 data points at each value of the corresponding average lab NOX values (i.e., lab nominal
value). The 60 PEMS signals were plotted against the corresponding laboratory signals
measured using lab equipment. Shown in Figure 6 are the 5th, 50th, and 95th percentiles
corresponding to the distribution of these 60 observations using the PEMS at each average NOX
concentration level (note that the distribution of data at each NOX level may not represent a
normal distribution). Since the 50th percentiles do not lie on the dashed (diagonal) line of perfect
agreement, the data suggest that there is a bias error between the PEMS and lab results. In
essence this graph summarizes the statistical distribution measured by the PEMS at each
concentration level sampled. The example plot in Figure 6 shows only 6 discrete average NOX
concentration levels (ranging from 100-350 ppm). However, the actual number of discrete
concentration levels was determined using the total number of operating conditions actually run
for all the tests on all three engines. In the section on Steady-State Repeat Engine Testing and
Error Surfaces it is reported that 10 operating conditions from an initial number of 40 operating
conditions were selected for construction of the steady-state NOX error surface. Thus, the plot
used in the Monte Carlo simulation contained 30 discrete NOX concentration levels (10 operating
conditions x 3 engines).
SwRI Report 03.12024.06 Page 22 of 371
-------
NOx Concentration Errors
394
§" ,50
LU
Q_
0_
Q.
Z
100
en
n
285
242/^^
/rt^se-^256
W/j?s'
I3U/ /• •
^f /^-214
155^^-^T^T--
i i^sX^55
..•••*'
..-•'' 85
./t' 380 -
341 /^ >i .-•'
xf .X-S62
/?£&
/^307
fir
• 5th percent! le
50th percent! le
(median)
— * — 95th percent! le
50 100 150 200 250
NOx PPM (Lab, nominal)
300
350
400
FIGURE 6. ERROR SURFACE CONSTRUCTION: PEMS VS. LABORATORY
RESULTS
(PEMS - Laboratory) Deltas vs. Lab
Figure 7 illustrates the "error band" measured during testing. This plot was created by
first subtracting the individual "lab nominal" NOX value from the corresponding individual
PEMS NOX measurement for each test run. This difference was defined as the "delta" error.
Second, these "PEMS - Laboratory" delta errors were pooled at each average lab nominal NOX
value to obtain the 5th, 50th, and 95th percentile values displayed in Figure 7. Therefore, the plot
represents the average NOX lab nominal at 30 discrete concentration levels versus the percentiles
of the delta errors computed from the PEMS and laboratory individual test results.
SwRI Report 03.12024.06
Page 23 of 371
-------
NOx Concentration Errors
60
Q.
Q.
50-
40-
9 30-
55
-95th percentile
50th percentile
(median)
-5th percentile
400
NOx PPM (Lab, nominal)
FIGURE 7. ERROR SURFACE CONSTRUCTION: (PEMS - LAB) VS. LABORATORY
RESULTS
Variability Index vs. (PEMS - Laboratory) Deltas and Lab Nominal
This step normalized the plot in Figure 7 using what is called a "variability index (ic)".
This index represented the value randomly drawn by the Monte Carlo simulation in order to
select a given error level. It was allowed to vary from -1 to +1. The likelihood of "ic" being any
value between -1 through +1 was specified by a "probability density function (PDF)" assigned to
ic. In the case of this example, ic. was assumed to vary according to a standard normal (i.e., bell-
shaped) distribution during the Monte Carlo simulations. This was because it was believed that
the distribution of NOX errors due to steady-state bias and precision would be centered about the
50th percentile of the full range of conditions measured according to the section on Steady-State
Repeat Engine Testing and Error Surfaces. Each set of data for each lab "set point" average
(i.e., lab nominal value) in Figure 7 was normalized by aligning the corresponding 5th percentile
error from Figure 7 with ic = -1, the 50th percentile error with ic = 0, and the 95th percentile error
with ic = +1. These values were then plotted in Figure 8, where the y-axis is the variability index,
the x-axis is the average lab nominal NOX value, and the z-axis is the delta NOX value. Notice
that, when using this normalization approach, the 5th, 50th, and 95th percentile values remain
equivalent between Figure 7 and Figure 8. Error surfaces such as the one presented in Figure 6
are the error deltas the Monte Carlo simulation program used during calculation of the BS
emissions "with errors".
SwRI Report 03.12024.06
Page 24 of 371
-------
NOx Concentration Errors
Error Surface: z-axis = DELTA(NOx PPM)
1.5
1 -
•§ 0.5 H
£
I1
15
n 0
-0.5-
Q.
Q.
-1 •
-1.5
55
40
42
35
41
44
•95th percent! le
50th percentile (median)
•5th percent! le
25
20
30
20
25
30
50 100 150 200 250 300 350 400
-15 +5 +14 +6 +7 +12
NOx PPM (Lab, nominal)
FIGURE 8. ERROR SURFACE CONSTRUCTION: ERROR AT VARIABILITY INDEX
VS. LABORATORY RESULTS
2.1.3 Error Surface Sampling and Interpolation
The error model used two different probability density functions to sample the error
surfaces, depending upon which experimental parameter the surface represented. To sample
error surfaces that were generated from the lab test results (Section on Engine Dynamometer
Laboratory Testing), and the environmental test results for shock and vibration, EMI/RFI, and
ambient hydrocarbons, the model used a truncated standard normal PDF because these tests were
designed to evenly cover the full, but finite, range of engine operation and ambient conditions.
To sample error surfaces that were generated from the pressure and temperature environmental
test results (Section on Environmental Chamber Testing), the model used a uniform PDF because
these tests were already designed to cover the typical range and frequency of the respective
conditions. Both of these sampling distributions are depicted in Figure 9.
SwRI Report 03.12024.06
Page 25 of 371
-------
,
Probability Density Functions for Sampling Error Surfaces Once Per NTE Event
Lab Tests, Normal, SD=0.60795, truncate @ -1 & 1
^—Environmental Tests, Random
Note: A non-trur
distribution with
values of 0.05 ai
ic=+1, respective
icated normal
SD=0.60795 has
id 0.95 at ic=-1 ar
;ly.
^
c
^
^^
p
d ^^^
^*^
^^
^^
^
*^^
^^^
^^^
> 4 3 2
Relative Probability
(
1.00
0.75
0.50
0.25
0.00 .-"
-0.25
-0.50
-0.75
-1.00
FIGURE 9. TRUNCATED STANDARD NORMAL AND UNIFORM PROBABILITY
DENSITY FUNCTIONS
When using the truncated standard normal PDF (see Figure 9), the Monte Carlo model
sampled normal deviates that ranged between -1 and +1. These were used as the ic values
defined in the section on Error Surfaces. Similarly, the pressure and temperature environmental
tests used a uniform PDF to sample test time, from which calculated errors were used. All
temperature error surfaces related to the four emissions were sampled uniformly from 1 to 1080
minutes while the error surfaces related to the pressure were sampled uniformly from 1 to 720
minutes. Exhaust flow error surface for temperature was sampled uniformly from 1 to 478
minutes while the exhaust flow for pressure was sampled uniformly from 1 to 360 minutes. The
errors from all the other tests were aligned with the truncated standard normal PDF such that
each of the 50th percentile error values at each of the tested signal magnitudes was centered at the
median (i.e., 0 value) of the PDF, and the 5th and 95th percentile error values at each of the tested
signal magnitudes were aligned with the extreme negative (ic = -1) and positive (ic = +1) edges of
the PDF, respectively.
Each error surface was sampled along its ic axis (y-axis) once per trial for a reference
NTE event simulation. Hence, every error surface had a separate randomly selected ic for each
trial. Since each reference NTE event contained second-by-second parameter data, the error
surface was sampled at a given ic on the y-axis and at the several selected parameter values on
the x-axis that corresponded to each second of the reference NTE event. The sampled error
value was determined for the given second and parameter along the error axis (z-axis) at the
intersection of the ic value and the parameter value from the reference NTE event. This was
accomplished by taking each second in the reference NTE event and finding the two adjacent x-
axis values from the error surface between which to linearly interpolate to obtain the error
surface x-value. Each second in the reference NTE event was linearly interpolated with the same
ic value for a particular trial at the error surface x-value. If any of the sampled lab nominal
SwRI Report 03.12024.06
Page 26 of 371
-------
values (NOX, NMHC, CO, Speed, Fuel Rate, etc.) exceeded the upper or lower limits of the
parameter error surface, the value of the closest endpoint of the error surface was assigned to
them.
Figure 10 depicts an example of the error surface sampling using a steady-state NOX error
surface containing 30 lab nominal NOX x-axis values. For this particular trial, the randomly
selected ic is -0.5. The example reference NTE event is noted by the symbol '*' and it plotted at
ic = -0.5 for each second in the NTE event.
SS Error Surface for NOx Concentration
Error Surface: z-axis = ASS_NOx_ppm
A A-44-At - A- A A- -A
100 200 300 400 500 600
'I o
-0.5
NOx ppm (lab,nom)
-A- - - 95th percentile
-•- - - 5th percentile
NTE Event
50th percentile (median)
ic = -0.5
FIGURE 10. STEADY-STATE NOX ERROR SURFACE WITH EXAMPLE SAMPLING
FOR A REFERENCE NTE EVENT
2.1.4 Brake-Specific Emissions Calculations
Errors from Sections 4 and 5 were combined by adding all of the sampled errors once per
trial for each reference NTE event simulation. For example, in order to assess the errors in NOX
concentration by calculation Method #1, several error surfaces were sampled and added to the
corresponding parameter in the Method #1 calculation and the resulting BSNOX "with errors"
SwRI Report 03.12024.06
Page 27 of 371
-------
was computed. The errors used in this calculation are the following (note that the corresponding
error surface numbers are provided in the subscripts):
NOX ppm 'with errors' =
Exhaust Flow % 'with errors'
Torque % 'with errors'
Jx ppm_reference ~"~ ^ J^'-'x ppm_l ~"~
A NOX ppm_2 + A NOX ppm_5
ExhaUSt Flow o/0 reference +
A Exhaust Flow % 20 A Exhaust Flow% 21 +
A Exhaust Flow % 22 + A Exhaust Flow % 23
A Exhaust Flow % 25 + A Exhaust Flow % 27
A Exhaust Flow % 28
Torque o/oreference +
A Torque % 29 + A Torque % 30"
* ^ i + A Torque % 32 -
A rr-i
A Torque % 31 + A Torque % 32 -
A Torque % 34 + A Torque % 35
Speed % 'with errors' = Speed % reference + A Speed % 43
where,
A 1,2 = NOX concentration errors due to steady-state and transient errors,
A 5 = NOX concentration errors due to ambient temperature,
A 20,21 = exhaust flow errors due to steady-state and transient errors,
A 22,23 = exhaust flow errors due to pulsation and swirl,
A 25 = exhaust flow errors due to ambient temperature,
A 27,28 = exhaust flow errors due to temperature and pressure,
A 29 = torque errors due to dynamic torque,
A 30,31 = torque errors due to DOE and warm-up,
A 32 = torque errors due to interacting parameters humidity and fuel,
A 3435 = torque errors due to interpolation and engine manufacturers,
A 43 = speed errors due to dynamic speed
Using the formulas for the calculation methods in Appendix B, the BSNOX for Method #1
was computed without errors ("ideal") and then with all the errors applied as outlined above.
Table 19 lists all error surfaces used by each calculation method for all three emissions.
SwRI Report 03.12024.06
Page 28 of 371
-------
TABLE 19. ERROR SURFACES USED FOR COMPUTING BRAKE-SPECIFIC
EMISSIONS BY THREE CALCULATION METHODS
Component
1. Delta NOc
2DeltaCO
3. Delta NVHC
NVHC=0.93THC
4 Delta Exhaust Rcw
5. Delta Torque
6. Delta BSFC
7. Delta Speed
8. Delta Fuel Rate
9. Delta C02
#
1
2
5
7
10
11
13
14
16
17
19
20
21
22
23
25
27
23
29
30
31
32
34
35
36
37
38
39
41
42
43
44
45
46
49
EirorSuface
Delta NOxSS
Delta NO Transient
Delta NO< Ardent Temperature
Delta 00 SS
Delta CO Arrospheric Pressure
Delta 00 Ardent Terrperatue
Delta NVHC SS
Delta NVHC Transient
Delta NVHC Amospheric Pressure
Delta NVHC Artient Temperature
Delta Ardent NVHC
Delta Exhaust FlowSS
Delta Exhaust Rcw Transient
Delta Exhaust Rcw Pulsation
Delta Exhaust RcwSwrl
Delta Exhaust EM/RR
Delta Exhaust Terrperature
Delta Exhaust Pressure
Delta Dynarric Torque
Delta Torque DOE Testing
Delta Torque V\6rmup
Delta Torque Huridty/Fuel
Delta Torque Interpolation
Delta Torque Engine IVanuf
Delta Dynarric BSFC
Delta BSFC DOE Testing
Delta BSFC Wsrm-up
Delta BSFC Hrridty/Fuel
Delta BSFC Interpolation
Delta BSFC ErgneManuf
Delta Dynarric Speed
Delta Dynarric Fuel Rate
Delta 002 SS
Delta 002 Transient
Delta 002 Ardent Temperature
Method Isolation
BSNOx
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
BSCO
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
BSMVHC
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
Method 2 (Violation
BSNOK
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
BSCO
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
BSNIVHC
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
Method 3 Cdolafion
BSNOx
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
BSCO
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
BSNIVHC
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
2.1.5 Periodic Drift Check
During the Monte Carlo simulation for a particular reference NTE event, the BS
emissions computed during each simulation trial (with ic selected randomly) was checked to
determine whether or not a periodic drift would have invalidated the NTE event trial. The drift
check results were simulated by computing the BS emissions with all the error surface errors
added except those due to the environmental error surfaces. Therefore, the following error
surfaces were excluded in computing the drift check: temperature error surfaces for NOX, CO,
CO2, and NMHC; pressure error surfaces for CO and NMHC; and ambient NMHC. If the
absolute difference in the BS emissions 'with all errors' and the BS emissions 'with all errors
except environmental' was greater than a percentage of the emissions threshold, then periodic
drift was detected and the simulation trial was eliminated from the analysis. The percentages
used in this study were 4% of the NOX and CO threshold (0.080 and 0.776 g/hp-hr, respectively)
and 10% of the NMHC threshold (0.021 g/hp-hr). Figure 11 represents the periodic drift
process.
SwRI Report 03.12024.06
Page 29 of 371
-------
Generate ic for each error surface.
Interpolate A errors.
Compute Emission "with errors"
Retain
Emission "with
errors"
Compute Emission with errors
except environmental"
ABS(Emission
"with errors" -
Emission "with
errors except
environmental)
Delete Emission "with
errors" from current i
< 4% (10%) of
Emission
Threshold?
(No Drift)
> 4% (10%) of
Emission
Threshold?
(Drift Detected)
FIGURE 11. PERIODIC DRIFT CHECK FLOWCHART
2.1.6 Time Alignment for NOX and CO
The time alignment adjustment measured the effect of errors in time alignment of the
various continuous PEMS data sources on the final BS emission results. This error source was
not originally included in the Test Plan, and no experiment had been designed to examine it.
However, it was later decided that time alignment was a significant source of potential error, and
that it should be incorporated into the Model. Time alignment values were only generated for
NOX and CO, because NMHC values were too low and too stable to see any discernible trends in
NMHC due to time alignment. Details regarding the methodology used to determine the time
alignment adjustment are given later in Section 4.12.
Although time alignment was not applied in the same fashion as the other error surfaces
in this model, it was described as an error surface because it was sampled as a normal
distribution similar to the other error surfaces. The time alignment adjustment was a
multiplicative factor which was applied to the BS emission result after all other error terms had
been added. The time alignment represented an adjustment up or down as a percentage of the BS
emission level "with errors". A separate time alignment factor was developed for each pollutant,
and for each of the three calculation methods allowed in the HDIUT program. Thus, during the
Monte Carlo simulation for each trial the brake-specific differences were computed as follows:
(BS emissions 'with errors' * Time Alignment Adjustment) - "Ideal" BS emissions
SwRI Report 03.12024.06
Page 30 of 371
-------
2.1.7 Convergence and Number of Trials
Since the Test Plan did not include a provision for convergence criteria, the Steering
Committee was tasked to develop a convergence method. The main goal was to define how
many simulation trials at a given reference NTE event were required to estimate the 95th
percentile BS emission differences with a given precision. Although the Crystal Ball software
contained precision control options, the method used to compute a confidence interval on
percentiles was based on an analytical bootstrapping method which was not adequately
documented. Thus, an independent convergence method was proposed and accepted by the
Steering Committee.
A nonparametric statistical technique [Reference: Practical Nonparametric Statistics,
WJ. Conover, John Wiley & Sons, 1971] was proposed which defined a 90% confidence
interval for the 95th percentile of the BS emissions differences for an individual reference NTE
simulation. If the width of the 90% confidence interval was less than 1% of the BS emissions
threshold, then convergence was met. The following steps define the convergence method:
1. Run the Monte Carlo simulation for TV trials.
2. Order the BS emissions differences from smallest to largest.
3. Identify the trial number at the lower end of the 90% confidence interval
niower = 0.95 * N -1.645V0.95 * 0.05 * N
4. Identify the trial number at the upper end of the 90% confidence interval
ripper = 0.95 * N + 1.645V0.95 * 0.05 * N
5. Compute (BS difference value at nupper)- (BS difference value at niower).
6. If the result in (5) < 1% of the BS emissions NTE threshold then convergence is met.
7. 1% of Thresholds g/hp-hr g/kW-hr
BSNOX 0.0200 0.026820
BSNMHC 0.0021 0.002816
BSCO 0.1940 0.260150
The Screening Committee agreed to the proposed convergence criteria outlined above.
During the initial simulation runs, all reference NTE events at an ideal BS emission level at the
threshold and below appeared to converge within the 1% level in 10,000 trials. However, there
were a number of reference NTE events with BS emissions levels that were as much as 3 times
the NTE threshold. This presented an initial problem in terms of the stated convergence criteria
since it was based on a fixed threshold value. Essentially this meant that in order to meet the
criterion, some of the higher BS emission level events (>5 g/kW-hr) would have had to converge
to a 90% confidence width of well below 0.5% of the threshold value, which would have
required an extremely high number of trials. To correct this problem the Steering Committee
chose to use the following two-step procedure in deciding the number of trials to run and the
convergence criteria:
1. For all reference NTE events with NOX values equal to or less than 2.6 g/kW-hr, a total of
10,000 trials were run and checked for convergence. It was expected that all of these would
converge well within the 1% criteria at this sample size. If any individual reference NTE events
did not converge at this run length, those events were run to 30,000 trials.
SwRI Report 03.12024.06 Page 31 of 371
-------
2. For reference NTE events with NOX values greater than 2.6 g/kW-hr, a total of 10,000 trials
were run and checked for convergence. If convergence was not achieved those same events were
run to 30,000 trials. If these events still did not converge within 1% of the threshold value, the
procedure was to do one of the following:
a. If there was convergence within at least 2% of the threshold value, the reference NTE
event was included as part of the simulation data set for the measurement allowance and
no additional runs were made.
b. If the reference NTE event did not converge within at least 2% of the threshold value, the
event was dropped from the simulation data set considered for the measurement
allowance.
In summary, the 195 reference NTE events were run at either 10,000 trials or 30,000 trials and
convergence was checked. For all but four reference NTE events, the convergence criteria was
met at the 1% NTE threshold value for all three emissions and all three calculations methods.
Since only four reference NTE events failed the initial criteria at 1% of the NTE threshold,
simulations for these four events were continued up to 50,000 trials. By that point all four NTE
events met the convergence criteria.
2.1.8 Simulation Output
During the simulation of a reference NTE event, differences between the BS emissions
"with errors", including time alignment adjustment, and the ideal BS emissions were obtained by
each of the three calculation methods. These differences were computed thousands of times
(once per trial) until the model converged. Then the 95th percentile difference value was
determined for each reference NTE event's distributions of BS differences for each emission
(NOX, NMHC, and CO) for all three calculation methods.
The output from the Crystal Ball simulation for each reference NTE event was saved in
two separate Excel files: an EXTRACT and a REPORT file. The EXTRACT file contained
descriptive statistics on all differences computed for BS emissions by all three calculation
methods, percentiles (0%, 5%, 10%,...95%, 100%) of the differences in BS emissions,
sensitivity data for all error surfaces, and differences in BS emissions computed at each trial in
the simulation.
The REPORT file contained a summary of the differences in the BS emissions for all
three calculation methods including descriptive statistics, the number of trials that were not
excluded due to periodic drift, a frequency histogram of the differences in BS emissions, and
percentiles (0%, 5%, 10%,...95%, 100%) of the differences in BS emissions. Also included
were descriptive statistics on each ic distribution sampled for each error surface. Lastly,
sensitivity charts for the differences in BS emissions for the three calculation methods were
stored. These charts provided information on how much each error surface influenced the
differences computed between the BS emissions "with errors" and the ideal BS emissions.
A more detailed description of the Crystal Ball output files can be found in Appendix C.
SwRI Report 03.12024.06 Page 32 of 371
-------
2.1.9 Step-by-Step Simulation Example
In order to clarify the simulation process the following step-by-step summary is provided.
This example assumes that a single reference NTE event was simulated for the BSNOX
difference computations. Figure 12 provides an overview of the simulation process.
Reference NTE
Monte-Carlo Simulation
NOx =NO
+ N02
(ppm)
NMHC
(ppm)
Exhflow
(scfm)
Torque
(N-m)
BSFC
(g/kW-hr)
Speed
(rpm)
Fuel
Rate
(L/sec)
CO,
Time
Alignment
Adjustment
95th percentile
BSNOx
differences
4-
Calculate
Differences*
(1) BSNOx = f (NOx, Exhflow, TECM, Speed)
(2) BSNOx = f (NOx, Exhflow, BSFCECM)
(3) BSNOx = f (NOx, CO2, CO, THC, TECM,
Fuel RateECM, Speed)
J
'Difference = (BSNOx "with errors" and Time Alignment Adjustment) — ("Ideal" BSNOx)
FIGURE 12. OVERVIEW OF MONTE CARLO SIMULATION FOR BSNOx
STEP 1 Enter the reference NTE input parameters into the Monte Carlo (MC) simulation
model. These include the emissions concentrations, exhaust flow, torque, BSFC, speed and fuel
rate data used in all three calculation methods.
STEP 2
NTE event.
Compute the "ideal" BSNOX by all three calculation methods from the reference
STEP 3 Set-up the Monte Carlo simulation parameters in Crystal Ball. An Excel
spreadsheet model was developed for use with Crystal Ball MC software for error analysis of
brake-specific emissions. Crystal Ball is a graphically-oriented forecasting and simulation
software that runs on Microsoft® Windows and Excel. The simulations run in this program used
Crystal Ball 7.1 and 7.2.2 Academic versions and were run on PCs configured with a Pentium 4
CPU, 3.0 GHz, 2.0 GB RAM, 232 GB hard drive and Windows XP operating system.
Microsoft® Excel 2003 SP was the spreadsheet software.
The options exercised in running Crystal Ball included the following:
• Number of trials = 10,000 or 30,000
SwRI Report 03.12024.06
Page 33 of 371
-------
o If the ideal emission < BS emission threshold then # trials = 10,000
o If the ideal emission > BS emission threshold then # trials = 30,000
o If convergence was not met then # trials = 50,000
• Monte Carlo sampling method with random initial seeds
• Normal speed run mode
• Suppress chart windows (fastest run time)
The Excel spreadsheet is in a modular structure following the specified model outline, and it
makes provisions for the three identified calculation modules. Input cells to the model are
clearly identified to facilitate any revisions that may become necessary for users who want to
exercise the model with other Monte Carlo software such as @Risk or newer versions of Crystal
Ball. The spreadsheet was tested with controlled test cases of simplified input distributions with
the Crystal Ball add-on to confirm correct model implementation in accordance with this test
plan. At least one typical analysis was run as an additional confirmation, and two independent
checks were made on the ideal emissions by other SwRI staff. A complete description of the
spreadsheet computations is contained in Appendix D.
STEP 4 Execute a single MC trial by randomly generating a separate ic for each error
surface used in the three calculations.
STEP 5 For each second in the reference NTE event, interpolate the A error for all error
surfaces at the input parameter values and the randomly generated ic. Figure 13 illustrates all the
error surfaces available (in yellow) and where the corresponding A errors are added. Error
surfaces depicted in blue were identified, discussed and eliminated by the Steering Committee.
SwRI Report 03.12024.06 Page 34 of 371
-------
Exhflow
|— H ANOxSSjTm
— H ANOxTOjTm
— >\ ANOxRMI/RFIJT,
-H ANOxrn,w_ITm
— *\ ANOx,
L- M ANOx,Hhmm
| — H ACO^ ~
— H AGO™ ~
— H ACCwI/PIJI 0/,
— H ACO^.. „,
— H ACO,.mr 0/.
>— H ACO.HK ox.
i— H ANMHC^ ^
— H ANMHCTO „,„
— H ANMHCm,roi.T
— H ANMHC^.,,^
— H ANMHC,.mrrr
— »| ANMHC.^j,^
1 — H ANMHC_^m,
i — H AExhflow.,.,
— H AExhflowTO
— H AExhflowr,,,,,«
— H AExhflow™,.,
— * AExhflow,,^^
— »• AExhflowBUI/p
— H AExhflow,^
— * AExhflow,_r
1 H AExhflow™,,,,,,
(1)
(2)
„ (3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
^ (15)
„ (16)
„ (17)
(18)
(19)
(20)
(21)
„ (22)
(23)
(24)
BT (25)
(26)
(27)
. (28)
1 Engine
J dyno
) Environ
chambers
)yno
Environ
chambers
} Engine
dyno
1
L Engine
1
J dyno
Environ
h K
chambers
Torque
Blue = Deleted by SC
Yellow = Included in MC
BSFC
-
r>| ATorqueDvmmic (29)
*
>
->
->
->
ATorqueDOE (30)
ATorque,^,, (31)
ATorquehmT]id/fllel (32)
ATorque^,,; (33)
ATorqueillerI,olation(34)
J
ATorqueenBne.manuf(35J
ABSFCDynamlc (36)
ABSFCDOE (37)
-H ABSFC_B (38)
-H ABSFChmmd/fhel (39)
-H ABSFC^ (40)
-1 ABSFCmternolatlon (41)
-H ABSFCemme.manuf(42)
73
e
$
ASpeedD
(43)
AFuel
c (44)
AC02S.. ,
(45)
ACO2T
(46)
AC02FMT/RFT % (47)
Engine
dyno
ACO2,,
(48)
AC02,m
(49)
AC02vih
(50)
FIGURE 13. ERROR SURFACES INCLUDED IN MONTE CARLO SIMULATION
STEP 6
Compute one BSNOX "with errors" for the given MC trial by adding all the A
error values to the reference NTE data and then calculating the BSNOX by all three calculation
methods.
STEP?
Check for periodic drift to determine if the BSNOX "with errors" for the given MC
trial is valid. If it is valid then continue to Step 8. Otherwise, eliminate the data from the current
trial and return to Step 4 to start a new trial.
STEP 8 Compute BSNOX difference for the current trial:
(BS emission "with errors" * Time Alignment Adjustment) - "Ideal" BS emission
STEP 9 Repeat Steps 4-8 until the number of trials is met.
STEP 10 Check the differences in BSNOX for all three calculation methods to be certain
that the convergence criteria are met. If convergence is met for all three calculation methods,
continue to Step 11. Otherwise, return to Step 4 and run the Monte Carlo simulation for an
additional 10,000 trials until the total number of trials is 50,000.
SwRI Report 03.12024.06
Page 35 of 371
-------
-th
STEP 11 Select the 95 percentile from the distribution of BSNOX differences for each of
the three calculation methods. Store the ideal BSNOX and the 95th percentile BSNOX differences
for computing the measurement allowance.
STEP 12 Repeat Steps 1-11 for each reference NTE event.
2.1.10 Measurement Allowance
At this point in the process there were nine distributions of 95th percentile differences,
where the 195 reference NTE events were pooled by the three emissions (NOX, CO, NMHC)
times three different calculation methods. Each of the 95th percentile distributions represented a
range of possible measurement allowances. From each of these nine distributions of possible
measurement allowances, one measurement allowance per distribution was determined. These
measurement allowances were computed by a regression method or a median method as
described below. Both of these calculations methods were decided by the Steering Committee
prior to the start of the program, and they were specified in the Test Plan.
Regression Method
This method involved determining the correlation between the 95th percentile differences
versus the ideal emission values for the reference NTE dataset. For each combination of
emissions and calculation method, a least squares linear regression of the 95th percentile
differences versus the ideal emissions results was computed. If the R2 value from the regression
model was greater than 0.90 and the SEE (standard error of the estimate or root-mean-squared-
error) was less than 5% of the median ideal emission result, then the linear regression equation
was used to determine the measurement allowance for that emissions and calculation method.
To determine the measurement allowance the NTE threshold was used to predict the
measurement allowance from the regression model. The NTE thresholds are given in Table 13.
The measurement allowance was then expressed as a percentage of the NTE threshold value.
Median Method
If the linear regression did not pass the aforementioned criteria for the R2 and SEE
statistics, then the median value of the 95th percentile differences from the 195 reference NTE
events was used as the single measurement allowance for a combination of emissions and
calculation method. The measurement allowance was then expressed as a percentage of the NTE
threshold value.
After all 95th percentile distributions were evaluated, there were nine measurement
allowances corresponding to the nine combinations of the three emissions and the three different
calculation methods.
Next the maximum allowance (in percent) among the three emissions was selected for
each of the given calculation methods. The calculation method corresponding to the minimum
of these three maximum values was chosen as the best method, and it provided the BS
measurement allowances (in percent) for NOX, NMHC, and CO, respectively. The final additive
SwRI Report 03.12024.06 Page 36 of 371
-------
BS measurement allowances were computed by multiplying each of the three measurement
allowances (in percent) times their corresponding threshold values. Each of these values would
be the very last value added to the actual brake-specific NTE threshold for a given engine, based
on actual family emissions limit, mileage, model year, etc. Note that if any measurement
allowance was determined to have a value less than zero, then that measurement allowance was
set equal to zero.
Table 20 below illustrates the selection of the calculation method for all of the
measurement allowances. The example is based on a hypothetical set of nine measurement
allowances for the three emissions and three calculation methods. The calculation method is
selected by first picking the maximum allowances of all the emissions for each of the given
calculation methods. For each column the maximum value is selected (highlighted in yellow).
Then the minimum of these maximums is used to select the best method (highlighted in blue). In
this hypothetical case, the BSFC method would be selected. Therefore, 18%, 17%, and 2%
would be selected as the best measurement allowances for NOX, NMHC, and CO, respectively.
TABLE 20. EXAMPLE OF SELECTION OF THE MEASUREMENT ERROR
Calc. Method ==>
BSNOx
BSNMHC
BSCO
Max Error ==>
Min of Max ==>
Selected Method==>
Measurement Errors at Respective NTE Threshold (%)
Method 1
Torque- Speed
18%
19%
3%
19%
Method 2
BSFC
18%
17%
2%
18%
18%
Method 3
ECM Fuel Specific
20%
14%
1%
20%
"BSFC "Method 2
For the data given in Table 20, the BS measurement allowances would be computed as:
• NOX
• NMHC
• CO
= 18%*2.00g/hp-hr =
= 17%* 0.21 g/hp-hr =
= 2% * 19.4g/hp-hr =
0.3600 g/hp-hr
0.0357 g/hp-hr
0.3880 g/hp-hr
2.1.11 Validation
The final validation methodology for the Monte Carlo model varied from the one that
was originally proposed in the Test Plan. This occurred for several reasons.
• The method described in the Test Plan required that CE-CERT be able to measure raw
emissions concentrations or determine dilution ratio accurately. However, CE-CERT's
mobile laboratory was only capable of making dilute measurements; therefore a dilution
SwRI Report 03.12024.06
Page 37 of 371
-------
ratio needed to be established. In addition, the mobile laboratory did not include any
direct method for the measurement of either exhaust flow or intake air flow. Although
the CE-CERT mobile laboratory could measure dilution ratio by measuring both the total
CVS flow and the dilution air flow rate, and subtracting to determine exhaust flow rate,
there was some concern about the accuracy of this measurement during short NTE events
involving a potentially wide dynamic range of dilution ratios. Since the success of this
measurement would be critical to the model validation under the methodology given in
the Test Plan, the Steering Committee decided that, due to the reliance on this dilution
ratio measurement method, there was a considerable degree of risk associated with the
original validation methodology, and that an alternative method might prove more robust.
• The Test Plan included an alternative methodology in the event that the CE-CERT
laboratory was unable to accurately determine raw exhaust flow or dilution ratio.
However, the proposed method had several potential problems, and the Steering
Committee decided that this option was not a good choice due to potential bias problems.
• A third option was also mentioned briefly in the Test Plan. It involved comparing the
NTE events recorded by the PEMS and the CE-CERT trailer. However, the Steering
Committee decided that the proposed method of comparison was not well defined.
After several discussions the Steering Committee selected an alternative approach that was
based on a robust validation method which did not rely on measurement of exhaust flow or raw
gaseous concentrations. This method was initially proposed by SwRI at the June 2006 Steering
Committee meeting in San Antonio. The proposed method had some similarity to the third
option proposed in the Test Plan, in that the deltas (PEMS vs. Lab) generated by CE-CERT were
to be compared with those generated by the Model. However, the method of comparison was
different. The key assumptions in using this method are listed below.
1. It was understood that CE-CERT could not measure torque directly and that no
reference torque would be available. This meant that the laboratory BS emission
values provided by CE-CERT were to use the same "torque-basis" as the PEMS
measurements.
2. It was assumed that CE-CERT would provide BS emission values for each on-road
NTE event by all three calculation methods for both PEMS and the mobile laboratory.
3. SwRI was to calculate the "deltas" between the PEMS and the CE-CERT laboratory
(i.e., PEMS - CE-CERT).
4. The CE-CERT data was to include both in-cab and on-frame mounted PEMS
measurements, but these were to be pooled together to provide a single data set.
5. When the Monte Carlo Model was run through the set of 195 reference NTE events,
two sets of results were to be generated. One set included BS emissions with all error
surface deltas applied, and a second set which included BS emissions with some of
the error surface deltas excluded (primarily those associated with torque, BSFC,
speed and fuel rate). These results were to be generated simultaneously during each
reference NTE model run. Essentially this yielded two Monte Carlo Model results, a
"Validation" result (used for the on-road validation) and a "Full Model" result (to be
used for the measurement allowance generation and for the lab replay validation).
SwRI Report 03.12024.06 Page 38 of 371
-------
The "Validation" result also included time alignment adjustment and checks for
periodic drift.
On-Road Validation Methodology
The Measurement Error Monte Carlo Simulation Model was validated by comparing the
simulation results using the data from the 195 reference NTE events to the on-road results using
the data from the 100 NTE events collected using the CE-CERT trailer and PEMS unit. This
was accomplished using the methodology described below.
1. Simulation Results
• The Monte Carlo Model was run using the data from the 195 reference NTE events. In
order to obtain Monte Carlo Model simulations representing similar conditions to those
obtained on-road, certain error surfaces needed to be suppressed in the simulations since
not all of them were applicable to the conditions used in collecting the on-road data. The
error surfaces excluded were all torque and BSFC error surfaces, dynamic speed and
dynamic fuel rate. This is the "Validation" result described earlier.
• For each reference NTE event, various percentiles, such as the 5th and 95th, of the
simulated distribution of the BS emissions differences, defined as
delta BS = BS emissions with "Validation" error - "Ideal" BS emissions,
were obtained. In essence the model produced a "distribution of deltas" for each NTE
event for all three calculation methods.
• The BS emissions included BSNMHC, BSCO, and BSNOX using all three calculation
methods. Thus, there were 9 sets of data (i.e., 3 emissions x 3 calculation methods).
• For each set of data and each percentile chosen in the study, the Monte Carlo Model
produced 195 BS delta values (i.e., one from each reference NTE event). These delta BS
values were ordered from smallest to largest, and then the empirical distribution function
(edf) of these delta BS values was plotted. The edf is a cumulative plot of the fraction of
the 195 delta BS observations that were less than or equal to x, versus each x, where x
was the observed BS delta value.
• For each of the nine sets of data (3 emissions and 3 calculation methods), the edf
corresponding to the 5th and 95th percentile distributions were plotted on the same plot.
Figure 14 contains an illustration of a plot of the edf for 5th and 95th percentiles. The region
between these two curves was designated as the validation region for comparison of the edf
obtained from the on-road data from CE-CERT.
SwRI Report 03.12024.06 Page 39 of 371
-------
1.0
0.8
(0
•§ 0.6
1 0.4
3
3
O
0.2
0.0
7
Model Simulation
5th %
Model Simulation
95th %
Delta BS Emissions
FIGURE 14. PLOT OF MODEL-GENERATED EMPIRICAL DISTRIBUTION
FUNCTIONS FOR TWO PERCENTILES
2. On-Road Results
• The CE-CERT trailer was driven on selected on-road routes to collect emissions data. In
addition, a PEMS installed in the tractor pulling the trailer collected emissions data.
From the routes driven with the CE-CERT trailer approximately 100 NTE events were
down-selected by the Steering Committee.
• For each on-road NTE event, a delta BS emissions value, defined as
delta BS emissions = PEMS BS emissions - CE-CERT BS emissions,
was computed.
• As before, the BS emissions included BSNMHC, BSCO, and BSNOX using all three
calculation methods. Thus, there were 9 sets of data (i.e., 3 emissions x 3 calculation
methods).
• For each set of data the delta BS values were ordered from smallest to largest and then
the empirical distribution function (edf) of these delta BS values were plotted on the same
plot as the matching simulation data.
The percentile edfs based on the "Validation" set of simulated data were used for comparison
with the edf obtained from the on-road data. Figure 15 contains an illustration of a plot of the
matching simulated and on-road edfs.
SwRI Report 03.12024.06
Page 40 of 371
-------
1.0
0.8
.Q
(0
| 0.6
ol
0.4
3
O
02
0.0
Model Simulation
5th %
Model Simulation
95th %
Delta BS Emissions
FIGURE 15. PLOT OF ON-ROAD AND MODEL-GENERATED EMPIRICAL
DISTRIBUTION FUNCTIONS
3. Comparison of Results
Several methods to compare the results of the model simulation deltas and the on-road
deltas for validation were presented to the Steering Committee at the June 2006 Steering
Committee meeting in San Antonio. At that meeting all the validation proposals were discussed
and a number of alternatives were presented.
Ultimately, the Steering Committee elected to proceed with the following method as a
validation methodology. From the on-road and model-generated empirical distribution functions
as shown in Figure 15, we would observe how many points of the on-road edf did not fall
between the points of the boundary edfs supplied by the simulation model. The Steering
Committee agreed that if at least 90% of the on-road data were within the 5th and 95th percentile
differences from the model simulation the model was considered valid for a particular BS
emissions and calculation method. However, if 10% of the on-road differences were outside the
model edfs either on the high or the low side, then the data would be investigated to try to
determine the cause. This analysis was performed independently for each pollutant and each
calculation method. This decision was later confirmed at the November 2006 Steering
Committee meeting in San Antonio.
SwRI Report 03.12024.06
Page 41 of 371
-------
It should be noted that none of these comparison methods had any effect on the final
generation of the Measurement Allowances. These were still generated using the 95th percentile
simulation results using the "Full Model."
Laboratory Replay Validation Methodology
The laboratory replay validation was not as well defined as the on-road validation in the
Test Plan, either in terms of testing scope or in terms of the validation methodology to be used
with the resulting data. According to the Test Plan, the laboratory replay was to involve
removing the engine from the test truck used by CE-CERT for the on-road validation, and using
that engine to "replay" some of the on-road testing episodes in the laboratory to the extent it was
possible to do so.
In the initial proposal for this program, SwRI established a planned level of effort for the
laboratory replay testing involving roughly one month of effort, based on initial discussions with
the Steering Committee and the limited details given in the Test Plan. This timeframe included
removal of the test engine from the truck, installation in the transient cell, cycle generation, cycle
tuning, and testing.
The final scope of the laboratory replay testing involved simulating one hour of operation
from each of the three test routes run by CE-CERT during the on-road validation testing. During
the course of the on-road validation exercise, personnel from Caterpillar were onsite with CE-
CERT in order to facilitate the recording of certain proprietary engine data channels from the
ECM. This was done in order to provide data to later assess the accuracy of the laboratory
replay simulation. However, this data was only successfully recorded during the portion of on-
road testing which was conducted with the PEMS mounted outside the truck cabin (i.e., "frame-
mounted" data). As a result, only the frame-mounted on-road operations were simulated during
the laboratory replay.
One hour of operation was selected from each route run by CE-CERT, with preference
generally given to hours of operation containing the highest frequency of NTE events. Each
hour of operation was the basis of an hour long test cycle, which was replayed in its entirety in
the laboratory. The data from this hour of operation was then divided into individual NTE
events via the standard entry and exit logic used throughout the program (i.e., evaluation on a 1
Hz basis). Successful replay operation was determined in close consultation with Caterpillar
personnel who aided in the interpretation of proprietary engine ECM data. After successful
replay operation was achieved, each cycle was repeated three times to generate a validation data
set.
Brake-specific deltas were determined using two different methods, which were
differentiated by the method of generating the laboratory brake-specific emissions levels for
comparison to PEMS generated values. The first method was to calculate the laboratory
reference values using the standard Laboratory Reference method for work calculation which
involved Laboratory measured engine Speed and engine Torque (via the test cell inline torque
meter). The PEMS values for each calculation method were compared to this Laboratory
SwRI Report 03.12024.06 Page 42 of 371
-------
Reference value to generate a delta. These are referred to as "full" deltas, since all error terms
are considered.
The second method was to generate deltas using a methodology similar to that used by
CE-CERT for the on-road validation data. In this method, the work term for each PEMS
calculation method was used for both the Laboratory data and the PEMS data. This essentially
eliminates any errors due to torque and BSFC measurement. A separate Laboratory value is
generated for each calculation method under this scenario. These values were compared for each
calculation method to generate another set of deltas. This second set of deltas is referred to as
"mass" deltas, because effectively only errors in the determination of emission mass rates are
considered.
The originally intended method for analysis of the replay data was to compare the "full"
deltas for the replay validation testing to the simulation deltas generated using the "Full Model."
However, the Steering Committee later decided that this comparison was not appropriate, due to
the fact that the laboratory replays were not able to test as wide a range of environmental
parameters as the on-road testing. Therefore, different percentiles from the model results needed
to be chosen to establish the proper validation window for the replay testing data. The validation
window from the "Full Model" result, which included environmental factors, would be too wide
for a proper validation.
An alternative method for the treatment of the replay data was discussed at the January
24th 2007 Steering Committee meeting. At that meeting, it was determined that the proper use
for the replay data was to examine the "incremental" errors arising from torque and BSFC
measurement errors which were not properly examined during the on-road validation, due to lack
of a reference torque measurement. Given this direction, SwRI determined an alternate method
of comparison, which was presented to the Steering Committee at the February 15th 2007
Steering Committee meeting, and is summarized below.
Monte Carlo Model Data
The model incremental deltas were determined by comparing the results of the Full
Model to the results of the Validation Model. This was done on an event-by-event basis for all
195 reference NTE events. In each case, a "work" delta was generated as follows:
Model,Work Model,Full Model,Validation
Replay Validation Data
As mentioned earlier, two deltas had been generated for each calculation method in
comparing the PEMS brake-specific values to the Laboratory values. The "full" deltas used the
Laboratory values generated using the lab measured torque as a basis for the work term. The
"mass" deltas used Laboratory values generated using the same work term as the associated
PEMS calculation method, essentially with any work differences cancelled out via calculation.
SwRI Report 03.12024.06 Page 43 of 371
-------
For each calculation method, an incremental "work" delta was generated by comparing its
associated "full" delta to the appropriate "mass" delta as follows:
replay,Work replay,Full replay,Mass
This calculation was performed individually for each replay NTE event and for each of
the three repeat runs.
Data Comparison
Initially, the model work deltas and the replay work deltas were plotted against the brake-
specific emission level for each event. For the model work deltas, the x-value was the ideal
brake-specific emission level for the reference NTE event in question. For the replay validation
work deltas, the x-value was the Laboratory Reference brake-specific emissions level calculated
using measured torque. The plots were initially examined to see if the replay validation work
deltas fell within the range of values produced by the model.
Assuming this initial assessment warranted a more rigorous comparison, the final replay
validation comparison would be made in a manner similar to that used for the on-road validation
data that was described above. An edf would be made using work deltas generated by the model
at both the 5th and 95th percentiles to generate a validation window. This would be compared to
an edf of the replay validation work deltas to determine of 90 percent or more of the replay data
fell within the validation window.
SwRI Report 03.12024.06 Page 44 of 371
-------
3.0 1065 PEMS AND LABORATORY AUDIT
3.1 Audit Objective
An initial task of the program was to audit the PEMS and dynamometer laboratory
according to 40 CFR Part 1065 Subpart D. The audit procedures were performed to insure the
equipment used for the In-Use Measurement Allowance Program met the minimum performance
requirements as regulated by the EPA.
3.2 Overview of 1065 Audit Activities
The list of audits to be conducted for both the laboratory and the PEMS was finalized by
the Steering Committee at the August, 2005 meeting in Ann Arbor, MI. Table 21 summarizes
the required audits for both the laboratory and PEMS instruments. Subsequent sections will
detail the results for the individual performance checks that were conducted as part of the audits,
as well as any corrective action taken as a result of those checks.
SwRI Report 03.12024.06 Page 45 of 371
-------
TABLE 21. 1065 AUDITS AND PERFORMANCE CHECKS REQUIRED FOR THE
MEASUREMENT ALLOWANCE PROGRAM
Description
Linearity
Torque Meter
Fuel Flow
Intake Flow
Exhaust Flow
CVS Verification
H2O Interference on CO2
H2O and CO2 Interference on CO
FID Optimization
Non-stoichiometric raw FID O2
Interference
Nonmethane cutter penetration fractions
CLD H2O and CO2 quench
NDUV HC and H2O Interference
Chiller NO2 penetration
NO2-to-NO converter check
CFR Reference
1065.307
1065.310
1065.320
1065.325
1065.330
1065.341
1065.350
1065.355
1065.360
1065.362
1065.365
1065.370
1065.372
1065.376
1065.378
Lab Raw
x1
X
X
X
X
X
x3
X
X
X
X
Lab Dilute
x1
X
X
X
X
x3
X
X
X
PEMS
x2
X
X
X
x3
X
X
X
1 Linearity for lab on gas analyzers, flow meters, torque meter, pressures, temperatures
2 Linearity for PEMS on gas analyzers, exhaust flow meters
3 Verify methane response factors only, THC instruments
3.2.1 Laboratory Audits
The results of the laboratory audit were presented to the Steering Committee at the
January, 2006 meeting in San Antonio, TX. The laboratory audit results indicated that the SwRI
reference laboratory met all of the requirements given under Part 1065 Subpart D. At that time,
the Steering Committee approved the laboratory audit results for both the raw and dilute
sampling systems, and SwRI was not directed to take any corrective actions for the laboratory.
Regular performance checks were performed throughout the program as required by Part
1065 Subpart D. However, only the results of the initial 1065 audit are included in this report.
Documentation of all regular performance checks is available at SwRI if needed.
SwRI Report 03.12024.06
Page 46 of 371
-------
3.2.2 PEMS Audits
The initial audits of the first four PEMS units (PEMS 1 through 4) were started in
January, 2006 and completed by mid-February, 2006. These initial audits included only the 5-
inch EFMs used for Engine 1 testing. Audits of the 4-inch and 3-inch EFMs were completed at a
later time, closer to the testing needs for Engines 2 and 3. The PEMS units were later modified
to address a 1065 NC>2 penetration check failure. PEMS 1 through 4 were audited again in June,
2006, once all modifications were completed. For these subsequent audits, only linearity and
NO2 penetration were checked.
Two additional Sensors Inc. PEMS units (PEMS 5 and 6) as well as a Horiba OBS-2200
arrived at SwRI in June, 2006. Upon arrival, PEMS 5 and 6 and the OBS-200 were given
complete 1065 audits as outlined in Table 21. A final PEMS unit (PEMS 7) arrived at SwRI in
October of 2006 to serve as a spare. PEMS 7 was given a complete 1065 audit at that time.
Additional PEMS linearity checks were performed as required by Part 1065 Subpart D
over the course of the program. In addition, a number of additional audits were required as a
result of maintenance or repairs performed on several of the PEMS units over the course of the
program. This report contains the results of initial audits on all PEMS units, as well as those
performed subsequent to the NO2 penetration modifications which were completed in June of
2006. In addition, relevant audit results are also given for any major repairs or maintenance
events which occurred on the PEMS equipment.
3.3 Gas Analyzer Linearity Verifications
Analyzer linearity checks were performed as specified in CFR Part 1065. The Federal
Register defines linearity in terms of the maximum concentration expected during testing.
Performing the PEMS and laboratory audits prior to engine testing, the maximum test
concentrations were unknown. Therefore, the mono-blend linearity verification gas
concentrations were used to define the 1065 linearity criteria. This interpretation of the
verification resulted in the most liberal linearity criteria. Mono-blend span gases were used with
a STEC Inc. Model SGD-710C 10-step gas divider, shown in Figure 16, to perform the PEMS
analyzer linearity verification. Span gas concentrations for the SEMTECH-DS and laboratory
analyzers were near the values listed in Table 22. Span concentrations for the PEMS were
selected based on manufacturer recommendations in the SEMTECH-DS user manual. All
linearity checks on the PEMS were performed using the PEMS span port.
SwRI Report 03.12024.06 Page 47 of 371
-------
FIGURE 16. STEC INC. MODEL SGD-710C GAS DIVIDER WITH SEMTECH-DS
PEMS
TABLE 22. SPAN CONCENTRATIONS USED FOR THE SEMTECH-DS AND
LABORATORY ANALYZERS
Analyzer Description
Dilute MEXA 7200D and
Horiba CH4 Bench
Raw MEXA 7200D and
Horiba CH4 Bench
SEMTECH-DS
NO
[ppm]
N/A
N/A
960
NO2
[ppm]
N/A
N/A
260
NOX
[ppm]
92
900
N/A
CO2
[%]
5.5
14.5
12
CO
[ppm]
47
47
960
THC
[ppmC]
9
9
660
CH4
[ppmC]
23
23
N/A
As shown in Table 23 and Table 24, the laboratory analyzers easily passed the 1065
linearity criteria. The MEXA benches use a Horiba GDC 703 gas divider and perform the
linearity checks in an automated process. The STEC Inc. manual gas divider was used to check
the dilute and raw CH4 benches. Linearity checks were performed monthly for all laboratory
analyzers during the program. The results of all monthly linearity checks are not included in this
report beyond those associated with the initial 1065 audit of the laboratory equipment.
SwRI Report 03.12024.06
Page 48 of 371
-------
TABLE 23. DILUTE MEXA 7200D AND HORIBA CH4 BENCH 1065 LINEARITY
VERIFICATION SUMMARY
Verification Description
Intercept
Slope
SEE
r2
CONDIR
Measured
Linearity Criteria
Pass / Fail
0.00
1.13
Pass
1.00
0.99-1.01
Pass
0.03
2.26
Pass
1.00
0.998
Pass
CO2 NDIR
Measured
Linearity Criteria
Pass / Fail
0.00
0.03
Pass
1.00
0.99-1.01
Pass
0.00
0.05
Pass
1.00
0.998
Pass
HCFID
Measured
Linearity Criteria
Pass / Fail
0.40
2.21
Pass
1.00
0.99-1.01
Pass
0.56
4.43
Pass
1.00
0.998
Pass
NOX CLD
Measured
Linearity Criteria
Pass / Fail
-0.04
1.36
Pass
1.00
0.99-1.01
Pass
0.12
2.72
Pass
1.00
0.998
Pass
CH4 FID with NMHC Cutter
Measured
Linearity Criteria
Pass / Fail
0.00
0.11
Pass
1.00
0.99-1.01
Pass
0.06
0.23
Pass
1.00
0.998
Pass
TABLE 24. RAW MEXA 7200D AND HORIBA CH4 BENCH 1065 LINEARITY
VERIFICATION SUMMARY
Verification Description
Intercept
Slope
SEE
r2
CONDIR
Measured
Linearity Criteria
Pass / Fail
0.80
4.52
Pass
1.00
0.99-1.01
Pass
2.25
9.04
Pass
1.00
0.998
Pass
CO2 NDIR
Measured
Linearity Criteria
Pass / Fail
-0.01
0.07
Pass
1.01
0.99-1.01
Pass
0.03
0.15
Pass
1.00
0.998
Pass
HCFID
Measured
Linearity Criteria
Pass / Fail
0.00
2.34
Pass
1.00
0.99-1.01
Pass
0.13
4.68
Pass
1.00
0.998
Pass
NOX CLD
Measured
Linearity Criteria
Pass / Fail
0.00
1.36
Pass
1.00
0.99-1.01
Pass
0.07
2.72
Pass
1.00
0.998
Pass
CH4 FID with NMHC Cutter
Measured
Linearity Criteria
Pass / Fail
0.00
0.11
Pass
1.00
0.99-1.01
Pass
0.06
0.23
Pass
1.00
0.998
Pass
SwRI Report 03.12024.06
Page 49 of 371
-------
Table 25 summarizes the linearity verifications performed on PEMS 1. During the initial
audit, PEMS 1 repeated failed the linearity check with NO and NO2. Because PEMS 1 was the
only unit to fail linearity with NO during the initial audit, the unit was returned to Sensors for
correction in accordance with the Test Plan. Sensors recalibrated the NO component of the
NDUV and sent PEMS 1 back to SwRI. The recalibrated unit passed the NO linearity check
PEMS 1 also failed the linearity check with NO2, as did many of the other PEMS during
the initial audit. All of these linearity failures involved the intercept being above the required
level. Sensors Inc. was offered a chance to correct the problem; however they declined,
indicating that they felt the units were operating correctly despite the intercept failures. Sensors
Inc. indicated that they felt there were problems with the 1065 linearity requirements as written,
and that widening the intercept linearity criteria should be considered. This was reported to the
Steering Committee during the February 23, 2006 conference call, and the decision was made to
continue testing as allowed in section 3.1.4 of the Test Plan.
During the June, 2006 linearity checks that followed the NO2 penetration upgrades,
PEMS 1 NDUV measurements became unstable and the instrument could not be zeroed or
spanned properly. Following subsequent diagnostics, the NDUV analyzer was replaced by
Sensors Inc. PEMS 1 passed the linearity checks for both NO and NO2 with the new analyzer.
SwRI Report 03.12024.06 Page 50 of 371
-------
TABLE 25. PEMS 1 1065 LINEARITY VERIFICATION SUMMARY
Verification Description
Intercept
Slope
SEE
r2
CO NDIR (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
0.00
4.52
Pass
1.00
0.99-1.01
Pass
0.00
9.04
Pass
1.00
0.998
Pass
CO NDIR (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
0.00
4.52
Pass
1.00
0.99-1.01
Pass
0.00
9.04
Pass
1.00
0.998
Pass
C02 NDIR (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
0.02
0.07
Pass
1.00
0.99-1.01
Pass
0.02
0.15
Pass
1.00
0.998
Pass
C02 NDIR (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
0.05
0.07
Pass
1.00
0.99-1.01
Pass
0.03
0.15
Pass
1.00
0.998
Pass
HC FID (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
-0.06
2.21
Pass
1.00
0.99-1.01
Pass
0.26
4.43
Pass
1.00
0.998
Pass
HC FID (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
-1.68
2.21
Pass
1.00
0.99-1.01
Pass
0.53
4.43
Pass
1.00
0.998
Pass
NO NDUV (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
8.25
4.42
Fail
0.98
0.99-1.01
Fail
4.46
8.84
Pass
1.00
0.998
Pass
NO NDUV (NO Recalibrated by Sensors Inc. 02-15-06)
Measured
Linearity Criteria
Pass / Fail
3.77
4.42
Pass
1.00
0.99-1.01
Pass
2.06
8.84
Pass
1.00
0.998
Pass
NO NDUV (New NDUV 06-07-06)
Measured
Linearity Criteria
Pass / Fail
3.75
4.42
Pass
0.99
0.99-1.01
Pass
2.09
8.84
Pass
1.00
0.998
Pass
N02 NDUV (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
2.15
1.42
Fail
1.00
0.99-1.01
Pass
0.92
2.83
Pass
1.00
0.998
Pass
N02 NDUV (New NDUV 06-07-06)
Measured
Linearity Criteria
Pass / Fail
-0.85
1.29
Pass
1.00
0.99-1.01
Pass
0.81
2.58
Pass
1.00
0.998
Pass
Table 26 shows the summarized linearity verification results for PEMS 2. PEMS 2
narrowly failed linearity for NC>2 during the initial audit. Per the steering committee's decision,
SwRI Report 03.12024.06
Page 51 of 371
-------
no action was taken to correct the linearity failure. During the June, 2006 audits that followed
the NC>2 penetration upgrades, PEMS 2 passed the NC>2 linearity check.
TABLE 26. PEMS 2 1065 LINEARITY VERIFICATION SUMMARY
Verification Description
Intercept
Slope
SEE
r2
CO NDIR (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
-0.91
4.52
Pass
0.99
0.99-1.01
Pass
4.96
9.04
Pass
1.00
0.998
Pass
CO NDIR (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
-2.27
4.52
Pass
1.01
0.99-1.01
Pass
2.75
9.04
Pass
1.00
0.998
Pass
CO2 NDIR (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
-0.02
0.07
Pass
1.00
0.99-1.01
Pass
0.02
0.15
Pass
1.00
0.998
Pass
CO2 NDIR (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
0.02
0.07
Pass
1.01
0.99-1.01
Pass
0.04
0.15
Pass
1.00
0.998
Pass
HC FID (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
0.98
2.21
Pass
1.00
0.99-1.01
Pass
0.73
4.43
Pass
1.00
0.998
Pass
HC FID (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
0.48
2.21
Pass
1.00
0.99-1.01
Pass
0.56
4.43
Pass
1.00
0.998
Pass
NO NDUV (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
3.65
4.42
Pass
1.00
0.99-1.01
Pass
2.91
8.84
Pass
1.00
0.998
Pass
NO NDUV (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
3.81
4.42
Pass
1.00
0.99-1.01
Pass
3.17
8.84
Pass
1.00
0.998
Pass
NO2 NDUV (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
2.10
1.42
Fail
1.01
0.99-1.01
Pass
1.88
2.83
Pass
1.00
0.998
Pass
NO2 NDUV (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
0.46
1.29
Pass
1.00
0.99-1.01
Pass
0.59
2.58
Pass
1.00
0.998
Pass
Table 27 summarizes the linearity verification results for PEMS 3. PEMS 3 also failed
linearity for NC>2 during the initial audit. As with PEMS 1, problems developed with the NDUV
SwRI Report 03.12024.06
Page 52 of 371
-------
during the June, 2006 audits that followed the NC>2 penetration upgrades, and the NDUV was
replaced by Sensors Inc. The new NDUV passed NC>2 linearity, but narrowly failed NO
linearity. Sensors did not elect to perform any corrective action on the unit as a result of this
failure. The NO linearity failure was reported to the Steering Committee during the regular
conference call on June 13th, 2006, and the decision was made to continue testing despite the
failure.
TABLE 27. PEMS 3 1065 LINEARITY VERIFICATION SUMMARY
Verification Description
Intercept
Slope
SEE
r2
CO NDIR (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
0.00
4.52
Pass
1.00
0.99-1.01
Pass
0.00
9.04
Pass
1.00
0.998
Pass
CO NDIR (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
-1.36
4.52
Pass
1.00
0.99-1.01
Pass
2.75
9.04
Pass
1.00
0.998
Pass
C02 NDIR (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
0.03
0.07
Pass
1.00
0.99-1.01
Pass
0.04
0.15
Pass
1.00
0.998
Pass
C02 NDIR (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
0.00
0.07
Pass
1.01
0.99-1.01
Pass
0.03
0.15
Pass
1.00
0.998
Pass
HC FID (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
0.60
2.21
Pass
1.00
0.99-1.01
Pass
0.51
4.43
Pass
1.00
0.998
Pass
HC FID (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
-0.96
2.21
Pass
1.00
0.99-1.01
Pass
0.27
4.43
Pass
1.00
0.998
Pass
NO NDUV (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
3.56
4.42
Pass
0.99
0.99-1.01
Pass
2.53
8.84
Pass
1.00
0.998
Pass
NO NDUV (New NDUV 06-06-06)
Measured
Linearity Criteria
Pass / Fail
5.70
4.42
Fail
1.00
0.99-1.01
Pass
2.02
8.84
Pass
1.00
0.998
Pass
N02 NDUV (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
7.38
1.42
Fail
0.98
0.99-1.01
Fail
1.29
2.83
Pass
1.00
0.998
Pass
N02 NDUV (New NDUV 06-06-06)
Measured
Linearity Criteria
Pass / Fail
0.73
1.29
Pass
1.01
0.99-1.01
Pass
1.17
2.58
Pass
1.00
0.998
Pass
SwRI Report 03.12024.06
Page 53 of 371
-------
Table 28 shows the summarized CO2, CO, and HC linearity verification results for PEMS
4. PEMS 4 passed the linearity check for CO2, CO, and HC during the initial audit as well as the
June 2006 checks that followed the NO2 penetration upgrades.
TABLE 28. PEMS 4 1065 LINEARITY VERIFICATION SUMMARY
Verification Description
Intercept
Slope
SEE
r2
CO NDIR (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
0.00
4.52
Pass
1.00
0.99-1.01
Pass
0.00
9.04
Pass
1.00
0.998
Pass
CO NDIR (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
-1.36
4.52
Pass
1.00
0.99-1.01
Pass
3.16
9.04
Pass
1.00
0.998
Pass
CO2 NDIR (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
-0.01
0.07
Pass
1.01
0.99-1.01
Pass
0.03
0.15
Pass
1.00
0.998
Pass
CO2 NDIR (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
0.02
0.07
Pass
1.01
0.99-1.01
Pass
0.04
0.15
Pass
1.00
0.998
Pass
HC FID (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
-0.22
2.21
Pass
1.00
0.99-1.01
Pass
0.26
4.43
Pass
1.00
0.998
Pass
HC FID (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
-1.11
2.21
Pass
1.00
0.99-1.01
Pass
0.36
4.43
Pass
1.00
0.998
Pass
Table 29 shows the PEMS 4 linearity check results for NO and NO2. PEMS 4 also failed
NO2 linearity during the initial audit and June, 2006 checks. As with PEMS 3, Sensors elected to
take no corrective action, and the Steering Committee elected to continue testing despite the
audit failure. During Engine 2 testing, the NDUV was replaced due to measurement stability
problems that eventually prevented proper zero and span operations. The new NDUV passed
NO linearity, but failed NO2 linearity. Shortly after installing the new NDUV, NO and NO2
measurements again became noisy and erratic. The NDUV was replaced again by Sensors Inc.,
after which NO and NO2 passed the linearity check.
SwRI Report 03.12024.06
Page 54 of 371
-------
TABLE 29. PEMS 4 1065 LINEARITY VERIFICATION SUMMARY CONTINUED
NO NDUV (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
2.54
4.42
Pass
0.99
0.99-1.01
Pass
1.93
8.84
Pass
1.00
0.998
Pass
NO NDUV (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
0.61
4.42
Pass
1.00
0.99-1.01
Pass
2.70
8.84
Pass
1.00
0.998
Pass
NO NDUV (New NDUV 09-14-06)
Measured
Linearity Criteria
Pass / Fail
0.36
4.41
Pass
1.00
0.99-1.01
Pass
2.81
8.83
Pass
1.00
0.998
Pass
N02 NDUV (New NDUV 09-25-06)
Measured
Linearity Criteria
Pass / Fail
3.55
4.41
Pass
0.99
0.99-1.01
Pass
2.30
8.83
Pass
1.00
0.998
Pass
N02 NDUV (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
3.46
1.42
Fail
1.01
0.99-1.01
Pass
3.37
2.83
Fail
1.00
0.998
Pass
N02 NDUV (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
4.60
1.29
Fail
0.99
0.99-1.01
Pass
2.60
2.58
Fail
1.00
0.998
Pass
N02 NDUV (New NDUV 09-14-06)
Measured
Linearity Criteria
Pass / Fail
-2.55
1.29
Fail
1.02
0.99-1.01
Fail
1.85
2.58
Pass
1.00
0.998
Pass
N02 NDUV (New NDUV 09-25-06)
Measured
Linearity Criteria
Pass / Fail
0.16
1.29
Pass
1.00
0.99-1.01
Pass
1.12
2.58
Pass
1.00
0.998
Pass
Shown in Table 30 are the linearity results for PEMS 5. No major repairs were
performed on this unit during the program, subsequent to the initial audits. During the initial
audit linearity checks, CO and CO2 measurement were unstable, so valid readings could not be
taken. The NDIR was replaced by Sensors Inc. PEMS 5 passed all linearity checks with the
exception of NO. No action was taken to correct the NO linearity failure with PEMS 5, and the
Steering Committee elected to continue testing with the unit. PEMS 5 was shipped to CE-CERT
and used during the in-use validation testing. Upon return to SwRI, analyzer linearity was
rechecked. Similar to the initial audit, NO failed the linearity check with high intercept. CO2
also repeatedly failed the linearity test with low intercept. Despite the linearity check failures,
PEMS 5 was used to perform the laboratory replay validation testing.
SwRI Report 03.12024.06
Page 55 of 371
-------
TABLE 30. PEMS 5 1065 LINEARITY VERIFICATION SUMMARY
Verification Description
Intercept
Slope
SEE
r2
CO NDIR (Initial Audit with New NDIR 06-07-06)
Measured
Linearity Criteria
Pass / Fail
0.00
4.52
Pass
1.00
0.99-1.01
Pass
0.00
9.04
Pass
1.00
0.998
Pass
CO NDIR (Returned from Ce-Cert 01-03-07)
Measured
Linearity Criteria
Pass / Fail
0.00
4.52
Pass
1.00
0.99-1.01
Pass
0.00
9.04
Pass
1.00
0.998
Pass
C02 NDIR (Initial Audit with New NDIR 06-07-06)
Measured
Linearity Criteria
Pass / Fail
-0.06
0.07
Pass
1.01
0.99-1.01
Pass
0.05
0.15
Pass
1.00
0.998
Pass
CO2 NDIR (Returned from Ce-Cert 01-03-07)
Measured
Linearity Criteria
Pass / Fail
-0.11
0.07
Fail
1.01
0.99-1.01
Pass
0.06
0.15
Pass
1.00
0.998
Pass
HC FID (Initial Audit 06-07-06)
Measured
Linearity Criteria
Pass / Fail
-0.41
2.21
Pass
1.00
0.99-1.01
Pass
0.53
4.43
Pass
1.00
0.998
Pass
HC FID (Returned from Ce-Cert 01-03-07)
Measured
Linearity Criteria
Pass / Fail
-1.18
2.21
Pass
1.00
0.99-1.01
Pass
0.46
4.43
Pass
1.00
0.998
Pass
NO NDUV (Initial Audit 06-07-06)
Measured
Linearity Criteria
Pass / Fail
6.65
4.42
Fail
0.99
0.99-1.01
Pass
2.50
8.84
Pass
1.00
0.998
Pass
NO NDUV (Returned from Ce-Cert 01-03-07)
Measured
Linearity Criteria
Pass / Fail
5.00
4.47
Fail
1.00
0.99-1.01
Pass
2.63
8.94
Pass
1.00
0.998
Pass
NO2 NDUV (Initial Audit 06-07-06)
Measured
Linearity Criteria
Pass / Fail
0.35
1.29
Pass
1.01
0.99-1.01
Pass
0.96
2.58
Pass
1.00
0.998
Pass
NO2 NDUV (Returned from Ce-Cert 01-03-07)
Measured
Linearity Criteria
Pass / Fail
0.81
1.23
Pass
1.01
0.99-1.01
Pass
1.45
2.45
Pass
1.00
0.998
Pass
Table 31 shows the summarized linearity results for PEMS 6. During the initial audit of
this unit, PEMS 6 failed NC>2 linearity. As with other similar failures, no corrective action was
taken by Sensors Inc., and the Steering Committee elected to proceed with testing using this unit.
During Engine 3 steady-state testing, PEMS 6 NOX measurements were biased low versus the
laboratory measurements. To determine the cause of the bias, additional NO and NO2 linearity
checks were performed, with both NO and NO2 passing the linearity tests. An addition linearity
SwRI Report 03.12024.06
Page 56 of 371
-------
check is also shown for the FID following a repair which was conducted following a failure
during environmental baseline testing.
TABLE 31. PEMS 6 1065 LINEARITY VERIFICATION SUMMARY
Verification Description
Intercept
Slope
SEE
r2
CO NDIR (Initial Audit 06-07-06)
Measured
Linearity Criteria
Pass / Fail
0.00
4.52
Pass
1.00
0.99-1.01
Pass
0.00
9.04
Pass
1.00
0.998
Pass
CO2 NDIR (Initial Audit 06-07-06)
Measured
Linearity Criteria
Pass / Fail
0.07
0.07
Pass
1.00
0.99-1.01
Pass
0.04
0.15
Pass
1.00
0.998
Pass
HC FID (Initial Audit 06-07-06)
Measured
Linearity Criteria
Pass / Fail
-0.05
2.21
Pass
1.00
0.99-1.01
Pass
0.52
4.43
Pass
1.00
0.998
Pass
HC FID (Repaired 08-14-06)
Measured
Linearity Criteria
Pass / Fail
-1.09
2.21
Pass
1.00
0.99-1.01
Pass
0.37
4.43
Pass
1.00
0.998
Pass
NO NDUV (Initial Audit 06-07-06)
Measured
Linearity Criteria
Pass / Fail
1.15
4.42
Pass
1.00
0.99-1.01
Pass
1.23
8.84
Pass
1.00
0.998
Pass
NO NDUV (Check 11-22-06)
Measured
Linearity Criteria
Pass / Fail
0.90
4.47
Pass
1.00
0.99-1.01
Pass
2.46
8.94
Pass
1.00
0.998
Pass
N02 NDUV (Initial Audit 06-07-06)
Measured
Linearity Criteria
Pass / Fail
1.95
1.29
Fail
1.01
0.99-1.01
Pass
1.68
2.58
Pass
1.00
0.998
Pass
N02 NDUV (Check 11-22-06)
Measured
Linearity Criteria
Pass / Fail
0.15
1.29
Pass
1.01
0.99-1.01
Pass
0.51
2.58
Pass
1.00
0.998
Pass
Linearity results for PEMS 7 are shown in Table 32. PEMS 7 arrived at SwRI late in the
project, and only initial audit results, performed in October of 2006, are shown for this unit.
PEMS 7 passed the linearity checks with all analyzers. It was understood that PEMS 7 was a
new unit that had been only recently received by EPA prior to shipment to SwRI.
SwRI Report 03.12024.06
Page 57 of 371
-------
TABLE 32. PEMS 7 1065 LINEARITY VERIFICATION SUMMARY
Verification Description
Intercept
Slope
SEE
r2
CO NDIR (Initial Audit 10-21-06)
Measured
Linearity Criteria
Pass / Fail
0.00
4.52
Pass
1.00
0.99-1.01
Pass
0.00
9.04
Pass
1.00
0.998
Pass
C02 NDIR (Initial Audit 10-21-06)
Measured
Linearity Criteria
Pass / Fail
0.02
0.07
Pass
1.00
0.99-1.01
Pass
0.05
0.15
Pass
1.00
0.998
Pass
HC FID (Initial Audit 10-21-06)
Measured
Linearity Criteria
Pass / Fail
-0.30
2.21
Pass
1.00
0.99-1.01
Pass
0.56
4.43
Pass
1.00
0.998
Pass
NO NDUV (Initial Audit 10-21-06)
Measured
Linearity Criteria
Pass / Fail
3.45
4.41
Pass
1.00
0.99-1.01
Pass
0.89
8.83
Pass
1.00
0.998
Pass
N02 NDUV (Initial Audit 10-21-06)
Measured
Linearity Criteria
Pass / Fail
0.24
1.29
Pass
1.00
0.99-1.01
Pass
0.51
2.58
Pass
1.00
0.998
Pass
Table 33 shows the linearity test results for the Horiba OBS-2200 PEMS unit. Mono-
blend span gases were used to check linearity. The gas concentrations used for the check were
14.67 % CO2, 904 ppm CO, 443 ppmC HC, and 892 ppm NOX. The OBS-2200 passed the 1065
linearity criteria for all gaseous emissions.
TABLE 33. HORIBA OBS-2200 1065 LINEARITY VERIFICATION SUMMARY
Verification Description
Intercept
Slope
SEE
r2
CO NDIR
Measured
Linearity Criteria
Pass / Fail
0.00
4.52
Pass
1.00
0.99-1.01
Pass
0.00
9.04
Pass
1.00
0.998
Pass
C02 NDIR
Measured
Linearity Criteria
Pass / Fail
0.00
0.07
Pass
1.00
0.99-1.01
Pass
0.01
0.15
Pass
1.00
0.998
Pass
HCFID
Measured
Linearity Criteria
Pass / Fail
0.05
2.21
Pass
1.00
0.99-1.01
Pass
0.49
4.43
Pass
1.00
0.998
Pass
NOX CLD
Measured
Linearity Criteria
Pass / Fail
-1.22
4.49
Pass
0.99
0.99-1.01
Pass
2.70
8.98
Pass
1.00
0.998
Pass
SwRI Report 03.12024.06
Page 58 of 371
-------
3.4 1065 Gas Analyzer Verifications
The results of gas analyzer performance checks not related to linearity verifications are
given in this section. In general, SwRI performed the analyzer audits as detailed in CFR Part
1065 Subpart D. A step-by-step description of each analyzer verification process as well as
discussion of the test results are presented in the following sections. In summary, all laboratory
instruments passed the verification tests. Similarly, the Horiba OBS-2200 PEMS unit passed all
verifications tests. PEMS 2 and PEMS 4 failed the non-stoichiometric raw exhaust FID O2
interference verifications. No corrective action was taken in regard to the FID O2 interference
test. All SEMTECH-DS PEMS units initially failed the chiller NO2 penetration check. A
system upgrade was implemented by Sensors Inc. after which all units passed the check. A
detailed account of the chiller NO2 penetration failure and subsequent actions are disused in the
chiller NO2 penetration section. The PEMS units passed all other 1065 verification tests
discussed below.
Generally, each of these verifications was performed once during the program, unless a
major instrument repair warranted an additional check.. The major exception was the NO2
chiller penetration check. The NO2 penetration test was repeated after an upgrade designed to
address the initial failure of all the PEMS, which is discussed in more detail below. The audit of
PEMS 5, 6, and 7 occurred after the implementation of the NO2 chiller penetration system
upgrade, therefore a chiller penetration failure was not documented for these units.
Several 1065 analyzer verification tests required the use of humidified and blended
gasses. SwRI therefore constructed a gas conditioning and blending cart, pictured in Figure 2, to
perform the PEMS and laboratory audits. Consisting of a heated bubbler, two flow meters, an
overflow system, a Vaisala dew point instrument, several thermocouples, heated rap, and various
valves and stainless steel connections, the humidification rig can control the dew point of a gas
blend up to 50 °C. The cart uses both a wet gas port and a dry gas port. Therefore, the cart can
overflow dry gas, wet gas, or a blend of a wet gas and dry gas. The gas blending feature is useful
when humidifying gases that are soluble in water, such as NO2. Many of the performance checks
involving humidified gases require that the gas be overflowed to the entry of the heated sample
line of either the laboratory emission bench or PEMS unit. The overflow procedure was done to
verify proper operation of sample handling and conditioning systems. Therefore, the cart was
built with the capacity to overflow gas to as many as two emission benches and/or PEMS units,
allowing for direct performance comparisons between units. This capability proved very useful
in diagnosing problems throughout the program.
SwRI Report 03.12024.06 Page 59 of 371
-------
FIGURE 17. SWRI GAS HUMIDIFICATION AND BLENDING CART
The results of the various 1065 performance checks for the laboratory analyzer benches
are summarize in Table 34 and Table 35. Results for the PEMS performance checks during the
initial audits are given in Table 36 through Table 43.
TABLE 34. DILUTE MEXA 7200D AND HORIBA NMHC BENCH 1065 ANALYZER
VERIFICATION SUMMARY
Verification Description
1065.350 H2O interference for CO2 NDIR [%]
1065.355 H2O and CO2 interference for CO NDIR [ppm]
1065.360 FID optimization (methane response)
1065.370 CO2 and H2O quench verification for NOX OLD [%]
1065.378 NO2-to-NO converter conversion [%]
1065.365 Nonmethane cutter penetration fractions [%]
Meas.
0.00%
0.4
1.12
-0.21%
97.3%
1 .8%
Verification
± 0.01%
± 4.0
N/A
± 2.00%
> 95%
< 2.0%
P/F
Pass
Pass
N/A
Pass
Pass
Pass
SwRI Report 03.12024.06
Page 60 of 371
-------
TABLE 35. RAW MEXA 7200D AND HORIBA NMHC BENCH 1065 ANALYZER
VERIFICATION SUMMARY
Verification Description
1065.350 H2O interference for CO2 NDIR [%]
1065.355 H2O and CO2 interference for CO NDIR [ppm]
1065.360 FID optimization (methane response)
1065.362 Non-stoichiometric FID O2 interference [%]
1065.370 CO2 and H2O quench verification for NOX CLD [%]
1065.378 NO2-to-NO converter conversion [%]
1065.362 Non-stoichiometric CH4 FID O2 interference [%]
1065.365 Nonmethane cutter penetration fractions [%]
Meas.
0.00%
1.4
1.15
-0.7%
-0.5%
96.7%
0.5%
1 .7%
Verification
± 0.07%
± 48.7
N/A
± 1 .5%
± 2.0%
> 95%
± 1 .5%
< 2.0%
P/F
Pass
Pass
N/A
Pass
Pass
Pass
Pass
Pass
TABLE 36. PEMS 1 1065 ANALYZER VERIFICATION SUMMARY
Verification Description
1065.350 H2O interference for CO2 NDIR [%]
1065.355 H2O and CO2 interference for CO NDIR [ppm]
1065.362 Non-stoichiometric FID O2 interference [%]
1065.372 HC and H2O interference for NOX NDUV [ppm]
1065.376 Chiller NO2 Penetration [%]
1065.376 Chiller NO2 Penetration [%] (Post Retrofit) (06-10-06)
Meas.
0.00%
21.8
-0.8%
-0.6
90.5%
100.7%
Verification
± 0.07%
± 48.7
± 1.5%
± 4.0
> 95%
> 95%
P/F
Pass
Pass
Pass
Pass
Fail
Pass
TABLE 37. PEMS 2 1065 ANALYZER VERIFICATION SUMMARY
Verification Description
1065.350 H2O interference forCO2 NDIR [%]
1065.355 H2O and CO2 interference for CO NDIR [ppm]
1065.362 Non-stoichiometric FID O2 interference [%]
1065.372 HC and H2O interference for NOX NDUV [ppm]
1065.376 Chiller NO2 Penetration [%]
1065.376 Chiller NO2 Penetration [%] (Post Retrofit) (06-10-06)
Meas.
0.00%
21.8
4.2%
0.6
89.0%
95.6%
Verification
± 0.07%
± 48.7
± 1 .5%
± 4.0
± 95%
> 95%
P/F
Pass
Pass
Fail
Pass
Fail
Pass
TABLE 38. PEMS 3 1065 ANALYZER VERIFICATION SUMMARY
Verification Description
1065.350 H2O interference forCO2 NDIR [%]
1065.355 H2O and CO2 interference for CO NDIR [ppm]
1065.362 Non-stoichiometric FID O2 interference [%]
1065.372 HC and H2O interference for NOX NDUV [ppm]
1065.376 Chiller NO2 Penetration [%]
1065.376 Chiller NO2 Penetration [%] (Post Retrofit) (06-10-06)
Meas.
0.00%
10.9
-0.1%
0.0
90.1%
100.6%
Verification
± 0.07%
± 48.7
± 1 .5%
± 4.0
± 95%
> 95%
P/F
Pass
Pass
Pass
Pass
Fail
Pass
SwRI Report 03.12024.06
Page 61 of 371
-------
TABLE 39. PEMS 4 1065 ANALYZER VERIFICATION SUMMARY
Verification Description
1065.350 H2O interference forCO2 NDIR [%]
1065.355 H2O and CO2 interference for CO NDIR [ppm]
1065.362 Non-stoichiometric FID O2 interference [%]
1065.372 HC and H2O interference for NOX NDUV [ppm]
1065.376 Chiller NO2 Penetration [%]
1065.376 Chiller NO2 Penetration [%] (Post Retrofit) (06-10-06)
1065.376 Chiller NO2 Penetration [%] (New NDUV) (09-25-06)
1065.376 Chiller NO2 Penetration [%] (RH Sensor) (09-28-06)
Meas.
0.01%
21.8
-0.5%
0.0
89.4%
101.2%
100.0%
98.1%
Verification
± 0.07%
± 48.7
± 1 .5%
± 4.0
± 95%
> 95%
> 95%
> 95%
P/F
Pass
Pass
Pass
Pass
Fail
Pass
Pass
Pass
TABLE 40. PEMS 5 1065 ANALYZER VERIFICATION SUMMARY
Verification Description
1065.350 H2O interference forCO2 NDIR [%]
1065.355 H2O and CO2 interference for CO NDIR [ppm]
1065.362 Non-stoichiometric FID O2 interference [%]
1065.372 HC and H2O interference for NOX NDUV [ppm]
1065.376 Chiller NO2 Penetration [%] (Post Retrofit) (06-10-06)
Meas.
0.00%
21.8
-0.1%
0.0
102.4%
Verification
± 0.07%
± 48.7
± 1 .5%
± 4.0
> 95%
P/F
Pass
Pass
Pass
Pass
Pass
TABLE 41. PEMS 6 1065 ANALYZER VERIFICATION SUMMARY
Verification Description
1065.350 H2O interference forCO2 NDIR [%]
1065.355 H2O and CO2 interference for CO NDIR [ppm]
1065.362 Non-stoichiometric FID O2 interference [%]
1065.372 HC and H2O interference for NOX NDUV [ppm]
1065.376 Chiller NO2 Penetration [%] (Post Retrofit) (06-10-06)
1065.376 Chiller NO2 Penetration [%] (RH Sensor) (9-15-06)
1065.376 Chiller NO2 Penetration [%] (RH Sensor) (9-21-06)
1065.376 Chiller NO2 Penetration [%] (RH Sensor) (9-28-06)
1065.376 Chiller NO2 Penetration [%] (Check) (11-27-06)
Meas.
0.00%
27.3
2.1%
0.0
95.7%
96.2%
100.0%
99.6%
99.0%
Verification
± 0.07%
± 48.7
± 1 .5%
± 4.0
> 95%
> 95%
> 95%
> 95%
> 95%
P/F
Pass
Pass
Fail
Pass
Pass
Pass
Pass
Pass
Pass
TABLE 42. PEMS 7 1065 ANALYZER VERIFICATION SUMMARY
Verification Description
1065.350 H2O interference forCO2 NDIR [%]
1065.355 H2O and CO2 interference for CO NDIR [ppm]
1065.362 Non-stoichiometric FID O2 interference [%]
1065.372 HC and H2O interference for NOX NDUV [ppm]
1065.376 Chiller NO2 Penetration [%] (Post Retrofit) (10-24-06)
Meas.
0.00%
11.0
1 .4%
1.6
96.8%
Verification
± 0.07%
± 48.7
± 1.5%
± 4.0
> 95%
P/F
Pass
Pass
Pass
Pass
Pass
SwRI Report 03.12024.06
Page 62 of 371
-------
TABLE 43. HORIBA OBS-2200 1065 ANALYZER VERIFICATION SUMMARY
Verification Description
1065.350 H20 interference for CO2 NDIR [%]
1065.355 H2O and CO2 interference for CO NDIR [ppm]
1065.362 Non-stoichiometric FID O2 interference [%]
1065.370 CO2 and H2O quench verification for NOx CLD [%]
1065.378 NO2-to-NO converter conversion [%]
Meas.
0.05%
4.8
-0.3%
-1 .4%
98.9%
Verification
± 0.07%
± 48.7
± 1 .5%
± 2.0%
± 95.0%
P/F
Pass
Pass
Pass
Pass
Pass
3.4.1 1065.350H2O Interference for CO2 NDIR
The CC>2 NDIR water interference check was performed on each of the PEMS units as
well as the laboratory dilute and raw analyzers. This check was performed to characterize CO2
interference caused by water when using a NDIR analyzer. The PEMS and laboratory analyzers
used sample dryers upstream of the NDIR analyzer and all passed this 1065 check.
To perform this verification, all analyzers were first zeroed and spanned as they would be
prior to an emissions test. Using the humidification rig, humidified zero air was overflowed to
the sample line of the analyzer. While maintaining the dew point of the zero air at 25 °C, the
response of the CC>2 NDIR analyzer was recorded. This recorded value was compared to ±2 %
of the lowest flow-weighted mean CO2 concentration expected during testing. The raw
verification value of ±0.07 % and dilute verification value of ±0.01 % were calculated using the
Detroit Diesel Series 60 engine and FTP engine cycle. Using the lowest flow-weighted CC>2
concentration provided the most stringent test, therefore verifying the analyzer performance
during all emissions tests.
3.4.2 1065.355 H2O and CO2 Interference for CO NDIR
The CO NDIR water and CO2 interference check was performed on each of the PEMS
units as well as the laboratory dilute and raw analyzers. This check was performed to
characterize CO interference caused by water and CO2 when using a NDIR analyzer. The PEMS
and laboratory analyzers all passed this 1065 check.
To perform this verification, all analyzers were first zeroed and spanned as they would be
prior to an emissions test. Using the humidification rig, humidified CO2 span gas was
overflowed to the sample line of the analyzer. While maintaining the dew point of the CO2 span
gas at 25 °C, the response of the CO NDIR analyzer was recorded. The recorded CO value was
multiplied by the highest flow-weighted mean CO2 concentration expected during testing, then
divided by the CO2 span gas concentration. For this check, the highest flow-weighted CO2
concentration provided the most stringent test. The CO2 value of 8 % was calculated using the
Detroit Diesel Series 60 engine and RMC engine cycle. The corrected CO concentration was
compared to ±2 % of the flow-weighted mean CO concentration expected at the standard. The
raw CO verification value of 48.7 ppm and dilute verification value of 4 ppm were calculated
using the DDC engine over a FTP heavy-duty transient cycle.
SwRI Report 03.12024.06
Page 63 of 371
-------
3.4.3 1065.3 60 FID Optimization Methane Response
The methane response factors were determined for the laboratory dilute and raw FID
analyzers. The FID analyzers were first zeroed and spanned as they would be prior to an
emissions test. Using a gas divider and methane span gas, the FID response to methane was
characterized over 10 evenly distributed points from near zero to span concentration. The
methane response factor was calculated by dividing the recorded FID response by the actual
methane concentration. The mean value of the 10 methane response factors was calculated. A
check was then performed to insure each of the 10 response factors was within ±2 % of the
mean.
3.4.4 1065.362 Non-stoichiometric Raw FID #2 Interference
The O2 raw FID interference check was performed on each of the PEMS units as well as
the raw laboratory THC FID and raw laboratory NMHC FID analyzers. This check was
performed to characterize C>2 interference when using a FID analyzer to measure raw exhaust
from a non-stoichiometric engine. PEMS 2 and PEMS 4 failed this 1065 check.
The first step performed during this test was to zero and span the analyzers. The FID
analyzers were then spanned using a propane span gas with balance nitrogen. Using a gas
divider, the propane in nitrogen span gas was cut with 20 % oxygen and sampled with the FID
analyzer. The FID response to the divided span gas was compared to the actual THC
concentration. For all analyzers except PEMS 2 and PEMS 4, the measurement concentration
was within 1.5 % of the actual concentration and therefore passed the interference check. By
spanning the analyzer with propane in nitrogen, and checking the analyzer with 20 % oxygen,
this verification insures the 62 interference is acceptable during typical diesel engine operation.
Per the Steering Committee's decision, no action was taken to remedy the C>2 raw FID
interference check failure of PEMS 2 and PEMS 4. The laboratory raw THC FID analyzer
initially failed this check. After the FID was re-optimized, the instrument passed the verification
test.
3.4.5 1065.365 Nonmethane Cutter Penetration Fractions
The nonmethane cutter (NMC) penetration verification was performed on the laboratory
raw and dilute methane analyzer benches, each of which employed a NMC. Both systems
passed the penetration check.
The instruments were spanned through the NMC using methane during testing and for
this performance check, therefore the methane penetration fraction was set to 1.0. For the
verification, the instruments were zeroed and spanned, after which ethane span gas was
introduced to the bench. The concentration of ethane (in ppmC) was near the methane span
value used during the check. The response of the NMC FID to ethane span gas was recorded.
The recorded value was divided by the ethane span gas concentration on a ppmC basis. This
fraction was less than 2 % and therefore passed the nonmethane cutter penetration check.
SwRI Report 03.12024.06 Page 64 of 371
-------
The dilute NMC bench initially failed the cutter penetration check. The temperature of
the NMC cutter oven was increased until the bench passed the penetration verification test.
3.4.6 1065.370 CO2 andH2O Quench Verification for NOx CLD
The NOX CLD water and CC>2 quench check was performed on the laboratory raw and
dilute analyzers. Both analyzers passed the quench check.
The CLD NOX analyzers were first zeroed and spanned as they would be prior to an
emissions test. The benches were then set to measure NO rather than total NOX. The NO span
gas was then sampled and the mean NO concentration was recorded. The NO span gas was then
humidified and the mean dry NO concentration as well as the water content of the gas was
recorded. The water quench was calculated by taking the difference between the dry and
humidified span gas measurements and correcting this value using the actual water content of the
span gas and the maximum water content expected during testing. The maximum water
concentration expected during testing was set to 12 % for the raw analyzer and 3.5 % for the
dilute analyzer. Because both raw and dilute CLD analyzers were operated in a dry mode during
this program, both analyzers showed negligible water quench, indicating that the drying systems
in the benches were able to successfully remove the water from the sample.
Using a gas divider, the CLD CO2 quench was determined by measuring a blend of 50 %
NO span gas and 50 % nitrogen. Next, 50 % NO span gas and 50 % CO2 span gas was
measured. The CO2 quench was calculated by taking the difference between the 50 % nitrogen
blend and the 50 % CO2 span gas blend. The quench value was then corrected using the CO2
concentration recorded during the test and the maximum CO2 concentration expected during
testing. The maximum CO2 concentration expected during testing was set to 10 % for the raw
analyzer and 2.2 % for the dilute analyzer. The combined water and CO2 quench for both
analyzers was less than 2 %, therefore passing the CLD quench verification test.
3.4.7 1065.372 HC andH2O Interference for NOX NDUV
The NOX NDUV water and HC interference check was performed on each of the PEMS
units. All PEMS devices passed this interference verification test.
The PEMS were first zeroed and spanned as they would be prior to an emissions test.
Next, a blend of humidified zero air and dry propane span gas were overflowed to the sample
line of the PEMS. The dew point of the gas mixture was maintained at 45 °C during these tests.
Allowing time for stabilization, the NO, NO2, and HC concentration values were recorded. The
NO and NO2 concentrations were added, and the resulting response was then adjusted to the
level to the flow-weighted mean HC concentration expected at the standard. The mean HC
concentration of 51 ppm was calculated using the DDC Series 60 engine over a FTP transient
cycle. The verification concentration was calculated as ±2 % of the flow-weighted mean NOX
concentration expected at the standard. The mean NOX concentration of 198 ppm was calculated
using the DDC Series 60 engine over a FTP transient cycle. All PEMS showed little water and
HC interference for the NO and NO2 measurements, and easily passed the 1065 interference
verification test.
SwRI Report 03.12024.06 Page 65 of 371
-------
3.4.8 1065.376 Chiller NO2 Penetration
The SEMTECH-DS PEMS uses a chiller to dry the exhaust sample prior to the NDUV
detector, but does not use a NO2-to-NO converter. Therefore, the chiller NO2 penetration check
was performed. Initially, SwRI performed the chiller penetration check using a procedure
similar to that performed by Sensors Inc. After the PEMS were zeroed and spanned, wet zero air
with a dew point of approximately 50 °C was overflowed to the sample line and sampled for 15
to 20 minutes. Next, dry NO2 span gas was overflowed to the sample line. Allowing time for
stabilization, the NO2 concentration was recorded and compared to the NO2 span gas bottle
concentration. The units initially read approximately 90 % of the NO2 span concentration, and
failed the verification criteria of 95 % penetration. Although this initial procedure was
successful in revealing problems with NO2 penetration, the method was less than ideal. For
example, following the switch from humidified zero air to dry NO2 span gas, the sampling
system of the PEMS is continually drying. Although this drying process is slow, the NO2
concentration does rise over time, making a stable measurement difficult to achieve.
As a result, a revised method was devised to perform the chiller NO2 penetration check,
required the use of the 1065 compliant laboratory CLD NOX analyzer as a reference. Humidified
zero air was blended with dry NO2 span gas. The blend was adjusted to maintain a mixture dew
point of approximately 45 °C with a NO2 concentration near the span concentration. The
humidified NO2 mixture was then overflowed simultaneously to both the PEMS and the
laboratory CLD NOx analyzer. The CLD NOX concentration was used as the reference in
calculating the NO2 penetration. It was felt that the CLD could serve as an appropriate reference
value for this check, due to the fact that the laboratory CLDs did not show significant water
quench (since they are run dry). In addition, the NO2-to-NO conversion efficiency was in excess
of 97% at concentrations well above those used for the chiller NO2 penetration check.
This method more accurately simulates in-use measurement, because the sample is
continuously humidified. The CLD-based penetration check method generated chiller
penetration results similar to method used by Sensors Inc., but resulted in more stable and
accurate values.
As discussed earlier, all of the PEMS failed the chiller NO2 penetration check initially.
Additional penetration checks were performed at varying concentrations of NO2. These
experiments revealed a trend of increasing NO2 loss with increased NO2 concentration. This
trend is illustrated in Figure 18. It should be noted that the NO2 span value used for testing was
near 300 ppm.
SwRI Report 03.12024.06 Page 66 of 371
-------
98%
96%
94%
92%
90%
88%
86%
50
100 150 200 250
NO2 Span Concentration [ppm]
300
350
FIGURE 18. NO2 CHILLER PENETRATION AUDIT RESULTS
These audit results were presented to the Steering Committee at the March 14th meeting
in Ann Arbor, MI. Although there was concern about these results, the Steering Committee
elected to run steady-state tests on Engine 1, to examine whether the performance check results
would translate into an observed negative bias in the test results.
The initial steady-state results for Engine 1 were presented to the Steering Committee at
the April 13, 2006 meeting in San Antonio. Figure 19 shows a summary the original steady-state
NOX concentration pooled delta data from the Detroit Diesel Series 60 engine. The data shows a
definite negative NOX bias for the PEMS at higher concentrations. This was due in part to the
higher fraction of NC>2 in the overall NOX which occur due to the use of a catalyzed DPF in the
exhaust stream.
SwRI Report 03.12024.06
Page 67 of 371
-------
I PEMS 5th % A PEMS 50th % • PEMS 95th % • Points
10
5
Q.
3 -5
c
o
"a
~ -10
HI
o
c
o
O -15
-20
-25
-30
. »40
9 15
'24
--3--
33
160
180
200 220 240 260
280
300 3:
Lab Reference Mean NOx Concentration (ppm)
FIGURE 19. NOX CONCENTRATION POOLED DELTA DATA FROM ENGINE 1
STEADY-STATE REPEAT TESTING PRIOR TO CHILLER NO2 PENETRATION
RETROFIT INSTALLATION
The Steering Committee deemed the low NOX bias unacceptable for continued testing.
Sensors Inc. was asked to develop an upgrade to correct the issue in as timely a manner as
possible. However, it was also stipulated that the upgrade had to be acceptable as a real-world
solution, and that the upgrade could be applied to all existing SEMTECH DS units.
Following the direction of the Steering Committee, Sensors Inc. developed a system
upgrade to estimate the chiller NC>2 loss and numerically correct the NC>2 measurement.
Upgrades were completed on all PEMS used during the measurement allowance program by the
end of May, 2006. The upgrade package includes a drain manifold relative humidity sensor and
software upgrade. After Sensors Inc. implemented the upgrade for the SEMTECH-DS, all units
passed the NC>2 penetration check. Details of the upgrade are not included in this report for
reasons of confidentiality.
After implementation of the NC>2 chiller penetration system upgrade, the Steering
Committee elected to repeat the steady-state testing for Engine 1. While these results are
summarized later in greater detail, a summary is given in Figure 20 to illustrate the effect of the
upgrade on the data. The figure shows the pooled NOX delta data with upgraded SEMTECH-DS
units. The negative bias of the original data set was replaced with pooled errors showing a slight
positive bias.
SwRI Report 03.12024.06
Page 68 of 371
-------
I PEMS 5th % A PEMS 50th % • PEMS 95th % • Points
30
25
20
a.
3 15 H
c
o
"a
r 10 H
c
o
o
5
-5
-10
.40
9__
15
2Y
33
.
160
180 200
220
240
260
280
300
Lab Reference Mean NOx Concentration (ppm)
FIGURE 20. NOX CONCENTRATION POOLED DELTA DATA FROM ENGINE 1
STEADY-STATE REPEAT TESTING AFTER CHILLER NO2 PENETRATION
RETROFIT INSTALLATION
The NC>2 penetration check failure and subsequent PEMS upgrade resulted in a schedule
delay of more than two months in the execution of the program.
During dynamometer testing on Engine 2, PEMS 4 and PEMS 6 began reporting several
faults stating the drain manifold relative humidity sensor was not responding. To remedy this
problem, several drain manifold humidity sensors were replaced, after which the NC>2 penetration
check was repeated. Sensors Inc. linked the frequent failure of the drain relative humidity
sensors to drain manifolds that were allowing exhaust gas to leak past the sensor. The leaking
exhaust gas carried liquid water past and onto the humidity sensors, causing the sensors to fail.
A simple leak check was used to screen for properly sealed new RH sensors, which were
installed in the PEMS. The PEMS passed all penetration checks after the new sensors and drain
manifolds were installed.
3.4.9 1 065. 3 78 NO2-to-NO Converter Conversion
The NO2-to-NO converter conversion verification was performed on the laboratory raw
and dilute benches. This check was performed using the automated Horiba bench software and
Horiba GDC-703 gas divider, which is also capable of performing the NOX converter check.
Both raw and dilute NCVto-NO converters had conversion efficiencies greater than 95 %, and
therefore passed this verification test.
SwRI Report 03.12024.06
Page 69 of 371
-------
3.5 1065 Exhaust Flow Meter Linearity Verification
The Sensors Inc. electronic flow meters (EFM) and Horiba flow meter were checked for
flow linearity using a flow calibration stand at SwRI. This flow stand incorporates a set of
reference laminar flow elements (LFEs), which are regularly sent for verification of NIST
traceability at CEESI. The SwRI flow stand is pictured in Figure 21. The flow stand uses a
positive displacement blower to pull air through the stand, therefore, the reference LFE and EFM
are under a slight negative pressure during testing. The reference meters are downstream of the
meter that is being calibrated. In the case of the Sensors Inc. EFMs, a length of straight pipe
matching the diameter of the EFM was installed upstream of the EFM. The flow stand
incorporates long lengths of straight pipe, well in excess of 10 diameters, between the two flow
meters, as well as downstream of the reference meter. The stand is designed in this manner
because most calibrations at SwRI focus on intake air measurement. Several manually
controlled flow restriction devices, located far downstream of the reference meter, are used to set
the desired flow rates during the linearity check. High precision mercury manometers are read
manually to record the LFE differential and inlet pressure, while a thermocouple is used to
measure the LFE inlet temperature.
The SwRI flow stand was also used to calibrate the intake air LFEs used to calculate the
laboratory reference raw exhaust flow. The raw exhaust flow rate was also checked during
testing by calculating a carbon balance fuel flow, using raw gaseous measurements and the raw
exhaust mass rate, and comparing to the measured fuel flow mass rate. During all steady-state
testing, the raw carbon balance error was generally less than 2 %.
FIGURE 21. SWRI LFE FLOW STAND MANOMETERS AND REFERENCE LFES
SwRI Report 03.12024.06
Page 70 of 371
-------
Sensors Inc. uses a similar LFE-based flow stand to calibrate their electronic flow meters.
However, the Sensors flow stand uses a blower to push air through the reference flow meter and
then through the test flow meter which is located downstream and is then vented directly to
atmosphere. Therefore, the flow meters calibrated at Sensors, are under slight positive pressure.
According to Sensors, the stand was designed in order to accurately simulate field conditions of
an exhaust tailpipe. This discrepancy between the SwRI and Sensors Inc. flow stands may have
been a factor contributing to increased exhaust flow errors during steady-state testing on Engines
2 and 3, which used the 4-inch and 3-inch flow meters respectively. However, this could not be
verified, and the issue did not manifest itself with the 5-inch flow meters.
At SwRI, the Sensors Inc. and Horiba flow meters were mounted inline with the
reference LFE as shown in Figure 22. A straight pipe, with length exceeding 10 diameters, was
connected to the inlet of the flow meters. The EFM flow was recorded using the Sensors Inc.
software. Data markers with the Sensors Inc. post processor software were used to average at
least 30 seconds of data at each flow rate.
FIGURE 22. SENSORS INC. EFM MOUNTED ON THE SWRI LFE FLOW STAND
The collected flow data was processed using the 1065 linearity verifications for a raw
exhaust measurement system. The sections below describe the calibration events and linearity
results for the 5-inch, 4-inch, and 3-inch flow meters.
3.5.1 Five-Inch Exhaust Flow Meter Linearity
As shown in Table 44, two 5-inch Sensors Inc. flow meters passed the linearity check,
while one 5-inch flow meter repeatedly failed the check. Per the Steering Committee's decision,
the failed 5-inch EFM was sent to Sensors Inc. where it was recalibrated using the Sensors Inc.
SwRI Report 03.12024.06
Page 71 of 371
-------
flow stand. The 5-inch flow meter was then tested at SwRI where it passed the 1065 linearity
check. These results appeared to indicate good agreement between the SwRI and Sensors flow
stands.
TABLE 44. 5-INCH SENSORS INC. EFM LINEARITY RESULTS
Verification Description
Intercept
Slope
SEE
r2
H05-SE05 -Initial, Test 1
Measured
Linearity Criteria
Pass / Fail
5.07
9.42
Pass
0.96
0.98-1.02
Fail
1.93
18.85
Pass
1.00
0.99
Pass
H05-SE05- Initial, Test 2
Measured
Linearity Criteria
Pass/ Fail
6.45
9.61
Pass
0.96
0.98-1.02
Fail
4.08
19.22
Pass
1.00
0.99
Pass
H05-SE03- Initial
Measured
Linearity Criteria
Pass/ Fail
1.60
9.54
Pass
1.00
0.98-1.02
Pass
4.50
19.09
Pass
1.00
0.99
Pass
I05-SE05- Initial
Measured
Linearity Criteria
Pass/ Fail
4.93
9.44
Pass
1.01
0.98-1.02
Pass
4.87
18.87
Pass
1.00
0.99
Pass
H05-SE05 - Test 1 After Recalibration at Sensors
Measured
Linearity Criteria
Pass/ Fail
-1.69
9.28
Pass
1.01
0.98-1.02
Pass
1.60
18.57
Pass
1.00
0.99
Pass
Table 45 shows the linearity results for the 5-inch Horiba exhaust flow meter. The
Horiba meter, also based on pitot tube technology, showed excellent correlation with the SwRI
flow stand; easily passing the 1065 linearity criteria.
TABLE 45. 5-INCH HORIBA EXHAUST FLOW METER LINEARITY RESULTS
Verification Description
Intercept
Slope
SEE
r2
Horiba 5-Inch Exhaust Flow Meter S/N: 050702G2
Measured
Linearity Criteria
Pass / Fail
0.81
9.58
Pass
1.00
0.98-1.02
Pass
6.45
19.17
Pass
1.00
0.99
Pass
Figure 23 shows the pooled delta data for the Sensors Inc. 5-inch flow meters versus the
laboratory calculated exhaust flow for the Detroit Diesel Series 60 engine during steady-state
repeat testing. Reporting median flow measurement deltas less than 2 % of point, the 5-inch
flow meters showed good agreement with the laboratory exhaust flow measurement. The
Sensors Inc. 5-inch EFM is rated at flows as high as 1700 scfm, therefore we were testing the
meter only in its mid to lower range.
SwRI Report 03.12024.06
Page 72 of 371
-------
30
25
20
— 15
-2- 10
5
1!
-5
-10
-15
-20
IPEMS 5th % A PEMS 50th % * PEMS 95th % • Points
40
37
27
24
33
9-
250
350
450
550
650
750
850
950
1050
Lab Reference Exhaust Flowrate (scfm)
FIGURE 23. 5-INCH SENSORS EFM EXHAUST FLOW POOLED DELTA DATA
FROM ENGINE 1 STEADY-STATE REPEAT TESTING
Because the Horiba OBS-2200 was delivered to SwRI late in the program, it was tested
only with Engine 3. Per the recommendation of Horiba, the 5-inch flow meter was used during
testing with the International VT365 engine. Figure 24 shows pooled delta data for the Horiba 5-
inch flow meter versus the laboratory calculated exhaust flow during steady-state repeat testing.
The Horiba flow meter showed good correlation with the SwRI calculated exhaust flows, with
median deltas less than 0.5 % of point.
SwRI Report 03.12024.06
Page 73 of 371
-------
IPEMS 5th % A PEMS 50th % » PEMS 95th % « Points
20
15
10
5
0
1
-5
-10
-15
-20
-25
-30
40
^
37
35 30
22
_2CL
-13-
-16-
A 200
A A
250
300
350
400
Lab Reference Exhaust Flowrate (scfm)
FIGURE 24. 5-INCH HORIBA EXHAUST FLOW METER POOLED DELTA DATA
FROM ENGINE 3 STEADY-STATE REPEAT TESTING
3.5.2 Four-Inch Exhaust Flow Meter Linearity
Table 46 shows the linearity test results for the 4-inch flow meters in chronological order.
During the initial 1065 linearity checks, all of the 4-inch flow meters failed the verification
criteria with low slope values for the regression lines, with an averaging regression line slope of
0.94. As a result, the 4-inch flow meters where sent to Sensors Inc. for recalibration. After
recalibration at Sensors Inc., one 4-inch flow meter, serial number H05-SE07, was returned to
SwRI to re-check linearity. The flow meter again failed the linearity check with low slope, at
roughly 0.97 on average. SwRI and Sensors performed a considerable number of diagnostic tests
and checks in an attempt to determine the cause of the apparent discrepancy between the two
flow stands. All tests with the SwRI LFE flow stand failed the linearity check with low slope.
An attempt was made to check linearity in actual engine exhaust from Engine 1 (14L
DDC Series 60), using the SwRI exhaust flow measurement (from the sum of intake air and fuel
flows) as the reference. It should be noted that the 4-inch flow meter was smaller than those
normally used for this engine. The engine check failed linearity, but with a high slope.
However, this was due in part to large negative errors at the low end of the flow range, which
caused the intercept to also fail by a wide margin.
The collection of 4-inch flow meter data was presented to the Steering Committee at the
July 27, 2006 meeting in Ann Arbor. The decision was made to recalibrate the 4-inch flow
meters using the SwRI flow stand data. SwRI ran 15-point curves as requested by Sensors Inc.
SwRI Report 03.12024.06
Page 74 of 371
-------
Using the SwRI data, Sensors Inc. supplied new calibration constants for the meters. As
expected, all 4-inch flow meters passed the 1065 linearity checks after they were calibrated and
checked using the SwRI flow stand.
TABLE 46. 4-INCH SENSORS INC. EFM LINEARITY RESULTS
Verification Description
Intercept
Slope
SEE
r2
H05-SE07 -Initial, Test 1
Measured
Linearity Criteria
Pass / Fail
7.28
9.25
Pass
0.95
0.98-1.02
Fail
5.99
18.51
Pass
1.00
0.99
Pass
H05-SE07- Initial, Test 2
Measured
Linearity Criteria
Pass / Fail
8.77
9.18
Pass
0.95
0.98-1.02
Fail
6.93
18.36
Pass
1.00
0.99
Pass
I05-SE03 -Initial, Test 1
Measured
Linearity Criteria
Pass/ Fail
4.09
9.30
Pass
0.98
0.98-1.02
Fail
6.49
18.60
Pass
1.00
0.99
Pass
I05-SE03- Initial, Test 2
Measured
Linearity Criteria
Pass / Fail
3.06
9.24
Pass
0.96
0.98-1.02
Fail
6.61
18.49
Pass
1.00
0.99
Pass
I05-SE01 -Initial, Test 1
Measured
Linearity Criteria
Pass / Fail
21.47
9.26
Fail
0.92
0.98-1.02
Fail
14.56
18.52
Pass
1.00
0.99
Pass
H05-SE07 - Test 1 After Recalibration at Sensors
Measured
Linearity Criteria
Pass / Fail
-8.11
7.31
Fail
0.98
0.98-1.02
Fail
6.42
14.61
Pass
1.00
0.99
Pass
H05-SE07 - Test 3 After Recalibration (check on DDC Series 60 in exhaust)
Measured
Linearity Criteria
Pass / Fail
-25.74
10.26
Fail
1.03
0.98-1.02
Fail
7.26
20.52
Pass
1.00
0.99
Pass
H05-SE07 - Test 4 After Recalibration (15-point calibration data generation)
Measured
Linearity Criteria
Pass / Fail
1.60
7.69
Pass
0.96
0.98-1.02
Fail
5.28
15.39
Pass
1.00
0.99
Pass
H05-SE07 - Test 5 (EFM calibrated using SwRI data)
Measured
Linearity Criteria
Pass/ Fail
1.87
7.32
Pass
1.00
0.98-1.02
Pass
11.70
14.65
Pass
1.00
0.99
Pass
I05-SE03 - EFM calibrated using SwRI data
Measured
Linearity Criteria
Pass/ Fail
-3.93
7.43
Pass
1.00
0.98-1.02
Pass
6.40
14.87
Pass
1.00
0.99
Pass
I05-SE01 - EFM calibrated using SwRI data
Measured
Linearity Criteria
Pass/ Fail
-3.93
7.43
Pass
1.00
0.98-1.02
Pass
6.40
14.87
Pass
1.00
0.99
Pass
SwRI Report 03.12024.06
Page 75 of 371
-------
Figure 25 shows the pooled delta data for the Sensors Inc. 4-inch flow meters versus the
laboratory calculated exhaust flow for the Caterpillar C9 engine during steady-state repeat
testing. The 4-inch flow meter showed a trend of increasing error as exhaust flow rate increased.
The median flow rate delta was near 5 % of point at the highest measured flow. Because Sensors
Inc. effectively increased the slope of their EFMs when calibrating to the SwRI flow stand data,
the observed engine deltas would likely have been smaller had the flow meters used the original
Sensors Inc. calibration. As discussed earlier, the differences in the calibrations between Sensors
Inc. and SwRI may be linked to the different designs of the two flow stands; however, the final
reason for the discrepancies is not know at this time. The Sensors Inc. 4-inch EFM is rated at
flows as high as 1100 scfm, therefore SwRI tested the meter over a broad range relative to the
maximum flow rate.
I PEMS 5th % A PEMS 50th % • PEMS 95th % • Points
60
50
40
£ 30
ra
^
u.
•K 20
10
-10
40
39
36
-34-
21
13
10
:
250
350
450
550
650
750
Lab Reference Exhaust Flowrate (scfm)
FIGURE 25. 4-INCH EFM EXHAUST FLOW POOLED DELTA DATA FROM ENGINE
2 STEADY-STATE REPEAT TESTING
3.5.3 Three-Inch Exhaust Flow Meter Linearity
The Sensors Inc. 3-inch EFM is rated at flows as high as 600 scfm, and was used on
Engine 3 during the program. On an initial set of linearity checks performed in January, 2006,
all of the 3-inch flow meters failed with positive slopes of 1.04 on average. The 3-inch flow
meters were all returned to Sensors Inc. for recalibration. When returned to SwRI, the 3-inch
EFMs failed linearity with a low slope of 0.96 on average. Table 47 summarizes the 3-inch EFM
linearity data in chronological order. One 3-inch flow meter, serial number H05-SE06, was
replaced by Sensors Inc. due to its outlying, low slope. Based on experiences with the 4-inch
flow meters, the Steering Committee elected to recalibrate the 3-inch flow meters using data
generated at SwRI. The 3-inch flow meters passed the linearity check after recalibration at
SwRI. The resulting calibration increased the slope of each regression line by 4 to 5 percent. It
SwRI Report 03.12024.06
Page 76 of 371
-------
should be noted that the final linearity checks indicated a slight positive bias of roughly 1 percent
on average, with one of the flow meters nearly failing linearity with a high slope.
TABLE 47. 3-INCH SENSORS INC. EFM LINEARITY RESULTS
Verification Description
Intercept
Slope
SEE
r2
H05-SE04 - Initial Test 1 (Jan, 2006)
Measured
Linearity Criteria
Pass / Fail
-12.54
5.91
Fail
1.04
0.98-1.02
Fail
8.68
11.82
Pass
1.00
0.99
Pass
H05-SE04 - Initial Test 2 (with Straight Pipe)
Measured
Linearity Criteria
Pass / Fail
-11.89
4.73
Fail
H05-SE04 - Initial 15-point w/ Straight Pipe
Measured
Linearity Criteria
Pass / Fail
-17.86
6.49
Fail
1.03
0.98-1.02
Fail
5.88
9.47
Pass
1.00
0.99
Pass
Jan, 2006)
1.05
0.98-1.02
Fail
14.55
12.99
Fail
1.00
0.99
Pass
H05-SE04 - Test 1 After Recalibration
Measured
Linearity Criteria
Pass/ Fail
1.67
6.37
Pass
0.95
0.98-1.02
Fail
8.10
12.74
Pass
1.00
0.99
Pass
H05-SE06 - Test 1 After Recalibration (EFM was replaced due to low slope)
Measured
Linearity Criteria
Pass/ Fail
4.01
6.77
Pass
0.90
0.98-1.02
Fail
4.23
13.54
Pass
1.00
0.99
Pass
I05-SE06 - Test 1 After Recalibration
Measured
Linearity Criteria
Pass/ Fail
1.90
6.27
Pass
0.97
0.98-1.02
Fail
7.09
12.54
Pass
1.00
0.99
Pass
H05-SE04 - EFM calibrated using SwRI data
Measured
Linearity Criteria
Pass/ Fail
3.23
5.97
Pass
1.02
0.98-1.02
Pass
5.99
11.93
Pass
1.00
0.99
Pass
I05-SE06 - EFM calibrated using SwRI data
Measured
Linearity Criteria
Pass/ Fail
4.93
9.44
Pass
1.01
0.98-1.02
Pass
4.87
18.87
Pass
1.00
0.99
Pass
I06-SE04 - EFM calibrated using SwRI data
Measured
Linearity Criteria
Pass/ Fail
4.89
6.01
Pass
1.00
0.98-1.02
Pass
6.43
12.02
Pass
1.00
0.99
Pass
Figure 26 shows the pooled delta data for the Sensors Inc. 3-inch flow meters versus the
laboratory calculated exhaust flow for the International VT365 engine during steady-state repeat
testing. The median flow rate delta for the 3-inch flow meters showed a nearly constant positive
bias of approximately 10 % of point. The engine exhaust flow deltas would likely have been
smaller had the 3-inch EFMs not been recalibrated using the SwRI LFE flow stand data.
However, the recalibration only resulted in an adjustment on the order of 4 % of point, which
was not large enough to explain the 10 % positive bias observed in the engine results. Because
SwRI Report 03.12024.06
Page 77 of 371
-------
the initial linearity check data on the SwRI flow stand indicated a positive bias, it is not clear
what the final source(s) of the positive measurement bias are.
60
50 -
,§ 40
u
30 -
3
ra
•5 20
10 --
40
150
25
37
IPEMS 5th % A PEMS 50th % • PEMS 95th % « Points
22
20
19
200
250 300 350
Lab Reference Exhaust Flowrate (scfm)
16
400
450
FIGURE 26. 3-INCH EFM EXHAUST FLOW POOLED DELTA DATA FROM ENGINE
3 STEADY-STATE REPEAT TESTING
SwRI Report 03.12024.06
Page 78 of 371
-------
4.0 ENGINE DYNAMOMETER LABORATORY TESTING
4.1 Engine Testing Objectives
Engine testing was performed to characterize bias and precision errors for the
SEMTECH-DS instruments versus lab grade emission measurement equipment. Analyzer and
exhaust flow rate measurements were compared over both steady-state and transient engine
operation. Several engine laboratory tests were designed to evaluate errors associated with
ECM-broadcast channels and subsequent interpolation errors of torque and BSFC. Finally, tests
were conducted to assess the exhaust flow measurement errors due to installation related factors.
4.2 Test Engines and Dynamometer Laboratory
FIGURE 27 PEMS INSTRUMENTATION SETUP IN DYNAMOMETER TEST CELL
SwRI Report 03.12024.06
Page 79 of 371
-------
FIGURE 28 ENGINE 1 (HHD) - 14L DDC SERIES 60
FIGURE 29 ENGINE 2 (MHD) - CATERPILLAR C9
SwRI Report 03.12024.06
Page 80 of 371
-------
FIGURE 30 ENGINE 3 (LHD) - INTERNATIONAL VT 365
SwRI Report 03.12024.06
Page 81 of 371
-------
FIGURE 31 TEST CELL EXHAUST SYSTEM SHOWING PEMS FLOWMETERS AND
SAMPLING POINTS
4.3 40-Point Torque and BSFC Map Generation and Error Surface
An initial task specified in the Test Plan was to generate 40-point torque and BSFC maps
as well as preview laboratory and PEMS emission and flow data. The toque and BSFC maps
were generated for a variety of reasons. First, the 40-point maps served as the data set used to
create interpolation surfaces for the estimation of ECM Torque and ECM BSFC from ECM
broadcast (CAN) speed and fuel rate signals. Second, the preview of the emission results from
these points was used to aid in down-selection of the 10 test points to be used in subsequent
steady-state error surface experiments. Finally, the preview data was used to determine whether
multiple PEMS units could be run in parallel on a given engine during steady-state experiments,
thus shortening the amount of time required for the steady-state testing.
The 40-points were chosen by the Steering Committee during the planning portions of the
program, and were designed to cover the entire NTE zone as evenly as possible. Several points
SwRI Report 03.12024.06
Page 82 of 371
-------
were positioned slightly beyond the NTE boundary to aid in interpolation near the edges of the
NTE zone.
After verifying the engines and aftertreatment systems were functioning properly, the lug
curves of the engine were mapped according to the procedures in CFR Part 1065 Subpart F. An
Excel spreadsheet provided by EPA, and approved by the Steering Committee, was used with the
map data to generate the 40 points within the NTE zone. The lug curves and 40 NTE points are
shown in Figure 32 through Figure 34 for Engine 1, Engine 2, and Engine 3, respectively. Using
the laboratory raw and dilute sampling systems, as well as the PEMS, each of the 40 points was
tested over 10-minute modes. The initially planned mode length was 3 minutes; however, the
mode length was extended to 10 minutes following initial Engine 1 testing to insure the fuel flow
measurement was stable.
1800
1600
1400
1200
i 1000
HI
I 80°
600
400
200
0 - Steering Committee Selected 10 Points
• 39
-•40
-•-26--
NTE Torque - Power Line
800 1000 1200 1400 1600 1800
Speed (rpm)
2000
2200
2400
FIGURE 32. ENGINE 1 - DETROIT DIESEL SERIES 60 LUG CURVE AND 40-POINT
MAP
SwRI Report 03.12024.06
Page 83 of 371
-------
1200
1000
800
0 - Steering Committee Selected 10 Points
/
Lug Curve /
;
' ^
NTE Speed Line — ^•
<
>-34 — -
• 32
I 35 • 31
36 • 30
I 37 • 29
t38
-**-4a
^-v
m^
• l?^.
• 20 • 16 ^•s5
• 21 • 15 »6\
M
• 22 • 14 07 *4
400 - - -
200
r-23- - -
-& --- I
NTE Torque - Power Line
FIGURE 33. ENGINE 2 - CATERPILLAR C9 LUG CURVE AND 40-POINT MAP
600
500
400
3
0"
.2
200
100
0 - Steering Committee Selected 10 Points
^
Lug Curve /
r/ \
<
NTE Speed Line — ^>
4
<
• 32 ^~- 35 • 31 • 20 • 16
» 36 • 30 • 21 • 15
> 37 • 29 • 22 • 14
M8
^^--^•28 • 23 • 13
> 40 • 26 • 25 T • 1 1
NTE Torque - Power Line
• 6 "^
\
• 7 414
\
\
• 8 • 3
• 10 *1
L
\
800
1300
1800 2300
Speed (rpm)
2800
3300
FIGURE 34. ENGINE 3 - INTERNATIONAL VT365 LUG CURVE AND 40-POINT MAP
The 40-point torque and BSFC interpolation surfaces were based on the laboratory torque
measurement and the laboratory BSFC calculation from the laboratory measured fuel rate and
SwRI Report 03.12024.06
Page 84 of 371
-------
measured engine power. The laboratory torque and BSFC measurements were referenced to the
ECM-broadcast (CAN) speed and fuel rates for each mode. Using these maps, torque and BSFC
values were interpolated based off of ECM-broadcast channels. A triangular plane interpolation
routine was developed by SwRI statisticians to aid in the interpolation process. The 40-point
interpolation maps were used throughout dynamometer engine testing to produce ECM
interpolated torque and BSFC values for comparison with the laboratory measured reference
values. The torque and BSFC interpolation maps for each engine can be found in Appendix E.
As requested by the Engine Manufacturers, only normalized map data is presented.
Originally, the 40-point torque and BSFC maps and interpolation routine were to be used
in the Monte Carlo Error Model. The reference NTE events, supplying ECM speed and fuel rate,
were to use the maps to interpolate torque and BSFC. The interpolation process in the Model
was problematic because the interpolation maps were different for each engine, and engine map
data was not available for the engine used to generate the reference NTE events. In addition,
there were questions regarding how to choose an interpolation surface for each event, as well as
the additional computational load of having to do repeated interpolations in the Model.
Therefore, the final reference torque and BSFC values were supplied with each reference NTE
events, and no torque and BSFC interpolations were performed in the Model.
In addition to map generation, the 40-point steady-state testing was used to preview the
performance of the PEMS and laboratory. As specified in the Test Plan, the results of the 40-
point testing were used to down-select the 40 points to the 10 points to be used for steady-state
repeat testing. SwRI reviewed the results of 40-point testing and recommended 10 points to be
used for steady-state testing. In the down-selecting process, SwRI attempted to have the selected
10 points evenly span the NOX concentration, exhaust flow rate, and NOX mass flow rate ranges
observed during the 40-point testing. In addition, the 10-points were selected to be somewhat
distributed over the NTE zone. In general, the Steering Committee approved the SwRI
recommended 10-point down-select!on, with only a couple points modified for the final steady-
state repeat testing. The selected 10 points are shown in Figure 32 through Figure 34.
Three PEMS units were used simultaneously during the 40-point mapping process. The
Test Plan called for the data to be examined to determine if running PEMS in this manner would
cause measurement issues that would require subsequent testing to be conducted with one PEMS
unit at a time. A particular area of concern was the use of multiple PEMS flow meters in series.
Following the 40-point testing on Engine 1, the data was examined by Sensors Inc. and the
Steering Committee. There was no evidence of a bias for any PEMS exhaust flow rate
measurement. Sensors Inc. agreed with this assessment, and the Steering Committee elected to
proceed with all further testing using the three PEMS units simultaneously. This decision was
made at the April 4, 2006 conference call. Data from the 40-point maps on Engine 2 and 3 was
also examined for evidence of an exhaust flow bias, but none was found.
4.4 Steady-State Repeat Engine Testing and Error Surfaces
Repeat steady-state engine testing was performed to quantify PEMS bias and precision
errors versus laboratory emission measurement equipment. The measurement errors evaluated
SwRI Report 03.12024.06 Page 85 of 371
-------
during steady-state repeat testing included gaseous emission concentration measurements and
exhaust flow rate measurements.
The steady-state test consisted of 10 modes that were selected by the Steering Committee
from the 40-point mapping procedure discussed previously. The 10-mode steady-state tests were
repeated 20 times. As specified in the Test Plan, the mode order of each of the steady-state tests
was randomized. The mode length was 3 minutes with data averaged over the last 30 seconds of
each mode. Each 10-mode test cycle was run essentially as a ramp modal cycle, although the
modes were processed individually. The laboratory reference analyzers were zeroed and
spanned before each cycle. The engine and laboratory sampling systems were preconditioned
before each cycle as outlined in 1065.520. Following the preconditioning, the engine was
brought to idle, both laboratory and PEMS sampling systems and data recording were started,
and the 10-mode test cycle was started. At the end of the cycle, laboratory systems were zero
and span checked. The PEMS were only spanned at the start of each test day, and were zeroed
prior to the start of each cycle. This was roughly equivalent to zeroing the instrument every
hour, which is the normal schedule for auto-zero maneuvers during field measurements.
Three PEMS units were tested simultaneously during steady-state testing. The SwRI
dynamometer laboratory conducted both raw and dilute emission measurements. The dilute
gaseous concentration measurements were converted to the equivalent raw concentrations using
the CVS flow rate and the calculated exhaust flow rate. This was done by first calculating a
dilute mass rate for a given pollutant, and then using the raw exhaust flow rate to back calculate
a raw concentration. These the dilute-to-raw emission concentrations were used as the
laboratory reference for comparison against the PEMS gaseous concentration measurements.
The laboratory raw measurements were used for quality assurance purposes by providing a check
on the dilute-to-raw measurements and on the raw exhaust flow measurement via carbon balance
verifications. The laboratory exhaust flow rate was determined using a LFE to measure the
intake air flow, a Micro-Motion fuel flow meter to measure fuel flow, and the laboratory
analyzers to measure raw exhaust emission concentrations. The intake LFE measurement and
the raw chemical balance were used with equation 1065.655-14 to calculate the reference
exhaust flow rate. The raw exhaust flow rate was also calculated using the LFE air flow rate and
measured fuel flow with the CFR Part 89 raw exhaust flow rate calculation. The two laboratory
exhaust flow rate calculation methods resulted in nearly identical exhaust flow rate results.
The wet gaseous PEMS concentration data and EFM data were compared to the
laboratory reference. Each PEMS measurement was compared individually to the laboratory
reference. These errors, or deltas, were pooled to generate the steady-state error surfaces. For
steady-state error generation, deltas were generated from paired data sets of PEMS and
laboratory reference measurements. In other words, each PEMS measurements were compared
directly to the associated laboratory reference measurement for that repeat.
4.4.1 Engine 1 Detroit Diesel Series 60 Steady-State
After generating the 40-point torque and BSFC maps, the Steering Committee selected 10
points to perform repeat steady-state testing. The modes selected for Engine 1 steady-state
testing are shown in Figure 32. As discussed in the audit section of the report, the PEMS units
SwRI Report 03.12024.06 Page 86 of 371
-------
used during the initial set of steady-state measurements had all failed the 1065 NC>2 Chiller
Penetration Check. The Steering Committee elected to proceed with steady-state testing to
determine if the NC>2 penetration failure would affect the performance of the PEMS units during
engine testing. PEMS 1, 3, and 4 were used for Engine 1 testing. PEMS 2 was not chosen
initially for Engine 1 testing due to the 1065 Non-stoichiometric C>2 FID Interference audit
failure.
After completion of the steady-state testing, the results were presented to the Steering
Committee. The individual delta data from each PEMS was pooled. The 5th, 50th and 95th
percentiles of the pooled error data was plotted against the mean laboratory reference value. As
shown in Figure 19, the PEMS showed a low bias for NOX, especially at high concentrations.
The Steering Committee deemed the NOX results unsatisfactory, and Sensors Inc. was asked to
design and implement a solution to the NC>2 chiller penetration problem. A complete discussion
of the NC>2 penetration solution can be found in the audit section of this report.
In June 2006, approximately two months after the initial Engine 1 steady-state testing,
Sensors Inc. installed the NC>2 penetration retrofit package on the PEMS units at SwRI. The NC>2
Chiller Penetration audits were then repeated. All upgraded PEMS units passed the 1065
penetration check.
Following the PEMS upgrades and audit checks, Engine 1 steady-state testing was
repeated. Again, the pooled PEMS delta data was plotted against the mean laboratory reference
values. As shown in Figure 35, the low NOX bias of the original testing was replaced with PEMS
data showing a slight positive NOX bias. The pooled gaseous concentration delta data for Engine
1 can be viewed in Appendix F. Delta data is included for both the PEMS as well as the
laboratory raw measurements, with the dilute-to-raw measurements as the reference.
SwRI Report 03.12024.06 Page 87 of 371
-------
IPEMS 5th % A PEMS 50th % • PEMS 95th % « Points
30
25
20
a.
3 15
c
o
15
~ 10
HI
Z
o
O 5
140
-5
-10
.40
10
15
24 *
--3-
33
160
180
200
220
240
260
280
300
Lab Reference Mean NOx Concentration (ppm)
FIGURE 35. NOX CONCENTRATION POOLED DELTAS FOR REPEAT STEADY-
STATE TESTING ON ENGINE 1 AFTER NO2 PENETRATION UPGRADE
With the Engine 1 catalyzed DPF, CO and HC emissions were very low. Although the
laboratory raw and dilute analyzers reported raw CO concentration levels generally between 10
to 25 ppm, the SEMTECH-DS consistently measured CO emission levels at approximately 40
ppm. Pooled deltas for CO are given in Figure 36 for Engine 1. The high CO bias may by due
in part to the low resolution of the CO detector, which has a reported resolution of 0.01%
(lOppm). In addition, the CO instruments would typically read between 20 and 40 ppm when
zero gas was introduced to the sample port of the SEMTECH DS using a sample probe overflow
technique. The positive CO bias was apparently due in part to the sampling handling system of
the unit and was observed on all three engines.
SwRI Report 03.12024.06
Page 88 of 371
-------
I PEMS 5th % A PEMS 50th % * PEMS 95th % • Points
60
10
50 '
Q.
HI
Q
I30
1:
HI
o
J 20
O
O
10 -
,-4 • V • y 33
A A
A
A A
10 12 14 16 18 20 22 24
Lab Reference Mean CO Concentration (ppm)
FIGURE 36. CO CONCENTRATION POOLED DELTAS FOR REPEAT STEADY-
STATE TESTING ON ENGINE 1
NMHC measurements presented a particular problem due to the very low tailpipe levels
associated with the use of the DPFs. The laboratory reference measurement was complicated by
several factors beyond the low concentration levels. First, the levels of THC and methane were
similar during all steady-state testing. Figure 37 shows mean raw exhaust hydrocarbon
concentrations for the steady-state modes. Individual THC and CH4 measurements varied
approximately ± 0.2 ppmC around the mean. Unfortunately, this level was similar to the final
NMHC concentrations. Therefore, the variability of the NMHC measurement was high, and in
some cases resulted in slight negative values.
SwRI Report 03.12024.06 Page 89 of 371
-------
-0.2
Steady-State Mode
FIGURE 37. MEAN RAW HC CONCENTRATIONS FOR ENGINE 1 STEADY-STATE
TESTING
The dilute NMHC measurement was further complicated because the raw exhaust
concentrations were below those in the background air. Typical background levels were 3 ppmC
for THC and 2.5 ppmC for NMHC. This resulted in a high occurrence of negative NMHC
results. The NMHC measurement errors were further exaggerated by the dilute-to-raw scaling
process. As a result, the Steering Committee elected to abandon the dilute-to-raw NMHC
concentration values as the reference at the April 2006 meeting. The laboratory direct raw
concentrations were chosen as the NMHC reference because the raw measurements were not
complicated by background concentrations and conversion problems.
Initially, Sensors Inc. supplied two methane analyzers which could, in principal, be added
to the SEMTECH-DS units. However, since these analyzers were external laboratory grade
analyzers, the Steering Committee decided the methane analyzers were not suitable for field
measurement. Therefore, the PEMS NMHC values were determined using only the THC
measurement. The THC concentrations were multiplied by 0.98 to generate NMHC values, as
given in Part 1065. Figure 38 shows PEMS NMHC concentrations for Engine 1 plotted against
the associated mean raw laboratory reference values. Tailpipe HC levels for Engine 1 were
below the resolution limits of the PEMS FID analyzer, which reported mostly zero THC values
throughout steady-state testing. Therefore, the data from Engine 1 was not useful in producing
an NMHC error surface. A final decision on how to process the NMHC data was deferred until
results from Engines 2 and 3 could be reviewed.
SwRI Report 03.12024.06
Page 90 of 371
-------
0.5
0.4
0.3
E 0.2
Q.
Q.
I 0.1
c
O -0.1
O
i -0.2
-0.3
-0.4
-0.5
IPEMS 5th % A PEMS 50th % • PEMS 95th %
-0.05
0.00
0.05 0.10 0.15 0.20
Lab Reference NMHC Concentration (ppm)
0.25
0.30
FIGURE 38. PEMS NMHC CONCENTRATIONS FOR ENGINE 1 STEADY-STATE
TESTING
As shown in Figure 39, the PEMS 5-inch EFMs showed good correlation with the
laboratory reference exhaust flow rate measurement, with deltas generally less than 2 to 3
percent of point. Although not used in the Model, NOX mass flow rate errors are also shown in
Appendix F. The PEMS NOX mass flow rate measurements were biased slightly high, which is
consistent with the slightly high bias observed in both NOX concentration and the EFM
measurements.
SwRI Report 03.12024.06
Page 91 of 371
-------
IPEMS 5th % A PEMS 50th % » PEMS 95th '
1 Points
30
25 -
20 -
— 15 -
-2- 10-
5 -
-5 -
-10 --
-15 -
-20
40
37
27
24
33
-TO 9-
250
350
450
550
B50
750
850
950H
1050
Lab Reference Exhaust Flowrate (scfm)
FIGURE 39. POOLED EFM DELTAS FOR ENGINE 1 STEADY-STATE TESTING- 5-
INCH FLOW METER
4.4.2 Engine 2 Caterpillar C9 Steady-State
Following the generation of the 40-point torque and BSFC maps, the Steering Committee
selected 10 NTE points to perform the repeat steady-state testing for Engine 2. The selected 10
points are shown in Figure 33. Prior to Engine 1 steady-state testing, PEMS 6, used in
environmental testing, experienced a FID failure. Because there was an immediate need to
continue environmental chamber testing, PEMS 3 was pulled from the dynamometer laboratory
and used as a replacement for PEMS 6. This resulted in a schedule delay while PEMS 6 was
repaired. PEMS 6 was therefore used for Engine 2 and 3 testing.
During initial Engine 2 steady-state testing, PEMS 4 NOX values showed several outlying
low points. The continuous NO and NO2 data from PEMS 4 indicated periods when both
channels were reporting zero values. Diagnostic efforts pointed to a bad NDUV lamp; therefore,
Sensors Inc. replaced the NDUV. Linearity checks were performed on the new NDUV before
proceeding with steady-state testing. After only a couple tests, the new NDUV in PEMS 4 began
behaving erratically as well. The PEMS 4 NDUV was replaced once again. Linearity checks as
well as a NO2 penetration check were performed before continuing with Engine 2 testing.
Shortly after PEMS 4 was repaired, PEMS 6 reported a fault stating the Manifold
Relative Humidity Sensor was not responding. With diagnostic support from Sensors Inc., the
exhaust manifold RH sensor and sensor manifold block were removed. This assembly is part of
the NO2 penetration upgrade package which had been developed by Sensors Inc. earlier in the
program. As shown in Figure 40, the sensor was found to be corroded, therefore a new sensor
and sensor manifold was installed. Because the exhaust manifold relative humidity sensor was
SwRI Report 03.12024.06
Page 92 of 371
-------
part of the NC>2 chiller penetration retrofit kit, a 1065 NC>2 penetration check was repeated.
PEMS 6 passed the penetration check with the new RH sensor. Shortly after continuing with
steady-state repeat testing, PEMS 6 again reported the relative humidity sensor was not
responding. The sensor was removed and found to be wet. A new sensor and sensor manifold
were installed and the 1065 NC>2 Chiller Penetration check was repeated. PEMS 6 passed the
audit and SwRI continued steady-state testing with Engine 2. After completing only a couple
steady-state tests, PEMS 6 reported the same RH sensor fault. Again, the sensor was found to be
wet.
FIGURE 40. CORRODED RH SENSOR (LEFT) COMPARED TO A NEW RH SENSOR
(RIGHT)
After the third failure, Sensors Inc. recommended checking the sensor manifold block for
leaks. The leak test was performed by slightly pressurizing the sensor manifold and checking for
air leaking past the RH sensor. All sensor manifolds tested by SwRI had air escaping the
manifold by the RH sensor. According to Sensors Inc., the escaping air likely caused liquid
water to be drawn up the manifold and in contact with the relative humidity sensor, thus causing
the fault. Sensors Inc. instructed SwRI to reseal the RH sensor in the sensor manifold block
using silicon. A picture of a RH sensor surrounded by silicon as well as the sensor manifold
block is shown in Figure 41. After reseating the RH sensor, the manifold was leak checked to
insure air was not escaping from the manifold. Another 1065 NC>2 penetration check was
performed after installing the new sensor. With the properly sealed sensor manifold, PEMS 6
operated without fault for the remainder of the steady-state testing.
SwRI Report 03.12024.06
Page 93 of 371
-------
FIGURE 41. DISASSEMBLED RH SENSOR MANIFOLD WITH RH SENSOR
During this time, PEMS 4 also began to have faults pertaining to the manifold relative
humidity sensor not responding. The RH sensor and manifold were replaced with a new, non-
leaking manifold. After replacement of the sensor, and subsequent NC>2 penetration check,
PEMS 4 reported no other problems related to the RH sensor.
Following the completion of the various repairs and diagnostic efforts, the remaining
Engine 2 steady-state tests were completed. However, examination of the data following the
completion of Engine 2 transient testing revealed a problem with the Caterpillar C9 steady-state
data. In generating the transient error surfaces, the transient data was corrected using the
variance measured during steady-state testing. The steady-state variance correction process is
described in detail under the transient testing section. However, the variance of the Engine 2
steady-state data was generally larger than the variance of the transient data. After reviewing the
Engine 2 steady-state data, the high variance was found to be related to the large time lapse
caused by the PEMS hardware failures. Almost half of steady-state points were run prior to the
PEMS hardware failures, with the remaining points run approximately 3 weeks afterward.
As seen in Figure 42, there is a definite shift in NOX concentration for the initial steady-
state points versus the points run after the PEMS repairs. Repeats 5, 9, and 12 through 20 were
run 2 V2 weeks after the other steady-state repeat tests. This shift was recorded for both
laboratory and PEMS analyzers. The bias error would not affect the steady-state error surfaces,
as the PEMS measurements are paired with the laboratory reference and this removes variances
caused by the engine. However, the variance of the pooled raw PEMS data, not the PEMS delta
data, was used to generating the transient error surfaces. The high variance of the Engine 2
PEMS steady-state data would have collapsed the Engine 2 transient error surfaces due to the
steady-state variance correction. Because the Engine 2 transient error surfaces would be
inaccurate using the high variance steady-state data, the Steering Committee elected to repeat
Engine 2 steady-state testing.
SwRI Report 03.12024.06 Page 94 of 371
-------
550
540
530
a.
520
510
500
490
480
Data Run Late
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
SS Repeat Number
FIGURE 42. ENGINE 2 INITIAL STEADY-STATE REPEAT NOX RESULTS
SHOWING TIME DEPENDENT CONCENTRATION SHIFT
The repeated steady-state testing for Engine 2 went smoothly, with no problems from the
PEMS or the laboratory. The PEMS and laboratory raw data was compared to the laboratory
reference dilute-to-raw measurements. The pooled delta data was plotted versus the mean
reference value. The results for the gaseous emission concentration errors are shown in
Appendix F. The SEMTECH-DS median NOX error levels for Engine 2 were generally less than
5 ppm and centered near zero.
In an attempt to address the issue of low tailpipe NMHC levels observed during Engine 1
testing, a different DPF was used for Engine 2. The Engine 2 DPF was a 2007 production DPF
supplied by Caterpillar. The production DPF likely had lower precious metal loadings than the
DPFs which SwRI had procured for the program. The Steering Committee hoped the production
DPF would result in more useable NMHC data. Figure 43 shows raw hydrocarbon levels for
Engine 2. While the NMHC concentrations were higher than Engine 1 levels, the methane and
THC levels were extremely similar, resulting in reference NMHC levels still near zero. The
similar THC and CH4 levels resulted in a large occurrence of negative values for the raw
laboratory reference. However, the PEMS showed measurable NMHC response for Engine 2, as
seen in Figure 44.
SwRI Report 03.12024.06
Page 95 of 371
-------
456
Steady-State Mode
10
FIGURE 43. RAW HYDROCARBON LEVELS FOR ENGINE 2 STEADY-STATE
TESTING
5.0
I PEMS 5th % A PEMS 50th '
# PEMS 95th '
4.5
4.0
3.5
I 3.0 -J
c
HI
u
c
o
o
O
2.5
2.0
1.5
LU
Q.
1.0 -
0.5
0.0
-0.9 -0.8 -0.7 -0.6 -0.5 -0.4 -0.3
Laboratory NMHC Concentration (ppm)
-0.2
-0.1
0.0
FIGURE 44. PEMS NMHC CONCENTRATIONS FOR ENGINE 2 STEADY-STATE
TESTING
SwRI Report 03.12024.06
Page 96 of 371
-------
Over the course of several conference calls and meetings, there was considerable
discussion among Steering Committee members as to how the NMHC data could be represented
in the Model. Ultimately, several decisions were made regarding the NMHC error surface.
First, it was determined that the laboratory reference method for NMHC was not accurate at the
low NMHC levels. As a result, the NMHC error surface was collapsed to a single x-axis point,
and all deltas would be generated using a reference value of zero. All of the NMHC data would
be pooled together to generate a single set of 5th, 50th, and 95th percentile values. Second, the
Steering Committee decided that only Engine 2 data would be used to populate the NMHC error
surface, because the data from Engines 1 and 3 showed no PEMS NMHC response. These
decisions were finalized at the November 2006 Steering Committee meeting in San Antonio. A
similar approach was to be used for the transient error surface as well.
Although the laboratory analyzers reported CO levels under 6 ppm for all modes during
Engine 2 testing, the PEMS median error was consistently near 50 ppm. Steady-state CO data
for Engine 2 is found in Appendix F. This high bias was similar to the data observed for Engine
1, and was consistent for all CO measurements during this program.
The deltas measured for the PEMS 4-inch EFMs versus the laboratory reference exhaust
flow rate are shown in Figure 45. Although the 4-inch flow meters passed the 1065 linearity
criteria on the SwRI flow stand, the EFMs showed a positive error at high flow rates during
engine testing. This error was on the order of a 5 percent positive bias. A discussion of the 4-
inch EFM error results and linearization issues is included in the flow meter audit section of the
report. Although not part of the measurement allowance, NOX mass flow rate deltas for Engine 2
are also included in Appendix F. With accurate NOX concentrations measurements, the NOX
mass flow rate error resembled the exhaust flow rate errors and had a positive bias at high NOX
mass flow rates.
SwRI Report 03.12024.06 Page 97 of 371
-------
IPEMS 5th % A PEMS 50th % • PEMS 95th % • Points
60
50
40
£ 30 -
ra
^
u.
•K 20 -
10 -
-10
40
39
36
-34-
2i-
13
10
:
250
350
450
550
650
750
Lab Reference Exhaust Flowrate (scfm)
FIGURE 45. POOLED EFM DELTAS FOR ENGINE 2 STEADY-STATE TESTING - 4
INCH FLOW METER
4.4.3 Engine 3 International VT365 Steady-State
Similar to Engine 1 and 2, 10 NTE points were selected by the Steering Committee from
the original 40 points tested for the torque and BSFC maps. The selected 10 points are shown in
Figure 34. As with Engine 2, PEMS 1, 4, and 6 were used for Engine 3 testing. A Horiba OBS-
2200 On Board Emission Measurement System was also tested during Engine 3 operation.
Engine 3 steady-state repeat testing went smoothly, with no equipment failures from the
PEMS or laboratory. However, PEMS 6 consistently showed a negative NOX bias at high
concentration levels. This surfaced initially during the 40-point map testing, and was confirmed
during repeat steady-state tests. Shown in Figure 46 is the PEMS 6 steady-state pooled NOX data
versus the mean laboratory reference concentrations. PEMS 1 also showed a slight negative NOX
bias at high concentrations, but not as severe as PEMS 6. Post-test span checks for all PEMS,
conducted using the instrument span port, indicated no problems despite the low bias. Several
diagnostic tests were performed immediately after steady-state testing with PEMS 6 to determine
the cause of the bias. NO and NO2 linearity verification results did not indicate any instrument
problems. As a check, dry span gas was then overflowed to the sample line of PEMS 6. At 100
% of span value, NO read nearly 7 % below the bottle value and NO2 read over 4 % low. At 70
% of span value, NO read approximately 2 % low while NO2 read slightly over 1 % low. At 30
% of span value, both NO and NO2 measurements were accurate. This confirmed the low bias
problem, but only when the gas is being introduced through the sample line.
SwRI Report 03.12024.06
Page 98 of 371
-------
•Lab Reference
PEMS6 5th % A PEMS6 50th '
PEMS6 95th %
Points
700
600
500
400
300
200
100
37 35
100
200
300 400 500
Lab Reference NOx Concentration (ppm)
600
700
FIGURE 46. PEMS NOX CONCENTRATION VERSUS MEAN LABORATORY
REFERENCE FOR ENGINE 3 STEADY-STATE TESTING
In early December 2006, an NC>2 penetration check was performed with PEMS 6. As
shown in Table 41, PEMS 6 passed a NC>2 chiller penetration check. This was unexpected,
considering the low biases observed before the long weekend. Therefore, the overflow checks
with dry NO and NO2 span gas were repeated. Low biases were not observed during the
repeated checks, indicating that something had changed while the PEMS were sampling ambient
air over the weekend. A possible explanation for the performance difference is the drying of
accumulated water in the sample handling system. It is not known at this time why some of the
PEMS showed a low bias while others did not during Engine 3 steady-state testing; however,
none of the analyzers failed any of the 1065 performance checks during this time. There was
considerable Steering Committee discussion regarding the Engine 3 steady-state data set. The
Steering Committee elected to accept the biased steady-state data because a specific cause for the
low NOX bias was not evident, and because the PEMS continued to pass all pertinent 1065 audit
verifications.
Another concern with the International steady-state data was high NOX concentration
variability at high concentration levels. NTE points 35, 37, and 40 were all near peak torque
speed and produced high NOX concentrations. Although the speed and torque for these modes
was consistent, the NOX concentrations showed unexpectedly high variability, which was evident
in both the lab reference data and the PEMS data. Figure 47 shows the laboratory dilute-to-raw
concentrations for NTE point 35 during steady-state repeat testing. The laboratory reference
NOX concentration median absolute deviation (MAD) value calculated for point 35 was over 40
ppm. An example MAD calculation is shown below for reference. As discussed for the
Caterpillar steady-state data, the high NOX variability did not adversely affect the Engine 3
SwRI Report 03.12024.06
Page 99 of 371
-------
steady-state error surface data because of the individual pairing with the laboratory reference.
However, the high steady-state variability does affect the transient error surfaces during the
steady-state variance correction. The solution to this problem is discussed in detail in the
transient engine testing section of the report.
MAD = median^xj -median(x^
800
•§• 750
.2 700
650
o
O
§ 600
HI
o
c
S 550
•s
OL
o 500
450
400
10
Steady-State Repeat Number
15
20
FIGURE 47. INTERNATIONAL VT365 POINT 35 NOX CONCENTRATIONS DURING
STEADY-STATE REPEAT TESTING
Another issue with the Engine 3 steady-state data was several instances of outlying data.
As seen in Figure 48, NTE point 30 had 5 repeats that were significantly higher than the other 15
events. This shift was observed on all of the measurement instruments, including the laboratory
dilute and raw and all of the PEMS. As a result, this instances were determined to be the result
of engine variability, rather than measurement errors. Per the Steering Committee's decision, the
outlying data points were removed from steady-state data set. For NOX concentration, events
were removed from NTE points 25, 30, and 37. Outlying events were also removed from the
CO, CO2, and NOX mass flow rate data sets.
SwRI Report 03.12024.06
Page 100 of 371
-------
400
E
Q.
350
o
IE
"E
HI
I
O
HI
o
£ 250
£
HI
OL
2 200
.Q
J3
150
Outlying Point;
10
Steady-State Repeat Number
15
20
FIGURE 48. INTERNATIONAL VT365 POINT 30 NOX CONCENTRATIONS DURING
STEADY-STATE REPEAT TESTING
After removal of the outlying data, the PEMS and laboratory raw data was compared to
the laboratory reference dilute-to-raw measurements. The pooled delta data was plotted versus
the mean reference values. The results for the gaseous emission concentration errors are shown
in Appendix F. As discussed previously, PEMS 1 and 6 showed a low bias for NOX
concentration at high levels. Similar to Engine 1 and 2, the median CO errors were near 50 ppm,
with the 95th percentile values reaching 90 ppm for Engine 3. Although the PEMS showed
occasional NMHC responses on Engine 3, the large body of data indicated essentially zero
PEMS response to the tailpipe exhaust, similar to what was observed for Engine 1. This data
ultimately reinforced the Steering Committee decision to use only Engine 2 data for NMHC error
surface generation.
The deltas measured for the PEMS 3-inch EFMs versus the laboratory reference exhaust
flow rate are shown in Figure 26. Although the 3-inch flow meters passed the 1065 linearity
criteria on the SwRI flow stand, the EFMs showed a positive error at high flow rates during
engine testing. Generally this error was on the order of 10 percent of point. A discussion of the
3-inch EFM error results and linearity is given in the flow meter audit section of the report. As
shown in Figure 50, the Horiba OBS-2200 exhaust flow rate measurements showed good
agreement with the laboratory reference.
SwRI Report 03.12024.06
Page 101 of 371
-------
PEMS 5th % A PEMS 50th % » PEMS 95th % • Points
60
50
,§ 40
§ 30 ^
u.
"w
3
re
x 20
LU
10 -
40
25
37
35 30
JU
22
20
19
16
•
A
A
150
200
250 300 350
Lab Reference Exhaust Flowrate (scfm)
400
450
FIGURE 49. POOLED EFM DELTAS FOR ENGINE 3 STEADY-STATE TESTING - 3-
INCH FLOW METER
PEMS 5th % A PEMS 50th % » PEMS 95th % • Points
20
15
10
I
u
* 0
£
ra 1
-10
-15
-20
-25
-30
40
22
-16-
A 200
250
300
350
400
Lab Reference Exhaust Flowrate (scfm)
FIGURE 50. POOLED HORIBA OBS-2200 EXHAUST FLOW RATE DELTAS FOR
ENGINE 3 STEADY-STATE TESTING
SwRI Report 03.12024.06
Page 102 of 371
-------
Although not part of the measurement allowance, NOX mass flow rate deltas are also
included in Appendix F. The NOX mass flow rate errors were biased high in the mid-to-lower
range due to the positive exhaust flow rate error. However, the negative NOX concentration bias
at high levels helped offset the NOX mass flow rate error at the higher levels in some cases.
Figure 51 shows the pooled Horiba OBS-2200 NOX concentration deltas measured during
Engine 3 steady-state testing. Median delta values were near 10 % of point. In addition to the
th
-th
median bias, NOX variability was also large, with the difference between the 95 and 5
percentile concentrations near 20% of point. The Engine 3 pooled steady-state deltas for all
gaseous emissions are shown in Appendix F. As shown in Figure 52, THC measurement with
the Horiba OBS-2200 showed good correlation to the SwRI raw THC concentrations, even at
levels between 0.5 and 2.5 ppmC. The OBS-2200 CO2 concentration measurements were
generally higher than the laboratory reference values, with median deltas ranging from 3 to 5 %
of point. With the laboratory reference CO concentrations ranging from 6 to 18 ppm, CO deltas
were near -100 ppm for the 5th percentile deltas, -60 ppm for the 50th percentile, and 110 ppm for
the 95th percentile error.
140
120 -
100 -
a.
80 -
I 60 4
c
o
o
40 -
20 -
-20 -
-40
I PEMS 5th % A PEMS 50th % » PEMS 95th % • Points
. .•
* 40
25 22° 16 20 19
*
A
A
» A
A f
A
0 " • ^40 " 340 440
• • •
• *
37 35
*
A
540 64fl 7.
Lab Reference NOx Concentration (ppm)
FIGURE 51. POOLED NOX DELTAS FOR THE HORIBA OBS-2200 DURING ENGINE
3 STEADY-STATE TESTING
SwRI Report 03.12024.06
Page 103 of 371
-------
- Lab Reference • PEMS 5th % A PEMS 50th % » PEMS 95th '
Points
3.5
3.0
O 2.5
Q.
Q.
o 2.0
1.5
o
o
o
1.04-
0.5
0.0
30
' '35
19 2$6
40
25
-*-
0.0
0.5
1.0 1.5
Lab Reference THC Concentration (ppmC)
2.0
2.5
FIGURE 52. POOLED THC MEASUREMENTS FOR THE HORIBA OBS-2200
DURING ENGINE 3 STEADY-STATE TESTING
4.4.4 Steady-State Concentration Error Surface Generation
The steady-state gaseous emission concentration error surfaces were generating using the
-th
-th
pooled PEMS EFM deltas versus the laboratory reference. The 5 , 50 , and 95 percentile
values of the pooled error terms were plotted against the mean laboratory reference
concentrations. Delta values were normally sampled from the steady-state gaseous concentration
error surfaces for each NTE event. Because the error surfaces were level dependent, linear
interpolation between points was used to determine the appropriate delta. Individual steady-state
error surfaces for each engine are contained in Appendix F.
The final steady-state concentration error surfaces were generated by pooling the Engine
1, 2, and 3 error surfaces. The combined error surface is shown in Figure 53. Because the NOX
concentration error profiles for the three engines were notably different, the combined final error
surface was extremely irregular, displaying sharp transitions in areas where concentration values
for the three engines overlapped. The original intent of testing three engines was to generate a
broad, uniform, well-distributed final error surface. The underlying assumption with this method
was that the three engines would produce similar errors, and the combined error surface would
therefore be relatively uniform.
SwRI Report 03.12024.06
Page 104 of 371
-------
•95th percentile
-50th percentile (median)
•5th percentile
a.
o.
u
Q
c
o
2 -20
c
HI
u
c
o
o
LU
Q.
-100
-40
-60
-80
Lab Reference Mean NOx Concentration [ppm]
FIGURE 53. COMBINED ERROR SURFACE FOR STEADY-STATE NOX
CONCENTRATION
The combined error surface shown in Figure 53 was presented to the Steering Committee
during the December 2006 meeting in San Antonio. After a lengthy discussion, the Committee
elected to reprocess the final error surfaces. Because the steady-state concentration error
surfaces were sampled normally, the 5th and 95th percentile deltas were to represent the largest, or
worst case, delta values. Following that argument, if all engines would have generated deltas at
all x-axis concentration levels, the 5th percentile value would have been generated by the engine
having the lowest bias and the 95
the highest bias.
th
percentile would have been generated by the engine reporting
The steering committee elected to reprocess the final error surfaces by linearly
interpolating between each engine's error surface data points to populate x-axis values generated
by the other engines. For example, the Engine 1 deltas were used to linearly interpolate Engine 1
deltas that would have occurred at the other x-axis concentrations. This method was applied to
the 5th, 50th, and 95th percentile values for each engine. For concentration values beyond the
range of data actually taken for a given engine, the method for generating delta values depended
on the trends observed in the measured data. If no definite trend was observed in the data for a
given engine, the nearest x-axis error value was repeated to generate the extrapolated data (i.e.,
the first or last data point of the data set). If a trend was evident, a regression line was fit through
the engine's delta data. The regression line was then used to generate deltas for points requiring
extrapolation.
-th
-.th
-th
Using the method described above, a 5 , 50 , and 95 percentile was generated for all
three engines at each of the 30 x-axis mean laboratory reference points. Once this was done, the
SwRI Report 03.12024.06
Page 105 of 371
-------
final pooled error surfaces were generated. At each of the 30 x-axis points, the 5th percentile of
the pooled error surface was generated by selecting the lowest 5th percentile value from the three
engines at that point. In a similar manner, the 95th percentile was selected by taking the highest
95l percentile value from the three engines, and the 50th percentile was taken as the middle 50th
percentile value from the three engines. The final steady-state NOX concentration error surface
used in the Model is shown in Figure 54 as an example of this process. Concentration error
surfaces for each engine as well as the final combined error surfaces are shown in Appendix F
for all pollutants. This process was not needed for NMHC data, because only Engine 2 data was
used to generate the NMHC error surface as discussed earlier.
•95th percentile
-50th percentile (median)
•5th percentile
7(10
-100
Lab Reference Mean NOx Concentration [ppm]
FIGURE 54. FINAL ERROR SURFACE FOR STEADY-STATE NOX
CONCENTRATION
The steady-state exhaust flow rate error surfaces were generating using the pooled PEMS
EFM deltas versus the laboratory reference. The 5th, 50th, and 95th percentile values of the
pooled error terms were plotted against the mean laboratory reference exhaust flow rates. The
data was normalized using the maximum EFM flow rate as specified in the user manual. The
maximum flow rates for the 3-inch, 4-inch, and 5-inch EFMs were 600, 1100, and 1700 scfm,
respectively. The reference NTE events used in the Model supply exhaust flow rate in scfm as
well as the EFM size. Using this information, the reference NTE exhaust flow rate measurement
was normalized similar to the laboratory generated error surface. Using the normalized flow
rate, a delta value was normally sampled from the steady-state exhaust flow error surface for
each NTE event. Because the error surface was level dependent, linear interpolation was used to
determine the appropriate flow rate delta. Individual steady-state error surfaces for each engine
are collected in Appendix F.
SwRI Report 03.12024.06
Page 106 of 371
-------
The combined steady-state exhaust flow rate error surface was generated by pooling the
Engine 1, 2, and 3 error surfaces. The combined error surface is shown in Figure 55. Similar to
the steady-state concentration combined error surfaces, the exhaust flow rate error surface was
not uniform due to error differences for each engine and EFM size.
&
o
LU
Q.
-95th percentile
-50th percentile (median)
-5th percentile
Lab Reference Mean Exhaust Flow Rate [% of Max]
FIGURE 55. COMBINED ERROR SURFACE FOR STEADY-STATE EXHAUST FLOW
RATE
The final steady-state exhaust flow rate error surface was reprocessed as described in the
steady-state concentration error surface generation section. The final steady-state exhaust flow
rate error surface used by the Model is shown in Figure 56.
SwRI Report 03.12024.06
Page 107 of 371
-------
-95th percentile
-50th percentile (median)
-5th percentile
Lab Reference Mean Exhaust Flow Rate [% of Max]
FIGURE 56. FINAL ERROR SURFACE FOR STEADY-STATE EXHAUST FLOW
RATE
4.5 Transient Engine Testing and Error Surfaces
The transient engine testing was performed to evaluate the errors involved in using the
SEMTECH-DS PEMS units to measure 30-second transient NTE events. It should be noted that
the intent of the transient experiments was to capture errors present over and above those already
observed during the steady-state experiments (i.e., errors resulting from the transient nature of
the events being measured). Transient error surfaces were generated for gaseous emission
concentrations, exhaust flow rate, and various ECM-related data, including ECM broadcast
speed and fuel rate, and ECM interpolated torque and BSFC. In addition, the transient test data
was used to generate an error surface based on time alignment errors of several key PEMS
parameters.
During the development of the Test Plan, there was concern over the lack of information
available regarding the accuracy and precision of the laboratory reference methods over 30-
second test events. Therefore, the Steering Committee elected not to compare the laboratory and
PEMS data during transient testing. Instead, the transient error surfaces account only for
precision errors of the PEMS with respect to their own median measurements. There is no bias
error term captured in the transient error surfaces. However, all laboratory instruments were
used during transient testing for comparative purposes and to evaluate the repeatability of the lab
over 32 second events. A secondary goal of the program was to assess the repeatability of the
1065-based reference laboratory methods over 30-second events of this nature.
Transient engine testing consisted of repeating 20-minute cycles containing 30 unique
32-second NTE events. An Excel spreadsheet, provided by EPA and approved by the Steering
SwRI Report 03.12024.06
Page 108 of 371
-------
Committee, was used to generate 30 unique NTE events for each engine, based on the engines'
lug curves. In addition, 31 unique transition events were generated, allowing for varying
amounts of time between NTE events during the cycle. Descriptions of the NTE and transition
events were taken directly from the Test Plan and are given in Table 48 and Table 49,
respectively. The NTE event order, as well as the transition order, was randomized to generate
20 different 20-minute cycles, each containing all 30 NTE events. As stated in the Test Plan,
only 4 to 5 cycles were run each day, so that transient testing occurred over a span of 4 to 5 days.
TABLE 48. NTE EVENT DESCRIPTIONS FROM THE TEST PLAN
Table 3.3.3-a: Dynamic Response NTE Events
NTE
Event
NTEj
NTE2
NTE3
NTE4
NTE5
NTE6
NTE7
NTE8
NTE9
NTE10
NT-En
NTE12
NTE13
NTE14
NTE15
NTE16
NTEJ 7
NTE18
NTEJ 9
NTE20
NTE21
NTE22
NTE23
NTE24
NTE25
NTE26
NTE27
NTE28
NTE29
NTE30
'Speed %
Range
17%
59%
Governor line
17%
59%
Governor line
17%
59%
100%
Lower third
Upper third
Middle third
17% - governed
17% - governed
17% - governed
2Torque %
Range
332%
332%
332%
66%
66%
66%
100%
100%
100%
332% - 100%
332% - 100%
332% - 100%
Lower third
Upper third
Middle third
Lower right diagonal
Upper left diagonal
Full diagonal; lower left to upper right
Lower left diagonal
Upper right diagonal
Full diagonal; lower right to upper left
Third light — heavy-duty NTE event from
International, Inc. data set
Cruise; ~ 50 mph
Cruise; ~ 75 mph
Small bulldozer
Large bulldozer
Second of three NTE events in FTP
Third light — heavy-duty NTE event from
International, Inc. data set
First of two NTE events in NRTC
First of two NTE events in NRTC
Description
Steady speed and torque; lower left of NTE
Steady speed and torque; lower center of NTE
Steady speed and torque; lower right of NTE
Steady speed and torque; middle left of NTE
Steady speed and torque; middle center of NTE
Steady speed and torque; middle right of NTE
Steady speed and torque; upper left of NTE
Steady speed and torque; upper center of NTE
Steady speed and torque; upper right of NTE
Highly transient torque; moderate transient speed
Highly transient torque; moderate transient speed
Highly transient torque; moderate transient speed
Highly transient speed; moderate transient torque
Highly transient speed; moderate transient torque
Highly transient speed; moderate transient torque
Transient; speed increases as torque increases
Transient; speed increases as torque increases
Transient; speed increases as torque increases
Transient; speed decreases as torque increases
Transient; speed decreases as torque increases
Transient; speed decreases as torque increases
Sample from LHDE
Sample from HDDE
Sample from HDDE
Sample from NRDE
Sample from NRDE
Seconds used from FTP: 714-725, 729-743, 751-755
Sample from LHDE
Seconds used from NRTC: 423-430, 444, 448-450, 462-
481, increased 464 speed from 40% to 42%
Seconds used from NRTC: 627-629, 657-664, 685-696,
714-722
1 Speed (rpm) = Curb Idle + (Speed % * (MTS - Curb Idle)
2 Torque (Ibf-ft) = Torque % * Maximum Torque At Speed (i.e. lug curve torque at speed)
3 Torque (Ibf-ft) = Maximum of (32 % * peak torque) and the torque at speed that produces (32 % * peak
power)
SwRI Report 03.12024.06
Page 109 of 371
-------
TABLE 49. NTE TRANSITION DESCRIPTIONS FROM THE TEST PLAN
Table 3.3.3-b: Dynamic Response Inter-NTE Events
INT Event1
INTj
INT2.6
INT7.10
INTn.14
INT15.18
INT19.21
INT22
INT23
INT24
INT25
INT26
INT27
INT28
INT29
INT30
INT31
Duration (s)
10
2
3
4
5
6
7
8
9
11
13
17
22
27
35
5
Frequency
1
5
4
4
4
o
J
Description
Initiation of cycle; INTj is always first
Shortest and most frequent inter-NTE events
Short and frequent inter-NTE events
Short and frequent inter-NTE events
Short and frequent inter-NTE events
Short and frequent inter-NTE events
Medium inter-NTE event
Medium inter-NTE event
Medium inter-NTE event
Medium inter-NTE event
Long inter-NTE event
Long inter-NTE event
Long inter-NTE event
Long inter-NTE event
Longest inter-NTE event
Termination of cycle; INT31. is always last
Interval speeds and torques are not identical, but they are clustered around zero torque and the speed at which 15%
of peak power and 15% of peak torque are output.
These tests were all run as hot-start transient tests. The engine and aftertreatment were
preconditioned before each test as recommended in 1065.520. The laboratory analyzers were
zeroed and spanned prior to each test, although again, they were run for reference only. The
PEMS were spanned only at the start of the day, and zeroed prior to each test. The total elapsed
time for each test was near one hour, which is the recommended auto-zero frequency for the
PEMS. The transient data was post-processed to extract the data associated with the 30
individual events so they could be compared across all 20 repeats.
4.5.1 Engine 1 Detroit Diesel Series 60 Transient
The transient engine testing followed the repeated Engine 1 steady-state testing with the
upgraded PEMS. The initial transient task was to generate the transient cycles and tune the
engine and dynamometer controls. Some of the NTE events contained highly transient speed
and load changes that challenged the laboratory dynamometer as well as the test engine. Next,
the lug curve from the Engine 1 was programmed into the PEMS. The PEMS used the J1939
Percent Load ECM-broadcast channel with the Engine 1 lug curve to estimate real time torque.
A number of prep cycles were then run to insure the laboratory and PEMS were distinguishing
the same 30 NTE events per cycle. Upon initial runs, the laboratory and PEMS missed several
NTE events. Causes for the missed NTE events included engine speeds running near the
governor that caused a drop in torque as well as highly transient speed and load profiles causing
torque or power to drop below the 30% NTE minimum values. In addition, the PEMS estimated
torque was often below the laboratory torque causing the torque or power to drop below the NTE
threshold levels. A number of slight speed and torque adjustments were made to the NTE cycles
before the laboratory and PEMS would consistently record 30 events per cycle.
SwRI Report 03.12024.06
Page 110 of 371
-------
Continuous engine speed data from the first 5 transient cycle repeats is shown in Figure
57, while torque traces are shown in Figure 58. In order to view the repeated data, the NTE
events were reordered and the transition events were removed from the data set. As seen from
the continuous data, many events contained highly dynamic speed and load combinations.
However, the laboratory was able to achieve very good repeatability in speed and torque for the
various events from cycle to cycle, even though the actual running order of events varied
considerably from one cycle to the next.
-R1 R2 R3 R4 R5 Event
2300
500
200
400 600
Time (sec)
800
1000
FIGURE 57. ENGINE 1 EXAMPLE SPEED TRACES DURING TRANSIENT TESTING
SwRI Report 03.12024.06
Page 111 of 371
-------
-R1
•R2 R3 R4 R5
- Event
1800
200
400 600
Time (sec)
800
1000
FIGURE 58. ENGINE 1 EXAMPLE TORQUE TRACES DURING TRANSIENT
TESTING
At the completion of transient testing, the gaseous concentration, exhaust flow rates, and
NOX mass rates were averaged over each NTE event for all 20 repeats, and then pooled. The 5th,
50th, and 95th percentile values of the averaged data were plotted versus the mean laboratory
dilute-to-raw measurements for each NTE event for reference purposes only, and to aid in data
review. This data processing structure was similar to the steady-state data analysis. In reviewing
the Engine 1 data, there was unexpectedly large variance for the several of the NTE events.
After further investigation, a number of outlying measurements were found in the data set. The
outlying points were found in both the laboratory raw and dilute measurements, as well as the
PEMS data. These outlying measurements were traced to changes in engine operation during the
NTE events. Shown in Figure 59, NTE Event 4 of transient cycle Repeat 2 shows a drastic drop
in NOX concentration, while the NOX concentration of the other repeats was relatively constant.
NTE Event 7 of Repeat 4 also shows a drop in NOX concentration. These engine operation shifts
may have been caused by different NTE modes orders or different transition events. The
underlying data processing method for the transient surfaces assumes that engine behavior will
be constant from run to run, and that any variance observed in the PEMS data is due to
measurement errors. Changes in engine behavior would therefore add additional, and potentially
overwhelming, variance error to the data.
SwRI Report 03.12024.06
Page 112 of 371
-------
-R1
-R2 R3 R4 R5
• Event
140
100
150 200 250
Time (sec)
300
350
400
FIGURE 59. ENGINE 1 NOX CONCENTRATION TRACES DURING TRANSIENT
TESTING SHOWING OUTLYING EVENTS
In order to prevent such outliers from artificially inflating the observed transient variance,
the Steering Committee decided to have SwRI remove the outlying data points at the May 2006
meeting in Ann Arbor. The removal of the outlying data was first done manually using scatter
plots and eliminating obvious outlying NTE points. A more rigorous outlier test was also
applied to the data set by SwRI statisticians. Outlier tests based on ASTM E 178 procedures
were used to identify outlying NTE data. All tests were made at the 5% level of significance.
The results from the statistical outlier tests and the scatter plot test gave similar results. Of the
600 NTE events generating during transient testing, 34 were deemed outliers and removed from
the data set.
A secondary task performed by SwRI was to evaluate the repeatability of the laboratory
and engine over 32-second NTE events. The laboratory brake-specific NOX emission results for
th
->th
-th
each NTE event were calculated and pooled. Figure 60 shows the 5 , 50 , and 95 percentile
lab dilute BS NOX results plotted against the mean BS NOX for the 30 NTE events repeated 20
times. This figure was generated with the outlying NTE points removed from the Engine 1 data
set. As shown in Figure 61, the median absolute deviation (MAD) was calculated for each of the
30 NTE events. Although some variation was observed, the MAD was generally in the range of
0.04 g/ (hp-hr), which is roughly 2 percent of the NTE threshold value of 2.0 g/(hp-hr) However,
some NTE events showed variance over 0.08 g/(hp-hr).
SwRI Report 03.12024.06
Page 113 of 371
-------
1.0
1.5
2.0 2.5 3.0 3.5
Lab Reference Dilute Mean BS NOx (g/hp-hr)
4.0
4.5
5.0
FIGURE 60. ENGINE 1 POOLED TRANSIENT TEST NTE BRAKE-SPECIFIC NOX
RESULTS
0.12
0.10
ji
a.
5
S 0.08
0.06
m
HI
u
c
HI
0.04
OL
0.02
0.00
• *
•
* "*-
0.5 1.0 1.5 2.0 2.5 3.0 3.5
Lab Reference Dilute Mean BS NOx (g/hp-hr)
4.0
4.5
5.0
FIGURE 61. ENGINE 1 POOLED TRANSIENT TEST NTE BRAKE-SPECIFIC NOX
MAD RESULTS
SwRI Report 03.12024.06
Page 114 of 371
-------
Although the laboratory data was not used to generate the transient error surfaces, plots
were generated to compare the PEMS performance with the laboratory measurements. As seen
in Figure 62 and Figure 63, the PEMS median values during transient testing nearly matched the
laboratory mean values for both NOX concentration and exhaust flow rate, respectively.
• Lab Reference
PEMS 5th % A PEMS 50th '
PEMS 95th '
800
700 -
600
.2 500
o 400
o
O
O 300
z
(/)
UJ 200
Q_
100
100
200 300 400 500 600
Lab Reference Dilute Mean NOx Concentration (ppm)
700
800
FIGURE 62. ENGINE 1 POOLED PEMS NTE NOX CONCENTRATION DATA VERSUS
THE LABORATORY MEAN
SwRI Report 03.12024.06
Page 115 of 371
-------
•Lab Reference
PEMS 5th % A PEMS 50th % • PEMS 95th '
1100
1000
o 900
800
3 700
LU
Q.
600
500 - - -
400
500
600
700 800 900
Lab Reference Mean Exhaust Flow Rate (scfm)
1000
FIGURE 63. ENGINE 1 POOLED PEMS NTE EXHAUST FLOW RATE DATA VERSUS
THE LABORATORY MEAN
4.5.2 Engine 2 Caterpillar C9 Transient
Engine 2 transient testing was conducted at the completion of the initial steady-state
testing. A process similar to Engine 1 transient testing was followed to perform the Engine 2
repeat NTE testing. The transient data generated with the Caterpillar C9 had no outlying data
due to engine operation, and the full data set was therefore used without alteration to generate the
Engine 2 transient error surfaces.
Similar to Engine 1, the repeatability of the laboratory and engine was evaluated by
comparing the brake-specific NOX emission results over the 20 NTE cycle repeats. Figure 61
shows the pooled BS NOX emission results for each of the 30 NTE events, while Figure 64 shows
the BS NOX MAD value for the 30 events. The MAD was generally around 0.04 g/(hp-hr),
similar to Engine 1.
SwRI Report 03.12024.06
Page 116 of 371
-------
Lab Reference I 5th % , 50th % » 95th
1.9
2.1 2.3 2.5 2.7
Lab Reference Dilute Mean BS NOx (g/hp-hr)
2.9
3.1
FIGURE 64. ENGINE 2 POOLED TRANSIENT TEST NTE BRAKE-SPECIFIC NOX
RESULTS
0.07
0.06
.c
I" 0.05
0.04
"£ 0.03
u
c
HI
0.02
&
2
0.01
0.00
+
• »
* *
1.7
1.9
2.1 2.3 2.5 2.7
Lab Reference Dilute Mean BS NOx (g/hp-hr)
2.9
3.1
FIGURE 65. ENGINE 2 POOLED TRANSIENT TEST NTE BRAKE-SPECIFIC NOX
MAD RESULTS
SwRI Report 03.12024.06
Page 117 of 371
-------
Although the laboratory data was not used to generate the transient error surfaces, plots
were generated to compare the PEMS performance with the laboratory measurements. As seen
in Figure 66, the PEMS median NOX concentrations during transient testing were bias slightly
low. As shown in Figure 67, the positive exhaust flow rate bias observed during Engine 2
steady-state testing was also evident during transient testing.
• Lab Reference • PEMS 5th % A PEMS 50th % • PEMS 95th '
500
450
E 400
Q.
_a
350
o 300
o
O
O 250
200
150
100
100
150
200 250 300 350 400
Lab Reference Dilute Mean NOx Concentration (ppm)
450
500
FIGURE 66. ENGINE 2 POOLED PEMS NTE NOX CONCENTRATION DATA VERSUS
THE LABORATORY MEAN
SwRI Report 03.12024.06
Page 118 of 371
-------
-Lab Reference • PEMS 5th % A PEMS 50th % • PEMS 95th %
950
350
350 400 450 500 550 600 650
Lab Reference Mean Exhaust Flow Rate (scfm)
700
750
800
FIGURE 67. ENGINE 2 POOLED PEMS NTE EXHAUST FLOW RATE DATA VERSUS
THE LABORATORY MEAN
4.5.3 Engine 3 International VT365 Transient
Engine 3 transient testing was conducted at the completion of the steady-state testing. A
process similar to Engine 1 and 2 transient testing was followed to perform the Engine 3 repeat
NTE testing. The transient data generated with the International VT365 had no outlying data and
was used without alteration to generate the Engine 3 transient error surfaces. Engine 3 transient
data showed higher variance than Engines 1 and 2. This was expected due to the higher variance
observed during steady-state repeat testing. For the Engine 3 data, both laboratory and PEMS
indicated a wider distribution of measurements with no obvious outlying points.
Similar to Engine 1 and 2, the repeatability of the laboratory and engine was evaluated by
comparing the brake-specific NOX emission results over the 20 NTE cycle repeats. Figure 68
shows the pooled BS NOX emission results for each of the 30 NTE events, while Figure 69 shows
the BS NOX MAD values for the 30 events. The Engine 3 MAD was at about 0.06 g/(hp-hr),
with some events over 0.1 g/(hp-hr). The higher variability of Engine 3 can be attributed to real
engine-out variations in NOX concentration. These values are roughly 3 to 5 percent of the 2.0
g/(hp-hr) threshold.
SwRI Report 03.12024.06
Page 119 of 371
-------
3.5
- Lab Reference • 5th % A 50th % » 95th '
3.0
a.
.c
2.5
m
2.0
HI
OL
1.5
1.0
1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 2.6
Lab Reference Dilute Mean BS NOx (g/hp-hr)
2.8 3.0
FIGURE 68. ENGINE 3 POOLED TRANSIENT TEST NTE BRAKE-SPECIFIC NOX
RESULTS
0.12
0.10
ji
a.
5
S 0.08
0.06
m
HI
u
c
HI
0.04
a:
.Q
0.02
0.00
1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4
Lab Reference Dilute Mean BS NOx (g/hp-hr)
2.6 2.8
3.0
FIGURE 69. ENGINE 1 POOLED TRANSIENT TEST NTE BRAKE-SPECIFIC NOX
MAD RESULTS
SwRI Report 03.12024.06
Page 120 of 371
-------
Although not used for transient error surface generation, the PEMS concentration and
exhaust flow rate data was plotted versus the laboratory mean values for comparative purposes.
Figure 70 shows the pooled PEMS NOX concentration data for each of the 30 repeated NTE
events. Interestingly, the low NOX concentration bias observed during Engine 3 steady-state
testing did not manifest in the transient data set, with all median PEMS NOX values near the
mean laboratory concentrations. As shown in Figure 71, the positive exhaust flow bias was
apparent in both the steady-state and transient testing.
• Lab Reference • PEMS 5th % A PEMS 50th % • PEMS 95th '
550
500
E 450
Q.
_a
c
.2 400
"
o 350
o
O
O 300
UJ 250
Q.
200 - - -
150
150
200 250 300 350 400 450
Lab Reference Dilute Mean NOx Concentration (ppm)
500
550
FIGURE 70. ENGINE 3 POOLED PEMS NTE NOX CONCENTRATION DATA VERSUS
THE LABORATORY MEAN
SwRI Report 03.12024.06
Page 121 of 371
-------
-Lab Reference
PEMS 5th % A PEMS 50th '
PEMS 95th %
500
450
o 400
350
3 300
LU
Q.
250
200
150
150
200
250 300 350
Lab Reference Mean Exhaust Flow Rate (scfm)
400
450
FIGURE 71. ENGINE 3 POOLED PEMS NTE EXHAUST FLOW RATE DATA VERSUS
THE LABORATORY MEAN
4.5.4 Transient Concentration Error Surface Generation
A number of steps were taken to generate transient error surfaces from the raw NTE data.
The concentrations used to generate the error surfaces were calculated as flow-weighted averages
over each NTE event. In other words, the continuous concentration data was multiplied by the
corresponding exhaust flow rate. The NTE event averaged concentration times exhaust flow rate
values were then divided by the NTE event averaged exhaust flow rate. The calculation of flow-
weighted concentrations was performed to capture transient variances that were pertinent to
emission calculations. For example, gaseous concentrations were multiplied by the exhaust flow
rate when calculating emission results, therefore, flow-weighting the concentration results
captured a more representative variance measurement.
As stated previously, the data generated by the laboratory during transient testing was not
used to generate the transient error surfaces. The averaged, flow-weighted PEMS concentrations
were pooled. The 5th, 50th, and 95th percentile values, as well as the MAD values, were
calculated for each of the 30 NTE events. To calculate the precision of the NTE testing, the 5th
and 95th percentile flow-weighted PEMS concentration values were subtracted from the 50th
percentile values. These delta values were then used to populate the transient error surface, with
the 95th percentile minus the 50th percentile concentration values set to the 95th percentile error
-th
th
values and the 5 percentile minus the 50 percentile concentrating values set to the 5
percentile error values. The 50th percentile error value was set to zero for all NTE events.
SwRI Report 03.12024.06
Page 122 of 371
-------
-th
-th
Figure 72 shows the PEMS 5 and 95 percentile error values plotted against the PEMS median
flow-weighted NOX concentration for Engine 1.
PEMS Median Flow-Weighted NOx (ppm)
FIGURE 72. ENGINE 1 UNCORRECTED TRANSIENT FLOW-WEIGHTED NOX
CONCENTRATION ERRORS
A final task performed on the transient error surfaces was to correct the variance
measured during transient testing for the variance already recorded during steady-state testing.
The variance correction was performed to insure steady-state precision errors were not double-
counted in the Model. The transient error surfaces would represent only the incremental
precision error associated with transient operation. The PEMS concentration MAD values from
both transient and steady-state testing were used to calculate a scaling factor. This scaling factor
was then used to shrink or collapse the transient error surfaces to remove the steady-state
variance. Figure 73 shows the transient and interpolated steady-state MAD values with the
resulting scaling factor. As anticipated, the transient MAD values were generally larger than the
steady-state MAD values.
SwRI Report 03.12024.06
Page 123 of 371
-------
20
•NOx ppm MAD TR
•NOx ppm MAD SS
-Scaling Factor
100 200 300 400 500
PEMS Median Flow-Weighted NOx (ppm)
600
700
- -1.0
-1.2
800
FIGURE 73. ENGINE 1 TRANSIENT AND INTERPOLATED STEADY-STATE MAD
VALUES WITH RESULTING SCALING FACTOR
To calculate the scaling factor, the steady-state PEMS MAD data was linearly
interpolated to generate steady-state MAD values at the 30 median PEMS concentration values
measured during NTE testing. In other words, the 10 steady-state PEMS concentration median
and MAD values were used with the 30 transient PEMS concentration median and MAD values
to linearly interpolate steady-state MAD values at the 30 transient median values. Next, the 30
interpolated steady-state MAD values were compared to the 30 transient MAD values. If the
steady-state interpolated MAD value was greater than the transient MAD value, the scaling
factor was set to zero. Otherwise, the scaling factor was calculated using the following equation.
Scaling _ Factor =
-MAD
MADt
The final corrected NOX concentration error surface for Engine 1 is shown in Figure 74.
With most scaling factor values greater than zero, the corrected error surface looks similar to the
uncorrected surface shown in Figure 72.
SwRI Report 03.12024.06
Page 124 of 371
-------
PEMS Median Flow-Weighted NOx (ppm)
FIGURE 74. ERROR SURFACE FOR ENGINE 1 TRANSIENT FLOW-WEIGHTED
NOX CONCENTRATION
A scaling factor of zero indicated the steady-state variance was greater than the transient
variance and mathematically collapsed the transient error surface value to zero. Although not
anticipated when designing the experiment, the steady-state variance was sometimes larger than
the transient variance, especially with Engine 3. Shown in Figure 75, the steady-state MAD
values were generally larger than the transient MAD values for Engine 3, resulting in zero level
scaling factors.
SwRI Report 03.12024.06
Page 125 of 371
-------
30
25
I 20
Q.
15
10
5
1.5
-1.0
0.5
r - - - - 0.0
D)
C
-0.5
- - -1.0
-1.5
100 150 200 250 300 350 400
PEMS Median Flow-Weighted NOx (ppm)
450
500
550
FIGURE 75. ENGINE 3 TRANSIENT AND INTERPOLATED STEADY-STATE MAD
VALUES WITH RESULTING SCALING FACTOR
Figure 76 shows the final corrected NOX concentration transient error surface for Engine
3. Due to the steady-state variance correction and zero level scaling factors, approximately two
thirds of the error surface points were zero values. This was problematic, especially when the
Engine 3 data was combined with data from the other two engines.
SwRI Report 03.12024.06
Page 126 of 371
-------
D)
I
25
20
15
10
5
0
1
-5
-10
-15
-20
-25
-95th % Delta —•— 5th % Delta
350 400 450 500 5
PEMS Median Flow-Weighted NOx (ppm)
FIGURE 76. ERROR SURFACE FOR ENGINE 3 TRANSIENT FLOW-WEIGHTED
NOX CONCENTRATION
Similar to the steady-state error surfaces, the final transient error surfaces were generated
by pooling the Engine 1, 2, and 3 final error surface data. Also similar to the final steady-state
error surfaces, the combined transient error surfaces were highly irregular. The unevenness of
the transient error surfaces was due to the variability of the transient delta data, the steady-state
variance correction, and error differences between the three engines. Shown in Figure 77, the
final NOX concentration error surface for transient testing was jagged and unpredictable.
SwRI Report 03.12024.06
Page 127 of 371
-------
PEMS Median Flow-Weighted NOx (ppm)
FIGURE 77. FINAL ERROR SURFACE FOR TRANSIENT FLOW-WEIGHTED NOX
CONCENTRATION
The transient error data was reviewed at the December 2006 Steering Committee meeting
in San Antonio. The Steering Committee suggested removing high variability Engine 3 steady-
state test points from the transient MAD correction to avoid collapsing the transient error
surfaces to zero. Unfortunately, the suggested correction had little impact on the transient error
surfaces, as illustrated in Figure 77. The Steering Committee decided the highly irregular error
surfaces may lead to erratic Model behavior. Additional analysis was performed by Steering
Committee members on the transient error data. The additional analyses confirmed that the data
for Engines 1 and 2 behaved as expected with larger transient MAD values as compared to
steady-state MAD values. Engine 3 generally showed a reversed trend, which was not expected.
It was initially proposed that Engine 3 data be eliminated from the final transient error surfaces.
The Steering Committee arrived at a solution that allowed most of the data from the three
engines to be used in the error surface generation, as originally intended in the Test Plan. The
solution was proposed in late December, and accepted by the Steering Committee on December
18, 2006 via email response
Steady-state and transient MAD data for the three engines was pooled into a single data
set. Selected outlier points were removed from the Engine 3 steady-state data set which showed
extremely large variations, as described earlier. In addition, some of the Engine 1 data points
were removed where the transient concentrations had been above all measured steady-state
concentrations for the engine, thus requiring extrapolation to generate steady-state MAD values.
The remaining data was pooled and root-mean-square (RMS) MAD values were generated for
both steady-state and transient data sets. The MAD values were compared to generate a transient
SwRI Report 03.12024.06
Page 128 of 371
-------
effect MAD. The steady-state MAD was subtracted from the transient MAD, and 5th and 95th
percentile values for the error surface were then generated using the following equations:
-MAD
Delta5thpercentaei = Concentration, * (-1.65* MAD terms)
Delta.
95thPercentile,
Concentration. * (+1.65 *MADte rms)
The 1.65 term in the equations above is the factor from a normal distribution which
covers 90 percent of the distribution around the median. This data analysis method essentially
produces an error surface which is a line, and makes the assumption that the transient errors are
dominated by span errors. This assumption is generally supported by the data.
The error surfaces for CO, NOX, and CO2 were all processed in this manner. In the case
of CO, the MADtranS;rms was actually less than the MADSSjrms, indicating that steady-state errors
were still larger than transient errors. Therefore, the CO transient error surface was set to zero
for all values. The final error surface values are given in Table 50 below. These values each
describe a pair of lines, with values at any given emission concentration determined via linear
interpolation.
TABLE 50. FINAL GASEOUS TRANSIENT ERROR SURFACE DELTAS
Pollutant /
Concentration
NOX
0
3000
C02
0
20
Percentiles
5th
50th
95th
delta, ppm
0.00
-72.03
0.00
0.00
0.00
72.03
delta, %
0.0000
-0.1904
0.0000
0.0000
0.0000
0.1904
1 Based on sampling with normal distribution
The transient concentration error surfaces are sampled normally in the Model, once per
NTE event. Concentration errors are linearly interpolated between x-axis points on the error
surface based on the reference NTE event concentrations and the error surface median x-axis
concentration levels. Transient error surface data can be found in Appendix G for all transient
testing.
4.5.5 Transient Flow Meter Error Surface Generation
Transient flow meter error surfaces were generated as described in the Transient
Concentration Error Surface Generation section of the report. Weighting was not performed
with the PEMS EFM data. The PEMS exhaust flow data was pooled and the 5th, 50th, and 95th
SwRI Report 03.12024.06
Page 129 of 371
-------
percentile values were calculated for each of the 30 repeated NTE events. Variance errors were
calculated by taking the difference between the 95th and 50th percentiles and the 5th and 50th
percentiles of the pooled PEMS exhaust flow rate data. Finally, the steady-state variance was
removed from the transient data set by calculating and applying a scaling factor based on the
interpolated steady-state MAD calculation and the transient MAD values.
The exhaust flow rate error surfaces were normalized as a percent of the maximum EFM
flow rate. Engine 1, 2, and 3 used the 5, 4, and 3-inch diameter EFMs, respectively. As
specified in the Sensors Inc. EFM user manual, the maximum flow rates for the 3-inch, 4-inch,
and 5-inch EFMs were 600, 1100, and 1700 scfm, respectively. Shown in Figure 78 through
Figure 80 are the transient exhaust flow error surfaces for engine 1, 2, and 3, respectively. Zero
values indicated the steady-state variability was larger than the transient variance at that exhaust
flow rate level.
-3.5
PEMS Median Exhaust Flow Rate (% of Max)
FIGURE 78. ERROR SURFACE FOR ENGINE 1 TRANSIENT EXHAUST FLOW
RATE
SwRI Report 03.12024.06
Page 130 of 371
-------
-3.5
PEMS Median Exhaust Flow Rate (% of Max)
FIGURE 79. ERROR SURFACE FOR ENGINE 2 TRANSIENT EXHAUST FLOW
RATE
-3.5
PEMS Median Exhaust Flow Rate (% of Max)
FIGURE 80. ERROR SURFACE FOR ENGINE 3 TRANSIENT EXHAUST FLOW
RATE
SwRI Report 03.12024.06
Page 131 of 371
-------
The combined, final exhaust flow rate error surface is shown in Figure 81. As with the
other final transient error surfaces, the final exhaust flow rate error was jagged. The unevenness
of the transient error surface was due to the variability of the transient delta data, the steady-state
variance correction, and error differences between the three engines.
-3.5
PEMS Median Exhaust Flow Rate (% of Max)
FIGURE 81. FINAL ERROR SURFACE FOR TRANSIENT EXHAUST FLOW RATE
The Steering Committee did not elect to re-analyze the transient EFM error surface data,
and therefore this error surface was used in the Model as shown in Figure 81.
4.5.6 Transient Dynamic Error Surface Generation
The dynamic error surfaces were generated to capture the variance of ECM-broadcast
speed and fuel rate measurements over the repeated 32-second NTE events. In addition, the
variance of the interpolated torque and BSFC from the 40-point maps was evaluated. The
generation of the dynamic error surfaces followed the procedure described in the Transient
Concentration Error Surface Generation section of the report and summarized in the Transient
Flow Meter Error Surface Generation section of the report. The dynamic error surface
generation process, however, did not include a steady-state variance correction. A steady-state
variance correction was not needed because the parameters evaluated for the dynamic error
surfaces did not have error surfaces for steady-state testing. Therefore, there was no concern of
double counting dynamic errors.
ECM-broadcast fuel rate was calculated as an average over the NTE event and received
no weighting. The dynamic fuel rate error surface was normalized using the engine's maximum
fuel rate, which was taken as the highest fuel rate recorded during the 40-point mapping
SwRI Report 03.12024.06
Page 132 of 371
-------
procedure. The Detroit Diesel Series 60 recorded a maximum fuel rate of 98 L/h, the Caterpillar
C9 measured 75 L/h, and the International VT365 delivered a maximum fuel rate of 46 L/h. The
final dynamic fuel rate error surface is shown in Figure 82. The fuel rate variance errors were
generally less than 1.0 % of the engine's maximum fuel rate.
PEMS Median Fuel Rate (% of Max)
FIGURE 82. FINAL ERROR SURFACE FOR DYNAMIC ECM FUEL RATE
ECM-broadcast speed was weighted using the interpolated torque from the 40-point
maps. The interpolated torque weighted ECM speed was calculated as an average over each
NTE event. The ECM speed error surface was normalized with ni0 speed equal to 0.0 % and n^
speed equal to 100 %. Table 51 shows the nlo and nhi speed definitions for each engine.
TABLE 51. NLO AND NHI SPEED DEFINITIONS FOR ENGINES 1, 2, AND 3
Engine 1 DDC
Engine 2 CAT
Engine 3 INT
nlo Speed (rpm)
1014
1099
1198
nhi Speed (rpm)
2129
2320
2839
The final combined dynamic ECM speed error surface is shown in Figure 83. The ECM-
broadcast speed showed little variation over the 20 repeated transient tests. The majority of the
5th and 95th percentile error terms were less than 0.2 % of normalized speed.
SwRI Report 03.12024.06
Page 133 of 371
-------
0.8
0.6
2
2
1!
1!
£
o.2
g 0.0
ai
a-
-0.2 -
LU
a.
-0.4
-0.6
PEMS Median Torque-Weighted Normalized Speed (%)
FIGURE 83. FINAL ERROR SURFACE FOR DYNAMIC ECM SPEED
Interpolated torque from the 40-point map was weighted using ECM-broadcast speed.
The ECM speed-weighted interpolated torque was calculated as an average over each NTE
event. The interpolated torque error surface was normalized as a percent of peak torque. Peak
torque measured during the lug curve tests at SwRI was 2195 N-m for Engine 1, 1464 N-m for
Engine 2, and 681 N-m for Engine 3. The final ECM speed-weighted interpolated torque error
surface in shown in Figure 84. Most variance errors were less than 1.0 % of peak torque for all
engines.
SwRI Report 03.12024.06
Page 134 of 371
-------
3.0
•x 2.0
re
'o
£ 1.0 J
HI
3
0"
0.0
20
-1.0
-2.0 -
-3.0
-4.0
PEMS Median Speed-Weighted Torque (% of Max)
FIGURE 84. FINAL ERROR SURFACE FOR DYNAMIC INTERPOLATED TORQUE
Interpolated BSFC from the 40-point map was weighted using ECM-broadcast fuel rate.
ECM fuel rate-weighted interpolated BSFC was calculated as an average over each NTE event.
Figure 85 shows the final dynamic interpolated BSFC error surface. Similar to the other
dynamic error surfaces, BSFC variability over the repeated NTE events was low, with most
variance errors less than 1.0 g/(kW-hr).
SwRI Report 03.12024.06
Page 135 of 371
-------
-3.0
PEMS Median Fuel-Weighted BSFC (g/kW-hr)
FIGURE 85. FINAL ERROR SURFACE FOR DYNAMIC INTERPOLATED BSFC
4.6 Interacting Parameters - Warm-Up Test Error Surface
The warm-up tests were conducted to evaluate ECM-broadcast torque and map errors due
to variations in oil viscosity, fuel temperature, oil temperature, and coolant temperature.
Because independently controlling these parameters was difficult, cold start tests were performed
to cumulatively estimate these ECM errors as the engine passed from cold to stable operating
temperatures. The errors associated with ECM fuel rate and ECM speed translated into torque
and BSFC errors through the 40-point map interpolation process. Warm-up tests were performed
on each of the three engines. The Detroit Diesel Series 60 and International VT365, both EGR
engines, were cooled to ambient temperature, approximately 18 °C, prior to the warm-up test.
The Caterpillar C9 was cooled to 0 °C for the warm-up test.
4.6.1 Interacting Parameters - Warm-Up Test Procedure
The original experimental design given in the Test Plan called for a single warm-up test
on each engine with the speed and load condition specified by the Steering Committee. The
initial choice was a high speed (Speed C), light load condition. However, when the first cycle
was run with the DDC engine, the intake manifold temperature never reached the NTE threshold
value. This data was decidedly unsatisfactory for the measurement allowance.
Following a discussion of the warm-up test results at the May 24, 2006 meeting in Ann
Arbor, the target speed and load was changed by the Steering Committee to Speed C and WOT
to insure the event would enter the NTE zone. Because the engine was at maximum operator
SwRI Report 03.12024.06
Page 136 of 371
-------
demand, actual torque varied throughout the warm-up test. The original intent of the warm-up
test was to hold torque constant throughout the cycle, therefore another test was run using Speed
C and a lower torque target (although still a high load point). Ultimately, the Steering
Committee elected to pool the data from both WOT and part-load tests. This decision was made
at the July 27, 2006 meeting in Ann Arbor after reviewing the results from both tests. Similar
tests were run for engines 2 and 3.
According to the finalized procedure, two 30-minute warm-up tests were run with each
engine. One test was run at C-speed and WOT, while the other test was performed at C-speed
and part load. The target torque values during the part load tests were set just low enough to
achieve constant torque control throughout the 30-minute warm-up cycle. The part load tests
were conducted by starting the engine and promptly ramping to the target speed and load, which
was held constant for the remainder of the cycle. The WOT tests were similar to the part load
tests, but the engines were ramped to the target speed and WOT. Using the recorded ECM speed
and fuel rate with the 40-point torque and BSFC maps, the interpolated torque and BSFC were
compared to the laboratory reference values. Although the 40-point BSFC map used fuel
consumption measurements from the laboratory fuel flow meter, BSFC calculated from the
dilute emission measurements was used as the lab reference for the warm-up tests. The
laboratory fuel flow meter system has an inherent time lag that would have resulted in incorrect
reference BSFC measurements during the transient warm-up test. In addition, there was also
concern with the fuel flow measurement accuracy due to the density change of the fuel during
the warm-up process.
In order to achieve cold start temperatures of 0° C, an insulating box was built to enclose
the Caterpillar engine. The partially built enclosure is shown in Figure 86, while the completed
insulating box is shown in Figure 87. The enclosure surrounded both the Caterpillar engine as
well the exhaust after treatment system. A re-circulating alcohol refrigeration system was used
with dry ice to achieve a heat sink with temperature below 0° C.
SwRI Report 03.12024.06 Page 137 of 371
-------
FIGURE 86. CATERPILLAR C9 ENGINE PARTIALLY ENCLOSED IN THE
INSULATING BOX PRIOR TO THE WARM-UP TEST
FIGURE 87. CATERPILLAR C9 ENGINE FULLY ENCLOSED IN THE INSULATING
BOX PRIOR TO THE WARM-UP TEST
4.6.2 Interacting Parameters - Warm-Up Data Analysis
The Test Plan did not initially include a method for how the data from the Interacting
Parameters Warm-up test would be used to generate an error surface. There was considerable
Steering Committee discussion of the course of several months regarding the appropriate
SwRI Report 03.12024.06
Page 138 of 371
-------
analysis of the data. The methodology was tentatively established following the completion of
Engine 1 testing, and was later adjusted with the completion of Engine 2 testing.
A number of concerns had to be balanced in the treatment of the warm-up data. On the
one hand, it was necessary to attempt to use the warm-up data to capture a wide range of possible
variations in engine fluid temperatures and viscosities. This was complicated by the fact that the
test was designed to explore cold temperatures and therefore only elevated viscosity levels. On
the other hand, there was a desire not to include any data that was not representative of operation
in the NTE zone.
An additional complicating factor was due to the interpolation of torque and BSFC from
the 40-point maps using ECM-broadcast speed and fuel rate. Because the warm-up cycle target
speed and load set points did not match a mode from the 40-point maps, a certain amount of
interpolation bias error was included in the data. This error was accounted for elsewhere in the
Model; therefore it was necessary to remove the bias due to the interpolation process prior to
generating error surfaces.
The data analysis method finally approved by the Steering Committee is described below.
Torque is used in the example, but the same methodology is also applied to BSFC. First, the
continuous data for the warm-up test was assembled, including the interpolated torque which was
generated via post processing. To remove the interpolation bias, data near the end of the warm-
up test, where all of the engine parameters had stabilized, was examined to generate an average
stabilized value for both the reference torque (from the laboratory torque-meter) and the
interpolated torque (based on ECM-broadcast speed and fuel rate). These two values were
compared in order to evaluate the interpolation bias error. This offset was then applied to the
continuous interpolated torque data set, shifting the data set to equalize the stabilized interpolated
torque values with the reference torque-meter values. An example of this bias correction is
illustrated in Figure 88.
SwRI Report 03.12024.06 Page 139 of 371
-------
1850
1800
1750 -
a 1700 -
O"
o
1650
1600
1550
•Lab Torque N-m
•Bias Cor. Interp. Torque
Start NTE Zone
Interpolated Torque
Bias Correction
= + 7Nm
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Time[s]
FIGURE 88 EXAMPLE OF WARM-UP TEST BIAS CORRECTION, DDC ENGINE
PART LOAD TEST
Once the interpolation bias correction was complete, the temperature data was examined
to determine when the NTE zone was entered. These entry points were based on the NTE zone
criteria given in CFR 40 Part 86.1370-2007. The primary trigger common to all three engines
was the aftertreatment outlet temperature, which must be 250 °C or higher. For Engines 1 and 3,
which were EGR equipped, additional trigger points are defined for engine coolant temperature
(ECT) and intake manifold temperature (IMT), as given in CFR 40 Part 86.1370-2007. An
example of the determination of NTE zone entry for Engine 1 is shown in Figure 89.
SwRI Report 03.12024.06
Page 140 of 371
-------
DPF Inlet Temp degC
•Actual ECT - CAN
-DPF Outlet Temp degC
-Target IMT
-Start NTE Zone
Actual IMT-CAN
-Target ECT
450
400
350 -
300
1- 200
DPF Outlet
Temp Crosses
250C
IMT Crosses Target - Start
NTE Zone @ 136 sec
150 -
100 -
50
100
150
Time[s]
200
250
300
FIGURE 89 EXAMPLE OF DETERMINATION OF NTE ZONE ENTRY FOR
WARMUP TEST, DDC ENGINE PART LOAD POINT
The continuous data was then examined to determine the maximum difference between
the bias-corrected interpolated torque and the reference torque after entry into the NTE zone. If
the difference resulted in a positive delta (interpolated minus reference), the value was set to the
95th percentile delta torque error value. The negative of the same value was set to the 5th
percentile error value for that test. If the delta from the data was negative, the value became the
-th
th
5 percentile for the test, while it's positive, or mirror-mage, became the 95 percentile delta.
The 50th percentile error values for all warm-up tests were set to zero. Torque was processed as
percent of maximum torque, while BSFC was calculated directly in engineering units.
Temperature, torque, and BSFC data is shown for each warm-up test in Appendix H.
These plots show temperature profiles related to the NTE zone, bias corrected interpolated torque
with laboratory reference torque, as well as bias corrected interpolated BSFC with laboratory
reference BSFC.
4.6.3 Interacting Parameters — Warm-Up Error Surface Generation
Using the process outlined above, torque and BSFC errors were calculated for each
warm-up test. Table 52 shows the torque deltas, while Table 53 summarizes the BSFC errors.
Torque errors for the Detroit Diesel Series 60 and International VT 365 engines were similar.
SwRI Report 03.12024.06
Page 141 of 371
-------
The Caterpillar C9 engine, with a cold-start temperature near 0 °C, had significantly larger
torque deltas. BSFC errors were similar among all engines with the exception of the Caterpillar
C9 part load test. The Engine 2 BSFC error was over three times as large as the deltas from the
Engine 1 and 3 part load tests.
TABLE 52. WARM-UP TEST TORQUE ERRORS SUMMARY
Engine
DDC HMD
CATMHD
INTLHD
Operating Point
C-Speed
WOT
78% Peak Torque
WOT
65% Peak Torque
WOT
73% Peak Torque
MEAN
5th % Torque Error
[% Peak Torque]
-3.4
-1.2
-14.2
-11.3
-3.5
-1.7
-5.9
50th % Torque Error
[% Peak Torque]
0.0
0.0
0.0
0.0
0.0
0.0
0.0
95th % Torque Error
[% Peak Torque]
3.4
1.2
14.2
11.3
3.5
1.7
5.9
TABLE 53. WARM-UP TEST BSFC ERRORS SUMMARY
Engine
DDC HMD
CAT MHD
INT LHD
Operating Point
C-Speed
WOT
78% Peak Torque
WOT
65% Peak Torque
WOT
73% Peak Torque
MEAN
5th % BSFC Error
[g/kW-hr]
-4
-7
-5
-24
-6
-7
-9
50th % BSFC Error
[g/kW-hr]
0
0
0
0
0
0
0
95th % BSFC Error
[g/kW-hr]
4
7
5
24
6
7
9
As decided by the Steering Committee, the mean values of the pooled torque and BSFC
deltas were used to create the final interacting parameters error surfaces. Figure 90 shows the
final warm-up torque error surface, while the BSFC error surface is shown in Figure 91. The
interacting parameters error surfaces are sampled normally and have a single x-axis point. The
warm-up deltas will be applied to each torque and BSFC value from the reference NTE events,
independent of level.
SwRI Report 03.12024.06
Page 142 of 371
-------
2 -,
E
ra
O"
o
0 -
-2 J
% Peak Torque Delta
5.9
0.0
-5.9
Single X-Axis Point
FIGURE 90. ERROR SURFACE FOR INTERACTING PARAMETERS - WARM-UP
DELTA TORQUE
2 -,
1 -
O 0
u.
-1 -
-2 J
BSFC Delta [g/kW-hr]
9
-9
Single X-Axis Point
FIGURE 91. ERROR SURFACE FOR INTERACTING PARAMETERS WARM-UP
DELTA BSFC
SwRI Report 03.12024.06
Page 143 of 371
-------
4.7 Torque and BSFC Interacting Parameters - Design of Experiment
The objective of the interacting parameters DOE experiment was to evaluate torque and
BSFC map errors due to a number of variable engine parameters. The list of parameters
included intake restriction, exhaust restriction, barometric pressure, and charge air cooler outlet
temperature. Because the 40-point maps were generated using nominal set points for the
parameters listed above, torque and BSFC values from the ECM interpolation would be
inaccurate due to engine parameter variations. The purpose of the interacting parameters DOE
was to compare the laboratory reference torque and BSFC with the interpolated values under a
broad range of engine operation. Ranges of adjustment for each parameter were defined
according to Table 54, which was copied from the Test Plan.
TABLE 54. INTERACTING PARAMETERS - DOE ADJUSTMENT GUIDANCE
Parameter
Intake air restriction
Exhaust gas restriction
Barometric pressure
Charge air cooler out
temperature
Minimum
Minimum capable*
Minimum capable*
82.7 kPa
Minimum per manufacturer
specifications and ambient
conditions**
Maximum
Max. allowed by
manufacturer*
Max. allowed by
manufacturer*
105 kPa
Maximum per manufacturer
specifications and ambient
conditions**
Consider removing after treatment to extend range of restrictions
Assume that a 1 deg. change in ambient temperature corresponds to a 1 deg. change in charge
air cooler out temperature
Although the program was run in a test cell capable of simulated high altitudes, the cell
could not simulate altitudes lower than approximately 689 feet. Therefore, the maximum
achievable barometric pressure was near 99 kPa, the typical atmospheric pressure for San
Antonio.
There was considerable Steering Committee discussion about exhaust backpressure set
points, because DPFs will be used on all 2007 engines. A final decision was reached on the
March 27, 2006 conference call. The Steering Committee agreed that the backpressure set points
should represent the minimum backpressure with a clean DPF installed, and the maximum
backpressure with a dirty DPF. That maximum was defined as the highest level of backpressure
the engine control system would allow before triggering an active regeneration based on DPF
differential pressure. SwRI was directed to obtain these values from the engine manufacturers
for each test engine.
The interacting parameter DOE test was performed on Engine 1 and Engine 3 only. For
each engine, SwRI worked with the engine manufacturers and the Steering Committee to define
appropriate adjustment ranges according to the guidance given in Table 54. A Design of
Experiment (DOE) test matrix (half factorial with resolution IV, 4 factors and 1 center point) was
used, resulting in nine test points. In addition, a tenth point was added by SwRI representing the
standard laboratory conditions used for steady-state and transient testing. In some cases, the
SwRI Report 03.12024.06
Page 144 of 371
-------
standard conditions were not at the center point of the adjustment range. The additional tenth
point was not originally intended as part of the error surface generation, but was to be used for
diagnostic and information purposes. The generic DOE test matrix is given in Table 55, while
specific set points used for each engine are given later in this section.
TABLE 55. INTERACTING PARAMETERS - DOE TEST MATRIX
Run#
1 -BL
2
3
4
5
6
7
8
9
10
Intake Air
Restriction
Center
Center
Min
Possible
Max
Allowed
Min
Possible
Max
Allowed
Max
Allowed
Min
Possible
Min
Possible
Max
Allowed
Exhaust
Restriction
Center
Center
Max Dirty
DPF
Min Clean
DPF
Min Clean
DPF
Max Dirty
DPF
Min Clean
DPF
Max Dirty
DPF
Min Clean
DPF
Max Dirty
DPF
Barometer
[kPa]
99.0
90.7
82.6
82.6
99.0
99.0
99.0
99.0
82.6
82.6
Inlet Air
Temp
[degC]
24
24
29
29
37
37
Min
Possible
Min
Possible
Min
Possible
Min
Possible
1 - Min Possible Inlet Air Temp = 9°C to 1 0°C
2 - Charge Air Cooler set point is Inlet Air Temp + Manufacturers'
allowed temperature rise
3 - Barometer of 99.0 kPa is estimated, actual max value varied
slightly due to ambient conditions
In order to allow for a larger range of adjustment of various parameters, the DPFs were
removed from the exhaust, and the LFEs used for intake air flow measurement were removed
from the intake air ducting.
Each DOE test matrix point was evaluated at five different steady-state load points. The
original mode definitions were given in the Test Plan as shown in Table 56.
TABLE 56. TEST PLAN DOE ENGINE OPERATING CONDITIONS
DOE Engine Operating Conditions (%speed, and %torque respectively)
17%, 32%
100%, 100%
59%, 49%
100%, 32%
100%, 100%
During the course of initial DOE testing on Engine 1, it was observed that the 100 % load
points were not repeatable because engine performance at WOT was not consistent across all of
the DOE test conditions. Therefore, the WOT points were adjusted to a lower level where the
torque set points could be maintained for all DOE tests. The 100 % speed points were also
SwRI Report 03.12024.06
Page 145 of 371
-------
lowered slightly to 97 % to insure repeatable load points. In addition, the modes were
lengthened to 10 minutes to allow for complete stabilization of the fuel flow measurement
system. These adjustments were approved by the Steering Committee during the May 23, 2006
meeting in Ann Arbor.
4.7.1 Interacting Parameters - DOE Data Analysis
During the evaluation of the Engine 1 DOE data, a consistent bias was evident in the
DOE torque and BSFC deltas for each test mode. Even the DOE test run with nominal engine
parameter set points showed a significant bias. It was found that the bias was the result of the
interpolation process which was used to generate the ECM torque and BSFC values. Because
interpolation error was already included in the error Model, the Steering Committee felt it was
necessary to remove the interpolation bias from the DOE error surface data. To address this
problem, SwRI proposed that the data from the additional baseline DOE test be used to generate
a bias correction for the nine DOE conditions. The correction was applied independently for
each of the five test modes. The Steering Committee approved this approach at the May 23,
2006 meeting in Ann Arbor. An example of this bias correction for Engine 1 torque data is
given in Table 57. In this example, the bias correction results in an upward shift of 1.1% to all
DOE data for Mode 1.
TABLE 57 EXAMPLE OF DOE BASELINE CORRECTION FOR ENGINE 1
DOE
Number
1 -Baseline
2
3
4
5
6
7
8
9
10
Delta Torq
Model
-1.1%
-1.0%
-0.2%
-0.6%
-0.9%
-0.1%
-1.2%
-1.2%
-1.3%
-0.4%
Mode 2
-1.8%
-1 .9%
2.1%
-1.1%
-1.0%
2.4%
-2.4%
-1 .6%
-2.6%
1.1%
ue (% of Peak Torque)
Mode3
-0.3%
-0.2%
0.4%
-0.1%
0.2%
1 .0%
-0.3%
-0.3%
-0.5%
0.0%
Mode 4
0.2%
0.3%
1 .3%
0.8%
0.6%
1 .3%
-0.2%
0.3%
0.0%
1.1%
ModeS
0.2%
0.9%
5.0%
2.6%
0.7%
3.9%
-0.6%
0.4%
0.4%
3.9%
Baseline Corrected Delta Torque (% of Peak Torque)
Model
N/A
0.1%
0.9%
0.5%
0.2%
1 .0%
-0.1%
-0.1%
-0.2%
0.7%
Mode 2
N/A
-0.1%
3.8%
0.7%
0.8%
4.2%
-0.6%
0.2%
-0.8%
2.9%
Mode3
N/A
0.0%
0.6%
0.1%
0.5%
1 .2%
-0.1%
0.0%
-0.3%
0.3%
Mode 4
N/A
0.1%
1 .2%
0.6%
0.4%
1 .2%
-0.4%
0.1%
-0.2%
1 .0%
ModeS
N/A
0.7%
4.8%
2.4%
0.5%
3.7%
-0.8%
0.2%
0.2%
3.7%
4.7.2 Engine 1 Detroit Diesel Series 60 DOE
The DOE matrix was run several times on the Detroit Diesel Series 60 engine as
adjustments were made to the test methodology. These changes were in response to the test
results and subsequent Steering Committee decisions. The final speed and torque set points used
for Engine 1 are given in Table 58. As noted above, these final points were different from those
used during the initial DOE run on this engine, due to the need to maintain the same torque level
for all DOE test conditions.
SwRI Report 03.12024.06
Page 146 of 371
-------
TABLE 58. INTERACTING PARAMETERS - DOE SPEED AND TORQUE STEADY-
STATE MODE DEFINITION FOR ENGINE 1
Model
Mode 2
ModeS
Mode 4
Mode 5
Speed
[% NTE]
17%
17%
59%
97%
97%
Torque
[% Peak]
43%
94%
49%
32%
71%
The actual engine parameter set points used for the Engine 1 DOE test matrix are given in
Table 59. The charge air cooler set point temperatures were based on a specification of 28 °C
temperature rise from ambient (inlet air) temperature. For inlet air temperatures at 10 °C, the
inlet air dew point temperature was lowered to 7 °C, rather than the standard set point of 15 °C,
in order to prevent condensation in the intake air stream.
TABLE 59. INTERACTING PARAMETERS - DOE TEST MATRIX FOR ENGINE 1
DOE
Number
1-BL
2
3
4
5
6
7
8
9
10
Intake Air
Restriction1
[kPa]
4.0
4.0
1.5
5.0
1.5
5.0
5.0
1.5
1.5
5.0
Exhaust
Restriction2
[kPa]
17
17
30
12
12
30
12
30
12
30
CVS
Pressure3
[kPa]
99
91
83
83
99
99
99
99
83
83
Boost After
Temp4
[°C]
52
52
57
57
64
64
38
38
38
38
Inlet Air
Temp5
[°C]
24
24
29
29
37
37
10
10
10
10
Dew Point
Temperature
[°C]
15
15
15
15
15
15
7
7
7
7
Notes:
1 . Minimum achievable intake air restriction was 1 .5 kPa
2. Maximum dirty DPF restriction was 30 kPa - Minimum clean DPF restriction was 12 kPa
3. Maximum achievable CVS pressure was 99 kPa
4. Temperature was set based on a fixed offset from the inlet air temperature
5. Minimum achievable inlet air temperature was 10 °C
The final baseline corrected errors for Engine 1 are given in Table 60 and Table 61 for
Torque and BSFC, respectively.
SwRI Report 03.12024.06
Page 147 of 371
-------
TABLE 60. INTERACTING PARAMETERS - DOE ENGINE 1 BIAS CORRECTED
TORQUE DELTAS
DOE
Number
1-Baseline
2
3
4
5
6
7
8
9
10
Baseline Corrected Delta Torque (% of Peak Torque)
Model
N/A
0.1%
0.9%
0.5%
0.2%
1 .0%
-0.1%
-0.1%
-0.2%
0.7%
Mode 2
N/A
-0.1%
3.8%
0.7%
0.8%
4.2%
-0.6%
0.2%
-0.8%
2.9%
ModeS
N/A
0.0%
0.6%
0.1%
0.5%
1 .2%
-0.1%
0.0%
-0.3%
0.3%
Mode 4
N/A
0.1%
1 .2%
0.6%
0.4%
1 .2%
-0.4%
0.1%
-0.2%
1 .0%
ModeS
N/A
0.7%
4.8%
2.4%
0.5%
3.7%
-0.8%
0.2%
0.2%
3.7%
TABLE 61. INTERACTING PARAMETERS - DOE ENGINE 1 BIAS CORRECTED
BSFC DELTAS
DOE
Number
1-Baseline
2
3
4
5
6
7
8
9
10
Baseline Corrected Delta BSFC Fuel Flow (g/kW-h)
Model
N/A
-2
-8
-5
-2
-7
2
1
1
-6
Mode 2
N/A
-1
-11
-3
-2
-14
3
-1
2
-7
Mode 3
N/A
-1
-6
-3
-2
-8
2
0
1
-5
Mode 4
N/A
-1
-9
-3
-2
-9
3
-1
1
-6
Mode 5
N/A
-3
-13
-8
0
-10
3
0
-1
-10
4.7.3 Engine 3 International VT365 DOE
The torque and speed set points used for Engine 3 are given in Table 62. Following the
direction of the Steering Committee, the points were selected to be identical to those used for
Engine 1. However, due to the shape of the torque curve for Engine 3, Mode 5 could not be run
at the desired combination of 97 % NTE speed and 71 % of maximum torque. In order to
maintain a consistent load point for use in the error surface, the speed set point was adjusted
down to the highest speed at which 71 % percent of maximum torque could be reliably
maintained at all DOE conditions. The mode 5 target speed was therefore adjusted from 97 %
NTE speed to 85 % NTE speed.
SwRI Report 03.12024.06
Page 148 of 371
-------
TABLE 62. INTERACTING PARAMETERS - DOE SPEED AND TORQUE STEADY-
STATE MODE DEFINITION FOR ENGINE 3
Model
Mode 2
ModeS
Mode 4
Mode 5
Speed
[% NTE]
17%
17%
59%
97%
85%
Torque
[% Peak]
43%
94%
49%
32%
71%
The engine parameter set points used for the Engine 3 DOE testing are given in Table 59.
Similar to Engine 1 testing, the engine manufacturer of Engine 3 was consulted to determine
appropriate set points for the DOE test matrix.
TABLE 63. INTERACTING PARAMETERS - DOE TEST MATRIX FOR ENGINE 3
DOE
Number
1-BL
2
3
4
5
6
7
8
9
10
Intake Air
Restriction1
[kPa]
3.5
3.5
0.7
3.7
0.7
3.7
3.7
0.7
0.7
3.7
Exhaust
Restriction2
[kPa]
17
17
24
12
12
24
12
24
12
24
CVS
Pressure3
[kPa]
99
91
83
83
99
99
99
99
83
83
Boost After
Temp4
[°C]
39
39
45
45
52
52
31
31
31
31
Inlet Air
Temp5
[°C]
24
24
29
29
37
37
10
10
10
10
Dew Point
Temperature
[°C]
15
15
15
15
15
15
7
7
7
7
Notes:
1 . Minimum achievable intake air restriction was 0.7 kPa
2. Maximum dirty DPF restriction was 24 kPa - Minimum clean DPF restriction was 12 kPa
3. Maximum achievable CVS pressure was 99 kPa
4. Temperature was set based on a fixed offset from the inlet air temperature
5. Minimum achievable inlet air temperature was 10 °C
The final baseline corrected data for Engine 3 is given in Table 64 and Table 65 for
Torque and BSFC, respectively.
SwRI Report 03.12024.06
Page 149 of 371
-------
TABLE 64. INTERACTING PARAMETERS - DOE ENGINE 3 BIAS CORRECTED
TORQUE DELTAS
DOE
Number
1-Baseline
2
3
4
5
6
7
8
9
10
Baseline Corrected Delta Torque (% of Peak Torque)
Model
N/A
0.5%
1 .9%
1.1%
0.7%
1 .0%
0.4%
1 .0%
1 .6%
1 .8%
Mode 2
N/A
2.4%
4.3%
3.7%
1 .8%
3.8%
-1 .5%
0.0%
0.9%
3.9%
ModeS
N/A
0.1%
-0.1%
-1.1%
0.4%
1 .4%
-0.8%
0.6%
-1 .5%
-0.8%
Mode 4
N/A
0.3%
1 .5%
1.1%
0.7%
1 .5%
0.4%
0.9%
1 .0%
1 .8%
ModeS
N/A
0.9%
1 .2%
0.9%
0.2%
2.4%
-0.6%
0.4%
-1 .2%
1 .7%
TABLE 65. INTERACTING PARAMETERS -DOE ENGINE 3 BIAS CORRECTED
BSFC DELTAS
DOE
Number
1-Baseline
2
3
4
5
6
7
8
9
10
Baseline Corrected Delta BSFC Fuel Flow (
Model
N/A
-3
-6
-3
-6
-8
-1
-4
-3
-6
Mode 2
N/A
-5
-9
-7
-4
-9
3
0
0
-7
ModeS
N/A
-2
-4
0
-3
-10
2
-4
3
-2
Mode 4
N/A
-3
-12
-9
-7
-14
-3
-8
-7
-15
3/kW-h)
ModeS
N/A
-3
-7
-5
-2
-11
3
-2
2
-8
4.7.4 Interacting Parameters - DOE Error Surface Generation
To generate the interacting parameters DOE error surfaces, the baseline corrected error
data for each mode was evaluated to generate a 5th, 50th, and 95th percentile delta across all nine
DOE conditions. The errors captured during this experiment included bias errors as well as
precision errors. However, the dynamic torque and BSFC error surfaces generated during
transient engine testing captured the precision errors associated with the interpolation process of
torque and BSFC. Not wanting to double count error sources in the Model, the variability of the
interpolation process was removed from the interacting parameters DOE error surfaces. This
was accomplished by shrinking the 5th, 50th, and 95th percentile delta values by the interpolated
torque and BSFC variance experienced during steady-state testing. The steady-state variance
was calculated as the mean of the 10 steady-state interpolated torque and BSFC MAD values
over the 20 repeats. The mean of the 10 MAD values was then used to collapse the raw DOE
error data. After the variance correction, the delta percentiles were then plotted with the x-axis
values calculated as the mean modal value.
SwRI Report 03.12024.06
Page 150 of 371
-------
The error correction due to the removal of the steady-state variance was minimal due to
the relatively small steady-state torque and BSFC MAD values. The mean interpolated torque
MAD values were 0.1 % of peak torque for Engine 1 and 0.3 % of peak torque for Engine 3.
The mean BSFC MAD values were 0.2 g/(kW-hr) for both Engine 1 and Engine 3. An example
of the MAD correction is show in Table 66. The bias corrected deltas, shown on the left, are
collapsed using the mean MAD value to generate the final MAD corrected deltas.
TABLE 66. EXAMPLE DOE STEADY-STATE VARIANCE CORRECTION USING
THE MEAN SS MAD
Bias Corrected
PEMS vs Lab
Delta
5th percentile
[ g/kW-hr]
-13.1
-7.6
-7.1
-11.9
-9.3
Bias Corrected
PEMS vs Lab
Delta
50th percentile
[ g/kW-hr]
-2.4
-1.7
-2.1
-2.7
-2.3
Bias Corrected
PEMS vs Lab
Delta
95th percentile
[ g/kW-hr]
2.8
1.8
1.5
1.8
2.0
SS BSFC
Mean of 10
MADs
[ g/kW-hr]
0.2
0.2
0.2
0.2
0.2
MAD Corrected
PEMSvsLab
Delta
5th percentile
[ g/kW-hr]
-12.9
-7.4
-6.9
-11.7
-9.1
MAD Corrected
PEMSvsLab
Delta
50th percentile
[ g/kW-hr]
-2.2
-1.5
-1.9
-2.5
-2.1
MAD Corrected
PEMSvsLab
Delta
95th percentile
[ g/kW-hr]
2.6
1.6
1.3
1.6
1.8
Figure 92 and Figure 93 show the corrected DOE torque and BSFC error surface data for
Engine 1.
% Peak Torque Lab
FIGURE 92. ERROR SURFACE FOR INTERACTING PARAMETERS - DOE ENGINE
1 DELTA TORQUE
SwRI Report 03.12024.06
Page 151 of 371
-------
-95th percentile
-50th percentile (median)
-5th percentile
-14
BSFC Lab [g/kW-hr]
FIGURE 93. ERROR SURFACE FOR INTERACTING PARAMETERS - DOE ENGINE
1 DELTA BSFC
Figure 94 and Figure 95 show the corrected interacting parameters DOE torque and
BSFC error surface data for Engine 3.
SwRI Report 03.12024.06
Page 152 of 371
-------
100.0
% Peak Torque Lab
FIGURE 94. ERROR SURFACE FOR INTERACTING PARAMETERS - DOE ENGINE
3 DELTA TORQUE
-95th percentile —•—50th percentile (median) —•—5th percentile
I
s
BSFC Lab [g/kW-hr]
FIGURE 95. ERROR SURFACE FOR INTERACTING PARAMETERS - DOE ENGINE
3 DELTA BSFC
SwRI Report 03.12024.06
Page 153 of 371
-------
To generate the final DOE error surfaces, the torque and BSFC errors from the Engine 1
and Engine 2 DOE matrix were pooled. The 5th, 50th, and 95th percentile deltas were then
calculated from the pooled data set. The variance correction was accomplished by calculating
the mean of the pooled steady-state interpolated torque and BSFC MAD values from Engine 1
and Engine 3. The mean MAD values for the pooled data were 0.2 % of peak torque and 0.5
g/(kW-hr) for BSFC. The final torque and BSFC error surfaces for the interacting parameters
DOE testing are shown in Figure 96 and Figure 97. The DOE error surfaces are sampled
normally. Having a broad range of x-axis torque and BSFC values, errors are linearly
interpolated from these error surfaces based on level.
1(10
% Peak Torque Lab
FIGURE 96. ERROR SURFACE FOR INTERACTING PARAMETERS - DOE DELTA
TORQUE FINAL
SwRI Report 03.12024.06
Page 154 of 371
-------
-95th percentile
50th percentile (median)
5th percentile
I
s
2! 10
BSFC Lab [g/kW-hr]
FIGURE 97. ERROR SURFACE FOR INTERACTING PARAMETERS - DOE DELTA
BSFC FINAL
4.8 Torque and BSFC Independent Parameters Sensitivity Analysis
The independent parameters test was conducted to evaluate torque and BSFC map errors
due to variations in intake air humidity and fuel properties. The Steering Committee had an
option to add additional parameters into this matrix, but other parameters were not added. This
test was performed only using Engine 2. The Test Plan called for SwRI to run a sensitivity
analysis using the parameters given in Table 67.
TABLE 67. INDEPENDENT PARAMETERS ADJUSTMENT GUIDANCE FROM THE
TEST PLAN
Sensitivity Parameter Set Points
Parameter
Intake air
humidity
Fuel properties
Minimum (#1)
Minimum possible (@30
deg. C); 0 grains/lb dry air
Fuel used in program
Mid. (#2)
50%RH(@30deg. C);
95 grains/lb dry air
Fuel selection #2
Maximum (#3)
95% RH (@30 deg. C)*;
180 grains/lb dry air
California ULSD
Run charge air cooler water inlet temperature of 30 deg. C
At each test condition, three steady-state modes were run according to the direction given
in the Test Plan and as shown in Table 68. Note that original mode definitions are given as NTE
percent speed and percent torque at speed.
SwRI Report 03.12024.06
Page 155 of 371
-------
TABLE 68. INDEPENDENT PARAMETERS TEST MODES FROM THE TEST PLAN
Sensitivity Engine Operating Conditions (%speed, and %torque respectively)
17%, 32%
59%, 49%
100%, 100%
Although the original Test Plan called for only running selected conditions across the
three modes, SwRI determined that the there was little difficulty in changing test conditions once
the test apparatus was set up and the fuels were procured. Therefore, SwRI elected to run a
complete test matrix for a total of nine test conditions.
As with the Interacting Parameters DOE, it was necessary to adjust the final test modes
slightly from those given in the Test Plan to position the points in the NTE zone and to insure the
target torque values could be maintained at all of the test conditions. The torque value for mode
1 was increased to insure the mode 1 power was always above the NTE limit of 30 % maximum
power. The speed and torque set points for mode 3 were decreased to pull the point away from
the governor line, thus insuring the mode was repeatable throughout the independent parameters
testing. Table 69 shows the final three modes of the steady-state test cycle run for the
independent parameters testing. The mode length was set to 10-minutes to insure stable fuel
flow measurement for the BSFC error surfaces.
TABLE 69. INDEPENDENT PARAMETERS SPEED AND TORQUE STEADY-STATE
MODE DEFINITIONS
Model
Mode 2
Mode 3
Speed
[% NTE]
17%
59%
97%
Torque
[% Peak]
43%
49%
56%
It was not possible to perform testing using an exhaustive matrix of fuel properties during
this program. Therefore, the Test Plan called for three fuels to represent a range of potential fuel
properties that might be available in the field. The first two fuels were specified in the Test Plan.
The first was the base ULSD 2-D certification grade fuel used during the program, while the
second was to be a representative California ULSD fuel. SwRI procured several drums of BP
ECD-1 ULSD fuel from California to meet this requirement. The third test fuel was to be
selected by the Steering Committee. Initially, SwRI proposed a very low aromatic (less than
10% by volume) fuel, but the Steering Committee felt that a high aromatic fuel would be more
representative of fuels available in the northern and eastern parts of the U.S. Therefore, SwRI
located a low API gravity ULSD test fuel from Chevron Phillips, which was selected as the third
test fuel by the Steering Committee. A summary of selected fuel properties for the three
Independent Parameters test fuels is given in Table 70.
SwRI Report 03.12024.06
Page 156 of 371
-------
TABLE 70. SELECTED FUEL PROPERTIES FOR INDEPENDENT PARAMETERS
TESTING
Property
Aromatics
Cetane Number
Viscosity
API Gravity
Sulfur
Distillation
10%
90%
Units
% vol
cSt @ 40C
ppm
deg F
deg F
Description
Test Fuels
Base Fuel
29.5
44.4
2.5
35.2
10
214
311
Haltermann
EPA 2-D Cert fuel
GARB ULSD
24.5
54.3
2.4
38.8
3
206
321
BP
EC Diesel-1
Third Fuel
35.0
40.4
2.6
33.1
6.2
207
344
Chevron Phillips
Low API ULSD
The mean intake air humidity levels recorded during testing were 4 gr/lb for the low
humidity points, 90 gr/lb for the middle humidity points, and 192 gr/lb for the high humidity
points. In order to reach the near zero humidity levels requested in the Test Plan, a specialized
humidity control system had to be used to condition the engine intake air. This custom-designed
system is incorporated into the intake air stream of the test cell at SwRI on an as needed basis in
order to achieve very low humidity levels, while imposing a very low additional restriction on
the intake air system. Shown in Figure 98, the humidity control system employs a large bed of
desiccant that is used to remove water from the intake air, and incorporates bypass legs and post-
bed cooling heat exchangers to maintain the desired intake air temperature and humidity
conditions.
SwRI Report 03.12024.06
Page 157 of 371
-------
FIGURE 98. INTAKE AIR LOW HUMIDITY CONTROL SYSTEM
4.8.1 Independent Parameters Data A nalysis
For the interacting parameters testing, ECM speed and ECM fuel rate were used with the
40-point maps to interpolate modal torque and BSFC values. These values were compared to the
laboratory reference measurements. As with the 40-point BSFC map, the reference BSFC was
calculated using the laboratory fuel flow meter. As expected, there were errors inherent in the
interpolation process using the 40-point maps. Because the interpolation errors were accounted
for in the Model, the Steering Committee elected to remove the interpolation bias errors. A
process was used similar that used for the Interacting Parameters DOE, wherein all data values
were bias corrected using the error values from a baseline condition. The baseline condition
chosen for the Independent Parameters test was Test Fuel 1 with normal 95 gr/lb humidity level.
Corrections were performed on a mode-by-mode basis.
Shown in Table 71 are the bias corrected interpolated torques versus laboratory torque
delta values for the Independent Parameters testing. Table 72 contains the bias corrected
interpolated BSFC versus laboratory BSFC delta values.
SwRI Report 03.12024.06
Page 158 of 371
-------
TABLE 71. INDEPENDENT PARAMETERS BIAS CORRECTED TORQUE DELTAS
Intake
Humidity
Min
Norm
High
Fuel
1
2
3
1
2
3
1
2
3
Delta Torque Bias Corrected (% of Peak Torque)
Model
-0.5%
0.8%
-1.1%
0.0%
1 .0%
-1 .0%
-0.2%
1 .9%
0.0%
Mode 2
-0.3%
1.1%
-0.9%
0.0%
1 .2%
-0.9%
0.0%
2.3%
0.1%
ModeS
-0.4%
1 .2%
-0.9%
0.0%
1 .7%
-1.1%
-0.3%
1 .7%
-0.7%
TABLE 72. INDEPENDENT PARAMETERS BIAS CORRECTED BSFC DELTAS
Intake
Humidity
Min
Norm
High
Fuel
1
2
3
1
2
3
1
2
3
Delta BSFC Bias Corrected (g/kW-h)
Model
3.5
2.3
0.5
0.0
2.8
0.6
-0.7
-4.1
-5.0
Mode 2
2.3
2.2
0.9
0.0
0.9
1.9
-0.7
-4.7
-4.0
ModeS
3.0
1.8
2.1
0.0
0.3
2.7
0.8
-2.3
-0.5
4.8.2 Independent Parameters Error Surface Generation
The Test Plan originally called for separate error surfaces to be generated for fuel and
humidity. However, while a clear trend was apparent with test fuel, no trends could be observed
related to humidity. Therefore, SwRI proposed that all the data be pooled into a single
Independent Parameters error surface each for torque and BSFC. Furthermore, the final torque
values for the three modes actually represented a relatively narrow range of torque, therefore no
trend in the data based on torque level could be determine. As a result, SwRI suggested that the
data for all three modes be pooled, and further that no x-axis be used on the final error surface.
The Steering Committee approved these changes during the November 21, 2006 conference call.
As a result, the overall 5th, 50th, and 95th percentiles were calculated from the pooled data sets
shown in Table 71 and Table 72.
Figure 99 shows the delta torque error surface for the independent parameters testing.
With only one x-axis point, the normally sampled delta torque values were applied to each torque
value in the reference NTE events in the Model.
SwRI Report 03.12024.06
Page 159 of 371
-------
2 -,
1 -
0 -
-1 -
-2 J
Delta Torque [% of Peak]
1.8
0.0
-1.0
Single X-Axis Point
FIGURE 99. ERROR SURFACE FOR INDEPENDENT PARAMETERS DELTA
TORQUE
Shown in Figure 100 is the delta BSFC error surface for the independent parameters
testing. Similar to the delta torque error surface, the BSFC surface was collapsed to a single x-
axis point. The pooled 5th, 50th, and 95th percentile values will be normally sampled and applied
to each reference NTE event BSFC value.
SwRI Report 03.12024.06
Page 160 of 371
-------
2 -,
1 -
o
u_
>
m
0 -
-1 -
-2 J
BSFC Delta [g/kW-hr]
3.0
0.6
-4.5
Single X-Axis Point
FIGURE 100. ERROR SURFACE FOR INDEPENDENT PARAMETERS DELTA BSFC
4.9 Torque and BSFC Interpolation Errors
During the design of the Test Plan, it was determined that a 40-point speed and fuel rate
matrix would be used to define an interpolation surface to predict Torque and BSFC from CAN
Speed and CAN Fuel Rate. While the 40-point matrix was used throughout the program, the
Steering Committee decided the 40-point matrix was too dense, placing an excessive mapping
burden on engine manufacturers. The Steering Committee determined that a 20-point matrix
would be more typical of actual field testing. However, the smaller matrix would lead to
increased interpolation errors.
The interpolation error surfaces were designed to capture the incremental error involved
in dropping from an interpolation surface based on a 40-point test matrix to one based on a 20-
point test matrix. The generation of these error surfaces was a computational exercise carried out
using the initial 40-point steady-state map data. For each engine, the Steering Committee down-
selected 20 points from the original 40 to generate the coarser grid. The 20-point maps selected
by the Steering Committee are shown in Figure 101 through Figure 103 for Engine 1, Engine 2,
and Engine 3, respectively.
SwRI Report 03.12024.06
Page 161 of 371
-------
1800
500 700 900 1100 1300 1500 1700 1900 2100 2300 2500
FIGURE 101. DETROIT DIESEL SERIES 60 DOWN SELECTED 20-POINT MAP
O"
onn
600
400
n
5C
'"/if \
^-^^
I
)0 1000
Selected Points for 20-Point Map
* ^^
• • • «
* * * * 1
^^~~~~~£} 0 9 0 9
^~~~— — A A A
»•$$;*•
1
1 500 2000 2500
Speed [rpm]
FIGURE 102. CATERPILLAR C9 DOWN SELECTED 20-POINT MAP
SwRI Report 03.12024.06
Page 162 of 371
-------
600
500
400
300
O"
o
200
100
Selected Points for 20-Point Map
r^t^t
500
1000
1500
2000
Speed [rpm]
2500
\'
3000
3500
FIGURE 103. INTERNATIONAL VT365 DOWN SELECTED 20-POINT MAP
For each engine a matrix of several thousand CAN-Speed and CAN-Fuel Rate
combinations was run using both 40-point and 20-point interpolation surfaces. The interpolated
torque and BSFC values were compared to generate the final deltas, with the 20-point values
subtracted from the 40-point values. An example of the results from this computational exercise
is shown in Figure 104 for Engine 1 interpolated torque. The results for all three engines are
given in Appendix I.
SwRI Report 03.12024.06
Page 163 of 371
-------
F
u
e
I
r
a
t
e
80-
60-
40-
20-
1200 1400 1600 1800 2000
Sfceed rpm
1.3466
0.7684
0.1902
-0.3880
-0.9662
-1.5444
-2.1226
-2.7007
-3.2789
-3.8571
Delta Tore
FIGURE 104. INTERPOLATED TORQUE ERROR (% PEAK TORQUE) BY SPEED
(RPM) AND FUEL RATE (G/S) FOR ENGINE #1
4.9.1 Interpolation Error Surface Generation
To generate the final error surfaces, the interpolated torque and BSFC 20-point versus 40-
point delta data was pooled for each engine, and 5th, 50th, and 95th percentile values were
generated. The percentile values for each engine were then averaged to generate the final deltas
for interpolation error surfaces. The error surfaces do not have an x-axis, as the interpolation
error was not found to scale with either speed or fuel rate, but remained relatively constant across
the entire performance map for each engine.
For torque the error surfaces are expressed as percent of maximum engine torque, while
BSFC errors are given in engineering units of g/(kW-hr). Each of these surfaces is sampled
normally, once per NTE event. The final error surface values are given in Table 73 and Table 74
for torque and BSFC, respectively. The error surfaces are depicted in Figure 105 and Figure 106
for torque and BSFC, respectively.
SwRI Report 03.12024.06
Page 164 of 371
-------
TABLE 73. TORQUE INTERPOLATION ERROR SURFACE VALUES
Engine
1
2
O
Number of
Points
8944
5741
5197
Averaged
Percentiles
5th
-0.82 %
-0.84 %
-1.00%
-0.89 %
50th
0.00 %
0.16%
0.01 %
0.06 %
95th
0.80%
2.55 %
1.34%
1.57 %
2 -,
1 -
o
Q.
HI
3
0"
.2
-1 -
-2 J
% Peak Torque Delta
1.57
0.06
-0.89
Single X-Axis Point
FIGURE 105. TORQUE INTERPOLATION ERROR SURFACE
TABLE 74. BSFC INTERPOLATION ERROR SURFACE VALUES
Engine
1
2
O
Number of
Points
8944
5741
5197
Averaged
Percentiles
5th
-3.96
-3.27
-0.12
-2.45
50th
0.49
0.05
1.98
0.84
95th
8.60
7.89
10.94
9.14
SwRI Report 03.12024.06
Page 165 of 371
-------
2 -,
1 -
o
m
0 -
-1 -
BSFC Delta [g/kW-hr]
9.14
0.84
-2.45
-2 J
Single X-Axis Point
FIGURE 106. BSFC INTERPOLATION ERROR SURFACE
4.10 Exhaust Flow Meter Testing
Exhaust flow meter testing was performed to evaluate potential bias errors due to
installation related factors. Flow meter testing included an exhaust pulsation test, an exhaust
swirl testing, and an exhaust tailpipe wind test. Using the Detroit Diesel Series 60 and 5-inch
EFM, steady-state tests were conducted to compare the SEMTECH-DS EFM flow rate to the
laboratory flow rate.
The ten steady-state points tested during the flow meter testing were identical to the
modes selected for Engine 1 steady-state repeat testing. The laboratory flow rate was determined
using a LFE to measure the intake air flow, a Micro-Motion fuel flow meter to measure fuel
flow, and the laboratory analyzers to measure raw exhaust emission concentrations. The intake
LFE measurement and the raw chemical balance were used with equation 1065.655-14 to
calculate the reference exhaust flow rate. As a check, the LFE air flow rate and measured fuel
flow were also used to calculate the exhaust flow rate using the CFR Part 89 raw exhaust flow
rate calculation. The two laboratory exhaust flow rate calculation methods gave nearly identical
exhaust flow rate results. The raw carbon balance error was modally calculated to insure the
laboratory reference exhaust flow rate was accurate to within two percent.
4.10.1 Pulsation Test
The pulsation test was performed to evaluate the bias and precision of the PEMS flow
meters when subjected to large pressure pulsations in the exhaust system. To conduct this test,
SwRI Report 03.12024.06
Page 166 of 371
-------
the DPFs were removed from the exhaust system, so that pulsations in the exhaust would not be
damped by its presence. The EFM was mounted 2 to 3 meters downstream of the turbocharger
outlet. Exhaust pipes, with lengths exceeding 10-diameters, were mounted before and after the
EFM. The exhaust was routed out the large overhead door of the laboratory, and was therefore
vented directly to the atmosphere.
The 10-mode steady-state test was repeated 5 times. The pooled SEMTECH-DS EFM
flow rate deltas versus the laboratory reference flows are shown in Figure 107. The PEMS flow
meters were biased high during the pulsation testing.
45
40
35
30
25
20
15
10
5
0
15th % A 50th % 95th '
A
-*•
140
240
340
440
540
640
740
840
940
1040
1140
Lab Reference Exhaust Flow Rate [scfm]
FIGURE 107. PULSATION TEST SEMTECH-DS EXHAUST FLOW RATE DELTAS -
RAW DATA
In order to avoid double counting exhaust flow errors, the bias recorded during steady-
state testing was subtracted from the pulsation test data. Figure 108 shows the EFM errors after
the mean steady-state exhaust flow rate bias was removed from the flow rate deltas. The steady-
state bias correction yielded a more uniform, positive exhaust flow rate bias. It should be noted
that, because this experiment was conducted using the 5-inch EFM, this steady-state bias
correction was small, because only a small amount of bias was observed during steady-state tests
involving the 5-inch flow meters (see Section 4.4.1 above).
SwRI Report 03.12024.06
Page 167 of 371
-------
45
40
35
I 30
I 25
o
20
15
15th % A 50th % *95th %
140 240 340 440 540 640 740 840
Lab Reference Exhaust Flow Rate [scfm]
940
1040
1140
FIGURE 108. PULSATION TEST SEMTECH-DS EXHAUST FLOW RATE DELTA -
CORRECTED FOR STEADY-STATE BIAS
4.10.2 Pulsation Error Surf ace Generation
Using the bias corrected EFM data, an exhaust flow rate pulsation error surface was
constructed for use in the Monte Carlo Model. The exhaust flow rate data was normalized using
the EFM maximum flow rate specification. In the case of the 5-inch EFM, the maximum flow
rate was 1700 scfm. Shown in Figure 109 are the flow rate delta values that were used in the
Model. Using normalized flow rate data from the reference NTE events, a flow rate delta was
normally sampled from the error surface. Linear interpolation was used for NTE reference
points within the data set. Points outside the data set were determined using the data set
maximum or minimum values, with no extrapolation beyond the values generated during testing.
The pulsation error surface generated positive exhaust flow errors when used in the Model.
SwRI Report 03.12024.06
Page 168 of 371
-------
o
£
>
2.5%
2.0% -
1.5% -
| 1.0%--
O
re
a
HI
Q
0.5% -
0.0%
-95th percentile
-50th percentile (median)
-5th percentile
10%
20%
30% 40% 50%
mol/s (lab,nom) / mol/s_max - [%]
60%
70%
FIGURE 109. ERROR SURFACE FOR PULSATION EXHAUST FLOW RATE
4.10.3 Non-Uniform Velocity Profile Swirl Test
The swirl test was conducted to evaluate PEMS flow meter errors when the EFM was
subjected to non-uniform flow velocity profiles upstream of the EFM. Two short radius 90°
elbows were connected in perpendicular planes to introduce exhaust swirl before the inlet of the
PEMS flow meter. The engine after-treatment system was installed during the swirl test. The
swirl exhaust system was also vented out the overhead door of the laboratory, directly to
atmosphere following the EFM.
Five repeats of the 10-mode steady-state test were run to characterize the EFM error due
to swirl. The pooled SEMTECH-DS EFM flow rate deltas versus the laboratory reference flows
for the swirl test are shown in Figure 110. The PEMS flow meter errors appeared to show a level
dependence, with increasing errors as flow rate increased.
SwRI Report 03.12024.06
Page 169 of 371
-------
30
25
20
15
10 -
5
-5
-10
15th % A 50th % *95th '
•
• A
200
600
800
1000
1200
t
Lab Reference Exhaust Flow Rate [scfm]
FIGURE 110. SWIRL TEST SEMTECH-DS EXHAUST FLOW RATE DELTAS - RAW
DATA
In order to avoid double counting exhaust flow errors, the bias recorded during steady-
state testing was subtracted from the swirl test data. Figure 111 shows the swirl test EFM errors
after the mean steady-state exhaust flow rate bias was removed from the flow rate deltas. The
steady-state bias correction eliminated much of the level dependency, resulting in a more
uniform, positive exhaust flow rate bias. . Again, it should be noted that, because this
experiment was conducted using the 5-inch EFM, this steady-state bias correction was small,
because only a small amount of bias was observed during steady-state tests involving the 5-inch
flow meters (see Section 4.4.1 above).
SwRI Report 03.12024.06
Page 170 of 371
-------
30
25
20
1 15
HI
5 10
o
5
-5
-10
I5th% A 50th % »95th%
-»--
A •
A"
200
400
600
800
1000
1200
Lab Reference Exhaust Flow Rate [scfm]
FIGURE 111. SWIRL TEST SEMTECH-DS EXHAUST FLOW RATE DELTAS
CORRECTED FOR STEADY-STATE BIAS
4.10.4 Swirl Error Surf ace Generation
Using the bias corrected EFM data, an exhaust flow rate swirl error surface was
constructed for use in the Monte Carlo Model. The exhaust flow rate data was normalized using
the EFM maximum flow rate specification. In the case of the 5-inch EFM, the maximum flow
rate was 1700 scfm. Shown in Figure 112 are the flow rate delta values that were used in the
Model. Using normalized flow rate data from the reference NTE events, a flow rate delta was
normally sampled from the error surface. Linear interpolation was used for NTE reference
points within the data set. Points outside the data set were determined using the data set
maximum or minimum values. The swirl error surface generated positive exhaust flow errors
when used in the Model.
SwRI Report 03.12024.06
Page 171 of 371
-------
E
o
c
.a"
E ^
> ^
1.2%
1.0%
0.8%
a E 0.6%
Q I
3 0.4%
0.2%
0.0%
10% 15% 20% 25% 30% 35% 40% 45%
mol/s (lab,nom) / mol/s_max - [%]
50%
55%
60%
FIGURE 112. ERROR SURFACE FOR SWIRL EXHAUST FLOW RATE
4.10.5 Tailpipe Wind Test
The tailpipe wind test was performed to determine EFM errors when the outlet of the
flow meter was subjected to high velocity air currents. The Steering Committee was initially
unsure if this experiment would result in significant errors, and how those errors would be
processed if they were found to be significant. Therefore, the Test Plan called for an initial
experiment to be run in order to determine the possible magnitude of this potential exhaust flow
error source. According to the Test Plan, if the initial experiment showed an error of less than 1
percent, no further experimentation would be performed, and the error surface would be dropped
from the Model. The experiment called for a high velocity air stream to be directed at the outlet
of the EFM at a variety of angles while the engine was operating at the 5 test modes. The flow
was to be designed to simulate a 60 mph wind velocity.
Figure 113 shows the experimental setup used for the initial test. A high velocity blower
system was used to direct air across the outlet of the EFM. A pitot tube device was used to
measure the air velocity at the outlet of the blower system. The air velocity was within the Test
Plan specification of 60 to 65 mph. Three steady-state tests, each consisting of the 5 steady-state
modes for EFM testing, were run with the blower system. One test was run with the high
velocity air stream perpendicular to the EFM, one with the air stream directed 45° into the EFM,
and one test with the air stream directed 45° out of the EFM. Figure 114 shows the three blower
orientations during the wind testing. A fourth steady-state test was run without the blower as a
baseline reference.
SwRI Report 03.12024.06
Page 172 of 371
-------
FIGURE 113. HIGH VELOCITY BLOWER SYSTEM
90-d
45-deg Out
Blower
45-deg In
Exhaust
FIGURE 114. EFM WIND TEST FLOW SCHEMATIC
Figure 115 shows the exhaust deltas for the three steady-state tests with the blower and
the one baseline test without the blower. The errors recorded during the blower tests where
similar to the baseline errors. When corrected for the baseline error, the blower deltas collapse to
SwRI Report 03.12024.06
Page 173 of 371
-------
near zero errors. Figure 116 shows the baseline corrected blower deltas. One outlying exhaust
flow delta was measured during the 45° out testing.
15
10
5
£
t -5]
-10
-15
-20
• PEMS BL * PEMS 90-deg • PEMS 45-deg Out A PEMS 45-deg In
A
200*
A
400
f^OO A
800
"lOOO
i
Lab Reference Exhaust Flow Rate [scfm]
30
FIGURE 115. SWIRL TEST SEMTECH-DS EXHAUST FLOW RATE DELTAS - RAW
DATA
SwRI Report 03.12024.06
Page 174 of 371
-------
15
10
5
-5]
-10
-15
-20
• PEMS 90-deg BPEMS 45-deg Out APEMS 45-deg In
200
600
I
-A-
•
A
800
I
A
1000
1200
Lab Reference Exhaust Flow Rate [scfm]
FIGURE 116. WIND TEST SEMTECH-DS EXHAUST FLOW RATE DELTAS -
CORRECTED FOR STEADY-STATE BIAS
As specified in the Test Plan, the results from the blower tests were reviewed by the
Steering Committee to determine if further testing and development of an error surface would be
needed for the tailpipe wind test. Figure 117 shows the mean baseline corrected exhaust flow
delta with 95 % confidence level error bars. This calculation was performed with and without
the one outlying flow rate delta. Because the 95 % confidence level bars nearly crossed zero
error, it was likely the errors generated from further tailpipe wind testing would be negligible. In
addition, the magnitude of all of the errors observed was considerably smaller than one percent
of the maximum flow for the flow meter. The Steering Committee therefore elected to not
perform further tailpipe wind testing and eliminated the wind exhaust flow rate error surface.
This decision was finalized at the June, 2006 Steering Committee meeting in San Antonio.
SwRI Report 03.12024.06
Page 175 of 371
-------
n n
c n
^Outlier Removed BOutlie
I
r Included
II
-•
FIGURE 117. WIND TEST MEAN DELTA VALUES WITH 95 % CONFIDENCE
LEVEL BARS
4.11 Torque and BSFC - OEM Supplied Error Surfaces
The purpose of the OEM supplied error surfaces was to capture ECM torque and BSFC
errors that could result from factors not characterized during this program. The additional error
sources included engine-to-engine production variability and the operation of non-deficiency
AECDs. As part of the Test Plan, participating engine manufacturers were asked to submit data
regarding these potential error sources to EPA. EPA was then tasked with analyzing the data and
developing a single error surface each for torque and BSFC which would combine the various
error sources.
Data was submitted by five engine manufacturers prior to the final deadline of August 1,
2006. EPA conducted an initial analysis, the results of which were reported to the Steering
Committee in a memo from EPA dated August 28, 2006. As part of the analysis, EPA held
private discussions with each manufacturer that submitted data, due to the confidential nature of
the information being submitted. Following the initial analysis, additional information was
requested regarding BSFC errors due to AECD operation, to resolve discrepancies in the data set.
Additional data was supplied by two manufacturers regarding this topic, after which EPA
completed a final analysis. The results of the final analysis were submitted to the Steering
Committee in a second memo dated November 2, 2006, and included a final proposal for the
error surface values. The Steering Committee approved the final form of the OEM error surfaces
at the November 2, 2006 meeting in San Antonio, as they appeared in the second memo. Copies
of both memos are included in Appendix J.
SwRI Report 03.12024.06
Page 176 of 371
-------
The final error surface values are summarized in Table 75 below. These surfaces are
sampled once per NTE event using a normal distribution. The error levels were determined to
scale with level, therefore the error surface values are defined as percent of point adjustments.
TABLE 75. OEM ERROR SURFACE DELTAS FOR TORQUE AND BSFC
Parameter
Torque
BSFC
Percentiles
-th
3 i
% point
-6.5%
-5.9%
50th,
% point
0%
0%
95th,
% point
+6.5%
+5.9%
Based on sampling with normal distribution
4.12 Time Alignment Error Surfaces
When processing the PEMS data recorded during transient dynamometer testing, the
question of time alignment was brought up by the Steering Committee. When using the PEMS
software to process test data, delay times for several variables can be used to time align the data
recorded from different sources. The variables that can be time aligned include gaseous
emission concentration data, the exhaust flow meter measurement, and the data recorded using
the vehicle interface. SwRI used the procedures detailed in Sensors Inc. Application Note #06-
001 titled Time Alignment of Raw Data to time align the data recorded during transient testing.
Transient data was also sent to Sensors to insure the data was time aligned correctly. Time
alignment of the SwRI transient data was relatively straightforward due to the sharp NTE event
entry and exit transitions, which were a deliberate part of the experimental design. Several
engine manufacturers indicated aligning data generated during field testing was often difficult
due to the difficulty of finding such clear transitions on many real-world field data sets. Several
examples of such difficult data sets were shared by Committee members during the course of the
discussions to illustrate the issue. With events as short as 30 seconds, small time alignment
errors result in significant differences in brake-specific emission results. Therefore, the Steering
Committee elected to account for time alignment errors in the Measurement Allowance Error
Model.
The first step in this analysis was to decide the level of typical alignment errors. Based
on input from the Engine Manufacturers, time alignment errors up to 1 second are possible,
however, errors near 0.5 seconds are more likely. Shown in Table 76, a matrix of time alignment
errors was generated. The gaseous emission delay times were left unchanged, while the EFM
and vehicle interface delay times were adjusted. To account for the relative likelihood of
occurrence, weighting factors were applied to each matrix point. Points with delay time errors
equal to or less than 0.5 were assumed to occur most often and received a relative weighting
factor of 8. Points with one variable at a 1 second delay time error and the other parameter at no
error received a weighting factor of 2. The diagonal points with the EFM and the vehicle
interface both having a delay time error of 1 second were assumed least likely to occur and
therefore received a relative weighting factor of 1.
SwRI Report 03.12024.06
Page 177 of 371
-------
TABLE 76. EFM AND VEHICLE INTERFACE ADJUSTMENT AND WEIGHTING
FACTORS USED FOR TIME ALIGNMENT ERROR GENERATION
Time
Alignment
Adjustment
Number
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
EFM
Adjustment
[sec]
-0.5
-0.5
-0.5
0.0
0.0
0.0
0.5
0.5
0.5
-1.0
-1.0
-1.0
0.0
0.0
1.0
1.0
1.0
Vehicle
Interface
Adjustment
[sec]
-0.5
0.0
0.5
-0.5
0.0
0.5
-0.5
0.0
0.5
-1.0
0.0
1.0
-1.0
1.0
-1.0
0.0
1.0
Relative
Weighting
Factor
8
8
8
8
8
8
8
8
8
1
2
1
2
2
1
2
1
One transient NTE test from each test engine was reprocessed using each of the time
alignment adjustment combinations shown in Table 76. Recall that each transient test was
comprised of 30 different 32-second NTE events. For each NTE event, brake-specific emissions
were calculated using each of the 3 calculations methods. The differences between the brake-
specific results calculated with the time alignment adjustments and the brake-specific results
calculated with the nominal time alignment were calculated as a percent of point for each NTE
event. With 30 NTE events per cycle and 17 time alignment combinations, 510 time alignment
errors were calculated for each engine. Once the time alignment error data was pooled for the 3
engines, a statistical routine was run to apply the specified relative weighting. The routine
essentially duplicated each error measurement as specified by the weighting factor, generating a
significantly larger pooled error data set. Finally, the 5th, 50th, and 95th percentile error values
were calculated for the pooled and weighted error data. The brake-specific time alignment error
data is shown in Table 77 for NOX, and Table 78 for CO. Time alignment data was not generated
for NMHC as nearly all PEMS THC measurements were zero.
SwRI Report 03.12024.06
Page 178 of 371
-------
TABLE 77. BRAKE-SPECIFIC TIME ALIGNMENT ERROR DATA FOR NOX
Engine
1
2
3
Pooled &
Weighted
Calculation
Method
1
2
3
1
2
3
1
2
3
1
2
3
NOx 5th
Percentile
[% of Point]
-3.3
-2.3
-1.6
-3.7
-0.9
-0.8
-2.0
-1.0
-1.6
-3.2
-1.3
-1.4
NOx 50th
Percentile
[% of Point]
0.0
0.0
0.0
-0.5
0.0
0.0
0.0
0.0
0.0
-0.1
0.0
0.0
NOx 95th
Percentile
[% of Point]
1.6
1.8
4.5
0.4
0.0
1.6
1.9
1.8
3.3
1.5
1.5
2.9
TABLE 78. BRAKE-SPECIFIC TIME ALIGNMENT ERROR DATA FOR CO
Engine
1
2
3
Pooled &
Weighted
Calculation
Method
1
2
3
1
2
3
1
2
3
1
2
3
CO 5th
Percentile
[% of Point]
-6.6
-5.9
-5.7
-8.4
-6.0
-5.6
-2.8
-3.0
-4.8
-7.5
-5.4
-5.2
CO 50th
Percentile
[% of Point]
-0.1
0.0
0.0
-0.1
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
CO 95th
Percentile
[% of Point]
5.2
5.1
12.3
5.8
6.1
12.7
2.2
3.6
12.5
4.6
5.1
12.3
Figure 118 graphically depicts the pooled and weighted brake-specific time alignment
errors for NOX and CO. In the Model, the time alignment errors were applied to the NTE BS
result with all errors applied just prior to the subtraction of the reference NTE brake-specific
result. The time alignment errors were sampled normally and were dependent on the calculation
method. An example of the time alignment error application is shown below.
Method 1 BSNOx (with full errors) = 3.8 g/kW-hr
Method 1 BSNOx (ideal) = 3.5 g/kW-hr
Method 1 BSNOx Time Alignment Error = 0.78197%
Delta BS NOx = (3.8 + (3.8*.0078197)) - 3.5 = 0.329714 g/kW-hr
SwRI Report 03.12024.06
Page 179 of 371
-------
15
10
HI
Q.
-10
0123.
Brake-Specific Calculation Method Number
FIGURE 118. POOLED AND WEIGHTED BRAKE-SPECIFIC TIME ALIGNMENT
ERROR DATA FOR NOX AND CO
SwRI Report 03.12024.06
Page 180 of 371
-------
5.0 ENVIRONMENTAL CHAMBER TESTING
5.1 Environmental Testing Objective
Environmental testing was performed with the SEMTECH-DS devices to quantify
gaseous emission concentration and exhaust flow measurement errors when the PEMS were
subjected to a variety of environmental disturbances. Environmental conditions evaluated during
the program included ambient temperature, ambient pressure, electromagnetic interference, and
ambient hydrocarbons. Each environmental test was designed to simulate environmental
disturbances that would likely be encountered during in-use field testing.
SwRI's Mechanical and Material Engineering Division (Division 18) performed the
environmental testing on site, as specified in the Test Plan and determined by the Steering
Committee. Eric Domes was the managerial Division 18 contact prior to and during the
environmental testing. Rick Pitman performed temperature, pressure, and vibration testing,
while David Smith and Herbert Walker performed the electromagnetic radiation testing.
5.2 Environmental Testing Procedure
During the various environmental tests, the performance of the PEMS was evaluated by
sampling and measuring reference gases. Bottled gases were selected to challenge each PEMS
gas analyzer at zero, audit, and span levels. The concentrations of the bottled gases were used as
the reference to evaluate the PEMS response to the various environmental disturbances. The
PEMS measured responses were compared to the reference concentrations to determine errors or
deltas during the environmental testing. The bottled reference gases and corresponding
concentrations are shown in Table 79. Reference gas concentrations were chosen based on
recommended audit and span levels in the Sensors Inc. user manual. AL size compressed gas
cylinders were procured from Scott Specialty Gases. During the program, the Scott gas bottle
concentration values were used as the reference. However, each Scott concentration was verified
by SwRI before being used for testing. The Test Plan originally specified the use of methane
audit and span bottles during environmental testing to challenge the PEMS methane analyzers.
However, the SEMTECH-DS methane analyzers were not accepted by the Steering Committee
as in-use field instruments and were not used in the Measurement Allowance Program. The
Steering Committee therefore elected to eliminate the methane reference gases from the
environmental testing procedure.
SwRI Report 03.12024.06 Page 181 of 371
-------
TABLE 79. REFERENCE GASES AND TYPICAL CONCENTRATIONS USED
DURING ENVIRONMENTAL CHAMBER TESTING
Bottle
Description
Zero Air
N02 Audit
NO2 Span
Quad Audit
Quad Span
THC
[ppm]
0
0
0
159.9
663
CO
[ppm]
0
0
0
178
960
CO2
[%]
0
0
0
6.04
12
NO
[ppm]
0
0
0
257
980
NO2
[ppm]
0
73
243
0
0
Balance
N/A
Air
Air
N2
N2
The reference gases were overflowed to the inlet of the PEMS sample lines during
environmental testing. Using an automated solenoid manifold, the reference gases were sampled
at a specific frequency and in a predetermined order. The Test Plan recommended sampling
each reference gas for 60 seconds. The first 30 seconds was intended to purge the system and
allow the analyzer responses to stabilize, with the final 30 seconds used to record a stable mean
measurement. Preliminary data indicated the NDUV NC>2 analyzers had not stabilized after the
30 second purge, therefore the purge duration was lengthened to 45 seconds for each reference
gas. With a 45-second purge time and 30-second sample length, each reference gas was sampled
for 75 seconds before switching to the next gas. Even after the 45-second purge, initial data
indicated the NC>2 concentration was still increasing after switching from the quad blend span gas
to the NC>2 span gas. The sample order of the reference gases was therefore set to minimize the
stabilization problem of the system, with audit gases preceding the corresponding span gases.
The reference gas sequence below decreased the stabilization times of the span gases and was
used throughout environmental testing.
1. Purified zero air reference gas
2. NO2 audit reference gas
3. NC>2 span reference gas
4. Quad blend audit reference gas
5. Quad blend span reference gas
During environmental testing, zero, audit, and span errors were recorded by comparing
the 30-second mean concentrations of the recorded PEMS measurements to the reference gas
concentrations. Although audit and span deltas could only be recorded when the corresponding
audit and span reference gases were being sampled, zero deltas were recorded whenever the
analyzer's audit or span gas was not flowing. For example, a zero delta was recorded for NC>2
during the zero air measurement as well as during the quad blend audit and quad blend span gas
measurements. Recording a zero delta for NO2 during the quad blend gas measurements was
possible because the quad blend gases contained negligible levels of NC>2. Likewise, zero deltas
were recorded for NO, CO, CO2, and THC during the NO2 audit and span measurements due to
the absence of the quad blend gases in the NO2 reference gases. The recording strategy used
during environmental testing is shown in Table 80. Again, it should be noted that the delta
recorded was always the actual analyzer reading minus the reference gas concentration.
SwRI Report 03.12024.06
Page 182 of 371
-------
TABLE 80. ZERO, AUDIT, AND SPAN DELTA RECORDING STRATEGY USED
DURING ENVIRONMENTAL CHAMBER TESTING
Bottle
Sequence
1
2
3
4
5
Bottle
Description
Zero Air
N02 Audit
NO2 Span
Quad Audit
Quad Span
THC
[ppm]
zero
zero
zero
audit
span
CO
[ppm]
zero
zero
zero
audit
span
CO2
[%]
See Note
zero
zero
audit
span
NO
[ppm]
zero
zero
zero
audit
span
NO2
[ppm]
zero
audit
span
zero
zero
Note:
Slow decay of CO2 following the quad span gas caused high zero measurements during the zero air test.
CO2 zero measurements after the quad span gas were not included in the delta data set.
After reviewing initial environmental data, it was evident CO2 zero deltas showed a
systematic trend. CC>2 zero errors recorded during zero air measurements were higher than zero
deltas recorded for NC>2 audit and NC>2 span gas measurements. Figure 119 shows the repeated
CC>2 zero delta behavior. The continuous PEMS concentration data indicated the high bias of the
zero air CO2 zero deltas were caused by the continuing decay of the CO2 measurement after
switching from the quad blend span gas to zero air. Figure 120 shows the response and slow
decay of a PEMS CC>2 analyzer when zero air is sampled after the quad span gas. After
reviewing the initial environmental delta data, the Steering Committee elected to remove the
biased CO2 zero delta recorded during the zero air measurement from the CO2 zero error
population.
SwRI Report 03.12024.06
Page 183 of 371
-------
0.020
0.015 -
^ 0.010
"oi
Q
C
o
2 0.005 -
"E
HI
u
c
o
0.000
8 *
-0.005 -
-0.010
High CO2 delta recorded during zero air reference gas measurement
62 64
68 70
74 76
78
Reference Gas Delta Observations
FIGURE 119. SYSTEMATIC HIGH CO2 BIAS DURING ZERO AIR REFERENCE GAS
MEASUREMENT
-CO2 [%] Quad Blend Start Zero Air Start - NO2 Audit Start NO2 Span Start Quad Audit
0.30
0.25
E
c
1 0-20
"E
HI
u
o 0.15
O
CM
O
° 0.10
0.05
n nn
Quad Blend Span
Zero Air
NO2 Audit
CO2 decay after quad blend span
Mr-w^
NO2 Span
reference gas
i
Stable CO2 measurement during NO2 au
r!E
Quad Blend
dit and span gases
TI
1025 1075 1125 1175 1225 1275 1325 1375 1425
Time [s]
FIGURE 120. CO2 CONCENTRATION DECAY DURING ZERO AIR MEASUREMENT
FOLLOWING THE QUAD BLEND SPAN REFERENCE GAS
SwRI Report 03.12024.06
Page 184 of 371
-------
During environmental testing, the PEMS were operated in a manner representative of in-
use field testing. Each SEMTECH-DS was started and allowed to thermally equilibrate while
sampling ambient air for 60 to 90 minutes prior to testing. The PEMS were then zeroed and
spanned as specified in the SEMTECH-DS user's manual. The PEMS were spanned with the
quad blend reference span gases measured during environmental testing. During testing, the
auto-zero feature of the PEMS was used to zero the devices hourly. At the completion of the
environmental chamber test, the PEMS were again zeroed and spanned. Span maneuvers were
therefore performed only at the beginning and end of each environmental test.
Zero air was used to zero the PEMS instruments throughout the environmental testing
program. Also, the PEMS were modified to use zero air as the SEMTECH-DS FID air source
rather than ambient air. The use of zero air eliminated potential hydrocarbon measurement
errors due to contaminated ambient air. The removal of ambient air hydrocarbon variability
during engine and environmental testing was essential because the Ambient Hydrocarbon
environmental test was specifically designed to capture FID measurement errors due to varying
levels and different species of ambient hydrocarbons. Therefore, zero air was used throughout
the program to avoid double counting measurement errors due to ambient hydrocarbons.
A Sensors Inc. 5-inch exhaust flow meter accompanied the SEMTECH-DS units during
environmental testing to evaluate the response of the PEMS EFM to various environmental
perturbations. One end of the EFM was capped to prevent air flow through the flow meter
during testing. Therefore, EFM measurements were recorded as 30-second mean zero errors
throughout each environmental test.
The SEMTECH-DS chassis is designed to house a small, high pressure FID fuel bottle.
Sensors Inc. recommends using the Scotty 104 aluminum gas cylinder from Scott Gas Company.
A full Scotty 104 FID fuel bottle can operate the FID for approximately 7 hours, which was not
sufficient for the 8-hour environmental chamber tests. Therefore, midway through each
environmental chamber test, the FID fuel bottle was replaced and the FID was re-zeroed and
spanned. During the 8-hour pressure test, the PEMS was enclosed in a sealed chamber, making
FID fuel bottle replacements impossible. Therefore, two FID bottles were plumbed in parallel
during the pressure test. Since the environmental chamber testing, Sensors Inc. has procured
FID fuel bottles with a higher pressure rating, allowing FID operation for over 8 hours.
5.3 Baseline Testing
Baseline testing was performed with three SEMTECH-DS devices to determine bias and
precision measurement errors for the PEMS with environmental conditions maintained at a
nominal level. It was assumed that each subsequent environmental chamber test would
inherently include the bias and precision errors recorded during baseline testing. Therefore, the
bias and variability errors measured during baseline testing were used to correct the
measurement errors generated during each environmental test.
Originally, PEMS 2, 5, and 6 were scheduled to be used for environmental testing.
However, during preliminary baseline tests, the PEMS 6 FID would not reach operating
temperature and would therefore not zero or span properly. Due to the environmental
SwRI Report 03.12024.06 Page 185 of 371
-------
temperature chamber schedule, it was necessary to complete the baseline testing as soon as
possible. In order to expedite baseline and temperature testing, PEMS 6 was replaced with
PEMS 3 from the dynamometer test lab. The use of PEMS 6 in the dynamometer laboratory was
deemed acceptable because Engine 1 testing was complete and Engine 2 testing had not started.
Therefore, all of Engine 2 and 3 testing was performed with PEMS 1, 4, and 6.
Baseline testing was performed in the Thermotron Walk-In temperature control chamber.
Although not used to control the ambient temperature, the chamber provided an environment that
was well ventilated, shielded from EMI and RFI, and maintained at relatively constant pressure
and temperature. The Walk-In chamber was also large enough to test 3 PEMS devices
simultaneously.
After the SEMTECH-DS devices and EFM had warmed and equilibrated, the PEMS
were zeroed and spanned. Next, the PEMS were set to sample the reference gases which were
controlled by the automated solenoid manifold and overflowed to the inlet of the SEMTECH-DS
sample lines. The PEMS measured the indexing reference gases for approximately 60 minutes,
after which the PEMS would perform an automated zero maneuver. Baseline testing was
conducted for 8 hours, generating 72 independent measurements for each gas. At the completion
of the 8-hour baseline test, the PEMS were zeroed and spanned.
The initial baseline testing indicated PEMS 2 and 3 had a NC>2 loss problem. Figure 121
shows the PEMS NC>2 delta data during the first 4 hours of baseline testing. As discussed in the
Environmental Test Procedure section, the NC>2 delta values were calculated by subtracting the
30-second mean PEMS NC>2 measurement from the NC>2 span bottle concentration. The initial
NC>2 delta values for all PEMS were accurate. However, as the test progressed, PEMS 2 and
PEMS 3 showed a decrease in NC>2 concentration measurements which resulted in large negative
deltas. Curiously, PEMS 2 and PEMS 3 biased NC>2 measurements recovered during the second
and third hours of the 8-hour baseline test.
SwRI Report 03.12024.06 Page 186 of 371
-------
10
15
20
25
30
35
g- 20
£j ~^u "
ra
"oi
Q 40
c -40-
0
TO
HI
U
c
o
O an
CM
O
<" -inn
LU
Q.
-140 -
if ;
A vv»v —.-<
•
A
A t
' *
' A
,..••••••!
>* A«« >*'
> * 9 • «^
,«**'
A
A
A
k
r •"•"•"•"•"!"
******
' A
;-.;-*"-
J»!
J
4
1
r»pi¥?r
k A^ A A A A
L PEMS 2
. PEMS 3
1 PEMS 5
— 1 Hour Marker
i • 1
A A t
1
. 1
i
Reference NO2 Span Gas Delta Observations
Bottle Cone. = 258 ppm
FIGURE 121. PEMS NO2 DELTA DATA DURING INITIAL ENVIRONMENTAL
BASELINE TESTING
The drastic loss of NC>2 during baseline testing prompted an investigation by SwRI and
Sensors Inc. Figure 122 shows the NO and NO2 response of PEMS 2 during the first hour of
baseline testing. During the first measurement of the NO2 span bottle, PEMS 2 reported a
concentration near the bottle concentration, yielding a relatively small delta measurement. As
expected, PEMS 2 reported near zero concentration levels of NO during the first measurement of
the NO2 span bottle. As the test progressed, the NO2 concentration measurement of PEMS 2
decreased significantly. As the measured NO2 concentration decreased, PEMS 2 reported
increased levels of NO during measurement the NO2 span gas. Because the NO2 span bottle
contained negligible levels of NO, measuring over 40 ppm of NO with PEMS 2 during the NO2
span gas measurement was unexpected. With reduced levels of NO2 and increased levels of NO,
it was apparent a NO2 to NO conversion was taking place. However, the sum of NO2 and NO
during the NO2 span gas measurement was still less than the NO2 span bottle concentration,
indicating NO2 was not only being converted to NO, but also lost.
SwRI Report 03.12024.06
Page 187 of 371
-------
• PEMS 2 NO ppm
PEMS 2 NO2 ppm
250
Gas Sequence: Zero Air, NO2 Audit, NO2 Span, Quad Audit, Quad Span (75 sec. each)
200
•g- 150 -
c
1 100
"E
ii*\J- W
1 ^~^ 500 1000
No NO response during NO2 span
r
span
V —
~^ frsi
1500
J^
V —
Red
S~.
c
/
K
uced
<
5
L
N02 res
I
^
M
pons
c
e during
/
I
/
~v
N02
C
span
IP
L
u
*= \^T* -\ w " ^
2000 \ 2500 3000 35
NO response during NO2 span
Time [s]
FIGURE 122. PEMS 2 NO AND NO2 RESPONSE DURING HOUR 1 OF INITIAL
ENVIRONMENTAL BASELINE TESTING
In trying to repeat the NC>2 loss problem, it was discovered that the PEMS units had to be
turned off or left idle for several hours before the NO2 loss phenomena could be repeated. PEMS
2 had to be left idle overnight to reproduce the results shown in Figure 121. To insure the SwRI
overflow gas delivery system was not causing the NC>2 loss/conversion problem observed with
PEMS 2 and 3, the Horiba OBS-2200 was fed gas from the SwRI supply manifold and operated
in NO mode during a repeated baseline type test. Similar to the initial baseline test, PEMS 2 and
3 showed a loss in NO2 and an increase in NO. The Horiba OBS-2200 showed no NO
concentration increase during the NO2 span gas measurement, indicating the NO2 loss and
conversion was not caused by the SwRI gas delivery hardware. Next, a test was run with the
sample time for zero air increased to 300 seconds to observe the effect of a lengthened zero air
purge. The sample times for the audit and span gases were left at 75 seconds. The NO2 loss with
extended zero air sample time was similar to the initial baseline test results.
Another test was performed with the quad blend audit and span gases removed from the
gas cycle sequence to determine if the presence of HC, CO, CO2, or NO was causing the NO2
loss problem. Although a slight NO2 loss was observed, the magnitude of the loss was greatly
reduced with the quad blend gases removed from the gas sampling sequence. A question then
surfaced about whether the reduction in NO2 loss was caused by the removal of HC, CO, CO2,
and NO, or the absence of sampling gases that contained no oxygen. Because the NO2 audit and
span gases are balanced with air, removing the quad blend audit and span gases (balance N2),
eliminated the sampling of gases with no oxygen. A test was therefore performed with N2 gas in
place of the quad blend audit and span gases. The gas sequence for this experiment was zero air,
NO2 audit gas, NO2 span gas, N2 and N2 again. All gases were sampled for 75 seconds. With
SwRI Report 03.12024.06
Page 188 of 371
-------
quad blend gases replaced with N2, the NC>2 loss was similar to the initial baseline test results.
Therefore, the NC>2 loss problem appeared to be dependant on the PEMS sampling oxygen-free
gases.
The next NC>2 conversion/loss test was performed with the PEMS filters and heated
sample line removed from the system. Again, results were similar to the initial baseline testing,
showing significant NC>2 loss. A test was then performed with the SEMTECH-DS
thermoelectric chiller bypassed in the sample handling system. With the chiller bypassed in the
system, the NC>2 measure was nearly perfect and showed no loss or conversion issues. Figure
123 shows the NC>2 and NO response for PEMS 2 with the thermoelectric chiller bypassed and
then reconnected. With the chiller bypassed, the NC>2 measurement was near the span bottle
concentration of 248 ppm. Once the chiller was reconnected in the sample handling system, the
NO2 loss/conversion problem became immediately apparent.
Concentration [ppm]
-i.-i.hOh
01 o 01 o c
3 O O O O C
-^n
Gas Sequence: Zero Air, NO2 Audit, NO2 Span, Quad Audit, Quad Span (75 se
1
L.
f
500
r-
Ik.
-sJ
1000
r-
•vyj T~— r VJ
1500
Chiller re
_N
k.
f
-*\
"i
c. each)
2000
connected
r-
f-
reivis i IN
PEMS 2 N
u ppm
O2 ppm
X
2500
rT
k.
3000
'
j.
Time [s]
FIGURE 123. PEMS 2 NO2 AND NO RESPONSE WITH THERMOELECTRIC
CHILLER BYPASSED AND RECONNECTED
PEMS 2 and 3 received new thermoelectric chillers. After installation of the replacement
chillers, no NO2 loss or conversion was evident. A possible explanation for the NO2 loss
problem is that the chillers' internal passivated coating may have been compromised. All PEMS
units, with the exception of PEMS 7, were used for emission testing prior to being sent to SwRI
for use in the Measurement Allowance Program. Not knowing the history of each PEMS, use or
misuse of the PEMS before arrival at SwRI may have caused the chiller NO2 loss/conversion
problem.
SwRI Report 03.12024.06
Page 189 of 371
-------
Environmental baseline testing was repeated with PEMS 2 and 3 after installation of the
new thermoelectric chillers. PEMS 5, showing no NC>2 loss or conversion during the initial
baseline test, did not undergo repeated baseline testing. Rather, PEMS 5 performed temperature
chamber testing shortly after the initial environmental baseline test. Immediately after
temperature testing, PEMS 5 was shipped to CE-CERT to avoid delaying the on-road model
validation testing.
The compiled zero delta data for the 8-hour environmental baseline test is shown in
Figure 124 for PEMS 2. During the test, 216 zero delta observations were recorded for each
gaseous emission with the exception of CC>2. Due to the elimination of the biased CC>2 zero
deltas measured while sampling zero air, 144 CO2 zero deltas were recorded during the baseline
test. With the replacement chiller, PEMS 2 showed NC>2 and NO zero deltas within ±5 ppm.
The NC>2 and NO deltas were added to produce the NOX zero delta measurement. Hydrocarbon
zero measurements were also accurate, with zero deltas less than 2 ppm. As discussed in the
Environmental Testing Procedure section of the report, the FID fuel bottle was replaced midway
through the baseline test; after which the FID was re-zeroed and spanned. Considering the 10
ppm resolution of the CO analyzer and past experience measuring positive CO biases through the
PEMS sample line, the 70 ppm range of CO zero deltas was not unexpected. The CO2 analyzer
provided accurate zero measurements with zero deltas within ±0.01 %. The environmental
chamber test results for PEMS 2, 3, and 5 are included in Appendix K.
THC [ppmC] • CO [ppm] NO [ppm] X NO2 [ppm] • NOx [ppm]
-New FID Bottle A CO2 [%]
0.030
Reference Gas Delta Observation
FIGURE 124. PEMS 2 ENVIRONMENTAL BASELINE ZERO DELTA
MEASUREMENTS
The audit delta observations for PEMS 2 are shown in Figure 125. During the 8-hour
baseline test, 72 audit deltas were recorded for each gaseous emission. As listed in Table 79, the
quad blend reference audit bottle concentrations were near 160 ppmC THC, 178 ppm CO, 6 %
CO2, and 247 ppm NO. The NO2 audit bottle concentration was near 73 ppm. The PEMS mean
SwRI Report 03.12024.06
Page 190 of 371
-------
gaseous measurements were compared to the reference audit bottle concentrations to generate the
audit delta values. NO audit deltas were centered around zero and generally within ±5 ppm.
NO2 audit deltas showed a negative bias of approximately 8 ppm. A noticeable positive shift in
the NO2 audit deltas is evident after observation number 45. The shift in the NO2 delta
measurement was due a zero calibration adjustment at the end of one of the 8 hour long segments
of testing. The zero adjust can also be seen in the zero delta data at observation number 45 in
Figure 124. Although the THC zero delta data was accurate, a positive bias of approximately 10
ppmC was evident with the THC audit measurement, indicating a possible span error. After
replacing the FID fuel bottle and re-zeroing and spanning the FID, the audit delta measurement
shifted to approximately 8 ppmC. CO2 showed a slight negative audit delta bias, with deltas
between -0.05 and 0.0 %. CO showed a slight positive audit delta bias, with deltas between 0
and 50 ppm. Bottle naming errors are included in the audit delta data set because the
SEMTECH-DS instruments were not spanned with the audit reference gases. However, the
reference gas concentrations were named by Scott Specialty Gas Company and checked by
SwRI. In general, the Scott and SwRI bottle measurements were within ±1.0 %. Baseline audit
deltas for PEMS 3 and 5 can be found in Appendix K.
THC[ppmC]
CO [ppm] X NO [ppm] X NO2 [ppm]
NOx [ppm] - New FID Bottle A CO2 [%]
0.000
0.010
0.020 rt
-A o.oso
-0.040 2
0.050
-0.060
-^ -0.070
-0.080
Reference Gas Delta Observation
FIGURE 125. PEMS 2 ENVIRONMENTAL BASELINE AUDIT DELTA
MEASUREMENTS
The span delta measurements for PEMS 2 are shown in Figure 126. During the 8-hour
baseline test, 72 span deltas were recorded for each gaseous emission. As listed in Table 79, the
quad blend reference span bottle concentrations were near 663 ppmC THC, 960 ppm CO, 12 %
CO2, and 980 ppm NO. The NO2 audit bottle concentration was near 243 ppm. The PEMS 30-
second mean gaseous measurements were compared to the reference span bottle concentrations
to generate the span delta values. With the replacement thermoelectric chiller, PEMS 2 NO2
measurements showed no significant negative bias and no conversion of NO2 to NO. NO2 span
deltas were generally within ±5 ppm. During the fist half of the baseline test, NO span deltas
SwRI Report 03.12024.06
Page 191 of 371
-------
were center around zero error. During the second half of the test, PEMS 2 NO measurements
drifted slightly negative, with NO deltas reaching -15 ppm. The THC span deltas were biased 10
ppmC high during the first half of the 8-hour baseline test. After replacing the FID fuel bottle
the FID was re-zeroed and spanned. After the THC zero and span maneuvers, the THC span
deltas were near zero. CO2 span deltas were between -0.02 and 0.04, while CO span deltas were
typically between -20 and 40 ppm. Because the PEMS were spanned with the reference span
gases used during baseline testing, bottle naming errors are not included span delta data set.
» THC [ppmC]
CO [ppm]
NO [ppm] X NO2 [ppm]
NOx [ppm]
- New Fl D Bottle A CO2 [%]
80
60 0.060
I 40
-40
A A
A A
..***
AA A
£t!3k
****.U**ci»*lfi
x^xx^o.
• •*•
^
X X
-20 -[-_-- --^-^-" -^--j-- -^- -4-0.020
%.
0.080
0.000
o
o
-0.040
Reference Gas Delta Observation
FIGURE 126. PEMS 2 ENVIRONMENTAL BASELINE SPAN DELTA
MEASUREMENTS
Deltas observed during environmental baseline testing were likely caused by a number of
factors. For example, the PEMS analyzers were zeroed and spanned through the zero and span
ports on the front of the SEMTECH-DS instruments. When using the zero and span ports, the
reference gases bypassed the majority of the sample handling system, including the stainless
steel cooler, the coalescing filter, and the thermoelectric chiller. Using a pneumatic path to zero
and span the analyzers that was different than the path used during sampling may have caused
environmental baseline errors.
Zero and span maneuver errors were also captured during the baseline test. The ability of
the PEMS to zero and span accurately was captured during environmental baseline testing.
Although 8 zero events occurred for each PEMS during baseline testing, only one span event was
performed for NO, NO2, CO, and CO2. THC was spanned twice during baseline testing. As
discussed in the environmental error surface sections of the report, having only one span event
for each environmental test complicated the extraction of PEMS measurement errors caused by
environmental factors. Deltas caused by span errors were often larger than the delta data for an
environmental test, thus resulting in biases that were not related to the environmental condition
SwRI Report 03.12024.06
Page 192 of 371
-------
being tested. The data for each environmental test and for each PEMS was therefore reviewed
before environmental error surfaces were calculated.
As discussed in the Environmental Test Procedure section of the report, a Sensors Inc. 5-
inch EFM was used to capture possible flow measurement errors due to environmental
disturbances. One end of the flow meter was capped to prevent air flow through the meter.
Throughout baseline testing, 30-second EFM flow rate averages were taken with each reference
gas observation. Shown in Figure 127, the observations were calculated as zero deltas for the
flow measurement system. Most 30-second mean measurements were below 0.6 scfm. Rated at
1700 scfm, the maximum observed flow meter error was less than 0.1 % of full scale. The
baseline EFM data was compared to the EFM data from other environmental tests to determine
flow measurement errors due to changes in environmental conditions.
100
150 200 250
Exhaust Flow Meter Zero Observations
300
350
FIGURE 127. 5-INCH EFM ENVIRONMENTAL BASELINE ZERO DELTA
MEASUREMENTS
5.4 Temperature Chamber Testing
Temperature chamber testing was performed with three SEMTECH-DS devices to
quantify PEMS gaseous concentration and exhaust flow measurement errors due to changes in
ambient temperature. The temperature test was designed to simulate real-world temperatures
and changes in temperature. Therefore, the temperature profile used during testing nearly
matched the atmospheric temperature distribution of EPA's 2002 National Emissions Inventory
(NEI) model. Taken from the Test Plan, Figure 128 shows the NEI temperature distribution as
well as the test cycle temperature distribution.
SwRI Report 03.12024.06
Page 193 of 371
-------
*inno/
OfiO/
yu/o
QftO/
oU/o
"5T
£ 7n<>/
— /U/o
"S Rn<>/
•s
a ww
>
O A(V>/
/
10%
n<>/
Temperature H
Real World Temperature 23 -100 F
Test Cycle Temperature
28.5 39.5 50.5 61.5
Temperatur
istograms
-m-=
72.5 83.5 94.5
e(F)
FIGURE 128. TEMPERATURE HISTOGRAMS FOR NEI MODEL AND TEST
PROFILE
The ambient temperature profile used for chamber testing was defined by a series of
temperature ramps with soaking periods between each transition. As written in the Test Plan,
Table 81 and Figure 129 define the 8-hour ambient temperature profile used during the program.
SwRI Report 03.12024.06
Page 194 of 371
-------
TABLE 81. TEMPERATURE TEST PROFILE DEFINITION
Ambient Temperature Test Sequence
Phase
1 Soak
2 Ramp
3 Soak
4 Ramp
5 Soak
6 Ramp
7 Soak
8 Ramp
9 Soak
10 Ramp
11 Soak
12 Ramp
13 Soak
Temperature
°C
13.89
13.89-5.00
-5.00
-5.00-12.78
12.78
12.78-28.33
28.33
28.33-37.78
37.78
37.78-22.22
22.22
22.22-13.89
13.89
°F
57
57-23
23
23-55
55
55-83
83
83-100
100
100-72
72
72-57
57
Time
min
10
5
5
145
40
5
52
5
8
100
60
5
40
Rate
°C/min
0.00
-3.78
0.00
0.12
0.00
3.11
0.00
1.89
0.00
-0.16
0.00
-1.67
0.00
Comments
Cool in-garage pre-test PEMS operations
Leaving cool garage into cold ambient
Operating at cold temperature outside of vehicle
Diurnal warming during cool day
Steady cool temperature during testing
Return to hot garage on a cool day
Hot in-garage pre- post- test PEMS operations
Leaving ho garage into hot ambient
Operating at hot temperature outside of vehicle
Diurnal cooling during hot day
Steady moderate temperature during testing
Return to cool garage on a moderate day
Cool in-garage post-test PEMS operations
Temperature-Time Environmental Test Cycle
-30
-40
-50
-60
— Temperature
1/2-hr moving average dT/dt (C/hr)
Vertical gridlines = hours
Vertical gridlines = 7-min gas cylinder cycle times
116
104
92
80
68
56
44
32
20
8
-4
-16
-28
-40
-52
-64
-76
4
Time (hr)
FIGURE 129. TEMPERATURE TEST PROFILE AND MOVING AVERAGE
Temperature testing was originally scheduled to be performed with a Thermotron Walk-
In temperature enclosure. The Walk-In chamber could easily house three PEMS devices,
therefore, temperature testing could be completed in one day. However, with the PEMS and
auxiliary hardware, it was unlikely the large Thermotron Walk-In would achieve the steepest
cooling ramps as defined in the Test Plan. Therefore, each PEMS was tested individually with a
smaller Thermotron SM-32 temperature control chamber, shown in Figure 130. The Thermotron
SwRI Report 03.12024.06
Page 195 of 371
-------
SM-32 used liquid nitrogen for supplemental cooling, and achieved all target temperatures and
ramp rates. The Thermotron SM-32 chamber housed the PEMS unit, EFM, and
temperature/relative humidity probe. The SEMTECH-DS heated sample lines, zero and span gas
lines, drain lines, and Ethernet cables were routed out of the chamber through ports on the
chamber sides.
FIGURE 130. THERMOTRON SM-32 TEMPERATURE CONTROL CHAMBER WITH
SUPPLEMENTAL LIQUID NITROGEN CYLINDER
Prior to executing the environmental temperature test, the PEMS were allowed to
thermally equilibrate while sampling ambient air for over one hour. The PEMS were then zeroed
and spanned at ambient temperature, approximately 23 °C. The environmental temperature test
was then started by ramping to the initial temperature soak point as specified in the Test Plan.
During the 8-hour temperature test, the PEMS were automatically zeroed every hour. The
temperature control chamber was not paused during the test, therefore, zero events occurred at
the temperatures defined by the Test Plan's temperature profile definition. Zero events occurred
near the hour markers shown in Figure 129. Similar to environmental baseline testing, zero,
audit, and span deltas were recorded by comparing the 30-second PEMS mean concentration
measurements to the reference gas concentrations. PEMS 3 performed temperature testing in a
Sensors Inc. environmental enclosure.
The zero deltas measured during the 8-hour temperature test are shown in Figure 131 for
PEMS 2. Mean temperature measurements from the PEMS temperature/relative humidity probe
are also shown in Figure 131. Analyzer zero drift caused by temperature variation was evident
throughout the temperature test. For example, during the steep temperature ramp at the
SwRI Report 03.12024.06
Page 196 of 371
-------
beginning of the cycle, NO, NO2, and CO drifted downward. The zero maneuver at the end of
the first hour of operation corrected the negative zero drift. Positive zero drift was evident
during the 4th and 5th hours of testing when the temperature was increasing aggressively. Again,
the hourly zero maneuvers continually corrected the zero drift. Slight negative drift occurred
during the last 3 hours of the 8-hour test, when the chamber temperature was decreasing. CO2
and THC measurements were largely unaffected by the temperature fluctuations experienced
during the environmental temperature test. Temperature data is included for all PEMS in
Appendix K. NO, NO2, CO, and CO2 behaved similarly with the three PEMS units during the
temperature tests. THC measurements with PEMS 3 and 5 showed slightly more susceptibility
to temperature induced zero drift than PEMS 2.
THC [ppmC] • CO [ppm] NO [ppm]
New FID Bottle Temp. [degC] A CO2 [%]
X NO2 [ppm]
NOx [ppm]
0.014
-0.004
Reference Gas Delta Observation
FIGURE 131. PEMS 2 ENVIRONMENTAL TEMPERATURE ZERO DELTA
MEASUREMENTS
Figure 132 shows the audit deltas measured during environmental temperature testing for
PEMS 2. For reference, the PEMS temperature probe mean measurement is plotted with the
audit delta values. During the first hour of the temperature cycle, the NO2 audit measurement
drifted negative, similar to the zero measurement. The zero maneuver at the end of the first hour
of testing not only corrected the zero drift, but also corrected the negative audit drift. The NO2
audit delta remained between -5 and -10 ppm for the last 7 hours of the 8-hour test. NO audit
deltas were between -10 and 10 ppm throughout the test. The THC audit deltas appeared
unaffected by temperature variation; however, the THC audit delta was near 20 ppm for the first
half of the test. After replacing the FID fuel bottle and zeroing and spanning the FID, the audit
deltas were below 10 ppm. CO and CO2 temperature audit deltas were similar to the audit deltas
observed during baseline testing. As seen in Appendix K, the audit delta behavior was similar
between the PEMS, with PEMS 3 and 5 showing slightly more THC temperature drift.
SwRI Report 03.12024.06
Page 197 of 371
-------
• THC [ppmC] • CO [ppm] X NO [ppm]
New FID Bottle Temp. [degC] A CO2 [%]
X NO2 [ppm]
NOx [ppm]
Reference Gas Delta Observation
FIGURE 132. PEMS 2 ENVIRONMENTAL TEMPERATURE AUDIT DELTA
MEASUREMENTS
Figure 133 shows the span deltas for PEMS 2 during the temperature test. NO2 span
deltas were minimal, generally within ±5 ppm. NO, CO, and CO2 span deltas were similar or
slightly more variable than the span deltas observed during baseline testing. PEMS 2 THC span
deltas showed large perturbations that followed a trend similar to the temperature profile. PEMS
3 and 5 also showed THC span deltas that were larger than the baseline test span deltas. An
explanation offered by Sensor Inc. in regard to the large PEMS 2 THC span delta measurements
was that the FID drain pressure may have been slightly elevated. The FID is sensitive to drain
backpressure, which may have been slightly elevated due to the extended length of the drain
lines during temperature testing. The Steering Committee elected to accept the temperature data
although the THC span delta data may have been influenced by the test setup. Due to the low
THC levels of the reference NTE events used in the Model, the THC span deltas would never be
used in the Model calculations. Because the NO, NO2, CO, and CO2 temperature delta data was
sound, and the THC span delta would not influence the Model, it was decided not to repeat the
temperature testing.
SwRI Report 03.12024.06
Page 198 of 371
-------
• THC [ppmC] • CO [ppm] X NO [ppm]
New FID Bottle Temp. [degC] A CO2 [%]
X NO2 [ppm]
NOx [ppm]
0.100
-0.050
Reference Gas Delta Observation
FIGURE 133. PEMS 2 ENVIRONMENTAL TEMPERATURE SPAN DELTA
MEASUREMENTS
A Sensors Inc. 5-inch EFM also underwent environmental temperature testing. The flow
meter with pressure transducer enclosure was placed in the temperature chamber during the 8-
hour test. Similar to baseline testing, one end of the EFM was capped to prevent air flow
through the meter. Figure 134 shows the 30-second mean flow meter measurements during the
temperature test. The zero deltas observed during temperature testing largely resembled the
EFM deltas recorded during baseline testing. Two periods midway through the temperature test
showed slightly increase EFM measurements. One perturbation occurred after the 150th mean
delta measurement, while the other occurred between observation number 200 and 250. The
deviations from zero were small, with the maximum zero error under 0.3 % of the EFM's rated
flow range of 1700 scfm.
SwRI Report 03.12024.06
Page 199 of 371
-------
5.0
4.5
4.0
3.5
3.0
2.5
2.0
1.5
1.0
0.5
0.0
* •
* *
50
100 150 200 250
Exhaust Flow Meter Zero Observation
300
350
FIGURE 134. 5-INCH EFM ENVIRONMENTAL TEMPERATURE ZERO DELTA
MEASUREMENTS
5.4.1 Temperature Error Surface Generation
Because the temperature test was designed to simulate real-world temperatures and
changes in temperature, the deltas measured during temperature testing were randomly sampled
in the Model. However, it was assumed that temperature chamber testing would inherently
include the bias and precision errors recorded during baseline testing. Therefore, the bias and
variability errors measured during baseline testing were used to correct the measurement errors
generated during each environmental test.
The initial step in generating the temperature error surfaces was to correct each
temperature measurement error for any bias measured during baseline testing. This discussion
focuses on NOX concentration, however, the same process was applied to each gaseous emission.
The median baseline NOX delta was calculated for each PEMS at the zero, audit, and span levels.
The median baseline deltas were subtracted from each delta measured during temperature
testing. For example, the PEMS 2 median baseline zero delta was subtracted from each PEMS 2
delta recorded during temperature testing. A similar procedure was performed for the audit and
span deltas. The median environmental baseline NOX concentrations for PEMS 2, 3, and 5 are
shown in Table 83. To remove the baseline variability from the temperature test data,
multiplicative scaling factors were calculated. The median absolute deviation (MAD) was
calculated for each baseline delta data set as well as the bias corrected temperature delta data set.
Scaling factors were calculated using the equation below. The scaling factors, shown in Table
82, were multiplied to each bias corrected temperature delta to reduce the variability of the data.
Similar to the bias correction, the variability correction was performed for each PEMS and at the
zero, audit, and span levels.
SwRI Report 03.12024.06
Page 200 of 371
-------
Scaling _ Factor =
-M4/
MAD
Rod
TABLE 82. MEDIAN ENVIRONMENTAL BASELINE NOX CONCENTRATIONS AND
ENVIRONMENTAL TEMPERATURE SCALING FACTORS
Zero
Audit
Span
PEMS2
PEMS3
PEMS5
PEMS2
PEMS3
PEMS5
PEMS2
PEMS3
PEMS5
Median
Baseline NOx
Delta
[ppm]
0.9
2.3
3.2
-6.1
-6.3
-10.5
-2.9
-4.7
-7.5
NOx
Temperature
Scaling Factor
0.85
0.66
0.77
0.90
0.93
0.88
0.94
0.91
0.97
Figure 135 through Figure 137 show the corrected zero, audit, and span NOX temperature
deltas for each PEMS. The corrected temperature deltas for all gaseous emission can be found in
Appendix K. The trends of the NOX deltas measured during temperature testing could be linked
to changes in temperature by comparing the deltas with the chamber temperature profile. Also,
the 3 PEMS showed similar NOX delta patterns, indicating a susceptibility to ambient
temperature. Therefore, inclusion of the NOX temperature error surface was justified. CO, CO2,
and NMHC delta trends were not as easily linked to changes in temperature. Also, the CO, CO2,
and NMHC delta patterns for the 3 PEMS were not as tightly matched as for NOX. Therefore, it
was not clear whether the CO, CO2, and NMHC deltas were caused by the ambient temperature
test or by some other factors. This problem was presented to the Steering Committee. After
review of the data and recommendations by SwRI, a decision was reached to include temperature
error surfaces for all of the gaseous emissions. Justification for the inclusion of all pollutants
included the following.
1. The variance of the temperature data was generally larger than the baseline data,
indicating an ambient temperature susceptibly.
2. The deltas from one or more PEMS showed a subtle correlation to the chamber
temperature profile.
3. There was slight agreement of the delta patterns between PEMS, indicating a common
error source.
SwRI Report 03.12024.06
Page 201 of 371
-------
- PEMS 2 * PEMS 3 PEMS 5
25 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
Reference Gas Delta Observation
FIGURE 135. ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE NOX
CONCENTRATION ZERO DELTA MEASUREMENTS
15
10 --
5 -
- PEMS 2 • PEMS 3 • PEMS 5
0 -
-5 ---
Q
c
o
B
re
m
o
c
o
o
o -10 -
z
-15 --
. -_*.--_
-20 H—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—h
0 10 20 30 40 50 60 70
Reference Gas Delta Observation
FIGURE 136. ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE NOX
CONCENTRATION AUDIT DELTA MEASUREMENTS
SwRI Report 03.12024.06
Page 202 of 371
-------
40
30 -
- PEMS 2 • PEMS 3 • PEMS 5
I 20
Q.
C
O
B
re
m
2
o
O
x
O
10 --
0 -- -<
-10 --
-20 -
-30
H—I-
-+-
H—I-
10
20 30 40
Reference Gas Delta Observation
50
60
70
FIGURE 137. ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE NOX
CONCENTRATION SPAN DELTA MEASUREMENTS
The baseline corrected deltas were sampled randomly in the Model. The Model was
initially programmed to randomly sample 360 zero, audit, and span observations for each PEMS.
With temperature data for 3 PEMS, the Model was programmed to use 1080 zero, audit, and
span deltas. PEMS 2 data was used for observation 1 to 360, PEMS 3 data was used for
observation 361 to 720, and PEMS 5 data was used for observation 721 to 1080. However,
during temperature testing, only 216 zero observations, 72 audit observations, and 72 span
observations were recorded for each PEMS. The data for each PEMS was expanded by
repeating delta observations to generate 360 zero, audit, and span observations for each PEMS.
The final NOX error surface for environmental temperature testing is shown in Figure 138. Final
temperature error surfaces for CO, CO2, and NMHC can be found in Appendix K.
SwRI Report 03.12024.06
Page 203 of 371
-------
-30
200
400 600 800
NOx Temperature Observation (1 to 1080)
1000
- NOx SPAN 1209ppm NOx
• NOx Audit 341 ppm NOx
— NOx Ze ro 0 ppm NOx
FIGURE 138. FINAL ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE
NOX CONCENTRATION
For each cycle of the Model, a number from 1 to 1080 was randomly selected. At the
selected observation, zero, audit, and span delta values were sampled. Based on the
concentrations of the reference NTE events, delta values were linearly interpolated from the
zero, audit, and span delta data.
On occasion, the variance measured during environmental baseline testing was greater
than the variance measured during temperature testing. During these instances, a scaling factor
could not mathematically be calculated. Therefore, each delta observation was set to the
difference between the median delta value measured during temperature testing and the median
delta value measured during the environmental baseline testing. An example of this correction is
shown in Figure 139 for CO span deltas.
SwRI Report 03.12024.06
Page 204 of 371
-------
0.0070 -i
0.0060 - * -
0.0050 --
0.0040 --
| 0.0030
o
o
0.0010 +
0.0000 --
-0.0010 --
-0.0020
- PEMS 2 * PEMS 3 • PEMS 5
Vr
H 1 1 1 H
H 1 h
-H I—
50
—I H
60
H 1—
70
10
20 30 40
Reference Gas Delta Observation
FIGURE 139. CORRECTED CO DELTAS MEASURED DURING ENVIRONMENTAL
TEMPERATURE TESTING
It was later decided that if the variance of the baseline test exceeded the variance of the
temperature test, it was unlikely the changes in ambient temperature adversely affected the
performance of the PEMS. Therefore, the bias differences captured by the subtraction of the
temperature and baseline median deltas are not likely due to changes in ambient temperature.
Following this argument, all delta observations in the final errors surfaces were set to zero if the
baseline MAD exceeded the temperature test MAD. An example of this correction is shown in
Figure 140 for CO.
SwRI Report 03.12024.06
Page 205 of 371
-------
0.007
0.006 -
0.005 -
0.004 -
g 0.003 -
3 0.002 -
&
I
0.001
0
-0.001 -
-0.002 -
-0.003
200
60.0
1000
CO Temperature Observation [1 to 1080]
- CO Span 0.0951% CO
• CO Audit 0.0179% CO
- CO Zero 0% CO
FIGURE 140. FINAL ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE
CO CONCENTRATION
A temperature error surface was also generated for exhaust flow rate. Similar to the
gaseous concentration error surfaces, the temperature exhaust flow rate data was corrected for
the bias error and variance recorded during baseline testing. The baseline correction process was
slightly modified for the exhaust flow rate because the PEMS automatically set all negative flow
rate measurements to zero. With all negative measurements set to zero, the distribution of the
flow rate data was inaccurate.
To generate the exhaust flow rate error surface for temperature, all zero flow rate
measurements were removed from both the temperature and baseline tests; thus generating a
more accurate variance comparison between the two data sets. With zero deltas removed, the
median exhaust flow rate measurement from the baseline test was subtracted from each
temperature exhaust flow rate delta. This process was inconsequential because the median
baseline delta was less than 0.01 % of the flow meter's maximum flow rating. Next the MAD of
the baseline data was compared to the MAD of the bias corrected temperature data. Using the
equation below, the MAD values were used to calculate a scaling factor. Each temperature
exhaust flow measurement was multiplied by the scaling factor to shrink the variance of the
temperature data by the variance measured during environmental baseline testing.
Scaling _ Factor =
-MAD
BL
MAD
Rod
All negative corrected temperature deltas were set to zero. The data was then mirrored
about the zero axis, generating twice the number of zero observations as well as negative deltas.
Negative deltas were generated to restore the negative data lost during the zero clipping process
and because exhaust measurements performed during engine operation in the NTE zone would
SwRI Report 03.12024.06
Page 206 of 371
-------
be subjected to both positive and negative exhaust flow rate errors. The final exhaust flow rate
error surface for temperature testing is shown in Figure 141. Temperature exhaust flow rate
deltas were sampled randomly in the Model and applied to each reference NTE event exhaust
flow rate independent of level.
0.25
0.20 -
100
150 200 250 300
Exhaust Flow Meter Zero Observation
350
400
450
FIGURE 141. ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE
EXHAUST FLOW RATE DELTA MEASUREMENTS
5.5 Pressure Chamber Testing
Pressure chamber testing was performed with two SEMTECH-DS devices to quantify
PEMS gaseous concentration and exhaust flow measurement errors due to changes in ambient
pressure. The pressure test was designed to simulate real-world pressures and changes in
pressure. Therefore, the pressure profile used during testing nearly matched the atmospheric
pressure distribution of EPA's 2002 National Emissions Inventory (NEI) model. Taken from the
Test Plan, Figure 142 shows the NEI pressure distribution as well as the test cycle pressure
distribution.
SwRI Report 03.12024.06
Page 207 of 371
-------
Pressure Histograms
QfiO/
yu/o
ono/
OU/o
•= 70%
]S
+J
•s
0s
o 40%
-------
TABLE 83. PRESSURE TEST PROFILE DEFINITION
Atmospheric Pressure Test Sequence
Phase
1 Soak
2 Ramp
3 Soak
4 Ramp
5 Soak
6 Ramp
7 Soak
8 Ramp
9 Soak
10 Ramp
11 Soak
12 Ramp
13 Soak
14 Ramp
15 Soak
16 Ramp
17 Soak
18 Ramp
19 Soak
20 Ramp
21 Soak
22 Ramp
23 Soak
Pressure
kPa
101
101-97
97
97-101.87
101.87
101.87-101
101
101-97
97
97-96.6
96.6
96.6-82.74
82.74
82.74-96.8
96.8
96.8-90
90
90-96.8
96.8
96.8-99.2
99.2
99.2-101
101
Alt. ft.
89
89-1203
1203
1203- -148
-148
-148-89
89
89-1203
1203
1203-1316
1316
1316-5501
5501
5501-1259
1259
1259-3244
3244
3244-1259
1259
1259-586
586
586-89
89
Time
min
10
20
20
60
20
20
20
20
25
20
20
20
20
30
20
15
10
20
20
20
20
10
20
Rate
ft/min
0
56
0
-23
0
12
0
56
0
6
0
209
0
-141
0
132
0
-99
0
-34
0
-50
0
Comments
Flat near sea-level
Moderate hill climb from sea level
Flat at moderate elevation
Moderate descent to below sea level
Flat at extreme low elevation
Moderate hill climb to near sea level
Flat near sea level
Moderate hill climb from sea level
Flat at moderate elevation
Slow climb from moderate elevation
Flat at moderate elevation
Rapid climb to NTE limit
Flat at NTE limit
Rapid descent from NTE limit
Flat at moderate elevation
Rapid hill climb to mid elevation
Flat at mid elevation
Rapid descent within middle of NTE
Flat at moderate elevation
Moderate descent to lower elevation
Flat at lower elevation
Moderate decent to near sea- level
Flat near sea-level
SwRI Report 03.12024.06
Page 209 of 371
-------
Pressure-Time Environmental Test Cycle
1/2-hr moving avg dA/dt (ft/hr)
Vertical gridlines = hours
Vertical gridlines = 7-min gas
cylinder cycle times
-9000
FIGURE 143. PRESSURE TEST PROFILE AND MOVING AVERAGE
The environmental pressure test was conducted in the altitude chamber shown in Figure
144. The chamber consisted of a cylindrical top that rested on a flat, circular base. The chamber
was specifically designed to simulate elevated altitudes and can attain pressure levels
representative of 65,000 feet of elevation. As the chamber pressure is lowered below ambient
pressure, the chamber top is pulled downward, creating a tight seal between the chamber base
and top. However, the Test Plan specified pressure levels up to 101.87 kPa or 148 feet below sea
level. Attaining positive pressure in the altitude chamber was problematic because the o-ring
sealing mechanism between the chamber top and base would shift and leak. Using weather
stripping, clay, and duct tape, a revised sealing mechanism was implemented that allowed the
chamber to achieve all pressures and pressure ramp rates as specified in the Test Plan.
SwRI Report 03.12024.06
Page 210 of 371
-------
ALTITUDE CHAMBER
ihide and Elevated Temperature
'rnglales Up to 65,000 Feet of
lilude
nal Dimensions: 5' Dia. x 7' Tall
FIGURE 144. ALTITUDE CHAMBER TOP - REMOVED FROM BASE
As shown in Figure 145, PEMS 2 and 3 were tested simultaneously in the altitude
chamber. PEMS 3 was tested in a Sensors Inc. environmental enclosure. A 5-inch EFM was
also tested in the chamber. In order to accurately simulate elevation changes, the PEMS sample
line overflow system, overflow FID zero air system, and PEMS drain lines were vented inside
the altitude chamber. Figure 145 shows a preliminary and incorrect setup with venting occurring
outside the chamber. Low restriction, electronic flow meters were installed in the sample line
overflow stream and FID zero air overflow stream to insure adequate bypass flow was
maintained during the 8-hour pressure test. Gas lines and Ethernet cables were routed out of the
chamber through a hole in the base plate. Clay and expanding foam insulation spray was used to
seal the lines and cables exiting the chamber base. With no access to the PEMS once the
chamber top was sealed to the base, replacing the FID fuel bottles during the 8-hour test was not
possible. Therefore, two Scotty 104 FID fuel bottles were plumbed together in parallel for each
PEMS unit. With two FID fuel bottles, the PEMS operated without FID fuel bottle replacement
during the altitude simulation test.
SwRI Report 03.12024.06
Page 211 of 371
-------
FIGURE 145. PEMS EQUIPMENT ON ALTITUDE CHAMBER BASE
Prior to executing the environmental pressure test, the PEMS were allowed to thermally
equilibrate while sampling ambient air for over one hour. The altitude chamber top was then
sealed to the chamber base. Figure 146 shows the assembled altitude chamber with pressure
control equipment. Next, the PEMS were zeroed and spanned at ambient pressure,
approximately 98 kPa. The environmental pressure test was then started by ramping to the initial
pressure soak point as specified in the Test Plan. During the 8-hour pressure test, the PEMS
were automatically zeroed every hour. The pressure control chamber was not paused during the
test, therefore, zero events occurred at the pressures defined by the Test Plan's pressure profile
definition. Zero events occurred near the hour markers shown in Figure 143. Similar to
environmental baseline and temperature testing, zero, audit, and span deltas were recorded by
comparing the 30-second PEMS mean concentration measurements to the reference gas
concentrations.
SwRI Report 03.12024.06
Page 212 of 371
-------
- Mi*
FIGURE 146. ALTITUDE CHAMBER AND PRESSURE CONTROL EQUIPMENT
DURING TESTING
The zero deltas measured during the 8-hour pressure test are shown in Figure 147 for
PEMS 2. Mean pressure measurements from the PEMS ambient pressure transducer are also
shown in Figure 147. Environmental pressure results for PEMS 3 are included in Appendix K.
In general, the PEMS zero errors showed little variation during environmental pressure test. NO2
measurements were relatively stable throughout the pressure test. PEMS 2 NO zero
measurements drifted upward during the first hour of pressure testing. However, the pressure
change was not significant during the first hour of testing and PEMS 3 showed no NO drift
during the initial hour of the test. Therefore, the PEMS 2 NO drift during the first hour of testing
was not likely caused by changes in pressure, but perhaps thermal equilibration of the NDUV.
PEMS 2 and 3 both showed slight negative NO drift during the steep negative pressure ramp to
the soak pressure of 82.74 kPa. The NO measurements drifted back in a positive direction
during the positive pressure ramp from the 82.74 kPa soak pressure. THC zero measurements
were relatively stable during the pressure test. Similar to NO, THC zero deltas showed slight
negative drift during negative pressure ramps, and slight positive drift during positive pressure
ramps. CO and CO2 zero deltas did not show significant deviations due to changes in pressure.
SwRI Report 03.12024.06
Page 213 of 371
-------
• THC [ppmC] BCO [ppm] NO [ppm] XNO2 [ppm] »NOx [ppm] Pressure [kpa] ACO2 [%]
120
„ 100 —,
0.014
-0.018
Reference Gas Delta Observation
FIGURE 147. PEMS 2 ENVIRONMENTAL PRESSURE ZERO DELTA
MEASUREMENTS
PEMS 2 audit level deltas measured during the environmental pressure test are shown in
Figure 148. NC>2, THC, and CC>2 deltas were relatively stable and showed no trends with
pressure. Similar to the zero deltas, PEMS 2 showed positive NO audit level drift during the first
hour of testing. Because PEMS 3 showed no NO drift during the first hour of pressure testing
and the pressure was relatively constant during this period, the PEMS 2 NO drift was likely
caused by a factor other than pressure change. CO audit measurements showed positive
response when the pressure in the altitude chamber was reduced. PEMS 2 CO audit deltas
reached 70 ppm during the pressure soak at 82.74 kPa.
SwRI Report 03.12024.06
Page 214 of 371
-------
• THC [ppmC] BCO [ppm] NO [ppm] XNO2 [ppm] «NOx [ppm] -Pressure [kpa] ACO2 [%]
- 0.054
-0.030
Reference Gas Delta Observation
FIGURE 148. PEMS 2 ENVIRONMENTAL PRESSURE AUDIT DELTA
MEASUREMENTS
The environmental pressure test span deltas are shown in Figure 149 for PEMS 2. NC>2
measurements were steady and unaffected by changes in pressure. NO span measurements were
more variable; however, it was difficult to determine a link between the NO deltas and the
chamber pressure. THC span measurements showed slight positive response with lower
chamber pressures. CO and CO2 span measurements were both affected by chamber pressure.
CO span deltas reached 140 ppm during the pressure soak at 82.74 kPa and 50 ppm during the 90
kPa soak. CO2 span measurements had a negative response to lowered chamber pressure, with
CO2 span deltas reaching -0.13 % during the 82.74 kPa soak. PEMS 3 pressure test span deltas,
which were similar to PEMS 2, can be found in Appendix K.
SwRI Report 03.12024.06
Page 215 of 371
-------
• THC [ppmC] BCO [ppm] NO [ppm] XNO2 [ppm] «NOx [ppm] -Pressure [kpa] ACO2 [%]
160
-60
Reference Gas Delta Observation
FIGURE 149. PEMS 2 ENVIRONMENTAL PRESSURE SPAN DELTA
MEASUREMENTS
The response of a 5-inch Sensors Inc. flow meter was also measured during the
environmental pressure test. With one end capped to prevent air flow through the meter, the 30-
second mean measurements were recorded at zero deltas. Figure 150 shows the EFM deltas
observed during pressure testing. Unlike baseline and temperature testing, most EFM
measurements were above zero during pressure testing. Although the EFM measurement errors
were small compared to the 1700 scfm flow rating of the meter, the PEMS EFM showed positive
interference during the environmental pressure test. When the chamber pressure was below
approximately 990 mbar, the EFM zero deltas were between 5 and 8 scfm. When the chamber
pressure was above 990 mbar, the EFM deltas were between 0 and 4 scfm.
SwRI Report 03.12024.06
Page 216 of 371
-------
* Flow Meter [scfm] • Pressure [mbar]
1200
100
150 200 250
Reference Gas Delta Observation
300
350
FIGURE 150. 5-INCH EFM ENVIRONMENTAL PRESSURE ZERO DELTA
MEASUREMENTS
5.5.1 Pressure Error Surface Generation
The process used to generate the pressure error surfaces was similar to the environmental
temperature error surface calculation method. The pressure delta data for each PEMS was
corrected for baseline bias and variance. As with the temperature error surfaces, it was difficult
to determine which gaseous emissions showed a susceptibility to the ambient pressure
disturbances. Criteria similar to the temperature error surfaces were used to decide which
pressure error surfaces should be included in the Model. As shown in Figure 151, the trends of
the NMHC span deltas followed the ambient pressure traces recorded in the pressure chamber.
Also, the two PEMS showed similar NMHC span delta behavior, indicating a common source of
error. CO also showed definite ties between the delta behavior and the pressure traces.
Therefore, NMHC and CO pressure error surfaces were included in the model. For NOX and
CO2, no correlation could be made between the delta data and the pressure profile or between the
two PEMS. Therefore, the NOX and CO2 deltas were not likely affected by the changes in
ambient pressure. NOX and CO2 pressure error surfaces were not included in the Model. The
environmental pressure error surfaces can be found in Appendix K.
SwRI Report 03.12024.06
Page 217 of 371
-------
10
Q.
5
Q 0
c
o
-5 --•
-PEMS2 »PEMS3
** -
o
o
o
-10 -
--*»*
-15
-H 1 1-
H H
-H
-H 1 H
-I-
-+-
H 1—
70
10
20 30 40 50
Reference Gas Delta Observation
60
FIGURE 151. NMHC CORRECTED DELTA DATA FOR THE ENVIRONMENTAL
PRESSURE TESTING
Similar to the temperature error surfaces, pressure delta data was spread to cover 360
observations for each PEMS at the zero, audit, and span levels. With only 2 PEMS evaluated
during pressure testing, the final NMHC and CO error surfaces were randomly sampled using
720 observations. Figure 152 shows the final NMHC error surface generated during pressure
testing. Delta data was linearly interpolated between the zero, audit, and span deltas at a given
observation based on the concentrations in the reference NTE events.
SwRI Report 03.12024.06
Page 218 of 371
-------
800
NMHC Pressure Observation [1 to 720]
- NMHC Span 663ppm NMHC
> NMHC Audit 159.9ppm NMHC
-NMHCZeroOppm NMHC
FIGURE 152. FINAL ERROR SURFACE FOR ENVIRONMENTAL PRESSURE NMHC
CONCENTRATION
A process similar to the temperature exhaust flow rate error surface calculation method
was used to generate the pressure exhaust flow rate error surface. The pressure exhaust flow
data was corrected for baseline bias and variance. The zero deltas were removed from the
baseline flow meter data. Interesting, all exhaust flow rate deltas were greater than zero during
pressure testing, therefore removal of zero data was not necessary. Although minor, the pressure
flow rate data was corrected for the baseline bias. After the bias correction, the variance
correction was applied using the scaling factor calculation. Due to excessive variability of the
pressure flow rate data, the scaling factor was calculated to be 0.995, and therefore had little
effect on the pressure flow rate data. The final pressure exhaust flow rate error surface is shown
in Figure 153. The pressure flow rate error surface was sampled randomly and without level
dependence.
SwRI Report 03.12024.06
Page 219 of 371
-------
£
HI
*ro
a:
5
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
0.1
0.1
0.0
-0.1
-::*.
V
_»_.
50
100 150 200 250
Exhaust Flow Zero Observation
300
350
FIGURE 153. ERROR SURFACE FOR ENVIRONMENTAL PRESSURE EXHAUST
FLOW RATE
5.6 Radiation Chamber Testing
Radiation chamber testing was performed with one SEMTECH-DS to quantify PEMS
gaseous concentration and exhaust flow measurement errors due to Electromagnetic Interference
(EMI) and Radio Frequency Interference (RFI). The four Society of Automotive Engineers
(SAE) tests selected for radiation chamber testing were Bulk Current Injection, Radiated
Immunity, Electrostatic Discharge, and Conducted Transients.
5.6.1 Bulk Current Injection
SAE test 111 13/4 titled Immunity to Radiated Electromagnetic Fields-Bulk Current
Injection (BCI) Method was performed to evaluate the PEMS response to radiated
electromagnetic fields on the PEMS cabling. Based on the SAE Standard test descriptions and
recommendations from SwRI specialists, the Steering Committee elected to test the PEMS using
the specifications detailed in Region 2, Class B of the Jl 113/4 test protocol. As shown in Figure
154, a calibrated current probe was used to inject RF current into the PEMS cables. For each
test, the probe was positioned 120 mm, 450 mm, and 750 mm from the cable connector. In other
words, a complete test was performed with the probe located 120 mm from the cable connector.
Another complete test was performed with the probe located 450 mm from the cable connector,
and another test at 750 mm. Each test consisted of stepping the current probe frequency from 1
MHz to 400 MHz. Listed below, SwRI used the maximum frequency step size as stated in the
SAE test protocol.
SwRI Report 03.12024.06
Page 220 of 371
-------
• 1 MHz to 10 MHz - 1 MHz step size
• 10 MHz to 200 MHz - 10 MHz step size
• 200 MHz to 400 MHz - 20 MHz step size
The SAE standard called for a minimum dwell time of 2 seconds at each frequency.
However, SwRI used a dwell time of 5 seconds to insure the electromagnetic field had stabilized.
As specified by the SAE Standard, the current probe was calibrated to deliver 60 milliamps of
current. Figure 155 shows the device used to calibrate the bulk current injection probe.
FIGURE 154. BULK CURRENT INJECTION PROBE
SwRI Report 03.12024.06
Page 221 of 371
-------
FIGURE 155. CALIBRATION DEVICE FOR THE BULK CURRENT INJECTION
PROBE
During Bulk Current Injection testing, the PEMS was powered with the Sensors Inc.
inverter as well as a 12-volt automotive battery. The battery was used during radiation testing to
simulate PEMS field testing. Figure 156 shows PEMS 7 setup in a radiation chamber during
Bulk Current Injection testing. Similar to baseline testing, the PEMS was zeroed and spanned
after warming for over one hour. The PEMS was zeroed approximately ever hour during testing.
Unlike temperature and pressure testing, which were continuous 8-hours tests, several BCI tests
were completed each hour. The Steering Committee elected to take advantage of the segmented
radiation testing and add short periods of baseline or zero stimulation testing during each hour of
testing. As observed with temperature and pressure testing, it was often difficult to determine if
the cause of increased delta measurements was due to changes in the environmental condition
being tested, or some other factor. Adding periods of baseline testing throughout the BCI test
offered direct comparison of PEMS measurement deltas with and without radiation stimulation.
The baseline comparisons aided in determining if PEMS deltas were caused by BCI or some
other factor.
SwRI Report 03.12024.06
Page 222 of 371
-------
FIGURE 156. PEMS 7 IN THE RADIATION CHAMBER UNDERGOING BULK
CURRENT INJECTION TESTING
The cables evaluated during BCI testing included the 12-volt power supply cables, the
auxiliary 1 cable, the Ethernet cable, the temperature/relative humidity probe cable, the heated
line power cable, and the EFM cables. Initially, PEMS 3 was used for BCI testing. However,
while testing the power cable, the PEMS reported several FID faults and eventually shut down
the FID due to high FID temperature. The probe current was reduced from 60 milliamps to 40
milliamps, but the FID still shutdown between 26 and 46 MHz. Fearing a problem with PEMS
3, PEMS 7 was used for BCI testing. Although similar FID faults and problems existed with
PEMS 7, the issues occurred less frequently. When testing the power cable, the probe current
was reduced to 40 milliamps due to FID shutdown problems at 60 milliamps. All other cables
were tested at 60 milliamps. Throughout testing, a number of faults and problems occurred.
Most faults were related to the FID. According to Sensors Inc., the FID faults and shutdowns
may have been related to problems with the FID DC to AC board. Communication between the
laptop and PEMS unit was disrupted several times. Communication between the PEMS and
EFM was also disrupted, requiring the PEMS to be restarted to restore communication. After the
PEMS was restarted, the analyzers were zeroed and spanned before continuing the BCI test.
SwRI Report 03.12024.06
Page 223 of 371
-------
Figure 157 shows the PEMS 7 gaseous emission concentration zero deltas measured
during Bulk Current Injection testing. PEMS zero events as well as the periods of baseline
testing are marked in Figure 157. Although difficult to see in the chart, NO zero deltas were
within a range of ±5 ppm and showed no noticeable difference between BCI testing and the
baseline portions of the test. In general, NO2 zero deltas during BCI and baseline testing were
between 0 and 5 ppm. One NO2 measurement showed an outlying positive delta of 7.6 ppm
while another outlying NO2 delta was at -15.5 ppm. With the exception of the two outlying NO2
zero measurements, NO2 deltas showed no difference between BCI and baseline testing. With
the exception of the first hour of testing, THC zero deltas were typically within ±1 ppmC.
Although the THC zero measurement drifted downward during the first hour of testing, the
baseline testing at the beginning and end of the hour test segment showed similar drift behavior;
indicating the drift was not caused by current injection. In general, CO and CO2 zero
measurements showed similar deltas during baseline and BCI testing. However, 4 CO zero
observations and 1 CO2 observation were outlying, low deltas that indicated a possible
susceptibility to the BCI test. With the exception of a few outlying points, there was no
noticeable difference between the measurements taken during BCI testing and those taken during
baseline testing. Therefore, the effect of the electromagnetic radiation on the PEMS cabling was
minor.
• THC [ppmC] BCO [ppm] «NO [ppm] XNO2 [ppm] «NOx [ppm] - Baseline Zero ACO2 [%]
Reference Gas Delta Observation
FIGURE 157. PEMS 7 ENVIRONMENTAL RADIATION BCI ZERO DELTA
MEASUREMENTS
Figure 158 shows the PEMS 7 audit delta measurements during BCI testing. The shift in
audit delta levels after the PEMS was restarted was due to a zero and span event for all
analyzers. Similar to the BCI zero deltas, almost all of the BCI audit deltas were similar to the
baseline audit deltas, indicating BCI testing had little effect on the PEMS gaseous measurement
systems. The only exceptions were 1 low CO audit measurement and 5 high CO2 measurements.
SwRI Report 03.12024.06
Page 224 of 371
-------
• THC [ppmC]
- Baseline
CO [ppm]
Zero
NO [ppm] X NO2 [ppm]
-Restart PEMS A CO2 [%]
NOx [ppm]
0.30
0.00
Reference Gas Delta Observation
FIGURE 158. PEMS 7 ENVIRONMENTAL RADIATION BCI AUDIT DELTA
MEASUREMENTS
The span deltas measured during BCI testing are shown in Figure 159. The PEMS was
restarted during the BCI test to restore communication with the EFM. The PEMS was zeroed
and spanned after being restarted, resulting in a shift in span delta measurements. Similar to the
BCI zero and audit deltas, the BCI span deltas nearly all matched the baseline deltas through the
environmental test. However, 1 CO span delta observation was outlying and low, 5 CO2 span
deltas were high, and 1 NO2 span delta was high. Because of the vast similarity between BCI
deltas and baseline deltas, the effect of the Bulk Current Inject testing on the PEMS span
measurements was minor.
SwRI Report 03.12024.06
Page 225 of 371
-------
0.375
0.300
0.225 «
0.150 *
•E
u
o
c
o
o
0.075
O
O
0.000
-0.075
Reference Gas Delta Observation
FIGURE 159. PEMS 7 ENVIRONMENTAL RADIATION BCI SPAN DELTA
MEASUREMENTS
As shown in Figure 160, the 5-inch EFM was susceptible to the BCI test. The EFM
reported several elevated measurements throughout BCI testing. EFM zero deltas were near zero
during baseline test segments, indicating the positive EFM measurements were most likely
caused by the BCI tests.
SwRI Report 03.12024.06
Page 226 of 371
-------
• Flow Meter [scfm] • Baseline
Reference Gas Delta Observation
FIGURE 160. 5-INCH EFM ENVIRONMENTAL RADIATION BCI ZERO DELTA
MEASUREMENTS
5.6.2 Radiated Immunity
SAE test 111 13/21 titled Electromagnetic Compatibility Measurement Procedure for
Vehicle Components - Part 21: Immunity to Electromagnetic Fields, 10 kHz to 18 GHz,
Absorber-Lined Chamber was performed to evaluate the PEMS response to continuous
narrowband electromagnetic fields on the PEMS and PEMS cabling. Based on the SAE
Standard test descriptions and recommendations from SwRI specialists, the Steering committee
elected to test the PEMS using the specifications detailed in Region 2, Class B of the 111 13/21
test protocol. As shown in Figure 161, the Radiated Immunity test was performed in an
absorber-lined radiation test room. The absorber medium was carbon-impregnated foam. A
series of antennas and a host of power electronics, shown in Figure 162, were used to generate
the RF fields specified in the SAE test protocol.
SwRI Report 03.12024.06
Page 227 of 371
-------
FIGURE 161. PEMS 7 AND RADIATION ANTENNA IN THE ABSORBER-LINED
RADIATION CHAMBER DURING RADIATED IMMUNITY TESTING
SwRI Report 03.12024.06
Page 228 of 371
-------
FIGURE 162. SIGNAL GENERATORS, AMPLIFIERS, OSCILLOSCOPES AND
OTHER ELECTRONICS USED TO PERFORM RADIATION TESTING
The Radiated Immunity tests consisted of stepping the electromagnetic field frequency
from 10 kHz to 1 GHz. Experts at SwRI recommended ending the Radiated Immunity testing at
1 GHz rather than 18 GHz as specified by the standard. Because the maximum oscillator speed
of the PEMS was relatively low, testing at higher frequencies would have likely shown no
measurement or operational susceptibilities. Listed below, SwRI used the maximum frequency
step size as stated in the SAE test protocol. The SAE standard called for a minimum dwell time
of 2 seconds at each frequency. However, SwRI used a dwell time of 5 seconds to insure the
electromagnetic field had stabilized. Although the SAE Standard specified using a field intensity
of 50 volts/meter, the intensity was reduced during testing to prevent the PEMS FID from
shutting down.
• 10 kHz to 100 kHz-10 kHz step size
• 100 kHz to 1 MHz - 100 kHz step size
• 1 MHz to 10 MHz - 1 MHz step size
• 10 MHz to 200 MHz - 2 MHz step size
• 200 MHz to 1 GHz - 20 MHz step size
During Radiated Immunity testing, PEMS 7 was operated in a manner similar to Bulk
Current Injection Testing. The PEMS was powered with the Sensors Inc. inverter as well as a
12-volt automotive battery. Similar to baseline testing, the PEMS was zeroed and spanned after
warming for over one hour. The PEMS was zeroed approximately every hour during Radiated
SwRI Report 03.12024.06
Page 229 of 371
-------
Immunity testing. Also, short periods of baseline or zero stimulation testing were included
during each hour of testing to determine if the PEMS deltas were caused by electromagnetic
radiation or some other factor.
During initial Radiated Immunity testing, the field intensity was set to the SAE Standard
specification of 50 volts/meter. At 50 volts/meter, PEMS 7 produced numerous faults and
warnings pertain to the FID, and eventually shutdown the FID. Testing was then repeated with a
field intensity of 25 volts/meter. PEMS 7 displayed several FID faults and warnings, and
shutdown the FID at an electromagnetic frequency of 164 MHz. The Radiated Immunity test
was repeated a third time, this time at the CE Standard field intensity specification of 10
volts/meter. At 10 volts/meter PEMS 7 performed similarly to the 25 volts/meter test, and
shutdown the FID at 164 MHz. After further testing, it was determined that the PEMS FID
would not operate with a field intensity of 10 volts/meter between the frequency range of 164 to
178 MHz. Therefore, testing was performed until the FID shutdown at 164 MHz. After the FID
was restarted, testing was continued from 178 MHz. Throughout Radiated Immunity testing, the
field intensity was set as high as possible without causing functional PEMS failures. If a large
number faults occurred, or if the PEMS FID shutdown, the radiation test was often repeated at a
lower field intensity.
Figure 163 shows the PEMS 7 zero deltas measured during Radiated Immunity testing.
Following the initial baseline testing, the electromagnetic frequency was ramped from 30 to 164
MHz at 25 volts/meter in a horizontal direction. At observation number 36 with a frequency of
164 MHz, the PEMS FID shutdown. The PEMS was then restarted. After baseline observations
37 through 45, the electromagnetic frequency was ramped from 30 to 164 MHz at 10
volts/meter, observation number 46 through 60. Even at 10 volts/meter, the FID shutdown at
164 MHz. Testing was then continued with the radiation frequency ramped from 178 to 260
MHz at 10 volts/meter during reference gas observations 61 through 72. The FID shutdown
again at 260 MHz and 10 volts/meter. During measurement 73 through 99, the electromagnetic
radiation was ramped from 260 to 300 MHz at 10 volts/meter, 300 MHz to 1000 MHz at 25
volts/meter, and after changing antennas, from 200 to 1000 MHz at 25 volts/meter in a vertical
direction. A short baseline segment was included from observation 91 through 96. Radiated
Immunity testing continued by ramping vertically from 30 to 50 MHz at 10 volts/meter and 50 to
200 volts/meter at 25 volts/meter. Using a bipolar antenna, the electromagnetic frequency was
ramped from 10 kHz to 6 MHz at 25 volts/meter and from 7 to 30 MHz at 10 volts/meter.
Testing was concluded by recording deltas with no radiation to generate a final baseline test
segment.
SwRI Report 03.12024.06 Page 230 of 371
-------
THC [ppmC] • CO [ppm] • NO [ppm] X NO2 [ppm]
Baseline A Zero Restart Test A CO2 [%]
NOx [ppm]
350
0.025
E
Q.
£
HI
Q
c
o
HI
o
c
o
o
Reference Gas Delta Observations
FIGURE 163. PEMS 7 ENVIRONMENTAL RADIATION RADIATED IMMUNITY
ZERO DELTA MEASUREMENTS
Throughout Radiated Immunity testing, the only zero deltas that showed notable
perturbations due to electromagnetic radiation were for CO when ramped from 120 to 164 MHz
at 25 volts/meter. One low CO2 concentration delta was recorded during radiation testing. All
other deltas resembled baseline measurements, indicating measurement errors were not caused
by electromagnetic radiation.
Figure 164 and Figure 165 show the PEMS 7 audit and span deltas recorded during
Radiated Immunity testing. Similar to the zero deltas, the electromagnetic radiation affected the
CO audit and span measurements from 120 to 164 MHz at 25 volts/meter. One high CO2 audit
and span delta was observed near 120 MHz at 25 volts/meter. All other audit and span
measurements appeared unaffected by the electromagnetic radiation.
SwRI Report 03.12024.06
Page 231 of 371
-------
THC [ppmC] • CO [ppm] • NO [ppm] X NO2 [ppm]
Baseline A Zero Restart Test A CO2 [%]
NOx [ppm]
350
300
250
200
150
100
50
0
-50
-100
-150
M»li»iil«>»
jiA*!
0.350
0.315
0.280
0.245
0.210
0.175
0.140
0.105
0.070
0.035
0.000
Reference Gas Delta Observations
FIGURE 164. PEMS 7 ENVIRONMENTAL RADIATION RADIATED IMMUNITY
AUDIT DELTA MEASUREMENTS
THC [ppmC] • CO [ppm] • NO [ppm] X NO2 [ppm]
Baseline A Zero Restart Test A CO2 [%]
NOx [ppm]
Reference Gas Delta Observations
FIGURE 165. PEMS 7 ENVIRONMENTAL RADIATION RADIATED IMMUNITY
SPAN DELTA MEASUREMENTS
SwRI Report 03.12024.06
Page 232 of 371
-------
Figure 166 shows the exhaust flow meter zero deltas recorded during Radiated Immunity
testing. Although most measurements were near zero, several segments of radiation testing
showed elevated EFM readings. EFM susceptibility was recorded at a field intensity of 25
volts/meter and electromagnetic frequencies ranging from 120 to 164 MHz (horizontal polarity),
300 to 1000 MHz (horizontal polarity), and 200 to 1000 MHz (vertical polarity).
* Flow Meter [scfm] Baseline
30
25 -
I20
u
~ 15
re lo
t 10
-5
•
v~
50
Exhaust Flow Meter Zero Observations
FIGURE 166. 5-INCH EFM ENVIRONMENTAL RADIATION RADIATED IMMUNITY
ZERO DELTA MEASUREMENTS
5.6.3 Electrostatic Discharge
SAE test 111 13/13 titled Electromagnetic Compatibility Measurement Procedure for
Vehicle Components—Part 13: Immunity to Electrostatic Discharge was performed to evaluate
the PEMS response to Electrostatic Discharges (ESDs) on the PEMS and auxiliary equipment.
Based on the SAE Standard test descriptions and recommendations from SwRI specialists, the
Steering committee elected to test the PEMS using the specifications detailed in Region 2, Class
B of the 111 13/13 test protocol. ESDs were delivered at over 80 locations on the PEMS, the
EFM, the EFM pressure transducer enclosure, the humidity probe, and PEMS connectors. Using
the Electrostatic Discharge Simulator shown in Figure 167, the discharge was delivered directly,
or with the simulator tip touching the discharge surface; as well as indirectly, or with the
simulator tip not touching the discharge surface. The direct discharge was performed by placing
the simulator tip on the discharge surface and energizing the discharge gun. The indirect
discharge was performed by energizing the discharge gun away from the discharge surface. The
energized gun tip was then moved towards the discharge surface until the voltage potential
SwRI Report 03.12024.06
Page 233 of 371
-------
caused an arc and the discharge was released. Both direct and indirect ESDs were performed at
each of the discharge locations.
FIGURE 167. ELECTROSTATIC DISCHARGE SIMULATOR USED DURING
ELECTROSTATIC DISCHARGE TESTING
As specified by the SAE Standard test procedure, the Electrostatic Discharge Simulator
was calibrated to deliver 4000 volts. An Electrostatic Voltmeter, shown in Figure 168, was used
to calibrate the Electrostatic Discharge Simulator.
SwRI Report 03.12024.06
Page 234 of 371
-------
FIGURE 168. ELECTROSTATIC VOLTMETER USED TO CALIBRATE THE
ELECTROSTATIC DISCHARGE SIMULATOR
During Electrostatic Discharge testing, PEMS 7 was operated in a manner similar to Bulk
Current Injection and Radiated Immunity testing. The PEMS was powered with the Sensors Inc.
inverter as well as a 12-volt automotive battery. Similar to baseline testing, the PEMS was
zeroed and spanned after warming for over one hour. Due to the reduced length of the ESD
testing, and the inclusion of baseline test segments, PEMS 7 was only zeroed once during the
Electrostatic Discharge test. Periods of baseline or zero stimulation testing were included at the
beginning and end of the ESD test to determine if the electrostatic discharge had any effect on
the PEMS measurements. To capture potential measurement errors, the discharge events, being
extremely brief, were timed to occur during the 30-second recorded measurements. Several
discharge locations were tested during each 30-second measurement.
Figure 169 shows the PEMS 7 zero delta measurements during Electrostatic Discharge
testing. Baseline test segments were included at the beginning and end of the test, with one zero
event occurring at observation number 28. In general, the zero deltas during testing resembled
the deltas measured during baseline testing, indicating the ESD testing had little effect on the
PEMS measurement. Two CC>2 zero measurements were outlying and high. With the ESDs
being extremely brief, short perturbations in the PEMS measurements may not have been
revealed in a 30-second average measurement. Therefore, the continuous data was reviewed for
each PEMS measurement to insure a short duration measurement error was not overlooked. The
continuous data showed no evidence of susceptibility to the ESDs.
SwRI Report 03.12024.06
Page 235 of 371
-------
• THC [ppmC] BCO [ppm] »NO [ppm] XNO2 [ppm] «NOx [ppm] -Baseline Zero ACO2 [%]
0.050
0.045
Q
c
o
B
re
m
u
c
o
o
Reference Gas Delta Observations
FIGURE 169. PEMS 7 ENVIRONMENTAL RADIATION ELECTROSTATIC
DISCHARGE ZERO DELTA MEASUREMENTS
Figure 170 and Figure 171 show the PEMS 7 audit and span deltas measured during
Electrostatic Discharge testing. After the zero event at observation number 10, the delta
measurements for NO, NO2, and CO2 had a noticeable shift. Although stable prior to the zero
calibration, the NO, NO2, and CO2 delta measurements showed noticeable positive drift after
being re-zeroed. The baseline test segment at the end of the test showed no shift in delta
measurements compared to those during ESD testing. Therefore, the deltas observed during
ESD testing were not likely caused by the Electrostatic Discharge. Furthermore, the continuous
data for each PEMS measurement was reviewed to insure short duration measurement
perturbations were not overlooked using the 30-second mean measurement. The continuous data
showed no evidence of short term measurement errors.
SwRI Report 03.12024.06
Page 236 of 371
-------
Concentration Delta [ppm]
hj .&. O CO C
D 0 0 0 0 C
-20
-4D
»THC [ppmC] BCD [ppm] «NO [ppm] XNO2 [ppm] »NOx [ppm] -Baseline Zero ACO2 [%]
'
A. • • A * A A A '
" • MJL • A A A A
A A A J »* " "t" .
•
A
A
A
5 8 * !Bx2£^5!lp*lXS*™»*25 30
~ • ( • •
"A" «
A
3 0 0 0 0 C
J Ul ^J O NJ C
n o m o m c
32 Concentration Delta [%]
- -n n?R
Reference Gas Delta Observations
FIGURE 170. PEMS 7 ENVIRONMENTAL RADIATION ELECTROSTATIC
DISCHARGE AUDIT DELTA MEASUREMENTS
»THC [ppmC] BCO [ppm] •NO [ppm] 5KNO2 [ppm] • NOx [ppm] -Baseline Zero ACO2 [%]
80
60 -^ ^- 0.070
40
Q.
Q.
20
HI
Q
c
o
S -20
u
c
o
o
-40
-80
- 0.040 ^
A A •
* * * * vv
- 0.010
• m 10
X *
• • A
X
,«x*fx*f°
X • •
-60 0.110
0.100
-0.020 ™
0.050
-- -0.080
-0.140
Reference Gas Delta Observations
FIGURE 171. PEMS 7 ENVIRONMENTAL RADIATION ELECTROSTATIC
DISCHARGE SPAN DELTA MEASUREMENTS
Figure 172 shows the EFM measurements during BSD testing. Nearly all of the EFM
zero deltas recorded during BSD testing were below 0.2 scfm. Curiously, the largest zero deltas
SwRI Report 03.12024.06
Page 237 of 371
-------
were measured at the end of the test during baseline testing. The 30-second mean delta data
suggests BSD testing had little affect on the exhaust flow measurements. Similar to gaseous
concentration measurements, the continuous EFM data was examined and found to show no
evidence of short duration measurement errors.
• Flow Meter [scfm] Baseline
3.5
3.0
2.5
2.0
1.5
•a o.s
o.o
-0.5
-1.0
-1.5
* *
»***
20
40
60
80
100
120
140
Exhaust Flow Meter Zero Observations
FIGURE 172. 5-INCH EFM ENVIRONMENTAL RADIATION ELECTROSTATIC
DISCHARGE ZERO DELTA MEASUREMENTS
5.6.4 Conducted Transients
SAE test 111 13/11 titled Immunity to Conducted Transients on Power Leads was
performed to evaluate the PEMS response to transient voltage disturbances on the PEMS 12-volt
power supply cable. Based on the SAE Standard test descriptions and recommendations from
SwRI specialists, the Steering committee elected to test the PEMS using the specifications
detailed in Region 2, Class B of the Jl 113/11 test protocol. Due to the high current draw of the
PEMS, a Schaffner NSG 5200 Automotive Electronics Test System was rented to perform the
Conducted Transients testing. A Schaffner test system is shown in Figure 173 The test system
included a Burst Generator Module, a Load Dump Module, a Pulse Generator Module, and an
Automotive ECM Test System with PC. The Schaffner system included all of the hardware and
software necessary to perform each of the Conducted Transients as specified in the SAE
Standard.
SwRI Report 03.12024.06
Page 238 of 371
-------
FIGURE 173. SCHAFFNER NSG 5200 AUTOMOTIVE ELECTRONICS TEST SYSTEM
During Conducted Transients testing, the power cable to the PEMS heated sample line
was disconnected to keep the PEMS current draw below 30 amps, which was the limit of the
Schaffner test system. The Schaffner NSG 5200 was connected in series between the PEMS and
12-volt power supply. With the test system installed, the PEMS supply voltage dropped to 8
volts using the Sensors Inc. power supply. Therefore, a SwRI 12-volt power supply was used to
power the PEMS during Conducted Transients testing. Similar to the other radiation tests, a 12-
volt automotive battery was connected in parallel with the power supply. The PEMS supply
voltage was maintained at approximately 13 volts during testing. Using the Schaffner test
system, a number of voltage disturbances were introduced through the PEMS power supply
cable. The voltage perturbations ranged from -200 to 100 volts with bursts as short as 250 ns
and voltage spikes lasting up to 2, 4, or 200 ms. The tests consisted of voltage spikes with slow
recovery, voltage spikes with quick recovery, repeated voltage bursts, and a load dump. All 200
ms voltage spikes with slow recovery would cause the PEMS to shut down. The 200 ms
duration tests were repeated at a quarter of the voltage disturbance amplitude, however, the
PEMS continued to shutdown. Therefore, the response of the PEMS to 200 ms voltage
disturbances was not characterized in this experiment. Similarly, the load dump experiment
caused the PEMS to shutdown, therefore, no PEMS response data was gathered for that portion
of the Conducted Transient testing.
Figure 174 shows an example voltage trace of a voltage spike with slow recovery during
Conducted Transient testing. The voltage disturbance had an amplitude of -100 volts and
recovery time of 4 ms.
SwRI Report 03.12024.06
Page 239 of 371
-------
Tek Run: 500ks/s Sample iUBE
[ T ]
C1 Min
-104.0 V
C1 Max
13.6V
Mill 20.0 V
M 500JJS Ch2 I 200mV 9 [\|OV 2006
16:21:10
FIGURE 174. EXAMPLE VOLTAGE TRACE DURING A VOLTAGE SPIKE WITH
SLOW RECOVERY
Figure 175 shows an example voltage trace of a voltage spike with quick recovery during
Conducted Transient testing. The voltage disturbance had an amplitude of 50 volts and recovery
time of 0.05 ms.
SwRI Report 03.12024.06
Page 240 of 371
-------
Tek Run: 10.0MS/S Sample
[ T-
A: 54.2V
67.0V
C1 Min
12.2V
Bill 10.0 V
M 25.0JJS Ch2 f 200mV g NQV 2006
16:26:16
FIGURE 175. EXAMPLE VOLTAGE TRACE DURING A VOLTAGE SPIKE WITH
QUICK RECOVERY
Figure 176 shows a voltage trace for a short duration voltage burst. The voltage burst
had an amplitude of -150 volts, a rise time of 2 ns, and a recovery time of 100 ns. Voltage burst
tests consisted of repeating a series of 10 voltage bursts.
SwRI Report 03.12024.06
Page 241 of 371
-------
Tek Run: I.OOCS/s Sample
[ T-
ann 20.0 v
M 250ns Ch1 f
A: 150.4 V
@: -136.0 V
-26.0 V 9 Nov 2006
17:58:00
FIGURE 176. EXAMPLE VOLTAGE TRACE DURING A VOLTAGE BURST
Prior to testing, PEMS 7 was zeroed and spanned after warming for over one hour.
Periods of baseline or zero stimulation testing were included throughout the Conducted Transient
test to determine if the voltage disturbances had any effect on the PEMS measurements. To
capture potential measurement errors, the voltage perturbations, being extremely brief, were
timed to occur during the 30-second recorded measurements. Typically, one Conducted
Transient test was performed during each 30-second measurement.
Figure 177 shows the PEMS 7 zero delta measurements during Conducted Transient
testing. Zero events occurred at zero observation number 16 and 77, while a day break occurred
at observation number 92; requiring the PEMS to be zeroed and spanned. Due to the extended
length of time required to setup each Conducted Transient test, a large portion of the data
collected was zero stimulation baseline testing. Differentiating between the test deltas and
baseline deltas for each gaseous measurement is difficult, therefore, the voltage disturbances
delivered to the PEMS during Conducted Transient testing had little to no affect on the PEMS
measurements.
SwRI Report 03.12024.06
Page 242 of 371
-------
THC [ppmC] • CO [ppm] • NO [ppm] X NO2 [ppm]
Baseline A Zero Day Break A CO2 [%]
NOx [ppm]
0.028
0.024
0.020
0.016
-30
0.012 5
il
&
c
o
13
£
O
0.008 g
O
0.004
0.000
Reference Gas Delta Observations
FIGURE 177. PEMS 7 ENVIRONMENTAL RADIATION CONDUCTED TRANSIENT
ZERO DELTA MEASUREMENTS
Figure 178 and Figure 179 show the audit and span deltas for PEMS 7 during Conducted
Transient testing. Zero events occurred at observation number 6 and 26. The PEMS was zeroed
and spanned following the day break at observation number 31. Although each gaseous
pollutant exhibited audit and span deltas, the baseline deltas transitioned smoothly with the test
deltas, indicating the Conducted Transient testing had little affect on the PEMS measurement
systems.
SwRI Report 03.12024.06
Page 243 of 371
-------
fin _
on Deltas [ppm]
= 5 g 8 g 8 S
2
I -10 -
u
8 -20
-30
-40
50
• THC [ppmC] • CO [ppm]
• NO [ppm] X NO2 [ppm] • NOx [ppm]
• ••
• " . . • • .
• • A
A r«A
A •
A A A_A A
AAA * ^ -"— -"-
fijl?"**?) ***!*»
jfjai********
____ ^ ___ __
m m
A " ••" "•
A
A A A •
••!• • •
**I *»******3*0 f"$|** 4
•••xxSlg • •
x**^X
A
Reference Gas Delta Observations
FIGURE 178. PEMS 7 ENVIRONMENTAL RADIATION CONDUCTED TF
AUDIT DELTA MEASUREMENTS
fin _
Deltas [ppm]
B & c
Concentration
3 k &
• THC[ppmC] • CO [ppm]
• NO [ppm] X NO2 [ppm] • NOx [ppm]
"I1.!
A
I AA* ••*£•*
*•**!* ixl**»**«
*•* • • •
!•!_ ..•*__ __
A A
A A A
A "*•"*•
A A A •
•
• i * * * *
r*s •••• • ? ---
25 • 30 x x 35 A x x 4
Jxjx*Xi* tJt*0 ••* •
g * A • A •
" •
0.160
0.145
0.131
0.116 ^
U)
0.102 |
0.087 o
13
0.073 •£
0)
0.058 §
O
0.044 o
O
0.029
0.015
0.000
IANSIENT
0 08
0.06
g
0.04 £
c
0
0.02 13
° 1
u
0.00 8
O
o
-0.02
-nn4
Reference Gas Delta Observations
FIGURE 179. PEMS 7 ENVIRONMENTAL RADIATION CONDUCTED TRANSIENT
SPAN DELTA MEASUREMENTS
Figure 180 shows the EFM zero measurements during Conducted Transient testing.
Most of the exhaust flow meter 30-second average measurements were less than 0.2 scfm.
SwRI Report 03.12024.06
Page 244 of 371
-------
Occasional measurements were recorded above 0.5 scfm, however, several of these
measurements occurred during baseline testing. The maximum observed delta of 1.5 scfm is less
than 0.1% of the meter's 1700 scfm full scale flow rating.
2.0
-*
Ol
1.0 -
O
Ol
HI
•s
OL
^
t 0.0
-0.5
* Flow Meter [scfm] Baseline
*
*
20
40
60
80
100
120
140
160
180
-1.5
Exhaust Flow Meter Zero Observations
FIGURE 180. 5-INCH EFM ENVIRONMENTAL RADIATION CONDUCTED
TRANSIENT ZERO DELTA MEASUREMENTS
5.6.5 Radiation Error Surface Generation
As described in the Test Plan, the measurement error data gathered during the Bulk
Current Injection, Radiated Immunity, Electrostatic Discharge, and Conducted Transients testing
was to be pooled to generate a radiation error surface. Because the radiation tests challenged the
PEMS at the most extreme radiation levels the equipment would likely be subjected to during in-
use testing, the radiation error surface was to be processed differently than the Temperature and
Pressure error surfaces. The pooled radiation data was to be corrected for the environmental
baseline bias and variation. The 5th, 50th, and 95th percentile values of the pooled, corrected
radiation data was to be used to generate error surfaces at the zero, audit and span levels. These
radiation error surfaces were to be sampled normally.
As described in the previous radiation test sections, nearly all of the radiation delta data
resembled the corresponding baseline deltas; indicating the PEMS showed very little
susceptibility to the radiation tests. The only notable radiation susceptibility was for a few high
biased CO and CO2 measurements. Because the radiation error surface would use the 95th
percentile deltas, the Steering Committee elected to calculate the 95th percentile deltas for zero,
audit, and span to determine if the high CO and CO2 deltas caused by the radiation testing were
SwRI Report 03.12024.06
Page 245 of 371
-------
outside the 95th percentile delta. Shown in Figure 181 through Figure 183 are the pooled zero,
audit, and span delta measurements for all of the radiation testing. The baseline test segments
were extracted from the pooled radiation delta data set. The 95th percentile values are shown as a
line for each gaseous emission. The high CO and CO2 deltas were greater than the 95th
percentile; therefore, the radiation error surfaces would not include any measurement error data
caused by radiation effects. Based on this data, radiation error surfaces were not generated for
use in the Model.
THC [ppmC] • CO [ppm] • NOx [ppm]
NOx95th% A CO2 [%] CO2 95th "/
-THC 95th '
-CO 95th '
350
300
0.05
-0.03
Reference Gas Delta Observations
FIGURE 181. COMBINED RADIATION CHAMBER ZERO DELTA TEST RESULTS
SwRI Report 03.12024.06
Page 246 of 371
-------
THC [ppmC] • CO [ppm] • NOx [ppm]
NOx 95th % A CO2 [%] CO2 95th "/
-THC 95th '
-CO 95th '
350
0.35
-0.05
Reference Gas Delta Observations
FIGURE 182. COMBINED RADIATION CHAMBER AUDIT DELTA TEST RESULTS
THC [ppmC] • CO [ppm] • NOx [ppm]
NOx 95th % A CO2 [%] CO2 95th °/c
•THC 95th %
•CO 95th %
350 ^
0.56
-0.16
Reference Gas Delta Observations
FIGURE 183. COMBINED RADIATION CHAMBER SPAN DELTA TEST RESULTS
SwRI Report 03.12024.06
Page 247 of 371
-------
Shown in Figure 184, the radiation exhaust flow rate zero measurements were pooled
after the baseline test segments were removed from each test. As directed in the Test Plan, the
radiation error surfaces were to be generated using the 5th, 50th, and 95th percentile delta
observations from the pooled measurement errors. However, the PEMS set all negative flow rate
measurements to zero. With an incomplete distribution of negative flow rate delta
measurements, an alternate error surface generation method was developed. All zero
measurements were removed from the pooled radiation exhaust flow measurements as well as
the environmental baseline exhaust flow measurement data set. Approximately one third of the
data was removed from the both the radiation and baseline exhaust flow data sets. Removal of
the zero level exhaust flow rate measurements was necessary to more accurately compare the
variance of the radiation and baseline error data.
To avoid over counting exhaust flow rate measurement errors, it was necessary to remove
the bias and variance measured during environmental baseline testing from the radiation exhaust
flow rate data. With zero deltas removed, the median exhaust flow rate measurement from the
baseline test was subtracted from each radiation exhaust flow rate delta. This process was
inconsequential because the median baseline delta was less than 0.01 % of the flow meter's
maximum flow rating. Next the MAD of the baseline data was compared to the MAD of the bias
corrected radiation data. Using the equation below, the MAD values were used to calculate a
scaling factor. Each radiation exhaust flow measurement was multiplied by the scaling factor to
shrink the variance of the radiation data by the variance measured during environmental baseline
testing.
jMADRad2 -MAD
Scaling _ Factor =
2
BL
SwRI Report 03.12024.06 Page 248 of 371
-------
Raw Exhaust Flow • Corrected Exhaust Flow
-0.2
100
200 300 400
Reference Gas Delta Observation
500
600
FIGURE 184. POOLED EXHAUST FLOW RATE ZERO DELTAS MEASURED
DURING ENVIRONMENTAL RADIATION TESTING
After the radiation exhaust flow rate data was corrected for the baseline bias and
variance, the 5th, 50th, and 95th percentile deltas were generated for use in the radiation exhaust
flow rate error surface. With an incomplete delta distribution, the Steering Committee elected to
calculate the 95th percentile of the corrected radiation delta data. The negative of the 95th
-th
,th
percentile delta value was set to the 5 percentile and the 50 percentile was set to zero. The
final error surface for radiation exhaust flow rate is shown in Figure 185.
SwRI Report 03.12024.06
Page 249 of 371
-------
1
0
-2
Exhaust Flow Rate Deltas
[% of Max Flow Rate]
0.3
A
0.0
-4-
-0.3
X-Axis - Single Point
FIGURE 185. ERROR SURFACE FOR ENVIRONMENTAL RADIATION EXHAUST
FLOW RATE DELTA MEASUREMENTS
5.7 Vibration Table Testing
Vibration testing was performed to determine PEMS gaseous concentration measurement
errors due to vehicle vibrations. Due to mounting issues, the PEMS EFM was not used during
vibration testing. Shown in Figure 186, an Unholtz-Dickie Corporation electro-dynamic shaker
system was used to perform vibration testing with PEMS 3 in a Sensors Inc. environmental
enclosure. The shaker system used random movement to reproduce vibration defined by the
desired Power Spectral Density (PSD).
SwRI Report 03.12024.06
Page 250 of 371
-------
FIGURE 186. PEMS 3 IN AN ENVIRONMENTAL ENCLOSURE DURING VIBRATION
TESTING USING AN UNHOLTZ-DICKIE SHAKER SYSTEM
The vibration test was intended to simulate vibration typically experienced by the PEMS
during in-use on-road testing. As stated in the Test Plan, the Steering Committee originally
decided to test the PEMS using the PSD from the Mil Standard 810, US Highway Truck
Vibration Exposure. However, Sensors Inc. independently performed the Mil Standard 810 and
observed functional failures shortly after commencing the vibration test. After Sensors Inc.
reported the failures to the Steering Committee, SwRI was asked to generate vibration spectra
representative of an on-road truck. Vibration data collected with an Army M915A2 Semi-
Tractor, shown in Figure 187, was used to generate vibration spectra. The vibration data
collected with the M915A2 Tractor was comprised of accelerometer data at 8 locations on the
truck. The accelerometers were located on the truck frame as well as the cab.
SwRI Report 03.12024.06
Page 251 of 371
-------
FIGURE 187. ARMY M915A2 SEMI-TRACTOR USED TO GENERATE VIBRATION
SPECTRA FOR VIBRATION TESTING
After reducing the raw vibration data, SwRI proposed a revised PSD. However, the
M915A2 Tractor was limited to 55 mph during testing. With the speed of most on-road trucks
exceeding 55 mph, the Steering Committee elected to have SwRI generate a PSD at 70 mph.
The 55 mph, 70 mph, and Mil Standard 810 are shown in Figure 188. The 70 mph PSD resulted
in a vibration energy increase factor of 2.69 over the 55 mph PSD. SwRI also performed a
comparison of the lateral acceleration data and vertical acceleration data. Overall, the lateral and
vertical accelerations were nearly equal, therefore, one PSD was used for vibration testing in
each PEMS axis. Over 86 % of the M915A2 Tractor vibration energy was below 500 Hz.
SwRI Report 03.12024.06
Page 252 of 371
-------
0.000001
10
10000
FIGURE 188. POWER SPECTRAL DENSITIES EVALUATED FOR VIBRATION
TESTING
After warming for over one hour, PEMS 3 was zeroed and spanned with no stimulus
prior to vibration testing. Similar to temperature and pressure testing, the PEMS was zeroed
hourly, during which the shaker table was turned off. Using the 70 mph PSD, PEMS 3 was first
tested for lateral, side-to-side vibration. During the first hour of testing, the PEMS reported a
warning that the environmental enclosure heated line temperature was low. Also, the enclosure
cooling fans were not operational. The cause of the aforementioned problems was due to a
failure of the 12-volt power connector from the PEMS to the environmental enclosure. After
repairing the broken connector, PEMS 3 was tested for 3 more hours of side-to-side vibration.
Next, the PEMS was turned and tested for lateral, front-to-back vibration. After only 10-minutes
of vibration testing, the PEMS reported a high temperature fault for the FID and automatically
shutdown the FID. The PEMS was shutdown and restarted, however, the PEMS could not
communicate with the compact flashcard during the restart procedure. Without flashcard
communication, the user cannot log onto the PEMS. After trying several different flash cards
and numerous diagnostic measures, PEMS 3 was shipped to Sensors Inc. for repair. Sensors Inc.
diagnosed the problem as a failed ribbon cable and returned PEMS 3 to SwRI for further testing.
Unfortunately, most of the PEMS data was lost due to the cable failure.
After considering the PEMS failures using the 70 mph PSD, the Steering Committee
elected to proceed with vibration testing using the 55 mph PSD. With the possibility of another
functional failure, vibration testing was initially performed for only two hours in each direction
to generate vibration deltas for each axis. Also, the shaker table was turned off for a segment of
each hour of vibration testing to generate baseline data. The table was also turned off during
zero and span operations.
SwRI Report 03.12024.06
Page 253 of 371
-------
Figure 189 shows the PEMS 3 zero deltas measured during environmental vibration
testing. Zero events were performed hourly, with baseline test segments included during each
hour of vibration testing. The first two hours of testing were lateral, side-to-side vibration, hours
3 and 4 were lateral, front-to-back vibration, hours 5 through 7 were vertical vibration, hour 8
was lateral, side-to-side vibration, and hour 9 was lateral, front-to-back vibration. NO, NC>2, CO,
and CO2 zero deltas showed no evidence of vibration susceptibility when compared to the
baseline test segments.
• THC [ppmC] BCO [ppm] »NO [ppm] XNO2 [ppm] »NOx [ppm] Zero -Baseline ACO2 [%]
0.020
-0.015
Reference Gas Delta Observations
FIGURE 189. PEMS 3 ENVIRONMENTAL VIBRATION ZERO DELTA
MEASUREMENTS
Figure 190 shows the PEMS 3 THC zero delta measurements during vibration testing.
During test hour 3, which was lateral, front-to-back vibration, the PEMS showed elevated zero
deltas during vibration testing. During the baseline segment of hour 3, the THC zero deltas
returned to near zero levels. At the end of test hour number 3, the PEMS reported a fault
indicating the FID internal reference pressure was out of its limits. Curiously, the FID fuel bottle
was nearly empty at the end of test hour number 3. When replacing FID fuel bottle, the quick-
connect device used to connect the FID bottle to the PEMS was found to be broken. With a
functional quick-connect, the THC zero deltas showed no evidence of susceptibility to vibration
in any axis. The elevated THC zero measurements and low FID fuel bottle pressure were likely
caused by the failure of the FID fuel bottle connection device.
At the end of test hour 6, the PEMS reported a fault indicating the FID gas flow was too
high or too low. After test 6, the FID would not zero or span, although the FID interface
indicated the FID was operating at the correct temperature and the FID flame was lit. The PEMS
was restarted several times in attempt to restore FID operation, however, the FID would not zero
or span. Therefore, testing was continued without THC measurements after test hour 6. During
test hour number 9, a loud pop was heard from the PEMS and the THC measurement returned.
SwRI Report 03.12024.06
Page 254 of 371
-------
According to Sensors Inc., the pop noise was likely the FID lighting during vibration testing.
When the FID fuel bottle was removed after test 9, the quick-connect was again found to be
broken. The connection failures were most likely due to the lateral, front-to-back vibration
causing the FID fuel bottle to slide forward and backward in the PEMS case and stress the
connection.
30
20
O
Q.
Q.
10
£ o
c
O
O
O
-10
-20
-30
*THC [ppmC] Zero -Baseline
Deltas during vibration testing
Note: THC susceptibility caused by broken FID bottle quick connect
Note: FID failure after test 6
50
100 150
* Deltas during baseline testing
200
Reference Gas Zero Observations
FIGURE 190. PEMS 3 ENVIRONMENTAL VIBRATION THC ZERO DELTA
MEASUREMENTS
Figure 191 and Figure 192 show the audit and span deltas for PEMS 3 during
environmental vibration testing. CO and CO2 audit and span deltas showed no differences
between the vibration test data and baseline test data, indicating CO and CO2 measurements were
not susceptible to vibration. THC audit and span deltas were similar to the THC zero deltas and
showed elevated measurements during test hour number 3. As discussed previously, the elevated
THC deltas during hour 3 were likely caused by a broken FID fuel bottle connector. With a
functional FID bottle connector, the THC audit and span deltas were similar for vibration and
baseline testing. As discussed below, NOX deltas showed slightly elevated measurements during
lateral, side-to-side vibration when compared to the baseline testing. Most of the shift between
vibration and baseline testing was cause by the NO2 measurement, with NO measurements being
unaffected by the vibration tests.
SwRI Report 03.12024.06
Page 255 of 371
-------
• THC [ppmC] BCO [ppm] • NO [ppm] XNO2 [ppm] »NOx [ppm] -Baseline Zero Span ACO2 [%]
0.12
-----*?
-0.02
Reference Gas Delta Observations
FIGURE 191. PEMS 3 ENVIRONMENTAL VIBRATION AUDIT DELTA
MEASUREMENTS
• THC [ppmC] BCO [ppm] • NO [ppm] XNO2 [ppm] «NOx [ppm] Zero -Baseline Span ACO2 [%]
0.16
30 • • - -• 0.12
0.04
- -0.08
-0.12
Reference Gas Delta Observations
FIGURE 192. PEMS 3 ENVIRONMENTAL VIBRATION SPAN DELTA
MEASUREMENTS
Figure 193 shows the NOX span deltas for PEMS 3 during vibration testing. During
lateral side-to-side vibration, test hours 1,2, and 8, there was a noticeable difference between the
SwRI Report 03.12024.06
Page 256 of 371
-------
deltas observed during vibration testing and the deltas observed during baseline testing. All
baseline deltas showed a definite negative shift compared to the vibration deltas directly prior to
and after the baseline segment. The slight NOX susceptibility to lateral, side-to-side vibration
was driven by the NO2 measurement deltas.
10
5
Q. "
£
TO
"oi
Q -5
c
o
"ra
1 -10
u
c
o
o
5 -15 f
-20
• NOx[ppm] Zero —Baseline Span
Note: Slight NOx susceptibility occurred during horizontal side-to-side vibration
30*
40
50
60
70
80
Baseline test segments
>*. **• ** -
A_
A
__
A A[ • I
Baseline test segmen
Reference Gas Delta Observation
FIGURE 193. PEMS 3 ENVIRONMENTAL VIBRATION NOX SPAN DELTA
MEASUREMENTS
Nearly all of the deltas measured during vibration matched the deltas measured during
the baseline test segments. Furthermore, the NOX susceptibility to lateral vibration was minor
and only evident during span measurements. Therefore, the Steering Committee elected not to
generate an environmental vibration error surface for use in the Monte Carlo Model.
5.8 Ambient Hydrocarbon Testing
The Ambient Hydrocarbon test was performed to determine the PEMS FID response to
varying levels and compositions of hydrocarbon in the ambient air. One source of potential FID
measurement errors included the use of ambient air as the FID burner air source. Hydrocarbon
in the ambient air would enter the FID reaction chamber as burner air, causing measurement
inaccuracy. Ambient air is also used to zero the FID, therefore, ambient hydrocarbons will also
affect the FID zero calibration. During laboratory engine testing and environmental testing, the
PEMS was zeroed using bottled zero air. Zero air was also overflowed to the FID burner air
inlet throughout the program. It was necessary to eliminate ambient hydrocarbon contamination
during engine and environmental testing to insure errors due to ambient hydrocarbons were only
capture during the Ambient Hydrocarbon test.
SwRI Report 03.12024.06
Page 257 of 371
-------
Ambient Hydrocarbon testing was performed with PEMS 2. Throughout testing, zero air
was overflowed to the inlet of the PEMS sample line. Instead of overflowing zero air to the FID
burner air inlet as was done during engine and environmental testing, air mixtures with varying
levels and compositions of hydrocarbons were introduced to the FID burner during Ambient
Hydrocarbon testing. Shown in Table 84, 9 combinations of Hexane and Methane were used to
contaminate the FID burner air. According to EPA, the Hexane and Methane concentrations
selected for Ambient Hydrocarbon testing were deliberately chosen to exceed typical ambient
hydrocarbon variation to insure the test capture the full range of possible THC measurement
errors. The hydrocarbon gas mixtures were generated using a 4 ppmC Hexane gas bottle, a 4
ppmC Methane gas bottle, a 16 ppmC Hexane bottle, and a 16 ppmC Methane bottle, all balance
air. A gas divider was used to blend the hydrocarbon gases 50/50 to achieve the desired
concentration levels. Two computer controlled electronic solenoid manifolds were used to
automatically control the hydrocarbon combinations.
TABLE 84. HEXANE AND METHANE CONTAMINATION COMBINATIONS USED
DURING AMBIENT HYDROCARBON TESTING
Hydrocarbon
Combination
1
2
3
4
5
6
7
8
9
Hexane
[ppmC]
0
2
8
0
2
8
0
2
8
Methane
[ppmC]
0
2
8
2
8
0
8
0
2
As specified in the Test Plan, the FID was stabilized using one of 7 different
combinations of FID air hydrocarbon contamination. After the FID had stabilized with a given
hydrocarbon combination, the FID was zeroed using zero air and spanned. The PEMS was then
set to sample zero air which was overflowed to the inlet of the heated sample line. Next, the FID
burner air was automatically cycled through the hydrocarbon combinations shown in Table 84.
Each hydrocarbon combination was sampled for 90 seconds; 60 seconds to purge and stabilize
and 30 seconds to record an averaged measurement. Taken from the Test Plan, Table 85 shows
the test sequence used for Ambient Hydrocarbon testing. The FID was first zeroed while using
the FID burner air hydrocarbon concentrations shown in the merged cells. After the FID was
zeroed, the PEMS was set to sample zero air and the FID burner air was cycled through the 10
hydrocarbon combinations levels to the right of the merged cells. Although only 9 hydrocarbon
combinations were possible, the first combination was repeated at the end of each test. The test
sequence was repeated one time.
SwRI Report 03.12024.06
Page 258 of 371
-------
TABLE 85. AMBIENT HYDROCARBON TEST SEQUENCE
Ambienet Hydrocarbons Test Sequence
Phase
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
Burner air
hydrocarbons during
Hexane,
ppm
0
2
8
0
2
Methane,
ppm
0
2
8
2
8
Burner air
hydrocarbons during
Hexane,
ppm
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
2
0
8
2
0
8
2
0
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
Methane,
ppm
0
2
8
2
8
0
8
0
2
0
2
8
0
8
0
2
0
2
8
2
8
2
0
2
0
8
0
8
2
8
0
2
8
2
8
0
8
0
2
0
2
8
0
8
0
2
0
2
8
2
Ambienet Hydrocarbons Test Sequence
Phase
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
Burner air
hydrocarbons during
Hexane,
ppm
8
0
2
8
Methane,
ppm
0
8
0
Burner air
hydrocarbons during
Hexane,
ppm
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
Methane,
ppm
8
0
2
0
2
8
2
8
0
8
0
2
8
2
8
0
8
0
2
0
2
8
0
8
0
2
0
2
8
2
8
0
2
0
2
8
2
8
0
8
Figure 194 and Table 86 show the THC measurements for the first test as specified in the
Ambient Hydrocarbon test sequence. As specified in the Test Plan, each test sequence was
repeated one time. The FID was zeroed with no FID burner air hydrocarbon contamination. The
FID burner air was then cycled through the hydrocarbon combinations shown in Table 86.
Having zeroed the FID with no Hexane or Methane FID air contamination, the initial and final
THC measurements, both with no FID air contamination, were near zero. With 2 ppmC Methane
SwRI Report 03.12024.06
Page 259 of 371
-------
contamination in the FID burner air, the PEMS measured approximately 18 ppmC THC while
sampling zero air. With 8 ppmC Methane contamination, the PEMS reported approximately 58
ppmC THC while sampling zero air. Hexane contamination introduced to the FID burner air had
little effect on the PEMS THC measurement. The PEMS use a charcoal filter in the FID burner
air line to absorb ambient hydrocarbon prior to reaching the FID. The charcoal filter apparently
absorbed nearly all of the Hexane, but had little effect on the Methane contamination.
Test 1a THC Test 1b THC
100
200
300
400
500 600
Time [s]
700
800
900
1000
FIGURE 194. THC MEASUREMENTS FOR TEST 1 OF THE AMBIENT
HYDROCARBON TEST SEQUENCE
SwRI Report 03.12024.06
Page 260 of 371
-------
TABLE 86. THC MEASUREMENTS FOR TEST 1 OF THE AMBIENT
HYDROCARBON TEST SEQUENCE
FID Air
Hexane
[ppmC]
0
2
8
0
2
8
0
2
8
0
FID Air
Methane
[ppmC]
0
2
8
2
8
0
8
0
2
0
Gas
Switch
Time [s]
90
180
270
360
450
540
630
720
810
900
Test 1a
THC
[ppmC]
0.5
18.3
58.1
16.6
58.1
2.9
58.9
2.9
17.8
0.3
Testlb
THC
[ppmC]
1.2
19.0
59.7
17.0
59.6
3.5
60.1
3.5
18.9
0.8
Pretest zero was performed with 0 ppmC Hexane and 0 ppmC Methane
Figure 195 and Table 87 show the THC measurements for the second test as specified in
the Ambient Hydrocarbon test sequence. Prior to performing the second test, the FID was
stabilized and zeroed with 2 ppmC Hexane and 2 ppmC Methane FID burner air contamination.
The FID air contamination was varied according to the concentrations specified in Table 87. The
first and last measurements occurred with FID air hydrocarbon contamination levels similar to
the hydrocarbon combination used to zero the FID. Therefore, the initial and final THC
measurements were near zero. With 8 ppmC Methane contamination, the PEMS reported
approximately 39 ppmC THC while sampling zero air. At 0 ppmC Methane contamination, the
PEMS reported approximately -15 ppmC THC while sampling zero air. As observed during test
number 1, Hexane contamination had little effect on the PEMS THC measurement.
SwRI Report 03.12024.06
Page 261 of 371
-------
Test 2a THC Test 2b THC
1000
Time [s]
FIGURE 195. THC MEASUREMENTS FOR TEST 2 OF THE AMBIENT
HYDROCARBON TEST SEQUENCE
TABLE 87. THC MEASUREMENTS FOR TEST 2 OF THE AMBIENT
HYDROCARBON TEST SEQUENCE
FID Air
Hexane
[ppmC]
2
8
0
2
8
0
2
8
0
2
FID Air
Methane
[ppmC]
2
8
0
8
0
2
0
2
8
2
Gas
Switch
Time [s]
90
180
270
360
450
540
630
720
810
900
Test 2a
THC
[ppmC]
0.5
40.6
-16.1
40.0
-13.8
-0.1
-13.9
0.0
40.3
0.3
Test 2b
THC
[ppmC]
0.2
39.0
-16.3
38.7
-14.0
-0.7
-14.1
0.0
39.0
0.0
Pretest zero was performed with 2 ppmC Hexane and 2 ppmC Methane
PEMS 2 would not zero with 8 ppmC Methane contamination in the FID burner air. The
PEMS would not perform an electronic zero with the high Methane FID air contamination
because the measured THC zero concentration was outside the zero limits of the PEMS.
Therefore, test 3, 5, and 7, each requiring the FID to be zeroed with 8 ppmC Methane
contamination, were eliminated from test sequence. Each test repeat resulted in similar zero
SwRI Report 03.12024.06
Page 262 of 371
-------
measurements with the exception of test 6. Shown in Figure 196, the test 6a THC trace appeared
normal, but the test 6b THC measurement was lower than expected. The test was repeated,
however, the test 6c THC trace appeared normal, and the test 6d trace appeared low. No
explanation for the variation in the test 6 results was evident. Also, test 9, which was identical to
test 6, had both repeats similar to test 6a and test 6c. The results from test 6 were not used in the
final data analysis due to the measurement variation and because test 9 was a duplicate of test 6.
1000
Time [s]
FIGURE 196. THC MEASUREMENTS FOR TEST 6 OF THE AMBIENT
HYDROCARBON TEST SEQUENCE
5.8.1 Ambient Hydrocarbon Error Surface Generation
The 30-second mean zero air THC measurements were pooled from test 1, 2, 4, 8, and 9
of the Ambient Hydrocarbon test sequence. The THC measurements were multiplied by 0.98 to
generate NMHC zero measurement errors. As discussed previously, Hexane had little effect on
the PEMS THC measurement due to the use of a charcoal filter. Therefore, the PEMS THC
measurement errors were driven by the Methane contamination in the FID burner air. Shown in
Figure 197, the pooled PEMS NMHC errors generated a nearly linear relationship with respect to
the FID air Methane contamination concentration.
SwRI Report 03.12024.06
Page 263 of 371
-------
FID Air Methane Contamination [ppmC]
FIGURE 197. PEMS 3 NMHC RESPONSE VERSUS FID AIR METHANE
CONTAMINATION MEASURED DURING AMBIENT HYDROCARBON TESTING
The linear relationship between the PEMS NMHC zero measurement error and the FID
air Methane contamination concentration was used to calculate the 5th and 95th percentile values
for the Ambient Hydrocarbon error surface. With the calculated NMHC response to ambient
Methane, the only task remaining was to determine appropriate 5th and 95th percentile values for
the real world ambient Methane levels. A great deal of deliberation was exercised by the
Steering Committee to determine appropriate levels of variation for ambient Methane. As
written in the test plan, Methane levels recorded by CE-CERT during the on-road validation
were used as a reference. During CE-CERT's testing, the maximum change in Methane
concentration was 1.8 ppmC. However, the objective of the Ambient Hydrocarbon test was to
capture the worst case Methane variation. Therefore, CE-CERT's Methane data could only be
used the minimum Methane variation. Historical Methane data was then examined from EPA,
SwRI and engine manufacturer test labs. The difference between the 5th and 95th percentile
values for each lab was considered the worst case Methane variation. Using the pool of Methane
data, the Steering Committee ultimately agreed to use a Methane variation of 2.2 ppmC.
With no justification to bias the error surface, the 5th percentile Methane concentration
was set to -1.1 ppmC, while the 95th percentile Methane concentration was set to 1.1 ppmC.
Using the linear relationship between the PEMS NMHC response and ambient Methane
variation, the 5th and 95th percentile Methane concentrations were used to calculate the 5th and
95th percentile PEMS NMHC deltas for the final Ambient Hydrocarbon error surface. Shown in
-th
-th
Figure 198, the 5 and 95 percentile NMHC deltas were -7.5 and 7.5 ppmC respectively. The
Ambient Hydrocarbon error surface was sampled normally and applied to each reference NTE
event NMHC concentration regardless of level.
SwRI Report 03.12024.06
Page 264 of 371
-------
2 -,
E
•*,
o
-2
NMHC Delta [ppmC]
7.5
0.0
I n ,
-7.5
Single X-Axis Point
FIGURE 198. ERROR SURFACE FOR AMBIENT HYDROCARBON TESTING
SwRI Report 03.12024.06
Page 265 of 371
-------
6.0 MODEL RESULTS AND VALIDATION
6.1 Model Results
The objective of this section is to present the results from the Monte Carlo simulation
runs, the post-processing computations of the measurement allowances, and the model validation
analyses. Useful background information is contained in Section 2.0 of this report where the
Monte Carlo error model is described along with the techniques and methods used in the model
simulation runs and the validation process. A total of 10,000 to 30,000 simulation trials (with
four NTE events running to 50,000 trials) were run for each of the reference 195 NTE events,
and approximately four million simulation trials were run for all the combination of settings for
these 195 reference events.
6.2 Results of Drift Correction
This section contains a summary of the number and percent of the simulation trials that
were deleted due to periodic drift. Section 2.1.5 on Periodic Drift Check contains a detailed
description and a flowchart of the procedure used to check whether or not a periodic drift
invalidated any of the reference NTE event trials. This procedure was applied to the simulation
data obtained for each of the three emissions for each of the three calculation methods.
No periodic drift was detected for the BSCO emissions for any of the three calculation
methods. Thus, no simulation trials were deleted in any of the drift correction checks for BSCO
for all 195 reference NTE events. However, for BSNOX and BSNMHC, periodic drift was
detected.
Figure 199 through Figure 201 display relative frequency (in percent) histograms for the
percent of simulation trials for BSNOx, using each of the three calculation methods, that were
deleted for each of the 195 reference NTE events due to periodic drift. A summary of the results
is given in Table 88. For the three calculation methods, the average percent of the simulation
trials that was deleted across the 195 reference NTE events ranged from 2.09% for Method 2 to
3.45% for Method 1. The maximum percent that was deleted ranged from 11.27% for Method 2
to 15.41% for Method 1.
SwRI Report 03.12024.06 Page 266 of 371
-------
BSNOx Method 1
0)
O)
CC
+J
c
a)
e
0)
0.
>
O
c
0)
60
50
40
30
20
10
0
n
D
2 4 6 8 10 12 14
% Trials Deleted Due to Periodic Drift
16
FIGURE 199. PERCENT OF TRIALS DELETED FOR EACH REFERENCE NTE
EVENT DUE TO PERIODIC DRIFT CHECK FOR BSNOx METHOD 1
60
o
t
e
«
o
c
0)
D
CT
BSNOx Method 2
2 4 6 8 10 12 14
% Trials Deleted Due to Periodic Drift
16
FIGURE 200. PERCENT OF TRIALS DELETED FOR EACH REFERENCE NTE
EVENT DUE TO PERIODIC DRIFT CHECK FOR BSNOx METHOD 2
SwRI Report 03.12024.06
Page 267 of 371
-------
BSNOx Method 3
0
O)
S
0)
o
60
50
£ 10
2 4 6 8 10 12 14
% Trials Deleted Due to Periodic Drift
16
FIGURE 201. PERCENT OF TRIALS DELETED FOR EACH REFERENCE NTE
EVENT DUE TO PERIODIC DRIFT CHECK FOR BSNOx METHOD 3
TABLE 88. SUMMARY OF THE TRIALS DELETED DUE TO PERIODIC DRIFT
CHECK FOR BSNOx
Method
1
2
3
# Reference
NTE Events
195
195
195
Mean %
3.45
2.09
2.11
Min %
0
0
0
Max %
15.41
11.27
11.32
Figure 202 through Figure 204 display relative frequency (in percent) histograms for the
percent of simulation trials for BSNMHC, using each of the three calculation methods, that were
deleted for each of the 195 reference NTE events due to periodic drift. A summary of the results
is given in Table 89. For the three calculation methods, the average percent of the simulation
trials that was deleted across the 195 reference NTE events ranged from 1.89% for Method 2 to
4.01% for Method 1. The maximum percent that was deleted ranged from 14.33% for Method 2
to 21.21% for Method 1.
SwRI Report 03.12024.06
Page 268 of 371
-------
BSNMHC Method 1
0)
100
4 8 12 16 20
% Trials Deleted Due to Periodic Drift
FIGURE 202. PERCENT OF TRIALS DELETED FOR EACH REFERENCE NTE
EVENT DUE TO PERIODIC DRIFT CHECK FOR BSNMHC METHOD 1
BSNMHC Method 2
IUU
0)
O)
re an
4-1 OU
c
0)
*- R(\
0) DU
a.
>
o An
^ 4U
0)
3
°" on
0) ZU
II
n
"
-
-
-
-
i
-
-
-
-
-
-
4 8 12 16 20 24
% Trials Deleted Due to Periodic Drift
FIGURE 203. PERCENT OF TRIALS DELETED FOR EACH REFERENCE NTE
EVENT DUE TO PERIODIC DRIFT CHECK FOR BSNMHC METHOD 2
SwRI Report 03.12024.06
Page 269 of 371
-------
BSNMHC Method 3
a, IUU
O)
JS on
£ Oil
0)
o
»r en
d) OU
0.
O An
0)
S- on
CT £\J
£
"- o
•
-
-
•
-
1
1
-
_
_
-
-
-
4 8 12 16 20 24
% Trials Deleted Due to Periodic Drift
FIGURE 204. PERCENT OF TRIALS DELETED FOR EACH REFERENCE NTE
EVENT DUE TO PERIODIC DRIFT CHECK FOR BSNMHC METHOD 3
TABLE 89. SUMMARY OF TRIALS DELETED FOR EACH REFERENCE NTE
EVENT DUE TO PERIODIC DRIFT CHECK FOR BSNMHC METHOD 3
Method
1
2
3
#RefNTE
Events
195
195
195
Mean %
4.01
1.89
1.93
Min %
0.05
0.00
0.00
Max %
21.21
14.33
14.35
6.3 Convergence Results from MC Runs
This section contains a summary of the checks to determine if the convergence criteria
were met for the simulation runs. Section 2.1.7 on Convergence and Number of Trials contains a
detailed description of the convergence methodology and the procedures followed to check for
convergence for the reference NTE event trials. This procedure was applied to the simulation
data obtained for each of the three emissions and all three calculation methods.
Figure 205 through Figure 207 contain relative frequency (in percent) histograms for the
widths of the 90% confidence intervals for the 95th percentiles of the corresponding BSNOx
differences for the 195 individual reference NTE events where the confidence interval widths are
expressed as a percent of the BSNOx emissions NTE threshold. This is done for each of the three
calculation methods. A summary of the results is given in Table 90. Of interest was whether or
SwRI Report 03.12024.06
Page 270 of 371
-------
not the simulations converged within 1% of the threshold value. For the three calculation
methods, the maximum percent of the confidence interval widths that were within 1% of the
threshold value across the 195 reference NTE events ranged from 0.481% for Method 2 to
0.984% for Method 1. Thus, all 195 reference events met the convergence criteria.
BSNOx Method 1
<1>
O)
CC
£
O
O
0.
O
c
a)
D
0)
30
T
0.2 0.4 0.6 0.8 1
Convergence Interval Width/Threshold, %
FIGURE 205. CONVERGENCE INTERVAL WIDTH AS A PERCENT OF
THRESHOLD FOR BSNOX METHOD 1
SwRI Report 03.12024.06
Page 271 of 371
-------
BSNOx Method 2
ou
Q)
O)
CO /in
j3 4U
c
a)
*- in
0) oil
0.
>
o on
c 20
03
3
O" *r\
o nu
i
n
-
-
-
-
-
-
,
n
-
-
-
-
-
-
0 0.2 0.4 0.6 0.8 1
Convergence Interval Width/Threshold, %
FIGURE 206. CONVERGENCE INTERVAL WIDTH AS A PERCENT OF
THRESHOLD FOR BSNOx METHOD 2
BSNOx Method 3
ou
O)
RJ Af\
+1 4U
*- in
O oU
0.
o on
c 20
O" -in
-------
TABLE 90. SUMMARY OF BSNOX CONVERGENCE INTERVAL WIDTH AS A
FUNCTION OF THRESHOLD FOR 195 REFERENCE NTE EVENTS
Method
1
2
3
#RefNTE
Events
195
195
195
Mean %
0.534
0.313
0.444
Min %
0.236
0.157
0.190
Max %
0.984
0.481
0.889
Figure 208 through Figure 210 contain relative frequency (in percent) histograms for the
widths of the 90% confidence intervals for the 95th percentiles of the corresponding BSNMHC
differences for the 195 individual reference NTE events where the confidence interval widths are
expressed as a percent of the BSNMHC emissions NTE threshold. This is done for each of the
three calculation methods. A summary of the results is given in Table 91. Of interest was
whether or not the simulations converged within 1% of the threshold value. For the three
calculation methods, the maximum percent of the confidence interval widths that were within 1%
of the threshold value across the 195 NTE events ranged from 0.429% for Method 2 to 0.604%
for Method 1. Thus, all 195 events met the convergence criteria.
BSNMHC Method 1
O)
c
0)
O
03
0.
>
O
c
0)
80
60
40
20
0.2 0.4 0.6 0.8
Convergence Interval Width/Threshold, %
FIGURE 208. CONVERGENCE INTERVAL WIDTH AS A PERCENT OF
THRESHOLD FOR BSNMHC METHOD 1
SwRI Report 03.12024.06
Page 273 of 371
-------
BSNMHC Method 2
0)
O)
CC
+J
c
0)
e
a)
0.
>
O
c
0)
40
20
0.2 0.4 0.6 0.8 1
Convergence Interval Width/Threshold, %
FIGURE 209. CONVERGENCE INTERVAL WIDTH AS A PERCENT OF
THRESHOLD FOR BSNMHC METHOD 1
BSNMHC Method 3
0)
o
(0
•+•»
0)
e
0)
0.
&
0)
0)
80
60
40
20
0 0.2 0.4 0.6 0.8 1
Convergence Interval Width/Threshold, %
FIGURE 210. CONVERGENCE INTERVAL WIDTH AS A PERCENT OF
THRESHOLD FOR BSNMHC METHOD 3
SwRI Report 03.12024.06
Page 274 of 371
-------
TABLE 91. SUMMARY OF BSNMHC CONVERGENCE INTERVAL WIDTH AS A
FUNCTION OF THRESHOLD FOR 195 REFERENCE NTE EVENTS
Method
1
2
3
#RefNTE
Events
195
195
195
Mean %
0.222
0.188
0.197
Min %
0.113
0.090
0.005
Max %
0.604
0.429
0.480
Figure 211 through Figure 213 contain relative frequency (in percent) histograms for the
widths of the 90% confidence intervals for the 95th percentiles of the corresponding BSCO
differences for the 195 individual reference NTE events where the confidence interval widths are
expressed as a percent of the BSCO emissions NTE threshold. This is done for each of the three
calculation methods. A summary of the results is given in Table 92. Of interest was whether or
not the simulations converged within 1% of the threshold value. For the three calculation
methods, the maximum percent of the confidence interval widths that were within 1% of the
threshold value across the 195 NTE events ranged from 0.076% for Method 2 to 0.202% for
Method 3. Thus, all 195 events met the convergence criteria.
BSCO Method 1
O)
1
0)
O
03
0.
>
O
c
0)
100
80
60
40
20
0.1 0.2 0.3 0.4 0.5
Convergence Interval Width/Threshold, %
FIGURE 211. CONVERGENCE INTERVAL WIDTH AS A PERCENT OF
THRESHOLD FOR BSCO METHOD 1
SwRI Report 03.12024.06
Page 275 of 371
-------
BSCO Method 2
IUU
0)
O)
+2 OU
c
*~ fin
Q) DU
0.
o AO
c
a)
°" on
0) ZU
ul
n
~
-
_
-
-
;
-
-
-
-
0 0.1 0.2 0.3 0.4 0.5
Convergence Interval Width/Threshold, %
FIGURE 212. CONVERGENCE INTERVAL WIDTH AS A PERCENT OF
THRESHOLD FOR BSCO METHOD 2
BSCO Method 3
a, IUU
O)
-2 on
£ OU
0)
*~ en
(1) OU
0.
u* An
o 4U
0)
= 20
7 ^U
0)
^ 0
.
-
-
.
-
_
-
-
-
0 0.1 0.2 0.3 0.4 0.5
Convergence Interval Width/Threshold, %
FIGURE 213. CONVERGENCE INTERVAL WIDTH AS A PERCENT OF
THRESHOLD FOR BSCO METHOD 3
SwRI Report 03.12024.06
Page 276 of 371
-------
TABLE 92. SUMMARY OF BSCO CONVERGENCE INTERVAL WIDTH AS A
FUNCTION OF THRESHOLD FOR 195 REFERENCE NTE EVENTS
Method
1
2
3
#RefNTE
Events
195
195
195
Mean %
0.034
0.025
0.032
Min %
0.013
0.009
0.013
Max %
0.102
0.076
0.202
6.4 Delta BS Emissions Plots for 95th Percentiles
This section contains plots of the 95th percentile delta emissions values obtained by
simulation for each reference NTE event distribution of BS differences for each emissions for all
three calculation methods. Section 2.1.8 on Simulation Output contains more details on the
simulation output and the methodology used to compute the delta values.
Figure 214 through Figure 216 display box plots of the 95th percentile delta emissions for
all three methods. The ends of the boxes mark the location of the 25th and 75th percentiles of the
delta emissions while the horizontal line inside the box indicates the location of the median, or
50th percentile, of the data. The plus sign inside the box denotes the location of the mean of the
data. The two vertical lines (i.e., whiskers) extending above and below the box denote the
distance to the farthest observation that does not exceed the endpoint +/- 1.5*(height of the box).
Observations outside these vertical lines are individually noted on the graph as square symbols.
Viewing Figure 214, it can be observed that the 95th percentile delta BSNOx values based
on Method 1 had larger values and more spread in the data than did the corresponding values
computed using Methods 2 and 3. Further, the delta values based on Method 2 had the smallest
variation and the lowest values of all three methods.
SwRI Report 03.12024.06
Page 277 of 371
-------
1.6
1.2
I
3>
5 °-8
o 0.4
Q
2
Method
FIGURE 214. BOX PLOT OF 95TH PERCENTILE DELTA BSNOX FOR THREE
METHODS FROM 195 REFERENCE NTE EVENTS
Viewing Figure 215, the 95th percentile delta BSNMHC values are not as different for the
three methods as the delta BSNOx values. The delta values based on Method 1 are slightly
higher than similar values based on the other two methods, and the data is more skewed to the
right than the data for the other two methods. Again, the delta values based on Method 2 had the
lowest values of all three methods.
SwRI Report 03.12024.06
Page 278 of 371
-------
.c
£
0)
o"
X
"53
Q
0.12
0.1
0.08
0.06
0.04
0.02
0
I •*- I
r * ~
~ * ~
n
I
1
_l_ 1 1 1 1 1 1
1 2 3
Method
-TH
FIGURE 215. BOX PLOT FOR 95ltt PERCENTILE DELTA BSCO FOR THREE
METHODS FROM 195 REFERENCE NTE EVENTS
Viewing Figure 216, the 95th percentile delta BSCO values are similar in spread to the
data for the delta BSNMHC values. The delta values based on Method 1 are again slightly
higher than similar values based on the other two methods, and the data is more skewed to the
right than the data for the other two methods. The delta values based on Method 2 had the
lowest values of all three methods, and there is some skewness to the right for the data based on
all three methods.
SwRI Report 03.12024.06
Page 279 of 371
-------
!_
^
O)
O"
o
£
0)
O
2.4
2
1.6
1.2
00
.0
0.4
0
— —
*•
*
*
*
_ *
D
T f
_l_ I * 1 I | 1
_
1 2 3
Method
FIGURE 216. BOX PLOT FOR 95TH PERCENTILE DELTA BSCO FOR THREE
METHODS FOR 195 REFERENCE NTE EVENTS
-th
An alternative way to compare the 95 percentile deltas across the three calculation
methods is to plot the deltas for the same reference NTE event on a scatter plot, as illustrated in
Figure 217 through Figure 219. As seen in Figure 217, the ideal BSNOX for each reference NTE
-th
event is plotted against its 95 percentile deltas for calculation methods 1, 2, and 3. This plot
-th
depicts the large range in 95 percentile deltas for Method 1 as compared to Methods 2 and 3.
Similar inferences can be made for BSNMHC (Figure 218) and BSCO (Figure 219).
SwRI Report 03.12024.06
Page 280 of 371
-------
0.00
0.00
NOx g/kW-hr Method 1, 2 and 3
With Time Alignment Adjustment
1.00 2.00
3.00 4.00 5.00
Ideal NOx g/kW-hr
6.00
7.00
8.00
• Delta NOx Method 1 • Delta NOx Method 2 Delta NOx Method 3
-TH
FIGURE 217. COMPARISON OF 951" PERCENTILE DELTA BSNOX FOR METHODS
1, 2, AND 3 FOR 195 REFERENCE NTE EVENTS
0.12
O
->•
o
% Delta NMHC g/kW-
O O O
b b b
-t*. en CD
NMHC g/kW-hr Method 1, 2 and 3
With Time Alignment Adjustment
/y^* ,,,,%*. . •
0021 i * \|yE? * 1 ~*~
0.00 -J , , , , , , , ,
0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45
Ideal NMHC g/kW-hr
• Delta NMHC Method 1 • Delta NMHC Method 2 Delta NMHC Method 3
-TH
FIGURE 218. COMPARISON OF 95ltt PERCENTILE DELTA BSNMHC FOR
METHODS 1, 2, AND 3 FROM 195 REFERENCE NTE EVENTS
SwRI Report 03.12024.06
Page 281 of 371
-------
2 50
JE
1
o
o
re
Q 1 00 ,
•£ 1
5* r
£
s r
050 -I
0 00
0.
CO g/kW-hr Method 1, 2 and 3
With Time Alignment Adjustment
•
»
•
•
*_: - - '
'&&**• • :• •
.,' • f^v^r •
J'fy^^m m
', _-,>>*."
DO 1.00 2.00 3.00 4.00 5.00 6.00 7.
Ideal CO g/kW-hr
• Delta CO Method 1 • Delta CO Method 2 Delta CO Method 3
DO
-TH
FIGURE 219. COMPARISON OF 95ltt PERCENTILE DELTA BSCO FOR METHODS
1, 2, AND 3 FROM 195 REFERENCE NTE EVENTS
6.5 Sensitivity Based on Variance
This section contains a summary of the error surfaces that contributed the most to the
variance of the generated BS emissions. During the MC simulation for each reference NTE
event, sensitivity charts produced by Crystal Ball were generated and stored in the REPORT
files. Crystal Ball calculates sensitivity by computing the rank correlation coefficient between
every assumption (error surface) and forecast value (delta BS emissions) while the simulation is
running. Positive rank correlations indicate that an increase in the assumption is associated with
an increase in the forecast. The larger the absolute value of the rank correlation the stronger the
relationship.
Sensitivity charts in Crystal Ball provide a means by which the variance of the error
surfaces affects the variance in the forecast values. Hence, the sensitivity charts developed
during a MC simulation are displayed as "Contribution to Variance" charts which are calculated
by squaring the rank correlation coefficients for all assumptions used in a particular forecast and
then normalizing them to 100%. The assumption (error surface) with the highest contribution to
variance (in absolute value of the percent) is listed first in the sensitivity chart.
Simulation results from all 195 reference NTE events produced sensitivity values for all
three 95th percentile delta emissions by all three calculation methods. Table 93 through Table
101 summarize the error surfaces in which the contribution to the variance sensitivity value was
at least 5% in magnitude compared to all the other error surfaces. Note that the number of error
SwRI Report 03.12024.06
Page 282 of 371
-------
surfaces whose sensitivity values were greater than 5% ranged from 3 to 6 for all three delta
emissions and three calculation methods. Also note that while some error surfaces were
sensitive for most of the 195 reference NTE events (e.g., 195 events for l_NOx_SS BSNOX
Method 2 in Table 94), others were sensitive for a small fraction of the reference NTE events
(e.g., only 17 events for 2_NOx_Transient for BSNOX Method 2 in Table 94).
Table 93 through Table 95 list the sensitivity descriptive statistics for the delta BSNOX
emissions for Methods 1, 2 and 3, respectively. In the first column of these tables the error
surfaces with at least a 5% contribution to variance are listed followed by the number of
reference NTE events in which this occurred. The mean contribution-to-variance normalized
percentage is also given along with the minimum and maximum values. For Methods 1 and 3,
the largest mean normalized variance was from error surface #31, torque warm-up, followed
closely by error surface #1, NOX steady state. For Method 2, the largest mean normalized
variance was from error surface #1, NOX steady-state, followed by error surface #42 due to
BSFC from the engine manufacturers' data.
TABLE 93. ERROR SURFACE SENSITIVITY TO VARIANCE FOR 195 REFERENCE
NTE EVENTS FOR BSNOX METHOD 1
Error Surface
1 NOxSS
20 Exhaust Flow SS
3 1 Torque Warm-up
35 Torque Engine
Manufacturers
#RefNTE
Events
195
185
193
192
Mean,
%
27.90
10.34
-34.65
-15.96
Minimum,
%
9.40
5.30
-61.40
-25.40
Maximum,
%
99.60
15.70
-11.40
-1.10
TABLE 94. ERROR SURFACE SENSITIVITY TO VARIANCE FOR 195 REFERENCE
NTE EVENTS FOR BSNOX METHOD 2
Error Surface
1 NOxSS
2 NOx Transient
37 BSFC DOE
38 BSFC Warm-up
41 BSFC Interpolation
42 BSFC Engine Manufacturers
#RefNTE
Events
195
17
93
182
56
193
Mean,
%
47.23
5.26
6.48
10.29
5.42
24.43
Minimum,
%
29.30
5.00
5.00
5.10
5.00
5.20
Maximum,
%
99.70
5.70
9.80
15.20
6.40
38.20
SwRI Report 03.12024.06
Page 283 of 371
-------
TABLE 95. ERROR SURFACE SENSITIVITY TO VARIANCE FOR 195 REFERENCE
NTE EVENTS FOR BSNOX METHOD 3
Error Surface
1 NOxSS
3 1 Torque Warm-up
35 Torque Engine
Manufacturers
#RefNTE
Events
195
194
193
Mean,
%
30.59
-38.37
-17.89
Minimum,
%
10.10
-65.80
-29.50
Maximum,
%
99.70
-12.00
-5.20
Table 96 through Table 98 list the sensitivity descriptive statistics for the delta BSNMHC
emissions for Methods 1, 2 and 3, respectively. For all three methods, the largest mean
normalized variance was from the error surface #19, NMHC ambient effect.
TABLE 96. ERROR SURFACE SENSITIVITY TO VARIANCE FOR 195 REFERENCE
NTE EVENTS FOR BSNMHC METHOD 1
Error Surface
13 NMHC SS
19 NMHC Ambient
20 Exhaust Flow SS
31 Torque Warm-up
35 Torque Engine
Manufacturers
#RefNTE
Events
134
195
14
38
30
Mean,
%
6.01
85.70
7.63
-16.78
-8.48
Minimum,
%
5.00
18.20
5.10
-48.20
-16.60
Maximum,
%
9.60
93.30
10.30
-5.20
-5.00
TABLE 97. ERROR SURFACE SENSITIVITY TO VARIANCE FOR 195 REFERENCE
NTE EVENTS FOR BSNMHC METHOD 2
Error Surface
13 NMHC SS
19 NMHC Ambient
37 BSFCDOE
38 BSFC Warm-up
42 BSFC Engine Manufacturers
#RefNTE
Events
122
195
1
7
30
Mean,
%
5.65
89.34
5.00
7.84
9.86
Minimum,
%
5.00
40.90
5.00
5.40
5.00
Maximum,
%
8.10
93.50
5.00
10.90
27.90
SwRI Report 03.12024.06
Page 284 of 371
-------
TABLE 98. ERROR SURFACE SENSITIVITY TO VARIANCE FOR 195 REFERENCE
NTE EVENTS FOR BSNMHC METHOD 3
Error Surface
13 NMHC SS
19 NMHC Ambient
31 Torque Warm-up
35 Torque Engine
Manufacturers
#RefNTE
Events
114
195
38
28
Mean,
%
5.64
86.98
-16.97
-8.86
Minimum,
%
5.00
22.30
-50.90
-16.70
Maximum,
%
8.20
93.30
-5.00
-5.10
Table 99 through Table 101 list the sensitivity descriptive statistics for the delta BSCO
emissions for Methods 1, 2 and 3, respectively. For all three methods, the largest mean
normalized variance was from the error surface #7, CO steady-state.
TABLE 99. ERROR SURFACE SENSITIVITY TO VARIANCE FOR 195 REFERENCE
NTE EVENTS FOR BSCO METHOD 1
Error Surface
7 COSS
20 Exhaust Flow SS
31 Torque Warm-up
35 Torque Engine
Manufacturers
52 CO Time Alignment
#RefNTE
Events
195
32
122
63
56
Mean,
%
76.20
7.48
-13.29
-1.11
7.40
Minimum,
%
9.30
5.00
-44.60
-17.60
5.00
Maximum,
%
96.30
11.60
-5.00
-5.00
15.80
TABLE 100. ERROR SURFACE SENSITIVITY TO VARIANCE FOR 195
REFERENCE NTE EVENTS FOR BSCO METHOD 2
Error Surface
7 COSS
37 BSFCDOE
38 BSFC Warm-up
42 BSFC Engine Manufacturers
52 CO Time Alignment
#RefNTE
Events
195
2
14
65
41
Mean,
%
85.82
5.35
7.07
9.12
8.61
Minimum,
%
20.10
5.00
5.00
5.00
5.00
Maximum,
%
98.10
5.70
10.90
28.10
21.70
SwRI Report 03.12024.06
Page 285 of 371
-------
TABLE 101. ERROR SURFACE SENSITIVITY TO VARIANCE FOR 195
REFERENCE NTE EVENTS FOR BSCO METHOD 3
Error Surface
7 COSS
3 1 Torque Warm-up
35 Torque Engine
Manufacturers
52 CO Time Alignment
#RefNTE
Events
195
120
62
97
Mean,
%
75.49
-13.40
-7.79
11.29
Minimum,
%
8.40
-43.20
-16.40
5.00
Maximum,
%
96.20
-5.10
-5.00
29.40
The contribution to normalized variance sensitivities from Table 93 through Table 95 are
illustrated pictorially as box plots in Figure 220 through Figure 222 for BSNOX Methods 1, 2 and
3, respectively. Only the error surfaces with at least 65 of the 195 reference NTE events (1/3 of
the events) are included as box plots. The mean normalized variance for each of the plotted error
surfaces is noted by a "+" symbol in the boxes. The error surface with the largest mean
normalized variance is plotted at the left of the chart. The error surface with the second largest
mean normalized variance is plotted second from the left, and so on. Figure 221 and Figure 223
demonstrate the high sensitivity to the error surface #31, torque warm-up. Figure 222 as well as
Figure 221 and Figure 223 illustrate the high sensitivity to the error surface #1, NOX steady-state.
SwRI Report 03.12024.06
Page 286 of 371
-------
Sensitivity Contribution to Variance for NCx Method 1
100-
Jf
I
I
(3
O
o
s
o-
-50-
-100 H
1
T
Vfermup_To rque_31
SS NOx 1
Eng_Manuf_To rq ue_35
SS Row 20
FIGURE 220. BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON
VARIANCE FOR BSNOX METHOD 1
SwRI Report 03.12024.06
Page 287 of 371
-------
Sensitivity Contribution to Variance for NOx Method 2
100-
of 75
I
50-
i
(3
Z
o
I
.Q
I"
o-t
T
SS NOx1
i i r
EncLManuf_BSFC_42 Vtermup_BSFC_38 DOE_BSFC_37
FIGURE 221. BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON
VARIANCE FOR BSNOX METHOD 2
SwRI Report 03.12024.06
Page 288 of 371
-------
Sensitivity Contribution to Variance for NOx Method 3
100-
50-
o
-50 -
-100-
n
D
a
a
n
a
i
T
Vtermup_To rque_31
SS NOx 1
E ng_Man uf_Torq ue_35
FIGURE 222. BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON
VARIANCE FOR BSNOX METHOD 3
The contribution to normalized variance sensitivities from Table 96 through Table 98 are
illustrated pictorially as box plots in Figure 223 through Figure 225 for BSNMHC Methods 1,2
and 3, respectively. Each of these figures demonstrates the high sensitivity to the error surface
#19, NMHC ambient effect.
SwRI Report 03.12024.06
Page 289 of 371
-------
Sensitivity Contribution to Variance for NMHC Method 1
100-
75-
I
50-
o
25-
o-t
NMHC Ambient 19
SS NMHC 13
FIGURE 223. BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON
VARIANCE FOR BSNMHC METHOD 1
SwRI Report 03.12024.06
Page 290 of 371
-------
Sensitivity Contribution to Variance for NMHC Method 2
100-
75-
I
50-
o
25-
o-t
NMHC Arrtiient 19
SS NMHC 13
FIGURE 224. BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON
VARIANCE FOR BSNMHC METHOD 2
SwRI Report 03.12024.06
Page 291 of 371
-------
Sensitivity Contribution to Variance for NMHC Method 3
100-
75-
I
50-
o
25-
o-t
NMHC Arrtiient 19
SS NMHC 13
FIGURE 225. BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON
VARIANCE FOR BSNMHC METHOD 3
The contribution to normalized variance sensitivities from Table 99 through Table 101
are illustrated pictorially as box plots in Figure 226 through Figure 228 for BSCO Methods 1,2
and 3, respectively. Each of these figures demonstrates the high sensitivity to error surface #7,
CO steady-state.
SwRI Report 03.12024.06
Page 292 of 371
-------
Sensitivity Contribution to Variance for CO Method 1
tf
o
§
100-
75-
50-
25-
-25"
-50 H
T
SS CO 7
Vfermup_1b rque_31
FIGURE 226. BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON
VARIANCE FOR BSCO METHOD 1
SwRI Report 03.12024.06
Page 293 of 371
-------
Sensitivity Contribution to Variance for CO Method 2
100-
6?
of 75
£
I
50-
o
"5
t 25-
o-t
n
n
SS CO 7
Eng_Manuf_BSFC_42
FIGURE 227. BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON
VARIANCE FOR BSCO METHOD 2
SwRI Report 03.12024.06
Page 294 of 371
-------
Sensitivity Contribution to Variance for CO Method 3
tf
o
§
100-
75-
501
25-
-25"
-50 H
T
D
D
SS CO 7
Vfermup_Tbrqiie_31
CO_Time_Align_52
FIGURE 228. BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON
VARIANCE FOR BSCO METHOD 3
6.6 Sensitivity Based on Bias and Variance
This section contains a summary of the error surfaces that contributed the most to the bias
of the generated BS emissions. The sensitivity charts developed in Crystal Ball help identify the
error surfaces (assumptions) that are sensitive to changes in variation with respect to their effect
on the three delta BS emissions. Another type of sensitivity examined in this study was
concerned with the effects of potential "bias" in error surfaces and their effects on the forecast
values. In order to study these effects a new error surface assumption was added to the MS
simulation model for each of the original 35 error surfaces (excluding the two error surfaces for
time alignment).
This assumption was sampled as a discrete binary distribution (i.e., on or off) during the
simulation. For each trial of the simulation, 35 original error surfaces and 35 'on/off error
surfaces were sampled according to their defined sample distribution. If the 'on/off error
surface produced an 'off condition, the delta emissions from that particular error surface were
not added to the BS emissions computations for the BS emissions 'with errors'. Similarly, if the
SwRI Report 03.12024.06
Page 295 of 371
-------
'on/off error surface produced an 'on' condition, the delta emissions from that particular error
surface were added to the BS emissions calculations.
During every trial of the simulation, the exclusions due to the 'off conditions resulted in
various combinations of the error surface delta emissions being added to the BS emissions 'with
errors' computations. Over the course of a MC simulation with thousands of trials, the
sensitivity of a particular error either 'on' or 'off was assessed by examining the change in the
forecast delta emission. Therefore, in a single MC simulation of a reference NTE event
sensitivities due to variance and/or bias were explored.
During this phase of the simulation, thirteen reference NTE events were selected to be re-
run with the additional 'on/off error surface assumptions included in the MC model. These
events were selected to bound the NTE BSNOX threshold of 2.6820 g/kW-hr and are listed in
Table 102 along with their ideal BSNOX values. Also included in Table 102 is the number of
trials run for each of the MC simulations. Figure 229 illustrates the distribution of the ideal
BSNOX values for the thirteen reference NTE events as a frequency histogram.
TABLE 102. IDEAL BSNOX VALUES FOR 13 REFERENCE NTE EVENTS
Reference NTE
Event #
38
44
87
148
82
163
63
46
51
69
157
1
25
Ideal BSNOx
g/kW-hr
0.0249
1.0730
1.5207
1.9985
2.4568
2.5907
2.6670
2.6957
2.8298
3.0257
3.4666
1.0713
5.4061
Trials Run
During MC
Simulation
10,000
10,000
10,000
10,000
10,000
10,000
10,000
30,000
10,000
30,000
30,000
30,000
30,000
SwRI Report 03.12024.06
Page 296 of 371
-------
CD
I 2
1
0
-1012345
Ideal NOx g/kW-hr
FIGURE 229. DISTRIBUTION OF IDEAL BSNOX FOR 13 REFERENCE NTE EVENTS
Monte Carlo simulations, including the additional 35 'on/off error surfaces, were run on
all thirteen reference NTE events for 10,000 or 30,000 trials each. EXTRACT data files and
REPORT files were generated for all three emissions and three calculation methods. All
reference NTE events converged within 1% of the NTE emissions threshold.
Simulation results from these reference NTE events produced sensitivity values for all
three 95th percentile delta emissions by all three calculation methods. Table 103 through Table
111 summarize the error surfaces in which either the contribution-to-variance normalized
sensitivity value or the 'on/off bias check for the error surface was at least 5% in magnitude
compared to all the other error surfaces. If the label in the error surface contains the words
'OnOff then it represents a check for bias; otherwise, the error surface indicates a check for
variance. Note that the number of error surfaces whose sensitivity values due to variance were
greater than 5% ranged from 3 to 5 for all three delta emissions and all three calculation
methods. Also note that all three emissions by all three calculation methods identified at least
one 'on/off error surface in which a bias effect was noted.
Table 103 through Table 105 list the sensitivity due to variance and bias descriptive
statistics for the delta BSNOX emissions for Methods 1, 2 and 3, respectively. In the first column
of these tables the error surfaces with at least a 5% contribution-to-variance are listed followed
by the number of reference NTE events in which this occurred. The mean contribution-to-
variance normalized percentage is also given along with the minimum and maximum values.
For Methods 1 and 3, the largest mean normalized variance was from error surface #31, torque
warm-up, which was the same result obtained in the previous analysis in Section 6.6. For
Method 2, the largest mean normalized variance was from the 'on/off error surface for CO2
steady-state followed by error surface #1 due to NOX steady-state.
SwRI Report 03.12024.06
Page 297 of 371
-------
TABLE 103. ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 13
REFERENCE NTE EVENTS FOR BSNOX METHOD 1
Error Surface
1 NOxSS
20 Exhaust Flow SS
3 1 Torque Warm-up
3 5_Torque Engine
Manufacturers
5 1 NOx Time Alignment
OnOff Exhaust Flow Pulsation
OnOff Exhaust Flow Swirl
OnOff Exhaust Flow SS
#RefNTE
Events
13
11
12
12
8
12
1
10
Mean,
%
21.02
6.33
-23.21
-9.78
6.05
14.70
5.70
14.58
Minimum,
%
8.40
5.40
-34.40
-12.20
5.40
5.60
5.70
8.70
Maximum,
%
92.70
8.10
-12.80
-7.40
7.10
19.00
5.70
21.30
TABLE 104. ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 13
REFERENCE NTE EVENTS FOR BSNOX METHOD 2
Error Surface
1 NOxSS
37 BSFCDOE
38 BSFC Warm-up
42 BSFC Engine Manufacturers
OnOff CO2 SS
OnOff BSFC DOE
#RefNTE
Events
13
1
6
12
12
8
Mean,
%
29.46
5.00
6.33
13.55
-34.93
-6.46
Minimum,
%
14.30
5.00
5.20
10.80
-39.40
-8.30
Maximum,
%
92.30
5.00
8.30
17.10
-20.80
-5.10
TABLE 105. ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 13
REFERENCE NTE EVENTS FOR BSNOX METHOD 3
Error Surface
1 NOxSS
31 Torque Warm-up
35 Torque Engine
Manufacturers
5 1 NOx Time Alignment
OnOff CO2 SS
#RefNTE
Events
13
12
12
7
12
Mean,
%
23.62
-27.83
-11.69
5.96
-25.22
Minimum,
%
11.20
-38.20
-15.00
5.00
-32.90
Maximum,
%
92.90
-15.90
-8.60
7.80
-15.90
Table 106 through Table 108 list the sensitivity and bias descriptive statistics for the delta
BSNMHC emissions for Methods 1, 2 and 3, respectively. For all three methods, the largest
mean normalized variance was from the error surface #19, NMHC ambient effect.
SwRI Report 03.12024.06
Page 298 of 371
-------
TABLE 106. ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 13
REFERENCE NTE EVENTS FOR BSNMHC METHOD 1
Error Surface
13 NMHC SS
19 NMHC Ambient
20 Exhaust Flow SS
31 Torque Warm-up
35 Torque Engine
Manufacturers
OnOff Exhaust Flow Pulsation
OnOff Exhaust Flow SS
OnOff NMHC SS
#RefNTE
Events
10
13
1
4
3
3
3
11
Mean,
%
7.05
58.33
6.40
-15.60
-8.37
14.80
11.00
13.89
Minimum,
%
6.10
11.90
6.40
-25.60
-9.80
9.20
8.20
5.00
Maximum,
%
8.70
73.80
6.40
-5.70
-5.90
19.30
12.40
18.80
TABLE 107. ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 13
REFERENCE NTE EVENTS FOR BSNMHC METHOD 2
Error Surface
13 NMHC SS
19 NMHC Ambient
38 BSFC Warm-up
42 BSFC Engine Manufacturers
OnOff CO2 SS
OnOff NMHC SS
#RefNTE
Events
10
13
2
3
3
11
Mean,
%
6.78
62.33
5.60
10.67
-24.67
13.95
Minimum,
%
6.00
21.60
5.50
7.40
-36.90
6.80
Maximum,
%
8.2
74.20
5.70
14.20
-12.60
19.30
TABLE 108. ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 13
REFERENCE NTE EVENTS FOR BSNMHC METHOD 3
Error Surface
13 NMHC SS
19 NMHC Ambient
3 1 Torque Warm-up
35 Torque Engine Manufacturers
OnOff CO2 SS
OnOff NMHC SS
#RefNTE
Events
10
13
4
3
3
11
Mean,
%
6.84
60.51
-18.68
-9.87
-18.63
13.64
Minimum,
%
6.00
14.90
-32.20
-11.80
-26.20
5.50
Maximum,
%
7.90
74.10
-5.40
-6.40
-11.00
17.50
Table 109 through Table 111 list the sensitivity and bias descriptive statistics for the delta
BSCO emissions for Methods 1, 2 and 3, respectively. For all three methods, the largest mean
normalized variance was from the 'on/off error surface for CO steady-state (average near 80%
for all three methods).
SwRI Report 03.12024.06
Page 299 of 371
-------
TABLE 109. ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 13
REFERENCE NTE EVENTS FOR BSCO METHOD 1
Error Surface
7 COSS
3 1 Torque Warm-up
3 5_Torque Engine
Manufacturers
52 CO Time Alignment
OnOff Exhaust Flow Pulsation
OnOff Exhaust Flow SS
OnOffSSCO
#RefNTE
Events
10
1
1
1
1
1
13
Mean,
%
6.06
-13.50
-5.20
20.00
10.80
6.20
80.63
Minimum,
%
5.00
-13.50
-5.20
20.00
10.80
6.20
31.30
Maximum,
%
7.30
-13.50
-5.20
20.00
10.80
6.20
91.30
TABLE 110. ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 13
REFERENCE NTE EVENTS FOR BSCO METHOD 2
Error Surface
7 COSS
42 BSFC Engine Manufacturers
52 CO Time Alignment
OnOff CO2 SS
OnOff CO SS
#RefNTE
Events
12
1
1
1
13
Mean,
%
6.31
5.80
20.00
-16.20
82.82
Minimum,
%
5.30
5.80
20.00
-16.20
43.80
Maximum,
%
7.40
5.80
20.00
-16.20
91.20
TABLE 111. ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 13
REFERENCE NTE EVENTS FOR BSCO METHOD 3
Error Surface
7 COSS
31 Torque Warm-up
52 CO Time Alignment
OnOff CO2SS
OnOff CO SS
#RefNTE
Events
9
1
6
1
13
Mean,
%
6.14
-13.10
12.10
-10.40
79.75
Minimum,
%
5.50
-13.10
5.50
-10.40
28.00
Maximum,
%
7.30
-13.10
36.00
-10.40
91.20
The contribution to normalized variance and bias sensitivities from Table 103 through
Table 105 are illustrated pictorially as box plots in Figure 230 through Figure 232 for BSNOX
Methods 1, 2 and 3, respectively. Only the error surfaces with at least 5 of the thirteen reference
NTE events (1/3 of the events) are included as box plots. The mean normalized variance for
each of the plotted error surfaces is noted by a "+" symbol in the boxes. The error surface with
the largest mean normalized variance is plotted at the left of the chart. The error surface with the
second largest mean normalized variance is plotted second from the left, and so on. Similar plots
could also be generated for BSNMHC and BSCO. Figure 230 and Figure 232 demonstrate the
high sensitivity to the error surface #31, torque warm-up. This was also seen in the analyses
using the 195 reference NTE events. Also note in Figure 230 that bias effects due to exhaust
SwRI Report 03.12024.06
Page 300 of 371
-------
flow pulsation and steady-state were important. Figure 231 and Figure 232 also show a bias
effect due to CC>2 steady-state.
Sensitivity Contribution to Variance and Bias for NOx Method 1
75
fl? 50
1
25 -
-25
-50 ~\
WaimupToiqie 31 SSHc« 1 PutseFlcw OnOff SSHow OnOff EngManufTorcjje35 SSHow 20 NoxTlmeAJi^i 51
FIGURE 230. BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON BIAS
AND VARIANCE FOR BSNOX METHOD 1
SwRI Report 03.12024.06
Page 301 of 371
-------
Sensitivity Contribution to Variance and Bias for NOx Method 2
100 -
75 -
50 -
#
o"
G
I 25
o
o
0 -'
-25 -
-50 H
D
n
SSCO2 OnOff
SSNox 1
EngMaiuBSFC 42
DOEBSFC OnOft
WarmupBSFC 38
FIGURE 231. BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON BIAS
AND VARIANCE FOR BSNOX METHOD 2
SwRI Report 03.12024.06
Page 302 of 371
-------
Sensitivity Contribution to Variance and Bias for NOx Method 3
100 -
#
o"
g 50 -
M
I
5
z
o
*-
c
o
i
•c
o
O
25 -
0 -
-25 -
-50 H
I
WarmupTorque_31
SSCO2 OnOft
SSNox 1
E ngManufTorque 35
NOxTimeAlign_51
FIGURE 232. BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON BIAS
AND VARIANCE FOR BSNOX METHOD 3
A summary of the sensitivity results due to variance only and both variance and bias MC
simulations are provided in Table 112 through Table 120 for all three emissions and all three
calculation methods.
SwRI Report 03.12024.06
Page 303 of 371
-------
TABLE 112. SUMMARY OF ERROR SURFACE SENSITIVE TO BIAS AND
VARIANCE FOR BSNOX METHOD 1
Sensitivity to Variance
1 95 RefNTE Events
#
1
20
31
35
Error Surface
SS NOx
SS Flow
Warmup Torque
Eng Manuf Torque
Avg
Contribution
to
Normalized
Variance, %
27.90
10.34
-34.65
-15.96
Sensitivity to Bias and Variance
13 RefNTE Events
#
1
20
31
35
51
Error Surface
SS NOx
SS Flow
Warmup Torque
Eng Manuf Torque
NOx Time Align
Pulse Flow OnOff
SS Flow OnOff
Avg
Contribution
to
Normalized
Variance, %
21.02
6.33
-23.21
-9.78
6.05
14.70
14.58
TABLE 113. SUMMARY OF ERROR SURFACE SENSITIVE TO BIAS AND
VARIANCE FOR BSNOX METHOD 2
Sensitivity to Variance
195 RefNTE Events
#
1
37
38
42
Error Surface
SS NOx
DOE BSFC
Warmup BSFC
Eng Manuf BSFC
Avg
Contribution
to
Normalized
Variance, %
47.22
6.48
10.29
24.43
Sensitivity to Bias and Variance
13 RefNTE Events
#
1
38
42
51
Error Surface
SS NOx
Warmup BSFC
Eng Manuf BSFC
NOx Time Align
SS CO2 OnOff
DOE BSFC OnOff
Avg
Contribution
to
Normalized
Variance, %
29.46
6.33
13.55
6.05
-34.93
-6.46
SwRI Report 03.12024.06
Page 304 of 371
-------
TABLE 114. SUMMARY OF ERROR SURFACE SENSITIVE TO BIAS AND
VARIANCE FOR BSNOX METHOD 3
Sensitivity to Variance
1 95 RefNTE Events
#
1
31
35
Error Surface
SS NOx
Warmup Torque
Eng Manuf Torque
Avg
Contribution
to
Normalized
Variance, %
30.59
-38.37
-17.89
Sensitivity to Bias and Variance
13 RefNTE Events
#
1
31
35
51
Error Surface
SS NOx
Warmup Torque
Eng Manuf Torque
NOx Time Align
SS CO2 OnOff
Avg
Contribution
to
Normalized
Variance, %
23.62
-27.83
-11.69
5.96
-25.22
TABLE 115. SUMMARY OF ERROR SURFACE SENSITIVE TO BIAS AND
VARIANCE FOR BSNMHC METHOD 1
Sensitivity to Variance
195 RefNTE Events
#
13
19
Error Surface
SS NMHC
NMHC Ambient
Avg
Contribution
to
Normalized
Variance, %
6.01
85.70
Sensitivity to Bias and Variance
13 RefNTE Events
#
13
19
Error Surface
SS NMHC
NMHC Ambient
SS NMHC OnOff
Avg
Contribution
to Normalized
Variance, %
7.05
58.33
13.86
TABLE 116. SUMMARY OF ERROR SURFACE SENSITIVE TO BIAS AND
VARIANCE FOR BSNMHC METHOD 2
Sensitivity to Variance
195 RefNTE Events
#
13
19
Error Surface
SS NMHC
NMHC Ambient
Avg
Contribution
to
Normalized
Variance, %
5.65
89.34
Sensitivity to Bias and Variance
13 RefNTE Events
#
13
19
Error Surface
SS NMHC
NMHC Ambient
SS NMHC OnOff
Avg
Contribution
to Normalized
Variance, %
6.78
62.33
13.95
SwRI Report 03.12024.06
Page 305 of 371
-------
TABLE 117. SUMMARY OF ERROR SURFACE SENSITIVE TO BIAS AND
VARIANCE FOR BSNMHC METHOD 3
Sensitivity to Variance
1 95 RefNTE Events
#
13
19
Error Surface
SS NMHC
NMHC Ambient
Avg
Contribution
to
Normalized
Variance, %
5.64
86.98
Sensitivity to Bias and Variance
13 RefNTE Events
#
13
19
Error Surface
SS NMHC
NMHC Ambient
SS NMHC OnOff
Avg
Contribution
to Normalized
Variance, %
6.84
60.51
13.64
TABLE 118. SUMMARY OF ERROR SURFACE SENSITIVE TO BIAS AND
VARIANCE FOR BSCO METHOD 1
Sensitivity to Variance
195 RefNTE Events
#
7
31
Error Surface
SS CO
Warmup Torque
Avg
Contribution
to
Normalized
Variance, %
76.20
-13.29
Sensitivity to Bias and Variance
13 RefNTE Events
#
7
Error Surface
SS CO
SS CO OnOff
Avg
Contribution to
Normalized
Variance, %
6.06
80.63
TABLE 119. SUMMARY OF ERROR SURFACE SENSITIVE TO BIAS AND
VARIANCE FOR BSCO METHOD 2
Sensitivity to Variance
195 RefNTE Events
#
7
42
Error Surface
SS CO
Eng Manuf BSFC
Avg
Contribution
to
Normalized
Variance, %
85.82
9.12
Sensitivity to Bias and Variance
13 RefNTE Events
#
7
Error Surface
SS CO
SS CO OnOff
Avg
Contribution to
Normalized
Variance, %
6.31
82.82
SwRI Report 03.12024.06
Page 306 of 371
-------
TABLE 120. SUMMARY OF ERROR SURFACE SENSITIVE TO BIAS AND
VARIANCE FOR BSCO METHOD 3
Sensitivity to Variance
1 95 RefNTE Events
#
7
31
52
Error Surface
SS CO
Warmup Torque
CO Time Align
Avg
Contribution
to
Normalized
Variance, %
75.49
-13.40
11.29
Sensitivity to Bias and Variance
13 RefNTE Events
#
7
52
Error Surface
SS CO
CO Time Align
SS CO OnOff
Avg
Contribution to
Normalized
Variance, %
6.14
12.10
79.74
6.7 CE-CERT Mobil Emission Laboratory Correlation
As mentioned previously, the primary means of validation for the Monte Carlo Model was
through the use of a comparison set of measurement deltas generated through in-field testing.
This was accomplished using the CE-CERT Mobile Emission Laboratory (MEL), which is
operating by staff from the University of California-Riverside. In order to insure that the
validation was not disturbed by some inherent bias between the SwRI Reference Laboratory and
the CE-CERT MEL Validation Reference, the Test Plan included a correlation exercise that was
to be performed between the two laboratories, prior to the start of on-road validation efforts. The
CE-CERT MEL was brought to SwRI's laboratory facilities in San Antonio, Texas, and a side-
by-side correlation test was run. This correlation testing was performed during June of 2006.
At this point in the program, SwRI had just recently completed dynamometer testing of
Engine 1, which was the DDC Series 60 heavy-heavy duty engine. This engine was still
installed in the test cell at that time, and therefore it was the engine that was used for the
correlation exercise.
The trailer housing the CE-CERT MEL was positioned directly behind the SwRI
laboratory facilities, in such a manner that the test cell could be readily accessed via a high-bay
access door. This position allowed for the easy connection of the MEL dilution tunnel exhaust
inlet to the test cell exhaust system. The exhaust system was constructed in such a manner that it
could be easily disconnected at a downstream of the DPFs, and an exhaust pipe extension was
fabricated and positioned in order to allow for relatively quick connect to the CE-CERT exhaust
inlet.
This arrangement allowed for easy switching of the exhaust between the two test facilities,
allowing for tests to be conducted on both facilities during a single day with minimum
interruption. The Test Plan did not initially include a test matrix for the correlation, and
therefore a correlation test matrix was developed by SwRI and proposed to the Steering
Committee for approval. Following approval, the correlation testing was conducted essentially
as proposed over the course of three days of testing.
SwRI Report 03.12024.06
Page 307 of 371
-------
The primary test cycle used during the correlation test was termed the "NTE Cycle." This
cycle was in fact one of the 20 NTE transient cycles which had been previously used to generate
data for the Transient Error Surface, which was described earlier in Section 4.5 of the report. It
was felt that a cycle which included a number of NTE events would be appropriate for basis for
correlation, because the CE-CERT MEL would later be used to generate validation data during
NTE operation. However, the basis for comparison between the labs was the overall cycle
average brake-specific emissions. The correlation was not assessed on an event-by-event basis,
although this data was available for comparison. This NTE cycle was run in triplicate each day
by both labs in succession.
In addition to the NTE cycle, the test matrix also included duplicate runs of the RMC 13-
mode SET test by each lab every day. This cycle was included in the test matrix in order to
provide some steady-state measurements that would hopefully aid in determining the cause of
any discrepancies between the two laboratories, if any such differences were found during NTE
cycle testing.
The test matrix is illustrated in Table 121. In total, nine NTE cycles were run for each
laboratory and six RMC cycles were run for each laboratory. On each day, one laboratory would
run in the morning, and then the exhaust would be switched for the afternoon runs on the other
laboratory.
TABLE 121. CORRELATION TEST MATRIX
Test Day
1
2
O
Test Laboratory
SwRI
CE-CERT
CE-CERT
SwRI
SwRI
CE-CERT
Test Cycles
3 x NTE Cycle
2 x RMC Cycle
3 x NTE Cycle
2 x RMC Cycle
3 x NTE Cycle
2 x RMC Cycle
3 x NTE Cycle
2 x RMC Cycle
3 x NTE Cycle
2 x RMC Cycle
3 x NTE Cycle
2 x RMC Cycle
Prior to the start of the correlation exercise, any periodic QA checks which were due
according to the schedule outlined in 40 CFR Part 1065 Subpart D were conducted. In addition,
CVS propane recovery checks were performed by both laboratories prior to the start of testing.
The start of testing was delayed several days be an electronic hardware failure with the MEL
CVS, but this was repaired once a replacement part was procured, and further test operations
proceeded without any major incident. The MEL CVS propane recovery check was repeated
following this repair, to insure that flow measurements were still correct.
SwRI Report 03.12024.06
Page 308 of 371
-------
As an additional QA measure, span bottles from each facility were read using the
instruments from the other facility. In all cases, span concentrations were within 1 percent of
expected values.
The brake-specific emission results from the correlation testing for all three days are
summarized in Table 122 for the NTE Transient Cycle and in Table 123 for the RMC cycle. The
results of greatest interest to the Steering Committee were the NOX results. On the NTE
Transient Cycle, the test labs showed a difference of 2.1 percent, with CE-CERT being higher
than SwRI. This difference was matched closely by a 2.7 percent difference in CC>2
measurements in the same direction. The close match between these numbers may have
indicated that the primary discrepancy between the two labs was related to measurement of total
CVS flow.
SwRI Report 03.12024.06 Page 309 of 371
-------
TABLE 122. CORRELATION TESTING RESULTS FOR NTE TRANSIENT CYCLE
Test Test
Day Date
1 6/29/2006
6/29/2006
6/29/2006
6/29/2006
6/29/2006
6/29/2006
2 6/30/2006
6/30/2006
6/30/2006
6/30/2006
6/30/2006
6/30/2006
3 7/5/2006
7/5/2006
7/5/2006
7/5/2006
7/5/2006
7/5/2006
Test
Number
SwRI-NTE-1
SwRI-NTE-2
SwRI-NTE-3
CE-CERT-NTE-1
CE-CERT-NTE-2
CE-CERT-NTE-3
Day 1 Difference
SwRI-NTE-1
SwRI-NTE-2
SwRI-NTE-3
CE-CERT-NTE-1
CE-CERT-NTE-2
CE-CERT-NTE-3
Day 2 Difference
SwRI-NTE-1
SwRI-NTE-2
SwRI-NTE-3
CE-CERT-NTE-1
CE-CERT-NTE-2
CE-CERT-NTE-3
Day 3 Difference
Standard for 2005 Series 60 Engine
Transient Emissions, g/hp-hr
THC
0.003
0.003
0.004
0.003
0.001
0.001
0.001
0.001
-287.6%
0.004
0.003
0.003
0.003
0.002
0.001
0.001
0.001
-148.3%
0.005
0.003
0.003
0.004
0.001
0.001
0.002
0.001
-159.4%
0.14
CH4
-0.005
0.001
0.003
0.000
0.001
0.002
0.002
0.002
119.7%
0.001
0.002
0.002
0.002
0.001
0.002
0.002
0.002
8.2%
-0.007
0.002
0.002
-0.001
0.002
0.002
0.002
0.002
152.1%
0.14
Overall Results - NTE
SwRI
CE-CERT
Mean
Stdev
CvaMSD
Mean
Stdev
CvaMSD
%point
%standard
0.004
0.001
0.001
0.0003
-65.1%
-1.6%
0.0001
0.0036
0.002
0.0003
2336.2%
1.3%
| NMHC |
0.008
0.002
0.001
0.004
-0.001
-0.001
-0.001
-0.001
546.0%
0.003
0.001
0.001
0.002
0.000
-0.001
-0.001
0.000
556.3%
0.012
0.001
0.001
0.005
-0.001
-0.001
0.000
-0.001
960.2%
0.14
Cycle
0.004
0.004
108.9%
-0.001
0.0004
59.0%
1-117.4%
-2.9%
CO
0.057
0.057
0.057
0.057
0.044
0.044
0.042
0.043
-31.7%
0.058
0.054
0.057
0.056
0.041
0.040
0.041
0.041
-38.2%
0.055
0.052
0.053
0.053
0.042
0.043
0.042
0.042
-26.2%
15.5
0.056
0.002
3.8%
0.042
0.001
3.3%
-24.2%
-0.1%
NOx
1.99
1.97
1.99
1.98
2.03
2.03
2.04
2.03
2.4%
2.04
2.01
2.02
2.02
2.04
2.05
2.04
2.04
1.0%
2.01
1.99
2.00
2.00
2.06
2.05
2.07
2.06
2.9%
2.2
2.001
0.020
1.0%
2.044
0.014
0.7%
2.1%
1.9%
CO2
540.4
540.9
542.0
541.1
557.6
558.0
557.8
557.8
3.0%
541.5
543.0
542.4
542.3
554.2
551.7
551.1
552.3
1.8%
539.5
540.4
541.2
540.4
558.5
558.0
554.8
557.1
3.0%
541.3
1.1
0.2%
555.7
2.9
0.5%
12.7%
n/a
SwRI Report 03.12024.06
Page 310 of 371
-------
TABLE 123. CORRELATION TEST RESULTS FOR RMC 13-MODE SET CYCLE
Test Test
Day Date
1 6/29/2006
6/29/2006
6/29/2006
6/29/2006
2 6/30/2006
6/30/2006
6/30/2006
6/30/2006
3 7/5/2006
7/5/2006
7/5/2006
7/5/2006
Test
Number
SwRI-RMC-1
SwRI-RMC-2
CE-CERT-RMC-1
CE-CERT-RMC-2
Day 1 Difference
SwRI-RMC-1
SwRI-RMC-2
CE-CERT-RMC-1
CE-CERT-RMC-2
Day 2 Difference
SwRI-RMC-1
SwRI-RMC-2
CE-CERT-RMC-1
CE-CERT-RMC-2
Day 3 Difference
Standard for 2005 Series 60 Engine
Transient Emissions, g/hp-hr
THC |
0.004
0.003
0.004
0.000
0.000
0.000
-109%
0.002
0.002
0.002
0.001
0.000
0.001
-72%
0.002
0.002
0.002
0.000
0.000
0.000
-84%
0.14
Overall Results
SwRI
CE-CERT
Mean
Stdev
CvaMSD
Mean
Stdev
CvaMSD
%point
%standard
0.003
0.001
0.0002
0.0005
-92.6%
-1.8%
CH4
0.000
0.002
0.001
0.001
0.001
0.001
23%
0.000
0.000
0.000
0.002
0.002
0.002
1586%
0.002
0.002
0.002
0.002
0.001
0.001
-35%
0.14
for RMC
0.001
0.001
0.002
0.001
42.9%
0.3%
NMHC |
0.004
0.001
0.003
-0.002
-0.001
-0.002
-160%
0.002
0.002
0.002
-0.001
-0.001
-0.001
-161%
0.000
0.001
0.000
-0.002
0.000
-0.001
-314%
0.14
Cycle
0.002
0.001
80.0%
-0.001
0.001
48.5%
1-171.9%
-2.1%
CO
0.054
0.057
0.055
0.048
0.052
0.050
-9.5%
0.054
0.053
0.053
0.043
0.041
0.042
-21.4%
0.052
0.052
0.052
0.041
0.045
0.043
-17.4%
15.5
0.053
0.002
3.4%
0.045
0.004
9.7%
-16.0%
-0.1%
NOx
1.79
1.80
1.80
1.88
1.88
1.88
4.6%
1.83
1.84
1.84
1.90
1.91
1.90
3.6%
1.84
1.85
1.85
1.92
1.89
1.91
3.2%
2.2
1.827
0.024
1.3%
1.897
0.015
0.8%
3.8%
3.1%
CO2
499.8
499.8
499.8
511.7
510.5
511.1
2.3%
500.6
501.1
500.8
508.1
509.0
508.5
1.5%
498.9
499.0
499.0
514.2
514.6
514.4
3.1%
499.9
0.9
0.2%
511.3
2.7
0.5%
2.3%
The RMC cycle showed a larger NOX difference of 3.8 percent, with CE-CERT again
being higher than SwRI. However, the CC>2 difference for this cycle was similar to that observed
during the NTE cycle at 2.3 percent. In this case it is unlikely that all of the NOX differences are
related to CVS flow discrepancies alone.
Continuous data for the RMC was examined in order to attempt to determine the reasons
for the larger discrepancies observed on that cycle. An example of continuous NOX mass rate
comparison is given in Figure 233. It was noted that NOX mass rates were closer together at
higher levels, associated with high load modes of operation during the RMC. The facilities
diverged more at lower load modes, which had correspondingly lower NOX levels. These
SwRI Report 03.12024.06
Page 311 of 371
-------
differences cannot be related entirely to flow rate measurements, because CC>2 correlated more
closely between the two laboratories than NOX. Examination of linearity data for both NOX
analyzers did not reveal any obvious reasons why the two systems should have diverged more at
lower concentration levels.
^—SwRI NOx Mass, g/seo ^—Ce-Cert NOx Mass, g/seo
^— Absolute Differences, g/seo Percent Difference (Ce-Cert - SwRI)
20%
500
1000 1500
Time, sec
2000
2500
FIGURE 233. COMPARISON OF TYPICAL CONTINUOUS NOX MASS RATE DATA
OVER RMC CYCLE - CE-CERT VERSUS SWRI
One area of difference noted between the two laboratories was in terms of CVS flow rate.
The SwRI test cell ran at an average flow rate of 4350 scfrn, while the CE-CERT tunnel ran at an
average flow rate of 2775 scfm. These flow rates translate to an average dilution ratio during the
RMC of 7.3 versus 4.6 for SwRI and CE-CERT respectively. It is not known if this difference is
related to any of the measurement differences observed between the two laboratories.
The Steering Committee examined these results during the July 7, 2006 conference call.
The primary area of interest was the NOX correlation between the two facilities. The NTE cycle
correlation of 2 percent was generally considered good by the Steering Committee; however,
there was some concern expressed over the larger discrepancy that was observed during the
RMC cycle.
SwRI Report 03.12024.06
Page 312 of 371
-------
The Test Plan did not include any guidance on assessing correlation. The initial proposal
was to use an analysis similar to that given in 40 CFR Part 1065.12 for alternative method
approvals. If this proved unsuccessful, the Steering Committee would assess the absolute
differences observed to determine if an acceptable level of correlation was achieved.
An initial data analysis was performed using the methodology laid out in that part of the
CFR. As outlined in that procedure, both a F-test and a t-test were performed to compare the two
sets of NOX data , using appropriate Fcrit and tcrit values given in the tables in Part 1065, and a 90
% confidence interval. This was done using the NTE cycle data, as that cycle was the primary
cycle of comparison. The result of this analysis indicated a failure of the t-test for NOX ,
meaning that the 2 percent difference in NOX observed was statistically significant. However, it
was noted that this was due in part to the high degree of test-to-test repeatability observed for
both laboratories over the three days of testing.
The Steering Committee examined all of these results along with the F-test and t-test
results. Ultimately, the Steering Committee determined at the July 7, 2006 conference call that
the level of correlation achieved between the two facilities was acceptable for the performance of
the validation testing. This determination was based primarily on the fact that the NTE cycle
was deemed more likely to be the kind of operation observed by CE-CERT during on-road
testing, and the correlation results for that cycle at 2 percent difference between the labs was
considered sufficient to proceed with testing.
It should be noted that this correlation exercise did represent a unique opportunity to
compare two 1065 compliant measurement systems using the same test engine and dynamometer
installation. In this comparison, any correlation issues related to test cell installation or cycle
operation were eliminated by the fact that the same test article was used for both comparisons.
6.8 CE-CERT On Road Validation Testing
The generation of the On-Road Validation data set was performed by CE-CERT using
their MEL. This on-road effort was conducted under a separate contract which was funded by
the California Air Resources Board (ARE), as part of their contribution to the Measurement
Allowance program. The test truck was supplied by Caterpillar, and it incorporated a C-15
heavy heavy duty diesel engine in it. Initially, the truck was configured only with diesel
oxidation catalysts (DOCs), rather than DPFs. A pair of DPFs was supplied for use during
validation testing by International. These DPFs were shipped to CE-CERT, and were installed
on the truck be a local Caterpillar dealer in Riverside, California.
The MEL was used as the reference laboratory measurement during the on-road testing.
The PEMS unit used during the on-road testing was PEMS 5, which had previously been audited
by SwRI, and was used during selected Environmental Chamber testing described earlier in
Section 5 of this report. In addition, one of the 5-inch EFM2 exhaust flow meters was also used
for the on-road testing. This flow meter had been previously audited by SwRI, and was also
used during Environmental Chamber testing. In addition, it had also been used for Engine 1
dynamometer tests at SwRI. The 5-inch flow meter was selected due to the size of the test
engine in the truck.
SwRI Report 03.12024.06 Page 313 of 371
-------
The actual performance and results of the CE-CERT on-road validation testing are not
included in this report. Full details on this part of the program are given in a separate report,
titled "Measurement Allowance On-Road Validation Project Report," dated March 2007. The
contents of that report are incorporated herein by reference.
SwRI had only a very limited role in the performance of the on-road testing itself,
generally limited to arranging for the delivery of the test truck and PEMS hardware both to and
from CE-CERT. Caterpillar representatives were onsite with CE-CERT staff during the replay
testing, but were there only in order to assist in the recording of proprietary engine ECM data
channels, which would later be used to aid in the Laboratory Replay Validation described later in
this report. All validation testing was conducted solely by CE-CERT staff.
Sensors staff was onsite briefly at CE-CERT prior to testing, but only to assist in the
installation of a purge option which was designed to keep the pressure sensing lines of the EFM2
exhaust flow meter free from condensation during on-road testing. The installation of this option
was approved by the Steering Committee prior to the start of on-road testing.
6.9 Laboratory Replay Testing
Because the CE-CERT MEL does not readily incorporate a means of direct torque
measurement on a vehicle, the on-road validation data set could not be used to validate model
errors associated with broadcast torque and derived BSFC. Therefore, an additional validation
exercise was conducted at SwRI. This involved removal of the Caterpillar CIS engine from the
test truck used by CE-CERT, and installation of the engine in the SwRI dynamometer test cell.
The dual DPFs used during the CE-CERT validation were also removed from the truck and
installed in the laboratory exhaust system. The SwRI test cell used for the laboratory replay
validation was the same dynamometer test cell used throughout the program. PEMS 5 with 5-
inch EFM, which was used for the CE-CERT validation exercise, was shipped back to SwRI for
use in the replay validation testing. Selected portions of the CE-CERT on-road tests were then
simulated in the laboratory, to the extent possible. Simultaneous laboratory and PEMS
measurements were again taken during this replay validation exercise. However, because the
laboratory incorporates actual torque measurement, it was possible to use this replay data set to
validate the portions of the model associated with torque and BSFC measurements.
After the replay engine installation was complete, the CAT CIS was power validated and
a lug curve was generated. Next, one hour test segments from the CE-CERT on-road validation
were extracted from the test data to be replayed in the SwRI test cell. During the CE-CERT
model validation exercise, CAN engine data was recorded with proprietary Caterpillar software.
Using the CAT engine data as well as the PEMS NTE event data, one 1-hour test segment was
extracted from each of the 3 routes tested during the CE-CERT validation exercise. The 1-hour
test segments were chosen to include a large number of NTE events while also offering diverse
engine operation. Each 1-hour cycle was repeated a minimum of 3 times.
The Caterpillar CAN engine data was used to insure the laboratory testing reproduced the
engine operation experience during the CE-CERT testing. With assistance from Caterpillar, the
SwRI Report 03.12024.06 Page 314 of 371
-------
CAN engine data recorded during SwRI laboratory testing was compared to the CAN data
recorded during the CE-CERT on-road validation. Upon initial replay attempts, it was apparent
the engine was producing significantly more boost at SwRI than it did during the on-road testing.
After reviewing the engine data, Caterpillar concluded that the truck likely had a boost leak
during the CE-CERT testing, causing the unexpectedly low boost pressure. To accurately
simulate the CE-CERT replay, SwRI implemented an adjustable boost leak in the intercooler
system. With the boost pressure similar between SwRI and CE-CERT testing, the correlation of
most of the engine parameters improved. After several iterations of testing at SwRI and
subsequent recommendations by Caterpillar, the accuracy of the engine replay was deemed
acceptable by Caterpillar and SwRI.
The first replay test cycle was taken from Route 1, which was the San Diego trip.
Because the average elevation during Route 1 nearly matched the elevation of San Antonio, no
altitude simulation was necessary. Cycle 2 was taken from Route 2, which was the trip to
Mammoth Mountain. The test cell simulated the average barometric pressure calculated for the
hour test segment used for the replay. Cycle 3 was extracted from Route 3, which was the return
trip from Mammoth Mountain back to Riverside. Similar to cycle 2, the test cell simulated the
average elevation calculated for the hour-long test segment of Route 3. For cycle 2 the elevation
simulation was 4500 feet, while for cycle 3 the average elevation was 3500 feet.
Figure 234 shows the average NTE event engine speed recorded with the Caterpillar
software for cycle 2 in the laboratory and for the CE-CERT on-road validation test. The SwRI
and CE-CERT speed traces showed excellent correlation for all replay tests.
2200
2000
„ 1800
Q.
.
-------
The SwRI and CE-CERT average NTE CAN fuel rate is shown in Figure 235 for cycle 2.
Cycle 2 and 3 showed tight fuel rate correlation. Cycle 1 showed good agreement with the
exception of a couple NTE events where the delta between SwRI and CE-CERT CAN fuel rate
data was near 3 gal/h.
30.0
25.0 -
20.0 -
• SwRI Fuel Rate gal/h BCeCert Fuel Rate gal/h
10.0 -
5.0 -
0.0
10 12
NTE Event Number
14
16
18
20
FIGURE 235. FUEL RATE CAN DATA COMPARISON FOR SWRI LABORATORY
TESTING AND CE-CERT ON-ROAD VALIDATION TESTING (ROUTE 2)
Figure 236 shows the boost pressure recorded during the SwRI replay and the CE-CERT
on-road test. With the laboratory intercooler system, it was impossible to match the boost
pressure for each NTE event. Therefore, SwRI attempted to center the NTE errors so that the
test average error was near zero.
SwRI Report 03.12024.06
Page 316 of 371
-------
200
180 -
140 -
120 -
100
80
• SwRI Boost P kPa • CeCert Boost P kPa
10 12
NTE Event Number
14
16
18
20
FIGURE 236. BOOST PRESSURE CAN DATA COMPARISON FOR SWRI
LABORATORY TESTING AND CE-CERT ON-ROAD VALIDATION TESTING
(ROUTE 2)
The timing discrepancies between the SwRI and CE-CERT testing are shown in Figure
237 for cycle 2. Nearly all timing differences for all cycles were within 1 degree of engine
rotation.
SwRI Report 03.12024.06
Page 317 of 371
-------
m
ra
0)
T3
• SwRI Timing deg BTDC BCeCert Timing deg BTDC
• : "*" : • I
_t_. __•__ .A.
2 degrees
V 1
1
10 12
NTE Event Number
14
16
18
20
FIGURE 237. INJECTION TIMING CAN DATA COMPARISON FOR SWRI
LABORATORY TESTING AND CE-CERT ON-ROAD VALIDATION TESTING
(ROUTE 2)
Figure 238 shows the intake manifold temperature comparison between SwRI and CE-
CERT. Similar to the boost pressure, an exact match between SwRI and CE-CERT could not be
achieved for all NTE events. The intake manifold temperature was set to achieve a balance
between positive and negative NTE event differences. The exhaust backpressure was likely
similar between the SwRI laboratory and CE-CERT tests because identical DPFs were used
during both validation efforts. The intake restriction was set per the manufacturers' specification
during laboratory testing. Different fuels were used during testing at SwRI and during CE-
CERT's on-road testing. Also, no effort was taken to reproduce the intake air humidity levels
observed during the CE-CERT testing.
SwRI Report 03.12024.06
Page 318 of 371
-------
70
60 -
50 -
O 40 -
ro
0>
2.
S 30
20 -
• SwRI IMT deg C BCeCert IMT deg C
I * *
. . •- •:y• --v- ---:•
10 --
10 12
NTE Event Number
14
16
18
20
FIGURE 238. INTAKE MANIFOLD TEMPERATURE CAN DATA COMPARISON FOR
SWRI LABORATORY TESTING AND CE-CERT ON-ROAD VALIDATION TESTING
(ROUTE 2)
Figure 239 shows the average NTE wet NOX concentrations measured using PEMS 5 in
the SwRI Laboratory and during the CE-CERT testing. PEMS 5 measured slightly lower wet
NOX concentrations at SwRI compared to during the CE-CERT validation. A possible
explanation for the difference in wet NOX concentration may have been a difference in intake air
humidity, as no effort was taken to match this parameter. Cycle 1 and 3 wet NOX concentration
correlations were tighter than those measured for cycle 2. Cycle 1 wet NOX deltas were centered,
with some positive and negative differences. Cycle 3 SwRI NOX deltas were biased slightly
negative.
SwRI Report 03.12024.06
Page 319 of 371
-------
700
600
500
Q.
Q.
o
•s
400
300
iu
0.
200 -
100
• SwRI NTE Ave. Wet NOx ppm BCeCert NTE Ave. Wet NOx ppm
10 12
NTE Event Number
14
16
18
20
FIGURE 239. PEMS 5 WET NOX COMPARISON FOR SWRI LABORATORY
TESTING AND CE-CERT ON-ROAD VALIDATION TESTING (ROUTE 2)
Figure 240 and Figure 241 show the PEMS 5 exhaust flow rate and wet NOX mass rate
for SwRI and CE-CERT validation tests. The exhaust flow rates showed good correlation for the
three cycles. Therefore, the wet NOX mass flow rates showed trends similar to wet NOX
concentration.
SwRI Report 03.12024.06
Page 320 of 371
-------
1400
• SwRI NTE Ave. Exh. Flow Rate scfm BCeCert NTE Ave. Exh. Flow Rate scfm
1200
1000 _
* m i *
800 f - - -J -
£ 600 - - -
LU
I 400 f
200
0
0 2 4 6 8 10 12 14 16 18 20
NTE Event Number
FIGURE 240. PEMS 5 EXHAUST FLOW RATE COMPARISON FOR SWRI
LABORATORY TESTING AND CE-CERT ON-ROAD VALIDATION TESTING
(ROUTE 2)
0.50
• SwRI NTE Ave. NOx Mass Rate g/s BCeCert NTE Ave. NOx Mass Rate g/s
0.45
0.40 -
035 fj . * * * * *---•-
• *
0.30 -
0.25 -
0.20 J- - - *-
0.15 -
LU
0.
0.10
0.05 -
0.00
8 10 12
NTE Event Number
14
16
18
20
FIGURE 241. PEMS 5 NOX MASS FLOW RATE COMPARISON FOR SWRI
LABORATORY TESTING AND CE-CERT ON-ROAD VALIDATION TESTING
(ROUTE 2)
SwRI Report 03.12024.06
Page 321 of 371
-------
Brake-specific emission results for each NTE event were calculated with data from
PEMS 5 using the three different calculation methods for each cycle repeat. The PEMS relied on
ECM broadcast parameters to calculate the work term of each calculation method. Based on
Caterpillar's recommendation, the percent torque, percent friction torque, and reference torque
J1939 CAN broadcast channels were used to estimate torque. ECM broadcast fuel rate was used
with the ECM torque estimation to calculate BSFC. The PEMS brake-specific emission results
were compared to the laboratory dilute reference brake-specific emission results. The laboratory
reference calculation method used torque from the inline torque meter to determine work.
The NOX emission deltas for PEMS 5 using calculation Method 1 are shown in Figure
242. Cycle 1 deltas were relatively small, with mostNOx deltas between -0.05 and 0.1 g/(hp-hr).
Cycle 2 and 3 deltas, however, were unexpected large, with deltas ranging from 0.0 to 0.6
g/(hp-hr) for NOX. To insure the large deltas were not caused by a problem with PEMS 5, cycle
1 and 2 data was compared for PEMS 4. The PEMS 4 and 5 delta results were similar,
indicating PEMS 5 was functioning properly. There was also concern that the altitude
simulation was causing PEMS measurement errors or errors with the ECM broadcast torque and
fuel rate parameters. Due to the experimental setup, the PEMS sample line and EFM were the
only components of the measurement system subjected to the reduced pressure during the cycle 2
and 3 altitude simulation. With all other PEMS ports referenced to ambient pressure, the PEMS
was near the fault limit for sample vacuum. To insure the altitude simulation was not the cause
of the increased brake-specific deltas, cycle 3 was repeated with no altitude simulation, as was
done with cycle 1. PEMS 5 cycle 3 deltas were similar with and without altitude simulation,
indicating the altitude simulation did not affect the PEMS measurements or the ECM broadcast
torque information. Similarly, the PEMS concentration data, as compared to the laboratory raw
concentration measurements, showed no susceptibility to the reduced pressure at the sample line
inlet.
SwRI Report 03.12024.06 Page 322 of 371
-------
IP5 Cycle 1 » P5 Cycle 2 A P4 Cycle 1 P4 Cycle 2 X P5 Cycle 3 • P5 Cycle 3 No Alt.
PEMS 5 Method 1 NOx - Lab Reference NOx
-0.2
Lab Reference NOx [g/hp-hr]
FIGURE 242. BRAKE-SPECIFIC NOX EMISSION DELTAS FOR PEMS 5 METHOD 1
CALCULATION VERSUS THE LABORATORY REFERENCE (LABORATORY
TORQUE AND BSFC)
With the cause of the substantial brake-specific deltas unknown, SwRI performed another
comparison between the PEMS and the laboratory NTE emission results. During the second
comparison, the laboratory used the ECM broadcast parameters used by the PEMS to perform
calculation by Method 1, 2, and 3. With similar torque, fuel rate, and BSFC values, as well as
similar calculation techniques, this second computational exercise was a comparison of emission
mass. The second SwRI correlation was similar to the comparison process used during the CE-
CERT on-road validation testing. The reprocessed Method 1 brake-specific deltas for the PEMS
versus the laboratory are shown in Figure 234. The large deltas observed during the PEMS
comparison to the laboratory reference are significantly diminished when similar torque and
BSFC terms are used for the PEMS and laboratory.
During altitude simulation, the PEMS EFM was subjected to pressures lower than
ambient pressure. According to Sensors Inc., the EFM static pressure measurement system was
not designed to operate with the EFM below the ambient pressure recorded by the barometric
pressure sensor in the PEMS. Therefore, operation with the EFM at reduced pressure and the
PEMS at ambient pressure caused inaccurate flow measurement. Cycle 3, at 3500 feet of
elevation simulation, showed similar brake-specific results with and without altitude simulation.
Therefore, the EFM was not likely affected by the 3500 elevation simulation. However, cycle 2,
at 4500 feet of elevation simulation, showed brake-specific deltas nearly double those measured
during cycle 2 testing. The EFM static pressure measurement problem may have caused the
increased deltas observed for cycle 2.
SwRI Report 03.12024.06
Page 323 of 371
-------
0.7
0.6
0.5
I 0.4
I)
| 0.3
"oi
Q
O 0.2
z
10
tn
0.1 -
LU
Q.
0.0
2
-0.1 -
-0.2
I P5 Cycle 1 » P5 Cycle 2 A P5 Cycle 3 P5 Cycle 3 NO Alt.
PEMS 5 Method 1 NOx - Lab Method 1 NOx
2.2
2.4
2.6 2.8
3.0
3.2
3.4 3.6
38
Lab Method 1 NOx [g/hp-hr]
FIGURE 243. BRAKE-SPECIFIC NOX EMISSION DELTAS FOR PEMS 5 METHOD 1
CALCULATION VERSUS THE LABORATORY METHOD 1 CALCULATION (ECM
TORQUE AND BSFC)
The purpose of the replay testing was to compare the ECM torque and BSFC errors
measured with the Caterpillar CIS engine to the incremental torque and BSFC errors predicted
by the Model. As mentioned earlier, two deltas had been generated for each calculation method
in comparing the PEMS brake-specific values to the Laboratory values. The "full" deltas, shown
in Figure 242, were calculated using the lab measured torque as the basis for the work term. The
"mass" deltas, shown in Figure 243, were calculated using ECM torque, fuel rate, and BSFC for
the laboratory as well as the PEMS. For each calculation method, an incremental "work" delta
was generated by comparing the associated "full" delta to the appropriate "mass" delta. As
shown in the equation below, this calculation was performed individually for each replay NTE
event.
A
replay,Work
= A
replay,Full
-A
replay,Mass
The Caterpillar CIS Method 1 work deltas generated using the equation above are shown
in Figure 244. The work deltas are cycle dependant and showed errors as large as those
calculated for the full replay. Therefore, the ECM torque and fuel rate errors were a key cause of
the brake-specific deltas observed in Figure 242.
SwRI Report 03.12024.06
Page 324 of 371
-------
I Cycle 1 * Cycle 2 A Cycle 3
Lab Reference NOx [g/hp-hr]
FIGURE 244. INCREMENTAL BRAKE-SPECIFIC NOX EMISSION DELTAS DUE TO
ECM TORQUE AND BSFC ERRORS FOR CALCULATION METHOD 1
The incremental torque and BSFC deltas predicted by the Model were determined by
comparing the results of the Full Model to the results of the Validation Model. The distinction
between these two sets of model results was described earlier in Section 2.1.11. The Full Model
results were calculated with all error surfaces active, including the torque and BSFC error
surfaces. The Validation Model deltas were calculated with error surfaces related to torque and
BSFC inactive. The difference between the Full Model deltas and the Validation Model deltas
yield deltas solely due to torque and BSFC error terms in the Model. The calculation shown
below was performed on an event-by-event basis for all reference NTE events, generating 195
Model predicted ECM torque and BSFC deltas.
A
Mode I, Work
= A
Mode I, Full
-A
Model, Validation
The comparison between the ECM torque and BSFC errors measured with the Caterpillar
CIS engine to the incremental torque and BSFC errors predicted by the Model are shown in
Figure 245 for calculation method number 1. Ideally, the replay torque and BSFC errors would
have been less than the incremental torque and BSFC deltas predicted by the model. However, it
should be noted that some of the replay error data exceeded the predicted values, with some of
the replay data work deltas being several times larger than the work deltas predicted by the
Model using the calculation given above. As described above, the predicted "work deltas" are
the difference between the Full Model and the Validation Model for a given NTE event.
SwRI Report 03.12024.06
Page 325 of 371
-------
I
5
x
O
Q
1.00
0.80 -
0.60 -
0.40 -
0.20 -
0.00
-0.20
NOx g/kW-hr Method 1
Model with Time Alignment Adjustment
BS Deltas Due to ECM Torque and Fuel Rate Errors
Ideal or Lab Reference NOx g/kW-hr
I 95th Full - 95th Valid • Replay
FIGURE 245. INCREMENTAL BRAKE-SPECIFIC NOX DELTAS COMPARED TO
THE INCREMENTAL TORQUE AND BSFC ERRORS PREDICTED BY THE MODEL
FOR CALCULATION METHOD 1
Figure 246 shows the brake-specific NOX emission results for PEMS 5 Method 1
calculations versus the laboratory reference emission results for each replay NTE event. The
laboratory reference results were calculated using torque measured with the inline torque meter.
Similar to the Method 1 calculations, Method 1 calculations with cycle 1 resulted in relatively
small deltas, while cycle 2 and 3 had substantially larger brake-specific emission errors.
SwRI Report 03.12024.06
Page 326 of 371
-------
Cycle 1 * Cycle 2 A Cycle 3 Cycle 3 No Alt.
PEMS 5 Method 2 NOx - Lab Reference NOx
-0.2
Lab Reference NOx [g/hp-hr]
FIGURE 246. BRAKE-SPECIFIC NOX EMISSION DELTAS FOR PEMS 5 METHOD 2
CALCULATION VERSUS THE LABORATORY REFERENCE (LABORATORY
TORQUE AND BSFC)
As described previously for calculation Method 1, Method 1 brake-specific deltas were
also calculated with the laboratory using ECM torque and fuel rate as well as the Method 1
emission calculation procedure. Using similar torque, fuel rate, and BSFC terms, PEMS 5 brake-
specific NOX results were similar to the laboratory dilute results. Essentially a comparison of
NTE mass, the results indicate the large brake-specific errors observed in Figure 246 were due to
torque and BSFC errors.
SwRI Report 03.12024.06
Page 327 of 371
-------
Cycle 1 » Cycle 2 A Cycle 3 Cycle 3 No Alt
PEMS 5 Method 2 NOx - Lab Method 2 NOx
-0.1 --
-0.2
Lab Method 2 NOx [g/hp-hr]
FIGURE 247. BRAKE-SPECIFIC NOX EMISSION DELTAS FOR PEMS 5 METHOD 2
CALCULATION VERSUS THE LABORATORY METHOD 2 CALCULATION (ECM
TORQUE AND BSFC)
The Method 1 incremental torque and BSFC errors measured during replay testing were
calculated by subtracting the full deltas, shown in Figure 246, from the mass deltas shown in
Figure 247. Shown in Figure 248, Method 1 torque and BSFC errors accounted for the majority
of the error observed when the PEMS brake-specific NOX results were compared to the
laboratory reference results.
SwRI Report 03.12024.06
Page 328 of 371
-------
I Cycle 1 * Cycle 2 A Cycle 3
0.8 i
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
1
-0.1
-0.2
8
Calculation Method 2 Incremental Deltas Due to Torque and BSFC Errors
2.0
2.2
2.4
Lab Reference NOx [g/hp-hr]
FIGURE 248. INCREMENTAL BRAKE-SPECIFIC NOX EMISSION DELTAS DUE TO
ECM TORQUE AND BSFC ERRORS FOR CALCULATION METHOD 2
Similar to calculation Method 1, the torque and fuel rate errors measured with PEMS 5
using calculation Method 1 were compared to the incremental torque and BSFC errors predicted
by the Model. Shown in Figure 249, the Method 1 torque and fuel rate errors measured during
replay testing with the Caterpillar CIS engine were considerably larger than the Model
prediction.
SwRI Report 03.12024.06
Page 329 of 371
-------
NOx g/kW-hr Method 2
Model with Time Alignment Adjustment
BS Deltas Due to ECM Torque and Fuel Rate Errors
O)
X
O
-------
Cycle 1 * Cycle 2 A Cycle 3 Cycle 3 No Alt.
PEMS 5 Method 3 NOx - Lab Reference NOx
-0.2
Lab Reference NOx [g/hp-hr]
FIGURE 250. BRAKE-SPECIFIC NOX EMISSION DELTAS FOR PEMS 5 METHOD 3
CALCULATION VERSUS THE LABORATORY REFERENCE (LABORATORY
TORQUE AND BSFC)
Figure 251 shows the brake-specific PEMS 5 deltas when the laboratory used the ECM
torque, fuel rate, and BSFC values to calculation brake-specific emissions. When torque and
BSFC errors were removed from the PEMS and laboratory comparison, thus yielding a mass
comparison, the deltas for the three cycles collapsed, with most errors between -0.1 and 0.1
g/(hp-hr). Therefore, ECM torque and fuel rate errors were the key factors driving the large
cycle 2 and 3 deltas shown in Figure 250.
SwRI Report 03.12024.06
Page 331 of 371
-------
0.7
0.6
0.5 -
I 0.4-
.c
| 0.3--
"oi
Q
O 0.2 --
z
10
(/)
0.1 --
LU
Q.
0.0
-0.1 --
-0.2
I Cycle 1 » Cycle 2 A Cycle 3 Cycle 3 No Alt.
PEMS 5 Method 3 NOx - Lab Method 3 NOx
2.2
3.4
3.6
38
Lab Method 3 NOx [g/hp-hr]
FIGURE 251. BRAKE-SPECIFIC NOX EMISSION DELTAS FOR PEMS 5 METHOD 3
CALCULATION VERSUS THE LABORATORY METHOD 3 CALCULATION (ECM
TORQUE AND BSFC)
Figure 252 shows the incremental ECM torque and BSFC errors measured during replay
testing. These deltas were determined by taking the difference between the PEMS errors
calculated against the laboratory reference results and the PEMS errors calculated against the
laboratory results using ECM torque and BSFC. Similar to the other calculation methods,
Method 1 ECM torque and fuel rate errors were large in comparison to the mass emission deltas
shown in Figure 251.
SwRI Report 03.12024.06
Page 332 of 371
-------
0.8
0.7
0.6
0.5
0.3
0.2
0.1
0.0
-0.1
-0.2
I Cycle 1 * Cycle 2 A Cycle 3
Calculation Method 3 Incremental Deltas Due to Torque and BSFC Errors
A
A
-t/_.
** -
•
-«-
V
2.0
2.2
2.4
2.6
2.
3.0
3.2
3.4
36
Lab Reference NOx [g/hp-hr]
FIGURE 252. INCREMENTAL BRAKE-SPECIFIC NOX EMISSION DELTAS DUE TO
ECM TORQUE AND BSFC ERRORS FOR CALCULATION METHOD 3
Similar to calculation Method 1 and 2, the torque and fuel rate errors measured with
PEMS 5 using calculation Method 1 were compared to the incremental Method 1 torque and
BSFC errors predicted by the Model. Shown in Figure 253, the Method 1 torque and fuel rate
errors measured during replay testing with the Caterpillar CIS engine were considerably larger
than the Model prediction.
SwRI Report 03.12024.06
Page 333 of 371
-------
NOx g/kW-hr Method 3
Model with Time Alignment Adjustment
BS Deltas Due to ECM Torque and Fuel Rate Errors
O)
X
-------
I Cycle 1 *Cycle 2 ACycle 3 Cycle 3 No Alt. X Cycle 3 No Leak •Cycle 3 SwRI Lug Curve Cycle 3 Cert Lug Curve
10%
5%
o
Q.
5 "«
3
O"
-5% H
-10%
<
O
-15%
-20%
• -.' v-r •**
0 _ 7|j) 800 _• 900 ^ 10000 1100 1200 tfOft 0 A^OO 1500
^
1600
A
X
x A
X
x
Lab Reference Average NTE Torque [Ibf-ft]
FIGURE 254. ECM BROADCAST TORQUE ERRORS MEASURED DURING REPLAY
VALIDATION TESTING WITH A CATERPILLAR CIS ENGINE
NTE event average deltas between the ECM fuel rate and the laboratory dilute carbon
balance fuel rate are shown in Figure 255 for the 3 cycles. Most fuel rate errors were between 0
and 6 % of point. The dilute carbon balance error versus the laboratory fuel flow meter was
approximately -1.0 to -1.5 %, therefore, the deltas shown in Figure 255 may be approximately
1.0% high.
SwRI Report 03.12024.06
Page 335 of 371
-------
14%
12%
o
Q.
« 8%
ra
"oi
Q
1 6%
<
O
4% -
2%
0%
Cycle 1 • Cycle 2 A Cycle 3
(CAN Fuel Rate - Lab CB Fuel Rate)/Lab CB Fuel Rate
-«
ft
10
12 14 16 18 20
NTE Event Average Lab Dilute Carbon Balance Fuel Flow [g/sec]
22
24
FIGURE 255. ECM BROADCAST FUEL FLOW RATE ERRORS MEASURED DURING
REPLAY VALIDATION TESTING WITH A CATERPILLAR CIS ENGINE
The BSFC values used for the PEMS brake-specific emission calculations were
calculated using the ECM broadcast torque and fuel rate parameters. The ECM-based BSFC
values were compared to the laboratory BSFC results which were calculated using the dilute
carbon balance fuel flow and the inline torque meter. The deltas between the ECM and
laboratory BSFC are shown in Figure 256. For cycle 1, BSFC deltas were between 0 and 5 % of
point. Cycle 2 and 3, however, had significantly higher BSFC deltas, with peak values near 20
SwRI Report 03.12024.06
Page 336 of 371
-------
25%
20%
15%
HI
Q
O
> 10%
m
I 5%
o
0%
-5%
Cycle 1 » Cycle 2 A Cycle 3 (CAN-Based BSFC - Lab BSFC)/Lab BSFC
A
A •
AA
A A
210
220
.230
240
250
260
NTE Event Average Lab BSFC [g/kw-hr]
FIGURE 256. ECM-BASED BSFC ERRORS MEASURED DURING REPLAY
VALIDATION TESTING WITH A CATERPILLAR CIS ENGINE
After reviewing the ECM torque and BSFC errors measured with the Caterpillar CIS
replay engine, the Steering Committee instructed SwRI to calculate ECM torque errors for the
other test engines used during the program. Using the 40-point engine map data, the ECM
broadcast torque estimates were compared to the laboratory inline torque meter. Figure 257
shows the ECM torque deltas for the Detroit Diesel Series 60 engine. The ECM torque errors
were calculated using two methods. One method used the percent torque, percent friction torque,
and reference torque J1939 CAN broadcast channels, while the other method used the percent
load at current speed CAN broadcast channel with the engine's certification lug curve. For the
DDC engine, the ECM broadcast torque errors were within ±10 % of point. Using the DDC
certification lug curve, the high level torque errors were similar to the ECM J1939 broadcast
torque errors. However, the lug curve method torque errors increased as the steady-state torque
level decreased.
According to the J1939 protocol, the percent load at current speed J1939 parameter
represents indicated engine torque, not brake torque. Therefore, the lug curve method most
likely neglected friction torque, and overestimated brake torque. At high loads, the friction
torque error is a small percentage of the total torque, therefore, the difference between indicated
torque and brake torque is inconsequential. However, at lighter loads, the friction torque errors
become significant, and most likely caused the large lug curve torque deltas shown in Figure
257. The SEMTECH-DS software allows the user to input a curb idle % load value that is used
to calculate brake torque from indicated torque. The equation was developed at the University of
West Virginia and is shown below. This calculation assumes friction torque is constant during
all engine operation.
SwRI Report 03.12024.06
Page 337 of 371
-------
% Load,-
ECM _ % _ Loadlndicated - % _ Loadpnction
100%-% Load
Friction
SwRI did not use the curb idle % load correction when processing the lug curve error
data.
LU
HI
3
O"
O
40
30
20
10
0
-10
-20
-30
-40
-50
-60
I •
.1
• DDC ECM Error [% of Point]
• DDC Lug Curve Error [% of Point]
I I
200
440
600
800
1000
1200
1400
1600
1800
40-Point Map Points 1 through 4
(High speed governer points)
(See DDC Map)
Laboratory Reference Torque [Ibf-ft]
FIGURE 257. ECM BROADCAST TORQUE ERRORS MEASURED DURING 40-
POINT MAP GENERATION WITH A DETROIT DIESEL SERIES 60 ENGINE
Figure 258 shows the ECM torque errors for the Caterpillar C9 test engine. The ECM
broadcast torque errors were similar to the errors generated using the certification lug curve
torque estimation technique. Similar to the DDC engine, torque errors were minimal at high load
points, but increased as the load level deceased. Engine 2 had torque errors near 50 % of point at
low load points.
SwRI Report 03.12024.06
Page 338 of 371
-------
o
Q.
HI
0"
o
LU
-100
»CAT ECM Error [% of Point]
• CAT Lug Curve Error [% of Point]
800
1000
1200
40-Point Map Points 1 through 4
(High speed governer points)
-80 --
Laboratory Reference Torque [Ibf-ft]
FIGURE 258. ECM BROADCAST TORQUE ERRORS MEASURED DURING 40-
POINT MAP GENERATION WITH A CATERPILLAR C9 ENGINE
Figure 259 shows the ECM torque errors for the International VT365 testing engine. The
International engine did not broadcast J1939 torque parameters, therefore, J1708 percent load at
current speed was used with a certification lug curve to calculate torque. At high torque levels,
the ECM torque error was almost 8 % low. Similar to engines 1 and 2, engine 3 torque errors
increased as the load level decreased. At the lowest load levels, the torque error was
approximately 20 %. For all lug curve torque analysis, the curb idle % load correction was not
used.
SwRI Report 03.12024.06
Page 339 of 371
-------
25
20
15
S. 10
"n" ^
o
w
d) 0
O"
-5
o
LU
-10
-15
-20
»INT Lug Curve Error [% of Point]
. i
100
200 0 300'
400
500
* *
40-Point Map Points 1 through 4
(High speed governer points)
Laboratory Reference Torque [Ibf-ft]
FIGURE 259. ECM BROADCAST TORQUE ERRORS MEASURED DURING 40-
POINT MAP GENERATION WITH AN INTERNATIONAL VT365 ENGINE
The ECM torque and BSFC errors measured during replay testing were substantially
larger than the incremental torque and BSFC errors predicted by the model, indicating the torque
and BSFC portion of the Model did not validate. One potential cause for the Model invalidation
was the that the deltas of the manufacturer supplied error surfaces for torque and BSFC were far
less than the ECM torque and BSFC deltas measured with the replay engine. In fact, the torque
deltas for all of the engine exceeded the OEM supplied torque and BSFC errors. A complete
discussion of the manufacturer supplied error surfaces can be found in the Torque and BSFC
OEMs Supplied Error Surfaces section of the report.
Although the replay engine testing showed torque and BSFC deltas that exceeded the
Model prediction, the Steering Committee decided to take no action in regard to the replay
validation testing results.
6.10 Validation Results
This section contains a summary of the model validation results, Section 2.1.11 on
Validation contains a more detailed description of the validation methodology utilized both in
the simulation and in the on-road data collection efforts.
During the Monte Carlo simulation of the 195 reference NTE events some of the error
surfaces were excluded in the computation of the BS emissions 'with errors' so that the
simulation represented conditions used in collecting the on-road data. The error surfaces
excluded were torque errors (# 29-32, 34, 35), BSFC errors (#36-39, 41, 42), dynamic speed
SwRI Report 03.12024.06
Page 340 of 371
-------
(#43) and dynamic fuel rate (#44). As described earlier in Section 2.1.11, this was necessary due
to the fact that there was no reference torque or reference fuel flow measurement during on-road
data collection. Therefore it would not be appropriate to use these error terms from the
simulation when comparing to the on-road data set, as the on-road data set would not include any
errors generate from those measurements. For each reference NTE event, the differences in BS
emissions were computed as:
delta BS emissions = BS emissions with "Validation error" - "Ideal" BS emissions.
These delta emissions were computed for each of the three emissions and all three calculation
methods. The validation results also included time alignment and checks for periodic drift.
The on-road results were gathered from selected routes driven to collect emissions data
with a CE-CERT trailer and a PEMS installed in the tractor pulling the trailer (see Section 2.1.11
on Validation). For each on-road NTE event, a delta BS emissions value was computed as
delta BS emissions = PEMS BS emissions - CE-CERT BS emissions.
These differences were computed for all three emissions and three calculation methods. The on-
road delta BS emissions were calculated from 81 NTE events for BSNOX and 87 NTE events for
BSNMHC and BSCO selected by the Steering Committee from the original set of data collected
on-road. All of the NTE events selected from the on-road data were drift corrected.
From the MC simulations, the 5th and 95th percentile delta BS emissions were extracted
from the output files for each reference NTE event. These percentiles were then plotted as
empirical distribution functions (edf) to form a validation interval for the on-road data. Also
plotted was the edf computed from the on-road NTE events. Figure 260 represents the validation
plot for the BSNOX Method 1 analyses. Since all of the on-road delta BSNOX emissions fell
within the 5th and 95th percentiles of the simulation results the model was validated for this
method and emissions.
SwRI Report 03.12024.06 Page 341 of 371
-------
Validation Analysis 5th and 95th Percentile Deltas
Compared Ref NTE #1—195 to Corrected 81 CE-CERT Deltas
(g/kW—hr) Method 1
100
c
il!
0
aj
a.
80-
f:0 •
40«
20 •
-0.4
-0.2
0.0
0.2 0.4 0.6
Delta NCx g/kW-hr
0.8
1.0
1.2
FIGURE 260. VALIDATION ON-ROAD AND MODEL GENERATED EMPIRICAL
DISTRIBUTION FUNCTIONS FOR BSNOX METHOD 1
The validation edf plot for the BSNOX Method 2 results is shown in Figure 261. Note
that approximately 45% of the on-road delta BSNOX emissions fell above the 95th percentile
deltas from the simulation model. Thus, the model was not considered valid for the BSNOX
Method 2 results.
SwRI Report 03.12024.06
Page 342 of 371
-------
Validation Analysis 5th and 95th Percentile Deltas
Compared Ref NTE #1—195 to Corrected 81 CE-CEFTT Deltas
NOx (g/kW-hr) Method 2
100*
6u
40«
20
—1.2
-0.8
-0.6
—0.4 -0.2 0.0
Delta NOc g/kW-rir
0,2
0.4
0.6
FIGURE 261. VALIDATION ON-ROAD AND MODEL GENERATED EMPIRICAL
DISTRIBUTION FUNCTIONS FOR BSNOX METHOD 2
The validation edf plot for the BSNOX Method 3 results is shown in Figure 262. Note
that approximately 55% of the on-road delta BSNOX emissions fell above the 95th percentile
deltas from the simulation model. Thus, the model was not considered valid for the BSNOX
Method 3 results.
SwRI Report 03.12024.06
Page 343 of 371
-------
Validation Analysis 5th and 95th Percentile Deltas
Compared Ref NTE #1—195 to Corrected 81 CE-CEFTT Deltas
NOx (g/kW-hr) Method 3
100*
6u
40«
20
—1.2
-0.8
-0.6
—0.4 -0.2 0.0
Delta NOc g/kW-hr
0,2
0.4
0.6
FIGURE 262. VALIDATION ON-ROAD AND MODEL GENERATED EMPIRICAL
DISTRIBUTION FUNCTIONS FOR BSNOX METHOD 3
Figure 263 through Figure 265 represent the validation plots for the BSNMHC analyses
for Methods 1, 2 and 3, respectively. Since the entire on-road delta BSNMHC emissions fell
within the 5th and 95th percentiles of the simulation results the model was validated for all three
methods for BSNMHC emissions.
SwRI Report 03.12024.06
Page 344 of 371
-------
Ill
CL
Validation Analysis 5th and 95th Percentile Deltas
Compared Ref NTE #1—195 to Corrected 87 CE-CERT Deltas
NMHC (g/kW-hr) Method 1
100'
6u
40«
20
95th % MC Delta
195
—0.02
0,00
0.02 0.04 0.06
Delta NMHC g/kW-hr
0.08
0.10
FIGURE 263. VALIDATION ON-ROAD AND MODEL GENERATED EMPIRICAL
DISTRIBUTION FUNCTIONS FOR BSNMHC METHOD 1
SwRI Report 03.12024.06
Page 345 of 371
-------
Validation Analysis 5th and 95th Percentile Deltas
Compared Ref NTE #1—195 to Corrected 87 CE-CERT Deltas
NMHC (g/kW—hr) Method 2
100
c
il!
0
aj
a.
80-
f:0 •
40«
20"
-0.035
-0.025
-0.015 -0.006 0.005
Delta NMHC g/kW-hr
0.015
0.025
u.u'jb
FIGURE 264. VALIDATION ON-ROAD AND MODEL GENERATED EMPIRICAL
DISTRIBUTION FUNCTIONS FOR BSNMHC METHOD 2
SwRI Report 03.12024.06
Page 346 of 371
-------
Validation Analysis 5th and 95th Percentile Deltas
Compared Ref NTE #1—195 to Corrected 87 CE-CERT Deltas
NMHC (g/kW—hr) Method 3
100'
-0.035
-0.025
-0.015 -0.005 0.005
Delta NMHC g/kW-hr
0.015
0.025
0.035
FIGURE 265. VALIDATION ON-ROAD AND MODEL GENERATED EMPIRICAL
DISTRIBUTION FUNCTIONS FOR BSNMHC METHOD 3
The validation edf plot for the BSCO Method 1 results is shown in Figure 266. Note that
all of the of the on-road delta BSCO emissions fell below the 5th percentile deltas from the
simulation model. Thus, the model was not considered valid for the BSCO Method 1 results.
SwRI Report 03.12024.06
Page 347 of 371
-------
Validation Analysis 5th and 95th Percentile Deltas
Compared Ref NTE #1—195 to Corrected 87 CE-CERT Deltas
CO (g/kW-hr) Method 1
100'
SO
40«
20
95th % MC Delta
196
—0.25 0.00 0.25 0.50 0.75 1.00
Delta CO g/kW-hr
1.25
1.50
1.75
2.00
FIGURE 266. VALIDATION ON-ROAD AND MODEL GENERATED EMPIRICAL
DISTRIBUTION FUNCTIONS FOR BSCO METHOD 1
The validation edf plots for the BSCO Methods 2 and 3 results are shown in Figure 267
and Figure 268, respectively. Note that approximately 20% of the on-road delta BSCO
emissions fell below the 5th percentile deltas from the simulation model for both methods. Thus,
the model was not considered valid for the BSCO Method 2 or Method 3 results.
SwRI Report 03.12024.06
Page 348 of 371
-------
Validation Analysis 5th and 95th Percentile Deltas
Compared Ref NTE #1—195 to Corrected 87 CE-CERT Deltas
CO (g/kW-hr) Method 2
100 H
-0.2
-0.1
0.0
0.1
0.2 0.3 0.4 0.5
Delta CO g/kW-hr
0.6
0.7
0.8
FIGURE 267. VALIDATION ON-ROAD AND MODEL GENERATED EMPIRICAL
DISTRIBUTION FUNCTIONS FOR BSCO METHOD 2
SwRI Report 03.12024.06
Page 349 of 371
-------
Validation Analysis 5th and 95th Percentile Deltas
Compared Ref NTE #1—195 to Corrected 87 CE-CERT Deltas
CO (g/kW-hr) Method 3
100
—0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1,1
Delta CO g/kW-hr
FIGURE 268. VALIDATION ON-ROAD AND MODEL GENERATED EMPIRICAL
DISTRIBUTION FUNCTIONS FOR BSCO METHOD 3
Table 124 summarizes the results of the model validation for each of the three emissions
and all the methods.
SwRI Report 03.12024.06
Page 350 of 371
-------
TABLE 124. SUMMARY OF MODEL VALIDATION RESULTS
Emission
CE-CERT Deltas < 95th Percentile Deltas from Validation-MC Model
Method 1 Method 2 Method 3
Torque-Speed BSFC ECM Fuel Specific
BSNOx
BSNMHC
BSCO
Yes
Yes
Yes
No
No
Yes Yes
Yes Yes
CE-CERT Deltas > 5th Percentile Deltas from Validation-MC Model
BSNOx Yes Yes Yes
BSNMHC Yes Yes Yes
BSCO No No No
SwRI Report 03.12024.06
Page 351 of 371
-------
7.0 MEASUREMENT ALLOWANCE GENERATION AND CONCLUSIONS
This section of the report contains a description of the final generation of the Measurement
Allowances, as well as a summary of some of the major conclusions derived from this program.
7.1 Measurement Error Allowance Results
This section contains a summary of the measurement error allowance results using both a
regression method and a median method to determine the measurement allowance. Section
2.1.10 on Measurement Allowance contains a detailed description of the methodology followed
in determining these values. This procedure was applied to the simulation data for all 195
reference NTE events obtained for each of the three emissions and all three calculation methods.
Figure 269 contains a regression plot of the 95th percentile delta BSNOx values (using
Method 1 and with time alignment adjustment) versus the Ideal BSNOX values for the 195
reference NTE events. Included in the right-hand corner of the plot is the equation for the fitted
regression line, and the R-square (R2) value and root mean square error (RMSE) value for the
regression fit. The R-square value of 90.56% met the criteria given in the Test Plan for use of
the regression line for generation of a potential measurement allowance, and indicates that the
magnitude of the 95th percentile BSNOX delta was linear with respect to the ideal BSNOX level.
The RMSE value of 0.0955 displays the size of the estimated standard deviation of the predicted
95th percentile BSNOX values.
Table 125 includes a comparison of the results of the regression method based on Figure
269 and the median method as described in the Section 2.1.10 on Measurement Allowance.
Under the heading of "Regression Method" in the table, it is shown that the R-square and RMSE
criteria are met by the data and that the measurement error at the BSNOX threshold, based on
using the regression line to predict the value, is 22.2981% when expressed as a percent of the
threshold value of 2.68204. Since the Regression Method is applicable, the Median Method,
though shown in the table for comparison purposes, is not applicable.
SwRI Report 03.12024.06 Page 352 of 371
-------
0.00
NOx g/kW-hr Method 1
With Time Alignment Adjustment
345
Ideal NOxg/kW-hr
-TH
FIGURE 269. REGRESSION PLOT OF 95ltt PERCENTILE DELTA BSNOX VERSUS
IDEAL BSNOx FOR METHOD 1
TABLE 125. MEASUREMENT ERROR AT THRESHOLD FOR BSNOX USING
REGRESSION AND MEDIAN METHODS FOR METHOD 1
Regression Method
R2
RMSE(SEE)
5% Median Ideal
Predicted 95th% Delta at
Threshold
Measurement Error @
Threshold=2.68204
0.9065
0.0955
0.1302
0.5980
22.2981%
Met Criteria
Met Criteria
Median Method
Median 95th% Delta
Measurement Error @
Threshold=2.68204
0.5558
20.7215%
Figure 270 contains a regression plot of the 95th percentile delta BSNOx values (using
Method 2 and with time alignment adjustment) versus the Ideal BSNOx values for the 195
reference NTE events. The R-square value indicates that 33.64% of the variation in the 95th
percentile BSNOx values is explained by the Ideal BSNOx values for the Method 2 data. The
RMSE value is 0.0181.
Table 126 includes a comparison of the results of the regression method based on Figure
270 and the median method. Under the heading of "Regression Method" in the table, it is shown
that the R-square criterion for using this method is not met by the data. Thus, the Median
Method must be used. Under the heading "Median Method" in the table, the measurement error
SwRI Report 03.12024.06
Page 353 of 371
-------
at the BSNOx threshold, based on using the median of the 195 95th percentile delta BSNOX
values, is 4.4507% when expressed as a percent of the threshold value of 2.68204.
0.25
0.20 -
0.15 -
0.10 -
0.05 -
0.00
NOx g/kW-hr Method 2
With Time Alignment Adjustment
y = 0.0087X + 0.097
R2 = 0.3364
RMSE=0.0181
345
Ideal NOxg/kW-hr
-TH
FIGURE 270. REGRESSION PLOT OF 95ltt PERCENTILE DELTA BSNOX VERSUS
IDEAL BSNOx FOR METHOD 2
TABLE 126. MEASUREMENT ERROR AT THRESHOLD FOR BSNOx USING
REGRESSION AND MEDIAN METHODS FOR METHOD 2
Regression Method
R2
RMSE(SEE)
5% Median Ideal
Predicted 95th% Delta at
Threshold
Measurement Error @
Threshold=2.68204
0.3364
0.0181
0.1302
0.1203
4.1853%
Did Not Meet Criteria
Met Criteria
Median Method
Median 95th% Delta
Measurement Error @
Threshold=2.68204
0.1194
4.4507%
Figure 271 contains a regression plot of the 95th percentile delta BSNOx values (using
Method 3 and with time alignment adjustment) versus the Ideal BSNOx values for the 195
reference NTE events. The R-square value indicates that 58.28% of the variation in the 95th
percentile BSNOx values is explained by the Ideal BSNOx values for the Method 3 data. The
RMSE value is 0.0603.
SwRI Report 03.12024.06
Page 354 of 371
-------
Table 127 includes a comparison of the results of the regression method based on Figure
271 and the median method. Under the heading of "Regression Method" in the table, it is shown
that the R-square criterion for using this method is not met by the data. Thus, the Median
Method must be used. Under the heading "Median Method" in the table, the measurement error
at the BSNOx threshold, based on using the median of the 195 95th percentile delta BSNOX
values, is 6.6088% when expressed as a percent of the threshold value of 2.68204.
5?
.c
0.70
0.60 -
0.50 -
0.40 -
0.30 -
0.20 -
0.10
0.00
NOx g/kW-hr Method 3
With Time Alignment Adjustment
y = 0.048x + 0.0629
R2 = 0.5828
RMSE = 0.0603—
345
Ideal NOxg/kW-hr
-TH
FIGURE 271. REGRESSION PLOT OF 95ltt PERCENTILE DELTA BSNOx VERSUS
IDEAL BSNOx FOR METHOD 3
TABLE 127. MEASUREMENT ERROR AT THRESHOLD FOR BSNOx USING
REGRESSION AND MEDIAN METHODS FOR METHOD 3
Regression Method
R2
RMSE(SEE)
5% Median Ideal
Predicted 95th% Delta at
Threshold
Measurement Error @
Threshold=2.68204
0.5828
0.0603
0.1302
0.1916
7.1455%
Did Not Meet Criteria
Met Criteria
Median Method
Median 95th% Delta
Measurement Error @
Threshold=2.68204
0.1773
6.6088%
-th
Figure 272 contains a regression plot of the 95 percentile delta BSNMHC values (using
Method 1 and with time alignment adjustment) versus the Ideal BSNMHC values for the 195
reference NTE events. The R-square value indicates that 79.73% of the variation in the 95th
SwRI Report 03.12024.06
Page 355 of 371
-------
percentile BSNOx values is explained by the Ideal BSNMHC values for the Method 1 data. The
RMSE value is 0.0049.
Table 128 includes a comparison of the results of the regression method based on Figure
272 and the median method. Under the heading of "Regression Method" in the table, it is shown
that both the R-square and RMSE criteria for using this method are not met by the data. Thus,
the Median Method must be used. Under the heading "Median Method" in the table, the
measurement error at the BSNMHC threshold, based on using the median of the 195 95th
percentile delta BSNMHC values, is 10.0778% when expressed as a percent of the threshold
value of 0.28161.
0.12
0.10 -
0.08 -
0.06 -
0.04
0.02
0.00
NMHC g/kW-hr Method 1
y = 0.1639X + 0.0258
R2 = 0.7973
RMSE= 0.0049
0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45
Ideal NMHC g/kW-hr
-TH
FIGURE 272. REGRESSION PLOT OF 95ltt PERCENTILE DELTA BSNMHC
VERSUS IDEAL BSNMHC FOR METHOD 1
SwRI Report 03.12024.06
Page 356 of 371
-------
TABLE 128. MEASUREMENT ERROR AT THRESHOLD FOR BSNMHC USING
REGRESSION AND MEDIAN METHODS FOR METHOD 1
Regression Method
R2
RMSE(SEE)
5% Median Ideal
Predicted 95th% Delta at
Threshold
Measurement Error @
Threshold=0.28161
0.7973
0.0049
0.0001
0.0719
25.5339%
Did Not Meet Criteria
Did Not Meet Criteria
Median Method
Median 95th% Delta
Measurement Error @
Threshold=0.28161
0.0284
10.0778%
th
Figure 273 contains a regression plot of the 95th percentile delta BSNMHC values (using
Method 2 and with time alignment adjustment) versus the Ideal BSNMHC values for the 195
reference NTE events. The R-square value indicates that 7.07% of the variation in the 95
percentile BSNOx values is explained by the Ideal BSNMHC values for the Method 2 data. The
RMSE value is 0.0042.
Table 129 includes a comparison of the results of the regression method based on Figure
273 and the median method. Under the heading of "Regression Method" in the table, it is shown
that both the R-square and RMSE criteria for using this method are not met by the data. Thus,
the Median Method must be used. Under the heading "Median Method" in the table, the
measurement error at the BSNMHC threshold, based on using the median of the 195 95th
percentile delta BSNMHC values, is 8.0310% when expressed as a percent of the threshold value
ofO.28161.
SwRI Report 03.12024.06
Page 357 of 371
-------
0.035
0.000
NMHC g/kW-hr Method 2
R2 = 0.0707
RMSE= 0.0042
0.010 -
0.005
0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45
Ideal NMHCg/kW-hr
-TH
FIGURE 273. REGRESSION PLOT OF 95ltl PERCENTILE DELTA BSNMHC
VERSUS IDEAL BSNMHC FOR METHOD 2
TABLE 129. MEASUREMENT ERROR AT THRESHOLD FOR BSNMHC USING
REGRESSION AND MEDIAN METHODS FOR METHOD 2
Regression Method
R2
RMSE(SEE)
5% Median Ideal
Predicted 95th% Delta at
Threshold
Measurement Error @
Threshold=0.28161
0.0707
0.0042
0.0001
0.0182
6.4671%
Did Not Meet Criteria
Did Not Meet Criteria
Median Method
Median 95th% Delta
Measurement Error @
Threshold=0.28161
0.0226
8.0310%
Figure 274 contains a regression plot of the 95th percentile delta BSNMHC values (using
Method 3 and with time alignment adjustment) versus the Ideal BSNMHC values for the 195
reference NTE events. The R-square value indicates that 0.21% of the variation in the 95th
percentile BSNOx values is explained by the Ideal BSNMHC values for the Method 3 data. The
RMSE value is 0.0047.
Table 130 includes a comparison of the results of the regression method based on Figure
274 and the median method. Under the heading of "Regression Method" in the table, it is shown
that both the R-square and RMSE criteria for using this method are not met by the data. Thus,
the Median Method must be used. Under the heading "Median Method" in the table, the
SwRI Report 03.12024.06
Page 358 of 371
-------
measurement error at the BSNMHC threshold, based on using the median of the 195 95th
percentile delta BSNMHC values, is 8.4436% when expressed as a percent of the threshold value
ofO.28161.
0.040
0.010
NMHC g/kW-hr Method 3
0.035 -
0.030
0.025
0.020
0.015 -
y = 0.0036X + 0.0234
R2 = 0.0021
RMSE = 0.0047
0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45
Ideal NMHC g/kW-hr
-TH
FIGURE 274. REGRESSION PLOT OF 95ltl PERCENTILE DELTA BSNMHC
VERSUS IDEAL BSNMHC FOR METHOD 3
TABLE 130. MEASUREMENT ERROR AT THRESHOLD FOR BSNMHC USING
REGRESSION AND MEDIAN METHODS FOR METHOD 3
Regression Method
R2
RMSE(SEE)
5% Median Ideal
Predicted 95th% Delta at
Threshold
Measurement Error @
Threshold=0.28161
0.0021
0.0047
0.0001
0.0244
8.6739%
Did Not Meet Criteria
Did Not Meet Criteria
Median Method
Median 95th% Delta
Measurement Error @
Threshold=0.28161
0.0238
8.4436%
Figure 275 contains a regression plot of the 95th percentile delta BSCO values (using
Method 1 and with time alignment adjustment) versus the Ideal BSCO values for the 195
reference NTE events. The R-square value indicates that 60.48% of the variation in the 95th
percentile BSCO values is explained by the Ideal BSCO values for the Method 1 data. The
RMSE value is 0.1403.
SwRI Report 03.12024.06
Page 359 of 371
-------
Table 131 includes a comparison of the results of the regression method based on Figure
275 and the median method. Under the heading of "Regression Method" in the table, it is shown
that both the R-square and RMSE criteria for using this method are not met by the data. Thus,
the Median Method must be used. Under the heading "Median Method" in the table, the
measurement error at the BSCO threshold, based on using the median of the 195 95th percentile
delta BSNMHC values, is 2.5775% when expressed as a percent of the threshold value of
26.015.
CO g/kW-hr Method 1
With Time Alignment Adjustment
2.50
y = 0.2428X + 0.5685
R2 = 0.6048
RMSE= 0.1403
0.00
3 4
Ideal COg/kW-hr
-TH
FIGURE 275. REGRESSION PLOT FOR 95ltt PERCENTILE DELTA BSCO VERSUS
IDEAL BSCO FOR METHOD 1
TABLE 131. MEASUREMENT ERROR AT THRESHOLD FOR BSCO USING
REGRESSION AND MEDIAN METHODS FOR METHOD 1
Regression Method
R2
RMSE(SEE)
5% Median Ideal
Predicted 95th% Delta at
Threshold
Measurement Error @
Threshold=26.015
0.6048
0.1403
0.0192
2.0242
7.7808%
Did Not Meet Criteria
Did Not Meet Criteria
Median Method
Median 95th% Delta
Measurement Error @
Threshold=26.015
0.6705
2.5775%
-th
Figure 276 contains a regression plot of the 95 percentile delta BSCO values (using
Method 2 and with time alignment adjustment) versus the Ideal BSCO values for the 195
SwRI Report 03.12024.06
Page 360 of 371
-------
reference NTE events. The R-square value indicates that 3.67% of the variation in the 95th
percentile BSCO values is explained by the Ideal BSCO values for the Method 2 data. The
RMSE value is 0.1139.
Table 132 includes a comparison of the results of the regression method based on Figure
276 and the median method. Under the heading of "Regression Method" in the table, it is shown
that both the R-square and RMSE criteria for using this method are not met by the data. Thus,
the Median Method must be used. Under the heading "Median Method" in the table, the
measurement error at the BSCO threshold, based on using the median of the 195 95th percentile
delta BSNMHC values, is 1.9924% when expressed as a percent of the threshold value of
26.015.
CO g/kW-hr Method 2
With Time Alignment Adjustment
0.90
0.80
0.70 -
0.60 -
0.50 -
0.40 -
0.30
y =0.0311x + 0.5156
R2 = 0.0367
—RMSE = 0.1139
3 4
Ideal COg/kW-hr
-TH
FIGURE 276. REGRESSION PLOT OF 95ltt PERCENTILE DELTA BSCO VERSUS
IDEAL BSCO FOR METHOD 2
SwRI Report 03.12024.06
Page 361 of 371
-------
TABLE 132. MEASUREMENT ERROR AT THRESHOLD FOR BSCO USING
REGRESSION AND MEDIAN METHODS FOR METHOD 2
Regression Method
R2
RMSE(SEE)
5% Median Ideal
Predicted 95th% Delta at
Threshold
Measurement Error @
Threshold=26.015
0.0367
0.1139
0.0192
0.7020
2.6986%
Did Not Meet Criteria
Did Not Meet Criteria
Median Method
Median 95th% Delta
Measurement Error @
Threshold=26.015
0.5183
1.9924%
th
Figure 277 contains a regression plot of the 95th percentile delta BSCO values (using
Method 3 and with time alignment adjustment) versus the Ideal BSCO values for the 195
reference NTE events. The R-square value indicates that only 22.09% of the variation in the 95
percentile BSCO values is explained by the Ideal BSCO values for the Method 3 data. The
RMSE value is 0.1254.
Table 133 includes a comparison of the results of the regression method based on Figure
277 and the median method. Under the heading of "Regression Method" in the table, it is shown
that both the R-square and RMSE criteria for using this method are not met by the data. Thus,
the Median Method must be used. Under the heading "Median Method" in the table, the
measurement error at the BSCO threshold, based on using the median of the 195 95th percentile
delta BSNMHC values, is 2.1117% when expressed as a percent of the threshold value of
26.015.
SwRI Report 03.12024.06
Page 362 of 371
-------
1.40
0.20
CO g/kW-hr Method 3
With Time Alignment Adjustment
y = 0.0932X + 0.5156
R2 = 0.2209
RMSE = 0.1254
3 4
Ideal COg/kW-hr
-TH
FIGURE 277. REGRESSION PLOT OF 95ltl PERCENTILE DELTA BSCO VERSUS
IDEAL BSCO FOR METHOD 3
TABLE 133. MEASUREMENT ERROR AT THRESHOLD FOR BSCO USING
REGRESSION AND MEDIAN METHODS FOR METHOD 3
Regression Method
R2
RMSE(SEE)
5% Median Ideal
Predicted 95th% Delta at
Threshold
Measurement Error @
Threshold=26.015
0.2209
0.1254
0.0192
1.0744
4.1299%
Did Not Meet Criteria
Did Not Meet Criteria
Median Method
Median 95th% Delta
Measurement Error @
Threshold=26.015
0.5494
2.1117%
Table 134 contains a summary of the measurement error values contained in Table 125
through Table 133. The values are categorized by emissions and by calculation method. The
maximum error by method is listed in the last row of the table. Although the Test Plan
methodology (i.e., see Section 2.1.10 on Measurement Allowance) initially indicated that the
minimum of these values was to be used to select the best method, the actual choice is dependent
on the ability to validate the model runs using the on-road CE-CERT data After reviewing the
validation results given in Table 124 and noting that only Method 1 validated for BSNOx, the
Steering Committee decided at a meeting held on February 26, 2007 to use Method 1 to
determine the measurement error allowance.
SwRI Report 03.12024.06
Page 363 of 371
-------
TABLE 134. MEASUREMENT ERROR IN PERCENT OF NTE THRESHOLD BY
EMISSIONS AND CALCULATION METHOD
Measurement Errors (%) at Respective NTE Threshold
Emission
BSNOx
BSNMHC
BSCO
Max Error
Method 1
Torque-Speed
22.30
10.08
2.58
22.30
Method 2
BSFC
4.45
8.03
1.99
8.03
Method 3
ECM Fuel Specific
6.61
8.44
2.11
8.44
Note: Values in table cells shaded white were successfully validation, while values
in cells shaded grey were not validated.
Table 135 includes in the last column of the table the measurement error allowances by
emissions for Method 1. The values are 0.44596 g/hp-hr for BSNOX, 0.02116 g/hp-hr for
BSNMHC, and 0.50002 g/hp-hr for BSCO.
TABLE 135. MEASUREMENT ALLOWANCE AT NTE THRESHOLD BY EMISSIONS
FOR METHOD 1
Emission
BSNOx
BSNMHC
BSCO
Method 1
Measurement
Error %
22.30
10.08
2.58
NTE
Threshold
g/kW-hr
2.68204
0.28161
26.015
Measurement
Allowance,
g/kW-hr
0.59804
0.02838
0.67054
Measurement
Allowance,
g/hp-hr
0.44596
0.02116
0.50002
7.2 Conclusions
The primary result of this program is the set of Measurement Allowances given in the
previous section. However, a number of other observations and conclusions may be drawn from
the experiences and data generated over the course of the program. This section of the report
details some of these observations, as well as recommendations arising from them.
SwRI Report 03.12024.06
Page 364 of 371
-------
7.2.1 Engine-Instattation-PEMS Variability
During the course of this program, a variety of PEMS were used to measure emissions
from three different engines. It became apparent that the measurement errors observed were not
consistent from installation to installation. In some cases the use of different equipment
contributed to the changes, for example the use of different size flow-meters. In other cases,
selected PEMS did not behave in the same fashion from engine to engine, as was the case for
PEMS 6 which experienced large negative errors during steady-state testing for Engine 3, but not
for Engine 2. At other times, PEMS equipment was repaired or replaced, after which different
behavior was observed, as was the case with PEMS 4 which demonstrated different NOX
measurement errors on Engine 2 after the NDUV instrument was replaced.
The steady-state error surface data reflected these variations in the fact that the original
errors surfaces were not very smooth in nature, and showed abrupt changes in error magnitudes
for many key parameters at similar reference levels. In many cases, these abrupt changes took
place because data at similar reference levels was generated on different test engines and test
installations. This behavior underscores the fact that there is considerable variability arising
from PEMS to PEMS and installation to installation.
The Test Plan was designed to use multiple engines in order to get some sense of this
variability. However, with only three examples or "samples" of different installations in the
plan, many on the Steering Committee felt in hindsight that this source of variation was not very
well characterized from a statistical point of view. It is suggested that any future efforts to assess
PEMS variation should somehow include a better means of assessing this source of error.
7.2.2 PEMS 1065 Audit Failures
In general, the PEMS passed most of the performance checks required under 40 CFR Part
1065 Subpart D. However, as was noted in Section 3 of the report, there were numerous
linearity failures associated with the SEMTECH-DS instruments, particularly in the case of the
NDUV analyzer which measures NO and NC>2. Although the Steering Committee approved the
continuation of the program despite these problems, this was an issue of concern to many on the
Committee. It has been noted earlier that, due to the manner in which the Test Plan was laid out,
the linearity requirements used to audit the PEMS represented a relatively liberal interpretation
of the regulations given in Subpart D, as a result of use of the span values for the instruments as
the "maximum concentration expected during testing" for scaling of the requirements. Despite
this interpretation, numerous intercept deviations were still observed. Due to the requirement
that testing under the HDIUT program must be conducted using instruments meeting all
requirements of 40 CFR Part 1065, such deviations could be problematic.
There was considerable Steering Committee debate regarding this issue. It is suggested
that in any future programs of this type, a complete set of audit data should be supplied by any
participating instrument manufacturers prior to the start of the program, in order to demonstrate
that all 1065 requirements can be met. However, Part 1065 audits should still be conducted as
part of any such program to verify compliance.
SwRI Report 03.12024.06 Page 365 of 371
-------
7.2.3 Method 2 and Method 3 Validation for NOX
There was considerable discussion among the Steering Committee regarding the reasons
that the Model results for Methods 2 and 3 did not validate with respect to NOX emissions. A
significant amount of effort was also directed at determining the cause for the lack of validation
of these two methods. These efforts were directed both at examining the Model results, and at
scrutinizing the CE-CERT on-road validation test results.
Examination of the CE-CERT results did uncover several relatively minor issues, which
were corrected by CE-CERT or dealt with by the Steering Committee. However, no significant
procedural issues were found either with the operation of the PEMS or of the MEL. The
Steering Committee found no major faults with the CE-CERT data which would have changed
the conclusions of the validation process. Ultimately the CE-CERT results were judged by all
Steering Committee members to be a valid data set, and this opinion is shared by SwRI staff as
well.
In a similar manner, the Steering Committee did not find that the lack of validation arose
from any deficiencies in the manner with which SwRI conducted the experiments that supplied
data to the Model. In addition, extensive quality assurance and checking were performed on the
Model itself in order to insure that it performed in the manner intended by the Test Plan.
As a result of these investigations, therefore, it must be assumed that the reason for the
lack of validation of Methods 2 and 3 for NOX arose from a real difference in PEMS behavior
between the laboratory tests at SwRI and the field tests conducted by CE-CERT. This would
seem to indicate that some source of variation that occurred during the field tests was not
captured by the various experiments which supplied data for the model. This difference could
not be related to any Torque or BSFC errors, as those were not relevant to the on-road validation
results, as explained earlier in Section 6. In addition, the difference could not be related to the
exhaust flow measurement, because this would have also affected the Method 1 results.
Given these observations, the only likely causes for the lack of validation are related
either to the measurement of CC>2 or of NOX itself. In the case of CC>2, the lower deltas would
have to be caused by a positive bias in the CC>2 errors predicted by the model as compared to
those observed during on-road testing. The Model does in fact incorporate a positive bias in CCh
errors, as reflected in the steady-state CC>2 error surface. This in turn reflects the fact that
negative PEMS CC>2 measurement errors were not observed during laboratory testing. However,
examination of the CC>2 data from on-road testing indicates that negative errors were also not
observed during the on-road validation. Therefore, it is unlikely that a CC>2 bias was the cause of
the lack of validation.
As a result, the root cause for the lack of validation is likely to lie with the NOX
measurement itself. This could be the result either of a bias between the two data sets, or the
result of variation. Both reference facilities had been correlated earlier in the program, and no
procedural errors were found either for SwRI or for CE-CERT. In addition, the PEMS used for
on-road validation testing was one of the units used at SwRI, and was later used again by SwRI
for laboratory validation and found to be in good working order. For these reasons, and
following a certain amount of engineering analysis, the Steering Committee could not determine
SwRI Report 03.12024.06 Page 366 of 371
-------
a reason that a bias should exist between the two data sets which would result in lack of
validation for Methods 2 and 3.
Another possibility which was raised during Steering Committee discussions was that the
level of variance predicted by the Model was not wide enough to encompass the results observed
during the CE-CERT validation testing. The steady-state NOX error surface generally predicts an
error of ± 20 ppm at the concentrations observed during on-road validation testing. The
sensitivity analysis given in Section 6 earlier indicates that this surface is a dominant driver of
NOX errors for Methods 2 and 3 in the Model. It is possible that the deltas observed during on-
road validation testing fell outside this range. Unfortunately, it is not possible to determine
concentration deltas directly from the CE-CERT data as no raw reference concentrations are
available for comparison with PEMS values.
An analysis of PEMS instrument QA data was conducted for both the CE-CERT data and
some of the steady-state experiments used to generate data for the Model at SwRI. Figure 278
shows zero calibration data observed during on-road testing, while Figure 279 shows similar data
for Engine 3 tests at SwRI. A comparison of these two data sets indicates much larger amounts
of variation in PEMS zero calibration adjustments during on-road testing, as compared to similar
adjustments in the laboratory. It should be noted that the range of these variations observed
during on-road testing is actually larger than the spread predicted by the steady-state NOX error
surface in the Model. The cause of these larger variations is not known, but they do indicate that
some source of error is present in the on-road data that was not apparent during laboratory tests.
A possible explanation for this difference in behavior is that it could be the result of the
installation and equipment variations described earlier. A concern of several Steering
Committee members was that there were essentially only three observations of this potential
error source, and that the on-road validation experiment may have represented a data point
outside that range of errors observed during these three cases. If this was the case, it would
suggest that the spread of the actual steady-state NOX error from 5th to 95th percentile is in fact
wider than what was given in the steady-state NOX error surface. This does not suggest that the
observations made during the Measurement Allowance program were incorrect, but rather that
not enough observations were made to fully characterize the error. Unfortunately, there was not
way to test this hypothesis within the scope of the program, and therefore there is no way of
knowing if this explanation is valid.
SwRI Report 03.12024.06 Page 367 of 371
-------
PEMs Total NO+NO2 Zero Calibrations
Af\
^-^ on
E 3°
Q.
Q.
C
O
c
0)
o
C n
0 °
0 (
CM
O 10
6
-in
_*n
PEMs average NO+NO2 concentration during correlation was ~450 ppm.
10 ppm divisions are ~2.2% of 450 ppm.
3
• <
) 5
,: ' *
< * * • • *• •
/ n 15 20 25 "^30 35 4ft 45 5
0
1
n
X
0)
•o
I
Zero Count
»Previoius Current A Day Index
FIGURE 278. VARIATION IN ZERO CALIBRATION OF PEMS DURING ON-ROAD
VALIDATION TESTING
SwRI Report 03.12024.06
Page 368 of 371
-------
PEMs 1 and 4 Total NO+NO2 Zero Calibrations Engine 3 Testing @ SwRI
5
I -2
s ^
c -4
o
O
-10
-12
-14
• 2
: «
1*4
18
20 g
12
10
6
(0
S
HI
Q.
Zero Count
»Previoius • Current A PEMS Index
FIGURE 279. PEMS ZERO CALIBRATION VARIATIONS DURING LABORATORY
TESTING
Ultimately, the Steering Committee decision was to accept the measurement allowances
predicted by the model for Method 1, as these were the only validated numbers. However, EPA
indicated that they would continue investigations into the possible causes for the lack of
validation of Methods 2 and 3, as well as look into modifications that might result in achieving
validation of those methods. It was agreed that any potential revision of the measurement
allowances that might arise from such investigations would be reviewed by the Steering
Committee and pursued as a cooperative effort between EPA, EMA and CARB. Further it was
determined that such changes would not apply before the 2010 model year.
7.2.4 PEMS Sampling Handling System Issues and Overflow Checks
As described throughout this report, and related in Appendix A, there were numerous
issues observed with the PEMS over the course of the program. Many of these issues were
ultimately traced to problems of deficiencies within the sample handling system of the
SEMTECH-DS. In many cases, an overflow span check proved to be the diagnostic exercise
which helped to isolate the source of the issue. In such a check, the instrument is zero and span
calibrated using the calibration port, and then the same span gas is overflowed to the sample
probe such that it enters the instrument through the sample line and passes through the sample
SwRI Report 03.12024.06
Page 369 of 371
-------
handling system. Although such a check is not required by 1065, SwRI found this to be an
invaluable tool for assessing PEMS and for diagnosing problems.
Therefore based on its experiences with these devices, SwRI would recommend the
inclusion of such an overflow check in the regular operations of a PEMS. Such a check could be
performed on installation of the PEMS on a vehicle, and again at the conclusion of testing.
While such a check is not strictly required under Part 1065, it is felt that performance of such a
check would help to identify many potential problems which could otherwise compromise PEMS
data.
7.2.5 Lessons Learned for Future Programs
Near the conclusion of this program, the Steering Committee engaged in several
discussions regarding various issues that arose during the program, and how they might be
addressed in any future work of this kind. This was particularly relevant given the fact that a
subsequent effort is about to be undertaken involving in-use PM measurement.
The result of these discussions was a list of issues, findings, and observations which were
collectively termed as "lessons learned." This list is not necessarily exhaustive, but it does
represent a group of observations that the Steering Committee collectively felt were the most
important in terms of how any future program of this kind might be conducted. These
observations are summarized in Table 136 below. It is hoped that this information will be useful
in planning for future efforts of this kind.
TABLE 136. LESSONS LEARNED DURING GASEOUS MEASUREMENT
ALLOWANCE PROGRAM
1. The Steering Committee felt that a number of the biases observed on several
error surfaces were likely the results of the limited number of engines and
installations tested, rather than a true systematic bias. Therefore, it was
recommended that error surfaces should be centered around zero unless there
is a technical reason not to do so. Observed biases during program could not
always be explained from a technical standpoint.
2. Broadcast torque needs to be revisited as an error term. The following ideas
were suggested on this topic:
a. Try to build an exception within the test program (extended data could
still be added in as before)
b. Variability
c. Hardware/Software changes
d. Check manufacturer supplied data vs. test data
3. Check engine to engine variability/differences. Try to structure the program
to account for this error source in a more robust manner.
4. Some error surfaces displayed highly irregular and erratic that were often the
result of different PEMS system behavior from on test engine to another, (ex,
exhaust flow delta varied between different sized flow meters for different
engines). Test Plan anticipate more consistent behavior from engine to
SwRI Report 03.12024.06 Page 370 of 371
-------
engine, and experimental design likely missed capturing sources of error
beyond the errors expected in Test Plan development (reproducibility are
larger than expected). Consider longer time frames.
CE-CERT validation was key to selecting Measurement Allowance.
6. Environmental errors were generally small compared to baseline variability
study. Limited number of exceptions. Environmental test generally caused
functional failures rather than measurement errors.
7. Vibration test was a frequency and direction spectrum of wide range of
frequencies to directly assess errors due to in-use vibration. It did not sweep
frequencies to identify susceptibility. This was out of scope. Limited field
vibration data. Consider adding 3-axis accelerometers on PEMS.
Errors in measurement of ambient temperature and humidity were not
included.
9. PEMS OEM manuals need to be available at start of test plan to better follow
manufacturer's recommendations. Any gray areas or multiple methods need
to be clarified before start of testing.
10. Incremental errors due to transient operation were smaller than expected.
11. Test engines emissions levels were much lower than threshold values for
NMHC and CO, data values primarily demonstrated PEMS performance near
zero levels, rather than at emission threshold values. Therefore, Measurement
Allowances may not entirely represent performance at threshold.
Recommended that future work try to use test articles near thresholds, even if
systems need to be modified to produce them.
12. Document control of Test Plan was weak.
13. Steering Committee decisions were not well documented during the course of
the program in an easily referenced manner. Improvement ideas include
meeting and conference call minutes, monthly reports, change history of test
plan.
14. Time alignment errors were larger than expected. Improvement idea is to
include phase errors in the model.
15. Test plan was not clear on the decision process if model did not validate.
16. Test plan did not specify how to use replay validation data.
17. Drift check in the model was weak.
18. PEMS issues burned up nearly all the contingency in the schedule. Back-up
PEMS was key to staying on schedule.
19. Method of pooling data into a single error surfaces changed based on actual
results.
20. PEMS vehicle installation process as a source of error was included in the
program only at the CE-CERT on road validation stage. Improvement idea is
to run the on-road validation earlier to provide input in developing the model.
21. Documents were shared via SwRI FTP website. Improvement idea is to use a
more standardized website layout. Need better data organization for online
data storage area.
22. 1065 has some gray areas with respect to PEMS performance checks. In the
process 1065 checks were prototyped and revisions were adapted.
23. Open issue on how to get PEMS to eventually PASS ALL 1065 checks.
SwRI Report 03.12024.06 Page 371 of 371
-------
Example: linearity check failures.
24. 1065 was not complete prior to start of gaseous programs. Improvement idea
is to expect PEMS supplier to have run and passed 1065 checks before
starting next program.
SwRI Report 03.12024.06 Page 372 of 371
-------
APPENDIX A
SEMTECH-DS OPERATION LOG
SwRI Report 03.12024.06
-------
SwRI Semtech PEMS Log
Date
1/4/2006
1/9/06 -
1/27/06
1/12/06-
2/15/06
1/15/2006
2/7/2006
2/15/06-
5/24/06
3/30/2006
4/3/2006
PEMS#
1,2,3,4
5" EFM
1
1,2,3,4
2
1,2,3,4
4
4
Summary
High CO
reading
1065 flow
measurement
linearity failure
1065 NO
linearity
failure
1065 N02
linearity
failure
1065 FID 02
interference
failure
1065 NO2
penetration
failure
Spikes in EFM
data
Unstable NO2
PEMS would
not
power-up
Problem
Occurred
During
Audit
Audit
Audit
Audit
Audit
Audit
Test
Test
Description
Symptoms
Problem
Solution
Symptoms
Problem
Solution
Symptoms
Problem
Solution
Symptoms
Problem
Solution
Symptoms
Problem
Solution
Symptoms
Problem
Solution
Symptoms
Problem
Solution
Symptoms
Problem
Solution
* CO reads 20-40 ppm when zero air is
sampled after a zero/span procedure
* Possibly caused by change in sample
pressure
* Different sample path for zero/span and
sample
* None
* 5-inch EFM repeatedly failed 1065 linearity
criteria
* Incorrect or outdated curve constants
* Sent flow meter to Sensors for recalibration
* Performed linearity check - passed
* NO repeatedly failed 1065 linearity - high
intercept, low slope
* Incorrect or outdated curve constants
* Sent PEMS 1 to Sensors for recalibration of
the NDUV
* Performed linearity check - passed
* NO2 repeatedly failed 1065 linearity - high
intercept, low slope
* Incorrect or outdated curve constants
* None taken
* FID repeatedly failed 1065.362 non-
stoichiometric FID O2 interference
* FID not tuned/optimized
* None taken
* PEMS repeatedly failed 1065.376 Chiller
NO2 Penetration
* Loss of NO2 in sample conditioning system
* Sensors filter bowl and chiller drain manifold
retrofit with RH/Temp sensor
* Correction Factor implemented by Sensors
using RH sensor
* EFM data had short spikes of excessive flow
* Unknown, may be linked to inverter problem
below (4/3/06)
* None
* Unstable and erratic NO2 readings
* Unit eventually would not turn on
* Inverter was supplying only 6 volts to the
PEMS
* Replaced inverter
SwRI Report 03.12024.06
A-l
-------
6/5/2006
6/5/2006
6/6/2006
6/6/2006
6/7/2006
6/7/2006
6/8/2006
6/9/2006
6/19/2006
7/12/06 -
8/2/06
3
3
5
1
5
6
6
4
1
4" EFMs
NDUV failure
1065 NO
linearity
failure
NDIR failure
NDUV failure
1065 NO
linearity
failure
1065 NO2
linearity
failure
1065 FID O2
interference
failure
NO and NO2
Unstable
Vehicle
Interface
not responding
4" EFMs failed
1065
linearity
Test
Audit
Test
Test
Audit
Audit
Audit
Test
Test
Audit
Symptoms
Problem
Solution
Symptoms
Problem
Solution
Symptoms
Problem
Solution
Symptoms
Problem
Solution
Symptoms
Problem
Solution
Symptoms
Problem
Solution
Symptoms
Problem
Solution
Symptoms
Problem
Solution
Symptoms
Problem
Solution
Symptoms
Problem
Solution
* NO and NO2 readings unstable and noisy
* NO and NO2 gains high or saturated
* NDUV lamp malfunction (Carl Ensfield)
* Replaced NDUV
* Performed linearity and NO2 penetration
check (NO linearity failed)
* NO repeatedly failed 1 065 linearity (high
intercept)
* Incorrect or outdated curve constants
* None taken
* CO and CO2 readings erratic and noisy
* NDIR failure
* Sensors replaced NDIR
* Performed linearity checks - passed
* Fault - NDUV not responding
* NDUV communication error
* Replaced NDUV
* Performed linearity and NO2 penetration
check - passed
* NO repeatedly failed 1 065 linearity (high
intercept)
* Incorrect or outdated curve constants
* None taken
* NO2 repeatedly failed 1065 linearity (high
intercept)
* Incorrect or outdated curve constants
* None taken
* FID repeatedly failed 1065.362 non-
stoichiometric FID O2 interference
* FID not tuned/optimized
* None taken
* NO and NO2 were unstable/noisy when
attempting to zero/span
* Unknown - possible NDUV lamp malfunction
* Restarted unit
* Fault - "Vehicle Interface not responding"
* Could not link to CAN bus
* Dearborn Group adapter not functioning
* Replaced Dearborn Group adapter
* 4-inch EFMs failed 1065 linearity check
compared to SwRI flow stand
* EFMs failed slope - low
* 4-inch EFMs showed similar biases
compared to SwRI engine exh. data
* Incorrect or outdated curve data
* Formulated new curve data based on SwRI
flow stand measurements
* Entered new linearization data for the flow
meters
* Performed linearity checks - passed
SwRI Report 03.12024.06
A-2
-------
7/14/2006
7/20/2006
8/8/2006
8/10/2006
2,5
6
2,3,5
6
24-volts
delivered to
digital input
FID failure
Low THC span
FID failure
Setup
Test
Test
Test
Notes
Symptoms
Problem
Solution
Symptoms
Problem
Solution
Symptoms
Problem
Solution
Symptoms
Problem
Solution
* SwRI connection error - 24 volts delivered to
digital input 1
* Aux 2 connector, pin 14 - digital input 1 was
delivered 24 volts
* Units shut down and would not restart
* Damaged I/O board in one unit, damaged
several boards in other unit
* Units sent to Sensors for repair - board
replacement
* FID temperature increases slowly
* FID does not reach operating temperature
* FID does not zero/span
* FID fault - "No zero/span performed, basic
data installed"
* FID fault - "FID Battery Backed Ram
Corrupt"
* FID block heater not functioning
* Sensors rebuilt FID at SwRI
* Replaced block heater
* FID appeared dirty and well used (Louciano)
* Performed linearity check - passed
* FIDs would not span to correct value (FIDs
would span low)
* FID digital gains saturated
* Adjustment of FID gain potentiometers did
not solve problem
* Excessive backpressure on FID drain
* Backpressure due to test setup -
long/restrictive drain lines
* FID drains and main drains were combined
* Rerouted drain lines, keeping FID and main
drains separate
* Used large 3" flexible hose to decrease drain
backpressure
* FID would not span to correct value (FID
would span low)
* FID digital gain saturated
* Adjustment of FID gain potentiometer did not
solve problem
* FID capillary tube obstructed
* Sensors rebuilt FID at SwRI
* Replaced FID capillary tube
* Problem likely caused by prior FID rebuild
(7/20/06)
* Performed linearity check - passed
SwRI Report 03.12024.06
A-3
-------
8/15/2006
8/22/2006
8/28/2006
9/6/2006
9/8/2006
2,3,5
2,3
5
3
3
CO span error
NO2
conversion/los
s
Temperature
probe
failure
NO2
conversion/los
s
Enclosure
wiring
problem
Test
Test
Test
Test
Test
Symptoms
Problem
Solution
Symptoms
Problem
Solution
Symptoms
Problem
Solution
Symptoms
Problem
Solution
Symptoms
Problem
Solution
* CO unstable and noisy during zero and span
operation
* CO readings from zero to several thousand
ppm during span
* CO digital gains saturated
* Likely caused from zeroing with span gas or
spanning with zero air
* Repeatedly zeroed and spanned CO
analyzers
* Instability subsided after first zero/span
operation, reading higher than span
* CO readings were correct after second
zero/span operation
* NO2 showed a significant drop during
environmental baseline testing
* NO2 span decreased during fist hour of
testing, then began to recover
* When NO2 decreased, NO increased -
possible conversion of NO2 to NO
* Problem with PEMS chillers - possible
deterioration of passivated coating
* Replaced chillers in PEMS 2 and 3
* Tested PEMS, PEMS 2 - no NO2 loss,
PEMS 3 - NO2 loss still prevalent
* Ambient temperature probe generating
unrealistic temperature data
* PEMS temperature data 1 1 degC below
SwRI measurement
* Temperature/humidity probe failure
* Replace temperature/humidity probe
* NO2 loss still prevalent with replacement
chiller from Sensors (8/22/06)
* NO2 span decreased during fist hour of
testing, then began to recover
* When NO2 decreased, NO increased -
possible conversion of NO2 to NO
* Problem with PEMS chillers - possible
deterioration of passivated coating
* Replaced chiller in PEMS 3 (replacement
#2)
* Tested PEMS, PEMS 3 - no NO2 loss
* Repeated baseline environmental testing on
PEMS 2 and 3
* Repeated temperature testing on PEMS 3
* Unit turned on and shut off after several
seconds when in the enclosure
* Unit eventually would not turn on when in the
enclosure
* 12-volt power cable in the enclosure had a
bad connection
* Repaired cable connection
SwRI Report 03.12024.06
A-4
-------
9/11/2006
9/14/2006
9/14/2006
9/15/2006
9/19/2006
9/20/2006
4
6
4
4
6
2
Manifold
RH/Temp
failure
Manifold
RH/Temp
failure and
communication
error
NDUV failure
1065 NO2
linearity
failure
Manifold
RH/Temp
failure
Drain pump
failure
Test
Test
Test
Audit
Test
Test
Symptoms
Problem
Solution
Symptoms
Problem
Solution
Symptoms
Problem
Solution
Symptoms
Problem
Solution
Symptoms
Problem
Solution
Symptoms
Problem
Solution
* Fault - "Manifold RH/Temp sensor not
responding"
* Manifold RH reading absent
* Fault cleared, then RH reading was erratic,
jumping between 60 and 100%
* Manifold RH/Temp sensor corroded
* Replaced manifold RH/Temp sensor
* Performed NO2 penetration check - passed
* Fault - "Manifold RH/Temp sensor not
responding"
* Manifold RH reading absent
* Manifold RH/Temp sensor wet
* Replaced manifold RH/Temp sensor
* New sensor would not
respond/communicate
* Reloaded firmware to reinitialize settings
* NO and NO2 readings lower than expected,
often zero
* NO and NO2 readings higher than expected
* NO and NO2 readings erratic and noisy
* NO and NO2 measurements have excessive
drift
* NO and NO2 gains high or saturated
* NDUV lamp malfunction (Carl Ensfield)
* NDUV gains were saturated (Louie Moret)
* Replaced NDUV
* Performed linearity and NO2 penetration
check - NO2 failed
* NO2 repeatedly failed 1065 linearity (low
intercept, high slope)
* Incorrect or outdated curve constants
* None taken
* Fault - "Manifold RH/Temp sensor not
responding"
* Manifold RH reading absent
* Manifold RH/Temp sensor wet
* Replaced manifold RH/Temp sensor
* Warning - "Low vacuum drain 2"
* Pressure #3 (Filter bowl drain) vacuum low
(near ambient pressure)
* Pressure #3 would not draw vacuum when
sample line disconnected
* Unit would not pass a leak check
* Drain pump not operating when checked
* Drain pump started to rotate, but would stop
under vacuum
* Replaced drain pump with PEMS 6 drain
pump
SwRI Report 03.12024.06
A-5
-------
9/20/2006
9/20/2006
9/21/2006
9/25/2006
9/27/2006
2
6
6
4
4,6
Drain pump
mounting
bracket broken
EFM data not
reported
in post-
processed file
(completion of
Fix from 9/20)
Vehicle
Interface
not responding
NDUV failure
Manifold
RH/Temp
failure
Test
Test
Audit
Test
Test
Test
Symptoms
Problem
Solution
Symptoms
Problem
Solution
Notes
Symptoms
Problem
Solution
Symptoms
Problem
Solution
Symptoms
Problem
Solution
* Aluminum drain pump mounting bracket
broken
* Mounting bracket cracked
* Welded bracket and reinstalled in PEMS
* EFM flow data was not reported in 4 SS post
processed files
* EFM flow data was in data section of the
.xml files
* EFM description data missing from header
section of the .xml files
* Added EFM description data to the header
section of the .xml file
* Reprocessed the data files
* Installed new drain pump in PEMS 6
* Performed leak check
* Performed NO2 penetration check (new RH
sensor and drain pump)
* Fault - "Vehicle Interface not responding"
* Could not link to CAN bus
* Dearborn Group adapter not functioning
* Replaced Dearborn Group adapter
* NO and NO2 readings lower than expected,
often zero
* NO and NO2 readings higher than expected
* NO and NO2 readings erratic and noisy
* NO and NO2 measurements have excessive
drift
* NO and NO2 gains high or saturated
* NDUV lamp malfunction (Carl Ensfield)
* Replaced NDUV
* Performed linearity and NO2 penetration
check - passed
* Fault - "Manifold RH/Temp sensor not
responding"
* Manifold RH reading absent, not updating, or
erratic
* Manifold RH/Temp sensor wet
* Manifold not sealed properly - air/water
leaking past sensor
* Disassembled new drain manifolds
* Resealed manifolds with Silicon
* Sealed manifolds around sensor connectors
* Performed leak checks on the manifolds -
passed
* Performed NO2 penetration checks - passed
SwRI Report 03.12024.06
A-6
-------
9/28/06 -
10/3/06
10/5/2006
10/11/2006
10/18/2006
3" EFMs
3
2
2
3" EFMs failed
1065
linearity
Vibration
failures
Problem 1
Problem 2
EFM data not
reported
in post-
processed file
FID battery
backed ram
corrupt
Audit
Test
Test
Test
Symptoms
Problem
Solution
Notes
Symptoms
Problem
Solution
Symptoms
Problem
Solution
Symptoms
Problem
Solution
Symptoms
Problem
Solution
* 2 3-inch EFMs failed 1065 linearity check
compared to SwRI flow stand
* EFMs failed slope
* One EFM close (0.95), one bad (.90)
* Incorrect or outdated curve data
* Replaced one bad EFM
* Entered new linearization data for both flow
meters
* Performed linearity checks - both EFMs
passed
* PEMS operated for 4 hours of side-to-side
horizontal vibration
* PEMS operated for 10 minutes of front-to-
back horizontal vibration
* During first hour of vib. testing, enclosure
heated line warning, low temp
* During first hour of vib. testing, enclosure
fans stopped operating
* 12-volt power connection to enclosure failed
on PEMS (soldered wires broke)
* Temporary solution - external 12-volt power
supply to enclosure
* During first 10-min. of side-to-side vib., FID
high temp fault
* FID fault - FID temperature exceeded limits -
reading 219 degF
* Unit was shut down - took approximately 3
minutes to restart
* Could not communicate with the compact
flash card
* Can not log onto unit without flash card
communication
* Some data from previous tests were lost
* Flash card ribbon cable failure (not known
prior to sending unit to Sensors)
* Unit sent to Sensors for repair
* Installed new flash card ribbon cable
* EFM flow data was not reported in EFM
linearity test
* EFM flow data was in data section of the
.xml files
* EFM description data missing from header
section of the .xml files
* Added EFM description data to the header
section of the .xml file
* Reprocessed the data files
* FID fault - "FID battery backed ram corrupt"
* Fault would not clear
* Unknown
* Restarting unit has cleared this fault in the
past
* After restart, lost communication with the
unit (see Communication error 10/18/06)
SwRI Report 03.12024.06
A-7
-------
10/18/2006
10/19/2006
2
3
Communication
and/or
processor
failure
Bulk Current
Injection
(BCI) FID
Failure
Test
Test
Symptoms
Problem
Solution
Symptoms
Problem
Solution
* Could not log onto PEMS - "Connection
Failed"
* CPU Status LED on, but not blinking
* Could not turn unit off with the power button
on the PEMS (had to unplug unit)
* Restarted unit several times, still no
communication
* Installed new flash card, still no
communication
* Corrupt flash card (Sensors Inc.)
* Sent PEMS 2 to Sensors Inc. for repair
* Sensors Inc. replaced flash card,
communication restored
* BCI to the 12-volt PEMS power cable
(Standard J 111 3-4 Class B, Region 2)
* 60 milliamps, 26-46 MHz
* Caused erratic readings for the FID oven
temperature, FID fuel pressure, and FID
internal
reference pressure
* Fault - "FID oven temperature out of limits"
* Fault - "FID internal reference pressure out
of limits"
* Warning - "Warming Up"
* High FID oven temperature reading caused
the FID to shutdown
* PEMS susceptibility to BCI, FID susceptibility
to BCI
* Replaced PEMS 3 with PEMS 7
* Sent PEMS 3 to Sensors for repair
* PEMS 7 did not turn off the FID when tested
over similar conditions
* PEMS 7 reported similar faults and
warnings, but to a lesser extent than PEMS 3
SwRI Report 03.12024.06
A-8
-------
10/23/2006
10/23/2006
7
7
Bulk Current
Injection
(BCI) Failures
EFM data not
reported
in post-
processed file
Test
Test
Testl
Solution
Test 2
Solution
Test3
Solution
Test 5a
Solution
Test 7
Solution
Test 9
Solution
Symptoms
Problem
Solution
* BCI to power supply cable
* Lost communication with EFM
* Turned EFM off and back on,
communication restored
* BCI to power supply cable
* Fault - "FID internal ref pressure out of
limits" (96 MHz)
* Cleared fault
* BCI to Aux 1 cable
* Fault - "FID oven temp out of range" (91
MHz)
* Warning - "Warming Up" (on briefly)
* Cleared fault
* BCI to Ethernet cable
* Fault - "FID internal ref pressure out of
limits" (80 MHz)
* Lost communication with the PEMS (96
MHz)
* Cleared fault
* Log back onto PEMS, continued test
* Fault - "FID internal ref pressure out of
limits" (96 MHz)
* Fault - "FID oven temp out of range" (360
MHz)
* Faults would not clear while BCI was active
* Stopped BCI, cleared faults and proceeded
with testing
* BCI to EFM cable at EFM box
* EFM not responding (prior to 170 MHz)
* Restarted PEMS and software to restore
communication with the EFM
* EFM flow data was not reported in BCI test
* EFM flow data was in data section of the
.xml files
* EFM description data missing from header
section of the .xml files
* Added EFM description data to the header
section of the .xml file
* Reprocessed the data files
SwRI Report 03.12024.06
A-9
-------
10/25/2006
11/9/2006
7
6
Radiated
Immunity
(Rl) Failures
Manifold
RH/Temp
failure
Test
CATSS
Testing
Testl
Solution
Test 3
Solution
Test 4
Solution
Symptoms
Problem
Solution
* Warning - "Warming Up" (on briefly) (42
MHz, 25 V/m)
* Warning - "Warming Up" (on briefly) (74
MHz, 25 V/m)
* FID oven temp reading 203 (74 MHz, 25
V/m)
* Fault - "FID oven temp out of range" (76
MHz, 25 V/m) - cleared fault
* Fault - "FID oven temp out of range" (78
MHz, 20 V/m) - cleared fault
* Fault - "FID oven temp out of range" (82
MHz, 15 V/m) -cleared fault
* Fault - "FID internal ref pressure out of
limits" (86 MHz, 15 V/m) - cleared fault
* Fault - "FID oven temp out of range" (128
MHz, ? V/m) - cleared fault
* Fault - "FID internal ref pressure out of
limits" (128 MHz, V/m) - cleared fault
* Warning - "Warming Up" (on briefly) (128
MHz, ? V/m)
* Lost communication with the PEMS (148
MHz, ? V/m) - logged back onto PEMS
* FID shut down, oven temp reading 213 (164
MHz, ? V/m)
* Restart test using lower intensity (10 V/m)
* FID oven temp reading 205 (164 MHz, 10
V/m)
* FID shut down, oven temp reading 215 (168
MHz, 10 V/m) - restarted FID
* Can not operate FID between 168 and 178
MHz at 10 V/m
* Continued testing at 178 MHz and 10 V/m
* FID shut down, oven temp reading 232 (260
MHz, 10 V/m) - restarted PEMS
* Chiller reading 20 degC when PEMS was
restarted - restarted PEMS
* Restarted PEMS and continued testing
* Fault - "Manifold RH/Temp sensor not
responding"
* Manifold RH/Temp sensor wet
* Manifold not sealed properly - air/water
leaking past sensor
* Fault cleared
* Checked RH reading - OK
* Continued testing
SwRI Report 03.12024.06
A-10
-------
11/16/2006
11/16/2006
1/30/2006
1/31/2006
6
3
5
5
Manifold
RH/Temp
failure
Vibration
Failures
Sample Line
Failure
Sample Line
Failure
INT 40-
Point
Testing
Vibration
Testing
CATC15
40-Point
Testing
CATC15
40-Point
Testing
Symptoms
Problem
Solution
Test3
Solution
Test 6
Solution
Test?
Solution
Test 9
Solution
Symptoms
Problem
Solution
Symptoms
Problem
Solution
* Fault - "Manifold RH/Temp sensor not
responding"
* Manifold RH reading absent, not updating, or
erratic
* Manifold RH/Temp sensor wet
* Manifold not sealed properly - air/water
leaking past sensor
* Removed drain manifold with RH sensor
* Performed leak check on new replacement
manifold - Passed
* Performed NO2 penetration checks -
Passed
* Fault - "FID internal ref pressure out of
limits"
* FID bottle low- broken FID bottle quick
connect
* Replaced FID bottle and quick connect
* Fault - "FID gas flow is too high or too low"
* Lost control of the FID
* Turned FID off overnight
* FID slow to come up to operating
temperature
* FID would not span - FID oven at 191 degC
* None taken, continued testing with no THC
measurement
* Heard a "pop" from the PEMS - FID
responding and reporting THC values
* Fault - "FID internal ref pressure out of
limits"
* FID bottle low- broken FID bottle quick
connect
* Replaced FID bottle and quick connect
* Fault - "High vacuum on drains 1 and 2"
* Fault occurred after PEMS 5 was set to
sample ambient air after being in standby
overnight
* Heated sample line blocked
* Cause of blockage unknown
* Replaced sample line
* Fault - "High vacuum on drains 1 and 2"
* Fault occurred after PEMS 5 was set to
sample ambient air after being in standby
overnight
* Heated sample line blocked
* Cause of blockage unknown
* Replaced sample line
SwRI Report 03.12024.06
A-ll
-------
APPENDIX B
BRAKE-SPECIFIC EMISSION CALCULATIONS FOR NOX, CO, AND NMHC
SwRI Report 03.12024.06
-------
The following sections detail the calculation formulas and the required input constants. For
Methods #1 and #2 the conversion ofexhaust flow rate from SCFM to mol/s is:
n.(SCFM)* IX-]*-
. (moT\ = 353l467(ft3) 60
mm
293.15(K)* 8.314472
J
Brake-Specific MX Concentration for Method #1
Input constants:
At = 1 (sec)
MNO = 46.0055MM
N°2 {molj
i V
z
(xNO2 (ppm) + xNO, (ppm)) * 10~6 * n\ —} * At
* f
Speedl(rpm)*Tl(N-m)*2*3.l4l59*At
60*1000*3600
Brake-Specific CO Concentration for Method #1
Input constants:
At = I (sec)
= 28.010lM-l
L co
mol)
eco(g/kW-hr) = -
S
(jcCO,(%))*10-2 *«/—V At
I s )
* 2* 3.14159*
60*1000*3600
SwRI Report 03.12024.06
B-l
-------
Brake-Specific NMHC Concentration for Method #1
Input constants:
A? = 1 (sec)
M NMHC *
6NMHc(g I kW • hr) = •
(xNMHC, (ppm)) * 1 (T6 * nt \ — } * A?
1\
I
Speedi(rpm)*Ti(N-m)*2*3.14159*At
60*1000*3600
Brake- Specific NOx Concentration for Method #2
Input constants:
w fuel =0.869 Mass fraction of carbon in the fuel.
At = 1 (sec )
(
M =12.0107
\rnol
NO~
= 46.0055
eNO(glkW-hr) = -
Wfuel
M *
1V1N02
i=\
(xNO2 (ppm)+xNO, (ppm))* 10~6 * ni — * A/
nt ( — ] * \xNMHCt (ppm) * 1 0~6 + (xCO, (%) + xCO2i (%))* 1 0 2 ]* A?
BSFCI g
\kW-hr
Brake-Specific CO Concentration for Method #2
SwRI Report 03.12024.06
B-2
-------
Input constants:
w 'fuel =0.869 Mass fraction of carbon in the fuel.
A? = 1 (sec )
Mc =12.0107
( mol
Mco =28.0101
\rnol
eco(glkW-hr} =
W
fue,
n,\— ] * [xNMHC, (ppm)* 10~6 + (xCO,(%) + JcCO2j (%))* 10~2]* A?
BSFC,\ g
\kW-hr
Brake-Specific NMHC Concentration for Method #2
Input constants:
wfud =0.869 Massfractionofcarbon inthefuel.
A? = 1 (sec)
M^ = 12.01071-^-1
\rnol;
MNMHC *
8NMHc(g I kW • hr) = •
i=\
(xNMHCt (ppm)) * 10"6 * nl — ] * A?
n, (— ] * [jc7VM//C, (ppm) * 10~6 + (jcCO;. (%) + JcCO2j (%)) * 1Q-2 ] *,
BSFCI g
\kW-hr
Brake-Specific NOY Concentration for Method #3
Input constants:
SwRI Report 03.12024.06
B-3
-------
w fuel =0.869 Mass fraction of carbon in the fuel.
NO~
( K \
Mc =12.0107 -2—
\rnol )
= 46.0055| -£-|
\rnol )
At = 1 (sec )
Example mass fuel mass rate calculatio n :
(v\ ( 1 \ ( v~\
m fuel. - = Fuelrate , | | * 851.0| -2-
sec
L
32 Wfuel ^^
A / ^-^
eNO(glkW-hr} = -
(xNO2i (ppm) + xNO, (ppm)) * 10~6 * mfuelj (
g_
s
xNMHC, (ppm) * 10~6 + (xCO, (%) + xCO2i (%)) * 10"2
IV
I
Speedi(rpm)*Ti(N-m)*2*3.\4\59*kt
60*1000*3600
Brake-Specific CO Concentration for Method #3
Input constants:
w'fud =0.869 Massfractionof carbon inthefuel.
Mr =12.010
1-*-}
\rnol)
Mco = 28.01 OlM-1
mol)
At = 1 (sec)
Examplemass fuel mass rate calculation :
m
L
«/»/. - = Fuelrate, — * 851.0 -2.
' V ; \ QPP / \ /
. J / \ scv^ y \ Jj ,
A/f ^117
ivico wfml
^"
6co(glkW -
'\ *
JC7VM//C, (/?p»i) * 1Q-6 + (xCO, (%) + xCO2i (%)) * 10~2
7V
I
60*1000*3600
SwRI Report 03.12024.06
B-4
-------
Brake-Specific NMHC Concentration for Method #3
Input constants:
^ fuel = 0- 869 Mass fractionof carbon in the fuel.
( K\
Mc = 12.0107 ^~
\rnol)
MtaaK= 13.87538/^-1
\rnol)
At = 1 (sec)
Examplemass fuel mass rate calculation :
r "^ ( I ~\ f?^
— *851.d-£
* w
NMHC w fuel
CNMHc(g I kW • hr) = •
IV
I
(xNMHCt (ppm)) *
g_
s
xNMHC, (ppm) * 10"6 + (xCO, (%) + xCO2j (%)) * 10~2
2*3.14159* A/'
60*1000*3600
SwRI Report 03.12024.06
B-5
-------
APPENDIX C
CRYSTAL BALL OUTPUT FILE DESCRIPTIONS
SwRI Report 03.12024.06
-------
EXTRACT DATA FILES
1.0 Simulation Variables
The simulation variables listed in Table 1 were extracted at the completion of the Monte
Carlo simulation run for each reference NTE event. Crystal Ball classifies variables into two
categories: assumptions and forecasts. Assumptions are the estimated inputs into the simulation
model such as the variability indices used to sample each error surface. All assumption variables
in this study are identified by an "ic" at the beginning of the variable name. Forecasts are values
calculated by a forecast formula in the spreadsheet cells. Examples of forecast variables used in
this study are "Full MC Delta NOX Method 1" and "Validation MC Delta CO Method 2".
TABLE 1. SIMULATION VARIABLES
Variable Name
001_DeNOx (g/kW-hr), Method 1_TA_DC
002_Valid DeNOx (g/kW-hr), Method 1_TA_DC
003_DeNOx (g/kW-hr), Method 2_TA_DC
004_Valid DeNOx (g/kW-hr), Method 2_TA_DC
005_DeNOx (g/kW-hr), Method 3_TA_DC
006_Valid DeNOx (g/kW-hr), Method 3_TA_DC
007_DeCO (g/kW-hr), Method 1_TA_DC
008_Valid DeCO (g/kW-hr), Method 1_TA_DC
009_DeCO (g/kW-hr), Method 2_TA_DC
010_Valid DeCO (g/kW-hr), Method 2_TA_DC
Oil DeCO (g/kW-hr), Method 3 TA DC
Description
Full MC Delta NOx Method 1
Time Alignment and Periodic Drift
Check
Validation MC Delta NOx Method
1 Time Alignment and Periodic
Drift Check
Full MC Delta NOx Method 2
Time Alignment and Periodic Drift
Check
Validation MC Delta NOx Method
2 Time Alignment and Periodic
Drift Check
Full MC Delta NOx Method 3
Time Alignment and Periodic Drift
Check
Validation MC Delta NOx Method
3 Time Alignment and Periodic
Drift Check
Full MC Delta CO Method 1 Time
Alignment and Periodic Drift
Check
Validation MC Delta CO Method 1
Time Alignment and Periodic Drift
Check
Full MC Delta CO Method 2 Time
Alignment and Periodic Drift
Check
Validation MC Delta CO Method 2
Time Alignment and Periodic Drift
Check
Full MC Delta CO Method 3 Time
SwRI Report 03.12024.06
C-l
-------
012_Valid DeCO (g/kW-hr), Method 3_TA_DC
013_DeNMHC (g/kW-hr), Method 1_DC
014_Valid DeNMHC (g/kW-hr), Method 1_DC
015_DeNMHC (g/kW-hr), Method 2 DC
016_Valid DeNMHC (g/kW-hr), Method 2 DC
017_DeNMHC (g/kW-hr), Method 3_DC
018_Valid DeNMHC (g/kW-hr), Method 3_DC
019_DeNOx (g/kW-hr), Method 1_DC
020_Valid DeNOx (g/kW-hr), Method 1_DC
021_DeNOx (g/kW-hr), Method 2_DC
022_Valid DeNOx (g/kW-hr), Method 2_DC
023_DeNOx (g/kW-hr), Method 3_DC
024_Valid DeNOx (g/kW-hr), Method 3_DC
025_DeCO (g/kW-hr), Method 1_DC
026_Valid DeCO (g/kW-hr), Method 1_DC
027_DeCO (g/kW-hr), Method 2_DC
028_Valid DeCO (g/kW-hr), Method 2_DC
029_DeCO (g/kW-hr), Method 3_DC
030_Valid DeCO (g/kW-hr), Method 3_DC
031_DeNOx (g/hp-hr), Method 1_TA_DC
032_Valid DeNOx (g/hp-hr), Method 1_TA_DC
Alignment and Periodic Drift
Check
Validation MC Delta CO Method 3
Time Alignment and Periodic Drift
Check
Full MC Delta NMHC Method 1
Periodic Drift Check
Validation MC Delta NMHC
Method 1 Periodic Drift Check
Full MC Delta NMHC Method 2
Periodic Drift Check
Validation MC Delta NMHC
Method 2 Periodic Drift Check
Full MC Delta NMHC Method 3
Periodic Drift Check
Validation MC Delta NMHC
Method 3 Periodic Drift Check
Full MC Delta NOx Method 1
Periodic Drift Check
Validation MC Delta NOx Method
1 Periodic Drift Check
Full MC Delta NOx Method 2
Periodic Drift Check
Validation MC Delta NOx Method
2 Periodic Drift Check
Full MC Delta NOx Method 3
Periodic Drift Check
Validation MC Delta NOx Method
3 Periodic Drift Check
Full MC Delta CO Method 1
Periodic Drift Check
Validation MC Delta CO Method 1
Periodic Drift Check
Full MC Delta CO Method 2
Periodic Drift Check
Validation MC Delta CO Method 2
Periodic Drift Check
Full MC Delta CO Method 3
Periodic Drift Check
Validation MC Delta CO Method 3
Periodic Drift Check
Full MC Delta NOx Method 1
Time Alignment and Periodic Drift
Check
Validation MC Delta NOx Method
1 Time Alignment and Periodic
SwRI Report 03.12024.06
C-2
-------
033_DeNOx (g/hp-hr), Method 2_TA_DC
034_Valid DeNOx (g/hp-hr), Method 2_TA_DC
035_DeNOx (g/hp-hr), Method 3_TA_DC
036_Valid DeNOx (g/hp-hr), Method 3_TA_DC
037_DeCO (g/hp-hr), Method 1_TA_DC
038_ValidDeCO (g/hp-hr), Method 1_TA_DC
039_DeCO (g/hp-hr), Method 2_TA_DC
040_Valid DeCO (g/hp-hr), Method 2_TA_DC
041_DeCO (g/hp-hr), Method 3_TA_DC
042_Valid DeCO (g/hp-hr), Method 3_TA_DC
043_DeNMHC (g/hp-hr), Method 1_DC
044_Valid DeNMHC (g/hp-hr), Method 1_DC
045_DeNMHC (g/hp-hr), Method 2_DC
046_Valid DeNMHC (g/hp-hr), Method 2_DC
047_DeNMHC (g/hp-hr), Method 3_DC
048_Valid DeNMHC (g/hp-hr), Method 3_DC
049_DeNOx (g/hp-hr), Method 1_DC
050 Valid DeNOx (g/hp-hr), Method 1 DC
Drift Check
Full MC Delta NOx Method 2
Time Alignment and Periodic Drift
Check
Validation MC Delta NOx Method
2 Time Alignment and Periodic
Drift Check
Full MC Delta NOx Method 3
Time Alignment and Periodic Drift
Check
Validation MC Delta NOx Method
3 Time Alignment and Periodic
Drift Check
Full MC Delta CO Method 1 Time
Alignment and Periodic Drift
Check
Validation MC Delta CO Method 1
Time Alignment and Periodic Drift
Check
Full MC Delta CO Method 2 Time
Alignment and Periodic Drift
Check
Validation MC Delta CO Method 2
Time Alignment and Periodic Drift
Check
Full MC Delta CO Method 3 Time
Alignment and Periodic Drift
Check
Validation MC Delta CO Method 3
Time Alignment and Periodic Drift
Check
Full MC Delta NMHC Method 1
Periodic Drift Check
Validation MC Delta NMHC
Method 1 Periodic Drift Check
Full MC Delta NMHC Method 2
Periodic Drift Check
Validation MC Delta NMHC
Method 2 Periodic Drift Check
Full MC Delta NMHC Method 3
Periodic Drift Check
Validation MC Delta NMHC
Method 3 Periodic Drift Check
Full MC Delta NOx Method 1
Periodic Drift Check
Validation MC Delta NOx Method
SwRI Report 03.12024.06
C-3
-------
051_DeNOx (g/hp-hr), Method 2 DC
052_Valid DeNOx (g/hp-hr), Method 2 DC
053_DeNOx (g/hp-hr), Method 3_DC
054_Valid DeNOx (g/hp-hr), Method 3_DC
055_DeCO (g/hp-hr), Method 1_DC
056_Valid DeCO (g/hp-hr), Method 1_DC
057_DeCO (g/hp-hr), Method 2_DC
058_Valid DeCO (g/hp-hr), Method 2_DC
059_DeCO (g/hp-hr), Method 3_DC
060_Valid DeCO (g/hp-hr), Method 3_DC
061_DeNOx (g/kW-hr), Method 1_TA
062_DeCO (g/kW-hr), Method 1_TA
063 DeNMHC (g/kW-hr), Method 1
064_DeNOx (g/kW-hr), Method 2_TA
065_DeCO (g/kW-hr), Method 2_TA
066_DeNMHC (g/kW-hr), Method 2
067_DeNOx (g/kW-hr), Method 3_TA
068_DeCO (g/kW-hr), Method 3_TA
069 DeNMHC (g/kW-hr), Method 3
070 DeNOx (g/kW-hr), Method 1
071 DeCO (g/kW-hr), Method 1
072 DeNOx (g/kW-hr), Method 2
073 DeCO (g/kW-hr), Method 2
074 DeNOx (g/kW-hr), Method 3
075 DeCO (g/kW-hr), Method 3
076_DeNOx (g/hp-hr), Method 1_TA
1 Periodic Drift Check
Full MC Delta NOx Method 2
Periodic Drift Check
Validation MC Delta NOx Method
2 Periodic Drift Check
Full MC Delta NOx Method 3
Periodic Drift Check
Validation MC Delta NOx Method
3 Periodic Drift Check
Full MC Delta CO Method 1
Periodic Drift Check
Validation MC Delta CO Method 1
Periodic Drift Check
Full MC Delta CO Method 2
Periodic Drift Check
Validation MC Delta CO Method 2
Periodic Drift Check
Full MC Delta CO Method 3
Periodic Drift Check
Validation MC Delta CO Method 3
Periodic Drift Check
Full MC Delta NOx Method 1
Time Alignment
Full MC Delta CO Method 1 Time
Alignment
Full MC Delta NMHC Method 1
Full MC Delta NOx Method 2
Time Alignment
Full MC Delta CO Method 2 Time
Alignment
Full MC Delta NMHC Method 2
Time Alignment
Full MC Delta NOx Method 3
Time Alignment
Full MC Delta CO Method 3 Time
Alignment
Full MC Delta NMHC Method 3
Full MC Delta NOx Method 1
Full MC Delta CO Method 1
Full MC Delta NOx Method 2
Full MC Delta CO Method 2
Full MC Delta NOx Method 3
Full MC Delta CO Method 3
Full MC Delta NOx Method 1
Time Alignment
SwRI Report 03.12024.06
C-4
-------
077_DeCO (g/hp-hr), Method 1_TA
078 DeNMHC (g/hp-hr), Method 1
079_DeNOx (g/hp-hr), Method 2_TA
080_DeCO (g/hp-hr), Method 2_TA
081 DeNMHC (g/hp-hr), Method 2
082_DeNOx (g/hp-hr), Method 3_TA
083_DeCO (g/hp-hr), Method 3_TA
084 DeNMHC (g/hp-hr), Method 3
085 DeNOx (g/hp-hr), Method 1
086 DeCO (g/hp-hr), Method 1
087 DeNOx (g/hp-hr), Method 2
088 DeCO (g/hp-hr), Method 2
089 DeNOx (g/hp-hr), Method 3
090 DeCO (g/hp-hr), Method 3
091_Valid DeNOx (g/kW-hr), Method 1_TA
092_Valid DeCO (g/kW-hr), Method 1_TA
093_Valid DeNMHC (g/kW-hr), Method 1
094_Valid DeNOx (g/kW-hr), Method 2_TA
095_Valid DeCO (g/kW-hr), Method 2_TA
096_Valid DeNMHC (g/kW-hr), Method 2
097_Valid DeNOx (g/kW-hr), Method 3_TA
098_Valid DeCO (g/kW-hr), Method 3_TA
099_Valid DeNMHC (g/kW-hr), Method 3
100_Valid DeNOx (g/kW-hr), Method 1
101 Valid DeCO (g/kW-hr), Method 1
102_Valid DeNOx (g/kW-hr), Method 2
103 Valid DeCO (g/kW-hr), Method 2
104_Valid DeNOx (g/kW-hr), Method 3
Full MC Delta CO Method 1 Time
Alignment
Full MC Delta NMHC Method 1
Full MC Delta NOx Method 2
Time Alignment
Full MC Delta CO Method 2 Time
Alignment
Full MC Delta NMHC Method 2
Full MC Delta NOx Method 3
Time Alignment
Full MC Delta CO Method 3 Time
Alignment
Full MC Delta NMHC Method 3
Full MC Delta NOx Method 1
Full MC Delta CO Method 1
Full MC Delta NOx Method 2
Full MC Delta CO Method 2
Full MC Delta NOx Method 3
Full MC Delta CO Method 3
Validation MC Delta NOx Method
1 Time Alignment
Validation MC Delta CO Method 1
Time Alignment
Validation MC Delta NMHC
Method 1
Validation MC Delta NOx Method
2 Time Alignment
Validation MC Delta CO Method 2
Time Alignment
Validation MC Delta NMHC
Method 2
Validation MC Delta NOx Method
3 Time Alignment
Validation MC Delta CO Method 3
Time Alignment
Validation MC Delta NMHC
Method 3
Validation MC Delta NOx Method
1
Validation MC Delta CO Method 1
Validation MC Delta NOx Method
2
Validation MC Delta CO Method 2
Validation MC Delta NOx Method
3
SwRI Report 03.12024.06
C-5
-------
105 Valid DeCO (g/kW-hr), Method 3
106_Valid DeNOx (g/hp-hr), Method 1_TA
107_Valid DeCO (g/hp-hr), Method 1_TA
108_Valid DeNMHC (g/hp-hr), Method 1
109_Valid DeNOx (g/hp-hr), Method 2_TA
1 10_Valid DeCO (g/hp-hr), Method 2_TA
1 1 l_Valid DeNMHC (g/hp-hr), Method 2
1 12_Valid DeNOx (g/hp-hr), Method 3_TA
1 13_Valid DeCO (g/hp-hr), Method 3_TA
1 14_Valid DeNMHC (g/hp-hr), Method 3
1 15_Valid DeNOx (g/hp-hr), Method 1
116 Valid DeCO (g/hp-hr), Method 1
1 17_Valid DeNOx (g/hp-hr), Method 2
118 Valid DeCO (g/hp-hr), Method 2
1 19_Valid DeNOx (g/hp-hr), Method 3
120 Valid DeCO (g/hp-hr), Method 3
121_eNOx (g/kW-hr), Method 1 Mode 2_TA
122_eCO (g/kW-hr), Method 1 Mode 2_TA
123_eNMHC (g/kW-hr), Method 1 Mode 2
124_eNOx (g/kW-hr), Method 2 Mode 2_TA
125_eCO (g/kW-hr), Method 2 Mode 2_TA
126_eNMHC (g/kW-hr), Method 2 Mode 2
127_eNOx (g/kW-hr), Method 3 Mode 2_TA
128_eCO (g/kW-hr), Method 3 Mode 2_TA
129_eNMHC (g/kW-hr), Method 3 Mode 2
Validation MC Delta CO Method 3
Validation MC Delta NOx Method
1 Time Alignment
Validation MC Delta CO Method 1
Time Alignment
Validation MC Delta NMHC
Method 1
Validation MC Delta NOx Method
2 Time Alignment
Validation MC Delta CO Method 2
Time Alignment
Validation MC Delta NMHC
Method 2 Time Alignment
Validation MC Delta NOx Method
3 Time Alignment
Validation MC Delta CO Method 3
Time Alignment
Validation MC Delta NMHC
Method 3
Validation MC Delta NOx Method
1
Validation MC Delta CO Method 1
Validation MC Delta NOx Method
2
Validation MC Delta CO Method 2
Validation MC Delta NOx Method
3
Validation MC Delta CO Method 3
Full MC BSNOx "with errors"
Method 1 and Time Alignment
Full MC BSCO "with errors"
Method 1 and Time Alignment
Full MC BSNMHC "with errors"
Method 1
Full MC BSNOx "with errors"
Method 2 and Time Alignment
Full MC BSCO "with errors"
Method 2 and Time Alignment
Full MC BSNMHC "with errors"
Method 2
Full MC BSNOx "with errors"
Method 3 and Time Alignment
Full MC BSCO "with errors"
Method 3 and Time Alignment
Full MC BSNMHC "with errors"
Method 3
SwRI Report 03.12024.06
C-6
-------
130_eNOx (g/kW-hr), Method 1 Mode 2
131_eCO (g/kW-hr), Method 1 Mode 2
132_eNOx (g/kW-hr), Method 2 Mode 2
133_eCO (g/kW-hr), Method 2 Mode 2
134_eNOx (g/kW-hr), Method 3 Mode 2
135_eCO (g/kW-hr), Method 3 Mode 2
136_eNOx (g/kW-hr), Method 1 Mode 1
137_eCO (g/kW-hr), Method 1 Mode 1
138_eNMHC (g/kW-hr), Method 1 Mode 1
139_eNOx (g/kW-hr), Method 2 Mode 1
140_eCO (g/kW-hr), Method 2 Mode 1
141_eNMHC (g/kW-hr), Method 2 Mode 1
142_eNOx (g/kW-hr), Method 3 Mode 1
143_eCO (g/kW-hr), Method 3 Mode 1
144_eNMHC (g/kW-hr), Method 3 Mode 1
145 eNOx (g/kW-hr), Method 1 Mode 0
146 eCO (g/kW-hr), Method 1 Mode 0
147_eNMHC (g/kW-hr), Method 1 Mode 0
148 eNOx (g/kW-hr), Method 2 Mode 0
149 eCO (g/kW-hr), Method 2 Mode 0
150_eNMHC (g/kW-hr), Method 2 Mode 0
151 eNOx (g/kW-hr), Method 3 Mode 0
152 eCO (g/kW-hr), Method 3 Mode 0
153_eNMHC (g/kW-hr), Method 3 Mode 0
154_eNOx (g/kW-hr), Method 1 Reject Flag
155 eCO (g/kW-hr), Method 1 Reject Flag
Full MC BSNOx "with errors"
Method 1
Full MC BSCO "with errors"
Method 1
Full MC BSNOx "with errors"
Method 2
Full MC BSCO "with errors"
Method 2
Full MC BSNOx "with errors"
Method 3
Full MC BSCO "with errors"
Method 3
Full MC BSNOx "with errors
except environmental" Method 1
Full MC BSCO "with errors except
environmental" Method 1
Full MC BSNMHC "with errors
except environmental" Method 1
Full MC BSNOx "with errors
except environmental" Method 2
Full MC BSCO "with errors except
environmental" Method 2
Full MC BSNMHC "with errors
except environmental" Method 2
Full MC BSNOx "with errors
except environmental" Method 3
Full MC BSCO "with errors except
environmental" Method 3
Full MC BSNMHC "with errors
except environmental" Method 3
Full MC BSNOx "ideal" Method 1
Full MC BSCO "ideal" Method 1
Full MC BSNMHC "ideal" Method
1
Full MC BSNOx "ideal" Method 2
Full MC BSCO "ideal" Method 2
Full MC BSNMHC "ideal" Method
2
Full MC BSNOx "ideal" Method 3
Full MC BSCO "ideal" Method 3
Full MC BSNMHC "ideal" Method
3
Full MC BSNOx Periodic Drift
Check Flag Method 1
Full MC BSCO Periodic Drift
SwRI Report 03.12024.06
C-7
-------
156_eNMHC (g/kW-hr), Method 1 Reject Flag
157_eNOx (g/kW-hr), Method 2 Reject Flag
158_eCO (g/kW-hr), Method 2 Reject Flag
159_eNMHC (g/kW-hr), Method 2 Reject Flag
160_eNOx (g/kW-hr), Method 3 Reject Flag
161_eCO (g/kW-hr), Method 3 Reject Flag
162_eNMHC (g/kW-hr), Method 3 Reject Flag
163 Valid eNOx (g/kW-hr), Method 1 Mode
2_TA
164_Valid eCO (g/kW-hr), Method 1 Mode 2_TA
165_Valid eNMHC (g/kW-hr), Method 1 Mode 2
166 Valid eNOx (g/kW-hr), Method 2 Mode
2_TA
167_Valid eCO (g/kW-hr), Method 2 Mode 2_TA
168_Valid eNMHC (g/kW-hr), Method 2 Mode 2
169 Valid eNOx (g/kW-hr), Method 3 Mode
2_TA
170_Valid eCO (g/kW-hr), Method 3 Mode 2_TA
171_Valid eNMHC (g/kW-hr), Method 3 Mode 2
172_Valid eNOx (g/kW-hr), Method 1 Mode 2
173_Valid eCO (g/kW-hr), Method 1 Mode 2
174_Valid eNOx (g/kW-hr), Method 2 Mode 2
175_Valid eCO (g/kW-hr), Method 2 Mode 2
176_Valid eNOx (g/kW-hr), Method 3 Mode 2
Check Flag Method 1
Full MC BSNMHC Periodic Drift
Check Flag Method 1
Full MC BSNOx Periodic Drift
Check Flag Method 2
Full MC BSCO Periodic Drift
Check Flag Method 2
Full MC BSNMHC Periodic Drift
Check Flag Method 2
Full MC BSNOx Periodic Drift
Check Flag Method 3
Full MC BSCO Periodic Drift
Check Flag Method 3
Full MC BSNMHC Periodic Drift
Check Flag Method 3
Validation MC BSNOx "with
errors" Method 1 and Time
Alignment
Validation MC BSCO "with errors"
Method 1 and Time Alignment
Validation MC BSNMHC "with
errors" Method 1
Validation MC BSNOx "with
errors" Method 2 and Time
Alignment
Validation MC BSCO "with errors"
Method 2 and Time Alignment
Validation MC BSNMHC "with
errors" Method 2
Validation MC BSNOx "with
errors" Method 3 and Time
Alignment
Validation MC BSCO "with errors"
Method 3 and Time Alignment
Validation MC BSNMHC "with
errors" Method 3
Validation MC BSNOx "with
errors" Method 1
Validation MC BSCO "with errors"
Method 1
Validation MC BSNOx "with
errors" Method 2
Validation MC BSCO "with errors"
Method 2
Validation MC BSNOx "with
errors" Method 3
SwRI Report 03.12024.06
-------
177_Valid eCO (g/kW-hr), Method 3 Mode 2
178_Valid eNOx (g/kW-hr), Method 1 Mode 1
179_Valid eCO (g/kW-hr), Method 1 Mode 1
180_Valid eNMHC (g/kW-hr), Method 1 Mode 1
181_Valid eNOx (g/kW-hr), Method 2 Mode 1
182_Valid eCO (g/kW-hr), Method 2 Mode 1
183_Valid eNMHC (g/kW-hr), Method 2 Mode 1
184_Valid eNOx (g/kW-hr), Method 3 Mode 1
185_Valid eCO (g/kW-hr), Method 3 Mode 1
186_Valid eNMHC (g/kW-hr), Method 3 Mode 1
187_Valid eNOx (g/kW-hr), Method 1 Reject Flag
188_Valid eCO (g/kW-hr), Method 1 Reject Flag
189 Valid eNMHC (g/kW-hr), Method 1 Reject
Flag
190_Valid eNOx (g/kW-hr), Method 2 Reject Flag
191_Valid eCO (g/kW-hr), Method 2 Reject Flag
192 Valid eNMHC (g/kW-hr), Method 2 Reject
Flag
193_Valid eNOx (g/kW-hr), Method 3 Reject Flag
194_Valid eCO (g/kW-hr), Method 3 Reject Flag
195 Valid eNMHC (g/kW-hr), Method 3 Reject
Flag
196_eNOx (g/hp-hr), Method 1 Mode 2_TA
Validation MC BSCO "with errors"
Method 3
Validation MC BSNOx "with
errors except environmental"
Method 1
Validation MC BSCO "with errors
except environmental" Method 1
Validation MC BSNMHC "with
errors except environmental"
Method 1
Validation MC BSNOx "with
errors except environmental"
Method 2
Validation MC BSCO "with errors
except environmental" Method 2
Validation MC BSNMHC "with
errors except environmental"
Method 2
Validation MC BSNOx "with
errors except environmental"
Method 3
Validation MC BSCO "with errors
except environmental" Method 3
Validation MC BSNMHC "with
errors except environmental"
Method 3
Validation MC BSNOx Periodic
Drift Check Flag Method 1
Validation MC BSCO Periodic
Drift Check Flag Method 1
Validation MC BSNMHC Periodic
Drift Check Flag Method 1
Validation MC BSNOx Periodic
Drift Check Flag Method 2
Validation MC BSCO Periodic
Drift Check Flag Method 2
Validation MC BSNMHC Periodic
Drift Check Flag Method 2
Validation MC BSNOx Periodic
Drift Check Flag Method 3
Validation MC BSCO Periodic
Drift Check Flag Method 3
Validation MC BSNMHC Periodic
Drift Check Flag Method 3
Full MC BSNOx "with errors"
Method 1 and Time Alignment
SwRI Report 03.12024.06
C-9
-------
197_eCO (g/hp-hr), Method 1 Mode 2_TA
198_eNMHC (g/hp-hr), Method 1 Mode 2
199_eNOx (g/hp-hr), Method 2 Mode 2_TA
200_eCO (g/hp-hr), Method 2 Mode 2_TA
201_eNMHC (g/hp-hr), Method 2 Mode 2
202_eNOx (g/hp-hr), Method 3 Mode 2_TA
203_eCO (g/hp-hr), Method 3 Mode 2_TA
204_eNMHC (g/hp-hr), Method 3 Mode 2
205_Valid eNOx (g/hp-hr), Method 1 Mode 2_TA
206_Valid eCO (g/hp-hr), Method 1 Mode 2_TA
207_Valid eNMHC (g/hp-hr), Method 1 Mode 2
208_Valid eNOx (g/hp-hr), Method 2 Mode 2_TA
209_Valid eCO (g/hp-hr), Method 2 Mode 2_TA
210_Valid eNMHC (g/hp-hr), Method 2 Mode 2
21 l_Valid eNOx (g/hp-hr), Method 3 Mode 2_TA
212_Valid eCO (g/hp-hr), Method 3 Mode 2_TA
213_Valid eNMHC (g/hp-hr), Method 3 Mode 2
01_ic_SS_NOx
02_ic_TR_NOx
05 ic Temperature NOx
Full MC BSCO "with errors"
Method 1 and Time Alignment
Full MC BSNMHC "with errors"
Method 1
Full MC BSNOx "with errors"
Method 2 and Time Alignment
Full MC BSCO "with errors"
Method 2 and Time Alignment
Full MC BSNMHC "with errors"
Method 2
Full MC BSNOx "with errors"
Method 3 and Time Alignment
Full MC BSCO "with errors"
Method 3 and Time Alignment
Full MC BSNMHC "with errors"
Method 3
Validation MC BSNOx "with
errors" Method 1 and Time
Alignment
Validation MC BSCO "with errors"
Method 1 and Time Alignment
Validation MC BSNMHC "with
errors" Method 1
Validation MC BSNOx "with
errors" Method 2 and Time
Alignment
Validation MC BSCO "with errors"
Method 2 and Time Alignment
Validation MC BSNMHC "with
errors" Method 2
Validation MC BSNOx "with
errors" Method 3 and Time
Alignment
Validation MC BSCO "with errors"
Method 3 and Time Alignment
Validation MC BSNMHC "with
errors" Method 3 and Time
Alignment
Random Sampling Variability
Index for SS NOx Error Surface
Random Sampling Variability
Index for Transient NOx Error
Surface
Random Sampling Variability
Index for NOx Temperature Error
Surface
SwRI Report 03.12024.06
C-10
-------
07_ic_SS_CO
10 ic Pressure CO
11 ic Temperature CO
13_ic_SS_NMHC
14_ic_TR_NMHC
1 6_ic_Pressure_NMHC
1 7_ic_Temperature_NMHC
1 9_ic_NMHC_Ambient
20_ic_SS_flow
21_ic_TR_Flowrate
22 ic Pulsation flow
23_ic_Swirl_flow
25 ic Radiation Exhaust Flow
27 ic Temperature Exhaust Flow
28 ic Pressure Exhaust Flow
29_ic_TR_Torque
30_ic_Torque_DOE
31 ic Torque Warm
32 ic Torque IP
34 ic Torque Interpolation
3 5_ic_Torque_Engine Manufacturers
Random Sampling Variability
Index for SS CO
Random Sampling Variability
Index for CO Pressure
Random Sampling Variability
Index for CO Temperature
Random Sampling Variability
Index for SS NMHC
Random Sampling Variability
Index for Transient NMHC
Random Sampling Variability
Index for NMHC Pressure
Random Sampling Variability
Index for NMHC Temperature
Random Sampling Variability
Index for Ambient NMHC
Random Sampling Variability
Index for SS Exhaust Flow
Random Sampling Variability
Index for Transient Exhaust Flow
Random Sampling Variability
Index for Exhaust Flow Pulsation
Random Sampling Variability
Index for Exhaust Flow Swirl
Random Sampling Variability
Index for Exhaust Flow EMI/RFI
Radiation
Random Sampling Variability
Index for Exhaust Flow
Temperature
Random Sampling Variability
Index for Exhaust Flow Pressure
Random Sampling Variability
Index for Dynamic Torque
Random Sampling Variability
Index for Torque Design of
Experiments Testing
Random Sampling Variability
Index for Torque Warm-up
Random Sampling Variability
Index for Torque Independent
Parameters Humidity and Fuel
Random Sampling Variability
Index for Torque Interpolation
Random Sampling Variability
Index for Torque Engine
SwRI Report 03.12024.06
C-ll
-------
36_ic_TR_BSFC
37_ic_BSFC_DOE
38_ic_BSFC_Warm
39_ic_BSFC_IP
41_ic_BSFC_Interpolation
42 ic BSFC Engine Manufacturers
43_ic_TR_Speed
44_ic_TR_Fuel Rate
45_ic_SS_CO2
46_ic_TR_CO2
49 ic Temperature CO2
5 1 ic NOx Time Alignment
52 ic CO Time Alignment
Manufacturers
Random Sampling Variability
Index for Dynamic BSFC
Random Sampling Variability
Index for BSFC Design of
Experiments
Random Sampling Variability
Index for BSFC Warm-up
Random Sampling Variability
Index for BSFC Independent
Parameters Humidity and Fuel
Random Sampling Variability
Index for BSFC Interpolation
Random Sampling Variability
Index for BSFC Engine
Manufacturers
Random Sampling Variability
Index for Dynamic Speed
Random Sampling Variability
Index for Dynamic Fuel Rate
Random Sampling Variability
Index for SS CO2
Random Sampling Variability
Index for Transient CO2
Random Sampling Variability
Index for CO2 Temperature
Random Sampling Variability
Index for NOx Time Alignment
Random Sampling Variability
Index for CO Time Alignment
2.0 Statistics
Descriptive statistics summarizing the values obtained during a single reference NTE event
simulation are provided in Table 2.
TABLE 2. DESCRIPTIVE STATISTICS FOR SIMULATION VARIABLES
Statistic
Trials
Mean
Median
Definition
Number of times the simulation was
repeated and not discarded due to periodic
drift
Arithmetic average
The value midway between the smallest
value and the largest value
SwRI Report 03.12024.06
C-12
-------
Mode
Standard Deviation
Variance
Skewness
Kurtosis
Coefficient of Variability
Minimum
Maximum
Range Width
Mean Standard Error
Filtered Values
Value that occurs most often
Measurement of variability of a
distribution. The square root of the
variance
The average of the squares of the
deviations of a number of values from their
mean
A measure of the degree of deviation of a
distribution from the norm of a symmetric
distribution
A measure of the degree of peakedness of a
distribution
Standard deviation/Mean
Smallest value
Largest value
Largest value - smallest value
Standard deviation of the distribution of
possible sample means
Number of trial discarded due to periodic
drift
3.0 Percentiles
Percentiles are the probability of achieving values below a particular percentage in the
following increments: 0, 5, 10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 95,
and 100%. Percentiles are computed for each of the simulations variables described in Section
1.1.
4.0 Sensitivity Data
Sensitivity data are provided by computing the rank correlation coefficient for all error
surfaces and all simulation variables. The EXTRACT data file contains the absolute value of
the rank correlation.
5.0 Trial Values
The value for all simulation variables is provided at each trial of the simulation.
REPORT FILES
1.0 Report Summary
This section includes the simulation start date and time, stop date and time, number of
trials run, sampling type (Monte Carlo), random seed used, and run statistics.
SwRI Report 03.12024.06
C-13
-------
2.0 Forecasts
Descriptive statistics, percentiles, and a frequency histogram are provided for forecast
variables 001_DeNOx (g/kW-hr), Method 1_TA_DC though 030_Valid DeCO (g/kW-hr),
Method 3_DC (see Table 1).
3.0 Assumptions
Descriptive statistics, percentiles, distribution parameters, and a distribution chart are
provided for assumption variables 01_ic_SS_NOx through 52_ic_CO_Time Alignment (see
Table 1).
4.0 Sensitivity Charts
Sensitivity charts are provided for forecast variables 001_DeNOx (g/kW-hr), Method
1_TA_DC though 030_Valid DeCO (g/kW-hr), Method 3_DC (see Table 1). Crystal Ball
calculates sensitivity by computing rank correlation coefficients between every assumption (error
surface) and forecast (BS emissions and delta BS emissions) while the simulation is running.
Positive rank correlations indicate that an increase in the assumption is associated with an
increase in the forecast. The larger the absolute value of the rank correlation the stronger the
relationship.
The sensitivity charts developed during the MC simulation are displayed as 'Contribution
to Variance" charts which are calculated by squaring the rank correlation coefficients for all
assumptions used in a particular forecast and then normalizing them to 100%. Figure 1 displays
a sensitivity chart for the delta NOX Method #1 with time alignment and periodic drift check
forecast. The assumption with the highest contribution to variance (in absolute value) is plotted
at the top of the chart. In this example, as you increase the torque warm-up there is a decrease in
the delta NOX Method #1 values. Only the top nine assumptions are plotted in the sensitivity
charts.
SwRI Report 03.12024.06 C-14
-------
Sensitivity 00 1 _DeNOx (g/kW-hr), Method 1_TA_DC
•40.0% -200%
0.0%
20.0%
31 _te_Tor<|ue_W«rm
01 _ic_SSJ*Dx
20_fc_SS_1tow
32JC_TwqueJP
3OJc_TocqueJWDe
51_te_NGx_I
02_te_TR_NOx
34Jc_Torque_lrt«rpololion
93%
FIGURE 1. SENSITIVITY CHART FOR DELTA NOX METHOD 1
SwRI Report 03.12024.06
C-15
-------
APPENDIX D
MONTE CARLO SPREADSHEET COMPUTATIONS
SwRI Report 03.12024.06
-------
1.0 DESCRIPTION OF ASSUMPTIONS
The following assumptions were made in running the Monte Carlo model:
• Only one reference NTE event can be run at a time through the Monte Carlo
simulation workbook.
• Uniform (1 second in duration) time steps are used in the reference NTE events.
• Standard format and engineering units for reference NTE data established for the
project are observed, and applied to the reference NTE event before the NTE event is
entered in the Error Model workbook for Monte Carlo simulation.
• Any wet - dry matter conversions, if not negligible, have been performed on the
appropriate reference NTE event values before the reference NTE event was entered
in the Error Model workbook for Monte Carlo simulation. No wet - dry conversions
are performed in the workbook.
• Any reference NTE event normalizations to produce similar emissions brake-specific
results from the three emissions calculation methods have been appropriately
performed before the reference NTE event was entered in the Error Model workbook
for Monte Carlo simulation. No normalizations among the three methods are
performed in the workbook.
• Emissions models for three calculation methods and three emissions are computed
during one MC simulation run.
• Error surface models and supporting data were approved by the Steering Committee.
• The error model spreadsheet has been correctly implemented, and its interaction with
Monte Carlo tools like Crystal Ball is correctly understood.
• Random number generation by a Monte Carlo tool like Crystal Ball is correct.
• Convergence of the completed MC simulation was processed and checked outside of
this workbook using a SAS® computer program.
2.0 WORKSHEET DESCRIPTIONS
2.1 Macro Description
The Macro can be viewed in the Excel spreadsheet with the menu selections
Tools>Macros>Macrol>Edit. The purpose of Macro 1 is to expedite clearing extra cells below
the reference NTE event in the Methods worksheet and to delete extra rows in the Delta error
worksheets. The macro also performs Mode 0 calculations and stores resultant 'ideal emissions'
values for application in subsequent Monte Carlo simulation.
SwRI Report 03.12024.06 D-l
-------
The user must begin with the starter version of the Excel file which has 300 rows of
equations in columns X - CF and in rows 52 - 351 in the Methods worksheet. The starter
spreadsheet also has 300 rows of equations below charts in columns B - F, or B - L, in
applicable Delta worksheets. The user copies the reference NTE event into columns A - V, row
52 and down, in the Methods worksheet. It is then necessary to confirm that cell J45 in the
Methods worksheet displays the correct number of rows of the reference NTE event.
Macro execution can be accomplished through the menu selections
Tools>Macros>Macrol>Run. Note that this macro clears cells without deleting rows in the
Methods worksheet, and deletes rows in the Delta worksheets. This macro will not work if the
reference NTE event has only one row. For a reference NTE event with exactly two rows, this
macro will corrupt the second "check" values in columns B-F type Delta worksheets. Check
values are not used in the simulation, but are provided as a diagnostic aid. Apply the macro for
reference NTE events with no more than 300 rows.
The reader can follow the description of execution that follows by viewing the macro and
observing the comment rows provided throughout the macro text. In execution, the macro first
reads the contents of J45 in the Methods worksheet. It uses the number of rows in the reference
NTE event defined by J45 to determine how many rows to clear and delete in the spreadsheet. It
checks that the number of rows is between 2 and 299, inclusive. It will also execute correctly for
300 rows.
Next, the macro clears cell contents in columns X - CF below the reference NTE event in
the Methods worksheet. Note the macro, as written, will not execute properly if the starter
spreadsheet has been revised with row insertion or deletion in certain areas of the spreadsheet.
As written, the macro initiates in cell X52, counts down through the NTE Event rows, and clears
contents in the range from there in column X through cell CF351.
Next, the macro deletes extra rows below the reference NTE event in Delta worksheet 1.
For Delta worksheet 1 it initiates in cell B79 and counts down through the rows of the reference
NTE event to the first row to be deleted. It selects the range of rows from there down through
row 378, deletes the rows, copies some equations and a value to the last row in the range the
charts use, and returns the cursor to cell F68 leaving the display more or less centered on the
charts in the worksheet.
Subsequently, the macro performs similar operations in other Delta worksheets; however,
the initiating cell and final row differ among the worksheets. The Delta worksheets processed in
this way are 1, 2, 5, 7, 8, 10, 11, 16, 17, 20, 21, 22, 23, 29, 30, 36, 37, 43, 44, 45, 46 and 49.
Following the row deletion operations in the Delta worksheets, or directly when the
reference NTE event has 300 rows, the macro prepares for the Mode 0 (ideal emissions)
calculation. First, in the Methods worksheet it copies the equations in row 52, columns X
through CF, to the last row in the reference NTE event. This clears any errors introduced in the
last row; however, it assumes that row 52 is correct. The last cell in column AC (At) is cleared
for aesthetics, since the At values are not applied in the model calculations.
SwRI Report 03.12024.06 D-2
-------
The Mode 0 calculation is performed by the macro by changing the value in cell A6 of
the Summary worksheet to 0. Then in the Methods worksheet, the values from cells CU22
through CU32 are pasted (values only) to cells O22 through O32 where they are referenced by
formulas during Monte Carlo simulation. The macro changes the value of A6 in the Summary
worksheet to 2 in preparation for the Monte Carlo simulation, and moves the cursor to cell CT18
of the Methods worksheet.
Additional comments regarding the macro operation are presented in the following
section descriptions of the model spreadsheet.
2.2 Worksheet 1: ErrorControl
The ErrorControl worksheet of the Error Model workbook implements 52 logic switch
functions. The user enters a numerical "1" in column D in each row corresponding to error
surfaces to be included in the calculation. A numerical "0" is applied to error surfaces to be
excluded in the calculation.
Error surfaces are numbered sequentially 1 through 50, and Time Alignment error surfaces
are designated 51 and 52. The numbered error surfaces are defined in columns A - C, and
information pertinent to their usage is presented in columns E - V of the worksheet. Column E
displays warning messages when an unusual value is entered in column D.
The control switch elements in the worksheet are deliberately placed on rows in the
worksheet corresponding to the error surfaces to expedite equation checking in the Methods
worksheet where the control switch variables are applied in conjunction with error surfaces from
the correspondingly numbered "Delta" worksheets
The numbered error surfaces and time alignment controls that have been implemented are
defined in the following Table 1.
TABLE 1. ERROR SURFACES USED IN SIMULATION
SwRI Report 03.12024.06 D-3
-------
Component
Delta NOX
Delta CO
Delta NMHC
NMHC = 0.98*THC
Delta Exhaust Flow
Delta Torque
Delta BSFC
Delta Speed
Delta Fuel Rate
Delta CO2
Time Alignment
No.
1
2
5
7
10
11
13
14
16
17
19
20
21
22
23
25
27
28
29
30
31
32
34
35
36
37
38
39
41
42
43
44
45
46
49
51
52
Error Surface
Delta NOXSS
Delta NOX Transient
Delta NOX Ambient Temperature
Delta CO SS
Delta CO Atmospheric Pressure
Delta CO Ambient Temperature
Delta NMHC SS
Delta NMHC Transient
Delta NMHC Atmospheric Pressure
Delta NMHC Ambient Temperature
Delta Ambient NMHC
Delta Exhaust Flow SS
Delta Exhaust Flow Transient
Delta Exhaust Flow Pulsation
Delta Exhaust Flow Swirl
Delta Exhaust EMI/RFI
Delta Exhaust Temperature
Delta Exhaust Pressure
Delta Dynamic Torque
Delta Torque DOE Testing
(Interacting Parameters Test)
Delta Torque Warm-up
(Interacting Parameters Test)
Delta Torque Humidity / Fuel
(Independent Parameters Test)
Delta Torque Interpolation
Delta Torque Engine Manufacturers
Delta Dynamic BSFC
Delta BSFC DOE Testing
(Interacting Parameters Test)
Delta BSFC Warm-up
(Interacting Parameters Test)
Delta BSFC Humidity / Fuel
(Independent Parameters Test)
Delta BSFC Interpolation
Delta BSFC Engine Manufacturers
Delta Dynamic Speed
Delta Dynamic Fuel Rate
Delta CO2 SS
Delta CO2 Transient
Delta CO2 Ambient Temperature
Delta NOX Time Alignment
Delta CO Time Alignment
SwRI Report 03.12024.06
D-4
-------
The thirty-five (35) error surfaces that have been implemented are included or excluded
by the controls numbered 1-49 identified in Table 1. Three NOX time alignment errors are
controlled by number 51 and three CO time alignment errors are controlled by number 52.
When all 35 error controls and both time alignment controls are on (included in calculation), the
sum of column D in the worksheet ErrorControl is 37.
2.3 Worksheet 2: Summary
The Summary worksheet in the Error Model workbook comprises input mode control in
rows 4-10 and output summary in rows 12 - 142.
The calculation mode control is accomplished with cell A6 where the user normally
confirms that a numerical value of "2" is designated. Mode 2 designates emissions calculation
with all errors applied. Mode 1 corresponds to a calculation of emissions with all errors applied
except environmental errors and time alignment. Mode 0 designates an "ideal" emissions
calculation with no errors applied. In Monte Carlo error model simulation performed in this
study Mode 2 was used.
Mode 0 is used off-line prior to Monte Carlo simulation to generate the "ideal" emissions
for a given reference NTE event. The Mode 0 values are calculated by entering a value of "0" in
cell A6. The Mode 0 calculation and subsequent storing of the "ideal" emissions results may be
accomplished manually (as described above) or by exercising a provided macro. The macro
automatically sets the value in cell A6 to zero, calculates and saves the "ideal" emissions values,
and returns the value in A6 to "2" in preparation for the Monte Carlo simulation. The locations
where the reference NTE event must be entered manually, and the locations where the "ideal"
emissions must be saved (done automatically if the macro is used) are described in the Methods
worksheet section.
Mode 1 calculations are fully implemented in the Error Model spreadsheet and used for
drift correction calculations. Mode 1 in cell A6 is not typically used but can be applied for
diagnostic purposes.
The output summary section of the Summary worksheet in rows 12 - 142 presents
numerically and descriptively labeled outputs of the emissions and emissions error calculations.
The suffix '_TA' indicates that time alignment has been applied to the result, and the suffix
'_DC' indicates that periodic drift correction has been applied to the result. Both time alignment
and periodic drift correction are applied in the Methods worksheet. However, a result in the
Summary worksheet with designation '_TA' has been calculated with an emissions value plus
errors all modified by the applicable time alignment percentage just prior to subtracting the
"ideal" emissions to produce the emissions difference. Time alignment was applied only to NOX
and CO emissions. Similarly, a result in the Summary worksheet with designation '_DC' has
been calculated by a drift correction formulation. A periodic drift rejection flag was checked by
logic.
Consider the logic in cell C19 for the output 001_DeNOx (g/kW-hr), Method 1_TA_DC.
The logic checks the drift rejection flag in cell C43. If the flag is 1, a huge negative number
SwRI Report 03.12024.06 D-5
-------
(typically -9999999, from cell D13) is returned in cell C19. Otherwise, the time aligned value
from the Methods worksheet is returned in cell C19 of the Summary worksheet.
In the output summary, the cells that are highlighted in turquoise color are designated by
Crystal Ball as "Forecast" (or output) random variables. When running the Monte Carlo
simulation with Crystal Ball, a filter is designated for the '_DC' (drift corrected) "Forecasts"
such that a value less than -9999 (or similar huge negative number, but greater than the value
typically -9999999 returned when drift flag is 1) will be rejected. The filter on the "Forecast"
rejects the values meeting the filter criterion (drift correction). Thus, when data is "Extracted"
from a Crystal Ball Monte Carlo simulation, drift corrected (rejected) values are omitted (cells
are empty) in the Excel spreadsheet of simulation output values. It is expected that other Monte
Carlo software comparable to Crystal Ball, such as @Risk, will have similar rejection provisions
with which periodic drift correction based on the calculated flag can be accomplished.
A total of 213 outputs ("Forecasts") are designated in the Summary worksheet covering
the number of output values from three emissions (NOx, CO and NMHC), three calculation
methods (Methods 1, 2 and 3), with and without time alignment, and with and without drift
correction, for the full error model and for the validation model (designated Valid in Summary
worksheet variable labels). All of these "Forecasts" are provided in both units of grams/kW-hr
and (for selected outputs) in grams/hp-hr. This variety of calculations was accomplished in the
Methods worksheet.
2.4 Worksheet 3: Methods
The Methods worksheet of the Error Model workbook comprises the following areas:
• Notes and diagnostic guides are located principally in rows I -22 in columns A - CF,
continuing on row 5 through column DD.
• Reference NTE event data are located in rows 35 - 351 of columns A - W. Actual
reference NTE event data must be entered manually starting on row 52 in columns A - V.
One to 300 rows of reference NTE event data are allowed. Uniform (one second interval)
time steps are assumed represented by the reference NTE data.
• Parameters calculated are located in rows 35-351 of columns X - CF. The number of
rows of these parameter equations must match the number of rows in the reference NTE
event. Excess cells in these columns may be cleared manually or automatically during
execution of the macro.
• Mode 0, Ideal Emissions for this reference NTE event are stored in column O rows 22 -
32 (either manually or automatically by the macro). Related data on the same rows are
located in columns CT - DD.
• Input ic random variable distributions (Crystal Ball uses the terminology "Assumptions"
for these inputs) are located in rows 26 - 32 of columns AG - CF.
SwRI Report 03.12024.06 D-6
-------
• Emissions calculations by three methods are located in rows 6 - 101 of columns CG - CJ
(Method 1), CK - CN (Method 2) and CO - CR (Method 3). This part of the worksheet
calculates full model, validation model, time alignment and drift correction.
2.5 Methods Worksheet:Notes and Diagnostic Guide
In rows 1-22 for columns A - CF, several descriptive labels and references are defined
for use in navigating through the worksheet. Row 5, columns A-DD, contains column
identification numbers referenced in rows 7 through 22 (depending on the column). For
example, in column H the values 65 and 66 in rows 8 and 9, respectively, indicate that the values
in column H (rows 52 and following rows) are applied in columns 65 (BM) and 66 (BN) labeled
on row 5. If the user scrolls to cells BM52 or BN52 it is observed that the spreadsheet formulas
in these cells refer to values from column H. Also, column H, rows 10, 11 and 12 indicate that
the values in column H (rows 52 down) are also applied in the Delta (emissions error surface)
tabs '45 Delta CO2 SS', '46 Delta CO2 Transient' and '49 Delta CO2 Ambient Temperature'.
The information in the notes and diagnostic guide was not applied by the spreadsheet in any of
the emissions calculations. It was included with the intent to simplify diagnostics by providing
information on locations where spreadsheet values were applied elsewhere in the spreadsheet.
Outside the areas indicated above, some other notes, comments and diagnostic guides may be
found in other areas of the spreadsheet.
2.5.1 Methods Worksheet: Reference NTE Event
The reference NTE event used in the simulation was entered in rows 35-351 of columns
A - W. Actual reference NTE event data must be entered manually starting on row 52 in
columns A - V. A minimum of one and a maximum of 300 rows of reference NTE event data
are allowed. Equal time steps (1 second intervals) are assumed in the reference NTE data rows.
The standard format and engineering units of reference NTE event data established for this
project must be observed. These are described in the column headings on rows 47-51, columns
A-V.
2.5.2 Methods Worksheet: Parameters
Parameters applied in the three emissions methods are calculated in rows 35 - 351 of
columns X - CF. The number of rows of these parameter equations must match the number of
rows in the reference NTE event. Excess cells in these columns may be cleared manually or
automatically during execution of the macro.
The formulas applied in rows 52 and down in columns X - CF have been produced by
normal edit-copy (typically of row 52 in these columns) and edit-paste to rows 53 and following
rows in these columns. The At values displayed in column AC are not used in any calculation,
but are displayed so a user can confirm uniform reference NTE event time sampling. The last
cell in column AC can be cleared (done automatically by the macro). Note that excess cells in
these columns must be cleared, and row deletion operations should not be applied since this
would affect other areas in the Methods worksheet.
SwRI Report 03.12024.06 D-7
-------
Certain sums are performed in several columns over the Parameter rows (range of the
reference NTE event). These are accomplished in row 46 in columns AU - AX, BC, BD, BI, BJ,
BO-BR and BU-CF. Certain constants applied in the calculation are stored in cells
AW42, BC42, BI42, BP40 and BP42. Other constants or conversion factors are incorporated
numerically in spreadsheet formulas. Typical of these is "0.01" to convert a percentage to a
fraction.
Specific parameters or variables are calculated in the various columns for application in
all three methods, full model and validation model, and for drift correction. Time alignment
distinctions are not generated in this area of the spreadsheet. Table 2 lists the parameters used in
the Methods worksheet, the columns where they are computed and a brief description of the
parameters.
TABLE 2. METHODS WORKSHEET PARAMETER COLUMN DESCRIPTIONS
Methods Worksheet Parameters Column Descriptions
Subject
Engine operating state
percentages
ATime
NMHC
Fuel Rate
Exhaust Flow
Calculations
Speed with error
Fuel rate with error
Torque
BSFC
NOx and ANOx, ppm
Column
X-AB
AC
AD
AE
AF
AG
AH
AI
AJ
AK
AL
AM
AN
AO
AP
AQ
AR
Description
Convert NTE Event variables to percentages: speed, torque, fuel rate,
exhaust flow
Displays At between NTE Event rows
Calculate NMHC ppm as 0.98 of THC ppm
Calculate fuel rate g/s based on fuel density of 851 g/L
Convert exhaust flow SCFM to mol/s
Sum exhaust flow errors from Delta tabs 20, 21, 22, 23, 25 27 and 28
expressed in % of mol/s maximum. Respective ErrorControl tab
switches are applied.
Convert the total exhaust flow error in % of maximum mol/s to mol/s
Add the mol/s exhaust flow error to the exhaust flow in mol/s. Mode
control logic is applied.
Add engine speed error from Delta tab 43 expressed as % of engine
range converted to rpm to engine speed in rpm. Mode control logic
and ErrorControl switch are applied.
Add fuel rate from Delta tab 44 expressed as % of maximum fuel rate
converted to g/s to engine fuel rate in g/s. Mode control logic and
ErrorControl switch are applied.
Sum torque errors from Delta tabs 29, 30, 31, 32, 34 expressed as % of
peak torque, and from Delta tab 35 expressed as % of NTE point
torque converted to % of peak torque. ErrorControl switches are
applied.
Add the total torque error expressed as % of peak torque converted to
N-m to engine torque in N-m. Mode control logic is applied.
Sum BSFC errors from Delta tabs 36, 37, 38, 39, 41 expressed as
g/kW-hr, and from Delta tab 42 expressed as % of NTE point BSFC
converted to g/kW-hr. ErrorControl switches are applied.
Add the total BSFC error expressed as g/kW-hr to engine BSFC in
g/kW-hr. Mode control logic is applied.
Sum engine NO ppm and NO2 ppm.
Sum environmental NOx errors. Error from Delta tab 5 is the only one
developed.
Sum other NOx errors including errors from Delta tabs 1 and 2.
SwRI Report 03.12024.06
D-8
-------
AS
AT
Add the total NOx errors expressed as ppm to engine ideal NOx in
ppm. Mode control logic is applied.
Add the NOx errors except environmental expressed as ppm to engine
ideal NOx in ppm for drift correction calculation. Mode control logic
is applied.
NOx • Exhaust Flow
AU
AV
Form product of NOx fraction (all errors case, column AS) and
exhaust flow (mol/s, column AI) for application in Methods 1 and 2.
Form product of NOx fraction (all errors except environmental,
column AT) and exhaust flow (mol/s, column AI) for application in
Methods 1 and 2 drift correction.
Speed • Torque
AW
AX
Form product of Speed (rpm, all errors case, column AJ) and Torque
(N-m, all errors case, column AM) for application in Methods 1 and 3.
Convert rpm to radians/sec with 2;tradians/revolution, minutes to
seconds with 60sec/min, N-m/sec to watt hr with 3600Joules/watt hr,
and watt to kW with lOOOw/kW.
Form product of Speed (rpm, no errors for validation case, column O)
and Torque (N-m, no errors for validation case, column T) for
application in Methods 1 and 3. Convert rpm to radians/sec with
27tradians/revolution, minutes to seconds with 60sec/min, N-m/sec to
watt hr with 3600Joules/watt hr, and watt to kW with IQOOw/kW.
CO and ACO, %
AY
AZ
BA
BB
Sum environmental CO errors including errors from Delta tabs 10 and
11.
Sum other CO errors. Error from Delta tab 7 is the only one
developed.
Add the total CO errors expressed as % to engine CO in %. Mode
control logic is applied.
Add the CO errors except environmental expressed as % to engine CO
in % for drift correction calculation. Mode control logic is applied.
CO • Exhaust Flow
BC
BD
Form product of CO fraction (all errors case, column BA) and exhaust
flow (mol/s, column AI) for application in Methods 1 and 2.
Form product of CO fraction (all errors except environmental, column
BB) and exhaust flow (mol/s, column AI) for application in Methods 1
and 2 drift correction.
NMHC and ANMHC,
ppm
BE
BF
BG
BH
Sum environmental NMHC errors including errors from Delta tabs 16,
17 and 19.
Sum other NMHC errors including errors from Delta tabs 13 and 14.
Add the total NMHC errors expressed as ppm to engine NMHC in
PPM. Mode control logic is applied.
Add the NMHC errors except environmental expressed as ppm to
engine NMHC in ppm for drift correction calculation. Mode control
logic is applied.
NMHC • Exhaust Flow
BI
BJ
Form product of NMHC fraction (all errors case, column BG) and
exhaust flow (mol/s, column AI) for application in Methods 1 and 2.
Form product of NMHC fraction (all errors except environmental,
column BH) and exhaust flow (mol/s, column AI) for application in
Methods 1 and 2 drift correction.
CO2 and ACO2, %
BK
BL
BM
BN
Sum environmental CO2 errors. Error from Delta tab 49 is the only
one developed.
Sum other CO2 errors including errors from Delta tabs 45 and 46.
Add the total CO2 errors expressed as % to engine CO2 in %. Mode
control logic is applied.
Add the CO2 errors except environmental expressed as % to engine
CO2 in % for drift correction calculation. Mode control logic is
applied.
SwRI Report 03.12024.06
D-9
-------
Exhaust Flow • |
NMHC + ( C0+C02)
/BSFC
BO
BP
BQ
BR
Form product of NMHC fraction plus CO and CO2 fractions (all errors
case, columns BG, BA and BM) and exhaust flow (mol/s, column AI)
divided by BSFC (g/kW-hr, all errors case, column AO) for
application in Method 2.
Form product of NMHC fraction plus CO and CO2 fractions (all errors
except environmental, columns BH, BB and BN) and exhaust flow
(mol/s, column AI) divided by BSFC (g/kW-hr, with errors, column
AO) for application in Method 2 drift correction.
Form product of NMHC fraction plus CO and CO2 fractions (all errors
case, columns BG, BA and BM) and exhaust flow (mol/s, column AI)
divided by BSFC (g/kW-hr, no errors case, column V) for application
in Method 2 validation.
Form product of NMHC fraction plus CO and CO2 fractions (all errors
except environmental, columns BH, BB and BN) and exhaust flow
(mol/s, column AI) divided by BSFC (g/kW-hr, no errors, column V)
for application in Method 2 validation drift correction.
NMHC + (CO+C02)
BS
BT
Form sum of NMHC fraction plus CO and CO2 fractions (all errors
case, columns BG, BA and BM) for application in Method 3.
Form sum of NMHC fraction plus CO and CO2 fractions (all errors
except environmental, columns BH, BB and BN) for application in
Method 3.
NOx • Fuel Rate /
NMHC + ( CO+CO2);
BU
BV
Form product of NOx fraction (all errors case, using column AS) and
Fuel Rate (g/s, all errors case, column AK) divided by sum of NMHC
fraction plus CO and CO2 fractions (all errors case, column BS) for
application in Method 3.
Form product of NOx fraction (all errors except environmental, using
column AT) and Fuel Rate (g/s, with error, column AK) divided by
sum of NMHC fraction plus CO and CO2 fractions (all errors except
environmental, column BT) for application in Method 3 drift
correction.
CO • Fuel Rate /
NMHC + ( CO+CO2);
BW
BX
Form product of CO fraction (all errors case, using column BA) and
Fuel Rate (g/s, all errors case, column AK) divided by sum of NMHC
fraction plus CO and CO2 fractions (all errors case, column BS) for
application in Method 3.
Form product of CO fraction (all errors except environmental, using
column BB) and Fuel Rate (g/s, with error, column AK) divided by
sum of NMHC fraction plus CO and CO2 fractions (all errors except
environmental, column BT) for application in Method 3 drift
correction.
NMHC • Fuel Rate /
NMHC + ( CO+CO2) ]
BY
BZ
Form product of NMHC fraction (all errors case, using column BG)
and Fuel Rate (g/s, all errors case, column AK) divided by sum of
NMHC fraction plus CO and CO2 fractions (all errors case, column
BS) for application in Method 3.
Form product of NMHC fraction (all errors except environmental,
using column BH) and Fuel Rate (g/s, with error, column AK) divided
by sum of NMHC fraction plus CO and CO2 fractions (all errors except
environmental, column BT) for application in Method 3 drift
correction.
NOx • Fuel Rate /
NMHC + (CO+C02);
CA
CB
Form product of NOx fraction (all errors case, using column AS) and
Fuel Rate (g/s, selected errors case, column AE) divided by sum of
NMHC fraction plus CO and CO2 fractions (all errors case, column
BS) for application in Method 3 validation.
Form product of NOx fraction (all errors except environmental, using
column AT) and Fuel Rate (g/s, selected errors case, column AE)
divided by sum of NMHC fraction plus CO and CO2 fractions (all
errors except environmental, column BT) for application in Method 3
validation drift correction.
SwRI Report 03.12024.06
D-10
-------
CO • Fuel Rate / [
NMHC + (CO+C02)]
CC
CD
Form product of CO fraction (all errors case, using column BA) and
Fuel Rate (g/s, selected errors case, column AE) divided by sum of
NMHC fraction plus CO and CO2 fractions (all errors case, column
BS) for application in Method 3 validation.
Form product of CO fraction (all errors except environmental, using
column BB) and Fuel Rate (g/s, selected errors case, column AE)
divided by sum of NMHC fraction plus CO and CO2 fractions (all
errors except environmental, column BT) for application in Method 3
validation drift correction.
NMHC • Fuel Rate / [
NMHC + ( CO+CO2) ]
CE
Form product of NMHC fraction (all errors case, using column BG)
and Fuel Rate (g/s, selected errors case, column AE) divided by sum of
NMHC fraction plus CO and CO2 fractions (all errors case, column
BS) for application in Method 3 validation.
CF
Form product of NMHC fraction (all errors except environmental,
using column BH) and Fuel Rate (g/s, selected errors case, column
AE) divided by sum of NMHC fraction plus CO and CO2 fractions (all
errors except environmental, column BT) for application in Method 3
validation drift correction.
2.5.3 Methods Worksheet: Mode 0 Ideal Emissions
For the reference NTE event in rows 52 and down in columns A - V, an ideal emissions
value must be calculated and stored for application in the emissions difference calculations. The
ideal case can be calculated either manually or automatically by the macro. Following the
calculation, the ideal values are stored by edit-copy edit-paste-special-values operation to the
cells in column O, rows 22 - 32. The manual operations described below are performed
automatically by the macro, if executed, after manually entering the reference NTE event.
After manually entering the reference NTE event to be simulated and checking that the
number of rows of equations in the Parameters section matches the rows in the reference NTE
event, a numerical "0" can be entered in cell A6 of the Summary worksheet. The Methods
worksheet should have calculated Mode 0 results using the reference NTE event. If error
messages like "#VALUE or #DIV/0!" are displayed, there is probably still a mismatch between
the rows of the reference NTE event and Parameter equations. When calculated properly (with 0
in Summary A6), the values displayed in the Methods worksheet columns CU, CV, CW, DB and
DD will be equal on each of the rows 22 - 32. The values in column CX are not yet equal
(unless previously calculated and stored for this reference NTE event) because they reflect the
values stored in Methods worksheet column O, rows 22 - 32. The next manual step is to edit-
copy column CU, rows 22 -32, and store the values by edit-paste-special-values in column O,
rows 22 -32. Now in rows 22 -32 the columns CU - DD should be equal. The final step is to
return to Summary worksheet cell A6 and change the value from 0 to 2. At this point the
spreadsheet could be run in Monte Carlo simulation to produce properly sampled values.
However, if the user desires to monitor charts provided in the Delta worksheets during the
simulation, further row-matching to the reference NTE event is required in most of the Delta
worksheets.
The manual operations described in the previous paragraph are intended to explain how
the Mode 0 ideal emissions are calculated and stored for use in the Monte Carlo simulation when
Aemissions values are calculated using the ideal emissions results stored in O22 - O32. The
reference NTE event must be entered with an operation such as a manual edit-copy and edit-
paste or edit-paste-special-values operation. At this point the macro can be executed with tools-
SwRI Report 03.12024.06
D-ll
-------
macro-macros-(macrol)-run operation. The macro automatically performs the mode 0
calculation, stores the mode 0 results in O22 - O32, and changes Summary A6 back to mode 2.
The macro also deletes extraneous rows from all the appropriate Delta worksheets so the
charts therein display properly. It is important to copy the reference NTE event into a fully
'loaded' starter file with equations filled on 300 rows in the Parameters area, and with full 300
row complement of equation-rows in each of the appropriate Delta worksheets for the macro to
modify the spreadsheet properly.
2.5.4 Methods Worksheet: Input ic Random Variable Distributions
Probability distribution parameters are applied, and simulation trial values of the inputs
are generated in rows 26 - 32 of columns AG - CF. Rows 26 and 27 are used to input
distribution parameters. Rows 28 and 29 contain descriptive labels brought from the appropriate
Delta worksheet. Row 30 is an information-only number, row 31 contains the name label
applied in Monte Carlo simulation to the input ic, and row 32 is where the Monte Carlo
simulation tool places generated randomly-sampled values during simulation. The values in row
31 are referenced by formula in the respective Delta worksheets where they are used for
interpolation on the error surfaces.
The Monte Carlo tool in Crystal Ball uses the terminology "Assumptions" for these
inputs. Two distribution forms are applied: truncated normal (Gaussian), and discrete uniform.
For the normal distribution, the applied standard deviation is in row 27. In Crystal Ball, the
standard deviation cell on row 27 and the label cell on row 31 were referenced by equation in the
Crystal Ball assumption setup window, the mean was input as 0, and the distribution was
truncated at -1 and at +1. Since all the truncated normal ic distributions are identical (although
the sampled trial values from each will be random in the Monte Carlo simulation), the Crystal
Ball define-copy data and defme-paste data operations were applied to define the truncated
normal distributions for other ic variables once the first one had been defined.
For the discrete uniform distributions, the minimum discrete value (1 in all cases) was
applied in row 26, the maximum discrete value was applied in row 27 and the other row
descriptions are the same as before. Again, one of these inputs was setup with Crystal Ball
"define assumption" and then applied with Crystal Ball define-copy data and defme-paste data
operations to other ic cells on row 32 where a discrete uniform distribution was applied. When
Crystal Ball "Assumptions" were defined, Crystal Ball colored each input cell bright green. It is
expected that @Risk has similar input definition procedures.
During a Monte Carlo simulation, the Monte Carlo tool (e.g. Crystal Ball) placed a
numerical value in each of the ic cells on row 32. Then the spreadsheet was exercised to perform
interpolations in all the Delta worksheets. The resulting error sample values for the entire
reference NTE event were returned to the Methods worksheet Parameters area, and then the
Methods worksheet Emission Calculations section computes Aemissions using three methods,
full model and validation, drift correction, time alignment and not, etc. to generate one set of the
213 output values described in the Summary section. The simulation tool stores the set of 37
random input values from row 32 as well as the 213 output values in an Excel data base from
SwRI Report 03.12024.06 D-12
-------
which the corresponding sets of values can later be extracted. Once each trial was completed,
the simulation tool randomly sampled a second set of input values from the respective
probability distributions, placed the values in the cells on row 32, exercised the spreadsheet
again, stored the input and output values, and went to a third trial, etc. Typically 10,000 and
30,000 trials, depending on the reference NTE event, were used in this project with this Error
Model workbook.
Note that there are three ways the user can control the effect of the ic values in the
emissions calculations:
1. Mode control in Summary A6,
2. Include / exclude switches in ErrorControl column D, and
3. Specification of input random variables ("Assumptions") and their probability
distributions in the Methods worksheet row 32.
These three ways of controlling the ic values are independent, but the effects are interdependent
as follows. Mode control determines what categories of errors are added into the calculations.
Mode controls categories of errors are classified as:
1. Mode 0 - no errors included
2. Mode 1 - "all" but 'environmental' errors included
3. Mode 2 - "all" errors added into the calculations.
"All" in this context represents those error surfaces turned on by the switches in the ErrorControl
worksheet. The input random variable distribution controls the distribution of the sampled ic
values applied during Monte Carlo simulation for the several Delta error surfaces. Mode and
ErrorControl switches must be appropriately turned on for the effects of the sampled ic values to
be included in the emissions difference results. These controls affect the calculations in the
Methods worksheet Parameters and Emission Calculations sections.
2.5.5 Methods Worksheet: Emission Calculations
In the area of rows 6 - 101 of columns CG - CR the brake-specific emissions and
Aemissions calculations are performed using the variables and parameters generated in the
Parameters section. Three sets of columns, structured similarly, calculate the full model,
validation model, time alignment and drift correction for the following methods:
1. Method 1 calculations are applied in columns CG - CJ,
2. Method 2 calculations are applied in columns CK - CN, and
3. Method 3 calculations are in columns CO - CR.
Columns CG - CJ for Method 1 are typical of the methods where the structure is the same, but
the formulas are a little different. Column CG is an information-only column that displays
Method 1 formulas implemented by equations in columns CH - CJ. Column CH performs the
SwRI Report 03.12024.06 D-13
-------
NOX emission calculations, while column CI performs the CO emissions and column CJ
performs the NMHC calculations. The structure of the three columns is the same. Formulas
implemented in the three columns are the same, but the equations implementing the formulas
apply variables and parameters appropriate to the respective emissions.
As an example of the calculation for NOX Method 1 we will examine column CH in
detail. The full model calculation was accomplished in cells CH48 - CH74. The ideal emissions
result was brought into the area by equation in CH51. Full model NOX emissions (eNOx) in
g/kW-hr were calculated in CH54. Cells CH55 - CH59 are information-only diagnostic aids.
The full model Method 1 result in CH54 is calculated by the formula in Figure 1.
N
N
z
<^\ppm)+xNUi\j>pmj
m°
60*1000*3600
FIGURE 1. BRAKE-SPECIFIC NOX BY METHOD 1
In the formula for the full model mode 2, delta error values sampled from the Delta worksheets
1, 2 and 5 have been added to xNO2, xNO. Similarly, delta error values sampled from Delta
worksheets 20-23 and 25-28 have been added to the exhaust flow, delta errors sampled from
worksheets 29-32 and 34-35 were added to torque, and worksheet 43 deltas were added to speed.
The At values are equal (1 second) and therefore cancel out of the equation.
'Full' model Mode 1 (with all errors except environmental errors from Delta worksheet
5) is calculated in CH64. Cells CH65 - CH69 are for information-only.
The full model rejection flag is calculated in CH74. A rejection flag is computed as
shown in Figure 2. For reference NTE events with ideal NOX emissions below the threshold of
2.682 g/kW-hr (2.0 g/hp-hr), the allowable band is defined by full model emissions (mode 1 -
mode 2| < 4% of threshold. For reference NTE events with ideal NOX emissions above the
threshold, the allowable band is defined by full model emissions (mode 1 - mode 2| < 4% of
ideal emissions.
SwRI Report 03.12024.06
D-14
-------
Drift Correction Rejection Flag
0.3
0.2-
•o
o
E
•o
i
0.1-
0)
•o
o
0)
O) 0
O
z
0)
-0.1 -
0)
o
c
gj
I -0-2
-0.3
Flag = 1
Flag = 0
Flag = 1
Ideal eNOx g/kw-hr (mode 0 )
FIGURE 2. PLOT ILLUSTRATING PERIODIC DRIFT CORRECTION FLAG
The validation model calculation was accomplished in the cells CH79 - CH101.
Validation model NOX emissions (g/kW-hr) was calculated in cell CH81. Cells CH82 - CH86
are information-only. Validation model Mode 1 (with 'all' errors except certain environmental
errors) was calculated in CH91. Cells CH92 - CH96 are information-only. The validation
model rejection flag was calculated in CH101. For reference NTE events with ideal NOX
emissions below the threshold of 2.682 g/kW-hr (2.0 g/hp-hr), the allowable band was defined
by validation model emissions (mode 1 - mode 2| < 4% of threshold. For reference NTE events
with ideal NOX emissions above the threshold, the allowable band was defined by validation
model emissions (mode 1 - mode 2| < 4% of ideal emissions.
The drift correction flag was computed similarly for CO and NMHC emissions with
different thresholds and allowable bands. For CO the threshold was 26.016 g/kW-hr (19.4 g/hp-
hr) and the allowable band was again defined at 4%. For NMHC the threshold was 0.282 g/kW-
hr (0.21 g/hp-hr) and the allowable band was defined at 10%.
Continuing with column CH for NOX, the time alignment indicated by the suffix '_TA' in
the label in CH24 was calculated for the full model in CH25. The Time Alignment was applied
as a modification by a percentage of the full model NOX emissions result from CH54. The time
alignment for NOX expressed as a percentage from Delta worksheet 51 was converted to a
fraction, and the fractional increment g/kW-hr was added to the NOX expressed in g/kW-hr. The
SwRI Report 03.12024.06
D-15
-------
mode control logic and ErrorControl switch are applied. The same calculation of time alignment
for the validation result from CH81 was done in CH42 again with mode control and
ErrorControl.
In CH14 the full model ANOX value in g/kW-hr was calculated for the time alignment
case by subtracting the ideal emissions in g/kW-hr from the result in CH25 described in the
previous paragraph. The full model ANOX value in g/kW-hr was converted to g/hp-hr in CHI 1.
Similar results for the validation model are in CHI 5 and CH12.
Rejection flags are taken to the Summary worksheet where drift correction explained in
the Summary worksheet section has been described. Additional result combinations are
calculated and presented for NOX Method 1 in column C of the Summary worksheet using results
described above from column CH of the Methods worksheet.
Calculations for NOX by Method 1 described above for column CH are similar for CO
and NMHC by Method 1 in columns CI and CJ, respectively. Similar calculations for NOX, CO
and NMHC by Method 2 are presented in columns CL - CN, and by Method 3 in columns CP -
CR.
2.6 Worksheet 4: Constants and Equations
The Constants&Eqns tab was strictly a snapshot of the equations used in the brake-
specific emissions calculations. It displayed the equations and constants implemented in
spreadsheet formulas of the Methods worksheet. The various parts all shown together in this
worksheet are redisplayed at appropriate locations in the Methods worksheet.
2.7 Worksheet 5: SS NOx Error Surface
The 1 Delta NOx SS worksheet was the first Delta worksheet. Its functional structure,
formulas, charts and operation are similar to the following worksheets:
• 7 Delta CO SS
• 20 Delta Exhaust Flow SS
• 22 Delta Exhaust Flow Pulsation
• 23 Delta Exhaust Flow Swirl
• 30 Delta Torque DOE Testing
• 37 Delta BSFC DOE Testing
• 45 Delta CO2 SS
With only minor changes in charts and structure, its function, formulas and operation are also
similar to the following worksheets:
• 2 Delta NOx Transient
• 21 Delta Exhaust Flow Transient
• 29 Delta Dynamic Torque
• 36 Delta Dynamic BSFC
SwRI Report 03.12024.06 D-16
-------
• 43 Delta Dynamic Speed
• 44 Delta Dynamic Fuel Rate
• 46 Delta CO2 Transient
The following provides a brief summary of the 1 Delta NOx SS worksheet:
• Rows 1-7 contain descriptive information about the error surface implemented in the
worksheet.
• Rows 8-42 present the error surface in columns A - L. Other columns, M - W, on these
rows generate a lookup table used with an interpolation routine.
• Figures A, B and C follow.
• Rows 76-379 calculate the ANOX SS error values for each row of the reference NTE
event. These values were returned to the Methods tab Parameters section.
The following paragraphs describe in further detail functions in the 1 Delta NOx SS worksheet:
Data from the error surface (rows 13 - 42, columns A - L, in this Delta worksheet) must
be entered in sorted order (sorted on Lab Nominal column C in ascending order) for proper
operation of the x-lookup-interpolation function. The three figures chart the error function.
Figure A plots several data sets versus the x-value, Lab Nominal (column C). Figure A y-values
are NOX ppm (PEMS) including Lab Nominal (column C), 95th percentile (column F), 50th
percentile (median) (column E) and 5th percentile (column D).
Related error surface data are plotted in Figure B. Figure B plots several data sets versus
the same x-value, Lab Nominal (column C). Figure B y-values are the difference, NOX ppm
(PEMS) - NOX ppm (lab, nom). The differences plotted may not correspond exactly to the
values shown in Figure A because of the statistical procedure applied in calculating the
differences shown in Figure B. This procedure is described in Sections 4.0 and 5.0 of this report.
Figure B plots the 95th percentile (column I), the 50th percentile (median) (column H) and the 5th
percentile (column G). In addition to the error surface data, Figure B also shows the
interpolation line designated ic = xx (column V), and the reference NTE event values on the
interpolation line (column F rows 80 through end of the reference NTE event versus Lab
Nominal x-values in column B rows 80 through end of the reference NTE event). When ic =
+1, the interpolation line plots on the 95th percentile. When ic = 0, the interpolation line plots on
the 50th percentile. When ic = -1, the interpolation line plots on the 5th percentile. The reference
NTE event always plots on the interpolation line, with points at the x-values in the reference
NTE event.
The error surface data were also plotted in the format of Figure C. Again the x-axis was
the same Lab nominal (column C). This time the y-axis data are the ic values. Thus, the 95th
percentile plots at +1, the 50th percentile plots at 0 and the 5th percentile plots at -1. The
interpolation line plots at the value of ic, and the reference NTE event plots on the interpolation
line at the x-values in the reference NTE event. If appropriate value labels were displayed in
Figure C, the values would represent the error surface plotted on a z-axis above the two-
dimensional x-y plane. These error surface values are displayed graphically in Figure B.
SwRI Report 03.12024.06 D-17
-------
Now consider inner rows 13 - 41 in the look-up table in columns T - W. Column T is a
repetition of the x-value from column C. Column U calculates a row-to-row A for the x-values
in column T for use in interpolation. Column V computes the interpolation line linearly
interpolated according to the value of ic between the median and the 95th percentile if ic > 0 (on
median if ic = 0 and on 95th percentile if ic = +1); and between the median and the 5th percentile if
ic < 0 (on median if ic = 0 and on 5th percentile if ic = -1). Only one ic value (from cell E80) is
applied in this calculation of the interpolation line. The Microsoft Excel vertical lookup function
VLOOKUP is applied to the table in rows 12 - 42 in columns T - W. This is done in rows 80
and down in column F. Because of the way the VLOOKUP function operates, the first row cells
T12 and V12, and the last row cell W42 (all three cells distinguished by darker line borders)
contain formulas or values different from the formulas of the inner rows. The formula in cell
T12 assures that the lookup function can always find an x-value in its table. The formula in V12
and the value in W12 assure that the interpolation in cells F80 to the end of the reference NTE
event data returns the nearest ANOx SS value on the interpolation line if the x-value is outside
the range of the error surface lab nominal values.
Before going to the interpolation accomplished in F80 and down, consider briefly the
formulation on rows 12 - 43 in columns O - R. This formulation considers one x-value from the
reference NTE event, the first one, in cell B80 and selects the two adjacent rows in the error
surface between which to interpolate on the B80 x-value. The result is formed on row 43 in
these columns and then the "check" cell G80 accomplishes the ic controlled interpolation. This
provides an alternative calculation check on one row in the reference NTE event.
Now consider the interpolation for each point in the reference NTE event. Column B,
row 80 and down, brings the lab nominal x-value from the Methods worksheet reference NTE
event. For this Delta worksheet, that x-value is xNO(ppm) + xNO2(ppm). The out-of-range
flags are information-only indicating points in the reference NTE event with x-value out of the
range of the error surface lab nominal. The ic value for this Delta worksheet was brought into
cell E80 from the Methods worksheet ic area. Each point in the reference NTE event was
interpolated with the same ic value, but with its own x-value. Recalling that the interpolation line
in column V was computed with this one ic value, the x-interpolation between the appropriate
two adjacent rows in the error surface can now be accomplished. This requires using the x-value
on each row in column B, B80 and down, in the VLOOKUP function, and performing the
required calculation using the looked-up values and deltas from the look-up table. The
calculation is done with the formulas in cell F80 and down. The values computed in column F,
cell F80 and down through the reference NTE event, could be considered elements of a column
matrix or vector, and are returned to the Methods worksheet Parameters section.
In the Monte Carlo simulation, the Methods worksheet combines this reference NTE
event result vector from the 1 Delta NOX SS worksheet with similar results from other error
surfaces, calculates Aemissions by three methods, full model and validation, drift reject flags,
time alignment, etc. to produce a set of 213 output values ("Forecasts" in Crystal Ball
terminology) described in the Summary worksheet section. This was done having input 35 ic
values (including 1 ic value for this Delta NOX SS) and two time alignment values all chosen by
random sample from the appropriate truncated normal or uniform distribution as explained in the
Methods worksheet section. Then another sample set of 35 plus two randomly sampled values
SwRI Report 03.12024.06 D-18
-------
was input (only one ic value coming to this Delta function again). The reference NTE event NOX
SS vector was recomputed with the one new ic value, returned to Methods worksheet and another
set of 213 output values was produced. This process was repeated many times until a statistical
convergence criterion, described in Section 2, was satisfied. Typically, 10,000 to 30,000 sets of
37 input values and 213 output values were produced to satisfy the convergence criterion with
this Error Model spreadsheet.
The number of rows in the Delta worksheet reference NTE event area (rows 80 and
down) should match the number of rows in the reference NTE event applied in the Methods
worksheet for proper function of Figures B and C. The starter spreadsheet has been set up with
the range of charted reference NTE event series extending through row 379 in this Delta
worksheet. The balance of the spreadsheet should calculate correctly when a reference NTE
event is properly entered in the Methods tab and Parameters formulas properly aligned, although
figures like B and C will not display properly until the last row of the reference NTE event is
coincident with the end of the range of the charted reference NTE event series. This could be
done manually in each Delta worksheet where needed, however, the macro was designed to
convert the fully 'loaded' starter workbook after the reference NTE event was entered in the
Methods worksheet. The macro uses the row count in the reference NTE event, aligns formulas
in the Methods worksheet Parameters area, and eliminates extra rows in the reference NTE event
area of each appropriate Delta worksheet. Again, the macro will do the operations correctly only
on a fully 'loaded' starter workbook set up with 300 rows of formulas in the Methods worksheet
Parameter area, and in each of the Delta worksheets using the reference NTE event.
One further detail in 1 Delta NOX SS is the value computed in cell E75 and its label in
E74. Each Delta worksheet has such a cell where the ic value is reflected. The intent is to
provide the user a further diagnostic capability through the user's designation of such cells as
outputs ("Forecasts") in Monte Carlo simulation. This would allow the user to statistically
process the input data with the same Crystal Ball (@Risk, etc.) tools used to process the outputs.
Crystal Ball allows extraction of all the input (Crystal Ball applies the term "Assumptions" to the
inputs) trial values without reflecting them as outputs ("Forecasts").
SwRI Report 03.12024.06 D-19
-------
APPENDIX E
40-POINT TORQUE AND BSFC MAP DATA
SwRI Report 03.12024.06
-------
Table Type
Engine Manufacturer
Engine Model
Model Year
Serial Number
Fuel Density
Peak Torque
nlo
nhi
Torque
DDC
Series 60
2005
06R0767368
851 [g/L]
2195 [N-m]
1014 [rpm]
2129 [rpm]
Point
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
Raw Data
ECM Speed
rpm
2152
2148
2142
2131
2095
2095
2095
2095
2095
2094
1866
1866
1867
1867
1867
1867
1867
1867
1636
1636
1636
1636
1636
1636
1636
1407
1407
1407
1407
1407
1407
1407
1407
1179
1179
1179
1178
1178
1178
1179
Lab Torque
N-m
541
632
764
1074
1599
1336
1074
811
658
548
548
658
811
1074
1336
1599
1735
1858
2052
1599
1336
1074
811
658
548
548
750
811
1074
1336
1599
1878
2136
2058
1599
1336
1074
896
811
548
Normalized Data
ECM Speed
nlo = 0%
nhi = 1 00%
102%
102%
101%
100%
97%
97%
97%
97%
97%
97%
76%
76%
76%
76%
76%
76%
76%
76%
56%
56%
56%
56%
56%
56%
56%
35%
35%
35%
35%
35%
35%
35%
35%
15%
15%
15%
15%
15%
15%
15%
ECM Fuel Rate
% of Maximum
Fuel Rate
34%
39%
46%
60%
97%
81%
66%
53%
44%
38%
33%
38%
46%
58%
73%
87%
95%
1 00%
94%
72%
60%
49%
38%
32%
28%
23%
30%
32%
42%
51%
60%
71%
84%
68%
53%
44%
36%
30%
27%
20%
Lab Torque
% of Peak
Torque
25%
29%
35%
49%
73%
61%
49%
37%
30%
25%
25%
30%
37%
49%
61%
73%
79%
85%
93%
73%
61%
49%
37%
30%
25%
25%
34%
37%
49%
61%
73%
86%
97%
94%
73%
61%
49%
41%
37%
25%
SwRI Report 03.12024.06
E-l
-------
Table Type
Engine Manufacturer
Engine Model
Model Year
Serial Number
Fuel Density
Peak Torque
nlo
nhi
BSFC
DDC
Series 60
2005
06R0767368
851 [g/L]
2195 [N-m]
1014 [rpm]
2129 [rpm]
Point
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
Raw Data
ECM Speed
rpm
2152
2148
2142
2131
2095
2095
2095
2095
2095
2094
1866
1866
1867
1867
1867
1867
1867
1867
1636
1636
1636
1636
1636
1636
1636
1407
1407
1407
1407
1407
1407
1407
1407
1179
1179
1179
1178
1178
1178
1179
Normalized Data
ECM Speed
nlo = 0%
nhi = 100%
1 02%
1 02%
101%
1 00%
97%
97%
97%
97%
97%
97%
76%
76%
76%
76%
76%
76%
76%
76%
56%
56%
56%
56%
56%
56%
56%
35%
35%
35%
35%
35%
35%
35%
35%
15%
15%
15%
15%
15%
15%
15%
ECM Fuel Rate
% of Maximum
Fuel Rate
34%
39%
46%
60%
97%
81%
66%
53%
44%
38%
33%
38%
46%
58%
73%
87%
95%
100%
94%
72%
60%
49%
38%
32%
28%
23%
30%
32%
42%
51%
60%
71%
84%
68%
53%
44%
36%
30%
27%
20%
BSFC
% of Max
100
96
92
85
81
82
85
90
96
106
95
91
88
84
83
82
81
81
79
81
77
79
82
86
89
85
81
80
77
80
75
76
77
77
77
78
78
79
80
86
SwRI Report 03.12024.06
E-2
-------
Table Type
Engine Manufacturer
Engine Model
Model Year
Serial Number
Fuel Density
Peak Torque
nlo
nhi
Torque
CAT
C9
2005
9DG05784
851 [g/L]
1464 [N-m]
1099 [rpm]
2320 [rpm]
Point
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
Raw Data
ECM Speed
rpm
2366
2348
2340
2325
2195
2195
2195
2194
2194
2194
1966
1966
1966
1966
1966
1966
1965
1966
1738
1738
1738
1738
1738
1738
1738
1510
1510
1510
1510
1510
1510
1510
1510
1281
1281
1281
1281
1281
1281
1281
Lab Torque
N-m
365
369
431
604
1043
874
705
535
439
366
366
439
535
705
874
1043
1135
1230
1372
1043
874
705
535
439
366
366
492
535
705
874
1043
1242
1436
1401
1043
874
705
580
535
366
Normalized Data
ECM Speed
nlo = 0%
nhi = 1 00%
104%
102%
102%
100%
90%
90%
90%
90%
90%
90%
71%
71%
71%
71%
71%
71%
71%
71%
52%
52%
52%
52%
52%
52%
52%
34%
34%
34%
34%
34%
34%
34%
34%
15%
15%
15%
15%
15%
15%
15%
ECM Fuel Rate
% of Maximum
Fuel Rate
55%
55%
59%
72%
1 00%
91%
74%
63%
57%
53%
41%
46%
52%
62%
74%
84%
93%
98%
88%
70%
63%
53%
48%
44%
41%
31%
37%
39%
47%
53%
57%
68%
78%
62%
51%
44%
37%
33%
32%
24%
Lab Torque
% of Peak
Torque
25%
25%
29%
41%
71%
60%
48%
37%
30%
25%
25%
30%
37%
48%
60%
71%
78%
84%
94%
71%
60%
48%
37%
30%
25%
25%
34%
37%
48%
60%
71%
85%
98%
96%
71%
60%
48%
40%
37%
25%
SwRI Report 03.12024.06
E-3
-------
Table Type
Engine Manufacturer
Engine Model
Model Year
Serial Number
Fuel Density
Peak Torque
nlo
nhi
BSFC
CAT
C9
2005
9DG05784
851 [g/L]
1464 [N-m]
1099 [rpm]
2320 [rpm]
Point
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
Raw Data
ECM Speed
rpm
2366
2348
2340
2325
2195
2195
2195
2194
2194
2194
1966
1966
1966
1966
1966
1966
1965
1966
1738
1738
1738
1738
1738
1738
1738
1510
1510
1510
1510
1510
1510
1510
1510
1281
1281
1281
1281
1281
1281
1281
Normalized Data
ECM Speed
nlo = 0%
nhi = 100%
104%
102%
102%
100%
90%
90%
90%
90%
90%
90%
71%
71%
71%
71%
71%
71%
71%
71%
52%
52%
52%
52%
52%
52%
52%
34%
34%
34%
34%
34%
34%
34%
34%
15%
15%
15%
15%
15%
15%
15%
ECM Fuel Rate
% of Maximum
Fuel Rate
55%
55%
59%
72%
100%
91%
74%
63%
57%
53%
41%
46%
52%
62%
74%
84%
93%
98%
88%
70%
63%
53%
48%
44%
41%
31%
37%
39%
47%
53%
57%
68%
78%
62%
51%
44%
37%
33%
32%
24%
BSFC
% of Max
100
99
95
87
77
82
83
87
93
98
90
86
81
78
76
73
74
72
69
71
74
76
81
84
89
85
80
79
75
73
69
69
68
67
70
71
74
77
79
80
SwRI Report 03.12024.06
E-4
-------
Table Type
Engine Manufacturer
Engine Model
Model Year
Serial Number
Fuel Density
Peak Torque
nlo
nhi
Torque
INT
VT365
2006
332259
851 [g/L]
681 [N-m]
1198 [rpm]
2839 [rpm]
Point
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
Raw Data
ECM Speed
rpm
2903
2890
2868
2832
2622
2621
2621
2621
2621
2620
2328
2328
2328
2328
2328
2328
2328
2328
2033
2033
2033
2032
2032
2032
2032
1739
1739
1739
1739
1739
1739
1739
1739
1444
1444
1443
1444
1444
1444
1444
Lab Torque
N-m
170
204
257
343
502
429
343
257
204
170
170
204
257
343
429
516
540
543
607
516
429
343
257
204
170
170
234
257
343
429
515
583
650
671
515
429
343
283
257
170
Normalized Data
ECM Speed
nlo = 0%
nhi = 1 00%
104%
103%
102%
100%
87%
87%
87%
87%
87%
87%
69%
69%
69%
69%
69%
69%
69%
69%
51%
51%
51%
51%
51%
51%
51%
33%
33%
33%
33%
33%
33%
33%
33%
15%
15%
15%
15%
15%
15%
15%
ECM Fuel Rate
% of Maximum
Fuel Rate
47%
53%
64%
81%
1 00%
88%
72%
58%
47%
41%
36%
41%
51%
63%
75%
87%
91%
92%
86%
74%
62%
52%
43%
36%
31%
26%
32%
34%
44%
52%
63%
68%
76%
64%
50%
42%
35%
30%
28%
21%
Lab Torque
% of Peak
Torque
25%
30%
38%
50%
74%
63%
50%
38%
30%
25%
25%
30%
38%
50%
63%
76%
79%
80%
89%
76%
63%
50%
38%
30%
25%
25%
34%
38%
50%
63%
76%
86%
96%
99%
76%
63%
50%
41%
38%
25%
SwRI Report 03.12024.06
E-5
-------
Table Type
Engine Manufacturer
Engine Model
Model Year
Serial Number
Fuel Density
Peak Torque
nlo
nhi
BSFC
INT
VT365
2006
332259
851 [g/L]
681 [N-m]
1198 [rpm]
2839 [rpm]
Point
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
Raw Data
ECM Speed
rpm
2903
2890
2868
2832
2622
2621
2621
2621
2621
2620
2328
2328
2328
2328
2328
2328
2328
2328
2033
2033
2033
2032
2032
2032
2032
1739
1739
1739
1739
1739
1739
1739
1739
1444
1444
1443
1444
1444
1444
1444
Normalized Data
ECM Speed
nlo = 0%
nhi = 100%
104%
103%
102%
100%
87%
87%
87%
87%
87%
87%
69%
69%
69%
69%
69%
69%
69%
69%
51%
51%
51%
51%
51%
51%
51%
33%
33%
33%
33%
33%
33%
33%
33%
15%
15%
15%
15%
15%
15%
15%
ECM Fuel Rate
% of Maximum
Fuel Rate
47%
53%
64%
81%
100%
88%
72%
58%
47%
41%
36%
41%
51%
63%
75%
87%
91%
92%
86%
74%
62%
52%
43%
36%
31%
26%
32%
34%
44%
52%
63%
68%
76%
64%
50%
42%
35%
30%
28%
21%
BSFC
% of Max
100
94
87
82
77
78
80
85
91
98
91
86
80
76
74
73
73
73
68
69
70
72
77
82
88
84
75
73
71
68
68
66
66
64
65
65
68
71
72
82
SwRI Report 03.12024.06
E-6
-------
APPENDIX F
STEADY-STATE ERROR SURFACE DATA
SwRI Report 03.12024.06
-------
I PEMS 5th % A PEMS 50th % • PEMS 95th % • Points
25
Concentration Delta (ppm)
-* -* N)
O1 O O1 O
-5
-10
10
15
.
24
33
160
180
200
220
240
260
280
300
Lab Reference Mean NOx Concentration (ppm)
PEMS NOX CONCENTRATION DELTA VERSUS THE LABORATORY REFERENCE
FOR ENGINE 1 STEADY-STATE TESTING
Raw 5th % A Raw 50th % * Raw 95th % • Points
25
? 20 -I
Q.
_a
2 15 -\
HI
Q
I 10 H
Concen
5
to
-5 ---
-10
10
M
9__
5
24 *
--3--
33
A
160
180
200
220
240
260
280
300
Lab Reference Mean NOx Concentration (ppm)
LAB RAW NOX CONCENTRATION DELTA VERSUS THE LABORATORY
REFERENCE FOR ENGINE 1 STEADY-STATE TESTING
SwRI Report 03.12024.06
F-l
-------
60
IPEMS 5th % A PEMS 50th % » PEMS 95th % • Points
50 -
Q.
S40
ra
"oi
Q
I30
"E
HI
u
J 20
O
O
10 --
10
A A
A A
37
»
A A
10
12
14 16 18 20
Lab Reference Mean CO Concentration (ppm)
22
24
PEMS CO CONCENTRATION DELTA VERSUS THE LABORATORY REFERENCE
FOR ENGINE 1 STEADY-STATE TESTING
60
I Raw 5th % A Raw 50th % » Raw 95th % • Points
50
E
Q.
Q.
S 30
HI
Q
I 20 Ji
HI
u
c
o
o
O
O
10
10
1524
40
27
33
37
-10
-20
\U i 14
16
i I1
20
Lab Reference Mean CO Concentration (ppm)
LAB RAW CO CONCENTRATION DELTA VERSUS THE LABORATORY
REFERENCE FOR ENGINE 1 STEADY-STATE TESTING
SwRI Report 03.12024.06
F-2
-------
I PEMS 5th % A PEMS 50th % * PEMS 95th % • Points
0.70
0.60
0.50
HI
Q
c
o
0.40
0.30
0.20
0.10
0.00
27
*to
15
37*3
A
56677889
Lab Reference Mean CO2 Concentration (%)
PEMS CO2 CONCENTRATION DELTA VERSUS THE LABORATORY REFERENCE
FOR ENGINE 1 STEADY-STATE TESTING
I Raw 5th % A Raw 50th % » Raw 95th % • Points
"oi
Q
O
is
c
0
o
CM
o °-20 "
n nn -
4 ; ;; "
* A
A
ill;
i *
37*3
5
A
i
"
5667788
Lab Reference Mean CO2 Concentration (%)
LAB RAW CO2 CONCENTRATION DELTA VERSUS THE LABORATORY
REFERENCE FOR ENGINE 1 STEADY-STATE TESTING
SwRI Report 03.12024.06
F-3
-------
I PEMS 5th % A PEMS 50th % » PEMS 95th % • Points
"g"
Q.
Q.
TO
Q
c
0
IS
ncentr;
O
O
I
S
z
40
A r\ r\
^ o
• n ^
1-5
-9 n
37 33 * 27* 24*
15 19 3 5
A
* ^. A
0 0.1 0.1 A o.2 A 0.2 A^ AH.3 0
• " • * "
Lab Reference Mean NMHC Concentration (ppm)
PEMS NMHC CONCENTRATION DELTA VERSUS THE LABORATORY
REFERENCE FOR ENGINE 1 STEADY-STATE TESTING
I PEMS 5th % A PEMS 50th % • PEMS 95th % • Points
30
25
20
15
10 H
5
3 1!
TO
•5 -5
-10
-15
-20
40
37
27 24
33
9-
250
350
450
550
650
750
850
950
1050
Lab Reference Mean Exhaust Flowrate (scfm)
PEMS EXHAUST FLOW RATE DELTA VERSUS THE LABORATORY REFERENCE
FOR ENGINE 1 STEADY-STATE TESTING
SwRI Report 03.12024.06
F-4
-------
15th % A 50th % 95th % • Points
4 -
I 2
o
3 1 -
40
37
27
24
33
10 9
15
i I :
I
250
350
450
550
650
750
850
950
1050
Lab Reference Mean Exhaust Flowrate (scfm)
LAB RAW EXHAUST FLOW RATE DELTA VERSUS THE LABORATORY
REFERENCE FOR ENGINE 1 STEADY-STATE TESTING
IPEMS 5th % A PEMS 50th % » PEMS 95th % « Points
70
60
_ 50
I
a 40
I 30
5
.2 20
10
0
-10
-20
40
37
33
_15_
200
400
600
800
1000
Lab Reference Mean NOx Mass Flow Rate (g/h)
PEMS NOX MASS FLOW RATE DELTA VERSUS THE LABORATORY REFERENCE
FOR ENGINE 1 STEADY-STATE TESTING
SwRI Report 03.12024.06
F-5
-------
Raw 5th % A Raw 50th % • Raw 95th % • Points
1
"oi
Q
HI
owRa
u.
to
to
TO
X
O
0
37t4 ; • *
T^ 9 3
ttf
I if r r I t
200 400 600 800
•
A
^000
Lab Reference Mean NOx Mass Flow Rate (g/h)
LAB RAW NOX MASS FLOW RATE DELTA VERSUS THE LABORATORY
REFERENCE FOR ENGINE 1 STEADY-STATE TESTING
SwRI Report 03.12024.06
F-6
-------
PEMS 5th % A PEMS 50th % • PEMS 95th % • Points
25
20 -
15
pp
_i
O
3 5
c
o
i °
" c
o -5
o
o
I -10
-15
-20
40 39
TO
.21- .20-
36
-1-9-
34
15CA 200 J50 3QB 350 400 450 500 5!
Lab Reference Mean NOx Concentration (ppm)
PEMS NOX CONCENTRATION DELTA VERSUS THE LABORATORY REFERENCE
FOR ENGINE 2 STEADY-STATE TESTING
Raw 5th % A Raw 50th % • Raw 95th % • Points
20 -
15
?
3 10
i"
& 5
c
o
« 0
r 100
o
O
x -10
O
-15
-20
40 39
TO
.21- .20-
36
-1-9-
34
• 15(§ ™ 200 250 300 350 400 450 500 5!
-*- 1
Lab Reference Mean NOx Concentration (ppm)
LAB RAW NOX CONCENTRATION DELTA VERSUS THE LABORATORY
REFERENCE FOR ENGINE 2 STEADY-STATE TESTING
SwRI Report 03.12024.06
F-7
-------
I PEMS 5th % A PEMS 50th % » PEMS 95th % • Points
80
70
60
50
40
30
20
10
0
-10
19
34
36 39 40
Lab Reference Mean CO Concentration (ppm)
PEMS CO CONCENTRATION DELTA VERSUS THE LABORATORY REFERENCE
FOR ENGINE 2 STEADY-STATE TESTING
Raw 5th % A Raw 50th % » Raw 95th % • Points
80
70
60
Q.
3 50
40
30
HI
Q
c
o
ra
HI
o
I 20
O
O
10
19
34
36 39
40
5 1 |
|6
-10
Lab Reference Mean CO Concentration (ppm)
LAB RAW CO CONCENTRATION DELTA VERSUS THE LABORATORY
REFERENCE FOR ENGINE 2 STEADY-STATE TESTING
SwRI Report 03.12024.06
F-8
-------
n en
CO2 Concentration Delta (%)
D O O O O O C
D ^ k> ijo !^ 01 i:
D O O O O O C
4
PEMS<
0.60
0.50
:= 0.40
ra
"oi
Q
c 0.30
0
TO
g 0.20
c
0
o
CM
8 0-1°
0.00
-n m
• PEMS 5th % A PEMS 50th % * PEMS 95th % • Points
* " • •
• 40 * 39 * 86 *34
10 13 21 20 g» 19
*
A
* * •
•
A *
•---*---: •
» A •
A
A A •
•
5566778899
Lab Reference Mean CO2 Concentration (%)
CO2 CONCENTRATION DELTA VERSUS THE LABORATORY REFERENCE
FOR ENGINE 2 STEADY-STATE TESTING
• Raw 5th % A Raw 50th % * Raw 95th % • Points
* • • •
. • 40 * 39 * 36 *34
10 13 21 2° 5 19
•
A
i
"*"" "•"
A
t " i i I
\ *5 m 6 6«7i7*8 8 9 !
Lab Reference Mean CO2 Concentration (%)
LAB RAW CO2 CONCENTRATION DELTA VERSUS THE LABORATORY
REFERENCE FOR ENGINE 2 STEADY-STATE TESTING
SwRI Report 03.12024.06
F-9
-------
IPEMS 5th % A PEMS 50th % » PEMS 95th % • Points
a.
o.
HI
Q
c
o
c
HI
o
c
o
o
o
I
-6*
-SrO"
40
39
-36--
-4r&
34
-SrO"
-2rO"
--ka
-OrS-
-0.8
-0.7
-0.6
10
-0.5
-0.4
-0.3
-0.2
-0.1
00
Lab Reference Mean NMHC Concentration (ppm)
PEMS NMHC CONCENTRATION DELTA VERSUS THE LABORATORY
REFERENCE FOR ENGINE 2 STEADY-STATE TESTING
I PEMS 5th % A PEMS 50th % PEMS 95th % • Points
50
40
£. 30
HI
Q
HI
20
10
-10
40 39
36
34
1ft)
21 299
•
5
250
350
450
550
650
750
Lab Reference Mean Exhaust Flowrate (scfm)
PEMS EXHAUST FLOW RATE DELTA VERSUS THE LABORATORY REFERENCE
FOR ENGINE 2 STEADY-STATE TESTING
SwRI Report 03.12024.06
F-10
-------
5.0
5th % A 50th % *95th % • Points
4.5
4.0
3.5
i 3.0
Q
HI
2.5
_o
u.
3
re
.c
2.0
1.5
1.0 -
0.5
0.0
40
39
36
34
150
250
350 450 550 650
Lab Reference Mean Exhaust Flowrate (scfm)
750
850
LAB RAW EXHAUST FLOW RATE DELTA VERSUS THE LABORATORY
REFERENCE FOR ENGINE 2 STEADY-STATE TESTING
70
£
3
re
a
HI
Q
HI
•re
OL
o
O
60
50
40
30
20
10
0
•10
-20
-30
PEMS 5th % A PEMS 50th % • PEMS 95th % • Points
40 39
36
~10~
_2J_
^20-
.34
-19-
200
400
600
800
1000
Lab Reference Mean NOx Mass Flow Rate (g/h)
PEMS NOX MASS FLOW RATE DELTA VERSUS THE LABORATORY REFERENCE
FOR ENGINE 2 STEADY-STATE TESTING
SwRI Report 03.12024.06
F-ll
-------
I Raw 5th % A Raw 50th % * Raw 95th % • Points
70
60
50
40
30
20
10
0
-10
-20
-30
40 39 . • 36
10 13--
-20-
.34
-19-
200
l~~l"
400
600
800
1000
I
Lab Reference Mean NOx Mass Flow Rate (g/h)
LAB RAW NOX MASS FLOW RATE DELTA VERSUS THE LABORATORY
REFERENCE FOR ENGINE 2 STEADY-STATE TESTING
SwRI Report 03.12024.06
F-12
-------
IPEMS 5th % A PEMS 50th % » PEMS 95th % • Points
40
20
I 0
f '
HI
Q -20
o
're
o
O
-60
-80
-100
25*
6
40
16 20 19
37 35
»»
/ A**
^A-
140
340
440
540
640
7.0
Lab Reference Mean NOx Concentration (ppm)
PEMS NOX CONCENTRATION DELTA VERSUS THE LABORATORY REFERENCE
FOR ENGINE 3 STEADY-STATE TESTING
I Raw 5th % A Raw 50th % » Raw 95th % • Points
40
20
I 0
c
o
o
-20
-60
-80
-100
2¥16 20 19
40
37 35
240
340
440
540
I 4
7. 0
Lab Reference Mean NOx Concentration (ppm)
LAB RAW NOX CONCENTRATION DELTA VERSUS THE LABORATORY
REFERENCE FOR ENGINE 3 STEADY-STATE TESTING
SwRI Report 03.12024.06
F-13
-------
I PEMS 5th % A PEMS 50th % » PEMS 95th % « Points
140
120 -
100 -
a.
80 -
I 60 4
o 40 -
o
O
£ 20 -
-20 -
-40
25*
--&-
16 20
40
19
37 35
140
^40
340
440
540
64fl
•
7. 0
Lab Reference NOx Concentration (ppm)
HORIBA OBS-2200 NOX CONCENTRATION DELTA VERSUS THE LABORATORY
REFERENCE FOR ENGINE 3 STEADY-STATE TESTING
I PEMS 5th % A PEMS 50th % » PEMS 95th % • Points
120
100 J
Q.
Q.
HI
Q
c
o
HI
o
c
o
o
o
O
80
60
40
20
25 22
19
•
37
35
10
12
14
16
13
-20
Lab Reference Mean CO Concentration (ppm)
PEMS CO CONCENTRATION DELTA VERSUS THE LABORATORY REFERENCE
FOR ENGINE 3 STEADY-STATE TESTING
SwRI Report 03.12024.06
F-14
-------
I Raw 5th % A Raw 50th % * Raw 95th % • Points
120
100
I 80 ^
"oi
Q 60
c
o
•£ 40
HI
u
c
o
o
O 20
O
25 22
.40
6
2?6
30
19
37
35
10
12
14
16
1 3
-20
Lab Reference Mean CO Concentration (ppm)
LAB RAW CO CONCENTRATION DELTA VERSUS THE LABORATORY
REFERENCE FOR ENGINE 3 STEADY-STATE TESTING
IPEMS 5th % A PEMS 50th % • PEMS 95th % • Points
150
100
50
a.
_a
c
o
c "
HI
o
c
o
o
O -50
O
-100
2
*
-40
*
37
35
10
12
14
16
13
Lab Reference CO Concentration (ppm)
HORIBA OBS-2200 CO CONCENTRATION DELTA VERSUS THE LABORATORY
REFERENCE FOR ENGINE 3 STEADY-STATE TESTING
SwRI Report 03.12024.06
F-15
-------
n en ^
CO2 Concentration Delta (%)
D O O O O O C
D ^ ki ijo !^ 01 i:
D O O O O O C
• PEMS 5th % A PEMS 50th % * PEMS 95th % • Points
25 37 22 ' 2016 3519 3°
b
^« *
* * A
• A A
AA B
A A A .. •
5.0 5.5 6.0 6.5 7.0 7.5 8.0 8.5 9.0
Lab Reference Mean CO2 Concentration (%)
PEMS CO2 CONCENTRATION DELTA VERSUS THE LABORATORY REFERENCE
FOR ENGINE 3 STEADY-STATE TESTING
n en ^
CO2 Concentration Delta (%)
D O O O O O C
D ^ ki ijo !^ 01 i:
D O O O O O C
• Raw 5th % A Raw 50th % * Raw 95th % • Points
• . * .
37 . 35 -,n
25 22 c 2016 19
* * * *
• • * AA Al •
• A 4 "• |
5.0 5.5 6.0 6.5 7.0 7.5 8.0 8.5 9.0
Lab Reference Mean CO2 Concentration (%)
LAB RAW CO2 CONCENTRATION DELTA VERSUS THE LABORATORY
REFERENCE FOR ENGINE 3 STEADY-STATE TESTING
SwRI Report 03.12024.06
F-16
-------
I PEMS 5th % A PEMS 50th % » PEMS 95th % « Points
1.0
0.8 -
0.6 -
IS °-4 -
c
o
is 0.2 -
0.0
40
-0.2 -
-0.4 -
-0.6 -
-0.8
40
25
37
22
35
2016 19
4.5
5.0
5.5
6.0
6.5
7.0
7.5
8.0
8.5
90
Lab Reference CO2 Concentration (%)
HORIBA OBS-2200 CO2 CONCENTRATION DELTA VERSUS THE LABORATORY
REFERENCE FOR ENGINE 3 STEADY-STATE TESTING
\ PEMS 5th % A PEMS 50th % * PEMS 95th % • Points
%
25 30 22
•
* 10-
A* n ^4
00*
• 0.0
.3 -0.2 -0.1 C
-rv^-
1 0-
1 5-
• •
40 35 37
5 19
6"
•
• •
* *
k A
A A ' A '
b • 0.1> 0.2 4l.3 0.4 0
"* •
Q.
0>
Q
c
o
-------
2.0
1.5
O
Q.
_a
C
O
c
HI
o
c
o
o
o
0.0
00
-0.5
PEMS 5th % A PEMS 50th % * PEMS 95th % • Points
•tn
30
3
19 2$
37
22
• •
0.5
1.0
1.5
Lab Reference THC Concentration (ppmC)
40
2.0
25
HORIBA OBS-2200 THC CONCENTRATION DELTA VERSUS THE LABORATORY
REFERENCE FOR ENGINE 3 STEADY-STATE TESTING
60
I PEMS 5th % A PEMS 50th % PEMS 95th % • Points
50 -
£. 40
re
a
HI
Q
HI
re 30
o
20 -
10 --
40
25
37
3530
22
• *
A>
20
19
16
150
200
250 300 350
Lab Reference Mean Exhaust Flowrate (scfm)
400
450
PEMS EXHAUST FLOW RATE DELTA VERSUS THE LABORATORY REFERENCE
FOR ENGINE 3 STEADY-STATE TESTING
SwRI Report 03.12024.06
F-18
-------
10
9
I 7
to
TO
? 6
Q
HI
2 5
o
il 4
1 3
15th % A 50th % 95th % • Points
1
40
25
37
3530
22
-20--
ft ft
-16--
150
200
250 300 350
Lab Reference Mean Exhaust Flowrate (scfm)
400
450
LAB RAW EXHAUST FLOW RATE DELTA VERSUS THE LABORATORY
REFERENCE FOR ENGINE 3 STEADY-STATE TESTING
u
to
0)
ra 1!
40
IPEMS 5th % A PEMS 50th % PEMS 95th % • Points
25
37
22_
_2Q_
-13-
200
A A
250
300
350
Lab Reference Exhaust Flowrate (scfm)
-16-
400
HORIBA OBS-2200 EXHAUST FLOW RATE DELTA VERSUS THE LABORATORY
REFERENCE FOR ENGINE 3 STEADY-STATE TESTING
SwRI Report 03.12024.06
F-19
-------
IPEMS 5th % APEMS 50th % *PEMS 95th % • Points
70
60 -
50 -
3 40 -
HI
Q
.2
re
OL
5
30 -
20 -
10 --
-10 --
-20 -
-30
25
30
40 .
_22-
--20-^6-
-19-
37
35
A A
53 100 150 200 250 300 350 400 450 5(
Lab Reference Mean NOx Mass Flow Rate (g/h)
PEMS NOX MASS FLOW RATE DELTA VERSUS THE LABORATORY REFERENCE
FOR ENGINE 3 STEADY-STATE TESTING
I Raw 5th % A Raw 50th % * Raw 95th % • Points
70
60 - --*>-- ~ -22-. --„-- --20-H6-
50
S 40
HI
Q
HI
•re
OL
o
30
20
10
-10
-20
-30
25
30
40
-19-
37
35
*• i 1 Ii^^i
53 100 150 200 250 300 350 • 400 45* 5i
Lab Reference Mean NOx Mass Flow Rate (g/h)
LAB RAW NOX MASS FLOW RATE DELTA VERSUS THE LABORATORY
REFERENCE FOR ENGINE 3 STEADY-STATE TESTING
SwRI Report 03.12024.06
F-20
-------
100
80
60
40
E 20
-20
-40 -
-60
25
100
I PEMS 5th % A PEMS 50th % » PEMS 95th % « Points
30
22
40
20 16
19
37
A A A
150 200 250
300
350 400
Lab Reference NOx Mass Flowrate (g/h)
450
5(10
HORIBA OBS-2200 NOX MASS FLOW RATE DELTA VERSUS THE LABORATORY
REFERENCE FOR ENGINE 3 STEADY-STATE TESTING
SwRI Report 03.12024.06
F-21
-------
•95th percentile —A—50th percentile (median) —•—5th percentile
40
20 "
Q.
n.
~
HI
Q
I -20
is
"E
HI
| -40 ^
O
11
200
400
500
600
LU
Q.
-60
-80
-100
Lab Reference Mean NOx Concentration [ppm]
ENGINE 1 ERROR SURFACE FOR STEADY-STATE NOX CONCENTRATION
-95th percentile —A—50th percentile (median) —•—5th percentile
-100
Lab Reference Mean NOx Concentration [ppm]
ENGINE 2 ERROR SURFACE FOR STEADY-STATE NOX CONCENTRATION
SwRI Report 03.12024.06
F-22
-------
•95th percentile —A—50th percentile (median) —•—5th percentile
-100
Lab Reference Mean NOx Concentration [ppm]
ENGINE 3 ERROR SURFACE FOR STEADY-STATE NOX CONCENTRATION
-95th percentile —A—50th percentile (median) —•—5th percentile
-100
Lab Reference Mean NOx Concentration [ppm]
FINAL ERROR SURFACE FOR STEADY-STATE NOX CONCENTRATION
SwRI Report 03.12024.06
F-23
-------
0.010
0.009 -
„ 0.008 -
= 0.007
HI
Q
0.006
0.005
0.004
0.003
0.002
0.001 -
0.000
0.0000
-95th percentile
-50th percentile (median)
-5th percentile
>-*-
-A-A
0.0005 0.0010 0.0015
Lab Reference Mean CO Concentration [%]
0.0020
0.0025
ENGINE 1 ERROR SURFACE FOR STEADY-STATE CO CONCENTRATION
-95th percentile
-50th percentile (median)
-5th percentile
0.010
0.009
„ 0.008
g
I °-007
Q
o 0.006
ra
| 0.005
o
c
O 0.004
O
W 0.003
0.002 -
0.001 -
0.000
0.0000
0.0005 0.0010 0.0015
Lab Reference Mean CO Concentration [%]
0.0020
0.0025
ENGINE 2 ERROR SURFACE FOR STEADY-STATE CO CONCENTRATION
SwRI Report 03.12024.06
F-24
-------
0.010
0.009 -
„ 0.008
= 0.007
HI
Q
o 0.006
0.005
u
c
O 0.004
O
W 0.003
0.002
0.001 -
0.000
0.0000
-95th percentile
-50th percentile (median)
-5th percentile
0.0005 0.0010 0.0015
Lab Reference Mean CO Concentration [%]
0.0020
0.0025
ENGINE 3 ERROR SURFACE FOR STEADY-STATE CO CONCENTRATION
0.010
0.009
„ 0.008
g
I °-007
Q
o 0.006
TO
| 0.005
o
c
O 0.004
O
W 0.003
s
LU
°- 0.002 -
0.001 -
0.000
0.0000
-95th percentile
-50th percentile (median)
-5th percentile
A A
MA A
0.0005 0.0010 0.0015
Lab Reference Mean CO Concentration [%]
•*—*
0.0020
0.0025
FINAL ERROR SURFACE FOR STEADY-STATE CO CONCENTRATION
SwRI Report 03.12024.06
F-25
-------
0.7
-95th percentile
•50th percentile (median)
-5th percentile
0.6
£ 0.5
c
o
''a 0.4
"E
HI
u
o 0.3
O
CM
O
O
> 0.2
LU
Q.
0.1
0.0
4.0 4.5 5.0 5.5 6.0 6.5 7.0 7.5
Lab Reference Mean CO2 Concentration [%]
8.0
8.5
9.0
ENGINE 1 ERROR SURFACE FOR STEADY-STATE CO2 CONCENTRATION
0.7
-95th percentile —A—50th percentile (median) —•—5th percentile
0.6
£ 0.5
HI
Q
c
o
is o.4
"c
HI
u
o 0.3
O
CM
O
O
> 0.2
LU
Q.
0.1 -
0.0
4.0 4.5 5.0 5.5 6.0 6.5 7.0 7.5
Lab Reference Mean CO2 Concentration [%]
8.0
8.5
9.0
ENGINE 2 ERROR SURFACE FOR STEADY-STATE CO2 CONCENTRATION
SwRI Report 03.12024.06
F-26
-------
0.7
-95th percentile
•50th percentile (median)
-5th percentile
0.6
£ 0.5
c
o
''a 0.4
"E
HI
u
o 0.3
O
CM
O
O
> 0.2
LU
Q.
0.1
0.0
4.0 4.5 5.0 5.5 6.0 6.5 7.0 7.5
Lab Reference Mean CO2 Concentration [%]
8.0
8.5
9.0
ENGINE 3 ERROR SURFACE FOR STEADY-STATE CO2 CONCENTRATION
0.7
-95th percentile —A—50th percentile (median) —•—5th percentile
0.6
£ 0.5
HI
Q
c
o
is o.4
"c
HI
u
o 0.3
O
CM
O
O
> 0.2
LU
Q.
0.1 -
0.0
4.0 4.5 5.0 5.5 6.0 6.5 7.0 7.5
Lab Reference Mean CO2 Concentration [%]
8.0
8.5
9.0
FINAL ERROR SURFACE FOR STEADY-STATE CO2 CONCENTRATION
SwRI Report 03.12024.06
F-27
-------
*95th percentile
A50th percentile (median)
• 5th percentile
0
Q.
£:
ra
"3
Q
c
o
"E
HI
c
o
o
s
(/)
LU
n
4
i
n n
>
1
Single X-Axis Point
FINAL ERROR SURFACE FOR STEADY-STATE NMHC CONCENTRATION
(ENGINE 2 DATA ONLY)
-95th percentile
-50th percentile (median)
-5th percentile
0 6
£
"oi
Q
I 4
&
O
2 -
UJ
Q.
-2
50
60
70
83
Lab Reference Mean Exhaust Flow Rate [% of Max]
ENGINE 1 ERROR SURFACE FOR STEADY-STATE EXHAUST FLOW RATE
SwRI Report 03.12024.06
F-28
-------
-95th percentile
-50th percentile (median)
-5th percentile
Lab Reference Mean Exhaust Flow Rate [% of Max]
ENGINE 2 ERROR SURFACE FOR STEADY-STATE EXHAUST FLOW RATE
-95th percentile
-50th percentile (median)
-5th percentile
8 -
0 6
"oi
Q
I
OL
o
2 -
LU 0
Q.
-2
10
20
30
40
50
60
70
Lab Reference Mean Exhaust Flow Rate [% of Max]
ENGINE 3 ERROR SURFACE FOR STEADY-STATE EXHAUST FLOW RATE
SwRI Report 03.12024.06
F-29
-------
-95th percentile
-50th percentile (median)
-5th percentile
Lab Reference Mean Exhaust Flow Rate [% of Max]
FINAL ERROR SURFACE FOR STEADY-STATE EXHAUST FLOW RATE
SwRI Report 03.12024.06
F-30
-------
APPENDIX G
TRANSIENT ERROR SURFACE DATA
SwRI Report 03.12024.06
-------
800
PEMS Median Flow-Weighted NOx (ppm)
ENGINE 1 ERROR SURFACE FOR TRANSIENT NOX CONCENTRATION
50
40
30
20
10
0
-10
-20
-30
-40
-50
- 95th % Delta
-5th% Delta
PEMS Median Flow-Weighted NOx (ppm)
ENGINE 2 ERROR SURFACE FOR TRANSIENT NOX CONCENTRATION
SwRI Report 03.12024.06
G-l
-------
800
PEMS Median Flow-Weighted NOx (ppm)
ENGINE 3 ERROR SURFACE FOR TRANSIENT NOX CONCENTRATION
PEMS Median Flow-Weighted NOx (ppm)
FINAL ERROR SURFACE FOR TRANSIENT NOX CONCENTRATION
SwRI Report 03.12024.06
G-2
-------
0.0080
-0.0025
PEMS Median Flow-Weighted CO Concentration (%)
ENGINE 1 ERROR SURFACE FOR TRANSIENT CO CONCENTRATION
0.0025 -,
0.0020
0.0015
3 0.0010
g 0.0005
I, o.oooo
0.0 )40
-0.0005 -
-0.0010
LU
Q.
-0.0015
-0.0020
-0.0025 J
- 95th % Delta
-5th% Delta
0.0045
0.0060
0.0065
0.0070
0.0075
0.0080
PEMS Median Flow-Weighted CO Concentration (%)
ENGINE 2 ERROR SURFACE FOR TRANSIENT CO CONCENTRATION
SwRI Report 03.12024.06
G-3
-------
0.0080
-0.0025
PEMS Median Flow-Weighted CO Concentration (%)
ENGINE 3 ERROR SURFACE FOR TRANSIENT CO CONCENTRATION
0.0080
-0.0025 J
PEMS Median Flow-Weighted CO Concentration (%)
FINAL ERROR SURFACE FOR TRANSIENT CO CONCENTRATION
SwRI Report 03.12024.06
G-4
-------
o
o
I
g>
I
i
0.5
0.4 --
0.3 -
0.2 -
0.1 --
0.0
4.0000
-0.1 ---
-0.2 -
LU
Q.
-0.3
-0.4 - -
-0.5
-95th % Delta —•— 5th % Delta
5.0000
8.0000 9.0000 10.0000 11.0000
PEMS Median Flow-Weighted CO2 Concentration (%)
ENGINE 1 ERROR SURFACE FOR TRANSIENT CO2 CONCENTRATION
11.0000
-0.5
PEMS Median Flow-Weighted CO2 Concentration (%)
ENGINE 2 ERROR SURFACE FOR TRANSIENT CO2 CONCENTRATION
SwRI Report 03.12024.06
G-5
-------
11.0000
-0.5
PEMS Median Flow-Weighted CO2 Concentration (%)
ENGINE 3 ERROR SURFACE FOR TRANSIENT CO2 CONCENTRATION
0.50
11.0000
-0.50
PEMS Median Flow-Weighted CO2 Concentration (%)
FINAL ERROR SURFACE FOR TRANSIENT CO2 CONCENTRATION
SwRI Report 03.12024.06
G-6
-------
-95th% Delta
-5th% Delta
-1-.5
-1-.9
-8.5
-GrS-
Engine 2 Median Flow-Weighted NMHC Concentration (ppmC)
FINAL ERROR SURFACE FOR TRANSIENT NMHC CONCENTRATION (ENGINE 2
DATA ONLY)
PEMS Median Exhaust Flow Rate (% of Max)
ENGINE 1 ERROR SURFACE FOR TRANSIENT EXHAUST FLOW RATE
SwRI Report 03.12024.06
G-7
-------
PEMS Median Exhaust Flow Rate (% of Max)
ENGINE 2 ERROR SURFACE FOR TRANSIENT EXHAUST FLOW RATE
PEMS Median Exhaust Flow Rate (% of Max)
ENGINE 3 ERROR SURFACE FOR TRANSIENT EXHAUST FLOW RATE
SwRI Report 03.12024.06
G-8
-------
PEMS Median Exhaust Flow Rate (% of Max)
FINAL ERROR SURFACE FOR TRANSIENT EXHAUST FLOW RATE
120
90 -
- 95th % Delta
-5th% Delta
200
400
600
800
1000
1200
1400
1600
1800
LU
Q.
-30 -
-60 -
-90 -
-120
PEMS Median NOx Mass Flow Rate (g/h)
ENGINE 1 ERROR SURFACE FOR TRANSIENT NOX MASS FLOW RATE
SwRI Report 03.12024.06
G-9
-------
120
1830
-120
PEMS Median NOx Mass Flow Rate (g/h)
ENGINE 2 ERROR SURFACE FOR TRANSIENT NOX MASS FLOW RATE
120
600
800
1000
1200
1400
1600
1800
-120
PEMS Median NOx Mass Flow Rate (g/h)
ENGINE 3 ERROR SURFACE FOR TRANSIENT NOX MASS FLOW RATE
SwRI Report 03.12024.06
G-10
-------
120
1830
-120
PEMS Median NOx Mass Flow Rate (g/h)
FINAL ERROR SURFACE FOR TRANSIENT NOX MASS FLOW RATE
2.0
1.5 -
1.0 --
0.5 -
re
'o
IS 0.0
~
o>
Q
I
OL
« -1.0 -
-95th % Delta —•— 5th % Delta
30
90
100
-0.5
-1.5 -
-2.0 -
-2.5 -
-3.0
PEMS Median Fuel Rate (% of Max)
ENGINE 1 ERROR SURFACE FOR TRANSIENT ECM FUEL RATE
SwRI Report 03.12024.06
G-ll
-------
2.0
1.5 -
1.0 --
0.5 -I
0.0
30
40
70
80
90
100
-0.5
-1.0 --
-1.5 -
-2.0 -
-2.5 -
-3.0
PEMS Median Fuel Rate (% of Max)
ENGINE 2 ERROR SURFACE FOR TRANSIENT ECM FUEL RATE
PEMS Median Fuel Rate (% of Max)
ENGINE 3 ERROR SURFACE FOR TRANSIENT ECM FUEL RATE
SwRI Report 03.12024.06
G-12
-------
2
HI
LU
Q.
0.8
0.6
0.4 -
0.2
PEMS Median Fuel Rate (% of Max)
FINAL ERROR SURFACE FOR TRANSIENT ECM FUEL RATE
g.
TO
&
1!
&
-95th % Delta —•— 5th % Delta
? '
I -0.2 4
OT
-0.4
-0.6
-0.8
PEMS Median Torque-Weighted Normalized Speed (%)
ENGINE 1 ERROR SURFACE FOR TRANSIENT ECM SPEED
SwRI Report 03.12024.06
G-13
-------
-0.8
0.8
0.6
0.4 -
0.2
PEMS Median Torque-Weighted Normalized Speed (%)
ENGINE 2 ERROR SURFACE FOR TRANSIENT ECM SPEED
&
1!
&
- 95th % Delta
- 5th % Delta
? c
I -0.2 4
OT
-0.4
-0.6
-0.8
PEMS Median Torque-Weighted Normalized Speed (%)
ENGINE 3 ERROR SURFACE FOR TRANSIENT ECM SPEED
SwRI Report 03.12024.06
G-14
-------
-0.8
PEMS Median Torque-Weighted Normalized Speed (%)
FINAL ERROR SURFACE FOR TRANSIENT ECM SPEED
95th % Delta —•— 5th % Delta
-4.0
PEMS Median Speed-Weighted Torque (% of Max)
ENGINE 1 ERROR SURFACE FOR TRANSIENT INTERPOLATED TORQUE
SwRI Report 03.12024.06
G-15
-------
-4.0
PEMS Median Speed-Weighted Torque (% of Max)
ENGINE 2 ERROR SURFACE FOR TRANSIENT INTERPOLATED TORQUE
95th % Delta —•— 5th % Delta
-4.0
PEMS Median Speed-Weighted Torque (% of Max)
ENGINE 3 ERROR SURFACE FOR TRANSIENT INTERPOLATED TORQUE
SwRI Report 03.12024.06
G-16
-------
-4.0
PEMS Median Speed-Weighted Torque (% of Max)
FINAL ERROR SURFACE FOR TRANSIENT INTERPOLATED TORQUE
95th % Delta —•— 5th % Delta
260
280
300
s::o
PEMS Median Fuel-Weighted BSFC (g/kW-hr)
ENGINE 1 ERROR SURFACE FOR TRANSIENT INTERPOLATED BSFC
SwRI Report 03.12024.06
G-17
-------
4.0
3.0 -
i-
1
Q 1.0
o
m
T3
I
0.0
-95th % Delta —•— 5th % Delta
200
280
|-1.0
300
s::o
-2.0
V V ^
LU
Q.
-3.0 -
-4.0
PEMS Median Fuel-Weighted BSFC (g/kW-hr)
ENGINE 2 ERROR SURFACE FOR TRANSIENT INTERPOLATED BSFC
95th % Delta —•— 5th % Delta
s::o
PEMS Median Fuel-Weighted BSFC (g/kW-hr)
ENGINE 3 ERROR SURFACE FOR TRANSIENT INTERPOLATED BSFC
SwRI Report 03.12024.06
G-18
-------
s::o
PEMS Median Fuel-Weighted BSFC (g/kW-hr)
FINAL ERROR SURFACE FOR TRANSIENT INTERPOLATED BSFC
SwRI Report 03.12024.06
G-19
-------
APPENDIX H
INTERACTING PARAMETERS - DOE ERROR SURFACE DATA
SwRI Report 03.12024.06
-------
uri- miei i emp aego — uri- uuuei i emp aego — ^sian IN i c ^.one - i argei co i
400
350
JJ 300
O)
HI
S 250
HI
3
1 200
Q.
|2 150
100
50
n
/
I
1
I
\\
1
TT*
t^
^*~- — i
^ ~[
_ _ — __ — — _^
r
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Time [s]
WARM-UP TEST TEMPERATURE PROFILES FOR ENGINE 1 AT WOT
2050
Lab Torque N-m
Bias Cor. Interp. Torque
1950
100 200 300 400 500 600 700 800 900 1000
1750
WARM-UP TEST LABORATORY AND INTERPOLATED TORQUE TRACES FOR
ENGINE 1 AT WOT
SwRI Report 03.12024.06
H-l
-------
230
220
i
S 210
O
u.
(/)
m
200
190
1RD .
u f
ftUWL..
?H^W^nh\»4*'WV*(^
0 100 200 300 400 500 600 700 800 900 1000
Time [s]
WARM-UP TEST LABORATORY AND INTERPOLATED BSFC TRACES FOR
ENGINE 1 AT WOT
450
- DPF Inlet Temp degC DPF Outlet Temp degC Start NTE Zone
•Actual ECT - CAN Target IMT Actual IMT - CAN
-Target ECT
400
350
300
a 250
I 200 -
150 -
100
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Time [s]
WARM-UP TEST TEMPERATURE PROFILES FOR ENGINE 1 AT PART LOAD
SwRI Report 03.12024.06
H-2
-------
1850
NTE Zone Bias Cor. Interp. Torque
100 200 300 400 500 600 700 800 900 1000
1550
WARM-UP TEST LABORATORY AND INTERPOLATED TORQUE TRACES FOR
ENGINE 1 AT PART LOAD
240
Lab Dilute CB BSFC g/kW-h Bias Cor. Interpolated BSFC Start NTE Zone
100 200 300 400 500 600 700 800 900 1000
180
WARM-UP TEST LABORATORY AND INTERPOLATED BSFC TRACES FOR
ENGINE 1 AT PART LOAD
SwRI Report 03.12024.06
H-3
-------
-DPF Inlet Temp degC
• DPF Outlet Temp degC
•Start NTE Zone
600
500
-- 400
O)
u
2,
u
= 300
0)
200
100
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Time [s]
WARM-UP TEST TEMPERATURE PROFILES FOR ENGINE 2 AT WOT
1650
1550 -
1450 -
1350 -
a
a. 1250
O"
1150
1050 -
950
•Lab Torque N-m Bias Cor. Interp. Torque ^^~Start NTE Zone
0 100 200 300 400 500 600 700 800 900 1000
Time[s]
WARM-UP TEST LABORATORY AND INTERPOLATED TORQUE TRACES FOR
ENGINE 2 AT WOT
SwRI Report 03.12024.06
H-4
-------
260
Lab Dilute CB BSFC g/kW-h Bias Cor. Interpolated BSFC ^—Start NTE Zone
100 200 300 400 500 600 700 800 900 1000
200
WARM-UP TEST LABORATORY AND INTERPOLATED BSFC TRACES FOR
ENGINE 2 AT WOT
-DPF Inlet Temp degC
-DPF Outlet Temp degC
•Start NTE Zone
500
450
400
350
O
250
200
150
100
50
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Time [s]
WARM-UP TEST TEMPERATURE PROFILES FOR ENGINE 2 AT PART LOAD
SwRI Report 03.12024.06
H-5
-------
1300
Lab Torque N-m Bias Cor. Interp. Torque ^^—Start NTE Zone
100 200 300 400 500 600 700 800 900 1000
600
WARM-UP TEST LABORATORY AND INTERPOLATED TORQUE TRACES FOR
ENGINE 2 AT PART LOAD
Lab Dilute CB BSFC g/kW-h Bias Cor. Interpolated BSFC ^—Start NTE Zone
100 200 300 400 500 600 700 800 900 1000
190
180
WARM-UP TEST LABORATORY AND INTERPOLATED BSFC TRACES FOR
ENGINE 2 AT PART LOAD
SwRI Report 03.12024.06
H-6
-------
500
-DPF Inlet Temp degC
-Target IMT
-DPF Outlet Temp degC
-Actual IMT - CAN
-Target ECT
•Start NTE Zone
Actual ECT - CAN
450
400
O
350 - - - /
300 -
1
HI
Q.
HI
250 - - -
200 H
150 H
100
50 "
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Time [s]
WARM-UP TEST TEMPERATURE PROFILES FOR ENGINE 3 AT WOT
1000
900
800
700 -
a 600
O"
£
500
400
300
200
•Lab Torque N-m Bias Cor. Interp. Torque ^^~Start NTE Zone
\
100 200 300 400 500 600 700 800 900 1000
Time[s]
WARM-UP TEST LABORATORY AND INTERPOLATED TORQUE TRACES FOR
ENGINE 3 AT WOT
SwRI Report 03.12024.06
H-7
-------
300
280
260
i
S 240 -
O
u.
220
200
180
•Lab Dilute CB BSFC g/kW-h Bias Cor. Interpolated BSFC Start NTE Zone
0 100 200 300 400 500 600 700 800 900 1000
Time [s]
WARM-UP TEST LABORATORY AND INTERPOLATED BSFC TRACES FOR
ENGINE 3 AT WOT
-DPF Inlet Temp degC DPF Outlet Temp degC Target ECT
• Target IMT Actual IMT - CAN Start NTE Zone
450
Actual ECT - CAN
350
JJ 300
- 250
200
150
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Time [s]
WARM-UP TEST TEMPERATURE PROFILES FOR ENGINE 3 AT PART LOAD
SwRI Report 03.12024.06
H-8
-------
800
700 -
600
f
a
o 500
O"
400
300
200
I
• Lab Torque N-m Bias Cor. Interp. Torque ^^—Start NTE Zone
W/vwvfl'vA
i^i»ii^A»^^«'*«»fliM*»VVM*<^*
-------
APPENDIX I
INTERPOLATION TORQUE AND BSFC ERROR MAPS
SwRI Report 03.12024.06
-------
F
u
e
I
r
a
t
e
80-
60-
40-
20-
1200 1400 1600 1800 2000
Sfceed rpm
1.3466
0.7684
0.1902
-0.3880
-0.9662
-1.5444
-2.1226
-2.7007
-3.2789
-3.8571
Delta Tore
INTERPOLATED TORQUE ERROR (% PEAK TORQUE) BY SPEED (RPM)
AND FUEL RATE (G/S) FOR ENGINE #1
F
u
e
I
r
a
t
e
60-
40-
20-
1400 1600 1800 2000 2200
Speed rpm
PI 4.6530
3.8707
3.0884
2.3061
1.5238
0.7414
-0.0409
-0.8232
-1.6055
-2.3878
Delta Tore
INTERPOLATED TORQUE ERROR (% PEAK TORQUE) BY SPEED (RPM)
AND FUEL RATE (G/S) FOR ENGINE #2
SwRI Report 03.12024.06
1-1
-------
F
u
e
I
r
a
t
e
40-
30-
20-
10-
1500
2000 2500
Speed rpm
2.5072
1.9951
1.4830
0.9710
0.4589
-0.0532
-0.5652
1-1.0773
-1.5893
-2.1014
Delta Tern
INTERPOLATED TORQUE ERROR (% PEAK TORQUE) BY SPEED (RPM)
AND FUEL RATE (G/S) FOR ENGINE #3
F
u
e
I
r
a
t
e
80-
60-
40-
20-
1200 1400 1600 1800 2000
SjDeed rpm
16.8087
8.7276
0.6465
-7.4346
-15.5157
-23.5968
-31.6780
-39.7591
-47.8402
-55.9213
Delta BSFC
INTERPOLATED BSFC ERROR (G/KW-HR) BY SPEED (RPM) AND FUEL
RATE (G/S) FOR ENGINE #1
SwRI Report 03.12024.06
1-2
-------
F
u
e
I
r
a
t
e
60-
40-
20-
1400 1600 1800 2000 2200
Speed rpm
12.449
10.271
8.0932
5.9149
3.7366
1.5583
-0.6200
-2.7983
-4.9765
-7.1548
Delta BSFC
INTERPOLATED BSFC ERROR (G/KW-HR) BY SPEED (RPM) AND FUEL
RATE (G/S) FOR ENGINE #2
F
u
e
I
r
a
t
e
40-
30-
20-
10-
1500
2000 2500
SjDeed rpm
Fll5.9089
14.1637
12.4185
10.6733
8.9282
7.1830
5.4378
! 3.6926
1.9474
0.2022
Del BSFC
INTERPOLATED BSFC ERROR (G/KW-HR) BY SPEED (RPM) AND FUEL
RATE (G/S) FOR ENGINE #3
SwRI Report 03.12024.06
1-3
-------
APPENDIX J
EVALUATION OF MANUFACTURER SUPPLIED ERROR SURFACES
SwRI Report 03.12024.06
-------
August 28, 2006
From: Matt Spears, US EPA
To: Measurement Allowance Steering Committee
Subject: Evaluation of Manufacturer-Submitted Error Surfaces
Background
In May 2005 EPA, CARB, and EMA signed a Memorandum of Agreement
(MOA), which among other things, described how to evaluate manufacturer voluntary
submissions of data that demonstrated non-deficiency AECD effects or production
variability effects on the ability to estimate NTE torque/BSFC values from ECM
parameters, using prescribed mapping procedures. In the Agreement it stated that, "EPA
and CARB, in consultation with HDOH engine manufacturers, will utilize this
information, if reasonably common among manufacturers, to determine and include a
margin component in the error model that accounts for the variability in the torque/BSFC
values used in the NTE brake-specific emission calculations. For example, EPA/CARB
would consider information for an additional allowance if variability due to non-
deficiency AECDs is consistent across manufacturers. If variability is inconsistent and
infrequent across the submissions or if there is a consistent bias, EPA and CARB would
expect manufacturers to account for these errors by creating more sophisticated
algorithms that decrease the infrequent large deviations or account for the consistent bias
that exists across manufacturers."
Methods
Five engine manufacturers submitted data to EPA and CARB for both torque and
BSFC. Three of the five manufacturers submitted error surface data that summarized
their total error of all of the component errors that they considered. The other two
manufacturers submitted data that depicted various sources of error, such as production
variability, deterioration, and ECM algorithm accuracy. For these two manufacturers
who did not send actual error surface data, I constructed or approximated error surfaces
so that the error surfaces from all manufacturers could be compared. For error types (e.g.
production variability) where I received data from several engines from a given
manufacturer, I determined the 5th, 50th, and 95th percentile errors from each engine's data
set, and then I averaged the percentile results across multiple engines because there was
almost no scatter of percentile errors between engines. Once I had single 5th 50th and 95th
percentile errors for each type of error, I added the 5th, 50th and 95th percentiles for each
type of error, respectively, to arrive at "worst case" composite errors. I used these results
to create error surfaces for torque and BSFC. Note that I did not take an RSS approach to
summing these error types.
To analyze the consistency of the various submitted error surfaces across
manufacturers, according to the MOA, I examined the error surfaces for bias (50th
percentile values), skew (symmetry of 5th and 95th percentile values), dependency of error
SwRI Report 03.12024.06 J-l
-------
magnitude as a function of level (torque, BSFC), and the magnitude of individual
manufacturer's error surfaces versus the average of the remaining manufacturer error
surfaces (outliers).
-,th
Observations
1. There was no consistent evidence of bias. Nearly all of the 50m percentile errors
were zero or near zero for both torque and BSFC from all manufacturers.
2. There was no consistent evidence of skew. Most of the manufacturers reported
symmetrical values (but opposite sign) for the 5th and 95th percentile values for
both torque and BSFC.
3. There was no consistent evidence of %-error being a function of level. One
individual manufacturer reported significant BSFC %-error changes as a function
of level. However, the magnitudes of the BSFC errors from that manufacturer
were also very inconsistent (an order of magnitude higher) compared to the other
four manufacturers' BSFC error data. When data from all five manufacturers was
aggregated, there was no correlation of torque %-error versus torque or—for the
four consistent manufacturers' error surfaces—BSFC %-error versus BSFC.
4. Because there was no consistent bias, skew, or dependency of %-error as a
function of level, I examined the average values of the manufacturers' 5th and 95th
percentile values. For torque the average value for all five manufacturers was -
5.68% for the 5th percentile value and +5.68% for the 95th percentile value. For
BSFC, I discarded one manufacturer's data based upon good engineering
judgment and based upon the terms of the MOA because every data point on the
error surface was over ten times the magnitude of any other manufacturer's
corresponding BSFC error. This manufacturer's BSFC data was deemed
inconsistent with the four other manufacturer's data. For BSFC the average value
for the four remaining manufacturers was -1.35% for the 5th percentile value and
+1.35% for the 95th percentile value.
Conclusions
Based upon my analysis, and in accordance with the terms of the Memorandum of
Agreement and associated Test Plan, I conclude that the following error surfaces should
be added to the error model to account for the consistent torque and BSFC error depicted
in the manufacturer-submitted data:
Parameter
Torque
BSFC
Sampling Distribution
Normal, once per NTE
Normal, once per NTE
5* percentile
-5. 7% of point
-1.4% of point
50th percentile
0
0
95th percentile
+5. 7% of point
+1.4% of point
Respectfully Submitted,
Matthew W Spears
SwRI Report 03.12024.06
J-2
-------
November 2, 2006
From: Matt Spears, US EPA
To: Measurement Allowance Steering Committee
Subject: Re-Evaluation of Manufacturer-Submitted Error Surfaces
Background
On August 28, 2006 I submitted a memo to the steering committee indicating the
results of my analysis of the manufacturer-submitted error surface data. This original
memo is attached for reference. Subsequently, we (the steering committee) had a
discussion of my original memo, and as a follow-up, the EMA members of the steering
committee requested an opportunity to submit to EPA/CARB some additional data.
Specifically, they requested to submit data that included the torque and BSFC ECM
errors due to non-deficiency AECD operation. I agreed to contact those engine
manufacturers that had not originally submitted such information. Ultimately, two of
those engine manufacturers submitted additional information.
Methods and Observations
One company, Cummins, Inc., submitted a comprehensive analysis of all the
errors that affect ECM torque and BSFC prediction. The Cummins submission was the
only data that I received, either originally or in this second round of data submission, that
contained a direct comparison of ECM-derived BSFC versus lab-determined BSFC.
Furthermore, the Cummins submission included a thorough statistical analysis of all of
the sources of error, including data previously submitted.
The other company submitted SET data for ECM torque and measured torque at
two different modes of engine operation. Based upon a discussion with that company, I
determined that one of the modes of operation was the mode from which ECM torque is
derived. Since the other mode of operation was of lower engine efficiency, ECM torque
and ECM derived BSFC reported during the less efficient mode of operation would result
in a consistent and significant low bias of reported emissions. Because this approach to
ECM torque programming might not be consistent among manufacturers, I was reluctant
to include the low bias indicated from this data. After correcting for the low bias, the
resulting torque error differed from the Cummins data by less than one percentage point.
I also reanalyzed the data from the one company that originally submitted non-
deficiency AECD data. This company's BSFC data was initially discarded as part of my
original analysis, which I explained in my original memo. This dataset only presented
data indicating deviations of torque/BSFC between modes of engine operation, and did
not identify which mode(s) of operation were used to determine ECM torque or BSFC.
This dataset also included several non-NTE data points. After filtering the non NTE data
from this data set, the resulting BSFC 5th and 95th percentage points were roughly
reduced by half. Because I could not identify which mode(s) of engine operation were
SwRI Report 03.12024.06 J-3
-------
used for ECM calibration, I could not subtract any bias from this spread. Even without
subtracting any bias, the resulting BSFC errors differed from the Cummins analysis by
less than two percentage points.
Conclusion
Based upon all of the data submitted, and in accordance with the terms of the
Memorandum of Agreement and associated Test Plan, I conclude that the error surfaces
proposed by Cummins should be included in the error model to account for the consistent
torque and BSFC error depicted in the manufacturer-submitted data. The Cummins
dataset and analysis is comprehensive and it imposes no bias on the error surface.
Furthermore, other less complete datasets indicate errors very close to those reported by
Cummins. Below are the revised values for the error surfaces:
Parameter
Torque
BSFC
Sampling Distribution
Normal, once per NTE
Normal, once per NTE
5th percentile
-6. 5% of point
-5. 9% of point
50th percentile
0
0
95th percentile
+6. 5% of point
+5. 9% of point
For a comparison of these values based upon the original data submission, please refer to
my original memo, attached.
Respectfully Submitted,
Matthew W Spears
End.
SwRI Report 03.12024.06
J-4
-------
APPENDIX K
ENVIRONMENTAL CHAMBER TESTING RESULTS AND ERROR SURFACES
SwRI Report 03.12024.06
-------
THC[ppmC] • CO [ppm] NO [ppm] X NO2 [ppm] • NOx [ppm] New FID Bottle A CO2 [%]
0.030
Reference Gas Delta Observation
PEMS 2 ENVIRONMENTAL BASELINE ZERO DELTA MEASUREMENTS
•
60
8
THC[ppmC] • CO [ppm] NO [ppm] X NO2 [ppm] • NOx [ppm] New FID Bottle A CO2 [%]
ppm
S
§
A ---**-*'
- A~A A -AA-
I A
« »»»«»•»«»<•••••»» »«>••»«»»»»•»••••
0.000
0.010
0.020 rt
g.
s
-0.030 g
p
trat
p
Con
p
-^r -0.070
-0.080
Reference Gas Delta Observation
PEMS 2 ENVIRONMENTAL BASELINE AUDIT DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-l
-------
THC [ppmC]
CO [ppm]
NO [ppm] X NO2 [ppm]
NOx [ppm]
-New FID Bottle A CO2 [%]
0.080
60 0.060
-0.040
Reference Gas Delta Observation
PEMS 2 ENVIRONMENTAL BASELINE SPAN DELTA MEASUREMENTS
» THC [ppmC] • CO [ppm] NO [ppm] X NO2 [ppm]
NOx [ppm]
-New FID Bottle A CO2 [%]
0.015
-0.009
Reference Gas Delta Observation
PEMS 3 ENVIRONMENTAL BASELINE ZERO DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-2
-------
THC [ppmC] • CO [ppm] NO [ppm] X NO2 [ppm]
NOx [ppm]
-New FID Bottle A CO2 [%]
0.060
20 - 0.048
-0.036
Reference Gas Delta Observation
PEMS 3 ENVIRONMENTAL BASELINE AUDIT DELTA MEASUREMENTS
» THC[ppmC] • CO [ppm] X NO [ppm] X NO2 [ppm] • NOx [ppm] New FID Bottle A CO2 [%]
20
»^^*s«^
• . AA* >. «A.
Reference Gas Delta Observation
PEMS 3 ENVIRONMENTAL BASELINE SPAN DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-3
-------
« THC [ppmC]
• CO [ppm]
NO [ppm]
A CO2 [%]
X NO2 [ppm]
• NOx [ppm]
0.030
-0.010
Reference Gas Delta Observation
PEMS 5 ENVIRONMENTAL BASELINE ZERO DELTA MEASUREMENTS
THC [ppmC] • CO [ppm] NO [ppm]
-Day Break —New FID Bottle A CO2 [%]
X NO2 [ppm]
NOx [ppm]
70
60
Q
c
o
Reference Gas Delta Observation
PEMS 5 ENVIRONMENTAL BASELINE AUDIT DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-4
-------
THC [ppmC] • CO [ppm] NO [ppm] X NO2 [ppm]
-Day Break New FID Bottle A CO2 [%]
NOx [ppm]
Reference Gas Delta Observation
PEMS 5 ENVIRONMENTAL BASELINE SPAN DELTA MEASUREMENTS
5.0 i-
4.5 -
4.0 -
•g- 3.5 --
^ 3.0 -
re
£
§ 2.5 -
u.
« 2.0 -
re
^
uj 1.5
1.0^
0.5 -
0.0 -L
» »
*
**
• «
50
100 150 200 250
Exhaust Flow Meter Zero Observation
300
350
5-INCH EFM ENVIRONMENTAL BASELINE ZERO DELTA
MEASUREMENTS
SwRI Report 03.12024.06
K-5
-------
THC [ppmC]
New FID Bottle
CO [ppm] X NO [ppm]
Temp. [degC] A CO2 [%]
X NO2 [ppm]
NOx [ppm]
0.014
-0.004
Reference Gas Delta Observation
PEMS 2 ENVIRONMENTAL TEMPERATURE ZERO DELTA
MEASUREMENTS
THC [ppmC]
-New FID Bottle
CO [ppm]
Temp. [degC]
NO [ppm]
A CO2 [%]
X NO2 [ppm]
NOx [ppm]
0.00
O
o>
m
T3
I
^
o
O
vt»^b8|.tVMMM»^5!*!:>A^AV - - V - A*
-*-- A * ArA4r-l-AA A ^ A ^ —
0.08
-0.09
Reference Gas Delta Observation
PEMS 2 ENVIRONMENTAL TEMPERATURE AUDIT DELTA
MEASUREMENTS
SwRI Report 03.12024.06
K-6
-------
THC [ppmC] • CO [ppm] NO [ppm]
New FID Bottle Temp. [degC] A CO2 [%]
X NO2 [ppm]
NOx [ppm]
0.100
-0.050
Reference Gas Delta Observation
PEMS 2 ENVIRONMENTAL TEMPERATURE SPAN DELTA
MEASUREMENTS
THC [ppmC] • CO [ppm] NO [ppm]
New FID Bottle Temp. [degC] A CO2 [%]
X NO2 [ppm]
NOx [ppm]
0.025
-0.010
Reference Gas Delta Observation
PEMS 3 ENVIRONMENTAL TEMPERATURE ZERO DELTA
MEASUREMENTS
SwRI Report 03.12024.06
K-7
-------
THC [ppmC] • CO [ppm] NO [ppm]
New FID Bottle Temp. [degC] A CO2 [%]
X NO2 [ppm]
NOx [ppm]
50
-30
Reference Gas Delta Observation
PEMS 3 ENVIRONMENTAL TEMPERATURE AUDIT DELTA
MEASUREMENTS
THC [ppmC] • CO [ppm] NO [ppm]
New FID Bottle Temp. [degC] A CO2 [%]
X NO2 [ppm]
NOx [ppm]
0.050
-0.090
Reference Gas Delta Observation
PEMS 3 ENVIRONMENTAL TEMPERATURE SPAN DELTA
MEASUREMENTS
SwRI Report 03.12024.06
K-8
-------
» THC [ppmC]
• NOx [ppm]
CO [ppm]
- Break
X NO [ppm]
Temp. [degC]
X NO2 [ppm]
A CO2 [%]
-40
Reference Gas Delta Observation
PEMS 5 ENVIRONMENTAL TEMPERATURE ZERO DELTA
MEASUREMENTS
« THC [ppmC]
• NOx [ppm]
• CO [ppm]
NO [ppm]
Temp. [degC]
X NO2 [ppm]
A CO2 [%]
-r 0.000
-0.150
Reference Gas Delta Observation
PEMS 5 ENVIRONMENTAL TEMPERATURE AUDIT DELTA
MEASUREMENTS
SwRI Report 03.12024.06
K-9
-------
• THC [ppmC]
• NOx [ppm]
• CO [ppm]
NO [ppm]
Temp. [degC]
X NO2 [ppm]
A CO2 [%]
120
100
80
60
40
20
0
-20
-40
A A*4 A
AA AAA /A,
A A A"
A>A A
Reference Gas Delta Observation
PEMS 5 ENVIRONMENTAL TEMPERATURE SPAN DELTA
MEASUREMENTS
5.0
4.5 -
4.0 -
|-3.5f
y,
|
£
I2'5
u.
« 2.0
3.0 -
" , _
iu 1.5
1.0 -
0.5 -
0.0
* *
>V -->-
**^ » UL* » «
50
100 150 200 250
Exhaust Flow Meter Zero Observation
300
350
5-INCH EFM ENVIRONMENTAL TEMPERATURE ZERO DELTA
MEASUREMENTS
SwRI Report 03.12024.06
K-10
-------
- PEMS 2 * PEMS 3 PEMS 5
25 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
Reference Gas Delta Observation
ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE NOX
CONCENTRATION ZERO DELTA MEASUREMENTS
15
10 --
5 -
- PEMS 2 • PEMS 3 • PEMS 5
0 -
-5 ---
Q
c
o
B
re
m
o
c
o
o
o -10 -
z
-15 --
. -_*.--_
-20 H—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—
0 10 20 30 40 50 60 70
Reference Gas Delta Observation
ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE NOX
CONCENTRATION AUDIT DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-ll
-------
40
30 -
- PEMS 2 • PEMS 3 • PEMS 5
I 20
Q.
10 --
2
o
O
-10 --
-20 -
-30 H—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I
0 10 20 30 40 50 60 70
Reference Gas Delta Observation
ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE NOX
CONCENTRATION SPAN DELTA MEASUREMENTS
40
30 --
I 20
£
° 10
£
0 ---
-10 --a
-20 --
-30
200
400 600 800
NOx Temperature Observation (1 to 1080)
1000
-NOx SPAN 1209ppm NOx
• NOx Audit 341 ppm NOx
- NOx Zero 0 ppm NOx
FINAL ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE NOX
CONCENTRATION
SwRI Report 03.12024.06
K-12
-------
I 1 1 1 1 1 1 1 1 1 1 1 1 1
-0.0020
100 150
Reference Gas Delta Observation
200
ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE CO
CONCENTRATION ZERO DELTA MEASUREMENTS
0.0020
0.0015 -
0.0010 -
0.0005 -
= 0.0000 -
§ -0.0005 -
-0.0010 -
- PEMS 2 • PEMS 3 • PEMS 5
»»**
* *
* **
-0.0015 -
-0.0020 -
-0.0025 ^—I—I—I—I-
10
20 30 40 50
Reference Gas Delta Observation
60
70
ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE CO
CONCENTRATION AUDIT DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-13
-------
0.0070
- PEMS 2 • PEMS 3 • PEMS 5
0.0060
0.0050 f - -m --
£ • i
re 0.0040
I
g 0.0030 -
"re
| 0.0020
o
c
" 0.0010 4-'
o
o
0.0000
-0.0010 -
-0.0020 H—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—h
0 10 20 30 40 50 60 70
Reference Gas Delta Observation
ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE CO
CONCENTRATION SPAN DELTA MEASUREMENTS
0.007
0.006
0.005 -
P| 0.004
«
g 0.003 -
3 0.002
^ 0.001 -I-- --«---.-- ^f r --Ji
80 - - ••'
200 «DO . »f " 600 85S ^tQDO
-0.001 -I- -*X--*-*
-0.002 -
-0.003
CO Temperature Observation [1 to 1080]
-COSpan 0.0951% CO
• CO Audit 0.0179% CO
- CO Zero 0% CO
FINAL ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE CO
CONCENTRATION
SwRI Report 03.12024.06
K-14
-------
0.015
-0.020
100 150
Reference Gas Delta Observation
200
ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE CO2
CONCENTRATION ZERO DELTA MEASUREMENTS
CO2 Concentration Delta [%]
0.08
0.06
0.04
0.02
0.00
-0.02
-0.04
-PEMS2 »PE
MS 3 BPEMS5
" % "V1^.- •+. v"
•
^m *
jf? "^
- ***** *• -+^*^ ~- - "*~ ^" * ^ ^
• * *
0 10 20 30 40 50 60 70
Reference Gas Delta Observation
ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE CO2
CONCENTRATION AUDIT DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-15
-------
CO2 Concentration Delta [%]
DpppppOC
-* o o o — * — * Kj K
DU10U10U10C
-PEMS 2 »PE
MS 3 • PEMS 5
•J"-.'-.'' f+ *f '.
— " •
*
•
** • • ** **_^~ A^^ ~ w ^9*m »^~~ ~~
•
g
|M»**% "
0 10 20 30 40 50 60 70
Reference Gas Delta Observation
ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE CO2
CONCENTRATION SPAN DELTA MEASUREMENTS
0.25
CO2 Temperature Observation [1 to 1080]
-C02Span12.0%C02
• C02 Audit 5.99% C02
-C02ZeroO%C02
FINAL ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE CO2
CONCENTRATION
SwRI Report 03.12024.06
K-16
-------
-7
100 150
Reference Gas Delta Observation
200
ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE NMHC
CONCENTRATION ZERO DELTA MEASUREMENTS
14
12
10 -
- PEMS 2 • PEMS 3 • PEMS 5
E
Q.
J±
&
C
O
13
***»
2 -
o
O
o
-2 -
-4 -
-6
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1-
10 20 30 40 50 60 70
Reference Gas Delta Observation
ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE NMHC
CONCENTRATION AUDIT DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-17
-------
70
60 -
50
- PEMS 2 • PEMS 3 • PEMS 5
I 40
I 30
O
I 20
13
1 1°
U
8 °
o
-20 -
-30 -
-40 H—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—h
0 10 20 30 40 50 60 70
Reference Gas Delta Observation
ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE NMHC
CONCENTRATION SPAN DELTA MEASUREMENTS
NMHC Temperature Observation [1 to 1080]
- MVIHC Span 669ppm MVIHC
MVIHC Audit 159.9ppm MVIHC
-NVIHCZeroOppm MVIHC
FINAL ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE NMHC
CONCENTRATION
SwRI Report 03.12024.06
K-18
-------
0.25
0.20 -
-0.25
50 100 150 200 250 300
Exhaust Flow Meter Zero Observation
350
400
450
ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE EXHAUST
FLOW RATE DELTA MEASUREMENTS
• THC [ppmC] BCO [ppm] NO [ppm] XNO2 [ppm] »NOx [ppm] Pressure [kpa] ACO2 [%]
120
„ 100
0.014
-0.018
Reference Gas Delta Observation
PEMS 2 ENVIRONMENTAL PRESSURE ZERO DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-19
-------
• THC [ppmC] BOO [ppm] NO [ppm] XNO2 [ppm] «NOx [ppm] -Pressure [kpa] ACO2 [%]
- 0.054
-0.030
Reference Gas Delta Observation
PEMS 2 ENVIRONMENTAL PRESSURE AUDIT DELTA MEASUREMENTS
• THC [ppmC] BCO [ppm] NO [ppm] XNO2 [ppm] «NOx [ppm] Pressure [kpa] ACO2 [%]
160
-60
Reference Gas Delta Observation
PEMS 2 ENVIRONMENTAL PRESSURE SPAN DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-20
-------
• THC [ppmC] BOO [ppm] NO [ppm] XNO2 [ppm] «NOx [ppm] -Pressure [kpa] ACO2 [%]
120
„ 100
0.015
-0.015
Reference Gas Delta Observation
PEMS 3 ENVIRONMENTAL PRESSURE ZERO DELTA MEASUREMENTS
• THC [ppmC] BCO [ppm] NO [ppm] XNO2 [ppm] «NOx [ppm] Pressure [mbar] ACO2 [%]
120
0.080
-0.060
Reference Gas Delta Observation
PEMS 3 ENVIRONMENTAL PRESSURE AUDIT DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-21
-------
• THC [ppmC] BCO [ppm] NO [ppm] XNO2 [ppm] «NOx [ppm] Pressure [mbar] ACO2 [%]
Reference Gas Delta Observation
PEMS 3 ENVIRONMENTAL PRESSURE SPAN DELTA MEASUREMENTS
* Flow Meter [scfm] • Pressure [mbar]
1200
1000
800
600
400
200
a.
LU
0.
100
150 200 250
Reference Gas Delta Observation
300
350
5-INCH EFM ENVIRONMENTAL PRESSURE ZERO DELTA
MEASUREMENTS
SwRI Report 03.12024.06
K-22
-------
100 150
Reference Gas Delta Observation
200
ERROR SURFACE (NOT USED IN THE MODEL) FOR ENVIRONMENTAL
PRESSURE NOX CONCENTRATION ZERO DELTA MEASUREMENTS
30
25 -
20 -
15 ------
m
o
c
o
o
10 --
• ---z*~
»•»»-* **
0 -
* -
-5 H—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I
0 10 20 30 40 50 60 70
Reference Gas Delta Observation
ERROR SURFACE (NOT USED IN THE MODEL) FOR ENVIRONMENTAL
PRESSURE NOX CONCENTRATION AUDIT DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-23
-------
30
-PEMS2 »PEMS3
20 -
I 10
D.
re
§ 0
c
o
I -10
o
o
5 -20
z
-30
-40 H—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—h
0 10 20 30 40 50 60 70
Reference Gas Delta Observation
ERROR SURFACE (NOT USED IN THE MODEL) FOR ENVIRONMENTAL
PRESSURE NOX CONCENTRATION SPAN DELTA MEASUREMENTS
0.0010
-PEMS2 »PEMS3
0.0008
0.0006
2 0.0004
re
Q 0.0002
c
o
is o.oooo
1
'_ -0.0002
o
O
O -0.0004
-0.0006
-0.0008
-0.0010
H 1 1 1-
H 1 1 1-
H 1 1 H
-I 1 1 1-
50
100 150
Reference Gas Delta Observation
200
ERROR SURFACE FOR ENVIRONMENTAL PRESSURE CO
CONCENTRATION ZERO DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-24
-------
0.004
0.003
0.002 -
0.001
m
o
c
o
o
O
O
0.000
-0.001
-0.002
•
* «
«r » **
»»
V
* *
**** _* * * *c >*
• ***„
-0.003 H—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I-
0 10 20 30 40 50 60 70
Reference Gas Delta Observation
ERROR SURFACE FOR ENVIRONMENTAL PRESSURE CO
CONCENTRATION AUDIT DELTA MEASUREMENTS
0.014
0.012
0.010
0.008
0.006
0.004
0.002
0.000
-0.002
-0.004
-0.006
-PEMS2 »PEMS3
n»*
»»**%**»
»**
-H 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1-
10 20 30 40 50 60 70
Reference Gas Delta Observation
ERROR SURFACE FOR ENVIRONMENTAL PRESSURE CO
CONCENTRATION SPAN DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-25
-------
0.014
0.012
0.01 -
0.008
s?
3 0.006
&
£ 0.004
w
I 0.002
8
-0.002
-0.004
-0.006
100
200
300
600
8(10
CO Pressure Observation [1 to 720]
- CO SPAN 0.0960% CO
• CO Audit 0.0178% CO
-COZeroO%CO
FINAL ERROR SURFACE FOR ENVIRONMENTAL PRESSURE CO
CONCENTRATION
0.012
-0.008 --
-0.010
100 150
Reference Gas Delta Observation
200
ERROR SURFACE (NOT USED IN THE MODEL) FOR ENVIRONMENTAL
PRESSURE CO2 CONCENTRATION ZERO DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-26
-------
0.09
0.08
0.07
0.06
S 0.05 -f
Q
I 0.04
•s
I 0-03
o
o
CM
O
O
0.02
0.01
0.00
-0.01
-0.02
-PEMS2 »PEMS3
** ••*-
****
»**
10
20 30 40 50
Reference Gas Delta Observation
60
70
ERROR SURFACE (NOT USED IN THE MODEL) FOR ENVIRONMENTAL
PRESSURE CO2 CONCENTRATION AUDIT DELTA MEASUREMENTS
0.15
0.10
„ 0.05
S
| 0.00
c
o
| -0.05
'c
-0.10
-0.15
-0.20
-PEMS2 »PEMS3
**---.*
•I •»*»-
***
»_
*»**.
-0.25 H—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—h
0 10 20 30 40 50 60 70
Reference Gas Delta Observation
ERROR SURFACE (NOT USED IN THE MODEL) FOR ENVIRONMENTAL
PRESSURE CO2 CONCENTRATION SPAN DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-27
-------
1 -
0
? -1 --
-2 -
-3
1 - 1 - 1 - 1 - 1 - 1 - 1
1
1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1
50
100 150
Reference Gas Delta Observation
200
ERROR SURFACE FOR ENVIRONMENTAL PRESSURE NMHC
CONCENTRATION ZERO DELTA MEASUREMENTS
2
Q.
s a
g -6
c
o
13 -8
o -10
O
o
I -12 f
-14
-PEMS2 «PEMS3
***********
*******
******
*********
-16 H—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I
0 10 20 30 40 50 60 70
Reference Gas Delta Observation
ERROR SURFACE FOR ENVIRONMENTAL PRESSURE NMHC
CONCENTRATION AUDIT DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-28
-------
10
5 -
a o ---
-10 -
-PEMS2 »PEMS3
***
__~ * * * ~-_
*****;* *
--***
***
-15 H—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I-
0 10 20 30 40 50 60 70
Reference Gas Delta Observation
ERROR SURFACE FOR ENVIRONMENTAL PRESSURE NMHC
CONCENTRATION SPAN DELTA MEASUREMENTS
800
NMHC Pressure Observation [1 to 720]
- NMHC Span 663ppm NMHC » NMHC Audit 159.9ppm NMHC - NMHC Zero Oppm NMHC
FINAL ERROR SURFACE FOR ENVIRONMENTAL PRESSURE NMHC
CONCENTRATION
SwRI Report 03.12024.06
K-29
-------
0.5
0.5
0.4
f 0.4
iE
I 0.3
f °'3
§ °'2
i °-2
(3
0.1
0.1
0.0
-0.1
LU
m \
* *A
•
-%--
t
50
100 150 200 250
Reference Gas Delta Observation
300
350
ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE EXHAUST
FLOW RATE DELTA MEASUREMENTS
• THC [ppmC] BCO [ppm] «NO [ppm] XNO2 [ppm] «NOx [ppm] - Baseline Zero ACO2 [%]
Reference Gas Delta Observation
PEMS 7 ENVIRONMENTAL RADIATION BCI ZERO DELTA
MEASUREMENTS
SwRI Report 03.12024.06
K-30
-------
• THC [ppmC] • CO [ppm]
- Baseline A Zero
NO [ppm] X NO2 [ppm]
-Restart PEMS A CO2 [%]
NOx [ppm]
0.30
0.00
Reference Gas Delta Observation
PEMS 7 ENVIRONMENTAL RADIATION BCI AUDIT DELTA
MEASUREMENTS
0.375
NO [ppm] X NO2 [ppm]
Restart PEMS A CO2 [%]
-0.075
Reference Gas Delta Observation
PEMS 7 ENVIRONMENTAL RADIATION BCI SPAN DELTA
MEASUREMENTS
SwRI Report 03.12024.06
K-31
-------
• Flow Meter [scfm] • Baseline
Reference Gas Delta Observation
5-INCH EFM ENVIRONMENTAL RADIATION BCI ZERO DELTA
MEASUREMENTS
THC [ppmC] • CO [ppm] • NO [ppm] X NO2 [ppm]
Baseline A Zero Restart Test A CO2 [%]
NOx [ppm]
Q
c
o
13
Reference Gas Delta Observations
PEMS 7 ENVIRONMENTAL RADIATION RADIATED IMMUNITY ZERO
DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-32
-------
THC [ppmC] • CO [ppm] • NO [ppm] X NO2 [ppm]
Baseline A Zero Restart Test A CO2 [%]
NOx [ppm]
E
Q.
£
tfl
§
0>
O
c
o
13
300
250
200
150
100
50
0
-50
-100
;;.>.;;,!... i:;::::;;;::;!!;;!;!'!!!!!!;
__ A « A_ $A — __ _____
0.350
0.315
0.280
0.245
0.210
0.175 S
"c
0.140 I
o
o
0.105 g
o
0.070
0.035
0.000
Reference Gas Delta Observations
PEMS 7 ENVIRONMENTAL RADIATION RADIATED IMMUNITY AUDIT
DELTA MEASUREMENTS
?
£
U)
J±
0)
Q
O
«
ncen
8
50
Q
I
IS
li
A
A'
• THC [ppmC] • CO [ppm] • NO [ppm] X NO2 [ppm] • NOx [ppm]
A
"
"
.... 1
•ii'ii'ft 15* 20 25 A 30 35 A
• • ^MMM . A A A ^' — T ~
A A A
04
k
0 450
- 0 150
0 100
- -n nsn
s?
s
X
c
0
In
^
(U
&
O
o
o
Reference Gas Delta Observations
PEMS 7 ENVIRONMENTAL RADIATION RADIATED IMMUNITY SPAN
DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-33
-------
• Flow Meter [scfm] • Baseline
Exhaust Flow Meter Zero Observations
5-INCH EFM ENVIRONMENTAL RADIATION RADIATED IMMUNITY ZERO
DELTA MEASUREMENTS
»THC [ppmC] BCO [ppm] «NO [ppm] XNO2 [ppm] «NOx [ppm] -Baseline Zero ACO2 [%]
0.050
0.045
3
0>
Q
c
o
13
Reference Gas Delta Observations
PEMS 7 ENVIRONMENTAL RADIATION ELECTROSTATIC DISCHARGE
ZERO DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-34
-------
Concentration Delta [ppm]
hj .&. O CO C
D 0 0 0 0 C
-20
-4D
»THC [ppmC] BCD [ppm] «NO [ppm] XNO2 [ppm] »NOx [ppm] -Baseline Zero ACO2 [%]
'
A. • • A * A A A '
" • MJL • A A A A
A A A J ^A " "t" .
•
A
A
A
5 8 * !Bx2£^5!lp*lXS*™»*25 30
~ • £ • •
^A «
A
3 0 0 0 0 C
J Ul ^J O NJ C
n o m o m c
32 Concentration Delta [%]
- -n n?R
Reference Gas Delta Observations
PEMS 7 ENVIRONMENTAL RADIATION ELECTROSTATIC DISCHARGE
AUDIT DELTA MEASUREMENTS
• THC [ppmC] BCO [ppm] •NO [ppm] XNO2 [ppm] «NOx [ppm] -Baseline Zero -Zero ACO2 [%]
40
"E"
&
— 20
(U
o
l '
I -20
u
8
-«n
• A A
A A A A A A A
AA"B •" •••• A
A A
AAA" A AA "••
»« k»»»»4»i»***»»»»"''*n>
* * •
***** AAAA
• " A ) • •
* 5i .•••*
A •
•
• •
- 0.040 —
3
- 0.010 g
0
0 020 ^
(U
u
- -0.050 o
O
CN|
O
_n ian
Reference Gas Delta Observations
PEMS 7 ENVIRONMENTAL RADIATION ELECTROSTATIC DISCHARGE
SPAN DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-35
-------
• Flow Meter [scfm] • Baseline
3.5
3.0
2.5
2.0
1.5
1.0
0.5
0.0
-0.5
-1.0
-1.5
--*-
20
60
80
100
120
140
Exhaust Flow Meter Zero Observations
5-INCH EFM ENVIRONMENTAL RADIATION ELECTROSTATIC
DISCHARGE ZERO DELTA MEASUREMENTS
THC [ppmC] • CO [ppm] • NO [ppm] X NO2 [ppm]
Baseline A Zero Day Break A CO2 [%]
NOx [ppm]
0.028
0.024
0.020 —
0.016
-30
0.012 5
O
0.008 g
O
0.004
0.000
Reference Gas Delta Observations
PEMS 7 ENVIRONMENTAL RADIATION CONDUCTED TRANSIENT ZERO
DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-36
-------
THC [ppmC] • CO [ppm]
Baseline A Zero
NO [ppm] X NO2 [ppm]
- Day Break A CO2 [%]
NOx [ppm]
60
50
E
Q.
£
tfl
§
0>
O
= 0
-10
8 -20
-30
-40
-50
A r«A
__A_. _•_!
A A
«X*»fj|g*j|ti(
A A
40
0.160
0.145
0.131
0.116
0.102 75
0.087
il
&
c
o
13
0.073 •£
o>
0.058 §
0.044
0.029
0.015
0.000
Reference Gas Delta Observations
PEMS 7 ENVIRONMENTAL RADIATION CONDUCTED TRANSIENT AUDIT
DELTA MEASUREMENTS
60
Q.
V)
2
&
~- n -
entratior
j
3 C
8
40
-fin
M
•i
• THC [ppmC] • CO [ppm] • NO [ppm] X NO2 [ppm]
• • A A
"I1-*4* ,A ".,.
A •
• i * *
*** • •^•^••« ••^* •
5 AA3»0jfi»»W20 25 ^ 30
•.TjiS.?*.!^!1.!.1.!*!^!!.?.1! iiii
A • X •** x i
1 ••* x '
A
" •
• NOx [ppm]
1 •-• -• •-•
•
I * • • *
L • "A •
» • • •
"
t
X A
•
0 08
g
V)
c
0
0 02 13
> 1
u
O
O
- -nn4
Reference Gas Delta Observations
PEMS 7 ENVIRONMENTAL RADIATION CONDUCTED TRANSIENT SPAN
DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-37
-------
2.0
1.5 -
1.0 -
0.5 -
0.0
-0.5
-1.5
• Flow Meter [scfm] • Baseline
*
**
* *
20
40
60
80
100
120
140
160
180
Exhaust Flow Meter Zero Observations
5-INCH EFM ENVIRONMENTAL RADIATION CONDUCTED TRANSIENT
ZERO DELTA MEASUREMENTS
• THC [ppmC] BCO [ppm] «NO [ppm] XNO2 [ppm] «NOx [ppm] Zero -Baseline ACO2 [%]
0.020
-0.015
Reference Gas Delta Observations
PEMS 3 ENVIRONMENTAL VIBRATION ZERO DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-38
-------
• THC [ppmC] BCO [ppm] • NO [ppm] XNO2 [ppm] »NOx [ppm] -Baseline Zero Span ACO2 [%]
0.12
-----*?
-0.02
Reference Gas Delta Observations
PEMS 3 ENVIRONMENTAL VIBRATION AUDIT DELTA MEASUREMENTS
• THC [ppmC] BCO [ppm] • NO [ppm] XNO2 [ppm] «NOx [ppm] Zero -Baseline Span ACO2 [%]
0.16
30 • • - -• 0.12
0.04
- -0.08
-0.12
Reference Gas Delta Observations
PEMS 3 ENVIRONMENTAL VIBRATION SPAN DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-39
-------
APPENDIX L
MEASUREMENT ALLOWANCE TEST PLAN
FINAL VERSION - OCTOBER, 2005
SwRI Report 03.12024.06
-------
Test Plan to Determine PEMS Measurement
Allowances for the Gaseous Emissions
Regulated under the Manufacturer-Run
Heavy-Duty Diesel Engine In-Use Testing
Program
Developed by:
United States Environmental Protection Agency,
California Air Resources Board, and
Engine Manufacturers Association
October 25, 2005
-------
Executive Summary
This test plan sets forth the agreed upon processes and methodologies to be utilized to
develop additive, brake-specific, data-driven measurement allowances for gaseous
emissions measured by PEMS (NOx, NMHC and CO) as required under the HDIUT
regulatory program. A separate test plan will be developed and agreed upon for the
determination of the PM measurement allowance.
As detailed in this test plan, there is a clear consensus on what components of
measurement error are intended to be covered by the measurement allowances. Namely,
the allowances are to be calculated in a manner that subtracts lab error from PEMS error.
Specifically, utilizing Part 1065 compliant emissions measurement systems and
procedures for both the lab and PEMS, the lab error associated with measuring heavy-
duty engine emissions at stabilized steady-state test points within the NTE zone, sampled
over 30-second durations will be subtracted from the PEMS error associated with
measuring heavy-duty engine emissions utilizing PEMS over 30-second transient NTE
sampling events under a broad range of environmental conditions. This subtraction will
yield "PEMS minus laboratory" measurement allowances. The error model will not
subtract any laboratory accuracy or precision that is determined from laboratory
measurements of transient ~30-second NTE events. The experimental methods and
procedures specified in this test plan for determining, modeling, and comparing each of
the various components of measurement error are designed to generate statistically robust
data-driven measurement allowances for each of the gaseous emissions, namely NOx,
NMHC, and CO.
Successful completion of this test plan is part of the resolution of a 2001 suit filed against
EPA by EMA and a number of individual engine manufacturers. The suit challenged,
among other things, certain supplemental emission requirements referred to as "not-to-
exceed" (NTE) standards. On June 3, 2003, the parties finalized a settlement of their
disputes pertaining to the NTE standards. The parties agreed upon a detailed outline for a
future regulation that would require a manufacturer-run heavy-duty in-use NTE testing
("HDIUT") program for diesel-fueled engines and vehicles. One section of the outline
stated:
"The NTE Threshold will be the NTE standard, including the margins built into the existing regulations,
plus additional margin to account for in-use measurement accuracy. This additional margin shall be
determined by the measurement processes and methodologies to be developed and approved by
EPA/CARB/EMA. This margin will be structured to encourage instrument manufacturers to develop more
and more accurate instruments in the future."
Given the foregoing, the work to be completed under this test plan is a vital component to
the fulfillment of the settlement agreement, and it is vital to the successful
implementation of a fully-enforceable HDIUT program. Because of this significance, it
is critically important that the work detailed in this test plan be carried out in as thorough,
careful and timely a manner as possible.
-------
Table of Contents
1 Introduction 6
2 Monte Carlo Error Model and Measurement Allowance 8
2.1 Objective 8
2.2 Background 8
2.3 Methods and Materials 13
2.4 Data Analysis 13
3 Engine Dynamometer Laboratory Tests 19
3.1 Preliminary Audits 19
3.1.1 Objective 19
3.1.2 Background 19
3.1.3 Methods and Materials 19
3.1.4 Data Analysis 20
3.2 Bias and Precision Errors under steady state engine operation 20
3.2.1 Objective 20
3.2.2 Background 20
3.2.3 Methods and Materials 20
3.2.4 Data Analysis 24
3.3 Precision Errors under transient engine operation (dynamic response) 26
3.3.1 Objective 26
3.3.2 Background 26
3.3.3 Methods and Materials 26
3.3.4 Data Analysis 29
3.4 Exhaust Flow Meter Installation 32
3.4.1 Objective 32
3.4.2 Background 32
3.4.3 Methods and Materials 32
3.4.4 Data Analysis 34
3.5 ECM Torque and BSFC 37
3.5.1 Objective 37
3.5.2 Background 37
3.5.3 Methods and Materials 39
3.5.4 Data Analysis 42
4 Environmental Chamber Tests 45
4.1 Data Analysis for all Environmental Tests 46
4.2 Baseline 47
4.2.1 Objective 47
4.2.2 Background 47
4.2.3 Methods and Materials 47
4.2.4 Data Analysis 48
4.3 Electromagnetic Radiation 48
4.3.1 Objective 48
Ba 49
4.3.2 ckground 49
4.3.3 Methods and Materials 49
-------
4.3.4 Data Analysis 50
4.4 Atmospheric Pressure 52
4.4.1 Objective 52
4.4.2 Background 52
4.4.3 Methods and Materials 53
4.4.4 Data Analysis 54
4.5 Ambient Temperature 55
4.5.1 Objective 55
4.5.2 Background 55
4.5.3 Methods and Materials 56
4.5.4 Data Analysis 57
4.6 Orientation, Shock, and Vibration 58
4.6.1 Objective 58
4.6.2 Background 58
4.6.3 Methods and Materials 59
4.6.4 Data Analysis 59
4.7 Ambient Hydrocarbons 60
4.7.1 Objective 60
4.7.2 Background 60
4.7.3 Methods and Materials 61
4.7.4 Data Analysis 63
5 Model Validation and Measurement Allowance Determination 64
5.1 Model validation 64
5.1.1 Objective 64
5.1.2 Background 64
5.1.3 Methods and Materials 65
5.1.4 Data Analysis 66
5.2 Measurement Allowance Determination 70
5.2.1 Objective 70
5.2.2 Background 70
5.2.3 Methods and Materials 70
5.2.4 Data Analysis 70
6 Time and Cost 71
7 Appendices 73
7.1 Description of Spreadsheet provided by Matt Spears 73
7.2 Abbreviations used in Brake Specific Equations (Table 3.3.4-a) 75
-------
Table of Figures
Fig 9
2.4-a. PDF 9
Table 2.2-a: Example of Calculation/Selection of Measurement Allowance 12
Figures 2.4 a, b, andc: Construction of an Error Surface 15
Table 3.2.3-a: Steady State Operating Conditions for 40 point matrix 23
Table 3.2.4-a: Steady State Error Surfaces (Refer to section 2.4) 25
Table 3.3.3-a: Dynamic Response NTE Events 27
Table 3.3.3-b: Dynamic Response Inter-NTE Events 28
Figure 3.3.3-a: Example of Transient Test Cycle 28
Table 3.3.4-a: Methods for Calculation of Brake Specific Emissions....Error! Bookmark
not defined.
Table 3.3.4-b: Dynamic response (Transient) Error Surfaces 31
Table 3.4.4.1-a: Exhaust Flow Configuration Error Surfaces (Refer to section 2.4) 35
Table 3.5.3.1-a: DOE Engine Operating Conditions (%speed, and %torque respectively)
40
Table 3.5.3.1-b: DOE Parameter Set Points 40
Table 3.5.3.3-a: Sensitivity Engine Operating Conditions (%speed, and %torque
respectively) 41
Table 3.5.3.3-b: Sensitivity Parameter Set Points 41
Tables 3.5.4.1 through 3.5.4.6: Torque Error Surfaces 43
Table 4-a: Gas Cylinder Contents for 5 of 6 Environmental Tests 45
Table 4-b: Gas Cylinder Contents for Ambient Hydrocarbons 46
Table 4.3.4-a: EMI / RFI Pooled Error Surface 51
Atmospheric Pressure Test Sequence 53
Time Series Chart of Atmospheric Pressure Test 53
Ambient Temperature Test Sequence 56
Time Series Chart of Ambient Temperature Test 56
Table 4.6.4-a: Shock and Vibration Pooled Error Surface 60
Table 4.7.3-a: Ambient Hydrocarbon Error Test Sequence 62
Table 5.1.3-a: CE-CERT Model Validation Test Sequence 66
Chart of Model Validation NTE Events 67
Chart of PEMS vs Tow-along lab Results 68
Budget 71
Timeline 72
-------
1 Introduction
This test plan will establish PEMS measurement allowances for the gaseous emissions
regulated by the manufacturer-run on-highway heavy-duty diesel engine in-use test
program. The measurement allowances will be established using various laboratory
facilities and PEMS. The measurement allowances will be established in units of brake-
specific emissions (g/hp-hr), and they will be added to the final NTE standard for a given
emission, after all the other additive and multiplicative allowances have been applied.
This test plan will establish three measurement allowances; one for NOx, one for NMHC,
and one for CO.
The PEMS used in this test plan must be standard in-production makes and models that
are for sale as commercially available PEMS. In addition, PEMS and any support
equipment must pass a "red-face" test with respect to being consistent with acceptable
practices for in-use testing. For example, use of large gas bottles that can not be utilized
by the EPA/ARB/EMA HDIU enforceable program is unacceptable. Furthermore, the
equipment must meet all safety and transportation regulations for use on-board heavy-
duty vehicles.
Even though the PEMS can not be "prototypes" nor their software "beta" versions, the
steering committee has already agreed that after delivery of PEMS to the contractor, there
may be a few circumstances in which PEMS modifications might be allowed, but these
modifications must meet certain deadlines, plus they are subject to approval by the
steering committee. Also, any implementation of such approved modifications will not
be allowed to delay the test plan, unless the steering committee specifically approves
such a delay. Table 1 summarizes these allowable modifications and their respective
deadlines:
Table 1: Allowed Modifications
Steering committee approved hardware and software modifications
that affect emissions results; including but not limited to fittings,
components, calibrations, compensation algorithms, sampling rates,
recording rates, etc.
Steering committee approved hardware modifications for DOT
approval or any other safety requirement approval
Delivery of any environmental / weather enclosure to contractor
Post-processing software to determine NTE results
DOT approval and documentation
Steering committee approved hardware or software that improves
the contractor's efficiency to conduct testing and data reduction
Before start of. . .
Section 3.2
Section 4.3
Section 4.3
Section 5.1
Section 5.1
Always Allowed
This test plan describes a computer model, a series of experiments that are used to
calibrate the model, and another series of experiments that are used to validate the
calibrated model.
-------
The test plan first describes the computer model. The computer model statistically
combines many sources of PEMS and lab error, which are nearly impossible to capture
simultaneously in a single test. The model will use statistics to apply the errors in a way
that simulates actual running of a PEMS in-use. The model will also consider only the
portion of error that is attributable to PEMS, and it will subtract the error that is already
tolerated in an emissions lab today. The model will also calculate and validate results
according to 40 CFR Part 1065.
The test plan then describes the series of experiments. These tests will characterize the
many sources of PEMS and lab error so that the specific nature of the errors can be
programmed into the computer model. The nature of the error has to do with the way
PEMS and the lab react to certain conditions. For example, under varying environmental
conditions such as temperature or vibration, a PEMS might exhibit signal drift, or it may
record noise that is not a part of the true emissions.
Next the experimental results will be entered into the computer model, and the
measurement allowances are calculated by the model. The model uses a "reference"
PEMS data set, which will have many "reference NTE events." The model statistically
applies all the errors to the reference data set, calculates results, and saves the results.
Then the model will be run with all errors set to zero to calculate the ideal results of the
reference data set. Each difference between a reference NTE event's result with errors
and its respective ideal result will be a brake-specific difference that is recorded for later
use. Then the process repeats using the same reference data set, to which new,
statistically selected errors are applied, and thus another unique set of differences is
calculated. As the model continues to iterate and generate more and more results,
patterns are expected to appear in the output data. These patterns should be the
distributions of differences, based upon the error that was statistically and repeatedly
applied to the reference data set. Many difference distributions will be determined: for
each reference NTE event, for each three regulated gaseous emissions, and for each of
three brake-specific calculation methods. It has been agreed that the 95th percentile
values of these distributions will be taken as reasonable "worst case" results for each
reference NTE event. Details on how all these distributions will be reduced to determine
the three gaseous measurement allowances are given in the "Error Model" section of this
test plan.
Finally, the test plan describes how the computer model will be validated against real-
world over-the-road in-use PEMS operation as well as additional lab testing. For the
over-the-road testing, PEMS emissions measurements will be conducted, while at the
same time a reference laboratory will be towed along to measure the same emissions. For
the lab testing, an attempt will be made to simulate real-world engine operation to
"replay" an over-the-road test in the lab. Data from these final experiments will be used
to validate the model, which must be done in order to gain sufficient confidence that the
model did not establish unreasonable measurement allowances.
The following sections of this test plan are written as instructions to the contractor or
contractors who will complete the test plan.
-------
2 Monte Carlo Error Model and Measurement Allowance
2.1 Objective
Use Monte Carlo (e.g. random sampling) techniques in an error model to simulate the
combined effects of all the agreed-upon sources of PEMS error incremental to lab error.
Create error "surfaces" for the Monte Carlo simulation to sample, based upon results
from the experiments described in Sections 3 and 4. Exercise the model over a wide
range of NTE events, based on a single, reference data set of at least 100 unique NTE
events. Determine the pollutant-specific brake-specific additive measurement allowances
for NOx, NMHC, and CO.
2.2 Background
The error model uses Monte Carlo techniques to sample error values from "error
surfaces" that are generated from the results of each of the experiments described in
Section 3: lab tests, and Section 4: environmental tests. The lab test error surfaces cover
the domain of error versus the magnitude of the signal to which the error is to be applied
(i.e. 5th to 95th percentile error vs. concentration, flow, torque, etc.). This is illustrated
later in this section. The environmental test error surfaces for shock & vibration and
electromagnetic & radio frequency interference (EMI/RFI) cover the same domain as the
lab tests, but only for concentration. The environmental test for ambient hydrocarbons is
similar but the error surface does not change as a function of concentration. The
environmental test error surfaces for pressure and temperature are characteristically
different because the cover the domain of environmental test cycle time versus the
magnitude of the signal to which the error is to be applied (i.e. error at a selected time vs.
concentration). Details on how each surface is generated are given in each of the
respective sections. These surfaces will already be adjusted to represent PEMS error
incremental to lab error; therefore, these surfaces are sampled directly by the model.
The error model will use two different probability density functions (PDFs) to sample the
error surfaces, depending upon which experiment the surface represents. To sample error
surfaces that are generated from all the laboratory test results (Section 3), and the
environmental test results for shock & vibration; EMI/RFI; and ambient hydrocarbons,
the model will use a truncated normal PDF because these tests are designed to evenly
cover the full, but finite, range of engine operation and ambient conditions.
-------
,
Probability Density Functions for Sampling Error Surfaces Once Per NTE Event
Lab Tests, Normal, SD=0.60795, truncate @ -1 & 1
^—Environmental Tests, Random
Note: A non-trur
distribution with
values of 0.05 ai
ic=+1, respective
icated normal
SD=0.60795 has
id 0.95 at ic=-1 ar
;ly.
^
c
^
^^
p
d ^^^
^*^
^^
^^
^
*^^
^^^
^^^
> 4 3 2
Relative Probability
(
1.00
0.75
0.50
0.25
0.00 .-"
-0.25
-0.50
-0.75
-1.00
Fig
2.4-a. PDF.
To sample error surfaces that are generated from the pressure and temperature
environmental test results (Section 4), the model will use a random PDF because these
tests are already designed to cover the typical range and frequency of the respective
conditions. Note that the lab's normal PDF samples ic, from -1 to 1, including -1 and 1,
and the pressure and temperature environmental tests use a random PDF to sample test
time, from which the nearest (in time) calculated errors are used. The errors from the
other tests will be aligned with the normal PDF such that each of the 50th percentile
values at each of the tested signal magnitudes is centered at the median of the PDF and
the 5th and 95th percentile error values at each of the tested signal magnitudes will be
aligned with the extreme negative (ic = -1) and positive (ic = +1) edges of the PDF,
respectively.
Each error surface will be sampled along its ic axis (y-axis) once per reference NTE event,
and it will be sampled along its parameter value axis (x-axis, e.g., concentration, flow,
torque, etc...) once per second, within a given reference NTE event. An error will be
determined for a given second and parameter along the error axis (z-axis) at the
intersection of an ic value and a parameter value.
To ensure that the magnitudes of the error surfaces are appropriate, each data point used
to generate the surfaces will be a mean or a weighted mean of 30 seconds of sampling.
Interpolation will be performed by first linearly interpolating error values at each tested
magnitude along the selected line perpendicular to the ic axis. Then from that line of
errors, individual error values will be linearly interpolated at each second-by-second
signal magnitude of the given NTE event in the reference data set. If good engineering
-------
judgment dictates that unevenly spaced data, sparse data, or other data irregularities
warrant more sophisticated interpolation techniques, the steering committee will consider
alternatives and provide guidance to the contractor.
The reference data set to which all errors will be applied will be a large data set of engine
operation over a wide range of NTE events. This reference data set will be initially
generated from collections of real-world PEMS data sets. The reference data set should
contain at least 100 unique NTE events. Parameters in the reference data set may be
scaled in order to exercise the model through a more appropriate range of parameters (i.e.
concentrations, flows, ambient conditions, etc.). If the parameters are scaled, care should
be taken to maintain the dynamic characteristics of the reference data set.
After the errors are applied, NTE brake-specific emissions results are calculated for NOx,
CO and NMHC, using each of the three agreed-upon NTE calculation methods. Next,
the NTE events are calculated by each of the three calculation methods, but with no error
sampled or applied to the reference data set. These results are considered the "ideal"
results of the reference NTE events. These ideal results are subtracted from each
respective NTE event, and the difference is recorded. Then a new set of errors are
sampled and applied to the reference data set, and the NTE results are calculated again.
The ideal results are again subtracted, and the difference is recorded. This is repeated
thousands or possibly even millions of times so that the model converges upon
distributions of brake-specific differences for each of the original NTE events in the
reference data set. Then the 95th percentile difference value is determined for each NTE
event's distributions of brake-specific differences for each emission (NOx, NMHC, and
CO) for each calculation method.
The three different brake-specific emission calculation methods referred to in this test
plan are i) Torque-Speed Method, ii) BSFC method, and iii) Fuel Specific method, and
these are illustrated in the same order in Figure 2.4-b, below. The symbol notation for
these equations is described in 40 CFR Part 1065 Subpa |