Determination of PEMS Measurement
  Allowances for Gaseous Emissions
  Regulated Under the Heavy-Duty Diesel
  Engine In-Use Testing Program


  Revised Final Report
United States
Environmental Protection
Agency

-------
                Determination of PEMS Measurement
                  Allowances for Gaseous Emissions
               Regulated Under the Heavy-Duty Diesel
                    Engine In-Use  Testing Program
                              Revised Final Report
v>EPA
                               Assessment and Standards Division
                              Office of Transportation and Air Quality
                              U.S. Environmental Protection Agency
                                    Prepared for EPA by
                                 Southwest Research Institute
                                 EPA Contract No. EP-C-05-018
                                Work Assignment No. 0-6,1 -6,2-6
    NOTICE

    This technical report does not necessarily represent final EPA decisions or
    positions. It is intended to present technical analysis of issues using data
    generated in the associated test program. The purpose in the release of such
    reports is to facilitate the exchange of technical information and to inform the
    public of technical developments which may form the basis for a final EPA
    decision, position, or regulatory action.


United States                                       EPA420-R-08-005
Environmental Protection                                 r ,    „„„
Agency                                          Feburary2008

-------
                               EXECUTIVE SUMMARY

       This report documents a program conducted by Southwest Research Institute® (SwRI®),
on behalf of the U.S. Environmental Protection Agency (EPA), the objective of which was to
determine a set of brake-specific measurement allowances for the gaseous pollutants regulated
under the Heavy-Duty In-Use Testing (HDIUT) program.  Those  pollutants are non-methane
hydrocarbons  (NMHC),  carbon  monoxide (CO),  and  oxides or nitrogen  (NOX).   Each
measurement allowance represents the incremental  error between  measuring emissions under
controlled conditions in a laboratory with lab-grade equipment, and measuring emissions in the
field using Portable Emissions Measurement Systems (PEMS).

       The  completion of this  program was part of the  resolution of a  2001 legal suit filed
against EPA by the Engine  Manufacturer's Association (EMA) and several individual engine
manufacturers regarding certain portions of the Not-to-Exceed (NTE)  standards. This dispute
was settled on June 3, 2003.  A portion of the settlement documents stated:

"The  NTE Threshold will be the NTE standard, including the margins built into the existing regulations,  plus
additional margin to account for in-use measurement accuracy. This additional margin shall be determined by the
measurement processes and methodologies to be developed and approved by EPA/CARB/EMA. This margin will
be structured to encourage instrument manufacturers to develop more and more accurate instruments in the future."

       The  program detailed  in  this  report  is the  result of the  aforementioned  statement.
Therefore, while this program was contracted through EPA, it represented a joint effort between
EPA, EMA, and the California Air Resources Board (CARB).  The Memorandum of Agreement
(MOA) that was part of the  settlement documents outlined a process during which a Test Plan
would be jointly developed  by  EPA, EMA, and the California Air Resources Board (CARB).
SwRI was chosen  as the contractor to carry out this Test  Plan. All efforts during the program
were conducted under the  direction  of a joint body, the HDIUT Measurement  Allowance
Steering Committee, referred to  in this report simply as the Steering Committee.  This group was
composed of representatives of EPA, CARB, EMA,  and various individual EMA member
companies.  The Steering Committee reviewed all decisions and results during this program,  and
any changes made to the Test  Plan were subject  to Steering Committee  review and approval
before being executed.

       The measurement allowances determined in this program were meant to be assessed in
comparison to certain NTE compliance threshold values. For this program, a single  set of NTE
threshold values was determined by the Steering Committee.  These values  served as the basis
for calculating  the final measurement allowances, as well as for the scaling of various other
parameters during  this program. The NTE threshold values used for the program are given in
Table 1.

       This revised version  of  the final report contains a number  of changes made following
EPA's peer review of the original final report.  None of the results or conclusions of the original
report were  affected as part of the revision.  The changes made to the report primarily involved
additional clarifying language in areas were the peer review  process indicated that the original
report was unclear or vague.
SwRI Report 03.12024.06

-------
 TABLE 1.  NTE THRESHOLD VALUES USED FOR MEASUREMENT ALLOWANCE
                                     PROGRAM
Pollutant
NMHC
NOX
CO
NTE Threshold,
g/hp-hr
0.21
2.0
19.4
       The final Measurement Allowance values determined at the conclusion of this program
are summarized in Table 2.   These values were  unanimously approved  by the Steering
Committee, and will be the values published by EPA for use during the HDIUT program.  The
effective date for these values was be March 1, 2007.

                 TABLE 2.  FINAL MEASUREMENT ALLOWANCES
Pollutant
NMHC
NOX
CO
Measurement Allowance,
g/hp-hr
0.02
0.45
0.5
       The remainder of this report details the process used to determine the values reported
above.

Acknowledgements

       This program could not have taken place without significant contributions from a wide
variety  of  participants.   SwRI would  like to acknowledge EPA,  CARB,  and the various
participating  EMA  member  companies for jointly funding  this effort.    Numerous other
contributions  were made as well.  Test engines were supplied by Daimler Chrysler, Caterpillar,
and International.  In addition Caterpillar also supplied the test truck used for validation testing,
and  International  contributed the diesel particulate filters  (DPFs)  used on that truck.   A
considerable quantity of PEMS equipment was provided by EPA in order to facilitate the timely
completion of this program.  Significant support in terms of training, diagnosis, and  technical
support was also provided by Sensors Inc. as the primary PEMS manufacturer participating in
the program.  SwRI would  like to thank all of the companies who made these contributions.
SwRI would  also like to acknowledge the efforts of CE-CERT in completing the on-road
validation tests that were a key portion of this program.

       SwRI  would  also like to extend a special thanks to the various individual representatives
who made  up the Steering Committee.  Participation in the Committee required a significant
amount of time and energy on the part of these individuals, and SwRI would like to acknowledge
the commitment of the Committee members to the successful completion of this program.

       SwRI  would  also like to thank Mr. Ed Oelkers for his efforts in developing and executing
the Crystal Ball Monte Carlo simulation model.
SwRI Report 03.12024.06
11

-------
Program Methodology

       Statistical Simulation Approach (Monte Carlo Model)

       During the Test Plan development, it was understood  that it would not be feasible to
conduct enough representative  experiments to directly quantify the  measurement allowances.
Therefore, the Steering Committee chose a methodology which involved the construction of a
statistical Model of the measurement errors.  The statistical Model incorporated Monte Carlo
(random sampling) methodology to simulate the variation in errors over repeat measurements.
The Model was then run thousands of times to generate a large data set to allow determination of
a robust set of measurement allowances.

       The Model incorporates a variety of error components, each of which represents a
different source of potential error between the laboratory and  the PEMS.  Each of these error
components was associated with a laboratory experiment designed to characterize and quantify
the effect of a potential error source. The result of each experiment was an empirical model,
often visualized as a three-dimensional surface, which related the chosen test conditions to the
error between a laboratory reference  measurement and a PEMS measurement.   These empirical
models are thus referred to as "error surfaces" in this report. The individual errors are generally
referred to as "deltas", and are  typically characterized as the PEMS measurement value minus
the laboratory reference value.  A positive delta indicates a PEMS measurement higher than the
reference, while a negative delta would indicate a PEMS value below that of the laboratory.

       A total  of  37  error surfaces  were incorporated into the Model.  The  individual error
surfaces encompassed a wide variety of error sources.  A number  of additional potential error
sources were investigated during the program beyond those which ultimately resulted in error
surfaces.   However,  in those cases, upon  reviewing the experimental data, the  Steering
Committee deemed that the errors from those sources were not significant; therefore, inclusion in
the final Model was not warranted.  A wide variety  of experiments were conducted to examine
the various error terms, but they can be grouped into several major categories.

       1.     Steady-State  error surfaces.  These error terms characterized precision and bias
             errors  over repeated steady-state measurements.  The  errors were characterized
             via  steady-state  testing  in  an engine  dynamometer test cell.   The Model
             incorporates steady-state  surfaces for each gaseous  pollutant and exhaust flow
             rate.

       2.     Transient error surfaces.  These error terms  characterized precision errors of
             repeated  measurement of 30-second  NTE events.  Note that bias errors were
             specifically not included in the transient error surfaces  due to concerns about the
             ability of the reference laboratory methods to accurately quantify emissions over
             30-second events.  These errors  were characterized by repeat transient testing.
             The transient  cycles were composed of a series of 30-second NTE events whose
             order was randomized for each repeat.   This testing was also run in an engine
             dynamometer test cell. The Model incorporates transient error surfaces for each
SwRI Report 03.12024.06                       iii

-------
             gaseous  pollutant and exhaust flow rate.  Transient error  surfaces also  were
             incorporated to look at dynamic errors in the ECM CAN broadcast signals.

       3.     Torque and BSFC error surfaces. These error terms were included to quantify
             the ability of the engine  ECM to accurately predict and broadcast torque and
             brake-specific fuel consumption (BSFC) in a wide variety of conditions. These
             experiments involved subjecting  the  test engines to changes  in  a variety of
             different conditions (altitude, temperature, fuel, etc.), and comparing laboratory
             reference measurements to values generated using parameters broadcast from the
             engine ECM via  the CAN bus. All of these experiments used steady-state tests
             conducted in an engine dynamometer test cell which was capable of simulating a
             wide variety of ambient conditions.

       4.     Exhaust Flow Measurement error surfaces.  These error terms characterized
             the effect of various installation/measurement conditions (wind, pipe bends, etc.)
             on the  PEMS exhaust flow  meter measurements.   These experiments  were
             conducted in an engine dynamometer  test  cell,  with PEMS measurements
             compared to the laboratory reference flow meters, again using steady-state testing.

       5.     Environmental Testing error surfaces.  These error terms were  designed to
             model the effects of various ambient conditions on the PEMS.   The conditions
             examined included ambient temperature and pressure, vibration, electromagnetic
             interference (EMI), etc.  These experiments were  conducted using a variety of
             environmental test facilities at SwRI, each of which was designed to simulate a
             wide variety of change to a given environmental parameter (such as altitude or
             EMI interference).  In these  cases, PEMS were set up  to measure  standard
             reference gases during testing, while the environmental conditions were varied
             according to the design of each experiment. The deltas generated for these tests
             were between the PEMS measurement and the known reference concentrations.

       6.     Miscellaneous  error  surfaces.    Several  additional  error   surfaces   were
             incorporated in the  Model  to account for diverse error  terms, such as  time
             alignment of data and engine production variability. These error terms involved
             computational  exercises  made using  data from some  of the  aforementioned
             experiments,  and in  some  cases,  data  supplied  by  participating  engine
             manufacturers.

       Test Methods and Equipment

       Engine dynamometer tests were  conducted using three different test engines, one Heavy
Heavy-Duty (HHD) engine,  one Medium Heavy-Duty (MHD) engine, and one Light Heavy-
Duty  (LHD)  engine.   These  engines were  contributed  to the  program  by  the  engine
manufacturers, along with all support  needed to insure successful  engine operation.  These
engines were generally model year 2005 or 2006 engines.  In order to simulate a post-2007 test
environment, SwRI procured several diesel particulate filters (DPFs) which were installed in the
exhaust of the various engines during all testing.  It should be noted that the filters selected were
SwRI Report 03.12024.06                        iv

-------
designed to regenerate primarily via passive regeneration, as active regeneration systems were
not available at the time of this program.

       The original intention of the program was to examine PEMS from more than one
manufacturer.  However, at the time of this program only one equipment manufacturer,  Sensors
Incorporated, was able to supply commercially available PEMS units to the program. Therefore,
the measurement allowance values are based only on measurements made using the Sensors Inc.
SEMTECH-DS instrument. Multiple examples of that instrument were used during the program,
often in parallel with each other, in order to encompass instrument-to-instrument variation errors.
Late in the  program,  several Horiba OBS-2200 units became available.  These units were
incorporated into the program as time and resources  allowed.   However, all Horiba PEMS
measurements  were performed for information purposes  only, and no Horiba PEMS data was
used in the generation of the measurement allowances.

       The primary engine laboratory reference measurements used for this program were made
using a transient  capable engine dynamometer test cell, which incorporated a full-flow CVS
dilution tunnel. The test cell was capable  of simulating a wide variety of ambient conditions in
order to facilitate some of the Torque and BSFC error experiments.   All of  the emission
concentration  deltas that went into the  Model were generated  using the dilute laboratory
measurements  as the reference value. However, raw exhaust laboratory measurements were also
conducted during this program for quality assurance purposes, and  as an  additional check on the
primary reference.  The laboratory reference values are summarized in Table 3. All calculations
were made using methods detailed in 40 CFR Part 1065. Unless otherwise stated, all engine tests
were run using U.S. EPA certification grade ultra-low sulfur 2-D diesel fuel.
                 TABLE 3. LABORATORY REFERENCE METHODS
PEMS Measurement
Gaseous Analyzers - engine
testing
Raw Exhaust Flow
Predicted Torque (from CAN)
Predicted BSFC (from CAN)
Gaseous Analyzers -
environmental chamber testing
Laboratory Reference
Dilute Emission Analyzers1
Measured Intake Air Flow and
Fuel Flow
Measured Torque
Measured Fuel flow and
Power
Standard reference gas
concentrations
Reference Method
Dilute mass calculated using
CVS flow, then raw
concentrations back-calculated
using laboratory raw exhaust
flow
Air Flow measured using
Laminar Flow Element (LFE).
Fuel Flow measured using
coriolis type meter.
Shaft mounted in-line torque
meter
see above notes
Reference values validated on all
bottles at SwRI
Reference NMHC levels were based on laboratory raw measurements due to very low levels.
SwRI Report 03.12024.06

-------
PEMS and Laboratory Audits

       In order to insure that all measurements were conducted at the highest quality level, a full
audit of the reference laboratory was conducted prior to the start of testing. In addition, the audit
was performed to verify that all requirements given in 40 CFR Part  1065 were met, and that any
recommended practices were followed to the extent possible.  Quality assurance procedures were
in place to insure that all test equipment was maintained within  the requirements of Part 1065
throughout the program.

       Audits based on  40 CFR Part 1065 were also conducted on all PEMS equipment used in
the program.  In addition, PEMS equipment was re-audited whenever equipment failures resulted
in major repairs to  one  or more PEMS.  This occurred on numerous occasions throughout the
program.  In general, the PEMS passed the requirements in  40 CFR Part 1065, but there were
exceptions.  In cases where the requirements were not initially met, the PEMS manufacturer was
offered  an opportunity  to correct the problem.  However,  in cases where no  correction was
available, the Steering Committee had the option to approve the deficiency and continue testing.

       Individual audit results for each PEMS are detailed in  Section 2 of this report.  However,
there were several general issues which arose during the audits which are summarized here.

       Gaseous Analyzer Linearity

       Numerous gas analyzers on the various PEMS units  failed to meet the 1065 Subpart D
linearity criteria during the  program. Nearly all of these failures were in the regression line
intercept criteria outlined in 1065.307 Table 1, which specify  a tolerance on the intercept of 0.5%
of the maximum value expected during testing. Because this  value was not known at the time of
the audits, this maximum value was interpreted as the span gas value used for the instrument.  It
should be noted that this interpretation resulted in a relatively loose tolerance for this particular
check, which the PEMS still failed  periodically.  A  number of the gas analyzers in  various
PEMS, particularly  the NDUV analyzers used for NO  and NC>2 measurement tended to fail this
requirement high. Certain units passed all linearity criteria.

       Sensors Inc. initially  re-calibrated one unit as a result of this failure,  but numerous other
units were deemed by Sensors Inc. to be operating correctly, and Sensors Inc. indicated that they
felt there were issues with the linearity procedure as written in 40  CFR Part 1065.  Due to the
difficulty in continually re-calibrating these units and comments from Sensors Inc., the Steering
Committee ultimately elected to allow testing to continue with PEMS units that failed the 1065
linearity verification.  However, this remains an issue to be addressed during in-use  testing,
wherein manufacturers will be legally bound to use equipment that meets all  1065 specifications.
It should be noted that the Horiba PEMS passed all 1065 linearity checks.

       NOi Penetration Checks and NOi Measurement

       Initially all  of the  PEMS failed the NC>2 penetration check in  1065.376 due to issues
within the sample handling systems of the SEMTECH DS units.  This issue turned out to have a
measurable effect on NOX emissions results,  due to the relatively  high tailpipe NC>2 fractions
resulting from the use of the catalyst-based DPFs. The result was a significant low bias in NOX
SwRI Report 03.12024.06                       vi

-------
measurements from the PEMS units as initially configured. As a result, it became necessary for
Sensors Inc. to modify the test equipment during the course of the program.  Although such a
modification  was initially not allowed in the Test Plan, the Steering Committee  approved a
retrofit  in  order to  address  this  significant measurement error.   This modification was
successfully accomplished, and is now commercially available on all new SEMTECH DS units
and as a retrofit for existing units. Following this retrofit, all PEMS passed the NC>2 penetration
check.  All of the data that was used in the Monte  Carlo Model reflects the use of this retrofit.
However, the issue resulted in significant program schedule delays as the retrofit was designed,
tested, and implemented.

       Exhaust Flow Meter Linearity and Calibration

       The exhaust flow meter linearity  checks required considerable effort on the part of both
SwRI and  Sensors Inc.,  and may ultimately have been a  source  of some of the bias  errors
observed during later testing as well. The issue was directly  linked to the size (diameter) of the
Sensors Inc. exhaust flow meter (EFM).  The 5-inch flow meters had little difficulty with the
linearity check at SwRI, with only one failing unit which was re-calibrated by Sensors Inc. and
then passed linearity at SwRI.  However, the initial linearity checks showed slope problems with
all the smaller flow meters, some low and some high. All of the 3-inch and 4-inch meters were
sent to Sensors Inc. for re-calibration. Linearity checks on the newly calibrated meters indicated
low slopes  for the 4-inch flow meters, and even lower on average for the 3-inch flow meters.
Considerable effort was directed into determining the root cause for these discrepancies.

       A possible cause for the linearity failures was a design difference between the flow stands
used by SwRI and Sensors Inc. for calibration and linearity checks.  The arrangement  of the
SwRI flow stand is test flow meter  followed by reference flow meter followed by pump, while
the Sensors Inc. flow stand uses the reverse order.  Thus the Sensors  Inc. calibrations were
performed with the EFM under a slight positive pressure, while  the SwRI linearity checks were
performed with the EFM under a slight negative  pressure.   According to the static pressure
measurement in the PEMS EFMs, the 5-inch meters, which showed minimal error, were under a
vacuum of about 2 kPa,  while the  4-inch and 3-inch meters both experienced slightly higher
vacuum levels of about 2.5 kPa.

       The Steering Committee ultimately decided to authorize recalibration of the Sensors Inc.
flow meters using the SwRI linearity data.  The average slope adjustment was on the order of a 4
percent  positive offset for both the 3-inch and 4-inch flow meters.

Engine Dynamometer Results

       Detailed  information  on  the results of all  of the engine  dynamometer  laboratory
experiments performed to generate individual error surfaces is given in Section 3 of this report.
However, there were  several  overall  trends which  affected  the  results of several  of the
dynamometer experiments, which are discussed below.
SwRI Report 03.12024.06                       vii

-------
       NMHC Measurement and Low NMHC Emissions

       Although the engine experiments were designed to quantify errors in NMHC, CO, and
NOX, the NMHC measurements presented a particular problem  due to the very low levels of
NMHC observed during testing.   This was due to a combination of relatively low engine-out
NMHC levels and the use of catalyst-based DPFs. In this program, the DPFs used were CRTs
procured from Johnson Matthey.  In addition, the DPFs were sized to be larger than normal for
these engines in  order to keep engine backpressure levels low, because none of these pre-2007
test engines were originally designed to operate with DPFs installed.  The resulting DPF-out
NMHC levels were at or near zero  all of the time.

       An attempt was made to address this issue by using a 2007 production-style DPF, which
likely had lower  precious metal loadings, on one of the test engines.  However, NMHC was still
at a level of less than 10 percent of the 2007 standard level of 0.14 g/hp-hr. The levels observed
were often  below the noise limitations of the laboratory reference method for all test engines,
thus complicating the generation of meaningful deltas.  As a result, the data analysis methods
and resulting error surfaces for NMHC were modified considerably from the Test Plan.

       An  additional issue with NMHC derived from the fact that no commercially  viable
method of in-field NMHC measurement existed at the time of the experiments.   All available
PEMS measured only total hydrocarbons (THC). Therefore, the PEMS NMHC measurements
were THC measurements multiplied by a factor of 0.98 as allowed in CFR 40 Part 1065.

       Engine-PEMS Installation Variability

       The Test Plan was designed to capture PEMS variability from unit-to-unit and engine-to-
engine. However, it was anticipated that there  would be a certain amount of uniformity in the
measurement error trends and the response of the PEMS  observed from one engine to the next,
despite the  different installations, and in some cases different measurement equipment (such as
different sized exhaust flow meters). When the results of experiments on all three  engines were
compared, it was apparent that reproducibility from engine installation to  engine installation was
a more important variable than expected.   This resulted in the need to modify some  of the
initially planned data analysis methods to account for the unexpectedly large source of variation.

       CO Measurement

       CO  measurements throughout the program were generally affected by the relatively poor
resolution of the NDIR detector used for CO measurement in the SEMTECH DS.  Tailpipe CO
levels during this program were orders of magnitude below the  CO NTE threshold levels (due to
the catalyst-based DPFs).  The NDIR detector in the SEMTECH DS uses the same percent scale
resolution for CO that is used for  CO2.   As a result, the minimum resolution is 0.001 %, or 10
ppm.  In addition, it was found that simply switching from the calibration gas port to sample line
generally resulted in a reading of roughly 20 to 60 ppm, even when reading zero gas through the
sample line. However, this lack of accuracy at low levels is not likely to be a compliance issue,
as the tailpipe levels observed with the SEMTECH-DS CO even  with resolution and bias issues
were still orders  of magnitude below the NTE threshold.  Therefore, no particular modifications
were made to the Test Plan to account for this issue.
SwRI Report 03.12024.06                       viii

-------
       Individual Error Surfaces

       The  following  is  a brief overview of  the  results  for each of  the  various  engine
dynamometer experiments. This summary gives  a broad overview of the general magnitude of
each error term, as well as any major issue or  findings associated with a given experiment.
Detailed information for each experiment is given later in the report.

       Steady-State Error Surfaces

       Considerable effort was expended in the generation of the steady-state data sets, as the
data from these experiments was also used in the  analysis of data from most of the other engine
experiments.    These  experiments  were run  on  all three engines,  with three PEMS  run
simultaneously  for all  of these experiments.  Due to various equipment failures, the same three
PEMS were not used on all three engines.  The steady-state error surfaces deal with both bias and
precision errors.  Steady-state  deltas were  generated by comparing PEMS measurements to
laboratory measurements  for each  individual  data point,  and  the data were  then  pooled to
generate the error surface.  Because these matched pairs of PEMS-laboratory data were used, the
steady-state error surfaces were not affected by variability of the test article itself.

       The final NMHC error surface incorporated only data from Engine 2, which was the only
engine showing a significant number of non-zero PEMS THC readings.  The magnitudes of the
error deltas for the steady-state error surfaces are summarized in Table 4. For errors which were
not level dependent, the size of the error is  shown as a percentage of the average value at the
appropriate NTE threshold.  It  should be noted that  these average  concentration values at the
thresholds are only estimates which were calculated by examining NTE data supplied by various
engine manufacturers, and that these calculations  assume certain average power levels and  flow
rates.  These values  are used only as a means to portray the magnitude of the steady-state errors.

     TABLE 4. MAGNITUDE OF ERROR TERMS FOR STEADY-STATE ERROR
                                      SURFACES
Percentile
5th
50th
95th
Error Magnitudes
NMHC,
% threshold1
0%
1%
7%
CO,
% threshold1
0.3%
1.1%
2.0%
NOX,
% threshold1
-5%3
0
5%
C02,
% threshold1
0.3%
0.4%
0.8%
EFM,
% max2
-1%
5%
11%
1 %threshold = percent of average concentration at NTE threshold, or for CO2 average value
during "typical" NTE event (NMHC = 60 ppm, CO = 4450 ppm, NOX = 290 ppm, CO2 = 8 %)
2 %max = percent of maximum value, varies by flow meter size
3 Above 400ppm, NOX 5th percentile appeared level dependent at -14% of point
       The NOX  error  surface was complicated by the engine-to-engine  variability issues
described earlier.   Steady-state NOX errors were generally independent of level.  However, at
SwRI Report 03.12024.06
IX

-------
NOX concentrations above 400 ppm, larger negative errors were observed.  These errors showed
a dependency on level; generally at about -14% of point (positive errors remained unchanged).
The NOX errors were  due to  negative biases observed for  some of the PEMS during tests on
Engine 3. The reason for this negative bias is not fully understood, as the PEMS passed all NOX
related Part 1065 QA checks during this time.

       The  steady-state exhaust flow  meter error  surfaces  were also complicated by large
variations in  observed  errors engine and test installation to another. A different size flow meter
was used for each of the three test engines, and each size flow meter appeared to have different
magnitudes of error.  In general, a net positive bias  was  observed with  the  PEMS EFMs as
compared to the laboratory, with larger biases for the smaller diameter EFMs. Some of this error
may have been the result of calibration method  differences  between Sensors Inc. and SwRI, as
discussed earlier, but the calibration differences were not large enough to account for all of the
positive bias  observed.

       Transient Error Surfaces

       The transient error surface experiments were also run on all three test engines.  This data
set was used not only to generate the transient error surfaces, but also to generate other error
surfaces  dealing with dynamic and time alignment errors that are described later.  As with the
steady-state experiments, three PEMS were run in parallel for the transient experiments, although
the same three PEMS were not used for all three engines.

       The transient error surfaces deal  with precision errors that result from transient operation.
Although bias errors could have been quantified, the Test Plan specifically excluded bias error
from the transient error surfaces.  As a result, PEMS variability was characterized with respect to
the median PEMS  value for a  given  NTE event, without  direct reference to the transient
laboratory data.  A secondary task for this experiment was to provide an initial assessment of the
laboratory's ability to repeat such short transient measurements.

       The transient error surface  data analysis was complicated by the  desire to correct for
precision errors already characterized by the steady-state measurements,  so  that the transient
surfaces  would characterize only the incremental error due  to transient operation.  The method
given in the Test Plan called for the variability of the steady-state measurements to be subtracted
from  the variability observed for the transient experiments, on  an  engine-by-engine basis.
However, because both of these variability terms are evaluated across all the repeats for a given
engine as a pooled data set, the analytical  method was particularly vulnerable to issues related to
variability in the test article itself (i.e. variability in the pooled NOX level during steady-state or
transient testing).

       This vulnerability manifested on several occasions throughout the transient error surface
experiments.  In some cases, it was addressed by removing selected outliers where the engine did
not repeat from the  pooled  data sets for both steady-state and transient experiments.  In other
cases, however, this approach was not adequate to address variability problems. For Engine 3,
steady-state variability was intermittently higher than transient variability for many concentration
levels. As a  result, the transient data analysis methodology  was modified considerably from the
one originally designed  in the Test Plan.  Because bias errors were  not included, all transient
SwRI Report 03.12024.06

-------
error surfaces have 50th percentile error values of zero.  A summary of the magnitude of the
transient error surfaces is given in Table 5. The gaseous emissions errors showed a dependency
on level, and are therefore given as percent of point values.  The EFM transient errors were not
as level dependant and are given is a percent of maximum flow.

TABLE 5. MAGNITUDE OF ERROR TERMS FOR TRANSIENT ERROR SURFACES -
                   GASEOUS ANALYZERS AND EXHAUST FLOW
Percentile
5th
50th
95th
Error Magnitudes
NMHC,
% point
-0.03%
0%
3%
CO,
% point
0%
0%
0%
NOX,
% point
-2.5%
0%
2.5%
COi,
% point
1%
0%
1%
EFM,
% max1
-0.7%2
0%
0.6%2
1 %max = percent of maximum flow rate, varies by flow meter size
2 Values represent average across range of range of flow, individual 5th and 95th percentile values
varied
       Another set of transient error surfaces were generated to capture the effects of transient
operations on ECM broadcast signals that are  used to predict torque and BSFC.  These error
surfaces were again designed only to capture precision errors, and therefore the PEMS deltas for
each repeat were generated with respect to the median PEMS value for a given event.  The
magnitude of these  ECM-related transient error surfaces is summarized in Table 6.

  TABLE 6. MAGNITUDE OF TRANSIENT ERROR TERMS FOR ECM VARIABLES
Percentile
rth3
50th3
95th3
Error Magnitudes
CAN- Speed,
% point
-0.2%
0%
0.2%
CAN-Fuel Rate,
% max1
-0.8%
0%
0.6%
Interpolated
Torque,
% max1
-0.9%
0%
0.7%
Interpolated
BSFC,
% average2
-0.2%
0%
0.2%
% max = percent of maximum engine torque or fuel rate
2 % average = percent of average BSFC over a "typical" NTE event = 245 g/kW-hr
3 Values represent average across range of measurement, individual 5th and 95th
percentile values varied and may be as large is 2-3 times averages
       Torque and BSFC Error Surfaces

       There were a number of engine experiments associated with various sources of error in
torque and BSFC estimation. Some of these were run only on selected engines, as noted below
for each error surface.  The PEMS values for these surfaces were not broadcast directly from the
engine ECMs. Rather ECM CAN broadcast speed and CAN broadcast fuel rate were recorded
during these experiments.  For  each test engine,  a  40-point steady-state map was run to
interpolate torque and/or BSFC from CAN-speed and CAN-fuel rate.  The recorded values were
SwRI Report 03.12024.06
XI

-------
post-processed via interpolation to provide the resulting PEMS values for comparison to the
laboratory reference values. A summary of the magnitude of each of the torque and BSFC error
surfaces is given in Table 7. Each of the error surface types is summarized briefly below.

        TABLE 7. MAGNITUDE OF TORQUE AND BSFC ERROR SURFACES
Error Surface
Torque
Interacting Parameters - DOE2
Interacting Parameters - Warm-up
Independent Parameters
Interpolation
BSFC
Interacting Parameters - DOE
Interacting Parameters - Warm-up
Independent Parameters
Interpolation
Error Magnitudes
5th
Percentile
50th 95th
Percentile Percentile
% max1
-0.5%
-5.9%
-1.0%
-0.9%
0.6%
0%
0%
0.06%
2.3%
5.9%
1.8%
1.6%
% average3
-4.2%
-3.6%
-1.8%
1.0%
-1.5%
0%
0.2%
0.3%
0.8%
3.6%
1.2%
3.7%
1 % max = percent of maximum engine torque
2 DOE percentiles are average percentiles for whole load range, values varied somewhat by level
3 % average = percent of average BSFC during "typical" NTE event = 245 g/kW-hr
       Interacting Parameters - Design of Experiment

       The Design  of Experiment (DOE) experiment was designed to characterize errors in
predicted torque and BSFC based on a variety of operating and environmental conditions. The
conditions included barometric pressure,  manifold temperature, exhaust restriction,  and inlet
restriction.  These parameters were all varied according to the DOE test matrix. Using steady-
state testing, this experiment was run on two of the three test  engines.  The data was all pooled
together to form a single error surface.   A "baseline" set of tests were used to remove
interpolation errors from the data set, because those errors are already accounted for elsewhere in
the Model.

       Interacting Parameters - Warm-up Experiment

       The warm-up experiment was designed to capture errors in predicted torque and BSFC
related to variations in engine fluid properties and operating temperatures, including viscosity
effects.  An exhaustive test matrix of these parameters could not readily be conducted; therefore,
these errors were dealt with collectively using a relatively simple cold-start warm-up experiment.
This experiment was run on all three engines.  However, two of the three engines (both of which
were EGR equipped) were started from low room temperature condition (roughly 15°C), while a
third engine (non-EGR equipped) was soaked to a temperature near 0°C prior to engine start.
The error surface values were characterized by finding the maximum error observed during the
experiment after the point in time where all engine temperatures had reached the entry point of
SwRI Report 03.12024.06
xil

-------
the NTE zone, as defined in CFR 40 Part 86.  The data from all tests was pooled together to
generate a final error surface.

       Independent Parameters

       The independent  parameters experiment  characterized errors in predicted  torque and
BSFC caused by changes in fuel or ambient humidity levels. Three different ULSD fuels were
tested which spanned a wide range of properties including aromatic content, density, and cetane
number.  Three humidity levels were also run from near zero humidity to levels near 28 g/kg.  A
full nine point test matrix was run testing all nine combinations of these parameters.  A clear
trend was observed for fuel changes, while humidity changes did not demonstrate an  obvious
trend. All of the data for all test points was collected into a single error surface. This experiment
was run only on the MHD engine.

       Interpolation Torque and BSFC Errors

       During the design of the Test Plan, it was determined that a 40-point  speed-load matrix
would be used to define an interpolation grid for predicted torque and BSFC from CAN-speed
and CAN-fuel rate.  While this matrix served the needs of the program, it was felt that in real-
world testing, the test matrix was too dense, placing an excessive mapping burden on individual
engine manufacturers.  The Steering Committee  determined that a 20-point  speed-load matrix
would be a more acceptable level of effort.  However, the less dense grid would likely lead to
more interpolation errors in use.

       The interpolation error surfaces were  designed to capture the incremental error involved
in dropping from a 40-point matrix to a 20-point matrix.  The generation of this surface was a
computational exercise carried out using the initial 40-point  steady-state map  data generated for
each engine.  The  Steering Committee down-selected  20  points from those 40 to generate the
coarser grid.  A matrix of several thousand CAN-Speed and CAN-Fuel Rate  combinations was
run using both 40-point and 20-point grids, and these  data sets were compared to generate the
final deltas, 20-point values minus the 40-point values.  Percentile values from this data set were
averaged for all three engines to derive the final error surface.

       Exhaust Flow Meter Error Surfaces

       There were three exhaust flow meter installation experiments,  each of which dealt with a
different potential error source.  All of these experiments were conducted only using Engine 1,
which used a 5-inch diameter Sensors Inc. exhaust flow meter.  The first dealt with errors due to
pulsations in the exhaust.  For this experiment, the DPFs were removed from the exhaust and the
flow meter was relocated to a position relatively close to the turbocharger outlet.  The second
experiment dealt with  non-uniform velocity profiles  in  the EFM introduced by  pipe bends
upstream of the flow meter.  This second  experiment was referred to  as  swirl error.  The
magnitude of these error terms is summarized below in Table 8.
SwRI Report 03.12024.06                      xiii

-------
     TABLE 8. MAGNITUDE OF ERROR TERMS FOR EXHAUST FLOW ERROR
                                     SURFACES
Percentile
50th
Total spread 5th to 95th 2
Pulsation
Errors,
%max1
0.5% to 2%
0.2% to 0.8%
"Swirl"
Errors,
%max1
0.1% to 0.9%
0.1% to 0.5%
%max = percent of maximum flow rate, varies by flow
meter size
2 This value is the total width of the error band between the
5th and 95th percentile boundaries.
       A third experiment was conducted to examine the possible effects of air currents up to 60
mph across the outlet of the EFM in various directions. The wind experiments resulted in no
significant errors; therefore this error surface was removed from the Model.

       Miscellaneous Error Surfaces

       There were several  error surfaces which either did not fit under the above categories, or
were based on data taken outside this program.  These error surfaces are described below.

       OEM Torque and BSFC Error Surfaces

       The OEM torque and BSFC error surfaces were generated based on data supplied by the
various engine manufacturers  directly to EPA. The intention for these error surfaces was to
characterize  errors based on a variety of terms  chosen by joint  agreement of the Steering
Committee members.  Some of these error sources include production variability,  the action of
various AECDs, etc. The data was combined and analyzed by EPA.   Discussions were held
between EPA and  individual engine manufacturers,  due to the confidential nature of much of the
information being disclosed. At the end  of this process, EPA submitted a single set of error
surfaces,  which was approved by the Steering Committee for inclusion in the Model.  The
magnitude of these errors is described in Table 9.
                TABLE 9. MAGNITUDE OF OEM ERROR SURFACES
Percentile
5th
50th
95th
Error Magnitudes
Torque,
% point
-5.9%
0%
5.9%
BSFC,
% point
-6.5%
0%
6.5%
SwRI Report 03.12024.06
xiv

-------
       Time Alignment Errors

       The time alignment error surface captured the effect of errors in time alignment of the
various continuous PEMS data sources on the final brake-specific emission results.  This error
source was not originally included in the Test Plan, and no  experiment had been designed to
examine  it.  However, during various Steering Committee discussions over the course of the
program, it was decided that time alignment was a potentially significant source of error, and that
it should be incorporated into the Model. This proved difficult because unlike many of the other
terms which dealt with  a single measurement term,  time alignment is  associated with the
collection of the various  data streams into the final result.  Therefore, a  single additive delta
could not easily be generated.

       Ultimately, the  Steering  Committee settled on a multiplicative adjustment factor which
would be applied after all  other error terms had been added and the final brake-specific result had
been determined.  A separate factor was developed for each pollutant, and  for each of the three
calculation methods allowed in the HDIUT program.  The error values were generated using a
set  of transient data from  each  engine.  Time  alignment  of three data  streams; the gaseous
analyzers, the exhaust flow meter, and the ECM vehicle interface data stream,  were perturbed
relative to one  another  by increments  of  0.5  and  1  second alignment  errors  in  various
combinations forward and backward.  The brake-specific emission levels for all  30 NTE events
in the cycle were calculated for each misaligned data set, and were compared to values calculated
using the nominal time alignment values.  The errors  were pooled across all three  engines to
arrive at  a final  set of error terms. Time alignment values were only generated for NOX and CO,
because NMHC values were too low and  stable to see any  discernible trends in NMHC due to
time alignment. The final time alignment values are given in Table 10 below.

        TABLE 10. MAGNITUDE OF TIME ALIGNMENT ERROR SURFACES
Calculation
Method
1
2
3
Percentile
5th
50th
95th
5th
50th
95th
5th
50th
95th
Error Values, % point (BS emission level)
CO
-7.5%
0.0%
4.6%
-5.4%
0.0%
5.1%
-5.2%
0.0%
12.3%
NOX
-3.2%
-0.1%
1.5%
-1.3%
0.0%
1.5%
-1.4%
0.0%
2.9%
       The use of the time alignment error term was  not universally accepted by all  of the
Steering Committee members, due to concerns over the method by which it was applied, and the
potential magnitude of its effects compared to all other error terms. However, the majority vote
of the Steering Committee was to include this error surface in the final Model.
SwRI Report 03.12024.06
xv

-------
Environmental Chamber Results

       Detailed information on the results of all of the  environmental  chamber experiments
performed to generate individual error surfaces is given in Section 4 of this report.  However,
several general observations can be made regarding the environmental chamber test results.

       These tests were different from the engine dynamometer tests in that they did not involve
the sampling of engine exhaust. Rather PEMS errors during environmental chamber testing were
conducted while sampling reference gases at various concentrations over an automated sequence.
These  gases were sampled continuously while the various  environmental  disturbances were
applied to the PEMS. Environmental error sources that were examined included the effects of
ambient temperature, ambient pressure (i.e., altitude), vibration, and electromagnetic interference
(EMI/RFI).  In addition, because the PEMS HC instrument used ambient air for the FID burner
air supply, the effect of ambient HC variation was examined on the NMHC measurement.

       In most cases,  the  observed effect of  most of the environmental disturbances  was
relatively small, as compared to other error sources examined during this program.  During the
course of environmental testing, it  was often noted that the PEMS exhibited similar variations on
analyzer response whether the environmental disturbances were applied or not. The exception to
this general trend was NMHC, which demonstrated considerable variation as a result of both
temperature and ambient HC variation.   This  is  despite the fact that a fairly broad range
disturbance was applied for each potential error source. It should be noted that in some cases,
particularly for the vibration and EMI/RFI experiments, the range of environmental disturbances
was actually sufficient to cause occasional functional failures of the PEMS.

       The  relative magnitude of the environmental error surfaces is  given in Table 11.  In
general, these error surfaces were  centered around a zero error, with the table value showing a
typical maximum range of the  error surface  values, as a  percentage of average value at the
appropriate NTE threshold.  Generally, this error could be either positive or negative.
   TABLE 11. MAGNITUDE OF ERROR TERMS FOR ENVIRONMENTAL ERROR
                                      SURFACES
Error Surface
Temperature
Pressure
EMI/RFI
Vibration
Ambient HC
Maximum Error Magnitudes
NMHC,
% threshold
7.5 %
2.5 %
2
2
10%
CO,
% threshold
0.1%
0.7 %
2
2
n/a
NOX,
% threshold
2.8 %
2
2
2
n/a
C02,
% threshold
0.5 %
2
2
2
n/a
EFM,
% max1
0.2 %
0.5 %
0.3 %
2
n/a
1 %max = percent of maximum value, varies by flow meter size
2 a dash (-) indicates that error effect was not deemed significant enough to justify inclusion in
the model
SwRI Report 03.12024.06
xvi

-------
       It should be noted that the effect of the environmental surfaces was limited in the overall
model by design. This was done in an attempt to simulate the effect of the drift check criteria in
40 CFR Part 1065.  The effect of this drift check was simulated in the model by comparing the
brake-specific results of each model run, both with and without the environmental errors applied.
If these two results diverged by more than the tolerance allowed in 40 CFR Part 1065, generally
4 percent, the result of that particular model run was discarded for the pollutant in question as
having failed the drift check.   Therefore, the maximum potential effect of the environmental
error surfaces was limited.

       Given the magnitude of the environmental error surfaces for NMHC, it was recognized
that a large number of model runs were likely to fail this drift check at a 4 percent tolerance.
Therefore,  EPA agreed to widen the drift check tolerance for NMHC for in-use testing from 4
percent to 10 percent.  This change will  be applied to 40  CFR Part 1065 Subpart J.

Monte Carlo Model Results and Validation

       The results  of the Monte Carlo simulation run, as well as the details of procedures used to
validate these results are given in Section 5 of this report.  A summary of these is given below,
including a brief description of the selection of the final Measurement Allowances which were
given at the front of the Executive Summary.

       The final model run to generate the measurement allowance values using all of the data
described above was a significant investment of time and resources. A data set of 195 "reference
NTE events" was used to conduct the model run.  Each  event was run through the model at least
10,000 times and in some cases many more times.  During each repeat, all of the error surfaces
described  above were  randomly sampled, and the resulting errors were  applied to appropriate
terms (i.e., concentrations, exhaust flow, etc.).  The model would then calculate several sets of
brake-specific results.  An "ideal"  result would be calculated from  the un-perturbed  reference
data for the event in question.  Then a set  of perturbated results would be calculated from the
data set after all errors had been applied.  Calculation of the perturbated results was done  using
each of the three brake-specific calculations allowed by 40 CFR Part 1065 Subpart J for in-use
testing.  The perturbated results were then compared to  the ideal result to generate a delta.  The
details of each  of  the  three calculation methods are described  in the background information
given in Section 1 of this report.

       For each of the 195 reference events, the resulting deltas for all of the 10,000 or  more
repeats were pooled together, and a 95th percentile delta was determined for each pollutant by
each of the three calculation methods.  These 95th percentile deltas were pooled together for all
195 reference events in order to generate a final potential measurement allowance value.  A set
of 9 candidate  measurement allowance values was determined, three values for each pollutant
(NMHC, CO, NOX), one for each calculation method.  The candidate values determined by the
model run are given in Table 12. The results of model validation are also shown in this table.
SwRI Report 03.12024.06                      xvii

-------
                  TABLE 12. MODEL RESULTS AND VALIDATION
Measurement Errors (%) at Respective NTE Threshold
Emission
BSNOx
BSNMHC
BSCO
Method 1
"Torque-Speed"
22.30
10.08
2.58
Method 2
"BSFC"
4.45
8.03
1.99
Method 3
"ECM Fuel Specific"
6.61
8.44
2.11
Note: values in white cells were validated successfully, while values shown in gray
cells were not validated.
       As is the case with all simulation results, no model values should be used until those
results are somehow validated against a set of real test data. The generation of a validation data
set was a considerable challenge, as it required in-use testing with PEMS to be performed, while
at the same time requiring comparison to some acceptable form of reference measurement.  This
was required because the output of the model is  a set of deltas between a PEMS measurement
and a laboratory reference measurement.

       Two methods of  generating  validation  data  were used in  this program.  The primary
method involved on-road field testing using one of the PEMS that was examined during this
program.  The reference for this on-road validation testing was the CE-CERT Mobile Emission
Laboratory,     which     is     operated    by    the     University     of    California-
Riverside.  This unique facility incorporates a full-flow CVS  dilution tunnel and measurement
system into a trailer which can be pulled behind a Class 8 heavy-duty truck.  During validation
testing, truck exhaust was sampled simultaneously by the PEMS and the mobile laboratory, in
order to generate deltas.  As an added quality assurance measure, the mobile laboratory was
correlated to the SwRI reference laboratory test cell,  in order to eliminate any potential effect of
biases between the two facilities on the model validation effort.

       A  secondary  validation  data set was generated  in the SwRI dynamometer laboratory
reference test cell.  This was done because the on-road validation could not incorporate any form
of reference torque and fuel flow measurements,  to  allow validation of torque and BSFC error
terms.  Therefore, selected portions of the on-road testing  operation were re-played  in the
dynamometer laboratory,  in order to try to validate the errors predicted by the model for torque
and BSFC sources.

       The final result of these validation exercises is depicted in Table 12. As noted  in that
table, the model result for NOX was validated only for calculation Method 1.  The model result
for NMHC was validated for all three calculation  methods, while the CO results did not validate
for any of the three calculation methods. As has been noted earlier, the test engines used during
this program generated very low levels of CO,  orders of magnitude below the NTE thresholds,
and therefore the model result based on that data  was not likely to be a good predictor of actual
measurement errors at the CO compliance threshold. However, the Steering Committee noted
SwRI Report 03.12024.06
xvin

-------
that actual engines which would be evaluated during the HDIUT program are also likely to be
several orders of magnitude below the NTE threshold, and therefore the lack of validation of CO
was not deemed to be a significant problem.

       Efforts were  made to examine  the  reasons  for the  lack  of validation of NOX for
calculations Methods 2 and 3.  These involved examination of both the model  results and the
validation data sets to determine if any errors were made or issues could be resolved.  It should
be noted that these two methods predicted considerably smaller overall measurement allowances,
as compared to Method 1.  However, the CE-CERT  on-road validation deltas  for those same
methods were larger, resulting in  a lack of validation.  As a result, after considerable Steering
Committee discussion, the values for Methods 2 and 3 were deemed not usable as candidates for
measurement allowance generation.

       The methodology for selecting measurement allowance  values from among the three
calculation methods called for a single method to be chosen for all three pollutants.  With CO not
considered relevant for this purpose, only Method 1  contained  validated  values  for the other
pollutants.  Therefore, the candidate measurement allowance values for Method  1 were adopted
as the basis for calculating the final measurement allowances.  These percentage values were
applied  to the appropriate NTE thresholds, as given in Table 1, in order  to generate the final
allowances which were given earlier in Table 2.

       It should be noted, however, that as part of the final agreement reached by the Steering
Committee on the Method 1 values, EPA indicated its desire to continue to  examine the possible
reasons for lack of validation, as well as the potential to modify the model and the error surfaces
in order to correct the issues. If upon further examination, this path appeared promising in terms
of being able to achieve validation of all three calculation methods,  then a further cooperative
program would be initiated to revise the model result.   However,  any revised  measurement
allowance values which were generated as a result of such a future program would not take effect
before the 2010 model year.
SwRI Report 03.12024.06                       xix

-------
                              LIST OF ACRONYMS

Auxiliary Emission Control Device        AECD
American Society for Testing and Materials ASTM
Brake-Specific                          BS
Brake-Specific Fuel Consumption          BSFC
Bulk Current Injection                   BCI
California Air Resources Board            CARB
Center for Environmental Research
& Technology                          CE-CERT
Code of Federal Regulations              CFR
Constant Volume Sampling               CVS
Controller Area Network                 CAN
Design of Experiment                   DOE
Diesel Particulate Filter                  DPF
Electromagnetic Interference              EMI
Electronic Flow Meter                   EFM
Electrostatic Discharge                   ESD
Empirical Distribution Function            EDF
Engine Coolant Temperature              ECT
Engine Control Module                  ECM
Engine Manufacturer's Association        EMA
Environmental Protection Agency          EPA
Heavy Duty In-Use Testing               HDIUT
Heavy Heavy Duty                      HHD
Intake Manifold Temperature             IMT
Laminar Flow Element                   LFE
Light Heavy Duty                       LHD
Median Absolute Deviation               MAD
Mobile Emissions Laboratory             MEL
Medium Heavy Duty                     MHD
Memorandum of Agreement              MO A
Non-Dispersive Ultraviolet               NDUV
Non-Dispersive Infrared                 NDIR
Nonmethane Cutter                      NMC
Nonmethane Hydrocarbon                NMHC
Not To Exceed                          NTE
Portable Emission Measurement System    PEMS
Power Spectral Density                  PSD
Radio Frequency Interference             RFI
Root Mean Square                      RMS
SEMTECH-DS SN G05-SDS04           PEMS 1
SEMTECH-DS SN G05-SDS02           PEMS 2
SEMTECH-DS SN G05-SDS03            PEMS 3
SEMTECH-DS SN G05-SDS01            PEMS 4
SwRI Report 03.12024.06
xx

-------
SEMTECH-DS SN D06-SDS01           PEMS 5
SEMTECH-DS SN D06-SDS06           PEMS 6
SEMTECH-DS SN F06-SDS02           PEMS 7
Society of Automotive Engineers          SAE
Southwest Research Institute             SwRI
Ultra-Low Sulfur Diesel                 ULSD
Wide Open Throttle                    WOT
SwRI Report 03.12024.06                    xxi

-------
                              TABLE OF CONTENTS

                                                                               Page

EXECUTIVE SUMMARY	i
LIST OF ACRONYMS	xx
LIST OF FIGURES	xxvi
LIST OF TABLES	xxxv
1.0   INTRODUCTION	1
1.1  Objective	1
1.2  Background	1
1.2.1    Measurement Allowance Program Test Plan	1
1.2.2    PEMS Steering Committee	2
1.2.3    Portable Emission Measurement Systems (PEMS) Description and Function	2
1.2.4    PEMS Operations at SwRI	5
1.2.5    Emission Calculation Methods for In-Use Testing	6
1.2.5.1   Calculation Method 1 - "Torque" Method	6
1.2.5.2   Calculation Method 2-"BSFC" Method	7
1.2.5.3   Calculation Method 3 - "Fuel Specific" Method	7
1.3  Monte Carlo Model Simulation	8
1.4  1065 PEMS and Laboratory Audit	9
1.5  Engine Dynamometer Laboratory Testing	9
1.6  Environmental Chamber Testing	10
1.7  Exhaust Flow Meter Testing	10
1.8  Model Validation	11
1.9  Measurement Allowance Generation	12
2.0   MONTE CARLO MODEL	14
2.1  Model Background	14
2.1.1    Reference NTE Events	14
2.1.2    Error Surfaces	19
2.1.3    Error Surface Sampling and Interpolation	25
2.1.4    Brake-Specific Emissions  Calculations	27
2.1.5    Periodic Drift Check	29
2.1.6    Time Alignment for NOX and CO	30
2.1.7    Convergence and Number of Trials	31
2.1.8    Simulation Output	32
2.1.9    Step-by-Step Simulation Example	33
2.1.10   Measurement Allowance	36
2.1.11   Validation	37
3.0   1065 PEMS AND LABORATORY AUDIT	45
3.1  Audit Objective	45
3.2  Overview of 1065 Audit Activities	45
3.2.1    Laboratory Audits	46
3.2.2    PEMS Audits	47
3.3  Gas Analyzer Linearity Verifications	47
3.4  1065 Gas Analyzer Verifications	59
SwRI Report 03.12024.06                     xxii

-------
3.4.1     1065.350 H2O Interference for CO2NDIR	63
3.4.2     1065.355 H2O and CO2 Interference for CO NDIR	63
3.4.3     1065.360 FID Optimization Methane Response	64
3.4.4     1065.362 Non-stoichiometric Raw FID O2 Interference	64
3.4.5     1065.365 Nonmethane Cutter Penetration Fractions	64
3.4.6     1065.370 CO2 and H2O Quench Verification for NOX CLD	65
3.4.7     1065.372 HC and H2O Interference for NOXNDUV	65
3.4.8     1065.376 Chiller NO2 Penetration	66
3.4.9     1065.378 NO2-to-NO Converter Conversion	69
3.5   1065 Exhaust Flow Meter Linearity Verification	70
3.5.1     Five-Inch Exhaust Flow Meter Linearity	71
3.5.2     Four-Inch Exhaust Flow Meter Linearity	74
3.5.3     Three-Inch Exhaust Flow Meter Linearity	76
4.0    ENGINE DYNAMOMETER LABORATORY TESTING	79
4.1   Engine Testing Objectives	79
4.2   Test Engines and Dynamometer Laboratory	79
4.3   40-Point Torque andBSFCMap Generation and Error Surface	82
4.4   Steady-State Repeat Engine Testing and Error Surfaces	85
4.4.1     Engine 1 Detroit Diesel Series 60 Steady-State	86
4.4.2     Engine 2 Caterpillar C9 Steady-State	92
4.4.3     Engine 3 International VT365 Steady-State	98
4.4.4     Steady-State Concentration Error Surface Generation	104
4.5   Transient Engine Testing and Error Surfaces	108
4.5.1     Engine 1 Detroit Diesel Series 60 Transient	110
4.5.2     Engine2 Caterpillar C9 Transient	116
4.5.3     Engine 3 International VT365 Transient	119
4.5.4     Transient Concentration Error Surface Generation	122
4.5.5     Transient Flow Meter Error Surface Generation	129
4.5.6     Transient Dynamic Error Surface Generation	132
4.6   Interacting Parameters - Warm-Up Test Error Surface	136
4.6.1     Interacting Parameters - Warm-Up Test Procedure	136
4.6.2     Interacting Parameters - Warm-Up Data Analysis	138
4.6.3     Interacting Parameters - Warm-Up Error Surface Generation	141
4.7   Torque and BSFC Interacting Parameters - Design of Experiment	144
4.7.1     Interacting Parameters - DOE Data Analysis	146
4.7.2     Engine 1 Detroit Diesel Series 60 DOE	146
4.7.3     Engines International VT3 65 DOE	148
4.7.4     Interacting Parameters - DOE Error Surface Generation	150
4.8   Torque and BSFC Independent Parameters Sensitivity Analysis	155
4.8.1     Independent Parameters Data Analysis	158
4.8.2     Independent Parameters Error Surface Generation	159
4.9   Torque and BSFC Interpolation Errors	161
4.9.1     Interpolation Error Surface Generation	164
4.10     Exhaust Flow Meter Testing	166
4.10.1    Pulsation Test	166
4.10.2   Pulsation Error Surface Generation	168
SwRI Report 03.12024.06                      xxiii

-------
4.10.3    Non-Uniform Velocity Profile Swirl Test	169
4.10.4    Swirl Error Surface Generation	171
4.10.5    Tailpipe Wind Test	172
4.11     Torque and BSFC - OEM Supplied Error Surfaces	176
4.12     Time Alignment Error Surfaces	177
5.0   ENVIRONMENTAL CHAMBER TESTING	181
5.1  Environmental Testing Objective	181
5.2  Environmental Testing Procedure	181
5.3  Baseline Testing	185
5.4  Temperature Chamber Testing	193
5.4.1     Temperature Error Surface Generation	200
5.5  Pressure Chamber Testing	207
5.5.1     Pressure Error Surface Generation	217
5.6  Radiation Chamber Testing	220
5.6.1     Bulk Current Injection	220
5.6.2     Radiated Immunity	227
5.6.3     Electrostatic Discharge	233
5.6.4     Conducted Transients	238
5.6.5     Radiation Error Surface Generation	245
5.7  Vibration Table Testing	250
5.8  Ambient Hydrocarbon Testing	257
5.8.1     Ambient Hydrocarbon Error Surface Generation	263
6.0   MODEL RESULTS AND VALIDATION	266
6.1  Model Results	266
6.2  Results of Drift Correction	266
6.3  Convergence Results from MC Runs	270
6.4  Delta BS Emissions Plots for 95th Percentiles	277
6.5  Sensitivity Based on Variance	282
6.6  Sensitivity Based on Bias and Variance	295
6.7  CE-CERT Mobil Emission Laboratory Correlation	307
6.8  CE-CERT On Road Validation Testing	313
6.9  Laboratory Replay Testing	314
6.10     Validation Results	340
7.0   MEASUREMENT ALLOWANCE GENERATION AND CONCLUSIONS	352
7.1  Measurement Error Allowance Results	352
7.2  Conclusions	364
7.2.1     Engine-Installation-PEMS Variability	365
7.2.2     PEMS 1065 Audit Failures	365
7.2.3     Method 2 and Method 3 Validation for NOX	366
7.2.4     PEMS Sampling Handling System Issues and Overflow Checks	369
7.2.5     Lessons Learned for Future Programs	370
SwRI Report 03.12024.06                     xxiv

-------
Appendix                                                                 No. of Pages

A     PEMS Operation Log	11
B     Brake-Specific Emission Calculations for NOX, CO, andNMHC	5
C     Crystal Ball Output File Description	15
D     Monte Carlo Spreadsheet Computations	19
E     40-Point Torque and BSFC Map Data	6
F      Steady-State Error Surface Data	30
G     Transient Error Surface Data	19
H     Interacting Parameters -DOE Error Surface Data	9
I      Interpolation Torque andBSFC Error Maps	3
J      Evaluation of Manufacturer Supplied Error Surfaces	4
K     Environmental Chamber Testing Results and Error Surfaces	39
L     Measurement Allowance Test Plan -Final Version	75
M     CE-CERT  On-road Validation Testing Report	93
SwRI Report 03.12024.06                      xxv

-------
                                 LIST OF FIGURES
1            Sensors Inc. SEMTECH-DS Portable Emissions Analyzer	3
2            SEMTECH EFM2 Exhaust Flow Meter and Control Unit	5
3            Method 1 BSNOX Values for Reference NTE Events	17
4            Method 1 BSCO Values for Reference NTE Events	17
5            Method 1 BSNMHC Values for Reference NTE Events	18
6            Error Surface Construction:  PEMS vs. Laboratory Results	23
7            Error Surface Construction:  (PEMS - Lab) vs. Laboratory Results	24
8            Error Surface Construction:  Error at Variability Index vs.  Laboratory Results .. 25
9            Truncated Standard Normal and Uniform Probability Density Functions	26
10           Steady-State NOX Error Surface with Example Sampling for a Reference NTE
             Event	27
11           Periodic Drift Check Flowchart	30
12           Overview of Monte Carlo Simulation for BSNOX	33
13           Error Surfaces Included in Monte Carlo Simulation	35
14           Plot of Model-Generated Empirical Distribution Functions for Two Percentiles 40
15           Plot of On-Road and Model-Generated Empirical Distribution Functions	41
16           STEC Inc. Model SGD-710C Gas Divider with SEMTECH-DS PEMS	48
17           SwRI Gas Humidification and Blending Cart	60
18           NO2 Chiller Penetration Audit Results	67
19           NOX Concentration Pooled Delta Data From Engine 1 Steady-State Repeat
             Testing Prior to Chiller NO2 Penetration Retrofit Installation	68
20           NOX Concentration Pooled Delta Data From Engine 1 Steady-State Repeat
             Testing After Chiller NO2 Penetration Retrofit Installation	69
21           SwRI LFE Flow Stand Manometers and Reference LFEs	70
22           Sensors Inc. EFM Mounted on the SwRI LFE Flow Stand	71
23           5-Inch Sensors EFM Exhaust Flow Pooled Delta Data From Engine 1 Steady-
             State Repeat Testing 	73
24           5-Inch Horiba Exhaust Flow Meter Pooled Delta Data From Engine 3 Steady-
             State Repeat Testing	74
25           4-Inch EFM Exhaust Flow Pooled Delta Data From Engine 2 Steady-State Repeat
             Testing	76
26           3-Inch EFM Exhaust Flow Pooled Delta Data From Engine 3 Steady-State Repeat
             Testing	78
27           PEMS Instrumentation Setup in Dynamometer Test Cell	79
28           Engine 1  (HHD) - 14L DDC Series 60	80
29           Engine 2  (MHD) - Caterpillar C9	80
30           Engine 3  (LHD) - International VT 365	81
31           Test Cell  Exhaust System showing PEMS Flowmeters and Sampling Points	82
32           Engine 1  - Detroit Diesel Series 60 Lug Curve and 40-Point Map	83
33           Engine 2  - Caterpillar C9 Lug Curve and 40-Point Map	84
34           EngineS  -International VT365 Lug Curve and 40-Point Map	84
SwRI Report 03.12024.06                     xxvi

-------
35           NOX Concentration Pooled Deltas for repeat steady-state testing on engine 1 after
             NO2 Penetration Upgrade	88
36           CO Concentration Pooled Deltas for Repeat Steady-State Testing on Engine 1.. 89
37           Mean Raw HC Concentrations for Engine 1 Steady-State Testing	90
3 8           PEMS NMHC Concentrations for Engine 1 Steady-State Testing	91
39           Pooled EFM Deltas for Engine 1 Steady-State Testing - 5-inch Flow Meter	92
40           Corroded RH Sensor (Left) Compared to a New RH Sensor (Right)	93
41           Disassembled RH Sensor Manifold with RH Sensor	94
42           Engine 2 Initial Steady-State Repeat NOX Results Showing Time Dependent
             Concentration Shift	95
43           Raw Hydrocarbon Levels for Engine 2 Steady-State Testing	96
44           PEMS NMHC Concentrations for Engine 2 Steady-State Testing	96
45           Pooled EFM Deltas for Engine 2 Steady-State Testing - 4 Inch Flow Meter	98
46           PEMS NOX Concentration versus Mean Laboratory Reference for Engine 3
             Steady-State Testing	99
47           International VT365 Point 35 NOX Concentrations During  Steady-State Repeat
             Testing	100
48           International VT365 Point 30 NOX Concentrations During  Steady-State Repeat
             Testing	101
49           Pooled EFM DelTas for Engine 3 Steady-State Testing - 3-Inch Flow Meter.. 102
50           Pooled Horiba OBS-2200 Exhaust Flow Rate Deltas For Engine 3 Steady-State
             Testing	102
51           Pooled NOX Deltas for the Horiba OBS-2200 During Engine 3 Steady-State
             Testing	103
52           Pooled THC Measurements for the Horiba OBS-2200 During Engine 3 Steady-
             State Testing	104
53           Combined Error Surface for Steady-State NOX Concentration	105
54           Final Error  Surface for Steady-State NOx Concentration	106
55           Combined Error Surface for Steady-State Exhaust Flow Rate	107
56           Final Error  Surface for Steady-State Exhaust Flow Rate	108
57           Engine 1 Example Speed Traces During Transient Testing	Ill
58           Engine 1 Example Torque Traces During Transient Testing	112
59           Engine 1 NOX Concentration Traces During Transient Testing Showing Outlying
             Events	113
60           Engine 1 Pooled Transient Test NTE Brake-Specific NOX Results	114
61           Engine 1 Pooled Transient Test NTE Brake-Specific NOX MAD Results	114
62           Engine 1 Pooled PEMS NTE NOX Concentration Data Versus the Laboratory
             Mean	115
63           Engine 1 Pooled PEMS NTE Exhaust Flow Rate Data Versus the Laboratory
             Mean	116
64           Engine 2 Pooled Transient Test NTE Brake-Specific NOX Results	117
65           Engine 2 Pooled Transient Test NTE Brake-Specific NOX MAD Results	117
66           Engine 2 Pooled PEMS NTE NOX Concentration Data Versus the Laboratory
             Mean	118
67           Engine 2 Pooled PEMS NTE Exhaust Flow Rate Data Versus the Laboratory
             Mean	119
SwRI Report 03.12024.06                     xxvii

-------
68           Engine 3 Pooled Transient Test NTE Brake-Specific NOX Results	120
69           Engine 1 Pooled Transient Test NTE Brake-Specific NOX MAD Results	120
70           Engine 3 Pooled PEMS NTE NOX Concentration Data Versus the Laboratory
             Mean	121
71           Engine 3 Pooled PEMS NTE Exhaust Flow Rate Data Versus the Laboratory
             Mean	122
72           Engine 1 Uncorrected Transient Flow-Weighted NOX Concentration Errors .... 123
73           Engine 1 Transient and Interpolated Steady-State MAD Values with Resulting
             Scaling Factor	124
74           Error Surface for Engine 1 Transient Flow-Weighted NOX Concentration	125
75           Engine 3 Transient and Interpolated Steady-State MAD Values with Resulting
             Scaling Factor	126
76           Error Surface for Engine 3 Transient Flow-Weighted NOX Concentration	127
77           Final Error Surface for Transient Flow-Weighted NOX Concentration	128
78           Error Surface for Engine 1 Transient Exhaust Flow Rate	130
79           Error Surface for Engine 2 Transient Exhaust Flow Rate	131
80           Error Surface for Engine 3 Transient Exhaust Flow Rate	131
81           Final Error Surface for Transient Exhaust Flow Rate	132
82           Final Error Surface for Dynamic ECM Fuel Rate	133
83           Final Error Surface for Dynamic ECM Speed	134
84           Final Error Surface for Dynamic Interpolated Torque	135
85           Final Error Surface for Dynamic Interpolated BSFC	136
86           Caterpillar C9 Engine Partially Enclosed in the Insulating Box Prior to the Warm-
             UpTest	138
87           Caterpillar C9 Engine Fully Enclosed in the Insulating Box Prior to the Warm-Up
             Test	138
88           Example of Warm-up Test Bias Correction, DDC Engine Part Load Test	140
89           Example of Determination of NTE Zone Entry for Warmup Test, DDC Engine
             Part Load Point	141
90           Error Surface for Interacting parameters - Warm-Up  Delta Torque	143
91           Error Surface for Interacting parameters Warm-Up Delta B SFC	143
92           Error Surface for Interacting Parameters - DOE Engine 1 Delta Torque	151
93           Error Surface for Interacting Parameters - DOE Engine 1 Delta B SFC	152
94           Error Surface for Interacting Parameters - DOE Engine 3 Delta Torque	153
95           Error Surface for Interacting Parameters - DOE Engine 3 Delta B SFC	153
96           Error Surface for Interacting Parameters - DOE Delta Torque Final	154
97           Error Surface for Interacting Parameters - DOE Delta B SFC Final	155
98           Intake Air Low Humidity Control  SYSTEM	158
99           Error Surface for Independent Parameters Delta Torque	160
100          Error Surface for Independent Parameters Delta BSFC	161
101          Detroit Diesel Series 60 Down Selected  20-Point Map	162
102          Caterpillar C9 Down Selected 20-Point Map	162
103          International VT365 Down Selected 20-Point Map	163
104          Interpolated Torque Error (% Peak Torque) by Speed (rpm) and Fuel Rate (g/s)
             for Engine #1	164
105          Torque Interpolation Error Surface	165
SwRI Report 03.12024.06                     xxviii

-------
106          BSFC Interpolation Error Surface	166
107          Pulsation Test SEMTECH-DS Exhaust Flow Rate Deltas - Raw Data	167
108          Pulsation Test SEMTECH-DS Exhaust Flow Rate Delta - Corrected for Steady-
             State Bias	168
109          Error Surface for Pulsation Exhaust Flow Rate	169
110          Swirl Test SEMTECH-DS Exhaust Flow Rate Deltas - Raw Data	170
111          Swirl Test SEMTECH-DS Exhaust Flow Rate Deltas - Corrected for Steady-State
             Bias	171
112          Error Surface for Swirl Exhaust Flow Rate	172
113          High Velocity Blower System	173
114          EFM Wind Test Flow Schematic	173
115          Swirl Test SEMTECH-DS Exhaust Flow Rate Deltas - Raw Data	174
116          Wind Test SEMTECH-DS Exhaust Flow Rate Deltas - Corrected for Steady-
             State Bias	175
117          Wind Test Mean Delta Values With 95 % Confidence Level Bars	176
118          Pooled and Weighted Brake-Specific Time Alignment Error Data for NOX and
             CO	180
119          Systematic High CC>2 Bias During Zero Air Reference Gas Measurement	184
120          CC>2 Concentration Decay During Zero Air Measurement Following the Quad
             Blend Span Reference Gas	184
121          PEMS NC>2 Delta Data During Initial Environmental Baseline Testing	187
122          PEMS 2 NO and NO2 Response During Hour 1  of Initial Environmental Baseline
             Testing	188
123          PEMS 2 NO2 and NO Response With Thermoelectric Chiller Bypassed and
             Reconnected	189
124          PEMS 2 Environmental Baseline Zero Delta Measurements	190
125          PEMS 2 Environmental Baseline Audit Delta Measurements	191
126          PEMS 2 Environmental Baseline Span Delta Measurements	192
127          5-Inch EFM Environmental Baseline Zero Delta Measurements	193
128          Temperature Histograms for NEI Model and Test Profile	194
129          Temperature Test Profile and Moving Average	195
130          Thermotron SM-32 Temperature Control Chamber with Supplemental Liquid
             Nitrogen Cylinder	196
131          PEMS 2 Environmental Temperature Zero Delta Measurements	197
132          PEMS 2 Environmental Temperature Audit Delta Measurements	198
133          PEMS 2 Environmental Temperature Span Delta Measurements	199
134          5-Inch EFM Environmental Temperature Zero Delta Measurements	200
135          Error Surface for Environmental Temperature NOX Concentration Zero Delta
             Measurements	202
136          Error Surface for Environmental Temperature NOX Concentration Audit Delta
             Measurements	202
137          Error Surface for Environmental Temperature NOX Concentration Span Delta
             Measurements	203
138          Final Error Surface for Environmental Temperature NOX Concentration	204
139          Corrected CO Deltas Measured During Environmental Temperature Testing... 205
140          Final Error Surface for Environmental Temperature CO Concentration	206
SwRI Report 03.12024.06                     xxix

-------
141          Error Surface for Environmental Temperature Exhaust Flow Rate Delta
             Measurements	207
142          Pressure Histograms for NEI Model and Test Profile	208
143          Pressure Test Profile and Moving Average	210
144          Altitude Chamber Top -Removed From Base	211
145          PEMS Equipment on Altitude Chamber Base	212
146          Altitude Chamber and Pressure Control Equipment During Testing	213
147          PEMS 2 Environmental Pressure Zero Delta Measurements	214
148          PEMS 2 Environmental Pressure Audit Delta Measurements	215
149          PEMS 2 Environmental Pressure Span Delta Measurements	216
150          5-Inch EFM Environmental Pressure Zero Delta Measurements	217
151          NMHC Corrected Delta Data for the Environmental Pressure Testing	218
152          Final Error Surface for Environmental Pressure NMHC Concentration	219
153          Error Surface for Environmental Pressure Exhaust Flow Rate	220
154          Bulk Current Injection Probe	221
155          Calibration Device for the Bulk Current Injection Probe	222
156          PEMS 7 in the Radiation Chamber Undergoing Bulk Current Injection Testing223
157          PEMS 7 Environmental Radiation BCI Zero Delta Measurements	224
158          PEMS 7 Environmental Radiation BCI Audit Delta Measurements	225
159          PEMS 7 Environmental Radiation BCI Span Delta Measurements	226
160          5-Inch EFM Environmental Radiation BCI Zero Delta Measurements	227
161          PEMS 7 and Radiation Antenna in the Absorber-Lined Radiation Chamber
             During Radiated Immunity Testing	228
162          Signal Generators, Amplifiers, Oscilloscopes and Other Electronics Used to
             Perform Radiation Testing	229
163          PEMS 7 Environmental Radiation Radiated Immunity Zero Delta Measurements
             	231
164          PEMS 7 Environmental Radiation Radiated Immunity Audit Delta Measurements
             	232
165          PEMS 7 Environmental Radiation Radiated Immunity Span Delta Measurements
             	232
166          5-Inch EFM Environmental Radiation Radiated Immunity Zero Delta
             Measurements	233
167          Electrostatic Discharge Simulator Used During Electrostatic Discharge Testing
             	234
168          Electrostatic Voltmeter Used to Calibrate the Electrostatic Discharge Simulator
             	235
169          PEMS 7 Environmental Radiation Electrostatic Discharge Zero Delta
             Measurements	236
170          PEMS 7 Environmental Radiation Electrostatic Discharge Audit Delta
             Measurements	237
171          PEMS 7 Environmental Radiation Electrostatic Discharge Span Delta
             Measurements	237
172          5-Inch EFM Environmental Radiation Electrostatic Discharge Zero Delta
             Measurements	238
173          Schaffner NSG 5200 Automotive Electronics Test System	239
SwRI Report 03.12024.06                     xxx

-------
174          Example Voltage Trace During a Voltage Spike with Slow Recovery	240
175          Example Voltage Trace During a Voltage Spike with Quick Recovery	241
176          Example Voltage Trace During a Voltage Burst	242
177          PEMS 7 Environmental Radiation Conducted Transient Zero Delta Measurements
             	243
178          PEMS 7 Environmental Radiation Conducted Transient Audit Delta
             Measurements	244
179          PEMS 7 Environmental Radiation Conducted Transient Span Delta
             Measurements	244
180          5-Inch EFM Environmental Radiation Conducted Transient Zero Delta
             Measurements	245
181          Combined Radiation Chamber Zero Delta Test Results	246
182          Combined Radiation Chamber Audit Delta Test Results	247
183          Combined Radiation Chamber Span Delta Test Results	247
184          Pooled Exhaust Flow Rate Zero Deltas Measured During Environmental
             Radiation Testing	249
185          Error Surface for Environmental Radiation Exhaust Flow Rate Delta
             Measurements	250
186          PEMS 3 in an Environmental Enclosure During Vibration Testing Using an
             Unholtz-Dickie Shaker System	251
187          Army M915 A2 Semi-Tractor Used to Generate Vibration  Spectra for Vibration
             Testing	252
188          Power Spectral Densities Evaluated For Vibration Testing	253
189          PEMS 3 Environmental Vibration Zero Delta Measurements	254
190          PEMS 3 Environmental Vibration THC Zero Delta Measurements	255
191          PEMS 3 Environmental Vibration Audit Delta Measurements	256
192          PEMS 3 Environmental Vibration Span Delta Measurements	256
193          PEMS 3 Environmental Vibration NOX Span Delta Measurements	257
194          THC Measurements for Test 1 of the Ambient Hydrocarbon Test Sequence.... 260
195          THC Measurements for Test 2 of the Ambient Hydrocarbon Test Sequence.... 262
196          THC Measurements for Test 6 of the Ambient Hydrocarbon Test Sequence.... 263
197          PEMS 3 NMHC Response Versus FID Air Methane Contamination Measured
             During Ambient Hydrocarbon Testing	264
198          Error Surface for Ambient Hydrocarbon Testing	265
199          Percent of Trials Deleted for Each Reference NTE Event Due to Periodic Drift
             Check for BSNOx Method 1	267
200          Percent of Trials Deleted for Each Reference NTE Event Due to Periodic Drift
             Check for BSNOx Method 2	267
201          Percent of Trials Deleted for Each Reference NTE Event Due to Periodic Drift
             Check for BSNOX Method 3	268
202          Percent of Trials Deleted for Each Reference NTE Event Due to Periodic Drift
             Check for BSNMHC Method 1	269
203          Percent of Trials Deleted for Each Reference NTE Event Due to Periodic Drift
             Check for BSNMHC Method 2	269
204          Percent of Trials Deleted for Each Reference NTE Event Due to Periodic Drift
             Check for BSNMHC Method 3	270
SwRI Report 03.12024.06                     xxxi

-------
205          Convergence Interval Width as a Percent of Threshold for BSNOX Method 1 ..271
206          Convergence Interval Width as a Percent of Threshold for BSNOX Method 2 .. 272
207          Convergence Interval Width as a Percent of Threshold for BSNOX Method 3 .. 272
208          Convergence Interval Width as a Percent of Threshold for BSNMHC Method 1
             	273
209          Convergence Interval Width as a Percent of threshold for BSNMHC Method 1274
210          Convergence Interval Width as a Percent of Threshold for BSNMHC Method 3
             	274
211          Convergence Interval Width as a Percent of Threshold for BSCO Method 1.... 275
212          Convergence Interval Width as a Percent of Threshold for BSCO Method 2.... 276
213          Convergence Interval Width as a Percent of Threshold for BSCO Method 3.... 276
214          Box Plot of 95th Percentile Delta BSNOX for Three Methods from 195 Reference
             NTE Events	278
215          Box Plot for 95th Percentile Delta BSCO for Three Methods from 195 Reference
             NTE Events	279
216          Box Plot for 95th Percentile Delta BSCO for Three Methods for 195 Reference
             NTE Events	280
217          Comparison of 95th Percentile Delta BSNOX for Methods 1, 2, and 3 for 195
             Reference NTE Events	281
218          Comparison of 95th Percentile Delta BSNMHC for Methods 1, 2, and 3 from 195
             Reference NTE Events	281
219          Comparison of 95th Percentile Delta BSCO for Methods 1, 2, and 3 from 195
             Reference NTE Events	282
220          Box Plot of Error Surface Sensitivity Based on Variance for BSNOX Method 1287
221          Box Plot of Error Surface Sensitivity Based on Variance for BSNOX Method 2288
222          Box Plot of Error Surface Sensitivity Based on Variance for BSNOX Method 3289
223          Box Plot of Error Surface Sensitivity Based on Variance for BSNMHC Method 1
             	290
224          Box Plot of Error Surface Sensitivity Based on Variance for BSNMHC Method 2
             	291
225          Box Plot of Error Surface Sensitivity Based on Variance for BSNMHC Method 3
             	292
226          Box Plot of Error Surface Sensitivity Based on Variance for BSCO Method  1. 293
227          Box Plot of Error Surface Sensitivity Based on Variance for BSCO Method 2. 294
228          Box Plot of Error Surface Sensitivity Based on Variance for BSCO Method 3.295
229          Distribution of Ideal BSNOX for 13 Reference NTE Events	297
230          Box Plot of error Surface Sensitivity Based on Bias and Variance for BSNOX
             Method 1	301
231          Box Plot of Error Surface Sensitivity Based on Bias and Variance for BSNOX
             Method 2	302
232          Box Plot of Error Surface Sensitivity Based on Bias and Variance for BSNOX
             Methods	303
233          Comparison of Typical CONTINUOUS NOx Mass Rate Data over RMC Cycle -
             CE-CERT versus SwRI	312
234          Engine Speed CAN Data Comparison for SwRI Laboratory Testing and CE-
             CERT On-Road Validation Testing (Route 2)	315
SwRI Report 03.12024.06                     xxxii

-------
235          Fuel Rate CAN Data Comparison for SwRI Laboratory Testing and CE-CERT
             On-Road Validation Testing (Route 2)	316
236          Boost Pressure CAN Data Comparison for SwRI Laboratory Testing and CE-
             CERT On-Road Validation Testing (Route 2)	317
237          Injection Timing CAN Data Comparison for SwRI Laboratory Testing and CE-
             CERT On-Road Validation Testing (Route 2)	318
238          Intake Manifold Temperature CAN Data Comparison for SwRI Laboratory
             Testing and CE-CERT On-Road Validation Testing (Route 2)	319
239          PEMS 5 Wet NOX Comparison for SwRI Laboratory Testing and CE-CERT On-
             Road Validation Testing (Route 2)	320
240          PEMS 5 Exhaust Flow Rate Comparison for SwRI Laboratory Testing and CE-
             CERT On-Road Validation Testing (Route 2)	321
241          PEMS 5 NOX Mass Flow Rate Comparison for SwRI Laboratory Testing and CE-
             CERT On-Road Validation Testing (Route 2)	321
242          Brake-Specific NOX Emission Deltas for PEMS 5 Method 1 Calculation Versus
             the Laboratory Reference (Laboratory Torque and BSFC)	323
243          Brake-Specific NOX Emission Deltas for PEMS 5 Method 1 Calculation Versus
             the Laboratory Method 1 Calculation (ECM Torque and BSFC)	324
244          Incremental Brake-Specific NOX Emission Deltas Due to ECM Torque and BSFC
             Errors for Calculation Method 1	325
245          Incremental Brake-Specific NOX Deltas Compared to the Incremental Torque and
             BSFC Errors Predicted by the Model for Calculation Method 1	326
246          Brake-Specific NOX Emission Deltas for PEMS 5 Method 2 Calculation Versus
             the Laboratory Reference (Laboratory Torque and BSFC)	327
247          Brake-Specific NOX Emission Deltas for PEMS 5 Method 2 Calculation Versus
             the Laboratory Method 2 Calculation (ECM Torque and BSFC)	328
248          Incremental Brake-Specific NOX Emission Deltas Due to ECM Torque and BSFC
             Errors for Calculation Method 2	329
249          Incremental Brake-Specific NOX Deltas Compared to the Incremental Torque and
             BSFC Errors Predicted by the Model for Calculation Method 2	330
250          Brake-Specific NOX Emission Deltas for PEMS 5 Method 3 Calculation Versus
             the Laboratory Reference (Laboratory Torque and BSFC)	331
251          Brake-Specific NOX Emission Deltas for PEMS 5 Method 3 Calculation Versus
             the Laboratory Method 3 Calculation (ECM Torque and BSFC)	332
252          Incremental Brake-Specific NOX Emission Deltas Due to ECM Torque and BSFC
             Errors for Calculation Method 3	333
253          Incremental Brake-Specific NOX Deltas Compared to the Incremental Torque and
             BSFC Errors Predicted by the Model for Calculation Method 3	334
254          ECM Broadcast Torque Errors Measured During Replay Validation Testing with
             a Caterpillar Cl5 Engine	335
255          ECM Broadcast Fuel Flow Rate Errors Measured During Replay Validation
             Testing with a Caterpillar C15 Engine	336
256          ECM-Based BSFC Errors  Measured During Replay Validation Testing with a
             Caterpillar C15 Engine	337
257          ECM Broadcast Torque Errors Measured During 40-Point Map Generation with a
             Detroit Diesel Series 60 Engine	338
SwRI Report 03.12024.06                     xxxiii

-------
258          ECM Broadcast Torque Errors Measured During 40-Point Map Generation with a
             Caterpillar C9 Engine	339
259          ECM Broadcast Torque Errors Measured During 40-Point Map Generation with
             an International VT365 Engine	340
260          Validation On-Road and Model Generated Empirical Distribution Functions for
             BSNOX Method 1	342
261          Validation On-Road and Model Generated Empirical Distribution Functions for
             BSNOX Method 2	343
262          Validation On-Road and Model Generated Empirical Distribution Functions for
             BSNOX Method 3	344
263          Validation On-Road and Model Generated Empirical Distribution Functions for
             BSNMHC Method 1	345
264          Validation On-Road and Model Generated Empirical Distribution Functions for
             BSNMHC Method 2	346
265          Validation On-Road and Model Generated Empirical Distribution Functions for
             BSNMHC Method 3	347
266          Validation On-Road and Model Generated Empirical Distribution Functions for
             BSCO Method 1	348
267          Validation On-Road and Model Generated Empirical Distribution Functions for
             BSCO Method 2	349
268          Validation On-Road and Model Generated Empirical Distribution Functions for
             BSCO Method 3	350
269          Regression Plot of 95th Percentile Delta BSNOX versus Ideal BSNOX for Method 1
             	353
270          Regression Plot of 95th Percentile Delta BSNOX versus Ideal BSNOX for Method 2
             	354
271          Regression Plot of 95th Percentile Delta BSNOX versus Ideal BSNOX for Method 3
             	355
272          Regression Plot of 95th Percentile Delta BSNMHC versus Ideal BSNMHC for
             Method  1	356
273          Regression Plot of 95th Percentile Delta BSNMHC versus Ideal BSNMHC for
             Method  2	358
274          Regression Plot of 95th Percentile Delta BSNMHC versus Ideal BSNMHC for
             Methods	359
275          Regression Plot for 95th Percentile Delta BSCO versus Ideal BSCO for Method 1
             	360
276          Regression Plot of 95th Percentile Delta BSCO versus Ideal BSCO for Method 2
             	361
277          Regression Plot of 95th Percentile Delta BSCO versus Ideal BSCO for Method 3
             	363
278          Variation in Zero Calibration of PEMS During On-Road Validation Testing... 368
279          PEMS Zero Calibration Variations during Laboratory Testing	369
SwRI Report 03.12024.06                     xxxiv

-------
                                 LIST OF TABLES
Table                                                                          Page

1            NTE Threshold Values Used for Measurement Allowance Program	ii
2            Final Measurement Allowances	ii
3            Laboratory Reference Methods	v
4            Magnitude of Error Terms for Steady-State Error Surfaces	ix
5            Magnitude of error terms for Transient Error Surfaces - gaseous analyzers and
             exhaust flow	xi
6            Magnitude of Transient Error TERMS  for ECM Variables	xi
7            Magnitude of Torque and BSFC Error  surfaces	xii
8            Magnitude of Error Terms for Exhaust Flow Error Surfaces	xiv
9            Magnitude of OEM Error Surfaces	xiv
10           Magnitude of Time Alignment Error Surfaces	xv
11           Magnitude of Error Terms for Environmental Error Surfaces	xvi
12           Model Results and Validation	xviii
13           NTE Thresholds for Measurement Allowance Program	12
14           Example of MEASUREMENT Allowance Determination from Test Plan	13
15           Reference NTE Events and Method 1 BS Emissions	15
16           Descriptive Statistics for BS Emissions for Reference NTE Events	18
17           Input Parameters for Reference NTE Events	19
18           Error Surfaces for Monte Carlo Simulation	21
19           Error Surfaces Used for Computing Brake-Specific Emissions by Three
             Calculation Methods	29
20           Example of Selection of the Measurement Error	37
21           1065 Audits and Performance Checks Required for the Measurement Allowance
             Program	46
22           Span Concentrations used for the SEMTECH-DS and Laboratory Analyzers	48
23           Dilute MEXA 7200D and Horiba CH4  Bench  1065 Linearity Verification
             Summary	49
24           Raw MEXA 7200D and Horiba CH4 Bench 1065 Linearity Verification Summary
             	49
25           PEMS 1 1065 Linearity Verification Summary	51
26           PEMS 2 1065 Linearity Verification Summary	52
27           PEMS 3 1065 Linearity Verification Summary	53
28           PEMS 4 1065 Linearity Verification Summary	54
29           PEMS 4 1065 Linearity Verification Summary Continued	55
30           PEMS 5 1065 Linearity Verification Summary	56
31           PEMS 6 1065 Linearity Verification Summary	57
32           PEMS 7 1065 Linearity Verification Summary	58
33           Horiba OBS-2200 1065 Linearity Verification Summary	58
34           Dilute MEXA 7200D and Horiba NMHC Bench 1065 Analyzer Verification
             Summary	60
SwRI Report 03.12024.06                     xxxv

-------
3 5           Raw MEXA 7200D and Horiba NMHC Bench 1065 Analyzer Verification
             Summary	61
36           PEMS 1 1065 Analyzer Verification Summary	61
37           PEMS 2 1065 Analyzer Verification Summary	61
38           PEMS 3 1065 Analyzer Verification Summary	61
39           PEMS 4 1065 Analyzer Verification Summary	62
40           PEMS 5 1065 Analyzer Verification Summary	62
41           PEMS 6 1065 Analyzer Verification Summary	62
42           PEMS 7 1065 Analyzer Verification Summary	62
43           Horiba OB S-2200 1065 Analyzer Verification Summary	63
44           5-Inch Sensors Inc. EFM Linearity Results	72
45           5-Inch Horiba Exhaust Flow Meter Linearity Results	72
46           4-Inch Sensors Inc. EFM Linearity Results	75
47           3-Inch Sensors Inc. EFM Linearity Results	77
48           NTE Event Descriptions from the Test Plan	109
49           NTE Transition Descriptions from the Test Plan	110
50           Final Gaseous Transient Error Surface Deltas	129
51           nlo and nhi Speed Definitions for Engines 1, 2, and 3	133
52           Warm-Up Test Torque Errors Summary	142
53           Warm-Up Test BSFC Errors Summary	142
54           Interacting Parameters - DOE Adjustment guidance	144
55           Interacting Parameters - DOE Test Matrix	145
56           Test Plan DOE Engine Operating Conditions	145
57           Example of DOE Baseline Correction for Engine 1	146
58           Interacting Parameters - DOE Speed and Torque Steady-State Mode Definition
             for Engine 1	147
59           Interacting Parameters - DOE Test Matrix for Engine 1	147
60           Interacting Parameters - DOE Engine 1 Bias Corrected Torque Deltas	148
61           Interacting Parameters - DOE Engine 1 Bias Corrected BSFC Deltas	148
62           Interacting Parameters - DOE Speed and Torque Steady-State Mode Definition
             for Engine 3	149
63           Interacting Parameters - DOE Test Matrix for Engine 3	149
64           Interacting Parameters - DOE Engine 3 Bias Corrected Torque Deltas	150
65           Interacting Parameters -DOE Engine 3 Bias Corrected BSFC Deltas	150
66           Example DOE Steady-State Variance Correction using the Mean SS MAD	151
67           Independent Parameters adjustment guidance from the Test Plan	155
68           Independent Parameters Test Modes from the Test Plan	156
69           Independent Parameters Speed and Torque Steady-State Mode Definitions	156
70           Selected Fuel Properties for INDEPENDENT Parameters  Testing	157
71           Independent Parameters Bias Corrected Torque Deltas	159
72           Independent Parameters Bias Corrected BSFC Deltas	159
73           Torque Interpolation Error Surface Values	165
74           BSFC interpolation error Surface Values	165
75           OEM Error Surface Deltas for Torque and BSFC	177
76           EFM and Vehicle Interface Adjustment and Weighting Factors Used for Time
             Alignment Error Generation	178
SwRI Report 03.12024.06                     xxxvi

-------
77           Brake-Specific Time Alignment Error Data forNOx	179
78           Brake-Specific Time Alignment Error Data for CO	179
79           Reference Gases and Typical Concentrations Used During Environmental
             Chamber Testing	182
80           Zero, Audit, and Span Delta Recording Strategy Used During Environmental
             Chamber Testing	183
81           Temperature Test Profile Definition	195
82           Median Environmental Baseline NOX Concentrations and Environmental
             Temperature Scaling Factors	201
83           Pressure Test Profile Definition	209
84           Hexane and Methane Contamination Combinations Used During Ambient
             Hydrocarbon Testing	258
85           Ambient Hydrocarbon Test Sequence	259
86           THC Measurements for Test 1 of the Ambient Hydrocarbon Test Sequence.... 261
87           THC Measurements for Test 2 of the Ambient Hydrocarbon Test Sequence.... 262
88           Summary of the Trials Deleted Due to Periodic Drift Check for BSNOX	268
89           Summary of Trials Deleted for Each Reference NTE Event Due to Periodic Drift
             Check for BSNMHC Method 3	270
90           Summary of BSNOX Convergence Interval Width as a Function of Threshold for
             195 Reference NTE Events	273
91           Summary of BSNMHC  Convergence Interval Width as a Function of Threshold
             for 195 Reference NTE  Events	275
92           Summary of BSCO Convergence Interval Width as a Function of Threshold for
             195 Reference NTE Events	277
93           Error Surface Sensitivity to Variance for 195 Reference NTE Events for BSNOX
             Method 1	283
94           Error Surface Sensitivity to Variance for 195 Reference NTE Events for BSNOX
             Method 2	283
95           Error Surface Sensitivity to Variance for 195 Reference NTE Events for BSNOX
             Methods	284
96           Error Surface Sensitivity to Variance for 195 Reference NTE Events for
             BSNMHC Method 1	284
97           Error Surface Sensitivity to Variance for 195 Reference NTE Events for
             BSNMHC Method 2	284
98           Error Surface Sensitivity to Variance for 195 Reference NTE Events for
             BSNMHC Method 3	285
99           Error Surface Sensitivity to Variance for 195 Reference NTE Events for BSCO
             Method 1	285
100          Error Surface Sensitivity to Variance for 195 Reference NTE Events for BSCO
             Method 2	285
101          Error Surface Sensitivity to Variance for 195 Reference NTE Events for BSCO
             Methods	286
102          Ideal BSNOX Values for 13 Reference NTE Events	296
103          Error Surface Sensitivity to Bias and Variance for 13 Reference NTE Events for
             BSNOX Method 1	298
SwRI Report 03.12024.06                    xxxvii

-------
104          Error Surface Sensitivity to Bias and Variance for 13 Reference NTE Events for
             BSNOxMethod2	298
105          Error Surface Sensitivity to Bias and Variance for 13 Reference NTE Events for
             BSNOX Method 3	298
106          Error Surface Sensitivity to Bias and Variance for 13 Reference NTE Events for
             BSNMHC Method 1	299
107          Error Surface Sensitivity to Bias and Variance for 13 Reference NTE Events for
             BSNMHC Method 2	299
108          Error Surface Sensitivity to Bias and Variance for 13 Reference NTE Events for
             BSNMHC Method 3	299
109          Error Surface Sensitivity to Bias and Variance for 13 Reference NTE Events for
             BSCO Method 1	300
110          Error Surface Sensitivity to Bias and Variance for 13 Reference NTE Events for
             BSCO Method 2	300
111          Error Surface Sensitivity to Bias and Variance for 13 Reference NTE Events for
             BSCO Method 3	300
112          Summary of Error Surface Sensitive to Bias and Variance for BSNOX Method 1
             	304
113          Summary of Error Surface Sensitive to Bias and Variance for BSNOX Method 2
             	304
114          Summary of Error Surface Sensitive to Bias and Variance for BSNOX Method 3
             	305
115          Summary of Error Surface Sensitive to Bias and Variance for BSNMHC Method
             1	305
116          Summary of Error Surface Sensitive to Bias and Variance for BSNMHC Method
             2	305
117          Summary of Error Surface Sensitive to Bias and Variance for BSNMHC Method
             3	306
118          Summary of Error Surface Sensitive to Bias and Variance for BSCO Method 1
             	306
119          Summary of Error Surface Sensitive to Bias and Variance for BSCO Method 2
             	306
120          Summary of Error Surface Sensitive to Bias and Variance for BSCO Method 3
             	307
121          Correlation Test Matrix	308
122          Correlation Testing Results for NTE Transient Cycle	310
123          Correlation Test Results for RMC 13-Mode SET Cycle	311
124          Summary of Model Validation Results	351
125          Measurement Error at Threshold for BSNOX Using Regression and Median
             Methods for Method 1	353
126          Measurement Error at Threshold for BSNOX Using Regression and Median
             Methods for Method 2	354
127          Measurement Error at Threshold for BSNOX Using Regression and Median
             Methods for Method 3	355
128          Measurement Error at Threshold for BSNMHC Using Regression and Median
             Methods for Method 1	357
SwRI Report 03.12024.06                    xxxviii

-------
129          Measurement Error at Threshold for BSNMHC Using Regression and Median
             Methods for Method 2	358
130          Measurement Error at Threshold for BSNMHC Using Regression and Median
             Methods for Method 3	359
131          Measurement Error at Threshold for B SCO Using Regression and Median
             Methods for Method 1	360
132          Measurement Error at Threshold for B SCO Using Regression and Median
             Methods for Method 2	362
133          Measurement Error at Threshold for B SCO Using Regression and Median
             Methods for Method 3	363
134          Measurement Error in Percent of NTE Threshold by Emissions and Calculation
             Method	364
135          Measurement Allowance at NTE Threshold by Emissions for Method 1	364
136          Lessons Learned During Gaseous Measurement Allowance Program	370
SwRI Report 03.12024.06                     xxxix

-------
                               1.0    INTRODUCTION

    The intent of this section of this report is to provide an overview of the program objectives,
background material on the test plan, test methods, and equipment, and to briefly discuss the
rationale behind each of the major components of the measurement allowance program which
will be discussed in detail in later sections of the report.

       This revised version of the final report contains a number of changes made following
EPA's peer review of the original final report.  None of the results or conclusions of the original
report were affected as  part of the revision.  The changes made to the report primarily involved
additional clarifying language in areas were the peer review process indicated that the original
report was unclear or vague.

1.1    Objective

       The objective of this program was to determine a set of brake-specific measurement
allowances for the gaseous pollutants regulated under the Heavy-Duty In-Use  Testing (HDIUT)
program.  These measurement allowances are intended to  represent the incremental error
between  measuring emissions under  controlled conditions  in  a laboratory  with  lab-grade
equipment, and measuring emissions in the field using Portable Emissions Measurement Systems
(PEMS).  Measurement allowance values were generated  for non-methane hydrocarbons
(NMHC), carbon monoxide (CO), and oxide of nitrogen (NOX).

       The measurement allowances are fixed brake-specific  values, which are  intended to be
added to a given NTE threshold in order to provide an additional  compliance margin which
accounts for the relative error between laboratory and field measurements.

     The completion of this program was part of the resolution of a 2001  legal suit filed against
EPA  by the  Engine  Manufacturer's  Association  (EMA)  and  several individual  engine
manufacturers  regarding certain portions of the Not-to-Exceed (NTE)  standards. This dispute
was settled on June 3, 2003. As such, this program represents a cooperative effort between EPA,
EMA, and the  California Air Resources Board  (CARB).  The program was jointly funded by all
three organizations, and was conducted under the direction of a Steering Committee composed of
representatives of all three organizations, as well as representatives  of a number of individual
engine manufacturers which are EMA members.

1.2    Background

1.2.1   Measurement Allowance Program Test Plan

     The measurement allowance program  was  conducted according  to  procedures and
guidelines which were  laid out in a detailed test plan document titled Test Plan to Determine
PEMS Measurement Allowances for Gaseous Emissions Regulated under the Manufacturer-Run
Heavy-Duty Diesel Engine In-Use Testing Program. The final version of this document, which
forms the basis of the program, is dated October 24, 2005.  This document will be referred to as


SwRI Report 03.12024.06                  Page 1 of 371

-------
the Test Plan throughout the remainder of the report. This final version was modified from the
initial version, dated May 20, 2005, which was distributed publicly by EPA and is available via
the Internet at http://www.epa.gov/otaq/regs/hd-hwy/inuse/testplan.pdf The two documents are
identical  in  terms of overall methodology and scope, and  differ primarily in certain details
pertaining to either test execution or data analysis.

     The Test Plan was developed as a collaborative effort by the Steering Committee and all
modifications  made to the Test Plan were discussed and approved by the Steering Committee
prior to being performed.  Throughout the program, every  effort was made to  adhere to the
procedures given in the Test Plan. However, on numerous occasions, these procedures had to be
modified in response to unexpected occurrences  during testing, or as  a result of test data
generated during the program.  All such modifications that were not captured in the final Test
Plan document are included  in this  report.  When such changes are noted  in the report, the
original Test Plan procedure is given, along with the rationale for any changes, and the date at
which these modifications were approved by the Steering Committee.

1.2.2   PEMS Steering Committee

     The PEMS  Steering Committee was composed primarily of representatives  from EPA,
EMA,  CARB, and the  following  engine manufacturers: Cummins  Engine Company,  Detroit
Diesel  Corporation, Volvo Powertrain, Caterpillar Inc., International Engine Company,  and
Isuzu.   Representatives of  other  engine  manufacturers were  also present for some of the
Committee meetings.  PEMS Steering Committee meetings were convened on an as needed basis
by agreement  of the Committee members.  Generally, these  meetings were held on a monthly
basis, although bi-monthly  meetings were held late  in the program as key decisions were
required.  During the  majority of the program, weekly teleconferences were held to update the
group on progress and to provide  feedback to SwRI.   It should be noted that this  required a
considerable time and travel commitment on the part of Steering  Committee members.  SwRI
would like to acknowledge this contribution, and thank the Committee members for their efforts.

     In  general,  efforts were  made to achieve  unanimity among all Steering  Committee
members before deciding on a  course of action.  On the occasions that a unanimous opinion
could not be formed,  a  majority vote of Committee members was required to decide a given
issue.  In such cases,  which were generally rare, dissenting votes  were noted for the record as
desired by those in dissent.

1.2.3   Portable Emission Measurement Systems (PEMS) Description and Function

     The focus of this program was the evaluation  of Portable Emission Measurement Systems,
which are referred to by the acronym PEMS throughout this report. A key provision of the Test
Plan was that the PEMS to be evaluated  had to represent commercially available hardware. The
intent of this provision is captured in the following language taken from the Test Plan:

     "The PEMS used in this test plan must be standard in-production makes  and models that are for sale as
commercially available PEMS. In addition, PEMS and any support equipment must pass a "red-face" test with
respect to being consistent with acceptable practices for in-use testing. For example, use of large gas bottles that can
SwRI Report 03.12024.06                  Page 2 of 371

-------
not be utilized by the EPA/ARB/EMA HDIU enforceable program is unacceptable.  Furthermore, the equipment
must meet all safety and transportation regulations for use on-board heavy-duty vehicles."

     The original intent of the program  was to evaluate PEMS from two suppliers, Sensors
Incorporated and Horiba Instruments.  However, at the time of the start of this program, the
Horiba PEMS was still in the final stages of development,  therefore Horiba was  not able to
supply a commercially available unit.  As a result, the program was conducted primarily with the
Sensors Inc. SEMTECH-DS hardware.  Horiba was able to supply examples  of its OBS-2200
PEMS hardware in the later stages of the  program, but this was evaluated only for purposes of
supplemental information as time permitted.  The measurement allowance values were generated
using data from only the  Sensors SEMTECH-DS PEMS hardware.  The Test Plan called for
three different PEMS units from each Manufacturer to be examined.  Ultimately, due to various
scheduling and hardware issues, a total of seven SEMTECH-DS units were evaluated during the
program.  However, all seven PEMS were  not evaluated for every error source, and no more than
three PEMS were used during any given error test.

     The SEMTECH-DS PEMS included several major components.  The first of these is the
SEMTECH-DS portable gaseous emission analyzer unit.  This unit housed the gaseous emission
analyzers, the sampling system, and the sampling conditioning system.  The unit also contained
electronics for analyzer functions, interaction with the other system components, as well  as for
communication with the  user.  User interface was  accomplished  using a  remote interface
program  running on  a laptop  computer,  which  was connected to the SEMTECH-DS  via  an
Ethernet  cable or using  wireless communication.   The  front of  a  SEMTECH-DS PEMS is
pictured in Figure 1, showing the connection points for various other components.
                        •
              SEmTECH-
              sensors, inc.
                               .FSSCOMMUNPCArtDWS
                                            in .-WP!U
                                            7-3.BBA

                                            ZERO


                                            H. . 4 1^
                  HEATED FILTER   HEATEP l.irst

                 • •
                         moBiLE  Emission  ARALVZER

                          READY

                          RD FLAME

                          ALBtT

                          ZERO

                          OtlSTAlUG

                          DATA LOG
           ••
 AUX-2   ANALOG TO

•   •
   FIGURE 1. SENSORS INC. SEMTECH-DS PORTABLE EMISSIONS ANALYZER
SwRI Report 03.12024.06
    Page3 of 371

-------
     The SEMTECH DS uses a variety of different analyzers to measure various gaseous
emissions.  Total hydrocarbons (HC) are measured using a heated flame ionization detector
(HFID).  Carbon monoxide (CO) and carbon dioxide (CC^) are measured using a non-dispersive
infrared (NDIR) instrument.   Oxides of nitrogen (NOX) are measured using a non-dispersive
ultraviolet (NDUV) instrument, in which NO and NO2 are measured separately and combined
mathematically to produce a final NOX value.

     The SEMTECH DS units were initially supplied along with an add-on FID analyzer for
methane  measurement to allow for the determination  of non-methane hydrocarbons (NMHC).
However, only two of these units were supplied, which was not enough for all the PEMS used in
the program.  In addition, upon  evaluation,  the Steering  Committee  determined that these
methane  analyzers did not pass the red-face requirement outlined in the Test Plan and were not
suitable for field use.  Therefore, the methane analyzers were not used in the program, and
NMHC for the PEMS was determined as 0.98  times THC, as allowed under CFR Title 40 Part
1065.

     A second  key component of the SEMTECH-DS  PEMS is the SEMTECH EFM2 exhaust
flow meter.  This unit is a pitot-tube based exhaust flow measurement meter which  is design to
be attached to the end of a vehicle tailpipe for  direct measurement of exhaust flow over a wide
dynamic  range.  The control box contains a set of pressure transducers for differential and static
pressure measurement. The EFM control unit is connected to the main SEMTECH-DS unit via a
digital  interface cable, and flow data is  recorded along with gaseous emissions data and other
parameters in a  single data file.  An example of the  SEMTECH EFM2 flow meter  is shown in
Figure  2.  This flow meter also incorporates the sampling probe through which the SEMTECH-
DS emission analyzer samples exhaust  for  delivery to the  gaseous analyzers.  This  probe is
connected to the main SEMTECH-DS unit via a heated sampling line which is controlled to a
temperature of 19PC, in accordance with CFR Title 40 Part 1065.

       The third key component of the SEMTECH-DS is the vehicle interface. This interface is
used to read engine variables  broadcast  from the ECM digitally via CAN.   Variables are read
according to either the SAE J-1939 or SAE J-1708 protocol, depending on what is available from
a given engine.   These ECM broadcast variables are required for estimation of torque and fuel
consumption during in-use testing, as well as to determine entry into or exit from the NTE zone.

       A fourth component of the SEMTECH-DS is a  temperature and humidity probe.  This is
used by the SEMTECH-DS to monitor and record ambient temperature and humidity during in-
use testing. This probe is plugged into the main SEMTECH-DS unit, which takes the raw  sensor
data from the probe and converts it to temperature and humidity measurement values.

       Data  from all of these components is generally  recorded simultaneously, and stored in a
single data file for each test run on  a memory card in the main SEMTECH-DS unit.  This data
can later be  retrieved via a laptop computer either over a wireless connection or via a cabled
Ethernet  link. The laptop software interface also provides a  means of user interface for manual
operations, diagnostics, and monitoring of the SEMTECH-DS during testing.  The data recorded
by the  SEMTECH-DS is then post-processed to determine emission values and to review quality
assurance parameters.
SwRI Report 03.12024.06                  Page 4 of 371

-------
   FIGURE 2. SEMTECH EFM2 EXHAUST FLOW METER AND CONTROL UNIT

1.2.4   PEMS Operations at SwRI

     Once the PEMS hardware was delivered to SwRI for this program, modifications could
only be conducted in accordance with strict guidelines given in the Test Plan.   In  general,
modifications could only be conducted following approval from the Steering  Committee.  In
addition, PEMS operations were conducted only by SwRI staff in accordance with procedures
given in the standard documentation  available for the PEMS.  SwRI staff members were trained
by PEMS manufacturer representatives prior to the start of the program.  PEMS representatives
were not allowed to be present during actual test operations.  SwRI technicians Billy Valuk and
Richard Mendez were the PEMS operators during the program.

     In general, PEMS manufacturers were allowed access to the hardware during this program
under only two conditions.  The first was the failure of a 1065 audit performance check, in which
case, the PEMS manufacturer was offered an opportunity to correct the problem.  The second
condition was in the event of an equipment malfunction which could not normally be repaired by
an end user. In the event of such repairs, appropriate  1065  audits were repeated to validate the
operation of the repaired systems before testing continued.

     Throughout the course of the  program, there were a variety of instances  of both audit
failures and equipment malfunctions.   An  operating  log  of all of these occurrences was
maintained by SwRI throughout the course of the program.  The complete log is  included  in
Appendix A of this report. For each incident, the log includes the date of occurrence, observed
failure  symptoms, diagnostic steps, root cause analysis (if known), and corrective actions taken.
This log represents the collective PEMS operation experience with seven sets of PEMS hardware
over the course of roughly one year.
SwRI Report 03.12024.06
Page 5 of 371

-------
1.2.5  Emission Calculation Methods for In- Use Testing

     Once a set  of data has  been recorded  using PEMS hardware, calculations  must be
performed to determine brake-specific emission values in accordance with methods outlined in
40 CFR Part 1065  Subparts G and J.  The symbolic notation given in the formulas shown later in
this section is fully described in 40 CFR Part 1065 Subpart K.

     40CFR Part  1065 allows for the use of any of three different calculation methods in order
to  determine brake-specific  emission values from in-use test data.  The basic calculation of
brake-specific emissions requires three main inputs as follows:


            nr^   .    .       Mass     Concentration x Flow Rate
           BShmission =	=	
                               Work                 Power

      The three calculation methods vary somewhat in the means used to determine either the
Flow component or the Work component  of this calculation.  Each  of the three methods is
summarized below.  Because each method relies on different  inputs,  it is possible that each
method  of calculation  will react  differently  to  various  measurement errors.   Therefore,
measurement  allowances  must  be  examined  independently  for each  method.   However,
according  to the Test Plan methodology, only one of the three calculation  methods would be
selected to generate the final measurement  allowances.  The selection methodology is outlined
later in this introduction under the Measurement Allowance Generation section.

1.2.5.1  Calculation Method 1 - "Torque" Method

     Calculation Method 1 is analogous to the method used by most dynamometer laboratories,
and relies on direct input of both exhaust flow and torque. In the case of exhaust flow, this is the
flow rate measured by the same form of exhaust flow meter.  In the  case of the Sensors  Inc.
PEMS, this is the  value measured by the SEMTECH EFM2 exhaust flow meter. Work is not
measured  directly, but is instead calculated using  ECM  broadcast engine speed and ECM
broadcast engine torque.  While engine speed is directly measured by  the engine ECM, ECM
broadcast torque is an estimate based on a variety of other  parameters, therefore, torque cannot
be directly verified during in-use testing. A simplified formula for this method is:

                                 Methodl = -
       The more complete formula used for Method 1, using NOX as an example, is as follows:
eNOx
(glkW
hr}-

N
M * V
1V± N02 / ,
i=\
N
s
z=l

Speedt
(«vo,(
(rpm)*
ppm)
Tt(N
)*1
m}
60*1000*
0~6 *ri
*2*3
3600
( m
{ .
°0 * A "
' J \
14159*Ar"


SwRI Report 03.12024.06                 Page 6 of 371

-------
       It should be noted that calculation Method 1 is directly dependent on the accuracy of both
the exhaust flow meter and the torque estimation,  as well as  on the measurement of gaseous
concentration.  This formula is applied similarly for CO and HC by replacing the measured
concentration and molecular weight values for NOX with those for the pollutant being calculated.

1.2.5.2  Calculation Method 2 - "BSFC" Method

     This calculation  is designated solely for in-use testing, and is designed to minimize the
effect of errors related to the accuracy of the exhaust flow measurement.  Calculation Method 2
relies on flow weighting of individual readings during a test event. This means that although the
flow meter must be linear, it does not necessarily have to be accurate.  In addition, Method 2
uses a carbon balance method  to predict the fuel consumption rate, and a brake-specific fuel
consumption (BSFC) value to determine a final work term for the calculation.  The BSFC value
is generally calculated using ECM broadcast values for fuel rate and for torque.  A simplified
version of this method can be expressed as:


                           Method 2= -
                                          CO 2 fuel
                                         	—	x Work
                                         ECMfuel
The more complete formula for Method 2, again using NOX as an example, is:

                                                        mol\ .  .
                                        (xNOXt (ppm)) * 1 (T6 * »,.
                                              '
                           «,• [ — I * [xHCi (pPm)* 1 °~6 + (xCOt (%) + xCO2i (%))* 1 (T2 ]* At
                                                BSFC
                                                       kW-hr
       As mentioned earlier, Method 2 is not subject to accuracy errors for the exhaust flow
measurement, although that measurement must still be linear for the method to function properly.
Application of this formula to HC and CO is the same as what was outlined for Method 1.

L 2.5.3  Calculation Method 3 - "Fuel Specific " Method

     Method 3  does not use direct measurement of exhaust flow, but relies on a carbon balance
and ECM broadcast fuel rate to determine  mass.  The work term for Method  3 is determined
identically to the work term for Method 1; using the ECM broadcast values for engine speed and
torque to calculate work.  Method 3 entirely circumvents the use of an exhaust flow meter, but
for the  HDIUT program, EPA must approve the use of Method  3  for  a  given test and
manufacturer. A simplified version of Method 1 may be expressed as:
SwRI Report 03.12024.06
                              Page 7 of 371

-------
                                         I
                              Method 3 = •
   ECU fuel
ex	
    CO 2 fuel
       The more complete formula for Method 3, using NOX as an example is:
M

*w N
O2 fuel % ^-i
Mc ^


N
v
/_J
2=1
(*A^

xHCj^ppm)*!®
Speed ' i(rpm}*Ti
60*
/• \
v-v . #A/
"6 + (xCO,. (%) + xCO2j (%)) * 1 0~2
(jV-y»)*2*3.14159*A/"
1000*3600
       It should be noted that Method 3 is not subject to exhaust flow measurement accuracy
errors, but also that this method is wholly dependent on ECM broadcast values for both mass and
work determination.  Application of this formula to HC and CO is similar to that described for
Method 1.

1.3    Monte Carlo Model Simulation

      The desire for this program was to generate measurement allowances based on rigorous
statistical methods applied to a large body of data.  At the same time, it was desirable to exclude
outlier data caused by extreme measurement errors which were not considered representative of
normal in-use operations. A direct approach could have been to test PEMS against some kind of
mobile laboratory reference  (such  as the CE-CERT Mobile Emission Laboratory) on a large
number of vehicles, and quantify errors directly. However, such an approach would have been
prohibitively expensive in terms of both time and  funding.  In  addition, the desired laboratory
reference point for error comparison was certification testing, which is normally conducted in a
dynamometer laboratory facility.

      Given these factors,  the Steering Committee ultimately  elected to use a simulation
approach in order to generate the measurement allowances.  In this approach,  the  Steering
Committee would define all of the expected  sources of PEMS  measurement errors, based on
existing in-use testing expertise and understanding  of how the PEMS functioned.  Each of these
errors would be quantified using a series of controlled laboratory experiments, each designed to
isolate errors related to a single error source.  The  results of each experiment would essentially
be an empirical model of a given source of measurement error. In this report, these error models
are referred to  as error surfaces.   It is  important to note that each  of these error surfaces
represents an incremental error of PEMS measurement, as compared to  an associated laboratory
reference measurement.

      All of these error surfaces were programmed into a computer model, which employed
Monte Carlo random sampling methods to simulate the combined effects of all of these sources
of error on the final measured brake-specific value.  An ideal  data set for a given test event was
SwRI Report 03.12024.06                  Page 8 of 371

-------
run through the Model,  and all the various errors were applied to that data set in a randomly
chosen manner.  Brake-specific emission values were then calculated for both the ideal and
error-applied data sets, which were compared to yield a final measurement error.  The process
was repeated thousands of times, with many different ideal data sets, to generate a large, robust
data set which was evaluated to determine a final  set of combined measurement errors.  These
final errors, referred to in this report as deltas, were generated for each pollutant and for each
calculation method, for a final set of nine deltas; three for each pollutant. A complete description
of the Monte Carlo methodology and of the model is given in Section 2 of this report.

1.4    1065 PEMS and Laboratory Audit

     A key provision of both certification testing and compliance testing under the HDIUT
program is that manufacturers must use measurement equipment which meets the  requirements
outlined in 40 CFR Part 1065.  In  particular 40  CFR Part 1065  Subpart D outlines a set of
performance  checks  which a  measurement  system must  pass  to insure the  accuracy  and
reliability of the instruments.

     In light of these requirements,  the Test Plan outlined a process wherein both the SwRI
reference laboratory and the PEMS would be audited prior to the start of testing, in  accordance
with the procedures outlined in 40 CFR Part 1065 Subpart D.  The audit was conducted on all
PEMS and laboratory instrumentation.  In addition, a similar audit was also conducted on the
CE-CERT  Mobile Emission  Laboratory (MEL), which was later  used during the validation
process outlined in Section 1.8.  The performance checks were regularly repeated for both the
PEMS and the reference laboratory, in accordance with the requirements given in Subpart D. In
the event that a given PEMS failed a given Subpart D performance check during the initial audit,
the PEMS manufacturer was given an opportunity to correct the issue prior to the start of actual
testing,  subject to the  approval of the Steering Committee. The 1065 audit process and results
for the SwRI laboratory and the individual PEMS are described fully in Section 3 of this report.

1.5    Engine Dynamometer Laboratory Testing

     A substantial number of the individual error experiments were conducted  in an engine
dynamometer test cell located in the Department of Engine and Emissions Research at SwRI.
The test cell used for this program was Heavy Duty Transient Test Cell 27.  This particular test
cell at SwRI is fully  compliant with the procedures and methods  of 40  CFR Part 1065.  In
addition, the test cell incorporates additional equipment that can be used to  simulate operation at
high altitudes, and also to simulate a wide range of ambient conditions in the intake air supply of
the engine. These expanded test cell capabilities were required for the proper conduct of some of
the experiments outlined in the Test Plan.   SwRI technicians Gabriel Hernandez and Brian
Moczygemba were the engine operators during the program.  Billy Valuk  was the Test Cell 27
emissions cart operator during the program.

     In general, the tests conducted at this location involved simultaneous measurements made
by both PEMS and the Laboratory on running engines.   The engines were all equipped with
diesel paniculate filters (DPFs), in order to simulate the exhaust conditions of a 2007 or later
model year engine. Because the engines that were  tested were not 2007 model year engines, the
SwRI Report 03.12024.06                  Page 9 of 371

-------
NOX levels were roughly twice as high as those expected for such an engine. Three different test
engines were supplied to SwRI by participating engine manufacturers.  These engines were a
heavy heavy duty (HHD) engine supplied by Daimler Chrysler, a medium heavy duty (MHD)
engine supplied by Caterpillar, and a light heavy  duty (LHD) engine supplied by International.

     Tests conducted in the  dynamometer test cell included both steady-state  and transient
exhaust emission measurements.  In addition, a  wide variety  of experiments were conducted to
quantify errors in ECM broadcast torque and fuel rate, as compared to Reference Laboratory
measured values.  Full details of all of these experiments, and their results are given in Section 4
of the report.

1.6    Environmental Chamber Testing

     Another major portion of the Test Plan was devoted to characterizing PEMS measurement
errors related to varying environmental conditions that might be experienced in the field during
in-use  testing.  These tests were performed at a variety of facilities which  are  part of the
Mechanical and Material Engineering Division  at SwRI.  Environmental factors included in
these experiments included  temperature,  altitude, vibration,  and electromagnetic interference.
PEMS were  installed in  specialized test  facilities designed  to  simulate a  wide  variety of
conditions for each of these factors.  In  addition, testing  was also performed to examine the
effect of ambient hydrocarbon levels on the PEMS HC measurement.

     No engines were involved in the environmental tests.  Instead, standard reference gases
were sampled by the PEMS  during  these tests.  Therefore,  the errors were determined by
comparing PEMS analyzer responses to the known, and verified, concentrations of the reference
gases.  The exhaust  flow meter was included in some of these tests, but because no exhaust was
flowing through the meter during environmental testing, only zero errors were examined for
exhaust flow during these tests. Full details of environmental  testing and test results are given in
Section 5 of this report.

1.7    Exhaust Flow Meter Testing

     A small  set of experiments was specified in the Test Plan to evaluate the effect of various
installation and operation conditions  on  the exhaust flow meter.  These conditions included
exhaust flow pulsations, non-uniform velocity profiles (possibly caused by pipe bends location
upstream of the flow meter), and the effect of  wind across  the open end of the exhaust flow
meter.  These experiments were also conducted in the dynamometer test cell described in Section
1.5.  Special exhaust systems and test rigs were set up for each of these experiments.  The PEMS
exhaust flow meter measurements were compared to the Laboratory Reference raw exhaust flow
measurement during these experiments. Exhaust flow meter experiments and the results of those
tests are described in a portion of Section 4 of this report.
SwRI Report 03.12024.06                 Page 10 of 371

-------
1.8    Model Validation

     For reasons discussed earlier, the measurement allowances were generated using a Monte
Carlo computer model.  As with all simulations, it is vital that such a model be validated through
comparison with real experimental data.  In this case, the Measurement Allowance model needed
to be validated against a data set  generated through actual in-use field  testing.  Because the
model generates an incremental error in comparison to a Laboratory Reference, a suitable in-use
reference measurement was needed for comparison to the PEMS measurements.  The Steering
Committee determined  that  the  CE-CERT Mobile Emission Laboratory, operated by the
University of California-Riverside, would be  an appropriate reference  for validation of the
model-based in-field testing.

     In order to insure that the validation was not disturbed by some inherent bias between the
SwRI Reference Laboratory and the CE-CERT MEL validation reference, a correlation exercise
was performed between the two laboratories, prior to the start of on-road validation efforts. The
CE-CERT MEL was brought  to SwRI's laboratory facilities in San Antonio,  Texas, and a side-
by-side correlation  test  was  run.   During this  test, exhaust  from the same test engine was
alternately routed to the measurements systems  of both SwRI and CE-CERT.  This was done
repeatedly over the  course of  three days of testing.  The data was then supplied to the Steering
Committee, in  order to allow for a determination to  be  made that correlation between the
facilities was  acceptable for the purposes of validation of the model.

     After the correlation exercise was completed, a test truck was  supplied to CE-CERT by
Caterpillar for use in this validation exercise.  In addition, one of the audited PEMS used at
SwRI during the program was also delivered to CE-CERT.  CE-CERT then conducted a series of
on-road test runs over various driving routes in California, which were designed to take the test
truck through a wide range  of environmental  and ambient  conditions.  During these tests,
simultaneous  measurements were  made with the PEMS and the  MEL in order to generate a
validation data set. This formed the primary validation set for the model.

     Because the CE-CERT MEL does  not readily  incorporate a means of direct torque
measurement on a vehicle, the on-road validation data set could not be used to validate model
errors associated with broadcast torque and derived BSFC.  Therefore, an additional validation
exercise was  conducted at SwRI.  This involved removal of the engine from the test truck used
by CE-CERT, and  installation  of  that engine in the SwRI dynamometer test cell.   Selected
portions of the CE-CERT on-road tests were then simulated in  the  laboratory, to the extent
possible.  Simultaneous  laboratory and PEMS  measurements were again  taken during  this
"replay"  validation exercise.    However,  because the  laboratory incorporates  actual torque
measurement, it was possible  to use this "replay" data set to validate the  portions of the model
associated with torque and BSFC measurements.

     Validation of the  model  was assessed independently for each of the three pollutants
(NMHC, CO, and NOX), and for each of the three calculation methods. A full description of the
validation efforts, including the data analysis methodology and the results of validation for each
pollutant by all three calculation methods is given in Section 6, with  the exception of the CE-
CERT  on road  validation testing.   This  effort is described fully in a  separate report, titled
SwRI Report 03.12024.06                  Page 11 of 371

-------
Measurement Allowance On-Road Validation Project Report dated March 2007.  The contents of
that report are incorporated herein by reference.

1.9    Measurement Allowance Generation

    The generation of a set of measurement allowances represented the final outcome of this
program.  The Test Plan provided a methodology by which all of the data from the millions of
Model simulation runs would be collected and analyzed statistically, in order to generate a set of
three potential measurement allowances for each pollutant, one for each of the three calculation
methods.  The Test Plan then outlined a specific method by which the final  set of allowances
would be chosen from among deltas generated for each of the three calculation methods.  The
assumption made by the Test Plan, was that the final outcome of all previous efforts would be a
set of three validated potential measurement allowance values for each pollutant, NMHC, CO,
and NOX.   Each potential  allowance was expressed  as  a percentage of  its associated  NTE
threshold.

    The NTE  thresholds used for this program are given in Table 13.  These NTE thresholds
were determined by EPA and approved by the Steering Committee during the generation of the
Test Plan.  The Test Plan values were  supplied in g/hp-hr as shown and calculated values in
g/kW-hr are also given for reference.
  TABLE 13.  NTE THRESHOLDS FOR MEASUREMENT ALLOWANCE PROGRAM
Pollutant
NMHC
CO
NOX
NTE Threshold
g/hp-hr
0.21
19.4
2.0
g/kW-hr
0.2816
26.02
2.682
    These threshold values are of critical importance to the program, as they provide the basis
for the scaling of measurement allowances, the assessment of model convergence, and a variety
of other calculations performed during this program.  The general philosophy of the Test Plan
was to determine measurement allowances based on errors at these emission levels, especially in
the case of any errors that scaled with emission level.

    The  anticipated outcome  from the  model runs,  analysis,  and validation efforts  can be
represented as a table  similar to the one  shown in  Table 14,  which is repeated herein from the
Test Plan.  The table illustrates both the model  outcome, and the process for selecting the final
measurement allowance values.
SwRI Report 03.12024.06
Page 12 of 371

-------
   TABLE 14. EXAMPLE OF MEASUREMENT ALLOWANCE DETERMINATION
                                 FROM TEST PLAN

Calc. Method ==>
BSNOx
BSNMHC
BSCO
Measurement Errors at respective NTE threshold (%)
Method 1
Torque-Speed
18%
19%
3%
Method 2
BSFC
18%
17%
2%
Method 3
ECM fuel specific
20%
14%
1 %

max error ==>
min of max ==>
selected method==>
19%

18%
18%
20%

"BSFC" method
    The intent of the final selection process was to first determine the largest percentage error
from among the three pollutants for each calculation method. These three largest errors would
then be compared with each other, and the method which produced the smallest of these three
would  be chosen for calculation of the  final measurement allowances.   At that  point,  the
percentages  given for the chosen calculation method would be applied to the NTE threshold
values  given in Table 13,  in order to generate the final  additive, brake-specific measurement
allowances for each pollutant.

    An implicit assumption  of the process, as described  in the Test Plan, was that the values
produced by the model for all three pollutants and all  three calculation methods would be
successfully validated. In the event that this did not occur, it would be necessary for the Steering
Committee to  determine a valid alternate course of action, in order  to determine the final
measurement allowance values.

    The  final  model run  and the selection and generation of measurement allowances  are
described fully in  Section 7 of this report, including the final allowances approved by  the
Steering Committee.
SwRI Report 03.12024.06
Page 13 of 371

-------
                           2.0    MONTE CARLO MODEL
2.1    Model Background
     The main objective of this portion of the project was to use Monte Carlo techniques (e.g.
random sampling) in an  error model to simulate the combined effects of all the agreed-upon
sources of PEMS error incremental to lab error on the components of the brake-specific (BS)
emissions.  This was accomplished by creating "error surfaces" for the Monte Carlo simulation
to sample, based upon the results of a variety of lab experiments.   The constructed model was
simulated for thousands of trials (i.e., iterations) using data taken from a reference data set of 195
unique NTE events. The model  results were used to  determine the  brake-specific  additive
measurement allowances for NOX, NMHC, and CO by three different calculation methods.

     The error surfaces were generated from the results of each of the engine dynamometer and
environmental chamber laboratory tests described in Sections 4 and 5, respectively. The engine-
lab-test error surfaces covered the domain of error versus the magnitude of the signal to which
the error was to be applied (i.e., 5th to 95th percentile error vs. concentration, flow, torque,  etc.).
The  environmental-test error  surfaces for shock  and vibration, and electromagnetic and radio
frequency interference  (EMI/RFI) covered the same domain as the engine tests, but  only for
concentration.   The environmental  test for ambient hydrocarbons was similar, but  the  error
surface did not change as a function of concentration. The environmental test error surfaces for
pressure  and temperature were characteristically  different because they covered the domain of
the environmental-test cycle time versus the magnitude of the signal to which the  error was to be
applied (i.e., error at a selected time vs. concentration).  Details  on  how  each surface was
generated are given in Sections 4 and 5  of this report.  Since these surfaces are  populated with
data  representing  the  incremental errors  between  PEMS  measurements and laboratory
measurements, they were sampled directly by the model.

2.1.1  Reference NTE Events

       The reference data set to which all the simulated  errors were applied represented engine
operations over a  wide range of NTE  events.  This reference data set was generated  from
collections of real-world  PEMS data sets. Parameters in the reference data set  were  scaled in
order to exercise the model  through  a more appropriate range of parameters (i.e.  concentrations,
flows,  ambient conditions, etc.). In this scaling process, care was taken to maintain the dynamic
characteristics of the reference data set.

       The  Monte Carlo simulation model was run  on a set of 195 reference NTE events
collected from a number of sources.  Five engine manufacturers provided a total of 97 events; 10
reference NTE events came from each of the three engines tested in the lab during the transient
testing; 54 reference NTE events were created by adjusting the engine transient tests to cover a
larger spread of the emissions; and  14 events came from the pre-pilot CE-CERT data. Before
and  after errors were applied in the Monte Carlo simulation for each of these  reference  NTE
events.
SwRI Report 03.12024.06                 Page 14 of 371

-------
       For all the reference data that was supplied by engine manufacturers, and the CE-CERT
data, it is understood that the NTE event data was based on actual field testing results.  This was
done to insure that the NTE reference events would be representative of in-field operation in the
NTE zone. The data from the engine manufacturers was supplied directly to EPA individually.
The  data  was  reviewed at EPA and then transmitted to  SwRI.  No additional information
regarding  procedures used to generate this data was supplied to SwRI along with the reference
NTE events.

       NTE brake-specific emissions results were calculated for NOX,  CO and  NMHC, using
each of the  three agreed-upon NTE calculation methods.   The three different BS  emissions
calculation methods referred to in this test plan are:

          1.  Method #1:  Torque-Speed Method
          2.  Method #2:  BSFC Method
          3.  Method #3:  Fuel Specific Method

The formulas and input constants for these three methods for each of the three emissions types
are provided in Appendix B.

       Table 15 lists the number of NTE events obtained from each data source and the three
corresponding BS emissions calculated using Method 1.  These emissions have been computed
with no error values  added to the input parameters. For this  report, emissions  with no errors
added will be labeled the "ideal" emissions.  In contrast, the emissions with errors added through
the Monte Carlo simulation will be labeled emissions "with errors".
      TABLE 15. REFERENCE NTE EVENTS AND METHOD 1 BS EMISSIONS
Source
International
DDC
Caterpillar
Cummins
Volvo
Engine #1
Engine #2
Engine #3
Pre-Pilot
Number
of NTE
Events
19
18
20
20
20
28
28
28
14
BSNOx
g/kW-hr
Min
1.858
3.148
0.025
2.667
1.396
0.844
1.815
1.586
5.328
Max
5.446
6.012
5.865
6.687
2.457
5.799
3.397
3.467
7.193
BSCO
g/kW-hr
Min
0.520
0.221
0.000
5.995
1.159
0.145
0.150
0.261
0.110
Max
1.3563
1.888
1.361
0.232
0.266
0.496
0.511
0.530
0.341
BSNMHC
g/kW-hr
Min
0.073
0.002
0.000
0.006
0.004
0.000
0.000
0.000
0.000
Max
0.276
0.087
0.059
0.426
0.014
0.000
0.004
0.004
0.000
SwRI Report 03.12024.06
Page 15 of 371

-------
       When the ideal brake-specific emission values were calculated for the various reference
NTE events, it was noted that these ideal emission values were frequently different from one
calculation method to another.  While it was recognized that this was a realistic outcome, the
Steering Committee was concerned that these discrepancies might introduce an unintended bias
into the results of the Monte Carlo simulation. Therefore, the Steering Committee directed SwRI
to adjust the NTE reference event data in order to align the brake-specific emission levels from
all the calculation methods. In general, the values for Methods 2 and 3 tended to be very close to
each other, while the Method 1 value would be farther from the other two.

       The adjustment was performed by first assuming that the Method 1  result was the desired
value, and that the other two calculation methods would be aligned to that result.  This  meant
that torque, speed,  and exhaust flow values were not changed.  The next  step of the alignment
process was to adjust CC>2 values for the NTE event, in order to line up the Method 2 NOX result
with the Method 1  value.  This was done by using a single multiplier on all CC>2 values for the
NTE event in question.   Finally the fuel rate values were adjusted  slightly in order to bring
Method 3  in line with Method 2.  This second adjustment was generally on the order  of 2 percent
or less, because Methods 2 and 3 were normally fairly close to each other.

       The alignment was performed in order to  get the NOX emission  levels from all  three
methods to line up precisely.  It was initially assumed that CO and NMHC would also line up,
once the NOX values were aligned.  In general, that is what happened; however, selected events
still  demonstrated  a misalignment  of CO  or NMHC once NOX was  aligned.  The Steering
Committee ultimately elected to accept small  discrepancies in  CO  and  NMHC between the
calculation methods as long as the magnitude of the differences were less than 1%  of the NTE
threshold for CO and 2% of the NTE threshold for NMHC.  Events which demonstrated larger
misalignment were removed from the reference data set.  This review resulted in the removal of
four of the original  events from the reference data set.

       The distribution  of the  BS  emissions data for  the 195 reference NTE events  to be
simulated  in the Monte Carlo model are depicted in Figure 3 through Figure 5 for NOX, CO and
NMHC, respectively.  Note that each emission has data values  spread above and below the
corresponding NTE threshold. The NTE thresholds used in this analysis were:

   •   BSNOX       2.0g/hp-hr    or     2.68204 g/kW-hr
   •   BSNMHC    0.21 g/hp-hr   or     0.28161 g/kW-hr
   •   BSCO        19.4 g/hp-hr   or     26.0150 g/kW-hr
SwRI Report 03.12024.06                  Page 16 of 371

-------
                Method 1  Ideal NOx (g/kW-hr) for 195 Ref NTE Events

              50

              40

              30

              20

              10

               0
                 0
2468
     NOx(g/kW-hr)
     FIGURE 3. METHOD 1 BSNOX VALUES FOR REFERENCE NTE EVENTS
                Method 1  Ideal CO (g/kW-hr) for 195 Ref MTE Events
            150 P
                           246
                               CO (g/kW-hr)
                             8
     FIGURE 4. METHOD 1 BSCO VALUES FOR REFERENCE NTE EVENTS
SwRI Report 03.12024.06
  Page 17 of 371

-------
                Method 1  Ideal NMHC (g/kW-hr) for 195 Ref NTE Events
OJ
Z!

QJ
              100 P

               80

               60

               40

               20

                0
                   0
                 0.1      0.2       0.3
                         NMHC (g/kW-hr)
0.4
0.5
    FIGURE 5. METHOD 1 BSNMHC VALUES FOR REFERENCE NTE EVENTS

      Table 16 provides a summary of some descriptive statistics for the reference NTE data set
for each of the three BS emissions.

  TABLE 16. DESCRIPTIVE STATISTICS FOR BS EMISSIONS FOR REFERENCE
                                 NTE EVENTS
Descriptive
Statistic
Minimum
Maximum
Mean
Median
Standard Deviation
BSNOX
g/kW-hr
0.0249
7.1927
3.0071
2.6033
1.4807
BSCO
g/kW-hr
0.0000
5.9949
0.5936
0.3836
0.7129
BSNMHC
g/kW-hr
0.0000
0.4258
0.0287
0.0021
0.0591
      The parameter data provided in each reference NTE event was on a second-by-second
basis with a minimum of 30 seconds and a maximum of 300 seconds.  The input parameters
required for the BS emissions calculation methods and the Monte Carlo simulation are listed in
Table 17. An Excel file with a specific input format structure was used to  standardize the format
of the input files.  Since the total hydrocarbons  (THC) was selected as an input parameter,
NMHC was computed as THC*0.98.
SwRI Report 03.12024.06
                       Page 18 of 371

-------
         TABLE 17. INPUT PARAMETERS FOR REFERENCE NTE EVENTS
Variable
Number
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

22
Input Variable
NTE Event Number
NTE Source
Engine Make
Engine Model
Engine Displacement
Date
Time Stamp
Wet CO2
Wet CO
Wet kNO
Wet kNO2
Wet THC
Exhaust Flow Rate
Flowmeter Diameter
Speed
Low Speed, nlo
High Speed, nhi
Fuel Rate
Max Fuel Rate
Derived Torque
Peak Torque

BSFC
Units
integer
alphanumeric
alphanumeric
alphanumeric
L
mm/dd/yyyy
hh:mm:ss.s
%
%
ppm
ppm
ppm
scfm
3, 4, or 5 (inches)
rpm
rpm
rpm
L/sec
L/sec
N-m
N-m

g/kW-hr
Description
All reference NTE events must be identified by an NTE number (e.g.,
001).
The source of the NTE event is the company, organization and/or lab
that created the event data.
Engine Make
Engine Model
Engine Displacement (L)
The day the NTE event data was created (mm/dd/yyyy).
Time in seconds. Each reference NTE must contain second-by-
second data only.
CO2 (%)
CO (%)
NO (ppm) with intake air-humidity correction
NO2 (ppm) with intake air-humidity correction
THC (ppm)
Exhaust flow rate (scfm)
To compute the % of PEMS flowmeter maximum flowrate we will need
to know what size flowmeter was used for each NTE event.
Enter either 3, 4, or 5 to represent the following flowmeters and
maximum flow rates:
3 = 3 inch EFM with maximum flow rate = 600 scfm
4 = 4 inch EFM with maximum flow rate = 1 100 scfm
5 = 5 inch EFM with maximum flow rate = 1700 scfm
Engine speed (rpm)
To compute the % of normalized speed we will need nlo and nhi for the
engine computed as follows:
nlo (rpm) = lowest speed below max power at which 50% max power
occurs
nhi (rpm) = highest speed above max power at which 70% max power
occurs
Fuel rate (L/hr))
To compute the % of maximum fuel rate we will need the max fuel rate
of the engine for each NTE event.
Max fuel rate (L/hr)
Torque (N-m)
To compute the % of maximum torque we will need the peak torque of
the engine for each NTE event
Peak torque (N-m)
BSFC (g/kW-hr), enter this based upon interpolating your own BSFC
table or use the calculation in this spreadsheet, which uses fuel rate,
torque, and speed to calculate BSFC, & spgr=0.85, use appropriate
conversion factors and spgr.
2.1.2   Error Surfaces

       During the initial review of the Test Plan and from discussions held at several  Steering
Committee meetings, 52 error surfaces were initially identified and considered for inclusion in
the Monte Carlo simulation model.  These individual error surfaces encompassed a wide variety
of error sources, and each of them was investigated in a specific experiment, as detailed later. In
some  cases, upon reviewing the experimental  data, the  Steering Committee deemed  that the
errors from certain sources were not significant; therefore, inclusion in the final Model was not
warranted.  The details regarding which errors were not  included in  the model are given later
SwRI Report 03.12024.06
Page 19 of 371

-------
under the description of the individual error experiments in Sections 4 and 5 of the report.  A
final total of 35 error surfaces were incorporated into the Model.  Two additional errors terms
were also included for time alignment as detailed later, bringing the total number of error terms
incorporated in the model to 37.

       Table 18 lists the error surfaces examined during the study with the surfaces excluded by
the Steering Committee designated in italics.  All  remaining ones were implemented in the
simulation model.   Each error surface was  assigned a  number for easy  identification.
Additionally, two error surfaces relating to the time alignment adjustment for NOX and CO  (i.e.,
see Section on Time Alignment for NOX and CO} were also included.
SwRI Report 03.12024.06                 Page 20 of 371

-------
        TABLE 18. ERROR SURFACES FOR MONTE CARLO SIMULATION
Measurement Error Surfaces and Deltas Used in BS Emissions Calculations
Component
1 . Delta NOx

2. Delta CO

3. Delta NMHC
NMHC = 0.98THC

4. Delta Exhaust Flow

5. Delta Torque

6. Delta BSFC

7. Delta Speed
8. Delta Fuel Rate
9. Delta CO2

#
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
Test Source
Engine Dyno
Engine Dyno
Environ
Environ
Environ
Environ
Engine Dyno
Engine Dyno
Environ
Environ
Environ
Environ
Engine Dyno
Engine Dyno
Environ
Environ
Environ
Environ
Environ
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Environ
Environ
Environ
Environ
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Engine Manuf
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Engine Manuf
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Environ
Environ
Environ
Environ
Error Surface
Delta NOx SS
Delta NOx Transient
Delta NOx EMI/RFI
Delta NOx Atmospheric Pressure
Delta NOx Ambient Temperature
Delta NOx Vibration
Delta CO SS
Delta CO Transient
Delta CO EMI/RFI
Delta CO Atmospheric Pressure
Delta CO Ambient Temperature
Delta CO Vibration
Delta NMHC SS
Delta NMHC Transient
Delta NMHC EMI/RFI
Delta NMHC Atmospheric Pressure
Delta NMHC Ambient Temperature
Delta NMHC Vibration
Delta Ambient NMHC
Delta Exhaust Flow SS
Delta Exhaust Flow Transient
Delta Exhaust Flow Pulsation
Delta Exhaust Flow Swirl
Delta Exhaust Flow Wind
Delta Exhaust EMI/RFI
Delta Exhaust Vibration
Delta Exhaust Temperature
Delta Exhaust Pressure
Delta Dynamic Torque
Delta Torque DOE Testing
Delta Torque Warm-up
Delta Torque Humidity/Fuel
Delta Torque Fuel
Delta Torque Interpolation
Delta Torque Engine Manufacturers
Delta Dynamic BSFC
Delta BSFC DOE Testing
Delta BSFC Warm-up
Delta BSFC Humidity/Fuel
Delta BSFC Fuel
Delta BSFC Interpolation
Delta BSFC Engine Manufacturers
Delta Dynamic Speed |
Delta Dynamic Fuel Rate
Delta CO2 SS
Delta CO2 Transient
Delta CO2 EMI/RFI
Delta CO2 Atmospheric Pressure
Delta CO2 Ambient Temperature
Delta CO2 Vibration
Committee Action


Deleted by Steering Committee
Deleted by Steering Committee

Deleted by Steering Committee

Deleted by Steering Committee
Deleted by Steering Committee


Deleted by Steering Committee


Deleted by Steering Committee


Deleted by Steering Committee





Deleted by Steering Committee

Deleted by Steering Committee






Combined with #32






Combined with #39






Deleted by Steering Committee
Deleted by Steering Committee

Deleted by Steering Committee
       For each of the measurement errors defined in Sections 4 and 5, an error surface was
created and used in the Monte Carlo simulation.  Each error surface represented an additive
error—or a subtractive error if the sign was negative—relative to the reference parameter value
to which it was applied.  Figure 6 through Figure 8 serve as a hypothetical example of how these
error surfaces  were created for every measurement error.  Details on the construction of each
SwRI Report 03.12024.06
Page 21 of 371

-------
error surface used in the simulation are provided in Sections 4 and 5. The example illustrated in
Figure 6 through Figure 8 represent the error  surface for steady-state bias and precision NOX
concentration  errors (Section on Steady-State Concentration error  Surface Generation).   The
plots shown correspond to hypothetical NOX  emissions concentration  data  acquired  in  the
laboratory with three PEMS and three engines, with all nine sets of PEMS data pooled together.

PEMS vs. Laboratory Nominal Results

       Figure 6 was constructed from raw data acquired from steady-state engine lab tests with
the PEMS at repeat testing at various concentration levels (NOX ppm).  The plot pools all bias
and precision errors for all  three PEMS and for all data from all three engines for all steady-state
modes. Twenty repeat measurements of NOX signals were taken for each of three PEMS yielding
60  data points at each value of the corresponding average lab NOX values (i.e., lab nominal
value).   The  60 PEMS  signals  were plotted against the corresponding  laboratory signals
measured using lab  equipment.   Shown in Figure  6 are the 5th, 50th,  and  95th  percentiles
corresponding to the distribution of these 60 observations using the PEMS at each average NOX
concentration  level (note that the distribution of data at each NOX level  may not represent a
normal distribution).  Since the  50th percentiles do not lie on the dashed (diagonal) line of perfect
agreement, the data suggest that there is a bias error between the PEMS and lab results.   In
essence this graph summarizes the  statistical  distribution measured by the  PEMS at each
concentration  level  sampled.  The example plot in Figure 6 shows only 6 discrete average NOX
concentration  levels (ranging from  100-350 ppm).   However, the actual number of discrete
concentration  levels was determined using the total number of operating conditions actually  run
for all the tests on all three engines. In the section on Steady-State Repeat Engine Testing and
Error Surfaces it is reported that 10 operating conditions from an initial number of 40 operating
conditions were selected for construction of the steady-state NOX error surface. Thus, the plot
used in the Monte Carlo simulation contained 30 discrete NOX concentration levels (10 operating
conditions x 3 engines).
SwRI Report 03.12024.06                  Page 22 of 371

-------
                                    NOx Concentration Errors
                                                               394


§" ,50
LU
Q_

0_
Q.
Z
100
en

n

285
242/^^
/rt^se-^256
W/j?s'
I3U/ /• •
^f /^-214
155^^-^T^T--
i i^sX^55
..•••*'
..-•'' 85


./t' 380 -
341 /^ >i .-•'
xf .X-S62
/?£&
/^307
fir



• 5th percent! le
50th percent! le
(median)
— * — 95th percent! le













                        50     100     150    200     250
                                    NOx PPM (Lab, nominal)
                                                        300
                                                               350
                                                                     400
    FIGURE 6.  ERROR SURFACE CONSTRUCTION: PEMS VS. LABORATORY
                                       RESULTS

(PEMS - Laboratory) Deltas vs. Lab

       Figure 7 illustrates the "error band" measured during testing.  This plot was created by
first subtracting the individual "lab nominal" NOX value from  the  corresponding individual
PEMS NOX measurement for each test run.  This difference was defined as the "delta" error.
Second, these "PEMS - Laboratory" delta errors were pooled at each  average lab nominal NOX
value to obtain the 5th, 50th, and 95th percentile values displayed in Figure 7.  Therefore, the plot
represents the average NOX lab nominal at 30 discrete concentration levels versus the percentiles
of the delta errors computed from the PEMS and laboratory individual test results.
SwRI Report 03.12024.06
Page 23 of 371

-------
                                     NOx Concentration Errors
                60
            Q.
            Q.
                50-

                40-
            9   30-
                               55
-95th percentile

 50th percentile
 (median)
-5th percentile
                                                                     400
                                    NOx PPM (Lab, nominal)

FIGURE 7. ERROR SURFACE CONSTRUCTION:  (PEMS - LAB) VS. LABORATORY
                                        RESULTS

Variability Index vs. (PEMS - Laboratory) Deltas and Lab Nominal

       This step normalized the plot in Figure 7 using what is called a "variability index (ic)".
This index represented the value randomly drawn by the Monte Carlo simulation in order to
select a given error level.  It was allowed to vary from -1 to +1. The likelihood of "ic" being any
value between -1 through +1 was specified by a "probability density function (PDF)" assigned to
ic. In the case of this example, ic. was assumed to vary according to a standard normal (i.e., bell-
shaped) distribution during the Monte Carlo simulations.  This was because it was believed that
the distribution of NOX errors due to steady-state bias and precision would be centered about the
50th percentile of the full range of conditions measured according to the section on Steady-State
Repeat Engine Testing and Error Surfaces.  Each set of data for each lab "set point" average
(i.e., lab nominal value) in Figure 7 was normalized by aligning the corresponding 5th percentile
error from Figure 7 with ic = -1,  the 50th percentile error with ic = 0, and the 95th percentile error
with ic = +1.  These values were then plotted in Figure 8, where the y-axis is the variability index,
the x-axis is the average lab nominal NOX value, and the z-axis is the delta NOX value.  Notice
that, when using this normalization approach, the 5th, 50th, and 95th percentile values  remain
equivalent between Figure 7 and Figure 8. Error surfaces such as the one presented in Figure 6
are the error deltas  the Monte  Carlo simulation program used during calculation of the BS
emissions "with errors".
SwRI Report 03.12024.06
                 Page 24 of 371

-------
                                 NOx Concentration Errors
                            Error Surface: z-axis = DELTA(NOx PPM)
            1.5
             1 -
•§ 0.5 H
£
I1
15
n   0
           -0.5-
         Q.
         Q.
             -1 •
           -1.5
                           55
                                  40
                                        42
                                               35
                                                     41
                                                            44
                   •95th percent! le
                   50th percentile (median)
                   •5th percent! le
                           25
                                  20
                                        30
                                               20
                                                     25
                                                            30
                     50     100    150    200     250    300     350    400
                           -15     +5     +14     +6     +7     +12
                                 NOx PPM (Lab, nominal)
FIGURE 8. ERROR SURFACE CONSTRUCTION: ERROR AT VARIABILITY INDEX
                             VS. LABORATORY RESULTS
2.1.3  Error Surface Sampling and Interpolation

      The error model  used two  different probability density  functions to sample the error
surfaces, depending upon which  experimental parameter the surface represented.   To sample
error surfaces that were generated from the lab test results (Section on Engine Dynamometer
Laboratory Testing), and the environmental test results for shock and vibration, EMI/RFI, and
ambient hydrocarbons, the model used a truncated standard normal PDF because these tests were
designed to evenly  cover the full,  but finite, range of engine operation and ambient  conditions.
To sample error surfaces that were generated from the pressure  and temperature environmental
test results (Section on Environmental Chamber Testing), the model used a uniform PDF because
these tests were already designed to cover the typical range and frequency of the respective
conditions. Both of these sampling distributions are depicted in Figure 9.
SwRI Report 03.12024.06
                            Page 25 of 371

-------
,
Probability Density Functions for Sampling Error Surfaces Once Per NTE Event
	 Lab Tests, Normal, SD=0.60795, truncate @ -1 & 1
^—Environmental Tests, Random

Note: A non-trur
distribution with
values of 0.05 ai
ic=+1, respective





icated normal
SD=0.60795 has
id 0.95 at ic=-1 ar
;ly.
^
c
^
^^


p
d ^^^
^*^


^^
^^

^
*^^




^^^
^^^
> 4 3 2
Relative Probability








(
1.00
0.75
0.50
0.25
0.00 .-"
-0.25
-0.50
-0.75
-1.00
  FIGURE 9. TRUNCATED STANDARD NORMAL AND UNIFORM PROBABILITY
                                DENSITY FUNCTIONS

       When using the truncated standard normal PDF (see Figure 9), the Monte Carlo model
sampled normal deviates that ranged between -1 and +1.  These were  used as the ic  values
defined in the section on Error Surfaces. Similarly, the pressure and temperature environmental
tests used a uniform PDF to sample test time,  from which calculated errors were used.  All
temperature error surfaces related to the four emissions were sampled uniformly from 1 to 1080
minutes while the error surfaces related to the pressure were sampled uniformly from 1  to 720
minutes.   Exhaust flow error surface for temperature was sampled uniformly from 1  to 478
minutes while the exhaust flow for pressure was sampled uniformly from 1 to 360 minutes.  The
errors from all the other  tests were aligned with the truncated standard normal PDF such that
each of the 50th percentile error values at each of the tested signal magnitudes was centered at the
median (i.e., 0 value) of the PDF, and the 5th and 95th percentile error values at each of the tested
signal magnitudes were aligned with the extreme negative (ic = -1) and positive (ic = +1) edges of
the PDF, respectively.

       Each error surface was sampled along its ic axis (y-axis) once per trial for a reference
NTE event simulation.  Hence, every error surface had a separate randomly selected ic for each
trial.  Since each reference NTE  event contained second-by-second parameter data, the error
surface was sampled at a given ic on the y-axis and at the several selected parameter values on
the x-axis that corresponded to each second of the reference NTE event.  The sampled error
value was determined for the given second and parameter along the error axis (z-axis) at the
intersection of the ic value and the parameter value from the reference NTE event.  This was
accomplished by taking each second in the reference NTE event and finding the two adjacent x-
axis  values from the error surface  between which to  linearly interpolate to obtain the error
surface x-value. Each second in the reference NTE event was linearly interpolated with the same
ic value for a particular trial at the error  surface x-value.  If any  of the sampled lab nominal
SwRI Report 03.12024.06
Page 26 of 371

-------
values (NOX, NMHC, CO, Speed, Fuel  Rate, etc.) exceeded the upper or lower limits of the
parameter error surface, the value of the closest endpoint of the error surface was assigned to
them.

       Figure 10 depicts an example of the error surface sampling using a steady-state NOX error
surface containing 30 lab  nominal NOX x-axis values.  For this particular trial, the randomly
selected ic is -0.5. The example reference NTE event is noted by the symbol '*' and it plotted at
ic = -0.5 for each second in the NTE event.
                          SS Error Surface for NOx Concentration
                             Error Surface: z-axis = ASS_NOx_ppm
                                     A A-44-At - A-	A A-	-A
                        100      200       300      400      500       600
           'I   o
            -0.5
                                       NOx ppm (lab,nom)
                -A- - - 95th percentile
                -•- - - 5th percentile
                    NTE Event
                  50th percentile (median)
                  ic = -0.5
FIGURE 10.  STEADY-STATE NOX ERROR SURFACE WITH EXAMPLE SAMPLING
                          FOR A REFERENCE NTE EVENT
2.1.4  Brake-Specific Emissions Calculations

       Errors from Sections 4 and 5 were combined by adding all of the sampled errors once per
trial  for each reference NTE event simulation.  For example, in order to assess the errors in NOX
concentration by calculation Method #1, several error surfaces were  sampled and added to the
corresponding parameter in the Method #1  calculation and the resulting BSNOX "with errors"
SwRI Report 03.12024.06
Page 27 of 371

-------
was computed. The errors used in this calculation are the following (note that the corresponding
error surface numbers are provided in the subscripts):
       NOX ppm 'with errors'  =


       Exhaust Flow % 'with errors'
       Torque % 'with errors'
      Jx ppm_reference  ~"~ ^ J^'-'x ppm_l ~"~
    A NOX ppm_2 + A NOX ppm_5

    ExhaUSt Flow o/0 reference +
    A Exhaust Flow % 20 A Exhaust Flow% 21 +
    A Exhaust Flow % 22 + A Exhaust Flow % 23
    A Exhaust Flow % 25 + A Exhaust Flow % 27
    A Exhaust Flow % 28

    Torque o/oreference +
    A Torque % 29 + A Torque % 30"
    * ^        i + A Torque % 32 -
                   A rr-i
                                         A Torque % 31 + A Torque % 32 -
                                         A Torque % 34 + A Torque % 35

       Speed % 'with errors' =             Speed % reference + A Speed % 43
where,

       A 1,2   =      NOX concentration errors due to steady-state and transient errors,
       A 5     =      NOX concentration errors due to ambient temperature,
       A 20,21  =      exhaust flow errors due to steady-state and transient errors,
       A 22,23  =      exhaust flow errors due to pulsation and swirl,
       A 25    =      exhaust flow errors due to ambient temperature,
       A 27,28  =      exhaust flow errors due to temperature and pressure,
       A 29    =      torque errors due to dynamic torque,
       A 30,31  =      torque errors due to DOE and warm-up,
       A 32    =      torque errors due to interacting parameters humidity and fuel,
       A 3435  =      torque errors due to interpolation and engine manufacturers,
       A 43    =      speed errors due to dynamic speed

       Using the formulas for the calculation methods in Appendix B, the BSNOX for Method #1
was  computed without errors ("ideal")  and then  with all the errors applied as outlined above.
Table 19 lists all error surfaces used by each calculation method for all three emissions.
SwRI Report 03.12024.06
Page 28 of 371

-------
    TABLE 19. ERROR SURFACES USED FOR COMPUTING BRAKE-SPECIFIC
                EMISSIONS BY THREE CALCULATION METHODS

Component
1. Delta NOc
2DeltaCO
3. Delta NVHC
NVHC=0.93THC
4 Delta Exhaust Rcw
5. Delta Torque
6. Delta BSFC
7. Delta Speed
8. Delta Fuel Rate
9. Delta C02

#
1
2
5
7
10
11
13
14
16
17
19
20
21
22
23
25
27
23
29
30
31
32
34
35
36
37
38
39
41
42
43
44
45
46
49

EirorSuface
Delta NOxSS
Delta NO Transient
Delta NO< Ardent Temperature
Delta 00 SS
Delta CO Arrospheric Pressure
Delta 00 Ardent Terrperatue
Delta NVHC SS
Delta NVHC Transient
Delta NVHC Amospheric Pressure
Delta NVHC Artient Temperature
Delta Ardent NVHC
Delta Exhaust FlowSS
Delta Exhaust Rcw Transient
Delta Exhaust Rcw Pulsation
Delta Exhaust RcwSwrl
Delta Exhaust EM/RR
Delta Exhaust Terrperature
Delta Exhaust Pressure
Delta Dynarric Torque
Delta Torque DOE Testing
Delta Torque V\6rmup
Delta Torque Huridty/Fuel
Delta Torque Interpolation
Delta Torque Engine IVanuf
Delta Dynarric BSFC
Delta BSFC DOE Testing
Delta BSFC Wsrm-up
Delta BSFC Hrridty/Fuel
Delta BSFC Interpolation
Delta BSFC ErgneManuf
Delta Dynarric Speed
Delta Dynarric Fuel Rate
Delta 002 SS
Delta 002 Transient
Delta 002 Ardent Temperature
Method Isolation
BSNOx
•/
•/
•/








•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/






•/




BSCO



•/
•/
•/





•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/






•/




BSMVHC






•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/






•/




Method 2 (Violation
BSNOK
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/






•/
•/
•/
•/
•/
•/


•/
•/
•/
BSCO



•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/






•/
•/
•/
•/
•/
•/


•/
•/
•/
BSNIVHC



•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/






•/
•/
•/
•/
•/
•/


•/
•/
•/
Method 3 Cdolafion
BSNOx
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/
•/







•/
•/
•/
•/
•/
•/






•/
•/
•/
•/
•/
BSCO



•/
•/
•/
•/
•/
•/
•/
•/







•/
•/
•/
•/
•/
•/






•/
•/
•/
•/
•/
BSNIVHC



•/
•/
•/
•/
•/
•/
•/
•/







•/
•/
•/
•/
•/
•/






•/
•/
•/
•/
•/
2.1.5   Periodic Drift Check

       During the Monte Carlo simulation  for  a particular  reference NTE  event, the BS
emissions computed during each simulation trial  (with ic  selected randomly) was checked to
determine whether or not a periodic drift would have invalidated the NTE event trial.  The drift
check results were simulated by computing the BS emissions with all the error surface errors
added  except those due to the environmental error surfaces.  Therefore, the following error
surfaces were excluded in computing the drift check: temperature error surfaces for NOX, CO,
CO2, and NMHC; pressure error  surfaces for CO and NMHC; and ambient  NMHC.  If the
absolute difference in the BS emissions 'with all errors' and the BS emissions  'with all errors
except environmental' was greater than a percentage of the emissions threshold, then periodic
drift was detected and the simulation trial was eliminated from the analysis. The percentages
used in this study were 4% of the NOX and CO threshold (0.080 and 0.776 g/hp-hr, respectively)
and 10%  of the NMHC threshold (0.021 g/hp-hr).  Figure 11 represents the periodic drift
process.
SwRI Report 03.12024.06
Page 29 of 371

-------
                        Generate ic for each error surface.
                              Interpolate A errors.
                         Compute Emission "with errors"
        Retain
     Emission "with
        errors"
               Compute Emission with errors
                   except environmental"
                                ABS(Emission
                                "with errors" -
                                Emission "with
                                 errors except
                                environmental)
      Delete Emission "with
      errors" from current i
< 4% (10%) of
  Emission
  Threshold?

  (No Drift)
> 4% (10%) of
   Emission
  Threshold?

(Drift Detected)
                FIGURE 11.  PERIODIC DRIFT CHECK FLOWCHART
2.1.6  Time Alignment for NOX and CO

       The time alignment adjustment measured the effect of errors in time alignment of the
various continuous PEMS data sources on the final BS emission results. This error source was
not originally included in the Test Plan, and no experiment had been  designed to examine it.
However, it was later decided that time alignment was a significant source of potential error, and
that it should be incorporated into the Model.  Time alignment values were only generated for
NOX and CO, because NMHC values were too low and too stable to see any discernible trends in
NMHC due to time alignment.  Details regarding the methodology used to determine the time
alignment adjustment are given later in Section 4.12.

       Although time alignment was not applied in the same fashion as the other error surfaces
in this model,  it was described as an error  surface because it was sampled as a  normal
distribution  similar  to  the  other  error surfaces. The  time  alignment  adjustment  was  a
multiplicative factor which was applied to the BS emission result after all other error terms had
been added. The time alignment represented an adjustment up or down as a percentage of the BS
emission level "with errors".  A separate time alignment factor was developed for each pollutant,
and for each of the three calculation methods allowed in the HDIUT program. Thus, during the
Monte Carlo simulation for each trial the brake-specific differences were computed as follows:

      (BS emissions 'with errors' * Time Alignment Adjustment) - "Ideal" BS emissions
SwRI Report 03.12024.06
                           Page 30 of 371

-------
2.1.7  Convergence and Number of Trials

       Since the Test Plan did not  include a provision for convergence criteria, the  Steering
Committee was tasked to develop a convergence method.  The main goal was to define how
many simulation trials at a given reference  NTE event were required to estimate  the  95th
percentile BS emission differences with a given precision.  Although the Crystal Ball software
contained precision control  options, the  method used to  compute  a confidence interval  on
percentiles was  based on  an  analytical bootstrapping method which was  not adequately
documented.  Thus, an independent convergence method was proposed and accepted by the
Steering Committee.

       A  nonparametric  statistical technique [Reference:  Practical  Nonparametric Statistics,
WJ. Conover, John Wiley  &  Sons,  1971] was proposed which defined a 90% confidence
interval for the 95th percentile of the BS emissions differences for an individual reference NTE
simulation.  If the width of the 90% confidence interval was less than 1% of the BS emissions
threshold, then convergence was met.  The following steps define the convergence method:

    1.  Run the Monte Carlo simulation for TV trials.
    2.  Order the BS emissions differences from smallest to largest.
    3.  Identify the trial number at the lower end of the 90% confidence interval
                           niower = 0.95 * N -1.645V0.95 * 0.05 * N
    4.  Identify the trial number at the upper end of the 90% confidence interval
                           ripper = 0.95 * N + 1.645V0.95 * 0.05 * N
    5.  Compute (BS difference value at nupper)- (BS difference value at niower).
    6.  If the result in (5) < 1% of the BS emissions NTE threshold then convergence is met.
    7.  1% of Thresholds     g/hp-hr       g/kW-hr
       BSNOX              0.0200        0.026820
       BSNMHC           0.0021        0.002816
       BSCO              0.1940        0.260150

       The Screening Committee agreed to the proposed convergence criteria outlined above.
During the initial simulation  runs, all reference NTE events at an ideal BS emission level at the
threshold and below appeared to converge within the 1% level in 10,000 trials.  However, there
were a number of reference NTE events with BS emissions levels that were as much as 3 times
the NTE threshold.  This presented an initial problem in terms of the stated convergence criteria
since it was based  on  a fixed threshold value.  Essentially this meant that in order to meet the
criterion, some of the higher BS emission level events (>5 g/kW-hr) would have had to converge
to a 90% confidence width of well below 0.5% of the threshold value,  which would have
required an  extremely high number  of trials. To correct this problem the Steering Committee
chose to use the following two-step  procedure in deciding  the number of trials to run and the
convergence criteria:

1.  For all reference NTE events with  NOX values equal to or less than 2.6 g/kW-hr, a total of
10,000 trials were  run and checked  for convergence.   It was expected that all of these would
converge well within the 1% criteria  at this sample size.  If any individual reference NTE events
did not converge at this run length, those events were run to 30,000 trials.
SwRI Report 03.12024.06                  Page 31 of 371

-------
2.  For reference NTE events with NOX values greater than 2.6 g/kW-hr, a total of 10,000 trials
were run and checked for convergence. If convergence was not achieved those same events were
run to 30,000 trials. If these events still did not converge within 1% of the threshold value, the
procedure was to do one  of the following:

    a.  If there was convergence within at least 2% of the threshold value, the reference NTE
       event was included as part of the simulation data set for the measurement allowance and
       no additional runs were made.
    b.  If the reference NTE event did not converge within at least 2% of the threshold value, the
       event was  dropped from the simulation data set  considered for  the  measurement
       allowance.

In summary, the 195 reference NTE events were run at either 10,000 trials or 30,000 trials and
convergence was checked. For all but four reference NTE events, the convergence criteria was
met at the 1% NTE threshold value for all three  emissions and all three calculations methods.
Since only four reference NTE events failed the initial criteria at 1%  of the NTE threshold,
simulations for these four events were continued up to 50,000 trials.  By that point all four NTE
events met the convergence criteria.

2.1.8   Simulation Output

       During the  simulation of a reference NTE event, differences between the  BS emissions
"with errors", including time alignment adjustment, and the ideal BS emissions were obtained by
each of the  three calculation methods.  These differences were computed thousands of times
(once per trial)  until  the  model converged.   Then the 95th percentile  difference value was
determined for  each reference NTE event's distributions of BS differences for each emission
(NOX, NMHC, and CO) for all three calculation methods.

       The output  from  the Crystal Ball simulation for each reference NTE event was saved in
two separate Excel files: an EXTRACT and a REPORT file.  The EXTRACT  file contained
descriptive statistics on  all differences computed for BS emissions by all three calculation
methods, percentiles  (0%, 5%,  10%,...95%, 100%)  of the  differences  in BS emissions,
sensitivity data for all  error surfaces, and differences in BS emissions computed at each trial in
the simulation.

       The REPORT file contained a summary of the differences in the BS  emissions for all
three  calculation methods including descriptive statistics, the number  of trials  that were not
excluded due to periodic drift, a frequency histogram of the differences in BS emissions,  and
percentiles (0%, 5%, 10%,...95%,  100%)  of the differences in  BS emissions.  Also included
were  descriptive statistics on  each ic  distribution  sampled for each  error surface.   Lastly,
sensitivity charts for the differences in BS emissions for the three calculation methods were
stored.  These  charts  provided  information  on how  much  each error  surface influenced the
differences computed between the BS emissions "with errors" and the ideal BS emissions.

       A more detailed description of the Crystal Ball output files can be found in Appendix C.


SwRI Report 03.12024.06                  Page 32 of 371

-------
2.1.9  Step-by-Step Simulation Example

       In order to clarify the simulation process the following step-by-step summary is provided.
This  example  assumes  that a  single reference NTE event was simulated  for  the  BSNOX
difference computations. Figure 12 provides an overview of the simulation process.
    Reference NTE
                      Monte-Carlo Simulation
       NOx =NO
        + N02
        (ppm)
       NMHC
        (ppm)
       Exhflow
        (scfm)
        Torque
        (N-m)
        BSFC
       (g/kW-hr)
        Speed
        (rpm)
        Fuel
        Rate
        (L/sec)
        CO,
                                               Time
                                             Alignment
                                             Adjustment
95th percentile
  BSNOx
 differences
    4-
                                                        Calculate
                                                       Differences*
                                          (1) BSNOx = f (NOx, Exhflow, TECM, Speed)
                                          (2) BSNOx = f (NOx, Exhflow, BSFCECM)
                                          (3) BSNOx = f (NOx, CO2, CO, THC, TECM,
                                                  Fuel RateECM, Speed)
                                                                     J
                    'Difference = (BSNOx "with errors" and Time Alignment Adjustment) — ("Ideal" BSNOx)
      FIGURE 12.  OVERVIEW OF MONTE CARLO SIMULATION FOR BSNOx

STEP 1       Enter the reference NTE input parameters into the Monte Carlo (MC) simulation
model.  These include the emissions concentrations, exhaust flow, torque, BSFC, speed and fuel
rate data used in all three calculation methods.
STEP 2
NTE event.
Compute the "ideal" BSNOX by all three calculation methods from the reference
STEP 3       Set-up the Monte  Carlo simulation parameters  in Crystal  Ball.   An Excel
spreadsheet model was developed for use with Crystal Ball MC software for error analysis of
brake-specific emissions.  Crystal Ball is  a graphically-oriented forecasting and simulation
software that runs on Microsoft® Windows and Excel.  The simulations run in this program used
Crystal Ball 7.1 and 7.2.2 Academic versions and were run on PCs configured with a Pentium 4
CPU,  3.0  GHz,  2.0 GB RAM,  232  GB  hard  drive  and Windows XP  operating  system.
Microsoft® Excel 2003 SP was the spreadsheet software.

The options exercised in running Crystal Ball included the following:
    •   Number of trials = 10,000 or 30,000
SwRI Report 03.12024.06
                        Page 33 of 371

-------
          o   If the ideal emission < BS emission threshold then # trials = 10,000
          o   If the ideal emission > BS emission threshold then # trials = 30,000
          o   If convergence was not met then # trials = 50,000
   •   Monte Carlo sampling method with random initial seeds
   •   Normal speed run mode
   •   Suppress chart windows (fastest run time)

The Excel spreadsheet is in a modular  structure following the  specified model outline, and it
makes provisions for the  three identified calculation modules.  Input cells to the model  are
clearly identified to facilitate any revisions that may become necessary for users who want to
exercise the model with other Monte Carlo software such as @Risk or newer versions of Crystal
Ball. The spreadsheet was tested with controlled test cases of simplified input distributions with
the Crystal Ball add-on to confirm correct model implementation in accordance with this test
plan.  At least one typical analysis was run as an additional confirmation, and two independent
checks were made on the  ideal emissions by other SwRI staff.  A complete description of the
spreadsheet computations is contained in Appendix D.

STEP 4       Execute a single MC trial by  randomly generating  a separate ic for each error
surface used in the three calculations.

STEP 5       For each second in the reference NTE event, interpolate the A error for all error
surfaces at the input parameter values and the randomly  generated ic.  Figure 13 illustrates all the
error  surfaces available (in yellow) and where the corresponding A  errors  are added.  Error
surfaces depicted in blue were identified, discussed and eliminated by the Steering Committee.
SwRI Report 03.12024.06                 Page 34 of 371

-------
  Exhflow
|— H ANOxSSjTm
— H ANOxTOjTm
— >\ ANOxRMI/RFIJT,
-H ANOxrn,w_ITm
— *\ ANOx,
L- M ANOx,Hhmm
| — H ACO^ ~
— H AGO™ ~
— H ACCwI/PIJI 0/,
— H ACO^.. „,
— H ACO,.mr 0/.
>— H ACO.HK ox.

i— H ANMHC^ ^
— H ANMHCTO „,„
— H ANMHCm,roi.T
— H ANMHC^.,,^
— H ANMHC,.mrrr
— »| ANMHC.^j,^
1 — H ANMHC_^m,
i — H AExhflow.,.,
— H AExhflowTO
— H AExhflowr,,,,,«
— H AExhflow™,.,
— * AExhflow,,^^
— »• AExhflowBUI/p
— H AExhflow,^

— * AExhflow,_r

1 	 H AExhflow™,,,,,,
(1)
(2)
„ (3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)

(13)
(14)
^ (15)
„ (16)
„ (17)
(18)
(19)
(20)
(21)
„ (22)
(23)
(24)
BT (25)
(26)

(27)

. (28)
1 Engine
J dyno

) Environ
chambers



)yno
Environ
chambers


} Engine
dyno





1
L Engine
1
J dyno


Environ

h K
chambers




Torque

Blue = Deleted by SC
Yellow = Included in MC



BSFC



-
r>| ATorqueDvmmic (29)
*
>
->
->
->
ATorqueDOE (30)
ATorque,^,, (31)
ATorquehmT]id/fllel (32)
ATorque^,,; (33)
ATorqueillerI,olation(34)
J
ATorqueenBne.manuf(35J
ABSFCDynamlc (36)
ABSFCDOE (37)
-H ABSFC_B (38)
-H ABSFChmmd/fhel (39)
-H ABSFC^ (40)
-1 ABSFCmternolatlon (41)
-H ABSFCemme.manuf(42)






                                                                                   73
                                                                                   e
                                                                                   $
                                                                  ASpeedD
                                                              (43)
                                                                  AFuel
                                                                            c (44)
                                                               AC02S.. ,
                                                          (45)
                                                               ACO2T
                                                          (46)
                                                               AC02FMT/RFT % (47)
Engine
dyno
                                                               ACO2,,
                                                          (48)
                                                               AC02,m
                                                          (49)
                                                               AC02vih
                                                          (50)
   FIGURE 13. ERROR SURFACES INCLUDED IN MONTE CARLO SIMULATION
STEP 6
Compute one BSNOX "with errors" for the given MC trial by  adding all the A
error values to the reference NTE data and then calculating the BSNOX by all three calculation
methods.
STEP?
Check for periodic drift to determine if the BSNOX "with errors" for the given MC
trial is valid.  If it is valid then continue to Step 8.  Otherwise, eliminate the data from the current
trial and return to Step 4 to start a new trial.

STEP 8      Compute BSNOX difference for the current trial:

          (BS emission "with errors" * Time Alignment Adjustment) - "Ideal" BS emission

STEP 9      Repeat Steps 4-8 until the number of trials is met.

STEP 10     Check the differences in BSNOX for all three calculation methods to be certain
that the convergence criteria  are met.  If convergence is met for all three calculation methods,
continue to Step  11.  Otherwise,  return to  Step 4 and run the Monte Carlo simulation for an
additional 10,000 trials until the total number of trials is 50,000.
SwRI Report 03.12024.06
                        Page 35 of 371

-------
                        -th
STEP 11     Select the 95  percentile from the distribution of BSNOX differences for each of
the three calculation methods. Store the ideal BSNOX and the 95th percentile BSNOX differences
for computing the measurement allowance.

STEP 12     Repeat Steps 1-11 for each reference NTE event.

2.1.10  Measurement Allowance

       At this point in the process there were nine  distributions  of 95th percentile differences,
where the 195 reference NTE events were pooled by the three emissions (NOX, CO, NMHC)
times three different calculation methods. Each of the 95th percentile distributions represented a
range of possible measurement allowances.  From each of these  nine distributions of possible
measurement allowances, one measurement  allowance per distribution was determined.  These
measurement allowances were  computed by  a regression  method or a  median method as
described below. Both of these calculations methods were decided by the Steering Committee
prior to the start of the program, and they were specified in the Test Plan.

Regression Method

       This method involved determining the correlation between the 95th percentile differences
versus  the  ideal emission values  for  the reference NTE  dataset.   For each combination of
emissions  and  calculation  method, a least squares linear  regression  of the  95th  percentile
differences versus the ideal emissions results was computed.  If the R2 value from the regression
model was greater than 0.90 and the SEE (standard  error of the estimate or root-mean-squared-
error) was less than 5% of the median ideal  emission result, then  the linear regression equation
was used to determine  the measurement  allowance  for that emissions and calculation method.
To determine the  measurement  allowance the NTE threshold  was  used to  predict the
measurement allowance from the regression model.  The NTE thresholds are given in Table 13.
The measurement allowance was then expressed as a  percentage of the NTE threshold value.

Median Method

       If the linear regression did not pass the aforementioned criteria for the  R2 and  SEE
statistics, then the median value of the 95th percentile differences from the 195 reference NTE
events  was used as the single measurement allowance for a combination of emissions and
calculation method.  The measurement allowance was then expressed as a percentage of the NTE
threshold value.

       After  all 95th percentile distributions were  evaluated, there were nine measurement
allowances corresponding to the nine combinations of the three emissions and the three different
calculation methods.

       Next the maximum allowance (in percent) among the three emissions was selected for
each of the given calculation methods. The calculation method corresponding to the  minimum
of these three  maximum  values  was chosen  as the  best method, and  it provided  the BS
measurement allowances (in percent) for NOX, NMHC, and CO, respectively. The final additive
SwRI Report 03.12024.06                 Page 36 of 371

-------
BS measurement allowances were  computed by multiplying each of the three measurement
allowances (in percent) times their corresponding threshold values.  Each of these values would
be the very last value added to the actual brake-specific NTE threshold for a given engine, based
on actual  family emissions limit, mileage, model  year, etc.  Note that if  any measurement
allowance was determined to have a value less than zero, then that measurement allowance was
set equal to zero.

       Table 20 below illustrates  the selection  of the calculation method for all  of  the
measurement allowances.  The example is based on a hypothetical set of nine measurement
allowances for the  three emissions and three calculation methods.  The calculation method is
selected by first picking the maximum allowances of all the emissions for  each of the given
calculation methods.  For each column the maximum value is selected (highlighted in yellow).
Then the minimum  of these maximums is used to select the best method (highlighted in blue).  In
this hypothetical case, the BSFC method would be selected. Therefore, 18%, 17%, and  2%
would be selected as the best measurement allowances for NOX, NMHC, and CO, respectively.
     TABLE 20. EXAMPLE OF SELECTION OF THE MEASUREMENT ERROR
Calc. Method ==>
BSNOx
BSNMHC
BSCO

Max Error ==>
Min of Max ==>
Selected Method==>
Measurement Errors at Respective NTE Threshold (%)
Method 1
Torque- Speed
18%
19%
3%

19%

Method 2
BSFC
18%
17%
2%

18%
18%
Method 3
ECM Fuel Specific
20%
14%
1%

20%

"BSFC "Method 2
   For the data given in Table 20, the BS measurement allowances would be computed as:
   •   NOX
   •   NMHC
   •   CO
= 18%*2.00g/hp-hr =
= 17%* 0.21 g/hp-hr =
= 2%  * 19.4g/hp-hr =
0.3600 g/hp-hr
0.0357 g/hp-hr
0.3880 g/hp-hr
2.1.11  Validation

       The final validation methodology for the Monte Carlo model varied from the one that
was originally proposed in the Test Plan. This occurred for several reasons.

   •   The method described in the Test Plan required that CE-CERT be able to measure raw
       emissions concentrations or determine dilution ratio accurately.  However, CE-CERT's
       mobile laboratory was only capable of making dilute measurements;  therefore a dilution
SwRI Report 03.12024.06
                Page 37 of 371

-------
       ratio needed to be established.  In addition, the mobile laboratory did not include any
       direct method for the measurement of either exhaust flow or intake air flow.  Although
       the CE-CERT mobile laboratory could measure dilution ratio by measuring both the total
       CVS flow and the dilution air flow rate, and subtracting to determine exhaust flow rate,
       there was some concern about the accuracy of this measurement during short NTE events
       involving a potentially wide dynamic range of dilution ratios.  Since the success of this
       measurement would  be critical to the model validation under the methodology given in
       the Test Plan, the  Steering Committee decided that, due to the reliance on this  dilution
       ratio measurement method, there was  a  considerable degree of risk associated with  the
       original validation methodology, and that an alternative method might prove more robust.

   •   The Test Plan included an alternative  methodology in the event that the CE-CERT
       laboratory was  unable to accurately determine raw  exhaust flow or  dilution  ratio.
       However,  the  proposed  method had  several  potential  problems,  and  the  Steering
       Committee decided that this option was not a good choice due to potential bias problems.

   •   A third option was also mentioned briefly in the Test Plan.  It involved comparing  the
       NTE events recorded by the PEMS and the CE-CERT trailer.  However, the Steering
       Committee decided that the proposed method of comparison was not well defined.

   After several discussions the  Steering Committee selected an alternative approach that was
based on a robust validation method which did not rely on measurement of exhaust flow or raw
gaseous concentrations. This method was initially proposed by SwRI at the June 2006 Steering
Committee  meeting in  San  Antonio. The proposed method had some similarity to the third
option proposed in the Test Plan, in that the deltas (PEMS vs. Lab) generated by CE-CERT were
to be  compared with those generated by the Model.  However, the method of comparison was
different. The key assumptions in using this method are listed below.

   1.      It was understood that CE-CERT could not measure torque  directly and  that no
          reference torque  would be available.  This meant  that the laboratory BS emission
          values provided  by CE-CERT were to use the same "torque-basis" as the PEMS
          measurements.
   2.      It was assumed that CE-CERT would provide BS emission values for each  on-road
          NTE event by all three calculation methods for both PEMS and the mobile laboratory.
   3.      SwRI was to calculate the "deltas" between the  PEMS and the CE-CERT laboratory
          (i.e., PEMS - CE-CERT).
   4.      The  CE-CERT data was to  include  both in-cab  and on-frame  mounted  PEMS
          measurements, but these were to be pooled together to provide a single data set.
   5.      When the Monte Carlo Model was run through the set of 195 reference NTE events,
          two sets of results were to be generated.  One set included BS emissions with all error
          surface deltas applied, and a second  set which included BS emissions  with some of
          the  error surface deltas  excluded (primarily those  associated with torque, BSFC,
          speed and fuel rate).  These results were to be generated simultaneously during each
          reference NTE model run.  Essentially this yielded two Monte Carlo Model results, a
          "Validation" result (used for the on-road validation) and a "Full Model" result (to be
          used for the measurement allowance generation and for the lab replay validation).
SwRI Report 03.12024.06                 Page 38 of 371

-------
          The "Validation" result  also included time alignment adjustment and checks for
          periodic drift.

On-Road Validation Methodology

The  Measurement Error Monte  Carlo  Simulation Model was validated by comparing the
simulation results using the data from the 195 reference NTE events to the on-road results using
the data from  the 100 NTE events collected using the CE-CERT trailer and PEMS unit.  This
was accomplished using the methodology described below.

   1.  Simulation Results

   •   The Monte Carlo Model was run using the data from the 195 reference NTE events.  In
       order to obtain Monte Carlo Model simulations representing similar conditions to those
       obtained on-road,  certain error surfaces needed to be suppressed in the simulations since
       not all  of them were applicable to the conditions used in collecting the on-road data. The
       error surfaces excluded were all torque and BSFC  error surfaces, dynamic  speed and
       dynamic fuel rate. This is the "Validation" result described earlier.
   •   For each  reference  NTE event,  various percentiles, such as the 5th  and 95th,  of the
       simulated distribution of the BS emissions differences, defined as

       delta BS = BS emissions with "Validation" error - "Ideal" BS emissions,

       were obtained.  In essence the model produced a "distribution of deltas" for  each NTE
       event for all three calculation methods.
   •   The BS emissions included BSNMHC, BSCO, and BSNOX using all  three calculation
       methods. Thus, there were 9 sets  of data (i.e., 3 emissions x 3 calculation methods).
   •   For each set of data and each percentile chosen in  the study, the Monte Carlo  Model
       produced 195 BS delta values (i.e., one from each reference NTE event). These delta BS
       values  were ordered from smallest to largest, and then the empirical distribution function
       (edf) of these delta BS values was plotted. The edf is a cumulative plot of the fraction of
       the 195 delta BS observations that were less than or equal to x, versus each x, where x
       was the observed BS delta value.
   •   For each  of the nine sets  of data (3 emissions and  3  calculation  methods), the edf
       corresponding to the 5th and 95th percentile distributions were plotted on the same plot.

Figure 14 contains an illustration of a plot of the edf for 5th and 95th percentiles. The region
between these two curves was designated as the validation region for comparison  of the edf
obtained from  the on-road data from CE-CERT.
SwRI Report 03.12024.06                 Page 39 of 371

-------
       1.0
       0.8
    (0
   •§  0.6
1 0.4
3
3
O
       0.2
       0.0
                                               7
                     Model Simulation
                         5th %
                                                  Model Simulation
                                                      95th %
                                 Delta BS Emissions
     FIGURE 14.  PLOT OF MODEL-GENERATED EMPIRICAL DISTRIBUTION
                       FUNCTIONS FOR TWO PERCENTILES

   2.  On-Road Results

   •   The CE-CERT trailer was driven on selected on-road routes to collect emissions data. In
       addition, a PEMS installed in the tractor pulling the trailer collected emissions data.
       From the routes driven with the CE-CERT trailer approximately  100 NTE events were
       down-selected by the Steering Committee.
   •   For each on-road NTE event, a delta BS emissions value, defined as

       delta BS emissions = PEMS BS emissions - CE-CERT BS emissions,

       was computed.
   •   As before, the BS emissions included BSNMHC, BSCO, and BSNOX using all three
       calculation methods.  Thus, there  were 9 sets of data (i.e., 3 emissions x 3 calculation
       methods).
   •   For each set of data the delta BS values  were ordered from smallest to largest and then
       the empirical distribution function (edf) of these delta BS values  were plotted on the same
       plot as the matching simulation data.

The percentile edfs based on the "Validation" set of simulated data were used for comparison
with the edf obtained from the on-road data. Figure 15 contains an  illustration of a plot of the
matching simulated and on-road edfs.
SwRI Report 03.12024.06
                                 Page 40 of 371

-------
       1.0
       0.8
   .Q
   (0
   |  0.6
   ol
       0.4
    3
   O
       02
       0.0
                     Model Simulation
                         5th %
                                                  Model Simulation
                                                      95th %
                                 Delta BS Emissions
     FIGURE 15. PLOT OF ON-ROAD AND MODEL-GENERATED EMPIRICAL
                            DISTRIBUTION FUNCTIONS

   3.  Comparison of Results

       Several methods to compare the results of the model simulation deltas and the on-road
deltas  for validation were  presented to the Steering  Committee  at the June 2006 Steering
Committee meeting in San Antonio. At that meeting all the validation proposals were discussed
and a number of alternatives were presented.

       Ultimately, the Steering Committee elected to proceed with the following method as  a
validation methodology. From the on-road and model-generated empirical distribution functions
as shown in Figure 15, we would observe how many points of the on-road edf did not fall
between  the points  of the  boundary edfs supplied  by the  simulation model.  The Steering
Committee agreed that if at least 90% of the on-road data were within the 5th and 95th percentile
differences from  the model simulation the model was considered valid for a particular BS
emissions and calculation method. However, if 10% of the on-road  differences were outside the
model  edfs either on the high or the low side, then the data would be investigated to try to
determine the cause.   This analysis was  performed independently for each  pollutant and  each
calculation method.   This decision  was  later confirmed  at the November 2006  Steering
Committee meeting in San Antonio.
SwRI Report 03.12024.06
Page 41 of 371

-------
   It should be noted that none  of these comparison methods had any  effect on the final
generation of the Measurement Allowances.  These were still generated using the 95th percentile
simulation results using the "Full Model."

Laboratory Replay Validation Methodology

       The laboratory replay validation was not as well defined as the on-road validation in the
Test Plan, either in terms of testing scope or in terms of the validation methodology to be used
with the resulting data.   According  to the Test Plan, the laboratory replay was to involve
removing the engine from the test truck used by CE-CERT for the on-road validation, and using
that engine to "replay" some of the on-road testing episodes in the laboratory to the extent it was
possible to do so.

       In the initial proposal for this program, SwRI established a planned level of effort  for the
laboratory replay testing involving roughly one month of effort, based on initial discussions with
the Steering Committee and the limited details given in the Test Plan.  This timeframe included
removal of the test engine from the truck, installation in the transient cell, cycle generation, cycle
tuning, and testing.

       The final scope of the laboratory replay testing involved simulating one hour of operation
from each of the three test routes run by CE-CERT during the on-road validation testing.  During
the course of the on-road validation exercise, personnel from Caterpillar  were onsite with CE-
CERT in  order to  facilitate the recording of certain proprietary engine data channels from the
ECM.  This was done in order to  provide data to later  assess the  accuracy of the laboratory
replay simulation.  However, this data was only successfully recorded during the portion of on-
road testing which was conducted with the PEMS mounted outside the truck cabin (i.e., "frame-
mounted" data). As a result, only the frame-mounted on-road operations were simulated  during
the laboratory replay.

       One hour of operation  was selected from each route run by CE-CERT, with preference
generally  given to hours of operation containing the highest frequency of NTE events.  Each
hour of operation was the basis of an  hour long test cycle, which was replayed in its entirety in
the laboratory.   The data from  this hour of operation was then divided into individual NTE
events via the standard entry and exit  logic used throughout the program (i.e., evaluation on a 1
Hz basis). Successful  replay  operation was determined  in close consultation with Caterpillar
personnel who  aided in the interpretation of proprietary engine ECM data.  After successful
replay operation was achieved, each cycle was repeated three times to generate a validation data
set.

       Brake-specific deltas  were determined  using  two  different methods,  which were
differentiated by the method  of generating the laboratory brake-specific emissions levels for
comparison  to  PEMS  generated values.  The first method was to calculate the laboratory
reference  values using the  standard Laboratory Reference  method for work calculation which
involved Laboratory measured engine Speed and engine  Torque (via the  test cell inline  torque
meter).   The PEMS values for each calculation  method  were compared to this Laboratory
SwRI Report 03.12024.06                 Page 42 of 371

-------
Reference value to generate a delta.  These are referred to as "full" deltas, since all error terms
are considered.

       The second method was to generate deltas using a methodology similar to that used by
CE-CERT for the on-road validation data.  In this  method, the work term  for each PEMS
calculation method was used for both the Laboratory data and the PEMS data.  This essentially
eliminates any errors due  to torque  and BSFC measurement. A separate Laboratory value is
generated for each calculation method under this scenario. These values were compared for each
calculation method to generate another set of deltas.  This second set of deltas is referred to as
"mass" deltas, because effectively only errors in the  determination of  emission mass rates  are
considered.

       The originally intended method for analysis of the replay data was to compare the "full"
deltas for the replay validation testing to the simulation deltas generated using the "Full Model."
However, the Steering Committee later decided that this comparison was not appropriate, due to
the fact that the  laboratory  replays were not able to test as wide a  range of environmental
parameters as the  on-road testing. Therefore, different percentiles from the model results needed
to be chosen to establish the proper validation window for the replay testing data. The validation
window from the  "Full Model" result, which included environmental factors, would be too wide
for a proper validation.

       An alternative method for the treatment of the replay  data was discussed at the January
24th 2007 Steering Committee meeting.  At that meeting, it was determined that the  proper use
for the replay data was to examine the "incremental" errors arising  from torque  and BSFC
measurement errors which  were not properly examined during the on-road validation,  due to lack
of a reference torque measurement.  Given this direction, SwRI determined an alternate method
of comparison, which was presented to the  Steering Committee at the February 15th  2007
Steering Committee meeting, and is summarized below.

Monte Carlo Model Data

       The model incremental deltas were determined by comparing the results of the Full
Model to the results of the Validation Model. This was done on an event-by-event basis for all
195 reference NTE events.  In each case, a "work" delta was generated as follows:



          Model,Work         Model,Full        Model,Validation


Replay Validation Data

       As mentioned earlier, two deltas had  been generated for each calculation  method in
comparing the PEMS brake-specific  values to the Laboratory  values.  The "full" deltas used the
Laboratory values generated using the lab measured torque as a basis for the work  term. The
"mass" deltas used Laboratory values generated using the same work term as the associated
PEMS calculation method, essentially with any work differences  cancelled out via calculation.
SwRI Report 03.12024.06                  Page 43 of 371

-------
For each calculation method,  an  incremental "work" delta was generated by comparing its
associated "full" delta to the appropriate "mass" delta as follows:



             replay,Work         replay,Full         replay,Mass


       This calculation was performed individually for each replay NTE event and for each of
the three repeat runs.

Data Comparison

       Initially, the model work deltas and the replay work deltas were plotted against the brake-
specific emission  level for each event.  For the model work deltas, the x-value was the ideal
brake-specific emission level for the reference NTE event in question. For the  replay validation
work deltas, the x-value was the Laboratory Reference brake-specific emissions level calculated
using measured torque.  The plots were initially examined to see if the replay validation work
deltas fell within the range of values produced by the model.

       Assuming  this initial assessment warranted  a more rigorous comparison, the final replay
validation comparison would be made in a manner similar to that used for the on-road validation
data that was described above.  An edf would be made using work deltas generated by the model
at both the 5th and 95th percentiles to generate a validation window.  This would be compared to
an edf of the replay validation work deltas to determine of 90 percent or more of the replay data
fell within the validation window.
SwRI Report 03.12024.06                 Page 44 of 371

-------
                   3.0    1065 PEMS AND LABORATORY AUDIT

3.1    Audit Objective

       An initial task of the  program was  to audit the PEMS and dynamometer  laboratory
according to 40 CFR Part 1065 Subpart D.  The audit procedures were  performed to insure the
equipment used for the In-Use Measurement Allowance Program met the minimum performance
requirements as regulated by the EPA.

3.2    Overview of 1065 Audit Activities

       The list of audits to be  conducted for both the laboratory and the PEMS was finalized by
the Steering Committee at the August, 2005  meeting in Ann Arbor, MI. Table 21 summarizes
the required audits for both the laboratory and PEMS  instruments.  Subsequent sections will
detail the results for the individual performance checks that were conducted as part of the audits,
as well as any corrective action taken as a result of those checks.
SwRI Report 03.12024.06                 Page 45 of 371

-------
   TABLE 21. 1065 AUDITS AND PERFORMANCE CHECKS REQUIRED FOR THE
                    MEASUREMENT ALLOWANCE PROGRAM
Description
Linearity
Torque Meter
Fuel Flow
Intake Flow
Exhaust Flow
CVS Verification
H2O Interference on CO2
H2O and CO2 Interference on CO
FID Optimization
Non-stoichiometric raw FID O2
Interference
Nonmethane cutter penetration fractions
CLD H2O and CO2 quench
NDUV HC and H2O Interference
Chiller NO2 penetration
NO2-to-NO converter check
CFR Reference
1065.307
1065.310
1065.320
1065.325
1065.330
1065.341
1065.350
1065.355
1065.360
1065.362
1065.365
1065.370
1065.372
1065.376
1065.378
Lab Raw
x1
X
X
X


X
X
x3
X
X
X


X
Lab Dilute
x1
X



X
X
X
x3

X
X


X
PEMS
x2



X

X
X
x3
X


X
X

1 Linearity for lab on gas analyzers, flow meters, torque meter, pressures, temperatures
2 Linearity for PEMS on gas analyzers, exhaust flow meters
3 Verify methane response factors only, THC instruments
3.2.1  Laboratory Audits

      The results of the laboratory audit were presented  to the Steering Committee at the
January, 2006 meeting in San Antonio, TX.  The laboratory audit results indicated that the SwRI
reference laboratory met all of the requirements given under Part 1065 Subpart D. At that time,
the Steering Committee approved  the laboratory  audit results for  both the  raw and dilute
sampling systems, and SwRI was not directed to take any corrective actions for the laboratory.

      Regular performance checks were performed throughout the program as required by Part
1065 Subpart D.  However, only the results of the initial 1065 audit are included in this report.
Documentation of all regular performance checks is available at SwRI if needed.
SwRI Report 03.12024.06
Page 46 of 371

-------
3.2.2   PEMS Audits

       The initial  audits of the  first four PEMS units (PEMS 1 through 4) were started in
January, 2006 and  completed by  mid-February, 2006. These initial audits included only the 5-
inch EFMs used for Engine 1 testing. Audits of the 4-inch and 3-inch EFMs were completed at a
later time, closer to the testing needs for Engines 2 and 3.  The PEMS units were later modified
to address a 1065 NC>2 penetration check failure.  PEMS  1 through 4 were audited again in June,
2006, once all modifications were completed.  For these subsequent audits, only linearity and
NO2 penetration were checked.

       Two additional Sensors Inc. PEMS units (PEMS  5 and 6) as well as a Horiba OBS-2200
arrived at SwRI in June, 2006.  Upon arrival, PEMS  5 and  6 and the  OBS-200  were given
complete 1065 audits as outlined  in Table 21.  A final PEMS unit  (PEMS 7) arrived at SwRI in
October of 2006 to serve as a spare. PEMS 7 was given a complete 1065 audit at that time.

       Additional PEMS linearity checks were performed as required by Part 1065 Subpart D
over the course of the program.  In addition, a number  of additional audits were required as a
result of maintenance or repairs performed on several of the PEMS units over the course of the
program.  This  report contains the results of initial audits on all PEMS units,  as well as those
performed subsequent to the NO2 penetration modifications which were completed in June of
2006.  In  addition, relevant  audit results  are also given for any major repairs or maintenance
events which  occurred on the PEMS equipment.

3.3    Gas Analyzer Linearity Verifications

       Analyzer linearity checks  were performed as specified  in CFR Part 1065.  The Federal
Register defines linearity in terms  of the  maximum concentration expected during testing.
Performing the  PEMS and laboratory  audits  prior to  engine  testing, the maximum  test
concentrations  were  unknown.     Therefore,  the  mono-blend linearity  verification  gas
concentrations were  used to define  the  1065 linearity criteria.  This interpretation  of the
verification resulted in the most liberal linearity criteria.  Mono-blend span gases were used with
a STEC Inc. Model SGD-710C 10-step gas divider, shown in Figure 16, to perform the PEMS
analyzer linearity verification.  Span gas concentrations for the SEMTECH-DS and laboratory
analyzers were  near the values listed in  Table 22.   Span  concentrations for the PEMS were
selected based  on  manufacturer  recommendations in the  SEMTECH-DS user  manual.  All
linearity checks on the PEMS were performed using the PEMS span port.
SwRI Report 03.12024.06                  Page 47 of 371

-------
   FIGURE 16. STEC INC. MODEL SGD-710C GAS DIVIDER WITH SEMTECH-DS
                                       PEMS
     TABLE 22. SPAN CONCENTRATIONS USED FOR THE SEMTECH-DS AND
                           LABORATORY ANALYZERS
Analyzer Description
Dilute MEXA 7200D and
Horiba CH4 Bench
Raw MEXA 7200D and
Horiba CH4 Bench
SEMTECH-DS
NO
[ppm]
N/A
N/A
960
NO2
[ppm]
N/A
N/A
260
NOX
[ppm]
92
900
N/A
CO2
[%]
5.5
14.5
12
CO
[ppm]
47
47
960
THC
[ppmC]
9
9
660
CH4
[ppmC]
23
23
N/A
      As shown in Table 23  and Table 24, the laboratory analyzers easily passed the  1065
linearity criteria.   The MEXA benches use a Horiba GDC 703 gas divider and perform the
linearity checks in an automated process.  The STEC Inc. manual gas divider was used to check
the dilute and raw CH4 benches.  Linearity checks were performed monthly for all laboratory
analyzers during the program. The results of all monthly linearity checks are not included in this
report beyond those associated with the initial 1065 audit of the laboratory equipment.
SwRI Report 03.12024.06
Page 48 of 371

-------
   TABLE 23. DILUTE MEXA 7200D AND HORIBA CH4 BENCH 1065 LINEARITY
                        VERIFICATION SUMMARY
Verification Description
Intercept
Slope
SEE
r2
CONDIR
Measured
Linearity Criteria
Pass / Fail
0.00
1.13
Pass
1.00
0.99-1.01
Pass
0.03
2.26
Pass
1.00
0.998
Pass
CO2 NDIR
Measured
Linearity Criteria
Pass / Fail
0.00
0.03
Pass
1.00
0.99-1.01
Pass
0.00
0.05
Pass
1.00
0.998
Pass
HCFID
Measured
Linearity Criteria
Pass / Fail
0.40
2.21
Pass
1.00
0.99-1.01
Pass
0.56
4.43
Pass
1.00
0.998
Pass
NOX CLD
Measured
Linearity Criteria
Pass / Fail
-0.04
1.36
Pass
1.00
0.99-1.01
Pass
0.12
2.72
Pass
1.00
0.998
Pass
CH4 FID with NMHC Cutter
Measured
Linearity Criteria
Pass / Fail
0.00
0.11
Pass
1.00
0.99-1.01
Pass
0.06
0.23
Pass
1.00
0.998
Pass
    TABLE 24. RAW MEXA 7200D AND HORIBA CH4 BENCH 1065 LINEARITY
                        VERIFICATION SUMMARY
Verification Description
Intercept
Slope
SEE
r2
CONDIR
Measured
Linearity Criteria
Pass / Fail
0.80
4.52
Pass
1.00
0.99-1.01
Pass
2.25
9.04
Pass
1.00
0.998
Pass
CO2 NDIR
Measured
Linearity Criteria
Pass / Fail
-0.01
0.07
Pass
1.01
0.99-1.01
Pass
0.03
0.15
Pass
1.00
0.998
Pass
HCFID
Measured
Linearity Criteria
Pass / Fail
0.00
2.34
Pass
1.00
0.99-1.01
Pass
0.13
4.68
Pass
1.00
0.998
Pass
NOX CLD
Measured
Linearity Criteria
Pass / Fail
0.00
1.36
Pass
1.00
0.99-1.01
Pass
0.07
2.72
Pass
1.00
0.998
Pass
CH4 FID with NMHC Cutter
Measured
Linearity Criteria
Pass / Fail
0.00
0.11
Pass
1.00
0.99-1.01
Pass
0.06
0.23
Pass
1.00
0.998
Pass
SwRI Report 03.12024.06
Page 49 of 371

-------
       Table 25 summarizes the linearity verifications performed on PEMS 1. During the initial
audit, PEMS 1  repeated failed the linearity check with NO and NO2. Because PEMS 1 was the
only unit to fail linearity with NO during the initial audit, the unit was returned to Sensors for
correction in accordance with the Test Plan.   Sensors  recalibrated the NO  component  of the
NDUV and sent PEMS 1 back to SwRI.  The recalibrated unit passed the NO linearity check

       PEMS 1 also failed the linearity check with NO2, as did many of the other PEMS during
the initial audit.  All of these linearity failures  involved the intercept being above the required
level.  Sensors Inc. was offered a chance to correct the problem;  however they declined,
indicating that they felt the units were operating correctly despite the intercept failures.  Sensors
Inc. indicated that they felt there were problems with the 1065 linearity requirements as written,
and that widening the intercept linearity criteria should be considered.  This was reported to the
Steering Committee during the February 23, 2006 conference call, and the decision was made to
continue testing as allowed in section 3.1.4 of the Test Plan.

       During  the  June, 2006 linearity  checks that followed the NO2 penetration upgrades,
PEMS 1 NDUV measurements became unstable and the instrument could not be zeroed or
spanned properly.  Following subsequent diagnostics,  the  NDUV analyzer was replaced by
Sensors Inc.  PEMS 1 passed the linearity checks for both NO and NO2 with the new analyzer.
SwRI Report 03.12024.06                  Page 50 of 371

-------
          TABLE 25. PEMS 1 1065 LINEARITY VERIFICATION SUMMARY
Verification Description
Intercept
Slope
SEE
r2
CO NDIR (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
0.00
4.52
Pass
1.00
0.99-1.01
Pass
0.00
9.04
Pass
1.00
0.998
Pass
CO NDIR (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
0.00
4.52
Pass
1.00
0.99-1.01
Pass
0.00
9.04
Pass
1.00
0.998
Pass
C02 NDIR (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
0.02
0.07
Pass
1.00
0.99-1.01
Pass
0.02
0.15
Pass
1.00
0.998
Pass
C02 NDIR (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
0.05
0.07
Pass
1.00
0.99-1.01
Pass
0.03
0.15
Pass
1.00
0.998
Pass
HC FID (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
-0.06
2.21
Pass
1.00
0.99-1.01
Pass
0.26
4.43
Pass
1.00
0.998
Pass
HC FID (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
-1.68
2.21
Pass
1.00
0.99-1.01
Pass
0.53
4.43
Pass
1.00
0.998
Pass
NO NDUV (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
8.25
4.42
Fail
0.98
0.99-1.01
Fail
4.46
8.84
Pass
1.00
0.998
Pass
NO NDUV (NO Recalibrated by Sensors Inc. 02-15-06)
Measured
Linearity Criteria
Pass / Fail
3.77
4.42
Pass
1.00
0.99-1.01
Pass
2.06
8.84
Pass
1.00
0.998
Pass
NO NDUV (New NDUV 06-07-06)
Measured
Linearity Criteria
Pass / Fail
3.75
4.42
Pass
0.99
0.99-1.01
Pass
2.09
8.84
Pass
1.00
0.998
Pass
N02 NDUV (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
2.15
1.42
Fail
1.00
0.99-1.01
Pass
0.92
2.83
Pass
1.00
0.998
Pass
N02 NDUV (New NDUV 06-07-06)
Measured
Linearity Criteria
Pass / Fail
-0.85
1.29
Pass
1.00
0.99-1.01
Pass
0.81
2.58
Pass
1.00
0.998
Pass
       Table 26  shows the summarized linearity verification results for PEMS 2.  PEMS 2
narrowly failed linearity for NC>2 during the initial audit. Per the steering committee's decision,
SwRI Report 03.12024.06
Page 51 of 371

-------
no action was taken to correct the linearity failure.  During the June, 2006 audits that followed
the NC>2 penetration upgrades, PEMS 2 passed the NC>2 linearity check.
          TABLE 26. PEMS 2 1065 LINEARITY VERIFICATION SUMMARY
Verification Description
Intercept
Slope
SEE
r2
CO NDIR (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
-0.91
4.52
Pass
0.99
0.99-1.01
Pass
4.96
9.04
Pass
1.00
0.998
Pass
CO NDIR (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
-2.27
4.52
Pass
1.01
0.99-1.01
Pass
2.75
9.04
Pass
1.00
0.998
Pass
CO2 NDIR (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
-0.02
0.07
Pass
1.00
0.99-1.01
Pass
0.02
0.15
Pass
1.00
0.998
Pass
CO2 NDIR (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
0.02
0.07
Pass
1.01
0.99-1.01
Pass
0.04
0.15
Pass
1.00
0.998
Pass
HC FID (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
0.98
2.21
Pass
1.00
0.99-1.01
Pass
0.73
4.43
Pass
1.00
0.998
Pass
HC FID (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
0.48
2.21
Pass
1.00
0.99-1.01
Pass
0.56
4.43
Pass
1.00
0.998
Pass
NO NDUV (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
3.65
4.42
Pass
1.00
0.99-1.01
Pass
2.91
8.84
Pass
1.00
0.998
Pass
NO NDUV (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
3.81
4.42
Pass
1.00
0.99-1.01
Pass
3.17
8.84
Pass
1.00
0.998
Pass
NO2 NDUV (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
2.10
1.42
Fail
1.01
0.99-1.01
Pass
1.88
2.83
Pass
1.00
0.998
Pass
NO2 NDUV (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
0.46
1.29
Pass
1.00
0.99-1.01
Pass
0.59
2.58
Pass
1.00
0.998
Pass
       Table 27 summarizes the linearity verification results for PEMS 3. PEMS 3 also failed
linearity for NC>2 during the initial audit.  As with PEMS 1, problems developed with the NDUV
SwRI Report 03.12024.06
Page 52 of 371

-------
during the June, 2006 audits that followed the NC>2 penetration upgrades, and the NDUV was
replaced by Sensors Inc.  The new NDUV passed NC>2  linearity, but narrowly failed NO
linearity.  Sensors did not elect to perform any corrective action on the unit as a result of this
failure.  The NO linearity failure was reported to the  Steering Committee during the regular
conference call on June 13th,  2006,  and the decision was made to continue testing despite the
failure.
          TABLE 27. PEMS 3 1065 LINEARITY VERIFICATION SUMMARY
Verification Description
Intercept
Slope
SEE
r2
CO NDIR (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
0.00
4.52
Pass
1.00
0.99-1.01
Pass
0.00
9.04
Pass
1.00
0.998
Pass
CO NDIR (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
-1.36
4.52
Pass
1.00
0.99-1.01
Pass
2.75
9.04
Pass
1.00
0.998
Pass
C02 NDIR (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
0.03
0.07
Pass
1.00
0.99-1.01
Pass
0.04
0.15
Pass
1.00
0.998
Pass
C02 NDIR (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
0.00
0.07
Pass
1.01
0.99-1.01
Pass
0.03
0.15
Pass
1.00
0.998
Pass
HC FID (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
0.60
2.21
Pass
1.00
0.99-1.01
Pass
0.51
4.43
Pass
1.00
0.998
Pass
HC FID (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
-0.96
2.21
Pass
1.00
0.99-1.01
Pass
0.27
4.43
Pass
1.00
0.998
Pass
NO NDUV (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
3.56
4.42
Pass
0.99
0.99-1.01
Pass
2.53
8.84
Pass
1.00
0.998
Pass
NO NDUV (New NDUV 06-06-06)
Measured
Linearity Criteria
Pass / Fail
5.70
4.42
Fail
1.00
0.99-1.01
Pass
2.02
8.84
Pass
1.00
0.998
Pass
N02 NDUV (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
7.38
1.42
Fail
0.98
0.99-1.01
Fail
1.29
2.83
Pass
1.00
0.998
Pass
N02 NDUV (New NDUV 06-06-06)
Measured
Linearity Criteria
Pass / Fail
0.73
1.29
Pass
1.01
0.99-1.01
Pass
1.17
2.58
Pass
1.00
0.998
Pass
SwRI Report 03.12024.06
Page 53 of 371

-------
       Table 28 shows the summarized CO2, CO, and HC linearity verification results for PEMS
4.  PEMS 4 passed the linearity check for CO2, CO, and HC during the initial audit as well as the
June 2006 checks that followed the NO2 penetration upgrades.
          TABLE 28. PEMS 4 1065 LINEARITY VERIFICATION SUMMARY
Verification Description
Intercept
Slope
SEE
r2
CO NDIR (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
0.00
4.52
Pass
1.00
0.99-1.01
Pass
0.00
9.04
Pass
1.00
0.998
Pass
CO NDIR (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
-1.36
4.52
Pass
1.00
0.99-1.01
Pass
3.16
9.04
Pass
1.00
0.998
Pass
CO2 NDIR (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
-0.01
0.07
Pass
1.01
0.99-1.01
Pass
0.03
0.15
Pass
1.00
0.998
Pass
CO2 NDIR (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
0.02
0.07
Pass
1.01
0.99-1.01
Pass
0.04
0.15
Pass
1.00
0.998
Pass
HC FID (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
-0.22
2.21
Pass
1.00
0.99-1.01
Pass
0.26
4.43
Pass
1.00
0.998
Pass
HC FID (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
-1.11
2.21
Pass
1.00
0.99-1.01
Pass
0.36
4.43
Pass
1.00
0.998
Pass
       Table 29 shows the PEMS 4 linearity check results for NO and NO2. PEMS 4 also failed
NO2 linearity during the initial audit and June, 2006 checks.  As with PEMS 3, Sensors elected to
take no corrective action,  and the Steering Committee  elected to continue testing despite the
audit failure. During Engine 2 testing, the NDUV was replaced due to measurement stability
problems that eventually prevented proper zero and span operations.  The new NDUV passed
NO linearity, but failed NO2 linearity.  Shortly after installing the new NDUV,  NO  and NO2
measurements again became noisy and erratic. The NDUV was replaced again by Sensors Inc.,
after which NO and NO2 passed the linearity check.
SwRI Report 03.12024.06
Page 54 of 371

-------
   TABLE 29. PEMS 4 1065 LINEARITY VERIFICATION SUMMARY CONTINUED
NO NDUV (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
2.54
4.42
Pass
0.99
0.99-1.01
Pass
1.93
8.84
Pass
1.00
0.998
Pass
NO NDUV (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
0.61
4.42
Pass
1.00
0.99-1.01
Pass
2.70
8.84
Pass
1.00
0.998
Pass
NO NDUV (New NDUV 09-14-06)
Measured
Linearity Criteria
Pass / Fail
0.36
4.41
Pass
1.00
0.99-1.01
Pass
2.81
8.83
Pass
1.00
0.998
Pass
N02 NDUV (New NDUV 09-25-06)
Measured
Linearity Criteria
Pass / Fail
3.55
4.41
Pass
0.99
0.99-1.01
Pass
2.30
8.83
Pass
1.00
0.998
Pass
N02 NDUV (Initial Audit)
Measured
Linearity Criteria
Pass / Fail
3.46
1.42
Fail
1.01
0.99-1.01
Pass
3.37
2.83
Fail
1.00
0.998
Pass
N02 NDUV (Check 06-05-06)
Measured
Linearity Criteria
Pass / Fail
4.60
1.29
Fail
0.99
0.99-1.01
Pass
2.60
2.58
Fail
1.00
0.998
Pass
N02 NDUV (New NDUV 09-14-06)
Measured
Linearity Criteria
Pass / Fail
-2.55
1.29
Fail
1.02
0.99-1.01
Fail
1.85
2.58
Pass
1.00
0.998
Pass
N02 NDUV (New NDUV 09-25-06)
Measured
Linearity Criteria
Pass / Fail
0.16
1.29
Pass
1.00
0.99-1.01
Pass
1.12
2.58
Pass
1.00
0.998
Pass
       Shown in Table 30  are  the  linearity results for PEMS 5.  No major  repairs were
performed on this unit during the program, subsequent to the initial audits.  During the initial
audit linearity checks, CO and CO2 measurement were unstable, so valid readings could not be
taken.  The NDIR was replaced by Sensors Inc.  PEMS 5 passed all linearity checks with the
exception of NO.  No action was taken to correct the NO linearity failure with PEMS 5, and the
Steering Committee elected to continue testing with the unit.  PEMS 5 was shipped to CE-CERT
and used during the in-use validation testing.  Upon return to SwRI,  analyzer  linearity was
rechecked.  Similar  to the initial  audit, NO failed the linearity check with high intercept.  CO2
also repeatedly failed the linearity test with low intercept.  Despite the linearity check failures,
PEMS 5 was used to perform the laboratory replay validation testing.
SwRI Report 03.12024.06
Page 55 of 371

-------
          TABLE 30. PEMS 5 1065 LINEARITY VERIFICATION SUMMARY
Verification Description
Intercept
Slope
SEE
r2
CO NDIR (Initial Audit with New NDIR 06-07-06)
Measured
Linearity Criteria
Pass / Fail
0.00
4.52
Pass
1.00
0.99-1.01
Pass
0.00
9.04
Pass
1.00
0.998
Pass
CO NDIR (Returned from Ce-Cert 01-03-07)
Measured
Linearity Criteria
Pass / Fail
0.00
4.52
Pass
1.00
0.99-1.01
Pass
0.00
9.04
Pass
1.00
0.998
Pass
C02 NDIR (Initial Audit with New NDIR 06-07-06)
Measured
Linearity Criteria
Pass / Fail
-0.06
0.07
Pass
1.01
0.99-1.01
Pass
0.05
0.15
Pass
1.00
0.998
Pass
CO2 NDIR (Returned from Ce-Cert 01-03-07)
Measured
Linearity Criteria
Pass / Fail
-0.11
0.07
Fail
1.01
0.99-1.01
Pass
0.06
0.15
Pass
1.00
0.998
Pass
HC FID (Initial Audit 06-07-06)
Measured
Linearity Criteria
Pass / Fail
-0.41
2.21
Pass
1.00
0.99-1.01
Pass
0.53
4.43
Pass
1.00
0.998
Pass
HC FID (Returned from Ce-Cert 01-03-07)
Measured
Linearity Criteria
Pass / Fail
-1.18
2.21
Pass
1.00
0.99-1.01
Pass
0.46
4.43
Pass
1.00
0.998
Pass
NO NDUV (Initial Audit 06-07-06)
Measured
Linearity Criteria
Pass / Fail
6.65
4.42
Fail
0.99
0.99-1.01
Pass
2.50
8.84
Pass
1.00
0.998
Pass
NO NDUV (Returned from Ce-Cert 01-03-07)
Measured
Linearity Criteria
Pass / Fail
5.00
4.47
Fail
1.00
0.99-1.01
Pass
2.63
8.94
Pass
1.00
0.998
Pass
NO2 NDUV (Initial Audit 06-07-06)
Measured
Linearity Criteria
Pass / Fail
0.35
1.29
Pass
1.01
0.99-1.01
Pass
0.96
2.58
Pass
1.00
0.998
Pass
NO2 NDUV (Returned from Ce-Cert 01-03-07)
Measured
Linearity Criteria
Pass / Fail
0.81
1.23
Pass
1.01
0.99-1.01
Pass
1.45
2.45
Pass
1.00
0.998
Pass
       Table 31 shows the summarized linearity results for PEMS 6.  During the initial audit of
this unit, PEMS 6 failed NC>2 linearity.  As with other similar failures, no corrective action was
taken by Sensors Inc., and the Steering Committee elected to proceed with testing using this unit.
During Engine 3 steady-state testing, PEMS 6 NOX measurements were biased low versus the
laboratory measurements.  To determine the cause of the bias, additional NO and NO2 linearity
checks were performed, with both NO and NO2 passing the linearity tests. An addition linearity
SwRI Report 03.12024.06
Page 56 of 371

-------
check is also shown for the FID following a repair which was conducted following a failure
during environmental baseline testing.
          TABLE 31. PEMS 6 1065 LINEARITY VERIFICATION SUMMARY
Verification Description
Intercept
Slope
SEE
r2
CO NDIR (Initial Audit 06-07-06)
Measured
Linearity Criteria
Pass / Fail
0.00
4.52
Pass
1.00
0.99-1.01
Pass
0.00
9.04
Pass
1.00
0.998
Pass
CO2 NDIR (Initial Audit 06-07-06)
Measured
Linearity Criteria
Pass / Fail
0.07
0.07
Pass
1.00
0.99-1.01
Pass
0.04
0.15
Pass
1.00
0.998
Pass
HC FID (Initial Audit 06-07-06)
Measured
Linearity Criteria
Pass / Fail
-0.05
2.21
Pass
1.00
0.99-1.01
Pass
0.52
4.43
Pass
1.00
0.998
Pass
HC FID (Repaired 08-14-06)
Measured
Linearity Criteria
Pass / Fail
-1.09
2.21
Pass
1.00
0.99-1.01
Pass
0.37
4.43
Pass
1.00
0.998
Pass
NO NDUV (Initial Audit 06-07-06)
Measured
Linearity Criteria
Pass / Fail
1.15
4.42
Pass
1.00
0.99-1.01
Pass
1.23
8.84
Pass
1.00
0.998
Pass
NO NDUV (Check 11-22-06)
Measured
Linearity Criteria
Pass / Fail
0.90
4.47
Pass
1.00
0.99-1.01
Pass
2.46
8.94
Pass
1.00
0.998
Pass
N02 NDUV (Initial Audit 06-07-06)
Measured
Linearity Criteria
Pass / Fail
1.95
1.29
Fail
1.01
0.99-1.01
Pass
1.68
2.58
Pass
1.00
0.998
Pass
N02 NDUV (Check 11-22-06)
Measured
Linearity Criteria
Pass / Fail
0.15
1.29
Pass
1.01
0.99-1.01
Pass
0.51
2.58
Pass
1.00
0.998
Pass
       Linearity results for PEMS 7 are shown in Table 32. PEMS 7 arrived at SwRI late in the
project, and only initial audit results, performed in October of 2006, are shown for this unit.
PEMS 7 passed the linearity checks with all analyzers.  It was understood that PEMS 7 was a
new unit that had been only recently received by EPA prior to shipment to SwRI.
SwRI Report 03.12024.06
Page 57 of 371

-------
         TABLE 32. PEMS 7 1065 LINEARITY VERIFICATION SUMMARY
Verification Description
Intercept
Slope
SEE
r2
CO NDIR (Initial Audit 10-21-06)
Measured
Linearity Criteria
Pass / Fail
0.00
4.52
Pass
1.00
0.99-1.01
Pass
0.00
9.04
Pass
1.00
0.998
Pass
C02 NDIR (Initial Audit 10-21-06)
Measured
Linearity Criteria
Pass / Fail
0.02
0.07
Pass
1.00
0.99-1.01
Pass
0.05
0.15
Pass
1.00
0.998
Pass
HC FID (Initial Audit 10-21-06)
Measured
Linearity Criteria
Pass / Fail
-0.30
2.21
Pass
1.00
0.99-1.01
Pass
0.56
4.43
Pass
1.00
0.998
Pass
NO NDUV (Initial Audit 10-21-06)
Measured
Linearity Criteria
Pass / Fail
3.45
4.41
Pass
1.00
0.99-1.01
Pass
0.89
8.83
Pass
1.00
0.998
Pass
N02 NDUV (Initial Audit 10-21-06)
Measured
Linearity Criteria
Pass / Fail
0.24
1.29
Pass
1.00
0.99-1.01
Pass
0.51
2.58
Pass
1.00
0.998
Pass
      Table 33 shows the linearity test results for the Horiba OBS-2200 PEMS unit. Mono-
blend span gases were used to check linearity. The gas concentrations used for the check were
14.67 % CO2, 904 ppm CO, 443 ppmC HC, and 892 ppm NOX. The OBS-2200 passed the 1065
linearity criteria for all gaseous emissions.
    TABLE 33. HORIBA OBS-2200 1065 LINEARITY VERIFICATION SUMMARY
Verification Description
Intercept
Slope
SEE
r2
CO NDIR
Measured
Linearity Criteria
Pass / Fail
0.00
4.52
Pass
1.00
0.99-1.01
Pass
0.00
9.04
Pass
1.00
0.998
Pass
C02 NDIR
Measured
Linearity Criteria
Pass / Fail
0.00
0.07
Pass
1.00
0.99-1.01
Pass
0.01
0.15
Pass
1.00
0.998
Pass
HCFID
Measured
Linearity Criteria
Pass / Fail
0.05
2.21
Pass
1.00
0.99-1.01
Pass
0.49
4.43
Pass
1.00
0.998
Pass
NOX CLD
Measured
Linearity Criteria
Pass / Fail
-1.22
4.49
Pass
0.99
0.99-1.01
Pass
2.70
8.98
Pass
1.00
0.998
Pass
SwRI Report 03.12024.06
Page 58 of 371

-------
3.4    1065 Gas Analyzer Verifications

       The results of gas analyzer performance checks not related to linearity verifications are
given in this section.  In general, SwRI performed the analyzer audits as detailed in CFR Part
1065  Subpart D.  A  step-by-step description of each analyzer verification process as well as
discussion of the test results are presented in the following sections. In summary, all laboratory
instruments passed the verification tests.  Similarly, the Horiba OBS-2200 PEMS unit passed all
verifications tests.  PEMS 2 and PEMS 4 failed the non-stoichiometric raw exhaust FID  O2
interference verifications. No corrective action was taken in regard to the FID O2 interference
test.  All  SEMTECH-DS PEMS units initially failed the chiller NO2 penetration check.  A
system upgrade  was implemented by Sensors Inc. after which all units passed the check.  A
detailed account of the chiller NO2 penetration failure and subsequent actions  are disused in the
chiller NO2 penetration  section.   The PEMS  units  passed all  other  1065  verification tests
discussed below.

       Generally, each of these verifications was performed once during the program, unless a
major instrument repair  warranted  an additional check..   The major  exception was the NO2
chiller penetration check. The NO2 penetration test was repeated after an upgrade designed to
address  the initial failure  of all the PEMS, which is discussed in more detail below. The audit of
PEMS 5,  6,  and 7 occurred after the implementation  of the NO2 chiller penetration  system
upgrade, therefore a chiller penetration failure was not documented for these units.

       Several  1065  analyzer verification tests required the use of humidified and  blended
gasses.  SwRI therefore constructed a gas conditioning and blending cart, pictured in Figure 2, to
perform the PEMS and laboratory audits. Consisting of a heated bubbler, two flow meters, an
overflow system, a Vaisala dew point instrument, several thermocouples, heated rap, and various
valves and stainless steel connections, the humidification rig can control the dew point of a gas
blend up to 50 °C. The cart uses both a wet gas port and a dry gas port. Therefore, the cart can
overflow dry gas, wet gas, or a blend of a wet gas and dry gas. The gas blending feature is useful
when humidifying gases that are soluble in water, such as NO2. Many of the performance checks
involving humidified gases require that the gas be overflowed to the entry of the heated sample
line of either the laboratory emission bench or PEMS unit. The overflow procedure was done to
verify proper operation of sample handling and conditioning systems.  Therefore, the cart was
built with the capacity to overflow gas to as many as two emission benches and/or PEMS units,
allowing for direct performance comparisons between units. This capability proved very useful
in diagnosing problems throughout the program.
SwRI Report 03.12024.06                 Page 59 of 371

-------
        FIGURE 17. SWRI GAS HUMIDIFICATION AND BLENDING CART

      The results of the various 1065 performance checks for the laboratory analyzer benches
are summarize in Table 34 and Table 35. Results for the PEMS performance checks during the
initial audits are given in Table 36 through Table 43.
  TABLE 34. DILUTE MEXA 7200D AND HORIBA NMHC BENCH 1065 ANALYZER
                           VERIFICATION SUMMARY
Verification Description
1065.350 H2O interference for CO2 NDIR [%]
1065.355 H2O and CO2 interference for CO NDIR [ppm]
1065.360 FID optimization (methane response)
1065.370 CO2 and H2O quench verification for NOX OLD [%]
1065.378 NO2-to-NO converter conversion [%]
1065.365 Nonmethane cutter penetration fractions [%]
Meas.
0.00%
0.4
1.12
-0.21%
97.3%
1 .8%
Verification
± 0.01%
± 4.0
N/A
± 2.00%
> 95%
< 2.0%
P/F
Pass
Pass
N/A
Pass
Pass
Pass
SwRI Report 03.12024.06
Page 60 of 371

-------
   TABLE 35. RAW MEXA 7200D AND HORIBA NMHC BENCH 1065 ANALYZER
                       VERIFICATION SUMMARY
Verification Description
1065.350 H2O interference for CO2 NDIR [%]
1065.355 H2O and CO2 interference for CO NDIR [ppm]
1065.360 FID optimization (methane response)
1065.362 Non-stoichiometric FID O2 interference [%]
1065.370 CO2 and H2O quench verification for NOX CLD [%]
1065.378 NO2-to-NO converter conversion [%]
1065.362 Non-stoichiometric CH4 FID O2 interference [%]
1065.365 Nonmethane cutter penetration fractions [%]
Meas.
0.00%
1.4
1.15
-0.7%
-0.5%
96.7%
0.5%
1 .7%
Verification
± 0.07%
± 48.7
N/A
± 1 .5%
± 2.0%
> 95%
± 1 .5%
< 2.0%
P/F
Pass
Pass
N/A
Pass
Pass
Pass
Pass
Pass
        TABLE 36. PEMS 1 1065 ANALYZER VERIFICATION SUMMARY
Verification Description
1065.350 H2O interference for CO2 NDIR [%]
1065.355 H2O and CO2 interference for CO NDIR [ppm]
1065.362 Non-stoichiometric FID O2 interference [%]
1065.372 HC and H2O interference for NOX NDUV [ppm]
1065.376 Chiller NO2 Penetration [%]
1065.376 Chiller NO2 Penetration [%] (Post Retrofit) (06-10-06)
Meas.
0.00%
21.8
-0.8%
-0.6
90.5%
100.7%
Verification
± 0.07%
± 48.7
± 1.5%
± 4.0
> 95%
> 95%
P/F
Pass
Pass
Pass
Pass
Fail
Pass
        TABLE 37. PEMS 2 1065 ANALYZER VERIFICATION SUMMARY
Verification Description
1065.350 H2O interference forCO2 NDIR [%]
1065.355 H2O and CO2 interference for CO NDIR [ppm]
1065.362 Non-stoichiometric FID O2 interference [%]
1065.372 HC and H2O interference for NOX NDUV [ppm]
1065.376 Chiller NO2 Penetration [%]
1065.376 Chiller NO2 Penetration [%] (Post Retrofit) (06-10-06)
Meas.
0.00%
21.8
4.2%
0.6
89.0%
95.6%
Verification
± 0.07%
± 48.7
± 1 .5%
± 4.0
± 95%
> 95%
P/F
Pass
Pass
Fail
Pass
Fail
Pass
        TABLE 38. PEMS 3 1065 ANALYZER VERIFICATION SUMMARY
Verification Description
1065.350 H2O interference forCO2 NDIR [%]
1065.355 H2O and CO2 interference for CO NDIR [ppm]
1065.362 Non-stoichiometric FID O2 interference [%]
1065.372 HC and H2O interference for NOX NDUV [ppm]
1065.376 Chiller NO2 Penetration [%]
1065.376 Chiller NO2 Penetration [%] (Post Retrofit) (06-10-06)
Meas.
0.00%
10.9
-0.1%
0.0
90.1%
100.6%
Verification
± 0.07%
± 48.7
± 1 .5%
± 4.0
± 95%
> 95%
P/F
Pass
Pass
Pass
Pass
Fail
Pass
SwRI Report 03.12024.06
Page 61 of 371

-------
        TABLE 39. PEMS 4 1065 ANALYZER VERIFICATION SUMMARY
Verification Description
1065.350 H2O interference forCO2 NDIR [%]
1065.355 H2O and CO2 interference for CO NDIR [ppm]
1065.362 Non-stoichiometric FID O2 interference [%]
1065.372 HC and H2O interference for NOX NDUV [ppm]
1065.376 Chiller NO2 Penetration [%]
1065.376 Chiller NO2 Penetration [%] (Post Retrofit) (06-10-06)
1065.376 Chiller NO2 Penetration [%] (New NDUV) (09-25-06)
1065.376 Chiller NO2 Penetration [%] (RH Sensor) (09-28-06)
Meas.
0.01%
21.8
-0.5%
0.0
89.4%
101.2%
100.0%
98.1%
Verification
± 0.07%
± 48.7
± 1 .5%
± 4.0
± 95%
> 95%
> 95%
> 95%
P/F
Pass
Pass
Pass
Pass
Fail
Pass
Pass
Pass
        TABLE 40. PEMS 5 1065 ANALYZER VERIFICATION SUMMARY
Verification Description
1065.350 H2O interference forCO2 NDIR [%]
1065.355 H2O and CO2 interference for CO NDIR [ppm]
1065.362 Non-stoichiometric FID O2 interference [%]
1065.372 HC and H2O interference for NOX NDUV [ppm]
1065.376 Chiller NO2 Penetration [%] (Post Retrofit) (06-10-06)
Meas.
0.00%
21.8
-0.1%
0.0
102.4%
Verification
± 0.07%
± 48.7
± 1 .5%
± 4.0
> 95%
P/F
Pass
Pass
Pass
Pass
Pass
        TABLE 41. PEMS 6 1065 ANALYZER VERIFICATION SUMMARY
Verification Description
1065.350 H2O interference forCO2 NDIR [%]
1065.355 H2O and CO2 interference for CO NDIR [ppm]
1065.362 Non-stoichiometric FID O2 interference [%]
1065.372 HC and H2O interference for NOX NDUV [ppm]
1065.376 Chiller NO2 Penetration [%] (Post Retrofit) (06-10-06)
1065.376 Chiller NO2 Penetration [%] (RH Sensor) (9-15-06)
1065.376 Chiller NO2 Penetration [%] (RH Sensor) (9-21-06)
1065.376 Chiller NO2 Penetration [%] (RH Sensor) (9-28-06)
1065.376 Chiller NO2 Penetration [%] (Check) (11-27-06)
Meas.
0.00%
27.3
2.1%
0.0
95.7%
96.2%
100.0%
99.6%
99.0%
Verification
± 0.07%
± 48.7
± 1 .5%
± 4.0
> 95%
> 95%
> 95%
> 95%
> 95%
P/F
Pass
Pass
Fail
Pass
Pass
Pass
Pass
Pass
Pass
        TABLE 42. PEMS 7 1065 ANALYZER VERIFICATION SUMMARY
Verification Description
1065.350 H2O interference forCO2 NDIR [%]
1065.355 H2O and CO2 interference for CO NDIR [ppm]
1065.362 Non-stoichiometric FID O2 interference [%]
1065.372 HC and H2O interference for NOX NDUV [ppm]
1065.376 Chiller NO2 Penetration [%] (Post Retrofit) (10-24-06)
Meas.
0.00%
11.0
1 .4%
1.6
96.8%
Verification
± 0.07%
± 48.7
± 1.5%
± 4.0
> 95%
P/F
Pass
Pass
Pass
Pass
Pass
SwRI Report 03.12024.06
Page 62 of 371

-------
    TABLE 43. HORIBA OBS-2200 1065 ANALYZER VERIFICATION SUMMARY
Verification Description
1065.350 H20 interference for CO2 NDIR [%]
1065.355 H2O and CO2 interference for CO NDIR [ppm]
1065.362 Non-stoichiometric FID O2 interference [%]
1065.370 CO2 and H2O quench verification for NOx CLD [%]
1065.378 NO2-to-NO converter conversion [%]
Meas.
0.05%
4.8
-0.3%
-1 .4%
98.9%
Verification
± 0.07%
± 48.7
± 1 .5%
± 2.0%
± 95.0%
P/F
Pass
Pass
Pass
Pass
Pass
3.4.1   1065.350H2O Interference for CO2 NDIR

       The CC>2 NDIR water interference check was performed on each of the PEMS units as
well as the laboratory dilute and raw analyzers.  This check was performed to characterize CO2
interference caused by water when using a NDIR analyzer. The PEMS and laboratory analyzers
used sample dryers upstream of the NDIR analyzer and all passed this 1065 check.

       To perform this verification, all analyzers were first zeroed and spanned as they would be
prior to an emissions test.  Using the humidification rig, humidified zero air was overflowed to
the sample line of the  analyzer.  While maintaining the dew point of the zero air at 25 °C, the
response of the CC>2 NDIR analyzer was recorded.  This recorded value was compared to ±2 %
of the lowest flow-weighted mean CO2 concentration  expected during  testing.  The raw
verification value of ±0.07 % and dilute verification value of ±0.01 % were calculated using the
Detroit Diesel Series 60 engine and FTP engine cycle.  Using the lowest flow-weighted CC>2
concentration provided the most stringent test, therefore verifying the analyzer  performance
during all emissions tests.

3.4.2   1065.355 H2O and CO2 Interference for CO NDIR

       The CO NDIR water and CO2 interference  check was performed on each of the PEMS
units  as well  as  the  laboratory dilute and raw  analyzers.   This check  was performed  to
characterize CO interference caused by water and CO2 when using a NDIR analyzer. The PEMS
and laboratory analyzers all passed this 1065 check.

       To perform this verification, all analyzers were first zeroed and spanned as they would be
prior  to an  emissions test.   Using the  humidification  rig,  humidified  CO2  span  gas was
overflowed to the  sample line of the analyzer. While maintaining the dew point of the CO2 span
gas at 25 °C, the response of the CO NDIR analyzer was recorded. The recorded CO value was
multiplied by the highest  flow-weighted mean CO2 concentration expected during testing, then
divided by the CO2 span gas  concentration. For this check,  the highest flow-weighted CO2
concentration provided the most stringent test.  The CO2 value of 8 % was calculated using the
Detroit Diesel Series 60 engine and RMC engine cycle.   The  corrected CO concentration was
compared  to ±2 % of the  flow-weighted mean CO concentration expected at the standard. The
raw CO verification value of 48.7 ppm and dilute verification value of 4 ppm were calculated
using the DDC engine over a FTP heavy-duty transient cycle.
SwRI Report 03.12024.06
Page 63 of 371

-------
3.4.3   1065.3 60 FID Optimization Methane Response

       The methane response factors were determined for the laboratory dilute and raw FID
analyzers.  The  FID analyzers  were first zeroed and spanned as they would be prior to an
emissions test.  Using a gas divider and methane span gas, the FID response to methane was
characterized over 10 evenly distributed points  from near zero  to  span concentration.   The
methane response factor was calculated by dividing the recorded FID response by the actual
methane concentration. The mean value of the 10 methane response  factors was calculated.  A
check was then performed to insure each of the 10 response factors was within  ±2 % of the
mean.

3.4.4   1065.362 Non-stoichiometric Raw FID #2 Interference

       The O2 raw FID interference check was performed on each of the PEMS units as well as
the raw  laboratory  THC FID and raw laboratory NMHC FID  analyzers.   This  check was
performed to characterize C>2 interference when using a FID analyzer to measure raw exhaust
from a non-stoichiometric engine. PEMS 2 and PEMS 4 failed this 1065 check.

       The first  step performed during this test was to zero and span the analyzers. The FID
analyzers were then spanned using a  propane span gas with balance nitrogen.  Using a gas
divider, the propane in nitrogen span gas was cut with 20 % oxygen  and sampled with the FID
analyzer.   The  FID response  to  the  divided span  gas was compared to the  actual  THC
concentration. For all analyzers except PEMS  2  and PEMS 4, the measurement concentration
was within 1.5 % of the actual  concentration and therefore passed the interference check. By
spanning the  analyzer with propane in nitrogen, and checking the analyzer with 20  % oxygen,
this verification insures the 62 interference is acceptable during typical diesel engine operation.

       Per the Steering Committee's decision, no action was taken to remedy the C>2 raw FID
interference  check failure of PEMS 2  and PEMS 4.   The laboratory raw THC FID analyzer
initially failed this check.  After the FID was re-optimized, the instrument passed the verification
test.

3.4.5   1065.365 Nonmethane Cutter Penetration Fractions

       The nonmethane cutter (NMC)  penetration verification was performed on the laboratory
raw and dilute methane analyzer benches, each  of which employed a NMC. Both systems
passed the penetration check.

       The instruments were spanned  through the NMC using methane during testing and for
this performance check, therefore  the  methane penetration fraction was  set to 1.0.  For the
verification,  the  instruments were zeroed and  spanned,  after  which  ethane span gas  was
introduced to the bench.  The concentration of ethane  (in ppmC) was near the methane span
value used during the check.  The response of the NMC FID to ethane span gas was recorded.
The recorded value was divided by the ethane  span gas concentration on a ppmC basis.  This
fraction was less  than 2 % and therefore passed the nonmethane cutter  penetration check.
SwRI Report 03.12024.06                  Page 64 of 371

-------
       The dilute NMC bench initially failed the cutter penetration check.  The temperature of
the NMC cutter oven was increased until the bench passed the penetration verification test.

3.4.6  1065.370 CO2 andH2O Quench Verification for NOx CLD

       The NOX CLD water and CC>2 quench check was performed on the laboratory raw and
dilute analyzers. Both analyzers passed the quench check.

       The CLD NOX  analyzers were first zeroed and spanned  as they would be prior to  an
emissions test.  The benches were then set to measure NO  rather  than total NOX.  The NO span
gas was then sampled and the mean NO concentration was recorded.  The NO span gas was then
humidified  and the mean dry NO concentration as well as  the  water content of the gas was
recorded.   The water quench  was calculated by taking the difference between the dry and
humidified  span gas measurements and correcting this value using the actual water content of the
span gas and the maximum water content  expected during testing.   The  maximum  water
concentration expected during testing was set to 12 % for the raw analyzer and 3.5  % for the
dilute analyzer. Because both raw and dilute CLD analyzers were operated in a dry mode during
this program, both analyzers showed negligible water quench, indicating that the drying systems
in the benches were able to successfully remove the water from the sample.

       Using a gas divider, the CLD CO2 quench was determined by measuring a blend of 50 %
NO  span gas and 50  %  nitrogen.  Next, 50 % NO span gas and 50 % CO2  span gas was
measured.  The CO2 quench was calculated by taking the difference between the 50 % nitrogen
blend and the 50 % CO2  span gas blend.  The quench value was then corrected using the CO2
concentration recorded during  the test and the maximum CO2 concentration expected  during
testing.  The maximum CO2 concentration expected during testing was set to 10 % for the raw
analyzer and 2.2 % for the dilute analyzer.   The combined water and CO2  quench for both
analyzers was less than 2 %, therefore passing the CLD quench verification test.

3.4.7  1065.372 HC andH2O Interference for NOX NDUV

       The NOX NDUV water and HC interference check was performed on each of the PEMS
units. All PEMS devices passed this interference verification test.

       The PEMS were first zeroed and spanned as  they would be prior to an emissions test.
Next, a blend of humidified zero air and dry propane span gas were overflowed to the sample
line of the PEMS.  The dew point of the gas mixture was maintained at 45 °C during these tests.
Allowing time for stabilization, the NO, NO2, and HC concentration values were recorded. The
NO and NO2 concentrations were added, and the resulting response was then adjusted to  the
level to the flow-weighted  mean HC concentration expected at  the standard.  The  mean HC
concentration of 51 ppm was calculated using the DDC Series 60 engine over a FTP transient
cycle.  The verification concentration was calculated as ±2 % of the flow-weighted mean NOX
concentration expected at the standard.  The mean NOX concentration of 198 ppm was  calculated
using the DDC Series 60 engine over a FTP transient cycle.  All PEMS showed little water and
HC interference for the NO and NO2 measurements, and easily passed the  1065 interference
verification test.
SwRI Report 03.12024.06                  Page 65 of 371

-------
3.4.8   1065.376 Chiller NO2 Penetration

       The SEMTECH-DS PEMS uses a chiller to dry the exhaust sample prior to the NDUV
detector, but does not use a NO2-to-NO converter.  Therefore, the chiller NO2 penetration check
was performed.   Initially,  SwRI performed the chiller penetration check  using  a procedure
similar to that performed by Sensors Inc. After the PEMS were zeroed and spanned, wet zero air
with a dew point of approximately 50 °C was overflowed to the sample line and sampled for 15
to 20  minutes. Next, dry NO2 span gas was overflowed to the sample line.  Allowing time for
stabilization,  the NO2 concentration was recorded and compared to the  NO2 span gas  bottle
concentration. The units initially read approximately 90 % of the NO2 span concentration, and
failed the verification criteria of 95  %  penetration.   Although  this initial  procedure was
successful in revealing problems with NO2 penetration, the method was less than ideal.  For
example, following the  switch from humidified zero air to dry NO2 span gas,  the sampling
system  of the PEMS is  continually drying.  Although this drying process is slow, the NO2
concentration does rise over time, making a stable measurement difficult to achieve.

       As a result, a revised method was devised to perform the chiller NO2 penetration check,
required the use of the 1065 compliant laboratory CLD NOX analyzer as a reference.  Humidified
zero air was blended with dry NO2 span gas.  The blend was adjusted to maintain a mixture dew
point  of approximately  45 °C with  a  NO2 concentration near the span concentration.  The
humidified NO2 mixture  was  then overflowed simultaneously to both the PEMS and the
laboratory CLD  NOx analyzer.  The CLD  NOX  concentration was  used as the  reference in
calculating the NO2 penetration.  It was felt that the CLD could serve as an appropriate reference
value  for this check, due to the fact that the laboratory CLDs did not show significant water
quench (since they are run dry). In addition, the NO2-to-NO conversion efficiency was in excess
of 97% at concentrations well above those used for the chiller NO2 penetration check.

       This method more accurately simulates in-use measurement, because  the sample is
continuously  humidified.   The  CLD-based penetration  check  method  generated  chiller
penetration results similar to method used by  Sensors Inc., but resulted in more  stable and
accurate values.

       As discussed earlier, all of the PEMS failed  the chiller NO2 penetration check initially.
Additional  penetration checks were performed at varying concentrations of NO2.   These
experiments revealed a trend of increasing NO2 loss with increased NO2 concentration.  This
trend is illustrated in Figure 18. It should be noted that the NO2 span value used for testing was
near 300 ppm.
SwRI Report 03.12024.06                 Page 66 of 371

-------
     98%
     96%
     94%
     92%
     90%
     88%
     86%
                   50
                              100         150        200         250
                                     NO2 Span Concentration [ppm]
                                                                         300
                                                                                    350
            FIGURE 18. NO2 CHILLER PENETRATION AUDIT RESULTS

       These audit results were presented to the Steering Committee at the March 14th meeting
in Ann Arbor, MI.  Although there was concern about these results, the Steering Committee
elected to run steady-state tests on Engine 1, to examine whether the performance check results
would translate into an observed negative bias in the test results.

       The initial steady-state results for Engine 1 were presented to the Steering Committee at
the April 13, 2006 meeting in San Antonio.  Figure 19 shows a summary the original steady-state
NOX concentration pooled delta data from the Detroit Diesel Series 60 engine.  The data shows a
definite negative NOX bias for the PEMS at higher concentrations.   This was due in part to the
higher fraction of NC>2 in the overall NOX which occur due to the use of a catalyzed DPF in the
exhaust stream.
SwRI Report 03.12024.06
Page 67 of 371

-------
                             I PEMS 5th % A PEMS 50th % • PEMS 95th % • Points
      10
      5
   Q.
   3  -5
   c
   o
   "a
   ~ -10
   HI
   o
   c
   o
   O -15
     -20
     -25
     -30
                              .   »40
                              9   15
        '24
       --3--
                                        33
               160
                        180
                                200      220      240      260
                                                                  280
                                                                          300      3:
                               Lab Reference Mean NOx Concentration (ppm)
   FIGURE 19. NOX CONCENTRATION POOLED DELTA DATA FROM ENGINE 1
   STEADY-STATE REPEAT TESTING PRIOR TO CHILLER NO2 PENETRATION
                             RETROFIT INSTALLATION
       The Steering Committee deemed the low NOX bias unacceptable for continued testing.
Sensors Inc. was  asked to develop an upgrade  to correct the issue in as  timely a manner  as
possible. However, it was also stipulated that the upgrade had to be acceptable as a real-world
solution, and that the upgrade could be applied to all existing SEMTECH DS units.

       Following the direction of the Steering  Committee, Sensors Inc.  developed a system
upgrade to estimate the chiller NC>2 loss and numerically  correct the  NC>2 measurement.
Upgrades were completed on all PEMS used during the measurement allowance program by the
end of May, 2006. The upgrade package includes a drain manifold relative  humidity sensor and
software upgrade. After Sensors Inc. implemented the upgrade for the SEMTECH-DS, all units
passed the NC>2 penetration check. Details of the upgrade are not included in this report for
reasons of confidentiality.

       After  implementation  of the  NC>2 chiller penetration  system upgrade, the Steering
Committee  elected to repeat  the  steady-state testing for  Engine 1.  While these results are
summarized later in greater detail,  a summary is given in Figure 20 to illustrate the effect  of the
upgrade on the data. The figure shows the pooled NOX delta data with upgraded SEMTECH-DS
units. The negative bias of the original data set was replaced with pooled errors showing a slight
positive bias.
SwRI Report 03.12024.06
Page 68 of 371

-------
                             I PEMS 5th % A PEMS 50th % • PEMS 95th % • Points
      30
      25
   20

a.
3  15 H
c
o
"a
r  10 H
   c
   o
   o
      5
      -5
     -10

.40
9__
                         15
                                          2Y
                                                             33
                         .
                160
                        180      200
                                         220
                                                 240
                                                         260
                                                                  280
                                                                          300
                               Lab Reference Mean NOx Concentration (ppm)
   FIGURE 20. NOX CONCENTRATION POOLED DELTA DATA FROM ENGINE 1
     STEADY-STATE REPEAT TESTING AFTER CHILLER NO2 PENETRATION
                             RETROFIT INSTALLATION

       The NC>2 penetration check failure and subsequent PEMS upgrade resulted in a schedule
delay of more than two months in the execution of the program.

       During dynamometer testing on Engine 2, PEMS 4 and PEMS 6 began reporting several
faults stating the drain manifold relative humidity  sensor was not responding.  To remedy this
problem, several drain manifold humidity sensors were replaced, after which the NC>2 penetration
check was repeated.  Sensors Inc. linked  the frequent failure  of the drain relative  humidity
sensors to drain manifolds that were allowing exhaust gas to leak past the sensor.  The leaking
exhaust gas carried liquid water past and onto the humidity sensors, causing the sensors to fail.
A simple leak  check was used to screen  for properly sealed  new  RH  sensors, which  were
installed in the PEMS.  The PEMS passed all penetration checks after the new sensors and drain
manifolds were installed.

3.4.9  1 065. 3 78 NO2-to-NO Converter Conversion

       The NO2-to-NO converter conversion verification was performed on the laboratory raw
and dilute benches.  This check was performed using the automated Horiba bench software and
Horiba GDC-703  gas divider, which is also capable  of performing the NOX converter check.
Both raw and dilute NCVto-NO converters had conversion efficiencies greater than 95 %, and
therefore passed this verification test.
SwRI Report 03.12024.06
                                 Page 69 of 371

-------
3.5    1065 Exhaust Flow Meter Linearity Verification

       The Sensors Inc. electronic flow meters (EFM) and Horiba flow meter were checked for
flow linearity using a flow calibration  stand at SwRI.  This flow stand incorporates  a  set of
reference laminar flow elements  (LFEs), which are regularly sent for  verification of NIST
traceability at CEESI.  The SwRI flow stand is pictured in Figure 21. The flow stand uses a
positive displacement blower to pull air through the stand, therefore, the reference LFE and EFM
are under a slight negative pressure during testing.  The reference meters are downstream of the
meter that is being calibrated.  In the case of the Sensors Inc. EFMs, a length of straight pipe
matching the diameter  of the EFM was installed upstream  of the EFM.  The  flow stand
incorporates long lengths of straight pipe, well in excess of 10 diameters, between the two flow
meters,  as well as  downstream of the reference meter.  The  stand is designed in this manner
because most  calibrations  at  SwRI focus on  intake  air measurement.  Several manually
controlled flow restriction devices, located far downstream of the reference meter, are used to set
the desired flow rates during the linearity check. High precision mercury manometers  are read
manually to record the LFE differential and inlet pressure, while a thermocouple is used to
measure the LFE inlet temperature.

       The SwRI flow  stand was also used to calibrate the intake air LFEs used to calculate the
laboratory reference raw exhaust flow.  The raw  exhaust flow rate was also checked during
testing by calculating a carbon balance fuel flow, using raw gaseous measurements and the raw
exhaust mass rate, and  comparing to the measured fuel flow mass rate. During all steady-state
testing, the raw carbon balance error was generally less than 2 %.
   FIGURE 21. SWRI LFE FLOW STAND MANOMETERS AND REFERENCE LFES
SwRI Report 03.12024.06
Page 70 of 371

-------
       Sensors Inc. uses a similar LFE-based flow stand to calibrate their electronic flow meters.
However, the Sensors flow stand uses a blower to push air through the reference flow meter and
then through the test flow meter which  is located  downstream and is then vented directly to
atmosphere. Therefore, the flow meters calibrated at Sensors, are under slight positive pressure.
According to Sensors, the stand was designed in order to accurately  simulate field conditions of
an exhaust tailpipe.  This discrepancy between the SwRI and Sensors Inc. flow stands may have
been a factor contributing to increased exhaust flow errors during steady-state testing on Engines
2 and 3, which used the 4-inch and 3-inch flow meters respectively.  However, this could not be
verified, and the issue did not manifest itself with the 5-inch flow meters.

       At SwRI,  the Sensors Inc.  and Horiba flow meters were mounted  inline  with the
reference LFE as shown in Figure 22.  A straight pipe, with length exceeding 10 diameters, was
connected to the inlet of the flow meters.  The EFM flow was recorded using the Sensors Inc.
software.  Data markers with the Sensors Inc. post  processor software were used to average at
least 30 seconds of data at each flow rate.
   FIGURE 22. SENSORS INC. EFM MOUNTED ON THE SWRI LFE FLOW STAND

       The collected flow data was processed using the 1065 linearity verifications for a raw
exhaust measurement system.  The sections below describe the calibration events and linearity
results for the 5-inch, 4-inch, and 3-inch flow meters.

3.5.1   Five-Inch Exhaust Flow Meter Linearity

       As shown in Table 44, two 5-inch Sensors Inc. flow meters passed the linearity check,
while one 5-inch flow meter repeatedly failed the check.  Per the Steering Committee's decision,
the failed 5-inch EFM was sent to Sensors Inc. where it was recalibrated using the Sensors Inc.
SwRI Report 03.12024.06
Page 71 of 371

-------
flow stand. The 5-inch flow meter was then tested at SwRI where it passed the 1065 linearity
check.  These results appeared to indicate good agreement between the SwRI and Sensors flow
stands.
           TABLE 44. 5-INCH SENSORS INC. EFM LINEARITY RESULTS
Verification Description
Intercept
Slope
SEE
r2
H05-SE05 -Initial, Test 1
Measured
Linearity Criteria
Pass / Fail
5.07
9.42
Pass
0.96
0.98-1.02
Fail
1.93
18.85
Pass
1.00
0.99
Pass
H05-SE05- Initial, Test 2
Measured
Linearity Criteria
Pass/ Fail
6.45
9.61
Pass
0.96
0.98-1.02
Fail
4.08
19.22
Pass
1.00
0.99
Pass
H05-SE03- Initial
Measured
Linearity Criteria
Pass/ Fail
1.60
9.54
Pass
1.00
0.98-1.02
Pass
4.50
19.09
Pass
1.00
0.99
Pass
I05-SE05- Initial
Measured
Linearity Criteria
Pass/ Fail
4.93
9.44
Pass
1.01
0.98-1.02
Pass
4.87
18.87
Pass
1.00
0.99
Pass
H05-SE05 - Test 1 After Recalibration at Sensors
Measured
Linearity Criteria
Pass/ Fail
-1.69
9.28
Pass
1.01
0.98-1.02
Pass
1.60
18.57
Pass
1.00
0.99
Pass
       Table  45  shows the linearity results for the 5-inch Horiba exhaust flow meter.   The
Horiba meter, also based on pitot tube technology, showed excellent correlation with the SwRI
flow stand; easily passing the 1065 linearity criteria.

   TABLE 45. 5-INCH HORIBA EXHAUST FLOW METER LINEARITY RESULTS
Verification Description
Intercept
Slope
SEE
r2
Horiba 5-Inch Exhaust Flow Meter S/N: 050702G2
Measured
Linearity Criteria
Pass / Fail
0.81
9.58
Pass
1.00
0.98-1.02
Pass
6.45
19.17
Pass
1.00
0.99
Pass
       Figure 23 shows the pooled delta data for the Sensors Inc. 5-inch flow meters versus the
laboratory calculated exhaust flow for the Detroit Diesel Series 60 engine during steady-state
repeat testing.  Reporting median flow measurement deltas less than 2 % of point, the 5-inch
flow meters showed good agreement with the laboratory exhaust flow measurement.  The
Sensors Inc. 5-inch EFM is rated at flows as high as 1700 scfm, therefore we were testing the
meter only in its mid to lower range.
SwRI Report 03.12024.06
Page 72 of 371

-------
      30
      25
      20
   —  15
   -2-  10
      5
       1!
      -5

     -10

     -15
     -20
                             IPEMS 5th % A PEMS 50th % * PEMS 95th % • Points
             40
                        37
                           27
                                  24
                                                                33
                                                9-
               250
                        350
                                450
                                        550
                                                 650
                                                         750
                                                                 850
                                                                         950
                                                                                 1050
                                Lab Reference Exhaust Flowrate (scfm)
   FIGURE 23. 5-INCH SENSORS EFM EXHAUST FLOW POOLED DELTA DATA
                FROM ENGINE 1 STEADY-STATE REPEAT TESTING


       Because the Horiba OBS-2200 was delivered to SwRI late in the program, it was tested
only with Engine 3.  Per the recommendation of Horiba, the 5-inch flow meter was used during
testing with the International VT365 engine. Figure 24 shows pooled delta data for the Horiba 5-
inch flow meter versus the laboratory calculated exhaust flow during steady-state repeat testing.
The Horiba flow meter showed good correlation with the SwRI calculated exhaust flows, with
median deltas less than 0.5 % of point.
SwRI Report 03.12024.06
Page 73 of 371

-------
                             IPEMS 5th % A PEMS 50th % » PEMS 95th % « Points
     20

     15

     10

      5

      0
       1
     -5

     -10

     -15

     -20

     -25

     -30
40
     ^
       37
              35 30
                        22
                                           _2CL
                                                  -13-
                                                           -16-
      A  200
              A A
                      250
                                  300
                                               350
                                                            400
                                 Lab Reference Exhaust Flowrate (scfm)
  FIGURE 24. 5-INCH HORIBA EXHAUST FLOW METER POOLED DELTA DATA
                FROM ENGINE 3 STEADY-STATE REPEAT TESTING
3.5.2   Four-Inch Exhaust Flow Meter Linearity

       Table 46 shows the linearity test results for the 4-inch flow meters in chronological order.
During the initial 1065  linearity checks,  all of the  4-inch flow meters failed the verification
criteria with low slope values for the regression lines, with an averaging regression line slope of
0.94.  As a result, the 4-inch flow meters where sent to Sensors Inc. for recalibration.  After
recalibration at Sensors Inc., one 4-inch flow meter, serial  number H05-SE07, was returned to
SwRI to  re-check linearity.  The flow meter again failed the  linearity check with low slope, at
roughly 0.97 on average.  SwRI and Sensors performed a considerable number of diagnostic tests
and checks in  an attempt to  determine the cause of the apparent discrepancy between the two
flow stands. All tests with the SwRI LFE flow stand failed the linearity check with low slope.

       An attempt was made to check linearity in actual engine exhaust from Engine 1 (14L
DDC  Series 60), using the SwRI exhaust flow measurement (from the sum of intake air and fuel
flows) as the reference.  It should be noted that the 4-inch flow meter was smaller than those
normally used for this  engine.   The engine  check failed linearity, but  with a  high  slope.
However, this  was due in  part to large negative errors at the  low end of the flow range,  which
caused the intercept to also fail by a wide margin.

       The collection of 4-inch flow meter data was presented to the Steering  Committee at the
July 27,  2006  meeting in  Ann Arbor.  The decision was made to  recalibrate the 4-inch flow
meters using the SwRI flow  stand  data. SwRI ran 15-point curves as requested by Sensors Inc.
SwRI Report 03.12024.06
                          Page 74 of 371

-------
Using the SwRI data, Sensors Inc. supplied  new calibration  constants  for the meters.  As
expected, all 4-inch flow meters passed the 1065 linearity checks after they were calibrated and
checked using the SwRI flow stand.
           TABLE 46. 4-INCH SENSORS INC. EFM LINEARITY RESULTS
Verification Description
Intercept
Slope
SEE
r2
H05-SE07 -Initial, Test 1
Measured
Linearity Criteria
Pass / Fail
7.28
9.25
Pass
0.95
0.98-1.02
Fail
5.99
18.51
Pass
1.00
0.99
Pass
H05-SE07- Initial, Test 2
Measured
Linearity Criteria
Pass / Fail
8.77
9.18
Pass
0.95
0.98-1.02
Fail
6.93
18.36
Pass
1.00
0.99
Pass
I05-SE03 -Initial, Test 1
Measured
Linearity Criteria
Pass/ Fail
4.09
9.30
Pass
0.98
0.98-1.02
Fail
6.49
18.60
Pass
1.00
0.99
Pass
I05-SE03- Initial, Test 2
Measured
Linearity Criteria
Pass / Fail
3.06
9.24
Pass
0.96
0.98-1.02
Fail
6.61
18.49
Pass
1.00
0.99
Pass
I05-SE01 -Initial, Test 1
Measured
Linearity Criteria
Pass / Fail
21.47
9.26
Fail
0.92
0.98-1.02
Fail
14.56
18.52
Pass
1.00
0.99
Pass
H05-SE07 - Test 1 After Recalibration at Sensors
Measured
Linearity Criteria
Pass / Fail
-8.11
7.31
Fail
0.98
0.98-1.02
Fail
6.42
14.61
Pass
1.00
0.99
Pass
H05-SE07 - Test 3 After Recalibration (check on DDC Series 60 in exhaust)
Measured
Linearity Criteria
Pass / Fail
-25.74
10.26
Fail
1.03
0.98-1.02
Fail
7.26
20.52
Pass
1.00
0.99
Pass
H05-SE07 - Test 4 After Recalibration (15-point calibration data generation)
Measured
Linearity Criteria
Pass / Fail
1.60
7.69
Pass
0.96
0.98-1.02
Fail
5.28
15.39
Pass
1.00
0.99
Pass
H05-SE07 - Test 5 (EFM calibrated using SwRI data)
Measured
Linearity Criteria
Pass/ Fail
1.87
7.32
Pass
1.00
0.98-1.02
Pass
11.70
14.65
Pass
1.00
0.99
Pass
I05-SE03 - EFM calibrated using SwRI data
Measured
Linearity Criteria
Pass/ Fail
-3.93
7.43
Pass
1.00
0.98-1.02
Pass
6.40
14.87
Pass
1.00
0.99
Pass
I05-SE01 - EFM calibrated using SwRI data
Measured
Linearity Criteria
Pass/ Fail
-3.93
7.43
Pass
1.00
0.98-1.02
Pass
6.40
14.87
Pass
1.00
0.99
Pass
SwRI Report 03.12024.06
Page 75 of 371

-------
       Figure 25 shows the pooled delta data for the Sensors Inc. 4-inch flow meters versus the
laboratory calculated  exhaust flow for the Caterpillar C9 engine during steady-state repeat
testing.  The 4-inch flow meter showed a trend of increasing error as exhaust flow rate increased.
The median flow rate delta was near 5 % of point at the highest measured flow.  Because Sensors
Inc. effectively increased the slope of their EFMs when calibrating to the SwRI flow stand data,
the observed engine deltas would likely have been smaller had the flow meters used the original
Sensors Inc. calibration. As discussed earlier, the differences in the calibrations between Sensors
Inc. and SwRI may be linked to the different designs of the two  flow stands; however, the final
reason for the discrepancies is not know at this time.   The Sensors Inc. 4-inch EFM is rated at
flows as high as 1100 scfm,  therefore SwRI tested the meter over a broad range relative to the
maximum flow rate.
                             I PEMS 5th % A PEMS 50th % • PEMS 95th % • Points
      60
      50
      40
   £  30
   ra
   ^
   u.
   •K  20
      10
     -10
                 40
                      39
                              36
                                          -34-
                                                           21
                                                       13
                                                        10
                 :
                  250
                             350
                                        450
                                                   550
                                                             650
                                                                        750
                                 Lab Reference Exhaust Flowrate (scfm)
FIGURE 25. 4-INCH EFM EXHAUST FLOW POOLED DELTA DATA FROM ENGINE
                        2 STEADY-STATE REPEAT TESTING

3.5.3   Three-Inch Exhaust Flow Meter Linearity

       The Sensors Inc. 3-inch EFM is rated at flows as high as 600 scfm, and was used on
Engine 3 during the program.  On an initial set of linearity checks performed in January, 2006,
all of the 3-inch flow meters failed with positive slopes of 1.04 on average.  The 3-inch flow
meters were all returned to Sensors Inc. for recalibration.  When returned to SwRI, the 3-inch
EFMs failed linearity with a low slope of 0.96 on average. Table 47 summarizes the 3-inch EFM
linearity  data in chronological order.   One 3-inch flow meter, serial number H05-SE06, was
replaced  by  Sensors Inc. due to its outlying, low slope.  Based on experiences with the 4-inch
flow meters, the Steering Committee elected to  recalibrate the 3-inch flow meters using data
generated at SwRI.  The 3-inch flow  meters passed the linearity check after recalibration at
SwRI.  The resulting calibration increased the slope of each regression line by 4 to 5 percent. It
SwRI Report 03.12024.06
Page 76 of 371

-------
should be noted that the final linearity checks indicated a slight positive bias of roughly 1 percent
on average, with one of the flow meters nearly failing linearity with a high slope.

           TABLE 47. 3-INCH SENSORS INC. EFM LINEARITY RESULTS
Verification Description
Intercept
Slope
SEE
r2
H05-SE04 - Initial Test 1 (Jan, 2006)
Measured
Linearity Criteria
Pass / Fail
-12.54
5.91
Fail
1.04
0.98-1.02
Fail
8.68
11.82
Pass
1.00
0.99
Pass
H05-SE04 - Initial Test 2 (with Straight Pipe)
Measured
Linearity Criteria
Pass / Fail
-11.89
4.73
Fail
H05-SE04 - Initial 15-point w/ Straight Pipe
Measured
Linearity Criteria
Pass / Fail
-17.86
6.49
Fail
1.03
0.98-1.02
Fail
5.88
9.47
Pass
1.00
0.99
Pass
Jan, 2006)
1.05
0.98-1.02
Fail
14.55
12.99
Fail
1.00
0.99
Pass
H05-SE04 - Test 1 After Recalibration
Measured
Linearity Criteria
Pass/ Fail
1.67
6.37
Pass
0.95
0.98-1.02
Fail
8.10
12.74
Pass
1.00
0.99
Pass
H05-SE06 - Test 1 After Recalibration (EFM was replaced due to low slope)
Measured
Linearity Criteria
Pass/ Fail
4.01
6.77
Pass
0.90
0.98-1.02
Fail
4.23
13.54
Pass
1.00
0.99
Pass
I05-SE06 - Test 1 After Recalibration
Measured
Linearity Criteria
Pass/ Fail
1.90
6.27
Pass
0.97
0.98-1.02
Fail
7.09
12.54
Pass
1.00
0.99
Pass
H05-SE04 - EFM calibrated using SwRI data
Measured
Linearity Criteria
Pass/ Fail
3.23
5.97
Pass
1.02
0.98-1.02
Pass
5.99
11.93
Pass
1.00
0.99
Pass
I05-SE06 - EFM calibrated using SwRI data
Measured
Linearity Criteria
Pass/ Fail
4.93
9.44
Pass
1.01
0.98-1.02
Pass
4.87
18.87
Pass
1.00
0.99
Pass
I06-SE04 - EFM calibrated using SwRI data
Measured
Linearity Criteria
Pass/ Fail
4.89
6.01
Pass
1.00
0.98-1.02
Pass
6.43
12.02
Pass
1.00
0.99
Pass
       Figure 26 shows the pooled delta data for the Sensors Inc. 3-inch flow meters versus the
laboratory calculated exhaust flow for the International VT365 engine during steady-state repeat
testing.  The median flow rate delta for the 3-inch flow meters showed a nearly constant positive
bias of approximately  10 % of point.  The engine exhaust flow deltas would likely have been
smaller  had the 3-inch  EFMs not been recalibrated using the SwRI LFE flow stand data.
However,  the recalibration only resulted in an adjustment on the order of 4 % of point, which
was not large enough to explain the 10 % positive bias observed in the engine results. Because
SwRI Report 03.12024.06
Page 77 of 371

-------
the initial linearity check data on the SwRI flow stand indicated a positive bias, it is not clear
what the final source(s) of the positive measurement bias are.
     60
     50 -
   ,§ 40
   u
     30 -
   3
   ra
   •5 20
     10 --
          40
       150
                25
                 37
                             IPEMS 5th % A PEMS 50th % • PEMS 95th % « Points
                                   22
                                                      20
                                                              19
                   200
                                250           300           350
                                 Lab Reference Exhaust Flowrate (scfm)
                                                                      16
                                                                       400
                                                                                   450
FIGURE 26. 3-INCH EFM EXHAUST FLOW POOLED DELTA DATA FROM ENGINE
                        3 STEADY-STATE REPEAT TESTING
SwRI Report 03.12024.06
Page 78 of 371

-------
            4.0    ENGINE DYNAMOMETER LABORATORY TESTING

4.1   Engine Testing Objectives

      Engine  testing was performed  to  characterize  bias  and precision  errors for the
SEMTECH-DS instruments versus lab grade emission measurement equipment.  Analyzer and
exhaust  flow rate measurements were compared over both  steady-state  and transient engine
operation.   Several engine laboratory tests were designed to evaluate errors associated with
ECM-broadcast channels and subsequent interpolation errors of torque and BSFC. Finally, tests
were conducted to assess the exhaust flow measurement errors  due to installation related factors.

4.2   Test Engines and Dynamometer Laboratory
  FIGURE 27 PEMS INSTRUMENTATION SETUP IN DYNAMOMETER TEST CELL
SwRI Report 03.12024.06
Page 79 of 371

-------
               FIGURE 28 ENGINE 1 (HHD) - 14L DDC SERIES 60
               FIGURE 29 ENGINE 2 (MHD) - CATERPILLAR C9
SwRI Report 03.12024.06
Page 80 of 371

-------
             FIGURE 30 ENGINE 3 (LHD) - INTERNATIONAL VT 365
SwRI Report 03.12024.06
Page 81 of 371

-------
FIGURE 31 TEST CELL EXHAUST SYSTEM SHOWING PEMS FLOWMETERS AND
                                SAMPLING POINTS
4.3    40-Point Torque and BSFC Map Generation and Error Surface

       An initial task specified in the Test Plan was to generate 40-point torque and BSFC maps
as well as preview laboratory and PEMS emission and flow data.  The toque and BSFC maps
were generated for a variety of reasons.  First, the 40-point maps served as the data set used to
create interpolation surfaces  for the estimation of ECM Torque and ECM BSFC from ECM
broadcast (CAN) speed and fuel  rate signals.  Second, the preview of the emission results from
these points was used to aid in down-selection of the 10 test points to be used in subsequent
steady-state error surface experiments. Finally, the preview data was used to determine whether
multiple PEMS units could be run in parallel on a given engine during steady-state experiments,
thus shortening the amount of time required for the steady-state testing.

       The 40-points were chosen by the Steering Committee during the planning portions of the
program, and were designed to cover the entire NTE zone as evenly as possible.  Several points
SwRI Report 03.12024.06
Page 82 of 371

-------
were positioned slightly beyond the NTE boundary to aid in interpolation near the edges of the
NTE zone.

       After verifying the engines and aftertreatment systems were functioning properly, the lug
curves of the engine were mapped according to the procedures in CFR Part 1065 Subpart F.  An
Excel spreadsheet provided by EPA, and approved by the Steering Committee, was used with the
map data to generate the 40 points within the NTE zone.  The lug curves and 40 NTE points are
shown in Figure 32 through Figure 34 for Engine  1, Engine 2, and Engine 3, respectively.  Using
the laboratory raw and dilute sampling systems, as well as the PEMS, each of the 40 points was
tested over 10-minute modes.  The initially planned mode length was 3 minutes; however, the
mode length was extended to 10 minutes following initial Engine 1 testing to insure the fuel flow
measurement was stable.
    1800
    1600
    1400
    1200
  i 1000
  HI

  I  80°
     600


     400


     200
                                                       0 - Steering Committee Selected 10 Points
 • 39
-•40
           -•-26--

             NTE Torque - Power Line
       800       1000       1200      1400       1600       1800
                                          Speed (rpm)
                                                                2000
                                                                          2200
                                                                                   2400
 FIGURE 32. ENGINE 1 - DETROIT DIESEL SERIES 60 LUG CURVE AND 40-POINT
                                         MAP
SwRI Report 03.12024.06
            Page 83 of 371

-------
    1200
    1000
    800
                                                      0 - Steering Committee Selected 10 Points
/
Lug Curve /
;
' ^
NTE Speed Line — ^•
<
>-34 — -
• 32
I 35 • 31
36 • 30
I 37 • 29
t38
-**-4a
^-v
m^
• l?^.
• 20 • 16 ^•s5
• 21 • 15 »6\
M
• 22 • 14 07 *4
    400 - - -
    200
                                                r-23- -    -
                                                                    -& --- I
                                               NTE Torque - Power Line
    FIGURE 33. ENGINE 2 - CATERPILLAR C9 LUG CURVE AND 40-POINT MAP
    600
    500
    400
  3
  0"


  .2
    200
    100
0 - Steering Committee Selected 10 Points
^
Lug Curve /
r/ \
<
NTE Speed Line — ^>
4
<
• 32 ^~- 35 • 31 • 20 • 16
» 36 • 30 • 21 • 15
> 37 • 29 • 22 • 14
M8
^^--^•28 • 23 • 13
> 40 • 26 • 25 T • 1 1
NTE Torque - Power Line

• 6 "^
\
• 7 414
\
\
• 8 • 3
• 10 *1
L
\
      800
                     1300
                                     1800            2300
                                          Speed (rpm)
                                                                    2800
                                                                                   3300
FIGURE 34. ENGINE 3 - INTERNATIONAL VT365 LUG CURVE AND 40-POINT MAP



       The 40-point torque and BSFC interpolation surfaces were based on the laboratory torque

measurement and the laboratory BSFC calculation from  the laboratory measured fuel rate and
SwRI Report 03.12024.06
Page 84 of 371

-------
measured engine power. The laboratory torque and BSFC measurements were referenced to the
ECM-broadcast (CAN) speed and fuel rates for each mode. Using these maps, torque and BSFC
values were interpolated based off of ECM-broadcast channels. A triangular plane interpolation
routine was developed by  SwRI statisticians to aid in the interpolation process.  The 40-point
interpolation  maps  were  used throughout  dynamometer engine  testing to produce ECM
interpolated torque and BSFC  values for comparison with the laboratory measured reference
values.  The torque and BSFC interpolation maps for each engine can be found in Appendix E.
As requested by the Engine Manufacturers, only normalized map data is presented.

       Originally, the 40-point torque and BSFC maps and interpolation routine were to be used
in the Monte Carlo Error Model. The reference NTE events, supplying ECM speed and fuel rate,
were to use the maps to interpolate torque  and BSFC.  The interpolation process in the Model
was problematic because the interpolation maps were different for each engine, and  engine map
data was not available for the engine used to generate the reference NTE  events.  In addition,
there were questions regarding how to choose an interpolation surface for each event, as well as
the  additional computational load  of  having to do repeated interpolations in  the  Model.
Therefore, the final reference torque and BSFC values were supplied with  each reference NTE
events, and no torque and BSFC interpolations were performed in the Model.

       In addition to map  generation, the 40-point steady-state testing was used to preview the
performance of the PEMS and  laboratory.  As specified in the Test Plan, the results of the 40-
point testing were used to down-select the 40 points to the 10 points to be used for  steady-state
repeat testing.  SwRI reviewed the results of 40-point testing and recommended 10 points to be
used for steady-state testing.  In the down-selecting process, SwRI attempted to have  the selected
10 points evenly span the NOX concentration, exhaust flow rate, and NOX mass flow rate ranges
observed during the 40-point testing. In addition, the 10-points were  selected  to be somewhat
distributed  over the  NTE  zone.   In  general, the  Steering  Committee approved the SwRI
recommended  10-point down-select!on, with only a couple points modified for the final  steady-
state repeat testing. The selected 10 points are shown in Figure 32 through Figure 34.

       Three PEMS units were used simultaneously during the 40-point mapping process. The
Test Plan called for the data to be examined to determine if running PEMS in this manner would
cause measurement issues that would require subsequent testing to be conducted with one PEMS
unit at a time.  A particular area of concern  was the use of multiple PEMS flow meters in series.
Following the 40-point testing on Engine  1, the data was examined  by Sensors Inc. and the
Steering  Committee.   There was no  evidence  of a bias  for any PEMS exhaust flow rate
measurement.  Sensors Inc. agreed with this assessment, and the Steering Committee elected to
proceed with all further testing using the three PEMS units simultaneously. This decision was
made  at the April 4, 2006 conference call. Data from the 40-point maps on Engine 2 and 3 was
also examined for evidence of an exhaust flow bias, but none was found.

4.4     Steady-State Repeat Engine Testing and Error  Surfaces

       Repeat steady-state engine testing was performed to quantify PEMS bias and  precision
errors versus laboratory emission measurement equipment.  The measurement  errors evaluated
SwRI Report 03.12024.06                  Page 85 of 371

-------
during steady-state repeat testing included gaseous emission concentration  measurements and
exhaust flow rate measurements.

       The steady-state test consisted of 10 modes that were selected by the Steering Committee
from the 40-point mapping procedure discussed previously.  The 10-mode steady-state tests were
repeated 20 times.  As specified in the Test Plan, the mode order of each of the steady-state tests
was randomized. The mode length was 3 minutes with data averaged over the last 30 seconds of
each mode.  Each 10-mode test cycle was run essentially as a ramp modal cycle, although the
modes were processed  individually.   The  laboratory  reference  analyzers  were zeroed and
spanned before each cycle.  The engine and laboratory sampling systems were preconditioned
before each cycle  as outlined in 1065.520.  Following the preconditioning, the engine was
brought to idle, both laboratory and PEMS sampling systems and data  recording were started,
and the 10-mode test cycle was started.  At the end of the cycle, laboratory  systems were zero
and span checked.  The PEMS were only spanned at the start of each test day, and were zeroed
prior to the  start of each cycle.  This was roughly equivalent to zeroing the instrument every
hour, which  is the normal schedule for auto-zero maneuvers during field measurements.

       Three PEMS units were tested  simultaneously during steady-state testing.  The SwRI
dynamometer laboratory  conducted both raw and dilute emission  measurements.  The dilute
gaseous concentration measurements were converted to the equivalent raw concentrations using
the CVS  flow rate and the calculated exhaust flow rate.  This was done by first calculating a
dilute mass rate for a given pollutant, and then using the raw exhaust flow rate to back calculate
a  raw concentration.   These the dilute-to-raw emission concentrations  were  used  as  the
laboratory reference  for comparison  against the PEMS  gaseous concentration measurements.
The laboratory raw measurements were used for quality assurance purposes by providing a check
on the dilute-to-raw measurements and on the raw exhaust flow measurement via carbon balance
verifications.  The laboratory exhaust flow rate was determined using a LFE to measure  the
intake air flow, a Micro-Motion fuel  flow meter to measure  fuel flow, and the  laboratory
analyzers to measure raw exhaust emission concentrations.  The intake LFE measurement and
the raw chemical  balance were used  with  equation 1065.655-14 to calculate the reference
exhaust flow rate. The raw exhaust flow rate was also calculated using the LFE air flow rate and
measured fuel flow with the CFR Part 89 raw exhaust flow rate calculation. The two laboratory
exhaust flow rate calculation methods resulted in nearly identical exhaust flow rate results.

       The  wet gaseous PEMS  concentration data and  EFM  data  were  compared  to  the
laboratory reference.  Each PEMS measurement was compared individually to the  laboratory
reference. These errors, or deltas, were pooled to generate the steady-state error surfaces. For
steady-state  error  generation,  deltas were generated from paired data sets  of PEMS and
laboratory reference measurements.  In  other words, each PEMS measurements were compared
directly to the associated laboratory reference measurement for that repeat.

4.4.1   Engine 1 Detroit Diesel Series 60 Steady-State

       After generating the 40-point torque and BSFC maps, the Steering Committee selected 10
points to perform repeat steady-state testing.  The modes selected for Engine  1 steady-state
testing are shown in Figure 32.  As discussed in the audit section of the report, the PEMS units
SwRI Report 03.12024.06                 Page 86 of 371

-------
used during the initial set of steady-state measurements had all failed the 1065 NC>2 Chiller
Penetration Check.  The  Steering Committee elected to  proceed with steady-state testing to
determine if the NC>2 penetration failure would affect the performance of the PEMS units during
engine testing.  PEMS 1, 3,  and 4 were used for Engine 1 testing.  PEMS 2 was not chosen
initially for Engine  1 testing due to the 1065 Non-stoichiometric C>2 FID Interference audit
failure.

       After  completion  of the steady-state testing,  the results were presented to the Steering
Committee.   The individual  delta data from each PEMS was pooled.  The 5th, 50th and 95th
percentiles of the pooled error data was plotted against the mean laboratory reference value. As
shown in Figure 19, the PEMS showed a low bias  for NOX, especially  at high concentrations.
The Steering Committee  deemed the NOX results unsatisfactory, and Sensors Inc. was asked to
design and implement a solution to the NC>2 chiller penetration problem.  A complete discussion
of the NC>2 penetration solution can be found in the audit section of this report.

       In  June  2006, approximately  two months after the initial Engine 1  steady-state testing,
Sensors Inc. installed the NC>2 penetration retrofit package on the PEMS units at SwRI.  The NC>2
Chiller  Penetration audits  were then  repeated.   All upgraded PEMS units passed the 1065
penetration check.

       Following the PEMS  upgrades and audit checks, Engine 1  steady-state  testing was
repeated.  Again, the pooled PEMS delta  data was plotted against the mean laboratory reference
values.  As shown in Figure 35, the low NOX bias of the original testing was replaced with PEMS
data showing a  slight positive NOX bias. The pooled gaseous concentration delta data for Engine
1 can be viewed in Appendix F. Delta data is included for both the PEMS as well as the
laboratory raw measurements, with the dilute-to-raw  measurements as the reference.
SwRI Report 03.12024.06                 Page 87 of 371

-------
                            IPEMS 5th % A PEMS 50th % • PEMS 95th % « Points
      30
      25
      20
   a.
   3  15
   c
   o
   15
   ~  10
   HI
   Z
   o
   O  5
       140
      -5
     -10
                .40
         10
                         15
24 *
 --3-
                                                               33
               160
                       180
                               200
                                        220
                                                240
                                                        260
                                                                280
                                                                        300
                              Lab Reference Mean NOx Concentration (ppm)
  FIGURE 35. NOX CONCENTRATION POOLED DELTAS FOR REPEAT STEADY-
      STATE TESTING ON ENGINE 1 AFTER NO2 PENETRATION UPGRADE

      With the Engine 1 catalyzed DPF, CO and HC emissions were very low. Although the
laboratory raw and dilute analyzers reported raw CO concentration levels generally between 10
to 25 ppm, the SEMTECH-DS  consistently measured CO emission levels at approximately 40
ppm. Pooled  deltas for CO are given in Figure 36 for Engine 1.  The high CO bias may by due
in part to the low resolution of the CO detector, which has a reported resolution  of 0.01%
(lOppm).  In  addition, the CO instruments would typically read between 20 and 40 ppm when
zero gas was introduced to the sample port of the SEMTECH DS using a sample probe overflow
technique.  The positive CO bias was apparently due in part to the sampling handling  system of
the unit and was observed on all  three engines.
SwRI Report 03.12024.06
         Page 88 of 371

-------
                             I PEMS 5th % A PEMS 50th % * PEMS 95th % • Points
    60
               10
    50 '
  Q.
  HI
  Q

  I30
  1:
  HI
  o
  J 20

  O
  O
    10  -
                       ,-4  •          V       • y       33
                       A  A
               A
A   A
      10          12          14          16          18          20          22          24
                              Lab Reference Mean CO Concentration (ppm)

   FIGURE 36. CO CONCENTRATION POOLED DELTAS FOR REPEAT STEADY-
                           STATE TESTING ON ENGINE 1

       NMHC measurements presented a particular problem due to the very low tailpipe levels
associated with the use of the DPFs.  The laboratory reference measurement was complicated by
several factors beyond the low concentration levels.  First, the levels of THC and methane were
similar during  all steady-state  testing.   Figure 37  shows mean  raw exhaust hydrocarbon
concentrations for the steady-state modes.  Individual THC  and CH4 measurements varied
approximately ± 0.2 ppmC around the mean.  Unfortunately, this level was similar to the final
NMHC concentrations. Therefore, the variability of the NMHC measurement was high, and in
some cases resulted in slight negative values.
SwRI Report 03.12024.06                  Page 89 of 371

-------
    -0.2
                                        Steady-State Mode
 FIGURE 37. MEAN RAW HC CONCENTRATIONS FOR ENGINE 1 STEADY-STATE
                                      TESTING

       The dilute  NMHC  measurement  was  further complicated because the raw exhaust
concentrations were below those in the background air.  Typical background levels were 3 ppmC
for THC and 2.5 ppmC for NMHC.  This resulted in a high occurrence of negative NMHC
results.  The NMHC  measurement errors were further exaggerated by the dilute-to-raw scaling
process.  As a result,  the  Steering Committee elected to abandon  the dilute-to-raw NMHC
concentration values  as the reference at the April 2006  meeting. The laboratory direct raw
concentrations were chosen as the NMHC reference because the raw measurements  were not
complicated by background concentrations and conversion problems.

       Initially, Sensors Inc. supplied two methane analyzers which could, in principal, be added
to the  SEMTECH-DS  units.  However, since these analyzers were  external laboratory  grade
analyzers, the Steering Committee  decided the methane analyzers were not suitable for field
measurement.   Therefore, the PEMS NMHC  values  were determined using only  the THC
measurement.  The THC concentrations were multiplied by 0.98 to generate NMHC values, as
given in Part  1065. Figure 38 shows PEMS NMHC concentrations for Engine 1 plotted against
the associated mean  raw laboratory reference values.  Tailpipe  HC  levels for Engine 1 were
below the resolution limits of the PEMS FID analyzer, which reported mostly zero THC values
throughout steady-state testing.  Therefore, the data from Engine 1  was  not useful in producing
an NMHC error surface. A final decision on how to process the NMHC data was  deferred until
results  from Engines 2 and 3 could be reviewed.
SwRI Report 03.12024.06
Page 90 of 371

-------
     0.5
     0.4
     0.3
   E 0.2
   Q.
   Q.

   I 0.1
   c
   O -0.1
   O

   i -0.2
     -0.3

     -0.4
     -0.5
                                 IPEMS 5th % A PEMS 50th % • PEMS 95th %
       -0.05
                  0.00
                             0.05        0.10         0.15         0.20
                                Lab Reference NMHC Concentration (ppm)
                                                                         0.25
                                                                                   0.30
  FIGURE 38. PEMS NMHC CONCENTRATIONS FOR ENGINE 1 STEADY-STATE
                                       TESTING

       As  shown  in Figure  39, the PEMS  5-inch EFMs showed good correlation with the
laboratory  reference exhaust flow rate measurement,  with deltas generally less than 2  to  3
percent of point. Although not used in the Model, NOX mass flow rate errors are also shown in
Appendix F.  The PEMS NOX mass flow rate measurements were biased slightly high, which is
consistent  with the slightly  high bias  observed in  both NOX  concentration and the  EFM
measurements.
SwRI Report 03.12024.06
Page 91 of 371

-------
                             IPEMS 5th % A PEMS 50th % » PEMS 95th '
                                                            1 Points
      30
      25 -
      20 -
   —  15 -
   -2-  10-
      5 -
      -5 -
     -10 --
     -15 -
     -20
             40
                         37
                           27
                                  24
                                                                 33
                                            -TO	9-
               250
                        350
                                450
                                         550
                                                 B50
                                                         750
                                                                  850
                                                                          950H
                                                                                  1050
                                 Lab Reference Exhaust Flowrate (scfm)
FIGURE 39.  POOLED EFM DELTAS FOR ENGINE 1 STEADY-STATE TESTING- 5-
                                 INCH FLOW METER

4.4.2   Engine 2 Caterpillar C9 Steady-State

       Following the generation of the 40-point torque and BSFC maps, the Steering Committee
selected 10 NTE points to perform the repeat steady-state testing for Engine 2.  The selected 10
points  are  shown in Figure  33.   Prior  to Engine  1 steady-state testing, PEMS 6, used in
environmental testing, experienced a FID failure.  Because there was an immediate need to
continue environmental chamber testing, PEMS 3 was pulled from the dynamometer laboratory
and used as a  replacement for PEMS 6.  This resulted in a schedule delay while PEMS 6 was
repaired. PEMS 6 was therefore used for Engine 2 and 3 testing.

       During initial Engine 2 steady-state testing, PEMS 4 NOX values showed several outlying
low points.  The continuous  NO  and  NO2 data from  PEMS  4  indicated periods when both
channels were reporting zero values. Diagnostic efforts pointed to a bad NDUV lamp; therefore,
Sensors Inc. replaced the NDUV.   Linearity checks were  performed on the new NDUV before
proceeding with steady-state testing. After only a  couple tests, the new NDUV in PEMS 4 began
behaving erratically as well. The PEMS 4 NDUV was replaced  once again. Linearity checks as
well as a NO2 penetration check were performed before continuing with Engine 2 testing.

       Shortly  after PEMS 4 was repaired,  PEMS  6  reported  a fault stating the Manifold
Relative Humidity Sensor was not responding. With diagnostic support from Sensors Inc., the
exhaust manifold RH sensor and sensor manifold  block were removed.  This assembly is  part of
the NO2 penetration upgrade package which had  been developed by Sensors Inc.  earlier in the
program.  As shown in Figure 40, the  sensor was found to be corroded, therefore  a new sensor
and sensor manifold was installed. Because the exhaust manifold relative humidity sensor was
SwRI Report 03.12024.06
Page 92 of 371

-------
part  of the NC>2 chiller penetration retrofit kit, a 1065 NC>2 penetration check was  repeated.
PEMS 6 passed the penetration check with the new RH sensor.  Shortly after continuing with
steady-state repeat  testing,  PEMS  6 again  reported  the relative  humidity sensor  was not
responding. The sensor was removed and found to be wet.  A new sensor and sensor manifold
were installed and the  1065  NC>2 Chiller Penetration  check was repeated. PEMS 6 passed the
audit and SwRI continued steady-state testing with Engine 2.  After completing only a couple
steady-state tests, PEMS 6 reported the same RH sensor fault.  Again, the sensor was found to be
wet.
 FIGURE 40. CORRODED RH SENSOR (LEFT) COMPARED TO A NEW RH SENSOR
                                       (RIGHT)

       After the third failure,  Sensors Inc. recommended checking the sensor manifold block for
leaks.  The leak test was performed by slightly pressurizing the sensor manifold and checking for
air leaking past the  RH sensor.  All  sensor manifolds tested by SwRI had air escaping the
manifold by the RH sensor.  According to Sensors Inc.,  the  escaping air  likely  caused liquid
water to be drawn up the manifold and in contact with the  relative humidity sensor, thus causing
the fault.  Sensors Inc. instructed SwRI to  reseal  the RH sensor in the  sensor manifold block
using silicon.  A picture of a RH sensor surrounded by silicon as well as  the sensor manifold
block is shown in Figure 41.  After reseating the RH sensor, the manifold was leak checked to
insure air was  not escaping  from the  manifold.   Another 1065  NC>2  penetration check was
performed after installing the new sensor.  With the properly  sealed sensor manifold, PEMS 6
operated without fault for the remainder of the steady-state testing.
SwRI Report 03.12024.06
Page 93 of 371

-------
     FIGURE 41. DISASSEMBLED RH SENSOR MANIFOLD WITH RH SENSOR

       During this time, PEMS 4 also began to have faults pertaining to the manifold relative
humidity sensor not responding. The RH sensor and manifold were  replaced with a new, non-
leaking manifold.  After replacement of the  sensor, and  subsequent NC>2 penetration check,
PEMS 4 reported no other problems related to the RH sensor.

       Following the completion of the various repairs and diagnostic  efforts, the remaining
Engine 2 steady-state tests were completed.  However, examination  of the data following the
completion of Engine 2 transient testing revealed a problem with the  Caterpillar C9 steady-state
data.   In generating the  transient  error surfaces,  the transient data was  corrected using the
variance measured during steady-state testing.  The  steady-state variance correction process is
described in detail under the transient testing  section.  However, the variance of the Engine 2
steady-state data was generally larger than the variance of the transient data.  After reviewing the
Engine 2 steady-state data, the high variance  was found to be related to the large time lapse
caused by the PEMS hardware failures. Almost half of steady-state points were run prior to the
PEMS hardware failures, with the remaining points run approximately 3 weeks afterward.

       As seen in Figure 42,  there is a definite shift in NOX concentration for the initial steady-
state points versus the points  run after the PEMS repairs. Repeats 5,  9, and 12 through 20 were
run 2  V2 weeks after  the other steady-state repeat tests.   This shift was recorded for both
laboratory and PEMS analyzers.  The bias error would not affect the  steady-state error surfaces,
as the PEMS measurements are paired with the laboratory reference and this removes variances
caused by the engine. However, the variance of the pooled  raw PEMS data, not the PEMS delta
data, was used to generating the transient error surfaces.  The high  variance of the Engine 2
PEMS steady-state data would have collapsed the  Engine 2 transient error surfaces due  to the
steady-state variance correction.    Because the Engine 2 transient  error surfaces would  be
inaccurate using the  high variance  steady-state data, the Steering Committee elected to repeat
Engine 2 steady-state testing.
SwRI Report 03.12024.06                  Page 94 of 371

-------
    550
    540
    530
   a.
    520
    510
    500
    490
    480
            Data Run Late
       0   1    2   3   4   5   6   7   8   9  10  11   12  13  14  15  16   17  18  19  20  21
                                        SS Repeat Number

      FIGURE 42. ENGINE 2 INITIAL STEADY-STATE REPEAT NOX RESULTS
             SHOWING TIME DEPENDENT CONCENTRATION SHIFT

       The repeated steady-state testing for Engine 2 went smoothly, with no problems from the
PEMS or the laboratory.  The PEMS  and  laboratory raw data was compared to the laboratory
reference dilute-to-raw measurements.  The pooled delta data was plotted versus the  mean
reference value.  The results  for  the gaseous  emission  concentration errors are shown in
Appendix F. The SEMTECH-DS median NOX error levels for Engine 2 were generally less than
5 ppm and centered near zero.

       In an attempt to address the issue of low tailpipe NMHC levels observed  during Engine 1
testing, a different DPF was used for Engine 2. The Engine 2 DPF was a 2007  production DPF
supplied by Caterpillar.  The production DPF likely had lower precious metal loadings than the
DPFs which SwRI had procured for the program.  The Steering Committee hoped the production
DPF would result in more useable NMHC data.  Figure 43 shows raw hydrocarbon levels for
Engine 2.  While the NMHC concentrations were higher than Engine 1  levels, the methane and
THC  levels were extremely similar, resulting in reference NMHC levels still  near zero.  The
similar THC  and  CH4 levels resulted in  a large occurrence of negative values for the raw
laboratory reference. However, the PEMS showed measurable NMHC response for Engine 2, as
seen in Figure 44.
SwRI Report 03.12024.06
Page 95 of 371

-------
                                 456


                                        Steady-State Mode
                                                                                 10
   FIGURE 43. RAW HYDROCARBON LEVELS FOR ENGINE 2 STEADY-STATE

                                       TESTING
     5.0
                              I PEMS 5th % A PEMS 50th '
                                                    # PEMS 95th '
     4.5
     4.0
     3.5
   I 3.0 -J
   c
   HI
   u
   c
   o
   o

   O
2.5
2.0
     1.5
   LU
   Q.
     1.0 -
     0.5
     0.0
       -0.9      -0.8      -0.7      -0.6      -0.5      -0.4       -0.3


                                 Laboratory NMHC Concentration (ppm)
                                                                  -0.2
                                                                           -0.1
                                                                                   0.0
  FIGURE 44. PEMS NMHC CONCENTRATIONS FOR ENGINE 2 STEADY-STATE

                                       TESTING
SwRI Report 03.12024.06
                                Page 96 of 371

-------
       Over  the  course  of several  conference  calls and  meetings,  there was considerable
discussion among Steering Committee members as to how the NMHC data could be represented
in the  Model.  Ultimately, several decisions were made regarding the NMHC error surface.
First, it was determined that the laboratory reference method for NMHC was not accurate at the
low NMHC levels.  As a result, the NMHC error surface was collapsed to a single x-axis point,
and all deltas would be generated using a reference value of zero.  All of the NMHC data would
be pooled together to generate a single set of 5th, 50th, and 95th percentile values.  Second, the
Steering Committee decided that only Engine 2 data would be used to populate the NMHC error
surface, because the data from Engines 1 and 3 showed no PEMS NMHC response.   These
decisions were finalized at the November 2006 Steering Committee meeting in San Antonio.  A
similar approach was to be used for the transient error surface as well.

       Although the laboratory analyzers reported CO levels under 6 ppm for all modes during
Engine 2 testing, the PEMS median error was consistently near 50  ppm.  Steady-state CO data
for Engine 2 is found in Appendix F.  This high bias was similar to the data observed for Engine
1, and was consistent for all CO measurements during this program.

       The deltas measured for the PEMS 4-inch EFMs versus  the laboratory reference exhaust
flow rate are shown in Figure 45.  Although the 4-inch flow meters passed the 1065 linearity
criteria on the SwRI flow stand, the EFMs showed a positive error at high flow rates during
engine testing.  This error was on the order of a 5 percent positive bias. A discussion of the 4-
inch EFM error results and  linearization issues is  included in the flow meter audit section of the
report. Although not part of the measurement allowance,  NOX mass flow rate deltas for Engine 2
are also included in Appendix  F.  With accurate NOX  concentrations  measurements, the NOX
mass flow rate error resembled the exhaust flow rate errors and had a positive bias at high NOX
mass flow rates.
SwRI Report 03.12024.06                 Page 97 of 371

-------
                             IPEMS 5th % A PEMS 50th % • PEMS 95th % • Points
      60
      50
      40
   £  30 -
   ra
   ^
   u.
   •K  20 -
      10 -
     -10
                40
                      39
                              36
                                         -34-
                                                           2i-
                                                       13
                                                       10
                 :
                  250
                             350
                                       450
                                                  550
                                                             650
                                                                        750
                                 Lab Reference Exhaust Flowrate (scfm)
 FIGURE 45. POOLED EFM DELTAS FOR ENGINE 2 STEADY-STATE TESTING - 4
                                 INCH FLOW METER

4.4.3   Engine 3 International VT365 Steady-State

       Similar to Engine 1 and 2, 10 NTE points were selected by the Steering Committee from
the original 40 points tested for the torque and BSFC maps. The selected 10 points are shown in
Figure 34. As with Engine 2, PEMS 1, 4, and 6 were used for Engine 3 testing. A Horiba OBS-
2200 On Board Emission Measurement System was also tested during Engine 3 operation.

       Engine 3  steady-state repeat testing went smoothly, with no equipment failures from  the
PEMS or laboratory.   However, PEMS 6 consistently showed a negative NOX bias  at high
concentration levels.  This surfaced initially during the 40-point map testing, and was confirmed
during repeat steady-state tests.  Shown in Figure 46 is the PEMS 6 steady-state pooled NOX data
versus the mean laboratory reference concentrations. PEMS 1  also showed a slight negative NOX
bias at high  concentrations,  but not as severe as PEMS 6. Post-test span checks for all PEMS,
conducted using  the instrument span port, indicated no problems despite the low bias.  Several
diagnostic tests were performed immediately after steady-state testing with PEMS 6 to determine
the cause of the bias.  NO and NO2 linearity verification results did not indicate any instrument
problems. As a check, dry span gas was then overflowed to the sample line of PEMS 6.  At 100
% of span value,  NO read nearly 7 % below the bottle value and NO2 read over 4 % low. At 70
% of span value,  NO read approximately 2  % low while NO2 read slightly over 1 % low. At 30
% of span value, both NO and NO2 measurements were accurate.  This confirmed the low bias
problem, but only when the gas is being introduced through the sample line.
SwRI Report 03.12024.06
Page 98 of 371

-------
                  •Lab Reference
                                 PEMS6 5th %   A PEMS6 50th '
                                                             PEMS6 95th %
                                                                           Points
    700
    600
    500
    400
    300
    200
    100
                                                                         37  35
      100
                   200
                                300           400          500
                                Lab Reference NOx Concentration (ppm)
                                                                      600
                                                                                   700
    FIGURE 46.  PEMS NOX CONCENTRATION VERSUS MEAN LABORATORY
               REFERENCE FOR ENGINE 3 STEADY-STATE TESTING

       In early December 2006, an NC>2 penetration check was performed with PEMS 6.  As
shown in Table 41, PEMS 6 passed a NC>2 chiller penetration check.  This was unexpected,
considering the low biases observed before the long weekend. Therefore, the overflow checks
with dry NO  and  NO2 span gas were  repeated.   Low biases  were not  observed during the
repeated checks, indicating that something had changed while the PEMS were sampling ambient
air over the weekend.  A possible explanation for the performance difference is the drying of
accumulated water in the sample handling system.  It is not known at this time why some of the
PEMS showed a low bias while others did not during Engine 3 steady-state testing; however,
none of the analyzers failed  any of the 1065 performance checks during this time.  There was
considerable Steering Committee discussion regarding the Engine 3  steady-state data  set.  The
Steering Committee elected to accept the biased steady-state data because a specific cause for the
low NOX bias was not evident, and because the PEMS continued to pass all pertinent 1065 audit
verifications.

       Another concern with the International steady-state data was high NOX concentration
variability at high  concentration levels.  NTE points 35, 37, and 40 were all near peak torque
speed and produced high NOX concentrations.  Although the speed and torque for these modes
was consistent, the NOX concentrations showed unexpectedly high variability, which was evident
in both the lab reference data and the PEMS data.  Figure 47 shows the laboratory dilute-to-raw
concentrations for  NTE point 35 during steady-state repeat testing.   The laboratory reference
NOX concentration median absolute  deviation (MAD) value calculated for point 35 was over 40
ppm.   An example MAD calculation  is shown below for reference.   As discussed for the
Caterpillar steady-state data, the high NOX variability  did  not  adversely affect the Engine 3
SwRI Report 03.12024.06
Page 99 of 371

-------
steady-state error surface data because of the individual pairing with the laboratory reference.
However, the high steady-state variability does affect the transient error surfaces during the
steady-state variance  correction.  The solution to this  problem is discussed  in  detail  in the
transient engine testing section of the report.
                            MAD = median^xj -median(x^
    800
  •§• 750
  .2 700
    650
  o
  O
  § 600
  HI
  o
  c
  S 550
  •s
  OL

  o 500
    450
    400
                                            10
                                     Steady-State Repeat Number
                                                               15
                                                                                  20
 FIGURE 47. INTERNATIONAL VT365 POINT 35 NOX CONCENTRATIONS DURING
                         STEADY-STATE REPEAT TESTING

       Another issue with the Engine 3 steady-state data was several instances of outlying data.
As seen in Figure 48, NTE point 30 had 5 repeats that were significantly higher than the other 15
events.  This shift was observed on all of the measurement instruments, including the laboratory
dilute and raw and all of the PEMS.  As a result, this instances were determined to be the result
of engine variability, rather than measurement errors.  Per the Steering Committee's decision, the
outlying data points were removed from steady-state data set.  For NOX concentration, events
were removed from NTE points 25,  30, and 37. Outlying events were also removed from the
CO, CO2, and NOX mass flow rate data sets.
SwRI Report 03.12024.06
Page 100 of 371

-------
    400
  E
  Q.
    350
  o
  IE
  "E
  HI
  I
  O
  HI
  o
  £ 250
  £
  HI
  OL
  2 200
  .Q
  J3
    150
Outlying Point;
                                            10
                                     Steady-State Repeat Number
                                                               15
                                                                                 20
 FIGURE 48. INTERNATIONAL VT365 POINT 30 NOX CONCENTRATIONS DURING
                         STEADY-STATE REPEAT TESTING

       After removal of the outlying data, the PEMS and laboratory raw data was compared to
the laboratory reference dilute-to-raw measurements.  The pooled delta data was plotted versus
the mean reference values. The results for the gaseous emission concentration errors are shown
in Appendix F.   As  discussed previously,  PEMS 1  and 6 showed a  low  bias for NOX
concentration at high levels. Similar to Engine 1 and 2, the median CO errors were near 50 ppm,
with the 95th percentile values  reaching 90 ppm for Engine 3.  Although the  PEMS  showed
occasional NMHC responses on Engine 3, the large body of data  indicated essentially zero
PEMS response to the  tailpipe exhaust, similar to what was observed for Engine 1.  This data
ultimately reinforced the Steering Committee decision to use only Engine 2 data for NMHC error
surface generation.

       The deltas measured for  the PEMS 3-inch EFMs versus the laboratory reference  exhaust
flow rate are shown in Figure 26.  Although the 3-inch flow meters passed the 1065 linearity
criteria on the SwRI flow stand, the EFMs showed a positive error at high flow rates during
engine testing. Generally this error was on the order of 10 percent of point.  A discussion of the
3-inch EFM error results and linearity is given in the flow meter audit section of the report.  As
shown in Figure 50,  the Horiba OBS-2200  exhaust flow rate  measurements showed  good
agreement with the laboratory reference.
SwRI Report 03.12024.06
                       Page 101 of 371

-------
                             PEMS 5th % A PEMS 50th % » PEMS 95th % • Points
     60
     50
   ,§ 40
   § 30 ^
   u.
   "w
   3
   re
   x 20
   LU
     10  -
          40
               25
                 37
                        35 30
                          JU
                                  22
                                                     20
                                                             19
                                                                     16
                                                           •

                                                           A
          A
       150
                   200
                                250          300          350
                                Lab Reference Exhaust Flowrate (scfm)
                                                                      400
                                                                                  450
 FIGURE 49. POOLED EFM DELTAS FOR ENGINE 3 STEADY-STATE TESTING - 3-
                                INCH FLOW METER
                             PEMS 5th % A PEMS 50th % » PEMS 95th % • Points
     20

     15

     10
   I
   u
   *  0
   £
   ra   1
     -10
     -15
     -20
     -25
     -30
40
                        22
                                                           -16-
      A  200
                                250
                                             300
                                                         350
                                                                      400
                                 Lab Reference Exhaust Flowrate (scfm)
  FIGURE 50. POOLED HORIBA OBS-2200 EXHAUST FLOW RATE DELTAS FOR
                        ENGINE 3 STEADY-STATE TESTING
SwRI Report 03.12024.06
                          Page 102 of 371

-------
       Although not part of the measurement allowance, NOX mass flow rate deltas are also
included in Appendix F.  The NOX mass flow rate errors were biased high in the mid-to-lower
range due to the positive exhaust flow rate error.  However, the negative NOX concentration bias
at high levels helped offset the NOX mass flow rate error at the higher levels in some cases.

       Figure 51 shows the pooled Horiba OBS-2200 NOX concentration deltas measured during
Engine 3 steady-state testing.  Median delta values were near  10 % of point.  In addition to the
                                                                              th
                                                                                     -th
median  bias, NOX variability was also  large, with the difference between  the  95   and 5
percentile concentrations near 20% of point.  The Engine 3  pooled  steady-state deltas for all
gaseous emissions are shown in Appendix F.  As shown in Figure 52, THC measurement with
the Horiba OBS-2200 showed good correlation to the SwRI raw THC concentrations, even at
levels between 0.5  and 2.5 ppmC.  The  OBS-2200 CO2 concentration  measurements were
generally higher than the laboratory reference values, with  median deltas ranging from 3 to 5 %
of point. With the laboratory reference CO concentrations  ranging from 6 to 18 ppm, CO deltas
were near -100 ppm for the 5th percentile deltas, -60 ppm for the 50th percentile, and 110 ppm for
the 95th  percentile error.
     140
     120 -
     100 -
   a.
      80 -
   I  60 4
   c
   o
   o
      40 -
      20 -
     -20 -
     -40
                             I PEMS 5th % A PEMS 50th % » PEMS 95th % • Points
. .•
* 40
25 22° 16 20 19


*
A
A
» A
A f
A
0 " • ^40 " 340 440
• • •

• *
37 35
*

A



540 64fl 7.

                                 Lab Reference NOx Concentration (ppm)
 FIGURE 51. POOLED NOX DELTAS FOR THE HORIBA OBS-2200 DURING ENGINE
                             3 STEADY-STATE TESTING
SwRI Report 03.12024.06
Page 103 of 371

-------
                    - Lab Reference  •  PEMS 5th %   A PEMS 50th %  »  PEMS 95th '
                                                                           Points
     3.5
     3.0
   O 2.5
   Q.
   Q.

   o 2.0
     1.5
   o
   o
   o
     1.04-
     0.5
     0.0
                         30
'  '35
19 2$6
                                                                     40
                                                                            25
                                                                           -*-
       0.0
                       0.5
                                      1.0              1.5
                                 Lab Reference THC Concentration (ppmC)
                                                                      2.0
                                                                                     2.5
     FIGURE 52. POOLED THC MEASUREMENTS FOR THE HORIBA OBS-2200
                    DURING ENGINE 3 STEADY-STATE TESTING
4.4.4  Steady-State Concentration Error Surface Generation

       The steady-state gaseous emission concentration error surfaces were generating using the
                                                               -th
                                                                             -th
pooled PEMS EFM deltas versus the laboratory reference.  The 5 ,  50 ,  and 95  percentile
values  of the  pooled  error  terms were  plotted  against  the mean  laboratory  reference
concentrations. Delta values were normally sampled from the steady-state gaseous concentration
error  surfaces for each NTE event.  Because the  error surfaces were level dependent, linear
interpolation between points was used to determine the appropriate delta. Individual steady-state
error surfaces for each engine are contained in Appendix F.

       The final steady-state concentration error surfaces were generated by pooling the Engine
1, 2, and 3 error surfaces. The  combined error surface is shown in Figure  53. Because the NOX
concentration error profiles for the three engines were notably different, the combined final error
surface was  extremely irregular, displaying sharp transitions in areas where concentration values
for the three engines overlapped.  The original intent of testing three engines was to generate a
broad, uniform, well-distributed final error surface.  The underlying assumption with this method
was that the three engines would produce similar errors, and the  combined error surface would
therefore be relatively uniform.
SwRI Report 03.12024.06
       Page 104 of 371

-------
                           •95th percentile
                                    -50th percentile (median)
•5th percentile
  a.
  o.
  u
  Q
  c
  o
  2  -20
  c
  HI
  u
  c
  o
  o
  LU
  Q.
    -100
-40
     -60
     -80
                                Lab Reference Mean NOx Concentration [ppm]
       FIGURE 53. COMBINED ERROR SURFACE FOR STEADY-STATE NOX
                                   CONCENTRATION
       The combined error surface shown in Figure 53 was presented to the Steering Committee
during the December 2006 meeting in San Antonio. After a lengthy discussion, the Committee
elected to reprocess the final error surfaces.   Because  the  steady-state concentration  error
surfaces were sampled normally, the 5th and 95th percentile  deltas were to represent the largest, or
worst case, delta values. Following that argument, if all engines would have generated deltas at
all x-axis  concentration levels, the 5th percentile value would have been generated by the engine
having the lowest bias and the 95
the highest bias.
                               th
                            percentile would have been generated by the engine reporting
       The  steering  committee elected to  reprocess  the final  error surfaces by  linearly
interpolating between each engine's error surface data points to populate x-axis values generated
by the other engines. For example, the Engine 1 deltas were used to linearly interpolate Engine 1
deltas that would have occurred at the other x-axis concentrations.  This method was applied to
the 5th,  50th, and 95th percentile values for each engine.  For concentration values beyond  the
range of data actually taken for a given engine, the method for generating delta values depended
on the trends observed in the measured data.  If no definite trend was observed in the data for a
given engine, the nearest x-axis error value was repeated to generate the extrapolated data (i.e.,
the first or last data point of the data set). If a trend was evident, a regression line was fit through
the engine's delta data.  The regression line was then used to generate deltas for points requiring
extrapolation.
                                           -th
                                                -.th
                                                         -th
       Using the method described above, a 5  , 50 , and 95  percentile was generated for all
three engines at each of the 30 x-axis mean laboratory reference points.  Once this was done, the
SwRI Report 03.12024.06
                                 Page 105 of 371

-------
final pooled error surfaces were generated.  At each of the 30 x-axis points, the 5th percentile of
the pooled error surface was generated by selecting the lowest 5th percentile value from the three
engines at that point.  In a similar manner, the 95th percentile was selected by taking the highest
95l percentile value from the three engines, and the 50th percentile was taken as the middle 50th
percentile value from  the three engines. The final steady-state NOX concentration error surface
used in the Model is  shown in Figure 54 as an example  of this process.   Concentration error
surfaces for each engine as well as the final combined error surfaces are shown in Appendix F
for all pollutants.  This process was not needed for NMHC data, because only Engine 2 data was
used to generate the NMHC error surface as discussed earlier.
                           •95th percentile
    -50th percentile (median)
•5th percentile
                                                                                     7(10
    -100
                               Lab Reference Mean NOx Concentration [ppm]

          FIGURE 54. FINAL ERROR SURFACE FOR STEADY-STATE NOX
                                  CONCENTRATION

       The steady-state exhaust flow rate error surfaces were generating using the pooled PEMS
EFM deltas versus the laboratory reference.  The  5th, 50th, and 95th percentile values of the
pooled error terms were plotted against the mean laboratory reference exhaust flow rates.  The
data was normalized using the maximum EFM flow rate as specified in the user manual.  The
maximum flow rates for the 3-inch, 4-inch, and 5-inch EFMs were  600,  1100, and  1700 scfm,
respectively.  The reference NTE events used in the Model supply exhaust flow rate in scfm as
well as the EFM size.  Using this information, the reference NTE exhaust flow rate measurement
was normalized similar to the laboratory generated error surface.  Using the normalized flow
rate, a delta value was normally  sampled  from the steady-state exhaust flow error  surface for
each NTE event. Because the error surface was level dependent, linear interpolation was used to
determine the appropriate  flow rate delta. Individual steady-state error surfaces for each engine
are collected in Appendix F.
SwRI Report 03.12024.06
Page 106 of 371

-------
       The combined steady-state exhaust flow rate error surface was generated by pooling the
Engine 1, 2, and 3 error surfaces. The combined error surface is shown in Figure 55. Similar to
the steady-state concentration combined error surfaces, the exhaust flow rate error surface was
not uniform due to error differences for each engine and EFM size.
  &
  o
  LU
  Q.
                          -95th percentile
   -50th percentile (median)
-5th percentile
                              Lab Reference Mean Exhaust Flow Rate [% of Max]

FIGURE 55. COMBINED ERROR SURFACE FOR STEADY-STATE EXHAUST FLOW
                                          RATE

       The final steady-state exhaust flow rate error surface was reprocessed as described in the
steady-state concentration error surface generation section. The final steady-state exhaust flow
rate error surface used by the Model is shown in Figure 56.
SwRI Report 03.12024.06
Page 107 of 371

-------
                          -95th percentile
   -50th percentile (median)
-5th percentile
                             Lab Reference Mean Exhaust Flow Rate [% of Max]

   FIGURE 56. FINAL ERROR SURFACE FOR STEADY-STATE EXHAUST FLOW
                                         RATE

4.5    Transient Engine Testing and Error Surfaces

       The transient engine testing was performed to evaluate the errors involved in using the
SEMTECH-DS PEMS units to measure 30-second transient NTE events.  It should be noted that
the intent of the transient experiments was to capture errors present over and above those already
observed during the steady-state experiments (i.e., errors resulting from the transient nature of
the events being measured).  Transient error surfaces were generated  for gaseous emission
concentrations, exhaust flow rate, and  various  ECM-related data, including  ECM broadcast
speed and fuel rate, and ECM interpolated torque and BSFC.  In addition, the transient test data
was  used to  generate an error surface based on time alignment  errors of several key  PEMS
parameters.

       During the development of the Test Plan,  there was concern over the lack of information
available regarding the  accuracy and precision  of the laboratory  reference methods over 30-
second test events.  Therefore, the Steering Committee elected not to compare the laboratory and
PEMS data during transient testing.   Instead,  the transient error surfaces account  only for
precision errors of the PEMS with respect to their own median measurements.  There is no bias
error term captured in the  transient error surfaces.  However, all  laboratory instruments were
used during transient testing for comparative purposes and to evaluate the repeatability of the lab
over 32 second events.  A  secondary goal of the program was to assess the repeatability of the
1065-based reference laboratory methods over 30-second events of this nature.

       Transient engine testing consisted of repeating 20-minute cycles  containing 30  unique
32-second NTE events.  An Excel spreadsheet, provided by EPA and approved by the Steering
SwRI Report 03.12024.06
Page 108 of 371

-------
Committee, was used to generate 30 unique NTE events for each engine, based on the engines'
lug curves.  In addition, 31  unique  transition events were generated, allowing for varying
amounts of time between NTE events  during the cycle.  Descriptions of the NTE and transition
events were taken directly from  the Test Plan  and are  given  in Table 48 and  Table 49,
respectively. The NTE event order, as well as the transition order, was randomized to generate
20 different 20-minute cycles, each containing all 30 NTE events.  As stated in the Test Plan,
only 4 to 5 cycles were run each day, so that transient testing occurred over a span of 4 to 5 days.
          TABLE 48. NTE EVENT DESCRIPTIONS FROM THE TEST PLAN
Table 3.3.3-a: Dynamic Response NTE Events
NTE
Event
NTEj
NTE2
NTE3
NTE4
NTE5
NTE6
NTE7
NTE8
NTE9
NTE10
NT-En
NTE12
NTE13
NTE14
NTE15
NTE16
NTEJ 7
NTE18
NTEJ 9
NTE20
NTE21
NTE22
NTE23
NTE24
NTE25
NTE26
NTE27
NTE28
NTE29
NTE30
'Speed %
Range
17%
59%
Governor line
17%
59%
Governor line
17%
59%
100%
Lower third
Upper third
Middle third
17% - governed
17% - governed
17% - governed
2Torque %
Range
332%
332%
332%
66%
66%
66%
100%
100%
100%
332% - 100%
332% - 100%
332% - 100%
Lower third
Upper third
Middle third
Lower right diagonal
Upper left diagonal
Full diagonal; lower left to upper right
Lower left diagonal
Upper right diagonal
Full diagonal; lower right to upper left
Third light — heavy-duty NTE event from
International, Inc. data set
Cruise; ~ 50 mph
Cruise; ~ 75 mph
Small bulldozer
Large bulldozer
Second of three NTE events in FTP
Third light — heavy-duty NTE event from
International, Inc. data set
First of two NTE events in NRTC
First of two NTE events in NRTC
Description
Steady speed and torque; lower left of NTE
Steady speed and torque; lower center of NTE
Steady speed and torque; lower right of NTE
Steady speed and torque; middle left of NTE
Steady speed and torque; middle center of NTE
Steady speed and torque; middle right of NTE
Steady speed and torque; upper left of NTE
Steady speed and torque; upper center of NTE
Steady speed and torque; upper right of NTE
Highly transient torque; moderate transient speed
Highly transient torque; moderate transient speed
Highly transient torque; moderate transient speed
Highly transient speed; moderate transient torque
Highly transient speed; moderate transient torque
Highly transient speed; moderate transient torque
Transient; speed increases as torque increases
Transient; speed increases as torque increases
Transient; speed increases as torque increases
Transient; speed decreases as torque increases
Transient; speed decreases as torque increases
Transient; speed decreases as torque increases
Sample from LHDE
Sample from HDDE
Sample from HDDE
Sample from NRDE
Sample from NRDE
Seconds used from FTP: 714-725, 729-743, 751-755
Sample from LHDE
Seconds used from NRTC: 423-430, 444, 448-450, 462-
481, increased 464 speed from 40% to 42%
Seconds used from NRTC: 627-629, 657-664, 685-696,
714-722
1 Speed (rpm) = Curb Idle + (Speed % * (MTS - Curb Idle)
2 Torque (Ibf-ft) = Torque % * Maximum Torque At Speed (i.e. lug curve torque at speed)
3 Torque (Ibf-ft) = Maximum of (32 % * peak torque) and the torque at speed that produces (32 % * peak
power)
SwRI Report 03.12024.06
Page 109 of 371

-------
      TABLE 49. NTE TRANSITION DESCRIPTIONS FROM THE TEST PLAN
Table 3.3.3-b: Dynamic Response Inter-NTE Events
INT Event1
INTj
INT2.6
INT7.10
INTn.14
INT15.18
INT19.21
INT22
INT23
INT24
INT25
INT26
INT27
INT28
INT29
INT30
INT31
Duration (s)
10
2
3
4
5
6
7
8
9
11
13
17
22
27
35
5
Frequency
1
5
4
4
4
o
J










Description
Initiation of cycle; INTj is always first
Shortest and most frequent inter-NTE events
Short and frequent inter-NTE events
Short and frequent inter-NTE events
Short and frequent inter-NTE events
Short and frequent inter-NTE events
Medium inter-NTE event
Medium inter-NTE event
Medium inter-NTE event
Medium inter-NTE event
Long inter-NTE event
Long inter-NTE event
Long inter-NTE event
Long inter-NTE event
Longest inter-NTE event
Termination of cycle; INT31. is always last
Interval speeds and torques are not identical, but they are clustered around zero torque and the speed at which 15%
of peak power and 15% of peak torque are output.
       These tests were all run as hot-start transient tests.  The engine and aftertreatment were
preconditioned before each test as recommended in 1065.520.  The laboratory analyzers were
zeroed and spanned prior to each test, although again, they were run for reference only. The
PEMS were spanned only at the start of the day, and zeroed prior to each test.  The total elapsed
time for each test was near one hour, which is the recommended auto-zero frequency for the
PEMS.   The transient data  was post-processed to  extract  the  data associated with the 30
individual events so they could be compared across all 20 repeats.

4.5.1   Engine 1 Detroit Diesel Series 60 Transient

       The transient engine testing followed the repeated Engine 1 steady-state testing with the
upgraded PEMS.  The initial transient task  was to generate the transient cycles and tune the
engine and dynamometer controls.  Some of the NTE events contained highly transient  speed
and load changes that challenged  the laboratory dynamometer as well  as the test engine.  Next,
the lug curve from the Engine 1 was programmed into the PEMS. The PEMS used the  J1939
Percent Load ECM-broadcast channel with the Engine 1 lug curve to estimate real time torque.
A number of prep cycles were then run to insure the laboratory and PEMS were distinguishing
the same 30 NTE events per cycle.  Upon initial runs, the laboratory and PEMS missed several
NTE  events.   Causes for the missed NTE events included engine  speeds running near the
governor that caused a drop in torque as well as highly transient speed and load profiles causing
torque or power to drop below the 30% NTE minimum values. In addition, the PEMS  estimated
torque was often below the laboratory torque  causing the torque or power to drop below the NTE
threshold levels.  A number of slight speed and torque adjustments were made to the NTE  cycles
before the laboratory and PEMS would consistently record 30 events per cycle.
SwRI Report 03.12024.06
Page 110 of 371

-------
       Continuous engine speed data from the first 5 transient cycle repeats is shown in Figure
57, while torque traces are  shown in Figure 58.  In order to view the repeated data, the NTE
events were reordered and the transition events were removed from the data set.  As seen from
the continuous  data, many events contained  highly  dynamic speed and load  combinations.
However, the laboratory was able to achieve very good repeatability in speed and torque for the
various  events from cycle  to cycle, even  though the actual running order of events varied
considerably from one cycle to the next.
                             -R1 	R2 	R3 	R4 	R5 	Event
     2300
     500
                     200
                                   400           600
                                         Time (sec)
                                                              800
                                                                           1000
 FIGURE 57. ENGINE 1 EXAMPLE SPEED TRACES DURING TRANSIENT TESTING
SwRI Report 03.12024.06
Page 111 of 371

-------
                              -R1
                                    •R2 	R3 	R4 	R5
                                                            - Event
    1800
                     200
                                   400           600
                                         Time (sec)
                                                              800
                                                                            1000
     FIGURE 58. ENGINE 1 EXAMPLE TORQUE TRACES DURING TRANSIENT
                                       TESTING

       At the completion of transient testing, the gaseous concentration, exhaust flow rates, and
NOX mass rates were averaged over each NTE event for all 20 repeats, and then pooled.  The 5th,
50th, and 95th percentile values of the averaged data were plotted versus the mean laboratory
dilute-to-raw measurements for each NTE event for reference purposes only,  and to aid in data
review. This data processing structure was similar to the steady-state data analysis. In reviewing
the Engine 1 data, there was unexpectedly large variance for the several of the NTE events.
After further investigation, a number of outlying measurements were found in the data set. The
outlying points were found in both the laboratory raw and dilute measurements, as well as the
PEMS data. These outlying measurements were traced to changes in engine operation during the
NTE events. Shown in Figure 59, NTE Event 4 of transient cycle Repeat 2 shows a drastic drop
in NOX concentration, while the NOX concentration of the other repeats was relatively  constant.
NTE Event 7 of Repeat 4 also shows a drop in NOX concentration. These engine operation shifts
may have been caused by different NTE modes orders or different transition events.   The
underlying data processing method for the transient surfaces  assumes that engine behavior will
be constant from run to run, and that any variance observed in the PEMS data is due to
measurement errors.  Changes in engine behavior would therefore add additional, and potentially
overwhelming, variance error to the data.
SwRI Report 03.12024.06
Page 112 of 371

-------
                            -R1
                                  -R2 	R3 	R4 	R5
                                                          • Event
    140
                        100
                                  150       200       250
                                        Time (sec)
                                                             300
                                                                      350
                                                                               400
   FIGURE 59. ENGINE 1 NOX CONCENTRATION TRACES DURING TRANSIENT
                     TESTING SHOWING OUTLYING EVENTS

       In order to prevent such outliers from artificially inflating the observed transient variance,
the Steering Committee decided to have SwRI remove the outlying data points at the May 2006
meeting in Ann Arbor.  The removal of the outlying data was first done manually using scatter
plots and eliminating obvious outlying NTE points.   A more rigorous outlier test was also
applied to the data set by  SwRI statisticians.  Outlier tests based on ASTM E 178 procedures
were used to  identify outlying NTE data.  All tests were made at the 5% level of significance.
The results from the statistical outlier tests and the scatter plot test gave similar results.  Of the
600 NTE events generating during transient testing, 34 were deemed outliers and removed from
the data set.

       A secondary task performed by SwRI  was to evaluate the repeatability  of the laboratory
and engine over 32-second NTE events. The  laboratory brake-specific NOX emission results for
                                                              th
                                                                  ->th
                                                                           -th
each NTE event were calculated and pooled.  Figure 60 shows the 5  , 50 ,  and 95   percentile
lab dilute BS NOX results plotted against the mean BS NOX for the 30 NTE  events repeated 20
times.  This figure was generated with the outlying NTE points removed from the Engine 1 data
set. As shown in Figure 61, the median absolute deviation (MAD) was calculated for each of the
30 NTE events. Although some variation was observed, the MAD was generally in the range of
0.04 g/ (hp-hr), which is roughly 2 percent of the NTE threshold value of 2.0 g/(hp-hr) However,
some NTE events showed variance over 0.08 g/(hp-hr).
SwRI Report 03.12024.06
Page 113 of 371

-------
               1.0
                        1.5
                                2.0       2.5       3.0       3.5
                               Lab Reference Dilute Mean BS NOx (g/hp-hr)
                                                                   4.0
                                                                           4.5
                                                                                    5.0
  FIGURE 60. ENGINE 1 POOLED TRANSIENT TEST NTE BRAKE-SPECIFIC NOX
                                       RESULTS
     0.12
     0.10
   ji
   a.
   5
   S 0.08
     0.06
   m
   HI
   u
   c
   HI
     0.04
   OL
     0.02
     0.00
•     *
    •
                 *   "*-
       0.5       1.0       1.5       2.0       2.5       3.0       3.5
                                Lab Reference Dilute Mean BS NOx (g/hp-hr)
                                                                   4.0
                                                                            4.5
                                                                                    5.0
  FIGURE 61. ENGINE 1 POOLED TRANSIENT TEST NTE BRAKE-SPECIFIC NOX
                                    MAD RESULTS
SwRI Report 03.12024.06
         Page 114 of 371

-------
       Although the laboratory data was not used to generate the transient error surfaces, plots
were generated to compare the PEMS performance with the laboratory measurements. As seen
in Figure 62 and Figure 63, the PEMS median values during transient testing nearly matched the
laboratory mean values for both NOX concentration and exhaust flow rate, respectively.
                         • Lab Reference
                                        PEMS 5th %   A PEMS 50th '
                                                                  PEMS 95th '
     800
     700  -
     600
   .2 500


   o 400
   o
   O
   O 300
   z
   (/)
   UJ 200
   Q_
     100
                 100
                           200        300       400       500        600
                             Lab Reference Dilute Mean NOx Concentration (ppm)
                                                                            700
                                                                                     800
FIGURE 62. ENGINE 1 POOLED PEMS NTE NOX CONCENTRATION DATA VERSUS
                              THE LABORATORY MEAN
SwRI Report 03.12024.06
Page 115 of 371

-------
                         •Lab Reference
                                       PEMS 5th %  A  PEMS 50th %  •  PEMS 95th '
     1100
     1000
   o  900
     800
   3  700
  LU
  Q.
     600
     500 - - -
     400
       500
                      600
                                    700            800            900
                               Lab Reference Mean Exhaust Flow Rate (scfm)
                                                                             1000
FIGURE 63. ENGINE 1 POOLED PEMS NTE EXHAUST FLOW RATE DATA VERSUS
                              THE LABORATORY MEAN
4.5.2  Engine 2 Caterpillar C9 Transient

       Engine 2 transient testing was conducted  at the completion of the initial steady-state
testing. A process similar to Engine 1 transient testing was followed to perform the Engine 2
repeat NTE testing.  The transient data generated with the Caterpillar C9 had no outlying data
due to engine operation, and the full data set was therefore used without alteration to generate the
Engine 2 transient error surfaces.

       Similar to Engine 1, the repeatability of the  laboratory and engine was evaluated  by
comparing the brake-specific NOX emission results over the 20 NTE cycle repeats.  Figure 61
shows the pooled BS NOX emission results for each of the 30 NTE events, while Figure 64 shows
the BS NOX MAD value for the 30 events.   The MAD was  generally around  0.04  g/(hp-hr),
similar to Engine 1.
SwRI Report 03.12024.06
Page 116 of 371

-------
                               Lab Reference   I  5th %   ,  50th %   » 95th
                  1.9
                             2.1         2.3         2.5          2.7
                                Lab Reference Dilute Mean BS NOx (g/hp-hr)
                                                                          2.9
                                                                                     3.1
  FIGURE 64. ENGINE 2 POOLED TRANSIENT TEST NTE BRAKE-SPECIFIC NOX
                                       RESULTS
     0.07
     0.06
   .c
   I" 0.05
     0.04
   "£ 0.03
   u
   c
   HI
     0.02
   &
   2
     0.01
     0.00
            +
         •      »
*    *
        1.7
                   1.9
                              2.1          2.3         2.5         2.7
                                Lab Reference Dilute Mean BS NOx (g/hp-hr)
                                                                          2.9
                                                                                     3.1
  FIGURE 65. ENGINE 2 POOLED TRANSIENT TEST NTE BRAKE-SPECIFIC NOX
                                    MAD RESULTS
SwRI Report 03.12024.06
     Page 117 of 371

-------
       Although the laboratory data was not used to generate the transient error surfaces, plots
were generated to  compare the PEMS performance with the laboratory measurements.  As seen
in Figure 66, the PEMS median NOX concentrations during transient testing were bias slightly
low.  As shown in Figure  67, the positive  exhaust flow rate bias observed during Engine 2
steady-state testing was also evident during transient testing.
                         • Lab Reference  •  PEMS 5th %   A PEMS 50th %  •  PEMS 95th '
     500
     450
   E  400
   Q.
   _a

     350
   o  300
   o
   O
   O  250
     200
     150
     100
       100
                 150
                          200       250       300        350        400

                             Lab Reference Dilute Mean NOx Concentration (ppm)
                                                                            450
                                                                                     500
FIGURE 66. ENGINE 2 POOLED PEMS NTE NOX CONCENTRATION DATA VERSUS
                              THE LABORATORY MEAN
SwRI Report 03.12024.06
Page 118 of 371

-------
                        -Lab Reference  •  PEMS 5th %  A  PEMS 50th %  •  PEMS 95th %
    950
    350
       350      400      450       500      550      600       650
                               Lab Reference Mean Exhaust Flow Rate (scfm)
                                                                   700
                                                                            750
                                                                                    800
FIGURE 67. ENGINE 2 POOLED PEMS NTE EXHAUST FLOW RATE DATA VERSUS
                              THE LABORATORY MEAN
4.5.3   Engine 3 International VT365 Transient

       Engine 3 transient testing was conducted at the completion of the steady-state testing.  A
process similar to Engine 1 and 2 transient testing was followed to perform the Engine 3 repeat
NTE testing.  The transient data generated with the International VT365 had no outlying data and
was used without alteration to generate the Engine 3 transient error surfaces. Engine 3 transient
data showed higher variance than Engines  1 and 2. This was expected due to the higher variance
observed during steady-state repeat testing.  For the Engine 3 data, both laboratory and PEMS
indicated a wider distribution of measurements with no obvious outlying points.

       Similar to Engine 1 and 2, the repeatability of the laboratory and engine was evaluated by
comparing the brake-specific NOX emission results over the 20 NTE cycle repeats.  Figure 68
shows the pooled BS NOX emission results for each of the 30 NTE events, while Figure 69 shows
the BS NOX MAD values for the 30 events.  The Engine 3 MAD was at about 0.06 g/(hp-hr),
with some events over 0.1 g/(hp-hr).  The higher variability  of Engine 3 can be attributed to real
engine-out variations in NOX concentration.  These values are roughly 3 to  5 percent of the 2.0
g/(hp-hr) threshold.
SwRI Report 03.12024.06
Page 119 of 371

-------
     3.5
                               - Lab Reference  • 5th %  A  50th %   » 95th '
     3.0
   a.
   .c
     2.5
   m
     2.0
   HI
   OL
     1.5
     1.0
       1.0      1.2      1.4      1.6      1.8      2.0      2.2       2.4       2.6
                                 Lab Reference Dilute Mean BS NOx (g/hp-hr)
                                                                                2.8      3.0
  FIGURE 68. ENGINE 3 POOLED TRANSIENT TEST NTE BRAKE-SPECIFIC NOX
                                         RESULTS
     0.12
     0.10
   ji
   a.
   5
   S 0.08
     0.06
   m
   HI
   u
   c
   HI
     0.04
   a:
   .Q
     0.02
     0.00
        1.0      1.2      1.4      1.6      1.8      2.0      2.2      2.4
                                  Lab Reference Dilute Mean BS NOx (g/hp-hr)
                                                                        2.6      2.8
                                                                                        3.0
  FIGURE 69. ENGINE 1 POOLED TRANSIENT TEST NTE BRAKE-SPECIFIC NOX
                                      MAD RESULTS
SwRI Report 03.12024.06
Page 120 of 371

-------
       Although not used for transient error surface generation, the PEMS concentration  and
exhaust flow rate data was plotted versus the laboratory mean values for comparative purposes.
Figure 70 shows the pooled PEMS NOX concentration data for each of the 30 repeated NTE
events.  Interestingly, the low NOX concentration bias observed during Engine 3 steady-state
testing did not manifest in the transient data set, with all median PEMS NOX  values near the
mean laboratory concentrations.  As shown in Figure 71, the positive exhaust flow bias was
apparent in both the steady-state and transient testing.
                         • Lab Reference  • PEMS 5th %   A PEMS 50th %  •  PEMS 95th '
     550
     500
   E 450
   Q.
   _a
   c
   .2 400
   "
   o 350
   o
   O
   O 300
   UJ 250
   Q.
     200 - - -
     150
       150
                 200        250        300       350       400        450
                             Lab Reference Dilute Mean NOx Concentration (ppm)
                                                                           500
                                                                                     550
FIGURE 70. ENGINE 3 POOLED PEMS NTE NOX CONCENTRATION DATA VERSUS
                              THE LABORATORY MEAN
SwRI Report 03.12024.06
Page 121 of 371

-------
                        -Lab Reference
                                       PEMS 5th %  A  PEMS 50th '
                                                                 PEMS 95th %
    500
    450
   o 400
    350
   3 300
  LU
  Q.
    250
    200
    150
       150
                    200
                                 250           300           350
                               Lab Reference Mean Exhaust Flow Rate (scfm)
                                                                        400
                                                                                     450
FIGURE 71. ENGINE 3 POOLED PEMS NTE EXHAUST FLOW RATE DATA VERSUS
                              THE LABORATORY MEAN
4.5.4  Transient Concentration Error Surface Generation

       A number of steps were taken to generate transient error surfaces from the raw NTE data.
The concentrations used to generate the error surfaces were calculated as flow-weighted averages
over each NTE event.  In other words, the continuous concentration data was multiplied by the
corresponding exhaust flow rate. The NTE event averaged concentration times exhaust flow rate
values were then divided by the NTE event averaged exhaust flow rate.  The calculation of flow-
weighted  concentrations was  performed to  capture transient variances that were pertinent to
emission calculations. For example, gaseous concentrations were multiplied by the exhaust flow
rate  when calculating  emission results, therefore, flow-weighting the concentration results
captured a more representative variance measurement.

       As stated previously, the data generated by the laboratory during transient testing was not
used to generate the transient error surfaces.  The averaged, flow-weighted PEMS concentrations
were  pooled.   The 5th,  50th,  and 95th percentile values, as well as the MAD values,  were
calculated for each of the 30 NTE events.  To calculate the precision of the NTE testing, the 5th
and 95th percentile flow-weighted PEMS concentration values were subtracted from  the 50th
percentile values. These delta values were then used to populate the transient error surface, with
the 95th percentile minus the 50th percentile concentration values  set to the 95th percentile error
                -th
                                                                                        th
values and  the  5  percentile minus the 50   percentile concentrating values set  to  the  5
percentile error  values.  The  50th percentile error value was set to zero for all NTE  events.
SwRI Report 03.12024.06
Page 122 of 371

-------
                          -th
                                  -th
Figure 72 shows the PEMS 5  and 95  percentile error values plotted against the PEMS median
flow-weighted NOX concentration for Engine 1.
                                 PEMS Median Flow-Weighted NOx (ppm)

   FIGURE 72. ENGINE 1 UNCORRECTED TRANSIENT FLOW-WEIGHTED NOX
                             CONCENTRATION ERRORS

       A final task performed  on  the  transient  error surfaces was to correct  the  variance
measured during transient testing for the variance already recorded during steady-state testing.
The variance correction was performed to insure steady-state precision errors were not double-
counted  in the  Model.  The transient  error surfaces  would  represent only  the incremental
precision error associated with transient operation.  The PEMS concentration MAD values from
both transient and steady-state testing were used to calculate a scaling factor.  This scaling factor
was then used to shrink or collapse  the transient  error surfaces  to remove  the steady-state
variance.  Figure 73  shows the transient and interpolated steady-state MAD  values  with the
resulting scaling factor. As anticipated, the transient MAD values were generally larger than the
steady-state MAD values.
SwRI Report 03.12024.06
Page 123 of 371

-------
     20
                         •NOx ppm MAD TR
    •NOx ppm MAD SS
-Scaling Factor
               100       200       300       400       500
                              PEMS Median Flow-Weighted NOx (ppm)
                                                             600
                                                                      700
                                                                                -  -1.0
                                                                                  -1.2
                                                                               800
  FIGURE 73. ENGINE 1 TRANSIENT AND INTERPOLATED STEADY-STATE MAD
                   VALUES WITH RESULTING SCALING FACTOR
       To  calculate the  scaling  factor,  the steady-state PEMS  MAD  data  was  linearly
interpolated to generate steady-state MAD values at the 30 median PEMS concentration values
measured during NTE testing.  In other words, the  10 steady-state PEMS concentration median
and MAD values were used with the 30 transient PEMS concentration median and MAD values
to linearly interpolate steady-state MAD values at the 30 transient median values. Next, the 30
interpolated steady-state MAD values were compared to the 30 transient MAD values.  If the
steady-state interpolated MAD  value  was greater  than the transient MAD value,  the scaling
factor was set to zero. Otherwise, the scaling factor  was calculated using the following equation.
                         Scaling _ Factor =
                                                      -MAD
                                                 MADt
       The final corrected NOX concentration error surface for Engine  1 is shown in Figure 74.
With most scaling factor values greater than zero, the corrected error surface looks similar to the
uncorrected surface shown in Figure 72.
SwRI Report 03.12024.06
Page 124 of 371

-------
                                PEMS Median Flow-Weighted NOx (ppm)

  FIGURE 74. ERROR SURFACE FOR ENGINE 1 TRANSIENT FLOW-WEIGHTED
                               NOX CONCENTRATION

       A scaling factor of zero indicated the steady-state variance was greater than the transient
variance and mathematically collapsed the transient error surface value to zero.  Although not
anticipated when designing the experiment, the steady-state variance was sometimes larger than
the transient variance, especially with Engine 3.  Shown in Figure 75, the  steady-state MAD
values were generally larger than the transient MAD values for Engine 3, resulting in zero level
scaling factors.
SwRI Report 03.12024.06
Page 125 of 371

-------
     30
     25
   I 20
   Q.
     15
     10
      5
                                                                                 1.5
                                                                               -1.0
                                             0.5
                                                                          r -   - - - 0.0
                                                                                    D)
                                                                                    C
                                                                                 -0.5
                                                                               - - -1.0
                                                                                 -1.5
      100      150      200      250      300     350     400
                              PEMS Median Flow-Weighted NOx (ppm)
                                                              450
                                                                      500
                                                                               550
  FIGURE 75. ENGINE 3 TRANSIENT AND INTERPOLATED STEADY-STATE MAD
                  VALUES WITH RESULTING SCALING FACTOR

       Figure 76 shows the final corrected NOX concentration transient error surface for Engine
3. Due to the steady-state variance correction and zero level scaling factors, approximately two
thirds of the error surface points were zero values.  This was problematic, especially when the
Engine 3 data was combined with data from the other two engines.
SwRI Report 03.12024.06
Page 126 of 371

-------
   D)

   I
25


20


15


10


 5


 0
  1
-5


-10


-15


-20


-25
              -95th % Delta —•— 5th % Delta
                                                  350      400       450      500      5
                                 PEMS Median Flow-Weighted NOx (ppm)
  FIGURE 76. ERROR SURFACE FOR ENGINE 3 TRANSIENT FLOW-WEIGHTED
                                NOX CONCENTRATION

       Similar to the steady-state error surfaces, the final transient error surfaces were generated
by pooling the Engine 1, 2, and 3 final error surface data.  Also similar to the final steady-state
error surfaces, the combined transient error surfaces were  highly irregular. The unevenness of
the transient error surfaces was due to the variability of the transient delta data, the steady-state
variance correction,  and error differences between the three engines.  Shown in Figure 77, the
final NOX concentration error surface for transient testing was jagged and unpredictable.
SwRI Report 03.12024.06
                                 Page 127 of 371

-------
                                 PEMS Median Flow-Weighted NOx (ppm)

  FIGURE 77. FINAL ERROR SURFACE FOR TRANSIENT FLOW-WEIGHTED NOX
                                  CONCENTRATION

       The transient error data was reviewed at the December 2006 Steering Committee meeting
in San Antonio.  The Steering Committee suggested removing high variability Engine 3 steady-
state test  points from  the  transient MAD correction to avoid collapsing  the  transient  error
surfaces to zero. Unfortunately, the suggested correction had little impact on the transient error
surfaces, as illustrated in Figure 77. The  Steering Committee decided the highly irregular error
surfaces may lead to erratic Model behavior.  Additional analysis was performed by Steering
Committee members on the transient error data.  The additional analyses confirmed that the data
for Engines 1 and 2 behaved as expected with larger transient MAD values as compared to
steady-state MAD values. Engine 3 generally showed a reversed trend, which was not expected.
It was initially proposed that Engine 3 data be eliminated from the final transient error surfaces.
The Steering Committee arrived at a  solution that allowed most of the data from the  three
engines to be used in the error surface  generation, as originally intended in the Test  Plan. The
solution was proposed in late December, and accepted by the Steering Committee on December
18, 2006 via email response

       Steady-state and transient MAD data for the three engines was pooled into a single data
set.  Selected outlier points were removed from the Engine 3 steady-state data set which showed
extremely large variations,  as described earlier.  In addition, some of the Engine 1 data points
were removed where the transient concentrations had been  above all  measured steady-state
concentrations for the engine, thus requiring extrapolation to generate steady-state MAD values.
The remaining data was pooled and root-mean-square (RMS) MAD values  were generated for
both steady-state and transient data sets. The MAD values were compared to  generate a transient
SwRI Report 03.12024.06
Page 128 of 371

-------
effect MAD.  The steady-state MAD was subtracted from the transient MAD, and 5th and 95th
percentile values for the error surface were then generated using the following equations:
                                                   -MAD
                    Delta5thpercentaei = Concentration, * (-1.65* MAD terms)
                    Delta.
                         95thPercentile,
Concentration. * (+1.65 *MADte rms)
       The  1.65 term  in the equations above is the  factor from a normal distribution which
covers 90 percent of the distribution around the median.  This data analysis method essentially
produces an error surface which is a line, and makes the assumption that the transient errors are
dominated by span errors.  This assumption is generally supported by the data.

       The error surfaces for CO, NOX, and CO2 were all processed in this manner.  In the case
of CO, the MADtranS;rms was actually less than the MADSSjrms, indicating that steady-state errors
were still  larger than transient errors.  Therefore,  the CO transient error surface was set to zero
for all values.  The final error surface values are given in Table 50 below.  These values  each
describe a pair of lines, with values at any given emission concentration determined via linear
interpolation.

       TABLE 50. FINAL GASEOUS TRANSIENT ERROR SURFACE DELTAS
Pollutant /
Concentration
NOX
0
3000
C02
0
20
Percentiles
5th
50th
95th
delta, ppm
0.00
-72.03
0.00
0.00
0.00
72.03
delta, %
0.0000
-0.1904
0.0000
0.0000
0.0000
0.1904
1 Based on sampling with normal distribution
       The transient concentration error surfaces are sampled normally in the Model, once per
NTE event.   Concentration errors are linearly interpolated between x-axis points  on the error
surface based on the reference NTE event concentrations and the error surface median x-axis
concentration levels. Transient error surface data can be found in Appendix G for all transient
testing.

4.5.5   Transient Flow Meter Error Surface Generation

       Transient flow meter  error  surfaces were generated  as described  in the Transient
Concentration Error Surface Generation  section of the report.  Weighting was not performed
with the PEMS EFM data.  The PEMS exhaust flow data was pooled and the 5th, 50th, and 95th
SwRI Report 03.12024.06
 Page 129 of 371

-------
percentile values were calculated for each of the 30 repeated NTE events.  Variance errors were
calculated by taking the difference between the 95th and 50th percentiles and the 5th and 50th
percentiles of the pooled PEMS exhaust flow rate data. Finally, the steady-state variance was
removed from the transient data set by calculating and applying a scaling factor based on the
interpolated steady-state MAD calculation and the transient MAD values.

       The exhaust flow rate  error surfaces were normalized as a percent of the maximum EFM
flow rate.  Engine 1, 2, and 3  used  the  5, 4, and 3-inch diameter EFMs, respectively.  As
specified in the Sensors Inc. EFM user manual, the maximum flow rates for the 3-inch, 4-inch,
and 5-inch EFMs were  600,  1100, and 1700 scfm, respectively. Shown in Figure 78 through
Figure 80 are the transient exhaust flow error surfaces for engine 1,  2, and 3, respectively.  Zero
values indicated the steady-state variability was larger than the transient variance at that exhaust
flow rate level.
     -3.5
                                PEMS Median Exhaust Flow Rate (% of Max)
   FIGURE 78. ERROR SURFACE FOR ENGINE 1 TRANSIENT EXHAUST FLOW
                                         RATE
SwRI Report 03.12024.06
Page 130 of 371

-------
    -3.5
                             PEMS Median Exhaust Flow Rate (% of Max)
   FIGURE 79. ERROR SURFACE FOR ENGINE 2 TRANSIENT EXHAUST FLOW
                                      RATE
    -3.5
                             PEMS Median Exhaust Flow Rate (% of Max)
   FIGURE 80. ERROR SURFACE FOR ENGINE 3 TRANSIENT EXHAUST FLOW
                                      RATE
SwRI Report 03.12024.06
Page 131 of 371

-------
       The combined, final exhaust flow rate error surface is shown in Figure 81.  As with the
other final transient error surfaces, the final exhaust flow rate error was jagged. The unevenness
of the transient error surface was due to the variability of the transient delta data, the steady-state
variance correction, and error differences between the three engines.
     -3.5
                                PEMS Median Exhaust Flow Rate (% of Max)
  FIGURE 81. FINAL ERROR SURFACE FOR TRANSIENT EXHAUST FLOW RATE

       The Steering Committee did not elect to re-analyze the transient EFM error surface data,
and therefore this error surface was used in the Model as shown in Figure 81.

4.5.6  Transient Dynamic Error Surface Generation

       The dynamic error surfaces were generated to capture the variance of ECM-broadcast
speed and fuel rate measurements over  the repeated 32-second NTE events.  In  addition, the
variance of the  interpolated torque and BSFC  from the 40-point maps  was evaluated.  The
generation  of the dynamic error surfaces followed the procedure described in  the Transient
Concentration Error Surface Generation  section of the report and summarized in the Transient
Flow Meter  Error  Surface  Generation section of the report.   The  dynamic  error  surface
generation process, however,  did not include a steady-state variance correction.  A steady-state
variance correction was not needed  because the parameters evaluated for the dynamic  error
surfaces did not  have error surfaces for steady-state testing.  Therefore, there was no concern of
double counting  dynamic errors.

       ECM-broadcast fuel rate was calculated as an average over the NTE event and received
no weighting.  The dynamic fuel rate error surface was normalized using the engine's maximum
fuel  rate, which was taken as the highest fuel  rate recorded during the 40-point mapping
SwRI Report 03.12024.06
Page 132 of 371

-------
procedure.  The Detroit Diesel Series 60 recorded a maximum fuel rate of 98 L/h, the Caterpillar
C9 measured 75 L/h, and the International VT365 delivered a maximum fuel rate of 46 L/h.  The
final dynamic fuel rate error surface is shown in Figure 82.  The fuel rate variance errors were
generally less than 1.0 % of the engine's maximum fuel rate.
                                 PEMS Median Fuel Rate (% of Max)

      FIGURE 82. FINAL ERROR SURFACE FOR DYNAMIC ECM FUEL RATE


       ECM-broadcast  speed was weighted using  the  interpolated torque  from  the  40-point
maps.  The interpolated torque weighted ECM speed was calculated as an  average over each
NTE event. The ECM speed error surface was normalized with ni0 speed equal to 0.0 % and n^
speed equal to 100 %. Table 51 shows the nlo and nhi speed definitions for each engine.


     TABLE 51. NLO AND NHI SPEED DEFINITIONS FOR ENGINES 1, 2, AND 3

Engine 1 DDC
Engine 2 CAT
Engine 3 INT
nlo Speed (rpm)
1014
1099
1198
nhi Speed (rpm)
2129
2320
2839
       The final combined dynamic ECM speed error surface is shown in Figure 83.  The ECM-
broadcast speed showed little variation over the 20 repeated transient tests.  The majority of the
5th and 95th percentile error terms were less than 0.2 % of normalized speed.
SwRI Report 03.12024.06
Page 133 of 371

-------
     0.8
     0.6
   2
   2
   1!
   1!
   £
     o.2
   g 0.0
   ai
   a-
     -0.2 -
   LU
   a.
     -0.4
     -0.6
                            PEMS Median Torque-Weighted Normalized Speed (%)

         FIGURE 83. FINAL ERROR SURFACE FOR DYNAMIC ECM SPEED

       Interpolated torque from the 40-point map was weighted using ECM-broadcast speed.
The  ECM  speed-weighted  interpolated  torque was  calculated  as an average over each NTE
event.  The interpolated torque error surface was normalized as a percent of peak torque. Peak
torque measured during the lug curve tests at  SwRI was 2195 N-m for Engine 1, 1464 N-m for
Engine 2, and 681 N-m for Engine 3.  The final ECM  speed-weighted interpolated torque error
surface in shown in Figure 84. Most variance errors were less than 1.0 % of peak torque for all
engines.
SwRI Report 03.12024.06
Page 134 of 371

-------
     3.0
   •x 2.0
   re

   'o
   £ 1.0 J
   HI
   3
   0"
0.0
       20
     -1.0
     -2.0 -
     -3.0
     -4.0
                             PEMS Median Speed-Weighted Torque (% of Max)
 FIGURE 84. FINAL ERROR SURFACE FOR DYNAMIC INTERPOLATED TORQUE

       Interpolated BSFC from the 40-point map was weighted using ECM-broadcast fuel rate.
ECM fuel rate-weighted interpolated BSFC was calculated as an average over each NTE event.
Figure 85 shows the final dynamic interpolated BSFC error surface.   Similar to the other
dynamic error surfaces, BSFC variability over the  repeated NTE  events was  low, with most
variance errors less than 1.0 g/(kW-hr).
SwRI Report 03.12024.06
                               Page 135 of 371

-------
     -3.0
                              PEMS Median Fuel-Weighted BSFC (g/kW-hr)
   FIGURE 85. FINAL ERROR SURFACE FOR DYNAMIC INTERPOLATED BSFC
4.6    Interacting Parameters - Warm-Up Test Error Surface

       The warm-up tests were conducted to evaluate ECM-broadcast torque and map errors due
to variations in oil  viscosity, fuel temperature,  oil temperature,  and coolant temperature.
Because independently controlling these parameters was difficult, cold start tests were performed
to cumulatively estimate these ECM errors as the engine passed from cold to stable operating
temperatures. The errors associated with ECM fuel rate and ECM speed translated into torque
and BSFC errors through the 40-point map interpolation process. Warm-up tests were performed
on each of the three engines.   The Detroit Diesel Series 60 and International VT365, both EGR
engines, were cooled to ambient temperature,  approximately 18 °C,  prior to the warm-up test.
The Caterpillar C9 was cooled to 0 °C for the warm-up test.

4.6.1   Interacting Parameters - Warm-Up Test Procedure

       The original experimental design given in the Test Plan called for a single warm-up test
on each engine with  the speed and load condition specified by the  Steering  Committee. The
initial choice was a high speed (Speed C), light load condition. However, when the first cycle
was run with the DDC engine, the intake manifold temperature never reached the NTE threshold
value.  This data was decidedly unsatisfactory for the measurement allowance.

       Following a discussion of the warm-up test  results at the May 24, 2006 meeting in Ann
Arbor, the target speed and load was changed by the Steering Committee to Speed C and WOT
to insure the event would enter the NTE zone. Because the engine was at maximum operator
SwRI Report 03.12024.06
Page 136 of 371

-------
demand, actual torque varied throughout the warm-up test.  The original intent of the warm-up
test was to hold torque constant throughout the cycle, therefore another test was run using Speed
C  and a lower  torque  target (although still a  high  load point).  Ultimately, the  Steering
Committee elected to pool the data from both WOT and part-load tests.  This decision was made
at the July 27, 2006 meeting in Ann Arbor after reviewing the results from both tests.  Similar
tests were run for engines 2 and 3.

       According to the finalized procedure, two 30-minute warm-up tests were run with each
engine.  One test was run at C-speed and WOT, while the other test was performed at C-speed
and part load.  The target torque values  during the part load tests were set just low enough to
achieve constant torque  control throughout the 30-minute warm-up cycle. The part load tests
were conducted by starting the engine and promptly ramping to the target speed and load, which
was held constant for the remainder of the cycle.  The WOT tests were similar to the part load
tests, but the engines were ramped to the target speed  and WOT.  Using the recorded ECM speed
and fuel rate with the 40-point torque and BSFC maps, the interpolated torque and BSFC were
compared to the laboratory reference values.   Although the 40-point BSFC  map used fuel
consumption measurements from  the  laboratory fuel flow meter, BSFC  calculated from the
dilute emission  measurements was used as the  lab reference for  the warm-up tests.  The
laboratory fuel flow meter system has an inherent time lag that would have resulted in incorrect
reference BSFC  measurements during the transient warm-up test.  In addition, there was also
concern with the fuel flow measurement accuracy due to the density change of the fuel  during
the warm-up process.

       In order to achieve cold start temperatures of 0° C, an insulating box was built to enclose
the Caterpillar engine. The partially built enclosure is shown in Figure 86, while the completed
insulating box is shown  in Figure 87.  The enclosure surrounded both the Caterpillar engine as
well the exhaust  after treatment system.  A re-circulating alcohol refrigeration system was used
with dry ice to achieve a heat sink with temperature below 0° C.
SwRI Report 03.12024.06                 Page 137 of 371

-------
     FIGURE 86. CATERPILLAR C9 ENGINE PARTIALLY ENCLOSED IN THE
               INSULATING BOX PRIOR TO THE WARM-UP TEST
 FIGURE 87. CATERPILLAR C9 ENGINE FULLY ENCLOSED IN THE INSULATING
                      BOX PRIOR TO THE WARM-UP TEST
4.6.2  Interacting Parameters - Warm-Up Data Analysis

      The Test Plan did not initially include a method for how the data from the Interacting
Parameters Warm-up test would be used to generate an error surface.  There was considerable
Steering Committee discussion of the course of  several months regarding the appropriate
SwRI Report 03.12024.06
Page 138 of 371

-------
analysis of the data.  The methodology was tentatively established following the completion of
Engine 1 testing, and was later adjusted with the completion of Engine 2 testing.

       A number of concerns had to be balanced in the treatment of the warm-up data.  On the
one hand, it was necessary to attempt to use the warm-up data to capture a wide range of possible
variations in engine fluid temperatures and viscosities.  This was complicated by the fact that the
test was designed to explore cold temperatures and therefore only elevated viscosity levels. On
the other hand, there was a desire not to include any data that was not representative of operation
in the NTE zone.

       An additional complicating factor was due to the interpolation of torque  and BSFC from
the 40-point maps using ECM-broadcast speed and fuel rate.  Because the warm-up cycle target
speed and load set points did not match a mode from the 40-point maps, a  certain amount of
interpolation bias error was included in the data.  This error was accounted for elsewhere in the
Model; therefore  it was necessary to remove the bias due to the interpolation  process prior to
generating error surfaces.

       The data analysis method finally approved by the Steering Committee is described below.
Torque is used in the example, but  the same methodology is also applied to BSFC.  First, the
continuous  data for the warm-up test was assembled, including the interpolated torque which was
generated via post processing.  To remove the interpolation bias, data near the end of the warm-
up test, where all of the engine parameters had stabilized, was examined to generate an average
stabilized value  for both the reference  torque (from the laboratory  torque-meter) and the
interpolated torque  (based  on ECM-broadcast speed  and fuel rate).  These two values  were
compared in order to evaluate the interpolation bias error.  This offset was then applied to the
continuous  interpolated torque data set, shifting the data set to equalize the stabilized interpolated
torque values with  the reference  torque-meter values. An example of this  bias correction is
illustrated in Figure 88.
SwRI Report 03.12024.06                 Page 139 of 371

-------
     1850
     1800
     1750 -
   a 1700 -
   O"
   o
     1650
     1600
     1550
                          •Lab Torque N-m
                          •Bias Cor. Interp. Torque
                Start NTE Zone
                Interpolated Torque
                                 Bias Correction
                                 = + 7Nm
         0      200      400     600     800     1000     1200     1400    1600     1800     2000
                                           Time[s]

   FIGURE 88 EXAMPLE OF WARM-UP TEST BIAS CORRECTION, DDC ENGINE
                                  PART LOAD TEST

       Once the interpolation bias correction was complete, the temperature data was examined
to determine when the NTE zone was entered.  These entry points were based on the NTE zone
criteria given in CFR 40 Part 86.1370-2007.  The primary trigger common to all three engines
was the aftertreatment outlet temperature, which must be 250 °C or higher.  For Engines 1 and 3,
which were EGR equipped, additional trigger points are defined for engine coolant temperature
(ECT)  and intake manifold temperature (IMT), as given in CFR 40 Part 86.1370-2007.  An
example of the determination of NTE zone entry for Engine 1 is shown in Figure 89.
SwRI Report 03.12024.06
Page 140 of 371

-------
          DPF Inlet Temp degC
         •Actual ECT - CAN
-DPF Outlet Temp degC
-Target IMT
-Start NTE Zone
 Actual IMT-CAN
-Target ECT
    450
    400
    350 -
    300
  1- 200
                         DPF Outlet
                         Temp Crosses
                         250C
                                              IMT Crosses Target - Start
                                              NTE Zone @ 136 sec
    150 -
    100 -
                   50
                                100
                                             150
                                            Time[s]
                                                          200
                                                                       250
                                                                                    300
     FIGURE 89 EXAMPLE OF DETERMINATION OF NTE ZONE ENTRY FOR
                 WARMUP TEST, DDC ENGINE PART LOAD POINT

       The continuous data was then examined to determine the maximum difference between
the bias-corrected interpolated torque and the reference torque after entry into the NTE zone.  If
the difference resulted in a positive delta (interpolated minus reference), the value was set to the
95th  percentile  delta torque error value.   The negative of the same value was set to the 5th
percentile error value for that test.  If the delta from the data was negative, the value became the
-th
                                                                        th
5  percentile for the test, while it's positive, or mirror-mage, became the 95   percentile delta.
The 50th percentile error values for all warm-up tests were set to zero.  Torque was processed as
percent of maximum torque, while BSFC was calculated directly in engineering units.

       Temperature, torque, and BSFC data is shown for each warm-up test in Appendix H.
These plots show temperature profiles related to the NTE zone, bias corrected interpolated torque
with laboratory reference torque,  as  well as bias  corrected interpolated BSFC with laboratory
reference BSFC.

4.6.3   Interacting Parameters — Warm-Up Error Surface Generation

       Using the process outlined above,  torque and BSFC  errors were  calculated for each
warm-up test.  Table 52 shows the torque deltas,  while Table 53 summarizes the BSFC errors.
Torque errors for the Detroit Diesel  Series 60 and International VT 365 engines were  similar.
SwRI Report 03.12024.06
        Page 141 of 371

-------
The Caterpillar C9 engine, with a cold-start temperature near 0 °C, had significantly larger
torque deltas. BSFC errors were similar among all engines with the exception of the Caterpillar
C9 part load test. The Engine 2 BSFC error was over three times as large as the deltas from the
Engine 1 and 3 part load tests.
            TABLE 52. WARM-UP TEST TORQUE ERRORS SUMMARY
Engine
DDC HMD
CATMHD
INTLHD
Operating Point
C-Speed
WOT
78% Peak Torque
WOT
65% Peak Torque
WOT
73% Peak Torque
MEAN
5th % Torque Error
[% Peak Torque]
-3.4
-1.2
-14.2
-11.3
-3.5
-1.7
-5.9
50th % Torque Error
[% Peak Torque]
0.0
0.0
0.0
0.0
0.0
0.0
0.0
95th % Torque Error
[% Peak Torque]
3.4
1.2
14.2
11.3
3.5
1.7
5.9
              TABLE 53. WARM-UP TEST BSFC ERRORS SUMMARY
Engine
DDC HMD
CAT MHD
INT LHD
Operating Point
C-Speed
WOT
78% Peak Torque
WOT
65% Peak Torque
WOT
73% Peak Torque
MEAN
5th % BSFC Error
[g/kW-hr]
-4
-7
-5
-24
-6
-7
-9
50th % BSFC Error
[g/kW-hr]
0
0
0
0
0
0
0
95th % BSFC Error
[g/kW-hr]
4
7
5
24
6
7
9
       As decided by the Steering Committee, the mean values of the pooled torque and BSFC
deltas were used to create the final interacting parameters error surfaces.  Figure 90 shows the
final warm-up torque error surface, while the BSFC error surface is shown in Figure 91.  The
interacting parameters error surfaces are sampled normally and have a single x-axis point.  The
warm-up deltas will be applied to each torque and BSFC value from the reference NTE events,
independent of level.
SwRI Report 03.12024.06
Page 142 of 371

-------
                             2 -,
E
ra


O"
o
                             0 -
                            -2 J
                                 % Peak Torque Delta

                                       5.9
                                       0.0
                                       -5.9
                                  Single X-Axis Point
  FIGURE 90. ERROR SURFACE FOR INTERACTING PARAMETERS - WARM-UP
                                DELTA TORQUE
                              2 -,
                              1 -
                           O  0
                           u.
                             -1 -
                             -2 J
                                   BSFC Delta [g/kW-hr]

                                        9
                                        -9
                                   Single X-Axis Point
  FIGURE 91. ERROR SURFACE FOR INTERACTING PARAMETERS WARM-UP
                                  DELTA BSFC
SwRI Report 03.12024.06
       Page 143 of 371

-------
4.7    Torque and BSFC Interacting Parameters - Design of Experiment

       The objective of the interacting parameters DOE experiment was to evaluate torque and
BSFC  map  errors due to a number of variable engine parameters.   The list of parameters
included intake restriction, exhaust restriction, barometric pressure, and charge air cooler outlet
temperature.   Because the 40-point  maps were generated using nominal set  points for  the
parameters listed above, torque and BSFC values from  the  ECM interpolation would be
inaccurate due to engine parameter variations. The purpose of the interacting parameters DOE
was to compare the laboratory reference torque and BSFC with the interpolated values under a
broad range of  engine operation.   Ranges  of  adjustment for each parameter were defined
according to Table 54, which was copied from the Test Plan.

    TABLE 54.  INTERACTING PARAMETERS - DOE ADJUSTMENT GUIDANCE
Parameter
Intake air restriction
Exhaust gas restriction
Barometric pressure
Charge air cooler out
temperature
Minimum
Minimum capable*
Minimum capable*
82.7 kPa
Minimum per manufacturer
specifications and ambient
conditions**
Maximum
Max. allowed by
manufacturer*
Max. allowed by
manufacturer*
105 kPa
Maximum per manufacturer
specifications and ambient
conditions**
 Consider removing after treatment to extend range of restrictions
  Assume that a 1 deg. change in ambient temperature corresponds to a 1 deg. change in charge
air cooler out temperature

       Although the program was run in a test cell capable of simulated high altitudes, the cell
could not simulate altitudes lower  than approximately  689 feet.  Therefore,  the maximum
achievable  barometric pressure was near 99 kPa,  the  typical atmospheric pressure for San
Antonio.

       There was considerable Steering  Committee discussion about exhaust backpressure set
points, because DPFs will be used on all 2007 engines.  A final  decision was reached on the
March 27, 2006 conference call. The Steering Committee agreed that the backpressure set points
should represent the minimum backpressure with a  clean DPF installed,  and  the maximum
backpressure with a dirty DPF.  That maximum was defined as the highest level of backpressure
the  engine control  system would allow before triggering an active regeneration  based on DPF
differential pressure.  SwRI was directed to  obtain these values from the engine manufacturers
for each test engine.

       The interacting parameter DOE test was performed on Engine 1 and Engine 3 only. For
each engine, SwRI worked with the engine manufacturers and the Steering Committee to define
appropriate  adjustment ranges  according to the guidance given  in Table  54.  A Design  of
Experiment (DOE) test matrix (half factorial with resolution IV, 4 factors and 1 center point) was
used, resulting in nine test points. In addition, a tenth point was added by SwRI representing the
standard laboratory conditions  used  for  steady-state and transient testing. In some cases,  the
SwRI Report 03.12024.06
Page 144 of 371

-------
 standard conditions were not at the center point of the adjustment range.  The additional tenth
 point was not originally intended as part of the error surface generation, but was to be used for
 diagnostic and information purposes.  The generic DOE test matrix is given in Table 55, while
 specific set points used for each engine are given later in this section.

           TABLE 55. INTERACTING PARAMETERS - DOE TEST MATRIX
Run#
1 -BL
2
3
4
5
6
7
8
9
10
Intake Air
Restriction
Center
Center
Min
Possible
Max
Allowed
Min
Possible
Max
Allowed
Max
Allowed
Min
Possible
Min
Possible
Max
Allowed
Exhaust
Restriction
Center
Center
Max Dirty
DPF
Min Clean
DPF
Min Clean
DPF
Max Dirty
DPF
Min Clean
DPF
Max Dirty
DPF
Min Clean
DPF
Max Dirty
DPF
Barometer
[kPa]
99.0
90.7
82.6
82.6
99.0
99.0
99.0
99.0
82.6
82.6
Inlet Air
Temp
[degC]
24
24
29
29
37
37
Min
Possible
Min
Possible
Min
Possible
Min
Possible
1 - Min Possible Inlet Air Temp = 9°C to 1 0°C
2 - Charge Air Cooler set point is Inlet Air Temp + Manufacturers'
allowed temperature rise
3 - Barometer of 99.0 kPa is estimated, actual max value varied
slightly due to ambient conditions
       In order to allow for a larger range of adjustment of various parameters, the DPFs were
 removed from the exhaust, and the LFEs used for intake air flow measurement were removed
 from the intake air ducting.

       Each DOE test matrix point was evaluated at five different steady-state load points. The
 original mode definitions were given in the Test Plan as shown in Table 56.

          TABLE 56. TEST PLAN DOE ENGINE OPERATING CONDITIONS

	DOE Engine Operating Conditions (%speed, and %torque respectively)	
    17%, 32%
100%, 100%
59%, 49%
100%, 32%
100%, 100%
       During the course of initial DOE testing on Engine 1, it was observed that the 100 % load
 points were not repeatable because engine performance at WOT was not consistent across all of
 the DOE test conditions. Therefore, the WOT points were adjusted to a lower level where the
 torque set  points could  be maintained for all DOE tests.  The 100 % speed points were also
 SwRI Report 03.12024.06
                Page 145 of 371

-------
lowered slightly to 97  %  to  insure repeatable  load  points.   In addition, the modes were
lengthened to  10 minutes to allow  for complete stabilization of the fuel  flow measurement
system.  These adjustments were approved by the  Steering Committee during the May 23, 2006
meeting in Ann Arbor.

4.7.1  Interacting Parameters - DOE Data Analysis

      During the evaluation of the Engine 1 DOE data, a consistent bias  was evident in the
DOE torque and BSFC deltas for each test mode.  Even the DOE test run with nominal  engine
parameter set points showed a  significant bias.  It was found that the bias was the result of the
interpolation process which was used to generate  the ECM torque and BSFC values.  Because
interpolation error was already included in the error Model, the Steering Committee felt it was
necessary to remove the interpolation bias from the DOE error surface  data.   To address this
problem, SwRI proposed that the data from the additional baseline DOE test be used to generate
a bias correction for the nine DOE conditions. The correction was applied independently for
each of the five  test modes. The Steering Committee  approved  this approach at the May 23,
2006 meeting in Ann  Arbor.  An example of this bias correction for Engine 1 torque data is
given in Table 57. In this example, the bias correction results in an upward shift of 1.1% to all
DOE data for Mode  1.
      TABLE 57 EXAMPLE OF DOE BASELINE CORRECTION FOR ENGINE 1
DOE
Number
1 -Baseline
2
3
4
5
6
7
8
9
10
Delta Torq
Model
-1.1%
-1.0%
-0.2%
-0.6%
-0.9%
-0.1%
-1.2%
-1.2%
-1.3%
-0.4%
Mode 2
-1.8%
-1 .9%
2.1%
-1.1%
-1.0%
2.4%
-2.4%
-1 .6%
-2.6%
1.1%
ue (% of Peak Torque)
Mode3
-0.3%
-0.2%
0.4%
-0.1%
0.2%
1 .0%
-0.3%
-0.3%
-0.5%
0.0%
Mode 4
0.2%
0.3%
1 .3%
0.8%
0.6%
1 .3%
-0.2%
0.3%
0.0%
1.1%
ModeS
0.2%
0.9%
5.0%
2.6%
0.7%
3.9%
-0.6%
0.4%
0.4%
3.9%
Baseline Corrected Delta Torque (% of Peak Torque)
Model
N/A
0.1%
0.9%
0.5%
0.2%
1 .0%
-0.1%
-0.1%
-0.2%
0.7%
Mode 2
N/A
-0.1%
3.8%
0.7%
0.8%
4.2%
-0.6%
0.2%
-0.8%
2.9%
Mode3
N/A
0.0%
0.6%
0.1%
0.5%
1 .2%
-0.1%
0.0%
-0.3%
0.3%
Mode 4
N/A
0.1%
1 .2%
0.6%
0.4%
1 .2%
-0.4%
0.1%
-0.2%
1 .0%
ModeS
N/A
0.7%
4.8%
2.4%
0.5%
3.7%
-0.8%
0.2%
0.2%
3.7%
4.7.2   Engine 1 Detroit Diesel Series 60 DOE

       The DOE matrix  was run  several times on the Detroit  Diesel  Series 60 engine  as
adjustments were made to the test  methodology. These changes  were in response to the test
results and subsequent Steering Committee decisions.  The final speed and torque set points used
for Engine 1 are given in Table 58.  As noted above, these final points were different from those
used during the initial DOE run on this engine, due to the need to maintain the same torque level
for all DOE test conditions.
SwRI Report 03.12024.06
Page 146 of 371

-------
 TABLE 58. INTERACTING PARAMETERS - DOE SPEED AND TORQUE STEADY-
                    STATE MODE DEFINITION FOR ENGINE 1

Model
Mode 2
ModeS
Mode 4
Mode 5
Speed
[% NTE]
17%
17%
59%
97%
97%
Torque
[% Peak]
43%
94%
49%
32%
71%
      The actual engine parameter set points used for the Engine 1 DOE test matrix are given in
Table 59.  The charge air cooler set point temperatures were based on a specification of 28 °C
temperature rise from ambient (inlet air) temperature.  For inlet air temperatures at 10 °C, the
inlet air dew point temperature was lowered to 7 °C, rather than the standard set point of 15 °C,
in order to prevent condensation in the intake air stream.

  TABLE 59. INTERACTING PARAMETERS - DOE TEST MATRIX FOR ENGINE 1

DOE
Number
1-BL
2
3
4
5
6
7
8
9
10
Intake Air
Restriction1
[kPa]
4.0
4.0
1.5
5.0
1.5
5.0
5.0
1.5
1.5
5.0
Exhaust
Restriction2
[kPa]
17
17
30
12
12
30
12
30
12
30
CVS
Pressure3
[kPa]
99
91
83
83
99
99
99
99
83
83
Boost After
Temp4
[°C]
52
52
57
57
64
64
38
38
38
38
Inlet Air
Temp5
[°C]
24
24
29
29
37
37
10
10
10
10
Dew Point
Temperature
[°C]
15
15
15
15
15
15
7
7
7
7
Notes:
1 . Minimum achievable intake air restriction was 1 .5 kPa
2. Maximum dirty DPF restriction was 30 kPa - Minimum clean DPF restriction was 12 kPa
3. Maximum achievable CVS pressure was 99 kPa
4. Temperature was set based on a fixed offset from the inlet air temperature
5. Minimum achievable inlet air temperature was 10 °C
      The final baseline corrected errors for Engine 1 are given in Table 60 and Table 61 for
Torque and BSFC, respectively.
SwRI Report 03.12024.06
Page 147 of 371

-------
  TABLE 60. INTERACTING PARAMETERS - DOE ENGINE 1 BIAS CORRECTED
                                TORQUE DELTAS
DOE
Number
1-Baseline
2
3
4
5
6
7
8
9
10
Baseline Corrected Delta Torque (% of Peak Torque)
Model
N/A
0.1%
0.9%
0.5%
0.2%
1 .0%
-0.1%
-0.1%
-0.2%
0.7%
Mode 2
N/A
-0.1%
3.8%
0.7%
0.8%
4.2%
-0.6%
0.2%
-0.8%
2.9%
ModeS
N/A
0.0%
0.6%
0.1%
0.5%
1 .2%
-0.1%
0.0%
-0.3%
0.3%
Mode 4
N/A
0.1%
1 .2%
0.6%
0.4%
1 .2%
-0.4%
0.1%
-0.2%
1 .0%
ModeS
N/A
0.7%
4.8%
2.4%
0.5%
3.7%
-0.8%
0.2%
0.2%
3.7%
  TABLE 61. INTERACTING PARAMETERS - DOE ENGINE 1 BIAS CORRECTED
                                  BSFC DELTAS
DOE
Number
1-Baseline
2
3
4
5
6
7
8
9
10
Baseline Corrected Delta BSFC Fuel Flow (g/kW-h)
Model
N/A
-2
-8
-5
-2
-7
2
1
1
-6
Mode 2
N/A
-1
-11
-3
-2
-14
3
-1
2
-7
Mode 3
N/A
-1
-6
-3
-2
-8
2
0
1
-5
Mode 4
N/A
-1
-9
-3
-2
-9
3
-1
1
-6
Mode 5
N/A
-3
-13
-8
0
-10
3
0
-1
-10
4.7.3  Engine 3 International VT365 DOE

      The torque and speed set points used for Engine 3 are given in Table 62.  Following the
direction of the Steering Committee,  the points were selected to be identical to  those used for
Engine 1.  However, due to the shape of the torque curve for Engine 3, Mode 5 could not be run
at the desired combination of 97 %  NTE  speed and 71 % of maximum torque.  In order to
maintain a consistent load point for use in the  error surface, the speed set point was adjusted
down to the highest speed at which 71  % percent of maximum torque could be reliably
maintained at all DOE conditions.  The mode 5 target speed was therefore adjusted from 97 %
NTE speed to 85 % NTE speed.
SwRI Report 03.12024.06
Page 148 of 371

-------
 TABLE 62. INTERACTING PARAMETERS - DOE SPEED AND TORQUE STEADY-
                    STATE MODE DEFINITION FOR ENGINE 3

Model
Mode 2
ModeS
Mode 4
Mode 5
Speed
[% NTE]
17%
17%
59%
97%
85%
Torque
[% Peak]
43%
94%
49%
32%
71%
      The engine parameter set points used for the Engine 3 DOE testing are given in Table 59.
Similar to Engine 1 testing, the engine manufacturer of Engine 3 was consulted to determine
appropriate set points for the DOE test matrix.

  TABLE 63. INTERACTING PARAMETERS - DOE TEST MATRIX FOR ENGINE 3

DOE
Number
1-BL
2
3
4
5
6
7
8
9
10
Intake Air
Restriction1
[kPa]
3.5
3.5
0.7
3.7
0.7
3.7
3.7
0.7
0.7
3.7
Exhaust
Restriction2
[kPa]
17
17
24
12
12
24
12
24
12
24
CVS
Pressure3
[kPa]
99
91
83
83
99
99
99
99
83
83
Boost After
Temp4
[°C]
39
39
45
45
52
52
31
31
31
31
Inlet Air
Temp5
[°C]
24
24
29
29
37
37
10
10
10
10
Dew Point
Temperature
[°C]
15
15
15
15
15
15
7
7
7
7
Notes:
1 . Minimum achievable intake air restriction was 0.7 kPa
2. Maximum dirty DPF restriction was 24 kPa - Minimum clean DPF restriction was 12 kPa
3. Maximum achievable CVS pressure was 99 kPa
4. Temperature was set based on a fixed offset from the inlet air temperature
5. Minimum achievable inlet air temperature was 10 °C
      The final baseline corrected data for Engine 3 is given in Table 64 and Table 65 for
Torque and BSFC, respectively.
SwRI Report 03.12024.06
Page 149 of 371

-------
   TABLE 64. INTERACTING PARAMETERS - DOE ENGINE 3 BIAS CORRECTED
                                  TORQUE DELTAS
DOE
Number
1-Baseline
2
3
4
5
6
7
8
9
10
Baseline Corrected Delta Torque (% of Peak Torque)
Model
N/A
0.5%
1 .9%
1.1%
0.7%
1 .0%
0.4%
1 .0%
1 .6%
1 .8%
Mode 2
N/A
2.4%
4.3%
3.7%
1 .8%
3.8%
-1 .5%
0.0%
0.9%
3.9%
ModeS
N/A
0.1%
-0.1%
-1.1%
0.4%
1 .4%
-0.8%
0.6%
-1 .5%
-0.8%
Mode 4
N/A
0.3%
1 .5%
1.1%
0.7%
1 .5%
0.4%
0.9%
1 .0%
1 .8%
ModeS
N/A
0.9%
1 .2%
0.9%
0.2%
2.4%
-0.6%
0.4%
-1 .2%
1 .7%
   TABLE 65. INTERACTING PARAMETERS -DOE ENGINE 3 BIAS CORRECTED
                                    BSFC DELTAS
DOE
Number
1-Baseline
2
3
4
5
6
7
8
9
10
Baseline Corrected Delta BSFC Fuel Flow (
Model
N/A
-3
-6
-3
-6
-8
-1
-4
-3
-6
Mode 2
N/A
-5
-9
-7
-4
-9
3
0
0
-7
ModeS
N/A
-2
-4
0
-3
-10
2
-4
3
-2
Mode 4
N/A
-3
-12
-9
-7
-14
-3
-8
-7
-15
3/kW-h)
ModeS
N/A
-3
-7
-5
-2
-11
3
-2
2
-8
4.7.4   Interacting Parameters - DOE Error Surface Generation

       To generate the interacting parameters DOE error surfaces, the baseline corrected error
data for each mode was evaluated to generate a 5th, 50th, and 95th percentile delta across all nine
DOE  conditions.  The errors captured during this experiment included bias errors as well as
precision errors.  However, the dynamic torque and BSFC error  surfaces generated  during
transient engine testing captured the precision errors associated with the interpolation process of
torque and BSFC. Not wanting to double count error sources in the Model, the variability of the
interpolation process was removed from the interacting  parameters DOE error surfaces. This
was accomplished by shrinking the 5th, 50th, and  95th percentile delta values by the interpolated
torque and  BSFC variance  experienced during steady-state testing.  The  steady-state variance
was calculated as the  mean of the 10 steady-state interpolated torque and BSFC  MAD values
over the 20 repeats. The mean of the 10 MAD values was then used to collapse the raw DOE
error data.  After the variance correction, the delta percentiles were then plotted with the x-axis
values calculated as the mean modal value.
SwRI Report 03.12024.06
Page 150 of 371

-------
      The error correction due to the removal of the steady-state variance was minimal due to
the relatively small steady-state torque and BSFC MAD values. The mean interpolated torque
MAD values were 0.1 % of peak torque for Engine 1 and 0.3  % of peak torque for Engine 3.
The mean BSFC MAD values were 0.2 g/(kW-hr) for both Engine 1 and Engine 3.  An example
of the MAD correction is show in Table 66.  The bias corrected deltas, shown on the left, are
collapsed using the mean MAD value to generate the final MAD corrected deltas.

   TABLE 66. EXAMPLE DOE STEADY-STATE VARIANCE CORRECTION USING
                               THE MEAN SS MAD
Bias Corrected
PEMS vs Lab
Delta

5th percentile
[ g/kW-hr]
-13.1
-7.6
-7.1
-11.9
-9.3
Bias Corrected
PEMS vs Lab
Delta

50th percentile
[ g/kW-hr]
-2.4
-1.7
-2.1
-2.7
-2.3
Bias Corrected
PEMS vs Lab
Delta

95th percentile
[ g/kW-hr]
2.8
1.8
1.5
1.8
2.0


SS BSFC
Mean of 10
MADs
[ g/kW-hr]
0.2
0.2
0.2
0.2
0.2
MAD Corrected
PEMSvsLab
Delta

5th percentile
[ g/kW-hr]
-12.9
-7.4
-6.9
-11.7
-9.1
MAD Corrected
PEMSvsLab
Delta

50th percentile
[ g/kW-hr]
-2.2
-1.5
-1.9
-2.5
-2.1
MAD Corrected
PEMSvsLab
Delta

95th percentile
[ g/kW-hr]
2.6
1.6
1.3
1.6
1.8
      Figure 92 and Figure 93 show the corrected DOE torque and BSFC error surface data for
Engine 1.
                                     % Peak Torque Lab

 FIGURE 92. ERROR SURFACE FOR INTERACTING PARAMETERS - DOE ENGINE
                                1 DELTA TORQUE
SwRI Report 03.12024.06
Page 151 of 371

-------
                          -95th percentile
    -50th percentile (median)
-5th percentile
     -14
                                        BSFC Lab [g/kW-hr]
 FIGURE 93. ERROR SURFACE FOR INTERACTING PARAMETERS - DOE ENGINE
                                    1 DELTA BSFC


       Figure 94 and Figure 95 show the corrected interacting parameters  DOE torque and
BSFC error surface data for Engine 3.
SwRI Report 03.12024.06
Page 152 of 371

-------
                                                                              100.0
                                     % Peak Torque Lab



FIGURE 94. ERROR SURFACE FOR INTERACTING PARAMETERS - DOE ENGINE

                                3 DELTA TORQUE
                         -95th percentile —•—50th percentile (median) —•—5th percentile
  I
  s
                                       BSFC Lab [g/kW-hr]
FIGURE 95. ERROR SURFACE FOR INTERACTING PARAMETERS - DOE ENGINE

                                  3 DELTA BSFC
SwRI Report 03.12024.06
Page 153 of 371

-------
       To generate the final DOE error surfaces, the torque and BSFC errors from the Engine 1
and Engine 2 DOE matrix were pooled.  The 5th, 50th, and  95th percentile deltas were  then
calculated from the pooled data  set.  The variance correction was accomplished by calculating
the mean of the pooled steady-state interpolated torque and BSFC MAD values from Engine 1
and Engine 3.  The mean MAD  values for the pooled data were 0.2 % of peak torque and 0.5
g/(kW-hr) for BSFC. The final  torque and BSFC error surfaces for the interacting parameters
DOE  testing are shown in Figure 96 and  Figure 97.  The DOE error surfaces are sampled
normally.   Having  a broad range of x-axis torque and BSFC values, errors  are  linearly
interpolated from these error surfaces based on level.
                                                                                  1(10
                                      % Peak Torque Lab

 FIGURE 96. ERROR SURFACE FOR INTERACTING PARAMETERS - DOE DELTA
                                  TORQUE FINAL
SwRI Report 03.12024.06
Page 154 of 371

-------
-95th percentile
    50th percentile (median)
                                                           5th percentile
  I
  s
                                                                                  2! 10
                                         BSFC Lab [g/kW-hr]
 FIGURE 97. ERROR SURFACE FOR INTERACTING PARAMETERS - DOE DELTA
                                    BSFC FINAL
4.8    Torque and BSFC Independent Parameters Sensitivity Analysis

       The independent parameters test was conducted to evaluate torque and BSFC map errors
due to variations in intake air humidity and fuel properties.  The Steering Committee had an
option to add additional parameters into this matrix,  but other parameters were not added.  This
test was performed only using Engine 2.  The Test Plan called for SwRI to run a sensitivity
analysis using the parameters given in Table 67.

 TABLE 67. INDEPENDENT PARAMETERS ADJUSTMENT GUIDANCE FROM THE
                                     TEST PLAN
Sensitivity Parameter Set Points
Parameter
Intake air
humidity
Fuel properties
Minimum (#1)
Minimum possible (@30
deg. C); 0 grains/lb dry air
Fuel used in program
Mid. (#2)
50%RH(@30deg. C);
95 grains/lb dry air
Fuel selection #2
Maximum (#3)
95% RH (@30 deg. C)*;
180 grains/lb dry air
California ULSD
 Run charge air cooler water inlet temperature of 30 deg. C

       At each test condition, three steady-state modes were run according to the direction given
in the Test Plan and as shown in Table 68.  Note that original mode definitions are given as NTE
percent speed and percent torque at speed.
SwRI Report 03.12024.06
Page 155 of 371

-------
 TABLE 68. INDEPENDENT PARAMETERS TEST MODES FROM THE TEST PLAN
          Sensitivity Engine Operating Conditions (%speed, and %torque respectively)
            17%, 32%
   59%, 49%
100%, 100%
       Although the original Test Plan called for only running selected conditions across the
three modes, SwRI determined that the there was little difficulty in changing test conditions once
the test apparatus was set up and the fuels were procured.  Therefore, SwRI elected to run a
complete test matrix for a total of nine test conditions.

       As with the Interacting Parameters DOE, it was necessary to adjust the final test modes
slightly from those given in the Test Plan to position the points in the NTE zone and to insure the
target torque values could be maintained at all of the test conditions.  The torque value for mode
1 was increased to insure the mode 1  power was always above the NTE limit of 30 % maximum
power.  The speed and torque set points for mode 3 were decreased to pull the point away from
the governor line, thus insuring the mode was repeatable throughout the independent parameters
testing.  Table  69 shows  the final  three modes  of the steady-state test cycle run  for  the
independent parameters testing.  The mode length  was set to 10-minutes to insure stable fuel
flow measurement for the BSFC error surfaces.

 TABLE 69. INDEPENDENT PARAMETERS SPEED AND TORQUE STEADY-STATE
                                MODE DEFINITIONS

Model
Mode 2
Mode 3
Speed
[% NTE]
17%
59%
97%
Torque
[% Peak]
43%
49%
56%
       It was not possible to perform testing using an exhaustive matrix of fuel properties during
this program. Therefore, the Test Plan called for three fuels to represent a range of potential fuel
properties that might be available in the field. The first two fuels were specified in the Test Plan.
The first was the base ULSD 2-D certification  grade fuel used during the program, while the
second was to be a representative California ULSD fuel.  SwRI procured several drums of BP
ECD-1 ULSD fuel from California to meet this requirement.  The third test fuel was to be
selected by the  Steering Committee.  Initially, SwRI proposed a very low aromatic (less  than
10% by volume) fuel, but the Steering Committee felt that a high aromatic fuel would be more
representative of fuels available in the northern  and eastern parts of the U.S. Therefore, SwRI
located a low API gravity ULSD test fuel from Chevron Phillips, which was selected as the third
test fuel by the Steering Committee.  A summary  of selected fuel properties for the three
Independent Parameters test fuels is given in Table 70.
SwRI Report 03.12024.06
Page 156 of 371

-------
  TABLE 70. SELECTED FUEL PROPERTIES FOR INDEPENDENT PARAMETERS
                                      TESTING
Property
Aromatics
Cetane Number
Viscosity
API Gravity
Sulfur
Distillation
10%
90%
Units
% vol

cSt @ 40C

ppm

deg F
deg F
Description
Test Fuels
Base Fuel
29.5
44.4
2.5
35.2
10

214
311
Haltermann
EPA 2-D Cert fuel
GARB ULSD
24.5
54.3
2.4
38.8
3

206
321
BP
EC Diesel-1
Third Fuel
35.0
40.4
2.6
33.1
6.2

207
344
Chevron Phillips
Low API ULSD
       The mean intake air humidity levels recorded during testing were 4 gr/lb for the low
humidity points, 90 gr/lb for the middle humidity points, and 192 gr/lb for the high humidity
points. In order to reach the near zero humidity levels requested in the Test Plan, a specialized
humidity control system had to be used to condition the engine intake air. This custom-designed
system is incorporated into the intake air stream of the test cell at SwRI on an as needed basis in
order to achieve very low humidity levels, while imposing a very low additional restriction on
the intake air system.  Shown in Figure 98, the humidity control  system employs a large bed of
desiccant that is used to remove water from the intake air, and incorporates bypass legs and post-
bed  cooling heat exchangers to maintain the desired  intake air temperature and humidity
conditions.
SwRI Report 03.12024.06
Page 157 of 371

-------
          FIGURE 98. INTAKE AIR LOW HUMIDITY CONTROL SYSTEM
4.8.1   Independent Parameters Data A nalysis

       For the interacting parameters testing, ECM speed and ECM fuel rate were used with the
40-point maps to interpolate modal torque and BSFC values.  These values were compared to the
laboratory reference measurements.  As with the 40-point BSFC map, the reference BSFC was
calculated using the laboratory fuel flow meter.  As expected, there were errors  inherent in the
interpolation process using the 40-point maps.  Because the interpolation errors were accounted
for in the Model, the  Steering Committee elected to remove the interpolation bias  errors.  A
process was used similar that used for the Interacting Parameters DOE, wherein all data values
were bias corrected using the error  values from a baseline condition.  The baseline condition
chosen for the Independent Parameters test was Test Fuel  1 with normal 95 gr/lb humidity level.
Corrections were performed on a mode-by-mode basis.

       Shown in  Table 71 are the bias corrected interpolated torques versus laboratory torque
delta values for the Independent Parameters testing.   Table 72 contains  the  bias corrected
interpolated BSFC versus laboratory BSFC delta values.
SwRI Report 03.12024.06
Page 158 of 371

-------
 TABLE 71. INDEPENDENT PARAMETERS BIAS CORRECTED TORQUE DELTAS
Intake
Humidity
Min
Norm
High
Fuel
1
2
3
1
2
3
1
2
3
Delta Torque Bias Corrected (% of Peak Torque)
Model
-0.5%
0.8%
-1.1%
0.0%
1 .0%
-1 .0%
-0.2%
1 .9%
0.0%
Mode 2
-0.3%
1.1%
-0.9%
0.0%
1 .2%
-0.9%
0.0%
2.3%
0.1%
ModeS
-0.4%
1 .2%
-0.9%
0.0%
1 .7%
-1.1%
-0.3%
1 .7%
-0.7%
   TABLE 72. INDEPENDENT PARAMETERS BIAS CORRECTED BSFC DELTAS
Intake
Humidity
Min
Norm
High
Fuel
1
2
3
1
2
3
1
2
3
Delta BSFC Bias Corrected (g/kW-h)
Model
3.5
2.3
0.5
0.0
2.8
0.6
-0.7
-4.1
-5.0
Mode 2
2.3
2.2
0.9
0.0
0.9
1.9
-0.7
-4.7
-4.0
ModeS
3.0
1.8
2.1
0.0
0.3
2.7
0.8
-2.3
-0.5
4.8.2   Independent Parameters Error Surface Generation

       The Test Plan originally called  for separate error surfaces to be generated for fuel and
humidity.  However, while a clear trend was apparent with test fuel, no trends could be observed
related to humidity.   Therefore, SwRI proposed  that all the data be pooled  into a single
Independent Parameters error surface each for torque and BSFC.  Furthermore, the final torque
values for the three modes actually represented a relatively narrow range of torque, therefore no
trend in the data based on torque level could be determine. As a result, SwRI suggested that the
data for all three modes be pooled, and further that  no x-axis be used on the final error surface.
The Steering Committee approved these changes during the November 21, 2006 conference call.
As a result, the overall 5th, 50th, and 95th percentiles were calculated from the pooled data sets
shown in Table 71 and Table 72.

       Figure 99 shows the delta torque error surface for the independent  parameters testing.
With only one x-axis point, the normally sampled delta torque values were applied to each torque
value in the reference NTE events in the Model.
SwRI Report 03.12024.06
Page 159 of 371

-------
                            2 -,
                            1 -
                            0 -
                            -1 -
                            -2 J
                                   Delta Torque [% of Peak]


                                          1.8
                                          0.0
                                          -1.0
                                     Single X-Axis Point
    FIGURE 99. ERROR SURFACE FOR INDEPENDENT PARAMETERS DELTA
                                       TORQUE

       Shown in Figure 100 is the delta BSFC error  surface for the independent parameters
testing.  Similar to the delta torque error surface, the BSFC surface was collapsed to a single x-
axis point.  The pooled 5th, 50th, and 95th percentile values will be normally sampled and applied
to each reference NTE event BSFC value.
SwRI Report 03.12024.06
Page 160 of 371

-------
                             2 -,
                             1 -
o
u_

m
                             0 -
                             -1 -
                             -2 J
                                     BSFC Delta [g/kW-hr]

                                           3.0
                                           0.6
                                           -4.5
                                      Single X-Axis Point
 FIGURE 100. ERROR SURFACE FOR INDEPENDENT PARAMETERS DELTA BSFC

4.9    Torque and BSFC Interpolation Errors

       During the design of the Test Plan,  it was determined that a 40-point speed and fuel rate
matrix would be used to define an interpolation surface to predict Torque and BSFC from CAN
Speed and CAN Fuel Rate.  While  the 40-point matrix was used throughout the program, the
Steering Committee decided the 40-point matrix was too dense, placing an excessive mapping
burden on engine manufacturers.  The Steering Committee determined that a 20-point matrix
would be more  typical  of actual field testing.   However,  the smaller matrix would lead to
increased interpolation errors.

       The interpolation error surfaces were designed to  capture the incremental  error involved
in dropping from an interpolation surface based on a 40-point test matrix to one based on a 20-
point test matrix. The generation of these error surfaces was a computational exercise carried out
using the initial 40-point steady-state map data.  For each  engine, the Steering Committee down-
selected 20 points from the original 40 to generate the coarser grid. The 20-point maps selected
by the Steering Committee are shown in Figure 101 through Figure 103 for Engine 1,  Engine 2,
and Engine 3, respectively.
SwRI Report 03.12024.06
          Page 161 of 371

-------
   1800
      500     700     900    1100     1300     1500     1700     1900     2100    2300    2500
   FIGURE 101. DETROIT DIESEL SERIES 60 DOWN SELECTED 20-POINT MAP
   O"

onn
600
400


n
5C


'"/if \
^-^^

I

)0 1000
Selected Points for 20-Point Map
* ^^
• • • «
* * * * 1
^^~~~~~£} 0 9 0 9
^~~~— — A A A
»•$$;*•
1
1 500 2000 2500
                                       Speed [rpm]
        FIGURE 102. CATERPILLAR C9 DOWN SELECTED 20-POINT MAP
SwRI Report 03.12024.06
Page 162 of 371

-------
     600
     500
     400
     300
   O"
   o
     200
     100
                                                Selected Points for 20-Point Map
                                           r^t^t
       500
                  1000
                              1500
                                          2000
                                       Speed [rpm]
                                                     2500
                                                                 \'
                                                                 3000
                                                                             3500
     FIGURE 103. INTERNATIONAL VT365 DOWN SELECTED 20-POINT MAP

      For  each  engine  a matrix  of several thousand CAN-Speed and CAN-Fuel Rate
combinations was run using both 40-point and 20-point interpolation surfaces. The interpolated
torque and BSFC values were compared to generate the final deltas, with the 20-point values
subtracted from the 40-point values.  An example of the results from this computational exercise
is shown in Figure 104 for Engine 1 interpolated torque.  The results for all three engines are
given in Appendix I.
SwRI Report 03.12024.06
Page 163 of 371

-------
           F
           u
           e
           I
           r
           a
           t
           e
80-
60-
40-
                 20-
                     1200   1400   1600   1800   2000
                                   Sfceed rpm
    1.3466
    0.7684
    0.1902
   -0.3880
   -0.9662

   -1.5444
   -2.1226
   -2.7007
   -3.2789
   -3.8571
Delta Tore
  FIGURE 104. INTERPOLATED TORQUE ERROR (% PEAK TORQUE) BY SPEED
                   (RPM) AND FUEL RATE (G/S) FOR ENGINE #1
4.9.1  Interpolation Error Surface Generation

     To generate the final error surfaces, the interpolated torque and BSFC 20-point versus 40-
point delta data was pooled for each engine, and 5th, 50th,  and 95th percentile values were
generated. The percentile values for each engine were then averaged to generate the final deltas
for interpolation error surfaces.  The error surfaces do not have an x-axis, as the interpolation
error was not found to scale with either speed or fuel rate, but remained relatively constant across
the entire performance map for each engine.

     For torque the error surfaces are expressed as percent of maximum engine torque, while
BSFC errors are given in engineering units of g/(kW-hr). Each of these surfaces is sampled
normally, once per NTE event.  The final error surface values are given in Table 73 and Table 74
for torque and BSFC, respectively. The error surfaces are depicted in Figure 105 and Figure 106
for torque and BSFC, respectively.
SwRI Report 03.12024.06
                 Page 164 of 371

-------
       TABLE 73. TORQUE INTERPOLATION ERROR SURFACE VALUES
Engine
1
2
O
Number of
Points
8944
5741
5197
Averaged
Percentiles
5th
-0.82 %
-0.84 %
-1.00%
-0.89 %
50th
0.00 %
0.16%
0.01 %
0.06 %
95th
0.80%
2.55 %
1.34%
1.57 %
                             2 -,
                             1 -
                          o
                          Q.
                          HI
                          3
                          0"


                          .2
                            -1 -
                            -2 J
% Peak Torque Delta




     1.57










     0.06










     -0.89
                                Single X-Axis Point
          FIGURE 105. TORQUE INTERPOLATION ERROR SURFACE
         TABLE 74. BSFC INTERPOLATION ERROR SURFACE VALUES
Engine
1
2
O
Number of
Points
8944
5741
5197
Averaged
Percentiles
5th
-3.96
-3.27
-0.12
-2.45
50th
0.49
0.05
1.98
0.84
95th
8.60
7.89
10.94
9.14
SwRI Report 03.12024.06
Page 165 of 371

-------
                              2 -,
                               1 -
                            o
                            m
                              0 -
                              -1 -
                                     BSFC Delta [g/kW-hr]


                                          9.14
                                          0.84
                                          -2.45
                              -2 J
                                     Single X-Axis Point

              FIGURE 106.  BSFC INTERPOLATION ERROR SURFACE
4.10   Exhaust Flow Meter Testing

       Exhaust flow  meter testing was performed  to  evaluate potential bias errors  due to
installation related factors.  Flow meter testing included an exhaust pulsation test, an exhaust
swirl testing, and an exhaust tailpipe wind test. Using the Detroit Diesel Series 60 and 5-inch
EFM, steady-state tests were conducted to compare the SEMTECH-DS EFM flow rate to the
laboratory flow rate.

       The ten steady-state points tested during the flow meter testing were identical to the
modes selected for Engine 1 steady-state repeat testing. The laboratory flow rate was determined
using a LFE to measure  the intake air flow,  a Micro-Motion fuel flow meter to measure fuel
flow, and the laboratory analyzers to measure raw exhaust emission concentrations. The intake
LFE measurement and the raw chemical balance were used with  equation 1065.655-14 to
calculate the reference exhaust flow rate. As a check, the LFE air flow rate and measured fuel
flow were also used to calculate the exhaust flow rate using the CFR Part 89 raw exhaust flow
rate calculation. The two laboratory exhaust flow rate calculation methods gave nearly identical
exhaust flow rate results.  The  raw carbon balance error was modally calculated to insure the
laboratory reference exhaust flow rate was accurate to  within two percent.

4.10.1  Pulsation Test

       The pulsation test was performed to evaluate  the bias and precision of the PEMS flow
meters when  subjected to large  pressure pulsations in the exhaust system.  To conduct this test,
SwRI Report 03.12024.06
Page 166 of 371

-------
the DPFs were removed from the exhaust system, so that pulsations in the exhaust would not be
damped by its presence. The EFM was mounted 2 to 3 meters downstream of the turbocharger
outlet.  Exhaust pipes, with lengths exceeding 10-diameters, were mounted before and after the
EFM.  The exhaust was routed out the large overhead door of the laboratory, and was therefore
vented directly to the atmosphere.

       The 10-mode steady-state test was repeated 5  times.  The pooled SEMTECH-DS EFM
flow rate deltas versus the laboratory reference flows are shown in Figure 107. The PEMS flow
meters were biased high during the pulsation testing.
     45
     40


     35


     30


     25


     20


     15


     10


      5


      0
            15th % A 50th %  95th '
 A
-*•
       140
              240
                      340
                             440
                                     540
                                            640
                                                    740
                                                            840
                                                                   940
                                                                          1040
                                                                                  1140
                                Lab Reference Exhaust Flow Rate [scfm]
 FIGURE 107. PULSATION TEST SEMTECH-DS EXHAUST FLOW RATE DELTAS -
                                     RAW DATA

       In order to avoid double counting exhaust flow errors, the bias recorded during steady-
state testing was subtracted from the pulsation test data.  Figure 108 shows the EFM errors after
the mean steady-state exhaust flow rate bias was removed from the flow rate deltas. The steady-
state bias correction yielded a more uniform, positive exhaust flow rate bias.  It should be noted
that, because this experiment was  conducted using the 5-inch EFM, this steady-state bias
correction was small, because only a small amount of bias was observed during steady-state tests
involving the 5-inch flow meters (see Section 4.4.1 above).
SwRI Report 03.12024.06
                       Page 167 of 371

-------
     45
     40
     35
   I 30
   I 25

   o
     20
     15
            15th % A 50th % *95th %
       140     240      340     440      540     640      740     840
                                Lab Reference Exhaust Flow Rate [scfm]
                                                                   940
                                                                          1040
                                                                                 1140
  FIGURE 108. PULSATION TEST SEMTECH-DS EXHAUST FLOW RATE DELTA -
                      CORRECTED FOR STEADY-STATE BIAS

4.10.2  Pulsation Error Surf ace Generation

       Using the  bias corrected EFM data, an exhaust flow rate pulsation error surface was
constructed for use in the Monte Carlo Model.  The exhaust flow rate data was normalized using
the EFM maximum flow rate specification.  In the case of the 5-inch EFM, the maximum flow
rate was  1700 scfm.  Shown in Figure 109 are the flow rate delta values that were used in the
Model.  Using normalized flow rate data from the reference NTE events, a flow rate delta was
normally sampled from the error surface.  Linear interpolation was used for NTE reference
points  within the data set.  Points outside the data  set were  determined using  the data set
maximum or minimum values, with no extrapolation beyond the values generated during testing.
The pulsation error surface generated positive exhaust flow errors when used in the Model.
SwRI Report 03.12024.06
Page 168 of 371

-------
  o
  £
  
      2.5%
      2.0% -
1.5% -
    |  1.0%--
  O

  re
  a
  HI
  Q
0.5% -
      0.0%
                          -95th percentile
                                   -50th percentile (median)
-5th percentile
         10%
                     20%
                                  30%          40%          50%
                                    mol/s (lab,nom) / mol/s_max - [%]
                                                                      60%
                                                                                   70%
     FIGURE 109. ERROR SURFACE FOR PULSATION EXHAUST FLOW RATE

4.10.3 Non-Uniform Velocity Profile Swirl Test

       The swirl test was conducted to evaluate PEMS flow meter errors when the EFM was
subjected to non-uniform flow velocity profiles upstream  of the EFM.  Two short radius  90°
elbows were connected in perpendicular planes to introduce exhaust swirl before the inlet of the
PEMS flow meter.  The engine after-treatment system was installed during the swirl test. The
swirl  exhaust  system was  also vented out the overhead  door of the laboratory, directly to
atmosphere following the EFM.

       Five repeats of the 10-mode steady-state test were run to characterize the EFM error due
to swirl.  The pooled SEMTECH-DS  EFM flow rate deltas versus the laboratory reference flows
for the swirl test are shown in Figure 110. The PEMS flow meter errors appeared to show a level
dependence, with increasing errors as  flow rate increased.
SwRI Report 03.12024.06
                              Page 169 of 371

-------
     30
     25
     20
     15
     10 -
      5
      -5
     -10
            15th % A 50th % *95th '
                •

            •   A
                    200
                                              600
                                                           800
                                                                        1000
                                                                                    1200
                      t
                                 Lab Reference Exhaust Flow Rate [scfm]
 FIGURE 110. SWIRL TEST SEMTECH-DS EXHAUST FLOW RATE DELTAS - RAW
                                         DATA

       In order to avoid  double  counting exhaust flow errors, the bias recorded during steady-
state testing was subtracted from the swirl test data. Figure 111 shows the swirl test EFM errors
after the mean steady-state exhaust flow rate bias was removed from the flow rate deltas.  The
steady-state bias correction eliminated much of the level dependency, resulting  in  a  more
uniform, positive  exhaust flow rate  bias.  .   Again,  it  should  be  noted  that,  because this
experiment was  conducted using the 5-inch EFM, this steady-state bias correction was small,
because only a small amount of bias was observed during steady-state tests involving the 5-inch
flow meters (see Section 4.4.1 above).
SwRI Report 03.12024.06
Page 170 of 371

-------
     30
     25
     20
   1 15
   HI
   5 10
   o
      5
      -5
     -10
            I5th% A 50th % »95th%
           -»--
            A  •
                                                             A"
                   200
                                400
                                            600
                                                         800
                                                                     1000
                                                                                  1200
                                Lab Reference Exhaust Flow Rate [scfm]
    FIGURE 111. SWIRL TEST SEMTECH-DS EXHAUST FLOW RATE DELTAS
                      CORRECTED FOR STEADY-STATE BIAS
4.10.4 Swirl Error Surf ace Generation

       Using the bias corrected EFM data,  an exhaust flow rate  swirl error surface was
constructed for use in the Monte Carlo Model.  The exhaust flow rate data was normalized using
the EFM maximum flow rate specification. In the case of the 5-inch EFM, the maximum flow
rate was  1700 scfm.  Shown in Figure 112 are the flow rate delta values that were used in the
Model.  Using normalized flow rate data from the reference NTE events, a flow  rate delta was
normally sampled  from the error surface.  Linear interpolation was used for NTE reference
points within the data set. Points outside the  data set  were determined using the  data set
maximum or  minimum values. The swirl error surface generated positive exhaust flow errors
when used in the Model.
SwRI Report 03.12024.06
Page 171 of 371

-------
   E
   o
   c
   .a"
   E ^
    ^
       1.2%
       1.0%
       0.8%
   a  E  0.6%
   Q  I
  3   0.4%
       0.2%
       0.0%
          10%     15%     20%    25%     30%    35%     40%     45%

                                      mol/s (lab,nom) / mol/s_max - [%]
                                                                      50%
                                                                             55%
                                                                                     60%
        FIGURE 112. ERROR SURFACE FOR SWIRL EXHAUST FLOW RATE
4.10.5 Tailpipe Wind Test

       The tailpipe wind test was performed to determine EFM errors when the outlet of the
flow meter was subjected to high velocity air currents.  The Steering Committee was initially
unsure if this experiment would result in significant errors, and how those errors would  be
processed if they were found to  be  significant.   Therefore, the Test Plan called for an initial
experiment to be run in order to determine the possible magnitude of this potential exhaust flow
error source.  According to the Test Plan, if the initial  experiment showed an error of less than 1
percent, no further experimentation would be performed, and the error surface would be dropped
from the Model. The experiment called for a high velocity air stream to be directed at the outlet
of the EFM at a variety of angles while the engine was operating at the 5 test modes.  The flow
was to be designed to simulate a 60 mph wind velocity.

       Figure 113  shows the experimental setup used for the initial test.  A high velocity blower
system was used to  direct air across the outlet of the EFM. A pitot tube device  was used to
measure the air velocity at the outlet  of the blower system.  The air velocity was within the Test
Plan specification of 60 to 65 mph. Three steady-state tests, each consisting of the 5 steady-state
modes for EFM testing, were  run with the blower system.  One test was run with the high
velocity air stream perpendicular to the EFM, one with the air stream  directed 45° into the EFM,
and one test with the air stream  directed 45° out of the EFM. Figure 114 shows the three blower
orientations during the wind testing.  A fourth steady-state test was run without the blower as a
baseline reference.
SwRI Report 03.12024.06
Page 172 of 371

-------
                 FIGURE 113. HIGH VELOCITY BLOWER SYSTEM
                                       90-d
                    45-deg Out
                                                 Blower
                     45-deg In
                          Exhaust
                 FIGURE 114. EFM WIND TEST FLOW SCHEMATIC

       Figure 115 shows the exhaust deltas for the three steady-state tests with the blower and
the one baseline test without the blower. The errors recorded during the blower tests where
similar to the baseline errors. When corrected for the baseline error, the blower deltas collapse to
SwRI Report 03.12024.06
Page 173 of 371

-------
near zero errors. Figure 116 shows the baseline corrected blower deltas.  One outlying exhaust
flow delta was measured during the 45° out testing.
     15
     10
      5
£
t  -5]
     -10
     -15
     -20
                       • PEMS BL * PEMS 90-deg  • PEMS 45-deg Out  A PEMS 45-deg In
                                                    A
                    200*

                      A
                                 400
                                           f^OO   A
                                                           800
                                                                       "lOOO
                                      i
                                 Lab Reference Exhaust Flow Rate [scfm]
                                                                                      30
 FIGURE 115. SWIRL TEST SEMTECH-DS EXHAUST FLOW RATE DELTAS - RAW
                                         DATA
SwRI Report 03.12024.06
                                  Page 174 of 371

-------
     15
     10
      5
     -5]
     -10
     -15
     -20
              • PEMS 90-deg BPEMS 45-deg Out APEMS 45-deg In
                   200
                                             600
 I
-A-
                               •
                               A
                                                          800
                                                              I
                                                              A
                                                                       1000
                                                                                    1200
                                 Lab Reference Exhaust Flow Rate [scfm]
     FIGURE 116. WIND TEST SEMTECH-DS EXHAUST FLOW RATE DELTAS -
                      CORRECTED FOR STEADY-STATE BIAS

       As specified in the Test Plan, the results  from the blower tests  were reviewed by the
Steering Committee to determine if further testing and development of an error surface would be
needed for the tailpipe wind test.  Figure  117 shows the mean baseline corrected exhaust flow
delta with 95 % confidence level  error bars.  This calculation was  performed with and without
the one outlying flow rate delta.  Because the 95 % confidence level  bars nearly crossed zero
error, it was likely the errors generated from further tailpipe wind testing would be negligible. In
addition, the magnitude of all of the errors observed was considerably  smaller than one percent
of the  maximum flow for the flow meter.  The  Steering Committee therefore elected  to not
perform further tailpipe wind testing and  eliminated the wind exhaust flow  rate error surface.
This decision was finalized at the June, 2006 Steering Committee meeting in San Antonio.
SwRI Report 03.12024.06
                Page 175 of 371

-------





n n


c n


^Outlier Removed BOutlie





I




r Included





II
-•



    FIGURE 117. WIND TEST MEAN DELTA VALUES WITH 95 % CONFIDENCE
                                    LEVEL BARS
4.11   Torque and BSFC - OEM Supplied Error Surfaces

     The purpose of the  OEM supplied error surfaces was to capture ECM torque and BSFC
errors that could result from factors not characterized during this program.  The additional error
sources included engine-to-engine production variability and  the operation  of non-deficiency
AECDs.  As part of the Test Plan, participating engine manufacturers were asked to submit data
regarding these potential error sources to EPA. EPA was then tasked with analyzing the data and
developing a single error  surface each for torque and BSFC which would combine the various
error sources.

     Data was submitted by five engine manufacturers prior to the final deadline of August 1,
2006.  EPA conducted an initial  analysis, the results of which were  reported to the Steering
Committee in a memo  from EPA dated August 28, 2006.  As part of the analysis,  EPA held
private discussions with each manufacturer that submitted data, due to the confidential nature of
the information being submitted.  Following the initial analysis, additional information was
requested regarding BSFC errors due to AECD operation, to resolve discrepancies in the data set.
Additional  data was  supplied by two manufacturers regarding this  topic,  after which EPA
completed a final analysis.  The results  of the final analysis were submitted to the Steering
Committee in a second memo dated November 2, 2006, and included a final proposal for the
error surface values.  The Steering Committee approved the final form of the OEM error surfaces
at the November 2, 2006 meeting in San Antonio, as they appeared in the second memo. Copies
of both memos are included in Appendix J.
SwRI Report 03.12024.06
Page 176 of 371

-------
     The final error surface values  are summarized in Table 75 below.  These surfaces are
sampled once per NTE event using a normal distribution. The error levels were determined to
scale with level, therefore the error surface values are defined as percent of point adjustments.
       TABLE 75. OEM ERROR SURFACE DELTAS FOR TORQUE AND BSFC
Parameter
Torque
BSFC
Percentiles
-th
3 i
% point
-6.5%
-5.9%
50th,
% point
0%
0%
95th,
% point
+6.5%
+5.9%
Based on sampling with normal distribution
4.12   Time Alignment Error Surfaces

       When processing the  PEMS  data recorded during transient dynamometer testing,  the
question of time alignment was brought up by the Steering Committee.  When using the PEMS
software to process test data, delay times for several variables can be used to time align the data
recorded from  different  sources.  The variables that can be  time  aligned  include gaseous
emission concentration data, the  exhaust flow meter measurement, and the data recorded using
the vehicle interface.  SwRI used the  procedures detailed in Sensors Inc. Application Note #06-
001 titled Time Alignment of Raw Data to time align the data recorded during transient testing.
Transient  data was  also sent to  Sensors to insure the data was time aligned  correctly.  Time
alignment of the SwRI transient data  was relatively straightforward due to the sharp NTE event
entry  and exit transitions, which were a deliberate  part of the experimental  design.   Several
engine manufacturers indicated aligning data generated during field testing was often difficult
due to the difficulty of finding such clear transitions on many real-world field data sets.  Several
examples of such difficult data sets were shared by Committee members during the course of the
discussions to illustrate  the issue.  With  events as short as 30  seconds, small time alignment
errors result in significant differences in brake-specific emission results.  Therefore, the Steering
Committee elected to account for time alignment errors in the Measurement Allowance Error
Model.

       The first step in this analysis was to decide the level of typical alignment errors.  Based
on input from the Engine Manufacturers, time  alignment  errors up to 1  second are  possible,
however, errors near 0.5 seconds  are more likely.  Shown in Table 76, a matrix of time alignment
errors was generated. The gaseous emission delay times were left unchanged, while the EFM
and vehicle interface delay  times were adjusted.   To account for the relative likelihood  of
occurrence, weighting factors were applied to each matrix point.  Points with delay time errors
equal  to or less than 0.5 were assumed to occur most often and received a relative weighting
factor of 8. Points with one variable at a 1 second delay time error and the other parameter at no
error  received a weighting factor of 2.   The diagonal points  with the EFM and the vehicle
interface both having a delay time error of 1 second  were assumed least likely  to occur and
therefore received a relative weighting factor of 1.
SwRI Report 03.12024.06
Page 177 of 371

-------
   TABLE 76. EFM AND VEHICLE INTERFACE ADJUSTMENT AND WEIGHTING
          FACTORS USED FOR TIME ALIGNMENT ERROR GENERATION
Time
Alignment
Adjustment
Number
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
EFM
Adjustment
[sec]
-0.5
-0.5
-0.5
0.0
0.0
0.0
0.5
0.5
0.5
-1.0
-1.0
-1.0
0.0
0.0
1.0
1.0
1.0
Vehicle
Interface
Adjustment
[sec]
-0.5
0.0
0.5
-0.5
0.0
0.5
-0.5
0.0
0.5
-1.0
0.0
1.0
-1.0
1.0
-1.0
0.0
1.0
Relative
Weighting
Factor
8
8
8
8
8
8
8
8
8
1
2
1
2
2
1
2
1
       One transient NTE test from each test engine was reprocessed using each of the time
alignment adjustment combinations shown in Table 76.  Recall that each transient test was
comprised of 30 different 32-second NTE events.  For each NTE event, brake-specific emissions
were calculated using each of the 3 calculations methods. The differences between the brake-
specific results  calculated with the time alignment adjustments and the brake-specific results
calculated with the nominal time alignment were  calculated as a percent of point for each NTE
event.  With 30 NTE events per cycle and 17 time alignment combinations, 510 time alignment
errors were calculated for each engine.  Once the time alignment error data was pooled for the 3
engines, a statistical routine  was run to apply the specified relative weighting.   The routine
essentially duplicated each error measurement as specified by the weighting factor, generating a
significantly larger pooled  error data set. Finally, the 5th, 50th, and 95th percentile error values
were calculated for the pooled and weighted error data. The brake-specific time alignment error
data is shown in Table 77 for NOX, and Table 78 for CO.  Time alignment data was not generated
for NMHC as nearly all PEMS THC measurements were zero.
SwRI Report 03.12024.06
Page 178 of 371

-------
     TABLE 77. BRAKE-SPECIFIC TIME ALIGNMENT ERROR DATA FOR NOX
Engine
1
2
3
Pooled &
Weighted
Calculation
Method
1
2
3
1
2
3
1
2
3
1
2
3
NOx 5th
Percentile
[% of Point]
-3.3
-2.3
-1.6
-3.7
-0.9
-0.8
-2.0
-1.0
-1.6
-3.2
-1.3
-1.4
NOx 50th
Percentile
[% of Point]
0.0
0.0
0.0
-0.5
0.0
0.0
0.0
0.0
0.0
-0.1
0.0
0.0
NOx 95th
Percentile
[% of Point]
1.6
1.8
4.5
0.4
0.0
1.6
1.9
1.8
3.3
1.5
1.5
2.9
     TABLE 78. BRAKE-SPECIFIC TIME ALIGNMENT ERROR DATA FOR CO
Engine
1
2
3
Pooled &
Weighted
Calculation
Method
1
2
3
1
2
3
1
2
3
1
2
3
CO 5th
Percentile
[% of Point]
-6.6
-5.9
-5.7
-8.4
-6.0
-5.6
-2.8
-3.0
-4.8
-7.5
-5.4
-5.2
CO 50th
Percentile
[% of Point]
-0.1
0.0
0.0
-0.1
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
CO 95th
Percentile
[% of Point]
5.2
5.1
12.3
5.8
6.1
12.7
2.2
3.6
12.5
4.6
5.1
12.3
      Figure 118 graphically depicts the pooled and weighted brake-specific time alignment
errors for NOX and CO.  In the Model, the time alignment errors were applied to the NTE BS
result with all errors applied just prior to the subtraction of the reference NTE brake-specific
result. The time alignment errors were sampled normally and were dependent on the calculation
method.  An example of the time alignment error application is shown below.

            Method 1 BSNOx (with full errors) = 3.8 g/kW-hr
            Method 1 BSNOx (ideal) = 3.5 g/kW-hr
            Method 1 BSNOx Time Alignment Error = 0.78197%
            Delta BS NOx = (3.8 + (3.8*.0078197)) - 3.5 = 0.329714 g/kW-hr
SwRI Report 03.12024.06
Page 179 of 371

-------
      15
      10
   HI
   Q.
     -10
        0123.

                                 Brake-Specific Calculation Method Number



  FIGURE 118. POOLED AND WEIGHTED BRAKE-SPECIFIC TIME ALIGNMENT

                          ERROR DATA FOR NOX AND CO
SwRI Report 03.12024.06
Page 180 of 371

-------
                   5.0    ENVIRONMENTAL CHAMBER TESTING

5.1    Environmental Testing Objective

       Environmental testing  was performed  with the SEMTECH-DS devices to quantify
gaseous emission concentration and exhaust flow measurement errors when the PEMS were
subjected to a variety of environmental disturbances. Environmental conditions evaluated during
the program included ambient temperature, ambient pressure, electromagnetic interference, and
ambient hydrocarbons.   Each environmental  test was designed to simulate  environmental
disturbances that would likely be encountered during in-use field testing.

       SwRI's Mechanical and Material Engineering Division (Division  18)  performed  the
environmental testing on site,  as  specified in  the Test Plan and determined by the Steering
Committee.   Eric  Domes was the  managerial Division  18 contact  prior  to and  during  the
environmental testing.  Rick Pitman performed temperature, pressure,  and vibration testing,
while David Smith and Herbert Walker performed the electromagnetic radiation testing.

5.2    Environmental Testing Procedure

       During the various environmental tests, the performance of the PEMS was evaluated by
sampling and measuring reference gases.  Bottled gases were selected to challenge each PEMS
gas analyzer at zero, audit, and  span levels.  The concentrations of the bottled gases were used as
the reference to  evaluate the PEMS response to the various environmental  disturbances.  The
PEMS  measured responses were compared to the reference concentrations to  determine errors or
deltas  during  the  environmental  testing.   The bottled  reference gases  and  corresponding
concentrations are shown in Table 79.   Reference gas concentrations were chosen based on
recommended audit and  span levels in the Sensors Inc.  user manual.  AL size compressed gas
cylinders were procured  from Scott Specialty Gases.  During the program, the Scott gas bottle
concentration values were used as the reference.  However, each Scott concentration was verified
by SwRI before  being used for testing.   The Test Plan originally  specified the use of methane
audit and span bottles during environmental testing to challenge the PEMS  methane analyzers.
However, the SEMTECH-DS methane analyzers were not accepted by the Steering Committee
as in-use field instruments and were not used  in the Measurement Allowance Program.  The
Steering  Committee therefore elected  to eliminate  the  methane  reference gases from  the
environmental testing procedure.
SwRI Report 03.12024.06                 Page 181 of 371

-------
     TABLE 79. REFERENCE GASES AND TYPICAL CONCENTRATIONS USED
                 DURING ENVIRONMENTAL CHAMBER TESTING
Bottle
Description
Zero Air
N02 Audit
NO2 Span
Quad Audit
Quad Span
THC
[ppm]
0
0
0
159.9
663
CO
[ppm]
0
0
0
178
960
CO2
[%]
0
0
0
6.04
12
NO
[ppm]
0
0
0
257
980
NO2
[ppm]
0
73
243
0
0
Balance
N/A
Air
Air
N2
N2
       The reference gases were overflowed to the inlet of the PEMS sample lines  during
environmental testing.  Using an automated solenoid manifold, the reference gases were sampled
at a specific frequency and in a predetermined order.  The Test Plan recommended sampling
each reference gas for 60 seconds.  The first 30  seconds was intended to purge the system and
allow the analyzer responses to stabilize, with the final 30 seconds used to record a stable mean
measurement.  Preliminary data indicated the NDUV NC>2 analyzers had not stabilized after the
30 second purge, therefore the purge duration was lengthened to 45 seconds for each reference
gas. With a 45-second purge time and 30-second sample length, each reference gas was sampled
for 75 seconds before switching to the next  gas. Even after the 45-second purge, initial data
indicated the NC>2 concentration was still increasing after switching from the quad blend span gas
to the NC>2 span gas.  The sample order of the reference gases was therefore set to minimize the
stabilization problem of the system, with  audit gases preceding the corresponding span gases.
The reference gas sequence below decreased the stabilization times of the span gases and was
used throughout environmental testing.

   1.  Purified zero air reference gas
   2.  NO2 audit reference gas
   3.  NC>2 span reference gas
   4.  Quad blend audit reference gas
   5.  Quad blend span reference gas

       During environmental  testing, zero, audit, and span errors were recorded by comparing
the 30-second mean concentrations of the  recorded PEMS measurements to the reference gas
concentrations.  Although audit and span deltas could only be recorded when the corresponding
audit  and span reference gases were being sampled, zero deltas were recorded whenever the
analyzer's audit or span gas was not flowing.  For example, a zero delta was recorded for NC>2
during the zero air measurement as well as during the quad blend audit and quad blend span gas
measurements.  Recording a zero delta for NO2 during the  quad blend gas measurements was
possible because the quad blend gases contained negligible levels of NC>2.  Likewise, zero deltas
were recorded for NO, CO, CO2, and THC during the NO2 audit and span measurements due to
the absence of the quad blend gases in the NO2 reference gases.  The recording strategy used
during environmental testing  is shown in Table 80.  Again, it  should be noted that the delta
recorded was always the actual analyzer reading minus the reference gas concentration.
SwRI Report 03.12024.06
Page 182 of 371

-------
   TABLE 80. ZERO, AUDIT, AND SPAN DELTA RECORDING STRATEGY USED
                 DURING ENVIRONMENTAL CHAMBER TESTING
Bottle
Sequence
1
2
3
4
5
Bottle
Description
Zero Air
N02 Audit
NO2 Span
Quad Audit
Quad Span
THC
[ppm]
zero
zero
zero
audit
span
CO
[ppm]
zero
zero
zero
audit
span
CO2
[%]
See Note
zero
zero
audit
span
NO
[ppm]
zero
zero
zero
audit
span
NO2
[ppm]
zero
audit
span
zero
zero
Note:
Slow decay of CO2 following the quad span gas caused high zero measurements during the zero air test.
CO2 zero measurements after the quad span gas were not included in the delta data set.
       After reviewing initial environmental  data,  it was evident CO2 zero deltas showed a
systematic trend. CC>2 zero errors recorded during zero air measurements were higher than zero
deltas recorded for NC>2 audit and NC>2 span gas measurements.  Figure 119 shows the repeated
CC>2 zero delta behavior.  The continuous PEMS concentration data indicated the high bias of the
zero air CO2 zero deltas were caused by the  continuing decay of the CO2 measurement after
switching from the quad blend span gas to zero air. Figure 120 shows the response and slow
decay of a PEMS  CC>2  analyzer when zero  air is  sampled after the quad  span  gas.  After
reviewing the initial environmental delta data, the  Steering  Committee elected to remove the
biased CO2 zero delta recorded  during the zero air measurement from the CO2 zero  error
population.
SwRI Report 03.12024.06
Page 183 of 371

-------
     0.020
     0.015 -
   ^ 0.010

   "oi
   Q

   C
   o

   2 0.005 -

   "E
   HI
   u
   c
   o
     0.000

   8     *
     -0.005 -
     -0.010
                          High CO2 delta recorded during zero air reference gas measurement
62       64
                       68       70
                                              74       76
                                                             78
                                     Reference Gas Delta Observations
FIGURE 119. SYSTEMATIC HIGH CO2 BIAS DURING ZERO AIR REFERENCE GAS

                                    MEASUREMENT
          -CO2 [%] 	Quad Blend Start 	Zero Air Start -   NO2 Audit Start 	NO2 Span Start 	Quad Audit
0.30
0.25
E
c
1 0-20
"E
HI
u
o 0.15
O
CM
O
° 0.10
0.05
n nn









Quad Blend Span














Zero Air




NO2 Audit


CO2 decay after quad blend span

Mr-w^


NO2 Span


reference gas
i
Stable CO2 measurement during NO2 au
r!E


Quad Blend



dit and span gases
TI
       1025       1075       1125      1175       1225       1275       1325       1375      1425

                                             Time [s]




FIGURE 120. CO2 CONCENTRATION DECAY DURING ZERO AIR MEASUREMENT

              FOLLOWING THE QUAD BLEND SPAN REFERENCE GAS
SwRI Report 03.12024.06
                    Page 184 of 371

-------
       During environmental testing, the PEMS were operated in a manner representative of in-
use field testing.  Each SEMTECH-DS was started and allowed to thermally equilibrate while
sampling ambient air for 60 to 90 minutes prior to testing.  The PEMS were then zeroed and
spanned as specified in the SEMTECH-DS user's manual.  The PEMS were spanned with the
quad blend reference span gases measured during environmental  testing.  During testing, the
auto-zero feature of the PEMS was used to zero the  devices hourly.  At the completion of the
environmental chamber test, the PEMS were again zeroed and spanned. Span maneuvers were
therefore performed only at the beginning and end of each environmental test.

       Zero air was used to zero the PEMS instruments throughout the  environmental testing
program.  Also, the PEMS were modified to use zero air as the  SEMTECH-DS FID air source
rather than ambient air.  The use of zero air eliminated potential hydrocarbon measurement
errors due to contaminated ambient air.  The removal of ambient air hydrocarbon variability
during engine and  environmental  testing was essential because  the Ambient Hydrocarbon
environmental test was specifically designed to capture FID measurement errors due to varying
levels and different species of ambient hydrocarbons.  Therefore, zero air was used throughout
the program to avoid double counting measurement errors due to ambient hydrocarbons.

       A Sensors Inc. 5-inch exhaust flow meter accompanied the SEMTECH-DS units during
environmental testing to  evaluate the  response of the PEMS EFM to various environmental
perturbations.  One end of the EFM was capped to prevent air flow through the flow meter
during testing.  Therefore, EFM measurements were recorded as 30-second mean zero errors
throughout each environmental test.

       The SEMTECH-DS chassis is designed to house a small, high pressure FID fuel bottle.
Sensors Inc. recommends using the Scotty  104 aluminum gas cylinder from Scott Gas Company.
A full Scotty  104 FID fuel bottle can operate the FID for approximately 7 hours, which was not
sufficient  for  the  8-hour environmental  chamber tests.   Therefore, midway  through each
environmental chamber test, the FID fuel bottle was replaced and the FID was re-zeroed and
spanned. During the 8-hour pressure test, the PEMS was enclosed in a sealed chamber, making
FID fuel bottle replacements impossible.  Therefore,  two FID bottles  were plumbed in parallel
during the pressure test.  Since the environmental chamber testing, Sensors Inc. has  procured
FID fuel bottles with a higher pressure rating, allowing FID operation for over 8 hours.

5.3    Baseline Testing

       Baseline testing was performed with three SEMTECH-DS devices to determine bias and
precision measurement errors for the  PEMS  with environmental  conditions maintained  at a
nominal  level.   It  was  assumed that each subsequent environmental  chamber test would
inherently include the bias and precision errors recorded during baseline testing. Therefore, the
bias  and  variability errors measured during  baseline testing  were  used to correct the
measurement errors generated during each environmental test.

       Originally, PEMS 2, 5,  and 6 were scheduled to be used for environmental testing.
However, during preliminary baseline tests,  the  PEMS  6 FID  would not reach  operating
temperature  and  would therefore  not zero  or span  properly.  Due to the environmental
SwRI Report 03.12024.06                 Page 185 of 371

-------
temperature  chamber schedule, it was necessary to complete the baseline testing as soon as
possible.  In order to expedite baseline  and temperature testing, PEMS 6 was replaced with
PEMS 3 from the dynamometer test lab.  The use of PEMS 6 in the dynamometer laboratory was
deemed acceptable because Engine 1 testing was complete and Engine 2 testing had not started.
Therefore, all of Engine 2 and 3 testing was performed with PEMS 1, 4, and 6.

       Baseline testing was performed in the Thermotron Walk-In temperature control chamber.
Although not used to control the ambient temperature, the chamber provided an environment that
was well ventilated, shielded from EMI and RFI, and maintained at relatively constant pressure
and temperature.  The  Walk-In  chamber was also  large enough to  test 3  PEMS  devices
simultaneously.

       After the SEMTECH-DS devices and EFM had warmed and equilibrated, the PEMS
were zeroed and spanned. Next, the PEMS were set to sample the reference gases which were
controlled by the automated solenoid manifold and overflowed to the inlet of the SEMTECH-DS
sample lines. The PEMS measured the indexing reference gases for approximately 60 minutes,
after which  the  PEMS would perform  an automated zero  maneuver.   Baseline testing was
conducted for 8 hours, generating 72 independent measurements for each  gas. At the completion
of the 8-hour baseline test, the PEMS were zeroed and spanned.

       The initial baseline testing indicated PEMS 2 and 3 had a NC>2 loss problem. Figure 121
shows the PEMS NC>2 delta data during the first 4 hours of baseline testing.  As discussed in the
Environmental Test Procedure section, the NC>2 delta values were calculated by subtracting the
30-second mean PEMS NC>2  measurement from the NC>2 span bottle concentration. The initial
NC>2 delta values for all PEMS were accurate.  However, as the test progressed, PEMS 2 and
PEMS 3 showed a decrease in NC>2 concentration measurements which resulted in large negative
deltas. Curiously, PEMS 2 and PEMS 3 biased NC>2 measurements recovered during the second
and third hours of the 8-hour baseline test.
SwRI Report 03.12024.06                 Page 186 of 371

-------
                             10
                                       15
                                                 20
                                                           25
                                                                      30
                                                                                35

g- 20
£j ~^u "
ra
"oi
Q 40
c -40-
0
TO
HI
U
c
o
O an
CM
O
<" -inn
LU
Q.

-140 -

if 	 ;
A vv»v —.-<
•

A
A t
' *
' A




,..••••••!
>* A«« >*'
> * 9 • «^
,«**'
A
A
A

k





r •"•"•"•"•"!"
******
' A
;-.;-*"-









J»!



J
4
1





r»pi¥?r
k A^ A A A A



L PEMS 2
. PEMS 3
1 PEMS 5
— 1 Hour Marker





i • 1
A A t










1
. 1
i









                                Reference NO2 Span Gas Delta Observations
                                        Bottle Cone. = 258 ppm

    FIGURE 121. PEMS NO2 DELTA DATA DURING INITIAL ENVIRONMENTAL
                                BASELINE TESTING

       The drastic loss of NC>2 during baseline testing prompted an investigation by SwRI and
Sensors Inc. Figure 122 shows the NO and NO2 response of PEMS 2 during the first hour of
baseline testing.  During the first measurement of the NO2 span bottle, PEMS 2 reported a
concentration near the bottle concentration, yielding a relatively small delta measurement. As
expected, PEMS 2 reported near zero concentration levels of NO during the first measurement of
the NO2 span bottle.   As the test progressed, the NO2  concentration measurement of PEMS 2
decreased  significantly.   As the measured NO2 concentration  decreased,  PEMS 2 reported
increased levels of NO during measurement the NO2 span gas.  Because the NO2 span bottle
contained negligible levels of NO, measuring over 40 ppm of NO with PEMS 2 during the NO2
span gas measurement was unexpected. With reduced levels of NO2 and increased levels of NO,
it was apparent a NO2 to NO conversion was taking place. However, the sum of NO2 and NO
during the NO2 span gas measurement was still  less than the NO2 span bottle concentration,
indicating NO2 was not only being converted to NO, but also lost.
SwRI Report 03.12024.06
Page 187 of 371

-------
• PEMS 2 NO ppm
                                                  PEMS 2 NO2 ppm
     250
                  Gas Sequence: Zero Air, NO2 Audit, NO2 Span, Quad Audit, Quad Span (75 sec. each)
     200
   •g- 150 -
   c
   1  100
   "E
   ii*\J- W
1 ^~^ 500 1000
No NO response during NO2 span



r
span



V —





~^ frsi
1500



J^




V —

Red
S~.
c

/

K

uced
<
5

L

N02 res


I
^

M

pons


c

e during
/


I
/

~v

N02


C

span




IP




L




u
*= \^T* -\ 	 w 	 " ^
2000 \ 2500 3000 35
NO response during NO2 span
                                           Time [s]
    FIGURE 122. PEMS 2 NO AND NO2 RESPONSE DURING HOUR 1 OF INITIAL
                      ENVIRONMENTAL BASELINE TESTING

       In trying to repeat the NC>2 loss problem, it was discovered that the PEMS units had to be
turned off or left idle for several hours before the NO2 loss phenomena could be repeated. PEMS
2 had to be left idle overnight to reproduce the results shown in Figure 121. To insure the SwRI
overflow gas delivery system was not causing the NC>2 loss/conversion problem  observed with
PEMS 2 and 3, the Horiba OBS-2200 was fed gas from the SwRI supply manifold and operated
in NO mode during a repeated baseline type test.  Similar to the initial baseline test, PEMS 2 and
3 showed a loss  in  NO2 and  an increase  in NO.  The  Horiba OBS-2200  showed no NO
concentration increase  during the NO2 span gas measurement, indicating the  NO2 loss and
conversion was not caused by the SwRI gas delivery hardware. Next, a test was run with the
sample time for zero  air increased to 300 seconds to observe the effect of a lengthened zero air
purge. The sample times for the audit and span gases were left at 75 seconds.  The NO2 loss with
extended zero air sample time was similar to the initial baseline test results.

       Another test was performed with the quad blend audit and span gases removed from the
gas cycle sequence to determine if the  presence of HC, CO, CO2,  or NO was causing the NO2
loss problem. Although a slight NO2 loss was observed, the magnitude  of the  loss was greatly
reduced with the quad blend gases removed  from the gas sampling sequence.  A question then
surfaced about whether the reduction in NO2 loss was caused by the  removal of HC, CO, CO2,
and NO, or the absence of sampling gases that contained no oxygen. Because the  NO2 audit and
span gases are balanced with air, removing the quad blend audit and span gases (balance N2),
eliminated the sampling of gases with no oxygen. A test was therefore performed  with N2 gas in
place of the quad blend audit and span gases.  The gas sequence for this experiment was zero air,
NO2 audit gas, NO2 span gas, N2 and N2 again.  All gases were sampled for 75 seconds.  With
SwRI Report 03.12024.06
 Page 188 of 371

-------
quad blend gases replaced with N2, the NC>2 loss was similar to the initial baseline test results.
Therefore, the NC>2 loss problem appeared to be dependant on the PEMS sampling oxygen-free
gases.

       The next NC>2 conversion/loss test was performed with the PEMS filters and heated
sample line removed from the system. Again, results were similar to the initial baseline testing,
showing  significant NC>2  loss.    A test  was  then performed  with  the  SEMTECH-DS
thermoelectric chiller bypassed in the  sample handling  system.  With the chiller bypassed in the
system, the NC>2 measure was nearly  perfect and showed no loss or conversion issues.  Figure
123 shows the NC>2  and NO response for PEMS 2 with the thermoelectric chiller bypassed and
then reconnected. With the chiller bypassed, the NC>2 measurement was near the span bottle
concentration of 248 ppm. Once the chiller was reconnected in the sample handling system, the
NO2 loss/conversion problem became immediately apparent.

Concentration [ppm]
-i.-i.hOh
01 o 01 o c
3 O O O O C
-^n
Gas Sequence: Zero Air, NO2 Audit, NO2 Span, Quad Audit, Quad Span (75 se





1

















L.
f















500
r-














Ik.
-sJ
1000
r-





















•vyj T~— r VJ
1500
Chiller re




_N 	




k.
f




-*\





"i
c. each)





2000
connected
r-
f-













reivis i IN
PEMS 2 N
u ppm
O2 ppm



X




2500




rT









k.
3000

'


j.






                                           Time [s]
     FIGURE 123. PEMS 2 NO2 AND NO RESPONSE WITH THERMOELECTRIC
                    CHILLER BYPASSED AND RECONNECTED

       PEMS 2 and 3 received new thermoelectric chillers. After installation of the replacement
chillers, no NO2 loss or conversion was evident.  A possible explanation for the NO2 loss
problem is that the chillers' internal passivated coating may have been compromised.  All PEMS
units, with the exception of PEMS 7, were used for emission testing prior to being sent to SwRI
for use in the Measurement Allowance Program. Not knowing the history of each PEMS, use or
misuse of the PEMS before arrival at SwRI may have caused the chiller NO2 loss/conversion
problem.
SwRI Report 03.12024.06
Page 189 of 371

-------
       Environmental baseline testing was repeated with PEMS 2 and 3 after installation of the
new thermoelectric  chillers.   PEMS 5, showing no NC>2 loss or conversion during the initial
baseline test, did not undergo repeated baseline testing. Rather, PEMS 5 performed temperature
chamber testing  shortly after  the  initial  environmental  baseline  test.   Immediately  after
temperature testing, PEMS 5  was shipped to CE-CERT  to avoid delaying the  on-road model
validation testing.

       The compiled  zero delta data for the 8-hour  environmental  baseline test is shown in
Figure 124 for PEMS 2. During the test, 216 zero delta observations were recorded for each
gaseous emission with the exception of CC>2.  Due to the  elimination of the biased CC>2 zero
deltas measured while sampling zero air, 144 CO2 zero deltas were recorded during the baseline
test. With the replacement chiller, PEMS 2 showed NC>2 and NO zero  deltas within ±5 ppm.
The NC>2 and NO deltas were  added to produce the NOX zero delta measurement. Hydrocarbon
zero measurements were also accurate, with zero deltas less than 2 ppm.  As discussed in the
Environmental Testing Procedure section of the report, the FID fuel bottle was replaced midway
through the baseline test; after which the FID was re-zeroed and spanned.   Considering the 10
ppm resolution of the CO analyzer and past experience measuring positive CO biases through the
PEMS sample line, the 70 ppm range of CO zero deltas was not unexpected.  The CO2 analyzer
provided accurate zero  measurements with zero deltas within ±0.01 %.   The environmental
chamber test results for PEMS 2, 3, and 5 are included in Appendix K.
THC [ppmC]  • CO [ppm]    NO [ppm]  X  NO2 [ppm]  • NOx [ppm]
                                                              -New FID Bottle  A  CO2 [%]
                                                                           0.030
                                 Reference Gas Delta Observation

         FIGURE 124. PEMS 2 ENVIRONMENTAL BASELINE ZERO DELTA
                                  MEASUREMENTS


       The audit delta observations for PEMS 2 are shown in Figure 125.  During the 8-hour
baseline test, 72 audit deltas were recorded for each gaseous emission. As listed in Table 79, the
quad blend reference audit bottle concentrations were near 160 ppmC THC, 178 ppm CO, 6 %
CO2, and 247 ppm NO. The NO2 audit bottle concentration was near 73 ppm.  The PEMS mean
SwRI Report 03.12024.06
                         Page 190 of 371

-------
gaseous measurements were compared to the reference audit bottle concentrations to generate the
audit delta values. NO audit deltas were centered around zero  and generally within ±5 ppm.
NO2 audit deltas showed a negative bias of approximately 8 ppm. A noticeable positive shift in
the NO2 audit  deltas is  evident after observation number 45.   The  shift in the NO2 delta
measurement was due a zero calibration adjustment at the end of one of the 8 hour long segments
of testing.  The zero  adjust can also be seen in the zero delta data at observation number 45 in
Figure 124.  Although the THC zero delta data was accurate, a positive bias of approximately 10
ppmC was evident with the THC audit measurement, indicating a  possible span error.  After
replacing the FID fuel bottle and re-zeroing and spanning the FID, the audit delta measurement
shifted to approximately 8 ppmC.  CO2 showed a slight negative audit delta bias, with deltas
between -0.05 and 0.0 %.  CO showed a slight positive audit delta bias, with deltas between 0
and 50 ppm.   Bottle  naming  errors are included in the audit  delta data  set because the
SEMTECH-DS instruments were not spanned with the audit reference gases. However, the
reference gas concentrations were named by Scott Specialty Gas  Company and checked by
SwRI.  In general, the Scott and SwRI bottle measurements were  within ±1.0 %. Baseline audit
deltas for PEMS 3 and 5 can be found in Appendix K.
THC[ppmC]
CO [ppm]  X  NO [ppm]  X NO2 [ppm]
                                                     NOx [ppm] - New FID Bottle  A  CO2 [%]
                                                                            0.000
                                                                          	0.010
                                                                          	0.020 rt
                                                                      -A	o.oso
                                                                            -0.040 2
                                                                          	0.050
                                                                            -0.060
                                                                         -^  -0.070
                                                                            -0.080
                                 Reference Gas Delta Observation
        FIGURE 125. PEMS 2 ENVIRONMENTAL BASELINE AUDIT DELTA
                                  MEASUREMENTS

       The span delta measurements for PEMS 2 are shown in Figure 126. During the 8-hour
baseline test, 72 span deltas were recorded for each gaseous emission. As listed in Table 79, the
quad blend reference span bottle concentrations were near 663  ppmC THC, 960 ppm CO, 12 %
CO2, and 980 ppm NO.  The NO2 audit bottle concentration was near 243 ppm. The PEMS 30-
second mean gaseous measurements were compared to the reference span bottle concentrations
to generate the span delta values.  With the replacement thermoelectric chiller, PEMS 2 NO2
measurements  showed no significant negative bias and no conversion of NO2 to NO. NO2 span
deltas were generally within ±5 ppm.  During the fist half of the baseline test, NO span deltas
SwRI Report 03.12024.06
              Page 191 of 371

-------
were center around zero error.  During the second half of the test, PEMS 2 NO measurements
drifted slightly negative, with NO deltas reaching -15 ppm.  The THC span deltas were biased 10
ppmC high during the first half of the 8-hour baseline test. After replacing the FID fuel bottle
the FID was re-zeroed and spanned.  After the THC zero and span maneuvers, the THC  span
deltas were near zero.  CO2 span deltas were between -0.02 and 0.04, while CO span deltas were
typically between -20  and 40 ppm.  Because the PEMS were spanned with the reference  span
gases used during baseline testing, bottle naming errors are not included span delta data set.
         » THC [ppmC]
CO [ppm]
NO [ppm]  X  NO2 [ppm]
NOx [ppm]
- New Fl D Bottle  A CO2 [%]
        80
        60	0.060
      I 40
        -40
                A A
               A A
    ..***
      AA A
                                       £t!3k
                                       ****.U**ci»*lfi
                                                             x^xx^o.
                                             • •*•
                                             ^
                                            X    X
        -20 -[-_--     --^-^-"  -^--j--   -^-                             -4-0.020
                     %.
                                                                           0.080
                                                                           0.000
                                               o
                                               o
                                                                           -0.040
                                Reference Gas Delta Observation
         FIGURE 126. PEMS 2 ENVIRONMENTAL BASELINE SPAN DELTA
                                  MEASUREMENTS

       Deltas observed during environmental baseline testing were likely caused by a number of
factors. For example, the PEMS analyzers were zeroed and spanned through the zero and span
ports on the front of the SEMTECH-DS instruments.  When using the zero and span ports, the
reference gases bypassed the majority of the  sample handling  system, including the stainless
steel cooler, the coalescing filter, and the thermoelectric chiller.  Using a pneumatic path to zero
and span the analyzers that was different than the path used during sampling may have caused
environmental baseline errors.

       Zero and span maneuver errors were also captured during the baseline test. The ability of
the PEMS  to zero and span accurately was  captured during environmental baseline testing.
Although 8 zero events occurred for each PEMS during baseline testing, only one span event was
performed for NO, NO2, CO, and CO2.  THC was  spanned twice during baseline testing.  As
discussed in the environmental error surface sections of the report, having only one span event
for each environmental test complicated the extraction of PEMS measurement errors caused by
environmental factors.  Deltas caused by span  errors were often  larger than the delta data for an
environmental test, thus resulting in biases that were not related to the environmental condition
SwRI Report 03.12024.06
              Page 192 of 371

-------
being tested.  The data for each environmental test and for each PEMS was therefore reviewed
before environmental error surfaces were calculated.

      As discussed in the Environmental Test Procedure section of the report, a Sensors Inc. 5-
inch EFM was  used  to capture possible  flow  measurement errors due  to environmental
disturbances.  One end of the flow meter was capped to prevent air flow through  the meter.
Throughout baseline testing, 30-second EFM flow rate averages were taken with each reference
gas observation.  Shown in Figure 127, the observations were calculated  as zero deltas for the
flow measurement system. Most 30-second mean measurements were below 0.6 scfm. Rated at
1700  scfm, the maximum observed flow meter error was  less than 0.1 % of full scale.  The
baseline EFM data was compared to the EFM data from other environmental tests to determine
flow measurement errors due to changes in environmental conditions.
                           100
                                      150        200        250
                                Exhaust Flow Meter Zero Observations
                                                                    300
                                                                               350
      FIGURE 127. 5-INCH EFM ENVIRONMENTAL BASELINE ZERO DELTA
                                 MEASUREMENTS
5.4    Temperature Chamber Testing

       Temperature chamber testing was  performed with three SEMTECH-DS devices  to
quantify PEMS gaseous concentration and exhaust flow measurement errors due to changes in
ambient temperature.  The temperature test was designed to simulate real-world temperatures
and changes in temperature.  Therefore, the  temperature profile used during testing nearly
matched the atmospheric temperature distribution of EPA's 2002 National Emissions Inventory
(NEI) model.  Taken from the Test Plan,  Figure 128 shows the NEI temperature distribution as
well as the test cycle temperature distribution.
SwRI Report 03.12024.06
Page 193 of 371

-------
*inno/

OfiO/
yu/o
QftO/
oU/o
"5T
£ 7n<>/
— /U/o
"S Rn<>/
•s
a ww
>
O A(V>/
/
10%
n<>/

Temperature H

Real World Temperature 23 -100 F

Test Cycle Temperature







28.5 39.5 50.5 61.5
Temperatur
istograms










-m-=
72.5 83.5 94.5
e(F)













     FIGURE 128. TEMPERATURE HISTOGRAMS FOR NEI MODEL AND TEST
                                     PROFILE

      The ambient temperature profile used for chamber testing was defined by a series of
temperature ramps with soaking periods between each transition.  As written in the Test Plan,
Table 81 and Figure 129 define the 8-hour ambient temperature profile used during the program.
SwRI Report 03.12024.06
Page 194 of 371

-------
              TABLE 81. TEMPERATURE TEST PROFILE DEFINITION
Ambient Temperature Test Sequence
Phase
1 Soak
2 Ramp
3 Soak
4 Ramp
5 Soak
6 Ramp
7 Soak
8 Ramp
9 Soak
10 Ramp
11 Soak
12 Ramp
13 Soak
Temperature
°C
13.89
13.89-5.00
-5.00
-5.00-12.78
12.78
12.78-28.33
28.33
28.33-37.78
37.78
37.78-22.22
22.22
22.22-13.89
13.89
°F
57
57-23
23
23-55
55
55-83
83
83-100
100
100-72
72
72-57
57
Time
min
10
5
5
145
40
5
52
5
8
100
60
5
40
Rate
°C/min
0.00
-3.78
0.00
0.12
0.00
3.11
0.00
1.89
0.00
-0.16
0.00
-1.67
0.00
Comments
Cool in-garage pre-test PEMS operations
Leaving cool garage into cold ambient
Operating at cold temperature outside of vehicle
Diurnal warming during cool day
Steady cool temperature during testing
Return to hot garage on a cool day
Hot in-garage pre- post- test PEMS operations
Leaving ho garage into hot ambient
Operating at hot temperature outside of vehicle
Diurnal cooling during hot day
Steady moderate temperature during testing
Return to cool garage on a moderate day
Cool in-garage post-test PEMS operations
                          Temperature-Time Environmental Test Cycle
        -30

        -40

        -50

        -60
  — Temperature
	1/2-hr moving average dT/dt (C/hr)
	Vertical gridlines = hours
   Vertical gridlines = 7-min gas cylinder cycle times
116
104
92
80
68
56
44
32
20
8
-4
-16
-28
-40
-52
-64
-76
                                           4
                                        Time (hr)
      FIGURE 129. TEMPERATURE TEST PROFILE AND MOVING AVERAGE

       Temperature testing was originally scheduled to be performed with a Thermotron Walk-
In temperature enclosure.  The  Walk-In chamber could easily house three PEMS devices,
therefore, temperature testing could be completed in one day.  However, with the PEMS and
auxiliary hardware,  it was unlikely the large Thermotron Walk-In would achieve the steepest
cooling ramps as defined in the Test Plan.  Therefore, each PEMS was tested individually with a
smaller Thermotron  SM-32 temperature control chamber, shown in Figure 130.  The Thermotron
SwRI Report 03.12024.06
      Page 195 of 371

-------
SM-32 used liquid nitrogen for supplemental cooling, and achieved all target temperatures and
ramp  rates.    The  Thermotron  SM-32  chamber housed the  PEMS unit,  EFM,  and
temperature/relative humidity probe. The SEMTECH-DS heated sample lines, zero and span gas
lines,  drain lines, and Ethernet cables were routed out  of the chamber through ports  on the
chamber sides.
 FIGURE 130. THERMOTRON SM-32 TEMPERATURE CONTROL CHAMBER WITH
                 SUPPLEMENTAL LIQUID NITROGEN CYLINDER

       Prior to executing the  environmental  temperature test, the PEMS  were  allowed  to
thermally equilibrate while sampling ambient air for over one hour. The PEMS were then zeroed
and spanned at ambient temperature, approximately 23 °C.  The environmental temperature test
was then started by ramping to the initial temperature soak point as specified in the Test Plan.
During the 8-hour temperature test, the PEMS were  automatically zeroed  every hour.  The
temperature control chamber was not paused during the test, therefore, zero events occurred at
the temperatures defined by the Test Plan's temperature profile definition.  Zero events occurred
near the hour markers shown in Figure 129.   Similar  to environmental  baseline testing, zero,
audit, and span deltas were recorded by comparing the 30-second PEMS mean concentration
measurements to the reference gas concentrations.  PEMS 3 performed temperature testing in a
Sensors Inc. environmental enclosure.

       The zero deltas measured during the 8-hour temperature test are shown in Figure 131 for
PEMS 2. Mean temperature measurements from the PEMS temperature/relative humidity probe
are also shown in Figure 131.  Analyzer zero drift caused by temperature variation was evident
throughout the  temperature test.  For example,  during  the  steep  temperature ramp at  the
SwRI Report 03.12024.06
Page 196 of 371

-------
beginning of the cycle, NO, NO2, and CO drifted downward. The zero maneuver at the end of
the first hour of operation corrected the negative zero drift.  Positive  zero  drift was evident
during the 4th and 5th hours of testing when the temperature was increasing aggressively. Again,
the hourly zero maneuvers continually  corrected the zero drift.  Slight negative drift occurred
during the last 3 hours of the 8-hour test, when the chamber temperature was decreasing.  CO2
and THC  measurements  were largely unaffected by the temperature fluctuations experienced
during the environmental temperature  test.   Temperature  data is included  for all  PEMS  in
Appendix K. NO, NO2, CO, and CO2 behaved similarly with the three PEMS units during the
temperature tests.  THC measurements with PEMS 3 and 5  showed slightly more susceptibility
to temperature induced zero drift than PEMS 2.
               THC [ppmC]   •  CO [ppm]        NO [ppm]
               New FID Bottle     Temp. [degC]   A  CO2 [%]
               X NO2 [ppm]
NOx [ppm]
                                                                            0.014
                                                                            -0.004
                                 Reference Gas Delta Observation
      FIGURE 131. PEMS 2 ENVIRONMENTAL TEMPERATURE ZERO DELTA
                                  MEASUREMENTS

       Figure 132 shows the audit deltas measured during environmental temperature testing for
PEMS 2.  For reference, the PEMS temperature probe mean measurement is plotted with the
audit delta values.  During the first hour of the temperature cycle, the NO2 audit measurement
drifted negative, similar to the zero measurement.  The zero maneuver at the end of the first hour
of testing not only corrected the zero drift, but also corrected the negative audit drift.  The NO2
audit delta remained between -5 and -10 ppm for the last 7 hours of the 8-hour test.  NO audit
deltas were  between -10  and  10 ppm throughout the test.  The THC audit deltas appeared
unaffected by temperature  variation; however, the THC audit delta was near 20 ppm for the first
half of the test.  After replacing the FID fuel bottle and zeroing  and spanning the FID, the audit
deltas were below 10 ppm.  CO and CO2 temperature audit deltas were similar to the audit deltas
observed during baseline testing.  As seen in Appendix K, the audit delta behavior was similar
between the PEMS, with PEMS 3 and 5 showing slightly more THC temperature drift.
SwRI Report 03.12024.06
Page 197 of 371

-------
             •  THC [ppmC]   •  CO [ppm]     X NO [ppm]
            	New FID Bottle     Temp. [degC]   A CO2 [%]
               X NO2 [ppm]
NOx [ppm]
                                 Reference Gas Delta Observation

     FIGURE 132. PEMS 2 ENVIRONMENTAL TEMPERATURE AUDIT DELTA
                                  MEASUREMENTS

       Figure  133  shows the span deltas for PEMS  2 during the temperature test.  NO2  span
deltas were minimal, generally within ±5 ppm. NO, CO, and CO2 span deltas were similar or
slightly more variable than the span deltas observed during baseline testing. PEMS 2 THC span
deltas showed large perturbations that followed a trend similar to the temperature profile.  PEMS
3 and 5 also showed THC span deltas that were larger than the baseline test span deltas.  An
explanation offered by Sensor Inc. in regard to the large PEMS 2 THC span delta measurements
was that the FID drain pressure may have been slightly elevated.  The FID is sensitive to drain
backpressure, which  may have been slightly elevated due to the  extended length of the drain
lines during temperature testing. The Steering Committee elected to accept the temperature data
although the THC  span delta data may have been influenced by the test setup.  Due to the low
THC levels of the reference NTE events used in the Model, the THC span deltas would never be
used in the Model calculations.  Because the NO, NO2, CO, and CO2 temperature delta data was
sound, and the THC span delta would not influence the Model, it was decided not to repeat the
temperature testing.
SwRI Report 03.12024.06
Page 198 of 371

-------
            •  THC [ppmC]   •  CO [ppm]     X NO [ppm]
            	New FID Bottle     Temp. [degC]   A CO2 [%]
               X  NO2 [ppm]
NOx [ppm]
                                                                           0.100
                                                                           -0.050
                                Reference Gas Delta Observation
      FIGURE 133. PEMS 2 ENVIRONMENTAL TEMPERATURE SPAN DELTA
                                  MEASUREMENTS

       A Sensors Inc. 5-inch EFM also underwent environmental temperature testing.  The flow
meter with pressure transducer enclosure was placed in the temperature chamber during the 8-
hour test.   Similar to baseline testing, one end of the EFM was capped to prevent air flow
through the meter. Figure 134 shows the 30-second mean flow meter measurements during the
temperature test.  The zero deltas  observed during temperature testing largely resembled the
EFM deltas recorded during baseline testing.  Two periods midway through the temperature test
showed slightly increase EFM measurements.  One perturbation occurred after the 150th mean
delta measurement, while the other occurred between  observation number 200 and 250. The
deviations from zero  were small, with the maximum zero error under 0.3 % of the EFM's rated
flow range of 1700 scfm.
SwRI Report 03.12024.06
Page 199 of 371

-------
        5.0

        4.5

        4.0

        3.5

        3.0

        2.5

        2.0

        1.5

        1.0

        0.5

        0.0
*   •
                       * *
                    50
                             100        150        200       250
                                  Exhaust Flow Meter Zero Observation
                                                                    300
                                                                              350
   FIGURE 134. 5-INCH EFM ENVIRONMENTAL TEMPERATURE ZERO DELTA
                                   MEASUREMENTS
5.4.1   Temperature Error Surface Generation

       Because  the temperature test was  designed to simulate  real-world temperatures and
changes in temperature, the deltas measured during temperature testing were randomly sampled
in the  Model.  However, it was  assumed  that temperature chamber testing would inherently
include the bias and precision errors recorded during baseline testing.  Therefore, the bias and
variability errors measured during baseline  testing were used to correct the measurement errors
generated during each environmental test.

       The  initial  step  in  generating  the temperature  error surfaces  was to  correct each
temperature measurement error for any bias measured during baseline testing.  This discussion
focuses on NOX concentration, however,  the same process was applied to each gaseous emission.
The median baseline NOX delta was calculated for each PEMS at the zero,  audit, and span levels.
The  median baseline deltas were subtracted from  each  delta measured  during  temperature
testing. For example,  the PEMS 2 median baseline zero delta was subtracted from each PEMS 2
delta recorded during  temperature testing.  A similar procedure was performed for the audit and
span deltas.   The median environmental baseline NOX concentrations for PEMS 2, 3, and 5  are
shown in Table  83.    To remove the baseline  variability from the temperature  test data,
multiplicative scaling factors  were calculated.  The median absolute deviation (MAD) was
calculated for each baseline delta data set as well as the bias corrected temperature delta data set.
Scaling factors were calculated using the equation below.  The scaling factors, shown in Table
82, were multiplied to each bias corrected temperature delta to reduce the variability of the data.
Similar to the bias correction, the variability correction was performed for  each PEMS and at the
zero, audit, and span levels.
SwRI Report 03.12024.06
                          Page 200 of 371

-------
                        Scaling _ Factor =
                                                    -M4/

                                               MAD
                                                    Rod
 TABLE 82. MEDIAN ENVIRONMENTAL BASELINE NOX CONCENTRATIONS AND
             ENVIRONMENTAL TEMPERATURE SCALING FACTORS

Zero
Audit
Span

PEMS2
PEMS3
PEMS5
PEMS2
PEMS3
PEMS5
PEMS2
PEMS3
PEMS5
Median
Baseline NOx
Delta
[ppm]
0.9
2.3
3.2
-6.1
-6.3
-10.5
-2.9
-4.7
-7.5
NOx
Temperature
Scaling Factor
0.85
0.66
0.77
0.90
0.93
0.88
0.94
0.91
0.97
       Figure 135 through Figure 137 show the corrected zero, audit, and span NOX temperature
deltas for each PEMS.  The corrected temperature deltas for all gaseous emission can be found in
Appendix K.  The trends of the NOX deltas measured during temperature testing could be linked
to changes in temperature by comparing the deltas with the chamber temperature profile.  Also,
the  3  PEMS  showed similar NOX delta  patterns, indicating  a susceptibility to ambient
temperature. Therefore, inclusion of the NOX temperature error surface was justified.  CO, CO2,
and NMHC delta trends were not as easily linked to changes in temperature. Also, the CO, CO2,
and NMHC delta patterns for the 3 PEMS were not as tightly matched as for NOX. Therefore, it
was not clear whether the CO, CO2, and NMHC deltas were caused by the ambient temperature
test or by some other factors. This problem was presented to the Steering Committee.  After
review of the data and recommendations by SwRI, a decision was reached to include temperature
error surfaces for all of the gaseous  emissions.  Justification  for the inclusion of all pollutants
included the following.

    1.  The variance  of the temperature  data was  generally  larger  than the  baseline data,
       indicating an ambient temperature susceptibly.
   2.  The deltas from  one  or  more  PEMS showed  a subtle correlation to the chamber
       temperature profile.
   3.  There was slight agreement of the  delta patterns between PEMS, indicating a common
       error source.
SwRI Report 03.12024.06
Page 201 of 371

-------
                                                     - PEMS 2 * PEMS 3  PEMS 5
       25	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1
                                Reference Gas Delta Observation
  FIGURE 135. ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE NOX

               CONCENTRATION ZERO DELTA MEASUREMENTS
       15
       10 --
        5 -

                                                      - PEMS 2 • PEMS 3 • PEMS 5
        0 -
       -5 ---
Q

c
o
B
re


m
o
c
o
o


o -10 -
z
       -15 --
              .   -_*.--_
       -20 H—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—h


         0        10       20        30        40       50        60        70


                                Reference Gas Delta Observation
  FIGURE 136. ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE NOX

              CONCENTRATION AUDIT DELTA MEASUREMENTS
SwRI Report 03.12024.06
                           Page 202 of 371

-------
        40
        30  -
                                                           - PEMS 2 • PEMS 3 • PEMS 5
      I 20
      Q.
      C
      O
      B
      re

      m
      2
      o
      O
      x
      O
        10 --
 0 -- -<
-10 --

        -20  -
        -30
                   H—I-
                                          -+-
                                                H—I-
                   10
                             20        30        40
                                   Reference Gas Delta Observation
                                                          50
                                                                   60
                                                                             70
   FIGURE 137. ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE NOX
                CONCENTRATION SPAN DELTA MEASUREMENTS

       The baseline corrected deltas were sampled randomly in the Model.  The Model was
initially programmed to randomly sample 360 zero, audit, and span observations for each PEMS.
With temperature data for 3  PEMS, the Model was programmed to use 1080 zero, audit, and
span deltas.  PEMS 2  data  was used for observation 1 to 360, PEMS 3  data was  used for
observation 361 to 720, and PEMS 5  data was used for observation 721 to 1080.  However,
during temperature testing, only 216  zero observations, 72 audit observations,  and  72 span
observations were recorded  for  each  PEMS.  The data for  each PEMS  was  expanded by
repeating delta observations to generate 360 zero,  audit, and span observations for each PEMS.
The final NOX error surface for environmental temperature testing is shown in Figure 138.  Final
temperature error surfaces for CO, CO2, and NMHC can be found in Appendix K.
SwRI Report 03.12024.06
                            Page 203 of 371

-------
      -30
                     200
                                   400           600           800
                                NOx Temperature Observation (1 to 1080)
                                                                           1000
               - NOx SPAN 1209ppm NOx
                                     • NOx Audit 341 ppm NOx
                                                          — NOx Ze ro 0 ppm NOx
 FIGURE 138. FINAL ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE
                                NOX CONCENTRATION

       For each cycle of the Model, a number from 1 to 1080 was randomly selected.  At the
selected  observation, zero,  audit,  and  span delta  values  were  sampled.   Based on  the
concentrations  of the reference NTE events, delta values were linearly interpolated  from the
zero, audit, and span delta data.

       On occasion, the variance measured during environmental baseline testing was greater
than the variance measured during temperature testing. During these instances,  a scaling factor
could not  mathematically be calculated.   Therefore, each delta observation  was set to the
difference between the median delta value measured during temperature testing  and the median
delta value measured during the environmental baseline testing.  An example of this  correction is
shown in Figure 139 for CO span deltas.
SwRI Report 03.12024.06
Page 204 of 371

-------
     0.0070 -i
     0.0060 - * -
     0.0050 --
     0.0040 --
   | 0.0030
   o
   o
     0.0010 +
     0.0000 --
     -0.0010 --
     -0.0020
                                                               - PEMS 2 * PEMS 3 • PEMS 5
Vr
                H	1	1	1	H
                                    H	1	h
-H	I—
  50
                                                                       —I	H
                                                                       60
                                                      H	1—
                                                        70
                    10
                              20         30         40
                                     Reference Gas Delta Observation
 FIGURE 139. CORRECTED CO DELTAS MEASURED DURING ENVIRONMENTAL
                              TEMPERATURE TESTING

       It was later decided that if the variance of the baseline test exceeded the variance of the
temperature  test,  it was  unlikely the changes in ambient temperature adversely affected the
performance of the PEMS.  Therefore, the bias  differences captured by the subtraction of the
temperature  and baseline median deltas are not likely due to changes in ambient temperature.
Following this argument, all delta observations in the final errors surfaces were set to zero if the
baseline MAD exceeded the temperature test MAD.   An example of this correction is shown in
Figure 140 for CO.
SwRI Report 03.12024.06
           Page 205 of 371

-------
        0.007
        0.006 -
        0.005 -
        0.004 -
     g  0.003 -
     3  0.002 -
      &
      I
0.001

   0
       -0.001 -
       -0.002 -
       -0.003

                       200
                                                60.0
                                                                         1000
                                 CO Temperature Observation [1 to 1080]
                 - CO Span 0.0951% CO
                                     • CO Audit 0.0179% CO
                                                         - CO Zero 0% CO
 FIGURE 140. FINAL ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE
                                CO CONCENTRATION

       A temperature error surface was also generated for exhaust flow rate.   Similar to the
gaseous concentration error surfaces, the temperature exhaust flow rate data was corrected for
the bias error and variance recorded during baseline testing.  The baseline correction process was
slightly modified for the exhaust flow rate because the PEMS automatically set all negative flow
rate measurements to zero.  With all negative measurements set to zero,  the distribution of the
flow rate data was inaccurate.

       To  generate  the exhaust flow  rate  error surface for temperature, all  zero flow rate
measurements were  removed from both the temperature and baseline tests; thus generating a
more accurate variance comparison between the two data sets.  With zero deltas removed, the
median exhaust  flow  rate  measurement from  the  baseline test was subtracted  from each
temperature exhaust flow rate  delta.   This process  was inconsequential because the median
baseline delta was less than 0.01 % of the flow meter's maximum flow rating. Next the MAD of
the baseline data was compared to the MAD of the bias corrected temperature data.  Using the
equation below, the MAD values were used to calculate a scaling factor.  Each temperature
exhaust flow measurement was multiplied by the scaling factor to shrink the variance of the
temperature data by the variance measured during environmental baseline testing.
                         Scaling _ Factor =
                                                      -MAD
                                                            BL
                                                 MAD
                                                      Rod
       All negative corrected temperature deltas were set to zero.  The data was then mirrored
about the zero axis, generating twice the number of zero observations as well as negative deltas.
Negative deltas were generated to restore the negative data lost during the zero clipping process
and because exhaust measurements performed during engine operation in the NTE zone would
SwRI Report 03.12024.06
                             Page 206 of 371

-------
be subjected to both positive and negative exhaust flow rate errors.  The final exhaust flow rate
error surface for temperature testing is  shown in Figure  141.  Temperature exhaust flow rate
deltas were sampled randomly in the Model and applied to each reference NTE event exhaust
flow rate independent of level.
        0.25
        0.20  -
                        100
                               150     200     250     300
                                 Exhaust Flow Meter Zero Observation
                                                           350
                                                                  400
                                                                         450
     FIGURE 141. ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE
                 EXHAUST FLOW RATE DELTA MEASUREMENTS
5.5    Pressure Chamber Testing

       Pressure chamber testing was performed with two SEMTECH-DS  devices to quantify
PEMS gaseous  concentration and exhaust flow measurement errors due to  changes in ambient
pressure.  The  pressure test was  designed to simulate real-world pressures and changes in
pressure.  Therefore, the pressure  profile used during testing nearly matched the  atmospheric
pressure distribution of EPA's 2002 National Emissions Inventory (NEI) model. Taken from the
Test Plan, Figure 142 shows the NEI pressure distribution as well as the test cycle pressure
distribution.
SwRI Report 03.12024.06
Page 207 of 371

-------
Pressure Histograms
QfiO/
yu/o
ono/
OU/o
•= 70%
]S
+J
•s
0s
o 40%

-------
              TABLE 83. PRESSURE TEST PROFILE DEFINITION
Atmospheric Pressure Test Sequence
Phase
1 Soak
2 Ramp
3 Soak
4 Ramp
5 Soak
6 Ramp
7 Soak
8 Ramp
9 Soak
10 Ramp
11 Soak
12 Ramp
13 Soak
14 Ramp
15 Soak
16 Ramp
17 Soak
18 Ramp
19 Soak
20 Ramp
21 Soak
22 Ramp
23 Soak
Pressure
kPa
101
101-97
97
97-101.87
101.87
101.87-101
101
101-97
97
97-96.6
96.6
96.6-82.74
82.74
82.74-96.8
96.8
96.8-90
90
90-96.8
96.8
96.8-99.2
99.2
99.2-101
101
Alt. ft.
89
89-1203
1203
1203- -148
-148
-148-89
89
89-1203
1203
1203-1316
1316
1316-5501
5501
5501-1259
1259
1259-3244
3244
3244-1259
1259
1259-586
586
586-89
89
Time
min
10
20
20
60
20
20
20
20
25
20
20
20
20
30
20
15
10
20
20
20
20
10
20
Rate
ft/min
0
56
0
-23
0
12
0
56
0
6
0
209
0
-141
0
132
0
-99
0
-34
0
-50
0
Comments
Flat near sea-level
Moderate hill climb from sea level
Flat at moderate elevation
Moderate descent to below sea level
Flat at extreme low elevation
Moderate hill climb to near sea level
Flat near sea level
Moderate hill climb from sea level
Flat at moderate elevation
Slow climb from moderate elevation
Flat at moderate elevation
Rapid climb to NTE limit
Flat at NTE limit
Rapid descent from NTE limit
Flat at moderate elevation
Rapid hill climb to mid elevation
Flat at mid elevation
Rapid descent within middle of NTE
Flat at moderate elevation
Moderate descent to lower elevation
Flat at lower elevation
Moderate decent to near sea- level
Flat near sea-level
SwRI Report 03.12024.06
Page 209 of 371

-------
                            Pressure-Time Environmental Test Cycle
                	1/2-hr moving avg dA/dt (ft/hr)
                    Vertical gridlines = hours
                    Vertical gridlines = 7-min gas
                    cylinder cycle times
         -9000
         FIGURE 143. PRESSURE TEST PROFILE AND MOVING AVERAGE

       The environmental pressure test was conducted in the altitude chamber shown in Figure
144. The chamber consisted of a cylindrical top that rested on a flat, circular base.  The chamber
was specifically  designed  to  simulate  elevated  altitudes and  can  attain pressure levels
representative of 65,000 feet of elevation.  As the chamber pressure is lowered below  ambient
pressure, the chamber top is pulled downward, creating  a tight seal between the chamber base
and top.  However, the Test Plan specified pressure levels up to 101.87 kPa or 148 feet below sea
level.  Attaining positive pressure  in the altitude chamber was problematic  because the o-ring
sealing mechanism between the  chamber top and base  would shift and leak.  Using  weather
stripping, clay,  and duct tape,  a revised sealing mechanism was implemented that allowed the
chamber to achieve all pressures and pressure ramp rates as specified in the Test Plan.
SwRI Report 03.12024.06
Page 210 of 371

-------
                                   ALTITUDE CHAMBER
                                   ihide and Elevated Temperature
                                   'rnglales Up to 65,000 Feet of
                                   lilude
                                    nal Dimensions: 5' Dia. x 7' Tall
         FIGURE 144. ALTITUDE CHAMBER TOP - REMOVED FROM BASE

       As  shown in Figure  145, PEMS  2 and 3 were tested simultaneously in the altitude
chamber.  PEMS 3  was tested in a Sensors Inc. environmental enclosure.  A 5-inch EFM was
also tested  in the chamber.  In order to accurately simulate elevation changes, the PEMS sample
line overflow system, overflow FID zero air system, and PEMS drain lines were vented inside
the altitude chamber. Figure 145 shows a preliminary and incorrect setup with venting occurring
outside the chamber.  Low restriction, electronic flow meters were installed in the sample line
overflow stream and  FID  zero  air  overflow  stream  to  insure  adequate bypass  flow was
maintained during the 8-hour pressure test.  Gas lines and Ethernet cables were  routed out of the
chamber through a hole in the base plate.  Clay and expanding foam insulation spray was used to
seal the  lines and cables exiting the chamber base.  With no access to the  PEMS  once the
chamber top was sealed to the base, replacing the FID fuel bottles during the 8-hour test was not
possible. Therefore, two Scotty 104 FID fuel bottles were plumbed together in parallel for each
PEMS unit. With two  FID  fuel bottles, the PEMS operated without FID fuel bottle replacement
during the altitude simulation test.
SwRI Report 03.12024.06
Page 211 of 371

-------
         FIGURE 145. PEMS EQUIPMENT ON ALTITUDE CHAMBER BASE

       Prior to executing the environmental pressure test, the PEMS were allowed to thermally
equilibrate while sampling ambient air for over one  hour.  The altitude chamber top was then
sealed  to the chamber base.  Figure 146 shows the  assembled altitude chamber with pressure
control  equipment.    Next, the  PEMS   were zeroed  and  spanned at  ambient pressure,
approximately 98 kPa. The environmental pressure test was then started by ramping to the initial
pressure soak point as specified in the Test Plan.  During the 8-hour pressure test, the PEMS
were automatically zeroed every hour.  The pressure control chamber was not paused during the
test, therefore, zero events occurred at the pressures defined by the Test Plan's pressure profile
definition.  Zero  events  occurred near the hour markers  shown in Figure 143.  Similar to
environmental baseline and temperature testing, zero, audit, and span deltas were recorded by
comparing  the  30-second PEMS  mean  concentration  measurements to  the reference  gas
concentrations.
SwRI Report 03.12024.06
Page 212 of 371

-------
                                                  -    Mi*
   FIGURE 146. ALTITUDE CHAMBER AND PRESSURE CONTROL EQUIPMENT
                                 DURING TESTING

       The zero  deltas measured during the 8-hour pressure test are shown in Figure 147 for
PEMS 2.  Mean pressure measurements from the PEMS ambient pressure transducer are also
shown in Figure  147. Environmental pressure results for PEMS 3 are included in Appendix K.
In general, the PEMS zero errors showed little variation during environmental pressure test. NO2
measurements  were  relatively  stable throughout  the pressure test.   PEMS  2 NO zero
measurements drifted upward during the first hour of pressure testing.  However, the pressure
change was not significant during the first hour of testing and PEMS 3 showed no NO drift
during the initial hour of the test. Therefore, the PEMS 2 NO drift during the first hour of testing
was not likely caused by changes in pressure, but perhaps thermal equilibration of the NDUV.
PEMS 2 and 3 both showed slight negative NO drift during the steep negative  pressure ramp to
the  soak pressure of 82.74 kPa.  The NO measurements drifted back in a positive direction
during the positive pressure ramp from the 82.74 kPa soak pressure.  THC zero measurements
were relatively stable during the pressure test.  Similar to NO, THC zero deltas showed slight
negative drift during negative pressure ramps, and slight positive drift during positive pressure
ramps. CO and CO2 zero deltas did not show significant deviations due to changes in pressure.
SwRI Report 03.12024.06
Page 213 of 371

-------
                • THC [ppmC] BCO [ppm]  NO [ppm] XNO2 [ppm] »NOx [ppm]  Pressure [kpa] ACO2 [%]
         120
      „ 100 —,
                                                                             0.014
                                                                             -0.018
                                 Reference Gas Delta Observation
         FIGURE 147. PEMS 2 ENVIRONMENTAL PRESSURE ZERO DELTA
                                  MEASUREMENTS

       PEMS 2 audit level deltas measured during the environmental pressure test are shown in
Figure 148.  NC>2,  THC, and CC>2 deltas  were relatively  stable and showed no trends with
pressure. Similar to the zero deltas, PEMS 2 showed positive NO audit level drift during the first
hour of testing.  Because PEMS 3 showed no NO drift during the first hour of pressure testing
and the pressure was relatively constant during this period, the  PEMS 2 NO drift  was likely
caused by  a factor  other than pressure change.   CO audit measurements showed positive
response when the  pressure in the altitude chamber was reduced. PEMS  2 CO audit deltas
reached 70  ppm during the pressure soak at 82.74 kPa.
SwRI Report 03.12024.06
Page 214 of 371

-------
                • THC [ppmC] BCO [ppm]  NO [ppm] XNO2 [ppm] «NOx [ppm] -Pressure [kpa] ACO2 [%]
                                                                        - 0.054
                                                                          -0.030
                                Reference Gas Delta Observation
        FIGURE 148. PEMS 2 ENVIRONMENTAL PRESSURE AUDIT DELTA
                                 MEASUREMENTS

       The environmental pressure test span deltas are shown in Figure 149 for PEMS 2.  NC>2
measurements were steady and unaffected by changes in pressure. NO span measurements were
more variable; however, it was difficult to determine a link  between the NO deltas and the
chamber pressure.   THC  span measurements showed slight  positive  response  with  lower
chamber pressures.  CO and CO2 span measurements were both affected by chamber pressure.
CO span deltas reached 140 ppm during the pressure soak at 82.74 kPa and 50 ppm during the 90
kPa soak.  CO2 span measurements had a negative response to lowered chamber pressure, with
CO2 span deltas reaching -0.13 % during the 82.74 kPa soak. PEMS 3  pressure test span deltas,
which were similar to PEMS 2, can be found in Appendix K.
SwRI Report 03.12024.06
Page 215 of 371

-------
                • THC [ppmC] BCO [ppm]  NO [ppm] XNO2 [ppm] «NOx [ppm] -Pressure [kpa] ACO2 [%]
        160
         -60
                                 Reference Gas Delta Observation
         FIGURE 149. PEMS 2 ENVIRONMENTAL PRESSURE SPAN DELTA
                                  MEASUREMENTS

       The response of  a  5-inch  Sensors  Inc. flow  meter was  also  measured during the
environmental pressure test. With one end capped to prevent air flow through the meter, the 30-
second mean measurements were recorded at zero deltas.  Figure  150 shows the EFM deltas
observed  during  pressure testing.   Unlike  baseline  and  temperature  testing,  most  EFM
measurements were above zero during pressure testing.  Although the EFM measurement errors
were small compared to the 1700 scfm flow rating of the meter, the PEMS EFM showed positive
interference  during the environmental  pressure test.  When the  chamber pressure was below
approximately 990 mbar,  the EFM zero deltas were between 5 and 8 scfm. When the chamber
pressure was above 990 mbar, the EFM deltas were between 0 and 4 scfm.
SwRI Report 03.12024.06
Page 216 of 371

-------
                               * Flow Meter [scfm] • Pressure [mbar]
                                                                                1200
                          100
                                    150        200       250
                                Reference Gas Delta Observation
                                                                 300
                                                                           350
      FIGURE 150. 5-INCH EFM ENVIRONMENTAL PRESSURE ZERO DELTA
                                  MEASUREMENTS
5.5.1   Pressure Error Surface Generation

       The process used to generate the pressure error surfaces was similar to the environmental
temperature error surface calculation method.  The pressure delta  data for each PEMS was
corrected for baseline bias and variance.  As with the temperature error surfaces, it was difficult
to determine  which  gaseous emissions  showed  a susceptibility  to the ambient pressure
disturbances.  Criteria similar to the temperature  error surfaces were used to decide which
pressure error surfaces should be included in the Model.  As shown in Figure 151, the trends of
the NMHC span deltas followed the ambient pressure traces recorded in the pressure chamber.
Also, the two PEMS showed similar NMHC span delta behavior, indicating a common source of
error.  CO  also  showed definite  ties between the  delta behavior and  the  pressure  traces.
Therefore, NMHC and CO pressure error  surfaces  were included in the model.  For NOX and
CO2, no correlation could be made between the delta data and the pressure profile or between the
two PEMS. Therefore, the NOX and CO2 deltas were  not likely affected by the changes in
ambient pressure. NOX and CO2 pressure error surfaces were not included in the Model.  The
environmental  pressure error surfaces can be found in Appendix K.
SwRI Report 03.12024.06
Page 217 of 371

-------
     10
   Q.
      5
   Q  0
   c
   o
      -5 --•
          -PEMS2 »PEMS3

                                                                    **    -
   o
   o
   o
     -10 -
                                                                                --*»*
     -15
               -H	1	1-
                                H	H
                                           -H
                                               -H	1	H
                                                        -I-
                                                            -+-
                                                                               H	1—
                                                                                  70
                  10
                            20         30          40         50
                                    Reference Gas Delta Observation
                                                                       60
   FIGURE 151. NMHC CORRECTED DELTA DATA FOR THE ENVIRONMENTAL
                                 PRESSURE TESTING

       Similar to the temperature  error surfaces,  pressure delta data was  spread to cover 360
observations for each PEMS at the zero, audit,  and span levels.  With only 2 PEMS evaluated
during pressure testing, the final NMHC and CO error surfaces were randomly sampled using
720 observations.  Figure 152 shows the final NMHC error surface generated during pressure
testing. Delta data was linearly interpolated between the zero, audit, and span deltas at a given
observation based on the concentrations in the reference NTE events.
SwRI Report 03.12024.06
Page 218 of 371

-------
                                                                                 800
                                 NMHC Pressure Observation [1 to 720]
             - NMHC Span 663ppm NMHC
                                   > NMHC Audit 159.9ppm NMHC
                                                         -NMHCZeroOppm NMHC
FIGURE 152. FINAL ERROR SURFACE FOR ENVIRONMENTAL PRESSURE NMHC
                                   CONCENTRATION

       A process similar to the temperature  exhaust flow rate error surface calculation method
was used to generate the pressure exhaust flow rate error surface.  The pressure exhaust flow
data was corrected for baseline bias and variance.  The  zero deltas were removed from the
baseline flow meter data. Interesting, all exhaust flow rate deltas were greater than zero during
pressure testing, therefore removal of zero data was not necessary.  Although minor, the pressure
flow rate data was  corrected  for the baseline  bias.  After  the bias correction, the variance
correction was applied using the scaling factor calculation.  Due to excessive variability  of the
pressure flow rate data,  the scaling  factor was calculated to  be 0.995, and therefore had little
effect on the pressure flow rate data.  The final pressure exhaust flow rate error surface is shown
in Figure 153.   The pressure flow rate error surface was sampled randomly and without level
dependence.
SwRI Report 03.12024.06
Page 219 of 371

-------
   £
   HI
   *ro
   a:
   5
0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0.0

-0.1
                                                                     -::*.
                   V
                  _»_.
                 50
                           100        150       200       250
                                  Exhaust Flow Zero Observation
                                                                  300
                                                                            350
  FIGURE 153. ERROR SURFACE FOR ENVIRONMENTAL PRESSURE EXHAUST
                                    FLOW RATE
5.6    Radiation Chamber Testing

       Radiation chamber testing was performed with one  SEMTECH-DS to quantify PEMS
gaseous concentration and exhaust flow measurement errors due to Electromagnetic Interference
(EMI)  and Radio Frequency Interference (RFI).  The four Society  of Automotive Engineers
(SAE)  tests selected for radiation chamber testing were  Bulk Current Injection,  Radiated
Immunity, Electrostatic Discharge, and Conducted Transients.

5.6.1   Bulk Current Injection

       SAE test 111 13/4 titled Immunity  to Radiated Electromagnetic Fields-Bulk Current
Injection  (BCI)  Method  was  performed  to   evaluate  the  PEMS response  to  radiated
electromagnetic fields on the PEMS cabling. Based on the SAE Standard test descriptions and
recommendations from SwRI specialists, the Steering Committee elected to test the PEMS using
the specifications detailed in Region 2, Class B of the Jl 113/4 test protocol. As shown in Figure
154, a  calibrated current probe was used to inject RF current into the PEMS  cables.  For each
test, the probe was positioned 120 mm, 450 mm,  and 750 mm from the cable connector. In other
words,  a complete test was performed with the probe located 120 mm from the cable connector.
Another complete test was performed  with the probe located 450 mm from the cable connector,
and another test at 750 mm. Each test consisted of stepping the current probe  frequency from 1
MHz to 400 MHz. Listed below, SwRI used the maximum frequency step size as stated in the
SAE test protocol.
SwRI Report 03.12024.06
                               Page 220 of 371

-------
   •   1 MHz to 10 MHz - 1 MHz step size
   •   10 MHz to 200 MHz - 10 MHz step size
   •   200 MHz to 400 MHz - 20 MHz step size

       The SAE standard called for a minimum dwell time of 2 seconds at each frequency.
However, SwRI used a dwell time of 5 seconds to insure the electromagnetic field had stabilized.
As specified by the SAE Standard, the current probe was calibrated to deliver 60 milliamps of
current. Figure 155 shows the device used to calibrate the bulk current injection probe.
                 FIGURE 154. BULK CURRENT INJECTION PROBE
SwRI Report 03.12024.06
Page 221 of 371

-------
   FIGURE 155. CALIBRATION DEVICE FOR THE BULK CURRENT INJECTION
                                        PROBE

       During Bulk Current Injection testing, the PEMS was powered with the  Sensors Inc.
inverter as well as a 12-volt automotive battery.  The battery was used during radiation testing to
simulate PEMS field testing.  Figure 156 shows PEMS 7 setup in a radiation chamber during
Bulk Current Injection testing. Similar to baseline testing, the PEMS was zeroed  and spanned
after warming for over one hour. The PEMS was zeroed approximately ever hour during testing.
Unlike temperature and pressure testing, which were continuous 8-hours tests, several BCI tests
were completed each hour.  The Steering Committee elected to take advantage of the segmented
radiation testing and add short periods of baseline or zero stimulation testing during each hour of
testing.  As observed with temperature and pressure testing, it was often difficult to determine if
the cause of increased delta measurements was due to changes in the environmental condition
being tested, or some other factor.  Adding periods of baseline testing throughout the BCI test
offered direct comparison of PEMS measurement deltas with and without radiation stimulation.
The baseline comparisons  aided in determining if PEMS  deltas were caused by BCI or some
other factor.
SwRI Report 03.12024.06
Page 222 of 371

-------
    FIGURE 156. PEMS 7 IN THE RADIATION CHAMBER UNDERGOING BULK
                          CURRENT INJECTION TESTING

       The cables evaluated during BCI testing included the 12-volt power supply cables, the
auxiliary 1  cable, the Ethernet cable, the temperature/relative humidity probe cable, the heated
line power  cable, and the EFM cables.  Initially, PEMS 3 was used for BCI testing.  However,
while testing the power cable, the PEMS reported several FID faults and eventually shut down
the FID due to high FID temperature.  The probe current was reduced from 60 milliamps to 40
milliamps, but the FID still shutdown between 26 and 46 MHz. Fearing a problem with PEMS
3,  PEMS 7 was used for BCI testing.  Although similar FID faults and problems existed with
PEMS 7, the  issues occurred less frequently.  When testing the power cable, the probe current
was reduced to 40 milliamps due to FID shutdown problems at 60 milliamps.  All other cables
were tested at 60  milliamps.  Throughout testing,  a number of faults and  problems occurred.
Most faults were related to the FID.  According to Sensors Inc., the FID faults and shutdowns
may have been related to problems with the FID DC to AC  board. Communication between the
laptop  and  PEMS unit was disrupted several times.  Communication between the PEMS and
EFM was also disrupted, requiring the PEMS to be restarted to restore communication.  After the
PEMS was  restarted, the analyzers were zeroed and spanned before continuing the BCI test.
SwRI Report 03.12024.06
Page 223 of 371

-------
       Figure  157 shows  the PEMS 7 gaseous  emission concentration zero  deltas measured
during Bulk Current Injection testing.  PEMS zero events  as well as the periods of baseline
testing are marked in Figure 157.  Although difficult to see in the chart, NO zero deltas were
within a range of ±5 ppm and  showed no noticeable  difference between BCI testing  and the
baseline portions  of the test.  In general, NO2 zero deltas during BCI and baseline testing were
between  0 and 5  ppm.   One NO2 measurement showed an  outlying positive delta of 7.6  ppm
while another outlying NO2 delta was at -15.5 ppm.  With the exception of the two outlying NO2
zero measurements, NO2 deltas  showed no difference between BCI and baseline testing.  With
the exception  of  the first  hour of testing, THC zero  deltas were typically within ±1 ppmC.
Although the  THC zero measurement drifted downward during the first hour of testing, the
baseline testing at the beginning and end of the hour test segment showed similar drift behavior;
indicating the drift  was  not caused by current injection.   In general,  CO  and CO2  zero
measurements  showed similar deltas during baseline and BCI testing.  However, 4 CO  zero
observations and  1  CO2  observation  were  outlying, low deltas that indicated  a  possible
susceptibility to  the BCI  test.   With the exception of a few  outlying points, there  was no
noticeable difference between the measurements taken during BCI testing and those taken during
baseline testing.  Therefore, the effect of the electromagnetic  radiation on the PEMS cabling was
minor.
               • THC [ppmC] BCO [ppm] «NO [ppm] XNO2 [ppm] «NOx [ppm] - Baseline Zero ACO2 [%]
                                 Reference Gas Delta Observation

      FIGURE 157. PEMS 7 ENVIRONMENTAL RADIATION BCI ZERO DELTA
                                  MEASUREMENTS


       Figure 158 shows the PEMS 7 audit delta measurements during BCI testing.  The shift in
audit  delta levels after the PEMS was restarted  was  due to a zero and span event for all
analyzers.  Similar to the BCI zero deltas, almost all of the BCI audit deltas were similar to the
baseline audit deltas, indicating BCI testing had little effect on the PEMS gaseous measurement
systems.  The only exceptions were 1 low CO audit measurement and 5 high CO2 measurements.
SwRI Report 03.12024.06
Page 224 of 371

-------
           •  THC [ppmC]
           -  Baseline
CO [ppm]
Zero
 NO [ppm]      X NO2 [ppm]
-Restart PEMS   A CO2 [%]
NOx [ppm]
                                                                                0.30
                                                                                0.00
                                Reference Gas Delta Observation
     FIGURE 158. PEMS 7 ENVIRONMENTAL RADIATION BCI AUDIT DELTA
                                  MEASUREMENTS

       The span deltas measured during BCI testing are shown in Figure 159. The PEMS was
restarted during the BCI test to restore communication with the EFM. The PEMS was zeroed
and spanned after being restarted, resulting in a shift in span delta measurements. Similar to the
BCI zero and audit deltas, the BCI span deltas nearly all matched the baseline deltas through the
environmental test.  However, 1 CO span delta observation was outlying and low, 5 CO2 span
deltas were high,  and 1 NO2 span delta was high.  Because of the vast similarity between BCI
deltas and baseline deltas, the effect of the  Bulk Current Inject testing  on the  PEMS span
measurements was minor.
SwRI Report 03.12024.06
         Page 225 of 371

-------
                                                                               0.375
                                                                               0.300
                                                                               0.225  «
                                                                               0.150  *
                                                                                   •E
                                                                                   u
                                                                                   o
                                                                                   c
                                                                                   o
                                                                                   o
                                                                               0.075
                                                                                   O
                                                                                   O
                                                                               0.000
                                                                               -0.075
                                Reference Gas Delta Observation
      FIGURE 159. PEMS 7 ENVIRONMENTAL RADIATION BCI SPAN DELTA
                                  MEASUREMENTS

       As shown in  Figure 160, the 5-inch EFM was susceptible to the BCI test.  The EFM
reported several elevated measurements throughout BCI testing.  EFM zero deltas were near zero
during baseline test  segments,  indicating  the positive EFM measurements were  most likely
caused by the BCI tests.
SwRI Report 03.12024.06
Page 226 of 371

-------
                                  • Flow Meter [scfm] • Baseline
                                  Reference Gas Delta Observation
   FIGURE 160. 5-INCH EFM ENVIRONMENTAL RADIATION BCI ZERO DELTA
                                  MEASUREMENTS
5.6.2   Radiated Immunity

       SAE test 111 13/21 titled Electromagnetic  Compatibility Measurement Procedure for
Vehicle Components - Part 21: Immunity to  Electromagnetic Fields,  10 kHz  to  18 GHz,
Absorber-Lined Chamber was  performed  to  evaluate the  PEMS  response  to continuous
narrowband electromagnetic fields on the PEMS and PEMS cabling.   Based  on  the  SAE
Standard test descriptions and recommendations from SwRI specialists, the Steering committee
elected to test the PEMS using the specifications detailed in Region 2, Class B of  the 111 13/21
test protocol.  As  shown in  Figure  161,  the Radiated Immunity test was  performed in an
absorber-lined  radiation test room.  The absorber medium was carbon-impregnated foam.  A
series of antennas and a host of power electronics, shown in Figure 162, were used to generate
the RF fields specified in the SAE test protocol.
SwRI Report 03.12024.06
Page 227 of 371

-------
  FIGURE 161. PEMS 7 AND RADIATION ANTENNA IN THE ABSORBER-LINED
       RADIATION CHAMBER DURING RADIATED IMMUNITY TESTING
SwRI Report 03.12024.06
Page 228 of 371

-------
    FIGURE 162. SIGNAL GENERATORS, AMPLIFIERS, OSCILLOSCOPES AND
        OTHER ELECTRONICS USED TO PERFORM RADIATION TESTING

       The Radiated Immunity tests consisted of stepping the electromagnetic field frequency
from 10 kHz to 1 GHz. Experts at SwRI recommended ending the Radiated Immunity testing at
1 GHz rather than 18 GHz as specified by the standard.  Because the maximum oscillator speed
of the  PEMS was relatively low, testing at higher frequencies would have likely shown no
measurement or operational susceptibilities. Listed below, SwRI used the maximum frequency
step size as stated in the SAE test protocol. The SAE standard called for a minimum dwell time
of 2 seconds at each frequency. However, SwRI used a dwell time of 5 seconds to insure the
electromagnetic field had  stabilized. Although the SAE Standard specified using a field intensity
of 50 volts/meter, the intensity was  reduced during testing  to  prevent the PEMS FID  from
shutting down.

   •   10 kHz to 100 kHz-10 kHz step size
   •   100 kHz to 1 MHz - 100 kHz step size
   •   1 MHz to 10 MHz - 1 MHz step size
   •   10 MHz to 200 MHz - 2 MHz step size
   •   200 MHz to 1 GHz - 20 MHz step size

       During Radiated Immunity testing, PEMS 7 was operated in a manner similar to  Bulk
Current Injection Testing. The PEMS was powered with the  Sensors Inc. inverter as well as a
12-volt automotive battery. Similar to baseline testing, the PEMS was zeroed and spanned after
warming for over one hour.  The PEMS was zeroed approximately every hour during Radiated
SwRI Report 03.12024.06
Page 229 of 371

-------
Immunity testing.  Also, short periods of baseline or zero stimulation testing were included
during each hour of testing to determine if the PEMS deltas were  caused by electromagnetic
radiation or some other factor.

       During initial Radiated Immunity testing, the field intensity was set to the SAE Standard
specification  of  50  volts/meter.  At  50 volts/meter, PEMS 7 produced numerous  faults  and
warnings pertain to the FID, and eventually shutdown the FID. Testing was then repeated with a
field intensity of 25 volts/meter.   PEMS 7 displayed several  FID faults and warnings,  and
shutdown the FID at an electromagnetic frequency of 164 MHz.  The Radiated Immunity  test
was  repeated a  third  time, this time at  the CE Standard field intensity specification of 10
volts/meter.   At 10 volts/meter PEMS 7 performed similarly to the 25 volts/meter test,  and
shutdown the FID at  164 MHz.  After further testing, it was determined that the PEMS FID
would not operate with a field intensity of 10 volts/meter between the frequency range of 164 to
178 MHz. Therefore, testing was performed until the FID shutdown at 164 MHz. After the FID
was restarted, testing was continued from 178 MHz. Throughout Radiated Immunity testing, the
field intensity was set as high as possible without causing functional PEMS  failures.  If a large
number faults occurred, or if the PEMS FID shutdown, the radiation test was often repeated  at a
lower field intensity.

       Figure 163 shows the PEMS 7 zero deltas measured during Radiated Immunity  testing.
Following the initial baseline testing, the electromagnetic  frequency was ramped from 30 to  164
MHz at 25 volts/meter in a horizontal direction. At observation number 36 with a frequency of
164 MHz, the PEMS FID shutdown. The PEMS was  then restarted. After baseline observations
37 through  45,  the electromagnetic  frequency  was ramped  from 30 to  164 MHz at  10
volts/meter, observation  number 46 through 60.  Even at 10 volts/meter, the FID shutdown at
164 MHz.  Testing  was then continued with the radiation frequency ramped from  178 to  260
MHz at 10 volts/meter during reference gas observations 61  through 72. The FID shutdown
again at 260 MHz and 10 volts/meter.  During measurement 73 through 99, the electromagnetic
radiation was ramped from 260 to  300 MHz at 10 volts/meter, 300 MHz to  1000  MHz at 25
volts/meter, and  after changing antennas, from 200 to 1000 MHz at 25 volts/meter in a vertical
direction.  A short baseline segment was included from  observation 91  through 96.  Radiated
Immunity testing continued by ramping vertically from 30 to 50 MHz at 10 volts/meter and 50 to
200 volts/meter at 25 volts/meter.  Using a bipolar antenna, the electromagnetic frequency was
ramped from 10 kHz to 6 MHz at 25  volts/meter and  from 7 to 30 MHz  at 10 volts/meter.
Testing was concluded by recording  deltas with no  radiation to generate a  final baseline  test
segment.
SwRI Report 03.12024.06                 Page 230 of 371

-------
                 THC [ppmC]  •  CO [ppm]    • NO [ppm]    X  NO2 [ppm]
                 Baseline    A  Zero      	Restart Test  A  CO2 [%]
                              NOx [ppm]
     350
                                                                               0.025
   E
   Q.
   £
   HI
   Q
   c
   o
   HI
   o
   c
   o
   o
                                Reference Gas Delta Observations

   FIGURE 163. PEMS 7 ENVIRONMENTAL RADIATION RADIATED IMMUNITY
                           ZERO DELTA MEASUREMENTS

       Throughout  Radiated Immunity testing, the  only  zero deltas  that  showed  notable
perturbations due to electromagnetic radiation were for CO when ramped from 120 to 164 MHz
at 25 volts/meter. One low CO2 concentration  delta was recorded during radiation testing.  All
other deltas resembled baseline measurements, indicating measurement errors were not caused
by electromagnetic radiation.

       Figure  164 and Figure 165 show the PEMS 7 audit and span deltas recorded during
Radiated Immunity testing.  Similar to the zero  deltas, the electromagnetic radiation affected the
CO audit and span measurements from 120 to 164 MHz at 25 volts/meter. One high CO2 audit
and span  delta was observed near 120 MHz  at 25  volts/meter.   All other audit and span
measurements appeared unaffected by the electromagnetic radiation.
SwRI Report 03.12024.06
Page 231 of 371

-------
                THC [ppmC]  •  CO [ppm]   •  NO [ppm]    X NO2 [ppm]
                Baseline    A  Zero      	Restart Test   A CO2 [%]
                                                        NOx [ppm]
     350

     300

     250

     200

     150

     100

      50

       0

     -50

     -100

     -150
M»li»iil«>»
                   jiA*!
0.350

0.315

0.280

0.245

0.210

0.175

0.140

0.105

0.070

0.035

0.000
                                Reference Gas Delta Observations
   FIGURE 164. PEMS 7 ENVIRONMENTAL RADIATION RADIATED IMMUNITY
                          AUDIT DELTA MEASUREMENTS
       THC [ppmC]  •  CO [ppm]    • NO [ppm]    X  NO2 [ppm]
       Baseline    A  Zero     	Restart Test  A  CO2 [%]
                                                                 NOx [ppm]
                               Reference Gas Delta Observations
   FIGURE 165. PEMS 7 ENVIRONMENTAL RADIATION RADIATED IMMUNITY
                           SPAN DELTA MEASUREMENTS
SwRI Report 03.12024.06
                          Page 232 of 371

-------
       Figure 166 shows the exhaust flow meter zero deltas recorded during Radiated Immunity
testing.   Although most measurements were near  zero, several segments of radiation testing
showed elevated EFM readings.  EFM susceptibility  was recorded at a field intensity of 25
volts/meter and electromagnetic frequencies ranging from 120 to 164 MHz (horizontal polarity),
300 to 1000 MHz (horizontal polarity), and 200 to 1000 MHz (vertical polarity).
                                    * Flow Meter [scfm]  Baseline
     30
     25 -
   I20
   u
   ~ 15
   re lo
   t 10
     -5
                       •

                      v~
                         50
                                 Exhaust Flow Meter Zero Observations
FIGURE 166. 5-INCH EFM ENVIRONMENTAL RADIATION RADIATED IMMUNITY
                           ZERO DELTA MEASUREMENTS
5.6.3   Electrostatic Discharge

       SAE test 111 13/13  titled Electromagnetic Compatibility Measurement Procedure for
Vehicle Components—Part 13: Immunity to Electrostatic Discharge was performed to evaluate
the PEMS response to Electrostatic Discharges (ESDs) on the PEMS and auxiliary equipment.
Based  on the SAE Standard test descriptions and recommendations from  SwRI specialists, the
Steering committee elected to test the PEMS using the specifications detailed in Region 2, Class
B  of the 111 13/13  test protocol.  ESDs were delivered at over 80 locations on the PEMS, the
EFM, the EFM pressure transducer enclosure, the humidity probe, and PEMS connectors. Using
the Electrostatic Discharge Simulator shown in Figure 167, the discharge was delivered directly,
or with the simulator tip  touching  the  discharge surface;  as  well  as indirectly,  or with the
simulator tip not touching the discharge surface.  The direct discharge was performed by placing
the simulator  tip  on the discharge  surface and energizing the discharge gun.   The  indirect
discharge was  performed by energizing the discharge gun away from the discharge surface. The
energized  gun tip  was then moved towards the discharge  surface until the voltage potential
SwRI Report 03.12024.06
Page 233 of 371

-------
caused an arc and the discharge was released. Both direct and indirect ESDs were performed at
each of the discharge locations.
     FIGURE 167. ELECTROSTATIC DISCHARGE SIMULATOR USED DURING
                     ELECTROSTATIC DISCHARGE TESTING

       As specified by the SAE Standard test procedure, the Electrostatic Discharge Simulator
was calibrated to deliver 4000 volts.  An Electrostatic Voltmeter, shown in Figure 168, was used
to calibrate the Electrostatic Discharge Simulator.
SwRI Report 03.12024.06
Page 234 of 371

-------
     FIGURE 168. ELECTROSTATIC VOLTMETER USED TO CALIBRATE THE
                   ELECTROSTATIC DISCHARGE SIMULATOR

       During Electrostatic Discharge testing, PEMS 7 was operated in a manner similar to Bulk
Current Injection and Radiated Immunity testing.  The PEMS was powered with the Sensors Inc.
inverter as well as a 12-volt automotive battery.  Similar to baseline testing, the PEMS was
zeroed and spanned after warming for over one  hour.  Due to the reduced length of the ESD
testing, and the inclusion of baseline test segments, PEMS 7 was  only zeroed once during the
Electrostatic Discharge test.  Periods of baseline or zero stimulation testing were included at the
beginning and end of the ESD test to determine if the electrostatic discharge had any effect on
the PEMS measurements. To capture potential measurement errors, the discharge events, being
extremely brief, were timed to occur  during the 30-second  recorded measurements.  Several
discharge locations were tested during each 30-second measurement.

       Figure 169 shows the PEMS 7 zero delta measurements during Electrostatic Discharge
testing.  Baseline test segments were included at the beginning and end of the test, with one zero
event occurring at observation number 28.  In general, the zero deltas during testing resembled
the deltas measured during  baseline testing, indicating the ESD testing had little  effect on the
PEMS measurement. Two  CC>2 zero measurements were outlying and high.  With the ESDs
being extremely brief,  short perturbations in the PEMS measurements may  not  have  been
revealed in a 30-second average measurement. Therefore, the continuous data was reviewed for
each PEMS measurement to insure a short duration measurement error was not overlooked.  The
continuous data showed no evidence of susceptibility to the ESDs.
SwRI Report 03.12024.06
Page 235 of 371

-------
              • THC [ppmC] BCO [ppm] »NO [ppm] XNO2 [ppm] «NOx [ppm] -Baseline  Zero ACO2 [%]
                                                                            0.050
                                                                            0.045
      Q
      c
      o
      B
      re

      m
      u
      c
      o
      o
                                Reference Gas Delta Observations

      FIGURE 169. PEMS 7 ENVIRONMENTAL RADIATION ELECTROSTATIC
                   DISCHARGE ZERO DELTA MEASUREMENTS

       Figure  170 and Figure 171 show the PEMS 7 audit and  span  deltas measured during
Electrostatic Discharge  testing.   After  the zero event at observation number  10,  the  delta
measurements  for NO, NO2, and CO2 had a noticeable shift.  Although stable prior to the zero
calibration, the NO, NO2, and CO2 delta measurements showed noticeable positive drift after
being re-zeroed.   The baseline test segment at the end of the test showed no  shift in  delta
measurements  compared to those during ESD  testing.  Therefore, the deltas observed during
ESD testing were not likely caused by the Electrostatic Discharge. Furthermore, the continuous
data for each PEMS  measurement was reviewed to insure short  duration  measurement
perturbations were not overlooked using the 30-second mean measurement.  The continuous data
showed no evidence of short term measurement errors.
SwRI Report 03.12024.06
Page 236 of 371

-------

Concentration Delta [ppm]
hj .&. O CO C
D 0 0 0 0 C
-20
-4D
»THC [ppmC] BCD [ppm] «NO [ppm] XNO2 [ppm] »NOx [ppm] -Baseline Zero ACO2 [%]

'
A. • • A * A A A '
" • MJL • A A A A
A A A J »* " "t" .
•
A
A
A
5 8 * !Bx2£^5!lp*lXS*™»*25 30
~ • ( • •
"A" «
A

3 0 0 0 0 C
J Ul ^J O NJ C
n o m o m c
32 Concentration Delta [%]

- -n n?R
                                Reference Gas Delta Observations
      FIGURE 170. PEMS 7 ENVIRONMENTAL RADIATION ELECTROSTATIC
                   DISCHARGE AUDIT DELTA MEASUREMENTS
            »THC [ppmC] BCO [ppm] •NO [ppm] 5KNO2 [ppm] • NOx [ppm] -Baseline   Zero ACO2 [%]
     80
     60	-^	^-	0.070
     40
   Q.
   Q.
     20
   HI
   Q
   c
   o
S -20
u
c
o
o
  -40
     -80
                                                                            -  0.040 ^
                                               A A     •
                                             * * * * vv
                                                                            -  0.010
                        •  m 10
                        X  *
                        •  • A
                             X
,«x*fx*f°
X              • •
     -60	0.110
                                                                              0.100
                                                                           -0.020 ™
	0.050
                                                                            -- -0.080
                                                                              -0.140
                               Reference Gas Delta Observations
      FIGURE 171. PEMS 7 ENVIRONMENTAL RADIATION ELECTROSTATIC
                   DISCHARGE SPAN DELTA MEASUREMENTS


       Figure 172 shows the EFM measurements during BSD testing.  Nearly all of the EFM
zero deltas recorded during BSD testing were below 0.2 scfm.  Curiously, the largest zero deltas
SwRI Report 03.12024.06
                                 Page 237 of 371

-------
were measured at the end of the test during baseline testing.  The 30-second mean delta data
suggests BSD  testing had little affect on the exhaust flow measurements.  Similar to gaseous
concentration measurements, the continuous EFM data was examined and found to show no
evidence of short duration measurement errors.
                                    • Flow Meter [scfm]  Baseline
        3.5

        3.0

        2.5

        2.0

        1.5
      •a o.s
        o.o
        -0.5

        -1.0

        -1.5
                                                                           * *

                         »***
                   20
                            40
                                     60
                                              80
                                                       100
                                                                120
                                                                        140
                                  Exhaust Flow Meter Zero Observations
   FIGURE 172. 5-INCH EFM ENVIRONMENTAL RADIATION ELECTROSTATIC
                   DISCHARGE ZERO DELTA MEASUREMENTS
5.6.4   Conducted Transients

       SAE test  111 13/11  titled  Immunity  to  Conducted Transients on Power Leads was
performed to evaluate the PEMS response to transient voltage disturbances on the PEMS 12-volt
power supply cable.  Based on the SAE Standard test descriptions and recommendations from
SwRI  specialists, the  Steering committee  elected to test the PEMS  using the specifications
detailed in Region 2, Class B of the Jl 113/11 test protocol.  Due to the high current draw of the
PEMS, a Schaffner NSG 5200 Automotive Electronics Test System was rented to perform the
Conducted Transients testing.  A Schaffner test system is shown in Figure 173  The test system
included a Burst Generator Module, a Load Dump Module, a Pulse Generator Module, and  an
Automotive ECM Test System with PC.  The Schaffner system included all of the hardware and
software necessary  to perform each of the Conducted Transients as specified in the  SAE
Standard.
SwRI Report 03.12024.06
Page 238 of 371

-------
FIGURE 173. SCHAFFNER NSG 5200 AUTOMOTIVE ELECTRONICS TEST SYSTEM

       During Conducted Transients testing, the power cable to the PEMS heated  sample line
was disconnected to keep the PEMS current draw below 30 amps, which was the  limit of the
Schaffner test system.  The Schaffner NSG 5200 was connected in series between the PEMS and
12-volt power supply.  With the test system installed, the  PEMS supply voltage dropped to 8
volts using the Sensors Inc. power supply.  Therefore, a SwRI 12-volt power supply  was used to
power the PEMS during Conducted Transients testing.  Similar to the other radiation tests, a 12-
volt automotive battery was  connected in parallel with the power supply.  The PEMS supply
voltage was maintained at approximately 13 volts  during testing.  Using the Schaffner test
system, a number of voltage disturbances were introduced through the PEMS power supply
cable.  The voltage perturbations ranged from -200 to 100  volts with bursts as short as 250 ns
and voltage spikes lasting up  to 2, 4, or 200 ms.  The tests consisted of voltage spikes with slow
recovery, voltage spikes with quick recovery, repeated voltage bursts,  and a load dump.  All 200
ms voltage spikes with slow recovery would cause the PEMS  to shut down.  The  200 ms
duration tests were repeated at a quarter of the voltage disturbance amplitude, however, the
PEMS  continued to  shutdown.  Therefore, the response  of the PEMS to 200  ms voltage
disturbances was not characterized  in this experiment.  Similarly, the load dump  experiment
caused the PEMS to shutdown, therefore, no PEMS response data was gathered for  that portion
of the Conducted Transient testing.

       Figure 174 shows an example voltage trace of a voltage spike with slow recovery during
Conducted  Transient testing.  The voltage disturbance had an  amplitude of -100 volts and
recovery time of 4 ms.
SwRI Report 03.12024.06
Page 239 of 371

-------
           Tek Run: 500ks/s    Sample   iUBE
                          [	T	]
                                                                  C1 Min
                                                                 -104.0 V
                                                                  C1 Max
                                                                  13.6V
              Mill  20.0 V
     M  500JJS  Ch2 I  200mV 9 [\|OV 2006
                            16:21:10
  FIGURE 174. EXAMPLE VOLTAGE TRACE DURING A VOLTAGE SPIKE WITH
                                 SLOW RECOVERY

       Figure 175 shows an example voltage trace of a voltage spike with quick recovery during
Conducted Transient testing. The voltage disturbance had an amplitude of 50 volts and recovery
time of 0.05 ms.
SwRI Report 03.12024.06
Page 240 of 371

-------
           Tek Run: 10.0MS/S  Sample
                          [	T-
                                                                A: 54.2V
                                                                   67.0V
                                                                   C1 Min
                                                                   12.2V
              Bill   10.0 V
     M 25.0JJS  Ch2 f   200mV g NQV 2006
                            16:26:16
  FIGURE 175. EXAMPLE VOLTAGE TRACE DURING A VOLTAGE SPIKE WITH
                                 QUICK RECOVERY

       Figure 176 shows a voltage trace for a short duration voltage burst.  The voltage burst
had an amplitude of -150 volts, a rise time of 2 ns, and a recovery time of 100 ns. Voltage burst
tests consisted of repeating a series of 10 voltage bursts.
SwRI Report 03.12024.06
Page 241 of 371

-------
           Tek Run: I.OOCS/s    Sample
                          [	T-	
              ann   20.0 v
     M 250ns Ch1 f
                                                                A: 150.4 V
                                                                @: -136.0 V
-26.0 V 9 Nov 2006
       17:58:00
     FIGURE 176. EXAMPLE VOLTAGE TRACE DURING A VOLTAGE BURST

       Prior to testing, PEMS 7 was zeroed and spanned after warming for over one hour.
Periods of baseline or zero stimulation testing were included throughout the Conducted Transient
test to determine if the voltage disturbances had any effect on the PEMS measurements.  To
capture potential measurement errors, the voltage perturbations, being extremely brief, were
timed  to occur during  the  30-second recorded measurements.  Typically,  one Conducted
Transient test was performed during each 30-second measurement.

       Figure 177  shows the PEMS 7 zero delta measurements during Conducted  Transient
testing. Zero events occurred at zero observation number 16 and 77, while a day break occurred
at observation number 92; requiring the PEMS to be zeroed and  spanned. Due to the extended
length  of time required to  setup each Conducted Transient test,  a  large portion of the data
collected was zero stimulation baseline testing.  Differentiating between the test deltas  and
baseline deltas for each gaseous measurement is difficult, therefore, the voltage disturbances
delivered to the PEMS during Conducted  Transient testing had little  to no affect on the PEMS
measurements.
SwRI Report 03.12024.06
Page 242 of 371

-------
                   THC [ppmC]  •  CO [ppm]    • NO [ppm]    X NO2 [ppm]
                   Baseline    A  Zero      	Day Break    A CO2 [%]
                             NOx [ppm]
                                                                              0.028
                                                                              0.024
                                                                              0.020
                                                                              0.016
        -30
                                                                              0.012 5
                                             il
                                             &
                                             c
                                             o
                                             13
                                             £
                                                                                  O
                                                                              0.008 g
                                                                                  O
                                                                              0.004
                                                                              0.000
                                 Reference Gas Delta Observations
 FIGURE 177. PEMS 7 ENVIRONMENTAL RADIATION CONDUCTED TRANSIENT
                           ZERO DELTA MEASUREMENTS

       Figure 178 and Figure 179 show the audit and span deltas for PEMS 7 during Conducted
Transient testing. Zero events occurred at observation number 6 and 26.  The PEMS was zeroed
and  spanned following the day break  at observation number 31.  Although each  gaseous
pollutant exhibited audit and span deltas, the baseline deltas transitioned smoothly with the test
deltas, indicating the Conducted Transient testing had little  affect on the PEMS measurement
systems.
SwRI Report 03.12024.06
Page 243 of 371

-------
fin _
on Deltas [ppm]
= 5 g 8 g 8 S
2
I -10 -
u
8 -20
-30
-40
50
• THC [ppmC] • CO [ppm]

• NO [ppm] X NO2 [ppm] • NOx [ppm]


• ••
• " . . • • .
• • A
A r«A
A •
A A A_A A
AAA * ^ -"— -"-
fijl?"**?) ***!*»
jfjai********

____ ^ ___ __

m m
A " ••" "•
A
A A A •

••!• • •
**I *»******3*0 f"$|** 4
•••xxSlg • •
x**^X

A

Reference Gas Delta Observations
FIGURE 178. PEMS 7 ENVIRONMENTAL RADIATION CONDUCTED TF
AUDIT DELTA MEASUREMENTS
fin _
Deltas [ppm]
B & c
Concentration
3 k &
• THC[ppmC] • CO [ppm]

• NO [ppm] X NO2 [ppm] • NOx [ppm]



"I1.!
A
I AA* ••*£•*
*•**!* ixl**»**«
*•* • • •
!•!_ ..•*__ __

A A
A A A
A "*•"*•
A A A •
•
• i * * * *
r*s •••• • ? ---
25 • 30 x x 35 A x x 4
Jxjx*Xi* tJt*0 ••* •
g * A • A •

" •
0.160
0.145
0.131
0.116 ^
U)
0.102 |
0.087 o
13
0.073 •£
0)
0.058 §
O
0.044 o
O
0.029
0.015
0.000
IANSIENT
0 08
0.06
g
0.04 £
c
0
0.02 13
° 1
u
0.00 8
O
o
-0.02
-nn4
                               Reference Gas Delta Observations

 FIGURE 179. PEMS 7 ENVIRONMENTAL RADIATION CONDUCTED TRANSIENT
                          SPAN DELTA MEASUREMENTS

      Figure 180 shows  the EFM  zero measurements during  Conducted Transient testing.
Most of the exhaust flow meter 30-second average  measurements were less than 0.2 scfm.
SwRI Report 03.12024.06
Page 244 of 371

-------
Occasional  measurements  were  recorded  above  0.5   scfm,  however,  several  of  these
measurements occurred during baseline testing.  The maximum observed delta of 1.5 scfm is less
than 0.1% of the meter's 1700 scfm full scale flow rating.
      2.0
-*
Ol
      1.0  -
O
Ol
HI
•s
OL
^
t  0.0
     -0.5
                                                            * Flow Meter [scfm]   Baseline
                      *
                       *

                20
                        40
                                60
                                        80
                                                100
                                                         120
                                                                 140
                                                                         160
                                                                                 180
     -1.5
                                  Exhaust Flow Meter Zero Observations
      FIGURE 180. 5-INCH EFM ENVIRONMENTAL RADIATION CONDUCTED
                    TRANSIENT ZERO DELTA MEASUREMENTS
5.6.5  Radiation Error Surface Generation

       As  described in the Test Plan,  the measurement error data gathered during the Bulk
Current Injection, Radiated Immunity, Electrostatic Discharge, and Conducted Transients testing
was to be pooled to generate a radiation error surface. Because the radiation tests challenged the
PEMS at the most extreme radiation levels the equipment would likely be subjected to during in-
use testing, the radiation error surface was to be processed differently than the Temperature and
Pressure error surfaces. The pooled radiation data was to be corrected for  the environmental
baseline bias and variation.  The  5th, 50th, and  95th percentile values of the pooled, corrected
radiation data was to be used to generate error surfaces at the zero, audit and span levels. These
radiation error surfaces were to be sampled normally.

       As described in the previous radiation test sections, nearly all of the radiation delta data
resembled  the  corresponding  baseline  deltas; indicating  the  PEMS  showed  very  little
susceptibility to the radiation tests.  The only notable radiation susceptibility was for a few high
biased CO and  CO2 measurements.  Because the radiation  error surface would use the 95th
percentile deltas, the Steering Committee elected to calculate the  95th percentile deltas for zero,
audit, and span to determine if the high CO and  CO2 deltas caused by the radiation testing were
SwRI Report 03.12024.06
                                  Page 245 of 371

-------
outside the 95th percentile delta.  Shown in Figure 181 through Figure 183 are the pooled zero,
audit, and span delta measurements for all of the radiation testing.  The baseline test segments
were extracted from the pooled radiation delta data set.  The 95th percentile values are shown as a
line for each gaseous emission.  The high CO and CO2 deltas  were  greater  than the 95th
percentile; therefore, the  radiation error surfaces would not include  any measurement  error data
caused by radiation effects.  Based on this data, radiation error surfaces were not generated for
use in the Model.
                 THC [ppmC]  •  CO [ppm]    • NOx [ppm]
                 NOx95th%  A  CO2 [%]       CO2 95th "/
                                                      -THC 95th '
                                                                   -CO 95th '
     350
     300
                                                                                 0.05
                                                                                 -0.03
                                 Reference Gas Delta Observations
  FIGURE 181. COMBINED RADIATION CHAMBER ZERO DELTA TEST RESULTS
SwRI Report 03.12024.06
Page 246 of 371

-------
                 THC [ppmC]  •  CO [ppm]    • NOx [ppm]
                 NOx 95th %  A  CO2 [%]       CO2 95th "/
                                                     -THC 95th '
                                                                 -CO 95th '
     350
                                                                               0.35
                                                                               -0.05
                                Reference Gas Delta Observations
 FIGURE 182. COMBINED RADIATION CHAMBER AUDIT DELTA TEST RESULTS
                 THC [ppmC]   • CO [ppm]    •  NOx [ppm]
                 NOx 95th %   A CO2 [%]        CO2 95th °/c
                                                     •THC 95th %
                                                                  •CO 95th %
     350 ^
                                                                               0.56
                                                                               -0.16
                                Reference Gas Delta Observations
  FIGURE 183. COMBINED RADIATION CHAMBER SPAN DELTA TEST RESULTS
SwRI Report 03.12024.06
Page 247 of 371

-------
       Shown in Figure 184, the radiation exhaust flow rate zero measurements were pooled
after the baseline test segments were removed from each test. As directed in the Test Plan, the
radiation error surfaces were to be  generated  using  the  5th,  50th,  and 95th  percentile  delta
observations from the pooled measurement errors. However, the PEMS set all negative flow rate
measurements  to  zero.    With  an  incomplete  distribution  of negative  flow  rate  delta
measurements,  an  alternate  error  surface  generation method  was  developed.   All  zero
measurements were removed from the pooled radiation exhaust flow measurements as well as
the environmental baseline exhaust flow measurement data  set.  Approximately one third of the
data was removed from the both  the radiation and baseline  exhaust flow data sets. Removal of
the zero level exhaust flow rate  measurements was necessary to more accurately compare the
variance of the radiation and baseline error data.

       To avoid over counting exhaust flow rate measurement errors, it was necessary to remove
the bias and variance measured during environmental baseline testing  from the radiation exhaust
flow rate data.  With zero deltas  removed, the median exhaust flow rate measurement from the
baseline test  was subtracted from each radiation  exhaust  flow rate  delta.  This process was
inconsequential because the median baseline delta was less than 0.01 % of the  flow  meter's
maximum flow rating. Next the MAD of the baseline data was compared to the MAD of the bias
corrected radiation data. Using the equation below, the MAD values were used to  calculate a
scaling factor. Each radiation exhaust flow measurement was multiplied by the  scaling factor to
shrink the variance of the radiation data by the variance measured during environmental baseline
testing.
                  jMADRad2 -MAD
Scaling _ Factor =	
                                                             2
                                                           BL
SwRI Report 03.12024.06                 Page 248 of 371

-------
                             Raw Exhaust Flow • Corrected Exhaust Flow
     -0.2
                    100
                                200           300          400
                                     Reference Gas Delta Observation
                                                                      500
                                                                                  600
     FIGURE 184. POOLED EXHAUST FLOW RATE ZERO DELTAS MEASURED
                 DURING ENVIRONMENTAL RADIATION TESTING

       After the radiation  exhaust flow  rate data was  corrected for the baseline bias and
variance, the 5th, 50th, and 95th percentile deltas were generated for use in the radiation exhaust
flow rate error surface.  With an incomplete delta distribution, the Steering Committee elected to
calculate the 95th percentile of the corrected radiation delta data.  The negative of the 95th
                                    -th
                                                          ,th
percentile delta value was set to the 5  percentile and the 50  percentile was set to zero.  The
final error surface for radiation exhaust flow rate is shown in Figure 185.
SwRI Report 03.12024.06
Page 249 of 371

-------
                               1
                               0
                              -2
Exhaust Flow Rate Deltas
  [% of Max Flow Rate]

       0.3
	A	
                                          0.0
                                         -4-
                                          -0.3
                                    X-Axis - Single Point
  FIGURE 185. ERROR SURFACE FOR ENVIRONMENTAL RADIATION EXHAUST
                       FLOW RATE DELTA MEASUREMENTS
5.7    Vibration Table Testing

       Vibration testing was performed to determine PEMS gaseous concentration measurement
errors due to vehicle vibrations.  Due to mounting issues, the PEMS EFM was not used during
vibration testing. Shown in Figure 186, an Unholtz-Dickie Corporation electro-dynamic shaker
system was used to perform vibration testing with PEMS 3  in a Sensors Inc.  environmental
enclosure.  The  shaker system used random movement to reproduce vibration defined by the
desired Power Spectral Density (PSD).
SwRI Report 03.12024.06
 Page 250 of 371

-------
FIGURE 186. PEMS 3 IN AN ENVIRONMENTAL ENCLOSURE DURING VIBRATION
             TESTING USING AN UNHOLTZ-DICKIE SHAKER SYSTEM

       The vibration test was intended to simulate vibration typically experienced by the PEMS
during in-use on-road testing.  As stated in the Test Plan, the Steering Committee originally
decided to  test the  PEMS using the PSD from the Mil  Standard  810, US Highway  Truck
Vibration Exposure. However, Sensors Inc. independently performed the Mil Standard 810 and
observed functional failures shortly after commencing the vibration test.  After Sensors  Inc.
reported the failures to the Steering Committee,  SwRI was asked to generate vibration spectra
representative  of an on-road truck.  Vibration data  collected with an  Army M915A2  Semi-
Tractor, shown in  Figure  187,  was used to generate  vibration  spectra.   The vibration data
collected with the M915A2 Tractor was comprised of accelerometer data at 8 locations on the
truck. The accelerometers were located on the truck frame as well as the cab.
SwRI Report 03.12024.06
Page 251 of 371

-------
 FIGURE 187. ARMY M915A2 SEMI-TRACTOR USED TO GENERATE VIBRATION
                       SPECTRA FOR VIBRATION TESTING

       After reducing the raw vibration data, SwRI proposed a revised PSD.  However, the
M915A2 Tractor was limited to 55 mph during testing. With the speed of most on-road trucks
exceeding 55 mph, the Steering Committee elected to have SwRI generate a PSD at 70 mph.
The 55 mph, 70 mph, and Mil Standard 810 are shown in Figure 188.  The 70 mph PSD resulted
in  a vibration energy increase factor of 2.69 over the 55 mph PSD.  SwRI also performed a
comparison of the lateral acceleration data and vertical acceleration data. Overall, the lateral and
vertical accelerations were nearly equal, therefore, one PSD was used for vibration testing in
each PEMS axis. Over 86  % of the M915A2 Tractor vibration energy was below 500 Hz.
SwRI Report 03.12024.06
Page 252 of 371

-------
     0.000001
          10
                                                                                  10000
   FIGURE 188. POWER SPECTRAL DENSITIES EVALUATED FOR VIBRATION
                                       TESTING

       After warming for over one hour,  PEMS 3 was zeroed and spanned with no stimulus
prior to vibration testing.   Similar to temperature and pressure testing, the PEMS was zeroed
hourly, during which the shaker table was turned off.  Using the 70 mph PSD, PEMS 3 was first
tested for lateral, side-to-side vibration.  During the first hour of testing, the PEMS reported a
warning that the environmental enclosure heated line temperature was low.  Also, the enclosure
cooling fans were not operational.  The cause of the aforementioned problems  was due to a
failure of the  12-volt power connector from the PEMS to the environmental enclosure.  After
repairing the broken connector, PEMS 3 was tested for 3 more hours of side-to-side vibration.
Next, the PEMS was turned and tested for lateral, front-to-back vibration.  After only 10-minutes
of vibration testing, the PEMS reported a high temperature fault for the FID and automatically
shutdown the  FID.   The PEMS was  shutdown and  restarted, however, the PEMS could not
communicate  with  the compact flashcard during  the restart procedure.   Without flashcard
communication, the user cannot log onto the PEMS.  After trying several  different flash cards
and numerous  diagnostic measures, PEMS 3 was shipped to Sensors Inc. for repair. Sensors Inc.
diagnosed the problem as a failed ribbon cable and returned PEMS 3 to SwRI for further testing.
Unfortunately, most of the PEMS data was lost due to the cable failure.

       After considering the  PEMS failures  using the 70 mph PSD,  the  Steering Committee
elected to proceed with vibration testing using the 55 mph PSD.  With the possibility of another
functional failure, vibration testing was initially performed for only two hours in each direction
to generate vibration deltas for each axis.  Also, the  shaker table was turned off for a segment of
each hour of vibration testing to generate baseline  data.  The table was  also turned off during
zero and span operations.
SwRI Report 03.12024.06
Page 253 of 371

-------
       Figure 189 shows the PEMS  3 zero deltas measured during environmental vibration
testing. Zero events were performed hourly, with baseline test segments included during each
hour of vibration testing.  The first two hours of testing were lateral, side-to-side vibration, hours
3 and 4 were lateral, front-to-back vibration, hours 5 through 7 were vertical vibration, hour 8
was lateral, side-to-side vibration, and hour 9 was lateral, front-to-back vibration. NO, NC>2, CO,
and CO2 zero deltas showed no evidence  of vibration susceptibility when compared to the
baseline test segments.
              • THC [ppmC] BCO [ppm] »NO [ppm] XNO2 [ppm] »NOx [ppm]  Zero -Baseline ACO2 [%]
                                                                             0.020
                                                                             -0.015
                                 Reference Gas Delta Observations
        FIGURE 189. PEMS 3 ENVIRONMENTAL VIBRATION ZERO DELTA
                                  MEASUREMENTS

       Figure 190 shows the PEMS 3 THC zero delta measurements during vibration testing.
During test hour 3, which was lateral, front-to-back vibration, the PEMS showed elevated zero
deltas during vibration testing.  During the baseline segment of hour 3, the THC zero deltas
returned to near zero levels.  At the end of test hour number 3, the PEMS reported a fault
indicating the FID internal reference pressure was out of its limits. Curiously, the FID fuel bottle
was nearly empty at the end of test hour number 3.  When replacing FID fuel bottle, the quick-
connect device used to connect the FID bottle to the PEMS was found to be broken.  With a
functional quick-connect, the THC zero deltas showed no evidence of susceptibility to vibration
in any axis.  The elevated THC zero measurements and low FID fuel bottle pressure were likely
caused by the failure of the FID fuel bottle connection device.

       At the end of test hour 6, the PEMS reported a fault indicating the FID gas flow was too
high or too low.   After test 6,  the FID would  not zero or span, although the FID  interface
indicated the FID was operating at the correct temperature and the FID flame was lit. The PEMS
was restarted several times in attempt to restore FID operation, however, the FID would not zero
or span. Therefore, testing was continued without THC measurements after test hour 6. During
test hour number 9, a  loud pop was heard from the PEMS and the THC measurement returned.
SwRI Report 03.12024.06
Page 254 of 371

-------
According to Sensors Inc., the pop noise was likely the FID lighting during vibration testing.
When the FID  fuel bottle was removed after test 9, the quick-connect was again found to be
broken.  The connection failures were most likely due to the lateral, front-to-back vibration
causing the FID  fuel bottle to slide forward and backward in the PEMS  case and stress the
connection.
      30
      20
   O

   Q.
   Q.
      10
   £  o
   c
   O
   O
   O
-10
     -20
     -30
                                                           *THC [ppmC]  Zero  -Baseline
                                          Deltas during vibration testing
                                         Note: THC susceptibility caused by broken FID bottle quick connect
                                                              Note: FID failure after test 6
                       50
                                      100             150

                               * Deltas during baseline testing
                                                                      200
                                    Reference Gas Zero Observations
      FIGURE 190. PEMS 3 ENVIRONMENTAL VIBRATION THC ZERO DELTA
                                   MEASUREMENTS

       Figure 191  and  Figure  192 show the  audit  and span deltas  for  PEMS 3  during
environmental vibration  testing.   CO and CO2  audit and span deltas showed no  differences
between the vibration test data and baseline test data, indicating CO and CO2 measurements were
not susceptible to vibration. THC audit and span deltas were similar to the THC zero deltas and
showed elevated measurements during test hour number 3. As discussed previously, the elevated
THC  deltas during  hour 3  were likely caused by a broken FID fuel bottle connector.  With a
functional FID bottle connector, the THC audit and span deltas were similar for vibration and
baseline testing.  As discussed below, NOX deltas showed slightly elevated measurements during
lateral, side-to-side  vibration when compared to the baseline testing.  Most of the shift between
vibration and baseline testing was cause by the NO2 measurement, with NO measurements being
unaffected by the vibration  tests.
SwRI Report 03.12024.06
                                Page 255 of 371

-------
             • THC [ppmC] BCO [ppm] • NO [ppm] XNO2 [ppm] »NOx [ppm] -Baseline  Zero  Span ACO2 [%]
                                                                                 0.12
               -----*?
                                                                                 -0.02
                                   Reference Gas Delta Observations
        FIGURE 191. PEMS 3 ENVIRONMENTAL VIBRATION AUDIT DELTA
                                    MEASUREMENTS
             • THC [ppmC] BCO [ppm] • NO [ppm] XNO2 [ppm] «NOx [ppm]  Zero -Baseline  Span ACO2 [%]
                                                                                 0.16
         30	•	• - -•	0.12
                                                                            	0.04
                                                                               - -0.08
                                                                                 -0.12
                                   Reference Gas Delta Observations
         FIGURE 192. PEMS 3 ENVIRONMENTAL VIBRATION SPAN DELTA
                                    MEASUREMENTS

       Figure  193 shows the NOX span deltas for PEMS 3 during vibration testing.  During
lateral side-to-side vibration, test hours 1,2, and 8, there was a noticeable difference between the
SwRI Report 03.12024.06
Page 256 of 371

-------
deltas observed during vibration testing and the deltas  observed during baseline testing.  All
baseline deltas showed a definite negative shift compared to the vibration deltas directly prior to
and after the baseline  segment.  The slight NOX susceptibility to lateral, side-to-side vibration
was driven by the NO2 measurement deltas.
      10
      5
Q.  "
£
TO
"oi
Q  -5
c
o
"ra
1 -10
u
c
o
o
5 -15 f
     -20
                                                      • NOx[ppm]  Zero —Baseline  Span
                                    Note: Slight NOx susceptibility occurred during horizontal side-to-side vibration
                                   30*
                                            40
                                                      50
                                                               60
                                                                        70
                                                                                 80
            Baseline test segments
>*.     **•  **      -
                                  A_
                                  A
                                      __
                                                        A       A[  •  I
                                                 Baseline test segmen
                                    Reference Gas Delta Observation

      FIGURE 193. PEMS 3 ENVIRONMENTAL VIBRATION NOX SPAN DELTA
                                   MEASUREMENTS

       Nearly all of the deltas measured during  vibration matched the deltas measured during
the baseline test segments.  Furthermore, the NOX susceptibility to lateral vibration was minor
and only evident during span measurements. Therefore, the Steering Committee elected not to
generate an environmental vibration error surface for use in the Monte Carlo Model.

5.8    Ambient Hydrocarbon Testing

       The Ambient Hydrocarbon test was performed to  determine the PEMS FID response to
varying levels and compositions of hydrocarbon in the  ambient air.  One source of potential FID
measurement errors included the use of ambient air as the FID burner  air source. Hydrocarbon
in the  ambient air would enter  the FID reaction chamber as burner air,  causing measurement
inaccuracy.  Ambient air is also used to zero the FID, therefore, ambient hydrocarbons will also
affect the FID zero calibration.  During laboratory engine testing and environmental testing, the
PEMS was zeroed using bottled zero air.  Zero air was also overflowed to the FID burner air
inlet throughout the program.  It was necessary to eliminate ambient hydrocarbon contamination
during engine and environmental testing to insure errors due to ambient hydrocarbons were only
capture during the Ambient Hydrocarbon test.
SwRI Report 03.12024.06
                                  Page 257 of 371

-------
       Ambient Hydrocarbon testing was performed with PEMS 2.  Throughout testing, zero air
was overflowed to the inlet of the PEMS sample line.  Instead of overflowing zero air to the FID
burner air inlet as was done during engine and environmental testing, air mixtures with varying
levels and compositions of hydrocarbons were introduced to the FID burner during Ambient
Hydrocarbon testing.  Shown in Table 84, 9 combinations of Hexane and Methane were used to
contaminate the FID  burner air.  According to EPA, the Hexane and Methane concentrations
selected for Ambient Hydrocarbon testing were deliberately  chosen to exceed typical ambient
hydrocarbon variation to insure the test  capture the full range of possible THC measurement
errors. The hydrocarbon gas mixtures were generated using a 4 ppmC Hexane gas bottle, a 4
ppmC Methane gas bottle, a 16 ppmC Hexane bottle, and a 16 ppmC Methane bottle, all balance
air.  A gas divider was used to blend  the hydrocarbon gases 50/50 to achieve  the desired
concentration levels.   Two computer controlled electronic solenoid  manifolds  were used to
automatically control the hydrocarbon combinations.

  TABLE 84. HEXANE AND METHANE CONTAMINATION COMBINATIONS USED
                  DURING AMBIENT HYDROCARBON TESTING
Hydrocarbon
Combination
1
2
3
4
5
6
7
8
9
Hexane
[ppmC]
0
2
8
0
2
8
0
2
8
Methane
[ppmC]
0
2
8
2
8
0
8
0
2
       As  specified  in  the  Test Plan,  the FID  was  stabilized  using  one of  7 different
combinations of FID air hydrocarbon contamination.  After the FID had stabilized with a given
hydrocarbon combination, the FID was zeroed using zero air and spanned.  The PEMS was then
set to sample zero air which was overflowed to the inlet of the heated sample line.  Next, the FID
burner air was automatically cycled through the hydrocarbon combinations shown in Table 84.
Each hydrocarbon combination was sampled for 90 seconds; 60 seconds to purge and stabilize
and 30 seconds to record an averaged measurement.  Taken from the Test Plan, Table 85 shows
the test sequence used for Ambient Hydrocarbon testing.  The FID was first zeroed while using
the FID burner air hydrocarbon concentrations shown in the merged cells.  After the FID was
zeroed, the PEMS was set to sample zero air and the FID burner air was cycled through the 10
hydrocarbon combinations levels to the right of the merged cells. Although only 9 hydrocarbon
combinations were possible, the first combination was repeated at the end of each test. The test
sequence was repeated one time.
SwRI Report 03.12024.06
Page 258 of 371

-------
             TABLE 85. AMBIENT HYDROCARBON TEST SEQUENCE

Ambienet Hydrocarbons Test Sequence
Phase
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
Burner air
hydrocarbons during
Hexane,
ppm
0
2
8
0
2
Methane,
ppm
0
2
8
2
8
Burner air
hydrocarbons during
Hexane,
ppm
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
2
0
8
2
0
8
2
0
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
Methane,
ppm
0
2
8
2
8
0
8
0
2
0
2
8
0
8
0
2
0
2
8
2
8
2
0
2
0
8
0
8
2
8
0
2
8
2
8
0
8
0
2
0
2
8
0
8
0
2
0
2
8
2



Ambienet Hydrocarbons Test Sequence
Phase
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
Burner air
hydrocarbons during
Hexane,
ppm
8
0
2
8
Methane,
ppm
0
8


0
Burner air
hydrocarbons during
Hexane,
ppm
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
0
2
8
Methane,
ppm
8
0
2
0
2
8
2
8
0
8
0
2
8
2
8
0
8
0
2
0
2
8
0
8
0
2
0
2
8
2
8
0
2
0
2
8
2
8
0
8

      Figure 194 and Table 86 show the THC measurements for the first test as specified in the
Ambient Hydrocarbon test sequence.  As specified in the Test Plan, each test sequence was
repeated one time. The FID was zeroed with no FID burner air hydrocarbon contamination.  The
FID burner air was then  cycled through the  hydrocarbon  combinations shown in Table 86.
Having zeroed the FID with no Hexane or Methane FID  air contamination, the initial and final
THC measurements, both with no FID air contamination, were near zero. With 2 ppmC Methane
SwRI Report 03.12024.06
Page 259 of 371

-------
contamination in the FID burner air, the PEMS measured approximately 18 ppmC THC while
sampling zero air. With 8 ppmC Methane contamination, the PEMS reported approximately 58
ppmC THC while sampling zero air. Hexane contamination introduced to the FID burner air had
little effect on the PEMS THC measurement. The PEMS use a charcoal filter in the FID burner
air line to absorb ambient hydrocarbon prior to reaching the FID. The charcoal filter apparently
absorbed nearly all of the Hexane, but had little effect on the Methane contamination.
                                                            Test 1a THC 	Test 1b THC
              100
                     200
                             300
                                    400
                                           500     600
                                          Time [s]
                                                          700
                                                                  800
                                                                         900
                                                                                1000
        FIGURE 194. THC MEASUREMENTS FOR TEST 1 OF THE AMBIENT
                         HYDROCARBON TEST SEQUENCE
SwRI Report 03.12024.06
Page 260 of 371

-------
         TABLE 86. THC MEASUREMENTS FOR TEST 1 OF THE AMBIENT
                        HYDROCARBON TEST SEQUENCE
FID Air
Hexane
[ppmC]
0
2
8
0
2
8
0
2
8
0
FID Air
Methane
[ppmC]
0
2
8
2
8
0
8
0
2
0
Gas
Switch
Time [s]
90
180
270
360
450
540
630
720
810
900
Test 1a
THC
[ppmC]
0.5
18.3
58.1
16.6
58.1
2.9
58.9
2.9
17.8
0.3
Testlb
THC
[ppmC]
1.2
19.0
59.7
17.0
59.6
3.5
60.1
3.5
18.9
0.8
Pretest zero was performed with 0 ppmC Hexane and 0 ppmC Methane
      Figure 195 and Table 87 show the THC measurements for the second test as specified in
the Ambient Hydrocarbon test  sequence.   Prior to performing the second test, the FID  was
stabilized and zeroed with 2 ppmC Hexane and 2 ppmC Methane FID burner air contamination.
The FID air contamination was varied according to the concentrations specified in Table 87.  The
first and last measurements occurred with FID air hydrocarbon contamination levels similar to
the hydrocarbon combination used  to  zero the FID.  Therefore,  the initial and  final THC
measurements were  near zero.   With  8  ppmC  Methane contamination, the PEMS reported
approximately 39 ppmC THC while sampling zero air. At 0 ppmC Methane contamination, the
PEMS reported approximately -15 ppmC THC while sampling zero air. As observed during test
number 1, Hexane contamination had little  effect on the PEMS THC measurement.
SwRI Report 03.12024.06
Page 261 of 371

-------
                                          Test 2a THC 	Test 2b THC
                                                                              1000
                                         Time [s]
       FIGURE 195. THC MEASUREMENTS FOR TEST 2 OF THE AMBIENT
                        HYDROCARBON TEST SEQUENCE
         TABLE 87. THC MEASUREMENTS FOR TEST 2 OF THE AMBIENT
                        HYDROCARBON TEST SEQUENCE
FID Air
Hexane
[ppmC]
2
8
0
2
8
0
2
8
0
2
FID Air
Methane
[ppmC]
2
8
0
8
0
2
0
2
8
2
Gas
Switch
Time [s]
90
180
270
360
450
540
630
720
810
900
Test 2a
THC
[ppmC]
0.5
40.6
-16.1
40.0
-13.8
-0.1
-13.9
0.0
40.3
0.3
Test 2b
THC
[ppmC]
0.2
39.0
-16.3
38.7
-14.0
-0.7
-14.1
0.0
39.0
0.0
Pretest zero was performed with 2 ppmC Hexane and 2 ppmC Methane
      PEMS 2 would not zero with 8 ppmC Methane contamination in the FID burner air.  The
PEMS would not perform an  electronic zero with the high Methane FID  air  contamination
because  the  measured  THC zero concentration was outside the zero limits of the PEMS.
Therefore, test 3, 5, and 7,  each requiring the FID to be zeroed with  8 ppmC  Methane
contamination, were eliminated from  test sequence.  Each test repeat resulted in  similar zero
SwRI Report 03.12024.06
Page 262 of 371

-------
measurements with the exception of test 6.  Shown in Figure 196, the test 6a THC trace appeared
normal, but the test 6b THC measurement was lower than expected.  The test was repeated,
however, the test 6c THC trace  appeared normal, and the test 6d trace appeared  low.  No
explanation for the variation in the test 6 results was evident.  Also, test 9, which was identical to
test 6, had both repeats similar to test 6a and test 6c. The results from test 6 were not used in the
final data analysis due to the measurement variation and because test 9 was  a duplicate of test 6.
                                                                                1000
                                          Time [s]
        FIGURE 196. THC MEASUREMENTS FOR TEST 6 OF THE AMBIENT
                        HYDROCARBON TEST SEQUENCE
5.8.1  Ambient Hydrocarbon Error Surface Generation

      The 30-second mean zero air THC measurements were pooled from test 1, 2, 4, 8, and 9
of the Ambient Hydrocarbon test sequence.  The THC measurements were multiplied by 0.98 to
generate NMHC zero measurement errors.  As discussed previously, Hexane had little effect on
the PEMS THC measurement due to the use of a charcoal filter.  Therefore, the PEMS THC
measurement errors were driven by the Methane contamination in the FID burner air.  Shown in
Figure 197, the pooled PEMS NMHC errors generated a nearly linear relationship with respect to
the FID air Methane contamination concentration.
SwRI Report 03.12024.06
Page 263 of 371

-------
                                 FID Air Methane Contamination [ppmC]

       FIGURE 197. PEMS 3 NMHC RESPONSE VERSUS FID AIR METHANE
  CONTAMINATION MEASURED DURING AMBIENT HYDROCARBON TESTING

       The linear relationship between the PEMS NMHC zero measurement error and the FID
air Methane contamination concentration was used to calculate the 5th and 95th percentile values
for the Ambient Hydrocarbon error surface.  With the calculated NMHC response to ambient
Methane, the only task remaining was to determine appropriate  5th and 95th percentile values for
the  real world  ambient Methane levels.  A great deal of deliberation  was  exercised by the
Steering Committee to determine appropriate levels  of variation for ambient Methane.   As
written in the test plan, Methane levels recorded by CE-CERT  during the on-road validation
were used as  a reference.  During  CE-CERT's testing, the  maximum change in  Methane
concentration was 1.8 ppmC.  However, the objective  of the Ambient Hydrocarbon test was to
capture the worst case Methane variation.  Therefore, CE-CERT's Methane data could only be
used the minimum Methane variation.  Historical  Methane data was  then examined from EPA,
SwRI and engine manufacturer test labs.   The difference between the 5th and 95th percentile
values for each lab was  considered the worst case Methane variation.  Using the pool of Methane
data, the Steering Committee ultimately agreed to use a Methane variation of 2.2 ppmC.

       With no justification to bias the error surface,  the 5th percentile Methane concentration
was set to -1.1  ppmC,  while the 95th percentile Methane  concentration  was  set to 1.1 ppmC.
Using  the  linear relationship  between the PEMS NMHC response  and ambient  Methane
variation, the 5th and 95th percentile Methane concentrations were used to calculate the 5th and
95th percentile PEMS NMHC deltas for the final Ambient Hydrocarbon error surface. Shown in
               -th
                       -th
Figure 198, the 5  and 95  percentile NMHC deltas were -7.5 and 7.5 ppmC respectively.  The
Ambient Hydrocarbon error surface was sampled normally and applied to each reference NTE
event NMHC concentration regardless of level.
SwRI Report 03.12024.06
Page 264 of 371

-------
                               2 -,
                             E

                             •*,
                             o
                               -2
                                     NMHC Delta [ppmC]




                                          7.5
                                          0.0
I n	,
                                          -7.5
                                    Single X-Axis Point
    FIGURE 198. ERROR SURFACE FOR AMBIENT HYDROCARBON TESTING
SwRI Report 03.12024.06
      Page 265 of 371

-------
                     6.0    MODEL RESULTS AND VALIDATION

6.1    Model Results

       The objective of this section is to present the results from the Monte Carlo simulation
runs, the post-processing computations of the measurement allowances, and the model validation
analyses.  Useful background information is contained in Section 2.0 of this report where the
Monte Carlo error model is described along with the techniques and methods used in the model
simulation runs and the validation process.  A total of 10,000 to 30,000 simulation trials (with
four NTE events running to 50,000 trials) were  run for each of the reference 195 NTE events,
and approximately four million simulation trials  were run for all the combination of settings for
these  195 reference events.

6.2    Results of Drift Correction

       This section  contains a summary  of the number and percent of the simulation trials that
were  deleted  due to periodic drift.  Section 2.1.5  on Periodic Drift Check contains a detailed
description and a flowchart of the  procedure used to check whether or not a periodic drift
invalidated any of the reference NTE event trials. This procedure was applied to the simulation
data obtained for each of the three emissions for each of the three calculation methods.

       No periodic  drift was detected  for the  BSCO emissions for any of the three calculation
methods. Thus, no simulation trials were deleted in any of the drift correction checks for BSCO
for all 195 reference  NTE events.  However, for BSNOX  and BSNMHC,  periodic drift was
detected.

       Figure 199 through Figure 201  display relative frequency (in percent) histograms  for the
percent of simulation trials for BSNOx,  using each of the three calculation methods, that were
deleted for each of the 195 reference NTE events due to periodic drift. A summary of the results
is given in Table 88.  For the three  calculation methods, the average percent of the simulation
trials that was deleted across the  195 reference NTE events ranged from 2.09% for Method 2 to
3.45% for Method 1. The maximum percent that was deleted ranged from 11.27% for Method 2
to  15.41% for Method 1.
SwRI Report 03.12024.06                Page 266 of 371

-------
                             BSNOx Method 1
      0)
      O)
      CC
      +J
      c
      a)

      e
      0)
      0.
      >
      O
      c
      0)
60


50


40


30


20


10


 0
n
                D
                   2     4    6     8    10    12   14

                    % Trials Deleted Due to Periodic Drift
                                              16
   FIGURE 199. PERCENT OF TRIALS DELETED FOR EACH REFERENCE NTE

       EVENT DUE TO PERIODIC DRIFT CHECK FOR BSNOx METHOD 1
           60
       o

       t

       e
       «
       o
       c
       0)
       D
       CT
                             BSNOx Method 2
                    2    4    6     8    10    12   14

                    % Trials Deleted Due to Periodic Drift
                                              16
   FIGURE 200. PERCENT OF TRIALS DELETED FOR EACH REFERENCE NTE

       EVENT DUE TO PERIODIC DRIFT CHECK FOR BSNOx METHOD 2
SwRI Report 03.12024.06
                  Page 267 of 371

-------
                                  BSNOx Method 3
        0
        O)
        S
        0)
        o
             60
50
        £   10
                       2     4     6      8    10    12    14
                       % Trials Deleted Due to Periodic Drift
                                                    16
   FIGURE 201. PERCENT OF TRIALS DELETED FOR EACH REFERENCE NTE
        EVENT DUE TO PERIODIC DRIFT CHECK FOR BSNOx METHOD 3
   TABLE 88. SUMMARY OF THE TRIALS DELETED DUE TO PERIODIC DRIFT
                               CHECK FOR BSNOx
Method
1
2
3
# Reference
NTE Events
195
195
195
Mean %
3.45
2.09
2.11
Min %
0
0
0
Max %
15.41
11.27
11.32
      Figure 202 through Figure 204 display relative frequency (in percent) histograms for the
percent of simulation trials for BSNMHC, using each of the three calculation methods, that were
deleted for each of the 195 reference NTE events due to periodic drift.  A summary of the results
is given in Table 89. For the three calculation methods, the average percent of the simulation
trials that was deleted across the 195 reference NTE events ranged from 1.89% for Method 2 to
4.01% for Method 1. The maximum percent that was deleted ranged from 14.33% for Method 2
to 21.21% for Method 1.
SwRI Report 03.12024.06
                     Page 268 of 371

-------
                            BSNMHC Method 1
       0)
          100
                      4      8      12     16     20
                    % Trials Deleted Due to Periodic Drift
   FIGURE 202. PERCENT OF TRIALS DELETED FOR EACH REFERENCE NTE
      EVENT DUE TO PERIODIC DRIFT CHECK FOR BSNMHC METHOD 1
                            BSNMHC Method 2
IUU
0)
O)
re an
4-1 OU
c
0)
*- R(\
0) DU
a.
>
o An
^ 4U
0)
3
°" on
0) ZU
II
n
"

-
-
-
-
	






















i 	






-
-
-
-
-
-
                      4      8     12     16    20     24
                    % Trials Deleted Due to Periodic Drift
   FIGURE 203. PERCENT OF TRIALS DELETED FOR EACH REFERENCE NTE
      EVENT DUE TO PERIODIC DRIFT CHECK FOR BSNMHC METHOD 2
SwRI Report 03.12024.06
Page 269 of 371

-------
                                BSNMHC Method 3
a, IUU
O)
JS on
£ Oil
0)
o
»r en
d) OU
0.
O An
0)
S- on
CT £\J
£
"- o
•
-
-
•
-


















	 1





1 	












-
_
_
-
-
-
                          4       8       12      16      20      24
                      % Trials Deleted  Due to Periodic Drift
   FIGURE 204.  PERCENT OF TRIALS DELETED FOR EACH REFERENCE NTE
       EVENT DUE TO PERIODIC DRIFT CHECK FOR BSNMHC METHOD 3
    TABLE 89. SUMMARY OF TRIALS DELETED FOR EACH REFERENCE NTE
       EVENT DUE TO PERIODIC DRIFT CHECK FOR BSNMHC METHOD 3
Method
1
2
3
#RefNTE
Events
195
195
195
Mean %
4.01
1.89
1.93
Min %
0.05
0.00
0.00
Max %
21.21
14.33
14.35
6.3   Convergence Results from MC Runs

      This section contains  a summary of the checks to determine if the convergence criteria
were met for the simulation runs. Section 2.1.7 on Convergence and Number of Trials contains a
detailed  description of the convergence methodology and the procedures followed to check for
convergence for the reference NTE event trials.  This procedure was  applied to the simulation
data obtained for each of the three emissions and all three calculation methods.

      Figure 205 through Figure 207 contain relative frequency (in percent) histograms for the
widths of the 90%  confidence  intervals for the  95th percentiles of the  corresponding BSNOx
differences for the 195 individual reference NTE events where the confidence interval widths are
expressed as a percent of the BSNOx emissions NTE threshold. This is  done for each of the three
calculation methods. A summary of the results is given in Table 90. Of interest was whether or
SwRI Report 03.12024.06
Page 270 of 371

-------
not the simulations converged within 1%  of the threshold  value.   For the three calculation

methods, the maximum percent of the confidence interval widths that were within 1% of the

threshold value across the 195 reference NTE  events ranged from  0.481% for Method 2 to

0.984% for Method 1. Thus, all 195 reference events met the convergence criteria.
                                  BSNOx Method 1
        <1>
        O)
        CC

        £
        O
        O


        0.

        O
        c
        a)
        D

        0)
30
                                    T
                          0.2       0.4       0.6       0.8        1

                      Convergence Interval Width/Threshold, %
       FIGURE 205. CONVERGENCE INTERVAL WIDTH AS A PERCENT OF

                      THRESHOLD FOR BSNOX METHOD 1
SwRI Report 03.12024.06
                     Page 271 of 371

-------
                             BSNOx Method 2
ou
Q)
O)
CO /in
j3 4U
c
a)
*- in
0) oil
0.
>
o on
c 20
03
3
O" *r\
o nu
i
n
-
-
-
-
-
-





, 	

















n












-
-
-
-
-
-
               0      0.2      0.4      0.6      0.8       1
                  Convergence Interval Width/Threshold, %


     FIGURE 206. CONVERGENCE INTERVAL WIDTH AS A PERCENT OF
                   THRESHOLD FOR BSNOx METHOD 2
                             BSNOx Method 3
ou
O)
RJ Af\
+1 4U
*- in
O oU
0.
o on
c 20
O" -in

-------
   TABLE 90.  SUMMARY OF BSNOX CONVERGENCE INTERVAL WIDTH AS A
         FUNCTION OF THRESHOLD FOR 195 REFERENCE NTE EVENTS
Method
1
2
3
#RefNTE
Events
195
195
195
Mean %
0.534
0.313
0.444
Min %
0.236
0.157
0.190
Max %
0.984
0.481
0.889
      Figure 208 through Figure 210 contain relative frequency (in percent) histograms for the
widths of the 90% confidence intervals for the 95th percentiles of the corresponding BSNMHC
differences for the 195 individual reference NTE events where the confidence interval widths are
expressed as a percent of the BSNMHC emissions NTE threshold. This is done for each of the
three calculation methods.  A summary  of the results is given in Table 91.  Of interest was
whether or  not the simulations converged within  1% of the  threshold value.  For the three
calculation methods, the maximum percent of the confidence interval widths that were within 1%
of the threshold value across the 195 NTE events ranged from 0.429% for Method 2 to 0.604%
for Method  1.  Thus, all 195  events met the convergence criteria.
                                 BSNMHC Method 1
        O)

        c
        0)
        O
        03
        0.
        >
        O
        c
        0)
             80
60
40
             20
                          0.2       0.4       0.6       0.8
                      Convergence Interval Width/Threshold, %
       FIGURE 208. CONVERGENCE INTERVAL WIDTH AS A PERCENT OF
                     THRESHOLD FOR BSNMHC METHOD 1
SwRI Report 03.12024.06
                     Page 273 of 371

-------
                            BSNMHC Method 2
       0)
       O)
       CC
       +J
       c
       0)

       e
       a)
       0.

       >
       O
       c
       0)
40
           20
                      0.2      0.4      0.6      0.8       1

                  Convergence Interval Width/Threshold, %
     FIGURE 209. CONVERGENCE INTERVAL WIDTH AS A PERCENT OF

                  THRESHOLD FOR BSNMHC METHOD 1
                           BSNMHC Method 3
       0)
       o
       (0
       •+•»

       0)
       e
       0)
       0.

       &

       0)
       0)
           80
60
40
           20
               0      0.2     0.4      0.6      0.8       1

                Convergence Interval Width/Threshold, %



     FIGURE 210. CONVERGENCE INTERVAL WIDTH AS A PERCENT OF

                  THRESHOLD FOR BSNMHC METHOD 3
SwRI Report 03.12024.06
                  Page 274 of 371

-------
  TABLE 91.  SUMMARY OF BSNMHC CONVERGENCE INTERVAL WIDTH AS A
         FUNCTION OF THRESHOLD FOR 195 REFERENCE NTE EVENTS
Method
1
2
3
#RefNTE
Events
195
195
195
Mean %
0.222
0.188
0.197
Min %
0.113
0.090
0.005
Max %
0.604
0.429
0.480
      Figure 211 through Figure 213 contain relative frequency (in percent) histograms for the
widths of the 90% confidence intervals for the 95th percentiles of the corresponding BSCO
differences for the 195 individual reference NTE events where the confidence interval widths are
expressed as a percent of the BSCO emissions NTE threshold. This is done for each of the three
calculation methods.  A summary of the results is given in Table 92. Of interest was whether or
not the simulations converged within 1% of the threshold value.  For the three calculation
methods,  the maximum  percent of the confidence interval widths that were within 1% of the
threshold value across the 195 NTE events ranged from 0.076% for Method 2 to 0.202% for
Method 3. Thus, all 195  events met the convergence criteria.
                                   BSCO Method 1
        O)

        1
        0)
        O
        03
        0.
        >
        O
        c
        0)
            100
80
60
40
             20
                          0.1       0.2       0.3       0.4       0.5
                      Convergence Interval Width/Threshold, %
       FIGURE 211. CONVERGENCE INTERVAL WIDTH AS A PERCENT OF
                       THRESHOLD FOR BSCO METHOD 1
SwRI Report 03.12024.06
                     Page 275 of 371

-------
                             BSCO Method 2
IUU
0)
O)
+2 OU
c
*~ fin
Q) DU
0.
o AO
c
a)
°" on
0) ZU
ul
n
~
-
_
-
-

























;
-
-
-
-
              0       0.1      0.2      0.3      0.4      0.5
                  Convergence Interval Width/Threshold, %
     FIGURE 212. CONVERGENCE INTERVAL WIDTH AS A PERCENT OF
                   THRESHOLD FOR BSCO METHOD 2
                            BSCO Method 3
a, IUU
O)
-2 on
£ OU
0)
*~ en
(1) OU
0.
u* An
o 4U
0)
= 20
7 ^U
0)
^ 0
.
-
-
.


























-
_
-
-
-
              0       0.1      0.2     0.3      0.4      0.5
                Convergence Interval Width/Threshold, %


     FIGURE 213. CONVERGENCE INTERVAL WIDTH AS A PERCENT OF
                   THRESHOLD FOR BSCO METHOD 3
SwRI Report 03.12024.06
Page 276 of 371

-------
    TABLE 92. SUMMARY OF BSCO CONVERGENCE INTERVAL WIDTH AS A
         FUNCTION OF THRESHOLD FOR 195 REFERENCE NTE EVENTS
Method
1
2
3
#RefNTE
Events
195
195
195
Mean %
0.034
0.025
0.032
Min %
0.013
0.009
0.013
Max %
0.102
0.076
0.202
6.4    Delta BS Emissions Plots for 95th Percentiles

       This section contains  plots of the 95th percentile delta  emissions values obtained by
simulation for each reference NTE event distribution of BS differences for each emissions for all
three  calculation methods.  Section 2.1.8  on Simulation Output contains more details on the
simulation output and the methodology used to compute the delta values.

       Figure 214 through Figure 216 display box plots of the 95th percentile delta emissions for
all three methods. The ends of the boxes mark the location of the 25th and 75th percentiles of the
delta emissions while the horizontal line inside the box indicates the location  of the median, or
50th percentile, of the data.  The plus sign inside the box denotes the location of the mean of the
data.  The  two vertical  lines (i.e., whiskers) extending above and below the box denote the
distance to the farthest observation that does not exceed the endpoint +/- 1.5*(height of the box).
Observations outside these vertical lines are individually noted on the graph as square symbols.

       Viewing Figure 214, it can be observed that the 95th percentile delta BSNOx values based
on Method 1 had larger values  and more spread in the data than did the corresponding values
computed using Methods 2 and 3.  Further, the delta values based on Method 2 had the smallest
variation and the lowest values of all three methods.
SwRI Report 03.12024.06
Page 277 of 371

-------
               1.6
               1.2
          I
           3>
          5   °-8
           o   0.4
          Q
                                            2
                                         Method
    FIGURE 214. BOX PLOT OF 95TH PERCENTILE DELTA BSNOX FOR THREE
                 METHODS FROM 195 REFERENCE NTE EVENTS

      Viewing Figure 215, the 95th percentile delta BSNMHC values are not as different for the
three methods as the delta BSNOx values.   The delta values based on Method 1  are slightly
higher than similar values based on the other two methods, and the data is more skewed to the
right than the data for the other two methods.  Again, the delta values based on Method 2 had the
lowest values of all three methods.
SwRI Report 03.12024.06
Page 278 of 371

-------
.c
£
0)
o"
X
"53
Q

0.12
0.1
0.08
0.06
0.04
0.02
0
I •*- I
r * ~
~ * ~
n
I
1
_l_ 1 	 1 	 1 1 1 1
1 2 3
Method
                                   -TH
    FIGURE 215. BOX PLOT FOR 95ltt PERCENTILE DELTA BSCO FOR THREE
                 METHODS FROM 195 REFERENCE NTE EVENTS
      Viewing Figure 216, the 95th percentile delta BSCO values are similar in spread to the
data for the delta BSNMHC values.  The delta values based on Method 1 are again slightly
higher than similar values based on the other two methods, and the data is more skewed to the
right than the data for the other two methods.  The delta values based on Method 2 had the
lowest values of all three methods, and there is some skewness to the right for the data based on
all three methods.
SwRI Report 03.12024.06
Page 279 of 371

-------

!_
^
O)
O"
o
£
0)
O



2.4
2
1.6

1.2
00
.0
0.4
0
— —
*•
*
*
*

_ *
D
T f

_l_ I 	 * 	 1 I | 1
_
1 2 3
Method
    FIGURE 216.  BOX PLOT FOR 95TH PERCENTILE DELTA BSCO FOR THREE
                  METHODS FOR 195 REFERENCE NTE EVENTS
                                         -th
       An alternative way to compare the 95  percentile deltas across the three calculation
methods is to plot the deltas for the same reference NTE event on a scatter plot, as illustrated in
Figure 217 through Figure 219.  As seen in Figure 217, the ideal BSNOX for each reference NTE
                          -th
event is plotted against its 95  percentile deltas for calculation methods 1, 2, and 3.  This plot
                         -th
depicts the large range in 95  percentile deltas for Method 1 as compared to Methods 2 and 3.
Similar inferences can be made for BSNMHC (Figure 218) and BSCO (Figure 219).
SwRI Report 03.12024.06
Page 280 of 371

-------
                  0.00
                    0.00
                                 NOx g/kW-hr Method 1, 2 and 3
                                With Time Alignment Adjustment
                          1.00    2.00
                                      3.00    4.00    5.00

                                        Ideal NOx g/kW-hr
                                                         6.00
                                                               7.00
                                                                     8.00
                          • Delta NOx Method 1 • Delta NOx Method 2  Delta NOx Method 3
                                    -TH
FIGURE 217.  COMPARISON OF 951" PERCENTILE DELTA BSNOX FOR METHODS
                   1, 2, AND 3 FOR 195 REFERENCE NTE EVENTS
                  0.12
O
->•
o
% Delta NMHC g/kW-
O O O
b b b
-t*. en CD
                                NMHC g/kW-hr Method 1, 2 and 3
                                With Time Alignment Adjustment
                     /y^* ,,,,%*.                         .        •
                  0021  i * \|yE? *  1        ~*~

                  0.00 -J	,	,	,	,	,	,	,	,	
                    0.00   0.05   0.10   0.15   0.20   0.25   0.30   0.35   0.40   0.45

                                       Ideal NMHC g/kW-hr
                        • Delta NMHC Method 1 • Delta NMHC Method 2  Delta NMHC Method 3
                                         -TH
     FIGURE 218. COMPARISON OF 95ltt PERCENTILE DELTA BSNMHC FOR
            METHODS 1, 2, AND 3 FROM 195 REFERENCE NTE EVENTS
SwRI Report 03.12024.06
Page 281 of 371

-------
2 50

JE
1
o
o
re
Q 1 00 ,
•£ 1
5* r
£
s r
050 -I
0 00
0.


CO g/kW-hr Method 1, 2 and 3
With Time Alignment Adjustment
•
»
•
•
*_: - - '
'&&**• • :• •
.,' • f^v^r •
J'fy^^m m
', _-,>>*."
DO 1.00 2.00 3.00 4.00 5.00 6.00 7.
Ideal CO g/kW-hr
• Delta CO Method 1 • Delta CO Method 2 Delta CO Method 3







DO


                                     -TH
 FIGURE 219. COMPARISON OF 95ltt PERCENTILE DELTA BSCO FOR METHODS
                  1, 2, AND 3 FROM 195 REFERENCE NTE EVENTS
6.5    Sensitivity Based on Variance

       This section contains a summary of the error surfaces that contributed the most to the
variance of the generated BS emissions.  During the MC simulation for each reference NTE
event,  sensitivity charts produced by Crystal Ball were generated and stored in the REPORT
files.  Crystal Ball calculates sensitivity by computing the rank correlation coefficient between
every assumption (error surface)  and forecast value (delta BS emissions) while the simulation is
running. Positive rank correlations indicate that an increase in the assumption is associated with
an increase in the forecast.  The larger the absolute value of the rank correlation the stronger the
relationship.

       Sensitivity charts in Crystal Ball provide a  means by which the variance  of the error
surfaces affects  the variance in  the forecast  values.  Hence, the sensitivity charts developed
during a MC simulation are displayed as "Contribution to Variance" charts which are calculated
by squaring the rank correlation coefficients for all assumptions used in a particular forecast and
then normalizing them to 100%.  The assumption (error surface) with the highest contribution to
variance (in absolute value of the percent) is listed first in the sensitivity chart.

       Simulation results from all 195 reference NTE events produced sensitivity values for all
three 95th percentile delta emissions by all three calculation methods.  Table 93 through Table
101 summarize the error surfaces in which the contribution to the variance sensitivity value was
at least 5% in magnitude compared to all the other error surfaces. Note that the number of error
SwRI Report 03.12024.06
Page 282 of 371

-------
surfaces whose sensitivity values were greater than 5% ranged from 3 to 6 for all three delta
emissions  and three  calculation methods.  Also  note  that while some  error surfaces were
sensitive for most of the 195 reference NTE events (e.g.,  195 events for l_NOx_SS BSNOX
Method 2 in Table 94), others were sensitive for a small fraction of the reference NTE events
(e.g., only 17 events for 2_NOx_Transient for BSNOX Method 2 in Table 94).

       Table 93 through Table 95 list the sensitivity descriptive statistics for the delta BSNOX
emissions for Methods 1, 2 and 3, respectively.  In the first column  of these tables the error
surfaces with at  least a 5% contribution to variance  are  listed followed by the number  of
reference NTE events in which this occurred.  The mean contribution-to-variance normalized
percentage is also given along with the minimum and maximum values.  For Methods 1 and 3,
the largest mean  normalized variance was from error surface #31, torque warm-up, followed
closely by error  surface #1, NOX steady state.  For Method 2, the largest mean normalized
variance was  from error surface #1, NOX steady-state,  followed by error surface #42  due  to
BSFC from the engine manufacturers' data.
 TABLE 93. ERROR SURFACE SENSITIVITY TO VARIANCE FOR 195 REFERENCE
                       NTE EVENTS FOR BSNOX METHOD 1
Error Surface
1 NOxSS
20 Exhaust Flow SS
3 1 Torque Warm-up
35 Torque Engine
Manufacturers
#RefNTE
Events
195
185
193
192
Mean,
%
27.90
10.34
-34.65
-15.96
Minimum,
%
9.40
5.30
-61.40
-25.40
Maximum,
%
99.60
15.70
-11.40
-1.10
 TABLE 94. ERROR SURFACE SENSITIVITY TO VARIANCE FOR 195 REFERENCE
                       NTE EVENTS FOR BSNOX METHOD 2
Error Surface
1 NOxSS
2 NOx Transient
37 BSFC DOE
38 BSFC Warm-up
41 BSFC Interpolation
42 BSFC Engine Manufacturers
#RefNTE
Events
195
17
93
182
56
193
Mean,
%
47.23
5.26
6.48
10.29
5.42
24.43
Minimum,
%
29.30
5.00
5.00
5.10
5.00
5.20
Maximum,
%
99.70
5.70
9.80
15.20
6.40
38.20
SwRI Report 03.12024.06
Page 283 of 371

-------
TABLE 95. ERROR SURFACE SENSITIVITY TO VARIANCE FOR 195 REFERENCE
                     NTE EVENTS FOR BSNOX METHOD 3
Error Surface
1 NOxSS
3 1 Torque Warm-up
35 Torque Engine
Manufacturers
#RefNTE
Events
195
194
193
Mean,
%
30.59
-38.37
-17.89
Minimum,
%
10.10
-65.80
-29.50
Maximum,
%
99.70
-12.00
-5.20
      Table 96 through Table 98 list the sensitivity descriptive statistics for the delta BSNMHC
emissions for Methods 1, 2 and 3, respectively.   For  all three methods, the largest mean
normalized variance was from the error surface #19, NMHC ambient effect.
TABLE 96. ERROR SURFACE SENSITIVITY TO VARIANCE FOR 195 REFERENCE
                    NTE EVENTS FOR BSNMHC METHOD 1
Error Surface
13 NMHC SS
19 NMHC Ambient
20 Exhaust Flow SS
31 Torque Warm-up
35 Torque Engine
Manufacturers
#RefNTE
Events
134
195
14
38
30
Mean,
%
6.01
85.70
7.63
-16.78
-8.48
Minimum,
%
5.00
18.20
5.10
-48.20
-16.60
Maximum,
%
9.60
93.30
10.30
-5.20
-5.00
TABLE 97. ERROR SURFACE SENSITIVITY TO VARIANCE FOR 195 REFERENCE
                    NTE EVENTS FOR BSNMHC METHOD 2
Error Surface
13 NMHC SS
19 NMHC Ambient
37 BSFCDOE
38 BSFC Warm-up
42 BSFC Engine Manufacturers
#RefNTE
Events
122
195
1
7
30
Mean,
%
5.65
89.34
5.00
7.84
9.86
Minimum,
%
5.00
40.90
5.00
5.40
5.00
Maximum,
%
8.10
93.50
5.00
10.90
27.90
SwRI Report 03.12024.06
Page 284 of 371

-------
TABLE 98. ERROR SURFACE SENSITIVITY TO VARIANCE FOR 195 REFERENCE
                    NTE EVENTS FOR BSNMHC METHOD 3
Error Surface
13 NMHC SS
19 NMHC Ambient
31 Torque Warm-up
35 Torque Engine
Manufacturers
#RefNTE
Events
114
195
38
28
Mean,
%
5.64
86.98
-16.97
-8.86
Minimum,
%
5.00
22.30
-50.90
-16.70
Maximum,
%
8.20
93.30
-5.00
-5.10
      Table 99 through Table 101 list the sensitivity descriptive statistics for the delta BSCO
emissions for Methods 1, 2  and 3,  respectively.   For  all three methods, the largest mean
normalized variance was from the error surface #7, CO steady-state.
TABLE 99. ERROR SURFACE SENSITIVITY TO VARIANCE FOR 195 REFERENCE
                     NTE EVENTS FOR BSCO METHOD 1
Error Surface
7 COSS
20 Exhaust Flow SS
31 Torque Warm-up
35 Torque Engine
Manufacturers
52 CO Time Alignment
#RefNTE
Events
195
32
122
63
56
Mean,
%
76.20
7.48
-13.29
-1.11
7.40
Minimum,
%
9.30
5.00
-44.60
-17.60
5.00
Maximum,
%
96.30
11.60
-5.00
-5.00
15.80
      TABLE 100. ERROR SURFACE SENSITIVITY TO VARIANCE FOR 195
               REFERENCE NTE EVENTS FOR BSCO METHOD 2
Error Surface
7 COSS
37 BSFCDOE
38 BSFC Warm-up
42 BSFC Engine Manufacturers
52 CO Time Alignment
#RefNTE
Events
195
2
14
65
41
Mean,
%
85.82
5.35
7.07
9.12
8.61
Minimum,
%
20.10
5.00
5.00
5.00
5.00
Maximum,
%
98.10
5.70
10.90
28.10
21.70
SwRI Report 03.12024.06
Page 285 of 371

-------
       TABLE 101. ERROR SURFACE SENSITIVITY TO VARIANCE FOR 195
                 REFERENCE NTE EVENTS FOR BSCO METHOD 3
Error Surface
7 COSS
3 1 Torque Warm-up
35 Torque Engine
Manufacturers
52 CO Time Alignment
#RefNTE
Events
195
120
62
97
Mean,
%
75.49
-13.40
-7.79
11.29
Minimum,
%
8.40
-43.20
-16.40
5.00
Maximum,
%
96.20
-5.10
-5.00
29.40
       The contribution to normalized variance sensitivities from Table 93 through Table 95 are
illustrated pictorially as box plots in Figure 220 through Figure 222 for BSNOX Methods 1, 2 and
3, respectively.  Only the error surfaces with at least 65 of the 195 reference NTE events (1/3 of
the events) are included as box plots. The mean normalized variance for each of the plotted error
surfaces is noted by a "+" symbol in the boxes.   The error surface with the largest mean
normalized variance is plotted at the left  of the chart.  The error surface with the second largest
mean normalized variance is plotted second from the left, and so on. Figure 221 and Figure 223
demonstrate the high sensitivity to the error surface #31, torque warm-up.  Figure 222 as well as
Figure  221 and Figure 223 illustrate the high sensitivity to the error surface #1, NOX steady-state.
SwRI Report 03.12024.06
Page 286 of 371

-------
                        Sensitivity Contribution to Variance for NCx Method 1
           100-
        Jf
        I
        I
(3

O
        o
       s
             o-
   -50-
          -100 H
1
T
          Vfermup_To rque_31
                      SS NOx 1
                            Eng_Manuf_To rq ue_35
SS Row 20
     FIGURE 220. BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON
                        VARIANCE FOR BSNOX METHOD 1
SwRI Report 03.12024.06
                            Page 287 of 371

-------
                        Sensitivity Contribution to Variance for NOx Method 2
          100-
        of 75
        I
           50-
        i
(3

Z
o

I
.Q
        I"
            o-t
              T
             SS NOx1
                         i                 i                  r
                  EncLManuf_BSFC_42    Vtermup_BSFC_38      DOE_BSFC_37
     FIGURE 221.  BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON
                        VARIANCE FOR BSNOX METHOD 2
SwRI Report 03.12024.06
                           Page 288 of 371

-------
                       Sensitivity Contribution to Variance for NOx Method 3
         100-
          50-
      o
         -50 -
        -100-
                                          n
                                          D
                                          a
                                          a
                                          n
                                         a
              i
T
        Vtermup_To rque_31
                         SS NOx 1
E ng_Man uf_Torq ue_35
      FIGURE 222. BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON
                         VARIANCE FOR BSNOX METHOD 3

       The contribution to normalized variance sensitivities from Table 96 through Table 98 are
illustrated pictorially as box plots in Figure 223 through Figure 225 for BSNMHC Methods 1,2
and 3, respectively.  Each of these figures  demonstrates the  high sensitivity to the error surface
#19, NMHC ambient effect.
SwRI Report 03.12024.06
                       Page 289 of 371

-------
                    Sensitivity Contribution to Variance for NMHC Method 1
      100-
       75-
    I
       50-
    o
       25-
        o-t
                     NMHC Ambient 19
             SS NMHC 13
     FIGURE 223. BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON
                      VARIANCE FOR BSNMHC METHOD 1
SwRI Report 03.12024.06
Page 290 of 371

-------
                     Sensitivity Contribution to Variance for NMHC Method 2
      100-
       75-
    I
       50-
    o
       25-
        o-t
                      NMHC Arrtiient 19
             SS NMHC 13
     FIGURE 224.  BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON
                      VARIANCE FOR BSNMHC METHOD 2
SwRI Report 03.12024.06
Page 291 of 371

-------
                      Sensitivity Contribution to Variance for NMHC Method 3
       100-
       75-
    I
       50-
     o
       25-
        o-t
                       NMHC Arrtiient 19
             SS NMHC 13
     FIGURE 225.  BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON
                       VARIANCE FOR BSNMHC METHOD 3

       The contribution to normalized variance sensitivities from Table 99 through Table 101
are illustrated pictorially as box plots in Figure 226 through Figure 228 for BSCO Methods  1,2
and 3, respectively.  Each  of these figures demonstrates the high sensitivity to error surface  #7,
CO steady-state.
SwRI Report 03.12024.06
Page 292 of 371

-------
                      Sensitivity Contribution to Variance for CO Method 1
    tf
    o

    §
       100-
        75-
        50-
        25-

       -25"
       -50 H
                            T
                         SS CO 7
           Vfermup_1b rque_31
     FIGURE 226.  BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON

                        VARIANCE FOR BSCO METHOD 1
SwRI Report 03.12024.06
Page 293 of 371

-------
                      Sensitivity Contribution to Variance for CO Method 2
        100-
      6?
      of 75
      £
      I
        50-
o
"5

t  25-
         o-t
                             n

                             n
                          SS CO 7
                                       Eng_Manuf_BSFC_42
     FIGURE 227. BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON
                        VARIANCE FOR BSCO METHOD 2
SwRI Report 03.12024.06
                             Page 294 of 371

-------
                       Sensitivity Contribution to Variance for CO Method 3
     tf
     o
     §
        100-
        75-
        501
        25-

       -25"
       -50 H
             T
              D
              D
          SS CO 7
Vfermup_Tbrqiie_31
CO_Time_Align_52
      FIGURE 228. BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON
                          VARIANCE FOR BSCO METHOD 3
6.6    Sensitivity Based on Bias and Variance

       This section contains a summary of the error surfaces that contributed the most to the bias
of the generated BS emissions. The sensitivity charts developed in Crystal Ball help identify the
error surfaces (assumptions) that are sensitive to changes in variation with respect to their effect
on the three delta BS  emissions.  Another type of sensitivity  examined  in this study was
concerned with the effects of potential "bias" in error surfaces and their effects on the forecast
values.  In order to study these effects a new  error surface assumption was added to the MS
simulation model for each of the original 35 error surfaces (excluding the two error surfaces for
time alignment).

       This assumption was sampled as a discrete binary distribution (i.e.,  on or off) during the
simulation.  For each trial of the simulation,  35 original error surfaces and 35 'on/off error
surfaces were sampled  according to  their defined  sample distribution.   If the 'on/off error
surface produced an 'off condition, the delta emissions from that particular error surface were
not added to the BS emissions computations for the BS emissions 'with errors'. Similarly, if the
SwRI Report 03.12024.06
 Page 295 of 371

-------
'on/off error surface produced an 'on' condition, the delta emissions from that particular error
surface were added to the BS emissions calculations.

       During every trial of the simulation, the exclusions due to the 'off conditions resulted in
various combinations of the error surface delta emissions being added to the BS emissions 'with
errors' computations.    Over the course of a MC simulation  with  thousands of trials,  the
sensitivity of a particular error either 'on' or 'off was assessed by examining the change in the
forecast delta emission.   Therefore, in a  single  MC  simulation of a reference  NTE event
sensitivities due to variance and/or bias were explored.

       During this phase of the simulation, thirteen reference NTE events were selected to be re-
run with the  additional  'on/off error surface  assumptions included in the MC model.  These
events were selected to  bound the NTE  BSNOX threshold of 2.6820 g/kW-hr and are listed in
Table 102 along with their ideal BSNOX values.  Also included in Table 102 is the number of
trials  run for each of the MC simulations.  Figure 229 illustrates the distribution  of the ideal
BSNOX values for the thirteen reference NTE events as a frequency histogram.
      TABLE 102. IDEAL BSNOX VALUES FOR 13 REFERENCE NTE EVENTS
Reference NTE
Event #
38
44
87
148
82
163
63
46
51
69
157
1
25
Ideal BSNOx
g/kW-hr
0.0249
1.0730
1.5207
1.9985
2.4568
2.5907
2.6670
2.6957
2.8298
3.0257
3.4666
1.0713
5.4061
Trials Run
During MC
Simulation
10,000
10,000
10,000
10,000
10,000
10,000
10,000
30,000
10,000
30,000
30,000
30,000
30,000
SwRI Report 03.12024.06
Page 296 of 371

-------
            CD

            I  2
                1

                0
                   -1012345
                               Ideal NOx  g/kW-hr
FIGURE 229. DISTRIBUTION OF IDEAL BSNOX FOR 13 REFERENCE NTE EVENTS

       Monte Carlo simulations, including the additional 35 'on/off error surfaces, were run on
all thirteen reference NTE events for 10,000 or 30,000 trials each. EXTRACT data files and
REPORT files were generated for all three emissions  and three calculation methods.   All
reference NTE events converged within 1% of the NTE emissions threshold.

       Simulation results from these reference NTE events  produced  sensitivity values for all
three 95th percentile delta emissions by all three calculation methods. Table 103 through Table
111  summarize  the  error surfaces in  which  either the contribution-to-variance  normalized
sensitivity value  or the 'on/off bias check for the error surface was at least 5% in magnitude
compared to all the other error surfaces.   If the label in the error surface contains the words
'OnOff then it represents a check for bias;  otherwise, the error surface indicates  a check for
variance.  Note that the number of error surfaces whose sensitivity values due to variance were
greater than 5% ranged from 3 to 5  for  all three  delta emissions and all three calculation
methods.  Also note that all three emissions by all three calculation methods identified at least
one 'on/off error surface in which a bias effect was noted.

       Table 103 through Table 105 list the sensitivity  due to variance and bias descriptive
statistics for the delta BSNOX emissions for Methods 1, 2 and  3, respectively. In the first column
of these tables the error surfaces with at least a 5% contribution-to-variance are listed followed
by the number of reference NTE  events in  which this occurred.   The mean  contribution-to-
variance normalized percentage is  also given along  with the minimum  and maximum values.
For Methods 1 and 3, the largest mean normalized variance was from error surface #31, torque
warm-up, which was the same result  obtained in the previous analysis in Section 6.6.   For
Method 2,  the largest mean  normalized variance was from  the 'on/off error surface for CO2
steady-state followed by error surface #1 due to NOX steady-state.
SwRI Report 03.12024.06
Page 297 of 371

-------
  TABLE 103. ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 13
               REFERENCE NTE EVENTS FOR BSNOX METHOD 1
Error Surface
1 NOxSS
20 Exhaust Flow SS
3 1 Torque Warm-up
3 5_Torque Engine
Manufacturers
5 1 NOx Time Alignment
OnOff Exhaust Flow Pulsation
OnOff Exhaust Flow Swirl
OnOff Exhaust Flow SS
#RefNTE
Events
13
11
12
12
8
12
1
10
Mean,
%
21.02
6.33
-23.21
-9.78
6.05
14.70
5.70
14.58
Minimum,
%
8.40
5.40
-34.40
-12.20
5.40
5.60
5.70
8.70
Maximum,
%
92.70
8.10
-12.80
-7.40
7.10
19.00
5.70
21.30
  TABLE 104. ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 13
              REFERENCE NTE EVENTS FOR BSNOX METHOD 2
Error Surface
1 NOxSS
37 BSFCDOE
38 BSFC Warm-up
42 BSFC Engine Manufacturers
OnOff CO2 SS
OnOff BSFC DOE
#RefNTE
Events
13
1
6
12
12
8
Mean,
%
29.46
5.00
6.33
13.55
-34.93
-6.46
Minimum,
%
14.30
5.00
5.20
10.80
-39.40
-8.30
Maximum,
%
92.30
5.00
8.30
17.10
-20.80
-5.10
  TABLE 105. ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 13
               REFERENCE NTE EVENTS FOR BSNOX METHOD 3
Error Surface
1 NOxSS
31 Torque Warm-up
35 Torque Engine
Manufacturers
5 1 NOx Time Alignment
OnOff CO2 SS
#RefNTE
Events
13
12
12
7
12
Mean,
%
23.62
-27.83
-11.69
5.96
-25.22
Minimum,
%
11.20
-38.20
-15.00
5.00
-32.90
Maximum,
%
92.90
-15.90
-8.60
7.80
-15.90
      Table 106 through Table 108 list the sensitivity and bias descriptive statistics for the delta
BSNMHC emissions for Methods  1, 2 and 3, respectively. For all three methods, the largest
mean normalized variance was from the error surface #19, NMHC ambient effect.
SwRI Report 03.12024.06
Page 298 of 371

-------
  TABLE 106.  ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 13
              REFERENCE NTE EVENTS FOR BSNMHC METHOD 1
Error Surface
13 NMHC SS
19 NMHC Ambient
20 Exhaust Flow SS
31 Torque Warm-up
35 Torque Engine
Manufacturers
OnOff Exhaust Flow Pulsation
OnOff Exhaust Flow SS
OnOff NMHC SS
#RefNTE
Events
10
13
1
4
3
3
3
11
Mean,
%
7.05
58.33
6.40
-15.60
-8.37
14.80
11.00
13.89
Minimum,
%
6.10
11.90
6.40
-25.60
-9.80
9.20
8.20
5.00
Maximum,
%
8.70
73.80
6.40
-5.70
-5.90
19.30
12.40
18.80
  TABLE 107.  ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 13
              REFERENCE NTE EVENTS FOR BSNMHC METHOD 2
Error Surface
13 NMHC SS
19 NMHC Ambient
38 BSFC Warm-up
42 BSFC Engine Manufacturers
OnOff CO2 SS
OnOff NMHC SS
#RefNTE
Events
10
13
2
3
3
11
Mean,
%
6.78
62.33
5.60
10.67
-24.67
13.95
Minimum,
%
6.00
21.60
5.50
7.40
-36.90
6.80
Maximum,
%
8.2
74.20
5.70
14.20
-12.60
19.30
  TABLE 108.  ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 13
              REFERENCE NTE EVENTS FOR BSNMHC METHOD 3
Error Surface
13 NMHC SS
19 NMHC Ambient
3 1 Torque Warm-up
35 Torque Engine Manufacturers
OnOff CO2 SS
OnOff NMHC SS
#RefNTE
Events
10
13
4
3
3
11
Mean,
%
6.84
60.51
-18.68
-9.87
-18.63
13.64
Minimum,
%
6.00
14.90
-32.20
-11.80
-26.20
5.50
Maximum,
%
7.90
74.10
-5.40
-6.40
-11.00
17.50
      Table 109 through Table 111 list the sensitivity and bias descriptive statistics for the delta
BSCO emissions for Methods 1, 2 and 3, respectively. For all three methods, the largest mean
normalized variance was from the 'on/off error surface for CO steady-state (average near 80%
for all three methods).
SwRI Report 03.12024.06
Page 299 of 371

-------
  TABLE 109.  ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 13
                REFERENCE NTE EVENTS FOR BSCO METHOD 1
Error Surface
7 COSS
3 1 Torque Warm-up
3 5_Torque Engine
Manufacturers
52 CO Time Alignment
OnOff Exhaust Flow Pulsation
OnOff Exhaust Flow SS
OnOffSSCO
#RefNTE
Events
10
1
1
1
1
1
13
Mean,
%
6.06
-13.50
-5.20
20.00
10.80
6.20
80.63
Minimum,
%
5.00
-13.50
-5.20
20.00
10.80
6.20
31.30
Maximum,
%
7.30
-13.50
-5.20
20.00
10.80
6.20
91.30
  TABLE 110.  ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 13
                REFERENCE NTE EVENTS FOR BSCO METHOD 2
Error Surface
7 COSS
42 BSFC Engine Manufacturers
52 CO Time Alignment
OnOff CO2 SS
OnOff CO SS
#RefNTE
Events
12
1
1
1
13
Mean,
%
6.31
5.80
20.00
-16.20
82.82
Minimum,
%
5.30
5.80
20.00
-16.20
43.80
Maximum,
%
7.40
5.80
20.00
-16.20
91.20
  TABLE 111.  ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 13
                REFERENCE NTE EVENTS FOR BSCO METHOD 3
Error Surface
7 COSS
31 Torque Warm-up
52 CO Time Alignment
OnOff CO2SS
OnOff CO SS
#RefNTE
Events
9
1
6
1
13
Mean,
%
6.14
-13.10
12.10
-10.40
79.75
Minimum,
%
5.50
-13.10
5.50
-10.40
28.00
Maximum,
%
7.30
-13.10
36.00
-10.40
91.20
      The contribution to normalized variance and bias sensitivities from Table 103 through
Table 105 are illustrated pictorially as box plots in Figure 230 through Figure 232 for BSNOX
Methods 1, 2 and 3, respectively.  Only the error surfaces with at least 5 of the thirteen reference
NTE events (1/3  of the events) are included as box plots. The mean normalized variance for
each of the plotted error surfaces is noted by a "+" symbol in the boxes. The  error surface with
the largest mean normalized variance is plotted at the left of the chart.  The error surface with the
second largest mean normalized variance is plotted second from the left, and so on.  Similar plots
could also be generated for BSNMHC and BSCO. Figure 230 and Figure 232 demonstrate the
high sensitivity to the error surface #31, torque warm-up. This was also seen  in the analyses
using the 195 reference NTE events.  Also note in Figure 230 that bias effects due to exhaust
SwRI Report 03.12024.06
Page 300 of 371

-------
flow pulsation and steady-state were important.   Figure 231 and Figure 232 also show a bias
effect due to CC>2 steady-state.

                  Sensitivity Contribution to Variance and Bias for  NOx Method 1
     75
  fl?  50
  1
     25 -
    -25
    -50 ~\
     WaimupToiqie 31    SSHc« 1    PutseFlcw OnOff  SSHow OnOff  EngManufTorcjje35   SSHow 20    NoxTlmeAJi^i 51


   FIGURE 230.  BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON BIAS
                       AND VARIANCE FOR BSNOX METHOD 1
SwRI Report 03.12024.06
Page 301 of 371

-------
                 Sensitivity Contribution to Variance and Bias for NOx Method 2
    100 -
     75 -
     50 -
  #
  o"

  G
  I  25
  o
  o
     0 -'
    -25 -
    -50 H
D

n
      SSCO2 OnOff
                        SSNox 1
                                     EngMaiuBSFC 42
                                                      DOEBSFC OnOft
                                                             WarmupBSFC 38
   FIGURE 231.  BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON BIAS
                      AND VARIANCE FOR BSNOX METHOD 2
SwRI Report 03.12024.06
                          Page 302 of 371

-------
                  Sensitivity Contribution to Variance and Bias for NOx Method 3
     100 -
  #
  o"
  g  50 -
  M
  I
5
z
o
*-
c
o
i
•c
o
O
     25 -
      0 -
    -25 -
    -50 H
         I
     WarmupTorque_31
                       SSCO2 OnOft
                                         SSNox 1
                                                    E ngManufTorque 35
NOxTimeAlign_51
   FIGURE 232. BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON BIAS
                       AND VARIANCE FOR BSNOX METHOD 3


       A summary of the sensitivity results due to variance only and both variance and bias MC
simulations are provided in Table 112 through Table 120 for all three emissions and all three
calculation methods.
SwRI Report 03.12024.06
                                   Page 303 of 371

-------
    TABLE 112. SUMMARY OF ERROR SURFACE SENSITIVE TO BIAS AND
                    VARIANCE FOR BSNOX METHOD 1
Sensitivity to Variance
1 95 RefNTE Events
#
1
20
31
35
Error Surface
SS NOx
SS Flow
Warmup Torque
Eng Manuf Torque
Avg
Contribution
to
Normalized
Variance, %
27.90
10.34
-34.65
-15.96


Sensitivity to Bias and Variance
13 RefNTE Events
#
1
20
31
35
51

Error Surface
SS NOx
SS Flow
Warmup Torque
Eng Manuf Torque
NOx Time Align
Pulse Flow OnOff
SS Flow OnOff
Avg
Contribution
to
Normalized
Variance, %
21.02
6.33
-23.21
-9.78
6.05
14.70
14.58
    TABLE 113. SUMMARY OF ERROR SURFACE SENSITIVE TO BIAS AND
                    VARIANCE FOR BSNOX METHOD 2
Sensitivity to Variance
195 RefNTE Events
#
1
37
38
42
Error Surface
SS NOx
DOE BSFC
Warmup BSFC
Eng Manuf BSFC
Avg
Contribution
to
Normalized
Variance, %
47.22
6.48
10.29
24.43


Sensitivity to Bias and Variance
13 RefNTE Events
#
1

38
42
51

Error Surface
SS NOx

Warmup BSFC
Eng Manuf BSFC
NOx Time Align
SS CO2 OnOff
DOE BSFC OnOff
Avg
Contribution
to
Normalized
Variance, %
29.46

6.33
13.55
6.05
-34.93
-6.46
SwRI Report 03.12024.06
Page 304 of 371

-------
    TABLE 114. SUMMARY OF ERROR SURFACE SENSITIVE TO BIAS AND
                   VARIANCE FOR BSNOX METHOD 3
Sensitivity to Variance
1 95 RefNTE Events
#
1
31
35
Error Surface
SS NOx
Warmup Torque
Eng Manuf Torque
Avg
Contribution
to
Normalized
Variance, %
30.59
-38.37
-17.89


Sensitivity to Bias and Variance
13 RefNTE Events
#
1
31
35
51

Error Surface
SS NOx
Warmup Torque
Eng Manuf Torque
NOx Time Align
SS CO2 OnOff
Avg
Contribution
to
Normalized
Variance, %
23.62
-27.83
-11.69
5.96
-25.22
    TABLE 115. SUMMARY OF ERROR SURFACE SENSITIVE TO BIAS AND
                  VARIANCE FOR BSNMHC METHOD 1
Sensitivity to Variance
195 RefNTE Events
#
13
19
Error Surface
SS NMHC
NMHC Ambient
Avg
Contribution
to
Normalized
Variance, %
6.01
85.70


Sensitivity to Bias and Variance
13 RefNTE Events
#
13
19

Error Surface
SS NMHC
NMHC Ambient
SS NMHC OnOff
Avg
Contribution
to Normalized
Variance, %
7.05
58.33
13.86
    TABLE 116. SUMMARY OF ERROR SURFACE SENSITIVE TO BIAS AND
                  VARIANCE FOR BSNMHC METHOD 2
Sensitivity to Variance
195 RefNTE Events
#
13
19
Error Surface
SS NMHC
NMHC Ambient
Avg
Contribution
to
Normalized
Variance, %
5.65
89.34


Sensitivity to Bias and Variance
13 RefNTE Events
#
13
19

Error Surface
SS NMHC
NMHC Ambient
SS NMHC OnOff
Avg
Contribution
to Normalized
Variance, %
6.78
62.33
13.95
SwRI Report 03.12024.06
Page 305 of 371

-------
    TABLE 117. SUMMARY OF ERROR SURFACE SENSITIVE TO BIAS AND
                  VARIANCE FOR BSNMHC METHOD 3
Sensitivity to Variance
1 95 RefNTE Events
#
13
19
Error Surface
SS NMHC
NMHC Ambient
Avg
Contribution
to
Normalized
Variance, %
5.64
86.98


Sensitivity to Bias and Variance
13 RefNTE Events
#
13
19

Error Surface
SS NMHC
NMHC Ambient
SS NMHC OnOff
Avg
Contribution
to Normalized
Variance, %
6.84
60.51
13.64
    TABLE 118. SUMMARY OF ERROR SURFACE SENSITIVE TO BIAS AND
                    VARIANCE FOR BSCO METHOD 1
Sensitivity to Variance
195 RefNTE Events
#
7
31

Error Surface
SS CO
Warmup Torque

Avg
Contribution
to
Normalized
Variance, %
76.20
-13.29


Sensitivity to Bias and Variance
13 RefNTE Events
#
7


Error Surface
SS CO

SS CO OnOff
Avg
Contribution to
Normalized
Variance, %
6.06

80.63
    TABLE 119. SUMMARY OF ERROR SURFACE SENSITIVE TO BIAS AND
                    VARIANCE FOR BSCO METHOD 2
Sensitivity to Variance
195 RefNTE Events
#
7
42

Error Surface
SS CO
Eng Manuf BSFC

Avg
Contribution
to
Normalized
Variance, %
85.82
9.12


Sensitivity to Bias and Variance
13 RefNTE Events
#
7


Error Surface
SS CO

SS CO OnOff
Avg
Contribution to
Normalized
Variance, %
6.31

82.82
SwRI Report 03.12024.06
Page 306 of 371

-------
      TABLE 120.  SUMMARY OF ERROR SURFACE SENSITIVE TO BIAS AND
                         VARIANCE FOR BSCO METHOD 3
Sensitivity to Variance
1 95 RefNTE Events
#
7
31
52
Error Surface
SS CO
Warmup Torque
CO Time Align
Avg
Contribution
to
Normalized
Variance, %
75.49
-13.40
11.29


Sensitivity to Bias and Variance
13 RefNTE Events
#
7

52

Error Surface
SS CO

CO Time Align
SS CO OnOff
Avg
Contribution to
Normalized
Variance, %
6.14

12.10
79.74
6.7    CE-CERT Mobil Emission Laboratory Correlation

     As mentioned previously, the primary means of validation for the Monte Carlo Model was
through the use of a comparison set of measurement deltas generated through in-field testing.
This was  accomplished using the CE-CERT Mobile Emission Laboratory (MEL),  which is
operating  by staff from the  University of California-Riverside. In order to insure that the
validation was not disturbed by some inherent bias between the SwRI Reference Laboratory and
the CE-CERT MEL Validation Reference, the Test Plan included a correlation exercise that was
to be performed between the two laboratories, prior to the start of on-road validation efforts. The
CE-CERT MEL was brought to SwRI's laboratory facilities in San Antonio, Texas, and a side-
by-side correlation test was run.  This correlation testing was performed during June of 2006.

     At this point in the program, SwRI had just recently completed dynamometer testing of
Engine  1, which  was the DDC Series 60  heavy-heavy duty engine.   This  engine  was still
installed in the test cell at that time, and therefore it was the  engine that was used for the
correlation exercise.

     The trailer  housing the  CE-CERT MEL was positioned  directly behind the  SwRI
laboratory facilities, in such a manner that the test cell could be readily accessed via a high-bay
access  door.  This position allowed for the easy connection of the MEL dilution tunnel exhaust
inlet to the test cell exhaust system. The exhaust system was constructed in such a manner that it
could be easily disconnected at a downstream of the DPFs, and an exhaust pipe extension was
fabricated and positioned in order to allow for relatively quick connect to the CE-CERT exhaust
inlet.

     This arrangement allowed for easy switching of the exhaust between the two test facilities,
allowing for tests to be conducted  on both  facilities during  a  single  day  with minimum
interruption.  The Test Plan did  not initially include a  test matrix  for the correlation,  and
therefore a  correlation test  matrix was developed  by SwRI and  proposed to the Steering
Committee for approval.  Following approval, the correlation testing was conducted essentially
as proposed over the course of three days of testing.
SwRI Report 03.12024.06
Page 307 of 371

-------
     The primary test cycle used during the correlation test was termed the "NTE Cycle." This
cycle was in fact one of the 20 NTE transient cycles which had been previously used to generate
data for the Transient Error Surface, which was described earlier in Section 4.5 of the report.  It
was felt that a cycle which included a number of NTE events would be appropriate for basis for
correlation, because the CE-CERT MEL would later be used to generate validation data during
NTE operation.  However, the basis for comparison  between the  labs was the overall cycle
average brake-specific emissions.  The correlation was not assessed  on an event-by-event basis,
although this data was available for comparison.  This NTE cycle was run in triplicate each day
by both labs in succession.

     In addition to the NTE cycle, the test matrix also included duplicate runs of the RMC 13-
mode SET test by each lab every day.   This cycle was included in the test matrix in order  to
provide some steady-state measurements that would hopefully aid in determining the cause  of
any discrepancies between the two laboratories, if any such differences were found  during NTE
cycle testing.

     The test matrix is illustrated in Table 121. In total, nine NTE cycles were run for each
laboratory and six RMC cycles were run for each laboratory.  On each day, one laboratory would
run in the morning, and then the exhaust would be switched for the  afternoon runs on the other
laboratory.

                    TABLE 121. CORRELATION TEST MATRIX
Test Day
1
2
O
Test Laboratory
SwRI
CE-CERT
CE-CERT
SwRI
SwRI
CE-CERT
Test Cycles
3 x NTE Cycle
2 x RMC Cycle
3 x NTE Cycle
2 x RMC Cycle
3 x NTE Cycle
2 x RMC Cycle
3 x NTE Cycle
2 x RMC Cycle
3 x NTE Cycle
2 x RMC Cycle
3 x NTE Cycle
2 x RMC Cycle
     Prior to the start of the correlation  exercise, any  periodic QA checks which were due
according to the schedule outlined in 40 CFR Part 1065 Subpart D were conducted.  In addition,
CVS propane recovery checks were performed by both laboratories prior to the start of testing.
The start of testing  was delayed several days be an electronic  hardware failure with the MEL
CVS, but this was repaired once  a replacement part was procured, and further test operations
proceeded without any major incident.   The MEL CVS  propane recovery check was repeated
following this repair, to insure that flow measurements were still correct.
SwRI Report 03.12024.06
Page 308 of 371

-------
     As an  additional  QA measure,  span bottles from  each facility  were read using the
instruments from the other facility.  In all cases, span concentrations were within 1 percent of
expected values.

     The brake-specific emission results from the correlation testing for all three days are
summarized in Table 122 for the NTE Transient Cycle and in Table 123  for the RMC cycle.  The
results of greatest interest to  the  Steering Committee were  the  NOX results.  On the NTE
Transient Cycle, the test labs showed a difference of 2.1 percent, with CE-CERT being higher
than SwRI.   This difference  was  matched  closely by  a  2.7  percent difference in CC>2
measurements in the same  direction.   The close match  between these numbers may have
indicated that the primary discrepancy between the two labs was related to measurement of total
CVS flow.
SwRI Report 03.12024.06                 Page 309 of 371

-------
 TABLE 122. CORRELATION TESTING RESULTS FOR NTE TRANSIENT CYCLE
Test Test
Day Date
1 6/29/2006
6/29/2006
6/29/2006

6/29/2006
6/29/2006
6/29/2006


2 6/30/2006
6/30/2006
6/30/2006

6/30/2006
6/30/2006
6/30/2006


3 7/5/2006
7/5/2006
7/5/2006

7/5/2006
7/5/2006
7/5/2006


Test
Number
SwRI-NTE-1
SwRI-NTE-2
SwRI-NTE-3

CE-CERT-NTE-1
CE-CERT-NTE-2
CE-CERT-NTE-3

Day 1 Difference
SwRI-NTE-1
SwRI-NTE-2
SwRI-NTE-3

CE-CERT-NTE-1
CE-CERT-NTE-2
CE-CERT-NTE-3

Day 2 Difference
SwRI-NTE-1
SwRI-NTE-2
SwRI-NTE-3

CE-CERT-NTE-1
CE-CERT-NTE-2
CE-CERT-NTE-3

Day 3 Difference
Standard for 2005 Series 60 Engine
Transient Emissions, g/hp-hr
THC
0.003
0.003
0.004
0.003
0.001
0.001
0.001
0.001
-287.6%
0.004
0.003
0.003
0.003
0.002
0.001
0.001
0.001
-148.3%
0.005
0.003
0.003
0.004
0.001
0.001
0.002
0.001
-159.4%
0.14
CH4
-0.005
0.001
0.003
0.000
0.001
0.002
0.002
0.002
119.7%
0.001
0.002
0.002
0.002
0.001
0.002
0.002
0.002
8.2%
-0.007
0.002
0.002
-0.001
0.002
0.002
0.002
0.002
152.1%
0.14
Overall Results - NTE
SwRI


CE-CERT



Mean
Stdev
CvaMSD
Mean
Stdev
CvaMSD
%point
%standard
0.004
0.001

0.001
0.0003

-65.1%
-1.6%
0.0001
0.0036

0.002
0.0003

2336.2%
1.3%
| NMHC |
0.008
0.002
0.001
0.004
-0.001
-0.001
-0.001
-0.001
546.0%
0.003
0.001
0.001
0.002
0.000
-0.001
-0.001
0.000
556.3%
0.012
0.001
0.001
0.005
-0.001
-0.001
0.000
-0.001
960.2%
0.14
Cycle
0.004
0.004
108.9%
-0.001
0.0004
59.0%
1-117.4%
-2.9%
CO
0.057
0.057
0.057
0.057
0.044
0.044
0.042
0.043
-31.7%
0.058
0.054
0.057
0.056
0.041
0.040
0.041
0.041
-38.2%
0.055
0.052
0.053
0.053
0.042
0.043
0.042
0.042
-26.2%
15.5

0.056
0.002
3.8%
0.042
0.001
3.3%
-24.2%
-0.1%
NOx
1.99
1.97
1.99
1.98
2.03
2.03
2.04
2.03
2.4%
2.04
2.01
2.02
2.02
2.04
2.05
2.04
2.04
1.0%
2.01
1.99
2.00
2.00
2.06
2.05
2.07
2.06
2.9%
2.2

2.001
0.020
1.0%
2.044
0.014
0.7%
2.1%
1.9%
CO2
540.4
540.9
542.0
541.1
557.6
558.0
557.8
557.8
3.0%
541.5
543.0
542.4
542.3
554.2
551.7
551.1
552.3
1.8%
539.5
540.4
541.2
540.4
558.5
558.0
554.8
557.1
3.0%


541.3
1.1
0.2%
555.7
2.9
0.5%
12.7%
n/a
SwRI Report 03.12024.06
Page 310 of 371

-------
   TABLE 123.  CORRELATION TEST RESULTS FOR RMC 13-MODE SET CYCLE
Test Test
Day Date
1 6/29/2006
6/29/2006
6/29/2006
6/29/2006

2 6/30/2006
6/30/2006
6/30/2006
6/30/2006

3 7/5/2006
7/5/2006
7/5/2006
7/5/2006

Test
Number
SwRI-RMC-1
SwRI-RMC-2
CE-CERT-RMC-1
CE-CERT-RMC-2
Day 1 Difference
SwRI-RMC-1
SwRI-RMC-2
CE-CERT-RMC-1
CE-CERT-RMC-2
Day 2 Difference
SwRI-RMC-1
SwRI-RMC-2
CE-CERT-RMC-1
CE-CERT-RMC-2
Day 3 Difference
Standard for 2005 Series 60 Engine
Transient Emissions, g/hp-hr
THC |
0.004
0.003
0.004
0.000
0.000
0.000
-109%
0.002
0.002
0.002
0.001
0.000
0.001
-72%
0.002
0.002
0.002
0.000
0.000
0.000
-84%
0.14
Overall Results
SwRI
CE-CERT
Mean
Stdev
CvaMSD
Mean
Stdev
CvaMSD
%point
%standard
0.003
0.001
0.0002
0.0005
-92.6%
-1.8%
CH4
0.000
0.002
0.001
0.001
0.001
0.001
23%
0.000
0.000
0.000
0.002
0.002
0.002
1586%
0.002
0.002
0.002
0.002
0.001
0.001
-35%
0.14
for RMC
0.001
0.001
0.002
0.001
42.9%
0.3%
NMHC |
0.004
0.001
0.003
-0.002
-0.001
-0.002
-160%
0.002
0.002
0.002
-0.001
-0.001
-0.001
-161%
0.000
0.001
0.000
-0.002
0.000
-0.001
-314%
0.14
Cycle
0.002
0.001
80.0%
-0.001
0.001
48.5%
1-171.9%
-2.1%
CO
0.054
0.057
0.055
0.048
0.052
0.050
-9.5%
0.054
0.053
0.053
0.043
0.041
0.042
-21.4%
0.052
0.052
0.052
0.041
0.045
0.043
-17.4%
15.5

0.053
0.002
3.4%
0.045
0.004
9.7%
-16.0%
-0.1%
NOx
1.79
1.80
1.80
1.88
1.88
1.88
4.6%
1.83
1.84
1.84
1.90
1.91
1.90
3.6%
1.84
1.85
1.85
1.92
1.89
1.91
3.2%
2.2

1.827
0.024
1.3%
1.897
0.015
0.8%
3.8%
3.1%
CO2
499.8
499.8
499.8
511.7
510.5
511.1
2.3%
500.6
501.1
500.8
508.1
509.0
508.5
1.5%
498.9
499.0
499.0
514.2
514.6
514.4
3.1%


499.9
0.9
0.2%
511.3
2.7
0.5%
2.3%
     The RMC  cycle showed a larger NOX difference of 3.8 percent, with CE-CERT again
being higher than SwRI.  However, the CC>2 difference for this cycle was similar to that observed
during the NTE cycle at 2.3 percent. In this case it is unlikely that all of the NOX differences are
related to CVS flow discrepancies alone.

     Continuous data for the RMC was examined in order to attempt to determine the reasons
for the larger discrepancies observed on that cycle.  An example of continuous NOX mass rate
comparison is given in Figure 233. It was noted that NOX mass rates were closer together at
higher levels,  associated with high load modes of operation during  the RMC.   The facilities
diverged more at lower load modes, which had correspondingly lower NOX levels.  These
SwRI Report 03.12024.06
Page 311 of 371

-------
differences cannot be related entirely to flow rate measurements, because CC>2 correlated more
closely between the two laboratories than NOX.  Examination of linearity data for both NOX
analyzers did not reveal any obvious reasons why the two systems should have diverged more at
lower concentration levels.
                                ^—SwRI NOx Mass, g/seo         ^—Ce-Cert NOx Mass, g/seo
      ^— Absolute Differences, g/seo        Percent Difference (Ce-Cert - SwRI)
                                                                                 20%
                    500
1000           1500
     Time, sec
2000
2500
 FIGURE 233. COMPARISON OF TYPICAL CONTINUOUS NOX MASS RATE DATA
                   OVER RMC CYCLE - CE-CERT VERSUS SWRI

     One area of difference noted between the two laboratories was in terms of CVS flow rate.
The SwRI test cell ran at an average flow rate of 4350 scfrn, while the CE-CERT tunnel ran at an
average flow rate of 2775 scfm.  These flow rates translate to an average dilution ratio during the
RMC of 7.3 versus 4.6 for SwRI and CE-CERT respectively. It is not known if this difference is
related to any of the measurement differences observed between the two laboratories.

       The Steering Committee examined these results during the July 7, 2006 conference call.
The primary area of interest was the NOX correlation between the two facilities. The NTE cycle
correlation of 2 percent was generally considered good  by the  Steering Committee; however,
there was some concern expressed over the larger discrepancy that was observed during  the
RMC cycle.
SwRI Report 03.12024.06
  Page 312 of 371

-------
     The Test Plan did not include any guidance on assessing correlation.  The initial proposal
was to use an analysis similar to  that given in 40 CFR Part  1065.12 for alternative method
approvals.   If this  proved unsuccessful, the Steering Committee would  assess the absolute
differences observed to determine if an acceptable level of correlation was achieved.

     An initial data analysis was performed using the methodology laid out in that part of the
CFR.  As outlined in that procedure, both a F-test and a t-test were performed to compare the two
sets of NOX data , using appropriate Fcrit and tcrit values given in the tables in Part 1065, and a 90
% confidence  interval.   This was done using the NTE cycle data, as that cycle was the primary
cycle  of comparison.   The  result  of this analysis indicated a failure of the t-test for NOX  ,
meaning that the 2 percent difference in NOX observed was statistically significant.  However, it
was noted that this was due in part to the high  degree of test-to-test repeatability observed for
both laboratories over the three days of testing.

     The Steering  Committee examined all  of these results along with the F-test and t-test
results.  Ultimately, the Steering Committee determined at the July  7, 2006 conference call that
the level of correlation achieved between the two facilities was acceptable for the performance of
the validation  testing.   This  determination was based primarily on  the fact that the NTE cycle
was deemed more likely to be the kind of operation observed  by CE-CERT during on-road
testing,  and the correlation results  for that cycle at 2 percent difference between the labs  was
considered sufficient to proceed with testing.

     It should be noted that  this correlation exercise did represent a  unique  opportunity to
compare two 1065 compliant measurement systems using the same test engine and dynamometer
installation. In this comparison, any correlation issues related to test cell  installation or cycle
operation were eliminated by the fact that the same test article was used for both comparisons.

6.8    CE-CERT On Road Validation Testing

     The generation of the On-Road Validation  data set was performed  by CE-CERT using
their MEL.  This on-road effort was  conducted  under a separate contract which was funded by
the California Air Resources Board  (ARE), as part  of their contribution to the Measurement
Allowance program.  The test truck was supplied by Caterpillar,  and it incorporated a C-15
heavy heavy duty diesel engine in  it.   Initially, the truck was configured only with  diesel
oxidation  catalysts (DOCs),  rather than DPFs.  A pair  of  DPFs was supplied for use during
validation testing by International.  These DPFs were shipped to  CE-CERT, and were installed
on the truck be a local Caterpillar dealer in Riverside, California.

     The MEL was used as the reference laboratory measurement during the on-road testing.
The PEMS unit used during the on-road testing was PEMS 5, which had previously been audited
by SwRI, and was  used during selected Environmental  Chamber  testing  described earlier in
Section 5 of this report.  In addition, one of the 5-inch EFM2 exhaust flow meters was also used
for the on-road testing.  This flow meter had been previously  audited by  SwRI, and was  also
used during Environmental Chamber testing.  In addition, it had also been used for Engine 1
dynamometer  tests at  SwRI.  The 5-inch flow  meter was selected due  to the size of the test
engine in the truck.
SwRI Report 03.12024.06                 Page 313 of 371

-------
     The actual performance and results of the CE-CERT on-road validation testing are not
included in this report. Full details on this part of the program are given in a separate report,
titled "Measurement Allowance On-Road Validation Project Report," dated March 2007.  The
contents of that report are incorporated herein by reference.

     SwRI  had only a very limited role in the performance  of the  on-road  testing  itself,
generally limited to arranging for the delivery of the test truck and PEMS hardware both to and
from CE-CERT. Caterpillar representatives were onsite with CE-CERT  staff during the replay
testing, but were there only in order to assist in the recording of proprietary engine ECM data
channels, which would later be used to aid in the Laboratory Replay Validation described later in
this report.  All validation testing was conducted solely by CE-CERT staff.

     Sensors  staff was  onsite  briefly at  CE-CERT  prior to testing,  but only  to assist in the
installation of a purge option which was designed to keep the pressure sensing lines of the EFM2
exhaust flow meter free from condensation during on-road testing. The installation of this option
was approved by the Steering Committee prior to the start of on-road testing.

6.9    Laboratory Replay Testing

       Because the CE-CERT  MEL does not readily incorporate a means  of direct torque
measurement on a  vehicle, the  on-road validation data set could not be used to validate model
errors associated with broadcast torque and derived BSFC. Therefore, an additional validation
exercise was conducted at SwRI. This involved removal of the Caterpillar CIS engine from the
test truck used by CE-CERT, and installation of the engine in the  SwRI dynamometer test cell.
The  dual DPFs used during the CE-CERT validation were also removed from the truck and
installed  in the laboratory  exhaust system. The SwRI test cell used for the laboratory replay
validation was the  same  dynamometer test cell used throughout the program. PEMS 5 with 5-
inch EFM, which was used for the CE-CERT  validation exercise, was shipped back to SwRI for
use in the replay validation testing. Selected  portions of the CE-CERT on-road tests were then
simulated in the laboratory, to the extent  possible.   Simultaneous  laboratory  and  PEMS
measurements were again taken during this replay validation exercise.   However, because the
laboratory incorporates actual torque measurement, it was possible to use this replay data set to
validate the portions of the model associated with torque and BSFC measurements.

       After the replay engine installation was complete, the CAT CIS was power validated and
a lug curve was generated.  Next, one hour test segments from the CE-CERT on-road validation
were extracted from the test data to be replayed in the SwRI test  cell.  During the CE-CERT
model validation exercise, CAN engine data was recorded with proprietary Caterpillar software.
Using the CAT engine data as well as the PEMS NTE event data, one 1-hour test segment was
extracted from each of the 3  routes tested during the CE-CERT validation exercise.  The 1-hour
test segments were chosen to include a large number of NTE events while also offering diverse
engine operation. Each 1-hour cycle was repeated a minimum of 3 times.

       The Caterpillar CAN engine data was used to insure the laboratory testing reproduced the
engine operation experience during the CE-CERT testing.  With assistance from Caterpillar, the
SwRI Report 03.12024.06                 Page 314 of 371

-------
CAN engine data recorded during SwRI laboratory testing was  compared to the CAN  data
recorded during the CE-CERT on-road validation. Upon initial replay attempts, it was apparent
the engine was producing significantly more boost at SwRI than it did during the on-road testing.
After reviewing the engine data, Caterpillar concluded that the truck likely had a boost  leak
during the CE-CERT testing,  causing the unexpectedly low  boost pressure.   To accurately
simulate the CE-CERT replay, SwRI implemented  an adjustable boost leak in the intercooler
system. With the boost pressure similar between SwRI and CE-CERT testing, the correlation of
most  of the engine parameters improved.  After  several  iterations of testing at SwRI  and
subsequent recommendations  by Caterpillar, the accuracy  of the engine replay was deemed
acceptable by Caterpillar and SwRI.

       The first replay test cycle was taken  from Route  1, which was the  San Diego  trip.
Because the average elevation during Route 1 nearly matched the elevation of San Antonio, no
altitude simulation was necessary.   Cycle 2 was taken from Route 2, which  was the trip to
Mammoth Mountain. The test cell simulated the average barometric pressure calculated for the
hour test segment used for the replay. Cycle 3 was extracted from Route 3, which was the return
trip from Mammoth Mountain back to Riverside.  Similar to cycle 2, the test cell simulated the
average elevation calculated for the hour-long test segment of Route 3. For cycle 2 the elevation
simulation was 4500 feet, while for cycle 3 the average elevation was 3500 feet.

       Figure 234  shows the  average  NTE event engine speed recorded with the Caterpillar
software for cycle 2 in the laboratory and  for the CE-CERT on-road validation test. The SwRI
and CE-CERT speed traces showed excellent correlation for all replay tests.
       2200
       2000
     „ 1800
     Q.
      .
     
-------
       The SwRI and CE-CERT average NTE CAN fuel rate is shown in Figure 235 for cycle 2.
Cycle 2 and 3 showed tight fuel rate correlation.  Cycle 1 showed good agreement with the
exception of a couple NTE events where the delta between SwRI and CE-CERT CAN fuel rate
data was near 3 gal/h.
       30.0
       25.0 -
       20.0 -
                              • SwRI Fuel Rate gal/h BCeCert Fuel Rate gal/h
       10.0 -
        5.0 -
        0.0
                                            10      12
                                       NTE Event Number
                                                         14
                                                                16
                                                                       18
                                                                             20
  FIGURE 235. FUEL RATE CAN DATA COMPARISON FOR SWRI LABORATORY
      TESTING AND CE-CERT ON-ROAD VALIDATION TESTING (ROUTE 2)

      Figure 236 shows the boost pressure recorded during the SwRI replay and the CE-CERT
on-road  test.  With the laboratory intercooler system, it was impossible to match the boost
pressure for each NTE event. Therefore, SwRI attempted to center the NTE errors so that the
test average error was near zero.
SwRI Report 03.12024.06
Page 316 of 371

-------
       200
       180 -
       140 -
       120 -
       100
        80
                               • SwRI Boost P kPa • CeCert Boost P kPa
                                           10     12
                                       NTE Event Number
                                                         14
                                                                16
                                                                       18
                                                                             20
      FIGURE 236. BOOST PRESSURE CAN DATA COMPARISON FOR SWRI
    LABORATORY TESTING AND CE-CERT ON-ROAD VALIDATION TESTING
                                    (ROUTE 2)

      The timing discrepancies between the SwRI and CE-CERT testing are shown in Figure
237 for cycle 2.  Nearly all timing differences for all cycles were within 1 degree of engine
rotation.
SwRI Report 03.12024.06
Page 317 of 371

-------
     m
     ra
     0)
     T3
                            • SwRI Timing deg BTDC BCeCert Timing deg BTDC
        •  :     "*"        :  •       I
_t_.                            __•__   .A.
                                                              2 degrees
                     V      1
                                       1
                                           10      12
                                      NTE Event Number
                                                         14
                                                                16
                                                                       18
                                                                              20
      FIGURE 237. INJECTION TIMING CAN DATA COMPARISON FOR SWRI
    LABORATORY TESTING AND CE-CERT ON-ROAD VALIDATION TESTING
                                     (ROUTE 2)

      Figure 238 shows the intake manifold temperature comparison between SwRI and CE-
CERT. Similar to the boost pressure, an exact match between SwRI and CE-CERT could not be
achieved  for all NTE events.  The intake manifold temperature was set to achieve a balance
between positive and negative NTE event differences.  The  exhaust backpressure was likely
similar between the SwRI laboratory and CE-CERT tests because identical DPFs were used
during both validation efforts. The intake restriction was set per the manufacturers' specification
during laboratory testing.  Different fuels were used during testing at SwRI and during CE-
CERT's on-road testing.  Also, no effort was taken to reproduce the intake air humidity levels
observed during the CE-CERT testing.
SwRI Report 03.12024.06
           Page 318 of 371

-------
       70
       60 -
       50 -
     O 40 -
     ro
     0>
     2.

     S 30
       20 -
                                • SwRI IMT deg C BCeCert IMT deg C
                                                  I   *  *
.      .  •-      •:y• --v-    ---:•
       10 --
                                           10     12
                                      NTE Event Number
                                                         14
                                                                16
                                                                       18
                                                                              20
FIGURE 238. INTAKE MANIFOLD TEMPERATURE CAN DATA COMPARISON FOR
 SWRI LABORATORY TESTING AND CE-CERT ON-ROAD VALIDATION TESTING
                                     (ROUTE 2)

      Figure 239 shows the average NTE wet NOX concentrations measured using PEMS 5 in
the SwRI Laboratory and during the CE-CERT testing.  PEMS 5 measured slightly lower wet
NOX concentrations at SwRI compared to during the CE-CERT validation.   A  possible
explanation for the difference in wet NOX concentration may have been a difference in intake air
humidity, as no effort was taken to match this parameter. Cycle 1 and 3 wet NOX concentration
correlations were tighter than those measured for cycle 2.  Cycle  1 wet NOX deltas were centered,
with some positive and negative differences.  Cycle 3  SwRI NOX deltas were biased slightly
negative.
SwRI Report 03.12024.06
                      Page 319 of 371

-------
       700
       600
       500
Q.
Q.
o
•s
       400
       300
     iu
     0.
       200 -
       100
                         • SwRI NTE Ave. Wet NOx ppm BCeCert NTE Ave. Wet NOx ppm
                                            10      12
                                       NTE Event Number
                                                          14
                                                                 16
                                                                        18
                                                                               20
     FIGURE 239. PEMS 5 WET NOX COMPARISON FOR SWRI LABORATORY
      TESTING AND CE-CERT ON-ROAD VALIDATION TESTING (ROUTE 2)


      Figure 240 and Figure 241  show the PEMS 5 exhaust flow rate and wet NOX mass rate
for SwRI and CE-CERT validation tests.  The exhaust flow rates showed good correlation for the
three cycles.  Therefore, the wet  NOX mass flow rates showed trends similar to wet NOX
concentration.
SwRI Report 03.12024.06
                              Page 320 of 371

-------
        1400
                      • SwRI NTE Ave. Exh. Flow Rate scfm BCeCert NTE Ave. Exh. Flow Rate scfm
        1200

        1000                      _
                         *             m  i          *
        800 f -                                  - -J -

     £  600 - - -
     LU

     I  400 f

        200

          0
           0      2       4      6      8      10      12      14     16     18     20
                                        NTE Event Number

      FIGURE 240. PEMS 5 EXHAUST FLOW RATE COMPARISON FOR SWRI
    LABORATORY TESTING AND CE-CERT ON-ROAD VALIDATION TESTING
                                      (ROUTE 2)



        0.50
                       • SwRI NTE Ave. NOx Mass Rate g/s BCeCert NTE Ave. NOx Mass Rate g/s
        0.45

        0.40 -

        035 fj             .      *   *     *  *   *---•-
                            •                               *
        0.30 -

        0.25 -

        0.20 J-   - - *-

        0.15 -
     LU
     0.
        0.10

        0.05 -
       0.00
                                       8      10     12
                                        NTE Event Number
                                                           14
                                                                  16
                                                                         18
                                                                                20
      FIGURE 241. PEMS 5 NOX MASS FLOW RATE COMPARISON FOR SWRI
    LABORATORY TESTING AND CE-CERT ON-ROAD VALIDATION TESTING
                                      (ROUTE 2)
SwRI Report 03.12024.06
Page 321 of 371

-------
       Brake-specific emission results for each NTE  event  were calculated with  data from
PEMS 5 using the three different calculation methods for each cycle repeat. The PEMS relied on
ECM broadcast parameters to calculate the work term  of each calculation method.  Based on
Caterpillar's recommendation, the percent torque, percent friction torque, and reference torque
J1939 CAN broadcast channels were used to estimate torque.  ECM broadcast fuel rate was used
with the ECM torque estimation to calculate BSFC.  The PEMS brake-specific emission results
were compared to the laboratory dilute reference brake-specific emission results.  The laboratory
reference calculation method used torque from the inline torque meter to determine work.

       The NOX emission deltas for PEMS 5 using calculation Method 1 are shown in Figure
242. Cycle 1 deltas were relatively small,  with mostNOx deltas between -0.05 and 0.1 g/(hp-hr).
Cycle 2 and 3 deltas,  however, were  unexpected large,  with deltas ranging from  0.0 to 0.6
g/(hp-hr) for NOX.  To insure the large deltas were not caused by a problem with PEMS 5, cycle
1 and  2  data was  compared for PEMS  4.   The PEMS 4 and  5 delta  results were similar,
indicating PEMS  5 was functioning  properly.  There  was also  concern that the  altitude
simulation was causing PEMS measurement errors or errors with the ECM broadcast torque and
fuel rate parameters. Due to the experimental setup, the PEMS sample line  and EFM were the
only components of the measurement system subjected to the reduced pressure during the cycle 2
and 3 altitude simulation.  With all  other PEMS ports referenced to ambient pressure,  the PEMS
was near the fault limit for sample vacuum. To  insure the altitude simulation was not the cause
of the increased brake-specific deltas,  cycle 3 was repeated with no altitude simulation, as was
done with cycle  1.  PEMS 5 cycle 3 deltas were similar  with and without altitude simulation,
indicating the altitude simulation did not affect the PEMS measurements or the ECM broadcast
torque information. Similarly, the PEMS  concentration  data, as compared to the laboratory raw
concentration measurements, showed no susceptibility to the reduced pressure at the sample line
inlet.
SwRI Report 03.12024.06                 Page 322 of 371

-------
                   IP5 Cycle 1 » P5 Cycle 2 A P4 Cycle 1   P4 Cycle 2 X P5 Cycle 3 • P5 Cycle 3 No Alt.
                                PEMS 5 Method 1 NOx - Lab Reference NOx
     -0.2
                                     Lab Reference NOx [g/hp-hr]
 FIGURE 242. BRAKE-SPECIFIC NOX EMISSION DELTAS FOR PEMS 5 METHOD 1
    CALCULATION VERSUS THE LABORATORY REFERENCE (LABORATORY
                                TORQUE AND BSFC)

       With the cause of the substantial brake-specific deltas unknown,  SwRI performed another
comparison between the PEMS and the laboratory NTE emission results.  During the second
comparison, the laboratory used the ECM broadcast  parameters used by the PEMS to perform
calculation by Method 1, 2, and 3. With similar torque, fuel rate, and  BSFC values, as well as
similar calculation techniques, this second computational exercise was a comparison of emission
mass. The second SwRI correlation was similar to the comparison process used during the CE-
CERT on-road validation testing.  The reprocessed Method 1 brake-specific deltas for the PEMS
versus the laboratory are shown in Figure 234.   The large deltas observed during the PEMS
comparison to the  laboratory reference are significantly diminished when similar torque and
BSFC terms are used for the PEMS and laboratory.

       During altitude simulation, the PEMS  EFM was subjected to pressures lower than
ambient pressure.  According to Sensors Inc., the EFM static pressure measurement system was
not designed to operate with the EFM below the ambient pressure recorded by  the barometric
pressure sensor in the PEMS.  Therefore,  operation with the EFM at reduced pressure  and the
PEMS at ambient  pressure caused inaccurate flow  measurement.   Cycle 3,  at 3500  feet of
elevation simulation, showed similar brake-specific results with and without altitude simulation.
Therefore, the EFM was not likely affected by the 3500 elevation simulation.  However, cycle 2,
at 4500 feet of elevation simulation, showed brake-specific deltas nearly double those measured
during cycle  2 testing.  The EFM static pressure measurement problem may have caused the
increased deltas observed for cycle 2.
SwRI Report 03.12024.06
Page 323 of 371

-------
     0.7


     0.6


     0.5


   I 0.4
   I)
   | 0.3
   "oi
   Q
   O 0.2
   z
   10
   tn
     0.1 -
   LU
   Q.
0.0
  2

-0.1  -
     -0.2
                    I P5 Cycle 1 » P5 Cycle 2 A P5 Cycle 3  P5 Cycle 3 NO Alt.
                       PEMS 5 Method 1 NOx - Lab Method 1 NOx
                2.2
                        2.4
                                2.6       2.8
                                                 3.0
                                                         3.2
                                                                  3.4      3.6
                                                                                   38
                                     Lab Method 1 NOx [g/hp-hr]
 FIGURE 243. BRAKE-SPECIFIC NOX EMISSION DELTAS FOR PEMS 5 METHOD 1
  CALCULATION VERSUS THE LABORATORY METHOD 1 CALCULATION (ECM
                                TORQUE AND BSFC)

       The purpose of the replay testing was to compare the ECM torque and  BSFC errors
measured with the Caterpillar CIS engine to the incremental  torque and BSFC errors predicted
by the Model.  As mentioned earlier, two deltas had been generated for each calculation method
in comparing the PEMS brake-specific values to the Laboratory values.  The "full" deltas, shown
in Figure 242, were calculated using the lab measured torque as the basis for the work term.  The
"mass" deltas, shown in Figure 243, were calculated using ECM torque, fuel rate, and BSFC for
the laboratory as well as the PEMS.  For each calculation method, an incremental  "work" delta
was generated by comparing  the associated "full" delta to the appropriate "mass" delta.  As
shown  in the equation below,  this calculation was performed individually for each replay NTE
event.
          A
             replay,Work
                          =  A
                                 replay,Full
-A
       replay,Mass
       The Caterpillar CIS Method 1 work deltas generated using the equation above are shown
in Figure 244.   The work deltas are cycle  dependant and showed errors as large  as those
calculated for the full replay.  Therefore, the ECM torque and fuel rate errors were a key cause of
the brake-specific deltas observed in Figure 242.
SwRI Report 03.12024.06
                               Page 324 of 371

-------
                                    I Cycle 1 * Cycle 2 A Cycle 3
                                    Lab Reference NOx [g/hp-hr]


 FIGURE 244. INCREMENTAL BRAKE-SPECIFIC NOX EMISSION DELTAS DUE TO
       ECM TORQUE AND BSFC ERRORS FOR CALCULATION METHOD 1
       The incremental torque and BSFC deltas predicted by the Model were determined by
comparing the results of the Full Model to the results of the Validation Model.  The distinction
between these two sets of model results was described earlier in Section 2.1.11. The Full Model
results were calculated with all error surfaces  active, including the torque and BSFC  error
surfaces.  The Validation Model deltas were calculated with error surfaces related to torque and
BSFC inactive.  The difference between the Full Model deltas and the Validation Model deltas
yield deltas solely due to torque and BSFC error terms  in the Model.   The calculation shown
below was performed on an event-by-event basis for all reference NTE events, generating 195
Model predicted ECM torque and BSFC deltas.
      A
          Mode I, Work
= A
       Mode I, Full
-A
       Model, Validation
       The comparison between the ECM torque and BSFC errors measured with the Caterpillar
CIS  engine to the incremental torque and BSFC errors predicted by the Model are  shown in
Figure 245 for calculation method number 1.  Ideally, the replay torque and BSFC errors would
have been less than the incremental torque and BSFC deltas predicted by the model. However, it
should be noted that some of the replay error data exceeded the predicted values, with some of
the replay data work deltas being several  times  larger than the work deltas predicted by the
Model using the calculation given above. As described above, the predicted "work deltas" are
the difference between the Full Model and the Validation Model for a given NTE event.
SwRI Report 03.12024.06
        Page 325 of 371

-------
I
5
x
O
      Q
          1.00
          0.80 -
          0.60 -
          0.40 -
          0.20 -
          0.00
         -0.20
                                NOx g/kW-hr Method 1
                        Model with Time Alignment Adjustment
                   BS Deltas Due to ECM Torque and Fuel Rate Errors
                              Ideal or Lab Reference NOx g/kW-hr
                                  I 95th Full - 95th Valid • Replay
  FIGURE 245. INCREMENTAL BRAKE-SPECIFIC NOX DELTAS COMPARED TO
 THE INCREMENTAL TORQUE AND BSFC ERRORS PREDICTED BY THE MODEL
                          FOR CALCULATION METHOD 1

       Figure 246 shows  the  brake-specific NOX emission results for PEMS  5 Method 1
calculations versus the laboratory reference emission results for each replay NTE event.  The
laboratory reference results were calculated using torque measured with the inline torque meter.
Similar to the Method 1 calculations, Method 1 calculations with cycle 1 resulted  in relatively
small deltas, while cycle 2 and 3 had substantially larger brake-specific emission errors.
SwRI Report 03.12024.06
                             Page 326 of 371

-------
                                Cycle 1 * Cycle 2 A Cycle 3  Cycle 3 No Alt.
                                PEMS 5 Method 2 NOx - Lab Reference NOx
     -0.2
                                     Lab Reference NOx [g/hp-hr]
 FIGURE 246. BRAKE-SPECIFIC NOX EMISSION DELTAS FOR PEMS 5 METHOD 2
    CALCULATION VERSUS THE LABORATORY REFERENCE (LABORATORY
                                TORQUE AND BSFC)

       As described previously  for calculation Method 1, Method 1 brake-specific deltas were
also calculated  with the laboratory using ECM torque and fuel rate as well as the Method 1
emission calculation procedure.  Using similar torque, fuel rate, and BSFC terms, PEMS 5 brake-
specific NOX results were similar to the laboratory dilute results.  Essentially a comparison of
NTE mass, the results indicate the large brake-specific errors observed in Figure 246 were due to
torque and BSFC errors.
SwRI Report 03.12024.06
Page 327 of 371

-------
                             Cycle 1  » Cycle 2 A Cycle 3  Cycle 3 No Alt
                             PEMS 5 Method 2 NOx - Lab Method 2 NOx

     -0.1 --
     -0.2
                                    Lab Method 2 NOx [g/hp-hr]
 FIGURE 247. BRAKE-SPECIFIC NOX EMISSION DELTAS FOR PEMS 5 METHOD 2
  CALCULATION VERSUS THE LABORATORY METHOD 2 CALCULATION (ECM
                                TORQUE AND BSFC)

       The Method 1 incremental torque and BSFC errors measured during replay testing were
calculated by subtracting the full deltas, shown in Figure 246, from the mass deltas shown  in
Figure 247.  Shown in Figure 248, Method 1 torque and BSFC errors accounted for the majority
of the error  observed  when  the  PEMS brake-specific NOX results were  compared to the
laboratory reference results.
SwRI Report 03.12024.06
Page 328 of 371

-------
                                      I Cycle 1 * Cycle 2 A Cycle 3
0.8 i

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0.0
  1
-0.1
     -0.2
         8
                    Calculation Method 2 Incremental Deltas Due to Torque and BSFC Errors
                2.0
                         2.2
                                  2.4
                                      Lab Reference NOx [g/hp-hr]
 FIGURE 248. INCREMENTAL BRAKE-SPECIFIC NOX EMISSION DELTAS DUE TO
        ECM TORQUE AND BSFC ERRORS FOR CALCULATION METHOD 2

       Similar to calculation Method 1, the torque and fuel rate errors measured with PEMS 5
using calculation Method 1 were compared to the incremental torque and BSFC errors predicted
by the Model.  Shown in Figure 249, the Method 1  torque and fuel rate errors measured during
replay testing  with the Caterpillar CIS  engine  were  considerably  larger  than the Model
prediction.
SwRI Report 03.12024.06
                                Page 329 of 371

-------
                                NOx g/kW-hr Method 2
                        Model with Time Alignment Adjustment
                   BS Deltas Due to ECM Torque and Fuel Rate Errors
      O)
      X
      O
      
-------
                                Cycle 1 * Cycle 2 A Cycle 3  Cycle 3 No Alt.
                                PEMS 5 Method 3 NOx - Lab Reference NOx
     -0.2
                                    Lab Reference NOx [g/hp-hr]
 FIGURE 250. BRAKE-SPECIFIC NOX EMISSION DELTAS FOR PEMS 5 METHOD 3
    CALCULATION VERSUS THE LABORATORY REFERENCE (LABORATORY
                                TORQUE AND BSFC)

       Figure 251 shows the brake-specific PEMS 5 deltas when the laboratory used the ECM
torque, fuel rate, and BSFC values to calculation brake-specific emissions.  When torque and
BSFC  errors were removed from the PEMS and laboratory comparison,  thus yielding a mass
comparison,  the deltas for the three cycles collapsed, with most errors between -0.1 and 0.1
g/(hp-hr).  Therefore, ECM torque and fuel rate errors were the key factors driving the large
cycle 2 and 3 deltas shown in Figure 250.
SwRI Report 03.12024.06
Page 331 of 371

-------
   0.7


   0.6


   0.5  -


I  0.4-
.c

|  0.3--
"oi
Q
O  0.2 --
z
10
(/)
   0.1 --
   LU
   Q.
     0.0
     -0.1 --
     -0.2
                             I Cycle 1 » Cycle 2 A Cycle 3  Cycle 3 No Alt.
                              PEMS 5 Method 3 NOx - Lab Method 3 NOx
                2.2
                                                                   3.4
                                                                           3.6
                                                                                    38
                                      Lab Method 3 NOx [g/hp-hr]
 FIGURE 251. BRAKE-SPECIFIC NOX EMISSION DELTAS FOR PEMS 5 METHOD 3
  CALCULATION VERSUS THE LABORATORY METHOD 3 CALCULATION (ECM
                                 TORQUE AND BSFC)

       Figure 252 shows the incremental ECM torque and BSFC errors measured during replay
testing.   These deltas were  determined by taking the  difference between the PEMS errors
calculated against the laboratory reference results and the PEMS  errors calculated against the
laboratory results using ECM torque and BSFC.  Similar to the other calculation methods,
Method 1 ECM torque and fuel rate errors were large  in comparison to the mass emission deltas
shown in Figure 251.
SwRI Report 03.12024.06
                                  Page 332 of 371

-------
0.8

0.7

0.6

0.5




0.3

0.2

0.1

0.0

-0.1

-0.2
                                      I Cycle 1 * Cycle 2 A Cycle 3
                     Calculation Method 3 Incremental Deltas Due to Torque and BSFC Errors
             A
              A
                              -t/_.
                                       **   -
 •
-«-
                                                                V
                2.0
                         2.2
                                  2.4
                                          2.6
                                                   2.
                                                            3.0
                                                                     3.2
                                                                             3.4
                                                                                      36
                                      Lab Reference NOx [g/hp-hr]
 FIGURE 252. INCREMENTAL BRAKE-SPECIFIC NOX EMISSION DELTAS DUE TO
        ECM TORQUE AND BSFC ERRORS FOR CALCULATION METHOD 3

       Similar to calculation Method 1  and 2, the torque and fuel rate  errors measured with
PEMS 5 using calculation Method 1 were compared to the incremental  Method 1  torque and
BSFC errors predicted by the Model.  Shown in Figure 253, the Method  1 torque and fuel rate
errors measured during replay testing with the Caterpillar CIS  engine were considerably  larger
than the Model prediction.
SwRI Report 03.12024.06
                                Page 333 of 371

-------
                                 NOx g/kW-hr Method 3
                         Model with Time Alignment Adjustment
                   BS Deltas Due to ECM Torque and Fuel Rate Errors
       O)
       X
       
-------
        I Cycle 1 *Cycle 2 ACycle 3  Cycle 3 No Alt. X Cycle 3 No Leak •Cycle 3 SwRI Lug Curve  Cycle 3 Cert Lug Curve
     10%
      5%
   o
   Q.
   5  "«
   3
   O"
      -5% H
     -10%
   <
   O
     -15%
     -20%
                         •    -.'          v-r   •**
0  _  7|j)     800  _• 900   ^ 10000   1100     1200     tfOft  0 A^OO    1500
                                          ^
1600
                                                      A
                                                      X

                      x A

                         X
                                          x
                                Lab Reference Average NTE Torque [Ibf-ft]
FIGURE 254. ECM BROADCAST TORQUE ERRORS MEASURED DURING REPLAY
           VALIDATION TESTING WITH A CATERPILLAR CIS ENGINE

      NTE event average deltas between the ECM fuel rate and the laboratory dilute carbon
balance fuel rate are  shown in Figure 255 for the 3 cycles.  Most fuel rate errors were between 0
and 6 %  of point. The dilute carbon balance error versus the laboratory fuel flow meter was
approximately -1.0 to -1.5 %, therefore, the deltas shown in Figure 255 may be approximately
1.0% high.
SwRI Report 03.12024.06
                          Page 335 of 371

-------
     14%
     12%
   o
   Q.
   «  8%
   ra
   "oi
   Q

   1  6%
   <
   O
      4% -
      2%
      0%
            Cycle 1  • Cycle 2 A Cycle 3
                                     (CAN Fuel Rate - Lab CB Fuel Rate)/Lab CB Fuel Rate
                      -«
ft
        10
                   12         14          16         18         20
                         NTE Event Average Lab Dilute Carbon Balance Fuel Flow [g/sec]
                                                                        22
                                                                                  24
FIGURE 255. ECM BROADCAST FUEL FLOW RATE ERRORS MEASURED DURING
      REPLAY VALIDATION TESTING WITH A CATERPILLAR CIS ENGINE

       The BSFC values used  for  the  PEMS  brake-specific  emission calculations  were
calculated using the ECM broadcast torque and fuel rate parameters. The ECM-based BSFC
values were compared to the laboratory BSFC results which were calculated using the dilute
carbon balance  fuel flow and  the  inline  torque  meter.  The  deltas between the ECM and
laboratory BSFC are shown in Figure 256.  For cycle 1, BSFC deltas were between 0 and 5 % of
point.  Cycle 2 and 3, however, had significantly higher BSFC deltas, with peak values near 20
SwRI Report 03.12024.06
Page 336 of 371

-------
     25%
     20%
     15%
   HI
   Q
   O
    10%
   m

-------
                   %   Load,-
                                  ECM _ % _ Loadlndicated - % _ Loadpnction
                                          100%-%  Load
                                                           Friction
       SwRI did not use the curb idle % load correction when processing the lug curve error
data.
   LU
   HI
   3
   O"
   O
40

30

20

10

 0

-10

-20

-30

-40

-50

-60
                          I  •
                                .1
• DDC ECM Error [% of Point]
• DDC Lug Curve Error [% of Point]
                                           I       I
                200
                         440
                                  600
                                           800
                                                   1000
                                                            1200
                                                                     1400
                                                                              1600
                                                                                       1800
                                                       40-Point Map Points 1 through 4
                                                         (High speed governer points)
                                                             (See DDC Map)
                                    Laboratory Reference Torque [Ibf-ft]
    FIGURE 257. ECM BROADCAST TORQUE ERRORS MEASURED DURING 40-
     POINT MAP GENERATION WITH A DETROIT DIESEL SERIES 60 ENGINE

       Figure 258 shows the ECM torque errors for the Caterpillar C9 test engine.  The ECM
broadcast torque errors were similar to the errors generated using the certification lug curve
torque estimation technique.  Similar to the DDC engine, torque errors were minimal at high load
points, but increased as the load level deceased.  Engine 2 had torque errors near 50 % of point at
low load points.
SwRI Report 03.12024.06
                                 Page 338 of 371

-------
   o
   Q.
   HI

   0"
   o
   LU
     -100
                                                               »CAT ECM Error [% of Point]
                                                               • CAT Lug Curve Error [% of Point]
                                                            800
                                                                         1000
                                                                                      1200
                  40-Point Map Points 1 through 4
                   (High speed governer points)
      -80 --
                                    Laboratory Reference Torque [Ibf-ft]
    FIGURE 258. ECM BROADCAST TORQUE ERRORS MEASURED DURING 40-
          POINT MAP GENERATION WITH A CATERPILLAR C9 ENGINE

       Figure 259 shows the ECM torque errors for the International VT365 testing engine. The
International engine did not broadcast J1939 torque parameters, therefore, J1708 percent load at
current speed was used with a certification lug curve to calculate torque.  At high torque levels,
the ECM torque error was almost 8 % low.  Similar to engines 1 and 2, engine 3 torque  errors
increased as the  load level  decreased.   At the  lowest load  levels, the torque  error was
approximately 20 %.  For all lug curve torque analysis, the curb idle % load correction was not
used.
SwRI Report 03.12024.06
Page 339 of 371

-------
     25
     20
      15
   S.  10
   "n"  ^
   o
   w
   d)  0
   O"
      -5
   o
   LU
     -10
     -15
     -20
                                                            »INT Lug Curve Error [% of Point]
  .    i
                    100
200     0    300'
                                                           400
                                                                        500
                                     * *
                                                        40-Point Map Points 1 through 4
                                                         (High speed governer points)
                                   Laboratory Reference Torque [Ibf-ft]
    FIGURE 259. ECM BROADCAST TORQUE ERRORS MEASURED DURING 40-
      POINT MAP GENERATION WITH AN INTERNATIONAL VT365 ENGINE

       The ECM torque and BSFC errors measured during replay testing were substantially
larger than the incremental torque and BSFC errors predicted by the model, indicating the torque
and BSFC portion of the Model did not validate.  One potential cause for the Model invalidation
was the that the deltas of the manufacturer supplied error surfaces for torque and BSFC were far
less than the ECM torque and BSFC deltas measured with the replay engine.  In fact, the torque
deltas for all  of the engine exceeded the OEM supplied torque and BSFC errors.  A complete
discussion of the manufacturer supplied error surfaces can be found in the Torque and BSFC
OEMs Supplied Error Surfaces section of the report.

       Although the replay engine testing showed torque and BSFC  deltas that exceeded the
Model prediction, the  Steering Committee decided to take no action in regard to the replay
validation testing results.

6.10   Validation Results

       This section  contains a summary  of the model validation results,   Section 2.1.11 on
Validation contains a more detailed description of the  validation methodology utilized both in
the simulation and in the on-road data collection efforts.

       During the Monte Carlo simulation of the  195 reference NTE  events some of the error
surfaces were excluded in  the  computation of the BS emissions 'with errors'  so that the
simulation  represented  conditions used in  collecting  the  on-road data.   The error surfaces
excluded were torque errors (# 29-32, 34, 35), BSFC  errors (#36-39, 41, 42),  dynamic speed
SwRI Report 03.12024.06
    Page 340 of 371

-------
(#43) and dynamic fuel rate (#44). As described earlier in Section 2.1.11, this was necessary due
to the fact that there was no reference torque or reference fuel flow measurement during on-road
data collection.   Therefore it would  not be appropriate to use these error terms from the
simulation when comparing to the on-road data set, as the on-road data set would not include any
errors generate from those measurements. For each reference NTE event, the differences in BS
emissions were computed as:

      delta BS emissions = BS emissions with "Validation error" - "Ideal" BS emissions.

These delta emissions were computed  for each of the three  emissions and all three  calculation
methods. The validation results also included time alignment and checks for periodic drift.

       The on-road results were  gathered from selected routes driven to collect emissions data
with a CE-CERT trailer and a PEMS installed in the tractor pulling the trailer (see Section 2.1.11
on Validation). For each on-road NTE  event, a delta BS emissions value was computed as

            delta BS emissions = PEMS BS emissions - CE-CERT BS emissions.

These differences were computed for all three emissions and three calculation methods. The on-
road delta BS emissions were calculated from 81 NTE events for BSNOX and  87 NTE events for
BSNMHC and BSCO selected by the Steering Committee from the original set of data collected
on-road.  All of the NTE events selected from the on-road data were drift corrected.

       From the MC simulations, the 5th and 95th percentile delta BS emissions were extracted
from the output files for each reference NTE event.  These percentiles were then plotted as
empirical distribution functions  (edf) to  form a validation interval  for the on-road data.  Also
plotted was the edf computed from the on-road NTE events.  Figure 260 represents the validation
plot for the BSNOX Method 1  analyses.  Since all of the on-road delta BSNOX emissions fell
within the 5th and 95th percentiles of  the simulation results the model  was validated for this
method and emissions.
SwRI Report 03.12024.06                 Page 341 of 371

-------
                           Validation Analysis 5th and 95th Percentile Deltas
                       Compared Ref NTE #1—195 to Corrected 81 CE-CERT Deltas
                                       (g/kW—hr) Method 1
       100
     c
     il!
     0
     aj
     a.
        80-
f:0 •
        40«
        20 •
         -0.4
                  -0.2
                            0.0
                                     0.2       0.4       0.6

                                        Delta NCx g/kW-hr
                                                                0.8
                                                                         1.0
                                                                                  1.2
  FIGURE 260. VALIDATION ON-ROAD AND MODEL GENERATED EMPIRICAL
                DISTRIBUTION FUNCTIONS FOR BSNOX METHOD 1

       The validation edf plot for the BSNOX Method 2 results is shown in Figure 261.  Note
that approximately 45% of the  on-road  delta BSNOX emissions fell above the 95th percentile
deltas from  the  simulation model.  Thus, the model was not considered valid for the BSNOX
Method 2 results.
SwRI Report 03.12024.06
                             Page 342 of 371

-------
                           Validation Analysis 5th and 95th Percentile Deltas
                       Compared Ref NTE #1—195 to Corrected 81 CE-CEFTT Deltas
                                    NOx (g/kW-hr) Method 2
       100*
        6u
        40«
        20
         —1.2
                        -0.8
                               -0.6
                                      —0.4     -0.2     0.0

                                        Delta NOc g/kW-rir
                                                             0,2
                                                                    0.4
                                                                           0.6
  FIGURE 261. VALIDATION ON-ROAD AND MODEL GENERATED EMPIRICAL
                DISTRIBUTION FUNCTIONS FOR BSNOX METHOD 2

       The validation edf plot for the BSNOX Method 3 results is shown in Figure 262.  Note
that approximately 55% of the  on-road delta BSNOX emissions fell above the 95th percentile
deltas from  the  simulation model.  Thus, the model was not considered valid for the BSNOX
Method 3 results.
SwRI Report 03.12024.06
Page 343 of 371

-------
                            Validation Analysis 5th and 95th Percentile Deltas
                        Compared Ref NTE #1—195 to Corrected 81 CE-CEFTT Deltas
                                    NOx (g/kW-hr) Method 3
       100*
        6u
        40«
        20
          —1.2
                        -0.8
                                -0.6
                                       —0.4    -0.2     0.0

                                         Delta NOc g/kW-hr
                                                              0,2
                                                                     0.4
                                                                            0.6
  FIGURE 262. VALIDATION ON-ROAD AND MODEL GENERATED EMPIRICAL
                 DISTRIBUTION FUNCTIONS FOR BSNOX METHOD 3

       Figure 263 through Figure 265 represent the validation plots for the BSNMHC analyses
for Methods 1, 2 and 3, respectively.  Since the  entire on-road delta BSNMHC emissions fell
within the 5th and 95th percentiles of the simulation results the model was validated for all three
methods for BSNMHC emissions.
SwRI Report 03.12024.06
Page 344 of 371

-------
     Ill
     CL
                             Validation Analysis 5th and 95th Percentile Deltas
                        Compared Ref NTE #1—195 to Corrected 87 CE-CERT Deltas
                                     NMHC (g/kW-hr) Method 1
       100'
        6u
        40«
        20
                                                                           95th % MC Delta
                                                                                     195
         —0.02
                       0,00
                                   0.02          0.04          0.06

                                          Delta NMHC g/kW-hr
                                                                          0.08
                                                                                       0.10
  FIGURE 263. VALIDATION ON-ROAD AND MODEL GENERATED EMPIRICAL
                DISTRIBUTION FUNCTIONS FOR BSNMHC METHOD 1
SwRI Report 03.12024.06
Page 345 of 371

-------
                             Validation Analysis 5th and 95th Percentile Deltas
                         Compared Ref NTE #1—195 to Corrected 87 CE-CERT Deltas
                                     NMHC (g/kW—hr) Method 2
        100
     c
     il!
     0
     aj
     a.
        80-
f:0 •
        40«
        20"
                   -0.035
                            -0.025
                                      -0.015    -0.006     0.005

                                          Delta NMHC g/kW-hr
                                                                   0.015
                                                                             0.025
                                                                                      u.u'jb
  FIGURE 264. VALIDATION ON-ROAD AND MODEL GENERATED EMPIRICAL
                DISTRIBUTION FUNCTIONS FOR BSNMHC METHOD 2
SwRI Report 03.12024.06
                                Page 346 of 371

-------
                           Validation Analysis 5th and 95th Percentile Deltas
                       Compared Ref NTE #1—195 to Corrected 87 CE-CERT Deltas
                                   NMHC (g/kW—hr) Method 3
       100'
                 -0.035
                          -0.025
                                   -0.015    -0.005     0.005

                                       Delta NMHC g/kW-hr
                                                               0.015
                                                                        0.025
                                                                                 0.035
  FIGURE 265. VALIDATION ON-ROAD AND MODEL GENERATED EMPIRICAL
               DISTRIBUTION FUNCTIONS FOR BSNMHC METHOD 3

       The validation edf plot for the BSCO Method 1 results is shown in Figure 266. Note that
all of the of the on-road delta BSCO emissions fell below the  5th percentile deltas from  the
simulation model.  Thus, the model was not considered valid for the BSCO Method 1 results.
SwRI Report 03.12024.06
Page 347 of 371

-------
                           Validation Analysis 5th and 95th Percentile Deltas
                       Compared Ref NTE #1—195 to Corrected 87 CE-CERT Deltas
                                    CO (g/kW-hr) Method 1
       100'
        SO
        40«
        20
                                                                       95th % MC Delta
                                                                                196
         —0.25    0.00     0.25     0.50      0.75      1.00

                                         Delta CO g/kW-hr
                                                          1.25
                                                                  1.50
                                                                          1.75
                                                                                  2.00
  FIGURE 266. VALIDATION ON-ROAD AND MODEL GENERATED EMPIRICAL
                 DISTRIBUTION FUNCTIONS FOR BSCO METHOD 1
       The validation edf plots for the BSCO Methods 2 and 3 results are shown in Figure 267
and  Figure  268, respectively.   Note that approximately  20% of the  on-road  delta  BSCO
emissions fell below the 5th percentile deltas from the simulation model for both methods.  Thus,
the model was not considered valid for the BSCO Method 2 or Method 3 results.
SwRI Report 03.12024.06
Page 348 of 371

-------
                             Validation Analysis 5th and 95th Percentile Deltas
                        Compared Ref NTE #1—195 to Corrected 87 CE-CERT Deltas
                                      CO (g/kW-hr) Method 2
       100 H
          -0.2
                 -0.1
                        0.0
                               0.1
                                      0.2     0.3     0.4     0.5

                                           Delta CO g/kW-hr
                                                                  0.6
                                                                         0.7
                                                                                0.8
  FIGURE 267.  VALIDATION ON-ROAD AND MODEL GENERATED EMPIRICAL
                  DISTRIBUTION FUNCTIONS FOR BSCO METHOD 2
SwRI Report 03.12024.06
Page 349 of 371

-------
                            Validation Analysis 5th and 95th Percentile Deltas
                        Compared Ref NTE #1—195 to Corrected 87 CE-CERT Deltas
                                     CO (g/kW-hr) Method 3
       100
          —0.2   -0.1    0.0    0.1    0.2    0.3    0.4    0.5   0.6    0.7    0.8    0.9    1.0    1,1
                                          Delta CO g/kW-hr
  FIGURE 268.  VALIDATION ON-ROAD AND MODEL GENERATED EMPIRICAL
                  DISTRIBUTION FUNCTIONS FOR BSCO METHOD 3

       Table 124 summarizes the results of the model validation for each of the three emissions
and all the methods.
SwRI Report 03.12024.06
Page 350 of 371

-------
          TABLE 124. SUMMARY OF MODEL VALIDATION RESULTS
       Emission
CE-CERT Deltas < 95th Percentile Deltas from Validation-MC Model
             Method 1      Method 2     Method 3
             Torque-Speed  BSFC        ECM Fuel Specific
       BSNOx

       BSNMHC

       BSCO
                  Yes

                  Yes

                  Yes
                                No
No
                               Yes              Yes

                               Yes              Yes

 CE-CERT Deltas > 5th Percentile Deltas from Validation-MC Model

BSNOx            Yes          Yes              Yes

BSNMHC          Yes          Yes              Yes

BSCO              No          No              No
SwRI Report 03.12024.06
                       Page 351 of 371

-------
     7.0    MEASUREMENT ALLOWANCE GENERATION AND CONCLUSIONS

    This section of the report contains a description of the final generation of the Measurement
Allowances, as well as a summary of some of the major conclusions derived from this program.

7.1    Measurement Error Allowance Results

       This section contains a summary of the measurement error allowance results using both a
regression method and a median method to determine the measurement  allowance.  Section
2.1.10 on Measurement Allowance contains a detailed description of the methodology followed
in determining these values.  This procedure was  applied to the simulation data for all 195
reference NTE events obtained for each of the three emissions and all three calculation methods.

       Figure 269 contains a  regression plot of the 95th percentile delta BSNOx values (using
Method 1 and with time alignment adjustment) versus the Ideal BSNOX values for the 195
reference NTE events.  Included in the right-hand corner of the plot is the equation  for the fitted
regression line, and the R-square (R2) value and root mean square error (RMSE) value for the
regression fit.  The R-square value of 90.56% met the criteria given  in the Test Plan for use of
the regression line for generation of a potential measurement allowance, and indicates that the
magnitude of the 95th percentile BSNOX delta was linear with  respect to the ideal BSNOX level.
The RMSE value of 0.0955 displays the size of the estimated standard deviation of the predicted
95th percentile BSNOX values.

       Table 125 includes a comparison of the results of the regression method based on Figure
269 and  the median  method  as described in the Section 2.1.10 on Measurement Allowance.
Under the heading of "Regression Method" in the table, it is shown that the R-square and RMSE
criteria are met by the  data and that the measurement error at the BSNOX threshold, based on
using the regression line to predict the value, is 22.2981% when expressed as a percent of the
threshold value of 2.68204. Since the Regression Method is applicable, the Median Method,
though shown in the table for comparison purposes, is not applicable.
SwRI Report 03.12024.06                 Page 352 of 371

-------

         0.00
                               NOx g/kW-hr Method 1
                          With Time Alignment Adjustment
                                    345

                                      Ideal NOxg/kW-hr
                                        -TH
 FIGURE 269.  REGRESSION PLOT OF 95ltt PERCENTILE DELTA BSNOX VERSUS
                          IDEAL BSNOx FOR METHOD 1
    TABLE 125. MEASUREMENT ERROR AT THRESHOLD FOR BSNOX USING
             REGRESSION AND MEDIAN METHODS FOR METHOD 1
Regression Method
R2
RMSE(SEE)
5% Median Ideal
Predicted 95th% Delta at
Threshold
Measurement Error @
Threshold=2.68204
0.9065
0.0955
0.1302
0.5980
22.2981%
Met Criteria
Met Criteria



Median Method

Median 95th% Delta
Measurement Error @
Threshold=2.68204
0.5558
20.7215%
      Figure 270 contains a regression plot of the 95th percentile delta BSNOx values (using
Method  2  and with time alignment adjustment) versus the Ideal BSNOx values for the 195
reference NTE events.  The R-square value indicates that 33.64% of the variation in the 95th
percentile BSNOx values is explained by the Ideal BSNOx values for the Method 2 data.  The
RMSE value is 0.0181.

      Table 126 includes a comparison of the results of the regression method based on Figure
270 and  the median method. Under the heading of "Regression Method" in the table, it is shown
that the  R-square criterion for using this method is  not met by the data.  Thus, the Median
Method  must be used. Under the heading "Median Method" in the table, the measurement error
SwRI Report 03.12024.06
Page 353 of 371

-------
at the BSNOx threshold, based  on using the median of the 195 95th percentile delta BSNOX
values, is 4.4507% when expressed as a percent of the threshold value of 2.68204.
         0.25
         0.20 -
         0.15 -
         0.10 -
         0.05 -
         0.00
                               NOx g/kW-hr Method 2
                          With Time Alignment Adjustment
                                                          y = 0.0087X + 0.097
                                                             R2 = 0.3364
                                                            RMSE=0.0181
                                    345

                                      Ideal NOxg/kW-hr
                                        -TH
 FIGURE 270. REGRESSION PLOT OF 95ltt PERCENTILE DELTA BSNOX VERSUS
                          IDEAL BSNOx FOR METHOD 2
    TABLE 126.  MEASUREMENT ERROR AT THRESHOLD FOR BSNOx USING
             REGRESSION AND MEDIAN METHODS FOR METHOD 2
Regression Method
R2
RMSE(SEE)
5% Median Ideal
Predicted 95th% Delta at
Threshold
Measurement Error @
Threshold=2.68204
0.3364
0.0181
0.1302
0.1203
4.1853%
Did Not Meet Criteria
Met Criteria



Median Method

Median 95th% Delta
Measurement Error @
Threshold=2.68204
0.1194
4.4507%
      Figure 271 contains a regression plot of the 95th percentile delta BSNOx values (using
Method  3  and with time alignment adjustment) versus the Ideal  BSNOx values  for the  195
reference NTE events.  The R-square value indicates that 58.28% of the variation in the  95th
percentile BSNOx values is explained by the Ideal BSNOx values for the Method 3 data.  The
RMSE value is 0.0603.
SwRI Report 03.12024.06
Page 354 of 371

-------
       Table 127 includes a comparison of the results of the regression method based on Figure
271 and the median method. Under the heading of "Regression Method" in the table, it is shown
that the R-square criterion for using this method is not met by the data.  Thus, the Median
Method must be used. Under the heading "Median Method" in the table, the measurement error
at the BSNOx threshold, based on using the median of the 195 95th percentile delta BSNOX
values, is 6.6088% when expressed as a percent of the threshold value of 2.68204.
      5?
      .c
         0.70
         0.60 -
         0.50 -
         0.40 -
         0.30 -
         0.20 -
         0.10
         0.00
                                NOx g/kW-hr Method 3
                           With Time Alignment Adjustment
                          y = 0.048x + 0.0629
                            R2 = 0.5828
                            RMSE = 0.0603—
                                     345

                                      Ideal NOxg/kW-hr
                                         -TH
 FIGURE 271. REGRESSION PLOT OF 95ltt PERCENTILE DELTA BSNOx VERSUS
                          IDEAL BSNOx FOR METHOD 3
    TABLE 127.  MEASUREMENT ERROR AT THRESHOLD FOR BSNOx USING
             REGRESSION AND MEDIAN METHODS FOR METHOD 3
Regression Method
R2
RMSE(SEE)
5% Median Ideal
Predicted 95th% Delta at
Threshold
Measurement Error @
Threshold=2.68204
0.5828
0.0603
0.1302
0.1916
7.1455%
Did Not Meet Criteria
Met Criteria



Median Method

Median 95th% Delta
Measurement Error @
Threshold=2.68204
0.1773
6.6088%
                                              -th
      Figure 272 contains a regression plot of the 95  percentile delta BSNMHC values (using
Method  1 and with time alignment adjustment) versus the Ideal BSNMHC values for the  195
reference NTE events.  The R-square  value indicates that 79.73% of the variation in the  95th
SwRI Report 03.12024.06
Page 355 of 371

-------
percentile BSNOx values is explained by the Ideal BSNMHC values for the Method 1 data.  The
RMSE value is 0.0049.

      Table 128 includes a comparison of the results of the regression method based on Figure
272 and the median method. Under the heading of "Regression Method" in the table, it is shown
that both the R-square and RMSE criteria for using this method are not met by the data. Thus,
the Median Method  must  be used.  Under the heading "Median Method"  in  the table, the
measurement  error at the BSNMHC  threshold, based on using  the median  of the 195  95th
percentile delta BSNMHC  values, is 10.0778% when expressed as a percent of the threshold
value of 0.28161.
         0.12
         0.10 -
         0.08 -
         0.06 -
         0.04
         0.02
         0.00
                               NMHC g/kW-hr Method 1
                                                         y = 0.1639X + 0.0258
                                                            R2 = 0.7973
                                                           RMSE= 0.0049
           0.00     0.05     0.10    0.15    0.20    0.25    0.30    0.35     0.40     0.45

                                     Ideal NMHC g/kW-hr
                                             -TH
     FIGURE 272. REGRESSION PLOT OF 95ltt PERCENTILE DELTA BSNMHC
                     VERSUS IDEAL BSNMHC FOR METHOD 1
SwRI Report 03.12024.06
Page 356 of 371

-------
   TABLE 128. MEASUREMENT ERROR AT THRESHOLD FOR BSNMHC USING
             REGRESSION AND MEDIAN METHODS FOR METHOD 1
Regression Method
R2
RMSE(SEE)
5% Median Ideal
Predicted 95th% Delta at
Threshold
Measurement Error @
Threshold=0.28161
0.7973
0.0049
0.0001
0.0719
25.5339%
Did Not Meet Criteria
Did Not Meet Criteria



Median Method

Median 95th% Delta
Measurement Error @
Threshold=0.28161
0.0284
10.0778%
                                                                                   th
      Figure 273 contains a regression plot of the 95th percentile delta BSNMHC values (using
Method  2 and with time alignment adjustment) versus the Ideal BSNMHC values for the 195
reference NTE events.  The R-square value  indicates that 7.07% of the variation in the 95
percentile BSNOx values is explained by the Ideal BSNMHC values for the Method 2 data.  The
RMSE value is 0.0042.

      Table 129 includes a comparison of the results of the regression method based on Figure
273 and  the median method.  Under the heading of "Regression Method" in the table, it is shown
that both the R-square and RMSE criteria for using this method are not met by the data. Thus,
the Median Method  must be  used.  Under the  heading  "Median Method" in  the table, the
measurement  error at the BSNMHC threshold,  based on using the median of the 195  95th
percentile delta BSNMHC values, is 8.0310% when expressed as a percent of the threshold value
ofO.28161.
SwRI Report 03.12024.06
                                   Page 357 of 371

-------
        0.035
        0.000
                               NMHC g/kW-hr Method 2
                                                       R2 = 0.0707
                                                     RMSE= 0.0042
        0.010 -
        0.005
            0.00    0.05    0.10    0.15    0.20    0.25    0.30     0.35    0.40    0.45

                                     Ideal NMHCg/kW-hr
                                            -TH
    FIGURE 273. REGRESSION PLOT OF 95ltl PERCENTILE DELTA BSNMHC
                    VERSUS IDEAL BSNMHC FOR METHOD 2
   TABLE 129. MEASUREMENT ERROR AT THRESHOLD FOR BSNMHC USING
             REGRESSION AND MEDIAN METHODS FOR METHOD 2
Regression Method
R2
RMSE(SEE)
5% Median Ideal
Predicted 95th% Delta at
Threshold
Measurement Error @
Threshold=0.28161
0.0707
0.0042
0.0001
0.0182
6.4671%
Did Not Meet Criteria
Did Not Meet Criteria



Median Method

Median 95th% Delta
Measurement Error @
Threshold=0.28161
0.0226
8.0310%
      Figure 274 contains a regression plot of the 95th percentile delta BSNMHC values (using
Method  3 and with time alignment adjustment) versus the Ideal BSNMHC values for the  195
reference NTE events.  The R-square value indicates that 0.21% of the variation in the  95th
percentile BSNOx values is explained by the Ideal BSNMHC values for the Method 3 data.  The
RMSE value is 0.0047.

      Table 130 includes a comparison of the results of the regression method based on Figure
274 and  the median method.  Under the heading of "Regression Method" in the table, it is shown
that both the R-square and RMSE criteria for using this method are not met by the data.  Thus,
the Median Method must be  used. Under the heading "Median  Method" in the table,  the
SwRI Report 03.12024.06
Page 358 of 371

-------
measurement  error at the BSNMHC threshold, based on using the median of the  195  95th
percentile delta BSNMHC values, is 8.4436% when expressed as a percent of the threshold value
ofO.28161.
         0.040
         0.010
                               NMHC g/kW-hr Method 3
         0.035 -
         0.030
         0.025
         0.020
         0.015 -
                   y = 0.0036X + 0.0234
                      R2 = 0.0021
                     RMSE = 0.0047
             0.00    0.05    0.10     0.15     0.20    0.25    0.30    0.35    0.40    0.45

                                      Ideal NMHC g/kW-hr
                                            -TH
     FIGURE 274. REGRESSION PLOT OF 95ltl PERCENTILE DELTA BSNMHC
                     VERSUS IDEAL BSNMHC FOR METHOD 3
   TABLE 130.  MEASUREMENT ERROR AT THRESHOLD FOR BSNMHC USING
             REGRESSION AND MEDIAN METHODS FOR METHOD 3
Regression Method
R2
RMSE(SEE)
5% Median Ideal
Predicted 95th% Delta at
Threshold
Measurement Error @
Threshold=0.28161
0.0021
0.0047
0.0001
0.0244
8.6739%
Did Not Meet Criteria
Did Not Meet Criteria



Median Method

Median 95th% Delta
Measurement Error @
Threshold=0.28161
0.0238
8.4436%
      Figure 275 contains a regression plot of the 95th percentile delta BSCO values (using
Method  1 and with time alignment adjustment) versus the Ideal BSCO values for the 195
reference NTE events.  The R-square value indicates that 60.48% of the variation in the 95th
percentile BSCO values is explained by the Ideal BSCO values for the Method 1  data.  The
RMSE value is 0.1403.
SwRI Report 03.12024.06
Page 359 of 371

-------
      Table 131 includes a comparison of the results of the regression method based on Figure
275 and the median method. Under the heading of "Regression Method" in the table, it is shown
that both the R-square and RMSE criteria for using this method are not met by the data.  Thus,
the Median Method must be used.  Under the heading  "Median Method"  in  the table, the
measurement error at the BSCO threshold, based on using the median of the 195  95th percentile
delta BSNMHC values, is 2.5775% when expressed as  a percent of the threshold value of
26.015.
                                CO g/kW-hr Method 1
                          With Time Alignment Adjustment
         2.50
                                                      y = 0.2428X + 0.5685
                                                          R2 = 0.6048
                                                         RMSE= 0.1403
         0.00
                                        3        4
                                      Ideal COg/kW-hr
                                          -TH
 FIGURE 275.  REGRESSION PLOT FOR 95ltt PERCENTILE DELTA BSCO VERSUS
                          IDEAL BSCO FOR METHOD 1
     TABLE 131. MEASUREMENT ERROR AT THRESHOLD FOR BSCO USING
             REGRESSION AND MEDIAN METHODS FOR METHOD 1
Regression Method
R2
RMSE(SEE)
5% Median Ideal
Predicted 95th% Delta at
Threshold
Measurement Error @
Threshold=26.015
0.6048
0.1403
0.0192
2.0242
7.7808%
Did Not Meet Criteria
Did Not Meet Criteria



Median Method

Median 95th% Delta
Measurement Error @
Threshold=26.015
0.6705
2.5775%
                                                -th
      Figure 276 contains a regression plot of the 95  percentile delta BSCO values (using
Method  2 and with time  alignment adjustment) versus the Ideal BSCO values for the  195
SwRI Report 03.12024.06
Page 360 of 371

-------
reference NTE events.  The R-square value indicates that 3.67% of the variation in the 95th
percentile BSCO values is explained by the Ideal BSCO  values for the Method 2 data.  The
RMSE value is 0.1139.

      Table 132 includes a comparison of the results of the regression method based on Figure
276 and the median method. Under the heading of "Regression Method" in the table, it is shown
that both the R-square and RMSE criteria for using this method are not met by the data.  Thus,
the Median Method  must be used. Under the heading "Median Method" in the table,  the
measurement error at the BSCO threshold, based on using the median of the 195 95th percentile
delta BSNMHC values, is 1.9924% when expressed as a percent of the threshold value of
26.015.
                                 CO g/kW-hr Method 2
                           With Time Alignment Adjustment
         0.90
         0.80
         0.70 -
         0.60 -
         0.50 -
         0.40 -
         0.30
                  y =0.0311x + 0.5156
                     R2 = 0.0367
                  —RMSE = 0.1139
                                         3        4

                                       Ideal COg/kW-hr
                                          -TH
  FIGURE 276.  REGRESSION PLOT OF 95ltt PERCENTILE DELTA BSCO VERSUS
                           IDEAL BSCO FOR METHOD 2
SwRI Report 03.12024.06
Page 361 of 371

-------
     TABLE 132. MEASUREMENT ERROR AT THRESHOLD FOR BSCO USING
             REGRESSION AND MEDIAN METHODS FOR METHOD 2
Regression Method
R2
RMSE(SEE)
5% Median Ideal
Predicted 95th% Delta at
Threshold
Measurement Error @
Threshold=26.015
0.0367
0.1139
0.0192
0.7020
2.6986%
Did Not Meet Criteria
Did Not Meet Criteria



Median Method

Median 95th% Delta
Measurement Error @
Threshold=26.015
0.5183
1.9924%
                                                                                  th
      Figure 277 contains a regression plot of the 95th percentile delta BSCO values (using
Method  3 and with time alignment adjustment)  versus the Ideal BSCO values for the  195
reference NTE events. The R-square value indicates that only 22.09% of the variation in the 95
percentile BSCO values is explained by the Ideal BSCO values for the Method 3 data.   The
RMSE value is 0.1254.

      Table 133 includes a comparison of the results of the regression method based on Figure
277 and  the median method. Under the heading of "Regression Method" in the table, it is shown
that both the R-square and RMSE criteria for using this method are not met by the data.  Thus,
the Median Method must be used. Under  the heading "Median Method"  in  the table, the
measurement error at the BSCO threshold, based on using the median of the 195  95th percentile
delta BSNMHC values, is 2.1117% when expressed as a percent of the threshold value of
26.015.
SwRI Report 03.12024.06
                                   Page 362 of 371

-------
         1.40
         0.20
                                CO g/kW-hr Method 3
                           With Time Alignment Adjustment
                                                       y = 0.0932X + 0.5156
                                                          R2 = 0.2209
                                                         RMSE = 0.1254
                                        3         4

                                       Ideal COg/kW-hr
                                         -TH
  FIGURE 277.  REGRESSION PLOT OF 95ltl PERCENTILE DELTA BSCO VERSUS
                           IDEAL BSCO FOR METHOD 3
     TABLE 133. MEASUREMENT ERROR AT THRESHOLD FOR BSCO USING
             REGRESSION AND MEDIAN METHODS FOR METHOD 3
Regression Method
R2
RMSE(SEE)
5% Median Ideal
Predicted 95th% Delta at
Threshold
Measurement Error @
Threshold=26.015
0.2209
0.1254
0.0192
1.0744
4.1299%
Did Not Meet Criteria
Did Not Meet Criteria



Median Method

Median 95th% Delta
Measurement Error @
Threshold=26.015
0.5494
2.1117%
       Table 134 contains a summary of the measurement error values contained in Table 125
through Table 133. The values are categorized by emissions and by calculation method.  The
maximum error by method is  listed in the last row  of the  table. Although the Test Plan
methodology (i.e., see Section 2.1.10  on Measurement Allowance) initially indicated that the
minimum of these values was to be used to select the best method, the actual choice is dependent
on the ability to validate the model runs using the on-road CE-CERT data  After reviewing the
validation results given in Table 124  and noting that only Method 1 validated for BSNOx, the
Steering Committee decided at a meeting held on February  26, 2007 to use Method  1 to
determine the measurement error allowance.
SwRI Report 03.12024.06
Page 363 of 371

-------
   TABLE 134. MEASUREMENT ERROR IN PERCENT OF NTE THRESHOLD BY
                   EMISSIONS AND CALCULATION METHOD
Measurement Errors (%) at Respective NTE Threshold
Emission
BSNOx
BSNMHC
BSCO
Max Error
Method 1
Torque-Speed
22.30
10.08
2.58
22.30
Method 2
BSFC
4.45
8.03
1.99
8.03
Method 3
ECM Fuel Specific
6.61
8.44
2.11
8.44
Note: Values in table cells shaded white were successfully validation, while values
in cells shaded grey were not validated.
      Table 135 includes in the last column of the table the measurement error allowances by
emissions for Method 1.   The values are 0.44596 g/hp-hr for BSNOX, 0.02116 g/hp-hr for
BSNMHC, and 0.50002 g/hp-hr for BSCO.
TABLE 135.  MEASUREMENT ALLOWANCE AT NTE THRESHOLD BY EMISSIONS
                                 FOR METHOD 1
Emission
BSNOx
BSNMHC
BSCO
Method 1
Measurement
Error %
22.30
10.08
2.58
NTE
Threshold
g/kW-hr
2.68204
0.28161
26.015
Measurement
Allowance,
g/kW-hr
0.59804
0.02838
0.67054
Measurement
Allowance,
g/hp-hr
0.44596
0.02116
0.50002
7.2   Conclusions

     The primary result of this program is the set of Measurement Allowances given in the
previous section.  However, a number of other observations and conclusions may be drawn from
the experiences and data generated over the course of the program. This section of the report
details some of these observations, as well as recommendations arising from them.
SwRI Report 03.12024.06
Page 364 of 371

-------
7.2.1   Engine-Instattation-PEMS Variability

       During the course of this program, a variety of PEMS were used to measure emissions
from three different engines. It became apparent that the measurement errors observed were not
consistent from installation to installation.  In some cases the use  of  different  equipment
contributed to the changes, for example the use of different size flow-meters.  In other cases,
selected PEMS did not behave in the same fashion from engine to engine, as was the case for
PEMS 6 which experienced large negative errors during steady-state testing  for Engine 3, but not
for Engine 2.  At other times,  PEMS equipment was repaired or replaced,  after which different
behavior was observed,  as was the  case  with PEMS 4 which demonstrated different NOX
measurement errors on Engine  2 after the NDUV instrument was replaced.

       The steady-state error surface data reflected these variations in the  fact that the original
errors surfaces were not very smooth in nature,  and showed abrupt changes in error magnitudes
for many key parameters at similar reference levels.  In many cases, these  abrupt changes took
place because data at similar reference levels was generated on different  test engines and test
installations.  This behavior underscores the fact that there is  considerable variability arising
from PEMS to PEMS and installation to installation.

       The Test Plan was designed to use  multiple engines in order to get some sense of this
variability.  However, with only three examples or "samples"  of different installations in the
plan, many on the Steering Committee felt in hindsight that this source of variation was not very
well characterized from a statistical point of view. It is suggested that any future efforts to assess
PEMS variation should somehow include a better means of assessing this source of error.

7.2.2  PEMS 1065 Audit Failures

       In general, the PEMS passed most of the  performance checks required under 40 CFR Part
1065  Subpart D.  However, as was  noted in  Section 3 of the report, there were numerous
linearity failures associated with the SEMTECH-DS instruments, particularly in the case of the
NDUV analyzer which measures NO and NC>2.  Although the Steering Committee approved the
continuation of the program despite these problems, this was an issue of concern to many on the
Committee.  It has been noted earlier that, due to the manner in which the Test Plan was laid out,
the linearity requirements used to audit the  PEMS  represented a relatively  liberal interpretation
of the regulations given in Subpart D, as a result of use of the span values for the instruments as
the "maximum  concentration expected during testing" for scaling of the requirements.  Despite
this interpretation, numerous intercept deviations were still observed.  Due to the requirement
that  testing  under the HDIUT  program must be conducted  using  instruments meeting all
requirements of 40 CFR Part 1065,  such deviations could be problematic.
       There was considerable Steering Committee debate regarding this issue.  It is suggested
that in any future programs of this type,  a complete set of audit data should be supplied by any
participating instrument manufacturers prior to the  start of the program, in  order to demonstrate
that all 1065  requirements can be met. However, Part 1065 audits  should  still be conducted as
part of any such program to verify compliance.
SwRI Report 03.12024.06                 Page 365 of 371

-------
7.2.3   Method 2 and Method 3 Validation for NOX

       There was considerable discussion among the Steering Committee regarding the reasons
that the Model results for Methods 2 and 3 did not validate with respect to NOX emissions.  A
significant amount of effort was also directed at determining the cause for the lack of validation
of these two methods.  These efforts were directed both at examining the Model results, and at
scrutinizing the CE-CERT on-road validation test results.

       Examination of the CE-CERT results did uncover several relatively minor issues, which
were corrected by CE-CERT or dealt with by the Steering Committee.  However, no significant
procedural issues were found either with the operation of the PEMS or of the  MEL.  The
Steering Committee found no major faults with the CE-CERT data which would have changed
the conclusions of the validation process.  Ultimately the CE-CERT results were judged by all
Steering Committee members to be a valid data set, and this opinion is shared by SwRI staff as
well.

       In a similar manner, the Steering Committee did not find that the lack of validation arose
from any deficiencies in the manner with which SwRI conducted the experiments that supplied
data to the Model.  In addition, extensive quality assurance  and checking were performed on the
Model itself in order to insure that it performed in the manner intended by the Test Plan.

       As a result of these investigations, therefore, it must be assumed that  the reason for the
lack of validation of Methods 2 and 3 for NOX arose from a real difference in PEMS behavior
between the laboratory tests at  SwRI and the field tests conducted by CE-CERT.  This would
seem  to indicate that  some source of variation  that  occurred  during  the field tests was not
captured by the various experiments which supplied data for the model. This difference could
not be related to  any Torque or BSFC errors, as those were not relevant to the on-road validation
results, as explained earlier in Section 6.  In addition, the difference could not be related to the
exhaust flow measurement, because this would have also affected the Method 1 results.

       Given these observations, the only  likely  causes for the lack of validation are related
either to the measurement of CC>2 or of NOX itself. In the  case of CC>2, the lower deltas would
have to be caused by a positive bias in the CC>2 errors predicted by the model as compared to
those  observed during on-road testing.  The Model  does in fact incorporate a positive bias in CCh
errors, as reflected in  the steady-state CC>2 error surface.  This in turn  reflects the fact that
negative PEMS CC>2 measurement errors were not observed during laboratory testing.  However,
examination of the CC>2 data from on-road testing indicates that negative errors were also not
observed during the on-road validation.  Therefore, it is unlikely that a CC>2 bias was the cause of
the lack of validation.

       As a  result, the root cause for the lack  of validation  is  likely to  lie  with the  NOX
measurement itself.  This could be the result either of a bias between the two data sets, or the
result of variation. Both reference facilities had been  correlated earlier in the program, and no
procedural errors were  found either for SwRI or for CE-CERT. In addition, the PEMS used for
on-road validation testing was one of the units used at  SwRI, and was later used again by  SwRI
for laboratory validation and  found to be in good working order.  For these reasons, and
following a certain amount of engineering analysis, the Steering Committee could not determine


SwRI Report 03.12024.06                 Page 366 of 371

-------
a reason that a bias  should exist between the  two data sets  which would result in lack of
validation for Methods 2 and 3.

       Another possibility which was raised during Steering Committee discussions was that the
level of variance predicted by the Model was not wide enough to encompass the results observed
during the CE-CERT validation testing.  The steady-state NOX error surface generally predicts an
error  of  ± 20  ppm at the  concentrations observed during  on-road validation testing.  The
sensitivity analysis  given in Section 6 earlier indicates  that this surface is a dominant driver of
NOX errors for Methods 2 and 3 in the Model. It is possible that the deltas observed during on-
road validation testing fell  outside this range.   Unfortunately, it is  not  possible to determine
concentration deltas directly from the  CE-CERT data as no raw reference concentrations are
available for comparison with PEMS values.

       An analysis  of PEMS instrument QA data was conducted for both the CE-CERT data and
some of the steady-state experiments used to generate data for the Model at SwRI.  Figure 278
shows zero calibration data observed during on-road testing, while Figure 279 shows similar data
for Engine 3  tests at SwRI.  A comparison of these two data sets indicates much larger amounts
of variation in PEMS zero calibration adjustments during on-road testing, as compared to similar
adjustments in the  laboratory.  It should be  noted that the range of these variations  observed
during on-road testing is  actually larger than the  spread predicted by  the steady-state NOX error
surface in the Model.  The cause of these larger variations is not known, but they do indicate that
some source of error is present in the on-road data that was not apparent during laboratory tests.

       A possible explanation for this difference in behavior is that it could be the result of the
installation  and equipment  variations  described  earlier.   A concern  of several  Steering
Committee members was that there  were essentially only three observations of this potential
error  source, and that the on-road validation experiment may have represented a data point
outside that range  of errors observed during these three  cases.  If this was the case, it would
suggest that the spread of the actual steady-state NOX error from 5th to 95th percentile is in fact
wider than what was given in the steady-state NOX  error surface. This does not suggest that the
observations  made  during the Measurement Allowance program were incorrect, but rather that
not enough observations were made to fully characterize the error. Unfortunately, there was not
way  to test this hypothesis within the scope of the program, and therefore there is no way of
knowing if this explanation is valid.
SwRI Report 03.12024.06                 Page 367 of 371

-------
                             PEMs Total NO+NO2 Zero Calibrations
Af\
^-^ on
E 3°
Q.
Q.
C
O
c
0)
o
C n
0 °
0 (
CM
O 10
6
-in
_*n
PEMs average NO+NO2 concentration during correlation was ~450 ppm.
10 ppm divisions are ~2.2% of 450 ppm.
3


• <
) 5

	
,: 	 ' *
< * * • • *• •
/ n 15 20 25 "^30 35 4ft 45 5








0

1
n
X
0)
•o
I
                                      Zero Count
                                 »Previoius  Current A Day Index
 FIGURE 278. VARIATION IN ZERO CALIBRATION OF PEMS DURING ON-ROAD
                              VALIDATION TESTING
SwRI Report 03.12024.06
Page 368 of 371

-------
                 PEMs 1 and 4 Total NO+NO2 Zero Calibrations Engine 3 Testing @ SwRI
  5
  I  -2
  s  ^
  c  -4
  o
  O
    -10
    -12
    -14
•   2
:  «
                                                         1*4
                                    18
20   g
                                                                                 12
                                                                                 10
                                             6
                                               (0
                                               S
                                               HI
                                               Q.
                                       Zero Count
                                 »Previoius • Current A PEMS Index
 FIGURE 279. PEMS ZERO CALIBRATION VARIATIONS DURING LABORATORY
                                      TESTING

       Ultimately, the Steering Committee decision was to accept the measurement allowances
predicted by the model for Method 1, as these were the only validated numbers.  However, EPA
indicated  that they  would  continue investigations into  the possible  causes for  the  lack  of
validation of Methods 2 and 3, as well as look into modifications that might result in achieving
validation of those  methods.   It was  agreed that any potential revision of the measurement
allowances that  might arise  from such investigations would  be reviewed by  the Steering
Committee and pursued as a cooperative effort between EPA, EMA and CARB. Further it was
determined that such changes would not apply before the 2010 model year.

7.2.4   PEMS Sampling Handling System Issues and Overflow Checks

       As described throughout this report, and related in Appendix A, there were numerous
issues observed with the  PEMS over the  course of the program. Many of these issues were
ultimately traced to problems  of deficiencies within the sample  handling  system of the
SEMTECH-DS.  In many cases, an overflow span check proved to be the diagnostic exercise
which helped to isolate the source of the issue. In such a check,  the instrument is zero and span
calibrated using the calibration port, and then the  same span gas is overflowed to the sample
probe such that it enters the instrument through the sample line  and passes through the sample
SwRI Report 03.12024.06
Page 369 of 371

-------
handling system.  Although such a check is not required by 1065, SwRI found this to be an
invaluable tool for assessing PEMS and for diagnosing problems.

       Therefore  based on its  experiences with these devices,  SwRI would  recommend the
inclusion of such an overflow check in the regular operations of a PEMS.  Such a check could be
performed  on installation of the PEMS on a vehicle, and again at the conclusion of testing.
While such a check is not strictly required under Part 1065, it is felt that performance of such a
check would help to identify many potential problems which could otherwise compromise PEMS
data.

7.2.5   Lessons Learned for Future Programs

       Near  the  conclusion  of this  program, the  Steering Committee  engaged  in several
discussions regarding various issues that  arose during the program,  and how they might be
addressed in  any  future work of this kind.  This was particularly relevant given the fact that a
subsequent effort is about to be undertaken involving in-use PM measurement.

       The result of these discussions was a list of issues, findings, and observations which were
collectively termed  as "lessons learned."  This list is not necessarily exhaustive, but it does
represent a group of observations that the Steering Committee collectively felt were the most
important in  terms of  how  any future program  of this kind  might be  conducted.   These
observations are summarized in Table 136 below. It is hoped that this information will be  useful
in planning for future efforts of this kind.

       TABLE 136. LESSONS LEARNED DURING GASEOUS MEASUREMENT
                              ALLOWANCE PROGRAM
       1.      The Steering Committee felt that a number of the biases observed on several
              error surfaces were likely the results of the limited number of engines and
              installations tested, rather than a true systematic  bias.   Therefore,  it was
              recommended that error surfaces should be centered around zero unless there
              is a technical reason not to do so.  Observed biases during program could not
      	always be explained from a technical standpoint.	
       2.      Broadcast torque needs to be revisited as an error term.  The following ideas
              were suggested on this topic:
              a.  Try to build an  exception within the test program (extended data could
                 still be added in as before)
              b.  Variability
              c.  Hardware/Software changes
              d.  Check manufacturer supplied data vs. test data
       3.      Check engine to engine variability/differences. Try to structure the program
              to account for this error source in a more robust manner.
       4.      Some error surfaces displayed highly irregular and erratic that were often the
              result of different PEMS system behavior from on test engine to another, (ex,
              exhaust flow delta varied between different sized flow meters for different
      	engines).    Test  Plan  anticipate more  consistent behavior from  engine to
SwRI Report 03.12024.06                 Page 370 of 371

-------
              engine, and experimental design likely missed  capturing sources of error
              beyond the errors expected in Test Plan development  (reproducibility  are
              larger than expected). Consider longer time frames.	
              CE-CERT validation was key to selecting Measurement Allowance.
       6.     Environmental errors were generally small compared to baseline variability
             study. Limited number of exceptions.  Environmental test generally caused
             functional failures rather than measurement errors.
       7.     Vibration test was a frequency and  direction  spectrum of wide range  of
             frequencies to directly assess errors due to in-use vibration.  It did not sweep
             frequencies to identify susceptibility.  This was out of scope. Limited field
      	vibration data. Consider adding 3-axis accelerometers on PEMS.	
             Errors in  measurement  of ambient  temperature  and humidity  were  not
             included.
       9.     PEMS OEM manuals need to be available at start of test plan to better follow
             manufacturer's recommendations.  Any gray areas or multiple methods need
      	to be clarified before start of testing.	
       10.    Incremental errors due to transient operation were smaller than expected.
       11.     Test engines emissions levels were much lower than threshold values for
              NMHC and CO, data values primarily demonstrated PEMS performance near
              zero levels, rather than at emission threshold values. Therefore, Measurement
              Allowances  may   not   entirely   represent  performance   at   threshold.
              Recommended that future work try to use test articles near thresholds, even if
      	systems need to be modified to produce them.	
       12.    Document control of Test Plan was weak.
       13.     Steering Committee decisions were not well documented during the course of
              the program in an easily referenced manner.  Improvement ideas include
              meeting and conference call minutes, monthly reports, change history of test
      	plan.	
       14.     Time alignment errors were larger than expected.  Improvement idea is to
      	include phase errors in the model.	
       15.     Test plan was not clear on the decision process if model did not validate.
       16.     Test plan did not specify how to use replay validation data.
       17.    Drift check in the model was weak.
       18.    PEMS issues burned up nearly all the contingency in the schedule.  Back-up
      	PEMS was key to staying on schedule.	
       19.    Method of pooling data into a single error surfaces changed based on actual
             results.
       20.    PEMS vehicle installation process as a source of error was included in the
             program only at the CE-CERT on road validation stage. Improvement idea is
      	to run the on-road validation earlier to provide input in developing the model.
       21.    Documents were shared via SwRI FTP website.  Improvement idea is to use a
             more standardized website layout.  Need better data organization for online
      	data storage area.	
       22.     1065 has some gray areas with respect to PEMS performance checks. In the
      	process 1065 checks were prototyped and revisions were adapted.	
       23.     Open issue on how to get PEMS to eventually PASS ALL 1065 checks.
SwRI Report 03.12024.06                 Page 371 of 371

-------
             Example: linearity check failures.
       24.    1065 was not complete prior to start of gaseous programs. Improvement idea
             is to  expect  PEMS supplier  to have run  and passed 1065 checks before
      	starting next program.	
SwRI Report 03.12024.06                 Page 372 of 371

-------
                                APPENDIX A
                       SEMTECH-DS OPERATION LOG
SwRI Report 03.12024.06

-------
SwRI Semtech PEMS Log
Date
1/4/2006




1/9/06 -
1/27/06




1/12/06-
2/15/06




1/15/2006



2/7/2006



2/15/06-
5/24/06




3/30/2006



4/3/2006



PEMS#
1,2,3,4




5" EFM




1




1,2,3,4



2



1,2,3,4




4



4



Summary
High CO
reading




1065 flow
measurement
linearity failure


1065 NO
linearity
failure



1065 N02
linearity
failure


1065 FID 02
interference
failure


1065 NO2
penetration
failure



Spikes in EFM
data



Unstable NO2
PEMS would
not
power-up

Problem
Occurred
During
Audit




Audit




Audit




Audit



Audit



Audit




Test



Test



Description
Symptoms
Problem

Solution

Symptoms
Problem
Solution


Symptoms
Problem
Solution


Symptoms
Problem
Solution

Symptoms
Problem
Solution

Symptoms
Problem
Solution


Symptoms
Problem
Solution

Symptoms

Problem
Solution
* CO reads 20-40 ppm when zero air is
sampled after a zero/span procedure
* Possibly caused by change in sample
pressure
* Different sample path for zero/span and
sample
* None

* 5-inch EFM repeatedly failed 1065 linearity
criteria
* Incorrect or outdated curve constants
* Sent flow meter to Sensors for recalibration
* Performed linearity check - passed

* NO repeatedly failed 1065 linearity - high
intercept, low slope
* Incorrect or outdated curve constants
* Sent PEMS 1 to Sensors for recalibration of
the NDUV
* Performed linearity check - passed

* NO2 repeatedly failed 1065 linearity - high
intercept, low slope
* Incorrect or outdated curve constants
* None taken

* FID repeatedly failed 1065.362 non-
stoichiometric FID O2 interference
* FID not tuned/optimized
* None taken

* PEMS repeatedly failed 1065.376 Chiller
NO2 Penetration
* Loss of NO2 in sample conditioning system
* Sensors filter bowl and chiller drain manifold
retrofit with RH/Temp sensor
* Correction Factor implemented by Sensors
using RH sensor

* EFM data had short spikes of excessive flow
* Unknown, may be linked to inverter problem
below (4/3/06)
* None

* Unstable and erratic NO2 readings
* Unit eventually would not turn on
* Inverter was supplying only 6 volts to the
PEMS
* Replaced inverter
SwRI Report 03.12024.06
A-l

-------
6/5/2006





6/5/2006



6/6/2006




6/6/2006




6/7/2006



6/7/2006



6/8/2006



6/9/2006



6/19/2006




7/12/06 -
8/2/06






3





3



5




1




5



6



6



4



1




4" EFMs






NDUV failure





1065 NO
linearity
failure


NDIR failure




NDUV failure




1065 NO
linearity
failure


1065 NO2
linearity
failure


1065 FID O2
interference
failure


NO and NO2
Unstable


Vehicle
Interface
not responding



4" EFMs failed
1065
linearity





Test





Audit



Test




Test




Audit



Audit



Audit



Test



Test




Audit






Symptoms

Problem
Solution


Symptoms
Problem
Solution

Symptoms
Problem
Solution


Symptoms
Problem
Solution


Symptoms
Problem
Solution

Symptoms
Problem
Solution

Symptoms
Problem
Solution

Symptoms
Problem
Solution

Symptoms

Problem
Solution

Symptoms


Problem
Solution


* NO and NO2 readings unstable and noisy
* NO and NO2 gains high or saturated
* NDUV lamp malfunction (Carl Ensfield)
* Replaced NDUV
* Performed linearity and NO2 penetration
check (NO linearity failed)

* NO repeatedly failed 1 065 linearity (high
intercept)
* Incorrect or outdated curve constants
* None taken

* CO and CO2 readings erratic and noisy
* NDIR failure
* Sensors replaced NDIR
* Performed linearity checks - passed

* Fault - NDUV not responding
* NDUV communication error
* Replaced NDUV
* Performed linearity and NO2 penetration
check - passed

* NO repeatedly failed 1 065 linearity (high
intercept)
* Incorrect or outdated curve constants
* None taken

* NO2 repeatedly failed 1065 linearity (high
intercept)
* Incorrect or outdated curve constants
* None taken

* FID repeatedly failed 1065.362 non-
stoichiometric FID O2 interference
* FID not tuned/optimized
* None taken

* NO and NO2 were unstable/noisy when
attempting to zero/span
* Unknown - possible NDUV lamp malfunction
* Restarted unit

* Fault - "Vehicle Interface not responding"
* Could not link to CAN bus
* Dearborn Group adapter not functioning
* Replaced Dearborn Group adapter

* 4-inch EFMs failed 1065 linearity check
compared to SwRI flow stand
* EFMs failed slope - low
* 4-inch EFMs showed similar biases
compared to SwRI engine exh. data
* Incorrect or outdated curve data
* Formulated new curve data based on SwRI
flow stand measurements
* Entered new linearization data for the flow
meters
* Performed linearity checks - passed
SwRI Report 03.12024.06
A-2

-------
7/14/2006





7/20/2006










8/8/2006








8/10/2006







2,5





6










2,3,5








6







24-volts
delivered to
digital input




FID failure










Low THC span








FID failure







Setup





Test










Test








Test







Notes

Symptoms
Problem
Solution

Symptoms




Problem
Solution




Symptoms


Problem


Solution


Symptoms


Problem
Solution



* SwRI connection error - 24 volts delivered to
digital input 1
* Aux 2 connector, pin 14 - digital input 1 was
delivered 24 volts
* Units shut down and would not restart
* Damaged I/O board in one unit, damaged
several boards in other unit
* Units sent to Sensors for repair - board
replacement

* FID temperature increases slowly
* FID does not reach operating temperature
* FID does not zero/span
* FID fault - "No zero/span performed, basic
data installed"
* FID fault - "FID Battery Backed Ram
Corrupt"
* FID block heater not functioning
* Sensors rebuilt FID at SwRI
* Replaced block heater
* FID appeared dirty and well used (Louciano)
* Performed linearity check - passed

* FIDs would not span to correct value (FIDs
would span low)
* FID digital gains saturated
* Adjustment of FID gain potentiometers did
not solve problem
* Excessive backpressure on FID drain
* Backpressure due to test setup -
long/restrictive drain lines
* FID drains and main drains were combined
* Rerouted drain lines, keeping FID and main
drains separate
* Used large 3" flexible hose to decrease drain
backpressure

* FID would not span to correct value (FID
would span low)
* FID digital gain saturated
* Adjustment of FID gain potentiometer did not
solve problem
* FID capillary tube obstructed
* Sensors rebuilt FID at SwRI
* Replaced FID capillary tube
* Problem likely caused by prior FID rebuild
(7/20/06)
* Performed linearity check - passed
SwRI Report 03.12024.06
A-3

-------
8/15/2006







8/22/2006






8/28/2006





9/6/2006








9/8/2006



2,3,5







2,3






5





3








3



CO span error







NO2
conversion/los
s






Temperature
probe
failure




NO2
conversion/los
s








Enclosure
wiring
problem


Test







Test






Test





Test








Test



Symptoms

Problem

Solution



Symptoms


Problem
Solution


Symptoms

Problem
Solution


Symptoms


Problem
Solution




Symptoms

Problem
Solution
* CO unstable and noisy during zero and span
operation
* CO readings from zero to several thousand
ppm during span
* CO digital gains saturated
* Likely caused from zeroing with span gas or
spanning with zero air
* Repeatedly zeroed and spanned CO
analyzers
* Instability subsided after first zero/span
operation, reading higher than span
* CO readings were correct after second
zero/span operation

* NO2 showed a significant drop during
environmental baseline testing
* NO2 span decreased during fist hour of
testing, then began to recover
* When NO2 decreased, NO increased -
possible conversion of NO2 to NO
* Problem with PEMS chillers - possible
deterioration of passivated coating
* Replaced chillers in PEMS 2 and 3
* Tested PEMS, PEMS 2 - no NO2 loss,
PEMS 3 - NO2 loss still prevalent

* Ambient temperature probe generating
unrealistic temperature data
* PEMS temperature data 1 1 degC below
SwRI measurement
* Temperature/humidity probe failure
* Replace temperature/humidity probe


* NO2 loss still prevalent with replacement
chiller from Sensors (8/22/06)
* NO2 span decreased during fist hour of
testing, then began to recover
* When NO2 decreased, NO increased -
possible conversion of NO2 to NO
* Problem with PEMS chillers - possible
deterioration of passivated coating
* Replaced chiller in PEMS 3 (replacement
#2)
* Tested PEMS, PEMS 3 - no NO2 loss
* Repeated baseline environmental testing on
PEMS 2 and 3
* Repeated temperature testing on PEMS 3

* Unit turned on and shut off after several
seconds when in the enclosure
* Unit eventually would not turn on when in the
enclosure
* 12-volt power cable in the enclosure had a
bad connection
* Repaired cable connection
SwRI Report 03.12024.06
A-4

-------
9/11/2006






9/14/2006






9/14/2006









9/15/2006



9/19/2006




9/20/2006






4






6






4









4



6




2






Manifold
RH/Temp
failure





Manifold
RH/Temp
failure and
communication
error



NDUV failure









1065 NO2
linearity
failure


Manifold
RH/Temp
failure



Drain pump
failure






Test






Test






Test









Audit



Test




Test






Symptoms


Problem
Solution


Symptoms

Problem
Solution



Symptoms




Problem

Solution


Symptoms
Problem
Solution

Symptoms

Problem
Solution

Symptoms



Problem

Solution
* Fault - "Manifold RH/Temp sensor not
responding"
* Manifold RH reading absent
* Fault cleared, then RH reading was erratic,
jumping between 60 and 100%
* Manifold RH/Temp sensor corroded
* Replaced manifold RH/Temp sensor
* Performed NO2 penetration check - passed

* Fault - "Manifold RH/Temp sensor not
responding"
* Manifold RH reading absent
* Manifold RH/Temp sensor wet
* Replaced manifold RH/Temp sensor
* New sensor would not
respond/communicate
* Reloaded firmware to reinitialize settings

* NO and NO2 readings lower than expected,
often zero
* NO and NO2 readings higher than expected
* NO and NO2 readings erratic and noisy
* NO and NO2 measurements have excessive
drift
* NO and NO2 gains high or saturated
* NDUV lamp malfunction (Carl Ensfield)
* NDUV gains were saturated (Louie Moret)
* Replaced NDUV
* Performed linearity and NO2 penetration
check - NO2 failed

* NO2 repeatedly failed 1065 linearity (low
intercept, high slope)
* Incorrect or outdated curve constants
* None taken

* Fault - "Manifold RH/Temp sensor not
responding"
* Manifold RH reading absent
* Manifold RH/Temp sensor wet
* Replaced manifold RH/Temp sensor

* Warning - "Low vacuum drain 2"
* Pressure #3 (Filter bowl drain) vacuum low
(near ambient pressure)
* Pressure #3 would not draw vacuum when
sample line disconnected
* Unit would not pass a leak check
* Drain pump not operating when checked
* Drain pump started to rotate, but would stop
under vacuum
* Replaced drain pump with PEMS 6 drain
pump
SwRI Report 03.12024.06
A-5

-------
9/20/2006



9/20/2006





9/21/2006








9/25/2006








9/27/2006








2



6





6








4








4,6








Drain pump
mounting
bracket broken


EFM data not
reported
in post-
processed file




(completion of
Fix from 9/20)



Vehicle
Interface
not responding



NDUV failure








Manifold
RH/Temp
failure







Test



Test





Audit



Test




Test








Test








Symptoms
Problem
Solution

Symptoms

Problem
Solution


Notes



Symptoms

Problem
Solution

Symptoms




Problem
Solution


Symptoms

Problem

Solution




* Aluminum drain pump mounting bracket
broken
* Mounting bracket cracked
* Welded bracket and reinstalled in PEMS

* EFM flow data was not reported in 4 SS post
processed files
* EFM flow data was in data section of the
.xml files
* EFM description data missing from header
section of the .xml files
* Added EFM description data to the header
section of the .xml file
* Reprocessed the data files

* Installed new drain pump in PEMS 6
* Performed leak check
* Performed NO2 penetration check (new RH
sensor and drain pump)

* Fault - "Vehicle Interface not responding"
* Could not link to CAN bus
* Dearborn Group adapter not functioning
* Replaced Dearborn Group adapter

* NO and NO2 readings lower than expected,
often zero
* NO and NO2 readings higher than expected
* NO and NO2 readings erratic and noisy
* NO and NO2 measurements have excessive
drift
* NO and NO2 gains high or saturated
* NDUV lamp malfunction (Carl Ensfield)
* Replaced NDUV
* Performed linearity and NO2 penetration
check - passed

* Fault - "Manifold RH/Temp sensor not
responding"
* Manifold RH reading absent, not updating, or
erratic
* Manifold RH/Temp sensor wet
* Manifold not sealed properly - air/water
leaking past sensor
* Disassembled new drain manifolds
* Resealed manifolds with Silicon
* Sealed manifolds around sensor connectors
* Performed leak checks on the manifolds -
passed
* Performed NO2 penetration checks - passed
SwRI Report 03.12024.06
A-6

-------
9/28/06 -
10/3/06







10/5/2006















10/11/2006





10/18/2006




3" EFMs







3















2





2




3" EFMs failed
1065
linearity






Vibration
failures

Problem 1



Problem 2









EFM data not
reported
in post-
processed file




FID battery
backed ram
corrupt



Audit







Test















Test





Test




Symptoms


Problem
Solution



Notes

Symptoms

Problem
Solution
Symptoms





Problem
Solution


Symptoms

Problem
Solution


Symptoms

Problem
Solution

* 2 3-inch EFMs failed 1065 linearity check
compared to SwRI flow stand
* EFMs failed slope
* One EFM close (0.95), one bad (.90)
* Incorrect or outdated curve data
* Replaced one bad EFM
* Entered new linearization data for both flow
meters
* Performed linearity checks - both EFMs
passed

* PEMS operated for 4 hours of side-to-side
horizontal vibration
* PEMS operated for 10 minutes of front-to-
back horizontal vibration
* During first hour of vib. testing, enclosure
heated line warning, low temp
* During first hour of vib. testing, enclosure
fans stopped operating
* 12-volt power connection to enclosure failed
on PEMS (soldered wires broke)
* Temporary solution - external 12-volt power
supply to enclosure
* During first 10-min. of side-to-side vib., FID
high temp fault
* FID fault - FID temperature exceeded limits -
reading 219 degF
* Unit was shut down - took approximately 3
minutes to restart
* Could not communicate with the compact
flash card
* Can not log onto unit without flash card
communication
* Some data from previous tests were lost
* Flash card ribbon cable failure (not known
prior to sending unit to Sensors)
* Unit sent to Sensors for repair
* Installed new flash card ribbon cable

* EFM flow data was not reported in EFM
linearity test
* EFM flow data was in data section of the
.xml files
* EFM description data missing from header
section of the .xml files
* Added EFM description data to the header
section of the .xml file
* Reprocessed the data files

* FID fault - "FID battery backed ram corrupt"
* Fault would not clear
* Unknown
* Restarting unit has cleared this fault in the
past
* After restart, lost communication with the
unit (see Communication error 10/18/06)
SwRI Report 03.12024.06
A-7

-------
10/18/2006








10/19/2006












2








3












Communication
and/or
processor
failure







Bulk Current
Injection
(BCI) FID
Failure











Test








Test












Symptoms




Problem
Solution


Symptoms







Problem
Solution



* Could not log onto PEMS - "Connection
Failed"
* CPU Status LED on, but not blinking
* Could not turn unit off with the power button
on the PEMS (had to unplug unit)
* Restarted unit several times, still no
communication
* Installed new flash card, still no
communication
* Corrupt flash card (Sensors Inc.)
* Sent PEMS 2 to Sensors Inc. for repair
* Sensors Inc. replaced flash card,
communication restored

* BCI to the 12-volt PEMS power cable
(Standard J 111 3-4 Class B, Region 2)
* 60 milliamps, 26-46 MHz
* Caused erratic readings for the FID oven
temperature, FID fuel pressure, and FID
internal
reference pressure
* Fault - "FID oven temperature out of limits"
* Fault - "FID internal reference pressure out
of limits"
* Warning - "Warming Up"
* High FID oven temperature reading caused
the FID to shutdown
* PEMS susceptibility to BCI, FID susceptibility
to BCI
* Replaced PEMS 3 with PEMS 7
* Sent PEMS 3 to Sensors for repair
* PEMS 7 did not turn off the FID when tested
over similar conditions
* PEMS 7 reported similar faults and
warnings, but to a lesser extent than PEMS 3
SwRI Report 03.12024.06
A-8

-------
10/23/2006






















10/23/2006




7






















7




Bulk Current
Injection
(BCI) Failures





















EFM data not
reported
in post-
processed file



Test






















Test




Testl

Solution
Test 2

Solution
Test3


Solution
Test 5a


Solution

Test 7

Solution

Test 9

Solution

Symptoms

Problem
Solution

* BCI to power supply cable
* Lost communication with EFM
* Turned EFM off and back on,
communication restored
* BCI to power supply cable
* Fault - "FID internal ref pressure out of
limits" (96 MHz)
* Cleared fault
* BCI to Aux 1 cable
* Fault - "FID oven temp out of range" (91
MHz)
* Warning - "Warming Up" (on briefly)
* Cleared fault
* BCI to Ethernet cable
* Fault - "FID internal ref pressure out of
limits" (80 MHz)
* Lost communication with the PEMS (96
MHz)
* Cleared fault
* Log back onto PEMS, continued test
* Fault - "FID internal ref pressure out of
limits" (96 MHz)
* Fault - "FID oven temp out of range" (360
MHz)
* Faults would not clear while BCI was active
* Stopped BCI, cleared faults and proceeded
with testing
* BCI to EFM cable at EFM box
* EFM not responding (prior to 170 MHz)
* Restarted PEMS and software to restore
communication with the EFM

* EFM flow data was not reported in BCI test
* EFM flow data was in data section of the
.xml files
* EFM description data missing from header
section of the .xml files
* Added EFM description data to the header
section of the .xml file
* Reprocessed the data files
SwRI Report 03.12024.06
A-9

-------
10/25/2006




















11/9/2006





7




















6





Radiated
Immunity
(Rl) Failures



















Manifold
RH/Temp
failure




Test




















CATSS
Testing





Testl











Solution
Test 3

Solution

Test 4

Solution

Symptoms
Problem

Solution


* Warning - "Warming Up" (on briefly) (42
MHz, 25 V/m)
* Warning - "Warming Up" (on briefly) (74
MHz, 25 V/m)
* FID oven temp reading 203 (74 MHz, 25
V/m)
* Fault - "FID oven temp out of range" (76
MHz, 25 V/m) - cleared fault
* Fault - "FID oven temp out of range" (78
MHz, 20 V/m) - cleared fault
* Fault - "FID oven temp out of range" (82
MHz, 15 V/m) -cleared fault
* Fault - "FID internal ref pressure out of
limits" (86 MHz, 15 V/m) - cleared fault
* Fault - "FID oven temp out of range" (128
MHz, ? V/m) - cleared fault
* Fault - "FID internal ref pressure out of
limits" (128 MHz, V/m) - cleared fault
* Warning - "Warming Up" (on briefly) (128
MHz, ? V/m)
* Lost communication with the PEMS (148
MHz, ? V/m) - logged back onto PEMS
* FID shut down, oven temp reading 213 (164
MHz, ? V/m)
* Restart test using lower intensity (10 V/m)
* FID oven temp reading 205 (164 MHz, 10
V/m)
* FID shut down, oven temp reading 215 (168
MHz, 10 V/m) - restarted FID
* Can not operate FID between 168 and 178
MHz at 10 V/m
* Continued testing at 178 MHz and 10 V/m
* FID shut down, oven temp reading 232 (260
MHz, 10 V/m) - restarted PEMS
* Chiller reading 20 degC when PEMS was
restarted - restarted PEMS
* Restarted PEMS and continued testing

* Fault - "Manifold RH/Temp sensor not
responding"
* Manifold RH/Temp sensor wet
* Manifold not sealed properly - air/water
leaking past sensor
* Fault cleared
* Checked RH reading - OK
* Continued testing
SwRI Report 03.12024.06
A-10

-------
11/16/2006







11/16/2006













1/30/2006





1/31/2006




6







3













5





5




Manifold
RH/Temp
failure






Vibration
Failures













Sample Line
Failure





Sample Line
Failure




INT 40-
Point
Testing







Vibration
Testing













CATC15
40-Point

Testing



CATC15
40-Point

Testing


Symptoms

Problem

Solution



Test3

Solution
Test 6

Solution
Test?

Solution
Test 9


Solution

Symptoms

Problem

Solution

Symptoms

Problem

Solution
* Fault - "Manifold RH/Temp sensor not
responding"
* Manifold RH reading absent, not updating, or
erratic
* Manifold RH/Temp sensor wet
* Manifold not sealed properly - air/water
leaking past sensor
* Removed drain manifold with RH sensor
* Performed leak check on new replacement
manifold - Passed
* Performed NO2 penetration checks -
Passed

* Fault - "FID internal ref pressure out of
limits"
* FID bottle low- broken FID bottle quick
connect
* Replaced FID bottle and quick connect
* Fault - "FID gas flow is too high or too low"
* Lost control of the FID
* Turned FID off overnight
* FID slow to come up to operating
temperature
* FID would not span - FID oven at 191 degC
* None taken, continued testing with no THC
measurement
* Heard a "pop" from the PEMS - FID
responding and reporting THC values
* Fault - "FID internal ref pressure out of
limits"
* FID bottle low- broken FID bottle quick
connect
* Replaced FID bottle and quick connect

* Fault - "High vacuum on drains 1 and 2"
* Fault occurred after PEMS 5 was set to
sample ambient air after being in standby
overnight
* Heated sample line blocked
* Cause of blockage unknown
* Replaced sample line

* Fault - "High vacuum on drains 1 and 2"
* Fault occurred after PEMS 5 was set to
sample ambient air after being in standby
overnight
* Heated sample line blocked
* Cause of blockage unknown
* Replaced sample line
SwRI Report 03.12024.06
A-ll

-------
                                APPENDIX B
    BRAKE-SPECIFIC EMISSION CALCULATIONS FOR NOX, CO, AND NMHC
SwRI Report 03.12024.06

-------
The following sections detail the calculation formulas and the required input constants.  For

Methods #1 and #2 the conversion ofexhaust flow rate from SCFM to mol/s is:
                       n.(SCFM)*	IX-]*-
              . (moT\ =           353l467(ft3)  60
                          mm
                                293.15(K)* 8.314472
                                                      J
Brake-Specific MX Concentration for Method #1



Input constants:



At = 1 (sec)



MNO  = 46.0055MM
  N°2          {molj
                       i V
                       z
                          (xNO2 (ppm) + xNO, (ppm)) * 10~6 * n\ —} * At
                               *                             f
Speedl(rpm)*Tl(N-m)*2*3.l4l59*At
                                    60*1000*3600
Brake-Specific CO Concentration for Method #1



Input constants:



At = I (sec)



     = 28.010lM-l
  L co
              mol)
eco(g/kW-hr) = -
                S
                          (jcCO,(%))*10-2 *«/—V At
                                            I  s )
                                       * 2* 3.14159*
                              60*1000*3600
SwRI Report 03.12024.06
             B-l

-------
Brake-Specific NMHC Concentration for Method #1
Input constants:




A? = 1 (sec)
                  M NMHC *
6NMHc(g I kW • hr) = •
     (xNMHC, (ppm)) * 1 (T6 * nt \ — } * A?
                     1\



                     I
Speedi(rpm)*Ti(N-m)*2*3.14159*At
                                   60*1000*3600
Brake- Specific NOx Concentration for Method #2




Input constants:




w fuel  =0.869   Mass fraction of carbon  in the fuel.



At = 1 (sec )



              (
M  =12.0107
              \rnol
   NO~
      = 46.0055
eNO(glkW-hr) = -
                Wfuel
                        M   *
                        1V1N02
                              i=\
         (xNO2 (ppm)+xNO, (ppm))* 10~6 * ni — * A/
                         nt ( — ] * \xNMHCt (ppm) * 1 0~6 + (xCO, (%) + xCO2i (%))* 1 0 2 ]* A?
                      BSFCI   g

                           \kW-hr
Brake-Specific CO Concentration for Method #2
SwRI Report 03.12024.06
               B-2

-------
Input constants:
w 'fuel =0.869   Mass fraction of carbon  in the fuel.

A? = 1  (sec )
Mc =12.0107
              ( mol
Mco  =28.0101
               \rnol
eco(glkW-hr} =
                W
                 fue,
                         n,\— ] * [xNMHC, (ppm)* 10~6 + (xCO,(%) + JcCO2j (%))* 10~2]* A?
        BSFC,\   g
             \kW-hr
Brake-Specific NMHC Concentration for Method #2


Input constants:


wfud =0.869  Massfractionofcarbon inthefuel.

A? = 1 (sec)


M^ = 12.01071-^-1
                 \rnol;
                              MNMHC *
8NMHc(g I kW • hr) = •
                                      i=\
 (xNMHCt (ppm)) * 10"6 * nl — ] * A?
                           n, (— ] * [jc7VM//C, (ppm) * 10~6 + (jcCO;. (%) + JcCO2j (%)) * 1Q-2 ] *,
                                                 BSFCI   g
                                                      \kW-hr
Brake-Specific NOY Concentration for Method #3
Input constants:
SwRI Report 03.12024.06
B-3

-------
w fuel =0.869  Mass fraction of carbon in the fuel.
   NO~
              (  K   \
Mc =12.0107  -2—
              \rnol )



      = 46.0055| -£-|
                \rnol )


At = 1 (sec )


Example  mass fuel mass rate calculatio n :


     (v\           (  1 \       ( v~\
m fuel.  -  = Fuelrate , |	| * 851.0| -2-
                      sec
                                   L
                    32  Wfuel ^^


                    A /      ^-^
eNO(glkW-hr} = -
                                   (xNO2i (ppm) + xNO, (ppm)) * 10~6 * mfuelj (
                                                                      g_

                                                                      s
                                xNMHC, (ppm) * 10~6 + (xCO, (%) + xCO2i (%)) * 10"2
                              IV

                             I
                                 Speedi(rpm)*Ti(N-m)*2*3.\4\59*kt
                                           60*1000*3600
Brake-Specific CO Concentration for Method #3



Input constants:

w'fud =0.869  Massfractionof carbon inthefuel.
Mr =12.010
           1-*-}
            \rnol)
Mco = 28.01 OlM-1
              mol)


At = 1 (sec)


Examplemass fuel mass rate calculation :
m
                    L
 «/»/. -  = Fuelrate, —  * 851.0 -2.

    '  V ;          \ QPP /      \  /
     . J /          \ scv^ y      \ Jj ,
               A/f   ^117
               ivico  wfml
                          ^"
6co(glkW -
'\ *
                              JC7VM//C, (/?p»i) * 1Q-6 + (xCO, (%) + xCO2i (%)) * 10~2
                             7V

                            I
                                          60*1000*3600
SwRI Report 03.12024.06
                                       B-4

-------
Brake-Specific NMHC Concentration for Method #3


Input constants:


^ fuel = 0- 869  Mass fractionof carbon in the fuel.

             (  K\
Mc = 12.0107 ^~
             \rnol)


MtaaK= 13.87538/^-1
                 \rnol)

At = 1 (sec)

Examplemass fuel mass rate calculation :

    r "^           ( I ~\       f?^
                    — *851.d-£
                        * w
                    NMHC  w fuel
CNMHc(g I kW • hr) = •
                                IV
                               I
   (xNMHCt (ppm)) *
g_

s
                                   xNMHC, (ppm) * 10"6 + (xCO, (%) + xCO2j (%)) * 10~2
                 2*3.14159* A/'
                                             60*1000*3600
SwRI Report 03.12024.06
B-5

-------
                              APPENDIX C
               CRYSTAL BALL OUTPUT FILE DESCRIPTIONS
SwRI Report 03.12024.06

-------
                               EXTRACT DATA FILES
1.0    Simulation Variables

     The simulation variables listed in Table 1 were extracted at the completion of the Monte
Carlo simulation run for each reference NTE event. Crystal Ball classifies variables into two
categories: assumptions and forecasts.  Assumptions are the estimated inputs into the simulation
model such as the variability indices used to sample each error surface.  All assumption variables
in this study are identified by an "ic" at the beginning of the variable name.  Forecasts are values
calculated by a forecast formula in the spreadsheet cells. Examples of forecast variables used in
this study are "Full MC Delta NOX Method 1" and "Validation MC Delta CO Method 2".

                        TABLE 1.  SIMULATION VARIABLES
Variable Name
001_DeNOx (g/kW-hr), Method 1_TA_DC
002_Valid DeNOx (g/kW-hr), Method 1_TA_DC
003_DeNOx (g/kW-hr), Method 2_TA_DC
004_Valid DeNOx (g/kW-hr), Method 2_TA_DC
005_DeNOx (g/kW-hr), Method 3_TA_DC
006_Valid DeNOx (g/kW-hr), Method 3_TA_DC
007_DeCO (g/kW-hr), Method 1_TA_DC
008_Valid DeCO (g/kW-hr), Method 1_TA_DC
009_DeCO (g/kW-hr), Method 2_TA_DC
010_Valid DeCO (g/kW-hr), Method 2_TA_DC
Oil DeCO (g/kW-hr), Method 3 TA DC
Description
Full MC Delta NOx Method 1
Time Alignment and Periodic Drift
Check
Validation MC Delta NOx Method
1 Time Alignment and Periodic
Drift Check
Full MC Delta NOx Method 2
Time Alignment and Periodic Drift
Check
Validation MC Delta NOx Method
2 Time Alignment and Periodic
Drift Check
Full MC Delta NOx Method 3
Time Alignment and Periodic Drift
Check
Validation MC Delta NOx Method
3 Time Alignment and Periodic
Drift Check
Full MC Delta CO Method 1 Time
Alignment and Periodic Drift
Check
Validation MC Delta CO Method 1
Time Alignment and Periodic Drift
Check
Full MC Delta CO Method 2 Time
Alignment and Periodic Drift
Check
Validation MC Delta CO Method 2
Time Alignment and Periodic Drift
Check
Full MC Delta CO Method 3 Time
SwRI Report 03.12024.06
C-l

-------

012_Valid DeCO (g/kW-hr), Method 3_TA_DC
013_DeNMHC (g/kW-hr), Method 1_DC
014_Valid DeNMHC (g/kW-hr), Method 1_DC
015_DeNMHC (g/kW-hr), Method 2 DC
016_Valid DeNMHC (g/kW-hr), Method 2 DC
017_DeNMHC (g/kW-hr), Method 3_DC
018_Valid DeNMHC (g/kW-hr), Method 3_DC
019_DeNOx (g/kW-hr), Method 1_DC
020_Valid DeNOx (g/kW-hr), Method 1_DC
021_DeNOx (g/kW-hr), Method 2_DC
022_Valid DeNOx (g/kW-hr), Method 2_DC
023_DeNOx (g/kW-hr), Method 3_DC
024_Valid DeNOx (g/kW-hr), Method 3_DC
025_DeCO (g/kW-hr), Method 1_DC
026_Valid DeCO (g/kW-hr), Method 1_DC
027_DeCO (g/kW-hr), Method 2_DC
028_Valid DeCO (g/kW-hr), Method 2_DC
029_DeCO (g/kW-hr), Method 3_DC
030_Valid DeCO (g/kW-hr), Method 3_DC
031_DeNOx (g/hp-hr), Method 1_TA_DC
032_Valid DeNOx (g/hp-hr), Method 1_TA_DC
Alignment and Periodic Drift
Check
Validation MC Delta CO Method 3
Time Alignment and Periodic Drift
Check
Full MC Delta NMHC Method 1
Periodic Drift Check
Validation MC Delta NMHC
Method 1 Periodic Drift Check
Full MC Delta NMHC Method 2
Periodic Drift Check
Validation MC Delta NMHC
Method 2 Periodic Drift Check
Full MC Delta NMHC Method 3
Periodic Drift Check
Validation MC Delta NMHC
Method 3 Periodic Drift Check
Full MC Delta NOx Method 1
Periodic Drift Check
Validation MC Delta NOx Method
1 Periodic Drift Check
Full MC Delta NOx Method 2
Periodic Drift Check
Validation MC Delta NOx Method
2 Periodic Drift Check
Full MC Delta NOx Method 3
Periodic Drift Check
Validation MC Delta NOx Method
3 Periodic Drift Check
Full MC Delta CO Method 1
Periodic Drift Check
Validation MC Delta CO Method 1
Periodic Drift Check
Full MC Delta CO Method 2
Periodic Drift Check
Validation MC Delta CO Method 2
Periodic Drift Check
Full MC Delta CO Method 3
Periodic Drift Check
Validation MC Delta CO Method 3
Periodic Drift Check
Full MC Delta NOx Method 1
Time Alignment and Periodic Drift
Check
Validation MC Delta NOx Method
1 Time Alignment and Periodic
SwRI Report 03.12024.06
C-2

-------

033_DeNOx (g/hp-hr), Method 2_TA_DC
034_Valid DeNOx (g/hp-hr), Method 2_TA_DC
035_DeNOx (g/hp-hr), Method 3_TA_DC
036_Valid DeNOx (g/hp-hr), Method 3_TA_DC
037_DeCO (g/hp-hr), Method 1_TA_DC
038_ValidDeCO (g/hp-hr), Method 1_TA_DC
039_DeCO (g/hp-hr), Method 2_TA_DC
040_Valid DeCO (g/hp-hr), Method 2_TA_DC
041_DeCO (g/hp-hr), Method 3_TA_DC
042_Valid DeCO (g/hp-hr), Method 3_TA_DC
043_DeNMHC (g/hp-hr), Method 1_DC
044_Valid DeNMHC (g/hp-hr), Method 1_DC
045_DeNMHC (g/hp-hr), Method 2_DC
046_Valid DeNMHC (g/hp-hr), Method 2_DC
047_DeNMHC (g/hp-hr), Method 3_DC
048_Valid DeNMHC (g/hp-hr), Method 3_DC
049_DeNOx (g/hp-hr), Method 1_DC
050 Valid DeNOx (g/hp-hr), Method 1 DC
Drift Check
Full MC Delta NOx Method 2
Time Alignment and Periodic Drift
Check
Validation MC Delta NOx Method
2 Time Alignment and Periodic
Drift Check
Full MC Delta NOx Method 3
Time Alignment and Periodic Drift
Check
Validation MC Delta NOx Method
3 Time Alignment and Periodic
Drift Check
Full MC Delta CO Method 1 Time
Alignment and Periodic Drift
Check
Validation MC Delta CO Method 1
Time Alignment and Periodic Drift
Check
Full MC Delta CO Method 2 Time
Alignment and Periodic Drift
Check
Validation MC Delta CO Method 2
Time Alignment and Periodic Drift
Check
Full MC Delta CO Method 3 Time
Alignment and Periodic Drift
Check
Validation MC Delta CO Method 3
Time Alignment and Periodic Drift
Check
Full MC Delta NMHC Method 1
Periodic Drift Check
Validation MC Delta NMHC
Method 1 Periodic Drift Check
Full MC Delta NMHC Method 2
Periodic Drift Check
Validation MC Delta NMHC
Method 2 Periodic Drift Check
Full MC Delta NMHC Method 3
Periodic Drift Check
Validation MC Delta NMHC
Method 3 Periodic Drift Check
Full MC Delta NOx Method 1
Periodic Drift Check
Validation MC Delta NOx Method
SwRI Report 03.12024.06
C-3

-------

051_DeNOx (g/hp-hr), Method 2 DC
052_Valid DeNOx (g/hp-hr), Method 2 DC
053_DeNOx (g/hp-hr), Method 3_DC
054_Valid DeNOx (g/hp-hr), Method 3_DC
055_DeCO (g/hp-hr), Method 1_DC
056_Valid DeCO (g/hp-hr), Method 1_DC
057_DeCO (g/hp-hr), Method 2_DC
058_Valid DeCO (g/hp-hr), Method 2_DC
059_DeCO (g/hp-hr), Method 3_DC
060_Valid DeCO (g/hp-hr), Method 3_DC
061_DeNOx (g/kW-hr), Method 1_TA
062_DeCO (g/kW-hr), Method 1_TA
063 DeNMHC (g/kW-hr), Method 1
064_DeNOx (g/kW-hr), Method 2_TA
065_DeCO (g/kW-hr), Method 2_TA
066_DeNMHC (g/kW-hr), Method 2
067_DeNOx (g/kW-hr), Method 3_TA
068_DeCO (g/kW-hr), Method 3_TA
069 DeNMHC (g/kW-hr), Method 3
070 DeNOx (g/kW-hr), Method 1
071 DeCO (g/kW-hr), Method 1
072 DeNOx (g/kW-hr), Method 2
073 DeCO (g/kW-hr), Method 2
074 DeNOx (g/kW-hr), Method 3
075 DeCO (g/kW-hr), Method 3
076_DeNOx (g/hp-hr), Method 1_TA
1 Periodic Drift Check
Full MC Delta NOx Method 2
Periodic Drift Check
Validation MC Delta NOx Method
2 Periodic Drift Check
Full MC Delta NOx Method 3
Periodic Drift Check
Validation MC Delta NOx Method
3 Periodic Drift Check
Full MC Delta CO Method 1
Periodic Drift Check
Validation MC Delta CO Method 1
Periodic Drift Check
Full MC Delta CO Method 2
Periodic Drift Check
Validation MC Delta CO Method 2
Periodic Drift Check
Full MC Delta CO Method 3
Periodic Drift Check
Validation MC Delta CO Method 3
Periodic Drift Check
Full MC Delta NOx Method 1
Time Alignment
Full MC Delta CO Method 1 Time
Alignment
Full MC Delta NMHC Method 1
Full MC Delta NOx Method 2
Time Alignment
Full MC Delta CO Method 2 Time
Alignment
Full MC Delta NMHC Method 2
Time Alignment
Full MC Delta NOx Method 3
Time Alignment
Full MC Delta CO Method 3 Time
Alignment
Full MC Delta NMHC Method 3
Full MC Delta NOx Method 1
Full MC Delta CO Method 1
Full MC Delta NOx Method 2
Full MC Delta CO Method 2
Full MC Delta NOx Method 3
Full MC Delta CO Method 3
Full MC Delta NOx Method 1
Time Alignment
SwRI Report 03.12024.06
C-4

-------
077_DeCO (g/hp-hr), Method 1_TA
078 DeNMHC (g/hp-hr), Method 1
079_DeNOx (g/hp-hr), Method 2_TA
080_DeCO (g/hp-hr), Method 2_TA
081 DeNMHC (g/hp-hr), Method 2
082_DeNOx (g/hp-hr), Method 3_TA
083_DeCO (g/hp-hr), Method 3_TA
084 DeNMHC (g/hp-hr), Method 3
085 DeNOx (g/hp-hr), Method 1
086 DeCO (g/hp-hr), Method 1
087 DeNOx (g/hp-hr), Method 2
088 DeCO (g/hp-hr), Method 2
089 DeNOx (g/hp-hr), Method 3
090 DeCO (g/hp-hr), Method 3
091_Valid DeNOx (g/kW-hr), Method 1_TA
092_Valid DeCO (g/kW-hr), Method 1_TA
093_Valid DeNMHC (g/kW-hr), Method 1
094_Valid DeNOx (g/kW-hr), Method 2_TA
095_Valid DeCO (g/kW-hr), Method 2_TA
096_Valid DeNMHC (g/kW-hr), Method 2
097_Valid DeNOx (g/kW-hr), Method 3_TA
098_Valid DeCO (g/kW-hr), Method 3_TA
099_Valid DeNMHC (g/kW-hr), Method 3
100_Valid DeNOx (g/kW-hr), Method 1
101 Valid DeCO (g/kW-hr), Method 1
102_Valid DeNOx (g/kW-hr), Method 2
103 Valid DeCO (g/kW-hr), Method 2
104_Valid DeNOx (g/kW-hr), Method 3
Full MC Delta CO Method 1 Time
Alignment
Full MC Delta NMHC Method 1
Full MC Delta NOx Method 2
Time Alignment
Full MC Delta CO Method 2 Time
Alignment
Full MC Delta NMHC Method 2
Full MC Delta NOx Method 3
Time Alignment
Full MC Delta CO Method 3 Time
Alignment
Full MC Delta NMHC Method 3
Full MC Delta NOx Method 1
Full MC Delta CO Method 1
Full MC Delta NOx Method 2
Full MC Delta CO Method 2
Full MC Delta NOx Method 3
Full MC Delta CO Method 3
Validation MC Delta NOx Method
1 Time Alignment
Validation MC Delta CO Method 1
Time Alignment
Validation MC Delta NMHC
Method 1
Validation MC Delta NOx Method
2 Time Alignment
Validation MC Delta CO Method 2
Time Alignment
Validation MC Delta NMHC
Method 2
Validation MC Delta NOx Method
3 Time Alignment
Validation MC Delta CO Method 3
Time Alignment
Validation MC Delta NMHC
Method 3
Validation MC Delta NOx Method
1
Validation MC Delta CO Method 1
Validation MC Delta NOx Method
2
Validation MC Delta CO Method 2
Validation MC Delta NOx Method
3
SwRI Report 03.12024.06
C-5

-------
105 Valid DeCO (g/kW-hr), Method 3
106_Valid DeNOx (g/hp-hr), Method 1_TA
107_Valid DeCO (g/hp-hr), Method 1_TA
108_Valid DeNMHC (g/hp-hr), Method 1
109_Valid DeNOx (g/hp-hr), Method 2_TA
1 10_Valid DeCO (g/hp-hr), Method 2_TA
1 1 l_Valid DeNMHC (g/hp-hr), Method 2
1 12_Valid DeNOx (g/hp-hr), Method 3_TA
1 13_Valid DeCO (g/hp-hr), Method 3_TA
1 14_Valid DeNMHC (g/hp-hr), Method 3
1 15_Valid DeNOx (g/hp-hr), Method 1
116 Valid DeCO (g/hp-hr), Method 1
1 17_Valid DeNOx (g/hp-hr), Method 2
118 Valid DeCO (g/hp-hr), Method 2
1 19_Valid DeNOx (g/hp-hr), Method 3
120 Valid DeCO (g/hp-hr), Method 3
121_eNOx (g/kW-hr), Method 1 Mode 2_TA
122_eCO (g/kW-hr), Method 1 Mode 2_TA
123_eNMHC (g/kW-hr), Method 1 Mode 2
124_eNOx (g/kW-hr), Method 2 Mode 2_TA
125_eCO (g/kW-hr), Method 2 Mode 2_TA
126_eNMHC (g/kW-hr), Method 2 Mode 2
127_eNOx (g/kW-hr), Method 3 Mode 2_TA
128_eCO (g/kW-hr), Method 3 Mode 2_TA
129_eNMHC (g/kW-hr), Method 3 Mode 2
Validation MC Delta CO Method 3
Validation MC Delta NOx Method
1 Time Alignment
Validation MC Delta CO Method 1
Time Alignment
Validation MC Delta NMHC
Method 1
Validation MC Delta NOx Method
2 Time Alignment
Validation MC Delta CO Method 2
Time Alignment
Validation MC Delta NMHC
Method 2 Time Alignment
Validation MC Delta NOx Method
3 Time Alignment
Validation MC Delta CO Method 3
Time Alignment
Validation MC Delta NMHC
Method 3
Validation MC Delta NOx Method
1
Validation MC Delta CO Method 1
Validation MC Delta NOx Method
2
Validation MC Delta CO Method 2
Validation MC Delta NOx Method
3
Validation MC Delta CO Method 3
Full MC BSNOx "with errors"
Method 1 and Time Alignment
Full MC BSCO "with errors"
Method 1 and Time Alignment
Full MC BSNMHC "with errors"
Method 1
Full MC BSNOx "with errors"
Method 2 and Time Alignment
Full MC BSCO "with errors"
Method 2 and Time Alignment
Full MC BSNMHC "with errors"
Method 2
Full MC BSNOx "with errors"
Method 3 and Time Alignment
Full MC BSCO "with errors"
Method 3 and Time Alignment
Full MC BSNMHC "with errors"
Method 3
SwRI Report 03.12024.06
C-6

-------
130_eNOx (g/kW-hr), Method 1 Mode 2
131_eCO (g/kW-hr), Method 1 Mode 2
132_eNOx (g/kW-hr), Method 2 Mode 2
133_eCO (g/kW-hr), Method 2 Mode 2
134_eNOx (g/kW-hr), Method 3 Mode 2
135_eCO (g/kW-hr), Method 3 Mode 2
136_eNOx (g/kW-hr), Method 1 Mode 1
137_eCO (g/kW-hr), Method 1 Mode 1
138_eNMHC (g/kW-hr), Method 1 Mode 1
139_eNOx (g/kW-hr), Method 2 Mode 1
140_eCO (g/kW-hr), Method 2 Mode 1
141_eNMHC (g/kW-hr), Method 2 Mode 1
142_eNOx (g/kW-hr), Method 3 Mode 1
143_eCO (g/kW-hr), Method 3 Mode 1
144_eNMHC (g/kW-hr), Method 3 Mode 1
145 eNOx (g/kW-hr), Method 1 Mode 0
146 eCO (g/kW-hr), Method 1 Mode 0
147_eNMHC (g/kW-hr), Method 1 Mode 0
148 eNOx (g/kW-hr), Method 2 Mode 0
149 eCO (g/kW-hr), Method 2 Mode 0
150_eNMHC (g/kW-hr), Method 2 Mode 0
151 eNOx (g/kW-hr), Method 3 Mode 0
152 eCO (g/kW-hr), Method 3 Mode 0
153_eNMHC (g/kW-hr), Method 3 Mode 0
154_eNOx (g/kW-hr), Method 1 Reject Flag
155 eCO (g/kW-hr), Method 1 Reject Flag
Full MC BSNOx "with errors"
Method 1
Full MC BSCO "with errors"
Method 1
Full MC BSNOx "with errors"
Method 2
Full MC BSCO "with errors"
Method 2
Full MC BSNOx "with errors"
Method 3
Full MC BSCO "with errors"
Method 3
Full MC BSNOx "with errors
except environmental" Method 1
Full MC BSCO "with errors except
environmental" Method 1
Full MC BSNMHC "with errors
except environmental" Method 1
Full MC BSNOx "with errors
except environmental" Method 2
Full MC BSCO "with errors except
environmental" Method 2
Full MC BSNMHC "with errors
except environmental" Method 2
Full MC BSNOx "with errors
except environmental" Method 3
Full MC BSCO "with errors except
environmental" Method 3
Full MC BSNMHC "with errors
except environmental" Method 3
Full MC BSNOx "ideal" Method 1
Full MC BSCO "ideal" Method 1
Full MC BSNMHC "ideal" Method
1
Full MC BSNOx "ideal" Method 2
Full MC BSCO "ideal" Method 2
Full MC BSNMHC "ideal" Method
2
Full MC BSNOx "ideal" Method 3
Full MC BSCO "ideal" Method 3
Full MC BSNMHC "ideal" Method
3
Full MC BSNOx Periodic Drift
Check Flag Method 1
Full MC BSCO Periodic Drift
SwRI Report 03.12024.06
C-7

-------

156_eNMHC (g/kW-hr), Method 1 Reject Flag
157_eNOx (g/kW-hr), Method 2 Reject Flag
158_eCO (g/kW-hr), Method 2 Reject Flag
159_eNMHC (g/kW-hr), Method 2 Reject Flag
160_eNOx (g/kW-hr), Method 3 Reject Flag
161_eCO (g/kW-hr), Method 3 Reject Flag
162_eNMHC (g/kW-hr), Method 3 Reject Flag
163 Valid eNOx (g/kW-hr), Method 1 Mode
2_TA
164_Valid eCO (g/kW-hr), Method 1 Mode 2_TA
165_Valid eNMHC (g/kW-hr), Method 1 Mode 2
166 Valid eNOx (g/kW-hr), Method 2 Mode
2_TA
167_Valid eCO (g/kW-hr), Method 2 Mode 2_TA
168_Valid eNMHC (g/kW-hr), Method 2 Mode 2
169 Valid eNOx (g/kW-hr), Method 3 Mode
2_TA
170_Valid eCO (g/kW-hr), Method 3 Mode 2_TA
171_Valid eNMHC (g/kW-hr), Method 3 Mode 2
172_Valid eNOx (g/kW-hr), Method 1 Mode 2
173_Valid eCO (g/kW-hr), Method 1 Mode 2
174_Valid eNOx (g/kW-hr), Method 2 Mode 2
175_Valid eCO (g/kW-hr), Method 2 Mode 2
176_Valid eNOx (g/kW-hr), Method 3 Mode 2
Check Flag Method 1
Full MC BSNMHC Periodic Drift
Check Flag Method 1
Full MC BSNOx Periodic Drift
Check Flag Method 2
Full MC BSCO Periodic Drift
Check Flag Method 2
Full MC BSNMHC Periodic Drift
Check Flag Method 2
Full MC BSNOx Periodic Drift
Check Flag Method 3
Full MC BSCO Periodic Drift
Check Flag Method 3
Full MC BSNMHC Periodic Drift
Check Flag Method 3
Validation MC BSNOx "with
errors" Method 1 and Time
Alignment
Validation MC BSCO "with errors"
Method 1 and Time Alignment
Validation MC BSNMHC "with
errors" Method 1
Validation MC BSNOx "with
errors" Method 2 and Time
Alignment
Validation MC BSCO "with errors"
Method 2 and Time Alignment
Validation MC BSNMHC "with
errors" Method 2
Validation MC BSNOx "with
errors" Method 3 and Time
Alignment
Validation MC BSCO "with errors"
Method 3 and Time Alignment
Validation MC BSNMHC "with
errors" Method 3
Validation MC BSNOx "with
errors" Method 1
Validation MC BSCO "with errors"
Method 1
Validation MC BSNOx "with
errors" Method 2
Validation MC BSCO "with errors"
Method 2
Validation MC BSNOx "with
errors" Method 3
SwRI Report 03.12024.06

-------
177_Valid eCO (g/kW-hr), Method 3 Mode 2
178_Valid eNOx (g/kW-hr), Method 1 Mode 1
179_Valid eCO (g/kW-hr), Method 1 Mode 1
180_Valid eNMHC (g/kW-hr), Method 1 Mode 1
181_Valid eNOx (g/kW-hr), Method 2 Mode 1
182_Valid eCO (g/kW-hr), Method 2 Mode 1
183_Valid eNMHC (g/kW-hr), Method 2 Mode 1
184_Valid eNOx (g/kW-hr), Method 3 Mode 1
185_Valid eCO (g/kW-hr), Method 3 Mode 1
186_Valid eNMHC (g/kW-hr), Method 3 Mode 1
187_Valid eNOx (g/kW-hr), Method 1 Reject Flag
188_Valid eCO (g/kW-hr), Method 1 Reject Flag
189 Valid eNMHC (g/kW-hr), Method 1 Reject
Flag
190_Valid eNOx (g/kW-hr), Method 2 Reject Flag
191_Valid eCO (g/kW-hr), Method 2 Reject Flag
192 Valid eNMHC (g/kW-hr), Method 2 Reject
Flag
193_Valid eNOx (g/kW-hr), Method 3 Reject Flag
194_Valid eCO (g/kW-hr), Method 3 Reject Flag
195 Valid eNMHC (g/kW-hr), Method 3 Reject
Flag
196_eNOx (g/hp-hr), Method 1 Mode 2_TA
Validation MC BSCO "with errors"
Method 3
Validation MC BSNOx "with
errors except environmental"
Method 1
Validation MC BSCO "with errors
except environmental" Method 1
Validation MC BSNMHC "with
errors except environmental"
Method 1
Validation MC BSNOx "with
errors except environmental"
Method 2
Validation MC BSCO "with errors
except environmental" Method 2
Validation MC BSNMHC "with
errors except environmental"
Method 2
Validation MC BSNOx "with
errors except environmental"
Method 3
Validation MC BSCO "with errors
except environmental" Method 3
Validation MC BSNMHC "with
errors except environmental"
Method 3
Validation MC BSNOx Periodic
Drift Check Flag Method 1
Validation MC BSCO Periodic
Drift Check Flag Method 1
Validation MC BSNMHC Periodic
Drift Check Flag Method 1
Validation MC BSNOx Periodic
Drift Check Flag Method 2
Validation MC BSCO Periodic
Drift Check Flag Method 2
Validation MC BSNMHC Periodic
Drift Check Flag Method 2
Validation MC BSNOx Periodic
Drift Check Flag Method 3
Validation MC BSCO Periodic
Drift Check Flag Method 3
Validation MC BSNMHC Periodic
Drift Check Flag Method 3
Full MC BSNOx "with errors"
Method 1 and Time Alignment
SwRI Report 03.12024.06
C-9

-------
197_eCO (g/hp-hr), Method 1 Mode 2_TA
198_eNMHC (g/hp-hr), Method 1 Mode 2
199_eNOx (g/hp-hr), Method 2 Mode 2_TA
200_eCO (g/hp-hr), Method 2 Mode 2_TA
201_eNMHC (g/hp-hr), Method 2 Mode 2
202_eNOx (g/hp-hr), Method 3 Mode 2_TA
203_eCO (g/hp-hr), Method 3 Mode 2_TA
204_eNMHC (g/hp-hr), Method 3 Mode 2
205_Valid eNOx (g/hp-hr), Method 1 Mode 2_TA
206_Valid eCO (g/hp-hr), Method 1 Mode 2_TA
207_Valid eNMHC (g/hp-hr), Method 1 Mode 2
208_Valid eNOx (g/hp-hr), Method 2 Mode 2_TA
209_Valid eCO (g/hp-hr), Method 2 Mode 2_TA
210_Valid eNMHC (g/hp-hr), Method 2 Mode 2
21 l_Valid eNOx (g/hp-hr), Method 3 Mode 2_TA
212_Valid eCO (g/hp-hr), Method 3 Mode 2_TA
213_Valid eNMHC (g/hp-hr), Method 3 Mode 2
01_ic_SS_NOx
02_ic_TR_NOx
05 ic Temperature NOx
Full MC BSCO "with errors"
Method 1 and Time Alignment
Full MC BSNMHC "with errors"
Method 1
Full MC BSNOx "with errors"
Method 2 and Time Alignment
Full MC BSCO "with errors"
Method 2 and Time Alignment
Full MC BSNMHC "with errors"
Method 2
Full MC BSNOx "with errors"
Method 3 and Time Alignment
Full MC BSCO "with errors"
Method 3 and Time Alignment
Full MC BSNMHC "with errors"
Method 3
Validation MC BSNOx "with
errors" Method 1 and Time
Alignment
Validation MC BSCO "with errors"
Method 1 and Time Alignment
Validation MC BSNMHC "with
errors" Method 1
Validation MC BSNOx "with
errors" Method 2 and Time
Alignment
Validation MC BSCO "with errors"
Method 2 and Time Alignment
Validation MC BSNMHC "with
errors" Method 2
Validation MC BSNOx "with
errors" Method 3 and Time
Alignment
Validation MC BSCO "with errors"
Method 3 and Time Alignment
Validation MC BSNMHC "with
errors" Method 3 and Time
Alignment
Random Sampling Variability
Index for SS NOx Error Surface
Random Sampling Variability
Index for Transient NOx Error
Surface
Random Sampling Variability
Index for NOx Temperature Error
Surface
SwRI Report 03.12024.06
C-10

-------
07_ic_SS_CO
10 ic Pressure CO
11 ic Temperature CO
13_ic_SS_NMHC
14_ic_TR_NMHC
1 6_ic_Pressure_NMHC
1 7_ic_Temperature_NMHC
1 9_ic_NMHC_Ambient
20_ic_SS_flow
21_ic_TR_Flowrate
22 ic Pulsation flow
23_ic_Swirl_flow
25 ic Radiation Exhaust Flow
27 ic Temperature Exhaust Flow
28 ic Pressure Exhaust Flow
29_ic_TR_Torque
30_ic_Torque_DOE
31 ic Torque Warm
32 ic Torque IP
34 ic Torque Interpolation
3 5_ic_Torque_Engine Manufacturers
Random Sampling Variability
Index for SS CO
Random Sampling Variability
Index for CO Pressure
Random Sampling Variability
Index for CO Temperature
Random Sampling Variability
Index for SS NMHC
Random Sampling Variability
Index for Transient NMHC
Random Sampling Variability
Index for NMHC Pressure
Random Sampling Variability
Index for NMHC Temperature
Random Sampling Variability
Index for Ambient NMHC
Random Sampling Variability
Index for SS Exhaust Flow
Random Sampling Variability
Index for Transient Exhaust Flow
Random Sampling Variability
Index for Exhaust Flow Pulsation
Random Sampling Variability
Index for Exhaust Flow Swirl
Random Sampling Variability
Index for Exhaust Flow EMI/RFI
Radiation
Random Sampling Variability
Index for Exhaust Flow
Temperature
Random Sampling Variability
Index for Exhaust Flow Pressure
Random Sampling Variability
Index for Dynamic Torque
Random Sampling Variability
Index for Torque Design of
Experiments Testing
Random Sampling Variability
Index for Torque Warm-up
Random Sampling Variability
Index for Torque Independent
Parameters Humidity and Fuel
Random Sampling Variability
Index for Torque Interpolation
Random Sampling Variability
Index for Torque Engine
SwRI Report 03.12024.06
C-ll

-------

36_ic_TR_BSFC
37_ic_BSFC_DOE
38_ic_BSFC_Warm
39_ic_BSFC_IP
41_ic_BSFC_Interpolation
42 ic BSFC Engine Manufacturers
43_ic_TR_Speed
44_ic_TR_Fuel Rate
45_ic_SS_CO2
46_ic_TR_CO2
49 ic Temperature CO2
5 1 ic NOx Time Alignment
52 ic CO Time Alignment
Manufacturers
Random Sampling Variability
Index for Dynamic BSFC
Random Sampling Variability
Index for BSFC Design of
Experiments
Random Sampling Variability
Index for BSFC Warm-up
Random Sampling Variability
Index for BSFC Independent
Parameters Humidity and Fuel
Random Sampling Variability
Index for BSFC Interpolation
Random Sampling Variability
Index for BSFC Engine
Manufacturers
Random Sampling Variability
Index for Dynamic Speed
Random Sampling Variability
Index for Dynamic Fuel Rate
Random Sampling Variability
Index for SS CO2
Random Sampling Variability
Index for Transient CO2
Random Sampling Variability
Index for CO2 Temperature
Random Sampling Variability
Index for NOx Time Alignment
Random Sampling Variability
Index for CO Time Alignment
2.0   Statistics

     Descriptive statistics summarizing the values obtained during a single reference NTE event
simulation are provided in Table 2.

      TABLE 2. DESCRIPTIVE STATISTICS FOR SIMULATION VARIABLES
Statistic
Trials
Mean
Median
Definition
Number of times the simulation was
repeated and not discarded due to periodic
drift
Arithmetic average
The value midway between the smallest
value and the largest value
SwRI Report 03.12024.06
C-12

-------
Mode
Standard Deviation
Variance
Skewness
Kurtosis
Coefficient of Variability
Minimum
Maximum
Range Width
Mean Standard Error
Filtered Values
Value that occurs most often
Measurement of variability of a
distribution. The square root of the
variance
The average of the squares of the
deviations of a number of values from their
mean
A measure of the degree of deviation of a
distribution from the norm of a symmetric
distribution
A measure of the degree of peakedness of a
distribution
Standard deviation/Mean
Smallest value
Largest value
Largest value - smallest value
Standard deviation of the distribution of
possible sample means
Number of trial discarded due to periodic
drift
3.0    Percentiles

     Percentiles are the probability of achieving values below a particular percentage in the
following increments: 0, 5, 10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 95,
and 100%.  Percentiles are computed for each of the simulations variables described in Section
1.1.

4.0    Sensitivity Data

     Sensitivity data are provided  by computing the rank correlation coefficient for all error
surfaces and all simulation variables.   The EXTRACT data file contains the absolute value of
the rank correlation.

 5.0    Trial Values

     The value for all simulation variables is provided at each trial of the  simulation.
                                    REPORT FILES
1.0    Report Summary
     This section includes the simulation start date and time, stop date and time, number of
trials run, sampling type (Monte Carlo), random seed used, and run statistics.
SwRI Report 03.12024.06
C-13

-------
2.0    Forecasts

     Descriptive statistics, percentiles, and a frequency histogram are provided for forecast
variables  001_DeNOx  (g/kW-hr),  Method  1_TA_DC  though  030_Valid DeCO  (g/kW-hr),
Method 3_DC (see Table 1).

3.0    Assumptions

     Descriptive statistics, percentiles, distribution parameters,  and  a distribution chart  are
provided for assumption variables 01_ic_SS_NOx through 52_ic_CO_Time  Alignment  (see
Table 1).

4.0    Sensitivity Charts

     Sensitivity charts are provided for forecast variables 001_DeNOx (g/kW-hr), Method
1_TA_DC though 030_Valid DeCO  (g/kW-hr), Method  3_DC (see  Table 1).   Crystal Ball
calculates sensitivity by computing rank correlation coefficients between every assumption (error
surface) and forecast (BS emissions and delta BS emissions) while the simulation  is running.
Positive rank  correlations indicate that an increase in  the assumption is associated with an
increase in the forecast.  The larger the absolute value of the rank correlation the stronger the
relationship.

       The sensitivity charts developed during the MC simulation are displayed as 'Contribution
to Variance" charts which  are calculated by  squaring the rank correlation coefficients for all
assumptions used in a particular forecast and then normalizing them to  100%. Figure 1 displays
a sensitivity chart for the delta NOX Method  #1 with time alignment  and periodic  drift check
forecast. The  assumption with the highest contribution to variance (in absolute value) is plotted
at the top of the chart. In this example, as you increase the torque warm-up there is a  decrease in
the delta NOX Method #1 values.  Only the top nine assumptions are  plotted in the sensitivity
charts.
SwRI Report 03.12024.06                      C-14

-------
                   Sensitivity 00 1 _DeNOx (g/kW-hr), Method 1_TA_DC
                                           •40.0%    -200%
                                                           0.0%
                                                                   20.0%
                         31 _te_Tor<|ue_W«rm






                             01 _ic_SSJ*Dx
                             20_fc_SS_1tow






                            32JC_TwqueJP






                          3OJc_TocqueJWDe
                     51_te_NGx_I






                             02_te_TR_NOx






                     34Jc_Torque_lrt«rpololion
                                                             93%
           FIGURE 1.  SENSITIVITY CHART FOR DELTA NOX METHOD 1
SwRI Report 03.12024.06
C-15

-------
                              APPENDIX D
              MONTE CARLO SPREADSHEET COMPUTATIONS
SwRI Report 03.12024.06

-------
                      1.0    DESCRIPTION OF ASSUMPTIONS

       The following assumptions were made in running the Monte Carlo model:

       •   Only one reference NTE event can be run at a time through the Monte Carlo
          simulation workbook.

       •   Uniform (1 second in duration) time steps are used in the reference NTE events.

       •   Standard format and engineering units for reference NTE data established for the
          project are observed, and applied to the reference NTE event before the NTE event is
          entered in the Error Model workbook for Monte Carlo simulation.

       •   Any wet - dry matter conversions, if not negligible, have been performed on the
          appropriate reference NTE event values before the reference NTE event was entered
          in the Error Model workbook for Monte Carlo simulation.  No wet - dry conversions
          are performed in the workbook.

       •   Any reference NTE event normalizations to produce similar emissions brake-specific
          results from the three emissions  calculation methods have been appropriately
          performed before the reference NTE event was entered in the Error Model workbook
          for Monte Carlo simulation.  No normalizations  among the three methods are
          performed in the workbook.

       •   Emissions models for three calculation methods  and three emissions are computed
          during one MC simulation run.

       •   Error surface models and supporting data were approved by the Steering Committee.

       •   The error model spreadsheet has been correctly implemented,  and its interaction with
          Monte Carlo tools like Crystal Ball is correctly understood.

       •   Random number generation by a Monte Carlo tool like Crystal Ball is correct.

       •   Convergence of the completed MC simulation was processed and checked outside of
          this workbook using a SAS® computer program.

                        2.0   WORKSHEET DESCRIPTIONS

2.1    Macro Description
       The  Macro  can  be viewed  in the  Excel  spreadsheet with the  menu  selections
Tools>Macros>Macrol>Edit.  The purpose of Macro 1 is to expedite clearing  extra cells below
the reference NTE event in  the Methods worksheet and to delete extra rows in the Delta error
worksheets.  The macro also performs Mode 0  calculations and stores resultant 'ideal  emissions'
values for application in subsequent Monte  Carlo simulation.
SwRI Report 03.12024.06                      D-l

-------
       The user must begin with the starter version of the Excel file which has 300 rows of
equations  in columns X - CF  and  in rows 52 -  351 in the Methods worksheet.  The starter
spreadsheet also has 300  rows of  equations below charts in columns B - F, or B - L, in
applicable Delta worksheets. The user copies the reference NTE event into columns A - V, row
52 and down,  in the Methods worksheet.  It is then necessary to confirm that cell J45 in the
Methods worksheet displays the correct number of rows of the reference NTE event.

       Macro    execution   can   be    accomplished   through   the    menu    selections
Tools>Macros>Macrol>Run. Note that this macro clears cells without  deleting rows in the
Methods worksheet, and deletes rows in the Delta worksheets.  This macro will not work if the
reference NTE event has only one row.  For a reference NTE event with exactly two rows, this
macro will corrupt the second "check" values in  columns B-F type Delta worksheets.  Check
values are not used in the simulation, but are provided as a diagnostic aid.  Apply the macro for
reference NTE events with no more than 300 rows.

       The reader can follow the description of execution that follows by viewing the macro and
observing the comment rows provided throughout the macro text.  In execution, the macro first
reads the contents of J45 in the Methods worksheet.  It uses the number of rows in the reference
NTE event defined by J45 to determine how many rows to clear and delete in the spreadsheet.  It
checks that the number of rows is between 2 and 299, inclusive. It will also execute correctly for
300 rows.

       Next, the macro clears cell contents in columns X - CF below the reference NTE event in
the Methods worksheet.  Note the  macro, as  written,  will not  execute properly  if the starter
spreadsheet has been revised with row insertion or deletion in certain areas of the spreadsheet.
As written, the macro initiates in cell X52, counts down through the NTE Event rows,  and clears
contents in the range from there in column X through cell CF351.

       Next, the macro deletes extra rows below the reference NTE event in Delta worksheet 1.
For Delta worksheet 1 it initiates in  cell B79 and counts down through the rows of the reference
NTE event to the first row to be deleted.   It selects the range of rows from there down through
row  378, deletes the rows, copies some equations and a value to the last  row in the range the
charts use, and returns the cursor to cell F68 leaving the display more or less centered on the
charts in the worksheet.

       Subsequently, the macro performs similar operations in  other Delta worksheets; however,
the initiating cell and final row differ among the worksheets. The Delta worksheets processed in
this way are 1, 2, 5, 7, 8, 10, 11, 16,  17, 20, 21, 22, 23, 29, 30, 36, 37, 43, 44, 45, 46 and 49.

       Following the row deletion operations in the Delta worksheets, or directly  when the
reference  NTE event has 300  rows, the  macro  prepares  for the Mode  0 (ideal emissions)
calculation.  First, in the  Methods worksheet it  copies the equations in row 52, columns X
through CF, to the last row in the reference NTE event.  This clears any errors introduced in the
last row; however, it assumes that row 52 is correct.  The last cell in column AC (At) is cleared
for aesthetics, since the At values are not applied in the model calculations.
SwRI Report 03.12024.06                      D-2

-------
       The Mode 0 calculation is performed by the macro by changing the value in cell A6 of
the Summary worksheet to 0.  Then in the Methods worksheet, the values from  cells CU22
through CU32 are pasted (values only) to cells O22 through O32 where they are referenced by
formulas during Monte Carlo simulation. The macro changes the value of A6 in the Summary
worksheet to 2 in preparation for the Monte Carlo simulation, and moves the cursor to cell CT18
of the Methods worksheet.

       Additional comments regarding the  macro operation are  presented in the following
section descriptions of the model spreadsheet.

2.2    Worksheet 1: ErrorControl

      The ErrorControl worksheet of the Error Model workbook implements 52 logic switch
functions.  The user enters a numerical "1" in  column D in each row corresponding to error
surfaces to be included in  the calculation.  A numerical "0" is applied to error surfaces to be
excluded in the calculation.

      Error surfaces are numbered sequentially 1 through 50, and Time Alignment error surfaces
are designated 51 and 52.   The numbered  error surfaces are defined in columns A - C, and
information pertinent to their usage is presented  in columns E - V of the worksheet.  Column E
displays warning messages  when an unusual value is entered in column D.

       The control  switch elements  in the worksheet are deliberately placed on rows in the
worksheet corresponding to the error  surfaces to expedite equation checking in the Methods
worksheet where the control switch variables are applied in conjunction with error surfaces from
the correspondingly numbered "Delta" worksheets

       The numbered error surfaces and time alignment controls that have been implemented are
defined in the following Table 1.

               TABLE 1. ERROR SURFACES USED IN SIMULATION
SwRI Report 03.12024.06                      D-3

-------
Component
Delta NOX
Delta CO
Delta NMHC
NMHC = 0.98*THC
Delta Exhaust Flow
Delta Torque
Delta BSFC
Delta Speed
Delta Fuel Rate
Delta CO2
Time Alignment
No.
1
2
5
7
10
11
13
14
16
17
19
20
21
22
23
25
27
28
29
30
31
32
34
35
36
37
38
39
41
42
43
44
45
46
49
51
52
Error Surface
Delta NOXSS
Delta NOX Transient
Delta NOX Ambient Temperature
Delta CO SS
Delta CO Atmospheric Pressure
Delta CO Ambient Temperature
Delta NMHC SS
Delta NMHC Transient
Delta NMHC Atmospheric Pressure
Delta NMHC Ambient Temperature
Delta Ambient NMHC
Delta Exhaust Flow SS
Delta Exhaust Flow Transient
Delta Exhaust Flow Pulsation
Delta Exhaust Flow Swirl
Delta Exhaust EMI/RFI
Delta Exhaust Temperature
Delta Exhaust Pressure
Delta Dynamic Torque
Delta Torque DOE Testing
(Interacting Parameters Test)
Delta Torque Warm-up
(Interacting Parameters Test)
Delta Torque Humidity / Fuel
(Independent Parameters Test)
Delta Torque Interpolation
Delta Torque Engine Manufacturers
Delta Dynamic BSFC
Delta BSFC DOE Testing
(Interacting Parameters Test)
Delta BSFC Warm-up
(Interacting Parameters Test)
Delta BSFC Humidity / Fuel
(Independent Parameters Test)
Delta BSFC Interpolation
Delta BSFC Engine Manufacturers
Delta Dynamic Speed
Delta Dynamic Fuel Rate
Delta CO2 SS
Delta CO2 Transient
Delta CO2 Ambient Temperature
Delta NOX Time Alignment
Delta CO Time Alignment
SwRI Report 03.12024.06
D-4

-------
       The thirty-five (35) error surfaces that have been implemented are included or excluded
by the controls numbered 1-49 identified in Table  1.  Three NOX time alignment errors are
controlled by number 51 and three CO time alignment errors are controlled by number 52.
When all 35 error controls and both time alignment controls are on (included in calculation), the
sum of column D in the worksheet ErrorControl is 37.

2.3    Worksheet 2: Summary

       The Summary worksheet in the Error Model workbook comprises input mode control in
rows 4-10 and output summary in rows 12 - 142.

       The calculation mode control is accomplished with cell A6 where the user normally
confirms that a numerical value of "2" is designated.  Mode 2 designates emissions calculation
with all errors applied.  Mode 1 corresponds to a calculation of emissions with all errors applied
except environmental errors  and time  alignment.  Mode 0 designates an  "ideal" emissions
calculation with no errors applied.   In  Monte Carlo error model  simulation performed in this
study Mode 2 was used.

       Mode 0 is used off-line prior to Monte Carlo simulation to generate the "ideal" emissions
for a given reference NTE event.  The Mode 0 values are calculated by entering a value of "0" in
cell A6.  The Mode 0 calculation and subsequent storing of the "ideal" emissions results may be
accomplished manually (as described above) or by exercising a provided macro.  The macro
automatically sets the value in cell A6 to zero, calculates and saves the "ideal" emissions values,
and returns the value in A6 to "2" in preparation for the Monte Carlo simulation.  The locations
where the reference NTE event must be entered manually, and the locations where the "ideal"
emissions must be saved (done automatically if the macro is used) are described in the Methods
worksheet section.

       Mode 1  calculations are fully  implemented in the Error Model spreadsheet and used for
drift correction calculations.   Mode 1 in cell A6 is not typically used but can be applied for
diagnostic purposes.

       The output summary  section of the Summary worksheet in rows  12 -  142  presents
numerically and descriptively labeled outputs of the emissions and emissions error calculations.
The suffix '_TA' indicates that time alignment has been applied to the result, and the  suffix
'_DC' indicates that periodic drift correction has been applied to the result. Both time alignment
and periodic  drift correction are applied in the Methods worksheet.  However, a result in the
Summary worksheet with designation '_TA' has been calculated with an emissions value plus
errors all modified by the applicable time alignment percentage just prior to subtracting the
"ideal" emissions to produce the emissions difference.  Time alignment was applied only to NOX
and CO emissions.  Similarly, a result in the Summary worksheet with designation '_DC' has
been calculated by a drift correction formulation. A periodic drift rejection flag was checked by
logic.

       Consider the logic in cell C19 for the output 001_DeNOx (g/kW-hr), Method 1_TA_DC.
The logic checks the drift rejection flag in cell C43.  If the flag is  1, a huge negative number
SwRI Report 03.12024.06                       D-5

-------
(typically -9999999, from cell D13) is returned in cell C19.  Otherwise, the time aligned value
from the Methods worksheet is returned in cell C19 of the Summary worksheet.

       In the output summary, the cells that are highlighted in turquoise color are designated by
Crystal Ball  as  "Forecast" (or output)  random variables.  When  running the Monte  Carlo
simulation with Crystal Ball, a filter  is designated for the '_DC' (drift corrected) "Forecasts"
such that a value less than -9999 (or  similar huge negative number, but greater than the value
typically -9999999 returned when drift flag is 1) will be rejected. The filter on the "Forecast"
rejects the values meeting the filter criterion (drift correction).  Thus, when data is "Extracted"
from a Crystal Ball Monte Carlo simulation,  drift corrected (rejected) values are omitted (cells
are empty) in the Excel spreadsheet of simulation output values. It is expected that other Monte
Carlo software comparable to Crystal Ball, such as @Risk,  will have similar rejection provisions
with which periodic drift correction based on the calculated flag can be accomplished.

       A total of 213  outputs ("Forecasts") are designated in the Summary worksheet covering
the number  of output values from three emissions  (NOx, CO and NMHC),  three calculation
methods (Methods 1,  2  and 3), with and without time alignment, and with and without drift
correction, for the full error model and for the validation model (designated Valid in Summary
worksheet variable labels).  All of these "Forecasts" are provided in both units of grams/kW-hr
and (for selected outputs) in grams/hp-hr. This variety of  calculations was accomplished in the
Methods worksheet.

2.4    Worksheet 3: Methods

       The Methods worksheet of the Error Model workbook comprises the following areas:

   •   Notes and diagnostic guides are located principally in rows I -22 in columns A - CF,
       continuing on row 5 through column DD.

   •   Reference NTE event data are located in rows 35 - 351 of columns A - W. Actual
       reference NTE event data must be entered manually starting on row 52 in columns A - V.
       One to 300 rows of reference NTE event data are allowed.  Uniform (one second interval)
       time steps are assumed represented by the reference  NTE data.

   •   Parameters calculated are located in rows 35-351 of columns X - CF. The number of
       rows of these parameter equations must match the number of rows in the reference NTE
       event. Excess cells in these columns may be cleared manually or automatically during
       execution of the macro.

   •   Mode 0, Ideal Emissions for this reference NTE event are stored in column O rows 22 -
       32 (either manually or automatically by the macro).  Related data on the same rows  are
       located in columns CT - DD.

   •   Input ic random variable distributions (Crystal Ball uses the terminology "Assumptions"
       for these inputs) are located in rows 26 - 32 of columns AG - CF.
SwRI Report 03.12024.06                       D-6

-------
   •   Emissions calculations by three methods are located in rows 6 - 101 of columns CG - CJ
       (Method 1), CK - CN (Method 2) and CO - CR (Method 3). This part of the worksheet
       calculates full model, validation model, time alignment and drift correction.

2.5    Methods Worksheet:Notes and Diagnostic Guide
       In rows 1-22 for columns A - CF, several descriptive labels and references are defined
for use in navigating through  the  worksheet.   Row 5, columns  A-DD,  contains column
identification numbers referenced in rows 7 through 22 (depending  on  the column).   For
example, in column H the values 65 and 66 in rows 8 and 9, respectively, indicate that the values
in column  H (rows 52 and following rows) are applied in columns 65 (BM) and 66 (BN) labeled
on row 5.  If the user scrolls to cells BM52 or BN52 it is observed that the spreadsheet formulas
in these cells refer to values from column H. Also, column H, rows 10, 11 and 12 indicate that
the values in column H (rows  52 down) are also applied in the Delta (emissions error surface)
tabs '45 Delta CO2  SS', '46 Delta CO2 Transient' and '49 Delta CO2 Ambient Temperature'.
The information in the notes and diagnostic guide was not applied by the spreadsheet in any of
the emissions calculations.  It was included with the intent to simplify diagnostics by  providing
information  on locations where spreadsheet values were  applied elsewhere in the spreadsheet.
Outside the  areas indicated  above, some other notes, comments and diagnostic guides may be
found in other areas of the spreadsheet.

2.5.1   Methods Worksheet: Reference NTE Event

       The reference NTE event used in the simulation was entered in rows 35-351 of columns
A - W.  Actual reference NTE  event data must  be entered manually starting on row 52 in
columns A - V. A minimum of one  and a maximum of 300 rows of reference NTE event data
are allowed. Equal time  steps (1 second intervals) are assumed in the reference NTE data rows.
The standard format and engineering units of reference NTE event data established for this
project must be observed. These are described in the column headings on rows 47-51, columns
A-V.

2.5.2   Methods Worksheet: Parameters

       Parameters applied in the three emissions methods are calculated in rows 35 - 351 of
columns X - CF.  The number of rows of these parameter equations must match the number of
rows in the  reference NTE event.  Excess cells in these  columns may be cleared manually or
automatically during execution of the macro.

       The formulas applied in rows 52 and down in columns X - CF have been produced by
normal edit-copy (typically of row 52 in these columns) and edit-paste to rows 53 and  following
rows in these columns.  The At values displayed in column AC are not used in any calculation,
but are displayed so a user can confirm uniform reference NTE  event time sampling.  The last
cell in column AC can be cleared (done automatically by the macro). Note that excess cells in
these  columns must be cleared, and  row deletion  operations should not be applied  since this
would affect other areas in the Methods worksheet.
SwRI Report 03.12024.06                      D-7

-------
       Certain sums are performed in several  columns over the Parameter rows (range of the
reference NTE event).  These are accomplished in row 46 in columns AU - AX, BC, BD, BI, BJ,
BO-BR and BU-CF.      Certain constants applied in the  calculation  are stored in  cells
AW42, BC42, BI42, BP40 and BP42. Other constants or conversion factors are incorporated
numerically in spreadsheet formulas.  Typical of these is "0.01" to convert a percentage to a
fraction.

       Specific parameters or variables are calculated in the various columns for application in
all three methods, full model and validation model, and for drift correction. Time alignment
distinctions are not generated in this area of the spreadsheet. Table 2 lists the parameters used in
the Methods worksheet, the columns where they are computed and a brief description of the
parameters.

   TABLE 2. METHODS WORKSHEET PARAMETER COLUMN DESCRIPTIONS
Methods Worksheet Parameters Column Descriptions
Subject
Engine operating state
percentages
ATime
NMHC
Fuel Rate
Exhaust Flow
Calculations
Speed with error
Fuel rate with error
Torque
BSFC
NOx and ANOx, ppm
Column
X-AB
AC
AD
AE
AF
AG
AH
AI
AJ
AK
AL
AM
AN
AO
AP
AQ
AR
Description
Convert NTE Event variables to percentages: speed, torque, fuel rate,
exhaust flow
Displays At between NTE Event rows
Calculate NMHC ppm as 0.98 of THC ppm
Calculate fuel rate g/s based on fuel density of 851 g/L
Convert exhaust flow SCFM to mol/s
Sum exhaust flow errors from Delta tabs 20, 21, 22, 23, 25 27 and 28
expressed in % of mol/s maximum. Respective ErrorControl tab
switches are applied.
Convert the total exhaust flow error in % of maximum mol/s to mol/s
Add the mol/s exhaust flow error to the exhaust flow in mol/s. Mode
control logic is applied.
Add engine speed error from Delta tab 43 expressed as % of engine
range converted to rpm to engine speed in rpm. Mode control logic
and ErrorControl switch are applied.
Add fuel rate from Delta tab 44 expressed as % of maximum fuel rate
converted to g/s to engine fuel rate in g/s. Mode control logic and
ErrorControl switch are applied.
Sum torque errors from Delta tabs 29, 30, 31, 32, 34 expressed as % of
peak torque, and from Delta tab 35 expressed as % of NTE point
torque converted to % of peak torque. ErrorControl switches are
applied.
Add the total torque error expressed as % of peak torque converted to
N-m to engine torque in N-m. Mode control logic is applied.
Sum BSFC errors from Delta tabs 36, 37, 38, 39, 41 expressed as
g/kW-hr, and from Delta tab 42 expressed as % of NTE point BSFC
converted to g/kW-hr. ErrorControl switches are applied.
Add the total BSFC error expressed as g/kW-hr to engine BSFC in
g/kW-hr. Mode control logic is applied.
Sum engine NO ppm and NO2 ppm.
Sum environmental NOx errors. Error from Delta tab 5 is the only one
developed.
Sum other NOx errors including errors from Delta tabs 1 and 2.
SwRI Report 03.12024.06
D-8

-------
                        AS
                        AT
                Add the total NOx errors expressed as ppm to engine ideal NOx in
                ppm.  Mode control logic is applied.
                Add the NOx errors except environmental expressed as ppm to engine
                ideal NOx in ppm for drift correction calculation. Mode control logic
                is applied.	
 NOx • Exhaust Flow
AU
                        AV
Form product of NOx fraction (all errors  case, column AS)  and
exhaust flow (mol/s, column AI) for application in Methods 1 and 2.
                Form product of NOx fraction (all errors  except environmental,
                column AT) and exhaust flow (mol/s, column AI) for application in
                Methods  1 and 2 drift correction.
 Speed • Torque
AW
                        AX
Form product of Speed (rpm, all errors case, column AJ) and Torque
(N-m, all errors case, column AM) for application in Methods 1 and 3.
Convert  rpm to radians/sec with 2;tradians/revolution, minutes to
seconds with 60sec/min, N-m/sec to watt hr with 3600Joules/watt hr,
and watt to kW with lOOOw/kW.
                Form product of Speed (rpm, no errors for validation case, column O)
                and  Torque (N-m,  no errors  for  validation  case, column T)  for
                application in  Methods 1 and  3.  Convert rpm to  radians/sec with
                27tradians/revolution, minutes to seconds  with 60sec/min, N-m/sec to
                watt hr with 3600Joules/watt hr, and watt to kW with IQOOw/kW.
 CO and ACO, %
                        AY
                        AZ
                         BA
                         BB
                Sum environmental CO errors including errors from Delta tabs 10 and
                11.
                Sum other CO errors.   Error from Delta tab  7 is  the  only  one
                developed.	
                Add the total CO errors expressed as % to engine CO in %.  Mode
                control logic is applied.	
                Add the CO errors except environmental expressed as % to engine CO
                in % for drift correction calculation.  Mode control logic is applied.
 CO • Exhaust Flow
BC
                         BD
Form product of CO fraction (all errors case, column BA) and exhaust
flow (mol/s, column AI) for application in Methods 1 and 2.
                Form product of CO fraction (all errors except environmental, column
                BB) and exhaust flow (mol/s, column AI) for application in Methods 1
                and 2 drift correction.
 NMHC  and  ANMHC,
 ppm
BE
                         BF
                         BG
                         BH
Sum environmental NMHC errors including errors from Delta tabs 16,
17 and 19.
                Sum other NMHC errors including errors from Delta tabs 13 and 14.
                Add the total NMHC errors expressed as ppm to engine NMHC in
                PPM. Mode control logic is applied.
                Add the NMHC errors  except environmental expressed as ppm to
                engine NMHC in ppm for drift correction calculation.  Mode control
                logic is applied.	
 NMHC • Exhaust Flow
BI
                         BJ
Form product of NMHC  fraction (all errors case, column BG) and
exhaust flow (mol/s, column AI) for application in Methods 1 and 2.
                Form product of NMHC fraction (all errors except environmental,
                column BH) and exhaust flow (mol/s, column AI) for application in
                Methods 1 and 2 drift correction.
 CO2 and ACO2, %
                         BK
                         BL
                         BM
                         BN
                Sum environmental CO2 errors.  Error from Delta tab 49 is the only
                one developed.	
                Sum other CO2 errors including errors from Delta tabs 45 and 46.
                Add the total CO2 errors expressed as % to engine CO2 in %.  Mode
                control logic is applied.	
                Add the CO2 errors except environmental expressed as % to engine
                CO2 in % for  drift correction calculation.   Mode control logic  is
                applied.	
SwRI Report 03.12024.06
                             D-9

-------
 Exhaust   Flow   •   |
 NMHC + ( C0+C02)
 /BSFC
                        BO
                        BP
                        BQ
                        BR
                Form product of NMHC fraction plus CO and CO2 fractions (all errors
                case, columns BG, BA and BM) and exhaust flow (mol/s, column AI)
                divided  by  BSFC  (g/kW-hr,  all errors  case,  column  AO) for
                application in Method 2.	
                Form product of NMHC fraction plus CO and CO2 fractions (all errors
                except environmental, columns BH, BB and BN) and exhaust flow
                (mol/s, column AI) divided by BSFC (g/kW-hr, with errors, column
                AO) for application in Method 2 drift correction.	
                Form product of NMHC fraction plus CO and CO2 fractions (all errors
                case, columns BG, BA and BM) and exhaust flow (mol/s, column AI)
                divided by BSFC (g/kW-hr, no errors case, column V) for application
                in Method 2 validation.
                Form product of NMHC fraction plus CO and CO2 fractions (all errors
                except environmental, columns BH, BB and BN) and exhaust flow
                (mol/s, column AI) divided by BSFC (g/kW-hr, no errors, column V)
                for application in Method 2 validation drift correction.	
 NMHC + (CO+C02)
BS
                        BT
Form sum of NMHC fraction plus CO and CO2 fractions (all errors
case, columns BG, BA and BM) for application in Method 3.
                Form sum of NMHC fraction plus CO and CO2 fractions (all errors
                except environmental, columns BH, BB and  BN) for application in
                Method 3.
 NOx  • Fuel  Rate /
 NMHC + ( CO+CO2);
BU
                        BV
Form product of NOx fraction (all errors case, using column AS) and
Fuel Rate (g/s, all errors case, column AK) divided by sum of NMHC
fraction plus CO and CO2 fractions (all errors case, column BS) for
application in Method 3.	
                Form product of NOx fraction (all errors except environmental, using
                column AT) and Fuel Rate (g/s, with error, column AK) divided by
                sum  of NMHC fraction plus CO and CO2 fractions (all errors except
                environmental,  column BT)  for  application  in  Method 3  drift
                correction.
 CO  •  Fuel Rate  /
 NMHC + ( CO+CO2);
BW
                        BX
Form product of CO fraction (all errors case, using column BA) and
Fuel Rate (g/s, all errors case, column AK) divided by sum of NMHC
fraction plus CO and CO2 fractions (all errors case, column BS) for
application in Method 3.	
                Form product of CO fraction (all errors except environmental, using
                column BB) and Fuel Rate (g/s, with error, column AK) divided by
                sum  of NMHC fraction plus CO and CO2 fractions (all errors except
                environmental,  column BT)  for  application  in  Method 3  drift
                correction.
 NMHC •  Fuel Rate /
 NMHC + ( CO+CO2) ]
BY
                        BZ
Form product of NMHC fraction (all errors case, using column BG)
and Fuel Rate (g/s, all errors case, column AK)  divided by sum of
NMHC fraction plus CO and CO2 fractions (all errors case, column
BS) for application in Method 3.	
                Form  product of NMHC  fraction (all errors except environmental,
                using column BH) and Fuel Rate (g/s, with error, column AK) divided
                by sum of NMHC fraction plus CO and CO2 fractions (all errors except
                environmental, column BT)  for  application  in  Method 3  drift
                correction.
 NOx  • Fuel  Rate /
 NMHC + (CO+C02);
CA
                        CB
Form product of NOx fraction (all errors case, using column AS) and
Fuel Rate (g/s, selected errors case, column AE) divided by sum of
NMHC fraction plus CO and CO2 fractions (all errors case, column
BS) for application in Method 3 validation.	
                Form product of NOx fraction (all errors except environmental, using
                column AT) and Fuel Rate (g/s, selected errors case, column AE)
                divided by  sum of NMHC fraction plus CO and CO2 fractions (all
                errors except environmental, column BT) for application in Method 3
                validation drift correction.
SwRI Report 03.12024.06
                            D-10

-------
 CO  •  Fuel Rate  /  [
 NMHC + (CO+C02)]
                    CC
                    CD
             Form product of CO fraction (all errors case, using column BA) and
             Fuel Rate (g/s, selected errors case, column AE) divided by sum of
             NMHC fraction plus CO and CO2 fractions (all errors case, column
             BS) for application in Method 3 validation.	
             Form product of CO fraction (all errors except environmental, using
             column BB) and Fuel Rate (g/s, selected errors case, column AE)
             divided by sum of NMHC fraction plus CO and CO2 fractions (all
             errors except environmental, column BT) for application in Method 3
             validation drift correction.
 NMHC • Fuel Rate / [
 NMHC + ( CO+CO2) ]
CE
Form product of NMHC fraction (all errors case, using column BG)
and Fuel Rate (g/s, selected errors case, column AE) divided by sum of
NMHC fraction plus CO and CO2 fractions (all errors case, column
BS) for application in Method 3 validation.	
                    CF
             Form product of NMHC fraction (all errors except environmental,
             using column BH) and Fuel Rate (g/s, selected errors case, column
             AE) divided by sum of NMHC fraction plus CO and CO2 fractions (all
             errors except environmental, column BT) for application in Method 3
             validation drift correction.
2.5.3  Methods Worksheet: Mode 0 Ideal Emissions

       For the reference NTE event in rows 52 and down in columns A - V, an ideal emissions
value must be calculated and stored for application in the emissions difference calculations. The
ideal case can be calculated either manually or automatically by  the macro.  Following the
calculation, the ideal values are stored by edit-copy edit-paste-special-values operation to the
cells in  column  O, rows 22 - 32.   The manual  operations described below are performed
automatically by the macro, if executed, after manually entering the reference NTE event.

       After manually entering the reference NTE  event to be simulated and checking that the
number of rows of equations in the Parameters section matches the rows in the reference NTE
event, a numerical "0" can be entered in cell A6  of the  Summary worksheet.  The Methods
worksheet should have  calculated Mode 0 results using the reference  NTE  event.  If error
messages like "#VALUE or #DIV/0!" are displayed, there is probably still a mismatch between
the rows of the reference NTE event and Parameter equations. When calculated properly (with 0
in Summary A6), the values displayed in the Methods worksheet columns CU, CV, CW, DB and
DD will be equal on each of the rows 22 - 32.  The values in column CX are not yet equal
(unless previously calculated and stored for this reference NTE event) because they reflect the
values stored in Methods worksheet column O, rows 22 - 32.  The  next  manual step is to edit-
copy column CU, rows 22 -32, and store the values by edit-paste-special-values in column  O,
rows 22 -32.  Now in rows 22 -32 the columns CU - DD  should be equal. The final step is to
return to Summary  worksheet  cell A6 and change the value from 0 to 2.  At this point the
spreadsheet could  be run in  Monte  Carlo  simulation to produce properly  sampled values.
However, if the user desires to monitor charts provided in the  Delta worksheets during the
simulation, further row-matching to the reference NTE event is required in most of the Delta
worksheets.

       The manual operations described in the previous paragraph are intended to explain how
the Mode 0 ideal emissions are calculated and stored for use in the Monte  Carlo simulation when
Aemissions values are calculated using the ideal emissions results stored in O22 - O32.  The
reference NTE event must be entered  with an operation such as  a  manual edit-copy and edit-
paste or edit-paste-special-values operation.  At this point the macro can be executed with tools-
SwRI Report 03.12024.06
                       D-ll

-------
macro-macros-(macrol)-run operation.    The  macro  automatically  performs the  mode  0
calculation, stores the mode 0 results in O22 - O32, and changes Summary A6 back to mode 2.

       The macro also deletes extraneous rows from all the appropriate Delta worksheets so the
charts therein  display properly.  It is important to copy the reference NTE event into a fully
'loaded' starter file with equations filled on 300 rows in the Parameters area, and with full 300
row complement of equation-rows in each of the appropriate Delta worksheets for the macro to
modify the spreadsheet properly.

2.5.4   Methods Worksheet: Input ic Random Variable Distributions

       Probability distribution parameters are applied, and simulation trial values of the inputs
are generated  in rows 26 - 32  of columns AG  -  CF.  Rows 26  and 27 are used to input
distribution parameters. Rows 28 and 29 contain descriptive labels brought from the appropriate
Delta worksheet.   Row 30  is an information-only  number, row 31 contains the name label
applied in Monte  Carlo  simulation to the input ic, and row 32 is where the Monte Carlo
simulation tool places generated randomly-sampled values during simulation. The values in row
31 are referenced by  formula  in the respective  Delta worksheets where  they  are  used for
interpolation on the error surfaces.

       The Monte Carlo tool  in Crystal Ball uses  the terminology "Assumptions"  for these
inputs. Two distribution forms are applied: truncated normal (Gaussian), and discrete uniform.
For the normal distribution, the  applied standard deviation  is in  row 27.  In Crystal  Ball, the
standard deviation cell on row 27 and the label cell on row 31 were referenced by equation in the
Crystal Ball assumption  setup window,  the mean  was input  as 0,  and  the  distribution was
truncated at -1 and at  +1.  Since  all the truncated normal ic distributions are identical (although
the sampled trial values from each will be random in the Monte Carlo simulation), the  Crystal
Ball  define-copy  data  and  defme-paste  data operations were applied to  define the  truncated
normal distributions for other ic variables once the first one had been defined.

       For the discrete uniform  distributions, the minimum discrete value (1 in all cases) was
applied in row 26, the maximum discrete value was  applied in row 27  and the other row
descriptions are the same as before.  Again, one of these inputs was setup with  Crystal  Ball
"define assumption" and then applied with Crystal Ball define-copy data and defme-paste data
operations to other ic cells on row 32 where a discrete uniform distribution was applied. When
Crystal Ball "Assumptions" were defined, Crystal Ball colored each input cell bright green.  It is
expected that @Risk has similar input definition procedures.

       During a Monte Carlo simulation, the Monte Carlo tool  (e.g.  Crystal Ball)  placed a
numerical value in each of the ic cells on row 32. Then the spreadsheet was exercised to perform
interpolations  in all the Delta worksheets.  The resulting  error  sample values for the entire
reference  NTE event  were  returned to the Methods worksheet Parameters  area, and then the
Methods worksheet Emission Calculations section computes Aemissions using three  methods,
full model and validation, drift correction, time alignment and not, etc. to generate one  set of the
213 output values described in the Summary section.  The simulation tool stores the  set of 37
random input values from row 32 as well as the 213 output values in an Excel data base from
SwRI Report 03.12024.06                      D-12

-------
which the corresponding sets of values can later be extracted.  Once each trial was completed,
the simulation tool  randomly  sampled  a second  set  of input  values  from  the  respective
probability distributions, placed the values in the  cells  on row 32, exercised the spreadsheet
again, stored the input and output values, and went to a third trial, etc.  Typically 10,000 and
30,000 trials, depending on the reference NTE event, were used in this project with this Error
Model workbook.

      Note that there are three ways the user can control the effect of the ic values in the
emissions calculations:

              1.  Mode control in Summary A6,
              2.  Include / exclude switches in ErrorControl column D, and
              3.  Specification of input random variables ("Assumptions") and their probability
                 distributions in the Methods worksheet row 32.

These three ways of controlling the ic values are independent, but the effects are interdependent
as follows.  Mode  control determines what categories of errors are added into the calculations.
Mode controls categories of errors are classified as:

              1.  Mode 0 - no errors included
              2.  Mode 1 - "all" but 'environmental' errors included
              3.  Mode 2 - "all" errors added into the calculations.

"All" in this context represents those error surfaces turned on by the switches in the ErrorControl
worksheet.  The input random  variable distribution controls the distribution of the sampled ic
values applied during Monte Carlo simulation for the  several Delta error surfaces. Mode and
ErrorControl switches must be appropriately turned on for the effects of the sampled ic values to
be included in the emissions difference  results. These  controls affect the calculations in the
Methods worksheet Parameters and Emission Calculations sections.

2.5.5 Methods Worksheet: Emission Calculations

      In the area of rows  6 - 101 of columns  CG  - CR the  brake-specific emissions and
Aemissions  calculations  are performed using the  variables and  parameters  generated in the
Parameters section.   Three sets of columns, structured  similarly,  calculate the full  model,
validation model, time alignment and drift correction for the following methods:

    1. Method 1 calculations are applied in columns CG - CJ,

   2. Method 2 calculations are applied in columns CK - CN, and

   3. Method 3 calculations are in columns CO - CR.

Columns CG  - CJ for Method 1 are typical of the methods where  the structure is the same, but
the formulas are a little different.   Column CG is an information-only column that displays
Method 1 formulas implemented by equations in columns CH - CJ.  Column CH performs the
SwRI Report 03.12024.06                      D-13

-------
NOX emission calculations, while column CI  performs  the  CO  emissions  and column  CJ
performs the NMHC calculations.  The structure of the three columns is the same.  Formulas
implemented in the three columns are the same, but the equations implementing the formulas
apply variables and parameters appropriate to the respective emissions.

       As an example of the  calculation for NOX Method 1 we will examine column CH in
detail.  The full model calculation was accomplished in cells CH48 - CH74. The ideal emissions
result was brought into the area by equation in CH51.  Full model NOX emissions (eNOx) in
g/kW-hr were calculated in CH54.  Cells CH55 - CH59 are information-only diagnostic aids.
The full model Method 1 result in CH54 is calculated by the formula in Figure 1.
                               N
                               N
                               z
                                      <^\ppm)+xNUi\j>pmj
                                                                    m°
                                             60*1000*3600
                 FIGURE 1.  BRAKE-SPECIFIC NOX BY METHOD 1

In the formula for the full model mode 2, delta error values sampled from the Delta worksheets
1, 2 and 5 have been added to xNO2, xNO.  Similarly, delta error values sampled from Delta
worksheets 20-23  and 25-28  have been  added to the exhaust flow, delta errors sampled from
worksheets 29-32 and 34-35 were added to torque, and worksheet 43 deltas were added to speed.
The At values are equal (1 second) and therefore cancel out of the equation.

       'Full' model Mode 1 (with all errors except environmental errors from Delta worksheet
5) is calculated in CH64.  Cells CH65 - CH69 are for information-only.

       The full model rejection flag is calculated in CH74.  A rejection flag is computed as
shown in Figure 2. For reference NTE events with ideal NOX emissions below the threshold of
2.682 g/kW-hr (2.0 g/hp-hr),  the allowable band is defined by full model emissions (mode 1 -
mode 2| < 4% of threshold.  For reference NTE  events with ideal NOX emissions above the
threshold, the allowable band is defined by full model emissions (mode 1 - mode 2| < 4% of
ideal emissions.
SwRI Report 03.12024.06
D-14

-------
                        Drift Correction Rejection Flag
      0.3
      0.2-
•o
o
E
•o
i
      0.1-
  0)
 •o
  o
  0)
   O) 0

  O
  z
   0)
    -0.1 -
0)
o
c
gj

I   -0-2
     -0.3
                                   Flag = 1
                                           Flag = 0
                                   Flag = 1
                              Ideal eNOx g/kw-hr (mode 0 )
     FIGURE 2. PLOT ILLUSTRATING PERIODIC DRIFT CORRECTION FLAG

       The validation model  calculation  was accomplished  in  the  cells  CH79 -  CH101.
Validation model NOX emissions (g/kW-hr) was calculated in cell CH81.  Cells CH82  - CH86
are information-only. Validation model Mode 1 (with 'all' errors except certain environmental
errors) was calculated in CH91.  Cells CH92  -  CH96 are information-only.   The validation
model  rejection flag was calculated in CH101.  For  reference NTE events  with ideal  NOX
emissions below the threshold of 2.682 g/kW-hr (2.0 g/hp-hr),  the allowable band was defined
by validation model  emissions (mode 1 - mode 2| < 4% of threshold.  For reference NTE events
with ideal NOX emissions above the threshold, the allowable band was defined by validation
model emissions (mode 1 - mode 2| < 4% of ideal emissions.

       The drift correction flag was computed similarly for CO and NMHC emissions  with
different thresholds and allowable bands. For CO the threshold  was 26.016 g/kW-hr (19.4 g/hp-
hr) and the allowable band was again defined at 4%.  For NMHC the threshold was 0.282 g/kW-
hr (0.21 g/hp-hr) and the allowable band was defined at 10%.

       Continuing with column CH for NOX, the time alignment indicated by the suffix '_TA' in
the label in CH24 was calculated for the full model in CH25.  The Time Alignment was applied
as a modification by a percentage of the full model NOX emissions result from CH54. The time
alignment  for NOX  expressed as a  percentage from Delta worksheet 51  was converted  to  a
fraction, and the fractional increment g/kW-hr was added to the  NOX expressed in g/kW-hr.  The
SwRI Report 03.12024.06
                                       D-15

-------
mode control logic and ErrorControl switch are applied.  The same calculation of time alignment
for the validation result from CH81  was  done in  CH42  again with mode  control and
ErrorControl.

       In CH14 the full model ANOX value in g/kW-hr was calculated for the time alignment
case by subtracting the ideal emissions in g/kW-hr from the result in CH25 described in the
previous paragraph.  The full model ANOX value in g/kW-hr was converted to g/hp-hr in CHI 1.
Similar results for the validation model are in CHI 5 and CH12.

       Rejection flags are taken to the  Summary worksheet where drift correction explained in
the Summary  worksheet section  has  been described.   Additional  result  combinations are
calculated and presented for NOX Method 1 in column C of the Summary worksheet using results
described above from column CH of the Methods worksheet.

       Calculations for NOX by Method 1 described above for  column CH are similar for CO
and NMHC by Method 1 in  columns CI and CJ, respectively. Similar calculations for NOX, CO
and NMHC by Method 2 are presented in columns CL - CN, and by  Method 3 in columns CP -
CR.

2.6    Worksheet 4: Constants and Equations

       The Constants&Eqns tab was  strictly  a snapshot of the equations used in the brake-
specific emissions calculations.   It displayed the equations  and  constants implemented in
spreadsheet formulas of the  Methods worksheet.   The various  parts all shown together in this
worksheet are redisplayed at appropriate locations in the Methods worksheet.

2.7    Worksheet 5: SS NOx Error Surface

       The 1  Delta NOx SS worksheet was the first Delta worksheet.  Its functional structure,
formulas, charts and operation are similar to the following worksheets:

   •   7 Delta CO SS
   •   20 Delta Exhaust Flow SS
   •   22 Delta Exhaust Flow Pulsation
   •   23 Delta Exhaust Flow Swirl
   •   30 Delta Torque DOE Testing
   •   37 Delta BSFC DOE Testing
   •   45 Delta CO2 SS

With only minor changes in charts and structure,  its function, formulas and  operation are also
similar to the following worksheets:

   •   2 Delta NOx Transient
   •   21 Delta Exhaust Flow Transient
   •   29 Delta Dynamic Torque
   •   36 Delta Dynamic BSFC


SwRI Report 03.12024.06                     D-16

-------
   •   43 Delta Dynamic Speed
   •   44 Delta Dynamic Fuel Rate
   •   46 Delta CO2 Transient

The following provides a brief summary of the 1 Delta NOx SS worksheet:

   •   Rows 1-7 contain descriptive information about the error surface implemented in the
       worksheet.
   •   Rows 8-42 present the error surface in columns A - L. Other columns, M - W, on these
       rows generate a lookup table used with an interpolation routine.
   •   Figures A, B and C follow.
   •   Rows 76-379 calculate the ANOX SS error values for each row of the reference NTE
       event. These values were returned to the Methods tab Parameters section.

The following paragraphs describe in further detail functions in the 1 Delta NOx SS worksheet:

       Data from the error surface (rows 13 - 42, columns A - L, in this Delta worksheet) must
be entered in sorted order (sorted  on Lab Nominal column C in ascending order) for proper
operation of the x-lookup-interpolation  function.   The  three figures chart the  error function.
Figure A plots several data sets versus the x-value, Lab Nominal (column C).  Figure A y-values
are NOX ppm (PEMS) including Lab Nominal (column C), 95th percentile (column F),  50th
percentile (median) (column E) and 5th percentile (column D).

       Related error surface data are plotted in  Figure B. Figure B plots several data sets versus
the same x-value,  Lab Nominal  (column C).   Figure B y-values are the difference, NOX ppm
(PEMS) - NOX ppm (lab, nom).  The differences plotted may not correspond exactly to the
values shown in  Figure  A because of the  statistical  procedure applied in calculating the
differences shown in Figure B. This procedure  is described in Sections 4.0 and 5.0 of this report.
Figure B plots the 95th percentile (column I), the 50th percentile (median) (column H) and the 5th
percentile  (column G).   In  addition to the  error  surface data, Figure B also shows the
interpolation line designated ic = xx (column  V), and the reference NTE  event values on the
interpolation line (column F rows  80 through end  of the  reference NTE  event versus Lab
Nominal x-values in  column B rows 80  through end of the reference NTE event).   When ic =
+1, the interpolation line plots on the 95th percentile. When ic = 0, the interpolation line plots on
the 50th percentile. When ic = -1, the interpolation line plots on the 5th percentile. The reference
NTE  event always plots on the interpolation line, with  points at the x-values in the  reference
NTE event.

       The error surface data were also plotted in the format of Figure C. Again the x-axis was
the same Lab nominal  (column C).   This time  the y-axis data are  the ic values.  Thus, the  95th
percentile plots at +1, the 50th percentile plots  at 0  and the 5th  percentile plots at -1.  The
interpolation line plots  at the value of ic, and the reference NTE event plots on the interpolation
line at the x-values in the reference NTE event.  If appropriate  value labels were  displayed in
Figure C, the values would  represent the  error surface plotted  on a z-axis above the  two-
dimensional x-y plane.  These error surface values are displayed graphically in Figure B.
SwRI Report 03.12024.06                      D-17

-------
       Now consider inner rows 13 - 41 in the look-up table in columns T - W.  Column T is a
repetition of the x-value from column C.  Column U calculates a row-to-row A for the x-values
in column  T for use in interpolation.  Column V computes the  interpolation line  linearly
interpolated according to the value of ic between the median and the  95th percentile if ic > 0 (on
median if ic = 0 and on 95th percentile if ic = +1); and between the median and the 5th percentile if
ic < 0 (on median if ic = 0 and on 5th percentile if ic  = -1).  Only one ic value (from cell E80) is
applied in this calculation of the interpolation line. The Microsoft Excel vertical lookup function
VLOOKUP is applied to the table in rows 12 - 42 in columns T - W.  This is done in rows 80
and down in column F. Because of the way the VLOOKUP function  operates, the first row cells
T12  and V12, and the last row cell W42 (all three cells distinguished by  darker line borders)
contain formulas or  values different from the formulas of the inner rows.  The formula in cell
T12 assures that the  lookup function can always find an x-value in its table.  The formula in V12
and the value in W12 assure that the interpolation in cells  F80 to the end of the reference NTE
event data returns the nearest ANOx SS value on the interpolation line if the x-value is outside
the range of the error surface lab nominal values.

       Before going to the interpolation accomplished in F80 and down,  consider briefly the
formulation on rows 12 - 43 in columns O - R.  This formulation considers one x-value from the
reference NTE event, the first one, in  cell B80 and selects the two  adjacent rows in the error
surface between which to interpolate on the B80 x-value.  The result is formed on row 43  in
these columns and then the "check"  cell G80  accomplishes the ic controlled interpolation.  This
provides an alternative calculation check on one row in the reference NTE event.

       Now consider the interpolation for each point in the  reference NTE event. Column B,
row 80 and down, brings the lab nominal x-value from the Methods worksheet reference NTE
event.  For this Delta worksheet, that  x-value is xNO(ppm) + xNO2(ppm).  The  out-of-range
flags are information-only indicating points in the reference NTE event with x-value out of the
range of the error surface lab nominal.  The ic value for this Delta worksheet was brought into
cell E80 from the Methods worksheet ic area.  Each  point in the  reference NTE event was
interpolated with the same ic value, but with its own x-value. Recalling that the interpolation line
in column V was computed with this one ic value, the x-interpolation between the appropriate
two adjacent rows in the error surface can now be accomplished.  This requires using the x-value
on each row in column B, B80 and down,  in the VLOOKUP function,  and performing the
required calculation using  the  looked-up values  and deltas  from the look-up  table.   The
calculation  is done with the formulas in cell F80 and down. The values computed in column F,
cell F80 and down through the reference NTE event, could be considered elements of a column
matrix or vector, and are returned to the Methods worksheet Parameters section.

       In the Monte Carlo simulation, the Methods worksheet combines  this reference NTE
event result vector from the 1 Delta NOX SS worksheet with similar results  from other error
surfaces, calculates  Aemissions by three  methods, full  model and validation, drift reject flags,
time  alignment,  etc.  to produce  a set  of 213 output  values ("Forecasts"  in Crystal  Ball
terminology) described in the Summary worksheet section.  This was done having input 35 ic
values (including  1 ic value for this Delta NOX SS) and two time alignment values all chosen by
random sample from the appropriate truncated normal or uniform distribution as explained in the
Methods worksheet  section.  Then another sample set of 35 plus two randomly sampled values
SwRI Report 03.12024.06                      D-18

-------
was input (only one ic value coming to this Delta function again). The reference NTE event NOX
SS vector was recomputed with the one new ic value, returned to Methods worksheet and another
set of 213 output values was produced.  This process was repeated many times until a statistical
convergence criterion, described in Section 2, was satisfied.  Typically, 10,000 to 30,000 sets of
37 input values and 213 output values were  produced to satisfy the convergence criterion with
this Error Model spreadsheet.

       The number of rows in the  Delta worksheet reference NTE event area (rows 80 and
down) should match the  number of rows in the reference NTE event applied in the Methods
worksheet for proper function of Figures B and C.  The starter spreadsheet has been set up with
the  range of charted reference NTE event  series extending through row 379 in this  Delta
worksheet.   The balance  of the spreadsheet  should calculate correctly when a reference NTE
event is properly entered in the Methods tab and Parameters formulas properly aligned, although
figures like B and C will not display properly until the last row of the reference NTE event is
coincident with the end of the range  of the charted reference NTE event series.  This could be
done  manually  in  each Delta worksheet  where needed, however, the macro was designed  to
convert the fully  'loaded' starter workbook  after the reference NTE event was  entered in the
Methods worksheet.  The macro uses the row count in the reference NTE event, aligns formulas
in the Methods worksheet Parameters area, and eliminates extra rows in the reference NTE event
area of each appropriate Delta worksheet.  Again, the macro will do the operations correctly only
on a fully 'loaded' starter workbook set up with 300 rows of formulas in the Methods worksheet
Parameter area, and in each of the Delta worksheets using the reference NTE event.

       One further detail in  1 Delta NOX SS is the value computed in cell E75  and its label  in
E74.  Each Delta worksheet has such a  cell where the ic value is reflected.  The intent is  to
provide the user a further diagnostic capability  through the user's designation of such cells  as
outputs  ("Forecasts") in Monte Carlo simulation.  This would allow the user to statistically
process the input data with the same Crystal Ball (@Risk, etc.) tools used to process the outputs.
Crystal Ball allows extraction of all the input  (Crystal Ball applies the term "Assumptions" to the
inputs) trial values without reflecting them as  outputs ("Forecasts").
SwRI Report 03.12024.06                     D-19

-------
                                APPENDIX E
                   40-POINT TORQUE AND BSFC MAP DATA
SwRI Report 03.12024.06

-------
      Table Type
      Engine Manufacturer
      Engine Model
      Model Year
      Serial Number
      Fuel Density
      Peak Torque
      nlo
      nhi
Torque
DDC
Series 60
2005
06R0767368
        851 [g/L]
       2195 [N-m]
       1014 [rpm]
       2129 [rpm]
Point
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
Raw Data
ECM Speed
rpm
2152
2148
2142
2131
2095
2095
2095
2095
2095
2094
1866
1866
1867
1867
1867
1867
1867
1867
1636
1636
1636
1636
1636
1636
1636
1407
1407
1407
1407
1407
1407
1407
1407
1179
1179
1179
1178
1178
1178
1179
Lab Torque
N-m
541
632
764
1074
1599
1336
1074
811
658
548
548
658
811
1074
1336
1599
1735
1858
2052
1599
1336
1074
811
658
548
548
750
811
1074
1336
1599
1878
2136
2058
1599
1336
1074
896
811
548
Normalized Data
ECM Speed
nlo = 0%
nhi = 1 00%
102%
102%
101%
100%
97%
97%
97%
97%
97%
97%
76%
76%
76%
76%
76%
76%
76%
76%
56%
56%
56%
56%
56%
56%
56%
35%
35%
35%
35%
35%
35%
35%
35%
15%
15%
15%
15%
15%
15%
15%
ECM Fuel Rate
% of Maximum
Fuel Rate
34%
39%
46%
60%
97%
81%
66%
53%
44%
38%
33%
38%
46%
58%
73%
87%
95%
1 00%
94%
72%
60%
49%
38%
32%
28%
23%
30%
32%
42%
51%
60%
71%
84%
68%
53%
44%
36%
30%
27%
20%
Lab Torque
% of Peak
Torque
25%
29%
35%
49%
73%
61%
49%
37%
30%
25%
25%
30%
37%
49%
61%
73%
79%
85%
93%
73%
61%
49%
37%
30%
25%
25%
34%
37%
49%
61%
73%
86%
97%
94%
73%
61%
49%
41%
37%
25%
SwRI Report 03.12024.06
                   E-l

-------
           Table Type
           Engine Manufacturer
           Engine Model
           Model Year
           Serial Number
           Fuel Density
           Peak Torque
           nlo
           nhi
BSFC
DDC
Series 60
2005
06R0767368
        851 [g/L]
       2195 [N-m]
       1014 [rpm]
       2129 [rpm]
Point
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
Raw Data
ECM Speed
rpm
2152
2148
2142
2131
2095
2095
2095
2095
2095
2094
1866
1866
1867
1867
1867
1867
1867
1867
1636
1636
1636
1636
1636
1636
1636
1407
1407
1407
1407
1407
1407
1407
1407
1179
1179
1179
1178
1178
1178
1179
Normalized Data
ECM Speed
nlo = 0%
nhi = 100%
1 02%
1 02%
101%
1 00%
97%
97%
97%
97%
97%
97%
76%
76%
76%
76%
76%
76%
76%
76%
56%
56%
56%
56%
56%
56%
56%
35%
35%
35%
35%
35%
35%
35%
35%
15%
15%
15%
15%
15%
15%
15%
ECM Fuel Rate
% of Maximum
Fuel Rate
34%
39%
46%
60%
97%
81%
66%
53%
44%
38%
33%
38%
46%
58%
73%
87%
95%
100%
94%
72%
60%
49%
38%
32%
28%
23%
30%
32%
42%
51%
60%
71%
84%
68%
53%
44%
36%
30%
27%
20%
BSFC
% of Max
100
96
92
85
81
82
85
90
96
106
95
91
88
84
83
82
81
81
79
81
77
79
82
86
89
85
81
80
77
80
75
76
77
77
77
78
78
79
80
86
SwRI Report 03.12024.06
             E-2

-------
      Table Type
      Engine Manufacturer
      Engine Model
      Model Year
      Serial Number
      Fuel Density
      Peak Torque
      nlo
      nhi
Torque
CAT
C9
2005
9DG05784
        851 [g/L]
       1464 [N-m]
       1099 [rpm]
       2320 [rpm]
Point
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
Raw Data
ECM Speed
rpm
2366
2348
2340
2325
2195
2195
2195
2194
2194
2194
1966
1966
1966
1966
1966
1966
1965
1966
1738
1738
1738
1738
1738
1738
1738
1510
1510
1510
1510
1510
1510
1510
1510
1281
1281
1281
1281
1281
1281
1281
Lab Torque
N-m
365
369
431
604
1043
874
705
535
439
366
366
439
535
705
874
1043
1135
1230
1372
1043
874
705
535
439
366
366
492
535
705
874
1043
1242
1436
1401
1043
874
705
580
535
366
Normalized Data
ECM Speed
nlo = 0%
nhi = 1 00%
104%
102%
102%
100%
90%
90%
90%
90%
90%
90%
71%
71%
71%
71%
71%
71%
71%
71%
52%
52%
52%
52%
52%
52%
52%
34%
34%
34%
34%
34%
34%
34%
34%
15%
15%
15%
15%
15%
15%
15%
ECM Fuel Rate
% of Maximum
Fuel Rate
55%
55%
59%
72%
1 00%
91%
74%
63%
57%
53%
41%
46%
52%
62%
74%
84%
93%
98%
88%
70%
63%
53%
48%
44%
41%
31%
37%
39%
47%
53%
57%
68%
78%
62%
51%
44%
37%
33%
32%
24%
Lab Torque
% of Peak
Torque
25%
25%
29%
41%
71%
60%
48%
37%
30%
25%
25%
30%
37%
48%
60%
71%
78%
84%
94%
71%
60%
48%
37%
30%
25%
25%
34%
37%
48%
60%
71%
85%
98%
96%
71%
60%
48%
40%
37%
25%
SwRI Report 03.12024.06
                  E-3

-------
            Table Type
            Engine Manufacturer
            Engine Model
            Model Year
            Serial Number
            Fuel Density
            Peak Torque
            nlo
            nhi
BSFC
CAT
C9
2005
9DG05784
        851 [g/L]
       1464 [N-m]
       1099 [rpm]
       2320 [rpm]
Point
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
Raw Data
ECM Speed
rpm
2366
2348
2340
2325
2195
2195
2195
2194
2194
2194
1966
1966
1966
1966
1966
1966
1965
1966
1738
1738
1738
1738
1738
1738
1738
1510
1510
1510
1510
1510
1510
1510
1510
1281
1281
1281
1281
1281
1281
1281
Normalized Data
ECM Speed
nlo = 0%
nhi = 100%
104%
102%
102%
100%
90%
90%
90%
90%
90%
90%
71%
71%
71%
71%
71%
71%
71%
71%
52%
52%
52%
52%
52%
52%
52%
34%
34%
34%
34%
34%
34%
34%
34%
15%
15%
15%
15%
15%
15%
15%
ECM Fuel Rate
% of Maximum
Fuel Rate
55%
55%
59%
72%
100%
91%
74%
63%
57%
53%
41%
46%
52%
62%
74%
84%
93%
98%
88%
70%
63%
53%
48%
44%
41%
31%
37%
39%
47%
53%
57%
68%
78%
62%
51%
44%
37%
33%
32%
24%
BSFC
% of Max
100
99
95
87
77
82
83
87
93
98
90
86
81
78
76
73
74
72
69
71
74
76
81
84
89
85
80
79
75
73
69
69
68
67
70
71
74
77
79
80
SwRI Report 03.12024.06
             E-4

-------
      Table Type
      Engine Manufacturer
      Engine Model
      Model Year
      Serial Number
      Fuel Density
      Peak Torque
      nlo
      nhi
Torque
INT
VT365
2006
     332259
        851 [g/L]
        681 [N-m]
       1198 [rpm]
       2839 [rpm]
Point
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
Raw Data
ECM Speed
rpm
2903
2890
2868
2832
2622
2621
2621
2621
2621
2620
2328
2328
2328
2328
2328
2328
2328
2328
2033
2033
2033
2032
2032
2032
2032
1739
1739
1739
1739
1739
1739
1739
1739
1444
1444
1443
1444
1444
1444
1444
Lab Torque
N-m
170
204
257
343
502
429
343
257
204
170
170
204
257
343
429
516
540
543
607
516
429
343
257
204
170
170
234
257
343
429
515
583
650
671
515
429
343
283
257
170
Normalized Data
ECM Speed
nlo = 0%
nhi = 1 00%
104%
103%
102%
100%
87%
87%
87%
87%
87%
87%
69%
69%
69%
69%
69%
69%
69%
69%
51%
51%
51%
51%
51%
51%
51%
33%
33%
33%
33%
33%
33%
33%
33%
15%
15%
15%
15%
15%
15%
15%
ECM Fuel Rate
% of Maximum
Fuel Rate
47%
53%
64%
81%
1 00%
88%
72%
58%
47%
41%
36%
41%
51%
63%
75%
87%
91%
92%
86%
74%
62%
52%
43%
36%
31%
26%
32%
34%
44%
52%
63%
68%
76%
64%
50%
42%
35%
30%
28%
21%
Lab Torque
% of Peak
Torque
25%
30%
38%
50%
74%
63%
50%
38%
30%
25%
25%
30%
38%
50%
63%
76%
79%
80%
89%
76%
63%
50%
38%
30%
25%
25%
34%
38%
50%
63%
76%
86%
96%
99%
76%
63%
50%
41%
38%
25%
SwRI Report 03.12024.06
                   E-5

-------
            Table Type
            Engine Manufacturer
            Engine Model
            Model Year
            Serial Number
            Fuel Density
            Peak Torque
            nlo
            nhi
BSFC
INT
VT365
2006
     332259
        851 [g/L]
        681 [N-m]
       1198 [rpm]
       2839 [rpm]
Point
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
Raw Data
ECM Speed
rpm
2903
2890
2868
2832
2622
2621
2621
2621
2621
2620
2328
2328
2328
2328
2328
2328
2328
2328
2033
2033
2033
2032
2032
2032
2032
1739
1739
1739
1739
1739
1739
1739
1739
1444
1444
1443
1444
1444
1444
1444
Normalized Data
ECM Speed
nlo = 0%
nhi = 100%
104%
103%
102%
100%
87%
87%
87%
87%
87%
87%
69%
69%
69%
69%
69%
69%
69%
69%
51%
51%
51%
51%
51%
51%
51%
33%
33%
33%
33%
33%
33%
33%
33%
15%
15%
15%
15%
15%
15%
15%
ECM Fuel Rate
% of Maximum
Fuel Rate
47%
53%
64%
81%
100%
88%
72%
58%
47%
41%
36%
41%
51%
63%
75%
87%
91%
92%
86%
74%
62%
52%
43%
36%
31%
26%
32%
34%
44%
52%
63%
68%
76%
64%
50%
42%
35%
30%
28%
21%
BSFC
% of Max
100
94
87
82
77
78
80
85
91
98
91
86
80
76
74
73
73
73
68
69
70
72
77
82
88
84
75
73
71
68
68
66
66
64
65
65
68
71
72
82
SwRI Report 03.12024.06
             E-6

-------
                                APPENDIX F
                   STEADY-STATE ERROR SURFACE DATA
SwRI Report 03.12024.06

-------
                            I PEMS 5th % A PEMS 50th % • PEMS 95th % • Points
     25
Concentration Delta (ppm)

-* -* N)
O1 O O1 O
     -5
    -10
         10
                        15
                             .
                           24
                                                               33

               160
                       180
                               200
                                       220
                                                240
                                                        260
                                                                280
                                                                        300
                              Lab Reference Mean NOx Concentration (ppm)
 PEMS NOX CONCENTRATION DELTA VERSUS THE LABORATORY REFERENCE
                     FOR ENGINE 1 STEADY-STATE TESTING
                              Raw 5th % A Raw 50th % * Raw 95th % • Points
     25
  ?  20 -I
  Q.
  _a

  2  15 -\
  HI
  Q

  I  10 H
Concen
      5
      to

     -5 ---
    -10
         10
M
9__
                         5
24 *
 --3--
                                                               33
                A
               160
                       180
                               200
                                       220
                                                240
                                                        260
                                                                280
                                                                        300
                             Lab Reference Mean NOx Concentration (ppm)
     LAB RAW NOX CONCENTRATION DELTA VERSUS THE LABORATORY
              REFERENCE FOR ENGINE 1 STEADY-STATE TESTING
SwRI Report 03.12024.06
                         F-l

-------
    60
                            IPEMS 5th % A PEMS 50th % » PEMS 95th % • Points
    50 -
Q.

S40
ra

"oi
Q




I30

"E
HI
u

J 20


O
O
    10 --
               10
                      A  A
                                    A         A
                                                                   37


                                                                   »
                                                                   A   A
      10
                12
                           14          16          18         20


                             Lab Reference Mean CO Concentration (ppm)
                                                                      22
                                                                                 24
 PEMS CO CONCENTRATION DELTA VERSUS THE LABORATORY REFERENCE

                     FOR ENGINE 1 STEADY-STATE TESTING
    60
                             I Raw 5th % A Raw 50th % » Raw 95th % • Points
    50
  E
  Q.
  Q.


  S  30

  HI
  Q



  I  20 Ji
  HI
  u
  c
  o
  o

  O
  O
  10
               10
                     1524
                                    40
                                               27
                                                       33
                                                                   37
   -10
   -20
                    \U  i 14
                                      16
                                             i I1
                                                           20
                             Lab Reference Mean CO Concentration (ppm)





      LAB RAW CO CONCENTRATION DELTA VERSUS THE LABORATORY

              REFERENCE FOR ENGINE 1 STEADY-STATE TESTING
SwRI Report 03.12024.06
                                       F-2

-------
                            I PEMS 5th % A PEMS 50th % * PEMS 95th % • Points
    0.70
    0.60
    0.50
  HI
  Q
  c
  o
0.40
    0.30
    0.20
    0.10
    0.00
                                 27
                                         *to
                                         15
37*3

A
       56677889
                              Lab Reference Mean CO2 Concentration (%)

 PEMS CO2 CONCENTRATION DELTA VERSUS THE LABORATORY REFERENCE
                    FOR ENGINE 1 STEADY-STATE TESTING
                             I Raw 5th % A Raw 50th % » Raw 95th % • Points


"oi
Q
O
is
c
0
o
CM
o °-20 "
n nn -
4 ; ;; "



* A
A
ill;
i *
37*3
5


A

i
"
       5667788
                              Lab Reference Mean CO2 Concentration (%)

     LAB RAW CO2 CONCENTRATION DELTA VERSUS THE LABORATORY
              REFERENCE FOR ENGINE 1 STEADY-STATE TESTING
SwRI Report 03.12024.06
                                    F-3

-------
                           I PEMS 5th % A PEMS 50th % » PEMS 95th % • Points

"g"
Q.
Q.
TO
Q
c
0
IS
ncentr;
O
O
I
S
z

40


A r\ r\
^ o
• n ^

1-5
-9 n
37 33 * 27* 24*
15 19 3 5


A
* ^. A
0 0.1 0.1 A o.2 A 0.2 A^ AH.3 0
• " • * "



                            Lab Reference Mean NMHC Concentration (ppm)





      PEMS NMHC CONCENTRATION DELTA VERSUS THE LABORATORY

              REFERENCE FOR ENGINE 1 STEADY-STATE TESTING
                           I PEMS 5th % A PEMS 50th % • PEMS 95th % • Points
     30
     25
     20
     15
     10 H
      5
3   1!
TO

•5  -5
     -10
     -15
     -20
             40
                       37
                          27      24
                                                              33
                                              9-
               250
                       350
                               450
                                       550
                                               650
                                                       750
                                                               850
                                                                       950
                                                                               1050
                             Lab Reference Mean Exhaust Flowrate (scfm)
 PEMS EXHAUST FLOW RATE DELTA VERSUS THE LABORATORY REFERENCE

                     FOR ENGINE 1 STEADY-STATE TESTING
SwRI Report 03.12024.06
                                     F-4

-------
                                 15th % A 50th %  95th % • Points
     4 -
   I 2
   o
   3 1 -
            40
                       37
                          27
                                24
                                                              33
                                          10  9
                                                                 15
                                          i   I   :
                                                                         I
              250
                      350
                              450
                                       550
                                               650
                                                       750
                                                               850
                                                                       950
                                                                               1050
                             Lab Reference Mean Exhaust Flowrate (scfm)

     LAB RAW EXHAUST FLOW RATE DELTA VERSUS THE LABORATORY
              REFERENCE FOR ENGINE 1 STEADY-STATE TESTING
                            IPEMS 5th % A PEMS 50th % » PEMS 95th % « Points
  70

  60

_ 50

I
a 40
   I 30
   5
   .2 20
   10

   0

  -10

  -20
               40
                         37
                                                           33
                                          _15_
                     200
                                  400
                                                600
                                                              800
                                                                           1000
                             Lab Reference Mean NOx Mass Flow Rate (g/h)

 PEMS NOX MASS FLOW RATE DELTA VERSUS THE LABORATORY REFERENCE
                     FOR ENGINE 1 STEADY-STATE TESTING
SwRI Report 03.12024.06
                                     F-5

-------
                            Raw 5th % A Raw 50th % • Raw 95th % • Points


1
"oi
Q
HI
owRa
u.
to
to
TO
X
O








0


37t4 ; • *
T^ 9 3



ttf
I if r r I t
200 400 600 800

•





A
^000

                            Lab Reference Mean NOx Mass Flow Rate (g/h)

     LAB RAW NOX MASS FLOW RATE DELTA VERSUS THE LABORATORY
             REFERENCE FOR ENGINE 1 STEADY-STATE TESTING
SwRI Report 03.12024.06
F-6

-------
                             PEMS 5th % A PEMS 50th % • PEMS 95th % • Points
     25
     20  -
     15
pp

_i
O
  3   5
  c
  o
  i   °
   "   c
   o  -5
   o
   o

   I -10

    -15
    -20
                  40   39
         TO
                              .21-   	.20-
                                               36
                                                               -1-9-
                                                                           34
               15CA     200      J50      3QB       350      400      450      500      5!
                              Lab Reference Mean NOx Concentration (ppm)



 PEMS NOX CONCENTRATION DELTA VERSUS THE LABORATORY REFERENCE

                     FOR ENGINE 2 STEADY-STATE TESTING
                              Raw 5th % A Raw 50th % • Raw 95th % • Points
     20  -
     15

  ?
  3 10

  i"
  &  5
  c
  o
  «  0
  r  100
   o
   O
   x -10
   O
    -15
    -20
                  40   39
          TO
                              .21-   	.20-
                                               36
                                                               -1-9-
                                                                           34
•    15(§ ™   200      250      300      350      400       450      500       5!


                    -*-                                          1
                              Lab Reference Mean NOx Concentration (ppm)



      LAB RAW NOX CONCENTRATION DELTA VERSUS THE LABORATORY

              REFERENCE FOR ENGINE 2 STEADY-STATE TESTING
SwRI Report 03.12024.06
                                F-7

-------
                            I PEMS 5th % A PEMS 50th % » PEMS 95th % • Points
  80


  70


  60


  50


  40


  30


  20


  10


   0


  -10
                                19
                                               34
                                                         36    39      40
                             Lab Reference Mean CO Concentration (ppm)
 PEMS CO CONCENTRATION DELTA VERSUS THE LABORATORY REFERENCE
                     FOR ENGINE 2 STEADY-STATE TESTING
                              Raw 5th % A Raw 50th % » Raw 95th % • Points
    80
    70
    60
  Q.
  3 50
    40
    30
HI
Q
c
o
ra

HI
o

I 20
O
O
    10
                                19
                                               34
                                                         36    39
                                                                     40
                                                     5    1    |
                                                                    |6
   -10
                             Lab Reference Mean CO Concentration (ppm)


      LAB RAW CO CONCENTRATION DELTA VERSUS THE LABORATORY
              REFERENCE FOR ENGINE 2 STEADY-STATE TESTING
SwRI Report 03.12024.06
                                       F-8

-------
n en
CO2 Concentration Delta (%)
D O O O O O C
D ^ k> ijo !^ 01 i:
D O O O O O C
4
PEMS<
0.60
0.50
:= 0.40
ra
"oi
Q
c 0.30
0
TO
g 0.20
c
0
o
CM
8 0-1°
0.00
-n m
• PEMS 5th % A PEMS 50th % * PEMS 95th % • Points

* " • •
• 40 * 39 * 86 *34
10 13 21 20 g» 19
*
A
* * •
•
A *
•---*---: •
» A •
A
A A •
•

5566778899
Lab Reference Mean CO2 Concentration (%)
CO2 CONCENTRATION DELTA VERSUS THE LABORATORY REFERENCE
FOR ENGINE 2 STEADY-STATE TESTING
• Raw 5th % A Raw 50th % * Raw 95th % • Points

* • • •
. • 40 * 39 * 36 *34
10 13 21 2° 5 19


•
A
i
"*"" "•"
A
t " i i I
\ *5 m 6 6«7i7*8 8 9 !

                            Lab Reference Mean CO2 Concentration (%)

     LAB RAW CO2 CONCENTRATION DELTA VERSUS THE LABORATORY
             REFERENCE FOR ENGINE 2 STEADY-STATE TESTING
SwRI Report 03.12024.06
F-9

-------
                            IPEMS 5th % A PEMS 50th % » PEMS 95th % • Points
   a.
   o.
   HI
   Q

   c
   o
   c
   HI
   o
   c
   o
   o

   o
   I
                                                                               -6*
                                                                               -SrO"
               40
                              39
                                                -36--
                                                                               -4r&
                                                             34
                                                                               -SrO"
                                                                               -2rO"
                                      --ka
                                                                               -OrS-
             -0.8
                      -0.7
                              -0.6
                                     10
                                       -0.5
                                               -0.4
                                                        -0.3
                                                                -0.2
                                                                         -0.1
                                                                                  00
                            Lab Reference Mean NMHC Concentration (ppm)





      PEMS NMHC CONCENTRATION DELTA VERSUS THE LABORATORY

              REFERENCE FOR ENGINE 2 STEADY-STATE TESTING
                             I PEMS 5th % A PEMS 50th %  PEMS 95th % • Points
      50
      40
   £.  30
   HI

   Q

   HI
      20
      10
     -10
                40    39
                              36
                                          34
                                                       1ft)
                                                           21 299
                                      •

                                      5
                  250
                             350
                                        450
                                                  550
                                                             650
                                                                        750
                               Lab Reference Mean Exhaust Flowrate (scfm)
 PEMS EXHAUST FLOW RATE DELTA VERSUS THE LABORATORY REFERENCE

                      FOR ENGINE 2 STEADY-STATE TESTING
SwRI Report 03.12024.06
F-10

-------
     5.0
                                   5th % A 50th % *95th % • Points
     4.5
     4.0
     3.5
   i 3.0

   Q

   HI
     2.5
   _o
   u.
3
re
.c
     2.0
     1.5
     1.0 -
     0.5
     0.0
                40
                      39
                              36
                                          34
       150
                  250
                             350        450        550         650


                               Lab Reference Mean Exhaust Flowrate (scfm)
                                                                        750
                                                                                   850
      LAB RAW EXHAUST FLOW RATE DELTA VERSUS THE LABORATORY

              REFERENCE FOR ENGINE 2 STEADY-STATE TESTING
      70
   £

   3
   re
   a
   HI
   Q
   HI
   •re
   OL


   o
   O
   60



   50



   40



   30



   20



   10



   0



  •10
     -20
     -30
                             PEMS 5th % A PEMS 50th % • PEMS 95th %  • Points
                 40  39
                                    36
                       ~10~
                                          _2J_
                                                    ^20-
                                                              .34
                                                                         -19-
                     200
                                    400
                                                  600
                                                                800
                                                                              1000
                              Lab Reference Mean NOx Mass Flow Rate (g/h)
 PEMS NOX MASS FLOW RATE DELTA VERSUS THE LABORATORY REFERENCE

                     FOR ENGINE 2 STEADY-STATE TESTING
SwRI Report 03.12024.06
                                      F-ll

-------
                             I Raw 5th % A Raw 50th % * Raw 95th % • Points
     70

     60

     50

     40

     30

     20

     10

      0

     -10

     -20

     -30
40   39  .     •      36
       10	13--
                              -20-
                                        .34
                                                  -19-
200
       l~~l"
              400
                            600
                                          800
                                                        1000
                                                   I
                              Lab Reference Mean NOx Mass Flow Rate (g/h)

     LAB RAW NOX MASS FLOW RATE DELTA VERSUS THE LABORATORY
              REFERENCE FOR ENGINE 2 STEADY-STATE TESTING
SwRI Report 03.12024.06
                    F-12

-------
                             IPEMS 5th % A PEMS 50th % » PEMS 95th % • Points
      40
      20
  I  0

  f   '
  HI
  Q  -20
  o
  're
   o
  O
     -60
     -80
    -100
            25*

             6
                                          40
       16  20     19
                                                                  37  35
»»


/  A**
^A-
       140
                    340
                                440
                                            540
                                                         640
                                                                     7.0
                              Lab Reference Mean NOx Concentration (ppm)
 PEMS NOX CONCENTRATION DELTA VERSUS THE LABORATORY REFERENCE

                     FOR ENGINE 3 STEADY-STATE TESTING
                              I Raw 5th % A Raw 50th % » Raw 95th % • Points
      40
      20
   I  0
   c
   o
  o
     -20
     -60
     -80
    -100
                2¥16  20     19
                                          40
                                                                  37  35
                   240
                                340
                                            440
                                                        540
                                                      I  4
                                                                                 7. 0
                              Lab Reference Mean NOx Concentration (ppm)
     LAB RAW NOX CONCENTRATION DELTA VERSUS THE LABORATORY

              REFERENCE FOR ENGINE 3 STEADY-STATE TESTING
SwRI Report 03.12024.06
                             F-13

-------
                            I PEMS 5th % A PEMS 50th % » PEMS 95th % « Points
     140
     120 -
     100 -
   a.
     80 -
   I 60 4
   o 40 -

   o
   O

   £ 20 -
     -20 -
     -40
       25*
       --&-
                   16  20
                                          40
                           19
                                                                 37  35
       140
                   ^40
                               340
                                            440
                                                        540
                                                                    64fl

                                                                  •
                                                                                7. 0
                                Lab Reference NOx Concentration (ppm)
 HORIBA OBS-2200 NOX CONCENTRATION DELTA VERSUS THE LABORATORY

              REFERENCE FOR ENGINE 3 STEADY-STATE TESTING
                             I PEMS 5th % A PEMS 50th % » PEMS 95th % • Points
    120
    100 J
  Q.
  Q.
  HI
  Q

  c
  o
  HI
  o
  c
  o
  o

  o
  O
    80
60
    40
20
           25    22
                                    19




                                     •
                                                           37
                                                                           35
                               10
                                            12
                                                        14
                                                                     16
                                                                                 13
    -20
                              Lab Reference Mean CO Concentration (ppm)
 PEMS CO CONCENTRATION DELTA VERSUS THE LABORATORY REFERENCE

                     FOR ENGINE 3 STEADY-STATE TESTING
SwRI Report 03.12024.06
                                    F-14

-------
                             I Raw 5th % A Raw 50th % * Raw 95th % • Points
   120
   100
  I 80 ^
 "oi
 Q  60
 c
 o
 •£  40
 HI
 u
 c
 o
 o

 O  20
 O
           25     22
                  .40


                  6
                               2?6
                                     30
                                        19
                                                           37
                                                                           35
                               10
                                           12
                                                        14
                                                                    16
                                                                                 1 3
    -20
                             Lab Reference Mean CO Concentration (ppm)




      LAB RAW CO CONCENTRATION DELTA VERSUS THE LABORATORY

              REFERENCE FOR ENGINE 3 STEADY-STATE TESTING
                            IPEMS 5th % A PEMS 50th % • PEMS 95th % • Points
     150
     100
      50
   a.
   _a

   c
   o
c   "
HI
o
c
o
o

O  -50
O
     -100
                   2
                  *
                    -40
                    *
                                                           37
                                                                           35
                                10
                                            12
                                                        14
                                                                    16
                                                                                 13
                                Lab Reference CO Concentration (ppm)





  HORIBA OBS-2200 CO CONCENTRATION DELTA VERSUS THE LABORATORY

              REFERENCE FOR ENGINE 3 STEADY-STATE TESTING
SwRI Report 03.12024.06
                                     F-15

-------
n en ^
CO2 Concentration Delta (%)
D O O O O O C
D ^ ki ijo !^ 01 i:
D O O O O O C
• PEMS 5th % A PEMS 50th % * PEMS 95th % • Points

25 37 22 ' 2016 3519 3°
b
^« *
* * A
• A A
AA B
A A A .. •



5.0 5.5 6.0 6.5 7.0 7.5 8.0 8.5 9.0
Lab Reference Mean CO2 Concentration (%)
PEMS CO2 CONCENTRATION DELTA VERSUS THE LABORATORY REFERENCE
FOR ENGINE 3 STEADY-STATE TESTING
n en ^
CO2 Concentration Delta (%)
D O O O O O C
D ^ ki ijo !^ 01 i:
D O O O O O C
• Raw 5th % A Raw 50th % * Raw 95th % • Points

• . * .
37 . 35 -,n
25 22 c 2016 19



* * * *
• • * AA Al •
• A 4 "• |


5.0 5.5 6.0 6.5 7.0 7.5 8.0 8.5 9.0
                            Lab Reference Mean CO2 Concentration (%)

     LAB RAW CO2 CONCENTRATION DELTA VERSUS THE LABORATORY
             REFERENCE FOR ENGINE 3 STEADY-STATE TESTING
SwRI Report 03.12024.06
F-16

-------
                            I PEMS 5th % A PEMS 50th % » PEMS 95th % « Points
     1.0
     0.8 -
     0.6 -
   IS °-4 -
   c
   o

   is 0.2 -
     0.0
       40


     -0.2 -
     -0.4 -
     -0.6 -
     -0.8
              40
                                    25
                                          37
                                                     22
                           35

                        2016  19
              4.5
                      5.0
                             5.5
                                     6.0
                                            6.5
                                                   7.0
                                                           7.5
                                                                  8.0
                                                                         8.5
                                                                                 90
                                 Lab Reference CO2 Concentration (%)
 HORIBA OBS-2200 CO2 CONCENTRATION DELTA VERSUS THE LABORATORY

              REFERENCE FOR ENGINE 3 STEADY-STATE TESTING
                            \ PEMS 5th % A PEMS 50th % * PEMS 95th % • Points
%
25 30 22
•
* 10-
A* n ^4
00*
• 0.0
.3 -0.2 -0.1 C
-rv^-
1 0-
1 5-

• •
40 35 37
5 19
6"
•
• •
* *
k A
A A ' A '
b • 0.1> 0.2 4l.3 0.4 0
"* •


   Q.
   0>
   Q

   c
   o
   
-------
     2.0
     1.5
   O


   Q.
   _a

   C
   O
   c
   HI
   o
   c
   o
   o
   o
     0.0
       00
     -0.5
                             PEMS 5th % A PEMS 50th % * PEMS 95th % • Points
                        •tn
                        30
   3
19 2$
37
22
                                        • •
                      0.5
                                     1.0
                                                   1.5
                               Lab Reference THC Concentration (ppmC)
                                                                 40
                                                                  2.0

                                                                                 25
 HORIBA OBS-2200 THC CONCENTRATION DELTA VERSUS THE LABORATORY

              REFERENCE FOR ENGINE 3 STEADY-STATE TESTING
     60
                            I PEMS 5th % A PEMS 50th %  PEMS 95th % • Points
     50  -
   £. 40
   re
   a
   HI
   Q
   HI
   re 30


   o
     20  -
     10 --
          40
               25
                 37
                       3530
                                  22
                        • *

                        A>
                                                    20
                                                            19
                                                                    16
      150
                   200
                               250          300          350

                              Lab Reference Mean Exhaust Flowrate (scfm)
                                                                    400
                                                                                 450
 PEMS EXHAUST FLOW RATE DELTA VERSUS THE LABORATORY REFERENCE

                     FOR ENGINE 3 STEADY-STATE TESTING
SwRI Report 03.12024.06
            F-18

-------
  10

   9
   I  7
   to
   TO
   ?  6
   Q
   HI
   2  5

   o
   il  4
   1  3
                                 15th % A 50th %  95th % • Points
      1
          40
               25
                37
                       3530
                                 22
                                                   -20--
                       ft ft
                                                                  -16--
      150
                  200
                               250          300          350
                              Lab Reference Mean Exhaust Flowrate (scfm)
                                                                   400
                                                                               450
     LAB RAW EXHAUST FLOW RATE DELTA VERSUS THE LABORATORY
              REFERENCE FOR ENGINE 3 STEADY-STATE TESTING
u
to
0)
ra   1!
          40
                            IPEMS 5th % A PEMS 50th %  PEMS 95th % • Points
               25
                 37
                                 22_
                                                   _2Q_
                                                          -13-
                   200
                       A A
                               250
                                           300
                                                       350
                                Lab Reference Exhaust Flowrate (scfm)
                                                                  -16-
                                                                   400
 HORIBA OBS-2200 EXHAUST FLOW RATE DELTA VERSUS THE LABORATORY
              REFERENCE FOR ENGINE 3 STEADY-STATE TESTING
SwRI Report 03.12024.06
                                     F-19

-------
                             IPEMS 5th % APEMS 50th % *PEMS 95th % • Points
      70
      60 -
      50 -
   3  40 -
   HI
   Q
   .2
   re
   OL
   5
30  -
20  -
      10 --
     -10 --
     -20 -
     -30
                25
                         30
                                    40    .
                            _22-
                                             --20-^6-
                                                          -19-
                                                              37
                                                                           35
                         A   A
       53      100       150      200      250       300      350      400       450      5(
                              Lab Reference Mean NOx Mass Flow Rate (g/h)
 PEMS NOX MASS FLOW RATE DELTA VERSUS THE LABORATORY REFERENCE
                     FOR ENGINE 3 STEADY-STATE TESTING
                              I Raw 5th % A Raw 50th % * Raw 95th % • Points
      70
60   -     --*>--     ~ -22-.        --„--  --20-H6-
      50
   S  40
   HI
   Q
   HI
   •re
   OL

   o
30
20
      10
     -10
     -20
     -30
                25
                         30
                                    40
                                                          -19-
                                                              37
                                                                           35
                *•   i      1   Ii^^i
       53      100      150      200      250       300      350   •  400      45*      5i
                              Lab Reference Mean NOx Mass Flow Rate (g/h)
     LAB RAW NOX MASS FLOW RATE DELTA VERSUS THE LABORATORY
              REFERENCE FOR ENGINE 3 STEADY-STATE TESTING
SwRI Report 03.12024.06
                                   F-20

-------
     100
     80
     60
     40
   E 20
     -20
     -40 -
     -60
                25
               100
                            I PEMS 5th % A PEMS 50th % » PEMS 95th % « Points
                        30
                            22
                                   40
                                             20  16
                                                         19
                                                             37
                         A  A      A
                       150      200      250
                                                300
                                                        350      400
                                Lab Reference NOx Mass Flowrate (g/h)
                                                                        450
                                                                                5(10
 HORIBA OBS-2200 NOX MASS FLOW RATE DELTA VERSUS THE LABORATORY
              REFERENCE FOR ENGINE 3 STEADY-STATE TESTING
SwRI Report 03.12024.06
F-21

-------
                             •95th percentile —A—50th percentile (median) —•—5th percentile
      40
      20     "
Q.
n.


~
HI
Q


I  -20

is

"E
HI

|  -40 ^

O
       11
                     200
                                                 400
                                                                500
                                                                              600
  LU

  Q.
     -60
     -80
    -100
                                  Lab Reference Mean NOx Concentration [ppm]
    ENGINE 1 ERROR SURFACE FOR STEADY-STATE NOX CONCENTRATION
                             -95th percentile —A—50th percentile (median) —•—5th percentile
    -100
                                  Lab Reference Mean NOx Concentration [ppm]
    ENGINE 2 ERROR SURFACE FOR STEADY-STATE NOX CONCENTRATION
SwRI Report 03.12024.06
                                            F-22

-------
                            •95th percentile —A—50th percentile (median) —•—5th percentile
    -100
                                 Lab Reference Mean NOx Concentration [ppm]
    ENGINE 3 ERROR SURFACE FOR STEADY-STATE NOX CONCENTRATION
                            -95th percentile —A—50th percentile (median) —•—5th percentile
    -100
                                 Lab Reference Mean NOx Concentration [ppm]
      FINAL ERROR SURFACE FOR STEADY-STATE NOX CONCENTRATION
SwRI Report 03.12024.06
F-23

-------
     0.010
     0.009 -
  „ 0.008 -
  =  0.007
  HI
  Q
0.006

0.005

0.004

0.003

0.002

0.001 -
     0.000
        0.0000
                               -95th percentile
                                           -50th percentile (median)
-5th percentile
                                                        >-*-
                                                                                      -A-A
                          0.0005             0.0010             0.0015
                                      Lab Reference Mean CO Concentration [%]
                                                                                0.0020
                                                                                                  0.0025
      ENGINE 1 ERROR SURFACE FOR STEADY-STATE CO CONCENTRATION
                               -95th percentile
                                           -50th percentile (median)
-5th percentile
     0.010
     0.009

  „ 0.008
  g

  I  °-007
  Q
  o  0.006
  ra
  |  0.005
  o
  c
  O  0.004
  O
  W  0.003
     0.002 -
     0.001  -
     0.000
        0.0000
                          0.0005             0.0010             0.0015
                                      Lab Reference Mean CO Concentration [%]
                                                                                0.0020
                                                                                                  0.0025
      ENGINE 2 ERROR SURFACE FOR STEADY-STATE CO CONCENTRATION
SwRI Report 03.12024.06
                                             F-24

-------
     0.010
     0.009 -
  „ 0.008
  =  0.007
  HI
  Q
  o  0.006
     0.005
  u
  c
  O  0.004
  O
  W  0.003
     0.002
     0.001 -
     0.000
        0.0000
                               -95th percentile
                                              -50th percentile (median)
-5th percentile
                          0.0005             0.0010             0.0015
                                       Lab Reference Mean CO Concentration [%]
                                                                                 0.0020
                                                                                                   0.0025
      ENGINE 3 ERROR SURFACE FOR STEADY-STATE CO CONCENTRATION
     0.010
   0.009

„ 0.008
g

I  °-007
Q
o  0.006
TO
|  0.005
o
c
O  0.004
O
W  0.003
s
LU
°-  0.002 -
     0.001 -
     0.000
        0.0000
                               -95th percentile
                                              -50th percentile (median)
-5th percentile
                                  A  A
                                                        MA  A
                          0.0005             0.0010             0.0015
                                       Lab Reference Mean CO Concentration [%]
                                                                                       •*—*
                                                                                 0.0020
                                                                                                   0.0025
        FINAL ERROR SURFACE FOR STEADY-STATE CO CONCENTRATION
SwRI Report 03.12024.06
                                                F-25

-------
     0.7
                               -95th percentile
•50th percentile (median)
-5th percentile
     0.6
  £  0.5
  c
  o

  ''a  0.4

  "E
  HI
  u

  o  0.3
  O
  CM
  O
  O
    0.2
  LU
  Q.
     0.1
     0.0
       4.0       4.5       5.0       5.5       6.0       6.5       7.0       7.5

                                     Lab Reference Mean CO2 Concentration [%]
                                                                                  8.0
                                                                                           8.5
                                                                                                    9.0
     ENGINE 1 ERROR SURFACE FOR STEADY-STATE CO2 CONCENTRATION
     0.7
                               -95th percentile —A—50th percentile (median) —•—5th percentile
     0.6
  £  0.5
  HI
  Q
  c
  o
  is  o.4

  "c
  HI
  u

  o  0.3
  O
  CM
  O
  O
    0.2
  LU
  Q.
     0.1  -
     0.0
       4.0       4.5       5.0       5.5       6.0       6.5       7.0       7.5

                                     Lab Reference Mean CO2 Concentration [%]
                                                                                  8.0
                                                                                           8.5
                                                                                                    9.0
     ENGINE 2 ERROR SURFACE FOR STEADY-STATE CO2 CONCENTRATION
SwRI Report 03.12024.06
  F-26

-------
     0.7
                               -95th percentile
•50th percentile (median)
-5th percentile
     0.6
  £  0.5
  c
  o

  ''a  0.4

  "E
  HI
  u

  o  0.3
  O
  CM
  O
  O
    0.2
  LU
  Q.
     0.1
     0.0
       4.0       4.5       5.0       5.5       6.0       6.5       7.0       7.5

                                     Lab Reference Mean CO2 Concentration [%]
                                                                                  8.0
                                                                                           8.5
                                                                                                    9.0
     ENGINE 3 ERROR SURFACE FOR STEADY-STATE CO2 CONCENTRATION
     0.7
                               -95th percentile —A—50th percentile (median) —•—5th percentile
     0.6
  £  0.5
  HI
  Q
  c
  o
  is  o.4

  "c
  HI
  u

  o  0.3
  O
  CM
  O
  O
    0.2
  LU
  Q.
     0.1  -
     0.0
       4.0       4.5       5.0       5.5       6.0       6.5       7.0       7.5

                                     Lab Reference Mean CO2 Concentration [%]
                                                                                  8.0
                                                                                           8.5
                                                                                                    9.0
       FINAL ERROR SURFACE FOR STEADY-STATE CO2 CONCENTRATION
SwRI Report 03.12024.06
  F-27

-------
                                       *95th percentile

                                       A50th percentile (median)

                                       • 5th percentile

0
Q.
£:
ra
"3
Q
c
o
"E
HI
c
o
o
s
(/)
LU
n


4






i
n n

>






1
                                       Single X-Axis Point



     FINAL ERROR SURFACE FOR STEADY-STATE NMHC CONCENTRATION

                                 (ENGINE 2 DATA ONLY)
                           -95th percentile
-50th percentile (median)
-5th percentile
  0  6
  £
  "oi
  Q


  I  4
  &


  O
     2 -
  UJ
  Q.
    -2
                                                          50
                                                                     60
                                                                                70
                                                                                          83
                               Lab Reference Mean Exhaust Flow Rate [% of Max]
    ENGINE 1 ERROR SURFACE FOR STEADY-STATE EXHAUST FLOW RATE
SwRI Report 03.12024.06
  F-28

-------
                            -95th percentile
-50th percentile (median)
-5th percentile
                               Lab Reference Mean Exhaust Flow Rate [% of Max]




     ENGINE 2 ERROR SURFACE FOR STEADY-STATE EXHAUST FLOW RATE
                            -95th percentile
-50th percentile (median)
-5th percentile
     8 -
  0  6
  "oi
  Q


  I
  OL


  o
     2 -
  LU  0
  Q.
    -2
                 10
                           20
                                      30
                                                 40
                                                            50
                                                                      60
                                                                                 70
                               Lab Reference Mean Exhaust Flow Rate [% of Max]
     ENGINE 3 ERROR SURFACE FOR STEADY-STATE EXHAUST FLOW RATE
SwRI Report 03.12024.06
  F-29

-------
                           -95th percentile
-50th percentile (median)
-5th percentile
                               Lab Reference Mean Exhaust Flow Rate [% of Max]
      FINAL ERROR SURFACE FOR STEADY-STATE EXHAUST FLOW RATE
SwRI Report 03.12024.06
  F-30

-------
                               APPENDIX G
                    TRANSIENT ERROR SURFACE DATA
SwRI Report 03.12024.06

-------
                                                                                   800
                                 PEMS Median Flow-Weighted NOx (ppm)
      ENGINE 1 ERROR SURFACE FOR TRANSIENT NOX CONCENTRATION
     50





     40





     30





     20





     10





      0





     -10





     -20





     -30





     -40





     -50
- 95th % Delta
            -5th% Delta
                                 PEMS Median Flow-Weighted NOx (ppm)
      ENGINE 2 ERROR SURFACE FOR TRANSIENT NOX CONCENTRATION
SwRI Report 03.12024.06
                            G-l

-------
                                                                               800
                               PEMS Median Flow-Weighted NOx (ppm)
      ENGINE 3 ERROR SURFACE FOR TRANSIENT NOX CONCENTRATION
                               PEMS Median Flow-Weighted NOx (ppm)
        FINAL ERROR SURFACE FOR TRANSIENT NOX CONCENTRATION
SwRI Report 03.12024.06
G-2

-------
                                                                                              0.0080
     -0.0025
                                   PEMS Median Flow-Weighted CO Concentration (%)
        ENGINE 1 ERROR SURFACE FOR TRANSIENT CO CONCENTRATION
      0.0025 -,
      0.0020
      0.0015
   3  0.0010
   g  0.0005
   I,  o.oooo
         0.0 )40

     -0.0005  -
     -0.0010
   LU
   Q.
     -0.0015
     -0.0020
     -0.0025 J
                   - 95th % Delta
                                 -5th% Delta
                    0.0045
                                                    0.0060
                                                              0.0065
                                                                         0.0070
                                                                                   0.0075
                                                                                              0.0080
                                   PEMS Median Flow-Weighted CO Concentration (%)
        ENGINE 2 ERROR SURFACE FOR TRANSIENT CO CONCENTRATION
SwRI Report 03.12024.06
G-3

-------
                                                                                    0.0080
     -0.0025
                               PEMS Median Flow-Weighted CO Concentration (%)
       ENGINE 3 ERROR SURFACE FOR TRANSIENT CO CONCENTRATION
                                                                                    0.0080
     -0.0025 J
                               PEMS Median Flow-Weighted CO Concentration (%)
         FINAL ERROR SURFACE FOR TRANSIENT CO CONCENTRATION
SwRI Report 03.12024.06
G-4

-------
   o
   o
   I
   g>
   I
   i
      0.5
      0.4 --
      0.3 -
      0.2 -
      0.1 --
0.0
 4.0000
-0.1 ---
     -0.2 -
   LU
   Q.
     -0.3
     -0.4 - -
     -0.5
                -95th % Delta —•— 5th % Delta
5.0000
                                    8.0000       9.0000       10.0000       11.0000
                                PEMS Median Flow-Weighted CO2 Concentration (%)
       ENGINE 1 ERROR SURFACE FOR TRANSIENT CO2 CONCENTRATION
                                                                                           11.0000
     -0.5
                                PEMS Median Flow-Weighted CO2 Concentration (%)
       ENGINE 2 ERROR SURFACE FOR TRANSIENT CO2 CONCENTRATION
SwRI Report 03.12024.06
                                          G-5

-------
                                                                                   11.0000
     -0.5
                             PEMS Median Flow-Weighted CO2 Concentration (%)
       ENGINE 3 ERROR SURFACE FOR TRANSIENT CO2 CONCENTRATION
     0.50
                                                                                   11.0000
     -0.50
                              PEMS Median Flow-Weighted CO2 Concentration (%)
        FINAL ERROR SURFACE FOR TRANSIENT CO2 CONCENTRATION
SwRI Report 03.12024.06
G-6

-------
            -95th% Delta
                         -5th% Delta
                                           -1-.5
                                           -1-.9
                                           -8.5
                                           -GrS-
                          Engine 2 Median Flow-Weighted NMHC Concentration (ppmC)

 FINAL ERROR SURFACE FOR TRANSIENT NMHC CONCENTRATION (ENGINE 2
                                      DATA ONLY)
                                 PEMS Median Exhaust Flow Rate (% of Max)
      ENGINE 1 ERROR SURFACE FOR TRANSIENT EXHAUST FLOW RATE
SwRI Report 03.12024.06
G-7

-------
                              PEMS Median Exhaust Flow Rate (% of Max)
      ENGINE 2 ERROR SURFACE FOR TRANSIENT EXHAUST FLOW RATE
                              PEMS Median Exhaust Flow Rate (% of Max)
      ENGINE 3 ERROR SURFACE FOR TRANSIENT EXHAUST FLOW RATE
SwRI Report 03.12024.06
G-8

-------
                                 PEMS Median Exhaust Flow Rate (% of Max)
        FINAL ERROR SURFACE FOR TRANSIENT EXHAUST FLOW RATE
    120
     90 -
               - 95th % Delta
                            -5th% Delta
               200
                        400
                                 600
                                          800
                                                  1000
                                                           1200
                                                                   1400
                                                                            1600
                                                                                     1800
  LU
  Q.
     -30 -
     -60 -
     -90 -
    -120
                                  PEMS Median NOx Mass Flow Rate (g/h)


      ENGINE 1 ERROR SURFACE FOR TRANSIENT NOX MASS FLOW RATE
SwRI Report 03.12024.06
G-9

-------
    120
                                                                                 1830
    -120
                                PEMS Median NOx Mass Flow Rate (g/h)





      ENGINE 2 ERROR SURFACE FOR TRANSIENT NOX MASS FLOW RATE
    120
                               600
                                       800
                                               1000
                                                        1200
                                                                1400
                                                                        1600
                                                                                 1800
    -120
                                PEMS Median NOx Mass Flow Rate (g/h)





      ENGINE 3 ERROR SURFACE FOR TRANSIENT NOX MASS FLOW RATE
SwRI Report 03.12024.06
G-10

-------
    120
                                                                                      1830
    -120
                                  PEMS Median NOx Mass Flow Rate (g/h)




        FINAL ERROR SURFACE FOR TRANSIENT NOX MASS FLOW RATE
     2.0
     1.5 -
     1.0 --
     0.5 -
re



'o


IS  0.0

~
o>
Q


I
OL
  « -1.0 -
               -95th % Delta —•— 5th % Delta
                30
                                                                     90
                                                                              100
    -0.5
    -1.5 -
    -2.0 -
    -2.5 -
    -3.0
                                    PEMS Median Fuel Rate (% of Max)




          ENGINE 1 ERROR SURFACE FOR TRANSIENT ECM FUEL RATE
SwRI Report 03.12024.06
                                         G-ll

-------
     2.0
     1.5  -
     1.0 --
     0.5 -I
     0.0
               30
                        40
                                                 70
                                                          80
                                                                  90
                                                                          100
    -0.5
    -1.0 --
    -1.5  -
    -2.0  -
    -2.5  -
    -3.0
                                   PEMS Median Fuel Rate (% of Max)





         ENGINE 2 ERROR SURFACE FOR TRANSIENT ECM FUEL RATE
                                   PEMS Median Fuel Rate (% of Max)





         ENGINE 3 ERROR SURFACE FOR TRANSIENT ECM FUEL RATE
SwRI Report 03.12024.06
G-12

-------
  2
  HI
  LU
  Q.
    0.8
    0.6
    0.4  -
    0.2
                                    PEMS Median Fuel Rate (% of Max)


            FINAL ERROR SURFACE FOR TRANSIENT ECM FUEL RATE
g.
TO

&

1!
&
               -95th % Delta —•— 5th % Delta
  ?    '
  I -0.2 4
  OT
    -0.4
    -0.6
    -0.8
                              PEMS Median Torque-Weighted Normalized Speed (%)


             ENGINE 1 ERROR SURFACE FOR TRANSIENT ECM SPEED
SwRI Report 03.12024.06
                                          G-13

-------
    -0.8
    0.8
    0.6
    0.4  -
    0.2
                              PEMS Median Torque-Weighted Normalized Speed (%)


             ENGINE 2 ERROR SURFACE FOR TRANSIENT ECM SPEED
&
1!
&
               - 95th % Delta
                             - 5th % Delta
  ?    c
  I -0.2 4
  OT
    -0.4
    -0.6
    -0.8
                              PEMS Median Torque-Weighted Normalized Speed (%)


             ENGINE 3 ERROR SURFACE FOR TRANSIENT ECM SPEED
SwRI Report 03.12024.06
                                           G-14

-------
    -0.8
                             PEMS Median Torque-Weighted Normalized Speed (%)





              FINAL ERROR SURFACE FOR TRANSIENT ECM SPEED
               95th % Delta —•— 5th % Delta
    -4.0
                               PEMS Median Speed-Weighted Torque (% of Max)





     ENGINE 1 ERROR SURFACE FOR TRANSIENT INTERPOLATED TORQUE
SwRI Report 03.12024.06
G-15

-------
    -4.0
                              PEMS Median Speed-Weighted Torque (% of Max)





     ENGINE 2 ERROR SURFACE FOR TRANSIENT INTERPOLATED TORQUE
               95th % Delta —•— 5th % Delta
    -4.0
                              PEMS Median Speed-Weighted Torque (% of Max)





     ENGINE 3 ERROR SURFACE FOR TRANSIENT INTERPOLATED TORQUE
SwRI Report 03.12024.06
G-16

-------
    -4.0
                              PEMS Median Speed-Weighted Torque (% of Max)





      FINAL ERROR SURFACE FOR TRANSIENT INTERPOLATED TORQUE
               95th % Delta —•— 5th % Delta
                                             260
                                                         280
                                                                      300
                                                                                  s::o
                               PEMS Median Fuel-Weighted BSFC (g/kW-hr)
       ENGINE 1 ERROR SURFACE FOR TRANSIENT INTERPOLATED BSFC
SwRI Report 03.12024.06
G-17

-------
     4.0
     3.0 -




   i-
   1
   Q 1.0
   o
m

T3


I
     0.0
              -95th % Delta —•— 5th % Delta
       200
                                                         280
   |-1.0
                                                                      300
                                                                                   s::o
     -2.0
                             V              V         ^
   LU

   Q.
     -3.0 -
     -4.0
                                PEMS Median Fuel-Weighted BSFC (g/kW-hr)
       ENGINE 2 ERROR SURFACE FOR TRANSIENT INTERPOLATED BSFC
               95th % Delta —•— 5th % Delta
                                                                                   s::o
                                PEMS Median Fuel-Weighted BSFC (g/kW-hr)
       ENGINE 3 ERROR SURFACE FOR TRANSIENT INTERPOLATED BSFC
SwRI Report 03.12024.06
                                       G-18

-------
                                                                                s::o
                               PEMS Median Fuel-Weighted BSFC (g/kW-hr)
        FINAL ERROR SURFACE FOR TRANSIENT INTERPOLATED BSFC
SwRI Report 03.12024.06
G-19

-------
                              APPENDIX H
         INTERACTING PARAMETERS - DOE ERROR SURFACE DATA
SwRI Report 03.12024.06

-------

	 uri- miei i emp aego — uri- uuuei i emp aego — ^sian IN i c ^.one - i argei co i



400
350
JJ 300
O)
HI
S 250
HI
3
1 200
Q.
|2 150
100
50
n

/
I
1
I
\\
1
TT*
t^

^*~- — i
^ 	 ~[





	 _ 	 _ — __ — — 	 _^
r
        0      200     400     600     800     1000     1200    1400     1600    1800    2000
                                          Time [s]


       WARM-UP TEST TEMPERATURE PROFILES FOR ENGINE 1 AT WOT
     2050
                        Lab Torque N-m
             Bias Cor. Interp. Torque
     1950
               100     200     300     400     500      600     700     800      900     1000
     1750
  WARM-UP TEST LABORATORY AND INTERPOLATED TORQUE TRACES FOR
                                 ENGINE 1 AT WOT
SwRI Report 03.12024.06
H-l

-------
230
220
i
S 210
O
u.
(/)
m
200
190
1RD .






u f






ftUWL..
?H^W^nh\»4*'WV*(^


        0      100     200     300      400     500     600      700     800      900     1000
                                            Time [s]


    WARM-UP TEST LABORATORY AND INTERPOLATED BSFC TRACES FOR
                                  ENGINE 1 AT WOT
    450
           - DPF Inlet Temp degC  	DPF Outlet Temp degC     Start NTE Zone
           •Actual ECT - CAN    	Target IMT		Actual IMT - CAN
                          -Target ECT
    400
    350
    300
   a 250


   I 200 -
    150   -
    100
       0      200      400      600     800      1000    1200     1400     1600    1800     2000
                                           Time [s]


   WARM-UP TEST TEMPERATURE PROFILES FOR ENGINE 1 AT PART LOAD
SwRI Report 03.12024.06
H-2

-------
     1850
                                           NTE Zone  	Bias Cor. Interp. Torque
               100     200     300     400     500     600     700     800     900     1000
     1550
  WARM-UP TEST LABORATORY AND INTERPOLATED TORQUE TRACES FOR
                             ENGINE 1 AT PART LOAD
     240
                   Lab Dilute CB BSFC g/kW-h  	Bias Cor. Interpolated BSFC     Start NTE Zone

               100     200     300      400     500      600     700     800     900     1000
     180
    WARM-UP TEST LABORATORY AND INTERPOLATED BSFC TRACES FOR
                             ENGINE 1 AT PART LOAD
SwRI Report 03.12024.06
H-3

-------
                  -DPF Inlet Temp degC
• DPF Outlet Temp degC
                                                                   •Start NTE Zone
    600
    500
  -- 400
  O)
  u
  2,

  u

  = 300
  0)
    200
     100
        0       200      400      600      800      1000     1200     1400     1600     1800     2000

                                             Time [s]



       WARM-UP TEST TEMPERATURE PROFILES FOR ENGINE 2 AT WOT
     1650
     1550   -
     1450   -
     1350   -
   a
   a. 1250
   O"
     1150
     1050   -
      950
                         •Lab Torque N-m   	Bias Cor. Interp. Torque  ^^~Start NTE Zone
         0      100     200     300      400      500      600      700      800      900     1000

                                               Time[s]



  WARM-UP TEST LABORATORY AND INTERPOLATED TORQUE TRACES FOR

                                    ENGINE 2 AT WOT
SwRI Report 03.12024.06
  H-4

-------
     260
                    Lab Dilute CB BSFC g/kW-h 	Bias Cor. Interpolated BSFC  ^—Start NTE Zone
               100      200      300     400     500      600      700     800     900      1000
     200
    WARM-UP TEST LABORATORY AND INTERPOLATED BSFC TRACES FOR
                                  ENGINE 2 AT WOT
                 -DPF Inlet Temp degC
-DPF Outlet Temp degC
                                                                 •Start NTE Zone
    500
    450
    400
    350
  O
    250
    200
    150
    100
     50
       0      200     400      600      800     1000     1200     1400     1600     1800    2000
                                            Time [s]

   WARM-UP TEST TEMPERATURE PROFILES FOR ENGINE 2 AT PART LOAD
SwRI Report 03.12024.06
  H-5

-------
     1300
                        Lab Torque N-m  	Bias Cor. Interp. Torque  ^^—Start NTE Zone
               100     200     300      400     500      600     700     800     900     1000
      600
  WARM-UP TEST LABORATORY AND INTERPOLATED TORQUE TRACES FOR
                              ENGINE 2 AT PART LOAD
                   Lab Dilute CB BSFC g/kW-h 	Bias Cor. Interpolated BSFC  ^—Start NTE Zone
               100     200      300     400      500     600     700     800     900     1000
     190
     180
    WARM-UP TEST LABORATORY AND INTERPOLATED BSFC TRACES FOR
                              ENGINE 2 AT PART LOAD
SwRI Report 03.12024.06
H-6

-------
     500
            -DPF Inlet Temp degC
            -Target IMT
                            -DPF Outlet Temp degC
                            -Actual IMT - CAN
-Target ECT
•Start NTE Zone
                                                                          Actual ECT - CAN
     450
     400
  O
     350 - - - /
     300  -
  1
  HI
  Q.
  HI
     250 - - -
200  H
     150  H
     100
     50 "
        0       200     400      600      800      1000     1200     1400     1600      1800     2000
                                              Time [s]

        WARM-UP TEST TEMPERATURE PROFILES FOR ENGINE 3 AT WOT
     1000
      900
      800
      700 -
   a  600
   O"
   £
      500
      400
      300
      200
                          •Lab Torque N-m  	Bias Cor. Interp. Torque   ^^~Start NTE Zone
          \
                 100      200      300      400      500      600      700      800      900     1000
                                               Time[s]
  WARM-UP TEST LABORATORY AND INTERPOLATED TORQUE TRACES FOR
                                    ENGINE 3 AT WOT
SwRI Report 03.12024.06
                                         H-7

-------
     300
     280
     260
   i
   S 240 -
   O
   u.
     220
     200
     180
                      •Lab Dilute CB BSFC g/kW-h  	Bias Cor. Interpolated BSFC      Start NTE Zone
        0       100      200      300     400     500     600     700     800     900     1000

                                              Time [s]



    WARM-UP TEST LABORATORY AND INTERPOLATED BSFC TRACES FOR

                                    ENGINE 3 AT WOT
            -DPF Inlet Temp degC  	DPF Outlet Temp degC 	Target ECT

            • Target IMT		Actual IMT - CAN    	Start NTE Zone
    450
                                                                        Actual ECT - CAN
    350
  JJ 300
  - 250
    200
     150
        0       200      400      600      800      1000     1200     1400     1600     1800     2000

                                             Time [s]




   WARM-UP TEST TEMPERATURE PROFILES FOR ENGINE 3 AT PART LOAD
SwRI Report 03.12024.06
H-8

-------
     800
     700  -
     600

   f
   a
   o 500
   O"
     400
     300
     200
                         I
                        • Lab Torque N-m  	Bias Cor. Interp. Torque  ^^—Start NTE Zone
W/vwvfl'vA
i^i»ii^A»^^«'*«»fliM*»VVM*<^*
-------
                               APPENDIX I
             INTERPOLATION TORQUE AND BSFC ERROR MAPS
SwRI Report 03.12024.06

-------
     F
     u
     e
     I
     r
     a
     t
     e
80-
60-
40-
          20-
             1200  1400  1600   1800   2000
                        Sfceed rpm
   1.3466
   0.7684
   0.1902
  -0.3880
  -0.9662
  -1.5444
  -2.1226
  -2.7007
  -3.2789
  -3.8571
Delta Tore
 INTERPOLATED TORQUE ERROR (% PEAK TORQUE) BY SPEED (RPM)
               AND FUEL RATE (G/S) FOR ENGINE #1
      F
      u
      e
      I
      r
      a
      t
      e
60-
40-
           20-
                1400  1600  1800  2000  2200
                         Speed rpm
  PI 4.6530
     3.8707
     3.0884
     2.3061
     1.5238
     0.7414
    -0.0409
    -0.8232
    -1.6055
    -2.3878
  Delta Tore
 INTERPOLATED TORQUE ERROR (% PEAK TORQUE) BY SPEED (RPM)
               AND FUEL RATE (G/S) FOR ENGINE #2
SwRI Report 03.12024.06
                      1-1

-------
      F
      u
      e
      I
      r
      a
      t
      e
40-
30-
20-
          10-
              1500
              2000       2500
              Speed rpm
    2.5072
    1.9951
    1.4830
    0.9710
    0.4589
   -0.0532
	-0.5652
   1-1.0773
   -1.5893
   -2.1014
Delta  Tern
 INTERPOLATED TORQUE ERROR (% PEAK TORQUE) BY SPEED (RPM)
               AND FUEL RATE (G/S) FOR ENGINE #3
      F
      u
      e
      I
      r
      a
      t
      e
80-
60-
40-
          20-
             1200  1400   1600   1800   2000
                        SjDeed rpm
   16.8087
   8.7276
   0.6465
  -7.4346
  -15.5157
  -23.5968
  -31.6780
  -39.7591
  -47.8402
  -55.9213
Delta BSFC
  INTERPOLATED BSFC ERROR (G/KW-HR) BY SPEED (RPM) AND FUEL
                   RATE (G/S) FOR ENGINE #1
SwRI Report 03.12024.06
                      1-2

-------
      F
      u
      e
      I
      r
      a
      t
      e
60-
40-
          20-
                1400  1600 1800  2000  2200
                         Speed rpm
    12.449
    10.271
    8.0932
    5.9149
    3.7366
    1.5583
   -0.6200
   -2.7983
   -4.9765
   -7.1548
Delta BSFC
  INTERPOLATED BSFC ERROR (G/KW-HR) BY SPEED (RPM) AND FUEL
                    RATE (G/S) FOR ENGINE #2
      F
      u
      e
      I
      r
      a
      t
      e
40-
30-
20-
          10-
              1500
              2000       2500
              SjDeed rpm
Fll5.9089
   14.1637
   12.4185
   10.6733
   8.9282
   7.1830
   5.4378
!   3.6926
   1.9474
   0.2022
 Del BSFC
  INTERPOLATED BSFC ERROR (G/KW-HR) BY SPEED (RPM) AND FUEL
                    RATE (G/S) FOR ENGINE #3
SwRI Report 03.12024.06
                      1-3

-------
                              APPENDIX J
      EVALUATION OF MANUFACTURER SUPPLIED ERROR SURFACES
SwRI Report 03.12024.06

-------
                                                                  August 28, 2006
From:        Matt Spears, US EPA

To:          Measurement Allowance Steering Committee

Subject:      Evaluation of Manufacturer-Submitted Error Surfaces

Background
       In May 2005 EPA, CARB, and EMA signed a Memorandum of Agreement
(MOA), which among other things, described how to evaluate manufacturer voluntary
submissions of data that demonstrated non-deficiency AECD effects or production
variability effects on the ability to estimate NTE torque/BSFC values from ECM
parameters, using prescribed mapping procedures. In the Agreement it stated that, "EPA
and CARB, in consultation with HDOH engine manufacturers, will utilize this
information, if reasonably common among manufacturers, to determine and include a
margin component in the error model that accounts for the variability in the torque/BSFC
values used in the NTE brake-specific emission calculations. For example, EPA/CARB
would consider information for an additional allowance if variability due to non-
deficiency AECDs is consistent across manufacturers. If variability is inconsistent and
infrequent across the submissions or if there is a consistent bias, EPA and CARB would
expect manufacturers to account for these errors by creating more sophisticated
algorithms that decrease the infrequent large deviations or account for the consistent bias
that exists across manufacturers."

Methods
       Five engine manufacturers submitted data to EPA and CARB  for both torque and
BSFC.  Three of the  five manufacturers submitted error surface data that summarized
their total error  of all of the  component errors that they considered.  The  other two
manufacturers submitted data that depicted various sources of error, such as production
variability,  deterioration,  and ECM  algorithm accuracy.  For these two manufacturers
who did not send actual error surface data, I constructed or approximated error surfaces
so that the error surfaces from all  manufacturers could be compared.  For error types (e.g.
production  variability) where I received  data  from  several engines from a  given
manufacturer, I determined the 5th, 50th,  and 95th percentile errors from each engine's data
set, and then I averaged the percentile results across  multiple engines because there was
almost no scatter of percentile errors between engines. Once I had single 5th 50th and 95th
percentile errors  for each type of error,  I added the 5th, 50th and 95th percentiles for each
type of error, respectively, to arrive at "worst case" composite errors.  I used these results
to create error surfaces for torque and BSFC. Note that I did not take an RSS approach to
summing these error types.

       To  analyze the consistency  of the various submitted  error surfaces across
manufacturers, according  to the MOA, I examined the error  surfaces for  bias  (50th
percentile values), skew (symmetry of 5th and 95th percentile values), dependency of error
SwRI Report 03.12024.06                      J-l

-------
magnitude  as  a function  of level (torque, BSFC), and  the  magnitude of individual
manufacturer's error surfaces versus the  average  of the remaining manufacturer error
surfaces (outliers).
                                                              -,th
Observations
    1.  There was no consistent evidence of bias. Nearly all of the 50m percentile errors
       were zero or near zero for both torque and BSFC from all manufacturers.

    2.  There was no consistent evidence of skew.  Most of the manufacturers reported
       symmetrical values (but opposite sign) for the 5th and 95th percentile values for
       both torque and BSFC.

    3.  There was no consistent evidence of %-error being a function of level.  One
       individual manufacturer reported significant BSFC %-error changes as a function
       of level. However, the magnitudes of the BSFC errors from that manufacturer
       were also very inconsistent (an order of magnitude higher) compared to the other
       four manufacturers' BSFC error data. When data from all five manufacturers was
       aggregated, there was no correlation of torque %-error versus torque or—for the
       four consistent manufacturers' error surfaces—BSFC %-error versus BSFC.

    4.  Because there was no consistent bias, skew, or dependency of %-error as a
       function of level, I examined the average values of the manufacturers' 5th and 95th
       percentile values. For torque the average value for all five manufacturers was -
       5.68% for the 5th percentile value and +5.68% for the 95th percentile value. For
       BSFC, I discarded one manufacturer's data based upon good engineering
       judgment and based upon the terms of the MOA because every data point on the
       error surface was over ten times the magnitude of any other manufacturer's
       corresponding BSFC error. This manufacturer's BSFC data was deemed
       inconsistent with the four other manufacturer's data.  For BSFC the average value
       for the four remaining manufacturers was -1.35% for the 5th percentile value and
       +1.35% for the 95th percentile value.

Conclusions
       Based upon my analysis, and in accordance with the terms of the Memorandum of
Agreement and associated Test Plan, I conclude that the following  error  surfaces should
be added to the error model to account for the consistent torque and BSFC error depicted
in the manufacturer-submitted data:
Parameter
Torque
BSFC
Sampling Distribution
Normal, once per NTE
Normal, once per NTE
5* percentile
-5. 7% of point
-1.4% of point
50th percentile
0
0
95th percentile
+5. 7% of point
+1.4% of point
Respectfully Submitted,
Matthew W Spears
SwRI Report 03.12024.06
                                          J-2

-------
                                                               November 2, 2006
From:        Matt Spears, US EPA

To:          Measurement Allowance Steering Committee

Subject:      Re-Evaluation of Manufacturer-Submitted Error Surfaces

Background
       On August 28, 2006 I submitted a memo to the steering committee indicating the
results of my analysis of the manufacturer-submitted error surface data. This original
memo is attached for reference. Subsequently, we (the steering committee) had a
discussion of my original memo, and as a follow-up, the EMA members of the steering
committee requested an opportunity to submit to EPA/CARB some additional data.
Specifically, they requested to submit data that included the torque and BSFC ECM
errors due to non-deficiency AECD operation.  I agreed to contact those engine
manufacturers that had not originally submitted such information. Ultimately, two of
those engine manufacturers submitted additional information.

Methods and Observations
       One company, Cummins,  Inc., submitted a comprehensive  analysis of all  the
errors that affect ECM torque and BSFC prediction.  The Cummins submission was the
only data that I received, either originally or in this second round of data submission, that
contained  a direct comparison of ECM-derived BSFC versus lab-determined BSFC.
Furthermore, the Cummins submission included a thorough statistical analysis of all of
the sources of error, including data previously submitted.

       The other company submitted SET data for ECM torque and  measured torque at
two different modes of engine operation.  Based upon a discussion with that company, I
determined that one of the modes of operation was the mode from which ECM torque is
derived.  Since the other mode of operation was  of lower engine efficiency,  ECM torque
and ECM derived BSFC reported during the less efficient mode of operation  would result
in a consistent and significant low bias of reported emissions.  Because this approach to
ECM torque programming might not be consistent among manufacturers, I was reluctant
to include the low bias indicated from this data. After correcting for the low bias,  the
resulting torque error differed from the Cummins data by less than one percentage point.

       I also reanalyzed the  data from the one company  that originally submitted non-
deficiency AECD data.  This company's BSFC data was initially discarded as part of my
original analysis, which I explained in my original memo.  This dataset only presented
data indicating deviations of torque/BSFC between modes of engine operation, and  did
not identify which mode(s) of operation were used to determine ECM torque or BSFC.
This dataset also included several non-NTE data points.  After filtering the non NTE data
from  this  data  set, the  resulting  BSFC 5th and 95th percentage  points were  roughly
reduced by half. Because I could not identify which mode(s) of engine operation were
SwRI Report 03.12024.06                       J-3

-------
used for ECM calibration, I could not subtract any bias from this spread. Even without
subtracting any bias, the resulting BSFC errors differed from the Cummins analysis by
less than two percentage points.

Conclusion
       Based upon  all of the data submitted, and in accordance with the terms of the
Memorandum  of Agreement and associated Test Plan, I conclude that the error surfaces
proposed by Cummins should be included in the error model to account for the consistent
torque and BSFC error depicted in  the manufacturer-submitted data.  The Cummins
dataset and analysis is comprehensive and it imposes no  bias on the error surface.
Furthermore, other less complete datasets indicate errors very close to those reported by
Cummins.  Below are the revised values for the error surfaces:
Parameter
Torque
BSFC
Sampling Distribution
Normal, once per NTE
Normal, once per NTE
5th percentile
-6. 5% of point
-5. 9% of point
50th percentile
0
0
95th percentile
+6. 5% of point
+5. 9% of point
For a comparison of these values based upon the original data submission, please refer to
my original memo, attached.

Respectfully Submitted,
Matthew W Spears

End.
SwRI Report 03.12024.06
J-4

-------
                           APPENDIX K
ENVIRONMENTAL CHAMBER TESTING RESULTS AND ERROR SURFACES
SwRI Report 03.12024.06

-------
        THC[ppmC]  •  CO [ppm]    NO [ppm]  X  NO2 [ppm]  • NOx [ppm] 	New FID Bottle  A  CO2 [%]
                                                                               0.030
                                Reference Gas Delta Observation
   PEMS 2 ENVIRONMENTAL BASELINE ZERO DELTA MEASUREMENTS
      •


     60
8
THC[ppmC]  • CO [ppm]     NO [ppm]  X  NO2 [ppm]  •  NOx [ppm] 	New FID Bottle  A CO2 [%]
ppm
   S

   §
                                                         A ---**-*'
                         - A~A A -AA-

                          I     A
                                           « »»»«»•»«»<•••••»» »«>••»«»»»»•»••••
                                                                               0.000
                                                                             	0.010
	0.020 rt
       g.

       s
  -0.030 g
p




trat
p




Con
p
                                                                            -^r  -0.070
                                                                               -0.080
                                Reference Gas Delta Observation
  PEMS 2 ENVIRONMENTAL BASELINE AUDIT DELTA MEASUREMENTS
SwRI Report 03.12024.06
                                     K-l

-------
        THC [ppmC]
CO [ppm]
NO [ppm]  X NO2 [ppm]
NOx [ppm]
-New FID Bottle  A CO2 [%]
                                                                             0.080
     60	0.060
                                                                             -0.040
                               Reference Gas Delta Observation
   PEMS 2 ENVIRONMENTAL BASELINE SPAN DELTA MEASUREMENTS
      » THC [ppmC]  • CO [ppm]     NO [ppm]  X NO2 [ppm]
                                NOx [ppm]
                                -New FID Bottle  A CO2 [%]
                                                                             0.015
                                                                             -0.009
                               Reference Gas Delta Observation
   PEMS 3 ENVIRONMENTAL BASELINE ZERO DELTA MEASUREMENTS
SwRI Report 03.12024.06
                        K-2

-------
THC [ppmC]  • CO [ppm]    NO [ppm]  X  NO2 [ppm]
                                                 NOx [ppm]
                 -New FID Bottle  A  CO2 [%]
                                                                        0.060
     20  -	0.048
                                                                        -0.036
                             Reference Gas Delta Observation
  PEMS 3 ENVIRONMENTAL BASELINE AUDIT DELTA MEASUREMENTS
     »  THC[ppmC]  • CO [ppm] X  NO [ppm]  X  NO2 [ppm]  • NOx [ppm] 	New FID Bottle  A CO2 [%]
     20
                                                   »^^*s«^
                                                     • .  AA*  >. «A.
                             Reference Gas Delta Observation
   PEMS 3 ENVIRONMENTAL BASELINE SPAN DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-3

-------
« THC [ppmC]

• CO [ppm]

NO [ppm]
A CO2 [%]
X NO2 [ppm]
• NOx [ppm]
                                                                         0.030
                                                                         -0.010
                              Reference Gas Delta Observation
  PEMS 5 ENVIRONMENTAL BASELINE ZERO DELTA MEASUREMENTS
 THC [ppmC]   •  CO [ppm]        NO [ppm]

-Day Break     —New FID Bottle  A  CO2 [%]
                                                 X  NO2 [ppm]
                       NOx [ppm]
     70
     60
   Q
   c
   o
                              Reference Gas Delta Observation
  PEMS 5 ENVIRONMENTAL BASELINE AUDIT DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-4

-------
          THC [ppmC]   •  CO [ppm]        NO [ppm]      X NO2 [ppm]
         -Day Break        New FID Bottle  A  CO2 [%]
                                                                    NOx [ppm]
                               Reference Gas Delta Observation
   PEMS 5 ENVIRONMENTAL BASELINE SPAN DELTA MEASUREMENTS
  5.0 i-


  4.5 -


  4.0 -


•g- 3.5 --


^ 3.0 -
re
£
§ 2.5 -
u.
« 2.0 -
re
^
uj 1.5


  1.0^


  0.5 -


  0.0 -L
                                      »   »
                                          *
                                             **
                               • «
                 50
                           100        150        200        250

                                Exhaust Flow Meter Zero Observation
                                                                    300
                                                                              350
          5-INCH EFM ENVIRONMENTAL BASELINE ZERO DELTA
                                 MEASUREMENTS
SwRI Report 03.12024.06
                                          K-5

-------
            THC [ppmC]

            New FID Bottle
                  CO [ppm]      X NO [ppm]

                  Temp. [degC]   A CO2 [%]
             X  NO2 [ppm]
                NOx [ppm]
                                                                            0.014
                                                                            -0.004
                               Reference Gas Delta Observation
          PEMS 2 ENVIRONMENTAL TEMPERATURE ZERO DELTA

                                MEASUREMENTS
     THC [ppmC]

    -New FID Bottle
                         CO [ppm]

                         Temp. [degC]
   NO [ppm]

A  CO2 [%]
X  NO2 [ppm]
NOx [ppm]
                                                                            0.00
  O
  o>
  m
  T3
  I
  ^
  o
  O
vt»^b8|.tVMMM»^5!*!:>A^AV - - V -  A*

-*--     A  *  ArA4r-l-AA  A ^         A    ^ —
                                                                          	0.08
                                                                            -0.09
                               Reference Gas Delta Observation
         PEMS 2 ENVIRONMENTAL TEMPERATURE AUDIT DELTA

                                MEASUREMENTS
SwRI Report 03.12024.06
                                    K-6

-------
            THC [ppmC]    • CO [ppm]        NO [ppm]
            New FID Bottle    Temp. [degC]   A CO2 [%]
      X  NO2 [ppm]
NOx [ppm]
                                                                              0.100
                                                                              -0.050
                                Reference Gas Delta Observation
          PEMS 2 ENVIRONMENTAL TEMPERATURE SPAN DELTA
                                 MEASUREMENTS
            THC [ppmC]    • CO [ppm]        NO [ppm]
            New FID Bottle    Temp. [degC]   A CO2 [%]
      X  NO2 [ppm]
NOx [ppm]
                                                                              0.025
                                                                              -0.010
                                Reference Gas Delta Observation
          PEMS 3 ENVIRONMENTAL TEMPERATURE ZERO DELTA
                                 MEASUREMENTS
SwRI Report 03.12024.06
K-7

-------
            THC [ppmC]   •  CO [ppm]        NO [ppm]
            New FID Bottle     Temp. [degC]   A  CO2 [%]
      X  NO2 [ppm]
NOx [ppm]
     50
     -30
                               Reference Gas Delta Observation
         PEMS 3 ENVIRONMENTAL TEMPERATURE AUDIT DELTA
                                 MEASUREMENTS
            THC [ppmC]   •  CO [ppm]        NO [ppm]
            New FID Bottle     Temp. [degC]   A  CO2 [%]
      X  NO2 [ppm]
NOx [ppm]
                                                                              0.050
                                                                              -0.090
                               Reference Gas Delta Observation
          PEMS 3 ENVIRONMENTAL TEMPERATURE SPAN DELTA
                                 MEASUREMENTS
SwRI Report 03.12024.06
K-8

-------
          » THC [ppmC]
          • NOx [ppm]
 CO [ppm]
- Break
X  NO [ppm]
   Temp. [degC]
X  NO2 [ppm]
A  CO2 [%]
     -40
                              Reference Gas Delta Observation
          PEMS 5 ENVIRONMENTAL TEMPERATURE ZERO DELTA
                                MEASUREMENTS
« THC [ppmC]
• NOx [ppm]
• CO [ppm]

NO [ppm]
Temp. [degC]
X NO2 [ppm]
A CO2 [%]
                                                                         -r 0.000
                                                                           -0.150
                              Reference Gas Delta Observation
         PEMS 5 ENVIRONMENTAL TEMPERATURE AUDIT DELTA
                                MEASUREMENTS
SwRI Report 03.12024.06
              K-9

-------
• THC [ppmC]
• NOx [ppm]
• CO [ppm]

NO [ppm]
Temp. [degC]
X NO2 [ppm]
A CO2 [%]
     120






     100






     80






     60






     40






     20






      0






     -20






     -40
            A  A*4 A
             AA AAA    /A,
A    A A"

A>A   A
                               Reference Gas Delta Observation
          PEMS 5 ENVIRONMENTAL TEMPERATURE SPAN DELTA

                                MEASUREMENTS
     5.0
     4.5  -
     4.0  -
   |-3.5f


   y,


   |
   £


   I2'5
   u.

   « 2.0
3.0 -
   " , _

   iu 1.5
     1.0  -
     0.5  -





     0.0
                                     *   *
           >V -->-



            **^  » UL*   » «
                 50
                          100        150        200        250


                               Exhaust Flow Meter Zero Observation
                                                                  300
                                                                            350
       5-INCH EFM ENVIRONMENTAL TEMPERATURE ZERO DELTA


                                MEASUREMENTS
SwRI Report 03.12024.06
                                      K-10

-------
                                                   - PEMS 2 * PEMS 3 PEMS 5
    25	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1
                             Reference Gas Delta Observation
     ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE NOX

            CONCENTRATION ZERO DELTA MEASUREMENTS
    15
    10 --
     5  -

                                                   - PEMS 2 • PEMS 3 • PEMS 5
     0  -
    -5 ---
Q

c
o
B
re


m
o
c
o
o


o -10  -
z
    -15 --
           .   -_*.--_
    -20 H—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—


      0         10       20       30       40        50        60        70


                             Reference Gas Delta Observation
     ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE NOX

           CONCENTRATION AUDIT DELTA MEASUREMENTS
SwRI Report 03.12024.06
                                    K-ll

-------
     40
     30  -
                                                           - PEMS 2 • PEMS 3 • PEMS 5
   I 20
   Q.
     10 --
   2
   o
   O
    -10 --
    -20  -
    -30 H—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I
       0          10         20         30         40        50         60         70
                                  Reference Gas Delta Observation
      ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE NOX
              CONCENTRATION SPAN DELTA MEASUREMENTS
      40
      30 --
   I 20

   £
   ° 10
   £
      0 ---
     -10 --a
     -20 --
     -30
                    200
                                 400           600           800
                              NOx Temperature Observation (1 to 1080)
                                                                       1000
              -NOx SPAN 1209ppm NOx
                                   • NOx Audit 341 ppm NOx
                                                       - NOx Zero 0 ppm NOx
  FINAL ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE NOX
                                CONCENTRATION
SwRI Report 03.12024.06
K-12

-------
             I	1	1	1	1	1	1	1	1	1	1	1	1	1
    -0.0020
                                         100              150
                                   Reference Gas Delta Observation
                                                                         200
       ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE CO
             CONCENTRATION ZERO DELTA MEASUREMENTS
     0.0020
     0.0015 -
     0.0010 -
     0.0005 -
  =  0.0000 -
  § -0.0005 -
    -0.0010 -
                                                           - PEMS 2 • PEMS 3 • PEMS 5
                                 »»**
                                                              *  *
                                * **
    -0.0015 -
    -0.0020 -
    -0.0025 ^—I—I—I—I-
                   10
                            20        30        40         50
                                   Reference Gas Delta Observation
                                                                   60
                                                                             70
       ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE CO
             CONCENTRATION AUDIT DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-13

-------
     0.0070
                                                                  - PEMS 2 • PEMS 3 • PEMS 5
     0.0060

     0.0050 f                     - -m --

   £                                •  i
   re 0.0040
   I
   g 0.0030 -
   "re
   | 0.0020
   o
   c
   " 0.0010 4-'
   o
   o
     0.0000

     -0.0010 -

     -0.0020 H—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—h
          0          10          20         30         40         50         60          70
                                       Reference Gas Delta Observation


        ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE CO
               CONCENTRATION SPAN DELTA MEASUREMENTS



     0.007

     0.006

     0.005  -

  P| 0.004
  «
  g  0.003  -

  3  0.002

   ^ 0.001 -I--                      --«---.--     ^f r      --Ji

  80                               - -   ••'
                      200            «DO . »f    " 600            85S         ^tQDO
    -0.001 -I-                         -*X--*-*

    -0.002  -
    -0.003
                                  CO Temperature Observation [1 to 1080]
               -COSpan 0.0951% CO
                                      • CO Audit 0.0179% CO
                                                            - CO Zero 0% CO
   FINAL ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE CO
                                   CONCENTRATION
SwRI Report 03.12024.06
K-14

-------
    0.015
    -0.020
                                    100             150
                               Reference Gas Delta Observation
                                                                 200
      ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE CO2
            CONCENTRATION ZERO DELTA MEASUREMENTS
CO2 Concentration Delta [%]
0.08
0.06
0.04
0.02
0.00
-0.02
-0.04

-PEMS2 »PE
MS 3 BPEMS5

" % "V1^.- •+. v"
•
^m *
jf? "^
- ***** *• -+^*^ ~- - "*~ ^" * ^ ^

• * *

       0        10        20        30        40       50       60       70
                               Reference Gas Delta Observation
      ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE CO2
           CONCENTRATION AUDIT DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-15

-------
CO2 Concentration Delta [%]
DpppppOC
-* o o o — * — * Kj K
DU10U10U10C

-PEMS 2 »PE
MS 3 • PEMS 5

•J"-.'-.'' f+ *f '.
— " •
*
•
** • • ** **_^~ A^^ ~ w ^9*m »^~~ ~~
•
g
|M»**% "

        0         10        20        30        40        50        60        70
                                Reference Gas Delta Observation
      ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE CO2
             CONCENTRATION SPAN DELTA MEASUREMENTS
    0.25
                            CO2 Temperature Observation [1 to 1080]
              -C02Span12.0%C02
                                    • C02 Audit 5.99% C02
                                                          -C02ZeroO%C02
  FINAL ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE CO2
                              CONCENTRATION
SwRI Report 03.12024.06
K-16

-------
    -7
                                    100              150
                              Reference Gas Delta Observation
                                                                  200
    ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE NMHC
            CONCENTRATION ZERO DELTA MEASUREMENTS
    14

    12

    10 -
                                                   - PEMS 2 • PEMS 3 • PEMS 5
E
Q.

J±
&
C
O
13
                               ***»
     2 -
  o
  O
  o
    -2 -
    -4 -
    -6
          1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1-
               10        20        30        40        50        60        70
                              Reference Gas Delta Observation
    ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE NMHC
            CONCENTRATION AUDIT DELTA MEASUREMENTS
SwRI Report 03.12024.06
                                     K-17

-------
     70
     60 -
     50
                                                          - PEMS 2 • PEMS 3 • PEMS 5
  I  40

  I  30
  O
  I  20
  13
  1  1°
  U

  8  °
  o
    -20 -
    -30 -
    -40 H—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—h
       0         10        20        30        40        50        60         70
                                 Reference Gas Delta Observation
     ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE NMHC
             CONCENTRATION SPAN DELTA MEASUREMENTS
                            NMHC Temperature Observation [1 to 1080]
      - MVIHC Span 669ppm MVIHC
MVIHC Audit 159.9ppm MVIHC
-NVIHCZeroOppm MVIHC
 FINAL ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE NMHC
                               CONCENTRATION
SwRI Report 03.12024.06
        K-18

-------
     0.25
     0.20  -
    -0.25
               50      100     150      200     250     300
                               Exhaust Flow Meter Zero Observation
                                                           350
                                                                  400
                                                                         450
   ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE EXHAUST
                    FLOW RATE DELTA MEASUREMENTS
             • THC [ppmC] BCO [ppm]  NO [ppm] XNO2 [ppm] »NOx [ppm]   Pressure [kpa] ACO2 [%]
     120
   „ 100
                                                                           0.014
                                                                           -0.018
                               Reference Gas Delta Observation
  PEMS 2 ENVIRONMENTAL PRESSURE ZERO DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-19

-------
              • THC [ppmC] BOO [ppm]  NO [ppm] XNO2 [ppm] «NOx [ppm] -Pressure [kpa] ACO2 [%]
                                                                           - 0.054
                                                                             -0.030
                                Reference Gas Delta Observation
  PEMS 2 ENVIRONMENTAL PRESSURE AUDIT DELTA MEASUREMENTS
              • THC [ppmC] BCO [ppm]  NO [ppm] XNO2 [ppm] «NOx [ppm]  Pressure [kpa] ACO2 [%]
     160
      -60
                                Reference Gas Delta Observation
   PEMS 2 ENVIRONMENTAL PRESSURE SPAN DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-20

-------
              • THC [ppmC] BOO [ppm]   NO [ppm] XNO2 [ppm] «NOx [ppm] -Pressure [kpa] ACO2 [%]
     120
   „ 100
                                                                              0.015
                                                                              -0.015
                                Reference Gas Delta Observation
  PEMS 3 ENVIRONMENTAL PRESSURE ZERO DELTA MEASUREMENTS
             • THC [ppmC] BCO [ppm]  NO [ppm] XNO2 [ppm] «NOx [ppm]  Pressure [mbar] ACO2 [%]
     120
                                                                              0.080
                                                                              -0.060
                                Reference Gas Delta Observation
  PEMS 3 ENVIRONMENTAL PRESSURE AUDIT DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-21

-------
             • THC [ppmC] BCO [ppm]  NO [ppm] XNO2 [ppm] «NOx [ppm]  Pressure [mbar] ACO2 [%]
                                Reference Gas Delta Observation
   PEMS 3 ENVIRONMENTAL PRESSURE SPAN DELTA MEASUREMENTS
                              * Flow Meter [scfm] • Pressure [mbar]
                                                                              1200
                                                                              1000
                                                                              800
                                                                              600
                                                                              400
                                                                              200
                                                                                  a.
                                                                                  LU
                                                                                  0.
                         100
                                   150        200       250

                               Reference Gas Delta Observation
                                                                300
                                                                          350
          5-INCH EFM ENVIRONMENTAL PRESSURE ZERO DELTA
                                 MEASUREMENTS
SwRI Report 03.12024.06
K-22

-------
                                   100             150


                             Reference Gas Delta Observation
                                                                200
  ERROR SURFACE (NOT USED IN THE MODEL) FOR ENVIRONMENTAL

    PRESSURE NOX CONCENTRATION ZERO DELTA MEASUREMENTS
    30
    25 -
    20 -
    15 ------
  m
  o
  c
  o
  o
    10 --
•       ---z*~
        »•»»-* **
    0 -
                                    *  -
    -5 H—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I


      0        10        20        30        40        50        60        70


                             Reference Gas Delta Observation
  ERROR SURFACE (NOT USED IN THE MODEL) FOR ENVIRONMENTAL

   PRESSURE NOX CONCENTRATION AUDIT DELTA MEASUREMENTS
SwRI Report 03.12024.06
                               K-23

-------
     30
                                                            -PEMS2 »PEMS3
     20 -



  I  10
  D.
  re

  §  0
  c
  o


  I -10

  o
  o
  5 -20
  z


    -30



    -40 H—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—h
       0         10        20         30         40         50         60         70

                                 Reference Gas Delta Observation



  ERROR SURFACE (NOT USED IN THE MODEL) FOR ENVIRONMENTAL

    PRESSURE NOX CONCENTRATION SPAN DELTA MEASUREMENTS







     0.0010
                                     -PEMS2 »PEMS3

     0.0008


     0.0006


  2  0.0004
  re

  Q  0.0002
  c
  o
  is  o.oooo

  1
  '_ -0.0002
  o
  O
  O -0.0004


    -0.0006


    -0.0008
    -0.0010
            H	1	1	1-
                            H	1	1	1-
                                           H	1	1	H
                                                           -I	1	1	1-
                         50
                                        100              150

                                  Reference Gas Delta Observation
                                                                        200
          ERROR SURFACE FOR ENVIRONMENTAL PRESSURE CO
             CONCENTRATION ZERO DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-24

-------
     0.004
     0.003
     0.002  -
     0.001
   m
   o
   c
   o
   o

   O
   O
     0.000
-0.001
    -0.002
                •



           *	«
                  
                               «r »  **

                                   »»
        V

            *  *
**** _* *   * *c >*
                           • ***„
    -0.003 H—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I-


         0         10        20        30         40         50         60         70


                                   Reference Gas Delta Observation
          ERROR SURFACE FOR ENVIRONMENTAL PRESSURE CO

             CONCENTRATION AUDIT DELTA MEASUREMENTS
     0.014




     0.012




     0.010




     0.008




     0.006




     0.004




     0.002




     0.000




    -0.002




    -0.004




    -0.006
      -PEMS2 »PEMS3
                                                             n»*
                                »»**%**»
                    »**
            -H	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1-


                  10        20        30         40         50         60         70


                                   Reference Gas Delta Observation
          ERROR SURFACE FOR ENVIRONMENTAL PRESSURE CO

              CONCENTRATION SPAN DELTA MEASUREMENTS
SwRI Report 03.12024.06
                                       K-25

-------
    0.014


    0.012


     0.01 -


    0.008
  s?

  3 0.006
  &
  £ 0.004
  w

  I 0.002


  8
    -0.002


    -0.004


    -0.006
                 100
                         200
                                  300
                                                             600
                                                                               8(10
                                 CO Pressure Observation [1 to 720]
              - CO SPAN 0.0960% CO
                                   • CO Audit 0.0178% CO
                                                       -COZeroO%CO
      FINAL ERROR SURFACE FOR ENVIRONMENTAL PRESSURE CO
                                CONCENTRATION
     0.012
    -0.008 --
    -0.010
                                         100              150

                                   Reference Gas Delta Observation
                                                                          200
  ERROR SURFACE (NOT USED IN THE MODEL) FOR ENVIRONMENTAL
    PRESSURE CO2 CONCENTRATION ZERO DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-26

-------
     0.09
     0.08
     0.07
     0.06
  S  0.05 -f
  Q

  I  0.04
  •s

  I  0-03
  o
  o
  CM
  O
  O
  0.02
     0.01
     0.00
    -0.01
    -0.02
                             -PEMS2 »PEMS3
                   **	••*-
                                                      ****
                                               »**
                 10
                           20        30        40         50

                                 Reference Gas Delta Observation
                                                                 60
                                                                           70
  ERROR SURFACE (NOT USED IN THE MODEL) FOR ENVIRONMENTAL

    PRESSURE CO2 CONCENTRATION AUDIT DELTA MEASUREMENTS
  0.15




  0.10




„ 0.05

S


| 0.00

c
o

| -0.05
'c
    -0.10
    -0.15
    -0.20
                           -PEMS2 »PEMS3
          **---.*
                        •I  •»*»-
                ***
                                     	    »_
                                                     *»**.
    -0.25 H—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—h

        0         10         20        30        40         50        60         70

                                 Reference Gas Delta Observation
  ERROR SURFACE (NOT USED IN THE MODEL) FOR ENVIRONMENTAL

    PRESSURE CO2 CONCENTRATION SPAN DELTA MEASUREMENTS
SwRI Report 03.12024.06
                                       K-27

-------
    1 -

    0
  ? -1 --
    -2 -
    -3
         1 - 1 - 1 - 1 - 1 - 1 - 1
                              1
                                 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1
                    50
                                   100              150

                              Reference Gas Delta Observation
                                                                  200
       ERROR SURFACE FOR ENVIRONMENTAL PRESSURE NMHC
            CONCENTRATION ZERO DELTA MEASUREMENTS
     2
  Q.
  s  a
  g  -6
  c
  o
  13  -8
  o -10
  O
  o

  I -12 f
    -14
                          -PEMS2 «PEMS3
                                                ***********
                                                          *******
                                           ******
                          *********
    -16 H—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I

      0        10        20        30       40        50        60        70

                              Reference Gas Delta Observation
       ERROR SURFACE FOR ENVIRONMENTAL PRESSURE NMHC
            CONCENTRATION AUDIT DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-28

-------
     10
     5 -

  a  o ---
    -10 -
         -PEMS2 »PEMS3

          ***
                                               __~ *    *   * ~-_
                                   *****;*                       *
                                                                        --***
                                                                       ***
    -15 H—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I—I-
      0         10         20        30        40         50        60        70
                                Reference Gas Delta Observation
       ERROR SURFACE FOR ENVIRONMENTAL PRESSURE NMHC
             CONCENTRATION SPAN DELTA MEASUREMENTS
                                                                           800
                             NMHC Pressure Observation [1 to 720]
          - NMHC Span 663ppm NMHC      » NMHC Audit 159.9ppm NMHC       - NMHC Zero Oppm NMHC
    FINAL ERROR SURFACE FOR ENVIRONMENTAL PRESSURE NMHC
                              CONCENTRATION
SwRI Report 03.12024.06
K-29

-------
     0.5

     0.5

     0.4

  f 0.4
  iE
  I  0.3

  f °'3

  §  °'2

  i  °-2
  (3
     0.1

     0.1

     0.0

    -0.1
LU
         m  \
         * *A
  •
-%--
t
                 50
                          100        150       200        250
                                 Reference Gas Delta Observation
                                                                300
                                                                          350
   ERROR SURFACE FOR ENVIRONMENTAL TEMPERATURE EXHAUST
                    FLOW RATE DELTA MEASUREMENTS
            • THC [ppmC] BCO [ppm] «NO [ppm] XNO2 [ppm] «NOx [ppm] - Baseline  Zero ACO2 [%]
                              Reference Gas Delta Observation
         PEMS 7 ENVIRONMENTAL RADIATION BCI ZERO DELTA
                                MEASUREMENTS
SwRI Report 03.12024.06
                                        K-30

-------
•  THC [ppmC]    • CO [ppm]
-  Baseline      A Zero
 NO [ppm]     X NO2 [ppm]
-Restart PEMS  A CO2 [%]
                                                                    NOx [ppm]
                                                                                0.30
                                                                                0.00
                                Reference Gas Delta Observation
         PEMS 7 ENVIRONMENTAL RADIATION BCI AUDIT DELTA
                                 MEASUREMENTS
                                                                               0.375
                                         NO [ppm]      X NO2 [ppm]
                                         Restart PEMS   A CO2 [%]
                                                                               -0.075
                               Reference Gas Delta Observation
          PEMS 7 ENVIRONMENTAL RADIATION BCI SPAN DELTA
                                 MEASUREMENTS
SwRI Report 03.12024.06
                                  K-31

-------
                                 • Flow Meter [scfm] • Baseline
                                Reference Gas Delta Observation
       5-INCH EFM ENVIRONMENTAL RADIATION BCI ZERO DELTA
                                MEASUREMENTS
                THC [ppmC]  •  CO [ppm]    •  NO [ppm]    X  NO2 [ppm]
                Baseline    A  Zero      	Restart Test  A  CO2 [%]
                    NOx [ppm]
  Q
  c
  o
  13
                               Reference Gas Delta Observations
   PEMS 7 ENVIRONMENTAL RADIATION RADIATED IMMUNITY ZERO
                            DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-32

-------
            THC [ppmC]  •  CO [ppm]   •  NO [ppm]   X NO2 [ppm]
            Baseline    A  Zero     	Restart Test  A CO2 [%]
                                                         NOx [ppm]
E
Q.
£
tfl
§
0>
O
c
o
13
300

250

200

150

100

 50

  0

-50

-100
        ;;.>.;;,!...  i:;::::;;;::;!!;;!;!'!!!!!!;
        __ A	« A_ $A —	__	_____
                                                                    0.350


                                                                    0.315


                                                                    0.280


                                                                    0.245


                                                                    0.210
0.175 S
    "c
0.140 I
    o
    o
0.105 g
    o
0.070

0.035

0.000
                            Reference Gas Delta Observations
  PEMS 7 ENVIRONMENTAL RADIATION RADIATED IMMUNITY AUDIT
                         DELTA MEASUREMENTS





?
£
U)
J±
0)
Q
O
«
ncen
8










50
Q
I











IS
li
A
A'
• THC [ppmC] • CO [ppm] • NO [ppm] X NO2 [ppm] • NOx [ppm]


A
"


"
.... 1

•ii'ii'ft 15* 20 25 A 30 35 A
• • ^MMM . A A A ^' — 	 T ~
A A A











04
k



0 450





- 0 150
0 100


- -n nsn





s?
s
X
c
0
In
^
(U
&
O
o
o


                           Reference Gas Delta Observations
   PEMS 7 ENVIRONMENTAL RADIATION RADIATED IMMUNITY SPAN
                         DELTA MEASUREMENTS
SwRI Report 03.12024.06
                                    K-33

-------
                                  • Flow Meter [scfm] • Baseline
                               Exhaust Flow Meter Zero Observations
5-INCH EFM ENVIRONMENTAL RADIATION RADIATED IMMUNITY ZERO

                           DELTA MEASUREMENTS
           »THC [ppmC] BCO [ppm] «NO [ppm] XNO2 [ppm] «NOx [ppm] -Baseline  Zero ACO2 [%]
                                                                           0.050
                                                                           0.045
  3
  0>
  Q
  c
  o

  13
                              Reference Gas Delta Observations
  PEMS 7 ENVIRONMENTAL RADIATION ELECTROSTATIC DISCHARGE

                        ZERO DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-34

-------

Concentration Delta [ppm]
hj .&. O CO C
D 0 0 0 0 C
-20
-4D
»THC [ppmC] BCD [ppm] «NO [ppm] XNO2 [ppm] »NOx [ppm] -Baseline Zero ACO2 [%]

'
A. • • A * A A A '
" • MJL • A A A A
A A A J ^A " "t" .
•
A
A
A
5 8 * !Bx2£^5!lp*lXS*™»*25 30
~ • £ • •
^A «
A

3 0 0 0 0 C
J Ul ^J O NJ C
n o m o m c
32 Concentration Delta [%]

- -n n?R
                            Reference Gas Delta Observations
  PEMS 7 ENVIRONMENTAL RADIATION ELECTROSTATIC DISCHARGE
                      AUDIT DELTA MEASUREMENTS
        • THC [ppmC] BCO [ppm] •NO [ppm] XNO2 [ppm] «NOx [ppm] -Baseline  Zero -Zero ACO2 [%]




40
"E"
&
— 20
(U

o
l '
I -20
u
8



-«n

• A A
A A A A A A A
AA"B •" •••• A
A A

AAA" A AA "••

»« k»»»»4»i»***»»»»"''*n>
* * •
***** AAAA


• " A ) • •
* 5i .•••*

A •
•
• •




- 0.040 —
3
- 0.010 g
0
0 020 ^
(U
u
- -0.050 o
O
CN|
O



_n ian
                            Reference Gas Delta Observations
  PEMS 7 ENVIRONMENTAL RADIATION ELECTROSTATIC DISCHARGE
                       SPAN DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-35

-------
                                     • Flow Meter [scfm] • Baseline
     3.5

     3.0

     2.5

     2.0

     1.5

     1.0

     0.5

     0.0

     -0.5

     -1.0

     -1.5
          --*-
20
                    60
                              80
                                        100
                                                 120
                                                           140
                                  Exhaust Flow Meter Zero Observations
       5-INCH EFM ENVIRONMENTAL RADIATION ELECTROSTATIC
                  DISCHARGE ZERO DELTA MEASUREMENTS
                 THC [ppmC]  •  CO [ppm]    •  NO [ppm]    X NO2 [ppm]
                 Baseline    A  Zero      	Day Break    A CO2 [%]
                                                 NOx [ppm]
                                                                                0.028
                                                                                0.024
                                                                                0.020 —
                                                                                0.016
     -30
                                                                                0.012 5
                                                                                     O
                                                                                0.008 g
                                                                                     O
                                                                                0.004
                                                                                0.000
                                Reference Gas Delta Observations
 PEMS 7 ENVIRONMENTAL RADIATION CONDUCTED TRANSIENT ZERO
                             DELTA MEASUREMENTS
SwRI Report 03.12024.06
                            K-36

-------
                 THC [ppmC]  • CO [ppm]

                 Baseline    A Zero
                        NO [ppm]    X NO2 [ppm]

                       - Day Break    A CO2 [%]
NOx [ppm]
     60


     50
   E
   Q.

   £

   tfl

   §
   0>
   O
   =  0
     -10
   8 -20


     -30


     -40


     -50
           A    r«A
            __A_. _•_!
A A

                         «X*»fj|g*j|ti(

                                                                  A       A
                                                                               40
                                                                                 0.160
                                                                                 0.145
                                                                                 0.131
                                                                                 0.116
                                                               0.102 75
                                                               0.087
                  il

                  &
                  c
                  o

                  13
              0.073 •£
                  o>

              0.058 §
                                                               0.044
                                                               0.029
                                                               0.015
                                                                                 0.000
                                Reference Gas Delta Observations
 PEMS 7 ENVIRONMENTAL RADIATION CONDUCTED TRANSIENT AUDIT

                             DELTA MEASUREMENTS


60

Q.
V)
2
&
~- n -
entratior
j
3 C
8
40
-fin






M
•i


• THC [ppmC] • CO [ppm] • NO [ppm] X NO2 [ppm]


• • A A
"I1-*4* ,A ".,.
A •
• i * *
*** • •^•^••« ••^* •
5 AA3»0jfi»»W20 25 ^ 30
•.TjiS.?*.!^!1.!.1.!*!^!!.?.1! iiii
A • X •** x i
1 ••* x '
A
" •
• NOx [ppm]


1 •-• -• •-•
•

I * • • *
L • "A •
» • • •






"
t
X A
•




0 08

g
V)
c
0
0 02 13
> 1
u
O
O

- -nn4
                                Reference Gas Delta Observations
  PEMS 7 ENVIRONMENTAL RADIATION CONDUCTED TRANSIENT SPAN

                             DELTA MEASUREMENTS
SwRI Report 03.12024.06
                            K-37

-------
     2.0
     1.5 -
     1.0  -
     0.5 -
     0.0
     -0.5
     -1.5
                                                           • Flow Meter [scfm] • Baseline
                      *
                  **
*       *

                20
                        40
                                60
                                        80
                                                100
                                                        120
                                                                140
                                                                        160
                                                                                180
                                  Exhaust Flow Meter Zero Observations
  5-INCH EFM ENVIRONMENTAL RADIATION CONDUCTED TRANSIENT


                         ZERO DELTA MEASUREMENTS
            • THC [ppmC] BCO [ppm] «NO [ppm] XNO2 [ppm] «NOx [ppm]  Zero -Baseline ACO2 [%]
                                                                                0.020
                                                                                -0.015
                                Reference Gas Delta Observations
  PEMS 3 ENVIRONMENTAL VIBRATION ZERO DELTA MEASUREMENTS
SwRI Report 03.12024.06
                  K-38

-------
         • THC [ppmC] BCO [ppm] • NO [ppm] XNO2 [ppm] »NOx [ppm] -Baseline  Zero  Span ACO2 [%]
                                                                                0.12
           -----*?
                                                                                -0.02
                                Reference Gas Delta Observations
 PEMS 3 ENVIRONMENTAL VIBRATION AUDIT DELTA MEASUREMENTS
         • THC [ppmC] BCO [ppm] • NO [ppm] XNO2 [ppm] «NOx [ppm]   Zero -Baseline  Span ACO2 [%]
                                                                                0.16
     30	•	• - -•	0.12
                                                                           	0.04
                                                                               - -0.08
                                                                                -0.12
                                Reference Gas Delta Observations
  PEMS 3 ENVIRONMENTAL VIBRATION SPAN DELTA MEASUREMENTS
SwRI Report 03.12024.06
K-39

-------
                            APPENDIX L
              MEASUREMENT ALLOWANCE TEST PLAN

                  FINAL VERSION - OCTOBER, 2005
SwRI Report 03.12024.06

-------
Test Plan to Determine PEMS Measurement
   Allowances for the Gaseous Emissions
   Regulated under the Manufacturer-Run
 Heavy-Duty Diesel Engine In-Use Testing
                 Program
                Developed by:

    United States Environmental Protection Agency,
        California Air Resources Board, and
         Engine Manufacturers Association
               October 25, 2005

-------
Executive Summary
This test plan sets forth the agreed upon processes and methodologies to be utilized to
develop additive, brake-specific, data-driven measurement allowances for gaseous
emissions measured by PEMS (NOx, NMHC and CO) as required under the HDIUT
regulatory program.  A separate test plan will be developed and agreed upon for the
determination of the PM measurement allowance.

As detailed in this test plan, there is a clear consensus on what components of
measurement error are intended to be covered by the measurement allowances.  Namely,
the allowances are to be calculated in a manner that subtracts lab error from PEMS error.
Specifically, utilizing Part 1065 compliant emissions measurement systems and
procedures for both the lab and PEMS, the lab error associated with measuring heavy-
duty engine emissions at stabilized steady-state test points within the NTE zone, sampled
over 30-second durations will be subtracted from the PEMS error associated with
measuring heavy-duty engine emissions utilizing PEMS over 30-second transient NTE
sampling events under a broad range of environmental conditions.  This subtraction will
yield "PEMS minus laboratory" measurement allowances. The error model will not
subtract any laboratory accuracy or precision that is determined from laboratory
measurements of transient ~30-second NTE events. The experimental methods and
procedures specified in this test plan for determining, modeling, and comparing each of
the various components of measurement error are designed to generate statistically robust
data-driven measurement allowances for each of the gaseous emissions, namely NOx,
NMHC, and CO.

Successful completion of this test plan is part of the resolution of a 2001 suit filed against
EPA by EMA and a number of individual engine manufacturers. The suit challenged,
among other things, certain supplemental emission requirements referred to as "not-to-
exceed" (NTE) standards. On June 3, 2003, the parties finalized a settlement of their
disputes pertaining to the NTE standards.  The parties agreed upon a detailed outline for a
future regulation that would require a manufacturer-run heavy-duty in-use NTE testing
("HDIUT") program for diesel-fueled engines and vehicles. One section of the outline
stated:

"The NTE Threshold will be the NTE standard, including the margins built into the existing regulations,
plus additional margin to  account for  in-use measurement accuracy. This  additional margin shall be
determined  by the measurement processes and methodologies to be developed  and approved by
EPA/CARB/EMA. This margin will be structured to encourage instrument manufacturers to develop more
and more accurate instruments in the future."

Given the foregoing, the work to be completed under this test plan is a vital component to
the fulfillment of the settlement agreement, and it is vital to the successful
implementation of a fully-enforceable HDIUT program. Because of this significance, it
is critically important that the work detailed in this  test plan be carried out in as thorough,
careful and timely a manner as possible.

-------
Table of Contents

1   Introduction	6
2   Monte Carlo Error Model and Measurement Allowance	8
  2.1     Objective	8
  2.2     Background	8
  2.3     Methods and Materials	13
  2.4     Data Analysis	13
3   Engine Dynamometer Laboratory Tests	19
  3.1     Preliminary Audits	19
    3.1.1      Objective	19
    3.1.2      Background	19
    3.1.3      Methods and Materials	19
    3.1.4      Data Analysis	20
  3.2     Bias and Precision Errors under steady state engine operation	20
    3.2.1      Objective	20
    3.2.2      Background	20
    3.2.3      Methods and Materials	20
    3.2.4      Data Analysis	24
  3.3     Precision Errors under transient engine operation (dynamic response)	26
    3.3.1      Objective	26
    3.3.2      Background	26
    3.3.3      Methods and Materials	26
    3.3.4      Data Analysis	29
  3.4     Exhaust Flow Meter Installation	32
    3.4.1      Objective	32
    3.4.2      Background	32
    3.4.3      Methods and Materials	32
    3.4.4      Data Analysis	34
  3.5     ECM Torque and BSFC	37
    3.5.1      Objective	37
    3.5.2      Background	37
    3.5.3      Methods and Materials	39
    3.5.4      Data Analysis	42
4   Environmental Chamber Tests	45
  4.1     Data Analysis for all Environmental Tests	46
  4.2     Baseline	47
    4.2.1      Objective	47
    4.2.2      Background	47
    4.2.3      Methods and Materials	47
    4.2.4      Data Analysis	48
  4.3     Electromagnetic Radiation	48
    4.3.1      Objective	48
         Ba	49
    4.3.2      ckground	49
    4.3.3      Methods and Materials	49

-------
    4.3.4    Data Analysis	50
  4.4    Atmospheric Pressure	52
    4.4.1    Objective	52
    4.4.2    Background	52
    4.4.3    Methods and Materials	53
    4.4.4    Data Analysis	54
  4.5    Ambient Temperature	55
    4.5.1    Objective	55
    4.5.2    Background	55
    4.5.3    Methods and Materials	56
    4.5.4    Data Analysis	57
  4.6    Orientation, Shock, and Vibration	58
    4.6.1    Objective	58
    4.6.2    Background	58
    4.6.3    Methods and Materials	59
    4.6.4    Data Analysis	59
  4.7    Ambient Hydrocarbons	60
    4.7.1    Objective	60
    4.7.2    Background	60
    4.7.3    Methods and Materials	61
    4.7.4    Data Analysis	63
5   Model Validation and Measurement Allowance Determination	64
  5.1    Model validation	64
    5.1.1    Objective	64
    5.1.2    Background	64
    5.1.3    Methods and Materials	65
    5.1.4    Data Analysis	66
  5.2    Measurement Allowance Determination	70
    5.2.1    Objective	70
    5.2.2    Background	70
    5.2.3    Methods and Materials	70
    5.2.4    Data Analysis	70
6   Time and Cost	71
7   Appendices	73
  7.1    Description of Spreadsheet provided by Matt Spears	73
  7.2    Abbreviations used in Brake Specific Equations  (Table 3.3.4-a)	75

-------
Table of Figures

Fig 9
2.4-a. PDF	9
Table 2.2-a: Example of Calculation/Selection of Measurement Allowance	12
Figures 2.4 a, b, andc: Construction of an Error Surface	15
Table 3.2.3-a: Steady State Operating Conditions for 40 point matrix	23
Table 3.2.4-a: Steady State Error Surfaces (Refer to section 2.4)	25
Table 3.3.3-a: Dynamic Response NTE Events	27
Table 3.3.3-b: Dynamic Response Inter-NTE Events	28
Figure  3.3.3-a: Example of Transient Test Cycle	28
Table 3.3.4-a: Methods for Calculation of Brake Specific Emissions....Error! Bookmark
    not defined.
Table 3.3.4-b:  Dynamic response (Transient) Error Surfaces	31
Table 3.4.4.1-a: Exhaust Flow Configuration Error Surfaces (Refer to section 2.4)	35
Table 3.5.3.1-a: DOE Engine Operating Conditions (%speed, and %torque respectively)
    	40
Table 3.5.3.1-b: DOE Parameter Set Points	40
Table 3.5.3.3-a: Sensitivity Engine Operating Conditions (%speed, and %torque
    respectively)	41
Table 3.5.3.3-b: Sensitivity Parameter Set Points	41
Tables  3.5.4.1 through 3.5.4.6: Torque Error Surfaces	43
Table 4-a: Gas Cylinder Contents for 5 of 6 Environmental Tests	45
Table 4-b: Gas Cylinder Contents for Ambient Hydrocarbons	46
Table 4.3.4-a:  EMI / RFI Pooled Error Surface	51
Atmospheric Pressure Test Sequence	53
Time Series Chart of Atmospheric Pressure Test	53
Ambient Temperature Test Sequence	56
Time Series Chart of Ambient Temperature Test	56
Table 4.6.4-a:  Shock and Vibration Pooled Error Surface	60
Table 4.7.3-a:  Ambient Hydrocarbon Error Test Sequence	62
Table 5.1.3-a:  CE-CERT Model Validation Test Sequence	66
Chart of Model Validation NTE Events	67
Chart of PEMS vs Tow-along lab Results	68
Budget	71
Timeline	72

-------
1   Introduction
This test plan will establish PEMS measurement allowances for the gaseous emissions
regulated by the manufacturer-run on-highway heavy-duty diesel engine in-use test
program. The measurement allowances will be established using various laboratory
facilities and PEMS. The measurement allowances will be established in units of brake-
specific emissions (g/hp-hr), and they will be added to the final NTE standard for a given
emission, after all the other additive and multiplicative allowances have been applied.
This test plan will establish three measurement allowances; one for NOx, one for NMHC,
and one for CO.

The PEMS used in this test plan must be standard in-production makes and models that
are for sale as commercially available PEMS.  In addition, PEMS and any support
equipment must pass a "red-face" test with respect to being consistent with acceptable
practices for in-use testing. For example, use of large gas bottles that can not be utilized
by the EPA/ARB/EMA HDIU enforceable program is unacceptable. Furthermore, the
equipment must meet all safety and transportation regulations for use on-board heavy-
duty vehicles.

Even though the PEMS can not be "prototypes" nor their  software "beta" versions, the
steering committee has already agreed that after delivery of PEMS to the contractor, there
may be a few circumstances in which PEMS modifications might be allowed, but these
modifications must meet certain deadlines, plus they are subject to approval by the
steering committee. Also, any implementation of such approved modifications will not
be allowed to delay the test plan, unless the steering committee specifically approves
such a delay. Table 1 summarizes these allowable modifications and their respective
deadlines:
Table 1: Allowed Modifications
Steering committee approved hardware and software modifications
that affect emissions results; including but not limited to fittings,
components, calibrations, compensation algorithms, sampling rates,
recording rates, etc.
Steering committee approved hardware modifications for DOT
approval or any other safety requirement approval
Delivery of any environmental / weather enclosure to contractor
Post-processing software to determine NTE results
DOT approval and documentation
Steering committee approved hardware or software that improves
the contractor's efficiency to conduct testing and data reduction
Before start of. . .
Section 3.2
Section 4.3
Section 4.3
Section 5.1
Section 5.1
Always Allowed
This test plan describes a computer model, a series of experiments that are used to
calibrate the model, and another series of experiments that are used to validate the
calibrated model.

-------
The test plan first describes the computer model.  The computer model statistically
combines many sources of PEMS and lab error, which are nearly impossible to capture
simultaneously in a single test. The model will use statistics to apply the errors in a way
that simulates actual running of a PEMS in-use.  The model will also consider only the
portion of error that is attributable to PEMS, and it will subtract the error that is already
tolerated in an emissions lab today. The model will also calculate and validate results
according to 40 CFR Part 1065.

The test plan then describes the series of experiments. These tests will characterize the
many sources of PEMS and lab error  so that the specific nature of the errors can be
programmed into the computer model. The nature of the error has to do with the way
PEMS and the lab react to certain conditions.  For example, under varying environmental
conditions such as temperature or vibration, a PEMS might exhibit signal drift, or it may
record noise that is not a part of the true emissions.

Next the experimental results will be  entered into the computer model, and the
measurement allowances are calculated by the model. The model uses a "reference"
PEMS data set, which will have many "reference NTE events." The model statistically
applies all the errors to the reference data set, calculates results, and saves the results.
Then the model will be run with all errors set to zero to calculate the ideal results of the
reference data set.  Each difference between a reference NTE event's result with errors
and its respective ideal result will be a brake-specific difference that is recorded for later
use. Then the process repeats using the same reference data set, to which new,
statistically selected errors are applied, and thus another unique set of differences is
calculated. As the model continues to iterate and generate more and more  results,
patterns are expected to appear in the output data. These  patterns should be the
distributions of differences, based upon the error that was statistically and repeatedly
applied to the reference data set.  Many difference distributions will be determined: for
each reference NTE event, for each three regulated gaseous emissions, and for each of
three brake-specific calculation methods. It has been agreed that the 95th percentile
values of these distributions will  be taken as reasonable "worst case"  results for each
reference NTE event.  Details on how all these distributions will be reduced to determine
the three gaseous measurement allowances are given in the "Error Model" section of this
test plan.

Finally, the test plan describes how the computer model will be validated against real-
world over-the-road in-use PEMS operation as well as additional lab testing. For the
over-the-road testing, PEMS emissions measurements will be conducted, while at the
same time a reference laboratory  will be towed along to measure the same emissions. For
the lab testing, an attempt will be made to simulate real-world engine operation to
"replay"  an over-the-road test in the lab. Data from these final experiments will be used
to validate the model, which must be  done in order to gain sufficient confidence that the
model did not establish unreasonable  measurement allowances.

The following sections of this test plan are written as instructions to the contractor or
contractors who will complete the test plan.

-------
2   Monte Carlo Error Model and Measurement Allowance

2.1  Objective
Use Monte Carlo (e.g. random sampling) techniques in an error model to simulate the
combined effects of all the agreed-upon sources of PEMS error incremental to lab error.
Create error "surfaces" for the Monte Carlo simulation to sample, based upon results
from the experiments described in Sections 3 and 4.  Exercise the model over a wide
range of NTE events, based on a single, reference data set of at least 100 unique NTE
events. Determine the pollutant-specific brake-specific additive measurement allowances
for NOx, NMHC, and CO.

2.2  Background
The error model uses Monte Carlo techniques to sample error values from "error
surfaces" that are generated from the results of each of the experiments described in
Section 3: lab tests, and Section 4: environmental tests. The lab test error surfaces cover
the domain of error versus the magnitude of the signal to which the error is to be applied
(i.e. 5th to 95th percentile error vs. concentration, flow, torque, etc.). This is illustrated
later in this section. The environmental test error surfaces for shock & vibration and
electromagnetic & radio frequency interference (EMI/RFI) cover the same domain as the
lab tests, but only for concentration.  The environmental test for ambient hydrocarbons is
similar but the error surface does not change as a function of concentration. The
environmental test error surfaces for pressure and temperature are characteristically
different because the cover the domain of environmental test cycle time versus the
magnitude of the signal to which the error is to be applied (i.e. error at a selected time vs.
concentration). Details on how each surface is generated are given in each of the
respective sections. These surfaces will already be adjusted to represent PEMS error
incremental to lab error; therefore, these surfaces are sampled directly by the model.

The error model will use two different probability density functions (PDFs) to sample the
error surfaces, depending upon which experiment the surface represents. To sample error
surfaces that are generated from all the laboratory test results  (Section 3), and the
environmental test results for  shock & vibration; EMI/RFI; and ambient hydrocarbons,
the model will use a truncated normal PDF because these tests are designed to evenly
cover the full, but finite, range of engine operation and ambient conditions.

-------
,
Probability Density Functions for Sampling Error Surfaces Once Per NTE Event
	 Lab Tests, Normal, SD=0.60795, truncate @ -1 & 1
^—Environmental Tests, Random

Note: A non-trur
distribution with
values of 0.05 ai
ic=+1, respective





icated normal
SD=0.60795 has
id 0.95 at ic=-1 ar
;ly.
^
c
^
^^


p
d ^^^
^*^


^^
^^

^
*^^




^^^
^^^
> 4 3 2
Relative Probability








(
1.00
0.75
0.50
0.25
0.00 .-"
-0.25
-0.50
-0.75
-1.00
                                        Fig
                                    2.4-a. PDF.

To sample error surfaces that are generated from the pressure and temperature
environmental test results (Section 4), the model will use a random PDF because these
tests are already designed to cover the typical range and frequency of the respective
conditions. Note that the lab's normal PDF samples ic, from -1 to 1, including -1 and 1,
and the pressure and temperature environmental tests use a random PDF to sample test
time, from which the nearest (in time) calculated errors are used. The errors from the
other tests will be aligned with the normal PDF such that each of the 50th percentile
values at each of the tested signal magnitudes is centered at the median of the PDF and
the 5th and 95th percentile error values at each of the tested signal magnitudes will be
aligned with the extreme negative (ic = -1) and positive (ic = +1) edges of the PDF,
respectively.

Each error surface will be sampled along its ic axis (y-axis)  once per reference NTE event,
and it will be sampled along its parameter value axis (x-axis, e.g., concentration, flow,
torque, etc...) once per second, within a given reference NTE event. An error will be
determined for a given second  and parameter along the error axis (z-axis) at the
intersection of an ic value and a parameter value.

To ensure that the magnitudes of the error surfaces are appropriate, each data point used
to generate the surfaces will be a mean or a weighted mean  of 30 seconds of sampling.

Interpolation will be performed by first  linearly interpolating error values at each tested
magnitude along the selected line perpendicular to the ic axis. Then from that line of
errors, individual error values will be linearly interpolated at each second-by-second
signal magnitude of the given NTE event in the reference data set.  If good engineering

-------
judgment dictates that unevenly spaced data, sparse data, or other data irregularities
warrant more sophisticated interpolation techniques, the steering committee will consider
alternatives and provide guidance to the contractor.

The reference data set to which all errors will be applied will be a large data set of engine
operation over a wide range of NTE events.  This reference data set will be initially
generated from collections of real-world PEMS data sets.  The reference data set should
contain at least 100 unique NTE events. Parameters in the reference data set may be
scaled in order to exercise the model through a more appropriate range of parameters (i.e.
concentrations, flows, ambient conditions, etc.). If the parameters are scaled, care should
be taken to maintain the dynamic characteristics of the reference data set.

After the errors are applied, NTE brake-specific emissions results are calculated for NOx,
CO and NMHC, using each of the three agreed-upon NTE calculation methods. Next,
the NTE events are calculated by each of the three calculation methods, but with no error
sampled or applied to the reference data set.  These results are considered the "ideal"
results of the reference NTE events. These ideal results are subtracted from each
respective NTE event, and the difference is recorded. Then a new set of errors are
sampled and applied to the reference data set, and the NTE results are calculated again.
The ideal results are again subtracted, and the difference is recorded. This is repeated
thousands or possibly even millions of times so that the model converges upon
distributions of brake-specific differences for each of the original NTE events in the
reference data set. Then the 95th percentile difference value is determined for each NTE
event's distributions of brake-specific differences for each emission (NOx,  NMHC, and
CO) for each calculation method.

The three different brake-specific emission calculation methods referred to  in this test
plan are i) Torque-Speed Method, ii) BSFC method, and iii) Fuel Specific method, and
these are illustrated in the  same order in Figure 2.4-b, below. The symbol notation for
these equations is described in 40 CFR  Part 1065 Subpa