United States       Air and Radiation      EPA420-R-01-025
            Environmental Protection               April 2001
            Agency                     M6.STE.001
vvEPA     Comparison of Start
            Emissions in the
            LA92 and ST01 Test Cycles
                                 > Printed on Recycled Paper

-------
                                                                          EPA420-R-01-025
                                                                                 April 2001
                                                            in  the
                            and

                               M6.STE.001
                                    Phil Enns
                                 David Brzezinski

                         Assessment and Standards Division
                      Office of Transportation and Air Quality
                       U.S. Environmental Protection Agency
                                    NOTICE

    This technical report does not necessarily represent final EPA decisions or positions.
It is intended, to present technical analysis of issues using data which are currently available.
         The purpose in the release of such reports is to facilitate the exchange of
      technical information and to inform the public of technical developments which
        may form the basis for a final EPA decision, position, or regulatory action.

-------
1.0    Introduction

       As part of the MOBILE inventory model revision, an effort has been undertaken to model
separately start emissions occurring at the beginning of a trip. A special start driving cycle,
described below, has been developed for the purpose of measuring this portion of emissions in
laboratory testing. However, sufficient data from such tests is not available for use in MOBILE
at this time. The Federal Test Procedure (FTP) and its California replacement the LA92, also
described below, do include engine start, but use different driving patterns than the special start
driving cycle.

       This document reports on a comparison of start emissions for two test cycles using data
from a sample of five vehicles tested at the EPA National Vehicle and Fuel Emissions
Laboratory in Ann Arbor, Michigan. The purpose of the analysis is to determine if, during the
start portion of the cycles, there is a significant difference between the cycles in their excess
emissions attributable to a cold start condition.

2.0    Vehicle Sample and Testing

       The two cycles used in this study were developed recently to serve the needs of revised
emissions testing. The 258-second "ST01" cycle was developed as part of EPA's Revised FTP
project. It is the first 258 seconds of a longer cycle known as the SC03.  For more information
about the FTP study and the development of the SC03 driving cycle, see the EPA web site
(http://www.epa.gov/otaq/sftp.htm). The ST01 is designed to simulate typical driving during the
beginning of atrip and is comprised of observed speed segments of real driving collected as part
of that project. The "LA92" cycle (also called the Unified Cycle) was created by the California
Air Resources Board to replace the FTP cycle for vehicle emission certification in  California.
More information about the Unified cyclecan be found on the California Air Resources Board
web site (http://www.arb.ca.gov/research/resnotes/notes/96-l 1.htm).  The LA92 also is
constructed from segments of actual driving, recorded in the Los Angeles area, and includes
elements of driving that are more "aggressive"  than any found on the FTP. While the full LA92
cycle lasts 1,436 seconds, only the first 298 seconds  are considered in this study. The first 298
seconds of the LA92 matches the elapsed time of the ST01  cycle plus the time needed for the
vehicle to return to idle. Figure 1 displays the speed traces for the two cycles.  The statistics for
these cycles, compared to the first 505 seconds of the LA4 cycle, are shown in Table 5.

       Determining when the cold-start portion of a trip ends is not a trivial problem. A typical
pattern of modal emissions for identical  driving under cold-start and (warm) no-start driving is
depicted in Figure 2. In the absence of test variation, the two graphs converge at a specific time
which can be defined as the end of the cold start portion. In practice, this point of convergence is
not obvious because of random fluctuations from one test to another. However, in the current
study if it is assumed that the cold-start and no-start emissions eventually converge within the
first 258 seconds, then knowledge  of the exact time of that convergence is not needed in order to
compute relative emissions from the two operating modes. When the difference between the cold
M6.STE.001                                2                                  April 2001

-------
and no-start emissions on a given cycle is calculated, the post-convergence values cancel.

       Using this assumption, the primary null hypothesis can be stated as follows: the average
difference between cold-start and no-start emissions is the same for the ST01 and LA92 cycles.
The average differences are shown graphically in Figure 4.  In the current experiment, each of
five vehicles was driven over the two cycles in both a cold-start and warm no-start condition, a
total of twenty tests. (A third "hot-start" test also was performed in which the warmed-up vehicle
started from engine off mode.)  Duplicate tests were not done.  From these tests, a simple paired
difference test can be constructed by first computing the excess of cold-start over no-start
emissions for each vehicle on each cycle, and then differencing these values by vehicle.

       Table 1 gives characteristics of the five vehicles employed in the study. Figure 3 shows
the second-by-second HC emissions for one of these vehicles on the first 298 seconds of the
LA92 cycle in the cold- and no-start conditions.  Also shown are the cumulative emissions for the
two tests. The difference in cumulative emissions at 258 seconds (the end of the shorter ST01
test) becomes the basic data measurement on which the cycle comparison is based. Table 2 lists
all the cumulative values for CO, HC, and NOx; excess cold-start emissions by start condition
and resulting cycle differences appear in Table 3.

3.0    Results and Conclusions

       Table 3 shows final t-test results for the hypotheses that the average difference between
cold-start and no-start emissions is the same for the ST01 and LA92 cycles. These small sample
tests are non-significant for all three pollutants. In other words,  they support the idea that,  on
average, excess emissions from cold-start operation are no different for the LA92 cycle than for
the ST01 cycle. The variability of the emission results, especially for NOx emissions, was very
high. Further analysis may be warranted to investigate the reasons for this variability or to
increase the sample size.

       It should be noted, of course, that these results do not conclusively prove that excess
emissions for a cold start are the same on ST01  as for any other driving pattern. Imputing that
conclusion to other cycles (such as the FTP cycle)  should be done with caution. Moreover,
because the number of vehicles tested was small, the power of the t-test to detect a difference
between cycles is limited.

       Nevertheless, this study supports the concept that the increment in emissions caused by
engine start is reasonably independent of the underlying driving cycle. For the MOBILE6 model,
the emissions caused by engine start will be extracted from existing FTP and LA92 emission
testing data. It is proposed, for purposes of modeling with MOBILE6, that the emissions from
engine start will be assumed to be the same, regardless of the driving which occurs after engine
start.  Other factors, such  as temperature, fuel composition and  soak time since the last engine
running will still be used to affect the emissions from engine start for particular, user specified
scenarios.

M6.STE.001                                 3                                  April 2001

-------
       There were no specific comments or criticisms of the conclusions found in this report
during the development of the MOBILE6 model.
M6.STE.001                               4                                 April 2001

-------
                                      Table 1
                         Test Vehicle Sample Characteristics
VEHICLE
5174
5177
5181
5182
5183
MODEL YEAR
91
94
94
94
94
MAKE
CHEVROLET
FORD
OLDSMOBILE
BUICK
SATURN
MODEL
CORSICA
THUNDERBIRD
ACHIEVA
ROADSTER
SATURN
VEHICLE
5174
5177
5181
5182
5183
VIN
1G1LT53G3ME142337
1FALP624ORH1 10885
1G3NL15D7RM029502
1G4BN52P3RR420339
1G8ZJ5574RZ301364
ENGINE FAMILY
M1G2.2V5JFG3
RFM3.8V8GAEA
R1G2.3VHGFEA
R1G5.7V8GAEE
R4G1.9VHGBEA
TRANSMISSION
AUTOMATIC
AUTOMATIC
AUTOMATIC
AUTOMATIC
MANUAL
VEHICLE
5174
5177
5181
5182
5183
CID
134
231
139
350
145
DRIVE TRAIN
FWD
RWD
FWD
RWD
FWD
CATALYST
3 -WAY
3 -WAY
3 -WAY
3 -WAY
3 -WAY
CYLINDERS
4
6
6
4
8
FUEL SYS
TBI
PFI
PFI
PFI
PFI
M6.STE.001
April 2001

-------
Table 2
258 Second Cumulative Emissions
(total emissions in grams)
Vehicle

5174
5177
5181
5182
5183
Cycle

LA92
ST01
LA92
ST01
LA92
ST01
LA92
ST01
LA92
ST01
No Engine Start
CO
5.74
6.17
9.22
15.76
3.58
5.3
0.39
0.78
5.91
4.95
THC
0.42
0.48
0.36
0.34
0.05
0.05
0.02
0.02
0.21
0.11
NOx
0.49
0.67
0.76
0.48
0.26
0.24
0.02
0.09
0.12
0.13
Cold Engine Start
CO
24.75
23.49
19.32
28.05
12.76
14.21
9.09
12.92
19.25
17.59
THC
1.96
2.25
1.92
2.08
1.31
1.26
1.13
1.2
1.71
1.39
NOx
1.16
2.41
1.87
0.51
0.88
0.97
0.29
0.53
0.49
0.19
M6.STE.001
April 2001

-------
Table 3
Excess of Cold-Start Over No-Start Emissions
By Vehicle and Cycle with Paired Difference (grams)
Vehicle
5174
5177
5181
5182
5183
Cycle
LA92
ST01
Difference
LA92
ST01
Difference
LA92
ST01
Difference
LA92
ST01
Difference
LA92
ST01
Difference
CO
19.01
17.32
1.69
10.1
12.29
-2.19
9.18
8.91
0.27
8.7
12.14
-3.44
13.34
12.64
0.7
THC
1.54
1.76
-0.22
1.56
1.75
-0.19
1.26
1.21
0.06
1.11
1.18
-0.07
1.5
1.28
0.22
NOx
0.67
1.74
-1.07
1.1
0.04
1.07
0.62
0.73
-0.11
0.26
0.44
-0.17
0.37
0.06
0.31
M6.STE.001
April 2001

-------
Table 4
Paired Difference T-Tests for Differences in Table 3
Statistics
Minimum (grams)
Maximum (grams)
Mean Difference (grams)
Std Deviation (grams)
Standard Error
T
PROB > T
CO
-3.44
1.69
-0.59
2.14
0.96
-0.62
0.57
THC
-0.22
0.22
-0.04
0.18
0.08
-0.49
0.65
NOx
-1.07
1.07
0.01
0.78
0.35
0.02
0.99
M6.STE.001
April 2001

-------
Table 5
Driving Cycle Statistics
Cycle
Time
Distance
Average Speed
Maximum Speed
Maximum Acceleration
Average Power*
LA92
298
1.2
13.6
41.1
5.8
19.5
ST01
258
1.4
19.4
40.1
5.1
21.4
LA4 Bag 1
505
3.6
25.6
56.7
3.3
19.6
Units
seconds
miles
miles per hour
miles per hour
mph/second
(mph)2
       *This metric represents the mean specific power of the entire driving cycle.  Specific
power was calculated for each second from the following equation:
0
                                                      , if St <^_,
       where St and St., are the vehicle speeds at times t and t-1, respectively. The average
power metric reported was calculated from the mean of the specific power calculated for each
second, including zeros.
M6.STE.001
                                           April 2001

-------
                                    Figure 1
                              LA92 SPEED TRACE
              50

              40

              30-

              20'
                                                  Kl	I
             50

             40

             30'

             20'

             1
                               100
                                   TIME (SECONDS;

                              ST01 SPEED TRACE
                               100             200
                                   TIME (SECONDS)
M6.STE.001
                                        10
April 2001

-------
H
W
o
o
                                                                   Figure 2
                     E
                     M
                     I
                     S
                     S
                     I
                     0
                     H
                     S
                                       N./
                                                                        TIME
to
O
o

-------
                                    Figure 3
                     HC  Emissions    Vehicle 5174
                0.031
               0.02-
               o.oi-
               0 . 0(H
                     0
 100
200
300
                                      SECONDS
                                COLD  STA   	NO_3TAET
              Cumulative HC  Emissions    Vehicle SI74
               2 . 0:


               1 . 5-
               1 . 0;
               o . 5:
               o .  o-t
                    0
100          200
     SECONDS
                                                           300
M6.STE.001
                                       12
                                        April 2001

-------
  14
  12
  10
                         Rgure4
          Average of the Emission Differences
                  Due to Engine Starts
 (A

 TO
 O)
ILA92
ISTO1
            CO
   THC
NOx
M6.STE.001
   13
     April 2001

-------
                                      Appendix A

                                       Test Plan
                       Evaluation of the Effects of Driving Cycles
                                on Cold Start Emissions
BACKGROUND

       A theory has been presented that cold start emissions are independent of specific driving
circles.  In other words, no matter what driving occurs immediately after a vehicle is started the
emissions over the time it takes for the engine to warm up are the same.

PURPOSE

       The purpose of this testing is to gain some insight into whether or not the cold start
emissions of a vehicle are independent of driving patterns. This is a preliminary investigation
which could result in further testing.

RECRUITMENT

       Five vehicles shall be selected to receive a series of cold start tests after the normal
Emission Factor Indolene test sequence. Because the data are needed quickly, the next five
available EF vehicles shall be used.  The vehicles selected shall be 'normal' emitting.  No high
emitting vehicles are to be included in this sample.  If it is determined, after testing begins, that a
vehicle is a high emitter it shall be removed from this program and another vehicle selected in its
place.

TESTING

       This testing shall be performed after the normal EF sequence on Indolene. The sequence
shall be as shown on the attached flowchart.  The cycles to be used for this testing will be the
ST01 (first 258  seconds of the SC03 cycle) and the first 298 seconds of the LA92 (Unified
Cycle).  Each of these cycles will be performed modally both as a cold start, hot start, and
running start test.

TEST SEQUENCE

       The test sequence shall begin no sooner than four hours after the Indolene testing has
been completed. The sequence shall begin by draining the fuel and filling the tank to 40%, by
volume, with Indolene test fuel.  An LA4 prep cycle shall be driven and vehicle soaked for a
minimum of 12 hours. An effort shall be made to soak the vehicle approximately the same
length of time after each prep, before each cold start. The vehicle will then receive a cold start
M6.STE.001                                14                                 April 2001

-------
ST01 with second by second dilute modal emission measurements and a bag sample.  An
unmeasured hot LA4 will then be driven followed by a 10 minute soak and a hot start ST01 and a
running ST01 after the hot start test (back to back ST01 cycles).

       No sooner than four hours later, an LA4 prep cycle shall be driven and the vehicle soaked
for a minimum of 12 hours. The vehicle will then receive a cold start with second by second
dilute modal emission measurements and a bag sample of the first 298 seconds of the LA92
cycle. An unmeasured hot LA4 will then be driven, followed by a 10 minute soak and a hot start
298 second test and a running 298 second test (back to back 298 second tests).

       The order of testing shall be alternated for each vehicle.  The first vehicle will have the
ST01 sequence first, the second vehicle will have the 298 second sequence first. The third
vehicle, the ST01 first, and so on.

DATA COLLECTION

       All data collected on these cycles will be second by second dilute modal and be
simultaneously collected in a bag.
M6.STE.001                               15                                April 2001

-------
                                       Appendix

                                 Response to Comments


       AIR Comments:

       Air Improvement Resource, Inc., commented on June 4, 1997 on this report:

       The major comment or question is why these two particular start cycles were chosen to
       determine the sensitivity of emissions to start cycle? Examination of the speed-time traces
      for the two test cycles in Figure 1 shows that they appear to be very similar with respect
       to average speed, acceleration, and the placement of the modal patterns. (No statistics on
       the cycles were presented in the report.) The vehicles tested experienced similar HC and
       CO for both cycles, but as was pointed out in the report, NOx emissions were quite
       different on both cycles, and there appeared to be no clear trends (one cycle was not
       necessarily higher than another for NOx). So, if the start cycles are very similar, then one
       would not expect to see much difference in the cold start increment. My recommendation
       would be to provide a comparison of the two cycles, in terms of average speed, minimum
       and maximum acceleration, power, etc. so that reviewers will have a better idea how
       different these cycles are.

       The ST01 cycle was developed specifically to represent the observed driving behavior
following an engine start using data from instrumented vehicles. The LA92 is more
representative of driving cycles (like the LA4 cycle used in the Federal Test Procedure, FTP)
which do not explicitly attempt to account for the effect of engine starts on the first portion of
driving behavior.  The hypothesis being tested is to determine if, within the relatively small
differences between the two cycles, there is a disproportionate difference in engine start
emissions.  If a large difference was observed, this would suggest that the choice of driving cycle
would be critical to estimating engine start emissions. The small differences we observed,
suggest that a fairly accurate estimate of engine start emissions can be obtained with any
reasonably constructed inventory driving cycle.  The statistics related to the driving cycles you
requested have been added to the report in Table 5.

       The report states that EPA will be developing cold start emissions from data collected on
       the LA-4 and the LA92. The LA92 was tested in this start emissions test program,  why
       wasn't the LA-4? The LA-4 start cycle (or first 250 seconds) is very different from these
       other two cycles (the second mode of the LA-4 exceeds 50 mph). I think that this is
       interesting work, but I would be much more comfortable with the conclusion (that the
       start increment is independent of the cycle) had EPA selected three or four start cycles
       that represented a fuller range of start driving behavior.

       At the time of this study, it was not clear that the LA4 would be used to estimate engine


M6.STE.001                                16                                 April  2001

-------
start emissions, since the ST01 was developed specifically to estimate emissions from driving
behavior following an engine start. The LA92 was a candidate cycle for estimating baseline
emission rates from vehicles, as a substitute for the LA4 driving cycle.  The initial focus of this
study was to determine if it would be necessary to measure an ST01 driving cycle in addition to
an LA92 driving cycle for each vehicle in order to appropriately estimate the effects of an engine
start. The scope of the conclusions of this study have been expanded to include the idea that the
LA4 would be used to determine engine start emissions, instead of the LA92.

       The test plan listed in the Appendix shows that EPA conducted some "running start" tests,
       and yet the results of these were not summarized in report. Does EPA plan to provide
       these at a later date?

       The emissions labeled "No Engine Start" in Table 2 are the "running start" test results for
these vehicles. It is the difference between the emissions measured which include an engine start
and those that do not which are analyzed in this report.

       Once the LA4 cycle was chosen to represent driving behavior after an engine start,  a "hot
running" 505 (HR505) measurement was added to a vehicle sample that was collected for use in
MOBILE6. This measurement was performed on a fully warmed engine and did not include an
engine start. The HR505 results  could then be directly compared with Bag 1 and Bag 3 of the
FTP test procedure results (which all use the first 505 seconds of the LA4 driving cycle) to
determine the effects of engine starts on emissions.  The results of this study are summarized in
the report, "The Determination of Hot Running Emissions From FTP Bag Emissions,"
(M6.STE.002) EPA-420-P-99-014.  The  data is included in the full Mobile Source Observation
Database (MSOD) which is available from EPA on request.
M6.STE.001                               17                                 April 2001

-------
      Review of "Comparison of Start Emissions in the LA92 and ST101 Test Cycles"
                                 John H. Warner, Ph.D
                              Brenda Wilson Gillespie, Ph.D
                    Center for Statistical Consultation  and Research
                              The University of Michigan
                                      May 6,1998

This document consists of a review, divided into two sections, of the report by US EPA Assessment
and Modeling Division prepared by Enns and Brzezinski (1997). Section 1, contains a brief synopsis
of the report;.  Section 2, is comprised of several recommendations for improving the statistical
methodology of the report.

In general, while the statistical tests presented in Enns and Brzezinski (1997) are sound and
appropriate, we suggest that:

       (1)    non-parametric tests and tests for normality should also be presented;
       (2)    that additional diagnostic tests be performed; and
       (3)    that several additional summary statistics be calculated to aid in the interpretation
             of the results.

We also suggest methods that could be used to analyze more complex experimental designs for
related future research.

1      Synopsis of the Report

The purpose of Enns and Brzezinski was to test the hypothesis that start up emissions are
independent of starting driving cycle.

The experiment conducted to test this hypothesis has the form of a classical cross-over design. (See
Jones and Kenward [1989], Ratkowsky, Evans, and Alldredge[ 1993], and  the references cited in
these  works.)  To describe the design in a  little more generality  than was  done by Enns and
Brzezinski:

       (1) let A and B represent two test cycles that are to be compared;,
       (2) let C be a third test cycle used to warm the vehicle up; and
       (3) let Wl3W2 and W3 be three waiting periods.

In Enns and Brzezinski A was the STO1 cycle, B was the LA92 cycle, C was the LA4  cycle, Wl
waiting periods was a 12 hours wait, W2 was 4 hour wait, and W3 was a 10 minute wait.
M6.STE.001                               18                                 April 2001

-------
In the experiment described by Enns and Brzezinski, roughly half the cars are put through
the sequence:
                                                                                      (1)

and the other half of the vehicles are put through the sequence

                                                                                      (2)
Although this design looks complicated, it can be considereedto be a simple 2x2 cross-over design,
where, abusing our notation slightly, (1) can be calledAB and (2) can be called BA. (There is a large
body of literature on cross-over designs and the issues involved in their analysis.) The standard
analysis for this type of design typically involves tests for treatment (A vs.B), period(l vs.  2),
sequence(AB vs. BA), and two-way interactions.  In a 2 x 2 cross-over trial, only three degrees of
freedom are available for performing tests, therefore, interpretation of significant results from some
tests may be ambiguous.

In biological experiments, tests of carry over effects from the first period to the second period  are
often of interest.  In the study of Enns and Brzezinski, it  is unlikely  that carryover effects will be
present, and period  effects are likely  to be small  or nonexistent.  However, given the cross-over
design, tests of such effects should be performed as a precautionary measure.

More formally, cars put through the sequence (1) will be  called Order 1 cars and cars put through
the sequence (2)) will be called Order 2 cars. A is measured in the first period for Order 1 cars and
the second period for Order 2 cars. Similarly, B is measured in the first period for Order 2 cars and
the second period for Order 1  cars.

Continuous time measurements of the emission of CO, HC, and NOx where taken at the first,
second, and third runs of cycle A and the  first, second,  and  third of runs of cycle B  These
measurements will be designated as the cold start, warm start, and nostart measurements of cycles
A and B respectively. Enns and Brzezinski analyze only the  cold start and nostart measurements
and (correctly in our view) perform a  separate analysis for each emittant.

Let X denote an arbitrary emittant. Interest focus on the measurement of the contrast:

           CA3 = (X(cold,A) - X(nostart,A)) - (X(cold,B) - X(nostart,B)).                  (3)

In particular,

           CA = X(cold,A) - X(nostart,A)                                               (4)

represents the emissions of X  in cycle A that can be attributed to the cold start and


M6.STE.001                                19                                 April 2001

-------
           CB = X(cold,B) - X((nostart,B)                                             (5)

represents the emissions of X in cycle B that can be attributed to the cold start, and CAB represents
the differences in such emissions between cycles Table 3 in Enns and Brzezinski shows that CAand
CB are both positive for all cars  and emittants cycles tested.  Enns and Brzezinski test the null
hypothesis that CAB is zero by using a two sided one sample T-test.  This is correct, although the
bottom row in Table 4 is labeled as though it contained p-values for a one sided test.

In order to insure the validity of the T-test described in the previous paragraph, we should verify that
the contrasts CA and CB do not depend on the order in which the measurements were made. To
make this explicit, we may label four new contrasts as follows:

           CA1  = X(cold,A,l) - X(nostart,A,l)
           CA'2  = X(cold,A,2) - X(nostart,A,2)
           CB1  = X(cold,B,l) - X(nostart,B,l)
           CB'2  = X(cold,B,2) - X(nostart,B,2)                                          (6)

Several hypothesis tests can be performed to investigate order effects among the contrasts in (6).
These include tests of the hypothesis:

           r   - r   = r   - r                                                        n~\
           ^B,2  ^A,l   ^B,l  ^A,2                                                     \'J
           CB,2 + CA^ = CAa+CB.                                                    (8)
                 CA,! = CA,2                                                        (9)
                 CB,2 = CB,!                                                        (10)

The first of these hypothesis tests, (7), is known, in terminology of cross-over designs, as a test for
a period effect;  the second, (8), is a test for a period by treatment interaction; (9) and (10) are tests
which can be used to identify the source of unwanted order effects.  In general, all of the hypothesis
(7-10) can be tested using two sample T-tests or two sample non-parametric tests. To see this, note
that all of the left hand sides  of the equations (7-10) can be computed from contrasts measured on
Order 1 cars and the right hand sides of the equations (7-10) can be measured from contrasts on
Order 2 cars.  If any of the above tests are  significant, it is evidence for an order dependence in the
measurement of CA and/or CB.  If such a dependence is found, it might be cured by lengthening the
soak time, Wt.

Alternatively, one might need to identify and neutralize a confounding factor that is correlated with
period. For example, it might be  discovered that first period experiments tended to be carried out
in the morning and second  period  experiments  tended to occur in the afternoon.  If weather
conditions tended to differ between morning and afternoon, this could, in theory,
affect emissions measurements at these times.

One way to analyze the above experiment is to use a repeated measures analysis of variance, as in
SAS  PROC REG (SAS,  1989).    This  ANOVA  would  include two within subjects factors,
M6.STE.001                                20                                  April 2001

-------
START(cold, no) and CYCLE(A,B), and a between subjects (or grouping) factor ORDER(1,2), 1
for the order in (1) and 2 for the order in (2). (A subject in this experiment is a specific car). In the
repeated measures framework, the test of the null hypothesis that CA B = 0 would be equivalent to the
test of the START by CYCLE interaction, the period effect  (7) would be tested by checking the
START by CYCLE by ORDER interaction, and the period by treatment effect (8)would be tested
by checking the START by ORDER interaction. All of the tests for the above repeated measures
ANOVA are  equivalent to one or two sample T-tests.  The repeated measures framework  can,
however, accommodate designs which cannot be analyzed by simple T-tests. This would occur if,
additional grouping factors were added to the design to test for effects caused by characteristics of
the vehicles being tested, i.e. high, medium, or low emitters,  or if additional levels of the START
factor (e.g. warm start) were added.

2   Recommendations

We suggest that the T-tests presented in Table 4 be supplemented with non-parametric Wilcoxon
signed rank tests and sign tests . As is well know, T-tests assume that the contrast in (3) is normally
distributed. Formal tests for normality should be performed and normal probability plots should be
consulted to evaluate the accuracy of this assumption.  With the extremely small sample sizes
encountered in the report, however, substantial doubt regarding the assumption of normality is likely
to  remain after all such tests have been completed and all of such plots observed.  Under these
conditions it is best to guard oneself against the possibility of error by performing parallel  analyses
which have lower power but fewer assumptions. Although less powerful than the Wilcoxon signed
rank test, we suggest that the sign test be presented, in addition to the Wilcoxon Signed Rank  test,
because exact small sample p-values are generally available for the sign test, even in the  presence
of ties.   Both  non-parametric tests and  tests for  normality are readily available in PROC
UNIVARIATE  in SAS (SAS (1990)).  Exact  p-values for the Wilcoxon signed rank test are
available (for example) in the S-PLUS statistics package (Statistical Sciences (1995) unless ties are
present in the data.

We suggest that standard errors and confidence intervals be presented for the  contrasts in (3), (4),
(5), and (6). Confidence intervals are useful, even in the absence of statistically significant results,
because they help one assess the strength of the evidence in favor of the null hypothesis.

We further suggest that  a standard battery of  tests, pertinent to a 2 x 2 cross-over design  be
performed, as described above. In particular, we suggest that two sample T-tests and non-parametric
Kruskal-Wallis tests be computed for testing the hypothesis in (7-10). These tests are available in
PROC T-TEST and PROC NPARONEWAY in SAS (1989).  The purpose of this analysis is to
attempt to identify order related dependencies in the contrasts CAand CB

We also suggest that additional summary statistics be presented that would help the reader assess the
relative magnitudes of the emissions attributable to the cold start compared to emissions caused by
normal running of the engine. In particular, we suggest that means and standard errors for  statistics
of the form

M6.STE.001                               21                                 April 2001

-------
                                    XCcold.A^) - XCnostart.A^)
                                         X(nostart,A)                              (H)
be computed. Means, standard errors, and T-tests could also be computed for dfference scores of
the form

                    X(cold.A^ - XCnostartA^) _ XCcold.B^) - XCnostart.B^).             (12)
                           X(nostart,A)                X(nostart,B)

Finally, we suggest that the analysis of the current experiment include an additional level of the
within factor (i.e. a warm start level for the START factor).  New experiments of the same type
might also include additional grouping factors as described in the previous section.  If more
complicated designs of this sort are considered, the analysis could make use of the repeated measures
facilities of SAS PROC GLM.
M6.STE.001                               22                                April 2001

-------
                                     References

Enns P. and Brzezinski D. (1997) "Comparison of Start Emissions in the LA92 and StOl Cycles."
Report No. M6.STE. 001.  US EPA Assessment and Modeling Division Ann Arbor.

 Jones, B. and Kenward, D. (1989).  "Design and Analysis of Cross-Over Trials." Chapman and
Hall Ltd, New York.

SAS Institute (1989).  "SAS/STAT User's Guide, Version 6, Fourth Edition." Volumes 1 and 2.
Cary, NC.

SAS Institute (1990).  "SAS Procedures Guide, Version 6, Third Edition." Cary, NC.

Statistical Sciences (1995). "S-PLUS Guide to Statistical and Mathematical Analysis, Version
3.3." StatSci, a division of MathSoft Inc., Seattle.

Ratkowsky, D. A., Evans, M.A. Alldredge J.R. (1993). "Cross-OverExperiments: Design, Analysis,
and Application." MarcelDekker Inc., New York.
M6.STE.001                              23                                April 2001

-------
       EPA Response to the Center for Statistical Consultation and Research Review
                                            of
            Comparison of Start Emissions in the LA92 and ST101 Test Cycles
       The reviewers at the Center for Statistical Consultation and Research Review have
suggested a series of detailed statistical descriptions of the data to assist readers in assessment of
the validity of the report conclusions.  Some of these suggestions are addressed in Tables 6
through 10 below.  If additional statistical information about the data is required, the data used in
this report has been made available along with this report.
Table 6
Statistics for the Engine Start Emissions Increment* by Cycle
CYCLE
LA92
LA92
ST01
ST01
Statistic
Mean
Std. Deviation
Mean
Std. Deviation
CO (grams)
12.0660
4.2827
12.6600
3.0077
THC (grams)
1.3940
.1992
1.4360
.2936
NOx (grams)
.6080
.3267
.6000
.6997
* Cold start emissions measurement minus the no start emissions.
Table 7
Paired Samples Test of the Engine Start Increment
Pollutant*
CO
(grams)
THC
(grams)
NOx
(grams)
Paired Differences
Mean
-.5940
-4.2000E-02
8.000E-03
Std.
Deviation
2.1380
.1819
.7823
Std. Error
Mean
.9562
8.133E-02
.3499
95% Confidence
Interval of the
Difference
Lower
-3.2487
-.2678
-.9634
Upper
2.0607
.1838
.9794
t
-.621
-.516
.023
df
4
4
4
Sig.
'2-tailed)
.568
.633
.983
* Cold start emissions measurement minus the no start emissions.
M6.STE.001
24
April 2001

-------
Table 8
Engine Start Emissions
As a Proportion of Emissions Without An Engine Start
Cycle
LA92
ST01
Pollutant
(grams)*
CO
THC
NOx
CO
THC
NOx
Minimum
1.10
3.67
1.37
.78
3.69
.06
Maximum
22.31
55.50
13.50
15.56
59.00
4.89
Mean
6.3073
19.1686
4.3592
4.6771
20.7283
2.2103
Std. Error
4.0160
9.9029
2.3069
2.7450
10.2311
.8856
Std. Deviation
8.9800
22.1436
5.1584
6.1380
22.8775
1.9803
  Cold start emissions measurement minus the no start emissions divided by the no start emissions.
Table 9
Difference in Engine Start Emissions
As a Proportion of Emissions Without An Engine Start
(LA92 versus ST01)
Pollutant
CO (grams)
THC (grams)
NOx (grams)
Mean
1.6301
-1.5597
2.1488
Std. Deviation
2.8901
2.3395
3.9318
Std. Error Mean
1.2925
1.0463
1.7583
* LA92 proportion (cold start emissions measurement minus the no start emissions divided by the no start
emissions) minus the ST01 proportion.
Table 10
One Sample Test of Engine Start Emissions
As a Proportion of Emissions Without An Engine Start
(LA92 versus ST01)
Pollutant*
CO (grams)
THC (grams)
NOx (grams)
Test Value = 0
t
1.261
-1.491
1.222
df
4
4
4
Sig.
(2-tailed)
.276
.210
.289
Mean
Difference
1.6301
-1.5597
2.1488
95% Confidence Interval of
the Difference
Lower
-1.9584
-4.4646
-2.7331
Upper
5.2187
1.3452
7.0308
* LA92 proportion (cold start emissions measurement minus the no start emissions divided by the no start
emissions) minus the ST01 proportion.
M6.STE.001
25
April 2001

-------