PM-PEMS Measurement Allowance
Determination
Final Report
&EPA
United States
Environmental Protection
Agency
-------
PM-PEMS Measurement Allowance
Determination
Final Report
Assessment and Standards Division
Office of Transportation and Air Quality
U.S. Environmental Protection Agency
and
California Air Resources Board
and
Engine Manufacturers Association
Prepared by
Southwest Research Institute
SwRI Project 03.14936.12
SEPA
United States
Environmental Protection
Agency
EPA-420-R-10-902
August 2010
-------
PM- PEMS MEASUREMENT ALLOWANCE
DETERMINATION
FINAL REPORT
rฎ
SwRI Project 03.14936.12
Prepared for
U.S. Environmental Protection Agency
California Air Resources Board
Engine Manufacturers Association
June 2010
Prepared by:
Imad A. Khalek, Program Manager
Approved by:
Jeff J. White, Director
DEPARTMENT OF EMISSIONS RESEARCH AND DEVELOPMENT
ENGINE, EMISSIONS AND VEHICLE RESEARCH DIVISION
This report shall not be reproduced, except in full, without the written approval of Southwest Research Institute .
Results and discussion given in this report relate only to the test items described in this report.
-------
FOREWORD
ThePM-PEMS measurement allowance program was performed by the Department of
Emissions R esearch a nd D evelopment unde r Mr. Jeff W hite, Director. Dr. Imad Khalek,
Program Manager, was the Principal Investigator and Project Manager, Mr. Thomas Bougher,
Research Engineer, was the Project Leader, and Mr. Daniel Preece, Research Assistant, was the
laboratory te chnical a ssistant. Dr. Robert Mason, I nstitute Analyst, was t he P rincipal Co-
Investigator responsible for statistical analysis, and Ms. Janet Buckingham, Staff Analyst, was
the Project Leader/Statistics. Other SwRI Emissions R&D staff with contribution to the project
were Mr. Michael F eist, Senior R esearch Engineer, Mr. Richard Mechler, Senior R esearch
Technologist, Mr. Donald Parker, Senior Technician, Mr. Jose Sosa, Principal Technician, Mr.
Keith Echtle, Laboratory A ssistant Manager, a nd M r. E rnest Krueger, L aboratory M anager.
Additional SwRI assistance during Environmental Testing was provided by Rick Pitman, Senior
Engineering T echnologist, Mr. Mike Negrete, S enior T echnician, Mr. David S mith, Staff
Technician, M r. Herbert W alker, S enior E ngineering T echnologist, and M r. E ric D ornes,
Principal Engineer.
This w ork w as pe rformed unde r E nvironmental P rotection A gency ( EPA) Work
Assignments 2-7, 3 -7, 4 -7, a nd 2 -12, und er SwRIP reject Numbers 12859.07, 13749.07,
14658.07, and 14936.12, respectively. The original EPA Work Assignment Manager was Mr.
Matthew Spears. A fter March of 2009, t he E PA W ork A ssignment M anager b ecame M r.
Christopher Laroo. This work started in June of 2007 a nd will end by June 26, 2010. T esting
started in June of 2008 and ended in September of 2009.
Funding for this work was provided by U.SEPA, Engine M anufacturers A ssociation
(EMA), a nd the C alifornia A ir R esources B oard ( C ARB). Funding b y EMA a nd C ARB w as
provided directly to EPA in support of this work.
In-kind engine and technical support were provided by Volvo Powertrain. In-kind PM-
PEMS and technical support were provided by AVL, Horiba, and Sensors. SwRI acknowledges
the following individuals for their laboratory technical and logistical support:
Mr. Jeffrey Saxon and Mr. Steven Trevitz, from Volvo Powertrain
Mr. Scott Porter, Dr. Michael Akard and Dr. Qiang Wei, from Horiba
Mr. William Silvis, Mr. Siegfried Roeck, Dr. Wolfgang Schindler, Dr. Roland Wanker,
Dr. Michael Arndt, Mr. P.J. Pankratz, and Ms. Sarah Kingham, from AVL
Dr. Andrew Reading, Dr. David Booker, Dr. Atul Shah, Mr. Carl Ensfield,
Mr. Timothy Bottomley, and Mr. Kevin Bouma, from Sensors
REPORT 03.14936.12 11
-------
A m easurement allowance steering committee (SC) com posed of EMA, EPA, CARB,
andPM-PEMS m anufacturer m embers m et on a regular basis throughout the entire project to
discuss the progress made and make recommendations. The SC has contributed significantly to
this project. SwRI acknowledges the following SC members for their active participation:
Mr. Rey Agama, Caterpillar
Dr. Michael Akard, Horiba
Dr. Dipak Bishnu, CARB
Dr. David Booker, Sensors
Dr. Bruce Cantrell, EPA Consultant
Mr. Timothy French, EMA
Dr. Robert Giannelli, EPA
Mr. Denny Hao, PACCAR
Dr. Kent Johnson, CE-CERT
Mr. Craig Kazmierczak, Detroit Diesel
Mr. John Kegebein, John Deere
Mr. Thomas Kramer, Navistar
Mr. Christopher Laroo, EPA
Mr. Hector Maldonado, CARB
Mr. William Martin, Cummins
Dr. Shirish Shimpi, Cummins
Mr. William Silvis, AVL
Ms. Carol Smith, Isuzu
Mr. Matthew Spears, EPA
Mr. Steven Trevitz, Volvo Powertrain
Dr. Qiang Wei, Horiba
REPORT 03.14936.12 111
-------
TABLE OF CONTENTS
Page
FOREWORD ii
LIST OF FIGURES vii
LIST OF TABLES xvi
ACRONYMS xix
EXECUTIVE SUMMARY xxi
1.0 INTRODUCTION 1
2.0 MODELING APPROACH 3
2.1 Purpose of Model 3
2.2 Model Improvement 3
2.3 Monte Carlo Simulation Approach 4
2.4 Calculation Methods 4
2.4.1 Calculation Method 1 - "Exhaust Flow-Torque-Speed" Method 5
2.4.2 Calculation Method 2 - "Exhaust and Fuel Flow-Torque-Speed" Method. 6
2.4.3 Calculation Method 3 - "Fuel Flow-Torque-Speed" Method. 7
2.5 Reference NTE Events 8
2.6 Error Surface Generation 12
2.6.1 PEMSvs. Laboratory Nominal Results 12
2.6.2 (PEMS-Laboratory) Deltas vs. Lab 14
2.6.3 Variability Index vs. (PEMS-Laboratory) Deltas and Lab Nominal 15
2.7 Error Surface Sampling and Interpolation 18
2.8 Brake-Specific Emissions Calculations 20
2.9 Convergence and Number of Trials 22
2.10 Simulation Output 23
2.11 Step-by-Step Simulation Example 24
2.12 Measurement Allowance Generation 27
2.12.1 Regression Method 27
2.72.2 Median Method 27
2.13 Model Validation 28
3.0 PART 1065 PEMS AND LABORATORY AUDIT 32
3.1 1065 Lab Audit 32
3.1 1065 PEMS Audit 34
3.2.7 HoribaFlow Audits 34
3.2.2 Exhaust Flow 37
REPORT 03.14936.12 IV
-------
TABLE OF CONTENTS (CONT'D)
Page
4.0 ENGINE DYNAMOMETER TESTING AND RESULTS 40
4.1 Testing Objective 40
4.2 Experimental Setup 40
4.2.1 Engine and Sampling System 40
4.2.2 Sensors PPMD 43
4.2.3 AVLMSS 45
4.2.4 HoribaTRPM. 46
4.3 Bypass Mixing Verification 47
4.4 PEMS Loss Corrections 50
4.4.1 Sensors PPMD Loss Correction 50
4.4.2 AVLMSSLoss Correction 50
4.4.3 AVL MSS Total PM Correction 51
4.5 Steady-State Testing Procedure 51
4.6 Data Yield During Steady-State Testing 58
4.6.1 Data Yield During Steady-State Testing 59
4.7 Accounting for CVS Variability During Steady-State Testing 62
4.8 Steady-State Testing Results 62
4.8.1 Comparison Between PEMS and Lab Delta PM 63
4.8.2 Correlation Between PEMS and Lab PM 70
4.8.3 Steady-State PM Error Surfaces 73
4.9 Transient Engine Results 76
4.10 CE-CERT Mobile Lab Correlation 86
4.11 Investigation of DPF Regeneration 89
4.12 Investigation of Storage and Release 91
4.13 Engine Manufacturers Torque and Fuel Error Surfaces 93
5.0 ENVIRONMENTAL TESTING AND RESULTS 96
5.1 Reference Measurement Testing 96
5.2 Pressure Chamber Testing 102
5.3 Temperature and Humidity Chamber Testing 109
5.4 Electromagnetic and Radio Frequency Interference Screening 116
5.4.1 Bulk Current Injection 117
5.4.2 Radiated Immunity 122
5.4.3 Electrostatic Discharge 125
5.4.4 Conducted Transients 126
5.5 Vibration Testing 128
6.0 MODELNG RESULTS 137
6.1 Convergence Results from MC Runs 137
6.2 Sensitivity Based on Bias and Variance 142
6.3 Validation Results 150
6.4 Measurement Error Allowance Results 160
REPORT 03.14936.12
-------
TABLE OF CONTENTS (CONT'D)
Page
7.0 SUMMARY 173
8.0 REFERENCES 174
APPENDICES Page Count
A TEST PLAN TO DETERMINE PEMS MEASUREMENT ALLOWANCE FOR
THE PM EMISSIONS REGULATED UDNER THE MANUFACFTURER-RUN
HEAVY-DUTY DIESEL ENGINE IN-USE TESTING PROGRAM 46
B STEERING COMMITTEE MEETING MINUTES 33
C CRYSTAL BALL OUTPUT FILE DESCRIPTIONS 6
D MONTE CARLO SPREADSHEET COMPUTATIONS 15
E PEMS OPERATION LOG 11
REPORT 03.14936.12 VI
-------
LIST OF FIGURES
Figure Page
1 Method 1 Ideal BSPM Values for Reference NTE Events 9
2 Error Surface Construction: PEMS vs. Laboratory Results 14
3 Error Surface Construction: (PEMS - Lab) vs. Laboratory Results 15
4 Error Surface Construction: Error at Variability Index for 5th and 95th percentiles
vs. Laboratory Results 16
5 Truncated Normal Distribution Percentiles 17
6 Error Surface Construction: Error at Variability Index for 1st and 99th Percentiles
vs. Laboratory Results 18
7 Truncated Standard Normal at 1st and 99th Percentiles and Uniform Probability
Density Functions 19
8 Steady-State PM Error Surface for AVL With Example Sampling for a Reference
NTE Event 20
9 Overview of Monte Carlo Simulation for BSPM 24
10 Error Surfaces Included in Monte Carlo Simulation 26
11 Linear Regression Fit To 5th and 95th Percentile Deltas 30
12 Loess Regression Fit To 5th and 95th Percentile Deltas 31
13 Linearity Check on PEMS-1 Exhaust Flow During Steady-State Engine Testing 38
14 Linearity Check on PEMS-2 Exhaust Flow During Steady-State Engine Testing 38
15 Linearity Check on PEMS-3 Exhaust Flow During Steady-State Engine Testing 39
16 Volvo MP7 Installed in a CVS Test Cell 40
17 DPF Bypass With DOC 41
18 Schematic of Engine Dynamometer Experimental Setup 42
19 PPMD Installed in the Test Cell 44
REPORT 03.14936.12 Vll
-------
LIST OF FIGURES (CONT'D)
Figure Page
20 TheAVLMSS 45
21 TheProbeandTPAfortheHoribaOBS-TRPM 46
22 TheHoribaOBS-TRPMandOBS-2200 47
23 Experimental Setup for Mixing Verification 48
24 Mixing Verification Sample Locations 48
25 Mixing Verification Results 49
26 Speed and Load for the 80 Points Cycle 52
27 Final Six Steady-State Modes 53
28 Example of Steady-State Cycle 55
29 Steady-State Sample Times 56
30 CVS Filter Weight Gain for Steady-State Testing 56
31 Steady-State Exhaust PM Concentration (jig/Mol) 57
32 Steady-State Brake-Specific PM, CVS Filter (Mg/Hp-Hr) 58
33 Number of Valid Data Points for Steady-State Testing 59
34 Horiba Sample Flow to the Filter and Dilution Flow While Compressor Stops 61
35 Sensors EFM Zero During Steady-State Cycle 61
36 Horiba-1 Steady-State PM Concentration Deltas 64
37 Sensors-1 Steady-State PM Concentration Deltas 64
3 8 AVL PM Concentration Deltas for Steady-State Testing on PEMS 1 65
39 Horiba-2 Steady-State PM Concentration Deltas 65
40 Sensors-2 Steady-State PM Concentration Deltas 66
REPORT 03.14936.12 Vlll
-------
LIST OF FIGURES (CONT'D)
Figure Page
41 Avl-2 Steady-State PM Concentration Deltas 66
42 Horiba-3 Steady-State PM Concentration Deltas 67
43 Sensors-3 Steady-State PM Concentration Deltas 67
44 AVL-3 Steady-State PM Concentration Deltas 68
45 Steady-State Concentration Deltas for Horiba 69
46 Steady-State Concentration Deltas for Sensors 69
47 Steady-State Concentration Deltas for Avl 70
48 Steady-State Horiba PEMS PM Concentration Versus the Laboratory Reference 71
49 Steady-State Sensors PEMS PM Concentration Versus the Laboratory Reference 71
50 Steady-State AVL PEMS PM Concentration Versus the Laboratory Reference 72
51 Linear Regression Correlation Between PEMS and Lab 72
52 Steady-State Concentration Deltas for Horiba 73
53 Steady-State Concentration Deltas for Sensors 74
54 Steady-State Concentration Deltas for AVL 74
55 Final Steady-State PM Error Surface - Horiba 75
56 Final Steady-State PM Error Surface - Sensors 75
57 Final Steady-State PM Error Surface - AVL 76
58 Repeat Engine Speed Traces for NTE Transient Cycle 77
59 Repeat Engine Torque Traces for NTE Transient Cycle 78
60 Repeat AVL Soot Concentration Traces for NTE Transient Cycle 79
61 AVL Soot Concentration During NTE Transient Cycles, Events 20-23 79
REPORT 03.14936.12 IX
-------
LIST OF FIGURES (CONT'D)
Figure Page
62 Comparison of AVL and HoribaReal Time Signals During Transient Cycle 80
63 CVS BSPM and Correction Factor for Transient Cycle 81
64 Horiba Concentration Deltas for Transient Engine Testing 82
65 Sensors Concentration Deltas for Transient Engine Testing 82
66 AVL Concentration Deltas for Transient Engine Testing 83
67 Final Horiba Transient PM Error Surface 84
68 Final Sensors Transient PM Error Surface 85
69 Final AVL Transient PM Error Surface 85
70 Exhaust Configuration for CE-CERT Correlation 86
71 Brake-Specific PM Results From CE-CERT Correlation 88
72 CVS Filter Weight Gain During Tunnel Blanks 88
73 Brake Specific PM Emissions During Active Regeneration 90
74 Total Exhaust Number Concentration During Storage and Release 92
75 Brake-Specific PM Emissions During Storage and Release Investigation 92
76 OEM Supplied Torque Errors 94
77 OEM Supplied Fuel Flow Errors 94
78 Target Dilution Ratio and PM Level Profile for Environmental Testing 97
79 Experimental Setup for Environmental Testing 98
80 Horiba Environmental Baseline Measurements 99
81 Sensors Environmental Baseline Measurements 99
82 AVL Environmental Baseline Measurements 100
REPORT 03.14936.12
-------
LIST OF FIGURES (CONT'D)
Figure Page
83 ReferenceMSS Environmental Baseline Measurements 100
84 Reference AVL Versus PEMS AVL for Environmental Baseline 101
85 Original and Revised Profile for Altitude Testing 102
86 Altitude Testing Chamber 103
87 PEMS Installed in the Altitude Chamber 104
88 Horiba Environmental Pressure Measurements 104
89 Horiba Total Flow and Dilution Ratio During Pressure Testing 105
90 Sensors Environmental Pressure Measurements 106
91 AVL Environmental Pressure Measurements 106
92 AVL Pressure Median Versus MAD 107
93 Horiba Pressure Median Versus MAD 107
94 Sensors Pressure Median Versus MAD 108
95 Final Error Surface for Environmental Pressure AVL PM Concentration 109
96 Temperature and Humidity Profile for Environmental Testing 110
97 PEMS Installed in the Temperature and Humidity Chamber 110
98 The PM Generator Installed Outside the Temperature and Humidity Chamber Ill
99 Horiba Environmental Temperature Measurements 112
100 Horiba Temperature during Environmental Temperature Testing 112
101 Sensors Environmental Temperature Measurements 113
102 AVL Environmental Temperature Measurements 113
103 Horiba Temperature Median Versus MAD 114
REPORT 03.14936.12 XI
-------
LIST OF FIGURES (CONT'D)
Figure Page
104 Sensors Temperature Median Versus MAD 115
105 Horiba Temperature and Humidity Median Versus MAD 115
106 Final Error Surface Environmental Temperature and Humidity AVL PM 116
107 AVL PEMS in the Radiation Chamber for EMI and RFI Testing 117
108 Bulk Current Injection Probe 118
109 Horiba PEMS Setup for Bulk Current Injection 118
110 Sensors Setup During Bulk Current Injection Testing 119
111 AVL Setup During Bulk Current Injection Testing 119
112 Horiba Flow Disturbance From Bulk Current Injection 120
113 Horiba Exhaust Flow Noise on Analog Cable During Bulk Current Injection 121
114 BCI Noise on AVL Analog Output Cable 122
115 Horiba PEMS Setup During Radiated Immunity Testing 123
116 Sensors PEMS Setup During Radiated Immunity Testing 124
117 AVL PEMS Setup During Radiated Immunity Testing 124
118 Horiba Dilution Ratio Fluctuations During Radiated Immunity 125
119 Electrostatic Discharge Simulator 126
120 Sensors Setup During Conducted Transients Testing 127
121 AVL Setup During Conducted Transients Testing 127
122 Sensors PEMS Mounted for Transverse Horizontal Vibration 128
123 Sensors PEMS Mounted for Longitudinal Horizontal Vibration 129
124 Sensors PEMS Mounted for Transverse 45ฐ Vibration 129
REPORT 03.14936.12 Xll
-------
LIST OF FIGURES (CONT'D)
Figure Page
125 Sensors PEMS Mounted for Longitudinal 45ฐ Vibration 130
126 Sensors PEMS Mounted for Vertical Vibration 130
127 Sensors PEMS Mounted for 45ฐ Vertical Vibration 131
128 Power Spectral Density for Vertical Vibration Testing 132
129 Power Spectral Density for Horizontal Vibration Testing 132
130 Horiba PEMS Vibration Positions 133
131 Sensors Total Flow During Vibration Testing 134
132 AVL PEMS Vibration Positions 135
133 AVL Soot Measurement Noise During Vibration Testing 135
134 Convergence for AVL Method 1 As a Percent of BSPM Threshold 138
135 Convergence for AVL Method 2 As a Percent of BSPM Threshold 138
136 Convergence for AVL Method 3 As a Percent of BSPM Threshold 139
137 Convergence for Horiba Method 1 As a Percent of BSPM Threshold 139
138 Convergence for Horiba Method 2 As a Percent of BSPM Threshold 140
139 Convergence for Sensors Method 1 As a Percent of BSPM Threshold 141
140 Convergence for Sensors Method 2 As a Percent of BSPM Threshold 141
141 Box Plot of Error Surface Sensitivity Based on Bias and Variance for AVL
BSPM Method 1 146
142 Box Plot of Error Surface Sensitivity Based on Bias and Variance for
AVL Method 2 146
143 Box Plot of Error Surface Sensitivity Based on Bias and Variance for
AVL Method 3 147
REPORT 03.14936.12 Xlll
-------
LIST OF FIGURES (CONT'D)
Figure Page
144 Box Plot of Error Surface Sensitivity Based on Bias and Variance for Horiba
BSPM Method 1 147
145 Box Plot of Error Surface Sensitivity Based on Bias and Variance for Horiba
BSPM Method 2 148
146 Box Plot of Error Surface Sensitivity Based on Bias and Variance for Sensors
BSPM Method 1 148
147 Box Plot of Error Surface Sensitivity Based on Bias and Variance for Sensors
BSPM Method 2 149
148 Validation Percentiles for the 141 Reference NTE Events for AVL Method 1 151
149 Validation Percentiles for the 141 Reference NTE Events for AVL Method 2 152
150 Validation Percentiles for 141 Reference NTE Events for AVL Method 3 152
151 Validation Percentiles for 141 Reference NTE Events for Horiba Method 1 153
152 Validation Percentiles for 141 Reference NTE Events for Horiba Method 2 153
153 Validation Percentiles for 141 Reference NTE Events for Sensors Method 1 154
154 Validation Percentiles for 141 Reference NTE Events for Sensors Method 2 154
155 Validation 95th Percentile BSPM Deltas Loess Fit for Sensors Method 1 155
156 Validation 5th Percentile BSPM Deltas Loess Fit for Sensors Method 1 156
157 Validation 95th Percentile BSPM Deltas Loess Fit for Sensors Method 2 156
158 Validation 5th Percentile BSPM Deltas Loess Fit for Sensors Method 2 157
159 Validati on 95th Percentile BSPM Deltas Loess Fit for AVL Method 1 157
160 Validati on 5th Percentile BSPM Deltas Loess Fit for AVL Method 1 158
161 Validati on 95th Percentile BSPM Deltas Loess Fit for AVL Method 2 158
162 Validati on 5th Percentile BSPM Deltas Loess Fit for AVL Method 2 159
REPORT 03.14936.12 XIV
-------
LIST OF FIGURES
Figure Page
163 Validation 95th PercentileBSPM Deltas Loess Fit for AVL Method 3 159
164 Validation 5th PercentileBSPM Deltas Loess Fit for AVL Method 3 160
165 Regression Plot of 95th Percentile Delta BSPM Versus Ideal BSPM for
AVL Method 1 162
166 Regression Plot of 95th Percentile Delta BSPM Versus Ideal BSPM for AVL
Method 2 162
167 Regression Plot of 95th Percentile Delta BSPM Versus Ideal BSPM for AVL
Methods 163
168 Regression Plot of 95th Percentile Delta BSPM Versus Ideal BSPM for Horiba
Method 1 164
169 Regression Plot of 95th Percentile Delta BSPM Versus Ideal BSPM for Horiba
Method 2 165
170 Regression Plot of 95th Percentile Delta BSPM Versus Ideal BSPM for Sensors
Method 1 166
171 Regression Plot of 95th Percentile Delta BSPM Versus Ideal BSPM for Sensors
Method 2 167
172 Validation On-Road and Regression Functions Based on the Simulation Model for
BSPM Sensors Method 1 WithNoRegen 170
173 Validation On-Road and Regression Functions Based on the Simulation Model for
BSPM Sensors Method 2 WithNoRegen 170
174 Validation On-Road and Regression Functions Based on the Simulation Model for
B SPM AVL Method 1 WithNoRegen 171
175 Validation On-Road and Regression Functions Based on the Simulation Model for
BSPM AVL Method 2 WithNoRegen 171
176 Validation On-Road and Regression Functions Based on the Simulation Model for
BSPM AVL Method 3 WithNoRegen 172
REPORT 03.14936.12 XV
-------
LIST OF TABLES
Table Page
1 PM-PEMS Used along with Serial Number 2
2 Reference NTE Events and Ideal BSPM Emissions 10
3 Descriptive Statistics for BSPM Emissions for 141 Reference NTE Events 10
4 Input Parameters For Reference NTE Events 11
5 Error Surfaces for Monte Carlo Simulation 13
6 Error Surfaces Used for Computing Brake-Specific PM Emissions by Three
Calculation Methods 22
7 Example of Selection of Measurement Allowance at 0.02 g/hp-hr NTE Threshold for
theAVLPEMS 28
8 Linearity Verification Results for Intake Air Flwo and Fuel Flow 32
9 CVS Propane Recovery Check Summary 33
10 Linearity Verification for PM Balance 33
11 Summary ofPart 1065 Audits 34
12 Linearity Verifications for HoribaPEMS 35
13 Linearity Verifications for Sensors PEMS 36
14 Linearity Verific Ations for AVL PEMS 37
15 List of DPF Bypass Configurations 42
16 Sample Order for Steady-State Cycle Testing 54
17 Data Yield by Each PM-PEMS 60
18 Test Procedure for CE-CERT Correlation 87
REPORT 03.14936.12 Xvi
-------
LIST OF TABLES (CONT'D)
Table Page
19 PM Emissions Results from Active DPF Regeneration 89
20 Average Brake-Specific Emissions During Storage and Release Cycle 93
21 OEM Error Surface Deltas for Torque and Fuel Flow 95
22 Summary of Number of Reference NTEs Meeting 2% Convergence 141
23 Error Surface Sensitivity to Bias and Variance for 141 Reference NTE Events for
AVLBSPM Method 1 143
24 Error Surface Sensitivity to Bias and Variance for 141 Reference NTE Events for
AVLBSPM Method 2 143
25 Error Surface Sensitivity to Bias and Variance for 141 Reference NTE Events for
AVLBSPM Method 3 143
26 Error Surface Sensitivity to Bias and Variance for 141 Reference NTE Events for
HORIB A BSPM Method 1 144
27 Error Surface Sensitivity to Bias and Variance for 141 Reference NTE Events for
HORIB A BSPM Method 2 144
28 Error Surface Sensitivity to Bias and Variance for 141 Reference NTE Events for
SENSORS BSPM Method 1 145
29 Error Surface Sensitivity to Bias and Variance for 141 Reference NTE Events for
SENSORS BSPM Method 2 145
30 Summary of Error Surface Sensitivities to Bias and Variance for BSPM Method 1 149
31 Summary of Error Surface Sensitive to Bias and Variance for BSPM Method 2 150
32 Summary of Error Surface Sensitive to Bias and Variance for BSPM Method 3 150
33 Loess Smoothing Parameters for Validation Percentiles 155
REPORT 03.14936.12 Xvii
-------
LIST OF TABLES (CONT'D)
Table Page
34 Measurement Error at Threshold for BSPM Using Regression and Median Methods
for AVL Method 1 161
35 Measurement Error at Threshold for BSPM Using Regression and Median Methods
for AVL Method 2 162
36 Measurement Error at Threshold for BSPM Using Regression and Median Methods
for AVL Method 3 163
37 Measurement Error at Threshold for BSPM Using Regression and Median Methods for
HORIBA Method 1 164
38 Measurement Error at Threshold for BSPM Using Regression and Median Methods
for HORIB A Method 2 165
39 Measurement Error at Threshold for BSPM Using Regression and Median Methods for
SENSORS Method 1 166
40 Measurement Error at Threshold for BSPM Using Regression and Median Methods for
SENSORS Method 2 167
41 BSPM Measurement Error in Percent of NTE Threshold by PEMS and Calculation
Method 168
42 Measurement Allowance at NTE Threshold by Emissions for Method 2 169
43 Summary of BSPM Model Validation Results 172
REPORT 03.14936.12 XV111
-------
ACRONYMS
ACES Advanced Collaborative Emissions Study
BCI Bulk Current Injection
BS Brake-Specific
BSFC Brake-Specific Fuel Consumption
BSPM Brake-specific Paniculate Matter
CARB California Air Resources Board
CE-CERT Bourns College of Engineering Center for Environmental Research & Technology
CF Correction Factor
CFR Code of Federal Regulations
C O V C oeffi ci ent of Vari ati on
CVS Constant Volume Sampling
DCS Diffusion Charge Sensor
DOC Diesel Oxidation Catalyst
DPF Diesel Particulate Filter
DR Dilution Ratio
EAD Electrical Aerosol Detector (TSI, Inc.)
EATS Exhaust after-treatment system
ECM Engine Control Module
EE Electrical Enclosure
EEPS Engine Exhaust Particle Sizer (TSI, Inc.)
EFM Electronic Flow Meter
EGR Exhaust Gas Recirculation
EMA Engine Manufacturers Association
EMI Electromagnetic Interference
EPA Environmental Protection Agency
ESD Electrostatic Discharge
HDIUT Heavy-Duty In-Use Testing
HE Heated Enclosure
HEPA High Efficiency Particulate Air
LFE Laminar Flow Element
MA Measurement Allowance
MAD Median Absolute Deviation
ME Mechanical Enclosure
MEL Mobile Emissions Laboratory
MSS Micro Soot Sensor (AVL)
NMHC Non-Methane Hydrocarbons
NOX The Oxides of Nitrogen (NO + NO2)\
NTE Not-to-exceed
OBS On-board systems
OC/EC Organic Carbon/Elemental Carbon
PEMS Portable Emission Measurement System
PM Particulate Matter
PPMD Portable Particulate Measurement Device (Sensors, Inc.)
REPORT 03.14936.12
XIX
-------
ACRONYMS (CONTD)
PSD Power Spectral Density
QCM Quartz Crystal Microbalance
RFI Radio Frequency Interference
RMS Root Mean Square
SAE Society of Automotive Engineers
SS Steady State
SwRI Southwest Research Institute
TPA Tail Pipe Adaptor
TRPM Transient Response Particulate Matter
ULSD Ultra-Low Sulfur Diesel
VGT Variable Geometry Turbocharger
REPORT 03.14936.12 XX
-------
EXECUTIVE SUMMARY
The U nited S tates E nvironmental P rotection A gency ( EPA), E ngine M anufacturers
Association ( EMA) a nd C alifornia A ir R esources B oard ( CARB) agreed t o pur sue an
experimental da ta dr iven program to establish a me asurement a llowance ( MA) f or in -use
paniculate matter (PM) testing using PM portable emissions measurement systems (PM-PEMS).
The M A i s a br ake-specific P M e missions e rror a ssociated with using in -use P M-PEMS
equipment compared to the laboratory reference filter method. If the MA error is a positive value
above zero, after the EPA rounding method, as described in the Code of Federal Title 40, Part
1065 [1], it will increase the EPA in-use not-to-exceed (NTE) standard by the rounded value. If
the error is negative or zero, it will not contribute to any changes to the in-use standard.
The measurement allowance steering committee (SC) accepted the following PM-PEMS
to be part of the MA program:
Sensors Portable Particulate Measuring Device (PPMD). This is a PM-PEMS that uses
proportional dilution and a series of 8 quartz crystal microbalances to measure total (solid
plus volatile) PM. The total PM is measured as a single flow-weighted value for an NTE
event.
Horiba Transient Particulate Matter (TRPM). This is a PM-PEMS that uses proportional
dilution, a real time electrical aerosol detector, and an integrated filter sample to report
total PM. The total PM is measured as a single flow-weighted value for a NTE event.
This instrument can report real time total PM, but was not used as such on this program.
AVL micro-soot sensor (MSS). This is an instrument that uses constant dilution and a
photo acoustic detector to measure soot or the elemental carbon portion of PM. Soot is
measured in real time during an NTE event.
The SC agreed that only the PPMD by Sensors and TRPM by Horiba would be used for
the official determination and validation of the measurement allowance generated because both
are designated as complete PEMS and both measure total (solid plus volatile) PM, as required by
US EPA to be valid PM-PEMS. The SC also agreed that only the PM-PEMS that produces the
lowest positive 95th percentile measurement allowance, based on Sensors or Horiba only, would
be chosen for in-use validation by CE-CERT due to funding limitation. The third instrument, the
MSS b y A VL, was not a com plete P M-PEMS a nd w as a Iways us ed i n c onjunction w ith t he
PPMD on t his program, based on a n agreement reached between S ensors and AVL. The S C
agreed that the MSS would not be considered as an option for official measurement allowance
determination, unless both the PPMD and the TRPM failed validation.
The PM-PEMS-MA project included four main elements:
Laboratory steady-state (SS) and transient engine NTE testing using PM-PEMS and CVS
filter m easurement dur ing S S t esting onl y. T he S S testing was us ed t o c apture bi as,
compared to the CVS, and the transient was used to capture precision only since there is
no reference method to measure PM during a short NTE event. A 2007 heavy-duty diesel
MackMP?, from Volvo Powertrain, was used to conduct the engine experiments. The
REPORT 03.14936.12 Xxi
-------
engine was configured with a bypass around the di esel particle filter (DPF) to provide
PM concentration levels similar to those expected at the NTE threshold limits between
0.02 g/hp-hr and 0.03 g/hp-hr.
Environmental testing such as the effect of shock and vibration, pressure, temperature
and relative humidity, and electric noise on PM-PEMS precision.
Monte C arlo simulation t o de termine a br ake-specific m easurement al lowance va lue
using error surfaces generated from the PM-PEMS laboratory and environmental testing,
and from error surfaces generated during the gaseous PEMS program
o A total of 141 Reference NTEs were provided by EMA as an input to the model
for calculating ideal brake specific emissions, prior to the Monte Carlo simulation
Model validation using data generated from in-use PM-PEMS testing by CE-CERT.
Three methods are used to determine in-use NTE brake-specific PM emissions:
Method 1 f(PM, torque;, speed;, exhaust-flow;)
Method 2 f(PM, torque;, speed;, exhaust- flow;, G-flow;,
Method 3 f(PM;, torque;, speed;, and fuelECMi, G-flow;)
Where PM is a flow-weighted PM m easurement, i is i nstantaneous, E CM i s engine
control module, and G-flow is gas-based fuel flow. All methods require ECM broadcasted torque
and speed. In addition, Method 1 requires measured exhaust flow but not fuel flow; Method 2
requires measured exhaust flow, ECM broadcasted fuel flow, and G-flow; Method 3 is similar to
Method 2, but it does not require measured exhaust flow. Besides real time PM measurement, all
methods can use a single flow-weighted PM measurement for an NTE event, except Method 3,
where real time PM measurement is required. Thus, Method 1 and Method 2 were applied to all
three PM-PEMS, but Method 3 was only applied to the AVL MSS.
Compared to the Horiba TRPM, the Sensors PPMD, as shown in Table ES-1, produced
the low est pos itive 95t h pe rcentile measurement a llowance of 0.00605 g/hp-hr for an NTE
threshold 1 evel of 0.02 g/hp-hr, us ing c alculation M ethod 2. T he H oriba T RPM pr oduced a
measurement allowance value of 0.0100 g/hp-hr. Thus, the PPMD was selected by the SC for in-
use te sting b y CE-CERT for Monte C arlo m odel validation. AIthough not accepted as a PM-
PEMS, the AVL MSS produced the lowest measurement allowance of essentially zero. The SC
agreed to include the AVL MSS during in-use validation because it was used in conjunction with
the PPMD during the laboratory portion of the testing.
REPORT 03.14936.12 XX11
-------
TABLE ES-1. MEASRUEMENT ALLOWANCE BASED ON 0.02 G/HP-HR
THRESHOLD
Measurement Errors at NTE Threshold, g/hp-hr
PEMS
AVL
Horiba
Sensors
Method 1
0.0001
0.0109
0.0069
Method 2
-0.0005
0.0100
0.0061
Method 3
-0.0005
n/a
n/a
Figures ES-1 a nd E S-2 show e xamples of va lidation pi ots f or S ensors P PMD us ing
Method 1 and AVL MSS using Method 3. The dots on each of the two figures represent the in-
use delta BSPM between PEMS and CE-CERT 47 mm filter measurements on the y-axis versus
CE-CERT B SPM on the x-axis (Ideal B SPM determined by the filter measurement). Similarly,
the lines represents the 5th and 95th percentile errors produced by Monte-Carlo simulation based
on laboratory testing versus a reference NTE BSPM value on the x-axis (Ideal BSPM using 47
mm measurements). To pass model validation, < 10% of thein-useNTE delta brake-specific
PM (BSPM) between the PEMS and the CE-CERT (dots on Figures ESI and ES2) must reside
outside the 95th percentile and 5th percentile lines.
The Sensors PPMD failed validation using Method 1 and Method 2, a s shown in Table
ES-2. For Method 1, 32 percent of the data were below the 5th percentile, and for Method 2, 34
percent of the data were below the 5th percentile. The failure was mainly due to negative bias.
The e xact c ause for ne gative bi as i s unknow n, and funding 1 imitation di d not pe rmit further
investigations to resolve this issue under the scope of the MA program.
The A VL M SS pa ssed va lidation us ing M ethod 3, a s s hown i n T able ES-3, w here
9.89 percent of the data were outside the 5th and 95th percentiles, with the majority of the failed
points being above the 95th percentile. Method 2 failed validation by one percentage point, and
Method 1 failed validation by having 18 percent of the data outside the 5th and 95th percentiles,
with the majority of failed points being above the 95th percentile.
REPORT 03.14936.12
XX111
-------
Validation 95th and 5th Percentile BSPM Loess Fit for 141 Ret NTE Events
BSPM (g/hp-hr) Sensors Method 1 Units 1,2,3 n= 217 (20 pts removed)
0.013:
0.008:
0.003:
-0.002:
r :
i -0.007:
01 -0.012:
w -0.017:
ID .
| -0.022:
Q :
-0.027:
-0.032:
-0.037:
-0.042-1
0.005 0.010 0.015 0.020 0.025 0.030 0.035 0.040 0.045 0.050 0.055
1 95th Percentile
Ideal BSPM g/hp-hr
5th Percenlile CE-CERT Deltas
FIGURE ES-1. IN-USE VALIDATION PLOT FOR SENSORS PPMD USING METHOD
1. (Y AXIS IS THE DIFFERENCE BETWEEN PEMS AND IDEAL BSPM. IDEAL BSPM
IS THE LABORATORY CVS BSPM FOR THE 5TH AND 95TH AND THE CE-CERT
BSPM FOR THE DOTS)
Validation 35th and 5th Percentile BSPM Loess Fit for 141 Ref NTE Events
BSPM (g/hp-hr) AVL Method 3 Units 2,3,4 n= 271 No Regen
1
m
C3
D
0.010 0.015 0.020 0.025 0.030 0.035 0.040 0.045 0.050 0.055 0.060
Ideal BSPM g/hp-hr
CE-CERT Deltas
95th Percentile
5th Percentile
FIGURE ES-2. IN-USE VALIDATION PLOT FOR AVL MSS USING METHOD 3. (Y
AXIS IS THE DIFFERENCE BETWEEN PEMS AND IDEAL BSPM. IDEAL BSPM IS
THE LABORATORY CVS BSPM FOR THE 5TH AND 95TH AND THE CE-CERT
BSPM FOR THE DOTS)
REPORT 03.14936.12
XXIV
-------
TABLE ES-2. SENSORS PPMD VALIDATION RESULTS
Total No. CE-CERT Points
No. CE-CERT Points within Ideal BSPM Range
No. CE-CERT Points above 95th Percentile
No. CE-CERT Points below 5th Percentile
No. CE-CERT Points between 5th and 95th Percentiles
% CE-CERT points that did not validate
Method 1
217
210
0
68
142
32.38
Method 2
217
210
1
70
139
33.81
TABLE ES-3. AVL MSS VALIDATION RESULTS
Total No. CE-CERT Points
No. CE-CERT Points within Ideal BSPM Range
No. CE-CERT Points above 95th Percentile
No. CE-CERT Points below 5th Percentile
No. CE-CERT P ointsbe tween 5t h a nd 9 5th
Percentiles
% CE-CERT points that did not validate
Method 1
271
263
47
2
222
18.08
Method 2
271
263
28
2
233
11.41
Method 3
271
263
23
3
237
9.89
Due t o t he 1 ack of additional f unding, t he measurement allowance pr ogram w as
concluded by the SC without being able to solve the lack of validations issue with the Sensors
PPMD, or pe rform a dditional C E-CERT te sting w ith the H oriba T RPM to determine if i t
validates the model. The AVL MSS passed the validation criteria using Method 3. H owever,
the MSS as used in this program is not accepted as a PM-PEMS by EPA, and the measurement
allowance generated b ased on t he performance of this i nstrument i s not an official p art of the
measurement allowance.
The final official measurement allowance accepted by the SC was based on the Sensors
PPMD. Further investigation of why the PPMD did not validate the model is being investigated
outside the MA program, and the details are expected to be part of the CE-CERT Final Report on
PM-PEMS In-Use Validation Testing.
REPORT 03.14936.12
XXV
-------
1.0 INTRODUCTION
The U.S. EPA, EMA, and CARB agreed to pursue an experimental data driven program
to determine PEMS bias and precision measurement errors expected in in-useNTE testing and
compliance b efore en forcement. The i dea i s t o combine t hese e rrors i nto a m easurement
allowance which will be used with the EPA in-use regulatory standard. The combination, in-use
standard plus m easurement al lowance, will al low for a 1 arger threshold due to i nstrument and
measurement uncertainties that the engine manufacturers must comply with. The gaseous PEMS
measurement a llowance pr ogram w as c ompleted b y S wRI i n A pril of 2007, a nd t his w ork
focuses on the PM-PEMS measurement allowance, as set forth in Test Plan and meeting minutes,
shown in Appendices A and B, respectively.
To determine all bias and precision errors associated with in-use PM-PEMS measurement
would take a very extensive set of experimental data and engines to cover all engine steady-state
and transient NTE operations, as well as, different environmental conditions and configurations
that may influence the measurement. Instead of focusing on a wide matrix of experimental data,
the SC approach was to:
a) Perform repeats on a series of six laboratory NTE steady-state laboratory tests using the
CVS PM filter method and the PM-PEMS, and establish a bias error surface for each of
the PEMS tested.
b) Perform repeats on a series of laboratory N TE t ransient c ycles c ontaining thirty 32
seconds NTE events, and use the PM-PEMS m easurement t o produce p recision error
surface for each of the PEMS tested.
c) Perform a series of e nvironmental te sts tha t inc ludes e lectromagnetic a nd radio
frequency i nterferences, shock and vibration, pressure, and temperature and humidity,
and produce an error surface, if any, for each PEMS associated with each parameter.
d) Use torque and fuel flow error surfaces provided by the engine manufacturers.
e) Use ot her error s urfaces es tablished during t he gaseous m easurement allowance
program.
f) Use M onte-Carlo s imulation ba sed on a set of 141 Ideal brake-specific PM(BSPM)
reference NTE events to predict the error distribution at each reference NTE.
g) Use the 95th percentile and 5th percentile of the error di stribution at each Ideal B SPM
reference N TE 1 evel as the uppe r a nd 1 ow bounda ry o f t he de Itas be tween P M-PEMS
B SPM and Ideal B SPM..
h) Perform act ual i n-use testing w ith the PM-PEMS using t he C E-CERT tr ailer to
determine whether or not the data generated in-use validates the model.
A total of nine PM-PEMS, three from each manufacturer, were used on this program. The
PM-PEMS, shown in Table 1, included the Sensors PPMD, Horiba TRPM, and AVL MSS. Each
set of three PM-PEMS, one PM-PEMS from each manufacturer, was tested simultaneously i n
parallel us ing a s eries of s teady-state and transient N TE engine experiments. After the
completion of steady-state testing and transient testing, one PM-PEMS from each manufacturer
was used on a series of environmental test conditions that included electromagnetic and radio
frequency i nterferences, shock a nd vi bration, atmospheric pr essure, a nd t emperature a nd
humidity.
REPORT 03.14936.12 1 of 174
-------
TABLE 1. PM-PEMS USED ALONG WITH SERIAL NUMBER
PEMS Name
Horiba 1
Horiba 2
Horiba 3
Sensors 1
Sensors 2
Sensors 3
AVL1
AVL2
AVL3
PEMS Serial Number
10107-01
10107-02
10107-03
E08-PD03
G08-PD02
A08-PD03
346
472
273
To validate the model, a series of in-use tests with the PM-PEMS was performed by CE-
CERT using the CE-CERT emissions trailer. Details about thi s te sting will be provided in a
separate report by CE-CERT on PM-PEMS In-Use Validation Testing.
This report describes:
Model approach used to perform the modeling portion of this work.
Engine ex periments with the P M-PEMS including steady-state and transient N TE P M
results
Environmental setup and PM results
Measurement allowance produced by the model for each of the three PM-PEMS provided
by each manufacturer
Model validation using the PM-PEMS selected for in-use testing
REPORT 03.14936.12
2 of 174
-------
2.0 MODELING APPROACH
2.1 Purpose of Model
This program was designed to generate BSPMm easurement al lowancesb ased on
rigorous statistical methods applied to a large body of data. At the same time, it was desirable to
exclude out Her da ta ca used by ex treme m easurement er rors w hich were not cons idered
representative of normal in-use operations. A direct approach could have been to test PEMS
against s ome ki nd of mobile 1 aboratory r eference ( such a s t he C E-CERT M obile E mission
Laboratory) on a 1 arge number o f ve hides, a nd qua ntify e rrors di rectly. H owever, s uch a n
approach would have been expensive in terms of both time and funding.
Given these f actors, the S teering C ommittee ultimately e lected to us e a s imulation
approach in order to generate the BSPM measurement allowances, similar to what was done in
the g aseous i n-use e missions t esting pr ogram [2]. I n t his a pproach, t he S teering C ommittee
defined all of the expected sources of PEMS measurement errors based on existing in-use testing
expertise and understanding of how the PEMS functioned. Priority was given to the Horiba's
and Sensors' PEMS in the design of experiments. Each of these errors was quantified using a
series of controlled laboratory experiments, each designed to isolate errors related to a single
error source. The results of each experiment would essentially be an empirical model of a given
source of measurement error. In this report, these error models are referred to as error surfaces.
It is important to note that each of these error surfaces represents an incremental error of PEMS
measurement, as compared to an associated laboratory reference measurement.
2.2 Model Improvement
Several i mprovements t o t he e xecution of t he M onte C arlo s imulation m odel w ere
implemented to improve the efficiency and post-processing of the simulation runs. Eight macros
were written to perform various tasks and are summarized in the section below.
Macro 1: Controls batch processing and allows the ability to run several simulations back-
to-back (batch mode). Reads each reference NTE event data, number of trials, and the number
of reference NTE events in the batch run. Calls other macros.
Macro 2: Clears and deletes ex tra r ows i n error m odel ( see A ppendix C f or a d etailed
description). Calculates ideal PM emissions for each calculation method.
Macro 3: Checks e rror s urfaces t urned ' off i n s imulation r un a nd c lears unus ed e rror
surface cha its and Excel t abs for c alculation s peedup (see A ppendix D ) f or a de tailed
description).
Macro 4: Controls Crystal Ball run preferences, runs Crystal Ball simulation for the given
NTE Event, and controls Crystal Ball creation of Report and Extract data files. No longer stores
each trial ic value (40,000 - 65,000 values). Reduced the number of sensitivity charts created.
Only stores BS emissions in g/hp-hr units. T his reduced EXTRACT and REPORT files from
133 MB to 17 MB per NTE event (~ 87% reduction).
REPORT 03.14936.12 3 of 174
-------
Macro 5: Reads EXTRACT file, controls calculation of 5th, 50th and 95th percentiles from
full model and calls Macro 6.
Macro 6: Computes 5*, 50* and 95* percentile delta BS emissions and confidence limits
from order statistics from the Full model. These values are used in the measurement allowance
plots and to check for convergence.
Macro 7: Reads Extract file Validation data, computes 5 ,50* and 95* percentiles for the
Validation model (calls Macro 6).
Macro 8: Reads RE PORT file, selects a nd formats s ensitivity da ta f or A ssumption
sensitivities greater than 5 percent or less than -5 percent contribution to variance and stores in
sensitivity file. Creates sensitivity file with "important" error surfaces.
The development of the macros resulted in substantial improvements in the simulation
run-time and post-processing of the extract files. The estimated reduction in model simulation
run-time and post processing was approximately 80 to 85 percent of the time used in the gaseous
PEMS program. The developed macros eliminated the need to manually post-process the files as
had been done in the gaseous PEMS program. The batch processing allowed up to 20 reference
NTE events to be simulated in a single Excel run. The size of the EXTRACT and REPORT files
was greatly reduced by only storing needed variables and delta PM emissions.
2.3 Monte Carlo Simulation Approach
The e rror s urfaces r epresenting incremental er rors of P EMS m easurement w ere
programmed into a computer model which employed Monte Carlo random sampling methods to
simulate t he c ombined effects of a 11 of t hese s ources of e rror on t he final m easured br ake-
specific value. An ideal reference NTE data set (see Section 2.5) for a given test event was run
through the Model, and all the various errors were applied to that data set in a randomly chosen
manner. Brake-specific PM emission values were then calculated for both the ideal and error-
applied data sets, which were compared to yield a final measurement error (see Appendix C and
D). T he p rocess w as repeated t housands of t imes, w ith m any different i deal da ta s ets, t o
generate a 1 arge, robust da ta s et w hich was ev aluated to determine a final s et of com bined
measurement errors. These final errors, referred to in this report as deltas, were generated for the
PM pollutant for each calculation method and three PEMS model units from each of the three
manufacturers, for a final set of seven deltas; Methods 1-3 for the AVL PEMS unit, Methods 1-2
for the Horiba PEMS unit and Methods 1 -2 for the Sensors PEMS unit. A complete description
of the Monte Carlo methodology and of the model is given in Section 2.5 of this report.
2.4 Calculation Methods
Calculations m ust be pe rformed on t he r ecorded da ta t o de termine br ake-specific P M
emission values in accordance with methods outlined in 40 C FRPart 1065 Subparts G and J.
The symbolic notation given in the formulas shown later in this section is fully described in 40
CFR Part 1065 SubpartK[l].
REPORT 03.14936.12 4 of 174
-------
40 C FR P art 1065 a Hows f or t he us e of a ny of t hree di fferent c alculation m ethods i n
order to determine brake-specific emission values from in-use test data. The basic calculation of
brake-specific emissions requires three main inputs as follows:
Mass Concentration x Flowrate
BS Emissions = =
Work Power
The three calculation methods vary somewhat in the means used to determine either the
Flow c omponent or t he W ork c omponent of t his c alculation. E ach of t he t hree m ethods i s
summarized be low. B ecause e ach m ethod r elies on di fferent i nputs, i t i s pos sible that e ach
method of c alculation w ill r eact di fferently t o va rious m easurement errors. T herefore,
measurement allowances m ust be e xamined i ndependently f or e ach m ethod. H owever,
according to the T est P Ian, see Appendix A and B , methodology, o nly on e of t he t hree
calculation methods would be s elected to generate t he f inal m easurement allowances. T he
selection methodology is outlined later in this introduction under the Measurement Allowance
Generation section.
2.4.1 Calculation Method 1 - "Exhaust Flow- Torque-Speed" Method
Calculation M ethod 1 i s a nalogous t o t he m ethod us ed b y most d ynamometer
laboratories, and relies on direct input of both exhaust flow and torque. In the case of exhaust
flow, this is the flow rate measured by the same form of exhaust flow meter. The Sensors and
AVL PEMS relied on the Sensors exhaust flow meter (EFM) while the Horiba PEMS had a tail
pipe adapter (TPA) it employed for exhaust flow measurement. Work is not measured directly,
but is instead calculated using ECM broadcast engine speed and ECM broadcast engine torque.
While eng ine s peed is directly m easured by t he en gine E CM, ECM b roadcast t orque i s an
estimate based on a variety of other parameters; torque cannot be directly verified during in-use
testing. A simplified formula for this method is:
Ymass
Method I = =
2, work
REPORT 03.14936.12 5 of 174
-------
The more complete formula used for Method 1 is as follows:
For all PM PEMS:
m PM is a flow weighted particulate matter exhaust concentration in g/mol
eru(glkW-hr)=
m
PM\
mol
IV
Z
'Speedi(rpm)*Ti(N-m)*2*3.l4l59*bt'
60*1000*3600
Where for AVL:
m PM is computed numerically as follows,
m
PM
=
mPM
g
mol)
-\*h.
mol
*Af
mol
1 V
I
It should be noted that calculation Method 1 is directly dependent on the accuracy of both
the e xhaust f low m eter a nd t he t orque e stimation, a s w ell a s on t he m easurement of PM
concentration.
2.4.2 Calculation Method 2 "Exhaust and Fuel Flow-Torque-Speed" Method
This calculation i s designated solely for in-use testing, and i s designed to minimize the
effect of errors r elated t o the a ccuracy o ft he ex haust f low m easurement. The M ethod 2
calculation adjusts the exhaust flow measurement by a ratio of the CO2-based fuel flow to the
ECM reported fuel flow. This means that although the flow meter must be linear, it does not
necessarily have to be accurate. In addition, Method 2 depends on t he ECM broadcast torque
and s peed, a nd on t he r atio of fuel flow c alculated from t he c arbon ba lance us ing ga seous
measurement over the fuel flow broadcast by the ECM. A simplified version of this method can
be expressed as:
Method 2 =
mass
C02 fuel
ECM fuel
x Work
REPORT 03.14936.12
6 of 174
-------
The more complete formula for Method 2 using PM as an example is:
For all PM PEMS:
m is a flow weighted particulate matter exhaust concentration in g/mol
PM
m
'PM
ePM(glkW-hr} = -
moi) tr
w
ซ. f1 * [xTHCt (ppm)*l Q-6 + (xCOt (%) + xC6>2 (%)) * 10~2 ] * A/
Speed, (rpm) * T( (N m) * 2 * 3.14159
60*1000*3600
Where for AVL:
m PM is computed numerically as follows,
m
PM
IV
I
w.
It should be noted that, as mentioned earlier, Method 2 i s not subject to accuracy errors
for the exhaust flow measurement, although that measurement must still be linear for the method
to function properly.
2.4.3 Calculation Method 3 - "Fuel Flow- Torque-Speed" Method
Method 3 doe s not us e di rect m easurement of e xhaust flow, but r elies on a carbon
balance and ECM broadcast fuel rate to determine m ass. The work term for Method 3 i s
determined identically t o t he w ork t erm f or M ethod 1; us ing t he E CM br oadcast va lues f or
engine speed and torque to calculate work. Method 3 entirely circumvents the use of an exhaust
flow meter, but for the HDIUT program, EPA must approve the use of Method 1 for a given test
and manufacturer. A simplified version of Method 1 may be expressed as:
Method 3 =
X
ECM fuel
C02fuel
REPORT 03.14936.12
7 of 174
-------
The more complete formula for Method 3 is:
For AVL Only:
hr} =
m
PM
S
mol
z
* 1 0~
* 1 0~
Speedi(rpm)*Ti(N-m)*2*3.l4l59*Al
60*1000*3600
Where:
mo
/)) *!ปฃ
xTHCt (ppm) * 10"6 + (^CO, (%) + xCO2i (%)) * 10"2
* 10"6 + ^CO % + xCO2i % * 10"2
It should be noted that Method 3 i s not subject to exhaust f low m easurement accuracy
errors, but also that this method is wholly dependent on ECM broadcast values for both mass and
work determination.
2.5 ReferenceNTE Events
The reference data set to which all the simulated errors were applied represented engine
operations over a wide range of laboratory NTE events. Parameters in the reference data set
were scaled in order to exercise the model through a more appropriate range of parameters (i.e.
concentrations, flows, ambient c onditions, e tc.). In t his s caling pr ocess, c are w as t aken t o
maintain the dynamic characteristics of the reference data set.
The Monte C arlo simulation m odel was run on a set of 141 reference NTE events that
were used during the gaseous MA program [2]. O nly the events that have different speed and
torque c ombinations w ere us ed; five e ngine m anufacturers pr ovided a t otal of 97 e vents; 10
reference NTE events came from each of the three engines tested in the lab during the gaseous
MA transient testing; and 14 e vents came from the pre-pilot CE-CERT data. Because no PM
data exist for these reference events, the PM concentration used to calculate the Reference NTE
PM emissions was developed by SwRI based on the Volvo engine used in this study. A simple
model w as de veloped t o pr edict t he P M c oncentration ba sed on s peed a nd t orque us ing 80
different s teady-state P M cone entrations t hat w ere m easured with the M SS us ing t he de sired
REPORT 03.14936.12
8 of 174
-------
exhaust b ypass c onfiguration. A dditional a djustment t o t he m odel us ing t he r ate of c hange i n
torque was added to better predict short NTE transient events. The model was further adjusted to
produce a brake-specific emissions distribution for the reference NTE events centered around
0.02 g/hp-hr, as shown in Figure 1.
Revised 141 Original 141
35%
30%
25%
20%
15%
10%
5%,
0%
BSPM (g/hp-hr)
FIGURE 1. METHOD 1 IDEAL BSPM VALUES FOR REFERENCE NTE EVENTS
NTE br ake-specific em issions r esults w ere calculated for PM us ing e ach of t he t hree
agreed-upon NTE calculation methods. T he three different BS emissions calculation methods
referred to in this test plan are:
1. Method No. 1: Exhaust Flow-Torque-Speed Method
2. Method No. 2: Exhaust and Fuel Flow-Torque-Speed Method
3. Method No. 3: Fuel Flow-Torque-Speed Method
Table 2 lists the number of NTE events obtained from each data source and the B SPM
emissions calculated using Method 1. These emissions have been computed with no error values
added to the input parameters. For this report, emissions with no errors added will be labeled the
"ideal" emissions. In contrast, t he e missions w ith e rrors added t hrough t he M onte C arlo
simulation will be labeled emissions "with errors."
REPORT 03.14936.12
9 of 174
-------
TABLE 2. REFERENCE NTE EVENTS AND IDEAL BSPM EMISSIONS
Source
International
DDC
Caterpillar
Cummins
Volvo
Engine No. 1
Engine No. 2
Engine No. 3
Number of
NTE Events
19
18
20
20
20
10
10
10
BSPM
Min
0.01146
0.01303
0.01097
0.01561
0.01541
0.01215
0.01675
0.01099
max
0.03069
0.03463
0.01789
0.03253
0.02701
0.04066
0.04872
0.03669
When the ideal brake-specific emission values were calculated for the various reference
NTE events, i t w as not ed that these i deal e mission values w ere frequently di fferent from one
calculation method to another. While it was recognized that this was a realistic outcome, the
Steering C ommittee felt tha t the se di screpancies mig ht int reduce a n unintended bias int o the
results of the M onte C arlo simulation. T herefore, the S teering C ommittee di rected SwRI to
adjust the NTE reference event data in order to align the brake-specific emission levels from all
the calculation methods.
The M ethod 1 r esult w as not c hanged, t herefore t orque, s peed, a nd e xhaust f low
remained unchanged as well. The CO2 concentration was adjusted to make the Method 2 result
equal to the one from Method 1. This was done by using a single multiplier on all CC>2 values for
the NTE event in question. Lastly the fuel rate values and alignments were adjusted to bring
Method 3 in line with Method 1 and 2.
The distribution of the ideal BSPM emissions data for the 141 reference NTE events was
presented to the Steering Committee. It was noted that very few reference NTE events were at or
below the B SPM threshold (0.02 g/hp-hr). T hus, the reference NTE events were adjusted to
produce more events with ideal BSPM values below 0.02 g/hp-hr. The original and revised ideal
BSPM di stributions are depicted i n Figure 1. Note that the B SPM emission data has values
spread above and below the corresponding NTE threshold.
Table 3 provides a summary of some descriptive statistics for the reference NTE data set
for BSPM emissions.
TABLE 3. DESCRIPTIVE STATISTICS FOR BSPM EMISSIONS FOR 141
REFERENCE NTE EVENTS
Descriptive Statistic
Minimum
Maximum
Mean
Median
Standard Deviation
BSPM g/hp-hr
0.010974
0.048717
0.020753
0.019101
0.006932
REPORT 03.14936.12
10 of 174
-------
The parameter data provided in each reference NTE event was on a second-by-second
basis with a minimum of 30 s econds and a maximum of 300 s econds. The input parameters
required for the BSPM emissions calculation methods and the Monte Carlo simulation are listed
in Table 4. An Excel file with a specific input format structure was used to standardize the
format of the input files. Since the total hydrocarbons (THC) was selected as an input parameter,
NMHC was computed as THC*0.98.
TABLE 4. INPUT PARAMETERS FOR REFERENCE NTE EVENTS
Variable
Number
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
Input Variable
NTE Event Number
NTE Source
Engine Make
Engine Model
Engine Displacement
Date
Time Stamp
Wet CO2
Wet CO
Wet kNO
Wet kNO2
Wet THC
Exhaust Flow Rate
Flowmeter Diameter
Speed
Low Speed, nlo
High Speed, nhi
Fuel Rate
Max Fuel Rate
Derived Torque
Peak Torque
Flow-weighted Average
PM Concentration for
Methods 1 & 2
Flow-weighted Average
PM Concentration for
Method 3
Units
integer
alphanumeric
alphanumeric
alphanumeric
L
mm/dd/yyyy
hh:mm:ss.s
%
%
ppm
ppm
ppm
scfrn
3, 4, or 5 (inches)
rpm
rpm
rpm
L/sec
L/sec
N-m
N-m
|jg/mol
|jg/mol
Description
All reference NTE events must be identified by an NTE number (e.g.,
001).
The source of the NTE event is the company, organization and/or lab
that created the event data.
Engine Make
Engine Model
Engine Displacement (L)
The day the NTE event data was created (mm/dd/yyyy).
Time in seconds. Each reference NTE must contain second-by-
second data only.
CO2 (%)
CO (%)
NO (ppm) with intake air-humidity correction
NO2 (ppm) with intake air-humidity correction
THC (ppm)
Exhaust flow rate (scfm)
To compute the % of PEMS flowmeter maximum flowrate we will
need to know what size flowmeter was used for each NTE event.
Enter either 3, 4, or 5 to represent the following flowmeters and
maximum flow rates:
3=3 inch EFM with maximum flow rate = 600 scfrn
4=4 inch EFM with maximum flow rate = 1100 scfrn
5=5 inch EFM with maximum flow rate = 1700 scfrn
Engine speed (rpm)
To compute the % of normalized speed we will need nlo and nhi for
the engine computed as follows:
nlo (rpm) = lowest speed below max power at which 50% max power
occurs
nhi (rpm) = highest speed above max power at which 70% max
power occurs
Fuel rate (L/sec)
To compute the % of maximum fuel rate we will need the max fuel
rate of the engine for each NTE event.
Max fuel rate (L/sec)
Torque (N-m)
To compute the % of maximum torque we will need the peak torque
of the enqine for each NTE event
Peak torque (N-m)
Flow-weighted average PM concentration, flow-weighted by the
exhaust flow. Values were calculated based on a predictive model
developed using transient and steady-state experimental data.
Flow-weighted average PM concentration, flow-weighted by the
exhaust flow. Values were calculated based on a predictive model
developed using transient and steady-state experimental data.
REPORT 03.14936.12
11 of 174
-------
2.6 Error Surface Generation
During the discussions he Id at s everal S teering Committee m eetings, 33 error s urfaces
were identified and considered for inclusion in the Monte Carlo simulation model. 25 of these
error surfaces were the same surfaces used in the gaseous emissions in-use testing program. [2]
Of the remaining ei ght error surfaces, two were di scarded (Delta PM EMI/RFI and Delta PM
Vibration), because the PM generator was not used during these tests (see Section 5) T he six
new er ror s urfaces f or t he P M pr ogram w ere P M S teady-State ( SS), PM T ransient, PM
Atmospheric Pressure, PM Ambient Temperature, Torque Engine Manufacturers and Fuel Rate
Engine Manufacturers. This resulted in a final total of 31 error surfaces that were incorporated
into the Model. These individual error surfaces encompassed a wide variety of error sources. In
addition, all error surfaces distributions used in this program included a range of sampled data
between the 1st and 99 * percentile t o expand the r ange of s ample d ata i n the M onte C arlo
simulation. T his w as done a 11 he request o f t he S C s ince s ampling for t he gaseous p rogram
covered only the range between the 5th and 95th.
Table 5 lists the error surfaces examined during the study with the surfaces excluded by
the S teering C ommittee de signaled in italics. A11 r emaining on es w ere impl emented in the
simulation model. Each error surface was assigned a number for easy identification.
For each of the measurement errors defined in Table 5, an error surface was created and
used i n t he M onte C arlo s imulation. E ach e rror s urface represented an a dditive e rroror a
subtractive error if the sign was negativerelative to the reference parameter value to which it
was applied. Figures 2 through 4 show an example of how these error surfaces were created for
every measurement error. Details on t he c onstruction of e ach e rror s urface us ed i n t he
simulation are provided in Section 0. The example illustrated in Figure 2 through 4 represents
the error surface for steady-state bias and precision PM concentration errors for an individual
PEMS unit
2.6.1 PEMS vs. Laboratory Nominal Results
Figure 2 was constructed from raw data acquired from steady-state engine lab tests with
the PEMS conducting repeat testing at various concentration levels (PM jig/mol). The plot pools
all bias and precision errors for the PEMS tested for all steady-state modes. A nominal target of
10 repeat m easurements of PM was taken on each PEMS unit for each value of the
corresponding average lab PM values (i.e., lab nominal value). The 10 PEMS measurements
were pi otted against the c orresponding measurements using 1 aboratory equipment. S hown i n
Figure 2 are t he 5 th, 50th, a nd 95th percentiles c orresponding t o t he di stribution of t hese 180
observations (30 observations at each of the six concentration levels) using the PEMS at each
average P M concentration 1 evel (note t hat t he distribution of da ta a t e ach P M level ma y not
represent a normal distribution). S ince the 50* percentiles do not lie on the dashed (diagonal)
line of perfect agreement, the data suggest that there is a bi as error between the PEMS and lab
results. In essence this graph summarizes the statistical distribution measured by the PEMS at
each concentration level sampled. The example plot in Figure 2 shows only 6 discrete average
PM concentration levels (ranging from 10-60 jig/mol). However, the actual number of discrete
concentration levels was determined using the total number of operating conditions actually run
for all the tests on the engine. In the section on Steady-State Repeat Engine Testing and Error
Surfaces it is reported that 6 operating modes conditions from an initial number of 80 operating
REPORT 03.14936.12 12 of 174
-------
conditions w ere s elected for con struct! on of t he steady-state P M error s urface. From t hese 6
operating modes several discrete PM concentration levels were defined which were used in the
error surface plots for the Monte Carlo simulation.
TABLE 5. ERROR SURFACES FOR MONTE CARLO SIMULATION
Measurement Error Surfaces and Deltas Used in BSPM Calculations
Component
1 . Delta PM
2. Delta CO
3. Delta NMHC
NMHC = 0.98*THC
4. Delta Exhaust Flow
5. Delta Torque
6. Delta Fuel Rate
7. Delta Speed
8. Delta Fuel Rate
9. Delta CO2
#
1
2
3
4
5
6
7
10
11
13
14
16
17
19
20
21
22
23
25
27
28
29
30
31
32
34
35
42
43
44
45
46
49
Test Source
Engine Dyno
Engine Dyno
Environ
Environ
Environ
Environ
Engine Dyno
Environ
Environ
Engine Dyno
Engine Dyno
Environ
Environ
Environ
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Environ
Environ
Environ
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Engine Manuf
Engine Manuf
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Environ
Error Surface
Delta PM SS
Delta PM Transient
Delta PM EMI/RFI
Delta PM Atmospheric Pressure
Delta PM Ambient Temperature
Delta PM Vibration
Delta CO SS
Delta CO Atmospheric Pressure
Delta CO Ambient Temperature
Delta NMHC SS
Delta NMHC Transient
Delta NMHC Atmospheric Pressure
Delta NMHC Ambient Temperature
Delta Ambient NMHC
Delta Exhaust Flow SS
Delta Exhaust Flow Transient
Delta Exhaust Flow Pulsation
Delta Exhaust Flow Swirl
Delta Exhaust EMI/RFI
Delta Exhaust Temperature
Delta Exhaust Pressure
Delta Dynamic Torque
Delta Torque DOE Testing
(Interacting Parameters Test)
Delta Torque Warm-up
(Interacting Parameters Test)
Delta Torque Humidity/Fuel
(Independent Parameters Test)
Delta Torque Interpolation
Delta Torque Engine Manuf
Delta Fuel Engine Manuf
Delta Dynamic Speed
Delta Dynamic Fuel Rate
Delta CO2 SS
Delta CO2 Transient
Delta CO2 Ambient Temperature
Description
AVL, Horiba and Sensors
AVL, Horiba and Sensors
Deleted by Steering Committee
AVL
AVL
Deleted by Steering Committee
Same as Gaseous Study but moved 5th% to 1st%
and 95th% to 99th%
Same as Gaseous Study
Same as Gaseous Study
Same as Gaseous Study but moved 5th% to 1st%
and 95th% to 99th%
Same as Gaseous Study but moved 5th% to 1st%
and 95th% to 99th%
Same as Gaseous Study
Same as Gaseous Study
Same as Gaseous Study but moved 5th% to 1st%
and 95th% to 99th%
Same as Gaseous Study but moved 5th% to 1st%
and 95th% to 99th%
Same as Gaseous Study but moved 5th% to 1st%
and 95th% to 99th%
Same as Gaseous Study but moved 5th% to 1st%
and 95th% to 99th%
Same as Gaseous Study but moved 5th% to 1st%
and 95th% to 99th%
Same as Gaseous Study but moved 5th% to 1st%
and 95th% to 99th%
Same as Gaseous Study
Same as Gaseous Study
Same as Gaseous Study but moved 5th% to 1st%
and 95th% to 99th%
Same as Gaseous Study but moved 5th% to 1st%
and 95th% to 99th%
Same as Gaseous Study but moved 5th% to 1st%
and 95th% to 99th%
Same as Gaseous Study but moved 5th% to 1st%
and 95th% to 99th%
Same as Gaseous Study but moved 5th% to 1st%
and 95th% to 99th%
New
New
Same as Gaseous Study but moved 5th% to 1st%
and 95th% to 99th%
Same as Gaseous Study but moved 5th% to 1st%
and 95th% to 99th%
Same as Gaseous Study but moved 5th% to 1st%
and 95th% to 99th%
Same as Gaseous Study but moved 5th% to 1st%
and 95th% to 99th%
Same as Gaseous Study
REPORT 03.14936.12
13 of 174
-------
Error Surface forSS PM Concentration
80
60
| 40
10
-se
^45
38
33
10 20 30 40 50
PM |jg/mole (lab, mean)
60
70
80
diagonal
50th percentile (median)
95th percentile
-5th percentile
FIGURE 2. ERROR SURFACE CONSTRUCTION: PEMS VS. LABORATORY
RESULTS
2.6.2 (PEMS - Laboratory) Deltas vs. Lab
Figure 3 illustrates the "error band" measured during testing. This plot was created by
first subtracting the individual "lab nominal" PM value from the corresponding individual PEMS
PM measurement for each test run. The sampling system used to obtain the "lab nominal" or
"lab reference" PM values is described in Section 4.2.1. The difference between the PEMS PM
and the lab reference was defined as the "delta" error. Second, these "PEMS - Laboratory" delta
errors w ere pool ed at each average 1 ab nom inal P M value t o obt ain t he 5 th, 50th, a nd 95 th
percentile v alues di splayed i n Figure 3. Therefore, the pi ot r epresents t he ave rage P M1 ab
nominal at 6 discrete concentration levels versus the percentiles of the d elta errors computed
from the PEMS and laboratory individual test results.
REPORT 03.14936.12
14 of 174
-------
Error Surface forSS PM Concentration
a.
55"
O
O)
-2
A
10
30
50
60
70
-1
-1
-2
PM |jg/mole (lab,mean)
-95th percentile
50th percentile (median)
-5th percentile
FIGURE 3. ERROR SURFACE CONSTRUCTION: (PEMS - LAB) VS. LABORATORY
RESULTS
2.6.3 Variability Index vs. (PEMS - Laboratory) Deltas and Lab Nominal
This step normalized the plot in Figure 3 using what is called a "variability index (ic)".
This i ndex r epresented t he va lue r andomly dr awn b y t he M onte C arlo s imulation i n or der t o
select a given error level. For the 5* and 95* percentile of the truncated normal it was allowed
to vary from -1 to +1, respectively. The likelihood of "ic" being any value between -1 through
+1 was specifiedby a "probability density function (PDF)" assigned to ic. In the case of this
example, ic. was assumed to vary according to a standard normal (i.e, bell-shaped) distribution
during the Monte C arlo simulations. T his was b ecause it was believed that the di stribution of
PM errors due to steady-state bias and precision would be centered about the 50* percentile of
the full range of conditions measured. Each set of data for each lab "setpoint" average (i.e., lab
nominal value) in Figure 3 was normalized by aligning the corresponding 5th percentile error
from Figure 3 with ic = -1, the 50* percentile error with ic = 0, and the 95* percentile error with
ic = +1. These values were then plotted in Figure 4, where the y-axis is the variability index, the
x-axis i s the average 1 ab nom inal PM value, and the z-axis is the delta PM value. Notice that,
when us ing t his nor malization a pproach, t he 5 th, 50 th, a nd 95 th percentile va lues r emain
REPORT 03.14936.12
15 of 174
-------
equivalent between Figure 3 and Figure 4. This development of the error surfaces from the lab
data was the procedure used in the gaseous emissions in-use testing program.
1 c
0.5 -
5
Q.
W' 0 -
V)
0 <
-0.5 -
1 .
1 > -
A-
Error Surface forSS PM Concentration
Error Surface: z-axis = ASS_PM_|jg/mole
868569
A Jr v x x m
346234
10 20 30 40 50 60 70
-2-13-11 1
PM ug/mole (lab, mean)
95th percentile 50th percentile (median) 5th percentile
FIGURE 4. ERROR SURFACE CONSTRUCTION: ERROR AT VARIABILITY INDEX
FOR 5 AND 95 PERCENTILES VS. LABORATORY RESULTS
For the PM measurement allowance program it was decided by the Steering Committee
to expand the tails of the truncated normal distribution in the error surface formation in order to
allow larger PM deltas to be sampled during the Monte Carlo simulation. Instead of truncating
-Hi
the lower tail at the 5m percentile, it was moved to the 1st percentile. Likewise on the upper tail,
the 95 * percentile w as m oved t o t he 99 * percentile. S ince t he or iginal t runcated normal
distribution was defined by a mean = 0 and a standard deviation = 0.60795, the resulting indices
corresponding to the 1st and 99th percentiles are -1.4143 and +1.1413, respectively, as illustrated
in Figure 5.
REPORT 03.14936.12
16 of 174
-------
Monte Carlo Truncated Normal Sampling Distribution
-1.4143
N(0,0.60795)
-3-2-10123
Percentiles-* 1st 5th 95th 99th
FIGURE 5. TRUNCATED NORMAL DISTRIBUTION PERCENTILES
Finally, the 5* and 95* percentile (PEMS - LAB) delta values at each lab nominal value
were us ed t o c ompute t he c orresponding va lues of t he t runcated nor mal f or t he 1st and 99th
percentiles. To redefine the error surface delta at the 1st percentile, the standard deviation for the
normal distribution below the 50* percentile is defined as follows:
Standard Deviation lst =
Delta Valu6rnth Delta Value<-tn
'50
1.6449
Using the mean = 50* percentile delta and the standard deviation computed above, the 1st
percentile can be found using the Excel NORMINV function: NORMINV(0.01,mean,standard
deviation ist). Similarly, to redefine the error surface delta at the 99th percentile, the standard
th
deviation for the normal distribution above the 50 percentile is defined as follows:
Standard Deviation 99th =
Delta ValueQrth Delta
95C
1.6449
Using the mean = 50th percentile delta and the standard deviation computed above, the
99th percentile c an be f ound us ing t he E xcel N ORMINV f unction:
NORMINV(0.99,mean,standard deviationggth). Taking the data for the error surface in Figure 4
and redefining it to include the 1st and 99* percentile truncated normal values results in the error
surface displayed in Figure 6. All the error surfaces carried over from the gaseous program that
were sampled using a truncated normal were redefined at the 1st and 99th percentiles. Those error
surfaces that were sampled using the uniform distribution remained unchanged. Error surfaces
such as the one presented in Figure 6 are the error deltas the Monte Carlo simulation program
used during calculation of the BSPM emissions "with errors".
REPORT 03.14936.12
17 of 174
-------
2 _
1.5 -
1 -
0.5 -
S.
Q_
W o -
w
o' (
-0.5 -
-1 -
-1.5 -
2 .
"^
Error Surf ace forSS PM Concentration
ErrorSurface: z-axis = ASS_PM_|jg/mole
10.1 6.8 8.8 6.2 7.2 11.1
A -A- A. A. A, .A
346234
10 20 30 40 50 60 7
-4.1 -3.1 1.8 -2.2 0.2 -0.2
PM |jg/mole (lab, mean)
99th percentile 50th percentile (median) Istpercentile
D
FIGURE 6. ERROR SURFACE CONSTRUCTION: ERROR AT VARIABILITY INDEX
FOR 1ST AND 99TH PERCENTILES VS. LABORATORY RESULTS
2.7 Error Surface Sampling and Interpolation
The error model used two different PDF to sample the error surfaces, depending upon
which e xperimental pa rameter the s urface r epresented. T o sample er ror s urfaces t hat w ere
generated from the lab test results (Section on Engine Dynamometer Laboratory Testing), and
the applicable environmental t est r esults, t he m odel us ed a t runcated s tandard nor mal PD F
because these tests were designed to evenly cover the full, but finite, range of engine operation
and ambient conditions. T o sample error surfaces that were generated from the pressure and
temperature environmental test results (Section on Environmental Chamber Testing), the model
used a uniform PDF because these tests were already d esigned to cover the typical range and
frequency of t he respective c onditions. Both of t hese s ampling di stributions a re depicted i n
Figure 7.
REPORT 03.14936.12
18 of 174
-------
ฃ
Probability Density Functions for Sampling Error Surfaces Once Per NTE Event
Lab Tests, Normal, SD=0.60795, truncate @ -1.4143 & 1.4143
Environmental Tests, Uniform
Note: A non-truncated normal
distribution with SD=0.60795 has P
values of 0.01 an
ic=+1,4143 respe
d0.99atic=-1.414
itively.
^
7
\.
"\
Sand ^^^
^^
_
"\.
^^
x"*^^
^^
^-*\
4321
Relative Probability
1 41
0 00 i
c
0
FIGURE 7. TRUNCATED STANDARD NORMAL AT 1ST AND 99 PERCENTILES
AND UNIFORM PROBABILITY DENSITY FUNCTIONS
When using the truncated standard normal PDF (see Figure 7), the Monte Carlo model
sampled normal deviates that ranged between -1.4143 and +1.4143. These were used as the ic
values de fined i n t he s ection on Error Surfaces. Similarly, t he pr essure a nd t emperature
environmental tests used a uniform PDF to sample test time, from which calculated errors were
used. All temperature error surfaces related to the four emissions were sampled uniformly from
1 to 1080 minutes while the error surfaces related to the pressure were sampled uniformly from 1
to 720 m inutes. E xhaust flow error surface for temperature was sampled uniformly from 1 t o
478 minutes while the exhaust flow for pressure was sampled uniformly from 1 to 360 minutes.
The errors from all the other tests were aligned with the truncated standard normal PDF such that
each of the 50* percentile error values at each of the tested signal magnitudes was centered at the
median (i.e., 0 value) of the PDF, and the 1st and 99* percentile error values at each of the tested
signal magnitudes were aligned with the extreme negative (ic = -1.4143) and positive (ic =
+1.4143) edges of the PDF, respectively.
Each error surface was sampled along itsic axis (y-axis) once per trial fora reference
NTE event simulation. Hence, every error surface had a separate randomly selected ic for each
trial. Since each reference NTE event contained second-by-second parameter data, except for
PM for the Sensors PPMD and Horiba TRPM PEMS, the error surface was sampled at a given ic
on the y-axis and at the several selected parameter values on the x-axis that corresponded to each
second of the reference NTE event. The sampled error value was determined for the given
second and parameter al ong t he error axis (z-axis) at the intersection of the ic value and the
parameter value from the reference NTE event. This was accomplished by taking each second in
the r eference N TE event and finding t he t wo adjacent x -axis va lues from t he er ror s urface
REPORT 03.14936.12 19 of 174
-------
between which t o 1 inearly i nterpolate t o obt ain the e rror s urface x -value. Each s econd i n the
reference NTE event was linearly interpolated with the same ic value for a particular trial at the
error surface x-value. If any of the sampled lab nominal values (PM, NMHC, CO, Speed, Fuel
Rate, etc.) exceeded theupperor lower limits of the parameter error surface, the value of the
closest endpoint of the error surface was assigned to them.
Figure 8 depicts an example of the error surface sampling using a steady-state PM error
surface c ontaining 10 1 ab nom inal P M x-axis v alues. F or thi s pa rticular tr ial, the r andomly
selected ic is -0.5. The example reference NTE event is noted by the symbol '*' and it plotted at
ic = -0.5 for each second in the NTE event.
SS Error Surface for AVL PM Concentration
Error Surface: z-axis = ASS_PM_AVL_ugmol
1.5 -
1 -
0.5 -
CO
CO
-0.5 -
-1.5 -
-2
50
100
150
200
250
300
350
400
450
*-*--
PM ug/mol (lab,nom)
- 99th percentile
ic = -0.5
50th percentile (median) 1st percentile
NTE Event
FIGURE 8. STEADY-STATE PM ERROR SURFACE FOR AVL WITH EXAMPLE
SAMPLING FOR A REFERENCE NTE EVENT
2.8 Brake-Specific Emissions Calculations
Errors from Sections 4, 5, and 6 were combined by adding all of the sampled errors once
per trial for each reference NTE event simulation. For example, in order to assess the errors in
PM concentration by calculation Method #1, several error surfaces were sampled and added to
the corresponding parameter in the Method #1 calculation and the resulting BSPM "with errors"
REPORT 03.14936.12
20 of 174
-------
was computed. The errors used in this calculation for the Horiba and Sensors are the following
(note that the corresponding error surface numbers are provided in the subscripts):
PM ng/mol_reference + A PM ng/mol_l +
PM ug/moi 'with errors'
A PM
Exhaust Flow % 'with errors' = Exhaust Flow % reference +
A Exhaust Flow %2o A Exhaust Flow_%2i +
A Exhaust Flow % 22 + A Exhaust Flow % 23 -
A Exhaust Flow %25 + A Exhaust Flow %2i ~
A Exhaust Flow o/o28
Torque % 'with errors'
Speed % 'with errors'
Torque o/oreference +
A Torque % 29 + A Torque % 30 +
A Torque o/o31 + A Torque %32 +
A Torque o/o34 + A Torque o/o35
Speed % reference + A Speed % 43
where,
A 1,2 = PM concentration errors due to steady-state and transient errors,
A 20,21 = exhaust flow errors due to steady-state and transient errors,
A 22,23 = exhaust flow errors due to pulsation and swirl,
A 25 = exhaust flow errors due to ambient temperature,
A 27,28 = exhaust flow errors due to temperature and pressure,
A 29 = torque errors due to dynamic torque,
A 30,31 = torque errors due to DOE and warm-up,
A 32 = torque errors due to interacting parameters humidity and fuel,
A 34,35 = torque errors due to interpolation and engine manufacturers,
A 43 = speed errors due to dynamic speed
Using the formulas for the calculation methods, the BSPM for Method #1 was computed
without errors ("ideal") and then with all the errors applied as outlined above. Table 6 lists all
error surfaces used by each calculation method for the PM emissions.
REPORT 03.14936.12
21 of 174
-------
TABLE 6. ERROR SURFACES USED FOR COMPUTING BRAKE-SPECIFIC PM
EMISSIONS BY THREE CALCULATION METHODS
Component
1 . Delta PM
2. Delta CO
3. Delta NMHC
NMHC = 0.98THC
4. Delta Exhaust Flow
5. Delta Torque
6. Delta Fuel Rate
7. Delta Speed
8. Delta Fuel Rate
9. Delta CO2
#
1
2
4
5
7
10
11
13
14
16
17
19
20
21
22
23
25
27
28
29
30
31
32
34
35
42
43
44
45
46
49
Error Surface
Delta PM SS
Delta PM Transient
Delta PM Atmospheric Pressure
Delta PM Ambient Temperature
Delta CO SS
Delta CO Atmospheric Pressure
Delta CO Ambient Temperature
Delta NMHC SS
Delta NMHC Transient
Delta NMHC Atmospheric Pressure
Delta NMHC Ambient Temperature
Delta Ambient NMHC
Delta Exhaust Flow SS
Delta Exhaust Flow Transient
Delta Exhaust Flow Pulsation
Delta Exhaust Flow Swirl
Delta Exhaust EMI/RFI
Delta Exhaust Temperature
Delta Exhaust Pressure
Delta Dynamic Torque
Delta Torque DOE Testing
(Interacting Parameters Test)
Delta Torque Warm-up
(Interacting Parameters Test)
Delta Torque Humidity/Fuel
(Independent Parameters Test)
Delta Torque Interpolation
Delta Torque Engine Manuf
Delta Fuel Engine Manuf
Delta Dynamic Speed
Delta Dynamic Fuel Rate
Delta CO, SS
Delta CO? Transient
Delta CO? Ambient Temperature
Method 1
^
S
S
S
s
s
s
s
s
s
s
s
s
/
s
s
s
s
Method 2
^
^
S
/
/
s
s
s
s
s
s
s
s
s
s
s
s
V
V
s
s
s
Method 3
S
-------
A nonparametric statistical technique [3] was proposed which defined a 90% confidence
interval for the 95th percentile of the BSPM em issions di fferences for an individual reference
NTE simulation. I f the width of the 90% confidence interval was less than 1% of the B SPM
emissions threshold, then c onvergence w as met. The following steps define the convergence
method:
1. Run the Monte Carlo simulation for N trials.
2. Order the BS emissions differences from smallest to largest.
3. Identify the trial number at the lower end of the 90% confidence interval
niower = 0.95 * N -1.645V0.95 * 0.05 * N
4. Identify the trial number at the upper end of the 90% confidence interval
riupper = 0.95 * N + 1.645V0.95 * 0.05 * N
5. Compute (BSPM difference value at nupper) - (BSPM difference value at niower)-
6. If the result in (5) < 1% of the BSPM emissions NTE threshold then convergence is met.
7. The BSPM threshold was defined as 0.02 g/hp-hr. Thus, 1% of the threshold was 0.0002
g/hp-hr.
The Steering Committee a greed to the proposed c onvergence c riteria outlined a bove.
During the initial simulation runs for 20 reference NTE events, convergence was not met at the 1
percent criteria level until 60,000 trials were run. T his only applied to the AVL PEMS unit at
each of the three calculation methods. The Horiba and Sensors units only reached convergence
at t he 1 percent cr iteria for appr oximately h alf of t he 20 reference N TE eve nts s imulations.
Upon e xamination of t he di stributions of t he de Ita BSPM e missions ge nerated f rom t he
simulations, some of the distributions were positively skewed which would make convergence
very difficult at the 1 percent level. This information was presented to the Steering Committee
wherein a decision was made to relax the convergence level to 2 percent or higher, depending on
the outcomes of the simulations.
In summary, the 141 r eference NTE events were run at 40,000 t rials and convergence
was checked. If the width of the confidence interval on the 95 percentile delta BSPM emission
was approaching 2 p ercent of the threshold, then the simulation was continued for up to 65,000
trials.
2.10 Simulation Output
During t he s imulati on of a reference NTE e vent, differences b etween theB SPM
emissions "with errors" and the ideal BSPM emissions were obtained by each of the three PEMS
model uni ts a nd e ach of t he t hree applicable c alculation m ethods. T hese differences were
computed thousands of times (once per trial) until the model converged. Then the 95 percentile
difference value was determined for each reference NTE event's distributions of BS differences
for the PM emissions for all three PEMS units and applicable calculation methods.
The output from the Crystal Ball simulation for each reference NTE event was saved in
two separate Excel files: an EXTRACT and a REPORT file. The EXTRACT file contained
descriptive s tatistics on all di fferences c omputed for B SPM e missions b y all thr ee c alculation
methods, pe rcentiles ( 0%, 5% , 10% ,.. .95%, 100% ) of t he di fferences i n B SPM e missions,
REPORT 03.14936.12 23 of 174
-------
sensitivity data for all error surfaces, and differences in BSPM emissions computed at each trial
in the simulation.
The REPORT file contained a summary of the differences in the BSPM emissions for all
three PEMS units and applicable calculation methods including descriptive statistics, the number
of trials, a frequency histogram of the differences in BSPM emissions, and percentiles (0%, 5%,
10%,... 95%, 100% ) of the di fferences i n B SPM e missions. A Iso i ncluded w ere de scriptive
statistics on each ic distribution sampled for each error surface. Lastly, sensitivity charts for the
differences in BSPM emissions for the three PEMS unit and applicable calculation methods were
stored. T hese c harts pr ovided i nformation on how m uch e ach e rror s urface i nfluenced t he
differences computed between the BSPM em issions "w ith errors" a nd the i deal BSPM
emissions. A m ore de tailed de scription of t he C rystal Ball out put f iles c an be f ound i n
Appendix C.
2.11 Step-by-Step Simulation Example
In o rder toe larify t he s imulation pr ocess, the f ollowing s tep-by-step summary is
provided. This example assumes that a single reference NTE event was simulated for the BSPM
difference computations. Figure 9 provides an overview of the simulation process.
Reference NTE
Monte-Carlo Simulation
Output
PM
(ULE mole)
CO %
NMHC
(ppm)
ExMlow
(scon)
Torque
(N-m)
Speed
(rpm)
Fuel
Fjte
(L sec)
CO-1
Fuel Rate -
AFuel Rate
CO, -AGO,
(1) BSPM = f (PM, Exhflow, Torque, Speed)
(2) BSPM = f (PM, Exhflow, BSFCECซ)
(3) BSPM = f (PM, CO-2, CO, THC, Torque,
Fuel Rate =c-M, Speed)
* Differences = BSPM "with errors" - "Ideal" BSPM
FIGURE 9. OVERVIEW OF MONTE CARLO SIMULATION FOR BSPM
REPORT 03.14936.12
24 of 174
-------
Step 1 - Enter the reference NTE input parameters into the Monte Carlo (MC) simulation
model. T hese include the emissions concentrations, exhaust flow, torque, speed and fuel rate
data used in all three calculation methods.
Step 2 - Compute the "ideal" B SPM b y al 1 three P EMS m odel uni t and a pplicable
calculation methods from the reference NTE event.
Step 3 - Set-up the M onte C arlo simulation parameters in Crystal B all. An Excel
spreadsheet m odel was de veloped for us e w ith Oracle Crystal Ball M C s oftware for e rror
analysis of brake-specific emissions. Crystal B all is graphically-oriented f orecasting a nd
simulation software that runs on M icrosoftฎ Windows and Excel. The simulations run in this
program us ed C rystal Ball V ersion 11.1.1 a nd were run on P Cs c onfigured w ith a P entium 4
CPU, 3.39 G Hz, 3.50 G B R AM, 232 GB ha rd dr ive a nd W indows X P ope rating s ystem.
Microsoftฎ Excel 2003 SP was the spreadsheet software.
The options exercised in running Crystal Ball included the following:
Number of trials = 40,000
o If convergence was not met at the 2% criteria then # trials = 65,000
Monte Carlo sampling method with random initial seeds
Normal speed run mode
Suppress chart windows (fastest run time)
The Excel spreadsheet i s i n a m odular s tructure following t he s pecified m odel out line, a nd i t
makes pr ovisions f or t he t hree i dentified c alculation m odules. Input c ells t o the m odel a re
clearly id entified to facilitate any revisions that may become ne cessary for us ers who want to
exercise the model with other Monte Carlo software such as @Risk or newer versions of Crystal
Ball. The spreadsheet was tested with controlled test cases of simplified input distributions with
the Crystal Ball add-on to confirm correct model impl ementation in accordance with this test
plan. At least one typical analysis was run as an additional confirmation, and two independent
checks were made on the ideal emissions by other SwRI staff. A complete description of the
spreadsheet computations is contained in Appendix D.
Step 4 - Execute a single MC trial by randomly generating a separate i c for each error
surface used in the three calculations.
Step 5 - For each second in the reference NTE event, interpolate the A error for all error
surfaces at the input parameter values and the randomly generated ic. Figure 10 illustrates all the
error s urfaces a vailable a nd w here t he corresponding A errors are a dded. T he num bers i n
parentheses represent the error surface number in the Monte Carlo simulation.
REPORT 03.14936.12 25 of 174
-------
PM
CO
NMHC
Exhflow
* AFMss ppm (i)
APMTR ppm
(2)
F=ป( APMDress
L-M APMtemn ~
r H ACOSS ppm
(4)
(5)
(7)
H ACOnress mm
L- H ACOtemp_ppm
H ANMHCSS ,,
ANMHCTR j,
(10)
(11)
(13)
(14)
i frl ANMHCp1P,, pp
(16)
>| ANMHC,Pm (17)
_^.
' H ANMHCamhiPrt (19)
(20)
(21)
H AExhflow ,,wim, (22)
H AExhflowewit1
(23)
H AExhflowPMTmFT (25)
H AExhflowtpnip
(27)
p (28)
I Engine
J dyno
(Environ
chambers
} Engine
dyno
} Environ
chambers
1 Engine
J dyno
Environ
chambers
, Engine
dyno
Environ
' chambers
Torque
->
+
-ป
->
ATorqueDynamic (29)
ATorqueDOE (30)
ATorquewannup (31)
ATorquehumidity (32)
ATorqueinterpolalion(34)
ATorqueengine.malmf(35
J
V
Engine dyno
ASpeedDymmlc (43)
AFuelRateDynamic (44)
AFuel Rate
(42)
AC02S
(45
ACO2
(46)
Engine
dyno
AC02te
(49)
FIGURE 10. ERROR SURFACES INCLUDED IN MONTE CARLO SIMULATION
Step 6 - Compute one BSPM "with errors" for the given MC trial by adding all the A
error values to the reference NTE data and then calculating the BSPM by all three PEMS units
and applicable calculation methods.
Step 7 - Compute BSPM difference for the current trial:
BSPM emission "with errors" - "Ideal" BSPM emission
Step 8 - Repeat Steps 4-7 until the number of trials is met.
Step 9 - Check the di fferences i n BSPM for all t hree P EMS uni ts a nd applicable
calculation methods to be certain that the convergence criteria are met. If convergence is met for
all three calculation methods, continue to Step 10. Otherwise, return to Step 4 and run the Monte
Carlo simulation for an additional 25,000 trials until the total number of trials is 65,000.
Step 10 - Select the 95* percentile from the distribution of BSPM differences for each of
the three PEMS units and applicable calculation methods. S tore the ideal B SPM and the 95th
percentile BSPM differences for computing the measurement allowance.
Step 11 - Repeat Steps 1-10 for each reference NTE event.
REPORT 03.14936.12
26 of 174
-------
2.12 Measurement Allowance Generation
The generation of a set of measurement allowances represented the final outcome of this
program. T he Test Plan provided a methodology by which all of the data from the millions of
Model simulation runs would be collected and analyzed statistically, in order to generate a set of
potential measurement allowances for each PEMS model unit, one for each of the calculation
methods. T he Test Plan then outlined a specific method by which the final set of allowances
would be chosen from among deltas generated for each of the three calculation methods. T he
assumption made by the Test Plan, was that the final outcome of all previous efforts would be a
set of validated pot ential measurement al lowance va lues for e ach PEMS uni t. E ach potential
allowance w as ex pressed as a pe rcentage of i ts as sociated BSPM N TE t hreshold. T hese
measurement allowances w ere computed b y a r egression m ethod or am edian m ethod a s
described below.
2.12.1 Regression Method
This method involved determining the correlation between the 95* percentile differences
versus the ideal emission values for the reference NTE dataset. For each combination of PEMS
units and calculation method, a least squares linear regression of the 95th percentile differences
versus the ideal emissions results was computed. If the R value from the regression model was
greater than 0.85 and the SEE (standard error of the estimate or root-mean-squared-error) was
less than 5 percent of the median ideal BS emissions, then the linear regression equation was
used to determine the measurement allowance for that PEMS unit and calculation method. To
determine the measurement allowance, the NTE threshold was used to predict the measurement
allowance from t he r egression model. T he m easurement al lowance was t hen expressed as a
percentage of the NTE BSPM threshold value (0.02 g/hp-hr).
2.12.2 Median Method
If the linear regression di d not pa ss t he a forementioned criteria for theR2 and SEE
statistics, then the median value of the 95th percentile differences from the 141 reference NTE
events was us ed as t he s ingle m easurement allowance f or a c ombination of e missions a nd
calculation method. The measurement allowance was then expressed as a percentage of the NTE
threshold value.
After a 11 95 l percentile distributions w ere e valuated, there w ere s even measurement
allowances c orresponding t o t he combinations of the three PEMS uni ts and the applicable
calculation methods.
Next, the calculation method with the minimum normalized PM value will be chosen and
the corresponding normalized PM value will be selected as the best measurement allowance for
PM, assuming it validates. This PM measurement allowance would be the very last value added
to the act ual br ake-specific N TE PM threshold f or a g iven e ngine, ba sed o n actual f amily
emissions limit, mileage, model year, etc. N ote tha t if a ny m easurement al lowance w as
determined to have a va lue 1 ess t han zero, then that m easurement al lowance w as s et equa 11 o
zero.
REPORT 03.14936.12 27 of 174
-------
The B SPM NTE threshold used for this program was 0.02 g/hp-hr. This NTE threshold
was determined by EPA and approved by the Steering Committee during the generation of the
Test Plan.
These threshold values are of critical importance to the program, as they provide the basis
for the scaling of measurement allowances, the assessment of model convergence, and a variety
of other calculations performed during this program. T he general philosophy of the Test Plan
was to determine measurement allowances based on errors at these emission levels, especially in
the case of any errors that scaled with emission level.
The a nticipated out come from t he m odel r uns, a nalysis, a nd va lidation efforts c an b e
represented as a table similar to the one shown in Table 7. The table illustrates both the model
outcome, a nd t he p rocess for s electing t he final m easurement allowance va lues for t he AVL
PEMS.
TABLE 7. EXAMPLE OF SELECTION OF MEASUREMENT ALLOWANCE AT 0.02
G/HP-HR NTE THRESHOLD FOR THE AVL PEMS
Calc. Method ->
BSPM
Selected Method ->
Allowance at Respective NTE threshold (%)
Method 1
Exhaust Flow
Torque-Speed
38%
Method 2
Exhaust and Fuel
Flow Torque-Speed
18%
Method 3
Fuel Flow Torque-
Speed
20%
Exhaust and Fuel Flow Torque-Speed Method
The i ntent of t he f inal s election pr ocess was t o choose the s mallest of the thr ee
normalized PM values for the final measurement allowance. At that point, the percentages given
for the chosen calculation method would be applied to the BSPM NTE threshold value in order
to generate the final additive, brake-specific measurement allowances.
An implicit assumption of the process, as described in the Test Plan, was that the values
produced by the model for all three calculation methods would be successfully validated. In the
event that this did not occur, it would be necessary for the Steering Committee to determine a
valid alternate course of action, in order to determine the final measurement allowance values.
The final model run and the selection and generation of measurement allowances are described
fully in Section 0 of this r eport, i ncluding t he f inal a llowances a pproved b y t he S teering
Committee.
2.13 Model Validation
For reasons discussed earlier, the measurement allowances were generated using a Monte
Carlo computer model. As with all simulations, it is vital that such a model be validated through
comparison with real experimental data. In this case, the Measurement Allowance model needed
to be va lidated against a da ta s et generated through actual i n-use field t esting. B ecause t he
model generates an incremental error in comparison to a Laboratory Reference, a suitable in-use
reference measurement was needed for comparison to the PEMS measurements. T he Steering
REPORT 03.14936.12 28 of 174
-------
Committee de termined that the C E-CERT M obile E mission L aboratory, op erated b y t he
University of C alifornia-Riverside, w ould be a n a ppropriate r eference f or va lidation of t he
model-based in-field testing.
To ensure that the validation was not disturbed by some inherent bias between the SwRI
Reference Laboratory and the CE-CERT MEL validation reference, a correlation exercise was
performed between the two laboratories, prior to the start of on-road validation efforts. The CE-
CERT MEL was brought to SwRI's laboratory facilities in San Antonio, Texas, and a side-by-
side correlation test was run. During this test, exhaust from the same test engine was alternately
routed to the m easurements systems of both SwRI and CE-CERT. This was don e repeatedly
over the course of three days of testing. The data was then supplied to the Steering Committee,
in order to allow for a determination to be made that correlation between the facilities was
acceptable for the purposes of validation of the model.
After the correlation exercise was completed, a 2 007 test truck with a Cummins engine
was procured by CE-CERT for use in this validation exercise. In addition, two Sensors PEMS
used at SwRI during the program were also delivered to CE-CERT. A third PEMS unit of the
same t ype w as pr ovided b y S ensors. The s teering c ommittee allowed Sensors t o pr ovide a
similar model with some small hardware upgrades for testing. CE-CERT then conducted a series
of on-road test runs over various driving routes in California, which were designed to take the
test truck through a wide range of environmental and ambient conditions. D uring these tests,
simultaneous m easurements w ere m ade w ith the P EMS and the M EL in order t o generate a
validation data set. This formed the primary validation set for the model.
Because t he C E-CERT M EL doe s not r eadily i ncorporate a m eans of di rect t orque
measurement on a vehicle, the on-road validation data set could not be used to validate model
errors associated with broadcast torque.
The di fference be tween t he P EMS r esults and t he CE-CERT trailer r esults w ill be
compared to the measurement allowance limits predicted by the Monte Carlo Model and defined
by the LOESS fit.
Validation will be based on the following procedure. For each reference NTE event, the
Monte C arlo m odel w ill be us ed to generate t he 5 th and 95 th percentiles of t he s imulated
distribution of t he br ake-specific P M em ission differences. In o rder t o obt ain s imulations
representing similar conditions to those obtained on-road, some error surfaces may need to be
suppressed in the simulations since not all of them may be applicable to the on-road conditions.
The c hoice of w hich error s urfaces t o s uppress w ould ne ed t o b e made b y t he S teering
Committee.
Next, the 5 * and 95 * delta pe rcentiles obt ained from the above s imulations will be
separately fit to a line or curve using two chosen methods: a linear regression procedure and a
local regression (loess) technique [41. Depending on w hich of the resulting two fits i s best for
each set of data (i.e., either for the 5* percentile deltas or the 95* percentile deltas), the resulting
line or curve will be used as one of the lower or upper limits for the on-road data.
REPORT 03.14936.12 29 of 174
-------
To determine the best fit for a given set of delta percentiles (i.e., 5* or 95*), a simple
regression line initially will be fit to the data. If a least squares linear regression of the 5th or 95th
percentile deltas versus the ideal PM emission has an r2 greater than 0.85 and an SEE less than 5
percent of the median ideal PM emissions, then the regression line will be used. If this set of
criteria is not met, then a loess fit will be used. Since a loess regression requires the selection of
a smoothing parameter [5] to smooth the data, the chosen smoothness parameter should balance
the residual sum of squares against the smoothness of the fit.
The on -road delta e rrors, obtained from the results of col lecting d ata on s everal N TE
th
-th
events during on-road operations, will be plotted on a graph containing the 5 and 95 percentile
delta limits determined from the regression fits chosen above. The graph will consist of a plot of
delta PM versus ideal PM. The number of on-road points outside these limits will be determined
and expressed as a percentage of the total number on on-road data points. If this number does not
exceed 10% of the total number o f on-road d ata, the simulation data will be considered to be
valid.
An example of a validation plot is given in Figure 11 and Figure 12. The plots shown
correspond to gaseous emissions concentration data that were collected in the prior PEMS study.
Figure 11 contains the 5th and 95th validation limits for NOx data determined by fitting a linear
regression model to the simulated data for both limits. Figure 12 contains t he 5 * and 95 *
validation limits for NOx data determined by fitting a loess model to the simulated data for both
limits.
Validation 5th and 95th Percentile Deltas for 50 Ref NTE Events
NOx (g/kW-hr) Method 1 Mod 1 Linear Fit
0.7000
0.6000
0.5000
0.4000
0.3000
0.2000
0.1000
0
-0.1000
-0.2000
-0.3000
-0.4000
01234567
Ideal NOx g/kW-hr
FIGURE 11. LINEAR REGRESSION FIT TO 5TH AND 95TH PERCENTILE DELTAS
REPORT 03.14936.12
30 of 174
-------
Validation 5th and 96th Percentile Delias tor 50 Ref NTE Events
NOx (g/kW-hr) Method 1 Mod 1 LOESS Fit
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
-0.1
-0.2
-0.3
-0.4-I
01234567
Ideal NOx g/kW-hr
FIGURE 12. LOESS REGRESSION FIT TO 5TH AND 95TH PERCENTILE DELTAS
Validation of the model was assessed independently for the PM pollutant for each of the
three P EMS m odel uni ts, a nd f or e ach of t he applicable t hree c alculation m ethods. A full
description of the validation efforts, including the data analysis methodology and the results of
PM validation for each PEMS unit by all three calculation methods is given in Section 2.4, with
the exception of the CE-CERT on road validation testing.
REPORT 03.14936.12
31 of 174
-------
3.0 PART 1065 PEMS AND LABORATORY AUDIT
Prior to the start of official testing both the laboratory system and each PEMS underwent
an extensive audit in accordance with CFR Part 1065. Table 8 summarizes the audits that were
performed on each type of system along with the CFR reference for each of the verifications.
TABLE 8. LINEARITY VERIFICATION RESULTS FOR INTAKE AIR FLOW AND
FUEL FLOW
Fuel Flow
Description
Measured
Criteria
Pass / Fail
|xmin(ai-l)+a0|
-0.02
0.55
Pass
ai
1.00
0.98-1.02
Pass
SEE
0.14
1.10
Pass
R2
1.000
0.990
Pass
Intake Air Flow
Description
Measured
Criteria
Pass / Fail
|xmin(ai-l)+a0|
0.00
15.68
Pass
ai
1.00
0.98-1.02
Pass
SEE
1.22
31.36
Pass
R2
1.000
0.990
Pass
3.1 1065 Lab Audit
The most important audits performed on the laboratory system were those directly related
to the P M cone entration measurement ac curacy, na mely t he 1 inearity of flows and the P M
balance. The intake air flow and fuel flow were verified for linearity, while the CVS and PM
sampling flows were verified using a p ropane recovery check. The linearity verifications were
performed in accordance with 40 CFR Part 1065.307 although they were performed within 180
days r ather t han 370. T he pr opane c hecks w ere pe rformed w eekly dur ing official t esting i n
accordance with 40 CFR Part 1065.341. The maximum allowable interval for the propane check
is 35 days. Table 8 shows the initial linearity verifications for fuel and intake air flow.
One additional verification was performed for the fuel and intake air flow with similar
results. Two different nominal flow rates were used for the PM secondary dilution system, one
for steady state testing and another for transient testing. A total nominal flow of 3.6 mVhr (2.0
scfm) was used to target a filter face velocity of 100 cm/s during steady state testing. Because the
filter measurement was unofficial the total flow was reduced during transient testing to prevent
overloading the filter. A nominal total flow rate of 2.4 m 3/hr was used during transient testing.
The propane recovery check was performed at a total secondary flow of either 3.6 or 2.4 mVhr
depending on whether t he t esting at t he t ime w as s teady s tate or t ransient. Table 9 shows a
summary of t he pr opane r ecovery results. The pe rcent di fference i s b etween the cal culated
propane concentration based on a kno wn f low of pr opane a nd t he m easured pr opane
concentration from a hydrocarbon analyzer. The pass limit for the CVS system is plus or minus
two percent and plus or minus five percent for the secondary sampling system.
REPORT 03.14936.12
32 of 174
-------
TABLE 9. CVS PROPANE RECOVERY CHECK SUMMARY
Test Date
10/29/08
1 1/05/08
11/14/08
12/01/08
12/08/08
12/18/08
01/05/09
01/12/09
01/20/09
02/1 1/09
02/18/09
02/25/09
03/06/09
03/16/09
03/24/09
04/07/09
CVS
Blower,
m3/hr
3,941
3,855
3,887
3,921
3,931
3,955
3,832
3,875
3,928
3,899
3,896
3,841
3,905
3,893
3,804
3,927
Secondary
Dilution,
nWhr
1.75
1.73
1.72
1.73
1.72
1.75
1.31
0.97
0.97
1.00
1.00
1.00
1.90
1.92
1.05
1.87
Secondary
Total,
nWhr
3.53
3.64
3.65
3.81
3.75
3.62
2.63
2.36
2.36
2.28
2.46
2.31
3.72
3.72
2.17
3.87
Secondary
Sample
Diff, %
-1.16
1.02
1.17
-1.1
0.06
-2.15
-1.89
-2.72
-2.98
-0.05
1.13
2.04
1.86
0.68
0.6
1.37
CVS
Diff, %
0.75
1.67
1.66
0.37
1.33
1.4
-0.61
-0.64
-0.17
-1.35
0.74
0.77
1.19
-0.56
0.51
0.28
Occasionally a propane check was outside of the allowable limits, but when the check
was r epeated i t us ually pa ssed unde r t he s ame c onditions. It w as ne cessary t o pa ss t wo
consecutive propane checks if the initial check failed. The final result from each day is shown in
the Table 9.
Linearity w as a Iso verified on the P M b alance us ed for filter w eights. T he line arity
verification results are shown in Table 10.
TABLE 10. LINEARITY VERIFICATION FOR PM BALANCE
PM Balance
Description
Measured
Criteria
Pass / Fail
|xmin(ai-l)+a0|
0.00
20.00
Pass
ai
1.00
0.99-1.01
Pass
SEE
0.00
20.00
Pass
R2
1.000
0.990
Pass
Verification of t he P M ba lance i s r equired e very 370 da ys although the che ck was
performed after 180 days for quality purposes and passed with similar results.
REPORT 03.14936.12
33 of 174
-------
3.2 1065 PEMS Audit
Before the start of engine testing, each PEMS was required to pass the verifications set
forth in CFR Part 1065. Because the measurement of PM is a non-standard process there are no
audits s pecified on the actual P M me asurement, but the ins truments s till n eeded to pass t he
necessary flow, temperature, and pressure audits. Table 11 lists the Part 1065 audits performed
on the lab and on the PEMS.
TABLE 11. SUMMARY OF PART 1065 AUDITS
Description
Linearity
Torque Meter
Pressure, Temperature, Dewpoint
Fuel Flow
Intake Flow
CVS Verification
PM Balance Verification
CFR Reference
1065.307
1065.310
1065.315
1065.320
1065.325
1065.341
1065.390
Lab
xa
X
X
X
X
X
X
PEMS
xb
a Linearity for lab performed on flow meters, torque meter, pressures, and temperatures
b Linearity for PEMS performed on flow meters
Since no analyzer ve rifications w ere p erformed, the mos t c ritical a udits on the P EMS
were the linearity verifications of flow measurements. Because all three types of PEMS use flow
measurements to calculate a dilution ratio, the accuracy of the flow measurements is directly
related to the accuracy of the reported PM emissions. The AVL and Horiba PEMS measure their
total and dilution flow to calculate a dilution ratio while the Sensors PEMS measures the dilution
and sample flow. The following equation is used to calculate the dilution ratio:
Dilution Ratio =
3.2.1 Horiba Flow Audits
Total Flow
Total Flow
Sample Flow Total Flow Dilution Flow
The Horiba PEMS has four flow measurements in the system: dilution flow, DCS flow,
make-up air flow, and total flow. During typical operation all flows but the dilution flow are held
constant. The dilution flow rate is varied to sample proportionally from the raw exhaust based on
changes in exhaust flow. The CFR requires linearity verifications on sample, dilution, and total
flow (whichever two of the three that are measured) [2]. The DCS and makeup flows affect the
accuracy of the dilution ratio and the filter flow, however both of these flows are maintained at a
nominal value of 2 1pm it was decided not to perform a linearity verification in this situation.
The steering committee elected to perform a spot check on the DCS flow to ensure its accuracy
and verify the filter flow which includes both the total and make-up air flow. The filter flow is
equal to the difference between the total flow and make-up air flow. Because the filter flow was
designed to operate at a constant flow of approximately 28 1pm it was not practical or logical to
verify the flow measurement over 10 e ven points down to zero flow as recommended in CFR
Part 1065.307. After discussing the issue with Horiba and the steering committee it was decided
REPORT 03.14936.12
34 of 174
-------
to verify the filter flow over a range of plus and minus ten percent of its operating target, using
eleven steps. The maximum range that the dilution flow could vary was approximately 21 to 31
1pm; the dilution flow was verified over this range in eleven even steps. The flow audits were
initially performed with a TSI flowmeter, but this was replaced with a bubble flowmeter when it
was discovered that the accuracy of the TSI flowmeter degraded as the pressure during the flow
measurement deviated from atmospheric. The results from the linearity verifications are shown
in Table 12 for the Horiba PEMS.
TABLE12. LINEARITY VERIFICATIONS FOR HORIBA PEMS
Verification Description
Intercept
Slope
SEE
R2
Horibal
Dilution Flow
Measured
Linearity Criteria
Pass/Fail
0.29
031
Pass
0.98
0.98-1.02
Pass
0.05
0.62
Pass
1.000
0.99
Pass
Filter Flow
Measured
Linearity Criteria
Pass/Fail
0.07
0.30
Pass
0.97
0.98-1.02
Fail
0.03
0.60
Pass
1.000
0.99
Pass
Horiba2
Dilution Flow
Measured
Linearity Criteria
Pass/Fail
0.08
0.31
Pass
1.00
0.98-1.02
Pass
0.3
0.61
Pass
0.999
0.99
Pass
Filter Flow
Measured
Linearity Criteria
Pass/Fail
0.18
0.30
Pass
0.98
0.98-1.02
Pass
0.03
0.60
Pass
1.000
0.99
Pass
Horiba3
Dilution Flow
Measured
Linearity Criteria
Pass/Fail
0.05
0.31
Pass
1.02
0.98-1.02
Pass
0.04
0.62
Pass
1.000
0.99
Pass
Filter Flow
Measured
Linearity Criteria
Pass/Fail
0.01
0.30
Pass
0.99
0.98-1.02
Pass
0.04
0.60
Pass
0.999
0.99
Pass
Horiba-1 narrowly missed passing linearity ve rification for tot al flow w ith a slope of
0.97. This check was repeated several times without passing. However, the error as a percent of
point was better than 0.5 percent for all eleven points. This result is a problem with applying the
linearity criteria from Part 1065 to a flow that is not verified over the range from zero to full
scale. Given the excellent agreement on a point-by-point basis, the steering committee elected to
proceed without taking corrective action.
REPORT 03.14936.12
35 of 174
-------
The linearity verification results for the Sensors PEMS are shown in Table 13.
TABLE 13. LINEARITY VERIFICATIONS FOR SENSORS PEMS
Verification Description
Intercept
Slope
SEE
R2
Sensorsl
Sample Flow
Measured
Linearity Criteria
Pass/Fail
-0.01
0.02
Pass
1.00
0.98-1.02
Pass
0.00
0.04
Pass
1.000
0.99
Pass
Major Dilution Flow
Measured
Linearity Criteria
Pass/Fail
0.05
0.09
Pass
0.98
0.98-1.02
Pass
0.08
0.17
Pass
0.999
0.99
Pass
Minor Dilution Flow
Measured
Linearity Criteria
Pass/Fail
-0.04
0.05
Pass
1.01
0.98-1.02
Pass
0.01
0.09
Pass
1.000
0.99
Pass
Sensors2
Sample Flow
Measured
Linearity Criteria
Pass/Fail
0.01
0.01
Pass
1.00
0.98-1.02
Pass
0.01
0.03
Pass
0.999
0.99
Pass
Major Dilution Flow
Measured
Linearity Criteria
Pass/Fail
0.07
0.07
Pass
1.01
0.98-1.02
Pass
0.05
0.13
Pass
0.999
0.99
Pass
Minor Silution Flow
Measured
Linearity Criteria
Pass/Fail
0.02
0.04
Pass
1.01
0.98-1.02
Pass
0.06
0.08
Pass
0.998
0.99
Pass
Sensors3
Sample Flow
Measured
Linearity Criteria
Pass/Fail
0.01
0.01
Pass
1.00
0.98-1.02
Pass
0.01
0.02
Pass
0.999
0.99
Pass
Major Dilution Flow
Measured
Linearity Criteria
Pass/Fail
0.01
0.07
Pass
1.00
0.98-1.02
Pass
0.05
0.14
Pass
1.000
0.99
Pass
Minor Dilution Flow
Measured
Linearity Criteria
Pass/Fail
0.01
0.02
Pass
0.96
0.98-1.02
Fail
0.01
0.04
Pass
1.000
0.99
Pass
The Sensors PEMS had the capability of performing a self-audit using 1065 criteria. An
external T SI f lowmeter was pr ovided a s pa it of t he S ensors e quipment a nd i ts m easurements
were recorded by the Sensors software to linearity verifications on the dilution and sample flows.
The total dilution flow i s calculated by the addition of the major and minor dilution flows, so
these two measurements are audited independently. The Sensors PEMS was able to pass all but
REPORT 03.14936.12
36 of 174
-------
the m inor di lution flow a udit for S ensors uni t 3. T his ve rification w as pe rformed r epeatedly
without passing. Since the total dilution flow would still pass a linearity verification in this case,
no further action was taken. In the case of both the Horiba and the Sensors, the absolute accuracy
of the flows are important in determining the mass of PM emitted. The AVL system is a real
time particle sensor rather than a proportional batch sampler. For this type of instrument, only
the di lution ratio a nd n ot t he a bsolute a ccuracy of t he t otal a nd di lution f lows a ffect t he
measurement accuracy. Forthis reason the dilution ratio was audited instead of the total and
dilution flow rates which are used to calculate the dilution ratio. It was also not possible to vary
the total flow, which is held constant during normal operation. Although the dilution ratio was
maintained at a constant of 5 throughout official testing, a six point check was performed ranging
from 2 to 6 in steps of 1. Table 14 shows the results for the linearity verifications for the AVL
units.
TABLE 14. LINEARITY VERIFICATIONS FOR AVL PEMS
Verification Description
Intercept
Slope
SEE
R2
AVL1
Flow Based Dilution Ratio
Measured
Linearity Criteria
Pass/Fail
0.01
0.07
Pass
0.96
0.98-1.02
Fail
0.01
0.15
Pass
1.000
0.99
Pass
AVL2
Flow Based Dilution Ratio
Measured
Linearity Criteria
Pass/Fail
0.03
0.70
Pass
0.97
0.98-1.02
Fail
0.02
0.14
Pass
1.000
0.99
Pass
AVL3
Flow Based Dilution Ratio
Measured
Linearity Criteria
Pass/Fail
0.05
0.07
Pass
1.02
0.98-1.02
Pass
0.00
0.14
Pass
1.000
0.99
Pass
The MSS dilution ratio was initially verified using both flow measurements as well as
CC>2 measurements. Because the AVL PEMS had an internal CC>2 measurement it would have
been pos sible t o us e a COz span bot tie t o ve rify t he di lution r atio, unfortunately i t w as not
possible to ever introduce an undiluted CC>2 sample to CC>2 sensor to provide a span. When the
sample is undiluted, the CC>2 cell is bypassed so that it does not measure. The results shown in
Table 15 w ere generated us ing T SI flowmeters t o m easure t he t otal and di lution flow a nd
calculate t he di lution ratio in the s ame m anner as t he P EMS. AVL-1 and A VL-2 w ere bot h
unable t o pa ss t he s lope c riteria but w ere w ithin t hree pe rcent of poi nt a cross t he s ix po int
verification. The s teering c ommittee a greed to accept the dilution ratio accuracy tol erance of
three percent.
3.2.2 Exhaust Flow
Official linearity verifications were not conducted on the PEMS exhaust flow meters at
SwRI. The steering committee decided that a calibration from the manufacturer was sufficient so
long as the flow meter was within five percent of the lab during engine testing. The three flow
meters tested from Horiba and the three from Sensors were all within found to be within five
REPORT 03.14936.12 37 of 174
-------
percent of the lab measurement. An unofficial linearity verification was performed using the data
from the steady state engine testing. The exhaust flow measurement was averaged during each
state measurement sample and compared with the laboratory measured value over the same time
period (50-270 seconds depending on the engine condition). There were between 64 and 84 data
points per exhaust flow meter. The linearity plots are shown in Figure 13 for PEMS-1, Figure 14
for PEMS-2, and Figure 15 for PEMS-3.
1200
ฐo 1100
fM
1000
J. 900
2 800
in
re
700
600
500
400
y = 0.995x-15.427
R2 = 0.999
y = 0.977x-16.188
R2 = 0.998
400 500 600 700 800 900 1000
Lab Exhaust Flow (m3/hr, 20ฐC)
1100 1200
*Horiba-l Sensors-1
FIGURE 13. LINEARITY CHECK ON PEMS-1 EXHAUST FLOW DURING STEADY-
STATE ENGINE TESTING
1200
IN
o
I/I
3
re
x
LLJ
in
LLJ
Q.
1100
1000
900
800
700
600
500
400 --
y = 1.021x-37.608
R2 = 0.997
y = 0.995x-17.619
400 500 600 700 800 900 1000
Lab Exhaust Flow (m3/hr, 20ฐC)
1100 1200
* Horiba-2 Sensors-2
FIGURE 14. LINEARITY CHECK ON PEMS-2 EXHAUST FLOW DURING STEADY-
STATE ENGINE TESTING
REPORT 03.14936.12
38 of 174
-------
1200
y=0.987x+3.218
R2 = 0.996
= 0.986x-13.106
R2 = 0.999
400 500 600 700 800 900 1000 1100
Lab Exhaust Flow (m3/hr, 20ฐC)
1200
*Horiba-3 Sensors-3
FIGURE 15. LINEARITY CHECK ON PEMS-3 EXHAUST FLOW DURING
STEADY-STATE ENGINE TESTING
Each of t he s ix ex haust flow m eters w as able to pa ss t he s tandard e rror, s lope, a nd
correlation coefficient c riteria f or a r aw exhaust f low m easurement s pecified in CFR P art
1065.307 however only one was able to pass the intercept criteria (Sensors-3). This is likely due
in part to the fact that the measurements did not extend below 470 m3/hr making it more difficult
to pa ss a n i ntercept c riteria t hat a ssumes e venly spaced d ata poi nts e xtended dow n t o z ero.
Conducting a linearity verification on the exhaust flow measurement during engine testing was
for informational purposes only.
REPORT 03.14936.12
39 of 174
-------
4.0 ENGINE DYNAMOMETER TESTING AND RESULTS
4.1 Testing Objective
The purpose of the engine dynamometer testing was to characterize the bias and precision
errors of t he P EMS dur ing s teady-state and t ransient en gine ope ration. D uring s teady-state
engine operation, the PEMS measurements were compared with that of the CVS filter-based PM
measurement to characterize the bias in each of the three PEMS. The transient engine testing was
used t o de termine t he p recision o f each PEMS b y quantifying the v ariability of the P EMS
measurement over a series of repeated transient NTE events.
4.2 Experimental Setup
4.2.1 Engine and Sampling System
Preliminary t esting was pe rformed using a 6. 4 liter lig ht he avy-duty di esel en gine
provided by Navistar, however the engine used to generate all official steady-state and transient
data was a 2007 V olvo MP7 provided by Volvo Powertrain. T he test plan initially called for
official dynamometer testing to be performed on two different engines, but funding constraints
reduced official testing to a single engine. The Volvo MP7 had a displacement of 10.8 liters and
was rated at 280 kilowatts (375 horsepower). The engine was equipped with a variable geometry
turbocharger (VGT) and a water-cooled high pressure exhaust gas recirculation (EGR) loop. The
engine intake system was connected to a test-cell water-cooled intercooler. The engine is shown
in Figure 16.
FIGURE 16. VOLVO MP7 INSTALLED IN A CVS TEST CELL
REPORT 03.14936.12
40 of 174
-------
The engine was also equipped with a close-coupled diesel oxidation catalyst (DOC) and a
diesel particulate filter (DPF) combination. For the purpose of producing higher PM emission
levels, a bypass was created around the aftertreatment system to allow an adjustable amount of
exhaust flow around the DPF. A DOC was added to the bypass so that all of the PM in the
exhaust would pass through an ox idation c atalyst s imulating a scenario of a cracked DPF. A
picture of the bypass is shown in Figure 17.
FIGURE 17. DPF BYPASS WITH DOC
The or iginal s tock aftertreatment w as 1 ocated in t he m ain leg of t he ex haust, while a
separate catalyst was procured for the bypass leg. The DOC was 76 millimeters (six inches) in
diameter with a length of 152 millimeters (12 inches). Three butterfly valves were placed in the
exhaust s ystem toe ontrol t he a mount of e xhaust pa ssing t hrough e ach 1 eg. T he D PF w as
regenerated via an exhaust fuel inj ection system. For all testing, the bypass was open to some
degree, how ever, the bypass leg was closed when active regenerations were performed on the
DPF. The DPF bypass went through multiple iterations until the proper PM level was achieved
during steady-state testing. A PM 1 evel of 0.025 g /hp-hr was easily obtainable during transient
cycles, however it w as extremely di fficult to obtain this same PM level during steady-state
engine operation. Table 15 lists the five different configurations of the bypass that were tested.
REPORT 03.14936.12
41 of 174
-------
TABLE 15. LIST OF DPF BYPASS CONFIGURATIONS
Iteration
1
2
3
4
5 (Final)
Pipe
Diameter
3
3
3
3
4
Inlet Probe
No
No
3", Upstream
3", Upstream
3", Upstream
DOC
Diameter
3
3
None
6
6
Outlet Probe
1", Upstream
1", Downstream
1", Downstream
1", Downstream
3", Downstream
In the final configuration (iteration 5), two butterfly valves in the main leg of the exhaust
were completely closed with only a one or two millimeter gap between the valve and the exhaust
pipe. This not only forced a majority of the exhaust through the bypass, but significantly raised
the exhaust backpressure. Based on measurements upstream and downstream of the bypass with
the AVL PEMS, it was estimated that well over 50 percent of the exhaust was routed through the
bypass in the final configuration. A diagram of the bypass system is shown in Figure 18.
Positive Displacement
Pump (POP)
Dilution
Air
Gas Meter
Pump
Sample Line -
Heated Line +
47mm PM Filter
Com Cable
Valve
FIGURE 18. SCHEMATIC OF ENGINE DYNAMOMETER EXPERIMENTAL SETUP
The schematic is not to scale so that some distances may appear incorrectly. There are
more t han 10 pipe diameters be tween t he m ixing poi nt of t he b ypass a nd t he first P EMS
sampling position for the Sensors SemtechPPMD. Additionally, there are approximately 10
pipe diameters b etween each of t he P EMS s ampling 1 ocations s o that an y flow di sturbances
REPORT 03.14936.12
42 of 174
-------
caused by a different pitot tube or sample probe should not affect the other PEMS. The portion of
exhaust between the first and the last sampling location is insulated to minimize the cooling of
the exhaust and thermophoretic deposition of particles. The 1 ast sampling probe i s that of the
Horiba OB S-TRPM, which i s j ust upstream of the entrance into the CVS tunnel. The gaseous
PEMS units (Sensors Semtech DS and Horiba OBS-2200) were used only for data acquisition
and t rigger s ignals. N o P EMS g aseous e missions w ere r ecorded dur ing t his pr ogram. T he
Semtech DS was used with both the Sensors PPMD and the AVL MSS, while the Horiba OBS-
2200 was used in conjunction with the Horiba OBS-TRPM.
The outlet of the aftertreatment was routed to a constant volume sampling (CVS) tunnel
for emissions measurement. The CVS consists of a positive displacement pump and an upstream
heat exchanger; it was maintained at a nominal flow rate of 3,740 m 3/hr (2,200 scfm) for this
testing. The di lution a ir i s e xtracted t hrough a filter pa ck from a t emperature a nd hum idity
controlled area. The po rtion of t he C VS t unnel t hat i s e xposed t o t he test c ell e nvironment
upstream of the exhaust is insulated to prevent heating of the dilution air from the test cell. The
particulate matter samples were extracted from the CVS as shown in Figure 18.
Intake air flow was measured by a laminar flow element (LFE) with a maximum flow of
1,700 nWhr (1,000 scfm). The LFE was oriented so that there were 10 diameters of straight pipe
before the inlet and after the outlet to minimize flow disturbances. The fuel flow was measured
by a M icro-Motion flow me ter. The addition of i ntake a ir flow a nd fuel flow w as us ed t o
determine the exhaust flow using the equations from CFR Part 89. The exhaust flow was used to
calculate t he exhaust P M cone entration from t he C VS filter as w ell as a che ck on t he P EMS
exhaust flow m eters. A r aw C O2 analyzer w as us ed i n c ombination w ith t he e xhaust flow
measurement t o calculate a r aw carbon balance f uel f low w hich was com pared with t he
measured fuel flow and the dilute carbon balance fuel flow as a quality check.
The PM sampling system consisted of a 47mm teflon membrane filter (Whatman Teflo),
a fine metal screen backing, and a plastic filter cartridge. The total flow is operated at a nominal
flow of 60 s tandard 1 iters pe r m inute (2.1s cfm) w hich r esults i n a filter f ace ve locity of
approximately 100 cm/s. The standard temperature and pressure used for all flow rates in this
report is 20 ฐC and 101.325 kPa as specified in CFR Part 1065. The nominal dilution flow is 30
slpm (1.1 scfm) which resulted in a dilution ratio of 2 and a sample flow of 30 slpm (1.1 scfm).
The system maintained a constant dilution ratio and achieved proportionality through sampling
from the CVS. The system was heated to 47ฐC and had an approximate residence time of 0.8
seconds from the inlet of the sample probe to the filter. A cyclone with a 2.5 micron cutpoint at
17 slpm was positioned just downstream of the sample probe. The aforementioned PM sampling
system was the laboratory reference used to generate all reference PM data used in this program.
4.2.2 Sensors PPMD
The PPMD was the PEMS unit installed closest to the outlet of the bypass, approximately
15 diameters downstream. An experiment was conducted to ensure the flow was fully mixed at
this location as will be discussed later. The PPMD was installed in the horizontal orientation as
shown in Figure 19.
REPORT 03.14936.12 43 of 174
-------
FIGURE 19. PPMD INSTALLED IN THE TEST CELL
The P PMD w as e quipped w ith a 1 ong s traight pipe c ontaining its e xhaust flow me ter
(EFM) just upstream of the 90 degree elbow from which the sample is extracted. The dilution in
the S ensors s ystem t akes pi ace i nside t he i nstrument. T he m ost c ommon i nstallation of t he
PPMD is in the vertical position, although it requires only the rotation of the moisture traps to
properly operate the PPMD in the horizontal orientation. The Sensors PEMS was equipped with
two s tages of di lution know n a s M PS1 a nd MPS2. A Ithough s ome pr eliminary t esting w as
conducted using both stages of dilution, the steering committee decided to use only a single stage
of di lution for all official te sts. The P PMD is a pr oportional s ampling system that va lies its
dilution ratio inversely with exhaust flow to maintain a mini mum di lution r atio of 6 a tthe
maximum e xhaust flow r ate o f a n e ngine. T he P PMD me asures P M u sing a Q uartz C rystal
Microbalance (QCM), which charges the particles using a corona needle, deposits the particle on
a Quartz Crystal, and then measures the change in frequency of the crystal to determine the mass
deposited. The PPMD is a batch sampling device meaning it does not report PM concentration in
real-time but instead reports a s ingle mass value per event. Because each crystal requires a pr e
and post frequency measurement to determine mass, a total of eight crystals are included to allow
for c ontinuous operationby switching crystals. Crystal s ampling b egins as soon asthe engine
enters the NTE zone and stops as soon as the engine exits the NTE zone. One of the of the eight
crystals was used as a r eference crystal to adjust the measurements for changes in temperature
and pressure. This left up to seven crystals available for measurement although it was common to
have one or two crystals not working on any given test. The timing of the samples during testing
was designed around having a minimum of five working crystals available for measurement. The
PPMD was included in the measurement allowance program because inertial microbalances had
already been approved for PEMS applications in 40 CFR Part 1065.
REPORT 03.14936.12
44 of 174
-------
4.2.3 AVLMSS
The AVL Micro Soot Sensor (MSS, also known as the Photo Acoustic Soot Sensor or
PASS) w as i nstalled dow nstream of t he P PMD i n t he m iddle of t he ve rtical por tion of t he
exhaust pipe leading to the CVS tunnel. The MSS is connected to the tunnel via a 2 meter heated
sample line which was maintained at 52ฐC. The mixing of dilution air takes place in the dilution
box just upstream of the sample probe so that the dilute sample is transported through the sample
line. The MSS is shown in Figure 20.
FIGURE 20. THE AVL MSS
The MSS consists of two boxes shown in the above figure. The top box is the measuring
unit which c ontains t he r esonance chamber for the s oot m easurement. The bottom box is the
conditioning unit which contains the sample and dilution pumps as well flow controllers. The
dilution pump is optional as the MSS can also provide dilution air via an external input of 300
kPa of compressed air. The steering committee requested that the MSS operate using its internal
dilution pump, since this is the way it would operate during in use testing. The MSS measures
soot by heating the elemental carbon using a laser. When the soot is heated it emits a sound wave
that is detected by a microphone. The MSS can report soot concentration on a IHz basis and uses
a constant dilution ratio, which was set at 5 for all official testing. Because the MSS measures
only soot and not total PM, it was included in this program as a partial participant. If both the
Sensors and Horiba units could not complete the measurement allowance program it was to be
considered for in-use. The AVL system does not have its own gaseous PEMS; or data storage
device; i nstead i t de pends on t he g aseous i nformation from t he S emtech D S a nd sends i ts
concentration signal to the Sensors Semtech DS where the necessary information is recorded. For
all testing in this program a single Semtech D S was used to record the signals from both the
REPORT 03.14936.12 45 of 174
-------
Sensors S emtech P PMD as well as the A VL M SS. The probe for the Horiba OBS-TRPM i s
located approximately 1.65 meters (5 feet) downstream of the MSS. The Horiba system contains
two separate exhaust pieces that are each about 150 mm (6 inches long). The upstream portion is
a tail pipe adapter (TPA) which is a pi tot tube exhaust flow measurement. The downstream pipe
contained the probe for the PM sampling. The TPA and sampling probe can be seen in Figure 21.
FIGURE 21. THE PROBE AND TPA FOR THE HORIBA OBS-TRPM
4.2.4 Horiba TRPM
The Horiba system is a proportional sampling device that varies its dilution ratio in the
same manner as the PPMD. The dilution air is introduced just downstream of the probe before
the heated sample line. The point of dilution can be seen in Figure 21 where the three stainless
steel lines converge into the stainless steel cylinder. The OBS-TRPM uses a T SI BAD (referred
to here as a DCS) real time particle instrument to measure the particle concentration on a second
by second basis and collects PM on a gravimetric filter simultaneously. The filter weight gain is
used to provide a calibration constant to the real time particle signal and apportion the PM mass
appropriately. The DCS instrument measures continuously, but the filter is designed to sample
during v alid N TE event ope ration from t he s ame di luted exhaust s tream. The filter s ampling
begins after five seconds in the NTE zone and will continue for a minimum sample time of 30
seconds even if the engine is no 1 onger in the NTE zone. Because the EPA's PM standard is
based on gravimetric filter analysis, the Horiba system was included in the program. The OBS-
TRPM is comprised of several different boxes including the heated enclosure (HE), the diffusion
charge sensors (DCS), the electrical enclosure (EE), and the mechanical enclosure (ME). The HE
contains the 47mm filter holder, and the DCS is the real time particle sensor. Dilution air was
REPORT 03.14936.12
46 of 174
-------
provided using a com mercially av ailable oi 1-less c ompressor. T he H oriba O BS-2200 ga seous
PEMS was used to log the ECM J1939 broadcast, measure the exhaust flow, and provide an NTE
trigger to the TRPM to start filter sampling. The combined TRPM-2200 system contains a total
of 6 box es plus an external compressor. T he components of the Horiba system are depicted in
Figure 22.
FIGURE 22. THE HORIBA OBS-TRPM AND OBS-2200
The TRPM used the same 47mm Whatman teflo filter, metal screen, and plastic cartridge
as the CVS system. All weighing and conditioning of both the CVS and the TRPM filters was
conducted i n t he S wRI filter w eighing r oom. The filter w eighing r oom is ma intained at a
temperature of 22 ฑ PC with a dewpoint of 9.5 ฑ PC in accordance with CFR Part 1065.190.
Filters were stabilized in the weighing environment for at least 1 hour prior to both the initial and
final weights. Each filter was weight was determined by the average of three weights on a scale
with a resolution of 0.1 jig.
4.3 Bypass Mixing Verification
The flow from the DPF bypass was reintroduced into the main exhaust stream using a 76
mm (3 i nch) pr obe facing dow nstream w ith a n or ifice ne ar t he t ip of the pr obe t o pr omote
mixing. T esting w as c onducted toe nsure t he e xhaust flow w as fully mixed pr ior t o t he first
sampling 1 ocation, w hich w as oc cupied b y t he Sensors P EMS. T wo pr obe or ientations w ere
created at the spot where the Sensors sample was extracted. One of the probe orientations was
parallel to the upstream exhaust elbow and one was perpendicular to the elbow. The orientation
of the AVL sample probe is shown in Figure 23.
REPORT 03.14936.12 47 of 174
-------
FIGURE 23. EXPERIMENTAL SETUP FOR MIXING VERIFICATION
At e ach of t he pr obe orientations t he M SS w as us ed t o m easure t he e xhaust s oot
concentration using a variable length probe that could traverse the length of the 127 mm (5 inch)
exhaust pipe. There were five sample locations for each orientation as shown in Figure 24.
Perpendicular
Parallel
FIGURE 24. MIXING VERIFICATION SAMPLE LOCATIONS
Each of the positions were one inch apart, with position 3,8 being at the center of the
exhaust pipe. For a measurement, the probe was started at location 1, moving down to 5 and then
back to 1. The probe was then moved to position 6m oving down to position 10 and back to
position 6. T he M SS m easurement w as r ecorded for 80 s econds ate ach pos ition, w ith t he
average of the last 30 seconds used for comparison. The steady-state modes with the highest and
REPORT 03.14936.12
48 of 174
-------
lowest e xhaust flow r ates w ere c hosen top erform t he m ixing ve rification, toe nsure pr oper
mixing over the entire range of test conditions.
The mixing verification was performed several times with different bypass configurations
with similar results in all cases. Only the results from the final mixing verification are presented
since the other results from different configurations are not relevant to the data in this report. The
data from the final verification is presented in Figure 25.
01
O
C
o
u
1.2
1.0
0.8
aj 0.6
-------
4.4 PEMS Loss Corrections
Each P EMS m anufacturer was given t he o pportunity toe orrect t heir f inal P M
measurement t o account for v arious pa rticle 1 osses e ncountered du ring the s ampling p rocess.
Any 1 oss c Direction ha d t o be pr esented t o t he s teering committee for a pproval be fore i t w as
implemented in the program. Sensors and AVL both chose to implement loss corrections, while
Horiba declined to apply a loss correction to their data.
4.4.1 Sensors PPMD Loss Correction
Sensors c onducted w ork unde r a separate pr oj ect at S wRI t o experimentally as sess t he
losses in the PPMD. David Booker presented the results of this work along with the proposed
Sensors loss corrections at the meeting on D ecember 10*, 2008 at SwRI. The final PPMD loss
correction included thermophoretic, electrostatic, and CVS loss factors. The CVS loss correction
factor was meant to estimate the particle losses in the CVS system, since this is the standard to
which t he P PMD i s c ompared. A Ithough t ypical 1 oss c orrection factors w ill i ncrease t he
estimated PM concentration, the CVS correction factor actually decreased the PPMD estimated
concentration. The S ensors s trategy was to use the th ermophoretic and electrostatic los s
corrections to determine what the true PM concentration is and then reduce that number by the
amount of PM mass they believe will be lost in the CVS system. They did not wish to merely
adjust to the correct concentration since the CVS system to which their instrument was compared
did not correct for losses. AtotalCVS loss of 15 percent was estimated by Sensors based on
general experience rather than specific data. The total loss correction was estimated to increase
the PM concentration by 5 to 10 percent when including the electrostatic and thermophoretic loss
factors. The proposed loss corrections were accepted by the steering committee and implemented
in the Sensors PPMD post processor. All Sensors data in this report includes these correction
factors unless otherwise stated.
4.4.2 AVLMSSLoss Correction
The proposed AVL loss correction was presented at the meeting on November 12th, 2008
at S wRI. The los s c orrection implemented by A VL w as int ended to correct f or the
thermophoretic losses in the system and is based off a paper by Stratmann et al [6]. The equation
for the loss correction is shown below:
Msoot = j mss(t + A/) qex (t) (l + L(lex (t)))dt
f . fOifT<150 }
L(T}=\ \
(a + b(T-l 50J/300 otherwise]
The m agnitude of t he correction i s t emperature de pendent a nd w as e stimated t o be
approximately 10 percent in most cases. This correction was accepted by the steering committee
and implemented in the AVL Concerto post processor.
At the meeting at SwRI on December 10th, 2008, AVL stated that their loss correction
was currently capped at a maximum loss of 25 percent regardless of the calculated value. AVL
REPORT 03.14936.12 50 of 174
-------
requested approval to remove this limitation and allow the equations output to be the correction,
regardless of its magnitude. This change was accepted by the steering committee.
4.4.3 AVLMSS TotalPMCorrection
The AVL Concerto post processor includes a function that estimates the total PM based
on the measured soot, exhaust temperature, and total hydrocarbon concentration, along with a
number of a djustable i nput pa rameters i ncluding t he vol ume of t he catalyst, t he 1 ight o ff
temperature of the catalyst and the sulfur content of the fuel. Although this calculation was not
approved for use with the official measurement allowance data, a portion of the AVL data was
processed to examine the results from the total PM model. All AVL data presented in this report
refers to the soot concentration corrected for losses unless otherwise stated.
4.5 Steady-State Testing Procedure
Originally the test plan called for two different engines at three different emission levels
ofPM: DPF out, 0.02 g/hp-hr 1 evel, and a 0.03 g /hp-hr level. TheDPF out level is simply
whatever the emissions happen to be with no bypass which was well below the 2007 standard of
0.01 g/hp-hr. The 0.02 and 0.03 g /hp-hr 1 evels would be s et b y adjusting the DPF b ypass to
produce t he corresponding b rake-specific P M n umber from t he C VS filter. D ue t o funding
limitations the testing was reduced to a single engine at a single emission level. Because 0.03
g/hp-hr will be used for the first year of compliance testing and 0.02 g/hp-hr i s used with the
following years, i t w as i mportant t o i nvestigate t he pe rformance of t he P EMS c overing t hese
levels. For this reason an average of the two threshold PM levels, 0.025 g/hp-hr, was used as the
target.
The obj ective of t he s teady-state t esting w as t o evaluate t he bi as and precision of t he
PEMS using 180 data points for each PEMS manufacturer. The 30 points consists of six steady-
state modes of engine operation (6), 10 repeats (10), one emission level (1), one engine (1), and
three different PEMS units (3), 6*10*1*1*3=180. A PM steady-state error surface, AssraPM (^),
was de veloped for each P EMS m anufacturer s o that t here a re t hree steady-state PM e rror
surfaces for use with calculation methods 1 and 2. For calculation method 3, the AVL MSS will
have a unique AssmPM based on the calculations for method 3. A s mentioned previously, the
Sensors and Horiba PEMS will only use methods 1 a nd 2. T o determine the most suitable six
steady engine modes for steady-state testing screening tests were performed using the 80 points
Cummins cycle and measuring the PM levels with the AVL MSS and the TSI Engine Exhaust
Particle S izer ( EEPS). The E EPS pr imarily pr ovides s ize di stribution information, but ma ss
concentration can be inferred using an assumed density. Since the AVL is the only PEMS that
can report a real time mass concentration without further efforts such as filter weighing and post
processing it was chosen to perform the screening work. The Cummins cycle steps through 80
steady-state engine modes as a transient cycle. The engine is stepped through 10 different speeds
and eight engine 1 oads at each of the selected speeds. The minimum speed of the cycle i s the
minimum NTE speed, and the minimum torque is 30 pe rcent of the torque at the given speed
meaning that the cycle effectively maps the NTE zone. Figure 26 shows the speed and torque
points of the cycle.
REPORT 03.14936.12 51ofl74
-------
120
100
2^ 80
o>
3
U1
,0 60
a*
LLJ
0
0
20
40 60 80
Engine Speed (%)
100
120
FIGURE 26. SPEED AND LOAD FOR THE 80 POINTS CYCLE
The engine remained at each of the points for 8 8 seconds with a one second transition
between modes. Because the speed and load were only incrementally changed in between each
mode, it was assumed that each point would stabilize relatively quickly. The fact that this cycle
was us ed onl y as a m ethod of s creening a Iso contributed t o t he de cision toe hange m odes
quickly. Even with such short modes, the total cycle length was 2 hour s. To allow for purging,
and calibrating the gas analyzers, the cycle was divided into two 40 poi nts cycles each lasting
one hour. The gas analyzers were necessary because this cycle was al so used to tabulate ECM
fuel rate errors for this error surface.
The PM emissions were estimated by measuring each of the 80 modes with the EEPS and
MSS, then choosing two of the modes to perform a filter measurement and compare the ratio of
the filter measurement to the EEPS and MSS as an estimate of the CVS filter B SPM at each of
the 80 modes. Although this method is not highly accurate, it provided a way to quickly obtain
rough estimates of the engine PM levels over a wide range of speed and torque conditions. Of the
initial 80 points, only 12 were estimated to produce brake-specific PM of greater than 0.02 g/hp-
hr. T he final s ix m odes w ere chosen w ith t he i ntent of c overing asm uch of t he N TE z one a s
possible while still maintaining high BSPM levels, and a range of PM exhaust concentrations.
Several iterations of adjusting the bypass, taking filter measurements, and narrowing down the
number of points occurred before the steering committee approved the final six points for steady-
state testing. Originally the DPF bypass was adjusted to produce the 0.025 g/hp-hr of PM based
on a filter measurement during a short version of the NTE transient cycle. The PM emissions at
steady-state were much lower at the same bypass setting. Since it was desirable to conduct the
steady-state testing at the same PM levels as the transient testing, it was necessary to adjust the
system to allow a much greater amount of the flow through the bypass. Six points were chosen
REPORT 03.14936.12
52 of 174
-------
out of t he t welve top rovide a range of P M concentration, exhaust flow r ates, a nd engine
operating conditions. The six points that were chosen are shown in Figure 27 along with the NTE
zone.
2500
0
500
1000 1500
Engine Speed (rpm)
2000
2500
Lug Curve NTE S Line -NTET-S Line NTE P-S Line SS Points
FIGURE 27. FINAL SIX STEADY-STATE MODES
Although the MSS was used to screen the 80 p oints, the actual concentration at each
position was verified with CVS filter measurements before selecting the points. The exhaust PM
concentration w as calculated b y m ultiplying t he CVS P M c oncentration by t he C VS di lution
ratio. T he C VS di lution r atio w as c alculated b y di viding t he a verage C VS flow r ate b y t he
average exhaust flow rate. The CVS dilution ratio ranged from 3 t o 7.5 r esulting in an overall
dilution ratio of 6 to 15 when including the secondary PM filter dilution.
The steady-state testing was conduc ted as a modal t ransient c ycle with e ach m ode
repeated twice for a total of 12 modes per cycle. 6 different cycles were created with the order of
the m odes r andomized in each cycle. Table 16 lists the sample order of the modes in the 6
steady-state cycles.
REPORT 03.14936.12
53 of 174
-------
TABLE 16. SAMPLE ORDER FOR STEADY-STATE CYCLE TESTING
Sample 1
Sample 2
Sample 3
Sample 4
Sample 5
Sample 6
Sample 7
Sample 8
Sample 9
Sample 10
Sample 1 1
Sample 12
Cycle 1
3
6
5
2
4
1
5
2
1
3
6
4
Cycle 2
1
4
3
6
5
2
2
1
4
5
6
3
Cycle 3
3
1
6
2
5
4
4
2
3
1
5
6
Cycle 4
3
4
1
2
6
5
6
4
1
2
5
3
Cycle 5
4
2
1
5
O
6
5
3
4
6
2
1
Cycle 6
6
1
4
3
5
2
4
1
6
3
5
2
The steady-state testing was conduc ted as a ramped modal c ycle w ith the eng ine
remaining at each operating condition for three minutes before the start of sampling. An external
trigger from t he 1 ab w as pr ovided toe ach of t he P EMS a nd t he C VS filter s ystem sot hat
sampling w ould be gin simultaneously for a 11 i nstruments. T he e ngine t hen r emained a 11 he
operating condition for 5 seconds after the end of sampling to ensure that no delay in the end of
sampling by any of the PEMS caused part of the transition period to be captured as a s ample.
The engine remained at the condition for five seconds after sampling had finished to ensure all
systems had finished sampling before the operating condition changed. The order of the modes
was randomized and each mode was repeated twice within a single cycle for a total of 12 data
points for each cycle. Six different cycles were created, which would create a total of 72 data
points. Although the target was only 60 valid points for each set of PEMS, in practice several
cycles had to be repeated to collect enough valid data. An example of one of the steady-state
cycles is shown in Figure 28.
REPORT 03.14936.12
54 of 174
-------
2500
Q.
~ 2000 H
0)
0)
Q.
1500 -
Zi 1000
z^
9> 500 -\
0
0 500 1000 1500 2000 2500 3000 3500 4000
Time (sec)
^Speed ^Torque
FIGURE 28. EXAMPLE OF STEADY-STATE CYCLE
To ensure a comparably accurate filter weight, a filter weight gain of 100 jig was targeted
for each mode. By collecting this amount of material on each filter, the weighing variability and
tunnel background contribution could be minimized. Collecting more material than this for each
steady state sample would have caused problems with the Horiba and Sensors PEMS by limiting
the amount of time they could operate before switching filters or cleaning crystals. To produce a
similar filter weight ga in at s ix di fferent s teady-state m odes w ith different m ass r ates, the
sampling time was adjusted for each mode to meet this target. The sample time ranged from 50
seconds to 245 seconds. Since the sampling time for each mode was different, the total length of
time spent at each mode was different as well. The sample time for each mode for each round of
the PEMS is shown in Figure 29.
REPORT 03.14936.12
55 of 174
-------
250
FIGURE 29. STEADY-STATE SAMPLE TIMES
Figure 30 shows t he ave rage filter w eight gain for all six modes for each ofthethree
PEMS.
140
0
Model Mode 2 Mode 3 Mode 4 Mode 5 Mode 6
PEMS1 PEMS 2 PEMS 3
FIGURE 30. CVS FILTER WEIGHT GAIN FOR STEADY-STATE TESTING
REPORT 03.14936.12
56 of 174
-------
Before presenting any of the PM concentration data it should be mentioned the choice of
units i n this w ork. T he t est pi an c ailed for the PM c oncentration t o be c alculated i n t erms of
|ig/mol. Although it i s an uncommon unit for describing mass concentration it was considered
for fundamental t o us e m ol f or vol ume r ather t han m 3 i n w hich case a s tandard r eference
condition must be defined. For reference, 1 m g/m3 of air at 20ฐC and 1 01.325 kPa is equal to
24.055 jig/mol. In several cases the values in mg/m3 are provided in parenthesis for reference,
but all official data was calculated and plotted using jig/mol.
The preliminary steady-state results from PEMS 1 were presented at the December 11*,
2008 meeting at SwRI. After reviewing the first set of steady-state data, the steering committee
felt t hat t he c oncentrations from t he s ix steady-state points w ere not e ffectively c overing t he
desired range. Five of the modes are clustered between 115 and 161 jig/mol (4.8 and 6.7 mg/m3)
with the remaining mode at 325 jig/mol (13.5 mg/m3). At the recommendation of the steering
committee the bypass setting was slightly adjusted for PEMS 2 and PEMS 3 in an attempt to fill
in s ome of t he r egion be tween 161 and 325 |i g/mol. D ue t o as hift in t he e ngine out P M
emissions, i t w as pos sible t o i ncrease t he c oncentration f or m odes 2, 3, 5, a nd 6 w hile
maintaining the same levels for modes 1 a nd 4.1 n fact, the dampers were actually adjusted to
flow 1 ess exhaust through the bypass indicating that the engine out PM had not only changed
relatively between operating conditions but increased overall. Figure 31 shows the median CVS
filter PM concentration for each of the three sets of PEMS.
Mode:
Mode 2 Mode 3 Mode4 Mode 5 Mode 6
PEMS1 HPEMS2 HPEMS3
FIGURE 31. STEADY-STATE EXHAUST PM CONCENTRATION (jiG/MOL)
REPORT 03.14936.12
57 of 174
-------
The m ode with the hi ghest c oncentration, m ode 2, i ncreased up t o ne arly 423 jig/mol
while the lowest concentration increased from 115 jig/mol on mode 5 to 132 jig/mol on mode 4.
It i s i nteresting t o not e that m ode 5 and 6 i ncreased i n c oncentration by 87 a nd 64 pe rcent,
respectively while mode 4 increased by less than 2 percent. This increase in concentration came
while opening the valve in the DPF leg of the exhaust, thereby increasing the flow of exhaust
through the DPF. Modes 2 and 3 both shifted downwards between PEMS 2 and PEMS 3 without
any change in the exhaust valve positions. Figure 32 shows the brake-specific PM values as
measured by the lab reference for all three sets of PEMS.
CO
Model Mode 2
ModeS Mode 4 Mode 5 Mode 6
PEMS1 BPEMS2 I PEMS3
FIGURE 32. STEADY-STATE BRAKE-SPECIFIC PM, CVS FILTER (MG/HP-HR)
The brake-specific PM ranged from 15.7 mg/hp-hr to 43.5 mg/hp-hr.
4.6 Data Yield During Steady-State Testing
The test plan called for a minimum of 10 valid data points for each mode for each PEMS,
allowing for a minimum data set of 30 for each 5* and 95* percentile delta that was generated. In
reality the target of 10 data points per PEMS per mode was not met in all cases due to additional
points that were invalidated by post processing software that had been updated after the testing
had been completed. While very few data points were removed during post processing for the
Horiba a nd AVL P EMS, S ensors s upplied S wRI w ith s everal n ew po st pr ocessors after t he
completion of te sting tha t inva lidated a significant por tion of t he S ensors da ta. S ince t he
information on what criteria would invalidate the data was not available at the time of testing, it
was not possible to know how many additional tests would be required to achieve the necessary
number of data points. The Sensor's post processor was revised to include some points that were
deemed valid data but had be en excluded by the postprocessor. The final set of data for the
Sensors PEMS had between 28 and 34 poi nts per mode. Figure 33 shows the number of valid
data points for steady-stat testing.
REPORT 03.14936.12
58 of 174
-------
0
Model Mode2 Mode3 Mode4 ModeS Mode6
Horiba Sensors AVL
FIGURE 33. NUMBER OF VALID DATA POINTS FOR STEADY-STATE TESTING
The total number of possible data points was either 40 or 41 depending onthemode.
Samples were taken 41 times at each mode, but modes 2, 3, 4, and 6 each had one point excluded
due to mishandling of the CVS filter.
4.6.1 Data Yield During Steady-State Testing
Table 17 shows t he s teady-state d ata yield by each of t he PM-PEMS relative to the
possible data yield obtained by the CVS. A total of 29 steady-state cycles were conducted for all
three PEMS with each of the six steady-state modes repeated twice in each cycle for a total of 12
data poi nts pe r cycle. Twenty-one of t hese cycles w ere cons idered valid tests from the
perspective of the function of the cycle command, NTE external trigger, filter sampling, and at
least one or more of the PEMS capturing valid data. During these valid tests four data points
were missed due to a mishandling of the CVS filter, but the rest were considered v alid data
points from the perspective of the lab measurements. Data from the PEMS was removed for a
variety of reasons i ncluding pr oblems w ith t he da talogging, s ampling, e xhaust f low
measurement, mechanical failures, and filter handling. Appendix E contains a complete list of
the reasons data was excluded, but several notable problems will be discussed here.
REPORT 03.14936.12
59 of 174
-------
TABLE 17. DATA YIELD BY EACH PM-PEMS
Mode 1
Mode 2
Mode3
Mode 4
Mode 5
Mode 6
Total
Possible
41
41
41
41
41
41
246
Horiba
37
36
36
35
37
37
218
Sensors -
NewPP
22
25
25
24
17
23
136
Sensors -
Revised PP
34
29
32
28
31
30
184
AVL
40
33
33
35
37
34
212
The Horiba system requires an external source of compressed air capable of supplying
approximately 30 1 pm at 400 kP a or higher. A commercially-available oil-less compressor was
provided by Horiba for use with the system. Unfortunately this compressor had a tendency to
stop working on qui te a few occasions during testing. Three different compressors of the same
model were provided and each experienced this problem. It was believed that the compressor
was overheating and shutting off to protect itself, although changes in the test cell temperature
did not seem to influence its performance. The compressor would begin to work again after 10 to
15m inutes presumably once i t ha d c ooled of f I f t he c ompressor s topped w orking w hile t he
Horiba system was in operation, the system lost all of its dilution air once the small air tank had
been depleted. This resulted in an undiluted exhaust stream being sampled onto the filter which
would quickly overload the filter at the PM concentration levels used in this work not to mention
fail pr oportionality r equired of t he H oriba s ystem. If t he c ompressor s topped w orking at a ny
point during an of ficial test, the Horiba data for that test was voi ded. Figure 34 shows an
example of a steady-state cycle where the compressor stopped working.
A problem occurred with Sensors 1 i nvolving the auto zero function of its exhaust flow
meter. Every hour the Sensors system would attempt to zero the exhaust flow meter by switching
the pressure transducers to ambient for a period of time less than a minute. On the Sensors 1 the
solenoid switching the pressure transducers from exhaust measurement to zero was not working
properly causing t he z ero function t o oc cur w hile pr essures w ere be ing m easured from t he
exhaust. Because t he steady-state cycle 1 asted longer t han on e hour , this w ould c ause a n
erroneous exhaust flow measurement on the last two modes of each steady-state cycle. Figure 35
shows an example of the EFM zero problems during a test.
The EFM auto zero function was disabled for the remaining tests on S ensors 1, and for
Sensors 2 and 3.
REPORT 03.14936.12
60 of 174
-------
0 1000 2000 3000 4000
Time (sec)
Sample Flow to Filter Dilution Flow
FIGURE 34. HORIBA SAMPLE FLOW TO THE FILTER AND DILUTION FLOW
WHILE COMPRESSOR STOPS
0
1000
2000 3000
Time (sec)
4000
Sensors Lab
FIGURE 35. SENSORS EFM ZERO DURING STEADY-STATE CYCLE
REPORT 03.14936.12
61 of 174
-------
4.7 Accounting for CVS Variability During Steady-State Testing
The s teady-state da ta h as va riability due to the P EMS, CVS, a nd test a rticle. By
computing a paired difference for each data point, the test article variability is removed from the
data. This concentration delta still contains variability associated with the PEMS and the CVS.
The data collected in this program does not allow for independent assessment of the PEMS and
CVS variability, but by assuming a CVS variability it becomes possible to assign the remaining
variability to the PEMS.
The following procedure for estimating the PEMS and CVS variability was proposed by
artin
committee.
Bill M artin at th e D ecember 12 th 2008 meeting at S wRI and w as a ccepted by t he s teering
1. Since test article variation is the same for each individual observation by the CVS (xPEMS,i)
and PEMS (xcvs,i~), compute the paired differences,
**i = \XPEMS,i ~ XCฅS,i) .
These paired differences (i.e., delta values or concentration deltas) contain random variation
from the CVS, random variation from the PEMS, and a mean offset between CVS and PEMS
(bias error).
2. Divide the entire set of delta values into j = 1 to M subsets based on the values of xcvsi- The
data sets are not subdivided by engine operating mode or PEMS serial number, but only the
level of the reference concentration.
3. For each subset, there are i = ItoNj values. The median and the M AD ar e us ed as the
descriptive statistics.
4. Calculate t he m edian delta va lue, A50;, them edian absolute deviation of the A. values,
, and the estimate of the standard deviation of the CVS random error, SDCVSrandomJ.
MAD ] = median^ -^O
N T.T ,
; !=1
CVS, filter, i
Tx
^ CVS,i
i=l
L
cVS, filter,]
where Lcvsfilteri is the PM s ample filter loa ding. 5 jig is the assumed CVS variability as
proposed by the steering committee based on a nominal filter loading of 100 jig.
REPORT 03. 14936. 12 62 of 174
-------
For each subset], calculate a corrected delta for each A. value, in subset], as follows:
if M<
then
if MAD]> 0.45495- SD2CVSrandomj
Note that this approach correctly passes through any significant offsets observed in the data.
These offsets should be passed through even if they persist only for a subset of data, such as
a given mode.
5. The entire set of corrected delta values is then to be used to establish the error surface for the
steady-state data. The 5th, 50th, and 95th delta values are used to establish the 1st, 50th, and
99th percentile values which are the inputs to the Monte Carlo model.
4.8 Steady-State Testing Results
4.8.1 Comparison between PEMS and Lab Delta PM
All steady-state data presented has already been corrected for the steady-state variability
as mentioned above. The steady-state concentration deltas from PEMS 1 forHoriba, Sensors,
and AVL, are shown in Figure 36, Figure 37, and Figure 38, respectively.
Each point on the x-axis represents the median exhaust PM concentration from the CVS
filter for a s ingle m ode. T he y-axis r epresents t he 5th, 50th, a nd 95th percentile of t he de Itas,
PEMS - Lab. As mentioned previously there was a clear gap in the data between 162 jig/mol and
325 jig/mol which represents a large portion of the target concentration range for a 0.025 g/hp-hr
level. Figure 39, Figure 40, and Figure 41 show the deltas for PEMS 2 where three of the modes
were between 208 and 267 jig/mol.
Horiba-1 and Horiba-2 performed similarly with mode 2 showing a significant negative
bias, mode 4 showing a positive bias and the other four modes closer to zero. Sensors-2 showed
a much greater negative bias than Sensors-1. Sensors-1 had a 50th percentile of between 0 and -
26 |i g/mol for five of the six modes while Sensors-2 was between -32 and -104 |i g/mol for the
50* percentile for the same five modes. In addition, mode 2 had a 5* percentile of -340 jig/mol
at a reference concentration of 442 ji g/mol. For the same mode on Sensors-1 the 5* percentile
was -157 |ig/mol at a reference concentration of 326 jig/mol. AVL-2 was lower than AVL-1 with
50th percentiles be tween -34 |i g/mol a nd -70 |i g/mol. T he 50 th percentiles for A VL-1 were
between 13 |ig/mol and -32 |ig/mol. No indication of the changes in performance for these
PEMS was discovered through the recommended checks and audits. The PM deltas for Horiba-3,
Sensors-3, and AVL-3 are shown in Figure 42, Figure 43, and Figure 44, respectively
REPORT 03.14936.12 63 of 174
-------
0
^
+ป
-------
___ J. JU
2 1 nn
C J.UU
i
nn
-zuu
300
A
A
' ' +
A ป
^ z
*
A
*
0
50
100 150 200 250 300 350 400 450
Lab Reference Concentration (ng/mol)
+ Horiba50th Horiba95th A Horiba 5th
FIGURE 39. HORIBA-2 STEADY-STATE PM CONCENTRATION DELTAS
REPORT 03.14936.12
65 of 174
-------
. -LDU
2 1 nn
c ฑuu
hft ^n
ou DU
"nT n
CD U
M
c 9nn
Q) Z.\J\J
u
c Ten
O ^-3U
u
_. ann
> jUU
ฐ- _ซn
1 1 1 1
*
ซt .
v
^
A
0
400
100 200 300
Lab Reference Concentration (ng/mol)
* Sensors 50th % Sensors 95th % A Sensors 5th %
FIGURE 40. SENSORS-2 STEADY-STATE PM CONCENTRATION DELTAS
PM Concentration Delta (ng/mol)
J.JU
1 nn
1UU
en
0
en
OU
1 nn
J.UU
1 C.D
"?OO
~^uu
9C.O
3nn
-jUU
3c.o
~
%
A
0 50 100 150 200 250 300 350 400 450
Lab Reference Concentration (|ig/mol)
* AVL 50th % AVL 95th % A AVL 5th %
FIGURE 41. AVL-2 STEADY-STATE PM CONCENTRATION DELTAS
REPORT 03.14936.12
66 of 174
-------
0
1
M
o
o
'^
I_
J. JU
o -i r>r>
C 1UU
a.
"ZT n
ro u
M
oป en
Q bU
ฃ i nn
o ~iuu
M
u
O ~25U
U
Q.
I I i i I I I I
^
* ป ป
A A
*
A
0 50 100 150 200 250 300 350
Lab Reference Concentration (ng/mol)
400 450
* Sensors 50th % Sensors 95th % A Sensors 5th %
FIGURE 43. SENSORS-3 STEADY-STATE PM CONCENTRATION DELTAS
REPORT 03.14936.12
67 of 174
-------
PM Concentration Delta (ng/mol)
J..3U
1 nn
CTf>
jU
n
Cf)
OU
-1 nn
1 c;n
-J.DU
->nn
-zuu
Ten
-ZDU
onn
_3t;n
+
A
ir
I I ^ US I X I * I I I
0
50
100 150 200 250 300 350 400 450
Lab Reference Concentration (ng/mol)
*AVL50th% AVL95th% 4AVL5th%
FIGURE 44. AVL-3 STEADY-STATE PM CONCENTRATION DELTAS
Horiba-3 and Sensors-3 are within the same ranges of their first two instruments, while
AVL-3 p reduced a n e xtremely high bi as a 11 he hi ghest concentration ( mode 2 ). T he 50 *
percentile of m ode 2 for AVL-3 was 84 |i g/mol, while it was 13 |i g/mol and -62 |i g/mol for
AVL-1 and AVL-2, respectively.
When the data from all three PEMS i s considered, the range of concentrations from 5
mg/m3 to 18 mg/m3 is covered, although there is still a majority of the data located between 5
and 7 mg/m3 as a result of the data from PEMS 1. The individual deltas for the Horiba, Sensors,
and AVL are shown in Figure 45, Figure 46, and Figure 47, respectively. Please note that each
point represents a single measurement, not pooled data.
REPORT 03.14936.12
68 of 174
-------
200
o
00
ro
+->
0)
Q
c
o
"p
ro
0)
u
c
o
u
Q.
-300
-400
0
100 200 300 400
Lab PM Concentration Reference (ng/mol)
500
* Horiba-1 n Horiba-2 AHoriba-3
FIGURE 45. STEADY-STATE CONCENTRATION DELTAS FOR HORIBA
200
-400
0
100 200 300 400
Lab PM Concentration Reference (ng/mol)
*Sensors-l n Sensors- 2 ASensors-3
FIGURE 46. STEADY-STATE CONCENTRATION DELTAS FOR SENSORS
500
REPORT 03.14936.12
69 of 174
-------
200
100
t>o
(U
Q
C
o -
100
ฃ -200
u
o
u
a.
-300
-400
100
200
300
400
500
Lab PM Concentration Reference (ng/mol)
*AVL-1 DAVL-2 A AVL-3
FIGURE 47. STEADY-STATE CONCENTRATION DELTAS FOR AVL
Because t he r eference concentration of s everal of t he m odes w as similar, the s teering
committee decided to group the data by reference concentration rather than by operating mode.
The da ta w as s plit up i nto g roups of a pproximately 20 da ta poi nts ba sed on r eference
concentration for developing the steady-state PM error surface. The Horiba data was grouped
into 11 sets, the Sensors data into 9 sets, and the AVL data into 10 sets.
Figure 48, Figure 49, and Figure 50 show PEMS PM concentration plotted against the
PM cone entration determined by the reference 1 aboratory filter m ethod for steady-state NTEs.
These plots mainly show the qualitative PEMS to PEMS scatter relative to the filter method.
4.8.2 Correlation between PEMS and Lab PM
Figure 5 1 shows a linear regression between each set of PM-PEMS and the CVS filter
method. G ood c orrelation w as obs erved be tween t he M SS a nd lab, w ith as coe fficient of
determination (R2) of 0.84 and a s lope of 0.89. The s lope suggests t hat t he M SS P M
concentration i s 11 percent lower than that determined by thelab. This trend is expected since
the MSS is measuring soot and the lab reports total PM. The slope seemed to be high because the
MSS PM concentration in the range between 15 and 20 mg/m3 was higher than that of the lab.
The linear regression between the Horiba TRPM and the lab resulted in R2 of 0.55 and a
slope of 0.86, indicating some correlation. After further investigation, it was recognized that the
weak correlation was due to Mode 4 (high speed, light load) of the SS testing, with concentration
levels of about 5 mg/m3. By removing this mode from the data, a R2 of 0.85 was obtained and the
slope w as moved from 0.861 o 0.88. It i s 1 ikely t hat M ode 4, could ha ve r esulted i n
overestimation of PM due to nanoparticle formation with the Horiba dilution system, although
CVS t esting a 11 his condition di d not s how a n anoparticle m ode. Based on t he s lope of t he
correlation, the Horiba PM concentration was 14 percent lower than that reported by the lab.
REPORT 03.14936.12
70 of 174
-------
The linear regression between the Sensors PPMD and the lab results in R2 of 0.34 and a
slope of 0.64. The weak correlation was due to data scatter. Except in the narrow range between
5 and 7 mg/m3, the Sensors PPMD showed underestimated PM. Based on the slope, the Sensors
PPMD PM concentration was 36 percent lower than that reported by the lab.
100
200
300
400
500
Lab PM Concentration Reference (ng/mol)
ซ Horiba-1 Horiba-2 A Horiba-3
Lab Reference
FIGURE 48. STEADY-STATE HORIBA PEMS PM CONCENTRATION VERSUS THE
LABORATORY REFERENCE
500
0 100 200 300 400 500
Lab PM Concentration Reference (ng/mol)
* Sensors-1 n Sensors-2 A Sensors-3 Lab Reference
FIGURE 49. STEADY-STATE SENSORS PEMS PM CONCENTRATION VERSUS THE
LABORATORY REFERENCE
REPORT 03.14936.12
71 of 174
-------
0
100
200
300
400
500
Lab PM Concentration Reference (ng/mol)
AVL-1 n AVL-2 A AVL-3 Lab Reference
FIGURE 50. STEADY-STATE AVL PEMS PM CONCENTRATION VERSUS THE
LABORATORY REFERENCE
25
20
-5. 15
c
o
I
01
10
5
5
10 15
Lab PM Concentration Reference (mg/m3)
20
25
ป Horiba Sensors AVL
FIGURE 51. LINEAR REGRESSION CORRELATION BETWEEN PEMS AND LAB
REPORT 03.14936.12
72 of 174
-------
4.8.3 Steady-State PM Error Surfaces
Figure 52, Figure 53, and Figure 54, show the original concentration deltas that were
presented to the steering committee on April 2n , 2009 at SwRI. The Sensors data includes some
additional poi nts t hat were a dded 1 ater due toe hanging t he e rror tolerances of t heir pos t
processor. Each marker on the plot represents one of the j = 1 to M subsets of data divided up by
concentration value. Each pi ot s hows both the 5th and 95th percentiles which are b ased on the
actual da ta. T he e rror s urfaces f or t he Monte Carlo m odel w ere ba sed on t he 1 st and 99
percentiles w hich were ex trapolated from t he 5
distribution of the data.
th
th
th
and 95 percentiles as suming a nor mal
The steering committee elected to smooth the error surface by not including some of the
data points that were within the envelope of surrounding points. A similar decision was made on
some of the error surfaces included in the gaseous measurement allowance program.
Figure 55, Figure 56, and Figure 57 show the final steady-state error surface ofHoriba,
Sensors, and AVL, respectively. The data presented in these plots has already been processed to
remove th e C VS va liability and adjusted to the 1st and 99 * percentile form the 5th and 95th
percentile. T he lines on t he plot represent the final error surfaces as approved by the steering
committee. The Horiba and AVL error surfaces were accepted by the steering committee during
the conference call on June 29*, 2009. S ome of the sensors data was reprocessed with different
tolerances to increase the data yield, so the sensors error surface was not accepted by the steering
committee until the July 14th, 2009 meeting in Indianapolis.
100
-200
o
100 200 300 400 500
Lab Reference Median PM Concentration (ng/mol)
-A-5th% -B-95tri% -*-50th%
FIGURE 52. STEADY-STATE CONCENTRATION DELTAS FOR HORIBA
600
REPORT 03.14936.12
73 of 174
-------
J.UU
O en
E 50
*c5 o
a.
~ZT en
ro OU
M
0) i nn
Q -1UU
S 1 en
O -LJU
ro ">nn
il -ZUU
F, 9^0
(1) Z JU
u
5 -3nn
O ouu
U
^ o en
^ -o jU
Q.
Ann
"M- UU
(
FIGURI
1 c;n
1DU
"o
ซc 1 nn
^5 1UU
2 c;n
+^ DU
0)
Q
ฃ n
.2
'?
(Q
ฃ en
C -DU
0)
u
C
o -i n.n
tj -1UU
Q.
1 en
.
" ??feiV"bj"
5*?*i" K:C%"-*^ *"" n
^ "y ^ '..si.''
..*- \ ,c..
) 100 200 300 400 500 6(
Lab Reference Median PM Concentration (ng/mol)
-A-5th% n 95th % -*-50th%
: 53. STEADY-STATE CONCENTRATION DELTAS FOR SENSORS
~$* '
^ET^S^^
^ ^.%Jl-^-:^ " ' A'. '
)0
0 100 200 300 400 500
Lab Reference Median PM Concentration (ng/mol)
-*-5th% a 95th % c 50th %
FIGURE 54. STEADY-STATE CONCENTRATION DELTAS FOR AVL
600
REPORT 03.14936.12
74 of 174
-------
o
TO
+->
0)
Q
Q.
D
0)
+j
u
0)
!_
O
u
0
100
200
300
400
500
CVS Filter PM Concentration (ng/mol)
-*-Deltalst% -CHDelta50th% -*-Delta99th%
FIGURE 55. FINAL STEADY-STATE PM ERROR SURFACE - HORIBA
200
-* 150
100
0
100
200
300
400
500
CVS Filter PM Concentration (ng/mol)
--Delta 1st % -ODelta50th% -A-Delta99th%
FIGURE 56. FINAL STEADY-STATE PM ERROR SURFACE - SENSORS
REPORT 03.14936.12
75 of 174
-------
o
00
ro
M
0)
Q
Q.
D
0)
+ป
u
0)
1_
o
u
0
100
200
300
400
500
CVS Filter PM Concentration (ng/mol)
--Delta 1st % -oDelta 50th % -A-Delta 99th %
FIGURE 57. FINAL STEADY-STATE PM ERROR SURFACE - AVL
With a few exceptions the PEMS generally exhibited a low bias, with the majority of the
50m percentile deltas falling below zero. The Horiba and Sensors both showed a degree of level
dependence with the negative bias generally increasing for higher PM concentrations. At a lab
reference concentration of 413 jig/mol the Horiba 50th percentile was -92 jig/mol and the Sensors
->th
->th
tii
50 percentile was -177 jig/mol at a lab reference concentration of 403 |i g/mol. The AVL 50
percentile w as ge nerally 1 evel i ndependent and remained between -8 a nd -58 ji g/mol for t he
entire r ange o f con centrations t ested. While t he 1 st percentile f or t he A VL also remained
relatively constant t hroughout t he c oncentration r ange, t he 99th percentile j umped to a m uch
higher va lue at hi gh co ncentration. The AVL 9 9th percentile j umped from 25 |i g/mol at 291
jig/mol to 165 jig/mol at 402 jig/mol. The cluster of positive deltas that caused this large increase
in the 99* percentile data i s due solely to mode 2 with PEMS 2. The majority of the positive
deltas observed for the Horiba were due to mode 4 which was a high speed light load condition.
Since it is understood that the DCS (EAD) real time particle sensor in the Horiba system is more
sensitive t o smaller pa rticles i t w as as sumed that m ay have a 1 arger n umber of s ub-50 nm
particles than the other 5 modes tested, causing more of the filter mass to be attributed to mode 4.
4.9 Transient Engine Results
The t ransient eng ine t esting w as de signed to characterize t he pr ecision error of t he
PEMS. A transient cycle consisting of 30 NTE events, 32 seconds each, was repeated multiple
times and any differences in the PEMS measurements were attributed to PEMS variability under
the assumption that the engine operation and PM emissions remained constant. No lab reference
value was captured on an individual NTE event basis, because it is not possible to collect PM on
a CVS filter for each NTE. Instead filter measurements were taken for each integrated transient
REPORT 03.14936.12
76 of 174
-------
NTE cycle as a general indicator of the PM emissions of the engine during the entire cycle. The
cycle used in this testing was developed using the cycle generator from the gaseous measurement
allowance program. In the gaseous program the cycle generator was used to develop 20 uni que
cycles with different orders of NTE events and different transitions between events. The steering
committee el ected to generate one single cycle for the current work that would be used for all
transient testing. Additional time was added in between NTE events to ensure that the PPMD
could sample throughout the cycle without missing any NTE events under the assumption of 7
working c rystals ( 6 for s ample and one r eference c rystal). In a ddition s everal s hort non -valid
NTE events were added to the cycle to challenge the PEMS for measurements of non-valid NTE
events.
The DPF bypass was adjusted to produce an integrated cycle BSPM of approximately
0.03 g/hp-hr based on the CVS filter. Instead of readjusting the bypass for each set of PEMS, the
exhaust valves were set to the same position each time. The NTE events were all approximately
34 s econds i n duration. The original events from the cycle generator were each 32 seconds,
however the Sensors PEMS would occasionally see these events as shorter than 30 seconds and
exclude t hem. Many of t he N TE eve nts cont ained extreme ace elerations and decelerations
stopping just short of the lower boundaries of the NTE window. Considerable time was spent to
ensure that the J1939 signal remained in the NTE window during events, although the engine
performance shifted slightly and occasionally an event was invalidated. If any NTE event did
not remain in the NTE windows for at least 30 s econds it was not included in the data for the
transient error surface. A total of 16 c ycles were run for PEMS, 17 f or PEMS2, and 18 f or
PEMS3. Figure 58 shows repeat engine speed traces for the transient cycle with the first and last
official cycle conduc ted w ith each PEMS. Figure 59 shows the repeat torque traces for the
transient cycle. The COV on cycle work was 0.7 percent over 51 cycles.
500
PEMSl-l
-PEMS1-16
PEMS2-1
1000
Time (sec)
PEMS2-17
1500
-PEMS3-1
2000
PEMS3-18
Event Number
FIGURE 58. REPEAT ENGINE SPEED TRACES FOR NTE TRANSIENT CYCLE
REPORT 03.14936.12
77 of 174
-------
2200
-200
0
500
1000
1500
2000
Time (sec)
-PEMSl-l PEMS1-16 PEMS2-1 PEMS2-17 PEMS3-1 PEMS3-18 Event Number
FIGURE 59. REPEAT ENGINE TORQUE TRACES FOR NTE TRANSIENT CYCLE
The exhaust concentration of soot, as measured by the AVL PEMS is shown in Figure
60. The measured exhaust concentrations of soot ranged from 5 t o 3800 jig/mol (0.2 to 160
mg/m3) during NTE operations, with concentrations of 5 to 14 jig/mol (0.2 to 0.6 mg/m3) outside
of the NTE window. The widest dynamic range observed during an NTE event was for event 9
where the initial spike in soot concentration was measured as high as 3800 jig/mol before falling
down to 55 jig/mol near the end of the event. Figure 61 gives a closer look at events 20 through
23 to show the variability of the real-time signal over the three sets of PEMS. It should be noted
that the AVL PEMS is shown here only because it reports the measured soot concentration on a
second by second basis without additional processing.
Although the Horiba PEMS was used in this program as a batch sampler rather than a
second by second instrument for official results, Figure 62 shows the second-by-second data of
the H oriba P EMS a long w ith t he A VL. T here were m ajor di fferences on s ome of t he pe ak
concentrations most notably NTE event 9, w here the measured AVL concentration was nearly
four times higher than the measured Horiba concentration. With no reference, the purpose of this
figure i s onl y t o s how t hat t here a re di fferences w ithout t rying t o qua ntify t he a ccuracy. T he
Sensors P EMS pe rforms i ts m easurements as a ba tch sampler s o that t here w as no real-time
exhaust concentration reported.
REPORT 03.14936.12
78 of 174
-------
500
1000
1500
2000
PEMS3-18
Time (sec)
PEMS3-1 PEMS2-17 PEMS2-1
FIGURE 60. REPEAT AVL SOOT CONCENTRATION TRACES FOR NTE
TRANSIENT CYCLE
1000
0
18
1440
1490
1540
1590
1640
PEMS3-18
Time (sec)
-PEMS3-1 PEMS2-17 PEMS2-1
FIGURE 61. AVL SOOT CONCENTRATION DURING NTE TRANSIENT CYCLES,
EVENTS 20-23
REPORT 03.14936.12
79 of 174
-------
500
1000
Time (sec)
1500
2000
AVL-Soot
Horiba-PM
FIGURE 62. COMPARISON OF AVL AND HORIBA REAL TIME SIGNALS DURING
TRANSIENT CYCLE
To properly quantify the precision of the PEMS it i s important that the reference value
remain constant otherwise changes in the source PM emissions will be attributed to the PEMS as
measurement variability. In examining the CVS filter results, it was clear that the engine PM
emissions va ried somewhat dur ing te sting. After r eviewing the tr ansient P EMS da ta a t the
January 28 *, 2009 meeting a t S wRI, the s teering c ommittee r equested that the i dea of a
correction t o t he P EMS data be a pplied t o a ccount for c hanges i n t he e ngine pe rformance. A
correction factor ba sed on the c ycle i ntegrated CVS B SPM w as pr esented to the com mittee
during the April 2nd, 2009 meeting at SwRI and was accepted for use on all of the transient data.
The correction factor was calculated as follows:
W rye. rye
it=i L v ^i L>V j,
avg
N CVSt CVSt
The correction factor is multiplicative and applied to the PEMS data in the following manner:
The CVS BSPM along with the correction factor is shown in Figure 63.
REPORT 03.14936.12
80 of 174
-------
1.4
[->
[->
O
p
p
i
p
u
ฃ
c
_g
+J
u
QJ
&_
&_
O
u
15
O.C
10
20 30 40
Transient Cycle Number
50
60
-fc-CVS Filter BSPM Correction Factor
FIGURE 63. CVS BSPM AND CORRECTION FACTOR FOR TRANSIENT CYCLE
The correction factor ranged from 0.78 to 1.24. There were two cycles in which the CVS
filter measurement was void; in these cases the correction factor was set to one. The CVS brake
specific PM ranged from 34.4 mg/hp-hr to 21.7 mg/hp-hr with an average of 27.2 mg/hp-hr. The
COV was 11.4 percent when cycles for all three PEMS are included. The COV was 7.2, 6.0, and
4.8 percent for cycles from PEMS1, 2, and 3, respectively.
While processing some of the AVL data and comparing to the lab, it was discovered that
the incorrect CVS BSPM value had been used for one of the cycles. The value of 31.9 mg/hp-hr
from cycle 1956 had also been used for cycle 1965 instead of the correct value of 28.4 mg/hp-hr.
The entire set of transient CVS data was scrutinized again and no additional errors were found.
This erroneous correction factor was included in the final transient error surface included in the
model. The transient error surface was found to shrink for all PEMS by between 1 and 2 percent
with t he c Direction a pplied. G iven t he s mall c hange i n t he out come no a ction w as t aken t o
correct thi s mistake. The transient error surfaces presented i n this report are those us ed i n the
Monte Carlo model.
Figure 64, Figure 65, and Figure 66 show the concentration deltas forHoriba, Sensors,
and AVL, respectively. In the transient testing there was no lab reference to create deltas. Rather
the delta was calculated from the 50th percentile of the lab data as follows:
Deltoij = CF; * mPMy - 50th;
Where mPMy is the average flow-weighted PM concentration for the jth repeat of the ith NTE
event. 50* ; is the 50* percentile of mPM for the i* NTE event. CF; is the previously described
correction factor to adjust for engine variability.
REPORT 03.14936.12
81 of 174
-------
PM Concentration Delta (ng/mol)
1 nn
J.UU
en
DU
n
en
-DU
1 nn
-J.UU
1 en
AAAAAAAA^**^ A 4
*W~*^; , ^ ^
0
100
200
300
400
500
Median Flow-Weighted PM Concentration (ng/mol)
5th-50th Percentile
95th-50th Percentile
FIGURE 64. HORIBA CONCENTRATION DELTAS FOR TRANSIENT ENGINE
TESTING
PM Concentration Delta (ng/mol)
J. -JU
1 nn
en
DU
en
1 nn
J.UU
1 c.n
A
4 I/'
A A A
I I I I I
* * *
0 100 200 300 400 500
Median Flow-Weighted PM Concentration (ng/mol)
* 5th-50th Percentile A 95th-50th Percentile
FIGURE 65. SENSORS CONCENTRATION DELTAS FOR TRANSIENT ENGINE
TESTING
REPORT 03.14936.12
82 of 174
-------
o
M
(0
Q
c
o
"+ป
(0
QJ
u
O
U
150
100
-100
-150
0
100 200 300 400 500
Median Flow-Weighted PM Concentration (ng/mol)
* 5th-50th Percentile A 95th-50th Percentile
FIGURE 66. AVL CONCENTRATION DELTAS FOR TRANSIENT ENGINE TESTING
The 5th and 95th percentiles of the Horiba and AVL PEMS were bounded by plus and
minus 50 jig/mol. T he Sensors P EMS ha d s lightly 1 arger de Itas e xtending j ust be yond 100
jig/mol for the 95* percentile and -150 jig/mol for the 5* percentile. The bias from the data was
removed by subtracting the 50th from the 5th and 95th percentiles.
The steady-state error surface was designed to quantify the accuracy of the PEMS while
the t ransient e rror s urface w as de signed t o qua ntify t he pr ecision of t he P EMS. H owever, a
portion of the PEMS precision error is inherently captured in the steady-state error surface. At
the meeting on J uly 15th, 2009 i n Indianapolis four different approaches were presented to the
steering c ommittee f or r emoving the steady-state contribution t o t he pr ecision e rror from t he
transient er ror s urface. T he s teering com mittee el ected to proceed w ith a pproach 3; f or
simplicity the other three approaches will not be described.
In this approach, the data from all three PEMS from the same manufacturer were pooled
together (ie Sensorsl, Sensors2, and SensorsS). This resulted in a total of 18 steady-state data
points and 90 transient data points. A median and a MAD value were calculated for each of the
108 data points and used to calculate a MAD relative error, transient effect, root mean squared or
MADre;tr,rms- The MADre;tr,rms is defined as follows:
REPORT 03.14936.12
83 of 174
-------
re.tr.rms
N
Mediant
MAD
SSii
N L-i Median^,-
. \ ii,i,
Where tr denotes transient and ss denotes steady-state. Using an rms value eliminates the need
for estimating the steady-state variability at each of 90 t ransient data points. The error surface
was defined as the 90 percent confidence interval around zero (or no bias) as:
5th Percentile = -1.65 PM MADretrrms
95th Percentile = 1.65 PM MADreitrirms
Effectively the error surface is a straight line with a slope of +/- 1.65-MADre,tr,ims and an
intercept of z ero. The transient error surface is shown in Figure 67 forHoriba, Figure 68 for
Sensors, and Figure 69 for AVL.
120
o
,ฃ
bo
ro
4->
OJ
Q
c
o
4->
ra
4-1
C
Ol
u
c
o
u
80
40 -
-40 -
-80
-120
0
100 200 300 400
Median Flow Weighted PM Concentration (ng/mol)
500
^1st Percentile ^99th Percentile
FIGURE 67. FINAL HORIBA TRANSIENT PM ERROR SURFACE
REPORT 03.14936.12
84 of 174
-------
o
E
ro
4-i
-------
The median flow weighted PM concentration on the x-axis of each graph is extended to
the highest measured concentration observed by any of the PEMS during transient testing. The
slope of the transient PM error surface is 0.083 (1.41) for Horiba, 0.228 (3.27) for Sensors, and
0.099 (1.66) for AVL. The 1st percentile for the PEMS are -34, -92, and -40 jig/molatthe
maximum PM concentrations observed by each of the PEMS during steady-state testing (around
400 jig/mol) for the Horiba, Sensors, and AVL systems. The 1st percentiles during steady-state
testing at the samePM level were -177, -366, and -110 jig/mol for the Horiba, Sensors, and
AVL, respectively. Because the steady-state error surface was much larger than the transient it
tended to dominate the model results as discussed later.
4.10 CE-CERT Mobile Lab Correlation
The t est pi an c ailed for t he v alidation of t he model t o be pe rformed b y the m obile
emissions 1 aboratory ( MEL) ope rated b y t he U niversity o f C alifornia a t R iverside Bourns
College of E ngineering C enter for E nvironmental R esearch and T echnology ( CE-CERT). T he
mobile laboratory consisted of a full-flow CVS system inside the trailer of a Class A truck. The
mobile lab was capable of measuring gaseous emissions and filter based PM. The mobile lab was
arrived at SwRI on April 9th, 2009 to compare brake-specific PM emissions and ensure that the
reference during in-use validation is similar to the reference used during laboratory testing. The
CE-CERT MEL was parked behind the SwRI test cell and an exhaust transfer line was fabricated
to allow the MEL to measure the full engine exhaust in the s ame w ay a s the SwRI test cell.
Because the CVS measurement technique requires the full flow of engine exhaust for emissions
measurement, the exhaust system was designed in such a way that the exhaust pipe could easily
be switched between the SwRI CVS and the MEL CVS using the same length and geometry of
exhaust pipe. Figure 70 shows the exhaust system used for the CE-CERT correlation.
i
^B ซ.
FIGURE 70. EXHAUST CONFIGURATION FOR CE-CERT CORRELATION^
REPORT 03.14936.12
86 of 174
-------
Prior t o t he s tart of t esting, t he M EL C VS a nd SwRI C VS hot h unde rwent a pr opane
check to ensure the sampling systems were operating at the correct flows. Further details of the
audits performed by CE-CERT on the MEL can be found in the CE-CERT report (reference?).
The MEL was set to the same CVS flow, secondary dilution ratio, and filter face velocity as the
SwRI C VS t o m ake t he m easurements as c lose as pos sible. O ne ke y factor t hat c ould not b e
controlled was the dilution air for the MEL CVS. The SwRI CVS was able to draw flow that is
conditioned and maintained between 20 and 30ฐC. The MEL CVS drew its dilution air from the
ambient which prevented the control of temperature and relative humidity of the dilution air.
Each sampling system was conditioned by sampling for 10 hours during steady-state DPF engine
operation a t a hi gh e xhaust t emperature. A ctive D PF r egeneration oc curred dur ing t his
conditioning period. A total of 16 s hort NTE transient cycles were conducted using each CVS
system. The short NTE transient cycle was a m odified version of the NTE transient cycle used
for official transient PEMS testing. The short cycle included 16 of the original 30 N TE events
and lasted 755 s econds compared to 2130 s econds for the full NTE cycle. Table 18 shows the
order of testing.
TABLE 18. TEST PROCEDURE FOR CE-CERT CORRELATION
Day 1
Day 2
Day 3
Day 4
Active
Regen
1
1
1
1
Cycle 1
SwRI
CE-CERT
SwRI
CE-CERT
Cycle 2
SwRI
CE-CERT
SwRI
CE-CERT
Cycle 3
SwRI
CE-CERT
SwRI
CE-CERT
Cycle 4
SwRI
CE-CERT
SwRI
CE-CERT
Active
Regen
1
1
1
1
Cycle 1
CE-CERT
SwRI
CE-CERT
SwRI
Cycle 2
CE-CERT
SwRI
CE-CERT
SwRI
Cycle 3
CE-CERT
SwRI
CE-CERT
SwRI
Cycle 4
CE-CERT
SwRI
CE-CERT
SwRI
A manually triggered active DPF regeneration was performed before the start of each set
of four test cycles to maintain a similar PM loading level on the DPF. Each day the test order
was switched so that the SwRI tests were first on days 1 and 3 and the CE-CERT tests were first
on days 2 and 4. Each test was conducted as a hot-start with a 20 minute hot soak in between test
cycles or in between the DPF regeneration and the first test cycle. Figure 71 shows the brake-
specific PM results from the 16 cycles.
The data from test 9 for CE-CERT was removed due to a filter weight that was deemed to
be an outlier. The average SwRI BSPM was 0.0287 g/hp-hr with a COV of 5.2 percent based on
16 repeats. The average CE-CERT BSPM was 0.0265 g/hp-hr with a COV of 3.5 percent based
on 15 r epeats. The reported CE-CERT emissions were on average 7.7 p ercent 1 ower than the
SwRI reported em issions. The average reported B SCO2 by CE-CERT was 2.6 p ercent lower
than average SwRI reported value. One possible source of discrepancy between the two systems
was the heat loss in the exhaust pipe prior to its entrance into the CVS. The SwRI system was
completely sheltered within the t est c ell, while the C E-CERT e xhaust pi pe w as pr otruding
outside in such a way that it was exposed to wind and ambient temperature effects. This could
have resulted in a higher thermophoretic deposition of particles inside the exhaust pipe resulting
in lower emissions for the CE-CERT system. However, the test plan stated that agreement within
ten percent was considered sufficient, so the correlation was considered complete and the issue
was not investigated further.
A series of tunnel blanks were measured from both systems over sample periods of 15,
30, and 60 minutes. Figure 72 shows the filter weight gains as a function of sample time.
REPORT 03.14936.12
87 of 174
-------
Onon
.U 3U
i_
.c
2" n n?^
.52
5 n n?n
Q.
u
ฃ^
~ n m c
u U.UlD
o>
Q.
LO
j n n 1 n
QJ U.U1U
CD
CQ n nn^
Onnn
1
[
)a
3
yi
5
7
i
9
1
D
1
a\
1
>2
3
1
5
1
7
1
D
9
ay 3
21
2.
3
i
i
2.
2"
D
7
a
2
/'
9
L
3
1
SwRI BCE-CERT
FIGURE 71. BRAKE-SPECIFIC PM RESULTS FROM CE-CERT CORRELATION
Filter Weight Gain (ng)
1 9
1 n
c
/i
M-
n
0
15
30 45
Sample Time (min)
60
75
*SwRI ICE-CERT
FIGURE 72. CVS FILTER WEIGHT GAIN DURING TUNNEL BLANKS
REPORT 03.14936.12
88 of 174
-------
The filter weight gains during tunnel blank operation were similar for the SwRI and CE-
CERT tunnels. The weight gains ranged from 1.9 to 12.6 jig a nd g enerally de creased a s t he
sampling time increased. The filter weight gains during the correlation testing were greater than
300 jig making any differences in the tunnel blanks insignificant.
4.11 Investigation ofDPF Regeneration
The steering committee requested that screening tests be performed on measurements of
active DPF regeneration. Although the majority of the time active DPF regeneration is excluded
from i n-use m easurement as t he r egulations ar e cur rently written there ar e cer tain situations
where regeneration could be included in a valid NTE event. In addition, if the Horiba system
were tot rigger filter s ampling dur ing D PF r egeneration t his w ould s till be i ncluded i n t heir
calibration factor. So even if all NTE events containing active DPF regeneration were considered
invalid, the measurement accuracy of the Horiba system during valid NTE events could still be
affected through the filter calibration. For this testing the engine was operated at a medium speed
medium load condition at steady-state and the DPF was allowed to accumulate PMuntil the
ECM aut omatically t riggered a r egeneration. The va Ives i n the b ypass w ere cl osed forcing a
large majority of the exhaust through the DPF although the bypass was not completely sealed. At
the poi nt w hen t he E CM i ndicated i t w as pr eparing for ana ctive r egeneration t he P EMS a nd
CVS filter w ere t riggered t o s ample. T he P EMS s ampled 40 s econds on, f ive s econds off
throughout t he r egeneration, w hile t he C VS filter s ampled c ontinuously. Table 19 lists the
brake-specific PM results of the PEMS and the CVS.
TABLE 19. PM EMISSIONS RESULTS FROM ACTIVE DPF REGENERATION
No. of Samples
Avg. BSPM, mg/hp-hr
CVS Filter
1
6.8
Horiba
36
4.6
Sensors
31
7.1
AVL
35
0.2
It i s 1 ikely t hat onl y a s mall por tion of t he e missions dur ing t he r egeneration e ven
included elemental carbon as shown by the fact that the AVL PEMS measured only three percent
of the emissions measured by the CVS filter. The Horiba and Sensors PEMS both measured
BSPM values on t he same order as the CVS filter although the e vent-by-event emissions were
much different. The EEPS also sampled continuously from the CVS through a long residence
time secondary dilution tunnel with a nominal secondary dilution ratio of 2. The EEPS number
concentration was converted to a mass concentration assuming spherical particles with a density
of 1 g/cm3. A comparison of the event-by-event emissions of the three PEMS and the EEPS are
shown in Figure 73.
REPORT 03.14936.12
89 of 174
-------
600
500 C
u
400
300
0)
I_
3
+ป
(Q
0)
Q.
200 ฃ
100
O
LL.
Q.
Q
0
0
20 40 60
Regeneration Time (%)
80
100
--Horiba -B-Sensors -*-AVL EEPS DPF Out Temp
FIGURE 73. BRAKE SPECIFIC PM EMISSIONS DURING ACTIVE REGENERATION
Although the average brake-specific PM value from the Sensors system was very close to
the lab value, it is unlikely that the Sensors instrument was properly capturing the behavior of the
regeneration. T his i s be cause i ts be havior w as i nsensitive t o t he a ctive regeneration r egion a s
shown i n F igure 73. F urthermore, t he i ndividual br ake-specific e missions s eemed t o be hi gh,
particularly during the first three events and the last two events, where no regeneration occurred.
The event-by-event emissions from the Horiba system had similar trend to the mass determined
using EEPS number-weighted size distribution measurement, assuming spherical particles with a
density o f 1 g/cm3. Comparing t he Horiba m easurements t o t he C VS filter a nd t he E EPS
indicate that it was more accurate than the other PEMS at measuring the regeneration emissions.
However, one aspect of the Horiba measurement that was not captured in this experiment, and
that is the change in sensitivity of the DCS particle instrument to different size particles. The
filter calibration constant for the regeneration event indicated that the DCS was approximately
15 times more sensitive to the particles emitted during the active regeneration compared to the
particles emitted during steady-state engine testing. This means that if a steady-state engine cycle
was sampled onto the s ame filter as the DPF regeneration the system would be much less
accurate. The Horiba system would over predict the emissions during the regeneration and under
predict t he e missions d uring nor mal e ngine op eration. T hese findings w ere pr esented t o t he
steering committee at the December 11th, 2008 meeting in San Antonio. Although this could be a
major issue in the accuracy of the measurement, due to budget limitation, the steering committee
did not add D PF r egeneration t esting t o t he pr ogram given t hat t he r egulated aspect of
regeneration during in-use testing was vague. No data from the DPF regeneration investigation
was included in the model. The AVL MSS was insensitive to regeneration events as shown in
Figure 73. This suggests that the majority of mass emitted during regeneration is likely to be
volatile materials that will not be detected by the MSS.
REPORT 03.14936.12
90 of 174
-------
4.12 Investigation of Storage and Release
At the same time as theDPF regeneration investigation, the steering committee also
requested that screening tests be performed in the area of storage and release of nanoparticles in
the aftertreatment system. It has commonly been observed that volatile material emitted by the
engine at low temperature will deposit on the DPF only to be released quickly when the engine
exhaust temperature climbs. Three different DPF loading conditions were tested:
Low idle: 650 rpm, 65 Nm
Medium idle: 1200 rpm, 135 Nm
High idle: 1800 rpm, 135 Nm.
Idling times of 20, 60, and 90 minutes were tested at each condition. Following the period
of low temperature loading, the engine was immediately brought to a high temperature condition
to promote the release of the stored particles. Each of the three idling conditions was tested with
two high temperature conditions:
Peak torque: 1200 rpm, 2170 Nm
Near rated power: 1800 rpm, 1425 Nm
These t ests, s hown i n F igure 74 were con ducted as a s creening exercise w ith
measurements b y t he EEPSto determine w hich combination produced t he 1 argest r elease of
nanoparticles to use for the PEMS testing. However none of the tested conditions resulted in any
significant p article e missions on a num ber or mass ba sis. Toe nsure t hat t he E EPS w as not
missing something that might have been captured by one of the PEMS, two tests were run with
the PEMS: low idle to peak torque and high idle to peak torque. Only the low idle test is shown
here because the results are very similar. The engine was allowed to i die for one hour before
going to peak torque for ten minutes; this process was repeated three times consecutively for a
cycle length of 3.5 hours. The PEMS only sampled during the peak torque portion of the cycle
with the same 40 seconds sample, five seconds off cycling used in the DPF regeneration study.
A spike of just over 2.0E6 particles/cm3 was observed during the transition from idle to
peak torque although this appears to be due to acceleration and a possible slight misalignment
between the dilution ratio and EEPS m easurement r ather t han a release of particles from the
aftertreatment. The spike was less than 1.0E6 for the second and third transitions to peak torque.
Figure 75 shows the brake-specific PM emissions measured by the three PEMS during the peak
torque portion of the storage and release cycle. All samples are taken during the three repeats of
the t en m inutes a t pe ak t orque a nd no e missions f rom t he i die por tion of t he c ycle a re
represented.
REPORT 03.14936.12 91 of 174
-------
2500
0 2000 4000 6000 8000 10000 12000
Time (sec)
Speed RPM
Torque N-m
EEPS
u
TO
a.
TO
+ป
0)
U
o
u
!_
0)
(Q
^
X
FIGURE 74. TOTAL EXHAUST NUMBER CONCENTRATION DURING STORAGE
AND RELEASE
-4.0
0
10
20 30
Sample Number
40
FIGURE 75. BRAKE-SPECIFIC PM EMISSIONS DURING STORAGE AND RELEASE
INVESTIGATION
REPORT 03.14936.12
92 of 174
-------
It is unclear why the Sensors were reporting emissions much higher than the other two
PEMS. A single CVS filter was sampled for the entire ten minutes at peak torque for each repeat
for a total of three filter measurements. The BSPM from the CVS filter was only between 0.3
and 0.4 m g/hp-hr for each of the three repeats. This was lower than any of the PEMS and an
order of magnitude lower than the Sensors PEMS. The three negative emissions were attributed
to t he s ame c rystal which was unusually noi sy during t he t est. A comparison of the average
BSPM values for the cycle is shown in Table 20.
TABLE 20. AVERAGE BRAKE-SPECIFIC EMISSIONS DURING STORAGE AND
RELEASE CYCLE
No. of Samples
Avg. BSPM, mg/hp-hr
CVS Filter
O
0.38
Horiba
39
0.54
Sensors
36
2.97
AVL
39
0.41
The effect of storage and release on the PEMS emissions was not cl ear because of the
inability to generate a s ignificant release of nanoparticles, but it was evi dent that investigating
this phenomenon using this experimental configuration was not worthwhile. These findings were
presented at t he D ecember 11*, 2008 m eeting i n S an A ntonio a nd t he s teering committee
declined t o pur sue a ny f urther w ork i n t his area. N o da ta f rom t he s torage and release
investigation was included in the model.
4.13 Engine Manufacturers Torque and Fuel Error Surfaces
The OEM supplied torque error surface was up dated from the gaseous PEMS program
and the OEM supplied BSFC error surface was replaced with a fuel flow error surface. Five
different engine manufacturers supplied data from 61 different engines. In addition data was used
from t he four engines t ested i n t he A CES pr ogram f or a t otal of 2,099 da ta poi nts from 65
engines. T he E CM t orque de Itas w ere nor malized b y t he m aximum E CM t orque. T he t orque
deltas are shown in Figure 76.
The e rrors as a p ercentage of t he m aximum E CM t orque a re r elatively constant
throughout t he e ntire m easured r ange i ndicating t hat t he e rror a s a pe rcentage of poi nt w ould
increase as t he abs olute t orque de creases. The much smaller da ta s et us ed for t he ga seous
measurement allowance program showed constant errors as a percentage of point rather than a
percentage of maximum. The plot of ECM fuel flow errors as a percentage of maximum ECM
fuel flow is shown in Figure 77.
REPORT 03.14936.12
93 of 174
-------
u
LLJ
X
ro
o
0)
u
-20%
0.0 0.2 0.4 0.6 0.8
ECM Torque/Max ECM Torque
FIGURE 76. OEM SUPPLIED TORQUE ERRORS
1.0
30%
0)
LL. -i/~>o/
U
LLJ
X
ro
0)
LL.
&
ro
20%
10%
0%
-10%
-20%
-30%
0.0 0.2 0.4 0.6 0.8
ECM Fuel/Max ECM Fuel
FIGURE 77. OEM SUPPLIED FUEL FLOW ERRORS
1.0
1.2
1.2
REPORT 03.14936.12
94 of 174
-------
The fuel flow errors were also constant in relation to max fuel flow rate above 20 percent
fuel flow. Since the fuel flow i s expected to remain well above 20 percent during NTE event
operation, the smaller errors at low fuel flows were not considered important. Although the OEM
supplied torque and fuel flow error surfaces were created as percentages of point for the gaseous
measurement al lowance, t hese s urfaces w ere generated as pe rcentages of m aximum f or t he
current program. The 1st, 50th, and 99th percentiles were calculated for torque and fuel flow and
sampled in the model using a normal distribution. The error surface deltas are shown in Table 21.
TABLE 21. OEM ERROR SURFACE DELTAS FOR TORQUE AND FUEL FLOW
Parameter
Torque
Fuel Flow
Percentiles
1st,
% Point
-7.6
-4.5
50th,
% Point
-1.7
-0.1
99th,
% Point
4.1
4.9
REPORT 03.14936.12
95 of 174
-------
5.0 ENVIRONMENTAL TESTING AND RESULTS
A s eries of t ests w ere conduc ted to characterize t he P EMS r esponse unde r i n-use
conditions such as changes in pressure, temperature, and humidity, as well as their response to
electromagnetic and radio frequency interference and shock and vibration.
Initially the te st pi an called for the EPA P M G enerator s ystem to be u sed as the P M
source for all e nvironmental t esting. T he P M generator i s c apable o f p reducing s oot, vol atile
hydrocarbons, and sulfuric acid to simulate the PM emitted from a diesel engine. However, after
the system was operated at SwRI it was discovered that the PM generator was too tall to fit into
the a Ititude c hamber. It was like ly th at it w ould not be pos sible to operate the P M generator
under atmospheric conditions while the output was subject to a constantly varying pressure. The
diffusion rate of the hydrocarbon vials is pressure dependant so it would not have been possible
to m aintain a c onstant PM s ource w hile v arying t he pr essure i nside of t he chamber. S wRI
proposed u sing a J ing mini-CAST s oot g enerator i n pi ace of t he P M g enerator. T he s oot
generator is only a fraction of the size and much easier to operate compared to the PM generator.
The steering committee agreed to allow the soot generator to be used for the altitude testing but
requested that the PM generator be used for the temperature and humidity testing.
It w as de cided to operate t he E lectromagnetic Interference / R adio Frequency
Interference ( EMI/RFI) and s hock a nd vi bration t esting ass creening. In t his c ase s creening
testing me ant tha t the PEMS w ere ope rated while s ampling z ero air to look for pot ential
problems. T he r esults f rom t he s creening t esting w ould t hen be pr esented t o t he s teering
committee to decide whether to proceed with official testing to generate an error surface. The
main motivation to conduct the EMI/RFI and vibration testing as screening was the result of the
finding of the gaseous measurement allowance program that in most cases a failure mode of the
PEMS was observed only as a malfunction of the system in which it could no longer operated. It
was not commonly observed that the accuracy of the PEMS was affected while it continued to
measure without detected problems.
SwRI's M echanical a nd M aterial E ngineering Division ( Division 18) performed the
environmental testing. T he altitude, temperature and humidity testing was performed by Rick
Pitman and MikeNegrete. The EMI/RFI testing was performed by David Smith and Herbert
Walker. T he s hock a nd vi bration t esting w as pe rformed b y D avid S mith, M ike N egrete, a nd
Mark Orlowski.
5.1 Reference Measurement Testing
An eight hour b aseline measurement was p erformed for comparison to the eight hour
environmental tests. Unlike the gaseous measurement allowance program it was not possible to
compare the accuracy of the PM-PEMS over the measurement period, only the variability. The
PM and soot generators provide a particle source, but the correct concentration of the source was
unknown.
REPORT 03.14936.12 96 of 174
-------
During the eight hours of testing the PM concentration and dilution ratio was cycled to allow a
more accurate assessment of the PEMS performance over a range of operating conditions. Three
PM levels and four dilution ratios were sampled for a total of 12 test conditions. Figure 78 shows
the schedule of target PM concentration and dilution ratio for one hour of environmental testing.
35
c
o
5 15
0)
M
S? 10
0
0)
0)
0)
M
- 1
0
0
10
20 30 40
Time (min)
50
60
Target DR
Target PM
FIGURE 78. TARGET DILUTION RATIO AND PM LEVEL PROFILE FOR
ENVIRONMENTAL TESTING
Each level was sampled at dilution ratios of 6, 12, 20, and 30 for the Horiba and Sensors
PEMS. T he A VL P EMS w as m aintained a t i ts c onstant di lution r atio of 5. T he P EMS w ere
maintained at a target dilution ratio for five minutes. With 35 seconds remaining, the sample
trigger w as ena bled for 30 seconds. T his a llowed f or f our m inutes a nd 25 s econds f or
stabilization, 30 seconds for sampling, and five seconds after sampling to ensure sampling on all
PEMS had stopped before the target dilution ratio was changed. The PEMS were cycled through
the four dilution ratios at a single PM concentration level before the process was repeated at the
next concentration level. It took one hour to cycle through each combination of dilution ratio and
PM concentration. This profile was repeated eight times for a total test time of eight hours. The
schematic showing the experimental setup can be seen in Figure 79.
REPORT 03.14936.12
97 of 174
-------
mini-CAST
soot
generator
Krypton-85
neutralizer
ฑ
Environmental Chamber
flow
dampening
volume
1
A\/| DfTC
AVL-rvtr
_ overflow
A *
Sensors
AVL-PEMS
Horiba
filter
overflow
orifice
pressure
control valve
PM-Generator,
Temp Test Only
FIGURE 79. EXPERIMENTAL SETUP FOR ENVIRONMENTAL TESTING
The particles from the soot generator passed through a Krypton-85 neutralizer to bring
the cha rge o f t he p articles t o a minimum B oltzmann distribution of c harge [ 7] to minimize
particle losses due to electric forces. A large volume was placed downstream of the neutralizer
to minimize the pressure fluctuations observed by the soot generator and also to smooth out any
changes in concentration from the generator. One of the AVL PEMS units was placed outside of
the environmental chamber upstream of the orifice and overflow so that it was isolated from the
chamber conditions. This PEMS served as a reference to verify that the soot concentration from
the ge nerator w as s table. T he A VL uni t w as chosen be cause i t c ould pr ovide a r eal t ime
measurement of the soot concentration. This was not a guarantee that the total PM concentration
from the generator was steady, but typically the volatile emissions from the generator would not
fluctuate significantly without some change in the soot concentration.
The mini-CAST soot generator was used as the particle source for the b aseline testing.
The number mean diameter was approximately 70 nm based on m easurement with the EEPS.
The three concentration levels were nominal concentration levels of 25, 75, and 125 jig/mol with
approximately 30 percent organic carbon based on the OC/EC measurement. Figures 80, 81, and
82 show the concentration measurements during the baseline testing for the Horiba, Sensors, and
AVL PEMS, respectively. Figure 83 s hows a comparison between the reference MSS and the
PEMS MSS which were measuring simultaneously during the baseline testing.
REPORT 03.14936.12
98 of 174
-------
0
20
40
60
80
100
Observation Number
FIGURE 80. HORIBA ENVIRONMENTAL BASELINE MEASUREMENTS
o
o
"P
ro
0)
u
c
o
u
0)
3
ro
0)
200
180
160
140
120
100
80
60
40
20
0
-20
0
20
40 60
Observation Number
80
100
FIGURE 81. SENSORS ENVIRONMENTAL BASELINE MEASUREMENTS
REPORT 03.14936.12
99 of 174
-------
o
00
c
.0
'?
ro
+j
0)
u
o
u
CL
D
0)
(Q
0)
200
180
160
0
20
40 60
Observation Number
80
100
FIGURE 82. AVL ENVIRONMENTAL BASELINE MEASUREMENTS
_,
"o
^
M
^
C
O
ro
c
0)
u
0
u
s
Q.
0)
(Q
ni
140
120
100
80
60
40
20
0
0
20
40 60
Observation Number
80
100
--RefMSS -*-PEMSMSS
FIGURE 83. REFERENCE MSS ENVIRONMENTAL BASELINE MEASUREMENTS
REPORT 03.14936.12
100 of 174
-------
Although t he s oot c oncentration from t he generator w as 1 ikely s table based on t he
measurements from both AVL PEMS it is not clear whether the total PM concentration remained
constant throughout the test. In Figure 80, the Horiba PEMS exhibited a clear downwards trend
in measured concentration as t he t est pr ogressed. The reported PM cone entrati on from the
Sensors P EMS inF igure 81 appeared to be s omewhati ndependent of t he act ual P M
concentration sampled. The Sensors data was too scattered to either confirm or disprove the PM
trend obs erved i n t he Horiba da ta. In a ddition t o t he general dow nwards t rend of t he
concentration, the Horiba data appeared to suggest that the accuracy of the di lution ratio was
playing a role i n the m easurement. In obs ervations 25 t hrough 96, t he reported c oncentration
decreased each time the dilution ratio target increased. This suggests that the Horiba PEMS is
has either a positive error on lower dilution ratios or a negative error on higher dilution ratios.
In Figure 83, it is clear that the PEMS and reference AVL units were both able to resolve
the di fferences be tween t he t hree P M1 evels cl early even showing s imilar r esponses t o small
changes i n concentration. Figure 84 depicts t he r elationship between t he A VL r eference
measurement and the AVL PEMS measurement.
120 t 1 T r r
1 100 y = 0.7762x
"So R2 = 0.9991
3
ง 80
"P
ro
i 60
0)
u
& 40
in
ฑ 20 ;
ง
0
0 20 40 60 80 100 120 140
AVL Reference Concentration (ng/mol)
FIGURE 84. REFERENCE AVL VERSUS PEMS AVL FOR ENVIRONMENTAL
BASELINE
The correlation coefficient between the two measurements is excellent, better than 0.99.
The slope of 0.78 is likely to be a combination of a difference in response of the two instruments
and line losses between the two points of measurement.
REPORT 03.14936.12 101 of 174
-------
5.2 Pressure Chamber Testing
The SwRI altitude chamber is capable of simulating altitudes up to 19.8 km. The chamber
is 1.5 m in diameter and 2.1 m tall. Typically the simulated altitude chamber at SwRI is operated
only unde r v acuum t o s imulate a Ititudes g reater t han t hat of S an A ntonio. T he g aseous
measurement allowance test plan called for pressures up to 101.87 kP a or 45 m eters below sea
level. T he e levation of S an A ntonio i s a pproximately 240 m eters a bove s ea 1 evel w ith a
barometric pressure near 99 (98.4 based on altitude) kPa. In the gaseous measurement allowance
program, the altitude chamber underwent significant alterations to achieve positive pressures and
still m any p roblems w ere en countered. The S wRI engineer i n cha rge o f t he altitude ch amber
requested that only negative pressures be tested to preserve the integrity of their test equipment.
Simply changing the positive pressures to ambient would have resulted in a large portion of the
testing being conducted at normal atmospheric pressure. Instead a s light ne gative pr essure of
94.3 kPa (610 m, 2,000 ft) was repeated twice, once at 1.8 hours and once at 7.2 hours. Figure 85
shows the original pressure profile from the test plan as well as the modified profile that was
used during testing.
110
-800
0
345
Time (hr)
Original -^Revised
FIGURE 85. ORIGINAL AND REVISED PROFILE FOR ALTITUDE TESTING
Significant efforts were devoted to ensuring that a stable PM source could be generated
that was insensitive to pressure. Because the soot generator contains an open flame operating at
atmospheric pressure, the properties of the flame, and hence the particle generation, tended to
change with the pressure of the outlet. By placing an orifice in the transfer line between the soot
generator and the pr essure cha mber i t w as pos sible t o operate t he s oot g enerator at a hi gher
pressure and maintain a constant pressure through adjustment of an overflow valve upstream of
the orifice. Adjustments were only necessary during the pressure ramps to 82.7 kP a and 90.0
kPa. The valve was adjusted to maintain a constant pressure upstream of the orifice. Figure 86
shows the setup outside of the altitude chamber.
REPORT 03.14936.12
102 of 174
-------
FIGURE 86. ALTITUDE TESTING CHAMBER
During the first practice run, the AVL 3 blew a fuse in its measurement unit. The test was
stopped and the fuse was replaced, but immediately blew again indicating an electrical failure
within the unit. The unit was replaced with AVL 2 and testing continued with the problem not
observed again. Due to space constraints, the Semtech DS and Horiba OBS-2200 gaseous PEMS
were located outside of the environmental chamber. Since the purpose of these devices was only
communications, it was not considered necessary to test them inside the chamber. The external
compressor us ed for t he H oriba di lution a ir w as a Iso i nstalled out side the c hamber. A1 arge
compressor was supplied by S wRI to provide oil-less dilution air. The steering committee had
requested t hat t he H oriba s upplied c ompressor s hould be us ed but then a greed t o allow t he
replacement compressor after it was determined that the original compressor could not operate
for t he e ntire e ight hour s w ithout s hutting off. A pi cture of t he P EMS installed i n t he a Ititude
chamber is shown in Figure 87.
REPORT 03.14936.12
103 of 174
-------
FIGURE 87. PEMS INSTALLED IN THE ALTITUDE CHAMBER
As mentioned previously, analysis of the environmental data was more difficult for the
PM-PEMS pr ogram c ompared t o t he gaseous PEMS pr ogram du e t o t he 1 ack of a know n
reference c oncentration. F or t his r eason, onl y the va riability o f t he P EMS m easurement i n
comparison to its average was compared. The individual data points were plotted as well as the
average levels to show how far each measurement deviated from the average. Figure 88 shows
the concentration measurements by the Horiba.
140 n r 100
re
CL
-------
The Horiba concentration measurements were not consistent for different dilution ratios.
In the baseline test it was observed that increasing the dilution ratio in the range of 6, 12, 20, and
30 t ended t o de crease t he r eported c oncentration. In t he pr essure t est, it appe ars t hat t he
measured concentration increases as the dilution ratio increases. The largest deviations from the
average occurred during observations 53-60 while the pressure was at its lowest indicating that
the a mbient pr essure doe s ha ve s ome ef feet on the accur acy of the H oriba r eported
concentration. D uring the period b etween observations 53 a nd 60, t he system was unable to
maintain its target total flow rate of 30 slpm, as shown in Figure 89.
0
0
20 40 60
Observation Number
80
- 82
80
100
CD
0.
ฃ1
3
in
in
0)
Total Flow
Dilution Ratio
Chamber Pressure
FIGURE 89. HORIBA TOTAL FLOW AND DILUTION RATIO DURING PRESSURE
TESTING
The total flow dropped to approximately 28.7 slpm during this period which is below the
acceptable t olerance a ccording t o t he m anufacturer, causing t he da ta t o be i nvalidated. T he
system was however able to maintain proportionality during this part of the test. The short spikes
in the total flow is from each switch from sample to bypass mode.
The S ensors data, asshown i n F igure 90, e xhibited a 1 arge amount of variability with
some data m ore than a factor of two hi gher and lower than the average. With that amount of
scatter it was difficult to visually discern an effect of pressure on the measurement.
The AVL data, as shown in Figure 91, was grouped tightly around the average except
during t he 1 ow pr essure e xcursion a round observances 53 -60. Because of t he ex cellent
repeatability of t he m easurement, the e ffect of pr essure on the m easurement w as r eadily
apparent. As the ambient pressure decreased, the measurement decreased as well.
REPORT 03.14936.12
105 of 174
-------
o
M
O
?
CD
01
U
c
o
u
O.
250
200
150
100
50
0
100
- 98
96
92
re
o.
94 =
88
3
in
10
0)
0)
.Q
re
0
20 40 60
Observation Number
80
100
Avg Sensors Sensors ^Pressure
FIGURE 90. SENSORS ENVIRONMENTAL PRESSURE MEASUREMENTS
100
CD
0.
(U
in
10
(U
(U
.0
CD
80
0
20 40 60
Observation Number
80
100
AvgAVL AVL Pressure
FIGURE 91. AVL ENVIRONMENTAL PRESSURE MEASUREMENTS
REPORT 03.14936.12
106 of 174
-------
To determine w hether a n error surface would be generated for each PEMS, the MAD
from the baseline was compared to the MAD of the pressure test. If the MAD of the pressure test
was greater, an error surface was generated to present to the steering committee. The plots of
median versus MAD shown in Figures 92, 93, and 94 were presented to the steering committee
at the meeting in Indianapolis on July 15*, 2009.
4.0
00
Q.
3.5
3.0
2.5
2.0
1.5
1.0
0.5
0.0
' = 0.047x + 0.539
R2 = 0.413
' = 0.022x-0.124
R2 = 0.999
0 20 40 60 80 100
Median PM Concentration (ng/mol)
AVL-Baseline AVL-Pressure
FIGURE 92. AVL PRESSURE MEDIAN VERSUS MAD
120
o
M
Q.
Q
<
^
J.O.U
16.0
14 0 -
1 ~) n
1Z.U
1 n n
J.U.U
8n
6n -
A n
2n
.u
n n
y- 0.073x + 5.936
R2- 0.427
y - O.lzlX-4.006
R2 = 0.991
^^f*^^
V
20 40 60 80
Median PM Concentration (|ig/mol)
100
120
Horiba-Baseline * Horiba-Pressure
FIGURE 93. HORIBA PRESSURE MEDIAN VERSUS MAD
REPORT 03.14936.12
107 of 174
-------
35.0
).0
ol 15.0
o
i 10.0
5.0
0.0
y = 0.155x+15.834
R2 = 0.756
= 0.111x+15.755
R2 = 0.609
0
20 40 60 80 100 120
Median PM Concentration (ng/mol)
140
Sensors-Baseline Sensors-Pressure
FIGURE 94. SENSORS PRESSURE MEDIAN VERSUS MAD
The M AD va lues were higher for the baseline compared to the pressure test for both
Horiba a nd S ensors i ndicating t hat no a dditional va riability in the me asurement c ould be
attributed toe hanges i n pr essure us ing t his t echnique. N o environmental e rror s urface w as
calculated for either Horiba or Sensors. The AVL MAD values were higher for the pressure test
compared t o t he ba seline, s o an e rror s urface w as de veloped. It s hould be not ed t hat t he
variability of the AVL is much lower for the AVL compared to the Horiba and Sensors, however
because of the high precision of the AVL measurements during the baseline it was still possible
to discern the added variability due to changes in ambient pressure. The environmental pressure
error surface was calculated using the same pooled rms technique that was used for the transient
error surface. Figure 95 shows the error surface that was generated for environmental pressure
on the AVL PM. The error surface was accepted for use by the steering committee during the
July 15th, 2009 meeting in Indianapolis.
REPORT 03.14936.12
108 of 174
-------
o
-------
100
o
'ฃ
3
cu
"J
_ro
oป
oc
10
0
345
Time (hr)
OriginalTemp ^Modified Temp
Original RH
Modified RH
FIGURE 96. TEMPERATURE AND HUMIDITY PROFILE FOR ENVIRONMENTAL
TESTING
The temperature chamber was unable to control the humidity under a temperature of 5ฐC
so t he hum idity was un controlled dur ing t he po rtion of t he cycle w here t he t emperature w as
between 2ฐC and 5ฐC. The moisture content was quite small during this portion of the cycle, so
this was not considered to be a significant issue. Figure 97 shows the PEMS in the temperature
and humidity chamber.
FIGURE 97. PEMS INSTALLED IN THE TEMPERATURE AND HUMIDITY
CHAMBER
REPORT 03.14936.12
110 of 174
-------
The c ompressor pi ctured w as pr ovided b y S wRI for us e dur ing environmental t esting.
The or iginal c ompressor s upplied b y H oriba was m uch s mailer, but a Iso ha d di fficulty
completing eight hours of operation without shutting off. The compressor was included in the
temperature chamber so that the dilution air for the Horiba system would be affected by the same
changes in temperature and humidity that the dilution air of the Sensors and AVL systems would
experience. The P M s ample w as t ransported into the cha mber us ing a he ated sample 1 ine
maintained at 60ฐC. The temperature w as set slightly above the maximum temperature of the
chamber so that a constant temperature in the sample could be maintained throughout the test.
There w as a s mall por tion of t he e nd of t he t ransfer 1 ine t hat w as n ot he ated, but i t w as
extensively i nsulated to minimize t emperature effects. The E PA P M ge nerator w as us ed in
conjunction with the soot generator in this work to provide the particle source. The experimental
setup was shown previously in Figure 79. The PM generator is shown in Figure 98.
FIGURE 98. THE PM GENERATOR INSTALLED OUTSIDE THE TEMPERATURE
AND HUMIDITY CHAMBER
The Horiba and Sensors gaseous PEMS were installed outside of the chamber along with
the PM generator. The PM generator is designed to add volatile hydrocarbons, sulfuric acid, and
water vapor to an elemental carbon source. An oxidation catalyst removed any volatile from the
soot generator before volatile was added from the diffusion vi al ovens. Unfortunately both the
sulfur oven and the syringes injecting water malfunctioned during testing and neither was able to
be quickly repaired. For the official temperature testing the particle source consisted of elemental
carbon from the mini-CAST and volatile hydrocarbon from the PM generator.
Figure 99 s hows t he i ndividual H oriba c oncentration m easurements along with a
comparison of t he average concentration measurement f or e ach P M level. T he ch amber
temperature is included for reference as well.
REPORT 03.14936.12
111 of 174
-------
200
u
01
a.
ฃ
01
.Q
ฃ
to
U
20
40
60
80
100
Observation Number
- AvgHoriba * Horiba ^Temperature
FIGURE 99. HORIBA ENVIRONMENTAL TEMPERATURE MEASUREMENTS
The Horiba data plotted in Figure 99 showed a significant amount of variability although
it is unc lear f rom thi s g raph whether the v ariability is di rectly r elated to the c hanging
temperature. For ex ample, them easurements be tween obs ervation 25 and 35 when the
temperature i s below 10ฐC tend to be below the cycle average. However, the second time the
temperature drops below 10ฐC around observation 70, t he measurements are above the cycle
average. Figure 100s hows t he c hanges i n filter di lution a ir a nd c hamber t emperature. T he
behavior of these variables cannot clearly explain why the PM concentration behaved the way it
did in Figure 99.
10
20
Filter Temp
40 60
Observation
Dilution Air Temp
80
Chamber Temp
100
FIGURE 100. HORIBA TEMPERATURES DURING ENVIRONMENTAL
TEMPERATURE TESTING
REPORT 03.14936.12
112 of 174
-------
Figure 101 shows t he i ndividual S ensor c oncentration m easurements as w ell a s t he
average concentration for each PM level. The temperature measurements for the Sensors PEMS
are shown in Figure 101. The Sensors data displayed a high degree of variation from the mean
with no c lear t rend r elated tot emperature. N o valid da ta w as captured f or t he 1 ast hour o f
operation because the S ensors PEMS was unable to maintain the crystals at a t emperature of
50ฐC when the chamber temperature was above 47ฐC.
250
20 40 60
Observation Number
80
100
Avg Sensors Sensors ^Temperature
FIGURE 101. SENSORS ENVIRONMENTAL TEMPERATURE MEASUREMENTS
The AVL data, shown in Figure 102, exhibited excellent repeatability in comparison to
the other two PEMS. Any temperature dependence by the AVL PEMS was extremely small.
160 60
- 50
40 5
30
a>
3
t-t
re
01
Q.
E
- 20
- 10
01
.a
E
re
u
20 40 60
Observation Number
80
100
Avg AVL AVL ^Temperature
FIGURE 102. AVL ENVIRONMENTAL TEMPERATURE MEASUREMENTS
REPORT 03.14936.12
113 of 174
-------
The temperature data was compared in the same manner as the pressure data with the
MAD values plotted in relation to the median, as shown in Figures 103, 104, and 105, for the
Horiba, Sensors, and the AVL PEMS, respectively. The MAD of the Horiba baseline is slightly
higher than during the temperature test meaning that an error surface was not generated in this
case. The Sensors and AVL MAD was higher for the baseline at lower concentrations, but higher
for the temperature data for higher concentrations. An attempt was made to calculate an error
surface for S ensors, but t he pool ed r ms t echnique r esulted i n a s lightly hi gher va lue f or t he
baseline indicating that an error surface was not necessary. The error surface calculated for AVL
using the same technique is shown in Figure 106. The PM median concentrations tested were
between 25 a nd 135 |i g/mol. It w as ne cessary toextend the e rror surface out t o ne arly 500
|ig/mol to encompass the range of concentrations encountered during engine testing. The steering
committee elected to cap the error surface at plus and minus 5.3 jig/mol because it was unclear
whether t he errors w ould c ontinue t o i ncrease outside of t he c oncentrations obs erved i n t he
temperature and humidity testing. Extending a straight line out to the median concentration of
481 |ig/mol would have resulted in a 5th percentile of 18.9 jig/mol.
00
0.
Q
18.0
16.0
14.0
12.0
1 n n
1U.U
8n
.u
6c>
.u
4n
.u
? n
n n
y-0.073x+5.936
p2_Q 427
y = 0.084x+3.106
R2 = 0.767
^^*
^^^^.^^
^ \*^^*
*^^
0 20 40 60 80 100 120
Median PM Concentration (ng/mol)
140
160
Horiba-Baseline * Horiba-Temperature
FIGURE 103. HORIBA TEMPERATURE MEDIAN VERSUS MAD
REPORT 03.14936.12
114 of 174
-------
35.0
Q.
Q
15.0
10.0
5.0
0.0
0
y = 0.155x+15.834
R2 = 0.756
y = 0.040x+22.144
R2 = 0.446
20
140
40 60 80 100 120
Median PM Concentration (ng/mol)
Sensors-Baseline Sensors-Temperature
FIGURE 104. SENSORS TEMPERATURE MEDIAN VERSUS MAD
160
o
a
2.5
2.0
1.5
0.5
0.0
0
y = 0.006x+1.072
R2 = 0.962
y = 0.022x-0.124
R2 = 0.999
20
140
160
40 60 80 100 120
Median PM Concentration (ng/mol)
AVL-Baseline A AVL-Temperature
FIGURE 105. HORIBA TEMPERATURE AND HUMIDITY MEDIAN VERSUS MAD
REPORT 03.14936.12
115 of 174
-------
The AVL temperature data is shown in Figure 106 including the individual and average
measurements as well as the chamber temperature.
12
o
1
OJ
Q
E
O
OJ
U
E
O
U
-4
-12
100 200 300 400
Median Flow Weighted PM Concentration (ng/mol)
500
IstPercentile
99th Percentile
FIGURE 106. FINAL ERROR SURFACE ENVIRONMENTAL TEMPERATURE AND
HUMIDITY AVL PM
5.4 Electromagnetic and Radio Frequency Interference Screening
The electromagnetic inference (EMI) and radio frequency interference (RFI) testing was
conducted a s a s creening e xercise w ithout t he us e of P M s ource. D uring t he gaseous
measurement allowance program the majority of the problems encountered caused a malfunction
of t he P EMS t o t he p oint w here i t w ould no 1 onger op erate. T he main pur pose of t he
environmental testing was not to test the durability of the PEMS, but to quantify any errors that
might occur during the operation of a PEMS that would cause a measurement error. The same
series of t ests from t he g aseous E MI/RFI t esting w ere c onducted a s a s eries of i ndividual
screening t ests us ing o nly FIEPA f iltered air. Based on the s creening results i t r emained a
possibility to conduct a full test cycle using a particle source to generate an error surface.
By providing filtered room air to the PEMS it w ould still has been possible to detect a
wide range of possible measurement accuracy issues, although some problems may only present
themselves when a particle sample is present. To shorten the test time, the PEMS were triggered
continuously s o tha t ti me in between te sts c ould be mini mized. This me ant th at the Horiba
system was continuously sampling on a filter, and the Sensors system was continuously sampling
on a crystal. The Horiba system was able to sample continuously for eight hours or more on the
filter since no particle source was loading on the filter. However, the Sensors system had a
maximum sample time that was adjusted so that it would cycle through the crystals every 120
seconds. This was desirable since it exercised the operation of all eight crystals however some
issues surfaced due to this testing technique. The problems included: both high voltage power
supplies turning on at the same time, two crystals sampling at the same time, no crystal sampling
REPORT 03.14936.12
116 of 174
-------
even t hough on e w as a vailable. E ach of t hese pr oblems oc curred i nfrequently, a nd i t w as
believed that the problems were a result of leaving the trigger on continuously for long periods of
time since none of these i ssues were obs erved during engine testing. These were functionality
issues that did not appear to have any impact on the accuracy of the measurement.
The EMI and RFI testing, shown in Figure 107, was conducted in a radiation chamber, as
shown in Figure 105, with walls covered with large cones of carbon impregnated foam designed
to absorb radiation and minimize reflections. F our standard S ociety of Automotive Engineers
(SAE) te sts w ere c onducted: B ulk C urrent Injection, Radiated Immunity, E lectrostatic
Discharge, and Conducted Transients.
FIGURE 107. AVL PEMS IN THE RADIATION CHAMBER FOR EMI AND RFI
TESTING
The S ensors a nd AVL PEMS w ere bot h de signed t o r un of f t he ve hide's 12 V olt
electrical s ystem s o bot h s ystems w ere pow ered us ing a 12 V a utomotive b artery t hat w as
continually charged using a 120 VAC charger supplied by Sensors. The Sensors PEMS was a
12V system so it connected directly to the battery. The AVL system is powered by 120 Volts 60
Hz AC power so a commercially available inverter was provided by AVL to convert the 12 volts
DC into 120 volts AC. The Horiba system was designed to operate using a generator, therefore it
was still powered using the 120 VAC wall outlets.
5.4.1 Bulk Current Injection
SAE test Jl 113/4 Immunity to Radiated Electromagnetic Fields - Bulk Current Injection
was performed to determine the effect of electromagnetic radiation on the electrical cables of the
PEMS. The specifications used are detailed in Region 2, Class B of the Jl 113/4 test protocol. A
calibrated current probe was place around the electrical cable and used to inject RF current into
the cable. Figure 108 shows a cable running through the bulk current injection probe.
REPORT 03.14936.12
117 of 174
-------
FIGURE 108. BULK CURRENT INJECTION PROBE
The probe was positioned at 120 mm, 450 mm, and 750 mm from the cable connector to
test the cable three times. For each test, the frequency of the current was stepped from 1 MHz to
400 MHz using the following step sizes.
1 MHz to 10 MHz - 1 MHz step size
10 MHz to 200 MHz - 10 MHz step size
200 MHz to 400 MHz - 20 MHz step size
These were the maximum allowed step sizes according to the SAE protocol. As with the
gaseous pr ogram a 5 second dwell time was used to ensure the electromagnetic field had
stabilized before s witching to the ne xt f requency. T he pr obe w as c alibrated t o de liver 60
milliamps of c urrent ass pecified in the te st p rocedure. Figure 109 s hows t he bul k c urrent
injection probe being used to test a cable on the Horiba PEMS. Figure 110 and Figure 111 show
the sensors and AVL bulk current injection setup, respectively.
FIGURE 109. HORIBA PEMS SETUP FOR BULK CURRENT INJECTION
118 of 174
REPORT 03.14936.12
-------
FIGURE 110. SENSORS SETUP DURING BULK CURRENT INJECTION TESTING
FIGURE 111. AVL SETUP DURING BULK CURRENT INJECTION TESTING
REPORT 03.14936.12
119 of 174
-------
The cables tested on the Horiba system were the sample line temperature, sample line
heater power, heated en closure heater power, sample trigger and exhaust flow signal, ethernet
connection from theDCStothee lectrical e nclosure, e thernet c onnection from the T RPM t o
2200, DCS 12 volt power, and two AC power cords. There were two main problems discovered
when the bulk current injection was applied to the cables of the Horiba PEMS. When the line
supplying power to the heater in the heated enclosure was probed, the reported total flow of the
system and started to rapidly fluctuate around the frequency of 10 M Hz. This frequency was
manually repeated after t he s weep and t he s ame be havior oc curred. Figure 112 shows t he
response of several of the Horiba signals to the current injection.
35 80
10 MHz During Sweep
10 MHz Repeated
70
60
50
- 40
30
20
h 10
0
u
o
0)
3
+ป
ro
0)
Q.
0
500
1000
Time (sec)
1500
2000
OB-PFSS_Total_l [L/min] OB-PFSS_Dil_l [L/min]
OB-PFSS_Filter_T [degC] OB-PFSS_Tot_T [degC]
FIGURE 112. HORIBA FLOW DISTURBANCE FROM BULK CURRENT INJECTION
The noise in the system appeared to have originated with several of the temperature and
pressure m easurements i ncluding t he filter face t emperature and total flow temperature s hown
here. The total flow temperature and pressures began to fluctuate causing fluctuations in the total
flow measurement, which in turn caused the dilution flow to fluctuate in an effort to maintain
proportionality. T his pr oblem w as onl y obs erved on t he c able s upplying he ater pow er t o t he
heated enclosure box.
The communications between the laptop and the PEMS was disrupted when the Ethernet
cables w ere pr obed. S everal t imes t he c ommunications dr opped out c ompletely. T he p roblem
was intermittent around 40 to 50 MHz and 140 to 160 MHz.
The final m ajor pr oblem ex perienced by t he Horiba s ystem dur ing B CI w as a 1 arge
amount of noise in the exhaust flow signal when the analog signal cable between the OBS-2200
and OBS-TRPM was probed. The exhaust flow is measured by the Horiba gaseous PEMS OBS-
2200. The OBS-2200 outputs the exhaust flow measurement as an analog voltage which is then
REPORT 03.14936.12
120 of 174
-------
read by the OBS-TRPM Over frequencies between 1 and 40 MHz the exhaust flow signal read as
high as 3,000 kg/hr on the OBS-TRPM while the measured value on the OBS-2200 was less than
10 kg/hr. The exhaust flow signal during the BCI sweep is shown in Figure 113.
3500
-500
0
200
400 600
Time (sec)
800
1000
OBS-TRPM
OBS-2200
FIGURE 113. HORIBA EXHAUST FLOW NOISE ON ANALOG CABLE DURING
BULK CURRENT INJECTION
It was clear that the problem involved only the transmission of the exhaust flow signal as
an analog voltage between the two systems, not the actual measurement of the exhaust flow. For
in use testing this problem would be easily detectable when the two exhaust flow data from the
two different files are compared; however, the TRPM relies on the analog exhaust flow signal for
adjusting i ts di lution r atio for pr oportional s ampling. E rrors i n the exhaust flow s ignal w ould
cause the TRPM to lose its proportionality, and therefore any data collected with the same filter
sampling would be voided.
The cables tested on the Sensors system were the communication cable from the PPMD
to DS, and the DC power. As mentioned previously, many of the problems that were encountered
with t he S ensors P EMS ha ppened i ntermittently a nd c ould not be reproduced a t any s pecific
frequencies. It was assumed that problems involving crystal sampling switching, high voltage
power s upplies, a nd fluctuations i n t he c orona current were not i nduced b y t he bul k c urrent
injection. H owever, t he S ensors s ystem di d e xperience s ignificant c ommunication pr oblems
when the serial cable between the PPMD and DS units was probed. The communication was
repeatedly disrupted over the entire frequency range causing a loss of data. In nearly every case
the c ommunication w as r ecovered one e t he r adiation s ubsided a llowing nor mal ope ration t o
continue.
REPORT 03.14936.12
121 of 174
-------
The cabl es t ested on the A VL s ystem w ere t he com munications cabl e be tween the
measuring and conditioning unit, the analog output to the Semtech DS, and two AC power cords.
The BCI induced a significant amount of noise on the analog signal cable between the AVL MSS
and the Sensors Semtech DS. The AVL system can log its own data, but the data used officially
in this program and during in use testing is the data logged by the Semtech DS. Two signals are
carried on the analog output wire: the measured concentration and the dilution ratio. These two
numbers are multiplied together for the final reported concentration. The measured concentration
did not see significant noise, but the dilution ratio signal was strongly influenced by radiation in
the range of 1 -3 MHz and 40 MHz. A comparison of the MSS measured dilution ratio and the
dilution ratio recorded by the Semtech DS is shown in Figure 114.
12
10
.2 8
+ป
ro
DC
I 6
+ป
5 4
0
0
100
200 300
Time (sec)
400
500
Semtech DS Analog DR AVL MSS DR
FIGURE 114. BCI NOISE ON AVL ANALOG OUTPUT CABLE
The dilution ratio doubled from 5 to 10 during current injection at 1-3 MHz. The data in
the pi ot w as recorded d uring t he frequency s weep, a nd t he pr oblem w as r eplicated m anually
afterwards. S ince t he t rue di lution r atio w as s till r ecorded i n t he M SS 1 og file, t he c orrect
reported concentration could be recovered after testing.
5.4.2 Radiated Immunity
The r esponse of t he P EMS toe ontinuous na rrowband e lectromagnetic f ields w as
measured using S AE test Jl 113/21 Electromagnetic Compatibility Measurement Procedure for
Vehicle Components - Part 21: Immunity to Electromagnetic Fields, 10 kHz to 18 GHz,
Absorber-Lined Chamber. The ex act s pecifications of t he t ests pe rformed were dr awn from
Region 2, C lass B of t he J1113/21 pr otocol. Several di fferent a ntennae were us ed t o generate
electromagnetic radiation over the frequency range of 10 kHz to 1 GHz. The electromagnetic
REPORT 03.14936.12
122 of 174
-------
susceptibility experts at S wRI recommended ending the test at a frequency of 1 GHz rather than
18 GHz due to the very small probability of detecting any susceptibility above 1 GHz. This same
approach was used in the gaseous PEMS program. The carbon impregnated foam walls of the
radiation chamber was designed to absorb any radiation so that the PEMS would only see the
direct r adiation ge nerated b y t he a ntenna. T he f ollowing s tep s izes were us ed dur ing t he
frequency sweeps:
10 kHz to 100 kHz-10 kHz step size
100 kHz to 1 MHz - 100 kHz step size
1 MHz to 10 MHz - 1 MHz step size
10 MHz to 200 MHz - 2 MHz step size
200 MHz to 1 GHz - 20 MHz step size
The S AE s tandard field i ntensity o f 50 vol ts/meter was us ed w ith bot h ve rtical a nd
horizontal electromagnetic radiation orientations. Figure 115, Figure 116, and Figure 117 show
the Horiba, Sensors, and AVL radiated immunity testing setup, respectively.
FIGURE 115. HORIBA PEMS SETUP DURING RADIATED IMMUNITY TESTING
REPORT 03.14936.12
123 of 174
-------
FIGURE 116. SENSORS PEMS SETUP DURING RADIATED IMMUNITY TESTING
FIGURE 117. AVL PEMS SETUP DURING RADIATED IMMUNITY TESTING
REPORT 03.14936.12
124 of 174
-------
The HoribaP EMS experienced several problems related to the radiated immunity test
including a 1 oss of c ommunications between t he 1 aptop a nd t he i nstrument. T his pr oblem
occurred at several different frequencies for both horizontal and vertical polarizations. Because
the data is logged on the laptop any loss of communication is also a loss of data. This would
result in a voided test since the filter calibration relies on collecting data the entire time the filter
is sampling. A problem was also observed with the control of the dilution ratio, as shown in
Figure 118. During EMI and RFI testing the Horiba dilution ratio was maintained constant at 6.
ro
DC
C
o
50-70 MHz
Vertical
120-150 MHz
Vertical
0
500
1000 1500 2000 2500
3000
3500
Time (sec)
Measured Dilution Ratio
Setpoint Dilution Ratio
FIGURE 118. HORIBA DILUTION RATIO FLUCTUATIONS DURING RADIATED
IMMUNITY
The dilution ratio varied between 338 and -938 but only a smaller portion of the graph is
shown for more detail. The external flowmeter on the end of the sample flow indicated that these
rapid changes in flow were real and not just reported. The problem with the exhaust flow analog
output signal first encountered during the bulk current injection testing was also observed during
radiated immunity.
The AVL system reported a supply voltage error in the range of 200 MHz to 1 GHz of
horizontal r adiation. T he i nverter pow ered do wn a t 300 M Hz s hutting dow n t he s ystem
completely.
5.4.3 Electrostatic Discharge
SAE test J 1113/13 Electromagnetic Compatibility Measurement Procedure for Vehicle
ComponentsPart 13: Immunity to Electrostatic Discharge was performed to test t he P EMS
response t o Electrostatic D ischarges (ESDs) on surfaces and connectors. T he ex act pr ocedure
used was taken from Region 2, Class B of the Jl 113/13 standard. Approximately 40 ESDs were
REPORT 03.14936.12
125 of 174
-------
supplied to eachPEMS at various connector ports, exposed screws, and general surfaces. The
test included both direct contact di scharges as well as indirect di scharges with the electrostatic
discharge gun placed near the surface of interest. The BSD gun is shown in Figure 119.
FIGURE 119. ELECTROSTATIC DISCHARGE SIMULATOR
The E SD g un w as c alibrated us ing a n e lectrostatic vol tmeter t o de liver 4000 vol ts pe r
discharge. Neither the Horiba nor the AVL PEMS exhibited any susceptibility to the ESD tests.
The Sensors unit shut off completely with an ESD near the auxiliary connector port to which an
external flow m eter c onnects for a udit pur poses. T his pr oblem w as c onfirmed w ith a s econd
ESD. No issues possibly related to measurement accuracy were found with any of the PEMS.
5.4.4 Conducted Transients
The response of the PEMS to voltage disturbances in the 12 volt power supply cable was
checked using SAE test Jl 113/11 Immunity to Conducted Transients on Power Leads. The tests
were conducted us ing specifications found in Region 2, C lass B of the J1113/11 protocol. A
Schaffner NSG 5000 Automotive Electronics Test System was installed in between the 12 volt
power s upply a nd t he P EMS. T he S chaffner E lectronics T est S ystem de livered vol tage
perturbations to the PEMS of varying magnitudes and durations. The voltage spikes ranged from
-200 to 100 vol ts and 1 asted anywhere between 250 ns and 200 m s. The tests i ncluded qui ck
voltage recovery, slow voltage recovery, repeated voltage bursts, and load dump. The conducted
transients t ests w ere no t pe rformed on the H oriba P EMS, because t he H oriba s ystem was
intended for use with an external generator and does not use 12 volt power. The Schaffner Test
System can be seen in Figure 120 with the Sensors PEMS and Figure 121 for the AVL PEMS.
REPORT 03.14936.12
126 of 174
-------
FIGURE 120. SENSORS SETUP DURING CONDUCTED TRANSIENTS TESTING
FIGURE 121. AVL SETUP DURING CONDUCTED TRANSIENTS TESTING
The S ensors P EMS po wered dow n dur ing a -100 vol t s pike w ith qu ick r ecovery a nd
again powered down when the magnitude of the spike was reduced to -50 volts and then -25
volts. The slow recovery voltage spikes also caused the PEMS to power down with the shortest
dwell time of 40 ms causing the unit to shut down. Longer dwell times were not tested.
The quick recovery voltage spike caused the AVL inverter to power down, although it
appeared t o be w orking pr operly w hen i t w as restarted. H owever, dur ing t he s low r ecovery
voltage spikes, the inverter shut down again, started smoking and stopped working. The test was
not repeated with another inverter to prevent further damage.
REPORT 03.14936.12
127 of 174
-------
The findings from the EMI and RFI testing were presented to the steering committee
during the May 20th, 2009 i n San Antonio. The steering committee declined to pursue further
testing to develop any EMI and RFI error surfaces. The steering committee requested that Horiba
investigate the i ssue of bul k current i njection noi se i n the analog ex haust flow cabl e and that
Sensors investigate the issue of exhaust flow errors during radiated immunity testing.
5.5 Vibration Testing
Figures 122, 123, 124, 1 25, 126, and 127, show the setup for Sensors vibrational testing
using di fferent c onfigurations. The vi bration testing w as condu cted as a s creening ex ercise
similar to the method used for the EMI and RFI testing. The PEMS were operated on H EPA
filtered room air while being subjected to vibration. The response of the PEMS to the screening
exercise was used to determine whether further testing was needed to generate an error surface
for vibration. Each PEMS was mounted on an Unholtze-Dickie Shaker Table which was capable
of vibrating the PEMS on all three axes separately by adjusting the orientation of the PEMS. The
system used a large table to vibrate horizontally. By rotating the orientation of the PEMS, this
table w as a ble t o s imulate 1 ongitudinal a nd t ransverse hor izontal vi bration. T he s haker w as
rotated into a vertical position and a smaller platform was attached to provide vertical vibration.
Due toe onsiderations for non -road i n-use te sting, the s teering c ommittee r equested that th e
PEMS a Iso be r otated a t a 45 de gree i ncline one ach axis a s w ell. T he hor izontal vi bration
platform was large enough to include all of the pieces of any one type of PEMS at a time. The
vertical platform and 45 degree stand were each capable of installing only a single piece of the
instrumentation at a time.
Direction of vibration
FIGURE 122. SENSORS PEMS MOUNTED FOR TRANSVERSE HORIZONTAL
VIBRATION
REPORT 03.14936.12
128 of 174
-------
Direction of vibration
FIGURE 123. SENSORS PEMS MOUNTED FOR LONGITUDINAL HORIZONTAL
VIBRATION
Direction of vibration
FIGURE 124. SENSORS PEMS MOUNTED FOR TRANSVERSE 45ฐ VIBRATION
REPORT 03.14936.12
129 of 174
-------
Direction of vibration
FIGURE 125. SENSORS PEMS MOUNTED FOR LONGITUDINAL 45ฐ VIBRATION
Direction of vibration
FIGURE 126. SENSORS PEMS MOUNTED FOR VERTICAL VIBRATION
REPORT 03.14936.12
130 of 174
-------
Direction of vibration
FIGURE 127. SENSORS PEMS MOUNTED FOR 45ฐ VERTICAL VIBRATION
The pow er s pectral de nsity ( PSD) from t he M il S tandard 810, U S H ighway T ruck
Vibration Exposure was used in this testing. To prevent damage to the PEMS, the PSD was only
operated at 25%, 50%, and 75% of the energy specified in the Mil Standard 810. A fter a few
tests at the 75% level, the energy levels were reduced to 10%, 25%, and 50% to maintain the
integrity of t he i nstruments. In a ddition e ach e nergy 1 evel w as onl y t ested f or 5 m inutes t o
minimize the chances of damaging the instruments. Figure 128 shows the PSD used for vertical
vibration testing and Figure 130 shows the PSD used for horizontal testing.
Figure 130 shows t he Horiba P EMS vi bration pos itions for di fferent configurations.
Because the OBS-TRPM included three separate boxes, some of the vibration testing had to be
performed on i ndividual boxes. The DCS, HE, and MEEE were all able to fit on the horizontal
vibration table however only a single box could fit at a time on the 45 degree angle stand. The
DCS and MEEE were tested on this stand; the HE box was not tested at 45 degrees, because it
was felt that no errors would be detected in a box housing only a filter holder and a cyclone. A
particle source would need to be present to detect problems in this portion of the system. The
vertical vibration platform was able to hold both the DCS and the MEEE, so these two boxes
were tested simultaneously while the HE was again not tested. Any piece of the system that was
not being tested on the vibration stand was positioned directly next to the stand so that all flow
and electrical cables could still connect to the system normally.
REPORT 03.14936.12
131 of 174
-------
high-abort(f[
tow-aboft(f)
high-alarm(f)
ow alarrn(f)
control(f)
0.0010
0.0001
1.00E-05
10.00
100.00
Frequency (Hz)
500.00
Level: SO %
ConfrolRMS: 0.520509 gn Full Level Elapsed Time: 00:05:00
DemandRMS: 0.522418gn Remaining Time: 00:00:00
Lines: 400 Frame Time: 0.800000 Seconds
DOF: 154 dF: 1.250000 Hz
FIGURE 128. POWER SPECTRAL DENSITY FOR VERTICAL VIBRATION TESTING
0.0010
0.0001 -I
1.00E-05
1.58E-06
10.00
100.00
Frequency (Hz]
500.00
Level: 50 %
Control RMS: 0,370955 gn Full Level Elapsed Time: 00:00:00 Lines: 400 Frame Time: 0.800000 Seconds
DemandRMS: 0.370640gn Remaining Time: 00:05:00 DOF: 154 dF: 1.250000Hz
FIGURE 129. POWER SPECTRAL DENSITY FOR HORIZONTAL VIBRATION
TESTING
REPORT 03.14936.12
132 of 174
-------
FIGURE 130. HORIBA PEMS VIBRATION POSITIONS
The H oriba P EMS ex perienced a m echanical failure dur ing t esting at t he 75% energy
level of t he t ransverse horizontal vi bration. A bracket hoi ding a filter inside t he M echanical
Enclosure (ME) box broke causing a flow line to become blocked. The test was stopped and the
bracket w as r eplaced with one f rom a nother uni t. T he t esting was r esumed unt il i t w as
discovered t hat one o f the pr essure t ransducers us ed t o m easure t he flow w as not working
properly c ausing an error i n the reported flow. This problem was b elieved to have originated
from t he br oken br acket be cause t he 1 ine t hat w as c losed of f w as a ttached t o t his pr essure
transducer. The pressure transducer was replaced with one from the other unit and the system
functioned properly. All future tests were conducted at 10%, 25%, and 50% energy for all PEMS
to prevent further damage. The only other damage incurred by the Horiba was a rubber foot on
the bottom of one of the boxes was sheared off. The measured dilution flow exhibited a higher
degree of fluctuations d uring t he vi bration t esting t han w as nor mally observed although t he
differences were not significant.
The S ensors P EMS w as ope rated w ithout t he exhaust flow m eter and sample e Ibow
attached to the unit. As can be seen in Figure 127, the two pieces were removed to allow a HEPA
filter t o be pi aced on t he i nlet. A Ithough i t w ould ha ve be en de sirable t o 1 eave t hese pi eces
attached it was considered more i mportant t o provide clean air to ensure a zero particle level.
The Sensors PEMS experienced no mechanical failures during vibration testing although several
functionality i ssues di d oc cur. T he exhaust flow m easurement be came n oisier w ith vi bration
although the magnitude of the noise was relatively small. In the EMI and RFI testing values as
high as 1000 kg/hr were observed, but in vibration testing the exhaust flow never went above 50
kg/hr.
REPORT 03.14936.12
133 of 174
-------
The Sensors PEMS experienced problems maintaining its total flow rate during vibration
in the vertical direction. This problem was related to the automatic drain valves on the moisture
traps i n t he di lution flow 1 ine. T hese t raps a re designed t o ope n a utomatically w hen e nough
downwards force is applied from accumulated moisture in the reservoir; however the vertical
vibration was causing the valves to repeatedly open allowing a portion of the flow to escape.
Figure 131 shows an example of the total flow dropping while experiencing vertical vibration.
10
6
0
None
10%
25%
50%
None
200 400 600 800 1000 1200 1400 1600
Time (sec)
FIGURE 131. SENSORS TOTAL FLOW DURING VIBRATION TESTING
It w as pos sible t o t est both of t he A VL boxes s imultaneously for t he horizontal a nd
vertical vibration, however they were tested separately for the 45 degree angle vibration. Figure
132 shows the AVL PEMS vibration testing using different configurations.
No mechanical problems were encountered with the AVL unit during vibration testing
although the measurement did exhibit vibration induced noise. The measured soot concentration
fluctuated with increasing amplitude as the vibration increased for all orientations tested. The
peak m easurement r ecorded w as about pi us a nd m inus 5 ji g/mol w ith m ore t ypical va lues
swinging between plus and minus 2.5 jig/mol. A typical measurement is shown in Figure 133.
REPORT 03.14936.12
134 of 174
-------
FIGURE 132. AVL PEMS VIBRATION POSITIONS
200
400
800
1000
1200
600
Time (sec)
Concentration dil. corrected [mg/m3] 30 Sec Avg
FIGURE 133. AVL SOOT MEASUREMENT NOISE DURING VIBRATION TESTING
REPORT 03.14936.12
135 of 174
-------
This particular data was from the 45 degree angle longitudinal vibration. The noise in the
measurement has no bi as indicating that it will average out over larger periods of time. The 30
second average is s hown s ince t his i s 1 ikely t he m easurement error t hat w ould be obs erved
during an NTE event. The maximum error of the 30 second average was -0.009 mg/m3 which is
relatively i nsignificant b ased on P M c oncentrations e xpected a 11 he t hreshold. H owever, i t is
unclear from this testing whether the noise could be greater if a particle source was present. The
results from the vibration testing were presented to the steering committee at the September 22nd,
2009 meeting in Riverside. The steering committee declined to perform additional tests to create
an error surface for vibration since the A VL system was excluded from being an official PM-
PEMS.
REPORT 03.14936.12 136 of 174
-------
6.0 MODELING RESULTS
The main objective of this portion of the project was to use Monte Carlo techniques (e.g.
random sampling) in an error mode 1 to simulate the combined effects of a 11 the agreed-upon
sources of PEMS error incremental to lab error on the components of the brake-specific (BS) PM
emissions. This was accomplished by creating "error surfaces" for the Monte Carlo simulation
to sample, based upon the results of a variety of lab experiments. The constructed model was
simulated for thousands of trials (i.e., iterations) using data taken from a reference data set of 141
unique N TE e vents. T he m odel r esults w ere u sed t o de termine t he br ake-specific a dditive
measurement allowances for PM by three different calculation methods for three different PEMS
model units.
The error surfaces were generated from the results of each of the engine dynamometer
and e nvironmental chamber laboratory tests. T he engine-lab-test error s urfaces cove red the
domain of error versus the magnitude of the signal to which the error was to be applied (i.e., 1st
to 99 * percentile er ror vs. concentration, flow, t orque, e tc.). T he en vironmental-test e rror
surfaces f or s hock and vi bration, a nd electromagnetic a nd radio f requency i nterference
(EMI/RFI) was not included because no error surfaces were generated. The environmental test
error surfaces for pressure and temperature were characteristically different because they covered
the domain of the environmental-test cycle time versus the magnitude of the signal to which the
error was to be applied (i.e., error at a selected time vs. concentration).
6.1 Convergence Results from MC Runs
This section contains a summary of the checks to determine if the convergence criteria
were met for the simulation runs. Section 2.9 on Convergence and Number of Trials contains a
detailed description of the convergence methodology and the procedures followed to check for
convergence for the reference NTE event trials. This procedure was applied to the simulation
data obtained for each of the three PEMS units and all applicable calculation methods.
Figure 134 through Figure 140 contain plots of the 90% confidence interval widths at the
95* percentile delta differences (expressed as a p ercent of the BSPM emissions NTE threshold)
versus the ideal BSPM emissions for the 141 individual reference NTE events. This is done for
each of the three PEMS units and the applicable calculation methods. A summary of the results
is given in Table 22. Of interest was whether or not the simulations converged within 1% of the
threshold value. A scan be seen in the plots, a majority of the reference NTE events did not
converge within 1% of the BSPM threshold. However, a majority did converge within 2% of the
threshold. F or t he t hree P EMS uni ts a nd c alculation m ethods, t he m aximum pe rcent of t he
confidence i nterval w idths r anged from 1.82% f or t he A VL M ethod 11 o 2.76% f or S ensors
Method 1. U pon examination of the delta B SPM di stributions for the various reference NTE
events, t hose t hat ha d a hi gh pe rcentage a bove t he B SPM t hreshold a 11 he 95 th percentile
generally had low input PM concentration levels.
REPORT 03.14936.12 137 of 174
-------
AVL Method 1
2.00
0.00
0
0.01 0.02 0.03 0.04 0.05 0.06
Ideal BSPM,g/hp-hr
FIGURE 134. CONVERGENCE FOR AVL METHOD 1 AS A PERCENT OF BSPM
THRESHOLD
AVL Method 2
2.40
0.00
0
0.01 0.02 0.03 0.04
Ideal BSPM, g/hp-hr
0.05
0.06
FIGURE 135. CONVERGENCE FOR AVL METHOD 2 AS A PERCENT OF BSPM
THRESHOLD
REPORT 03.14936.12
138 of 174
-------
AVL Method 3
2
o
w
ฃ
CD
2.40
2.00
0.00
0
0.01 0.02 0.03 0.04 0.05 0.06
Ideal BSPM,g/hp-hr
FIGURE 136. CONVERGENCE FOR AVL METHOD 3 AS A PERCENT OF BSPM
THRESHOLD
Horiba Method 1
2.80
ฐ 2.00
j= 1.60
i-
ง 1.20
Q_
CO 0.80
0.40
0.00
*
0
0.01 0.02 0.03 0.04 0.05 0.06
Ideal BSPM, g/hp-hr
FIGURE 137. CONVERGENCE FOR HORIBA METHOD 1 AS A PERCENT OF BSPM
THRESHOLD
REPORT 03.14936.12
139 of 174
-------
Horiba Method 2
2.80
2.40
ฐ 2.00
w
| 1.60
I-
S 1.20
CL
DO 0.80
^
0.40
0.00
**'
0 0.01 0.02 0.03 0.04
Ideal BSPM,g/hp-hr
0.05
0.06
FIGURE 138. CONVERGENCE FOR HORIBA METHOD 2 AS A PERCENT OF BSPM
THRESHOLD
Sensors Method 1
3.20
2.80
| 2.40
| 2.00
H 1.60
0.40
0.00
**vป%
^*^T*4
--
* **
0 0.01 0.02 0.03 0.04
Ideal BSPM, g/hp-hr
0.05
0.06
FIGURE 139. CONVERGENCE FOR SENSORS METHOD 1 AS A PERCENT OF
BSPM THRESHOLD
REPORT 03.14936.12
140 of 174
-------
2.80
0.00
0
Sensors Method 2
0.01 0.02 0.03 0.04
Ideal BSPM,g/hp-hr
0.05
0.06
FIGURE 140. CONVERGENCE FOR SENSORS METHOD 2 AS A PERCENT OF
BSPM THRESHOLD
TABLE 22. SUMMARY OF NUMBER OF REFERENCE NTES MEETING 2%
CONVERGENCE
PEMS Unit
AVL
Horiba
Sensors
Method
1
2
3
1
2
1
2
Min
0.4467
0.4446
0.4240
0.4158
0.4414
0.7883
0.8613
Max
1.8211
1.9766
1.9215
2.5663
2.5447
2.7664
2.6544
No. NTEs within
2% Convergence
141
141
141
129
131
125
131
% NTEs within
2% Convergence
100%
100%
100%
91%
93%
87%
93%
REPORT 03.14936.12
141 of 174
-------
6.2 Sensitivity Based on Bias and Variance
This section contains a summary of the error surfaces that contributed the most to the bias
of the generated BS emissions. The sensitivity charts developed in Crystal Ball help identify the
error surfaces (assumptions) that are sensitive to changes in variation with respect to their effect
on the t hree de Ita BS emissions. A nother t ype of s ensitivity e xamined i n t his s tudy w as
concerned with the effects of potential "bias" in error surfaces and their effects on the forecast
values. In or der t o s tudy these e ffects a ne w e rror s urface a ssumption was a dded t o t he M C
Monte Carlo simulation model for each of the original 31 error surfaces.
This assumption was sampled as a discrete binary distribution (i.e., on or off) during the
simulation. For each trial of the simulation, 31 original error surfaces and 31 ' on/off error
surfaces w ere s ampled according tot heir de fined s ample di stribution. If t he 'on/off e rror
surface produced an 'off condition, the delta emissions from that particular error surface were
not added to the BS emissions computations for the BS emissions 'with errors'. Similarly, if the
'on/off error surface produced an ' on' condition, the delta emissions from that particular error
surface were added to the BS emissions calculations.
During every trial of the simulation, the exclusions due to the 'off conditions resulted in
various combinations of the error surface delta emissions being added to the BS emissions 'with
errors' c omputations. Over t he c ourse of a MC s imulation w ith t housands of t rials, t he
sensitivity of a particular error either 'on' or 'off was assessed by examining the change in the
forecast de Ita em ission. T herefore, in a s ingle M C s imulation of a reference N TE ev ent
sensitivities due to variance and/or bias were explored.
Simulation results from the reference NTE events produced sensitivity values for all 95th
percentile delta emissions by all three PEMS units and applicable calculation methods. Table 23
through Table 2 9 summarize t he e rror s urfaces i n w hich e ither t he c ontribution-to-variance
normalized sensitivity value or the ' on/off bias check for the error surface was at least 5% in
magnitude compared to all the other error surfaces. If the label in the error surface contains the
words ' Delta' then it represents a che ck for bi as; otherwise, the error surface indicates a ch eck
for variance. Table 23 through Table 25 lists the sensitivity and bias descriptive statistics for the
delta B SPM emissions for the AVL PEMS for Methods 1, 2 a nd 3, respectively. For all three
methods, the largest mean normalized variance was from the bias effect due to error surface #1,
SSPM.
REPORT 03.14936.12 142 of 174
-------
TABLE 23. ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 141
REFERENCE NTE EVENTS FOR AVL BSPM METHOD 1
Error
Surface
No.
1
2
20
31
35
Error Surface
SSPM
TRPM
SS Exhaust Flow
Torque Warm-up
Torque Engine
Manufacturer
Delta Exhaust
Flow Pulsation
Delta SS PM
No. Ref
NTE
Events
141
4
4
10
1
3
141
Avg. Contribution
to Normalized
Variance, %
11.34
7.00
5.57
-6.64
-6.27
5.78
-74.37
Min
Contribution,
%
7.59
6.42
5.32
-11.22
-6.27
5.56
-83.67
Max
Contribution,
%
35.25
7.34
6.05
-5.32
-6.27
5.99
-21.02
TABLE 24. ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 141
REFERENCE NTE EVENTS FOR AVL BSPM METHOD 2
Error
Surface
No.
1
2
31
35
45
Error Surface
SSPM
TRPM
Torque Warm-
up
Torque Engine
Manuf
SSCO2
Delta SS PM
No. Ref
NTE
Events
141
38
8
1
4
141
Avg. Contribution
to Normalized
Variance, %
21.48
9.20
-6.82
-6.12
-6.89
-63.27
Min
Contribution,
%
7.44
5.53
-10.77
-6.12
-7.82
-83.77
Max
Contribution,
%
60.56
12.31
-5.24
-6.12
-5.32
-20.95
TABLE 25. ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 141
REFERENCE NTE EVENTS FOR AVL BSPM METHOD 3
Error
Surface
No.
1
2
31
35
45
Error Surface
SSPM
TRPM
Torque Warm-
up
Torque Engine
Manuf
SSCO2
Delta SS PM
No. Ref
NTE
Events
141
36
9
1
4
141
Avg. Contribution
to Normalized
Variance, %
20.91
9.27
-6.62
-6.12
-6.96
-64.04
Min
Contribution,
%
7.71
5.17
-10.77
-6.12
-7.89
-83.76
Max
Contribution,
%
60.50
12.52
-5.14
-6.12
-5.40
-20.93
REPORT 03.14936.12
143 of 174
-------
Table 26 and Table 27 list the s ensitivity a nd bias descriptive statistics for the delta
BSPM emissions for the Horiba PEMS for Methods 1 and 2, respectively. For both methods, the
highest mean normalized variances were from the bias and variance due to error surface #1, SS
PM.
TABLE 26. ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 141
REFERENCE NTE EVENTS FOR HORIBA BSPM METHOD 1
Error
Surface
No.
1
2
20
31
35
Error Surface
SSPM
TRPM
SS Exhaust
Flow
Torque Warm-
up
Torque Engine
Manuf
Delta Exhaust
Flow Pulsation
Delta SS PM
No. Ref
NTE
Events
138
10
26
100
29
31
83
Avg. Contribution
to Normalized
Variance, %
41.86
5.84
8.94
-8.34
-5.72
6.67
-32.05
Min
Contribution,
%
6.65
5.04
5.00
-15.96
-8.07
5.07
-83.42
Max
Contribution,
%
80.39
7.18
7.80
-5.08
-5.00
10.01
73.96
TABLE 27. ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 141
REFERENCE NTE EVENTS FOR HORIBA BSPM METHOD 2
Error
Surface
No.
1
2
31
35
42
45
Error Surface
SSPM
TRPM
Torque Warm-
up
Torque Engine
Manuf
Fuel Rate
Engine Manuf
SSCO2
Delta SS PM
No. Ref
NTE
Events
138
10
95
24
7
54
89
Avg. Contribution
to Normalized
Variance, %
39.55
5.89
-8.19
-5.72
5.52
-6.60
-42.60
Min
Contribution,
%
6.99
5.00
-15.04
-7.65
5.07
-9.83
-83.42
Max
Contribution,
%
80.30
7.42
-5.03
-5.08
6.34
-5.09
75.48
REPORT 03.14936.12
144 of 174
-------
Table 28 and Table 29 list the s ensitivity a nd bias descriptive statistics for the delta
BSPM emissions for the Sensors PEMS for Methods 1 and 2, respectively. F or both methods,
the highest mean normalized variance was from the bias and variance due to error surface #1,
SSPM.
TABLE 28. ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 141
REFERENCE NTE EVENTS FOR SENSORS BSPM METHOD 1
Error
Surface
No.
1
2
20
31
35
Error Surface
SSPM
TRPM
SS Exhaust Flow
Torque Warm-up
Torque Engine
Manuf
Delta Exhaust
Flow Pulsation
Delta SS PM
No. Ref
NTE
Events
138
100
8
30
10
9
120
Avg. Contribution
to Normalized
Variance, %
46.92
8.48
6.11
-8.28
-5.60
7.01
-31.19
Min
Contribution,
%
6.57
5.00
8.06
-11.57
-6.06
5.56
-86.58
Max
Contribution,
%
78.61
12.31
7.56
-5.02
-5.14
8.79
75.35
TABLE 29. ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 141
REFERENCE NTE EVENTS FOR SENSORS BSPM METHOD 2
Error
Surface
No.
1
2
31
35
42
45
Error Surface
SSPM
TRPM
Torque Warm-up
Torque Engine
Manuf
Fuel Rate Engine
Manuf
SSCO2
Delta SS PM
No. Ref
NTE
Events
138
101
24
5
4
10
127
Avg. Contribution
to Normalized
Variance, %
45.92
8.48
-7.84
-5.61
6.18
-6.84
-28.96
Min
Contribution,
%
6.68
5.03
-12.01
-6.06
5.72
-9.77
-86.67
Max
Contribution,
%
77.78
12.52
-5.08
-5.01
6.53
-5.15
75.75
The c ontribution to normalized variance and bias s ensitivities from Table 23 through
Table 29 are illustrated pictorially as box plots in Figure 141 to Figure 147 for BSPM by PEMS
unit for Methods 1, 2 and 3. Only the error surfaces with at least 35 of the 141 reference NTE
events (1/4 of the events) are included as box plots. The mean normalized variance for each of
the pi otted e rror s urfaces i s not ed b y a " +" s ymbol i n t he box es. T he e rror s urface with t he
largest m ean normalized variance i s pi otted at the 1 eft of the chart. The error surface with the
second largest mean normalized variance is plotted second from the left, and so on. Figure 142
and Figure 143 demonstrate the high sensitivity to the negative bias for error surface #1, PM SS.
Figure Figurel46 and Figure 147 show a 1 arge variance effect due to PM S S. Table 3 0 and
Table 3 1 show a summary of the error surface sensitivity to bias and variance for the different
PEMS using Method 1 and Method 2. T able 32 shows a similar summary using Method 3 for
the AVL PEMS only.
REPORT 03.14936.12
145 of 174
-------
Sensitivity Contribution to Variance and Bias for BSPM AVL Method 1
50
25-
o
i
I
1
TB
-25
s
p -50
|
| -75
-100
01 ic SS PM
Delta PM SS
FIGURE 141. BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON BIAS
AND VARIANCE FOR AVL BSPM METHOD 1
Sensitivity Contribution to Variance and Bias for BSPM AVL Method 2
100
I
TB
E
o
JO
s
o
o
50
0
-50
-100
01 ic SS PM
02 ic TR PM
Delta PM SS
FIGURE 142. BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON BIAS
AND VARIANCE FOR AVL METHOD 2
REPORT 03.14936.12
146 of 174
-------
Sensitivity Contribution ID Variance and Bias for BSPM AVL Method 3
I
1
a
100
50
5 -50"
1:
o
o
-100
01 ic SS PM
02 ic TR PM
Delta PM SS
FIGURE 143. BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON BIAS
AND VARIANCE FOR AVL METHOD 3
Sensitivity Contribution to Variance and Bias for BSPM Horiba Method 1
100-
50-
ซ
o
I
I
i
E
o
I -501
ฃ
o
o
-100
T
01 ic SS PM
31_fc_Toique_Wami
Delta PM SS
FIGURE 144. BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON BIAS
AND VARIANCE FOR HORIBA BSPM METHOD 1
REPORT 03.14936.12
147 of 174
-------
Sensitivity Contribution to Variance and Bias for BSPM Horiba Method 2
o
ง
I
I
'
100
50
0
o
z
ฃ
o
a
=ง -50
1
O
-no
01_ic_SS_PM 31JObiqueJttrm
Delta PM SS
FIGURE 145. BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON BIAS
AND VARIANCE FOR HORIBA BSPM METHOD 2
Sensitivity Contribution to Variance and Bias for BSPM Sensors Method 1
s
s
I
13
E
o
?
=ง
s
o
o
50
o-
-50
-100
T
01 ic SS PM
02 ic TR PM
Delta PM SS
FIGURE 146. BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON BIAS
AND VARIANCE FOR SENSORS BSPM METHOD 1
REPORT 03.14936.12
148 of 174
-------
Sensitivity Contribution to Variance and Bias for BSPM Sensors Method 2
100
*
g
(3
1
"S
_N
1 o
z
2
*|
_o
3
=ง -50
s
o
o
-100
n
T Q
n
E
]
_._ n
T
H
1
t
I
\~
\
3
01 Jc SS_PM 02 Jc_TR_PM Delta PM
FIGURE147. BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON BIAS AND
VARIANCE FOR SENSORS BSPM METHOD 2
TABLE 30 SUMMARY OF ERROR SURFACE SENSITIVITIES TO BIAS AND
VARIANCE FOR BSPM METHOD 1
Method 1
Error
Surface
No.
1
1
20
31
35
Error Surface
SSPM
TRPM
SS Exhaust
Flow
Torque Warm-
up
Torque Engine
Manuf
Delta Exhaust
Flow Pulsation
Delta SS PM
AVL
No. NTE
Events
141
4
4
10
1
3
141
Avg
Contribution
to Normalized
Variance, %
11.34
7.00
5.57
-6.64
-6.27
5.78
-74.37
Ho rib a
No. NTE
Events
138
10
26
100
29
31
83
Avg
Contribution to
Normalized
Variance, %
41.86
5.84
8.94
-8.34
-5.72
6.67
-32.05
Sensors
No. NTE
Events
138
100
8
30
10
9
120
Avg
Contribution to
Normalized
Variance, %
46.92
8.48
6.11
-8.28
-5.60
7.01
-31.19
REPORT 03.14936.12
149 of 174
-------
TABLE 31. SUMMARY OF ERROR SURFACE SENSITIVE TO BIAS AND
VARIANCE FOR BSPM METHOD 2
Method 2
Error
Surface
No.
1
2
31
35
42
45
Error Surface
SSPM
TRPM
Torque Warm-
up
Torque Engine
Manuf
Fuel Rate
Engine Manuf
SSC02
Delta SS PM
AVL
No. NTE
Events
141
38
8
1
4
141
Avg
Contribution
to Normalized
Variance, %
21.48
9.20
-6.82
-6.12
-6.89
-63.27
Ho rib a
No. NTE
Events
138
10
95
24
7
54
89
Avg
Contribution to
Normalized
Variance, %
39.55
5.89
-8.19
-5.72
5.52
-6.60
-42.60
Sensors
No. NTE
Events
138
101
24
5
4
10
127
Avg
Contribution to
Normalized
Variance, %
45.92
8.48
-7.84
-5.61
6.18
-6.84
-28.96
TABLE 32. SUMMARY OF ERROR SURFACE SENSITIVE TO BIAS AND
VARIANCE FOR BSPM METHOD 3
Method 3
Error
Surface
No.
1
2
31
35
45
Error Surface
SSPM
TRPM
Torque Warm-up
Torque Engine
Manuf
SSCO2
Delta SS PM
AVL
No. NTE
Events
141
36
9
1
4
141
Avg Contribution to
Normalized
Variance, %
20.91
9.27
-6.62
-6.12
-6.96
-64.04
6.3 Validation Results
This section contains a summary of the model validation results; Section 0 on Validation
contains a m ore d etailed de scription of t he validation m ethodology ut ilized hot h i n t he
simulation and in the on-road data collection efforts.
During the Monte Carlo simulation of the 141 r eference NTE events some of the error
surfaces w ere e xcluded i n t he c omputation of t he B S e missions ' with e rrors' sot hat t he
simulation r epresented c onditions us ed i n c ollecting t he on -road data. T he er ror s urfaces
excluded were torque errors (Nos. 29-32, 34, 35), fuel rate engine manufacturers (#42), dynamic
REPORT 03.14936.12
150 of 174
-------
speed (#43) and dynamic fuel rate (#44). For each reference NTE event, the difference in BSPM
emissions was computed as
delta BSPM = BSPM with "Validation error" - "Ideal" BSPM.
These delta BSPM emissions were computed for each of the three P EMS units and all
applicable calculation methods. T he 5 th, 50th and 95 th percentiles w ere id entified from the
distributions of t he de Ita B SPM e missions dur ing t he M onte C arlo simulation us ing t he
validation error surfaces only. Figure 148 through Figure 150 depict the validation percentiles
for the AVL PEMS unit for methods 1, 2 and 3, respectively. Similar validation plots for the
Horiba PEMS unit are illustrated in Figure 151 and Figure 152 for methods 1 and 2, respectively.
Sensors PEMS validation plots for methods 1 and 2 are shown in Figure 153 through Figure 154.
0.005
0.000
si
Q.
s:
o> -0.005
s
0.
CQ -0.010
ฃ
-0.015
-0.020
0.00
0.01
0.02 0.03 0.04
Ideal BSPM, g/hp-hr
0.05
0.06
Validation AVL 1 5th DValidation AVL 1 50th
Validation AVL 1 95th
FIGURE 148. VALIDATION PERCENTILES FOR THE 141 REFERENCE NTE
EVENTS FOR AVL METHOD 1
REPORT 03.14936.12
151 of 174
-------
0.005
-0.020
0.00 0.01 0.02 0.03 0.04
Ideal BSPM, g/hp-hr
0.05
0.06
^Validation AVL 2 5th DValidation AVL 2 50th Validation AVL 2 95th
FIGURE 149. VALIDATION PERCENTILES FOR THE 141 REFERENCE NTE
EVENTS FOR AVL METHOD 2
0.005
0.000
o) -0.005
s"
Q.
m -0.010
0)
Q
-0.015
-0.020
*v..
0.00
0.01
0.02 0.03 0.04
Ideal BSPM, g/hp-hr
0.05
0.06
^Validation AVL 3 5th DValidation AVL 3 50th Validation AVL 3 95th
FIGURE 150. VALIDATION PERCENTILES FOR 141 REFERENCE NTE EVENTS
FOR AVL METHOD 3
REPORT 03.14936.12
152 of 174
-------
0.025
0.020
- 0.015
0.010
Q.
w
to
o
Q
-0.015
0.00 0.01 0.02 0.03 0.04
Ideal BSPM, g/hp-hr
0.05
0.06
Validation Horiba 1 5th DValidation Horiba 1 50th
Validation Horiba 1 95th
FIGURE 151. VALIDATION PERCENTILES FOR 141 REFERENCE NTE EVENTS
FOR HORIBA METHOD 1
0.00
0.01
0.02 0.03 0.04
Ideal BSPM, g/hp-hr
0.05
0.06
| Validation Horiba 2 5th DValidation Horiba 2 50th Validation Horiba 2 95th |
FIGURE 152. VALIDATION PERCENTILES FOR 141 REFERENCE NTE EVENTS
FOR HORIBA METHOD 2
REPORT 03.14936.12
153 of 174
-------
Q.
"oi
2
0.015
0.010
0.005
0.000
-0.005
-0.010
-0.015
-0.020
-0.025
-0.030
-0.035
0.00
0.01
0.02 0.03 0.04
Ideal BSPM, g/hp-hr
0.05
0.06
Validation Sensors 1 5th DValidation Sensors 1 50th Validation Sensors 1 95th
FIGURE 153. VALIDATION PERCENTILES FOR 141 REFERENCE NTE EVENTS
FOR SENSORS METHOD 1
Q.
.C
^)
Q.
V)
m
Qi
Q
0.00
0.01
0.02 0.03 0.04
Ideal BSPM, g/hp-hr
0.05
0.06
Validation Sensors 2 5th DValidation Sensors 2 50th Validation Sensors 2 95th
FIGURE 154. VALIDATION PERCENTILES FOR 141 REFERENCE NTE EVENTS
FOR SENSORS METHOD 2
REPORT 03.14936.12
154 of 174
-------
The 5* and 95* percentiles of the validation delta BSPM were separately fit for the AVL
and S ensors P EMS uni ts us ing a s imple 1 inear regression m odel. H owever, t he c riteria for
accepting the linear fits were not met for any of the calculations methods for these two PEMS
units. T hus, loess regression fits were used to determine the best functional representation for
the 5* and 95* delta BSPM based on the validation simulation modeling. These loess fits for the
95th and 5th percentiles for the Sensors unit methods 1 and 2 can be found in Figure 155 through
Figure 158 , respectively. T he 1 oess fits f or t he 95 th and 5 th percentiles f or t he A VL uni ts
methods 1, 2 a nd 3 c an be found in Figure 159 t hroughFigure 164, respectively. The loess
smoothing parameters for the regression fits are listed in Table 33.
TABLE 33. LOESS SMOOTHING PARAMETERS FOR VALIDATION PERCENTILES
PEMS
Sensors
AVL
Method
1
2
1
2
3
5th Percentile
0.290
0.290
0.294
0.294
0.294
95th Percentile
0.570
0.570
0.755
0.777
0.777
0.012:
0.010:
0.008:
0.006:
0.004:
0.002:
0.000:
-0.002:
D -0.004:
- 0.006:
-0.008:
-0.010:
-0.012:
I
EL
I
I
m
03
Validation 95th Percentile BSPM Deltas for 141 Ref NTE Events
BSPM (g/hp hr) Sensors 1 LOESS Fit Smoothing Parameter= 0.57
0.010
0.015
0.020
0.025
0.030
0.035
0.040
0.045
0.050
Ideal BSPM g/hp-hr
FIGURE 155. VALIDATION 95TH PERCENTILE BSPM DELTAS LOESS FIT FOR
SENSORS METHOD 1
REPORT 03.14936.12
155 of 174
-------
Validation Sth Percentile BSPM Deltas for 141 Ref NTE Events
BSPM (g/hphr) Sensors 1 LOESS Fit Smoothing Parameter= 0.29
Q.
w
m
-0.010
-0.011
-0.012
-0.013
-0.014-
-0.015-
-0.016-
-0.017
-0.018-
-0.019
-0.020-
-0.021'
-0.022
-0.023-
-0.024
-0.025
-0.026-
-0.027-
-0.028-
-0.029
-0.030-
-0.031
0.010
0.015
0.020
0.025
0.030
0.035
0.040
0.045
0.050
Ideal BSPM g/hp-hr
FIGURE 156. VALIDATION STH PERCENTILE BSPM DELTAS LOESS FIT FOR
SENSORS METHOD 1
Validation 95th Percentile BSPM Deltas for 141 Ref NTE Events
BSPM (g/hphr) Sensors 2 LOESS Fit Smoothing Parameters 0.57
I
ol
5
ft
m
0.012-
0.010-
0.008
0.006-
0.004-
0.002
o.ooo-
-0.002-
-0.004-
-0.006-
-0.008-
-0.010-
-0.012-
-0.014-
0.010
0.015
0.020
0.025
0.030
0.035
0.040
0.045
0.050
Ideal BSPM g/hp-hr
FIGURE 157. VALIDATION 95TH PERCENTILE BSPM DELTAS LOESS FIT FOR
SENSORS METHOD 2
REPORT 03.14936.12
156 of 174
-------
Validation 5th Percentile BSPM Deltas for 141 Ref NTE Events
BSPM (g/hphr) Sensors 2 LOESS Fit Smoothing Parameter= 0.29
I
m
-0.010;
-0.01V
-0.012
-0.013;
-0.014;
-0.015-
-0.016
-0.017
-0.018-
-0.019
-0.020;
-0.021;
-0.022-
-0.023
-0.024
-0.025;
-0.026;
-0.027-
-0.028
-0.029-
-0.030
-0.031
0.010
0.015
0.020 0.025
0.030
0.035
0.040 0.045
0.050
Ideal BSPM g/hp-hr
FIGURE 158. VALIDATION 5TH PERCENTILE BSPM DELTAS LOESS FIT FOR
SENSORS METHOD 2
Validation 95th Percentile BSPM Deltas for 141 Ref NTE Events
BSPM (g/hp-rr) AVL Method 1 LOESS Fit Smoothing Parameter 0.755
0.004
0.003
0.002
0.001
m
Q 0.000
-0.001
-0.002-I
*.
aV '.-.'
" '
0.010 0.015 0.020 0.025 O.OX 0.035 0.040 0.045 0.050
Ideal BSPM g/hp-hr
FIGURE 159. VALIDATION 95TH PERCENTILE BSPM DELTAS LOESS FIT FOR
AVL METHOD 1
REPORT 03.14936.12
157 of 174
-------
Validation 5th Percentile BSPM Deltas for 141 Ref NTE Events
BSPM (g/hp-hr) AVL Method 1 LOESS Fit Smoothing Parameter 0.294
Q.
W
CD
ฑฑ
Q
-0.0050
-0.0075
-0.0100
0.0125
-0.0150
-0.0175 H
0.010 0.015 0.020 0.025 0.030 0.035 0.040 0.045 0.050
Ideal BSPM g/hp-hr
FIGURE 160. VALIDATION 5TH PERCENTILE BSPM DELTAS LOESS FIT FOR
AVL METHOD 1
Validation 95th Percentile BSPM Deltas for 141 Ref NTE Events
BSPM (g/hp-hr) AVL Method 2 LOESS Fit Smoothing Parameter 0.777
0.003
O)
I
0.002-
0.001
0.000
-0.001-
-0.002 H
V. "
ป_
0.010 0.015 0.020 0.025 0.030 0.035 0.040 0.045 0.050
Ideal BSPM g/hp-hr
FIGURE 161. VALIDATION 95TH PERCENTILE BSPM DELTAS LOESS FIT FOR
AVL METHOD 2
REPORT 03.14936.12
158 of 174
-------
Validation 5th Percentile BSPM Deltas for 141 Ref NTE Events
BSPM (g/hp-hr) AVL Method 2 LOESS Fit Smoothing Parameter 0.294
1
m
ฃ
S
-0.0050
-0.0075
-o.oioo
-0.0125
-0.0150
-0.0175
0.010 0.015
0.020 0.025 0.030 0.035 0.040 0.045 0.050
Ideal BSPM g/hp-hr
FIGURE 162. VALIDATION 5TH PERCENTILE BSPM DELTAS LOESS FIT FOR
AVL METHOD 2
Validation 95th Percentile BSPM Deltas for 141 Ref NTE Events
BSPM (g/hp-hr) AVL Method 3 LOESS Fit Smoothing Parameter 0.777
I
m
0.003
0.002
0.001
0.000
-0.001
-0.002
0.010 0.015 0.020 0.025 O.OX 0.035 0.040 0.045 0.050
Ideal BSPM g/hp-hr
FIGURE 163. VALIDATION 95TH PERCENTILE BSPM DELTAS LOESS FIT FOR
AVL METHOD 3
REPORT 03.14936.12
159 of 174
-------
Validation 5th Percentile BSPM Deltas for 141 Ref NTE Events
BSPM (g/hp-hr) AVL Method 3 LOESS Fit Smoothing Parameter 0.294
a.
I
1
m
(V
Q
-0.0050
-0.0075
-0.0100
-0.0125
-0.0150
-0.0175
0.010
0.015
0.020
0.025
0.030
0.035
0.040
0.045
0.050
Ideal BSPM g/hp-hr
FIGURE 164. VALIDATION 5TH PERCENTILE BSPM DELTAS LOESS FIT FOR
AVL METHOD 3
6.4 Measurement Error Allowance Results
This section contains a summary of the measurement error allowance results using both a
regression method and a median method to determine the measurement allowance. Section 2.12
on Measurement Allowance contains a de tailed de script! on of t he m ethodology followed i n
determining these values. This procedure was applied to the simulation data for all 141 reference
NTE events obtained for all three calculation methods for the AVL PEMS and for calculations
methods 1 and 2 for the Horiba and Sensors PEMS.
Figure65 contains a regression piot of the 95* percentile delta B SPM values versus the
Ideal BSPM values for the 141 reference NTE events for AVL Method 1. Included in the plot is
the equation for the fitted regression line, and the R-square (R2) value and root mean square error
(RMSE) va lue f or t he r egression fit. T he t wo symbols i n t he pi ot r epresent r eference N TE
events where there was a dominant bias effect due to the SSPM error surface (diamond symbol)
or there was a dominant variance effect due to the SSPM error surface (square symbol). The R-
square va lue i ndicates t hat 47.95% of t he va riation in t he 95 th percentile B SPM va lues i s
explained by the ideal BSPM values for the AVL Method 1 data. The RMSE value of 0.0008
displays t he s ize of t he e stimated s tandard de viation of t he pr edicted 95
values.
th
percentile B SPM
REPORT 03.14936.12
160 of 174
-------
Table 34 includes a comparison of the results of the regression method based on Figure
165 and the median method as described in the Section 0 on Measurement Allowance. U nder
the heading of "Regression Method" in the table, it is shown that only the R-square criterion was
not m et b y t he da ta. T hus, t he M edian M ethod m ust be us ed. Under t he he ading " Median
Method" in the table, the measurement error at the BSPM threshold, based on using the median
of the 141 95 th percentile delta BSPM values, is 0.661% when expressed as a percent of the
threshold of 0.02 g/hp-hr.
AVL Method 1
0.0050
5. 0.0040 -
1) 0.0030 -
S
0.0020 -
3
0.0010 -
0 0.0000 -
-0.0010 -
io
-0.
y = 0.116x-0.002
R2 = 0.4795
RMSE = 0.0008
0
0.01
0.02 0.03 0.04
Ideal BSPM, g/hp-hr
0.05
0.06
Delta SSPM nicSSPM
FIGURE165. REGRESSION PLOT OF 95 PERCENTILE DELTA BSPM VERSUS
IDEAL BSPM FOR AVL METHOD 1
TABLE 34. MEASUREMENT ERROR AT THRESHOLD FOR BSPM USING
REGRESSION AND MEDIAN METHODS FOR AVL METHOD 1
R2
Regression Method
0.4795
Median Method
RMSE (SEE)
5% Median Ideal
Predicted 95th % Delta
at Threshold
Measurement Error @
Threshold = 0.02
0.0008
0.0191007
0.0003399
1.6993%
Did Not Meet
Criteria
Met Criteria
Median 95th % Delta
Measurement Error @
Threshold = 0.02
0.0001322
0.661%
REPORT 03.14936.12
161 of 174
-------
Figure 166 contains a regression plot of the 95* percentile delta BSPM values versus the
Ideal B SPM values for the 141 r eference NTE events for AVL Method 2. T he R-square value
indicates that 46.02% of the variation in the 95th percentile B SPM values is explained by the
Ideal BSPM values for the AVL Method 2 data. The RMSE value is 0.0008.
Table 35 includes a comparison of the results of the regression method based on Figure
166 and the median method. Under the heading of "Regression Method" in the table, it is shown
that the R -square c riterion for using t his m ethod i s not m et b y t he da ta. T hus, t he M edian
Method must be used. Under the heading "Median Method" in the table, the measurement error
at the BSPM threshold, based on using the median of the 141 95th percentile delta BSPM values,
is -2.375% when expressed as a percent of the threshold value of 0.02.
AVL Method 2
0.0040
TO
0.
W
ns
0.0030 -
0.0020 H
0.0010 -
0.0000 -
-0.0010 -
IO
o>
-0.0020
y = 0.0996x-0.0022
R2 = 0.4602
RMSE = 0.0008
0
0.01
0.02 0.03 0.04
Ideal BSPM, g/hp-hr
0.05
0.06
ป Delta SSPM nicSSPM
FIGURE 166. REGRESSION PLOT OF 95 PERCENTILE DELTA BSPM VERSUS
IDEAL BSPM FOR AVL METHOD 2
TABLE 35. MEASUREMENT ERROR AT THRESHOLD FOR BSPM USING
REGRESSION AND MEDIAN METHODS FOR AVL METHOD 2
Regression Method
0.4602
Median Method
RMSE (SEE)
5% Median Ideal
Predicted 95th % Delta at
Threshold
Measurement E rror @
Threshold = 0.02
0.0008
0.0191007
-0.0002124
-1.0618%
Did Not
Meet Criteria
Met Criteria
Median 95 * %
Delta
Measurement E rror
@ Threshold = 0.02
-0.0004751
-2.375%
REPORT 03.14936.12
162 of 174
-------
Figure 167 contains a regression plot of the 95* percentile delta BSPM values versus the
Ideal B SPM values for the 141 reference NTE events for AVL Method 3. T he R-square value
indicates that 46.20% of the variation in the 95th percentile B SPM values is explained by the
Ideal BSPM values for the AVL Method 3 data. The RMSE value is 0.0008.
Table 36 i ncludes a comparison of the results of the regression method based on Figure
167 and the median method. Under the heading of "Regression Method" in the table, it is shown
that the R -square c riterion for u sing t his m ethod i s not m et b y t he da ta. T hus, t he M edian
Method must be used. Under the heading "Median Method" in the table, the measurement error
at the BSPM threshold, based on using the median of the 141 95th percentile delta BSPM values,
is -2.383% when expressed as a percent of the threshold value of 0.02.
AVL Method 3
0.0040
0.0030 -
0.0020 H
0.0010 -
re
ป o.oooo H
Q
-0.0010 H
U)
o
-0.0020
y = 0.1001x-0.0022
R2 = 0.462
RMSE = 0.0008
0
0.01
0.02 0.03 0.04
Ideal BSPM, g/hp-hr
0.05
0.06
ปDeltaSSPM DicSSPM
FIGURE167. REGRESSION PLOT OF 95 PERCENTILE DELTA BSPM VERSUS
IDEAL BSPM FOR AVL METHOD 3
TABLE 36. MEASUREMENT ERROR AT THRESHOLD FOR BSPM USING
REGRESSION AND MEDIAN METHODS FOR AVL METHOD 3
Regression Method
0.4620
RMSE (SEE)
5% Median Ideal
Predicted 95th % Delta at
Threshold
Measurement E rror @
Threshold = 0.02
0.0008
0.0191007
-0.0002118
-1.0592%
Median Method
Did Not
Meet Criteria
Met Criteria
Median 95 th %
Delta
Measurement E rror
(a), Threshold = 0.02
-0.0004766
-2.383%
~~
REPORT 03.14936.12
163 of 174
-------
Figure 168 contains a regression plot of the 95* percentile delta BSPM values versus the
Ideal BSPM values for the 141 reference NTE events for Horiba Method 1. The R-square value
indicates that 26.57% of the variation in the 95th percentile B SPM values is explained bythe
Ideal BSPM values for the Horiba Method 1 data. The RMSE value is 0.0041.
Table 37 includes a comparison of the results of the regression method based on Figure
168 and the median method. Under the heading of "Regression Method" in the table, it is shown
that the R-square and the RMSE criteria for using this method were not met by the data. Thus,
the M edian M ethod m ust be us ed. U nder t he heading " Median M ethod" i n t he t able, t he
measurement error at the BSPM threshold, based on using the median of the 141 95th percentile
delta BSPM values, is 54.379 % when expressed as a percent of the threshold value of 0.02.
Horiba Method 1
0.0250
ฃ 0.0200 -
J 0.0150 -
Q.
% 0.0100 H
3
55 0.0050 -
^ o.oooo H
-0.0050
y = -0.3555x + 0.0174
R2 = 0.2657
RMSE = 0.0041
0 0.01 0.02 0.03 0.04 0.05 0.06
Ideal BSPM, g/hp-hr
| ปDeltaSSPM DicSSPM~|
FIGURE168. REGRESSION PLOT OF 95 PERCENTILE DELTA BSPM VERSUS
IDEAL BSPM FOR HORIBA METHOD 1
TABLE 37. MEASUREMENT ERROR AT THRESHOLD FOR BSPM USING
REGRESSION AND MEDIAN METHODS FOR HORIBA METHOD 1
RMSE (SEE)
5% Median Ideal
Regression Method
0.2657
0.0041
0.0191007
Median Method
th
Predicted 95 % Delta at
Threshold
Measurement E rror @
Threshold = 0.02
0.0102566
51.2831%
Did Not
Meet Criteria
Did Not
Meet Criteria
Median 95 * %
Delta
Measurement E rror
@ Threshold = 0.02
0.0108759
54.379%
REPORT 03.14936.12
164 of 174
-------
Figure 169 contains a regression plot of the 95* percentile delta BSPM values versus the
Ideal BSPM values for the 141 reference NTE events for Horiba Method 2. The R-square value
indicates that 25.97 % of the variation in the 95th percentile BSPM values i s explained by the
Ideal BSPM values for the Horiba Method 2 data. The RMSE value is 0.0041.
Table 38 includes a comparison of the results of the regression method based on Figure
169 and the median method. Under the heading of "Regression Method" in the table, it is shown
that the R-square and the RMSE criteria for using this method were not met by the data. Thus,
the M edian M ethod m ust be us ed. U nder t he heading " Median M ethod" i n t he t able, t he
measurement error at the BSPM threshold, based on using the median of the 141 95th percentile
delta BSPM values, is 50.079 % when expressed as a percent of the threshold value of 0.02.
Horiba Method 2
O)
0.0200
0.0150 -
t 0.0100 -
/
y =-0.3515x +0.0165
R2 = 0.2597
RMSE = 0.0041
0.01
0.02 0.03 0.04
Ideal BSPM, g/hp-hr
0.05
0.06
Delta SSPM DicSSPM
TH
FIGURE169. REGRESSION PLOT OF 95ltt PERCENTILE DELTA BSPM VERSUS
IDEAL BSPM FOR HORIBA METHOD 2
TABLE 38. MEASUREMENT ERROR AT THRESHOLD FOR BSPM USING
REGRESSION AND MEDIAN METHODS FOR HORIBA METHOD 2
Regression Method
0.2597
Median Method
RMSE (SEE)
5% Median Ideal
Predicted 95th % Delta at
Threshold
Measurement E rror @
Threshold = 0.02
0.0041
0.0191007
0.0094282
47.1408%
Did Not
Meet Criteria
Did Not
Meet Criteria
Median 95 %
Delta
Measurement E rror
(2> Threshold = 0.02
0.0100158
50.079%
REPORT 03.14936.12
165 of 174
-------
Figure 170 contains a regression plot of the 95* percentile delta BSPM values versus the
Ideal BSPM values for the 141 reference NTE events for Sensors Method 1. The R-square value
indicates that 45.49 % of the variation in the 95th percentile BSPM values i s explained by the
Ideal BSPM values for the Sensors Method 1 data. The RMSE value is 0.0029.
Table 39 i ncludes a comparison of the results of the regression method based on Figure
170 and the median method. Under the heading of "Regression Method" in the table, it is shown
that the R-square and the RMSE criteria for using this method were not met by the data. Thus,
the M edian M ethod m ust be us ed. U nder t he heading " Median M ethod" i n t he t able, t he
measurement error at the BSPM threshold, based on using the median of the 141 95th percentile
delta BSPM values, is 34.361 % when expressed as a percent of the threshold value of 0.02.
Sensors Method 1
0.0150
!_
Q. 0.0100 -
^ 0.0050 -
o.
& 0.0000 -
S
o> -0.0050 -
J -0.0100 H
35
m -0.0150
y = -0.3866x + 0.0144
R2 = 0.4549
RMSE = 0.0029
0
0.01
0.02 0.03 0.04
Ideal BSPM, g/hp-hr
0.05
0.06
Delta SSPM nicSSPM
-TH
FIGURE 170. REGRESSION PLOT OF 95ltt PERCENTILE DELTA BSPM VERSUS
IDEAL BSPM FOR SENSORS METHOD 1
TABLE 39. MEASUREMENT ERROR AT THRESHOLD FOR BSPM USING
REGRESSION AND MEDIAN METHODS FOR SENSORS METHOD 1
R2
RMSE (SEE)
Regression Method
0.4549
Median Method
5% Median Ideal
Predicted 95th % Delta
at Threshold
Measurement Error @
Threshold = 0.02
0.0029
0.0191007
0.0066785
33.3924%
Did Not
Meet Criteria
Did Not
Meet Criteria
Median 95th %
Delta
Measurement Error
(a), Threshold = 0.02
0.00687227
34.361%
REPORT 03.14936.12
166 of 174
-------
Figure 171 contains a regression plot of the 95* percentile delta BSPM values versus the
Ideal BSPM values for the 141 reference NTE events for Sensors Method 2. The R-square value
indicates that 45.03 % of the variation in the 95th percentile BSPM values i s explained by the
Ideal BSPM values for the Sensors Method 1 data. The RMSE value is 0.0030.
Table 40 includes a comparison of the results of the regression method based on Figure
171 and the median method. Under the heading of "Regression Method" in the table, it is shown
that the R-square and the RMSE criteria for using this method were not met by the data. Thus,
the M edian M ethod m ust be us ed. U nder t he heading " Median M ethod" i n t he t able, t he
measurement error at the BSPM threshold, based on using the median of the 141 95th percentile
delta BSPM values, is 30.285 % when expressed as a percent of the threshold value of 0.02.
Sensors Method 2
0.0150
Q- 0.0100 H
s- 0.0050 H
Q.
ฃ 0.0000 -
(0
-o.ooso H
-0.0100 H
01 -0.0150
55
y = -0.3875x +0.0137
R2 = 0.4503
RMSE = 0.0030
0
0.01
0.02 0.03 0.04
Ideal BSPM, g/hp-hr
0.05
0.06
ปDelta SSPM nicSSPM
FIGURE 171. REGRESSION PLOT OF 95 PERCENTILE DELTA BSPM VERSUS
IDEAL BSPM FOR SENSORS METHOD 2
TABLE 40. MEASUREMENT ERROR AT THRESHOLD FOR BSPM USING
REGRESSION AND MEDIAN METHODS FOR SENSORS METHOD 2
RMSE (SEE)
5% Median Ideal
Regression Method
0.4503
0.0030
0.0191007
Median Method
th
Predicted 95 % Delta at
Threshold
Measurement E rror @
Threshold = 0.02
0.0059333
29.6663%
Did Not
Meet Criteria
Did Not
Meet Criteria
Median 95 %
Delta
Measurement E rror
(a), Threshold = 0.02
0.0060569
30.285%
REPORT 03.14936.12
167 of 174
-------
Table 41 contains a s ummary of t he m easurement e rror va lues c ontained i n Table 3 4
through Table 40. The values are categorized by PEMS unit and by calculation method.
TABLE 41. BSPM MEASUREMENT ERROR IN PERCENT OF NTE THRESHOLD BY
PEMS AND CALCULATION METHOD
Measurement Errors (%) at Respective NTE Threshold
PEMS
AVL
Horiba
Sensors
Method 1
Exhaust Flow
Torque-Speed
0.661
54.379
34.361
Method 2
Exhaust and Fuel
Flow Torque-Speed
-2.375
50.079
30.285
Method 3
Fuel Flow
Torque-Speed
-2.383
n/a
n/a
Table 42 includes i n the m easurement al lowance s elected based on the m inimum
normalized PM. The AVL was not used in the measurement allowance determination because
the AVL at the start of the program was not accepted as an official PEMS, and the measurement
Steering Committee had decided that the measurement allowance would only be based on t he
Sensors or the Horiba PEMS.
TABLE 42. MEASUREMENT ALLOWANCE AT NTE THRESHOLD BY EMISSIONS
FOR METHOD 2
PEMS
Sensors
Method 2
Measurement
Error %
30.285
NTE
Threshold
g/hp-hr
0.02
Measurement
Allowance,
g/hp-hr
0.00605
On-road PM emissions were gathered from selected routes driven to collect emissions
data with a CE-CERT trailer and a PEMS installed on the tractor pulling the trailing. For each
on-road NTE event, a delta BSPM emissions value was computed as follows:
Delta BSPM = PEMS BSPM - CE-CERT BSPM.
These differences were computed for the BSPM emissions for each PEMS unit tested in-
use. The in-use BSPM was computed using Method 1 and 2 for the Sensors PPMD and Method
1,2, and 3 for AVL MSS. The in-use delta B SPM emissions calculated forthe AVLPEMS
using methods 1, 2, a nd 3. CE-CERT validation data were produced without any diesel particle
filter (DPF) active regeneration (referred to as "no regen"). (referred to a s " informational
purposes active AVL methods 1, 2 and 3 including three individual units (#2, #3 and #4) and 271
NTEevents. This data set was computedas "noregen". T he second PEMS unit tested for
validation w as t he S ensors. T he on -road delta BSPM em issions w ere calculated for S ensors
methods 1 and 2 also using three individual units in the "no regen" scenario and resulted in 217
NTE events.
REPORT 03.14936.12
168 of 174
-------
The va lidation pi ots for t he S ensors a nd t he A VL s ystems a re s hown in Figures 172
through 176. The y-axis scale on each figure was intended to make the best representation of the
data relative to the validation lines shown on each plot. Because Method 1 depends strongly on
exhaust flow, and some concerns were raised about the accuracy of exhaust flow measurement
during CE-CERT testing, results reported using Method 1 m ight not be accurate. Details about
exhaust flow measurement are expected to be part of CE-CERT Final Report on PM-PEMS In-
Use Validation Testing.
The loess regression fits in Figures 176 through 176 and the Sensors CE-CERT B SPM
differences col lected on -road were pi otted i n order t o determine i f t he s imulation m ethod
validated. If the number of CE-CERT delta BSPM values does not exceed 10% of the total
number of on-road NTE events collected, then the simulation method would be considered valid.
Figure 172 represents the validation plot for the BSPM method 1 analysis for Sensors. Note that
7 of the 217 on -road NTE events were either below or above the range of the ideal B SPM and
were excluded from the validation percentage calculation. T herefore, 68 of the 210 C E-CERT
NTE e vents (32.38%) fell be low t he s imulation m odel 5 * percentile ba sed on the loess
regression. Thus, the model was not considered valid for the BSPM Sensors Method 1.
Figure 173 represents the validation pi ot for the BSPM m ethod 2 a nalysis for S ensors.
Note that again 7 of the 217 on -road NTE events were either below or above the range of the
ideal BSPM and were excluded from the validation percentage calculation. Therefore, 71 of the
210 CE-CERT NTE events (33.81%) fell above the simulation model 95th percentile or below
the s imulation model 5th percentile ba sed on t he 1 oess r egressions. T hus, t he m odel w as not
considered valid for the BSPM Sensors Method 2.
Figure 174 represents the validation plot for the BSPM method 1 analysis for AVL with
'no regen'. In this case, 8 of the 271 on-road NTE events were either below or above the range
of the ideal BSPM and were excluded from the validation percentage calculation. Therefore, 49
of the 263 C E-CERT NTE events (18.63%) fell above the simulation model 95th percentile or
below the simulation model 5th percentile based on the loess regressions. Thus, the model was
not considered valid for the BSPM AVL Method 1.
Figure 175 represents the validation plot for the BSPM method 2 analysis for AVL with
'no regen'. Again in this case 8 of the 271 on-road NTE events were either below or above the
range o f t he i deal B SPM and were ex eluded f rom t he v alidation percentage calculation.
Therefore, 30 of the 263 CE-CERT NTE events (11.41%) fell above the simulation model 95th
percentile or below the simulation model 5* percentile based on the loess regressions. Thus, the
model was not considered valid for the BSPM AVL Method 2.
Figure 176 represents the validation plot for the BSPM method 3 analysis for AVL with
'no regen'. Again in this case 8 of the 271 on-road NTE events were either below or above the
range o f t he i deal B SPM and were ex eluded f rom t he v alidation percentage calculation.
Therefore, 26 of the 263 CE-CERT NTE events (9.89%) fell above the simulation model 95th
percentile or below the simulation model 5* percentile based on the loess regressions. Thus, the
model was considered valid for the BSPM AVL Method 3 since the number of CE-CERT NTE
events outside the 5th and 95th percentile loess regression was less than 10%.
REPORT 03.14936.12 169 of 174
-------
Validation 95th and 5th Percent Me BSPM Loess Fit for 141 Ref NTE Events
BSPM (g/hp-hr) Sensors Method 1 Units 1,2,3 n=217 (20 pis removed)
.c
1
m
IF
a
0.013-
0.008-
0.003-
-0.002-
-0.007-
-0.012-
-0.017-
-0.022-
-0.027-
-0.032-
-0.037-
-0.042 !,_,_,
0.005
0.010 0.015 0.020 0.025 0.030 0.035 0.040 0.045 0.050 0.055
Ideal BSPM g/hp-hr
95th Percentile 5th Percentile CE-CERT Deltas
FIGURE 172. VALIDATION ON-ROAD AND REGRESSION FUNCTIONS BASED ON
THE SIMULATION MODEL FOR BSPM SENSORS METHOD 1 WITH NO REGEN
Validation 95th and 5th Percentile BSPM Loess Fit for 141 Ref NTE Events
BSPM (g/hp-hr) Sensors Method 2 Units 1,2,3 n= 217 (20 pts removed)
0.03-
0.02
0.01
k.
a. 0.00
1 -0.01
w
m
ฃ -0.02
Q
-0.03
-0.04
-0.05 H
0.005 0.010 0.015 0.020
95th Percentile
0.025 0.030 0.035 0.040 0.045 0.050 0.055
Ideal BSPM g/hp-hr
5th Percentile CE-CERT Deltas
FIGURE 173. VALIDATION ON-ROAD AND REGRESSION FUNCTIONS BASED ON
THE SIMULATION MODEL FOR BSPM SENSORS METHOD 2 WITH NO REGEN
REPORT 03.14936.12
170 of 174
-------
ft
m
Validation 95th and 5th Percent Me BSPM Loess Fit for 141 Ref MTE Events
BSPM (g/hp-hr) AVL Method 1 Units 2,3,4 n=271 No Regen
0.010;
0.008:
0.006;
0.004;
0.002:
0.000;
-0.002-
-0.004;
-0.006:
-0.008;
-0.010;
-0.012;
-0.014;
-0.016:
tr. . .*
., . .ฃL_&Iฑ-?ฃ
^&&
**&&%
^
0.005 0.010 0.015 0.020
95th Percentile
0.025 0.030 0.035 0.040
Ideal BSPM g/hp-hr
5th Percentile
0.045 0.050 0.055 0.060
CE-CERT Deltas
FIGURE 174. VALIDATION ON-ROAD AND REGRESSION FUNCTIONS BASED ON
THE SIMULATION MODEL FOR BSPM AVL METHOD 1 WITH NO REGEN
Validation 95th and 5th Percentile BSPM Loess Fit for 141 Ref NTE Events
BSPM (g/hp-hr) AVL Method 2 Units 2,3,4 n=271 No Regen
D.
W
m
Tit
a
0.006:
0.004:
0.002:
0.000:
-0.002:
-0.004;
-0.006:
-0.008:
-0.010;
-0.012:
-0.014;
-0.016;
-0.018:
FIGURE
THE
O.X5 0.010 0.015 0.020 0.025 0.030 0.035 0.040 0.045 0.050 0.055 0.060
Ideal BSPM g/hp-hr
95th Percentile 5th Percentile CE-CERT Deltas
175. VALIDATION ON-ROAD AND REGRESSION FUNCTIONS BASED ON
SIMULATION MODEL FOR BSPM AVL METHOD 2 WITH NO REGEN
REPORT 03.14936.12
171 of 174
-------
Validation 95th and 5th Percentile BSPM Loess Fit for 141 Ref NTE Events
BSPM (g/hp-hr) AVL Method 3 Units 2,3,4 n= 271 No Regen
0.006-
0.004-
0.002-
0.000
h_
^ -0.0021
a.
- 0.004 ^
O
Q.
m
ra
i
Q
-0.006:
-0.008-
-0.010:
-0.012:
-0.014:
-0.016:
-0.018-
0.005 0.010 0.015 0.020 0.025 0.030 0.035 0.040 0.045 0.050 0.055 0.060
Ideal BSPM g/hp-hr
95th Percentile 5th Percentile CE-CERT Deltas
FIGURE 176. VALIDATION ON-ROAD AND REGRESSION FUNCTIONS BASED ON
THE SIMULATION MODEL FOR BSPM AVL METHOD 3 WITH NO REGEN
Table 43 summarizes the model validation results. Only the AVL Method 3 pa ssed the
model validation.
TABLE 43. SUMMARY OF BSPM MODEL VALIDATION RESULTS
PEMS Unit
Sensors "no regen"
AVL "no regen"
Method 1
Exhaust Flow
Torque-Speed
No
No
Method 2
Exhaust and Fuel
Flow Torque-Speed
No
No
Method 3
Fuel Flow
Torque-Speed
No
Yes
REPORT 03.14936.12
172 of 174
-------
7.0 SUMMARY
A s eries of en gine ex periments and environmental t ests w ere p erformed on three P M-
PEMS that included the Sensors PPMD, Horiba TRPM, and AVL MSS. The Sensors PPMD and
Horiba TRPM were treated as official PEMS, and only those were used for official results and
determination of next steps. The AVL MSS were used in conjunction with the Sensors PPMD
based on an agreement reached between Sensors and AVL. Error surfaces were developed based
on e xperimental w ork t o de termine P M-PEMS bi as a nd pr ecision e rrors us ing M onte C arlo
simulation model. The output of the model was to determine the error distribution at a set of
reference N TE ev ents, compared t o their i deal va lue. The P M-PEMS w ith t he 1 owest 95 th
percentile error that is greater than zero was selected for in-use validation testing of the model.
The P M-PEMS t hat w as s elected for i n-use va lidation w as t he S ensors P PMD. T he
PPMD produced a 95th percentile measurement allowance error of 0.006 g/hp-hr at a threshold
NTE limit of 0.02 g/hp-hr using Method 2, c ompared to the 0.01 g/hp-hr that was produced by
the Horiba TRPM. As for the AVL MSS, the instrument produced zero measurement allowance,
but its measurement allowance value was not officially used because the MSS only measures the
carbon fraction of PM, compared to the required total (solid plus volatile) PM measured by the
other two PM-PEMS. However, because the AVL MSS was used in conjunction with the PPMD,
the SC a greed t o i nclude i t dur ing i n-use va lidation testing. D ue to funding limita tion, the
Horiba TRPM was not included in in-use validation testing.
Based on i n-use v alidation t esting, t he S ensors P PMD f ailed va lidation be cause 3 2
percent and 34 percent of the data produced in-use were below the 5 percentile of the validation
window, using Method 1 and Method 2, respectively. The SC agreed during the development of
the TestPlan that less or equal 10 percent (< 10%) of the data are allowed to be outside the
validation window to pass validation.
As for the AVL MSS, it passed validation using Method 3 by having 9.89 percent of the
data outside the validation window, with the majority of these data be ing higher than the 95*
percentile. Method 2 failed by two percentage points and Method 1 failed by 8 percentage points.
Because the MSS using Method 3 passed validation and funding run out to do any further
work with the Sensors PPMD and/or to perform in-use validation testing with the Horiba TRPM,
the SC concluded the measurement allowance program. The SC also accepted the measurement
allowance ba sed on t he S ensors P PMD a s t he final P M-PEMS m easurement al lowance for
Methods 1,2, and 3
REPORT 03.14936.12 173 of 174
-------
8.0 REFERENCES
1. "Electronic Code of Federal Regulations, Title 40 Protection of the Environment, Vol. 15
Part 1065," http://www.ecfr.gpoaccess.gov, 2009.
2. "Determination of PEMS Measurement Allowance for Gaseous Emissions Regulated Under
theH eavy-Dutyl n-Use T estingP rogram: R evised Final R eport," EPA420-R-08-005,
February 2008.
3. Conover, W.J., PracticalNonparametric Statistics., John Wiley & Sons, 1971
4. Cleveland, W .S., " Robust Locally-Weighted Regression a nd S moothing S catterplots,"
Journal of the American Statistical Association, 1979, 74, pp. 829-836.
5. Hurviuch, C .M., S imonoff, J .S., a nd T sai, C .L., "Smoothing P arameters S election i n
Nonparametric Regression Using and Improved Akaike Information Criterion," Journal of
the Royal Statistical Society B, 1998, 60, pp.271-293.
6. Stratmann, F., Otto, E., and Fissan, H., "Thermophoretical and Diffusional Particle Transport
in a Cooled Laminar Tube Flow." Journal of Aerosol Science, March 28th, 1994, vol. 25; pp.
1359-1365.
7. Liu, B . Y.H and D .Y.H. P ui, " Equilibrium B ipolar C harge D istribution of A erosols," J .
Colloid Interface Science, Vol. 49, 1974.
REPORT 03.14936.12 174 of 174
-------
APPENDIX A
TEST PLAN TO DETERMINE PEMS MEASUREMENT ALLOWANCE
FOR THE PM EMISSIONS REGULATED UNDER THE
MANUFACTURER-RUN HEAVY-DUTY DIESEL ENGINE IN-USE
TESTING PROGRAM
DEVELOPED BY:
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY,
CALIFORNIA AIR RESOURCES BOARD, AND
ENGINE MANUFACTURERS ASSOCIATION
NOVEMBER 11, 2008
REPORT 03.14936.12
-------
EXECUTIVE SUMMARY
This test plan sets forth the agreed upon processes and methodologies to be utilized to develop
additive, br ake-specific, da ta-driven measurement allowance for PM emissions me asured b y
PEMS as required under the HDIUT regulatory program.
As detailed in this test plan, there is a clear consensus on what components of measurement error
are i ntended to be covered by t he m easurement allowance. Namely, the allowance is to be
calculated in a manner that subtracts lab error from PEMS error. Specifically, utilizing Part 1065
compliant emissions measurement systems and procedures for both the lab and PEMS, the lab
error associated with measuring heavy-duty engine emissions at stabilized steady-state test points
within the NTE zone, will be subtracted from the PEMS error associated with measuring heavy-
duty e ngine e missions utilizing P EMS ove r events unde r a br oad r ange of environmental
conditions. This subtraction will yield "PEMS minus laboratory" measurement allowance. The
experimental methods and procedures specified in this test plan for determining, modeling, and
comparing e ach of t he various c omponents of measurement error a re designed t o generate
statistically robust data-driven measurement allowance for the PM emissions.
Successful completion of this test plan is part of the resolution of a 2001 suit filed against EPA
by EMA and a number of individual engine manufacturers. The suit challenged, among other
things, c ertain s upplemental e mission r equirements r eferred to as " not-to-exceed" ( NTE)
standards. On June 3, 2 003, the parties finalized a settlement of their disputes pertaining to the
NTE standards. T he parties agreed upon a detailed outline for a future regulation that would
require a manufacturer-run heavy-duty in-use NTE testing ("HDIUT") program for diesel-fueled
engines and vehicles. One section of the outline stated:
"The N TE T hreshold w ill be t he N TE s tandard, i ncluding t he m argins bui It i nto t he e xisting
regulations, plus additional margin to account for in-use measurement accuracy. This additional
margin shall be determined by the measurement processes and methodologies to be developed
and a pproved b y E PA/CARB/EMA. T his m argin w ill be s tructured t o encourage i nstrument
manufacturers to develop more and more accurate instruments in the future."
Given the foregoing, the work to be completed under this test plan i s a vital component to the
fulfillment of the settlement agreement, and it is vital to the successful implementation of a fully-
enforceable HDIUT p rogram. Because of thi s significance, it is c ritically important that the
work de tailed i n t his t est pi an be c arried out i n a s t horough, c areful a nd t imely a m anner a s
possible.
REPORT 03.14936.12 A-l
-------
TABLE OF CONTENTS
Executive Summary A-l
Table of Contents A-2
List of Figures A-4
List of Tables A-4
1 Introduction A-5
2 Monte Carlo Error Model and Measurement Allowance A-7
2.1 Objective A-7
2.2 Background A-7
2.3 Methods and Materials A-13
2.4 Simulation Procedure A-13
2.4.1 Construction of the Error Surface A-15
2.5 Model Considerations A-20
2.5.1 Convergence A-20
2.6 Simulation Output A-20
2.6.1 Sensitivity Variation Effect A-20
2.6.2 Sensitivity Bias Effect A-21
3 Engine Dynamometer Laboratory Tests A-21
3.1 Preliminary Audits A-21
3.1.1 Objective A-21
3.1.2 Background A-21
3.1.3 On-site meeting to establish 1065 compliance requirements A-22
3.1.4 Methods and Materials A-22
3.1.5 Data Analysis A-22
3.1.6 PEMS Manufacturer PM PEMS Commissioning A-22
3.2 Bias and Precision Errors under steady state engine operation A-23
3.2.1 Objective A-23
3.2.2 Background A-24
3.2.3 Methods and Materials A-24
3.2.4 Data Analysis A-25
3.3 Precision Errors under transient engine operation (dynamic response) A-26
3.3.1 Objective A-26
3.3.2 Background A-26
3.3.3 Methods and Materials A-26
3.3.4 Data Analysis A-27
3.4 ECM Torque and BSFC A-30
3.4.1 Objective A-30
3.4.2 Data Analysis A-30
4 Environmental Chamber A-30
4.1 Data Analysis for Environmental Tests A-31
4.2 PM Generator Commissioning A-32
4.3 Baseline A-32
4.3.1 Objective A-32
4.3.2 Background A-32
4.3.3 Methods and Materials A-32
4.3.4 Data Analysis A-33
4.4 Electromagnetic Radiation A-33
REPORT 03.14936.12 A-2
-------
4.4.1 Objective A-33
4.4.2 Methods and Materials A-33
4.5 Atmospheric Pressure A-33
4.5.1 Objective A-33
4.5.2 Background A-33
4.5.3 Methods and Materials A-34
4.5.4 Data Analysis A-36
4.6 Ambient Temperature and Humidity A-36
4.6.1 Objective A-36
4.6.2 Background A-36
4.6.3 Methods and Materials A-37
4.6.4 Data Analysis A-39
4.7 Orientation and Vibration A-39
4.7.1 Objective A-39
5 SwRI CVS and CE-CERT Trailer Correlation A-39
5.1 Method and Materials A-39
6 Model Validation and Measurement Allowance Determination A-40
6.1 Model validation A-41
6.1.1 Objective A-41
6.2 Measurement Allowance Determination A-43
6.2.1 Objective A-43
6.2.2 Background A-44
6.2.3 Methods and Materials A-44
6.2.4 Data Analysis A-44
7 Time and Cost A-44
7.1 Timeline A-44
7.2 Cost A-44
8 Abbreviations used in Brake Specific Equations A-45
REPORT 03.14936.12 A-3
-------
LIST OF FIGURES
FIGURE 1. PROBABILITY DENSITY FUNCTIONS FOR SAMPLING ERROR SURFACES A-8
FIGURE 2. BRAKE-SPECIFIC PM EMISSIONS CALCULATION FOR METHOD 1 A-9
FIGURE 3. BRAKE-SPECIFIC PM EMISSIONS CALCULATION FOR METHOD 2 A-10
FIGURE 4. BRAKE-SPECIFIC PM EMISSIONS CALCULATION FOR METHOD 3 A-ll
FIGURES. ERROR SURFACE: PEMSVS. LAB A-14
FIGURES. ERRORSURFACE: (PEMS-LAB) VS. LAB A-14
FIGURE?. ERRORSURFACE: FINAL VERSION A-15
FIGURE 8. OVERVIEW OF MONTE CARLO SIMULATION A-18
FIGURE 9. EXAMPLE OF A NTE CYCLE A-29
FIGURE 10. PRESSURE HISTOGRAM A-34
FIGURE 11. PRESSURE-TIME ENVIRONMENTAL TEST CYCLE A-35
FIGURE 12. TEMPERATURE HISTOGRAM A-37
FIGURE 13. TIME SERIES CHART OF AMBIENT TEMPERATURE TEST A-38
LIST OF TABLES
TABLE 1. ALLOWED MODIFICATIONS A-5
TABLE 2. EXAMPLE OF SELECTION OF MEASUREMENT ALLOWANCE AT 0.02 G/HP-HR NTE THRESHOLDA-12
TABLES. ERROR SURFACES FOR THE BSPM SIMULATION A-19
TABLE 4. ENGINE, EXHAUST CONFIGURATION, AND STEADY-STATE MODES A-23
TABLES. EXAMPLE OF SS ERROR SURFACE A-25
TABLES. NTE TRANSIENT CYCLE A-278
TABLE 7. DYNAMIC RESPONSE INTER-NTE EVENTS A-29
TABLE 8. CONCENTRATION AND DILUTION RATIO SCHEDULE WITH PM GENERATOR A-31
TABLE 9. ATMOSPHERIC PRESSURE TEST SEQUENCE A-35
TABLE 10. AMBIENT TEMPERATURE TEST SEQUENCE A-38
TABLE 11. PROJECTED PM-PEMS TIMELINE A-44
TABLE 12. PROJECTED COST ESTIMATE A-45
REPORT 03.14936.12 A-4
-------
1 INTRODUCTION
This te st pi an will e stablish a P EMS me asurement a llowances for P M, as r egulated b y the
manufacturer-run on -highway h eavy-duty di esel engine i n-use test program. T he m easurement
allowance will be established using various laboratory facilities and PEMS. The measurement
allowance will be established in units of brake-specific emissions (g/hp-hr), and it will be added
to the final NTE PM standard, after all the other additive and multiplicative allowances have
been applied. This test plan will establish the PM measurement allowance.
The PEMS used in this test plan must be standard in-production makes and models that are for
sale as commercially available PEMS. In addition, PEMS and any support equipment must pass
a "red-face" test with respect to being consistent with acceptable practices for in-use testing. For
example, t he e quipment m ust m eet all s afety and t ransportation regulations for us e on -board
heavy-duty vehicles.
Even though the PEMS cannot be "prototypes" nor their software "beta" versions, the steering
committee has already agreed that after delivery of PEMS to the contractor, there may be a few
circumstances i n which PEMS m odifications m ight be allowed, but these m edifications must
meet certain deadlines, plus they are subject to approval by the steering committee. Also, any
implementation of such approved modifications will not be allowed to delay the test plan, unless
the steering committee specifically approves such a delay. Table 1 summarizes these allowable
modifications and their respective deadlines:
TABLE 1. ALLOWED MODIFICATIONS
Allowed Modifications
Steering committee approved hardware and software modifications
that affect emissions results; including but not limited to fittings,
components, calibrations, compensation algorithms, sampling rates,
recording rates, etc.
Steering c ommittee a pproved ha rdware m odifications f or D OT
approval or any other safety requirement approval
Delivery of any environmental / weather enclosure to contractor
Post-processing software to determine NTE results
DOT approval and documentation
Steering committee appr oved hardware or s oftware t hat i mproves
the contractor's efficiency to conduct testing and data reduction
Before start of. . .
Steady-State
Testing
Environmental
Chamber Testing
Environmental
Chamber Testing
Model Validation
Model Validation
Always Allowed
The s teering com mittee appr oved three di fferent P EMS t hat i ncludes t he A VL M icro-Soot
Sensor (MS S), the Horiba Transient P articulate Matter ( TRPM), and the Sensors Proportional
Particulate M atter D iluter ( PPMD). H owever, because of t he different m easurement
technologies employed by each of these systems, the three different PEMS hold slightly different
status w ith respect t o de termining t he P Mm easurement allowance. Because i nertial
microbalances are already approved for PEMS applications i n 40 C FR P art 1065, t he S ensors
PPMD will be one of the PEMS used to determine the measurement allowance. A nd because
EPA's PM standard is based upon a gravimetric filter analysis, the Horiba TRPM will also be
used to determine t he measurement al lowance. T he 1 owest m easurement al lowance v alue
REPORT 03.14936.12 A-5
-------
between the two will be selected as the final measurement allowance for PM. If that value does
not validate, then the lowest validated value will be chosen. If the lowest validated value chosen
is within 0.0075 g /hp-hr from the 1 owest non-validated value, then the 1 owest validated value
will be the measurement allowance. Otherwise, the MASC will spend up to a $100,000 to figure
out a resolution to the problem by generating more data or changing the way the validation was
performed. If that does not lead to a resolution, then Executive Management of EMA and EPA
will have to settle the issue.
Note that at the conclusion of successful te sting of the H oriba s ystem in this me asurement
allowance program, EPA intends to approve the Horiba system as an alternative for use, or EPA
may elect to amend 40 CFR Parts 86 and/or 1065 to allow the use of the Horiba TRPM or other
PEMS t hat ope rate upon similar m easurement pr inciples. B ecause t he A VL s ystem m easures
only the soot component of PM, the measurement allowance will not be determined using the
AVL r esults, unl ess bot h t he S ensors and H oriba s ystems fail to complete the me asurement
allowance program. Note that the steering committee may determine at the conclusion of the
program that the AVL MSS is a viable alternative for demonstrating compliance. Under such a
circumstance EPA may amend the Heavy-Duty In-Use regulation to allow for its use.
This test plan describes a computer model, a series of experiments that are used to calibrate the
model, and another series of experiments that are used to validate the calibrated model.
The test plan first describes the computer model. T he computer model statistically combines
many sources of PEMS and lab error, which are nearly impossible to capture simultaneously in a
single test. The model will use statistics to apply the e rrors in a way that s imulates actual
running o f a P EMS i n-use. T he m odel w ill a Iso c onsider onl y t he po rtion of e rror t hat i s
attributable to PEMS, and it will subtract the error that is already tolerated in an emissions lab
today. The model will also calculate and validate results according to 40 CFR Part 1065.
The test plan then describes the series of experiments. These tests will characterize the many
sources of PEMS and lab error so that the specific nature of the errors can be programmed into
the computer model. The nature of the error has to do with the way PEMS and the lab react to
certain conditions. For example, under varying environmental conditions such as temperature or
vibration, a PEMS might exhibit signal drift, or it may record noise that is not a part of the true
emissions.
Next, the experimental results will be entered into the c omputer m odel, and the measurement
allowances are calculated by the model. The model uses a "reference" PEMS data set, which
will have many "reference NTE events." The model statistically applies all the errors to the
reference data set, calculates results, and saves the results. Then the model will be run with all
errors set to zero to calculate the ideal results of the reference data set. Each difference between
a reference NTE event's result with errors and its respective ideal result will be a brake-specific
difference that is recorded for later use. Then the process repeats using the same reference data
set, tow hich ne w, s tatistically s elected e rrors are a pplied, and t hus another unique s et of
differences is calculated. As the model continues to iterate and generate more and more results,
patterns are expected to appear in the output data. These patterns should be the distributions of
differences, based upon the error that was statistically and repeatedly ap plied to the reference
data set. Many difference di stributions will be determined: for each reference NTE event, for
each of the two brake-specific calculation methods (three in case of the AVL system only), and
REPORT 03.14936.12 A-6
-------
for each PEMS. It has been agreed that the 95* percentile values of these distributions will be
taken as reasonable "worst case" results for each reference NTE event. Details on how all these
distributions will be reduced to determine the PM measurement allowance is given in the "Error
Model" section of this test plan.
Because the calculation based on Method 2 and Method 3 require gas-based fuel flow calculation
based on t he measurement of CO2, CO, andNMHC, a decision was made to use the gaseous
PEMS data for this purpose, without the need to perform gaseous measurement during the PM-
PEMS program.
Finally, the te st pi an describes how the c omputer mode 1 w ill be v alidated against r eal-world
over-the-road i n-use P EMS ope ration a s w ell a s a dditional 1 ab t esting. For t he ove r-the-road
testing, PEMS emissions measurements will be conducted, while at the same time a reference
laboratory will be towed along to measure the same emissions. For the lab testing, an attempt
will be made to simulate real-world engine operation to "replay" an over-the-road test in the lab.
Data from these final experiments will be used to validate the model, which must be done in
order to gain sufficient confidence that the model did not establish unreasonable measurement
allowances.
The following sections of this test plan are written as instructions to the contractor or contractors
who will complete the test plan.
2 MONTE CARLO ERROR MODEL AND MEASUREMENT ALLOWANCE
2.1 OBJECTIVE
Use Monte Carlo (e.g. random sampling) techniques in an error model to simulate the combined
effects of al 11 he a greed-upon s ources of P EMS e rror i ncremental t o 1 ab e rror. C reate e rror
"surfaces" for the Monte Carlo simulation to sample, based upon results from the experiments
described in Sections 3 and 4. Exercise the model over a wide range of NTE events, based on a
single, reference data set of at least 150 but no more than 200 unique NTE events. Determine the
pollutant-specific brake-specific additive measurement allowance for PM.
2.2 BACKGROUND
The error model uses Monte Carlo techniques to sample error values from "error surfaces" that
are generated from the results of each of the experiments de scribed in Section 3 on engine
dynamometer laboratory tests and Section 4 on e nvironmental chamber tests. The lab test error
surfaces cover the domain of error versus the magnitude of the signal to which the error is to be
applied (i.e. 1st to 99* percentile error vs. concentration, flow, torque, etc.). T his i s illustrated
later in this section. The e nvironmental te st e rror surfaces f or s hock & vi bration a nd
electromagnetic & radio frequency interference (EMI/RFI) cover t he s ame dom ain as the lab
tests. The environmental test error surfaces for pressure and temperature are characteristically
different because they cover the domain of environmental test cycle time versus the magnitude of
the signal to which the error is to be applied (i.e. error at a s elected time vs. concentration).
Details on how each surface i s generated are given in each of the respective sections. These
surfaces will al ready b e adj usted to represent P EMS error i ncremental to lab error; therefore,
these surfaces are sampled directly by the model.
REPORT 03.14936.12 A-7
-------
The error model will use two different probability density functions (PDFs) as shown in Figure 1
to sample t he e rror s urfaces, de pending upon which e xperiment t he s urface r epresents. T o
sample error surfaces that are generated from all the laboratory test results (Section 3), and the
environmental t est r esults f or s hock & vi bration ( Section 4), t he m odel w ill u se a t runcated
normal PDF because these tests are designed to evenly cover the full, but finite, range of engine
operation and ambient conditions. To sample error surfaces that are generated from the pressure
and temperature e nvironmental te st r esults ( Section 4) , t he m odel w ill us e a uni form P DF
because t hese t ests are al ready d esigned to covert he t ypical range and frequency of t he
respective conditions.
.
Probability Density Functions for Sampling Error Surfaces Once Per NTE Event
Lab Tests, Normal, SD=0.60795, truncate @ -1 & 1
Environmental Tests, Uniform
Note: A non-trun
distribution with
values of 0.01 an
ic=+1, respective
cated normal
SD=0.60795 has P
d0.99atic=-1 anc
y.
^
z
V
_^\
^^
^^
^_
^^
^^
ซ^*^
^^^
^^^
> 4 3 2
Relative Probability
- 0.75
- 0.50
- 0.25
- 0.00 i c
- -0.25
- -0.50
- -0.75
0
FIGURE 1. PROBABILITY DENSITY FUNCTIONS FOR SAMPLING ERROR
SURFACES
The random values that are obtained from both distributions are labeled ic in Figure 1 and range
from -1 to 1. Note that for the pressure and temperature environmental tests, a uniform PDF
will be used to sample test time, from which the nearest (in time) calculated errors are used. The
errors from the other tests will be aligned with the truncated normal PDF such that each of the
50th percentile values at each of the tested signal magnitudes i s centered at the median of the
PDF (ic = 0), and the 1st and 99th percentile error values at each of the tested signal magnitudes
will be aligned with the extreme negative (ic = -1) and positive (ic = +1) edges of the PDF,
respectively.
Each error surface will be sampled along its ic axis (y-axis) once per reference NTE event trial,
and it will be sampled along its parameter value axis (x-axis, e.g., concentration (only for AVL
MSS), flow, torque, etc...) once per second, within a given reference NTE event trial. An error
will be de termined for a given second and parameter al ong t he error ax is ( z-axis) a t the
intersection of an ic value and a parameter value.
REPORT 03.14936.12
A-8
-------
To e nsure t hat t he m agnitudes of t he error s urfaces are a ppropriate, each da ta poi nt us ed t o
generate the surfaces will be a mean or a weighted mean of 30 seconds of sampling.
Interpolation will be p erformed b y first line arly int erpolating e rror values at ea ch tested
magnitude al ong t he s elected line pe rpendicular t o the i c axis. T hen from t hat 1 ine of e rrors,
individual error values will be linearly interpolated at each second-by-second signal magnitude
of the given NTE event in the reference data set.
The r eference d ata s et t o which all er rors w ill be appl ied will be a 1 arge d ata s et of engine
operation over a wide range of NTE events. This reference data set will be initially generated
from collections of real-world PEMS data sets. The reference data set should contain at least
150 but no m ore than 200 unique NTE events. Parameters in the reference data set may be
scaled in order t o exercise t he m odel t hrough a m ore appr opriate r ange of p arameters (i.e.
concentrations, flows, ambient conditions, etc.). If the parameters are scaled, c are s hould be
taken to maintain the dynamic characteristics of the reference data set.
After the errors are applied, NTE brake-specific PM emissions results are calculated, using each
of the three agreed-upon NTE calculation methods. The three different brake-specific emission
calculation methods for PM referred to in this test plan are i) Torque-Speed method, ii) BSFC
method, a nd i ii) E CM-Fuel S pecific m ethod, and these ar e i llustrated in Figure 2, 3, a nd 4,
respectively.
For all PM PEMS:
is a flow weighted particulate matter exhaust concentration in g/mol
m
PM
ePM(glkW-hr) = -
PM
--
mol
n
mol
'At
i\
I
Speedi(rpm)*Ti(N-m)*2*3.\4\59*At
60*1000*3600
Where for AVL:
m PM is computed numerically as follows,
m
PM
g
mol
1 V
I
l
'mol
IV
I
FIGURE 2. BRAKE-SPECIFIC PM EMISSIONS CALCULATION FOR METHOD 1
REPORT 03.14936.12
A-9
-------
For all PM PEMS:
m PM is a flow weighted particulate matter exhaust concentration in g/mol
em(glkW-hr) = .
w
fuel 1=1
m
TM
g
mol
Speedt(rpm)*Tt(N-m)*2*3.U159
60*1000*3600
Where for AVL:
m PM is computed numerically as follows,
m
l
'mol
PM
mol
i\
I
. mol
J
FIGURE 3. BRAKE-SPECIFIC PM EMISSIONS CALCULATION FOR METHOD 2
REPORT 03.14936.12
A-10
-------
For AVL Only:
ePM(glkW-hr} =
mE
M
-*Af
Speedi(rpm)*Ti(N-m)*2*3.14159*At
60*1000*3600
Where:
w
W
fuel
m
PM
_g_
mol
(mPMt(glmol))*mL
xTHC, (ppm) * 1 0~6 + (xCO, (%) + xCO2i (%)) * 1 0~2
-*Af
xTHC, (ppm) * 10~6 + (xCO, (%) -
-*A/
FIGURE 4. BRAKE-SPECIFIC PM EMISSIONS CALCULATION FOR METHOD 3
Next, the NTE events are calculated by each of the three calculation methods, but with no error
sampled or applied to the reference data set. These results are considered the "ideal" results of
the reference NTE events. These i deal results are subtracted from each respective NTE event
result 'with errors', and the difference i s recorded. T hen a new set of errors are sampled and
applied to the reference NTE event, and the NTE results 'with errors' are calculated again. The
ideal results are again subtracted, and the difference is recorded. This is repeated thousands of
times so that the model converges upon distributions of brake-specific differences for each of the
original NTE events in the reference data set.
Then the 95th percentile difference value is determined for each NTE event distribution of brake-
specific differences for PM for each calculation method. At this point there is one distribution of
95* percentile differences for PM, where all the NTE events are pooled by the PM emissions for
each of t he t hree di fferent c alculation m ethods. E ach of t he 95
represents a range of possible measurement allowance values.
th
percentile di stributions
From e ach of t hese t hree di stributions of p ossible m easurement a llowance v alues, one
measurement allowance per distribution must be determined. First the correlation between 95*
percentile differences versus the ideal PM emission is tested. For each calculation method, if a
least squares linear regression of 95th percentile differences versus ideal PM emissions has an r2
(squared correlation coefficient) > 0.85 and an SEE (standard error of the estimate or root-mean-
REPORT03.14936.12
A-ll
-------
squared-error) < 5 % of the median ideal PM emission, then that linear regression equation will
be used to determine the measurement allowance for that calculation method at the following
NTE threshold:
PM = 0.02 g/hp-hr and 0.03 g/hp-hr
In cases where extrapolation is required to determine the m easurement allowance at the NTE
threshold, t he m easurement a llowance w ill be de termined u sing t he 1 inear r egression, but
evaluated at the ideal PM emission that is closest to the NTE threshold, not extrapolated to the
NTE threshold itself. If the linear regression does not pass the aforementioned r2 and SEE
criteria, then the m edian value of the 95 th percentile di fferences i s us ed as t he s ingle
measurement allowance for that calculation method.
Next, the calculation method is selected. The above procedure will provide three measurement
allowances, where applicable, one for each of the three different calculation methods. To make
them com parable, the t hree m easurement al lowance va lues w ill be n ormalized by t he P M
threshold and expressed as a percent. Also, if any measurement allowance is determined to have
a va lue 1 ess t han zero, t hen that m easurement al lowance w ill be s et equa 11 o zero. T he
calculation m ethod w ith t he m inimum nor malized P M va lue w ill be c hosen a nd t he
corresponding normalized PM value will be selected as the best measurement allowance for PM,
assuming it va lidates. If it doe s not va lidate, then the mini mum va lue that va lidates w ill be
chosen as long as it is within 0.0075 g/hp-hr from the minimum value that did not validate. If the
difference b etween the minimum va lue tha t va lidates a nd the mini mum va lue tha t di d not
validate i s greater than 0.0075 g/hp-hr, additional i nvestigation with up t o a $100,000 w ill be
spent in order to understand why the minimum value chosen did not validate. If the problem is
not r esolved a fter s pending t he $100,000, t hen t he m atter w ill be r eferred to executive
management of EPA and EMA to decide on the PM measurement allowance.
Table 2 below i llustrates t he s election of t he c alculation m ethod. T he example i s ba sed on a
hypothetical s et of nor malized PM me asurements for the th ree calculation m ethods. The
minimum of these normalized allowances is used to select the best method (highlighted in blue).
In this hypothetical case, the BSFC method would be selected.
TABLE 2. EXAMPLE OF SELECTION OF MEASUREMENT ALLOWANCE AT 0.02
G/HP-HR NTE THRESHOLD
Calc. Method ==>
BSPM
Selected Method==>
Allowance at Respective NTE Threshold (%)
Torque-Speed
38%
BSFC
18%
ECM fuel specific
N/A
BSFC Method
Therefore, 18% would be selected as the be stm easurement al lowance for PM, assuming it
validates. Otherwise, the 38 % will be chosen if it validates. T hus, the additive brake-specific
measurement allowance would be:
PM = 18 % * 0.02 g/hp-hr = 0.0036 g/hp-hr, if it validates, and if not, then:
REPORT 03.14936.12 A-12
-------
PM = 38 % *0.02 g/hp-hr = 0.0076 g/hp-hr, if it validates, and if not, then:
spend up to a $100,000 to figure out why it did not validate in the first place, and then apply the
above strategy again, assuming the value now validates. If not, then EPA and EMA executive
management will decide on the PM measurement allowance value.
This PM value would be the value added to the actual brake-specific NTE threshold for a given
engine, based on actual family emissions limit, mileage, model year, etc.
2.3 METHODS AND MATERIALS
Exercise the model using three different calculation methods: a) Torque-Speed method, b) BSFC
method, and c) ECM-Fuel Specific method (only for AVL MSS). Determine which calculation
method is the most accurate, and use it to estimate the measurement allowance. Each calculation
method is described in Figured 2, 3, and 4.
Prepare an Excel spreadsheet model for use with the Crystal Ball Monte Carlo software for error
analysis of brake specific emissions, BSE, as outlined in section 2.4. C hanges t o t he m odel
specifications m ay b e requested as a greed up on by t he S teering C ommittee. Prepare t he
spreadsheet in a modular structure following the specified model outline, and make provisions
for the identified calculation modules. Additionally, clearly identify and easily locate input cells
to the model to facilitate any revisions that may become necessary for users who want to exercise
the model with other Monte Carlo add-ins such as @Risk or the newest versions of Crystal Ball.
Test the spreadsheet with controlled test cases of simplified input distributions with the Crystal
Ball add-in to confirm correct model implementation in accordance with this test plan. Run at
least one typical analysis as an additional confirmation.
Deliver t he electronic s preadsheet and a br ief r eport de scribing t he m odel, pr esenting t he t est
cases, and describing pertinent information including the Crystal Ball version number, the Excel
version number, the operating system and the computer. Use standard spreadsheet calculations
so that no serious di fficulties w ill be anticipated r egarding a pplication i n ot her s preadsheet
versions. Use Crystal Ball Version 7 or higher, and confirm test cases using Excel 2003.
Control revisions of the spreadsheet model using descriptive file names. Extensive revisions or
testing with other software versions beyond that initially proposed may be re-proposed by the
Steering Committee if and when a need for such additional work is identified.
2.4 SIMULATION PROCEDURE
For each of the measurement errors in Section 3, create an error surface and sample it according
to the aforementioned PDFs. Each error surface represents an additive erroror a subtractive
error if the sign i s negativerelative to the reference value to which it i s applied. F igure 5,
Figure 6, and Figure 7 serve as a hypothetical PM example of how these error surfaces should be
created f or e very e rror. T he pi ots s hown c orrespond t o PM emissions c oncentration da ta
representing 1 PEMS, two engines, and three exhaust configurations each, with all 6 s ets of
PEMS data pooled together. Note that separate error surfaces will be constructed for each of the
three P EMS uni ts ( AVL, H oriba a nd S ensors). The e xample a pplies t o the e rror m odule for
REPORT 03.14936.12 A-13
-------
steady-state (SS) bias and precision PM concentration errors (Section 3.2). These figures will
be referenced by each "Data Analysis" section for the various errors discussed in this test plan.
70
v> *n
^ 40
B>
3.
I 30
20
10
<
Error Surface for SS PM Concentration
69
X64-''
'
4 /^
45 / ~/r
J
-------
0.5
m
o' '
-0.5
Error Surface for SS PM Concentration
Error Surface: z-axis = ASS_PM_ug/mole
10.1 6.8 8.8 6. 7.2 11.1
346234
) 10 20 30 40 50 60 70
-4.1 -3.1 1.8 -2.2 0.2 -0.2
PM ug/mole (lab, mean)
Ert*U *'l / J-
FIGURE?. ERROR SURFACE: FINAL VERSION
Errors from Section 3 ( Engine D ynamometer Laboratory t ests) and S ection 4 ( Environmental
Chamber Tests) are combined by adding all of the sampled errors once per NTE event trial. For
example, in order to assess the errors in PM concentration for each NTE event, several modules
will be created such that:
PM_with errors = PM _ideal + A(ug/mole)i + A(ug/mole)2 + A(ug/mole)3 + ...
where,
A(ug/mole)i = PM concentration errors due to steady state bias and precision errors,
A(ug/mole)2 = PM concentration errors due to ambient temperature,
A(ug/mole)3 = PM concentration errors due to ambient pressure,
etc....
2.4.1 Construction of the Error Surface
2A.I.I PEMSvs. Lab
Acquire raw data with the PEMS at various average concentration levels as per Section 3.2. Plot
the "P EMS" s ignals ve rsus t he c orresponding "lab" s ignals t hat w ere m easured us ing 1 ab
equipment. This plot pools all bias and precision errors for one PEMS and for all data from all
engines for all steady-state modes. Shown in Figure 5 are the 5*, 50* and 95* percentiles at the
mean PM concentration level from the lab (note that the distribution of data at each level is not
necessarily Gaussian). If the 50th percentile is di fferent tha n the line of p erfect a greement
(diagonal), the data suggests that there is a bias error between PEMS and Lab. In essence this
graph shows the statistical di stribution m easured b y the PEMS at each average concentration
REPORT 03.14936.12
A-15
-------
level sampled. The example shows only 6 discrete PM concentration levels (ranging from 10-60
ug/mole). However, the actual number of discrete levels will be determined by the total number
of operating conditions actually run for all the tests of all the engines. For example, the SS PM
testing will select 6 modes representing typical operating conditions. Thus, the actual plot for SS
PM will likely have 36 discrete concentration levels (6 modes x 1 PEMS x 2 engines x 3 exhaust
configurations).
2.4.1.2 (PEMS - Lab) vs. Lab
The plot in Figure 6 basically shows the "additive error band" measured during testing. The plot
is created by first subtracting the "lab" PM value from the corresponding individual PEMS PM
measurement for each test run. This difference is defined as the 'delta' error. Next, the "PEMS
- Laboratory" delta errors are pooled at each average lab PM value to obtain the 95*, 50*, and
5th percentile values, respectively, displayed in Figure 5. Notice that if lab error exceeds PEMS
error at a given percentile, crossover o f va lues can occur. T his i s ac ceptable b ecause t he
crossover effectively reduces PEMS error whenever lab error exceeds PEMS error.
In order to obtain estimates of the 1st and 99th percentiles for the delta errors for a given "lab"
PM value, each side of the corresponding error distribution will be assumed to independently fit
a normal di stribution. Because of the asymmetry of the data, this methodology will yield two
halves of a normal di stribution. The m edian of e ach normal di stribution will be the m edian
based on the delta errors given in Figure 6. The 95th percentile delta error will form the upper
boundary of one half of the normal distribution, and the 5th percentile delta error will form the
lower bounda ry of t he ot her ha If of t he nor mal di stribution. W hen e ach s ide of t he da ta
distribution is fitted to a normal distribution using the above boundary conditions, one can then
expand each half of the distribution from the error surface to obtain the 1st and 99th percentiles of
the data for the given "lab" PM value.
2.4.1.3 Error Surf ace
This step normalizes the data in Figure 7 using what is called a "variability index (ic)", which
represents the random sampling by the Monte Carlo technique, in order to select a given error
level. This variability index is allowed to vary from -1 to +1. The likelihood of ic being any
value between -1 through +1 is specified by the PDF assigned to ic. In the given example, ic is
assumed to vary according to a normal di stribution during Monte C arlo calculations. T his i s
because it is believed that the distribution of errors due to steady-state bias and precision will be
centered about the 50* percentile of the full range of conditions measured according to Section
3.2. The pressure and temperature environmental error modules use uniform probability density
functions for their respective variability index. Each set of data for each lab set-point mean (i.e.,
lab reference value) in Figure 6 i s normalized by aligning the 1st percentile error from the fitted
normal distributions with ic = -1, the 50* percentile error with ic = 0, and the 99 percentile error
from the fitted normal distribution with ic = +1.
Error surfaces such as the one presented in Figure 7 are the input modules that the Monte Carlo
simulation program will use during calculations of brake-specific PM emissions. For example,
for a given NTE calculation a random ic value is chosen once per NTE event trial. Let us assume
that the first random sample produced an ic = 0.5. Let us also assume that during this NTE event
trial, the reference PM concentration is 10 ug/mole. In this case,
REPORT 03.14936.12 A-16
-------
A(ng/mole)i = (3 + 10.1) / 2 = 6.55 ug/mole.
Also, from Figure 7, for ic = 0.5, the reference PM =10 ug/mole.
For that step in the calculation, the Monte Carlo approach will add this "delta" to the reference
concentration value of 10 ug/mole (10 ug/mole + 6.55 ug/mole = 16.55 ug/mole) to represent
errors in steady-state bias and precision for ic = 0.5, and reference NTE PM = 10 ug/mole. If
during the same NTE event in the reference data set, a reference concentration of 35 ug/mole is
read, then,
A(ug/mole)i = ((6 + 8.8) / 2 + (2 + 6.2) / 2) / 2 = 5.75 ug/mole (from Figure 7)
Note that first the error along the i c line perpendicular to the i c axis (in this case the line along
0.5) is linearly interpolated at each discrete concentration level. Then those interpolated values
are t hemselves 1 inearly interpolated to determine t he e rror corresponding t o each reference
concentration in the NTE event. Note that the random selection is once per reference NTE event
trial, but the error along that ic line is applied to every second-by-second value within the given
reference NTE event, except for PM concentration in the case of Horiba and Sensors, where no
second-by-second i nformation a re a vailable, b ut di fferent P M c oncentration 1 evels m ay be
available for a specific NTE event.
Now let us assume that the error in PM concentration is composed of only 3 deltas: A(ug/mole)i,
A(ug/mole)2, and A(ug/mole)3. And let us assume that for a given reference NTE event trial
we have the following values:
Reference PM at one second= 30 ug/mole
A(ug/mole)i = 6 ug/mole
A(ug/mole)2 = -2 ug/mole
A(ug/mole)3 = -3 ug/mole.
When the model calculates brake-specific emissions by each of the three calculation methods, it
will use the following PM value, which has all of its error applied:
PM = 30 + 6 -2 - 3 = 31 ug/mole.
The appl ication of error at the first s elected ic continues during the entire NTE event without
having to randomly sample again. In other words, i c will not change during that random trial.
For all of the variables except for mPM , the errors may continue to change during an NTE event
on a second-by-second basis if their error surface happens to be a function of level. For the
second randomly selected ic this entire process of determining the Aug/mole errors is repeated.
The s imulation will c ontinue to randomly s elected ic values f or t housands of t rials unt il
convergence is met.
For the Horiba and Sensors generated reference NTE events, there is only one flow-weighted PM
value for the entire NTE event. D uring the simulation for these types of reference NTEs, the
single PM value will be used in the interpolation of the corresponding PM error surfaces (i.e.,
REPORT 03.14936.12 A-17
-------
steady-state PM, transient PM) at all seconds of the reference NTE event. Since the PM value
will not vary from second-to-second, the only interpolation will occur according to the ic value at
each of the simulation trials.
The same second-by-second sampling and interpolation approach would be used for other deltas
such a s a mbient t emp, a mbient pr essure, s hock a nd vi bration, B SFC i nterpolation, t orque,
exhaust flow r ate, e tc. An ove rview o f t he Monte C arlo s imulation f or P M i s de tailed i n
Figure 8.
Reference NTE
co%
NMHC
(ppm)
Exhflow
(scfm)
Torque
(N-m)
Speed
(rpm)
Fuel
Rate
(L/sec)
Monte-Carlo Simulation
c_PM
c CO
1c_MMHC
'c_Exhflow
c_Torque
'c_Speed
1c_Fuel Rate
lc_CO2
(1) BSPM = f (PM, Exhflow, Torque, Speed)
(2) BSPM = f (PM, Exhflow, BSFCECM)
(3) BSPM - f (PM, C02, CO, THC, Torque,
Fuel RateECM, Speed)
J
'Differences = BSPM "with errors" - "Ideal" BSPM
FIGURE 8. OVERVIEW OF MONTE CARLO SIMULATION
REPORT 03.14936.12
A-18
-------
Table 3 lists the e rror s urfaces tha t w ill be c reated for us e in simulating the B SPM e rror
differences.
TABLE 3. ERROR SURFACES FOR THE BSPM SIMULATION
Calculation
Component
Delta PM
Delta PM
Delta PM
Delta PM
Delta PM
Delta PM
Delta CO
Delta CO
Delta CO
Delta CO
Delta CO2
Delta CO2
Delta CO2
Delta NMHC
Delta NMHC
Delta NMHC
Delta NMHC
Delta NMHC
Delta Exhaust Flow
Delta Exhaust Flow
Delta Exhaust Flow
Delta Exhaust Flow
Delta Exhaust Flow
Delta Exhaust Flow
Delta Exhaust Flow
Delta Torque
Delta Torque
Delta Torque
Delta Torque
Delta Torque
Delta Torque
Delta Speed
Delta Fuel Rate
Test Source
Engine Dyno
Engine Dyno
Environ
Environ
Environ
Environ
Engine Dyno
Environ
Environ
Engine Dyno
Engine Dyno
Engine Dyno
Environ
Engine Dyno
Engine Dyno
Environ
Environ
Environ
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Environ
Environ
Environ
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Engine Dyno
Engine Manuf
Engine Dyno
Engine Dyno
Error Surface
Delta PMSS
Delta PM Transient
Delta PM Ambient Temperature
Delta PM EMI/RFI
Delta PM Atmospheric Pressure
Delta PM Vibration
Delta CO SS
Delta CO Atmospheric Pressure
Delta CO Ambient Temperature
Delta CO Time Alignment
Delta CO2SS
Delta CO2 Transient
Delta CO2 Ambient Temperature
Delta NMHC SS
Delta NMHC Transient
Delta NMHC Atmospheric Pressure
Delta NMHC Ambient Temperature
Delta Ambient NMHC
Delta Exhaust Flow SS
Delta Exhaust Flow Transient
Delta Exhaust Flow Pulsation
Delta Exhaust Flow Swirl
Delta Exhaust EMI/RFI
Delta Exhaust Temperature
Delta Exhaust Pressure
Delta Dynamic Torque
Delta Torque DOE Testing
Delta Torque Warm-up
Delta Torque Humidity/Fuel
Delta Torque Interpolation
Delta Torque Engine Manuf
Delta Dynamic Speed
Delta Dynamic Fuel Rate
REPORT 03.14936.12
A-19
-------
2.5 MODEL CONSIDERATIONS
2.5.1 Convergence
The main goal of the convergence criteria is to define how many simulation trials at a given
reference NTE event are required to estimate the 95th percentile BSPM emission differences with
a given precision. T he convergence method to be used i s based on a nonparametric statistical
technique3 which de fines a 90% confidence i nterval f or t he 95 * percentile of t he BSPM
emissions di fferences for an individual r eference N TE s imulation. I f t he w idth of t he 90%
confidence interval is less than 1% of the BSPM emissions threshold, then convergence is met.
The following steps define the convergence method:
1. Run the Monte Carlo simulation for TV trials for a single reference NTE event.
2. Order the BSPM emissions differences from smallest to largest.
3. Identify the trial number at the lower end of the 90% confidence interval
niower = 0.95 * N - 1.645^0.95 * 0.05 * N
4. Identify the trial number at the upper end of the 90% confidence interval
riupper = 0.95 * N + 1.645V0.95 * 0.05 * N
5. Compute (BSPM difference value at nupper) - (BSPM difference value at niower).
6. If the result in (5) < 1% of the BSPM emissions NTE threshold (0.02 g/hp-hr) then
convergence is met.
2.6 SIMULATION OUTPUT
It is i mportant t o unde rstand a nd i dentify w hat e rror s urfaces ha ve t he m ost influence (i.e.,
sensitivity) on t he B SPM e missions ' with e rrors' a nd, t hus, t he resulting B S e missions
differences. Contributions to sensitivity can be attributable to changes in variance and/or bias.
2.6.1 Sensitivity Variation Effect
During the Monte Carlo simulation for each reference NTE event, sensitivity charts produced by
Crystal B all w ill be generated and stored in output REPORT files. Crystal B all c alculates
sensitivity b y computing t he r ank c orrelation coefficient be tween every assumption ( error
surface) and forecast value (delta BS emissions) while the simulation is running. Positive rank
correlations i ndicate that an increase in the assumption is as sociated with an increase i n t he
forecast. The larger the absolute value of the rank correlation the stronger the relationship.
Sensitivity charts in Crystal Ball provide a means to determine how the variances of the error
surfaces affect the variance in the forecast values. Hence, the sensitivity charts developed during
a simulation are displayed as "Contribution to Variance" charts which are calculated by squaring
the r ank c orrelation coefficients for all a ssumptions us ed i n a pa rticular f orecast a nd t hen
normalizing the m to 100%. T he assumption ( error s urface) w ith t he hi ghest c ontribution t o
variance (in absolute value of the percent) is listed first in the sensitivity chart.
Simulation results from all reference NTE events will produce sensitivity values for the 95
percentile delta PM emissions by all three calculation methods.
REPORT 03.14936.12 A-20
-------
2.6.2 Sensitivity Bias Effect
Another type of sensitivity to be examined in this study is concerned with the effects of potential
"bias" in error surfaces and their effects on the forecast values. In order to study these effects a
new error surface assumption will be added to the simulation model for each of the original error
surfaces.
This assumption will be sampled as a discrete binary distribution (i.e., on or off) during the
simulation. For each trial of the simulation, the original error surfaces and 'on/off error surfaces
will be s ampled a ccording tot heir de fined s ample di stribution. I f t he ' on/off e rror s urface
produces an 'off condition, the delta emissions from that particular error surface will not be
added to the BSPM emissions computations for the BSPM emissions 'with errors'. Similarly, if
the ' on/off error surface produces an ' on' condition, the delta emissions from that particular
error surface will be added to the BSPM emissions calculations.
During e very t rial of t he s imulation, t he exclusions due t o t he ' off c onditions w ill r esult i n
various combinations of the error surface delta emissions being added to the B SPM emissions
'with errors' computations. O vert he course of a simulation with thousands of tr ials, the
sensitivity of a particular error either ' on' or 'off will be assessed by examining the change in
the forecast delta emission. Therefore, in a single Monte Carlo simulation of a reference NTE
event sensitivities due to variance and/or bias will be explored.
3 ENGINE DYNAMOMETER LABORATORY TESTS
Utilize e ngine d ynamometer laboratory te sting t o establish the di fference be tween PM P EMS
and PM based on laboratory measurement in accordance with Part 1065. Also establish how well
ECM parameters can be used to estimate torque and BSFC.
First, however, audit all the PEMS and lab equipment to ensure that they are operating properly,
according t o 40 C FR P art 1065, S ubpart D. N ext, c onduct s teady-state engine d ynamometer
tests to establish PEMS s teady-state bi as a nd pr ecision r elative t o t he 1 ab. T hen, c onduct
transient e ngine d ynamometer te sting to determine P EMS tr ansient pr ecision by r epeating
transient NTE events. Finally, compare ECM derived torque and BSFC to laboratory measured
torque and BSFC.
3.1 PRELIMINARY AUDITS
3.1.1 Objective
Conduct 40 CFR Part 1065, Subpart D audits of all engine dynamometer laboratory systems and
all PEMS.
3.1.2 Background
Because the overall purpose of this entire test plan is to establish measurement allowance that
account for the incremental difference in the performance of PEMS versus engine dynamometer
laboratory systems, the first task is to audit all of the measurement systems to ensure that the
REPORT 03.14936.12 A-21
-------
specific systems used for testing meet EPA's minimum performance requirements. T he audits
also help to minimize bias errorsbetweenPEMS and lab systems measurements. However, in
case a specific PM-PEMS does not meet the specifics of Part 1065 requirement, the MASC will
decide on how to move forward by perhaps allowing some flexibility in passing Part 1065 audit,
in situations where it mi ghtbe needed, especially if the performance of a system is within the
expectation of the manufacturer.
3.1.3 On-site meeting to establish 1065 compliance requirements
In order to clarify what are all the requirements expected from the lab-grade instrumentation and
PEMS equipment, with respect to 1065 compliance, a meeting will be held between the test plan
steering c ommittee a nd the c ontractor at the c ontractor s ite to provide t he c ontractor w ith
guidance regarding which specific sections of Part 1065 S ubpart D are required and which are
optional. In case Part 1065 requirement i s demonstrated to be too stringent or impractical, the
contractor may seek approval from the MASC to lessen the stringency of Part 1065 in relation to
the PEMS.
3.1.4 Methods and Materials
Use the methods and materials described in 40 CFR Part 1065, Subpart D to conduct audits of all
lab and PEMS measurement systems. E ven if lab systems and PEMS pass initial S ubpart D
audits, allow lab operators and PEMS manufacturers to make on-site adjustments to improve the
performance oft heir s ystems pr ior t o e ngine t esting. A How a djustments t o be ba sed on
recalibrations w ith reference s ignals t hat a re a llowed in 40 C FR P art 1065. T he s teering
committee may direct the contractor to calibrate or adjust the laboratory sampling system based
on audit results. The steering committee may also suggest that a PEMS manufacturer calibrate
or adjust one or more PEMS based on lab audits.
3.1.5 Data Analysis
Use the data analyses described in CFR Part 1 065 SubpartsD, J and G. F or all subsequent
testing, use onl y thos e measurement s ystems tha t pa ss the mini mum p erformance c riteria in
Subpart D, unless a deficiency i s deemed acceptable in writing by all parties including PEMS
manufacturers. Provide a list and brief description of all the audits conducted for each PEMS
manufacturer type. EPA would likely use this list as a template for the data requirements in the
PM portion of the HDIU testing program.
3.1.6 PEMS Manufacturer PM PEMS Commissioning
Notify PEMS manufacturers when the 1065 audits are complete and the first set of PM PEMS
are completely ins tailed in the e ngine d ynamometer te st c ellin pr eparation f or emissions
testing. Schedule dates and times that are prior to the start of emissions testing for each PEMS
manufacturer to conduct a final commissioning of all their PEMS that are on site, including those
PEMS that are not installed in the test cell. PEMS manufacturers may inspect their PEMS and
make a ny final a djustments to their r espective PEMS in order for the P EMS to meet the ir
specifications. Allow PEMS manufacturers to inspect the installation of their PEMS in the test
cell. If P EMS m anufacturers t ake exception to any portion of t he i nstallation or on -site
configuration, a ttempt tor esolve any s uch i nstallation i ssues. If s uch i ssues a re not e asily
REPORT 03.14936.12 A-22
-------
resolvable, notify the steering committee, who will determine a course of action. Once PEMS
manufacturers have completed their commissioning, notify the steering committee. From this
point any further modifications to the PEMS may only be made according to Table 1 of this test
plan.
3.2 BIAS AND PRECISION E RRORS UNDE RS TEADY S TATE EN GINE
OPERATION
3.2.1 Objective
Evaluate the bias and precision using one engine and one exhaust configuration, shown in Table
4, and 10 r epeats of steady-state modes, and three sets of PEMS units, each set including the
MSS, TRPM, and PPMD. Thus, the total number of NTE steady-state points required to conduct
the steady-state experiments is 30. This constitutes six steady-state modes of engine operation
(6), 10 r epeats (10), one exhaust configuration, one engine (1), and three different PEMS units
(3), 6x10x1x1x3= 180.
TABLE 4. ENGINE, EXHAUST CONFIGURATION, AND STEADY-STATE MODES
07 Engine
1
No. of Steady-State Modes
for Bypass Setting 1
(BSPM and PM
Concentration,
representative of PM
threshold of 0.025 g/hp-hr
under NTE Transient
Operation)
SSI, SS2, SS3, SS4, SS5,
SS6
PM-PEMS Units
Three Sets of (MSS,
TRPM, and PPMD)
Number of Repeats
10 per Mode per PM-
PEMS Set
Determine the AssmPM MM surface pi ots for the error m odel based upon a 11 data pool ed. Note
that e ach br and of P EMS w ill ha ve i ts ow n AsswPM error s urface generated for us e i n bot h
calculation methods 1 and 2. For calculation method 3, the AVL brand PM PEMS will have a
unique Ass/wPM calculated according to Figure 4 of this test plan.
Recommend six steady-state points based on the PM measurement, using the AVL MSS, of 80
SS points of the Cummins cycle that is typically used to generate ECM torque and BSFC errors
versus laboratory. The MASC will accept the six steady-state points or choose alternative points
for each exhaust configuration. The obj ective for the MASC will be to select steady-state points
within a given exhaust configuration that provides a nominal spread of concentrations within that
configuration's target brake-specific levels. Note that to achieve the brake-specific targets under
steady-state conditions, the bypass might have to be opened further, relative to the transient NTE
bypass settings.
REPORT 03.14936.12
A-23
-------
3.2.2 Background
Testing will be conducted to capture bias and precision errors in PEMS' emissions instruments
versus the laboratory filter-based method. The tests will be steady-state only.
Note: S ection 3.3 ( next s ection) will e valuate pr ecision e rrors (not bi as) due to the d ynamic
response of the PEMS instrumentation. The precision error captured during steady state testing
(section 3.2) will have to be subtracted from the overall precision error captured in section 3.3 in
order not t o doubl e-count t he s teady s tate precision e rrors of P EMS i nstrumentation. T his
process is detailed in Section 3.3.
3.2.3 Methods and Materials
Use the following systems:
a) One model year 2007 heavy duty diesel engines, equipped with a DPF in the exhaust (Mack
MP9)
b) Nine PM PEMS (3 Sensors PPMD, 3 AVL MSS, 3 Horiba TRPM)
c) One PEMS exhaust flow-meter from Sensors, Inc., and one and from Horiba, applicable to
the engine to be tested
d) DPF with Bypass Setting 1 for SS testing, representing a threshold level of about 0.025 g/hp-
hr under NTE transient testing
Use the following overall guidelines:
e) Measure PM via the CVS, Part 1065 Lab Method (most recent publication)
f) Measure engine inlet airflow through use of LFE or equivalent
g) Use a series of six steady-state modes, and set each mode time to collect a CVS filter mass of
at least 75 microgram per mode, simultaneously with other PM-PEMS
h) Regenerate DPF system prior to each series of steady-state tests
i) Capture ECM broadcast channels and other common diagnostic channels, as recommended
by engine manufacturer(s), to ensure proper engine operation
j) Do not measure gaseous species by the PEMS
k) Stabilization time =180 seconds, with a different running time per mode to achieve a 75
microgram or higher of PM on the CVS filter
1) Always power off PEMS equipment at end of each day, according to PEMS manufacturer
instructions. Re-start start-up process every day according to PEMS manufacturer
instructions and Part 1065, Subpart J.
m) Whenever PEMS are exchanged, swap the order of the Horiba and Sensors flowmeters, if the
steup allows for it.
6 point steady-state repeat-testing, evaluate bias and precision errors:
a) The MASC will select 6 SS operating conditions for repeat testing from a matrix of 80 SS
points containing information on PM emissions using the AVL MSS
b) Randomize the order of the six modes
c) Repeat each six steady-state cycle two or three times, prior to DPF regeneration
d) Each test will use three PEMS (Sensors, AVL, and Horiba) at a time, to measure PM
emissions concentration and exhaust flow rate.
e) Expected test duration is 5 days per PEMS set, with a total of 15 days for all three sets.
REPORT 03.14936.12 A-24
-------
Bypass Setting:
a) Run NTE transient cycle using the CVS filter-based method
b) Set bypass to produce CVS filter-based average brake-specific of about 0.025 g/hp-hr
c) Determine the average PM mass concentration
d) Run the 80 S S Cummins cycle to capture PM concentration at each mode using the AVL
MSS
e) Check the PM concentration levels and select the six-steady state modes from the 80 poi nt
matrix. As a first order, check the concentration at the pre-selected steady-state modes to see
if they spread within reason around the concentration produced for the NTE transient cycle.
If not, adjust the bypass as needed to establish the right spread in brake-specific emissions
and concentration for the six steady-state modes
f) Make sure that the points selected spread around a brake specific level and concentration
level of a threshold of 0.025 g/hp-hr, and concentration range of 4 to 15 milligram per cubic
meter.
3.2.4 Data Analysis
Use the acquired data to create the "error surfaces" to be used by the Monte Carlo simulation.
An example of the steady-state error surface determination is shown in Table 5 for PM.
TABLE 5. EXAMPLE OF SS ERROR SURFACE
Error Surface for SS PM Concentration
Figure 5
x-axis
y-axis
PM ug/mole (lab mean at setpoint)
PM ug/mole (PEMS)
Figure 6
x-axis
y-axis 5th percentile
y-axis 50th percentile
y-axis 95th percentile
PM ug/mole (lab mean at setpoint)
5th [PM ug/mole (PEMS) - PM ug/mole (lab)]
50th [PM ug/mole (PEMS) - PM ug/mole (lab)]
95th [PM ug/mole (PEMS) - PM ug/mole (lab)]
The 5th, 50th and 95th percentiles from the (PEMS - lab) delta data will be used to
estimate the 1st and 99th percentiles from assumed Gaussian distributions.
Figure 7
x-axis
y-axis
z-axis =
ASS_PM_ug/mole
ic sample frequency
ic sample distribution
PM ug/mole (lab mean at setpoint)
ic SS PM
1st Percentile from Gaussian distribution based on 5th and
50th [PM ug/mole (PEMS) - PM ug/mole (lab)] deltas.
99th Percentile from Gaussian distribution based on 50th and
95th [PM ug/mole (PEMS) - PM ug/mole (lab)] deltas.
50th Percentile based on [PM ug/mole (PEMS) - PM ug/mole
(lab)] deltas.
once per NTE event trial
Gaussian (normal distribution)
REPORT 03.14936.12
A-25
-------
3.3 PRECISION ERRORS UNDER TRANSIENT ENGINE OPERATION (DYNAMIC
RESPONSE)
3.3.1 Objective
The objective of this portion of the work is to determine the precision error, ATR/wPM , with each
PM-PEMS under NTE transient engine operation. This will be achieved by creating a 20 to 25-
minute transient NTE cycle where the PEMS measure in each NTE.
3.3.2 Background
PEMS are expected to operate in a repeatable manner over NTE events as short as 30 s econds.
Two sources of PEMS precision error are hypothesized: 1) dynamic response to rapidly changing
signals, and 2) susceptibility to "history" effects. Dynamic response error includes error due to
measurement signal time alignment, and the dissimilarity of the dynamic response and aliasing
of signals; including those signals used to determine entry into and exit from the NTE zone.
History effects i nclude the e ffects of p reviously measured qu antities on c urrently m easured
quantities. F or e xample, this m ay be c aused b y i neffective s ample ex change in the P M
emissions sampling volumes, or it may be caused by one or more sensors' characteristic rise time
or fall time. To account for any dynamic response precision error, the increase in precision error
incremental to the steady-state emissions measurement precision will be incorporated into the
overall error model.
Selection of short NTE cycles (each 32 seconds) maximizes the sensitivity of this test to effects
of dynamic response. Thirty-two seconds was chosen as the minimum instead of thirty seconds,
which i s the shortest NTE event time, to ensure that 1 H z ECM updating of torque and speed
values w ould be unl ikely to i nterfere with c apturing N TE e vents. For each r epeat of t he t est
cycle, the order of the 3 0 differentNTE events will be the same. In addition the 29 di fferent
intervals separating each NTE event from the next will have a range of durations and these will
be randomly arranged in each test cycle as well. Fixed arrangement of the NTE events and the
inter-NTE events w ill maximize the s ensitivity of thi s te st to dynamic r esponse a nd history
effects, and make the DPF and bypass operation very consistent.
The total length of the NTE transient cycle will assume that only 5 quartz crystal of the Sensors
PPMD are working, and it takes five minutes of stabilization time for reusing a crystal after PM
collection. Thus, the same NTE transient cycle used in the gaseous PEMS program will be used
here, except for changes in the inter-NTE times to accommodate the Sensors PPMD.
3.3.3 Methods and Materials
a. Use a transient engine dynamometer emissions laboratory.
b. Use a 1 aboratory that can accommodate at 1 east three PEMS, their power supplies, the
PEMS flow meters, cables and lines.
c. Use s ame over all guidelines described in section 3.2, but applied t o t ransient e ngine
testing.
d. Record the EEPS' total mass signal during transient testing.
REPORT 03.14936.12 A-26
-------
Challenge PEMS to 30 different 32-second NTE events, shown in Table 5, over about 23 minute
test cycle, or whatever needed to accommodate the need for five crystals of the PPMD to be
operational. Randomize the NTE events shown in Table 6 once, scale up every fifth inter-NTE
time, shown in Table 7, to accommodate the PPMD, and use the same order for repeat testing.
Repeat the test cycle 10 times for each set of three PEMS. Note that for any torque command
that is less than zero, command closed throttle (i.e. zero or minimum fuel command), and motor
the engine at the commanded speed for that data point. An example of an NTE transient cycle is
shown in Figure 9.
Based on 10 repeats with each set of PEMS, the total number of repeats will be 30 cycles,
assuming 1 NTE cycle x 10 repeats x one exhaust configuration x 3 sets of PEMS x one engine
(1x10x1x3x1 = 30). Assuming a 25 minutes of NTE with 30 minutes of forced regeneration and
preparation for the second repeat, the total number of days for NTE transient testing is 10 days (8
hours per day). This time includes PEMS and engine setup, PEMS warm up, and daily checks.
Prior to executing the first repeat, setup each PEMS and stabilize engine operation at the first
inter-NTE ope rating poi nt. S etup t he P EMS a ccording t o 40 C FR P art 1065 a nd P EMS
manufacturer instructions, including any warm-up time, zero-spans of the analyzers and the setup
of all accessories including flow meters, ECM interpreters, etc. Then, when the test cycle starts,
switch the PEMS' to sample emissions from the engine. When the text cycle ends, switch the
PEMS back to ambient sampling. Complete all post-test lab and PEMS validations according to
40 CFR Part 1065 and according to PEMS manufacturer instructions.
3.3.4 Data Analysis
Discard from further data analysis any NTE events invalidated by any criteria in 40 CFR Part
1065 Subpart J. For each NTE; event (i=l to 30), which was repeated 30 times per engine with a
specific exhaust configuration (j = 11 o 30), calculate the transient median absolute deviation,
MADiRi, for thpM, where for each NTE; event, MADiRi = median[| NTE;j - median (NTE;j) |].
Next cal culate t he di fference of M AD b y s ubtracting a cor responding s teady-state M AD,
MADssi for mPM . MADTRi-ssi =MADTRi - MADSsi. To determine a corresponding MADSsi,
calculate t he P EMS M ADSs at each steady-state me dian lab va lue, a nd t hen us e t he m edian
PEMS N TE; value a long t he m edian 1 ab va lue's a xis t o find M ADssi for t he c orresponding
MADiRi . D o not extrapolate any MAD ssi beyond the mini mum or ma ximum me dian lab
values. Note that some MADss; values might be zero because the lab data for that median failed
the F-test in the previous section.
For any MADxRi-ssi less than zero, set that MADxRi-ssi equal to zero.
Create a transient error surfaces using all of the MADTRi-ssi- Be sure to include any MADTRi-ssi
data points that are equal to zero because they will affect the 1st and 99th percentile values.
REPORT 03.14936.12 A-27
-------
TABLE 6. NTE TRANSIENT CYCLE
NTE
Event
NTE1
NTE2
NTE3
NTE4
NTE5
NTE6
NTE7
NTE8
NTE9
NTE 10
NTE 11
NTE 12
NTE 13
NTE 14
NTE 15
NTE 16
NTE 17
NTE 18
NTE 19
NTE20
NTE21
NTE22
NTE23
NTE24
NTE25
NTE26
NTE27
NTE28
NTE29
NTE30
1 Speed % Range
17%
59%
Governor line
17%
59%
Governor line
17%
59%
100%
Lower third
Upper third
Middle third
17% -governed
17% -governed
17% -governed
2Torque % Range
332%
332%
332%
66%
66%
66%
100%
100%
100%
332% - 100%
332% - 100%
332% - 100%
Lower third
Upper third
Middle third
Lower right diagonal
Upper left diagonal
Full diagonal; lower left to upper right
Lower left diagonal
Upper right diagonal
Full diagonal; lower right to upper left
Third light heavy-duty NTE event
from International, Inc. data set
Cruise; ~ 50 mph
Cruise; ~ 75 mph
Small bulldozer
Large bulldozer
Second of three NTE events in FTP
Third light heavy-duty NTE event
from International, Inc. data set
First of two NTE events in NRTC
First of two NTE events in NRTC
Description
Steady speed and torque; lower left of NTE
Steady speed and torque; lower center of NTE
Steady speed and torque; lower right of NTE
Steady speed and torque; middle left of NTE
Steady speed and torque; middle center of NTE
Steady speed and torque; middle right of NTE
Steady speed and torque; upper left of NTE
Steady speed and torque; upper center of NTE
Steady speed and torque; upper right of NTE
Highly transient torque; moderate transient speed
Highly transient torque; moderate transient speed
Highly transient torque; moderate transient speed
Highly transient speed; moderate transient torque
Highly transient speed; moderate transient torque
Highly transient speed; moderate transient torque
Transient; speed increases as torque increases
Transient; speed increases as torque increases
Transient; speed increases as torque increases
Transient; speed decreases as torque increases
Transient; speed decreases as torque increases
Transient; speed decreases as torque increases
Sample from LHDE
Sample from HDDE
Sample from HDDE
Sample from NRDE
Sample from NRDE
Seconds used from FTP: 714-725, 729-743, 751-
755
Sample from LHDE
Seconds used from NRTC: 423-430, 444, 448-450,
462-481, increased 464 speed from 40% to 42%
Seconds used from NRTC: 627-629, 657-664,
685-696, 714-722
1 Speed (rpm) = Curb Idle + (Speed % * (MTS - Curb Idle)
2 Torque (Ibf-ft) = Torque % * Maximum Torque At Speed (i.e. lug curve torque at speed)
3 Torque (Ibf-ft) = Maximum of (32 % * peak torque) and the torque at speed that produces (32 % * peak
power)
REPORT 03.14936.12
A-28
-------
TABLE 7. DYNAMIC RESPONSE INTER-NTE EVENTS
INT
Event1
INT1
INT2-6
INT7-10
INT11-14
INT15-18
INT 19-21
INT22
INT23
INT24
INT25
INT26
INT27
INT28
INT29
INT30
INT31
Duration
(s)
10
2
3
4
5
6
7
8
9
11
13
17
22
27
35
5
Frequency
1
5
4
4
4
O
1
1
1
1
1
1
1
1
1
1
Description
Initiation of cycle; INT1 is always first
Shortest and most frequent inter-NTE events
Short and frequent inter-NTE events
Short and frequent inter-NTE events
Short and frequent inter-NTE events
Short and frequent inter-NTE events
Medium inter-NTE event
Medium inter-NTE event
Medium inter-NTE event
Medium inter-NTE event
Long inter-NTE event
Long inter-NTE event
Long inter-NTE event
Long inter-NTE event
Longest inter-NTE event
Termination of cycle; INT31* is always last
Interval speeds and torques are not identical, but they are clustered around zero torque and the
speed at which 15% of peak power and 15% of peak torque are output.
Torque-Speed Domain
7nn
/ uu
600
500
ฃ"
.a
7T 300
g-
o
H 200
100
~
1 nn
/
/
/
Lug Curve
NTES line
NTE T-S line
O 1 SS Lower Left
O 2 SS Lower Mid
O 3 SS Lower Right
0 4 SS Mid Left
O 5SS Mid Mid
O 6 SS Mid Right
O 7 SS Upper Left
O 8 SS Upper Mid
O 9 SS Upper Right
NTE Cycle
22 LD NTES
^^23 OH SOmph Cruise
24 OH 70mph Cruise
25 NR Small Dozer
26 NR Large Dozer
-0-27FTPNTE2
28 LD NTE6
o 29 NRTC NTE1
--30 NRTC NTE2
.Q .
,/ ...'.'
ง'
::. i
Tw
\P
i'
,
'
? Q
' 8 \>
O o: '
: -0 --;
' %>
I ?!
"IA-:
&? V
^SD&'LO- >
' !y.
X
' '" t)
Q ffi^W ' . fS
1 ^Jf "?: ::.i--r
1 \i
6 ฐ 8^ '
6 :
500 1000 1500 2000 2500 3000
.0
o
\
\
\
\
\
1 1
3500 4000 45
00
Speed (rpm)
FIGURE 9. EXAMPLE OF A NTE CYCLE
REPORT 03.14936.12
A-29
-------
3.4 ECM TORQUE AND BSFC
3.4.1 Objective
Compare t he E CM-based t orque a nd fuel r ate w ith t hat of t he 1 aboratory-based measurement
using the Cummins 80 SS mode cycles. For the laboratory purposes, use the gas-based fuel flow
values instead of the measured fuel flow. Repeat the Cummins 80 S S cycle three times, and use
the average values produced.
Use at least six engines for these experiments that include the one engine to be used in the PM
PEMS program and Engine B, C, and D of the ACES program.
3.4.2 Data Analysis
Use the acquired data pooled and normalized to % of max torque and % of maximum fuel rate to
replace t he m anufacturer s ubmitted e rror s urfaces t hat w ere pr eviously us ed i n t he g aseous
portion of the Monte Carlo simulation. Refer to section 2.4 for description and example of an
error surface. Include any bias error, unless there is an assignable cause that would not occur in-
use and the steering committee approves to eliminate such bias error.
4 ENVIRONMENTAL CHAMBER
The envi ronmental cha mber t ests cha llenge P EMS t o a variety o f e nvironmental di sturbances,
namely electromagnetic interference, atmospheric pressure, ambient temperature and humidity,
and shock and vibration.
During e ach of t he t ests, pi us a ba seline t est, the P EMS w ill c ycle t hrough s ampling four
different di lution pr eparations of a erosol pa rticles t hat c ontain vol atile h ydrocarbon a nd
elemental carbon using a particle generator that mimics the formation of diesel particles. The
OC/EC w ill be us ed to determine t he cone entration levels ne eded for t he P M g enerator.
Essentially, after de termining the s teady-state p oints t o r un on t he e ngine, t he O C/EC s emi-
continuous ins trument w ill be us ed along w ith the f ilter-based m ethod. T hen, f or t he
concentration levels to be used with the PM generator, the OC/EC instrument will be used to set
the PM generator to produce the desired composition and concentration levels, similar to those
encountered under steady-state. Three particle concentration levels of 5, 10, and 15 mg/m3, as
shown in Table 8, will be generated by the particle generator. Each concentration will be fed to
the PEMS after applying dilution ratios of 6, 12, 20, and 30. For each concentration and dilution
ratio combination, the PM generator will be stabilized for 4.5 minutes, and data will be collected
by the PEMS for 30 seconds. The test will continue for a period of 8 hours. The first six cycles of
every test will serve to be the baseline before any environmental change is made.
The t emperature/humidity and pr essure t ests a re de signed t o mimic r eal-world e nvironmental
disturbances w ith t he m agnitude a nd f requency of t he di sturbances a djusted tor eal-world
conditions. Randomly sample a uniform distribution of probability for their /'c. , from any minute
of the test. By randomly sampling from the minutes of these tests the magnitude and frequency
of the real-world error will be built into the error model, which is described in Section 2. T he
other e nvironmental t ests r epresent t he full r ange o f pos sible c onditions. F or t hese t ests,
randomly sample the normal distribution in Figure 1 for their /c.
REPORT 03.14936.12 A-30
-------
TABLE 8. CONCENTRATION AND DILUTION RATIO SCHEDULE WITH PM
GENERATOR
Raw PM Concentration,
|ig/m3
5000
10000
15000
Dilution Ratio
DR1
6
DR2
12
DR3
20
DR4
30
Concentration at Above Dilution Ratio
833.3
1666.7
2500.0
416.7
833.3
1250.0
250.0
500.0
750.0
166.7
333.3
500.0
For E MI/RFI and vi bration, t he i nstruments w ill be s ubjected t o s creening t ests w ith H EPA
filtered air to detect if there any changes in the r esponse of t he i nstruments. Based on these
results, the MASC will decide if the particle generator will need to be used with these tests.
For the vibration screening test, in order to avoid damage to the instruments, a frequency sweep
will be us ed at low a mplitude. The ide a he re i s to detect th e frequency that ma y tr igger a
response by the instrument, without doing any damage due to high amplitude.
4.1 DATA ANALYSIS FOR ENVIRONMENTAL TESTS
Reduce data by first calculating means for each 30-second period of stabilized measurements.
Subtract f rom each mean the respective ba seline con centration. T he r esults ar e errors or
"deltas". C orrect e ach of t hese er ror di stributions b y r emoving their r espective b aseline
variances, which were determined by quantifying PM Generator output with no e nvironmental
perturbations. C alculate t he va riance of each of t he di stributions. S ubtractthe respective
baseline variance from each calculated variance. Use the resulting difference in variance as the
target variance for adjusting the error di stributions. If the target variance i s zero or negative,
leave all error values of the di stribution as i s and do not proceed to the next step. If the target
variance is positive, iteratively solve to find a single numerical value that can be used to divide
each error in a given distribution such that the resulting distribution has a variance equal to the
target variance. Now each of the errors is corrected for baseline variance.
Then, calculate the NTE result with all errors, including torque and flow errors set to zero. This
is the true value. Then subtract the true NTE value from the result with all errors and record this
difference i n one of the 7 m easurement allowance di stributions: OTPM times three cal culation
methods (torque-speed, fuel-specific * BSFC, ECM fuel flow) times three PEMS manufacturers,
except Sensors and Horiba can not use the ECM fuel flow calculation method. Then proceed to
the next NTE event in the nominal data set. R epeat the entire nominal data set over and over
until all 7 measurement allowance distributions converge. Follow the data reduction steps set out
in Section 2 to select the final measurement allowance.
REPORT 03.14936.12
A-31
-------
4.2 PM GENERATOR COMMISSIONING
The P M g enerator i s de veloped by E PA. The P M g enerator can create various h ydrocarbon
mixtures along with solid particle generation using carbon rods arcing. The PM generator is also
equipped with a micro-proportional diluter, and is intended to simulate diesel exhaust particle
phase compounds.
EPA will ship the PM generator to SwRI. EPA (Matt Spears) will train SwRI staff on using it. In
addition, SwRI together with EPA may incorporate to it a soot particle generation mechanism
that i s di fferent t han t he c arbon r od a rcing, using i nstead a pr opane f lame m ini-CAST
technology.
The PM generator will be used during atmospheric chamber testing, temperature and humidity
testing, and may be used during EMI/RFI and vibration experiments.
4.3 BASELINE
4.3.1 Objective
The ba seline variance will be e stablished us ing a n 8 hour ba seline t est i n w hich t he PM
generator cycles through the same compositions and concentrations of PM used during the actual
environmental tests. Mean values will be determined from the first five cycles through the PM
concentrations. D eviations (deltas) from these mean values during subsequent cycles through
the c oncentrations w ill be us ed t o d etermine the ba seline va riance. This va riance w ill be
subtracted from the environmental test results.
4.3.2 Background
All of the ot her environmental te sts inhe rently inc orporate the ba seline bi as v ariance o f the
PEMS. B ecause t he M onte C arlo s imulation m odel a dds all the e rrors de termined from the
various environmental tests, it would add the baseline variance of PEMS to the model too many
times. In order to compensate for this in the model, the baseline variance of PEMS is determined
and subtracted from each of the environmental tests' results.
Note that the baseline variance of PEMS is measured and modeled (i.e. added) once as part of
the steady-state engine dynamometer laboratory experiment.
4.3.3 Methods and Materials
For thi s e xperiment us e a w ell ve ntilated EMI/RFI s hielded r oom c apable of m aintaining
reasonably constant temperature and pressure. Use a room that can house one of each PEMS,
their power supplies, the PEMS flow meters, cables and lines.
Prior to executing t he b aseline t est, s etup each PEMS and stabilize the PEMS in the room.
Perform P EMS s etup a ccording t o 40 C FR P art 1065 S ubpart J a nd P EMS m anufacturer
instructions, including any warm-up time, and audit. Then supply the PEMS' sample ports with
the sequence of PM from the PM generator as described at the beginning of Section 4.
REPORT 03.14936.12 A-32
-------
At each PM concentration, flow PM1 ong enough so that stable readings of the PEMS can be
recorded. W hen t he O C/EC a nalyzer i s us ed t o s pot-check t he out put of t he P M g enerator,
ensure that enough time has elapsed to achieve an accurate OC/EC analysis.
Position PEMS and configure PM transport tubing to minimize transport delays and PM losses.
Test at least one PEMS from each PEMS manufacturer.
4.3.4 Data Analysis
Reduce t he b aseline d ata f or each P M P EMS, using artificial N TE s ampling e vent time s.
Subtract from each mPM the mean mPM from the initial (short) baseline test of six cycles through
the PM concentrations, which were conducted at the beginning of the test. The results are errors
or "deltas". Calculate the variance of these values, and use them for baseline variance correction
in the data reduction of the remaining environmental tests.
4.4 ELECTROMAGNETIC RADIATION
4.4.1 Objective
Evaluate the effect of Electromagnetic Interference (EMI) and Radio frequency Interference
(RFI) on the performance of the PEMS and determine error factors for the PEMS due to these
effects. First, a screening test on each instrument will be performed with HEPA filtered air to
determine if the EMI/RFI affects the instrument response. If it doe s, the MASC will decide on
the test matrix required for this evaluation.
4.4.2 Methods and Materials
Use an EMI test facility capable of running the SAE tests listed above. This would include:
Signal generators, Power amplifiers, Transmit antennas, Electric Field Sensors, Measurement
Receiver, Data recording device, LISNs (Line Impedance Stabilization Networks) and shielded
enclosure.
4.5 ATMOSPHERIC PRESSURE
4.5.1 Objective
Evaluate the effects of ambient pressure on PEMS PM concentration outputs.
4.5.2 Background
PEMS are expected to operate over ranges of ambient pressures. It is hypothesized that some of
the errors of the PEMS concentration outputs may be a function of ambient pressure. Therefore,
this experiment will change the ambient pressure surrounding PEMS to evaluate its effects on
PEMS m easured c oncentrations a nd f low m eter t ransducer out puts. A s w ith a 11 of t he
environmental te sts, the test cycle for thi s te st is ba sed on t he be st-known di stribution of real
world conditions. For this test, the test cycle pressure distribution was matched to the county-by-
county annual average atmospheric pressure di stribution i n EPA's 2002 N ational E missions
REPORT 03.14936.12 A-33
-------
Inventory (NEI) model. Figure 10 depicts the NEI data distribution (based on 3149 data points)
and the test cycle pressure distribution.
Pressure Histograms
QfiO/
QfiO/
OU/O
= 70%
"o R0%
Q
S? 50/o
-------
TABLE 9. ATMOSPHERIC PRESSURE TEST SEQUENCE
Atmospheric Pressure Test Sequence
Phase
1 Soak
2 Ramp
3 Soak
4 Ramp
5 Soak
6 Ramp
7 Soak
8 Ramp
9 Soak
10 Ramp
11 Soak
12 Ramp
13 Soak
14 Ramp
15 Soak
16 Ramp
17 Soak
18 Ramp
19 Soak
20 Ramp
21 Soak
22 Ramp
23 Soak
Pressure
kPa
101
101-97
97
97-101.87
101.87
101.87-101
101
101-97
97
97-96.6
96.6
96.6-82.74
82.74
82.74-96.8
96.8
96.8-90
90
90-96.8
96.8
96.8-99.2
99.2
99.2-101
101
Alt. ft.
89
89-1203
1203
1203- -148
-148
-148-89
89
89-1203
1203
1203-1316
1316
1316-5501
5501
5501-1259
1259
1259-3244
3244
3244-1259
1259
1259-586
586
586-89
89
Time
min
10
20
20
60
20
20
20
20
25
20
20
20
20
30
20
15
10
20
20
20
20
10
20
Rate
ft/min
0
56
0
-23
0
12
0
56
0
6
0
209
0
-141
0
132
0
-99
0
-34
0
-50
0
Comments
Flat near sea-level
Moderate hill climb from sea level
Flat at moderate elevation
Moderate descent to below sea
level
Flat at extreme low elevation
Moderate hill climb to near sea
level
Flat near sea level
Moderate hill climb from sea level
Flat at moderate elevation
Slow climb from moderate
elevation
Flat at moderate elevation
Rapid climb to NTE limit
Flat at NTE limit
Rapid descent from NTE limit
Flat at moderate elevation
Rapid hill climb to mid elevation
Flat at mid elevation
Rapid descent within middle of
NTE
Flat at moderate elevation
Moderate descent to lower
elevation
Flat at lower elevation
Moderate decent to near sea-level
Flat near sea-level
Pressure-Time Environmental Test Cycle
9000
= -3000 -I-
-5000
-7000 J-
-9000
1/2-hr moving avg dA/dt (ft/hr)
Vertical gridlines = hours
Vertical gridlines = 7-min gas
cylinder cycle times
115 ง;
1
119
123
127
131
8
234567
Time (hr)
FIGURE 11. PRESSURE-TIME ENVIRONMENTAL TEST CYCLE
REPORT 03.14936.12
A-35
-------
Prior toe xecuting t his pressure s equence, s etup e ach P EMS a nd s tabilize t he P EMS i n t he
chamber's first pressure. P erform PEMS setup according to 40 C FR Part 1065 S ubpart J and
PEMS manufacturer instructions, including any warm-up time, zero-span-audits of the analyzers
and the setup of all accessories including flow meters, ECM interpreters, etc. T hen supply the
PMPEMS' sample port with the sequence of PM from the PM generator as described at the
beginning of Section 4.
Flow each generated PM sample long enough so that at least 30 seconds of stable readings are
recorded for the slowest responding gas concentration output of all the PEMS. P osition PEMS
and configure gas transport tubing to minimize transport delays. T arget t o s ample aboutSO
seconds. Repeat this cycle over the 8-hr test cycle, by cycling through the concentration shown
in Table 8, w hich represents one hour of testing, using a 4.5 m inutes of s tabilization and 30
seconds of sampling at each condition.
Perform this test once for one set of PEMS with as many PEMS tested at once.
4.5.4 Data Analysis
Perform data analysis according to Section 4.1.
4.6 AMBIENT TEMPERATURE AND HUMIDITY
4.6.1 Objective
Evaluate the effects of ambient temperature and humidity on P EMS PM concentration outputs.
The histogram i n F igure 12, a long w ith T able 10 a nd F igure 13, w ill be updated by a ne w
temperature profile that takes into consideration the data generated by CE-CERT.
4.6.2 Background
PEMS ar e ex pected t o operate ove r a wide r ange of changing ambient t emperatures. It i s
hypothesized t hat s ome of t he e rrors of t he P EMS out puts may be a function of c hanges i n
ambient te mperature. T herefore, this experiment w ill c hange the a mbient te mperature
surrounding P EMS toe valuate i ts e ffects on P EMS m easured c oncentrations and flow m eter
transducer outputs. As with all of the environmental tests, the test cycle for this test is based on
the be st-known di stribution of r eal w orld c onditions. F or t his t est, t he t est c ycle t emperature
distribution was matched to the hour-by hour county-by-county average atmospheric temperature
distribution, weighted by vehicle miles traveled according to EPA's 2002 National Emissions
Inventory (NEI) m odel. F igure 12 d epicts the NEI data di stribution (based on ov er 900,000
temperatures and over 270 trillion vehicle miles) and the test cycle temperature distribution.
REPORT 03.14936.12 A-36
-------
inno/
QftO/
yu/o
ono/
'oT
E 70%
~ /U/O
"(5
"X RIW
5
cn%
>
o dn%
-------
TABLE 10. AMBIENT TEMPERATURE TEST SEQUENCE
Ambient Temperature Test Sequence
Phase
1 Soak
2 Ramp
3 Soak
4 Ramp
5 Soak
6 Ramp
7 Soak
8 Ramp
9 Soak
10 Ramp
11 Soak
12 Ramp
13 Soak
Temperature
ฐC
13.89
13.89-5.00
-5.00
-5.00-12.78
12.78
12.78-28.33
28.33
28.33-37.78
37.78
37.78-22.22
22.22
22.22-13.89
13.89
ฐF
57
57-23
23
23-55
55
55-83
83
83-100
100
100-72
72
72-57
57
Time
min
10
5
5
145
40
5
52
5
8
100
60
5
40
Rate
ฐC/min
0.00
-3.78
0.00
0.12
0.00
3.11
0.00
1.89
0.00
-0.16
0.00
-1.67
0.00
Comments
Cool in-garage pre-test PEMS operations
Leaving cool garage into cold ambient
Operating at cold temperature outside of
vehicle
Diurnal warming during cool day
Steady cool temperature during testing
Return to hot garage on a cool day
Hot in-garage pre- post- test PEMS
operations
Leaving ho garage into hot ambient
Operating at hot temperature outside of
vehicle
Diurnal cooling during hot day
Steady moderate temperature during
testing
Return to cool garage on a moderate day
Cool in-garage post-test PEMS operations
Temperature-Time Environmental Test Cycle
Temperature
1/2-hr moving average dT/dt (C/hr)
Vertical gridlines = hours
Vertical gridlines = 7-min gas cylinder cycle times
TO
I
-60
0
6
8
12345
Time (hr)
FIGURE 13. TIME SERIES CHART OF AMBIENT TEMPERATURE TEST
REPORT 03.14936.12
A-38
-------
Prior to executing this temperature sequence, setup each PEMS and stabilize the PEMS in the
chamber's first temperature. Perform PEMS setup according to 40 CFR Part 1065 Subpart J and
PEMS manufacturer instructions, including any warm-up time, zero-span-audits of the analyzers
and the setup of all accessories including flow meters, ECM interpreters, etc.
Run t he 8 -hour c ycle t est b y s tepping t hrough t he c oncentration a nd di lution r atio s hown i n
Table 8.
4.6.4 Data Analysis
Perform data analysis according to Section 4.1.
4.7 ORIENTATION AND VIBRATION
4.7.1 Objective
Evaluate the effect of vehicle vibration on the performance of the PEMS and determine error
factors for the PEMS due to these effects. Prior to doing extensive vibration work, perform a
screening using HEPA filtered air sampling at a sweep of different frequencies with low
amplitude. If any of the PEMS shows a response to a particular frequency, propose a frequency
test and submit it for the MASC for approval.
5 SWRI CVS AND CE-CERT TRAILER CORRELATION
Prior to performing the in-use work with the PM-PEMS, it is important to establish the degree of
correlation between SwRI C VS-based PM m easurement and CE-CERT CV S-based PM
measurement. For t his pur pose, t he C E-CERT tr ailer w ill move to SwRI f acilities a nd PM
measurement will be conducted on the engine used for the PM-PEMS program.
Prior to the correlation with SwRI, the CE-CERT MEL will conduct a 1065 a udit of the PM
measurement system and associated weighing chambers and stations and associated electronic
sensors a nd m onitors. T his a udit w ill i nclude ve rification of t he s econdary di lution flow a nd
temperature c ontrollers. T he s ampling s ystem w ill be c hecked t o m ake s ure i t hoi ds t he
appropriate temperatures and within the appropriate limits. The filter holders will be checked for
compliance a nd t he 1 og books w ill be e xamined toe nsure t hey a ppropriately m onitor a 111 he
parameters ne eded for 1 065 c ompliance. C E-CERT w ill i dentify ar eas where t he cu rrent C E-
CERT pr ocedures or equipment doe s not m eet the 106 5 r egulations a nd w ill upg rade t hese
systems or procedures so that they are compliant with the 1065 regulations.
The CE-CERT's MEL will be cross-correlated with an engine eel 1 at SwRI us ing an engine
selected by the SC. Testing will be conducted under the NTE transient cycle with the 0.025 g/hp-
hr bypass setting as determined by the SC.
5.1 METHOD AND MATERIALS
Below is a list of a step by step approach for the correlation between SwRI and CE-CERT
REPORT 03.14936.12 A-39
-------
1. Perform a propane check on SwRI CVS and 47 mm filter and CE-CERT CVS and 47
mm filter. Both systems should pass Part 1065 on propane. However, even if they
pass, note any difference between the two.
2. Set the CVS flow rate to be the same on both systems
3. Set the filter face temperature and velocity to be the same on both systems
4. Set the secondary dilution ratio to be the same on both systems.
5. Use Whatman PTFE membrane filters (7592-104), and filter screens that meet the
latest Part 1065.
6. Modify the exhaust path to SwRI CVS to be comparable with that for the CE-CERT
Trailer
7. Since SwRI is using a test cell that may have had various PM levels, both SwRI and
CE-CERT s hould pr econdition on t he s ame e ngine. T hus, i t i s r ecommended t hat
CE-CERT c lean t heir t unnel pr ior t o t raveling t o S wRI sot hat bot h c an be
conditioned on a similar emissions level.
8. Pre-condition t he S wRI C VS t unnel a nd t he C E-CERT tr ailer C VS tu nnel for a
period of 10 hour sate ngine r ated pow er us ing e xhaust c onfiguration w ith D PF
without bypass. The conditioning time may include active DPF regenerations.
9. Run a total of 12 repeats of the NTE transient cycle using DPF with Bypass Level at
0.025 g/hp-hr emission level, over a period of three days. Four repeats per day with
the CE-CERT followed by four repeats with SwRI CVS and then alternate. Prior to
each set of four repeats manually regenerate the DPF.
10. Use SwRIDMM-230 and CE-CERT DMM-230 to make sure that the e ngine PM
source is not shifting and being consistent.
11. SwRI s hould h andle a nd w eigh a 111 he f liters for bot h S wRI and C E-CERT i n
accordance with their protocol.
12. The CE-CERT trailer is needed at SwRI for at least two weeks per engine. One week
for setup and two weeks of testing assuming the above schedule.
13. In as eparate t ask, EPA w ill equi librate and pre-weigh 20 f liters us ing E PA's
weighing protocol. EPA will then ship them to SwRI for repeat preweighing using
their protocol. SwRI will then ship the s ame filters to EPA for reweighing. After
reweighing at EPA, EPA will ship the filters to CE-CERT for weighing using CE-
CERT's weighing protocol. F inally, CE-CERT w ill s hip the filters to EPA for
reweighing. R esults will be reported b y EPA for M ASC di scussion. N o threshold
for acceptance has been established at the time of this testplan writing.
14. The t arget f or c orrelation a 11 he 0.025 g /hp-hr le vel is C E-CERT's m ean of 12
repeats being within +/-10% of the mean value reported by SwRI.
6 MODEL VALIDATION AND MEASUREMENT ALLOWANCE DETERMINATION
The pr e-validated measurement allowance va lue f or bot h or at 1 east one P EMS w ill b e
determined pr ior t o t he i n-use model va lidation at C E-CERT. If bot h P EMS s ystems ha ve
determined reasonable measurment allowances, then the validation testing will be performed on
the PEMS that shows the lowest measurement allowance.
The MA SC decided to validate only one of the complete PM PEMS systems Horiba or Sensors
where AVL will "piggy back" on either PEMS as part of the model validation. Thus the testing is
a full set of tests where three model PEMS from one manufactures over three routes with one
bypass setting and one vehicle will be tested. If the selected model PEMS does not validate the
REPORT 03.14936.12 A-40
-------
MA S C ha s t he opt ion t o t ry va lidating t he second m anufactures P EMS. T his a dditional
validation i s not c overed i n t he C E-CERT s cope of w ork a nd w ould r equire a budg et
modification.
6.1 MODEL VALIDATION
6.1.1 Objective
The objective of the validation testing is to validate the Monte Carlo model by
1. Testing the PEMS in parallel with the CE-CERT trailer
2. Checking the data to see if it fits the model predicted based on the SwRI laboratory
efforts
For the mode 1 va lidation testing effort, CE-CERT w ill c onduct pr eliminary pi anning for t he
PEMS installation and commissioning. For each PEMS model tested, a total of 5 t est days are
allocated for commissioning. Subsequent PEMS commissioning of like models should take less
time and thus only 3 test days are allocated. The PEMS commissioning will be performed with
the assistance of the PEMS manufacturer on site. CE-CERT will procure Whatman filters for
both CE-CERT & Horiba filter weighing process.
CE-CERT will design, construction, and install a bypass. CE-CERT will purchase the parts for a
bypass. The bypass will be "tuned" to the BSPM level requested by the committee. Initially, it is
planned to tune to 0.025 g/hp-h at clean DPF condition, which could give a range of values from
0.01 to 0.04 g /hp-h depending on i n-use conditions and DPF regeneration status. The use of a
PM PEMS may be incorporated into this part of tuning to provide instantaneous feedback on the
PM level in addition PM filters will be used to determine actual level. This tuning data will be
made available to the MA SC as additional PEMs-MEL deltas, but be denoted as preliminary
tuned data since values could exceed the desires of the MA program.
Long line lengths will be employed to ensure good mixing. CE-CERT will use good engineering
judgment to determine if good mixing i s established. CE-CERT will evaluate good mixing by
measuring the real time PM with the AVL PEMS while attempting to traverse the exhaust stack.
Given the limitation to work around a vehicle during in-use testing, traversing the exhaust may
require s ome t ype of al ternative t est pr ocedure once t he b ypass i s fabricated. CE-CERT w ill
work with the MA SC to determine when a suitable well mixed bypass has been achieved.
The test matrix and test costs depend on the actual number of PEMS tested, number of bypass
configurations, and the number of routes. For this scope of work three model PEMS from one
manufactures over three routes with one bypass setting and one vehicle will be tested. This test
matrix i s ba sed on t he recommendation of t he SC. I f a s econd m anufacture P EMS r equires
testing then a new scope of work will be needed and a budget change.
The pr imary te sting w ill be focused on true N TE e vents if pr actically p ossible a nd/or forced
triggered events. The target level of> 50 ug will be set for the filter measurements by the MEL.
If H oriba P EMS i s c hosen t hen t he H oriba filter w ill be r eplaced one t ime f or e very 8 M EL
filters to simulate an 8 hour operation for the Horiba filter. All filter weighing for both the CE-
REPORT03.14936.12 A-41
-------
CERT a nd Horiba filters ( if s elected) w ill be pe rformed by C E-CERT us ing CE -CERT's
weighing procedures.
For t he t est m atrix c hosen t hree m odels of one m anufacture P EMS w ill be t ested ove r t hree
routes with one additional test day allocated, see Table below. One additional day is allocated for
repeating a test route or for operating the PEMS in a "true" NTE mode or combination of both.
The PEMS will be tested over each route/test-bypass configuration for a total of 4 test days. A
total of 10 prep days are allocated for the preparation and installation of the first PEMS for each
manufacturer, 5 test days are allocated for commissioning each PEMS, and 3 days are allocated
for changing between PEMS of a single manufacturer. Subsequent PEMS commissioning of like
models should also take less time and thus only 3 test days are allocated.
Table - Three Models of One PEMS Manufacture Test Matrix
Mfg
PEMS
plus AVL
PEMS
plus AVL
PEMS
plus AVL
Unit#
1
2
3
Route
Palm Springs
San Diego
Baker
Palm Springs
San Diego
Baker
Palm Springs
San Diego
Baker
Test
Conditions
1 bypass
0.025 g/hp-h
1 bypass
0.025 g/hp-h
1 bypass
0.025 g/hp-h
Total test
days
4
4
4
Truck rental for extended period of time for setup and PEMS installation is included under this
task. This could include a Volvo because of parts availability or a different model for ease of
bypass installation.
Data analysis with engines outside of the NTE requires additional data processing for Method 2.
During Method 2 calculation there i s a s ummation of the inverse of fuel rate. The fuel rate on
some conditions outside the NTE can go to zero causing the calculation to go to infinity. In these
situations it was decided by the MASC to freeze the bsFC to a constant value during out-of-NTE
operation using the last valid BSFC NTE value. CE-CERT will perform this bsFC freezing in the
Method 2 summation during data post processing for both the PEMS and MEL. The logic to start
and stop freezing will be determined by the MASC and provided to CE-CERT before processing
Method 2 results.
Methodl =
I
Carbon^
_ ECMfuel
xWork
REPORT 03.14936.12
A-42
-------
6.1.1.1 CE-CERT Validation
The difference between the PEMS results and the CE-CERT trailer results will be compared to
the error predicted by the Monte Carlo model. To validate the Monte Carlo model, data must be
run t hrough t he m odel a nd t he m odel r esults must pr edict t he a ctual t est r esults w ithin a
reasonable level of accuracy.
Validation will be based on the following procedure. For each reference NTE event, the Monte
Carlo model will be used to generate the 5* and 95* percentiles of the simulated distribution of
the brake-specific PM emission differences. In order to obtain simulations representing similar
conditions t o those obt ained on -road, some e rror s urfaces may ne ed t o be s uppressed i n t he
simulations since not all of them may be applicable to the on-road conditions. T he choice of
which error surfaces to suppress would need to be made by the Steering Committee.
Next, the 5th and 95th delta percentiles obtained from the above simulations will be separately fit
to a line or curve using two chosen methods: a linear regression procedure and a local regression
(loess) technique1. Depending on which of the resulting two fits is best for each set of data (i.e.,
either for the 5th percentile deltas or the 95th percentile deltas), the resulting line or curve will be
used as one of the lower or upper limits for the on-road data.
To determine the best fit for a given set of delta percentiles (i.e., 5* or 95*), a simple regression
line initially will be fit to the data. If a least squares linear regression of the 5th or 95th percentile
deltas versus the ideal PM emission has an r2>0.85 and an SEE < 5 % of the median ideal PM
emissions, then the regression line will be used. If this set of criteria is not met, then a loess fit
will be used. Since a loess regression requires the selection of a smoothing parameter to smooth
the data, the chosen smoothness parameter should balance the residual sum of squares against the
smoothness of the fit.
The on -road de Ita e rrors, obt ained from t he r esults of c ollecting da ta on s everal N TE e vents
during on-road operations, will be plotted on a graph containing the 5th and 95th percentile delta
limits determined from the regression fits chosen above. The graph will consist of a plot of delta
PM versus ideal PM. The number of on-road points outside these limits will be determined and
expressed as a percentage of the total number on on-road data points. If this number does not
exceeds 10% of the total number of on-road data, the simulation data will be considered to be
valid.
6.2 MEASUREMENT ALLOWANCE DETERMINATION
6.2.1 Objective
Use the Monte C arlo simulation program developed with data from s ections 2, 3 a nd 5, and
validated with section 5.1 to determine the measurement allowance for all regulated emissions, at
2007 emissions standards.
REPORT 03.14936.12 A-43
-------
6.2.2 Background
After the Monte Carlo model has been validated and confidence in its ability to predict errors
from PEMS instrumentation, the last step in this program will be to actually calculate a single set
of measurement allowance for PM.
6.2.3 Methods and Materials
Using the criteria explained in section 2.2 calculate the various levels of measurement accuracy
corresponding t o t he t hree P EMS m anufacturers a nd t he b rake s pecific P M e missions
calculations. U se all the various error surfaces developed during this test program, including
those provided by engine manufacturers to the EPA and ARB.
6.2.4 Data Analysis
Use the methodology explained in section 2.2, and Table 2.2 to arrive at the final measurement
allowance.
7 TIME AND COST
7.1 TIMELINE
Table 11 is a te ntative time line pr ejecting th e ma jor ta sks to be a ccomplished during thi s
program. The additional work if needed option is the work that may need to be done if the model
did not validate. Otherwise, the final report will be submitted by September 30, 2009.
TABLE 11. PROJECTED PM-PEMS TIMELINE
Tentative Timeline
il 1.13 y June J LI I. -LI; . st iei: em : T '. :tcber Noveii'ibe Decernbe Jdnuar.-i-eiiTur-i' ',1.3 :h -pnl 1.1.? June Jul Uigust 'I'ept&mbe October
7.2 COST
The rough estimated cost is shown in Table 12. Based on the current estimate, a $125,000 of the
$200,000 is needed to complete the project.
REPORT 03.14936.12
A-44
-------
TABLE 12. PROJECTED COST ESTIMATE
PEMS Training, Setup, Audit, and Debug
Steady-State and Transient Experiments
SwRI and CE-CERT Correlation (1 engine)
PM Generator and Environmental Testing Activities
Modeling Activities (Including CO2)
Data and analysis, reporting, and final report
Contingency if validation fails
General Contingency
Grand Total
Grand Total Without General Contingency
Grand Total without General Contingency and
Contingency if Validation Fails
$660,000
$190,000.00
$75,000.00
$200,000.00
$225,000.00
$150,000.00
$100,000.00
$200,000.00
$1,800,000.00
$1,600,000.00
$1,500,000.00
8 ABBREVIATIONS USED IN BRAKE SPECIFIC EQUATIONS
Method 1:
ePM = brake-specific emission, PM (g/hp-hr)
N = total number (of time intervals) in series
x = amount of substance fraction (mol PM/mol exhaust; note that Ijjmol (emission
constituent)/mol (exhaust) = Ippm (part per million)
n = amount of substance rate (mol/sec, in this case, mol (exhaust)/sec
Dt = time interval (sec)
fn = rotational frequency (shaft), rev/min
T = torque (N-m)
NOTE: The units of the numerator work out to gemission as is. However, using the units given
for the denominator (RPM * N-m * s), you would still need to divide by 1.978 to getto hp-hr
(using RPM * N-m = kW * 9550, 1 hour = 3600 sec, and kW = hp*0.7457)
Method 2:
ePM = brake-specific emission, PM (g/hp-hr)
MNO2 = Molecular weight, NO2 (-46 g/mol)
N = total number (of time intervals) in series
x = amount of substance fraction (mol PM/mol exhaust; note that Ijjmol (emission
constituent)/mol (exhaust) = Ippm (part per million)
n = amount of substance rate (mol/sec, in this case, mol (exhaust)/sec) that is linearly
proportional to n (Note: this is a proportional sample, which means that you may use a flow
meter that has a span error, as long as its calibration is linear)
Dt = time interval (sec)
REPORT 03.14936.12 A-45
-------
MC = Atomic weight of carbon (-12 g/mol)
wfuel = g (carbon)/g (fuel); Note fuel is roughly 86% carbon by mass
xCproddry = amount of carbon products on a Cl basis per dry mol of measured flow (exhaust),
mol/mol, solved iteratively per 1065.655
xH2O = amount of water in measured flow, mol/mol (see 1065.645 for calculations)
efuel = brake-specific fuel consumption (g (fuel)/hp-hr)
Methods:
ePM = brake-specific emission, PM (g/hp-hr)
MNO2 = Molecular weight, NO2 (-46 g/mol)
wfuel = g (carbon)/g (fuel); Note fuel is roughly 86% carbon by mass
MC = Atomic weight of carbon (-12 g/mol)
N = total number (of time intervals) in series
x = amount of substance fraction (mol PM/mol exhaust; note that Ijjmol (emission
constituent)/mol (exhaust) = Ippm (part per million)
nifuei = mass rate of fuel (g/sec)
xH2O = amount of water in measured flow, mol/mol (see 1065.645 for calculations)
xCproddry = amount of carbon products on a Cl basis per dry mol of measured flow (exhaust),
mol/mol
Dt = time interval (sec)
fn = rotational frequency (shaft), rev/min
T = torque (N-m)
Dt = time interval (sec)
NOTE: The units of the numerator work out to gemission as is. However, using the units given
for the denominator (RPM * N-m * s), you would still need to divide by 1.978 to getto hp-hr
(using RPM * N-m = kW * 9550, 1 hour = 3600 sec, and kW = hp*0.7457)
REPORT 03.14936.12 A-46
-------
APPENDIX B
STEERING COMMITTEE MEETING MINUTES
REPORT 03.14936.12
-------
PM Measurement Allowance Steering Committee Meeting
Meeting at CE-CERT
April 23, 24, and 25, 2008
Future Meetings:
1. May 15-16 meeting, SwRI
2. June 12 and 13 at for PM-PEMS and 10 and 11 for EMTC, ACS.
3. July 29 and 30, San Antonio, SwRI
4. August 28, 29, San Antonio, SwRI
On-Road PM PEMS Discussion and Action Items
COi Activities
1. Use CC>2 data provided by CE-CERT during the gaseous program for CO2 validation,
and share the information during the May 15-16 meeting at SwRI.
Test Plan Activities
a. Explain in test plan why Horiba can't use Method 3
b. Replace BSFC i n t he e quation of M ethod 2 with ( fuel flow/torque * speed).
Essentially, remove BSFC error surface
c. Show how (mp bar) for the AVL is calculated
d. For AVL Method 3, use a different error surface for PM.
e. For reference NTEs, use the existing reference NTEs, and PM concentrations to
produce different concentration from DPF out to threshold levels.
f Check section 2.3 with Bob Mason to make sure it is correct
g. Change t he example t o PM in the t est pi an and give an appropriate P M
concentration range per mole basis.
h. Reexamine the text to explain the new equations better. No second by second for
AVL
i. Change Figure 8 i n the test pi an to reflect the fuel flow i nclusion, and remove
BSFC error.
j. Change Figure 8 to update all errors that are required
k. State t he t ime and d ate b y when t he m odel could be available t o t he group,
assuming no last minute changes are required.
General Notes:
1. For AVL, use a 3 t o 1 dilution ratio and change the selectable range if needed but not
the dilution ratio.
2. For the NTE windowing, use the EEPS to determine the windowing sensitivity, assuming
a maximum of 6 seconds delay at the beginning and end.
3. Measure the CVS dilution air temperature as close as close as possible to the exhaust and
dilution air mixing point. This may require insulation of SwRI CVS system.
4. Use Whatman Teflon membrane, 2 micrometer for the entire program
REPORT 03.14936.12 B-l
-------
5. Use the fine ambient backup screen instead of the coarse or diesel backup screen
6. List all the error surfaces in the test plan
7. Plan Training and Comissioning of the PM Generator
8. If 32 seconds NTE is not sufficient for the PM-PEMS, extend the length
9. Use 10 % difference as an acceptable difference between SwRI and CE-CERT
1065 Audit (For reference see SwRI presentation)
TSI Flow Meters
Using reference flow meters like the TSI flow meter as a transfer standards is
okay, if independently checked via a master flow meter
Sensors PPMD
a. Verify that the external and bypass flow TSI flow meter certificates are valid for
duration of testing
b. Verify tha t the da ta publ ished on their indi vidual c ertificates me et the 1065
linearity thresholds
Horiba TRPM
Use total flow and dilution air flow for Horiba and make sure they meet 1065
linearity verifications.
AVL MSS
Meeting +/- 3 % per point on dilution ratio is acceptable.
Vibration
Perform a frequency s weep w ith m oderate a mplitude a nd 1 og t he r eal t ime s ignals from t he
PEMS using a fixed PM generator level. If changes occur, design a frequency cycle around that
frequency range to test for delta changes in PM with and without vibration. Obtain approval on
the frequency cycle s elected from t he M ASC be fore pr oceeding. If no changes are observed
during a frequency sweep, there is not a need to test for vibration.
Use t wo o dentations ve rtical a nd ho rizontal for t he P PMD. If t here i s enough s pace on t he
vibration table at SwRI, test all PEMS units at once. If not, use them one by one.
EMI
Use E MI on and of f, and s creen t he r eal t ime r esponse of t he i nstrument a t a s pecific
concentration 1 evel. If a r esponse i s obs erved d uring E MI on/ off s witching, t he M ASC w ill
decide on what the next steps are.
REPORT 03.14936.12 B-2
-------
Environmental Chamber:
1. Temperature and Humidity cycle will be made available by Matt Spears
2. Use a high velocity fan to blow over the 4 inch inlet section of the PPMD
3. PM Generator Setup and Verification
4. How to do the environmental pressure test?
a) Use three different levels concentration at the 0.02 and 0.03 g/hp-hr
b) Run a long baseline of 8 hour s using three different toggeled PM concentration levels.
For baseline, toggle the process five times, and use that as a baseline, followed by the
continuation of the baseline.
c) Run the temperature and humidity test using the same toggeling profile. For temperature,
humidity, and p ressure, start w ith a b aseline with five r epeats, and t hen ki ck of ft he
environmental cycle.
d) A toggeling of zero, mid, and high is on the order of 15 minutes.
There i s a pos sibility of e liminating t he z ero a nd a dd one c oncentration 1 evel, a nd r andomly
sample from all the deltas generated. We need to talk to Bob Mason about this.
Use just one of each of the PEMS for environmental activities.
Finally, changes w ere m ade t o the t est p Ian during di scussions. A Iso, S wRI pr esented t hat
attached document on the test plan. More test plan discussions will take place in the next meeting
on May 15 and 16, at SwRI.
REPORT 03.14936.12 B-3
-------
PM Measurement Allowance Steering Committee Meeting
Meeting at SwRI
May 15-16, 2008
Dayl
1. Tim French indicates that there are several i ssues came up with the PM-PEMS onboard
testing performed at CE-CERT before the CE-CERT meeting was presented.
2. Kent's presentation, posted on FTP website:
Some highlights, but see presentation for more details:
a. Problems with valve timing opening and closing consistency and long delays over
3 seconds.
b. There was one crystal that was very noi sy but the software did not reject it, and
reported data using it.
c. A drop in frequency but an increase in mass deposit. (This is a non-issue)
d. Semtech DS 10[1].09 SP2 b5. This beta J1939 includes filtering and was able to
include NTEs.
e. Post test J1939 filtering ve rsus r eal time J 1939 filtering. Post te st c aptured
different start up NTE than the real time filtering one. Essentially, post processing
software needs to be scrutinized.
f NTE yield produced by PPMD was low. Some of the valid NTE that was captured
in PPMD post processing was rejected, after carefully reviewing the data.
g. Others. See presentation.
3. SwRI update:
a. Horiba linearity check was resolved by using the Gilibrator directlty
b. Engine A, B, C, and D 80 steady-state testing was complete
c. Engine A was tested for 40 SS points in sub-NTE runs
d. International engine was also tested
e. All above will be posted on the website after careful review
4. Janet and Bob Presentation, please see website.
Bill Martin questioned the idea of excluding the 95th to 100th data twice. One in the error
surfaces, and the other in the measurement allowance. He was concerned whether such
practice will lead to truncation of the distribution. Janet will show in the next meeting
that such practice will not lead to any truncation of data or it will effect the shape of the
error distribution.
5. General Discussion and Action Items:
a. Find ways to load up the can bus during the actual testing to simulate real world
operation by making the can bus communication busy.
REPORT 03.14936.12 B-4
-------
b. Keep using the gaseous error surfaces for Method 2 a nd Method 3 on t he PM-
PEMS.
c. Environmental Chamber:
1. Use a 9 m inute c ycle for the environmental chamber a nd go into three
concentration levels, 3 m inutes per level. Sample for a period of 35 s econds
from each level.
2. Use five levels of dilution ratio ranging from 6 to 50.
3. Check with Sensors to see if they accept an exhaust flow meter analog input.
For M odel out put, ke ep t he 95 * as t he m easurement a llowance. H owever, i f i t di d not
validate c onsider t he p otential of us ing ot her t han t he 95 *, if t he MASC r eaches an
agreement on that.
Day 2:
1. Move forward with the International Engine
2. Communicate via J1939 to load the engine can bus during testing. Use only the J1939
communication with the heavy-heavy duty diesel engine
3. Accept the idea that Horiba will use test cell engine speed and load analog output signals
4. The right to remove outliers using good engineering judgement.
5. Starting on June 2nd with commissioning
6. Spend one week of commissioning before we start.
Plot:
x-axis percent of max torque or fuel rate
y-axis absolute difference over max torque or fuel rate
Mart's Discussion:
Model Validation Testing at CE-CERT
a. Number of engines/vehicles
b. Number of bypass conditions, at least two, maybe 3
c. Number of PEMS: at least two of each, highly desirable three
d. Number of NTE events: total 100-200 per PEMS
e. Number of route repeats: 1 to 3
f. Types of NTE events
i. CE-CERT limits
1. Minimum filter loading: 50 micrograms
REPORT 03.14936.12 B-5
-------
Gaseous plus PM. But PM set priorities. Sensors goes first.
Horiba plus AVL
Sensors plus AVL
Horiba plus AVL
Sensors plus AVL
Horiba plus AVL
Sensors plus AVL
NTE Events
30-50
30-50
30-50
30-50
30-50
30-50
Route
Palm Springs
Palm Springs
San Diego
San Diego
Baker
Baker
Bypass
1
1,
review
1 or
change
1
1
1
Repeat
One-Run
One-Run
One-Run
One-Run
One-Run
One-Run
REPORT 03.14936.12
-------
PM Measurement Allowance Steering Committee Meeting
Meeting in Madison, Wisconsin
June 12-13, 2008
The meeting started by reviewing and approving the last meeting minutes.
PPMD Commissioning
a. Following the meeting minutes, Imad Khalek gave a presentation on the status of
the PPMD commissioning at SwRI. The presentation was sent to the MASC, but
was not posted on the website.
b. David Booker also gave a presentation on the SwRI commissioning activities. The
presentation w as sent to the MASC, but was not posted ont he website at the
request of Sensors.
c. As a r esult of t hese t wo pr esentations, t he M ASC de cided t o g ive S ensors a
chance t o fix s ome of t he pr oblems e ncountered a nd c ome ba ck t o S wRI for
additional commissioning during the week of June 16. Also, the MASC requested
that a conference call should take place on Friday, June 20, and SwRI should give
a status update on whether or not the Semtech-DS/PPMD issues were fixed to a
satisfactory level so the program can proceed.
d. The MASC also made the following points on the PPMD:
i. It is up to Sensors to decide on the quartz crystal equilibration time, after a
crystal goes into a invalid NTE window.
ii. It i s r equested that w hen all c rystals ar e 1 ocked out, a nd one o ft hem
becomes available during an NTE, the crystal should wait until the current
NTE terminates, and a new NTE starts before it samples from an NTE.
PM-PEMS Engine Selection
Imad Khalek pointed out to the MASC the fact that the PM-PEMS program is moving
forward with some deficiencies related to the Horiba system inability to communicate with the
Navistar engine ECM ISO protocol. It was recommended to the MASC that a heavy-heavy duty
diesel engine be installed in the test cell first so the Horiba system can communicate with the
engine ECM using the J1939 protocol. In addition, this will give a chance for Horiba to upgrade
their system in preparation for the Navistar engine after a heavy heavy-duty diesel engine.
EMA ag reed to take a 1 ook at t he pos sibility of pr oviding a he avy h eavy-duty di esel
engine in a timely manner. EMA agreed to make a final decision on this issue by the June 20
conference call.
REPORT 03.14936.12 B-7
-------
Horiba Concern
Horiba was concerned about the fact that Sensors brought a new model of the PPMD to be used
on the PM-PEMS-MA program. They wanted to have a chance to update their system as well.
The MASC decided to give Horiba a chance to work and upgrade their systems. It was decided
that SwRI should ship back to Horiba one of their system present at SwRI, and give them until
July 14 to ship back the system.
Horiba was also given until September 15 to upgrade their system so it can communicate with
the Navistar ECM protocol using ISO-15765.
Bill M artin made c omments about the di fficulty he s ees in the H oriba system me eting 1065
requirements. Matt indicated that EPA would enter an allowance for Horiba specifically, at the
time of Direct Final Rule (DFR), through the alternate procedure approval.
AVL Presentation
Bill Silvis presented results on the MSS, where a compensation algorithm is added to account for
organic carbon and sulfate. No copy of the presentation was given to SwRI for distribution. Matt
Spears was not convinced that such compensation will be acceptable for EPA approval due to the
significant correction required and due to the MSS principle itself. SwRI also had some technical
reservation about the process due to its technical complexity. E ssentially, the problem i s not
trivial and more thorough work and understanding is still needed.
AVL was encouraged to continue working on this issue and refine it. It is understood that they
will submit the compensating algorithms prior to the start of steady-state testing. They may also
submit their compensating algorithms at any time for the MASC to have analyzed by SwRI or
CE-CERT.
Rey A gama s uggested t hat t ime w ould be t he be st w indow of oppor tunity f or A VL t o be
included via DFR through alternate procedure approval.
SwRI Test Cell
The MASC requested the following:
SwRI should install all PM-PEMS as close as practically possible to the entry
oft he C VS. This w ill r educe an y p article 1 osses be tween t he poi nt of
measurement amongt heP M-PEMS and also relative t o the C VS.
Furthermore, this will minimize the backpressure experienced by the PPMD,
by shortening the length of exhaust piping present downstream of the PPMD,
prior to entry to the CVS. The target backpressure is (-1 to +4 kpa)
SwRI should try to maintain temperature of 35ฐC ฑ 5ฐC in the vicinity of the
PM-PEMS inside the test cell.
REPORT 03.14936.12
-------
CE-CERT Presentation
Kent Johnson gave a presentation on the temperature distribution on various location of the MEL
such a s be hind t he C AB, unde r M EL t rail, a nd unde r pa ssenger door. K ent' s pr esentation i s
posted on t he website. One of i ntriguing hi ghlights of the presentation is that the temperature
profile can reach as high as 60 ฐC to 90 ฐC during in-use in the vicinity of the PM-PEMS. This
triggers t he i dea of c hanging t he e nvironmental t emperature pr ofile t hat w as us ed dur ing t he
gaseous PEMS program that was based on ambient temperature. Matt spears will be modifying
the existing environmental chamber temperature profile, taking into account Kent's finding.
Budget
CE-CERT presented their budget with some options. The overall impression was that the total
budget for both SwRI and CE-CERT i s beyond the funding 1 evel available for this program.
Below are some of the options entertained that will be discussed during the next meeting at SwRI
starting on July 21st.
1. Reduce the scope of work by cutting the number of routes and the number of PEMS used
by CE-CERT
2. Reduce the scope of work at SwRI by reducing the number of engines to be tested from
two engines to one engine
3. Increase the overall budget by eliminating this year EMA pilot program requirement, and
add funding to the MA program instead, assuming that the funding will be cost shared
among EMA, EPA, and CARB
Additional Comments and Action Items:
1. If PPMD and OBS200, both, resulted in negative allowance, e.g. -0.01 to -0.02 g/hp-hr,
the MA will be zero, essentially one MA for the entire program and for all the PM-PEMS
used. Under such circumstances, both instruments will be allowed to be used with a zero
MA.
2. Keep aware of alignment issues. One may want to investigate the brake-specific emission
value reported by PM-PEMS by shifting the numerator and denominators relative to one
another and relative to absolute time. This could be done with initial transient test results
with bypass.
3. Sensors w ill pr ovide a VI to simulate the ex haust flow rate s o we c an exercise the
multiple dilution ratios in the environmental chamber.
4. SwRI will ne ed t o r esubmit s ome budge t opt ions of doi ng t he w ork w ith one e ngine
versus two engines.
REPORT 03.14936.12 B-9
-------
PM Measurement Allowance Steering Committee Meeting
Meeting at SwRI, San Antonio
July 21-23, 2008
July 21
The meeting started at 1:00 PM. It started by reviewing and approving the previous minutes, and
also by discussing future meeting schedules, which were already sent by EMA as future meeting
notices. The next meeting is scheduled for August 27 (starts at 2:00PM), 28, and 29 (ends at
2:00 PM).
Imad Khalek started a presentation, posted on FTP website, on project status update. He showed
a comparison between laboratory-based exhaust flow and the 4-inch Sensors exhaust flow that
was used on the Navistar engine. The slope of the correlation was too high at 1.27. As a result of
the discussions, the MASC decided on the following with the current 5-inch exhaust flow meters
used w ith t he M ack engine, which t he f irst engine t o be us ed as a pa it of t he of ficial
measurement allowance work.
Check 1 aboratory-based exhaust flow rate with Horiba and Sensors 5-inch exhaust flow
meters. If a problem is obvious or the slope of the correlation differs by more than +/-
0.05 from a slope of 1, then send the flow meters back to the manufacturers to check on
the calibration.
After finishing a part of the presentation, Janet B uckingham and Bob Mason showed up f or a
scheduled presentation at 3:00 PM. The presentation is posted on the FTP website and addressed
the double truncation issue raised by Bill Martin at the 5* and 95* percentile. The presentation is
posted on the FTP website. Based on the presented work, the following was agreed upon:
a. The 95* percentile is still desired for the measurement allowance
b. The 5* and 95* are still acceptable to bound the validation range.
c. There was still a remaining unresolved issue about where you assign the -1, 0,1 on
the error surfaces for the delta change between lab and PEMS. It was decided that
this issue should be addressed during the last day of the meeting, but that was
never brought up again. There was a proposal by Bill Martin to fit both sides of
the error distribution independently using a normal distribution fit. The median of
such distribution will be the median based on previous practice along with 95th for
one s ide and 5 * for t he ot her s ide. W hen e ach side of t he di stribution i s fitted
using the above boundary conditions, one can then expand the data picked from
the error surfaces to cover 1 % to 99 % of the data or even 0.1 % to 99.9 %. No
decision has been made on this issue yet. This topic will require more discussion
during the next meeting.
July 22
A significant part of the day was allocated for budget discussion. The different budget scenarios
are posted in the FTP website. Below are some of the highlights:
REPORT 03.14936.12 B-10
-------
For budget cutting purposes, one engine will be used for the measurement allowance at SwRI. In
addition, work will be performed with one bypass setting that gives brake specific PM emissions
levels between the two thresholds of 0.02 and 0.03 g/hp-hr. For CE-CERT, choose the PEMS
with the 1 owestpos itivem easurement al lowance. If one P EMS cl early s hows a 1 ower
measurement a llowance t han t he r est, a nd bot h a re pos itive, pi ck t he one w ith t he 1 ow
measurement allowance for the rest of the validation testing.
If one allowance is positive by one PEMS, and if the other one is negative, then, in principle,
choose the positive allowance if it slightly positive, (no clear cut agreement yet).
After the budget di scussion, Tim French from EMA walked the group (via phone) through the
EMA proposal posted on F TP website, to substitute year 1 pi lot program and to also provide
some supplemental funding on the order of $200 K to inject more funding into the measurement
allowance program. The overall EMA budget funding was projected to be on the order of $500 K
to $700 K . Most or all the proposed activities will be performed in-use by hiring a third party
that i s not part of EMA and does not belong to one of the engine manufacturer to conduct the
testing. As a result of the EMA proposal the following will take place:
Matt Spears will speak with EPA upper-management to consider the EMA proposal in
the context of the one year pilot program.
EPA will also make a final determination on whether to use the shortened version of the
measurement allowance program. E.g. one versus two engines, by-pass, no-bypass, etc...
After the budget discussions, Imad Khalek continued his presentation that was started on the first
day. T he w ork s howed the di fferent t orque a nd fuel errors s urfaces de termined be tween t he
laboratory and the engine ECM public broadcast. The data covered a total of five engines (four
heavy-heavy and one 1 ight-heavy) that i ncluded a DDC S eries 60, C AT C13, Cummins ISM,
Mack MP7, and Navistar 6.4 liter engine.
Also the work covered sub-NTE fuel flow errors. Based the sub-NTE results, it was decided that
for forced NTEs, if the engine operation falls belowNTE, use the last BSFC value observed
within the NTE. Use that only for Method 2 calculations.
After this presentation, Imad Khalek refreshed the memory of the group by giving a status update
presentation, posted on t he FTP website, based on the last commissioning work done at SwRI.
The pr esentation w as p osted on t he F TP w ebsite. A fter t hat, K ent J ohnson from C E-CERT
shared s ome obs ervations a bout t he S ensors da ta pr oduced b y S wRI d uring c ommissioning.
There was no conflict between SwRI and C E-CERT r eporting on t he results. K ent's W ord
document that was shared with the group will be posted at the FTP website when it becomes
available to SwRI.
July 23, 2008
The focus of this day was on the Test Plan, particularly the environmental testing. However, at
the beginning of this day, the idea brought by SwRI earlier of adding short NTE windows for the
laboratory transient testing, along with s ome 1 ow or m edium i die operation prior to the NTE
transient cycle was discussed.
REPORT 03.14936.12 B-ll
-------
It was decided that Short NTE windows should be added to the thirty 3 2-seconds NTE
cycle, in the inter time. A total of 10 short NTEs will be added as follows:
1. 4 five seconds NTEs, with two back to back within an inter-time
2. 3 10 seconds NTEs
3. 318 seconds NTEs
Position two five seconds NTEs back to back in the inter-NTEs. For Horiba, they
need about 21 o 3 seconds to exit an NTE. Thus, place the short NTEs at least five
seconds after the end of a valid NTE to avoid continuous sampling with Horiba.
On the conditioning prior to the start of a transient NTE, start with five to 10 minutes at
medium idle time before the NTE transient cycle to observe if there is an effect on the
laboratory P M e missions r esults. If t here i s a nd e ffect, t hen pr opose an i die t ime o r
something similar prior to starting the NTE transient cycle. (Note that before each NTE
transient cycle, the plan is to force-regenerate the DPF first, medium speed idle will then
be added after the forced regeneration).
EMI/RFI/Shock &Vibration
1. For EMI/RFI, expose the PM PEMS to EMI/RFI using HEPA filtered air, similar to the
gaseous program. Then decide with the M ASC after reviewing the results, what i s the
next step. As an option, we could use the PM generator to perform this work. One PEMS
from each manufacturer will be used for these activities.
2. Vibration and Orientation (non-road only)
a. On t he or ientation, a sk PEMS m anufacturers on t he w orst or ientation s cenario
postion. Use such orientation at 45ฐ with the appropriate orientation, and survey
the worst case scenario for all gaseous and for PPMD, and TRPM.
b. Perform a frequency s weep, with a v ery m oderate am plitude, then share with
MASC to decide on how to go forward. Eric should propose the sweep frequency,
amplitude and duration. Do two repeats. One of each instrument will be used.
This is only for nonroad. Outside the scope of the program.
3. As for shock, ask Eric about a recommended on-highway profile for shock.
Temperature/Humidity chamber and Pressure Chamber
For bot h t emperature a nd hum idity a nd pr essure c hamber w ork, us e t hree P M c oncentration
levels around the two threshold levels such as 4000, 5000, a nd 6000 ji g/m3. Use a total of four
dilution r atios of 6, 12, 20 a nd 30. A t e ach di lution r atio, s tabilize for 4.5m inutes ate ach
concentration level and measure for 30 seconds before you go to the next concentration level to
do the same. E.g.
1. Set the dilution ratio to 6, stabilize at 4000 |ig/m3 for a 4.5 minutes
2. Measure PM using PEMS for 30 seconds
3. Move to next concentration level of 5,000 |ig/m3 and stabilize for 4.5 minutes
4. Measure PM using PEMS for 30 seconds
5. Move to the next concentration level of 6,000 |ig/m3
REPORT 03.14936.12 B-12
-------
6. Measure PM using PEMS for 30 seconds
7. Repeat 1 through 6 at different dilution ratio
A spreadsheet on PM concentration and loading was originally made by Matt Spears during the
meeting and was very slightly modified by Imad is posted to the FTP website
As a result of the proposed s cenario above, S ensors PPMD will require a s econdary di lution
using MPS2 to perform an 8 hour s of activities using the above scenarios. In case of Horiba,
either we need to allow higher dilution ratio than those adheres to 1065 or we need to allow more
loading on the filter past a 0.4 mg. More discussions on this will need to take place during the
next meeting.
The PM cycle to run by the PM generator contains five 15 minutes cycle to be used as a baseline.
E.g. R un t he first five 15m inutes a t nor mal t emperature, us e t hose a s a ba seline 1 ine, t hen
proceed for the rest of the day to capture an eight hours of similar repeats. Do the same thing
prior to starting the Temperature/Humidity profile and well as the Pressure Chamber profile. One
of the remaining i ssues that has not b een resolved around this topic i s how do w e capture the
baseline i nformation w ith t he H oriba s ystem without the ne ed to change the f ilter. More
discussion on this subject is needed during the next meeting.
New Temperature Profile
Take the mean of CE-CERT Cab minus ambient and add it to original temperature profile, and
solve for new temperature and humidity profile that maintains derived from the real world data.
Perform a F ourier transform on t he CE-CERT Cab data, eliminate frequency content that are
similar to the base ambient profile, and use a magic synthesizer to superimpose the frequency on
top of the new temperature profile. Matt Spears is assigned to do this task.
Pressure Chamber
We resolve the logistical issue of the pressure chamber work by placing the PM generator inside
the chamber. Matt will need to send me some dimension on the PM generator to see if it fits in
the Chamber. Other gases as well as 30 amp circuits should all be accommodated.
For Horiba, heat trace the segment of the transition to 250 ฐC, similar to the PM generator outlet
temperature for both the Horiba and AVL.
Use each instrument separately for these tests.
REPORT 03.14936.12 B-13
-------
PM Measurement Allowance Steering Committee Meeting
Meeting at SwRI, San Antonio
August 27,28, & 29, 2008
Meetings
October: 9,10-atEPA
Nov: 12,13, and 14-at SwRI
Dec.: 10,11,12-atSwRI
Discussion Points and Decisions Made
For t he e rror di stribution c hoose t he 1 st and 99 th at -1 a nd 1, ba sed on a nor mal
distribution fitting to extrapolate down to the 1st and 99 . For the height of the median, if
there is a discontinuity, choose the average or randomly pick one or the other.
Recheck the 4 inch flowmeter calibration that was on the International engine
For Horiba, filter 1 oading is allowed to go beyond 400 microgram up to 700 microgram
or beyond as long as the flow is controlled.
For Horiba, do the five cycle baseline at the beginning for humidity and temperature and
do it at the end for the pressure. This will require changing the filter. For the pressure, if
it can be done within an hour than do it at the beginning, otherwise, do it at the end.
Which of the three units are acceptable to be used in the pilot program:
PPMD is approved
Horiba might be approved
AVL will have to make the case at EPA to see if approved
EMA would 1 ike to know what i nstrument would be acceptable for pi lot
one program
Can EMA combine pilot efforts within companies
EMA will fund the additional funding required that will be required to do
the intermediate testing that involves:
One bypass setting, one engine at SwRI
Three PEMS, one manufacturer with CE-CERT
We s hould m ove forward on RMI, RFI, and vi bration s weep, as s oon a s w e s tart t he
official testing
Due to spikes, we may revisit the post processing of the Horiba results, especially during
the N TE tr ansient te st. As of r ight now, Horiba first c orrelates the e ntire E AD s ignal
(including spike) with the filter weight (not including spikes), then apply the relationship
for the NTE window portion of the cycle.
SwRI Presentation and Action Items
During t he m eeting, SwRI gave a s tatus upda te vi a pr esentation, see enc losed
presentation (Update 6), on the PPMD, MSS, and TrPM. SwRI, also discussed the bypass tuning
for using 10 NTE transient cycle, 80 poi nts steady-state using MSS, and projected filter m ass
concentration, us ing a r elationship be tween t he filter a nd t he M SS. In a ddition, t he b ypass
mixing w as s hown qualitatively. Furthermore, the transient concentration trace with the MSS
was presented for the 10 NTE transient cycle.
REPORT 03.14936.12 B-14
-------
As a result of the work reported several action items were born:
Make sure that the filter-based steady-state projected concentration is consistent with the
filter-based measured for t he s ix s teady-state poi nts s elected. T hus, pe rform an
experiment to determine the f ilter-based m easured c oncentration f or the s ix poi nts
selected and share the results with MASC
Instead of s howing a qualitative r esults on t he m ixing, pr ovide s ome qua ntitative
assessment such as a T-test.
Based on the high spikes observed with the MSS, check if the PPMD results in a low bias
due to the fact that it m ay be mis sing the early spike. F or these experiments, pi ck the
transient N TE w indow with t he hi ghest s pike and create a cycle that consists of 10
repeats of that same window. During this exercise, use only three working crystals and
vary the PPMD trigger into the NTE with time advancement of 1,2 and 3 seconds, time
delay of 1 second, no time delay, and ECM trigger
The above work and the problems encountered were presented to the MASC via two conference
calls t hat w ere don e on S eptember 12 a nd S eptember 30, 2008. T he p resentations a re al so
enclosed as Update 7, and Update 8.
REPORT 03.14936.12 B-15
-------
PM Measurement Allowance Steering Committee Meeting
Meeting at EPA, Ann Arbor
October 9 and 10, 2008
SwRI gave a status update on the progress of testing. See enclosed presentation.
There was a con cern about the predicted lasting time of the PPMDin the field, which was
predicted to be on the order of 1 hour, using an average NTE threshold of 0.02 g/hp-hr. There
was a desire to extend the running time of thePPMD to at least 2 hours. This is indeed met by
the instrument if one takes into account overall collection efficiency of the instrument. In the
example given, it was assumed that the collection efficiency was a 100 %, where in reality it was
on the order of 50 %. Thus the one hour of lasting time reported is in reality two hours.
A decision was made to move forward with testing using only the MPS1, as shown below, after
final c ommissioning of the P PMD b y S ensors. Actually, D avid Booker from S ensors flew t o
SwRI late on October 9 to be at SwRI on October 10.
1. Test on MPS1 only, as its currently configured at one microgram.
2. Okay to use external trigger for steady-state testing
3. For steady-state testing:
a. Target 100 microgram on the filter
b. 50 microgram for TRPM
c. 0.66 microgram for PPMD
Do not clean crystals until it is apparent that the next run will likely overload the filter.
The second day of the meeting was spent at Sensors. Matt Spears gave a presentation on the PM
generator. Later, he s howed t he P M g enerator s etup, a nd e xplained t o t he g roup t he v arious
elements of the PM generator and the equipment used. A copy of Mart's presentation is enclosed.
Also, a copy of Matt's note is shown below, particularly to 1065 PEMS changes:
Matt's Note:
1. Review minutes from last meeting
2. Discuss 1065 changes required for PM PEMS
3. Update from SwRI on recent activities
4. Update from PEMS manufacturers regarding recent phone conference
5. Friday afternoon @ Sensors
a. SUN conference presentation
b. PM Generator
6. Discuss 1065 changes required for **PEMS field testing only**
a. Dilution Air
i. Temperature control
1. 1065=25ฑ5ฐC
REPORT 03.14936.12 B-16
-------
2. PM P EMS= no di rect f eedback c ontrol f rom a di lution a ir
temperature m easurement r equired- use good e ngineering
judgment; if directly and actively controlled then target 25 ฐC .
ii. You mayuse afixed molar mass of the diluted exhaust mixture for all
PEMS field testing, as determined by engineering analysis.
b. "Filter"
i. Media
1. PTFE membrane or TX-40
ii. Face velocity
1. 1065= target near 100 cm/s actual, unless overloading
2. PMPEMS
a. Flow-through media: (10 to 100) cm/s actual, which can be
verified by engineering analysis
b. Non-flow through media: no specification
iii. Temperature
1. 1065is47ฑ5ฐC
2. PM PEMS target (42 to 52) at all times, with a minimum tolerance
of 32 ฐC and a maximum tolerance of 62 ฐC, where the tolerances
apply only during filter sampling.
iv. Conditions during mass determination
1. 1065=see subpart B
2. PMPEMS
a. If mass is not determined in-situi.e. within the PEMS
then t he s ample c ollection m edia m ust be pr e a nd pos t
analyzed according to 1065.190x.
b. If mass is determined in-situ, follow .195.
c. In s ubpart J , ha ve no r equirement t o hoi d t o de wpoint
specs for in-situ analyzers.
c. Absolute reference for inertial balance
i. Current status: QCM OEM stated specs are assumed.
ii. For 1065 m easurement allowance audit we had S ensors verify frequency
measurement circuit.
iii. No immediate solution available
d. Cleanup 915 table for inertial batch PM analyzers: no freq, or rise/fall time specs.
Recommend a process for determining noise, accuracy, and repeatability
e. 1065 Subpart J needs to state that field testing applies at any ambient temperature,
pressure and humidity, unless otherwise specified in the standard setting part.
f. State that EPA approves of electrostatic deposition technique for PM collection.
Must meet 95% collection efficiency, as stated by the manufacturer.
g. Overall P EMS t est r equirement s hould be r eread a nd e dited t o be applicable t o
batch analyzers. For example describe how to use a combination of steady-state
and transient test modes to determine accuracy and repeatability separately; like
what we're doing in the measurement allowance.
h. 1065.308-09: a Iso r equired f ore ontinuous PM a nalyzersread a nd edit
accordingly
i. 1065 clarify that options after 400 ug loading are optional
j. Clarify whether or not ambient air may be used for zero air for PEMS, including
for hangup check.
REPORT 03.14936.12 B-17
-------
k. Drift: allow any drift that doesn't affect your ability to demonstrate compliance
with the applicable standard.
7. Timetable
a. Next meetings
8. Test Plan-documentation reflecting the latest agreements
a. Validationget from CE-CERT
b. Modeling
9. For PEMS testing set dilution ratio based on manufacturers literature regarding maximum
exhaust f low. Y ou m ay also us e ot her m anufacturer i nformation t o pe rform a n
engineering analysis to estimate the maximum.
10. There will be no dilution ratio verification.
REPORT 03.14936.12 B-18
-------
PM Measurement Allowance Steering Committee Meeting
Meeting at SwRI, San Antonio
November 12 and 14, 2008
Flow Alignment in PEMS
In case of continuous sampling, use a step function to measure the delay from the probe to the
instrument.
In case of batch sampling, use the geometry to account for time delay from the probe to the batch
sampler
The above is done to align the flow or to account for time delay.
Leave alone any time alignment between exhaust flow and ECM torque and speed. Use the ECM
torque and speed to determine the integral over the NTE.
Loss Correction
a. The principle of PM loss corrections for PM PEMS is agreed upon by the steering
committee
b. EMA desires a legal construct in 1065 for allowing the use of PM loss corrections
i. Open up 1065.295 to allow more types of compensating algorithms, based
upon other variables
ii. Utilize S ubpart J ove rail a pproval te st to validate e ntire P M P EMS,
including its loss corrections.
The PEMS manufacturers will be allowed to use compensation algorithms.
Horiba decided to use no particle loss algorithm.
AVL p resented a 1 oss a Igorithm toe orrect f or t hermophoresis. T he 1 oss c Direction i s
already defined and will be implemented via a post processor provided by AVL
Sensors plans to correct for particle loss and will share the process with MASC during
the next meeting
As of today, no filter data can be shared with Sensors unless the loss correction is shared with the
MASC.
Test Matrix, DPF out
Full day no QCM cleaning or filter changing
Storage, high speed, light load
Release at peak torque, run for 20 minutes
Store again at high speed, light load
Release at rated power
REPORT 03.14936.12 B-19
-------
Do another storage at low idle, equivalent
Cycle the PEMS every one minute to sample into NTE.
Cycle the PEMS every 32 seconds
5 second dwell time in between the NTEs
PEMS Daily Checks Tolerance
Slope of 0.96 is acceptable by Sensors
For Horiba, 3% on filter flow, and 3 percent at a dilution ratio of 5, and 5 percent at a dilution
ratio of 15, and Mike will reconfirm.
We will get a feedback from AVL on the accepted tolerance for a dilution ratio of 5.f
Milestones
Milestone for the Model dry run
PM generator milestone for commissioning, week of January 5
Milestone for the fuel flow error surface delivery by the engine manufacturers (EMA will target
the end of February for these data to be available. The eight or nine engines that are available
now will be used for the dry run).
Fixed date for the delivery of environmental chamber kit by the PEMS manufacturer.
Ship it on the 19th of January by Sensors and Horiba.
We will schedule EMI, RFI, on t he 26 of January, and shock and vibration on t he following
week.
Matt Spears' Note:
Wednesday
1. Upcoming meetings, December, January, and March all at SwRI
a. December 10-12 at SwRI (10th 2pm start, 12 2pm close)
b. January 28th - 30th (28th 2pm start, 30th 2pm close)
c. March 18th - 20th (18th 2pm start, 20th 2pm close)
2. Review October meeting minutes
3. EPA / SwRI / CE-CERT PM filter round-robin
a. Initial results
4. Temperature-Humidity test cycle
a. EPA cycle
b. SwRI addition of CE-CERT frequency content
5. Status and progress at SwRI since last meeting
a. Update on mission time projections
i. Relook at projections to see if it is possible to collect 20 failed NTE events
( 0.02 g/bhp-hr) in one mission
b. Test pi an revi ew
c. Decisions for MASC
6. Budget update
a. Arrangements for EPA/EMA/ARB
REPORT 03.14936.12 B-20
-------
Thursday
7. PEMS PM loss algorithms
a. AVL
b. Sensors
c. Horiba
8. Test plan development
a. Inclination discussion
9. Continue if necessary: Status and progress at SwRI since last meeting
a. Decisions for MASC
Friday
10. Continue PM PEMS related Part 1065 changes
11. Other meetings during our November meetings
a. Wednesday
i. Wrap-up by 5:30pm due to another room reservation
b. Thursday
i. 12:00pm EM A, Rey Agama - break for lunch at 12:00pm
ii. l:00pm - 2:00pm, Shirish Shimpi, continue meeting
c. Friday
i. Matt Spears 10:00am to 10:45am, continue meeting
12. Loss corrections resolutions
a. The principle of PM loss corrections for PM PEMS is agreed upon by the steering
committee, including EM A, EPA, ARB.
b. EMA desires a legal construct in 1065 for allowing the use of PM loss corrections
i. Open up 1065.295 to allow more types of compensating algorithms, based
upon other variables.
ii. EPA agrees that PM los s corrections will not be applied to certification
testing. If i n t he future E PA de sires t o a pply P M 1 oss c Directions t o
certification t esting, s uch a pr ovision w ould b e pr oposed as pa it of a
notice of pr oposed r ulemaking be cause E PA a cknowledges t hat s uch a
change would cause a cha nge in the s tringency oft he ce rtification
standard.
iii. EPA w ill be t he a pproving bod y w ith r espect t o P M P EMS P M 1 oss
correction.
1. May make case-by-case approvals, based upon specific PM PEMS
manufacturer circumstances, such as, but not limited to submitted,
models, theory, validation data, or even simply the magnitude of
the correction.
2. May utilize Subpart J overall approval test to validate entire PM
PEMS, including its loss corrections.
3. May develop (with consultation with EMA) other procedures for
codification within Part 1065.
13. Sensors w ill pr ovide ana dvance c opy of S ensors' D ecember pr esentation, r equesting
EMA question consolidation ahead of meeting.
REPORT 03.14936.12 B-21
-------
PM Measurement Allowance Steering Committee Meeting
Meeting at SwRI, San Antonio
December 10-12, 2008
SwRI p resented t he s teady-state t esting r esults, and the s torage and release and r egeneration
results. The presentations were posted on the FTP site.
Sensors presented their approach to particle 1 oss correction. A presentation i s available at the
FTP site.
Craig Kazmierczak presented some in-use PM-PEMS work that was done on one of DDC trucks
using the PM-PEMS equipment. The work was done by Sensors and presented by Craig.
The MASC requested that the reference crystal be used during SwRI laboratory activities, if that
is to be used on the road.
The t esting done b y S wRI s o f ar i s acceptable. F or t he pa rticular P PMD us ed, di sable t he
reference crystal, and use a working c rystal t o be a r eference. Essentially, operate w ith six
working crystals, and use one for reference.
Sensors intends to use the reference crystal for correction, but they are going to use a logic to
decide whether or not it will be used in the post processor.
The pos t pr ocessor s hould be a vailable be fore January 23, 2009. T he post pr ocessor s hould
include any loss correction intended or any reference crystal correction model, remove all Part
1065 excursions, and all other miscellaneous items that will make a measurement invalid.
In t he current A VL pos t pr ocessor, t hermophoretic 1 oss w as c apped at 25 % . A VL w ants t o
change that to include the full range of the model, and remove the 25 % cap. Both corrected and
uncorrected data will be provided in the output of the post processor.
We agreed to use the paired analysis for steady-state and for model validation.
We agreed to use 5 % COV at a 1 sigma standard deviation for the CVS at 100 microgram filter
loading.
The a bove s ubject w as t abled f or f urther di scussion on how t o s ubtract t he C VS e rror
contribution. Bill Martin will send his proposal to Bob and Imad.
How to account for a regeneration in the field:
NTE event > 30 seconds to be valid
1-Discrete
2-Triggered by ECM
3-The regen is defined between two regen flags
4-If the regen occurs during an NTE event, and the length between two consecutive NTE flag x 2
is shorter than the NTE event, then the regeneration is counted
REPORT 03.14936.12 B-22
-------
5-If the regen occurs before an NTE event, and end outside the NTE event, the NTE will count.
Tim F rench mentioned that t he s ubject of r egeneration a nd i ts i nclusion i n N TE ne ed t o be
discussed in a different forum.
Blow-By and how it will be computed for the measurement allowance. Next meeting
As of now, the results reported for storage and release and for regeneration will not impact the
MA program.
Matt will share the filter results during the next meeting.
REPORT 03.14936.12 B-23
-------
PM Measurement Allowance Steering Committee Meeting
Meeting at SwRI, San Antonio
January 28-30, 2009
The 1 ast m inutes w as reviewed and a pproved. T he s cheduling for future m eetings and
conference calls was as follows:
Conference Call, Thursday, Feb. 26, 2009, 9:00 am Central, length will be decided on Friday.
Next MA-SC, April 1st (2:00 to 6:00), 2nd (9:00 to 5:00) and 3rd (8:00 to 2:00), San Antonio.
Week of the 18th of May for EMTC and MASC (May 20, 21, 22, same as the April meeting)
Some of the in-use testing performed by Ce-Cert will include regeneration events. No
action will be taken on modifying the test plan at SwRI. The test plan will remain the
same and it will not include active regeneration.
Bill Martin explained his proposal on the CVS variability addition to SS delta data.
Bill Martin presented the analysis on how to assign the CVS variability into the delta
between PM-PEMS and CVS in order to shrink the 95th and 5th A copy of his write-
up is on the FTP website
SwRI presented work on the progress made. A copy of the presentations is posted on
the FTP website
Rey Agama presented an argument about using standard deviation i nstead of MAD
for the data analysis. Matt Spears suggested that the MAD should be used, and if no
validation w as obs erved a 11 he e nd of t he p rogram, ot her pos sibilities c an be
considered. The group agreed to move forward with this approach. Rey asked that we
apply a normality test on the data, and he will consult with Bob Mason on that. This
is in relation to applying a MAD or SD for the model. Bob Mason will present some
material on the normality criteria during the next meeting.
The contribution of blow-by will be a constant based on the data presented from the
four ACES engines at 0.00042 g/hp-hr. If the crankcase is vented to the atmosphere,
this value will be added to every NTE emissions value. If the crankcase is closed
loop, t he bl ow-by contribution of 0.000 42 g /hp-hr w ill not be a dded t o t he N TE
emission values.
Horiba was allowed to fix a bug in their system in relation to delays between engine
and OBS and OBS and TRPM
Horiba was allowed to make modifications on the ion trap voltage of the EAD for the
third TRPM unit.
REPORT 03.14936.12 B-24
-------
The engine manufacturers agreed to submit the ECM/Lab Torque and Fuel Flow by
March 31st, 2009. The data will not be linked to a particular manufacturer.
The i nstrument m anufacturers ne eds t o s ubmit t he 1 atest pos t pr ocessor b y F riday,
March 6, 2009
The MASC agreed that the AVL measurement allowance will be performed based on
data m easured plus t hermophoretic cor rection. The M ASC a grees t o also see d ata
from the AVL MSS total PM prediction on a non-interference basis from the core of
the measurement allowance program.
The MASC was updated on the PM generator via a laboratory tour. The PM generator
is currently set up at SwRI Particle Laboratory.
Below is the unedited Matt Spears' minutes:
1. January meeting agenda
a. Next meetings?
b. EPA participation/management in MASC
c. Review of Meeting minutes
d. Regen in NTE discussion
e. Bill Martin's paired testing proposal
f. SwRI data
i. Transient
ii. 2ndsetofPEMS
iii. Lessons learned, problems?
iv. Analysis of MSS with sulfate and HC corrections
v. ECM vs test cell torque and BSFC error surface update
1. SwRI approach
g. PM Generator update
h. Environmental chamber update
i. PEMS mfr readiness
ii. SwRI readiness & schedule
iii. PM Generator readiness
i. Next face-to-face meeting: April 1 afternoon, to April 3r afternoonSwRI
j. May 13-15, placeholder-SwRI
k. Bypass sizing for model validation work
i. DDC engine
1. Filter results
m. Friday Schedule
i. 9:00am start
ii. Review test plan timeline
iii. Environmental chamber update
1. PEMS mfr readiness
2. SwRI readiness & schedule
3. PM Generator readiness
REPORT 03.14936.12 B-25
-------
iv. AVL data with corrections
v. Horiba general topic
vi. Finalize agenda for February 26th meetingfinalized, finalize time. 9am
central. 4 h rs. 12: 00pm C ST - 4:00pm C ST ( lpm-5pm E ST) S hirish
travelling to NRMM in Ispra Feb 26th
1. Agenda
a. Make this a LiveMeeting, this is ok w SwRI
b. Update on PEMS manufacturer post processors: 1-hr
c. Timeline upda teSwRI: t est eel 1 and environmental
chamber testing: 30 min
d. CE-CERT upda te on a bility toe ome t o S wRI f or
correlation testing & progress on bypass: 30 min
e. SwRI data
i. PEMS s etnu mber2:s teady-state & pe rhaps
transient r esultssummary onl y: a nything
remarkably different than the 1st set of PM PEMS:
1-hr
ii. Any new issues or difficulties: 30 min
f.
vii. PM Generator / nanoparticle lab tour
viii. Engine manufacturers to submit ECM/test cell torque/fuel rate data
by March 31st meeting
ix. Rudy's for lunch
REPORT 03.14936.12 B-26
-------
PM Measurement Allowance Steering Committee Meeting
Meeting at SwRI, San Antonio
April 1-3, 2009
PM-PEMS Meeting: April 1-3, 2009.
Wednesday, 2:00-6:00 PM
Meeting Minutes from previous meeting were reviewed
Practice measurement allowance on incomplete set of data was presented by Janet, see
presentation on website
The viability of using Method 3 was discussed. There was a general agreement that this
method ne eds t o be dr opped out m ainly b ecause t he 1 ack o f i nformation on t ime
alignment between ECM fuel flow and gas-based fuel flow, but such decision needs to be
reconfirmed during the next meeting.
Thursday, 9:00-5:00 PM
The nor mality t est r equirement w as di scussed b y B ob M ason, s ee p resentation on
website. The decision for now i s not to assume normal di stribution and use the MAD
instead of SD. If the MAD failed to produce a validated measurement allowance, the SD
will be revisited.
David Booker from Sensors presented the features of the new PPMD post processor, see
presentation on website
Mike Akard from Horiba gave an update on t he status of Horiba's post processor, see
presentation on website
SwRI gave a presentation on the following (see presentation on FTP website):
Status update and project progress
SS data from all the PM-PEMS
Transient data with and without engine drift from all the PM-PEMS
OC/EC data for SS
EEPS data for SS
Some MSS corrected data for sulfate and HC
Reference N TE using m ethod 3 g ave di fferent va lues t han M ethod 1 and 2, a nd
decision will need to be made on Method 3 during the next meeting.
Friday, 8:00-2:00
Vibration for offroad was di scussed. SwRI pi ans to do a vibration sweep similar to on-
highway but while the PEMS sitting at the vibration table at 45 degree angle. SwRI will
present a cost estimate for this additional activities when the vibration activities start.
SwRI s howed t he f uel a nd t orque da ta s ubmitted by all e ngine m anufacturers t hat
included C aterpillar, C ummins, D etroit D iesel, Navistar, a nd V olvo P owertrain. S wRI
will present all data together during the next meeting.
REPORT 03.14936.12 B-27
-------
Some of the decisions/action items made during this meeting were as follows:
A decision on whether or not Method 3 should be used (AVL only) needs to be made
Reference NTEs will be tweaked so the main distribution of events will be between
0.015 g/hp-hr and 0.035 g/hp-hr
SS error surfaces will be presented during the next meeting
Transient data was drift corrected using the CVS, but the MASC desired to look at the
integrated AVL data as compared with the CVS to look at the possibility of using real
time AVL data for drift correction. The integrated AVL for the NTE cycles will be
compared with the i ntegrated C VS during the next m eeting, before m aking a f inal
okay on the CVS drift correction method
EPA was to provide some information on t emperatures experienced during off-road
in-use a ctivities s o i t c an be i ncorporated w ith t emperature a nd hum idity pr ofile
during environmental testing. EPA was to propose a final temperature and humidity
profile for the program, after incorporating the CE-CERT and off-road data
Off-road vibration tests will be added at 45 degree angle
Fuel a nd t orque e rrors us ing t he e ngine m anufacturers' s ubmitted fuel a nd t orque
needs to be presented during the next meeting
REPORT 03.14936.12 B-28
-------
PM Measurement Allowance Steering Committee Meeting
Meeting at SwRI, San Antonio
May 20-22, 2009
PM-PEMS Meeting, May 20-22, Meeting Agenda
1. Review of last meeting minutes
2. Next Meeting Schedule
3. Overall Proj ect Status Update
4. CE-CERT and SwRI Correlation
5. Update on EMI and RFI Testing
6. Update on Environmental Chamber Testing
7. SS Error Surfaces and Adjusted Reference NTE events
8. Other agenda items and questions
The last meeting minutes were reviewed
The next meeting schedule was set to July 15-17 in Indianapolis
The overall project status we presented. See presentation by SwRI
CE-CERT and SwRI Correlation was presented. See presentation by SwRI
EMI and RFI testing results were presented. See presentation by SwRI. No error
surfaces will be generated. The problems mainly affected instrument functions.
i. Horiba will investigate the issue related to Bulk Current Injection effect on
the Horiba exhaust flow
ii. Sensors will demonstrate a fix to the exhaust flow problem during the next
meeting. Sensors will conduct their own testing if necessary.
SS error surfaces were presented. Sensors data were not fully analyzed because
the new post processor was not provided to SwRI. Sensors promised to provide a
new pos t pr ocessor r esolving t he i ssues i dentified b y S wRI dur ing t he 1 ast
meeting to increase data yield. Sensors later provided the new post processor, and
the S S error surfaces were presented during a co nference call that took place on
June 29, 2009. See SwRI presentation for the June 29 conference call.
The transient errors were also di scussed, and the approach for the transient error
surface will be presented during the next meeting in Indianapolis, along with a
final recommendation
Sensitivity on fuel flow a nd CO2 flow f or M ethod 2 a nd M ethod 3 will be
discussed during the next meeting in Indianapolis
REPORT 03.14936.12 B-29
-------
PM Measurement Allowance Steering Committee Meeting
Meeting at SwRI, Indianapolis
July 14-17, 2009
1. Last Minute Review
2. SwRI Presentation
a. Project Update
b. SS and Transient Error Surfaces treatment and results
c. Environmental T esting ( Atmospheric P ressure, T emperature and H umidity)
Results and Discussion
i. Three approaches were presented on how to treat the error surfaces for the
environmental testing.
ii. Approach 3 was used. The steering committee agreed with using approach
3
iii. No error surfaces were obtained for Horiba and Sensors for environmental
testing
iv. Error s urfaces for A VL, but a nything b elow t he 1 owest M AD, s et t o a
constant equal to the lowest MAD, and anything above the highest MAD
set to a constant equals to the highest MAD
d. For Method 3, and for the reference NTEs, we agreed that the gas based fuel flow
will be advanced in order to match the ECM fuel flow.
3. Rules on Measurement Allowance:
a. Pick t he pos itive m easurement a llowance t hat i s c loset t o zero ba sed on t he
Horiba's and Sensors' PEMS.
b. If both Horiba's and S ensors' PEMS have a negative m easurement a llowance,
pick the one that is closest to zero.
4. Preliminary conference call scheduled for August 20, 2009. We will confirm it tomorrow.
At the August's conference call a de cision on the measurement allowance will be made
and an instrument will be selected to go to CE-CERT. After that, SwRI will ship the units
to CE-CERT to arrive at CE-CERT earlier than September 1st
5. Meeting at CE-CERT on September 22nd to observe and check PEMS installation by CE-
CERT.
REPORT 03.14936.12 B-30
-------
PM Measurement Allowance Steering Committee LiveMeeting
August 20, September 11, 2009
1. SwRI presented the Monte Carlo Simulation (See Presented at FTP website)
2. The m easurement a llowance w as de termined b ased on M ethod 2 us ing S ensors'
PPMD
3. The measurement allowance was 0.00605 g/hp-hr
4. Sensors' PPMD was chosen for CE-CERT in-use testing
5. It was agreed by the MASC that the AVL MSS will also participate in in-use testing
along with the PPMD, just like it was done in the laboratory
6. The MASC requested the following from SwRI
a. plot the Sensors and Horiba 95th delta on the same plot
b. show the results of the simulation based on reference data available within the
concentration range obtained in the laboratory
c. plot the 95th, 50th, and 5th, for the validation
d. Refreshment on the regression rules for the validation deltas
7. Address Item 6 above in a Livemeeting on September 11
8. The requests in Item 6 above were addressed in a livemeeting on S eptember 11 (see
presentation on FTP website)
a. TheMASC requested that the validation deltas forthe 95th, 50th, and 5th be
regressed using the LOESS f itting r ule s ince the c riteria s et f or 1 inear
regression is not met.
b. The LOESS fit should be done on Sensors' validation deltas based on Method
1 and Method 2.
c. SwRI should present the regression the CE-CERT September 22 meeting.
REPORT 03.14936.12 B-3 1
-------
PM Measurement Allowance Steering Committee Meeting
February 17-18, 2010, US EPA, Ann Arbor
Sensors gave a presentation, proposing a correction factor for the exhaust flow.
Sensors calibrated the flow meter in house with the same wrong pressure configuration in
the field
Sensors de termined a c Direction factor of 1.52 for e xhaust flow c orrection for U nit 3.
This correction factor was accepted by the MASC
A similar calibration will be done on Unit 2 to determine the exhaust flow correction
factor to be used for this unit
Sensors i ndicated t hat a dditional c Directions r elated t o b ypass flow t hat w as not c orrected for
during the in-use testing is needed:
The sample flow during NTE is determined as the difference between bypass flow before
sampling and bypass flow during sampling.
If t he b ypass flow i s n ot c orrected for ba rometric pr essure dur ing s ampling, a nd t he
barometric changes during the long NTEs, then there would be an error introduced to the
sample flow.
AVL di scussed t he a pproach t hey t ook toe alculate t he br ake-specific P M e missions us ing
Method 2. The ECM broadcast fuel term in the equation was essentially frozen if it went below
5% of the max fuel flow encountered during the test. However, to move forward with Method 2
calculation, the MASC agreed that to the following:
The MAX fuel flow provided by the engine manufacturer should be used not the max
fuel determined during a test. Thus, the engine manufacturer of the CE-CERT vehicle
should provide the information to CE-CERT to do proper Method 2 calculation
The fuel flow along with the gas concentration terms in the equation will be frozen if the
ECM fuel flow dipped below 10% of the MAX fuel flow.
CE-CERT gave a presentation on the different exhaust flow correction attempts they made. See
CE-CERT presentation at the FTP website.
SwRI pr esented va lidation f or P PMD U nit 2 and 3 us ing M ethod 1 with a n e xhaust flow
correction of 1.52. T he P PMD ba sed on M ethod 1 f ailed t he va lidation c riteria. S wRI a Iso
showed different scenarios for the AVL, including Method 1, 2, and 3. The presentation is listed
on the FTP website.
Below is a summary of the action items for CE-CERT and SwRI as a results of this meeting.
These action items were compiled by Chris Laroo:
Follow-up Work to Finish PM MA Test Program
1) Agreed to exhaust flow correction factor of 1.52 for unit #3.
REPORT 03.14936.12 B-32
-------
2) Will use exhaust flow correction factor for unit #2 that comes out of Sensors check of the
unit #2 flow meter on their flow stand (value anticipated to be at or near 1.52 but needs to
be determined). Value is anticipated to be known the week of March 1st.
3) Correct PPMD data for incorrect setting in QCM bypass flow. S oftware setting change
from 0 to 1.
4) Use Semtech D S barometric pressure readings to account for altitude change effects on
data. If it is determined that the PPMD barometric pressure reading is faulty, we will
request that S ensors up grade t he s ensor to the quality used in the DS. This could be
deemed a special source of error and accounted for via a future hardware improvement.
5) PPMD Unit #1 data will be reported for single crystal use only to reflect that fact that the
Mass S ensitivity w as i ncorrectly e ntered into the s oftware dur ing te sting. T his s ingle
crystal use data will then be pooled with the multi-crystal use results from units #2 and #3
to determine the final validation % results.
6) Report PPMD Unit #2 and #3 deltas for single crystal use only as a probing exercise to
see if multiple crystal use has an effect on mass loss. The intent of this exercise is to
gauge the effectiveness of proposed fixes by Sensors to eliminate the PPMD low bias for
future pilot and compliance program testing. T his data will be pooled with the unit #1
single crystal use results for plotting in the validation window for experimental purposes
only. These results will not be used to determine validation.
7) Method #2, i f the fuel flow rate drops below 10% of manufacturer declared maximum
fuel rate value, then the ratio of the emission concentration terms and the ECM broadcast
fuel rate will be frozen at that value.
8) CE-CERT will reprocess all PPMD and MSS data with correct factors.
9) CE-CERT w ill w rite a final r eport t o be reviewed by t he s teering committee be fore
fmalization.
10) SwRI w ill pi ace r evised CE-CERT da ta i n v alidation w indows a nd c alculate ne w
validation percentages. CE-CERT should have the data to SwRI by the end of March.
11) SwRI will also place revised CE-CERT single crystal usage data in validation windows
and calculate new validation percentages (for experimental purposes only). T his again
will help gauge whether or not there is an effect on mass loss from multiple crystal usage.
12) SwRI w ill w rite a f inal report t o be r eviewed by t he s teering committee be fore
fmalization.
REPORT 03.14936.12 B-33
-------
APPENDIX C
CRYSTAL BALL OUTPUT FILE DESCRIPTIONS
REPORT 03.14936.12
-------
EXTRACT DATA FILES
1.0 Simulation Variables
The simulation variables listed in Table B-l were extracted at the completion of the Monte
Carlo simulation run for each referenceNTE event. Crystal Ball classifies variables into two
categories: assumptions and forecasts. Assumptions are the estimated inputs into the simulation
model such as the variability indices used to sample each error surface. Assumption variables in
this study are identified by an "ic" at the beginning of the variable name, or by "Delta" at the
beginning of the variable name. The "ic" variables are the simulation error model inputs such as
"01_ic_SS_PM". The "Delta" variables serve as switches that turn a given error surface on or
off in the model, e.g. "Delta PM SS". The Delta switch variables when turned on and off during
a s imulation a re applied i n pos t-simulation analysis to determine s ensitivity of results to the
particular error surfaces. Forecasts are values calculated by a forecast formula in the spreadsheet
cells. Examples of forecast variables used in this study are "001 AVL_DePM (g/hp-hr), Method
1" and "005AVL_Valid DePM (g/hp-hr), Method 2".
TABLE B-l. SIMULATION VARIABLES
Variable Name
001AVL DePM (g/hp-hr), Method 1
OOlHoriba DePM (g/hp-hr), Method 1
001 Sensors DePM (g/hp-hr), Method 1
002AVL DePM (g/hp-hr), Method 2
002Horiba DePM (g/hp-hr), Method 2
002Sensors DePM (g/hp-hr), Method 2
003AVL DePM (g/hp-hr), Method 3
004AVL_Valid DePM (g/hp-hr), Method 1
004Horiba_Valid DePM (g/hp-hr), Method 1
004Sensors_Valid DePM (g/hp-hr), Method 1
005AVL_Valid DePM (g/hp-hr), Method 2
005Horiba_l 18_Valid DePM (g/hp-hr), Method 2
005Sensors_Valid DePM (g/hp-hr), Method 2
006AVL_Valid DePM (g/hp-hr), Method 3
01_ic_SS_PM
02_ic_TR_PM
04_ic_Atm.Pres_PM_AVL
Description
MC Delta PM Method 1 for AVL PEMS
MC Delta PM Method 1 for Horiba PEMS
MC Delta PM Method 1 for Sensors PEMS
MC Delta PM Method 2 for AVL PEMS
MC Delta PM Method 2 for Horiba PEMS
MC Delta PM Method 2 for Sensors PEMS
MC Delta PM Method 3 for AVL PEMS
Validation M C D elta P M M ethod 1 f or
AVL PEMS
Validation M C D elta P M M ethod 1 f or
Horiba PEMS
Validation M C D elta P M M ethod 1 f or
Sensors PEMS
Validation M C D elta P M M ethod 2 f or
AVL PEMS
Validation M C D elta P M M ethod 2 f or
Horiba PEMS
Validation M C D elta P M M ethod 2 f or
Sensors PEMS
Validation MC D elta P M Method 3 f or
AVL PEMS
Random Sampling Variability Index for SS
PM Error Surface, applied to AVL, Horiba
and Sensors PEMS
Random S ampling V ariability I ndex f or
Transient P M E rror S urface, applied to
AVL, Horiba and Sensors PEMS
Random S ampling V ariability I ndex f or
PM A tmospheric P ressure E rror S urface,
applied to AVL PEMS
REPORT 03.14936.12
C-l
-------
05_ic_Amb .Temp_PM_AVL
07_ic_SS_CO
10 ic Pressure CO
11 ic Temperature CO
13_ic_SS_NMHC
14_ic_TR_NMHC
1 6_ic_Pressure_NMHC
17 ic Temperature NMHC
1 9_ic_NMHC_Ambient
20_ic_SS_flow
21 ic TR Flowrate
22 ic Pulsation flow
23 ic Swirl flow
25_ic_Radiation_Exhaust Flow
27 ic Temperature Exhaust Flow
28 ic Pressure Exhaust Flow
29_ic_TR_Torque
30_ic_Torque_DOE
31 ic Torque Warm
32 ic Torque IP
34 ic Torque Interpolation
35_ic_Torque_Engine Manufacturers
42 ic Fuel Engine Manufacturers
43 ic TR Speed
44 ic TR Fuel Rate
Random S ampling V ariability I ndex f or
PM Temperature Error Surface, applied to
AVL PEMS
Random Sampling Variability Index for SS
CO
Random S ampling V ariability I ndex f or
CO Pressure
Random S ampling V ariability I ndex f or
CO Temperature
Random Sampling Variability Index for SS
NMHC
Random S ampling V ariability I ndex f or
Transient NMHC
Random S ampling V ariability I ndex f or
NMHC Pressure
Random S ampling V ariability I ndex f or
NMHC Temperature
Random S ampling V ariability Index f or
Ambient NMHC
Random Sampling Variability Index for SS
Exhaust Flow
Random S ampling V ariability I ndex f or
Transient Exhaust Flow
Random S ampling V ariability I ndex f or
Exhaust Flow Pulsation
Random S ampling V ariability I ndex f or
Exhaust Flow Swirl
Random S ampling V ariability I ndex f or
Exhaust Flow EMI/RFI Radiation
Random S ampling V ariability I ndex f or
Exhaust Flow Temperature
Random S ampling V ariability I ndex f or
Exhaust Flow Pressure
Random S ampling V ariability I ndex f or
Dynamic Torque
Random S ampling V ariability I ndex f or
Torque Design of Experiments Testing
Random S ampling V ariability I ndex f or
Torque Warm-up
Random S ampling V ariability I ndex f or
Torque I ndependent P arameters H umidity
and Fuel
Random S ampling V ariability I ndex f or
Torque Interpolation
Random S ampling V ariability I ndex f or
Torque Engine Manufacturers
Random S ampling V ariability I ndex f or
Fuel Engine Manufacturers
Random S ampling V ariability Index f or
Dynamic Speed
Random S ampling V ariability I ndex f or
REPORT 03.14936.12
C-2
-------
45_ic_SS_CO2
46_ic_TR_CO2
49 ic Temperature CO2
Delta PM SS
Delta PM Transient
Delta PM Atmospheric Pressure
Delta PM Ambient Temperature
Delta CO SS
Delta CO Atmospheric Pressure
Delta CO Ambient Temperature
Delta NMHCSS
Delta NMHC Transient
Delta NMHC Atmospheric Pressure
Delta NMHC Ambient Temperature
Delta Ambient NMHC
Delta Exhaust Flow SS
Delta Exhaust Flow Transient
Delta Exhaust Flow Pulsation
Delta Exhaust Flow Swirl
Delta Exhaust EMI/RFI
Delta Exhaust Temperature
Delta Exhaust Pressure
Delta Dynamic Torque
Delta Torque DOE Testing
Delta Torque Warm-up
Dynamic Fuel Rate
Random Sampling Variability Index for SS
CO2
Random S ampling V ariability I ndex f or
Transient CO2
Random Sampling V ariability I ndex f or
CO2 Temperature
Model s witch c ontrolling P M S S e rror
surface application
Model s witch c ontrolling P M Transient
error surface application
Model switch controlling PM Atmospheric
Pressure error surface application
Model s witch c ontrolling P M A mbient
Temperature error surface application
Model s witch c ontrolling C O S S e rror
surface application
Model switch controlling CO Atmospheric
Pressure error surface application
Model s witch c ontrolling C O A mbient
Temperature error surface application
Model switch controlling NMHC S S error
surface application
Model switch controlling NMHC Transient
error surface application
Model s witch c ontrolling N MHC
Atmospheric P ressure er ror su rface
application
Model switch controlling NMHC Ambient
Temperature error surface application
Model switch controlling A mbient NMHC
error surface application
Model switch controlling Exhaust Flow SS
error surface application
Model switch c ontrolling E xhaust F low
Transient error surface application
Model s witch c ontrolling E xhaust F low
Pulsation error surface application
Model s witch c ontrolling E xhaust F low
Swirl error surface application
Model switch controlling Exhaust EMI/RFI
error surface application
Model s witch c ontrolling E xhaust
Temperature error surface application
Model switch controlling Exhaust Pressure
error surface application
Model switch controlling Dynamic Torque
error surface application
Model s witch c ontrolling T orque D OE
Testing error surface application
Model switch controlling Torque Warm-up
REPORT 03.14936.12
C-2
-------
Delta Torque Humidity
Delta Torque Interpolation
Delta Torque Engine Manuf
Delta Fuel Engine Manuf
Delta Dynamic Speed
Delta Dynamic Fuel Rate
Delta CO2 SS
Delta CO2 Transient
Delta CO2 Ambient Temperature
error surface application
Model switch controlling Torque Humidity
error surface application
Model s witch c ontrolling T orque
Interpolation error surface application
Model s witch c ontrolling Torque ( Engine
Manufacturer) error surface application
Model s witch c ontrolling F uel ( Engine
Manufacturer) error surface application
Model s witch c ontrolling D ynamic Speed
error surface application
Model s witch c ontrolling Dynamic F uel
Rate error surface application
Model s witch controlling C O2 S S error
surface application
Model s witch c ontrolling C O2 Transient
error surface application
Model s witch controlling C O2 A mbient
Temperature error surface application
2.0 Statistics
Descriptive statistics summarizing the values obtained during a single reference NTE event
simulation are provided in Table B-2.
TABLE B-2. DESCRIPTIVE STATISTICS FOR SIMULATION VARIABLES
Statistic
Trials
Mean
Median
Mode
Standard Deviation
Variance
Skewness
Kurtosis
Coefficient of Variability
Minimum
Maximum
Range Width
Mean Standard Error
Definition
Number of times the simulation was repeated
Arithmetic average
The value midway b etween the smallest value and the largest
value
Value that occurs most often
Measurement of variability of a distribution. The square root of
the variance
The a verage of t he s quares of t he de viations of a num ber of
values from their mean
A measure of the degree of deviation of a distribution from the
norm of a symmetric distribution
A measure of the degree of peakedness of a distribution
Standard deviation/Mean
Smallest value
Largest value
Largest value - smallest value
Standard deviation of the distribution of possible sample means
REPORT 03.14936.12
C-4
-------
3.0 Percentiles
Percentiles a re t he probability of achieving va lues be low a particular percentage in the
following increments: 0, 5, 10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 95,
and 100%. Percentiles are computed for each of the simulation variables described in Table.
4.0 Sensitivity Data
Sensitivity da ta a re pr ovided bye omputing t he r ank c orrelation c oefficient for a 11 e rror
surfaces and all simulation variables. T he EXTRACT data file contains the absolute value of
the rank correlation. In post-simulation processing, values of control variable Delta PM S S in
the simulation results were applied to dichotomize the data.
5.0 Trial Values
The value for all simulation variables is provided at each trial of the simulation.
REPORT FILES
1.0 Report Summary
This section includes the simulation start date and time, stop date and time, number of
trials run, sampling type (Monte Carlo), random seed used, and run statistics.
2.0 Forecasts
Descriptive s tatistics, percentiles, and a frequency histogram a re p rovided for forecast
variables 001AVL_DePM (g/hp-hr), Method 1 through 006AVL_Valid DePM (g/hp-hr), Method
3 (see Table).
3.0 Assumptions
Descriptive s tatistics, percentiles, distribution parameters, and a di stribution chart a re
provided for assumption variables 01_ic_SS_PM through 49_ic_Temperature_CO2 (see Table).
4.0 Sensitivity Charts
Sensitivity charts are provided for forecast variables 001 AVLJDePM (g/hp-hr), Method 1
through 006AVL_Valid DePM ( g/hp-hr), M ethod 3 (see Table). C rystal Ball cal culates
sensitivity by computing rank correlation coefficients between every assumption (error surface)
and forecast (delta BSPM emissions) while the simulation is running. Positive rank correlations
indicate that an increase in the as sumption is associated with an increase in the forecast. The
larger the absolute value of the rank correlation the stronger the relationship.
The sensitivity charts developed during the MC simulation are displayed as 'Contribution
to Variance" charts which are cal culated by squaring the rank correlation coefficients for all
assumptions used in a particular forecast and then normalizing them to 100%. Figure displays a
sensitivity c hart f or t he A VL delta P M M ethod #1. T he a ssumptions w ith t he hi ghest
contribution to variance (in absolute value) are plotted at the top of the chart. T his example
REPORT 03.14936.12 C-5
-------
shows a typical dominant effect of the PM S S error surface represented by the 79.5% negative
effect of Delta PM SS. As seen in the example in Figure B-l, as you increase the SS PM there is
an increase i n t he de Ita P M M ethod #1 values r epresented by t he 7.6 % pos itive ef feet o f
ic_SS_PM, and as you increase the torque warm-up there is a decrease in the delta PM Method
#1 va lues r epresented b y the 2.8 % ne gative e ffect of i c_Torque_Warm. O nly t he t op e ight
assumptions are plotted in this sensitivity chart.
Sensitivity: 001 AVL_DePM (g/hp-hr). Method 1
Delta PM SS
01_iC_SS_PM
31_ic_Torque_Warm
02_ic_TR_PM
04_ic_Atm.Pres_PM_AVL
20_ic_SS_flow
35_ic_Torque_Engine
Manufacturers
05_ic_Amb.Temp_PM_AVL
7JW
24%
1.4ฐ
1.4ฐ
-14%
10.7%
FIGURE B-l. SENSITIVITY CHART FOR AVL DELTA PM METHOD 1
REPORT 03.14936.12
C-6
-------
APPENDIX D
MONTE CARLO SPREADSHEET COMPUTATIONS
REPORT 03.14936.12
-------
1.0 DESCRIPTION OF ASSUMPTIONS
The following assumptions were made in running the Monte Carlo model:
Only on e r eference N TE e vent can b e r un at a t ime t hrough t he M onte Carlo
simulation workbook. However, NTE event cases can be stacked and run in a batch
mode.
Uniform (1 second in duration) time steps are used in the reference NTE events.
Standard format and e ngineering uni ts f or r eference N TE da ta established f or t he
project are observed, and applied to the reference NTE event before the NTE event is
entered in the Error Model workbook for Monte Carlo simulation.
Any we t - dry m atter c onversions, i f not ne gligible, h ave b een p erformed on t he
appropriate reference NTE event values before the reference NTE event was entered
in the Error Model workbook for Monte Carlo simulation. No wet - dry conversions
are performed in the workbook.
Any reference NTE event normalizations to produce similar emissions brake-specific
results f rom the thr ee e missions calculation m ethods ha ve be en a ppropriately
performed before the reference NTE event was entered in the Error Model workbook
for M onte C arlo s imulation. N o no rmalizations a mong t he t hree methods a re
performed in the workbook.
PM e missions models f or t hree c alculation m ethods a re c omputed f or t he A VL
PEMS. Only calculation methods 1 and 2 are computed for the Horiba and Sensors
PEMS.
Error surface models and supporting data were approved by the Steering Committee.
The error model spreadsheet has been correctly implemented, and its interaction with
Monte Carlo tools like Crystal Ball is correctly understood.
Random number generation by a Monte Carlo tool like Crystal Ball is correct.
Convergence of the completed MC simulation was processed and checked outside of
the simulation workbook. B enchmark checks on the convergence calculations were
made using a SASฎ computer program.
2.0 WORKSHEET DESCRIPTIONS
2.1 Macro Description
The M aero c an be vi ewed i n t he E xcel s preadsheet' Batch C ontrol' with t he m enu
selections Tools>Macros>Macrol>Edit. The purpose of Macro 1 is to control NTE event batch
processing of stacked cases. For each NTE event case processed, the macro expedites clearing
extra cells below the reference NTE event in the spreadsheet 'Error Model' Methods worksheet
REPORT 03.14936.12 D-l
-------
and deletes extra rows i n the D elta error w orksheets. T he m aero a Iso pe rforms M ode 0
calculations and s tores r esultant' ideal e missions' va lues for a pplication i n s ubsequent M onte
Carlo simulation.
The user must begin with the starter version of the 'Error Model' Excel file which has
300 rows of equations in columns X - CF and in rows 52 - 351 in the Methods worksheet. The
starter spreadsheet also has 300 rows of equations below charts in columns B - F, or B - L, in
applicable Delta worksheets. The us er (when n ot unde r automatic b atch c ontrol) c opies the
reference NTE event into columns A - V, row 52 and down, in the Methods worksheet. It is can
then be confirmed that cell J45 in the Methods worksheet displays the correct number of rows of
the reference NTE event.
Macro e xecution c an be a ccomplished t hrough t he m enu s elections
Tools>Macros>Macrol>Run. Note that t his macro clears eel Is w ithout de leting rows i n t he
Methods worksheet, and deletes rows in the Delta worksheets. This macro will not work if the
reference NTE event has only one row. For a reference NTE event with exactly two rows, this
macro will corrupt the second " check" values in columns B-F type Delta worksheets. C heck
values are not used in the simulation, but are provided as a diagnostic aid. Apply the macro for
reference NTE events with no more than 300 rows.
The reader can follow the description of execution that follows by viewing the macro and
observing the comment rows provided throughout the macro text. In execution, the macro first
reads the contents of J45 in the Methods worksheet. It uses the number of rows in the reference
NTE event defined by J45 to determine how many rows to clear and delete in the spreadsheet. It
checks that the number of rows is between 2 and 299, inclusive. It will also execute correctly for
300 rows.
Next, the macro clears cell contents in columns X - CF below the reference NTE event in
the M ethods w orksheet. N ote t he m aero, as written, will not execute properly if the starter
spreadsheet has been revised with row insertion or deletion in certain areas of the spreadsheet.
As written, the macro initiates in cell X52, counts down through the NTE Event rows, and clears
contents in the range from there in column X through cell CF351.
Next, the macro deletes extra rows below the reference NTE event, where applicable, for
example in Delta worksheet 07. For Delta worksheet 07 it initiates in cell B79 and counts down
through the rows of the reference NTE event to the first row to be deleted. It selects the range of
rows from there down through row 378, deletes the rows, copies some equations and a value to
the last row in the range the charts use, and returns the cursor to cell F68 leaving the display
more or less centered on the charts in the worksheet.
Subsequently, the macro performs similar operations in other Delta worksheets; however,
the initiating cell and final row differ among the worksheets. The Delta worksheets processed in
this way are 7, 10, 11, 16, 17, 20, 21, 22, 23, 29, 30, 43, 44, 45, 46 and 49.
Following t he r ow d eletion ope rations i n t he D elta w orksheets, or di rectly w hen t he
reference N TE e vent ha s 300 r ows, t he m aero pr epares f or t he M ode 0 ( ideal e missions)
calculation. F irst, i n t he M ethods w orksheet i t c opies t he e quations i n r ow 52, c olumns X
through CF, to the last row in the reference NTE event. This clears any errors introduced in the
REPORT 03.14936.12 D-2
-------
last row; however, it assumes that row 52 is correct. The last cell in column AC (At) is cleared
for aesthetics, since the At values are not applied in the model calculations.
The Mode 0 c alculation is performed by the macro by changing the value in cell A6 of
the Summary worksheet t o 0. T hen i n t he M ethods w orksheet, t he va lues from c ells C U22
through CU30 are pasted (values only) to cells O22 through O30 where they are referenced by
formulas during Monte Carlo simulation. The macro changes the value of A6 in the Summary
worksheet to 2 in preparation for the Monte Carlo simulation, and moves the cursor to cell CT18
of the Methods worksheet.
Additional c omments r egarding t he m aero ope ration a re presented in the f ollowing
section descriptions of the model spreadsheet.
2.2 Worksheet 1: ErrorControl
The E rrorControl w orksheet of t he E rror M odel w orkbook i mplements 31 logic s witch
functions. T he user enters a numerical " 1" in column AD in each row corresponding to error
surfaces t o be i ncluded in the cal culation. A n umerical "0 " i s appl ied to error s urfaces t o be
excluded i n t he c alculation. Corresponding r andom va riables for e rror-surface on -off s witch
random effects sensitivity modeling are implemented in column W under Crystal Ball control.
Error surfaces are numbered 1 through 49. T he numbered error surfaces are defined in
columns A - C, and information pertinent to their usage i s presented in columns E - V of the
worksheet. Column E displays warning messages when an unusual value is monitored in column
D.
The c ontrol s witch e lements i n t he w orksheet a re de liberately pi aced on r ows i n t he
worksheet c orresponding t o t he e rror s urfaces toe xpedite e quation c hecking i n t he M ethods
worksheet where the control switch variables are applied in conjunction with error surfaces from
the correspondingly numbered "Delta" worksheets
The numbered error surfaces and time alignment controls that have been implemented are
defined in the following Table C-l.
REPORT 03.14936.12 D-3
-------
TABLE C-l. ERROR SURFACES USED IN SIMULATION
Component
Delta PM
Delta CO
Delta NMHC
NMHC = 0.98*THC
Delta Exhaust Flow
Delta Torque
Delta Fuel
Delta Speed
Delta Fuel Rate
Delta CO2
No.
1
2
4
5
7
10
11
13
14
16
17
19
20
21
22
23
25
27
28
29
30
31
32
34
35
42
43
44
45
46
49
Error Surface
Delta PMSS
Delta PM Transient
Delta PM Atmospheric Pressure
Delta PM Ambient Temperature
Delta CO SS
Delta CO Atmospheric Pressure
Delta CO Ambient Temperature
Delta NMHC SS
Delta NMHC Transient
Delta NMHC Atmospheric Pressure
Delta NMHC Ambient Temperature
Delta Ambient NMHC
Delta Exhaust Flow SS
Delta Exhaust Flow Transient
Delta Exhaust Flow Pulsation
Delta Exhaust Flow Swirl
Delta Exhaust EMI/RFI
Delta Exhaust Temperature
Delta Exhaust Pressure
Delta Dynamic Torque
Delta Torque DOE Testing (Interacting Parameters Test)
Delta Torque Warm-up(Interacting Parameters Test)
Delta Torque Humidity / Fuel(Independent Parameters Test)
Delta Torque Interpolation
Delta Torque Engine Manufacturers
Delta Fuel Engine Manufacturers
Delta Dynamic Speed
Delta Dynamic Fuel Rate
Delta CO2 SS
Delta CO2 Transient
Delta CO2 Ambient Temperature
The thirty-one (31) error surfaces that have been implemented are included or excluded
by the controls numbered 1-49 identified in Table. When all 31 error controls are on (included
in calculation), the sum of column D in the worksheet ErrorControl is 31.
2.3 Worksheet 2: Summary
The Summary worksheet in the Error Model workbook comprises input mode control in
rows 4-10 and output summary in rows 88 and 119. Other rows in this worksheet are available
for diagnostic purposes.
REPORT 03.14936.12
D-4
-------
The c alculation m ode c ontrol i s a ccomplished with c ell A 6 w here t he us er nor mally
confirms that a numerical value of "2" i s designated. M ode 2 de signates emissions calculation
with all errors applied. Mode 1 corresponds to a calculation of emissions with all errors applied
except environmental errors. Mode 0 designates an "ideal" emissions calculation with no errors
applied. In Monte Carlo error model simulation performed in this study Mode 2 was used.
Mode 0 i s used prior to Monte Carlo simulation to generate the "ideal" emissions for a
given reference NTE event. The Mode 0 values are calculated by entering a value of "0" in cell
A6. T he Mode 0 c alculation and subsequent storing of the "i deal" emissions results may b e
accomplished m anually (as de scribed above) or by ex ercising a pr ovided macro. T he m aero
automatically sets the value in cell A6 to zero, calculates and saves the "ideal" emissions values,
and returns the value in A6 to "2" in preparation for the Monte Carlo simulation. The locations
where the reference NTE event must be entered manually, and the locations where the "ideal"
emissions must be saved (done automatically if the macro is used) are described in the Methods
worksheet section.
Mode 1 in cell A6 is not typically used but can be applied for diagnostic purposes.
The out put s ummary s ection of t he S ummary worksheet i n r ows 88 a nd 119 presents
numerically and descriptively labeled outputs of the emissions and emissions error calculations.
In the output summary, the cells that are highlighted in turquoise color are designated by
Crystal Ball as "Forecast" (or output) random variables.
A total of 14 outputs ("Forecasts") are designated in the Summary worksheet rows 88 and
119 covering t he num ber of out put va lues f rom PM e mission, t hree c alculation m ethods
(Methods 1, 2 and 3) for the AVL PEMS and two methods (Methods 1 and 2) for the Horiba and
Sensors P EMS, for t he ful 1 e rror m odel a nd f or t he va lidation m odel (designated Valid in
Summary worksheet variable labels). All of these "Forecasts" are provided in units of grams/hp-
hr. This variety of calculations was accomplished with the Methods worksheet.
2.4 Worksheet 3: Methods
The Methods worksheet of the Error Model workbook comprises the following areas:
Notes and diagnostic guides are located principally in rows 1 - 21 in columns A - CF,
continuing on row 5 through column DD.
Reference NTE event d ata a re 1 ocated i n rows 35 - 351 of columns A - W. Actual
reference NTE event d ata m ust be ent ered manually (or aut omatically under ba tch
control) starting on r ow 52 i n columns A - V. Cell W52 data must be entered, and i s
provided f or s pecial c ase s tudy w here t he M ethod 3 f low-weighted PM cone entration
may di ffer from Methods 1 a nd 2. One to 300 rows of reference NTE event data are
allowed. Uniform (one s econd i nterval) time steps ar e as sumed r epresented by t he
reference NTE data.
Parameters calculated are located in rows 35 - 351 of columns X - CF. The number of
rows of these parameter equations must match the number of rows in the reference NTE
REPORT 03.14936.12 D-5
-------
event. Excess cells in these columns may be cleared manually or automatically during
execution of the macro.
Mode 0, Ideal Emissions for this reference NTE event are stored in column O rows 22 -
30 (either manually or automatically by the macro). Related data on the same rows are
located in columns CT - DD.
Input ic random variable distributions (Crystal Ball uses the terminology "Assumptions"
for these inputs) are located in rows 26 - 32 of columns AG - CC.
Emissions calculations by three methods are located in rows 6 - 81 of columns CH - CJ
(Method 1), C L - CN (M ethod 2) a nd C P (Method 3). T his part of t he w orksheet
calculates full model and validation model.
2.5 Methods Worksheet:Notes and Diagnostic Guide
In rows 1-22 for columns A - CF, several descriptive labels and references are defined
for us e i n n avigating t hrough t he w orksheet. R ow 5 , columns A -DD, contains column
identification num bers r eferenced i n r ows 7 t hrough 22 ( depending on t he c olumn). F or
example, in column H the values 65 on row 8 indicates that the values in column H (rows 52 and
following rows) are applied in column 65 (BM) labeled on r ow 5. If the user scrolls to cell
BM52 it is observed that the spreadsheet formula in the cell refers to values from column H. The
information in the notes and diagnostic guide was not applied by the spreadsheet in any of the
emissions c alculations. It w as i ncluded w ith the intent to simplify di agnostics b y pr oviding
information on 1 ocations where spreadsheet values were applied elsewhere in the spreadsheet.
Outside the areas indicated above, some other notes, comments and diagnostic guides may be
found in other areas of the spreadsheet.
2.5.1 Methods Worksheet: Reference NTE Event
The reference NTE event used in the simulation was entered in rows 35-351 of columns
A - W. A ctual reference N TE e vent data m ust be e ntered m anually s tarting on r ow 52 i n
columns A - V. Cell W52 data must be entered, and is provided for special case study where the
Method 3 flow-weighted PM concentration may differ from Methods 1 and 2. A minimum of
one and a maximum of 300 rows of reference NTE event data are allowed. Equal time steps (1
second intervals) are as sumed in the reference NTE d ata rows. T he s tandard f ormat a nd
engineering uni ts of reference NTE event da ta e stablished f or t his pr oj ect m ust be obs erved.
These are described in the column headings on rows 47-51, columns A - V.
2.5.2 Methods Worksheet: Parameters
Parameters applied in the three emissions methods are calculated in rows 35 - 351 of
columns X - CF. The number of rows of these parameter equations must match the number of
rows i n the reference NTE event. Excess eel Is i n these col umns m ay b e cl eared manually o r
automatically during execution of the macro.
The formulas applied in rows 52 and down in columns X - CF have been produced by
normal edit-copy (typically of row 52 in these columns) and edit-paste to rows 53 and following
REPORT 03.14936.12 D-6
-------
rows in these columns. The At values displayed in column AC are not used in any calculation,
but are displayed so a user can confirm uniform reference NTE event time sampling. The last
cell in column AC can be cleared (done automatically by the macro). Note that excess cells in
these c olumns m ust be cleared, a nd r ow de letion ope rations s hould not be a pplied s ince t his
would affect other areas in the Methods worksheet.
Certain sums are performed in several columns over the Parameter rows (range of the
reference NTE event). These are accomplished in row 46 in columns AI, AW, AX, BO, BQ, BU
and CA. Flow-weighted PM concentration SS and TR errors are consolidated, with and without
environmental errors, i n t he area of c ells A N40:AB47. Certain constants appl ied in the
calculation are s tored i n c ells A W42, BC42, BI42, BP40 and BP42. O ther c onstants or
conversion factors a re i ncorporated num erically i n s preadsheet formulas. T ypical of the se is
"0.01" to convert a percentage to a fraction.
Specific parameters or variables are calculated in the various columns for application in
all three methods, full model and validation model. Table C-2 lists the parameters used in the
Methods w orksheet, t he c olumns w here t hey are c omputed a nd a br ief de scription of t he
parameters.
TABLEC-2. METHODS WORKSHEET PARAMETER COLUMN DESCRIPTIONS
Methods Worksheet Parameters Column Descriptions
Subject
Engine operating
state percentages
ATime
NMHC
Fuel Rate
Exhaust Flow Calculations
Speed with error
Fuel rate with error
Torque
PM, ng/mol
Column
X-AB
AC
AD
AE
AF
AG
AH
AI
AJ
AK
AL
AM
AN42,
AN43
AO42
AP42
AQ42,
AQ43
AR42
Description
Convert NTE Event variables to percentages: speed, torque, fuel rate, exhaust flow
Displays At between NTE Event rows
Calculate NMHC ppm as 0.98 of THC ppm
Calculate fuel rate g/s based on fuel density of 851 g/L
Convert exhaust flow SCFM to mol/s
Sum exhaust flow errors from Delta tabs 20, 21, 22, 23, 25 27 and 28 expressed in %
of mol/s maximum. Respective ErrorControl tab switches are applied.
Convert the total exhaust flow error in % of maximum mol/s to mol/s
Add the mol/s exhaust flow error to the exhaust flow in mol/s. Mode control logic is
applied.
Add engine speed error from Delta tab 43 expressed as % of engine range converted
to engine speed in rpm. Mode control logic and ErrorControl switch are applied.
Combine D elta t ab 4 2 F uel ( engine m anufacturer) with fuel r ate from D elta t ab 4 4
expressed a s % o f maximum fuel r ate converted to g/s to en gine fuel r ate i n g/s.
Mode control logic and ErrorControl switch are applied.
Sum torque errors from Delta tabs 29, 30, 31, 32, 34 expressed as % of peak torque,
and from Delta tab 35 expressed as % of NTE point torque converted to % of peak
torque. ErrorControl switches are applied.
Add the total torque error expressed as % of peak torque converted to N-m to engine
torque in N-m. Mode control logic is applied.
SumPM SS and TR errors from Delta tabs 1 and 2 for A VL PEMS expressed as
|ig/mol, AN 42 for Methods 1 and 2, AN43 for Method 3. ErrorControl switches are
applied.
Sum PM S S and TR errors from Delta tabs 1 and 2 for Horiba PEMS expressed as
Hg/mol, for Methods 1 and 2. ErrorControl switches are applied.
Sum PM SS and TR errors from Delta tabs 1 and 2 for Sensors PEMS expressed as
|ig/mol, for Methods 1 and 2. ErrorControl switches are applied.
Sum AVL f low- weighted PM concentration to all errors except environmental PM
errors, A Q 42 f or M ethods 1 a nd 2, A Q43 for M ethod 3 . M ode c ontrol 1 ogic i s
applied.
Sum Horiba flow-weighted PM concentration to all errors except environmental PM
REPORT 03.14936.12
D-7
-------
AS42
AQ46,
AQ47
AR46
AS46
errors. Mode control logic is applied.
Sum Sensors flow-weighted PM concentration to all errors except environmental PM
errors. Mode control logic is applied.
Sum A VL flow-weighted PM concentration to all errors including environmental PM
errors Delta tabs 4 and 5, AQ 46 for Methods 1 and 2, AQ47 for Method 3. Mode
control logic is applied.
Same formula as AR42 as no environmental PM error model applied to Horiba flow-
weighted PM concentration.
Same formula as AS42 as no environmental PM error model applied to Sensors flow-
weighted PM concentration.
Speed Torque
AW
Form product of Speed (rpm, all errors case, column AJ) and Torque (N-m, all errors
case, column AM) for application in Methods 1 and 3. C onvert rpm to radians/sec
with 2 Ttradians/revolution, m inutes t o s econds w ith 60s ec/min, N -in/sec to w att h r
with 3600Joules/watt hr, and watt to kw with IQOOw/kw.
AX
Form product of Speed (rpm, no errors for validation case, column O) and Torque
(N-m, no e rrors for validation case, column T) for application in Methods 1 and 3.
Convert r pm to r adians/sec w ith 2 ^radians/revolution, m inutes t o s econds w ith
60sec/min, N -in/sec to watt h r w ith 3 600Joules/watt h r, a nd w att to k w with
lOOOw/kw.
CO and ACO, %
NMHC and ANMHC, ppm
CO2 and ACO2,%
AY
Sum environmental CO errors including errors from Delta tabs 10 and 11.
AZ
Sum other CO errors. Error from Delta tab 7 is the only one developed.
BA
Add the total CO errors expressed as % to engine CO in %. Mode control logic is
applied.
BE
Sum environmental NMHC errors including errors from Delta tabs 16,17 and 19.
BF
Sum other NMHC errors including errors from Delta tabs 13 and 14.
BG
Add t he t otal N MHC e rrors e xpressed a s ppm toe ngine N MHC i n PPM. M ode
control logic is applied.
BK
BL
Sum environmental CO2 errors. Error from Delta tab 49 is the only one developed.
Sum other CO2 errors including errors from Delta tabs 45 and 46.
BM
Add the total CO2 errors expressed as % to engine CO2 in %. Mode control logic is
applied.
Exhaust Flow
[NMHC+(CO+C02)] /
[fuel mass flow rate / Speed
Torque ]
BO
Form product of hydrocarbons fraction plus CO and CO2 fractions (all errors case,
columns BG, BA and BM) and exhaust flow (mol/s, column AI) divided by ratio of
fuel rate (g/s, all errors case, column AK) to speed-torque product (all errors case,
column AW) for application in PM Method 2.
BQ
Form product of NMHC fraction plus CO and CO2 fractions (all errors case, columns
BG, B A and BM) and exhaust flow (mol/s, column AI) divided by ratio of fuel rate
(g/s, no errors case, column AE) to speed-torque product (no errors case, column AX)
for application in PM Method 2 validation.
NMHC + ( C0+C02)
BS
Form sum ofNMHC fraction plus CO and CO2 fractions (all errors case, columns
BG, BA and BM) for application in Method 3.
FuelRate/ [ N MHC + (
CO+CO2) ]
BU
Form quotient, Fuel Rate (g/s, all errors case, column AK) divided by sum ofNMHC
fraction plus CO andCO2 fractions (all errors case, column BS) for application in
Methods.
FuelRate/ [ N MHC + (
CO+CO2) ]
CA
Form quotient, Fuel Rate (g/s, no errors case, column AE) divided by sum ofNMHC
fraction plus C O and C O2 fractions (all errors c ase, c olumn B S) for application in
Method 3 validation.
2.5.3 Methods Worksheet: Mode 0 Ideal Emissions
For the reference NTE event in rows 52 and down in columns A - V, an ideal emissions
value must be calculated and stored for application in the emissions difference calculations. The
ideal cas e c an be cal culated either m anually, or a utomatically b y the ma cro. F ollowing the
calculation, the i deal v alues ar e stored by edit-copy edit-paste-special-values operation to the
cells i n c olumn O , r ows 22-30. T he m anual ope rations de scribed below ar e p erformed
automatically by the macro, if executed, after manually entering the reference NTE event.
REPORT 03.14936.12
D-8
-------
After manually entering the reference NTE event to be simulated and checking that the
number of rows of equations in the Parameters section matches the rows in the reference NTE
event, a num erical "0" can be ent ered i n c ell A 6 of t he S ummary w orksheet. T he M ethods
worksheet should h ave calculated M ode 0 r esults us ing t he reference NTE event. If e rror
messages like "#VALUE or #DIV/0!" are displayed, there is probably still a mismatch between
the rows of the reference NTE event and Parameter equations. When calculated properly (with 0
in Summary A6), the values displayed in the Methods worksheet columns CU, CV and DB will
be e qual on e ach of t he r ows 22 - 30. T he va lues i n c olumn CX a re not yet e qual (unless
previously calculated and stored for this reference NTE event) be cause they reflect the values
stored i n M ethods worksheet column O, rows 22 - 30. T he next manual step i s to edit-copy
column CU, rows 22-30, and store the values by edit-paste-special-values in column O, rows 22
- 30. Now in rows 22 - 30 the columns CU, CV, CX and DB should be equal. The final step is
to return to Summary w orksheet cell A 6 and change the value from Oto2. Atthis point the
spreadsheet c ould be r un i n M onte C arlo s imulation t o pr oduce pr operly s ampled va lues.
However, i f t he us er de sires t o m onitor charts pr ovided i n t he D elta worksheets dur ing t he
simulation, further r ow-matching to the reference NTE event i s required in most of the D elta
worksheets.
The manual operations described in the previous paragraph are intended to explain how
the Mode 0 ideal emissions are calculated and stored for use in the Monte Carlo simulation when
Aemissions values are calculated using the ideal emissions results stored in O22 - O30. The
reference NTE event must be entered with an operation such as a manual edit-copy and edit-
paste or edi t-paste-special-values op eration. T he m aero a utomatically performs t he m ode 0
calculation, stores the mode 0 results in O22 - O30, and changes Summary A6 back to mode 2.
The macro also deletes extraneous rows from all the appropriate Delta worksheets so the
charts t herein di splay pr operly. It i s i mportant to c opy t he reference NTE event i nto a fully
'loaded' starter file with equations filled on 300 rows in the Parameters area, and with full 300
row complement of equation-rows in each of the appropriate Delta worksheets for the macro to
modify the spreadsheet properly.
2.5.4 Methods Worksheet: Input ic Random Variable Distributions
Probability distribution parameters are applied, and simulation trial values of the inputs
are generated i n rows 26 - 32 o f c olumns A G - CC. R ows 26 a nd 27 a re us ed t o i nput
distribution parameters. Rows 28 and 29 contain descriptive labels brought from the appropriate
Delta w orksheet. Row 30 i s a n i nformation-only num ber, row 31 c ontains t he na me 1 abel
applied i n M onte C arlo s imulation to the inpu t i c, and r ow 32 is w here t he M onte C arlo
simulation tool places generated randomly-sampled values during simulation. The values in row
32 are r eferenced by f ormula i n the r espective D elta w orksheets w here t hey are us ed for
interpolation on the error surfaces.
The M onte C arlo t ool in Crystal B all us es the te rminology "Assumptions" for the se
inputs. Two distribution forms are applied: truncated normal (Gaussian), and discrete uniform.
For the normal di stribution, the applied standard deviation i s i n row 27 . In C rystal Ball, the
standard deviation cell on row 27 and the label cell on row 31 were referenced by equation in the
Crystal Ball a ssumption s etup w indow, t he m ean w as i nput as 0, a nd t he di stribution w as
truncated at -1.414319083 and at +1.414319083. Since all the truncated normal ic distributions
REPORT 03.14936.12 D-9
-------
are identical (although the sampled trial values from each will be random in the Monte Carlo
simulation), the Crystal Ball define-copy data and define-paste data operations were applied to
define t he t runcated no rmal di stributions for ot her i c variables on ce t he first one ha d be en
defined.
For the di screte uniform di stributions, the minimum discrete value (1 in all cases) was
applied i n r ow 26, t he m aximum di screte va lue w as appl ied in r ow 2 7 a nd t he ot her r ow
descriptions ar e t he s ame as before. A gain, on e of t hese i nputs w as s etup w ith C rystal B all
"define assumption" and then applied with Crystal Ball define-copy data and defme-paste data
operations to other ic cells on row 32 where a discrete uniform distribution was applied. When
Crystal Ball "Assumptions" were defined, Crystal Ball colored each input cell bright green
During a M onte C arlo s imulation, t he M onte Carlo tool (e.g. Crystal B all) pi aced a
numerical value in each of the ic cells on row 32. Then the spreadsheet was exercised to perform
interpolations i n a 111 he D elta w orksheets. T he r esulting e rror s ample va lues f or t he e ntire
reference NTE event were r eturned t o t he M ethods worksheet Parameters ar ea, and then the
Methods w orksheet Emission Calculations s ection computes Aemissions us ing three methods,
full model and validation to generate one set of the 14 output values described in the Summary
section. T he simulation tool stores the set of random input values from row 32 a s well as the
output values i n an Excel data b ase from which the corresponding s ets of values can later b e
extracted. Once each trial was completed, the simulation tool randomly sampled a second set of
input values from the respective probability distributions, placed the values in the cells on row
32, exercised the spreadsheet again, stored the input and output values, and went to a third trial,
etc. Typically 40,000 to 65,000 trials, depending on the reference NTE event, were used in this
project with this Error Model workbook.
Note t hat t here ar e t hree w ays t he us er c an control the effect of the ic values in the
emissions calculations:
Mode control in Summary A6,
Include / exclude switches in ErrorControl column AD, and
Specification of i nput r andom va riables ( "Assumptions") a nd t heir pr obability
distributions in the Methods worksheet row 32.
These three ways of controlling the ic values are independent, but the effects are interdependent
as follows. M ode control determines what categories of errors are added into the calculations.
Mode controls categories of errors are classified as:
1. Mode 0 - no errors included
2. Mode 1 - "all" but 'environmental' errors included
3. Mode 2 - "all" errors added into the calculations
"All" in this context represents those error surfaces turned on by the switches in the ErrorControl
worksheet. T he i nput r andom va liable di stribution c ontrols t he di stribution of t he s ampled i c
values applied during Monte Carlo simulation for the several Delta error surfaces. Mode and
ErrorControl switches must be appropriately turned on for the effects of the sampled ic values to
be i ncluded i n t he emissions di fference results. These controls affect t he calculations i n t he
Methods worksheet Parameters and Emission Calculations sections.
REPORT 03.14936.12 D-10
-------
2.5.5 Methods Worksheet: Emission Calculations
In t he area of rows 6 - 81 of c olumns C H - CP the brake-specific emissions and
Aemissions cal culations ar e pe rformed using t he va riables and parameters ge nerated in the
Parameters s ection. T hree s ets of col umns, structured similarly, calculate the f ull mode 1,
validation model, time alignment and drift correction for the following methods:
1. Method 1 calculations are applied in columns CH - CJ,
2. Method 2 calculations are applied in columns CL - CN, and
3. Method 3 calculations are in columns CP.
Columns CH - CJ for Method 1 are typical of the methods where the structure is the same, but
the formulas are a little different. Column CH performs the PM emission calculations for the
AVLPEMS, column CI performs for the Horiba PEMS and column CJ for the Sensors PEMS.
The structure of the three columns is the same. Formulas implemented in the three columns are
the s ame, but t he equations i mplementing t he f ormulas a pply va riables a nd pa rameters
appropriate to the respective PEMS.
As an example of the calculation for PM Method 1 we will examine column CH in detail.
The full model calculation was accomplished in cells CH48 - CH54. The ideal emissions result
was brought into the areaby equation in CH51. Full model PM emissions (ePM) in g/kw-hr
were calculated in CH54. C ells CH55 - CH59 are information-only diagnostic aids. T he full
model Method 1 result in CH54 is calculated by the formulas in Figure C-l.
m PM is a flow weighted paniculate matter exhaust concentration in g/mol
ป>^}*fl^m0r
ePM(g/kW-hr) =
Speedi(rpm)*Ti(N-m)*2*3.\4l59*kt
60*1000*3600
FIGURE C-l. BRAKE-SPECIFIC PM BY METHOD 1
In the formula for the full model mode 2, delta error values sampled from the Delta worksheets
1,2, 4 a nd 5 ha ve be en a dded i n m PM S imilarly, de Ita e rror va lues s ampled from D elta
worksheets 20-23, 25, 27 and 28 have been added to the exhaust flow, delta errors sampled from
worksheets 29-32 and 34-35 were added to torque, and worksheet 43 deltas were added to speed.
The At values are equal (1 second) and therefore cancel out of the equation.
Theva lidation m odel c alculation was accomplished in the eel Is C H79 - CH81.
Validation model PM emissions (g/kW-hr) was calculated in cell CH81.
REPORT 03.14936.12 D-ll
-------
Calculations for PM by Method 1 described above for the AVL PEMS in column CH are
similar for the H oriba a nd S ensors P EMS by Method line olumns CI and C J, r espectively.
Similar calculations for PM by Method 2 are presented in columns CL - CN, and by Method 3 in
column CP.
2.6 Worksheet 4: Constants and Equations
The C onstants&Eqns tab was strictly a snapshot of equations used in the brake-specific
emissions c alculations. It di splayed the e quations a nd c onstants i mplemented i n s preadsheet
formulas of the Methods worksheet
2.7 Worksheet 5: SS PM Error Surface
The 7 Delta C O SS worksheet is a t ypical D elta w orksheet. Its functional s tructure,
formulas, charts and operation are very similar to the following worksheets:
20 Delta Exhaust Flow SS
22 Delta Exhaust Flow Pulsation
23 Delta Exhaust Flow Swirl
30 Delta Torque DOE Testing
45 Delta CO2 SS
With minor changes in charts and structure, its function, formulas and operation are also similar
to the following worksheets:
1 AVL Delta PM SS
IHoriba Delta PMSS
1 Sensors Delta PMSS
2 AVL Delta PM Transient
2 Horiba Delta PM Transient
2 Sensors Delta PM Transient
4 AVL Delta PM AtmosPressure
5 AVL Delta PM Ambient Temp
13 DeltaNMHC SS
14 Delta NMHC Transi ent
19 Delta Ambient NMHC
21 Delta Exhaust Flow Transient
25 Delta Exhaust EMI-RFI
29 Delta Dynamic Torque
31 Delta Torque Warm-up
32 Delta Torque Humidity
34 Delta Torque Interpolation
35 Delta Torque Engine Manuf
42 Delta Fuel Engine Manuf
43 Delta Dynamic Speed
44 Delta Dynamic Fuel Rate
REPORT 03.14936.12 D-12
-------
46 Delta CO2 Transient
The following provides a brief summary of the 7 Delta CO SS worksheet:
Rows 1-7 contain descriptive information about the error surface implemented in the
worksheet.
Rows 8-42 present the error surface in columns A - L. Other columns, M - W, on these
rows generate a lookup table used with an interpolation routine.
Figures A, B and C follow.
Rows 76 - 379 calculate the AGO S S error values for each row of the reference NTE
event. These values were returned to the Methods tab Parameters section.
The following paragraphs describe in further detail functions in the 7 Delta CO SS worksheet:
Data from the error surface (rows 13 - 42, columns A - L, in this Delta worksheet) must
be e ntered i n s orted or der ( sorted on Lab N ominal c olumn C i n a scending or der) for pr oper
operation of the x-lookup-interpolation function. T he three figures chart t he e rror function.
Figure A, in similar Delta tabs, may pi ot several da ta s ets v ersus t he x-value, Lab Nominal
(column C ). F igure A y-values are C O % Lab N ominal ( column C ), and m ay also pi ot 99th
percentile, 50th percentile (median) and 1st percentile.
Related error surface data are plotted in Figure B. Figure B plots several data sets versus
the s ame x -value, Lab Nominal (column C ). F igure B y -values ar e t he di fference, CO %
(PEMS) - CO % (lab, nom). The differences plotted may not correspond exactly to the values
shown i n F igure A be cause of t he s tatistical pr ocedure a pplied i n calculating t he di fferences
shown in Figure B. Figure B plots the 99* percentile (column I), the 50* percentile (median)
(column H) and the 1st percentile (column G). In addition to the error surface data, Figure B also
shows the interpolation line designated ic = xx (column V), and the reference NTE event values
on the interpolation line (column F rows 80 through end of the reference NTE event versus Lab
Nominal x-values in column B rows 80 through end of the reference NTE event). W hen ic =
+1.414319083, the interpolation line plots on the 99th percentile. When ic = 0, the interpolation
line plots on the 50th percentile. When ic = -1.414319083, the interpolation line plots on the 1st
percentile. The reference NTE event always plots on the interpolation line, with points at the x-
values in the reference NTE event.
The error surface data were also plotted in the format of Figure C. Again the x-axis was
the same Lab nominal (column C). T his time the y-axis data are the i c values. T hus, the 99*
percentile pi ots at +1.414319083, the 50* percentile pi ots at 0 a nd the 1st percentile pi ots at -
1.414319083. The interpolation line pi ots at the value of ic, and the reference NTE event pi ots
on the interpolation line at the x-values in the reference NTE event. If appropriate value labels
were di splayed i n F igure C, t he va lues w ould represent t he e rror s urface pi otted on a z -axis
above the two-dimensional x-y plane. These error surface values are di splayed graphically in
Figure B.
Now consider inner rows 13 - 41 in the look-up table in columns T - W. Column T is a
repetition of the x-value from column C. C olumn U calculates a row-to-row A for the x-values
in c olumn T f or us e i n i nterpolation. C olumn V c omputes t he i nterpolation 1 ine 1 inearly
interpolated according to the value of ic between the median and the 99* percentile if ic > 0 (on
median if ic = 0 and on 99* percentile if ic = +1.414319083); and between the median and the 1st
REPORT 03.14936.12 D-13
-------
percentile if ic < 0 (on median if ic = 0 and on 1st percentile if ic = -1.414319083). Only one ic
value (from cell E80) is applied in this calculation of the interpolation line. The Microsoft Excel
vertical lookup function VLOOKUP is applied to the table in rows 12 - 42 in columns T - W.
This i s done in rows 80 and down in column F. B ecause of the way the VLOOKUP function
operates, the first row cells T12 and V12, and the last row cell W42 (all three cells distinguished
by darker line borders) contain formulas or values different from the formulas of the inner rows.
The formula in cell T12 assures that the lookup function can always find an x-value in its table.
The formula in V12 and the value in W12 assure that the interpolation in cells F80 to the end of
the reference NTE event data returns the nearest AGO SS value on the interpolation line if the x-
value is outside the range of the error surface lab nominal values.
Before going to the interpolation accomplished in F 80 and down, consider briefly the
formulation on rows 12 - 43 in columns O - R. This formulation considers one x-value from the
reference NTE event, the first one, in cell B80 and selects the two adjacent rows in the error
surface between which to interpolate on t he B80 x-value. T he result i s formed on r ow 43 i n
these columns and then the "check" cell G80 accomplishes the ic controlled interpolation. This
provides an alternative calculation check on one row in the reference NTE event.
Now consider the interpolation for each point in the reference NTE event. Column B,
row 80 and down, brings the lab nominal x-value from the Methods worksheet reference NTE
event. For this Delta worksheet, that x-value is CO % (lab,nom). The out-of-range flags are
information-only indicating points in the reference NTE event with x-value out of the range of
the error surface lab nominal. The i c value for this Delta worksheet was brought into cell E80
from the Methods worksheet ic area. E ach point in the reference NTE event was interpolated
with the same ic value, but with its own x-value. Recalling that the interpolation line in column
V was computed with this one ic value, the x-interpolation between the appropriate two adjacent
rows in the error surface can now be accomplished. This requires using the x-value on each row
in c olumn B , B 80 a nd dow n, i n the V LOOKUP f unction, a nd pe rforming t he r equired
calculation using the looked-up values and deltas from the look-up table. The calculation is done
with the formulas in cell F80 and down. The values computed in column F, cell F80 and down
through the reference NTE event, could be considered elements of a column matrix or vector,
and are returned to the Methods worksheet Parameters section.
In t he M onte C arlo s imulation, t he M ethods w orksheet combines t his reference NTE
event r esult ve ctor from t he 7 Delta C O SS worksheet with similar r esults from ot her e rror
surfaces, calculates Aemissions by three methods, full model and validation to produce a set of
14 output values ("Forecasts" in Crystal Ball terminology) described in the Summary worksheet
section. This was done having input ic values (including one ic value for this Delta CO SS) all
chosen b y random s ample from t he a ppropriate t runcated no rmal or u niform di stribution as
explained i n t he M ethods w orksheet section. T hen a nother s ample s et of randomly s ampled
values was input (only one ic value coming to this Delta function again). T he reference NTE
event CO SS vector was recomputed with the one new ic value, returned to Methods worksheet
and another set of 14 output values was produced. This process was repeated many times until a
statistical conve rgence criterion, described i n S ection 2, w as s atisfied. Typically, 40,000 t o
65,000 s ets of i nput va lues and 14 out put va lues w ere produced t o s atisfy t he c onvergence
criterion with this Error Model spreadsheet.
REPORT 03.14936.12 D-14
-------
The number of r ows i n t he D elta w orksheet r eference N TE e vent a rea (rows 80 a nd
down) s hould m atch t he num her of r ows i n the reference NTE event applied i n t he M ethods
worksheet for proper function of Figures B and C. The starter spreadsheet has been set up with
the r ange of charted reference NTE event s eries e xtending t hrough r ow 379 i n t his D elta
worksheet. T he ba lance of t he s preadsheet s hould calculate cor rectly w hen a r eference N TE
event is properly entered in the Methods tab and Parameters formulas properly aligned, although
figures like B and C will not di splay properly until the last row of the reference NTE event i s
coincident with the end of the range of the charted reference NTE event series. T his could be
done m anually i n each Delta w orksheet where needed, ho wever, t he m aero was d esigned to
convert the fully 'loaded' starter workbook after the reference NTE event was entered in the
Methods worksheet. The macro uses the row count in the reference NTE event, aligns formulas
in the Methods worksheet Parameters area, and eliminates extra rows in the reference NTE event
area of each appropriate Delta worksheet. Again, the macro will do the operations correctly only
on a fully 'loaded' starter workbook set up with 300 rows of formulas in the Methods worksheet
Parameter area, and in each of the Delta worksheets using the reference NTE event.
REPORT 03.14936.12 D-15
-------
APPENDIX E
EMS OPERATION LOG
REPORT 03.14936.12
-------
Date
2/25/2008
2/28/2008
2/28/2008
3/3/2008
3/4/2008
3/4/2008
3/14/2008
3/21/2008
3/24/2008
3/26/2008
PEMS
Horiba
Horiba 1
Horiba 2
Horiba 2
Sensors 1
Horiba 1
Horiba 1
Horiba
1,2,&3
Horiba
1,2,&3
Horiba
1,2,&3
Description
Lost c ommunication with
TRPM laptop on a
regular basis
Could not control dilution
ratio i f e xternal flow
meter is not connected
Java software does not
display the measured
values that are in the
Labview software
Dilution air flow not
stable
Lookup table for MPS2
repeatedly failed
Dilution Ratio c ontrol is
still s omewhat e rratic
Pressure transducer Ptl
would not respond to
calibration
Unable to pass Part 1065
sample flow linearity
verification
Sample flow failed
Horiba check with
provided external flow
meter
Unable to pass Part 1065
dilution flow linearity
verification
Reason
Standard ethernet
cable does not fit
properly into this
laptop
Unknown
Unknown
Dilution air
pressure too high
Unknown
Bad PID
constants
Pressure
transducer was
broken
Sample flow is an
inferred not
measured value
Flow calibrations
needed updating
Dilution flow
coefficients
needed updating
Solution
A industrial grade ethernet
cable was provided by
Horiba which s dvedt his
problem
Horiba fixed the problem
Software update fixed the
problem
Performeda dilution air
flow adjustment per
Horiba
Sensors said the criteria is
too s tringent and it is fine
as longas itvisually looks
good
Adjusted Pro constants
to new values suggested
by Horiba
Replaced with new part
from Horiba
Linearity verification
performed on dilution and
total flow at MASC
request
Flow coefficients updated
for allthree units
Performed " Dilution Ratio
Accuracy Adjustment" as
instructed by Horiba
1REPORT03.14936.12
E-l
-------
Date
4/14/2008
4/22/2008
6/13/2008
6/19/2008
6/28/2008
7/31/2008
8/8/2008
8/12/2008
8/14/2008
8/15/2008
PEMS
Horiba 3
Horiba 3
Horiba
2200
Horiba 1
Sensors 1
Horiba 2
Horiba 2
Horiba
2200
Sensors 2
Horiba
2200
Description
Java software freezes
when loading
Java softare is not
communicating with the
Labview software
Horiba system is unable
to log the ISO- 1576
ECM broadcast from the
International Engine
TRPM software not
reading the same exhaust
flow as OBS-2200
software
Sample valve for crystal
remains in "transient"
state every time it
attempts to sample
The OBS-2200 software
would n ot trigger the
OBS-TRPM to start
sampling du ring an N TE
Dilution flow is tool ow
OBS-2200 software
unable to read the
reference torque value
from J1939 broadcast
C ould n ot get any of the
crystals to os dilate
OBS-2200 laptop would
not boot up
Reason
Wrong
parameters set in
aconfigfile
Improper
software
configuation
Software did not
have this
capability
Calibration
coefficient is
wrong
Stepper motor
attempting to turn
too quickly (not
enough torque)
Connector wired
incorrectly
Unable to enter
the proper data
bit location
Power supply in
CQCM head was
likely burnt out
Unknown
Solution
AVL programmed an
offset into their software
to account for this, laterit
appeared to be a
grounding pr obi em in the
Semtech DS
Parameters adjusted to fix
the problem
International engine was
replaced with a heavy-
duty Volvo e ngine that
uses J1939 broadcast
Manually adj ust the
calibration coefficient to
get the readings to match
Sensors readjusted the
stepper motor speed
Reduced the speed of the
stepper motor
Replaced with connector
from second unit
Perform di lution flow
adj ustment using internal
pressure regulator
Enter the reference torque
value manually
Hard drive was placed in
another identical laptop,
broken laptop was
shipped back to Horiba
2REPORT03.14936.12
E-2
-------
Date
8/18/2008
8/21/2008
8/26/2008
8/27/2008
8/27/2008
8/27/2008
8/27/2008
8/27/2008
8/27/2008
8/27/2008
8/27/2008
8/27/2008
PEMS
Sensors 1
Sensors 1
AVL1
Horiba 1
Horiba 1
Horiba 1
Horiba 1
Horiba 1
AVL1
AVL1
Sensors 1
Sensors 1
Description
Bypass flow was not
updating
Crystal frequencies,
corona currents, and
voltages were dropping
out
Unit switched! ntoZ ero
Checki nthe midd e of
the test
OBS-2200 laptop failed,
would no longer boot
OBS-2200 laptop would
not read Jl 93 9
TP A would not zero
TRPM Java software
unable to 1 og exhaust
flow (Labview works)
BAD c heck w ill fail
repeatedly
Analog output signal to
MS S would clip at 2
mg/m3 range
DRwouldoc casionally
stop controlling
PPMD internal
temperature was out of
limit
Bypass flow from TSI
flowmeter was not
reading in s oftware
Reason
High temperature
causing
communication
problems
Uknown, possibly
a pressure out of
limit
Laptop broken
Unknown
Zero function tied
to gas analyzers,
looking for gas
flow
Error in software
Tolerances are
too tight for test
cell operation,
tolerances
relaxed.
Soot
concentration too
high
Too much
moisture in the
system
Test cell
temperature too
hot
Faulty com cable
Solution
Attempted to reduce test
cell temperature, but
problem persisted
Unit switched back into
sample after approx. 30
seconds
New laptop provided by
Horiba
Sent back to Horiba for
repair
Use the C AL function
instead of the ZERO to
just zero TPA
Installed n ew version of
Java software
Ignored tolerances set in
software for check
Output switched to 0- 10
mg/m3 , D R s etpoi nt
increased from 3 to 6
Unit purged for moisture
overnight, a nd firmware
upgraded
A new back panel was
installedw ith two fans to
promote cooling
Replaced TSI cable
3REPORT03.14936.12
E-3
-------
Date
8/27/2008
8/27/2008
8/27/2008
8/27/2008
9/10/2008
9/10/2008
9/12/2008
9/12/2008
9/18/2008
9/18/2008
PEMS
Sensors 1
Sensors 1
Sensors 1
Sensors 1
Horiba 1
Horiba 1
AVL1
Sensors 1
Sensors 1
Sensors 1
Description
C Q CM c ommunication
dropped out during
testing
C Q CM c ommunication
dropped out during
testing
Corona needle high
voltage erratic
PPMD c ommunication
was dropping out
Compressor tripped
breaker, turned off AVL
unit
Make up air flow was
0.7 1pm instead of 2. 3
1pm
Analog output signal to
MS S would still briefly
clip at 10m g/m3 range
Negative emissions
reported e ven at high
emission levels on N IE
The bypass flow would
increase when crystal 1
on PPMD 1 would
sample
Sample flow dr ifting
during steady state engine
operation
Reason
CQCM power
supply failed
Unknown
Unknown
High internal
temperature
Combining AVL
and compressor
on same ISA
circuit was too
much current
PID constants
incorrect
Soot
concentration too
high
High dilution,
crystal saturation
(no grease),
crystal
stabilization
Crystal may be
installed
backwards so
that it is always
sampling
Temperature
estimate is based
on mixing of two
flows, if the
estimate is off
temperature will
change the flow
Solution
Power supply replaced
CQCMcommchip
replaced with a newer
model
Fixingthe commissue
resolved this problem
Directed cooling air at
PPMD
Horiba compressor
moved to dedicated ISA
circuit
Modified PID constants
Logarithmic analog output
added as a firmware
update
Do not use MPS2, grease
crystals, wait longer for
PPMD to warm up
Set crystal 1 to be the
reference crystal
Adjusted the parameters
in the temperature
estimate
4REPORT03.14936.12
E-4
-------
Date
9/18/2008
9/18/2008
9/20/2008
9/26/2008
9/30/2008
10/9/2008
10/9/2008
10/9/2008
10/9/2008
PEMS
Sensors 1
Sensors 1
Sensors 1
AVL1
Sensors 1
Sensors 1
Sensors 1
Sensors 1
Horiba 1
Description
No corona current was
measured for two crystals
High voltage reading was
low on two crystals
Sample flow
measurement was too
low
Error: MFD Temperature
out of Spec
Lower than expected
emissions for the PPMD
PPMD s ampling de layed
several seconds after
trigger
Inaccuracies in flow
measurement
Crystal s ampling ass oon
as it becomes available,
including in an N TE event
Compressor supplying
dilution air stops working
in middl e of t est ( 2 u nits)
Reason
Crystals were
shorted to ground
Corona needles
were too close to
the crystals
Sample flow TC
was in excess
flow return, which
had low flow due
to connection to
CVS
Test cell
temperature too
hot
Inaccurate sample
flow temp causing
low sample flow
meas, corona
needles not
positioned
properly
Delay in
communications
System uses an
assumed! nlet
pressure for
MPS2
Software logic
Unit shutting off
due to overheat
protection
Solution
Crystals were replaced
Needles repositioned
TC movedi ntot he main
exhaust
Repos ition ed c hiller a ir t o
blow on the A VL unit
Repos ition the sample
flow thermocouple, adjust
corona needle
Increased residcence time
inside PPMD up to 3
seconds
Added pressure
measurement downstream
of MPS1 to account for
changes in pressure due
toMPS2
Software modified so that
an available crystal waits
for the start of the next
NTE event
Cool air provided to the
compressor (temporary
solution)
5REPORT03.14936.12
E-5
-------
Date
10/9/2008
10/10/2008
10/15/2008
10/15/2008
10/15/2008
10/16/2008
10/28/2008
1 1/3/2008
1 1/4/2008
1 1/4/2008
PEMS
AVL1
Sensors 1
Sensors 1
Sensors 1
Sensors 1
Sensors 1
Horiba 1
Sensors 1
Sensors
1
Horiba 1
Description
MSS concentration
reporting too h igh in
Sensors software
Bypass flow too low
PPMD emission results
were inconsistent
PPMD emissions were
lower than expected
PPMD wouldn't s ample
when external trigger
activates on Semtech DS
PPMD unable to
communicate withD S
Dilution flow inaccurate
PPMD block pressure
low
Multiple crystals stopped
oscillating, when one was
enabled another would
disable
Unable torn aintain
setpoint for total flow
near end of the cycle
Reason
Sensor snot
correcting
concentration
from 0ฐC to
20ฐC (-8%)
Flow leaking
through the
carbon filter
connection
Booker had
decreased the
crystal flow from
0.4to0.2slpmto
increase loading
time
Uknown
Unknown
Unknown
Unknown
Uknown, block
pressures were
activated in the
software
Grease loading
slightly too high?
High filter loading
Solution
Semtech DS software
updated to include the log
range and volume
correction
Tightened the carbon filter
Final c rystal s ample flow
set at 0.5 slpm
Crystal sensitivity
adjusted from 125 hz/pg
to 100 h z/pg (increasing
sensitivity by 25% )
Powered off hardware
and laptop and restarted,
problem was resolved
TriggeredthePPMD
manually through Host
software (unofficial
testing)
Recalibrated VFM
Aborted cycle, powered
down the PPMD and
restarted, problem was
fixed
Re cleaned and greased
crystals
Adjusted the valve on the
total flow pump to allow
more flow
6REPORT03.14936.12
E-6
-------
Date
1 1/5/2008
1 1/6/2008
1 1/7/2008
1 1/7/2008
1 1/7/2008
11/12/2008
11/12/2008
11/12/2008
11/12/2008
1 1/24/2008
1 1/25/2008
PEMS
Horiba 1
Horiba 1
Sensors 1
Horiba 1
Horiba 1
Sensors 1
Horiba 1
Sensors 1
Horiba 1
Horiba 1
AVL1
Description
Sample flow accuracy is
out of spec
Dilution flow is still
inaccurate
MPS Dilution Flow
Major audit could not
pass
External compressor
stopped working
External compressor
stopped working
PPMD results not
include d i n the D S results
file
Dilution compressor still
stoppi ng s onetimes
Slope on flow for daily
audits s onetimes fails to
0.97 or 1.03
Sample flow check is
sometimes not within
tolerance
Sample flow accuracy is
out of spec
MSS failed the external
DR audit repeatedly
Reason
Caused by
inaccurate dilution
flow measurement
(cause of this
uknown)
Unknown
Overheated
Overheated
Overheated
Descrepancies in
the time stamps
Overheating
Monthly
tolerances too
tight for daily
checks
Monthly
tolerances too
tight for daily
checks
Inaccurate
dilution flow
measurement
Unknown- all
internal checks
passed
Solution
Recalibrated VFM
Recalibrated VFM
None, cycle voided
None, cycle voided
None, cycle voided,
installed 2nd compressor
in parallel
Never resolved
Connected two
compressos inpa rallel,
so t he 2n d w ill run if the
first stops
Tolerances relaxed slightly
for daily checks to save
time
Tolerances relaxed slightly
for daily checks to save
time
Recalibrated VFM
Requirement of
performing external audit
on daily basis was
removed
7REPORT03.14936.12
E-7
-------
Date
12/1/2008
12/1/2008
12/1/2008
12/11/2008
12/15/2008
12/16/2008
12/18/2008
1/14/2009
1/14/2009
1/26/2009
1/28/2009
PEMS
Sensors 1
Horiba 1
Horiba 1
Horiba 1
Horiba 1
Horiba 1
Horiba 1
Horiba 1
Horiba 1
Sensors 2
Horiba 2
Description
EFM reading inaccurate
at the end of each test
Newest compressor that
was shipped still stopping
sometimes
Post processor file size
too big for Excel (>200
MB)
DCS signal flat lined
during testing
TRPM data filed was not
saved by Java software
External compressor
stopped working
External compressor
stopped working
Sample flow accuracy is
out of spec
Dilution flow inaccurate
SemtechDSlost
communication with
laptop
IP A switched from
measure to standby at the
start of the test
Reason
Solenoid not
switching
properly during
the 1 h our
autozero, EFM
zeroingwhile
open to flow
Overheating
Steady state data
processed at IHz
instead of lOHz
Uknown
Unknown
Overheated
Overheated
Caused by
inaccurate dilution
flow measurement
(cause of this
uknown)
Unknown
Unknown
Software glitch?
Solution
Solenoid fixed, auto zero
disabled?
C ontinued t o u se two
compressors in parallel, a
new compressor was
shipped from Horiba
C ontinued t o u se two
compressors in parallel
Problem did not occur
again when system was
restarted
None, cycle was void
None, cycle voided
None, cycle voided
Recalibrated VFM
Recalibrated VFM
Couldn't monitor data, but
data was still recorded on
compact flash card
Did not occur again
8REPORT03.14936.12
E-S
-------
Date
1/28/2009
1/28/2009
1/30/2009
2/2/2009
2/5/2009
2/18/2009
3/2/2009
3/3/2009
3/5/2009
3/6/2009
3/23/2009
3/24/2009
PEMS
Horiba 2
Horiba 2
AVL2
Sensors 2
Sensors 2
Horiba 2
Horiba 3
Horiba 3
Sensors 3
Horiba 3
Horiba
2200
Sensors
DS
Description
TRPM would s witch out
of filter s ample mode
once 30 seconds had
elapsed
Filter sampling was
beginning 10 s ec after
start of NTE instead of 5
Error : N o di lution air
available
SemtechDSlost
communication with
laptop, could not
reconnect
Semtech DS was losing
communication with
laptop
External compressor
stopped working
Software zero pressure
transducer function would
not work after repeated
attempts
External compressor
stopped working
SemtechDSlost
communication with
laptop
External compressor
stopped working
OBS-2200 software
locked up during test
C ouldn't c onnect to t he
Semtech DS repeatedly
Reason
Faulty software
logic
OBS-2200 is
delayed 5
seconds in its
response
On-board pump
was leaking
Unknown
Unknown
Overheated
Unknown
Overheated
Unknown
Overheated
Unknown
LAN circuit
board was likely
damaged
Solution
Software modified so that
filter will never switch ou t
of s ample mode while in
an NTE event
Horiba chose not to make
any changes
Pump was replaced with
new part shipped from
AVL
Powered down unit,
recovered data the next
day
Couldn't monitor data, but
data was still recorded on
compact flash card
None, cycle voided
Shut dow n equipment
attempted it again several
hours later and it worked
None, cycle voided
Couldn't monitor data, but
data was still recorded on
compact flash card
None, cycle voided
Software worked when
rebooted, but data for the
cycle was lost
Unit sent back to Sensors
for repair
9REPORT03.14936.12
E-9
-------
Date
3/25/2009
4/3/2009
4/13/2009
4/15/2009
4/20/2009
4/21/2009
4/21/2009
4/21/2009
4/24/2009
4/27/2009
4/28/2009
PEMS
Sensors 3
AVL1
AVL2
Sensors 2
Sensors 2
AVL2
AVL2
Sensors 2
Horiba 3
Horiba 3
Horiba 3
Description
PPMD wouldn't s ample
when N IE trigger
activates on the Semtech
DS
There is a voltage offset
between what the AVL
unit outputs and the
Semtech DS reads
Power inverter shut dow n
during r adi ated i mmunity
test
Communication with the
laptop dropped out
repe atedly du ring bu Ik
current injection
Communication with the
laptop dropped out
during r adi ated i mmunity
Power inverter shut dow n
during conducted
transient test
Power inverter shut
down, then start smoking
during conducted
transient
PPMD shut down
completely several times
during conducted
transient tests
Flow was erratic during
bulk c urrent inj ection test
Exhaust flow was reading
very high du ring bu Ik
current injection test
Lost c ommunication with
laptop during radiated
immunitytest
Reason
Unknown
Unknown
EMI/RFI Test
EMI/RFI Test
EMI/RFI Test
EMI/RFI Test
EMI/RFI Test
EMI/RFI Test
EMI/RFI Test
EMI/RFI Test
EMI/RFI Test
Solution
Turning the unit of f and
back on fixed the problem
Switched constants
Replaced bypass pump
Head sent back for repair
Replaced with bracket
from other unit
No corrective action
taken
Unit was shipped back to
AVL, replaced with other
unit, problem was not
observed again
No corrective action
taken
No corrective action
taken
No corrective action
taken
Inverter replaced with
backup, test was not
repeated
10REPORT03.14936.12
E-10
-------
Date
4/28/2009
5/15/2009
7/15/2009
7/15/2009
7/21/2009
7/22/2009
7/22/2009
7/24/2009
PEMS
Horiba 3
AVL3
Sensors 3
Sensors 3
AVL2
Horiba 3
Horiba 3
Sensors 2
Description
Exhaust flow was reading
very high during radiated
immunitytest
AVL PEMS blew a fuse
when trying to pow er up
after 1st pressure test
PPMD reported a
barometric pressure
increase when altitude
chamber was at vacuum
PPMD could not
maintain a by pa ss flow of
4 slpm
Soot concentration
reading erratic during
vibr ation ( all or ientation s)
L- bracket holding filter in
ME box broke during
vibration testing
Total Pi pressure
transducer would n ot
read correctly after
problem with L-bracket
occurred
PPMD could not
maintaintotal flow du e to
moisture traps opening
Reason
EMI/RFI Test
Uknown
Constants in the
software were
backwards for
the barometric
pressure
Bypass pump
was dying
Vibration
Vibration
Vibration Test
Vibration Test
Solution
No corrective action
taken
Replaced with AVL 2
No corrective action
taken
No corrective action
taken
Rebooted hardware to
reconnect
Did n ot c ontinue with
more extreme tests to
prevent damage
Pressure transducer was
replaced
No corrective action
taken
11REPORT03.14936.12
E-ll
------- |