Test Plan to Determine PEMS Measurement Allowance for the PM Emissions Regulated under the Manufacturer-Run Heavy-Duty Diesel Engine in-use Testing Program &EPA United States Environmental Protection Agency ------- Test Plan to Determine PEMS Measurement Allowance for the PM Emissions Regulated under the Manufacturer-Run Heavy-Duty Diesel Engine in-use Testing Program Assessment and Standards Division Office of Transportation and Air Quality U.S. Environmental Protection Agency and California Air Resources Board and Engine Manufactures Association SEPA United States Environmental Protection Agency EPA-420-B-10-901 August 2010 ------- EXECUTIVE SUMMARY This test plan sets forth the agreed upon processes and methodologies to be utilized to develop additive, brake-specific, data-driven measurement allowance for PM emissions measured by PEMS as required under the HDIUT regulatory program. As detailed in this test plan, there is a clear consensus on what components of measurement error are intended to be covered by the measurement allowance. Namely, the allowance is to be calculated in a manner that subtracts lab error from PEMS error. Specifically, utilizing Part 1065 compliant emissions measurement systems and procedures for both the lab and PEMS, the lab error associated with measuring heavy- duty engine emissions at stabilized steady-state test points within the NTE zone, will be subtracted from the PEMS error associated with measuring heavy-duty engine emissions utilizing PEMS over events under a broad range of environmental conditions. This subtraction will yield "PEMS minus laboratory" measurement allowance. The experimental methods and procedures specified in this test plan for determining, modeling, and comparing each of the various components of measurement error are designed to generate statistically robust data-driven measurement allowance for the PM emissions. Successful completion of this test plan is part of the resolution of a 2001 suit filed against EPA by EMA and a number of individual engine manufacturers. The suit challenged, among other things, certain supplemental emission requirements referred to as "not-to- exceed" (NTE) standards. On June 3, 2003, the parties finalized a settlement of their disputes pertaining to the NTE standards. The parties agreed upon a detailed outline for a future regulation that would require a manufacturer-run heavy-duty in-use NTE testing ("HDIUT") program for diesel-fueled engines and vehicles. One section of the outline stated: "The NTE Threshold will be the NTE standard, including the margins built into the existing regulations, plus additional margin to account for in-use measurement accuracy. This additional margin shall be determined by the measurement processes and methodologies to be developed and approved by EPA/CARB/EMA. This margin will be structured to encourage instrument manufacturers to develop more and more accurate instruments in the future." Given the foregoing, the work to be completed under this test plan is a vital component to the fulfillment of the settlement agreement, and it is vital to the successful implementation of a fully-enforceable HDIUT program. Because of this significance, it is critically important that the work detailed in this test plan be carried out in as thorough, careful and timely a manner as possible. ------- TABLE OF CONTENTS Executive Summary 2 Table of Contents 3 List of Figures 5 List of Tables 5 1 Introduction 6 2 Monte Carlo Error Model and Measurement Allowance 8 2.1 Objective 8 2.2 Background 9 2.3 Methods and Materials 15 2.4 Simulation Procedure 16 2.4.1 Construction of the Error Surface 16 2.5 Model Considerations 24 2.5.1 Convergence 24 2.6 Simulation Output 24 2.6.1 Sensitivity Variation Effect 24 2.6.2 Sensitivity Bias Effect 25 3 Engine Dynamometer Laboratory Tests 26 3.1 Preliminary Audits 26 3.1.1 Objective 26 3.1.2 Background 26 3.1.3 On-site meeting to establish 1065 compliance requirements 26 3.1.4 Methods and Materials 26 3.1.5 Data Analysis 27 3.1.6 PEMS Manufacturer PM PEMS Commissioning 27 3.2 Bias and Precision Errors under steady state engine operation 27 3.2.1 Objective 27 3.2.2 Background 28 3.2.3 Methods and Materials 28 3.2.4 Data Analysis 30 3.3 Precision Errors under transient engine operation (dynamic response) 31 3.3.1 Objective 31 3.3.2 Background 31 3.3.3 Methods and Materials 31 3.3.4 Data Analysis 35 3.4 ECM Torque and BSFC 37 3.4.1 Objective 37 3.4.2 Data Analysis 37 4 Environmental Chamber 37 4.1 Data Analysis for Environmental Tests 38 4.2 PM Generator Commissioning 39 4.3 Baseline 39 4.3.1 Objective 39 4.3.2 Background 39 ------- 4.3.3 Methods and Materials 40 4.3.4 Data Analysis 40 4.4 Electromagnetic Radiation 40 4.4.1 Objective 40 4.4.2 Methods and Materials 40 4.5 Atmospheric Pressure 41 4.5.1 Objective 41 4.5.2 Background 41 4.5.3 Methods and Materials 41 4.5.4 Data Analysis 43 4.6 Ambient Temperature and Humidity 44 4.6.1 Objective 44 4.6.2 Background 44 4.6.3 Methods and Materials 45 4.6.4 Data Analysis 46 4.7 Orientation and Vibration 46 4.7.1 Objective 46 5 SwRI CVS and CE-CERT Trailer Correlation 47 5.1 Method and Materials 47 6 Model Validation and Measurement Allowance Determination 48 6.1 Model validation 48 6.1.1 Objective 48 6.2 Measurement Allowance Determination 49 6.2.1 Objective 49 6.2.2 Background 49 6.2.3 Methods and Materials 49 6.2.4 Data Analysis 49 7 Time and Cost 50 7.1 Timeline 50 7.2 Cost 51 8 Abbreviations used in Brake Specific Equations 51 ------- LIST OF FIGURES Figure 1. Probability Density Functions For Sampling Error Surfaces 9 Figure 2. Brake-Specific PM Emissions Calculation For Method 1 11 Figure 3. Brake-Specific PM Emissions Calculation For Method 2 12 Figure 4. Brake-Specific PM Emissions Calculation For Method 3 13 Figure 5. Error Surface: PEMS Vs. LAB 17 Figure 6. Error Surface: (PEMS-LAB) Vs. LAB 18 Figure 7. Error Surface: Final Version 19 Figure 8. Overview of Monte Carlo Simulation 22 Figure 9. Example of ANTE Cycle 35 Figure 10. Pressure Histogram 41 Figure 11. Pressure-Time Environmental Test Cycle 43 Figure 12. temperature histogram 44 Figure 13. Time Series Chart of Ambient Temperature Test 46 LIST OF TABLES Table 1. Allowed Modifications 6 Table 2. Example Of Selection Of Measurement Allowance At 0.02 g/hp-hr NTE Threshold 15 Table 3. Error Surfaces for the BSPM Simulation 23 Table 4. Engine, Exhaust Configuration, And Steady-State Modes 28 Table 5. Example Of SS Error Surface 30 Table 6. NTE Transient Cycle 32 Table 7. Dynamic response inter-NTE Events 34 Table 8. Concentration and Dilution Ratio Schedule with PM Generator 38 Table 9. Atmospheric Pressure Test Sequence 42 Table 10. Ambient Temperature test sequence 45 Table 11. Projected PM-PEMS Timeline 50 Table 12. projected cost estimate 51 ------- 1 INTRODUCTION This test plan will establish a PEMS measurement allowances for PM, as regulated by the manufacturer-run on-highway heavy-duty diesel engine in-use test program. The measurement allowance will be established using various laboratory facilities and PEMS. The measurement allowance will be established in units of brake-specific emissions (g/hp-hr), and it will be added to the final NTE PM standard, after all the other additive and multiplicative allowances have been applied. This test plan will establish the PM measurement allowance. The PEMS used in this test plan must be standard in-production makes and models that are for sale as commercially available PEMS. In addition, PEMS and any support equipment must pass a "red-face" test with respect to being consistent with acceptable practices for in-use testing. For example, the equipment must meet all safety and transportation regulations for use on-board heavy-duty vehicles. Even though the PEMS can not be "prototypes" nor their software "beta" versions, the steering committee has already agreed that after delivery of PEMS to the contractor, there may be a few circumstances in which PEMS modifications might be allowed, but these modifications must meet certain deadlines, plus they are subject to approval by the steering committee. Also, any implementation of such approved modifications will not be allowed to delay the test plan, unless the steering committee specifically approves such a delay. Table 1 summarizes these allowable modifications and their respective deadlines: TABLE 1. ALLOWED MODIFICATIONS Allowed Modifications Steering committee approved hardware and software modifications that affect emissions results; including but not limited to fittings, components, calibrations, compensation algorithms, sampling rates, recording rates, etc. Steering committee approved hardware modifications for DOT approval or any other safety requirement approval Delivery of any environmental / weather enclosure to contractor Post-processing software to determine NTE results DOT approval and documentation Steering committee approved hardware or software that improves the contractor's efficiency to conduct testing and data reduction Before start of. . . Steady- State Testing Environmental Chamber Testing Environmental Chamber Testing Model Validation Model Validation Always Allowed The steering committee approved three different PEMS that includes the AVL Micro- Soot Sensor (MSS), the Horiba Transient Particulate Matter (TRPM), and the Sensors Proportional Particulate Matter Diluter (PPMD). However, because of the different measurement technologies employed by each of these systems, the three different PEMS hold slightly different status with respect to determining the PM measurement allowance. ------- Because inertial microbalances are already approved for PEMS applications in 40 CFR Part 1065, the Sensors PPMD will be one of the PEMS used to determine the measurement allowance. And because EPA's PM standard is based upon a gravimetric filter analysis, the Horiba TRPM will also be used to determine the measurement allowance. The lowest measurement allowance value between the two will be selected as the final measurement allowance for PM. If that value does not validate, then the lowest validated value will be chosen. If the lowest validated value chosen is within 0.0075 g/hp-hr from the lowest non-validated value, then the lowest validated value will be the measurement allowance. Otherwise, the MASC will spend up to a $100,000 to figure out a resolution to the problem by generating more data or changing the way the validation was performed. If that does not lead to a resolution, then Executive Management of EMA and EPA will have to settle the issue. Note that at the conclusion of successful testing of the Horiba system in this measurement allowance program, EPA intends to approve the Horiba system as an alternative for use, or EPA may elect to amend 40 CFR Parts 86 and/or 1065 to allow the use of the Horiba TRPM or other PEMS that operate upon similar measurement principles. Because the AVL system measures only the soot component of PM, the measurement allowance will not be determined using the AVL results, unless both the Sensors and Horiba systems fail to complete the measurement allowance program. Note that the steering committee may determine at the conclusion of the program that the AVL MSS is a viable alternative for demonstrating compliance. Under such a circumstance EPA may amend the Heavy-Duty In-Use regulation to allow for its use. This test plan describes a computer model, a series of experiments that are used to calibrate the model, and another series of experiments that are used to validate the calibrated model. The test plan first describes the computer model. The computer model statistically combines many sources of PEMS and lab error, which are nearly impossible to capture simultaneously in a single test. The model will use statistics to apply the errors in a way that simulates actual running of a PEMS in-use. The model will also consider only the portion of error that is attributable to PEMS, and it will subtract the error that is already tolerated in an emissions lab today. The model will also calculate and validate results according to 40 CFR Part 1065. The test plan then describes the series of experiments. These tests will characterize the many sources of PEMS and lab error so that the specific nature of the errors can be programmed into the computer model. The nature of the error has to do with the way PEMS and the lab react to certain conditions. For example, under varying environmental conditions such as temperature or vibration, a PEMS might exhibit signal drift, or it may record noise that is not a part of the true emissions. Next, the experimental results will be entered into the computer model, and the measurement allowances are calculated by the model. The model uses a "reference" PEMS data set, which will have many "reference NTE events." The model statistically ------- applies all the errors to the reference data set, calculates results, and saves the results. Then the model will be run with all errors set to zero to calculate the ideal results of the reference data set. Each difference between a reference NTE event's result with errors and its respective ideal result will be a brake-specific difference that is recorded for later use. Then the process repeats using the same reference data set, to which new, statistically selected errors are applied, and thus another unique set of differences is calculated. As the model continues to iterate and generate more and more results, patterns are expected to appear in the output data. These patterns should be the distributions of differences, based upon the error that was statistically and repeatedly applied to the reference data set. Many difference distributions will be determined: for each reference NTE event, for each of the two brake-specific calculation methods (three in case of the AVL system only), and for each PEMS. It has been agreed that the 95th percentile values of these distributions will be taken as reasonable "worst case" results for each reference NTE event. Details on how all these distributions will be reduced to determine the PM measurement allowance is given in the "Error Model" section of this test plan. Because the calculation based on Method 2 and Method 3 require gas-based fuel flow calculation based on the measurement of CC>2, CO, and NMHC, a decision was made to use the gaseous PEMS data for this purpose, without the need to perform gaseous measurement during the PM-PEMS program. Finally, the test plan describes how the computer model will be validated against real- world over-the-road in-use PEMS operation as well as additional lab testing. For the over-the-road testing, PEMS emissions measurements will be conducted, while at the same time a reference laboratory will be towed along to measure the same emissions. For the lab testing, an attempt will be made to simulate real-world engine operation to "replay" an over-the-road test in the lab. Data from these final experiments will be used to validate the model, which must be done in order to gain sufficient confidence that the model did not establish unreasonable measurement allowances. The following sections of this test plan are written as instructions to the contractor or contractors who will complete the test plan. 2 MONTE CARLO ERROR MODEL AND MEASUREMENT ALLOWANCE 2.1 Objective Use Monte Carlo (e.g. random sampling) techniques in an error model to simulate the combined effects of all the agreed-upon sources of PEMS error incremental to lab error. Create error "surfaces" for the Monte Carlo simulation to sample, based upon results from the experiments described in Sections 3 and 4. Exercise the model over a wide range of NTE events, based on a single, reference data set of at least 150 but no more than 200 unique NTE events. Determine the pollutant-specific brake-specific additive measurement allowance for PM. ------- 2.2 Background The error model uses Monte Carlo techniques to sample error values from "error surfaces" that are generated from the results of each of the experiments described in Section 3 on engine dynamometer laboratory tests and Section 4 on environmental chamber tests. The lab test error surfaces cover the domain of error versus the magnitude of the signal to which the error is to be applied (i.e. 1st to 99th percentile error vs. concentration, flow, torque, etc.). This is illustrated later in this section. The environmental test error surfaces for shock & vibration and electromagnetic & radio frequency interference (EMI/RFI) cover the same domain as the lab tests. The environmental test error surfaces for pressure and temperature are characteristically different because they cover the domain of environmental test cycle time versus the magnitude of the signal to which the error is to be applied (i.e. error at a selected time vs. concentration). Details on how each surface is generated are given in each of the respective sections. These surfaces will already be adjusted to represent PEMS error incremental to lab error; therefore, these surfaces are sampled directly by the model. The error model will use two different probability density functions (PDFs) as shown in Figure 1 to sample the error surfaces, depending upon which experiment the surface represents. To sample error surfaces that are generated from all the laboratory test results (Section 3), and the environmental test results for shock & vibration (Section 4), the model will use a truncated normal PDF because these tests are designed to evenly cover the full, but finite, range of engine operation and ambient conditions. To sample error surfaces that are generated from the pressure and temperature environmental test results (Section 4), the model will use a uniform PDF because these tests are already designed to cover the typical range and frequency of the respective conditions. 1 Probability Density Functions for Sampling Error Surfaces Once Per NTE Event Lab Tests, Normal, SD=0.60795, truncate @ -1 & 1 — Environmental Tests, Uniform Note: A non-truncated normal distribution with SD=0. 60795 has P values of 0.01 an ic=+1, respective d 0.99 at ic=-1 anc y. ^ z V ^^ ^^ ^^ ^>^_ ^^ ^^ f^*^ ^^^ ^^^^ 432 Relative Probability - -0.25 - -0.50 - -0.75 0 FIGURE 1. PROBABILITY DENSITY FUNCTIONS FOR SAMPLING ERROR SURFACES ------- The random values that are obtained from both distributions are labeled ic in Figure 1 and range from -1 to 1. Note that for the pressure and temperature environmental tests, a uniform PDF will be used to sample test time, from which the nearest (in time) calculated errors are used. The errors from the other tests will be aligned with the truncated normal PDF such that each of the 50th percentile values at each of the tested signal magnitudes is centered at the median of the PDF (ic = 0), and the 1st and 99th percentile error values at each of the tested signal magnitudes will be aligned with the extreme negative (ic = -1) and positive (ic = +1) edges of the PDF, respectively. Each error surface will be sampled along its ic axis (y-axis) once per reference NTE event trial, and it will be sampled along its parameter value axis (x-axis, e.g., concentration (only for AVL MSS), flow, torque, etc...) once per second, within a given reference NTE event trial. An error will be determined for a given second and parameter along the error axis (z-axis) at the intersection of an ic value and a parameter value. To ensure that the magnitudes of the error surfaces are appropriate, each data point used to generate the surfaces will be a mean or a weighted mean of 30 seconds of sampling. Interpolation will be performed by first linearly interpolating error values at each tested magnitude along the selected line perpendicular to the ic axis. Then from that line of errors, individual error values will be linearly interpolated at each second-by-second signal magnitude of the given NTE event in the reference data set. The reference data set to which all errors will be applied will be a large data set of engine operation over a wide range of NTE events. This reference data set will be initially generated from collections of real-world PEMS data sets. The reference data set should contain at least 150 but no more than 200 unique NTE events. Parameters in the reference data set may be scaled in order to exercise the model through a more appropriate range of parameters (i.e. concentrations, flows, ambient conditions, etc.). If the parameters are scaled, care should be taken to maintain the dynamic characteristics of the reference data set. After the errors are applied, NTE brake-specific PM emissions results are calculated, using each of the three agreed-upon NTE calculation methods. The three different brake- specific emission calculation methods for PM referred to in this test plan are i) Torque- Speed method, ii) BSFC method, and iii) ECM-Fuel Specific method, and these are illustrated in Figure 2, Figure 3, and Figure 4, respectively. 10 ------- For all PM PEMS: m PM is a flow weighted particulate matter exhaust concentration in g/mol m m ePM(glkW-hr} = - i=\ . (mol\ n J V I 2*3.14159* A/' 60*1000*3600 Where for AVL: m p,, is computed numerically as follows, m 'PM mol I • «.. I * A v * A? FIGURE 2. BRAKE-SPECIFIC PM EMISSIONS CALCULATION FOR METHOD 1 11 ------- For all PM PEMS: m PM is a flow weighted particulate matter exhaust concentration in g/mol em(g/kW-hr)=- 4c *V r~ 2. Wfuel vmo/j tr • n 'mo/ Where for AVL: is computed numerically as follows, PM mol rnol S • «. ± A. * A? , 4 *icr2 * ,(*] >6UJ ,^peeti( (?pm) *7^(Ar-m) *2*3. 14159 60*1000*3600 FIGURE 3. BRAKE-SPECIFIC PM EMISSIONS CALCULATION FOR METHOD 2 12 ------- For AVL Only: ePM(g/kW-hr) = ( a \ W \mol) Mc ' !=1 Where: 1/1; N F __i *^f) ,77/C!(^)*10-+(^(%) + ^2 Speedf (rpm\*Ti (7V-#2)*2*3. 14159* A? - 60*1000*3600 Wfuel * ^ f a- ^ m g - mPM\ , \rnol ) n; c Z-. ¥ fuel *\^ C (m,PM;(^/mo/))*ml * A/ V| s ^ / ^ = Af JcTT/C, (/?p»i) * 1 0"6 + (jcCO;. (%) + xCO2i (%))*! O"2 • f^l m/K \^y ^ JcTT/C, (/?p»i) * 1 0~6 + (xCOt (%) + xCO2j (%))*! O"2 :A? FIGURE 4. BRAKE-SPECIFIC PM EMISSIONS CALCULATION FOR METHOD 3 Next, the NTE events are calculated by each of the three calculation methods, but with no error sampled or applied to the reference data set. These results are considered the "ideal" results of the reference NTE events. These ideal results are subtracted from each respective NTE event result 'with errors', and the difference is recorded. Then a new set of errors are sampled and applied to the reference NTE event, and the NTE results 'with errors' are calculated again. The ideal results are again subtracted, and the difference is recorded. This is repeated thousands of times so that the model converges upon distributions of brake-specific differences for each of the original NTE events in the reference data set. Then the 95th percentile difference value is determined for each NTE event distribution of brake-specific differences for PM for each calculation method. At this point there is one distribution of 95th percentile differences for PM, where all the NTE events are pooled by the PM emissions for each of the three different calculation methods. Each of the 95th percentile distributions represents a range of possible measurement allowance values. 13 ------- From each of these three distributions of possible measurement allowance values, one measurement allowance per distribution must be determined. First the correlation between 95th percentile differences versus the ideal PM emission is tested. For each calculation method, if a least squares linear regression of 95th percentile differences versus ideal PM emissions has an r2 (squared correlation coefficient) > 0.85 and an SEE (standard error of the estimate or root-mean-squared-error) < 5 % of the median ideal PM emission, then that linear regression equation will be used to determine the measurement allowance for that calculation method at the following NTE threshold: PM = 0.02 g/hp-hr and 0.03 g/hp-hr In cases where extrapolation is required to determine the measurement allowance at the NTE threshold, the measurement allowance will be determined using the linear regression, but evaluated at the ideal PM emission that is closest to the NTE threshold, not extrapolated to the NTE threshold itself. If the linear regression does not pass the aforementioned r2 and SEE criteria, then the median value of the 95th percentile differences is used as the single measurement allowance for that calculation method. Next, the calculation method is selected. The above procedure will provide three measurement allowances, where applicable, one for each of the three different calculation methods. To make them comparable, the three measurement allowance values will be normalized by the PM threshold and expressed as a percent. Also, if any measurement allowance is determined to have a value less than zero, then that measurement allowance will be set equal to zero. The calculation method with the minimum normalized PM value will be chosen and the corresponding normalized PM value will be selected as the best measurement allowance for PM, assuming it validates. If it does not validate, then the minimum value that validates will be chosen as long as it is within 0.0075 g/hp-hr from the minimum value that did not validate. If the difference between the minimum value that validates and the minimum value that did not validate is greater than 0.0075 g/hp-hr, additional investigation with up to a $100,000 will be spent in order to understand why the minimum value chosen did not validate. If the problem is not resolved after spending the $100,000, then the matter will be referred to executive management of EPA and EMA to decide on the PM measurement allowance. Error! Reference source not found, below illustrates the selection of the calculation method. The example is based on a hypothetical set of normalized PM measurements for the three calculation methods. The minimum of these normalized allowances is used to select the best method (highlighted in blue). In this hypothetical case, the BSFC method would be selected. 14 ------- TABLE 2. EXAMPLE OF SELECTION OF MEASUREMENT ALLOWANCE AT 0.02 G/HP-HR NTE THRESHOLD Calc. Method ==> BSPM Selected Method==> Allowance at Respective NTE Threshold (%) Torque-Speed 38% BSFC 18% ECM fuel specific N/A BSFC Method Therefore, 18% would be selected as the best measurement allowance for PM, assuming it validates. Otherwise, the 38 % will be chosen if it validates. Thus, the additive brake- specific measurement allowance would be: PM = 18 % * 0.02 g/hp-hr = 0.0036 g/hp-hr, if it validates, and if not, then: PM = 38 % *0.02 g/hp-hr = 0.0076 g/hp-hr, if it validates, and if not, then: spend up to a $100,000 to figure out why it did not validate in the first place, and then apply the above strategy again, assuming the value now validates. If not, then EPA and EMA executive management will decide on the PM measurement allowance value. This PM value would be the value added to the actual brake-specific NTE threshold for a given engine, based on actual family emissions limit, mileage, model year, etc. 2.3 Methods and Materials Exercise the model using three different calculation methods: a) Torque-Speed method, b) BSFC method, and c) ECM-Fuel Specific method (only for AVL MSS). Determine which calculation method is the most accurate, and use it to estimate the measurement allowance. Each calculation method is described in Figure 2, 3, and 4. Prepare an Excel spreadsheet model for use with the Crystal Ball Monte Carlo software for error analysis of brake specific emissions, BSE, as outlined in section 2.4. Changes to the model specifications may be requested as agreed upon by the Steering Committee. Prepare the spreadsheet in a modular structure following the specified model outline, and make provisions for the identified calculation modules. Additionally, clearly identify and easily locate input cells to the model to facilitate any revisions that may become necessary for users who want to exercise the model with other Monte Carlo add-ins such as @Risk or the newest versions of Crystal Ball. Test the spreadsheet with controlled test cases of simplified input distributions with the Crystal Ball add-in to confirm correct model implementation in accordance with this test plan. Run at least one typical analysis as an additional confirmation. Deliver the electronic spreadsheet and a brief report describing the model, presenting the test cases, and describing pertinent information including the Crystal Ball version number, the Excel version number, the operating system and the computer. Use standard 15 ------- spreadsheet calculations so that no serious difficulties will be anticipated regarding application in other spreadsheet versions. Use Crystal Ball Version 7 or higher, and confirm test cases using Excel 2003. Control revisions of the spreadsheet model using descriptive file names. Extensive revisions or testing with other software versions beyond that initially proposed may be re- proposed by the Steering Committee if and when a need for such additional work is identified. 2.4 Simulation Procedure For each of the measurement errors in Section 3, create an error surface and sample it according to the aforementioned PDFs. Each error surface represents an additive error— or a subtractive error if the sign is negative—relative to the reference value to which it is applied. Figure 5, Figure 6, and Figure 7 serve as a hypothetical PM example of how these error surfaces should be created for every error. The plots shown correspond to PM emissions concentration data representing 1 PEMS, two engines, and three exhaust configurations each, with all 6 sets of PEMS data pooled together. Note that separate error surfaces will be constructed for each of the three PEMS units (AVL, Horiba and Sensors). The example applies to the error module for steady-state (SS) bias and precision PM concentration errors (Section 3.2). These figures will be referenced by each "Data Analysis" section for the various errors discussed in this test plan. Errors from Section 3 (Engine Dynamometer Laboratory tests) and Section 4 (Environmental Chamber Tests) are combined by adding all of the sampled errors once per NTE event trial. For example, in order to assess the errors in PM concentration for each NTE event, several modules will be created such that: PM_with errors = PM _ideal + A(ug/mole)i + A(ug/mole)2 + A(ug/mole)3 + ... where, A(ug/mole)i = PM concentration errors due to steady state bias and precision errors, A(ug/mole)2 = PM concentration errors due to ambient temperature, A(ug/mole)3 = PM concentration errors due to ambient pressure, etc.... 2.4.1 Construction of the Error Surface 2.4.1.1 PEMS vs. Lab Acquire raw data with the PEMS at various average concentration levels as per Section 3.2. Plot the "PEMS" signals versus the corresponding "lab" signals that were measured using lab equipment. This plot pools all bias and precision errors for one PEMS and for all data from all engines for all steady-state modes. Shown in Figure 5 are the 5th, 50th 16 ------- and 95th percentiles at the mean PM concentration level from the lab (note that the distribution of data at each level is not necessarily Gaussian). If the 50th percentile is different than the line of perfect agreement (diagonal), the data suggests that there is a bias error between PEMS and Lab. In essence this graph shows the statistical distribution measured by the PEMS at each average concentration level sampled. The example shows only 6 discrete PM concentration levels (ranging from 10-60 ug/mole). However, the actual number of discrete levels will be determined by the total number of operating conditions actually run for all the tests of all the engines. For example, the SS PM testing will select 6 modes representing typical operating conditions. Thus, the actual plot for SS PM will likely have 36 discrete concentration levels (6 modes x 1 PEMS x 2 engines x 3 exhaust configurations). Error Surface forSS PM Concentration 10 20 30 40 50 PM |jg/mole (lab, mean) 60 70 80 diagonal 50th percentile (median) •95th percentile -5th percentile FIGURES. ERROR SURFACE: PEMS VS. LAB 17 ------- Error Surface forSS PM Concentration PM |jg/mole (lab,mean) -95th percentile 50th percentile (median) •5th percentile FIGURE 6. ERROR SURFACE: (PEMS-LAB) VS. LAB 18 ------- 1.5 0.5 tn -0.5 -1 -1.5 Error Surface forSS PM Concentration Error Surface: z-axis = ASS_PM_ug/mole 10.1 A-- 10 • -• -4.1 6.8 --A-- 20 -3.1 8.8 •-A-- 1.8 6. -A- -2.2 7.2 -A-- 0.2 PM ug/mole (lab,mean) -A--- 99th percentile 50th percentile (median) -•-- - 1st percentile FIGURE 7. ERROR SURFACE: FINAL VERSION 19 ------- 2.4.1.2 (PEMS - Lab) vs. Lab The plot in Figure 6 basically shows the "additive error band" measured during testing. The plot is created by first subtracting the "lab" PM value from the corresponding individual PEMS PM measurement for each test run. This difference is defined as the 'delta' error. Next, the "PEMS - Laboratory" delta errors are pooled at each average lab PM value to obtain the 95th, 50th, and 5th percentile values, respectively, displayed in Figure 5. Notice that if lab error exceeds PEMS error at a given percentile, crossover of values can occur. This is acceptable because the crossover effectively reduces PEMS error whenever lab error exceeds PEMS error. In order to obtain estimates of the 1st and 99th percentiles for the delta errors for a given "lab" PM value, each side of the corresponding error distribution will be assumed to independently fit a normal distribution. Because of the asymmetry of the data, this methodology will yield two halves of a normal distribution. The median of each normal distribution will be the median based on the delta errors given in Figure 6. The 95th percentile delta error will form the upper boundary of one half of the normal distribution, and the 5th percentile delta error will form the lower boundary of the other half of the normal distribution. When each side of the data distribution is fitted to a normal distribution using the above boundary conditions, one can then expand each half of the distribution from the error surface to obtain the 1st and 99th percentiles of the data for the given "lab" PM value. 2.4.1.3 Error Surface This step normalizes the data in Figure 7 using what is called a "variability index (ic)", which represents the random sampling by the Monte Carlo technique, in order to select a given error level. This variability index is allowed to vary from -1 to +1. The likelihood of ic being any value between -1 through +1 is specified by the PDF assigned to ic. In the given example, ic is assumed to vary according to a normal distribution during Monte Carlo calculations. This is because it is believed that the distribution of errors due to steady-state bias and precision will be centered about the 50th percentile of the full range of conditions measured according to Section 3.2. The pressure and temperature environmental error modules use uniform probability density functions for their respective variability index. Each set of data for each lab set-point mean (i.e., lab reference value) in Figure 6 is normalized by aligning the 1st percentile error from the fitted normal distributions with ic = -1, the 50th percentile error with ic = 0, and the 99th percentile error from the fitted normal distribution with ic= +1. Error surfaces such as the one presented in Figure 7 are the input modules that the Monte Carlo simulation program will use during calculations of brake-specific PM emissions. For example, for a given NTE calculation a random ic value is chosen once per NTE event trial. Let us assume that the first random sample produced an ic= 0.5. Let us also 20 ------- assume that during this NTE event trial, the reference PM concentration is 10 ug/mole. In this case, A(ug/mole)i = (3 + 10.1) / 2 = 6.55 ug/mole. Also, from Figure 7, for ic = 0.5, the reference PM = 10 ug/mole. For that step in the calculation, the Monte Carlo approach will add this "delta" to the reference concentration value of 10 ug/mole (10 ug/mole + 6.55 ug/mole = 16.55 ug/mole) to represent errors in steady-state bias and precision for ic= 0.5, and reference NTE PM =10 ug/mole. If during the same NTE event in the reference data set, a reference concentration of 35 ug/mole is read, then, A(ug/mole)i = ((6 + 8.8) / 2 + (2 + 6.2) / 2) / 2 = 5.75 ug/mole (from Figure 7) Note that first the error along the ic line perpendicular to the ic axis (in this case the line along 0.5) is linearly interpolated at each discrete concentration level. Then those interpolated values are themselves linearly interpolated to determine the error corresponding to each reference concentration in the NTE event. Note that the random selection is once per reference NTE event trial, but the error along that ic line is applied to every second-by-second value within the given reference NTE event, except for PM concentration in the case of Horiba and Sensors, where no second-by-second information are available, but different PM concentration levels may be available for a specific NTE event. Now let us assume that the error in PM concentration is composed of only 3 deltas: A(ug/mole)i, A(ug/mole)2 , and A(ug/mole)3 . And let us assume that for a given reference NTE event trial we have the following values: • Reference PM at one second= 30 ug/mole • A(ug/mole)i = 6 ug/mole • A(ug/mole)2 = -2 ug/mole • A(ug/mole)3 = -3 ug/mole. When the model calculates brake-specific emissions by each of the three calculation methods, it will use the following PM value, which has all of its error applied: PM = 30 + 6 -2 - 3 = 31 ug/mole. The application of error at the first selected ic continues during the entire NTE event without having to randomly sample again. In other words, ic will not change during that random trial. For all of the variables except for mPM, the errors may continue to change during an NTE event on a second-by-second basis if their error surface happens to be a function of level. For the second randomly selected ic this entire process of determining 21 ------- the Aug/mole errors is repeated. The simulation will continue to randomly selected ic values for thousands of trials until convergence is met. For the Horiba and Sensors generated reference NTE events, there is only one flow- weighted PM value for the entire NTE event. During the simulation for these types of reference NTEs, the single PM value will be used in the interpolation of the corresponding PM error surfaces (i.e., steady-state PM, transient PM) at all seconds of the reference NTE event. Since the PM value will not vary from second-to-second, the only interpolation will occur according to the ic value at each of the simulation trials. The same second-by-second sampling and interpolation approach would be used for other deltas such as ambient temp, ambient pressure, shock and vibration, BSFC interpolation, torque, exhaust flow rate, etc. An overview of the Monte Carlo simulation for PM is detailed in Figure 8. Reference NTE Monte-Carlo Simulation PM CO% NMHC (ppm) Exhfiow (scfm) Torque (N-m) Speed (rprn) Fuel Rate (L/sec) CO.% 95th percentile BSPM differences 4 Torque + ATorque c_Exhflow *c_Torque 1c_Speed Fuel Rate + AFuel Rate (1) BSPM = f (PM, Exhflow, Torque, Speed) (2) BSPM = f (PM, Exhflow, BSFCECM) (3) BSPM = f (PM, CO2, CO, THC, Torque, Fuel RateECM, Speed) * Differences = BSPM "with errors" - "Ideal" BSPM FIGURE 8. OVERVIEW OF MONTE CARLO SIMULATION Table 3 lists the error surfaces that will be created for use in simulating the BSPM error differences. 22 ------- TABLE 3. ERROR SURFACES FOR THE BSPM SIMULATION Calculation Component Delta PM Delta PM Delta PM Delta PM Delta PM Delta PM Delta CO Delta CO Delta CO Delta CO Delta CO2 Delta CO2 Delta CO2 Delta NMHC Delta NMHC Delta NMHC Delta NMHC Delta NMHC Delta Exhaust Flow Delta Exhaust Flow Delta Exhaust Flow Delta Exhaust Flow Delta Exhaust Flow Delta Exhaust Flow Delta Exhaust Flow Delta Torque Delta Torque Delta Torque Delta Torque Delta Torque Delta Torque Delta Speed Delta Fuel Rate Test Source Engine Dyno Engine Dyno Environ Environ Environ Environ Engine Dyno Environ Environ Engine Dyno Engine Dyno Engine Dyno Environ Engine Dyno Engine Dyno Environ Environ Environ Engine Dyno Engine Dyno Engine Dyno Engine Dyno Environ Environ Environ Engine Dyno Engine Dyno Engine Dyno Engine Dyno Engine Dyno Engine Manuf Engine Dyno Engine Dyno Error Surface Delta PM SS Delta PM Transient Delta PM Ambient Temperature Delta PM EMI/RFI Delta PM Atmospheric Pressure Delta PM Vibration Delta CO SS Delta CO Atmospheric Pressure Delta CO Ambient Temperature Delta CO Time Alignment Delta CO2 SS Delta CO2 Transient Delta CO2 Ambient Temperature Delta NMHC SS Delta NMHC Transient Delta NMHC Atmospheric Pressure Delta NMHC Ambient Temperature Delta Ambient NMHC Delta Exhaust Flow SS Delta Exhaust Flow Transient Delta Exhaust Flow Pulsation Delta Exhaust Flow Swirl Delta Exhaust EMI/RFI Delta Exhaust Temperature Delta Exhaust Pressure Delta Dynamic Torque Delta Torque DOE Testing Delta Torque Warm-up Delta Torque Humidity /Fuel Delta Torque Interpolation Delta Torque Engine Manuf Delta Dynamic Speed Delta Dynamic Fuel Rate 23 ------- 2.5 Model Considerations 2.5.1 Convergence The main goal of the convergence criteria is to define how many simulation trials at a given reference NTE event are required to estimate the 95th percentile BSPM emission differences with a given precision. The convergence method to be used is based on a nonparametric statistical technique3 which defines a 90% confidence interval for the 95th percentile of the BSPM emissions differences for an individual reference NTE simulation. If the width of the 90% confidence interval is less than 1% of the BSPM emissions threshold, then convergence is met. The following steps define the convergence method: 1. Run the Monte Carlo simulation for TV trials for a single reference NTE event. 2. Order the BSPM emissions differences from smallest to largest. 3. Identify the trial number at the lower end of the 90% confidence interval niower = 0.95 * N -1.645-VO.95 * 0.05 * N 4. Identify the trial number at the upper end of the 90% confidence interval = 0.95 * N + 1.645V0.95 * 0.05 * N 5. Compute (BSPM difference value at nupper)- (BSPM difference value at niower). 6. If the result in (5) < 1% of the BSPM emissions NTE threshold (0.02 g/hp-hr) then convergence is met. 2.6 Simulation Output It is important to understand and identify what error surfaces have the most influence (i.e., sensitivity) on the BSPM emissions 'with errors' and, thus, the resulting BS emissions differences. Contributions to sensitivity can be attributable to changes in variance and/or bias. 2.6.1 Sensitivity Variation Effect During the Monte Carlo simulation for each reference NTE event, sensitivity charts produced by Crystal Ball will be generated and stored in output REPORT files. Crystal Ball calculates sensitivity by computing the rank correlation coefficient between every assumption (error surface) and forecast value (delta BS emissions) while the simulation is running. Positive rank correlations indicate that an increase in the assumption is associated with an increase in the forecast. The larger the absolute value of the rank correlation the stronger the relationship. Sensitivity charts in Crystal Ball provide a means to determine how the variances of the error surfaces affect the variance in the forecast values. Hence, the sensitivity charts developed during a simulation are displayed as "Contribution to Variance" charts which 24 ------- are calculated by squaring the rank correlation coefficients for all assumptions used in a particular forecast and then normalizing them to 100%. The assumption (error surface) with the highest contribution to variance (in absolute value of the percent) is listed first in the sensitivity chart. Simulation results from all reference NTE events will produce sensitivity values for the 95th percentile delta PM emissions by all three calculation methods. 2.6.2 Sensitivity Bias Effect Another type of sensitivity to be examined in this study is concerned with the effects of potential "bias" in error surfaces and their effects on the forecast values. In order to study these effects a new error surface assumption will be added to the simulation model for each of the original error surfaces. This assumption will be sampled as a discrete binary distribution (i.e., on or off) during the simulation. For each trial of the simulation, the original error surfaces and 'on/off error surfaces will be sampled according to their defined sample distribution. If the 'on/off error surface produces an 'off condition, the delta emissions from that particular error surface will not be added to the BSPM emissions computations for the BSPM emissions 'with errors'. Similarly, if the 'on/off error surface produces an 'on' condition, the delta emissions from that particular error surface will be added to the BSPM emissions calculations. During every trial of the simulation, the exclusions due to the 'off conditions will result in various combinations of the error surface delta emissions being added to the BSPM emissions 'with errors' computations. Over the course of a simulation with thousands of trials, the sensitivity of a particular error either 'on' or 'off will be assessed by examining the change in the forecast delta emission. Therefore, in a single Monte Carlo simulation of a reference NTE event sensitivities due to variance and/or bias will be explored. 25 ------- 3 ENGINE DYNAMOMETER LABORATORY TESTS Utilize engine dynamometer laboratory testing to establish the difference between PM PEMS and PM based on laboratory measurement in accordance with Part 1065. Also establish how well ECM parameters can be used to estimate torque and BSFC. First, however, audit all the PEMS and lab equipment to ensure that they are operating properly, according to 40 CFR Part 1065, Subpart D. Next, conduct steady-state engine dynamometer tests to establish PEMS steady-state bias and precision relative to the lab. Then, conduct transient engine dynamometer testing to determine PEMS transient precision by repeating transient NTE events. Finally, compare ECM derived torque and BSFC to laboratory measured torque and BSFC. 3.1 Preliminary Audits 3.1.1 Objective Conduct 40 CFR Part 1065, Subpart D audits of all engine dynamometer laboratory systems and all PEMS. 3.1.2 Background Because the overall purpose of this entire test plan is to establish measurement allowance that account for the incremental difference in the performance of PEMS versus engine dynamometer laboratory systems, the first task is to audit all of the measurement systems to ensure that the specific systems used for testing meet EPA's minimum performance requirements. The audits also help to minimize bias errors between PEMS and lab systems measurements. However, in case a specific PM-PEMS does not meet the specifics of Part 1065 requirement, the MASC will decide on how to move forward by perhaps allowing some flexibility in passing Part 1065 audit, in situations where it might be needed, especially if the performance of a system is within the expectation of the manufacturer. 3.1.3 On-site meeting to establish 1065 compliance requirements In order to clarify what are all the requirements expected from the lab-grade instrumentation and PEMS equipment, with respect to 1065 compliance, a meeting will be held between the test plan steering committee and the contractor at the contractor site to provide the contractor with guidance regarding which specific sections of Part 1065 Subpart D are required and which are optional. In case Part 1065 requirement is demonstrated to be too stringent or impractical, the contractor may seek approval from the MASC to lessen the stringency of Part 1065 in relation to the PEMS. 3.1.4 Methods and Materials Use the methods and materials described in 40 CFR Part 1065, Subpart D to conduct audits of all lab and PEMS measurement systems. Even if lab systems and PEMS pass initial Subpart D audits, allow lab operators and PEMS manufacturers to make on-site 26 ------- adjustments to improve the performance of their systems prior to engine testing. Allow adjustments to be based on recalibrations with reference signals that are allowed in 40 CFR Part 1065. The steering committee may direct the contractor to calibrate or adjust the laboratory sampling system based on audit results. The steering committee may also suggest that a PEMS manufacturer calibrate or adjust one or more PEMS based on lab audits. 3.1.5 Data Analysis Use the data analyses described in CFR Part 1065 Subparts D, J and G. For all subsequent testing, use only those measurement systems that pass the minimum performance criteria in Subpart D, unless a deficiency is deemed acceptable in writing by all parties including PEMS manufacturers. Provide a list and brief description of all the audits conducted for each PEMS manufacturer type. EPA would likely use this list as a template for the data requirements in the PM portion of the HDIU testing program. 3.1.6 PEMS Manufacturer PM PEMS Commissioning Notify PEMS manufacturers when the 1065 audits are complete and the first set of PM PEMS are completely installed in the engine dynamometer test cell—in preparation for emissions testing. Schedule dates and times that are prior to the start of emissions testing for each PEMS manufacturer to conduct a final commissioning of all their PEMS that are on site, including those PEMS that are not installed in the test cell. PEMS manufacturers may inspect their PEMS and make any final adjustments to their respective PEMS in order for the PEMS to meet their specifications. Allow PEMS manufacturers to inspect the installation of their PEMS in the test cell. If PEMS manufacturers take exception to any portion of the installation or on-site configuration, attempt to resolve any such installation issues. If such issues are not easily resolvable, notify the steering committee, who will determine a course of action. Once PEMS manufacturers have completed their commissioning, notify the steering committee. From this point any further modifications to the PEMS may only be made according to Table 1 of this test plan. 3.2 Bias and Precision Errors under steady state engine operation 3.2.1 Objective Evaluate the bias and precision using one engine and one exhaust configuration, shown in Table 4, and 10 repeats of steady-state modes, and three sets of PEMS units, each set including the MSS, TRPM, and PPMD. Thus, the total number of NTE steady-state points required to conduct the steady-state experiments is 30. This constitutes six steady- state modes of engine operation (6), 10 repeats (10), one exhaust configuration, one engine (1), and three different PEMS units (3), 6x10x1x1x3= 180. Determine the AsswPM (-Jjl surface plots for the error model based upon all data pooled. Note that each brand of PEMS will have its own AsswPM error surface generated for use 27 ------- in both calculation methods 1 and 2. For calculation method 3, the AVL brand PM PEMS will have a unique AssmPM calculated according to Figure 4 of this test plan. Recommend six steady-state points based on the PM measurement, using the AVL MSS, of 80 SS points of the Cummins cycle that is typically used to generate ECM torque and BSFC errors versus laboratory. The MASC will accept the six steady-state points or choose alternative points for each exhaust configuration. The objective for the MASC will be to select steady-state points within a given exhaust configuration that provides a nominal spread of concentrations within that configuration's target brake-specific levels. Note that to achieve the brake-specific targets under steady-state conditions, the bypass might have to be opened further, relative to the transient NTE bypass settings. TABLE 4. ENGINE, EXHAUST CONFIGURATION, AND STEADY-STATE MODES 07 Engine 1 No. of Steady-State Modes for Bypass Setting 1 (BSPM and PM Concentration, representative of PM threshold of 0.025 g/hp-hr under NTE Transient Operation) SSI, SS2, SS3, SS4, SS5, SS6 PM-PEMS Units Three Sets of (MSS, TRPM, and PPMD) Number of Repeats 10 per Mode per PM- PEMS Set 3.2.2 Background Testing will be conducted to capture bias and precision errors in PEMS' emissions instruments versus the laboratory filter-based method. The tests will be steady-state only. Note: Section 3.3 (next section) will evaluate precision errors (not bias) due to the dynamic response of the PEMS instrumentation. The precision error captured during steady state testing (section 3.2) will have to be subtracted from the overall precision error captured in section 3.3 in order not to double-count the steady state precision errors of PEMS instrumentation. This process is detailed in Section 3.3. 3.2.3 Methods and Materials Use the following systems: a) One model year 2007 heavy duty diesel engines, equipped with a DPF in the exhaust (Mack MP9) b) Nine PM PEMS (3 Sensors PPMD, 3 AVL MSS, 3 Horiba TRPM) c) One PEMS exhaust flow-meter from Sensors, Inc., and one and from Horiba, applicable to the engine to be tested 28 ------- d) DPF with Bypass Setting 1 for SS testing, representing a threshold level of about 0.025 g/hp-hr under NTE transient testing Use the following overall guidelines: e) Measure PM via the CVS, Part 1065 Lab Method (most recent publication) f) Measure engine inlet airflow through use of LFE or equivalent g) Use a series of six steady-state modes, and set each mode time to collect a CVS filter mass of at least 75 microgram per mode, simultaneously with other PM-PEMS h) Regenerate DPF system prior to each series of steady-state tests i) Capture ECM broadcast channels and other common diagnostic channels, as recommended by engine manufacturer(s), to ensure proper engine operation j) Do not measure gaseous species by the PEMS k) Stabilization time =180 seconds, with a different running time per mode to achieve a 75 microgram or higher of PM on the CVS filter 1) Always power off PEMS equipment at end of each day, according to PEMS manufacturer instructions. Re-start start-up process every day according to PEMS manufacturer instructions and Part 1065, Subpart J. m) Whenever PEMS are exchanged, swap the order of the Horiba and Sensors flowmeters, if the steup allows for it. 6 point steady-state repeat-testing, evaluate bias and precision errors: a) The MASC will select 6 SS operating conditions for repeat testing from a matrix of 80 SS points containing information on PM emissions using the AVL MSS b) Randomize the order of the six modes c) Repeat each six steady-state cycle two or three times, prior to DPF regeneration d) Each test will use three PEMS (Sensors, AVL, and Horiba) at a time, to measure PM emissions concentration and exhaust flow rate. e) Expected test duration is 5 days per PEMS set, with a total of 15 days for all three sets. Bypass Setting: a) Run NTE transient cycle using the CVS filter-based method b) Set bypass to produce CVS filter-based average brake-specific of about 0.025 g/hp-hr c) Determine the average PM mass concentration d) Run the 80 SS Cummins cycle to capture PM concentration at each mode using the AVL MSS e) Check the PM concentration levels and select the six-steady state modes from the 80 point matrix. As a first order, check the concentration at the pre-selected steady-state modes to see if they spread within reason around the concentration produced for the NTE transient cycle. If not, adjust the bypass as needed to establish the right spread in brake-specific emissions and concentration for the six steady-state modes 29 ------- f) Make sure that the points selected spread around a brake specific level and concentration level of a threshold of 0.025 g/hp-hr, and concentration range of 4 to 15 milligram per cubic meter. 3.2.4 Data Analysis Use the acquired data to create the "error surfaces" to be used by the Monte Carlo simulation. An example of the steady-state error surface determination is shown in Table 5 forPM. TABLE 5. EXAMPLE OF SS ERROR SURFACE Error Surface for SS PM Concentration Figure 5 x-axis y-axis PM ug/mole (lab mean at setpoint) PM ug/mole (PEMS) Figure 6 x-axis y-axis 5th percentile y-axis 50th percentile y-axis 95th percentile PM ug/mole (lab mean at setpoint) 5th [PM ug/mole (PEMS) - PM ug/mole (|ab)] 50th [PM ug/mole (PEMS) - PM ug/mole (|ab)] 95th [PM ug/mole (PEMS) - PM ug/mole (|ab)] 77?e 5th, 50th and 95th percentiles from the (PEMS - lab) delta data will be used to estimate the 1st and 99th percentiles from assumed Gaussian distributions. Figure 7 x-axis y-axis Z-axiS = Ass_PM_|jg/mole ic sample frequency ic sample distribution PM ug/mole (lab mean at setpoint) ic ss PM 1st Percentile from Gaussian distribution based on 5th and 50th [PM ug/mole (PEMS) - PM ug/mole (|ab)] deltas. 99th Percentile from Gaussian distribution based on 50th and 95th [PM ug/mole (PEMS) - PM ug/mole (|ab)] deltas. 50th Percentile based on [PM ug/mole (PEMS) - PM ug/mole (lab)] deltas. once per NTE event trial Gaussian (normal distribution) 30 ------- 3.3 Precision Errors under transient engine operation (dynamic response) 3.3.1 Objective The objective of this portion of the work is to determine the precision error, ATRwPM, with each PM-PEMS under NTE transient engine operation. This will be achieved by creating a 20 to 25-minute transient NTE cycle where the PEMS measure in each NTE. 3.3.2 Background PEMS are expected to operate in a repeatable manner over NTE events as short as 30 seconds. Two sources of PEMS precision error are hypothesized: 1) dynamic response to rapidly changing signals, and 2) susceptibility to "history" effects. Dynamic response error includes error due to measurement signal time alignment, and the dissimilarity of the dynamic response and aliasing of signals; including those signals used to determine entry into and exit from the NTE zone. History effects include the effects of previously measured quantities on currently measured quantities. For example, this may be caused by ineffective sample exchange in the PM emissions sampling volumes, or it may be caused by one or more sensors' characteristic rise time or fall time. To account for any dynamic response precision error, the increase in precision error incremental to the steady-state emissions measurement precision will be incorporated into the overall error model. Selection of short NTE cycles (each 32 seconds) maximizes the sensitivity of this test to effects of dynamic response. Thirty-two seconds was chosen as the minimum instead of thirty seconds, which is the shortest NTE event time, to ensure that 1 Hz ECM updating of torque and speed values would be unlikely to interfere with capturing NTE events. For each repeat of the test cycle, the order of the 30 different NTE events will be the same. In addition the 29 different intervals separating each NTE event from the next will have a range of durations and these will be randomly arranged in each test cycle as well. Fixed arrangement of the NTE events and the inter-NTE events will maximize the sensitivity of this test to dynamic response and history effects, and make the DPF and bypass operation very consistent. The total length of the NTE transient cycle will assume that only 5 quartz crystal of the Sensors PPMD are working, and it takes five minutes of stabilization time for reusing a crystal after PM collection. Thus, the same NTE transient cycle used in the gaseous PEMS program will be used here, except for changes in the inter-NTE times to accommodate the Sensors PPMD. 3.3.3 Methods and Materials a. Use a transient engine dynamometer emissions laboratory. ------- b. Use a laboratory that can accommodate at least three PEMS, their power supplies, the PEMS flow meters, cables and lines. c. Use same overall guidelines described in section 3.2, but applied to transient engine testing. d. Record the EEPS' total mass signal during transient testing. Challenge PEMS to 30 different 32-second NTE events, shown in Table 5, over about 23 minute test cycle, or whatever needed to accommodate the need for five crystals of the PPMD to be operational. Randomize the NTE events shown in Table 6 once, scale up every fifth inter-NTE time, shown in Table 7, to accommodate the PPMD, and use the same order for repeat testing. Repeat the test cycle 10 times for each set of three PEMS. Note that for any torque command that is less than zero, command closed throttle (i.e. zero or minimum fuel command), and motor the engine at the commanded speed for that data point. An example of an NTE transient cycle is shown in Figure 9. Based on 10 repeats with each set of PEMS, the total number of repeats will be 30 cycles, assuming 1 NTE cycle x 10 repeats x one exhaust configuration x 3 sets of PEMS x one engine (1x10x1x3x1 = 30). Assuming a 25 minutes of NTE with 30 minutes of forced regeneration and preparation for the second repeat, the total number of days for NTE transient testing is 10 days (8 hours per day). This time includes PEMS and engine setup, PEMS warm up, and daily checks. TABLE 6. NTE TRANSIENT CYCLE NTE Event NTE1 NTE2 NTE3 NTE4 NTE5 NTE6 NTE7 NTE8 NTE9 NTE10 1 Speed % Range 17% 59% Governor line 17% 59% Governor line 17% 59% 100% Lower third Torque % Range J32% 332% J32% 66% 66% 66% 100% 100% 100% 332% - 100% Description Steady speed and torque; lower left of NTE Steady speed and torque; lower center of NTE Steady speed and torque; lower right of NTE Steady speed and torque; middle left of NTE Steady speed and torque; middle center of NTE Steady speed and torque; middle right of NTE Steady speed and torque; upper left of NTE Steady speed and torque; upper center of NTE Steady speed and torque; upper right of NTE Highly transient torque; moderate transient 32 ------- NTE11 NTE12 NTE13 NTE14 NTE15 NTE16 NTE17 NTE18 NTE19 NTE20 NTE21 NTE22 NTE23 NTE24 NTE25 NTE26 NTE27 NTE28 NTE29 NTE30 Upper third Middle third 17% - governed 17% - governed 17% - governed 332% - 100% 332% - 100% Lower third Upper third Middle third Lower right diagonal Upper left diagonal Full diagonal; lower left to upper right Lower left diagonal Upper right diagonal Full diagonal; lower right to upper left Third light — heavy-duty NTE event from International, Inc. data set Cruise; ~ 50 mph Cruise; ~ 75 mph Small bulldozer Large bulldozer Second of three FTP NTE events in Third light — heavy-duty NTE event from International, Inc. data set First of two NTE First of two NTE events in NRTC events in NRTC speed Highly transient torque; moderate transient speed Highly transient torque; moderate transient speed Highly transient speed; moderate transient torque Highly transient speed; moderate transient torque Highly transient speed; moderate transient torque Transient; speed increases as torque increases Transient; speed increases as torque increases Transient; speed increases as torque increases Transient; speed decreases as torque increases Transient; speed decreases as torque increases Transient; speed decreases as torque increases Sample from LHDE Sample from HDDE Sample from HDDE Sample from NRDE Sample from NRDE Seconds used from FTP: 714-725, 729- 743,751-755 Sample from LHDE Seconds used from NRTC: 423-430, 444, 448-450, 462-481, increased 464 speed from 40% to 42% Seconds used from NRTC: 627-629, 657- 664, 685-696, 714-722 1 Speed (rpm) = Curb Idle + (Speed % * (MTS - Curb Idle) 2 Torque (Ibf-ft) = Torque % * Maximum Torque At Speed (i.e. lug curve torque at speed) 3 Torque (Ibf-ft) = Maximum of (32 % * peak torque) and the torque at speed that produces (32 % * peak power) 33 ------- TABLE 7. DYNAMIC RESPONSE INTER-NTE EVENTS INT Event1 INT1 INT2-6 INT7-10 INT11-14 INT15-18 INT 19-21 INT22 INT23 INT24 INT25 INT26 INT27 INT28 INT29 INT30 INT31 Duration (s) 10 2 3 4 5 6 7 8 9 11 13 17 22 27 35 5 Frequency 1 5 4 4 4 O 1 1 1 1 1 1 1 1 1 1 Description Initiation of cycle; INT1 is always first Shortest and most frequent inter-NTE events Short and frequent inter-NTE events Short and frequent inter-NTE events Short and frequent inter-NTE events Short and frequent inter-NTE events Medium inter-NTE event Medium inter-NTE event Medium inter-NTE event Medium inter-NTE event Long inter-NTE event Long inter-NTE event Long inter-NTE event Long inter-NTE event Longest inter-NTE event Termination of cycle; INT31* is always last Interval speeds and torques are not identical, but they are clustered around zero torque and the speed at which 15% of peak power and 15% of peak torque are output. 34 ------- Torque-Speed Domain 700 600 500 _ 400 |