Populations, Activity and Emissions of
Diesel Nonroad Equipment in
EPA Region 7
&EPA
United States
Environmental Protection
Agency
-------
Populations, Activity and Emissions of
Diesel Nonroad Equipment in
EPA Region 7
Assessment and Standards Division
Office of Transportation and Air Quality
U.S. Environmental Protection Agency
Prepared for EPA by
Eastern Research Group, Inc. (ERG)
EPA Contract No. EP-C-06-080
NOTICE
This technical report does not necessarily represent final EPA decisions or
positions. It is intended to present technical analysis of issues using data
that are currently available. The purpose in the release of such reports is to
facilitate the exchange of technical information and to inform the public of
technical developments.
&EPA
United States
Environmental Protection
Agency
EPA-420-R-12-009
July 2012
-------
Eastern Research Group, Inc.
Populations, Activity and
Emissions of Diesel Nonroad
Equipment in EPA Region 7
FINAL REPORT
Prepared for:
U.S. Environmental Protection
Agency
October 19, 2010
Revised by EPA Staff on January 7, 2011, April
11,2011 &May 13,2011
-------
www.erg.com
ERGNo. 0218.04.001.001
Populations, Activity and Emissions of Diesel Nonroad Equipment in EPA
Region 7
FINAL REPORT
EPA Contract # EP-C-06-080
Prepared for:
Constance Hart
Work Assignment Manager
Carl Fulper
Project Officer
James Warila
Alternate Work Assignment Manager
U.S. Environmental Protection Agency
2000 Traverwood Drive
Ann Arbor, MI 48105
Sandeep Kishan
Scott Fincher
Michael Sabisch
ERG, Inc.
3508FarWestBlvd,
Suite 210
Austin, TX 78731
Prepared by:
Rob Santos
The Urban
Institute
Mia Zmud
Sudeshna Sen
Sally Amen
NuStats
Carl Ensfield
Sensors, Inc.
October 19, 2010
5608 Parkcrest Drive, Suite 100. Austin. TX 78731-4947. Telephone: (512) 407-1820. Fax (512) 419-0089
Arlington. VA Austin. TX Boston, MA Chantclly. VA Chicago. !L Lexington, MA Portland, ME Morrisville. NC Sacramento. CA
Equal ( ;i- , ftccycied Paper-
-------
Table of Contents
Acronyms vii
Glossary of Terms ix
Acknowledgements xi
Executive Summary ES-1
Test Program Overview Overview-1
1.0 Introduction 1-1
2.0 Background 2-1
3.0 Study Design Overview 3-1
3.1 Workplan and Survey Design Development 3-1
3.2 Development of the Draft Quality Assurance Project Plan (QAPP) 3-9
3.3 Conducting Equipment Ownership Surveys, Revising QAPP 3-10
3.4 Acquiring and Preparing EAM Equipment 3-12
3.5 Overview of Fieldwork Activities 3-21
3.6 EOIPhase 1 Initial EOT/ EAM Interview 3-32
3.7 ES Phase 1 Emission and Activity Measurements 3-33
3.8 Integrated Sample Surveys, Phase 2 EAMS 3-34
3.9 Enhancements Prior to Phase 3 3-36
3.10 ES Phase 3 EAMS 3-41
3.11 Data Processing, Analysis and Submission 3-42
4.0 Sample Design Performance 4-1
4.1 EOIPhase 1 Results 4-2
4.2 EOI Phase 2 Results (Integrated EOI & ESI in PSU1) 4-7
4.3 EOI Phase 2 Results (Integrated EOI & ESI, PSUs 2 and 3) 4-13
4.4 EOI Phase 3 Results (SSI/EDA Combined Sample, PSUs 4 and 5) 4-17
4.5 Incentive Test Analysis 4-26
5.0 Fieldwork Operations 5-1
5.1 Overview of field protocols 5-1
5.2 Trade organization recruitment 5-4
5.3 Mail out and FAQ 5-4
5.4 EOI/EAM Script Development 5-5
5.5 EOI Performance 5-10
5.6 Summary of Onsite Inventories and Instrumentation 5-12
6.0 Study Results and Conclusions 6-1
6.1 Recruiting and EOI Findings 6-1
6.2 PEMS Measurement Results 6-3
6.3 PAMS Measurement Results 6-36
7.0 Lessons Learned and Program Recommendations 7-1
7.1 Sample Design and Recruitment 7-1
7.2 General Fieldwork Lessons and Recommendations 7-2
7.3 PEMS Lessons Learned 7-4
7.4 PAMS Lessons Learned 7-8
8.0 Data Conversion and Delivery 8-1
8.1 MySQL Database Delivery 8-1
9.0 References 9-1
10.0 Index of Appendices 10-1
-------
The Appendices cited in this document have been provided separately in electronic format. An
index of these appendix files is provided at the end of this report.
11
-------
List of Figures
Figure OV-1 PM Emissions from Backhoe Loaders, Work Basis, Filters 1 -3 OV-26
Figure OV-2 Gaseous Emissions from Backhoe Loaders, Work Basis, Overall test OV-26
Figure OV-3 PM Emissions from Dozers, 50-99 hp, Work Basis, Filters 1-3 OV-27
Figure OV-4 PM Emissions from Dozers, > 100 hp, Work Basis, Filters 1-3 OV-27
Figure OV-5 Gaseous Emissions from Dozers, 50-99 hp, Work Basis, Overall test OV-28
Figure OV-6 Gaseous Emissions from Dozers, > 100 hp, Work Basis, Overall test OV-28
Figure OV-7 PM Emissions from Excavators, < 300 hp, Work Basis, Filters 1 -3 OV-29
Figure OV-8 PM Emissions from Excavators, > 300 hp, Work Basis, Filters 1 - 3 OV-29
Figure OV-9 Gaseous Emissions from Excavators, < 300 hp, Work Basis, Overall Test... OV-30
Figure OV-10 Gaseous Emissions from Excavators, > 300 hp, Work Basis, Overall Test.. OV-30
Figure OV-11 PM Emissions From Loaders, Work Basis, Filters 1 -3 OV-31
Figure OV-12 Gaseous Emissions From Loaders, Work Basis, Overall test OV-31
Figure OV-13 PM Emissions From Other Equipment, Work Basis, Filters 1 -3 OV-32
Figure OV-14 Gaseous Emissions From Other Equipment, Work Basis, Overall Test OV-32
Figure OV-15 PM Emissions From Backhoe Loaders, Fuel Basis, Filters 1 -3 OV-34
Figure OV-16 Gaseous Emissions From Backhoe Loaders, Fuel Basis, Overall Test OV-35
Figure OV-17 PM Emissions From Dozers, 50-99hp, Fuel Basis, Filters 1 -3 OV-35
Figure OV-18 PM Emissions From Dozers, > 100 hp, Fuel Basis, Filters 1-3 OV-36
Figure OV-19 Gaseous Emissions From Dozers, 50-99 hp, Fuel Basis, Overall Test OV-36
Figure OV-20 Gaseous Emissions From Dozers, > 100 hp, Fuel Basis, Overall Test OV-37
Figure OV-21 PM Emissions From Excavators, < 300 hp, Fuel Basis, Filters 1 - 3 OV-37
Figure OV-22 PM Emissions From Excavators, > 300 hp, Fuel Basis, Filters 1 -3 OV-38
Figure OV-23 Gaseous Emissions From Excavators, < 300 hp, Fuel Basis, Overall Test... OV-38
Figure OV-24 Gaseous Emissions From Excavators, > 300 hp, Fuel Basis, Overall Test... OV-39
Figure OV-25 PM Emissions From Loaders, Fuel Basis, Filters 1-3 OV-39
Figure OV-26 Gaseous Emissions From Loaders, Fuel Basis, Overall Test OV-40
Figure OV-27 PM Emissions From Other Equipment, Fuel Basis, Filters 1 -3 OV-40
Figure OV-28 Gaseous Emissions From Other Equipment, Fuel Basis, Overall Test OV-41
Figure OV-29 Activity Summary by Equipment Category (in Hours) OV-42
Figure OV-30 Activity Summary by Equipment Category (in Percentages) OV-43
Figure 3.4-1 PEMS Rack and Trailer 3-13
Figure 3.4-2 Installation of PEMS Rack Using Truck-Mounted Boom 3-13
Figure 3.4-3 PEMS Rack During Construction (open sides), Front 3-17
Figure 3.4-4 PEMS Rack During Construction (open sides), Rear 3-17
Figure 3.4-5 Corsa EZII, Isaac V8, and HEMDATA Dawn Dataloggers 3-19
Figure 3.7-1 Emissions and Activity Measurements in ES Phase 1 3-34
Figure 3.8-1 Field Schedule for ES Phase 2 3-36
Figure 3.9-1 Optical Sensor on a Bracket With a Magnetic Base 3-39
Figure 3.9-2 Optical Sensor on an Aluminum Bracket 3-40
Figure 3.10-1 Field Schedule for ES Phase 3 3-42
Figure 4.4-1: Composition ofPSU4 and 5 Establishments by Sampling Frame Status 4-18
Figure 5.1-1 Original Field Protocol 5-2
Figure 5.1-2 Sampling Selection Weighting Criteria 5-3
Figure 6.2-1 PM Emissions from Backhoe Loaders, Work Basis, Filters 1 -3 6-20
in
-------
Figure 6.2-2 Gaseous Emissions from Backhoe Loaders, Work Basis, Overall test 6-20
Figure 6.2-3 PM Emissions from Dozers, 50-99 hp, Work Basis, Filters 1-3 6-21
Figure 6.2.4 PM Emissions from Dozers, > 100 hp, Work Basis, Filters 1 -3 6-21
Figure 6.2-5 Gaseous Emissions from Dozers, 50-99 hp, Work Basis, Overall test 6-22
Figure 6.2-6 Gaseous Emissions from Dozers, > 100 hp, Work Basis, Overall test 6-22
Figure 6.2-7 PM Emissions from Excavators, < 300 hp, Work Basis, Filters 1-3 6-23
Figure 6.2-8 PM Emissions from Excavators, > 300 hp, Work Basis, Filters 1-3 6-23
Figure 6.2-9 Gaseous Emissions from Excavators, < 300 hp, Work Basis, Overall Test 6-24
Figure 6.2-10 Gaseous Emissions from Excavators, > 300 hp, Work Basis, Overall Test 6-24
Figure 6.2-11 PM Emissions From Loaders, Work Basis, Filters 1 - 3 6-25
Figure 6.2-12 Gaseous Emissions From Loaders, Work Basis, Overall test 6-25
Figure 6.2-13 PM Emissions From Other Equipment, Work Basis, Filters 1 -3 6-26
Figure 6.2-14 Gaseous Emissions From Other Equipment, Work Basis, Overall Test 6-26
Figure 6.2-15 PM Emissions From Backhoe Loaders, Fuel Basis, Filters 1 -3 6-28
Figure 6.2-16 Gaseous Emissions From Backhoe Loaders, Fuel Basis, Overall Test 6-29
Figure 6.2-17 PM Emissions From Dozers, 50-99hp, Fuel Basis, Filters 1-3 6-29
Figure 6.2.18 PM Emissions From Dozers, > 100 hp, Fuel Basis, Filters 1-3 6-30
Figure 6.2-19 Gaseous Emissions From Dozers, 50-99 hp, Fuel Basis, Overall Test 6-30
Figure 6.2-20 Gaseous Emissions From Dozers, > 100 hp, Fuel Basis, Overall Test 6-31
Figure 6.2-21 PM Emissions From Excavators, < 300 hp, Fuel Basis, Filters 1 -3 6-31
Figure 6.2-22 PM Emissions From Excavators, > 300 hp, Fuel Basis, Filters 1 -3 6-32
Figure 6.2-23 Gaseous Emissions From Excavators, < 300 hp, Fuel Basis, Overall Test 6-32
Figure 6.2-24 Gaseous Emissions From Excavators, > 300 hp, Fuel Basis, Overall Test 6-33
Figure 6.2-25 PM Emissions From Loaders, Fuel Basis, Filters 1 - 3 6-33
Figure 6.2-26 Gaseous Emissions From Loaders, Fuel Basis, Overall Test 6-34
Figure 6.2-27 PM Emissions From Other Equipment, Fuel Basis, Filters 1 - 3 6-34
Figure 6.2-28 Gaseous Emissions From Other Equipment, Fuel Basis, Overall Test 6-35
Figure 6.3-1 Activity Summary by Equipment Category (in Hours) 6-39
Figure 6.3-2 Activity Summary by Equipment Category (in Percentages) 6-40
Figure 7.3-1 Burned and Melted Exhaust Tubing 7-4
Figure 7.3-2 Delaminated Exhaust Tubing 7-5
Figure 7.3-3 Backhoe Loader with Metal Tubing Installed 7-5
Figure 7.4-1 Pelican Case Housing a Corsa Datalogger 7-8
Figure 8.1-1 Entity-Relationship Diagram for MySQL Database 8-3
IV
-------
List of Tables
Table OV-1 Primary Sampling Units for the Pilot Study OV-4
Table OV-2 Study Goals by Measurement Type OV-5
Table OV-3 Expected Distribution of Completions by Sample and Measurement Type OV-6
Table OV-4 Revised Total Sample Needed to Achieve Sample Targets OV-12
Table OV-5 Expected Dispositions for each Sample Frame and Study Phase OV-15
Table OV-6 Actual Performance Rates for EOT, All Phases of Integrated Sample OV-16
Table O V-7 Actual Performance Rates for ESI, All Phases of Integrated Sample O V-16
Table OV- 8 Summary Counts of Establishments Inventoried OV-17
Table OV-9 Summary Counts of Equipment Inventoried and Instrumented OV-17
Table OV-10 Summary of data collected throughout the Nonroad PEMS Study OV-18
Table OV-11 Frame Characteristics of PSU 1 OV-20
Table OV-12 PSU 1 Expected vs. Actual Design Parameters OV-20
Table OV-13 PSU 2 and 3 Actual Versus Expected Design Parameters OV-22
Table OV-14 PSU 4 and 5 Actual Versus Expected Design Parameters OV-22
Table OV-15 Combined PSU 4 and 5 Incentive Test on Instrumentation OV-23
Table OV-16 Combined PSU 4 and 5 Incentive Test on Instrumentation Follow-Through OV-24
Table OV-17 Combined PSUs 2-5 Incentive Tests on Instrumentation Follow-Through ... OV-24
Table OV-18 Activity Summary by Equipment Category (in Hours) OV-41
Table OV-19 Descriptive Statistics for Key Study Variables OV-46
Table 3.1-1 Targeted Distribution of Observations by Measurement Type 3-3
Table 3.1-2 Original Plan for Conducting Establishment Sample Surveys 3-3
Table 3.1-3 Original Plan for Conducting Equipment Sample Measurements 3-4
Table 3.1-4 Expected Completions by Sample and Measurement Type 3-4
Table 3.1-5 Anticipated Sample Needed to Achieve Sample Targets 3-5
Table 3.1-6 Primary Sampling Units for the Pilot Study 3-6
Table 3.5-1 Summary of data collected throughout the Nonroad PEMS Study 3-22
Table 3.8-1 Revised Total Sample Needed to Achieve Sample Targets 3-35
Table 3.11-1 Flowmeter Usage During Each Phase of Fieldwork 3-45
Table 4.1-1 Frame Characteristics of PSU 1 4-3
Table 4.1-2 Expected Vs. Actual Design Parameters by Stratum for EOI Phase 1 4-4
Table 4.1-3 PSU 2-5 Projections for the Combined Establishment and Equipment Samples.... 4-6
Table 4.2-1 Distribution of Pilot and Fresh Samples by SSU Type for PSU 1 4-8
Table 4.2-2 EOI Phase 2 Screening and Recruitment Response Disposition Hierarchies 4-10
Table 4.2-3 Screening Rates for Phase 2 by Sample Frame 4-11
Table 4.2-4 EOI Phase 2 Eligibility Rates by Sample Source for PSU 1 4-12
Table 4.2-5 EOI Phase 2 Recruitment Rates by Sample Type for PSU 1 4-12
Table 4.2-6 EOI Phase 2 Overall Response Rates by Sample Source for PSU 1 4-12
Table 4.2-7 EOI Phase 2 Actual Versus Expected Design Parameters for PSU 1 4-13
Table 4.3-1 Comparing Two Sets of ESI Eligibility Criteria Used for PSUs 2 and 3 4-14
Table 4.3-2 Screening Rates for Combined PSUs 2 and 3 by Criteria Period 4-15
Table 4.3-3 Eligibility Rates for Combined PSUs 2 and 3 by Criteria Period 4-15
Table 4.3-4 PSU 2 and 3 Eligible Establishment Recruitment Rates by Criterion Period 4-16
Table 4.3-5 Overall Response Rates for Combined PSUs 2 and 3 by Criterion Period 4-16
Table 4.3-6 Combined PSU 2 and 3 Actual Versus Expected Design Parameters 4-17
-------
Table 4.4-1 Distribution of Establishments by Frame Status and PSU 4-18
Table 4.4-2 Combined SSI-ED A EOT Phase 3 Sample by Usability Status and PSU 4-19
Table 4.4-2a SSI-Only EOT Phase 3 Sample by Usability Status and PSU 4-19
Table 4.4-2b EDA-Only EOT Phase 3 Sample by Usability Status and PSU 4-20
Table 4.4-2c Both SSI & EDA EOI Phase 3 Sample by Usability Status and PSU 4-20
Table 4.4-3 Final Disposition of EOI Phase 3 Fielded Sample by PSU 4-22
Table 4.4-4 Actual and Expected Design Parameters for EOI Phase 3 Sample by PSU 4-23
Table 4.4-5 Comparison of SSI and EDA Sample Performance Rates 4-25
Table 4.4-6 Expected Eligible Establishments by Frame Using Phase 3 Eligibility Rates 4-25
Table 4.5-1 Combined PSU 4 and 5 Incentive Test on the Invitation to Instrumentation 4-27
Table 4.5-2 Combined PSU 4 and 5 Incentive Test on Instrumentation Follow-Through 4-27
Table 4.5-3 Combined PSUs 2-5 Incentive Tests on Instrumentation Follow-Through 4-28
Table 5.4-1 Eligibility Criteria by PSU 5-6
Table 5.4-2 Profile of Participants in Cognitive Interviews 5-7
Table 5.5-1 Expected Dispositions for each Sample Frame and Study Phase 5-11
Table 5.5-2 Actual Performance Rates for EOI, All Phases of Integrated Sample 5-12
Table 5.5-3 Actual Performance Rates for ESI, All Phases of Integrated Sample 5-12
Table 5.6-1 Summary Counts of Establishments Inventoried 5-13
Table 5.6-2 Summary Counts of Equipment Inventoried and Instrumented 5-13
Table 5.6-3 PEMS Testing Summary 5-15
Table 5.6-4 PAMS Testing Summary 5-18
Table 6.2-1 Descriptive Statistics for Key Study Variables 6-3
Table 6.2-2 PM Blind Study Results 6-4
Table 6.2-3 Dynamic and Field Blank Measurement Results 6-5
Table 6.2-4 PEMS Gaseous Results, Overall Average Work-Based Emissions 6-8
Table 6.2-5 PEMS Gaseous Results, Overall Average Fuel-Based Emissions 6-11
Table 6.2-6 PEMS Gaseous Results, Overall Average Time-Based Emissions 6-12
Table 6.2-7 By-Filter PEMS Results, Average Work-Based Emissions 6-15
Table 6.2-8 Nonroad Emission Standards Summary 6-19
Table 6.3-1 Activity Measurement Result Summary 6-37
Table 6.3-2 Activity Summary by Equipment Category (in Hours) 6-39
VI
-------
Acronyms
AMBII Automotive Micro-Bench II
BSFC Brake-Specific Fuel Consumption
CAN Controller Area Network
CAT ET Caterpillar Electronic Technician
CATI Computer Assisted Telephone Interviewing
CBS Comprehensive Business Samples
CDT Central Daylight Time
CO Carbon Monoxide
CO2 Carbon Dioxide
CRC Coordinating Research Council
CVS Tortoise Concurrent Versions System
cQCM Carousel Quartz Crystal Microbalance
CSV Comma-Separated Variable file
DRI Desert Research Institute
EAM Emissions and Activity Measurements
ECU Electronic Control Unit
EDA Equipment Data Associates, Inc.
EFM Exhaust Flow Meter
EOI Equipment Ownership Interview
EPA United States Environmental Protection Agency
ERG Eastern Research Group
ES Equipment Sample
ESI Equipment Sample Interview
FID Flame lonization Detector
FIPS Federal Information Processing Standard
g grams
GPS Global Positioning System
HP Horsepower
hr hour
kW Kilowatt
mA Milliamp
MOS Measure of Size
MPS Micro-Proportional Sampler
MSOD Mobile Source Observation Database
NDIR Non-dispersive infrared
NOx Oxides of Nitrogen
NTE Not to Exceed
OBD On-Board Diagnostics
OTAQ Office of Transportation and Air Quality
PEMS Portable Emissions Measurement System
PAMS Portable Activity Measurement System
PM Particulate Matter
PPS Probability Proportional to Size
PSU Primary Sampling Unit
vn
-------
QA Quality Assurance
QAPP Quality Assurance Proj ect Plan
QC Quality Control
QMP Quality Management Plan
RPM Revolutions Per Minute
SAE Society of Automotive Engineers
SAS Statistical Analysis Software
scfm Standard Cubic Feet per Minute
SOPs Standard Operating Procedures
SRI Southern Research Institute
SRS Simple Random Sampling
SSI Survey Sampling International
SSU secondary sampling unit
THC Total Hydrocarbons
Vlll
-------
Glossary of Terms
Caller ID: Establishments that could not be reached due to privacy manager i.e., the number
called automatically blocks unknown numbers
Eligibility rate: The rate at which fully screened establishments were determined to be eligible
for the study (e.g., they were eligible to be invited to participate in the equipment inventory). The
denominator is the total number of screened establishments.
EOI Phase 1: Equipment Ownership Interview, Phase 1 (telephone interviewing in PSU 1)
EOI Phase 2: Equipment Ownership Interview, Phase 2 (telephone interviewing and recruiting
inPSUsl,2and3)
EOI Phase 3: Equipment Ownership Interview, Phase 3 (telephone interviewing and recruiting
in PSUs 4 and 5)
ES Phase 1: Equipment Sample, Phase 1 (inventories and instrumentation fieldwork in PSU 1)
ES Phase 2: Equipment Sample, Phase 2 (inventories and instrumentation fieldwork in PSUs 2
and 3)
ES Phase 3: Equipment Sample, Phase 3 (inventories and instrumentation fieldwork in PSUs 4
and 5)
First refusal: An establishment kindly declined to participate in the interview during the
introduction or before screening could begin.
Final refusal: Establishment strongly declined to participate during the introduction or before
screening could begin.
General callback: Establishments wherein the respondent or screener provided a general day
and time to call them back.
Interview response rate: The rate at which screened and eligible establishments completed the
survey (can be used for either the EOI or ESI, depending on context). With regard to the ESI, the
recruitment rate and the interview rate represent the same thing. The denominator is the total
number of known (screened) eligible establishments.
Measure of size: The measure of the units in the target population or another measure of
influence that is assumed to correlate fairly strongly with the target population in each sampling
unit
IX
-------
Overall response rate: This rate is equal to the product of the screening and interview response
rates. It reflects net "overall" response from all sources and is typically used to document and
assess survey quality (vis a vis the potential for nonresponse bias).
Partial complete: Interview was initiated but stopped before it was completed and it was not
possible to recontact the establishment to complete the interview. Typically used with EOI
survey process
Partial refusal: Establishment initiated the interview and refused after the screening for
eligibility began.
Recruitment rate: The rate at which screened and eligible establishments agreed to the
inventory and/or instrumentation phase of the study. The denominator is the total number of
known (screened) eligible establishments.
Screening rate: This rate reflects the fraction of establishments for which we were able to make
an eligibility determination. This rate does not consider the actual eligibility status of an
establishment. Instead it reflects the fraction of establishments for which sufficient information
(via answers to survey questions) was provided to establish whether or not they are eligible to
participate in the EOI and/or ESI. The denominator of this rate is the original sample size.
Short complete: Those establishments that completed an EOI but did not complete the ESI.
Specific callback respondent: Establishments wherein the respondent or screener provided a
specific day and time to call them back.
-------
Acknowledgements
This multi-year pilot study was made possible through funding provided by the U.S.
Environmental Protection Agency (EPA) and the Coordinating Research Council (CRC). ERG
greatly appreciates having had the opportunity work on this complex and exciting project and
wishes to acknowledge the significant contributions of EPA and CRC was well as those of
integral partners NuStats, Sensors, Inc., the Urban Institute, Southern Research Institute and the
Desert Research Institute. The contributions of all our team members along with the project
design, planning and oversight support provided by EPA and CRC have been critical in this
study's success.
We must thank EPA management and technical expertise from John Koupal, Carl Fulper,
Connie Hart, Jim Warila and Bob Giannelli for their continual support throughout the study.
Several other EPA personnel also provided valuable project field support such as Ethan Schauer,
Brian Ratkos, Robert Caldwell, Chris Bahn, and Tim Quast; Nancy Tschirhart and Rosemary
Gehyk for the fuel analyses; Zuimdie Guerra for the PM filter analyses; and Donna Reinhart for
her contractual support.
XI
-------
(This page was intentionally left blank.)
xn
-------
Executive Summary
This study was a multi-year pilot project to develop recruitment and data collection
protocols as part of a broad effort to understand the population, emissions and activities of
nonroad equipment in various economic sectors. This study, which was supported by the U.S.
Environmental Protection Agency (EPA) and the Coordinating Research Council (CRC),
focused on commercial establishments within the construction sector. Statistical sampling was
applied to randomize the recruitment and screening of participants and the selection of
equipment pieces, with weighting applied toward size and usage history. Fieldwork involved
the installation and operation of portable on-board instruments to measure exhaust emissions and
equipment usage in EPA's Region 7 area (states of Iowa, Kansas and Missouri). Data was
collected during normal operation at construction worksites in three different phases over a 17
month period between June 2007 and October 2008.
Inventories were conducted at 79 worksites, with testing at 29 of those sites. Emissions
and activity data was collected on approximately 29 pieces of equipment each. Gaseous and
particulate-matter emissions data were collected over a typical working day using a specially
constructed enclosure housing a SEMTECH-DS, a micro-proportional Sampler (MPS), a three-
chamber 47mm gravimetric sampler and various-sized exhaust flowmeters, all manufactured by
Sensors, Inc. Activity measurements (1 Hz date / time / engine speed) were collected over a
period of approximately one month using Isaac and Corsa dataloggers.
Recruiting, technical and logistical challenges encountered during the study resulted in a
refinement of equipment and procedures for conducting future studies of this nature. These
challenges and the steps taken to address them are described in detail in this report. Data
collected have been subjected to extensive review, analysis and validation / correction.
Emissions are presented on a work basis, fuel basis and time basis. Although uncertainties are
presented with the emission results, these "in-use" duty cycles differ from certification test
cycles, and a comparison of these "in-use" emissions with emission standards would not be
appropriate.
This report and the associated data collected throughout the study represent a first step in
the process of improving the quantity and representativeness of data available for nonroad engine
inventory modeling. EPA will continue this process by performing additional review, including
comparison of this data with data from other EPA and non-EPA emission test programs. It is
anticipated that these data will finally be used to develop relationships and emission rates in a
non-road version of the MOVES model. This report and data from this study will be released to
ES-1
-------
the general public after EPA's comprehensive review has been completed and approval from
sponsors and EPA senior management has been granted, and findings from EPA's review will be
included in subsequent EPA reports.
ES-2
-------
Test Program Overview
Introduction
Previous work has shown that nonroad equipment contributes substantially to mobile-
source emissions, with their contribution in relative terms expected to increase as emissions from
highway vehicles are controlled (Kean, Sawyer & Harley 2000). Generally, nonroad equipment
includes vehicles powered by combustion engines, designed to perform a wide variety of tasks
other than street or highway transportation. Thus, the term "nonroad equipment" covers a broad
variety of machines including forklifts, graders, crawler dozers, backhoe loaders, excavators and
other equipment.
A report published by the National Academy of Sciences emphasized the need for EPA to
design and implement programs to expand and improve the data used to support emissions
inventory estimates from nonroad equipment. This and other NRC report recommendations have
influenced the concept and design of EPA's new inventory model for highway vehicles, the
Motor Vehicle Emissions Simulator (MOVES). Therefore, this study was intended as the first
step in a program to respond to recommendations concerning the quantity and representativeness
of the data supporting inventory modeling for nonroad engines.
The Nonroad PEMS and PAMS study was a multi-year pilot study funded and supported
by the U.S. Environmental Protection Agency (EPA) and the Coordinating Research Council
(CRC) which was intended to refine methods of developing larger-scale estimations of
populations, usage and emissions of heavy-duty non-road diesel equipment in various economic
sectors. During this pilot study, the ERG team, consisting of ERG, Sensors, Inc., NuStats, the
Urban Institute, Southern Research Institute and the Desert Research Institute worked with the
United States Environmental Protection Agency (EPA) to integrate statistical sampling
techniques, the latest activity and emissions measurement technology and rigorous quality
assurance and quality control methods to characterize in-use, real-world emissions from nonroad
diesel engines within the commercial construction sector in EPA Region 7 (work was performed
in the states of Iowa, Kansas and Missouri).
The study focused on commercial establishments within the construction sector, with the
objective of developing recruitment and data collection protocols for ultimately expanding the
study to other commercial sectors that operate fleets of nonroad diesel engines (owned or leased)
in their daily operations such as agriculture, mining, and utility sectors. These protocols
included:
Overview-1
-------
Sample Frame - the list from which a sample of commercial establishments is
drawn.
Sample Design - the approach for randomly selecting commercial establishments
within the sample frame to meet a specified sampling target.
Recruitment Protocols - the process and materials used to (1) provide advance
notice to prospective participants about the study, (2) screen establishments for
eligibility as a study participant, and (3) recruit qualified establishments to
participate in the study (this later process includes the use of monetary
incentives).
Data Processing and Statistical Analysis- procedures for processing and
analyzing the dispositions or outcomes of sampling and recruitment stages for the
purposes of data weighting and analysis.
While these protocols and data collection processes were developed, tested and modified
within the construction sector, it is reasonable to expect that they will generally apply to other
commercial sectors with some modifications needed to address nuances with sample frames and
fleet operations that may be specific to a particular commercial sector.
In total, 549 establishments were interviewed regarding their equipment ownership and
usage. From those establishments, 119 volunteered to allow project team members to conduct
onsite inventories and emissions and activity measurements on their eligible nonroad equipment.
Inventories were conducted at sites of 79 of those establishments, and emissions and/or activity
measurements were performed at sites of 29 establishments. Emissions tests were ultimately
conducted on 58 different pieces of nonroad equipment, and activity information was collected
from 30 pieces of nonroad equipment. Statistical sampling was applied prior to and during
fieldwork in order to randomize the recruitment and screening of participants and the selection of
equipment to be instrumented, and various ways of establishing a rapport with and minimizing
our testing burden on participants were explored throughout the study.
For the emissions measurements, portable on-board emission measurement systems
(PEMS) were used to perform 40 CFR 1065-compliant onboard measurements of gaseous and
aggregate particulate matter (PM) exhaust emissions on 50 horsepower or greater diesel engines
in nonroad construction equipment. In order to withstand the rigors of testing in a nonroad
environment, equipment had to be "ruggedized", and equipment modifications and
enhancements were made throughout the course of the study to achieve test goals.
For the activity measurements, various commercially-available portable activity
measurement systems (PAMS) were evaluated, three different types of systems purchased, and
Overview-2
-------
these systems were used during the work assignment to collect activity (date / time / engine
speed) over a period of approximately one month. This activity testing provided information on
long-term equipment usage patterns of this equipment and also provided an opportunity to test
and evaluate the performance of PAMS equipment and data quality during the work assignment.
Study data was analyzed, quality assured and processed for input into the EPA Office of
Transportation and Air Quality's (OTAQ's) Mobile Source Observation Database (MSOD),
where it may be used to help expand and improve the data currently supporting emission
inventory modeling for nonroad engines.
Multi-Stage Sample Plan
The sample design for this pilot study employed stratified multi-stage probability
sampling with probabilities proportional to size. The number of selection stages varied by the
type of data collection (i.e., establishment vs. equipment samples). The survey of establishments
employed two-stages of selection, while the equipment samples involved a three-stage design, as
follows:
First Stage (Primary) County level
Second Stage (Secondary) Establishment level (within county)
Third Stage (Tertiary) Equipment piece (within establishment)
Because the first- and second-stage sampling units were different sizes, in terms of the
numbers of equipment expected within each unit, the second-stage samples were drawn with
probability proportional to size (PPS). In PPS, the first and second-state selection probabilities
would then be managed to compensate for the fact that third-stage probabilities cannot be
managed.
The measure of size (MOS) is ideally a measure of the units in the target population
(equipment piecesor diesel enginesin this case), or alternatively, another measure of
influence that is assumed to correlate fairly strongly with the target population in each sampling
unit. Lacking direct estimates of equipment populations by county or establishment, the number
of employees per establishment was selected because it was assumed to be correlated to
equipment population.
Overview-3
-------
In the third stage, one or more eligible equipment pieces would be drawn from each
selected establishment using simple random sampling (SRS). Specifics of the design at each
stage are discussed below.
First Stage. The first stage of selection was utilized by all samples in the study and
involved a sample of 5 counties (primary sampling units or PSUs) with probabilities proportional
to size from the collection of counties selected within EPA Region 7. EPA designed the sample
and selected these counties, which are shown in Table OV-1.
Table OV-1 Primary Sampling Units for the Pilot Study
PSU
1
2
3
4
5
FIPS
29095
19113
19163
29047
20177
STATE
MO
IA
IA
MO
KS
COUNTY
Jackson
Linn
Scott
Clay
Shawnee
EST. NO.
EMPLOYEES
63,800
25,400
18,500
16,500
13,000
SAMPLING
PROBABILITY
0.3063
0.1216
0.4277
0.3813
0.3006
Second Stage. Within each PSU, commercial establishments were the secondary
sampling unit (SSU), drawn with selection probabilities based on the same measure of size used
to draw the first-stage sample. For construction establishments, the establishment MOS was the
estimated number of employees, as derived for use in the first stage. After compiling the
establishment frame for each PSU, an estimated number of employees was assigned to each
establishment (MOS), based on its assigned size class.
Third Stage. The third stage of selection was relevant to the sampling of equipment from
the equipment inventory performed prior to instrumentation.
Overview of Study Plan
Unlike motor vehicles, for which registration databases exist, it is not practical to
construct or obtain sample frames listing individual equipment pieces. To compensate for this
difficulty, coverage was expressed in terms of the owners or users of equipment within the
construction sector. Thus, for this pilot study, coverage of the target population included engines
owned, rented or leased by commercial establishments in the construction sector that employed
at least one person on a full-time or part-time basis during the previous twelve months. Elements
Overview-4
-------
of the target population were identified and selected through the constituent establishments that
operated this equipment.l
At the onset of this study, the ERG team employed the Comprehensive Business Samples
(CBS) supplied by Survey Sampling International (SSI) of Fairfield, Connecticut as the sampling
frame. SSI compiles listings from telephone directories and additional industry-specific sources
including government listings, bank records, trade directories, city directories and proprietary
sources. Listings are verified and updated on a continuous basis.
The study objectives called for the collection of emissions and activity data from 50
nonroad diesel engines operated within the study area. To achieve this, it was necessary to
collect a total of 550 observations from two independent samples:
Establishment sample - a sample of 500 establishments for a telephone survey,
referred to as the Equipment Ownership Interview (EOI), involving the
administration of an equipment ownership survey to screen and qualify
establishments for the study.
Equipment sample - a sample of establishments from which inventories, 25 emissions
measurements and 25 activity measurements would be drawn. These were
establishments qualified in the Establishment sample who agreed to participate in an
inventory and emissions and activity testing of their equipment fleet.2
The administration of the EOI was intended as a predecessor to the administration of the
Equipment Sample Interview (ESI) to recruit participants for an equipment inventory and
random selection of equipment for instrumentation.
Specific targets were provided for the numbers of establishments and pieces of equipment
for which data were collected. Table OV-2 presents the study goals by measurement type.
Table OV-2 Study Goals by Measurement Type
1
MEASUREMENT
TYPE
ESTABLISHMENT
SAMPLE
EQUIPMENT 1
SAMPLE |
1 Government and other non-commercial establishments were excluded from the target population.
For the 50 observations involving equipment measurements, an experiment was embedded to test for response rate
effects associated with incentives. The incentive was offered to a random half of the eligible Equipment Sample
subjects upon completion of the EOI (telephone) portion of the survey (and as part of the recruitment process into
the inventory and measurement components of data collection).
Overview-5
-------
Economic Sector
Construction
Total EOI and
Instrumentations
550
Equipment
Ownership Interview
500
Emissions
Measurement
25
Activity
Measurement
25
This pilot study was planned to be implemented in two phases for the Establishment
Sample and three phases for the Equipment Samples. The Pilot would commence with the first
phase of the Establishment Sample (involving conduct of only the EOI). EOI data collection
would start in one of the five study areas: Primary Sampling Unit 1 (PSU1) followed by Phase 2
of the Establishment Sample (EOI Phase 2). The Equipment Sample would be conducted
following completion of the Establishment Sample. The rollout of both planned phases of the
Establishment Sample and all three phases of the Equipment Sample are shown in Table OV-3
below. A total of 185 EOI interviews were expected to be needed in order to secure 60 total
equipment emissions and activity measurements. This table also presents the by-phase
distribution of inventories and instrumentations which were expected. Oversampling was
employed to ensure emissions and activity measurement test targets were met.
Table OV-3 Expected Distribution of Completions by Sample and Measurement
Type
COMPLETES BY TYPE
EOI - Establishment Sample
EOI - Equipment Sample
Inventory
Emissions measurements
Activity measurements
EOI
PHASE 1
(PSU 1)
100
0
0
0
EOI
PHASE 2
(PSUs 2-5)
400
0
0
0
ES
PHASE 1
(PSU 1)
37
12
6
6
ES
PHASE 2
(PSUs 2,3)
74
24
12
12
ES
PHASE 3
(PSUs 4,5)
74
24
12
12
TOTAL
500
185
60
30
30
PEMS and PAMS Equipment Used in Study
The ERG team used SEMTECH-DS PEMS manufactured by Sensors, Inc. and provided
by the EPA for collection of emissions data for this work assignment, and researched and
acquired from commercial vendors the PAMS used for collection of activity measurements.
For PEMS testing, the ERG team leased a trailer from Sensors, Inc. which was used to
house, transport and maintain the PEMS and all associated support equipment. Sensors
personnel transported this trailer to and from EPA Region 7 and the various work locations
within the region using a flatbed truck with a boom lift, which was also part of the lease
Overview-6
-------
agreement. This boom lift allowed the PEMS installation team members to place the
approximately 400-pound PEMS rack onto each piece of equipment being tested.
For emissions measurements, the PEMS rack collected the following information in one-
second intervals, as specified in the work assignment:
engine speed (revolutions per minute, rpm),
oxygen concentration in the exhaust stream ([62], percent by weight, wt%),
carbon-dioxide concentration in the exhaust stream ([CO2], percent by weight, wt%),
oxides of nitrogen concentration in the exhaust stream ([NOX], parts per million, ppm),
carbon monoxide concentration in the exhaust stream ([CO], percent by weight, wt%)
total hydrocarbon concentration in the exhaust stream, ([THC] parts per million, ppm)
aggregate particulate matter by gravimetric methods (g),
ambient temperature (°C),
exhaust temperature (°C),
exhaust mass flow rate (via the Sensors EFM)
relative humidity (%), and
barometric pressure (kilo-Pascals, kPa).
date/time stamp.
The following derived measurements, also specified in the work assignment, were provided
for all emissions measurements:
exhaust flow volume (adjusted to standard temperature and pressure, cu. ft/min (scfm)),
fuel flow volume (g/sec, gal/sec),
carbon dioxide emission rate (g/sec, g/kg fuel),
pollutant emission rates for NOx, CO, THC, and PM, (g/sec, g/gal).
The PEMS rack consisted of the EPA-provided SEMTECH-DS PEMS (as well as a
backup SEMTECH-DS PEMS), a Sensors micro-proportional Sampler (MPS), a Sensors
Gravimetric Filter Sampler and 2", 3", 4" and 5" diameter exhaust flowmeters (EFMs).
Flowmeter diameter selection was based on each particular installation. A small air compressor
and filtration unit was used to operate the MPS, and to automatically back-purge the EFM
pressure lines at specified intervals. This air compressor operated using A/C power provided by
a Honda portable generator. Automated zero calibrations of the SEMTECH-DS analyzers were
performed throughout the sampling period using ambient air which was scrubbed with a carbon
filter and a particulate filter.
All emissions measurements throughout the study included gravimetric filter sampling
using a micro-proportional sampling system (MPS) and Sensors' 3-chamber gravimetric filter
sampler provided by EPA. The SEMTECH MPS is a two-stage dilute proportional sampler in
which a proportional sample flow is extracted from the exhaust flow. This sample flow is
controlled to be a constant fraction of the varying exhaust flow by way of a two-stage dilution
Overview-7
-------
system. The first stage performs a fine "valving" function and the second stage is a venturi
which adds the major part of the dilution flow and forces the sample plus the primary dilution
flow to exit the MPS (Fulper, Giannelli, et al., 2010). The MPS total flow (sample flow plus
primary and secondary dilution) was held to a constant rate of approximately 12.5 liters per
minute in this study. Maximum exhaust flowrates and major and minor dilution ratios were
tailored for each installation based on engine size and anticipated workload according to
guidelines in the PEMS installation SOPs (Appendix F). Additional information regarding
flowrates and system settings are available in Appendix F (PEMS Installation SOPS) and
Appendix I (PEMS Data QC Criteria).
Gravimetric PM samples were collected on 47mm Teflon filters housed in the
gravimetric filter sampling unit which was heated to 1065 specifications. Filter flow was
maintained at a rate of approximately 18 liters per minute. The filter sampler automatically
switched between the three gravimetric filters, based on an integral timer, input voltage signal
indicating engine start, or the filter could also be switched by a wireless remote electrical signal.
One filter was used to capture the first start at the beginning of the day or shift (generally a 10-
minute cold-start), and the second and third filters captured continued warm operation or hot
start emissions (20-minutes sampling for the second filter and 30-minutes sampling for the third
filter). Each "start" episode was defined as the first ten (10) minutes of operation after the
engine was turned on.
ERG acquired "core" portable activity measurement systems (PAMS) conforming to the
PAMS specifications outlined in the work assignment. These systems measured and recorded
engine on and off times, engine speed and associated date and time stamps over an
approximately one month period for each activity instrumentation.
Initial Surveys and Phase 1 of Field Testing
The Establishment Sample EOT was administered to Phase 1 (PSU 1) survey participants
from April 20, 2007 through June 20, 2007. Data was collected under a pledge of confidentiality
that responses would be used for statistical purposes only. During collection, storage and
reporting, steps were taken to protect the identity of respondents or establishments with the
information collected, as required by the workplan and the QAPP.
Prior to the administering the EOT survey, the study team conducted a number of
activities with the purpose of improving response and participation rates. These included
securing study support from area trade associations, conducting an advance mailing of a letter
Overview-8
-------
and brochure to prospective respondents and performing cognitive testing / structured interviews
(these followed Phase 1 of the EOT surveys).
Following the initial EOT PSU 1 survey, the EOT Phase 1 EOT / ESI interview was
conducted (initial surveys followed by recruiting of eligible establishments). To accomplish this,
a sample of businesses was drawn, the EOT was administered (survey only, not recruiting),
followed by the administration of an incentive experiment, and finally the administration of the
Equipment Sample Interview (ESI) with the objective of recruiting qualified establishments to
participate in the inventory and instrumentation phase of the study. During the ESI, data on the
qualified businesses (e.g., business name, site address selected for the inventory, contact name
and phone number, etc.) that completed the EOI and agreed to be inventoried were electronically
transferred to ERG by way of a secure FTP site. Personnel in ERG's Kansas City office
retrieved the establishment information from the secure FTP site and contacted the participating
establishments to schedule the inventory appointments.
Inventories were conducted at one or two sites for each establishment. For each site
inventoried, information regarding all equipment belonging to, leased by, or used by a particular
establishment was collected on a site inventory form. Sufficient information was gathered for
each piece of equipment in order to allow determination of equipment model year and engine
power using supplemental information, such as EquipmentWatch or other commercially
available equipment specification resources. Equipment serial number and cumulative hours of
use (from the equipment's hour meter) were recorded and each piece of equipment's PEMS and
PAMS testability was also assessed. Digital photographs of equipment, serial numbers and
equipment specification tags were taken whenever possible to help clarify or correct any
ambiguous or inaccurate information recorded during on-site inventories. Inventory information
collected in the field was entered into a master spreadsheet posted on an ERG-internal project-
specific secure server. In Austin, equipment horsepower and model year information was
determined using equipment specification literature. Information pertaining to the equipment's
age, engine size, and cumulative usage was used to assign each piece of equipment to a specific
weighted bin for PEMS and PAMS selection. After equipment was classified in the weighted
stratification bins, individual pieces of equipment were selected for PEMS and PAMS
instrumentation. Each eligible piece of equipment was selected either as a primary, or "first
choice" selection, or as "backup" equipment in case one or more of the primary pieces of
equipment could not be tested. After the equipment information entry was verified and PEMS
and PAMS instrumentation selections were made, this information was posted back to the
project-specific secure server for onsite field staff to retrieve.
Overview-9
-------
After retrieval of the equipment selections, the ERG onsite installation manager
scheduled instrumentations with appropriate establishment contacts. If a site contact indicated
the selected equipment was not to be used or not available for testing during the anticipated
measurement period, an alternate piece of equipment was selected for instrumentation from the
list of secondary selections.
At the outset of fieldwork, the sampling plan permitted up to four pieces of equipment
(two emissions measurements and two activity measurements) to be sampled per establishment.
However, EPA and the ERG team decided to allow some of the sampled pieces of equipment to
be measured for both activity and emissions, and during the course of the work assignment, the
teams discovered that fewer establishments used PEMS-eligible equipment than originally
anticipated. Because of this, PEMS sampling requirements were relaxed to allow up to five
pieces of equipment to be sampled per establishment (three emissions and two activity
measurements), again with some equipment possibly having both PEMS and PAMS installed.
PAMS installations were performed at the outset of each phase of testing, prior to the
start of PEMS testing. PAMS units were then revisited, monitored and maintained throughout
the PEMS test period for each phase of the study. Teams of two to three people performed the
PAMS installations, and on average installed two PAMS per day (typically at the same site or
establishment). Both Corsa and Isaac PAMS units were used during each phase, and engine
speed (revolutions per minute, or RPM) was collected in several different ways, including via a
Capelec voltage processor connected to the equipment's battery, an optical sensor directed at a
rotating object to which reflective tape was applied, a magnetic pickup mounted near a rotating
object to which a magnet was affixed, or by non-destructively tapping into the equipment's
electronic tachometer signal. Non-destructive taps were accomplished using supplemental
connectors with harnesses which connected inline with the equipment's original harness.
After PAMS installations were complete (or nearly complete), PEMS emissions
measurements were commenced. PEMS emissions measurements were usually performed by
three person teams, with a fourth field technician providing fieldwork management, PEMS rack
mounting support, testing oversight and other fieldwork logistic support. Emission
measurements were typically gathered over a one-day period in an effort to collect emissions
information throughout the equipment's entire work day. PEMS installation, operation and
maintenance was scheduled and performed in such a way as to minimize interruption of
equipment use. PEMS instrumentation teams generally performed installations during each
site's non-working hours (after the equipment was no longer needed for that working day).
Hence, PEMS installations usually took place the evening prior to the day of testing, and the
Overview-10
-------
instrumentation team would then arrive the next morning at least two hours prior to site
operations to warm-up, calibrate and verify the equipment's operation prior to emissions testing.
This schedule usually allowed the PEMS instrumentation team to obtain cold-start emissions
data for both gaseous pollutants and PM.
For every PEMS and PAMS instrumentation, detailed information pertaining to the
equipment being instrumented as well as calibration, filter sampling and other PEMS and PAMs
test details were collected on PEMS and PAMS instrumentation forms.
Field staff followed the methodology provided in the project QAPP and associated
standard operating procedures (SOPs) for both PEMS and PAMS testing. Significant detail was
associated with PEMS and PAMS instrumentations, and procedural training was provided to all
team members prior to fieldwork. Daily calibrations of PEMS equipment were performed, and
laboratory calibration and verification of flowmeter measurements and SEMTECH gaseous
measurement linearity results were performed by Sensors, Inc. prior to and following each phase
of fieldwork.
Operation of all PEMS test equipment was continuously monitored by PEMS
instrumentation teams throughout the test day. During breaks in usage of the construction
equipment, team members would attempt to access the PEMS to refill the generator, replace the
gravimetric filters and calibrate the SEMTECH-DS, as necessary. Real-time monitoring of test
parameters was performed using remote laptops connected to the PEMS rack wirelessly in order
to identify and correct any data or equipment issues. In addition, test files were extracted during
and immediately after each test, processed and reviewed for data quality issues or problems.
PEMS installation teams attempted to collect diesel fuel and crankcase lubricating oil
samples on all pieces of equipment that received emissions measurements. Fuels samples were
gathered so that adjustments can be made to the emission measurements based on fuel properties
(i.e density, C/H ratios, etc). Oil samples were taked because they might be able to determine
the engine status or wear. All fuel and oil samples which were collected were stored and shipped
in appropriate containers provided by EPA.
Phase 1 fieldwork, which was performed in the county of Jackson, Missouri, began June
4th 2007 and continued through July 24th, 2007. At the completion of ES Phase Ifieldwork, the
team revised the Emissions and Activity Measurement sections of the QAPP (as well as the
associated SOPS) based on field-testing experience. SOPs were continually revised and
redistributed throughout each phase of testing as procedural and equipment refinements were
made.
Overview-11
-------
Phase 2 Surveys and Field Testing in PSUs 2 and 3
According to our study design, Phase 2 was to proceed with conducting the establishment
sample EOT with PSUs 2 through 5. However, partly due to the skewed nature of the
establishments according to the measure of size (MOS) and also due to the lower than expected
amount of eligible pieces of equipment to test, a full integration of the Phase 2 Establishment and
Equipment Samples (i.e., the EOT & ESI) into a single, unified design was performed. In
addition, due to the lower than anticipated yield rate of establishments, it was discovered that a
census of all construction establishments in the region would be needed in order to achieve the
goals of the study. Therefore, the study design was modified to integrate the establishment and
equipment sample into a single, integrated interview process, as shown in Table OV-4. This
table shows the revised sample size goals by showing censuses conducted for Inventory and
Instrumentation for Phases II (PSUs 2&3) and III (PSUs 4&5). As shown in Table OV-4, a total
of 3541 selections were expected to be needed to complete the EOIs for the Establishment
Sample and the EOI portion of the Equipment Sample.
Table OV-4 Revised Total Sample Needed to Achieve Sample Targets
TYPE OF DATA
COLLECTION
EOIs
Inventory &
Instrumentations
ESTABLISHMENT SAMPLE
PHASE 1
243
0
PHASE 2
n/a
0
EQUIPMENT SAMPLE
PHASE 1
404
37
PHASE 2
1522
74
PHASE 3
1372
74
TOTAL
3541
185
With the exception of some changes made to establishment eligibility criteria (relaxing
eligibility in an effort to increase recruitment yield), the process for the integrated sample
mirrored the establishment sample process for the first PSU. However, following the initial EOI
survey, the extended recruitment interview was performed with all eligible establishments in
order to select a site at which an in-field equipment inventory would be conducted and
equipment selected for instrumentation.
Field testing in PSUs 2 and 3 mirrored that performed in PSU1, but targeted
establishments located in the counties of Linn and Scott, Iowa. Fieldwork in PSUs 2 and 3
began September 5, 2007 and continued through October 27th, 2007.
Study Enhancements Made Prior to Phase 3
Overview-12
-------
As a pilot project, the design strategy was to capitalize on experience with previous
phases of the project and build upon that experience before proceeding with the subsequent
phase. Previous enhancements to the study design included revising eligibility criteria3 and
integrating the establishment sample EOT and equipment sample ESI into a single survey
application. As a result of the EOT Phase 2 effort, in order to meet data collection goals, a
decision was made to explore the viability and utility of drawing a supplemental sample from a
database provided by Equipment Data Associates, Inc. (EDA).4 To that end, EDA data was
purchased using the same specifications employed by EPA in its earlier acquisition of EDA data
for PSU 1 (Jackson, MO). The data set structure was organized and merged with the existing
SSI data set. The result of this assessment was a merged data set of 2,209 records, comprising
the sample frame for Phase 3. Accordingly, all 2,209 records in the combined SSI-EDA frame
were loaded into the sample management system for the issuance of advance letters and
subsequent calling by the telephone facility.
In addition to enhancements made to the establishment sample frame, data collection
equipment and procedures were refined prior to ES Phase 3 fieldwork, based on information
learned during ES Phase 1 and II testing. The team considered several changes, but eventually
focused on enhancing PEMS RPM collection and PEMS ECU data collection.
For PEMS RPM collection enhancements, the team abandoned use of "Capelec" RPM
measurement devices (which were the primary RPM measurement device used during PEMS
tests in ES Phases 1 and 2 of this study) as the method of RPM collection during PEMS testing.
This decision was made because only limited success was achieved obtaining an accurate and
reliable RPM signal. Instead, the ERG team and EPA decided to use optical sensors for PEMS
RPM acquisition during ES Phase 3 testing. The ERG team and EPA worked together to
identify dedicated optical sensor holders which could be attached to high-powered rare-earth
magnets (acquired separately). These high-powered mounts proved to be capable of securely
attaching the optical sensors for both day-long PEMS testing and month-long PAMS testing.
This provided a much more reliable RPM signal during the third phase of fieldwork.
In addition to the RPM collection enhancement, the team focused efforts on
supplementing PEMS test procedures and equipment in order to allow the collection of
3 Certain eligibility criteria were revised, including establishments that (according to the SSI sample frame) reported
having zero employees were no longer excluded from the study as a result of Phase 1 and establishments that were
non-prime contractors were considered eligible during Phase 2.
4The EDA data provide a list of establishments that have financed construction equipment purchases. The data set
contains identifying company information, equipment pieces financed (by equipment type) and date of transaction.
Overview-13
-------
Caterpillar ECU data from electronically-controlled equipment that would be PEMS-tested
throughout the remainder of this work assignment. Therefore, ERG, EPA and Sensors worked
with Caterpillar in acquiring "CAT ET" (Caterpillar Electronic Technician) equipment and
software necessary for ECU data collection on nonroad equipment, and the project team
established training sessions with Caterpillar for use of this equipment and software. New
procedures were developed to allow this data to be collected in the field during emissions testing
(rather than collection of data during engine diagnosis or repairs, the typical application for this
equipment). The CAT ET software was installed on a remote laptop for which the "sleep" power
mode was disabled (allowing the laptop to operate with the lid closed). This laptop was placed
in the cab of the equipment being tested, connected directly to the ECU CAN port (via the Cat
ET communication module) and to a power supply (taken from the PEMS onboard generator).
Since acquisition could not start until after the equipment was turned on, remote control of the
acquisition laptop was necessary and was achieved by way of a standard Wi-Fi computer
transmitter/receiver. Preliminary testing showed this configuration adequate for remotely
collecting ECU data from electronically-controlled Caterpillar equipment. This data could then
later be merged and time-aligned with the PEMS data for the same test.
Phase 3 Surveys and Field Testing in PSUs 4 and 5
Phase 3 EOIs and Equipment Sample recruitment were performed with the integrated
sample and merged SSI-EDA frame. Phase 3 fieldwork was conducted with establishments
located in the counties of Clay, Missouri and Shawnee, Kansas. Because of the distance which
separated these two counties, inventories, emissions and activity measurements were conducted
independently in each county (work for each task was completed in Clay, Missouri before
moving on to Shawnee, Kansas). Phase 3 fieldwork began June 30th 2008 and continued through
October 10th 2008. The quality of RPM data collected during ES Phase 3 was higher than that in
ES Phases 1 and 2, and although only one piece of electronically-controlled Caterpillar
equipment was tested, ECU data collection was successful for this test. Several weeks of CRC-
funded PEMS testing were added to the scope of work during ES Phase 3, increasing the total
number of PEMS tests collected during the study, as shown in the next section.
Summary of EOI Surveys and Equipment Sample Recruitment
A hierarchical disposition analysis was conducted using the survey data to determine the
data collection performance and instrumentation recruitment rates throughout the study.
Specifically, the analysis provided documentation of the screening rate, eligibility rates,
interview rates, and overall response rates for each of the study phases. The response rates that
Overview-14
-------
had been expected during the planning phase are shown in Table OV-5 below. In Table OV-5,
an "eligible establishment" was an establishment which had verified that it was the establishment
that had been selected for the sample, operated in the construction industry, used diesel powered
equipment and employed one or more persons (although the "one or more persons employed"
eligibility requirement was later eliminated in an effort to yield more Equipment Sample). If a
determination was made regarding eligibility, an establishment was "screened". If insufficient
information was obtained to determine eligibility; the establishment disposition was "not
screened."
Table OV-5 Expected Dispositions for each Sample Frame and Study Phase
Screening Response
Interview Response
Overall Response*
Eligibility Rate
Total Sample
Total Interviewed
(EOI)
Total Agreeing to
Instrumentation
Instrumentation
Response/ Total
ESTABLISHMENT
SAMPLE
PHASE 1
PSU1
75%
85%
64%
85%
304
100
NA
NA
PHASE 2
PSU2-5
75%
85%
64%
85%
1214
400
NA
NA
EQUIPMENT SAMPLE
PHASE 1
PSU1
75%
85%
26%
75%
318
93
37
40%
PHASE 2
PSU2+3
75%
85%
26%
75%
635
185
74
40%
PHASE 3
PSU 4+5
75%
85%
26%
75%
635
185
74
40%
* For the Equipment Sample, the overall response rate assumes that 40% of the Establishment Sample would agree to
instrumentation. We obtain the overall Equipment Sample response rate by multiplying the overall Establishment Sample
response rate by 40%. Thus, 64% x 40% = 26%.
Table OV-6 shows the actual performance rates for the EOI portion of the study. From
the onset of the study, it is clear that actual dispositions were with few exceptions lower than
originally expected. A significant reason for this was the lower-than-expected eligibility rate.
In response to this, in subsequent phases of the study (Phase 2, integrated sample, and Phase 3),
the eligibility requirements were revised to increase the eligibility rate. Tables ES-6 and ES-7
illustrate the impact of this.
Overview-15
-------
Table OV-6 Actual Performance Rates for EOI, All Phases of Integrated Sample
Screening Response
Interview Response
Overall Response
Eligibility*
Total Sample
Total Interviewed
EOI PHASE 1
PSU1
58%
94%
54%
38%
304
162
EOI PHASE 2
PSU1
50%
70%
35%
15%
2015
101
PSU 2+3
60%
87%
53%
14%
1453
107
EOI PHASE 3
PSU 4+5
28%
100%
28%
31%
2048
179
* Eligibility criteria were revised for each 'column' of data collection shown above.
The overall performance rates for Equipment Sample recruiting are presented in Table
OV-7 below.
Table OV-7 Actual Performance Rates for ESI, All Phases of Integrated Sample
Screening Response
Interview Response
Overall Response
Eligibility
Total Sample
Total Agreed to Inventory
Estabs Inventoried
Estabs Instrumented
EOI PHASE 1/2
PSU1
36%
28%
10%
9%
2319
22
11
7
EOI PHASE 2
PSU 2+3
60%
37%
23%
14%
1453
43
30
9
EOI PHASE 3
PSU 4+5
28%
35%
8%
31%
2048
54
38
13
Summary of PEMS and PAMS Data Collected Throughout Study
Table OV-8 provides counts of the numbers of establishments that were originally
recruited for inventories, establishments which were inventoried, establishments which were
recruited but then refused inventories, and establishments which were not inventoried for reasons
Overview-16
-------
other than establishment refusal, such as all sites being outside the study area or the
establishment would have no active sites until after the end of the phase.
Table OV- 8 Summary Counts of Establishments Inventoried
ES
Phase
1
2
3
Totals
Establishments
Recruited for
Inventories
22
43
54
119
Establishments
Inventoried
11
30
38
79
Establishments
Refusing
Inventories
7 (32%)
1 1 (26%)
12 (22%)
30 (25%)
Establishments Not
Inventoried for
Other Reasons
4
2
4
10
As can be seen in Table OV-8, approximately 25% of the establishments who originally
agreed to participate in the inventory and measurement phase of the study later reversed their
decision and declined to participate any further in the study. Some of these were categorized as
"passive" refusals, i.e., field inventory teams were never able to reach a contact, or were given
extraordinarily unusual reasons that participation was not possible at that time.
Table OV-9 provides counts of equipment inventoried and instrumented throughout the
study.
Table OV-9 Summary Counts of Equipment Inventoried and Instrumented
Count of equipment inventoried
Count of PEMS-eligible equipment
Count of PAMS installations
Count of PEMS installations
Overall
292
179
30
40
ES
Phase 1
56
41
7
6
ES
Phase 2
110
65
11
13
ES
Phase 3
126
73
12
21
Thirty-five of the 119 establishments that were inventoried were also asked to participate
in instrumentation (PAMS, PEMS, or both). It is interesting that only 6 of those 35
establishments refused to participate in the instrumentation process after the inventory. In Table
OV-9, PEMS eligibility was generally based on whether sufficient room was available for
securing the PEMS rack, as approximately 4 ft by 3 ft (footprint) was required to mount the rack.
In addition, the PEMS rack could not be mounted on equipment where it would hinder work or
pose a visibility or safety hazard.
Overview-17
-------
Table OV-10 provides a summary of PEMS and PAMS data that was collected throughout all
three phases of study fieldwork. As previously noted, Phase 3 PEMS tests were enhanced with
several additional weeks of CRC-funded testing.
Table OV-10 Summary of data collected throughout the Nonroad PEMS Study
ES Phase / PSU
ESPh 1, Summer 07
Jackson, MO (PSU 1)
Es Ph 2, Fall 07
Linn, IA (PSU 2) &
Scott, IA (PSU 3)
ES Ph3, Summer&Fall 08
Clay, MO (PSU 4) &
Shawnee, KS (PSU 5)
Totals
PEMS
Target
PEMS
5
10
16
31
PEMS
Attempts
6
13
21
40
PEMS Successful
Gaseous
3
10
16
29
PM
2
10
15
27
RPM
1
9
15
25
PAMS
Target
5
10
10
25
Install
Attempts
7
11
12
30
Data
Collected
7
10
12
29
As shown in Table OV-10, a higher number of PEMS and PAMS "attempts" were made than
"target" test counts, in order to account for loss of data to equipment malfunctions and other
problems.
Data Processing and QC
At the completion of EOI/ESI data collection, NuStats processed the establishment
interview and recruiting database, conducting quality control and edit checks, using
specifications established for this study. Study data was analyzed regarding sample
performance. Final data files were prepared and transferred to ERG according to the protocols
required in the statement of work.
All PEMS and PAMS data was monitored during collection followed by extensive QC
and processing performed after data collection was completed. For PEMS data, QC and analysis
included time-alignment, RPM scaling, estimation of brake-specific emissions based on engine
RPM and fuel rate, review of analyzer gaseous drift, review of exhaust mass flow rates, review
of MPS proportionality to exhaust flow, and a detailed second-by-second review of all recorded
parameters by way of analysis of plots and raw data. This second-by-second review entailed
evaluation of all gaseous pollutants, review of RPM quality, reviewing sampling system
pressures such as the MPS inlet pressure and SEMTECH pressures, evaluating all system flows
including the exhaust mass flow rates, review of the calculated fuel flow rate, MPS sampling
Overview-18
-------
flowrates and gravimetric filter flowrates, and evaluating all system and sampling temperatures
such as exhaust temperatures, external heated line, chiller, cyclone, manifold and gravimetric
filter temperatures and ambient and internal PEMS rack temperatures. Corrections were applied
to the data as needed and uncorrectable or erroneous data has been excluded from reporting
summaries. In addition, a number of sources of uncertainty in the emissions results are described
in Section 6.2.2 of this report and have been quantified in Appendix AO. These uncertainty
estimates are shown in the emission results presented in this overview.
PAMS data processing varied from test to test depending on the type of installation,
PAMS equipment used and equipment being tested. In general, review and correction of PAMS
data involved reviewing and correcting date / time stamp assignments (including on dataloggers
with sub-second acquisition), performing RPM calibration corrections, flagging observations that
could be associated with ERG team activities (installation, removal and revisit dates), assigning
equipment activity and key-position flags, assigning RPM validity flags and defining a "correct"
RPM for each test.
Study Results
Phase 1 EOI and Recruiting Results
Analysis of Phase 1 EOIs showed that the distribution of construction establishments was
highly skewed with respect to number of employees much in the way that most business
productivity and revenue distributions are distributed in the U.S. a relatively small subset of
establishments account for the majority of productivity or revenue. Such was the case with the
employee distribution across construction establishments in PSU 1: the distribution pattern
approximately followed a Pareto distribution (i.e., about 20% of establishments accounted for
roughly 80% of employees). If the correlation assumption between employees and equipment
held up, then the observed distribution of establishments in the sampling frame would support
the use of PPS sampling.
For PSU 1, we encountered a large number of self-representing/certainty establishments.5
Table OV-11 shows the frequency and percentage distributions of establishments and number of
employees in our PSU 1 frame by self-representing status.
5 Self-representing establishments in PSU1 were those for whom SSI reported 15 or more employees; non self-
representing establishments had 1-14 employees.
Overview-19
-------
Table OV-11 Frame Characteristics of PSD 1
Self-representing
SSUs
Nonself-representing
SSUs
Excluded
establishments
(0 employees)
TOTAL
NUMBER OF
ESTABLISHMENTS
267
2,052
77
2,396
PERCENT OF
ESTABLISHMENTS
11%
86%
3.2%
100%
NUMBER OF
EMPLOYEES
18,248
6,317
0
24,565
PERCENT OF
EMPLOYEES
74%
26%
0
100%
The need for self-representing units in the PSU 1 sample design precipitated a major
modification in the overall design. This was because a large number of establishments (i.e., the
self-representing units) belonged to both the Establishment and Equipment Samples by virtue of
their large measures of size. This meant that all self-representing SSUs needed to be taken
through the both the EOT and ESI survey protocols (e.g., EOT, recruitment for inventory and
instrumentation, incentive experiment, etc.).
Table OV-12 lists the various response rates by stratum for EOT (interviews) in Phase 1.
The net effect is seen as the last row of Table OV-12 - Net Yield. Net Yield refers to the bottom
line percentage of the sample that will yield a completed interview. Using our design parameters
a net yield of 41% was expected. The actual yield for EOT Phase 1 Establishment Sample was
21%, or about half of what we planned. Net yield varied twofold by PSU status - 33% for self-
representing units and 16% for non-self-representing units. Both figures fell well below the
expected/planned value of 41%.
Table OV-12 PSU 1 Expected vs. Actual Design Parameters
DESIGN
PARAMETER:
eligibility rate
screen rate
interview rate
overall response
rate
Net Yield
ACTUAL
SELF REP
51%
72%
89%
64%
33%
ACTUAL
NONSELF-REP
32%
51%
97%
49%
16%
OVERALL
ACTUAL
40%
58%
93%
54%
21%
OVERALL
EXPECTED
65%
75%
85%
64%
41%
The EOT Phase 1 Interviews were successful in that they achieved the primary goal of
preparing for the remainder of the Study. The results are summarized as follows:
The distribution of construction establishments in the sampling frame was highly
skewed with respect to number of employees (generally following a Pareto
distribution 20% of establishments acct for 80% of employees), and the sample
and data collection designs were adapted accordingly;
Overview-20
-------
If the number of employees is to be used as a measure of size for sampling
establishments, then within-PSU sampling of establishments requires a two-step
approach. First, identify a set of "self-representing" establishments; second,
subsample the non-self-rep establishments;
The overall response rates were generally favorable; however there was
considerable variation by self-representing and non-self representing response
rates;
Eligibility was considerably lower than planned in our design parameters, and this
was differential by PSU; this suggested that higher levels (than planned/budgeted)
of screening and calling would be required per completed EOT; under a model of
fixed level of resources, the target number of completed interviews would need to
be reduced;
There is no efficient way (short of calling) to identify ineligible sample; however,
the screening response rate can be increased by adopting a protocol that requires a
nominal amount of research of the disconnected numbers. This would verify that
there are no other listings for that establishment and/or that all additional listings
are disconnected or 'wrong numbers' and a conclusion could be drawn that the
establishment is no longer in business (which in turn helps the screening response
rate).
Phase 2 EOI and Recruiting Results
Analysis of the remainder of the PSU 2-5 Frame data for the combined Establishment and
Equipment samples indicated that even performing a census of all establishments in PSUs 2-5
might fail to yield the recruitment goals for the Equipment Sample. This potential shortfall,
along with the need for self-representing establishments to be taken through both the EOI and
ESI survey protocols, led to a full integration of the EOI Phase 2 Establishment and Equipment
Samples into a single, unified design.
Table OV-13 compares the PSU 2 and 3 recruiting rates with the expected values used
for planning. The last row shows that the actual net yield was under that anticipated by a factor
of 7. The rightmost column shows the ratios of actual-to-expected rates and most are relatively
near an ideal value of 1.0. However the biggest departure is due to the discrepancy in eligibility
rates. The actual eligibility rate was 14% while the planned value was 75 percent. This factor
alone represented a lower-than-expected net yield by a factor of 5.4.
Overview-21
-------
Table OV-13 PSU 2 and 3 Actual Versus Expected Design Parameters
Screening Rate
Eligibility Rate
Recruitment Rate
Overall Response rate
COMBINED PSU2& 3
ACTUAL
60%
14%
37%
23%
EXPECTED
75%
75%
34%
26%
Actual increase relative to Expected
RATIO
(EXPECTED/ACTUAL)
1.3
5.4
0.9
1.1
7
Phase 3 EOI and Recruiting Results
Table OV-14 presents actual and expected screening response rates and eligibility rates;
also, interview response rates are provided separately for EOI and ESI. All rates are reported by
PSU (Columns A and B) and for the overall sample (Column C), along with the corresponding
expected values that were used in planning (Column D).
Table OV-14 PSU 4 and 5 Actual Versus Expected Design Parameters
EOI PHASE 3
Screening response
Eligibility Rate
EOI Interview Response
ESI Interview Response
Overall Response EOI
Overall Response - ESI
ESI Net Yield (1 in N)
A
PSU 4
27%
33%
100%
31%
27%
8%
35.4
B
PSU 5
29%
29%
100%
29%
29%
8%
41.0
C
Total
Actual
28%
31%
100%
30%
28%
8%
37.9
D
Expected
75%
75%
85%
40%
64%
26%
5.2
The values in columns A-C of Table OV-14 indicate that the screening, eligibility and
response rates were consistent across PSUs 4 and 5. Columns C and D can be used to compare
actual and expected response and eligibility rates for EOI Phase 3. Actual screening response
rates were substantially below the expected/planned value (28% actual vs. 75% expected). This
was due to a number of reasons including the quality of the business contact data, the need to
conduct additional searches to obtain business information and the need for multiple call backs to
businesses to reach a knowledgeable contact.
Even among the successfully screened establishments, the actual eligibility rate was less
than half of the expected rate (31% actual vs. 75% expected). This was disappointing in that the
eligibility criteria had been loosened (e.g., companies with 0 employees were eligible provided
Overview-22
-------
they met other criteria, removing the restriction that establishments be prime contractors). The
lower-than-expected eligibility rate led to higher screening burden to identify eligible
establishments (requiring 2.4 times the expected amount of screening than what was planned).
Incentive Test Analysis Results
The incentive tests for each of EOT Phases 1 and 2 were not definitive and a decision was
made to continue the testing in EOT Phase 3. The design of the test involved a random
assignment to an incentive offering to establishments just before commencing the ESI portion of
the survey. The incentive was an offer of a $100 check sent to the establishment regardless of
their participation in the ESI (recruitment). The experimental design called for a random half of
respondents to receive the incentive.
The results of the incentive experiment are presented in Table OV-15. The results
combine PSUs 4 and 5 for the sake of parsimony (since the separate tables show the same result).
A total of 171 establishments participated in the incentive test. To "participate" in the incentive
test, the introduction must be read to the respondent. The results show that the incentive had no
observed impact on accepting the invitation to participate in the inventory and instrumentation.
With one degree of freedom, a chi-square statistic whose value is 2.71 or greater would have
been needed to establish a 10% level of significance; the observed value was 1.98.
Table OV-15 Combined PSU 4 and 5 Incentive Test on Instrumentation
Experimental Group:
Incentive
No incentive
Total
Chi-Squared
OUTCOME
Recruited
33
21
54
1.98
Declined
58
59
117
Total
91
80
171
NOT Signif. at 10%
The possibility that the incentive offer might impact actual participation in the
inventory/instrumentation even if we failed to detect a treatment effect at the recruitment stage
was explored. That is, follow-through to instrumentation as the outcome instead of agreement
to participate was explored because some establishments agreed during the CATI interview but
then later declined when the reality of inventory/instrumentation was at hand.
Table OV-16 presents the results of the incentive test where the outcome is the actual
follow-through to instrumentation. Unfortunately, the incentive did not show a significant
impact on instrumentation follow-through. The results here are striking contrast to the incentive
Overview-23
-------
test results for PSUs 2 and 3 (EOT Phase 2). In EOT Phase 2 there was a highly significant
incentive effect detected for follow-though to instrumentation.
Table OV-16 Combined PSD 4 and 5 Incentive Test on Instrumentation Follow-
Through
Experimental Group:
Incentive
No incentive
Total
Chi-Squared
OUTCOME
Instrumented
14
10
24
0.29
Declined
77
70
147
Total
91
80
171
NOT Signif. At 10%
To explore the EOT Phase 2 and 3 results we combined the EOT Phase 2 and 3 incentive
tests to see if the incentive effect remained. Table OV-17 shows the results of that analysis. We
see that the incentive effect remains highly significant when the outcome measure is follow-
through to instrumentation.
Table OV-17 Combined PSUs 2-5 Incentive Tests on Instrumentation Follow-
Through
Experimental Group:
Incentive
No incentive
Total
Chi-Squared
OUTCOME
Instrumented
34
17
51
5.41
Declined
116
122
238
Signif. At 2.5% level
Total
150
139
289
These findings are insightful. First, it is clearly more important to generate follow-
through to instrumentation rather than assent at the recruitment stage. As such, we recommend
that future research focus on this as the outcome of interest/treatment effect. Secondly, the
findings from Tables ES-16 and ES-17 are decidedly mixed. There is a very strong incentive
effect from EOT Phase 2 when combining the data from EOT Phases 2 and 3. However, the
absence of a treatment effect in EOT Phase 3 is troubling and therefore the EOT Phase 2-3
incentive experience begs further analysis and investigation. Clear, significant incentive effects
would have conclusively led to a recommendation to adopt incentives in all future studies of this
type.
These mixed results give pause to a wholehearted acceptance of incentives. What could
lead to such outcomes? The following are some possible explanations:
Overview-24
-------
Site effects: it could be that PSUs 2-3 contain fundamentally different
establishments than those of PSUs 4-5, although it is hard to believe that the
effect is due to the peculiarities of establishments within PSUs.
Interviewer effects: there may have been a difference in the composition of the
field interviewer staff between EOT Phases; this could occur if, say, in EOT Phase
2 a highly experienced interviewer staff was used but less experienced
interviewers were used in EOT Phase 3.
Incongruent samples: EOT Phase 3 sampling involved a census of construction
establishments regardless of employee size; even establishments showing zero
employees (according to SSI) were fielded; this was not the case for EOT Phase 2,
where zero-employee establishments were not sampled and if identified in the
EOT were terminated; differences in sample universe make-up might explain the
incentive results.
The conclusion we reach from these findings is that incentives are cautiously
recommended. If at all possible incentive testing should be continued in a way that helps inform
our understanding of what factors are associated with differential instrumentation.
PEMS Results
Figures ES-1 through ES-14 present PM and gaseous emissions on a "brake specific" or
mass / work basis (in units of grams or kg per kW-hr), by equipment category. PM emissions are
based on the first three filters collected, and gaseous emissions are based on the overall test
average (including times when filters were and were not sampled). When reviewing these
results, it should be noted that these emission results were collected during real-world operation
and may be biased due to extensive idle or low engine speed operation. Due to differences in
work cycles, direct comparisons should not be made between emission standards and the
emission results presented here.
Overview-25
-------
Figure OV-1 PM Emissions from Backhoe Loaders, Work Basis, Filters 1 - 3
I
"S3
c
o
'
Backhoe Loaders
I Filter 1
I Filter 2
Filters
'95 Deere 410D
(75 hp)
'06 Deere 310G
(Tier 2)
(84 hp)
'07 Deere 310J
(Tier 2)
(84 hp)
Figure OV-2 Gaseous Emissions from Backhoe Loaders, Work Basis, Overall test
Backhoe Loaders
'95 Deere 410D 06 Deere 310G* '07 Deere 310J
(75 hp) (Tier 2) (Tier 2)
(84 hp) (84 hp)
*NoCO or CO2 results due to NDIR signal loss on this test
lHC(g/kW-hr)
lCO(g/kW-hr)
lCO2(kg/kW-hr)
IIMox(g/kW-hr)
Overview-26
-------
Figure OV-3 PM Emissions from Dozers, 50-99 hp, Work Basis, Filters 1-3
Dozers, 50 hp-99 hp
I Filter 1
I Filter 2
Filters
'96CatD4CXL
(87 hp)
'99 Deere 550H
(Tierl)
(84 hp)
Figure OV-4 PM Emissions from Dozers, > 100 hp, Work Basis, Filters 1 - 3
^£
"SB
c
o
Dozers, > 100 hp
Filter 1
'88 Cat 953 '95Cat963CB '97CatD6RXL 99Cat953C* '98Cat963B
(121 hp) (160 hp) (Tierl) (Tierl) (Tierl)
(175 hp) 170 hp) (220 hp)
* PM results invalid for Filter #3 on this test
Overview-27
-------
Figure OV-5 Gaseous Emissions from Dozers, 50-99 hp, Work Basis, Overall test
3
o
100 hp, Work Basis, Overall test
3
O
OJ
V)
TO
(3
12
10
8
6
Dozers, > 100 hp
HC(g/kW-hr)
CO(e/kW-hrl
'85Cat963 '88Cat953 '95 Cat 963CB'97 Cat D6RXL 99 Cat 953C* '98 Cat 963B
(150 hp) (121 hp) (160 hp) (Tierl) (Tierl) (Tierl)
(175 hp) 170 hp) (220 hp)
*NoCO or C02 results due to NDIR signal loss on this test
Overview-28
-------
Figure OV-7 PM Emissions from Excavators, < 300 hp, Work Basis, Filters 1 - 3
Excavators, < 300 hp
I Filter 1
I Filter 2
Filters
'85 Case 1085B
(120 hp)
'97Cat320B
(Tierl)
(128 hp)
'03 Komatsu PC300LC
(Tier 2)
(255 hp)
Figure OV-8 PM Emissions from Excavators, > 300 hp, Work Basis, Filters 1 - 3
Excavators, >300hp
I Filter 1
I Filter 2
Filters
'93 Komatsu PC400LC '00 Komatsu PC400LC '06Cat325D
(330 hp) (Tierl) (Tier 3)
(321 hp) (300 hp)
Overview-29
-------
Figure OV-9 Gaseous Emissions from Excavators, < 300 hp, Work Basis, Overall
Test
Excavators, < 300 hp
lHC(g/kW-hr)
I CO (g/kW-hr)
C02(kg/kW-hr)
|Nox(g/kW-hr)
'85 Case '97Cat320B '98Komatsu '03 Komatsu
1085B (Tierl) PC300LC PC300LC
(120 hp) (128 hp) (Tierl) (Tier2)
(232 hp) (255 hp)
Figure OV-10 Gaseous Emissions from Excavators, > 300 hp, Work Basis, Overall
Test
c
o
o
OJ
V)
TO
ID
Excavators, > 300 hp
lHC(g/kW-hr)
iCO(g/kW-hr)
!C02(kg/kW-hr)
|Nox(g/kW-hr)
'93 Komatsu '00 Komatsu '06Cat325D '06 Deere
PC400LC PC400LC (Tier 3) 450D
(330 hp) (Tierl) (300 hp) (Tier 3)
(321 hp) (349 hp)
Overview-30
-------
Figure OV-11 PM Emissions From Loaders, Work Basis, Filters 1 - 3
Loaders
'04Cat953C
(Tier 2)
(128 hp)
' PM results invalid for Filters #1 & #3 on this test
Figure OV-12 Gaseous Emissions From Loaders, Work Basis, Overall test
Loaders
'97Case Komatsu '02 Cat963'03 Deere '04Cat
570LXT WA180 (Tierl) 544H 953C
(68 hp) (124 hp) (160 hp) (Tier 2) (Tier 2)
(130 hp) (128 hp)
IHC(g/kW-hr)
!CO(g/kW-hr)
IC02(kg/kW-hr)
INox(g/kW-hr)
Overview-31
-------
Figure OV-13 PM Emissions From Other Equipment, Work Basis, Filters 1 - 3
o
1
E
8
7
6
5
4
3
2
1
0
Other Equipment
I Filter 1
Filter2
I Filters
'87 Cummins4B-3.9
(76 hp)
96 Cat 12H*
(140 hp)
'02 CatTH83
(Tierl)
(101 hp)
* PM results invalid for Filterttl on this test
Figure OV-14 Gaseous Emissions From Other Equipment, Work Basis, Overall
Test
o
OJ
V)
TO
ID
16
14
12 -
10 -
6 -
4 -
2
0
Other Equipment
'87Cummins4B-3.9
(76 hp)
'02CatTH83
(Tierl)
(101 hp)
lHC(g/kW-hr)
iCO(g/kW-hr)
lC02(kg/kW-hr)
lNox(g/kW-hr)
Overview-32
-------
Figures OV-15 through OV-28 present PM and gaseous emissions on a fuel basis (grams
or kg of emissions per gallon of fuel consumed), rather than a work basis, again grouped by
equipment category. PM emissions are based on the first three filters collected, and gaseous
emissions are based on the overall test average (including times when filters were and were not
sampled).
Using a fuel basis to evaluate emissions eliminates several of the points of uncertainty
inherent in work basis estimates. However, as with the work basis emissions, extensive idle
periods can have an influence on the fuel-based emissions, and the accuracy of the PM
measurements are still dependent on the performance of the micro-proportional sampler
throughout the range of operation (including outside of the NTE zone). In addition, any errors
associated with the SEMTECH-DS' determination of the second-by-second (and hence
cumulative) fuel consumption rate will affect the accuracy of the fuel-based emissions estimates.
These errors have been estimated in Appendix AO, Nonroad Error Estimates, and are shown in
Figures OV-15 through OV-28.
Overview-33
-------
Figure OV-15 PM Emissions From Backhoe Loaders, Fuel Basis, Filters 1 - 3
25
20
S
T 15
to
'wi
W)
E 10
BackhoeLoaders
I Filter 1
I Filter 2
Filters
'95 Deere 410D
(75 hp)
'06 Deere 310G
(Tier 2)
(84 hp)
'07 Deere 310J
(Tier 2)
(84 hp)
Overview-34
-------
Figure OV-16 Gaseous Emissions From Backhoe Loaders, Fuel Basis, Overall
Test
300
BackhoeLoaders
'95 Deere 410D 06 Deere 310G* '07 Deere 310J
(75 hp) (Tier 2) (Tier 2)
(84 hp) (84 hp)
*NoCO or C02 results due to NDIR signal loss on this test
lHC(g/gal)
lCO(g/gal)
C02 (kg/gal)
lNox(g/gal)
Figure OV-17 PM Emissions From Dozers, 50-99hp, Fuel Basis, Filters 1-3
Dozers,50hp-99 hp
'96CatD4CXL
(87 hp)
'99 Deere 550H
(Tierl)
(84 hp)
I Filter 1
I Filter 2
Filters
Overview-35
-------
Figure OV-18 PM Emissions From Dozers, > 100 hp, Fuel Basis, Filters 1 - 3
Dozers, >100 hp
FiltPi-1
'88 Cat 953 '95Cat963CB '97CatD6RXL 99Cat953C* '98Cat963B
(121 hp) (160 hp) (Tierl) (Tierl) (Tierl)
(175 hp) 170 hp) (220 hp)
* PM results invalid for Filter#3 on this test
Figure OV-19 Gaseous Emissions From Dozers, 50-99 hp, Fuel Basis, Overall
Test
250
200
m 150
o
13
100
Dozers,50hp-99 hp
'96CatD4CXL*
(87 hp)
'99 Deere 550H
(Tierl)
(84 hp)
* No HC results due to FID signal loss on this test
lHC(g/gal)
lCO(g/gal)
C02 (kg/gal)
INox(g/gal)
Overview-36
-------
Figure OV-20 Gaseous Emissions From Dozers, > 100 hp, Fuel Basis, Overall
Test
(fL
c
o
a
E
o
9!
re
Dozers, >100 hp
'85 Cat 963 '88 Cat 953 '95 Cat 963CB'97 Cat D6RXL 99 Cat 953C* '98Cat963B
(150 hp) (121 hp) (160 hp) (Tierl) (Tierl) (Tierl)
(175 hp) 170 hp) (220 hp)
*NoCO or C02 results due to NDIR signal loss on this test
Figure OV-21 PM Emissions From Excavators, < 300 hp, Fuel Basis, Filters 1 - 3
Excavators, <300hp
I Filter 1
I Filter 2
Filters
'85 Case 1085B
(120 hp)
'97Cat320B
(Tierl)
(128 hp)
'03 Komatsu PC300LC
(Tier 2)
(255 hp)
Overview-37
-------
Figure OV-22 PM Emissions From Excavators, > 300 hp, Fuel Basis, Filters 1 - 3
c
o
Excavators, >300hp
I Filter 1
I Filter 2
Filters
'93 Komatsu PC400LC
(330 hp)
'00 Komatsu PC400LC
(Tierl)
(321 hp)
'06Cat325D
(TierS)
(300 hp)
Figure OV-23 Gaseous Emissions From Excavators, < 300 hp, Fuel Basis, Overall
Test
Excavators, < 300 hp
300
lHC(g/gal)
lCO(g/gal)
C02 (kg/gal)
INox(g/gal)
'85 Case '97Cat320B '98 Komatsu '03 Komatsu
1085B (Tierl) PC300LC PC300LC
(120 hp) (128 hp) (Tierl) (Tier 2)
(232 hp) (255 hp)
Overview-38
-------
Figure OV-24 Gaseous Emissions From Excavators, > 300 hp, Fuel Basis, Overall
Test
Excavators, > 300 hp
160
c
o
o
01
u)
ro
13
IHC(g/gal)
iCO(g/gal)
C02 (kg/gal)
INox(g/gal)
'93 Komatsu
PC400LC
(330 hp)
'00 Komatsu
PC400LC
(Tierl)
(321 hp)
'06Cat325D '06 Deere
(Tier3) 450D
(300 hp) (Tier3)
(349 hp)
Figure OV-25 PM Emissions From Loaders, Fuel Basis, Filters 1 - 3
Loaders
I Filter 1
Filter 2
I Filter 3
'03 Deere 544H '04Cat953C
(Tier 2) (Tier 2)
(130 hp) (128 hp)
* PM results invalid for Filters #1 & #3 on this test
Overview-39
-------
Figure OV-26 Gaseous Emissions From Loaders, Fuel Basis, Overall Test
Loaders
300
lHC(g/gal)
lCO(g/gal)
C02 (kg/gal)
iNox(g/gal)
'97 Case Komatsu '02 Cat 963 '03 Deere '04 Cat
570LXT WA180 (Tierl) 544H 953C
(68 hp) (124 hp) (160 hp) (Tier 2) (Tier 2)
(130 hp) (128 hp)
Figure OV-27 PM Emissions From Other Equipment, Fuel Basis, Filters 1 - 3
120
Other Equipment
I Filter 1
Filter 2
I Filter 3
'87Cummins4B-3.9
(76 hp)
96 Cat 12H*
(140 hp)
'02CatTH83
(Tierl)
(101 hp)
* PM results invalid for Filter #1 on this test
Overview-40
-------
Figure OV-28 Gaseous Emissions From Other Equipment, Fuel Basis, Overall
Test
Other Equipment
c
o
I/I
3
o
d
I/)
re
lHC(g/gal)
lCO(g/gal)
C02 (kg/gal)
lNox(g/gal)
'87 Cummins 4B-3.9
(76 hp)
'02CatTH83
(Tierl)
(101 hp)
PAMS Results
Table OV-18 lists total usage of the PAMS-instrumented equipment in hours, summed by
equipment category. In this table, "weekday days" refers to operation on Mondays through
Fridays, 7 am - 7:59 pm. "Weekday nights" operation is defined as operation on Mondays thru
Fridays, beginning at 8 pm each evening (Monday through Friday) and ending at 6:59 am the
following morning (Tuesday through Saturday). Weekend operation is defined as operation
beginning Saturday at 7 am and ending Monday at 6:59 am. These summed categories are
shown graphically in Figure OV-29. Figure OV-30 shows these same categories normalized to
percentages.
Table OV-18 Activity Summary by Equipment Category (in Hours)
Equipment
Category
Backhoe loader
Boring / Trenching
Dozer
Excavator
Loader
Overall
286.5
179.7
157.6
45.7
199.9
Weekday
days
268.1
179.6
150.0
35.5
169.3
Weekday
Nights
0.5
0.0
0.2
1.0
9.6
Weekend
Days/Nights
17.9
0.0
7.4
9.2
21.0
Overview-41
-------
Other
Telescope Forklift
72.7
107.6
67.6
58.9
0.1
1.7
5.1
3.7
Figure OV-29 Activity Summary by Equipment Category (in Hours)
Equipment Usage by Category
dWeekdays
Weekday Nights
QWeekend Days/Nights
n
Backhoe loader Boring /Trenching
Excavator
Equipment Category
Telescope Forklift
Overview-42
-------
Figure OV-30 Activity Summary by Equipment Category (in Percentages)
Equipment Usage by Category
DWeekdays
Weekday Nights
D Weekend Days/Nights
-n
Backhoe loader Boring/Trenching
Excavator
Equipment Category
Telescope Forklift
Study Conclusions and Recommendations
Sample Design and Recruitment
Sample frames: This study utilized Survey Sampling International (SSI) as the primary
sampling frame and tested the use of Equipment Data Associates (EDA) as a replacement or
supplemental frame. SSI remains a viable and productive sampling frame; however, as
discussed in Section 4.4, we recommend future considerations of the EDA frame as a
supplemental fame in a dual-frame design to increase coverage. The relative costs of processing
EDA and SSI sample need to be considered and analyzed before implementing this
recommendation.
Two stage sampling: Given the unexpected low prevalence of eligible establishments in
the pilot study, combined with the absence of correlation between data items on the SSI sampling
frame and the actual number of eligible equipment pieces for an establishment, we now believe
that in most if not all situations a census of establishments will be needed even to instrument a
small number of equipment pieces. If censuses are used, then issues of sample design within
PSUs become moot.
Overview-43
-------
However, there may be large metropolitan area such as New York City, Chicago or Los
Angeles where the number of establishments for sampling would far exceed that needed for this
type of study. In these cases we would resist the use of PPS sampling of establishments based on
our findings in this pilot. Instead, we would encourage the creation of a few strata based on
number of employees as follows: first exploit the skewed Pareto distribution of establishments
to create a "large stratum" (say all establishments in the top 20th percentile according to number
of employees), a "zero employee" stratum and a residual stratum. Based on our pilot study we
expect that the eligibility rates to be highest among the "large stratum" and lowest among the
"zero employee" stratum. This could lead to either a proportional allocation sample of
establishments, or a mild optimum (Neyman) allocation stratified sample that employs 'best
estimates' of eligibility rates across strata. But we would not recommend a PPS sample using
number of employees as a measure of size.
Use of incentives: While the incentive tests conducted during this study were
inconclusive, we cautiously recommend their use in future studies. Section 4.5 discusses a
number of explanations for this including possible site or interviewer effects and differences
among establishments. Clearly, it is more important to generate follow-through to
instrumentation rather than assent at the recruitment stage. As such, we recommend that future
research focus on this as the outcome of interest/treatment effect.
Establishment eligibility: Clearly, the number of eligibility requirements included in a
survey impacts eligibility and ultimately response rates and sample design. Revisions were made
to the questionnaire throughout the study to clarify issues such as fuel type (i.e., diesel versus
gasoline-fueled equipment) or prime versus subcontractor status (the requirement of being a
prime contractor was relaxed following Phase 1). Future studies with other industry sectors or
other geographic locations within the construction sector should incorporate modifications made
during this study. Appendix AK contains the survey questionnaires used throughout the study,
by EOI phase of study.
Survey instrument introduction: A number of enhancements were made to the survey
instrument introduction over the course of the study to reduce the likelihood that an
establishment would refuse to participate in the study at the onset of the interview. The
introduction should mention the Environmental Protection Agency and provide a very concise
one-sentence description of the study that does not allude to eligibility (allowing prospective
establishment to self-determine eligibility at the onset and giving them an easy way to opt out of
the survey). For example, rather than, "... we are conducting a study with construction
companies about the diesel equipment and machinery used in their daily operations" the
Overview-44
-------
following is preferable, "... we are conducting a study with companies about the equipment used
in their daily operations."
Advance letter: We recommend continuing the use of an advance letter with FAQ
brochure and endorsement from trade associations in future work, as these serves a critical
function of pre-notifying establishments about the study.
Fraction of establishments using diesel nonroad equipment: About 49 percent of the
establishments participating in the EOT Phases 2 and 3 survey use diesel non-road equipment.
Specifically, out of 1,099 establishments, 544 reported they have (1) at least one rented or leased
item of equipment or machinery that runs on diesel fuel or (2) at least 1% of equipment that run
on diesel.
Proportion of establishments employing at least one person on a part-time or full-
time basis: Data collected in the EOT Phases 2 and 3 suggest that approximately 82% of the
establishments in the targeted sectors employed at least one person on a part-time or full-time
basis during the previous twelve months. EOT Phase 1 data is excluded from this analysis
because zero-employee establishments were excluded from the sample drawn for EOT Phase 1.
Correlation of number of persons employed in a company and the amount of
eligible equipment: A regression analysis was employed to explore the correlation between the
number of employees in a company and the amount of eligible equipment. The findings indicate
that there was no significant correlation between these variables. As a result we decided to field
the 0-employee establishments for EOT Phases 2 and 3.
Variances in key variables (to inform sample size estimation for subsequent data
collection efforts): The descriptive statistics for the key variables are provided in Table OV-19.
However, we recommend caution in the use of these parameters for sample size estimation. For
instance, the number of paid employees is not related to equipment usage, so its use in the design
of a study on nonroad equipment has limited or no value. The number of equipment pieces is
useful but only available after data collection. It is not available prior to data collection.
Nonetheless, it could be useful in determining sample size for statistics where the establishment
is the unit of analysis. If the desired unit of analysis is nonroad diesel equipment, then the
clustered nature of our sample must be addressed, since equipment pieces are clustered within
establishments (as well as within work sites).
Overview-45
-------
Table OV-19 Descriptive Statistics for Key Study Variables
No. of paid employees
No. of diesel equipment pieces
VALID NO. OF
ESTABLISHMENTS
409
454
MIN.
0
1
MAX.
2300
3200
MEAN
42.07
24.39
STD.
DEVIATION
175.40
173.44
VARIANCE
30765.32
30082.61
Emissions and Activity Data
Reviewing data and emissions plots from this study suggests engine size and regulatory
tier may not be meaningful stratification variables for estimating emissions rates of diesel
engines, as emission rate variations seemed to be influenced less by these observational
parameters as by other parameters such as engine speed range and engine load. The type of work
being done (power-take off, equipment transport, or both) might be a good indicator of both
engine speed ranges and loads used and may be a good operational parameter to consider when
selecting stratification variables for future work. Although potentially hindered by the small
sample size, data collected during this study could be evaluated in order to identify appropriate
stratification variables for future studies, considering both observational and operational
parameters in a regression analysis. Such an analysis was beyond the scope of work in this
study.
With consideration of the small sample set of data collected, reviewing the PAMS usage
data does suggest the majority of equipment usage occurs during typical weekday hours. Some
types of equipment did appear to have higher night / weekend usage rates, although this could be
attributed in part to rain, mud and other conditions which prevented operations during typical
hours. Throughout the three ES phases of fieldwork, our experience does indicate that the type
of industry in which each establishment worked did have an effect on what days and times
equipment was operated. Type of work (and hence equipment type) may therefore be a good
indicator of hours of operation (days / nights / weekends). Work hours did appear to be fairly
consistent within establishments.
Overview-46
-------
1.0 Introduction
The Nonroad PEMS and PAMS study was a multi-year pilot study funded and supported
by the U.S. Environmental Protection Agency (EPA) and the Coordinating Research Council
(CRC) which was intended to refine methods of developing larger-scale estimations of
populations, usage and emissions of heavy-duty non-road diesel equipment in various economic
sectors. During this pilot study, the ERG team, consisting of ERG, Sensors, Inc., NuStats, the
Urban Institute, Southern Research Institute and the Desert Research Institute worked with the
United States Environmental Protection Agency (EPA) to integrate statistical sampling
techniques, the latest activity and emissions measurement technology and rigorous quality
assurance and quality control methods to characterize in-use, real-world emissions from nonroad
diesel engines within the commercial construction sector in EPA Region 7 (work was performed
in the states of Iowa, Kansas and Missouri).
The study focused on commercial establishments within the construction sector (defined
by NAICS code 23), with the objective of developing recruitment and data collection protocols
for ultimately expanding the study to other commercial sectors that operate fleets of nonroad
diesel engines (owned or leased) in their daily operations such as agriculture, mining, and utility
sectors. These protocols included:
Sample Frame - the list from which a sample of commercial establishments is
drawn.
Sample Design - the approach for randomly selecting commercial establishments
within the sample frame to meet a specified sampling target.
Recruitment Protocols - the process and materials used to (1) provide advance
notice to prospective participants about the study, (2) screen establishments for
eligibility as a study participant, and (3) recruit qualified establishments to
participate in the study (this later process includes the use of monetary
incentives). See Section 5, Fieldwork Operations, for details on these protocols.
Data Processing and Statistical Analysis- procedures for processing and
analyzing the dispositions or outcomes of sampling and recruitment stages for the
purposes of data weighting and analysis.
While these protocols and data collection processes were developed, tested and modified
within the construction sector, it is reasonable to expect that they will generally apply to other
commercial sectors with some modifications needed to address nuances with sample frames and
fleet operations that may be specific to a particular commercial sector.
1-1
-------
In total, 549 establishments were interviewed regarding their equipment ownership and
usage. From those establishments, 119 volunteered to allow project team members to conduct
onsite inventories and emissions and activity measurements on their eligible nonroad equipment.
Inventories were conducted at sites of 79 of those establishments, and emissions and/or activity
measurements were performed at sites of 29 establishments. Emissions tests were ultimately
conducted on 58 different pieces of nonroad equipment, and activity information was collected
from 30 pieces of nonroad equipment. Statistical sampling was applied prior to and during
fieldwork in order to randomize the recruitment and screening of participants and the selection of
equipment to be instrumented, and various ways of establishing a rapport with and minimizing
our testing burden on participants were explored throughout the study.
For the emissions measurements, portable on-board emission measurement systems
(PEMS) were used to perform 40 CFR 1065-compliant onboard measurements of gaseous and
aggregate paniculate matter (PM) exhaust emissions on 50 horsepower or greater diesel engines
in nonroad construction equipment. In order to withstand the rigors of testing in a nonroad
environment, equipment had to be "ruggedized", and equipment modifications and
enhancements were made throughout the course of the study to achieve test goals.
For the activity measurements, various commercially-available portable activity
measurement systems (PAMS) were evaluated, three different types of systems purchased, and
these systems were used during the work assignment to collect activity (date / time / engine
speed) over a period of approximately one month. This activity testing provided information on
long-term equipment usage patterns of this equipment and also provided an opportunity to test
and evaluate the performance of PAMS equipment and data quality during the work assignment.
Study data was analyzed, quality assured and processed for input into the EPA Office of
Transportation and Air Quality's (OTAQ's) Mobile Source Observation Database (MSOD),
where it may be used to help expand and improve the data currently supporting emission
inventory modeling for nonroad engines. Information regarding the sample design and field
team approach is presented in this report, and study results, including sampling / recruiting
results and also emissions and activity results, are provided herein. Lessons learned from this
study and recommendations for future work based on what was learned during this study are also
provided.
1-2
-------
2.0 Background
As described in the EPA's statement of work for this work assignment, nonroad
equipment contributes substantially to mobile-source emissions, with their contribution in
relative terms expected to increase as emissions from highway vehicles are controlled (Kean,
Sawyer & Harley 2000). Based on estimates derived from diesel fuel sales published by the
Energy Information Administration (EIA 2001) and estimates from the NONROAD model, fuel
consumption in nonroad equipment accounts for 15-18% of all diesel fuel supplied to the U.S.
market in 2000 (57.2 billion gallons), where "diesel" includes No. 1 and No. 2 distillates
(excluding kerosene, jet fuel and fuel oils). In addition, the sector selected for inclusion in this
survey (construction) is important in terms of diesel fuel consumption and emissions. In the same
year, this sector accounted for approximately 20% of nonroad diesel fuel consumption, which
corresponds to approximately 4% of all diesel fuel consumed by mobile sources (44.9 billion
gallons) where mobile sources include highway vehicles, locomotives, marine vessels and
nonroad equipment (EPA, 2006).
Generally, nonroad equipment includes vehicles powered by combustion engines,
designed to perform a wide variety of tasks other than street or highway transportation. Thus, the
term "nonroad equipment" covers a broad variety of machines including forklifts, graders,
crawler dozers, backhoe loaders, excavators and other equipment.
At the request of Congress, the National Academy of Sciences published a report on
EPA's emissions inventory modeling for mobile sources (NRC 2000). A committee of technical
experts was given the primary charge of conducting a detailed review of the MOBILE model,
which estimates fleet average emission factors for motor vehicles. Nonetheless, the report bears
mention in the context of nonroad equipment inventories because the committee also took the
opportunity to make comments concerning EPA's inventory modeling for nonroad equipment.
Under the heading of "Technical Issues Associated with the MOBILE Model," the committee
remarked that
As future Tier 2 vehicle standards and corresponding sulfur-reduction regulations reduce
on-road mobile-source emissions, non-road emissions will become a larger fraction of the
total emissions. The NONROAD model is extremely data driven, and there are many gaps in
the available data. EPA should place more emphasis on improving both the emissions
factors and activity data in this model, (p. 74)
In the executive summary, the committee emphasized the need for EPA to design and
implement programs to expand and improve the data used to support emissions inventory
estimation for nonroad equipment, and added that:
2-1
-------
The plan should include the population and activity data and real-world emissions
factors for gasoline and diesel engines (p. 13).
The recommendations in the NRC report have influenced the concept and design of
EPA's new inventory model for highway vehicles, the Motor Vehicle Emissions Simulator
(MOVES). Similarly, this work assignment is intended as the first step in a program to respond
to recommendations concerning the quantity and representativeness of the data supporting
inventory modeling for nonroad engines.
2-2
-------
3.0 Study Design Overview
3.1 Workplan and Survey Design Development
With the goal of characterizing emissions from nonroad diesel engines, EPA and the
ERG team developed a workplan that outlined the study design to characterize emissions from
50 nonroad diesel engines operated in specified counties within EPA Region 7 (in the states of
Iowa, Kansas, and Missouri) by establishments in the construction sector, NAICS 23.
This section describes the key elements of the study design including the definition of the
target population and coverage, a discussion of the sample frame, sample sizes and the sampling
method.
3.1.1 Target Population
The population of inference for the pilot study was composed of:
all nonroad equipment powered by nonroad diesel engines from
commercial establishments within the construction sector (defined
by NAICS code 23) that are located within the geographic area of
study.
Unlike motor vehicles, for which registration databases exist, it is not practical to construct
sample frames listing individual equipment pieces. To compensate for this difficulty, coverage
was expressed in terms of the owners or users of equipment within the construction sector.
Thus, for this pilot study, coverage of the target population included engines owned,
rented or leased by commercial establishments in the construction sector (NAICS 23) that
employed at least one person on a full-time or part-time basis during the previous twelve months.
Elements of the target population were identified and selected through the constituent
establishments that operated this equipment.6
3.1.2 Sample Frame
The term sampling frame denotes the list from which a sample is drawn. Ideally, the list
is all-inclusive of the target population. At the onset of this study, the ERG team employed the
Comprehensive Business Samples (CBS) supplied by Survey Sampling International (SSI) of
Fairfield, Connecticut as the sampling frame. SSI compiles listings from telephone directories
6 Government and other non-commercial establishments were excluded from the target population.
5-1
-------
and additional industry-specific sources including government listings, bank records, trade
directories, city directories and proprietary sources. Listings are verified and updated on a
continuous basis. Because it was the most recent sample available at the time the task was
initiated, it was used (and not updated) throughout the duration of the study. 7
3.1.3 Study Objectives and Sample Sizes
The study objectives called for the collection of emissions and activity data from 50
nonroad diesel engines operated within the study area. To achieve this, it was necessary to
collect a total of 550 observations from two independent samples:
Establishment sample - a sample of 500 establishments for a telephone survey,
referred to as the Equipment Ownership Interview (EOT), involving the
administration of an equipment ownership survey to screen and qualify
establishments for the study.
Equipment sample - a sample of establishments from which 50 equipment
measurements consisting of 25 emissions measurements and 25 activity
measurements would be drawn. These were establishments qualified in the
Establishment sample who agreed to participate in an inventory and emissions and
activity testing of their equipment fleet.8
The administration of the EOT was intended as a predecessor to the administration of the
Equipment Sample Interview (ESI) to recruit participants for an equipment inventory and
random selection of equipment for instrumentation.
Specific targets were provided for the numbers of establishments and pieces of equipment
for which data were collected. Table 3.1-1 presents the targeted distribution of observations by
measurement type.
7 As is presented later in this section, this original design was modified based on the performance of Phase 1 and
subsequent data collection efforts of the study.
o
For the 50 observations involving equipment measurements, an experiment was embedded to test for response rate
effects associated with incentives. The incentive was offered to a random half of the eligible Equipment Sample
subjects upon completion of the EOI (telephone) portion of the survey (and as part of the recruitment process into
the inventory and measurement components of data collection).
5-2
-------
Table 3.1-1 Targeted Distribution of Observations
by Measurement Type
Economic Sector
Construction
MEASUREMENT
TYPE
Total EOI and
Instrumentations
550
ESTABLISHMENT
SAMPLE
Equipment
Ownership Interview
500
EQUIPMENT
SAMPLE
Emissions
Measurement
25
Activity
Measurement
25
The numbers of establishments recruited into the Equipment Sample via the EOI were
purposely larger than the number that actually completed the emissions and/or the activity
measurements. This was because the study team anticipated (1) inevitable non-cooperation post-
EOI (i.e., agreed on the telephone but declined at the time of the on-site visit); and (2)
recognition that a certain fraction of completed measurements would nonetheless result in
unusable measurement data due to a variety of reasons (i.e., equipment ineligible, equipment
failure, equipment not operated as planned or moved to another construction site). The data
collection plan reflected these factors.
This pilot study was planned to be implemented in two phases for the Establishment
Sample and three phases for the Equipment Samples, depending on funding levels and observed
performance of the sample. The Pilot would commence with the first phase of the Establishment
Sample (involving conduct of only the EOI). EOI data collection would start slowly in one of
the five study areas: PSU 1. EOI Phase 2 of the Establishment Sample would follow the
completion of EOI Phase 1, and a similar roll out of the Equipment sample was planned
following completion of the Establishment Sample. Tables 3.1-2 and 3.1-3 provide the original
plan for sample sizes by phase for the Establishment and Equipment Samples, respectively.
These tables show that the Establishment Sample was to be limited to two phases of activity,
while the Equipment Sample would involve three sequenced phases of activity.
Table 3.1-2 Original Plan for Conducting Establishment Sample Surveys
ESTABLISHMENT SURVEY
PHASE
EOIs, Phase 1
EOIs, Phase 2
TOTAL Establishment Sample
#PSUS
1
2-5
5
TARGET #
COMPLETES
100
400
500
5-3
-------
Table 3.1-3 Original Plan for Conducting Equipment Sample Measurements
MEASUREMENT TYPE
(INCLUDING EOI)
# of PSUs
Activity Measurements
(with oversampling)
Emissions Measurements
(with oversampling)
ES PHASE 1
1
6
6
ES PHASE 2
2
12
12
ES PHASE 3
2
12
12
TOTAL
5
30
30
The Equipment Sample called for the conduct of an EOI (via centralized CATI), after
which the incentive experiment for recruitment was given, followed by preliminary "onsite
equipment inventory" interview, then followed by a face-to-face visit to the establishment to
conduct a site inventory (or inventory of more than one site), and finally selection of equipment
and the emissions and activity measurements. Moreover, because the EOI was the precursor to
the onsite equipment inventory and instrumentation phase, the study team expected that a
number of establishments in the Equipment Sample would participate in the EOI interview, then
agree to a face-to-face visit but then decline to participate in the inventory and/or the emissions
and/or activity testing (despite earlier indications of cooperation). As such, the EOI sample sizes
for the Equipment Sample were planned larger than the final targeted number of equipment
measurements in order to allow for this inevitable attrition.
The impact of this on the number of EOIs conducted in the Equipment Sample is shown
in the second data row of Table 3.1-4. A total of 185 EOI interviews were expected to be needed
in order to secure 60 total inventories and equipment measurements. This table also presents the
by-phase distribution of inventories and instrumentations which were expected (similar to that
shown in Table 3.1-3). Summaries of the actual outcomes of onsite inventories and
instrumentations are provided in Section 5.6.
Table 3.1-4 Expected Completions by Sample and Measurement Type
COMPLETES BY TYPE
EOI - Establishment Sample
EOI - Equipment Sample
Inventory
Emissions measurements
Activity measurements
EOI
PHASE 1
100
0
0
0
EOI
PHASE 2
400
0
0
0
ES
PHASE 1
37
12
6
6
ES
PHASE 2
74
24
12
12
ES
PHASE 3
74
24
12
12
TOTAL
500
185
60
30
30
5-4
-------
The sample sizes by study phases that would be needed to achieve the target number of
completions for the pilot were estimated. As shown in Table 3.1-5, a total of 2,803 selections
would be needed, including the reserve sample to complete the EOIs for the Establishment
Sample and the EOT portion of the Equipment Sample. It was estimated that more than 185 EOIs
would need to be conducted in order to get 185 establishments agreeing to be instrumented;
under these design a total of 1,588 EOIs would be necessary. However, these were conservative
estimates and assumed that only one instrumentation would occur per piece of equipment (i.e.,
either an activity measurement or an emissions measurement, but not both).
Table 3.1-5 Anticipated Sample Needed to Achieve Sample Targets
TYPE OF DATA
COLLECTION
EOIs
Inventory &
Instrumentations
ESTABLISHMENT SAMPLE (EOI)
PHASE 1
243
0
PHASE 2
971
0
EQUIPMENT SAMPLE
PHASE 1
318
37
PHASE 2
635
74
PHASE 3
635
74
TOTAL
2803
185
3.1.4 Sampling Methods
The sample design for this pilot study employed stratified multi-stage probability
sampling with probabilities proportional to size. The number of selection stages varied by the
type of data collection (i.e., establishment vs. equipment samples). The survey of establishments
employed two-stages of selection, while the equipment samples involved a three-stage design.
To reduce travel time and associated expense for field technicians installing and
maintaining instrumentation on site, the equipment sample was drawn in three stages as follows:
First Stage
(Primary) County level
Second Stage (Secondary) Establishment level (within county)
Third Stage
(Tertiary) Equipment piece (within establishment)
Because the first- and second-stage sampling units were different sizes, in terms of the
numbers of equipment expected within each unit, the second-stage samples were drawn with
probability proportional to size (PPS). This technique is commonly employed in combination
with state sampling to reduce differences in the final sampling weights among individual sample
units. In PPS, the first and second-state selection probabilities would then be managed to
compensate for the fact that third-stage probabilities cannot be managed.
5-5
-------
The measure of size (MOS) is ideally a measure of the units in the target population
(equipment piecesor diesel enginesin this case), or alternatively, another measure of
influence that is assumed to correlate fairly strongly with the target population in each sampling
unit. Lacking direct estimates of equipment populations by county or establishment, the number
of employees per establishment was selected because it was assumed to be correlated to
equipment population.
In the third stage, one or more eligible equipment pieces would be drawn from each
selected establishment using simple random sampling (SRS). Specifics of the design at each
stage are discussed below.
First Stage. The first stage of selection was utilized by all samples in the study and
involved a sample of 5 counties (primary sampling units or PSUs) with probabilities proportional
to size (pps) from the collection of counties selected within EPA Region 7. EPA designed the
sample and selected these counties, which are shown in Table 3.1-6.
Table 3.1-6 Primary Sampling Units for the Pilot Study
PSU
1
2
3
4
5
FIPS
29095
19113
19163
29047
20177
STATE
MO
IA
IA
MO
KS
COUNTY
Jackson
Linn
Scott
Clay
Shawnee
EST. NO.
EMPLOYEES
63,800
25,400
18,500
16,500
13,000
SAMPLING
PROBABILITY
0.3063
0.1216
0.4277
0.3813
0.3006
Second Stage. Within each PSU, commercial establishments were the secondary
sampling unit (SSU), drawn with selection probabilities based on the same measure of size used
to draw the first-stage sample. For construction establishments, the establishment MOS was the
estimated number of employees, as derived for use in the first stage (Table 3.1-6). After
compiling the establishment frame for each PSU, an estimated number of employees was
assigned to each establishment (MOS), based on its assigned size class.
EPA required that the universe of establishments be obtained from SSI with the
expectation the data would contain sufficient information (i.e., number of employees) so that the
same calculations can be implemented to create the MOS.
-------
It was likely that the initial measure of size estimate MOS*(ij), when summed over the
relevant establishments in county j, would not match the specific county-level total MOSQg)
used by EPA to select the first phase sample. The size measure could be adjusted accordingly so
the adjusted MOS*(ij) in county./ would match the corresponding MOS from the stage one
sampling. Such an adjustment would have no impact on the Phase 2 selection probabilities.
Nevertheless, the distribution of adjustment factors might provide some insights regarding the
performance of the proposed MOS. For instance, if the adjustment factors were roughly equal
across the sampled counties, this would suggest a robust formulation for MOS.
At stage two of sampling it would be necessary to draw independent samples by PSU.
Table 3.1-5 in the preceding subsection showed the required sample sizes by sample type and
phase to achieve the targeted number of completed interviews and instrumentations.
To illustrate the stage two approach, let m be the number of construction establishments
selected per PSU. Furthermore, \etMOS*(ij) denote the measure of size for establishment i and
PSU/ (In practice constructed the MOS* from the universe file obtained from SSI.)
The second stage conditional selection probability of establishment i and PSUj would be:
Prob(establishment i \ PSUj) = mxMOS*(if)/Sum i\j{MOS*(ij)}
The denominator sum ofMOS*(ij) occurs over all establishments i in PSU/ Depending
on the prevalence and sizes of construction establishments in a given PSU, the second stage
selection probabilities were expected to vary significantly. Another consequence is that it was
expected to be difficult or impossible to achieve the desired PSU sample size targets. That is,
there was no guarantee that a nominal number of construction establishments would be available
for selection from each PSU.
Considering the overall selection probability of an establishment was relevant for the
equipment ownership survey in Phases I and II, letting MOSQ) denote the stage one measure of
size of PSU/ the overall (unconditional) probability of selecting establishment i in PSUj would
be:
Prob(establishment i) = Prob(establishment i \ PSUj) x Prob(PSVj)
= m* MOS*(ij) x 5 x MOS(j)/{Sum][MOS (j)] x Sum i[j [MOS*(ij)]}
= c(ij) x {MOSQ) /Sum i[j [MOS*(ij)]}
5-7
-------
where c(ij) = m x MOS*(ij) x 5 /Sum j[MOS (j)] and
r(j) = MOSO) /Sum i[j [MOS*(ij)J
Thus, c(ij) is proportional to MOS*(ij), and r(j) depends only on PSU/
The overall probability of an establishment would be proportional to its second stage
measure of size MOS*(ij) when the ratio r(j) is a constant But r(j) is the ratio of the first stage
PSU measure of size MOSQ) to the summed measures of size Sum i\j [MOS*(ij)J used in the
second stage of selection. The first stage measure of size MOSQ) is summed over all
establishments used in the selection of the PSUs (the nature of the establishments used for
developing theMOSwas not clear), while the second stage denominator term Sum i|/° /
MOS*(ij)J is summed only over a subset of establishments only those in PSU./ for the
construction sector.
Third Stage. The third stage of selection was relevant to the sampling of equipment from
the equipment inventory performed prior to instrumentation. The conditional probability of
selection at this last stage of sampling was simply l/t(i), where t(i) denotes the total number of
equipment pieces for establishment i.
Assuming the selection of one piece of equipment from each establishment, the overall
selection probability of equipment piece h in establishment i, and PSUj is simply the product of
the second stage overall selection probability and the conditional equipment probability l/t(i).
Thus, the overall probability is written as:
Prob(equipment h) = Prob (equipment h \ establishment i) x Prob (establishment i \ PSUj)
x Prob(PSUj)
= l/t(i) x m x MOS*(y) x 5/ SumjfMOS (/)] x rQ)
= {m* 5/SumjfMOS (j)]} x {MOS*(ij) /t(i)} x rQ)
Where r(j) is defined as before,
c* = mx 5 /Sum }[MOS (j)] = a constant, and
s(i)=MOS*(ij)/t(i)
-------
To the extent that the number of pieces of equipment in establishment i is correlated to
the second stage measure of size used in selecting establishments, MOS*(ij), the factor s(i) will
be constant. And if both s(i) and r(j) are constant, then the three stage design produces an equal
probability of selection (epsem) sample for equipment. We have discussed reasons why r(j)
would likely not be constant, but if the measure of size proposed by EPA was correlated with
equipment pieces, then the factors s(i) would be relatively stable within the construction sector.
3.2 Development of the Draft Quality Assurance Project Plan (QAPP)
As part of the proposal effort, the ERG Team tailored the corporate Quality Management
Plan (QMP) into a guidance document which provides corporate quality guidelines for work
under this contract. In addition, a Quality Assurance Project Plan (QAPP) was developed as part
of this study to provide project-specific guidelines that cover all facets of the study. This QAPP
was drafted prior to the commencement of project activities, and underwent multiple revisions at
different times during the project. As specified in the work assignment, the QAPP was based on
the following two guidance documents:
Requirements for Quality Assurance Project Plans. EPA QA/R-5. EPA/240/B-01/003.
USEPA Office of Environmental Information. Washington, D.C. (Available at
http://www.epa.gov/quality/qs-docs/r5-fmal.pdf).
Guidance for Quality Assurance Project Plans. EPA QA/G-5. EPA/240/R-02/009. USEPA
Office of Environmental Information. Washington, D.C. (Available at
http://www.epa.gov/quality/qs-docs/g5-fmal.pdf).
The QAPP conforms to requirements specified in Section 2.1 of the work assignment, and
describes the following measures:
standard procedures for calibration of all portable measurement instruments
standard schedules for regular calibration of portable measurement instruments, and the
maintenance of permanent and retrievable records of all calibrations
procedures or decision rules for verifying proper operation of a portable measurement
system when reviewing records of calibrations, spans, or zeroes
maintenance of operating logs for all portable measurement systems
standard operating procedures for equipment used to perform calibrations
standard operating procedures for portable measurement instruments (PEMS/PAMS)
procedures for sampling and recruitment of respondents
5-9
-------
procedures for data transfer, entry and management
procedures for regular transfer of all data generated within this project to the Work
Assignment Manager for review and audit, and
procedures for the protection of respondent confidentiality
data tracking and chain of custody procedures
A copy of the QAPP, and its associated appendices, is provided in Appendix N of this report.
3.3 Conducting Equipment Ownership Surveys, Revising QAPP
An Equipment Ownership Interview (EOT) was administered to Phase 1 (PSU 1) survey
participants during a field period running April 20, 2007 through June 20, 2007. While the
protocols developed for the study administration are presented in Section 5 of this report, this
section describes the general approach followed in conducting the EOT.
Prior to the administering the EOT survey, the study team conducted a number of
activities with the purpose of improving response and participation rates. This included (1)
securing study support from area trade associations; (2) conducting an advance mailing to
prospective respondents that contains a letter and brochure informing them about the survey and
assuring study confidentiality; and, (3) cognitive testing / structured interviews following Phase
1 of the establishment ownership questionnaire. Each is briefly discussed below.
Obtain Trade industry support. Obtaining support from industry trade associations
can positively influence prospective respondents decision to participate in the study
(e.g., complete an Equipment Ownership Interview or agree to instrumentation during
the Equipment Ownership Interview). Early in the study, contact was made with
trade associations to enlist their support in the study. Specifically, the following
associations agreed to and provided their logos for use in the advance letter:
American Road & Transportation Builders Association, Associated Equipment
Distributors, Associated General Contractors of Iowa, Associate General Contractors
of Missouri, Associated General Contractors of Kansas, and the Kansas Contractors
Association. Additional details on trade organization recruitment are provided in
Section 5.2.
Conduct advanced mailing. Administration of the study began with an advance letter
to inform business owners of the purpose of the survey, to reassure the business
owners of study participation confidentiality as described below, and to enlist their
3-10
-------
participation in the study. The advance letter also provided prospective respondents
with a fact sheet containing more detailed information about the study and
demonstrating industry support. The letter was printed on EPA letterhead to increase
the perceived legitimacy of the survey. Prior to conducting the survey, each sample
record was sent an advance mail packet containing the advance letter and information
brochure, as shown in Appendices AI and AJ.
Conduct cognitive interviews. Cognitive testing9 of the Establishment Sample
Equipment Ownership Interview was conducted after Phase 1 of the Establishment
Sample Equipment Ownership Interviews in order to have the benefit of initial
analysis of data and response rates. The cognitive interviews were conducted with
persons who participated in the Phase 1 EOI interview. Minor adjustments to
question wording and flow were made based on the cognitive test results. In addition,
the cognitive interviews also assessed the effectiveness of the advance mail materials
(e.g., whether the use of EPA letterhead and sponsorship of trade associations
influenced their decision to participate in the study).
For the Establishment Sample, the EOI was to be conducted by telephone over two
phases. In Phase 1, only the first PSU was sampled with the goal of completing 100 interviews.
As previously described, within a week following receipt of the advance letter, a phone call was
made to a knowledgeable respondent and the EOI was completed. Data was collected under a
pledge of confidentiality that responses would be used for statistical purposes only. During
collection, storage, and reporting, steps were taken to protect the identity of respondents or
establishments with the information collected, as required by the workplan and the QAPP.
During data collection, NuStats monitored call attempts, dispositions and progress in
completing interviews. At the completion of Phase 1 interviews, the study team processed and
analyzed the data collected, revised the Equipment Ownership Questionnaire and the QAPP
based upon the performance of Phase 1 data collection efforts
As part of this assessment, seven cognitive interviews were conducted as described
above. The study team prepared a report that analyzed the questionnaire's performance with the
final standard call dispositions, provided recommendations on revisions to the questionnaire and
on the study design based on the analysis of Phase 1 data collection results.
A cognitive interview provides an assessment of the mental processes (e.g., question comprehension, memory retrieval,
response formation and editing) associated with the questions in a survey instrument using persons that possess similar
characteristics to the survey's intended audience, involving in-person interviewing. These interviews were also used to assess the
adequacy of the questionnaire flow (structure and design), and to test advance mailing materials and other aspects of the study.
3-11
-------
3.4 Acquiring and Preparing EAM Equipment
The acquisition and preparation of emissions and activity measurement (EAM)
equipment began in February 2007, several months before establishments were first contacted
for EOIs (and participation in the study). The following sections describe the equipment used
during the study and the steps taken to prepare equipment for use in the study.
3.4.1 Overview of Emissions and Activity Test Equipment
The ERG team used SEMTECH-DS PEMS manufactured by Sensors, Inc. and provided
by the EPA for collection of emissions data for this work assignment, and researched and
acquired from commercial vendors the PAMS used for collection of activity measurements. A
description of the research for and procurement of the PAMS equipment is provided in Section
3.4.3.
Under this contract, the ERG team leased a trailer from Sensors, Inc. which was used to
house, transport and maintain the PEMS and all associated support equipment. Sensors
personnel transported this trailer to and from EPA Region 7 and the various work locations
within the region using a flatbed truck with a boom lift, which was also part of the lease
agreement. This boom lift allowed the PEMS installation team members to place the
approximately 400-pound PEMS rack onto each piece of equipment being tested. Figure 3.4-1
shows the PEMS rack on a rolling cart, prior to installation on a backhoe loader, with the leased
SEMTECH support trailer in the background. Figure 3.4-2 shows the leased four-wheel drive
truck with flatbed-mounted boom lift being used for installation of the PEMS rack on the
backhoe loader.
3-12
-------
Figure 3.4-1 PEMS Rack and Trailer
Figure 3.4-2 Installation of PEMS Rack Using Truck-Mounted Boom
3-13
-------
For emissions measurements, the PEMS rack collected the following information in one-
second intervals, as specified in the work assignment:
engine speed (revolutions per minute, rpm),
oxygen concentration in the exhaust stream ([62], percent by weight, wt%),
carbon-dioxide concentration in the exhaust stream ([CO2], percent by weight, wt%),
oxides of nitrogen concentration in the exhaust stream ([NOX], parts per million, ppm),
carbon monoxide concentration in the exhaust stream ([CO], percent by weight, wt%)
total hydrocarbon concentration in the exhaust stream, ([THC] parts per million, ppm)
aggregate paniculate matter by gravimetric methods (g),
ambient temperature (°C),
exhaust temperature (° C),
exhaust mass flow rate (via the Sensors EFM)
relative humidity (%), and
barometric pressure (kilo-Pascals, kPa).
date/time stamp.
The following derived measurements, also specified in the work assignment, were provided
for all emissions measurements:
exhaust flow volume (adjusted to standard temperature and pressure, cu. ft/min (scfm)),
fuel flow volume (g/sec, gal/sec),
carbon dioxide emission rate (g/sec, g/kg fuel),
pollutant emission rates for NOx, CO, THC, and PM, (g/sec, g/gal).
The PEMS rack consisted of the EPA-provided SEMTECH-DS PEMS (as well as a
backup SEMTECH-DS PEMS), a Sensors micro-proportional Sampler (MPS), a Sensors
Gravimetric Filter Sampler and 2", 3", 4" and 5" diameter exhaust flowmeters (EFMs).
Flowmeter diameter selection was based on each particular installation. A small air compressor
and filtration unit was used to operate the MPS, and to automatically back-purge the EFM
pressure lines at specified intervals. This air compressor operated using A/C power provided by
a Honda portable generator. Automated zero calibrations of the SEMTECH-DS analyzers were
performed throughout the sampling period using ambient air which was scrubbed with a carbon
filter and a particulate filter.
All emissions measurements throughout the study included gravimetric filter sampling
using a micro-proportional sampling system (MPS) and Sensors' 3-chamber gravimetric filter
sampler provided by EPA . The SEMTECH MPS is a two-stage dilute proportional sampler in
which a proportional sample flow is extracted from the exhaust flow. This sample flow is
controlled to be a constant fraction of the varying exhaust flow by way of a two-stage dilution
system. The first stage performs a fine "valving" function and the second stage is a venturi
which adds the major part of the dilution flow and forces the sample plus the primary dilution
flow to exit the MPS (Fulper, Giannelli, et al., 2010). The MPS total flow (sample flow plus
3-14
-------
primary and secondary dilution) was held to a constant rate of approximately 12.5 liters per
minute in this study. Maximum exhaust flowrates and major and minor dilution ratios were
tailored for each installation based on engine size and anticipated workload according to
guidelines in the PEMS installation SOPs (Appendix F). Additional information regarding
flowrates and system settings are available in Appendix F (PEMS Installation SOPS) and
Appendix I (PEMS Data QC Criteria).
Gravimetric PM samples were collected on 47mm Teflon filters housed in the
gravimetric filter sampling unit which was heated to 1065 specifications. Filter flow was
maintained at a rate of approximately 18 liters per minute. The filter sampler automatically
switched between the three gravimetric filters, based on an integral timer, input voltage signal
indicating engine start, or the filter could also be switched by a wireless remote electrical signal.
One filter was used to capture the first start at the beginning of the day or shift (generally a 10-
minute cold-start), and the second and third filters captured continued warm operation or hot
start emissions (20-minutes sampling for the second filter and 30-minutes sampling for the third
filter). Each "start" episode was defined as the first ten (10) minutes of operation after the
engine was turned on.
For testing during PSU1, Desert Research Institute (DRI) supplied pre-weighed 47 mm
Teflon filters to the PEMS installation team in pre-loaded URG-2000-30FL filter cassettes
provided by EPA. DRI provided all gravimetric laboratory analysis for PSU 1 (and EPA
subsequently provided the filters and gravimetric laboratory analysis in ES Phases 2 and 3 of the
study). After PM collection, filter samples were kept cold and returned to the DRI or EPA
laboratories in the URG filter cassettes for post-test gravimetric measurements. Resultant data
was provided to ERG on a total mass per filter basis (i.e., mg/filter).
Dynamic and field gravimetric filter blanks were collected during the study in order to
identify and quantify any transport and handing contamination on the gravimetric filters. Field
blanks were treated as actual samples, including all shipping, handling and transport to the field
during testing, although they were never removed from their shipping cassettes or placed in the
gravimetric sample system holder. Dynamic blanks were also treated as actual samples but in
addition to field handling they were placed in the gravimetric sample system holder rack during
emissions testing. However, no exhaust sample (or air) was routed through the dynamic blanks
(the flow-control solenoid on the gravimetric sampler which held the dynamic blank remained
closed during testing). Comparison of the measurement results for dynamic blanks and field
blanks can help provide information regarding potential contamination in the filter holder as well
as any leaks in the flow-control solenoids used to isolate that specific filter holder in which the
3-15
-------
dynamic blank was placed. Five percent of all filters were designated as field blanks and five
percent as dynamic blanks, as defined in Appendix K (Onsite Installation Manager SOPs).
Initial study design included the use of a carousel-equipped quartz crystal microbalance
(cQCM) to allow up to 8-hours of continuous PM measurement. However, since cQCM
availability was limited due to EPA's concurrent Heavy-Duty In-use Verification Program and
the double-dilution which would be required to operate the cQCM would require extensive
PEMS rack modifications beyond the project's budget and schedule constraints, PM emissions
measurements were limited to gravimetric filter sampling and no continuous sampling was
performed.
ERG acquired "core" portable activity measurement systems (PAMS) conforming to the
PAMS specifications outlined in the work assignment. These systems measured and recorded
engine on and off times, engine speed and associated date and time stamps over approximately
one month for each activity instrumentation. Section 3.4.3 describes the PAMS research and
purchase performed in support of this work assignment.
3.4.2 Fabrication of the PEMS Equipment Rack
Prior to the start of fieldwork, Sensors Inc., under subcontract to ERG, fabricated an
enclosed rack to house all PEMS sampling equipment, including the SEMTECH-DS PEMS,
exhaust flow meter and heated sample line, external weather station, the micro-proportional
sampler (MPS), air compressor and flow control unit to provide filtered dilution air, the three-
stage 47mm gravimetric sampling system, a rotary vane vacuum pump for filter sampling, power
supply and backup battery, nitrogen bottle for switching gravimetric filter solenoids and
Carousel Quartz Crystal Microbalance (cQCM) continuous PM sampling system (although the
cQCM was not used during this study). Figures 3.4-3 and 3.4-4 show the PEMS rack during
construction, before the side covers were installed. In these images, the gravimetric sampling
system is not yet housed in the PM filter box, and several components (such as the air
compressor and nitrogen bottle) are not included or cannot be seen. This rack was used
throughout the work assignment.
3-16
-------
Figure 3.4-3 PEMS Rack During Construction (open sides), Front
Exhaust Flow Meter
II
MPS
SEMTECH
Flow Control
Unit
PM Filter Box
Figure 3.4-4 PEMS Rack During Construction (open sides), Rear
^^
Heated Sample Line
Backup Battery *^** '
Weather Station
PM Filter Box
3-17
-------
3.4.3 Acquisition and Preparation of PAMS Equipment
Prior to the start of fieldwork, ERG was tasked with identifying portable activity
measurement systems (PAMS) that could be used to collect activity measurements. Cost and
conformance with work assignment specifications (as listed in Appendix C, PAMS
Specifications, of the work assignment) were used as the primary basis for evaluation, with
additional consideration given to factors such as a qualitative assessment of overall product
design and anticipated responsiveness and support of the PAMS supplier.
Market research was conducted in order to identify the universe of candidates from
which the most suitable units could be selected. Although no PAMS were identified which
conformed entirely to the PAMS Specification, several candidates were identified which did
satisfy a number of the requirements, especially with respect to "core" units. Once these units
were identified, primary consideration was given to factors such as unit cost, ability to
automatically shift into "sleep" mode and auto-initiate, ability to operate from the vehicle or
equipment's power source with minimal electrical current drain, OBD/CAN datastream
communication capabilities, availability of additional analog and digital input channels for
auxiliary sensor inputs, and ability to be placed into service without the need to independently
develop acquisition and configuration software (i.e., units are provided with suitable
configuration and processing software). Although units meeting only the "core" specification
requirements were considered during our search, emphasis was placed on those units that could
be modularly expanded into "comprehensive" units. In this way, the PAMS selected for this
study could also be used in future studies with different system requirements.
Nine systems were identified which appeared to meet a number of the critical PAMS
Specification requirements. Each of these units was investigated to assess overall conformance
with the specification. Manufacturers and distributors were contacted in order to obtain technical
information, cost information and demonstrations of interfaces and capabilities. Ratings were
assigned based on how well each unit satisfied the PAMS Specification requirements.
Eventually, ERG recommended purchasing dataloggers from three manufacturers, and in
coordination with EPA, ERG then purchased only enough units in order to perform testing in the
first phase of the study (6 activity tests, plus possible concurrent testing during emissions testing,
and a backup unit). The initial purchase was limited to only enough units for the first phase in
order to allow additional evaluation of these systems to be made regarding during ES Phase 1
testing, prior to a second purchase order being placed for the following two phases of the study
(in which 12 activity tests plus backups would be necessary). For ES Phase 1, five Corsa EZII
3-18
-------
Dataloggers, one Isaac V8 Sealed Datalogger and one Hemdata DAWN-LOG16 datalogger were
purchased. Figure 3.4-5 shows a Corsa datalogger, an Isaac datalogger, and the Hemdata DAWN-
LOG^ datalogger which were part of our initial purchase for this study.
Figure 3.4-5 Corsa EZII, Isaac V8, and HEMDATA Dawn Dataloggers
3-19
-------
Appendix O contains three memos, the first two describe results of the market review of
dataloggers and recommendations for purchasing dataloggers for ES Phase 1 of the study, and
the third memo provides recommendations for purchasing five additional Isaac dataloggers to be
used for the remainder of the study (ES Phases 2 and 3). In all, five Corsa dataloggers, six Isaac
dataloggers and one Hemdata datalogger were purchased for support of this study. EPA
provided an additional two Corsa dataloggers which were also used during field testing.
3.4.4 Preliminary Installation and Testing of EAM Equipment
Prior to the commencement of field activities, EPA, Sensors, ERG and SRI assembled a
"mock-up" of the emissions and activity measurement systems on a piece of diesel equipment at
Sensors' facility in Saline, Michigan. At this time, Sensors also provided training on the
installation, operation and maintenance of the gaseous and particulate sampling systems, and
ERG provided training on installation and operation of the PAMS systems. This "mock-up"
served several purposes, including:
Installation team members were able to increase their familiarity with equipment and
procedures to be used in the field
Team members were able to identify and acquire (or develop) remaining equipment,
tools, materials and procedures that were lacking prior to the start of field activities
Previously unforeseen technical or logistical challenges were identified and resolved
prior to field activities
Equipment operation was confirmed and systematic performance was evaluated and
optimized
A gravimetric filter "blind" comparison study between DRI and EPA was performed,
and
All consumables required for field testing were sourced and acquired
3.4.5 Gravimetric Participate Matter Blind Study
The ERG team coordinated a round of interlaboratory gravimetric mass measurement
comparisons between EPA and DRI prior to ES Phase 1 emission testing. This "blind study"
was conducted in the following sequence:
1) DRI pre-weighed and sent 12 Teflon filters with unique identification numbers to EPA
in EPA-provided filter holders (URG-2000-30FL filter cassettes). DRI provided these
pre-weights, along with filter ID, to ERG.
2) EPA weighed these filters (pre-weights) and collected samples on 8 of these filters.
Four of these filters were treated as field/transport blanks. EPA provided these pre-
weights, along with filter ID, to ERG.
3-20
-------
3) After collecting samples on 8 of these filters, EPA re-weighed the filters and provided
these post-weights, along with Filter ID, to ERG.
3) All twelve filters (including the four blanks) were then returned to DRI. DRI weighed
all twelve filters, then removed a tiny portion from the filter ring on one of the blank
filters. DRI then re-weighed the one altered blank, and provided all weights, along with
filter ID, to ERG. DRI then sent these twelve filters to EPA.
4) EPA then weighed all twelve filters and provided these post-weights, along with filter
IDs, to ERG
Results of the blind study are provided in Section 6.2.1.
3.5 Overview of Fieldwork Activities
The fieldwork for this study was conducted in three phases (ES Phases 1, 2 and 3),
encompassing testing throughout 5 counties, or Primary Sampling Units (PSUs). Only one
county (PSU 1) was sampled in ES Phase 1, while two counties each were sampled in ES Phases
2 and 3 (PSUs 2 and 3 were sampled in ES Phase 2, and PSUs 4 and 5 were sampled in ES Phase
3). The work assignment specified a total of 25 emissions and 25 activity measurements to be
conducted over the course of the five-county study (five emission and five activity measurements
in each county). However, more tests were attempted than were specified in the work
assignment to account for loss of sample due to equipment or data issues. In addition, several
weeks of PEMS testing were added to the scope of work during ES Phase 3, so a total of 40
emissions and 30 activity measurements were ultimately conducted during the study. Table 3.5-
1 provides a summary of emissions testing conducted throughout the three phases of the study.
3-21
-------
Table 3.5-1 Summary of data collected throughout the Nonroad PEMS Study
ES Phase / PSU
ESPh 1, Summer 07
Jackson, MO (PSU 1)
ES Ph 2, Fall 07
Linn, IA (PSU 2) &
Scott, IA (PSU 3)
ES Ph3, Summer&Fall 08
Clay, MO (PSU 4) &
Shawnee, KS (PSU 5)
Totals
PEMS
Target
PEMS
5
10
16
31
PEMS
Attempts
6
13
21
40
PEMS Successful
Gaseous
3
10
16
29
PM
2
10
15
27
RPM
1
9
15
25
PAMS
Target
5
10
10
25
Install
Attempts
7
11
12
30
Data
Collected
7
10
12
29
At the outset of fieldwork, the sampling plan permitted up to four pieces of equipment
(two emissions measurements and two activity measurements) to be sampled per establishment.
However, EPA and the ERG team decided to allow some of the sampled pieces of equipment to
be measured for both activity and emissions (which would reduce the total number of pieces of
equipment sampled per site). In addition, during the course of the work assignment, the teams
discovered that fewer establishments used PEMS-eligible equipment than originally anticipated.
Because of this, PEMS sampling requirements were relaxed to allow up to five pieces of
equipment to be sampled per establishment (three emissions and two activity measurements).
Again, some of these sampled pieces could receive both emissions and activity measurements.
The following subsections describe the inventory and equipment selection process used
throughout the study.
3.5.1 Conducting the Onsite Inventory and Selecting Equipment to Test
As previously described, initial contact with each establishment was made by NuStats'
call center, Datasource. A subset of establishments contacted by Datasource were solicited for
and agreed to participate in onsite inventories and emissions and activity measurements (field
testing). For those establishments who agreed to participate, NuStats transferred to ERG (via a
secure FTP site) all relevant establishment information, including equipment counts and contact
information. Updated information was generally posted at least twice per week, or more
frequently if needed.
Personnel in ERG's Kansas City office retrieved the establishment information from the
secure FTP site and attempted to call each establishment. Inventory appointments were
3-22
-------
scheduled for each participating establishment and details regarding locations, times and contact
information for the onsite inventories were established. Inventory appointment information was
then transmitted to the ERG onsite manager and the ERG inventory team leader.
Inventories were either scheduled to occur at a specific work site or instead to be initiated
at a general location, usually the establishment office. If the meeting took place at the
establishment office and a choice was provided regarding sites which could be inventoried, the
ERG inventory team leader selected one or two sites at which to conduct inventories. When
more than one site was available for inventory, the ERG inventory team leader had the discretion
to decide whether one or two sites would be inventoried. Once a decision was made regarding
the number of sites to be inventoried, the inventory team was then required to inventory all sites
selected (even if an abundant amount of equipment suitable for instrumentation was identified at
the first site inventoried). If more than two sites were available to inventory, the inventory team
leader randomly selected which one or two sites would be inventoried. This was only necessary
if more sites were available for inventory than were actually to be inventoried.
For each site inventoried, information regarding all equipment belonging to, leased by, or
used by a particular establishment was collected on a site inventory form, as shown in Appendix
C. Sufficient information was gathered for each piece of equipment in order to allow
determination of equipment model year and engine power using supplemental information, such
as EquipmentWatch or other commercially available equipment specification resources.
Equipment serial number and cumulative hours of use (from the equipment's hour meter) were
recorded and each piece of equipment's PEMS and PAMS testability was also assessed. In order
to help inventory personnel locate serial numbers and ensure accurate data collection, guidelines
were made available to field personnel describing common serial number locations and formats
for various types of nonroad equipment, such as the "Serial Number Location Index" provided in
the Equipment Watch Serial Number Guide.
Digital photographs of equipment, serial numbers and equipment specification tags were
taken whenever possible, to help clarify or correct any ambiguous or inaccurate information
recorded during on-site inventories. Photographs were also taken of engine tags, when possible,
in order to help confirm horsepower ratings. Maintenance hours written on or near the
equipment's oil filter were also recorded in order to help confirm hour meter information.
Inventory information collected in the field was entered into a master spreadsheet posted
on a project-specific secure server. Hardcopy inventory forms were also sent to ERG's Austin
office to allow independent confirmation of data which had been entered. Also in Austin,
3-23
-------
equipment horsepower and model year information was determined using equipment
specification literature. Information pertaining to the equipment's age, engine size, and
cumulative usage was used to assign each piece of equipment to a specific weighted bin for
PEMS or PAMS selection, as described in more detail in Section 5.1.3, Equipment Sampling in
The Field.
After equipment was classified in the weighted stratification bins, individual pieces of
equipment were selected for PEMS and PAMS instrumentation by ERG's Austin staff. Each
eligible piece of equipment was selected either as a primary, or "first choice" piece of
equipment, or as "backup" equipment in case one or more of the primary pieces of equipment
could not be tested. After the equipment information entry was verified and PEMS and PAMS
instrumentation selections were made, all information was updated in the master equipment
inventory spreadsheet, and the spreadsheet was posted back to the project secure server for onsite
field staff. The process for selecting equipment is described in more detail in Section 5.1.3.
A complete list of equipment inventoried for each establishment in each phase of the
study, including equipment selection results, is provided in Appendix W. Establishment-specific
identifiers (names, addresses and ID numbers) have been removed from this list but will be
provided as part of the final deliverable to EPA for this study. Additional details regarding
onsite inventory tasks are provided in Appendix J.
3.5.2 Performing PEMS and PAMS Testing
After equipment selection for PEMS and PAMS instrumentation, the ERG onsite
installation manager scheduled instrumentations with the appropriate establishment contact,
sought permission to instrument the specific pieces of equipment, and asked the site contact
about each selected piece of equipment's general usage and anticipated usage during the
measurement period (as specified in questions 8-11 of Appendix B, Onsite Equipment
Questions). If the site contact indicated the equipment was not to be used or not available for
testing during the anticipated measurement period, an alternate piece of equipment was selected
for instrumentation from the list of backup equipment. Backup equipment selections were made
according to numerical rankings, as shown in Appendix W. PAMS installation and PEMS
testing appointments were scheduled and conducted independently.
3.5.2.1 PAMS Installations
PAMS installations were performed at the outset of each phase of testing, prior to the
start of PEMS testing. PAMS units were then revisited, monitored and maintained throughout
3-24
-------
the PEMS test period for each phase of the study. Teams of two to three people performed the
PAMS installations, and on average installed two PAMS per day (typically at the same site or
establishment). Both Corsa and Isaac PAMS units were used during each phase, and engine
speed (revolutions per minute, or RPM) was collected in several different ways, including via a
Capelec voltage processor connected to the equipment's battery, an optical sensor directed at a
rotating object to which reflective tape was applied, a magnetic pickup mounted near a rotating
object to which a magnet was affixed, or by non-destructively tapping into the equipment's
electronic tachometer signal. Non-destructive taps were accomplished using supplemental
connectors with harnesses which connected inline with the equipment's original harness.
In ES Phase 1, an attempt was made to acquire two independent RPM signals for each
PAMS installation, usually via a Capelec RPM device and an optical or magnetic RPM device
or the equipment's own RPM signal (using the non-destructive tap). Although it was recognized
that collecting two RPM signals would pose additional complications during data processing
(i.e., trying to determine the "correct" RPM), this was done to in order to increase the likelihood
of obtaining a valid RPM signal in the event one of the signals was lost or incorrect. Dual RPM
collection (and the use of Capelec devices) was largely abandoned in the second and third phases
of the fieldwork (ES Phases 2 and 3).
In ES Phase 1, PAMS were either left "active" or configured to switch into "active" mode
(or into "standby" mode) based on RPM, equipment battery voltage changes, or a switched
power signal. For Corsa dataloggers, standby mode was found to be problematic for equipment
that was inactive for several or more days, as this could drain batteries that were already weak
(the standby current drain for Corsa dataloggers was approximately 175 mA). Therefore, in ES
Phases 2 and 3, Corsa dataloggers were installed to be "dead" (draw no "standby" power) when
the equipment was turned off. This was accomplished by using switched power as the main
power source for the logger. For the Isaac dataloggers, installation teams typically attempted to
provide constant input voltage to the unit, with a supplemental switched 12V signal to indicate
key on / key off condition. Occasionally, it was not possible to use this configuration, and either
switched power was used for the main power, or a constant (non-switched) power source
powered the PAMS unit.
For units that went into "standby" mode or were switched entirely dead, some delay was
experienced when the equipment was first turned on. Testing showed that on average, Corsa
dataloggers required approximately four seconds to awaken from "dead" mode after switched
power was provided, and Isaac dataloggers required about 2 seconds to awaken from "dead"
mode. Once a Corsa or Isaac unit was "active" (recording data), engine RPM data had an
3-25
-------
approximately 2 to 3 second delay. Results of data acquisition delay testing are provided in
Appendix AE, PAMS Acquisition Delay Test Results. Details regarding configurations,
equipment used, calibrations performed and other details on all PAMS installations are provided
in Appendix U, PEMS and PAMS Testing Details.
3.5.2.2 PEMS Testing
Emissions measurements were usually performed by three person teams, with a fourth
field technician providing fieldwork management, PEMS rack mounting support, testing
oversight and other fieldwork logistic support. Emission measurements were typically gathered
over a one-day period in an effort to collect emissions information throughout the equipment's
entire work day. PEMS installation, operation and maintenance was scheduled and performed in
such a way as to minimize interruption of equipment use. PEMS instrumentation teams
generally performed installations during each site's non-working hours (after the equipment was
no longer needed for that working day). Hence, PEMS installations usually took place the
evening prior to the day of testing, and the instrumentation team would then arrive the next
morning at least two hours prior to site operations to warm-up, calibrate and verify the
equipment's operation prior to emissions testing. This schedule usually allowed the PEMS
instrumentation team to obtain cold-start emissions data for both gaseous pollutants and PM.
However, occasionally it was not possible to obtain cold start data because of inaccessibility of
equipment during facility off-hours. Whenever possible, installations were scheduled and
performed at times when equipment was to be dormant for 12 or more hours (in order to get true
"cold-start" results).
For every PEMS and PAMS instrumentation, detailed information pertaining to the
equipment being instrumented as well as calibration, filter sampling and other PEMS and PAMs
test details were collected on PEMS and PAMS instrumentation forms, shown in Appendix E. A
compilation of all this information collected for all PEMS and PAMS instrumentations
throughout the study is provided in Appendix U. The information in Appendix U will also be
provided as part of the MSOD data submission for this project. In Appendix U (and elsewhere),
the primary key field for each test is an eight digit test ID number. The first four digits of the test
ID correspond to the last four digits of the unique establishment ID (assigned by NuStats) and
the last four digits of the test ID correspond to the last four digits of the chassis serial number of
the equipment being tested.
For both PEMS and PAMS testing, field staff followed the methodology provided in the
project QAPP and associated standard operating procedures (SOPs). Significant detail was
3-26
-------
associated with PEMS and PAMS instrumentations, and procedural training was provided to all
team members prior to fieldwork. PEMS and PAMS SOPs are provided in Appendices F and G,
respectively. Training and SOPs were also provided for other facets of PEMS and PAMS
testing, including gravimetric filter handing (Appendix L), oil and diesel fuel sampling
(Appendix M), onsite manager responsibilities (Appendices J and K) and QA checks of PEMS
results (Appendix I). Quality control steps were integrated into all fieldwork processes in an
effort to identify and correct any problems as soon as possible.
Daily calibrations of PEMS equipment were performed and results are provided in
Appendices Q, R and S. In addition, laboratory calibration and verification of flowmeter
measurements and SEMTECH gaseous measurement linearity results were performed by
Sensors, Inc. prior to and following each phase of fieldwork and are being compiled by EPA.
Laboratory results which have been made available are provided in Appendices AB and AC.
PEMS installation teams attempted to collect diesel fuel and crankcase lubricating oil
samples on all pieces of equipment that received emissions measurements. Occasionally, it was
not possible to collect a sample because of a locked or inaccessible filler neck, anti-siphon
equipment, or an inability to access the engine crankcase oil through the oil dipstick or fill tube.
All fuel and oil samples which were collected were stored and shipped in appropriate containers
provided by EPA, according to guidelines listed in Appendix M, Oil and Diesel Sampling SOPs.
Fuels samples were gathered so that adjustments can be made to the emission measurements
based on fuel properties (i.e density, C/H ratios, etc). Oil samples were taked because they might
be able to determine the engine status or wear. All fuel and oil samples which were collected
were stored and shipped in appropriate containers provided by EPA.
All samples were stored in the Sensors support trailer and returned to EPA at the end of
each phase of testing.
Gravimetric filters were loaded into the filter sampling equipment prior to each test and
switched during breaks in equipment operation. Gravimetric filters were transported and
handled in plastic holders sealed in Ziploc plastic bags and carried/stored in small ice-chest
coolers. However, since these filters were loaded in the field, they were briefly exposed to
ambient contamination as they were removed from their bags and holders and placed in the
sampling equipment. An effort was made to shelter filters from wind during loading (to prevent
filter contamination). Although wind speed was not recorded during the study, dynamic and
field blanks were collected to help quantify the extent of ambient contamination on the PM
results. Dynamic and field blank results are presented in Section 6.2.1 (PM Filter Weights).
3-27
-------
As described in Section 3.4.1, an attempt was made to capture PM emissions from the
first 10 minutes of cold-start operation with the first filter used each day. The second and third
filters captured continued warm operation or hot start emissions (generally 20-minutes sampling
for the second filter and 30-minutes sampling for the third filter). If testing permitted, filters
were switched during the day's operations to obtain additional PM data. The fourth and
subsequent filters captured continued warm operation or hot start emissions. Filters were not
dedicated to a specific type of operation, and operations (and work load) varied from test to test
and filter to filter. Appendix V (PEMS Filter Log) lists mass weights and sampling information
for each sampled filter, including sample type such as cold start, warm operation, hot start, blank,
etc. Appendix AH (PEMS Measurement Results) lists sample times and by-filter emissions for
all sampled filters.
Significant differences were seen in equipment operation and workloads from job site to
job site and from test to test. For instance, at one job site a backhoe was used to gently move
piles of loose, fine aggregate soil, while at another job site an excavator was heavily loaded
while excavating deep, rocky compacted soil. In addition to job demands and soil conditions, the
overall operation of the equipment varied widely from site to site (and person to person). For
example, in one test, a backhoe was gently used move light soil and dig a residential trench,
while at another job site a backhoe was operated so aggressively that violent shaking and
vibrations broke several mounts and flipped breakers in the test equipment, rendering the test
equipment inoperable for several days. As another example, at one job site (a commercial waste
collection facility), an excavator with an inoperable transmission was "permanently" outriggered
onto railroad ties and used only to compact trash in a roll-away dumpster, while at another
jobsite where a commercial excavation was taking place, excavators were very heavily loaded
while removing rock and compacted, damp soil and also used with jackhammer attachments to
assist with rock excavation. Even soil moisture content varied greatly throughout the study, and
work at jobsites was frequently suspended during our study due to heavy rains. It was beyond
the scope of this pilot study to attempt to assign and classify the types of workloads based on
worker operation, soil types, moisture content or types of operation. However, as described in
Section 6.2, emission results are presented in a work-basis and in a fuel-basis in an attempt to
normalize emissions based on work output. Some limitations to this methodology are also
presented and described in Section 6.2 (PEMS Measurement Results).
Operation of all PEMS test equipment was continuously monitored by PEMS
instrumentation teams throughout the test day. During breaks in usage of the construction
equipment, team members would attempt to access the PEMS to refill the generator, replace the
gravimetric filters and calibrate the SEMTECH-DS, as necessary. Real-time monitoring of test
3-28
-------
parameters was performed using remote laptops connected to the PEMS rack wirelessly in order
to identify and correct any data or equipment issues. In addition, test files were extracted during
and immediately after each test, processed and reviewed for data quality issues or problems.
At the completion of ES Phase 1 fieldwork, the team revised the Emissions and Activity
Measurement sections of the QAPP (as well as the associated SOPS) based on field-testing
experience. SOPs were continually revised and redistributed throughout each phase of testing as
procedural and equipment refinements were made.
3.5.2.3 Testing Challenges
Significant challenges were encountered throughout the study which reduced the overall
productivity of the field teams. These challengers were not unexpected, since the team was
attempting to perform gaseous and PM testing on in-use equipment in the field without
interfering with everyday construction site operations. In addition, the PEMS rack described in
Section 3.4.2 was designed and built specifically for this study, so this study was essentially an
initial field validation of the PM and gaseous sampling equipment integrated in the rack.
Previous testing with these systems has primarily been in on-road and laboratory environments.
Non-road testing subjected the equipment to extreme conditions not typically encountered in
other test environments. Highlights of some of the challenges encountered are provided in the
following sections:
Logistical Issues This was an "in-use" study in which it was imperative to not interfere
with the operations of the participating facility (construction company). Consequently, test
scheduling had to be extremely flexible and tailored to each company's operations (and
downtime). Frequently, it was difficult to obtain access to job sites and equipment during off-
hours (nights/weekends), and many times on-site contacts did not know upcoming work
schedules for equipment to be tested, which made staffing and test scheduling very challenging.
Heavy rain and mud though much of the fieldwork prevented site operations and equipment
testing and hampered test planning, since PEMS installations were not performed if no site
operations were planned or if the chance of appreciable rain over the test period was significant.
Occasionally, equipment used during the day was transported to and domiciled nightly at
the establishment's headquarters. The equipment would then be transported back to a job site the
following day. This usage pattern generally prevented instrumentations from taking place on
these pieces of equipment, since the equipment usually could not be transported with the PEMS
rack installed (due to height concerns) and insufficient time was available for instrumentation in
the mornings after the equipment had been transported to the site but before it was used.
3-29
-------
Often, after a test was initiated and the equipment was placed into service, the equipment
was not available again to field team members until much later in the day or the evening. Any
equipment problems which arose during these periods of inaccessibility could not be addressed,
including maintenance issues such as filter changes and generator refueling. As safety and onsite
operations permitted, team members did maintain and calibrate equipment, change filters and
refuel generators whenever equipment use was paused.
Variability in onsite operations occasionally resulted in equipment being needed before
the PEMS installation had been completed. In these instances, the PEMS hardware (generally
the complete rack, less generator) was secured and the equipment was released for service with
an incomplete installation. The installation was then completed later in the day or evening (once
the equipment was no longer in use) for testing over the next usage cycle.
Finally, variability in usage made it difficult to predict which installations would be
"safe" installations. Some equipment was used so aggressively that the PEMS rack and PEMS
and PM equipment was significantly damaged during the work day. Typical damage included
broken welds, dislodged electrical boards, broken electrical connections, internal SEMTECH and
PPMD damage, etc.
Equipment Issues - Since the PEMS rack had been specifically constructed for this
study, the durability of the system integration had not been field-verified prior to the study. As a
result, some failures were encountered during testing, and a significant portion of the fieldwork
effort was dedicated to repairing and maintaining the equipment. Highlights of the equipment
problems encountered are provided below, and additional details of PEMS equipment issues
encountered during testing are provided in Appendix Y, PEMS Data QC Results.
Electrical problems were encountered throughout the study due to circuit boards
and wiring becoming dislodged and disconnected during testing, excessive
electrical loads, and faulty ground and electrical leads. The impact of these
electrical problems ranged from loss of flows necessary for gaseous or PM
sampling to loss of an entire test (i.e., after a power supply failure or system shut
down). Circuit breakers used in the PEMS rack would also occasionally shut off,
apparently due to extreme shaking and vibration.
Excessive vibration and rough equipment usage resulted not only in electrical
problems but also problems such as occasional loss of MPS calibration and loss of
various bench signals during the study. In addition, the NDIR bench signal would
intermittently fail during a test, resulting in a loss of CO, CO2, oxygen and RPM
data for the remainder of that test.
MPS flow control and gravimetric sampling system malfunctions prevented PM
filter collection for some tests, and gaseous sampling system leaks were also
3-30
-------
encountered. This was likely due in part to vibrations and moisture encountered
during testing, MPS controller board failure (vibration), and also disconnection
and kinking of tubing connecting the various systems in the integrated PEMS
rack.
Since the PEMS team had to maintain distance from the in-use equipment, remote
control (via wireless antenna) of the PEMS rack and filter switching system was
occasionally lost, depending on the distance from the equipment and the PEMS
monitoring team.
All issues were identified and corrected when they occurred, and tests were redone as
necessary. Throughout the study, the team continued to enhance the equipment to withstand the
rigors of this type of testing, and the experience gained in this study can be used to refine the
integrated package for future non-road studies.
Installation Issues
As described in Sections 7.2 and 7.3 (Lessons Learned), installing and operating the
PEMS rack and associated equipment was a challenging learning process. One obstacle simply
involved installing the equipment. Because the PEMS rack contained the SEMTECH DS, MPS,
gravimetric sampler, compressor and all other equipment needed for PM and gaseous sampling,
a crane was required for all installations. This was particularly challenging at some locations
where truck access was limited. The mounting location for each installation varied depending on
the equipment to be tested, and all installations required mounting points for rack tie downs.
Four people were typically required to mount the rack onto the nonroad equipment. Even
finding a location with suitable room and strength to support the rack was not always possible,
and occasionally wooden platforms were fabricated in order to support the PEMS rack on the
nonroad equipment to be tested. Sufficient space was not always available to install the PEMS
in a safe and secure manner, in which case that piece of equipment was deemed untestable. It is
possible that some of the equipment not testable with the PEMS rack would be testable if the
individual PEMS system components (SEMTECH DS, MPS, gravimetric sampler, etc) were
individually mounted in separate locations.
Once installed, hooking up the exhaust collection system, sample lines, nitrogen and FID
fuel lines, power supply lines, RPM sensor and calibration lines and securing the tie-downs was
typically challenging since the PEMS rack was mounted on top of the nonroad equipment.
Access to the rack required the use of ladders and climbing/standing on various areas of the
equipment to reach the rack (see Figure 3.4-2). If a system malfunctioned while on the nonroad
equipment, disassembly of the PEMS rack while installed or removal of the rack was required,
generally adding several or more hours to the installation process.
3-31
-------
Also described in Section 7.3, high exhaust flow rates and temperatures frequently
resulted in the high-temperature silicon exhaust connections melting and/or blowing off the
equipment's tailpipe. These melting exhaust connections resulted in contamination of the PM
sampling system, including internal MPS contamination which could result in loss of
proportional sampling and would require full disassembly for cleaning and recalibration.
Due to the effort and time required for test equipment installation and preparation, in-use
test monitoring and post-test equipment maintenance, daily back-to-back testing was not possible
during this study. However, daily (back-to-back) testing would be possible with additional staff
beyond the 4-person teams described in Section 3.5.2.2. In addition, multiple complete PEMS
test systems would be required so equipment could be prepared and installations could be
initiated while existing testing was still underway (i.e., two teams working independently with
complete equipment sets).
Significant time was spent during each installation connecting the exhaust system,
securing the PEMS rack, installing the RPM pickup device, gas bottles, generator and power
lines, calibrating the gaseous sampling system and MPS system, and ensuring all systems were
functioning prior to the start of the test. Maintaining, calibrating and operating the PM and
gaseous PEMS system is complex and requires equipment expertise and a significant amount of
attention to detail. While certainly a manageable task, staff with adequate training and
experience are required, and testing expectations should include the above challenges, in
particular when performing this work in "real world" settings. Sufficient staff with dedicated
duties should be provided for testing support, with heavy use of SOPs, guidelines, and checklists
to ensure safety and proper test preparation and procedures.
3.6 EOI Phase 1 Initial EOI / EAM Interview
This effort involved the recruitment of construction businesses from PSU 1 for the
purpose of PEMS and PAMS instrumentation. To accomplish this, a sample of businesses was
drawn and called by a centralized telephone facility. For most of the sample,10 the EOI was
administered (questions 1 through 13 of Appendix A), followed by the administration of the
incentive experiment, and finally the administration of the Equipment Sample Interview (ESI)
with the objective of recruiting qualified establishments to participate in the inventory and
instrumentation phase of the study (questions 14 through 22 of Appendix A). During the ESI,
data on the qualified businesses (e.g., business name, site selected for the inventory and address,
10 A small portion of the sample involved re-calling the businesses that participated in the Phase 1 Pilot. The Pilot
only involved the administration of the EOI and was conducted in early 2007.
3-32
-------
contact name and phone number, etc.) that completed the EOT and agreed to be inventoried were
electronically transferred to ERG by NuStats for scheduling.
It is important to note that the eligibility requirements for the EOT in Phase 1 differed
from those of the ESI in Phase 2. Both required confirmation of the business name, operating in
the construction sector, having more than one employee, and owning or leasing diesel powered
nonroad equipment. But the Phase 2 ESI employed an additional eligibility criterion:
The establishment had to be a prime contractor.
The reason for this restriction (which in Phase 3 was lifted in order to increase eligibility rates)
lies in the selection probabilities of equipment. We were trying to control the paths by which a
piece of equipment could fall into the sample for instrumentation. For those establishments that
served as both a prime and subcontractor, the establishment would have multiple chances of
selection - one as a prime contractor, plus multiple chances as a subcontractor (one per
subcontract for each distinct prime). By limiting the ESI to prime contractors, we would
effectively limit the selection probability of an establishment (and its equipment) to a known
value.11
3.7 ES Phase 1 Emission and Activity Measurements
Figure 3.7-1 shows the sequence of events in ES Phase 1 of fieldwork, performed in the
county of Jackson, Missouri. Phase 1 fieldwork was initiated with on-site inventories (conducted
June 4 through 14 , 2007) and PAMS installations began shortly after the start of on-site
inventories (PAMS installations were initiated June 6th and were completed June 11th).
Inventory information regarding equipment test eligibility was transmitted from the inventory
team to the PAMS installation team on a daily basis. To maximize the amount of activity data
collected, all PAMS installations were performed by ERG and Southern Research Institute (SRI)
personnel prior to the commencement of PEMS testing. PEMS testing, involving staff from
ERG, Sensors, Inc., Southern Research Institute and EPA began after the completion of PAMS
installations. PEMS testing in Phase 1 was conducted from June 15* through July 24* , 2007.
PAMS revisits and downloads were performed periodically during PEMS testing, as schedules
permitted. Hence, PAMS revisits were generally performed on weekends (and on days when a
PEMS test was not being conducted).
11 Unfortunately, due to the extremely low eligibility rates we encountered using the prime contractor criterion, we
were forced to drop this restriction for the sake of being able to implement the instrumentation to attain our desired
sample size.
3-33
-------
Week beginning:
6/4
11 Sites
Figure 3.7-1 Emissions and Activity Measurements in ES Phase 1
Phase 1, Summer 2007
Task
Onsite Inventory
PAMS Installs
PEMS Tests
PAMS Removals
-T i
j^7 PAMS Installs
6/11
6/18
6/25
^
[7PEMSTests ]
^ '
1 PAMS Removals
7/2
7/9
7/16
*-
7/23
3.8 Integrated Sample Surveys, Phase 2 EAMS
According to our study design, Phase 2 was to proceed with conducting the establishment
sample EOT with PSUs 2 through 5. However, analysis of EOT Phase 1 demonstrated that a full
integration of the Phase 2 Establishment and Equipment Samples (i.e., the EOT & ESI) into a
single, unified design would lead to cost efficiencies and produce more accurate reporting.
The reason for integrating the EOI-ESI sample design was partly due to the skewed
nature of the establishments according to the measure of size (MOS) which led to a large number
of self-representing units needing to be shared by the Establishment and Equipment Samples.
Another factor was the finding of lower net yield rates relative to what we had planned and
expected12. The Phase 1 Establishment Sample yield rates were half that of the expected rates,
and meant that twice the sample than was originally planned would be needed for the Phase 2
Establishment and Equipment Samples. Taken together, these factors suggested that even a
census of all establishments in PSUs 2-5 would fail to produce the targeted number of EOIs in
order to achieve the study's Equipment Sample targets.
Based on these findings, the study design was modified to integrate the establishment and
equipment sample into a single, integrated interview process. Table 3.8-1 presents the revised
sample size goals by study phase and sample type that would be needed to achieve the target
number of completions for the pilot. It reflects our integrated design and Phase 1 Establishment
Sample experience by showing censuses conducted for Inventory and Instrumentation for Phases
The net yield rate is the 'bottom line' number of sampled establishments required to secure a single completed
EOI survey; it combines sample losses from ineligibility, screening nonresponse and interview nonresponse.
3-34
-------
II (PSUs 2&3) and III (PSUs 4&5). A second stage sample was estimated to be needed to be
drawn only within PSU 1 for the Inventories and Instrumentations in order to achieve the
targeted sample size specified in the survey objectives. For the other PSUs, the revised plan
involved conducting censuses of all establishments in order to maximize the possibility of
achieving the targeted numbers of EOIs and instrumentations. As shown in Table 3.8-1, a total
of 3541 selections were expected to be needed to complete the EOIs for the Establishment
Sample and the EOI portion of the Equipment Sample.
Table 3.8-1 Revised Total Sample Needed to Achieve Sample Targets
TYPE OF DATA
COLLECTION
EOIs
Inventory &
Instrumentations
ESTABLISHMENT SAMPLE
PHASE 1
243
0
PHASE 2
n/a
0
EQUIPMENT SAMPLE
PHASE 1
404
37
PHASE 2
1522
74
PHASE 3
1372
74
TOTAL
3541
185
The process for the integrated sample mirrored the establishment sample process for the
first PSU with the exception of an extended interview to select a site at which an in-field
equipment inventory would be conducted and equipment selected for instrumentation. All
prospective respondents received an advance letter to alert them about the study and encourage
their participation. This step of data collection began on June 15, 2008 and continued through
August 2, 2008.
Figure 3.8-1 shows the sequence of field events in ES Phase 2, which targeted
establishments located in the counties of Linn and Scott, Iowa. ES Phase 2 inventories began
September 5, 2007, and PAMS installations began September 10th. ES Phase 2 PEMS testing
was conducted from September 17th through October 27th, 2007.
Although geographic considerations were taken into account when scheduling
inventories, PAMS installations and PEMS testing, these counties were contiguous and activities
were not segregated based on county. As can be seen in Figure 3.8-1, inventories continued for a
much longer duration than in ES Phase 1, primarily due to the larger number of establishments
participating in Phase 2, a result of a 2-county census being performed. Later during the phase,
as the rate of inventory appointments decreased, the inventories were scheduled in "groups",
which allowed inventory personnel to assist with PAMS monitoring support and PEMS testing
tasks.
3-35
-------
Figure 3.8-1 Field Schedule for ES Phase 2
Phase 2, Fall 2007
Task
Onsite Inventory
PAMS Installs
PEMS Tests
PAMS Removals
Week Beginning:
9/3
9/10
10/22
3.9 Enhancements Prior to Phase 3
As a pilot project, the design strategy was to capitalize on experience with previous
phases of the project and build upon that experience before proceeding with the subsequent
phase. Previous enhancements to the study design, based on the earlier study phases included
revising eligibility criteria13 and integrating the establishment sample EOT and equipment sample
ESI into a single survey application. As a result of the EOT Phase 2 effort, in order to meet data
collection goals, a decision was made to explore the viability and utility of drawing a
supplemental sample from a file provided by Equipment Data Associates, Inc. (EDA).
14
The focus of this section is on the design enhancements that were made in acquiring and
processing this supplemental database into the PSU 4 and PSU 5 sampling of establishments.
This includes a review of the process for obtaining and processing the EDA data and assessment
of the EDA data for PSU 4 and PSU 5 sampling.
3.9.1 EDA Data Acquisition and Processing for Sampling
EDA data was purchased for PSUs 1-5 using the same specifications employed by EPA
in its earlier acquisition of EDA data for PSU 1 (Jackson, MO). Data for PSU 1 was purchased
as an update from the previous purchase date (July 31, 2003). For the remaining PSUs, data was
purchased anew, as there was no previous EDA acquisition for those areas.
13 Certain eligibility criteria were revised, including establishments that (according to the SSI sample frame)
reported having zero employees were no longer excluded from the study as a result of Phase 1 and establishments
that were non-prime contractors were considered eligible during Phase 2.
14The EDA data provide a list of establishments that have financed construction equipment purchases. The data set
contains identifying company information, equipment pieces financed (by equipment type) and date of transaction.
3-36
-------
Once the data were obtained, they were reviewed for integrity and completeness. This
was accomplished through quality checks (e.g., running summary reports by PSU and verifying
against existing EDA-produced reports). The data set structure was then organized to match
against the SSI data set. This was done by merging the EDA data with the SSI sampling frame
data sets. In doing so, the SSI and EDA data were treated as two collections of records
aggregated across PSUs 1 through 5 (rather than 10 distinct data sets requiring pair-wise
merging). This step reduced processing time and guaranteed a standardized approach for all
PSUs.
This merging process resulted in three distinct sets of establishment records:
Records that appear in the SSI frame only.
Records that appear in the EDA frame only.
Records that appear in both SSI and EDA lists.
The result of this assessment was a merged data set of 2,209 records, comprising the
sample frame for EOI Phase 3. Accordingly, all 2,209 records in the combined SSI-EDA frame
were loaded into the sample management system for the issuance of advance letters and
subsequent calling by the telephone facility.
3.9.2 Improvements to Sampling Protocols
Merging the EDA-SSI data onto the ESI and EOI survey data for PSUs 1-3 provided a
unique opportunity to explore important survey design and survey quality issues. The analysis
involved two steps with one related to sampling frame development, design parameter
assessment and weighting, and the other focusing on response error and coverage.
For the first analysis we focused on the use of EDA data to improve understanding of
eligibility and nonresponse, as described below:
Eligibility. To begin, the EDA data would aid in determining the eligibility status of a
portion if not all of the establishments that were not able to be screened using the
conventional CATI calling protocols. This would allow the development of enhanced
estimates of eligibility by PSU. Secondly, the correlates of eligibility status could
also be explored. A key component of this analysis would be the ability of the EDA
equipment list to identify establishments that are eligible for ESI/EOI prior to mailing
and calling. Such "pre-screening" via EDA could potentially lead to a high level of
3-37
-------
efficiency. Analysis would also explore opportunities to more efficiently partition
the sampling frame into groups of establishments that exhibit highly differential
eligibility rates across strata. This will be important for developing optimal
allocation designs that take into account the high cost of screening.
Response Rate Analysis. The EDA data was anticipated to also provide new extant
data on most of the establishments appearing in the SSI sampling frame. This would
allow an additional avenue for exploring correlates of nonresponse at the screening
and interviewing stages of data collection. The extent to which nonresponse for EOT
and ESI is related to establishments with certain patterns of equipment purchases (i.e.,
by equipment type), and/or certain equipment inventories (by amount and
configuration of equipment) would also be explored. The results could provide
insight into the risks of nonresponse bias.
Weighting adjustments. The nonresponse analysis was expected to yield specific
recommendations for enhanced nonresponse weight adjustments that rely on the use
of the EDA data (which was previously unavailable and therefore not an option for
such adjustments).
To explore coverage, an analysis was conducted on three principal subgroups that
comprise the EDA-SSI merged data set: (1) matched records; (2) records in SSI but not in EDA;
and (3) records in EDA but not in SSI. Results of this analysis are presented in Section 4.
3.9.3 ES Phase 3 Data Collection Enhancements
A delay between ES Phase 2 and ES Phase 3 of this work assignment provided the team
an opportunity to refine data collection equipment and procedures prior to ES Phase 3 fieldwork,
based on information learned during ES Phase 1 and II testing. The team considered several
changes, but eventually focused on enhancing PEMS RPM collection and PEMS ECU data
collection. These two enhancements are described in the following subsections.
3.9.3.1 RPM Collection Enhancement
"Capelec" RPM measurement devices were the primary RPM measurement device used
during PEMS tests in ES Phases 1 and 2 of this work assignment. These devices, which were
specifically purchased for use in this study, determine an engine's RPM by processing voltage
fluctuations measured at the terminals of the nonroad equipment's battery (these fluctuations
result from alternator output fluctuations during equipment operation). Although these units are
3-38
-------
very simple to install and operate, only limited success was achieved obtaining an accurate and
reliable RPM signal. Specifically, RPM signals were often erroneous and erratic (would drop
out) during equipment operation, and they would jump to a high level when the equipment was
turned off. It appears much of the Capelec's unusual behavior was likely due to the unit picking
up signals from the PEMS test equipment, rather than true RPM information from the equipment
being tested.
Conversely, optical and magnetic RPM collection equipment were the primary RPM
collection devices used during ES Phase 1 and 2 PAMS testing and were shown to provide a
fairly accurate and reliable signal, although installation procedures were somewhat more time
consuming. However, because of the reliability and universal applicability of optical sensors for
RPM collection, the ERG team and EPA decided to use optical sensors for PEMS RPM
acquisition during ES Phase 3 testing. The ERG team and EPA worked together to identify
dedicated optical sensor holders which could be attached to high-powered rare-earth magnets
(acquired separately). These high-powered mounts proved to be capable of securely attaching
the optical sensors for both day-long PEMS testing and month-long PAMS testing. Figure 3.9-1
shows an optical sensor mounted using a magnetic base, with an extension allowing the sensor to
point at a rotating pulley (not shown). Although a long extension was used in this particular
installation, in general an effort was made to minimize the distance from the optical sensor to the
sensor's mounting base, in order to minimize vibration and mis-alignment potential of the optical
sensor. Figure 3.9-2 shows an optical sensor mounted using an aluminum bracket, as was done
when equipment configurations permitted.
Figure 3.9-1 Optical Sensor on a Bracket With a Magnetic Base
3-39
-------
Figure 3.9-2 Optical Sensor on an Aluminum Bracket
3.9.3.2 Enhancement of Collection of Electronic Control Unit Operation Data (ECU
data) for Electronically-Controlled Equipment
The majority of equipment tested in ES Phases 1 and 2 was mechanically-controlled,
rather than electronically controlled by way of an electronic control unit, or ECU. However, for
equipment which was electronically-controlled, attempts to collect ECU data during ES Phase 1
and 2 PEMS testing were not successful. The intent of this task, therefore, was to supplement
PEMS test procedures and equipment in order to allow the team to successfully collect ECU data
from electronically-controlled equipment that would be PEMS-tested throughout the remainder
of this work assignment.
Initial research suggested that although SAE J1939 is intended by SAE to be the
standard protocol for 2004 and later heavy-duty on-road vehicles (superseding the use of
J1587/1708), the J1939 standard thus far has seen limited penetration into the heavy-duty (on-
road) vehicle market and non-standardization appeared to be even greater in the non-road heavy-
duty equipment sector.
Because of the prevalence of Caterpillar equipment inventoried and tested during the first
two phases of this study, the ERG team decided to focus efforts on only Caterpillar ECU data
acquisition, in hopes that the information learned could then be applied to other equipment
manufacturers in future studies. Therefore, ERG, EPA and Sensors worked with Caterpillar in
acquiring "CAT ET" (Caterpillar Electronic Technician) equipment and software necessary for
ECU data collection on nonroad equipment. The project team established training sessions with
3-40
-------
Caterpillar for use of this equipment and software. These training sessions were also used as an
opportunity to explore alternative manufacturer-specific RPM-collection devices (such as
inductive flywheel signal pickups), although ultimately the decision was made to focus efforts on
use of optical RPM pickups, as this is universally applicable in the field.
After hardware and software for Caterpillar ECU data collection were acquired, new
procedures were developed to allow this data to be collected in the field during emissions testing
(rather than collection of data during engine diagnosis or repairs, the typical application for this
equipment). Due to time and budget constraints, it was not feasible to log the data directly into
the SEMTECH-DS. Instead, the CAT ET software was installed on a remote laptop for which
the "sleep" power mode was disabled (allowing the laptop to operate with the lid closed). This
laptop was placed in the cab of the equipment being tested, connected directly to the ECU CAN
port (via the Cat ET communication module) and to a power supply (taken from the PEMS
onboard generator). Since acquisition could not start until after the equipment was turned on,
remote control of the acquisition laptop was necessary and was achieved by way of a standard
Wi-Fi computer transmitter/receiver. Preliminary testing showed this configuration adequate for
remotely collecting ECU data from electronically-controlled Caterpillar equipment. This data
could then later be merged and time-aligned with the PEMS data for the same test.
3.10 ES Phase 3 EAMS
Figure 3.10-1 shows the sequence of fieldwork events in ES Phase 3 testing, which was
conducted with establishments located in the counties of Clay, Missouri and Shawnee, Kansas.
Because of the distance which separated these two counties, inventories, emissions and activity
measurements were conducted independently in each county (work for each task was completed
in Clay, Missouri before moving on to Shawnee, Kansas). This reduced the amount field teams
needed to commute long distances between the two non-adjacent counties.
Inventories in Clay County were conducted from June 30th through July 24th, 2008 and in
Shawnee County from August 4th through August 14th, 2008. PAMS were installed in Clay
County from July 7th through July 11th, 2008 and in Shawnee County from August 11th through
August 14th 2008. PEMS testing began in Clay County on July 23rd, and ended on Aug 26th,
2008 and in Shawnee County PEMS testing began on September 23r and continued through
October 10th, 2008. The quality of RPM data collected during ES Phase 3 was higher than that in
ES Phases 1 and 2, and although only one piece of electronically-controlled Caterpillar
equipment was tested, ECU data collection was successful for this test.
3-41
-------
Figure 3.10-1 Field Schedule for ES Phase 3
Phases, Summer & Fall, 2008
Task
Onsite Inventory
RAMS Installs
PEMS Tests
PAMS Removals
Week Beginning:
6/30
-
^r
,
==[6 PAMS Installs
6 PAMS Removals I
7/7
7/14
7/21
7/28
==
8/4
8/11
6 PAMS Installs ]
J
J 14 PEMS Tests
L
6 PAMS Removals L
8/18
8/25
9/1
9/8
, 8 F
1
1
9/15
3E MS Tests
9/22
^H
9/29
^
10/6
3.11 Data Processing, Analysis and Submission
At the completion of EOI/ESI data collection, NuStats processed the database,
conducting quality control and edit checks, using specifications established for this study and
analyzed the data regarding sample performance. Final data files were prepared and transferred
to ERG according to the protocols required in the statement of work.
All PEMS and PAMS data was monitored during collection followed by extensive QC
and processing performed after data collection was completed. Data processing, QC and
analysis steps are described for PEMS and PAMS data in the following sections.
3.11.1 PEMS Data Processing and QC
As mentioned in Section 3, the operation of PEMS test equipment was continuously
monitored by PEMS instrumentation teams throughout the test. Real-time monitoring of test
parameters was performed using remote laptops communicating wirelessly with the PEMS rack
in order to identify and correct any data or equipment issues. In addition, test files were
extracted during and immediately after each test, processed and reviewed for data quality issues
or problems. This real-time and "in-field" QC was intended to identify issues with exhaust and
MPS flows and gravimetric filter system flowrates, system temperatures and pressures, pollutant
concentrations and other measured and recorded parameters. An overview of the work done
with the data after fieldwork was completed follows.
3.11.1.1 Summary of PEMS Data Processing and Analysis
After testing was completed, all PEMS data was read into Statistical Analysis Software
(SAS) and thorough processing and quality control (QC) steps were performed in order to
3-42
-------
identify and flag any suspect data. QC involved SAS screening, a review of information
recorded in field logs and data processing logs and a detailed second-by-second review of all
recorded parameters by way of analysis of plots. Second-by-second review involved evaluation
of all gaseous pollutants, reviewing sampling system pressures such as the MPS inlet pressure
and SEMTECH pressures, evaluating all system flows including the exhaust mass flow rates,
calculated fuel flow rate, all MPS sampling flowrates and gravimetric filter flowrates, and
evaluating all system and sampling temperatures such as exhaust temperatures, external heated
line, chiller, cyclone, manifold and gravimetric filter temperatures, and ambient and internal
PEMS rack temperatures. Data identified as suspect has been eliminated from emissions
summaries reported in Section 6.2, PEMS Data. A detailed list of criteria used during review of
the second-by-second data is provided in Appendix I, and Appendix Y contains notes on the
review performed for each of the datafiles collected during PEMS testing.
In addition to the second-by-second review of PEMS data, the following analysis steps
were performed and corrections applied to the PEMS data collected during this study:
Time alignment - Time alignment was applied by Sensors during the initial processing
of the xml files into CSV files. All measured parameters such as gaseous pollutants, MPS
system flows, exhaust flow and RPM and GPS signals were aligned to within one second.
Uncertainty associated with time-alignment errors is included in the emission estimates provided
in Section 6.2.2 and in Appendix AO, Nonroad Error Estimates.
RPM scaling - RPM scaling was applied by Sensors during the initial processing of the
xml files into CSV files. Scale factors were based on the known RPM ranges for the equipment
being tested as well as the RPM verification and calibration information collected prior to and
after each PEMS test.
Estimation of brake-specific emissions based on engine RPM and fuel rate - ERG,
Sensors and EPA worked together to develop protocols to calculate cumulative brake-specific
fuel consumption (BSFC) mass-based gaseous and PM emissions using existing PEMS test data
(including exhaust flow, calculated fuel flow rate, RPM and MPS data), gaseous measurement
results and gravimetric filter results. Using this methodology, in SAS, ERG calculated brake-
specific emissions using the SEMTECH-DS' fuel consumption rate and the manufacturer-
specific BSFC vs. RPM curves (lug curves) for equipment that had been tested. The engine
power output for each observation was based on maximum power output at that RPM scaled by
the ratio of "measured" fuel consumption (via PEMS) to maximum fuel consumption (from lug
curves) . BSFC vs. RPM curves were acquired by EPA and provided to ERG. Although some
3-43
-------
of these curves were provided by the engine manufacturers for the specific engine models,
curves for other tests were "generic" curves for similar or multiple models of engines. These
"generic" curves likely provide a reasonable estimation of brake-specific emissions, although a
more accurate estimate could be obtained using curves specific to each engine model. These
"generic" curves likely add approximately 5% variability, mainly at low and high engine speeds.
The methodology used for calculating BSFC emissions is provided in Appendix AD,
BSFC Calculation Methodology. Use of the methodology developed for this work assignment is
based on the assumption that brake-specific fuel consumption is constant across varying engine
loads at any given RPM. Comparing results from this methodology to BSFC emission estimates
using in-house EPA laboratory results, EPA has shown this "constant fuel consumption"
assumption to be reasonable. A list of sources lug curves used for estimating BSFC emissions is
provided in Appendix Y. However, because some of these lug curves are confidential business
information, they are not included as an appendix to this report. Uncertainties associated with
estimating brake-specific emissions based on engine RPM and fuel rate and from the use of
generic lug curves are included in the emission estimates provided in Section 6.2.2. These
uncertainty estimates are derived and presented in detail in Appendix AO, Nonroad Error
Estimates.
By-test analyzer drift check (test record review) - PEMS gaseous span calibrations
were generally performed prior to and following each day's testing (and mid-day spans were
performed as field testing conditions permitted). However, occasionally a post-test span was not
performed (such as when testing or equipment failures occurred). For this task, ERG and
Sensors compiled all span results conducted throughout the study, in order to quantify the
amount of system drift occurring during the test on a percentage basis, through review of the
post-test span results. A summary of drift results (on a percentage basis) is provided in
Appendices Q, R and S for all three phases of the fieldwork. These drift checks were not applied
to the test data during post-processing of the data.
Comparison of known exhaust mass flowrates vs. measured flowrates: For engines
for which displacements were known, ERG compared both the idle flowrates and maximum
flowrates for each test with empirical data from engines with similar displacements operating at
similar speeds. Scale factors were used to correct for differences in displacement and engine
speed between measured and empirical data. This comparison of known versus measured
flowrates helped confirm the proper flowmeter diameters were used and entered into the test
setup screen and data processing calculations. However, in general, for the equipment tested
during the study, the idle flowrates were high enough that even the largest diameter flowmeter
3-44
-------
used in the study (5") would provide acceptable resolution at low (idle) flows, but conversely the
measured flows would not reach the maximum flow limits if a smaller than optimum diameter
flowmeter was used for a test. Guidelines for expected flowrates based on displacement and
engine speed are provided in Appendix F, PEMS Installation SOPS, and results of the
comparison of measured flowrates with expected parameters, for engines for which displacement
is known is provided in Appendix Y, PEMS Data QC Results.
MPS proportionality to flow review: In SAS, ERG plotted the MPS average sample
flow rate (iMPS_Average_Q, SCCM) against the exhaust mass flow rate (icMASS_FLOW,
r\
kg/hr) and calculated the best fit line slope, intercept, coefficient of determination (r ) and root
mean square error (RMSE) in order to assess the quality of proportionality. Results are provided
in Appendix Z, MPS to Exhaust Flow Proportionality Plots. These proportionality plots only
pertain to time periods when filter sampling was being performed.
Review of flowmeter usage during each phase of the fieldwork: In order to provide
information regarding the reliability and durability of flowmeters used during the study, ERG
summarized the usage time of each flowmeter tube during each study phase, provided in Table
3.11-1 below. Only one of each diameter flowmeter was used during each of the three phases of
the fieldwork. For example, only 3 flowmeters were used in ES Phase 1, one 3" flowmeter, only
one 4" flowmeter, and one 5" flowmeter. Any results of pre and post-fieldwork MPS and
flowmeter testing which has been made available is provided in Appendix AB.
Table 3.11-1 Flowmeter Usage During Each Phase of Fieldwork
ES Phase
1
2
3
2" usage
(mins)
208
"
"
3" usage
(mins)
784
3959
"
4" usage
(mins)
"
3522
5" usage
(mins)
24
696
3.11.1.2
Issues Identified during PEMS Data Analysis
In this section we discuss issues that were encountered during PEMS data processing and
analysis, and steps taken to resolve each issue.
3-45
-------
Post-fieldwork laboratory verification of PEMS equipment showed a bias in the 5"
flowmeter used during ES Phase 3: The 5" flowmeter used during Phase 3 of the fieldwork
failed the subsequent February 2009 calibration, apparently due to an incorrectly installed Torbar
exhaust flow measurement pitot tube in the exhaust flow meter. The incorrect installation of the
pitot tube would bias exhaust flow readings from this flowmeter low by approximately 8%.
However, review of ES Phase 3 testing shows the Torbar pitot tube was incorrectly installed after
test 8418_0961, which was conducted on August 25th, 2008 (the Torbar had been removed for
cleaning after an exhaust boot melted and contaminated the sampling system). However, it was
determined that this was the last test in which this 5" flowmeter had been used, so no subsequent
tests were affected and no data corrections were necessary.
Post-fieldwork laboratory verification of the PEMS equipment showed a bias in the
mass flow controller used during ES Phase 3: The mass-flow controller used for proportional
exhaust sampling in ES Phase 3 was removed from service August 26th, 2008 and sent to the
manufacturer for repairs. During this repair process, different coefficients were entered into the
firmware which resulted in a small (approximately 10%) bias in the gravimetric filter mass
controller flow rate. However, investigation of this issue revealed that no correction to the PM
data was necessary, since this bias in the gravimetric filter flow rate only affected the amount of
dilution added to the sample after it was collected, and hence the entire PM sample was still
passed through the PM filter. The total PM collection rate and the overall dilution ratios used for
determining tailpipe PM emission rates were unaffected.
Invalid time stamps were identified for several of the tests: - PEMS second-by-
second timestamps for some tests in ES Phases 2 and 3 were found to be erroneous, most likely
due to a weak or dead battery in the SEMTECH-DS' internal clock. In order to correct for this,
in SAS ERG corrected all second-by-second times using the GPS timestamp's Greenwich Mean
Time (GMT) value corrected to central daylight time (CDT).
RPM data issues: As described in Section 3.9.3, RPM acquisition was problematic
during the first two phases of the study, when the team used a device which calculated an RPM
signal based on slight changes in voltage readings measured at the battery. These problems
prompted the team to use an optical RPM collection device during ES Phase 3 of the study.
Although using the optical RPM sensor during ES Phase 3 greatly increased the reliability and
accuracy of the RPM signal, RPM data corrections were necessary for all three phases of the
study, to correct the following problems:
3-46
-------
Erroneous or spikedRPM- RPM spikes (unreasonably high transient values) and
suspect RPM values (RPM higher or lower than expected based on measured data
such as exhaust flow rate) were seen in the data, both during engine-off periods
and during engine operation. Erroneous data during engine-off periods were most
common with the Capelec RPM device (ES Phases 1 and 2), likely due to the
processor misidentifying signals from the PEMS hardware or portable generator
as engine RPM. All erroneous and spiked RPM data was corrected as described
below.
RPM scale - RPM calibration checks were performed prior to and following each
PEMS test, whenever possible (equipment failures or other factors sometimes
prevented pre or post-test calibrations). Using this calibration check information
collected in the field along with knowledge of engine operating ranges and
manufacturer specifications, RPM scale factors were applied during post-
processing and verified during data analysis. Final RPM scale corrections were
applied in SAS, as needed.
Missing RPM- RPM data is missing during certain segments of some data files.
This missing data was caused by loss of signal from the RPM collection device
(due to a malfunctioning processor or misaligned sensor) or due to equipment
failures such loss of the AMBII board signal in the SEMTECH-DS, through
which the RPM signal was routed. Segments of test data where RPM was
missing were identified by manual review of plots and second-by-second data,
and when RPM data was missing, a new RPM field was provided as described
below.
RPM issues such as spikes and missing or erroneous values were identified during review
of plots and data such as pollutants and exhaust flow rates from each test file. All original RPM
data was kept intact, but a new "corrected RPM" field was added to each test file where RPM
corrections were made. If possible, spikes and other transient RPM problems were corrected
using interpolation between valid points (verifying operating range using exhaust mass flow
rate). If a significant portion of the PEMS RPM appeared invalid and PAMS or ECU data was
available (i.e., PAMS or ECU data was collected concurrently with the PEMS test), the PAMS or
ECU RPM data was merged (in SAS) and time-aligned with the PEMS data to obtain a new
"corrected" RPM. If no concurrent PAMS or ECU data was available but the test had large
segments of missing or invalid data, ERG developed the new "corrected RPM" field by
multiplying the measured exhaust mass flow rate by constants which were calculated based on
the ratio of RPM and exhaust mass flow rates during periods of "valid" RPM collection. These
test-specific RPM to exhaust "scale factors" were developed for low-range and high-range
exhaust mass flow rates. ERG attempted to model RPM from exhaust mass flow rate using non-
linear relationships (such as polynomials), but it was seen that when flow was segregated into
high/low ranges, linear relationships defined flow versus rpm in each individual range as well as
3-47
-------
non-linear relationships. This is likely due to the large variability in the relationship between
engine volumetric throughput and RPM due to variations in throttle position and demand (i.e.,
varying loads at constant RPM). Therefore, simple constant multipliers were used, one for the
low-flow region and one for the high-flow region, where the low and high flow ranges and
constants were specifically defined for each test.
After the factors were developed, this newly calculated RPM was compared with data
from a segment of the test where "valid" RPM had been collected. Comparing plots of this new
RPM with actual, recorded RPM provided an indication of validity of the new RPM field, and
when necessary, an upper bound was applied to limit the newly-developed RPM's maximum
values. These limits were based on actual RPM values seen during periods of valid data
collection (maximum or governed engine speed). This correlation was then applied to those
segments of test data where RPM was erroneous or missing.
Occasionally, no "reasonable" operating RPM was available in the data, in which case a
new RPM field was developed based entirely on RPM to exhaust scale factors seen during field
comparisons between RPM and exhaust mass flow rates for that piece of equipment (i.e., factors
seen during test setup were used instead of collected test data). Unfortunately, when using
factors based only on RPM to exhaust ratios seen during setup and not on test data during which
valid RPM was collected, no assessment of the quality of this newly-derived RPM could be
made, and upper bounds (based on measured data) could not be applied.
Details of the all RPM corrections, including the RPM scale factors and upper bounds
used for this study, are provided in Appendix Y, PEMS Data QC Results. A summary of the
analysis used for RPM factor development is provided in Appendix AA. Appendix AA shows
time-series comparisons of the "calculated" to "measured" RPM values (during time periods
when the "measured" RPM appears reasonable) and it also shows scatter plots between
"measured" and "calculated" RPM (again during these "valid" regions). As can be seen from
reviewing these plots, some tests have a much better correlation than others, most likely due to
fluctuations in load at maximum (or governed) RPM and also due to the influence of
turbochargers (each of these influences affect exhaust mass flow rate at certain RPMs, based on
load). ERG has performed a preliminary assessment of improving this relationship by modeling
a second independent variable in the relationship, thereby improving the accuracy of the
"calculated" RPM. Completion and application of this new methodology was not possible
within the schedule and budget of the current proj ect, but is described in Section 3.11.1.3 as
possible follow-on work. However, estimates of error associated with use of this "derived" RPM
3-48
-------
are included with the emission estimates in Section 6.2.2. Details regarding uncertainty
estimates are presented in Appendix AO, Nonroad Error Estimates.
Autozeros were performed during gaseous sampling: Since engine RPM and exhaust
emission rates weren't set to zero during autozeros, autozeros performed during gaseous
sampling could produce a slight bias in cumulative gaseous emission results (since pollutants are
artificially low during autozeros). This bias could be more problematic for those time periods
when PM filter sampling was being performed (as occurred during ES Phases 1 and 2), as these
were relatively short sampling durations (so the relative bias would be greater) and also because
this would bias the cumulative gaseous results low relative to the unbiased PM results for the
filter that was being sampled during the autozero. The PEMS firmware was updated in the
enhancement stage between ES Phase 2 and 3 sampling so that autozeros would not be
performed during filter sampling in ES Phase 3.
The US EPA has reviewed all test files and identified all instances where autozeros were
performed during filter sampling in ES Phases 1 and 2, and they have confirmed autozeros were
not performed during filter sampling in ES Phase 3. All filters that were affected by gaseous
autozeros are identified in Appendix Y, PEMS Data QC Results. In order to correct for gaseous
autozeros, all observations (seconds of data collection) during which an autozero was being
performed are excluded from the cumulative gaseous emission results (gaseous emissions and
the work performed during each second of data are not included in the cumulative total).
However, if a PM filter was being collected during an autozero, the PM total flow and total work
basis are left intact for the PM results (only the gaseous data is adjusted).
In order to exclude gaseous observations where autozeros took place, in SAS, ERG
screened CO2, CO, NOX and 62 records on a second-by-second basis. For each second of data
during which CC>2 was under one percent, CO and NOX were under 50 PPM and 62 was greater
than 19 percent (these conditions indicated an autozero was underway), RPM and exhaust mass
flow rate were set to zero, thereby eliminating that observation from cumulative reporting (both
gaseous and work). This adjustment was made only for the gaseous reporting summaries which
accompany this report. The MSOD submission will contain the complete original data prior to
the application of these adjustments.
Incorrect NOx correction was applied to some PEMS data: EPA has requested that
the methodology outlined in 40 CFR 1065.670 be applied to all data collected during the study.
However, the NOx emission results in ES Phase 2 data was corrected using the methodology
outlined in 40 CFR 86.1370-2007NTE, because the PEMS data processor did not offer 1065.670
3-49
-------
methodology at the time ES Phase 2 data was processed. In addition, some ES Phase 1 and 3
data was also inadvertently processed using 40 CFR 86.1370-2007NTE methodology. In SAS,
ERG has applied 1065 corrections to all data processed using NTE methodology, so 40 CFR
1065.670 humidity and temperature-corrected NOx values are applied consistently to all data.
Some gaseous data was null during filter sampling: Occasionally, equipment issues
resulted in gaseous emission data not being collected during filter sampling. Similar to the
autozero situation previously described, if gaseous pollutants are invalid or zero for a portion of
time during which a filter was being sampled, the cumulative gaseous results will be biased low
relative to the PM results for that filter. The column entitled "Some gaseous null during filters"
in Appendix Y, PEMS Data QC Results, identifies filters affected by this, and these gaseous
results are excluded from the PEMS gaseous result reporting in Section 6.2. The column entitled
"Observations to Exclude From reporting" in Appendix Y lists all invalid data contained in the
final dataset which will be excluded from reporting, whether or not during filter sampling. This
data will be included but flagged in the final data submission for this project.
Some post-test span calibrations are missing: Pre and post-test span calibrations may
be used to quantify instrument drift during each testing episode, per 40 CFR 1065 guidelines.
Occasionally, equipment failures or other scenarios would prevent post-test spans from being
performed. In these situations, it is not possible to quantify the instrument drift during the test.
However, post-test spans are missing for other tests conducted during the study which could
have received a post-test span, typically when other testing problems, such as PM data collection
problems, occurred. For these tests, as well as for tests with equipment failures, post-test span
data is not available. All available audit and span results are compiled in Appendices Q, R and
S.
Other issues identified during second-by-second review of the PEMS data:
A summary of some of the more common issued identified during the second-by-second
review of PEMS data is provided below. These issues are listed under "General Review
Comments" in Appendix Y, PEMS Data QC Results, and criteria applied during the review are
listed in Appendix I, PEMS data QC Criteria.
Some variations and out of range temperatures were seen in various gravimetric sampling
components, including the cyclone, filter holders, and manifold. Many of the temperature
anomalies appear to have been due to switched or malfunctioning thermocouples.
Some pulsations were seen in some of the measured values (mostly sampling system
parameters such as the SEMTECH-DS' sampling pumps). Sensors reports that these
3-50
-------
pulsations result from electrical current fluctuations, as power was taken directly from the
12V rail in the PEMS rack. However, constant current control in these components is not
necessary, and regulated power was used for current needs for any component which
required constant control.
- For a select few tests, some MPS total flowrates were lower than optimal - Sensors reports
that additional flow control under our testing conditions was not feasible and these MPS
flowrate variations are inherent in this type of test system. Although the impact of this will
be quantified in the PM measurement allowance study being conducted by EPA, flow
proportionality is the critical parameter for MPS sampling. Appendix Z provides plots of
MPS flow proportionality during periods of filter sampling, and estimates of error associated
with MPS flow proportionality issues are included with the emission results in Section 6.2.2
and are derived in Appendix AO, Nonroad Error Estimates.
Engine-off flowrates for some tests (such as 0062-0748 in ES Phase 3) are higher than
expected (around 14 SCFM). Since the flowmeter has four pitot tube systems, each one
measuring flows in different ranges, this bias only affects the low/zero flows, and not higher
flows, and consequently this zero-flow bias will have a negligible effect on overall flow
measurements. In addition, the zero flow measurement is also more susceptible to drift than
the other 3 systems, so this bias is limited to the zero flow and low flow regimes.
Details regarding other issues identified for each test are provided in the "General
Review Comments" in Appendix Y, PEMS Data QC Results.
3.11.1.3 Possible Future Work
Below is a summary of additional analysis which could be performed as resources permit.
Apply drift corrections to test data: The amount of drift for each PEMS test for which
a post-test span is available has been calculated on a percentage basis. Using this drift
percentage, drift corrections could be applied to the final PEMS gaseous data on a work (or
other) basis.
Apply corrections to emissions data using results of laboratory fuel analysis:
PEMS gaseous data was processed using default diesel fuel properties (such as a 0.85 specific
gravity and a 1.8 hydrogen to carbon ratio). However, after fieldwork was completed, the EPA's
NVFEL fuel laboratory performed analysis on all fuel samples collected during the study, and
samples were also sent to an outside laboratory for additional analysis, so results from the fuels
analysis could be used in place of the default fuel properties used during initial processing in
order to improve the accuracy of the reported emissions estimates. Use of these "actual"
properties would either require reprocessing the xml data files using the SEMTECH-DS's post
processor or applying corrections to the data already in SAS. For changes to the fuel hydrogen
3-51
-------
to carbon ratio, reprocessing would probably be the most efficient method, although it is likely
the laboratory-determined hydrogen to carbon ratio is already so close to the default ratio of 1.8
used for processing that the small influence on emissions changes would not warrant
reprocessing. A comparison of data from one or two files processed with different values might
be helpful in determining the impact of the change. Specific gravity changes could easily be
made to the existing data in SAS, by ratioing the specific gravity used with the "actual" specific
gravity (divide by the default specific gravity and multiply by the actual).
Collection of MPS / Flowmeter calibration verification data - Laboratory calibration
checks of the exhaust flowmeters and microproportional samplers (EFM / MPS) used in the
study were performed by Sensors prior to and following each phase of the Nonroad PEMS study.
EPA is currently working with Sensors to acquire and compile results of the EFM / MPS
calibration verifications performed in support of this study. Review of this data can be used to
confirm accuracy of exhaust flow measurements or develop correction factors for any bias in the
exhaust flow measurements. EFM / MPS laboratory verification data received thus far is
provided in Appendix AB.
Collection of SEMTECH multipoint linearity verification data - Laboratory linearity
verification of the SEMTECH-DS PEMS used in the study was performed by Sensors prior to
and following each phase of the Nonroad PEMS study. EPA is currently working with Sensors
to acquire and compile results of the SEMTECH-DS' multipoint linearity verification results
performed in support of this study. Review of this data can be used to confirm accuracy of the
gaseous measurements or develop correction factors for any non-linearity in the data.
SEMTECH-DS multipoint linearity verification data received thus far is provided in Appendix
AC.
Test alternate corrections for SEMTECH gaseous autozeros performed during filter
sampling - Although data has been corrected for autozeros performed during filter sampling (by
way of exclusion), it is likely one or more additional methodologies could be developed which
would allow correction of cumulative emission results during which autozeros were performed
without altering the cumulative work basis used for reporting. One simple methodology might
be to calculate an average one-second emission rate for each pollutant, based on the overall test
average emission rate, then use that average for each pollutant during all seconds an autozero
was being performed. Another more refined methodology would be to classify emissions for the
test being analyzed based on engine speed (RPM) and load (as indicated by exhaust mass flow
rate). These power-based emission rates could then be used in appropriate power regimes in
order to estimate instantaneous emission rates during periods of autozero. Obviously, some
3-52
-------
uncertainty would be associated with either of these methodologies, as emissions variability and
transient emission rates have not been captured and would not be known for these time periods.
Improve accuracy of estimated RPM for tests where RPM was invalid or missing -
As described in Section 3.11.1.2, an engine's exhaust mass flow rate was occasionally used to
estimate the engine's RPM during a portion of a test collected when the measured RPM signal
was lost or erroneous. As shown in Appendix AA, some tests had a less than ideal correlation
between exhaust and RPM, which is most likely due to changes in engine load at maximum (or
governed) RPM and also due to the influence of turbochargers on the exhaust to RPM
relationship. ERG has performed a preliminary analysis which suggests using a second variable,
such as the exhaust's oxygen content, air / fuel ratio or pollutant concentration such as carbon
dioxide content, along with exhaust mass flow rate, might be helpful in calculating a more
accurate estimate of "true" RPM. This second variable could help correct for engine load, in
particular at higher RPMs. Currently, this correction for load is done by "capping" the
maximum RPM based on data trends seen during periods of valid RPM collection. Estimates of
error associated with use of this "derived" RPM are included with the emission estimates in
Section 6.2.2.
Obtain revised lug curves used for BSFC emissions estimates - As previously
described, EPA provided lug curves for all engines tested during the study, but many of these
were "generic" lug curves. Use of engine model/family specific lug curves could improve the
accuracy of the work-based emission estimates. A list of the types of curves (i.e., generic from a
website or engine specific from the manufacturer) used for estimating BSFC emissions is
provided in Appendix Y. For those tests for which only generic curves are available, engine
manufacturers could be contacted in order to obtain engine-model "specific" lug curves. The
engine model and serial number information listed in Appendix U could be used to specify
engines for which curves are needed. Once obtained, equations could be developed which
represent the curves over the full operating range, and these equations could replace the
"generic" curve equations currently used in SAS to calculate work-based emissions. In SAS,
caps should be placed on the maximum brake-specific fuel consumption for engine speeds above
the maximum RPM listed in the lug curve, in order to avoid overestimating emissions during
times when the RPM value exceeds the range provided in the lug curve. An assessment has been
made with the existing curves and data to ensure the majority of engine operation occurs within
the limits of the applied lug curve equations. Limits have been applied to BSFC values outside
of valid RPM ranges, and estimates of error associated with the use of "generic" lug curves are
included with the emission estimates in Section 6.2.2.
3-53
-------
Reprocessing data files (if necessary) - In the event any reprocessing of the
SEMTECH-DS xml files is required, all test-specific input settings, processing parameters or any
other test inputs originally used for file processing is included in the "Summary Information"
section of the processed file (this information follows the second-by-second data in each test's
*.csv file). For example, starting at the "Vehicle Description" section, fuel specific gravity, H/C
molar ratio of fuel, and AMBIIRPM multiplier are all given. The time delays (time alignment)
are all provided in the "delays" section ("NDIR Delay(s)", "NDUV Delay(s)", "THC FID
Delay(s)", "Methane FID Delay(s)"). If the flow was scaled for any reason, this would be listed
in the "Overrides" section.
3.11.2 PAMS Data Processing and QC
Several different configurations of PAMS installations were used throughout the study.
These configurations were dependent both upon equipment used for the PAMS installation as
well as how the PAMS and accessories were connected to the equipment being tested. PAMS
data processing varied from test to test depending on the type of installation, PAMS equipment
used, and equipment being tested. A summary of the PAMS data processing and QC steps is
provided below.
Process, export, consolidate, read into SAS - All PAMS datalogger files were
processed and exported to comma-separated variable (CSV) files. Duplicate records resulting
from interim PAMS data downloads during site revisits were identified and extracted, and all
remaining unique CSV files were read into Statistical Analysis Software (SAS) and compiled
into a single, chronological dataset for each piece of equipment (i.e., each installation).
Perform date and time assignments and corrections - All date and times were
reviewed and corrected, as necessary. Occasionally, PAMS installers experienced some
problems setting PAMS units to the current time zone, so some date / time corrections involved
adjusting the times to the proper time zone (CDT). Initially, the Isaac software only provided a
date and time at the start of each monitoring episode, but did not provide second-by-second dates
and times, so date and time stamps were assigned to each Isaac second-by-second record
(observation) based on the starting date/time provided for the sampling episode. Note that
updated Isaac processing software now allows dates/times to be included with each second-by-
second observation.
Although the Corsa data was collected on a 1-second basis, the Corsa datalogger
timestamps were in a sub-second format, so in SAS the timestamps were truncated to the nearest
second. As a result, after truncation, two records were sometimes obtained for any individual
3-54
-------
second. In these instances, the second observation was flagged and not used in reporting
summaries. All original and "corrected" data will be provided as part of the MSOD data
submission for this work assignment.
Perform engine speed calibration corrections - Based on the installation, revisit or
removal records, in SAS, RPM calibration corrections were applied to the data as needed.
Flag observations on installation, revisit and removal dates - Provided in Appendix
U (PEMS and PAMS Testing Details), records were kept regarding when the PAMS were
installed, revisited, and removed. Based on these installation, revisit and removal records, all
second-by-second records which were collected on days when the PAMS was installed, revisited
or removed were flagged, since any engine activity on these days could be the result of the ERG
PAMS team, rather than actual equipment use by the establishment.
Assign equipment activity flags - Depending on the type of installation, new variables
were assigned to each dataset in order to indicate whether each second-by-second record was
collected when the equipment was active (equipment active), the engine was on (engine on) or
the key was on (key on).
For some installations, the datalogger was always active (always collecting data, even if
the key and engine were off). In other installations, voltage was used as an indicator of engine
on/off status, resulting in a number of observations collected after the equipment had been shut
off (as the equipment's battery voltage slowly decayed below the shutoff limit). For these two
situations, RPM was not always a reliable indicator of engine activity, because the PAMS would
sometimes record an RPM signal when the equipment was off (erroneous RPM signal), and also
because RPM was sometimes lost during actual engine usage (these two problems were
generally only an issue in ES Phase 1, when the Capelec RPM signal device was used). Because
of these two issues (data being collected in inactive equipment and unreliable RPM signals), new
fields were defined which indicated whether or not the equipment was being used. These
assignments were made based both on RPM readings and voltage readings (which fluctuated
predictably based on equipment activity). "Equipment active" was used in ES Phase 1 to
indicate whether the equipment was in operation (engine running) at the time each observation
was recorded by the datalogger. "Engine on" was used for ES Phase 2 and 3 installations for
which switched power wasn't recorded. "Key on" was used for ES Phase 2 and 3 installations
for which switched power was recorded. Additional details regarding assignment of these three
parameters are provided in the PAMS Data Dictionary in Appendix X.
3-55
-------
Assign RPM validity flags - Erroneous RPM signals were sometimes recorded by the
PAMS, primarily during ES Phase 1 when the Capelec RPM signal devices were used. ERG
assigned "RPM valid" flags to the ES Phase 1 second-by-second data to indicate when RPM
was being collected but the equipment was not active (hence the RPM signal was invalid noise),
or if the RPM was too high (RPM greater than 6000). In ES Phases 2 and 3, unrealistic RPM
values (RPM greater than 6000) were flagged as "too high".
Assignment of "Correct" RPM - For all PAMS data in all three phases, the "ERG
RPM" field was used to indicate, for each observation, the "true" RPM. The ERG RPM is "null"
for observations where the RPM is invalid (zero when the engine was on or values above 6000
RPM). For installations where two RPM signals were collected, the "ERG RPM" is the value
felt to be correct. Again, if neither of the two values were felt to be correct, this field was left
null. Non-destructive taps into engine harnesses to obtain RPM values appeared to provide the
most reliable signal, followed by optical and then magnetic pickups, and lastly Capelec RPM
signal devices. Initial defaults for "ERG RPM" were based on this hierarchy, then comparison
of RPM with validity flags and manual review of data was used to further refine the "ERG"
RPM values assigned in SAS.
Assignment of trips and trip counts - In SAS, counts of trips (and number of
observations for each trip) were assigned for each test file. Trips were defined as episodes of
engine operation. Trip counts were the total number of "trips", separated by engine off periods.
Details regarding review of each PAMS datafile are provided in Appendix AF, PAMS
Data Review Notes. All original and processed data will be provided as part of this work
assignment.
3-56
-------
4.0 Sample Design Performance
At the onset of any study, it is important to develop estimates of how many sampling
elements must be selected to meet the specified sampling targets for the study. Key design
parameters must be predefined and expected dispositions of the sample from selection through
actual interview completion and instrumentation must be estimated. These estimates are used for
planning the survey operation and estimating the level of effort that will be required to screen
businesses for eligibility in the study and then to recruit sufficient number of businesses to
participate in the instrumentation of their nonroad equipment.
The sample design for this study, described in Section 3 of this report, employed a multi-
stage probability sample with probabilities proportional to size. The EOT Phase 1 sample of the
study involved four-stages of selection (county, establishment, site, equipment) while the
integrated sample (EOT Phases II and III) involved a three-stage design (since a census of all
establishments was taken). The expected performance of the sample used in this pilot study was
based upon several basic design parameters. These included:
Establishing the sample number called is a business and verifying a correct
address
Verifying the business conducts construction activity
Calculating an overall screening response
Confirming business eligibility to participate in the study (e.g., owns or leases one
or more pieces of diesel nonroad equipment, the business is a prime contractor,
etc.)
Completing the interview questions (agreeing to or not agreeing to participate in
the inventory/instrumentation)
Following each phase of the study, the sample design and data collection performance
was analyzed using these parameters. Based on the findings of this analysis, modifications were
made to the study design based on the outcome of the sample performance. For example, while
it was initially assumed that a single sample frame would be sufficient to achieve the study goals
(e.g., obtain the requisite number of completed equipment measurements), an additional sample
frame was obtained in EOI Phase 3 of the study to augment the initial sample obtained from SSI.
This section presents the results of the study design performance using the above
parameters and design elements for each Phase of the study. The findings of each study phase
4-1
-------
are presented by first reviewing the sample design / sample performance findings, followed by
data collection findings. Implications of these findings on the sample or data collection design of
the subsequent phase are also presented.
4.1 EOI Phase 1 Results
4.1.1 Sample Analysis
EOI Phase 1 represented our first opportunity to assess the performance of the SSI
sampling frame. Within PSU 1, construction establishments (the secondary sampling unit or
SSU) were drawn with selection probabilities proportional to the estimated number of employees
(pps). Our decision to use a pps sampling and in particular to choose the number of employees as
our measure of size was driven by a desire to increase efficiency. We plausibly expected that
employees in establishments operated nonroad equipment and that the number employees in the
establishment would be positively correlated with the number of pieces of eligible equipment.
Conversely, we expected that establishments with no/zero employees would not be operating
equipment and could therefore be eliminated from the frame prior to selection. To get a feel for
the implications of these assumptions on the second stage sampling process, we ran a frequency
distribution on the number of employees in our PSU 1 frame.
Our analysis showed that the distribution of construction establishments was highly
skewed with respect to number of employees much in the way that most business productivity
and revenue distributions are distributed in the U.S. - a relatively small subset of establishments
account for the majority of productivity or revenue. Such was the case with the employee
distribution across construction establishments in PSU 1: the distribution pattern approximately
followed a Pareto distribution (i.e., about 20% of establishments accounted for roughly 80% of
employees). If the correlation assumption between employees and equipment held up, then the
observed distribution of establishments in the sampling frame would support the use of pps
sampling.
An important concomitant to the use of number of employees as a measure of size is the
likelihood of encountering 'self-representing'/'certainty' establishments. This happens when one
or more establishments feature such large measures of size that they must be selected with
certainty into a given sample. For PSU 1, we encountered a large number of self-
representing/certainty establishments.15 Table 4.1-1 shows the frequency and percentage
15 Self-representing establishments in PSU1 were those for whom SSI reported 15 or more employees; non self-
representing establishments had 1-14 employees.
4-2
-------
distributions of establishments and number of employees in our PSU 1 frame by self-
representing status.
Table 4.1-1 Frame Characteristics of PSU 1
Self-representing
SSUs
Non self-representing
SSUs
Excluded
establishments
(0 employees)
TOTAL
NUMBER OF
ESTABLISHMENTS
267
2,052
77
2,396
PERCENT OF
ESTABLISHMENTS
11%
86%
3.2%
100%
NUMBER OF
EMPLOYEES
18,248
6,317
0
24,565
PERCENT OF
EMPLOYEES
74%
26%
0
100%
The need for self-representing units in the PSU 1 sample design precipitated a major
modification in the overall design. This was because a large number of establishments (i.e., the
self-representing units) belonged to both the Establishment and Equipment Samples by virtue of
their large measures of size. This meant that all self-representing SSUs needed to taken through
the both the EOT and ESI survey protocols (e.g., EOT, recruitment for inventory and
instrumentation, incentive experiment, etc.). In consequence, the Equipment and Establishment
sample designs were integrated to allow the same self-representing units to be included in both
samples. Field protocols were needed to incorporate this design change, as well.
A significant complication to the field protocol integration involved the sequential timing
of the data collection for the Establishment and Equipment Samples. In terms of field schedule,
the two samples had been planned to be fielded several months apart. However, we wanted to
avoid a significant gap in time between the conduct of the EOI survey and the recruitment of the
establishment and conduct of instrumentation. Conventional survey practice suggested that the
EOIs for the self-representing units be conducted during the latter Equipment Sample data
collection. To address the timing issue, we decided to draw a small equal probability sample
from the collection of self-representing establishments for fielding in the Establishment Sample.
This would leave the bulk of the largest establishments in the Equipment Sample yet permit
some learning from self-representing units through the EOI interviews in the Establishment
Sample. Accordingly, a subsample of n = 100 (out of 267 total) self-representing units was used
in the PSU 1 Establishment Sample, and the balance was allocated to the Equipment Sample.
4.1.2 Data Collection Analysis
Before presenting the results of data collection we must first describe the eligibility
criteria used for EOI Phase 1 PSU 1 (namely, for conducting the EOI). Establishments were
4-3
-------
eligible16 for the EOT if they had the following attributes (each of which is tied to an EOT survey
question):
the establishment verified that it was the establishment that had been
selected into the sample;
the establishment operated in the construction sector;
the establishment used diesel powered nonroad equipment; and
the establishment employed one or more persons.
The EOT Phase 1 Establishment sample consisted of a total of 304 establishments: 100
Self-representing establishments, and 204 non-self-representing establishments. Table 4.1-2
shows the performance of the Establishment Sample. An overall response rate of 54% was
achieved even though we planned for a 64% overall response rate. The screening rate was
principally responsible for the response rate shortfall - 58% actual versus 75% expected. Once
an establishment provided screening information and was found eligible, it appeared that
cooperation for conducting the EOT was very high (93% actual versus 85% planned).
Table 4.1-2 Expected Vs. Actual Design Parameters by Stratum for EOI Phase 1
DESIGN
PARAMETER:
eligibility rate
screen rate
interview rate
overall response
rate
Net Yield
ACTUAL
SELF REP
51%
72%
89%
64%
33%
ACTUAL
NONSELF-REP
32%
51%
97%
49%
16%
OVERALL
ACTUAL
40%
58%
93%
54%
21%
OVERALL
EXPECTED
65%
75%
85%
64%
41%
Table 4.1-2 also demonstrates the substantial differential response rates by PSU type. In
particular, the screening response rate for self-representing units (72%) is over 20 percentage
points higher than the screening rate of the non-self-representing units (51%). Moreover, the
eligibility rates17 between self- and non-self-representing units show striking differences: 51%
eligibility for self-representing units versus only 32% for non-self-representing units. In
We note that for Phase 2 PSU1, the ESI which is used to recruit establishments for instrumentation used an
additional criterion to establish eligibility prior to administering the recruitment questions. We will describe the
additional criteria in the next subsection.
It is important to remind the reader that Phase 1 EOI and subsequent ESI (recruitment) employed different
eligibility criteria, with ESI being more restrictive for PSUs 1-3. With regard to PSU 1, the subsequent ESI (in
Phase 2) added a restriction that establishments must be prime contractors. We will see later in the report that the
eligibility rates for ESI are significantly lower than what we see above in Table 5.2-1, and the reason lies with the
additional ESI eligibility requirements that were imposed.
4-4
-------
addition, both of these figures fell substantially below our expectation of 65% that we used to
plan the data collection.
The net effect is seen as the last row of Table 4.1-2- Net Yield. Net Yield refers to the
bottom line percentage of the sample that will yield a completed interview. Using our design
parameters a net yield of 41% was expected. The actual yield for EOT Phase 1 Establishment
Sample was 21%, or about half of what we planned. Net yield varied twofold by PSU status -
33% for self-representing units and 16% for non-self-representing units. Both figures fell well
below the expected/planned value of 41%.
4.1.3 Implications of Findings on Study Design
The EOT Phase 1 Establishment Sample was successful in that it achieved its primary
goal of preparing for the remainder of the Pilot Study. The results of the EOT Phase 1
Establishment Sample are summarized as follows:
The distribution of construction establishments in the sampling frame was highly
skewed with respect to number of employees (generally following a Pareto
distribution 20% of establishments acct for 80% of employees), and the sample
and data collection designs were adapted accordingly;
If the number of employees is to be used as a measure of size for sampling
establishments, then within-PSU sampling of establishments requires a two-step
approach. First, identify a set of "self-representing" establishments; second,
subsample the non-self-rep establishments
The overall response rates were generally favorable; however there was
considerable variation by self-rep and non-self rep response and this should be
incorporated into the EOT Phase 2 sample design;
Eligibility was considerably lower than planned in our design parameters, and this
was differential by PSU Status; this suggested that higher levels (than
planned/budgeted) of screening and calling would be required per completed EOT;
under a model of fixed level of resources, the target number of completed
interviews would need to be reduced;
There is no efficient way (short of calling) to identify ineligible sample; however,
the screening response rate can be increased by adopting a protocol that requires a
nominal amount of research of the disconnected numbers. This would verify that
there are no other listings for that establishment and/or that all additional listings
are disconnected or 'wrong numbers', and a conclusion could be drawn that the
establishment is no longer in business (which in turn helps the screening response
rate).
4-5
-------
To develop recommendations for Establishment Samples in EOT Phase 2 and Equipment
Sampling, we reviewed the sampling frame data for our remaining PSUs 2-5. Table 4.1-3
presents a expected data collection outcome distributions using Establishment Sample PSU 1
empirical data applied to PSUs 2-5. Column A presents the number of establishments that
appear in the sampling frame (including ineligible establishments with only one employee).
Note the relatively small number of establishments available for sampling in these PSUS,
ranging from the mid-600s to the mid-800s. Column B provides the expected number of self-
representing units for PSUs 2-5. These estimates are obtained by applying the proportion of self-
representing establishments (about 11%) that were found in PSU 1 to the remaining PSUs.
Column C exhibits the remaining non-self-representing establishments after the self-representing
are taken into account. Column D provides the total number of completed EOIs for the
combined Establishment and Equipment samples. Column E is a projection of the number of
completed EOIs from the self-representing units (for the combined Establishment and Equipment
Samples) using the effective yield that was attained from EOT Phase 1 Establishment Sample
PSU 1. More importantly, Column F applies the effective yield (16%) from non-self-
representing units in the EOT Phase 1 Establishment Sample to create a projection of the number
of completed EOIs if a census of all establishments were taken. Thus, Column G presents the
total expected number of EOIs combining the Establishment and Equipment Samples under a
census design. Note that all projections of completed EOIs are well under the targeted value of
192. The shortfalls appear in Column H.
Table 4.1-3 PSU 2-5 Projections for the Combined Establishment and Equipment
Samples
PSUl
PSU 2
PSU 3
PSU 4
PSUS
A
Number of
establishments in
Sampling Frame
2396
833
689
709
663
B
Expected
Number of
Self Rep
SSUs*
266
92
76
79
74
C
Expected
Number of
NSR SSUs
2130
741
613
630
589
D
TARGET
EOIs for
Both Phases
792
792
792
792
792
E
Expected
EOIs from
Self-Rep
@33% yield
88
31
25
26
24
F
Expected
EOIs with
census of
NSRs
341
118
98
101
94
G
Total EOIs
for both
Phases with
Census
n/a
149
123
127
119
H
Expected
SHORTFALL
in EOIs
n/a
43
69
65
73
The analysis results shown in Table 4.1-3 demonstrated that based on the EOI Phase 1
Establishment sample experience, even a census of all establishments in PSUs 2-5 would fail to
4-6
-------
produce the targeted number of EOIs in order to achieve the Establishment and Equipment
sample targets. This is irrespective of budgetary issues.
The significance of the Table 4.1-3 analysis was profound. Under the initial study
design, the Equipment Sample was scheduled for fielding after the Establishment Sample. But
instrumentations were to be based primarily on recruitment that occurs during the latter
(Equipment Sample) data collection. Table 4.1-3 indicated that there was insufficient sample
for both an 'earlier' Establishment Sample (EOT only) and a 'later' Equipment Sample (EOT,
inventory, instrumentation).
Moreover, a crucial design parameter in the Equipment Sample had not yet been verified
- the percentage of establishments that complete the EOT then agree to instrumentation. The
study design had planned for 40% cooperation rate (among those completing the EOT), but EOT
Phase 1 showed that other design parameters were off by as much as 50% (e.g., effective yield).
This was a significant area of risk and suggested that a robust approach needed to be adopted for
the Equipment Sample to ensure adequate sample for instrumentation recruiting.
To address this issue, a full integration of the EOT Phase 2 Establishment and Equipment
Samples into a single, unified design was adopted. Under this design, a single data collection
effort was conducted in which the only difference between the "Establishment" and "Equipment"
samples would be the recruitment of establishments for instrumentation at the end of the EOT.
The transition from one to the other was seamless, represented by the asking of a few additional
questions at the end of the EOT. All self-representing units were included in the Equipment
Sample. Then non-self-representing establishments would be partitioned into random replicates
and fielded sequentially, beginning with the "Equipment Sample". If the target number of
instrumentation recruitments in a given PSU was achieved (i.e., 92 x .040 = 37), then all
remaining non-self-representing replicates would be designated "Establishment Sample,"
yielding only EOIs.
4.2 EOI Phase 2 Results (Integrated EOI & ESI in PSU1)
EOI Phase 2 involved the recruitment of construction businesses from PSU 1 for the
purpose of PEMS and PAMS instrumentations. To accomplish this, a sample of businesses
1 &
(from SSI) was drawn and called by a centralized telephone facility. For most of the sample ,
the EOI was administered, followed by the administration of the incentive experiment, and
finally the administration of the recruitment protocol. Businesses that completed the EOI and
18 A small portion of the sample involved re-calling the businesses that participated in the Phase 1 Pilot. The Pilot
involved only the administration of the EOI and was conducted early in 2007.
4-7
-------
agreed to be instrumented were then sent to the field for site visits to conduct an inventory of
eligible off-road equipment, and instrumentation on a sample of that equipment.
4.2.1 Sample Analysis
The sample involved businesses that were in the construction industry (according to SIC
codes) according to the SSI sampling frame. Because the eligibility of businesses was
sufficiently low, a census of all construction establishments in PSU 2 was conducted. Thus, the
sample was composed of two parts:
The Pilot Sample; a small portion of the sample included all businesses that
participated in the EOI Phase 1 Pilot and thus had conducted an EOI; these
businesses were re-called at EOI Phase 2 because of the need for additional
sample.
Fresh sample; this represents the residual, random sample of establishments in the
PSU 1 sampling frame that were being contacted for the first time at EOI Phase 2;
they were asked to complete the EOI prior to being recruited for instrumentation;
Table 4.2-1 shows the distribution of Pilot and Fresh Sample by SSU type19 (self-
representing versus non-self-representing) used in the EOI Phase 2 data collection effort.
Column E shows that a total sample of 2,319 was used. It should be noted that only 62 of the
304 total establishments in Column A were called because there were 62 completed EOIs as a
result of the EOI Phase 1 Pilot calling effort.
Table 4.2-1 Distribution of Pilot and Fresh Samples by SSU Type for PSU 1
SSU Type:
Self-rep
Non-self rep
Total
PILOT SAMPLE
A
N
100
204
304
B
%
33%
67%
100%
FRESH SAMPLE
C
N
167
1848
2015
D
%
8%
92%
100%
TOTAL
E
N
267
2052
2319
F
%
12%
88%
100%
Prior to recruiting an establishment for site inventory and instrumentation, a series of 13
questions was administered, five of which were designed to establish eligibility to participate in
instrumentation. Thus, to determine whether or not an establishment was screened, the responses
to five questions were assessed:
Self-representing establishments in PSU1 were those for whom SSI reported 15 or more employees; non-self-
representing establishments were reported by SSI to have 1-14 employees.
4-8
-------
Ql: whether the business on the telephone line is actually the establishment
drawn into the sample;
Q2: whether or not the business is in the construction industry;
Q3: the use of diesel powered off-road equipment;
Q4: whether the business has one or more number of employees; and
90
Q13: whether the business operates as a prime contractor (note: this
requirement was removed during the ESI field period in order to increase
eligibility)
An establishment that adhered to these criteria was eligible to participate (and be recruited) into
instrumentation was established by this set of responses. And the provision of responses
(regardless of eligibility) amounted to a "successful screening." Note that any point in the
question sequence, if an establishment failed an eligibility criterion, the screening was
immediately terminated (to reduce respondent burden) and the case was declared "screened,
ineligible." Only those cases whose responses indicated eligibility to all criteria were advanced
to the incentive experiment and subsequent recruitment.
Once the screening status of an establishment was determined, the calling history for that
establishment was reviewed to obtain the proper final disposition for reporting, as shown in
Table 4.2-2. It was necessary to prioritize the dispositions in order to gain full insight into the
nature of the non-responding cases. For instance, a "not-screened" establishment that was coded
as a 'first refusal' at the first call (meaning that a later call would be made to convert the refusal
to full cooperation) may have had a busy signal on its last call before data collection ceased. We
would want the disposition of this case to be "not screened - first refusal". This determination
can only be made using a hierarchical disposition protocol for assigning final dispositions. Table
4.2-2 provides the hierarchical structure we used to assign the final dispositions.
20 Note that the last criterion (Q 13) transcends the Phase 1 eligibility criteria used for the EOI (even though it was
subsequently removed during data collection).
4-9
-------
Table 4.2-2 EOI Phase 2 Screening and Recruitment Response Disposition
Hierarchies
SCREENED:
HIERARCHY
Eligible
Completed Recruitment
Recruitment Not Completed
Complete
1
Partial Complete
Final Refusal
Over Quota
Partial Refusal
Short Completes
Specific Callback, Respondent
First Refusal
Privacy Manager (Caller ID)
Language Barrier/Deaf
Hang Up
Disconnect
General Callback
Fax/Modem
Wrong Number
Answering Machine
Busy
No Answer
Household Number
3
3
3
4
4
5
5
5
5
6
6
6
6
6
6
6
6
6
Ineligible
Q2: No Construction Services
Q3: No Diesel Equipment
Q4: No Paid Employees
Q13: Subcontractor
Out-Of-Area Completes
NOT SCREENED:
Final Refusal
First Refusal
Privacy Manager (Caller ID)
Language Barrier/Deaf
Hang Up
Disconnect
General Callback
Fax/Modem
Wrong Number
Answering Machine
Busy
No Answer
Household Number
Not dialed
2
2
2
2
2
Hierarchy
3
5
5
5
6
6
6
6
6
6
6
6
6
7
4-10
-------
4.2.2 Screening Results
Table 4.2-3 presents the screening rates for EOT Phase 2 by sample source (fresh versus
pilot). Columns A and B show that the pilot EOT Phase 1 screening effort was more successful
than that of the EOT Phase 2 calling. This was likely due to the higher mix of self representing
(i.e., larger) establishments in the pilot EOT Phase 1 sample. Column C exhibits the combined
screening rate for Phase 2 - 36%. The screening rate was about half of what was expected in the
initial study design (as shown by columns C and D of Table 4.2-3).
Table 4.2-3 Screening Rates for Phase 2 by Sample Frame
PSUl
SCREENING RATE
Sample size
PHII SAMPLE SOURCE
Fresh
A
35%
2015
Phi Pilot
B
44%
304
PHASE 2
COMBINED
C
36%
2319
EXPECTED
D
75%
n/a
The eligibility rates for instrumentation recruitment were calculated from the 844
establishments that were actually screened. Table 4.2-4 presents the eligibility rates experienced
in the EOT Phase 2 recruitment calling. The pilot sample achieved an eligibility rate that was
over twice that of the fresh EOT Phase 1 sample. This also reflects the higher proportion of large
(self-representing) establishments in the pilot sample. Column C shows the overall eligibility
rate of all establishments among those screened (recall that for EOT Phase 1 a census of all
establishments in PSU 1 was taken). The actual eligibility rate was lower than planned by a
factor of (75/8.6) = 8.5. This represents a major departure from expectation. Such a huge
reduction factor begs the question as to whether the eligibility criteria are conceptually
relevant/coherent (i.e., worthy of a major 'reality check'). The restriction to prime-contractor-
only establishments had been lifted during data collection, but that did not seem to appreciably
increase the overall rate compared to what was encountered in the Phase 1 EOIs (i.e., 40%).
One possible explanation was that the reduction was due to response error. Both the
survey introduction and some of the question wording announced or hinted at the eligibility
requirements. It was possible that the respondents were responding negatively in an effort to
quickly terminate the call and thereby reduce their own burden. To address this possibility we
reviewed and revised the survey protocols (i.e., introduction and question wordings) to remove
any hint regarding eligibility. The impact of these changes are seen in later in this report when
presenting the findings of EOT Phase 3 activity.
4-11
-------
Table 4.2-4 EOI Phase 2 Eligibility Rates by Sample Source for PSU 1
PSUl
Eligibility RATE
Total Screened
PH 2 SAMPLE FRAME
Fresh
A
7.5%
710
Ph1 Pilot
B
15.7%
134
PHASE 2
COMBINED
C
8.6%
844
EXPECTED
D
75%
n/a
4.2.3 Data Collection Analysis
The screening effort yielded a total of 74 establishments that were eligible to be recruited
for instrumentation. Of these, 21 were recruited by telephone and sent to the field for site
inventories, representing an overall recruitment rate of 28%. Table 4.2-5 presents the EOI Phase
2 recruitment rates for eligible establishments by sample type. The recruitment rates among the
fresh and pilot samples were fairly consistent. However, compared to our expectations,
recruitments were down by (1 - 28.4/34) = 17 percent. This was not viewed as substantial in
the larger picture (compared to, say, the discrepancy in expected versus actual eligibility rates
that lead us to re-think the concept of eligibility).
Table 4.2-5 EOI Phase 2 Recruitment Rates by Sample Type for PSU 1
PSUl
Recruitment RATE
Total Eligible
Total Recruited
PH2 SAMPLE
FRAME
Fresh
28.3%
53
15
Phi Pilot
28.6%
21
6
PHASE 2
COMBINED
28.4%
74
21
EXPECTED
34%
n/a
n/a
The overall response rate for the EOI Phase 2 recruitment process was calculated by
taking the product of the screening and recruitment rates (see Table 4.2-6). This yielded an
overall response rate of (35% x 28.4%) = 10.2%, which was lower than we expected by a factor
of 2.6.
Table 4.2-6 EOI Phase 2 Overall Response Rates by Sample Source for PSU 1
PSUl
Overall Response
Total Recruited
PH2 SAMPLE FRAME
Fresh
A
9.9%
15
Phi Pilot
B
12.6%
6
PHASE 2
COMBINED
C
10.2%
21
EXPECTED
D
26%
n/a
4-12
-------
4.2.4 Implications of Findings on Sample Design
Table 4.2-7 summarizes the EOT Phase 2 recruitment effort and compares the actual
design parameters that were encountered to those that were used for planning. The ratios of the
expected to actual rates show the increased effort factors for each component of the design. The
biggest contributor of additional effort was the unexpected 7.8 factor increase in screening to
identify eligible establishments. That is followed by the twofold increase in sample size needed
to compensate for a lower-than-expected screening response. The bottom row of Table 4.2-7
shows the cumulative effect: roughly 55 times more sample would be needed than expected to
achieve the original target number of recruits from telephone recruiting.
EOT Phase 2 sample and data collection analysis showed that higher than expected effort
was needed to recruit establishments due to significantly lower than expected eligibility rates,
compounded by lower screening and recruitment rates relative to what was expected. Similar
performance was expected and planned for EOT Phase 3.
Table 4.2-7 EOI Phase 2 Actual Versus Expected Design Parameters for PSU 1
PSUl
Screening Rate
Eligibility Rate
Recruitment Rate
Overall Response rate
PHASE 2 ACTUAL
36%
8.8%
28%
10%
EXPECTED
75%
75%
34%
26%
Actual increase relative to Expected
RATIO
(EXP/ACT)
2.1
8.5
1.2
2.6
55
4.3 EOI Phase 2 Results (Integrated EOI & ESI, PSUs 2 and 3)
EOI Phase 2 also included the data collection component for PSUs 2 and 3 that were
launched after the PSU 1 fieldwork. The results for PSUs 2 and 3 are reported in this subsection.
The objective for this portion of EOI Phase 2 was to test and refine our survey field protocols
which featured a fully integrated EOI-ESI instrument as well as an incentive test to establish the
utility of cash incentive for equipment participation.
4.3.1 Sample Analysis
The sample consisted of a census of all establishments appearing in the SSI frame for
PSUs 2 and 3. There were 1,522 total listings in the two PSUs, with PSU 2 accounting for 55
percent of the total. After deleting 69 listings which were not reachable (e.g., no name,
telephone number or address), the remaining 1,453 establishments were fielded for the EOI and
4-13
-------
ESI. We note that in reality there was a single survey protocol which concatenated the EOT and
ESI in a seamless fashion. The separation between the "EOT" and "ESI" simply represented the
transition from one section of a survey instrument to the next. Essentially, the EOI was
composed of a series of screening questions to establish eligibility, while the ESI was the
recruitment (to instrumentation) section that was administered only to those establishments for
which eligibility was established.
Because of the relatively small sizes of the PSU 2 and 3 samples and the fact that they
were counties in eastern Iowa that were similar socio-demographically, we combined the
91
samples for the presentation of these analyses. More important is a change to eligibility
criteria that was implemented midway into the screening operation, which is discussed in the
next section.
4.3.2 Screening Results
Screening results for this data collection require separate reporting by two important time
periods - (1) prior to September 27, 2007; and (2) September 27 and afterwards. September 27
marks the date that two criteria that restricted eligibility were removed. Prior to September 27,
eligibility criteria included a requirement that the establishment have at least one employee. On
September 27 that criterion was removed, thus allowing establishments with zero employees to
be eligible for instrumentation if they used nonroad diesel equipment. (All other eligibility
criteria remained in place.)
Table 4.3-1 Comparing Two Sets of ESI Eligibility Criteria Used for PSUs 2 and 3
EOI PHASE 2 (PSUS 2-3) ESI ELIGIBILITY CRITERIA
prior to 9/27
Ql verify Company name
Q2 type of business = construction
Q3B % diesel equipment < 1%
Q4 paid employees > 0
9/27 and after
Ql
Q2
verify Company name
type of business = construction
Q3B % diesel equipment < 1%
Table 4.3-2 presents the screening rates for PSUs 2 and 3 by criteria period.
Unfortunately, the screening rates in Columns A and B cannot be substantively compared
because the entire sample was fielded at the beginning of data collection. Because of that, the
'easy', more cooperative establishments tended to be screened prior to September 27 under the
There is little if any insight that would be gained by examining results separately PSU because of the PSUs'
similarities.
4-14
-------
more stringent criteria. The rate in column B reflects the screening rate for the harder-to-reach
establishments. A more appropriate statistic is the combined rate in column C. This shows that
the overall screening rate (regardless of the criteria used) was 60 percent. This is fairly close to
the expected rate of 71 percent. This suggests that the field protocols were roughly performing
as expected, based on our PSU 1 experience.
Table 4.3-2 Screening Rates for Combined PSUs 2 and 3 by Criteria Period
SCREENING RATE
Sample size
CRITERIA PERIOD
Prior to 9/27
(1+ employees)
A
78%
935
9/27 and after
(0+ employees)
B
28%
518
COMBINED
C
60%
1453
EXPECTED
D
75%
n/a
Eligibility rates for instrumentation recruitment were calculated using the 878
establishments that were screened. Table 4.3-3 presents the eligibility rates experienced in the
EOT Phase 2 recruitment calling. The effect of removing the 0 employee restriction on eligibility
is clearly seen. The eligibility rate after 9/27 (0+ employee criterion) is more than twice that of
the rate prior to 9/27 (1+ employee criterion). And this does not take into account the fact that
the rates in Column B are based on harder-to-reach establishments. In this table Column C is
less informative because it provides the overall average eligibility rate. The important statistic
lies in Column B because we would now expect a 26 percent eligibility rate under the criteria
used in Column B. This value could now be used for planning the implementation of PSUs 4
and 5.
Table 4.3-3 Eligibility Rates for Combined PSUs 2 and 3 by Criteria Period
Eligibility RATE
Total Screened
PRIOR TO 9/27
(1+ EMPLOYEES)
A
12%
733
9/27 AND AFTER
(0+ EMPLOYEES)
B
26%
145
COMBINED
C
14%
878
EXPECTED
D
75%
n/a
4.3.3 Data Collection Analysis
The data collection process produced the expected rate of recruitment: 37% actual versus
34% expected. The total of 123 eligible screened establishments yielded 46 agreeing to be
instrumented. There was a 7 percentage point higher rate of recruitment among the "0+
4-15
-------
employee group". This may be associated with the less constrained eligibility criterion, but
ultimately the difference is not so great to have substantive value in terms of findings or design
recommendations. The bottom line result is that the expected rate of cooperation with
instrumentation recruitment can be reasonably planned at 35 to 37 percent.
Table 4.3-4 PSU 2 and 3 Eligible Establishment Recruitment Rates by Criterion
Period
Recruitment RATE
Total Eligible
Total Recruited
PRIOR TO 9/27
(1+ EMPLOYEES)
35%
85
30
9/27 AND AFTER
(0+ EMPLOYEES)
42%
38
16
COMBINED
37%
123
46
EXPECTED
34%
n/a
n/a
Table 4.3-5 presents the overall response rates for combined PSUs 2 and 3 by criterion
period. As expected, the initial group (i.e., prior to 9/27) displays a substantially higher response
rate than its later counterpart. This is simply a reflection of fact that easier, more cooperative
establishments were more likely to be encountered at the beginning of data collection (i.e., in the
prior to 9/27 group) than in the latter group. The overall response rate for the "prior to 9/27
group" is over twice that of the post-9/27 group (i.e., 28% vs. 12%).
It is more meaningful to review the overall response rate for the combined groups. This
is seen in the third column (23%). This value is close to the expected value of 26 percent.
Table 4.3-5 Overall Response Rates for Combined PSUs 2 and 3 by Criterion
Period
Overall Response
Total Recruited
PRIOR TO 9/27
(1+ EMPLOYEES)
28%
30
9/27 AND AFTER
(0+ EMPLOYEES)
12%
16
COMBINED
23%
46
EXPECTED
26%
n/a
4.3.4 Implications of Findings on Study Design
Table 4.3-6 compares the empirical design parameters that resulted from our computer
assisted telephone interviewing (CATI) field experience with combined PSU 2 and 3 to the
expected values used for planning. Each row of the table offers a comparison and the bottom
row shows the overall impact in terms of a factor representing net increase. The last row shows
that the actual net yield was under that anticipated by a factor of 7. The rightmost column shows
the ratios of actual-to-expected rates and most are relatively near an ideal value of 1.0. However
4-16
-------
the biggest departure is due to the discrepancy in eligibility rates. The actual eligibility rate was
14% while the planned value was 75 percent. This factor alone represents a lower-than-expected
net yield by a factor of 5.4.
The good news from the data collection is that the screening and recruitment rates appear
to be predictable. This can help in planning as well as set the stage for exploring methods to
increase response rates.
Table 4.3-6 Combined PSU 2 and 3 Actual Versus Expected Design Parameters
Screening Rate
Eligibility Rate
Recruitment Rate
Overall Response rate
COMBINED PSU2& 3
ACTUAL
60%
14%
37%
23%
EXPECTED
75%
75%
34%
26%
Actual increase relative to Expected
RATIO
(EXPECTED/ACTUAL)
1.3
5.4
0.9
1.1
7
4.4 EOI Phase 3 Results (SSI/EDA Combined Sample, PSUs 4 and 5)
4.4.1 Sample Analysis
The sample design for EOI Phase 3 was unique from the other Phases in that it employed
a dual frame sample design. Instead of using only the Survey Sampling International (SSI) for
selecting establishments, we drew a supplemental sample from a file provided by a vendor
Equipment Data Associates (EDA). EDA furnished a complete listing of companies who leased
or purchased heavy construction equipment and whose transactions occurred over an 18 year
period spanning January, 1990 and February, 2008.
Because companies can purchase or lease equipment repeatedly over time, change names
and/or merge or move their offices, considerable processing was needed to prepare the EDA file
for fielding. Moreover, since we were combining samples from two sources, extensive
processing was required to identify and remove duplicate and other ineligible companies. This
process resulted in 2,209 total listings - 1,155 from PSU 4 and 1,054 from PSU 5.
Figure 4.4-1 presents visually the decomposition of PSU 4 and 5 according to three states
of nature:
The records appear in the SSI frame only.
The records appear in the EDA frame only.
4-17
-------
The records appear in both SSI and EDA lists.
This information is also given in tabular form in Table 4.4-1.
Figure 4.4-1: Composition of PSU 4 and 5 Establishments by Sampling Frame
Status
PSU4
SSI
EDA
PSU 5
SSI
EDA
Table 4.4-1 Distribution of Establishments by Frame Status and PSU
SSI only
Both SSI & EDA
EDA only
Total
Total Establishments
PSU 4
68%
11%
21%
100%
1,155
PSU 5
63%
15%
22%
100%
1,054
Total
66%
13%
21%
100%
2,209
Both Figure 4.4-1 and Table 4.4-1 relay a consistent pattern showing that a relatively
small percentage of establishments appeared in both SSI and EDA frames (roughly 1 in eight
establishments appear in both frames). Moreover, the coverage value that was added by EDA
represented a 27% increase in numbers of listings (since 217(66+13)) = 0.27 in Column C of
Table 4.4-1). Finally, the SSI contributed distinct frame-specific establishments to the overall
sample by a ratio of more than a 3:1 (since 66/21 = 3.1).
4-18
-------
As in previous study Phases, a census of all establishments was undertaken in EOT Phase
3. Accordingly, all 2,209 records in the combined SSI-EDA frame were loaded into the sample
management system for the issuance of advance letters and subsequent calling by the telephone
facility.
4.4.2 Survey Disposition Documentation
In this section the calling outcomes of the EOI-ESI data collection effort are presented.
Before discussing the disposition of the sample, a further refinement of the sample is noted: The
list of 2,209 establishments to be fielded included 161 unusable listings that could not be fielded.
Unusable listings represent seeming valid establishments but were missing key contact
information such as telephone number and/or address, and for which pre-field research efforts
failed to gather sufficient information necessary in order to make contact. Such listings included
establishments that no longer existed, moved out of the area, or were bought, merged or
renamed. There was no benefit from fielding a listing that could not be contacted. Only those
listings that contained at least a company name and telephone number were retained in the
sample.
Table 4.4-2 shows the distribution of the 2,209 establishments by usability status and
PSU. As discussed earlier, 161 listings were deemed unusable, representing about 7 percent of
the sample. This percentage of unusable listings was consistent across PSUs 4 and 5.
Table 4.4-2 Combined SSI-EDA EOI Phase 3 Sample by Usability Status and PSU
Listing Unusable
Fielded sample
Total
Fielded Sample
PSU 4
8%
92%
100%
1,063
PSUS
7%
93%
100%
985
TOTAL
7%
93%
100%
2,048
Table 4.4-2a SSI-Only EOI Phase 3 Sample by Usability Status and PSU
Listing Unusable
Fielded sample
Total
Fielded Sample
PSU 4
<1%
100%
100%
785
PSUS
<1%
100%
100%
665
TOTAL
<1%
100%
100%
1,450
4-19
-------
Table 4.4-2b EDA-Only EOI Phase 3 Sample by Usability Status and PSU
Listing Unusable
Fielded sample
Total
Fielded Sample
PSU 4
38%
62%
100%
146
PSU 5
29%
71%
100%
162
TOTAL
34%
66%
100%
308
Table 4.4-2c Both SSI & EDA EOI Phase 3 Sample by Usability Status and PSU
Listing Unusable
Fielded sample
Total
Fielded Sample
PSU 4
0%
100%
100%
132
PSU 5
0%
100%
100%
158
TOTAL
0%
100%
100%
290
The recommendations from our EOI Phase 1 experience included the full integration of
the EOI and ESI process, meaning that the transition from "EOI" survey questions to the series
of "ESI" questions was seamless and completely transparent to the respondent. Essentially, the
first thirteen questions of the integrated survey gather information about the eligibility and other
characteristics of the establishment and represent the EOI. The last nine questions involve
recruitment into the equipment inventory and instrumentation; they comprise the ESI.
Questionnaires, by phase, are provided in Appendix AK.
4.4.3 Screening
The final disposition of the sample of 2,048 fielded establishments was based on the
pattern of survey responses to the questionnaire. In order for either an EOI or ESI to be
completed, the establishment needed to screened in order to determine its eligibility status. The
screening process comprised the first three questions in the survey:
Ql: verification that the telephone number was associated with the intended
establishment;
Q2: verification that the establishment operated in the construction industry; and
Q3: verification that at least 1 percent of the equipment used by the establishment
was diesel powered.
An eligible establishment needed to be contacted (Ql), had to operate in the construction
industry (Q2) and had to employ diesel powered equipment in its operations for at least 1 percent
4-20
-------
of its fleet (Q3B). The screening process failed when insufficient information from QQ1-3 was
obtained to establish eligibility; the disposition of such establishments was "not screened''
Screened establishments were those for which a pattern of survey responses to QQ1-3 allowed a
definitive determination of eligibility status.
It should be noted that unlike earlier Phases of the pilot, EOT Phase 2 employed a singular
eligibility standard for both the EOT and ESI (as detailed above). In EOI Phase 2 PSU 1, for
instance, ESI 'eligibility' required the establishment to be a prime contractor (with no similar
restriction for EOI eligibility).
The disposition of an establishment was deemed an "EOI-complete" if (1) the
establishment was screened eligible and (2) valid survey responses were obtained
through the QQ 12-13 series (which asked about purchasing equipment and
financing such purchases, respectively).
The disposition of an establishment was deemed an "ESI-complete" if (1) the
establishment was screened eligible and (2) valid survey responses were obtained
through Q19 (soliciting the best time to contact a specific establishment
representative to arrange for the agreed site visit and instrumentation).
Table 4.4-3 represents the final disposition of the sample by PSU and for the combined
overall sample. Note that among eligible establishments, there are two possible interview
dispositions (see the two "screened eligible" rows of Table 4.4-3):
Establishments where the EOI was conducted but the ESI was not completed
(e.g., refusal to be recruited for instrumentation); and
Establishments where both the EOI and ESI were completed.
Also note the absence of the scenario where a screened eligible establishment did not
complete an EOI. This is due to the integration of the screening and "EOI" interview survey
questions. Once respondents began answering questions, those from eligible establishments
continued to answer questions up to the point of being asked to participate in inventory and
instrumentation (i.e., at Q 14 or at the incentive offering if it was triggered). This point in the
questionnaire was where break-offs first commenced (although they could occur later in the
questioning as well). Consequently, in this Phase of the study, once a respondent commences the
interview, they uniformly continued to participate until the "stakes were raised" in the ESI
recruitment process.
4-21
-------
Table 4.4-3 Final Disposition of EOI Phase 3 Fielded Sample by PSU
Final Disposition Counts
NOT screened
Screened - INELIGIBLE
Screened eligible - EOI ONLY Completed
Screened eligible - EOI & ESI Completed
Total
A
PSU 4
772
194
67
30
1,063
B
PSU 5
701
202
58
24
985
C
Total
1,473
396
125
54
2,048
The EOI Phase 3 expected and actual design parameters were next examined. The term
design parameter refers to the factors that determine sample outcomes for a given sample
survey. Four design parameters can be reported from the EOI Phase 3 work:
screening response., representing the ability of field calling to gather enough
information to determine the eligibility of an establishment;
eligibility, which reflects the rate at which screened establishments meet the
criterion for inclusion in the study (i.e., are in the construction industry and use
diesel powered equipment);
EOI interview response rate, reflecting the rate at which screened/known eligible
establishments participate in the EOI interview; and
ESI interview response rate, which is the rate at which screened/known eligible
establishments participate in the ESI recruitment question series.
The net yield was also examined - the number of sample listings needed to be processed
(based on the design parameters) in order to produce a single recruitment.
Table 4.4-4 presents actual and expected screening response rates and eligibility rates;
also, interview response rates are provided separately for EOI and ESI. All rates are reported by
PSU (Columns A and B) and for the overall sample (Column C), along with the corresponding
expected values that were used in planning (Column D).
4-22
-------
Table 4.4-4 Actual and Expected Design Parameters for EOI Phase 3 Sample by
PSU
EOI PHASE 3
Screening response
Eligibility Rate
EOI Interview Response
ESI Interview Response
Overall Response EOI
Overall Response - ESI
ESI Net Yield (1 in N)
A
PSU 4
27%
33%
100%
31%
27%
8%
35.4
B
PSU 5
29%
29%
100%
29%
29%
8%
41.0
C
Total
Actual
28%
31%
100%
30%
28%
8%
37.9
D
Expected
75%
75%
85%
40%
64%
26%
5.2
The values in columns A-C of Table 4.4-4 indicate that the screening, eligibility and
response rates were consistent across PSUs 4 and 5. Columns C and D can be used to compare
actual and expected response and eligibility rates for EOI Phase 3. Actual screening response
rates were substantially below the expected/planned value (28% actual vs. 75% expected). This
was due to a number of reasons including the quality of the business contact data, the need to
conduct additional searches to obtain business information and the need for multiple call backs to
businesses to reach a knowledgeable contact. This experience with EOI Phase 3 is consistent
with that from earlier Phases of data collection. It is difficult to secure initial cooperation with
the establishment personnel who are willing or able to provide the needed information.
Even among the successfully screened establishments, the actual eligibility rate was less
than half of the expected rate (31% actual vs. 75% expected). This was disappointing in that the
eligibility criteria had been loosened (e.g., companies with 0 employees were eligible provided
they met other criteria, removing the restriction that establishments be prime contractors). The
lower-than-expected eligibility rate led to higher screening burden to identify eligible
establishments (requiring 2.4 times the expected amount of screening than what was planned).
However, the actual EOI interview rates were consistently 100%, which simply reflected
the integrated nature of the screening questions and the EOI survey questions. The transition
was seamless and the respondent never realized the transition (as there was no need to demarcate
it).
The actual ESI interview response rate was about a quarter lower than expected (30%
actual vs. 40% expected). It was anticipated that the EOI experience would allow a rapport
between the interviewer and respondent to take hold and 'power' a successful transition from the
process of asking survey questions to being recruited into instrumentation. But this did not
4-23
-------
happen. In part this may be to the relatively short duration of the EOT (only 13 questions); such a
short duration may not allow much if any rapport to be nurtured. Another issue is certainly the
magnitude of the recruitment request. The higher than expected recalcitrance for recruitment led
to larger samples required to secure a single instrumentation.
All design parameters can be combined into the ESI net yield which estimated the total
number of sample establishments fielded and processed in order to produce a single recruitment
(i.e., an ESI complete). Net yield accounts for both the overall nonresponse rate and the
eligibility rate - two chief factors that determine how much sample is fielded to achieve a
targeted sample size. The last row of Table 4.4-4 shows that fielding roughly 5 establishments
per completed ESI was planned, but fewer than 38 actual fielded establishments were needed to
secure a completed ESI/recruitment. This represents an over sevenfold increase in fielding effort
compared to what was originally anticipated.
4.4.4 EDA-SSI Performance
A principal objective of EOI Phase 3 was to explore the utility of acquiring EDA lists of
establishments that purchase or lease diesel off-road equipment. To explore this issue, we
compiled the disposition performance measures for samples associated with each frame.
Table 4.4-5 presents the full set of sample performance measures for the SSI frame and
the EDA frame. Note that 13% of the sample appeared in both frames; for the purpose of this
analysis the overlap cases in each frame were used in the calculations for both Columns A and B
and are included. The data is presented this way because each frame should be assessed on its
own merits as a stand-alone frame.
A comparison of Table 4.4-5 Columns A and B suggest that both frames demonstrate
similar performance with regard to screening and interview response rates (as one might expect
given that a standardized field protocol was employed for both samples). Table 4.4-5 Column C
reveals one striking difference between frames: the EDA sample exhibited over twice the
eligibility rate as that of SSI (62% vs. 30%). The net effect is a 56 percent reduction needed to
produce an ESI recruit for EDA relative to that needed for SSI.
4-24
-------
Table 4.4-5 Comparison of SSI and EDA Sample Performance Rates
Response & Eligibility
Screening response
Eligibility Rate
EOI Interview Response
ESI Interview Response
Overall Response - EOI
Overall Response - ESI
ESI Net Yield (1 in N)
% of original sample* *
A
SSI
30%
30%
100%
30%
30%
9%
37.8
79%
B
EDA
28%
62%
100%
35%
28%
10%
16.6
34%
C = B/A
Ratio
0.94
2.10
1.00
1.15
0.94
1.08
0.44
0.43
* % is based on original sample of 2,209; each Frame includes overlaps
4.4.5 Analysis of the EDA as a replacement frame or a screening tool
All else being equal, the increased eligibility of the EDA frame appears very attractive
because of its high eligibility rates. The EDA data set is extremely expensive, costing $10,139
compared to the SSI frame which costs $2,262 (for all PSUs). SSI provides over 2.3 times as
many listings as EDA. And over 38% of the EDA listings (290/756) appeared in the SSI frame.
On the other hand a clear comparison of the two frames should be made on the basis of 'eligible
establishments' rather than listings, especially when the eligibility of the EDA frame is over
twice that of the SSI. To explore the distribution of expected "eligible listings" in each frame,
the expected number of eligible establishments was calculated using the differential eligibility
rates in Table 4.4-5 and applying them to the initial frame distributions in Table 4.4-1 (showing
sample listings by frame status). The results appear in Table 4.4-6.
Table 4.4-6 Expected Eligible Establishments by Frame Using Phase 3 Eligibility
Rates
SSI only
Both SSI & EDA
EDA only
Total
A
Sample
1,453
290
466
2,209
B
% Exp
Eligible
24%
62%
62%
37%
C = AXB
Exp #
Eligible
343
180
289
812
D
Combined
Frames
42%
22%
36%
100%
E
SSI
Frame
66%
34%
-
100%
F
EDA
Frame
--
38%
62%
100%
Table 4.4-6 reveals that the distribution of expected eligible establishments is more
equally distributed between SSI and EDA frames than one might think based on the raw
(unscreened) sample shown in Column A. As separate frames, SSI and EDA overlap with the
4-25
-------
other by a little over a third of their listings (ranging 34% and 38%, respectively, as shown in
Columns E and F).
The SSI frame contributes more 'eligible' sample (343+180 = 523) than the EDA frame
(180+289 = 469), as shown in Column C. Finally, in terms of assessing the EDA as a
supplemental sampling frame (to increase coverage), the EDA adds 289/523 = 55 percent more
expected eligible sample than would otherwise not be available from the SSI. This is
considerable, especially since the eligibility rate from the EDA frame is twice as high as that of
the SSI.
These findings are sufficient to conclude that EDA would be inappropriate as a
replacement frame. The findings also contraindicate the use of EDA as a pre-screening tool to
help reduce screening costs: the high cost of the EDA frame outweigh the screening benefits.
However, the EDA holds promise as a supplemental frame in a dual frame design to increase
coverage. The principal driver is cost - is the EDA purchase expense offset by its accompanying
decrease in screening costs due to a twofold increase in eligibility rates (relative to SSI)?
The relative costs of processing EDA and SSI sample require further analysis (which is
outside the scope of this report). This future analysis is encouraged.
4.5 Incentive Test Analysis
The incentive tests for each of EOI Phases 1 and 2 were not definitive and a decision was
made to continue the testing in EOI Phase 3. The design of the test involved a random
assignment to an incentive offering to establishments just before commencing the ESI portion of
the CATI survey (at Question 14). The incentive was an offer of a $100 check sent to the
establishment regardless of their participation in the ESI (recruitment). The experimental design
called for a random half of respondents to receive the incentive.
The results of the incentive experiment are presented in Table 4.5-1. The results combine
PSUs 4 and 5 for the sake of parsimony (since the separate tables show the same result). A total
of 171 establishments participated in the incentive test. To "participate" in the incentive test, the
introduction must be read to the respondent. The results show that the incentive had no observed
impact on accepting the invitation to participate in the inventory and instrumentation. With one
degree of freedom, a chi-square statistic whose value is 2.71 or greater would have been needed
to establish a 10% level of significance; the observed value was 1.98.
4-26
-------
Table 4.5-1 Combined PSU 4 and 5 Incentive Test on the Invitation to
Instrumentation
Experimental Group:
Incentive
No incentive
Total
Chi-Squared
OUTCOME
Recruited
33
21
54
1.98
Declined
58
59
117
Total
91
80
171
NOT Signif. at 10%
The possibility that the incentive offer might impact actual participation in the
inventory/instrumentation even if we failed to detect a treatment effect at the recruitment stage
was explored. That is, follow-through to instrumentation as the outcome instead of agreement
to participate was explored because some establishments agreed during the CATI interview but
then later declined when the reality of inventory/instrumentation was at hand.
Table 4.5-2 presents the results of the incentive test where the outcome is the actual
follow-through to instrumentation. Unfortunately, the incentive did not show a significant
impact on instrumentation follow-through. The results here are striking contrast to the incentive
test results for PSUs 2 and 3 (EOT Phase 2). In EOT Phase 2 there was a highly significant
incentive effect detected for follow-though to instrumentation.
Table 4.5-2 Combined PSU 4 and 5 Incentive Test on Instrumentation Follow-
Through
Experimental Group:
Incentive
No incentive
Total
Chi-Squared
OUTCOME
Instrumented
14
10
24
0.29
Declined
77
70
147
Total
91
80
171
NOT Signif. At 10%
To explore the EOT Phase 2 and 3 results we combined the EOT Phase 2 and 3 incentive
tests to see if the incentive effect remained. Table 4.5-3 shows the results of that analysis. We
see that the incentive effect remains highly significant when the outcome measure is follow-
through to instrumentation.
4-27
-------
Table 4.5-3 Combined PSUs 2-5 Incentive Tests on Instrumentation Follow-
Through
Experimental Group:
Incentive
No incentive
Total
Chi-Squared
OUTCOME
Instrumented
34
17
51
5.41
Declined
116
122
238
Signif. At 2.5% level
Total
150
139
289
These findings are insightful. First, it is clearly more important to generate follow-
through to instrumentation rather than assent at the recruitment stage. As such, we recommend
that future research focus on this as the outcome of interest/treatment effect. Secondly, the
findings from Tables 4.5-2 and 4.5-3 are decidedly mixed. There is a very strong incentive
effect from EOT Phase 2 when combining the data from EOT Phases 2 and 3. However, the
absence of a treatment effect in EOT Phase 3 is troubling and therefore the EOT Phase 2-3
incentive experience begs further analysis and investigation. Clear, significant incentive effects
would have conclusively led to a recommendation to adopt incentives in all future studies of this
type.
These mixed results give pause to a wholehearted acceptance of incentives. What could
lead to such outcomes? The following are some possible explanations:
Site effects: it could be that PSUs 2-3 contain fundamentally different
establishments than those of PSUs 4-5, although it is hard to believe that the
effect is due to the peculiarities of establishments within PSUs.
Interviewer effects: there may have been a difference in the composition of the
field interviewer staff between EOT Phases; this could occur if, say, in EOT Phase
2 a highly experienced interviewer staff was used but less experienced
interviewers were used in EOT Phase 3.
Incongruent samples: EOT Phase 3 sampling involved a census of construction
establishments regardless of employee size; even establishments showing zero
employees (according to SSI) were fielded; this was not the case for EOT Phase 2,
where zero-employee establishments were not sampled and if identified in the
EOT were terminated; differences in sample universe make-up might explain the
incentive results.
4-28
-------
The conclusion we reach from these findings is that incentives are cautiously
recommended. If at all possible incentive testing should be continued in a way that helps inform
our understanding of what factors are associated with differential instrumentation.
4-29
-------
5.0 Fieldwork Operations
5.1 Overview of field protocols
5.1.1 Initial protocols
The study objectives called for the collection of emissions data from nonroad diesel
engines operated within the study area. The protocols to achieve this were initially based on a
plan to collect data using computer assisted telephone interviewing (CATI) from two distinct
samples, establishment sample and equipment sample, for equally distinct purposes. An
overview of the protocol for the study as originally intended is illustrated in the figure below.
Data collection for the first sample, the establishment sample, was designed to estimate the
prevalence of equipment ownership in the study area through an equipment ownership interview
(EOT) survey. This sample would be administered in two phases in which EOT Phase 1 would
serve as a pilot in which the first PSU would be included. Analysis on the EOT Phase 1 would
lead to revisions to the questionnaire that would be used in EOT Phase 2 with the remaining
PSUs. The data collected in the EOT was intended to (1) evaluate the sample frames in relations
to the target study populations; (2) Obtain direct estimates of proposed measure of establishment
size; and (3) estimate proportions of eligible establishments.
For the second sample, the equipment sample, a new sample would be drawn in which
participants would participate in an EOT interview followed by an Equipment Sample Interview
(ESI) in which qualified establishments would next be invited to participate in an inventory of
their nonroad diesel construction equipment (with the goal of achieving their consent to allow
one randomly selected piece of equipment be instrumented for activity or emissions monitoring
immediately following the inventory). Prior to the telephone interview, participants in this
sample would receive an advanced letter containing a fact sheet about the study. Furthermore, as
part of the study design for this sample, incentives would be offered to establishments with the
intended effect of increasing likeliness of participation.
5-1
-------
Figure 5.1-1 Original Field Protocol
ESTABLISHMENT & EQUIPMENT SAMPLES
EQUIPMENT SAMPLES
Administer Equipment
Ownership Interview
YES
nsent ^\
ned? ,^
^^ NO
Schedule Inital Site Visit(s)
\
r
Complete Site Inventory
(Select Site)
(Select shift)
i
Complete On-site
Equipment Inventory
(Select Equipment Piece)
i
r
Instrument
Equipment Piece
i
r
Solicit consent
for instrumentation
5.1.2 Revision for Phase 2 (complete integration of EOI & ESI (EAM))
The study protocol was modified following ES Phase 1 of the establishment sample due
to low instrumentation yield rates of the sample frame which demonstrated that there was not
sufficient sample in the study region (within all five PSUs, in fact) to conduct the study as
originally intended. Rather, to achieve the study goals it was determined that a census of
establishments would be necessary. As such, the field protocols were modified through an
integration of the establishment and equipment sample and administration of their respective
surveys. The administration of the EOI remained the predecessor to the recruitment for an
equipment inventory and random selection of equipment for instrumentation.
5-2
-------
5.1.3 Equipment Sampling in The Field
As described in Section 3.5.1, weighted equipment selection for each site was performed
in Austin after the site inventory was completed. Equipment selection was based on stratified
sampling by size and usage, to provide a semi-random sample weighted toward large, heavily-
used equipment. This process was similar to picking every nth vehicle from a weighted
distribution of randomly arranged vehicles. The weightings were used to increase the likelihood
for the selection of vehicles based on size and usage. Equipment "size" categories were based on
maximum power output, with small equipment having engine with 100 hp or less and large
equipment having engines over 100 hp. Heavily-used equipment was classified as equipment
which had a lifetime average of at least 500 hours/year, as determined using hour meter readings
and the equipment's age (model year). Relative weights to various classification categories are
shown Figure 5.1-2.
Figure 5.1-2 Sampling Selection Weighting Criteria
Equipment Size
Classification
4*
M)
=
a ,2
4* C
£ '%
1 S
W)
H
o
J
=
Large
3
2
1
Small
2
1
1
Unk
1
1
1
As described in Section 3.5.1, equipment model year (hence age) and engine power were
determined after each site inventory was completed. Using the above weighting scheme,
equipment selection took place for sites inventoried for each establishment using the equipment
selection workbook developed by EPA and included in Appendix D. Equipment for which
model year, power, or cumulative hours used could not be determined were classified as
"unknown" as shown in the above figure. After equipment selections were made, this workbook
was archived for each establishment, and the equipment selection results were then imported into
the master equipment inventory sheet and transmitted back to field personnel. A complete copy
of the equipment inventory sheet with all equipment selections made during the study is provided
in Appendix W.
5-3
-------
5.2 Trade organization recruitment
In order to increase establishment participation in the study, national and local trade
organizations to which construction establishments within the study area likely belonged or dealt
with were initially recruited by ERG to seek their support in the study. Specifically, trade
organization representatives were asked if they would review our stakeholder contact list to
ensure all relevant stakeholders were included, provide input regarding how to improve
participation in the study, and they were asked to notify their membership of this study and
encourage participation through newsletters and or faxes. Additionally, the logos of trade
organizations who agreed to participate in the study were included in the footer of the program
introduction letter initially mailed out to establishments.
Eighteen trade organizations in EPA's region 7 were contacted for support. A call script
was developed briefly explaining the program and seeking support, and ERG contacted all
organizations to explain the program and request participation. A follow-up letter was also
developed to thank the trade organization contact person for their time and to provide additional
information regarding the study and type of support that was being requested. Ultimately, seven
trade organizations agreed to participate, as shown in the footer of the establishment program
introduction letter provided in Appendix AI.
5.3 Mail out and FAQ
NuStats sent an advanced letter describing the study to candidate establishments. The
advance letter served multiple purposes. First, it informed business owners about the study and
about an upcoming telephone call to complete the survey. Second, it asked them to inform
NuStats if they are not the most knowledgeable person about their non-road equipment fleet.
Finally, it informed prospective respondents they were under no obligation to participate in the
survey and, if they did, their responses would remain confidential.
To lend credibility to the study, EPA letterhead was used (rather than NuStats or another
study team member) and seven trade associations allowed the use of their logo on the letter to
demonstrate their support of the study. A copy of this letter is provided in Appendix AI.
In addition to the advance letters, a "frequently asked questions" (FAQ) brochure was
also included with a focus on the process for conducting the equipment inventory and selection
of equipment for instrumentation. The advance letter and FAQ brochure and were not originally
intended to be administered in the establishment sample portion of the study, but once the
5-4
-------
samples were integrated, beginning with EOT Phase 2, the letter and brochure were sent to all
establishments. A copy of the FAQ brochure is provided in Appendix AJ.
5.4 EOI/EAM Script Development
The Equipment Ownership Interview and Equipment Sample Interview were
administered to a person with knowledge of the establishment's operation, and were purposefully
designed to not require company records or consultation with other employees.
Draft versions of the Establishment Ownership Interview (EOI) and the Equipment
Sample Interview (ESI) questionnaire were included in the work assignment. Using these draft
questionnaires, NuStats reviewed the EOI survey for minor editing in preparation for Computer
Assisted Telephone Interviewing (CATI) programming. A number of revisions were made in
consultation with EPA. The following summarizes those modifications:
The sequence of questions was changed so that all qualifying items (those that could
potentially deem a respondent as being eligible) were at the beginning of the survey instrument.
This was done from a productivity perspective so that resources are not spent interviewing a
prospective respondent that does not qualify for the study.
In lieu of asking a business to confirm their NAICS code, respondents were asked to
verify that their organization's primary function was construction-related. The list of potential
activities was broad and representative of the types of activities as possible.
Once EPA reviewed and approved the survey, it was programmed into CATI and internal
tests were conducted to check for correct skip patterns and overall functioning of the program.
However, as a result of the findings of the EOI Phase 1 effort with PSU 1, a decision was
made to change the study design into an integrated sample. This necessitated combining the two
survey instruments (EOI and ESI) into a single survey instrument for EOI Phase 2 of the study.
During subsequent survey administrations, the instrument was modified prior to each study
phase to account for low eligibility rates necessitating the removal of certain qualifying
screening questions or streamlining the interview flow to improve response rates. The survey
instruments for each phase are contained in Appendix AK, and a summary of the eligibility
criteria, by EOI Phase, is provided in Table 5.4-1.
5-5
-------
Table 5.4-1 Eligibility Criteria by PSU
PSU
Survey
Eligibility
Criteria by
Question
Number
Ql Verify
establishment
Q2 Primary
function as
construction
Q3 Rent/own
diesel fuel
equipment
Q4 One or
more paid
employees
Q13 Prime
contractor
EOI Phase 1
PSUl
EOI
^
/
S
s
NA
EOI Phase 2
PSUl
PSUs 2 & 3
BEFORE 9/27
PSUs 2 & 3
9/27 & AFTER
EOI
Phase 3
PSUs 4 & 5
Integrated EOI/ESI
^
S
S
S
S
^
S
S
s
NA
^
s
V
NA
NA
^
S
V
NA
NA
5.4.1 Incentive Tests
Incentive tests were conducted in Phases 2 and 3 to test the extent (if any) that a cash
incentive would facilitate recruitment into instrumentation. The experiment called for half of
the screened eligible establishments to be offered an 'advance incentive' prior to being recruited
into instrumentation (which constitutes participation in ESI). The incentive test was offered
following question 13 of the survey instrument (see Appendix AK). Among those respondents
who were eligible for recruitment for the instrumentation phase of the study, half were flagged
via CATI programming for the incentive test. This was done via a "coin toss" programmed into
the CATI script in which a sample item received a 50/50 chance of being offered an incentive.
Those that were flagged as an incentive were read a script designed to offer the incentive. This
script emphasized that the offer of the incentive was not for their guaranteed participation in the
instrumentation part of the study. All respondents who were offered an incentive received the
incentive regardless of their ultimate consent to participate in the study. Those that were not
flagged as an incentive skipped the incentive script altogether. Results of the incentive test were
presented in Section 4.5.
5.4.2 Cognitive Test
Cognitive or one-on-one interviews were conducted with construction business owners in
the Kansas City, MO, region to test reactions to the materials designed to mail to business
5-6
-------
owners to invite and encourage their participation in the study. These materials included an
advance letter and Frequently Asked Questions Fact Sheet. As part of the testing process,
NuStats conducted a probe into a number of study design issues, primarily the incentive process.
Using a list of the 64 respondents who previously participated in EOT Phase 1 (PSU1) of
the project, seven Kansas City area business owners or managers were recruited to be part of the
interviews. Two trained cognitive interviewers from NuStats conducted the research in one-on-
one interviews. Four interviews took place at Delve, a professional focus group facility in Kansas
City, MO. For the other three interviews, a member of the NuStats research team visited the
business and conducted the interview on site. Interviews lasted approximately one hour and
participants received compensation for their time.
Table 5.4-2 summarizes the profiles of the seven participants based on their responses
provided in the Phase 1 Equipment Ownership Interviews.
Table 5.4-2 Profile of Participants in Cognitive Interviews
No.
1
2
o
3
4
5
6
7
Sample
Type*
NSR
NSR
SR
SR
SR
NSR
NSR
Services Performed
Building & general contracting
Special trade contractor
Excavation
Building and general contracting
Building & general contracting
Heavy construction
Excavation
Wrecking or demolition
Other: Roofing contractor
Other: Plumbing contractor
Other: Paving contractor
Building & general contracting
Heavy construction
Excavation
Wrecking or demolition
Number of
Employees
7
3
50
35
5
8
13
Number of
Equipment
o
J
4
50
7
8
10
7
*NSR=Non Self Representing and SR = Self Representing
NuStats designed the interview to capture insight and reactions to the letter and FAQ
sheet and to guide research design issues. A copy of the cognitive interview script is provided
in Appendix AM. Overall, the study materials were well received. Participants seemed to
understand the reason for the study and why their company would receive information in the
mail. The letter and fact sheet were generally clear, easy to read and communicated the
appropriate information. Knowing the letter was from EPA played a strong role in the
5-7
-------
willingness of participants to take the survey. Most participants indicated the materials would
provide enough information about the study, and in particular, the inventory and instrumentation
stages.
Discussions and in-depth probing about the letter and FAQ sheet generated positive
reaction to the materials, their questions and statements were telling in terms of what additional
detail we should consider for inclusion as well as recommendations for the study design as it
relates to enlisting participation in the equipment inventory/instrumentation portion of this pilot
study. All but two of the participants expressed willingness to participate in this later portion of
the study after reading the study materials. When asked whether or not compensation would
increase willingness to participate, they felt it would; however, it did not alter the decision of the
two who had previously said they would still not agree to participate. Regardless, monetary
compensation in the amount of one hundred dollars or more was recommended. Because the
general contractor agreeing to the equipment inventory/instrumentation at a project site may also
have subcontractors with equipment on the site, participants also recommended that
compensation be offered to the subcontractors if they are asked to participate in the equipment
instrumentation.
Finally, participants generally felt that the study's sample design assumption that
business sizenumber of employeeswas correlated to the number of the equipment in their
fleet was not always valid. For instance, if the company was a general contractor, and primarily
subcontracted much of the work requiring owned or leased equipment, it may have many
employees but few pieces of equipment.
The following summarizes the recommendations that resulted from the cognitive
interviews and guided the revisions to the study materials and design.
Letter
Keep EPA logo, have a local EPA official sign the letter; also contact information
for a local EPA employee as well as a key NuStats person.
Reword the beginning of the letter to immediately state why we need to talk to
these business owners and why they are in a unique position to help with this
study. The immediate "hook" should be as soon as possible in that first paragraph.
Incorporate more language about the bottom line reason for the study: cleaner air.
Use 'off-road' instead of'nonroad' diesel equipment.
5-S
-------
Keep trade association logos.
Consider switching the two sections on participation so the confidentiality comes
before the steps involved.
FAQ Sheet
Consider removing the word "trouble" from last question.
Consider adding a photo of the instrument, and include dimensions (i.e., 5" by 8")
and size approximations such as "about the size of a pallet" or "similar in size to a
lunchbox."
Remove the words "pilot" and "population" throughout.
Bold the instrumentation web site URL.
Appeal to protecting the air we all breathe, and consider not focusing so heavily
on the equipment and technical terminology. Explain that
leased/rented/subcontracted equipment will be treated identical to equipment
owned by the prime contractor during the study.
Equipment Inventory /Instrumentation Recruitment Design
Training of interviewers for inventory / instrumentation recruitment (immediately
following the EOT) should include the findings from these interviews, especially
those related to hesitancy among prospective participants. Training should
include the FAQ's and / or a script to address these types of concerns.
Fieldwork protocols (field manuals) for the technical specialists performing the
equipment inventory / instrumentation should account for the following:
If equipment on a site is operated by subcontractors, it may be necessary to obtain
permission from subcontractors to inventory and /or instrument equipment. Field
protocols will instruct specialists to approach a site as if they have "blanket
approval" from the prime contractor to inventory and instrument equipment
operated at the construction site.
The process for randomly selecting equipment for instrumentation should
incorporate steps to identify up to 3 equipment items in case a subcontractor will
not allow instrumentation of their equipment. If agreement for all four items is
not obtained, then consider that instrumentation attempt an "incomplete."
Equipment owners may want to be present with their equipment is being
instrumented. This may require addition time to accommodate their schedules.
5-9
-------
Include script to prepare specialists for responding to potential concerns (e.g.,
details on the instrumentation of PEMS or PAMS units such as size of units,
length of instrumentation, liability for damaged units and compensation if their
equipment is damaged.
5.5 EOI Performance
5.5.1 Outcomes
At the onset of the study, specified targets were developed for sampling and simulated
the expected dispositions of the sample frame selection through actual interview completion and
recruitment for instrumentation for each of the study phases and for each of the sample frames.
These were used to estimate how many sampling elements must be purchased and selected to
achieve specific sampling and data collection goals and were also used for planning the survey
operation and developing the level of effort for the survey interviewing effort. Appendix AL
contains the expected disposition assumptions for the study.
At the completion of each study phase survey data collection effort and following data
processing, we conducted analysis of the sample and data collection outcomes. This analysis
consisted primarily of a hierarchical analysis of the call history dispositions to determine
screening status, eligibility status (among screened establishments) and recruitment status
(among 'eligibles'). This analysis entailed determining final call history for each survey
question encompassing the screening for eligibility process.
Screening status of an establishment was determined first using the call history for that
establishment to obtain the proper final disposition for reporting. The dispositions were
prioritized in order to gain full insight into the nature of the non-responding cases. For instance,
a "not-screened" establishment that was coded as a 'first refusal' at the first call (meaning that a
later call will be made to covert the refusal to full cooperation) may have had a busy signal on its
last call before data collection ceased. We would want the disposition of this case to be "not
screened - first refusal". This determination can only be made using a hierarchical disposition
protocol for assigning final dispositions, as shown in Table 5.2-2.
The hierarchical disposition analysis ultimately provided the data necessary for a full
analysis of the sample and data collection performance in conducting the EOI and ESI
(instrumentation recruitment). Specifically, it provided documentation of the screening rate,
eligibility rates, interview rates, and overall response rates for each of the study phases.
5-10
-------
5.5.2 Discussion
The table below shows the expected dispositions of the study over each of the study
phases and for each of the sample frames.
Table 5.5-1 Expected Dispositions for each Sample Frame and Study Phase
Screening Response
Interview Response
Overall Response*
Eligibility Rate
Total Sample
Total Interviewed
(EOI)
Total Agreeing to
Instrumentation
Instrumentation
Response/ Total
ESTABLISHMENT
SAMPLE
PHASE 1
PSU1
75%
85%
64%
85%
304
100
NA
NA
PHASE 2
PSU 2-5
75%
85%
64%
85%
1214
400
NA
NA
EQUIPMENT SAMPLE
PHASE1
PSU1
75%
85%
26%
75%
318
93
37
40%
PHASE 2
PSU 2+3
75%
85%
26%
75%
635
185
74
40%
PHASE 3
PSU 4+5
75%
85%
26%
75%
635
185
74
40%
* For the Equipment Sample, the overall response rate assumes that 40% of the Establishment Sample would agree to
instrumentation. We obtain the overall Equipment Sample response rate by multiplying the overall Establishment Sample
response rate by 40%. Thus, 64% x 40% = 26%.
Table 5.5-2 shows the actual performance rates for the EOI portion of the study. From
the onset of the study, it is clear that actual dispositions were with few exceptions lower than
originally indicated. A significant reason for this was the lower-than-expected eligibility rate.
In response to this, in subsequent phases of the study (Phase 2, integrated sample, and Phase 3),
the eligibility requirements were revised to increase the eligibility rate. Tables 5.5-2 and 5.5-3
illustrate the impact of this.
5-11
-------
Table 5.5-2 Actual Performance Rates for EOI, All Phases of Integrated Sample
Screening Response
Interview Response
Overall Response
Eligibility*
Total Sample
Total Interviewed
EOI PHASE 1
PSU1
58%
94%
54%
38%
304
162
EOI PHASE 2
PSU1
50%
70%
35%
15%
2015
101
PSU 2+3
60%
87%
53%
14%
1453
107
EOI PHASE 3
PSU 4+5
28%
100%
28%
31%
2048
179
* Eligibility criteria were revised for each 'column' of data collection shown above.
The overall performance rates for ESI are presented in Table 5.5-3 below.
Table 5.5-3 Actual Performance Rates for ESI, All Phases of Integrated Sample
Screening Response
Interview Response
Overall Response
Eligibility
Total Sample
Total Agreed to Inventory
Estabs Inventoried
Estabs Instrumented
EOI PHASE 1/2
PSU1
36%
28%
10%
9%
2319
22
11
7
EOI PHASE 2
PSU 2+3
60%
37%
23%
14%
1453
43
30
9
EOI PHASE 3
PSU 4+5
28%
35%
8%
31%
2048
54
38
13
5.6 Summary of Onsite Inventories and Instrumentation
The process of conducting the onsite inventories and selecting equipment to test is
discussed in Section 3.5.1, and a description of PEMS and PAMS testing is provided in Section
3.5.2. Section 5.1.3 outlines the process used for selecting inventoried pieces of equipment for
instrumentation. This section provides counts of establishments and pieces of equipment which
5-12
-------
were inventoried and instrumented, and also provides an overview of the types of equipment on
which instrumentations took place.
5.6.1 Establishments and Equipment Inventoried
Table 5.6-1 provides counts of the numbers of establishments that were originally
recruited for inventories, establishments which were inventoried, establishments which were
recruited but then refused inventories, and establishments which were not inventoried for reasons
other than establishment refusal, such as all sites being outside the study area or the
establishment would have no active sites until after the end of the phase.
Table 5.6-1 Summary Counts of Establishments Inventoried
ES
Phase
1
2
3
Totals
Establishments
Recruited for
Inventories
22
43
54
119
Establishments
Inventoried
11
30
38
79
Establishments
Refusing
Inventories
7 (32%)
11(26%)
12 (22%)
30 (25%)
Establishments Not
Inventoried for
Other Reasons
4
2
4
10
As can be seen in Table 5.6-1, approximately 25% of the establishments who originally
agreed to participate in the inventory and measurement phase of the study later reversed their
decision and declined to participate any further in the study. Some of these were categorized as
"passive" refusals, i.e., field inventory teams were never able to reach a contact, or were given
extraordinarily unusual reasons that participation was not possible at that time.
Table 5.6-2 provides counts of equipment inventoried and instrumented throughout the
study.
Table 5.6-2 Summary Counts of Equipment Inventoried and Instrumented
Count of equipment inventoried
Count of PEMS-eligible equipment
Count of PAMS installations
Count of PEMS installations
Overall
292
179
30
40
ES
Phase 1
56
41
7
6
ES
Phase 2
110
65
11
13
ES
Phase 3
126
73
12
21
5-13
-------
Thirty-five of the 79 establishments that were inventoried were also asked to participate
in instrumentation (either PAMS, PEMS, or both). It is interesting that only 6 of those 35
establishments refused to participate in the instrumentation process after the inventory. In Table
5.6-2, PEMS eligibility was generally based on whether sufficient room was available for
securing the PEMS rack, as approximately 4 ft by 3 ft (footprint) was required to mount the rack.
In addition, the PEMS rack could not be mounted on equipment where it would hinder work or
pose a visibility or safety hazard.
A more detailed breakdown of the above counts of establishments and equipment,
including by-establishment counts of equipment and counts of installations per establishment, is
provided in Appendix AG. A complete list of all equipment inventoried is provided in Appendix
W.
5.6.2 PEMS and PAMS Testing Summary
As shown in Table 5.6-3, 40 PEMS installations were attempted throughout the duration
of the study. If more than one installation attempt was made on any individual piece of
equipment, both dates are listed in Table 5.6-3. The status of gaseous (gas), particulate matter
(PM) and engine speed (RPM) acquisition for each test are also shown in Table 5.6-3.
Additional information pertaining to each PEMS test, including details on the piece of equipment
that was instrumented and PEMS operating and setup parameters for that test, is provided in
Appendix U. Appendix Y contains detailed information regarding results of QC and analysis
performed on each PEMS test.
Table 5.6-4 provides a summary of the 30 PAMS installations throughout the duration of
the study. Activity data was successfully collected for all installations except as noted in the
"notes" column. Additional information pertaining to each PAMS installation, including details
regarding the piece of equipment that was instrumented and PAMS setup parameters and revisits
for each installation, is provided in Appendix U. Appendix AF provides additional information
pertaining to QC analysis of data from each PAMS installation.
An overall and "by-phase" summary of the counts of PEMS and PAMS tests conducted
throughout the study is also provided in Table 3.5-1.
5-14
-------
Table 5.6-3 PEMS Testing Summary
ES
Phase
1
1
1
1
1
1
2
2
2
2
2
2
2
2
2
Test ID
2208-
1918
0685-
2214
0685-
1214
0008-
1644
1688-
1462
0619-
0968
3858-
1482
3858-
5754
2523-
0713
2523-
6087
2523-
0210
3597-
095K
3597-
0726
2745-
1190
3858-
4862
Equip Type
Backhoe loader
Track/Crawler
Loader
Grader
Backhoe loader
Horizontal Boring
Machine
Track/Crawler
Loader
Crawler Dozer
Wheel Loader
Track dozer
Articulated Loader
Track Excavator
Roller Compactor
Grader
Well Driller
Telescopic Lift Truck
Mfr
John Deere
Caterpillar
Komatsu
JC Bamford Excavators
Vermeer
Caterpillar
Caterpillar
Case
Caterpillar
John Deere
Caterpillar
Hyster
Caterpillar
Cummins / James W Bell
Co
Caterpillar
Model
41 OB Turbo
963C
GD655
210S Series 2
Navigator
D16x20A
953C
D4CXL
480FLL
D6RXL
544H
325D
C340C
12H
4B-3.9
TH83
Model
Year
1983
2002
2005
1977
2006
2004
1996
1992
1997
2003
2006
1997
1996
1987
2002
Test Date
6/15/07
6/21/07
6/26/07
6/29/07
7/2/07
7/24/07
9/18, 9/20
9/21/07
9/27/07
9/28/07
10/2/07
10/4/07
10/9/07
10/11/07
10/13,
10/15
Gas
Yes
No
No
Yes
No
Yes
Yes
Yes
Yes
Yes
Yes
No
No
Yes
Yes
PM
No
Yes
No
No
No
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
RPM
No
No
No
No
No
Yes
Yes
Yes
Yes
Yes
No
Yes
Yes
Yes
Yes
5-15
-------
ES
Phase
2
2
3
3
3
3
3
3
3
3
3
3
3
3
3
3
Test ID
3597-
4734
3597-
9706
8925-
2466
9960-
6086
0229-
3781
0229-
0045
9960-
5674
8391-
3333
8418-
0997
8418-
0377
8418-
0961
0349-
1836
0349-
2422
9272-
3481
9272-
2494
9272-
0853
Equip Type
Tractor Loader
Crawler Dozer
Track Dozer
Articulated Loader
Excavator
Backhoe
Excavator
Excavator
Track Dozer
Track Dozer
Excavator
Track Dozer
Excavator
Excavator
Track Dozer
Excavator
Mfr
Case
John Deere
Caterpillar
Komatsu
Case
John Deere
Komatsu
John Deere
Caterpillar
Caterpillar
Komatsu
Caterpillar
Caterpillar
Komatsu
Caterpillar
Komatsu
Model
570 LXT
550H
953C
WAI 80
1085B
410DTurbo
PC300LC
450D
963CB
963
PC300LC-6LC
953
320B
PC400LC
963B
PC400LC
Model
Year
1997
1999
1999
Unk
1985
1995
Unk
Unk
1995
1985
1998
1988
1997
2000
1998
1993
Test Date
10/24/07
10/27/07
7/23/08
7/28, 8/5
7/31/08
8/1/08
8/6/08
8/12, 8/13
8/18, 8/19
8/22, 8/26
8/25/08
9/23/08
9/24/08
9/30/08
10/1, 10/2
10/6/08
Gas
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
PM
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
RPM
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No
Yes
Yes
Yes
Yes
Yes
5-16
-------
ES
Phase
3
3
Test ID
0062-
0748
0062-
6092
Equip Type
Backhoe Loader
Backhoe Loader
Mfr
John Deere
John Deere
Model
310J
310G
Model
Year
2007
2006
Test Date
10/9/08
10/10/08
Gas
Yes
Yes
PM
Yes
Yes
RPM
Yes
Yes
5-17
-------
Table 5.6-4 PAMS Testing Summary
ES
Phase
1
1
1
1
1
1
1
2
2
2
2
2
2
2
2
2
2
2
3
3
3
3
3
3
3
3
3
3
3
3
Test ID
1437-0399
1688-0216
1688 - 1462
1437-1396
2208-1918
1911 -1916
1911 -9540
3597-095K
3597 - 0265
3928 - 1649
3854-9162
3868-0304
3868 - 8720
3702 - 9726
3858 - 1482
3858-5754
2535-9216
2535-2754
8555-2757
0229-3781
8597 - 1096
8597-0194
8542 - 1271
0229 - 0045
0062 - 0934
0062 - 6976
9429 - 7232
9429 - 0323
9679 - 6459
0349 - 0567
Equip Type
Compact Skid Steer Loader
Horizontal Boring Machine
Horizontal Boring Machine
Skidsteer loader
Backhoe loader
Tracked Loader
Concrete Saw
Roller Compactor
Wheel Loader
Telescopic Lift Truck
Straight-Mast Lift Truck
Wheeled Crane
Telescopic Lift Truck
Telescopic Lift Truck
Crawler Dozer
Backhoe loader
Backhoe loader
Backhoe loader
Compact Skid Steer Loader
Wheeled Excavator
Compact Track Loader
Compact Track Loader
Mini Track Excavator
Backhoe loader
Tracked Dozer
Backhoe loader
Backhoe loader
Directional Boring Machine
Skid Steer Loader
Track Excavator
Mfr
IR Bobcat
Vermeer
Vermeer
IR Bobcat
John Deere
IR Bobcat
Core Cut
Hyster
Caterpillar
Lull
Case
Grove
Ingersoll Rand
Skytrak
Caterpillar
Case
Case
Case
IR Bobcat
Case
IR Bobcat
IR Bobcat
IR Bobcat
John Deere
Caterpillar
John Deere
Caterpillar
Ditch Witch
New Holland
Caterpillar
Model
873 Turbo
D20 x 22
D16x20A
S3 00 Turbo
41 OB Turbo
T300 Turbo
CC6560 XLS
C340C
962G
644B-42
586D
RT640C
VR-90B
6042
D4CXL
480FLL
5 80 Super L
5 80 Super M
SI 85 Turbo
1085B
T190
T250
329G
410DTurbo
D4C
310G
420E
JT2020
LX665 Turbo
330D
Model
Yr
1999
2007
2006
2004
1983
2003
2006
1997
1999
1998
1985
1999
1997
2005
1996
1992
1999
2002
2002
1985
2008
2008
2007
1995
2001
2006
2008
2006
1995
2006
Install
Date
6/7/07
6/8/07
6/8/07
6/9/07
6/10/07
6/11/07
6/19/07
9/10/07
9/14/07
9/11/07
9/12/07
9/13/07
9/19/07
9/15/07
9/16/07
9/16/07
9/17/07
9/17/07
7/7/08
7/8/08
7/9/08
7/10/08
7/10/08
7/11/08
8/11/08
8/11/08
8/12/08
8/12/08
8/13/08
8/14/08
Removal
Date
7/25/07
7/26/07
7/26/07
7/25/07
7/25/07
7/3/07
7/3/07
10/24/07
9/20/07
10/29/07
10/25/07
10/12/07
10/26/07
10/29/07
10/25/07
10/25/07
10/29/07
10/29/07
8/7/08
8/8/08
8/9/08
8/9/08
8/9/08
8/8/08
9/16/08
9/16/08
9/16/08
9/18/08
9/17/08
9/16/08
Notes
No data collected
No date / time stamps
No RPM, "activity"
via voltage
5-18
-------
5.6.4 Sample management system development
This study required a system for tracking establishments which had agreed to participate
in an equipment inventory through to instrumentation for PEMS and PAMS. The original scope
was an electronic (real-time) reporting tool, with overall establishment tracking, but not
equipment-inventory-level tracking. However, after PSU1, ERG provided to NuStats a list of all
data requirements for conducting fieldwork (including all inventoried equipment, equipment
selection, equipment backup, etc.). NuStats had worked on developing a system intended to
accommodate these needs. However, with the inventory and equipment selection facets of this
project, several requirements were difficult to accommodate with an online system:
Site locations were subjected to change following initial field contact
All pieces of equipment (including details) for all sites for an establishment
needed to be documented and editable on one page (including instrumentation
selections and backup selections).
In addition to canned reports, being able to parse/sort/manipulate the equipment
data in other ways (i.e., sort by engine HP, sort by Mfr, extract all the backhoes,
etc.) was necessary
Lists need to be available offline
Import of data and rapid editing and review of large amounts of data was needed.
For these reasons, the focus shifted to preparing a system as an overall tracking/reporting
tool, but not as a comprehensive field-management tool. The system was designed to have the
ability to pull in all Equipment Sample establishments (info on those establishments which agree
to participate) from CAT!
The system contained the following information:
Establishment Level
1) ES Phase of Study (Phase 1, Phase 2, Phase 3) (entered by NuStats/CATI
import)
2) Establishment ID # (entered by NuStats/CATI import)
3) Establishment Type (Self-Rep vs. non Self-Rep) (entered by NuStats/CATI
import)
4) Incentive Offered? (Yes / No) (entered by NuStats/CATI import)
5-19
-------
Date Inventoried (ERG enter thru web-based system)
1) Number of pieces of equipment for each establishment (ERG enter thru web-
based system)
2) Number of PEMS eligible pieces of equipment (ERG enter thru web-based
system)
3) Number of pieces of equipment on which a PAMS was installed (ERG enter
thru web-based system)
4) Number of pieces which received a PEMS test (ERG enter thru web-based
system)
5) Number of pieces of equipment on which a PAMS was installed and which
received a PEMS test (ERG enter thru web-based system)
6) Establishment Status (Options are "Active" and "Closed", Default to "Active",
and changed to "Closed" when completed at all the establishment's sites) (ERG
enter thru web-based system)
Equipment Level
1) Establishment ID # (where the equipment is located) (ERG enter thru web-
based system)
2) ES Phase (this would be linked by Establishment ID)
3) Equipment ID (this is the serial #) (ERG enter thru web-based system)
4) Type of equipment (open field for text entry, ERG enter thru web-based system)
5) Instrumentation type (PAMS, PEMS, or both) (ERG enter thru web-based
system)
6) If "Instrumentation Type" is "PAMS" or "both"
7) PAMS Install date (ERG enter thru web-based system)
8) PAMS Removal date (ERG enter thru web-based system)
9) If "Instrumentation Type" is PEMS or both
10) PEMS Test date (ERG enter thru web-based system)
11) PEMS Removal date (ERG enter thru web-based system)
Reporting by ES Phase (Phase 1, Phase 2, Phase 3) and total
1) # of establishments recruited (Representing vs. non self-representing/ Incentive
vs. non-incentive
2) # of establishments inventoried
3) # of establishments where PAMS have been installed (but no PEMS tests)
4) # of establishments where PEMS tests have been conducted (but no PAMS
installs)
5) # of establishments where both PEMS tests have been conducted and where
PAMS have been installed
6) Total # of PAMS installations (here, PAMS installations is defined by PAMS
install date)
7) Total # of PEMS tests
5-20
-------
6.0 Study Results and Conclusions
6.1 Recruiting and EOI Findings
6.1.1 Recruiting Database
Since this was a pilot study, our design strategy capitalized on learning, revising,
implementing and assessing from one phase to the next. Our EOI Phase 1 experience suggested
that we eliminate specific eligibility criteria in order to generate sufficient numbers of
establishments so that the instrumentation could be fielded. It also suggested that the EOI and
ESI data collection vehicles be integrated into a single, seamless survey application. The sample
design was also modified when it was realized that a census of establishments would be needed
in order to even approach the objectives of the pilot. The need for a census also rendered moot
the need to explore the merits of alternative measures of size for establishment sampling within
PSUs (since there is no need for MOS when all establishments are being taken into the sample).
Another insight came from finding a lower net yield rate relative to what we had expected, i.e., a
much larger number of sampled establishments was actually required to recruit a single
establishment.
Our EOI Phase 2 experience suggested that we no longer exclude establishments that
(according to the SSI sampling frame) reported having zero employees. This was because the
measure of size (MOS), i.e., number of employees per establishment used for sampling did not
correlate strongly with the equipment population. In addition, EOI Phase 2 suggested that we
further loosened our eligibility criteria (to include establishments that are non-prime contractors.
For EOI Phase 2 we continued the incentive experiment, where half of EOI respondents were
offered an 'advance incentive' prior to being recruited into instrumentation (which constitutes
participation in ESI). The EOI Phase 2 recruitment process for instrumentation showed that a
higher than expected effort was needed per recruit due to significantly lower than expected
eligibility rates, compounded by lower screening and recruitment rates relative to our original
expectations.
Our experience with PSUs 2 and 3 was insightful, as well. We demonstrated that the full
integration into a seamless EOI/ESI instrument was both feasible and efficient. Our 'institutional
learning' from EOI Phase 1 allowed us to realize/achieve our expected response rate targets.
And the continued striking gap between expected versus actual (much lower) eligibility rate
confirmed our suspicion that reality means low prevalence of eligible construction
establishments in the SSI frame (despite our focus on the construction sector listings). On the
other hand, the lower than expected eligibility rates also spurred our thinking that there may be
6-1
-------
response error to our screening questions and introductory scripts. In this spirit, our PSU 2-3
experience led us to refine our scripts (e.g., no longer announcing the eligibility criteria before
asking a respondent about the nature of their business) and questions (e.g., asking about the
nature of their business in open-ended fashion rather than a yes/no question about construction).
The merit of these enhancements would be seen in EOT Phase 3, where the eligibility rate more
than doubled from 15% to 31%.
We also decided to explore the utility of the EDA database which comprises a list of
purchasers of eligible off-road equipment for EOT Phase 3. We purchased EDA data - at
considerably greater expense in order to explore its utility as (1) a separate or supplemental
sampling frame, (2) as a method of pre-screening establishments to determine eligibility, and (c)
as a potential method of exploring response error (i.e., situations where an establishment reports
in the EOT to not having any eligible equipment when in fact they have purchased such
equipment according to EDA). The findings suggested that while the EDA database is promising
as a supplemental frame in a dual-frame design to increase coverage, it is inappropriate as a
replacement frame due to lower contribution towards 'eligible' sample (in comparison to the SSI
database) and high cost that outweighs the screening benefits.
6.1.2 Construction Sector Findings
The following summarizes the general findings regarding the study as applied to the
targeted construction sector:
Fraction of establishments using diesel nonroad equipment. About 49 percent of the
establishments participating in the EOT Phases 2 and 3 survey use diesel non-road
equipment. Specifically, out of 1,099 establishments, 544 reported they have (1) at
least one rented or leased item of equipment or machinery that runs on diesel fuel or
(2) at least 1% of equipment that run on diesel.
Proportion of establishments employing at least one person on a part-time or full-time
basis. Data collected in the EOT Phases 2 and 3 suggest that approximately 82% of
the establishments in the targeted sectors employed at least one person on a part-time
or full-time basis during the previous twelve months. EOT Phase 1 data is excluded
from this analysis because zero-employee establishments were excluded from the
sample drawn for EOT Phase 1.
Correlation of number of persons employed in a company and the amount of eligible
equipment. A regression analysis was employed to explore the correlation between
the number of employees in a company and the amount of eligible equipment. The
findings indicate that there was no significant correlation between these variables.
As a result we decided to field the 0-employee establishments for EOT Phases 2 and 3.
6-2
-------
Variances in key variables (to inform sample size estimation for subsequent data
collection efforts). The descriptive statistics for the key variables are provided in
Table 6.2-1. However, we recommend caution in the use of these parameters for
sample size estimation. For instance, the number of paid employees is not related to
equipment usage, so its use in the design of a study on nonroad equipment has limited
or no value. The number of equipment pieces is useful but only available after data
collection. It is not available prior to data collection. Nonetheless, it could be useful
in determining sample size for statistics where the establishment is the unit of
analysis. If the desired unit of analysis is nonroad diesel equipment, then the
clustered nature of our sample must be addressed, since equipment pieces are
clustered within establishments (as well as within work sites).
Table 6.2-1 Descriptive Statistics for Key Study Variables
No. of paid employees
No. of diesel equipment pieces
VALID NO. OF
ESTABLISHMENTS
409
454
MIN.
0
1
MAX.
2300
3200
MEAN
42.07
24.39
STD.
DEVIATION
175.40
173.44
VARIANCE
30765.32
30082.61
6.2 PEMS Measurement Results
The following subsections provide results of PEMS particulate matter and gaseous
emissions measurements collected throughout the study. Emission results are provided in units
of emissions per work performed, emissions per fuel used, and emissions per time. Results of a
PM blind study conducted prior to fieldwork in order to compare DRI and EPA laboratory
measurement results (this blind study is described in Section 3.4.5) and results of gravimetric
measurements of dynamic and field blanks collected during the field study are also presented.
Summaries listed here include gaseous results for the overall test as well as gaseous and PM
results for the first three filters collected. Additional emissions information from filters 4
through 9 (if available), as well as test and filter sampling durations, fuel used during each test
and filter, and a summary of data "flags" (potentially invalid data) overall and by filter are
provided in Appendix AH.
6.2.1 PM Filter Weights
All gravimetric sample particulate measurements were collected in accordance with
guidelines provided in Appendix L (Gravimetric Filter Handling SOPs) and Appendix F (PEMS
Installation SOPs). Weights of all gravimetric filters collected throughout the study are provided
in Appendix V (PEMS Filter Log).
6-3
-------
6.2.1.1 PM Blind Study
As described in Section 3.4.5, the ERG team coordinated a round of interlaboratory
gravimetric mass measurement comparisons between EPA and DRI prior to ES Phase 1
emissions testing. Results from this "blind study" are provided in Table 6.2-2. All EPA
measurements are buoyancy corrected. Copies of the complete EPA and DRI laboratory results
from the PM blind study are provided in Appendix P.
Table 6.2-2 PM Blind Study Results
DRI Filter ID
PEMGT001
PEMGT002
PEMGT003
PEMGT004
PEMGT005
PEMGT006
PEMGT007
PEMGT008
PEMGT009
PEMGT009 -
With Ring Cut
PEMGT010
PEMGT011
PEMGT012
DRI Pre-test Mass
(mg)
(5/9/07)
143.140
145.650
143.573
142.391
144.709
145.072
144.561
143.043
141.756
N/A
139.224
144.181
140.774
EPA Pre-test
Mass (mg)
(5/14/07)
143.2874
145.8060
143.7601
142.5794
144.8656
145.2474
144.7396
143.2262
141.9189
N/A
139.4020
144.3591
140.9449
First EPA
post-test Mass
(mg)
(5/24/07)
143.7841
146.3091
145.2798
144.1330
147.8169
148.2430
149.0786
147.6319
141.9310
N/A
139.4219
144.3708
140.9602
DRI Post-test
Mass (mg)
(6/4/07)
143.645
146.168
145.153
144.004
147.677
148.113
148.945
147.505
141.786
141.327
139.280
144.227
140.818
Final EPA
post-test
Mass (mg)
(6/7/09)
143.8029
146.3238
145.3145
144.1666
147.8481
148.2890
149.1243
147.6890
N/A
141.4719
139.4238
144.3727
140.9612
6.2.1.2 Dynamic and Field Blanks
As described in Section 3.4 1, dynamic and field blanks were collected throughout the
study to quantify the effect of handling and system contamination on the gravimetric filters
collected during the study. Table 6.2-3 provides the laboratory measurement results from all
dynamic and field blanks collected during the study. These results (and additional information
pertaining to associated tests, etc.) are included in the PEMS Filter Log, Appendix V.
6-4
-------
Table 6.2-3 Dynamic and Field Blank Measurement Results
Field
Tracking #
PEMGT037
PEMGT072
PEMGT073
5070730
PEMGT116
PEMGT121
PEMGT123
8004436
8004437
8004439
8001550
8000039
8000042
8000044
7078362
EPA
Filter ID
N/A
6075325
6075326
5070730
5070707
6075346
7049845
8004436
8004437
8004439
8001550
8000039
8000042
8000044
7078362
EstID
0619
2523
2523
3858
3597
3597
3597
N/A
N/A
N/A
8418
9272
9272
9272
9272
Equip ID
N/A
6087
6087
4862
4734
9706
9706
N/A
N/A
N/A
0097
0853
0853
0853
2494
ES
Phase
1
2
2
2
2
2
2
3
3
3
3
3
3
3
3
Date
Collected
7/24/07
9/28/07
9/28/07
10/13/07
10/24/07
10/27/07
10/27/07
7/30/08
7/30/08
7/30/08
8/18/08
10/6/08
10/6/08
10/6/08
10/1/08
Blank
Type
Field
Dynamic
Field
Dynamic
Field
Dynamic
Field
Dynamic
Dynamic
Field
Dynamic
Dynamic
Dynamic
Dynamic
Dynamic
Mass Collected
(mg)
-0.005
0.0010
0.0064
0.0273
0.0099
0.0020
0.0097
0.0075
0.0079
0.0048
0.0000
0.0233
0.0708
0.0179
0.0085
6.2.2 Summary of Gaseous and PM Emission Results
Tables 6.2-4, 6.2-5 and 6.2-6 list gaseous pollutant emissions for each PEMS test
conducted during the study. Results are cumulative for the overall test. Results in Table 6.2-4
are provided on a work basis (grams per kilowatt-hour, or g/kW-hr), calculated as outlined in
Section 3.11.1 (with a more detailed derivation in Appendix AD). Emissions in Tables 6.2-5 and
6.2-6 are provided in a fuel basis (grams/gallon) and a time basis (grams/second), respectively.
Potentially invalid or "suspect" results are excluded from Tables 6.2-4, 6.2-5 or 6.2-6.
Additional details regarding these results, including notes regarding data corrections and
potentially invalid data that has been excluded from these summaries, are provided in Appendix
AH, PEMS Measurement Results, and additional details regarding PEMS data quality checks are
provided in Appendix Y, PEMS Data QC Results.
When reviewing Tables 6.2-4, 6.2-5 and 6.2-6, it can be seen that work-based results are
not listed for some tests which do have fuel-based and time-based results listed. For these tests,
either RPM was not available and could not be developed, or no lug curve was available (both
RPM and lug curves are necessary to develop work-based emissions estimates but not fuel or
time-based emissions estimates). Appendix Y provides details regarding lug curve and RPM
availability for each test. As can be seen in Appendix Y, lug curves were provided for all
equipment except four pieces, and test results for these four pieces were either suspect, missing
gaseous data, or very short (under 15 minutes).
6-5
-------
NOx in all emissions results refers to the total exhaust nitrogen oxides, corrected for
humidity using methodology defined in 40CFR 1065.670. THC refers to the total hydrocarbon
content of the exhaust, and CO and CO2 refer to carbon monoxide and carbon dioxide exhaust
content, respectively. PM refers to the total exhaust particulate matter, determined using
dilution-corrected weight measurement results of heated 47mm gravimetric filters collected
during testing.
Several points should be kept in mind when reviewing all emission results from this
study, in particular when comparing these results with emission standards:
The calculation methodology used to estimate the work basis for each test
assumes no change in engine efficiency across varying loads at any given RPM.
That is, we assume
bsfc (N) = fc (N) / work(N)
= fc (N) / P(N) At
« fCmax (N) / Pmax (N) ' At = bsfcmax (N)
Engine efficiencies are engine load and speed dependent. For a more detailed
discussion of the assumptions used in these calculations, please see Appendix
AD.
The calculated brake specific emissions for many of these tests are calculated
using "generic" engine lug curves, in particular the brake specific fuel
consumption, bsfc, as a function of engine speed, N. Any variation between the
"generic" bsfc curve used and the true bsfc curve for each engine will result in
error in the calculated brake specific emissions for the test (or filter).
PM and gaseous results were collected during real-world operation, including
extensive idle periods for some tests. Extensive idle (or low engine speed / load)
operation may result in a higher estimate of brake specific emissions than higher
speed/load operation. This is due to the ratio of bsfc(N) emissions to small values
of fuel consumption in the denominator, i.e., the ratio of emissions to actual work
output..
The bsfc(N) for many of these tests are based in part on an estimated engine speed
that is calculated using exhaust mass flow rate, as described in Section 3.11.1.2.
The amount of error between this engine speed estimation and the "actual" engine
speed will vary from test to test. This engine speed error will be propagated to the
bsfc(N) calculation which is used to calculate the gaseous (or PM) brake specific
emissions estimations for the test (or filter).
6-6
-------
Although measured and estimated engine speeds have been range checked and
corrected for any unreasonableness, any engine speed in an individual test
outside of the range defined for any bsfc curve could result in a bias in the brake
specific emissions calculated for each test (or filter). However, BSFC values have
been limited at max BSFC for work-based emission estimates with out-of-range
(high) RPM.
As mentioned above, engine loads were determined using the ratio of the
"measured" fuel rate (calculated by the SEMTECH-DS) to the maximum fuel rate
(determined using each engine's lug curve). Some variation between this
calculated load and the true load of the engine is possible depending on the
engine's efficiency, the accuracy of the bsfc curve and the accuracy of the
SEMTECH-DS' estimate of fuel usage on a second-by-second basis.
Because of differences in soil conditions and types of work performed at each
jobsite, workloads varied widely among the different tests performed in this
study. Some equipment was used very heavily under great loads, and some
equipment use was very light. The way a piece of equipment is used (heavy load
and high RPM vs. light load) can have an influence on the emissions from one test
(or piece of equipment) to another. Time-based emission estimates are most
susceptible to these differences in workloads, since the emission estimates are not
normalized by the work produced (or energy consumed) as with work-based and
fuel-based emission estimates. Work-based estimates do account for this as
emissions are presented on a work-output basis.
Accuracy of the PM measurements are based on the ability of the micro-
proportional sampling system to collect a partial sample proportional to total
exhaust mass flow rate at any point in time. The system used for sampling was
designed for performing micro-proportional sampling during operation in the
"Not To Exceed Zone" (NTE Zone), hence the proportionality varied from test to
test. Accuracy of proportionality will have an influence on total PM emission
results reported for each test. Plots of proportionality for each test during filter
sampling are provided in Appendix Z.
Any PEMS / PM system measurement error including exhaust and sample flow
measurement error, instrument drift, time alignment errors, temperature measurement errors,
bench errors (errors in measuring specific pollutants), and errors associated with PM collection
and measurement, including filter contamination and lab procedures, proportionality and sample
flow control, sample loss and collection efficiency, etc. will affect overall emission results.
These errors, as well as the other errors discussed in this section have been estimated as shown in
Appendix AO, Nonroad Error Estimates, and are summarized in Tables 6.2-4 through 6.2-7 and
shown in Figures 6.2-1 through 6.2-28. Uncertainty associated with test-to-test emissions
variability (changes in an engine's "true" emissions from one test to the next) and changes in
fuel properties (specific gravity and hydrogen to carbon ratio) were not quantified in this study.
6-7
-------
Table 6.2-4 PEMS Gaseous Results, Overall Average Work-Based Emissions
Test
ID
2208-
1918
0685-
2214
0685-
1214
0008-
1644
1688-
1462
0619-
0968
3858-
1482
3858-
1482-1
3858-
5754
2523-
0713
2523-
6087
2523-
0210
3597-
095K
3597-
0726
2745-
1190
Equipment
Type
Backhoe
loader
Track
Loader
Grader
Backhoe
loader
Boring
Machine
Track
Loader
Crawler
Dozer
Crawler
Dozer
Wheel
Loader
Track dozer
Articulated
Loader
Track
Excavator
Roller
Compactor
Grader
Well Driller
Manufacturer
John Deere
Caterpillar
Komatsu
JC Bamford
Excavators
Vermeer
Caterpillar
Caterpillar
Caterpillar
Case
Caterpillar
John Deere
Caterpillar
Hyster
Caterpillar
Cummins / JW
Bell
Model
410B
Turbo
963C
GD655
210S
Series 2
Navigator
D16x20A
953C
D4CXL
D4CXL
480FLL
D6RXL
544H
325D
C340C
12H
4B-3.9
Model
Year
1983
2002
2005
1977
2006
2004
1996
1996
1992
1997
2003
2006
1997
1996
1987
Rated
HP
75
160
197
64
64
128
87
87
63
175
130
300
83
140
76
Test
Time
(mins)
206.33
161.38
7.17
-
-
305.13
99.82
469.65
275.42
86.42
263.42
327.07
185.85
482.97
399.72
Fuel
Used
(gals)
4.365
11.558
0.276
-
-
15.854
3.366
13.058
2.803
5.767
8.987
26.589
-
-
8.690
THC
(g/kW-hr)
-
0.586
-
-
-
0.351
-
-
-
0.976
0.677
0.575
-
-
2.297
CO
(g/kW-hr)
-
1.278
-
-
-
2.869
3.857
3.925
-
3.613
2.585
2.313
-
-
3.678
CO2
(kg/kW-hr)
-
0.679
-
-
-
1.211
0.752
0.736
-
0.730
0.721
0.787
-
-
0.709
NOX
(g/kW-hr)
-
7.97
-
-
-
7.50
13.68
13.26
-
4.57
7.49
3.44
-
-
12.46
-------
Test
ID
3858-
4862-1
3858-
4862-2
3597-
4734
3597-
9706
8925-
2466
9960-
6086
0229-
3781
0229-
0045
9960-
5674
8391-
3333 1
8391-
3333 2
8418-
0097 1
8418-
0097 2
8418-
0377 1
8418-
0961
8418-
0377 2
Equipment
Type
Telescopic
Lift
Telescopic
Lift
Tractor
Loader
Crawler
Dozer
Track Dozer
Articulated
Loader
Excavator
Backhoe
loader
Excavator
Excavator
Excavator
Track Dozer
Track Dozer
Track Dozer
Excavator
Track Dozer
Manufacturer
Caterpillar
Caterpillar
Case
John Deere
Caterpillar
Komatsu
Case
John Deere
Komatsu
John Deere
John Deere
Caterpillar
Caterpillar
Caterpillar
Komatsu
Caterpillar
Model
TH83
TH83
570 LXT
550H
953C
WAI 80
1085B
410D
Turbo
PC300LC
450D
450D
963CB
963CB
963
PC300LC-
6LC
963
Model
Year
2002
2002
1997
1999
1999
Unk
1985
1995
2003
2006
2006
1995
1995
1985
1998
1985
Rated
HP
101
101
68
84
170
124
120
75
255
349
349
160
160
150
232
150
Test
Time
(mins)
56.98
188.47
215.13
244.27
136.13
424.37
60.52
68.08
482.65
344.38
82.33
14.88
111.32
8.70
265.35
-
Fuel
Used
(gals)
1.146
3.441
2.053
8.726
9.327
8.280
0.794
1.457
33.323
36.937
0.641
0.694
10.093
1.071
20.820
-
THC
(g/kW-hr)
0.591
0.497
2.561
0.520
0.717
3.045
4.132
1.174
0.648
0.202
-
0.702
0.782
1.517
0.529
-
CO
(g/kW-hr)
2.249
2.437
6.242
1.539
-
3.527
5.757
2.441
2.197
1.024
-
3.222
2.865
4.436
1.249
-
CO2
(kg/kW-hr)
1.035
1.182
0.725
0.726
-
0.725
0.682
0.712
0.805
0.659
-
0.722
0.705
0.739
0.690
-
NOX
(g/kW-hr)
10.46
11.35
13.72
6.71
8.470
17.654
17.443
11.292
5.061
3.971
-
7.345
6.826
7.653
9.420
-
6-9
-------
Test
ID
0349-
1836
0349-
2422
9272-
3481
9272-
2494 1
9272-
2494 2
9272-
0853
0062-
0748
0062-
6092
Equipment
Type
Track Dozer
Excavator
Excavator
Track Dozer
Track Dozer
Excavator
Backhoe
Loader
Backhoe
Loader
Manufacturer
Caterpillar
Caterpillar
Komatsu
Caterpillar
Caterpillar
Komatsu
John Deere
John Deere
Model
953
320B
PC400LC
963B
963B
PC400LC
310J
310G
Model
Year
1988
1997
2000
1998
1998
1993
2007
2006
Rated
HP
121
128
321
220
220
330
84
84
Test
Time
(mins)
94.98
113.97
605.68
17.03
186.43
268.55
365.42
167.55
Fuel
Used
(gals)
5.455
9.030
77.240
1.160
14.929
30.592
8.362
2.068
THC
(g/kW-hr)
0.445
0.738
0.352
0.474
0.317
0.752
0.710
1.395
CO
(g/kW-hr)
3.428
1.328
1.274
2.277
1.876
1.631
3.474
-
CO2
(kg/kW-hr)
0.745
0.736
0.685
0.717
0.735
0.936
0.735
-
NOX
(g/kW-hr)
8.959
7.596
5.394
6.838
7.184
12.859
8.963
18.269
Note: Various sources of bias and uncertainty exist in the emission estimates provided in this table, and reported work-based
gaseous emissions could vary by as much as 20% from "true" emission values, as listed in Appendix AO. In addition, these "in-use"
estimates may differ from certification standards due to differences between the certification test cycles and this study's "in-use"
work cycles.
6-10
-------
Table 6.2-5 PEMS Gaseous Results, Overall Average Fuel-Based Emissions
Test ID
2208-1918
0685-2214
0685 - 1214
0008 - 1644
1688- 1462
0619-0968
3858-1482
3858-1482-1
3858-5754
2523-0713
2523 - 6087
2523-0210
3597-095K
3597 - 0726
2745-1190
3858-4862-1
3858-4862-2
3597.4734
3597 - 9706
8925 - 2466
9960 - 6086
0229-3781
0229 - 0045
9960 - 5674
8391-3333 1
8391-3333 2
8418-0097 1
8418-0097 2
8418-0377 1
8418-0961
Equipment Type
Backhoe loader
Track Loader
Grader
Backhoe loader
Boring Machine
Track Loader
Crawler Dozer
Crawler Dozer
Wheel Loader
Track dozer
Articulated Loader
Track Excavator
Roller Compactor
Grader
Well Driller
Telescopic Lift
Telescopic Lift
Tractor Loader
Crawler Dozer
Track Dozer
Articulated Loader
Excavator
Backhoe loader
Excavator
Excavator
Excavator
Track Dozer
Track Dozer
Track Dozer
Excavator
Manufacturer
John Deere
Caterpillar
Komatsu
JC Bamford Excavators
Vermeer
Caterpillar
Caterpillar
Caterpillar
Case
Caterpillar
John Deere
Caterpillar
Hyster
Caterpillar
Cummins / JW Bell
Caterpillar
Caterpillar
Case
John Deere
Caterpillar
Komatsu
Case
John Deere
Komatsu
John Deere
John Deere
Caterpillar
Caterpillar
Caterpillar
Komatsu
Model
41 OB Turbo
963C
GD655
210S Series 2
Navigator D16x20A
953C
D4CXL
D4CXL
480FLL
D6RXL
544H
325D
C340C
12H
4B-3.9
TH83
TH83
570 LXT
550H
953C
WAI 80
1085B
410DTurbo
PC300LC
450D
450D
963CB
963CB
963
PC300LC-6LC
Model
Year
1983
2002
2005
1977
2006
2004
1996
1996
1992
1997
2003
2006
1997
1996
1987
2002
2002
1997
1999
1999
Unk
1985
1995
2003
2006
2006
1995
1995
1985
1998
Rated
HP
75
160
197
64
64
128
87
87
63
175
130
300
83
140
76
101
101
68
84
170
124
120
75
255
349
349
160
160
150
232
HC
(g/gal)
22.141
8.699
10.151
-
-
2.967
-
-
56.102
13.610
9.595
7.474
-
-
32.264
5.852
4.302
35.491
7.339
9.798
41.719
59.833
16.626
8.167
3.123
-
9.848
11.231
20.602
7.803
CO
(g/gal)
43.839
18.960
83.365
-
-
24.235
52.203
54.138
62.055
50.394
36.655
30.045
-
-
51.661
22.259
21.086
86.495
21.708
-
48.321
83.368
34.575
27.687
15.865
-
45.187
41.139
60.225
18.434
CO2
(kg/gal)
10.142
10.078
10.098
-
-
10.231
10.181
10.154
9.623
10.181
10.221
10.219
-
-
9.952
10.245
10.229
10.041
10.235
-
9.932
9.878
10.080
10.147
10.202
-
10.129
10.118
10.039
10.187
NOX
(g/gal)
165.1
118.2
65.9
-
-
63.4
185.1
183.0
136.7
63.8
106.2
44.7
-
-
175.0
103.6
98.2
190.1
94.6
115.670
241.867
252.600
159.959
63.790
61.504
-
103.027
98.014
103.908
139.045
6-11
-------
Test ID
8418-0377 2
0349-1836
0349 - 2422
9272-3481
9272-2494 1
9272-2494 2
9272 - 0853
0062 - 0748
0062 - 6092
Equipment Type
Track Dozer
Track Dozer
Excavator
Excavator
Track Dozer
Track Dozer
Excavator
Backhoe Loader
Backhoe Loader
Manufacturer
Caterpillar
Caterpillar
Caterpillar
Komatsu
Caterpillar
Caterpillar
Komatsu
John Deere
John Deere
Model
963
953
320B
PC400LC
963B
963B
PC400LC
310J
310G
Model
Year
1985
1988
1997
2000
1998
1998
1993
2007
2006
Rated
HP
150
121
128
321
220
220
330
84
84
HC
(g/gal)
-
6.057
10.217
5.266
6.731
4.406
8.227
9.771
-
CO
(g/gal)
-
46.626
18.396
19.054
32.345
26.056
17.829
47.829
-
CO2
(kg/gal)
-
10.138
10.200
10.248
10.192
10.214
10.232
10.119
-
NOX
(g/gal)
-
121.849
105.217
80.685
97.136
99.771
140.608
123.384
-
Note: Various sources of bias and uncertainty exist in the emission estimates provided in this table, and reported fuel-based gaseous
emissions could vary by as much as 6% from "true" emission values, as listed in Appendix AO. In addition, these "in-use"
estimates may differ from laboratory-derived emission rates due to differences between laboratory test cycles and this study's "in-
use" work cycles.
Table 6.2-6 PEMS Gaseous Results, Overall Average Time-Based Emissions
Test ID
2208-1918
0685-2214
0685 - 1214
0008 - 1644
1688 - 1462
0619-0968
3858 - 1482
3858 - 1482-1
3858-5754
2523-0713
2523 - 6087
2523-0210
Equipment Type
Backhoe loader
Track Loader
Grader
Backhoe loader
Boring Machine
Track Loader
Crawler Dozer
Crawler Dozer
Wheel Loader
Track dozer
Articulated Loader
Track Excavator
Manufacturer
John Deere
Caterpillar
Komatsu
JC Bamford Excavators
Vermeer
Caterpillar
Caterpillar
Caterpillar
Case
Caterpillar
John Deere
Caterpillar
Model
41 OB Turbo
963C
GD655
210S Series 2
Navigator D16x20A
953C
D4CXL
D4CXL
480FLL
D6RXL
544H
325D
Model
Year
1983
2002
2005
1977
2006
2004
1996
1996
1992
1997
2003
2006
Rated
HP
75
160
197
64
64
128
87
87
63
175
130
300
HC
(mg/sec)
7.806
10.384
6.506
-
-
2.569
-
-
9.515
15.137
5.456
10.126
CO
(mg/sec)
15.456
22.631
53.435
-
-
20.987
29.342
25.088
10.525
56.051
20.842
40.708
CO2
(g/sec)
3.576
12.030
6.473
-
-
8.860
5.723
4.705
1.632
11.324
5.811
13.846
NOX
(mg/sec)
58.198
141.136
42.209
-
-
54.869
104.069
84.790
23.183
70.980
60.361
60.595
6-12
-------
Test ID
3597-095K
3597 - 0726
2745-1190
3858-4862-1
3858-4862-2
3597.4734
3597 - 9706
8925 - 2466
9960 - 6086
0229-3781
0229 - 0045
9960 - 5674
8391 -3333 1
8391 -3333 2
8418-0097 1
8418-0097 2
8418-0377 1
8418-0961
8418-0377 2
0349 - 1836
0349 - 2422
9272-3481
9272-2494 1
9272 - 2494 2
9272 - 0853
0062 - 0748
0062 - 6092
Equipment Type
Roller Compactor
Grader
Well Driller
Telescopic Lift
Telescopic Lift
Tractor Loader
Crawler Dozer
Track Dozer
Articulated Loader
Excavator
Backhoe loader
Excavator
Excavator
Excavator
Track Dozer
Track Dozer
Track Dozer
Excavator
Track Dozer
Track Dozer
Excavator
Excavator
Track Dozer
Track Dozer
Excavator
Backhoe Loader
Backhoe Loader
Manufacturer
Hyster
Caterpillar
Cummins / JW Bell
Caterpillar
Caterpillar
Case
John Deere
Caterpillar
Komatsu
Case
John Deere
Komatsu
John Deere
John Deere
Caterpillar
Caterpillar
Caterpillar
Komatsu
Caterpillar
Caterpillar
Caterpillar
Komatsu
Caterpillar
Caterpillar
Komatsu
John Deere
John Deere
Model
C340C
12H
4B-3.9
TH83
TH83
570 LXT
550H
953C
WAI 80
1085B
410DTurbo
PC300LC
450D
450D
963CB
963CB
963
PC300LC-6LC
963
953
320B
PC400LC
963B
963B
PC400LC
310J
310G
Model
Year
1997
1996
1987
2002
2002
1997
1999
1999
Unk
1985
1995
2003
2006
2006
1995
1995
1985
1998
1985
1988
1997
2000
1998
1998
1993
2007
2006
Rated
HP
83
140
76
101
101
68
84
170
124
120
75
255
349
349
160
160
150
232
150
121
128
321
220
220
330
84
84
HC
(mg/sec)
-
-
11.691
1.962
1.309
5.645
4.370
10.955
13.566
13.085
5.929
9.398
5.582
-
7.656
16.972
42.267
10.204
-
5.797
13.492
11.192
7.642
5.881
15.619
3.726
-
CO
(mg/sec)
-
-
18.719
7.464
6.416
13.756
12.925
-
15.714
18.232
12.331
31.860
28.359
-
35.130
62.167
123.562
24.107
-
44.626
24.293
40.498
36.724
34.775
33.851
18.241
-
CO2
(g/sec)
-
-
3.606
3.435
3.113
1.597
6.094
-
3.230
2.160
3.595
11.676
18.237
-
7.874
15.291
20.597
13.322
-
9.703
13.470
21.782
11.572
13.632
19.426
3.859
-
NOX
(mg/sec)
-
-
63.415
34.729
29.881
30.240
56.333
129.332
78.652
55.243
57.048
73.403
109.943
-
80.096
148.115
213.185
181.831
-
116.623
138.950
171.489
110.286
133.157
266.961
47.056
-
Note: Various sources of bias and uncertainty exist in the emission estimates provided in this table, and reported time-based
gaseous emissions could vary by as much as 6% from "true" emission values, as listed in Appendix AO. In addition, these "in-use"
estimates may differ from laboratory-derived emission rates due to differences between laboratory test cycles and this study's "in-
use" work cycles.
6-13
-------
Table 6.2-7 lists work-based (grams/kW-hr) PM and gaseous emissions for the first three
filters collected for each PEMS test. By-filter results are provided on a fuel basis (mass per
gallon) and a time basis (mass per second) in Appendix AH. If more than three filters were
collected for a test, these additional results are also provided in Appendix AH.
Although not always possible, Filter 1 was generally collected on a cold-start, while
filters two and three were collected once the engine had been warmed. Additional details
pertaining to each filter collected are provided in Appendix V, PEMS Filter Log.
Uncertainties associated with each filter are listed in Table 6.2-7. As described in
Appendix AO, several factors contribute to the large range in uncertainties associated with each
filter. These factors include changes in proportionality from one test (or filter) to the next, and
the relative magnitude of filter contamination and laboratory measurement uncertainty (described
in Appendix AO) to filter mass. For filters with light loading (low PM accumulation due to high
dilution, low sample times or low PM emission rates), the relative magnitude of filter
contamination and lab measurement uncertainty (both absolute numbers in mg) increases,
thereby increasing the uncertainty in the overall PM emission rate.
6-14
-------
Table 6.2-7 By-Filter PEMS Results, Average Work-Based Emissions
Test
ID
2208-
1918
0685-
2214
0685-
1214
0008-
1644
1688-
1462
0619-
0968
3858-
1482
3858-
1482-1
3858-
5754
2523-
0713
2523-
6087
2523-
0210
3597-
095K
3597-
0726
2745-
1190
Equip Desc.
'83Deere410B
,02 Cat
963 C
'05 Kmtsu
GD655
'77 JCB
210S
'06 Vmr
D16X20A
'04 Cat
953C
'96CatD4CXL
'96CatD4CXL
'92 Case
480FLL
'97CatD6RXL
'03 Deere 544H
'06 Cat
325D
'97 Hyster
C340C
'96 Cat
12H
'87 Cmns 4B-
3.9
Filter 1 Results
HC
CO
NOX
g/kW-hr
-
-
-
-
-
0.56
-
-
-
1.93
1.54
0.88
-
-
1.82
-
-
-
-
-
5.95
17.8
20.1
-
5.82
6.53
3.62
-
-
4.60
-
-
-
-
-
9.02
13.3
13.7
-
5.44
6.40
7.52
-
-
16.3
C02
kg/kwh
-
-
-
-
-
1.14
0.72
0.74
-
0.71
0.68
0.76
-
-
0.71
PM
PM
+ /-
g/kW-hr
-
-
-
-
-
0.28
0.23
0.28
-
0.64
0.19
0.57
-
-
0.31
0.10
0.11
0.09
0.09
0.15
0.22
0.21
0.16
0.08
0.11
0.18
0.14
0.10
0.08
Filter 2 Results
HC
CO
NOX
g/kW-hr
-
0.70
-
-
-
0.48
-
-
-
2.08
1.11
0.93
-
-
2.16
-
1.09
-
-
-
3.08
6.83
4.39
-
4.58
3.86
2.01
-
-
3.31
-
11.9
-
-
-
7.54
14.5
13.3
-
4.75
7.70
2.76
-
-
14.6
C02
kg/kwh
-
0.65
-
-
-
1.19
0.74
0.73
-
0.69
0.71
0.77
-
-
0.72
PM
PM
+ /-
g/kW-hr
-
0.48
-
-
-
0.55
0.22
0.37
-
0.29
0.26
0.30
-
5.54
0.14
0.43
0.40
0.17
0.14
0.07
0.05
0.14
0.15
0.09
0.07
0.08
0.06
0.09
0.07
1.7
1.3
0.04
0.03
Filter 3 Results
HC
CO
NOX
g/kW-hr
-
-
-
-
-
0.38
-
-
-
1.00
1.02
0.62
-
-
2.16
-
-
-
-
-
2.88
4.12
3.71
-
3.56
3.82
2.66
-
-
3.01
-
-
-
-
-
7.77
14.4
13.9
-
4.47
7.16
3.48
-
-
12.6
C02
kg/kwh
-
-
-
-
-
1.20
0.74
0.72
-
0.73
0.70
0.77
-
0.69
PM
PM
+ /-
g/kW-hr
-
-
-
-
-
0.44
0.16
0.18
-
0.47
0.22
0.10
-
3.06
0.12
0.13
0.10
0.05
0.04
0.06
0.05
0.15
0.12
0.07
0.05
0.03
0.03
0.94
0.72
0.04
0.03
6-15
-------
Test
ID
3858-
4862-1
3858-
4862-2
3597-
4734
3597-
9706
8925-
2466
9960-
6086
0229-
3781
0229-
0045
9960-
5674
8391 -
3333 1
8391 -
3333 2
8418-
0097 1
8418-
0097 2
8418-
0377 1
8418-
0961
8418-
0377 2
0349-
1836
Equip Desc.
'02 Cat
TH83
'02 Cat
TH83
'97 Case
570LXT
'99 Deere 550H
'99 Cat
953C
Kmtsu
WAI 80
'85 Case 1085B
'95Deere410D
,03 Kmtsu
PC300LC
'06 Deere 450D
'06 Deere 450D
'95 Cat 963CB
'95 Cat 963CB
'85 Cat
963
'98 Kmtsu
PC300LC
'85 Cat
963
'88 Cat
953
Filter 1 Results
HC
-
0.57
2.06
1.15
0.73
4.23
4.41
0.99
0.76
-
-
0.68
0.54
-
-
-
-
CO
NOX
S/kW-hr
-
3.53
8.44
1.97
5.25
7.50
11.1
2.52
1.00
-
-
3.38
3.09
-
-
-
-
-
15.1
14.1
6.89
6.93
10.3
9.90
11.0
4.79
-
-
7.41
6.35
-
-
-
-
C02
kg/kwh
-
1.20
0.72
0.72
0.69
0.73
0.65
0.71
0.80
-
-
0.72
0.70
-
-
-
-
PM
PM
+ /-
g/kW-hr
-
0.60
0.16
0.28
0.24
0.14
0.35
0.30
0.38
-
-
0.74
0.54
-
-
-
0.23
0.19
0.15
0.06
0.08
0.09
0.07
0.07
0.06
0.06
0.07
0.13
0.14
0.09
0.07
0.12
0.09
0.23
0.17
0.16
0.13
0.07
0.05
Filter 2 Results
HC
CO
NOX
g/kW-hr
-
0.50
2.54
1.30
0.56
5.27
5.67
1.14
0.66
-
-
-
0.77
-
-
-
0.53
-
2.69
7.40
3.83
4.46
6.64
8.47
2.13
1.62
-
-
-
2.57
-
-
-
3.60
-
11.3
14.3
9.35
6.53
9.65
11.2
11.0
4.54
-
-
-
6.91
-
-
-
10.2
C02
kg/kwh
-
1.09
0.72
0.72
0.73
0.72
0.65
0.71
0.80
-
-
-
0.71
-
-
-
0.69
PM
PM
+ /-
g/kW-hr
-
0.40
0.07
0.09
0.37
0.17
0.18
0.07
0.12
-
-
-
0.41
-
-
-
0.14
0.13
0.10
0.04
0.05
0.05
0.07
0.11
0.09
0.06
0.07
0.06
0.07
0.03
0.03
0.04
0.03
0.12
0.10
0.04
0.03
Filter 3 Results
HC
CO
NOX
g/kW-hr
-
0.54
2.56
0.57
5.52
3.59
1.32
0.57
-
-
-
0.83
-
-
-
0.36
-
2.69
6.42
1.13
7.08
3.93
2.73
1.49
-
-
-
2.86
-
-
-
2.82
-
11.5
14.3
6.55
9.01
20.8
11.3
4.42
-
-
-
6.88
-
-
-
8.72
C02
kg/kwh
-
1.18
0.72
0.72
0.73
0.70
0.71
0.81
-
-
-
0.71
-
-
-
0.73
PM
PM
+ /-
g/kW-hr
-
0.58
0.07
0.16
0.17
0.14
0.14
0.11
-
-
-
0.36
-
-
-
0.29
0.19
0.15
0.03
0.03
0.05
0.04
0.06
0.07
0.04
0.03
0.04
0.03
0.03
0.03
0.11
0.09
0.09
0.07
6-16
-------
Test
ID
0349-
2422
9272-
3481
9272-
2494 1
9272-
2494 2
9272-
0853
0062-
0748
0062-
6092
Equip Desc.
'97 Cat
320B
'00 Kmtsu
PC400LC
'98 Cat
963B
'98 Cat
963B
'93 Kmtsu
PC400LC
'07Deere310J
'06Deere310G
Filter 1 Results
HC
0.79
0.44
0.61
1.16
0.75
1.49
10.1
CO
NOX
S/kW-hr
1.21
1.09
2.48
3.77
1.50
7.75
-
7.63
5.59
6.76
9.63
13.3
16.6
93.9
C02
kg/kwh
0.73
0.65
0.71
0.62
0.95
0.70
-
PM
PM
+ /-
g/kW-hr
0.22
0.18
0.33
0.40
0.28
0.51
1.08
0.07
0.05
0.06
0.04
0.39
0.33
0.13
0.10
0.09
0.07
0.18
0.20
0.69
1.06
Filter 2 Results
HC
CO
NOX
g/kW-hr
0.81
0.45
-
0.78
0.73
1.59
11.2
1.50
1.28
-
2.58
1.51
6.23
-
7.47
5.47
-
7.33
13.2
13.9
71.4
C02
kg/kwh
0.73
0.68
-
0.67
0.92
0.70
-
PM
PM
+ /-
g/kW-hr
0.12
0.15
-
0.47
0.16
0.61
0.57
0.04
0.04
0.05
0.04
0.15
0.12
0.05
0.04
0.19
0.14
0.36
0.56
Filter 3 Results
HC
CO
NOX
g/kW-hr
0.69
0.56
-
0.28
0.78
0.85
7.74
1.29
1.52
-
1.79
1.51
3.14
-
7.67
5.57
-
7.13
12.9
8.78
83.3
C02
kg/kwh
0.74
0.70
-
0.73
0.94
0.74
-
PM
PM
+ /-
g/kW-hr
0.02
0.15
-
0.19
0.16
0.28
1.35
0.01
0.01
0.05
0.04
0.06
0.05
0.05
0.04
0.08
0.07
0.43
0.33
Note: Various sources of bias and uncertainty exist in the emission estimates provided in this table, and reported work-based gaseous
emissions could vary by as much as 20% from "true" emission values, as listed in Appendix AO. Uncertainties for work-based PM
emissions are listed in the PM +/- column for each filter. The "in-use" estimates reported in this table may differ from certification
standards due to differences between the certification test cycles and this study's "in-use" work cycles
6-17
-------
Figures 6.2-1 through 6.2-14 present PM and gaseous emissions on a "brake specific" or
mass / work basis (in units of grams or kg per kw-hr), by equipment category. These brake
specific emissions, Ebs(N,t), were calculated using vehicle brakes specific fuel consumption
curves, bsfcmax (N), with time resolved engine speed, N, estimated fuel consumption, fc(N,t), and
the gaseous and PM emissions measurements, E(N,t)
Ebs(N,t) = bsfcmax (N)* E(N,t) / fc(N,t)
where
bsfcmax (N) = fcmax (N) / Pmax (N) At
is the fuel consumption at maximum engine load, Pmax (N) , for a given engine speed, N, and
time interval, At. Appendix AD give the details of the calculations and an analysis detailing the
validity of using the estimation,
P(N,t) / Pmax(N) « fc(N,t) / fc max(N).
From these time resolved estimates, test sums and averages could then be computed. PM
emissions are based on the first three filters collected, and gaseous emissions are based on the
overall test average (including times when filters were and were not sampled). Uncertainty
designations are provided in these figures based on the methodology presented in Appendix AO.
Engine tier designations listed in Figures 6.2-1 through 6.2-28 are based on categories
shown in Table 6.2-8 (Dieselnet, 2009). However, the emission limits shown in Table 6.2-8 are
based on test methods that differ from the in-use work that generated data reported here, and
direct comparisons should not be made between the work-based emission standards listed in
Table 6.2-8 and the work-based emission results presented in Tables 6.2-4 and 6.2-7.
6-18
-------
Table 6.2-8 Nonroad Emission Standards Summary
EPA Tier 1-3 Nonroad Diesel Engine Emission Standards, g/kWh (g/bhp-hr)
Engine Power
kW<8
(hp 560
(hp > 750)
Tier
Tier 1
Tier 2
Tier 1
Tier 2
Tier 1
Tier 2
Tier 1
Tier 2
Tier 3
Tier 1
Tier 2
Tier 3
Tier 1
Tier 2
Tier 3
Tier 1
Tier 2
Tier 3
Tier 1
Tier 2
Tier 3
Tier 1
Tier 2
Year
2000
2005
2000
2005
1999
2004
1998
2004
2008
1997
2003
2007
1996
2003
2006
1996
2001
2006
1996
2002
2006
2000
2006
CO
8.0 (6.0)
8.0 (6.0)
6.6 (4.9)
6.6 (4.9)
5.5(4.1)
5.5(4.1)
-
5.0(3.7)
5.0(3.7)
-
5.0(3.7)
5.0(3.7)
11.4(8.5)
3.5 (2.6)
3.5 (2.6)
11.4(8.5)
3.5 (2.6)
3.5 (2.6)
11.4(8.5)
3.5 (2.6)
3.5 (2.6)
11.4(8.5)
3.5 (2.6)
HC
-
-
-
-
-
-
-
-
-
-
-
-
1.3(1.0)
-
-
1.3(1.0)
-
-
1.3(1.0)
-
-
1.3(1.0)
-
NMHC+NOx
10.5 (7.8)
7.5 (5.6)
9.5(7.1)
7.5 (5.6)
9.5(7.1)
7.5 (5.6)
-
7.5 (5.6)
4.7(3.5)
-
6.6 (4.9)
4.0(3.0)
-
6.6 (4.9)
4.0(3.0)
-
6.4 (4.8)
4.0(3.0)
-
6.4 (4.8)
4.0(3.0)
-
6.4 (4.8)
NOx
-
-
-
-
-
-
9.2 (6.9)
-
-
9.2 (6.9)
-
-
9.2 (6.9)
-
-
9.2 (6.9)
-
-
9.2 (6.9)
-
-
9.2 (6.9)
-
PM
1.0(0.75)
0.8 (0.6)
0.8 (0.6)
0.8 (0.6)
0.8 (0.6)
0.6 (0.45)
-
0.4 (0.3)
-t
-
0.3 (0.22)
-t
0.54 (0.4)
0.2(0.15)
-t
0.54 (0.4)
0.2(0.15)
-t
0.54 (0.4)
0.2(0.15)
-t
0.54 (0.4)
0.2(0.15)
t Not adopted, engines must meet Tier 2 PM standard.
6-19
-------
Figure 6.2-1 PM Emissions from Backhoe Loaders, Work Basis, Filters 1 - 3
J£
"SB
o
Backhoe Loaders
I Filter 1
I Filter 2
Filters
'95 Deere 410D
(75 hp)
'06 Deere 310G
(Tier 2)
(84 hp)
'07 Deere 310J
(Tier 2)
(84 hp)
Figure 6.2-2 Gaseous Emissions from Backhoe Loaders, Work Basis, Overall test
Backhoe Loaders
'95 Deere 410D
(75 hp)
06 Deere310G*
(Tier 2)
(84 hp)
'07 Deere 310J
(Tier 2)
(84 hp)
lHC(g/kW-hr)
lCO(g/kW-hr)
lCO2(kg/kW-hr)
lNox(g/kW-hr)
*NoCO or CO2 results due to NDIR signal loss on this test
6-20
-------
Figure 6.2-3 PM Emissions from Dozers, 50-99 hp, Work Basis, Filters 1-3
Dozers, 50 hp-99 hp
I Filter 1
I Filter 2
Filters
'96CatD4CXL
(87 hp)
'99 Deere 550H
(Tierl)
(84 hp)
Figure 6.2.4 PM Emissions from Dozers, > 100 hp, Work Basis, Filters 1 - 3
"Sa
I
Dozers, > 100 hp
Filter 1
'88 Cat 953 '95Cat963CB '97CatD6RXL 99Cat953C* '98Cat963B
(121 hp) (160 hp) (Tierl) (Tierl) (Tierl)
(175 hp) 170 hp) (220 hp)
* PM results invalid for Filter #3 on this test
6-21
-------
Figure 6.2-5 Gaseous Emissions from Dozers, 50-99 hp, Work Basis, Overall test
c
o
8
£
3
o
8!
nj
Dozers, 50 hp - 99 hp
'96 Cat D4CXL*
(87 hp)
'99 Deere 550H
(Tierl)
(84 hp)
*NoHC results due to FID signal loss on this test
lHC(g/kW-hr)
I CO (g/kW-hr)
!C02(kg/kW-hr)
lNox(g/kW-hr)
Figure 6.2-6 Gaseous Emissions from Dozers, > 100 hp, Work Basis, Overall test
3
O
OJ
V)
TO
(3
12
10
8
6
Dozers, > 100 hp
HC(g/kW-hr)
COfe/kW-hrl
'85Cat963 '88Cat953 '95 Cat 963CB'97 Cat D6RXL 99 Cat 953C* '98 Cat 963B
(150 hp) (121 hp) (160 hp) (Tierl) (Tierl) (Tierl)
(175 hp) 170 hp) (220 hp)
*NoCO or C02 results due to NDIR signal loss on this test
6-22
-------
Figure 6.2-7 PM Emissions from Excavators, < 300 hp, Work Basis, Filters 1 - 3
Excavators, < 300 hp
I Filter 1
I Filter 2
Filters
'85 Case 1085B
(120 hp)
'97Cat320B
(Tierl)
(128 hp)
'03 Komatsu PC300LC
(Tier 2)
(255 hp)
Figure 6.2-8 PM Emissions from Excavators, > 300 hp, Work Basis, Filters 1 - 3
Excavators, >300hp
I Filter 1
I Filter 2
Filters
'93 Komatsu PC400LC '00 Komatsu PC400LC '06Cat325D
(330 hp) (Tierl) (Tier 3)
(321 hp) (300 hp)
6-23
-------
Figure 6.2-9 Gaseous Emissions from Excavators, < 300 hp, Work Basis, Overall
Test
Excavators, < 300 hp
lHC(g/kW-hr)
I CO (g/kW-hr)
C02(kg/kW-hr)
|Nox(g/kW-hr)
'85 Case '97Cat320B '98Komatsu '03 Komatsu
1085B (Tierl) PC300LC PC300LC
(120 hp) (128 hp) (Tierl) (Tier2)
(232 hp) (255 hp)
Figure 6.2-10 Gaseous Emissions from Excavators, > 300 hp, Work Basis, Overall
Test
Excavators, > 300 hp
'93 Komatsu '00 Komatsu '06Cat325D '06 Deere
PC400LC
(330 hp)
PC400LC
(Tierl)
(321 hp)
(Tier 3)
(300 hp)
450D
(Tier 3)
(349 hp)
lHC(g/kW-hr)
I CO (g/kW-hr)
!C02(kg/kW-hr)
|Nox(g/kW-hr)
6-24
-------
Figure 6.2-11 PM Emissions From Loaders, Work Basis, Filters 1 - 3
Loaders
I Filter 1
Filter 2
I Filter 3
'04Cat953C
(Tier 2)
(128 hp)
' PM results invalid for Filters #1 & #3 on this test
Figure 6.2-12 Gaseous Emissions From Loaders, Work Basis, Overall test
Loaders
'97Case Komatsu '02 Cat963'03 Deere '04Cat
570LXT WA180 (Tierl) 544H 953C
(68 hp) (124 hp) (160 hp) (Tier 2) (Tier 2)
(130 hp) (128 hp)
IHC(g/kW-hr)
!CO(g/kW-hr)
IC02(kg/kW-hr)
INox(g/kW-hr)
6-25
-------
Figure 6.2-13 PM Emissions From Other Equipment, Work Basis, Filters 1 - 3
c
o
a
£
Other Equipment
'87 Cummins4B-3.9
(76 hp)
96 Cat 12H*
(140 hp)
'02CatTH83
(Tierl)
(101 hp)
* PM results invalid for Filterttl on this test
Figure 6.2-14 Gaseous Emissions From Other Equipment, Work Basis, Overall
Test
o
OJ
V)
TO
ID
16
14
12 -
10 -
6 -
4 -
2
0
Other Equipment
'87Cummins4B-3.9
(76 hp)
'02CatTH83
(Tierl)
(101 hp)
lHC(g/kW-hr)
iCO(g/kW-hr)
lC02(kg/kW-hr)
lNox(g/kW-hr)
6-26
-------
Figures 6.2-15 through 6.2-28 present PM and gaseous emissions on a fuel basis (grams
or kg of emissions per gallon of fuel consumed), rather than a work basis, again grouped by
equipment category. PM emissions are based on the first three filters collected, and gaseous
emissions are based on the overall test average (including times when filters were and were not
sampled).
Using a fuel basis to evaluate emissions eliminates several of the points of uncertainty
inherent in work basis estimates. In particular, errors in the estimated engine speed (RPM) have
no effect on emissions, and since load is not used in the emissions estimate, errors associated
with the use of "generic" lug curves, and errors in engine load and efficiency estimations are
eliminated. However, the way a piece of equipment is used (heavy load and high RPM vs. low
load and light RPM or extensive idle periods) can have an influence on the fuel-based emissions
from one test (or piece of equipment) to another. Also, as with the work-based emissions, the
accuracy of the PM measurements are still dependent on the performance of the micro-
proportional sampler. In addition, any errors associated with the SEMTECH-DS' determination
of the second-by-second (and hence cumulative) fuel consumption rate will affect the accuracy
of the fuel-based emissions estimates. These errors have been estimated in Appendix AO,
Nonroad Error Estimates, and are shown in Figures 6.2-15 through 6.2-28.
6-27
-------
Figure 6.2-15 PM Emissions From Backhoe Loaders, Fuel Basis, Filters 1 - 3
BackhoeLoaders
I Filter 1
I Filter 2
Filters
'95 Deere 410D
(75 hp)
'06 Deere 310G
(Tier 2)
(84 hp)
'07 Deere 310J
(Tier 2)
(84 hp)
6-28
-------
Figure 6.2-16 Gaseous Emissions From Backhoe Loaders, Fuel Basis, Overall
Test
BackhoeLoaders
300
lHC(g/gal)
lCO(g/gal)
C02 (kg/gal)
lNox(g/gal)
'95 Deere 410D 06 Deere 310G* '07 Deere 310J
(75 hp) (Tier 2) (Tier 2)
(84 hp) (84 hp)
*NoCO or C02 results due to NDIR signal loss on this test
Figure 6.2-17 PM Emissions From Dozers, 50-99hp, Fuel Basis, Filters 1-3
Dozers,50hp-99 hp
I Filter 1
I Filter 2
Filters
'96CatD4CXL
(87 hp)
'99 Deere 550H
(Tierl)
(84 hp)
6-29
-------
Figure 6.2.18 PM Emissions From Dozers, > 100 hp, Fuel Basis, Filters 1 - 3
Dozers, >100 hp
FiltPi-1
'88 Cat 953 '95Cat963CB '97CatD6RXL 99Cat953C* '98Cat963B
(121 hp) (160 hp) (Tierl) (Tierl) (Tierl)
(175 hp) 170 hp) (220 hp)
* PM results invalid for Filter#3 on this test
Figure 6.2-19 Gaseous Emissions From Dozers, 50-99 hp, Fuel Basis, Overall
Test
250
200
m 150
o
13
100
Dozers,50hp-99 hp
'96CatD4CXL*
(87 hp)
'99 Deere 550H
(Tierl)
(84 hp)
* No HC results due to FID signal loss on this test
lHC(g/gal)
lCO(g/gal)
C02 (kg/gal)
INox(g/gal)
6-30
-------
Figure 6.2-20 Gaseous Emissions From Dozers, > 100 hp, Fuel Basis, Overall
Test
(fL
c
o
a
E
o
9!
re
Dozers, >100 hp
'85 Cat 963 '88 Cat 953 '95 Cat 963CB'97 Cat D6RXL 99 Cat 953C* '98Cat963B
(150 hp) (121 hp) (160 hp) (Tierl) (Tierl) (Tierl)
(175 hp) 170 hp) (220 hp)
*NoCO or C02 results due to NDIR signal loss on this test
Figure 6.2-21 PM Emissions From Excavators, < 300 hp, Fuel Basis, Filters 1 - 3
Excavators, <300hp
'85 Case 1085B
(120 hp)
'97Cat320B
(Tierl)
(128 hp)
I Filter 1
I Filter 2
Filters
'03 Komatsu PC300LC
(Tier 2)
(255 hp)
6-31
-------
Figure 6.2-22 PM Emissions From Excavators, > 300 hp, Fuel Basis, Filters 1 - 3
c
_g
i/i
£
Excavators, >300hp
I Filter 1
I Filter 2
Filters
'93 Komatsu PC400LC
(330 hp)
'00 Komatsu PC400LC
(Tierl)
(321 hp)
'06Cat325D
(TierS)
(300 hp)
Figure 6.2-23 Gaseous Emissions From Excavators, < 300 hp, Fuel Basis, Overall
Test
Excavators, < 300 hp
300
iHC(g/gal)
lCO(g/gal)
C02 (kg/gal)
|Nox(g/gal)
'85 Case
1085B
(120 hp)
'97Cat320B
(Tierl)
(128 hp)
'98 Komatsu
PC300LC
(Tierl)
(232 hp)
'03 Komatsu
PC300LC
(Tier 2)
(255 hp)
6-32
-------
Figure 6.2-24 Gaseous Emissions From Excavators, > 300 hp, Fuel Basis, Overall
Test
160
140
120
.52 100
3
o
01
Excavators, > 300 hp
IHC(g/gal)
iCO(g/gal)
C02 (kg/gal)
INox(g/gal)
'93Komatsu '00 Komatsu '06Cat325D '06 Deere
PC400LC PC400LC (Tier 3) 450D
(330 hp) (Tierl) (300 hp) (Tier3)
(321 hp) (349 hp)
Figure 6.2-25 PM Emissions From Loaders, Fuel Basis, Filters 1 - 3
14
12
58
c
o
£ 6
Loaders
I Filter 1
Filter 2
I Filter 3
97Case570 Komatsu 02Cat963* '03 Deere 544H '04 Cat 953C
(68 hp) WA180 (Tierl) (Tier 2) (Tier 2)
(124 hp) (160 hp) (130 hp) (128 hp)
* PM results invalid for Filters #1 & #3 on this test
6-33
-------
Figure 6.2-26 Gaseous Emissions From Loaders, Fuel Basis, Overall Test
Loaders
300
c
o
in
O
tu
I/I
as
(3
lHC(g/gal)
lCO(g/gal)
C02 (kg/gal)
lNox(g/gal)
'97 Case Komatsu '02 Cat 963 '03 Deere '04 Cat
570LXT WA180 (Tierl) 544H 953C
(68 hp) (124 hp) (160 hp) (Tier 2) (Tier 2)
(130 hp) (128 hp)
Figure 6.2-27 PM Emissions From Other Equipment, Fuel Basis, Filters 1 - 3
Other Equipment
120
I Filter 1
Filter 2
I Filter 3
'87 Cummins4B-3.9
(76 hp)
96 Cat 12H*
(140 hp)
'02CatTH83
(Tierl)
(101 hp)
* PM results invalid for Filter #1 on this test
6-34
-------
Figure 6.2-28 Gaseous Emissions From Other Equipment, Fuel Basis, Overall
Test
Other Equipment
c
o
HI
1/1
3
O
d
1/1
re
iHC(g/gal)
lCO(g/gal)
C02 (kg/gal)
lNox(g/gal)
'87 Cummins 4B-3.9
(76 hp)
'02CatTH83
(Tierl)
(101 hp)
Comparing the profiles of the work basis and fuel basis emissions results between
equivalent plots (i.e., comparing Figure 6.2-1 with 6.2-15, 6.2-2 with 6.2-16, etc.) may help
illustrate the differences between work-based and fuel-based (or other) emissions estimates. As
previously described in Section 6.2.2, extensive idle (or low engine speed / load) operation may
result in higher work-based (brake specific) emissions than work-based emissions calculated
over periods of higher speed/load operation.
Reviewing data and emissions plots from this study suggests engine size and regulatory
tier may not be meaningful stratification variables for estimating emissions rates of diesel
engines, as emission rate variations appear to be influenced less by these observational
parameters as by other parameters such as engine speed range and engine load. The type of work
being done (power-take off, equipment transport, or both) might be a good indicator of both
engine speed ranges and loads used and may be a good operational parameter to consider when
selecting stratification variables for future work. Although potentially limited by the small
sample size, data collected during this study could be evaluated in an effort to identify
6-35
-------
appropriate stratification variables for future studies, considering both observational and
operational parameters in a regression analysis. Such an analysis was beyond the scope of work
in this study.
6.3 PAMS Measurement Results
Table 6.3-1 lists a summary of activity measurements made throughout the study. In this
table, "weekday days" refers to operation on Mondays through Fridays, 7 am - 7:59 pm.
"Weekday nights" operation is defined as operation on Mondays thru Fridays, beginning at 8 pm
each evening (Monday through Friday) and ending at 6:59 am the following morning (Tuesday
through Saturday). Weekend operation is defined as operation beginning Saturday at 7 am and
ending Monday at 6:59 am.
6-36
-------
Table 6.3-1 Activity Measurement Result Summary
Test ID
2208-
1918
3858-
5754
2535-
9216
2535-
2754
0229-
0045
0062-
6976
9429-
7232
1688-
0216
1688-
1462
9429-
0323
3858-
1482
0062-
0934
0229-
3781
8542-
1271
0349-
0567
Equipment
Category
Backhoe
loader
Backhoe
loader
Backhoe
loader
Backhoe
loader
Backhoe
loader
Backhoe
loader
Backhoe
loader
Boring /
Trenching
Boring /
Trenching
Boring /
Trenching
Dozer
Dozer
Excavator
Excavator
Excavator
Equip Type
Backhoe loader
Backhoe loader
Backhoe loader
Backhoe loader
Backhoe loader
Backhoe loader
Backhoe loader
Horizontal Boring
Machine
Horizontal Boring
Machine
Directional Boring
Machine
Crawler Dozer
Tracked Dozer
Wheeled Excavator
Mini Track
Excavator
Track Excavator
Mfr
John Deere
Case
Case
Case
John Deere
John Deere
Caterpillar
Vermeer
Vermeer
Ditch Witch
Caterpillar
Caterpillar
Case
IR Bobcat
Caterpillar
Model
41 OB Turbo
480FLL
5 80 Super L
5 80 Super M
410DTurbo
310G
420E
D20 x 22
D16x20A
JT2020
D4CXL
D4C
1085B
329G
330D
Calendar Days
PAMS
Installed
45
39
42
42
28
36
35
48
48
37
39
36
31
30
33
Equip
Used
17
30
18
18
20
22
18
27
26
19
27
22
23
10
18
Total Minutes of Operation
Overall
1,861
6,069
868
1,966
1,003
2,705
2,715
4,592
3,300
2,888
4,565
4,892
369
801
1,573
Weekday
days
1861
5247
697
1962
991
2610
2715
4590
3300
2888
4109
4892
369
437
1321
Weekday
Nights
0
19
0
0
12
0
0
2
0
0
13
0
0
60
0
Weekend
Days/Nights
0
803
171
3
0
95
0
0
0
0
442
0
0
303
252
6-37
-------
Test ID
1437-
0399
1437-
1396
1911-
1916
8555-
2757
8597-
1096
8597-
0194
9679-
6459
1911-
9540
3597-
095K
3854-
9162
3868-
0304
3928-
1649
3868-
8720
3702-
9726
Equipment
Category
Loader
Loader
Loader
Loader
Loader
Loader
Loader
Other
Other
Other
Other
Telescope
Forklift
Telescope
Forklift
Telescope
Forklift
Equip Type
Compact Skid Steer
Loader
Skidsteer loader
Tracked Loader
Compact Skid Steer
Loader
Compact Track
Loader
Compact Track
Loader
Skid Steer Loader
Concrete Saw
Roller Compactor
Straight-Mast Lift
Truck
Wheeled Crane
Telescopic Lift
Truck
Telescopic Lift
Truck
Telescopic Lift
Truck
Mfr
IR Bobcat
IR Bobcat
IR Bobcat
IR Bobcat
IR Bobcat
IR Bobcat
New Holland
Core Cut
Hyster
Case
Grove
Lull
Ingersoll Rand
Skytrak
Model
873 Turbo
S3 00 Turbo
T300 Turbo
SI 85 Turbo
T190
T250
LX665 Turbo
CC6560 XLS
C340C
586D
RT640C
644B-42
VR-90B
6042
Calendar Days
PAMS
Installed
48
46
22
31
31
30
35
14
44
43
29
48
37
44
Equip
Used
13
25
14
11
21
20
27
11
20
10
6
16
N/A
36
Total Minutes of Operation
Overall
1,008
2,846
1,006
666
1,330
1,828
3,310
1,960
1,768
296
340
1,602
2,599
2,255
Weekday
days
606
2423
1006
628
1294
1388
2811
1960
1456
296
340
1343
N/A
2189
Weekday
Nights
48
26
0
2
24
435
42
0
5
0
0
101
N/A
0
Weekend
Days/Nights
355
397
0
36
12
5
457
0
307
0
0
158
N/A
66
6-38
-------
Table 6.3-2 lists total usage in hours, summed by equipment category. These summed
categories are shown graphically in Figure 6.3-1. Figure 6.3-2 shows these same categories
normalized to percentages.
Table 6.3-2 Activity Summary by Equipment Category (in Hours)
Equipment
Category
Backhoe loader
Boring / Trenching
Dozer
Excavator
Loader
Other
Telescope Forklift
Overall
286.5
179.7
157.6
45.7
199.9
72.7
107.6
Weekday
days
268.1
179.6
150.0
35.5
169.3
67.6
58.9
Weekday
Nights
0.5
0.0
0.2
1.0
9.6
0.1
1.7
Weekend
Days/Nights
17.9
0.0
7.4
9.2
21.0
5.1
3.7
Figure 6.3-1 Activity Summary by Equipment Category (in Hours)
Equipment Usage by Category
3
O
£_
0)
O)
re
"c
0)
Q.
'5
D-
m -innn
o.o -
DWeekdays
Weekday Nights
dWeekend Days/Nights
n
n
n m
rn T^\ r~ i i _^-,
Backhoe loader Boring /Trenching Dozer Excavator Loader Other Telescope Forklift
Equipment Category
6-39
-------
Figure 6.3-2 Activity Summary by Equipment Category (in Percentages)
Equipment Usage by Category
100%
90%
DWeekdays
Weekday Nights
D Weekend Days/Nights
10%
0% H
Backhoe loader Boring/Trenching
Dozer Excavator
Equipment Category
Loader
Other
Telescope Forklift
With consideration of the small sample set of data collected, reviewing the PAMS usage
data does suggest the majority of equipment usage occurs during typical weekday hours. Some
types of equipment did appear to have higher night / weekend usage rates, although this could be
attributed in part to rain, mud and other conditions which prevented operations during typical
hours. Throughout the three ES phases of fieldwork, our experience does indicate that the type
of industry in which each establishment worked did have an effect on what days and times
equipment was operated. Type of work (and hence equipment type) may therefore be a good
indicator of hours of operation (days / nights / weekends). Work hours did appear to be fairly
consistent within establishments.
6-40
-------
7.0 Lessons Learned and Program Recommendations
7.1 Sample Design and Recruitment
Sample frames: This study utilized Survey Sampling International (SSI) as the primary
sampling frame and tested the use of Equipment Data Associates (EDA) as a replacement or
supplemental frame. SSI remains a viable and productive sampling frame; however, as
discussed in Section 4.4, we recommend future considerations of the EDA frame as a
supplemental fame in a dual-frame design to increase coverage. The relative costs of processing
EDA and SSI sample need to be considered and analyzed before implementing this
recommendation.
Two stage sampling: Given the unexpected low prevalence of eligible establishments in
the pilot study, combined with the absence of correlation between data items on the SSI sampling
frame and the actual number of eligible equipment pieces for an establishment, we now believe
that in most if not all situations a census of establishments will be needed even to instrument a
small number of equipment pieces. If censuses are used, then issues of sample design within
PSUs become moot.
However, there may be large metropolitan area such as New York City, Chicago or Los
Angeles where the number of establishments for sampling would far exceed that needed for this
type of study. In these cases we would resist the use of PPS sampling of establishments based on
our findings in this pilot. Instead, we would encourage the creation of a few strata based on
number of employees as follows: first exploit the skewed Pareto distribution of establishments
to create a "large stratum" (say all establishments in the top 20th percentile according to number
of employees), a "zero employee" stratum and a residual stratum. Based on our pilot study we
expect that the eligibility rates to be highest among the "large stratum" and lowest among the
"zero employee" stratum. This could lead to either a proportional allocation sample of
establishments, or a mild optimum (Neyman) allocation stratified sample that employs 'best
estimates' of eligibility rates across strata. But we would not recommend a PPS sample using
number of employees as a measure of size.
Use of incentives: While the incentive tests conducted during this study were
inconclusive, we cautiously recommend their use in future studies. Section 4.5 discusses a
number of explanations for this including possible site or interviewer effects and differences
among establishments. Clearly, it is more important to generate follow-through to
instrumentation rather than assent at the recruitment stage. As such, we recommend that future
research focus on this as the outcome of interest/treatment effect.
7-1
-------
Establishment eligibility: Clearly, the number of eligibility requirements included in a
survey impacts eligibility and ultimately response rates and sample design. Revisions were made
to the questionnaire throughout the study to clarify issues such as fuel type (i.e., diesel versus
gasoline-fueled equipment) or prime versus subcontractor status (the requirement of being a
prime contractor was relaxed following Phase 1). Future studies with other industry sectors or
other geographic locations within the construction sector should incorporate modifications made
during this study. Appendix AK contains the survey questionnaires used throughout the study,
by EOT phase of study.
Survey instrument introduction: A number of enhancements were made to the survey
instrument introduction over the course of the study to reduce the likelihood that an
establishment would refuse to participate in the study at the onset of the interview. The
introduction should mention the Environmental Protection Agency and provide a very concise
one-sentence description of the study that does not allude to eligibility (allowing prospective
establishment to self-determine eligibility at the onset and giving them an easy way to opt out of
the survey). For example, rather than, "... we are conducting a study with construction
companies about the diesel equipment and machinery used in their daily operations" the
following is preferable, "... we are conducting a study with companies about the equipment used
in their daily operations."
Advance letter: We recommend continuing the use of an advance letter with FAQ
brochure and endorsement from trade associations in future work, as these serves a critical
function of pre-notifying establishments about the study.
7.2 General Fieldwork Lessons and Recommendations
Installations during non-working hours was generally found to be ideal for PEMS and
PAMS installations, although occasionally off-hour site access was not possible. When off-hour
site access was not possible, installations either took place during working hours when
equipment was inactive, or installations were not performed at that particular site. Because of
their nature of work, some establishments generally trailered equipment between job sites and
the establishment on a daily basis (out to a job site in the morning, returning to the establishment
at night). PEMS installations were generally not possible for equipment trailered on a daily
basis, because the PEMS rack installed on the equipment was typically higher than could be
safely transported on a trailer (due to interference with traffic lights, bridges, and electrical
wires). In these situations, PEMS installations were performed during periods of inactivity or
were not performed at all.
7-2
-------
In general, conducting fieldwork (both PEMS and PAMS testing) required more field
personnel than originally anticipated. For the PAMS units, locating and revisiting the units was
time consuming because establishments and sites were generally separated by long distances,
and equipment and work sites were generally more transient than originally anticipated. A
significant amount of time was required to coordinate the logistics associated with ongoing
PEMS and PAMS installations and testing.
Due to the transient nature of work of many establishments, fieldwork testing schedules
and plans were difficult to establish in advance. Although it was usually possible to make
tentative plans with establishments, it was also very typical to not know whether a PEMS
installation was going to occur until late in the afternoon the day of the installation. Weather and
establishment work and worksite unpredictability both factored largely into this uncertainty, as
well as the installation team's need to conduct the installation in as unobtrusive of a manner as
possible. This type of field testing required a large degree of flexibility of field team members
who could conduct support activities (such as PEMS calibration and maintenance support and
PAMS revisits) during times when PEMS installations and testing were not taking place.
For both PEMS and PAMS testing, although quick to install, the Capelec RPM collection
devices did not provide a reliable RPM signal. Optical sensors typically worked well, as long as
care was taken to mount in a location which minimized exposure to ambient light, dust and dirt
and moisture (such as rainwater). Caterpillar optical sensor mounts attached to high-strength
magnet bases worked well to hold optical sensors in place. Brackets fabricated on-site using
existing bolts for mounting also provided a secure mount for optical sensors. Non-destructive
taps into the vehicle's engine speed signal harness also worked well for RPM pickup.
For nonroad equipment which is equipped with a 24-V electrical system, PEMS and
PAMS voltage needs to either be taken from a 12V point in the equipments electrical system, or
for PAMS installations, a 24V to 12V power transducer is needed to step down the input voltage
to within the PAMS operating range.
The ERG team originally intended on developing an Internet-based establishment and
equipment sample management system. However, due to the amount of information which
needed to be collected and analyzed by many different people, this approach was abandoned in
favor of using various spreadsheets transferred among participants via a secure Internet-based
file storage, archival and retrieval system (Tortoise Concurrent Versions System (CVS)). For
future projects, an Internet-based sample management system could be developed using
information gathered and learned during this study.
7-3
-------
7.3 PEMS Lessons Learned
Depending on the type of equipment being instrumented, PEMS installation and setup
times typically ranged from two to four hours the prior evening, with an additional two hours of
warm-up and system verification required on the day of testing (prior to the start of emissions
testing).
Due to the size and weight of the PEMS measurement system, installation was facilitated
through use of an outrigger-equipped flatbed truck with an electric crane (leased as part of this
contract). Three to four field staff were needed to safely install PEMS equipment. Many
installations took place in muddy, off-road locations, necessitating the use of four-wheel drive
installation vehicles.
During the first two ES phases of the study, silicon boots and hoses were used to connect
the equipment's exhaust system to the PEMS exhaust flowmeter. However, it was discovered
that this silicon exhaust tubing was not always capable of withstanding the high temperatures and
exhaust flowrates of some of this equipment, which resulted in exhaust tubing burning, melting
through, and delaminating. This process was accelerated for equipment with a tapered, side-exit
exhaust tip, as this type of exhaust tip directed the exhaust flow directly onto the inside of the
tubing. Figure 7.4-1 shows an image of exhaust tubing which was burned and melted, causing
the exhaust connection to come loose from the exhaust tailpipe. Figure 7.4-2 shows exhaust
tubing which became delaminated during testing.
Figure 7.3-1 Burned and Melted Exhaust Tubing
7-4
-------
Figure 7.3-2 Delaminated Exhaust Tubing
In order to correct this problem, tapered, side-exit exhaust tips were removed from
equipment prior to installing exhaust tubing, and the EPA acquired and provided metal tubing
and clamps to be used directly from the exhaust pipe to the exhaust flowmeter, with no silicon
pieces used. Metal boots joined the tailpipe with heavy-duty truck-style clamps. Figure 7.4-3
shows a backhoe loader in use with metal tubing.
Figure 7.3-3 Backhoe Loader with Metal Tubing Installed
Primarily due to equipment space constraints, ambient air was used to zero the
instrument. For future studies, portable disposable containers of zero air could be used rather
7-5
-------
than ambient air. In addition, zeros should be limited to non-sampling episodes because of the
resulting bias in emission measurements and work-basis calculations.
EPA laboratory testing performed after the completion of fieldwork identified the
potential for leaks to occur at the union leading to the gravimetric filter holder. These leaks
would occur if the sample line was not adequately pushed into the sample holder prior to
tightening the assembly. In future studies, leak checks of the gravimetric sampling system
performed prior to collecting sample on each set of filters would help ensure the system had no
leaks. In order to do this, upstream and downstream pressure transducers could be installed prior
to and after the gravimetric filter holders, and prior to each test, a vacuum could be applied to the
gravimetric assembly, and the rate of vacuum decay could be monitored in order to ensure
airtight seals are achieved each time gravimetric filters were replaced. Alternatively, pre and
post-filter mass flow measurements could be performed as part of the system setup process.
Although "real-time" data QC was performed as PEMS testing was being performed,
some issues identified during analysis performed after the testing was completed revealed
additional areas where real-time QC would be beneficial. The "Summary of Nonroad PEMS
data QC criteria" include in Appendix I could be used during review of PEMS test parameter
plots in the field in order to ensure all systems are functioning properly during PEMS testing. In
addition, review of test support data such as pre- and post-test spans (necessary if drift
corrections are to be performed) and time stamps should be performed during or immediately
after data collection. Appendix I provides a good starting point for PEMs QC review that could
be performed in the field.
Accurate RPM collection is critical for developing work-based (brake-specific) emission
estimates. Accurate RPM should always be collected using a reliable method (such as an optical
sensor or a non-destructive tap into the equipment's tachometer signal, if so equipped). ECU
data can also provide an accurate RPM signal, if the delay in initiating acquisition upon
equipment start-up is minimal.
Percent load is also used in determining work-based emission estimates. For this study,
fuel used was compared with maximum fuel rate (at any given RPM) from the engine's lug curve
in order to determine percent load. In order to comply with the work assignment goals of
minimizing the establishment's work interference, no requests were made to equipment operators
regarding how to operate their equipment. For future work, however, it would be beneficial
during PEMS testing to collect data during intentional full load conditions at different engine
speeds which could be compared with fuel consumption estimates from lug curves. This could
7-6
-------
provide information on the accuracy of second-by-second fuel usage estimates used in
determining work-based emissions. Alternatively, load comparisons could be made with ECU
data or rack position as measured using a string potentiometer installed with the PEMS
assembly.
During ES Phase 3 of this study, an optical sensor's RPM signal was input into an analog
channel of the SEMTECH-DS' Automotive Micro-Bench II (AMBII) non-dispersive infrared
(NDIR) analyzer bench. This bench also processes signals for the NDIR CO and CO2 emissions
and also 62 emissions (from the oxygen sensor input through another analog channel). Also
during ES Phase 3, output from the AMBII bench was lost on five of the tests, resulting in loss of
CO, CO2, O2 and RPM information for at least part of those tests. Signal losses continued to
occur after replacement of the AMBII circuit board, and also after switching PEMS units.
Additional investigation of the AMBII board and RPM sensor assembly in order to determine
and rectify the root cause of this type of failure would be beneficial.
For future studies of this nature, the acquisition of ECU data collection equipment for
nonroad equipment from other manufacturers would be of benefit, especially as SAE J1939
becomes more prevalent in the nonroad sector.
Because of the bias caused in cumulative emission estimates (especially PM relative to
gaseous results), autozeros should not be performed during emissions sampling. If autozeros are
necessary, output of an "autozero" flag in the SEMTECH-DS data file is useful to identify and
exclude the data from emissions reporting, and autozeros should never be performed during filter
sampling.
Additional efforts to continue to ruggedize and vibration-isolate the equipment would be
beneficial. Because of the rough usage of some of the tested equipment, some of the test failures
which occurred during the study were due to sample lines becoming disconnected or kinking,
PEMS rack hardware mounts and harnesses breaking, and breaker switches flipping as an
apparent result of heavy vibration. This usage took its toll on the PEMS rack through the course
of the study, and illustrated areas where improvements could be made.
As size was an installation limitation on many pieces of equipment, decreasing the size of
the PEMS rack would increase the number of types of equipment eligible for a PEMS test. As
future redesigns of the PEMS rack are being considered, any effort made toward decreasing the
size (and mass) of the PEMS rack would be of benefit.
7-7
-------
7.4 PAMS Lessons Learned
Corsa dataloggers required supplemental protection against water and the elements, so
sealed Pelican cases were acquired, modified and used to house the Corsa dataloggers during
installations in the later phases of the study. Rubber mounts were placed inside each case to
suspend the datalogger within the case, and one-way valves (duckbill valves) were used to allow
any water that happened to enter the cases to drain. Silicon sealant was used to seal the cases at
the point where the wiring harness passed through, as shown in Figure 7.4-4.
Figure 7.4-1 Pelican Case Housing a Corsa Datalogger
The Corsa dataloggers we used were equipped with an antenna for remote activation and
data collection (via a laptop using a Corsa antenna attached through a USB hub). However,
wireless retrieval of data was unsuccessful due to the slow transmission speed, so the wireless
capability was of little benefit in this type of study. Data was collected by powering down the
datalogger and manually removing the compact flash card.
Due to the wireless transmission capabilities which never went into standby, the Corsa
dataloggers drew 175 mA in standby mode. On equipment with weak batteries which sat
dormant for two or more days, this drain rate was enough to drain the equipment's battery below
a charge necessary for starting the equipment. To prevent this, all Corsa installations were
eventually performed with switched power used as the main power source.
Isaac dataloggers have low standby drain rate. Setting the Isaac datalogger to enter
standby mode based on a switched voltage signal or an RPM on/off value (such as 300 RPM)
was found to work well for installations. However, switched power was generally favorable to
7-S
-------
using an RPM signal as this would likely provide a quicker wake-up time. Isaac dataloggers
were sealed sufficiently well such that an additional enclosure was not warranted, and in fact use
of an additional enclosure was discouraged by the manufacturer because of possible system heat
retention issues.
7-9
-------
8.0 Data Conversion and Delivery
Data from this study was provided in the following formats:
Exports from the recruiting database from all three survey stages were provided,
along with the complete establishment sample frames used during the study.
These database extracts were provided in Excel workbook format.
All raw PAMS data files were provided in csv format, along with the SAS code
used to import and process the PAMS data
All raw PEMS data files (containing continuous gaseous emission rates) were
provided in both unprocessed xml and processed csv formats, along with the SAS
code used to import and process the PEMS data
All scanned PEMS and PAMS instrumentation forms were provided in PDF
format
All scanned site inventory forms were provided in PDF format
The project's comprehensive MySQL database was also provided. This
deliverable is described below:
8.1 MySQL Database Delivery
The data deliverable provided to EPA as part of this project consisted of a MySQL
database in script form. The database was designed based on a structure originally outlined in
Appendix B of the work assignment, and includes tables, relationships, field names, field
formats, and descriptors similar to those proposed.
The following text describes the design of the database, the process of its creation, and
issues ERG staff encountered during preparation of the deliverable related to the structure and
contents of the data.
8.1.2 Creation and Population of Database
ERG used the MySQL Workbench tool to create tables, fields, format, and descriptors
consistent with Appendix B of the work assignment. In some cases, fields were defined in
Appendix B that no longer applied to the data collected at the conclusion of the project, or
otherwise needed to be changed to reflect the data actually collected. Such fields are noted in
more detail in Figure 8.1-1, which presents an entity-relationship diagram (ERD) detailing the
structure of the database, which consists of eight separate tables.
3-1
-------
With the design of the database complete, ERG began gathering raw data collected over
the course of the project, and copying it into Microsoft Excel spreadsheets modeled after each
individual table in the database. This data was obtained from a variety of sources, including
NuStats interview results, inventories of potential sites, logs of sites and equipment visited in the
field, and filter logs. A separate spreadsheet was created for each table to be input into the
database, and was populated accordingly using the data described above. Extensive use was
made of translation tables supplied as part of Appendix B in the work assignment to populate
data fields with applicable codes.
Having populated these spreadsheets, corrections were made to the data in order for it to
be readable by the database in the formats described by Appendix B. Examples of the corrections
made here include changes to date and time formats, filling of nulls, addition of fields for clarity,
and removal of fields no longer applicable to the project. These changes are described in
additional detail below for each individual table and field. Once the spreadsheets were fully
populated with the appropriate data, they were exported in tab-delimited format in order for each
file to be directly imported into the database. In the case of PEMS and PAMS data collected via
instrumentation, SAS programs were written at this point to export the previously quality
checked data to tab-delimited files that could be read directly into the database. Additional edits
were then made to correct line endings and to remove extraneous data characters introduced by
the export from Microsoft Excel.
8-2
-------
Figure 8.1-1 Entity-Relationship Diagram for MySQL Database
331 eqtOwnlwiew V 33 eqtlnvlntenriew T
respID VARCHAR(IO)
sample INT
pssu DECIMAL (8)
dateCompleted VARCHAR(S)
iwerlD VARCHAR(IO)
respNAICS INT
respFIPS INT
respIncentiveGrp INT
iviewMode INT
iwFinal VARCHAR(2)
frameMOS INT
timebeg TIME
Ql INT
Q2A INT
Q2B INT
Q2C INT
Q2D INT
Q2E INT
. Q2F INT
Q2G INT
Q2H INT
Q2I INT
Q2J INT
Q2K INT
Q2t INT
Q2M INT
O_Q2 VARCHAR(75)
Q3businAct INT
Q3AJ>2345 INT
Q3B j>2345 INT
Q4isEmployer INT
respID VARCHAR(IO)
dateCompleted VARCHAR(S)
invFinal INT
timebeg TIME
Q14 INT
Q17MS INT
Q20MS INT
Q21MS INT
Q22MS INT
O_Q22MS VARCHARpS)
Q15INT
Q17A INT
Q20S1 INT
Q21S1 INT
Q22S1 INT
1 O_Q22S1 VARCHAR(75)
| Q17B INT
Q20S2INT
Q21S2 INT
Q22S2 INT
O_Q22S2 VARCHAR(75)
Q17CINT
Q20S3INT
Q21S3 INT
Q22S3INT
O_Q22S3 VARCHAR(75)
AilnualVsPerodic INT
Monthl_Usage INT
Monrh2_Usage INT
Month 3._Usage INT
Month4_Usage INT
II
1
33 eqtlnvEqtLlst W
respID VARCHAR(IO)
siteNum INT
shiftSelectCode INT
eqtPcNum INT
eqtType INT
eqtTypeDesc VARCHAR(50)
eqtMfr VARCHAR(SO)
eqtModel VARCHAR(50)
eqtModelYr INT
eqtSerialNum VARCHAR(120)
PAMSEIigible INT
PEMSEIigble INT
PEMSSelected INT
PAMSSelected INT
pPiece DECIMAL(S)
comments VARCHARPOO)
k
X
33 eqtlnvSlteList
"l
1
1
|
1
1
1
|
1
1
> ^
'
t
| respID VARCHAR(IO)
siteNum INT
siteCode INT
distCodeINT
eqtShifiCode INT
shiftSelectCode INT
siteDesc VARCHAR(250)
EquipmentAtSitelnsbumented
INT
NumSitesOperatJngOnlnvDay INT
_J eqtlnstParam
..respIDVARCHAR(lO)
siteNum INT
, eqtPcNum INT
testIDVARCHAR(12)
eqtSerlalNum VARCHAR(30)
numSites INT
numShlfts INT
numPieces INT
eqtType INT
eqtTypeDesc VARCHAR(50)
eqtMfr VARCHAR(25)
eqtModel VARCHAR(25)
eqtModelYr INT
eqtPlateCode INT
eqtComments VARCHAR(250)
engMfr VARCHAR(25)
engModel VARCHAR(20)
engModelYr INT
engSerialNum VARCHAR(20)
engFamily VARCHAR(20)
engPlateCode INT
engCommenbs VARCHAR(250)
hrMeterCodel INT
hrMetErcode2 INT
, hrMeterBegDate VARCHAR(S)
hrMeterReading DECIMAL
hrMetei Comments VARCHAR(250)
IsExhteak INT
IsAltSignal INT
IsModMal INT
visInspectComm VARCHAR(350)
H J
HI
J eqtftctMty ₯
, testID VARCHAR(12)
rawDateTime DATETIME
rruncDateTime DATETIME
corrRPM DECIMAL
. supplyVoltage DECIMAL
. tripnum INT
ntiip INT
tDtintrip INT
RPM1 DECIMAL
AFCalc DOUBLE
AFStoich DOUBLE
AMBII_CO DOUBLE
AMBII_CO2 DOUBLE
AMBII_COPPM DOUBLE
AMBII_O2 DOUBLE
Abs_Humid DOUBLE
AutozeroFlag INT
AuxTemp DOUBLE
CJCTemp DOUBLE
Chiller Temp DOUBLE
ExhMassFlowCorr DOUBLE
3 eqifllters W
FllterlD VARCHAR(IO) J
. FilterWtmg DECIMAL
DateRcvd VARCHAR(8)
DateUsed VARCHAR(8)
, testIDVARCHAR(12)
HolderlD INT
SequencelD INT
SampleType VARCHAR(20)
FinalStatus VARCHAR(40)
FilterCommenti VARCHAR(250) |
-------
Once the tab-files containing the data had been created, we executed a MySQL script
(FullDB.SQL) generated by the Workbench tool to create a blank database within a MySQL
instance. A separate script (LoadAllData.SQL) was used to load the information contained in the
tab-delimited files into the newly created database structure. Some iteration took place at this
point to ensure that all of the data loaded into the database without errors related to table
relationship integrity, primary key violations or field format errors. Once the data loaded
successfully, we exported the database for transport to EPA by executing the backup function of
the MySQL Administrator tool, which created a self-executing SQL script containing the entirety
of the database. This file was transmitted to EPA via secure FTP and DVD by tracked courier.
8.1.3 Table List/Fields of Interest
Table eqtOwnlview: This table was designed to include results from the equipment
ownership interview, and is related to its child table eqtlnvlview by the RespID field. Fields in
this table that required correction or additional explanation include:
pSSU: This field, the respondent's 2nd stage probability of selection, was originally defined
in Appendix B. However, based on the need to integrate the sample and conduct a census (as
described in Section 3.8 of this report), the relevance of this field was lost, therefore this
field was set to null.
DateCompleted: This information was adjusted to mmddyyyy format to be consistent with
the database structure.
respNAICS: The data in this field was, in most cases, looked up based on SIC provided by
NuStats - these are 4 digit NAICS codes. In some cases in PSU 4 & 5, this data was directly
provided - these are the 3 digit NAICS codes. This data is later used as a basis for eqtType
codes in subsequent child tables.
TimeBeg and TimeEnd fields: The times were converted from 4-digit military time using the
following Excel formula: ROUNDDOWN(Ll,-2) / 2400 + MOD(L1,100) / 1440. The
obtained decimal result was then converted to Time format in Excel.
Table eqtlnvlview: This table was designed to include results from the interview portion of the
on-site equipment inventory, and is related to its child table eqtlnvSiteList by the RespID
field. Fields in this table that required correction or additional explanation include:
DateCompleted: This information was adjusted to mmddyyyy format to be consistent with
the database structure.
invFinal: This field is derived from Column D ("Inventoried?") of Appendix AG of this
report. The numeric codes were based on suggested values presented in translation tables in
Appendix B of the Work Assignment.
8-4
-------
Timebeg and timeend: Times were not recorded for on-site inventories, so these fields were
set to null.
AnnualvsPeriodic: Annual usage information was only collected for some sites in PSUs 4
and 5. If this information was unavailable, the field was set to null. A value of 1 represents
annual operation, and 0 represents periodic operation.
MonthX_Usage: These 12 fields contain usage operating information by month. Values of 1
and 0 represent operation and non-operation, respectively.
Comments: Derived from column L ("Comments") of Appendix AG of the final report.
Note that not all text in that column pertains to inventories; some is relevant only to
instrumentation.
Table eqtlnvSiteList: This table was designed to include results from the site list portion
of the on-site equipment inventory, and is related to its child table eqtlnvEqtList by the RespID
and Sitenum fields. Fields in this table that required correction or additional explanation include:
Sitecode: Primary and secondary visits are represented by codes 01 and 02, respectively.
The first visit to a given site, by date, was designated primary; any other visits were
considered secondary.
DistCode: According to Appendix B translation tables, this field is related to a variable
called traveldistance, which was not recorded. Thus, this field was set to null.
EquipmentAtSitelnstrumented?: This field replaces the wasSiteSelected field originally
proposed in Appendix B. Values of 1 and 0 indicate that equipment at a given site either was
or was not instrumented, respectively.
NumSitesOperatingOnlnvDay: This field replaces the pSite field originally proposed in
Appendix B, and represents the number of sites a given establishment is operating on the
day the inventory was conducted.
Table eqtlnvEqtList: This table was designed to include results from the site list portion of the
on-site equipment inventory, and is related to its child table eqtlnstParam by the RespID,
Sitenum, and EqtPcNum fields. Fields in this table that required correction or additional
explanation include:
eqtType: These codes are derived from those provided in Table A-1.2 of Appendix A-l of
the work assignment, and are base on a cross-reference of NAICS codes and equipment
descriptions.
pPiece. This field represents an individual piece of equipment's selection probability, and
was calculated based on the Sample Selection Weighting Criteria matrix (Figure Figure 5.1-
2 ) presented in Section 5.1.3 of this report. pPiece is equal to the weight for an individual
piece divided by the sum of weights at a given site.
3-5
-------
Table eqtlnsParam: This table was designed to include results from the site list portion
of the on-site equipment inventory, and is related to its child tables eqtActivity, eqtPEMS, and
eqtFilters by the testID field. Fields in this table that required correction or additional
explanation include:
testID: A code consisting of the last 4 digits of a site's RespID and the last 4 digits of the
equipment piece's serial number. This was introduced to the table for linking to PEMS,
PAMS, and filter data in child tables.
numSites: This information was not collected at the testing level, but rather in NuStats
questionnaires and during the inventory, both of which are presented above as parent tables.
This field was set to null.
eqtType: These codes are derived from those provided in Table A-1.2 of Appendix A-l of
the work assignment, and are base on a cross-reference of NAICS codes and equipment
descriptions.
engRatePwrUnits: This field was changed from numeric to character, because the codes
suggested in Appendix B of the work assignment did not allow for definition of horsepower
on an unknown basis (as opposed to gross or net).
engUnitsRating: Similarly, this field was added in order to distinguish between net, gross, or
unknown bases for horsepower.
tailpipeOD2: this field was added to allow for a second dimension of outside pipe diameter,
as specified on data forms used in the field.
InstCode: This field was intended to distinguish between equipment instrumented with
PAMS (code 01) or SEMTECH-D (code 03). We added a code of 05 to allow for equipment
that was tested with both types of instruments.
spotBoxnum: This field was populated with the SEMTECH serial number present in the
header of the raw XML data file.
spotNOxO2sensorID: This field was populated with the NDUV serial number present in the
header of the raw XML data file.
preCalibration fields specified in Appendix B of the work assignment: Due to formatting
issues, these fields are not included in the database structure but instead calibration data is
provided in Appendices Q, R, S, AB and AC.
Visit fields: additional fields were added to allow for multiple site visits to check on and
maintain PAMS equipment
Tables eqtActivity, eqtPEMS, and eqtFilters were designed to include PAMS, PEMS,
and filter data collected in the field, respectively, and have no child tables. The format for these
tables were not explicitly specified in Appendix B of the work assignment, so they were created
to comprehensively include all of the data contained in the QC'd SAS datasets from which they
-------
were derived. In the case ofeqtPEMS, it is important to note that because of differences in
instrumentation settings between different phases of the project, certain fields that apply to data
collected in PSU 1, for example, may not have existed in PSUs 4 & 5. In such cases, the fields
defined in PSU 1 were set to null for PSU 4 and 5 data. The list of all fields included in these
tables is presented in the database schema report provided in Appendix AN.
3-7
-------
9.0 References
Kean, A.J.; Sawyer, R.F.; Harley, R.A. (Nov 2000). A Fuel-Based Assessment of Off-
Road Diesel Engine Emissions." Journal of the Air & Waste Management Association 50, 1929.
Maclntyre, S.; Maples, J; Lee, H.; Kendell, I; Bawks, B. (May 2001). The Transition to
Ultra-Low-Sulfur Diesel Fuel: Effects on Prices and Supply, Energy Information Administration
Hart, C.; Warila, J. (Sept 2006). Activity and Emissions of Diesel Nonroad Equipment in
EPA Region 7, Statement of Work, Work Assignment 01, US Environmental Protection Agency
2-3
National Research Council (2000). Modeling Mobile Sources Emissions, National
Academy of Sciences.
Fulper, C.; Giannelli, R.; Hart, C.; Hawkins, D.; Hu, J.; Warila, J.; (2010). In-Use
Emissions From Nonroad Equipment for EPA Emissions Inventory Modeling (MOVES), SAE
International.
Dieselnet.com. United States Nonroad Diesel Emission Standards, accessed December
30, 2009. http://www.dieselnet.com/standards/us/nonroad.php.
9-1
-------
10.0 Index of Appendices
Appendices will be provided electronically. The following is a list of the appendices to
be provided. Page counts for those appendices formatted for printing are given in parenthesis.
Appendix A - Equipment Ownership Questions (6pages)
Appendix B - Onsite Equipment Questions (3 pages)
Appendix C - Onsite Inventory Data Collection Form (2 pages)
Appendix D - Onsite Inventory Equipment Selection Template (9 pages)
Appendix E - PEMS & PAMS Instrumentation Forms (16pages)
Appendix F - PEMS Installation SOPS (46pages)
Appendix G - PAMS Installation SOPS (16pages)
Appendix H - WA Data Tables (13 pages)
Appendix I - PEMS data QC Criteria (7pages)
Appendix J - Onsite Inventory Team Leader Duties (1 page)
Appendix K - Onsite Installation Manager Duties (2 pages)
Appendix L - Grav Filter Handling SOPS (2 pages)
Appendix M - Oil and Diesel Sampling SOPS (2pages)
Appendix N - Nonroad QAPP & Appendices (217pages)
Appendix O - PSU 1 PAMS Purchase Recommendation Memos (43pages)
Appendix P - EPA & DRIPM Blind Study Results (5 pages)
Appendix Q - ES Phase 1 SEMTECH Daily Span Results (1 page)
Appendix R - ES Phase 2 SEMTECH Daily Span Results (3 pages)
Appendix S - ES Phase 3 SEMTECH Daily Span Results (8 pages)
Appendix T - Nonroad PEMS Fuel and Oil Analysis Results (122 pages)
Appendix U - PEMS and PAMS Testing Details (155pages)
Appendix V - PEMS Filter Log (7pages)
Appendix W - Equipment Inventory Results, all phases (110 pages)
Appendix X - PAMS Data Dictionary and Summary Statistics (6pages)
Appendix Y - PEMS Data QC Results (94pages)
Appendix Z - MPS to Exhaust Flow Proportionality Plots (30pages)
Appendix AA - Summary of RPM correlations developed (17pages)
Appendix AB - EFM / MPS calibration verification results (Not formatted for printing)
Appendix AC - SEMTECH multipoint linearity verification results (Not formatted for printing)
Appendix AD - BSFC Calculation Methodology (6pages)
Appendix AE - PAMS Acquisition Delay Test Results (1 page)
Appendix AF - PAMS Data Review Notes (11 pages)
Appendix AG - Inventory and Instrumentation Counts (3 pages)
Appendix AH - PEMS Measurement Results (15 pages)
Appendix AI - Advance Mailout Letter (1 page)
Appendix AJ - Advance FAQ Brochure (2 pages)
Appendix AK - Survey Questionnaires by Phase (20 pages)
Appendix AL - Expected Study Dispositions (5pages)
Appendix AM - Cognitive Interview Script (3 pages)
Appendix AN - MySQL Database Schema Report (20pages)
Appendix AO - Uncertainty and Bias Estimates for Gaseous and PM Emissions (11 pages)
10-1
------- |