Peer Review of LBNL Statistical
Analysis of the Effect of Vehicle Mass &
Footprint Reduction on Safety (LBNL
Phase 1 and 2 Reports)
&EPA
United States
Environmental Protection
Agency
-------
Peer Review of LBNL Statistical
Analysis of the Effect of Vehicle Mass &
Footprint Reduction on Safety (LBNL
Phase 1 and 2 Reports)
Assessment and Standards Division
Office of Transportation and Air Quality
U.S. Environmental Protection Agency
Prepared for EPA by
Systems Research and Application Corporation
EPA Contract No. EP-C-11-007
&EPA
United States
Environmental Protection
Agency
EPA-420-R-12-020
August 2012
-------
Peer Review of LBNL Statistical Analysis of the
Effect of Vehicle Mass & Footprint Reduction
on Safety (LBNL Phase 1 and 2 Reports)
Table of Contents
Peer Review of the LBNL Phase 1 and 2 Reports, Conducted by SRA International
p. 4
1. Background p. 4
2. Description of Review Process p. 4
3. Compilation of Review Comments p. 6
4. References p. 53
Appendices
A. Resumes of Peer Reviewers p. 54
B. Conflict of Interest Statements p. 86
C. Peer Review Charge & Conference Call Notes p. 98
D. Reviews p. 100
EPA's Response to Peer Review Comments p. 158
-------
Executive Summary
In September 2011, EPA contracted with SRA International (SRA) to conduct a peer review of LBNL
Statistical Analysis of the Effect of Vehicle Mass & Footprint Reduction on Safety) (LBNL Phase 1 and 2
Reports), prepared by Tom Wenzel, Lawrence Berkeley National Laboratory.
The peer reviewers selected by SRA were Donna Chen and Kara Kockelman (University of Texas at
Austin), Charles Farmer (Insurance Institute for Highway Safety), David Greene (Oak Ridge National
Laboratory), and Michael Van Auken (Dynamic Research, Inc.). EPA would like to extend its appreciation
to all three reviewers for their efforts in evaluating this survey. The reviewers brought useful and
distinctive views in response to the charge questions.
The first section of this document contains the final SRA report summarizing the peer review of the LBNL
Phase 1 and 2 Reports, including the detailed comments of each peer reviewer and a compilation of
reviewer comments according to the series of specific questions set forth in the peer review charge.
The SRA report also contains the peer reviewers' resumes, completed conflict of interest and bias
questionnaires for each reviewer, and the peer review charge letter. The second major section contains
our responses to the peer reviewers' comments. In this section, we repeat the compiled comments
provided by SRA and, after each section of comments, provide our response. We have retained the
organization reflected in SRA's compilation of the comments to aid the reader in moving from the SRA
report to our responses.
-------
TO: Cheryl Caffrey, U.S. Environmental Protection Agency, Office of Transportation and Air
Quality (OTAQ)
FROM: Brian Menard, SRA International
DATE: February 28, 2012
SU BJ ECT: Peer Review of LBNL Statistical Analysis of the Effect of Vehicle Mass & Footprint
Reduction on Safety) (LBNL Phase 1 and 2 Reports), prepared by Tom Wenzel, Lawrence
Berkeley National Laboratory.
1. Background
In developing programs to reduce greenhouse gas (GHG) emissions and increase fuel economy of light-
duty highway vehicles, the U.S. Environmental Protection Agency (EPA) and the National Highway
Transportation Safety Administration (NHTSA) have to evaluate the safety of mass reduction
technologies likely to be used to meet future standards. The U.S. Department of Energy (DOE)
contracted with Lawrence Berkeley National Laboratory (LBNL) to perform a statistical analysis of the
effect of vehicle mass and footprint reduction on safety. LBNL's analysis of the relationship between
vehicle mass, footprint, and societal fatality and casualty risk consisted of two phases. Phase 1 was an
assessment of the NHTSA report Relationships between Fatality Risk, Mass, and Footprint in Model Year
2000-2007 Passenger Cars and LTVs. This study used logistic regression analysis to estimate the
relationship of changes in vehicle mass and footprint on U.S. fatality risk per vehicle mile traveled.
Phase 2 was an independent logistic regression analysis to estimate the relationship between vehicle
mass, footprint and total casualty (fatality plus serious injury) risk, per police-reported crash, using state-
level data on all crashes.
This report documents the peer review of Assessment of NHTSA's Report "Relationships Between
Fatality Risk, Mass, and Footprint in Model Year 2000-2007 Passenger Cars and LTVs" (LBNL Phase 1
Report) and Analysis of the Relationship between Casualty Risk Per Crash and Vehicle Mass and Footprint
for Model Year 2000-2007 Light-Duty Vehicles (LBNL Phase 2 Report). Section 2 of this memorandum
describes the process for selecting reviewers, administering the review process, and closing the peer
review. Section 3 summarizes reviewer comments according to the series of specific questions set forth
in the peer review charge. The appendices to the memorandum contain the peer reviewers' resumes,
completed conflict of interest and bias questionnaires for each reviewer, and the peer review charge
letter.
2. Description of Review Process
In September 2011, OTAQ contacted SRA International to facilitate the peer review of the LBNL Phase 1
and 2 Reports. The reports were prepared by Tom Wenzel, Lawrence Berkeley National Laboratory.
EPA provided SRA with a short list of subject matter experts from academia and industry to serve as a
"starting point" from which to assemble a list of peer reviewer candidates. SRA selected three
independent (as defined in Sections 1.2.6 and 1.2.7 of EPA's Peer Review Handbook, Third Edition)
subject matter experts to conduct the requested reviews. SRA selected subject matter experts with
-------
experience in statistics and statistical model analysis, vehicle mass safety, and vehicle crash and safety
engineering. The coverage of these subject areas is shown below in Table A.
Table A:
Peer Reviewer Experience and Expertise
Name
David Green
Charles Farmer
Donna Chen& Kar
Kockelman
Michael
Van Auken
Affiliation
Oak Ridge
National
Laboratory
IIHS
University of
Texas
Dynamic
Research Inc.
Coverage
Statistics &
methodology
Y
Y
Y
Y
Knowledge of
past vehicle
mass safety
statistics study
Y
Y
Y
Y
Knowledge of
vehicle
crash/safety
engineering
/
Y
Y
Y
Knowledge of
vehicle safety
database (FAS,
State Crash &
Vehicle Attributes)
Y
Y
Y
Y
Statistical
model
analysis
Y
Y
Y
Y
To ensure the independence and impartiality of the peer review, SRA was solely responsible for
selecting the peer review panel. Appendix A of this report contains the resumes of the three peer
reviewers. A crucial element in selecting peer reviewers was to determine whether reviewers had any
actual or perceived conflicts of interest or bias that might prevent them from conducting a fair and
impartial review of the CVCM and documentation. SRA required each reviewer to complete and sign a
conflict of interest and bias questionnaire. Appendix B of this report contains an explanation of the
process and standards for judging conflict and bias along with copies of each reviewer's signed
questionnaire.
SRA provided the reviewers a copy of the most recent version of the LBNL Phase 1 and 2 Reports as well
as the peer review charge containing specific questions EPA asked the reviewers to address. The charge
included a matrix of questions issues upon which the reviewers were asked to comment. Reviewers
were also encouraged to provide additional comments, particularly in their areas of expertise and work
experience. Appendix C of this report contains the memo to reviewers from SRA with the peer review
charge and response matrix.
EPA sought peer reviewers' expert opinions on the statistic methodologies used in the LBNL Phase 1 and
2 Reports and whether they are likely to yield realistic estimates of the relationship between vehicle
mass, footprint, and total fatality or casualty risk. EPA requested that each reviewer comment on all
aspects of the two LBNL studies, with particular emphasis on the methodologies employed, assumptions
inherent to the analysis, sources of information employed, methods of calculation and any other key
issues the reviewer may identify. Reviewers were encouraged to examine and evaluate the NHTSA
study in helping them to understand the LBNL assessment analysis. Findings of this peer review may be
-------
used toward validation and improvement of the statistical analysis conducted by LBNL, and to inform
EPA staff on potential use of the regression results for predicting the safety effect of future standards in
reducing mass and footprint.
A teleconference between EPA, LBNL, the reviewers, and SRA was held to allow reviewers the
opportunity to raise any questions or concerns they might have about the LBNL Phase 1 and 2 Reports,
and to raise any other related issues with EPA and SRA, including EPA's expectations for the reviewers'
final review comments. SRA delivered the final review comments to EPA by the requested date. These
reviews, contained in Appendix D of this report, included the reviewers' response to the specific charge
questions and any additional comments they might have had.
3. Compilation of Review Comments
The LBNL Phasel and 2 Reports were reviewed by Donna Chen and Kara Kockelman (University of Texas
at Austin), Charles Farmer (Insurance Institute for Highway Safety), David Greene (Oak Ridge National
Laboratory), and Michael Van Auken (Dynamic Research, Inc.). Appendix A contains detailed resumes
for each of the reviewers. This section provides a compilation of their comments. The complete
comments may be found in Appendix D.
-------
Assessment of NHTSA's Report
''Relationships Between Fatality Risk, Mass, and Footprint in Model Year 2000-2007 Passenger Cars and LTVs"
(LBNL Phase 1 Report)
1. ASSUMPTIONS
COMMENTS
Please comment on the validity of any
assumptions embedded in the LBNL
assessment analysis and the independent
casualty analysis that could affect the
projected relationship between vehicle
mass/footprint reductions and
fatality/casualty risk. Examples might include
assumptions regarding whether recent
historical relationships between vehicle
weight, size, and safety will continue into the
future; potential future improvements in
vehicle technology and design may result in
compensatory safety benefits; and the annual
baseline fatality distribution.
[Chen and Kockelman] [1] The report does a nice job discussing recent trends in vehicles, such as the increase
of ESC, side airbags, and light truck crash compatibility with passenger cars - which will improve safety
outcomes for all vehicles, but perhaps most significantly the smaller and lighter vehicles. . It also mentions the
phasing out of the lightest and smallest vehicles between model years 2000-2007 (but doesn't mention the
makes and models somehow), which were particularly poor safety performers in the past. However, with the
introduction of urban commuter vehicles, such as the SmartCar, Mini Cooper, and Fiat 500m, and the growing
popularity of smaller, fuel-efficient compact vehicles following gas price increases, this trend does not seem so
obvious. Such vehicles should be discussed.
[2] The simplistic logistic model employed in this analysis only accounts for two crash outcomes (fatal versus
non-fatal) and so neglects the more detailed, and ordered nature of injury severity data, which is unfortunate.
The model also assumes error-term homoscedasticity from one crash or individual to the next; in reality
certain vehicle types (e.g., pickups) and crash contexts (e.g., high speed crashes) have more uncertainty
associated with their severity outcomes. It would be good to point out such limitations for readers.
[Van Auken] The basic assumptions, methodology, and data are primarily the same as in the Kahane (2011)
report. These include the following:
1) The probability of a crash fatality is proportional to the vehicle miles travelled (VMT), except as noted in
Section 5.1
2) The logarithm of probability of fatality per VMT for a given curb weight, footprint, and control variable
values varies as a linear combination of the curb weight, footprint, and control variables within the domain
of the data.
3) The logistic regression methods determine a maximum likelihood estimate of model coefficients.
4) It is assumed that the above relationships remain constant in the recent past (i.e., 2000-2007 model year
vehicles in the 2002-2008 calendar years), present, and near future (i.e., 2017-2025 model year vehicles).
The first assumption that crash fatalities are proportional to VMT ratherthan the number of vehicle
-------
registration years (VRY) is appropriate because the fatalities cannot occur if the vehicles are not driven on the
road (i.e., VMT = 0). This assumption is qualified however because VMT is more difficult to measure than VRY
and therefore may be less accurate. On the other hand the probability of a fatal crash or the number of
fatalities in a crash may also depend on the vehicle occupancy. The analysis in Section 5.1 is a commendable
attempt to explore the sensitivity to this assumption, however the Kahane (1997) and DRI (2003-2005) reports
have shown that some driver, vehicle, and environmental factors may be underrepresented or
overrepresented in unweighted induced-exposure data. VRY could have also been considered as a measure of
exposure.
The second and third assumptions are appropriate provided that it is recognized that it is essentially
impossible with currently available knowledge and information to model all of the factors that could affect the
probability of fatality in a crash, and that the objective of the analysis is to identify overall trends versus
vehicle weight and footprint. In general the probability of fatality depends on other many other factors which
have not been modeled (e.g., driver behavior factors, vehicle design factors, roadway design factors, EMT
factors) and these unmodeled factors are assumed to be uncorrelated with vehicle weight and footprint,
and/or are represented by the other control variables. The latter assumption might or might not be valid.
The fourth assumption is perhaps the weakest because it assumes that future vehicles will have the same
design characteristics as past vehicles, and that the characteristics of the vehicle population (e.g., collision
partner weight, size, type) will also remain the same. A commendable attempt to partially address this effect
is described in Section 6. These effects can be perhaps better addressed by the "Volpe model" described in
Kahane (2011) of the Honda-DRI fleet systems model described in Refs (2324), which can be used to forecast
the effects of mass reductions of individual makes and models on a year-by-year basis.
Please comment on any apparent unstated or
implicit assumptions and related caveats or
limitations.
[Chen and Kockelman] The role of driver behavior is briefly addressed in the report but not emphasized
sufficiently. Fatality risk is a combination of driver, vehicle, and roadway characteristics. Driver behavioral
differences are many and do not solely exist for pickup truck drivers versus car drivers. Socioeconomic data
such driver household income, size, and education influence driver attitudes and driving environments. For
example, Chen et al. (2010) found that crash risk increases for those living in socioeconomically disadvantaged
areas (including households more likely to drive less expensive and older vehicles). Though such data is not
typically available in state and national crash databases, the importance of these driver and environmental
characteristics on crash rates (per mile driven) and fatality risk should be stressed in both reports. It is clearly
very difficult to control for, but a major caveat to the NHTSA (& now LBNL) results. We expect that crash
severity could be probably be lower for many of the small cars and pickups if they were driven by those who
-------
tend to drive more expensive vehicles, under the same settings (e.g., daytime, urban freeway). Similarly, in the
second LBNL report (which uses VMT estimates), we expect that crash rates would probably be lower for these
types of driver-vehicle-setting combinations.
[Farmer] The statistical models assume no interaction between the vehicle size/weight measures and any of
the numerous covariates, but this may not be true. For example, size/weight reductions may differently affect
vehicles with and without ESC if they affect vehicle handling. It is risky to make statements such as that on p.
11 of the Phase I report: Therefore, the mass of a lighter car could be reduced by 800 Ibs while adding ESC,
without increasing fatality risk.
[Van Auken] The induced-exposure data set provided by NHTSA is based on the "non-culpable" vehicle in two-
vehicle crashes. It is assumed that the dataset is a reprehensive sample of the driver and environmental
exposure factors for vehicle use. However, since these cases include moving vehicles, some vehicle-driver-
environmental conditions may be under or over represented in this data depending on how they affect the
ability of a non-culpable vehicle to avoid a crash. Results in Ref (17) indicate that the estimated effect of
weight and size reduction are sensitive to whether the induced-exposure data are based on the Kahane (2003)
non-culpable vehicle definition of the Kahane (1996) stopped vehicle definition.
Unfortunately it is not currently possible to test this sensitivity with the NHTSA-provided induced-exposure
data.
ADDITIONAL COMMENTS:
[Chen and Kockelman] Chen, H.Y., Ivers, R.Q., Mariniuk, A.L.C., Boufous, S., Senserrick, T., Woodward, M., Stevenson, M. and Norton R. Socioeconomic status
and risk of car crash injury, independent of place of residence and driving exposure: Results from the DRIVE study. Journal of Epidemiology and Community
Health 64(10), 2010, pp. 998-1003.
-------
2. CONTROL AND DEPENDENT
VARIABLES USED IN THE REGRESSION
MODELS
COMMENTS
Please comment on the adequacy of control
and dependent variables used in the
assessment analysis and independent casualty
analysis, and recommend any alternative
control or dependent variables that are
available for possible inclusion in the analysis.
For example, what are the relative merits of
the main dependent variables used, fatality
risk per estimated VMT, and casualty risk per
police-reported crash?
[Chen and Kockelman] [1] As alluded to above, a primary concern is that the NHTSA analysis (& thus the LBNL
analyses) largely neglect the idea that vehicle type (make & model) is very much a proxy for driver type, and a
vehicle's crash avoidance may have very little to do with vehicle type. It has a lot to do with the person behind
the wheel, and gender & age simply aren't enough to control for such distinctions. Education, risk aversion,
ability, wealth, etc., are important covariates. But existing data sets are quite limiting (though the MVOSS &
FAR with 3-year driver violation history do offer some valuable insights, not discussed in these reports). In
reality, small cars may be less crash prone than Kahane's & Wenzel's results suggest, because they are driven
by lower-income, younger, less risk averse people driving in more crash prone settings (e.g., commercial strips
rather than pricey residential suburbs). Such key caveats need thoughtful discussion. Four relevant papers on
the topics of crash frequency and vehicle size-and-weight implications (by Knipling, Kweon & Kockelman, Wang
and Kockelman, and Chen & Kockelman) have been sent to Tom Wenzel. These all include useful literature
reviews for further connections to useful findings for citation in the reports, as time allows the contractor.
[2] The grouping of the vehicles into heavier- and lighter-than-average weight categories essentially splits a
"typical" weight vehicle of that type into two categories. The impacts of curb weight and footprint on fatality
risk may be easier to interpret if the vehicles were grouped into 3 weight categories (light, average, and heavy)
& by type (with the average category representing vehicles within one standard deviation of average weight).
Furthermore, the grouping of CUVs and minivans into the same vehicle type category neglects the fact that
these vehicles have faced rather different ground clearance requirements (impacting rollover potential), door
types (sliding vs. standard), and, perhaps most importantly, can appeal to different types of drivers (as
indicated in the market shift of car drivers to CUV drivers).
[Farmer] One needs to restrict control variables to those that are available and reliable. A problem when
combining state databases is that the states often are not consistent as to the variables coded and the
definitions of those variables. This severely limits the list of possible control variables.
[Van Auken] [1] The main metric used in both the Kahane (2011) and Wenzel (2011a) reports is the total
number of fatalities (except as noted). Reducing the total number of fatalities, which includes both subject
vehicle occupants and collision partner fatalities, is desirable from a societal viewpoint. Fatal crash occurrence
is related to the total number of fatalities, which has been used by Kahane (2003, 2011) to address concerns
10
-------
about double counting.
[2] VMT is a good measure of accident exposure provided that it can be accurately determined. [Note: This
peer review does not address the Wenzel (2011b) companion report (Ref 3) which examines the risks per
police-reported crash. See Ref 4 for comments on the companion report.]
What additional control variables, such as
vehicle make or model, might be included in
the regression models?
[Chen and Kockelman] [1] Vehicle height, a variable which may be more valuable than vehicle type for
similarly structured vehicles such as sedans, wagons, CUVs, and minivans, would be a valuable control variable.
In addition to a wider track, a lower center of gravity also increases vehicle stability, thereby reducing the risk
of rollover. Relevant literature & findings exist, and should be cited.
[2] Other variables which have been found in past studies to influence fatality risk such as seat belt use,
roadway geometry and division type are not included in this study (which is largely a repeat of the NHTSA
study, as specifically contracted by the EPA).
[3] To account for driver characteristics that contribute to fatality risk, socioeconomic variables such as
household income, education, household size, etc. would be valuable additions. Unfortunately, both state and
national crash databases typically do not include such information (outside of MVOSS). Such issues should be
flagged for readers. It seems the contractor has done his duty, and the key limitations lie with the original
methodology he was to essentially duplicate.
Please comment on any caveats or limitations
that these dependent variable or control
variables entail with respect to use of the
results as the basis for estimating the safety
effect of mass reduction.
[Chen and Kockelman] Please see above comment (in Assumptions section) regarding driver behavior and
environment.
[Farmer] Model overspecification could be the reason for results that are non-intuitive, especially in the Phase
II analyses of police-reported crashes. Control variables may be correlated with each other or with the size
and weight variables. For example, Figure 2.9 of the Phase I report implies that torso side airbags increase
fatality risk in CUVs.
ADDITIONAL COMMENTS:
[Chen and Kockelman] Table 2.1 has many indicator variables labeled as "C" for continuous variable (such as ABS, ESC, AWD, DRVMALE, etc). These C's should
be removed.
[Van Auken] The underlying reasons for some of the estimated effects are unknown at this time, but presumably involve driver, vehicles, environment or
11
-------
accident factors that have not been controlled for in the Kahane (2011) and Wenzel analyses. See, for example, Refs 17 and 25.
12
-------
3. METHODOLOGY AND STATISTICS
COMMENTS
Please comment on the validity and
applicability of the methodology LBNL used in
assessing the NHTSA 2011 study and its
analysis of the relationship between mass,
footprint, and risks per police-reported crash.
[Chen and Kockelman] The report assesses the NHTSA 2011 study in a fair amount of detail and seeks to
introduce some additional analyses to better examine the relationship between mass, footprint, and fatality
risks. However, due to a lack of control for very specific vehicle differences (which vary by make & sub-model),
the exclusion of driver characteristics and crash setting details (which cannot always be controlled for, but are
often correlated with vehicle type), the effects of downweighting vehicles and/or shifting vehicle styles and
sizes may be overestimated. Simply changing the vehicle on a risky driver in a high-risk setting is unlikely to
influence outcomes significantly.
[Van Auken] The logistic regression methods seem to be appropriate. The confidence intervals are based on
the logistic regression Wald Chi-Square statistic, which as Kahane (2003, 2011) has demonstrated does not
include all sources of variation. However, these confidence intervals are useful because they do provide some
indication of the uncertainty in the results. [Note: This peer review does not address the Wenzel (2011b)
companion report, which examines the risks per police-reported crash. See Ref 4 for comments on the
companion report.]
Please review other statistical methods LBNL
has used in the analysis, in addition to the
logistic regression methodology. Examples
include the alternative approaches used by
LBNL to assess NHTSA interval estimation
results, and LBNL's linear regression analysis
of actual, predicted, and residual risk by
vehicle model.
[Chen and Kockelman] [1] In the alternative measures of exposure, the author examines the effect of vehicle
manufacturer on fatality risk and treats the luxury models produced by Toyota, Honda, and Nissan as separate
manufacturers. However, domestic luxury brands (such as Cadillac & Lincoln) are categorized with their
nameplate manufacturers (GM and Ford), which appears inconsistent.
[2] The effect of calendar year variables on fatality risk may be overestimated here, since VMT is tracked by
vehicle model and not by calendar year. The trend of greatest fatality risk reductions in light trucks, CUVs and
minivans with increasing calendar year may simply be a reflection of rising gas prices in combination with the
ailing economy contributing to lower VMT (in these relatively low-fuel-economy vehicles).
[3] It is unclear how the author determined the various percentage replacements of vehicle types in the
aggressive vehicle market share shift scenario. (For example, why are 50% of SUVs replaced by CUVs and 60%
of small pickups replaced by CUVs? The CUV is a more natural replacement for an SUV, and an SUV a more
natural replacement for a pickup.)
[Van Auken] [1] The correlations in Section 3 appear to be assessed using the Coefficient of Multiple
Determination (R2) based on a linear fit to the data (e.g., the correlation between footprint versus curb weight
in Figure 3.1 on p. 14). The linear regression model attributes the differences between the dependent variable
13
-------
(vertical axis_ and the linear fit to the independent variable (horizontal axis) to random effects. If there is no
preference as to the choice of independent and dependent variables (e.g., footprint versus curb weight, or
curb weight versus footprint), then the linear trend and R2 result would be different if the two variables were
interchanged, and having two different yet equally valid results would be undesirable.
[2] If the variation in the data can be attributed to both variables (e.g., footprint and curb weight), then it
would be better to report the square of the sample correlation coefficient r2,, where r is computed according
to Eqn (1). The trend lines in these correlation figures should not be computed using a linear regression.
Instead, the trend line should pass through the sample means (I.e. (x, y)), and have a slope equal to the ratio of
the sample standard deviations in the data (i.e., sy/sx). Therefore, the reported correlation results do not
depend on the ordering of the data variables.
Note this comments does not apply to linear trends indicated in Section 4, for which the Coefficient of Multiple
Determination (R2) seems appropriate.
[3] The Coefficient of Multiple Determination (R2) is frequently used in the Wenzel (2011a) report as an
indicator of the statistical importance of a linear trend (e.g., lvalues in Tables 4.1 and 4.2 that are greater
than 0.3 are shown in blue font). It would be better to report the standard error, confidence interval, and/or
probability value as measures of the statistical significance of a linear trend.
Please comment on caveats or limitations of
using non-significant regression estimates to
project the safety impact of mass reduction.
[Chen and Kockelman] First, the t-statistics are not provided in the report which makes it difficult for the
reader to assess statistical significance of specific regression estimates (except where noted by the author).
Second, inclusion of a statistically insignificant variable can influence the estimates of coefficients associated
with related variables. Nevertheless, in general, it is best to keep insignificant estimates if one has a strong
defense for their role, since removing such variables (& thus their parameters) will shift the burden of
response to a correlated covariate's parameter, thus biasing the latter. We generally keep key covariates in a
model up to a pvalue of 0.20 or 0.25 or so, especially in relatively small data sets (e.g., n < 1,000). Covariates
for which we have no strong basis can be removed for pvalues > 0.10.
[Van Auken] Regression estimates are random numbers which have an unknown expected value and variance,
and known sample value and standard error. If the sample value can be explained by a zero expected value
and known standard error then the result is considered not statistically significantly different than zero and
therefore the result is not considered to be statistically significant. However, If we can combine this estimate
with other estimates then the unknown expected values and variances can also be combined using the same
14
-------
transformation, and the statistical significance of the combined result can be tested. Therefore, depending on
the sample values and inter-correlation, the combined result may be statistically significant even if the
individual estimates are not statistically significant.
For example, the results from each of the nine different crash types can be combined into an overall estimate
and the standard error calculated assuming that the results for each crash type are independent of each other.
Then the statistical significance of the combined effect can be determined.
However, and Kahane (2011) points out there are two sources of uncertainty in the regression results. The
first is the PARS based sampling error which is uncorrelated across crash types because they are based on
different fatal cases (Kahane 2011, p. 77). The second is the state based include-exposure sampling error
which is correlated across crash types because they are based on the same included-exposure cases.
Therefore a confidence interval estimated using the jackknife method described by Kahane (2011) and
accounting for correlation of these two error sources would be more accurate than a simple estimate based on
the Wald Chi-Square statistic and assumed independence.
How might the LBNL methodology be
strengthened to better represent future
vehicle designs and reduce multi-collinearity
between mass and footprint in the regression
analysis?
[Chen and Kockelman] Including more vehicle-specific characteristics (such as vehicle height and engine size)
reduces the analysis' dependence on vehicle type, since vehicle shapes and structures will continue to evolve.
There is also correlation with context (e.g., pickups are driven in more rural locations, with greater hazards
[like less lighting, higher speed, & few medians]). Disaggregate data are almost always best, to avoid ecological
fallacies & such.
[Van Auken] [1] The effects of multi-collinearity can be mitigated by 1) obtaining more data, 2) pooling data
from different crash type or vehicle types, or 3) reducing the number of regression variables. The first option
would require more calendar years and/or model years, which would involve added newer data as it becomes
available (or using older data). The second option might be to recombine the CUVs and minivans with truck
based vans and adding a control variable to compensate for the differences in the vehicles types. The third
option might involve removing statistically insignificant control variables or removing control variables that
would not be expected to have an effect on the probability of fatality in the crash (e.g., the side airbag variable
is not included in pedestrian crashes because it is not expected to affect pedestrian fatality risk). The number
of driver age control variables might be reduced from eight to three (as in the Kahane (1997) and DRI (2002-
2005) studies). Finally, a linear curb weight model instead of a two-piece linear model may help to better
elucidate the general trend.
15
-------
[2] The Variance Inflation Factor (VIF) has been suggested as a measure of multi-collinearity in the Kahane
(2010 and 2011) reports, however this diagnostic metric does not account for differences in database size (i.e.,
Options 1 and 2 above). The Wenzel (2011a) report does not discuss the Variance Inflation Factor or report
any VIF results.
ADDITIONAL COMMENTS:
[Chen and Kockelman] On page 55, it is unclear what is meant by "however; if anything, reduction of this type of fatality will increase detrimental effect of
mass reduction in cars."
16
-------
4. DATA SETS
COMMENTS
Please comment on the validity and
applicability of the datasets used to project
changes in risk resulting from reduction in
vehicle mass. LBNL's casualty analysis used
police-reported crash data from 16 states,
while the 2011 NHTSA study used national
fatality data, combined with a subset of non-
culpable vehicles involved in two-vehicle
crashes from police-reported crash data from
13 states.
[Chen and Kockelman] [1] The acquisition of Polk data for VMT estimates by make & model is valuable, and a
contribution to the literature. However, these estimates come from vehicles found in repair shops in
non-attainment areas, and so will be biased towards problem-prone vehicles, wealthier households who
service their vehicles more regularly, and/or urban (smoggier) areas. Such issues merit careful discussion in the
paper, so that readers are well aware of caveats.
[2] Related to this, Tom Wenzel indicated (by phone) that he did take a look at CA's extensive odometer reads,
which go into some semi-rural locations (not too rural), and he indicated that the VMT values by vehicle type
(not controlling for HH attributes & such) are very similar (just 5% longer in rural areas) -except for vans
(which are used much more extensively in rural areas). This is interesting to me, and is not that different from
what we've seen in the past. For example, Kockelman & Zhao's JTS paper from 2000 (pre- print at
http://www.ce.utexas.edu/prof/kockelman/public_html/
BTSJournalLDTs.pdf) suggests that, after controlling for various HH attributes & vehicle types, density is/was
still very important (tables 1 & 2), but a shift from a density of Ik to 5k persons per sq mile (which is 1.5 vs. 7.8
persons per acre) means an increase of 750 mi/yr/vehicle (which is about 7.5% of annual VMT). Such
differences, and their practical significance (or lack thereof) should be discussed in the reports.
[Van Auken] The inducted-exposure data set provided by NHTSA is based on the non-culpable vehicles in two-
vehicle crashes. See the comments in Table 1 on the limitations of this data. In addition, there are also many
differences in the coding variables and values used by the different states, which tend to make the receding to
a common data set imprecise. [Note: This peer review does not address the Wenzel (2011b) companion
report, which examines the risks per police-reported crash. See Ref 4 for comments on the companion report.]
Please comment on any apparent, unstated,
or implicit impact on estimated risks inherent
in the two different approaches, and any
related caveats or limitations. For example,
what are the strengths and weaknesses of the
two measures of vehicle exposure, miles of
vehicle traveled scaled up from crash data
from 13 states, and number of police-reported
crashes?
[Chen and Kockelman] The use of non-culpable vehicles in two-vehicle crashes as a proxy for vehicles which
are "just there" may be distorting the overall distribution of vehicle models. VMT may differ between vehicles
that are more prone to run-off-road accidents, at-fault two-car crash vehicles, and non-culpable vehicles.
[Van Auken] [1] The number of fatal cases tends to be much less than the number of induced-exposure cases.
Therefore the effective numbers of degrees-of-freedom in the statistical estimates tend to be limited by the
available number of fatal cases. For example, it would not be possible to estimate the effects of two variables
(e.g., just curb weight and footprint) if we had data for only one fatal case even if we had thousands of
induced-exposure cases. Therefore it is desirable to use data for the entire US in order to get a large sample of
fatal cases for the logistic regressions. This then requires the available induced-exposure data (i.e., from 13
17
-------
states) to be "scaled up" the US level using the method described in Kahane (2003 and 2011). The result is the
best currently available estimate of vehicle exposure.
[2] There may be some concerns about the accuracy of the vehicle miles-traveled data because the difficulty
estimating the number of vehicle miles travelled at the make-model-year=state level of detail. [Note: This
peer review does not address the Wenzel (2011b) companion report, which examines the risks per police-
reported crash. See Ref 4 for comments on the companion report.]
ADDITIONAL COMMENTS:
18
-------
5. RECOMMENDATIONS
COMMENTS
Please comment on whether the LBNL
assessment adequately addresses the NHTSA
2011 study and identifies the safety impact
from mass reduction. Are the analytic
methods and data used to assess the NHTSA
study, and estimate the relationship between
risk, mass, and footprint, appropriate? Is
casualty risk per crash a legitimate measure of
vehicle safety? What other methods or data
could be used to better predict the effect of
future vehicle designs on safety?
[Chen and Kockelman] While driver fatalities per crash seems a useful measure of vehicle design safety, and
examination of fatal crash rates is very valuable (using Polk- based exposure estimates), there are many
caveats to work of this type. As noted above: a primary concern remains a neglect of the notion that the type
of car is very much a proxy for driver type, and a vehicle's crash avoidance may have very little to do with
vehicle type. It has a lot to do with the person behind the wheel. Simply including gender and age variables
cannot account for important covariates such as education, risk aversion, driving ability, wealth, etc. In reality,
small cars may be less crash prone than Kahane's and Wenzel's results suggest, because they are driven by
lower-income, younger, less risk averse people driving in more crash prone settings (e.g., commercial strips
rather than pricey residential suburbs). Of course, as noted above, it is very difficult to control for all these
variables, and the contractor was asked to rely on the original data. In reality, the best the report authors can
do with such data sets is to explain how all the other, relevant attributes may factor in (e.g., quality of driver
and typical driving settings), and how they can generate biased estimation (sometimes in either direction).
Discussion of relevant literature that looks more deeply at crash outcomes (e.g., Wang or Chen's papers,
mentioned above, allowing for heteroscedasticity and individual vehicle attributes, non-driver outcomes, etc.)
will also be useful.
[Van Auken] [1] The basic methodology described by Kahane (2011) seems appropriate; however some
results using this method and data are not well understood and need further diagnosis.
[2] The induced-exposure data set provided by NHTSA is based on the non-culpable vehicles in two-vehicle
crashes. See the Table 1 comments on the limitations of this data. [Note: This peer review does not address
the Wenzel (2011b) companion report, which examines the risks per police-reported crash.]
Please comment on the overall adequacy of
LBNL's assessment of the 2011 NHTSA report
and its independent study of casualty risk for
predicting the effect of vehicle mass or
footprint reduction on safety. Provide any
recommended improvements that might
reasonably be adopted by the author to
improve the analysis.
[Chen and Kockelman] Overall, the study is a comprehensive assessment of the 2011 NHTSA report and
introduces interesting additional analyses to examine the relationship of vehicle mass and footprint reduction
on safety. However, as stated previously in the comments here, driver preference for specific car types
(including size and mass) is related to driver socioeconomic characteristics and driving behavior. As vehicle,
driver, and roadway environment characteristics all contribute to fatality risk, the effects of physical vehicle
changes such as mass or footprint reduction on safety should not be overstated when the other two types of
characteristics are not sufficiently accounted for.
[Farmer] Overall these are reasonably good studies. The Phase I report does a very good job of assessing the
NHTSA report of fatality risk.
19
-------
[Van Auken] The Wenzel (2011a) report provides a valuable supplement to the analysis and results in the
Kahane (2011) report. [Note: This peer review does not address the Wenzel (2011b) companion report, which
examines the risks per police-reported crash.]
ADDITIONAL COMMENTS:
[Van Auken] See attached tables 6 and 7 below.
20
-------
Table 6. Additional General Comments and Recommendations
Mike Van Auken
Section
COMMENTS AND RECOMMENDATIONS
Use of R2 is confusing. Suggest using lower case "r" when referring to the sample correlation coefficient (Box, Hunter, Hunter, 1978,
P. 61); or upper case R when referring to the regression coefficient of multiple determination (Draper and Smith, 1981, p. 90).
All
In most cases the reported results are just estimates, but are not described as such. For example, "The effect of mass reduction on
heavier cars and CUVs and minivans are not statistically significant" on p. iii should say "The estimated effect of mass reduction on
heavier cars and CUVs and minivans are not statistically significant."
This distinction is important when comparing results based on different models and assumptions because the different models and
assumptions do not change the effect itself, but rather the estimate of the effect. For example, the statement "The first sensitivity,
in dark purple, includes the weight variables in the regression model but excludes the footprint variable; this model tests the effect
of mass reduction while allowing footprint to vary with vehicle mass. This sensitivity increases the risk from a 100-lb mass reduction
in cars (from 1.43% to 2.64% for lighter cars, and from 0.48% to 1.94% for heavier cars) and CUVs/minivans (from a 0.47% decrease
in risk to a 0.52% increase in risk); however, there is no change in fatality risk in light-duty trucks" on page 15 is misleading. It would
be better to state that "The first sensitivity, in dark purple, includes the weight variables in the regression model but excludes the
footprint variable; this model tests the estimated effect of mass reduction while allowing footprint to vary with vehicle mass. Th4&
GGnsitivity/?emoi//'ng the footprint variable from the regression model increases the estimated risk from a 100-lb mass reduction in
cars (from 1.43% to 2.64% for lighter cars, and from 0.48% to 1.94% for heavier cars) and CUVs/minivans (from a 0.47% decrease in
risk to a 0.52% increase in risk); however, there is nothe change in the estimated fatality risk in light-duty trucks is very small and
not statistically significant."
This also applies to table and figure captions. For example, "Table ES.l. Effect of mass and footprint reduction on fatality risk, under
alternative regression model specifications" should say "Table ES.l. Estimated effects of mass and footprint reduction on fatality
risk, under alternative regression model specifications." "Figure 3.3 Effect of reduction in mass or footprint on US fatality risk per
VMT, by vehicle type: mass only, footprint only, and both" should say "Figure 3.3 Estimated effects of reduction in mass or footprint
on US fatality risk per VMT, by vehicle type: mass only, footprint only, and both."
Overall the word "effect" appears over 200 times in this report with the "estimated" or other qualifier. In some cases this may be
appropriate and in other cases it is not appropriate. It is recommended that the author review each instance and revise as
appropriate.
21
-------
Figures 4.1 through 4.17 do not control for the effect of vehicle size (e.g. footprint), which has been shown to be correlated with
vehicle weight (e.g., Figure 3.1), and therefore these figures may be misleading. It is strongly suggested that the horizontal axis label
be changed to "Curb weight (Ibs) and corresponding changes in size," and/or a note such as the following be added to each figure:
"Note these results do not control for the effect of vehicle size on fatality risk. Therefore the horizontal axis represents changes to
both vehicle weight and vehicle size."
The statistical significance of the linear trends in Figures 4.1 through 4.17 are not reported. It would be helpful if the confidence
intervals or statistical significance of the linear trends were reported, either in addition to or instead of R2.
The confidence intervals for the estimated slopes should be added to the results in Tables 4.1 and 4.2.
22
-------
Table 7. Additional Specific Comments and Recommendations
Mike Van Auken
Section
Executive
Summary, 7
Executive
Summary
Executive
Summary, 4
Executive
Summary, 7
4
4
5.1
5.2
5.2
5.2
5.3
6.4
Page
iii, 65
iv
iv, v, 22, 66
viii, 69
31-32
35
37
39
39
39
44
63
COMMENTS AND RECOMMENDATIONS
2nd paragraph refers to "our analysis," however the results are the same as the NHTSA analysis. The author
should clarify who or what "our analysis" refers to and how it relates to the NHTSA analysis. Perhaps the
statement "LBNL was able to reproduce the NHTSA analysis, which finds that..." would be more appropriate.
Last bullet - suggest changing the statement that "Logistic regression does not allow a statistic" to "Logistic
regression methods do not have a statistic."
Suggest changing "variance in risk" to variation in risk" throughout.
The numerical results for the NHTSA preferred model in Tables ES.l and 7.1 are slightly different than the results
reported in the NHTSA report. For example 1.43%/0.48%/0.52%/-0.40%/-0.47% should be 1.44%/0.47%/0.52%/-
0.39%/-0.46%
Figures 4.12 through 4.14 have the results for small and heavy-duty pickups combined, which is inconsistent with
the results in Table 4.1
The R2 values in Table 4.1 are different than the values in Figures 4.6, 4.8, 4.9, 4.11.
The subsection title should be "Alternative measures of exposure and outcome" because fatal crashes and
fatalities are measures of the crash outcome, not exposure.
It would be helpful to list the 18 manufacturer dummy variables in a table.
It is unclear why Lexus, Acura, and Infinity are treated as separate manufacturers, but Cadillac and Lincoln are
not.
It is unclear why AM General is considered a Chrysler brand. The AM General Hummer was sold by GM
beginning with the 2001 model year.
It would be helpful if the figures include error bars or shading to indicate the confidence intervals.
Table 6.3 - Suggest adding a note that the 72,316 total includes fatalities that are counted more than once in
crashes involving more than one vehicle type.
23
-------
24
-------
An Analysis of the Relationship between Casualty Risk Per Crash and Vehicle Mass and Footprint
for Model Year 2000-2007 Light-Duty Vehicles
(LBNL Phase 2 Report)
1. ASSUMPTIONS
COMMENTS
Please comment on the validity of any
assumptions embedded in the LBNL
assessment analysis and the independent
casualty analysis that could affect the
projected relationship between vehicle
mass/footprint reductions and
fatality/casualty risk. Examples might include
assumptions regarding whether recent
historical relationships between vehicle
weight, size, and safety will continue into the
future; potential future improvements in
vehicle technology and design may result in
compensatory safety benefits; and the annual
baseline fatality distribution.
[Chen and Kockelman] [1] The Phase 2 report serves as a complimentary document to the Phase 1 report by
isolating the effect of vehicle mass and footprint on crashworthiness. Whereas the Phase 1 report analyzes
fatality risk per estimated VMT, the Phase 2 report analyzes casualty risk per crash. The parallel structure of
the two reports makes it easy for the reader to compare the results of the two analyses.
[2] The binary logistic model employed in this analysis can only account for two injury outcome categories;
here it is used to distinguish crashes resulting in serious injury or death from all other crash outcomes. Thus,
the model does not account for the ordinal nature of injury severity and neglects the difference between a
serious injury and a death.
[3] The report states that "a serious incapacitating injury can be just as traumatic to the victim and her family,
and costly from an economic perspective, as a fatality." While serious injuries are very costly to society (and
may have similar economic cost implications as deadly crashes), willingness to-pay estimates (which include
pain and suffering) price the cost of a fatality at almost 20 times the cost of an incapacitating injury (NSC
2010). Thus, it is difficult to assess the economic cost of the estimates of increases in casualty risk per crash
without distinguishing whether that outcome is a serious injury or a death. This limitation of the model should
be addressed in the report.
[4] The logistic model also assumes error-term homoscedasticity and cannot account for increases and
decreases in the variation of injury outcomes due to vehicle and driver type, for example. Such limitations of
the model should be discussed.
[Farmer] The report concludes "that much of the detrimental effect of mass or footprint reduction on risk can
be attributed to the tendency for mass or footprint reduction to increase crash frequency, rather than to
reduce vehicle crashworthiness (risk once a crash has occurred)." However, the interpretation of casualties
per crash as inversely proportional to crashworthiness ignores the possibility that injury severity also depends
upon the circumstances of the crash. Casualties per crash must be divided into casualties per severe crash and
severe crashes per crash, where a severe crash would be one involving more energy, e.g., high-speed or
25
-------
rollover. It could be that weight reduction increases casualties per severe crash (i.e., reduces
crashworthiness), but reduces the likelihood that a crash is severe.
[Van Auken ] The basic assumptions, methodology, and data are primarily the same as in the Kahane (2011)
report, but have been extended to include serious injuries as well as fatalities, and also address crash
involvement (i.e., fatalities and serious injuries per accident, and also accidents per VMT).. These include the
following:
1) The probability of a crash fatality or serious injury is proportional to the number of accidents (provided the
crash conditions remain the same); and the probability of an accident is proportional to the vehicle miles
travelled (VMT).
2) The logarithm of probabilities of fatality or serious injury per accident, and accidents per VMT for a given
curb weight, footprint, and control variable values varies as a linear combination of the curb weight,
footprint, and control variables within the domain of the data.
3) The logistic regression methods determine a maximum likelihood estimate of model coefficients.
4) It is assumed that the above relationships remain constant in the recent past (i.e., 2000-2007 model year
vehicles in the 2002-2008 calendar years), present, and near future (i.e., 2017-2025 model year vehicles).
The first assumption that crash fatalities and serious injuries are proportional to the number of accidents
provided the crash conditions remain the same seems self-evident (e.g., if two fatal crashes had exactly the
same conditions, then the expected number of fatalities for the two crashes would be twice the value for just
one of the crashes). The assumption that the number of accidents are proportional to VMT rather than the
number of vehicle registration years (VRY) is also appropriate because accidents cannot occur if the vehicles
are not driven on the road (i.e., VMT = 0). This assumption is qualified however because VMT is more difficult
to measure than VRY and therefore may be less accurate. On the other hand the probability of a fatal crash or
the number of fatalities in a crash may also depend on the vehicle occupancy.
The second and third assumptions are appropriate provided that it is recognized that it is essentially
impossible with currently available knowledge and information to model all of the factors that could affect the
probability of fatality in a crash, and that the objective of the analysis is to identify overall trends versus
vehicle weight and footprint. In general the probability of fatality depends on other many other factors which
have not been modeled (e.g., driver behavior factors, vehicle design factors, roadway design factors, EMT
factors), and these unmodeled factors are assumed to be uncorrelated with vehicle weight and footprint,
and/or are represented by the other control variables. The latter assumption might or might not be valid.
26
-------
Please comment on any apparent unstated or
implicit assumptions and related caveats or
limitations.
The fourth assumption is perhaps the weakest because it assumes that future vehicles will have the same
design characteristics as past vehicles, and that the characteristics of the vehicle population (e.g., collision
partner weight, size, type) will also remain the same.
[Chen and Kockelman] The role of driver behavior is briefly addressed in the report but not emphasized
sufficiently. Casualty risk is a combination of driver, vehicle, and roadway characteristics. Whereas vehicle
characteristics significantly influence crashworthiness, driver behavioral differences play a significant, if not
primary, role in determining crash frequency. Socioeconomic data such driver household income, size, and
education influence driver attitudes and driving environments. For example, Chen et al. (2010) found that
crash risk increases for those living in socioeconomically disadvantaged areas (including households more likely
to drive less expensive and older vehicles). Though such data is not typically available in state and national
crash databases, the importance of these driver and environmental characteristics on crash rates (per mile
driven) and casualty risk should be stressed in both reports. It is clearly very difficult to control for, but is a
major caveat to the NHTSA (& now LBNL) results. We expect that crash severity could be probably be lower for
many of the small cars and pickups if they were driven by those who tend to drive more expensive vehicles,
under the same settings (e.g., daytime, urban freeway). Thus, statements like "a 100-lb reduction in the mass
of lighter cars leads to a 1.84% increase in crash frequency" should be accompanied by an explanation of the
possibility of the mass variable accounting/proxying for effects of lower income households owning smaller
vehicles.
[Van Auken] [1] The induced-exposure data set provided by NHTSA is based on the "non-culpable" vehicle in
two-vehicle crashes. It is assumed that the dataset is a reprehensive sample of the driver and environmental
exposure factors for vehicle use. However, since these cases include moving vehicles, some vehicle-driver-
environmental conditions may be under or over represented in this data depending on how they affect the
ability of a non-culpable vehicle to avoid a crash. Results in Ref (17) indicated that the estimated effect of
weight and size reduction are sensitive to whether the induced-exposure data are based on the Kahane (2003)
non-culpable vehicle definition of the Kahane (1997) stopped vehicle definition. Unfortunately it is not
currently possible to test this sensitivity with the NHTSA-provided induced-exposure data.
[2] It is also assumed that the accident data from the 13 or 16 states are representative of all US states. Figure
2.1 in Wenzel (2011b) provides a useful comparison of the distribution of fatalities in the US and 13 states by
the nine different crash types.
ADDITIONAL COMMENTS:
27
-------
[Chen and Kockelman] National Safety Council (2010) Estimating the Costs of Unintentional Injuries. Available online at
http://www.nsc.org/news_resources/injury_and_death_statistics/Pages/
EstimatingtheCostsof Unintentional Injuries
Chen, H.Y., Ivers, R.Q., Mariniuk, A.L.C., Boufous, S., Senserrick, T., Woodward, M., Stevenson, M. and Norton R. Socioeconomic status and risk of car crash
injury, independent of place of residence and driving exposure: Results from the DRIVE study. Journal of Epidemiology and Community Health 64(10), 2010, pp.
998-1003.
[Farmer] NHTSA's fatality analysis covered calendar years 2001-08, but the casualty analysis excludes 2008. Such exclusion is understandable given that 2008
data were at the time unavailable for a majority of the states (I think they are available now). However, 2008 was an unusual year and may have affected the
size and weight effect estimates. The footnote on p. 6 of the Phase II report states that an analysis including the available 2008 data will be summarized in
Appendix A. I don't see Appendix A. Is an analysis planned including 2008 data?
28
-------
2. CONTROL AND DEPENDENT
VARIABLES USED IN THE REGRESSION
MODELS
COMMENTS
Please comment on the adequacy of control
and dependent variables used in the
assessment analysis and independent casualty
analysis, and recommend any alternative
control or dependent variables that are
available for possible inclusion in the analysis.
For example, what are the relative merits of
the main dependent variables used, fatality
risk per estimated VMT, and casualty risk per
police-reported crash?
[Chen and Kockelman] [1] As alluded to above, a primary concern is that the NHTSA analysis (& thus the
LBNL analyses) largely neglect the idea that vehicle type (make & model) is very much a proxy for driver type,
and a vehicle's crash frequency may have very little to do with physical vehicle characteristics. It has a lot to do
with the person behind the wheel, and gender and age simply aren't enough to control for such distinctions.
Education, risk aversion, ability, wealth, etc., are important covariates. But existing data sets are quite limiting
(though the MVOSS & FAR with 3-year driver violation history do offer some valuable insights, not discussed in
these reports). In reality, small cars may be less crash prone than Kahane's & Wenzel's results suggest, because
they are driven by lower-income, younger, less risk averse people driving in more crash prone settings (e.g.,
commercial strips rather than pricey residential suburbs). Such key caveats need thoughtful discussion. Four
relevant papers on the topics of crash frequency and vehicle size-and-weight implications (by Knipling, Kweon
& Kockelman, Wang and Kockelman, and Chen & Kockelman) have been sent to Tom Wenzel. These all include
useful literature reviews for further connections to useful findings for citation in the reports, as time allows the
contractor.
[2] Independent variables such as vehicle mass and footprint may be accounting for effects of driver
socioeconomic factors as discussed in the Assumptions section. Furthermore, vehicle option variables such as
AWD and side curtain airbags may be reflecting the effects of driver environment (e.g., those living in areas
with icy winters opting for AWD) and attitude (e.g., more risk-averse drivers opting for side curtain airbags)
rather than the vehicle technology themselves. While extremely heavy and extremely large vehicles may have
significantly different handling and braking characteristics which influence crash frequency and casualty risk, it
is unlikely that given the same driver in the same environment, a small change in vehicle mass or footprint
would influence the driver's crash proneness.
[Farmer] One needs to restrict control variables to those that are available and reliable. A problem when
combining state databases is that the states often are not consistent as to the variables coded and the
definitions of those variables. This severely limits the list of possible control variables.
[Van Auken] [1] Reducing the total number of fatalities and serious injuries is desirable from a societal
viewpoint. This includes both subject vehicle occupant and collision partner (e.g., other vehicle occupant,
pedestrian) fatalities and serious injuries.
29
-------
[2] VMT is a good measure of accident exposure provided that it can be accurately determined.
[3] The number of fatalities and serious injuries per accident is a measure of vehicle crashworthiness (i.e.,
effect of a crash on the subject vehicle occupants) and crash compatibility (i.e., effect of a crash on the other
vehicle occupants or vulnerable road users). Subject vehicle occupant fatalities and serious injuries per
accident are a measure of the subject vehicle crashworthiness. Collision partner fatalities and serious injuries
per accident are a measure of vehicle crash compatibility.
[4] The number of accidents per VMT is a measure of the crash avoidance capabilities of a given vehicle.
What additional control variables, such as
vehicle make or model, might be included in
the regression models?
[Chen and Kockelman] [1] Vehicle height, a variable which may be more valuable than vehicle type for
similarly structured vehicles such as sedans, wagons, CUVs, and minivans, would be a valuable control variable.
In addition to a wider track, a lower center of gravity also increases vehicle stability, thereby reducing the risk
of rollover. Relevant literature & findings exist, and should be cited.
[2] Other variables which have been found in past studies to influence fatality risk such as seat belt use,
roadway geometry and division type are not included in this study.
[3] To account for driver characteristics that contribute to casualty risk, socioeconomic variables such as
household income, education, household size, etc. would be valuable additions. Unfortunately, both state and
national crash databases typically do not include such information (outside of MVOSS). Such issues should be
flagged for readers.
[Farmer] I think that already there are too many control variables in the regression models. Instead I would
consider defining different classifications of crash types. Table 2.2 of the Phase II report shows that the
distribution of crash types for casualty crashes is very different from that for fatal crashes.
Please comment on any caveats or limitations
that these dependent variable or control
variables entail with respect to use of the
results as the basis for estimating the safety
effect of mass reduction.
[Chen and Kockelman] Please see above comment (in Assumptions section) regarding driver behavior and
environment.
[Farmer] Model overspecification could be the reason for results that are non-intuitive, especially in the Phase
II analyses of police-reported crashes. Control variables may be correlated with each other or with the size
and weight variables. For example, Figure 2.9 of the Phase I report implies that torso side airbags increase
fatality risk in CUVs.
30
-------
ADDITIONAL COMMENTS:
[Chen and Kockelman] Table 2.1 has many indicator variables labeled as "C" for continuous variable (such as ABS, ESC, AWD, DRVMALE, etc). These C's should
be removed.
[Farmer] The sensitivity results of Chapter 5 (Phase II) point out the extreme differences in results when changing the control variables. For example, including
vehicle make changes the effect of a 100-lb reduction in heavier cars from -0.91% to +0.55% (see Fig 5.3).
[Van Auken] The underlying reasons for some of the estimated effects are unknown at this time, but presumably involve driver, vehicle, environment or
accident factors than have not been controlled for in the Kahane (2011) and Wenzel (2011b) analyses. See, for example, Refs 17 and 23.
31
-------
3. METHODOLOGY AND STATISTICS
COMMENTS
Please comment on the validity and
applicability of the methodology LBNL used in
assessing the NHTSA 2011 study and its
analysis of the relationship between mass,
footprint, and risks per police-reported crash.
[Chen and Kockelman] The Phase 2 report enhances the findings in the Phase 1 report by isolating the effect
of vehicle mass and footprint on crashworthiness. Like the Phase 1 report, this analysis goes into a fair amount
of detail and seeks to introduce additional analyses to better examine the relationship between mass,
footprint, and casualty risks. However, due to a lack of control for very specific vehicle differences (which vary
by make & sub-model), the exclusion of driver characteristics and crash setting details (which cannot always be
controlled for, but are often correlated with vehicle type), the effects of downweighting vehicles and/or
shifting vehicle styles and sizes may be overestimated. Simply changing the vehicle mass or footprint on a risky
driver in a high-risk setting is unlikely to influence crash outcomes significantly.
[Farmer] Figure 2.11 of the Phase II report implies that NHTSA's fitting of a separate regression model for each
of the 9 crash types was unnecessary, at least for the analysis of casualty risk per crash. I don't recall seeing a
similar analysis for fatality risk per VMT. Is it possible to get essentially the same results as the NHTSA study
using a single regression model?
[Van Auken] [1] The logistic regression methods seem to be appropriate. The confidence intervals are based
on the logistic regression Wald Chi-Square statistic, which as Kahane (2003, 2011) has demonstrated does not
include all sources of variation. However, these confidence intervals are useful because they do provide some
indication of the uncertainty in the results.
[2] The two-stage results for the 13-state fatalities per crash and 13-state crashes per VMT, and the one-stage
result for 13-state fatalities per VMT reported in Tables ES.l and 6.1 and Figure 2.7 were computed using
independent logistic regressions. The differences between the two-stage results and the one-stage results for
fatalities per crash could have been eliminated by using the "simultaneous three-way" logistic regression
method described in DRI (2003). This method imposes the constraint that the combined two-stage estimated
and the one-stage estimated are equal.
Please review other statistical methods LBNL
has used in the analysis, in addition to the
logistic regression methodology. Examples
include the alternative approaches used by
LBNL to assess NHTSA interval estimation
results, and LBNL's linear regression analysis
of actual, predicted, and residual risk by
[Chen and Kockelman] In the alternative measures of exposure, the author examines the effect of vehicle
manufacturer on fatality risk and treats the luxury models produced by Toyota, Honda, and Nissan as separate
manufacturers. However, domestic luxury brands (such as Cadillac & Lincoln) are categorized with their
nameplate manufacturers (GM and Ford), which appears inconsistent.
[Van Auken] [1] The correlations in Section 3 appear to be assessed using the Coefficient of Multiple
Determination (R2) based on a linear fit to the data (e.g., the correlation between footprint versus curb weight
32
-------
vehicle model.
in Figure 3.1 on p. 32). The linear regression model attributes the differences between the dependent variable
(vertical axis) and the linear fit to the independent variable (horizontal axis) to random effects. If there is no
preference as to the choice of independent and dependent variables (e.g., footprint versus curb weight, or
curb weight versus footprint), then the linear trend and R2 result would be different if the two variables were
interchanged, and having two different yet equally valid results would be undesirable.
[2] If the variation in the data can be attributed to both variables (e.g., footprint and curb weight), then it
would be better to report the square of the sample correlation coefficient r2, where r is computed according to
Eqn (1). The trend lines in these correlation figures should not be computed using a linear regression. Instead,
the trend line should pass through the sample means (i.e., (x, y)), and have a slope equal to the ratio of the
sample standard deviations in the data (i.e., sy/sx). Therefore, the reported correlation results do not depend
on the ordering of the data variables.
Note this comments does not apply to linear trends indicated in Section 4, for which the Coefficient of Multiple
Determination (R2) seems appropriate.
[3] The Coefficient of Multiple Determination (R2) is frequently used in the Wenzel (2011b) report as an
indicator of the statistical importance of a linear trend (e.g., lvalues in Tables 4.1 and 4.2 were compared to
0.3). It would be better to report the standard error, confidence interval, and/or probability value as measures
of the statistical significance of a linear trend.
Please comment on caveats or limitations of
using non-significant regression estimates to
project the safety impact of mass reduction.
[Chen and Kockelman] First, the t-statistics are not provided in the report which makes it difficult for the
reader to assess statistical significance of specific regression estimates (except where noted by the author).
Second, inclusion of a statistically insignificant variable can influence the estimates of coefficients associated
with related variables. Nevertheless, in general, it is best to keep insignificant estimates if one has a strong
defense for their role, since removing such variables (& thus their parameters) will shift the burden of
response to a correlated covariate's parameter, thus biasing the latter. We generally keep key covariates in a
model up to a pvalue of 0.20 or 0.25 or so, especially in relatively small data sets (e.g., n < 1,000). Covariates
for which we have no strong basis can be removed for pvalues > 0.10.
[Farmer] Making projections from non-significant regression estimates is proper so long as the resulting
confidence intervals are constructed conservatively (to account for the accumulated imprecision). In that
sense, I prefer NHTSA's jackknife approach to the standard errors produced by SAS (see p. 13 of Phase II).
33
-------
[Van Auken] [1] Regression estimates are random numbers which have an unknown expected value and
variance, and known sample value and standard error. If the sample value can be explained by a zero
expected value and known standard error then the result is considered not statistically significantly different
than zero and therefore the result is not considered to be statistically significant. However, If we can combine
this estimate with other estimates then the unknown expected values and variances can also be combined
using the same transformation, and the statistical significance of the combined result can be tested.
Therefore, depending on the sample values and inter-correlation, the combined result may be statistically
significant even if the individual estimates are not statistically significant.
For example, the results from each of the nine different crash types can be combined into an overall estimate
and the standard error calculated assuming that the results for each crash type are independent of each other.
Then the statistical significance of the combined effect can be determined.
[2] However, and Kahane (2011) points out there are two sources of uncertainty in the regression results. The
first is the PARS based sampling error which is uncorrelated across crash types because they are based on
different fatal cases (Kahane 2011, p. 77). The second is the state based induced-exposure sampling error
which is correlated across crash types because they are based on the same induced-exposure cases. Therefore
a confidence interval estimated using the "jackknife" method described by Kahane (2011) and accounting for
correlation of these two error sources would be more accurate than a simple estimate based on the Wald Chi-
Square statistic and assumed independence.
How might the LBNL methodology be
strengthened to better represent future
vehicle designs and reduce multi-collinearity
between mass and footprint in the regression
analysis?
[Chen and Kockelman] Including more vehicle-specific characteristics (such as vehicle height and engine size)
reduces the analysis' dependence on vehicle type, since vehicle shapes and structures will continue to evolve.
There is also correlation with context (e.g., pickups are driven in more rural locations, with greater hazards
[like less lighting, higher speed, & few medians]). Disaggregate data are almost always best, to avoid ecological
fallacies & such.
[Van Auken] [1] The effects of multi-collinearity can be mitigated by 1) obtaining more data, 2) pooling data
from different crash type or vehicle types, or 3) reducing the number of regression variables. The first option
would require more states (for serious injuries and police-reported accidents), calendar years and/or model
years, which would involve added newer data as it becomes available (or using older data). The second option
might be to recombine the CUVs and minivans with truck based vans and adding a control variable to
compensate for the differences in the vehicles types. The third option might involve removing statistically
insignificant control variables or removing control variables that would not be expected to have an effect on
34
-------
the probability of crash or crash outcome (e.g., the side airbag variable is not included in pedestrian crashes
because it is not expected to affect pedestrian fatality risk). The number of driver age control variables might
be reduced from eight to three (as in the Kahane (1997) and DRI (2002-2005) studies). Finally, a linear curb
weight model instead of a two-piece linear model may help to better elucidate the general trend.
[2] The Variance Inflation Factor (VIF) has been suggested as a measure of multi-collinearity in the Kahane
(2010 and 2011) reports, however this diagnostic metric does not account for differences in database size (i.e.,
Options 1 and 2 above). The Wenzel (2011b) report does not discuss the Variance Inflation Factor or report
any VIF results.
ADDITIONAL COMMENTS:
35
-------
4. DATA SETS
COMMENTS
Please comment on the validity and
applicability of the datasets used to project
changes in risk resulting from reduction in
vehicle mass. LBNL's casualty analysis used
police-reported crash data from 16 states,
while the 2011 NHTSA study used national
fatality data, combined with a subset of non-
culpable vehicles involved in two-vehicle
crashes from police-reported crash data from
13 states.
[Chen and Kockelman] [1] The Phase 2 report uses an unusually extensive data set of police-reported crash
data from 13 states which the author compares in detail to national data sets to illustrate similarities and
differences. The author is very thorough in addressing the difference in definitions of "serious" and
"incapacitating" injuries across different states and the effects of such inconsistency on the regression results.
[2] Since casualty risk in the report accounts for serious injuries but not minor injuries, the author should note
that police-reported injury levels may also be poor indicators of the actual or Modified Abbreviated Injury
Scale (MAIS) level, following medical evaluation. Farmer (2003) found that 41% of injuries reported by U.S.
police as incapacitating received MAIS ratings of "minor injury" by health care professionals using NASS
Crashworthiness Data System (CDS). Thus, the results of the estimated casualty risk increases and decreases
rely heavily on the assumption that police errors in reporting actual MAIS ratings are consistent across states.
[Farmer] A major limitation of the Phase II analysis is a bias that may be due to the patterns of missing data.
In particular, the vehicle identification number (VIN) is missing or mistyped for many crash records. High-
severity crashes (especially fatal) are more likely to have detailed police investigation, so VINs (and other
variables) in these crashes may be more complete. State crash files are therefore much less reliable than
PARS.
[Van Auken] [1] The induced-exposure data set provided by NHTSA is based on the non-culpable vehicles in
two-vehicle crashes. See the comments in Table 1 on the limitations of this data.
[2] The use of property damage accident data and cases with serious injury from the 13 states seems
appropriate (with the noted qualification that the different states may have different accident reporting
thresholds and injury reporting criteria). The concerns about the use of data for the 3 additional states
(Georgia, Illinois, and New Mexico) have also been noted.
[3] In addition, there are also many differences in the coding variables and values used by the different states,
which tend to make the receding to a common data set (either induced-exposure, police-reported accident, or
severe injury) imprecise.
36
-------
Please comment on any apparent, unstated,
or implicit impact on estimated risks inherent
in the two different approaches, and any
related caveats or limitations. For example,
what are the strengths and weaknesses of the
two measures of vehicle exposure, miles of
vehicle traveled scaled up from crash data
from 13 states, and number of police-reported
crashes?
[Chen and Kockelman] The Phase 1 analysis used non-culpable vehicles in two-vehicle crashes as a proxy for
induced exposure crashes. In contrast, Phase 2 analysis uses data from vehicles involved in one-car crashes and
the responsible vehicle in two-car crashes. The exclusion of the not-at-fault vehicle in two-car crashes may be
distorting the distribution of crash frequency and casualty risk across different vehicle makes and models if
crash-prone drivers are more likely to drive certain types of vehicles.
[Farmer] The VMT weights provided by NHTSA were scaled to represent the entire US. Comments on pp. 9
and 18 of the Phase II report seem to acknowledge this deficiency, promising to adjust these to the 13 states in
the future. Was any adjustment made, such as multiplying the weights by the proportion of annual US VMT
accounted for by each of these states? The accuracy of the VMT weights is critical is we are to believe the
somewhat surprising results concerning crashes per VMT.
[Van Auken] [1] The number of fatal or serious injury cases tends to be much less than the number of
induced-exposure cases (and the number of police-reported accidents). Therefore the effective numbers of
degrees-of-freedom in the statistical estimates tend to be limited by the available number of fatal or serious
injury cases. For example, it would not be possible to estimate the effects of two variables (e.g., just curb
weight and footprint) if we had data for only one fatal of serious injury case even if we had thousands of
induced-exposure cases. Therefore it is desirable to use data for the entire US in order to get a large sample of
fatal cases for the logistic regressions. This then requires the available induced-exposure data (i.e., from 13
states) to be "scaled up" the US level using the method described in Kahane (2003 and 2011). The result is the
best currently available estimate of vehicle exposure.
[2] There may be some concerns about the accuracy of the vehicle miles-travelled data because the difficulty
estimating the number of vehicle miles travelled at the make-model-year-state level of detail.
ADDITIONAL COMMENTS:
[Chen and Kockelman] Farmer, C.M. Reliability of police-reported information for determining crash and injury severity. Traffic Injury Prevention 4(1), 2003,
pp.38-44.
[Farmer] Statements above Figure 2.7 in the Phase II report imply that the effects of weight reduction on crashes per VMT and fatalities per crash should add
up to the effect on fatalities per VMT. This is not the case. For example, a 1.43% increase in crashes per VMT and a 0.76% decrease in fatalities per crash
would imply a 2.16% decrease in fatalities per VMT (i.e., 1 - 0.9924/1.0143). The fact that the model on fatalities per VMT yields an estimated 1.08% increase
should be a cause for concern. Either the VMT weights are inaccurate or the control variables have different effects on crash frequency and crashworthiness.
37
-------
38
-------
5. RECOMMENDATIONS
COMMENTS
Please comment on whether the LBNL
assessment adequately addresses the NHTSA
2011 study and identifies the safety impact
from mass reduction. Are the analytic
methods and data used to assess the NHTSA
study, and estimate the relationship between
risk, mass, and footprint, appropriate? Is
casualty risk per crash a legitimate measure of
vehicle safety? What other methods or data
could be used to better predict the effect of
future vehicle designs on safety?
[Chen and Kockelman] As noted above, a primary concern remains a neglect of the notion that the type of car
is very much a proxy for driver type, and a vehicle's crash avoidance may have very little to do with vehicle
type. It has a lot to do with the person behind the wheel. Simply including gender and age variables cannot
account for important covariates such as education, risk aversion, driving ability, wealth, etc. In reality, small
cars may be less crash prone than Kahane's and Wenzel's results suggest, because they are driven by
lower-income, younger, less risk averse people driving in more crash prone settings (e.g., commercial strips
rather than pricey residential suburbs). Alas, it is very difficult to control for all these variables, since they are
not readily available in data sets. In reality, the best the report authors can do with such data sets is to explain
how all the other, relevant attributes may factor in (e.g., quality of driver and typical driving settings), and how
they can generate biased estimation (sometimes in either direction). Discussion of relevant literature that
looks more deeply at crash outcomes (e.g., Wang or Chen's papers, mentioned above, allowing for
heteroscedasticity and individual vehicle attributes, non-driver outcomes, etc.) will also be useful.
[Farmer] Casualty risk per crash does not fully measure the effects of vehicle size and weight reductions on
society. Casualty risk per VMT best coincides with the NHTSA analysis of fatalities per VMT. The breakdown of
casualty risk per VMT into the crash frequency and crashworthiness components is of interest. However, the
surprising results reported here make everything suspect. For example, the Phase II report concludes that "the
detrimental effect of male drivers has to do with their higher tendency of getting into a serious crash rather
than their sensitivity to injury once a serious crash has occurred" (p. 24). A few pages later it concludes that
"male drivers have essentially no effect on crash frequency, but cause a statistically significant increase in
fatality risk once a crash occurs" (p. 28).
[Van Auken] [1] The basic methodology described by Kahane (2011) seems appropriate; and the extension by
Wenzel (2011b) are also appropriate. However some results using these methods and data are not well
understood and need further diagnosis.
[2] The induced-exposure data set provided by NHTSA is based on the non-culpable vehicles in two-vehicle
crashes. See the Table 1 comments on the limitations of this data.
[3] The state accident data files tend to have different database variable and coding definitions and criteria,
which could confound the results.
Please comment on the overall adequacy of
[Chen and Kockelman] Overall, the study is an enriching complementary document to the Phase 1 assessment
39
-------
LBNL's assessment of the 2011 NHTSA report
and its independent study of casualty risk for
predicting the effect of vehicle mass or
footprint reduction on safety. Provide any
recommended improvements that might
reasonably be adopted by the author to
improve the analysis.
of the 2011 NHTSA report. The parallel structure of the two reports allows the reader to easily compare and
contrast the various additional analyses which examine the relationship of vehicle mass and footprint
reduction on safety. However, as stated previously in the comments here, driver preference for specific car
types (including size and mass) is related to driver socioeconomic characteristics and driving behavior. As
vehicle, driver, and roadway environment characteristics all contribute to fatality risk, the effects of physical
vehicle changes such as mass or footprint reduction on safety should not be overstated when the other two
types of characteristics are not sufficiently accounted for.
[Farmer] Overall these are reasonably good studies. The Phase I report does a very good job of assessing the
NHTSA report of fatality risk. However, the Phase II report should be more cautious in its conclusions
concerning casualty risk. The casualty analysis is based solely on police-reported data from 13 states, which:
1. May not be representative of the US as a whole.
2. Are inconsistent in the information given and the way in which it is coded.
3. Suffer from information that is missing, inaccurate, or unclear.
[Van Auken] The Wenzel (2011b) report provides a valuable supplement to the analysis and results in the
Kahane (2011) report.
ADDITIONAL COMMENTS:
[Farmer] Column G of Table 6.1 in the Phase II report provides the most appropriate comparison to results from the NHTSA report (Column A). For both
fatalities and casualties per VMT, a 100-lb weight reduction is most harmful in lighter cars, less harmful in heavier cars and lighter light trucks, and slightly
beneficial in heavier light trucks, minivans, and crossovers.
[Van Auken] See attached tables 6 and 7 below.
40
-------
Table 6. Additional General Comments and Recommendations
Mike Van Auken
Section
4
All
All
4
COMMENTS AND RECOMMENDATIONS
Use of R2 is confusing. Suggest using lower case "r" when referring to the sample correlation coefficient (Box, Hunter, Hunter,
1978, P. 61); or upper case R when referring to the regression coefficient of multiple determination (Draper and Smith, 1981, p.
90).
In most cases the reported results are just estimates, but are not described as such. The word "effect" appears several hundred
times in this report with the "estimated" or other qualifier. In some cases this may be appropriate and in other cases it is not
appropriate. It is recommended that the author review each instance and revise as appropriate.
"Crashworthiness" in most instances should be changed to "crashworthiness and crash compatibility" because the fatalities
and/or serious injuries may either be in the subject vehicle (crashworthiness effect) or collision partner (crash compatibility
effect).
The statistical significance of the linear trends in Figures 4.1 through 4.9 are not reported. It would be helpful if the confidence
intervals or statistical significance of the linear trends were reported, either in addition to or instead of R2.
The confidence intervals for the estimated slopes should be added to the results in Tables 4.1 and 4.2.
41
-------
Table 7. Additional Specific Comments and Recommendations
Mike Van Auken
Section
Executive
Summary, 4
Executive
Summary, 6
5.3
5.3
5.3
Page
iv, v, 22, 66
vii, 63
49
48-49
49
COMMENTS AND RECOMMENDATIONS
Suggest changing "variance in risk" to "variation in risk" throughout.
The statement "In conclusion, casualty risk per crash is not necessarily a better metric than fatality risk per VMT
for evaluating the effect of mass or footprint reduction on risk; rather, it provides a different perspective in
assessing the benefits or drawbacks of mass and footprint reduction on safety in vehicles. However, it does
allow the separation of risk per VMT to be separated into its two components, crash frequency and risk per
crash" suggests that the casualty risk per crash metric was needed in order to assess the crash frequency and risk
per crash, which is incorrect. The DRI (2003-2012) methods have also estimated the effects of weight and size
on crash frequency (A/E) and risk per crash (F/E) in terms of fatalities.
It would be helpful to list the 18 manufacturer dummy variables in a table.
It is unclear why Lexus, Acura, and Infinity are treated as separate manufacturers, but Cadillac and Lincoln are
not.
It is unclear why AM General is considered a Chrysler brand. The AM General Hummer was sold by GM
beginning with the 2003 model year.
42
-------
Review of "An Analysis of the Relationship between Casualty Risk per Crash and Vehicle Mass and
Footprint for Model Year 2000-2007 Light-Duty Vehicles"
David L Greene
January 10, 2012
Summary
The Phase I and Phase II analyses by Tom Wenzel of LBNL have been executed diligently and consistently
in accord with the methods and data used in the original NHTSA analysis. The studies contain many
valuable, new insights. The phase I study highlights the weakening relationship between vehicle mass
and highway fatalities. This is not only seen in decreasing coefficient estimates but in the very large
number of results that are not statistically significant. When regressions were done separately by
footprint deciles, vehicle mass was statistically significantly positively related to fatalities only for light-
duty trucks in rollovers, there were almost as many cases in which mass was negatively related to
fatalities (9 vs. 13 out of 27) and there were more instances of statistically significant negative
relationships than positive relationships. Given that so many tests are being jointly conducted, it is quite
possible that when joint probabilities are considered, there is no significant relationship between mass
and fatalities (more on joint probabilities later). Showing the weakness and inconsistency of these
results is an important contribution.
Another meaningful contribution of the phase I study, and one that deserves more emphasis, is a logical
inference from the following findings: 1) much of the variance in risk remains unexplained even by the
most complete models, 2) control variables explain 1 to 2 orders of magnitude more of the variance
than the variables of interest (mass and size), 3) when key control variables are removed or changed it
strongly influences the coefficients of mass and size. These results have very important implications for
the robustness of the results and the likelihood that some or all of the apparently statistically significant
relationships are due to spurious correlations with omitted or imperfectly controlled factors. Noting
that exposure measures are control variables with constrained coefficients, the following observation
from the phase I study is especially perceptive.
"Calculating risk as total fatalities per induced exposure crash, rather than per vehicle mile
traveled, reverses the sign of mass reductions on risk in cars and the lighter light trucks, with mass
reduction leading to a reduction in risk in all vehicle types."
Finally, the phase I report notes that if only the control variables are included in the regression and
not size or mass, the resulting residuals from the regression are uncorrelated with size or mass.
Given these findings (as well as those of phase II) the conclusions that,
"The 2011 NHTSA study, and this report, conclude that the effect of mass reduction on US fatality
risk is small."
43
-------
should be revised with the following emendation, "... and probably non-existent."
Both studies, like the NHTSA analysis, have shortcomings in terms of interpreting the results and the
language used to describe the results, and acknowledging the limitations of the data and methodology.
The limitations are extensive. The interpretation of the results of the LBNL studies commits two
important, related errors. The first is to attribute inferred coefficients of mass and size as representing
only the effects of vehicle mass and size when, as the phase I and II study results indicate, there is a
virtual certainty that aliasing effects are present due to a combination of omitted variables, errors in
variables and correlations among variables. Given that estimated driver and environmental factors
tend to have 1-2 orders of magnitude larger impacts on safety outcomes than vehicle factors, the almost
certain presence of aliasing effects must be explicitly acknowledged as severely limiting the ability to
draw inferences about the effects of vehicle attributes. Second, the language used in interpreting
results fails to acknowledge that the analysis does not address the effects of down-weighting or down-
sizing specific vehicles or vehicle designs, but instead relies on correlations between vehicle weight and
size in existing vehicle designs. In existing vehicles, weight and size are correlated with each other and
many other vehicle attributes (and driver and environmental attributes, as well). Thus, the study is not
actually measuring the effects of down-weighting via the material substitution and design changes likely
to occur as a consequence of fuel economy and emissions standards. An early example of the kind of
misleading language referred to here can be found on page iv.
"For example, a 100-lb reduction in the mass of lighter cars leads to a 1.84% increase in crash
frequency (columns B), while mass reduction leads to a 0.76% decrease in the number of fatalities
per crash (column C);"
This statement is misleading in that it implies causality rather than correlation, and it is additionally
misleading in that it implies that the inference applies to removing weight from specific vehicles.
Neither is correct. A better statement would be the following.
"For example, vehicles in the lighter class that are 100 Ibs. lighter are correlated with a 1.84%
increase in crash frequency...."
There are so many examples of this misleading language that it is not feasible to list them all. All
should be corrected, however. Failure to correct them could lead to serious misinterpretation of the
studies' findings.
Following in the footsteps of the seminal study by DRI, the NHTSA and LBNL studies contribute to the
literature in three important ways: 1) the LBNL and NHTSA studies recognize that the societal safety
perspective is the correct perspective to when assessing the impacts of fuel economy and emissions
regulations, 2) they recognize that vehicle dimensions and vehicle mass may have separate and
potentially different impacts on both the likelihood of a crash and the outcomes of the crash and, 3) the
LBNL phase two analysis makes an additional contribution by attempting to disentangle factors affecting
the likelihood of a crash and factors affecting the outcomes of a crash.
44
-------
Speaking of the DRI study, I am puzzled about why there are no references cited in the phase I study and
only a handful all by Kahane and Wenzel, in the phase II study. This is perhaps due to the scope of work
defined for the two studies but there are highly relevant studies in the literature that could have been
cited, those by DRI foremost among them. Making use of the insights from these studies would have
been helpful in interpreting the results of both phase I and phase II.
Lack of a Theory or Model of the Phenomenon
Both the NHTSA and LBNL studies lack a rigorous theory of the process by which down-weighting at
constant size or down-sizing at constant mass affect societal safety either through crash avoidance or
crashworthiness. This is not a trivial shortcoming because it affects the ability to formulate hypotheses
and interpret results. Prior to the dissenting report on safety of the NRC 2002 CAFE report, the physics
of elastic collisions between objects was typically cited as the underlying physical model. That report
showed how taking the societal perspective renders that model inappropriate. What remains appears
to be far more complex, involving the quantity of kinetic energy, the ability of vehicle designs to absorb
that energy so as to minimize maximum deceleration rates, stability, maneuverability, safety
technologies, and more.
The consequences of the lack of a rigorous theory are that it is not known, a priori, what the signs of
coefficients are expected to be, let alone what their quantitative relationships should be. Hypotheses
must be formulated based on intuition and the interpretation of results is likewise ad hoc. One
implication of this is that results that suggest that lower fatalities are associated with lower vehicle mass
have equal standing, a priori, with results that indicate that higher fatalities are associated with lower
vehicle mass, and similarly for vehicle size. There are no surprising or unsurprising results, in theory.
This also makes it difficult to develop a plan for statistically testing the model or theory and its
implications. It would have been helpful to the reader to have been presented early on in the report
with such a plan of analysis.
On the Virtual Certainty of Aliasing
The LBNL report typically attributes causal effects to correlations between mass or size and safety. In
fact, most or all of the observed correlations are almost certainly affected by aliasing effects. There is
ample evidence for this inference in the results presented in the LBNL phase II report.
The coefficients of mass and size change in important ways when different model formulations are
estimated. Removing and adding control variables changes the magnitudes and sometimes the signs of
the mass and size variables. This means that, at a minimum, the mass and size variables alias the effects
of the omitted control variables. The question is whether the aliasing is eliminated entirely by the
inclusion of the control variables available or whether some aliasing remains either because not all
relevant and correlated control variables have been included or because the included control variables
are imperfect measures of the factors they are intended to represent.
45
-------
The latter seems highly likely for the following reasons. First, the overall explanatory power of the full
models (including control variables), as measured by their R2 is low. Most of the variance in casualties
and fatalities remains unexplained. Second, at least some of the important included control variables
are only crude approximations of the factors they are intended to represent. For example, dummy
variables represent differences in state reporting practices, age and gender represent risky driving
behavior differences among owners of different sizes and masses of vehicles, the presence or absence
of a kind of safety equipment represents both its performance and use in a particular vehicle, and
calendar year dummy variables represent unknown factors associated with the respective calendar
years. Such practices are common and their use is appropriate. Third, the control variables generally
account for 1-2 orders of magnitude more variance in the casualty and fatality variables than do vehicle
weight and size. To recap, the amount of unexplained variability in the dependent variables is larger
relative to the variance statistically explained by the most complete models. Control variables are
correlated with size and mass, and they account for 1-2 orders of magnitude more variability in the
dependent variables than the variables of interest, mass and size. Therefore, even small correlations of
size and mass with omitted variables or with errors (imprecision) of the control variables could easily
result in biased estimates for the effects of size and mass on the dependent variables.
Tom Wenzel is to be commended for providing the results that definitively demonstrate the three key
points made above. The above is not a criticism of the analysis nor of the results, per se. It is a criticism
of their interpretation. In light of the above, the results should be interpreted in light of the virtual
certainty that many of the estimated coefficients are likely to be biased in ways that make their
interpretation highly uncertain. The implication is that phrases such as "down-weighting or down-sizing
caused" to "mass (or size) and unobserved correlated factors are associated with..."
On Joint Probabilities
The NHTSA and LBNL studies do not correctly interpret their results as joint statistical tests. When
testing a hypothesis on, for example, 5 vehicle classes simultaneously, a result for one equation that
might be statistically significant on its own may not be statistically significant as one of five related tests.
Statistical analyses comprised of multiple regressions too often overlook the fact that tests of statistical
significance designed for individual regressions may not apply in the case of multiple regressions. That is
the case here. NHTSA conducted 5 analyses to infer relationships between mass differences among
vehicles holding footprint constant for 5 classes of cars. The results showed one relationship out of five
was statistically significant. As table 1 illustrates, using a simple example, if one conducts 5 trials, each
with a 0.05 probability of given result, there is a 22.6% probability of finding at least one such result in
the five trials. Thus, the joint significance level of the overall result (1 statistically significant regression
out of 5) is 0.226, rather than 0.05.
46
-------
Table 1. Simplified Illustration of the Joint Probability of Inferences in Multiple Regressions
# Significant
Regressions Combinations
0 1
1 5
2 10
3 10
4 5
5 1
0.95
0.773780938
0.81450625
0.857375
0.9025
0.95
1
0.05
1
0.05
0.0025
0.000125
0.00000625
3.125E-07
Joint
Probability
77.37809%
20.36266%
2.14344%
0.11281%
0.00297%
0.00003%
100.0%
22.6%
So there is between a 1:4 and a 1:5 chance of getting one statistically significant result by pure chance.
In fact, the actual significance level of the results is more complicated to calculate, and probably a bit
smaller than 0.226. Thus, it is very appropriate for Dr. Kahane to add the qualifier "if any" to his
conclusions about the relationship between the societal highway fatalities and mass reduction, holding
footprint constant. Had appropriate tests of joint statistical significance been used to evaluate the
results in the NHTSA and LBNL studies, the significance levels very likely would not meet accepted
criteria for statistical significance. This could change the conclusions of the studies from the inference
that mass is correlated with fatalities or casualties in some case but not others to the lack of statistically
significant evidence that mass is correlated with fatalities or injuries on the highway. This is an
important difference.
Page-by-Page Comments
I will make page by page comments on the phase II study only, since that contains the overwhelming
share of original contributions and the key findings of the phase I study are recapitulated there.
p. iii Paragraph 3. This would be a very good place to acknowledge the importance of driver behavior
and environment on crash avoidance especially.
p. iv Para. 3. This would be a good place to discuss probability inference in joint tests.
Para 4. The statement about a 100-lb reduction in the mass, etc., is a good example of
misleading language.
p. v Para. 1. Again, it is misleading to say that mass reduction increases crash frequency, for reasons
stated above.
Para. 2. It is more accurate to describe the association of lower vehicle mass with casualty risk
than the "effect of mass reduction on..." casualty risk.
Para. 3. Would benefit greatly from joint probability inferences.
47
-------
Para. 4. As noted above, this shows how much more important the control variables are than
the variables of interest.
Para 5. Again, these are correlations not necessarily effects.
p. vi Para. 1. Again, mass reduction is misleading terminology and you do not know if it increases
casualty risk or not, you know only a correlation. Why is this so important? It is the virtual
certainty of spurious correlations, or aliasing, as noted above.
Para. 4. (1st bullet) This is clear evidence of aliasing. Take variables out of the regression and the
coefficients of interest change in important ways. Are there no important factors still missing?
Are the variables included perfect measures of the factors of interest? Of course not. Thus,
there must be remaining aliasing. How bad is it? We don't know.
The third bullet shows the same effect with a different set of variables.
p. vii Para. 4. No, your analysis does not indicate "...that much of the detrimental effect of mass or
footprint reduction on risk can be attributed to the tendency for mass or footprint reduction to
increase crash frequency." Again, you have correlation, not causation and you have good
reason to believe that what you are seeing is affected by spurious correlations.
Para. 5. The "effect" is small, 1-2 orders of magnitude smaller than correlations with other
control variables, and IS strongly affected by which variables are in the equation, as stated on
the previous page, and there is a great deal of unexplained variance. Please reconsider the
meaning of these results in light of the comments above.
Finally, as the last paragraph of the ES implies, it would be far better not to speak in terms of
"reducing" mass or size. That is not what is happening in your data set.
p. 1 Para. 4. Risk per VMT includes the effects of how well vehicles are driven as well as how well
they can be driven. I think there is no chance that you have fully accounted for how well
vehicles are driven.
p. 2 Para. 2. Exposure measures are explanatory variables whose elasticity is constrained to 1. That
is, it is assumed that an increase in vehicle use of 1 vehicle mile produces a 1 unit increase in the
chance of a fatality (or casualty as the case may be). This is actually a maintained hypothesis. If
this hypothesis is incorrect, it can bias the other coefficients in the equation. Thus, the change
from fatalities/VMT to fatalities/registration-year, to fatalities per crash not only changes the
meaning of the analysis, it also may bias coefficients in the event that the true relationship
between fatalities and VMT is not an elasticity of 1.
p. 3 Line 1. Please acknowledge that your "accounting for differences in driver characteristics, crash
locations, and other vehicle attributes" is incomplete and that this could affect your inferences
about size and mass.
p. 3 Para. 2. NHTSA's use of "non-culpable" vehicles involved in two vehicle crashes as an exposure
measure raises its own issues. How non-culpable was the non-culpable vehicle. Often this is a
matter of degree, rather than black or white. Driver behavior may also be involved. It seems to
48
-------
me this is just another potential measure of exposure that may or may not be better than any
other measure and may introduce new sources of bias in the analysis.
p. 3 Para. 4. Induced exposure needs to be defined. What is it intended to mean? This needs to be
explicit.
Also, I am startled that there are no equations in these reports. Equations can provide an
unequivocal explanation of the assumed relationships that cannot be adequately accomplished
by words, in many cases. Why no equations?
p. 7 Para 3. CUVs and minivans are involved in fewer crashes with stationary objects than cars.
Why? Is it the drivers, the vehicles, or the passengers? How well can you control for such
differences? Not well. What does this mean for your analysis?
p. 9 Para 1. Here an equation showing how the weighting was done would be very helpful.
p. 9 Bullet 1. Excluding these vehicle types implies that the control variables in the model are not
adequate to account for whatever makes these vehicle types different from the vehicles
included in the analysis. First, this is an admission that the model is not adequate to explain the
fatalities associated with these vehicles. Second, it is an admission that if they were included
the coefficients on the variables of interest would likely be biased by spurious correlations.
Clearly, it would not even be sufficient to include the vehicles along with a control variable (e.g.,
X = 1 if vehicle is a police car, 0 otherwise). This is yet another indication that the model suffers
generally from omitted variables, errors in variables and correlation among right-hand side
variables.
p. 10 Line 3. Sentence does not make sense. Please correct.
p. 12 Sect. 2.3. Please provide an equation.
p. 13 Were the confidence intervals calculated using ex-l? Please state explicitly or, better, show an
equation.
Para. 2. Here another instance where you say "mass reduction increases societal fatality risk"
but you really are not entitled to say that. It is misleading. Also, the NHTSA CI's are larger, as
they should be in a joint test.
Para. 3. These results require an underlying theory for interpretation. The lack of one makes it
seem like there is just no consistency in the results.
Para. 4. The fact that the results for fatalities per crash differ substantially from fatality per VMT
may be very important. Taken at face value, it would imply that any negative effect of reduced
mass is due to its effect on crash avoidance (crash probability) rather than crashworthiness.
This is where the lack of a theory is most troubling. Why would that be? Are lighter vehicles
less easily controlled, etc.? Or, as seems much more likely, is there a spurious correlation
between mass and other omitted or imperfectly measured factors (including driver behavior)
that lead to an increased probability of a crash? Consider, for example, driver age. Driver age is
related to crash involvement. Driver age is a control variable. But are all young drivers the
49
-------
same? Is it possible that young drivers more prone to risky behavior tend to drive lighter
vehicles? If so, this could partly explain the result observed. Of course, this is just speculative,
but the point is that correlations with imperfectly measured and omitted factors are highly likely
to be present in the data and, if there, could easily affect the statistical inferences.
p. 14 Here we see that changing the exposure measure influence the effect of mass and size on
fatalities and casualties, which is more evidence that spurious correlations are likely to be
biasing estimated coefficients for mass and size.
The bar graphs with confidence intervals are well done and convey a great deal of information
effectively. The patterns of magnitude and statistical significance are difficult to interpret, partly
because there is no explicit theory of what should happen and partly, perhaps, because the
relationships are actually not real.
p. 16 Para. 1. Reduction in the mass of lighter cars increases crash frequency but reduces fatalities
per crash. This is contradictory to the previously maintained theory that mass protects due to
the physics of velocity changes in elastic collisions. Indeed, there is no theoretical explanation
for these results, only speculation.
p. 18 Para. 1. Here is a good example of such casual speculation.
p. 19 Para. 1. Developing VMT weights for the 13 states is a good idea, given the effect of exposure
measures on inferences. Still the results would not be definitive.
p. 20 Para. 2. Mass reduction leads to a large reduction in risk only in crashes with objects for heavier
cars? There is only one type of crash in which the simple physics of collisions leads to an
unambiguous benefit for increased mass, and that is collisions with moveable or breakable
objects. This finding contradicts even that.
Para. 3. More speculation, this time about rollovers.
p. 21 Para. 3. "Curiously,..." Curiouser and curiouser.
Figure 2.15 printed without labels. Could be my computer but the other graphs were fine.
p. 24 Para. 2. This is probably a very important finding that needs further investigation and
explanation. As figure 2.16 illustrates well, the correlations with mass and size are orders of
magnitude smaller than the correlations with driver and environmental factors. This is why
even small correlations with omitted or imperfectly measured control factors could be, are even
likely to be, predominantly responsible for the estimated coefficients of mass and size.
p. 25 Para. 2. The results for minivans discussed here could be due to what is going on inside the
vehicle as much or more than the vehicle itself. How could these results be explained in terms
of the vehicle itself. Figure 2.17 shows this again. The effects of calendar year dummies, which
can only be considered rough approximations to unknown and various time-related factors,
have much large effects than size or mass. Again in figure 2.20.
50
-------
p. 28 Para. 1. "Surprisingly,..." How can side airbags, which deploy only in a crash, reduce crash
frequency but not fatality risk in a crash? Only if the real effect is a reflection of who buys a
CUV/minivan with side airbags and how and where they drive. There are more surprising
inferences in paragraph 2 about male and female drivers. Surprising relative to what theory?
p. 32 The problem here is not numerical multicollinearity (numerical difficulties inverting the cross-
product matrix) but the more complex problem of correlations among right-hand side variables,
omitted variables, errors in variables, and correlations of included variables with omitted and
imperfectly measured variables. This leads to biased estimates.
p. 36 Table 3.1 cries out for inference based on joint probabilities. What is the probability of
observing "success" in at least 3 out of 27 trials when the true probability of success is only 0.05.
See discussion above. The probability is certainly much higher than 0.05 and probably closer to
0.5. The implication is that, taken together, these results do not show any statistically
significant relationship between mass or size and risk per crash. If there were a rigorous
underlying theory, the interpretation might be different (patterns of significance could matter)
but there is none. Again, good graphs on succeeding pages.
p. 41 The statistical significance of such a relationship should be the same whether bins are used or
not. Is it?
p. 42 Para. 3. R-squared is not the correct measure of statistical significance. Is the coefficient of
weight significantly different from zero?
p. 43 Para. 2. This is perhaps the key finding of the phase I and II analyses. Control variables explain
1-2 orders of magnitude more variance than size or mass. Still, most of the variance remains
unexplained and is uncorrelated with mass or size. It is very likely there is nothing going on
here.
p. 47 Para. 2. More evidence of correlation of mass and size with control variables and how changing
definitions or excluding control variables results in important changes in the coefficient of mass
and size. Such results are considered unstable.
p. 49 Para. 1. The rationale for the grouping of manufacturers is not obvious. Can you explain it?
Para. 2. Yet more evidence for the instability of the model and likelihood that variables still
missing from the model, plus errors in measuring the included control variables are likely biasing
the inferences. The results described in this paragraph do not make sense to me. How can they
be explained other than random results?
p. 50 Para. 1. More casual interpretation of results. OK, maybe, maybe not. Same for paragraph 2.
The economy faltered in 2008 but the big negative effect was in 2007. The downward trend
started in 2004. This correlates neither with vehicle mass changes over time nor economic
growth as measured by real GDP. Idle speculation.
p. 51 Para. 1. Good discussion of the gratuitous speculation by NHTSA about the meaning of the
observed correlations. This is more a Rorschach test than statistical analysis.
51
-------
p. 51 Para. 2. "We have no explanation for why..." More of this kind of honest appraisal is needed in
studies like these.
p. 52 More results for which there is no explanation.
p. 53 Why the interaction between calendar year dummy variables and safety equipment? The
presence of the safety equipment on a particular vehicle is established. What has calendar year
(not model year) to do with it? Again, one suspects spurious correlations.
p. 55 One needs to think carefully about the reasons why vehicles would be excluded. It does not
appear that NHTSA did that. First line of first paragraph "used" rather than "sused".
p. 57 Well reasoned. It is interesting that NHTSA resisted including footprint or size in previous
analyses on the grounds of correlation with mass. These results show that assertion was
groundless.
p. 59 Para. 3. Rather than say risk per VMT accounts for two effects, it is better to say it includes or
comprises two effects. But this statement also ignores the important influences of drivers and
environment and their potential correlations with other factors. Yes, it includes how well a
vehicle can be driven, but more importantly it includes how well a vehicle IS driven. That is in
there too and is very likely to be correlated with make, model, and other vehicle attributes.
p. 60 Para. 3. Here again, the conclusions are misstated. It is not a genuine "reduction" in mass, but
an association with the mass of vehicles. And how does it "lead" to and increase in crash
frequency? What is the theory or model that predicts this? Driver and environment are very
likely mixed up in these results to an unknown but likely substantial degree. So what can we
really conclude? Not this.
As I read the conclusions and inferences I find myself asking, why?, why?, repeatedly without
any sound explanations. Page 61, paragraph 3 contains more "surprising" results. Surprising
because they are contrary to theory? Surprising because they are contrary to intuition?
Surprising because they are random? To what can we attribute so many "surprising" results,
and how many must there be before one concludes that the analysis is not revealing what we
had hoped it would.
Para. 4. Again, this cries out for joint statistical inference. Three statistically significant results
out of 27 is probably nothing statistically significant at all.
p. 62 Para. 1. What this shows, again, is that the coefficients of mass and size are strongly influenced
by which control variables are included in the model and how they are defined. These results
and their implications need explanation. The bottom line is that the effects of mass and size are
likely to be (after the necessary joint significance calculations are done) not statistically
significant, not consistent, and not robust.
p. 63 How do mass and footprint reduction (again, it's not really reduction in the sense of designing
lighter vehicles to increase fuel economy or reduce GHG emissions, the issue at hand) increase
crash frequency. What is the theory? I don't find a theory in either the NHTSA report or the
phase I and phase II studies. Absent a theory, these results seem sufficiently unstable and
52
-------
inconsistent to be highly questionable as evidence of any relationship between mass or size and
crashworthiness or crash avoidance. I think joint estimation of significance levels would provide
additional support for this view.
53
-------
4. References
Tom Wenzel. Assessment of NHTSA's Report "Relationships Between Fatality Risk, Mass, and Footprint
in Model Year 2000-2007 Passenger Cars and LTVs". 2011.
Tom Wenzel. Analysis of the Relationship between Casualty Risk Per Crash and Vehicle Mass and
Footprint for Model Year 2000-2007 Light-Duty Vehicles. 2011.
54
-------
Appendix A: Resumes of Peer Reviewers
T. Donna Chen, P.E.
1206 Arcadia Ave, Austin, TX 78757 • (501)908-9620 • chen.donna@gmail.com
EDUCATION
University of Texas, Austin, TX
Doctorate of Philosophy, Civil Engineering - Transportation (3.9 GPR) August 2010- Present
• Thrust Fellow
• Advanced Institute for Transportation Infrastructure Engineering and Management Fellow
University of Texas-Arlington, Arlington, TX
Master of Engineering, Civil Engineering - Transportation (3.9 GPR) December 2008
Texas A&M University, College Station, TX
Bachelor of Science, Civil Engineering (3.8 GPR) May 2005
• National Merit Scholar
• Recipient of the Texas A&M President's Endowed and Merit Plus scholarships
• Recipient of the Dallas/Fort Worth Women's Transportation Seminar scholarship
• Recipient of Society of Women Engineers A&M Student Chapter scholarship
• Alumni of Texas A&M Engineering Scholars Program
WORK EXPERIENCE
Center for Transportation Research, Austin, TX
Graduate Research Assistant August 2010- Present
• Developing a reference on transportation economics for TxDOT
Richland College, Dallas, TX
Instructor, College of Business, Engineering, and Technology January 2010- August 2010
• Teaching statics and dynamics as part of core engineering curriculum.
HNTB Corp., Piano, TX
Transportation Planning Engineer II December 2008 - November 2009
Transportation Planning Engineer I June 2005 - December 2008
Performed geometric design, cost estimation, and operational analysis for projects such as:
New Route Studies
• Lavon Lake Bridge Study, Collin County, Texas - collected constraints data; developed and evaluated alignment
alternatives for ten mile bridge route; assisted in developing GIS exhibits; answered citizen questions at public
meeting.
• Border Highway-East, El Paso, Texas - developed four alignment and two interchange alternatives; evaluated
alternatives and selected preferred alternative based on constraints; assisted in development of technical documents.
• Collin County Outer Loop - developed alternative corridors and alignments; evaluated alternatives and selected locally
preferred alignment for 40 mile study from US 75 to Rockwall County Line.
Traffic Analyses
• I-95/395 HOT Lanes, Arlington, Virginia - performed LOS analysis for revised managed lane access for 24 mile
corridor.
• Beltway 8/SH 249 Connector, Houston, Texas - modeled two proposed ramping alternatives in CORSIM.
• I-30, Greenville, Texas - performed LOS analysis using HCS and prepared Interstate Access Justification report.
• I-35, Lorena, Texas - modeled traffic along corridor using CORSIM and prepared Interstate Access Justification
report.
Design Schematics
• Dallas North Tollway Ramping Modifications, Piano, Texas - designed various ramping improvements along DNT
between PGBT and SH 121 to improve congestion and accommodate for an additional mainlane in each direction.
• Dallas North Tollway/SH 121 Interchange, Piano, Texas - designed four ramping options for the retrofit of a fully
directional interchange between existing mainlanes; conducted LOS analysis for toll and non-toll options along SH
121.
• 1-10, El Paso, Texas - developed two alternatives for the addition of two managed lanes to the existing four-lane
facility.
• US 67, Cedar Hill and Midlothian, Texas - produced schematic plans for the addition of center reversible HOV lane.
55
-------
Participated in leadership development activities such as:
Leadership HNTB - completed professional development/project management training program for younger staff.
H4 Community Outreach Program - worked with Leadership HNTB classmates to implement a community outreach
program which yielded over 900 hours of combined service and over $13,000 in charitable donations in one year.
HNTB College Recruitment Team - represented HNTB as college recruiter at Texas A&M Engineering Career Fair
and Society of Women Engineers National Conference Career Fair.
Texas Transportation Institute, College Station, TX
Undergraduate Research Assistant May 2003 - May 2004
Analyzed travel behavior data for the Houston QuickRide (HOT lane) project using Excel and SPSS
Received the Dwight Look College of Engineering Undergraduate Summer Research Grant and published paper in
Southwest Regional University Transportation Center Compendium
PROFESSIONAL MEMBERSHIPS/HONOR SOCIETIES
Intelligent Transportation Society of America
UT Student Chapter President November 2010- Present
o Coordinated new student and intern peer support activities
o Organized prospective student recruitment activities
o Served as UT liaison for the 2011 Southwest Region University Transportation Center Student
Symposium
o Represented ITS as Explore UT day volunteer
American Society of Civil Engineers
Dallas Branch History and Heritage Committee Chair 2008-2010
Society of Women Engineers
Dallas Section Career Night Chair 2006 - 2008
Tau Beta Pi Member 2004 - Present
Chi Epsilon Member 2004 - Present
56
-------
CHARLES M. FARMER
1005 North Glebe Road, Suite 800
Arlington, Virginia 22201
cfarmer@iihs.org
(703)247-1590
EDUCATION
December, 1986
August, 1981
Iowa State University, Ames, Iowa
Doctor of Philosophy
Major: Statistics
Old Dominion University, Norfolk, Virginia
Master of Science
Major: Applied Mathematics
Concentration: Statistics
May, 1979
St. John Fisher College, Rochester, New York
Bachelor of Arts
Major: Mathematics
Minor: Political Science
PROFESSIONAL
EXPERIENCE
2004-Present
1997-2004
1994-1997
1987-1994
1986-1987
Director, Statistical Services
Senior Statistician
Statistician
Insurance Institute for Highway Safety
Arlington, Virginia
Assistant Professor
Department of Mathematics
James Madison University
Harrisonburg, Virginia
Visiting Assistant Professor
Department of Statistics
University of Kentucky
Lexington, Kentucky
PROFESSIONAL
AFFILIATIONS
American Statistical Association
57
-------
RECENT
PUBLICATIONS
June, 2011
October, 2010
Farmer, C.M. Methods for estimating driver death rates by
vehicle make and series. Arlington, VA: Insurance Institute
for Highway Safety.
Farmer, C.M., Braitman, K.A., and Lund, A.K. Cell phone use
while driving and attributable crash risk. Traffic Injury
Prevention 11:466-470.
August, 2010
May, 2010
April, 2010
February, 2010
February, 2010
McCartt, A.T., Farmer, C.M., and Jenness, J.W. Perceptions
and experiences of participants in a study of in-vehicle
monitoring of teenage drivers. Traffic Injury Prevention
11:361-370.
Farmer, C.M. Effects of electronic stability control on
fatal crash risk. Arlington, VA: Insurance Institute for
Highway Safety.
McCartt, A.T., Hellinga, L.A., Strouse, L.M., and Farmer,
C.M. Long-term effects of hand-held cell phone laws on
driver hand-held cell phone use. Traffic Injury Prevention
11:133-141.
Farmer, C.M. and Wells, J.K. Effect of enhanced seat belt
reminders on driver fatality risk. Journal of Safety
Research 41:53-57.
Farmer, C.M., Kirley, B.B., and McCartt, A.T. In-vehicle
monitoring and the driving behavior of teenagers. Journal
of Safety Research 41:39-45.
58
-------
DAVID L. GREENE
Home: 212 Way Station Trail • Farragut, Tennessee 37922 • (865) 966-0891
Work: Oak Ridge National Laboratory • National Transportation Research Center • 2360 Cherahala Boulevard •
Knoxville, Tennessee 37932 • (865) 946-1310
PERSONAL
Born: November 18, 1949, New York, New York
Married, two children
EDUCATION
THE JOHNS HOPKINS UNIVERSITY
Ph.D., Geography and Environmental Engineering, 1973—78
UNIVERSITY OF OREGON
M.A., 1972-73
COLUMBIA UNIVERSITY
B.A., 1967-71
EMPLOYMENT
UNIVERSITY OF TENNESSEE, KNOXVILLE 2010-PRESENT
1/2010-Present Senior Fellow, Howard H. Baker, Jr. Center for Public Policy
Research Professor, Department of Economics
INSTITUTE FOR TRANSPORTATION STUDIES, UNIVERSITY OF CALIFORNIA, DAVIS 2008-2009
9/2008-6/2009 Visiting Research Faculty
OAK RIDGE NATIONAL LABORATORY (ORNL) 1977-PRESENT
1999—Present Corporate Fellow, Oak Ridge National Laboratory
1989—1999 Senior Research Staff Member II and Manager of Energy Policy Research
Programs, Center for Transportation Analysis
1988—1989 Senior Research Analyst, Office of Policy Integration, U.S. Department of
Energy (On assignment from ORNL)
1987—1988 Head, Transportation Research Section
1984-1987 Senior Research Staff Member I
1982-1984 Research Staff Member
1980—1982 Leader, Transportation Energy Group
1977-1980 Research Associate
AWARDS AND HONORS
2011 DOE Vehicle Technologies Program R&D Award, U.S. Department of Energy (with Z. Lin)
2011 Edward L. Ullman Award, Association of American Geographers
2009 Alliance to Save Energy, Energy Efficiency Hall of Fame
2008 Science Communicator Award, UT-Battelle
Recognition by the Intergovernmental Panel on Climate Change for Contributions to the Award of the 2008
Nobel Peace Prize to the IPCC
59
-------
2007 Department of Energy Hydrogen Program R&D Award (with P.N. Leiby)
Barry D. McNutt Award for Excellence in Automotive Policy Analysis, Society of Automotive Engineers,
2007
Member Emeritus, Transportation Research Board Committee on Alternative Fuels, 2006
Barry D. McNutt Award for best paper of 2004, Energy Committee, Transportation Research Board
Lifetime National Associate of the National Academies, 2002
UT-Battelle Award for Excellence in Science and Technology, 2001
Oak Ridge National Laboratory Significant Event Award, 2001
Corporate Fellow of Oak Ridge National Laboratory, 1999
Outstanding Paper of 1999, The Energy Journal, International Association for Energy Economics
Lockheed-Martin Significant Event Award, 1999
Member Emeritus, Transportation Research Board Committee on Transportation Energy, 1998
Lockheed-Martin Significant Event Award, 1996
Distinguished Service Certificate, Transportation Research Board, 1993
ORNL Special Achievement Award, 1991
Distinguished Service Certificate, Transportation Research Board, 1989
Energy Specialty Group Paper Award, Association of American Geographers, 1986
ORNL Special Recognition Award, Oak Ridge National Laboratory, 1986
Technical Achievement Award, Martin Marietta Energy Systems, 1985
Pyke Johnson Award, Transportation Research Board, 1984
PROFESSIONAL ACTIVITIES
• Board of Directors, American Council for an Energy Efficiency Economy
• Board of Advisors, Institute for Transportation Studies, University of California, Davis
• Editorial Advisory Board, Transportation Research Part D, 1996-2006
• Editorial Board Member, Energy Policy, 2001-present
• Editorial Board Member, Journal of Transportation and Statistics, 2001-2006
• Editorial Board Member, Transportation Quarterly, 1999-2005
• Editor-in-Chief, Journal of Transportation and Statistics, 1997-2000
• Editorial Board Member, Macmillan Encyclopedia of Energy, 1998-2001
• Editorial Advisory Board, Transportation Research A, 1986-1997
• National Research Council
Transportation Research Board Standing Committees:
Committee on Transportation and Sustainability, Member, 2006-present
Committee on Energy, A1F01, Chairman 1983-1986, 1986-1990; Member, 1993-1998;
Member Emeritus, 1999-present
Subcommittee on Forecasting Transportation Energy Demand,
A1F01(2), Chairman, 1982-1983
Section F, Energy and Environmental Concerns, Chairman, 1990-1992
Committee on Alternative Fuels, A1F05, Member, 1993-2006,
Member Emeritus, 2006-present
Task Force on Freight Transportation Data, A1B51, Secretary, 1989-1996
Committee on Transportation Information Systems and Data Requirements,
Member, 1983-1986, 1986-1989
Ad Hoc Committees:
Committee on Transitions to Alternative Vehicles and Fuels, 2011-2012
Special Task Force on Energy and Climate Change, 2008-2009
Committee on the Assessment of Fuel Economy Technologies for Light-Duty Vehicles,
2007-2010
Planning Group for Workshop on Issues Related to Peaking of Global Oil Production, 2005
Committee on State Practices in Setting Mobile Source Emissions Standards, 2004-2006
Chair, Committee for the Symposium on Introducing Sustainability into Surface
Transportation Planning, 2003-2004
Panel on Combating Global Warming through Sustainable Surface Transportation Policy,
TCRP Project Panel H-21A, 2002-2005
60
-------
Committee on Effectiveness and Impacts of Corporate Average Fuel Economy (CAFE)
Standards, 2001
Committee for the Study of the Impacts of Highway Capacity Improvements on Air Quality
and Energy Consumption, 1993-1994
Committee on Fuel Economy of Automobiles and Light Trucks, Energy Engineering Board,
Commission on Engineering and Technical Systems, 1991-1992
Committee for the Study of High-Speed Surface Transportation in the United States, 1990
Planning Group on Strategic Issues in Domestic Freight Transportation, 1990
Steering Committee for Conference on Transportation, Urban Form, and the Environment,
1990
National Cooperative Highway Research Program, Panel on "Evaluating Alternative Methods
of Highway Finance," 1991-1992
Intergovernmental Panel on Climate Change
Lead Author, Working Group III, Fourth Assessment Report, 2007
Lead Author, Working Group III, Third Assessment, 2001
Lead Author, Working Group III, Aviation and the Global Atmosphere, 1999
Principal Lead Author, Working Group II, Second Assessment Report, 1995
Association of American Geographers
Board of Directors, Transportation Specialty Group, 1989-1991
Secretary-Treasurer, Transportation Geography Specialty Group, 1980-1982
Editor, Transportation Geography Newsletter, 1980-1982
Society of Automotive Engineers, member, 1985-present
International Association for Energy Economics, member
Consulting
International Council for Clean Transportation, 2011
International Transport Forum, 2007
Addx Corporation, 2007
United Nations Framework Convention on Climate Change, 2007
Securing America's Future Energy, 2007
Center for Clean Air Policy, 2007
Pollution Probe Canada, 2006-2007
The Energy Foundation China Project, 2005—present
The Pew Center on Global Climate Change, 2004—present
Eno Transportation Foundation, 1991-1996
Transportation Research Board, 1996-1997
BOOKS
and D.W. Jones and Mark Delucchi, eds., The Full Costs and Benefits of Transportation, Springer-Verlag,
Heidelberg, 1997.
Transportation and Energy, Eno Foundation for Transportation, Lansdowne, Virginia, 1996.
and D. J. Santini, eds., Transportation and Global Climate Change, American Council for an Energy
Efficient Economy, Washington, DC, 1993.
ARTICLES IN PROFESSIONAL JOURNALS
"What's Greener than a VMT Tax? The Case for an Indexed Energy User Fee to Finance U.S. Surface
Transportation," Transportation Research D-Environment, vol. 16, pp. 451-458.
"Uncertainty, Loss Aversion and Markets for Energy Efficiency," Energy Economics, vol. 33, pp. 608-
616,2011.
61
-------
Z. Lin and D.L. Greene, Predicting Individual On-road Fuel Economy Using Simple Consumer and
Vehicle Attributes, SAE Technical Paper Series No. 11SDP-0014, Society of Automotive Engineers,
Warrendale, PA, April 12, 2011.
and P.R. Boudreaux, D.J. Dean, W. Fulkerson, A.L. Gaddis, R.L. Graham, R.L. Graves, J.L. Hopson,
P. Hughes, M.V. Lapsa, I.E. Mason, R.F. Standaert, T.J. Wilbanks and A. Zucker, "The Importance of
Advancing Technology to America's Energy Goals," Energy Policy, vol. 38, no. 8, pp. 3886-3890, March
2010.
Rubin, J., P.N. Leiby and D.L. Greene, "Tradable Fuel Economy Credits: Competition and Oligopoly,"
Journal of Environmental Economics and Management, vol. 58, no. 3, pp. 315-328, 2009.
"Measuring Energy Security: Can the United States Achieve Oil Independence?" Energy Policy, 2009, Vol.
38, No. 4, pp. 1614-1621.
"Feebates, Footprints and Highway Safely," Transportation Research Part D, vol. 14, pp. 375-384, 2009.
"Vehicles and E85 Stations Needed to Achieve to Achieve Ethanol Goals," Transportation Research
Record No. 2058,-pp. 172-178.
and P.N. Leiby, P.D. Patterson, S.E. Plotkin and M. Sing, "Oil Independence: Achievable National Goal or
Empty Slogan?" Transportation Research Record, No. 2017, pp. 47-53, Washington, DC, 2007.
and J.L. Hopson, R. Goeltz and J. Li, "Analysis of In-Use Fuel Economy Shortfall Based on Voluntarily
Reported Mile-per-Gallon Estimates," Transportation Research Record, No. 1983, pp. 99-105, 2007.
Leiby, P.N., D.L. Greene, D. Bowman and E. Tworek, "Systems Analysis of Hydrogen Transition with
HyTrans," Transportation Research Record, No. 1983, pp. 129-139, 2007.
and J.L. Hopson and J. Li, "Have We Run Out of Oil Yet? Oil Peaking Analysis from an Optimist's
Perspective," Energy Policy, vol. 34, pp. 515-531, 2006.
S. Ahmad and D.L. Greene, "The Effect of Fuel Economy on Automobile Safely: A Reexamination,"
Transportation Research Record No. 1941, pp. 1-7, Washington, DC, January 2005.
and J.L.Hopson and J. Li, "Running Out of and Into Oil: Analyzing Global Depletion and Transition
Through 2050, Transportation Research Record 1880, pp. 1-9, Transportation Research Board,
Washington, DC, 2005.
and P.D. Patterson, M. Singh and J. Li, "Feebates, Rebates and Gas-Guzzler Taxes: A Study of Incentives
for Increased Fuel Economy," Energy Policy, vol. 33, no. 6, pp. 721-827, 2005.
Sheffield, J., et al., "Energy Options for the Future," Journal of Fusion Energy, vol. 23, no. 2, pp. 63-109,
2004.
and J. Hopson, "An Analysis of Alternative Forms of Automotive Fuel Economy Standards for the United
States," Transportation Research Record No. 1842, pp. 20-28, Transportation Research Board,
Washington, DC, 2003.
H.L. Hwang, S.M. Chin and D.L. Greene, "In, Out, Within and Through: Geography of Truck Freight in
the Lower 48," Transportation Research Record, no. 1768, pp. 18-25, Transportation Research Board,
Washington, DC, 2001.
and S.E. Plotkin, "Energy Futures for the U.S. Transportation Sector," Energy Policy, vol. 29, no. 14, pp.
1255-1270,2001.
62
-------
and N. Tishchishyna, "The Costs of Oil Dependence: A 2000 Update," Transportation Quarterly, vol. 55,
no. 3, pp. 11-32,2001.
H.L. Hwang, D.L. Greene, S.M. Chin, J. Hopson and A.A. Gibson, "Real-time Indicators of VKT and
Congestion: One Year of Experience," Transportation Research Record, no. 1719, pp. 209-214,
Transportation Research Board, Washington, DC, 2000.
and J.M. DeCicco, "Engineering-Economic Analyses of Automotive Fuel Economy Potential in the United
States," Annual Review of Energy and the Environment, vol. 25, pp. 477-536, 2000.
L.A. Greening, D.L. Greene and C. Difiglio, "Energy Efficiency and Consumption—The Rebound
Effect—A Survey," Energy Policy, vol. 28, pp. 389-401, 2000.
R.N. Schock, W. Fulkerson, M.L. Brown, R.L. San Martin, D.L. Greene and J. Edmonds, "How Much Is
Energy R&D Worth as Insurance?" Annual Review of Energy and the Environment, vol. 24, pp. 487-512,
Annual Review, Palo Alto, California, 1999.
S.M. Chin, D.L. Greene, J. Hopson, H.L. Hwang and B. Thompson, "Towards Real-Time Indices of U.S.
Vehicle Travel and Traffic Congestion," Transportation Research Record, no. 1660, pp. 132-139, National
Academy Press, Washington, DC, 1999.
and J. Kahn and R. Gibson, "Fuel Economy Rebound Effect for U.S. Household Vehicles," The Energy
Journal, vol. 20, no. 3, pp. 1-31, 1999.
"Survey Evidence on the Importance of Fuel Availability to Choice of Alternative Fuels and Vehicles,"
Energy Studies Review, vol. 8, no. 3, pp. 215-231, 1998.
"Why CAFE Worked," Energy Policy, vol. 26, no. 8, pp. 595-614, 1998.
and Donald W. Jones and Paul N. Leiby, "The Outlook for U.S. Oil Dependence," Energy Policy, vol. 26,
no. l,pp. 55-69, 1998.
and Michael Wegener, "Sustainable Transport," in Journal of Transport Geography, vol. 5, no. 3, pp. 177-
190, 1997.
Steven E. Plotkin and David Greene, "Prospects for Improving the Fuel Economy of Light-Duty Vehicles,"
Energy Policy, vol. 25, no. 14-15, pp. 1179-1188, 1997.
"Economic Scarcity: Monopoly, Not Geology, Threatens Global Supply," Harvard International Review,
vol. XIX, no. 3, Summer, 1997.
"Environmental Impacts," Journal of Transport Geography, vol. 5, no. 1, pp. 28-29, 1997.
"Energy for Transportation," Journal of Transport Geography, vol. 5, no. 1, pp. 30-32, 1997.
and Y. Fan, "Transportation Energy Intensity Trends, 1972-1992," Transportation Research Record, no.
1475, pp. 10-19, Energy and Environment, Transportation Research Board, Washington, DC, 1995.
M.A. Deluchi, D.L. Greene and Quanlu Wang, "Motor Vehicle Fuel Economy: The Forgotten Hydrocarbon
Control Strategy?" Transportation Research A, vol. 28A, no. 3, pp. 223-244, 1994.
"Transportation and Energy," Transportation Quarterly, vol. 48, no. 1, pp. 91-101, Winter, 1994.
and K.G. Duleep, "Costs and Benefits of Automotive Fuel Economy Improvement," Transportation
Research, vol. 27A, no. 3, pp. 217-236, May, 1993.
63
-------
"Transportation and Energy: The Global Environmental Challenge," Transportation Research, vol. 27A,
no. 3, pp. 163-166, May, 1993.
"Energy Efficiency Improvement Potential of Commercial Aircraft," Annual Review of Energy and
Environment, vol. 17, pp. 537-573, 1992.
"Vehicle Use and Fuel Economy: How Big is the Rebound Effect?" The Energy Journal, vol. 13, no. 1, pp.
117-143, Apnl 1992.
"A Note on OPEC Market Power and Oil Prices," Energy Economics, vol. 13, no. 2, pp. 123-129, April
1991.
"The Cost of Short-Run Pricing Strategies to Increase Corporate Average Fuel Economy," Economic
Inquiry, vol. XXIX, no. 1, pp. 101-114, January 1991.
"Fuel Choice for Multifuel Vehicles," Contemporary Policy Issues, vol. VIII, no. 4, pp. 118-137, October
1990.
"CAFE or PRICE? An Analysis of the Effects of Federal Fuel Economy Regulations and Gasoline Price
on New Car MPG, 1978-89," The Energy Journal, vol. 11, no. 3, pp. 37-57, September 1990.
"Technology and Fuel Efficiency," Forum for Applied Research and Public Policy, vol. 5, no. 1, pp. 23-
29, University of Tennessee, Spring 1990.
Carmen Difiglio, K.G. Duleep and D.L. Greene, "Cost Effectiveness of Future Fuel Economy
Improvements," The Energy Journal, vol. 11, no. 1, 1990.
"Short-Term Options for Controlling CO2 Emissions of Light-Duty Vehicles," SAE Technical Paper Series
901111, Society of Automotive Engineers, 1990.
"Motor Fuel Choice: An Econometric Analysis," Transportation Research A, vol. 23A, no. 3, pp. 243-253,
1989.
"Fuel Choice for Dual-Fuel Vehicles: An Analysis of the Canadian Natural Gas Vehicles Survey," SAE
Technical Paper Series 892067, Society of Automotive Engineers, Warrendale, Pennsylvania, 1989.
J. J. Erickson, D.L. Greene and A. J. Sabadell, "An Analysis of Transportation Energy Conservation Projects
in Developing Countries," Transportation, vol. 15, no. 3, pp. 163-189, 1988.
and J.T. Liu, "Automotive Fuel Economy Improvements and Consumers' Surplus," Transportation
Research A, vol. 22A, no. 3, pp. 203-218, 1988.
"Advances in Automobile Technology and the Market for Fuel Efficiency, 1978-1985," Transportation
Research Record 1155, pp. 18-27, Transportation Research Board, National Research Council,
Washington, DC, 1987.
and Anthony Araya Jacome, Robert Kowalski and Patricia S. Hu, "Road Transport Energy Conservation in
Costa Rica," Energy, vol. 12, no. 12, pp. 1299-1308, 1987.
and P.S. Hu, "A Functional Form Analysis of the Short-Run Demand for Travel and Gasoline by One-
Vehicle Households," Transportation Research Record, no. 1092, pp. 10-15, Transportation Research
Board, Washington, DC, 1986.
and N. Meddeb and J.T. Liu, "Vehicle Stock Modeling of Highway Energy Use: Tunisian and U.S.
Applications," Energy Policy, pp. 437-446, October 1986.
64
-------
"Efficiency-Related Changes in Automobile and Light Truck Markets," SAE Technical Paper Series, no.
861423, Society of Automotive Engineers, Warrendale, Pennsylvania, September 1986.
"The Market Share of Diesel Cars in the U.S., 1979-83," Energy Economics, vol. 8, no. 1, pp. 13-21,
January 1986.
and P.S. Hu and L. Till, "An Analysis of Trends in Automotive Fuel Economy from 1978 to 1984,"
Transportation Research Record, no. 1049, pp. 51-56, Washington, DC, 1985.
"Estimating Daily Vehicle Usage Distributions and the Implications for Limited-Range Vehicles,"
Transportation Research B, vol. 19B, no. 4, pp. 347-358, 1985.
and P.S. Hu, "Vehicle Usage in Multi-Vehicle Households and the Price of Gasoline," Transportation
Research Record, no. 988, pp. 19-24, Washington, DC, 1984.
and P.S. Hu and G.F. Roberts, "An Analysis of Geographical and Temporal Variation in Vehicles
Classification Count Statistics," Transportation Research Record, no. 987, pp. 21-28, Washington, DC,
1984.
and G.F. Roberts, "A Comment on Fuel Consumption for Road Transport in the U.S.A," Energy
Economics, vol. 6, no. 2, pp. 145-147, April 1984.
"A Derived Demand Model of Regional Highway Diesel Fuel Use," Transportation Research B, vol. 18B,
no. 1, pp. 43-61, 1984.
P.D. Patterson, F.W. Westbrook, D.L. Greene and G.F. Roberts, "Reasons for Changes in MPG Estimates,
Model Year 1978 to the Present," SAE Technical Paper Series, no. 840500, Society of Automotive
Engineers, Warrendale, Pennsylvania, February/March, selected for inclusion in 1984 SAE Transactions,
1984.
J. Soderstrom, E. Hirst, D. Greene and J. Trimble, "Have Department of Energy Conservation Programs
Saved Energy?" Evaluation Review, vol. 8, no. l,pp. 93-112, February 1984.
E. Hirst, R. Marlay, D. Greene and R. Barnes, "Recent Changes in U.S. Energy Consumption: What
Happened and Why?" Annual Review of Energy, vol. 8, pp. 193-243, Annual Reviews, Inc., Palo Alto,
Calrforma, 1983.
"Streamlining the Collection and Processing of Traffic Count Statistics, A Comment," Transportation
Research Record, no. 928, pp. 18-19, 1983.
"A Note on Implicit Consumer Discounting of Automobile Fuel Economy: Reviewing the Available
Evidence," Transportation Research, vol. 17B, no. 6, pp. 491^-99, 1983.
and C.K. Chen, "A Time Series Analysis of State Gasoline Demand, 1975-1980," The Professional
Geographer, vol. 35, no. 1, pp 40-51, February 1983.
G.F. Roberts and D.L. Greene, "A Method for Assessing the Market Potential of New Energy-Saving
Technologies," Transactions on Systems, Man, and Cybernetics, Institute of Electrical and Electronics
Engineers, vol. SMC-13, no. 1, pp. 3-7, January/February 1983.
Eric Hirst et al, "Effects of Improved Energy Efficiency on U.S. Energy Use: 1973-1980," Energy, vol. 7,
no. 11, pp. 897-907, 1982.
and G. Kulp, "An Analysis of the 1979-1980 Decline in gasoline Consumption in the United States,"
Energy, vol. 7, no. 4, pp. 367-375, 1982.
65
-------
and G. Walton, "Data and Methodological Problems in Establishing State Gasoline Conservation Targets,"
Transportation Research Record, no. 815, pp. 24-30, 1981.
and E. Chen, "Scrappage and Survival Rates of Passenger Cars and Trucks in the U.S., 1966-77,"
Transportation Research, vol. ISA, no. 5, pp. 383-389, 1981.
"City Size Distribution and Income Distribution in Space: A Comment," Regional Development Dialogue,
vol. II, no. 1, pp. 124-126, Spring 1981.
"A State Level Stock System Model of Gasoline Demand," Transportation Research Record, no. 801, pp.
44-50, 1981.
"Estimated Speed/Fuel Consumption Relationships for a Large Sample of Cars," Energy, vol. 6, pp. 441-
446, 1981.
"The Spatial Dimension of Gasoline Demand," Geographical Survey, vol. 9, no. 2, pp. 19-28, April 1980.
"Regional Demand for Gasoline: Comment," Journal of Regional Science, vol. 20, no. 1, pp. 103-109,
1980.
"Urban Subcenters: Recent Trends in Urban Spatial Structure," Growth and Change, vol. 11, no. 1, pp.
103-109, January 1980.
R. Dubin, D.L. Greene and C. Begovich, "Multivariate Classification of Automobiles Using an
Automobile's Characteristics' DataBase," Transportation Research Record, no. 726, pp. 29-27, 1979.
"State Differences in the Demand for Gasoline: An Econometric Analysis," Energy Systems and Policy,
vol. 3, no. 2, pp. 191-212, 1978.
and Joern Barnbrock, "A Note on Problems in Estimating Urban Density Models," Journal of Urban
Economics, vol. 5, April 1978.
and Rolf R. Schmitt, "An Alternative Derivation of the Intervening Opportunities Model," Geographical
Analysis, vol. 10, no. 1, January 1978.
Joem Barnbrock and D.L. Greene, "A Comment on Population Density and Trend Surface Analysis," Land
Economics, vol. 53, no. 2, May 1977.
CONTRIBUTIONS TO NATIONAL RESEARCH COUNCIL REPORTS
"Assessment of Fuel Economy Technologies for Light-Duty Vehicles," Report of the Committee on the
assessment of Technologies for Improving Light-duty Vehicle Fuel Economy, National Academies Press,
Washington, D.C., 2010.
J. Zucchetto, Trends in Oil Supply and Demand, Potential for Peaking of Conventional Oil Production, and
Possible Mitigation Options, a summary report of the Modeling the Oil Transition workshop, Member,
Planning Group and Keynote Speaker, Washington, DC, April 2006.
"State and Federal Standards for Mobile Source Emissions," Report of the Committee on State Practices in
Setting Mobile Source Emissions Standards, National Research Council, National Academies Press,
Washington, DC, March 2006.
"Integrating Sustainability into the Transportation Planning Process," Conference Proceedings 37,
Transportation Research Board of the National Academies, Washington, DC, 2005.
66
-------
"Effectiveness and Impact of Corporate Average Fuel Economy (CAFE) Standards," Report of the
Committee, National Research Council, National Academy Press, Washington, 2002.
"Ecological, Environmental and Energy-Related Issues, in The Future Highway Transportation System and
Society, Transportation Research Board, National Research Council, National Academy Press, Washington,
DC, 1997.
Expanding Metropolitan Highways: Implications for Air Quality and Energy Use, Special Report 245,
Transportation Research Board, National Research Council, Washington, DC, July 1995.
Automotive Fuel Economy: How Far Can We Go? Report of the Committee on Automobile and Light
Truck Fuel Economy, National Research Council, National Academy Press, Washington, DC, 1992.
In Pursuit of Speed: New Options for Intercity Passenger Transport, Special Report 233, Transportation
Research Board, National Research Council, Washington, DC, 1991.
and D. Sperling and B. McNutt, "Transportation Energy to the Year 2020," pp. 207-231, in A Look Ahead:
Year 2020, Special Report 220, Transportation Research Board, National Research Council, Washington,
DC, 1988.
CONGRESSIONAL TESTIMONY
"Technology-Neutral Incentives for Energy Efficient Low Greenhouse Gas Emitting Vehicles," Testimony
to the Finance Committee, United States Senate, April 23, 2009.
"Near-Term Options to Increase Fuel Economy and Reduce Petroleum Demand," Testimony to the
Committee on Energy and Natural Resources, United States Senate, July 23, 2008.
"Facing the Challenges of Oil Dependence and Climate Change: What Will It Take?" Testimony to the
Subcommittee on Energy and Water Development, U.S. House of Representatives Committee on
Appropriations, February 14, 2008.
"Policies to Increase Passenger Car and Light Truck Fuel Economy," Testimony to the U.S. Senate
Committee on Energy and Natural Resources, January 30, 2007.
"Is Cap-and-Trade a Sufficient Carbon Policy for Transportation?" Testimony to the U.S. Senate
Committee on Environment and Public Works, November 13, 2007, Washington, DC.
"Energy Challenges for Transportation in the 21st Century," Testimony to the National Surface
Transportation Policy and Revenue Study Commission, March 19, 2007, Washington, DC.
"Corporate Average Fuel Economy (CAFE) Standards," Testimony to the U.S. Senate Commerce
Committee, March 6, 2007, Washington, DC.
"Policies to Increase Passenger Car and Light Truck Fuel Economy," Testimony to the U.S. Senate
Committee on Energy and Natural Resources, January 30, 2007, Washington, DC .
"Observations on the H-Prize Act of 2006 (H.R. 5143)," Testimony to the U.S. House of Representatives
Committee on Science, April 27, 2006, Washington, DC.
"Improving the Nation's Energy Security: Can Cars and Trucks be Made More Fuel Efficient?" Testimony
to the U.S. House of Representatives Committee on Science, February 9, 2005, Serial No. 109-3,
U.S.G.P.O, Washington, DC.
67
-------
"The Outlook for Surface Transportation Growth," Testimony to the Subcommittee on Surface
Transportation of the Committee on Transportation Infrastructure of the United States House of
Representatives, March 28, 1996.
CONTRIBUTIONS TO BOOKS
"Measuring Energy Sustainability: Making Progress," in Linkages of Sustainability, T.E. Graedel and E.
van der Voet, eds., Strungmann Forum Reports, MIT Press, Cambridge, MA, 2010.
Loschel, A., J. Johnston, M.A. Delucchi, T.N. Demayo, D.L. Gauthier, D.L. Greene, J. Ogden, S. Rayner
and E. Worrell, "Energy: Stocks, Flows, and Prospects," in Linkages of Sustainability, T.E. Graedel and E.
van der Voet, eds., Strungmann Forum Reports, MIT Press, Cambridge, MA, 2010.
D.L. Greene, "What Will Replace Liquid Hydrocarbon Fuels?" in M.J. Benton, ed., The Seventy Great
Mysteries of the Natural World, Thames and Hudson, Inc., New York, 2008.
and J. German and M. A. Delucchi, "Fuel Economy: The Case for Market Failure," in Reducing Climate
Impacts of the Transportation Sector, D. Sperling and J.S. Cannon, eds., Springer Science+Business Media,
springer.com, 2008.
Ribeiro, S.K., S. Kobayashi, D.L. Greene et al., "Transport and its Infrastructure," Chapter 5 in Climate
Change 2007: Mitigation of Climate Change, Working Group III contribution to the Fourth Assessment
Report of the Intergovernmental Panel on Climate Change, Cambridge Press, Cambridge, United Kingdom.
A. Dumas, D.L. Greene and A. Bourbeau, "North American Feebate Analysis Model," in Driving Climate
Change, D. Sperling and J.S. Cannon, eds., Academic Press, San Francisco, CA, 2007.
"Transportation and Energy," in The Geography of Urban Transportation, S. Hanson and G. Giuliano, eds.,
The Guilford Press, New York, 2004.
"Transportation and Energy Overview," in Encyclopedia of Energy, Cutler J. Cleveland, ed., Elsevier,
Oxford, United Kingdom, 2004.
M. Wegener and D.L. Greene, "Sustainable Transport" in Social Change and Sustainable Transport, pp.
35-42, W.R. Black and P. Nijkamp, eds., Indiana University Press, Bloomington, IN, 2002.
"Sustainable Transportation," in International Encyclopedia of the Social and Behavioral Sciences, N.J.
Smelser and P.B. Baltes, eds., Elsevier, Oxford, pp. 15,335-15,339, 2002.
W.R. Moomaw, J.R. Moreira, K. Blok, D.L. Greene et al., "Technological and Economic Potential of
Greenhouse Gas Emissions Reduction," Chapter 3 in Climate Change 2001: Mitigation, contribution of
Working Group III to the Third Assessment Report of the Intergovernmental Panel on Climate Change,
Cambridge University Press, Cambridge, UK, 2001.
"Transportation's Oil Dependence and Energy Security in the 21st Century," in Environmental Change,
Adaptation, and Security, S.C. Lonergan, ed., NATO Advanced Science Institute Series, Klumer Academic
Publishers, Boston, 1999.
S.L. Baughcum, J.J. Begin, F. Franco, D.L. Greene et al., "Aircraft Emissions: Current Inventories and
Future Scenarios," Chapter 9 in Aviation and Global Atmosphere, Intergovernmental Panel on Climate
Change, Cambridge University Press, Cambridge, UK, 1999.
and Robert Gibson, "Transportation, Energy and the Environment," Chapter 4 in Transportation Statistics
Annual Report 1998, BTS98-S-01, Bureau of Transportation Statistics, U.S. Department of Transportation,
Washington, DC, September 1998.
68
-------
"Commercial Air Transport Energy Use and Emissions: Is Technology Enough?" pp. 207-228 in DeCicco
and Delucchi, eds., Transportation, Energy, and the Environment: How Far Can Technology Take Us?
American Council for an Energy Efficient Economy, Washington, DC, 1997.
B. McNutt, L. Fulton and D.L. Greene, "Is Technology Enough? A Synthesis of Views Expressed at the
Conference," pp. 251-256 in DeCicco and Delucchi, eds., Transportation, Energy, and Environment: How
Far Can Technology Take Us? American Council for an Energy Efficient Economy, Washington, DC,
1997.
and Robert Gibson, "Transportation, Energy, and the Environment," Chapter 4 in Transportation Statistics
Annual Report 1997, BTS97-5-01, Bureau of Transportation Statistics, U.S. Department of Transportation,
Washington, DC, October 1997.
and Donald W. Jones, "The Full Costs and Benefits of Transportation: Conceptual and Theoretical Issues,"
Chapter 1 of The Full Costs and Benefits of Transportation, Springer-Verlag, Heidelberg, 1997.
"The Cost of Transportation's Oil Dependence," in O. Hohmeyer, R.L. Ottinger and K. Rennings, eds.,
Social Costs and Sustainability, Springer-Verlag, Berlin, 1996.
"Twenty Years of Energy Policy: What Should We Have Learned?" in David L. Feldman, ed., The Energy
Crisis, The Johns Hopkins University Press, Baltimore, 1996.
"Environmental Impacts of Transportation," Chapter 6 in Transportation Statistics Annual Report 1996,
Bureau of Transportation Statistics, U.S. Department of Transportation, Washington, DC, October 1996.
R. Gibson and D.L. Greene, "Environmental Trends and the U.S. Transportation System," Chapter 7 in
Transportation Statistics Annual Report 1996, Bureau of Transportation Statistics, U.S. Department of
Transportation, Washington, DC, October 1996.
"Energy and Transportation," Chapter 4 in Transportation Statistics Annual Report 1996, Bureau of
Transportation Statistics, U.S. Department of Transportation, Washington, DC, October 1996.
"Transportation," Chapter 5 in Policies and Measures for Reducing Energy Related Greenhouse Emissions,
DOE/PO-0047, U.S. Department of Energy, Office of Policy and International Affairs, Washington, DC,
July 1996.
Guest editor, "Energy and Global Climate Change," Transportation Research, Special Issue, vol. 27A,
no. 3, May 1993.
"Regional Demand Implication for Gasoline Supply Shortages," in T.R. Lakshmanan and P. Nijkamp, eds.,
Systems and Models for Energy and Environmental Analysis, pp. 206-233, Gower, UK, 1983.
G. Samuels, A.B. Rose, D.L. Greene and J.N. Hooker, "Energy Conservation in Transportation," a chapter
in Vol. Ill of Advances in Energy Systems and Technology, Peter Auer, ed., Academic Press, New York,
1982.
"A Regional Stock System Model of Highway Gasoline Demand," in Changing Energy Use Futures,
Volume 1, Proceedings of the Second International Conference on Energy Use and Management, Los
Angeles, California, October 22-26, 1979.
69
-------
OTHER PUBLICATIONS
and K.G. Duleep, and G. Upreti, "Status and Outlook for the U.S. Non-Automotive Fuel Cell Industry:
Impacts of Government Policies and Assessment of Future Opportunities", ORNL/TM-2011/101, Oak
Ridge National Laboratory, Oak Ridge, Tennessee.
Bunch, D.S. and D.L. Greene, "Potential Design, Implementation, and Benefits of a Feebate Program for
New Passenger Vehicles in California," State of California Air Resources Board and the California
Environmental Protection Agency, Sacramento, California, 2011, available at
http://76.12.4.249/artman2/uploads/l/Feebate_Progr am_for_New_Passenger_Vehicles_in_California.pdf.
"Oil Peak or Panic?" Book review, Science, vol. 328, no. 5980, p. 828, May 14, 2010
"What's Greener than a VMT Tax? Discussion Paper, Howard H. Baker, Jr. Center for Public Policy, The
University of Tennessee, Knoxville, TN, May 2010.
and S.E. Plotkin, Reducing Greenhouse Gas Emissions from U.S. Transportation, Pew Center on Global
Climate Change, Arlington, VA, 2011.
How Consumers Value Fuel Economy: A Literature Review, EPA-420-R-10-008, U.S. Environmental
Protection Agency, March 2010.
"Why the Market for New Passenger Cars Generally Undervalues Fuel Economy," Discussion Paper No.
2010-6, International Transport Forum, OECD, Paris, January 2010.
D. McCollum, G. Gould and D. Greene, Greenhouse Gas Emissions from Aviation and Marine
Transportation: Mitigation Potential and Policies, Pew Center on Global Climate Change, Arlington,
Virginia, available at http://www.pewclimate.org/technologv/report/aviation-an-marine, December 2009.
"Energy Assurance: Essential Energy Technologies for Climate Protection and Energy Security,"
ORNL/TM-2009/314, Oak Ridge National Laboratory, Oak Ridge, TN, December 2009.
S. Plotkin et &\.,Multipath Transportation Futures Study: Vehicle Characterization and Scenario Analyses,
ANL/ESD/09-5, Argonne National Laboratory, Argonne, Illinois, July 2009.
and K.G. Duleep, "Bootstrapping a Sustainable North American PEM Fuel Cell Industry: Could a Federal
Acquisition Program Make a Difference?" ORNL/TM-2008/183, October, 2008.
"Future Prices and Availability of Transport Fuels," International Transport Forum, Research Round Table
on Oil Dependence: Is Transport Running Out of Affordable Fuel? November 14-15, 2007, OECD, Paris.
American Physical Society Study Group, Energy Future: Think Efficiency, American Physical Society,
College Park, Maryland, 2008.
D.L. Greene et al, Analysis of the Transition to Hydrogen Fuel Cell Vehicles and the Potential Hydrogen
Infrastructure Requirements, ORNL/TM-2008/30, Oak Ridge National Laboratory, Oak Ridge, Tennessee,
March, 2008.
"Future Prices and Availability of Transport Fuels," Discussion Paper No. 2007-15, International Transport
Forum, Joint Transportation Research Center, OECD, Paris,
http://www.internationltransport.forum.org/jtre/DiscussionPapers/DiscussionPaperl5.pdf.
"Transportation," in The First State of the Carbon Cycle Report (SOCCR): The North American Budget
and Implications for the Global Carbon Cycle. A Report by the U.S. Climate Change Science Program and
the Subcommittee on Global Change Research [King, A.W., L. Dilling, G.R. Zimmerman, D.M. Ciarman,
70
-------
R.A. Houghton, G. Marland, A.Z. Rose, and T.J. Wilbanks (eds)]. National Oceanic and Atmospheric
Administration, National Climate Data Center, Asheville, North Carolina, USA, pp. 73-84.
and P.N. Leiby and D. Bowman, Integrated Analysis of Market Transformation Scenarios with HyTrans,
ORNL/TM-2007/094, Oak Ridge National Laboratory, Oak Ridge, Tennessee, June 2007.
D. Gordon, D.L. Greene, M.H. Ross and T.P. Wenzel, "Sipping Fuel and Saving Lives: Increasing Fuel
Economy without Sacrificing Safety," International Council on Clean Transportation, Washington, DC,
June 2007.
and P.N. Leiby, "Oil Independence: Realistic Goal or Empty Slogan," published as "Expert Commentary"
on The Lugar Energy Initiative website,
http ://lugar. senate, gov/energy/links/commentary708 greene summary .html, 2007.
Editor, Modeling the Oil Transition: A Summary of the Proceedings of the DOE/EPA Workshop on the
Economic and Environmental Implications of Global Energy Transitions, ORNL/TM-2007/014, Oak Ridge
National Laboratory, Oak Ridge, Tennessee, February 2007.
and P.N. Leiby, The Oil Security Metrics Model, ORNL/TM-2006/505, Oak Ridge National Laboratory,
Oak Ridge, Tennessee, May 2006.
and S. Ahmad, Costs of U.S. Oil Dependence: 2005 Update, ORNL/TM-2005/45, Oak Ridge National
Laboratory, Oak Ridge, Tennessee, March 2005.
"Climate Change Policy for Transportation While Waiting for H2," in The 10-50 Solution: Technologies
and Policies for a Low-Carbon Future, Pew Center on Global Climate Change, National Commission on
Energy Policy, Washington, DC, March 2004.
and K.G. Duleep and W. McManus, Future Potential of Hybrid and Diesel Powertrains in the U.S. Light-
Duty Vehicle Market, ORNL/TM-2004/181, Oak Ridge National Laboratory, Oak Ridge, Tennessee,
August 2004.
and J.L. Hopson, Running Out of and Into Oil: Analyzing Global Depletion and Transition Through 2050,
ORNL/TM-2003/259, Oak Ridge National Laboratory, Oak Ridge, Tennessee, October 2003.
and R.C. Gibson and K.G. Duleep, Energy Star Concepts for Highway Vehicles, ORNL/TM-2003/37, Oak
Ridge National Laboratory, Oak Ridge, Tennessee, June 2003.
R. Nye, D.L. Greene and J.W. Saulsbury, Providing Consumers with Web-based Information on the
Environmental Effects of Automobiles, ORNL/TM-2003/166, Oak Ridge National Laboratory, Oak Ridge,
Tennessee, June 2003.
and A. Schafer, "Reducing Greenhouse Gas Emissions from U.S. Transportation," Pew Center on Global
Climate Change, Arlington, Virginia, May 2003.
and S. Plotkin and K.G. Duleep, Examining the Potential for Voluntary Fuel Economy Standards in the
United States and Canada, ANL/ESD/02-5, Center for Transportation Research, Argonne National
Laboratory, Argonne, Illinois, October 2002.
S.M. Chin, O. Franzese, D.L. Greene, H.L. Hwang and R.C. Gibson, Temporary Losses of Highway
Capacity and Impacts on Performance, ORNL/TM-2002/3, Oak Ridge National Laboratory, Oak Ridge,
Tennessee, May 2002.
TAFV Alternative Fuels and Vehicles Choice Model Documentation, ORNL/TM-2001/134, Oak Ridge
National Laboratory, Oak Ridge, Tennessee, July 2001.
71
-------
R.N. Schock, W. Fulkerson, M.L. Brown, R.L. San Martin, D.L. Greene and J.A. Edmonds, The Insurance
Value of Energy R&D, UCEL-ID-141815, CGSR-2001, 003, Center for Global Security Research,
Lawrence Livermore National Laboratory, Livermore, California, May 2001.
"Energy," in Chapter 5 of The Changing Face of Transportation, U.S. Department of Transportation,
Bureau of Transportation Statistics, BTSOO-007, Washington, DC, 2000.
with S. Plotkin, "Transportation Sector," Chapter 6 in Scenarios for a Clean Energy Future, Interlaboratory
Working Group on Energy-Efficient and Clean Energy Technologies, Oak Ridge National Laboratory,
ORNL/CON-476, Lawrence Berkeley National Laboratory, LBNL-44029, November 2000.
R.N. Schock, W. Fulkerson, M.L. Brown, R.L. San Martin, D.L. Greene and J. Edmonds, "How Much Is
Energy R&D Worth?" Energy, vol. 25, no. 2, pp. 4-8, 2000.
Costs of Oil Dependence: A 2000 Update, ORNL/TM-2000/152, Oak Ridge National Laboratory, Oak
Ridge, Tennessee, May 2000.
and J.M. DeCicco, Engineering-Economic Analyses of Automotive Fuel Economy Potential in the United
States, ORNL/TM-2000/26, Oak Ridge National Laboratory, Oak Ridge, Tennessee, February 2000.
and J.M. DeCicco, "Energy and Transportation Beyond 2000," Millennium Paper Series, Transportation
Research Board, National Research Council, Washington, DC, January 2000.
and many others, National Transportation Statistics 1999, BTS99-04, U.S. Department of Transportation,
Bureau of Transportation Statistics, Washington, DC, 1999.
and many others, Transportation Statistics Annual Report 1999, BTS99-03, U.S. Department of
Transportation, Bureau of Transportation Statistics, Washington, DC, 1999.
An Assessment of Energy and Environmental Issues Related to the Use of Gas-to-Liquid Fuels in
Transportation, ORNL/TM-1999/258, Oak Ridge National Laboratory, Oak Ridge, Tennessee, November
1999.
and J. Kahn and R. Gibson, An Econometric Analysis of the Elasticity of Vehicle Travel with Respect to
Fuel Cost per Mile Using RTEC Survey Data, ORNL-6950, Oak Ridge National Laboratory, Oak Ridge,
Tennessee, March 1999.
"Why CAFE Worked," Transportation Research Circular No. 492, Transportation Research Board,
National Research Council, Washington, DC, 1999.
R.N. Schock, W. Fulkerson, M.L. Brown, R.L. San Martin, D.L. Greene and J. Edmonds, How Much is
Energy R&D Worth as Insurance? UCRL-JC-131205, Lawrence Livermore National Laboratory,
Livermore, California, October 1998.
"Transportation and Energy Policy," in Twenty-Five Years of Energy and Environmental Policy, pp. 125-
140, University of Illinois at Chicago, Energy Resources Center, Chicago, Illinois, 1998.
"Ecological, Environmental and Energy-Related Issues," pp. 127-149 in The Future Highway
Transportation System and Society, Transportation Research Board, National Research Council, 1997.
and S. Plotkin, "Transportation Sector," Chapter 5 in Scenarios of U.S. Carbon Reductions, Interlaboratory
Working Group on Energy-Efficient and Low-Carbon Technologies, ORNL/CON-444, report to the Office
of Energy Efficiency and Renewable Energy, U.S. Department of Energy, Washington, DC, September
1997.
72
-------
and many others, Energy Technology R&D: What Could Make a Difference, ORNL-6921/V2-P1, Oak
Ridge National Laboratory, Oak Ridge, Tennessee, September 1997.
"Energy and Environmental Consequences of Transportation: Indicators of Sustainability," 51st Session of
the International Statistical Institute Proceedings, Istanbul, Turkey, August 18-26, 1997.
"Oil Dependence: The Value of R&D," 32" Intersociety Energy Conversion Engineering Conference
Proceedings, volume 3, pp. 2148-2153, Honolulu, Hawaii, July 28-30, 1997.
and Steve Plotkin and K.G. Duleep, "The Potential for Energy-Efficient Technologies to Reduce Carbon
Emissions in the United States: Transport Sector," 32nd Intersociety Energy Conversion Engineering
Conference Proceedings, volume 3, pp. 2114-2119, Honolulu, Hawaii, July 28-30, 1997.
"Grin and Obey," Energy, vol. XXII, p. 6, April 1997.
"Consequences of U.S. Oil Dependence," Energy, vol. XXI, no. 5, pp. 3-8, November 1996.
D. Streets and many others, "Inventory of Technologies, Methods, and Practices for Reducing Emissions of
Greenhouse Gases," technical appendix to Climate Change 1995: Impacts, Adaptations, and Mitigation of
Climate Change: Scientific-Technical Analyses, Argonne National Laboratory, Argonne, Illinois, May
1996.
one of Principal Lead Authors, "Industry, Energy and Transportation: Impacts and Adaptation," in Climate
Change 1995: Impacts, Adaptations and Mitigation of Climate Change: Scientific-Technical Analyses, R.T.
Watson, M.C. Zinyowera, R.H. Moss and D.J. Dokken, eds., Intergovernmental Panel on Climate Change,
Cambridge University Press, 1996.
one of Contributing Authors, "Mitigation Options for the Transportation Sector," in Climate Change 1995:
Impacts, Adaptations and Mitigation of Climate Change: Scientific-Technical Analyses, R.T. Watson, M.C.
Zinyowera, R.H. Moss and D.J. Dokken, eds., Intergovernmental Panel on Climate Change, Cambridge
University Press, 1996.
"Commercial Air Transport Energy Use and Emissions: Is Technology Enough?" in Sustainable
Transportation: Is Technology Enough? Proceedings of the 1995 Asilomar Conference on Transportation
and Energy, American Council for an Energy-Efficient Economy, Washington, DC, 1995.
"Transportation, Energy and Environment: Global Policy Issues for the Early 21st Century," in
International Workshop on Motor Vehicles and Global Environmental Problems, Japan Automotive
Research Institute, Tokyo, Japan, October-November 1995.
P.N. Leiby, D.L. Greene and Harry Vidas, Market Potential and Impacts of Alternative Fuel use in Light
Duty Vehicles: A 2000/2010 Analysis, DOE/PO-0042, Office of Policy Analysis, U.S. Department of
Energy, Washington, DC, January 1996.
and X. Han, "The Unintended Consequences of Transportation," Chapter 3 in Transportation Statistics
Annual Report 1995, pp. 43-86, Bureau of Transportation Statistics, U.S. Department of Transportation,
Washington, DC, 1995.
with D.W. Jones and P.N. Leiby, The Outlook of U.S. Oil Dependence, ORNL-6783, Oak Ridge National
Laboratory, Oak Ridge, Tennessee, May 1995.
with S.M. Chin and R. Gibson, Aggregate Vehicle Travel Forecasting Model, ORNL-6872, Oak Ridge
National Laboratory, Oak Ridge, Tennessee, May 1995.
73
-------
D. Belzer, J. Roop, R. Sands and D.L. Greene, Energy Conservation Trends, Energy Efficiency in the U.S.
Economy, Report Three, DOE/PO-0034, U.S. Department of Energy, Office of Energy Efficiency and
Alternative Fuels Policy, Washington, DC, April 1995.
Transportation Energy Efficiency Trends, 1972-1992, ORNL-6828, Oak Ridge National Laboratory, Oak
Ridge, Tennessee, December 1994.
Alternative Fuels and Vehicles Choice Model, ORNL/TM-12738, Oak Ridge National Laboratory, Oak
Ridge, Tennessee, October 1994.
and E. Hillsman, L. Fulton, A.K. Wolfe and J. Niles, Energy Emissions and Social Consequences of
Telecommuting, DOE/PO-0026, Technical Report One, Energy Efficiency in the U.S. Economy Series,
U.S. Department of Energy, Office of Policy, Planning and Program Evaluation, Washington, DC, June
1994.
R.R. Schmitt, D.L. Greene et al., Transportation Statistics Annual Report, Bureau of Transportation
Statistics, U.S. Department of Transportation, Washington, DC, 1994.
and Paul N. Leiby, The Social Costs to the U.S. of Monopolization of the World Oil Market, 1972-1991,
ORNL-6744, Oak Ridge National Laboratory, Oak Ridge, Tennessee, March 1993.
Impacts on Home Heating Costs of Incentives for Alternative Fuel Vehicles, Technical Report Eight, U.S.
Department of Energy, Office of Domestic and International Energy Policy, Washington, DC, May 1992.
with K.G. Duleep, Costs and Benefits of Automotive Fuel Economy Improvement: A Partial Analysis,
ORNL-6704, Oak Ridge National Laboratory, Oak Ridge, Tennessee, March 1992.
M.A. Deluchi, Q. Wang and D.L. Greene, Motor Vehicle Fuel Economy: The Forgotten Hydrocarbon
Control Strategy? ORNL-6715, Oak Ridge National Laboratory, Oak Ridge, Tennessee, July 1992.
"Transportation Energy Policy: Back to the Past or Ahead to the Future?" in Twenty Years of Energy
Policy: Looking Toward the Twenty-First Century, Proceedings of the Twentieth Annual Illinois Energy
Conference, Energy Resources Center, University of Illinois at Chicago, Chicago, Illinois, November 23-
24, 1992.
with M. Singh, E. Ecklund, R. Bechtold and C. Saricks, "Second Interim Report of the Interagency
Commission on Alternative Motor Fuels," Office of Energy Demand, Office of Policy, Planning and
Analysis, Washington, DC, 1992.
Q. Wang, D.L. Greene and M.A. Deluchi, "Effects of Increasing Fuel Economy on Gasoline Vehicle HC
Emissions," Proceedings of the 84th Annual Meeting
Association, Vancouver, BC, Canada, June 16-21, 1991.
Emissions," Proceedings of the 84th Annual Meeting and Exhibition, Air and Waste Management
"Coverage and Quality Problems with Existing Data Resources for Freight Transportation," Proceedings of
the Special Conference on Freight Transportation Data: The Changing Federal Role Since Deregulation,
Transportation Research Circular, No. 367, Transportation Research Board, Washington, DC, 1990.
and M. Singh, First Interim Report on the Interagency Commission on Alternative Motor Fuels, Office of
Energy Demand Policy, Office of Policy, Planning and Analysis, U.S. Department of Energy, Washington,
DC, September 30, 1990.
"Commercial Aircraft Fuel Efficiency Potential Through 2010," Proceedings 1990 Intersociety Energy
Conversion Engineering Conference, Reno, Nevada, published by the American Institute of Chemical
Engineers, August 12-17, 1990.
74
-------
Energy Efficiency Improvement Potential of Commercial Aircraft to 2010, ORNL-6622, Oak Ridge
National Laboratory, Oak Ridge, Tennessee, June 1990.
and Anju Rathi, Alternative Motor Fuel Use Model: Model Theory and Design, and User's Guide,
ORNL/TM-11448, Oak Ridge National Laboratory, Oak Ridge, Tennessee, April 1990.
with Donald Igo, "Energy Dependence," Chapter 3 in National Strategic Transportation Study, U.S.
Department of Transportation, Washington, DC, 1989.
"A Context for Estimating Economic and Energy Security Benefits," Technical Report Two, Assessment of
Costs and Benefits of Flexible and Alternative Fuel Use in the U.S. Transportation Sector, DOE/PE-0091,
Office of Policy, Planning and Analysis, U.S. Department of Energy, Washington, DC, September 1989.
and J.C. Liang, Modeling Operating Weight and Axle Weight Distributions for Highway Vehicles, ORNL-
6437, Oak Ridge National Laboratory, Oak Ridge, Tennessee, July 1988.
et al., "Research Priorities in Transportation and Energy," Transportation Research Circular, no. 323,
Transportation Research Board, Washington, DC, September 1987.
Simulating the Market for Automotive Fuel Efficiency: The SHRSIM Model, ORNL/TM-10074, Oak Ridge
National Laboratory, Oak Ridge, Tennessee, February 1987.
and M.C. Holcomb, Off-Highway Use of Gasoline in the United States, U.S. Department of Transportation,
Federal Highway Administration, Office of Highway Information Management, Washington, DC, July
1986.
RUMS, A PC-Based FORTRAN Program for Estimating Consumer Surplus Changes Using Multinomial
Logit and Hedonic Demand Models, ORNL/TM-10069, Oak Ridge National Laboratory, Oak Ridge,
Tennessee, August 1986.
Driver Energy Conservation Awareness Training: Review and Recommendations, ORNL/TM-9897, Oak
Ridge National Laboratory, Oak Ridge, Tennessee, May 1986.
R.L. Graves, D.L. Greene, E.W. Gregory, II, Application of the Adiabatic Diesel to Heavy Trucks: A
Technology Assessment, ORNL/TM-9554, Oak Ridge National Laboratory, Oak Ridge, Tennessee, March
1986.
and R. Kowalski and F. Southworth, The Transportation Sector in Costa Rica and Opportunities for
Energy Conservation, a report of the Energy Conservation Services Program, U.S. Agency for International
Development, Office of Energy, Washington, DC, May 1985.
P.S. Hu, D.L. Greene and L.E. Till, Motor Vehicle MPG and Market Shares Report: First Six Months of
Model Year 1984, ORNL/TM-9391, Oak Ridge National Laboratory, Oak Ridge, Tennessee, October 1984.
and P.S. Hu and A.B. Rose, Transportation Energy Use and Efficiency in Tunisia, ORNL-6066, Oak Ridge
National Laboratory, Oak Ridge, Tennessee, August 1984.
"Highway Fuel Use: Trends and Factors," Proceedings of the Energy Information Administration
Symposium on Petroleum Information, DOE/EIA-0425, Energy Information Administration, Washington,
DC, September 1983.
G.F. Roberts and D.L. Greene, Trends in Heavy Truck Energy Use and Efficiency, ORNL/TM-8843, Oak
Ridge National Laboratory, Oak Ridge, Tennessee, October 1983.
75
-------
et al., An Overview of the Wharton School Assessments of the Short-Term Integrated Forecasting System,
DOE/NBB-0046, Energy Information Administration, Office of Statistical Standards, Washington, DC,
July 1983.
"Regional Demand Implications for Gasoline Supply Shortages," in T.R. Lakshmanan and P. Nijkamp,
eds., Systems and Models for Energy and Environmental Analysis, Gower, UK, pp. 206-233, 1983.
and E. Hirst, J. Soderstrom and J. Trimble, Estimating the Total Impact on Energy Consumption of
Department of Energy Conservation Programs, ORNL-5925, Oak Ridge National Laboratory, Oak Ridge,
Tennessee, November 1982.
"Gasoline use in the United States," Petroleum Supply Monthly, DOE/EIA-0109 (82-05), U.S. Department
of Energy, Washington, DC, 1982.
and G. Kulp, G.H. Walton and D.B. Shonka, Transportation Energy Use 1973-80: Changes, Trends, and
Causes, ORNL/TM-7953, Oak Ridge National Laboratory, Oak Ridge, Tennessee, April 1982.
G. Samuels, A.B. Rose, D.L. Greene and J.N. Hooker, "Energy Conservation in Transportation," a chapter
in Vol. Ill of Advances in Energy Systems and Technology, Peter Auer, ed., Academic Press, New York,
1982.
E. Hirst, D.L. Greene et al., Energy Use from 1973 to 1980: The Role of Improved Energy Efficiency,
ORNL/CON-79, Oak Ridge National Laboratory, Oak Ridge, Tennessee, December 1981.
The Aggregate Demand for Gasoline and Highway Passenger Vehicles in the United States: A Review of
the Literature, 1938-1978, ORNL-5728, Oak Ridge National Laboratory, Oak Ridge, Tennessee, July
1981.
J.N. Hooker, A.B. Rose and D.L. Greene, End Use Energy Consumption Data Base: Transportation
Sector, DOE/EIA/CR-7405-01, U.S. Department of Energy, Washington, DC, February 1980.
A Statistical Analysis of State VMT Estimates in VMT Statistics, Lifetime VMT, and Current State Methods
of Estimating VMT, ORNL/TM-6327, Oak Ridge National Laboratory, Oak Ridge, Tennessee, February
1979.
G. Kulp, D.L. Greene et al., Regional Analyses of Highway Energy Use, ORNL-5587, Oak Ridge National
Laboratory, Oak Ridge, Tennessee, December 1979.
"A Regional Stock System Model of Highway Gasoline Demand," in Changing Energy Use Futures,
Volume 1, Proceedings of the Second International Conference on Energy Use Management, Los Angeles,
California, October 22-26, 1979.
and T.P. O'Conner, P.D. Patterson, A.B. Rose and D.B. Shonka, Regional Transportation Energy
Conservation Data Book, ORNL-5435, Oak Ridge National Laboratory, Oak Ridge, Tennessee, September
1978.
Econometric Analysis of the Demand for Gasoline at the State Level, ORNL/TM-6326, Oak Ridge National
Laboratory, Oak Ridge, Tennessee, July 1978.
An Investigation of the Variability of Gasoline Consumption among States, ORNL-5391, Oak Ridge
National Laboratory, Oak Ridge, Tennessee, April 1978.
R.R. Schmitt and D.L. Greene, "Evaluating Transportation Innovations with the Intervening Opportunities
Model," Proceedings of the Northeast American Institute for Decision Sciences, April 1977.
76
-------
M.G. Wolman, D.L. Greene and N.L. Froomer, An Analysis of the Physical Characteristics of the
Shanandoah River Which Affects Its Navigability and An Analysis of the Physical Characteristics of the
Potomac River Which Affects Its Navigability, prepared for the Baltimore District, U.S. Army Corps of
Engineers, Chesapeake Research Consortium, pub. nos. 37S and 38S, resp., March 1975.
G. Power, M.G. Wolman et al., Report on the Shenandoah River: An Investigation to Determine
Navigability and Report on the Potomac River: An Investigation to Determine Navigability, prepared for
the Baltimore District, U.S. Army Corps of Engineers, Chesapeake Research Consortium, pub. nos. 37 and
38, resp., March 1975.
FORTHCOMING PUBLICATIONS
"Rebound 2007: Analysis of National Light-Duty Vehicle Travel Statistics," article in press, Energy Policy.
C. Liu and D.L. Greene, "Impacts of Feebates in Combination with Fuel Economy and Emissions
Standards on U.S. Light-Duty Vehicles Fuel Use and Greenhouse Gas Emissions," forthcoming,
Transportation Research Record.
Z. Lin and D.L. Greene, "Significance of Daily VMT Variation over Time and Among Drivers on
Assessment of PHEV Energy Impact," forthcoming, Transportation Research Record.
Z. Lin and D.L. Greene, "Promoting the Market for Plug-in Hybrid and Battery Electric Vehicles: The Role
of Recharge Availability," forthcoming, Transportation Research Record.
77
-------
KARA M. KOCKELMAN
Professor & William J. Murray Jr. Fellow
Department of Civil, Architectural & Environmental Engineering - University of Texas at Austin
Note: Full CV available at www.ce.utexas.edu/prof/kockelman
EDUCATION
University of California at Berkeley BSCE 1991, MSCE 1996, PhD 1998, MCP 1996
PROFESSIONAL EXPERIENCE - Academic
University of Texas at Austin Assistant Professor of Civil Engineering 1998-2004
University of Texas at Austin Associate Professor of Civil Engineering 2004-2009
University of Texas at Austin Professor of Civil Engineering 2009 - Present
Licensed Professional Engineer, California (Certification #C057380) and Texas (Certification #93443)
Licensed City Planner, AICP 2008 & Member of American Planning Assoc. National & Texas Chapters,
2008
SELECTED RELEVANT REFEREED PUBLICATIONS
1. Musti, S., K. Kortum and K. Kockelman. "Household Energy Use and Travel: Opportunities for
Behavioral Change." Transportation Research 16D: 49-56, 2011.
2. Kockelman, K., M. Thompson and C. Frei. "Americans' Contributions to Climate Change:
Opportunities for Meeting Carbon Targets." Journal of Urban Planning and Development, Vol. 137
(2): 91-100,2011.
3. Paul, B., K. Kockelman and S. Musti. "The Light-Duty-Vehicle Fleet's Evolution: Anticipating
PHEV Adoption and GHG Emissions across the U.S. Fleet." Forthcoming in Transp Research
Record, 2011.
4. Kockelman K., M. Bomberg, M. Thompson and C. Whitehead. "GHG Emissions Control Options:
Opportunities for Conservation." Report commissioned by National Academy of Science's
Committee for the Study on the Relationships among Development Patterns, VMT, and Energy
Conservation. June 2009.
5. Khan, M., and K. Kockelman. Predicting the Market Potential of Plug-In Electric Vehicles Using
Multiday GPS Data. Proc'gs of the Annual Meeting of the Transportation Research Board and under
review for publication in Transportation Research Record.
SELECTED OTHER REFEREED PUBLICATIONS
1. Tirumalachetty, S. and K. Kockelman. "Forecasting Greenhouse Gas Emissions from Urban Regions:
Microsimulation of Land Use and Transport Patterns in Austin, Texas." Under review for publication
in Journal of Transport Geography.
2. Turtle, D. P. and K. Kockelman. "Electrified Vehicle Technology Trends, Infrastructure Implications
and Cost Comparisons." Forthcoming in Journal of the Transportation Research Forum.
Tirumalachetty, S. and K. Kockelman. "The Welfare Implications of Carbon Taxes and Carbon Caps:
A Look at U.S. Households." Under review for publication to NOVA Science Publishers as a chapter
in Household Energy: Economics, Consumption and Efficiency. Bomberg, M., K. Kockelman, M.
Thompson. "GHG Emissions Control Options: Assessing Transportation & Electricity Generation
Technologies & Policies to Stabilize Climate Change." Under review for publication in the Journal of
the Transportation Research Forum. Kockelman, K. "Transportation and Land Use Solutions for
Low-Carbon Cities." Under review for publication in the Journal of the Transportation Research
Forum.
Over 95 refereed publications to date, and over 50 technical reports, total.
Over 100 referred conference proceedings and more than 250 additional presentations of research.
78
-------
SELECTED HONORS AND AWARDS
U.C. Berkeley's University Medal - "Most Distinguished Graduate" of Graduating Class of 5,300 students,
1991
National Science Foundation Faculty Early Career Development (CAREER) Award, 2000-2004
Ford CAREER Award, given by Ford Motor Company's Ford Fund, 2002
One the World's Top 100 Young Innovators, according to MIT's Technology Review, May 2002
Named to the National Academy of Engineering's Gallery of Women in Engineering, August 2002
Inaugural recipient of the Annual New Faculty Award, sponsored by the Council of University Transportation
Centers and the American Road and Transportation Builders Association, recognizing "outstanding
teaching and research contributions to the transportation field" December 2002
Recipient of the Geoffrey J.D. Hewings Award for 2006 presented by the Regional Science Assoc.
International
Recipient of ASCE's 2007 Harland Bartholomew Award, in recognition of contributions to the
enhancement of the role of the civil engineer in urban planning and development
Recipient of the Women's Transportation Seminar 2007 Heart of Texas Chapter Woman of the Year
Award
Recipient of ASCE's 2010 Walter L. Huber Research Prize in Transportation Engineering, for
contributions in the areas of data acquisition and analysis to facilitate decisions in transport planning
and policy-making
RELATED PROFESSIONAL ASSIGNMENTS, SYNERGISTIC TO THIS PROPOSAL
Member, Executive Committee, UT Austin & Texas A&M, NSF-sponsored Plug-in Electric Vehicle
Industry-University Research Center (IURC), 2010 - present.
Member, UT Austin's NSF IGERT for Sustainable Grid Integration of Distributed & Renewable Energy
Systems, 2010-present, & NSF RCN SEES, for Sustainable Cities: People, Infrastructures & the Energy-
Climate-Water Nexus, 2011-present.
Member, Advisory Council on Transportation Statistics (ACTS), Bureau of Transportation Statistics
Research and Innovative Technology Administration (RITA), U.S. Department of Transportation,
2010-present.
Member, National Research Council Committee for the Study on Relationships among Development
Patterns, VMT, and Energy Conservation, 2007-2009.
Member, RAND Corporation's Panel on Transportation and Climate Change, Washington, DC, June
2008
Member, TRB Committees on Transportation and Land Development, Transport Economics, and
Statistical Methods (and Past Chair of TRB's Survey Methods Committee).
Member, Editorial Advisory Boards of Transportation Research (Part B), Journal of Transport and Land
Use, Journal of Regional Science, and Papers in Regional Science.
PAST RELATED ENGAGEMENTS
Member, Transportation Electrification Panel, Sponsored by Indiana University's School of Public and
Environmental Affairs, producing report "Plug-in Electric Vehicles: A Practical Plan for Progress"
report (available at http://www.indiana.edu/~spea/pubs/TEP combined.pdf), 2010-2011.
Invited presentation of "Urban Planning, Land Use, and Vehicle Technologies to Reduce GHG Emissions
in Cities" for the National Academy of Science's EU-US Frontiers of Engineering Symposium, at UC
Irvine, November 3, 2011; and for UT Austin's Energy and Urban Development Conference,
November 18,2011.
Presentation of "PEV Market Potential Using Multi-Day GPS Data" for the NSF EV-TEC Industry-
University Research Center workshop on Electric Vehicles - Transportation and Electricity
Convergence, Industry Advisory Panel Meeting, Houston, TX, November 2, 2011.
Invited Presenter, US-China NSF Workshop titled "Pathways Toward Low Carbon Cities: Quantifying
Baselines and Interventions", Hong Kong, December 13-14, 2010.
79
-------
Invited Lecture titled "Environmental Impacts of Land Use and Transportation Policies" for Europe's
Kuhmo-Nectar Summer School in Transportation Economics, Valencia, Spain, July 2010.
Invited presentation, titled "Incorporating Environment and Equity into Evaluations of Land Use
Transportation Systems" for UCLA Symposium titled "Transportation - Land Use -Environment
Connection," Lake Arrowhead, CA, October 18-20, 2009.
Invited presentation, titled "Credit Based Congestion Pricing: An Opportunity for Reductions in Greenhouse
Gas Emissions?" at the International Colloquium on Transport, Energy and Greenhouse Gases: Will
Rationing be Necessary?, 19th Entretiens du Centre Jacques Cartier in Lyon, France, December 4-5, 2006.
Please visit http://www.caee.utexas.edu/prof/kockelman/ for access to pre-prints of all papers and
complete vitae (listing all research projects and publications).
80
-------
CURRICULUM VITAE
R. MICHAEL VAN AUKEN
Dynamic Research Inc.
355 Van Ness Avenue
Torrance, California 92691
(310)212-5211
EDUCATION
Bachelor of Science in Aerospace Engineering,
University of Michigan, 1980
Master of Science in Aeronautical and Astronautical Engineering,
Stanford University, 1984
Engineer in Aeronautical and Astronautical Engineering,
Stanford University, 1987.
Thesis was on Model Reduction with Generalized Input/Output Weightings, a
model reduction method for linearized road vehicle dynamics models with
potential application to automotive active suspensions.
PROFESSIONAL EXPERIENCE
1989 Principal Engineer
to Dynamic Research Inc., Torrance, California
present
Involved in various technical activities in the areas of vehicle
dynamics and control, crashworthiness and crash avoidance;
including ride characteristics, handling, occupant injury assessments
and technology effectiveness estimates, for automobiles, motorcycles,
and ATVs. This involved mathematical modeling and computer
simulation of driver and vehicle systems, data analysis, and
interpretation of results; as well as full scale and component testing
and model validation. Other activities have included development of
tire-road math models; Fourier analysis, sound signal and other types
of signal analysis; large scale, computer simulations of multi-body
dynamics; finite element analysis; and various types of statistical
analyses and experimental design.
Applications in these areas have included studies related to the
development of transient ride and noise discomfort metrics for
passenger cars, Weibull analysis of automobile transmission failures,
and injury and fatality assessment of accident data. The ride and
noise discomfort metric development included, for example, a large
scale computational effort to analyze, correlate, and interpret driver
and vehicle objective and subjective data. Similar studies have
81
-------
recently been accomplished on the topic of driver attentional workload
1987 Sr. Staff Engineer
to Hughes Aircraft Company, El Segundo, California
1989
Involved in several automotive systems engineering tasks including
defining vehicle performance requirements for a chassis subsystem;
participating in the design, development, and verification of several
vehicle dynamics systems, including developing tire-road contact
force and moment math models; and identification of significant
driver/vehicle feedback factors for driving simulators.
1980 to Sr. Research Engineer
1987 Lockheed Missiles & Space Company, Inc., Sunnyvale, California
Involved in the design, development, and verification of a attitude
determination system based on optimal estimation methods, including
allocating requirements to subsystems, simulation based trade
studies, identified system anomalies and developed appropriate
countermeasures, developed hardware math models, and refined
simulations.
Refined and applied simulations to assess dynamic system
performance under extreme non-linear disturbance conditions.
Refined and applied simulations to assess guidance system
performance and mission planning.
1979 Engineer
to Airflow Sciences Corporation, Plymouth, Michigan
1980
Assisted in the development of computational fluid dynamics (CFD)
software. Used CFD software to solve specific aerodynamic
problems. Assisted in wind tunnel test preparation.
TECHNICAL AND SCIENTIFIC PUBLICATIONS
Van Auken, R.M., Model Reduction with Generalized Input/Output Weighting, Engineer
Thesis. Stanford University, Stanford, CA, May, 1987.
Van Auken, M., Zellner, J.W., and Kunkel, D.T., "Correlation of Zwicker's Loudness and
Other Noise Metrics with Drivers' Over-the-Road Transient Noise Discomfort," Technical
Paper 980585, SAE, 1998.
Kebschull, S.A., Zellner, J.W., Van Auken, M., Rogers, N.M., "Injury Risk/Benefit
Analysis of Motorcycle Protective Devices Using Computer Simulation and ISO 13232",
Paper Number 98-S10-W-26, Proc. 16th International Technical Conference on the
82
-------
Enhanced Safety of Vehicles, Windsor, Canada, National Highway Traffic Safety
Administration, Washington D.C., 1998, pp. 2357-2374.
Zellner, J.W., Kebschull, Van Auken, R.M., "Analysis of Vehicle Tip Stability in Side
Impact Tests," Technical Paper 2000-01-1650, SAE, 2000.
Van Auken, R.M., Zellner, J.W., Kebschull, S.A., Wiley, K.D., Smith, T., Shewchenko, N,
Rogers, N.M., "Development of Neck Injury Assessment Criteria for the ISO 13232
Motorcyclist Anthropometric Test Dummy with the Revised Neck," Paper No. 417, Proc.
18th International Technical Conference on the Enhanced Safety of Vehicles, Nagoya,
Japan, May 2003.
Van Auken, R.M., and Zellner, J.W., "An Assessment of the Effects of Vehicle Weight
and Size on Fatality Risk in 1985 to 1998 Model Year Passenger Cars and 1985 to
1997 Model Year Light Trucks and Vans," Technical Paper 2005-01-1354, SAE
Transactions, Vol. 114, Parts, Detroit, April 11-15, 2005, pp 1607-1622.
Van Auken, R.M., Zellner, J.W., Smith, T., Rogers, N.M., "Development of an Improved
Neck Injury Assessment Criteria for the ISO 13232 Motorcyclist Anthropometric Test
Dummy," Paper No. 05-0227, Proc. 19th International Technical Conference on the
Enhanced Safety of Vehicles, 2005.
Van Auken, R.M., Kebschull, S.A., Broen, P.C., Zellner, J.W., Rogers, N.M.,
"Development of a Rider Size and Position Model to Determine Motorcycle Protective
Device Test Conditions," Paper Number 05-0392, Proc. 19th International Technical
Conference on the Enhanced Safety of Vehicles, 2005.
Suzuki, H., Sugimoto, Y., Van Auken, R.M., and Zellner, J.W., "Development of a
Prototype Safety Analysis System to Assess and Forecast Vehicle Safety," SAE
Technical Paper 2006-01-0718, 2006.
Van Auken, R.M., "Active Suspension Control Synthesis Using Reduced Order Models
that Account for the Time Delay Between Road Inputs," ESDA2006-95570, Proc. 8th
Biennial ASME Conference on Engineering Systems Design and Analysis, Torino, Italy,
July 4-7, 2006.
Zellner, J.W., Van Auken, R.M., Kebschull, S.A., Mun oz, S. Injury Risk-Benefit
Analysis of Rollover Protection Systems (ROPS) For All Terrain Vehicles (ATVs) Using
Computer Simulation, Full-Scale Testing and ISO 13232, Paper F2008-08-009, Fisita
World Congress, Munich, Germany, 2008.
Zellner, J.W., Van Auken, R.M., Chiang, D.P, Broen, P.C., Joseph Kelly, J.K., Sugimoto,
Y , "Extension of the Honda-DRI "Safety Impact Methodology" (SIM) for the NHTSA
Advanced Crash Avoidance Technology (ACAT) Program and Application to a
Prototype Advanced Collision Mitigation Braking System," Technical Paper 2009-01-
0781, SAE Int. J. Passenger. Cars-Mech. Syst. 2(1):875-894, 2009.
83
-------
Van Auken, R.M., "Development and Comparison of Laplace Domain and State-Space
Models of a Half-Car with Flexible Body," ESDA2010-24518, Proc. 10th Biennial
Conference on Engineering Systems Design and Analysis, Istanbul, Turkey, July 12-14,
2010.
Van Auken, R.M., Zellner, J.W., Silberling, J.Y., Kelly, J., Chiang, D.P., Broen, P.,
Kirsch, A., Sugimoto, Y., "Extension of the Honda-DRI Safety Impact Methodology for
the NHTSA Advanced Crash Avoidance Technology (ACAT) Program and Application
to the Evaluation of an Advanced Collision Mitigation Braking System - Final Results of
the ACAT-I Program," Technical Paper 2011-01-0581, SAE Int. J. Passenger Cars -
Mech. Syst. 4(1):488-508, 2011.
Van Auken, R.M., Smith, T.A., Zellner, J.W., "Development of a Probabilistic Skull
Fracture Model for a 50th Percentile Adult Male Motorcyclist ATD Headform," Paper
Number 11-0035, Proc. 21st International Technical Conference on the Enhanced
Safety of Vehicles, Washington, June 2011.
Van Auken, R.M., Zellner, J.W., Silberling, J.Y., Sugimoto, Y., Urai, Y., "Progress
Report on Evaluation of a Pre-Production Head-On Crash Avoidance Assist System
Using an Extended "Safety Impact Methodology" (SIM)," Paper Number 11-0270, Proc.
21st International Technical Conference on the Enhanced Safety of Vehicles,
Washington, June 2011.
Sugimoto, Y., Sugimoto, Y., Kawakami, K., Hashimoto, T., Zellner, J.W., Van Auken,
R.M., "Safety Impact Methodology for Advanced Crash Avoidance Technology (ACAT)
Program and Application to Advanced Collision Mitigation Braking System and Head-on
Crash Avoidance Assist System," First International Symposium on Future Active Safety
Technology Toward Zero-Traffic-Accident, JSAE, Tokyo, September 5-9, 2011.
Van Auken, R.M., Zellner, J.W., "Extension of the Honda-DRI "Safety Impact
Methodology" (SIM) for the NHTSA Advanced Crash Avoidance Technology (ACAT) II
Program and Application to the Evaluation of a Preproduction Head-On Crash
Avoidance Assist System," Technical Paper 2012-01-0291, SAE, forthcoming.
PUBLISHED TECHNICAL REPORTS
VanAuken, R. M. and Zellner, J. W., ATV ROPS Tests and Simulations. DRI-TR-98-2,
Dynamic Research, Inc., Torrance, California, October 1998.
Van Auken, R.M., and Zellner, J.W., An Assessment of the Effects of Vehicle Weight
and Size Parameters on Fatality Risk in Model Year 1985-98 Passenger Cars and
1985-97 Light Trucks. DRI-TR-02-02, Dynamic Research, Inc., Torrance, California,
DOT Docket No. NHTSA-2003-16318-2, February 2002.
84
-------
Van Auken, R.M., and Zellner, J.W., A Further Assessment of the Effects of Vehicle
Weight on Fatality Risk in Model Year 1985-98 Passenger Cars and 1985-97 Light
Trucks, DRI-TR-03-01, Dynamic Research, Inc., Torrance, California, DOT Docket No.
NHTSA-2003-16318-3, January 2003.
Van Auken, R.M., and Zellner, J.W., A Review of the Results in the 1997 Kahane. 2002
DRI. 2003 DRI. and 2003 Kahane Reports on the Effects of Passenger Car and Light
Truck Weight and Size on Fatality Risk, DRI-TR-04-02, Dynamic Research, Inc.,
Torrance, California, DOT Docket No. NHTSA-2003-16318-7, March 2004.
Kebschull, S.A., Kelly, J., Van Auken, R.M., and Zellner, J.W., An Analysis of the Effects
of SUV Weight and Length on SUV Crashworthiness and Compatibility Using Systems
Modeling and Risk-Benefit Analysis, DRI-TR-04-04-2, Dynamic Research, Inc.,
Torrance, California, DOT Docket No. NHTSA-2003-16128-1452, July 2004.
Van Auken, R.M., and Zellner, J.W., Supplemental Results on the Independent Effects
of Curb Weight. Wheelbase. and Track on Fatality Risk in 1985-1998 Model Year
Passenger Cars and 1985-1997 Model Year LTVs. DRI-TR-05-01, Dynamic Research,
Inc., Torrance, California, Federal Docket No. NHTSA-2003-16318-17, 20 May 2005.
Van Auken, R.M., and Zellner, J.W., DRI Comments On Safety Impacts of EPA-NHTSA
Proposed Rule To Establish Light-Duty Vehicle Greenhouse Gas Emission Standards
and Corporate Average Fuel Economy Standards Dated September 28 2009, DRI-TM-
09-86, Dynamic Research, Inc., Torrance, California, Federal Docket No. NHTSA-2009-
0059-112.1, 27 November 2009.
Van Auken, R.M., Zellner, J.W., Chiang, D.P., Kelly, J., Silberling, J.Y., Dai, R., Broen,
P.C., Kirsch, A.M., Sugimoto, Y., Advanced Crash Avoidance Technologies Program -
Final Report of the Honda-DRI Team, Volume I: Executive Summary and Technical
Report. DOT HS 811 454, Accomplished by Honda and DRI for the National Highway
Traffic Safety Administration, Washington, June, 2011.
Van Auken, R.M., and Zellner, J.W., Updated Analysis of the Effects of Passenger
Vehicle Size and Weight On Safety. Phase I: Updated Analysis Based on 1995 to 2000
Calendar Year Data for 1991 to 1999 Model Year Light Passenger Vehicles. DRI-TR-
11-01, Dynamic Research Inc., Torrance, California, Federal Docket No. NHTSA-2010-
0152-0030, January 2011.
Zellner, J.E., Van Auken, R.M., Kelly, J., Silberling, J.Y., Hagoski, B.K., Sugimoto, Y.,
Advanced Crash Avoidance Technologies (ACAT) II Program - Final Report of the
Honda-DRI Team, Accomplished by Honda and DRI for the National Highway Traffic
Safety Administration, Washington, D.C., forthcoming.
Van Auken, R.M., and Zellner, J.W., Updated Analysis of the Effects of Passenger
Vehicle Size and Weight On Safety. Phase II: Analysis Based on 2002 to 2008
Calendar Year Data for 2000 to 2007 Model Year Light Passenger Vehicles. DRI-TR-
12-01, Dynamic Research Inc., Torrance, California, forthcoming.
85
-------
PUBLIC PRESENTATIONS
Van Auken, R.M., and Zellner, J.W., "Effects of Vehicle Weight and Size Reductions on
Safety," P03-6853, Transportation Research Board 82nd Annual Meeting, Washington,
D.C., January 15, 2003.
Van Auken, R.M., and Erwin, S., "Injury Severity in Side Impact Mismatch," CIREN
Quarterly Meeting, Washington, D.C., April 3, 2003.
Van Auken, R.M., and Zellner, J.W. "Updated Analysis of the Effects of Passenger
Vehicle Size and Weight on Safety," NHTSA Workshop on Vehicle Mass-Size-Safety,
Washington, D.C., February 25, 2011.
PEER REVIEWS
Proc. 8th Biennial ASME Conference on Engineering Systems Design and Analysis,
Torino, Italy, July 4-7, 2006.
Proc. 10th Biennial Conference on Engineering Systems Design and Analysis, Istanbul,
Turkey, July 12-14, 2010.
86
-------
Appendix B: Conflict of Interest Statements
Conflict of Interest and Bias for Peer Review
Background
Identification and management of potential conflict of interest (COI) and bias issues are vital to
the successes and credibility of any peer review consisting of external experts. The
questionnaire that follows is consistent with EPA guidance concerning peer reviews.1
Definitions
Experts in a particular field will, in many cases, have existing opinions concerning the subject of
the peer review. These opinions may be considered bias, but are not necessarily conflicts of
interest.
Bias: For a peer review, means a predisposition towards the subject matter to be discussed that
could influence the candidate's viewpoint.
Examples of bias would be situations in which a candidate:
1. Has previously expressed a position on the subject(s) under consideration by the panel; or
2. Is affiliated with an industry, governmental, public interest, or other group which has
expressed a position concerning the subject(s) under consideration by the panel.
Conflict of Interest: For a peer review, as defined by the National Academy of Sciences,2
includes any of the following:
1. Affiliation with an organization with financial ties directly related to the outcome;
2. Direct personal/financial investments in the sponsoring organization or related to the
subject; or
3. Direct involvement in the documents submitted to the peer review panel... that could
impair the individual's objectivity or create an unfair competitive advantage for the
individual or organization.
1 U.S. EPA (2009). Science Policy Council Peer Review Handbook.
OMB (2004). Final Information Quality Bulletin for Peer Review.
NAS (2003). "Policy and Procedures on Committee Composition and Balance and Conflict or Interest for Committees Used in
the Development of Reports" (www.nationalacademies.org/coi).
87
-------
Policy and Process
• Candidates with COI, as defined above, will not be eligible for membership on those panels
where their conflicts apply.
• In general, candidates with bias, as defined above, on a particular issue will be eligible for all
panel memberships; however, extreme biases, such as those likely to impair a candidate's
ability to contribute to meaningful scientific discourse, will disqualify a candidate.
• Ideally, the composition of each panel will reflect a range of bias for a particular subject,
striving for balance.
• Candidates who meet scientific qualifications and other eligibility criteria will be asked to
provide written disclosure through a confidential questionnaire of all potential COI and bias
issues during the candidate identification and selection process.
• Candidates should be prepared, as necessary, to discuss potential COI and bias issues.
• All bias issues related to selected panelists will be disclosed in writing in the final peer
review record.
88
-------
Conflict of Interest and Bias Questionnaire
Peer Review of LBNL Reports on Vehicle Mass/Footprint Reduction & Safety
Instructions to Candidate Reviewers
1. Please check YES/NO/DON'T KNOW in response to each question.
2. If your answer is YES or DON'T KNOW, please provide a brief explanation of the
circumstances.
3. Please make a reasonable effort to answer accurately each question. For example, to the
extent a question applies to individuals (or entities) other than you (e.g., spouse,
dependents, or their employers), you should make a reasonable inquiry, such as emailing
the questions to such individuals/entities in an effort to obtain information necessary to
accurately answer the questions.
Questions
1. Are you (or your spouse/partner or dependents) or your current employer, an author,
contributor, or an earlier reviewer of the document(s) being reviewed by this panel?
YES NO X DON'T KNOW
2. Do you (or you spouse/partner or dependents) or your current employer have current
plans to conduct or seek work related to the subject of this peer review following the
completion of this peer review panel?
YES X NO DON'T KNOW
3. Do you (or your spouse/partner or dependents) or your current employer have any known
financial stake in the outcome of the review (e.g., investment interest in a business related
to the subject of peer review)?
YES NO X DON'T KNOW
4. Have you (or your spouse/partner or dependents) or your current employer commented,
reviewed, testified, published, made public statements, or taken positions regarding the
subject of this peer review?
YES NO DON'T KNOW X
89
-------
5. Do you hold personal values or beliefs that would preclude you from conducting an
objective, scientific evaluation of the subject of the review?
YES NO X DON'T KNOW
6. Do you know of any reason that you might be unable to provide impartial advice or
comments on the subject review of the panel?
YES NO X DON'T KNOW
7. Are you aware of any other factors that may create potential conflict of interest or bias
issues for you as a member of the panel?
YES NO X DON'T KNOW
Acknowledgment
I declare that the disclosed information is true and accurate to the best of my knowledge, and that
no real, potential, or apparent conflict of interest or bias is known to me except as disclosed. I
further declare that I have made reasonable effort and inquiry to obtain the information needed to
answer the questions truthfully, and accurately. I agree to inform SRA promptly of any change
in circumstances that would require me to revise the answers that I have provided.
Tong Donna Chen
Name
.- ^ ' 11/27/2011
Signature Date
90
-------
Conflict of Interest and Bias Questionnaire
Peer Review of LBNL Reports on Vehicle Mass/Footprint Reduction & Safety
Instructions to Candidate Reviewers
1. Please check YES/NO/DON'T KNOW in response to each question.
2. If your answer is YES or DON'T KNOW, please provide a brief explanation of the
circumstances.
3. Please make a reasonable effort to answer accurately each question. For example, to the
extent a question applies to individuals (or entities) other than you (e.g., spouse,
dependents, or their employers), you should make a reasonable inquiry, such as emailing
the questions to such individuals/entities in an effort to obtain information necessary to
accurately answer the questions.
Questions
1. Are you (or your spouse/partner or dependents) or your current employer, an author,
contributor, or an earlier reviewer of the document(s) being reviewed by this panel?
YES X NO DON'T KNOW
[I am a peer reviewer for the NHTSA study "Relationships Between Fatality Risk, Mass,
and Footprint in Model Year 200-2007 Passenger Cars andLTVs. "]
2. Do you (or you spouse/partner or dependents) or your current employer have current
plans to conduct or seek work related to the subject of this peer review following the
completion of this peer review panel?
YES X NO DON'T KNOW
[The Insurance Institute for Highway Safety has in the past and will continue to study the
relationship between vehicle size/weight and crash injury risk.]
3. Do you (or your spouse/partner or dependents) or your current employer have any known
financial stake in the outcome of the review (e.g., investment interest in a business related
to the subject of peer review)?
YES NO X DON'T KNOW
4. Have you (or your spouse/partner or dependents) or your current employer commented,
reviewed, testified, published, made public statements, or taken positions regarding the
subject of this peer review?
91
-------
YES X NO DON'T KNOW
[The Insurance Institute for Highway Safety has in the past and will continue to study the
relationship between vehicle size/weight and crash injury risk.]
5. Do you hold personal values or beliefs that would preclude you from conducting an
objective, scientific evaluation of the subject of the review?
YES NO X DON'T KNOW
6. Do you know of any reason that you might be unable to provide impartial advice or
comments on the subject review of the panel?
YES NO X DON'T KNOW
7. Are you aware of any other factors that may create potential conflict of interest or bias
issues for you as a member of the panel?
YES NO X DON'T KNOW
Acknowledgment
I declare that the disclosed information is true and accurate to the best of my knowledge, and that
no real, potential, or apparent conflict of interest or bias is known to me except as disclosed. I
further declare that I have made reasonable effort and inquiry to obtain the information needed to
answer the questions truthfully, and accurately. I agree to inform SRA promptly of any change
in circumstances that would require me to revise the answers that I have provided.
Charles M. Farmer, Ph.D.
Director of Statistical Services
Insurance Institute for Highway Safety
Arlington, VA
Name
l(\ October 24, 2011
Signature Date
92
-------
Conflict of Interest and Bias Questionnaire
Peer Review of LBNL Reports on Vehicle Mass/Footprint Reduction & Safety
Instructions to Candidate Reviewers
1. Please check YES/NO/DON'T KNOW in response to each question.
2. If your answer is YES or DON'T KNOW, please provide a brief explanation of the
circumstances.
3. Please make a reasonable effort to answer accurately each question. For example, to the
extent a question applies to individuals (or entities) other than you (e.g., spouse,
dependents, or their employers), you should make a reasonable inquiry, such as emailing
the questions to such individuals/entities in an effort to obtain information necessary to
accurately answer the questions.
Questions
1. Are you (or your spouse/partner or dependents) or your current employer, an author,
contributor, or an earlier reviewer of the document(s) being reviewed by this panel?
YES NO X DON'T KNOW
2. Do you (or you spouse/partner or dependents) or your current employer have current
plans to conduct or seek work related to the subject of this peer review following the
completion of this peer review panel?
YES NO X DON'T KNOW
3. Do you (or your spouse/partner or dependents) or your current employer have any known
financial stake in the outcome of the review (e.g., investment interest in a business related
to the subject of peer review)?
YES NO X DON'T KNOW
4. Have you (or your spouse/partner or dependents) or your current employer commented,
reviewed, testified, published, made public statements, or taken positions regarding the
subject of this peer review?
YES X NO DON'T KNOW
93
-------
5. Do you hold personal values or beliefs that would preclude you from conducting an
objective, scientific evaluation of the subject of the review?
YES NO X DON'T KNOW
6. Do you know of any reason that you might be unable to provide impartial advice or
comments on the subject review of the panel?
YES NO X DON'T KNOW
7. Are you aware of any other factors that may create potential conflict of interest or bias
issues for you as a member of the panel?
YES NO X DON'T KNOW
Acknowledgment
I declare that the disclosed information is true and accurate to the best of my knowledge, and that
no real, potential, or apparent conflict of interest or bias is known to me except as disclosed. I
further declare that I have made reasonable effort and inquiry to obtain the information needed to
answer the questions truthfully, and accurately. I agree to inform SRA promptly of any change
in circumstances that would require me to revise the answers that I have provided.
David L. Greene
Name
10/31/2011
Signature Date
94
-------
Conflict of Interest and Bias Questionnaire
Peer Review of LBNL Reports on Vehicle Mass/Footprint Reduction & Safety
Instructions to Candidate Reviewers
1. Please check YES/NO/DON'T KNOW in response to each question.
2. If your answer is YES or DON'T KNOW, please provide a brief explanation of the
circumstances.
3. Please make a reasonable effort to answer accurately each question. For example, to the
extent a question applies to individuals (or entities) other than you (e.g., spouse,
dependents, or their employers), you should make a reasonable inquiry, such as emailing
the questions to such individuals/entities in an effort to obtain information necessary to
accurately answer the questions.
Questions
1. Are you (or your spouse/partner or dependents) or your current employer, an author,
contributor, or an earlier reviewer of the document(s) being reviewed by this panel?
YES NO X DON'T KNOW
2. Do you (or you spouse/partner or dependents) or your current employer have current
plans to conduct or seek work related to the subject of this peer review following the
completion of this peer review panel?
YES NO X DON'T KNOW
3. Do you (or your spouse/partner or dependents) or your current employer have any known
financial stake in the outcome of the review (e.g., investment interest in a business related
to the subject of peer review)?
YES NO X DON'T KNOW
4. Have you (or your spouse/partner or dependents) or your current employer commented,
reviewed, testified, published, made public statements, or taken positions regarding the
subject of this peer review?
YES NO DON'T KNOW X
[I have a paper (to be presented at TRB) about crash safety, controlling for vehicle size,
fuel economy, etc.]
95
-------
5. Do you hold personal values or beliefs that would preclude you from conducting an
objective, scientific evaluation of the subject of the review?
YES NO X DON'T KNOW
6. Do you know of any reason that you might be unable to provide impartial advice or
comments on the subject review of the panel?
YES NO X DON'T KNOW
7. Are you aware of any other factors that may create potential conflict of interest or bias
issues for you as a member of the panel?
YES NO X DON'T KNOW
Acknowledgment
I declare that the disclosed information is true and accurate to the best of my knowledge, and that
no real, potential, or apparent conflict of interest or bias is known to me except as disclosed. I
further declare that I have made reasonable effort and inquiry to obtain the information needed to
answer the questions truthfully, and accurately. I agree to inform SRA promptly of any change
in circumstances that would require me to revise the answers that I have provided.
Kara Kockelman
Name
~~— 10/21/11
Signature *- Date
96
-------
Conflict of Interest and Bias Questionnaire
Peer Review of LBNL Reports on Vehicle Mass/Footprint Reduction & Safety
Instructions to Candidate Reviewers
1. Please check YES/NO/DON'T KNOW in response to each question.
2. If your answer is YES or DON'T KNOW, please provide a brief explanation of the
circumstances.
3. Please make a reasonable effort to answer accurately each question. For example, to the
extent a question applies to individuals (or entities) other than you (e.g., spouse,
dependents, or their employers), you should make a reasonable inquiry, such as emailing
the questions to such individuals/entities in an effort to obtain information necessary to
accurately answer the questions.
Questions
1. Are you (or your spouse/partner or dependents) or your current employer, an author,
contributor, or an earlier reviewer of the document(s) being reviewed by this panel?
YES NO X DON'T KNOW
2. Do you (or you spouse/partner or dependents) or your current employer have current
plans to conduct or seek work related to the subject of this peer review following the
completion of this peer review panel?
YES NO X DON'T KNOW
3. Do you (or your spouse/partner or dependents) or your current employer have any known
financial stake in the outcome of the review (e.g., investment interest in a business related
to the subject of peer review)?
YES NO X DON'T KNOW
4. Have you (or your spouse/partner or dependents) or your current employer commented,
reviewed, testified, published, made public statements, or taken positions regarding the
subject of this peer review?
YES NO X DON'T KNOW
97
-------
5. Do you hold personal values or beliefs that would preclude you from conducting an
objective, scientific evaluation of the subject of the review?
YES NO X DON'T KNOW
6. Do you know of any reason that you might be unable to provide impartial advice or
comments on the subject review of the panel?
YES NO X DON'T KNOW
7. Are you aware of any other factors that may create potential conflict of interest or bias
issues for you as a member of the panel?
YES NO X DON'T KNOW
Acknowledgment
I declare that the disclosed information is true and accurate to the best of my knowledge, and that
no real, potential, or apparent conflict of interest or bias is known to me except as disclosed. I
further declare that I have made reasonable effort and inquiry to obtain the information needed to
answer the questions truthfully, and accurately. I agree to inform SRA promptly of any change
in circumstances that would require me to revise the answers that I have provided.
R. Michael Van Auken
Name
2011-10-24
r —
Signature Date
98
-------
SRA
Honesty and Service
Appendix C: Peer Review Charge
Charge to Peer Reviewers of LBN L Statistical Analysis of the Effect of Vehicle Mass & Footprint
Reduction on Safety
In developing programs to reduce greenhouse gas (GHG) emissions and increase fuel economy of light-
duty highway vehicles, the U.S. Environmental Protection Agency (EPA) and the National Highway
Transportation Safety Administration (NHTSA) have to evaluate the safety of mass reduction
technologies likely to be used to meet future standards. The U.S. Department of Energy (DOE) has
contracted with Lawrence Berkeley National Laboratory (LBNL) to perform a statistical analysis of the
effect of vehicle mass and footprint reduction on safety. LBNL's analysis of the relationship between
vehicle mass, footprint, and societal fatality and casualty risk is comprised of two phases. Phase 1 is an
assessment of the NHTSA report Relationships between Fatality Risk, Mass, and Footprint in Model Year
2000-2007 Passenger Cars and LTVs. This study uses logistic regression analysis to estimate the
relationship of changes in vehicle mass and footprint on US fatality risk per vehicle mile traveled. Phase
2 is an independent logistic regression analysis to estimate the relationship between vehicle mass,
footprint and total casualty (fatality plus serious injury) risk, per police-reported crash, using state-level
data on all crashes.
The focus of this peer review is to evaluate the assumptions made, data and methods of statistical
analysis used, and conclusions from the analysis, for both the LBNL assessment of the NHTSA study and
the independent LBNL study using state-level data. A comprehensive peer review by third party experts
is an important step for validation of the results of the studies, and how the results of the studies are
used in modeling the effect of new fuel economy and greenhouse gas emission standards on vehicle
safety. In order to review the LBNL analysis, you should understand and be familiar with the data,
assumptions, conclusions, and statistical approach used in the NHTSA 2011 report "Relationship
between Fatality Risk, Mass, and Footprint in Model Year 2000-2007 Passenger Cars and LTVs." Also,
you may want to review the 2003 NHTSA report
http://www.nhtsa.gov/cars/rules/regrev/evaluate/pdf/809662.pdfand the 2010 NHTSA report
http://www.nhtsa.gov/staticfiles/rulemaking/pdf/cafe/CAFE 2012-2016 FRIA 04012010.pdf (beginning
page 464), depending upon your familiarity with them.
You are asked to review and provide expert comments on the LBNL Phase 1 and LBNL Phase 2 draft
reports described above. You are being provided LBNL's Phase 1 draft report and the NHTSA 2011
report, which it addresses. Work is being completed on the Phase 2 study, and a draft report is
expected to be available for review by November 22.
EPA is seeking peer reviewers' expert opinions on the statistic methodologies used in the two LBNL
studies and whether they are likely to yield realistic estimates of the relationship between vehicle mass,
footprint, and total fatality or casualty risk. EPA requests that each reviewer comment on all aspects of
the two LBNL studies, with particular emphasis on the methodologies employed, assumptions inherent
to the analysis, sources of information employed, methods of calculation and any other key issues the
reviewer may identify. Reviewers are encouraged to examine and evaluate the NHTSA study in helping
99
-------
them to understand the LBNL assessment analysis. Findings of this peer review may be used toward
validation and improvement of the statistical analysis conducted by LBNL, and to inform EPA staff on
potential use of the regression results for predicting the safety effect of future standards in reducing
mass and footprint. No independent data analysis will be required for this review.
Reviewers are asked to orient their comments toward these five general areas: (1) assumptions; (2)
control and dependent variables used in the regression models; (3) methodology and statistics; (4) data
sets; and (5) recommendations. Possible topics are provided in each area as illustrative examples.
Reviewers are expected to identify additional topics or depart from these examples as necessary to best
apply their particular set of expertise toward review of the LBNL reports.
Please note that the author intends to make the database and statistical programs available to the
public to ensure that the assumptions made and the methods of calculation are transparent and
replicable. Thus it will be helpful for reviewers to recommend improvements to the analysis that would
utilize publicly available information rather than those that would make use of proprietary information.
Comments should be sufficiently clear and detailed to allow readers to thoroughly understand their
relevance to the LBNL studies. Please deliver your final written comments to SRA International no
later than Thursday, December 22.
All materials provided to reviewers as well as reviewer comments should be treated as confidential, and
should neither be released nor discussed with others outside of the review panel. Once EPA, LBNL, and
NHTSA have made their reports and supporting documentation public, EPA will notify reviewers that
they may release or discuss the peer review materials and their review comments with others.
Should reviewers have questions about what is required in order to complete this review or need
additional background material, please contact Brian Menard at SRA (Brian Menard@sra.com) or (434-
817-4133). If a reviewer has any questions about the EPA peer review process itself, please contact Ms.
Ruth Schenk in EPA's Quality Office, National Vehicle and Fuel Emissions Laboratory
(schenk.ruth@epa.gov) or (734-214-4017).
100
-------
Appendix D: Reviews
Donna Chen & Kara Kockelman
Review of LBNL Phase 1 Report
1. ASSUMPTIONS
COMMENTS
Please comment on the validity of any assumptions embedded in the LBNL
assessment analysis and the independent casualty analysis that could affect
the projected relationship between vehicle mass/footprint reductions and
fatality/casualty risk. Examples might include assumptions regarding
whether recent historical relationships between vehicle weight, size, and
safety will continue into the future; potential future improvements in vehicle
technology and design may result in compensatory safety benefits; and the
annual baseline fatality distribution.
The report does a nice job discussing recent trends in vehicles, such as the
increase of ESC, side airbags, and light truck crash compatibility with
passenger cars - which will improve safety outcomes for all vehicles, but
perhaps most significantly the smaller and lighter vehicles. . It also mentions
the phasing out of the lightest and smallest vehicles between model years
2000-2007 (but doesn't mention the makes and models somehow), which
were particularly poor safety performers in the past. However, with the
introduction of urban commuter vehicles, such as the SmartCar, Mini Cooper,
and Fiat 500m, and the growing popularity of smaller, fuel-efficient compact
vehicles following gas price increases, this trend does not seem so obvious.
Such vehicles should be discussed.
The simplistic logistic model employed in this analysis only accounts for two
crash outcomes (fatal versus non-fatal) and so neglects the more detailed,
and ordered nature of injury severity data, which is unfortunate. The model
also assumes error-term homoscedasticity from one crash or individual to
the next; in reality certain vehicle types (e.g., pickups) and crash contexts
(e.g., high speed crashes) have more uncertainty associated with their
severity outcomes. It would be good to point out such limitations for readers.
Please comment on any apparent unstated or implicit assumptions and
related caveats or limitations.
The role of driver behavior is briefly addressed in the report but not
emphasized sufficiently. Fatality risk is a combination of driver, vehicle, and
roadway characteristics. Driver behavioral differences are many and do not
solely exist for pickup truck drivers versus car drivers. Socioeconomic data
such driver household income, size, and education influence driver attitudes
and driving environments. For example, Chen et al. (2010) found that crash
risk increases for those living in socioeconomically disadvantaged areas
(including households more likely to drive less expensive and older vehicles).
Though such data is not typically available in state and national crash
101
-------
databases, the importance of these driver and environmental characteristics
on crash rates (per mile driven) and fatality risk should be stressed in both
reports. It is clearly very difficult to control for, but a major caveat to the
NHTSA (& now LBNL) results. We expect that crash severity could be
probably be lowerfor many of the small cars and pickups if they were driven
by those who tend to drive more expensive vehicles, under the same settings
(e.g., daytime, urban freeway). Similarly, in the second LBNL report (which
uses VMT estimates), we expect that crash rates would probably be lower for
these types of driver-vehicle-setting combinations.
ADDITIONAL COMMENTS:
Chen, H.Y., Ivers, R.Q., Mariniuk, A.L.C., Boufous, S., Senserrick, T., Woodward, M., Stevenson, M. and Norton R. Socioeconomic status and risk of car crash
injury, independent of place of residence and driving exposure: Results from the DRIVE study. Journal of Epidemiology and Community Health 64(10), 2010,
pp. 998-1003.
102
-------
2. CONTROL AND DEPENDENT VARIABLES USED IN THE REGRESSION
MODELS
COMMENTS
Please comment on the adequacy of control and dependent variables used
in the assessment analysis and independent casualty analysis, and
recommend any alternative control or dependent variables that are available
for possible inclusion in the analysis. For example, what are the relative
merits of the main dependent variables used, fatality risk per estimated
VMT, and casualty risk per police-reported crash?
As alluded to above, a primary concern is that the NHTSA analysis (& thus the
LBNL analyses) largely neglect the idea that vehicle type (make & model) is
very much a proxy for driver type, and a vehicle's crash avoidance may have
very little to do with vehicle type. It has a lot to do with the person behind
the wheel, and gender & age simply aren't enough to control for such
distinctions. Education, risk aversion, ability, wealth, etc., are important
covariates. But existing data sets are quite limiting (though the MVOSS & FAR
with 3-year driver violation history do offer some valuable insights, not
discussed in these reports). In reality, small cars may be less crash prone than
Kahane's & Wenzel's results suggest, because they are driven by
lower-income, younger, less risk averse people driving in more crash prone
settings (e.g., commercial strips rather than pricey residential suburbs). Such
key caveats need thoughtful discussion. Four relevant papers on the topics of
crash frequency and vehicle size-and-weight implications (by Knipling, Kweon
& Kockelman, Wang and Kockelman, and Chen & Kockelman) have been sent
to Tom Wenzel. These all include useful literature reviews for further
connections to useful findings for citation in the reports, as time allows the
contractor.
The grouping of the vehicles into heavier- and lighter-than-average weight
categories essentially splits a "typical" weight vehicle of that type into two
categories. The impacts of curb weight and footprint on fatality risk may be
easier to interpret if the vehicles were grouped into 3 weight categories
(light, average, and heavy) & by type (with the average category representing
vehicles within one standard deviation of average weight). Furthermore, the
grouping of CUVs and minivans into the same vehicle type category neglects
the fact that these vehicles have faced rather different ground clearance
requirements (impacting rollover potential), door types (sliding vs. standard),
and, perhaps most importantly, can appeal to different types of drivers (as
indicated in the market shift of car drivers to CUV drivers).
103
-------
What additional control variables, such as vehicle make or model, might be
included in the regression models?
Vehicle height, a variable which may be more valuable than vehicle type for
similarly structured vehicles such as sedans, wagons, CUVs, and minivans,
would be a valuable control variable. In addition to a wider track, a lower
center of gravity also increases vehicle stability, thereby reducing the risk of
rollover. Relevant literature & findings exist, and should be cited.
Other variables which have been found in past studies to influence fatality
risk such as seat belt use, roadway geometry and division type are not
included in this study (which is largely a repeat of the NHTSA study, as
specifically contracted by the EPA).
To account for driver characteristics that contribute to fatality risk,
socioeconomic variables such as household income, education, household
size, etc. would be valuable additions. Unfortunately, both state and national
crash databases typically do not include such information (outside of
MVOSS). Such issues should be flagged for readers. It seems the contractor
has done his duty, and the key limitations lie with the original methodology
he was to essentially duplicate.
Please comment on any caveats or limitations that these dependent variable
or control variables entail with respect to use of the results as the basis for
estimating the safety effect of mass reduction.
Please see above comment (in Assumptions section) regarding driver
behavior and environment.
ADDITIONAL COMMENTS:
Table 2.1 has many indicator variables labeled as "C" for continuous variable (such as ABS, ESC, AWD, DRVMALE, etc). These C's should be removed.
104
-------
3. METHODOLOGY AND STATISTICS
COMMENTS
Please comment on the validity and applicability of the methodology LBNL
used in assessing the NHTSA 2011 study and its analysis of the relationship
between mass, footprint, and risks per police-reported crash.
The report assesses the NHTSA 2011 study in a fair amount of detail and
seeks to introduce some additional analyses to better examine the
relationship between mass, footprint, and fatality risks. However, due to a
lack of control for very specific vehicle differences (which vary by make &
sub-model), the exclusion of driver characteristics and crash setting details
(which cannot always be controlled for, but are often correlated with vehicle
type), the effects of downweighting vehicles and/or shifting vehicle styles
and sizes may be overestimated. Simply changing the vehicle on a risky driver
in a high-risk setting is unlikely to influence outcomes significantly.
Please review other statistical methods LBNL has used in the analysis, in
addition to the logistic regression methodology. Examples include the
alternative approaches used by LBNL to assess NHTSA interval estimation
results, and LBNL's linear regression analysis of actual, predicted, and
residual risk by vehicle model.
In the alternative measures of exposure, the author examines the effect of
vehicle manufacturer on fatality risk and treats the luxury models produced
by Toyota, Honda, and Nissan as separate manufacturers. However, domestic
luxury brands (such as Cadillac & Lincoln) are categorized with their
nameplate manufacturers (GM and Ford), which appears inconsistent.
The effect of calendar year variables on fatality risk may be overestimated
here, since VMT is tracked by vehicle model and not by calendar year. The
trend of greatest fatality risk reductions in light trucks, CUVs and minivans
with increasing calendar year may simply be a reflection of rising gas prices in
combination with the ailing economy contributing to lower VMT (in these
relatively low-fuel-economy vehicles).
It is unclear how the author determined the various percentage
replacements of vehicle types in the aggressive vehicle market share shift
scenario. (For example, why are 50% of SUVs replaced by CUVs and 60% of
small pickups replaced by CUVs? The CUV is a more natural replacement for
an SUV, and an SUV a more natural replacement for a pickup.)
105
-------
Please comment on caveats or limitations of using non-significant regression
estimates to project the safety impact of mass reduction.
First, the t-statistics are not provided in the report which makes it difficult for
the reader to assess statistical significance of specific regression estimates
(except where noted by the author). Second, inclusion of a statistically
insignificant variable can influence the estimates of coefficients associated
with related variables. Nevertheless, in general, it is best to keep insignificant
estimates if one has a strong defense for their role, since removing such
variables (& thus their parameters) will shift the burden of response to a
correlated covariate's parameter, thus biasing the latter. We generally keep
key covariates in a model up to a pvalue of 0.20 or 0.25 or so, especially in
relatively small data sets (e.g., n < 1,000). Covariates for which we have no
strong basis can be removed for pvalues > 0.10.
How might the LBNL methodology be strengthened to better represent
future vehicle designs and reduce multi-collinearity between mass and
footprint in the regression analysis?
Including more vehicle-specific characteristics (such as vehicle height and
engine size) reduces the analysis' dependence on vehicle type, since vehicle
shapes and structures will continue to evolve. There is also correlation with
context (e.g., pickups are driven in more rural locations, with greater hazards
[like less lighting, higher speed, & few medians]). Disaggregate data are
almost always best, to avoid ecological fallacies & such.
ADDITIONAL COMMENTS:
On page 55, it is unclear what is meant by "however; if anything, reduction of this type of fatality will increase detrimental effect of mass reduction in cars."
106
-------
4. DATA SETS
Please comment on the validity and applicability of the datasets used to
project changes in risk resulting from reduction in vehicle mass. LBNL's
casualty analysis used police-reported crash data from 16 states, while the
2011 NHTSA study used national fatality data, combined with a subset of
non-culpable vehicles involved in two-vehicle crashes from police-reported
crash data from 13 states.
Please comment on any apparent, unstated, or implicit impact on estimated
risks inherent in the two different approaches, and any related caveats or
limitations. For example, what are the strengths and weaknesses of the two
measures of vehicle exposure, miles of vehicle travelled scaled up from crash
data from 13 states, and number of police-reported crashes?
COMMENTS
The acquisition of Polk data for VMT estimates by make & model is valuable,
and a contribution to the literature. However, these estimates come from
vehicles found in repair shops in non-attainment areas, and so will be biased
towards problem-prone vehicles, wealthier households who service their
vehicles more regularly, and/or urban (smoggier) areas. Such issues merit
careful discussion in the paper, so that readers are well aware of caveats.
Related to this, Tom Wenzel indicated (by phone) that he did take a look at
CA's extensive odometer reads, which go into some semi-rural locations (not
too rural), and he indicated that the VMT values by vehicle type (not
controlling for HH attributes & such) are very similar (just 5% longer in rural
areas) - except for vans (which are used much more extensively in rural
areas). This is interesting to me, and is not that different from what we've
seen in the past. For example, Kockelman & Zhao's JTS paper from 2000 (pre-
print at http://www.ee. utexas.edu/prof/kockelman/public_html/
BTSJournalLDTs.pdf) suggests that, after controlling for various HH attributes
& vehicle types, density is/was still very important (tables 1 & 2), but a shift
from a density of Ik to 5k persons per sq mile (which is 1.5 vs. 7.8 persons
per acre) means an increase of 750 mi/yr/vehicle (which is about 7.5% of
annual VMT). Such differences, and their practical significance (or lack
thereof) should be discussed in the reports.
The use of non-culpable vehicles in two-vehicle crashes as a proxy for
vehicles which are "just there" may be distorting the overall distribution of
vehicle models. VMT may differ between vehicles that are more prone to
run-off-road accidents, at-fault two-car crash vehicles, and non-culpable
vehicles.
ADDITIONAL COMMENTS:
107
-------
5. RECOMMENDATIONS
COMMENTS
Please comment on whether the LBNL assessment adequately addresses the
NHTSA 2011 study and identifies the safety impact from mass reduction. Are
the analytic methods and data used to assess the NHTSA study, and estimate
the relationship between risk, mass, and footprint, appropriate? Is casualty
risk per crash a legitimate measure of vehicle safety? What other methods
or data could be used to better predict the effect of future vehicle designs
on safety?
While driver fatalities per crash seems a useful measure of vehicle design
safety, and examination of fatal crash rates is very valuable (using Polk-
based exposure estimates), there are many caveats to work of this type. As
noted above: a primary concern remains a neglect of the notion that the type
of car is very much a proxy for driver type, and a vehicle's crash avoidance
may have very little to do with vehicle type. It has a lot to do with the person
behind the wheel. Simply including gender and age variables cannot account
for important covariates such as education, risk aversion, driving ability,
wealth, etc. In reality, small cars may be less crash prone than Kahane's and
Wenzel's results suggest, because they are driven by lower-income, younger,
less risk averse people driving in more crash prone settings (e.g., commercial
strips rather than pricey residential suburbs). Of course, as noted above, it is
very difficult to control for all these variables, and the contractor was asked
to rely on the original data. In reality, the best the report authors can do with
such data sets is to explain how all the other, relevant attributes may factor
in (e.g., quality of driver and typical driving settings), and how they can
generate biased estimation (sometimes in either direction). Discussion of
relevant literature that looks more deeply at crash outcomes (e.g., Wang or
Chen's papers, mentioned above, allowing for heteroscedasticity and
individual vehicle attributes, non-driver outcomes, etc.) will also be useful.
Please comment on the overall adequacy of LBNL's assessment of the 2011
NHTSA report and its independent study of casualty risk for predicting the
effect of vehicle mass or footprint reduction on safety. Provide any
recommended improvements that might reasonably be adopted by the
author to improve the analysis.
Overall, the study is a comprehensive assessment of the 2011 NHTSA report
and introduces interesting additional analyses to examine the relationship of
vehicle mass and footprint reduction on safety. However, as stated
previously in the comments here, driver preference for specific car types
(including size and mass) is related to driver socioeconomic characteristics
and driving behavior. As vehicle, driver, and roadway environment
characteristics all contribute to fatality risk, the effects of physical vehicle
changes such as mass or footprint reduction on safety should not be
overstated when the other two types of characteristics are not sufficiently
accounted for.
ADDITIONAL COMMENTS:
Donna Chen & Kara Kockelman
108
-------
Review of LBNL Phase 2 Report
1. ASSUMPTIONS
COMMENTS
Please comment on the validity of any assumptions embedded in the LBNL
assessment analysis and the independent casualty analysis that could affect
the projected relationship between vehicle mass/footprint reductions and
fatality/casualty risk. Examples might include assumptions regarding
whether recent historical relationships between vehicle weight, size, and
safety will continue into the future; potential future improvements in vehicle
technology and design may result in compensatory safety benefits; and the
annual baseline fatality distribution.
The Phase 2 report serves as a complimentary document to the Phase 1
report by isolating the effect of vehicle mass and footprint on
crashworthiness. Whereas the Phase 1 report analyzes fatality risk per
estimated VMT, the Phase 2 report analyzes casualty risk per crash. The
parallel structure of the two reports makes it easy for the reader to compare
the results of the two analyses.
The binary logistic model employed in this analysis can only account for two
injury outcome categories; here it is used to distinguish crashes resulting in
serious injury or death from all other crash outcomes. Thus, the model does
not account for the ordinal nature of injury severity and neglects the
difference between a serious injury and a death.
The report states that "a serious incapacitating injury can be just as traumatic
to the victim and her family, and costly from an economic perspective, as a
fatality." While serious injuries are very costly to society (and may have
similar economic cost implications as deadly crashes), willingness to-pay
estimates (which include pain and suffering) price the cost of a fatality at
almost 20 times the cost of an incapacitating injury (NSC 2010). Thus, it is
difficult to assess the economic cost of the estimates of increases in casualty
risk per crash without distinguishing whether that outcome is a serious injury
or a death. This limitation of the model should be addressed in the report.
The logistic model also assumes error-term homoscedasticity and cannot
account for increases and decreases in the variation of injury outcomes due
to vehicle and driver type, for example. Such limitations of the model should
be discussed.
Please comment on any apparent unstated or implicit assumptions and
related caveats or limitations.
The role of driver behavior is briefly addressed in the report but not
emphasized sufficiently. Casualty risk is a combination of driver, vehicle, and
roadway characteristics. Whereas vehicle characteristics significantly
influence crashworthiness, driver behavioral differences play a significant, if
not primary, role in determining crash frequency. Socioeconomic data such
109
-------
driver household income, size, and education influence driver attitudes and
driving environments. For example, Chen et al. (2010) found that crash risk
increases for those living in socioeconomically disadvantaged areas (including
households more likely to drive less expensive and older vehicles). Though
such data is not typically available in state and national crash databases, the
importance of these driver and environmental characteristics on crash rates
(per mile driven) and casualty risk should be stressed in both reports. It is
clearly very difficult to control for, but is a major caveat to the NHTSA (& now
LBNL) results. We expect that crash severity could be probably be lower for
many of the small cars and pickups if they were driven by those who tend to
drive more expensive vehicles, under the same settings (e.g., daytime, urban
freeway). Thus, statements like "a 100-lb reduction in the mass of lighter cars
leads to a 1.84% increase in crash frequency" should be accompanied by an
explanation of the possibility of the mass variable accounting/proxying for
effects of lower income households owning smaller vehicles.
ADDITIONAL COMMENTS:
National Safety Council (2010) Estimating the Costs of Unintentional Injuries. Available online at
http://www.nsc.org/news_resources/injury_and_death_statistics/Pages/EstimatingtheCostsofUnintentionallnjuries
Chen, H.Y., Ivers, R.Q., Mariniuk, A.L.C., Boufous, S., Senserrick, T., Woodward, M., Stevenson, M. and Norton R. Socioeconomic status and risk of car crash
injury, independent of place of residence and driving exposure: Results from the DRIVE study. Journal of Epidemiology and Community Health 64(10), 2010,
pp. 998-1003.
110
-------
2. CONTROL AND DEPENDENT VARIABLES USED IN THE REGRESSION
MODELS
COMMENTS
Please comment on the adequacy of control and dependent variables used
in the assessment analysis and independent casualty analysis, and
recommend any alternative control or dependent variables that are available
for possible inclusion in the analysis. For example, what are the relative
merits of the main dependent variables used, fatality risk per estimated
VMT, and casualty risk per police-reported crash?
As alluded to above, a primary concern is that the NHTSA analysis (& thus the
LBNL analyses) largely neglect the idea that vehicle type (make & model) is
very much a proxy for driver type, and a vehicle's crash frequency may have
very little to do with physical vehicle characteristics. It has a lot to do with
the person behind the wheel, and gender and age simply aren't enough to
control for such distinctions. Education, risk aversion, ability, wealth, etc., are
important covariates. But existing data sets are quite limiting (though the
MVOSS & FAR with 3-year driver violation history do offer some valuable
insights, not discussed in these reports). In reality, small cars may be less
crash prone than Kahane's & Wenzel's results suggest, because they are
driven by lower-income, younger, less risk averse people driving in more
crash prone settings (e.g., commercial strips rather than pricey residential
suburbs). Such key caveats need thoughtful discussion. Four relevant papers
on the topics of crash frequency and vehicle size-and-weight implications (by
Knipling, Kweon & Kockelman, Wang and Kockelman, and Chen &
Kockelman) have been sent to Tom Wenzel. These all include useful
literature reviews for further connections to useful findings for citation in the
reports, as time allows the contractor.
Independent variables such as vehicle mass and footprint may be accounting
for effects of driver socioeconomic factors as discussed in the Assumptions
section. Furthermore, vehicle option variables such as AWD and side curtain
airbags may be reflecting the effects of driver environment (e.g., those living
in areas with icy winters opting for AWD) and attitude (e.g., more risk-averse
drivers opting for side curtain airbags) rather than the vehicle technology
themselves. While extremely heavy and extremely large vehicles may have
significantly different handling and braking characteristics which influence
crash frequency and casualty risk, it is unlikely that given the same driver in
the same environment, a small change in vehicle mass or footprint would
influence the driver's crash proneness.
Ill
-------
What additional control variables, such as vehicle make or model, might be
included in the regression models?
Vehicle height, a variable which may be more valuable than vehicle type for
similarly structured vehicles such as sedans, wagons, CUVs, and minivans,
would be a valuable control variable. In addition to a wider track, a lower
center of gravity also increases vehicle stability, thereby reducing the risk of
rollover. Relevant literature & findings exist, and should be cited.
Other variables which have been found in past studies to influence fatality
risk such as seat belt use, roadway geometry and division type are not
included in this study.
To account for driver characteristics that contribute to casualty risk,
socioeconomic variables such as household income, education, household
size, etc. would be valuable additions. Unfortunately, both state and national
crash databases typically do not include such information (outside of
MVOSS). Such issues should be flagged for readers.
Please comment on any caveats or limitations that these dependent variable
or control variables entail with respect to use of the results as the basis for
estimating the safety effect of mass reduction.
Please see above comment (in Assumptions section) regarding driver
behavior and environment.
ADDITIONAL COMMENTS:
Table 2.1 has many indicator variables labeled as "C" for continuous variable (such as ABS, ESC, AWD, DRVMALE, etc). These C's should be removed.
112
-------
3. METHODOLOGY AND STATISTICS
COMMENTS
Please comment on the validity and applicability of the methodology LBNL
used in assessing the NHTSA 2011 study and its analysis of the relationship
between mass, footprint, and risks per police-reported crash.
The Phase 2 report enhances the findings in the Phase 1 report by isolating
the effect of vehicle mass and footprint on crashworthiness. Like the Phase 1
report, this analysis goes into a fair amount of detail and seeks to introduce
additional analyses to better examine the relationship between mass,
footprint, and casualty risks. However, due to a lack of control for very
specific vehicle differences (which vary by make & sub-model), the exclusion
of driver characteristics and crash setting details (which cannot always be
controlled for, but are often correlated with vehicle type), the effects of
downweighting vehicles and/or shifting vehicle styles and sizes may be
overestimated. Simply changing the vehicle mass or footprint on a risky
driver in a high-risk setting is unlikely to influence crash outcomes
significantly.
Please review other statistical methods LBNL has used in the analysis, in
addition to the logistic regression methodology. Examples include the
alternative approaches used by LBNLto assess NHTSA interval estimation
results, and LBNL's linear regression analysis of actual, predicted, and
residual risk by vehicle model.
In the alternative measures of exposure, the author examines the effect of
vehicle manufacturer on fatality risk and treats the luxury models produced
by Toyota, Honda, and Nissan as separate manufacturers. However, domestic
luxury brands (such as Cadillac & Lincoln) are categorized with their
nameplate manufacturers (GM and Ford), which appears inconsistent.
Please comment on caveats or limitations of using non-significant regression
estimates to project the safety impact of mass reduction.
First, the t-statistics are not provided in the report which makes it difficult for
the reader to assess statistical significance of specific regression estimates
(except where noted by the author). Second, inclusion of a statistically
insignificant variable can influence the estimates of coefficients associated
with related variables. Nevertheless, in general, it is best to keep insignificant
estimates if one has a strong defense for their role, since removing such
variables (& thus their parameters) will shift the burden of response to a
correlated covariate's parameter, thus biasing the latter. We generally keep
key covariates in a model up to a pvalue of 0.20 or 0.25 or so, especially in
relatively small data sets (e.g., n < 1,000). Covariates for which we have no
strong basis can be removed for pvalues > 0.10.
How might the LBNL methodology be strengthened to better represent
future vehicle designs and reduce multi-collinearity between mass and
footprint in the regression analysis?
Including more vehicle-specific characteristics (such as vehicle height and
engine size) reduces the analysis' dependence on vehicle type, since vehicle
shapes and structures will continue to evolve. There is also correlation with
context (e.g., pickups are driven in more rural locations, with greater hazards
[like less lighting, higher speed, & few medians]). Disaggregate data are
113
-------
almost always best, to avoid ecological fallacies & such.
ADDITIONAL COMMENTS:
114
-------
4. DATA SETS
COMMENTS
Please comment on the validity and applicability of the datasets used to
project changes in risk resulting from reduction in vehicle mass. LBNL's
casualty analysis used police-reported crash data from 16 states, while the
2011 NHTSA study used national fatality data, combined with a subset of
non-culpable vehicles involved in two-vehicle crashes from police-reported
crash data from 13 states.
The Phase 2 report uses an unusually extensive data set of police-reported
crash data from 13 states which the author compares in detail to national
data sets to illustrate similarities and differences. The author is very
thorough in addressing the difference in definitions of "serious" and
"incapacitating" injuries across different states and the effects of such
inconsistency on the regression results.
Since casualty risk in the report accounts for serious injuries but not minor
injuries, the author should note that police-reported injury levels may also be
poor indicators of the actual or Modified Abbreviated Injury Scale (MAIS)
level, following medical evaluation. Farmer (2003) found that 41% of injuries
reported by U.S. police as incapacitating received MAIS ratings of "minor
injury" by health care professionals using NASS Crashworthiness Data System
(CDS). Thus, the results of the estimated casualty risk increases and
decreases rely heavily on the assumption that police errors in reporting
actual MAIS ratings are consistent across states.
Please comment on any apparent, unstated, or implicit impact on estimated
risks inherent in the two different approaches, and any related caveats or
limitations. For example, what are the strengths and weaknesses of the two
measures of vehicle exposure, miles of vehicle travelled scaled up from crash
data from 13 states, and number of police-reported crashes?
The Phase 1 analysis used non-culpable vehicles in two-vehicle crashes as a
proxy for induced exposure crashes. In contrast, Phase 2 analysis uses data
from vehicles involved in one-car crashes and the responsible vehicle in two-
car crashes. The exclusion of the not-at-fault vehicle in two-car crashes may
be distorting the distribution of crash frequency and casualty risk across
different vehicle makes and models if crash-prone drivers are more likely to
drive certain types of vehicles.
ADDITIONAL COMMENTS:
Farmer, C.M. Reliability of police-reported information for determining crash and injury severity. Traffic Injury Prevention 4(1), 2003, pp.38-44.
115
-------
5. RECOMMENDATIONS
COMMENTS
Please comment on whether the LBNL assessment adequately addresses the
NHTSA 2011 study and identifies the safety impact from mass reduction. Are
the analytic methods and data used to assess the NHTSA study, and estimate
the relationship between risk, mass, and footprint, appropriate? Is casualty
risk per crash a legitimate measure of vehicle safety? What other methods
or data could be used to better predict the effect of future vehicle designs
on safety?
As noted above, a primary concern remains a neglect of the notion that the
type of car is very much a proxy for driver type, and a vehicle's crash
avoidance may have very little to do with vehicle type. It has a lot to do with
the person behind the wheel. Simply including gender and age variables
cannot account for important covariates such as education, risk aversion,
driving ability, wealth, etc. In reality, small cars may be less crash prone than
Kahane's and Wenzel's results suggest, because they are driven by
lower-income, younger, less risk averse people driving in more crash prone
settings (e.g., commercial strips rather than pricey residential suburbs). Alas,
it is very difficult to control for all these variables, since they are not readily
available in data sets. In reality, the best the report authors can do with such
data sets is to explain how all the other, relevant attributes may factor in
(e.g., quality of driver and typical driving settings), and how they can
generate biased estimation (sometimes in either direction). Discussion of
relevant literature that looks more deeply at crash outcomes (e.g., Wang or
Chen's papers, mentioned above, allowing for heteroscedasticity and
individual vehicle attributes, non-driver outcomes, etc.) will also be useful.
Please comment on the overall adequacy of LBNL's assessment of the 2011
NHTSA report and its independent study of casualty risk for predicting the
effect of vehicle mass or footprint reduction on safety. Provide any
recommended improvements that might reasonably be adopted by the
author to improve the analysis.
Overall, the study is an enriching complementary document to the Phase 1
assessment of the 2011 NHTSA report. The parallel structure of the two
reports allows the reader to easily compare and contrast the various
additional analyses which examine the relationship of vehicle mass and
footprint reduction on safety. However, as stated previously in the
comments here, driver preference for specific car types (including size and
mass) is related to driver socioeconomic characteristics and driving behavior.
As vehicle, driver, and roadway environment characteristics all contribute to
fatality risk, the effects of physical vehicle changes such as mass or footprint
reduction on safety should not be overstated when the other two types of
characteristics are not sufficiently accounted for.
ADDITIONAL COMMENTS:
116
-------
Charles Farmer
Review of LBNL Phase 1 and 2 Reports
1. ASSUMPTIONS
COMMENTS
Please comment on the validity of any assumptions embedded in the LBNL
assessment analysis and the independent casualty analysis that could affect
the projected relationship between vehicle mass/footprint reductions and
fatality/casualty risk. Examples might include assumptions regarding whether
recent historical relationships between vehicle weight, size, and safety will
continue into the future; potential future improvements in vehicle technology
and design may result in compensatory safety benefits; and the annual
baseline fatality distribution.
The report concludes "that much of the detrimental effect of mass or footprint
reduction on risk can be attributed to the tendency for mass or footprint
reduction to increase crash frequency, rather than to reduce vehicle
crashworthiness (risk once a crash has occurred)." However, the
interpretation of casualties per crash as inversely proportional to
crashworthiness ignores the possibility that injury severity also depends upon
the circumstances of the crash. Casualties per crash must be divided into
casualties per severe crash and severe crashes per crash, where a severe crash
would be one involving more energy, e.g., high-speed or rollover. It could be
that weight reduction increases casualties per severe crash (i.e., reduces
crashworthiness), but reduces the likelihood that a crash is severe.
Please comment on any apparent unstated or implicit assumptions and related
caveats or limitations.
The statistical models assume no interaction between the vehicle size/weight
measures and any of the numerous covariates, but this may not be true. For
example, size/weight reductions may differently affect vehicles with and
without ESC if they affect vehicle handling. It is risky to make statements such
as that on p. 11 of the Phase I report:
Therefore, the mass of a lighter car could be reduced by 800 Ibs while adding
ESC, without increasing fatality risk.
ADDITIONAL COMMENTS:
NHTSA's fatality analysis covered calendar years 2001-08, but the casualty analysis excludes 2008. Such exclusion is understandable given that 2008 data were
at the time unavailable for a majority of the states (I think they are available now). However, 2008 was an unusual year and may have affected the size and
weight effect estimates. The footnote on p. 6 of the Phase II report states that an analysis including the available 2008 data will be summarized in Appendix A.
I don't see Appendix A. Is an analysis planned including 2008 data?
117
-------
2. CONTROL AND DEPENDENT VARIABLES USED IN THE REGRESSION
MODELS
COMMENTS
Please comment on the adequacy of control and dependent variables used in
the assessment analysis and independent casualty analysis, and recommend
any alternative control or dependent variables that are available for possible
inclusion in the analysis. For example, what are the relative merits of the main
dependent variables used, fatality risk per estimated VMT, and casualty risk
per police-reported crash?
One needs to restrict control variables to those that are available and reliable.
A problem when combining state databases is that the states often are not
consistent as to the variables coded and the definitions of those variables.
This severely limits the list of possible control variables.
What additional control variables, such as vehicle make or model, might be
included in the regression models?
I think that already there are too many control variables in the regression
models. Instead I would consider defining different classifications of crash
types. Table 2.2 of the Phase II report shows that the distribution of crash
types for casualty crashes is very different from that for fatal crashes.
Please comment on any caveats or limitations that these dependent variable
or control variables entail with respect to use of the results as the basis for
estimating the safety effect of mass reduction.
Model overspecification could be the reason for results that are non-intuitive,
especially in the Phase II analyses of police-reported crashes. Control variables
may be correlated with each other or with the size and weight variables. For
example, Figure 2.9 of the Phase I report implies that torso side airbags
increase fatality risk in CUVs.
ADDITIONAL COMMENTS:
The sensitivity results of Chapter 5 (Phase II) point out the extreme differences in results when changing the control variables. For example, including vehicle
make changes the effect of a 100-lb reduction in heavier cars from -0.91% to +0.55% (see Fig 5.3).
118
-------
3. METHODOLOGY AND STATISTICS
COMMENTS
Please comment on the validity and applicability of the methodology LBNL
used in assessing the NHTSA 2011 study and its analysis of the relationship
between mass, footprint, and risks per police-reported crash.
Figure 2.11 of the Phase II report implies that NHTSA's fitting of a separate
regression model for each of the 9 crash types was unnecessary, at least for
the analysis of casualty risk per crash. I don't recall seeing a similar analysis
for fatality risk per VMT. Is it possible to get essentially the same results as
the NHTSA study using a single regression model?
Please review other statistical methods LBNL has used in the analysis, in
addition to the logistic regression methodology. Examples include the
alternative approaches used by LBNL to assess NHTSA interval estimation
results, and LBNL's linear regression analysis of actual, predicted, and residual
risk by vehicle model.
The graphs in LBNL's analysis of risk by vehicle model seem to indicate
different trends for light and heavy vehicles (e.g., Figure 4.1 of Phase I).
However, only simple linear relationships are examined, unlike the NHTSA
analyses, which modeled piecewise linear relationships. Also, it's not clear
why the 3rd and last rows of Table 4.1 list different numbers (both are labeled
CUVs/minivans), and some entries in Table 4.1 disagree with Figures 4.6-4.11.
Please comment on caveats or limitations of using non-significant regression
estimates to project the safety impact of mass reduction.
Making projections from non-significant regression estimates is proper so long
as the resulting confidence intervals are constructed conservatively (to
account for the accumulated imprecision). In that sense, I prefer NHTSA's
jackknife approach to the standard errors produced by SAS (see p. 13 of Phase
How might the LBNL methodology be strengthened to better represent future
vehicle designs and reduce multi-collinearity between mass and footprint in
the regression analysis?
ADDITIONAL COMMENTS:
119
-------
4. DATA SETS
COMMENTS
Please comment on the validity and applicability of the datasets used to
project changes in risk resulting from reduction in vehicle mass. LBNL's
casualty analysis used police-reported crash data from 16 states, while the
2011 NHTSA study used national fatality data, combined with a subset of non-
culpable vehicles involved in two-vehicle crashes from police-reported crash
data from 13 states.
A major limitation of the Phase II analysis is a bias that may be due to the
patterns of missing data. In particular, the vehicle identification number (VIN)
is missing or mistyped for many crash records. High-severity crashes
(especially fatal) are more likely to have detailed police investigation, so VINs
(and other variables) in these crashes may be more complete. State crash files
are therefore much less reliable than PARS.
Please comment on any apparent, unstated, or implicit impact on estimated
risks inherent in the two different approaches, and any related caveats or
limitations. For example, what are the strengths and weaknesses of the two
measures of vehicle exposure, miles of vehicle traveled scaled up from crash
data from 13 states, and number of police-reported crashes?
The VMT weights provided by NHTSA were scaled to represent the entire US.
Comments on pp. 9 and 18 of the Phase II report seem to acknowledge this
deficiency, promising to adjust these to the 13 states in the future. Was any
adjustment made, such as multiplying the weights by the proportion of annual
US VMT accounted for by each of these states? The accuracy of the VMT
weights is critical is we are to believe the somewhat surprising results
concerning crashes per VMT.
ADDITIONAL COMMENTS:
Statements above Figure 2.7 in the Phase II report imply that the effects of weight reduction on crashes per VMT and fatalities per crash should add up to the
effect on fatalities per VMT. This is not the case. For example, a 1.43% increase in crashes per VMT and a 0.76% decrease in fatalities per crash would imply a
2.16% decrease in fatalities per VMT (i.e., 1 - 0.9924/1.0143). The fact that the model on fatalities per VMT yields an estimated 1.08% increase should be a
cause for concern. Either the VMT weights are inaccurate or the control variables have different effects on crash frequency and crashworthiness.
120
-------
5. RECOMMENDATIONS
COMMENTS
Please comment on whether the LBNL assessment adequately addresses the
NHTSA 2011 study and identifies the safety impact from mass reduction. Are
the analytic methods and data used to assess the NHTSA study, and estimate
the relationship between risk, mass, and footprint, appropriate? Is casualty
risk per crash a legitimate measure of vehicle safety? What other methods or
data could be used to better predict the effect of future vehicle designs on
safety?
Casualty risk per crash does not fully measure the effects of vehicle size and
weight reductions on society. Casualty risk per VMT best coincides with the
NHTSA analysis of fatalities per VMT. The breakdown of casualty risk per VMT
into the crash frequency and crashworthiness components is of interest.
However, the surprising results reported here make everything suspect. For
example, the Phase II report concludes that "the detrimental effect of male
drivers has to do with their higher tendency of getting into a serious crash
rather than their sensitivity to injury once a serious crash has occurred" (p.
24). A few pages later it concludes that "male drivers have essentially no
effect on crash frequency, but cause a statistically significant increase in
fatality risk once a crash occurs" (p. 28).
Please comment on the overall adequacy of LBNL's assessment of the 2011
NHTSA report and its independent study of casualty risk for predicting the
effect of vehicle mass or footprint reduction on safety. Provide any
recommended improvements that might reasonably be adopted by the author
to improve the analysis.
Overall these are reasonably good studies. The Phase I report does a very
good job of assessing the NHTSA report of fatality risk. However, the Phase I
report should be more cautious in its conclusions concerning casualty risk.
The casualty analysis is based solely on police-reported data from 13 states,
which:
1. May not be representative of the US as a whole.
2. Are inconsistent in the information given and the way in which it is
coded.
3. Suffer from information that is missing, inaccurate, or unclear.
ADDITIONAL COMMENTS:
Column G of Table 6.1 in the Phase II report provides the most appropriate comparison to results from the NHTSA report (Column A). For both fatalities and
casualties per VMT, a 100-lb weight reduction is most harmful in lighter cars, less harmful in heavier cars and lighter light trucks, and slightly beneficial in
heavier light trucks, minivans, and crossovers.
121
-------
Review of "An Analysis of the Relationship between Casualty Risk per Crash and Vehicle Mass and
Footprint for Model Year 2000-2007 Light-Duty Vehicles"
David L Greene
January 10, 2012
Summary
The Phase I and Phase II analyses by Tom Wenzel of LBNL have been executed diligently and consistently
in accord with the methods and data used in the original NHTSA analysis. The studies contain many
valuable, new insights. The phase I study highlights the weakening relationship between vehicle mass
and highway fatalities. This is not only seen in decreasing coefficient estimates but in the very large
number of results that are not statistically significant. When regressions were done separately by
footprint deciles, vehicle mass was statistically significantly positively related to fatalities only for light-
duty trucks in rollovers, there were almost as many cases in which mass was negatively related to
fatalities (9 vs. 13 out of 27) and there were more instances of statistically significant negative
relationships than positive relationships. Given that so many tests are being jointly conducted, it is quite
possible that when joint probabilities are considered, there is no significant relationship between mass
and fatalities (more on joint probabilities later). Showing the weakness and inconsistency of these
results is an important contribution.
Another meaningful contribution of the phase I study, and one that deserves more emphasis, is a logical
inference from the following findings: 1) much of the variance in risk remains unexplained even by the
most complete models, 2) control variables explain 1 to 2 orders of magnitude more of the variance
than the variables of interest (mass and size), 3) when key control variables are removed or changed it
strongly influences the coefficients of mass and size. These results have very important implications for
the robustness of the results and the likelihood that some or all of the apparently statistically significant
relationships are due to spurious correlations with omitted or imperfectly controlled factors. Noting
that exposure measures are control variables with constrained coefficients, the following observation
from the phase I study is especially perceptive.
"Calculating risk as total fatalities per induced exposure crash, rather than per vehicle mile
traveled, reverses the sign of mass reductions on risk in cars and the lighter light trucks, with mass
reduction leading to a reduction in risk in all vehicle types."
Finally, the phase I report notes that if only the control variables are included in the regression and
not size or mass, the resulting residuals from the regression are uncorrelated with size or mass.
Given these findings (as well as those of phase II) the conclusions that,
"The 2011 NHTSA study, and this report, conclude that the effect of mass reduction on US fatality
risk is small."
122
-------
should be revised with the following emendation, "... and probably non-existent."
Both studies, like the NHTSA analysis, have shortcomings in terms of interpreting the results and the
language used to describe the results, and acknowledging the limitations of the data and methodology.
The limitations are extensive. The interpretation of the results of the LBNL studies commits two
important, related errors. The first is to attribute inferred coefficients of mass and size as representing
only the effects of vehicle mass and size when, as the phase I and II study results indicate, there is a
virtual certainty that aliasing effects are present due to a combination of omitted variables, errors in
variables and correlations among variables. Given that estimated driver and environmental factors
tend to have 1-2 orders of magnitude larger impacts on safety outcomes than vehicle factors, the almost
certain presence of aliasing effects must be explicitly acknowledged as severely limiting the ability to
draw inferences about the effects of vehicle attributes. Second, the language used in interpreting
results fails to acknowledge that the analysis does not address the effects of down-weighting or down-
sizing specific vehicles or vehicle designs, but instead relies on correlations between vehicle weight and
size in existing vehicle designs. In existing vehicles, weight and size are correlated with each other and
many other vehicle attributes (and driver and environmental attributes, as well). Thus, the study is not
actually measuring the effects of down-weighting via the material substitution and design changes likely
to occur as a consequence of fuel economy and emissions standards. An early example of the kind of
misleading language referred to here can be found on page iv.
"For example, a 100-lb reduction in the mass of lighter cars leads to a 1.84% increase in crash
frequency (columns B), while mass reduction leads to a 0.76% decrease in the number of fatalities
per crash (column C);"
This statement is misleading in that it implies causality rather than correlation, and it is additionally
misleading in that it implies that the inference applies to removing weight from specific vehicles.
Neither is correct. A better statement would be the following.
"For example, vehicles in the lighter class that are 100 Ibs. lighter are correlated with a 1.84%
increase in crash frequency...."
There are so many examples of this misleading language that it is not feasible to list them all. All
should be corrected, however. Failure to correct them could lead to serious misinterpretation of the
studies' findings.
Following in the footsteps of the seminal study by DRI, the NHTSA and LBNL studies contribute to the
literature in three important ways: 1) the LBNL and NHTSA studies recognize that the societal safety
perspective is the correct perspective to when assessing the impacts of fuel economy and emissions
regulations, 2) they recognize that vehicle dimensions and vehicle mass may have separate and
potentially different impacts on both the likelihood of a crash and the outcomes of the crash and, 3) the
LBNL phase two analysis makes an additional contribution by attempting to disentangle factors affecting
the likelihood of a crash and factors affecting the outcomes of a crash.
123
-------
Speaking of the DRI study, I am puzzled about why there are no references cited in the phase I study and
only a handful all by Kahane and Wenzel, in the phase II study. This is perhaps due to the scope of work
defined for the two studies but there are highly relevant studies in the literature that could have been
cited, those by DRI foremost among them. Making use of the insights from these studies would have
been helpful in interpreting the results of both phase I and phase II.
Lack of a Theory or Model of the Phenomenon
Both the NHTSA and LBNL studies lack a rigorous theory of the process by which down-weighting at
constant size or down-sizing at constant mass affect societal safety either through crash avoidance or
crashworthiness. This is not a trivial shortcoming because it affects the ability to formulate hypotheses
and interpret results. Prior to the dissenting report on safety of the NRC 2002 CAFE report, the physics
of elastic collisions between objects was typically cited as the underlying physical model. That report
showed how taking the societal perspective renders that model inappropriate. What remains appears
to be far more complex, involving the quantity of kinetic energy, the ability of vehicle designs to absorb
that energy so as to minimize maximum deceleration rates, stability, maneuverability, safety
technologies, and more.
The consequences of the lack of a rigorous theory are that it is not known, a priori, what the signs of
coefficients are expected to be, let alone what their quantitative relationships should be. Hypotheses
must be formulated based on intuition and the interpretation of results is likewise ad hoc. One
implication of this is that results that suggest that lower fatalities are associated with lower vehicle mass
have equal standing, a priori, with results that indicate that higher fatalities are associated with lower
vehicle mass, and similarly for vehicle size. There are no surprising or unsurprising results, in theory.
This also makes it difficult to develop a plan for statistically testing the model or theory and its
implications. It would have been helpful to the reader to have been presented early on in the report
with such a plan of analysis.
On the Virtual Certainty of Aliasing
The LBNL report typically attributes causal effects to correlations between mass or size and safety. In
fact, most or all of the observed correlations are almost certainly affected by aliasing effects. There is
ample evidence for this inference in the results presented in the LBNL phase II report.
The coefficients of mass and size change in important ways when different model formulations are
estimated. Removing and adding control variables changes the magnitudes and sometimes the signs of
the mass and size variables. This means that, at a minimum, the mass and size variables alias the effects
of the omitted control variables. The question is whether the aliasing is eliminated entirely by the
inclusion of the control variables available or whether some aliasing remains either because not all
relevant and correlated control variables have been included or because the included control variables
are imperfect measures of the factors they are intended to represent.
124
-------
The latter seems highly likely for the following reasons. First, the overall explanatory power of the full
models (including control variables), as measured by their R2 is low. Most of the variance in casualties
and fatalities remains unexplained. Second, at least some of the important included control variables
are only crude approximations of the factors they are intended to represent. For example, dummy
variables represent differences in state reporting practices, age and gender represent risky driving
behavior differences among owners of different sizes and masses of vehicles, the presence or absence
of a kind of safety equipment represents both its performance and use in a particular vehicle, and
calendar year dummy variables represent unknown factors associated with the respective calendar
years. Such practices are common and their use is appropriate. Third, the control variables generally
account for 1-2 orders of magnitude more variance in the casualty and fatality variables than do vehicle
weight and size. To recap, the amount of unexplained variability in the dependent variables is larger
relative to the variance statistically explained by the most complete models. Control variables are
correlated with size and mass, and they account for 1-2 orders of magnitude more variability in the
dependent variables than the variables of interest, mass and size. Therefore, even small correlations of
size and mass with omitted variables or with errors (imprecision) of the control variables could easily
result in biased estimates for the effects of size and mass on the dependent variables.
Tom Wenzel is to be commended for providing the results that definitively demonstrate the three key
points made above. The above is not a criticism of the analysis nor of the results, per se. It is a criticism
of their interpretation. In light of the above, the results should be interpreted in light of the virtual
certainty that many of the estimated coefficients are likely to be biased in ways that make their
interpretation highly uncertain. The implication is that phrases such as "down-weighting or down-sizing
caused" to "mass (or size) and unobserved correlated factors are associated with..."
On Joint Probabilities
The NHTSA and LBNL studies do not correctly interpret their results as joint statistical tests. When
testing a hypothesis on, for example, 5 vehicle classes simultaneously, a result for one equation that
might be statistically significant on its own may not be statistically significant as one of five related tests.
Statistical analyses comprised of multiple regressions too often overlook the fact that tests of statistical
significance designed for individual regressions may not apply in the case of multiple regressions. That is
the case here. NHTSA conducted 5 analyses to infer relationships between mass differences among
vehicles holding footprint constant for 5 classes of cars. The results showed one relationship out of five
was statistically significant. As table 1 illustrates, using a simple example, if one conducts 5 trials, each
with a 0.05 probability of given result, there is a 22.6% probability of finding at least one such result in
the five trials. Thus, the joint significance level of the overall result (1 statistically significant regression
out of 5) is 0.226, rather than 0.05.
125
-------
Table 1. Simplified Illustration of the Joint Probability of Inferences in Multiple Regressions
# Significant
Regressions Combinations
0 1
1 5
2 10
3 10
4 5
5 1
0.95
0.773780938
0.81450625
0.857375
0.9025
0.95
1
0.05
1
0.05
0.0025
0.000125
0.00000625
3.125E-07
Joint
Probability
77.37809%
20.36266%
2.14344%
0.11281%
0.00297%
0.00003%
100.0%
22.6%
So there is between a 1:4 and a 1:5 chance of getting one statistically significant result by pure chance.
In fact, the actual significance level of the results is more complicated to calculate, and probably a bit
smaller than 0.226. Thus, it is very appropriate for Dr. Kahane to add the qualifier "if any" to his
conclusions about the relationship between the societal highway fatalities and mass reduction, holding
footprint constant. Had appropriate tests of joint statistical significance been used to evaluate the
results in the NHTSA and LBNL studies, the significance levels very likely would not meet accepted
criteria for statistical significance. This could change the conclusions of the studies from the inference
that mass is correlated with fatalities or casualties in some case but not others to the lack of statistically
significant evidence that mass is correlated with fatalities or injuries on the highway. This is an
important difference.
Page-by-Page Comments
I will make page by page comments on the phase II study only, since that contains the overwhelming
share of original contributions and the key findings of the phase I study are recapitulated there.
p. iii Paragraph 3. This would be a very good place to acknowledge the importance of driver behavior
and environment on crash avoidance especially.
p. iv Para. 3. This would be a good place to discuss probability inference in joint tests.
Para 4. The statement about a 100-lb reduction in the mass, etc., is a good example of
misleading language.
p. v Para. 1. Again, it is misleading to say that mass reduction increases crash frequency, for reasons
stated above.
Para. 2. It is more accurate to describe the association of lower vehicle mass with casualty risk
than the "effect of mass reduction on..." casualty risk.
Para. 3. Would benefit greatly from joint probability inferences.
126
-------
Para. 4. As noted above, this shows how much more important the control variables are than
the variables of interest.
Para 5. Again, these are correlations not necessarily effects.
p. vi Para. 1. Again, mass reduction is misleading terminology and you do not know if it increases
casualty risk or not, you know only a correlation. Why is this so important? It is the virtual
certainty of spurious correlations, or aliasing, as noted above.
Para. 4. (1st bullet) This is clear evidence of aliasing. Take variables out of the regression and the
coefficients of interest change in important ways. Are there no important factors still missing?
Are the variables included perfect measures of the factors of interest? Of course not. Thus,
there must be remaining aliasing. How bad is it? We don't know.
The third bullet shows the same effect with a different set of variables.
p. vii Para. 4. No, your analysis does not indicate "...that much of the detrimental effect of mass or
footprint reduction on risk can be attributed to the tendency for mass or footprint reduction to
increase crash frequency." Again, you have correlation, not causation and you have good
reason to believe that what you are seeing is affected by spurious correlations.
Para. 5. The "effect" is small, 1-2 orders of magnitude smaller than correlations with other
control variables, and IS strongly affected by which variables are in the equation, as stated on
the previous page, and there is a great deal of unexplained variance. Please reconsider the
meaning of these results in light of the comments above.
Finally, as the last paragraph of the ES implies, it would be far better not to speak in terms of
"reducing" mass or size. That is not what is happening in your data set.
p. 1 Para. 4. Risk per VMT includes the effects of how well vehicles are driven as well as how well
they can be driven. I think there is no chance that you have fully accounted for how well
vehicles are driven.
p. 2 Para. 2. Exposure measures are explanatory variables whose elasticity is constrained to 1. That
is, it is assumed that an increase in vehicle use of 1 vehicle mile produces a 1 unit increase in the
chance of a fatality (or casualty as the case may be). This is actually a maintained hypothesis. If
this hypothesis is incorrect, it can bias the other coefficients in the equation. Thus, the change
from fatalities/VMT to fatalities/registration-year, to fatalities per crash not only changes the
meaning of the analysis, it also may bias coefficients in the event that the true relationship
between fatalities and VMT is not an elasticity of 1.
p. 3 Line 1. Please acknowledge that your "accounting for differences in driver characteristics, crash
locations, and other vehicle attributes" is incomplete and that this could affect your inferences
about size and mass.
p. 3 Para. 2. NHTSA's use of "non-culpable" vehicles involved in two vehicle crashes as an exposure
measure raises its own issues. How non-culpable was the non-culpable vehicle. Often this is a
matter of degree, rather than black or white. Driver behavior may also be involved. It seems to
127
-------
me this is just another potential measure of exposure that may or may not be better than any
other measure and may introduce new sources of bias in the analysis.
p. 3 Para. 4. Induced exposure needs to be defined. What is it intended to mean? This needs to be
explicit.
Also, I am startled that there are no equations in these reports. Equations can provide an
unequivocal explanation of the assumed relationships that cannot be adequately accomplished
by words, in many cases. Why no equations?
p. 7 Para 3. CUVs and minivans are involved in fewer crashes with stationary objects than cars.
Why? Is it the drivers, the vehicles, or the passengers? How well can you control for such
differences? Not well. What does this mean for your analysis?
p. 9 Para 1. Here an equation showing how the weighting was done would be very helpful.
p. 9 Bullet 1. Excluding these vehicle types implies that the control variables in the model are not
adequate to account for whatever makes these vehicle types different from the vehicles
included in the analysis. First, this is an admission that the model is not adequate to explain the
fatalities associated with these vehicles. Second, it is an admission that if they were included
the coefficients on the variables of interest would likely be biased by spurious correlations.
Clearly, it would not even be sufficient to include the vehicles along with a control variable (e.g.,
X = 1 if vehicle is a police car, 0 otherwise). This is yet another indication that the model suffers
generally from omitted variables, errors in variables and correlation among right-hand side
variables.
p. 10 Line 3. Sentence does not make sense. Please correct.
p. 12 Sect. 2.3. Please provide an equation.
p. 13 Were the confidence intervals calculated using ex-l? Please state explicitly or, better, show an
equation.
Para. 2. Here another instance where you say "mass reduction increases societal fatality risk"
but you really are not entitled to say that. It is misleading. Also, the NHTSA CI's are larger, as
they should be in a joint test.
Para. 3. These results require an underlying theory for interpretation. The lack of one makes it
seem like there is just no consistency in the results.
Para. 4. The fact that the results for fatalities per crash differ substantially from fatality per VMT
may be very important. Taken at face value, it would imply that any negative effect of reduced
mass is due to its effect on crash avoidance (crash probability) rather than crashworthiness.
This is where the lack of a theory is most troubling. Why would that be? Are lighter vehicles
less easily controlled, etc.? Or, as seems much more likely, is there a spurious correlation
between mass and other omitted or imperfectly measured factors (including driver behavior)
that lead to an increased probability of a crash? Consider, for example, driver age. Driver age is
related to crash involvement. Driver age is a control variable. But are all young drivers the
128
-------
same? Is it possible that young drivers more prone to risky behavior tend to drive lighter
vehicles? If so, this could partly explain the result observed. Of course, this is just speculative,
but the point is that correlations with imperfectly measured and omitted factors are highly likely
to be present in the data and, if there, could easily affect the statistical inferences.
p. 14 Here we see that changing the exposure measure influence the effect of mass and size on
fatalities and casualties, which is more evidence that spurious correlations are likely to be
biasing estimated coefficients for mass and size.
The bar graphs with confidence intervals are well done and convey a great deal of information
effectively. The patterns of magnitude and statistical significance are difficult to interpret, partly
because there is no explicit theory of what should happen and partly, perhaps, because the
relationships are actually not real.
p. 16 Para. 1. Reduction in the mass of lighter cars increases crash frequency but reduces fatalities
per crash. This is contradictory to the previously maintained theory that mass protects due to
the physics of velocity changes in elastic collisions. Indeed, there is no theoretical explanation
for these results, only speculation.
p. 18 Para. 1. Here is a good example of such casual speculation.
p. 19 Para. 1. Developing VMT weights for the 13 states is a good idea, given the effect of exposure
measures on inferences. Still the results would not be definitive.
p. 20 Para. 2. Mass reduction leads to a large reduction in risk only in crashes with objects for heavier
cars? There is only one type of crash in which the simple physics of collisions leads to an
unambiguous benefit for increased mass, and that is collisions with moveable or breakable
objects. This finding contradicts even that.
Para. 3. More speculation, this time about rollovers.
p. 21 Para. 3. "Curiously,..." Curiouser and curiouser.
Figure 2.15 printed without labels. Could be my computer but the other graphs were fine.
p. 24 Para. 2. This is probably a very important finding that needs further investigation and
explanation. As figure 2.16 illustrates well, the correlations with mass and size are orders of
magnitude smaller than the correlations with driver and environmental factors. This is why
even small correlations with omitted or imperfectly measured control factors could be, are even
likely to be, predominantly responsible for the estimated coefficients of mass and size.
p. 25 Para. 2. The results for minivans discussed here could be due to what is going on inside the
vehicle as much or more than the vehicle itself. How could these results be explained in terms
of the vehicle itself. Figure 2.17 shows this again. The effects of calendar year dummies, which
can only be considered rough approximations to unknown and various time-related factors,
have much large effects than size or mass. Again in figure 2.20.
129
-------
p. 28 Para. 1. "Surprisingly,..." How can side airbags, which deploy only in a crash, reduce crash
frequency but not fatality risk in a crash? Only if the real effect is a reflection of who buys a
CUV/minivan with side airbags and how and where they drive. There are more surprising
inferences in paragraph 2 about male and female drivers. Surprising relative to what theory?
p. 32 The problem here is not numerical multicollinearity (numerical difficulties inverting the cross-
product matrix) but the more complex problem of correlations among right-hand side variables,
omitted variables, errors in variables, and correlations of included variables with omitted and
imperfectly measured variables. This leads to biased estimates.
p. 36 Table 3.1 cries out for inference based on joint probabilities. What is the probability of
observing "success" in at least 3 out of 27 trials when the true probability of success is only 0.05.
See discussion above. The probability is certainly much higher than 0.05 and probably closer to
0.5. The implication is that, taken together, these results do not show any statistically
significant relationship between mass or size and risk per crash. If there were a rigorous
underlying theory, the interpretation might be different (patterns of significance could matter)
but there is none. Again, good graphs on succeeding pages.
p. 41 The statistical significance of such a relationship should be the same whether bins are used or
not. Is it?
p. 42 Para. 3. R-squared is not the correct measure of statistical significance. Is the coefficient of
weight significantly different from zero?
p. 43 Para. 2. This is perhaps the key finding of the phase I and II analyses. Control variables explain
1-2 orders of magnitude more variance than size or mass. Still, most of the variance remains
unexplained and is uncorrelated with mass or size. It is very likely there is nothing going on
here.
p. 47 Para. 2. More evidence of correlation of mass and size with control variables and how changing
definitions or excluding control variables results in important changes in the coefficient of mass
and size. Such results are considered unstable.
p. 49 Para. 1. The rationale for the grouping of manufacturers is not obvious. Can you explain it?
Para. 2. Yet more evidence for the instability of the model and likelihood that variables still
missing from the model, plus errors in measuring the included control variables are likely biasing
the inferences. The results described in this paragraph do not make sense to me. How can they
be explained other than random results?
p. 50 Para. 1. More casual interpretation of results. OK, maybe, maybe not. Same for paragraph 2.
The economy faltered in 2008 but the big negative effect was in 2007. The downward trend
started in 2004. This correlates neither with vehicle mass changes over time nor economic
growth as measured by real GDP. Idle speculation.
p. 51 Para. 1. Good discussion of the gratuitous speculation by NHTSA about the meaning of the
observed correlations. This is more a Rorschach test than statistical analysis.
130
-------
p. 51 Para. 2. "We have no explanation for why..." More of this kind of honest appraisal is needed in
studies like these.
p. 52 More results for which there is no explanation.
p. 53 Why the interaction between calendar year dummy variables and safety equipment? The
presence of the safety equipment on a particular vehicle is established. What has calendar year
(not model year) to do with it? Again, one suspects spurious correlations.
p. 55 One needs to think carefully about the reasons why vehicles would be excluded. It does not
appear that NHTSA did that. First line of first paragraph "used" rather than "sused".
p. 57 Well reasoned. It is interesting that NHTSA resisted including footprint or size in previous
analyses on the grounds of correlation with mass. These results show that assertion was
groundless.
p. 59 Para. 3. Rather than say risk per VMT accounts for two effects, it is better to say it includes or
comprises two effects. But this statement also ignores the important influences of drivers and
environment and their potential correlations with other factors. Yes, it includes how well a
vehicle can be driven, but more importantly it includes how well a vehicle IS driven. That is in
there too and is very likely to be correlated with make, model, and other vehicle attributes.
p. 60 Para. 3. Here again, the conclusions are misstated. It is not a genuine "reduction" in mass, but
an association with the mass of vehicles. And how does it "lead" to and increase in crash
frequency? What is the theory or model that predicts this? Driver and environment are very
likely mixed up in these results to an unknown but likely substantial degree. So what can we
really conclude? Not this.
As I read the conclusions and inferences I find myself asking, why?, why?, repeatedly without
any sound explanations. Page 61, paragraph 3 contains more "surprising" results. Surprising
because they are contrary to theory? Surprising because they are contrary to intuition?
Surprising because they are random? To what can we attribute so many "surprising" results,
and how many must there be before one concludes that the analysis is not revealing what we
had hoped it would.
Para. 4. Again, this cries out for joint statistical inference. Three statistically significant results
out of 27 is probably nothing statistically significant at all.
p. 62 Para. 1. What this shows, again, is that the coefficients of mass and size are strongly influenced
by which control variables are included in the model and how they are defined. These results
and their implications need explanation. The bottom line is that the effects of mass and size are
likely to be (after the necessary joint significance calculations are done) not statistically
significant, not consistent, and not robust.
p. 63 How do mass and footprint reduction (again, it's not really reduction in the sense of designing
lighter vehicles to increase fuel economy or reduce GHG emissions, the issue at hand) increase
crash frequency. What is the theory? I don't find a theory in either the NHTSA report or the
phase I and phase II studies. Absent a theory, these results seem sufficiently unstable and
131
-------
inconsistent to be highly questionable as evidence of any relationship between mass or size and
crashworthiness or crash avoidance. I think joint estimation of significance levels would provide
additional support for this view.
132
-------
Michael Van Auken
Review of LBNL Phase 1 Report
1. ASSUMPTIONS
COMMENTS
Please comment on the validity of any assumptions embedded in the
LBNL assessment analysis and the independent casualty analysis that
could affect the projected relationship between vehicle mass/footprint
reductions and fatality/casualty risk. Examples might include
assumptions regarding whether recent historical relationships between
vehicle weight, size, and safety will continue into the future; potential
future improvements in vehicle technology and design may result in
compensatory safety benefits; and the annual baseline fatality
distribution.
The basic assumptions, methodology, and data are primarily the same
as in the Kahane (2011) report. These include the following:
1) The probability of a crash fatality is proportional to the vehicle miles
travelled (VMT), except as noted in Section 5.1
2) The logarithm of probability of fatality per VMT for a given curb
weight, footprint, and control variable values varies as a linear
combination of the curb weight, footprint, and control variables
within the domain of the data.
3) The logistic regression methods determine a maximum likelihood
estimate of model coefficients.
4) It is assumed that the above relationships remain constant in the
recent past (i.e., 2000-2007 model year vehicles in the 2002-2008
calendar years), present, and near future (i.e., 2017-2025 model
year vehicles).
The first assumption that crash fatalities are proportional to VMT
rather than the number of vehicle registration years (VRY) is
appropriate because the fatalities cannot occur if the vehicles are not
driven on the road (i.e., VMT = 0). This assumption is qualified
however because VMT is more difficult to measure than VRY and
therefore may be less accurate. On the other hand the probability of a
fatal crash or the number of fatalities in a crash may also depend on
the vehicle occupancy. The analysis in Section 5.1 is a commendable
attempt to explore the sensitivity to this assumption, however the
Kahane (1997) and DRI (2003-2005) reports have shown that some
driver, vehicle, and environmental factors may be underrepresented or
overrepresented in unweighted induced-exposure data. VRY could
have also been considered as a measure of exposure.
133
-------
The second and third assumptions are appropriate provided that it is
recognized that it is essentially impossible with currently available
knowledge and information to model all of the factors that could affect
the probability of fatality in a crash, and that the objective of the
analysis is to identify overall trends versus vehicle weight and
footprint. In general the probability of fatality depends on other many
other factors which have not been modelled (e.g., driver behavior
factors, vehicle design factors, roadway design factors, EMTfactors)
and these unmodeled factors are assumed to be uncorrelated with
vehicle weight and footprint, and/or are represented by the other
control variables. The latter assumption might or might not be valid.
The fourth assumption is perhaps the weakest because it assumes that
future vehicles will have the same design characteristics as past
vehicles, and that the characteristics of the vehicle population (e.g.,
collision partner weight, size, type) will also remain the same. A
commendable attempt to partially address this effect is described in
Section 6. These effects can be perhaps better addressed by the
"Volpe model" described in Kahane (2011) of the Honda-DRI fleet
systems model described in Refs (2324), which can be used to forecast
the effects of mass reductions of individual makes and models on a
year-by-year basis.
Please comment on any apparent unstated or implicit assumptions and
related caveats or limitations.
The induced-exposure data set provided by NHTSA is based on the
"non-culpable" vehicle in two-vehicle crashes. It is assumed that the
dataset is a reprehensive sample of the driver and environmental
exposure factors for vehicle use. However, since these cases include
moving vehicles, some vehicle-driver-environmental conditions may be
under or over represented in this data depending on how they affect
the ability of a non-culpable vehicle to avoid a crash. Results in Ref (17)
indicate that the estimated effect of weight and size reduction are
sensitive to whether the induced-exposure data are based on the
Kahane (2003) non-culpable vehicle definition of the Kahane (1996)
stopped vehicle definition.
134
-------
Unfortunately it is not currently possible to test this sensitivity with the
NHTSA-provided induced-exposure data.
ADDITIONAL COMMENTS:
None come to mind.
135
-------
2. CONTROL AND DEPENDENT VARIABLES USED IN THE REGRESSION
MODELS
COMMENTS
Please comment on the adequacy of control and dependent variables
used in the assessment analysis and independent casualty analysis, and
recommend any alternative control or dependent variables that are
available for possible inclusion in the analysis. For example, what are
the relative merits of the main dependent variables used, fatality risk
per estimated VMT, and casualty risk per police-reported crash?
The main metric used in both the Kahane (2011) and Wenzel (2011a)
reports is the total number of fatalities (except as noted). Reducing the
total number of fatalities, which includes both subject vehicle
occupants and collision partner fatalities, is desirable from a societal
viewpoint. Fatal crash occurrence is related to the total number of
fatalities, which has been used by Kahane (2003, 2011) to address
concerns about double counting.
VMT is a good measure of accident exposure provided that it can be
accurately determined.
[Note: This peer review does not address the Wenzel (2011b)
companion report (Ref 3) which examines the risks per police-reported
crash. See Ref 4 for comments on the companion report.]
What additional control variables, such as vehicle make or model,
might be included in the regression models?
None come to mind.
Please comment on any caveats or limitations that these dependent
variable or control variables entail with respect to use of the results as
the basis for estimating the safety effect of mass reduction.
None come to mind.
ADDITIONAL COMMENTS:
The underlying reasons for some of the estimated effects are unknown at this time, but presumably involve driver, vehicles, environment or
accident factors that have not been controlled for in the Kahane (2011) and Wenzel analyses. See, for example, Refs 17 and 25.
136
-------
3. METHODOLOGY AND STATISTICS
COMMENTS
Please comment on the validity and applicability of the methodology
LBNL used in assessing the NHTSA 2011 study and its analysis of the
relationship between mass, footprint, and risks per police-reported
crash.
The logistic regression methods seem to be appropriate. The
confidence intervals are based on the logistic regression Wald Chi-
Square statistic, which as Kahane (2003, 2011) has demonstrated does
not include all sources of variation. However, these confidence
intervals are useful because they do provide some indication of the
uncertainty in the results.
[Note: This peer review does not address the Wenzel (2011b)
companion report, which examines the risks per police-reported crash.
See Ref 4 for comments on the companion report.]
Please review other statistical methods LBNL has used in the analysis,
in addition to the logistic regression methodology. Examples include
the alternative approaches used by LBNL to assess NHTSA interval
estimation results, and LBNL's linear regression analysis of actual,
predicted, and residual risk by vehicle model.
The correlations in Section 3 appear to be assessed using the
Coefficient of Multiple Determination (R2) based on a linear fit to the
data (e.g., the correlation between footprint versus curb weight in
Figure 3.1 on p. 14). The linear regression model attributes the
differences between the dependent variable (vertical axis) and the
linear fit to the independent variable (horizontal axis) to random
effects. If there is no preference as to the choice of independent and
dependent variables (e.g., footprint versus curb weight, or curb weight
versus footprint), then the linear trend and R2 result would be different
if the two variables were interchanged, and having two different yet
equally valid results would be undesirable.
If the variation in the data can be attributed to both variables (e.g.,
footprint and curb weight), then it would be better to report the
square of the sample correlation coefficient r2, where r is computed
according to Eqn (1). The trend lines in these correlation figures should
not be computed using a linear regression. Instead, the trend line
should pass through the sample means (i.e., (x, y)), and have a slope
equal to the ratio of the sample standard deviations in the data (i.e., sy
/sx). Therefore, the reported correlation results do not depend on the
ordering of the data variables.
137
-------
Note this comments does not apply to linear trends indicated in
Section 4, for which the Coefficient of Multiple Determination (R2)
seems appropriate.
The Coefficient of Multiple Determination (R2) is frequently used in the
Wenzel (2011a) report as an indicator of the statistical importance of a
linear trend (e.g., R2 values in Tables 4.1 and 4.2 that are greater than
0.3 are shown in blue font). It would be better to report the standard
error, confidence interval, and/or probability value as measures of the
statistical significance of a linear trend.
Please comment on caveats or limitations of using non-significant
regression estimates to project the safety impact of mass reduction.
Regression estimates are random numbers which have an unknown
expected value and variance, and known sample value and standard
error. If the sample value can be explained by a zero expected value
and known standard error then the result is considered not statistically
significantly different than zero and therefore the result is not
considered to be statistically significant. However, If we can combine
this estimate with other estimates then the unknown expected values
and variances can also be combined using the same transformation,
and the statistical significance of the combined result can be tested.
Therefore, depending on the sample values and inter-correlation, the
combined result may be statistically significant even if the individual
estimates are not statistically significant.
For example, the results from each of the nine different crash types can
be combined into an overall estimate and the standard error calculated
assuming that the results for each crash type are independent of each
other. Then the statistical significance of the combined effect can be
determined.
However, and Kahane (2011) points out there are two sources of
uncertainty in the regression results. The first is the PARS based
sampling error which is uncorrelated across crash types because they
are based on different fatal cases (Kahane 2011, p. 77). The second is
138
-------
the state based induced-exposure sampling error which is correlated
across crash types because they are based on the same induced-
exposure cases. Therefore a confidence interval estimated using the
"jackknife" method described by Kahane (2011) and accounting for
correlation of these two error sources would be more accurate than a
simple estimate based on the Wald Chi-Square statistic and assumed
independence.
How might the LBNL methodology be strengthened to better represent
future vehicle designs and reduce multi-collinearity between mass and
footprint in the regression analysis?
The effects of multi-collinearity can be mitigated by 1) obtaining more
data, 2) pooling data from different crash type or vehicle types, or 3)
reducing the number of regression variables. The first option would
require more calendar years and/or model years, which would involve
added newer data as it becomes available (or using older data). The
second option might be to recombine the CUVs and minivans with
truck based vans and adding a control variable to compensate for the
differences in the vehicles types. The third option might involve
removing statistically insignificant control variables or removing control
variables that would not be expected to have an effect on the
probability of fatality in the crash (e.g., the side airbag variable is not
included in pedestrian crashes because it is not expected to affect
pedestrian fatality risk). The number of driver age control variables
might be reduced from eight to three (as in the Kahane (1997) and DRI
(2002-2005) studies). Finally, a linear curb weight model instead of a
two-piece linear model may help to better elucidate the general trend.
The Variance Inflation Factor (VIF) has been suggested as a measure of
multi-collinearity in the Kahane (2010 and 2011) reports, however this
diagnostic metric does not account for differences in database size (i.e.,
Options 1 and 2 above). The Wenzel (2011a) report does not discuss
the Variance Inflation Factor or report any VIF results.
ADDITIONAL COMMENTS:
None come to mind.
139
-------
4. DATA SETS
COMMENTS
Please comment on the validity and applicability of the datasets used
to project changes in risk resulting from reduction in vehicle mass.
LBNL's casualty analysis used police-reported crash data from 16
states, while the 2011 NHTSA study used national fatality data,
combined with a subset of non-culpable vehicles involved in
two-vehicle crashes from police-reported crash data from 13 states.
The induced-exposure data set provided by NHTSA is based on the non-
culpable vehicles in two-vehicle crashes. See the comments in Table 1
on the limitations of this data. In addition, there are also many
differences in the coding variables and values used by the different
states, which tend to make the receding to a common data set
imprecise.
[Note: This peer review does not address the Wenzel (2011b)
companion report, which examines the risks per police-reported crash.
See Ref 4 for comments on the companion report.]
Please comment on any apparent, unstated, or implicit impact on
estimated risks inherent in the two different approaches, and any
related caveats or limitations. For example, what are the strengths and
weaknesses of the two measures of vehicle exposure, miles of vehicle
travelled scaled up from crash data from 13 states, and number of
police-reported crashes?
The number of fatal cases tends to be much less than the number of
induced-exposure cases. Therefore the effective numbers of degrees-
of-freedom in the statistical estimates tend to be limited by the
available number of fatal cases. For example, it would not be possible
to estimate the effects of two variables (e.g., just curb weight and
footprint) if we had data for only one fatal case even if we had
thousands of induced-exposure cases. Therefore it is desirable to use
data for the entire US in order to get a large sample of fatal cases for
the logistic regressions. This then requires the available induced-
exposure data (i.e., from 13 states) to be "scaled up" the US level using
the method described in Kahane (2003 and 2011). The result is the
best currently available estimate of vehicle exposure.
There may be some concerns about the accuracy of the vehicle miles-
travelled data because the difficulty estimating the number of vehicle
miles travelled at the make-model-year-state level of detail.
[Note: This peer review does not address the Wenzel (2011b)
companion report, which examines the risks per police-reported crash.
See Ref 4 for comments on the companion report.]
140
-------
ADDITIONAL COMMENTS:
None come to mind.
141
-------
5. RECOMMENDATIONS
COMMENTS
Please comment on whether the LBNL assessment adequately
addresses the NHTSA 2011 study and identifies the safety impact from
mass reduction. Are the analytic methods and data used to assess the
NHTSA study, and estimate the relationship between risk, mass, and
footprint, appropriate? Is casualty risk per crash a legitimate measure
of vehicle safety? What other methods or data could be used to better
predict the effect of future vehicle designs on safety?
The basic methodology described by Kahane (2011) seems appropriate;
however some results using this method and data are not well
understood and need further diagnosis.
The induced-exposure data set provided by NHTSA is based on the non-
culpable vehicles in two-vehicle crashes. See the Table 1 comments on
the limitations of this data.
[Note: This peer review does not address the Wenzel (2011b)
companion report, which examines the risks per police-reported crash.]
Please comment on the overall adequacy of LBNL's assessment of the
2011 NHTSA report and its independent study of casualty risk for
predicting the effect of vehicle mass or footprint reduction on safety.
Provide any recommended improvements that might reasonably be
adopted by the author to improve the analysis.
The Wenzel (2011a) report provides a valuable supplement to the
analysis and results in the Kahane (2011) report.
[Note: This peer review does not address the Wenzel (2011b)
companion report, which examines the risks per police-reported crash.
ADDITIONAL COMMENTS:
See additional comments and recommendations in Tables 6 and 7.
142
-------
Table 6. Additional General Comments and Recommendations
Section
COMMENTS AND RECOMMENDATIONS
Use of R2 is confusing. Suggest using lower case "r" when referring to the sample correlation coefficient (Box, Hunter, Hunter,
1978, P. 61); or upper case R when referring to the regression coefficient of multiple determination (Draper and Smith, 1981, p.
90).
All
In most cases the reported results are just estimates, but are not described as such. For example, "The effect of mass reduction
on heavier cars and CUVs and minivans are not statistically significant" on p. iii should say "The estimated effect of mass
reduction on heavier cars and CUVs and minivans are not statistically significant."
This distinction is important when comparing results based on different models and assumptions because the different models
and assumptions do not change the effect itself, but rather the estimate of the effect. For example, the statement "The first
sensitivity, in dark purple, includes the weight variables in the regression model but excludes the footprint variable; this model
tests the effect of mass reduction while allowing footprint to vary with vehicle mass. This sensitivity increases the risk from a
100-lb mass reduction in cars (from 1.43% to 2.64% for lighter cars, and from 0.48% to 1.94% for heavier cars) and
CUVs/minivans (from a 0.47% decrease in risk to a 0.52% increase in risk); however, there is no change in fatality risk in light-
duty trucks" on page 15 is misleading. It would be better to state that "The first sensitivity, in dark purple, includes the weight
variables in the regression model but excludes the footprint variable; this model tests the estimated effect of mass reduction
while allowing footprint to vary with vehicle mass. This GGnsitivity/?emoi//'ng the footprint variable from the regression model
increases the estimated risk from a 100-lb mass reduction in cars (from 1.43% to 2.64% for lighter cars, and from 0.48% to
1.94% for heavier cars) and CUVs/minivans (from a 0.47% decrease in risk to a 0.52% increase in risk); however, there is nothe
change in the estimated fatality risk in light-duty trucks is very small and not statistically significant."
This also applies to table and figure captions. For example, "Table ES.l. Effect of mass and footprint reduction on fatality risk,
under alternative regression model specifications" should say "Table ES.l. Estimated effects of mass and footprint reduction on
fatality risk, under alternative regression model specifications." "Figure 3.3 Effect of reduction in mass or footprint on US
fatality risk per VMT, by vehicle type: mass only, footprint only, and both" should say "Figure 3.3 Estimated effects of reduction
in mass or footprint on US fatality risk per VMT, by vehicle type: mass only, footprint only, and both."
Overall the word "effect" appears over 200 times in this report with the "estimated" or other qualifier. In some cases this may
be appropriate and in other cases it is not appropriate. It is recommended that the author review each instance and revise as
appropriate.
143
-------
Figures 4.1 through 4.17 do not control for the effect of vehicle size (e.g. footprint), which has been shown to be correlated
with vehicle weight (e.g., Figure 3.1), and therefore these figures may be misleading. It is strongly suggested that the horizontal
axis label be changed to "Curb weight (Ibs) and corresponding changes in size," and/or a note such as the following be added to
each figure: "Note these results do not control for the effect of vehicle size on fatality risk. Therefore the horizontal axis
represents changes to both vehicle weight and vehicle size."
The statistical significance of the linear trends in Figures 4.1 through 4.17 are not reported. It would be helpful if the
confidence intervals or statistical significance of the linear trends were reported, either in addition to or instead of R2.
The confidence intervals for the estimated slopes should be added to the results in Tables 4.1 and 4.2.
144
-------
Table 7. Additional Specific Comments and Recommendations
Section
Executive
Summary, 7
Executive
Summary
Executive
Summary, 4
Executive
Summary, 7
4
4
5.1
5.2
5.2
5.2
5.3
6.4
Page
iii, 65
iv
Iv, v, 22, 66
viii, 69
31-32
35
37
39
39
39
44
63
COMMENTS AND RECOMMENDATIONS
2nd paragraph refers to "our analysis," however the results are the same as the NHTSA analysis. The author
should clarify who or what "our analysis" refers to and how it relates to the NHTSA analysis. Perhaps the
statement "LBNL was able to reproduce the NHTSA analysis, which finds that..." would be more appropriate.
Last bullet - suggest changing the statement that "Logistic regression does not allow a statistic" to "Logistic
regression methods do not have a statistic."
Suggest changing "variance in risk" to variation in risk" throughout.
The numerical results for the NHTSA preferred model in Tables ES.l and 7.1 are slightly different than the results
reported in the NHTSA report. For example 1.43%/0.48%/0.52%/-0.40%/-0.47% should be 1.44%/0.47%/O.S2%/-
0.39%/-0.46%
Figures 4.12 through 4.14 have the results for small and heavy-duty pickups combined, which is inconsistent with
the results in Table 4.1
The R2 values in Table 4.1 are different than the values in Figures 4.6, 4.8, 4.9, 4.11.
The subsection title should be "Alternative measures of exposure and outcome" because fatal crashes and
fatalities are measures of the crash outcome, not exposure.
It would be helpful to list the 18 manufacturer dummy variables in a table.
It is unclear why Lexus, Acura, and Infinity are treated as separate manufacturers, but Cadillac and Lincoln are
not.
It is unclear why AM General is considered a Chrysler brand. The AM General Hummer was sold by GM
beginning with the 2001 model year.
It would be helpful if the figures include error bars or shading to indicate the confidence intervals.
Table 6.3 - Suggest adding a note that the 72,316 total includes fatalities that are counted more than once in
crashes involving more than one vehicle type.
145
-------
Michael Van Auken
Review of LBNL Phase 2 Report
1. ASSUMPTIONS
COMMENTS
Please comment on the validity of any assumptions embedded in the
LBNL assessment analysis and the independent casualty analysis that
could affect the projected relationship between vehicle mass/footprint
reductions and fatality/casualty risk. Examples might include
assumptions regarding whether recent historical relationships between
vehicle weight, size, and safety will continue into the future; potential
future improvements in vehicle technology and design may result in
compensatory safety benefits; and the annual baseline fatality
distribution.
The basic assumptions, methodology, and data are primarily the same
as in the Kahane (2011) report, but have been extended to include
serious injuries as well as fatalities, and also address crash involvement
(i.e., fatalities and serious injuries per accident, and also accidents per
VMT).. These include the following:
1) The probability of a crash fatality or serious injury is proportional to
the number of accidents (provided the crash conditions remain the
same); and the probability of an accident is proportional to the
vehicle miles travelled (VMT).
2) The logarithm of probabilities of fatality or serious injury per
accident, and accidents per VMT for a given curb weight, footprint,
and control variable values varies as a linear combination of the
curb weight, footprint, and control variables within the domain of
the data.
3) The logistic regression methods determine a maximum likelihood
estimate of model coefficients.
4) It is assumed that the above relationships remain constant in the
recent past (i.e., 2000-2007 model year vehicles in the 2002-2008
calendar years), present, and near future (i.e., 2017-2025 model
year vehicles).
The first assumption that crash fatalities and serious injuries are
proportional to the number of accidents provided the crash conditions
remain the same seems self-evident (e.g., if two fatal crashes had
exactly the same conditions, then the expected number of fatalities for
the two crashes would be twice the value for just one of the crashes).
The assumption that the number of accidents are proportional to VMT
rather than the number of vehicle registration years (VRY) is also
appropriate because accidents cannot occur if the vehicles are not
146
-------
driven on the road (i.e., VMT = 0). This assumption is qualified
however because VMT is more difficult to measure than VRY and
therefore may be less accurate. On the other hand the probability of a
fatal crash or the number of fatalities in a crash may also depend on
the vehicle occupancy.
The second and third assumptions are appropriate provided that it is
recognized that it is essentially impossible with currently available
knowledge and information to model all of the factors that could affect
the probability of fatality in a crash, and that the objective of the
analysis is to identify overall trends versus vehicle weight and
footprint. In general the probability of fatality depends on other many
other factors which have not been modelled (e.g., driver behavior
factors, vehicle design factors, roadway design factors, EMT factors),
and these unmodeled factors are assumed to be uncorrelated with
vehicle weight and footprint, and/or are represented by the other
control variables. The latter assumption might or might not be valid.
The fourth assumption is perhaps the weakest because it assumes that
future vehicles will have the same design characteristics as past
vehicles, and that the characteristics of the vehicle population (e.g.,
collision partner weight, size, type) will also remain the same.
Please comment on any apparent unstated or implicit assumptions and
related caveats or limitations.
The induced-exposure data set provided by NHTSA is based on the
"non-culpable" vehicle in two-vehicle crashes. It is assumed that the
dataset is a reprehensive sample of the driver and environmental
exposure factors for vehicle use. However, since these cases include
moving vehicles, some vehicle-driver-environmental conditions may be
under or over represented in this data depending on how they affect
the ability of a non-culpable vehicle to avoid a crash. Results in Ref
(17) indicated that the estimated effect of weight and size reduction
are sensitive to whether the induced-exposure data are based on the
Kahane (2003) non-culpable vehicle definition of the Kahane (1997)
stopped vehicle definition.
147
-------
Unfortunately it is not currently possible to test this sensitivity with the
NHTSA-provided induced-exposure data.
It is also assumed that the accident data from the 13 or 16 states are
representative of all US states. Figure 2.1 in Wenzel (2011b) provides a
useful comparison of the distribution of fatalities in the US and 13
states by the nine different crash types.
ADDITIONAL COMMENTS:
None come to mind.
148
-------
2. CONTROL AND DEPENDENT VARIABLES USED IN THE REGRESSION
MODELS
COMMENTS
Please comment on the adequacy of control and dependent variables
used in the assessment analysis and independent casualty analysis, and
recommend any alternative control or dependent variables that are
available for possible inclusion in the analysis. For example, what are
the relative merits of the main dependent variables used, fatality risk
per estimated VMT, and casualty risk per police-reported crash?
Reducing the total number of fatalities and serious injuries is desirable
from a societal viewpoint. This includes both subject vehicle occupant
and collision partner (e.g., other vehicle occupant, pedestrian) fatalities
and serious injuries.
VMT is a good measure of accident exposure provided that it can be
accurately determined.
The number of fatalities and serious injuries per accident is a measure
of vehicle crashworthiness (i.e., effect of a crash on the subject vehicle
occupants) and crash compatibility (i.e., effect of a crash on the other
vehicle occupants or vulnerable road users). Subject vehicle occupant
fatalities and serious injuries per accident are a measure of the subject
vehicle crashworthiness. Collision partner fatalities and serious injuries
per accident are a measure of vehicle crash compatibility.
The number of accidents per VMT is a measure of the crash avoidance
capabilities of a given vehicle.
What additional control variables, such as vehicle make or model,
might be included in the regression models?
None come to mind.
Please comment on any caveats or limitations that these dependent
variable or control variables entail with respect to use of the results as
the basis for estimating the safety effect of mass reduction.
None come to mind.
ADDITIONAL COMMENTS:
The underlying reasons for some of the estimated effects are unknown at this time, but presumably involve driver, vehicle, environment or
accident factors than have not been controlled for in the Kahane (2011) and Wenzel (2011b) analyses. See, for example, Refs 17 and 23.
149
-------
3. METHODOLOGY AND STATISTICS
COMMENTS
Please comment on the validity and applicability of the methodology
LBNL used in assessing the NHTSA 2011 study and its analysis of the
relationship between mass, footprint, and risks per police-reported
crash.
The logistic regression methods seem to be appropriate. The
confidence intervals are based on the logistic regression Wald Chi-
Square statistic, which as Kahane (2003, 2011) has demonstrated does
not include all sources of variation. However, these confidence
intervals are useful because they do provide some indication of the
uncertainty in the results.
The two-stage results for the 13-state fatalities per crash and 13-state
crashes per VMT, and the one-stage result for 13-state fatalities per
VMT reported in Tables ES.l and 6.1 and Figure 2.7 were computed
using independent logistic regressions. The differences between the
two-stage results and the one-stage results for fatalities per crash
could have been eliminated by using the "simultaneous three-way"
logistic regression method described in DRI (2003). This method
imposes the constraint that the combined two-stage estimated and the
one-stage estimated are equal.
Please review other statistical methods LBNL has used in the analysis,
in addition to the logistic regression methodology. Examples include
the alternative approaches used by LBNL to assess NHTSA interval
estimation results, and LBNL's linear regression analysis of actual,
predicted, and residual risk by vehicle model.
The correlations in Section 3 appear to be assessed using the
Coefficient of Multiple Determination (R2) based on a linear fit to the
data (e.g., the correlation between footprint versus curb weight in
Figure 3.1 on p. 32). The linear regression model attributes the
differences between the dependent variable (vertical axis) and the
linear fit to the independent variable (horizontal axis) to random
effects. If there is no preference as to the choice of independent and
dependent variables (e.g., footprint versus curb weight, or curb weight
versus footprint), then the linear trend and R2 result would be different
if the two variables were interchanged, and having two different yet
equally valid results would be undesirable.
If the variation in the data can be attributed to both variables (e.g.,
footprint and curb weight), then it would be better to report the
square of the sample correlation coefficient r2, where r is computed
according to Eqn (1). The trend lines in these correlation figures should
150
-------
not be computed using a linear regression. Instead, the trend line
should pass through the sample means (i.e., (x, y)), and have a slope
equal to the ratio of the sample standard deviations in the data (i.e., sy
/sx). Therefore, the reported correlation results do not depend on the
ordering of the data variables.
Note this comments does not apply to linear trends indicated in
Section 4, for which the Coefficient of Multiple Determination (R2)
seems appropriate.
The Coefficient of Multiple Determination (R2) is frequently used in the
Wenzel (2011b) report as an indicator of the statistical importance of a
linear trend (e.g., R2 values in Tables 4.1 and 4.2 were compared to
0.3). It would be better to report the standard error, confidence
interval, and/or probability value as measures of the statistical
significance of a linear trend.
Please comment on caveats or limitations of using non-significant
regression estimates to project the safety impact of mass reduction.
Regression estimates are random numbers which have an unknown
expected value and variance, and known sample value and standard
error. If the sample value can be explained by a zero expected value
and known standard error then the result is considered not statistically
significantly different than zero and therefore the result is not
considered to be statistically significant. However, If we can combine
this estimate with other estimates then the unknown expected values
and variances can also be combined using the same transformation,
and the statistical significance of the combined result can be tested.
Therefore, depending on the sample values and inter-correlation, the
combined result may be statistically significant even if the individual
estimates are not statistically significant.
For example, the results from each of the nine different crash types can
be combined into an overall estimate and the standard error calculated
assuming that the results for each crash type are independent of each
other. Then the statistical significance of the combined effect can be
151
-------
determined.
However, and Kahane (2011) points out there are two sources of
uncertainty in the regression results. The first is the PARS based
sampling error which is uncorrelated across crash types because they
are based on different fatal cases (Kahane 2011, p. 77). The second is
the state based induced-exposure sampling error which is correlated
across crash types because they are based on the same induced-
exposure cases. Therefore a confidence interval estimated using the
"jackknife" method described by Kahane (2011) and accounting for
correlation of these two error sources would be more accurate than a
simple estimate based on the Wald Chi-Square statistic and assumed
independence.
How might the LBNL methodology be strengthened to better represent
future vehicle designs and reduce multi-collinearity between mass and
footprint in the regression analysis?
The effects of multi-collinearity can be mitigated by 1) obtaining more
data, 2) pooling data from different crash type or vehicle types, or 3)
reducing the number of regression variables. The first option would
require more states (for serious injuries and police-reported accidents),
calendar years and/or model years, which would involve added newer
data as it becomes available (or using older data). The second option
might be to recombine the CUVs and minivans with truck based vans
and adding a control variable to compensate for the differences in the
vehicles types. The third option might involve removing statistically
insignificant control variables or removing control variables that would
not be expected to have an effect on the probability of crash or crash
outcome (e.g., the side airbag variable is not included in pedestrian
crashes because it is not expected to affect pedestrian fatality risk).
The number of driver age control variables might be reduced from
eight to three (as in the Kahane (1997) and DRI (2002-2005) studies).
Finally, a linear curb weight model instead of a two-piece linear model
may help to better elucidate the general trend.
The Variance Inflation Factor (VIF) has been suggested as a measure of
multi-collinearity in the Kahane (2010 and 2011) reports, however this
152
-------
diagnostic metric does not account for differences in database size (i.e.,
Options 1 and 2 above). The Wenzel (2011b) report does not discuss
the Variance Inflation Factor or report any VIF results.
ADDITIONAL COMMENTS:
None come to mind.
153
-------
4. DATA SETS
COMMENTS
Please comment on the validity and applicability of the datasets used
to project changes in risk resulting from reduction in vehicle mass.
LBNL's casualty analysis used police-reported crash data from 16
states, while the 2011 NHTSA study used national fatality data,
combined with a subset of non-culpable vehicles involved in
two-vehicle crashes from police-reported crash data from 13 states.
The induced-exposure data set provided by NHTSA is based on the non-
culpable vehicles in two-vehicle crashes. See the comments in Table 1
on the limitations of this data.
The use of property damage accident data and cases with serious injury
from the 13 states seems appropriate (with the noted qualification that
the different states may have different accident reporting thresholds
and injury reporting criteria). The concerns about the use of data for
the 3 additional states (Georgia, Illinois, and New Mexico) have also
been noted.
In addition, there are also many differences in the coding variables and
values used by the different states, which tend to make the receding to
a common data set (either induced-exposure, police-reported accident,
or severe injury) imprecise.
Please comment on any apparent, unstated, or implicit impact on
estimated risks inherent in the two different approaches, and any
related caveats or limitations. For example, what are the strengths and
weaknesses of the two measures of vehicle exposure, miles of vehicle
travelled scaled up from crash data from 13 states, and number of
police-reported crashes?
The number of fatal or serious injury cases tends to be much less than
the number of induced-exposure cases (and the number of police-
reported accidents). Therefore the effective numbers of degrees-of-
freedom in the statistical estimates tend to be limited by the available
number of fatal or serious injury cases. For example, it would not be
possible to estimate the effects of two variables (e.g., just curb weight
and footprint) if we had data for only one fatal of serious injury case
even if we had thousands of induced-exposure cases. Therefore it is
desirable to use data for the entire US in order to get a large sample of
fatal cases for the logistic regressions. This then requires the available
induced-exposure data (i.e., from 13 states) to be "scaled up" the US
level using the method described in Kahane (2003 and 2011). The
result is the best currently available estimate of vehicle exposure.
There may be some concerns about the accuracy of the vehicle miles-
travelled data because the difficulty estimating the number of vehicle
miles travelled at the make-model-year-state level of detail.
154
-------
ADDITIONAL COMMENTS:
None come to mind.
155
-------
5. RECOMMENDATIONS
COMMENTS
Please comment on whether the LBNL assessment adequately
addresses the NHTSA 2011 study and identifies the safety impact from
mass reduction. Are the analytic methods and data used to assess the
NHTSA study, and estimate the relationship between risk, mass, and
footprint, appropriate? Is casualty risk per crash a legitimate measure
of vehicle safety? What other methods or data could be used to better
predict the effect of future vehicle designs on safety?
The basic methodology described by Kahane (2011) seems appropriate;
and the extension by Wenzel (2011b) are also appropriate. However
some results using these methods and data are not well understood
and need further diagnosis.
The induced-exposure data set provided by NHTSA is based on the non-
culpable vehicles in two-vehicle crashes. See the Table 1 comments on
the limitations of this data.
The state accident data files tend to have different database variable
and coding definitions and criteria, which could confound the results.
Please comment on the overall adequacy of LBNL's assessment of the
2011 NHTSA report and its independent study of casualty risk for
predicting the effect of vehicle mass or footprint reduction on safety.
Provide any recommended improvements that might reasonably be
adopted by the author to improve the analysis.
The Wenzel (2011b) report provides a valuable supplement to the
analysis and results in the Kahane (2011) report.
ADDITIONAL COMMENTS:
See additional comments and recommendations in Tables 6 and 7.
156
-------
Table 6. Additional General Comments and Recommendations
Section
COMMENTS AND RECOMMENDATIONS
Use of R2 is confusing. Suggest using lower case "r" when referring to the sample correlation coefficient (Box, Hunter, Hunter,
1978, P. 61); or upper case R when referring to the regression coefficient of multiple determination (Draper and Smith, 1981, p.
90).
All
In most cases the reported results are just estimates, but are not described as such. The word "effect" appears several hundred
times in this report with the "estimated" or other qualifier. In some cases this may be appropriate and in other cases it is not
appropriate. It is recommended that the author review each instance and revise as appropriate.
All
"Crashworthiness" in most instances should be changed to "crashworthiness and crash compatibility" because the fatalities
and/or serious injuries may either be in the subject vehicle (crashworthiness effect) or collision partner (crash compatibility
effect).
The statistical significance of the linear trends in Figures 4.1 through 4.9 are not reported. It would be helpful if the confidence
intervals or statistical significance of the linear trends were reported, either in addition to or instead of R2.
The confidence intervals for the estimated slopes should be added to the results in Tables 4.1 and 4.2.
157
-------
Table 7. Additional Specific Comments and Recommendations
Section
Executive
Summary, 4
Executive
Summary, 6
5.3
5.3
5.3
Page
iv, v, 22, 66
vii, 63
49
48-49
49
COMMENTS AND RECOMMENDATIONS
Suggest changing "variance in risk" to "variation in risk" throughout.
The statement "In conclusion, casualty risk per crash is not necessarily a better metric than fatality risk per VMT
for evaluating the effect of mass or footprint reduction on risk; rather, it provides a different perspective in
assessing the benefits or drawbacks of mass and footprint reduction on safety in vehicles. However, it does
allow the separation of risk per VMT to be separated into its two components, crash frequency and risk per
crash" suggests that the casualty risk per crash metric was needed in order to assess the crash frequency and risk
per crash, which is incorrect. The DRI (2003-2012) methods have also estimated the effects of weight and size
on crash frequency (A/E) and risk per crash (F/E) in terms of fatalities.
It would be helpful to list the 18 manufacturer dummy variables in a table.
It is unclear why Lexus, Acura, and Infinity are treated as separate manufacturers, but Cadillac and Lincoln are
not.
It is unclear why AM General is considered a Chrysler brand. The AM General Hummer was sold by GM
beginning with the 2003 model year.
158
-------
August 9, 2012
MEMORANDUM
SUBJECT: EPA Response to Comments on the peer review of LBNL Statistical Analysis of the Effect
of Vehicle Mass & Footprint Reduction on Safety) (LBNL Phase 1 and 2 Reports),
prepared by Tom Wenzel, Lawrence Berkeley National Laboratory.
FROM: Cheryl Caffrey, Assessment and Standards Division
Office of Transportation and Air Quality, U.S. Environmental Protection Agency
The LBNL Phase 1 and 2 Reports were reviewed by Donna Chen and Kara Kockelman (University of Texas
at Austin), Charles Farmer (Insurance Institute for Highway Safety, David Greene (Oak Ridge National
Laboratory), and Michael Van Auken (Dynamic Research, Inc.).
This memo includes a compilation of comments prepared by SRA International; LBNL's responses to, and
actions in response to, those comments are interspersed with the comments and noted by [LBNL].
159
-------
Assessment of NHTSA's Report
''Relationships Between Fatality Risk, Mass, and Footprint in Model Year 2000-2007 Passenger Cars and LTVs"
(LBNL Phase 1 Report)
1. ASSUMPTIONS
COMMENTS
Please comment on the validity of any
assumptions embedded in the LBNL
assessment analysis and the independent
casualty analysis that could affect the
projected relationship between vehicle
mass/footprint reductions and
fatality/casualty risk. Examples might include
assumptions regarding whether recent
historical relationships between vehicle
weight, size, and safety will continue into the
future; potential future improvements in
vehicle technology and design may result in
compensatory safety benefits; and the annual
baseline fatality distribution.
[Chen and Kockelman] [1] The report does a nice job discussing recent trends in vehicles, such as the increase
of ESC, side airbags, and light truck crash compatibility with passenger cars - which will improve safety
outcomes for all vehicles, but perhaps most significantly the smaller and lighter vehicles.. It also mentions the
phasing out of the lightest and smallest vehicles between model years 2000-2007 (but doesn't mention the
makes and models somehow), which were particularly poor safety performers in the past. However, with the
introduction of urban commuter vehicles, such as the SmartCar, Mini Cooper, and Fiat 500m, and the growing
popularity of smaller, fuel-efficient compact vehicles following gas price increases, this trend does not seem so
obvious. Such vehicles should be discussed.
[LBNL] The vehicles mentioned were particular models of small light vehicles that had poor on-road safety
records. However as Figures 4.7 and 4.10 indicate, there are several recent car models weighing less than
2,500 Ibs that have fatality risks similar to models weighing 1,000 Ibs more, even after accounting for all
differences in vehicle, driver and crash characteristics except mass and footprint.
[2] The simplistic logistic model employed in this analysis only accounts for two crash outcomes (fatal versus
non-fatal) and so neglects the more detailed, and ordered nature of injury severity data, which is unfortunate.
The model also assumes error-term homoscedasticity from one crash or individual to the next; in reality
certain vehicle types (e.g., pickups) and crash contexts (e.g., high speed crashes) have more uncertainty
associated with their severity outcomes. It would be good to point out such limitations for readers.
[LBNL] The LBNL Phase 1 report followed the methodology of the NHTSA 2011 report, which analyzes the
estimated relationships between vehicle mass, footprint and US fatalities per VMT. The LBNL Phase 2 report
analyzes the relationships between vehicle mass, footprint, and casualties, defined as fatalities plus serious or
incapacitating injuries. The effects on other injury severities (i.e. minor injuries) were not analyzed because of
the differences in reporting of injury severity in the thirteen states.
[Van Auken] The basic assumptions, methodology, and data are primarily the same as in the Kahane (2011)
160
-------
report. These include the following:
1) The probability of a crash fatality is proportional to the vehicle miles travelled (VMT), except as noted in
Section 5.1
2) The logarithm of probability of fatality per VMT for a given curb weight, footprint, and control variable
values varies as a linear combination of the curb weight, footprint, and control variables within the domain
of the data.
3) The logistic regression methods determine a maximum likelihood estimate of model coefficients.
4) It is assumed that the above relationships remain constant in the recent past (i.e., 2000-2007 model year
vehicles in the 2002-2008 calendar years), present, and near future (i.e., 2017-2025 model year vehicles).
The first assumption that crash fatalities are proportional to VMT ratherthan the number of vehicle
registration years (VRY) is appropriate because the fatalities cannot occur if the vehicles are not driven on the
road (i.e., VMT = 0). This assumption is qualified however because VMT is more difficult to measure than VRY
and therefore may be less accurate. On the other hand the probability of a fatal crash or the number of
fatalities in a crash may also depend on the vehicle occupancy. The analysis in Section 5.1 is a commendable
attempt to explore the sensitivity to this assumption, however the Kahane (1997) and DRI (2003-2005) reports
have shown that some driver, vehicle, and environmental factors may be underrepresented or
overrepresented in unweighted induced-exposure data. VRY could have also been considered as a measure of
exposure.
[LBNL] NHTSA and LBNL used vehicle miles traveled, rather than vehicle registration-years, as the measure of
exposure because a vehicle that is not driven has zero risk. Nonetheless, LBNL conducted a sensitivity using
vehicle registration years rather than VMT as the measure of exposure. This alternative resulted in lower
estimated effects of mass reduction on risk in lighter cars and light trucks, no change in CUVs/minivans, but a
substantially higher estimated effect of mass reduction in heavier cars (from an estimated 0.51% increase in
risk to an estimated 2.40% increase in risk, as shown in Alternative 6 in new Table ES.l).
The second and third assumptions are appropriate provided that it is recognized that it is essentially
impossible with currently available knowledge and information to model all of the factors that could affect the
probability of fatality in a crash, and that the objective of the analysis is to identify overall trends versus
vehicle weight and footprint. In general the probability of fatality depends on other many other factors which
have not been modeled (e.g., driver behavior factors, vehicle design factors, roadway design factors, EMT
factors) and these unmodeled factors are assumed to be uncorrelated with vehicle weight and footprint,
161
-------
and/or are represented by the other control variables. The latter assumption might or might not be valid.
The fourth assumption is perhaps the weakest because it assumes that future vehicles will have the same
design characteristics as past vehicles, and that the characteristics of the vehicle population (e.g., collision
partner weight, size, type) will also remain the same. A commendable attempt to partially address this effect
is described in Section 6. These effects can be perhaps better addressed by the "Volpe model" described in
Kahane (2011) of the Honda-DRI fleet systems model described in Refs (2324), which can be used to forecast
the effects of mass reductions of individual makes and models on a year-by-year basis.
Please comment on any apparent unstated or
implicit assumptions and related caveats or
limitations.
[Chen and Kockelman] The role of driver behavior is briefly addressed in the report but not emphasized
sufficiently. Fatality risk is a combination of driver, vehicle, and roadway characteristics. Driver behavioral
differences are many and do not solely exist for pickup truck drivers versus car drivers. Socioeconomic data
such driver household income, size, and education influence driver attitudes and driving environments. For
example, Chen et al. (2010) found that crash risk increases for those living in socioeconomically disadvantaged
areas (including households more likely to drive less expensive and older vehicles). Though such data is not
typically available in state and national crash databases, the importance of these driver and environmental
characteristics on crash rates (per mile driven) and fatality risk should be stressed in both reports. It is clearly
very difficult to control for, but a major caveat to the NHTSA (& now LBNL) results. We expect that crash
severity could be probably be lower for many of the small cars and pickups if they were driven by those who
tend to drive more expensive vehicles, under the same settings (e.g., daytime, urban freeway). Similarly, in the
second LBNL report (which uses VMT estimates), we expect that crash rates would probably be lower for these
types of driver-vehicle-setting combinations.
[LBNL] We agree that it would be preferable to include additional variables that account for driver behavior
rather than just driver age and gender. However, variables such as driver age, gender, or other socio-economic
characteristics probably have a less direct effect on risk than the actual driving skill or behavior of individual
drivers. The PARS data includes information on whether alcohol or drug use, or speeding or reckless driving,
was a factor in the current crash, as well as the driver's record for the last three years. NHTSA 2003 found that
including a "bad driver" rating variable comprised of 8 of these variables did not have an appreciable effect on
the relationship between vehicle mass and risk. Although these variables are included in the PARS fatal crash
records, they are not included in the state crash databases, so a similar variable cannot be used in the
regression models. However, Alternative 12 in new Table ES.2, and the third column in new Figure 5.14,
indicates that excluding bad drivers (after also excluding reported driver alcohol or drug use) results in
additional estimated increases in fatality risk as vehicle mass decreases, and estimated decreases in risk as
162
-------
vehicle footprint decreases. LBNL believes that driver behavior in the current crash and their recent driving
record, and not general socio-economic information, is the best available measure to account for the effect
driver skill and behavior has on risk in individual vehicles.
Nonetheless, LBNL explored the effect of adding a control variable for one additional driver socio-economic
characteristic, household income. Unfortunately, crash data do not include household income of vehicle
drivers. PARS includes the driver's zip code, from his or her license, which could be merged with US Census data
by zip code to obtain the average median income of households in a particular zip code. However, very few
states include driver zip code in their databases of police-reported crashes.
Kweon and Kockelman (2003) merged police-reported crash data from GES with household demographic data,
including household income, from the National Household Travel Survey (NHTS), to examine crash rates per
annual VMTby driver age, gender, and vehicle type. Each dataset includes population weights to derive
national estimates of crashes and households, which the authors use to estimate national crash rates for driver
age, gender and vehicle type. However, there are several limitations with the NHTS data. First, the survey
includes only 25,000 national households, supplemented by an additional 124,000 households from "add-on"
state and municipal jurisdictions. The national weights provided in the NHTS most likely do not represent the
national distribution of vehicle registrations by make and model. Second, 30% of the households in the 2009
NHTS reported income "over $100,000"; using the income bins provided in the NHTS to estimate household
income will likely understate the actual income of the households participating in the survey.
Chen el al (2010) made use of an Australian survey of 20,000 young adult drivers that asked questions
regarding driving behavior. The injury severity of 127 of these participants who were admitted to hospitals
after a vehicle crash was analyzed as a function of socio-economic status. The socio-economic status of the
drivers was based on average or median values from the zip code where they (or their parents, since most were
at University) lived.
Using 2010 California vehicle registration data, LBNL calculated the median household income by registered
owner's zip code, using 2000 census data, then took the average household income by vehicle make and model.
Predicted US fatality risk is correlated with this measure of average household income by vehicle model, with
risk decreasing as income increases; the correlation is highest for cars and CUVs, with an R2 of 0.60 (or a
correlation coefficient r of 0.77) or higher, as shown in new Figure 4.20 and the first column of new Table 4.7.
However, while income tends to increase as mass increases, the correlation is not strong (R2 is highest for 4-
door cars, at 0.16, and CUVs, at 0.19, as shown in new Figure 4.21 and the middle columns of new Table 4.7).
163
-------
As with predicted fatality risk, there is a wide range in the average income of households owning vehicle
models with similar mass. This analysis is summarized at the end of Section 4 of the Final Phase 1 report.
Using average household income by vehicle model, rather than driver alcohol or drug use, or reckless driving, as
an alternative indicator of driver behavior substantially reduces the estimated detrimental effect of mass
reduction on risk in cars, and slightly increases the estimated detrimental effect of mass reduction on risk in
light trucks. This analysis is summarized at the end of Section 5.4 of the Final Phase 1 report.
[Farmer] The statistical models assume no interaction between the vehicle size/weight measures and any of
the numerous covariates, but this may not be true. For example, size/weight reductions may differently affect
vehicles with and without ESC if they affect vehicle handling. It is risky to make statements such as that on p.
11 of the Phase I report: Therefore, the mass of a lighter car could be reduced by 800 Ibs while adding ESC,
without increasing fatality risk.
[LBNL] The sentence has been revised to: "For instance, a 100-lb reduction in curb weight for an underweight
car is estimated to increase risk by 1.4%, while installing ESC would reduce risk by 11.4%; the models estimate
that the beneficial effect of adding ESC is nearly ten times that of reducing mass by 100 Ibs. "
[Van Auken] The induced-exposure data set provided by NHTSA is based on the "non-culpable" vehicle in two-
vehicle crashes. It is assumed that the dataset is a reprehensive sample of the driver and environmental
exposure factors for vehicle use. However, since these cases include moving vehicles, some vehicle-driver-
environmental conditions may be under or over represented in this data depending on how they affect the
ability of a non-culpable vehicle to avoid a crash. Results in Ref (17) indicate that the estimated effect of
weight and size reduction are sensitive to whether the induced-exposure data are based on the Kahane (2003)
non-culpable vehicle definition of the Kahane (1996) stopped vehicle definition.
Unfortunately it is not currently possible to test this sensitivity with the NHTSA-provided induced-exposure
data.
[LBNL] NHTSA developed vehicle miles traveled weighting factors for induced exposure cases based on stopped,
rather than non-culpable, vehicles. LBNL investigated what effect this alternative method of exposure has on
the estimated effect of mass or footprint reduction on risk, in Alternative 15 in Table ES.3 and Section 5.6 in the
Final Phase 1 report.
164
-------
ADDITIONAL COMMENTS:
[Chen and Kockelman] Chen, H.Y., Ivers, R.Q., Mariniuk, A.L.C., Boufous, S., Senserrick, T., Woodward, M., Stevenson, M. and Norton R. Socioeconomic status
and risk of car crash injury, independent of place of residence and driving exposure: Results from the DRIVE study. Journal of Epidemiology and Community
Health 64(10), 2010, pp. 998-1003.
165
-------
2. CONTROL AND DEPENDENT
VARIABLES USED IN THE REGRESSION
MODELS
COMMENTS
Please comment on the adequacy of control
and dependent variables used in the
assessment analysis and independent casualty
analysis, and recommend any alternative
control or dependent variables that are
available for possible inclusion in the analysis.
For example, what are the relative merits of
the main dependent variables used, fatality
risk per estimated VMT, and casualty risk per
police-reported crash?
[Chen and Kockelman] [1] As alluded to above, a primary concern is that the NHTSA analysis (& thus the LBNL
analyses) largely neglect the idea that vehicle type (make & model) is very much a proxy for driver type, and a
vehicle's crash avoidance may have very little to do with vehicle type. It has a lot to do with the person behind
the wheel, and gender & age simply aren't enough to control for such distinctions. Education, risk aversion,
ability, wealth, etc., are important covariates. But existing data sets are quite limiting (though the MVOSS &
FAR with 3-year driver violation history do offer some valuable insights, not discussed in these reports). In
reality, small cars may be less crash prone than Kahane's & Wenzel's results suggest, because they are driven
by lower-income, younger, less risk averse people driving in more crash prone settings (e.g., commercial strips
rather than pricey residential suburbs). Such key caveats need thoughtful discussion. Four relevant papers on
the topics of crash frequency and vehicle size-and-weight implications (by Knipling, Kweon & Kockelman, Wang
and Kockelman, and Chen & Kockelman) have been sent to Tom Wenzel. These all include useful literature
reviews for further connections to useful findings for citation in the reports, as time allows the contractor.
[LBNL] As discussed above, LBNL believes that driver behavior in the current crash and their recent driving
record, and not general socio-economic information, is the best available measure to account for the effect
driver skill and behavior have on risk in individual vehicles. As described in Section 5.4, LBNL used this measure
of driver behavior in one of its sensitivity analyses in the draft final report. Nonetheless, LBNL examined the
effect of including a measure of driver household income on risk in Section 5.4 in the Final Phase 1 report.
[2] The grouping of the vehicles into heavier- and lighter-than-average weight categories essentially splits a
"typical" weight vehicle of that type into two categories. The impacts of curb weight and footprint on fatality
risk may be easier to interpret if the vehicles were grouped into 3 weight categories (light, average, and heavy)
& by type (with the average category representing vehicles within one standard deviation of average weight).
Furthermore, the grouping of CUVs and minivans into the same vehicle type category neglects the fact that
these vehicles have faced rather different ground clearance requirements (impacting rollover potential), door
types (sliding vs. standard), and, perhaps most importantly, can appeal to different types of drivers (as
indicated in the market shift of car drivers to CUV drivers).
[LBNL] As described in NHTSA 2011, NHTSA included CUVs and minivans in the same vehicle category in part
because in most respects they are more "car-like" than conventional light-duty trucks (i.e. pickups, SUVs, and
166
-------
fullsize vans). A MINIVAN indicator variable is included in the regressions to estimate the separate effect of
minivan vs. CUV mass reduction on risk. The relatively small number of CUVs and minivans during the analysis
period precludes them from being separated into two distinct groups for analysis.
[Farmer] One needs to restrict control variables to those that are available and reliable. A problem when
combining state databases is that the states often are not consistent as to the variables coded and the
definitions of those variables. This severely limits the list of possible control variables.
[LBNL] The control variables used from the state crash databases are either derived from the VIN, or based on
the driver age and gender, or the time or location of the crash. LBNL believes that these variables are
consistently coded among the thirteen states.
[Van Auken] [1] The main metric used in both the Kahane (2011) and Wenzel (2011a) reports is the total
number of fatalities (except as noted). Reducing the total number of fatalities, which includes both subject
vehicle occupants and collision partner fatalities, is desirable from a societal viewpoint. Fatal crash occurrence
is related to the total number of fatalities, which has been used by Kahane (2003, 2011) to address concerns
about double counting.
[2] VMT is a good measure of accident exposure provided that it can be accurately determined. [Note: This
peer review does not address the Wenzel (2011b) companion report (Ref 3) which examines the risks per
police-reported crash. See Ref 4 for comments on the companion report.]
What additional control variables, such as
vehicle make or model, might be included in
the regression models?
[Chen and Kockelman] [1] Vehicle height, a variable which may be more valuable than vehicle type for
similarly structured vehicles such as sedans, wagons, CUVs, and minivans, would be a valuable control variable.
In addition to a wider track, a lower center of gravity also increases vehicle stability, thereby reducing the risk
of rollover. Relevant literature & findings exist, and should be cited.
[LBNL] The propensity for a vehicle to roll over increases as its center of gravity increases (and static stability
factor decreases), while risk in frontal crashes may decline as bumper height increases (and the degree of
overlap with the bumper of a crash partner increases). However, overall vehicle height is only a crude proxy for
these two other vehicle dimensions that are thought to correlate with safety, and therefore was not included in
the regression models.
[2] Other variables which have been found in past studies to influence fatality risk such as seat belt use,
167
-------
roadway geometry and division type are not included in this study (which is largely a repeat of the NHTSA
study, as specifically contracted by the EPA).
[LBNL] Seat belt use was not included as a control variable because it is notoriously under-reported for non-
fatal injuries. Roadway geometry and roadway division were not included because they are not consistently
reported or coded in the state databases of police-reported crashes used to develop the induced exposure
cases.
[3] To account for driver characteristics that contribute to fatality risk, socioeconomic variables such as
household income, education, household size, etc. would be valuable additions. Unfortunately, both state and
national crash databases typically do not include such information (outside of MVOSS). Such issues should be
flagged for readers. It seems the contractor has done his duty, and the key limitations lie with the original
methodology he was to essentially duplicate.
[LBNL] As discussed above, LBNL believes that driver behavior in the current crash and their recent driving
record, and not general socio-economic information, is the best available measure to account for the effect
driver skill and behavior have on risk in individual vehicles. As described in Section 5.4, LBNL used this measure
of driver behavior in one of its sensitivity analyses in the draft final report. Nonetheless, LBNL examined the
effect of including a measure of driver household income on risk in Section 5.4 in the Final Phase 1 report.
Please comment on any caveats or limitations
that these dependent variable or control
variables entail with respect to use of the
results as the basis for estimating the safety
effect of mass reduction.
[Chen and Kockelman] Please see above comment (in Assumptions section) regarding driver behavior and
environment.
[LBNL] As discussed above, an additional sensitivity analysis was run using a control variable for household
income, based on California vehicle registration data in 2010.
[Farmer] Model overspecification could be the reason for results that are non-intuitive, especially in the Phase
II analyses of police-reported crashes. Control variables may be correlated with each other or with the size
and weight variables. For example, Figure 2.9 of the Phase I report implies that torso side airbags increase
fatality risk in CUVs.
[LBNL] It is possible that inclusion of too many control variables leads to non-intuitive results in certain cases.
This problem might be resolved by removing control variables (other than vehicle weight or footprint) that are
168
-------
not statistically significant. The example cited suggests that torso side airbags increase fatality risk in
CUVs/minivans by an estimated 0.7%, is a small effect, and not statistically-significant. NHTSA and LBNL ran a
sensitivity where non-significant control variables are removed; this sensitivity has only a small effect on the
original estimates, as shown in Alternative 19 in Table ES.3 and in Section 5.6.
ADDITIONAL COMMENTS:
[Chen and Kockelman] Table 2.1 has many indicator variables labeled as "C" for continuous variable (such as ABS, ESC, AWD, DRVMALE, etc). These C's should
be removed.
[LBNL] These vehicle variables are continuous because, for some models, the VIN does not indicate whether a particular vehicle is equipped with that option or
not. In these cases the fraction of that model that is equipped with the particular feature is used. On the other hand, the DRVMALE variable is always a discrete
variable, and is now coded as such in the table.
[Van Auken] The underlying reasons for some of the estimated effects are unknown at this time, but presumably involve driver, vehicles, environment or
accident factors that have not been controlled for in the Kahane (2011) and Wenzel analyses. See, for example, Refs 17 and 25.
[LBNL] The report notes that other vehicle, driver, or crash characteristics may account for the remaining residual risk not explained by the control variables
included in the regression models. As discussed above, additional sensitivity analyses were run using an alternative control variable for vehicle characteristics,
initial vehicle purchase price (using values in Folk's VIN decoding software), and an alternative control variable for driver behavior, household income (based on
California vehicle registration data in 2010). The results of these sensitivities are included in Sections 5.2 and 5.4 in the Final Phase 1 report.
169
-------
3. METHODOLOGY AND STATISTICS
COMMENTS
Please comment on the validity and
applicability of the methodology LBNL used in
assessing the NHTSA 2011 study and its
analysis of the relationship between mass,
footprint, and risks per police-reported crash.
[Chen and Kockelman] The report assesses the NHTSA 2011 study in a fair amount of detail and seeks to
introduce some additional analyses to better examine the relationship between mass, footprint, and fatality
risks. However, due to a lack of control for very specific vehicle differences (which vary by make & sub-model),
the exclusion of driver characteristics and crash setting details (which cannot always be controlled for, but are
often correlated with vehicle type), the effects of downweighting vehicles and/or shifting vehicle styles and
sizes may be overestimated. Simply changing the vehicle on a risky driver in a high-risk setting is unlikely to
influence outcomes significantly.
[LBNL] As discussed above, additional sensitivity analyses were run using an alternative control variable for
vehicle characteristics, initial vehicle purchase price (using values in Folk's VIN decoding software), and an
alternative control variable for driver behavior, household income (based on California vehicle registration data
in 2010). The results of these sensitivities are included in Sections 5.2 and 5.4 in the Final Phase 1 report.
[Van Auken] The logistic regression methods seem to be appropriate. The confidence intervals are based on
the logistic regression Wald Chi-Square statistic, which as Kahane (2003, 2011) has demonstrated does not
include all sources of variation. However, these confidence intervals are useful because they do provide some
indication of the uncertainty in the results. [Note: This peer review does not address the Wenzel (2011b)
companion report, which examines the risks per police-reported crash. See Ref 4 for comments on the
companion report.]
Please review other statistical methods LBNL
has used in the analysis, in addition to the
logistic regression methodology. Examples
include the alternative approaches used by
LBNL to assess NHTSA interval estimation
results, and LBNL's linear regression analysis
of actual, predicted, and residual risk by
vehicle model.
[Chen and Kockelman] [1] In the alternative measures of exposure, the author examines the effect of vehicle
manufacturer on fatality risk and treats the luxury models produced by Toyota, Honda, and Nissan as separate
manufacturers. However, domestic luxury brands (such as Cadillac & Lincoln) are categorized with their
nameplate manufacturers (GM and Ford), which appears inconsistent.
[LBNL] The analysis has been revised to include the luxury models Lexus, Acura, and Infiniti with their
nameplate manufacturers Toyota, Honda, and Nissan. An additional sensitivity was conducted using separate
indicator variables for the five luxury brands Lexus, Acura, Infiniti, Cadillac, and Lincoln. Adding variables for
the five luxury brands increases the estimated detrimental effect of mass reduction on risk for lighter cars (from
1.55% to 2.04 %), and dramatically increases the estimated detrimental effect of mass reduction on risk for
heaver cars (from 0.51% in NHTSA's preferred model, to 0.75% after including 14 major vehicle makes, to 1.80%
after including 14 makes plus the five luxury brands), as shown in Alternative 8 in Table ES.2. The vehicle
manufacturer variables have little effect on the estimated effect of lighter light truck mass reduction, but both
170
-------
analyses reduce the slightly beneficial estimated effect of mass reduction in heavier light trucks. Including the
manufacturer variables changes the estimated effect of mass reduction in CUVs/minivansfrom an estimated
0.38% reduction in risk to an estimated 1.62% or 1.28% increase in risk).
LBNL also examined the effect of replacing the vehicle make indicator variables with a single continuous
variable for vehicle initial purchase price, taken from the Polk VIN decoder. Initial purchase price can be seen as
a proxy for the general quality of vehicle design; we expect that more expensive vehicles will have lower risk
than less expensive vehicles. Replacing the vehicle make variables with the variable for purchase price slightly
reduces the estimated effect of mass reduction on lighter cars and light trucks, increases the estimated effect of
mass reduction in heavier cars (from 0.51% to 0.84%), and substantially decreases the estimated effect of mass
reduction in CUVs/minivans (from a 0.38% reduction in risk to a 0.92% reduction in risk). This analysis is
summarized in Section 5.2 of the Final Phase 1 report.
[2] The effect of calendar year variables on fatality risk may be overestimated here, since VMT is tracked by
vehicle model and not by calendar year. The trend of greatest fatality risk reductions in light trucks, CUVs and
minivans with increasing calendar year may simply be a reflection of rising gas prices in combination with the
ailing economy contributing to lower VMT (in these relatively low-fuel-economy vehicles).
[LBNL] As discussed in Section 5.3 of the LBNL Final Phase 1 report, the VMT weights NHTSA developed do not
reflect the reduction in driving that occurred in 2008 (as shown in Figure 5.9). Therefore, the large reduction in
risk estimated by the CY08 variables likely is the result of less driving in that year, and not the result of
widespread changes in the fleet of vehicles on the road as potential crash partners, as NHTSA has
surmised/postulated. However, the lower VMT in CY08 does not explain the consistent trend in reduced
estimated fatality risk over the other calendar years.
[3] It is unclear how the author determined the various percentage replacements of vehicle types in the
aggressive vehicle market share shift scenario. (For example, why are 50% of SUVs replaced by CUVs and 60%
of small pickups replaced by CUVs? The CUV is a more natural replacement for an SUV, and an SUV a more
natural replacement for a pickup.)
The aggressive shift in market share scenario was selected to maximize replacement of truck-based vehicles,
such as pickups and SUVs, with car-based vehicles, such as CUVs and minivans, that tend to be safer and have
171
-------
higher fuel economy than truck-based vehicles.
[Van Auken] [1] The correlations in Section 3 appear to be assessed using the Coefficient of Multiple
Determination (R2) based on a linear fit to the data (e.g., the correlation between footprint versus curb weight
in Figure 3.1 on p. 14). The linear regression model attributes the differences between the dependent variable
(vertical axis_ and the linear fit to the independent variable (horizontal axis) to random effects. If there is no
preference as to the choice of independent and dependent variables (e.g., footprint versus curb weight, or
curb weight versus footprint), then the linear trend and R2 result would be different if the two variables were
interchanged, and having two different yet equally valid results would be undesirable.
[LBNL] The R2 values are the same if the two variables are interchanged.
[2] If the variation in the data can be attributed to both variables (e.g., footprint and curb weight), then it
would be better to report the square of the sample correlation coefficient r2,, where r is computed according
to Eqn (1). The trend lines in these correlation figures should not be computed using a linear regression.
Instead, the trend line should pass through the sample means (I.e. (x, y)), and have a slope equal to the ratio of
the sample standard deviations in the data (i.e., sy/sx). Therefore, the reported correlation results do not
depend on the ordering of the data variables.
[LBNL] Again, the R2 values are the same if the two variables are interchanged. The R2 values in Figures 3.1 and
3.2, and Figures 4.1 through 4.5, have been replaced with the correlation coefficient r, and a new Table 3.1 has
been added to Section 3 that lists r and VIFfor curb weight and footprint by vehicle type.
Note this comments does not apply to linear trends indicated in Section 4, for which the Coefficient of Multiple
Determination (R2) seems appropriate.
[3] The Coefficient of Multiple Determination (R2) is frequently used in the Wenzel (2011a) report as an
indicator of the statistical importance of a linear trend (e.g., lvalues in Tables 4.1 and 4.2 that are greater
than 0.3 are shown in blue font). It would be better to report the standard error, confidence interval, and/or
probability value as measures of the statistical significance of a linear trend.
[LBNL] Whether the estimated relationships shown in Tables 4.1 and 4.2 in the Final Draft report are
statistically significant has been added to the tables (see new Tables 4.2 and 4.4 in the Final Phase 1 report).
However, the point of the section is to indicate that, while the relationship may be statistically significant on
172
-------
average, as indicated by the confidence level around the estimated effect, there remains a large range in risk
for individual vehicle models with similar mass, as indicated by the relatively low R2 values.
Please comment on caveats or limitations of
using non-significant regression estimates to
project the safety impact of mass reduction.
[Chen and Kockelman] First, the t-statistics are not provided in the report which makes it difficult for the
reader to assess statistical significance of specific regression estimates (except where noted by the author).
Second, inclusion of a statistically insignificant variable can influence the estimates of coefficients associated
with related variables. Nevertheless, in general, it is best to keep insignificant estimates if one has a strong
defense for their role, since removing such variables (& thus their parameters) will shift the burden of
response to a correlated covariate's parameter, thus biasing the latter. We generally keep key covariates in a
model up to a pvalue of 0.20 or 0.25 or so, especially in relatively small data sets (e.g., n < 1,000). Covariates
for which we have no strong basis can be removed for p values > 0.10.
[LBNL] The 95% confidence intervals are included in every figure in the document, to indicate whether the
estimated variable is statistically different from zero.
NHTSA is planning to run a sensitivity where non-significant control variables are removed; LBNL will run similar
sensitivities for its revisions to its Phase 1 and Phase 2 reports.
[Van Auken] Regression estimates are random numbers which have an unknown expected value and variance,
and known sample value and standard error. If the sample value can be explained by a zero expected value
and known standard error then the result is considered not statistically significantly different than zero and
therefore the result is not considered to be statistically significant. However, If we can combine this estimate
with other estimates then the unknown expected values and variances can also be combined using the same
transformation, and the statistical significance of the combined result can be tested. Therefore, depending on
the sample values and inter-correlation, the combined result may be statistically significant even if the
individual estimates are not statistically significant.
For example, the results from each of the nine different crash types can be combined into an overall estimate
and the standard error calculated assuming that the results for each crash type are independent of each other.
Then the statistical significance of the combined effect can be determined.
However, and Kahane (2011) points out there are two sources of uncertainty in the regression results. The
first is the FARS based sampling error which is uncorrelated across crash types because they are based on
173
-------
different fatal cases (Kahane 2011, p. 77). The second is the state based include-exposure sampling error
which is correlated across crash types because they are based on the same included-exposure cases.
Therefore a confidence interval estimated using the jackknife method described by Kahane (2011) and
accounting for correlation of these two error sources would be more accurate than a simple estimate based on
the Wald Chi-Square statistic and assumed independence.
[LBNL] LBNL agrees that NHTSA's jack-knife method is a preferable method of estimating confidence intervals;
however, the jack-knife method obtains even larger estimates of uncertainty than using the standard errors
output from the regression models.
How might the LBNL methodology be
strengthened to better represent future
vehicle designs and reduce multi-collinearity
between mass and footprint in the regression
analysis?
[Chen and Kockelman] Including more vehicle-specific characteristics (such as vehicle height and engine size)
reduces the analysis' dependence on vehicle type, since vehicle shapes and structures will continue to evolve.
There is also correlation with context (e.g., pickups are driven in more rural locations, with greater hazards
[like less lighting, higher speed, & few medians]). Disaggregate data are almost always best, to avoid ecological
fallacies & such.
[LBNL] As discussed above, overall vehicle height is only a crude proxy for two vehicle dimensions, center of
gravity and bumper height, that are thought to correlate with safety in rollover and two-vehicle crashes,
respectively, and therefore was not included in the regression models. Other vehicle attributes that might
affect risk are engine power-to-weight ratio, braking distance, and handling capabilities; LBNL may examine
the effect of these accounting for these vehicle attributes in future analyses.
LBNL's analysis does account for whether a vehicle is driven in a rural area, defined as a county in which the
population density is less than 250 residents per square mile of land area.
[Van Auken] [1] The effects of multi-collinearity can be mitigated by 1) obtaining more data, 2) pooling data
from different crash type or vehicle types, or 3) reducing the number of regression variables. The first option
would require more calendar years and/or model years, which would involve added newer data as it becomes
available (or using older data). The second option might be to recombine the CUVs and minivans with truck
based vans and adding a control variable to compensate for the differences in the vehicles types. The third
option might involve removing statistically insignificant control variables or removing control variables that
would not be expected to have an effect on the probability of fatality in the crash (e.g., the side airbag variable
174
-------
is not included in pedestrian crashes because it is not expected to affect pedestrian fatality risk). The number
of driver age control variables might be reduced from eight to three (as in the Kahane (1997) and DRI (2002-
2005) studies). Finally, a linear curb weight model instead of a two-piece linear model may help to better
elucidate the general trend.
[2] The Variance Inflation Factor (VIF) has been suggested as a measure of multi-collinearity in the Kahane
(2010 and 2011) reports, however this diagnostic metric does not account for differences in database size (i.e.,
Options 1 and 2 above). The Wenzel (2011a) report does not discuss the Variance Inflation Factor or report
any VIF results.
[LBNL] A table of VIF results, Table 3.1, has been added to the Final Phase 1 report.
ADDITIONAL COMMENTS:
[Chen and Kockelman] On page 55, it is unclear what is meant by "however; if anything, reduction of this type of fatality will increase detrimental effect of
mass reduction in cars."
[LBNL] Full adoption of ESC is expected to reduce the number of, and fatalities in, rollovers and one-vehicle crashes. Because rollovers and one-vehicle crashes
are the only types of crashes in which mass reduction is estimated to reduce fatality risk, reducing the number of these fatalities by full adoption of ESC
increases the net estimated effect of mass reduction over all types of crashes. Similarly, full adoption of side airbags is estimated to reduce the number of
fatalities in side impact crashes, as least for cars (Figure 6.4). However, Figure 6.7 indicates that the estimated effect of mass reduction on risk when a car is
struck in the side is the same or lower than when a car is involved in a fatal crash. Therefore the reduction in the number of side impact fatalities expected from
full adoption of side airbags will likely increase the estimated effect of mass reduction on risk across all types of crashes.
175
-------
4. DATA SETS
COMMENTS
Please comment on the validity and
applicability of the datasets used to project
changes in risk resulting from reduction in
vehicle mass. LBNL's casualty analysis used
police-reported crash data from 16 states,
while the 2011 NHTSA study used national
fatality data, combined with a subset of non-
culpable vehicles involved in two-vehicle
crashes from police-reported crash data from
13 states.
[Chen and Kockelman] [1] The acquisition of Polk data for VMT estimates by make & model is valuable, and a
contribution to the literature. However, these estimates come from vehicles found in repair shops in
non-attainment areas, and so will be biased towards problem-prone vehicles, wealthier households who
service their vehicles more regularly, and/or urban (smoggier) areas. Such issues merit careful discussion in the
paper, so that readers are well aware of caveats.
[LBNL] The largest source of vehicles in the Polk odometer data are vehicles that report for a regularly-
scheduled vehicle emission inspection and maintenance (I/M) program inspection. In urban areas with poor air
quality, all eligible vehicles are required to report for inspection every one or two years. While the Polk VMT
data may be skewed towards urban areas (which typically include suburban areas, and sometimes rural areas,
depending on the state I/M program), they are not biased towards problem-prone vehicles or those owned by
wealthier households.
[2] Related to this, Tom Wenzel indicated (by phone) that he did take a look at CA's extensive odometer reads,
which go into some semi-rural locations (not too rural), and he indicated that the VMT values by vehicle type
(not controlling for HH attributes & such) are very similar (just 5% longer in rural areas) -except for vans
(which are used much more extensively in rural areas). This is interesting to me, and is not that different from
what we've seen in the past. For example, Kockelman & Zhao's JTS paper from 2000 (pre- print at
http://www.ce.utexas.edu/prof/kockelman/public_html/
BTSJournalLDTs.pdf) suggests that, after controlling for various HH attributes & vehicle types, density is/was
still very important (tables 1 & 2), but a shift from a density of Ik to 5k persons per sq mile (which is 1.5 vs. 7.8
persons per acre) means an increase of 750 mi/yr/vehicle (which is about 7.5% of annual VMT). Such
differences, and their practical significance (or lack thereof) should be discussed in the reports.
[LBNL] The vehicle mile traveled weights NHTSA assigns to each vehicle in the induced exposure dataset are
based on the average odometer reading by vehicle make, model, and year from a database of odometer
readings provided by Polk. Many of these odometer readings were obtained through state-run emissions
inspection/maintenance programs, which tend to operate in urban areas of states with poor air quality. In a
previous analysis LBNL (LBNL 2011a) found that vehicles registered in rural counties (<200 population per
square mile) are driven between 2% and 8% more miles than vehicles registered in urban counties (>200
population per square mile), depending on vehicle type (full size vans registered in urban counties are driven 3%
176
-------
more miles than those registered in rural counties; however this difference is not statistically significant).
Therefore the Polk database of odometer readings is likely skewed towards vehicles driven in urban areas,
which tend to be driven fewer miles than comparable vehicles registered in rural areas.
An additional limitation of the VMT weights developed by NHTSA is that they do not account for the reduction
in miles driven in response to higher gas prices and the economic recession in 2008, as noted in Section 5.3 of
LBNL's Final Phase 1 report. However, despite these limitations, the average VMT weights NHTSA has
developed are an improvement over the averages used in the 2003 analysis.
[Van Auken] The inducted-exposure data set provided by NHTSA is based on the non-culpable vehicles in two-
vehicle crashes. See the comments in Table 1 on the limitations of this data. In addition, there are also many
differences in the coding variables and values used by the different states, which tend to make the receding to
a common data set imprecise. [Note: This peer review does not address the Wenzel (2011b) companion
report, which examines the risks per police-reported crash. See Ref 4 for comments on the companion report.]
[LBNL] As discussed above, NHTSA has developed vehicle miles traveled weighting factors for induced exposure
cases based on stopped, rather than non-culpable, vehicles. LBNL summarized the effect this alternative
method of exposure has on the estimated effect of mass or footprint reduction on risk in Alternative 15 in Table
ES.3, and Section 5.6, in the Final Phase 1 report.
Please comment on any apparent, unstated,
or implicit impact on estimated risks inherent
in the two different approaches, and any
related caveats or limitations. For example,
what are the strengths and weaknesses of the
two measures of vehicle exposure, miles of
vehicle traveled scaled up from crash data
from 13 states, and number of police-reported
crashes?
[Chen and Kockelman] The use of non-culpable vehicles in two-vehicle crashes as a proxy for vehicles which
are "just there" may be distorting the overall distribution of vehicle models. VMT may differ between vehicles
that are more prone to run-off-road accidents, at-fault two-car crash vehicles, and non-culpable vehicles.
[LBNL] As discussed above, NHTSA has developed vehicle miles traveled weighting factors for induced exposure
cases based on stopped, rather than non-culpable, vehicles. LBNL summarized the effect this alternative
method of exposure has on the estimated effect of mass or footprint reduction on risk in Alternative 15 in Table
ES.3, and Section 5.6, in the Final Phase 1 report.
[Van Auken] [1] The number of fatal cases tends to be much less than the number of induced-exposure cases.
Therefore the effective numbers of degrees-of-freedom in the statistical estimates tend to be limited by the
177
-------
available number of fatal cases. For example, it would not be possible to estimate the effects of two variables
(e.g., just curb weight and footprint) if we had data for only one fatal case even if we had thousands of
induced-exposure cases. Therefore it is desirable to use data for the entire US in order to get a large sample of
fatal cases for the logistic regressions. This then requires the available induced-exposure data (i.e., from 13
states) to be "scaled up" the US level using the method described in Kahane (2003 and 2011). The result is the
best currently available estimate of vehicle exposure.
[2] There may be some concerns about the accuracy of the vehicle miles-traveled data because the difficulty
estimating the number of vehicle miles travelled at the make-model-year=state level of detail. [Note: This
peer review does not address the Wenzel (2011b) companion report, which examines the risks per police-
reported crash. See Ref 4 for comments on the companion report.]
ADDITIONAL COMMENTS:
178
-------
5. RECOMMENDATIONS
COMMENTS
Please comment on whether the LBNL
assessment adequately addresses the NHTSA
2011 study and identifies the safety impact
from mass reduction. Are the analytic
methods and data used to assess the NHTSA
study, and estimate the relationship between
risk, mass, and footprint, appropriate? Is
casualty risk per crash a legitimate measure of
vehicle safety? What other methods or data
could be used to better predict the effect of
future vehicle designs on safety?
[Chen and Kockelman] While driver fatalities per crash seems a useful measure of vehicle design safety, and
examination of fatal crash rates is very valuable (using Polk- based exposure estimates), there are many
caveats to work of this type. As noted above: a primary concern remains a neglect of the notion that the type
of car is very much a proxy for driver type, and a vehicle's crash avoidance may have very little to do with
vehicle type. It has a lot to do with the person behind the wheel. Simply including gender and age variables
cannot account for important covariates such as education, risk aversion, driving ability, wealth, etc. In reality,
small cars may be less crash prone than Kahane's and Wenzel's results suggest, because they are driven by
lower-income, younger, less risk averse people driving in more crash prone settings (e.g., commercial strips
rather than pricey residential suburbs). Of course, as noted above, it is very difficult to control for all these
variables, and the contractor was asked to rely on the original data. In reality, the best the report authors can
do with such data sets is to explain how all the other, relevant attributes may factor in (e.g., quality of driver
and typical driving settings), and how they can generate biased estimation (sometimes in either direction).
Discussion of relevant literature that looks more deeply at crash outcomes (e.g., Wang or Chen's papers,
mentioned above, allowing for heteroscedasticity and individual vehicle attributes, non-driver outcomes, etc.)
will also be useful.
[LBNL] As discussed above, additional sensitivity analyses were run using a control variable for household
income, based on California vehicle registration data in 2010 (see Section 5.4 in the Final Phase 1 report). And
sensitivity analyses were also run using a control variable for initial vehicle purchase price as a proxy for quality
of vehicle design, using values in Folk's VIN decoding software (see Section 5.2 in the Final Phase 1 report).
[Van Auken] [1] The basic methodology described by Kahane (2011) seems appropriate; however some
results using this method and data are not well understood and need further diagnosis.
[2] The induced-exposure data set provided by NHTSA is based on the non-culpable vehicles in two-vehicle
crashes. See the Table 1 comments on the limitations of this data. [Note: This peer review does not address
the Wenzel (2011b) companion report, which examines the risks per police-reported crash.]
[LBNL] As discussed above, NHTSA has developed vehicle miles traveled weighting factors for induced exposure
cases based on stopped, rather than non-culpable, vehicles. LBNL summarized the effect this alternative
method of exposure has on the estimated effect of mass or footprint reduction on risk in Alternative 15 in Table
ES.3, and Section 5.6, in the Final Phase 1 report.
179
-------
Please comment on the overall adequacy of
LBNL's assessment of the 2011 NHTSA report
and its independent study of casualty risk for
predicting the effect of vehicle mass or
footprint reduction on safety. Provide any
recommended improvements that might
reasonably be adopted by the author to
improve the analysis.
[Chen and Kockelman] Overall, the study is a comprehensive assessment of the 2011 NHTSA report and
introduces interesting additional analyses to examine the relationship of vehicle mass and footprint reduction
on safety. However, as stated previously in the comments here, driver preference for specific car types
(including size and mass) is related to driver socioeconomic characteristics and driving behavior. As vehicle,
driver, and roadway environment characteristics all contribute to fatality risk, the effects of physical vehicle
changes such as mass or footprint reduction on safety should not be overstated when the other two types of
characteristics are not sufficiently accounted for.
[Farmer] Overall these are reasonably good studies. The Phase I report does a very good job of assessing the
NHTSA report of fatality risk.
[Van Auken] The Wenzel (2011a) report provides a valuable supplement to the analysis and results in the
Kahane (2011) report. [Note: This peer review does not address the Wenzel (2011b) companion report, which
examines the risks per police-reported crash.]
ADDITIONAL COMMENTS:
[Van Auken] See attached tables 6 and 7 below.
180
-------
Table 6. Additional General Comments and Recommendations
Mike Van Auken
Section
COMMENTS AND RECOMMENDATIONS
Use of R2 is confusing. Suggest using lower case "r" when referring to the sample correlation coefficient (Box, Hunter, Hunter, 1978,
P. 61); or upper case R when referring to the regression coefficient of multiple determination (Draper and Smith, 1981, p. 90).
[LBNL] Figures 3.1 and 3.2 and 4.1 through 4.5 report r and not R2 in the Final Phase 1 report. In addition, a new Table 3.1 with r and
VIF, and a new Table 4.1 with r and R2, have been added.
All
In most cases the reported results are just estimates, but are not described as such. For example, "The effect of mass reduction on
heavier cars and CUVs and minivans are not statistically significant" on p. iii should say "The estimated effect of mass reduction on
heavier cars and CUVs and minivans are not statistically significant."
This distinction is important when comparing results based on different models and assumptions because the different models and
assumptions do not change the effect itself, but rather the estimate of the effect. For example, the statement "The first sensitivity,
in dark purple, includes the weight variables in the regression model but excludes the footprint variable; this model tests the effect
of mass reduction while allowing footprint to vary with vehicle mass. This sensitivity increases the risk from a 100-lb mass reduction
in cars (from 1.43% to 2.64% for lighter cars, and from 0.48% to 1.94% for heavier cars) and CUVs/minivans (from a 0.47% decrease
in risk to a 0.52% increase in risk); however, there is no change in fatality risk in light-duty trucks" on page 15 is misleading. It would
be better to state that "The first sensitivity, in dark purple, includes the weight variables in the regression model but excludes the
footprint variable; this model tests the estimated effect of mass reduction while allowing footprint to vary with vehicle mass. Th4&
GGnsitivity/?emoi//'ng the footprint variable from the regression model increases the estimated risk from a 100-lb mass reduction in
cars (from 1.43% to 2.64% for lighter cars, and from 0.48% to 1.94% for heavier cars) and CUVs/minivans (from a 0.47% decrease in
risk to a 0.52% increase in risk); however, there is nothe change in the estimated fatality risk in light-duty trucks is very small and
not statistically significant."
This also applies to table and figure captions. For example, "Table ES.l. Effect of mass and footprint reduction on fatality risk, under
alternative regression model specifications" should say "Table ES.l. Estimated effects of mass and footprint reduction on fatality
risk, under alternative regression model specifications." "Figure 3.3 Effect of reduction in mass or footprint on US fatality risk per
VMT, by vehicle type: mass only, footprint only, and both" should say "Figure 3.3 Estimated effects of reduction in mass or footprint
on US fatality risk per VMT, by vehicle type: mass only, footprint only, and both."
Overall the word "effect" appears over 200 times in this report with the "estimated" or other qualifier. In some cases this may be
181
-------
appropriate and in other cases it is not appropriate. It is recommended that the author review each instance and revise as
appropriate.
[LBNL] The word "estimated" will be used extensively in the final report. In addition the following text was added in the Executive
Summary, Section 1, and Section 7:
Although the purpose of the NHTSA and LBNL reports is to estimate the effect of vehicle mass reduction on societal risk, this is not
exactly what the regression models are estimating. Rather, they are estimating the recent historical relationship between mass and
risk, after accounting for most measurable differences between vehicles, drivers, and crash times and locations. In essence, the
regression models are comparing the risk of a 2600-lb Dodge Neon with that of a 2500-lb Honda Civic, after attempting to account
for all other differences between the two vehicles. The models are not estimating the effect of literally removing 100 Ibsfrom the
Neon, leaving everything else unchanged.
In addition, the analyses are based on the relationship of vehicle mass and footprint on risk for recent vehicle designs (model year
2000 to 2007). These relationships may or may not continue into the future as manufacturers utilize new vehicle designs and
incorporate new technologies, such as more extensive use of strong lightweight materials and specific safety technologies. Therefore,
throughout this report we use the phrase "the estimated effect of mass (or footprint) reduction on risk" as shorthand for "the
estimated change in risk as a function of its relationship to mass (or footprint) for vehicle models of recent design."
Figures 4.1 through 4.17 do not control for the effect of vehicle size (e.g. footprint), which has been shown to be correlated with
vehicle weight (e.g., Figure 3.1), and therefore these figures may be misleading. It is strongly suggested that the horizontal axis label
be changed to "Curb weight (Ibs) and corresponding changes in size," and/or a note such as the following be added to each figure:
"Note these results do not control for the effect of vehicle size on fatality risk. Therefore the horizontal axis represents changes to
both vehicle weight and vehicle size."
[LBNL] The figures, as well as Tables 4.2 and 4.4, show the relationship between risk and vehicle mass, before and after accounting
for all modelled differences across vehicle models except mass and footprint, and are labelled as such. A new Table 4.3 has been
added to the final report that compares predicted risk after accounting for all modelled differences across vehicle models except mass
(but including footprint), and a new Table 4.5 shows the relationship between predicted risk and footprint after accounting for all
differences except footprint (but including mass); this increases the estimated risks in some cases, makes some statistically
significant, but does not appreciably reduce the range in risk by vehicle model.
The statistical significance of the linear trends in Figures 4.1 through 4.17 are not reported. It would be helpful if the confidence
182
-------
intervals or statistical significance of the linear trends were reported, either in addition to or instead of R2.
The confidence intervals for the estimated slopes should be added to the results in Tables 4.1 and 4.2.
[LBNL] Tables 4.2 through 4.5 in the Final Phase 1 report summarize the linear trends and statistical significance of Figures 4.1
through 4.17.
183
-------
Table 7. Additional Specific Comments and Recommendations
Mike Van Auken
Section
Page
COMMENTS AND RECOMMENDATIONS
Executive
Summary, 7
iii, 65
-,nd
2n paragraph refers to "our analysis," however the results are the same as the NHTSA analysis. The author
should clarify who or what "our analysis" refers to and how it relates to the NHTSA analysis. Perhaps the
statement "LBNL was able to reproduce the NHTSA analysis, which finds that..." would be more appropriate.
[LBNL] The suggested change has been made.
Executive
Summary
IV
Last bullet - suggest changing the statement that "Logistic regression does not allow a statistic" to "Logistic
regression methods do not have a statistic."
[LBNL] The suggested change has been made.
Executive
Summary, 4
iv, v, 22, 66
Suggest changing "variance in risk" to variation in risk" throughout.
[LBNL] The suggested change has been made.
Executive
Summary, 7
viii, 69
The numerical results for the NHTSA preferred model in Tables ES.l and 7.1 are slightly different than the results
reported in the NHTSA report. For example 1.43%/0.48%/0.52%/-0.40%/-0.47% should be 1.44%/0.47%/0.52%/-
0.39%/-0.46%
[LBNL] We are not sure why LBNL's results are slightly different from NHTSA's; perhaps due to rounding
differences, or the factor LBNL used to convert log-odds ratios to probabilities, as mentioned in the Final Phase 1
report.
31-32
Figures 4.12 through 4.14 have the results for small and heavy-duty pickups combined, which is inconsistent with
the results in Table 4.1
[LBNL] The figures in the Final Phase 1 report present the data for small and heavy-duty pickups separately.
35
The R2 values in Table 4.1 are different than the values in Figures 4.6, 4.8, 4.9, 4.11.
[LBNL] The R2 values in the bottom four rows of the table are incorrect; these have been corrected in the Final
Phase 1 report.
184
-------
5.1
37
The subsection title should be "Alternative measures of exposure and outcome" because fatal crashes and
fatalities are measures of the crash outcome, not exposure.
[LBNL] The title has been changed to "Alternative measures of risk"
5.2
39
It would be helpful to list the 18 manufacturer dummy variables in a table.
[LBNL] The indicator variables for the vehicle manufacturers have been added to the final report.
5.2
39
It is unclear why Lexus, Acura, and Infinity are treated as separate manufacturers, but Cadillac and Lincoln are
not.
[LBNL] The results of two regressions have been included in the Final Phase 1 report: one including the five luxury
brands in their parent manufacturers, and one accounting for each of the five luxury brands (both in Figure 5.2).
Accounting for the five luxury brands substantially increases the estimated detrimental effect of car mass
reduction on risk, as shown in Section 5.3 in the Final Phase 1 report.
5.2
39
It is unclear why AM General is considered a Chrysler brand. The AM General Hummer was sold by GM
beginning with the 2001 model year.
[LBNL] AM General has been removed from the Chrysler brand and included in the Other category.
5.3
44
It would be helpful if the figures include error bars or shading to indicate the confidence intervals.
6.4
63
Table 6.3 - Suggest adding a note that the 72,316 total includes fatalities that are counted more than once in
crashes involving more than one vehicle type.
[LBNL] Table 6.3 is mis-labelled; it represents the total number of fatalities between 2004 and 2008 for model
year 2000 to 2007 light duty vehicles, not the average annual fatalities. The text in the paragraph above explains
that the number of fatalities in mulit-vehicle crashes is divided by the number of case vehicles, to avoid double-
counting of fatalities. The title of Table 6.3 has been corrected in the Final Phase 1 report, and VMTfor each
vehicle type added.
185
-------
An Analysis of the Relationship between Casualty Risk Per Crash and Vehicle Mass and Footprint
for Model Year 2000-2007 Light-Duty Vehicles
(LBNL Phase 2 Report)
1. ASSUMPTIONS
COMMENTS
Please comment on the validity of any
assumptions embedded in the LBNL
assessment analysis and the independent
casualty analysis that could affect the
projected relationship between vehicle
mass/footprint reductions and
fatality/casualty risk. Examples might include
assumptions regarding whether recent
historical relationships between vehicle
weight, size, and safety will continue into the
future; potential future improvements in
vehicle technology and design may result in
compensatory safety benefits; and the annual
baseline fatality distribution.
[Chen and Kockelman] [1] The Phase 2 report serves as a complimentary document to the Phase 1 report by
isolating the effect of vehicle mass and footprint on crashworthiness. Whereas the Phase 1 report analyzes
fatality risk per estimated VMT, the Phase 2 report analyzes casualty risk per crash. The parallel structure of
the two reports makes it easy for the reader to compare the results of the two analyses.
[2] The binary logistic model employed in this analysis can only account for two injury outcome categories;
here it is used to distinguish crashes resulting in serious injury or death from all other crash outcomes. Thus,
the model does not account for the ordinal nature of injury severity and neglects the difference between a
serious injury and a death.
[LBNL] The LBNL Phase 2 report analyzes the relationships between vehicle mass, footprint, and casualties,
defined as fatalities plus serious or incapacitating injuries. The effects on other injury severities (i.e. minor
injuries) were not analyzed because of the differences in reporting of injury severity in the thirteen states.
[3] The report states that "a serious incapacitating injury can be just as traumatic to the victim and her family,
and costly from an economic perspective, as a fatality." While serious injuries are very costly to society (and
may have similar economic cost implications as deadly crashes), willingness to-pay estimates (which include
pain and suffering) price the cost of a fatality at almost 20 times the cost of an incapacitating injury (NSC
2010). Thus, it is difficult to assess the economic cost of the estimates of increases in casualty risk per crash
without distinguishing whetherthat outcome is a serious injury or a death. This limitation of the model should
be addressed in the report.
[LBNL] The focus of the analysis is to estimate the relationship between changes in vehicle mass or footprint on
fatality or injury risk. Estimating the economic cost of any estimated increase in risk is beyond the scope of the
analysis.
[4] The logistic model also assumes error-term homoscedasticity and cannot account for increases and
186
-------
decreases in the variation of injury outcomes due to vehicle and driver type, for example. Such limitations of
the model should be discussed.
[LBNL] As mentioned above, the effects on other injury severities (i.e. minor injuries) were not analyzed because
of the differences in reporting of injury severity in the thirteen states.
[Farmer] The report concludes "that much of the detrimental effect of mass or footprint reduction on risk can
be attributed to the tendency for mass or footprint reduction to increase crash frequency, rather than to
reduce vehicle crashworthiness (risk once a crash has occurred)." However, the interpretation of casualties
per crash as inversely proportional to crashworthiness ignores the possibility that injury severity also depends
upon the circumstances of the crash. Casualties per crash must be divided into casualties per severe crash and
severe crashes per crash, where a severe crash would be one involving more energy, e.g., high-speed or
rollover. It could be that weight reduction increases casualties per severe crash (i.e., reduces
crashworthiness), but reduces the likelihood that a crash is severe.
[LBNL] The state crash data do not consistently provide a measure of crash severity, such as an accurate
estimate of travel speed, to change the measure of exposure from all police-reported crashes to all severe
police-reported crashes. Figure 2.10 in the Final Phase 2 report shows that using casualty, as opposed to all,
crashes as the measure of exposure does change the estimated detrimental effect of mass reduction on fatality
risk per crash
[Van Auken ] The basic assumptions, methodology, and data are primarily the same as in the Kahane (2011)
report, but have been extended to include serious injuries as well as fatalities, and also address crash
involvement (i.e., fatalities and serious injuries per accident, and also accidents per VMT).. These include the
following:
1) The probability of a crash fatality or serious injury is proportional to the number of accidents (provided the
crash conditions remain the same); and the probability of an accident is proportional to the vehicle miles
travelled (VMT).
2) The logarithm of probabilities of fatality or serious injury per accident, and accidents per VMT for a given
curb weight, footprint, and control variable values varies as a linear combination of the curb weight,
footprint, and control variables within the domain of the data.
3) The logistic regression methods determine a maximum likelihood estimate of model coefficients.
4) It is assumed that the above relationships remain constant in the recent past (i.e., 2000-2007 model year
187
-------
vehicles in the 2002-2008 calendar years), present, and near future (i.e., 2017-2025 model year vehicles).
The first assumption that crash fatalities and serious injuries are proportional to the number of accidents
provided the crash conditions remain the same seems self-evident (e.g., if two fatal crashes had exactly the
same conditions, then the expected number of fatalities for the two crashes would be twice the value for just
one of the crashes). The assumption that the number of accidents are proportional to VMT rather than the
number of vehicle registration years (VRY) is also appropriate because accidents cannot occur if the vehicles
are not driven on the road (i.e., VMT = 0). This assumption is qualified however because VMT is more difficult
to measure than VRY and therefore may be less accurate. On the other hand the probability of a fatal crash or
the number of fatalities in a crash may also depend on the vehicle occupancy.
The second and third assumptions are appropriate provided that it is recognized that it is essentially
impossible with currently available knowledge and information to model all of the factors that could affect the
probability of fatality in a crash, and that the objective of the analysis is to identify overall trends versus
vehicle weight and footprint. In general the probability of fatality depends on other many other factors which
have not been modeled (e.g., driver behavior factors, vehicle design factors, roadway design factors, EMT
factors), and these unmodeled factors are assumed to be uncorrelated with vehicle weight and footprint,
and/or are represented by the other control variables. The latter assumption might or might not be valid.
The fourth assumption is perhaps the weakest because it assumes that future vehicles will have the same
design characteristics as past vehicles, and that the characteristics of the vehicle population (e.g., collision
partner weight, size, type) will also remain the same.
Please comment on any apparent unstated or
implicit assumptions and related caveats or
limitations.
[Chen and Kockelman] The role of driver behavior is briefly addressed in the report but not emphasized
sufficiently. Casualty risk is a combination of driver, vehicle, and roadway characteristics. Whereas vehicle
characteristics significantly influence crashworthiness, driver behavioral differences play a significant, if not
primary, role in determining crash frequency. Socioeconomic data such driver household income, size, and
education influence driver attitudes and driving environments. For example, Chen et al. (2010) found that
crash risk increases for those living in socioeconomically disadvantaged areas (including households more likely
to drive less expensive and older vehicles). Though such data is not typically available in state and national
crash databases, the importance of these driver and environmental characteristics on crash rates (per mile
driven) and casualty risk should be stressed in both reports. It is clearly very difficult to control for, but is a
major caveat to the NHTSA (& now LBNL) results. We expect that crash severity could be probably be lower for
many of the small cars and pickups if they were driven by those who tend to drive more expensive vehicles,
188
-------
under the same settings (e.g., daytime, urban freeway). Thus, statements like "a 100-lb reduction in the mass
of lighter cars leads to a 1.84% increase in crash frequency" should be accompanied by an explanation of the
possibility of the mass variable accounting/proxying for effects of lower income households owning smaller
vehicles.
[LBNL] We agree that it would be preferable to include additional variables that account for driver behavior
rather than just driver age and gender. However, variables such as driver age, gender, or other socio-economic
characteristics probably have a less direct effect on risk than the actual driving skill or behavior of individual
drivers. The PARS data includes information on whether alcohol or drug use, or speeding or reckless driving,
was a factor in the current crash, as well as the driver's record for the last three years. NHTSA 2003 found that
including a "bad driver" rating variable comprised of 8 of these variables did not have an appreciable effect on
the relationship between vehicle mass and risk. Although these variables are included in the PARS fatal crash
records, they are not included in the state crash databases, so a similar variable cannot be used in the
regression models. However, Alternative 12 in Tables ES.2 and 7.1, and the third column in Figure 5.14, in the
Final Phase 1 report indicates that excluding bad drivers (after also excluding reported driver alcohol or drug
use) results in additional estimated increases in fatality risk as vehicle mass decreases, and estimated decreases
in risk as vehicle footprint decreases. LBNL believes that driver behavior in the current crash and their recent
driving record, and not general socio-economic information, is the best available measure to account for the
effect driver skill and behavior has on risk in individual vehicles.
Nonetheless, LBNL explored the effect of adding a control variable for one additional driver socio-economic
characteristic, household income. Unfortunately, crash data do not include household income of vehicle
drivers. PARS includes the driver's zip code, from his or her license, which could be merged with US Census data
by zip code to obtain the average median income of households in a particular zip code. However, very few
states include driver zip code in their databases of police-reported crashes.
Kweon and Kockelman (2003) merge police-reported crash data from GES with household demographic data,
including household income, from the National Household Travel Survey (NHTS), to examine crash rates per
annual VMTby driver age, gender, and vehicle type. Each dataset includes population weights to derive
national estimates of crashes and households, which the authors use to estimate national crash rates for driver
age, gender and vehicle type. However, there are several limitations with the NHTS data. First, the survey
includes only 25,000 national households, supplemented by an additional 124,000 households from "add-on"
state and municipal jurisdictions. The national weights provided in the NHTS most likely do not represent the
national distribution of vehicle registrations by make and model. Second, 30% of the households in the 2009
189
-------
NHTS reported income "over $100,000"; using the income bins provided in the NHTS to estimate household
income will likely understate the actual income of the households participating in the survey.
Chen el al (2010) makes use of an Australian survey of 20,000 young adult drivers that asked questions
regarding driving behavior. The injury severity of 127 of these participants who were admitted to hospitals
after a vehicle crash was analyzed as a function of socio-economic status. The socio-economic status of the
drivers was based on average or median values from the zip code where they (or their parents, since most were
at University) lived.
Using 2010 California vehicle registration data, LBNL calculated the median household income by registered
owner's zip code, using 2000 census data, then took the average household income by vehicle make and model.
As discussed in the Final Phase 1 report, predicted US fatality risk is correlated with this measure of average
household income by vehicle model, with risk decreasing as income increases; the correlation is highest for cars
and CUVs, with an R2 of 0.60 (or a correlation coefficient r of 0.77) or higher, as shown in new Figure 4.20 and
the first column of new Table 4.7. However, while income tends to increase as mass increases, the correlation is
not strong (R2 is highest for 4-door cars, at 0.16, and CUVs, at 0.19, as shown in new Figure 4.21 and the middle
columns of new Table 4.7). As with predicted fatality risk, there is a wide range in the average income of
households owning vehicle models with similar mass. This analysis is summarized at the end of Section 4 of the
Final Phase 1 report.
Using average household income by vehicle model, rather than driver alcohol or drug use, or reckless driving, as
an alternative indicator of driver behavior substantially reduces the estimated detrimental effect of mass
reduction on risk in cars and CUVs/minivans, and has little effect on the relationship in light trucks. This
analysis is summarized in Section 5.5 of the Final Phase 2 report.
[Van Auken] [1] The induced-exposure data set provided by NHTSA is based on the "non-culpable" vehicle in
two-vehicle crashes. It is assumed that the dataset is a reprehensive sample of the driver and environmental
exposure factors for vehicle use. However, since these cases include moving vehicles, some vehicle-driver-
environmental conditions may be under or over represented in this data depending on how they affect the
ability of a non-culpable vehicle to avoid a crash. Results in Ref (17) indicated that the estimated effect of
weight and size reduction are sensitive to whether the induced-exposure data are based on the Kahane (2003)
non-culpable vehicle definition of the Kahane (1997) stopped vehicle definition. Unfortunately it is not
190
-------
currently possible to test this sensitivity with the NHTSA-provided induced-exposure data.
[LBNL] NHTSA has developed vehicle miles traveled weighting factors for induced exposure cases based on
stopped, rather than non-culpable, vehicles. LBNL investigated what effect this alternative method of exposure
has on the estimated effect of mass or footprint reduction on risk, in Alternative 15 in Table ES.3 and Section
5.6 in the Final Phase 1 report.
[2] It is also assumed that the accident data from the 13 or 16 states are representative of all US states. Figure
2.1 in Wenzel (2011b) provides a useful comparison of the distribution of fatalities in the US and 13 states by
the nine different crash types.
ADDITIONAL COMMENTS:
[Chen and Kockelman] National Safety Council (2010) Estimating the Costs of Unintentional Injuries. Available online at
http://www.nsc.org/news_resources/injury_and_death_statistics/Pages/
EstimatingtheCostsof Unintentional Injuries
Chen, H.Y., Ivers, R.Q., Mariniuk, A.L.C., Boufous, S., Senserrick, T., Woodward, M., Stevenson, M. and Norton R. Socioeconomic status and risk of car crash
injury, independent of place of residence and driving exposure: Results from the DRIVE study. Journal of Epidemiology and Community Health 64(10), 2010, pp.
998-1003.
[Farmer] NHTSA's fatality analysis covered calendar years 2001-08, but the casualty analysis excludes 2008. Such exclusion is understandable given that 2008
data were at the time unavailable for a majority of the states (I think they are available now). However, 2008 was an unusual year and may have affected the
size and weight effect estimates. The footnote on p. 6 of the Phase II report states that an analysis including the available 2008 data will be summarized in
Appendix A. I don't see Appendix A. Is an analysis planned including 2008 data?
[LBNL] The footnote is incorrect, there was no Appendix A in the final draft report. For the Final Phase 2 report LBNL included the 12 states that have provided
crash data for 2008 (all but Wyoming).
191
-------
2. CONTROL AND DEPENDENT
VARIABLES USED IN THE REGRESSION
MODELS
COMMENTS
Please comment on the adequacy of control
and dependent variables used in the
assessment analysis and independent casualty
analysis, and recommend any alternative
control or dependent variables that are
available for possible inclusion in the analysis.
For example, what are the relative merits of
the main dependent variables used, fatality
risk per estimated VMT, and casualty risk per
police-reported crash?
[Chen and Kockelman] [1] As alluded to above, a primary concern is that the NHTSA analysis (& thus the
LBNL analyses) largely neglect the idea that vehicle type (make & model) is very much a proxy for driver type,
and a vehicle's crash frequency may have very little to do with physical vehicle characteristics. It has a lot to do
with the person behind the wheel, and gender and age simply aren't enough to control for such distinctions.
Education, risk aversion, ability, wealth, etc., are important covariates. But existing data sets are quite limiting
(though the MVOSS & FAR with 3-year driver violation history do offer some valuable insights, not discussed in
these reports). In reality, small cars may be less crash prone than Kahane's & Wenzel's results suggest, because
they are driven by lower-income, younger, less risk averse people driving in more crash prone settings (e.g.,
commercial strips rather than pricey residential suburbs). Such key caveats need thoughtful discussion. Four
relevant papers on the topics of crash frequency and vehicle size-and-weight implications (by Knipling, Kweon
& Kockelman, Wang and Kockelman, and Chen & Kockelman) have been sent to Tom Wenzel. These all include
useful literature reviews for further connections to useful findings for citation in the reports, as time allows the
contractor.
[LBNL] As discussed above, LBNL believes that driver behavior in the current crash and their recent driving
record, and not general socio-economic information, is the best available measure to account for the effect
driver skill and behavior have on risk in individual vehicles. As described in Section 5.4 of the Final Phase 1
report, LBNL used these measures of driver behavior in one of its sensitivity analyses. Unfortunately, the states
do not consistently report this information in their crash databases. LBNL examined the effect of including a
measure of driver household income on risk in Section 5.5 in the Final Phase 2 report.
[2] Independent variables such as vehicle mass and footprint may be accounting for effects of driver
socioeconomic factors as discussed in the Assumptions section. Furthermore, vehicle option variables such as
AWD and side curtain airbags may be reflecting the effects of driver environment (e.g., those living in areas
with icy winters opting for AWD) and attitude (e.g., more risk-averse drivers opting for side curtain airbags)
rather than the vehicle technology themselves. While extremely heavy and extremely large vehicles may have
significantly different handling and braking characteristics which influence crash frequency and casualty risk, it
is unlikely that given the same driver in the same environment, a small change in vehicle mass or footprint
would influence the driver's crash proneness.
192
-------
[LBNL] We agree that it would be preferable to include other variables to control for individual driver behavior
and skill; unfortunately such data are not available.
[Farmer] One needs to restrict control variables to those that are available and reliable. A problem when
combining state databases is that the states often are not consistent as to the variables coded and the
definitions of those variables. This severely limits the list of possible control variables.
[LBNL] The control variables used from the state crash databases are either derived from the VIN, or based on
the driver age and gender, or the time or location of the crash. LBNL believes that these variables are
consistently coded among the thirteen states.
[Van Auken] [1] Reducing the total number of fatalities and serious injuries is desirable from a societal
viewpoint. This includes both subject vehicle occupant and collision partner (e.g., other vehicle occupant,
pedestrian) fatalities and serious injuries.
[2] VMT is a good measure of accident exposure provided that it can be accurately determined.
[3] The number of fatalities and serious injuries per accident is a measure of vehicle crashworthiness (i.e.,
effect of a crash on the subject vehicle occupants) and crash compatibility (i.e., effect of a crash on the other
vehicle occupants or vulnerable road users). Subject vehicle occupant fatalities and serious injuries per
accident are a measure of the subject vehicle crashworthiness. Collision partner fatalities and serious injuries
per accident are a measure of vehicle crash compatibility.
[4] The number of accidents per VMT is a measure of the crash avoidance capabilities of a given vehicle.
What additional control variables, such as
vehicle make or model, might be included in
the regression models?
[Chen and Kockelman] [1] Vehicle height, a variable which may be more valuable than vehicle type for
similarly structured vehicles such as sedans, wagons, CUVs, and minivans, would be a valuable control variable.
In addition to a wider track, a lower center of gravity also increases vehicle stability, thereby reducing the risk
of rollover. Relevant literature & findings exist, and should be cited.
[LBNL] The propensity for a vehicle to roll over increases as its center of gravity increases (and static stability
factor decreases), while risk in frontal crashes may decline as bumper height increases (and the degree of
overlap with the bumper of a crash partner increases). However, overall vehicle height is only a crude proxy for
these two other vehicle dimensions that are thought to correlate with safety, and therefore was not included in
193
-------
the regression models.
[2] Other variables which have been found in past studies to influence fatality risk such as seat belt use,
roadway geometry and division type are not included in this study.
[LBNL] Seat belt use was not included as a control variable because it is notoriously under-reported for non-
fatal injuries. Roadway geometry and roadway division were not included because they are not consistently
reported or coded in the state databases of police-reported crashes.
[3] To account for driver characteristics that contribute to casualty risk, socioeconomic variables such as
household income, education, household size, etc. would be valuable additions. Unfortunately, both state and
national crash databases typically do not include such information (outside of MVOSS). Such issues should be
flagged for readers.
[LBNL] As discussed above, an additional sensitivity analysis was run using a control variable for household
income, based on California vehicle registration data in 2010; see Section 5.5 in the Final Phase 2 report.
[Farmer] I think that already there are too many control variables in the regression models. Instead I would
consider defining different classifications of crash types. Table 2.2 of the Phase II report shows that the
distribution of crash types for casualty crashes is very different from that for fatal crashes.
[LBNL] It is possible that inclusion of too many control variables leads to non-intuitive results in certain cases.
This problem might be resolved by removing control variables (other than vehicle weight or footprint) that are
not statistically significant. NHTSA and LBNL ran a sensitivity where non-significant control variables were
removed; see Section 5.8 in the Final Phase 2 report.
Please comment on any caveats or limitations
that these dependent variable or control
variables entail with respect to use of the
results as the basis for estimating the safety
effect of mass reduction.
[Chen and Kockelman] Please see above comment (in Assumptions section) regarding driver behavior and
environment.
[LBNL] As discussed above, an additional sensitivity analysis was run using a control variable for household
income, based on California vehicle registration data in 2010; see Section 5.5 in the Final Phase 2 report.
194
-------
[Farmer] Model overspecification could be the reason for results that are non-intuitive, especially in the Phase
II analyses of police-reported crashes. Control variables may be correlated with each other or with the size
and weight variables. For example, Figure 2.9 of the Phase I report implies that torso side airbags increase
fatality risk in CUVs.
[LBNL] It is possible that inclusion of too many control variables leads to non-intuitive results in certain cases.
This problem might be resolved by removing control variables (other than vehicle weight or footprint) that are
not statistically significant. The example cited suggests that torso side airbags increase fatality risk in
CUVs/minivans by an estimated 0.7%, is a small effect, and not statistically-significant. LBNL ran a sensitivity
where non-significant control variables were removed; see Section 5.8 in the Final Phase 2 report.
ADDITIONAL COMMENTS:
[Chen and Kockelman] Table 2.1 has many indicator variables labeled as "C" for continuous variable (such as ABS, ESC, AWD, DRVMALE, etc). These C's should
be removed.
[LBNL] These vehicle variables are continuous because, for some models, the VIN does not indicate whether a particular vehicle is equipped with that option or
not. In these cases the fraction of that model that is equipped with the particular feature is used. On the other hand, the DRVMALE variable is always a discrete
variable, and is be coded as such in the table in the Final Phase 2 report.
[Farmer] The sensitivity results of Chapter 5 (Phase II) point out the extreme differences in results when changing the control variables. For example, including
vehicle make changes the effect of a 100-lb reduction in heavier cars from -0.91% to +0.55% (see Fig 5.3).
[Van Auken] The underlying reasons for some of the estimated effects are unknown at this time, but presumably involve driver, vehicle, environment or
accident factors than have not been controlled for in the Kahane (2011) and Wenzel (2011b) analyses. See, for example, Refs 17 and 23.
[LBNL] The report notes that other vehicle, driver, or crash characteristics may account for the residual risk not explained by the control variables included in the
regression models. As discussed above, additional sensitivity analyses were run using an alternative control variable for vehicle characteristics, initial vehicle
purchase price (using values in Folk's VIN decoding software), and a control variable for driver behavior, household income (based on California vehicle
registration data in 2010). The results of these sensitivities are shown in Sections 5.3 and 5.5 in the Final Phase 2 report.
195
-------
3. METHODOLOGY AND STATISTICS
COMMENTS
Please comment on the validity and
applicability of the methodology LBNL used in
assessing the NHTSA 2011 study and its
analysis of the relationship between mass,
footprint, and risks per police-reported crash.
[Chen and Kockelman] The Phase 2 report enhances the findings in the Phase 1 report by isolating the effect
of vehicle mass and footprint on crashworthiness. Like the Phase 1 report, this analysis goes into a fair amount
of detail and seeks to introduce additional analyses to better examine the relationship between mass,
footprint, and casualty risks. However, due to a lack of control for very specific vehicle differences (which vary
by make & sub-model), the exclusion of driver characteristics and crash setting details (which cannot always be
controlled for, but are often correlated with vehicle type), the effects of downweighting vehicles and/or
shifting vehicle styles and sizes may be overestimated. Simply changing the vehicle mass or footprint on a risky
driver in a high-risk setting is unlikely to influence crash outcomes significantly.
[LBNL] As discussed above, additional sensitivity analyses were run using an alternative control variable for
vehicle characteristics, initial vehicle purchase price (using values in Folk's VIN decoding software), and an
alternative control variable for driver behavior, household income (based on California vehicle registration data
in 2010). The results of these sensitivities are shown in Sections 5.3 and 5.5 in the Final Phase 2 report.
[Farmer] Figure 2.11 of the Phase II report implies that NHTSA's fitting of a separate regression model for each
of the 9 crash types was unnecessary, at least for the analysis of casualty risk per crash. I don't recall seeing a
similar analysis for fatality risk per VMT. Is it possible to get essentially the same results as the NHTSA study
using a single regression model?
[LBNL] Figure 2.1 of the Final Phase 1 report has the same analysis (although the second and third columns are
reversed). NHTSA ran a separate regression model for each of nine crash types in order to reweight the
estimated coefficients to reflect an estimated reduction in the number of fatal rollovers and crashes with a
stationary object from widespread adoption of ESC in the vehicle fleet. Running a single regression model
across all crash types obtains essentially the same result as running nine regression models.
[Van Auken] [1] The logistic regression methods seem to be appropriate. The confidence intervals are based
on the logistic regression Wald Chi-Square statistic, which as Kahane (2003, 2011) has demonstrated does not
include all sources of variation. However, these confidence intervals are useful because they do provide some
indication of the uncertainty in the results.
[2] The two-stage results for the 13-state fatalities per crash and 13-state crashes per VMT, and the one-stage
196
-------
result for 13-state fatalities per VMT reported in Tables ES.l and 6.1 and Figure 2.7 were computed using
independent logistic regressions. The differences between the two-stage results and the one-stage results for
fatalities per crash could have been eliminated by using the "simultaneous three-way" logistic regression
method described in DRI (2003). This method imposes the constraint that the combined two-stage estimated
and the one-stage estimated are equal.
Please review other statistical methods LBNL
has used in the analysis, in addition to the
logistic regression methodology. Examples
include the alternative approaches used by
LBNL to assess NHTSA interval estimation
results, and LBNL's linear regression analysis
of actual, predicted, and residual risk by
vehicle model.
[Chen and Kockelman] In the alternative measures of exposure, the author examines the effect of vehicle
manufacturer on fatality risk and treats the luxury models produced by Toyota, Honda, and Nissan as separate
manufacturers. However, domestic luxury brands (such as Cadillac & Lincoln) are categorized with their
nameplate manufacturers (GM and Ford), which appears inconsistent.
[LBNL] The analysis has been revised to include the luxury models Lexus, Acura, and Infiniti with their
nameplate manufacturers Toyota, Honda, and Nissan. An additional sensitivity was conducted using separate
indicator variables for the five luxury brands Lexus, Acura, Infiniti, Cadillac, and Lincoln. The results of these
analyses are summarized in Section 5.3 in the Final Phase 2 report.
LBNL also examined the effect of replacing the vehicle make indicator variables with a single continuous
variable for vehicle initial purchase price, taken from the Polk VIN decoder. Initial purchase price can be seen as
a proxy for the general quality of vehicle design; we expect that more expensive vehicles will have lower risk
than less expensive vehicles. This analysis is summarized in Section 5.5 in the Final Phase 2 report.
[Van Auken] [1] The correlations in Section 3 appear to be assessed using the Coefficient of Multiple
Determination (R2) based on a linear fit to the data (e.g., the correlation between footprint versus curb weight
in Figure 3.1 on p. 32). The linear regression model attributes the differences between the dependent variable
(vertical axis) and the linear fit to the independent variable (horizontal axis) to random effects. If there is no
preference as to the choice of independent and dependent variables (e.g., footprint versus curb weight, or
curb weight versus footprint), then the linear trend and R2 result would be different if the two variables were
interchanged, and having two different yet equally valid results would be undesirable.
[LBNL] The R2 values are the same if the two variables are interchanged.
197
-------
[2] If the variation in the data can be attributed to both variables (e.g., footprint and curb weight), then it
would be better to report the square of the sample correlation coefficient r2, where r is computed according to
Eqn (1). The trend lines in these correlation figures should not be computed using a linear regression. Instead,
the trend line should pass through the sample means (i.e., (x, y)), and have a slope equal to the ratio of the
sample standard deviations in the data (i.e., sy/sx). Therefore, the reported correlation results do not depend
on the ordering of the data variables.
[LBNL] Again, the R2 values are the same if the two variables are interchanged. The R2 values in Figures 3.1 and
3.2, and Figures 4.1 through 4.6, have been replaced with the correlation coefficient r, and a new Table 3.1 has
been added to Section 3 that lists r and VIFfor curb weight and footprint by vehicle type, in the Final Phase 2
report.
Note this comments does not apply to linear trends indicated in Section 4, for which the Coefficient of Multiple
Determination (R2) seems appropriate.
[3] The Coefficient of Multiple Determination (R2) is frequently used in the Wenzel (2011b) report as an
indicator of the statistical importance of a linear trend (e.g., lvalues in Tables 4.1 and 4.2 were compared to
0.3). It would be better to report the standard error, confidence interval, and/or probability value as measures
of the statistical significance of a linear trend.
[LBNL] The significance of the estimated relationships have been added to the tables in Section 4 of the Final
Phase 2 report. However, the point of the section is to indicate that while the relationship may be statistically
significant, on average (as indicated by the confidence level around the estimated effect), there remains a large
range in risk for individual vehicle models with similar mass (as indicated by the relatively low R2 values).
Please comment on caveats or limitations of
using non-significant regression estimates to
project the safety impact of mass reduction.
[Chen and Kockelman] First, the t-statistics are not provided in the report which makes it difficult for the
reader to assess statistical significance of specific regression estimates (except where noted by the author).
Second, inclusion of a statistically insignificant variable can influence the estimates of coefficients associated
with related variables. Nevertheless, in general, it is best to keep insignificant estimates if one has a strong
defense for their role, since removing such variables (& thus their parameters) will shift the burden of
response to a correlated covariate's parameter, thus biasing the latter. We generally keep key covariates in a
model up to a pvalue of 0.20 or 0.25 or so, especially in relatively small data sets (e.g., n < 1,000). Covariates
for which we have no strong basis can be removed for pvalues > 0.10.
198
-------
[LBNL] The 95% confidence intervals are included in every figure in the document, to indicate whether the
estimated variable is statistically different from zero. LBNL ran a sensitivity where non-significant control
variables are removed; see Section 5.8 of the Final Phase 2 report.
[Farmer] Making projections from non-significant regression estimates is proper so long as the resulting
confidence intervals are constructed conservatively (to account for the accumulated imprecision). In that
sense, I prefer NHTSA's jackknife approach to the standard errors produced by SAS (see p. 13 of Phase II).
[LBNL] LBNL agrees that NHTSA's jack-knife method is a preferable method of estimating confidence intervals;
however, the jack-knife method obtains even larger estimates of uncertainty than using the standard errors
output from the regression models.
[Van Auken] [1] Regression estimates are random numbers which have an unknown expected value and
variance, and known sample value and standard error. If the sample value can be explained by a zero
expected value and known standard error then the result is considered not statistically significantly different
than zero and therefore the result is not considered to be statistically significant. However, If we can combine
this estimate with other estimates then the unknown expected values and variances can also be combined
using the same transformation, and the statistical significance of the combined result can be tested.
Therefore, depending on the sample values and inter-correlation, the combined result may be statistically
significant even if the individual estimates are not statistically significant.
For example, the results from each of the nine different crash types can be combined into an overall estimate
and the standard error calculated assuming that the results for each crash type are independent of each other.
Then the statistical significance of the combined effect can be determined.
[2] However, and Kahane (2011) points out there are two sources of uncertainty in the regression results. The
first is the PARS based sampling error which is uncorrelated across crash types because they are based on
different fatal cases (Kahane 2011, p. 77). The second is the state based induced-exposure sampling error
which is correlated across crash types because they are based on the same induced-exposure cases. Therefore
a confidence interval estimated using the "jackknife" method described by Kahane (2011) and accounting for
correlation of these two error sources would be more accurate than a simple estimate based on the Wald Chi-
Square statistic and assumed independence.
[LBNL] LBNL agrees that NHTSA's jack-knife method is a preferable method of estimating confidence intervals;
199
-------
however, the jack-knife method obtains even larger estimates of uncertainty than using the standard errors
output from the regression models.
How might the LBNL methodology be
strengthened to better represent future
vehicle designs and reduce multi-collinearity
between mass and footprint in the regression
analysis?
[Chen and Kockelman] Including more vehicle-specific characteristics (such as vehicle height and engine size)
reduces the analysis' dependence on vehicle type, since vehicle shapes and structures will continue to evolve.
There is also correlation with context (e.g., pickups are driven in more rural locations, with greater hazards
[like less lighting, higher speed, & few medians]). Disaggregate data are almost always best, to avoid ecological
fallacies & such.
[LBNL] As discussed above, overall vehicle height is only a crude proxy for two vehicle dimensions, center of
gravity and bumper height, that are thought to correlate with safety in rollover and two-vehicle crashes,
respectively, and therefore was not included in the regression models. Other vehicle attributes that might
affect risk are engine power-to-weight ratio, braking distance, and handling capabilities; LBNL may examine
the effect of these accounting for these vehicle attributes in future analyses.
LBNL's analysis does account for whether a vehicle is driven in a rural area, defined as a county in which the
population density is less than 250 residents per square mile of land area.
[Van Auken] [1] The effects of multi-collinearity can be mitigated by 1) obtaining more data, 2) pooling data
from different crash type or vehicle types, or 3) reducing the number of regression variables. The first option
would require more states (for serious injuries and police-reported accidents), calendar years and/or model
years, which would involve added newer data as it becomes available (or using older data). The second option
might be to recombine the CUVs and minivans with truck based vans and adding a control variable to
compensate for the differences in the vehicles types. The third option might involve removing statistically
insignificant control variables or removing control variables that would not be expected to have an effect on
the probability of crash or crash outcome (e.g., the side airbag variable is not included in pedestrian crashes
because it is not expected to affect pedestrian fatality risk). The number of driver age control variables might
be reduced from eight to three (as in the Kahane (1997) and DRI (2002-2005) studies). Finally, a linear curb
weight model instead of a two-piece linear model may help to better elucidate the general trend.
[2] The Variance Inflation Factor (VIF) has been suggested as a measure of multi-collinearity in the Kahane
200
-------
(2010 and 2011) reports, however this diagnostic metric does not account for differences in database size (i.e.,
Options 1 and 2 above). The Wenzel (2011b) report does not discuss the Variance Inflation Factor or report
any VIF results.
[LBNL] A table of VIF results, Table 3.1, has been added to the Final Phase 2 report.
ADDITIONAL COMMENTS:
201
-------
4. DATA SETS
COMMENTS
Please comment on the validity and
applicability of the datasets used to project
changes in risk resulting from reduction in
vehicle mass. LBNL's casualty analysis used
police-reported crash data from 16 states,
while the 2011 NHTSA study used national
fatality data, combined with a subset of non-
culpable vehicles involved in two-vehicle
crashes from police-reported crash data from
13 states.
[Chen and Kockelman] [1] The Phase 2 report uses an unusually extensive data set of police-reported crash
data from 13 states which the author compares in detail to national data sets to illustrate similarities and
differences. The author is very thorough in addressing the difference in definitions of "serious" and
"incapacitating" injuries across different states and the effects of such inconsistency on the regression results.
[2] Since casualty risk in the report accounts for serious injuries but not minor injuries, the author should note
that police-reported injury levels may also be poor indicators of the actual or Modified Abbreviated Injury
Scale (MAIS) level, following medical evaluation. Farmer (2003) found that 41% of injuries reported by U.S.
police as incapacitating received MAIS ratings of "minor injury" by health care professionals using NASS
Crashworthiness Data System (CDS). Thus, the results of the estimated casualty risk increases and decreases
rely heavily on the assumption that police errors in reporting actual MAIS ratings are consistent across states.
[LBNL] Inconsistencies in injury reporting across states are another reason to include a control variable for each
state. The following text has been added to Section 2.3 of the Final Phase 2 report:
"Another type of bias is inaccurate reporting of injury outcomes by police officers at the scene of a crash. Using
detailed NASS CDS records, in which a crash investigator tracks hospital records of victims in a small sample of
police-reported crashes, Farmer (2003) found that 41% of injuries that police responders coded as serious or
incapacitating received Modified Abbreviated Injury Scale (MAIS) ratings of "minor injury" by health care
professionals. The possibility that these injury reporting errors are not consistent across states is another
reason to include a control variable for the state in which the crash occurred."
[Farmer] A major limitation of the Phase II analysis is a bias that may be due to the patterns of missing data.
In particular, the vehicle identification number (VIN) is missing or mistyped for many crash records. High-
severity crashes (especially fatal) are more likely to have detailed police investigation, so VINs (and other
variables) in these crashes may be more complete. State crash files are therefore much less reliable than
PARS.
[LBNL] LBNL assumes that missing or erroneous VINs are equally distributed among vehicle year, make and
model. In its primary analysis LBNL used only vehicles whose model year reported in the crash database
matched the model year from the decoded VIN (as well as all vehicles in certain years in Washington where
model year was not recorded in the state database). In addition, in Section 5.7 of the Final Phase 2 report LBNL
202
-------
corrected obviously erroneous VINs, by translating VIN position 10, shifting the VIN one position to the right
starting at position 9 (both based on the reported model year), and translating VIN position 8, the vehicle
engine code (based on a large database of known valid VINS, as described in Wenzel 2011a). Combined these
changes increased the number of available records by only 1%.
[Van Auken] [1] The induced-exposure data set provided by NHTSA is based on the non-culpable vehicles in
two-vehicle crashes. See the comments in Table 1 on the limitations of this data.
[2] The use of property damage accident data and cases with serious injury from the 13 states seems
appropriate (with the noted qualification that the different states may have different accident reporting
thresholds and injury reporting criteria). The concerns about the use of data for the 3 additional states
(Georgia, Illinois, and New Mexico) have also been noted.
[LBNL] Because of the differences in crash or injury reporting thresholds and injury severity reporting across
states, LBNL used a control variable for each state in which the crash occurred.
[3] In addition, there are also many differences in the coding variables and values used by the different states,
which tend to make the receding to a common data set (either induced-exposure, police-reported accident, or
severe injury) imprecise.
[LBNL] The coding of the variable for type of crash may vary across states. However, the remaining control
variables used from the state crash databases are either derived from the VIN, or based on the driver age and
gender, or the time or location of the crash. LBNL believes that these variables are consistently coded among
the thirteen states.
203
-------
Please comment on any apparent, unstated,
or implicit impact on estimated risks inherent
in the two different approaches, and any
related caveats or limitations. For example,
what are the strengths and weaknesses of the
two measures of vehicle exposure, miles of
vehicle traveled scaled up from crash data
from 13 states, and number of police-reported
crashes?
[Chen and Kockelman] The Phase 1 analysis used non-culpable vehicles in two-vehicle crashes as a proxy for
induced exposure crashes. In contrast, Phase 2 analysis uses data from vehicles involved in one-car crashes and
the responsible vehicle in two-car crashes. The exclusion of the not-at-fault vehicle in two-car crashes may be
distorting the distribution of crash frequency and casualty risk across different vehicle makes and models if
crash-prone drivers are more likely to drive certain types of vehicles.
[LBNL] The Phase 1 analysis used a subset of non-culpable vehicles in two-vehicle crashes to assign vehicle,
driver, and crash characteristics to the total number of US vehicle registrations and miles driven. The measure
of exposure in the Phase 1 analysis is national miles driven. The Phase 2 analysis used all non-fatality (or non-
casualty) crashes in the 13 states as the measure of exposure for estimating fatalities (or casualties) per crash;
the same VMTfrom the Phase 1 study was used to estimate crashes per VMT, and fatalities (or casualties) per
VMT.
[Farmer] The VMT weights provided by NHTSA were scaled to represent the entire US. Comments on pp. 9
and 18 of the Phase II report seem to acknowledge this deficiency, promising to adjust these to the 13 states in
the future. Was any adjustment made, such as multiplying the weights by the proportion of annual US VMT
accounted for by each of these states? The accuracy of the VMT weights is critical is we are to believe the
somewhat surprising results concerning crashes per VMT.
[LBNL] LBNL requested that NHTSA recalculate the VMT weights to represent the total number of vehicles
registered, and the estimated total number of miles driven, in the thirteen states, rather than for the entire US.
NHTSA declined to provide LBNL VMT weights for the 13 states.
In their previous analyses DRI similarly found that crash frequency (per VMT) increases as mass decreases. And
in Section 1.6 of its 2011 report NHTSA notes that small and light vehicles historically have had higher crash
and insurance claim frequency per vehicle mile traveled, despite their theoretical advantage in terms of
handling, braking, and accelerating.
[Van Auken] [1] The number of fatal or serious injury cases tends to be much less than the number of
induced-exposure cases (and the number of police-reported accidents). Therefore the effective numbers of
degrees-of-freedom in the statistical estimates tend to be limited by the available number of fatal or serious
injury cases. For example, it would not be possible to estimate the effects of two variables (e.g., just curb
weight and footprint) if we had data for only one fatal of serious injury case even if we had thousands of
induced-exposure cases. Therefore it is desirable to use data for the entire US in order to get a large sample of
204
-------
fatal cases for the logistic regressions. This then requires the available induced-exposure data (i.e., from 13
states) to be "scaled up" the US level using the method described in Kahane (2003 and 2011). The result is the
best currently available estimate of vehicle exposure.
[2] There may be some concerns about the accuracy of the vehicle miles-travelled data because the difficulty
estimating the number of vehicle miles travelled at the make-model-year-state level of detail.
[LBNL] RL Polk provided NHTSA a database of average odometer reading by vehicle year, make and model
based on hundreds of thousands of odometer readings. Because many of these records came from state
emission inspection and maintenance programs, it is likely that they are skewed towards vehicles driven in
urban areas, which tend to be driven fewer miles than comparable vehicles registered in rural areas.
An additional limitation of the VMT weights developed by NHTSA is that they do not account for the reduction
in miles driven in response to higher gas prices and the economic recession in 2008, as noted in Section 5.3 of
LBNL's Phase 1 report. Despite these limitations, the average VMT weights NHTSA has developed are an
improvement over the averages used in the 2003 analysis.
ADDITIONAL COMMENTS:
[Chen and Kockelman] Farmer, C.M. Reliability of police-reported information for determining crash and injury severity. Traffic Injury Prevention 4(1), 2003,
pp.38-44.
[Farmer] Statements above Figure 2.7 in the Phase II report imply that the effects of weight reduction on crashes per VMT and fatalities per crash should add
up to the effect on fatalities per VMT. This is not the case. For example, a 1.43% increase in crashes per VMT and a 0.76% decrease in fatalities per crash
would imply a 2.16% decrease in fatalities per VMT (i.e., 1 - 0.9924/1.0143). The fact that the model on fatalities per VMT yields an estimated 1.08% increase
should be a cause for concern. Either the VMT weights are inaccurate or the control variables have different effects on crash frequency and crashworthiness.
[LBNL] Table ES.l reports for lighter-than-average cars a 2.00% increase in crash frequency, a 0.54% decrease in fatalities per crash, and a 1.42% increase in
fatalities per VMT; in this case the first two estimates sum almost exactly to the third (2.00% - 0.54% = 1.46%). In other cases the first two estimates do not sum
to the third. In its previous studies DRI solved the three equations, crashes per VMT, fatalities per crash, and fatalities per VMT, simultaneously, which forces
the estimated effects on fatalities per VMT to equal the sum of the estimated effects on crashes per VMT and fatalities per crash.
205
-------
5. RECOMMENDATIONS
COMMENTS
Please comment on whether the LBNL
assessment adequately addresses the NHTSA
2011 study and identifies the safety impact
from mass reduction. Are the analytic
methods and data used to assess the NHTSA
study, and estimate the relationship between
risk, mass, and footprint, appropriate? Is
casualty risk per crash a legitimate measure of
vehicle safety? What other methods or data
could be used to better predict the effect of
future vehicle designs on safety?
[Chen and Kockelman] As noted above, a primary concern remains a neglect of the notion that the type of car
is very much a proxy for driver type, and a vehicle's crash avoidance may have very little to do with vehicle
type. It has a lot to do with the person behind the wheel. Simply including gender and age variables cannot
account for important covariates such as education, risk aversion, driving ability, wealth, etc. In reality, small
cars may be less crash prone than Kahane's and Wenzel's results suggest, because they are driven by
lower-income, younger, less risk averse people driving in more crash prone settings (e.g., commercial strips
rather than pricey residential suburbs). Alas, it is very difficult to control for all these variables, since they are
not readily available in data sets. In reality, the best the report authors can do with such data sets is to explain
how all the other, relevant attributes may factor in (e.g., quality of driver and typical driving settings), and how
they can generate biased estimation (sometimes in either direction). Discussion of relevant literature that
looks more deeply at crash outcomes (e.g., Wang or Chen's papers, mentioned above, allowing for
heteroscedasticity and individual vehicle attributes, non-driver outcomes, etc.) will also be useful.
[LBNL] As discussed above, additional sensitivity analyses were run using an alternative control variable for
vehicle characteristics, initial vehicle purchase price (using values in Folk's VIN decoding software), and an
alternative control variable for driver behavior, household income (based on California vehicle registration data
in 2010). The results of these sensitivities are shown in Sections 5.3 and 5.5 in the Final Phase 2 report.
[Farmer] Casualty risk per crash does not fully measure the effects of vehicle size and weight reductions on
society. Casualty risk per VMT best coincides with the NHTSA analysis of fatalities per VMT. The breakdown of
casualty risk per VMT into the crash frequency and crashworthiness components is of interest. However, the
surprising results reported here make everything suspect. For example, the Phase II report concludes that "the
detrimental effect of male drivers has to do with their higher tendency of getting into a serious crash rather
than their sensitivity to injury once a serious crash has occurred" (p. 24). A few pages later it concludes that
"male drivers have essentially no effect on crash frequency, but cause a statistically significant increase in
fatality risk once a crash occurs" (p. 28).
[LBNL] A summary discussing unexpected results has been added to the end of Section 2 in the Final Phase 2
report.
[Van Auken] [1] The basic methodology described by Kahane (2011) seems appropriate; and the extension by
Wenzel (2011b) are also appropriate. However some results using these methods and data are not well
206
-------
understood and need further diagnosis.
[2] The induced-exposure data set provided by NHTSA is based on the non-culpable vehicles in two-vehicle
crashes. See the Table 1 comments on the limitations of this data.
[3] The state accident data files tend to have different database variable and coding definitions and criteria,
which could confound the results.
Please comment on the overall adequacy of
LBNL's assessment of the 2011 NHTSA report
and its independent study of casualty risk for
predicting the effect of vehicle mass or
footprint reduction on safety. Provide any
recommended improvements that might
reasonably be adopted by the author to
improve the analysis.
[Chen and Kockelman] Overall, the study is an enriching complementary document to the Phase 1 assessment
of the 2011 NHTSA report. The parallel structure of the two reports allows the reader to easily compare and
contrast the various additional analyses which examine the relationship of vehicle mass and footprint
reduction on safety. However, as stated previously in the comments here, driver preference for specific car
types (including size and mass) is related to driver socioeconomic characteristics and driving behavior. As
vehicle, driver, and roadway environment characteristics all contribute to fatality risk, the effects of physical
vehicle changes such as mass or footprint reduction on safety should not be overstated when the other two
types of characteristics are not sufficiently accounted for.
[LBNL] As discussed above, additional sensitivity analyses were run using an alternative control variable for
vehicle characteristics, initial vehicle purchase price (using values in Folk's VIN decoding software), and an
alternative control variable for driver behavior, household income (based on California vehicle registration data
in 2010). The results of these sensitivities are shown in Sections 5.3 and 5.5 in the Final Phase 2 report.
[Farmer] Overall these are reasonably good studies. The Phase I report does a very good job of assessing the
NHTSA report of fatality risk. However, the Phase II report should be more cautious in its conclusions
concerning casualty risk. The casualty analysis is based solely on police-reported data from 13 states, which:
1. May not be representative of the US as a whole.
2. Are inconsistent in the information given and the way in which it is coded.
3. Suffer from information that is missing, inaccurate, or unclear.
[Van Auken] The Wenzel (2011b) report provides a valuable supplement to the analysis and results in the
Kahane (2011) report.
ADDITIONAL COMMENTS:
207
-------
[Farmer] Column G of Table 6.1 in the Phase II report provides the most appropriate comparison to results from the NHTSA report (Column A). For both
fatalities and casualties per VMT, a 100-lb weight reduction is most harmful in lighter cars, less harmful in heavier cars and lighter light trucks, and slightly
beneficial in heavier light trucks, minivans, and crossovers.
[Van Auken] See attached tables 6 and 7 below.
208
-------
Table 6. Additional General Comments and Recommendations
Mike Van Auken
Section
COMMENTS AND RECOMMENDATIONS
Use of R2 is confusing. Suggest using lower case "r" when referring to the sample correlation coefficient (Box, Hunter, Hunter,
1978, P. 61); or upper case R when referring to the regression coefficient of multiple determination (Draper and Smith, 1981, p.
90).
[LBNL] Figures 3.1 and 3.2, and 4.1 through 4.5, report r and not R2 in the Final Phase 2 report. In addition, a new Table 3.1 with
r and VIF, and a new Table 4.1 with r and R2, have been added.
All
In most cases the reported results are just estimates, but are not described as such. The word "effect" appears several hundred
times in this report with the "estimated" or other qualifier. In some cases this may be appropriate and in other cases it is not
appropriate. It is recommended that the author review each instance and revise as appropriate.
[LBNL] The word "estimated" will be used extensively in the final report. In addition the following text will be included in the
Executive Summary, Section 1, and Section 6:
Although the purpose of the NHTSA and LBNL reports is to estimate the effect of vehicle mass reduction on societal risk, this is
not how the regression models should be interpreted. Rather, they are estimating the recent historical relationship between
mass and risk, after accounting for most measurable differences between vehicles, drivers, and crash times and locations. In
essence, the regression models are comparing the risk of a 2600-lb Dodge Neon with that of a 2500-lb Honda Civic, after
attempting to account for all other differences between the two vehicles. The models are not estimating the effect of literally
removing 100 Ibsfrom the Neon, leaving everything else unchanged.
In addition, the analyses are based on the relationship of vehicle mass and footprint on risk for recent vehicle designs (model
year 2000 to 2007). These relationships may or may not continue into the future as manufacturers utilize new vehicle designs
and incorporate new technologies, such as more extensive use of strong lightweight materials and specific safety technologies.
Therefore, throughout this report we use the phrase "the estimated effect of mass (or footprint) reduction on risk" as shorthand
for "the estimated change in risk as a function of its relationship to mass (or footprint) for vehicle models of recent design."
All
"Crashworthiness" in most instances should be changed to "crashworthiness and crash compatibility" because the fatalities
and/or serious injuries may either be in the subject vehicle (crashworthiness effect) or collision partner (crash compatibility
209
-------
effect).
[LBNL] Text has been changed
The statistical significance of the linear trends in Figures 4.1 through 4.9 are not reported. It would be helpful if the confidence
intervals or statistical significance of the linear trends were reported, either in addition to or instead of R2.
The confidence intervals for the estimated slopes should be added to the results in Tables 4.1 and 4.2.
[LBNL] New Tables 4.1 through 4.5 summarize the linear trends and statistical significance of Figures 4.1 through 4.17 in the
Final Phase 2 report.
210
-------
Table 7. Additional Specific Comments and Recommendations
Mike Van Auken
Section
Page
COMMENTS AND RECOMMENDATIONS
Executive
Summary, 4
iv, v, 22, 66
Suggest changing "variance in risk" to "variation in risk" throughout.
[LBNL] The suggested change has been made.
Executive
Summary, 6
vii, 63
The statement "In conclusion, casualty risk per crash is not necessarily a better metric than fatality risk per VMT
for evaluating the effect of mass or footprint reduction on risk; rather, it provides a different perspective in
assessing the benefits or drawbacks of mass and footprint reduction on safety in vehicles. However, it does
allow the separation of risk per VMT to be separated into its two components, crash frequency and risk per
crash" suggests that the casualty risk per crash metric was needed in order to assess the crash frequency and risk
per crash, which is incorrect. The DRI (2003-2012) methods have also estimated the effects of weight and size
on crash frequency (A/E) and risk per crash (F/E) in terms of fatalities.
[LBNL] Text has been changed.
5.3
49
It would be helpful to list the 18 manufacturer dummy variables in a table.
[LBNL] The indicator variables for the vehicle manufacturers has been added to the final report.
5.3
48-49
It is unclear why Lexus, Acura, and Infinity are treated as separate manufacturers, but Cadillac and Lincoln are
not.
[LBNL] The results of two regressions has been included in the final report: one including the five luxury brands in
their parent manufacturers, and one accounting for each of the five luxury brands. Accounting for the five luxury
brands has little effect on the estimated detrimental effect of car mass reduction on risk, as shown in Figure 5.3
in the Final Phase 2 report.
5.3
49
It is unclear why AM General is considered a Chrysler brand. The AM General Hummer was sold by GM
beginning with the 2003 model year.
[LBNL] AM General has been removed from the Chrysler brand and included in the Other category.
211
-------
212
-------
Review of "An Analysis of the Relationship between Casualty Risk per Crash and Vehicle Mass and
Footprint for Model Year 2000-2007 Light-Duty Vehicles"
David L Greene
January 10, 2012
Summary
The Phase I and Phase II analyses by Tom Wenzel of LBNL have been executed diligently and consistently
in accord with the methods and data used in the original NHTSA analysis. The studies contain many
valuable, new insights. The phase I study highlights the weakening relationship between vehicle mass
and highway fatalities. This is not only seen in decreasing coefficient estimates but in the very large
number of results that are not statistically significant. When regressions were done separately by
footprint deciles, vehicle mass was statistically significantly positively related to fatalities only for light-
duty trucks in rollovers, there were almost as many cases in which mass was negatively related to
fatalities (9 vs. 13 out of 27) and there were more instances of statistically significant negative
relationships than positive relationships. Given that so many tests are being jointly conducted, it is quite
possible that when joint probabilities are considered, there is no significant relationship between mass
and fatalities (more on joint probabilities later). Showing the weakness and inconsistency of these
results is an important contribution.
Another meaningful contribution of the phase I study, and one that deserves more emphasis, is a logical
inference from the following findings: 1) much of the variance in risk remains unexplained even by the
most complete models, 2) control variables explain 1 to 2 orders of magnitude more of the variance
than the variables of interest (mass and size), 3) when key control variables are removed or changed it
strongly influences the coefficients of mass and size. These results have very important implications for
the robustness of the results and the likelihood that some or all of the apparently statistically significant
relationships are due to spurious correlations with omitted or imperfectly controlled factors. Noting
that exposure measures are control variables with constrained coefficients, the following observation
from the phase I study is especially perceptive.
"Calculating risk as total fatalities per induced exposure crash, rather than per vehicle mile
traveled, reverses the sign of mass reductions on risk in cars and the lighter light trucks, with mass
reduction leading to a reduction in risk in all vehicle types."
Finally, the phase I report notes that if only the control variables are included in the regression and
not size or mass, the resulting residuals from the regression are uncorrelated with size or mass.
Given these findings (as well as those of phase II) the conclusions that,
"The 2011 NHTSA study, and this report, conclude that the effect of mass reduction while
maintaining footprint on societal US fatality risk is small, and may be non-existent."
213
-------
should be revised with the following emendation, "... and probably non-existent."
[LBNL] Text will be revised as indicated in bold above.
Both studies, like the NHTSA analysis, have shortcomings in terms of interpreting the results and the
language used to describe the results, and acknowledging the limitations of the data and methodology.
The limitations are extensive. The interpretation of the results of the LBNL studies commits two
important, related errors. The first is to attribute inferred coefficients of mass and size as representing
only the effects of vehicle mass and size when, as the phase I and II study results indicate, there is a
virtual certainty that aliasing effects are present due to a combination of omitted variables, errors in
variables and correlations among variables. Given that estimated driver and environmental factors
tend to have 1-2 orders of magnitude larger impacts on safety outcomes than vehicle factors, the almost
certain presence of aliasing effects must be explicitly acknowledged as severely limiting the ability to
draw inferences about the effects of vehicle attributes. Second, the language used in interpreting
results fails to acknowledge that the analysis does not address the effects of down-weighting or down-
sizing specific vehicles or vehicle designs, but instead relies on correlations between vehicle weight and
size in existing vehicle designs. In existing vehicles, weight and size are correlated with each other and
many other vehicle attributes (and driver and environmental attributes, as well). Thus, the study is not
actually measuring the effects of down-weighting via the material substitution and design changes likely
to occur as a consequence of fuel economy and emissions standards. An early example of the kind of
misleading language referred to here can be found on page iv.
"For example, a 100-lb reduction in the mass of lighter cars leads to a 1.84% increase in crash
frequency (columns B), while mass reduction leads to a 0.76% decrease in the number of fatalities
per crash (column C);"
This statement is misleading in that it implies causality rather than correlation, and it is additionally
misleading in that it implies that the inference applies to removing weight from specific vehicles.
Neither is correct. A better statement would be the following.
"For example, vehicles in the lighter class that are 100 Ibs. lighter are correlated with a 1.84%
increase in crash frequency...."
There are so many examples of this misleading language that it is not feasible to list them all. All
should be corrected, however. Failure to correct them could lead to serious misinterpretation of the
studies' findings.
[LBNL] The word "estimated" will be used extensively in the final report. For example, the above
sentence will be rewritten as:
"For example, the models estimate that 100-lb lower mass in lighter-than-average cars is associated with
a 2.00% increase in crash frequency (column B), while lower mass is associated with a 0.54% decrease in
the number of fatalities per crash (column C);"
214
-------
In addition, the following text will be included in the Executive Summary, Section 1 and Section 6 of the
reports:
"Although the purpose of the NHTSA and LBNL reports is to estimate the effect of vehicle mass reduction
on societal risk, this is not exactly what the regression models are estimating. Rather, they are
estimating the recent historical relationship between mass and risk, after accounting for most
measurable differences between vehicles, drivers, and crash times and locations. In essence, the
regression models are comparing the risk of a 2600-lb Dodge Neon with that of a 2500-lb Honda Civic,
after attempting to account for all other differences between the two vehicles. The models are not
estimating the effect of literally removing 100 Ibsfrom the Neon, leaving everything else unchanged.
In addition, the analyses are based on the relationship of vehicle mass and footprint on risk for recent
vehicle designs (model year 2000 to 2007). These relationships may or may not continue into the future
as manufacturers utilize new vehicle designs and incorporate new technologies, such as more extensive
use of strong lightweight materials and specific safety technologies. Therefore, throughout this report
we use the phrase "the estimated effect of mass (or footprint) reduction on risk" as shorthand for "the
estimated change in risk as a function of its relationship to mass (or footprint) for vehicle models of
recent design.""
Following in the footsteps of the seminal study by DRI, the NHTSA and LBNL studies contribute to the
literature in three important ways: 1) the LBNL and NHTSA studies recognize that the societal safety
perspective is the correct perspective to when assessing the impacts of fuel economy and emissions
regulations, 2) they recognize that vehicle dimensions and vehicle mass may have separate and
potentially different impacts on both the likelihood of a crash and the outcomes of the crash and, 3) the
LBNL phase two analysis makes an additional contribution by attempting to disentangle factors affecting
the likelihood of a crash and factors affecting the outcomes of a crash.
Speaking of the DRI study, I am puzzled about why there are no references cited in the phase I study and
only a handful all by Kahane and Wenzel, in the phase II study. This is perhaps due to the scope of work
defined for the two studies but there are highly relevant studies in the literature that could have been
cited, those by DRI foremost among them. Making use of the insights from these studies would have
been helpful in interpreting the results of both phase I and phase II.
[LBNL] LBNL has summarized the 2003 DRI studies, as well as updated results published by DRI in 2011;
see Sections 2 and 3 in the Final Phase 2 report.
Lack of a Theory or Model of the Phenomenon
Both the NHTSA and LBNL studies lack a rigorous theory of the process by which down-weighting at
constant size or down-sizing at constant mass affect societal safety either through crash avoidance or
crashworthiness. This is not a trivial shortcoming because it affects the ability to formulate hypotheses
and interpret results. Prior to the dissenting report on safety of the NRC 2002 CAFE report, the physics
of elastic collisions between objects was typically cited as the underlying physical model. That report
showed how taking the societal perspective renders that model inappropriate. What remains appears
215
-------
to be far more complex, involving the quantity of kinetic energy, the ability of vehicle designs to absorb
that energy so as to minimize maximum deceleration rates, stability, maneuverability, safety
technologies, and more.
The consequences of the lack of a rigorous theory are that it is not known, a priori, what the signs of
coefficients are expected to be, let alone what their quantitative relationships should be. Hypotheses
must be formulated based on intuition and the interpretation of results is likewise ad hoc. One
implication of this is that results that suggest that lower fatalities are associated with lower vehicle mass
have equal standing, a priori, with results that indicate that higher fatalities are associated with lower
vehicle mass, and similarly for vehicle size. There are no surprising or unsurprising results, in theory.
This also makes it difficult to develop a plan for statistically testing the model or theory and its
implications. It would have been helpful to the reader to have been presented early on in the report
with such a plan of analysis.
[LBNL] Section 1.5 of NHTSA's 2011 report summarizes the hypothetical physical relationships between
vehicle mass, footprint, and societal fatality risk. LBNL has added a section summarizing this discussion
at the end of Section 1 of the Final Phase 1 and Phase 2 reports.
On the Virtual Certainty of Aliasing
The LBNL report typically attributes causal effects to correlations between mass or size and safety. In
fact, most or all of the observed correlations are almost certainly affected by aliasing effects. There is
ample evidence for this inference in the results presented in the LBNL phase II report.
The coefficients of mass and size change in important ways when different model formulations are
estimated. Removing and adding control variables changes the magnitudes and sometimes the signs of
the mass and size variables. This means that, at a minimum, the mass and size variables alias the effects
of the omitted control variables. The question is whether the aliasing is eliminated entirely by the
inclusion of the control variables available or whether some aliasing remains either because not all
relevant and correlated control variables have been included or because the included control variables
are imperfect measures of the factors they are intended to represent.
The latter seems highly likely for the following reasons. First, the overall explanatory power of the full
models (including control variables), as measured by their R2 is low. Most of the variance in casualties
and fatalities remains unexplained. Second, at least some of the important included control variables
are only crude approximations of the factors they are intended to represent. For example, dummy
variables represent differences in state reporting practices, age and gender represent risky driving
behavior differences among owners of different sizes and masses of vehicles, the presence or absence
of a kind of safety equipment represents both its performance and use in a particular vehicle, and
calendar year dummy variables represent unknown factors associated with the respective calendar
years. Such practices are common and their use is appropriate. Third, the control variables generally
account for 1-2 orders of magnitude more variance in the casualty and fatality variables than do vehicle
weight and size. To recap, the amount of unexplained variability in the dependent variables is larger
216
-------
relative to the variance statistically explained by the most complete models. Control variables are
correlated with size and mass, and they account for 1-2 orders of magnitude more variability in the
dependent variables than the variables of interest, mass and size. Therefore, even small correlations of
size and mass with omitted variables or with errors (imprecision) of the control variables could easily
result in biased estimates for the effects of size and mass on the dependent variables.
[LBNL] The only control variables used in the NHTSA regression models that are correlated with size or
mass are the HD_PKP variable (r of 0.65 on OVERWTOO and r of 0.54 on FOOTPRNT) and the SUV
variable (r of 0.62 on FOOTPRNT). The control variable LBNL has added to the sensitivity analyses for
initial vehicle purchase price, PRICEOOO, also is correlated with mass and footprint. However it is possible
that other controls not included in the regressions may be correlated with mass or footprint.
Tom Wenzel is to be commended for providing the results that definitively demonstrate the three key
points made above. The above is not a criticism of the analysis nor of the results, per se. It is a criticism
of their interpretation. In light of the above, the results should be interpreted in light of the virtual
certainty that many of the estimated coefficients are likely to be biased in ways that make their
interpretation highly uncertain. The implication is that phrases such as "down-weighting or down-sizing
caused" to "mass (or size) and unobserved correlated factors are associated with..."
[LBNL] The following text will be added to the executive summary:
"It is unclear why lower vehicle mass is associated with higher crash frequency, but lower risk per crash,
in the regression models. It is possible that including variables that more accurately account for
important differences among vehicles and driver behavior would reverse this relationship. On the other
hand, it is also possible that over thirty years of improvements in vehicle design to achieve high crash test
ratings have enabled manufacturers to use clever vehicle design to mitigate the hypothetical safety
penalty of low mass vehicles."
"The large remaining unexplained variance in risk by vehicle model could be attributable to other
differences in vehicle design, or how drivers who select certain vehicles drive them. It is possible that
including variables that account for these factors in the regression models would change the estimated
relationship between mass or footprint and risk."
Additional text, discussing unexpected results, has been added to the end of Section 2 of the Final Phase
2 report.
On Joint Probabilities
The NHTSA and LBNL studies do not correctly interpret their results as joint statistical tests. When
testing a hypothesis on, for example, 5 vehicle classes simultaneously, a result for one equation that
might be statistically significant on its own may not be statistically significant as one of five related tests.
Statistical analyses comprised of multiple regressions too often overlook the fact that tests of statistical
significance designed for individual regressions may not apply in the case of multiple regressions. That is
the case here. NHTSA conducted 5 analyses to infer relationships between mass differences among
217
-------
vehicles holding footprint constant for 5 classes of cars. The results showed one relationship out of five
was statistically significant. As table 1 illustrates, using a simple example, if one conducts 5 trials, each
with a 0.05 probability of given result, there is a 22.6% probability of finding at least one such result in
the five trials. Thus, the joint significance level of the overall result (1 statistically significant regression
out of 5) is 0.226, rather than 0.05.
Table 1. Simplified Illustration of the Joint Probability of Inferences in Multiple Regressions
# Significant
Regressions
0
1
2
3
4
5
Combinations
1
5
10
10
5
1
0.95
0.773780938
0.81450625
0.857375
0.9025
0.95
1
0.05
1
0.05
0.0025
0.000125
0.00000625
3.125E-07
Joint
Probability
77.37809%
20.36266%
2.14344%
0.11281%
0.00297%
0.00003%
100.0%
22.6%
So there is between a 1:4 and a 1:5 chance of getting one statistically significant result by pure chance.
In fact, the actual significance level of the results is more complicated to calculate, and probably a bit
smaller than 0.226. Thus, it is very appropriate for Dr. Kahane to add the qualifier "if any" to his
conclusions about the relationship between the societal highway fatalities and mass reduction, holding
footprint constant. Had appropriate tests of joint statistical significance been used to evaluate the
results in the NHTSA and LBNL studies, the significance levels very likely would not meet accepted
criteria for statistical significance. This could change the conclusions of the studies from the inference
that mass is correlated with fatalities or casualties in some case but not others to the lack of statistically
significant evidence that mass is correlated with fatalities or injuries on the highway. This is an
important difference.
[LBNL] The comment implies that the significance of the estimate on mass for one of five types of
vehicles, or for a handful of nine crash types, does not translate into whether the estimate of mass on
risk across all vehicle types, or across all crash types, is statistically significant. For this reason we
compared the results from NHTSA's nine regressions for nine types of crashes, in which the estimated
coefficients were weighted by the number of fatalities in each type of crash, with results from a single
regression model for all crash types combined (without controlling for crash type). The second and third
columns in Figure 2.1 of the Final Phase 1 report present this comparison. The figure indicates that the
estimated effects of mass reduction on risk, and the statistical uncertainty of those estimates, are
virtually identical, whether one takes the weighted average estimates from the nine regression models
by crash type or whether one runs a single regression model including all types of crashes.
218
-------
Figure 6.13 in the Final Phase 1 report similarly shows the estimated effect of mass reduction from nine
regression models, each including all vehicle types (with control variables for vehicle type) in each
regression model. However, this analysis does not run a single regression model for all types of crashes.
We ran a single regression model, across nine crash types and three vehicle types, including control
variables for vehicle types but not for crash types. The estimated effect of a 100-lb reduction in mass is a
0.15% (+/- 0.23%) increase in US fatality risk per VMT holding footprint constant, while the estimated
effect of a 1-squarefoot reduction in footprint is 0.43% (+/- 0.23%) increase in risk, holding mass
constant.
These results suggest that, across the entire vehicle fleet, the estimated effect of mass reduction on
fatality risk is very small, and not statistically significant, even though mass is associated with risk for
certain vehicle types in certain types of crashes. However, we believe NHTSA is justified in examining the
relationship between mass and risk for certain classes of vehicles.
NHTSA found that there is a statistically-significant, albeit small, estimated increase in fatality risk as
mass decreases in one of five vehicle classes studied, lighter-than-average cars. Although it is possible
that this result was obtained by pure chance, as expected by joint probabilities or the multiple-
comparison fallacy, the fact that this result was obtained for the lightest cars, the vehicle class we expect
to be most sensitive to mass reduction, does not appear to be a purely random result.
Page-by-Page Comments
I will make page by page comments on the phase II study only, since that contains the overwhelming
share of original contributions and the key findings of the phase I study are recapitulated there.
p. iii Paragraph 3. This would be a very good place to acknowledge the importance of driver behavior
and environment on crash avoidance especially.
p. iv Para. 3. This would be a good place to discuss probability inference in joint tests.
Para 4. The statement about a 100-lb reduction in the mass, etc., is a good example of
misleading language.
p. v Para. 1. Again, it is misleading to say that mass reduction increases crash frequency, for reasons
stated above.
Para. 2. It is more accurate to describe the association of lower vehicle mass with casualty risk
than the "effect of mass reduction on..." casualty risk.
Para. 3. Would benefit greatly from joint probability inferences.
Para. 4. As noted above, this shows how much more important the control variables are than
the variables of interest.
Para 5. Again, these are correlations not necessarily effects.
219
-------
p. vi Para. 1. Again, mass reduction is misleading terminology and you do not know if it increases
casualty risk or not, you know only a correlation. Why is this so important? It is the virtual
certainty of spurious correlations, or aliasing, as noted above.
Para. 4. (1st bullet) This is clear evidence of aliasing. Take variables out of the regression and the
coefficients of interest change in important ways. Are there no important factors still missing?
Are the variables included perfect measures of the factors of interest? Of course not. Thus,
there must be remaining aliasing. How bad is it? We don't know.
The third bullet shows the same effect with a different set of variables.
p. vii Para. 4. No, your analysis does not indicate "...that much of the detrimental effect of mass or
footprint reduction on risk can be attributed to the tendency for mass or footprint reduction to
increase crash frequency." Again, you have correlation, not causation and you have good
reason to believe that what you are seeing is affected by spurious correlations.
Para. 5. The "effect" is small, 1-2 orders of magnitude smaller than correlations with other
control variables, and IS strongly affected by which variables are in the equation, as stated on
the previous page, and there is a great deal of unexplained variance. Please reconsider the
meaning of these results in light of the comments above.
Finally, as the last paragraph of the ES implies, it would be far better not to speak in terms of
"reducing" mass or size. That is not what is happening in your data set.
p. 1 Para. 4. Risk per VMT includes the effects of how well vehicles are driven as well as how well
they can be driven. I think there is no chance that you have fully accounted for how well
vehicles are driven.
[LBNL] Including driver age does account for how well vehicles are driven, to some extent. In
addition, LBNL ran a sensitivity where vehicles with "bad" drivers (i.e. crashes involving drug or
alcohol use, or the driver was cited for speeding, another traffic violation, or had been cited in
the previous three years) were excluded from the analysis. Figure 5.14 of the Final Phase 1
report shows that this sensitivity found that the estimated detrimental effect of mass reduction
on risk per VMT increased substantially for cars and lighter light trucks, and the estimated
beneficial effect of mass reduction on risk decreased substantially for heavier light trucks and
CUVs/minivans. Unfortunately, this detailed information on the driver is not available in the
state crash data, so a similar analysis cannot be done to estimate the effect of removing bad
drivers on crash frequency or on risk per crash. However, LBNL did run a sensitivity including
average household income; see Section 5.5 in the Final Phase 2 report.
p. 2 Para. 2. Exposure measures are explanatory variables whose elasticity is constrained to 1. That
is, it is assumed that an increase in vehicle use of 1 vehicle mile produces a 1 unit increase in the
chance of a fatality (or casualty as the case may be). This is actually a maintained hypothesis. If
this hypothesis is incorrect, it can bias the other coefficients in the equation. Thus, the change
from fatalities/VMT to fatalities/registration-year, to fatalities per crash not only changes the
meaning of the analysis, it also may bias coefficients in the event that the true relationship
between fatalities and VMT is not an elasticity of 1.
220
-------
[LBNL] We expect that the regressions using a different measure of exposure or of risk would
estimate a different relationship between vehicle mass and risk.
p. 3 Line 1. Please acknowledge that your "accounting for differences in driver characteristics, crash
locations, and other vehicle attributes" is incomplete and that this could affect your inferences
about size and mass.
[LBNL] The phrase "...included in the NHTSA regression models" will be added to the end of this
sentence.
p. 3 Para. 2. NHTSA's use of "non-culpable" vehicles involved in two vehicle crashes as an exposure
measure raises its own issues. How non-culpable was the non-culpable vehicle. Often this is a
matter of degree, rather than black or white. Driver behavior may also be involved. It seems to
me this is just another potential measure of exposure that may or may not be better than any
other measure and may introduce new sources of bias in the analysis.
[LBNL] DRI suggested using stopped vehicles, rather than non-culpable vehicles, in two vehicle
crashes as the measure of exposure. NHTSA and LBNL ran sensitivity analyses using this
alternative measure of exposure; see Section 5.6 in the Final Phase 1 report.
p. 3 Para. 4. Induced exposure needs to be defined. What is it intended to mean? This needs to be
explicit.
Also, I am startled that there are no equations in these reports. Equations can provide an
unequivocal explanation of the assumed relationships that cannot be adequately accomplished
by words, in many cases. Why no equations?
[LBNL] The method that NHTSA used to develop the vehicle registration and vehicle miles
traveled weights using the non-culpable vehicles from the crash data from 13 states, national
and state vehicle registration data from R.L. Polk, and average vehicle odometer reading by
vehicle year, make, and model is complicated, and thoroughly described in Sections 2.3 through
2.6 of the 2012 NHTSA final report.
p. 7 Para 3. CUVs and minivans are involved in fewer crashes with stationary objects than cars.
Why? Is it the drivers, the vehicles, or the passengers? How well can you control for such
differences? Not well. What does this mean for your analysis?
p. 9 Para 1. Here an equation showing how the weighting was done would be very helpful.
[LBNL] Please refer to Sections 2.3 through 2.6 of the 2012 NHTSA final report for a detailed
description of how NHTSA created the VMT weights.
p. 9 Bullet 1. Excluding these vehicle types implies that the control variables in the model are not
adequate to account for whatever makes these vehicle types different from the vehicles
included in the analysis. First, this is an admission that the model is not adequate to explain the
fatalities associated with these vehicles. Second, it is an admission that if they were included
the coefficients on the variables of interest would likely be biased by spurious correlations.
Clearly, it would not even be sufficient to include the vehicles along with a control variable (e.g.,
221
-------
X = 1 if vehicle is a police car, 0 otherwise). This is yet another indication that the model suffers
generally from omitted variables, errors in variables and correlation among right-hand side
variables.
[LBNL] LBNL agrees that excluding these vehicle types is a tacit admission by NHTSA that the
control variables used in their regression models do not fully account for differences in vehicle
types or the behavior of their drivers. However, LBNL ran a sensitivity in both Phase 1 and Phase
2 draft final reports where these four vehicle types were included in the regression models, with
a control variable for each vehicle type. Figure 5.15 in the Final Phase 1 report indicates that
including these vehicle types increases the estimated detrimental effect of mass reduction on US
fatality risk per VMTfor lighter-than-average cars, but increases the estimated beneficial effect
for heavier-than-average light trucks. Figure 5.12 in the Final Phase 2 report shows that
including these vehicle types has little effect on the estimated effect of mass reduction on
casualty risk per crash.
p. 10 Line 3. Sentence does not make sense. Please correct. [LBNL] The sentence fragment has been
deleted.
p. 12 Sect. 2.3. Please provide an equation. [LBNL] The term "equation" has been replaced with the
term "con version factor."
p. 13 Were the confidence intervals calculated using ex-l? Please state explicitly or, better, show an
equation. [LBNL] Yes, the end of this paragraph states that the confidence intervals were
calculated the same way.
Para. 2. Here another instance where you say "mass reduction increases societal fatality risk"
but you really are not entitled to say that. It is misleading. Also, the NHTSA CI's are larger, as
they should be in a joint test.
Para. 3. These results require an underlying theory for interpretation. The lack of one makes it
seem like there is just no consistency in the results.
[LBNL] A section summarizing the physical theory of vehicles and fatality risk has been added
after Section 1 in both the Final Phase 1 and Phase 2 reports.
Para. 4. The fact that the results for fatalities per crash differ substantially from fatality per VMT
may be very important. Taken at face value, it would imply that any negative effect of reduced
mass is due to its effect on crash avoidance (crash probability) rather than crashworthiness.
This is where the lack of a theory is most troubling. Why would that be? Are lighter vehicles
less easily controlled, etc.? Or, as seems much more likely, is there a spurious correlation
between mass and other omitted or imperfectly measured factors (including driver behavior)
that lead to an increased probability of a crash? Consider, for example, driver age. Driver age is
related to crash involvement. Driver age is a control variable. But are all young drivers the
same? Is it possible that young drivers more prone to risky behavior tend to drive lighter
vehicles? If so, this could partly explain the result observed. Of course, this is just speculative,
but the point is that correlations with imperfectly measured and omitted factors are highly likely
to be present in the data and, if there, could easily affect the statistical inferences.
222
-------
[LBNL] We expect that the regressions using a different measure of exposure would estimate a
different relationship between vehicle mass and risk, although the differences could also be
explained by other factors not included in the regression models.
p. 14 Here we see that changing the exposure measure influence the effect of mass and size on
fatalities and casualties, which is more evidence that spurious correlations are likely to be
biasing estimated coefficients for mass and size.
[LBNL] Again, we expect that the regressions using a different measure of exposure would
estimate a different relationship between vehicle mass and risk. Although the differences could
also be explained by other factors not included in the regression models.
The bar graphs with confidence intervals are well done and convey a great deal of information
effectively. The patterns of magnitude and statistical significance are difficult to interpret, partly
because there is no explicit theory of what should happen and partly, perhaps, because the
relationships are actually not real.
p. 16 Para. 1. Reduction in the mass of lighter cars increases crash frequency but reduces fatalities
per crash. This is contradictory to the previously maintained theory that mass protects due to
the physics of velocity changes in elastic collisions. Indeed, there is no theoretical explanation
for these results, only speculation.
[LBNL] It is unclear why lower vehicle mass is associated with higher crash frequency, but lower
risk per crash, in the regression models. It is possible that including variables that more
accurately account for important differences among vehicles and driver behavior would reverse
this relationship.
Adding vehicle purchase price substantially reduces the estimated increase in crash frequency as
vehicle mass decreases for all vehicle types; mass reduction is now estimated to slightly decrease
crash frequency in the case of heavier-than-average cars, but is still estimated to increase crash
frequency for the other four types of vehicles. See Figure 4.10 in the Final Phase 2 report.
On the other hand, it is also possible that over thirty years of improvements in vehicle design to
achieve high crash test ratings have enabled manufacturers to use clever vehicle design to
mitigate the hypothetical safety penalty of low mass vehicles.
Text to this effect has been added to the Final Phase 2 report.
p. 18 Para. 1. Here is a good example of such casual speculation.
p. 19 Para. 1. Developing VMT weights for the 13 states is a good idea, given the effect of exposure
measures on inferences. Still the results would not be definitive.
p. 20 Para. 2. "Mass reduction leads to a large reduction in risk only in crashes with objects for heavier
cars?" There is only one type of crash in which the simple physics of collisions leads to an
unambiguous benefit for increased mass, and that is collisions with moveable or breakable
objects. This finding contradicts even that.
223
-------
[LBNL] As summarized in the new Section 1.1 in the Final reports, additional mass may be
beneficial if it enables the vehicle to knock down the object and continue moving, thereby
reducing its delta V than if the vehicle was stopped by the object. In a previous study NHTSA
estimated that the object is knocked down in about 25% of frontal collisions with stationary
objects (Partyka, 1995). However, if the object is immovable, reducing vehicle mass would lower
the kinetic energy of the crash, thereby reducing the amount of energy for the vehicle's structure
to absorb, and likely reducing risk. Additional crush space or structural strength would increase
the amount of crash energy the vehicle can absorb, tending to reduce risk to occupants.
Figures 2.12 through 2.14, and Table 2.3, in the Final Phase 2 report indicates that lower mass is
associated with a reduction in 13-state casualty risk per crash in crashes with a stationary object,
for four of the five vehicle classes, although the estimated reduction is large enough to become
statistically-significant only for heavier-than-average cars (3.75%) and CUVs/minivans (2.60%).
Note that in the same table lower footprint is associated with a significant 3.55% increase in
casualty risk in car crashes with an object; smaller size substantially increases risk in crashes with
objects for light trucks (0.99%) and CUVs/minivans (5.56%) as well.
The relationship between lower mass and fatality risk per crash in crashes with stationary
objects is even stronger (not shown in the Final Phase 2 report, but shown in the table below); for
fatality risk per crash, mass reduction is estimated to reduce fatality risk per crash in crashes
with stationary objects in all five vehicle classes, and the estimated reductions are statistically
significant for cars and the lighter light trucks. As in casualty risk per crash, footprint reduction is
estimated to increase risk in all three types of vehicles.
Estimated effect of mass or footprint reduction on risk per
crash, in crashes with a stationary object
Variable
Weight
Footprint
Vehicle type
Cars < 3106
Cars > 3106
LTs < 4594
LTs > 4594
CUVs/minivans
Cars
LTs
CUVs/minivans
Casualty risk
per crash
-0.80%
-3.75%
-0.78%
0.04%
-2.60%
3.55%
0.99%
5.56%
Fatality risk
per crash
-3.96%
-5.14%
-3.98%
-0.81%
-5.77%
4.41%
2.11%
5.93%
Para. 3. More speculation, this time about rollovers.
[LBNL] A section summarizing the physical theory of vehicles and fatality risk has been added
after Section 1 in both final reports. As noted in that section, reducing a vehicle's mass without
changing its roof structure would reduce the force applied on the roof once a vehicle rolls over,
and thus reducing risk.
p. 21 Para. 3. "Curiously,..." Curiouser and curiouser.
Figure 2.15 printed without labels. Could be my computer but the other graphs were fine.
224
-------
p. 24 Para. 2. This is probably a very important finding that needs further investigation and
explanation. As figure 2.16 illustrates well, the correlations with mass and size are orders of
magnitude smaller than the correlations with driver and environmental factors. This is why
even small correlations with omitted or imperfectly measured control factors could be, are even
likely to be, predominantly responsible for the estimated coefficients of mass and size.
p. 25 Para. 2. The results for minivans discussed here could be due to what is going on inside the
vehicle as much or more than the vehicle itself. How could these results be explained in terms
of the vehicle itself. Figure 2.17 shows this again. The effects of calendar year dummies, which
can only be considered rough approximations to unknown and various time-related factors,
have much large effects than size or mass. Again in figure 2.20.
[LBNL] Text discussing unexpected results such as this one will be added to the end of Section 2
in the Final Phase 2 report.
p. 28 Para. 1. "Surprisingly,..." How can side airbags, which deploy only in a crash, reduce crash
frequency but not fatality risk in a crash? Only if the real effect is a reflection of who buys a
CUV/minivan with side airbags and how and where they drive. There are more surprising
inferences in paragraph 2 about male and female drivers. Surprising relative to what theory?
[LBNL] The coefficients for side airbags in cars have the expected effect; airbags have a small
effect on crash frequency, but a large beneficial effect on risk per crash. The unexpected results
for CUVs/minivans could be attributable to the relatively small number of vehicles included in the
analysis. Text discussing unexpected results such as this one will be added to the end of Section
2 in the Final Phase 2 report.
p. 32 The problem here is not numerical multicollinearity (numerical difficulties inverting the cross-
product matrix) but the more complex problem of correlations among right-hand side variables,
omitted variables, errors in variables, and correlations of included variables with omitted and
imperfectly measured variables. This leads to biased estimates.
p. 36 Table 3.1 cries out for inference based on joint probabilities. What is the probability of
observing "success" in at least 3 out of 27 trials when the true probability of success is only 0.05.
See discussion above. The probability is certainly much higher than 0.05 and probably closer to
0.5. The implication is that, taken together, these results do not show any statistically
significant relationship between mass or size and risk per crash. If there were a rigorous
underlying theory, the interpretation might be different (patterns of significance could matter)
but there is none. Again, good graphs on succeeding pages.
[LBNL] We believe that the results in Table 3.1 (Table 3.2 in the Final Phase 2 report) speak for
themselves, without having to calculate joint probabilities. Casualty risk increases with
decreasing mass in more than half of the footprint deciles in only 9 of the 27 vehicle/crash
combinations, and the increase is statistically significant in fewer than half of the footprint
deciles in each of these 9 cases.
p. 41 The statistical significance of such a relationship should be the same whether bins are used or
not. Is it?
225
-------
[LBNL] Because the risk of a casualty is a binary outcome for an individual vehicle, binning is
necessary, either by increments of weight, make and model, or some other factor, to present the
relationship between risk per VMT and weight in a plot. The relationship between risk per VMT
and curb weight is statistically significant when the values for each vehicle weight bin are
weighted by the number of observations in each bin.
The statistical significance of the relationship between casualty risk and weight bin has been
included in a new Table 4.1 in the Final Phase 2 report.
p. 42 Para. 3. R-squared is not the correct measure of statistical significance. Is the coefficient of
weight significantly different from zero?
[LBNL] A new Table 4.1 has been added to both final reports which includes the statistical
significance of the relationship. However, the R2 value is shown to indicate that, when the data
are binned by vehicle weight the relationship appears stronger than when the data are plotted
by vehicle make and model, in the first column of new Table 4.2. For example, the relationship
between US fatality risk per VMT and weight for individual car models in new Table 4.2 in the
Final Phase 1 report (4.0%) is statistically significant; however, the R2 is only 0.17. Therefore, on
average the relationship is statistically significant, but there is a wide range in risk for individual
car models.
p. 43 Para. 2. This is perhaps the key finding of the phase I and II analyses. Control variables explain
1-2 orders of magnitude more variance than size or mass. Still, most of the variance remains
unexplained and is uncorrelated with mass or size. It is very likely there is nothing going on
here.
p. 47 Para. 2. More evidence of correlation of mass and size with control variables and how changing
definitions or excluding control variables results in important changes in the coefficient of mass
and size. Such results are considered unstable.
[LBNL] Section 5.1 in the Final Phase 2 report discusses the importance of including control
variables for each of the thirteen states, to account for differences among the states in reporting
rates for non-injury crashes.
p. 49 Para. 1. The rationale for the grouping of manufacturers is not obvious. Can you explain it?
[LBNL] The analysis has been revised to include the luxury models Lexus, Acura, and Infiniti with
their nameplate manufacturers Toyota, Honda, and Nissan. An additional sensitivity was
conducted using separate indicator variables for the five luxury brands Lexus, Acura, Infiniti,
Cadillac, and Lincoln; see Section 5.3 in the Final Phase 2 report.
Para. 2. Yet more evidence for the instability of the model and likelihood that variables still
missing from the model, plus errors in measuring the included control variables are likely biasing
the inferences. The results described in this paragraph do not make sense to me. How can they
be explained other than random results?
226
-------
p. 50 Para. 1. More casual interpretation of results. OK, maybe, maybe not. Same for paragraph 2.
The economy faltered in 2008 but the big negative effect was in 2007. The downward trend
started in 2004. This correlates neither with vehicle mass changes over time nor economic
growth as measured by real GDP. Idle speculation.
p. 51 Para. 1. Good discussion of the gratuitous speculation by NHTSA about the meaning of the
observed correlations. This is more a Rorschach test than statistical analysis.
p. 51 Para. 2. "We have no explanation for why..." More of this kind of honest appraisal is needed in
studies like these.
p. 52 More results for which there is no explanation.
p. 53 Why the interaction between calendar year dummy variables and safety equipment? The
presence of the safety equipment on a particular vehicle is established. What has calendar year
(not model year) to do with it? Again, one suspects spurious correlations.
p. 55 One needs to think carefully about the reasons why vehicles would be excluded. It does not
appear that NHTSA did that. First line of first paragraph "used" rather than "sused". [LBNL]
Text has been changed.
p. 57 Well reasoned. It is interesting that NHTSA resisted including footprint or size in previous
analyses on the grounds of correlation with mass. These results show that assertion was
groundless.
p. 59 Para. 3. Rather than say risk per VMT accounts for two effects, it is better to say it includes or
comprises two effects. [LBNL] Text has been changed. But this statement also ignores the
important influences of drivers and environment and their potential correlations with other
factors. Yes, it includes how well a vehicle can be driven, but more importantly it includes how
well a vehicle IS driven. That is in there too and is very likely to be correlated with make, model,
and other vehicle attributes.
p. 60 Para. 3. Here again, the conclusions are misstated. It is not a genuine "reduction" in mass, but
an association with the mass of vehicles. And how does it "lead" to and increase in crash
frequency? What is the theory or model that predicts this? Driver and environment are very
likely mixed up in these results to an unknown but likely substantial degree. So what can we
really conclude? Not this. [LBNL] Text has been changed.
As I read the conclusions and inferences I find myself asking, why?, why?, repeatedly without
any sound explanations. Page 61, paragraph 3 contains more "surprising" results. Surprising
because they are contrary to theory? Surprising because they are contrary to intuition?
Surprising because they are random? To what can we attribute so many "surprising" results,
and how many must there be before one concludes that the analysis is not revealing what we
had hoped it would.
Para. 4. Again, this cries out for joint statistical inference. Three statistically significant results
out of 27 is probably nothing statistically significant at all.
227
-------
p. 62 Para. 1. What this shows, again, is that the coefficients of mass and size are strongly influenced
by which control variables are included in the model and how they are defined. These results
and their implications need explanation. The bottom line is that the effects of mass and size are
likely to be (after the necessary joint significance calculations are done) not statistically
significant, not consistent, and not robust.
p. 63 How do mass and footprint reduction (again, it's not really reduction in the sense of designing
lighter vehicles to increase fuel economy or reduce GHG emissions, the issue at hand) increase
crash frequency. What is the theory? I don't find a theory in either the NHTSA report or the
phase I and phase II studies. Absent a theory, these results seem sufficiently unstable and
inconsistent to be highly questionable as evidence of any relationship between mass or size and
crashworthiness or crash avoidance. I think joint estimation of significance levels would provide
additional support for this view.
[LBNL] Section 1.5 of NHTSA's 2011 report summarizes the hypothetical physical relationships
between vehicle mass, footprint, and societal fatality risk. LBNL has added a section
summarizing this discussion at the end of Section 1 in each of the final reports.
4. References
Chen, H.Y., Ivers, R.Q., Mariniuk, A.L.C., Boufous, S., Senserrick, T., Woodward, M., Stevenson,
M. and Norton R. 2010. "Socioeconomic status and risk of car crash injury, independent of
place of residence and driving exposure: Results from the DRIVE study." Journal of Epidemiology
and Community Health 64(10), pp. 998-1003.
Kweon, Y-J. and Kockelman, K.M. 2003. "Overall Injury Risk to Different Drivers: Combining
Exposure, Frequency, and Severity Models." Accident Analysis and Prevention 35(4), pp. 441-
450.
Wenzel, Tom P. 2011a. Assessment of NHTSA's Report "Relationships Between Fatality Risk,
Mass, and Footprint in Model Year 2000-2007 Passenger Cars and LTVs". Preliminary report
prepared for the Office of Energy Efficiency and Renewable Energy, US Department of Energy,
Berkeley, California. August. LBNL-.
Wenzel, Tom P. 2011b. An Analysis of the Relationship between Casualty Risk Per Crash and
Vehicle Mass and Footprint for Model Year 2000-2007 Light-Duty Vehicles. Preliminary report
prepared for the Office of Energy Efficiency and Renewable Energy, US Department of Energy,
Berkeley, California. December. LBNL-.
Wenzel, Tom P. 2012a. Assessment of NHTSA's Report "Relationships Between Fatality Risk,
Mass, and Footprint in Model Year 2000-2007 Passenger Cars and LTVs". Final report prepared
for the Office of Energy Efficiency and Renewable Energy, US Department of Energy, Berkeley,
California. August. LBNL-.
Wenzel, Tom P. 2012b. An Analysis of the Relationship between Casualty Risk Per Crash and
Vehicle Mass and Footprint for Model Year 2000-2007 Light-Duty Vehicles. Final report prepared
228
-------
for the Office of Energy Efficiency and Renewable Energy, US Department of Energy, Berkeley,
California. August. LBNL-.
229
------- |