Peer Review of LBNL Statistical
            Analysis of the Effect of Vehicle Mass &
            Footprint Reduction on Safety (LBNL
            Phase 1 and 2  Reports)
&EPA
United States
Environmental Protection
Agency

-------
                   Peer Review of LBNL Statistical
              Analysis of the Effect of Vehicle Mass &
               Footprint Reduction on Safety (LBNL
                        Phase 1 and 2  Reports)
                            Assessment and Standards Division
                            Office of Transportation and Air Quality
                            U.S. Environmental Protection Agency
                                 Prepared for EPA by
                         Systems Research and Application Corporation
                              EPA Contract No. EP-C-11-007
&EPA
United States
Environmental Protection
Agency
EPA-420-R-12-020
August 2012

-------
Peer Review of LBNL Statistical Analysis of the
 Effect of Vehicle Mass & Footprint Reduction
    on Safety (LBNL Phase 1 and 2 Reports)
                     Table of Contents
      Peer Review of the LBNL Phase 1 and 2 Reports, Conducted by SRA International
         p. 4
      1.  Background                                         p. 4
      2.  Description of Review Process                              p. 4
      3.  Compilation of Review Comments                            p. 6
      4.  References                                         p. 53
      Appendices
         A.  Resumes of Peer Reviewers                             p. 54
         B.  Conflict of Interest Statements                           p. 86
         C.  Peer Review Charge & Conference Call Notes                   p. 98
         D.  Reviews                                         p. 100
      EPA's Response to Peer Review Comments                         p. 158

-------
                                Executive Summary
In September 2011, EPA contracted with SRA International (SRA) to conduct a peer review of LBNL
Statistical Analysis of the Effect of Vehicle Mass & Footprint Reduction on Safety) (LBNL Phase 1 and 2
Reports), prepared by Tom Wenzel, Lawrence Berkeley National Laboratory.

The peer reviewers selected by SRA were Donna Chen and Kara Kockelman (University of Texas at
Austin), Charles Farmer (Insurance Institute for Highway Safety), David Greene (Oak Ridge National
Laboratory), and Michael Van Auken (Dynamic Research, Inc.).  EPA would like to extend its appreciation
to all three reviewers for their efforts in evaluating this survey.  The reviewers brought useful and
distinctive views in response to the charge questions.

The first section of this document contains the final SRA report  summarizing the peer review of the LBNL
Phase 1 and 2 Reports, including the detailed comments of each peer reviewer and a compilation of
reviewer comments according to the series of specific questions set forth in the peer review charge.
The SRA report also contains the peer reviewers' resumes, completed  conflict of interest and bias
questionnaires  for each reviewer, and the peer review charge letter. The second major section contains
our responses to the peer reviewers' comments.  In this section, we repeat the compiled comments
provided by SRA and, after each section of comments, provide our response. We have retained the
organization reflected in SRA's compilation of the comments to aid the reader in moving from the SRA
report to our responses.

-------
TO:            Cheryl Caffrey, U.S. Environmental Protection Agency, Office of Transportation and Air
               Quality (OTAQ)

FROM:         Brian Menard, SRA International

DATE:          February 28, 2012

SU BJ ECT:       Peer Review of LBNL Statistical Analysis of the Effect of Vehicle Mass & Footprint
               Reduction on Safety) (LBNL Phase 1 and 2 Reports), prepared by Tom Wenzel, Lawrence
               Berkeley National Laboratory.

1.      Background

In developing programs to reduce greenhouse gas (GHG) emissions and increase fuel economy of light-
duty highway vehicles, the U.S. Environmental Protection Agency (EPA) and the National Highway
Transportation Safety Administration (NHTSA) have to evaluate the safety of mass reduction
technologies likely to be used to meet future standards.  The U.S. Department of Energy (DOE)
contracted with Lawrence Berkeley National Laboratory (LBNL) to perform a statistical analysis of the
effect of vehicle mass and footprint reduction on safety. LBNL's analysis of the relationship between
vehicle mass, footprint, and  societal fatality and casualty risk consisted of two phases.  Phase 1 was an
assessment of the NHTSA report Relationships between Fatality Risk, Mass, and Footprint in Model Year
2000-2007 Passenger Cars and LTVs. This study used logistic regression analysis to estimate the
relationship of changes in vehicle  mass and footprint on U.S. fatality risk per vehicle mile traveled.
Phase 2 was an independent logistic regression analysis to estimate the relationship between vehicle
mass, footprint and total casualty (fatality plus serious injury) risk, per police-reported crash, using state-
level data on all crashes.

This report documents the peer review of Assessment of NHTSA's Report "Relationships Between
Fatality Risk, Mass, and Footprint  in Model Year 2000-2007 Passenger Cars and LTVs" (LBNL Phase 1
Report) and Analysis of the Relationship between Casualty Risk Per Crash and Vehicle Mass and Footprint
for Model Year 2000-2007 Light-Duty Vehicles (LBNL Phase 2 Report).  Section 2 of this memorandum
describes the process for selecting reviewers, administering the review process, and closing the peer
review. Section 3 summarizes reviewer comments according to the series of specific questions set forth
in the peer review charge. The appendices to the memorandum contain the peer reviewers' resumes,
completed conflict of interest and bias questionnaires for each reviewer, and the peer review charge
letter.

2.      Description of Review Process

In September 2011, OTAQ contacted SRA International to facilitate the peer review of the LBNL Phase 1
and 2 Reports. The reports were prepared  by Tom Wenzel, Lawrence Berkeley National Laboratory.

EPA provided SRA with a short list of subject matter experts from academia and industry to serve as a
"starting point" from which to assemble  a list of peer reviewer candidates.  SRA selected three
independent (as defined in Sections 1.2.6 and 1.2.7 of EPA's Peer Review Handbook, Third Edition)
subject matter experts to conduct the requested reviews. SRA selected subject matter experts with

-------
experience in statistics and statistical model analysis, vehicle mass safety, and vehicle crash and safety
engineering. The coverage of these subject areas is shown below in Table A.
                                           Table A:
                            Peer Reviewer Experience and Expertise
Name
David Green
Charles Farmer
Donna Chen& Kar
Kockelman
Michael
Van Auken
Affiliation
Oak Ridge
National
Laboratory
IIHS
University of
Texas
Dynamic
Research Inc.
Coverage
Statistics &
methodology
Y
Y
Y
Y
Knowledge of
past vehicle
mass safety
statistics study
Y
Y
Y
Y
Knowledge of
vehicle
crash/safety
engineering
/
Y
Y
Y
Knowledge of
vehicle safety
database (FAS,
State Crash &
Vehicle Attributes)
Y
Y
Y
Y
Statistical
model
analysis
Y
Y
Y
Y
To ensure the independence and impartiality of the peer review, SRA was solely responsible for
selecting the peer review panel. Appendix A of this report contains the resumes of the three peer
reviewers. A crucial element in selecting peer reviewers was to determine whether reviewers had any
actual or perceived conflicts of interest or bias that might prevent them from conducting a fair and
impartial review of the CVCM and documentation.  SRA required each reviewer to complete and sign a
conflict of interest and bias questionnaire. Appendix B of this report contains an explanation of the
process and standards for judging conflict and bias along with copies of each  reviewer's signed
questionnaire.

SRA provided the reviewers a copy of the most recent version of the LBNL Phase 1 and 2 Reports as well
as the peer review charge containing specific questions EPA asked the reviewers to address. The charge
included a matrix of questions issues upon which the reviewers were asked to comment.  Reviewers
were also encouraged to provide additional comments, particularly in their areas of expertise and work
experience. Appendix C of this report contains the memo to reviewers from SRA with the peer review
charge and response matrix.

EPA sought peer reviewers' expert opinions on the statistic methodologies used in the LBNL Phase 1 and
2 Reports and whether they are likely to yield realistic estimates of the relationship between vehicle
mass, footprint, and total fatality or casualty risk. EPA requested that each reviewer comment on all
aspects of the two LBNL studies, with particular emphasis on the methodologies employed, assumptions
inherent to the analysis, sources of information employed,  methods of calculation and any other key
issues the reviewer may identify.  Reviewers were encouraged to examine and evaluate the NHTSA
study in  helping them to understand the LBNL assessment analysis. Findings  of this peer review may be

-------
used toward validation and improvement of the statistical analysis conducted by LBNL, and to inform
EPA staff on potential use of the regression results for predicting the safety effect of future standards in
reducing mass and footprint.

A teleconference between EPA, LBNL, the reviewers, and SRA was held to allow reviewers the
opportunity to raise any questions or concerns they might have about the LBNL Phase 1 and 2 Reports,
and to raise any other related issues with EPA and SRA, including EPA's expectations for the reviewers'
final review comments.  SRA delivered the final review comments to  EPA by the requested date. These
reviews, contained in Appendix D of this report, included the reviewers' response to the specific charge
questions and any additional comments they might have had.

3.     Compilation  of Review Comments

The LBNL Phasel and 2 Reports were reviewed by Donna Chen and Kara Kockelman (University of Texas
at Austin), Charles Farmer (Insurance Institute for Highway Safety), David Greene (Oak  Ridge National
Laboratory), and Michael Van Auken (Dynamic Research, Inc.).  Appendix A contains detailed resumes
for each of the reviewers. This section provides a compilation of their comments. The complete
comments may be found in Appendix D.

-------
                                                        Assessment of NHTSA's Report
                 ''Relationships Between Fatality Risk, Mass, and Footprint in Model Year 2000-2007 Passenger Cars and LTVs"
                                                            (LBNL Phase 1 Report)
    1.  ASSUMPTIONS
                                            COMMENTS
Please comment on the validity of any
assumptions embedded in the LBNL
assessment analysis and the independent
casualty analysis that could affect the
projected relationship between vehicle
mass/footprint reductions and
fatality/casualty risk. Examples might include
assumptions regarding whether recent
historical relationships between vehicle
weight, size, and safety will continue into the
future; potential future improvements in
vehicle technology and design may result in
compensatory safety benefits; and the annual
baseline fatality distribution.
[Chen and Kockelman]  [1] The report does a nice job discussing recent trends in vehicles, such as the increase
of ESC, side airbags, and light truck crash compatibility with passenger cars - which will improve safety
outcomes for all vehicles, but perhaps most significantly the smaller and lighter vehicles. .  It also mentions the
phasing out of the lightest and smallest vehicles between model years 2000-2007 (but doesn't mention the
makes and models somehow), which were particularly poor safety performers in the past. However, with the
introduction of urban commuter vehicles, such as the SmartCar, Mini Cooper, and  Fiat 500m, and the growing
popularity of smaller, fuel-efficient compact vehicles following gas price increases, this trend does not seem so
obvious. Such vehicles should be discussed.

[2] The simplistic logistic model employed in this analysis only accounts for two crash  outcomes (fatal versus
non-fatal) and so neglects the more detailed, and ordered nature of injury severity data, which is unfortunate.
The model also assumes error-term homoscedasticity from  one crash or individual to  the next; in reality
certain vehicle types (e.g., pickups) and crash contexts (e.g., high speed crashes) have more uncertainty
associated with their severity outcomes. It would be good to point out such limitations for readers.

[Van Auken] The basic assumptions, methodology, and data are primarily the same as in the Kahane (2011)
report.  These include the following:

1) The probability of a crash fatality is proportional to the vehicle miles travelled (VMT), except as noted in
   Section 5.1
2) The logarithm of probability of fatality per VMT for a given curb weight, footprint,  and control variable
   values varies as a linear combination of the curb weight, footprint, and control  variables within the  domain
   of the data.
3) The logistic regression  methods determine a maximum likelihood estimate of model  coefficients.
4) It is assumed that the above relationships remain constant in the recent past (i.e.,  2000-2007 model year
   vehicles in the 2002-2008 calendar years), present, and near future (i.e., 2017-2025 model year vehicles).

The first assumption that crash fatalities are proportional to VMT ratherthan the number of vehicle

-------
                                            registration years (VRY) is appropriate because the fatalities cannot occur if the vehicles are not driven on the
                                            road (i.e., VMT = 0). This assumption is qualified however because VMT is more difficult to measure than VRY
                                            and therefore may be less accurate. On the other hand the probability of a fatal crash or the number of
                                            fatalities in a crash may also depend on the vehicle occupancy. The analysis in Section 5.1 is a commendable
                                            attempt to explore the sensitivity to this assumption, however the Kahane (1997) and DRI (2003-2005) reports
                                            have shown that some driver, vehicle, and environmental factors may be underrepresented or
                                            overrepresented in unweighted induced-exposure data. VRY could have also been considered as a measure of
                                            exposure.

                                           The second and third assumptions are appropriate provided that it is recognized that  it is essentially
                                            impossible with currently available knowledge and information to model  all of the factors that could affect the
                                            probability of fatality in a crash, and that the objective of the analysis is to identify overall trends versus
                                            vehicle weight and footprint.  In general the probability of fatality depends on  other many other factors which
                                            have not been modeled (e.g., driver behavior factors, vehicle design factors, roadway design factors,  EMT
                                            factors) and these unmodeled factors are assumed to be uncorrelated with vehicle weight and footprint,
                                            and/or are represented by the other control variables. The latter assumption might or might not be valid.

                                           The fourth assumption is  perhaps the weakest because it assumes that future vehicles will have the same
                                           design characteristics as past vehicles, and that the characteristics of the vehicle population (e.g., collision
                                           partner weight, size, type) will also remain the same. A commendable attempt to partially address this effect
                                           is described in Section 6.  These effects can be perhaps better addressed by the "Volpe model" described in
                                           Kahane (2011) of the Honda-DRI fleet systems model described in Refs  (2324),  which can be used to forecast
                                           the effects of mass reductions of individual makes and models on a year-by-year basis.
Please comment on any apparent unstated or
implicit assumptions and related caveats or
limitations.
[Chen and Kockelman] The role of driver behavior is briefly addressed in the report but not emphasized
sufficiently. Fatality risk is a combination of driver, vehicle, and roadway characteristics. Driver behavioral
differences are many and do not solely exist for pickup truck drivers versus car drivers. Socioeconomic data
such driver household income, size, and education influence driver attitudes and driving environments.  For
example, Chen et al. (2010) found that crash risk increases for those living in socioeconomically disadvantaged
areas (including households more likely to drive less expensive and older vehicles). Though such data is not
typically available in state and national crash databases, the importance of these driver and environmental
characteristics on crash rates (per mile driven) and fatality risk should be stressed in both reports. It is clearly
very difficult to control for, but a major caveat to the NHTSA (& now LBNL) results. We expect that crash
severity could be probably be lower for many of the small cars and pickups if they were driven by those  who

-------
                                           tend to drive more expensive vehicles, under the same settings (e.g., daytime, urban freeway).  Similarly, in the
                                           second LBNL report (which uses VMT estimates), we expect that crash rates would probably be lower for these
                                           types of driver-vehicle-setting combinations.

                                           [Farmer] The statistical models assume no interaction between the vehicle size/weight measures and any of
                                           the numerous covariates, but this may not be true.  For example, size/weight reductions may differently affect
                                           vehicles with and without ESC if they affect vehicle handling. It is risky to make statements such as that on p.
                                           11 of the Phase I report:  Therefore, the mass of a lighter car could be reduced by 800 Ibs while adding ESC,
                                           without increasing fatality risk.

                                           [Van Auken] The  induced-exposure data set provided by NHTSA is based on the "non-culpable" vehicle in two-
                                           vehicle crashes. It is assumed that the dataset is a reprehensive sample of the driver and environmental
                                           exposure factors for vehicle use. However, since these cases include moving vehicles, some vehicle-driver-
                                           environmental conditions may be under or over represented in this data depending on how they affect the
                                           ability of a non-culpable vehicle to avoid a crash. Results in Ref (17) indicate that the estimated effect of
                                           weight and size reduction are sensitive to whether the induced-exposure data are based on the Kahane (2003)
                                           non-culpable vehicle definition of the Kahane (1996) stopped vehicle definition.
                                           Unfortunately it is not currently possible to test this sensitivity with the  NHTSA-provided induced-exposure
                                           data.
ADDITIONAL COMMENTS:

[Chen and Kockelman]  Chen, H.Y., Ivers, R.Q., Mariniuk, A.L.C., Boufous, S., Senserrick, T., Woodward, M., Stevenson, M. and Norton R. Socioeconomic status
and risk of car crash injury, independent of place of residence and driving exposure: Results from the DRIVE study. Journal of Epidemiology and Community
Health 64(10), 2010, pp. 998-1003.

-------
    2.  CONTROL AND DEPENDENT
       VARIABLES USED IN THE REGRESSION
       MODELS
                                             COMMENTS
Please comment on the adequacy of control
and dependent variables used in the
assessment analysis and independent casualty
analysis, and recommend any alternative
control or dependent variables that are
available for possible inclusion in the analysis.
For example, what are the relative merits of
the main dependent variables used, fatality
risk per estimated VMT, and casualty risk per
police-reported crash?
[Chen and Kockelman] [1] As alluded to above, a primary concern is that the NHTSA analysis (& thus the LBNL
analyses) largely neglect the idea that vehicle type (make & model) is very much a proxy for driver type, and a
vehicle's crash avoidance may have very little to do with vehicle type. It has a lot to do with the person behind
the wheel, and gender & age simply aren't enough to control for such distinctions. Education, risk aversion,
ability, wealth, etc., are important covariates. But existing data sets are quite limiting (though the MVOSS &
FAR with 3-year driver violation history do offer some valuable insights, not discussed in these  reports). In
reality, small cars may be less crash prone than Kahane's & Wenzel's results suggest, because they are driven
by lower-income, younger, less risk averse people driving in more crash prone settings (e.g., commercial strips
rather than pricey residential suburbs). Such key caveats need thoughtful discussion. Four relevant papers on
the topics of crash frequency and vehicle size-and-weight implications (by Knipling, Kweon & Kockelman, Wang
and Kockelman, and Chen & Kockelman) have been sent to Tom Wenzel. These all include useful  literature
reviews for further connections to useful findings for citation in the reports, as time allows the contractor.

[2] The grouping of the vehicles into heavier- and lighter-than-average weight categories essentially splits a
"typical" weight vehicle of that type into two categories. The impacts of curb weight and footprint on fatality
risk may be easier to interpret if the vehicles were grouped into 3 weight categories (light, average, and heavy)
& by type (with the average category representing vehicles within one standard deviation of average weight).
Furthermore, the grouping of CUVs and minivans into the same vehicle type category neglects  the fact that
these vehicles have faced rather different ground clearance requirements (impacting rollover potential), door
types (sliding vs. standard), and, perhaps most importantly, can appeal to different types of drivers (as
indicated in the market shift of car drivers to CUV drivers).

[Farmer] One needs to restrict control variables to  those that are available  and reliable.  A problem when
combining state databases is that the states often are not consistent as to the variables coded  and the
definitions of those variables. This severely limits the list of possible control variables.

[Van Auken]  [1] The main metric used in both the Kahane (2011) and Wenzel (2011a) reports  is the total
number of fatalities (except as noted).  Reducing the total  number of fatalities, which  includes  both subject
vehicle occupants and collision partner fatalities, is  desirable from a societal viewpoint.  Fatal crash occurrence
is related to the total number of fatalities, which has been used by Kahane (2003, 2011) to address concerns
                                                                                                                                       10

-------
                                            about double counting.

                                            [2] VMT is a good measure of accident exposure provided that it can be accurately determined. [Note: This
                                            peer review does not address the Wenzel (2011b) companion report (Ref 3) which examines the risks per
                                            police-reported crash. See Ref 4 for comments on the companion report.]
What additional control variables, such as
vehicle make or model, might be included in
the regression  models?
[Chen and Kockelman]  [1] Vehicle height, a variable which may be more valuable than vehicle type for
similarly structured vehicles such as sedans, wagons, CUVs, and minivans, would be a valuable control variable.
In addition to a wider track, a lower center of gravity also increases vehicle stability, thereby reducing the risk
of rollover.  Relevant literature & findings exist, and should be cited.

[2] Other variables which have been found in past studies to influence fatality risk such as seat belt use,
roadway geometry and  division type are not included in this study (which is largely a repeat of the NHTSA
study, as specifically contracted by the EPA).

[3] To account for driver characteristics that contribute to fatality risk, socioeconomic variables such as
household income, education, household size, etc. would be valuable additions. Unfortunately, both state and
national crash databases typically do not include such information (outside of MVOSS). Such issues should be
flagged for readers. It seems the contractor has done his duty, and the key limitations lie with the original
methodology he was to essentially duplicate.
Please comment on any caveats or limitations
that these dependent variable or control
variables entail with respect to use of the
results as the basis for estimating the safety
effect of mass reduction.
[Chen and Kockelman]  Please see above comment (in Assumptions section) regarding driver behavior and
environment.

[Farmer] Model overspecification could be the reason for results that are non-intuitive, especially in the Phase
II analyses of police-reported crashes. Control variables may be correlated with each other or with the size
and weight variables. For example, Figure 2.9 of the Phase I report implies that torso side airbags increase
fatality risk in CUVs.
ADDITIONAL COMMENTS:

[Chen and Kockelman]  Table 2.1 has many indicator variables labeled as "C" for continuous variable (such as ABS, ESC, AWD, DRVMALE, etc). These C's should
be removed.

[Van Auken] The underlying reasons for some of the estimated effects are unknown at this time, but presumably involve driver, vehicles, environment or
                                                                                                                                       11

-------
accident factors that have not been controlled for in the Kahane (2011) and Wenzel analyses. See, for example, Refs 17 and 25.
                                                                                                                                    12

-------
    3.  METHODOLOGY AND STATISTICS
                                            COMMENTS
Please comment on the validity and
applicability of the methodology LBNL used in
assessing the NHTSA 2011 study and its
analysis of the  relationship between mass,
footprint, and risks per police-reported crash.
[Chen and Kockelman]  The report assesses the NHTSA 2011 study in a fair amount of detail and seeks to
introduce some additional analyses to better examine the relationship between mass, footprint, and fatality
risks. However, due to a lack of control for very specific vehicle differences (which vary by make & sub-model),
the exclusion of driver characteristics and crash setting details (which cannot always be controlled for, but are
often correlated with vehicle type), the effects of downweighting vehicles and/or shifting vehicle styles and
sizes may be overestimated. Simply changing the vehicle on a risky driver in a high-risk setting is  unlikely to
influence outcomes significantly.

[Van Auken] The logistic regression methods seem to be appropriate. The confidence intervals are based on
the logistic regression Wald Chi-Square statistic, which as Kahane (2003, 2011) has demonstrated does  not
include all sources of variation. However, these confidence intervals are useful because they do  provide some
indication of the uncertainty in the results. [Note: This peer review does not address the Wenzel (2011b)
companion report, which examines the risks per police-reported crash. See Ref 4 for comments  on the
companion report.]
Please review other statistical methods LBNL
has used in the analysis, in addition to the
logistic regression methodology. Examples
include the alternative approaches used by
LBNL to assess NHTSA interval estimation
results, and LBNL's linear regression analysis
of actual, predicted, and residual risk by
vehicle model.
[Chen and Kockelman]  [1] In the alternative measures of exposure, the author examines the effect of vehicle
manufacturer on fatality risk and treats the luxury models produced by Toyota, Honda, and Nissan as separate
manufacturers. However, domestic luxury brands (such as Cadillac & Lincoln) are categorized with their
nameplate manufacturers (GM and Ford), which appears inconsistent.

[2] The effect of calendar year variables on fatality risk may be overestimated here, since VMT is tracked by
vehicle model and not by calendar year. The trend of greatest fatality risk reductions in light trucks,  CUVs and
minivans with increasing calendar year may simply be a reflection of rising gas prices in combination with the
ailing economy contributing to lower VMT (in these relatively low-fuel-economy vehicles).

[3] It is unclear how the author determined the various percentage replacements of vehicle types in the
aggressive vehicle market share shift scenario. (For example, why are  50% of SUVs replaced  by CUVs and 60%
of small pickups replaced by CUVs? The CUV is a more natural replacement for an SUV, and an SUV a more
natural replacement for a pickup.)

[Van Auken] [1]  The correlations in Section 3 appear to be assessed using the Coefficient of Multiple
Determination (R2) based on a linear fit to the data (e.g., the correlation between footprint versus curb weight
in Figure 3.1 on p. 14). The linear regression  model attributes the differences between the dependent variable
                                                                                                                                      13

-------
                                            (vertical axis_ and the linear fit to the independent variable (horizontal axis) to random effects. If there is no
                                            preference as to the choice of independent and dependent variables (e.g., footprint versus curb weight, or
                                            curb weight versus footprint), then the linear trend and R2 result would be different if the two variables were
                                            interchanged, and having two different yet equally valid results would be undesirable.

                                            [2] If the variation in the data can be attributed to both variables (e.g., footprint and curb weight), then it
                                            would be better to report the square of the sample correlation coefficient r2,, where r is computed according
                                            to Eqn (1). The trend lines in these correlation figures should not be computed using a linear regression.
                                            Instead, the trend line should  pass through the sample means (I.e. (x, y)), and have a slope equal to the ratio of
                                            the sample standard deviations in the data (i.e., sy/sx). Therefore, the reported correlation results  do  not
                                            depend  on the ordering of the data variables.

                                            Note this comments does not apply to linear trends indicated in Section 4, for which the  Coefficient of  Multiple
                                            Determination (R2) seems appropriate.

                                            [3] The Coefficient of Multiple Determination (R2) is frequently used in the Wenzel (2011a) report as an
                                            indicator of the statistical importance of a linear trend (e.g., lvalues in Tables 4.1 and 4.2 that are  greater
                                            than 0.3 are shown in blue font).  It would be better to report the standard error, confidence interval, and/or
                                            probability value as measures of the statistical  significance of a linear trend.
Please comment on caveats or limitations of
using non-significant regression estimates to
project the safety impact of mass reduction.
[Chen and Kockelman] First, the t-statistics are not provided in the report which makes it difficult for the
reader to assess statistical significance of specific regression estimates (except where noted by the author).
Second, inclusion of a statistically insignificant variable can influence the estimates of coefficients associated
with related variables. Nevertheless, in general, it is best to keep insignificant estimates if one has a strong
defense for their role, since removing such variables (& thus their parameters) will shift the burden of
response to a correlated covariate's parameter, thus biasing the latter. We generally keep key covariates in a
model  up to a pvalue of 0.20 or 0.25 or so, especially in relatively small data  sets (e.g., n < 1,000). Covariates
for which we have no strong basis can be removed for pvalues > 0.10.

[Van Auken] Regression estimates are random numbers which have an unknown expected value and variance,
and known sample value and standard error.  If the sample value can be explained by a zero expected value
and known standard error then the result is considered not statistically significantly different than zero and
therefore the result is not considered to be statistically significant. However, If we can combine this estimate
with other estimates then the unknown expected values and variances can also be combined using the same
                                                                                                                                        14

-------
                                            transformation, and the statistical significance of the combined result can be tested. Therefore, depending on
                                            the sample values and inter-correlation, the combined result may be statistically significant even if the
                                            individual estimates are not statistically significant.

                                            For example, the results from each of the nine different crash types can be combined into an overall estimate
                                            and the standard error calculated assuming that the results for each crash type are independent of each other.
                                            Then the statistical significance of the combined effect can be determined.

                                            However, and Kahane (2011) points out there are two sources of uncertainty in the regression results. The
                                            first is the  PARS based sampling error which is uncorrelated across crash types because they are based on
                                            different fatal cases (Kahane 2011, p. 77). The second is the state based include-exposure sampling error
                                            which is correlated across crash types because they are based on the same included-exposure cases.
                                            Therefore  a confidence interval estimated using the jackknife method described by Kahane (2011) and
                                            accounting for correlation of these two error sources would  be more accurate than a simple estimate based on
                                            the Wald Chi-Square statistic and assumed independence.
How might the LBNL methodology be
strengthened to better represent future
vehicle designs and reduce multi-collinearity
between mass and footprint in the regression
analysis?
[Chen and Kockelman] Including more vehicle-specific characteristics (such as vehicle height and engine size)
reduces the analysis' dependence on vehicle type, since vehicle shapes and structures will continue to evolve.
There is also correlation with context (e.g., pickups are driven in more rural locations, with greater hazards
[like less lighting, higher speed, & few medians]). Disaggregate data are almost always best, to avoid ecological
fallacies & such.

[Van Auken]  [1] The effects of multi-collinearity can be mitigated by 1) obtaining more data, 2) pooling data
from different crash type or vehicle types, or 3) reducing the number of regression variables. The first option
would require more calendar years and/or model years, which would involve added newer data as it becomes
available (or using older data).  The second option might be to recombine the CUVs and minivans with truck
based vans and adding a control variable to compensate for the differences in the vehicles types. The third
option might involve removing statistically insignificant control variables or removing control variables that
would not be expected to have an effect on the probability of fatality in the crash (e.g., the side airbag variable
is not included in pedestrian crashes because it is not expected to affect pedestrian fatality risk). The number
of driver age control variables might be reduced from eight to three (as in the Kahane (1997) and DRI (2002-
2005) studies). Finally, a linear curb weight model instead of a two-piece linear model may help to better
elucidate the general trend.
                                                                                                                                        15

-------
                                           [2] The Variance Inflation Factor (VIF) has been suggested as a measure of multi-collinearity in the Kahane
                                           (2010 and 2011) reports, however this diagnostic metric does not account for differences in database size (i.e.,
                                           Options 1 and 2 above).  The Wenzel (2011a) report does not discuss the Variance Inflation Factor or report
                                           any VIF results.
ADDITIONAL COMMENTS:

[Chen and Kockelman]  On page 55, it is unclear what is meant by "however; if anything, reduction of this type of fatality will increase detrimental effect of
mass reduction in cars."
                                                                                                                                      16

-------
    4.  DATA SETS
                                            COMMENTS
Please comment on the validity and
applicability of the datasets used to project
changes in risk resulting from reduction in
vehicle mass. LBNL's casualty analysis used
police-reported crash data from 16 states,
while the 2011 NHTSA study used national
fatality data, combined with a subset of non-
culpable vehicles involved in two-vehicle
crashes from police-reported crash data from
13 states.
[Chen and Kockelman]  [1] The acquisition of Polk data for VMT estimates by make & model is valuable, and a
contribution to the literature. However, these estimates come from vehicles found in repair shops in
non-attainment areas, and so will be biased towards problem-prone vehicles, wealthier households who
service their vehicles more regularly, and/or urban (smoggier) areas. Such issues merit careful discussion in the
paper, so that readers are well aware of caveats.

[2] Related to this, Tom Wenzel indicated (by phone) that he did take a look at CA's extensive odometer reads,
which go into some semi-rural locations (not too rural), and he indicated that the VMT values by vehicle type
(not controlling for HH attributes & such) are very similar (just 5% longer in rural areas) -except for vans
(which are used much more extensively in rural areas). This is interesting to me, and is not that different from
what we've seen in the  past. For example, Kockelman & Zhao's JTS paper from 2000 (pre- print at
http://www.ce.utexas.edu/prof/kockelman/public_html/
BTSJournalLDTs.pdf) suggests that, after controlling for various HH attributes & vehicle types, density is/was
still very important (tables 1 & 2), but a shift from a density of Ik to 5k persons per sq mile (which is 1.5 vs. 7.8
persons per acre) means an increase of 750 mi/yr/vehicle (which is about 7.5% of annual VMT). Such
differences, and their practical significance (or lack thereof) should be discussed  in the reports.

[Van Auken] The inducted-exposure data set provided by NHTSA is based on the non-culpable vehicles in two-
vehicle crashes. See the comments  in Table 1 on the limitations of this data.  In addition, there are also many
differences in the coding variables and values used by the different states, which tend to make the receding to
a common data set imprecise. [Note: This peer review does not address the Wenzel (2011b) companion
report, which examines the risks per police-reported crash.  See Ref 4 for comments on the companion report.]
Please comment on any apparent, unstated,
or implicit impact on estimated risks inherent
in the two different approaches, and any
related caveats or limitations. For example,
what are the strengths and weaknesses of the
two measures of vehicle exposure, miles of
vehicle traveled scaled up from crash data
from 13 states, and number of police-reported
crashes?
[Chen and Kockelman]  The use of non-culpable vehicles in two-vehicle crashes as a proxy for vehicles which
are "just there" may be distorting the overall distribution of vehicle models. VMT may differ between vehicles
that are more prone to  run-off-road accidents, at-fault two-car crash vehicles, and non-culpable vehicles.

[Van Auken] [1] The number of fatal cases tends to be much less than the number of induced-exposure cases.
Therefore the effective  numbers of degrees-of-freedom in the statistical estimates tend to be limited by the
available number of fatal cases. For example, it would not be possible to estimate the effects of two variables
(e.g., just curb weight and footprint) if we had data for only one fatal case even if we had thousands of
induced-exposure cases. Therefore it is desirable to use data for the entire US in order to get a large sample of
fatal cases for the logistic regressions.  This then requires the available induced-exposure data (i.e., from 13
                                                                                                                                      17

-------

states) to be "scaled up" the US level using the method described in Kahane (2003 and 2011). The result is the
best currently available estimate of vehicle exposure.
[2] There may be some concerns about the accuracy of the vehicle miles-traveled data because the difficulty
estimating the number of vehicle miles travelled at the make-model-year=state level of detail. [Note: This
peer review does not address the Wenzel (2011b) companion report, which examines the risks per police-
reported crash. See Ref 4 for comments on the companion report.]
ADDITIONAL COMMENTS:
18

-------
    5.  RECOMMENDATIONS
                                            COMMENTS
Please comment on whether the LBNL
assessment adequately addresses the NHTSA
2011 study and identifies the safety impact
from mass reduction. Are the analytic
methods and data used to assess the NHTSA
study, and estimate the relationship between
risk, mass, and footprint, appropriate?  Is
casualty risk per crash a legitimate measure of
vehicle safety?  What other methods or data
could be  used to better predict the effect of
future vehicle designs on safety?
[Chen and Kockelman]  While driver fatalities per crash seems a useful measure of vehicle design safety, and
examination of fatal crash rates is very valuable (using Polk- based exposure estimates), there are many
caveats to work of this type. As noted above: a primary concern remains a neglect of the notion that the type
of car is very much a proxy for driver type, and a vehicle's crash avoidance may have very little to do with
vehicle type. It has a lot to do with the person behind the wheel. Simply including gender and age variables
cannot account for important covariates such as education, risk aversion, driving ability, wealth, etc. In reality,
small cars may be less crash prone than Kahane's and Wenzel's results suggest, because they are driven by
lower-income, younger, less risk averse people driving in more crash prone settings (e.g., commercial strips
rather than pricey residential suburbs). Of course, as noted above, it is very difficult to control for all these
variables, and the contractor was  asked to rely on the original data. In reality, the best the report authors can
do with such data sets is to explain how all the other, relevant attributes may factor in (e.g., quality of driver
and typical driving settings), and how they can generate biased estimation (sometimes in either direction).
Discussion of relevant literature that looks more deeply at crash outcomes (e.g., Wang or Chen's papers,
mentioned above, allowing for heteroscedasticity and individual vehicle attributes, non-driver outcomes, etc.)
will also be useful.

[Van Auken] [1] The basic methodology described by Kahane (2011) seems appropriate; however some
results using this method and data are not well understood and need further diagnosis.

[2] The induced-exposure data set provided by NHTSA is based on the non-culpable vehicles in two-vehicle
crashes.  See the Table 1 comments on the limitations of this data.  [Note: This peer review does not address
the Wenzel (2011b) companion report, which examines the risks per police-reported crash.]
Please comment on the overall adequacy of
LBNL's assessment of the 2011 NHTSA report
and its independent study of casualty risk for
predicting the effect of vehicle mass or
footprint reduction on safety.  Provide any
recommended improvements  that might
reasonably be adopted by the  author to
improve the analysis.
[Chen and Kockelman]  Overall, the study is a comprehensive assessment of the 2011 NHTSA report and
introduces interesting additional analyses to examine the relationship of vehicle mass and footprint reduction
on safety. However, as stated previously in the comments here, driver preference for specific car types
(including size and mass) is related to driver socioeconomic characteristics and driving behavior. As vehicle,
driver, and roadway environment characteristics all contribute to fatality risk, the effects of physical vehicle
changes  such as mass or footprint reduction on safety should not  be overstated  when the other two types of
characteristics are not sufficiently accounted for.

[Farmer] Overall these are reasonably good studies. The Phase I report does a very good job of assessing the
NHTSA report of fatality risk.
                                                                                                                                      19

-------
                                          [Van Auken] The Wenzel (2011a) report provides a valuable supplement to the analysis and results in the
                                          Kahane (2011) report. [Note: This peer review does not address the Wenzel (2011b) companion report, which
                                          examines the risks per police-reported crash.]
ADDITIONAL COMMENTS:
[Van Auken] See attached tables 6 and 7 below.
                                                                                                                                  20

-------
                                  Table 6. Additional General Comments and Recommendations
                                                          Mike Van Auken
 Section
                                          COMMENTS AND RECOMMENDATIONS
           Use of R2 is confusing.  Suggest using lower case "r" when referring to the sample correlation coefficient (Box, Hunter, Hunter, 1978,
           P. 61); or upper case R when referring to the regression coefficient of multiple determination (Draper and Smith, 1981, p. 90).
All
In most cases the reported results are just estimates, but are not described as such. For example, "The effect of mass reduction on
heavier cars and CUVs and minivans are not statistically significant" on p. iii should say "The estimated effect of mass reduction on
heavier cars and CUVs and minivans are not statistically significant."

This distinction  is important when comparing results based on different models and assumptions because the different models and
assumptions do not change the effect itself, but rather the estimate of the effect. For example, the statement "The first sensitivity,
in dark purple, includes the weight variables in the regression model but excludes the footprint variable; this model tests the effect
of mass reduction while allowing footprint to vary with vehicle mass. This sensitivity increases the risk from a 100-lb mass reduction
in cars (from 1.43% to 2.64% for lighter cars, and from 0.48% to 1.94% for heavier cars) and CUVs/minivans (from a 0.47% decrease
in risk to a 0.52% increase in  risk); however, there is no change in fatality risk in light-duty trucks" on page 15 is misleading.  It would
be better to state that "The first sensitivity, in dark purple, includes the weight variables in the regression model but excludes the
footprint variable; this model tests the estimated effect of mass reduction while allowing footprint to vary with vehicle mass. Th4&
GGnsitivity/?emoi//'ng the footprint variable from the regression model increases the estimated risk from a 100-lb mass reduction  in
cars (from 1.43% to 2.64% for lighter cars, and from 0.48% to 1.94% for heavier cars) and  CUVs/minivans (from a 0.47% decrease  in
risk to a 0.52% increase in risk); however, there is nothe change in  the estimated fatality risk in light-duty trucks is very small and
not statistically significant."

This also applies to table and figure captions. For example, "Table  ES.l. Effect of mass and footprint reduction on fatality risk, under
alternative regression model specifications" should say "Table ES.l. Estimated effects of mass and footprint reduction  on fatality
risk, under alternative regression model specifications."  "Figure 3.3 Effect of reduction in mass or footprint on US fatality risk per
VMT, by vehicle type: mass only, footprint only, and both" should say "Figure 3.3 Estimated effects of reduction in mass or footprint
on US fatality risk per VMT, by vehicle type: mass only, footprint only, and both."

Overall the word "effect" appears over 200 times in this report with the "estimated" or other qualifier. In some cases this may be
appropriate and in other cases it is not appropriate.  It is recommended that the author review each instance and revise as
appropriate.
                                                                                                                                 21

-------
Figures 4.1 through 4.17 do not control for the effect of vehicle size (e.g. footprint), which has been shown to be correlated with
vehicle weight (e.g., Figure 3.1), and therefore these figures may be misleading.  It is strongly suggested that the horizontal axis label
be changed to "Curb weight (Ibs) and corresponding changes in size," and/or a note such as the following be added to each figure:
"Note these results do not control for the effect of vehicle size on fatality risk.  Therefore the horizontal axis represents changes to
both vehicle weight and vehicle size."
The statistical significance of the linear trends in Figures 4.1 through 4.17 are not reported. It would be helpful if the confidence
intervals or statistical significance of the linear trends were reported, either in addition to or instead of R2.
The confidence intervals for the estimated slopes should be added to the results in Tables 4.1 and 4.2.
                                                                                                                         22

-------
Table 7. Additional Specific Comments and Recommendations
Mike Van Auken
Section
Executive
Summary, 7
Executive
Summary
Executive
Summary, 4
Executive
Summary, 7
4
4
5.1
5.2
5.2
5.2
5.3
6.4
Page
iii, 65
iv
iv, v, 22, 66
viii, 69
31-32
35
37
39
39
39
44
63
COMMENTS AND RECOMMENDATIONS
2nd paragraph refers to "our analysis," however the results are the same as the NHTSA analysis. The author
should clarify who or what "our analysis" refers to and how it relates to the NHTSA analysis. Perhaps the
statement "LBNL was able to reproduce the NHTSA analysis, which finds that..." would be more appropriate.
Last bullet - suggest changing the statement that "Logistic regression does not allow a statistic" to "Logistic
regression methods do not have a statistic."
Suggest changing "variance in risk" to variation in risk" throughout.
The numerical results for the NHTSA preferred model in Tables ES.l and 7.1 are slightly different than the results
reported in the NHTSA report. For example 1.43%/0.48%/0.52%/-0.40%/-0.47% should be 1.44%/0.47%/0.52%/-
0.39%/-0.46%
Figures 4.12 through 4.14 have the results for small and heavy-duty pickups combined, which is inconsistent with
the results in Table 4.1
The R2 values in Table 4.1 are different than the values in Figures 4.6, 4.8, 4.9, 4.11.
The subsection title should be "Alternative measures of exposure and outcome" because fatal crashes and
fatalities are measures of the crash outcome, not exposure.
It would be helpful to list the 18 manufacturer dummy variables in a table.
It is unclear why Lexus, Acura, and Infinity are treated as separate manufacturers, but Cadillac and Lincoln are
not.
It is unclear why AM General is considered a Chrysler brand. The AM General Hummer was sold by GM
beginning with the 2001 model year.
It would be helpful if the figures include error bars or shading to indicate the confidence intervals.
Table 6.3 - Suggest adding a note that the 72,316 total includes fatalities that are counted more than once in
crashes involving more than one vehicle type.
23

-------
24

-------
                       An Analysis of the Relationship between Casualty Risk Per Crash and Vehicle Mass and Footprint
                                                for Model Year 2000-2007 Light-Duty Vehicles
                                                            (LBNL Phase 2 Report)
    1.  ASSUMPTIONS
                                            COMMENTS
Please comment on the validity of any
assumptions embedded in the LBNL
assessment analysis and the independent
casualty analysis that could affect the
projected relationship between vehicle
mass/footprint reductions and
fatality/casualty risk.  Examples might include
assumptions regarding whether recent
historical relationships between vehicle
weight, size, and safety will continue into the
future; potential future improvements in
vehicle technology and design may result in
compensatory safety benefits; and the annual
baseline fatality distribution.
[Chen and Kockelman]  [1] The Phase 2 report serves as a complimentary document to the Phase 1 report by
isolating the effect of vehicle mass and footprint on crashworthiness. Whereas the Phase 1 report analyzes
fatality risk per estimated VMT, the Phase 2 report analyzes casualty risk per crash. The parallel structure of
the two reports makes it easy for the reader to compare the results of the two analyses.

[2] The binary logistic model employed in this analysis can only account for two injury outcome categories;
here it is used to distinguish crashes resulting in serious injury or death from all other crash outcomes. Thus,
the model does not account for the ordinal nature of injury severity and neglects the difference between a
serious injury and  a death.

[3] The report states that "a serious incapacitating injury can be just as traumatic to the victim and her family,
and costly from an economic perspective, as a fatality." While serious injuries are very costly to society (and
may have similar economic cost implications as deadly crashes), willingness to-pay estimates (which include
pain and suffering) price the cost of a fatality at almost 20 times the cost of an incapacitating injury (NSC
2010). Thus, it is difficult to assess the economic cost of the estimates of increases in casualty risk per crash
without distinguishing whether that outcome is a serious injury or a death. This limitation of the model should
be addressed in the report.

[4] The logistic model also assumes error-term homoscedasticity and cannot account for increases and
decreases in the variation of injury outcomes due to vehicle and driver type, for example. Such limitations of
the model should  be discussed.

[Farmer]  The report concludes "that much of the detrimental effect of mass or footprint reduction on risk can
be attributed to the tendency for mass or footprint reduction to increase crash frequency, rather than to
reduce vehicle crashworthiness (risk once a  crash has occurred)."  However, the interpretation of casualties
per crash as inversely proportional to crashworthiness  ignores the possibility that injury severity also depends
upon the circumstances of the crash. Casualties per crash must be divided into casualties per severe crash and
severe crashes per crash, where a severe crash would be one involving  more energy, e.g., high-speed or
                                                                                                                                      25

-------
rollover.  It could be that weight reduction increases casualties per severe crash (i.e., reduces
crashworthiness), but reduces the likelihood that a crash is severe.

[Van Auken ] The basic assumptions, methodology,  and data are primarily the same as in the Kahane (2011)
report, but have been extended to include serious injuries as well as fatalities, and also address crash
involvement (i.e., fatalities and serious injuries per accident, and also accidents per VMT).. These include the
following:

1) The probability of a crash fatality or serious injury is proportional to the number of accidents (provided the
   crash  conditions remain the same); and the probability of an accident is proportional to the vehicle miles
   travelled (VMT).
2) The logarithm of probabilities of fatality or serious injury per accident, and accidents per VMT for a given
   curb weight, footprint, and control variable values varies as a linear combination of the curb weight,
   footprint, and control variables within the domain of the data.
3) The logistic regression methods determine a maximum likelihood estimate of model coefficients.
4) It is assumed that the above relationships remain constant in the recent past (i.e., 2000-2007  model year
   vehicles in the 2002-2008 calendar years), present, and  near future (i.e., 2017-2025 model year vehicles).

The first assumption that crash fatalities and serious injuries are proportional to the number of accidents
provided the crash conditions remain the same seems self-evident (e.g., if two fatal crashes had exactly the
same conditions, then the expected number of fatalities for the two crashes would be twice the value for just
one of the crashes). The assumption that the number of accidents are proportional to VMT rather than the
number of vehicle registration years (VRY) is also appropriate because accidents cannot occur if the vehicles
are not driven on the road (i.e., VMT = 0).  This assumption is qualified however because VMT is more difficult
to measure than VRY and therefore may be less accurate. On the other hand the probability of a fatal crash or
the number of fatalities in a crash may also depend on the  vehicle occupancy.

The second and third assumptions are appropriate provided that it is recognized that it is essentially
impossible with currently available  knowledge and information to model all of the factors that could affect the
probability of fatality in a crash, and that the objective of the analysis is to identify overall trends versus
vehicle weight and footprint.  In general the probability of fatality depends on other many other  factors which
have not been modeled (e.g., driver behavior factors, vehicle design factors, roadway design factors, EMT
factors),  and these unmodeled factors are assumed to be uncorrelated with vehicle weight and footprint,
and/or are represented by the other control variables.  The latter assumption might or might not be valid.
                                                                                            26

-------

Please comment on any apparent unstated or
implicit assumptions and related caveats or
limitations.
The fourth assumption is perhaps the weakest because it assumes that future vehicles will have the same
design characteristics as past vehicles, and that the characteristics of the vehicle population (e.g., collision
partner weight, size, type) will also remain the same.
[Chen and Kockelman] The role of driver behavior is briefly addressed in the report but not emphasized
sufficiently. Casualty risk is a combination of driver, vehicle, and roadway characteristics. Whereas vehicle
characteristics significantly influence crashworthiness, driver behavioral differences play a significant, if not
primary, role in determining crash frequency. Socioeconomic data such driver household income, size, and
education influence driver attitudes and driving environments. For example, Chen et al. (2010) found that
crash risk increases for those living in socioeconomically disadvantaged areas (including households more likely
to drive less expensive and older vehicles). Though such data is not typically available in state and national
crash databases, the importance of these driver and environmental characteristics on crash rates (per mile
driven) and casualty risk should be stressed in both reports. It is clearly very difficult to control for, but is a
major caveat to the NHTSA (& now LBNL) results. We expect that crash severity could be probably be lower for
many of the small cars and pickups if they were driven by those who tend to drive more expensive vehicles,
under the same settings (e.g., daytime, urban freeway). Thus, statements like "a 100-lb reduction in the mass
of lighter cars leads to a 1.84% increase in crash frequency" should be accompanied by an explanation of the
possibility of the mass variable accounting/proxying for effects of lower income households owning smaller
vehicles.
[Van Auken] [1] The induced-exposure data set provided by NHTSA is based on the "non-culpable" vehicle in
two-vehicle crashes. It is assumed that the dataset is a reprehensive sample of the driver and environmental
exposure factors for vehicle use. However, since these cases include moving vehicles, some vehicle-driver-
environmental conditions may be under or over represented in this data depending on how they affect the
ability of a non-culpable vehicle to avoid a crash. Results in Ref (17) indicated that the estimated effect of
weight and size reduction are sensitive to whether the induced-exposure data are based on the Kahane (2003)
non-culpable vehicle definition of the Kahane (1997) stopped vehicle definition. Unfortunately it is not
currently possible to test this sensitivity with the NHTSA-provided induced-exposure data.
[2] It is also assumed that the accident data from the 13 or 16 states are representative of all US states. Figure
2.1 in Wenzel (2011b) provides a useful comparison of the distribution of fatalities in the US and 13 states by
the nine different crash types.
ADDITIONAL COMMENTS:
27

-------
[Chen and Kockelman]  National Safety Council (2010) Estimating the Costs of Unintentional Injuries. Available online at
http://www.nsc.org/news_resources/injury_and_death_statistics/Pages/
EstimatingtheCostsof Unintentional Injuries

Chen, H.Y., Ivers, R.Q., Mariniuk, A.L.C., Boufous, S., Senserrick, T., Woodward, M., Stevenson, M. and Norton R. Socioeconomic status and risk of car crash
injury, independent of place of residence and driving exposure: Results from the DRIVE study. Journal of Epidemiology and Community Health 64(10), 2010, pp.
998-1003.

[Farmer] NHTSA's fatality analysis covered calendar years 2001-08, but the casualty analysis excludes 2008. Such exclusion is understandable given that 2008
data were at the time unavailable for a majority of the states (I think they are available now). However, 2008 was an unusual year and may have affected the
size and weight effect estimates. The footnote on p. 6 of the Phase II report states that an analysis including the available 2008 data will be summarized in
Appendix A.  I don't see Appendix A. Is an analysis planned including 2008 data?
                                                                                                                                       28

-------
    2.  CONTROL AND DEPENDENT
       VARIABLES USED IN THE REGRESSION
       MODELS
                                             COMMENTS
Please comment on the adequacy of control
and dependent variables used in the
assessment analysis and independent casualty
analysis, and recommend any alternative
control or dependent variables that are
available for possible inclusion in the analysis.
For example, what are the relative merits of
the main dependent variables used, fatality
risk per estimated VMT, and casualty risk per
police-reported crash?
[Chen and Kockelman] [1] As alluded to above, a primary concern is that the NHTSA analysis (& thus the
LBNL analyses) largely neglect the idea that vehicle type (make & model) is very much a proxy for driver type,
and a vehicle's crash frequency may have very little to do with physical vehicle characteristics. It has a lot to do
with the person behind the wheel, and gender and age simply aren't enough to control for such distinctions.
Education, risk aversion, ability, wealth, etc., are important covariates. But existing data sets are quite limiting
(though the MVOSS & FAR with 3-year driver violation history do offer some valuable insights, not discussed in
these reports). In reality, small cars may be less  crash prone than Kahane's & Wenzel's results suggest, because
they are driven by lower-income, younger, less risk averse people driving in more crash prone settings (e.g.,
commercial strips rather than pricey residential  suburbs). Such key caveats need thoughtful discussion. Four
relevant papers on  the topics of crash frequency and vehicle size-and-weight implications (by Knipling, Kweon
& Kockelman, Wang and Kockelman, and  Chen & Kockelman) have been sent to Tom Wenzel. These all include
useful literature reviews for further connections to useful findings for  citation in the reports, as time allows the
contractor.

[2] Independent variables such as vehicle mass and footprint  may be accounting for effects of driver
socioeconomic factors as discussed in the Assumptions section. Furthermore, vehicle option variables such as
AWD and side curtain airbags may be  reflecting  the effects of driver environment (e.g., those living in areas
with icy winters opting for AWD) and attitude (e.g., more risk-averse drivers opting for side curtain airbags)
rather than the vehicle technology themselves. While extremely heavy and extremely large vehicles may have
significantly different handling and braking characteristics which influence crash frequency and casualty risk, it
is unlikely that given the same driver in the same environment, a small change in vehicle mass or footprint
would influence the driver's crash proneness.

[Farmer] One  needs to restrict control variables to those that are available and reliable.  A problem when
combining state databases is that the  states often are not consistent as to the variables coded and the
definitions of those variables. This severely limits the list of possible control variables.

[Van Auken] [1] Reducing the total number of fatalities and serious injuries is desirable from a societal
viewpoint. This includes both subject vehicle occupant and collision partner (e.g.,  other vehicle  occupant,
pedestrian) fatalities and serious injuries.
                                                                                                                                       29

-------
                                            [2] VMT is a good measure of accident exposure provided that it can be accurately determined.

                                            [3] The number of fatalities and serious injuries per accident is a measure of vehicle crashworthiness (i.e.,
                                            effect of a crash on the subject vehicle occupants) and crash compatibility (i.e., effect of a crash on the other
                                            vehicle occupants or vulnerable road users).  Subject vehicle occupant fatalities and serious injuries per
                                            accident are a measure of the subject vehicle crashworthiness.  Collision partner fatalities and serious injuries
                                            per accident are a measure of vehicle crash compatibility.

                                            [4] The number of accidents per VMT is a measure of the crash avoidance capabilities of a given vehicle.
What additional control variables, such as
vehicle make or model, might be included in
the regression models?
[Chen and Kockelman] [1] Vehicle height, a variable which may be more valuable than vehicle type for
similarly structured vehicles such as sedans, wagons, CUVs, and minivans, would be a valuable control variable.
In addition to a wider track, a lower center of gravity also increases vehicle stability, thereby reducing the risk
of rollover. Relevant literature & findings exist, and should be cited.

[2] Other variables which have been found  in past studies to influence fatality risk such as seat belt use,
roadway geometry and division type are not included in this study.

[3] To account for driver characteristics that contribute to casualty risk, socioeconomic variables such as
household income, education, household size, etc. would be valuable additions. Unfortunately,  both state and
national crash databases typically do not include such information (outside of MVOSS). Such issues should be
flagged for readers.

[Farmer] I think that already there are too many control variables in the regression models. Instead  I would
consider defining different classifications of crash types. Table 2.2 of the Phase II report shows that the
distribution of crash types for casualty crashes is very different from that for fatal crashes.
Please comment on any caveats or limitations
that these dependent variable or control
variables entail with respect to use of the
results as the basis for estimating the safety
effect of mass reduction.
[Chen and Kockelman] Please see above comment (in Assumptions section) regarding driver behavior and
environment.

[Farmer] Model overspecification could be the reason for results that are non-intuitive, especially in the Phase
II analyses of police-reported crashes.  Control variables may be correlated with each other or with the size
and weight variables.  For example, Figure 2.9 of the Phase I report implies that torso side airbags increase
fatality risk in CUVs.
                                                                                                                                         30

-------
ADDITIONAL COMMENTS:

[Chen and Kockelman] Table 2.1 has many indicator variables labeled as "C" for continuous variable (such as ABS, ESC, AWD, DRVMALE, etc). These C's should
be removed.

[Farmer] The sensitivity results of Chapter 5 (Phase  II) point out the extreme differences in results when changing the control variables.  For example, including
vehicle make changes the effect of a 100-lb reduction in heavier cars from -0.91% to +0.55% (see Fig 5.3).

[Van Auken] The underlying reasons for some of the estimated effects are unknown at this time, but presumably involve driver, vehicle, environment or
accident factors than have not been controlled for in the Kahane  (2011) and Wenzel (2011b) analyses. See, for example, Refs 17 and 23.
                                                                                                                                    31

-------
    3.  METHODOLOGY AND STATISTICS
                                             COMMENTS
Please comment on the validity and
applicability of the methodology LBNL used in
assessing the NHTSA 2011 study and its
analysis of the relationship between mass,
footprint, and risks per police-reported crash.
[Chen and Kockelman] The Phase 2 report enhances the findings in the Phase 1 report by isolating the effect
of vehicle mass and footprint on crashworthiness. Like the Phase 1 report, this analysis goes into a fair amount
of detail and seeks to introduce additional analyses to  better examine the relationship between mass,
footprint, and casualty risks. However, due to a lack of control for very specific vehicle differences (which vary
by make & sub-model), the exclusion of driver characteristics and crash setting details (which cannot always be
controlled for, but are often correlated with vehicle type), the effects of downweighting vehicles and/or
shifting vehicle styles and sizes may be overestimated. Simply changing the vehicle mass or footprint on a risky
driver in a high-risk setting is unlikely to influence crash outcomes significantly.

[Farmer] Figure 2.11 of the Phase II report implies that NHTSA's fitting of a separate regression model for each
of the 9 crash types was unnecessary, at least for the analysis of casualty  risk per crash.  I don't recall seeing a
similar analysis for fatality risk per VMT. Is it possible to get essentially the same results as the NHTSA study
using a single regression model?

[Van Auken]  [1] The logistic regression methods seem to be appropriate. The confidence intervals are based
on the logistic regression Wald Chi-Square statistic, which as Kahane (2003, 2011) has demonstrated does not
include all sources of variation. However, these confidence  intervals are useful because they do provide some
indication of the uncertainty in the results.

[2] The two-stage  results for the 13-state fatalities per crash and 13-state crashes per VMT, and the one-stage
result for 13-state fatalities per VMT reported in Tables ES.l and 6.1 and Figure 2.7 were computed using
independent logistic regressions. The differences between the two-stage results and the one-stage results for
fatalities  per crash could have been eliminated by using the  "simultaneous three-way" logistic regression
method described in  DRI (2003). This  method imposes the constraint that the combined two-stage estimated
and the one-stage estimated are equal.
Please review other statistical methods LBNL
has used in the analysis, in addition to the
logistic regression methodology. Examples
include the alternative approaches used by
LBNL to assess NHTSA interval estimation
results, and LBNL's linear regression analysis
of actual, predicted, and residual risk by
[Chen and Kockelman] In the alternative measures of exposure, the author examines the effect of vehicle
manufacturer on fatality risk and treats the luxury models produced by Toyota, Honda, and Nissan as separate
manufacturers. However, domestic luxury brands (such as Cadillac & Lincoln) are categorized with their
nameplate manufacturers (GM and Ford), which appears inconsistent.

[Van Auken] [1] The correlations in Section 3 appear to be assessed using the Coefficient of Multiple
Determination (R2) based on a linear fit to the data (e.g., the correlation between footprint versus curb weight
                                                                                                                                       32

-------
vehicle model.
in Figure 3.1 on p. 32). The linear regression model attributes the differences between the dependent variable
(vertical axis) and the linear fit to the independent variable (horizontal axis) to random effects.  If there is no
preference as to the choice of independent and dependent variables (e.g., footprint versus curb weight, or
curb weight versus footprint), then the linear trend and R2 result would be different if the two variables were
interchanged, and having two different yet equally valid results would be undesirable.

[2] If the variation in the data can be attributed to both variables (e.g., footprint and curb weight), then it
would be better to report the square of the sample correlation coefficient r2, where r is computed according to
Eqn (1). The trend lines in these correlation figures should not be computed  using a linear regression. Instead,
the trend line should pass through the sample means (i.e., (x, y)), and have a  slope equal to the ratio of the
sample standard deviations in the data (i.e., sy/sx). Therefore, the reported correlation results do not depend
on the ordering of the data variables.

Note this comments does not apply to linear trends indicated in Section 4, for which the Coefficient of Multiple
Determination (R2) seems appropriate.

[3] The Coefficient of Multiple Determination (R2) is frequently used in the Wenzel (2011b) report as an
indicator of the statistical importance of a linear trend  (e.g., lvalues in Tables 4.1 and 4.2 were compared to
0.3). It would be better to report the standard error, confidence interval, and/or probability value as measures
of the statistical significance of a  linear trend.
Please comment on caveats or limitations of
using non-significant regression estimates to
project the safety impact of mass reduction.
[Chen and Kockelman] First, the t-statistics are not provided in the report which makes it difficult for the
reader to assess statistical significance of specific regression estimates (except where noted by the author).
Second, inclusion of a statistically insignificant variable can influence the estimates of coefficients associated
with related variables. Nevertheless, in general, it is best to keep insignificant estimates if one has a strong
defense for their role, since removing such variables (& thus their parameters) will shift the burden of
response to a correlated covariate's parameter, thus biasing the latter. We generally keep key covariates in a
model  up to a pvalue of 0.20 or 0.25 or so, especially in relatively small data sets (e.g., n < 1,000). Covariates
for which we have no strong basis can  be removed for pvalues > 0.10.

[Farmer] Making projections from non-significant regression estimates is proper so long as the resulting
confidence intervals are constructed conservatively (to account for the accumulated imprecision). In that
sense,  I prefer NHTSA's jackknife approach to the standard errors produced by SAS (see p. 13 of Phase II).
                                                                                                                                         33

-------
                                            [Van Auken] [1] Regression estimates are random numbers which have an unknown expected value and
                                            variance, and known sample value and standard error.  If the sample value can be explained by a zero
                                            expected value and known standard error then the result is considered not statistically significantly different
                                            than zero and therefore the result is not considered to be statistically significant.  However, If we can combine
                                            this estimate with other estimates then the unknown expected values and variances can also be combined
                                            using the same transformation, and the statistical significance of the combined result can be tested.
                                            Therefore, depending on the sample values and inter-correlation, the combined result may be statistically
                                            significant even if the individual estimates are not statistically significant.

                                            For example, the results from each of the nine different crash types can be combined into an overall estimate
                                            and the standard error calculated assuming that the results for each crash type are independent of each other.
                                            Then the statistical significance of the combined effect can be determined.

                                            [2] However, and Kahane (2011)  points out there are two sources of uncertainty in the regression results. The
                                            first is the PARS based sampling error which is uncorrelated across crash types because they are based on
                                            different fatal cases (Kahane 2011, p. 77). The second is the state based induced-exposure sampling error
                                            which is correlated across crash types because they are based on the same induced-exposure cases. Therefore
                                            a confidence interval estimated using the "jackknife" method described by Kahane (2011) and accounting for
                                            correlation of these two error sources would  be more accurate than a simple estimate based on the Wald Chi-
                                            Square statistic and assumed independence.
How might the LBNL methodology be
strengthened to better represent future
vehicle designs and reduce multi-collinearity
between mass and footprint in the regression
analysis?
[Chen and Kockelman] Including more vehicle-specific characteristics (such as vehicle height and engine size)
reduces the analysis' dependence on vehicle type, since vehicle shapes and structures will continue to evolve.
There is also correlation with context (e.g., pickups are driven in more rural locations, with greater hazards
[like less lighting, higher speed, & few medians]). Disaggregate data are almost always best, to avoid ecological
fallacies & such.

[Van Auken]  [1] The effects of multi-collinearity can be mitigated by 1) obtaining more data, 2) pooling data
from different crash type or vehicle types, or 3) reducing the number of regression variables. The first option
would require more states (for serious injuries and police-reported accidents), calendar years and/or model
years, which would involve added newer data as it becomes available (or using older data). The second option
might be to recombine the CUVs and minivans with truck based vans and adding a control variable to
compensate for the differences in the vehicles types.  The third option might involve removing statistically
insignificant  control variables or removing control variables that would not be expected to have an effect on
                                                                                                                                       34

-------

the probability of crash or crash outcome (e.g., the side airbag variable is not included in pedestrian crashes
because it is not expected to affect pedestrian fatality risk). The number of driver age control variables might
be reduced from eight to three (as in the Kahane (1997) and DRI (2002-2005) studies). Finally, a linear curb
weight model instead of a two-piece linear model may help to better elucidate the general trend.
[2] The Variance Inflation Factor (VIF) has been suggested as a measure of multi-collinearity in the Kahane
(2010 and 2011) reports, however this diagnostic metric does not account for differences in database size (i.e.,
Options 1 and 2 above). The Wenzel (2011b) report does not discuss the Variance Inflation Factor or report
any VIF results.
ADDITIONAL COMMENTS:
35

-------
    4.  DATA SETS
                                             COMMENTS
Please comment on the validity and
applicability of the datasets used to project
changes in risk resulting from reduction in
vehicle mass. LBNL's casualty analysis used
police-reported crash data from 16 states,
while the 2011 NHTSA study used national
fatality data, combined with a subset of non-
culpable vehicles involved  in two-vehicle
crashes from police-reported crash data from
13 states.
[Chen and Kockelman] [1] The Phase 2 report uses an unusually extensive data set of police-reported crash
data from 13 states which the author compares in detail to national data sets to illustrate similarities and
differences. The author is very thorough in addressing the difference in definitions of "serious" and
"incapacitating" injuries across different states and the effects of such inconsistency on the regression results.

[2] Since casualty risk in the report accounts for serious injuries but not minor injuries, the author should note
that police-reported injury levels may also be poor indicators of the actual or Modified Abbreviated Injury
Scale (MAIS) level, following medical evaluation. Farmer (2003) found that 41% of injuries reported by U.S.
police as incapacitating received MAIS ratings of "minor injury" by health care professionals using NASS
Crashworthiness Data System (CDS). Thus, the results of the estimated casualty risk increases and decreases
rely heavily on the assumption that police errors in reporting actual MAIS ratings are consistent across states.

[Farmer] A major limitation of the Phase II analysis is a bias that may be due to the patterns of missing data.
In particular, the vehicle identification  number (VIN) is missing or mistyped for many crash records.  High-
severity crashes (especially fatal) are more likely to have detailed police investigation, so VINs (and other
variables) in these crashes may be more complete. State crash files are therefore much less reliable than
PARS.

[Van Auken]  [1] The induced-exposure data set provided by NHTSA is based on the non-culpable vehicles in
two-vehicle crashes.  See the comments in Table 1 on the limitations of this data.

[2] The use of property damage accident data and cases with serious injury from the 13 states seems
appropriate (with the noted qualification that the different states may have different accident reporting
thresholds and  injury reporting criteria). The concerns about the use of data for the 3 additional states
(Georgia, Illinois, and New Mexico) have also been noted.

[3] In addition, there are also many differences in the coding variables and values used by the different states,
which tend to make the receding to a common data set (either induced-exposure, police-reported accident, or
severe injury) imprecise.
                                                                                                                                         36

-------
Please comment on any apparent, unstated,
or implicit impact on estimated risks inherent
in the two different approaches, and any
related caveats or limitations.  For example,
what are the strengths and weaknesses of the
two measures of vehicle exposure, miles of
vehicle traveled scaled up from crash data
from 13 states, and number of police-reported
crashes?
[Chen and Kockelman]  The Phase 1 analysis used non-culpable vehicles in two-vehicle crashes as a proxy for
induced exposure crashes. In contrast, Phase 2 analysis uses data from vehicles involved in one-car crashes and
the responsible vehicle in two-car crashes. The exclusion of the not-at-fault vehicle in two-car crashes may be
distorting the distribution of crash frequency and casualty risk across different vehicle makes and models if
crash-prone drivers are  more likely to drive certain types of vehicles.

[Farmer] The VMT weights provided by NHTSA were scaled to represent the entire US. Comments on pp. 9
and 18 of the Phase II report seem to acknowledge this deficiency, promising to adjust these to the 13 states in
the future.  Was any adjustment made, such as multiplying the weights by the proportion of annual US VMT
accounted for by each of these states?  The accuracy of the VMT weights is critical is we are to believe the
somewhat surprising results concerning crashes per VMT.

[Van Auken]  [1] The number of fatal or serious injury cases tends to be much  less than the number of
induced-exposure cases (and the number of police-reported accidents). Therefore the effective numbers of
degrees-of-freedom in the statistical estimates tend to be  limited by the available number of fatal or serious
injury cases. For example, it would not be possible to estimate the effects of two variables (e.g., just curb
weight and footprint) if we had data for only one fatal of serious injury case even if we had thousands of
induced-exposure cases. Therefore it is desirable to use data for the entire US in order to get a large sample of
fatal cases for the logistic regressions. This then requires the available induced-exposure data (i.e., from 13
states) to be "scaled up" the US level using the method described in Kahane (2003 and 2011). The result is the
best currently available  estimate of vehicle exposure.

[2] There may be some concerns about the accuracy of the vehicle miles-travelled data because the difficulty
estimating the number of vehicle miles travelled at the make-model-year-state level of detail.
ADDITIONAL COMMENTS:

[Chen and Kockelman]  Farmer, C.M. Reliability of police-reported information for determining crash and injury severity. Traffic Injury Prevention 4(1), 2003,
pp.38-44.

[Farmer] Statements above Figure 2.7 in the Phase II report imply that the effects of weight reduction on crashes per VMT and fatalities per crash should add
up to the effect on fatalities per VMT. This is not the case.  For example, a 1.43% increase in crashes per VMT and a 0.76% decrease in fatalities per crash
would imply a 2.16% decrease in fatalities per VMT (i.e., 1 - 0.9924/1.0143). The fact that the model on fatalities per VMT yields an estimated 1.08% increase
should be a cause for concern.  Either the VMT weights are  inaccurate or the control variables have different effects on crash frequency and crashworthiness.
                                                                                                                                      37

-------
38

-------
    5.  RECOMMENDATIONS
                                            COMMENTS
Please comment on whether the LBNL
assessment adequately addresses the NHTSA
2011 study and identifies the safety impact
from mass reduction. Are the analytic
methods and data used to assess the NHTSA
study, and estimate the relationship between
risk, mass, and footprint, appropriate?  Is
casualty risk per crash a legitimate measure of
vehicle safety?  What other methods or data
could be  used to better predict the effect of
future vehicle designs on safety?
[Chen and Kockelman]  As noted above, a primary concern remains a neglect of the notion that the type of car
is very much a proxy for driver type, and a vehicle's crash avoidance may have very little to do with vehicle
type. It has a lot to do with the person behind the wheel. Simply including gender and age variables cannot
account for important covariates such as education, risk aversion, driving ability, wealth, etc. In reality, small
cars may be less crash prone than Kahane's and Wenzel's results suggest, because they are driven by
lower-income, younger, less risk averse people driving in more crash prone settings (e.g., commercial strips
rather than pricey residential suburbs). Alas, it is very difficult to control for all these variables, since they are
not readily available in data  sets. In reality, the best the  report authors can do with such data sets is to explain
how all the other, relevant attributes may factor in (e.g., quality of driver and typical driving settings), and how
they can generate biased estimation (sometimes in either direction). Discussion of relevant literature that
looks more deeply at crash outcomes (e.g., Wang or Chen's papers, mentioned above, allowing for
heteroscedasticity and individual vehicle attributes, non-driver outcomes, etc.) will also be useful.

[Farmer] Casualty risk per crash does not fully measure the effects of vehicle size and weight reductions on
society.  Casualty risk per VMT best coincides with the NHTSA analysis of fatalities per VMT. The breakdown of
casualty risk per VMT into the crash frequency and crashworthiness components is of interest. However, the
surprising results reported here make everything suspect. For example, the Phase II report concludes that "the
detrimental effect of male drivers has to do with their higher tendency of getting into a serious crash rather
than their sensitivity to  injury once a serious crash has occurred" (p. 24). A few pages later it concludes that
"male drivers have essentially no effect on crash frequency, but cause a statistically significant increase in
fatality risk once a crash occurs" (p. 28).

[Van Auken]  [1] The basic methodology described by Kahane (2011) seems appropriate; and the extension by
Wenzel  (2011b) are also appropriate. However some results using these methods and data are not well
understood and need further diagnosis.

[2] The induced-exposure data set provided by NHTSA is based on the non-culpable vehicles in two-vehicle
crashes. See the Table 1 comments on  the limitations of this data.

[3] The state accident data files tend to have different database variable and coding definitions and criteria,
which could confound the results.
Please comment on the overall adequacy of
[Chen and Kockelman]  Overall, the study is an enriching complementary document to the Phase 1 assessment
                                                                                                                                       39

-------
LBNL's assessment of the 2011 NHTSA report
and its independent study of casualty risk for
predicting the effect of vehicle mass or
footprint reduction on safety. Provide any
recommended improvements that might
reasonably be adopted by the author to
improve the analysis.
of the 2011 NHTSA report. The parallel structure of the two reports allows the reader to easily compare and
contrast the various additional analyses which examine the relationship of vehicle mass and footprint
reduction on safety. However, as stated previously in the comments here, driver preference for specific car
types (including size and mass) is related to driver socioeconomic characteristics and driving behavior. As
vehicle, driver, and roadway environment characteristics all contribute to fatality risk, the effects of physical
vehicle changes such as mass or footprint reduction on safety should not be overstated when the other two
types of characteristics are not sufficiently accounted for.

[Farmer] Overall these are reasonably good studies. The Phase I report does a very good job of assessing the
NHTSA report of fatality risk.  However, the Phase II report should be more cautious in its conclusions
concerning casualty risk. The casualty analysis is based solely on police-reported data from 13 states, which:

    1.  May not be representative of the US as a whole.
    2.  Are inconsistent in the information given and the way in which it is coded.
    3.  Suffer from information that is missing, inaccurate, or unclear.

[Van Auken] The Wenzel (2011b) report provides a valuable supplement to the analysis and results in the
Kahane (2011) report.
ADDITIONAL COMMENTS:

[Farmer] Column G of Table 6.1 in the Phase II report provides the most appropriate comparison to results from the NHTSA report (Column A). For both
fatalities and casualties per VMT, a 100-lb weight reduction is most harmful in lighter cars, less harmful in heavier cars and lighter light trucks, and slightly
beneficial in heavier light trucks, minivans, and crossovers.

[Van Auken] See attached tables  6 and 7 below.
                                                                                                                                      40

-------
Table 6. Additional General Comments and Recommendations
Mike Van Auken
Section
4
All
All
4
COMMENTS AND RECOMMENDATIONS
Use of R2 is confusing. Suggest using lower case "r" when referring to the sample correlation coefficient (Box, Hunter, Hunter,
1978, P. 61); or upper case R when referring to the regression coefficient of multiple determination (Draper and Smith, 1981, p.
90).
In most cases the reported results are just estimates, but are not described as such. The word "effect" appears several hundred
times in this report with the "estimated" or other qualifier. In some cases this may be appropriate and in other cases it is not
appropriate. It is recommended that the author review each instance and revise as appropriate.
"Crashworthiness" in most instances should be changed to "crashworthiness and crash compatibility" because the fatalities
and/or serious injuries may either be in the subject vehicle (crashworthiness effect) or collision partner (crash compatibility
effect).
The statistical significance of the linear trends in Figures 4.1 through 4.9 are not reported. It would be helpful if the confidence
intervals or statistical significance of the linear trends were reported, either in addition to or instead of R2.
The confidence intervals for the estimated slopes should be added to the results in Tables 4.1 and 4.2.
41

-------
Table 7. Additional Specific Comments and Recommendations
Mike Van Auken
Section
Executive
Summary, 4
Executive
Summary, 6
5.3
5.3
5.3
Page
iv, v, 22, 66
vii, 63
49
48-49
49
COMMENTS AND RECOMMENDATIONS
Suggest changing "variance in risk" to "variation in risk" throughout.
The statement "In conclusion, casualty risk per crash is not necessarily a better metric than fatality risk per VMT
for evaluating the effect of mass or footprint reduction on risk; rather, it provides a different perspective in
assessing the benefits or drawbacks of mass and footprint reduction on safety in vehicles. However, it does
allow the separation of risk per VMT to be separated into its two components, crash frequency and risk per
crash" suggests that the casualty risk per crash metric was needed in order to assess the crash frequency and risk
per crash, which is incorrect. The DRI (2003-2012) methods have also estimated the effects of weight and size
on crash frequency (A/E) and risk per crash (F/E) in terms of fatalities.
It would be helpful to list the 18 manufacturer dummy variables in a table.
It is unclear why Lexus, Acura, and Infinity are treated as separate manufacturers, but Cadillac and Lincoln are
not.
It is unclear why AM General is considered a Chrysler brand. The AM General Hummer was sold by GM
beginning with the 2003 model year.
42

-------
   Review of "An Analysis of the Relationship between Casualty Risk per Crash and Vehicle Mass and
                      Footprint for Model Year 2000-2007 Light-Duty Vehicles"

                                         David L Greene

                                        January 10, 2012
Summary

The Phase I and Phase II analyses by Tom Wenzel of LBNL have been executed diligently and consistently
in accord with the methods and data used in the original NHTSA analysis. The studies contain many
valuable, new insights.  The phase I study highlights the weakening relationship between vehicle mass
and highway fatalities.  This is not only seen in decreasing coefficient estimates but in the very large
number of results that are not statistically significant. When regressions were done separately by
footprint deciles, vehicle mass was statistically significantly positively related to fatalities only for light-
duty trucks in rollovers, there were almost as many cases in which mass was negatively related to
fatalities (9 vs. 13 out of 27) and there were more instances of statistically significant negative
relationships than positive relationships. Given that so many tests are being jointly conducted, it is quite
possible that when joint probabilities are considered, there is no significant relationship between mass
and fatalities (more on joint probabilities later).  Showing the weakness and inconsistency of these
results is an important contribution.

Another meaningful contribution of the phase I study, and one that deserves more emphasis, is a logical
inference from the following findings: 1) much of the variance in risk remains unexplained even by the
most complete models, 2) control variables explain 1 to 2 orders of magnitude more of the variance
than the variables of interest (mass and size), 3) when key control variables are removed or changed it
strongly influences the coefficients of mass and size.  These results have very important implications for
the robustness of the results and the likelihood that some  or all of the apparently statistically significant
relationships are due to spurious correlations with omitted or imperfectly controlled factors. Noting
that exposure measures are control variables with constrained coefficients, the following observation
from the phase I study is especially perceptive.

   "Calculating risk as total fatalities per induced exposure crash, rather than per vehicle mile
   traveled, reverses the sign of mass reductions on risk in cars and the lighter light trucks, with  mass
   reduction leading to a reduction in risk in all vehicle types."

Finally, the phase I report notes that if only the control variables are included in the regression and
not size or mass, the resulting residuals from the regression are uncorrelated with size or mass.
Given these findings (as well as those of phase II) the conclusions that,

   "The 2011 NHTSA study, and this report, conclude that the effect of mass reduction on US fatality
   risk is small."
                                                                                              43

-------
should be revised with the following emendation, "... and probably non-existent."

Both studies, like the NHTSA analysis,  have shortcomings in terms of interpreting the results and the
language used to describe the results, and acknowledging the limitations of the data and methodology.
The limitations are extensive.  The interpretation of the results of the LBNL studies commits two
important, related  errors. The first is to attribute inferred coefficients of mass and size as representing
only the effects of vehicle mass and size when, as the phase I and II study results  indicate, there is a
virtual certainty that aliasing effects are present due to a combination of omitted variables, errors in
variables and correlations among variables.  Given that estimated driver and environmental factors
tend to have 1-2 orders of magnitude  larger impacts on safety outcomes than vehicle factors, the almost
certain presence of aliasing effects must be explicitly acknowledged as severely limiting the ability to
draw inferences about the effects of vehicle attributes.  Second, the language used in interpreting
results fails to  acknowledge that the analysis does not address the effects of down-weighting or down-
sizing specific vehicles or vehicle designs, but instead relies on correlations between vehicle weight and
size in existing vehicle designs. In existing vehicles, weight and size are correlated with each other and
many other vehicle attributes  (and driver and environmental attributes,  as well).  Thus, the study is not
actually measuring the effects of down-weighting via the material substitution and design changes likely
to occur as a consequence of fuel economy and emissions standards. An early example of the kind of
misleading language referred to here can be found on page iv.

   "For example, a 100-lb reduction in the mass of lighter cars leads to a 1.84% increase in crash
   frequency (columns B), while mass reduction leads to a 0.76% decrease in the number of fatalities
   per crash (column C);"

This statement is misleading in that it  implies causality rather than correlation, and it is additionally
misleading in that it implies that the inference applies to removing weight from specific vehicles.
Neither is correct.  A better statement would be the following.

   "For example, vehicles in the lighter class that are 100 Ibs. lighter are correlated with a 1.84%
   increase in  crash frequency...."

There are so many examples of this misleading language that it is not feasible to list them all. All
should be corrected, however. Failure to correct them could lead to serious misinterpretation of the
studies' findings.

Following in the footsteps of the seminal study by DRI, the NHTSA and LBNL studies contribute to the
literature in three important ways: 1) the LBNL and NHTSA studies recognize that the societal safety
perspective is the correct perspective  to when assessing the impacts of fuel economy and emissions
regulations, 2) they recognize that vehicle dimensions and vehicle mass may have separate and
potentially different impacts on both the likelihood of a crash and the outcomes of the crash and, 3) the
LBNL phase two analysis makes an additional contribution by attempting to disentangle factors affecting
the likelihood of a crash and factors affecting the outcomes of a crash.
                                                                                             44

-------
Speaking of the DRI study, I am puzzled about why there are no references cited in the phase I study and
only a handful all by Kahane and Wenzel, in the phase II study. This is perhaps due to the scope of work
defined for the two studies but there are highly relevant studies in the literature that could have been
cited, those by DRI foremost among them.  Making use of the insights from these studies would have
been helpful in interpreting the results of both phase I and phase  II.

Lack of a Theory or Model of the Phenomenon

Both the  NHTSA and LBNL studies lack a rigorous theory of the process by which down-weighting at
constant  size or down-sizing at constant mass affect societal safety either through crash avoidance or
crashworthiness. This is not a trivial shortcoming because it affects the ability to formulate hypotheses
and interpret results. Prior to the dissenting report on safety of the NRC 2002 CAFE report, the physics
of elastic collisions between objects was typically cited as the underlying physical model. That report
showed how taking the societal perspective renders that model inappropriate.  What remains appears
to be far  more complex, involving the quantity of kinetic energy, the ability of vehicle designs to absorb
that energy so as to minimize maximum deceleration rates, stability, maneuverability, safety
technologies, and  more.

The consequences of the lack of a rigorous theory are that it is not known, a priori, what the signs of
coefficients are expected to be, let alone what their quantitative relationships should be. Hypotheses
must be formulated based on  intuition and the interpretation of results is likewise ad hoc. One
implication of this is that results that suggest that lower fatalities are associated with lower vehicle mass
have equal standing, a priori, with results that indicate that higher fatalities are associated with  lower
vehicle mass, and  similarly for vehicle size. There are no surprising or unsurprising results, in theory.

This also  makes it  difficult to develop a plan for statistically testing the model or theory and its
implications.  It would have been helpful to the reader to have been presented  early on  in the report
with such a plan of analysis.

On the Virtual Certainty of Aliasing

The LBNL report typically attributes causal effects to correlations between mass or size and safety.  In
fact, most or all of the observed correlations are almost certainly affected by aliasing effects. There is
ample evidence for this inference in the results presented in the LBNL phase II report.

The coefficients of mass and size change in important ways when  different model formulations are
estimated.  Removing and adding control variables changes the magnitudes and sometimes the  signs of
the mass and size variables. This means that, at a minimum, the mass and size variables alias the effects
of the omitted control variables. The question is whether the aliasing is eliminated entirely by the
inclusion  of the control variables available or whether some aliasing remains either because not all
relevant and correlated control variables have been included or because the included control variables
are imperfect measures of the factors they are intended to represent.
                                                                                            45

-------
The latter seems highly likely for the following reasons.  First, the overall explanatory power of the full
models (including control variables), as measured by their R2 is low. Most of the variance in casualties
and fatalities remains unexplained. Second, at least some of the important included control variables
are only crude approximations of the factors they are intended to represent.  For example, dummy
variables represent differences in state reporting practices, age and gender represent risky driving
behavior differences among owners of different sizes and masses of vehicles, the presence or absence
of a kind  of safety equipment represents both its performance and use in a  particular vehicle, and
calendar year dummy variables represent unknown factors associated with  the respective calendar
years.  Such practices are common and their use is appropriate. Third, the control variables generally
account for 1-2 orders of magnitude more variance in the casualty and fatality variables than do vehicle
weight and size. To recap, the amount of unexplained variability in the dependent variables is larger
relative to the variance statistically explained by the most complete models. Control variables are
correlated with size and mass, and they account for 1-2 orders of magnitude more variability in the
dependent variables than the variables of interest, mass and size. Therefore, even small correlations of
size and mass with omitted variables or with errors (imprecision) of the control variables could easily
result in biased estimates for the effects of size and mass on the dependent variables.

Tom Wenzel is to be commended for providing the results that definitively demonstrate the three key
points made above. The above is not a criticism of the analysis nor of the results, per se.  It is a criticism
of their interpretation.  In light of the above, the results should be interpreted in light of the virtual
certainty that many of the estimated coefficients are likely to be biased in ways that make their
interpretation highly uncertain. The implication is that phrases such as "down-weighting or down-sizing
caused" to "mass (or size) and unobserved correlated factors are associated with..."

On Joint  Probabilities

The NHTSA and LBNL studies do not  correctly interpret their results as joint statistical tests. When
testing a  hypothesis on, for example, 5 vehicle classes simultaneously, a result for one equation that
might be statistically significant on its own may not be statistically significant as one  of five related tests.

Statistical analyses comprised of multiple  regressions too often overlook the fact that tests of statistical
significance designed for individual regressions may not apply in the case of multiple regressions. That is
the case here. NHTSA conducted 5 analyses to infer relationships between  mass differences among
vehicles holding footprint constant for 5 classes of cars. The results showed one relationship out of five
was statistically significant. As table 1 illustrates, using a simple example, if one conducts 5 trials, each
with a 0.05 probability of given result, there is a  22.6% probability of finding at least  one such result in
the five trials. Thus, the joint  significance level of the overall result (1 statistically significant regression
out of 5)  is 0.226, rather than  0.05.
                                                                                              46

-------
Table 1. Simplified Illustration of the Joint Probability of Inferences in Multiple Regressions

   # Significant
   Regressions    Combinations
                0                1
                1                5
                2               10
                3               10
                4                5
                5                1

0.95
0.773780938
0.81450625
0.857375
0.9025
0.95
1



0.05
1
0.05
0.0025
0.000125
0.00000625
3.125E-07


Joint
Probability
77.37809%
20.36266%
2.14344%
0.11281%
0.00297%
0.00003%
100.0%
22.6%
So there is between a 1:4 and a 1:5 chance of getting one statistically significant result by pure chance.
In fact, the actual significance level of the results is more complicated to calculate, and probably a bit
smaller than 0.226.  Thus, it is very appropriate for Dr. Kahane to add the qualifier "if any" to his
conclusions about the relationship between the societal highway fatalities and mass reduction, holding
footprint constant.  Had appropriate tests of joint statistical significance been used to evaluate the
results in the NHTSA and LBNL studies, the significance levels very likely would not meet accepted
criteria for statistical significance.  This could change the conclusions of the studies from the inference
that mass is correlated with fatalities or casualties in some case but not others to the lack of statistically
significant evidence that mass is correlated with fatalities or injuries on the highway. This is an
important difference.

Page-by-Page Comments

I will make page by page comments on the phase II study only, since that contains the overwhelming
share of original contributions and the key findings of the phase I study are recapitulated there.

p. iii    Paragraph 3. This would be a very good place to acknowledge the importance of driver behavior
        and environment on crash avoidance especially.

p. iv    Para. 3. This would be a good place to discuss probability inference in joint tests.

        Para 4. The statement about a 100-lb reduction in the mass, etc., is a good example of
        misleading language.

p. v     Para. 1. Again, it is misleading to say that mass reduction increases crash frequency, for reasons
        stated above.

        Para. 2. It is more accurate to describe the association of lower vehicle mass with casualty risk
        than the "effect of mass reduction on..." casualty risk.
        Para. 3. Would benefit greatly from joint probability inferences.
                                                                                              47

-------
       Para. 4. As noted above, this shows how much more important the control variables are than
       the variables of interest.

       Para 5.  Again, these are correlations not necessarily effects.

p. vi    Para. 1. Again, mass reduction is misleading terminology and you do not know if it increases
       casualty risk or not, you know only a correlation.  Why is this so important? It is the virtual
       certainty of spurious correlations, or aliasing, as noted above.

       Para. 4. (1st bullet) This is clear evidence of aliasing. Take variables  out of the regression and the
       coefficients of interest change in important ways.  Are there no important factors still missing?
       Are the variables included perfect measures of the factors of interest? Of course not. Thus,
       there must be remaining aliasing. How bad is it?  We  don't know.

       The third bullet shows the same effect with a different set of variables.

p. vii   Para. 4. No, your analysis does not indicate "...that much of the detrimental effect of mass or
       footprint reduction  on risk can be attributed to the tendency for mass or footprint reduction to
       increase crash frequency." Again, you have correlation, not causation and  you have good
       reason to believe that what you are seeing is affected by spurious correlations.

       Para. 5. The "effect" is small, 1-2 orders of magnitude smaller than correlations with other
       control  variables, and  IS strongly affected by which variables are in  the equation, as stated on
       the previous page, and there is a great deal of unexplained variance. Please reconsider the
       meaning of these results  in light of the comments above.

       Finally,  as the last paragraph of the ES implies, it would be far better not to speak in terms of
       "reducing" mass or  size. That is not what is happening in your data set.

p. 1    Para. 4. Risk per VMT includes the effects of how well vehicles are  driven as well as how well
       they can be driven.  I think there is no chance that you have fully accounted for how well
       vehicles are driven.

p. 2    Para. 2. Exposure measures are explanatory variables whose elasticity is constrained to 1. That
       is, it is assumed that an increase in vehicle use of 1 vehicle mile produces a 1 unit increase in the
       chance  of a fatality  (or casualty as the case may be). This is actually a maintained hypothesis. If
       this hypothesis is incorrect, it can  bias the other coefficients in the equation.  Thus, the change
       from fatalities/VMT to fatalities/registration-year, to fatalities per crash not only changes the
       meaning of the analysis, it also may bias coefficients in the event that the true relationship
       between fatalities and VMT is not an elasticity of 1.

p. 3    Line 1.  Please acknowledge that your "accounting for differences in driver characteristics, crash
       locations, and other vehicle attributes" is incomplete and that this could affect your inferences
       about size and mass.

p. 3    Para. 2. NHTSA's use of "non-culpable" vehicles involved in two vehicle crashes as an  exposure
       measure raises its own issues.  How non-culpable was the non-culpable vehicle. Often this is a
       matter of degree, rather than black or white. Driver behavior may also be  involved. It seems to
                                                                                             48

-------
       me this is just another potential measure of exposure that may or may not be better than any
       other measure and may introduce new sources of bias in the analysis.

p. 3    Para. 4. Induced exposure needs to be defined.  What is it intended to mean? This needs to be
       explicit.

       Also, I am startled that there are no equations in these reports.  Equations can provide an
       unequivocal explanation of the assumed relationships that cannot be adequately accomplished
       by words, in many cases. Why no equations?

p. 7    Para 3.  CUVs and minivans are involved in fewer crashes with stationary objects than  cars.
       Why?  Is it the drivers, the vehicles, or the passengers?  How well can you control for such
       differences?  Not well.  What does this mean for your analysis?

p. 9    Para 1.  Here an equation showing how the weighting was done would be very helpful.

p. 9    Bullet 1. Excluding these vehicle types implies that the control variables in the model are not
       adequate to account for whatever makes these vehicle types different from the vehicles
       included in the analysis. First, this is an admission that the model is not adequate to explain the
       fatalities associated with these vehicles. Second, it is an admission that if they were included
       the coefficients on the variables of interest would likely  be biased by spurious correlations.
       Clearly, it would not even be sufficient to include the vehicles along with a control variable (e.g.,
       X = 1 if vehicle is a police car, 0 otherwise). This is yet another indication that the model suffers
       generally from omitted variables, errors in variables and correlation among right-hand side
       variables.

p. 10   Line 3. Sentence does not make sense. Please correct.

p. 12   Sect. 2.3.  Please provide an equation.

p. 13   Were the confidence intervals calculated using ex-l? Please state explicitly or, better, show an
       equation.

       Para. 2. Here another instance where you say "mass reduction increases societal fatality risk"
       but you really are not entitled to say that. It is misleading. Also, the NHTSA CI's are larger, as
       they should be in a joint test.

       Para. 3. These results require an underlying theory for interpretation.  The lack of one makes it
       seem like there is just no consistency in the  results.

       Para. 4. The fact that the results for fatalities per crash differ substantially from fatality per VMT
       may be very important.  Taken at face value, it would imply that any negative effect of reduced
       mass is due to its effect on crash avoidance  (crash probability) rather than crashworthiness.
       This is where the lack of a theory is most troubling.  Why would  that be? Are lighter vehicles
       less easily controlled, etc.? Or, as seems much more likely, is there a spurious correlation
       between mass and other omitted or imperfectly measured factors (including  driver behavior)
       that lead to an increased probability of a crash?  Consider, for example, driver age. Driver age  is
       related to crash involvement. Driver age is a control variable. But are all young drivers the
                                                                                             49

-------
       same? Is it possible that young drivers more prone to risky behavior tend to drive lighter
       vehicles? If so, this could partly explain the result observed. Of course, this is just speculative,
       but the point is that correlations with imperfectly measured and omitted factors are highly likely
       to be present in the data and, if there, could easily affect the statistical inferences.

p. 14   Here we  see that changing the exposure measure influence the effect of mass and size on
       fatalities and casualties, which is more evidence that spurious correlations are likely to be
       biasing estimated coefficients for mass and size.

       The  bar graphs with confidence intervals are well done and convey a great deal of information
       effectively. The patterns of magnitude and statistical significance are difficult to interpret, partly
       because there is no explicit theory of what should happen and partly, perhaps, because the
       relationships are actually not real.

p. 16   Para. 1. Reduction in the mass of lighter cars increases crash frequency but reduces fatalities
       per crash. This is contradictory to the  previously maintained theory that mass protects due to
       the physics of velocity changes in elastic collisions.  Indeed, there is no theoretical explanation
       for these results, only speculation.

p. 18   Para. 1. Here is a good example of such casual speculation.

p. 19   Para. 1. Developing VMT weights for the 13 states is a good idea, given the effect of exposure
       measures on inferences. Still the results would not be definitive.

p. 20   Para. 2. Mass reduction leads to a large reduction in risk only in crashes with objects for heavier
       cars?  There is only one type of crash in which the simple physics of collisions leads to an
       unambiguous benefit for increased mass, and that is collisions with moveable or breakable
       objects.  This finding contradicts even that.

       Para. 3. More speculation, this time about  rollovers.

p. 21   Para. 3. "Curiously,..." Curiouser and curiouser.

       Figure 2.15 printed without labels. Could be my computer but the other graphs were fine.

p. 24   Para. 2. This is probably a very important finding that needs further investigation and
       explanation.  As figure 2.16 illustrates well, the correlations with mass and size are orders of
       magnitude smaller than the correlations with driver and environmental factors. This is why
       even small correlations with  omitted or imperfectly measured control factors could be, are even
       likely to be, predominantly responsible for the estimated coefficients of mass and size.

p. 25   Para. 2. The results for minivans discussed here could be due to what is going on inside the
       vehicle as much or more than the vehicle itself.  How could these results be explained in terms
       of the vehicle itself. Figure 2.17 shows this again. The effects of calendar year dummies, which
       can only  be considered rough approximations to unknown and various time-related factors,
       have much large effects than size or mass.  Again in figure 2.20.
                                                                                              50

-------
p. 28   Para. 1. "Surprisingly,..." How can side airbags, which deploy only in a crash, reduce crash
       frequency but not fatality risk in a crash?  Only if the real effect is a reflection of who buys a
       CUV/minivan with side airbags and how and where they drive. There are more surprising
       inferences in paragraph 2 about male and female drivers. Surprising relative to what theory?

p. 32   The problem here is not numerical multicollinearity (numerical difficulties inverting the cross-
       product matrix) but the more complex problem of correlations among right-hand side variables,
       omitted variables, errors in variables, and correlations of included variables with omitted and
       imperfectly measured variables.  This leads to biased estimates.

p. 36   Table 3.1 cries out for inference based on joint probabilities. What is the probability of
       observing "success" in at least 3 out of 27 trials when the true probability of success is only 0.05.
       See discussion above. The probability is certainly much higher than 0.05 and probably closer to
       0.5.  The implication is that, taken together, these results do not  show any statistically
       significant relationship between mass or size and risk per crash.  If there were a  rigorous
       underlying theory, the interpretation might be different (patterns of significance could  matter)
       but there is none. Again, good graphs on succeeding pages.

p. 41   The statistical significance  of such a relationship should be the same whether bins are used or
       not. Is it?

p. 42   Para. 3. R-squared is not the correct measure of statistical significance.  Is the coefficient of
       weight significantly different from zero?

p. 43   Para. 2. This is perhaps the key finding of the phase I and II analyses. Control variables explain
       1-2 orders of magnitude more variance than  size or mass. Still, most of the variance remains
       unexplained and is uncorrelated with mass or size. It is very likely there is nothing going on
       here.

p. 47   Para. 2. More evidence of correlation of mass and size with control variables and how changing
       definitions or excluding control variables results in important changes in the coefficient of mass
       and size. Such results are considered unstable.

p. 49   Para. 1. The rationale for the grouping of manufacturers is not obvious. Can you explain it?

       Para. 2. Yet more evidence for the instability of the model and likelihood that variables still
       missing from the  model, plus errors in measuring the included control variables  are likely biasing
       the inferences. The results described in this paragraph do not make sense to me. How can they
       be explained other than random  results?

p. 50   Para. 1. More casual interpretation of results.  OK, maybe, maybe not.  Same for paragraph 2.
       The economy faltered in 2008 but the big negative effect was in 2007. The downward trend
       started in 2004. This correlates neither with  vehicle mass changes over time nor economic
       growth as measured by real GDP. Idle speculation.

p. 51   Para. 1. Good discussion of the gratuitous speculation by NHTSA about the meaning of the
       observed correlations.  This is more a Rorschach test than statistical analysis.
                                                                                             51

-------
p. 51   Para. 2. "We have no explanation for why..." More of this kind of honest appraisal is needed in
       studies like these.

p. 52   More results for which there is no explanation.

p. 53   Why the interaction between calendar year dummy variables and safety equipment? The
       presence of the safety equipment on a particular vehicle is established.  What has calendar year
       (not model year) to do with it? Again, one suspects spurious correlations.

p. 55   One needs to think carefully about the reasons why vehicles would be excluded. It does not
       appear that NHTSA did that. First line of first paragraph "used" rather than "sused".

p. 57   Well reasoned. It is interesting that NHTSA resisted including footprint or size in previous
       analyses on the grounds of correlation with mass. These results show that assertion was
       groundless.

p. 59   Para. 3. Rather than  say risk per VMT accounts for two effects, it is better to say it includes or
       comprises two effects.  But this statement also ignores the important influences of drivers and
       environment and their potential correlations with other factors. Yes, it includes how well a
       vehicle can be driven, but more importantly it includes how well a vehicle IS driven. That is in
       there too and is very likely to be correlated with make, model, and other vehicle attributes.

p. 60   Para. 3. Here again, the conclusions are misstated. It is not a genuine "reduction" in mass, but
       an association with the mass of vehicles. And how does it "lead" to and increase in crash
       frequency? What is the theory or model that predicts this? Driver and environment are very
       likely mixed up in these results to an unknown but likely substantial degree. So what can we
       really conclude? Not this.

       As I read the conclusions and inferences I find myself asking, why?, why?, repeatedly without
       any sound explanations. Page 61, paragraph  3 contains more "surprising" results. Surprising
       because they are contrary to theory?  Surprising because they are contrary to intuition?
       Surprising because they are random?  To what can we attribute so many "surprising" results,
       and how many must there be before one concludes that the analysis is not  revealing what we
       had hoped it would.

       Para. 4. Again, this cries out for joint statistical inference. Three statistically significant results
       out of  27 is probably nothing statistically significant at all.

p. 62   Para. 1. What this shows, again, is that the coefficients of mass and size are strongly influenced
       by which control variables are included in the model and how they are defined. These results
       and their implications need explanation. The bottom line is that the effects of mass and size are
       likely to be (after the necessary joint significance calculations are done)  not statistically
       significant, not consistent, and not robust.

p. 63   How do mass and footprint reduction (again, it's not really reduction in the sense of designing
       lighter vehicles to increase fuel economy or reduce GHG emissions, the issue at hand) increase
       crash frequency. What is the theory?  I don't find a theory in either the  NHTSA  report or the
       phase  I and phase II studies. Absent a theory, these results seem sufficiently unstable and
                                                                                             52

-------
inconsistent to be highly questionable as evidence of any relationship between mass or size and
crashworthiness or crash avoidance. I think joint estimation of significance levels would provide
additional support for this view.
                                                                                      53

-------
4.     References

Tom Wenzel. Assessment of NHTSA's Report "Relationships Between Fatality Risk, Mass, and Footprint
       in Model Year 2000-2007 Passenger Cars and LTVs". 2011.

Tom Wenzel. Analysis of the Relationship between Casualty Risk Per Crash and Vehicle Mass and
       Footprint for Model Year 2000-2007 Light-Duty Vehicles. 2011.
                                                                                         54

-------
                                  Appendix A: Resumes of Peer Reviewers



                                         T. Donna Chen, P.E.

1206 Arcadia Ave, Austin, TX 78757         •          (501)908-9620         •         chen.donna@gmail.com

EDUCATION
University of Texas, Austin, TX
  Doctorate of Philosophy, Civil Engineering - Transportation (3.9 GPR)               August 2010- Present
•   Thrust Fellow
•   Advanced Institute for Transportation Infrastructure Engineering and Management Fellow
University of Texas-Arlington, Arlington, TX
  Master of Engineering, Civil Engineering - Transportation (3.9 GPR)                 December 2008
Texas A&M University, College Station, TX
    Bachelor of Science, Civil Engineering (3.8 GPR)                                May 2005
•   National Merit Scholar
•   Recipient of the Texas A&M President's Endowed and  Merit Plus scholarships
•   Recipient of the Dallas/Fort Worth Women's Transportation Seminar scholarship
•   Recipient of Society of Women Engineers A&M Student Chapter scholarship
•   Alumni of Texas A&M Engineering Scholars Program

WORK EXPERIENCE
Center for Transportation Research, Austin, TX
   Graduate Research Assistant                                                  August 2010- Present
        •    Developing a reference on transportation economics for TxDOT
Richland College, Dallas, TX
  Instructor, College of Business, Engineering, and Technology                       January 2010- August 2010
        •    Teaching statics and dynamics as part of core engineering curriculum.
HNTB Corp., Piano, TX
  Transportation Planning Engineer II                                             December 2008 - November 2009
  Transportation Planning Engineer I                                              June 2005 - December 2008
        Performed geometric design, cost estimation, and operational analysis for projects such  as:
        New Route Studies
        •    Lavon Lake  Bridge Study, Collin County, Texas - collected constraints  data; developed and evaluated alignment
            alternatives for ten mile  bridge route; assisted  in developing GIS exhibits; answered  citizen questions at public
            meeting.
        •    Border Highway-East,  El Paso, Texas -  developed four  alignment and two interchange alternatives; evaluated
            alternatives and selected preferred alternative based on constraints; assisted in development of technical documents.
        •    Collin County Outer Loop - developed alternative corridors and alignments; evaluated alternatives and selected locally
            preferred alignment for 40 mile study from US 75 to Rockwall County Line.
        Traffic Analyses
        •    I-95/395 HOT Lanes, Arlington, Virginia - performed LOS analysis for revised  managed lane access for 24 mile
            corridor.
        •    Beltway 8/SH 249 Connector, Houston, Texas - modeled two proposed ramping alternatives in CORSIM.
        •    I-30, Greenville, Texas - performed LOS analysis using HCS and prepared Interstate Access Justification report.
        •    I-35,  Lorena,  Texas  - modeled traffic along  corridor using CORSIM and prepared Interstate  Access  Justification
            report.
        Design Schematics
        •    Dallas North Tollway Ramping Modifications, Piano,  Texas  - designed various  ramping improvements along  DNT
            between PGBT and SH 121 to improve congestion and accommodate for an additional mainlane in each direction.
        •    Dallas North Tollway/SH  121 Interchange, Piano, Texas - designed four ramping options for the  retrofit of a fully
            directional  interchange between existing mainlanes;  conducted LOS analysis for toll and non-toll options along SH
            121.
        •    1-10, El Paso, Texas - developed two alternatives for the addition of two managed lanes to the existing four-lane
            facility.
        •    US 67, Cedar Hill and Midlothian, Texas - produced schematic plans for the addition of center reversible HOV lane.


                                                                                                            55

-------
        Participated in leadership development activities such as:
           Leadership HNTB - completed professional development/project management training program for younger staff.
           H4 Community Outreach Program - worked with Leadership HNTB classmates to implement a community outreach
           program which yielded over 900 hours of combined service and over $13,000 in charitable donations in one year.
           HNTB College Recruitment Team - represented HNTB as college recruiter at Texas A&M Engineering Career Fair
           and Society of Women Engineers National Conference Career Fair.
Texas Transportation Institute, College Station, TX
    Undergraduate Research Assistant                                                 May 2003 - May 2004
           Analyzed travel behavior data for the Houston QuickRide (HOT lane) project using Excel and SPSS
           Received the  Dwight Look College of Engineering  Undergraduate Summer Research Grant and published paper in
           Southwest Regional University Transportation Center Compendium

PROFESSIONAL MEMBERSHIPS/HONOR SOCIETIES

Intelligent Transportation Society of America
           UT Student Chapter President                                             November 2010- Present
               o   Coordinated new student and intern peer support activities
               o   Organized prospective student recruitment activities
               o   Served as UT liaison for the 2011 Southwest Region University Transportation Center Student
                   Symposium
               o   Represented ITS as Explore UT day volunteer
American Society of Civil Engineers
           Dallas  Branch History and Heritage Committee Chair                          2008-2010
Society of Women Engineers
           Dallas  Section Career Night Chair                                          2006 - 2008
Tau Beta Pi Member                                                                2004 - Present
Chi Epsilon Member                                                                2004 - Present
                                                                                                           56

-------
                              CHARLES  M.  FARMER
                       1005 North Glebe Road, Suite 800
                           Arlington, Virginia 22201
                               cfarmer@iihs.org
                                 (703)247-1590
EDUCATION
December, 1986
August, 1981
Iowa State University, Ames, Iowa
Doctor of Philosophy
Major: Statistics

Old Dominion University, Norfolk, Virginia
Master of Science
Major: Applied Mathematics
Concentration: Statistics
May, 1979
St. John Fisher College, Rochester, New York
Bachelor of Arts
Major: Mathematics
Minor: Political Science
PROFESSIONAL
EXPERIENCE
2004-Present
1997-2004
1994-1997
1987-1994
1986-1987
Director, Statistical Services
Senior Statistician
Statistician
Insurance Institute for Highway Safety
Arlington, Virginia

Assistant Professor
Department of Mathematics
James Madison University
Harrisonburg, Virginia

Visiting Assistant Professor
Department of Statistics
University of Kentucky
Lexington, Kentucky
PROFESSIONAL
AFFILIATIONS
American Statistical Association
                                                                            57

-------
RECENT
PUBLICATIONS
June, 2011
October, 2010
Farmer, C.M. Methods for estimating driver death rates by
vehicle make and series. Arlington, VA: Insurance Institute
for Highway Safety.

Farmer, C.M., Braitman, K.A., and Lund, A.K. Cell phone use
while driving and attributable crash risk. Traffic  Injury
Prevention 11:466-470.
August, 2010
May, 2010
April, 2010
February, 2010
February, 2010
McCartt, A.T., Farmer, C.M., and Jenness, J.W. Perceptions
and experiences of participants in a study of in-vehicle
monitoring of teenage drivers. Traffic Injury Prevention
11:361-370.

Farmer, C.M. Effects of electronic stability control on
fatal crash risk. Arlington, VA: Insurance Institute for
Highway Safety.

McCartt, A.T., Hellinga, L.A., Strouse, L.M., and Farmer,
C.M. Long-term effects of hand-held cell phone laws on
driver hand-held cell phone use. Traffic Injury Prevention
11:133-141.

Farmer, C.M. and Wells, J.K. Effect of enhanced seat belt
reminders on driver fatality risk. Journal of Safety
Research 41:53-57.

Farmer, C.M., Kirley, B.B., and McCartt, A.T. In-vehicle
monitoring and the driving behavior of teenagers. Journal
of Safety Research 41:39-45.
                                                                             58

-------
DAVID L. GREENE

Home:  212 Way Station Trail • Farragut, Tennessee 37922 • (865) 966-0891
Work:  Oak Ridge National Laboratory • National Transportation Research Center • 2360 Cherahala Boulevard •
       Knoxville, Tennessee 37932 • (865) 946-1310

    PERSONAL

       Born: November 18, 1949, New York, New York
       Married, two children

    EDUCATION	

       THE JOHNS HOPKINS UNIVERSITY
       Ph.D., Geography and Environmental Engineering, 1973—78

       UNIVERSITY OF OREGON
       M.A., 1972-73

       COLUMBIA UNIVERSITY
       B.A., 1967-71


    EMPLOYMENT	

       UNIVERSITY OF TENNESSEE, KNOXVILLE                                          2010-PRESENT

       1/2010-Present    Senior Fellow, Howard H. Baker, Jr. Center for Public Policy
                       Research Professor, Department of Economics

       INSTITUTE FOR TRANSPORTATION STUDIES, UNIVERSITY OF CALIFORNIA, DAVIS           2008-2009

       9/2008-6/2009     Visiting Research Faculty

       OAK RIDGE NATIONAL LABORATORY (ORNL)                                     1977-PRESENT

       1999—Present      Corporate Fellow, Oak Ridge National Laboratory
       1989—1999        Senior Research Staff Member II and Manager of Energy Policy Research
                       Programs, Center for Transportation Analysis
       1988—1989        Senior Research Analyst, Office of Policy Integration, U.S. Department of
                       Energy (On assignment from ORNL)
       1987—1988       Head, Transportation Research Section
       1984-1987        Senior Research Staff Member I
       1982-1984       Research Staff Member
       1980—1982       Leader, Transportation Energy Group
       1977-1980       Research Associate

    AWARDS AND HONORS

       2011 DOE Vehicle Technologies Program R&D Award, U.S. Department of Energy (with Z. Lin)
       2011 Edward L. Ullman Award, Association of American Geographers
       2009 Alliance to Save Energy, Energy Efficiency Hall of Fame
       2008 Science Communicator Award, UT-Battelle
       Recognition by the Intergovernmental Panel on Climate Change for Contributions to the Award of the 2008
           Nobel Peace Prize to the IPCC
                                                                                           59

-------
    2007 Department of Energy Hydrogen Program R&D Award (with P.N. Leiby)
    Barry D. McNutt Award for Excellence in Automotive Policy Analysis, Society of Automotive Engineers,
        2007
    Member Emeritus, Transportation Research Board Committee on Alternative Fuels, 2006
    Barry D. McNutt Award for best paper of 2004, Energy Committee, Transportation Research Board
    Lifetime National Associate of the National Academies, 2002
    UT-Battelle Award for Excellence in Science and Technology, 2001
    Oak Ridge National Laboratory Significant Event Award, 2001
    Corporate Fellow of Oak Ridge National Laboratory, 1999
    Outstanding Paper of 1999, The Energy Journal, International Association for Energy Economics
    Lockheed-Martin Significant Event Award, 1999
    Member Emeritus, Transportation Research Board Committee on Transportation Energy, 1998
    Lockheed-Martin Significant Event Award, 1996
    Distinguished Service Certificate, Transportation Research Board, 1993
    ORNL Special Achievement Award, 1991
    Distinguished Service Certificate, Transportation Research Board, 1989
    Energy Specialty Group Paper Award, Association of American Geographers, 1986
    ORNL Special Recognition Award, Oak Ridge National Laboratory,  1986
    Technical Achievement Award, Martin Marietta Energy Systems, 1985
    Pyke Johnson Award, Transportation Research Board, 1984

PROFESSIONAL ACTIVITIES
    •    Board of Directors, American Council for an Energy Efficiency Economy
    •    Board of Advisors, Institute for Transportation Studies, University of California, Davis
    •    Editorial Advisory Board, Transportation Research Part D, 1996-2006
    •    Editorial Board Member, Energy Policy, 2001-present
    •    Editorial Board Member, Journal of Transportation and Statistics, 2001-2006
    •    Editorial Board Member, Transportation Quarterly, 1999-2005
    •    Editor-in-Chief, Journal of Transportation and Statistics, 1997-2000
    •    Editorial Board Member, Macmillan Encyclopedia of Energy, 1998-2001
    •    Editorial Advisory Board, Transportation Research A, 1986-1997
    •    National Research Council
            Transportation Research Board Standing Committees:
                Committee on Transportation and Sustainability, Member, 2006-present
                Committee on Energy, A1F01, Chairman 1983-1986, 1986-1990; Member, 1993-1998;
                   Member Emeritus, 1999-present
                Subcommittee on Forecasting Transportation Energy Demand,
                   A1F01(2), Chairman, 1982-1983
                Section F, Energy and Environmental Concerns, Chairman, 1990-1992
                Committee on Alternative Fuels, A1F05, Member, 1993-2006,
                   Member Emeritus, 2006-present
                Task Force on Freight Transportation Data, A1B51, Secretary, 1989-1996
                Committee on Transportation Information Systems and Data Requirements,
                   Member, 1983-1986, 1986-1989
            Ad Hoc Committees:
                Committee on Transitions to Alternative Vehicles and Fuels, 2011-2012
                Special Task Force on Energy and Climate Change, 2008-2009
                Committee on the Assessment of Fuel Economy Technologies for Light-Duty Vehicles,
                2007-2010
                Planning Group for Workshop on Issues Related to Peaking of Global Oil Production, 2005
                Committee on State Practices in Setting Mobile Source  Emissions Standards, 2004-2006
                Chair, Committee for the Symposium on Introducing Sustainability into Surface
                   Transportation Planning, 2003-2004
                Panel on Combating Global Warming through Sustainable Surface Transportation Policy,
                   TCRP Project Panel H-21A, 2002-2005
                                                                                              60

-------
                Committee on Effectiveness and Impacts of Corporate Average Fuel Economy (CAFE)
                   Standards, 2001
                Committee for the Study of the Impacts of Highway Capacity Improvements on Air Quality
                   and Energy Consumption, 1993-1994
                Committee on Fuel Economy of Automobiles and Light Trucks, Energy Engineering Board,
                   Commission on Engineering and Technical Systems, 1991-1992
                Committee for the Study of High-Speed Surface Transportation in the United States, 1990
                Planning Group on Strategic Issues in Domestic Freight Transportation, 1990
                Steering Committee for Conference on Transportation, Urban Form, and the Environment,
                   1990
                National Cooperative Highway Research Program, Panel on "Evaluating Alternative Methods
                   of Highway Finance," 1991-1992
        Intergovernmental Panel on Climate Change
            Lead Author, Working Group III, Fourth Assessment Report, 2007
            Lead Author, Working Group III, Third Assessment, 2001
            Lead Author, Working Group III, Aviation and the Global Atmosphere, 1999
            Principal Lead Author, Working Group II, Second Assessment Report, 1995
        Association of American Geographers
            Board of Directors, Transportation Specialty Group, 1989-1991
            Secretary-Treasurer, Transportation Geography Specialty Group, 1980-1982
            Editor, Transportation Geography Newsletter, 1980-1982
        Society of Automotive Engineers, member, 1985-present
        International Association for Energy Economics, member
        Consulting
            International Council for Clean Transportation, 2011
            International Transport Forum, 2007
            Addx Corporation, 2007
            United Nations Framework Convention on Climate Change, 2007
            Securing America's Future Energy, 2007
            Center for Clean Air Policy, 2007
            Pollution Probe Canada, 2006-2007
            The Energy Foundation China Project,  2005—present
            The Pew Center on Global Climate Change, 2004—present
            Eno Transportation Foundation, 1991-1996
            Transportation Research Board, 1996-1997
BOOKS
    and D.W. Jones and Mark Delucchi, eds., The Full Costs and Benefits of Transportation, Springer-Verlag,
    Heidelberg, 1997.

    Transportation and Energy, Eno Foundation for Transportation, Lansdowne, Virginia, 1996.

    and D. J.  Santini, eds., Transportation and Global Climate  Change, American Council for an Energy
    Efficient Economy, Washington, DC, 1993.

ARTICLES IN PROFESSIONAL JOURNALS	

    "What's Greener than a VMT  Tax?  The Case  for an Indexed Energy User Fee to Finance U.S. Surface
    Transportation," Transportation Research D-Environment, vol. 16, pp. 451-458.

    "Uncertainty, Loss Aversion and  Markets for Energy Efficiency,"  Energy Economics, vol. 33, pp.  608-
    616,2011.
                                                                                              61

-------
Z. Lin and D.L.  Greene, Predicting Individual On-road Fuel Economy Using Simple Consumer and
Vehicle Attributes, SAE Technical Paper Series No. 11SDP-0014, Society  of Automotive Engineers,
Warrendale, PA, April 12, 2011.

and P.R.  Boudreaux, D.J. Dean, W. Fulkerson,  A.L. Gaddis, R.L. Graham,  R.L. Graves, J.L. Hopson,
P. Hughes, M.V. Lapsa, I.E. Mason, R.F. Standaert, T.J. Wilbanks and A. Zucker,  "The Importance of
Advancing Technology to America's Energy Goals," Energy Policy, vol. 38, no. 8, pp. 3886-3890, March
2010.

Rubin, J., P.N.  Leiby  and D.L.  Greene, "Tradable Fuel Economy Credits: Competition and Oligopoly,"
Journal of Environmental Economics and Management, vol. 58, no. 3, pp. 315-328, 2009.

"Measuring Energy Security: Can the United States Achieve Oil Independence?" Energy Policy, 2009, Vol.
38, No. 4, pp. 1614-1621.

"Feebates, Footprints and Highway Safely," Transportation Research Part D, vol. 14, pp. 375-384, 2009.


"Vehicles and E85  Stations  Needed to Achieve to Achieve Ethanol  Goals," Transportation Research
Record No.  2058,-pp. 172-178.

and P.N.  Leiby, P.D. Patterson, S.E. Plotkin and M. Sing, "Oil Independence: Achievable National Goal or
Empty Slogan?" Transportation  Research Record, No. 2017, pp. 47-53, Washington, DC, 2007.

and J.L. Hopson, R. Goeltz and  J. Li, "Analysis of In-Use Fuel Economy Shortfall Based on Voluntarily
Reported Mile-per-Gallon Estimates," Transportation Research Record, No. 1983, pp.  99-105, 2007.

Leiby, P.N., D.L.  Greene, D. Bowman and E. Tworek, "Systems Analysis of Hydrogen Transition with
HyTrans," Transportation Research Record, No. 1983, pp.  129-139, 2007.

and J.L.  Hopson and  J. Li, "Have We  Run Out of Oil Yet? Oil Peaking Analysis from an Optimist's
Perspective," Energy Policy, vol. 34, pp. 515-531, 2006.

S. Ahmad and  D.L. Greene, "The Effect  of Fuel  Economy on Automobile  Safely: A Reexamination,"
Transportation Research Record No. 1941, pp. 1-7, Washington, DC, January 2005.

and J.L.Hopson and J. Li, "Running Out of and Into Oil:  Analyzing Global Depletion and Transition
Through  2050,  Transportation  Research Record  1880,  pp.  1-9,  Transportation  Research  Board,
Washington, DC, 2005.

and P.D.  Patterson, M. Singh and J. Li, "Feebates, Rebates and Gas-Guzzler Taxes: A Study of Incentives
for Increased Fuel Economy," Energy Policy, vol. 33, no. 6, pp. 721-827, 2005.

Sheffield, J., et  al., "Energy Options for the Future," Journal of Fusion Energy, vol. 23, no. 2, pp. 63-109,
2004.

and J. Hopson, "An Analysis  of Alternative Forms of Automotive Fuel Economy Standards for the United
States,"   Transportation Research Record No.   1842,  pp.  20-28,  Transportation  Research  Board,
Washington, DC, 2003.

H.L. Hwang, S.M. Chin and D.L. Greene,  "In,  Out, Within and Through:  Geography of Truck Freight in
the Lower 48," Transportation Research Record, no. 1768, pp. 18-25, Transportation Research Board,
Washington, DC, 2001.

and S.E.  Plotkin, "Energy Futures for the U.S.  Transportation Sector," Energy Policy, vol. 29, no. 14, pp.
1255-1270,2001.
                                                                                            62

-------
and N. Tishchishyna, "The Costs of Oil Dependence: A 2000 Update," Transportation Quarterly, vol. 55,
no. 3, pp. 11-32,2001.

H.L. Hwang, D.L.  Greene, S.M. Chin, J. Hopson and A.A. Gibson, "Real-time Indicators of VKT and
Congestion:  One Year  of Experience,"  Transportation Research  Record,  no.  1719,  pp.  209-214,
Transportation Research Board, Washington, DC, 2000.

and J.M. DeCicco, "Engineering-Economic Analyses of Automotive Fuel Economy Potential in the United
States," Annual Review of Energy and the Environment, vol. 25, pp. 477-536, 2000.

L.A.  Greening, D.L. Greene and  C.  Difiglio, "Energy  Efficiency and  Consumption—The Rebound
Effect—A Survey," Energy Policy, vol. 28, pp. 389-401, 2000.

R.N. Schock, W. Fulkerson, M.L. Brown, R.L. San Martin, D.L. Greene and J. Edmonds, "How Much Is
Energy R&D Worth as Insurance?"  Annual Review of Energy and the Environment, vol. 24, pp. 487-512,
Annual Review, Palo Alto, California, 1999.

S.M. Chin, D.L. Greene, J. Hopson, H.L. Hwang and B. Thompson, "Towards Real-Time Indices of U.S.
Vehicle Travel and  Traffic Congestion," Transportation Research Record, no. 1660, pp. 132-139, National
Academy Press, Washington, DC, 1999.

and J. Kahn and R. Gibson, "Fuel Economy Rebound Effect for U.S. Household Vehicles," The Energy
Journal, vol. 20, no. 3, pp. 1-31, 1999.

"Survey Evidence on the Importance of Fuel Availability to Choice of Alternative Fuels and Vehicles,"
Energy Studies Review, vol. 8, no. 3, pp. 215-231,  1998.

"Why CAFE Worked," Energy Policy, vol. 26, no. 8, pp. 595-614, 1998.

and Donald W. Jones and Paul N. Leiby, "The Outlook for U.S. Oil Dependence," Energy Policy, vol. 26,
no. l,pp. 55-69, 1998.

and Michael Wegener, "Sustainable Transport," in Journal of Transport Geography, vol. 5, no. 3, pp. 177-
190, 1997.

Steven E. Plotkin and David Greene, "Prospects for Improving the Fuel Economy of Light-Duty Vehicles,"
Energy Policy, vol.  25, no. 14-15, pp. 1179-1188, 1997.

"Economic Scarcity: Monopoly, Not Geology, Threatens Global Supply," Harvard International Review,
vol. XIX, no. 3, Summer, 1997.

"Environmental Impacts," Journal of Transport Geography, vol. 5, no.  1, pp. 28-29, 1997.

"Energy for Transportation," Journal of Transport Geography, vol.  5, no. 1, pp. 30-32, 1997.

and Y. Fan,  "Transportation Energy Intensity Trends, 1972-1992," Transportation Research Record, no.
1475, pp. 10-19, Energy and Environment, Transportation Research Board, Washington, DC, 1995.

M.A. Deluchi, D.L. Greene and Quanlu Wang, "Motor Vehicle Fuel Economy: The Forgotten Hydrocarbon
Control Strategy?"  Transportation Research A, vol. 28A, no. 3, pp. 223-244, 1994.

"Transportation and Energy," Transportation Quarterly, vol. 48, no. 1, pp. 91-101, Winter, 1994.

and  K.G.  Duleep,  "Costs  and Benefits of  Automotive Fuel Economy Improvement," Transportation
Research, vol. 27A, no. 3, pp. 217-236, May, 1993.
                                                                                            63

-------
"Transportation and Energy: The Global Environmental Challenge," Transportation Research, vol. 27A,
no. 3, pp. 163-166, May, 1993.

"Energy Efficiency Improvement  Potential  of  Commercial Aircraft," Annual Review  of Energy and
Environment, vol.  17, pp. 537-573,  1992.

"Vehicle Use and Fuel Economy: How Big is the Rebound Effect?" The Energy Journal, vol. 13, no. 1, pp.
117-143, Apnl 1992.

"A Note on  OPEC Market Power and Oil Prices," Energy Economics, vol. 13, no. 2, pp. 123-129, April
1991.

"The Cost of Short-Run Pricing Strategies to Increase Corporate Average Fuel Economy," Economic
Inquiry, vol.  XXIX, no. 1, pp. 101-114, January 1991.

"Fuel Choice for Multifuel Vehicles," Contemporary Policy Issues, vol. VIII, no. 4, pp. 118-137, October
1990.

"CAFE or PRICE?  An Analysis of the Effects of Federal Fuel Economy Regulations and Gasoline Price
on New Car MPG, 1978-89," The Energy Journal, vol. 11, no. 3, pp. 37-57, September 1990.

"Technology and Fuel Efficiency," Forum for Applied Research and Public Policy, vol. 5, no. 1, pp. 23-
29, University of Tennessee, Spring 1990.

Carmen Difiglio,  K.G.  Duleep  and  D.L.  Greene, "Cost  Effectiveness of Future  Fuel Economy
Improvements," The Energy Journal, vol. 11, no.  1, 1990.

"Short-Term Options for Controlling CO2 Emissions of Light-Duty Vehicles," SAE Technical Paper Series
901111, Society of Automotive Engineers, 1990.

"Motor Fuel  Choice: An Econometric Analysis,"  Transportation Research A, vol. 23A, no. 3, pp. 243-253,
1989.

"Fuel Choice for Dual-Fuel Vehicles:  An Analysis of the  Canadian Natural Gas Vehicles Survey," SAE
Technical Paper Series 892067, Society of Automotive Engineers, Warrendale, Pennsylvania, 1989.

J. J. Erickson, D.L. Greene and A. J. Sabadell, "An Analysis of Transportation Energy Conservation Projects
in Developing Countries," Transportation, vol. 15, no. 3, pp. 163-189, 1988.

and  J.T. Liu,  "Automotive Fuel  Economy Improvements  and Consumers' Surplus," Transportation
Research A, vol. 22A, no. 3, pp. 203-218, 1988.

"Advances in Automobile Technology and the Market for Fuel Efficiency,  1978-1985," Transportation
Research  Record  1155,  pp.  18-27, Transportation Research  Board, National  Research Council,
Washington, DC, 1987.

and Anthony Araya Jacome, Robert Kowalski and Patricia S. Hu, "Road Transport Energy Conservation in
Costa Rica," Energy, vol. 12, no. 12, pp. 1299-1308, 1987.

and P.S. Hu, "A Functional Form Analysis of the Short-Run Demand for Travel and Gasoline by One-
Vehicle Households," Transportation  Research Record, no. 1092, pp. 10-15, Transportation Research
Board,  Washington, DC, 1986.

and N.  Meddeb and J.T. Liu,  "Vehicle Stock  Modeling  of Highway Energy Use: Tunisian and  U.S.
Applications," Energy Policy, pp. 437-446, October 1986.
                                                                                           64

-------
"Efficiency-Related Changes in Automobile and Light Truck Markets," SAE Technical Paper Series, no.
861423, Society of Automotive Engineers, Warrendale, Pennsylvania, September 1986.

"The Market Share of Diesel Cars in the U.S., 1979-83," Energy Economics, vol.  8, no.  1, pp.  13-21,
January 1986.

and P.S. Hu and  L. Till, "An Analysis of Trends in Automotive Fuel Economy from 1978 to  1984,"
Transportation Research Record, no.  1049, pp. 51-56, Washington, DC, 1985.

"Estimating  Daily Vehicle  Usage  Distributions  and the  Implications  for  Limited-Range Vehicles,"
Transportation Research B, vol. 19B, no. 4, pp.  347-358, 1985.

and P.S. Hu, "Vehicle Usage in  Multi-Vehicle Households and the Price of Gasoline," Transportation
Research Record, no. 988, pp. 19-24, Washington, DC, 1984.

and P.S. Hu and  G.F. Roberts,  "An  Analysis of  Geographical  and Temporal  Variation in  Vehicles
Classification Count Statistics," Transportation Research Record, no. 987, pp. 21-28, Washington, DC,
1984.

and G.F.  Roberts, "A Comment on  Fuel  Consumption  for Road  Transport in the U.S.A," Energy
Economics, vol. 6, no. 2, pp. 145-147, April 1984.

"A Derived Demand Model of Regional Highway Diesel Fuel Use," Transportation Research B, vol. 18B,
no. 1, pp. 43-61, 1984.

P.D. Patterson, F.W. Westbrook, D.L. Greene and G.F. Roberts, "Reasons for Changes in MPG Estimates,
Model Year  1978 to the Present,"  SAE Technical  Paper  Series, no. 840500,  Society of Automotive
Engineers, Warrendale, Pennsylvania, February/March, selected for inclusion in 1984 SAE Transactions,
1984.

J.  Soderstrom, E. Hirst, D. Greene and J. Trimble, "Have Department of Energy Conservation Programs
Saved Energy?" Evaluation Review, vol. 8, no.  l,pp. 93-112, February 1984.

E. Hirst, R.  Marlay, D.  Greene and R. Barnes, "Recent Changes in U.S. Energy  Consumption: What
Happened and Why?"  Annual Review  of Energy, vol. 8, pp. 193-243, Annual Reviews, Inc., Palo Alto,
Calrforma, 1983.

"Streamlining the Collection  and Processing of Traffic  Count Statistics, A Comment," Transportation
Research Record, no. 928, pp. 18-19, 1983.

"A Note on  Implicit Consumer Discounting of Automobile  Fuel Economy:  Reviewing the Available
Evidence," Transportation Research, vol. 17B, no. 6, pp. 491^-99, 1983.

and C.K.  Chen, "A Time Series Analysis of State Gasoline Demand,  1975-1980," The  Professional
Geographer, vol. 35, no. 1, pp 40-51, February  1983.

G.F. Roberts and D.L.  Greene, "A Method for Assessing the Market Potential of  New Energy-Saving
Technologies,"  Transactions on Systems, Man, and  Cybernetics, Institute of Electrical and Electronics
Engineers, vol. SMC-13, no. 1, pp. 3-7, January/February 1983.

Eric Hirst et al, "Effects of Improved Energy Efficiency on U.S. Energy Use:  1973-1980," Energy, vol. 7,
no. 11, pp. 897-907, 1982.

and G. Kulp, "An Analysis  of the  1979-1980 Decline in gasoline Consumption in the United States,"
Energy, vol.  7, no. 4, pp. 367-375, 1982.
                                                                                            65

-------
        and G. Walton, "Data and Methodological Problems in Establishing State Gasoline Conservation Targets,"
        Transportation Research Record, no. 815, pp. 24-30, 1981.

        and E.  Chen, "Scrappage and Survival Rates of  Passenger Cars and Trucks  in the U.S.,  1966-77,"
        Transportation Research, vol. ISA, no. 5, pp. 383-389, 1981.

        "City Size Distribution and Income Distribution in Space: A Comment," Regional Development Dialogue,
        vol. II, no. 1, pp.  124-126, Spring 1981.

        "A State Level Stock System Model of Gasoline Demand," Transportation Research Record, no. 801, pp.
        44-50, 1981.

        "Estimated Speed/Fuel Consumption Relationships for a Large Sample of Cars," Energy, vol. 6, pp. 441-
        446, 1981.

        "The Spatial Dimension of Gasoline Demand," Geographical Survey, vol. 9, no. 2, pp. 19-28, April 1980.

        "Regional Demand for Gasoline: Comment," Journal of Regional Science, vol.  20, no. 1, pp. 103-109,
        1980.

        "Urban Subcenters: Recent Trends in Urban Spatial Structure," Growth and Change, vol. 11, no. 1, pp.
        103-109, January 1980.

        R.  Dubin, D.L.  Greene and  C. Begovich, "Multivariate  Classification  of Automobiles  Using an
        Automobile's Characteristics' DataBase," Transportation Research Record, no. 726, pp. 29-27,  1979.

        "State Differences in the Demand for Gasoline:  An Econometric Analysis," Energy Systems and Policy,
        vol. 3, no. 2, pp. 191-212, 1978.

        and Joern Barnbrock, "A Note  on Problems in Estimating  Urban Density Models," Journal of Urban
        Economics, vol. 5, April  1978.

        and Rolf R. Schmitt, "An Alternative Derivation of the  Intervening Opportunities Model," Geographical
        Analysis, vol. 10, no. 1, January 1978.

        Joem Barnbrock and D.L. Greene, "A Comment on Population Density and Trend Surface Analysis," Land
        Economics, vol. 53, no. 2, May 1977.

CONTRIBUTIONS TO NATIONAL RESEARCH COUNCIL REPORTS	

        "Assessment of Fuel Economy Technologies for Light-Duty Vehicles," Report  of the Committee  on the
        assessment of Technologies for Improving Light-duty Vehicle Fuel Economy, National Academies Press,
        Washington, D.C., 2010.

        J. Zucchetto, Trends in Oil Supply and Demand, Potential for Peaking of Conventional Oil Production, and
        Possible Mitigation  Options, a summary report  of the Modeling the  Oil Transition workshop, Member,
        Planning Group and Keynote Speaker, Washington, DC, April 2006.

        "State and Federal Standards for Mobile Source Emissions," Report of the Committee on State Practices in
        Setting  Mobile Source  Emissions Standards, National Research Council,  National Academies Press,
        Washington, DC, March 2006.

        "Integrating  Sustainability into  the Transportation Planning  Process," Conference  Proceedings  37,
        Transportation Research  Board of the National Academies, Washington, DC, 2005.
                                                                                                  66

-------
        "Effectiveness  and Impact of Corporate Average Fuel Economy (CAFE) Standards," Report of the
        Committee, National Research Council, National Academy Press, Washington, 2002.

        "Ecological, Environmental and Energy-Related Issues, in The Future Highway Transportation System and
        Society, Transportation Research Board, National Research Council, National Academy Press, Washington,
        DC, 1997.

        Expanding Metropolitan Highways:  Implications for Air Quality  and Energy Use,  Special  Report 245,
        Transportation Research Board, National Research Council, Washington, DC, July 1995.

        Automotive Fuel Economy: How Far Can We Go?  Report of the Committee on Automobile  and Light
        Truck Fuel Economy, National Research Council, National Academy Press, Washington, DC, 1992.

        In Pursuit of Speed:  New Options for Intercity Passenger Transport, Special Report 233, Transportation
        Research Board, National Research Council, Washington, DC, 1991.

        and D. Sperling and B. McNutt, "Transportation Energy to the Year 2020," pp. 207-231, in A Look Ahead:
        Year 2020, Special Report 220, Transportation Research Board, National Research Council, Washington,
        DC, 1988.

CONGRESSIONAL TESTIMONY	

        "Technology-Neutral Incentives for Energy Efficient Low Greenhouse Gas Emitting Vehicles," Testimony
        to the Finance Committee, United States Senate, April  23, 2009.

        "Near-Term  Options to  Increase  Fuel Economy  and  Reduce Petroleum  Demand," Testimony to the
        Committee on Energy and Natural Resources, United States Senate, July 23, 2008.

        "Facing  the Challenges of Oil Dependence and Climate Change: What Will It Take?" Testimony to the
        Subcommittee  on  Energy  and  Water Development, U.S.  House of  Representatives  Committee  on
        Appropriations, February  14, 2008.

        "Policies to Increase  Passenger Car and Light Truck Fuel Economy," Testimony to  the  U.S. Senate
        Committee on Energy and Natural Resources, January  30, 2007.

        "Is Cap-and-Trade a  Sufficient  Carbon  Policy for  Transportation?"   Testimony  to the U.S. Senate
        Committee on Environment and Public Works, November 13, 2007,  Washington, DC.

        "Energy  Challenges for  Transportation  in  the 21st  Century,"  Testimony to the National  Surface
        Transportation Policy and Revenue Study Commission, March 19, 2007, Washington, DC.

        "Corporate  Average Fuel  Economy (CAFE)  Standards,"  Testimony to  the U.S. Senate Commerce
        Committee, March 6, 2007, Washington, DC.

        "Policies to Increase Passenger Car and Light  Truck Fuel  Economy,"  Testimony to  the U.S. Senate
        Committee on Energy and Natural Resources, January  30, 2007, Washington,  DC .

        "Observations  on the H-Prize Act of 2006 (H.R. 5143)," Testimony to the U.S. House of Representatives
        Committee on Science, April 27, 2006, Washington, DC.

        "Improving the Nation's Energy Security: Can Cars and Trucks be Made More Fuel Efficient?" Testimony
        to  the U.S. House of Representatives Committee on Science, February  9,  2005, Serial No. 109-3,
        U.S.G.P.O, Washington, DC.
                                                                                                   67

-------
        "The  Outlook for  Surface  Transportation  Growth,"  Testimony to  the  Subcommittee  on  Surface
        Transportation of the Committee on  Transportation Infrastructure of  the  United  States House  of
        Representatives, March 28, 1996.

CONTRIBUTIONS TO BOOKS	

        "Measuring Energy Sustainability: Making Progress," in Linkages of Sustainability, T.E. Graedel and E.
        van der Voet, eds., Strungmann Forum Reports, MIT Press, Cambridge, MA, 2010.

        Loschel, A., J. Johnston, M.A. Delucchi, T.N. Demayo, D.L. Gauthier, D.L. Greene, J.  Ogden, S. Rayner
        and E. Worrell, "Energy: Stocks, Flows, and Prospects," in Linkages of Sustainability, T.E. Graedel and E.
        van der Voet, eds., Strungmann Forum Reports, MIT Press, Cambridge, MA, 2010.

        D.L.  Greene,  "What Will Replace Liquid Hydrocarbon Fuels?" in M.J. Benton, ed., The Seventy Great
        Mysteries of the Natural World, Thames  and Hudson, Inc., New York, 2008.

        and J. German and M. A. Delucchi, "Fuel Economy:  The Case for Market Failure," in  Reducing Climate
        Impacts of the Transportation Sector, D.  Sperling and J.S. Cannon, eds., Springer Science+Business Media,
        springer.com,  2008.

        Ribeiro,  S.K., S. Kobayashi, D.L.  Greene et al., "Transport and its Infrastructure," Chapter 5 in Climate
        Change 2007: Mitigation of Climate Change, Working Group III contribution to the Fourth Assessment
        Report of the Intergovernmental Panel on Climate Change, Cambridge Press, Cambridge, United Kingdom.

        A. Dumas, D.L. Greene and A. Bourbeau, "North American Feebate Analysis Model," in Driving Climate
        Change, D. Sperling and J.S.  Cannon, eds., Academic Press, San Francisco, CA, 2007.

        "Transportation and Energy," in The Geography of Urban Transportation, S. Hanson and G. Giuliano, eds.,
        The Guilford Press, New York, 2004.

        "Transportation and  Energy  Overview," in Encyclopedia of Energy,  Cutler J. Cleveland, ed.,  Elsevier,
        Oxford, United Kingdom, 2004.

        M. Wegener and D.L. Greene, "Sustainable Transport" in Social Change and Sustainable Transport, pp.
        35-42, W.R. Black and P. Nijkamp, eds., Indiana University  Press, Bloomington, IN, 2002.

        "Sustainable Transportation," in International Encyclopedia of the Social and Behavioral Sciences,  N.J.
        Smelser and P.B. Baltes, eds., Elsevier, Oxford, pp. 15,335-15,339, 2002.

        W.R. Moomaw, J.R. Moreira, K.  Blok, D.L.  Greene et al., "Technological and Economic Potential of
        Greenhouse Gas Emissions Reduction," Chapter 3  in Climate Change 2001:  Mitigation, contribution of
        Working  Group III to the Third Assessment Report of the Intergovernmental Panel on Climate Change,
        Cambridge University Press, Cambridge, UK, 2001.

        "Transportation's Oil Dependence and Energy  Security in the 21st Century," in Environmental Change,
        Adaptation,  and Security, S.C. Lonergan, ed., NATO Advanced Science Institute Series, Klumer Academic
        Publishers, Boston, 1999.

        S.L. Baughcum, J.J.  Begin, F. Franco, D.L. Greene et al., "Aircraft Emissions: Current  Inventories  and
        Future Scenarios," Chapter 9 in Aviation and Global Atmosphere, Intergovernmental  Panel  on Climate
        Change,  Cambridge University Press, Cambridge, UK, 1999.

        and Robert Gibson, "Transportation, Energy and the Environment," Chapter 4 in Transportation  Statistics
        Annual Report 1998, BTS98-S-01, Bureau of Transportation Statistics,  U.S. Department of Transportation,
        Washington, DC, September  1998.
                                                                                                   68

-------
"Commercial Air Transport Energy Use and Emissions: Is Technology Enough?" pp. 207-228 in DeCicco
and Delucchi, eds., Transportation, Energy,  and the Environment: How Far Can Technology Take Us?
American Council for an Energy Efficient Economy, Washington, DC, 1997.

B. McNutt, L. Fulton and D.L.  Greene, "Is Technology Enough?  A Synthesis of Views Expressed at the
Conference," pp. 251-256 in DeCicco and Delucchi, eds., Transportation, Energy,  and Environment: How
Far Can Technology Take Us?  American Council for  an Energy Efficient Economy, Washington, DC,
1997.

and Robert Gibson, "Transportation, Energy,  and the Environment," Chapter 4 in Transportation Statistics
Annual Report 1997, BTS97-5-01, Bureau of Transportation Statistics, U.S. Department of Transportation,
Washington, DC, October 1997.

and Donald W. Jones, "The Full Costs and Benefits of Transportation:  Conceptual and Theoretical Issues,"
Chapter 1 of The Full Costs and Benefits of Transportation, Springer-Verlag, Heidelberg, 1997.

"The Cost of Transportation's Oil Dependence," in O. Hohmeyer, R.L. Ottinger and K. Rennings, eds.,
Social Costs and Sustainability,  Springer-Verlag, Berlin, 1996.

"Twenty Years of Energy Policy: What Should We Have Learned?" in David L. Feldman, ed., The Energy
Crisis, The Johns Hopkins University Press, Baltimore, 1996.

"Environmental Impacts of Transportation,"  Chapter 6 in Transportation Statistics Annual Report 1996,
Bureau of Transportation Statistics, U.S. Department of Transportation, Washington, DC, October 1996.

R. Gibson and D.L. Greene,  "Environmental Trends and the U.S. Transportation System," Chapter 7 in
Transportation Statistics Annual Report 1996, Bureau of Transportation Statistics, U.S. Department of
Transportation, Washington, DC, October 1996.

"Energy  and Transportation," Chapter 4 in Transportation  Statistics  Annual Report 1996, Bureau  of
Transportation Statistics, U.S. Department of Transportation, Washington, DC, October 1996.

"Transportation," Chapter 5 in Policies and Measures for Reducing Energy Related Greenhouse Emissions,
DOE/PO-0047, U.S. Department of Energy,  Office of Policy and  International  Affairs, Washington, DC,
July 1996.

Guest editor, "Energy and Global Climate Change," Transportation Research, Special Issue, vol. 27A,
no. 3, May  1993.

"Regional Demand Implication for Gasoline Supply Shortages," in  T.R. Lakshmanan and P.  Nijkamp, eds.,
Systems and Models for Energy and Environmental Analysis, pp. 206-233, Gower, UK, 1983.

G. Samuels, A.B. Rose, D.L.  Greene and J.N. Hooker, "Energy Conservation in  Transportation," a chapter
in Vol. Ill of Advances in Energy Systems and Technology, Peter  Auer, ed., Academic Press, New York,
1982.

"A Regional Stock  System Model  of Highway  Gasoline Demand," in Changing Energy Use Futures,
Volume  1,  Proceedings of the  Second International Conference on Energy Use and Management, Los
Angeles, California, October 22-26, 1979.
                                                                                            69

-------
OTHER PUBLICATIONS
        and K.G. Duleep, and G.  Upreti,  "Status and Outlook for the U.S. Non-Automotive Fuel Cell Industry:
        Impacts of Government Policies  and Assessment of Future  Opportunities", ORNL/TM-2011/101,  Oak
        Ridge National Laboratory, Oak Ridge, Tennessee.

        Bunch, D.S. and D.L. Greene, "Potential Design, Implementation, and Benefits of a Feebate Program for
        New Passenger Vehicles  in  California," State  of  California Air Resources Board and  the California
        Environmental    Protection     Agency,    Sacramento,    California,    2011,    available     at
        http://76.12.4.249/artman2/uploads/l/Feebate_Progr am_for_New_Passenger_Vehicles_in_California.pdf.

        "Oil Peak or Panic?" Book review, Science, vol. 328, no. 5980, p. 828, May 14, 2010

        "What's Greener than a VMT Tax? Discussion Paper, Howard H. Baker, Jr. Center for Public Policy, The
        University of Tennessee, Knoxville, TN, May 2010.

        and S.E. Plotkin,  Reducing Greenhouse Gas Emissions from  U.S. Transportation, Pew Center on Global
        Climate Change, Arlington, VA, 2011.

        How Consumers  Value Fuel Economy: A  Literature  Review, EPA-420-R-10-008, U.S. Environmental
        Protection Agency, March 2010.

        "Why the Market for New Passenger Cars Generally Undervalues Fuel Economy," Discussion Paper No.
        2010-6, International Transport Forum, OECD, Paris, January 2010.

        D.  McCollum, G.  Gould and D.  Greene,  Greenhouse  Gas  Emissions from Aviation and Marine
        Transportation: Mitigation Potential and Policies,  Pew Center  on  Global Climate Change, Arlington,
        Virginia, available at http://www.pewclimate.org/technologv/report/aviation-an-marine, December 2009.

        "Energy  Assurance: Essential Energy Technologies for Climate  Protection and  Energy Security,"
        ORNL/TM-2009/314, Oak Ridge National Laboratory, Oak Ridge, TN, December 2009.

        S. Plotkin et &\.,Multipath Transportation Futures Study: Vehicle  Characterization and Scenario Analyses,
        ANL/ESD/09-5, Argonne National Laboratory, Argonne, Illinois, July 2009.

        and K.G. Duleep, "Bootstrapping a Sustainable North American PEM Fuel Cell Industry: Could a Federal
        Acquisition Program Make a Difference?" ORNL/TM-2008/183, October, 2008.

        "Future Prices and Availability of Transport Fuels," International Transport Forum, Research Round Table
        on Oil Dependence: Is Transport Running Out of Affordable Fuel?  November 14-15, 2007, OECD, Paris.

        American  Physical Society Study Group, Energy Future: Think Efficiency,  American Physical Society,
        College Park, Maryland, 2008.

        D.L. Greene et al, Analysis of the Transition to Hydrogen Fuel Cell Vehicles and the Potential Hydrogen
        Infrastructure Requirements, ORNL/TM-2008/30, Oak Ridge National Laboratory, Oak Ridge, Tennessee,
        March, 2008.

        "Future Prices and Availability of Transport Fuels," Discussion Paper No. 2007-15, International Transport
        Forum, Joint Transportation Research Center, OECD, Paris,
        http://www.internationltransport.forum.org/jtre/DiscussionPapers/DiscussionPaperl5.pdf.

        "Transportation," in The First State of the Carbon Cycle Report (SOCCR): The North American Budget
        and Implications for the Global Carbon Cycle. A Report by the U.S. Climate Change Science Program and
        the  Subcommittee on Global Change Research [King, A.W., L. Dilling, G.R. Zimmerman, D.M. Ciarman,
                                                                                                    70

-------
R.A. Houghton, G. Marland, A.Z. Rose, and T.J. Wilbanks (eds)].  National Oceanic and Atmospheric
Administration, National Climate Data Center, Asheville, North Carolina, USA, pp. 73-84.

and P.N. Leiby and D. Bowman, Integrated Analysis of Market Transformation Scenarios with HyTrans,
ORNL/TM-2007/094, Oak Ridge National Laboratory, Oak Ridge, Tennessee, June 2007.

D. Gordon, D.L. Greene, M.H. Ross and T.P. Wenzel, "Sipping Fuel and Saving Lives:  Increasing Fuel
Economy without Sacrificing  Safety," International  Council on  Clean Transportation, Washington, DC,
June 2007.

and P.N. Leiby, "Oil Independence: Realistic Goal or Empty Slogan," published as "Expert Commentary"
on The Lugar Energy Initiative website,
http ://lugar. senate, gov/energy/links/commentary708 greene summary .html, 2007.

Editor, Modeling the Oil Transition: A Summary of the Proceedings of the DOE/EPA Workshop on  the
Economic and Environmental Implications of Global Energy Transitions, ORNL/TM-2007/014, Oak Ridge
National Laboratory, Oak Ridge, Tennessee, February 2007.

and P.N. Leiby, The Oil Security Metrics Model, ORNL/TM-2006/505, Oak Ridge National Laboratory,
Oak Ridge, Tennessee, May 2006.

and S. Ahmad, Costs  of U.S.  Oil Dependence:  2005 Update, ORNL/TM-2005/45, Oak  Ridge National
Laboratory, Oak Ridge, Tennessee, March 2005.

"Climate Change Policy  for Transportation While Waiting for H2," in The 10-50 Solution: Technologies
and Policies for a Low-Carbon Future, Pew Center on Global Climate Change, National  Commission on
Energy Policy, Washington, DC, March 2004.

and K.G. Duleep and W. McManus, Future Potential of Hybrid and Diesel Powertrains in the U.S. Light-
Duty  Vehicle Market, ORNL/TM-2004/181,  Oak Ridge National Laboratory,  Oak  Ridge,  Tennessee,
August 2004.

and J.L. Hopson, Running Out of and Into Oil: Analyzing Global Depletion and Transition Through 2050,
ORNL/TM-2003/259, Oak Ridge National Laboratory, Oak Ridge, Tennessee, October 2003.

and R.C. Gibson and K.G. Duleep, Energy Star Concepts for Highway Vehicles, ORNL/TM-2003/37, Oak
Ridge National Laboratory, Oak Ridge, Tennessee, June 2003.

R. Nye, D.L.  Greene  and J.W. Saulsbury, Providing Consumers with  Web-based Information  on  the
Environmental Effects  of Automobiles, ORNL/TM-2003/166, Oak Ridge National Laboratory, Oak Ridge,
Tennessee, June 2003.

and A. Schafer, "Reducing Greenhouse Gas Emissions from U.S. Transportation," Pew Center on Global
Climate Change, Arlington, Virginia, May 2003.

and S. Plotkin and K.G.  Duleep, Examining the Potential for  Voluntary Fuel Economy Standards in  the
United States  and Canada, ANL/ESD/02-5,  Center for Transportation Research,  Argonne  National
Laboratory, Argonne, Illinois, October 2002.

S.M.  Chin, O. Franzese, D.L. Greene,  H.L.  Hwang and R.C.  Gibson,  Temporary Losses of Highway
Capacity and Impacts  on Performance,  ORNL/TM-2002/3, Oak Ridge National Laboratory, Oak Ridge,
Tennessee, May 2002.

TAFV Alternative Fuels  and Vehicles Choice Model Documentation, ORNL/TM-2001/134,  Oak Ridge
National Laboratory, Oak Ridge, Tennessee, July 2001.
                                                                                            71

-------
R.N. Schock, W. Fulkerson, M.L. Brown, R.L. San Martin, D.L. Greene and J.A. Edmonds, The Insurance
Value  of Energy R&D,  UCEL-ID-141815, CGSR-2001,  003, Center for Global Security Research,
Lawrence Livermore National Laboratory, Livermore, California, May 2001.

"Energy," in Chapter  5  of The  Changing Face  of Transportation, U.S. Department of Transportation,
Bureau of Transportation Statistics, BTSOO-007, Washington, DC, 2000.

with S. Plotkin, "Transportation Sector," Chapter 6 in Scenarios for a Clean Energy Future, Interlaboratory
Working Group  on Energy-Efficient and Clean Energy  Technologies, Oak Ridge  National Laboratory,
ORNL/CON-476, Lawrence Berkeley National Laboratory, LBNL-44029, November 2000.

R.N. Schock, W. Fulkerson, M.L. Brown, R.L. San Martin, D.L. Greene and J. Edmonds, "How Much Is
Energy R&D Worth?"  Energy, vol. 25, no. 2, pp. 4-8, 2000.

Costs of Oil Dependence: A 2000 Update,  ORNL/TM-2000/152,  Oak Ridge National Laboratory, Oak
Ridge, Tennessee, May 2000.

and J.M. DeCicco, Engineering-Economic Analyses of Automotive Fuel Economy Potential in the United
States, ORNL/TM-2000/26, Oak Ridge National Laboratory, Oak Ridge, Tennessee, February 2000.

and J.M. DeCicco, "Energy and Transportation Beyond 2000," Millennium Paper Series, Transportation
Research Board, National Research Council, Washington, DC, January 2000.

and many others, National Transportation Statistics 1999, BTS99-04, U.S. Department  of Transportation,
Bureau of Transportation Statistics, Washington, DC, 1999.

and  many  others, Transportation Statistics Annual Report  1999, BTS99-03,  U.S.  Department  of
Transportation, Bureau of Transportation Statistics, Washington, DC, 1999.

An  Assessment of Energy and Environmental Issues Related to  the  Use of Gas-to-Liquid Fuels  in
Transportation, ORNL/TM-1999/258, Oak Ridge National Laboratory, Oak Ridge, Tennessee, November
1999.

and J.  Kahn and R. Gibson, An  Econometric Analysis of the Elasticity of Vehicle Travel with Respect to
Fuel Cost per Mile Using RTEC Survey Data, ORNL-6950, Oak Ridge National Laboratory, Oak Ridge,
Tennessee, March 1999.

"Why  CAFE  Worked," Transportation  Research  Circular No. 492, Transportation Research Board,
National Research Council, Washington, DC,  1999.

R.N. Schock, W. Fulkerson, M.L. Brown, R.L. San Martin, D.L. Greene and  J. Edmonds, How Much is
Energy R&D  Worth  as  Insurance?  UCRL-JC-131205,  Lawrence Livermore  National  Laboratory,
Livermore, California,  October 1998.

"Transportation and Energy Policy," in Twenty-Five Years of Energy and Environmental Policy, pp. 125-
140, University of Illinois at Chicago, Energy Resources Center, Chicago, Illinois, 1998.

"Ecological, Environmental  and   Energy-Related Issues,"  pp.  127-149  in  The  Future  Highway
Transportation System  and Society, Transportation Research Board, National Research Council, 1997.

and S.  Plotkin, "Transportation Sector," Chapter 5 in Scenarios of U.S. Carbon Reductions, Interlaboratory
Working Group on Energy-Efficient and Low-Carbon Technologies, ORNL/CON-444, report to the Office
of Energy Efficiency and Renewable Energy, U.S. Department of Energy,  Washington, DC, September
1997.
                                                                                            72

-------
and many others, Energy Technology R&D:   What Could Make a Difference, ORNL-6921/V2-P1, Oak
Ridge National Laboratory, Oak Ridge, Tennessee, September 1997.

"Energy and Environmental Consequences of Transportation: Indicators of Sustainability," 51st Session of
the International Statistical Institute Proceedings, Istanbul, Turkey, August 18-26, 1997.

"Oil Dependence:  The Value of R&D," 32"  Intersociety Energy Conversion Engineering Conference
Proceedings, volume 3, pp. 2148-2153, Honolulu, Hawaii, July 28-30,  1997.

and Steve Plotkin and K.G. Duleep, "The Potential for Energy-Efficient Technologies to Reduce Carbon
Emissions in the United States: Transport Sector," 32nd Intersociety  Energy Conversion Engineering
Conference Proceedings, volume 3, pp. 2114-2119, Honolulu, Hawaii, July 28-30, 1997.

"Grin and Obey," Energy, vol. XXII, p. 6, April 1997.

"Consequences of U.S. Oil Dependence," Energy, vol. XXI, no. 5, pp. 3-8, November 1996.

D. Streets and many others, "Inventory of Technologies, Methods, and Practices for Reducing Emissions of
Greenhouse Gases," technical appendix to Climate Change 1995:  Impacts, Adaptations, and Mitigation of
Climate Change:  Scientific-Technical Analyses, Argonne National Laboratory, Argonne, Illinois, May
1996.

one of Principal Lead Authors, "Industry, Energy and Transportation:  Impacts and Adaptation," in Climate
Change 1995: Impacts, Adaptations and Mitigation  of Climate Change: Scientific-Technical Analyses, R.T.
Watson, M.C. Zinyowera, R.H. Moss and D.J. Dokken, eds., Intergovernmental Panel on Climate Change,
Cambridge University Press,  1996.

one of Contributing Authors, "Mitigation Options for the Transportation Sector," in Climate Change 1995:
Impacts, Adaptations and Mitigation of Climate Change: Scientific-Technical Analyses, R.T. Watson, M.C.
Zinyowera, R.H. Moss and D.J. Dokken, eds., Intergovernmental Panel on  Climate  Change,  Cambridge
University Press, 1996.

"Commercial Air  Transport Energy Use and Emissions:   Is Technology Enough?" in  Sustainable
Transportation:  Is Technology Enough?  Proceedings of the 1995 Asilomar Conference on Transportation
and Energy, American Council for an Energy-Efficient Economy, Washington, DC, 1995.

"Transportation,  Energy  and  Environment:    Global  Policy  Issues for the  Early 21st Century,"  in
International Workshop on  Motor  Vehicles  and  Global Environmental Problems,  Japan  Automotive
Research Institute, Tokyo, Japan, October-November 1995.

P.N. Leiby, D.L. Greene and Harry Vidas, Market  Potential and Impacts of Alternative Fuel use in Light
Duty  Vehicles: A 2000/2010 Analysis, DOE/PO-0042,  Office of Policy  Analysis, U.S. Department  of
Energy, Washington, DC, January 1996.

and X. Han, "The Unintended Consequences of Transportation," Chapter 3  in Transportation Statistics
Annual Report 1995, pp.  43-86, Bureau of Transportation Statistics, U.S. Department of Transportation,
Washington, DC, 1995.

with D.W. Jones and P.N. Leiby, The Outlook of U.S. Oil Dependence, ORNL-6783,  Oak Ridge National
Laboratory, Oak  Ridge, Tennessee, May 1995.

with S.M. Chin  and R. Gibson, Aggregate Vehicle Travel Forecasting Model, ORNL-6872,  Oak Ridge
National Laboratory, Oak Ridge, Tennessee, May 1995.
                                                                                             73

-------
D. Belzer, J. Roop, R. Sands and D.L. Greene, Energy Conservation Trends, Energy Efficiency in the U.S.
Economy, Report Three, DOE/PO-0034, U.S.  Department of Energy, Office of Energy  Efficiency  and
Alternative Fuels Policy, Washington, DC, April 1995.

Transportation Energy Efficiency Trends, 1972-1992, ORNL-6828, Oak Ridge National Laboratory, Oak
Ridge, Tennessee, December 1994.

Alternative Fuels and Vehicles Choice Model, ORNL/TM-12738, Oak Ridge National Laboratory, Oak
Ridge, Tennessee, October 1994.

and E. Hillsman, L.  Fulton, A.K. Wolfe and J. Niles, Energy Emissions and Social Consequences of
Telecommuting, DOE/PO-0026, Technical Report One, Energy  Efficiency in the U.S. Economy Series,
U.S. Department of Energy, Office of Policy, Planning and Program Evaluation, Washington, DC, June
1994.

R.R.  Schmitt, D.L. Greene et al., Transportation Statistics Annual Report, Bureau of Transportation
Statistics, U.S. Department of Transportation, Washington, DC, 1994.

and Paul N. Leiby, The Social Costs to the U.S. of Monopolization  of the World Oil Market,  1972-1991,
ORNL-6744, Oak Ridge National Laboratory, Oak Ridge, Tennessee, March 1993.

Impacts on Home Heating Costs of Incentives for Alternative Fuel Vehicles, Technical Report Eight, U.S.
Department of Energy, Office of Domestic and International Energy Policy, Washington, DC, May 1992.

with K.G. Duleep, Costs  and Benefits of Automotive Fuel Economy Improvement:   A Partial Analysis,
ORNL-6704, Oak Ridge National Laboratory, Oak Ridge, Tennessee, March 1992.

M.A.  Deluchi, Q. Wang and D.L. Greene, Motor Vehicle Fuel Economy:  The Forgotten Hydrocarbon
Control Strategy? ORNL-6715, Oak Ridge National Laboratory, Oak Ridge, Tennessee, July 1992.

"Transportation Energy Policy:  Back to the Past or Ahead to the Future?" in Twenty Years of Energy
Policy:  Looking Toward the Twenty-First Century, Proceedings of the Twentieth Annual Illinois Energy
Conference, Energy Resources Center, University of Illinois at Chicago, Chicago, Illinois, November 23-
24, 1992.

with M.  Singh, E. Ecklund, R. Bechtold  and  C.  Saricks,  "Second  Interim Report of  the  Interagency
Commission  on  Alternative Motor Fuels," Office of Energy Demand, Office  of Policy, Planning  and
Analysis, Washington, DC, 1992.
Q. Wang, D.L. Greene and M.A. Deluchi, "Effects of Increasing Fuel Economy on Gasoline Vehicle HC
Emissions," Proceedings of the 84th Annual  Meeting
Association, Vancouver, BC, Canada, June 16-21, 1991.
Emissions,"  Proceedings of the 84th  Annual Meeting and  Exhibition,  Air and  Waste Management
"Coverage and Quality Problems with Existing Data Resources for Freight Transportation," Proceedings of
the Special Conference on Freight Transportation Data:  The Changing Federal Role Since Deregulation,
Transportation Research Circular, No. 367, Transportation Research Board, Washington, DC, 1990.

and M. Singh, First Interim Report on the Interagency Commission on Alternative Motor Fuels, Office of
Energy Demand Policy, Office of Policy, Planning and Analysis, U.S. Department of Energy, Washington,
DC, September 30, 1990.

"Commercial Aircraft Fuel Efficiency  Potential  Through 2010," Proceedings 1990 Intersociety Energy
Conversion Engineering Conference, Reno, Nevada,  published  by the American  Institute of Chemical
Engineers, August 12-17, 1990.
                                                                                            74

-------
Energy  Efficiency  Improvement Potential  of Commercial Aircraft to  2010,  ORNL-6622,  Oak Ridge
National Laboratory, Oak Ridge, Tennessee, June 1990.

and Anju Rathi, Alternative Motor Fuel  Use Model:  Model Theory  and Design,  and User's Guide,
ORNL/TM-11448,  Oak Ridge National Laboratory, Oak Ridge, Tennessee, April 1990.

with Donald Igo,  "Energy  Dependence," Chapter 3 in National Strategic Transportation Study,  U.S.
Department of Transportation, Washington, DC, 1989.

"A Context for Estimating Economic and Energy Security Benefits," Technical Report Two, Assessment of
Costs and Benefits  of Flexible and Alternative Fuel Use in the  U.S. Transportation Sector, DOE/PE-0091,
Office of Policy, Planning and Analysis, U.S. Department of Energy, Washington, DC,  September 1989.

and J.C. Liang, Modeling Operating Weight and Axle Weight Distributions for Highway Vehicles, ORNL-
6437, Oak Ridge National Laboratory, Oak Ridge, Tennessee, July 1988.

et al., "Research Priorities  in Transportation and Energy," Transportation Research Circular, no.  323,
Transportation Research Board, Washington, DC, September 1987.

Simulating the Market for Automotive Fuel Efficiency: The SHRSIM Model, ORNL/TM-10074, Oak Ridge
National Laboratory, Oak Ridge, Tennessee, February 1987.

and M.C. Holcomb, Off-Highway Use of Gasoline in the United States, U.S. Department of Transportation,
Federal  Highway Administration, Office of Highway Information Management,  Washington, DC, July
1986.

RUMS, A PC-Based FORTRAN Program for Estimating Consumer Surplus  Changes Using Multinomial
Logit and Hedonic Demand Models, ORNL/TM-10069, Oak Ridge National  Laboratory, Oak Ridge,
Tennessee, August  1986.

Driver Energy Conservation Awareness  Training: Review and Recommendations, ORNL/TM-9897, Oak
Ridge National Laboratory, Oak Ridge, Tennessee, May 1986.

R.L. Graves, D.L.  Greene,  E.W. Gregory,  II, Application  of the Adiabatic Diesel to Heavy Trucks:  A
Technology Assessment, ORNL/TM-9554, Oak Ridge National Laboratory, Oak Ridge, Tennessee, March
1986.

and R. Kowalski  and  F.  Southworth, The Transportation Sector in Costa  Rica and Opportunities for
Energy Conservation, a report of the Energy Conservation Services Program, U.S. Agency for International
Development, Office of Energy, Washington, DC, May 1985.

P.S. Hu, D.L. Greene and L.E.  Till, Motor Vehicle MPG and Market Shares Report:  First Six Months of
Model Year 1984, ORNL/TM-9391, Oak Ridge National Laboratory, Oak Ridge, Tennessee, October 1984.

and P.S. Hu  and A.B. Rose,  Transportation Energy Use and Efficiency in Tunisia, ORNL-6066, Oak Ridge
National Laboratory, Oak Ridge, Tennessee, August 1984.

"Highway Fuel Use:   Trends  and  Factors," Proceedings of the Energy  Information  Administration
Symposium on Petroleum Information, DOE/EIA-0425, Energy Information Administration, Washington,
DC, September 1983.

G.F. Roberts and D.L.  Greene,  Trends in Heavy Truck Energy Use and Efficiency, ORNL/TM-8843, Oak
Ridge National Laboratory, Oak Ridge, Tennessee, October 1983.
                                                                                            75

-------
et al., An Overview of the Wharton School Assessments of the Short-Term Integrated Forecasting System,
DOE/NBB-0046, Energy  Information Administration,  Office of Statistical  Standards,  Washington, DC,
July 1983.

"Regional Demand Implications for Gasoline Supply Shortages," in T.R. Lakshmanan and P. Nijkamp,
eds., Systems and Models for Energy and Environmental Analysis, Gower, UK, pp. 206-233, 1983.

and  E. Hirst, J.  Soderstrom and J. Trimble, Estimating the Total Impact on Energy Consumption of
Department of Energy Conservation Programs, ORNL-5925, Oak Ridge National Laboratory, Oak Ridge,
Tennessee, November 1982.

"Gasoline use in the United States," Petroleum Supply Monthly, DOE/EIA-0109 (82-05), U.S. Department
of Energy, Washington, DC, 1982.

and G. Kulp, G.H. Walton and D.B. Shonka, Transportation Energy  Use  1973-80:  Changes,  Trends, and
Causes, ORNL/TM-7953, Oak Ridge National Laboratory, Oak Ridge, Tennessee, April 1982.

G. Samuels, A.B. Rose, D.L. Greene and J.N. Hooker, "Energy Conservation in Transportation," a chapter
in Vol. Ill of Advances in Energy Systems and Technology, Peter Auer, ed., Academic  Press, New York,
1982.

E. Hirst,  D.L. Greene et al., Energy Use from  1973 to 1980:  The Role of Improved Energy Efficiency,
ORNL/CON-79, Oak Ridge National Laboratory, Oak Ridge, Tennessee, December 1981.

The Aggregate Demand for Gasoline and Highway Passenger Vehicles in the United States: A Review of
the Literature, 1938-1978,  ORNL-5728, Oak  Ridge National Laboratory,  Oak Ridge, Tennessee, July
1981.

J.N.  Hooker,  A.B.  Rose and D.L. Greene, End  Use Energy Consumption  Data Base:  Transportation
Sector, DOE/EIA/CR-7405-01, U.S. Department of Energy, Washington, DC, February 1980.

A Statistical Analysis of State VMT Estimates in VMT Statistics, Lifetime VMT,  and Current State Methods
of Estimating VMT, ORNL/TM-6327, Oak Ridge National Laboratory, Oak Ridge, Tennessee, February
1979.

G. Kulp,  D.L. Greene et al., Regional Analyses of Highway Energy Use, ORNL-5587, Oak Ridge National
Laboratory, Oak Ridge, Tennessee, December 1979.

"A Regional  Stock  System Model of Highway Gasoline Demand," in  Changing Energy Use Futures,
Volume 1, Proceedings of the Second International Conference on Energy Use Management, Los Angeles,
California, October 22-26, 1979.

and  T.P.  O'Conner, P.D. Patterson,  A.B.  Rose and D.B.  Shonka, Regional Transportation Energy
Conservation Data Book, ORNL-5435, Oak Ridge National Laboratory, Oak Ridge, Tennessee, September
1978.

Econometric Analysis of the Demand for Gasoline at the State Level, ORNL/TM-6326, Oak Ridge National
Laboratory, Oak Ridge, Tennessee, July 1978.

An Investigation of the  Variability of Gasoline  Consumption  among States, ORNL-5391, Oak  Ridge
National Laboratory, Oak Ridge, Tennessee, April 1978.

R.R. Schmitt and D.L. Greene, "Evaluating Transportation Innovations with the Intervening Opportunities
Model," Proceedings of the Northeast American Institute for Decision Sciences, April 1977.
                                                                                            76

-------
        M.G. Wolman, D.L.  Greene and N.L. Froomer, An Analysis  of the Physical Characteristics of the
        Shanandoah River Which Affects Its Navigability and An Analysis  of the Physical Characteristics of the
        Potomac River Which Affects Its Navigability, prepared for the Baltimore District, U.S. Army Corps of
        Engineers, Chesapeake Research Consortium, pub. nos. 37S and 38S, resp., March 1975.

        G. Power, M.G.  Wolman et al., Report on  the Shenandoah River:   An Investigation  to  Determine
        Navigability and Report on the Potomac River:  An Investigation to Determine Navigability, prepared for
        the Baltimore District, U.S. Army Corps of Engineers, Chesapeake Research Consortium, pub. nos. 37 and
        38, resp., March 1975.

FORTHCOMING PUBLICATIONS	


        "Rebound 2007: Analysis of National Light-Duty Vehicle Travel Statistics," article in press, Energy Policy.

        C. Liu and D.L. Greene, "Impacts of Feebates  in Combination with Fuel Economy  and Emissions
        Standards on  U.S. Light-Duty  Vehicles  Fuel  Use  and  Greenhouse  Gas  Emissions,"  forthcoming,
        Transportation Research Record.

        Z. Lin  and  D.L.  Greene,  "Significance of Daily VMT Variation over Time  and Among Drivers  on
        Assessment of PHEV Energy Impact," forthcoming, Transportation Research Record.

        Z. Lin and D.L. Greene, "Promoting the Market for Plug-in Hybrid and Battery Electric Vehicles: The Role
        of Recharge Availability," forthcoming, Transportation Research Record.
                                                                                                  77

-------
                                   KARA M. KOCKELMAN

                               Professor & William J. Murray Jr. Fellow
       Department of Civil, Architectural & Environmental Engineering - University of Texas at Austin
                        Note: Full CV available at www.ce.utexas.edu/prof/kockelman
EDUCATION
University of California at Berkeley    BSCE 1991, MSCE 1996, PhD 1998, MCP 1996

PROFESSIONAL EXPERIENCE - Academic
 University of Texas at Austin      Assistant Professor of Civil Engineering      1998-2004
 University of Texas at Austin      Associate Professor of Civil Engineering     2004-2009
 University of Texas at Austin      Professor of Civil Engineering               2009 - Present

Licensed Professional Engineer, California (Certification #C057380) and Texas (Certification #93443)
Licensed City Planner, AICP 2008 & Member of American Planning Assoc. National & Texas Chapters,
    2008

SELECTED RELEVANT REFEREED PUBLICATIONS
1.   Musti, S., K. Kortum and K. Kockelman. "Household Energy Use and Travel: Opportunities for
    Behavioral Change."  Transportation Research 16D: 49-56, 2011.
2.   Kockelman, K., M. Thompson and C. Frei. "Americans' Contributions to Climate Change:
    Opportunities for Meeting Carbon Targets." Journal of Urban Planning and Development, Vol. 137
    (2): 91-100,2011.
3.   Paul, B., K. Kockelman and S. Musti. "The Light-Duty-Vehicle  Fleet's Evolution: Anticipating
    PHEV Adoption and GHG Emissions across the U.S. Fleet." Forthcoming in Transp Research
    Record, 2011.
4.   Kockelman K., M. Bomberg, M. Thompson and C. Whitehead. "GHG Emissions Control Options:
    Opportunities for Conservation." Report commissioned by National Academy of Science's
    Committee for the Study on the Relationships among Development Patterns, VMT, and Energy
    Conservation. June 2009.
5.   Khan, M., and K. Kockelman. Predicting the Market Potential of Plug-In Electric Vehicles Using
    Multiday GPS Data. Proc'gs of the Annual Meeting of the Transportation Research Board and under
    review for publication in Transportation Research Record.

SELECTED OTHER REFEREED PUBLICATIONS
1.   Tirumalachetty, S. and K. Kockelman. "Forecasting Greenhouse Gas Emissions from Urban Regions:
    Microsimulation of Land Use and Transport Patterns in Austin, Texas." Under review for publication
    in Journal of Transport Geography.
2.   Turtle, D. P. and K. Kockelman. "Electrified Vehicle Technology Trends, Infrastructure Implications
    and Cost Comparisons."  Forthcoming in Journal of the  Transportation Research Forum.
    Tirumalachetty, S. and K. Kockelman. "The Welfare Implications of Carbon Taxes and Carbon Caps:
    A Look at U.S. Households." Under review for publication to NOVA Science Publishers as a chapter
    in Household Energy: Economics, Consumption and Efficiency.  Bomberg, M., K. Kockelman, M.
    Thompson. "GHG Emissions Control Options: Assessing Transportation & Electricity Generation
    Technologies & Policies to Stabilize Climate Change." Under review for publication in the Journal of
    the Transportation Research Forum. Kockelman, K. "Transportation and Land Use Solutions for
    Low-Carbon Cities." Under review for publication in the Journal of the Transportation Research
    Forum.

Over 95 refereed publications to date, and over 50 technical reports, total.
Over 100 referred conference proceedings and more than 250 additional presentations of research.
                                                                                          78

-------
SELECTED HONORS AND AWARDS
U.C. Berkeley's University Medal - "Most Distinguished Graduate" of Graduating Class of 5,300 students,
     1991
National Science Foundation Faculty Early Career Development (CAREER) Award, 2000-2004
Ford CAREER Award, given by Ford Motor Company's Ford Fund, 2002
One the World's Top 100 Young Innovators, according to MIT's Technology Review, May 2002
Named to the National Academy of Engineering's Gallery of Women in Engineering, August 2002
Inaugural recipient of the Annual New Faculty Award, sponsored by the Council of University Transportation
    Centers and the American Road and Transportation Builders Association, recognizing "outstanding
    teaching and research contributions to the transportation field" December 2002
Recipient of the Geoffrey J.D. Hewings Award for 2006 presented by the Regional Science Assoc.
    International
Recipient of ASCE's 2007 Harland Bartholomew Award, in recognition of contributions to the
    enhancement of the role of the civil engineer in urban planning and development
Recipient of the Women's Transportation Seminar 2007 Heart of Texas Chapter Woman of the Year
    Award
Recipient of ASCE's 2010 Walter L. Huber Research Prize in Transportation Engineering, for
    contributions in the areas of data acquisition and analysis to facilitate decisions in transport planning
    and policy-making

RELATED PROFESSIONAL ASSIGNMENTS, SYNERGISTIC TO THIS PROPOSAL
Member, Executive Committee, UT Austin & Texas A&M, NSF-sponsored Plug-in Electric Vehicle
    Industry-University Research Center (IURC), 2010 - present.
Member, UT Austin's NSF IGERT for Sustainable Grid Integration of Distributed & Renewable Energy
    Systems, 2010-present, & NSF RCN SEES, for Sustainable Cities: People, Infrastructures & the Energy-
    Climate-Water Nexus, 2011-present.
Member, Advisory Council on Transportation Statistics (ACTS), Bureau of Transportation Statistics
    Research and Innovative Technology Administration (RITA), U.S. Department of Transportation,
    2010-present.
Member, National Research Council Committee for the Study on Relationships among Development
    Patterns, VMT, and Energy Conservation, 2007-2009.
Member, RAND Corporation's  Panel on Transportation and Climate Change, Washington, DC, June
    2008
Member, TRB Committees on Transportation and Land Development, Transport Economics, and
    Statistical Methods (and Past Chair of TRB's Survey Methods Committee).
Member, Editorial Advisory Boards of Transportation Research (Part B), Journal of Transport and Land
    Use, Journal of Regional Science, and Papers in Regional Science.

PAST RELATED ENGAGEMENTS
Member, Transportation Electrification Panel, Sponsored by Indiana University's School of Public and
    Environmental Affairs, producing report "Plug-in Electric Vehicles: A Practical Plan for Progress"
    report (available at http://www.indiana.edu/~spea/pubs/TEP combined.pdf), 2010-2011.
Invited presentation of "Urban Planning, Land Use, and Vehicle Technologies to Reduce GHG Emissions
    in Cities"  for the National Academy of Science's EU-US Frontiers of Engineering Symposium, at UC
    Irvine, November 3, 2011; and for UT Austin's Energy and Urban Development Conference,
    November 18,2011.
Presentation of "PEV Market Potential Using Multi-Day GPS Data" for the NSF EV-TEC Industry-
    University Research Center workshop on Electric Vehicles - Transportation and Electricity
    Convergence, Industry Advisory Panel Meeting, Houston, TX, November 2, 2011.
Invited Presenter, US-China NSF Workshop titled "Pathways Toward Low Carbon Cities: Quantifying
    Baselines and Interventions", Hong Kong, December 13-14, 2010.
                                                                                         79

-------
Invited Lecture titled "Environmental Impacts of Land Use and Transportation Policies" for Europe's
    Kuhmo-Nectar Summer School in Transportation Economics, Valencia, Spain, July 2010.
Invited presentation, titled "Incorporating Environment and Equity into Evaluations of Land Use
    Transportation Systems" for UCLA Symposium titled "Transportation - Land Use -Environment
    Connection," Lake Arrowhead, CA, October 18-20, 2009.
Invited presentation, titled "Credit Based Congestion Pricing: An Opportunity for Reductions in Greenhouse
    Gas Emissions?" at the International Colloquium on Transport, Energy and Greenhouse Gases: Will
    Rationing be Necessary?, 19th Entretiens du Centre Jacques Cartier in Lyon, France, December 4-5, 2006.

Please visit http://www.caee.utexas.edu/prof/kockelman/ for  access to pre-prints of all papers and
complete vitae (listing all research projects and publications).
                                                                                             80

-------
                             CURRICULUM VITAE
                           R. MICHAEL VAN AUKEN
                             Dynamic Research  Inc.
                             355 Van Ness Avenue
                           Torrance, California 92691
                                (310)212-5211
EDUCATION
     Bachelor of Science in Aerospace Engineering,
         University of Michigan, 1980

     Master of Science in Aeronautical and Astronautical Engineering,
         Stanford University, 1984

     Engineer in Aeronautical and Astronautical Engineering,
         Stanford University, 1987.

         Thesis was on Model Reduction with Generalized Input/Output Weightings, a
         model reduction method for linearized road vehicle dynamics models with
         potential application to automotive active suspensions.

PROFESSIONAL EXPERIENCE

  1989    Principal Engineer
    to     Dynamic Research  Inc., Torrance, California
 present
          Involved in various technical activities in the areas of vehicle
          dynamics and control, crashworthiness and crash avoidance;
          including ride characteristics, handling, occupant injury assessments
          and technology effectiveness estimates, for automobiles, motorcycles,
          and ATVs. This involved mathematical modeling and computer
          simulation of driver and vehicle systems, data analysis, and
          interpretation of results; as well as full scale and component testing
          and model validation. Other activities have included development of
          tire-road math models; Fourier analysis, sound signal and other types
          of signal analysis; large scale, computer simulations of multi-body
          dynamics; finite element analysis; and various types of statistical
          analyses and experimental design.

          Applications in these areas have included  studies related to the
          development of transient ride and noise discomfort metrics for
          passenger cars, Weibull analysis of automobile transmission failures,
          and injury and fatality assessment of accident data. The ride and
          noise discomfort metric development included, for example, a large
          scale computational effort to analyze, correlate, and interpret driver
          and vehicle objective and subjective data.  Similar studies have
                                                                            81

-------
          recently been accomplished on the topic of driver attentional workload

  1987   Sr. Staff Engineer
    to     Hughes Aircraft Company, El Segundo, California
  1989
          Involved in several automotive systems engineering tasks including
          defining vehicle performance requirements for a chassis subsystem;
          participating in  the design, development, and verification of several
          vehicle dynamics systems, including developing tire-road contact
          force and moment math models; and identification of significant
          driver/vehicle feedback factors for driving simulators.

 1980 to  Sr. Research Engineer
  1987   Lockheed Missiles & Space Company, Inc., Sunnyvale, California

          Involved in the  design, development, and verification of a attitude
          determination system based on optimal estimation methods, including
          allocating requirements to subsystems, simulation based trade
          studies, identified system anomalies and developed appropriate
          countermeasures, developed hardware math models, and refined
          simulations.

          Refined and applied simulations to assess dynamic system
          performance under extreme non-linear disturbance conditions.
          Refined and applied simulations to assess guidance system
          performance and mission planning.

  1979   Engineer
    to     Airflow Sciences Corporation, Plymouth, Michigan
  1980
          Assisted in the  development of computational fluid dynamics (CFD)
          software. Used CFD software to solve specific aerodynamic
          problems. Assisted in wind tunnel test preparation.
TECHNICAL AND SCIENTIFIC PUBLICATIONS

Van Auken, R.M., Model Reduction with Generalized Input/Output Weighting, Engineer
Thesis. Stanford University, Stanford, CA, May, 1987.

Van Auken, M., Zellner, J.W., and Kunkel, D.T., "Correlation of Zwicker's Loudness and
Other Noise Metrics with Drivers' Over-the-Road Transient Noise Discomfort," Technical
Paper 980585, SAE, 1998.

Kebschull, S.A., Zellner, J.W., Van Auken, M., Rogers, N.M.,  "Injury Risk/Benefit
Analysis of Motorcycle Protective Devices Using Computer Simulation and ISO 13232",
Paper Number 98-S10-W-26, Proc. 16th International Technical Conference on the
                                                                            82

-------
Enhanced Safety of Vehicles, Windsor, Canada, National Highway Traffic Safety
Administration, Washington D.C., 1998, pp. 2357-2374.

Zellner, J.W., Kebschull, Van Auken, R.M., "Analysis of Vehicle Tip Stability in Side
Impact Tests," Technical Paper 2000-01-1650, SAE, 2000.

Van Auken, R.M., Zellner, J.W., Kebschull, S.A., Wiley, K.D., Smith, T., Shewchenko, N,
Rogers, N.M., "Development of Neck Injury Assessment  Criteria for the ISO 13232
Motorcyclist Anthropometric Test Dummy with the Revised Neck," Paper No. 417, Proc.
18th International Technical Conference on the  Enhanced Safety of Vehicles, Nagoya,
Japan, May 2003.

Van Auken, R.M., and Zellner, J.W., "An Assessment of the Effects of Vehicle Weight
and Size on Fatality Risk in 1985 to 1998  Model Year Passenger Cars and 1985 to
1997 Model Year Light Trucks and Vans," Technical Paper 2005-01-1354, SAE
Transactions, Vol. 114, Parts, Detroit, April 11-15, 2005, pp 1607-1622.

Van Auken, R.M., Zellner, J.W., Smith, T., Rogers, N.M., "Development of an Improved
Neck Injury Assessment Criteria for the ISO 13232 Motorcyclist Anthropometric Test
Dummy," Paper No. 05-0227, Proc. 19th International Technical Conference on the
Enhanced Safety of Vehicles, 2005.

Van Auken, R.M., Kebschull, S.A., Broen, P.C.,  Zellner, J.W., Rogers, N.M.,
"Development of a Rider Size and Position Model to Determine Motorcycle Protective
Device Test Conditions," Paper Number 05-0392, Proc. 19th International Technical
Conference on the Enhanced Safety of Vehicles, 2005.

Suzuki, H., Sugimoto, Y., Van Auken, R.M., and Zellner,  J.W.,  "Development of a
Prototype Safety Analysis System to Assess and Forecast Vehicle Safety," SAE
Technical Paper 2006-01-0718, 2006.

Van Auken, R.M., "Active Suspension Control Synthesis  Using Reduced  Order Models
that Account for the Time Delay Between  Road  Inputs," ESDA2006-95570, Proc. 8th
Biennial ASME Conference on Engineering Systems Design and Analysis, Torino, Italy,
July 4-7, 2006.

Zellner, J.W., Van Auken, R.M., Kebschull, S.A., Mun  oz, S. Injury Risk-Benefit
Analysis of Rollover Protection Systems (ROPS) For All Terrain Vehicles (ATVs) Using
Computer Simulation, Full-Scale Testing and  ISO 13232, Paper F2008-08-009,  Fisita
World Congress, Munich, Germany, 2008.

Zellner, J.W., Van Auken, R.M., Chiang, D.P, Broen, P.C., Joseph Kelly,  J.K., Sugimoto,
Y , "Extension of the Honda-DRI "Safety Impact Methodology"  (SIM) for the NHTSA
Advanced Crash Avoidance Technology (ACAT) Program and Application to a
Prototype Advanced Collision Mitigation Braking System," Technical Paper 2009-01-
0781, SAE Int.  J.  Passenger. Cars-Mech. Syst. 2(1):875-894, 2009.
                                                                           83

-------
Van Auken, R.M., "Development and Comparison of Laplace Domain and State-Space
Models of a Half-Car with Flexible Body," ESDA2010-24518, Proc. 10th Biennial
Conference on Engineering Systems Design and Analysis, Istanbul, Turkey, July 12-14,
2010.

Van Auken, R.M., Zellner, J.W., Silberling, J.Y., Kelly, J., Chiang, D.P., Broen, P.,
Kirsch, A.,  Sugimoto, Y., "Extension of the Honda-DRI Safety Impact Methodology for
the NHTSA Advanced Crash Avoidance Technology (ACAT) Program and Application
to the Evaluation of an Advanced Collision Mitigation Braking System - Final Results of
the ACAT-I Program," Technical Paper 2011-01-0581, SAE Int. J. Passenger Cars -
Mech. Syst. 4(1):488-508, 2011.

Van Auken, R.M., Smith, T.A., Zellner, J.W., "Development of a Probabilistic Skull
Fracture Model for a 50th Percentile Adult Male Motorcyclist ATD Headform," Paper
Number 11-0035, Proc. 21st International Technical Conference on the Enhanced
Safety of Vehicles, Washington, June 2011.

Van Auken, R.M., Zellner, J.W., Silberling, J.Y., Sugimoto, Y., Urai, Y., "Progress
Report on Evaluation of a Pre-Production Head-On Crash Avoidance Assist System
Using an Extended "Safety Impact Methodology" (SIM)," Paper Number 11-0270, Proc.
21st  International Technical Conference on the Enhanced Safety of Vehicles,
Washington, June 2011.

Sugimoto, Y.,  Sugimoto, Y., Kawakami, K., Hashimoto, T., Zellner, J.W., Van Auken,
R.M., "Safety Impact Methodology for Advanced Crash Avoidance Technology (ACAT)
Program and Application to Advanced  Collision Mitigation Braking  System and Head-on
Crash Avoidance Assist System," First International Symposium on Future Active Safety
Technology Toward Zero-Traffic-Accident, JSAE, Tokyo, September 5-9, 2011.

Van Auken, R.M., Zellner, J.W., "Extension of the Honda-DRI "Safety Impact
Methodology" (SIM) for the NHTSA Advanced  Crash Avoidance Technology (ACAT) II
Program and Application to the Evaluation of a Preproduction Head-On Crash
Avoidance Assist System," Technical Paper 2012-01-0291, SAE, forthcoming.
PUBLISHED TECHNICAL REPORTS

VanAuken, R. M. and Zellner, J. W., ATV ROPS Tests and Simulations. DRI-TR-98-2,
Dynamic Research, Inc., Torrance, California, October 1998.

Van Auken, R.M., and Zellner, J.W., An Assessment of the Effects of Vehicle Weight
and Size Parameters on Fatality Risk in Model Year 1985-98 Passenger Cars and
1985-97 Light Trucks. DRI-TR-02-02, Dynamic Research, Inc., Torrance,  California,
DOT Docket No. NHTSA-2003-16318-2, February 2002.
                                                                          84

-------
Van Auken, R.M., and Zellner, J.W., A Further Assessment of the Effects of Vehicle
Weight on Fatality Risk in Model Year 1985-98 Passenger Cars and 1985-97 Light
Trucks, DRI-TR-03-01, Dynamic Research, Inc., Torrance, California, DOT Docket No.
NHTSA-2003-16318-3, January 2003.

Van Auken, R.M., and Zellner, J.W., A Review of the Results in the 1997 Kahane. 2002
DRI. 2003 DRI. and 2003 Kahane Reports on the Effects of Passenger Car and Light
Truck Weight and Size on Fatality Risk, DRI-TR-04-02, Dynamic Research, Inc.,
Torrance, California, DOT Docket No. NHTSA-2003-16318-7, March 2004.
Kebschull, S.A., Kelly, J., Van Auken, R.M., and Zellner, J.W., An Analysis of the Effects
of SUV Weight and Length on SUV Crashworthiness and Compatibility Using Systems
Modeling and Risk-Benefit Analysis, DRI-TR-04-04-2, Dynamic Research, Inc.,
Torrance, California, DOT Docket No. NHTSA-2003-16128-1452, July 2004.

Van Auken, R.M., and Zellner, J.W., Supplemental Results on the Independent Effects
of Curb Weight. Wheelbase. and Track on Fatality Risk in 1985-1998 Model Year
Passenger Cars and 1985-1997 Model Year LTVs. DRI-TR-05-01, Dynamic Research,
Inc., Torrance,  California,  Federal Docket No. NHTSA-2003-16318-17, 20 May 2005.

Van Auken, R.M., and Zellner, J.W., DRI Comments On Safety Impacts of EPA-NHTSA
Proposed Rule To Establish Light-Duty Vehicle Greenhouse Gas Emission Standards
and Corporate Average Fuel Economy Standards Dated September 28 2009, DRI-TM-
09-86, Dynamic Research, Inc., Torrance, California, Federal Docket No. NHTSA-2009-
0059-112.1, 27 November 2009.

Van Auken, R.M., Zellner, J.W., Chiang, D.P., Kelly, J., Silberling, J.Y., Dai, R., Broen,
P.C., Kirsch, A.M., Sugimoto, Y., Advanced Crash Avoidance Technologies Program -
Final Report of the Honda-DRI Team, Volume I: Executive Summary and Technical
Report. DOT HS 811  454, Accomplished by Honda and DRI for the National Highway
Traffic Safety Administration,  Washington, June, 2011.

Van Auken, R.M., and Zellner, J.W., Updated Analysis of the Effects of Passenger
Vehicle Size and Weight On Safety. Phase I: Updated Analysis Based on  1995 to 2000
Calendar Year Data for 1991  to 1999 Model Year Light Passenger Vehicles. DRI-TR-
11-01, Dynamic Research Inc., Torrance, California, Federal  Docket No. NHTSA-2010-
0152-0030, January 2011.

Zellner, J.E., Van Auken, R.M., Kelly, J., Silberling, J.Y., Hagoski, B.K., Sugimoto, Y.,
Advanced Crash Avoidance Technologies (ACAT) II Program - Final Report of the
Honda-DRI Team, Accomplished by Honda and DRI for the National Highway Traffic
Safety Administration, Washington, D.C.,  forthcoming.

Van Auken, R.M., and Zellner, J.W., Updated Analysis of the Effects of Passenger
Vehicle Size and Weight On Safety. Phase II: Analysis Based on 2002 to 2008
Calendar Year Data for 2000 to 2007 Model Year Light Passenger Vehicles. DRI-TR-
12-01, Dynamic Research Inc., Torrance, California, forthcoming.
                                                                          85

-------
PUBLIC PRESENTATIONS

Van Auken, R.M., and Zellner, J.W., "Effects of Vehicle Weight and Size Reductions on
Safety," P03-6853, Transportation Research Board 82nd Annual Meeting, Washington,
D.C., January 15, 2003.

Van Auken, R.M., and Erwin, S., "Injury Severity in Side Impact Mismatch," CIREN
Quarterly Meeting, Washington, D.C., April 3, 2003.

Van Auken, R.M., and Zellner, J.W. "Updated Analysis of the Effects of Passenger
Vehicle Size and Weight on Safety," NHTSA Workshop on Vehicle Mass-Size-Safety,
Washington, D.C., February 25, 2011.
PEER REVIEWS

Proc. 8th Biennial ASME Conference on Engineering Systems Design and Analysis,
Torino, Italy, July 4-7, 2006.

Proc. 10th Biennial Conference on Engineering Systems Design and Analysis, Istanbul,
Turkey, July 12-14, 2010.
                                                                           86

-------
                     Appendix B:  Conflict of Interest Statements
                       Conflict of Interest and Bias for Peer Review
Background

Identification and management of potential conflict of interest (COI) and bias issues are vital to
the successes and credibility of any peer review consisting of external experts. The
questionnaire that follows is consistent with EPA guidance concerning peer reviews.1

Definitions

Experts in a particular field will, in many cases, have existing opinions concerning the subject of
the peer review. These opinions may be considered bias, but are not necessarily conflicts of
interest.

Bias: For a peer review, means a predisposition towards the subject matter to be discussed that
could influence the candidate's viewpoint.

Examples of bias would be situations in which a candidate:

    1.  Has previously expressed a position on the subject(s) under consideration by the panel; or

    2.  Is affiliated with an industry, governmental, public interest, or other group which has
       expressed a position concerning the subject(s) under consideration by the panel.

Conflict of Interest: For a peer review, as defined by the National Academy of Sciences,2
includes any of the following:

    1.  Affiliation with an organization with financial ties directly related to the outcome;

    2.  Direct personal/financial investments in the sponsoring organization or related to the
       subject; or

    3.  Direct involvement in the documents submitted to the peer review panel... that could
       impair the individual's objectivity or create an unfair competitive advantage for the
       individual or organization.
1 U.S. EPA (2009). Science Policy Council Peer Review Handbook.

 OMB (2004). Final Information Quality Bulletin for Peer Review.

 NAS (2003). "Policy and Procedures on Committee Composition and Balance and Conflict or Interest for Committees Used in
the Development of Reports" (www.nationalacademies.org/coi).
                                                                                         87

-------
Policy and Process

•   Candidates with COI, as defined above, will not be eligible for membership on those panels
    where their conflicts apply.

•   In general, candidates with bias, as defined above, on a particular issue will be eligible for all
    panel memberships; however, extreme biases, such as those likely to impair a candidate's
    ability to contribute to meaningful scientific discourse,  will disqualify a candidate.

•   Ideally, the composition of each panel will reflect a range of bias for a particular subject,
    striving for balance.

•   Candidates who meet scientific qualifications and other eligibility criteria will be asked to
    provide written disclosure through a confidential questionnaire of all potential COI and bias
    issues during the candidate identification and selection  process.

•   Candidates should be prepared, as necessary, to discuss potential COI and bias issues.

•   All bias issues related to selected panelists will be disclosed in writing in the final peer
    review record.
                                                                                         88

-------
                       Conflict of Interest and Bias Questionnaire

      Peer Review of LBNL Reports on Vehicle Mass/Footprint Reduction & Safety


Instructions to Candidate Reviewers

    1.  Please check YES/NO/DON'T KNOW in response to each question.

    2.  If your answer is YES or DON'T KNOW, please provide a brief explanation of the
      circumstances.

    3.  Please make a reasonable effort to answer accurately each question. For example, to the
      extent a question applies to individuals (or entities) other than you (e.g., spouse,
      dependents, or their employers), you should make a reasonable inquiry, such as emailing
      the questions to such individuals/entities in an effort to obtain information necessary to
      accurately answer the questions.

Questions

    1.  Are you (or your spouse/partner or dependents) or your current employer, an author,
      contributor, or an earlier reviewer  of the document(s) being reviewed by this panel?

      YES         NO X              DON'T KNOW
   2.  Do you (or you spouse/partner or dependents) or your current employer have current
       plans to conduct or seek work related to the subject of this peer review following the
       completion of this peer review panel?

       YES X      NO                 DON'T KNOW
   3.   Do you (or your spouse/partner or dependents) or your current employer have any known
       financial stake in the outcome of the review (e.g., investment interest in a business related
       to the subject of peer review)?

       YES         NO X              DON'T KNOW
   4.  Have you (or your spouse/partner or dependents) or your current employer commented,
       reviewed, testified, published, made public statements, or taken positions regarding the
       subject of this peer review?

       YES         NO                 DON'T KNOW X
                                                                                     89

-------
   5.  Do you hold personal values or beliefs that would preclude you from conducting an
       objective, scientific evaluation of the subject of the review?

       YES         NO X              DON'T KNOW
   6.  Do you know of any reason that you might be unable to provide impartial advice or
       comments on the subject review of the panel?

       YES         NO X              DON'T KNOW
   7.  Are you aware of any other factors that may create potential conflict of interest or bias
       issues for you as a member of the panel?

       YES         NO X              DON'T KNOW
Acknowledgment

I declare that the disclosed information is true and accurate to the best of my knowledge, and that
no real, potential, or apparent conflict of interest or bias is known to me except as disclosed. I
further declare that I have made reasonable effort and inquiry to obtain the information needed to
answer the questions truthfully, and accurately. I agree to inform SRA promptly of any change
in circumstances that would require me to revise the answers that I have provided.
Tong Donna Chen
Name
       .-  ^ '                                   11/27/2011
Signature                                       Date
                                                                                     90

-------
                       Conflict of Interest and Bias Questionnaire

      Peer Review of LBNL Reports on Vehicle Mass/Footprint Reduction & Safety


Instructions to Candidate Reviewers

    1. Please check YES/NO/DON'T KNOW in response to each question.

    2. If your answer is YES or DON'T KNOW, please provide a brief explanation of the
      circumstances.

    3. Please make a reasonable effort to answer accurately each question. For example, to the
      extent a question applies to individuals (or entities) other than you (e.g., spouse,
      dependents, or their employers), you should make a reasonable inquiry, such as emailing
      the questions to such individuals/entities in an effort to obtain information necessary to
      accurately answer the questions.

Questions

    1. Are you (or your spouse/partner or dependents) or your current employer, an author,
      contributor, or an earlier reviewer  of the document(s) being reviewed by this panel?

      YES X      NO                 DON'T KNOW
       [I am a peer reviewer for the NHTSA study "Relationships Between Fatality Risk, Mass,
       and Footprint in Model Year 200-2007 Passenger Cars andLTVs. "]

   2.  Do you (or you spouse/partner or dependents) or your current employer have current
       plans to conduct or seek work related to the subject of this peer review following the
       completion of this peer review panel?

       YES  X       NO                 DON'T KNOW
       [The Insurance Institute for Highway Safety has in the past and will continue to study the
       relationship between vehicle size/weight and crash injury risk.]

   3.  Do you (or your spouse/partner or dependents) or your current employer have any known
       financial stake in the outcome of the review (e.g., investment interest in a business related
       to the subject of peer review)?

       YES         NO X              DON'T KNOW
   4.  Have you (or your spouse/partner or dependents) or your current employer commented,
       reviewed, testified, published, made public statements, or taken positions regarding the
       subject of this peer review?
                                                                                     91

-------
       YES X      NO                 DON'T KNOW
       [The Insurance Institute for Highway Safety has in the past and will continue to study the
       relationship between vehicle size/weight and crash injury risk.]

    5.  Do you hold personal values or beliefs that would preclude you from conducting an
       objective, scientific evaluation of the subject of the review?

       YES         NO X              DON'T KNOW
   6.  Do you know of any reason that you might be unable to provide impartial advice or
       comments on the subject review of the panel?

       YES         NO  X              DON'T KNOW
   7.  Are you aware of any other factors that may create potential conflict of interest or bias
       issues for you as a member of the panel?

       YES         NO  X              DON'T KNOW
Acknowledgment

I declare that the disclosed information is true and accurate to the best of my knowledge, and that
no real, potential, or apparent conflict of interest or bias is known to me except as disclosed.  I
further declare that I have made reasonable effort and inquiry to obtain the information needed to
answer the questions truthfully, and accurately. I agree to inform SRA promptly of any change
in circumstances that would require me to revise the answers that I have provided.
Charles M. Farmer, Ph.D.
Director of Statistical Services
Insurance Institute for Highway Safety
Arlington, VA	
Name
                                   l(\          October 24, 2011
Signature                                      Date
                                                                                     92

-------
                       Conflict of Interest and Bias Questionnaire

      Peer Review of LBNL Reports on Vehicle Mass/Footprint Reduction & Safety


Instructions to Candidate Reviewers

    1.  Please check YES/NO/DON'T KNOW in response to each question.

    2.  If your answer is YES or DON'T KNOW, please provide a brief explanation of the
      circumstances.

    3.  Please make a reasonable effort to answer accurately each question. For example, to the
      extent a question applies to individuals (or entities) other than you (e.g., spouse,
      dependents, or their employers), you should make a reasonable inquiry, such as emailing
      the questions to such individuals/entities in an effort to obtain information necessary to
      accurately answer the questions.

Questions

    1.  Are you (or your spouse/partner or dependents) or your current employer, an author,
      contributor, or an earlier reviewer  of the document(s) being reviewed by this panel?

      YES         NO X              DON'T KNOW
   2.  Do you (or you spouse/partner or dependents) or your current employer have current
       plans to conduct or seek work related to the subject of this peer review following the
       completion of this peer review panel?

       YES         NO X              DON'T KNOW
   3.   Do you (or your spouse/partner or dependents) or your current employer have any known
       financial stake in the outcome of the review (e.g., investment interest in a business related
       to the subject of peer review)?

       YES         NO X              DON'T KNOW
   4.  Have you (or your spouse/partner or dependents) or your current employer commented,
       reviewed, testified, published, made public statements, or taken positions regarding the
       subject of this peer review?

       YES X      NO                 DON'T KNOW
                                                                                     93

-------
    5.  Do you hold personal values or beliefs that would preclude you from conducting an
       objective, scientific evaluation of the subject of the review?

       YES         NO X              DON'T KNOW
   6.  Do you know of any reason that you might be unable to provide impartial advice or
       comments on the subject review of the panel?

       YES         NO X              DON'T KNOW
   7.  Are you aware of any other factors that may create potential conflict of interest or bias
       issues for you as a member of the panel?

       YES         NO X              DON'T KNOW
Acknowledgment

I declare that the disclosed information is true and accurate to the best of my knowledge, and that
no real, potential, or apparent conflict of interest or bias is known to me except as disclosed. I
further declare that I have made reasonable effort and inquiry to obtain the information needed to
answer the questions truthfully, and accurately. I agree to inform SRA promptly of any change
in circumstances that would require me to revise the answers that I have provided.
David L. Greene
Name
                                                      10/31/2011
Signature                                              Date
                                                                                     94

-------
                       Conflict of Interest and Bias Questionnaire

      Peer Review of LBNL Reports on Vehicle Mass/Footprint Reduction & Safety


Instructions to Candidate Reviewers

    1. Please check YES/NO/DON'T KNOW in response to each question.

    2. If your answer is YES or DON'T KNOW, please provide a brief explanation of the
      circumstances.

    3. Please make a reasonable effort to answer accurately each question. For example, to the
      extent a question applies to individuals (or entities) other than you (e.g., spouse,
      dependents, or their employers), you should make a reasonable inquiry, such as emailing
      the questions to such individuals/entities in an effort to obtain information necessary to
      accurately answer the questions.

Questions

    1. Are you (or your spouse/partner or dependents) or your current employer, an author,
      contributor, or an earlier reviewer  of the document(s) being reviewed by this panel?

      YES         NO X              DON'T KNOW
   2.  Do you (or you spouse/partner or dependents) or your current employer have current
       plans to conduct or seek work related to the subject of this peer review following the
       completion of this peer review panel?

       YES         NO X              DON'T KNOW
   3.   Do you (or your spouse/partner or dependents) or your current employer have any known
       financial stake in the outcome of the review (e.g., investment interest in a business related
       to the subject of peer review)?

       YES         NO X              DON'T KNOW
   4.  Have you (or your spouse/partner or dependents) or your current employer commented,
       reviewed, testified, published, made public statements, or taken positions regarding the
       subject of this peer review?

       YES         NO                 DON'T KNOW X
       [I have a paper (to be presented at TRB) about crash safety, controlling for vehicle size,
      fuel economy, etc.]
                                                                                     95

-------
   5.  Do you hold personal values or beliefs that would preclude you from conducting an
       objective, scientific evaluation of the subject of the review?

       YES         NO X              DON'T KNOW
   6.  Do you know of any reason that you might be unable to provide impartial advice or
       comments on the subject review of the panel?

       YES         NO X              DON'T KNOW
   7.  Are you aware of any other factors that may create potential conflict of interest or bias
       issues for you as a member of the panel?

       YES         NO X              DON'T KNOW
Acknowledgment

I declare that the disclosed information is true and accurate to the best of my knowledge, and that
no real, potential, or apparent conflict of interest or bias is known to me except as disclosed. I
further declare that I have made reasonable effort and inquiry to obtain the information needed to
answer the questions truthfully, and accurately. I agree to inform SRA promptly of any change
in circumstances that would require me to revise the answers that I have provided.
Kara Kockelman
Name
  	~~—            10/21/11
Signature       *-                                  Date
                                                                                     96

-------
                       Conflict of Interest and Bias Questionnaire

      Peer Review of LBNL Reports on Vehicle Mass/Footprint Reduction & Safety


Instructions to Candidate Reviewers

    1.  Please check YES/NO/DON'T KNOW in response to each question.

    2.  If your answer is YES or DON'T KNOW, please provide a brief explanation of the
      circumstances.

    3.  Please make a reasonable effort to answer accurately each question. For example, to the
      extent a question applies to individuals (or entities) other than you (e.g., spouse,
      dependents, or their employers), you should make a reasonable inquiry, such as emailing
      the questions to such individuals/entities in an effort to obtain information necessary to
      accurately answer the questions.

Questions

    1.  Are you (or your spouse/partner or dependents) or your current employer, an author,
      contributor, or an earlier reviewer  of the document(s) being reviewed by this panel?

      YES         NO X              DON'T KNOW
   2.  Do you (or you spouse/partner or dependents) or your current employer have current
       plans to conduct or seek work related to the subject of this peer review following the
       completion of this peer review panel?

       YES         NO X              DON'T KNOW
   3.   Do you (or your spouse/partner or dependents) or your current employer have any known
       financial stake in the outcome of the review (e.g., investment interest in a business related
       to the subject of peer review)?

       YES         NO X              DON'T KNOW
   4.  Have you (or your spouse/partner or dependents) or your current employer commented,
       reviewed, testified, published, made public statements, or taken positions regarding the
       subject of this peer review?

       YES         NO X              DON'T KNOW
                                                                                     97

-------
   5.  Do you hold personal values or beliefs that would preclude you from conducting an
       objective, scientific evaluation of the subject of the review?

       YES         NO X              DON'T KNOW
   6.  Do you know of any reason that you might be unable to provide impartial advice or
       comments on the subject review of the panel?

       YES         NO X              DON'T KNOW
   7.  Are you aware of any other factors that may create potential conflict of interest or bias
       issues for you as a member of the panel?

       YES         NO X              DON'T KNOW
Acknowledgment

I declare that the disclosed information is true and accurate to the best of my knowledge, and that
no real, potential, or apparent conflict of interest or bias is known to me except as disclosed. I
further declare that I have made reasonable effort and inquiry to obtain the information needed to
answer the questions truthfully, and accurately. I agree to inform SRA promptly of any change
in circumstances that would require me to revise the answers that I have provided.
R. Michael Van Auken
Name
                                                  2011-10-24
  r                                                	—
Signature                                          Date
                                                                                     98

-------
                                     SRA
                                     Honesty and Service


                           Appendix C:  Peer Review Charge

    Charge to Peer Reviewers of LBN L Statistical Analysis of the Effect of Vehicle Mass & Footprint
                                     Reduction on Safety

In developing programs to reduce greenhouse gas (GHG) emissions and increase fuel economy of light-
duty highway vehicles, the U.S. Environmental Protection Agency (EPA) and the National Highway
Transportation Safety Administration (NHTSA) have to evaluate the safety of mass reduction
technologies likely to be used to meet future standards.  The U.S. Department of Energy (DOE) has
contracted with Lawrence Berkeley National Laboratory (LBNL) to perform a statistical analysis of the
effect of vehicle mass and footprint reduction on safety.  LBNL's analysis of the relationship between
vehicle mass, footprint, and societal fatality and casualty risk is comprised of two phases. Phase 1 is an
assessment of the NHTSA report Relationships between Fatality Risk, Mass, and Footprint in Model  Year
2000-2007 Passenger Cars and LTVs. This study uses logistic regression analysis to estimate the
relationship of changes in vehicle mass and footprint on US fatality risk per vehicle mile traveled. Phase
2 is an independent logistic regression analysis to estimate the relationship between vehicle mass,
footprint and total casualty (fatality plus serious injury) risk, per police-reported crash, using state-level
data on all crashes.

The focus of this peer review is to evaluate the assumptions made, data and methods of statistical
analysis used, and conclusions from the analysis, for both the LBNL assessment of the NHTSA study and
the independent LBNL study using state-level data.  A comprehensive peer review by third party experts
is an important step for validation of the results of the studies, and how the results of the studies are
used in modeling the effect of new fuel economy and greenhouse gas emission standards on vehicle
safety. In order to review the LBNL analysis, you should understand and be familiar with the data,
assumptions, conclusions, and statistical approach used in the NHTSA 2011 report "Relationship
between Fatality Risk, Mass, and Footprint in Model Year 2000-2007 Passenger Cars and LTVs."  Also,
you may want to review the 2003 NHTSA report
http://www.nhtsa.gov/cars/rules/regrev/evaluate/pdf/809662.pdfand the 2010 NHTSA report
http://www.nhtsa.gov/staticfiles/rulemaking/pdf/cafe/CAFE 2012-2016 FRIA 04012010.pdf (beginning
page 464), depending upon your familiarity with them.

You are asked to review and provide expert comments on the LBNL Phase 1 and LBNL Phase 2 draft
reports described above. You are being provided LBNL's Phase 1 draft report and the NHTSA 2011
report, which it addresses. Work is being completed on the Phase 2 study, and a draft report is
expected to be available for review by November 22.

EPA is seeking peer reviewers' expert opinions on the statistic methodologies used  in the two LBNL
studies and whether they are likely to yield realistic estimates of the relationship between vehicle mass,
footprint, and total fatality or casualty risk. EPA requests that each reviewer comment on all aspects of
the two LBNL studies, with particular emphasis on the methodologies employed, assumptions inherent
to the analysis, sources of information employed, methods of calculation and any other key issues the
reviewer may identify. Reviewers are encouraged to examine and evaluate the NHTSA study in helping
                                                                                         99

-------
them to understand the LBNL assessment analysis. Findings of this peer review may be used toward
validation and improvement of the statistical analysis conducted by LBNL, and to inform EPA staff on
potential use of the regression results for predicting the safety effect of future standards in reducing
mass and footprint. No independent data analysis will be required for this review.

Reviewers are asked to orient their comments toward these five general areas: (1) assumptions; (2)
control and dependent variables used in the regression models; (3) methodology and statistics; (4) data
sets; and (5) recommendations.  Possible topics are provided in each area as illustrative examples.
Reviewers are expected to identify additional topics or depart from these examples as necessary to best
apply their particular set of expertise toward review of the LBNL reports.

Please note that the author intends to make the database and statistical programs available to the
public to ensure that the assumptions made and the methods of calculation are transparent and
replicable. Thus it will be helpful for reviewers to recommend improvements to the analysis that would
utilize publicly available information rather than those that would make use of proprietary information.

Comments should be sufficiently clear and detailed to allow readers to thoroughly understand their
relevance to the LBNL studies. Please deliver your final written comments to SRA International no
later than Thursday, December 22.

All materials provided to reviewers as well as reviewer comments should be treated as confidential, and
should neither be released nor discussed with others outside of the review panel.  Once EPA, LBNL, and
NHTSA have made their reports and supporting documentation public, EPA will notify reviewers that
they may release or discuss the peer review materials and their review comments with others.

Should reviewers have questions about what is required in order to complete this review or need
additional background material, please  contact Brian Menard at  SRA (Brian Menard@sra.com) or (434-
817-4133). If a reviewer has any questions about the EPA peer review process itself, please contact Ms.
Ruth Schenk in EPA's Quality Office, National Vehicle and Fuel Emissions Laboratory
(schenk.ruth@epa.gov) or (734-214-4017).
                                                                                          100

-------
                                                          Appendix D: Reviews
                                                        Donna Chen & Kara Kockelman
                                                        Review of LBNL Phase 1 Report
1. ASSUMPTIONS
                             COMMENTS
Please comment on the validity of any assumptions embedded in the LBNL
assessment analysis and the independent casualty analysis that could affect
the projected relationship between vehicle mass/footprint reductions and
fatality/casualty risk. Examples might include assumptions regarding
whether recent historical relationships between vehicle weight, size, and
safety will continue into the future; potential future improvements in vehicle
technology and design may result in compensatory safety benefits; and the
annual baseline fatality distribution.
The report does a nice job discussing recent trends in vehicles, such as the
increase of ESC, side airbags, and light truck crash compatibility with
passenger cars - which will improve safety outcomes for all vehicles, but
perhaps most significantly the smaller and lighter vehicles. . It also mentions
the phasing out of the lightest and smallest vehicles between model years
2000-2007 (but doesn't mention the makes and models somehow), which
were particularly poor safety performers in the past. However, with the
introduction of urban commuter vehicles, such as the SmartCar, Mini  Cooper,
and Fiat 500m, and the growing popularity of smaller, fuel-efficient compact
vehicles following gas price increases, this trend does not seem so obvious.
Such vehicles should be discussed.

The simplistic logistic model employed in this analysis only accounts for two
crash outcomes (fatal versus non-fatal) and so neglects the more detailed,
and ordered nature of injury severity data, which is unfortunate. The model
also assumes error-term homoscedasticity from one  crash or individual to
the next; in reality certain vehicle types (e.g., pickups) and crash contexts
(e.g., high speed crashes) have more uncertainty associated with their
severity outcomes. It would be good to point out such limitations for readers.
Please comment on any apparent unstated or implicit assumptions and
related caveats or limitations.
The role of driver behavior is briefly addressed in the report but not
emphasized sufficiently. Fatality risk is a combination of driver, vehicle, and
roadway characteristics. Driver behavioral differences are many and do not
solely exist for pickup truck drivers versus car drivers. Socioeconomic data
such driver household income, size, and education influence driver attitudes
and driving environments.  For example, Chen et al. (2010) found that crash
risk increases for those living in socioeconomically disadvantaged areas
(including households more likely to drive less expensive and older vehicles).
Though such data is not typically available in state and national crash
                                                                                                                                           101

-------
                                                                      databases, the importance of these driver and environmental characteristics
                                                                      on crash rates (per mile driven) and fatality risk should be stressed in both
                                                                      reports. It is clearly very difficult to control for, but a major caveat to the
                                                                      NHTSA (& now LBNL) results. We expect that crash severity could be
                                                                      probably be lowerfor many of the small cars and pickups if they were driven
                                                                      by those who tend to drive more expensive vehicles, under the same settings
                                                                      (e.g., daytime, urban freeway).  Similarly, in the second LBNL report (which
                                                                      uses VMT estimates), we expect that crash rates would probably be lower for
                                                                      these types of driver-vehicle-setting combinations.
ADDITIONAL COMMENTS:

Chen, H.Y., Ivers, R.Q., Mariniuk, A.L.C., Boufous, S., Senserrick, T., Woodward, M., Stevenson, M. and Norton R. Socioeconomic status and risk of car crash
injury, independent of place of residence and driving exposure: Results from the DRIVE study. Journal of Epidemiology and Community Health 64(10), 2010,
pp. 998-1003.
                                                                                                                                          102

-------
2. CONTROL AND DEPENDENT VARIABLES USED IN THE REGRESSION
MODELS
                             COMMENTS
Please comment on the adequacy of control and dependent variables used
in the assessment analysis and independent casualty analysis, and
recommend any alternative control or dependent variables that are available
for possible inclusion in the analysis. For example, what are the relative
merits of the main dependent variables used, fatality risk per estimated
VMT, and casualty risk per police-reported crash?
As alluded to above, a primary concern is that the NHTSA analysis (& thus the
LBNL analyses) largely neglect the idea that vehicle type (make & model) is
very much a proxy for driver type, and a vehicle's crash avoidance may have
very little to do with vehicle type. It has a lot to do with the person behind
the wheel, and gender & age simply aren't enough to control for such
distinctions. Education, risk aversion, ability, wealth, etc., are important
covariates. But existing data sets are quite limiting (though the MVOSS & FAR
with 3-year driver violation history do offer some valuable insights, not
discussed in these reports). In reality, small cars may be less crash prone than
Kahane's & Wenzel's results suggest, because they are driven by
lower-income, younger, less risk averse people driving in more crash prone
settings (e.g., commercial strips rather than pricey residential suburbs). Such
key caveats need thoughtful discussion.  Four relevant papers on the topics of
crash frequency and vehicle size-and-weight implications (by Knipling,  Kweon
& Kockelman, Wang and Kockelman, and Chen & Kockelman) have been sent
to Tom Wenzel. These all include useful  literature reviews for further
connections to useful findings for citation in the reports, as time allows the
contractor.

The grouping of the vehicles into heavier- and lighter-than-average weight
categories essentially splits a  "typical" weight vehicle of that type  into  two
categories. The impacts of curb weight and footprint on fatality risk may be
easier to interpret if the vehicles were grouped into 3 weight categories
(light, average, and heavy) &  by type (with the average category representing
vehicles within one standard  deviation of average weight). Furthermore, the
grouping of CUVs and minivans into the  same vehicle type category neglects
the fact that these vehicles have faced rather different ground clearance
requirements (impacting rollover potential), door types  (sliding vs. standard),
and, perhaps most importantly, can appeal to different types of drivers (as
indicated in the market shift of car drivers to CUV drivers).
                                                                                                                                            103

-------
What additional control variables, such as vehicle make or model, might be
included in the regression models?
Vehicle height, a variable which may be more valuable than vehicle type for
similarly structured vehicles such as sedans, wagons, CUVs, and minivans,
would be a valuable control variable. In addition to a wider track, a lower
center of gravity also increases vehicle stability, thereby reducing the risk of
rollover. Relevant literature & findings exist, and should be cited.

Other variables which have been found in past studies to influence fatality
risk such as seat belt use, roadway geometry and division type are not
included in this study (which is largely a repeat of the NHTSA study, as
specifically contracted by the EPA).

To account for driver characteristics that contribute to fatality risk,
socioeconomic variables such as household income, education, household
size, etc. would be valuable additions. Unfortunately, both state and national
crash databases typically do not include such information (outside of
MVOSS). Such issues should be flagged for readers. It seems the contractor
has done his duty, and the key limitations lie with the original methodology
he was to essentially duplicate.
Please comment on any caveats or limitations that these dependent variable
or control variables entail with respect to use of the results as the basis for
estimating the safety effect of mass reduction.
Please see above comment (in Assumptions section) regarding driver
behavior and environment.
ADDITIONAL COMMENTS:

Table 2.1 has many indicator variables labeled as "C" for continuous variable (such as ABS, ESC, AWD, DRVMALE, etc). These C's should be removed.
                                                                                                                                            104

-------
3.  METHODOLOGY AND STATISTICS
                             COMMENTS
Please comment on the validity and applicability of the methodology LBNL
used in assessing the NHTSA 2011 study and its analysis of the relationship
between mass, footprint, and risks per police-reported crash.
The report assesses the NHTSA 2011 study in a fair amount of detail and
seeks to introduce some additional analyses to better examine the
relationship between mass, footprint, and fatality risks. However, due to a
lack of control for very specific vehicle differences (which vary by make &
sub-model), the exclusion of driver characteristics and crash setting details
(which cannot always be controlled for, but are often correlated with vehicle
type), the effects of downweighting vehicles and/or shifting vehicle styles
and sizes may be overestimated. Simply changing the vehicle on a risky driver
in a high-risk setting is unlikely to influence outcomes significantly.
Please review other statistical methods LBNL has used in the analysis, in
addition to the logistic regression methodology. Examples include the
alternative approaches used by LBNL to assess NHTSA interval estimation
results, and LBNL's linear regression analysis of actual, predicted, and
residual risk by vehicle model.
In the alternative measures of exposure, the author examines the effect of
vehicle manufacturer on fatality risk and treats the luxury models produced
by Toyota, Honda, and Nissan as separate manufacturers. However, domestic
luxury brands (such as Cadillac & Lincoln) are categorized with their
nameplate manufacturers (GM and Ford), which appears inconsistent.

The effect of calendar year variables on fatality risk may be overestimated
here, since VMT is tracked by vehicle model and not by calendar year. The
trend of greatest fatality risk reductions in light trucks, CUVs and  minivans
with increasing calendar year may simply be a reflection of rising gas prices in
combination with the ailing economy contributing to lower VMT (in these
relatively low-fuel-economy vehicles).

It is unclear how the author determined the various percentage
replacements of vehicle types in the aggressive vehicle market share shift
scenario. (For example, why are 50% of SUVs replaced by CUVs and 60% of
small pickups replaced by CUVs? The CUV is a more natural replacement for
an SUV, and an SUV a more natural replacement for a pickup.)
                                                                                                                                           105

-------
Please comment on caveats or limitations of using non-significant regression
estimates to project the safety impact of mass reduction.
First, the t-statistics are not provided in the report which makes it difficult for
the reader to assess statistical significance of specific regression estimates
(except where noted by the author). Second, inclusion of a statistically
insignificant variable can influence the estimates of coefficients associated
with related variables.  Nevertheless, in general, it is best to keep insignificant
estimates if one has a strong defense for their role, since removing such
variables (& thus their parameters) will shift the burden of response to a
correlated covariate's parameter, thus biasing the latter. We generally keep
key covariates in a model up to a pvalue of 0.20 or 0.25 or so, especially in
relatively small data sets (e.g., n < 1,000). Covariates for which we have no
strong basis can be removed for pvalues > 0.10.
How might the LBNL methodology be strengthened to better represent
future vehicle designs and reduce multi-collinearity between mass and
footprint in the regression analysis?
Including more vehicle-specific characteristics (such as vehicle height and
engine size) reduces the analysis' dependence on vehicle type, since vehicle
shapes and structures will continue to evolve. There is also correlation with
context (e.g., pickups are driven in more rural locations, with greater hazards
[like less lighting, higher speed, & few medians]). Disaggregate data are
almost always best, to avoid ecological fallacies & such.
ADDITIONAL COMMENTS:

On page 55, it is unclear what is meant by "however; if anything, reduction of this type of fatality will increase detrimental effect of mass reduction in cars."
                                                                                                                                               106

-------
4. DATA SETS
Please comment on the validity and applicability of the datasets used to
project changes in risk resulting from reduction in vehicle mass. LBNL's
casualty analysis used police-reported crash data from 16 states, while the
2011 NHTSA study used national fatality data, combined with a subset of
non-culpable vehicles involved in two-vehicle crashes from police-reported
crash data from 13 states.
Please comment on any apparent, unstated, or implicit impact on estimated
risks inherent in the two different approaches, and any related caveats or
limitations. For example, what are the strengths and weaknesses of the two
measures of vehicle exposure, miles of vehicle travelled scaled up from crash
data from 13 states, and number of police-reported crashes?
COMMENTS
The acquisition of Polk data for VMT estimates by make & model is valuable,
and a contribution to the literature. However, these estimates come from
vehicles found in repair shops in non-attainment areas, and so will be biased
towards problem-prone vehicles, wealthier households who service their
vehicles more regularly, and/or urban (smoggier) areas. Such issues merit
careful discussion in the paper, so that readers are well aware of caveats.
Related to this, Tom Wenzel indicated (by phone) that he did take a look at
CA's extensive odometer reads, which go into some semi-rural locations (not
too rural), and he indicated that the VMT values by vehicle type (not
controlling for HH attributes & such) are very similar (just 5% longer in rural
areas) - except for vans (which are used much more extensively in rural
areas). This is interesting to me, and is not that different from what we've
seen in the past. For example, Kockelman & Zhao's JTS paper from 2000 (pre-
print at http://www.ee. utexas.edu/prof/kockelman/public_html/
BTSJournalLDTs.pdf) suggests that, after controlling for various HH attributes
& vehicle types, density is/was still very important (tables 1 & 2), but a shift
from a density of Ik to 5k persons per sq mile (which is 1.5 vs. 7.8 persons
per acre) means an increase of 750 mi/yr/vehicle (which is about 7.5% of
annual VMT). Such differences, and their practical significance (or lack
thereof) should be discussed in the reports.
The use of non-culpable vehicles in two-vehicle crashes as a proxy for
vehicles which are "just there" may be distorting the overall distribution of
vehicle models. VMT may differ between vehicles that are more prone to
run-off-road accidents, at-fault two-car crash vehicles, and non-culpable
vehicles.
ADDITIONAL COMMENTS:
107

-------
5. RECOMMENDATIONS
                             COMMENTS
Please comment on whether the LBNL assessment adequately addresses the
NHTSA 2011 study and identifies the safety impact from mass reduction. Are
the analytic methods and data used to assess the NHTSA study, and estimate
the relationship between risk, mass, and footprint, appropriate? Is casualty
risk per crash a legitimate measure of vehicle safety? What other methods
or data could be used to better predict the effect of future vehicle designs
on safety?
While driver fatalities per crash seems a useful measure of vehicle design
safety, and examination of fatal crash rates is very valuable (using Polk-
based exposure estimates), there are many caveats to work of this type. As
noted above: a primary concern remains a neglect of the notion that the type
of car is very much a proxy for driver type, and a vehicle's crash avoidance
may have very little to do with vehicle type. It has a lot to do with the person
behind the wheel. Simply including gender and age variables cannot account
for important covariates  such as education, risk aversion, driving ability,
wealth, etc.  In reality, small cars may be less  crash prone than Kahane's and
Wenzel's results suggest, because they are driven by lower-income, younger,
less risk averse people driving in more crash prone settings (e.g., commercial
strips rather than pricey  residential suburbs). Of course, as noted above, it is
very difficult to control for all these variables, and the contractor was asked
to rely on the original data. In reality, the best the  report authors can do with
such data sets is to explain how all the other, relevant attributes may factor
in (e.g., quality of driver and typical driving settings), and how they can
generate biased estimation (sometimes in either direction). Discussion of
relevant literature that looks more deeply at crash outcomes (e.g., Wang or
Chen's papers, mentioned above, allowing for heteroscedasticity and
individual vehicle attributes, non-driver outcomes, etc.) will also be useful.
Please comment on the overall adequacy of LBNL's assessment of the 2011
NHTSA report and its independent study of casualty risk for predicting the
effect of vehicle mass or footprint reduction on safety. Provide any
recommended improvements that might  reasonably be adopted by the
author to improve the analysis.
Overall, the study is a comprehensive assessment of the 2011 NHTSA report
and introduces interesting additional analyses to examine the relationship of
vehicle mass and footprint reduction on safety. However, as stated
previously in the comments here, driver preference for specific car types
(including size and mass) is related to driver socioeconomic characteristics
and driving behavior. As vehicle, driver, and roadway environment
characteristics all contribute to fatality risk, the effects of physical vehicle
changes such as mass or footprint reduction on safety should not be
overstated when the other two types  of characteristics are not sufficiently
accounted for.
ADDITIONAL COMMENTS:
                                                        Donna Chen & Kara Kockelman
                                                                                                                                           108

-------
                                                        Review of LBNL Phase 2 Report
1. ASSUMPTIONS
                             COMMENTS
Please comment on the validity of any assumptions embedded in the LBNL
assessment analysis and the independent casualty analysis that could affect
the projected relationship between vehicle mass/footprint reductions and
fatality/casualty risk. Examples might include assumptions regarding
whether recent historical relationships between vehicle weight, size, and
safety will continue into the future; potential future improvements in vehicle
technology and design may result in compensatory safety benefits; and the
annual baseline fatality distribution.
The Phase 2 report serves as a complimentary document to the Phase 1
report by isolating the effect of vehicle mass and footprint on
crashworthiness. Whereas the Phase 1 report analyzes fatality risk per
estimated VMT, the Phase 2 report analyzes casualty risk per crash. The
parallel structure of the two reports makes it easy for the reader to compare
the results of the two analyses.

The binary logistic model employed in this analysis can only account for two
injury outcome categories; here it is used to distinguish crashes resulting in
serious injury or death from all other crash outcomes. Thus, the model does
not account for the ordinal nature  of injury severity and neglects the
difference between a serious injury and a death.

The report states that "a serious incapacitating injury can be just as traumatic
to the victim and her family, and costly from an  economic perspective,  as a
fatality." While serious injuries are very costly to society (and may have
similar economic cost implications as deadly crashes), willingness to-pay
estimates (which include pain and suffering) price the cost of a fatality at
almost 20 times the cost of an incapacitating injury (NSC 2010). Thus, it is
difficult to assess the economic cost of the estimates of increases in casualty
risk per crash without distinguishing whether that outcome is a serious injury
or a death. This limitation of the model should be addressed in the report.

The logistic model also assumes error-term homoscedasticity and cannot
account for increases and decreases in the variation  of injury outcomes due
to vehicle and  driver type, for example. Such limitations of the model should
be discussed.
Please comment on any apparent unstated or implicit assumptions and
related caveats or limitations.
The role of driver behavior is briefly addressed in the report but not
emphasized sufficiently. Casualty risk is a combination of driver, vehicle, and
roadway characteristics. Whereas vehicle characteristics significantly
influence crashworthiness, driver behavioral differences play a significant, if
not primary, role in determining crash frequency. Socioeconomic data such
                                                                                                                                            109

-------
                                                                       driver household income, size, and education influence driver attitudes and
                                                                       driving environments. For example, Chen et al. (2010) found that crash risk
                                                                       increases for those living in socioeconomically disadvantaged areas (including
                                                                       households more likely to drive less expensive and older vehicles). Though
                                                                       such data is not typically available in state and national crash databases, the
                                                                       importance of these driver and environmental characteristics on crash rates
                                                                       (per mile driven) and casualty risk should be stressed in both reports. It is
                                                                       clearly very difficult to control for, but is a major caveat to the NHTSA (& now
                                                                       LBNL) results. We expect that crash severity could be probably be lower for
                                                                       many of the small cars and pickups if they were driven by those  who tend to
                                                                       drive more expensive vehicles, under the same settings (e.g., daytime, urban
                                                                       freeway). Thus, statements like "a 100-lb reduction in the mass  of lighter cars
                                                                       leads to a 1.84% increase in crash frequency" should be accompanied by an
                                                                       explanation of the possibility of the mass variable accounting/proxying for
                                                                       effects of lower income households owning smaller vehicles.
ADDITIONAL COMMENTS:

National Safety Council (2010) Estimating the Costs of Unintentional Injuries. Available online at
http://www.nsc.org/news_resources/injury_and_death_statistics/Pages/EstimatingtheCostsofUnintentionallnjuries

Chen, H.Y., Ivers, R.Q., Mariniuk, A.L.C., Boufous, S., Senserrick, T., Woodward, M., Stevenson, M. and Norton R. Socioeconomic status and risk of car crash
injury, independent of place of residence and driving exposure: Results from the DRIVE study. Journal of Epidemiology and Community Health 64(10), 2010,
pp. 998-1003.
                                                                                                                                            110

-------
2. CONTROL AND DEPENDENT VARIABLES USED IN THE REGRESSION
MODELS
                             COMMENTS
Please comment on the adequacy of control and dependent variables used
in the assessment analysis and independent casualty analysis, and
recommend any alternative control or dependent variables that are available
for possible inclusion in the analysis. For example, what are the relative
merits of the main dependent variables used, fatality risk per estimated
VMT, and casualty risk per police-reported crash?
As alluded to above, a primary concern is that the NHTSA analysis (& thus the
LBNL analyses) largely neglect the idea that vehicle type (make & model) is
very much a proxy for driver type, and a vehicle's crash frequency may have
very little to do with physical vehicle characteristics. It has a lot to do with
the person behind the wheel, and gender and age simply aren't enough to
control for such distinctions.  Education, risk aversion, ability, wealth, etc., are
important covariates. But existing data sets are quite limiting (though the
MVOSS & FAR with 3-year driver violation history do offer some valuable
insights, not discussed in these reports). In reality, small cars may be less
crash prone than Kahane's & Wenzel's results suggest, because they are
driven by lower-income, younger, less risk averse people driving in more
crash prone settings (e.g., commercial strips rather than pricey residential
suburbs). Such key caveats need thoughtful discussion. Four relevant papers
on the topics of crash frequency and vehicle size-and-weight implications (by
Knipling, Kweon & Kockelman, Wang and Kockelman, and  Chen  &
Kockelman) have been sent to Tom Wenzel. These all include useful
literature reviews for further connections to useful findings for citation in the
reports, as time allows the contractor.

Independent variables such as vehicle mass and footprint may be accounting
for effects of driver socioeconomic factors as discussed in the Assumptions
section. Furthermore, vehicle option variables such as AWD and side curtain
airbags may be reflecting the effects of driver environment (e.g., those living
in areas with icy winters opting for AWD) and attitude (e.g., more risk-averse
drivers opting for side curtain airbags) rather than the vehicle technology
themselves. While extremely heavy and extremely large vehicles may have
significantly different handling and braking characteristics which influence
crash frequency and casualty risk, it is unlikely that given the same driver in
the same environment,  a small change in vehicle mass or footprint would
influence the driver's crash proneness.
                                                                                                                                           Ill

-------
What additional control variables, such as vehicle make or model, might be
included in the regression models?
Vehicle height, a variable which may be more valuable than vehicle type for
similarly structured vehicles such as sedans, wagons, CUVs, and minivans,
would be a valuable control variable. In addition to a wider track, a lower
center of gravity also increases vehicle stability, thereby reducing the risk of
rollover. Relevant literature & findings exist, and should be cited.

Other variables which have been found in past studies to influence fatality
risk such as seat belt use, roadway geometry and division type are not
included in this study.

To account for driver characteristics that  contribute to casualty risk,
socioeconomic variables such as household income, education, household
size, etc. would be valuable additions. Unfortunately, both state and national
crash databases typically do not include such information  (outside of
MVOSS). Such issues should be flagged for readers.
Please comment on any caveats or limitations that these dependent variable
or control variables entail with respect to use of the results as the basis for
estimating the safety effect of mass reduction.
Please see above comment (in Assumptions section) regarding driver
behavior and environment.
ADDITIONAL COMMENTS:

Table 2.1 has many indicator variables labeled as "C" for continuous variable (such as ABS, ESC, AWD, DRVMALE, etc). These C's should be removed.
                                                                                                                                            112

-------
3. METHODOLOGY AND STATISTICS
                              COMMENTS
Please comment on the validity and applicability of the methodology LBNL
used in assessing the NHTSA 2011 study and its analysis of the relationship
between mass, footprint, and risks per police-reported crash.
The Phase 2 report enhances the findings in the Phase 1 report by isolating
the effect of vehicle mass and footprint on crashworthiness. Like the Phase 1
report, this analysis goes into a fair amount of detail and seeks to introduce
additional analyses to better examine the relationship between mass,
footprint, and casualty risks. However, due to a lack of control for very
specific vehicle differences (which vary by make & sub-model), the exclusion
of driver characteristics and crash setting details (which  cannot always be
controlled for, but are often correlated with vehicle type), the effects of
downweighting vehicles and/or shifting vehicle styles and sizes may be
overestimated. Simply changing the vehicle mass or footprint on a risky
driver in  a high-risk setting is unlikely to influence crash outcomes
significantly.
Please review other statistical methods LBNL has used in the analysis, in
addition to the logistic regression methodology. Examples include the
alternative approaches used by LBNLto assess NHTSA interval estimation
results, and LBNL's linear regression analysis of actual, predicted, and
residual risk by vehicle model.
In the alternative measures of exposure, the author examines the effect of
vehicle manufacturer on fatality risk and treats the luxury models produced
by Toyota, Honda, and Nissan as separate manufacturers. However, domestic
luxury brands (such as Cadillac & Lincoln) are categorized with their
nameplate manufacturers (GM and Ford), which appears inconsistent.
Please comment on caveats or limitations of using non-significant regression
estimates to project the safety impact of mass reduction.
First, the t-statistics are not provided in the report which makes it difficult for
the reader to assess statistical significance of specific regression estimates
(except where noted by the author). Second, inclusion of a statistically
insignificant variable can influence the estimates of coefficients associated
with related variables. Nevertheless, in general, it is best to keep insignificant
estimates if one has a strong defense for their role, since removing such
variables (& thus their parameters) will shift the burden of response to a
correlated covariate's parameter, thus biasing the latter. We generally keep
key covariates in a model up to a pvalue of 0.20 or 0.25 or so, especially in
relatively small data sets (e.g., n < 1,000). Covariates for which we have no
strong basis can be removed for pvalues > 0.10.
How might the LBNL methodology be strengthened to better represent
future vehicle designs and reduce multi-collinearity between mass and
footprint in the regression analysis?
Including more vehicle-specific characteristics (such as vehicle height and
engine size) reduces the analysis' dependence on vehicle type, since vehicle
shapes and structures will continue to evolve. There is also correlation with
context (e.g., pickups are driven in more rural locations, with greater hazards
[like less lighting, higher speed, & few medians]). Disaggregate data are
                                                                                                                                              113

-------
                                                                      almost always best, to avoid ecological fallacies & such.
ADDITIONAL COMMENTS:
                                                                                                                                          114

-------
4. DATA SETS
                              COMMENTS
Please comment on the validity and applicability of the datasets used to
project changes in risk resulting from reduction in vehicle mass. LBNL's
casualty analysis used police-reported crash data from 16 states, while the
2011 NHTSA study used national fatality data, combined with a subset of
non-culpable vehicles involved in two-vehicle crashes from police-reported
crash data from 13 states.
The Phase 2 report uses an unusually extensive data set of police-reported
crash data from 13 states which the author compares in detail to national
data sets to illustrate similarities and differences. The author is very
thorough in addressing the difference in definitions of "serious" and
"incapacitating" injuries across different states and the effects of such
inconsistency on the regression results.

Since casualty risk in the report accounts for serious injuries but not minor
injuries, the author should note that police-reported  injury  levels may also be
poor indicators of the actual or Modified Abbreviated Injury Scale (MAIS)
level, following medical evaluation. Farmer (2003) found that 41% of injuries
reported by U.S.  police as incapacitating received MAIS ratings of "minor
injury" by health care professionals using NASS Crashworthiness Data System
(CDS). Thus, the results of the estimated casualty risk increases and
decreases rely heavily on the assumption that police errors in reporting
actual MAIS ratings are consistent across states.
Please comment on any apparent, unstated, or implicit impact on estimated
risks inherent in the two different approaches, and any related caveats or
limitations. For example, what are the strengths and weaknesses of the two
measures of vehicle exposure, miles of vehicle travelled scaled up from crash
data from 13 states, and number of police-reported crashes?
The Phase 1 analysis used non-culpable vehicles in two-vehicle crashes as a
proxy for induced exposure crashes. In contrast, Phase 2 analysis uses data
from vehicles involved in one-car crashes and the responsible vehicle in two-
car crashes. The exclusion of the not-at-fault vehicle in two-car crashes may
be distorting the distribution of crash frequency and casualty risk across
different vehicle makes and models if crash-prone drivers are more likely to
drive certain types of vehicles.
ADDITIONAL COMMENTS:

Farmer, C.M. Reliability of police-reported information for determining crash and injury severity. Traffic Injury Prevention 4(1), 2003, pp.38-44.
                                                                                                                                              115

-------
5. RECOMMENDATIONS
                             COMMENTS
Please comment on whether the LBNL assessment adequately addresses the
NHTSA 2011 study and identifies the safety impact from mass reduction. Are
the analytic methods and data used to assess the NHTSA study, and estimate
the relationship between risk, mass, and footprint, appropriate? Is casualty
risk per crash a legitimate measure of vehicle safety? What other methods
or data could be used to better predict the effect of future vehicle designs
on safety?
As noted above, a primary concern remains a neglect of the notion that the
type of car is very much a proxy for driver type, and a vehicle's crash
avoidance may have very little to do with vehicle type.  It has a lot to do with
the person behind the wheel. Simply including gender and age variables
cannot account for important covariates such as education, risk aversion,
driving ability, wealth, etc. In reality, small cars may be less crash prone than
Kahane's and Wenzel's results suggest, because they are driven by
lower-income, younger, less risk averse people driving in more crash prone
settings (e.g., commercial strips rather than pricey residential suburbs). Alas,
it is very difficult to control for all these variables, since they are not readily
available in data sets. In reality, the best the report authors can do with such
data sets is to explain how all the other, relevant attributes may factor in
(e.g., quality of driver and typical driving settings), and how they can
generate biased estimation (sometimes in either direction). Discussion of
relevant literature that looks more deeply at crash outcomes (e.g., Wang or
Chen's papers, mentioned above, allowing for heteroscedasticity and
individual vehicle  attributes, non-driver outcomes, etc.) will also be useful.
Please comment on the overall adequacy of LBNL's assessment of the 2011
NHTSA report and its independent study of casualty risk for predicting the
effect of vehicle mass or footprint reduction on safety. Provide any
recommended improvements that might  reasonably be adopted by the
author to improve the analysis.
Overall, the study is an enriching complementary document to the Phase 1
assessment of the 2011 NHTSA report. The parallel structure of the two
reports allows the reader to easily compare and contrast the various
additional analyses which examine the relationship of vehicle mass and
footprint reduction on safety. However, as stated previously in the
comments here, driver preference for specific car types (including size and
mass) is related to driver socioeconomic characteristics and driving behavior.
As vehicle, driver, and roadway environment characteristics all contribute to
fatality risk, the effects of physical vehicle changes such as mass or footprint
reduction on safety should not be overstated when the other two types of
characteristics are not sufficiently accounted for.
ADDITIONAL COMMENTS:
                                                                                                                                           116

-------
                                                                 Charles Farmer
                                                      Review of LBNL Phase 1 and 2 Reports
    1.  ASSUMPTIONS
                              COMMENTS
Please comment on the validity of any assumptions embedded in the LBNL
assessment analysis and the independent casualty analysis that could affect
the projected relationship between vehicle mass/footprint reductions and
fatality/casualty risk.  Examples might include assumptions regarding whether
recent historical relationships between vehicle weight, size, and safety will
continue into the future; potential future improvements in vehicle technology
and design may result in compensatory safety benefits; and the annual
baseline fatality distribution.
The report concludes "that much of the detrimental effect of mass or footprint
reduction on risk can be attributed to the tendency for mass or footprint
reduction to increase crash frequency, rather than to reduce vehicle
crashworthiness (risk once a crash has occurred)." However, the
interpretation of casualties per crash as inversely proportional to
crashworthiness ignores the possibility that injury severity also depends upon
the circumstances of the crash. Casualties per crash must be divided into
casualties per severe crash and severe crashes per crash, where a severe crash
would be one involving more energy, e.g., high-speed or rollover.  It could be
that weight reduction increases casualties per severe crash (i.e., reduces
crashworthiness), but reduces the likelihood that a crash  is severe.
Please comment on any apparent unstated or implicit assumptions and related
caveats or limitations.
The statistical models assume no interaction between the vehicle size/weight
measures and any of the numerous covariates, but this may not be true. For
example, size/weight reductions may differently affect vehicles with and
without ESC if they affect vehicle handling.  It is risky to make statements such
as that on p. 11 of the Phase I report:
Therefore, the mass of a lighter car could be reduced by 800 Ibs while adding
ESC, without increasing fatality risk.
ADDITIONAL COMMENTS:

NHTSA's fatality analysis covered calendar years 2001-08, but the casualty analysis excludes 2008. Such exclusion is understandable given that 2008 data were
at the time unavailable for a majority of the states (I think they are available now). However, 2008 was an unusual year and may have affected the size and
weight effect estimates.  The footnote on p. 6 of the Phase II report states that an analysis including the available 2008 data will be summarized in Appendix A.
I don't see Appendix A. Is an analysis planned including 2008 data?
                                                                                                                                             117

-------
    2.  CONTROL AND DEPENDENT VARIABLES USED IN THE REGRESSION
       MODELS
                              COMMENTS
Please comment on the adequacy of control and dependent variables used in
the assessment analysis and independent casualty analysis, and recommend
any alternative control or dependent variables that are available for possible
inclusion in the analysis.  For example, what are the relative merits of the main
dependent variables used, fatality risk per estimated VMT, and casualty risk
per police-reported crash?
One needs to restrict control variables to those that are available and reliable.
A problem when combining state databases is that the states often are not
consistent as to the variables coded and the definitions of those variables.
This severely limits the list of possible control variables.
What additional control variables, such as vehicle make or model, might be
included in the regression models?
I think that already there are too many control variables in the regression
models. Instead I would consider defining different classifications of crash
types. Table 2.2 of the Phase II report shows that the distribution of crash
types for casualty crashes is very different from that for fatal crashes.
Please comment on any caveats or limitations that these dependent variable
or control variables entail with respect to use of the results as the basis for
estimating the safety effect of mass reduction.
Model overspecification could be the reason for results that are non-intuitive,
especially in the Phase II analyses of police-reported crashes. Control variables
may be correlated with each other or with the size and weight variables.  For
example, Figure 2.9 of the Phase I report implies that torso side airbags
increase fatality risk in CUVs.
ADDITIONAL COMMENTS:

The sensitivity results of Chapter 5 (Phase II) point out the extreme differences in results when changing the control variables. For example, including vehicle
make changes the effect of a 100-lb reduction in heavier cars from -0.91% to +0.55% (see Fig 5.3).
                                                                                                                                              118

-------
    3.  METHODOLOGY AND STATISTICS
                              COMMENTS
Please comment on the validity and applicability of the methodology LBNL
used in assessing the NHTSA 2011 study and its analysis of the relationship
between mass, footprint, and risks per police-reported crash.
Figure 2.11 of the Phase II report implies that NHTSA's fitting of a separate
regression model for each of the 9 crash types was unnecessary, at least for
the analysis of casualty risk per crash. I don't recall seeing a similar analysis
for fatality risk per VMT. Is it possible to get essentially the same results as
the NHTSA  study using a single regression model?
Please review other statistical methods LBNL has used in the analysis, in
addition to the logistic regression methodology.  Examples include the
alternative approaches used by LBNL to assess NHTSA interval estimation
results, and LBNL's linear regression analysis of actual, predicted, and residual
risk by vehicle model.
The graphs in LBNL's analysis of risk by vehicle model seem to indicate
different trends for light and heavy vehicles (e.g., Figure 4.1 of Phase I).
However, only simple linear relationships are examined, unlike the NHTSA
analyses, which modeled piecewise linear relationships. Also, it's not clear
why the 3rd and last rows of Table 4.1 list different numbers (both are labeled
CUVs/minivans), and some entries in Table 4.1 disagree with Figures 4.6-4.11.
Please comment on caveats or limitations of using non-significant regression
estimates to project the safety impact of mass reduction.
Making projections from non-significant regression estimates is proper so long
as the resulting confidence intervals are constructed conservatively (to
account for the accumulated imprecision).  In that sense, I prefer NHTSA's
jackknife approach to the standard errors produced by SAS (see p.  13 of Phase
How might the LBNL methodology be strengthened to better represent future
vehicle designs and reduce multi-collinearity between mass and footprint in
the regression analysis?
ADDITIONAL COMMENTS:
                                                                                                                                              119

-------
    4.  DATA SETS
                              COMMENTS
Please comment on the validity and applicability of the datasets used to
project changes in risk resulting from reduction in vehicle mass. LBNL's
casualty analysis used police-reported crash data from  16 states, while the
2011 NHTSA study used national fatality data, combined with a subset of non-
culpable vehicles involved in two-vehicle crashes from police-reported  crash
data from 13 states.
A major limitation of the Phase II analysis is a bias that may be due to the
patterns of missing data. In particular, the vehicle identification number (VIN)
is missing or mistyped for many crash records.  High-severity crashes
(especially fatal) are more likely to have detailed police investigation, so VINs
(and other variables) in these crashes may be more complete. State crash files
are therefore much less reliable than PARS.
Please comment on any apparent, unstated, or implicit impact on estimated
risks inherent in the two different approaches, and any related caveats or
limitations. For example, what are the strengths and weaknesses of the two
measures of vehicle exposure, miles of vehicle traveled scaled up from crash
data from 13 states, and number of police-reported crashes?
The VMT weights provided by NHTSA were scaled to represent the entire US.
Comments on pp. 9 and 18 of the Phase II report seem to acknowledge this
deficiency, promising to adjust these to the 13 states in the future. Was any
adjustment made, such as multiplying the weights by the proportion of annual
US VMT accounted for by each of these states? The  accuracy of the VMT
weights is critical is we are to believe the somewhat  surprising  results
concerning crashes per VMT.
ADDITIONAL COMMENTS:

Statements above Figure 2.7 in the Phase II report imply that the effects of weight reduction on crashes per VMT and fatalities per crash should add up to the
effect on fatalities per VMT. This is not the case.  For example, a 1.43% increase in crashes per VMT and a 0.76% decrease in fatalities per crash would imply a
2.16% decrease in fatalities per VMT (i.e., 1 - 0.9924/1.0143).  The fact that the model on fatalities per VMT yields an estimated 1.08% increase should be a
cause for concern.  Either the VMT weights are inaccurate or the control variables have different effects on crash frequency and crashworthiness.
                                                                                                                                            120

-------
    5.  RECOMMENDATIONS
                              COMMENTS
Please comment on whether the LBNL assessment adequately addresses the
NHTSA 2011 study and identifies the safety impact from mass reduction. Are
the analytic methods and data used to assess the NHTSA study, and estimate
the relationship between risk, mass, and footprint, appropriate?  Is casualty
risk per crash a legitimate measure of vehicle safety? What other methods or
data could be used to better predict the effect of future vehicle designs on
safety?
Casualty risk per crash does not fully measure the effects of vehicle size and
weight reductions on society. Casualty risk per VMT best coincides with the
NHTSA analysis of fatalities per VMT. The breakdown of casualty risk per VMT
into the crash frequency and crashworthiness components is of interest.
However, the surprising results reported here make everything suspect. For
example, the Phase II report concludes that "the detrimental effect of male
drivers has to do with their higher tendency of getting into a serious crash
rather than their sensitivity to injury once a serious crash has occurred" (p.
24). A few pages later it concludes that "male drivers have essentially no
effect on crash frequency, but cause a statistically significant increase in
fatality risk once a crash occurs" (p. 28).
Please comment on the overall adequacy of LBNL's assessment of the 2011
NHTSA report and its independent study of casualty risk for predicting the
effect of vehicle mass or footprint reduction on safety. Provide any
recommended improvements that might reasonably be adopted by the author
to improve the analysis.
Overall these are reasonably good studies. The Phase I report does a very
good job of assessing the NHTSA report of fatality risk. However, the Phase I
report should be more cautious in its conclusions concerning casualty risk.
The casualty analysis is based solely on police-reported data from 13 states,
which:

    1.   May not be representative of the US as a whole.
    2.   Are inconsistent in the information given and the way in which it is
        coded.
    3.   Suffer from information that is missing, inaccurate, or unclear.
ADDITIONAL COMMENTS:

Column G of Table 6.1 in the Phase II report provides the most appropriate comparison to results from the NHTSA report (Column A). For both fatalities and
casualties per VMT, a 100-lb weight reduction is most harmful in lighter cars, less harmful in heavier cars and lighter light trucks, and slightly beneficial in
heavier light trucks, minivans, and crossovers.
                                                                                                                                            121

-------
   Review of "An Analysis of the Relationship between Casualty Risk per Crash and Vehicle Mass and
                      Footprint for Model Year 2000-2007 Light-Duty Vehicles"

                                         David L Greene

                                        January 10, 2012
Summary

The Phase I and Phase II analyses by Tom Wenzel of LBNL have been executed diligently and consistently
in accord with the methods and data used in the original NHTSA analysis. The studies contain many
valuable, new insights.  The phase I study highlights the weakening relationship between vehicle mass
and highway fatalities.  This is not only seen in decreasing coefficient estimates but in the very large
number of results that are not statistically significant. When regressions were done separately by
footprint deciles, vehicle mass was statistically significantly positively related to fatalities only for light-
duty trucks in rollovers, there were almost as many cases in which mass was negatively related to
fatalities (9 vs. 13 out of 27) and there were more instances of statistically significant negative
relationships than positive relationships. Given that so many tests are being jointly conducted, it is quite
possible that when joint probabilities are considered, there is no significant relationship between mass
and fatalities (more on joint probabilities later).  Showing the weakness and inconsistency of these
results is an important contribution.

Another meaningful contribution of the phase I study, and one that deserves more emphasis, is a logical
inference from the following findings: 1) much of the variance in risk remains unexplained even by the
most complete models, 2) control variables explain 1 to 2 orders of magnitude more of the variance
than the variables of interest (mass and size), 3) when key control variables are removed or changed it
strongly influences the coefficients of mass and size.  These results have very important implications for
the robustness of the results and the likelihood that some or all of the apparently statistically significant
relationships are due to spurious correlations with omitted or imperfectly controlled factors. Noting
that exposure measures are control variables with constrained coefficients, the following observation
from the phase I study is especially perceptive.

   "Calculating risk as total fatalities per induced exposure crash, rather than per vehicle mile
   traveled, reverses the sign of mass reductions on risk in  cars and the lighter light trucks, with  mass
   reduction leading to a reduction in risk in all vehicle types."

Finally, the phase I report notes that if only the control variables are included in the regression and
not size or mass, the resulting residuals from the  regression are uncorrelated with size or mass.
Given these findings (as well as those of phase II) the conclusions that,

   "The 2011 NHTSA study, and this report, conclude that the effect of mass reduction on US fatality
   risk is small."
                                                                                             122

-------
should be revised with the following emendation, "... and probably non-existent."

Both studies, like the NHTSA analysis, have shortcomings in terms of interpreting the results and the
language used to describe the results, and acknowledging the limitations of the data and methodology.
The limitations are extensive.  The interpretation of the results of the LBNL studies commits two
important, related  errors. The first is to attribute inferred coefficients of mass and size as representing
only the effects of vehicle mass and size when, as the phase I and II study results  indicate, there is a
virtual certainty that aliasing effects are present due to a combination of omitted variables, errors in
variables and correlations among variables.  Given that estimated driver and environmental factors
tend to have 1-2 orders of magnitude larger impacts on safety outcomes than vehicle factors, the almost
certain presence of aliasing effects must be explicitly acknowledged as severely limiting the ability to
draw inferences about the effects of vehicle attributes.  Second, the language used in interpreting
results fails to  acknowledge that the analysis does not address the effects of down-weighting or down-
sizing specific vehicles or vehicle designs, but instead relies on correlations between vehicle weight and
size in existing vehicle designs. In existing vehicles, weight and size are correlated with each other and
many other vehicle attributes  (and driver and environmental attributes,  as well).  Thus, the study is not
actually measuring the effects of down-weighting via the material substitution and design changes likely
to occur as a consequence of fuel economy and emissions standards. An early example of the kind of
misleading language referred to here can be found on  page iv.

   "For example, a 100-lb reduction in the mass of lighter cars leads to a 1.84% increase in crash
   frequency (columns B), while mass reduction leads to a 0.76% decrease in the number of fatalities
   per crash (column C);"

This statement is misleading in that it implies causality rather than correlation, and  it is additionally
misleading in that it implies that the inference applies to removing weight from specific vehicles.
Neither is correct.  A better statement would be the following.

   "For example, vehicles in the lighter class that are 100 Ibs. lighter are correlated with a 1.84%
   increase in  crash frequency...."

There are so many examples of this misleading language that it is not feasible to list them all. All
should be corrected, however. Failure to correct them could lead to serious misinterpretation of the
studies' findings.

Following in the footsteps of the seminal study by DRI, the NHTSA and LBNL studies contribute to the
literature in three important ways: 1) the LBNL and NHTSA studies recognize that the societal safety
perspective is the correct perspective to when assessing the impacts of fuel economy and emissions
regulations, 2) they recognize that vehicle dimensions and vehicle mass may have separate and
potentially different impacts on both the likelihood of a crash and the outcomes of the crash and, 3) the
LBNL phase two analysis makes an additional contribution by attempting to disentangle factors affecting
the likelihood of a crash and factors affecting the outcomes of a crash.
                                                                                            123

-------
Speaking of the DRI study, I am puzzled about why there are no references cited in the phase I study and
only a handful all by Kahane and Wenzel, in the phase II study. This is perhaps due to the scope of work
defined for the two studies but there are highly relevant studies in the literature that could have been
cited, those by DRI foremost among them.  Making use of the insights from these studies would have
been helpful in interpreting the results of both phase I and phase  II.

Lack of a Theory or Model of the Phenomenon

Both the  NHTSA and LBNL studies lack a rigorous theory of the process by which down-weighting at
constant  size or down-sizing at constant mass affect societal safety either through crash avoidance or
crashworthiness. This is not a trivial shortcoming because it affects the ability to formulate hypotheses
and interpret results. Prior to the dissenting report on safety of the NRC 2002 CAFE report, the physics
of elastic collisions between objects was typically cited as the underlying physical model. That report
showed how taking the societal perspective renders that model inappropriate.  What remains appears
to be far  more complex, involving the quantity of kinetic energy, the ability of vehicle designs to absorb
that energy so as to minimize maximum deceleration rates, stability, maneuverability, safety
technologies, and  more.

The consequences of the lack of a rigorous theory are that it is not known, a priori, what the signs of
coefficients are expected to be, let alone what their quantitative relationships should be. Hypotheses
must be formulated based on  intuition and the interpretation of results is likewise ad hoc. One
implication of this is that results that suggest that lower fatalities are associated with lower vehicle mass
have equal standing, a priori, with results that indicate that higher fatalities are associated with  lower
vehicle mass, and  similarly for vehicle size. There are no surprising or unsurprising results, in theory.

This also  makes it  difficult to develop a plan for statistically testing the model or theory and its
implications.  It would have been helpful to the reader to have been presented early on in the report
with such a plan of analysis.

On the Virtual Certainty of Aliasing

The LBNL report typically attributes causal effects to correlations between mass or size and safety. In
fact, most or all of the observed correlations are almost certainly affected by aliasing effects. There is
ample evidence for this inference in the results presented in the LBNL phase II report.

The coefficients of mass and size change in important ways when  different model formulations are
estimated.  Removing and adding control variables changes the magnitudes and sometimes the  signs of
the mass and size variables. This means that, at a minimum, the mass and size variables alias the effects
of the omitted control variables. The question is whether the aliasing is eliminated entirely by the
inclusion  of the control variables available or whether some aliasing remains either because not all
relevant and correlated control variables have been included or because the included control variables
are imperfect measures of the factors they are intended to represent.
                                                                                           124

-------
The latter seems highly likely for the following reasons.  First, the overall explanatory power of the full
models (including control variables), as measured by their R2 is low. Most of the variance in casualties
and fatalities remains unexplained. Second, at least some of the important included control variables
are only crude approximations of the factors they are intended to represent.  For example, dummy
variables represent differences in state reporting practices, age and gender represent risky driving
behavior differences among owners of different sizes and masses of vehicles, the presence or absence
of a kind  of safety equipment represents both its performance and use in a  particular vehicle, and
calendar year dummy variables represent unknown factors associated with  the respective calendar
years.  Such practices are common and their use is appropriate. Third, the control variables generally
account for 1-2 orders of magnitude more variance in the casualty and fatality variables than do vehicle
weight and size. To recap, the amount of unexplained variability in the dependent variables is larger
relative to the variance statistically explained by the most complete models. Control variables are
correlated with size and mass, and they account for 1-2 orders of magnitude more variability in the
dependent variables than the variables of interest, mass and size. Therefore, even small correlations of
size and mass with omitted variables or with errors (imprecision) of the control variables could easily
result in biased estimates for the effects of size and mass on the dependent variables.

Tom Wenzel is to be commended for providing the results that definitively demonstrate the three key
points made above. The above is not a criticism of the analysis nor of the results, per se.  It is a criticism
of their interpretation.  In light of the above, the results should be interpreted in light of the virtual
certainty that many of the estimated coefficients are likely to be biased in ways that make their
interpretation highly uncertain. The implication is that phrases such as "down-weighting or down-sizing
caused" to "mass (or size) and unobserved correlated factors are associated with..."

On Joint  Probabilities

The NHTSA and LBNL studies do not  correctly interpret their results as joint statistical tests. When
testing a  hypothesis on, for example, 5 vehicle classes simultaneously, a result for one equation that
might be statistically significant on its own may not be statistically significant as one  of five related tests.

Statistical analyses comprised of multiple  regressions too often overlook the fact that tests of statistical
significance designed for individual regressions may not apply in the case of multiple regressions. That is
the case here. NHTSA conducted 5 analyses to infer relationships between  mass differences among
vehicles holding footprint constant for 5 classes of cars. The results showed one relationship out of five
was statistically significant. As table 1 illustrates, using a simple example, if one conducts 5 trials, each
with a 0.05 probability of given result, there is a  22.6% probability of finding at least  one such result in
the five trials. Thus, the joint  significance level of the overall result (1 statistically significant regression
out of 5)  is 0.226, rather than  0.05.
                                                                                             125

-------
Table 1. Simplified Illustration of the Joint Probability of Inferences in Multiple Regressions

   # Significant
   Regressions    Combinations
                0                1
                1                5
                2               10
                3               10
                4                5
                5                1

0.95
0.773780938
0.81450625
0.857375
0.9025
0.95
1



0.05
1
0.05
0.0025
0.000125
0.00000625
3.125E-07


Joint
Probability
77.37809%
20.36266%
2.14344%
0.11281%
0.00297%
0.00003%
100.0%
22.6%
So there is between a 1:4 and a 1:5 chance of getting one statistically significant result by pure chance.
In fact, the actual significance level of the results is more complicated to calculate, and probably a bit
smaller than 0.226.  Thus, it is very appropriate for Dr. Kahane to add the qualifier "if any" to his
conclusions about the relationship between the societal highway fatalities and mass reduction, holding
footprint constant.  Had appropriate tests of joint statistical significance been used to evaluate the
results in the NHTSA and LBNL studies, the significance levels very likely would not meet accepted
criteria for statistical significance.  This could change the conclusions of the studies from the inference
that mass is correlated with fatalities or casualties in some case but not others to the lack of statistically
significant evidence that mass is correlated with fatalities or injuries on the highway. This is an
important difference.

Page-by-Page Comments

I will make page by page comments on the phase II study only, since that contains the overwhelming
share of original contributions and the key findings of the phase I study are recapitulated there.

p. iii    Paragraph 3. This would be a very good place to acknowledge the importance of driver behavior
        and environment on crash avoidance especially.

p. iv    Para. 3. This would be a good place to discuss probability inference in joint tests.

        Para 4. The statement about a 100-lb reduction in the mass, etc., is a good example of
        misleading language.

p. v     Para. 1. Again, it is misleading to say that mass reduction increases crash frequency, for reasons
        stated above.

        Para. 2. It is more accurate to describe the association of lower vehicle mass with casualty risk
        than the "effect of mass reduction on..." casualty risk.
        Para. 3. Would benefit greatly from joint probability inferences.
                                                                                             126

-------
       Para. 4. As noted above, this shows how much more important the control variables are than
       the variables of interest.

       Para 5.  Again, these are correlations not necessarily effects.

p. vi    Para. 1. Again, mass reduction is misleading terminology and you do not know if it increases
       casualty risk or not, you know only a correlation.  Why is this so important? It is the virtual
       certainty of spurious correlations, or aliasing, as noted above.

       Para. 4. (1st bullet) This is clear evidence of aliasing. Take variables out of the regression and the
       coefficients of interest change in important ways.  Are there no important factors still missing?
       Are the variables included perfect measures of the factors of interest? Of course not. Thus,
       there must be remaining aliasing. How bad is it?  We don't know.

       The third bullet shows the same effect with a different set of variables.

p. vii   Para. 4. No, your analysis does not indicate "...that much of the detrimental effect of mass or
       footprint reduction  on risk can be attributed to the tendency for mass or footprint reduction to
       increase crash frequency." Again, you have correlation, not causation and  you have good
       reason to believe that what you are seeing is affected by spurious correlations.

       Para. 5. The "effect" is small, 1-2 orders of magnitude smaller than correlations with other
       control  variables, and  IS strongly affected by which variables are in the equation, as stated on
       the previous page, and there is a great deal of unexplained variance. Please reconsider the
       meaning of these results  in light of the comments above.

       Finally,  as the last paragraph of the ES implies, it would be far better not to speak in terms of
       "reducing" mass or  size. That is not what is happening in your data set.

p. 1    Para. 4. Risk per VMT includes the effects of how well vehicles are driven as well as how well
       they can be driven.  I think there is no chance that you have fully accounted for how well
       vehicles are driven.

p. 2    Para. 2. Exposure measures are explanatory variables whose elasticity is constrained to 1. That
       is, it is assumed that an increase in vehicle use of 1 vehicle mile produces a 1 unit increase in the
       chance  of a fatality  (or casualty as the case may be).  This is actually a maintained hypothesis. If
       this hypothesis is incorrect, it can  bias the other coefficients in the equation.  Thus, the change
       from fatalities/VMT to fatalities/registration-year, to fatalities  per crash not only changes the
       meaning of the analysis, it also may bias coefficients in the event that the true relationship
       between fatalities and VMT is not an elasticity of 1.

p. 3    Line 1.  Please acknowledge that your "accounting for differences in driver characteristics, crash
       locations, and other vehicle attributes" is incomplete and that this could affect your inferences
       about size and mass.

p. 3    Para. 2. NHTSA's use of "non-culpable" vehicles involved in two vehicle crashes as an exposure
       measure raises its own issues.  How non-culpable was the non-culpable vehicle. Often this is a
       matter of degree, rather than black or white. Driver behavior may also be  involved. It seems to
                                                                                            127

-------
       me this is just another potential measure of exposure that may or may not be better than any
       other measure and may introduce new sources of bias in the analysis.

p. 3    Para. 4. Induced exposure needs to be defined.  What is it intended to mean? This needs to be
       explicit.

       Also, I am startled that there are no equations in these reports.  Equations can provide an
       unequivocal explanation of the assumed relationships that cannot be adequately accomplished
       by words, in many cases. Why no equations?

p. 7    Para 3.  CUVs and minivans are involved in fewer crashes with stationary objects than  cars.
       Why?  Is it the drivers, the vehicles, or the passengers?  How well can you control for such
       differences?  Not well.  What does this mean for your analysis?

p. 9    Para 1.  Here an equation showing how the weighting was done would be very helpful.

p. 9    Bullet 1. Excluding these vehicle types implies that the control variables in the model are not
       adequate to account for whatever makes these vehicle types different from the vehicles
       included in the analysis. First, this is an admission that the model is not adequate to explain the
       fatalities associated with these vehicles. Second, it is an admission that if they were included
       the coefficients on the variables of interest would likely be biased by spurious correlations.
       Clearly, it would not even be sufficient to include the vehicles along with a control variable (e.g.,
       X = 1 if vehicle is a police car, 0 otherwise). This is yet another indication that the model suffers
       generally from omitted variables, errors in variables and correlation among right-hand side
       variables.

p. 10   Line 3. Sentence does not make sense. Please correct.

p. 12   Sect. 2.3.  Please provide an equation.

p. 13   Were the confidence intervals calculated using ex-l? Please state explicitly or, better,  show an
       equation.

       Para. 2. Here another instance where you say "mass reduction increases societal fatality risk"
       but you really are not entitled to say that. It is misleading. Also, the NHTSA CI's are larger, as
       they should be in a joint test.

       Para. 3. These results require an underlying theory for interpretation.  The lack of one makes it
       seem like there is just no consistency in the  results.

       Para. 4. The fact that the results for fatalities per crash differ substantially from fatality per VMT
       may be very important.  Taken at face value, it would imply that any negative effect of reduced
       mass is due to its effect on crash avoidance  (crash probability) rather than crashworthiness.
       This is where the lack of a theory is most troubling.  Why would  that be? Are lighter vehicles
       less easily controlled, etc.? Or, as seems much more likely, is there a spurious correlation
       between mass and other omitted or imperfectly measured factors (including driver behavior)
       that lead to an increased probability of a crash?  Consider, for example, driver age. Driver age  is
       related to crash involvement. Driver age is a control variable. But are all young drivers the
                                                                                            128

-------
       same? Is it possible that young drivers more prone to risky behavior tend to drive lighter
       vehicles? If so, this could partly explain the result observed. Of course, this is just speculative,
       but the point is that correlations with imperfectly measured and omitted factors are highly likely
       to be present in the data and, if there, could easily affect the statistical inferences.

p. 14   Here we  see that changing the exposure measure influence the effect of mass and size on
       fatalities and casualties, which is more evidence that spurious correlations are likely to be
       biasing estimated coefficients for mass and size.

       The  bar graphs with confidence intervals are well done and convey a great deal of information
       effectively. The patterns of magnitude and statistical significance are difficult to interpret, partly
       because there is no explicit theory of what should happen and partly, perhaps, because the
       relationships are actually not real.

p. 16   Para. 1. Reduction in the mass of lighter cars increases crash frequency but reduces fatalities
       per crash. This is contradictory to the  previously maintained theory that mass protects due to
       the physics of velocity changes in elastic collisions.  Indeed, there is no theoretical explanation
       for these results, only speculation.

p. 18   Para. 1. Here is a good example of such casual speculation.

p. 19   Para. 1. Developing VMT weights for the 13 states is a good idea, given the effect of exposure
       measures on inferences. Still the results would not be definitive.

p. 20   Para. 2. Mass reduction leads to a large reduction in risk only in crashes with objects for heavier
       cars?  There is only one type of crash in which the simple physics of collisions leads to an
       unambiguous benefit for increased mass, and that is collisions with moveable or breakable
       objects.  This finding contradicts even that.

       Para. 3. More speculation, this time about  rollovers.

p. 21   Para. 3. "Curiously,..." Curiouser and curiouser.

       Figure 2.15 printed without labels. Could be my computer but the other graphs were fine.

p. 24   Para. 2. This is probably a very important finding that needs further investigation and
       explanation.  As figure 2.16 illustrates well, the correlations with mass and size are orders of
       magnitude smaller than the correlations with driver and environmental factors. This is why
       even small correlations with  omitted or imperfectly measured control factors could be, are even
       likely to be, predominantly responsible for the estimated coefficients of mass and size.

p. 25   Para. 2. The results for minivans discussed here could be due to what is going on inside the
       vehicle as much or more than the vehicle itself.  How could these results be explained in terms
       of the vehicle itself. Figure 2.17 shows this again. The effects of calendar year dummies, which
       can only  be considered rough approximations to unknown and various time-related factors,
       have much large effects than size or mass.  Again in figure 2.20.
                                                                                             129

-------
p. 28   Para. 1. "Surprisingly,..." How can side airbags, which deploy only in a crash, reduce crash
       frequency but not fatality risk in a crash?  Only if the real effect is a reflection of who buys a
       CUV/minivan with side airbags and how and where they drive. There are more surprising
       inferences in paragraph 2 about male and female drivers. Surprising relative to what theory?

p. 32   The problem here is not numerical multicollinearity (numerical difficulties inverting the cross-
       product matrix) but the more complex problem of correlations among right-hand side variables,
       omitted variables, errors in variables, and correlations of included variables with omitted and
       imperfectly measured variables.  This leads to biased estimates.

p. 36   Table 3.1 cries out for inference based on joint probabilities. What is the probability of
       observing "success" in at least 3 out of 27 trials when the true probability of success is only 0.05.
       See discussion above. The probability is certainly much higher than 0.05 and probably closer to
       0.5.  The implication is that, taken together, these results do not  show any statistically
       significant relationship between mass or size and risk per crash.  If there were a  rigorous
       underlying theory, the interpretation might be different (patterns of significance could  matter)
       but there is none. Again, good graphs on succeeding pages.

p. 41   The statistical significance  of such a relationship should be the same whether bins are used or
       not. Is it?

p. 42   Para. 3. R-squared is not the correct measure of statistical significance.  Is the coefficient of
       weight significantly different from zero?

p. 43   Para. 2. This is perhaps the key finding of the phase I and II analyses. Control variables explain
       1-2 orders of magnitude more variance than  size or mass. Still, most of the variance remains
       unexplained and is uncorrelated with mass or size. It is very likely there is nothing going on
       here.

p. 47   Para. 2. More evidence of correlation of mass and size with control variables and how changing
       definitions or excluding control variables results in important changes in the coefficient of mass
       and size. Such results are considered unstable.

p. 49   Para. 1. The rationale for the grouping of manufacturers is not obvious. Can you explain it?

       Para. 2. Yet more evidence for the instability of the model and likelihood that variables still
       missing from the  model, plus errors in measuring the included control variables  are likely biasing
       the inferences. The results described in this paragraph do not make sense to me. How can they
       be explained other than random  results?

p. 50   Para. 1. More casual interpretation of results.  OK, maybe, maybe not.  Same for paragraph 2.
       The economy faltered in 2008 but the big negative effect was in 2007. The downward trend
       started in 2004. This correlates neither with  vehicle mass changes over time nor economic
       growth as measured by real GDP. Idle speculation.

p. 51   Para. 1. Good discussion of the gratuitous speculation by NHTSA about the meaning of the
       observed correlations.  This is more a Rorschach test than statistical analysis.
                                                                                            130

-------
p. 51   Para. 2. "We have no explanation for why..." More of this kind of honest appraisal is needed in
       studies like these.

p. 52   More results for which there is no explanation.

p. 53   Why the interaction between calendar year dummy variables and safety equipment? The
       presence of the safety equipment on a particular vehicle is established.  What has calendar year
       (not model year) to do with it? Again, one suspects spurious correlations.

p. 55   One needs to think carefully about the reasons why vehicles would be excluded. It does not
       appear that NHTSA did that. First line of first paragraph "used" rather than "sused".

p. 57   Well reasoned. It is interesting that NHTSA resisted including footprint or size in previous
       analyses on the grounds of correlation with mass. These results show that assertion was
       groundless.

p. 59   Para. 3. Rather than  say risk per VMT accounts for two effects, it is better to say it includes or
       comprises two effects.  But this statement also ignores the important influences of drivers and
       environment and their potential correlations with other factors. Yes, it includes how well a
       vehicle can be driven, but more importantly it includes how well a vehicle IS driven. That is in
       there too and is very likely to be correlated with make, model, and other vehicle attributes.

p. 60   Para. 3. Here again, the conclusions are misstated. It is not a genuine "reduction" in mass, but
       an association with the mass of vehicles. And how does it "lead" to and increase in crash
       frequency? What is the theory or model that predicts this? Driver and environment are very
       likely mixed up in these results to an unknown but likely substantial degree. So what can we
       really conclude? Not this.

       As I read the conclusions and inferences I find myself asking, why?, why?, repeatedly without
       any sound explanations. Page 61, paragraph 3 contains more "surprising" results. Surprising
       because they are contrary to theory?  Surprising because they are contrary to intuition?
       Surprising because they are random?  To what can we attribute so many "surprising" results,
       and how many must there be before one concludes that the analysis is not  revealing what we
       had hoped it would.

       Para. 4. Again, this cries out for joint statistical inference. Three statistically significant results
       out of  27 is probably nothing statistically significant at all.

p. 62   Para. 1. What this shows, again, is that the coefficients of mass and size are strongly influenced
       by which control variables are included in the model and how they are defined. These results
       and their implications need explanation. The bottom line is that the effects of mass and size are
       likely to be (after the necessary joint significance calculations are done)  not statistically
       significant, not consistent, and not robust.

p. 63   How do mass and footprint reduction (again, it's not really reduction in the sense of designing
       lighter vehicles to increase fuel economy or reduce GHG emissions, the issue at hand) increase
       crash frequency. What is the theory?  I don't find a theory in either the  NHTSA  report or the
       phase  I and phase II studies. Absent a theory, these results seem sufficiently unstable and
                                                                                            131

-------
inconsistent to be highly questionable as evidence of any relationship between mass or size and
crashworthiness or crash avoidance. I think joint estimation of significance levels would provide
additional support for this view.
                                                                                    132

-------
                                                        Michael Van Auken
                                                   Review of LBNL Phase 1 Report
1. ASSUMPTIONS
                           COMMENTS
Please comment on the validity of any assumptions embedded in the
LBNL assessment analysis and the independent casualty analysis that
could  affect the projected relationship between vehicle mass/footprint
reductions and fatality/casualty risk. Examples might include
assumptions regarding whether recent historical relationships between
vehicle weight, size, and safety will continue into the future; potential
future improvements in vehicle technology and design may result in
compensatory safety benefits; and the annual baseline fatality
distribution.
The basic assumptions, methodology, and data are primarily the same
as in the Kahane (2011) report. These include the following:

1) The probability of a crash fatality is proportional to the vehicle miles
   travelled (VMT), except as noted in Section 5.1
2) The logarithm of probability of fatality per VMT for a given curb
   weight, footprint, and control variable values varies as a linear
   combination of the curb weight, footprint, and control variables
   within the domain of the data.
3) The logistic regression methods determine a maximum likelihood
   estimate of model coefficients.
4) It is assumed that the above relationships remain constant in the
   recent past (i.e., 2000-2007 model year vehicles in the 2002-2008
   calendar years), present, and near future (i.e., 2017-2025 model
   year vehicles).

The first assumption that crash fatalities are proportional to VMT
rather than the number of vehicle registration years (VRY) is
appropriate because the fatalities cannot occur if the vehicles are not
driven on the road (i.e., VMT = 0). This assumption is qualified
however because VMT is more difficult to measure than VRY and
therefore may be less  accurate. On the other hand the  probability of a
fatal crash or the number of fatalities in a crash may also depend on
the vehicle occupancy. The analysis in Section 5.1  is a commendable
attempt to explore the sensitivity to this assumption, however the
Kahane (1997) and DRI (2003-2005) reports have shown that some
driver, vehicle, and environmental factors may be underrepresented or
overrepresented in unweighted induced-exposure data. VRY could
have also been  considered as a measure of exposure.
                                                                                                                              133

-------
                                                                  The second and third assumptions are appropriate provided that it is
                                                                  recognized that it is essentially impossible with currently available
                                                                  knowledge and information to model all of the factors that could affect
                                                                  the probability of fatality in a crash, and that the objective of the
                                                                  analysis is to identify overall trends versus vehicle weight and
                                                                  footprint. In general the probability of fatality depends on other many
                                                                  other factors which have not been modelled (e.g., driver behavior
                                                                  factors, vehicle design factors, roadway design factors, EMTfactors)
                                                                  and these unmodeled  factors are assumed to be uncorrelated with
                                                                  vehicle weight and footprint, and/or are represented by the other
                                                                  control variables. The latter assumption might or might not be valid.

                                                                  The fourth assumption is perhaps the weakest because it assumes that
                                                                  future vehicles will have the same design characteristics as past
                                                                  vehicles, and that the characteristics of the vehicle population (e.g.,
                                                                  collision partner weight, size, type) will also remain the same. A
                                                                  commendable attempt to partially address this effect is described in
                                                                  Section 6. These effects can be perhaps better addressed by the
                                                                  "Volpe model" described in Kahane (2011) of the Honda-DRI fleet
                                                                  systems model described in Refs (2324), which can be used to forecast
                                                                  the effects of mass reductions of individual makes and models on a
                                                                  year-by-year basis.
Please comment on any apparent unstated or implicit assumptions and
related caveats or limitations.
The induced-exposure data set provided by NHTSA is based on the
"non-culpable" vehicle in two-vehicle crashes.  It is assumed that the
dataset is a reprehensive sample of the driver and environmental
exposure factors for vehicle use. However, since these cases include
moving vehicles, some vehicle-driver-environmental conditions may be
under or over represented in this data depending on how they affect
the ability of a non-culpable vehicle to avoid a crash. Results in Ref (17)
indicate that the estimated effect of weight and size reduction are
sensitive to whether the induced-exposure data are based on the
Kahane (2003) non-culpable vehicle definition of the Kahane (1996)
stopped vehicle definition.
                                                                                                                               134

-------
                                                                 Unfortunately it is not currently possible to test this sensitivity with the
                                                                 NHTSA-provided induced-exposure data.
ADDITIONAL COMMENTS:
None come to mind.
                                                                                                                             135

-------
2. CONTROL AND DEPENDENT VARIABLES USED IN THE REGRESSION
MODELS
                          COMMENTS
Please comment on the adequacy of control and dependent variables
used in the assessment analysis and independent casualty analysis, and
recommend any alternative control or dependent variables that are
available for possible inclusion in the analysis. For example, what are
the relative merits of the main dependent variables used, fatality risk
per estimated VMT, and casualty risk per police-reported crash?
The main metric used in both the Kahane (2011) and Wenzel (2011a)
reports is the total number of fatalities (except as noted).  Reducing the
total number of fatalities, which includes both subject vehicle
occupants and collision partner fatalities, is desirable from a societal
viewpoint. Fatal crash occurrence is related to the total number of
fatalities, which has been used by Kahane (2003, 2011) to address
concerns about double counting.

VMT is a good measure of accident exposure provided that it can be
accurately determined.

[Note: This peer review does not address the Wenzel (2011b)
companion report (Ref 3) which examines the risks per police-reported
crash. See Ref 4 for comments on the companion report.]
What additional control variables, such as vehicle make or model,
might be included in the regression models?
None come to mind.
Please comment on any caveats or limitations that these dependent
variable or control variables entail with respect to use of the results as
the basis for estimating the safety effect of mass reduction.
None come to mind.
ADDITIONAL COMMENTS:

The underlying reasons for some of the estimated effects are unknown at this time, but presumably involve driver, vehicles, environment or
accident factors that have not been controlled for in the Kahane (2011) and Wenzel analyses. See, for example, Refs 17 and 25.
                                                                                                                             136

-------
3. METHODOLOGY AND STATISTICS
                           COMMENTS
Please comment on the validity and applicability of the methodology
LBNL used in assessing the NHTSA 2011 study and its analysis of the
relationship between mass, footprint, and risks per police-reported
crash.
The logistic regression methods seem to be appropriate. The
confidence intervals are based on the logistic regression Wald Chi-
Square statistic, which as Kahane (2003, 2011) has demonstrated does
not include all sources of variation. However, these confidence
intervals are useful because they do provide some indication of the
uncertainty in the results.

[Note: This peer review does not address the Wenzel (2011b)
companion report, which examines the risks per police-reported crash.
See Ref 4 for comments on the companion report.]
Please review other statistical methods LBNL has used in the analysis,
in addition to the logistic regression methodology. Examples include
the alternative approaches used by LBNL to assess NHTSA interval
estimation results, and LBNL's linear regression analysis of actual,
predicted, and residual risk by vehicle model.
The correlations in Section 3 appear to be assessed using the
Coefficient of Multiple Determination (R2) based on a linear fit to the
data (e.g., the correlation between footprint versus curb weight in
Figure 3.1 on p. 14). The linear regression model attributes the
differences between the dependent variable (vertical axis) and the
linear fit to the independent variable (horizontal axis) to random
effects.  If there is no preference as to the choice of independent and
dependent variables (e.g., footprint versus curb weight, or curb weight
versus footprint), then the linear trend and  R2 result would be different
if the two variables were interchanged, and having two different yet
equally valid results would be undesirable.

If the variation in the data can be attributed to both  variables (e.g.,
footprint and curb weight), then it would be better to report the
square of the sample correlation coefficient r2, where r is computed
according to Eqn (1). The trend lines in these correlation figures should
not be computed using a linear regression.  Instead, the trend line
should pass through the sample means (i.e., (x, y)), and have a slope
equal to the ratio of the sample standard deviations in the data (i.e., sy
/sx). Therefore, the reported correlation results do not depend on the
ordering of the data variables.
                                                                                                                                137

-------
                                                                  Note this comments does not apply to linear trends indicated in
                                                                  Section 4, for which the Coefficient of Multiple Determination (R2)
                                                                  seems appropriate.

                                                                  The Coefficient of Multiple Determination (R2) is frequently used in the
                                                                  Wenzel (2011a) report as an indicator of the statistical importance of a
                                                                  linear trend (e.g., R2 values in Tables 4.1 and 4.2 that are greater than
                                                                  0.3 are shown in blue font). It would be better to report the standard
                                                                  error, confidence interval, and/or probability value as measures of the
                                                                  statistical significance of a linear trend.
Please comment on caveats or limitations of using non-significant
regression estimates to project the safety impact of mass reduction.
Regression estimates are random numbers which have an unknown
expected value and variance, and known sample value and standard
error. If the sample value can be explained by a zero expected value
and known standard error then the result is considered not statistically
significantly  different than zero and therefore the result is not
considered to be statistically significant. However, If we can combine
this estimate with other estimates then the unknown expected values
and variances can also be combined using the same transformation,
and the statistical significance of the combined result can be tested.
Therefore, depending on the sample values and inter-correlation, the
combined result may be statistically significant even if the individual
estimates are not statistically significant.

For example, the results from each of the nine different crash types can
be combined into an overall estimate and the standard error calculated
assuming that the results for each  crash type are independent of each
other. Then the statistical significance  of the combined effect can be
determined.

However,  and Kahane (2011) points out there are two sources of
uncertainty in the regression results. The first is the PARS based
sampling error which is uncorrelated across crash types because they
are based on different fatal cases (Kahane  2011, p. 77).  The second is
                                                                                                                                138

-------
                                                                  the state based induced-exposure sampling error which is correlated
                                                                  across crash types because they are based on the same induced-
                                                                  exposure cases.  Therefore a confidence interval estimated using the
                                                                  "jackknife" method described by Kahane (2011) and accounting for
                                                                  correlation of these two error sources would be more accurate than a
                                                                  simple estimate  based on the Wald Chi-Square statistic and assumed
                                                                  independence.
How might the LBNL methodology be strengthened to better represent
future vehicle designs and reduce multi-collinearity between mass and
footprint in the regression analysis?
The effects of multi-collinearity can be mitigated by 1) obtaining more
data, 2) pooling data from different crash type or vehicle types, or 3)
reducing the number of regression variables. The first option would
require more calendar years and/or model years, which would involve
added  newer data as it becomes available (or using older data). The
second option might be to recombine the CUVs and minivans with
truck based vans and adding a control variable to compensate for the
differences in the vehicles types. The third option  might involve
removing statistically insignificant control variables or removing control
variables that would not be expected to have an effect on the
probability of fatality in the crash (e.g., the side airbag variable is not
included in pedestrian  crashes because it is not expected to affect
pedestrian fatality risk). The number of driver age control variables
might be reduced from eight to three (as in the Kahane (1997)  and DRI
(2002-2005) studies).  Finally, a linear curb weight  model instead of a
two-piece  linear model may help to better elucidate the general trend.

The Variance Inflation  Factor (VIF) has been suggested as a measure of
multi-collinearity in the Kahane (2010 and 2011) reports, however this
diagnostic metric does not account for differences in database size (i.e.,
Options 1  and 2 above). The Wenzel  (2011a) report does not discuss
the Variance Inflation Factor or report any VIF results.
ADDITIONAL COMMENTS:
None come to mind.
                                                                                                                               139

-------
4. DATA SETS
                          COMMENTS
Please comment on the validity and applicability of the datasets used
to project changes in risk resulting from reduction in vehicle mass.
LBNL's casualty analysis used police-reported crash data from 16
states, while the 2011 NHTSA study used national fatality data,
combined with a subset of non-culpable vehicles involved in
two-vehicle crashes from police-reported crash data from 13 states.
The induced-exposure data set provided by NHTSA is based on the non-
culpable vehicles in two-vehicle crashes. See the comments in Table 1
on the limitations of this data. In addition, there are also many
differences in the coding variables and values used by the different
states, which tend to make the receding to a  common data set
imprecise.

[Note: This peer review does not address the Wenzel (2011b)
companion report, which examines the risks per police-reported crash.
See Ref 4 for comments on the companion report.]
Please comment on any apparent, unstated, or implicit impact on
estimated risks inherent in the two different approaches, and any
related caveats or limitations. For example, what are the strengths and
weaknesses of the two measures of vehicle exposure,  miles of vehicle
travelled scaled up from crash data from 13 states, and number of
police-reported crashes?
The number of fatal cases tends to be much less than the number of
induced-exposure cases. Therefore the effective numbers of degrees-
of-freedom in the statistical estimates tend to be limited by the
available number of fatal cases. For example, it would not be possible
to estimate the effects of two variables (e.g., just curb weight and
footprint) if we had data for only one fatal case even if we had
thousands of induced-exposure cases. Therefore it is desirable to use
data for the entire US in order to get a large sample of fatal  cases for
the logistic regressions. This then requires the available induced-
exposure data  (i.e., from 13 states) to be "scaled up" the US level using
the method described in Kahane (2003 and 2011). The result is the
best currently available estimate of vehicle exposure.

There may be some concerns about the accuracy of the vehicle miles-
travelled data because the difficulty estimating the number of vehicle
miles travelled at the make-model-year-state level of detail.

[Note:  This peer review does not address the Wenzel (2011b)
companion report, which examines the risks per police-reported crash.
See Ref 4 for comments on the companion report.]
                                                                                                                              140

-------
ADDITIONAL COMMENTS:





None come to mind.
                                                                                                               141

-------
5.  RECOMMENDATIONS
                          COMMENTS
Please comment on whether the LBNL assessment adequately
addresses the NHTSA 2011 study and identifies the safety impact from
mass reduction. Are the analytic methods and data used to assess the
NHTSA study, and estimate the relationship between risk, mass, and
footprint, appropriate? Is casualty risk per crash a legitimate measure
of vehicle safety? What other methods or data could be used to better
predict the effect of future vehicle designs on safety?
The basic methodology described by Kahane (2011) seems appropriate;
however some results using this method and data are not well
understood and need further diagnosis.

The induced-exposure data set provided by NHTSA is based on the non-
culpable vehicles in two-vehicle crashes. See the Table 1 comments on
the limitations of this data.

[Note: This peer review does not address the Wenzel (2011b)
companion report, which examines the risks per police-reported crash.]
Please comment on the overall adequacy of LBNL's assessment of the
2011 NHTSA report and its independent study of casualty risk for
predicting the effect of vehicle mass or footprint reduction on safety.
Provide any recommended improvements that might reasonably be
adopted by the author to improve the analysis.
The Wenzel (2011a) report provides a valuable supplement to the
analysis and results in the Kahane (2011) report.

[Note:  This peer review does not address the Wenzel (2011b)
companion report, which examines the risks per police-reported crash.
ADDITIONAL COMMENTS:
See additional comments and recommendations in Tables 6 and 7.
                                                                                                                           142

-------
                                     Table 6. Additional General Comments and Recommendations
    Section
                                        COMMENTS AND RECOMMENDATIONS
               Use of R2 is confusing. Suggest using lower case "r" when referring to the sample correlation coefficient (Box, Hunter, Hunter,
               1978, P. 61); or upper case R when referring to the regression coefficient of multiple determination (Draper and Smith, 1981, p.
               90).
All
In most cases the reported results are just estimates, but are not described as such. For example, "The effect of mass reduction
on heavier cars and CUVs and minivans are not statistically significant" on p. iii should say "The estimated effect of mass
reduction on heavier cars and CUVs and minivans are not statistically significant."

This distinction is important when comparing results based on different models and assumptions because the different models
and assumptions do not change the effect itself, but rather the estimate of the effect. For example, the statement "The first
sensitivity, in dark purple, includes the weight variables in the regression model but excludes the footprint variable; this model
tests the effect of mass reduction while allowing footprint to vary with vehicle mass. This sensitivity increases the risk from a
100-lb mass reduction in cars (from 1.43% to 2.64% for lighter cars, and from 0.48% to 1.94% for heavier cars) and
CUVs/minivans (from a 0.47% decrease in risk to a 0.52% increase in risk); however, there is no change in fatality risk in light-
duty trucks" on page 15 is misleading.  It would be better to state that "The first sensitivity, in dark purple, includes the weight
variables in the regression model but excludes the footprint variable; this model tests the estimated effect of mass reduction
while allowing footprint to vary with vehicle mass. This GGnsitivity/?emoi//'ng the footprint variable from the regression model
increases the estimated risk from a 100-lb mass reduction in cars (from 1.43% to 2.64% for lighter cars, and from 0.48% to
1.94% for heavier cars) and CUVs/minivans (from  a 0.47% decrease in risk to a 0.52% increase in risk); however, there is nothe
change in the estimated fatality risk in light-duty trucks is very small and not statistically significant."

This also applies  to table and figure captions. For  example, "Table ES.l. Effect of mass and footprint reduction on fatality risk,
under alternative regression  model specifications" should say "Table ES.l. Estimated effects of mass and footprint reduction on
fatality risk, under alternative regression  model specifications."  "Figure 3.3 Effect of reduction in mass or footprint on US
fatality risk per VMT, by vehicle type: mass only, footprint only, and both" should say "Figure 3.3 Estimated effects of reduction
in mass or footprint on US fatality risk per VMT, by vehicle type: mass only, footprint only, and both."

Overall the word "effect" appears over 200 times  in this report with the "estimated" or other qualifier. In some cases this may
be appropriate and in other cases it is not appropriate. It is recommended that the author review each instance and revise as
appropriate.
                                                                                                                                 143

-------
Figures 4.1 through 4.17 do not control for the effect of vehicle size (e.g. footprint), which has been shown to be correlated
with vehicle weight (e.g., Figure 3.1), and therefore these figures may be misleading. It is strongly suggested that the horizontal
axis label be changed to "Curb weight (Ibs) and corresponding changes in size," and/or a note such as the following be added to
each figure: "Note these results do not control for the effect of vehicle size on fatality risk. Therefore the horizontal axis
represents changes to both vehicle weight and vehicle size."
The statistical significance of the linear trends in Figures 4.1 through 4.17 are not reported. It would be helpful if the
confidence intervals or statistical significance of the linear trends were reported, either in addition to or instead of R2.
The confidence intervals for the estimated slopes should be added to the results in Tables 4.1 and 4.2.
                                                                                                                   144

-------
Table 7. Additional Specific Comments and Recommendations
Section
Executive
Summary, 7
Executive
Summary
Executive
Summary, 4
Executive
Summary, 7
4
4
5.1
5.2
5.2
5.2
5.3
6.4
Page
iii, 65
iv
Iv, v, 22, 66
viii, 69
31-32
35
37
39
39
39
44
63
COMMENTS AND RECOMMENDATIONS
2nd paragraph refers to "our analysis," however the results are the same as the NHTSA analysis. The author
should clarify who or what "our analysis" refers to and how it relates to the NHTSA analysis. Perhaps the
statement "LBNL was able to reproduce the NHTSA analysis, which finds that..." would be more appropriate.
Last bullet - suggest changing the statement that "Logistic regression does not allow a statistic" to "Logistic
regression methods do not have a statistic."
Suggest changing "variance in risk" to variation in risk" throughout.
The numerical results for the NHTSA preferred model in Tables ES.l and 7.1 are slightly different than the results
reported in the NHTSA report. For example 1.43%/0.48%/0.52%/-0.40%/-0.47% should be 1.44%/0.47%/O.S2%/-
0.39%/-0.46%
Figures 4.12 through 4.14 have the results for small and heavy-duty pickups combined, which is inconsistent with
the results in Table 4.1
The R2 values in Table 4.1 are different than the values in Figures 4.6, 4.8, 4.9, 4.11.
The subsection title should be "Alternative measures of exposure and outcome" because fatal crashes and
fatalities are measures of the crash outcome, not exposure.
It would be helpful to list the 18 manufacturer dummy variables in a table.
It is unclear why Lexus, Acura, and Infinity are treated as separate manufacturers, but Cadillac and Lincoln are
not.
It is unclear why AM General is considered a Chrysler brand. The AM General Hummer was sold by GM
beginning with the 2001 model year.
It would be helpful if the figures include error bars or shading to indicate the confidence intervals.
Table 6.3 - Suggest adding a note that the 72,316 total includes fatalities that are counted more than once in
crashes involving more than one vehicle type.
                                                                                   145

-------
                                                        Michael Van Auken
                                                   Review of LBNL Phase 2 Report
1. ASSUMPTIONS
                           COMMENTS
Please comment on the validity of any assumptions embedded in the
LBNL assessment analysis and the independent casualty analysis that
could  affect the projected relationship between vehicle mass/footprint
reductions and fatality/casualty risk. Examples might include
assumptions regarding whether recent historical relationships between
vehicle weight, size, and safety will continue into the future; potential
future improvements in vehicle technology and design may result in
compensatory safety benefits; and the annual baseline fatality
distribution.
The basic assumptions, methodology, and data are primarily the same
as in the Kahane (2011) report, but have been extended to include
serious injuries as well as fatalities, and also address crash involvement
(i.e., fatalities and serious injuries per accident, and also accidents per
VMT).. These include the following:

1) The probability of a crash fatality or serious injury is proportional to
   the number of accidents (provided the crash conditions remain the
   same); and the probability of an  accident is proportional to the
   vehicle miles travelled (VMT).
2) The logarithm of probabilities of fatality or serious injury per
   accident, and accidents per VMT for a given curb weight, footprint,
   and control variable values varies as a linear combination of the
   curb weight, footprint, and control variables within the domain of
   the data.
3) The logistic regression methods determine a maximum likelihood
   estimate of model coefficients.
4) It is assumed that the above relationships remain constant in the
   recent past (i.e., 2000-2007 model year vehicles in the 2002-2008
   calendar years), present, and near future (i.e., 2017-2025 model
   year vehicles).

The first assumption that crash fatalities and serious injuries are
proportional to the number of accidents provided the crash  conditions
remain the same seems self-evident (e.g., if two fatal crashes had
exactly the same conditions, then the expected number of fatalities for
the two crashes would be twice  the value for just one of the crashes).
The assumption that the number of accidents are proportional to VMT
rather than the number of vehicle registration years (VRY) is also
appropriate because accidents cannot occur if the vehicles are not
                                                                                                                               146

-------
                                                                  driven on the road (i.e., VMT = 0).  This assumption is qualified
                                                                  however because VMT is more difficult to measure than VRY and
                                                                  therefore may be less accurate.  On the other hand the probability of a
                                                                  fatal crash or the number of fatalities in a crash may also depend on
                                                                  the vehicle occupancy.

                                                                  The second and third assumptions are appropriate provided that it is
                                                                  recognized that it is essentially impossible with currently available
                                                                  knowledge and information to model all of the factors that could affect
                                                                  the probability of fatality in a crash, and that the objective of the
                                                                  analysis is to identify overall trends versus vehicle weight and
                                                                  footprint. In general the probability of fatality depends on other many
                                                                  other factors which have not been modelled (e.g., driver behavior
                                                                  factors, vehicle design factors, roadway design factors, EMT factors),
                                                                  and these unmodeled factors are assumed to be uncorrelated with
                                                                  vehicle weight and footprint, and/or are represented by the other
                                                                  control variables. The latter assumption  might or might not be valid.

                                                                  The fourth assumption is perhaps the weakest because it assumes that
                                                                  future vehicles will have the same design characteristics as past
                                                                  vehicles, and that the characteristics of the vehicle  population (e.g.,
                                                                  collision partner weight, size, type) will also remain the same.
Please comment on any apparent unstated or implicit assumptions and
related caveats or limitations.
The induced-exposure data set provided by NHTSA is based on the
"non-culpable" vehicle in two-vehicle crashes. It is assumed that the
dataset is a reprehensive sample of the driver and environmental
exposure factors for vehicle use. However, since these cases include
moving vehicles, some vehicle-driver-environmental conditions may be
under or over represented in this data depending on how they affect
the ability of a non-culpable vehicle to avoid a crash.  Results in Ref
(17) indicated that the estimated effect of weight and size reduction
are sensitive to whether the induced-exposure data are based on the
Kahane (2003) non-culpable vehicle definition of the Kahane (1997)
stopped vehicle definition.
                                                                                                                               147

-------
                                                                  Unfortunately it is not currently possible to test this sensitivity with the
                                                                   NHTSA-provided induced-exposure data.

                                                                  It is also assumed that the accident data from the 13 or 16 states are
                                                                  representative of all US states. Figure 2.1 in Wenzel (2011b) provides a
                                                                  useful comparison of the distribution of fatalities in the US and 13
                                                                  states by the nine different crash types.
ADDITIONAL COMMENTS:
None come to mind.
                                                                                                                                148

-------
2. CONTROL AND DEPENDENT VARIABLES USED IN THE REGRESSION
MODELS
                           COMMENTS
Please comment on the adequacy of control and dependent variables
used in the assessment analysis and independent casualty analysis, and
recommend any alternative control or dependent variables that are
available for possible inclusion in the analysis. For example, what are
the relative merits of the main dependent variables used, fatality risk
per estimated VMT, and casualty risk per police-reported crash?
Reducing the total number of fatalities and serious injuries is desirable
from a societal viewpoint. This includes both subject vehicle occupant
and collision partner (e.g., other vehicle occupant, pedestrian) fatalities
and serious injuries.

VMT is a good measure of accident exposure provided that it can be
accurately determined.

The number of fatalities and serious injuries per accident is a measure
of vehicle crashworthiness (i.e., effect of a crash on the subject vehicle
occupants) and crash compatibility (i.e., effect of a crash on the other
vehicle occupants or vulnerable road users).  Subject vehicle occupant
fatalities and serious injuries per accident are a measure of the subject
vehicle crashworthiness. Collision partner fatalities and serious injuries
per accident are a measure of vehicle crash compatibility.

The number of accidents per VMT is a measure of the crash avoidance
capabilities of a given vehicle.
What additional control variables, such as vehicle make or model,
might be included in the regression models?
None come to mind.
Please comment on any caveats or limitations that these dependent
variable or control variables entail with respect to use of the results as
the basis for estimating the safety effect of mass reduction.
None come to mind.
ADDITIONAL COMMENTS:

The underlying reasons for some of the estimated effects are unknown at this time, but presumably involve driver, vehicle, environment or
accident factors than have not been controlled for in the Kahane (2011) and Wenzel (2011b) analyses.  See, for example, Refs 17 and 23.
                                                                                                                               149

-------
3. METHODOLOGY AND STATISTICS
                           COMMENTS
Please comment on the validity and applicability of the methodology
LBNL used in assessing the NHTSA 2011 study and its analysis of the
relationship between mass, footprint, and risks per police-reported
crash.
The logistic regression methods seem to be appropriate. The
confidence intervals are based on the logistic regression Wald Chi-
Square statistic, which as Kahane (2003, 2011)  has demonstrated does
not include all sources of variation.  However, these confidence
intervals are useful because they do provide some indication of the
uncertainty in the results.

The two-stage results for the 13-state fatalities per crash and 13-state
crashes per VMT, and the one-stage result for 13-state fatalities per
VMT reported in Tables ES.l and 6.1 and Figure 2.7 were computed
using independent logistic regressions. The differences between the
two-stage results and the one-stage results for fatalities per crash
could have been eliminated by using the "simultaneous three-way"
logistic regression method described in DRI (2003). This method
imposes the constraint that the combined two-stage estimated and the
one-stage estimated are equal.
Please review other statistical methods LBNL has used in the analysis,
in addition to the logistic regression methodology. Examples include
the alternative approaches used by LBNL to assess NHTSA interval
estimation results, and LBNL's linear regression analysis of actual,
predicted, and residual risk by vehicle model.
The correlations in Section 3 appear to be assessed using the
Coefficient of Multiple Determination (R2) based on a linear fit to the
data (e.g., the correlation between footprint versus curb weight in
Figure 3.1 on p. 32). The linear regression model attributes the
differences between the dependent variable (vertical axis) and the
linear fit to the independent variable (horizontal axis) to random
effects.  If there is no preference as to the choice of independent and
dependent variables (e.g., footprint versus curb weight, or curb weight
versus footprint), then the linear trend and R2 result would be different
if the two variables were interchanged, and having two different yet
equally valid results would be undesirable.

If the variation in the data can be attributed to both variables (e.g.,
footprint and curb weight), then it would be  better to report the
square of the sample correlation coefficient r2, where r is computed
according to Eqn (1).  The trend lines in these correlation figures should
                                                                                                                                150

-------
                                                                  not be computed using a linear regression. Instead, the trend line
                                                                  should pass through the sample means (i.e., (x, y)), and have a slope
                                                                  equal to the ratio of the sample standard deviations in the data (i.e., sy
                                                                  /sx). Therefore, the reported correlation results do not depend on the
                                                                  ordering of the data variables.

                                                                  Note this comments does not apply to linear trends indicated in
                                                                  Section 4, for which the Coefficient of Multiple Determination (R2)
                                                                  seems appropriate.

                                                                  The Coefficient of Multiple Determination (R2) is frequently used in the
                                                                  Wenzel (2011b) report as an indicator of the statistical  importance of a
                                                                  linear trend (e.g., R2 values in Tables 4.1 and 4.2  were compared to
                                                                  0.3). It would be better to report the standard error, confidence
                                                                  interval, and/or probability value as measures of the statistical
                                                                  significance of a linear trend.
Please comment on caveats or limitations of using non-significant
regression estimates to project the safety impact of mass reduction.
Regression estimates are random numbers which have an unknown
expected value and variance, and known sample value and standard
error. If the sample value can be explained by a zero expected value
and known standard error then the result is considered not statistically
significantly  different than zero and therefore the result is not
considered to be statistically significant. However,  If we can combine
this estimate with other estimates then the unknown expected values
and variances can also be combined using the same transformation,
and the statistical significance of the combined result can be tested.
Therefore, depending on the sample values and inter-correlation, the
combined result may be statistically significant even if the individual
estimates are not statistically significant.

For example, the results from each of the nine different crash types can
be combined into an overall estimate and the standard error calculated
assuming that the results for each  crash type are independent of each
other. Then the statistical significance  of the combined effect can be
                                                                                                                                151

-------
                                                                  determined.

                                                                  However, and Kahane (2011) points out there are two sources of
                                                                  uncertainty in the regression results. The first is the PARS based
                                                                  sampling error which is uncorrelated across crash types because they
                                                                  are based on different fatal cases (Kahane 2011, p. 77). The second is
                                                                  the state based induced-exposure sampling error which is correlated
                                                                  across crash types because they are based on the same induced-
                                                                  exposure cases. Therefore a confidence  interval estimated using the
                                                                  "jackknife" method described by Kahane (2011) and accounting for
                                                                  correlation of these two error sources would  be more accurate than a
                                                                  simple estimate based on the Wald Chi-Square statistic and assumed
                                                                  independence.
How might the LBNL methodology be strengthened to better represent
future vehicle designs and reduce multi-collinearity between mass and
footprint in the regression analysis?
The effects of multi-collinearity can be mitigated by 1) obtaining more
data, 2) pooling data from different crash type or vehicle types, or 3)
reducing the number of regression variables. The first option would
require more states (for serious injuries and police-reported accidents),
calendar years and/or model years, which would involve added newer
data as it becomes available (or using older data). The second option
might be to recombine the CUVs and minivans with truck based vans
and adding a control variable to compensate for the differences in the
vehicles types. The third option might involve removing statistically
insignificant control variables or removing control variables that would
not be expected to have an effect on the probability of crash or crash
outcome (e.g., the side airbag variable is not included in pedestrian
crashes because it is not expected to affect pedestrian fatality risk).
The number of driver age control variables might be reduced from
eight to three (as in the Kahane (1997) and DRI (2002-2005) studies).
Finally, a linear curb weight model instead of a two-piece linear model
may help to better elucidate the general trend.

The Variance Inflation Factor (VIF) has been suggested as a measure of
multi-collinearity in the Kahane (2010 and 2011) reports, however this
                                                                                                                               152

-------
                                                                 diagnostic metric does not account for differences in database size (i.e.,
                                                                 Options 1 and 2 above). The Wenzel (2011b) report does not discuss
                                                                 the Variance Inflation Factor or report any VIF results.
ADDITIONAL COMMENTS:
None come to mind.
                                                                                                                             153

-------
4. DATA SETS
                           COMMENTS
Please comment on the validity and applicability of the datasets used
to project changes in risk resulting from reduction in vehicle mass.
LBNL's casualty analysis used police-reported crash data from 16
states, while the 2011 NHTSA study used national fatality data,
combined with a subset of non-culpable vehicles involved in
two-vehicle crashes from police-reported crash data from 13 states.
The induced-exposure data set provided by NHTSA is based on the non-
culpable vehicles in two-vehicle crashes. See the comments in Table 1
on the limitations of this data.

The use of property damage accident data and cases with serious injury
from the  13 states seems appropriate (with the noted qualification that
the different states may have different accident reporting thresholds
and injury reporting criteria). The concerns about the use of data for
the 3 additional states (Georgia, Illinois, and New Mexico) have also
been noted.

In addition, there are also many differences in the coding variables and
values used by the different states, which tend to make the receding to
a common data set (either induced-exposure, police-reported accident,
or severe injury) imprecise.
Please comment on any apparent, unstated, or implicit impact on
estimated risks inherent in the two different approaches, and any
related caveats or limitations. For example, what are the strengths and
weaknesses of the two measures of vehicle exposure, miles of vehicle
travelled scaled up from crash data from 13 states, and number of
police-reported crashes?
The number of fatal or serious injury cases tends to be much less than
the number of induced-exposure cases (and the number of police-
reported accidents). Therefore the effective numbers of degrees-of-
freedom in the statistical estimates tend to be limited by the available
number of fatal or serious injury cases. For example, it would not be
possible to estimate the effects of two variables (e.g., just curb weight
and footprint) if we had data for only one fatal of serious injury case
even if we had thousands of induced-exposure cases. Therefore it is
desirable to use data for the entire US in order to get a large sample of
fatal cases for the logistic regressions. This then requires the available
induced-exposure data (i.e., from 13 states) to be "scaled up" the  US
level using the method described in  Kahane (2003 and 2011). The
result is the best currently available  estimate of vehicle exposure.

There may be some concerns about the accuracy of the vehicle miles-
travelled data because the difficulty estimating the number of vehicle
miles travelled at the make-model-year-state level of detail.
                                                                                                                               154

-------
ADDITIONAL COMMENTS:





None come to mind.
                                                                                                               155

-------
5. RECOMMENDATIONS
                          COMMENTS
Please comment on whether the LBNL assessment adequately
addresses the NHTSA 2011 study and identifies the safety impact from
mass reduction. Are the analytic methods and data used to assess the
NHTSA study, and estimate the relationship between risk, mass, and
footprint, appropriate? Is casualty risk per crash a legitimate measure
of vehicle safety? What other methods or data could be used to better
predict the effect of future vehicle designs on safety?
The basic methodology described by Kahane (2011) seems appropriate;
and the extension by Wenzel (2011b) are also appropriate.  However
some results using these methods and data are not well understood
and need further diagnosis.

The induced-exposure data set provided by NHTSA is based on the non-
culpable vehicles in two-vehicle crashes.  See the Table 1 comments on
the limitations of this data.

The state accident data files tend to have different database variable
and coding definitions and criteria, which could confound the  results.
Please comment on the overall adequacy of LBNL's assessment of the
2011 NHTSA report and its independent study of casualty risk for
predicting the effect of vehicle mass or footprint reduction on safety.
Provide any recommended improvements that might reasonably be
adopted by the author to improve the analysis.
The Wenzel (2011b) report provides a valuable supplement to the
analysis and results in the Kahane (2011) report.
ADDITIONAL COMMENTS:
See additional comments and recommendations in Tables 6 and 7.
                                                                                                                            156

-------
                                    Table 6. Additional General Comments and Recommendations
   Section
                                        COMMENTS AND RECOMMENDATIONS
               Use of R2 is confusing. Suggest using lower case "r" when referring to the sample correlation coefficient (Box, Hunter, Hunter,
               1978, P. 61); or upper case R when referring to the regression coefficient of multiple determination (Draper and Smith, 1981, p.
               90).
All
In most cases the reported results are just estimates, but are not described as such. The word "effect" appears several hundred
times in this report with the "estimated" or other qualifier. In some cases this may be appropriate and in other cases it is not
appropriate.  It is recommended that the author review each instance and revise as appropriate.
All
"Crashworthiness" in most instances should be changed to "crashworthiness and crash compatibility" because the fatalities
and/or serious injuries may either be in the subject vehicle (crashworthiness effect) or collision partner (crash compatibility
effect).
               The statistical significance of the linear trends in Figures 4.1 through 4.9 are not reported. It would be helpful if the confidence
               intervals or statistical significance of the linear trends were reported, either in addition to or instead of R2.
               The confidence intervals for the estimated slopes should be added to the results in Tables 4.1 and 4.2.
                                                                                                                                157

-------
Table 7. Additional Specific Comments and Recommendations
Section
Executive
Summary, 4
Executive
Summary, 6
5.3
5.3
5.3
Page
iv, v, 22, 66
vii, 63
49
48-49
49
COMMENTS AND RECOMMENDATIONS
Suggest changing "variance in risk" to "variation in risk" throughout.
The statement "In conclusion, casualty risk per crash is not necessarily a better metric than fatality risk per VMT
for evaluating the effect of mass or footprint reduction on risk; rather, it provides a different perspective in
assessing the benefits or drawbacks of mass and footprint reduction on safety in vehicles. However, it does
allow the separation of risk per VMT to be separated into its two components, crash frequency and risk per
crash" suggests that the casualty risk per crash metric was needed in order to assess the crash frequency and risk
per crash, which is incorrect. The DRI (2003-2012) methods have also estimated the effects of weight and size
on crash frequency (A/E) and risk per crash (F/E) in terms of fatalities.
It would be helpful to list the 18 manufacturer dummy variables in a table.
It is unclear why Lexus, Acura, and Infinity are treated as separate manufacturers, but Cadillac and Lincoln are
not.
It is unclear why AM General is considered a Chrysler brand. The AM General Hummer was sold by GM
beginning with the 2003 model year.
                                                                                    158

-------
August 9, 2012

MEMORANDUM
SUBJECT:      EPA Response to Comments on the peer review of LBNL Statistical Analysis of the Effect
              of Vehicle Mass & Footprint Reduction on Safety) (LBNL Phase 1 and 2 Reports),
              prepared by Tom Wenzel, Lawrence Berkeley National Laboratory.

FROM:        Cheryl Caffrey, Assessment and Standards Division
              Office of Transportation and Air Quality, U.S. Environmental Protection Agency

The LBNL Phase 1 and 2 Reports were reviewed by Donna Chen and Kara Kockelman (University of Texas
at Austin), Charles Farmer (Insurance Institute for Highway Safety, David Greene (Oak Ridge National
Laboratory), and Michael Van Auken (Dynamic Research, Inc.).

This memo includes a compilation of comments prepared by SRA International; LBNL's responses to, and
actions in response to, those comments are interspersed with the comments and noted by [LBNL].
                                                                                         159

-------
                                                        Assessment of NHTSA's Report
                 ''Relationships Between Fatality Risk, Mass, and Footprint in Model Year 2000-2007 Passenger Cars and LTVs"
                                                            (LBNL Phase 1 Report)
    1.  ASSUMPTIONS
                                            COMMENTS
Please comment on the validity of any
assumptions embedded in the LBNL
assessment analysis and the independent
casualty analysis that could affect the
projected relationship between vehicle
mass/footprint reductions and
fatality/casualty risk.  Examples might include
assumptions regarding whether recent
historical relationships between vehicle
weight, size, and safety will continue into the
future; potential future improvements in
vehicle technology and design may result in
compensatory safety benefits; and the annual
baseline fatality distribution.
[Chen and Kockelman]  [1] The report does a nice job discussing recent trends in vehicles, such as the increase
of ESC, side airbags, and light truck crash compatibility with passenger cars - which will improve safety
outcomes for all vehicles, but perhaps most significantly the smaller and lighter vehicles.. It also mentions the
phasing out of the lightest and smallest vehicles between model years 2000-2007 (but doesn't mention the
makes and models somehow), which were particularly poor safety performers in the past. However, with the
introduction of urban commuter vehicles, such as the SmartCar, Mini  Cooper, and Fiat 500m, and the growing
popularity of smaller, fuel-efficient compact vehicles following gas price increases, this trend does not seem so
obvious. Such vehicles should be discussed.

[LBNL] The vehicles mentioned were particular models of small light vehicles that had poor on-road safety
records. However as Figures 4.7 and 4.10 indicate, there are several recent car models weighing less than
2,500 Ibs that have fatality risks similar to models weighing 1,000 Ibs more, even  after accounting for all
differences in vehicle, driver and crash characteristics except mass and footprint.

[2] The simplistic logistic model employed in this analysis only accounts for two crash outcomes (fatal versus
non-fatal) and so neglects the more detailed, and ordered  nature of injury severity data,  which is unfortunate.
The model also assumes error-term homoscedasticity from one crash or individual to the next; in reality
certain vehicle types (e.g., pickups) and crash contexts (e.g., high speed crashes) have more  uncertainty
associated with their severity outcomes. It would be good to point out such limitations for readers.

[LBNL] The LBNL Phase 1 report followed the methodology of the NHTSA 2011 report, which  analyzes the
estimated relationships between vehicle mass, footprint and US fatalities per VMT.  The LBNL Phase 2 report
analyzes the relationships between vehicle mass, footprint, and casualties, defined as fatalities plus serious or
incapacitating injuries.  The effects on other injury severities (i.e. minor injuries) were not analyzed because of
the differences in reporting of injury severity in the thirteen states.
                                           [Van Auken]  The basic assumptions, methodology, and data are primarily the same as in the Kahane (2011)
                                                                                                                                     160

-------
report. These include the following:

1) The probability of a crash fatality is proportional to the vehicle miles travelled (VMT), except as noted in
   Section 5.1
2) The logarithm of probability of fatality per VMT for a given curb weight, footprint, and control variable
   values varies as a linear combination of the curb weight, footprint, and control variables within the domain
   of the data.
3) The logistic regression methods determine a maximum likelihood estimate of model coefficients.
4) It is assumed that the above relationships remain constant in the recent past (i.e., 2000-2007 model year
   vehicles in the 2002-2008 calendar years), present, and near future (i.e., 2017-2025 model year vehicles).

The first assumption that crash fatalities are proportional to VMT ratherthan the number of vehicle
registration years (VRY) is appropriate because the fatalities cannot occur if the vehicles are not driven on the
road (i.e., VMT = 0). This assumption  is qualified however because VMT is more difficult to measure than VRY
and therefore may be less accurate. On the other hand the probability of a fatal  crash or the number of
fatalities in a crash may also depend on the vehicle occupancy. The analysis in Section 5.1 is a commendable
attempt to explore the sensitivity to this assumption, however the Kahane (1997) and DRI (2003-2005) reports
have shown that some driver, vehicle, and environmental factors may be underrepresented or
overrepresented in unweighted induced-exposure data.  VRY could  have also been considered as a measure of
exposure.

[LBNL] NHTSA and LBNL used vehicle miles traveled, rather than vehicle registration-years, as the measure of
exposure because a vehicle that is not driven has zero risk.  Nonetheless, LBNL conducted a sensitivity using
vehicle registration years rather than  VMT as the measure of exposure. This alternative resulted in lower
estimated effects of mass reduction on risk in lighter cars and light trucks, no change in CUVs/minivans, but a
substantially higher estimated effect of mass reduction in heavier cars (from an estimated 0.51% increase in
risk to an  estimated 2.40% increase in risk, as shown in Alternative 6 in new Table ES.l).

The second and third assumptions are appropriate provided that it is recognized that it is essentially
impossible with currently available knowledge and information to model all of the factors that could affect the
probability of fatality in a crash, and that the objective of the analysis is to identify overall trends versus
vehicle weight and footprint.  In general the probability of fatality depends on other many other factors which
have not been modeled (e.g., driver behavior factors, vehicle design factors, roadway design factors, EMT
factors) and these unmodeled factors are assumed to  be uncorrelated with vehicle weight and footprint,
                                                                                           161

-------
                                            and/or are represented by the other control variables. The latter assumption might or might not be valid.

                                            The fourth assumption is perhaps the weakest because it assumes that future vehicles will have the same
                                            design characteristics as past vehicles, and that the characteristics of the vehicle population (e.g., collision
                                            partner weight, size, type) will also remain the same.  A commendable attempt to partially address this effect
                                            is described in Section 6. These effects can be perhaps better addressed by the "Volpe model" described in
                                            Kahane (2011) of the Honda-DRI fleet systems model  described in Refs (2324), which can be used to forecast
                                            the effects of mass reductions of individual makes and models on a year-by-year basis.
Please comment on any apparent unstated or
implicit assumptions and related caveats or
limitations.
[Chen and Kockelman] The role of driver behavior is briefly addressed in the report but not emphasized
sufficiently. Fatality risk is a combination of driver, vehicle, and roadway characteristics. Driver behavioral
differences are many and do not solely exist for pickup truck drivers versus car drivers. Socioeconomic data
such driver household income, size, and education  influence driver attitudes and driving environments. For
example, Chen et al. (2010) found that crash risk increases for those living in socioeconomically disadvantaged
areas (including households more likely to drive less expensive and older vehicles). Though such data is not
typically available in state and national crash databases, the importance of these driver and environmental
characteristics on crash rates (per mile driven) and  fatality risk should be stressed in both  reports. It is clearly
very difficult to control for, but a major caveat to the NHTSA (& now LBNL) results. We expect that crash
severity could be probably be lower for many of the small cars and pickups if they were driven by those who
tend to drive more expensive vehicles, under the same settings (e.g., daytime,  urban freeway). Similarly, in the
second LBNL report (which uses VMT estimates), we expect that crash rates would probably be lower for these
types of driver-vehicle-setting combinations.

[LBNL] We agree that it would be preferable to include additional variables that account for driver behavior
rather than just driver age and gender. However, variables such as driver age,  gender, or other socio-economic
characteristics probably have a less direct effect on risk than the actual driving  skill or behavior of individual
drivers.  The PARS data includes information on whether alcohol or drug use, or speeding or reckless driving,
was a factor in the  current crash, as well as the driver's record for the last three years. NHTSA 2003 found that
including a  "bad driver" rating variable comprised of 8 of these variables did not have an appreciable effect on
the relationship between vehicle mass and risk. Although  these variables are included in the PARS fatal crash
records, they are not included in the state crash databases, so a similar variable cannot be used in the
regression models.  However, Alternative 12 in new Table  ES.2, and the third column in new Figure 5.14,
indicates that excluding bad drivers (after also excluding reported driver alcohol or drug use) results in
additional estimated increases in fatality risk as vehicle mass decreases, and estimated decreases in risk as
                                                                                                                                       162

-------
vehicle footprint decreases.  LBNL believes that driver behavior in the current crash and their recent driving
record, and not general socio-economic information, is the best available measure to account for the effect
driver skill and behavior has on risk in individual vehicles.

Nonetheless, LBNL explored the effect of adding a control variable for one additional driver socio-economic
characteristic, household income. Unfortunately, crash data do not include household income of vehicle
drivers.  PARS includes the driver's zip code, from  his or her license, which could be merged with US Census data
by zip code to obtain the average median income of households in a particular zip code.  However, very few
states include driver zip code in their databases of police-reported crashes.

Kweon and Kockelman (2003) merged police-reported crash data from GES with household demographic data,
including household income, from the National Household Travel Survey (NHTS), to examine crash rates per
annual VMTby driver age, gender, and vehicle type. Each dataset includes population weights to derive
national estimates of crashes and households, which the authors use to estimate national crash rates for driver
age, gender and vehicle type. However, there are several limitations with the NHTS data. First, the survey
includes only 25,000 national households, supplemented by an additional 124,000 households from "add-on"
state and municipal jurisdictions. The national weights provided in the NHTS most likely do not represent the
national distribution of vehicle registrations by make and model.  Second, 30% of the households in the 2009
NHTS reported income "over $100,000"; using the income bins provided in the NHTS to estimate household
income will likely understate the actual income of the households participating in the survey.

Chen el al (2010) made use of an Australian survey of 20,000 young adult drivers that asked questions
regarding driving behavior.  The injury severity of 127 of these participants who were admitted to hospitals
after a vehicle crash was analyzed as a function of socio-economic status. The socio-economic status of the
drivers was based on average or median values from the zip code where they (or their parents, since most were
at University) lived.

Using 2010 California vehicle registration data, LBNL calculated the median household income by registered
owner's zip code, using 2000 census data, then took the average household income by vehicle make and model.
Predicted US fatality risk is correlated with this measure of average household income by vehicle model,  with
risk decreasing as income increases; the correlation is highest for cars and CUVs, with an R2 of 0.60 (or a
correlation coefficient r of 0.77) or higher, as shown in new Figure 4.20 and the first column  of new Table 4.7.
However, while income tends to increase as mass increases, the correlation is not strong (R2 is highest for 4-
door cars, at 0.16, and CUVs, at 0.19, as shown in new Figure 4.21 and the middle columns of new Table 4.7).
                                                                                          163

-------
As with predicted fatality risk, there is a wide range in the average income of households owning vehicle
models with similar mass. This analysis is summarized at the end of Section 4 of the Final Phase 1 report.

Using average household income by vehicle model, rather than driver alcohol or drug use, or reckless driving, as
an alternative indicator of driver behavior substantially reduces the estimated detrimental effect of mass
reduction on risk in cars, and slightly increases the estimated detrimental effect of mass reduction on risk in
light trucks.  This analysis is summarized at the end of Section 5.4 of the Final Phase 1 report.
[Farmer] The statistical models assume no interaction between the vehicle size/weight measures and any of
the numerous covariates, but this may not be true. For example, size/weight reductions may differently affect
vehicles with and without ESC if they affect vehicle handling. It is risky to make statements such as that on p.
11 of the Phase I report: Therefore, the mass of a lighter car could be reduced by 800 Ibs while adding ESC,
without increasing fatality risk.

[LBNL] The sentence has been revised to: "For instance, a 100-lb reduction in curb weight for an underweight
car is estimated to increase risk by 1.4%, while installing ESC would reduce risk by 11.4%; the models estimate
that the beneficial effect of adding ESC is nearly ten times that of reducing mass by 100 Ibs. "
[Van Auken]  The induced-exposure data set provided by NHTSA is based on the "non-culpable" vehicle in two-
vehicle crashes.  It is assumed that the dataset is a reprehensive sample of the driver and environmental
exposure factors for vehicle use. However, since these cases include moving vehicles, some vehicle-driver-
environmental conditions may be under or over represented in this data depending on how they affect the
ability of a non-culpable vehicle to avoid a crash. Results in Ref (17) indicate that the estimated effect of
weight and size reduction are sensitive to whether the induced-exposure data are based on the Kahane (2003)
non-culpable vehicle definition of the Kahane (1996) stopped vehicle definition.
Unfortunately it is not currently possible to test this sensitivity with the NHTSA-provided induced-exposure
data.

[LBNL] NHTSA developed vehicle miles traveled weighting factors for induced exposure cases based on stopped,
rather than non-culpable, vehicles. LBNL investigated what effect this alternative method of exposure has on
the estimated effect of mass or footprint reduction on risk, in Alternative 15 in Table ES.3 and Section 5.6 in the
Final Phase 1 report.
                                                                                           164

-------
ADDITIONAL COMMENTS:

[Chen and Kockelman] Chen, H.Y., Ivers, R.Q., Mariniuk, A.L.C., Boufous, S., Senserrick, T., Woodward, M., Stevenson, M. and Norton R. Socioeconomic status
and risk of car crash injury, independent of place of residence and driving exposure: Results from the DRIVE study. Journal of Epidemiology and Community
Health 64(10), 2010, pp. 998-1003.
                                                                                                                                   165

-------
    2.  CONTROL AND DEPENDENT
       VARIABLES USED IN THE REGRESSION
       MODELS
                                            COMMENTS
Please comment on the adequacy of control
and dependent variables used in the
assessment analysis and independent casualty
analysis, and recommend any alternative
control or dependent variables that are
available for possible inclusion in the analysis.
For example, what are the relative merits of
the main dependent variables used, fatality
risk per estimated VMT, and casualty risk per
police-reported crash?
[Chen and Kockelman] [1] As alluded to above, a primary concern is that the NHTSA analysis (& thus the LBNL
analyses) largely neglect the idea that vehicle type (make & model) is very much a proxy for driver type, and a
vehicle's crash avoidance may have very little to do with vehicle type. It has a lot to do with the person behind
the wheel, and gender & age simply aren't enough to control for such distinctions. Education, risk aversion,
ability, wealth, etc., are important covariates. But existing data sets are quite limiting (though the MVOSS &
FAR with 3-year driver violation history do offer some valuable insights, not discussed in these reports). In
reality, small cars may be less crash prone than Kahane's & Wenzel's results suggest, because they are driven
by lower-income, younger, less risk averse people driving in more crash prone settings (e.g., commercial strips
rather than pricey residential suburbs). Such key caveats need thoughtful discussion. Four relevant papers on
the topics of crash frequency and vehicle size-and-weight implications (by Knipling, Kweon & Kockelman, Wang
and Kockelman, and Chen & Kockelman) have been sent to Tom Wenzel. These all include useful  literature
reviews for further connections to useful findings for citation in the reports, as time allows the contractor.

[LBNL] As discussed above, LBNL believes that driver behavior in the current crash and their recent driving
record, and not general socio-economic information, is the best available measure to account for  the effect
driver skill and behavior have on risk in individual vehicles. As described in Section 5.4, LBNL used this measure
of driver behavior in one of its sensitivity analyses in the draft final report. Nonetheless, LBNL examined the
effect of including a measure of driver household income on risk in Section 5.4 in the Final Phase 1 report.

[2] The grouping of the vehicles into heavier- and lighter-than-average weight categories essentially splits a
"typical" weight vehicle of that type into two categories. The impacts of curb weight and footprint on fatality
risk may be easier to interpret if the vehicles were grouped into 3 weight categories (light, average, and heavy)
& by type (with the average category representing vehicles within one standard deviation of average weight).
Furthermore, the grouping of CUVs and minivans into the same vehicle type category neglects the fact that
these vehicles have faced rather different ground clearance requirements (impacting rollover potential), door
types (sliding vs. standard), and, perhaps most importantly, can appeal to different types of drivers (as
indicated in the market shift of car drivers to CUV drivers).

[LBNL] As described in NHTSA 2011, NHTSA included CUVs and minivans in the same vehicle category in part
because in most respects they are more "car-like" than conventional light-duty trucks (i.e. pickups, SUVs, and
                                                                                                                                      166

-------
                                            fullsize vans). A MINIVAN indicator variable is included in the regressions to estimate the separate effect of
                                            minivan vs. CUV mass reduction on risk.  The relatively small number of CUVs and minivans during the analysis
                                            period precludes them from being separated into two distinct groups for analysis.

                                            [Farmer] One needs to restrict control variables to those that are available and reliable. A problem when
                                            combining state databases is that the states often are  not consistent as to the variables coded and the
                                            definitions of those variables.  This severely limits the list of possible control variables.

                                            [LBNL] The control variables used from the state crash databases are either derived from the VIN, or based on
                                            the driver age and gender, or the time or location of the crash. LBNL believes that these variables are
                                            consistently coded among the thirteen states.

                                            [Van Auken]  [1] The main metric used in both the Kahane (2011) and Wenzel (2011a) reports is the total
                                            number of fatalities (except as noted). Reducing the total number of fatalities, which includes both subject
                                            vehicle occupants and collision partner fatalities, is desirable from a societal viewpoint. Fatal crash occurrence
                                            is related to the total number of fatalities, which has been used by Kahane (2003, 2011) to address concerns
                                            about double counting.

                                            [2] VMT is a good  measure of accident exposure provided that it can be accurately determined.  [Note: This
                                            peer review does not address  the Wenzel (2011b) companion report (Ref 3) which examines the risks per
                                            police-reported crash. See Ref 4 for comments on the companion report.]
What additional control variables, such as
vehicle make or model, might be included in
the regression models?
[Chen and Kockelman] [1] Vehicle height, a variable which may be more valuable than vehicle type for
similarly structured vehicles such as sedans, wagons, CUVs, and minivans, would be a valuable control variable.
In addition to a wider track, a lower center of gravity also increases vehicle stability, thereby reducing the risk
of rollover. Relevant literature & findings exist, and should be cited.

[LBNL] The propensity for a vehicle to roll over increases as its center of gravity increases (and static stability
factor decreases), while risk in frontal crashes may decline as bumper height increases (and the degree of
overlap with the bumper of a crash partner increases).  However, overall vehicle height is only a crude proxy for
these two other vehicle dimensions that are thought to correlate with safety, and therefore was not included in
the regression models.

[2] Other variables which have been found  in past studies to influence fatality risk such as seat belt use,
                                                                                                                                       167

-------
                                            roadway geometry and division type are not included in this study (which is largely a repeat of the NHTSA
                                            study, as specifically contracted by the EPA).

                                            [LBNL] Seat belt use was not included as a control variable because it is notoriously under-reported for non-
                                            fatal injuries. Roadway geometry and roadway division were not included because they are not consistently
                                            reported or coded in the state databases of police-reported crashes used to develop the induced exposure
                                            cases.

                                            [3] To account for driver characteristics that contribute to fatality risk, socioeconomic variables such as
                                            household  income, education, household size, etc. would be valuable additions. Unfortunately, both state and
                                            national crash databases typically do not include such information (outside of MVOSS). Such issues should be
                                            flagged for readers. It seems the contractor has done his duty, and the key limitations lie with the original
                                            methodology he was to essentially duplicate.

                                            [LBNL] As discussed above, LBNL believes that driver behavior in the current crash and their recent driving
                                            record, and not general socio-economic information, is the best available measure to account for the effect
                                            driver skill and behavior have on risk in individual vehicles. As described in Section 5.4, LBNL used this measure
                                            of driver behavior in one of its sensitivity analyses in the draft final report.  Nonetheless, LBNL examined the
                                            effect of including a measure of driver household income on risk in Section 5.4 in the Final Phase 1 report.
Please comment on any caveats or limitations
that these dependent variable or control
variables entail with respect to use of the
results as the basis for estimating the safety
effect of mass reduction.
[Chen and Kockelman] Please see above comment (in Assumptions section) regarding driver behavior and
environment.

[LBNL] As discussed above, an additional sensitivity analysis was run using a control variable for household
income, based on California vehicle registration data in 2010.

[Farmer] Model overspecification could be the reason for results that are non-intuitive, especially in the Phase
II analyses of police-reported crashes. Control variables may be correlated with each other or with the size
and weight variables.  For example, Figure 2.9 of the Phase I report implies that torso side airbags increase
fatality risk in CUVs.

[LBNL] It is possible that inclusion of too many control variables leads to non-intuitive results in certain cases.
This problem might be resolved by removing control variables (other than vehicle weight or footprint)  that are
                                                                                                                                       168

-------
                                            not statistically significant.  The example cited suggests that torso side airbags increase fatality risk in
                                            CUVs/minivans by an estimated 0.7%, is a small effect, and not statistically-significant. NHTSA and LBNL ran a
                                            sensitivity where non-significant control variables are removed; this sensitivity has only a small effect on the
                                            original estimates, as shown in Alternative 19 in Table ES.3 and in Section 5.6.
ADDITIONAL COMMENTS:

[Chen and Kockelman] Table 2.1 has many indicator variables labeled as "C" for continuous variable (such as ABS, ESC, AWD, DRVMALE, etc). These C's should
be removed.

[LBNL] These vehicle variables are continuous because, for some models, the VIN does not indicate whether a particular vehicle is equipped with that option or
not.  In these cases the fraction of that model that is equipped with the particular feature is used. On the other hand, the DRVMALE variable is always a discrete
variable, and is now coded as such in the table.

[Van Auken] The underlying reasons for some of the estimated effects are unknown at this time, but presumably involve driver, vehicles, environment or
accident factors that have not been controlled for in the Kahane (2011) and Wenzel analyses. See, for example, Refs 17 and 25.

[LBNL] The report notes that other vehicle,  driver, or crash characteristics may account for the remaining residual risk not explained by the control variables
included in the regression models. As discussed above, additional sensitivity analyses were run using an alternative control variable for vehicle characteristics,
initial vehicle purchase price (using values in Folk's VIN decoding software), and an alternative control variable for driver behavior, household income (based on
California vehicle registration data in 2010). The results of these sensitivities are included in Sections 5.2 and 5.4 in the Final Phase 1 report.
                                                                                                                                        169

-------
    3.  METHODOLOGY AND STATISTICS
                                             COMMENTS
Please comment on the validity and
applicability of the methodology LBNL used in
assessing the NHTSA 2011 study and its
analysis of the relationship between mass,
footprint, and risks per police-reported crash.
[Chen and Kockelman] The report assesses the NHTSA 2011 study in a fair amount of detail and seeks to
introduce some additional analyses to better examine the relationship between mass, footprint, and fatality
risks. However, due to a lack of control for very specific vehicle differences (which vary by make & sub-model),
the exclusion of driver characteristics and crash setting details (which cannot always be controlled for, but are
often correlated with vehicle type), the effects of downweighting vehicles and/or shifting vehicle styles and
sizes may be overestimated. Simply changing the vehicle on a risky driver in a high-risk setting is unlikely to
influence outcomes significantly.

[LBNL] As discussed above, additional sensitivity analyses were run using an alternative control variable for
vehicle characteristics, initial vehicle purchase price (using values in Folk's VIN decoding software), and an
alternative control variable for driver behavior, household income (based on  California vehicle registration data
in 2010). The results of these sensitivities are included in Sections 5.2 and 5.4 in the Final Phase 1 report.

[Van Auken]  The logistic  regression methods seem to be appropriate.  The confidence intervals are based on
the logistic regression Wald Chi-Square statistic, which as Kahane (2003, 2011) has demonstrated does not
include all sources of variation.  However, these confidence intervals are useful because they do provide some
indication of the uncertainty in the results.  [Note: This peer review does not address the Wenzel (2011b)
companion report, which  examines the risks per police-reported crash. See  Ref 4 for comments on the
companion report.]
Please review other statistical methods LBNL
has used in the analysis, in addition to the
logistic regression methodology.  Examples
include the alternative approaches used by
LBNL to assess NHTSA interval estimation
results, and LBNL's linear regression analysis
of actual, predicted, and residual risk by
vehicle model.
[Chen and Kockelman] [1] In the alternative measures of exposure, the author examines the effect of vehicle
manufacturer on fatality risk and treats the luxury models produced by Toyota, Honda, and Nissan as separate
manufacturers. However, domestic luxury brands (such as Cadillac & Lincoln) are categorized with their
nameplate manufacturers (GM and Ford), which appears inconsistent.

[LBNL] The analysis has been revised to include the luxury models Lexus, Acura, and Infiniti with their
nameplate manufacturers Toyota, Honda, and Nissan. An additional sensitivity was conducted using separate
indicator variables for the five luxury brands Lexus, Acura, Infiniti, Cadillac, and Lincoln. Adding variables for
the five luxury brands increases the estimated detrimental effect of mass reduction on risk for lighter cars (from
1.55% to 2.04 %), and dramatically increases the estimated detrimental effect of mass reduction on risk for
heaver cars (from 0.51% in NHTSA's preferred model, to 0.75% after including 14 major vehicle makes, to 1.80%
after including 14 makes plus the five luxury brands), as shown in Alternative 8 in Table ES.2. The  vehicle
manufacturer variables have little effect on the estimated effect of lighter light truck mass reduction, but both
                                                                                                                                       170

-------
analyses reduce the slightly beneficial estimated effect of mass reduction in heavier light trucks. Including the
manufacturer variables changes the estimated effect of mass reduction in CUVs/minivansfrom an estimated
0.38% reduction in risk to an estimated 1.62% or 1.28% increase in risk).

LBNL also examined the effect of replacing the vehicle make indicator variables with a single continuous
variable for vehicle initial purchase price, taken from the  Polk VIN decoder.  Initial purchase price can be seen as
a proxy for the general quality of vehicle design; we expect that more expensive vehicles will have lower risk
than less expensive vehicles. Replacing the vehicle make variables with the variable for purchase price slightly
reduces the estimated effect of mass reduction on lighter cars and light trucks, increases the estimated effect of
mass reduction in heavier cars (from 0.51% to 0.84%), and substantially decreases the estimated effect of mass
reduction in CUVs/minivans (from a 0.38% reduction in risk to a 0.92% reduction in risk). This analysis is
summarized in Section 5.2 of the Final Phase 1 report.
[2] The effect of calendar year variables on fatality risk may be overestimated here, since VMT is tracked by
vehicle model and not by calendar year. The trend of greatest fatality risk reductions in light trucks, CUVs and
minivans with increasing calendar year may simply be a reflection of rising gas prices in combination with the
ailing economy contributing to lower VMT (in these relatively low-fuel-economy vehicles).

[LBNL] As discussed in Section 5.3 of the LBNL Final Phase 1 report, the VMT weights NHTSA developed do not
reflect the reduction in driving that occurred in 2008 (as shown in Figure 5.9).  Therefore, the large reduction in
risk estimated by the CY08 variables likely is the result of less  driving in that year, and not the result of
widespread changes in the fleet of vehicles on the road as potential crash partners, as NHTSA has
surmised/postulated.  However, the lower VMT in CY08 does not explain the consistent trend in reduced
estimated fatality risk over the other calendar years.
[3] It is unclear how the author determined the various percentage replacements of vehicle types in the
aggressive vehicle market share shift scenario. (For example, why are 50% of SUVs replaced by CUVs and 60%
of small pickups replaced by CUVs? The CUV is a more natural replacement for an SUV, and an SUV a more
natural replacement for a pickup.)

The aggressive shift in market share scenario was selected to maximize replacement of truck-based vehicles,
such as pickups and SUVs, with car-based vehicles, such as CUVs and minivans, that tend to be safer and have
                                                                                           171

-------
higher fuel economy than truck-based vehicles.

[Van Auken]  [1] The correlations in Section 3 appear to be assessed using the Coefficient of Multiple
Determination (R2) based on a linear fit to the data (e.g., the correlation between footprint versus curb weight
in Figure 3.1 on p. 14). The linear regression model attributes the differences between the dependent variable
(vertical axis_ and the linear fit to the independent variable (horizontal axis) to random effects.  If there is no
preference as to the choice of independent and dependent variables (e.g., footprint versus curb weight, or
curb weight versus footprint), then the linear trend and R2 result would be different if the two variables were
interchanged, and having two different yet equally valid results would be undesirable.

[LBNL] The R2 values are the same if the two variables are interchanged.

[2] If the variation in the data can be attributed to both variables (e.g., footprint and curb weight), then  it
would be better to report the square of the sample correlation coefficient r2,, where r is computed according
to Eqn (1). The trend lines in these correlation figures should not be computed using a  linear regression.
Instead, the trend line should pass through the sample means (I.e. (x, y)), and have a slope equal to the ratio of
the sample standard deviations in the data (i.e., sy/sx). Therefore, the reported correlation results do not
depend on the ordering of the data variables.

[LBNL] Again, the R2 values are the same if the two variables are interchanged. The R2 values in Figures 3.1 and
3.2, and Figures 4.1 through 4.5, have been replaced with the correlation coefficient r, and a new Table 3.1 has
been added to Section 3 that lists r and VIFfor curb weight and footprint by vehicle type.

Note this comments does not apply to linear trends indicated  in Section 4, for which the Coefficient of Multiple
Determination (R2) seems appropriate.

[3] The Coefficient  of Multiple Determination (R2) is frequently used in the Wenzel  (2011a) report as an
indicator of the statistical importance of a linear trend (e.g., lvalues in Tables 4.1 and 4.2 that are greater
than 0.3 are  shown  in blue font).  It would be better to report the standard error, confidence interval, and/or
probability value as measures of the statistical significance of a linear trend.

[LBNL] Whether the estimated relationships shown in Tables 4.1 and 4.2 in the Final Draft report are
statistically significant has been added to the tables (see new Tables 4.2 and 4.4 in the Final Phase 1 report).
However, the point of the section is to indicate that, while the relationship may be statistically significant on
                                                                                           172

-------
                                            average, as indicated by the confidence level around the estimated effect, there remains a large range in risk
                                            for individual vehicle models with similar mass, as indicated by the relatively low R2 values.
Please comment on caveats or limitations of
using non-significant regression estimates to
project the safety impact of mass reduction.
[Chen and Kockelman] First, the t-statistics are not provided in the report which makes it difficult for the
reader to assess statistical significance of specific regression  estimates (except where noted by the author).
Second, inclusion of a statistically insignificant variable can influence the estimates of coefficients associated
with related variables. Nevertheless, in general, it is best to keep insignificant estimates if one has a strong
defense for their role, since removing such variables (& thus  their parameters) will shift the burden of
response to a correlated covariate's parameter, thus biasing  the latter. We generally keep key covariates in a
model  up to a pvalue of 0.20 or 0.25 or so, especially in relatively small data sets (e.g., n < 1,000). Covariates
for which we have no strong basis can  be removed for p values > 0.10.

[LBNL] The 95% confidence intervals are included in every figure in the document, to indicate whether the
estimated variable is statistically different from zero.

NHTSA is planning to run a sensitivity where non-significant control variables are removed; LBNL will run similar
sensitivities for its revisions to its Phase 1 and Phase 2 reports.
                                            [Van Auken]  Regression estimates are random numbers which have an unknown expected value and variance,
                                            and known sample value and standard error. If the sample value can be explained by a zero expected value
                                            and known standard error then the result is considered not statistically significantly different than zero and
                                            therefore the result is not considered to be statistically significant.  However, If we can combine this estimate
                                            with other estimates then the unknown expected values and variances can also be combined using the same
                                            transformation, and the statistical significance of the combined result can be tested. Therefore, depending on
                                            the sample values and inter-correlation, the combined result may be statistically significant even if the
                                            individual estimates are not statistically significant.

                                            For example, the results from each of the nine different crash types can be combined into an overall estimate
                                            and the standard error calculated assuming that the results for each crash type are independent of each other.
                                            Then the statistical significance of the combined effect can be determined.

                                            However, and Kahane (2011) points out there are two sources of uncertainty in the regression results. The
                                            first is the FARS based sampling error which is uncorrelated across crash types because they are based on
                                                                                                                                        173

-------
                                            different fatal cases (Kahane 2011, p. 77).  The second is the state based include-exposure sampling error
                                            which is correlated across crash types because they are based on the same included-exposure cases.
                                            Therefore a confidence interval estimated using the jackknife method described by Kahane (2011) and
                                            accounting for correlation of these two error sources would be more accurate than a simple estimate based on
                                            the Wald Chi-Square statistic and assumed independence.

                                            [LBNL] LBNL agrees that NHTSA's jack-knife method is a preferable method of estimating confidence intervals;
                                            however, the jack-knife method obtains even larger estimates of uncertainty than using the standard errors
                                            output from the regression models.
How might the LBNL methodology be
strengthened to better represent future
vehicle designs and reduce multi-collinearity
between mass and footprint in the regression
analysis?
[Chen and Kockelman] Including more vehicle-specific characteristics (such as vehicle height and engine size)
reduces the analysis' dependence on vehicle type, since vehicle shapes and structures will continue to evolve.
There is also correlation with context (e.g., pickups are driven in more rural locations, with greater hazards
[like less lighting, higher speed, & few medians]). Disaggregate data are almost always best, to avoid ecological
fallacies & such.

[LBNL] As discussed above, overall vehicle height is only a crude proxy for two vehicle dimensions, center of
gravity and bumper height, that are thought to correlate with safety in rollover and two-vehicle crashes,
respectively, and therefore was not included in the regression models. Other vehicle attributes that might
affect risk are engine power-to-weight ratio, braking distance, and handling capabilities; LBNL may examine
the effect of these accounting for these vehicle attributes in future analyses.

LBNL's analysis does account for whether a vehicle is driven in a rural area, defined as a county in which the
population density is less than 250 residents per square mile of land area.
                                            [Van Auken]  [1] The effects of multi-collinearity can be mitigated by 1) obtaining more data, 2) pooling data
                                            from different crash type or vehicle types, or 3) reducing the number of regression variables. The first option
                                            would require more calendar years and/or model years, which would involve added newer data as it becomes
                                            available (or using older data). The second option might be to recombine the CUVs and minivans with truck
                                            based vans and  adding a control variable to compensate for the differences in the vehicles types.  The third
                                            option might involve removing statistically insignificant control variables or removing control variables that
                                            would not be expected to have an effect on the probability of fatality in the crash (e.g., the side airbag variable
                                                                                                                                       174

-------
                                            is not included in pedestrian crashes because it is not expected to affect pedestrian fatality risk). The number
                                            of driver age control variables might be reduced from eight to three (as in the Kahane (1997) and DRI (2002-
                                            2005) studies).  Finally, a linear curb weight model instead of a two-piece linear model may help to better
                                            elucidate the general trend.

                                            [2] The Variance Inflation Factor (VIF) has been suggested as a measure of multi-collinearity in the Kahane
                                            (2010 and 2011) reports, however this diagnostic metric does not account for differences in database size (i.e.,
                                            Options 1 and 2 above). The Wenzel (2011a) report does not discuss the Variance Inflation Factor or report
                                            any VIF results.

                                            [LBNL] A table of VIF results, Table 3.1, has been added to the Final Phase 1 report.
ADDITIONAL COMMENTS:

[Chen and Kockelman] On page 55, it is unclear what is meant by "however; if anything, reduction of this type of fatality will increase detrimental effect of
mass reduction in cars."

[LBNL] Full adoption of ESC is expected to reduce the number of, and fatalities in, rollovers and one-vehicle crashes. Because rollovers and one-vehicle crashes
are the only types of crashes in which mass reduction is estimated to reduce fatality risk, reducing the number of these fatalities by full adoption of ESC
increases the net estimated effect of mass reduction over all types of crashes. Similarly, full adoption of side airbags is estimated to reduce the number of
fatalities in side impact crashes, as least for cars (Figure 6.4). However, Figure 6.7 indicates that the estimated effect of mass reduction on risk when a car is
struck in the side is the same or lower than when a car is involved in a fatal crash.  Therefore the reduction in the number of side impact fatalities expected from
full adoption of side airbags will likely increase the estimated effect of mass reduction on risk across all types of crashes.
                                                                                                                                       175

-------
    4.  DATA SETS
                                            COMMENTS
Please comment on the validity and
applicability of the datasets used to project
changes in risk resulting from reduction in
vehicle mass. LBNL's casualty analysis used
police-reported crash data from 16 states,
while the 2011 NHTSA study used national
fatality data, combined with a subset of non-
culpable vehicles involved in two-vehicle
crashes from police-reported crash data from
13 states.
[Chen and Kockelman]  [1] The acquisition of Polk data for VMT estimates by make & model is valuable, and a
contribution to the literature. However, these estimates come from vehicles found in repair shops in
non-attainment areas, and so will be biased towards problem-prone vehicles, wealthier households who
service their vehicles more regularly, and/or urban (smoggier) areas. Such issues merit careful discussion in the
paper, so that readers are well aware of caveats.

[LBNL] The largest source of vehicles in the Polk odometer data are vehicles that report for a regularly-
scheduled vehicle emission inspection and maintenance (I/M) program inspection. In urban areas with poor air
quality, all eligible vehicles are required to report for inspection every one or two years.  While the Polk VMT
data may be skewed towards urban areas (which typically include suburban areas, and sometimes rural areas,
depending on the state I/M program), they are not biased towards problem-prone vehicles or those owned by
wealthier households.
                                            [2] Related to this, Tom Wenzel indicated (by phone) that he did take a look at CA's extensive odometer reads,
                                            which go into some semi-rural  locations (not too rural), and he indicated that the VMT values by vehicle type
                                            (not controlling for HH attributes & such) are very similar (just 5% longer in rural areas) -except for vans
                                            (which are used much more extensively in rural areas).  This is interesting to me, and is not that different from
                                            what we've seen in the past. For example, Kockelman & Zhao's JTS paper from 2000 (pre- print at
                                            http://www.ce.utexas.edu/prof/kockelman/public_html/
                                            BTSJournalLDTs.pdf) suggests that, after controlling for various HH attributes & vehicle types, density is/was
                                            still very important (tables 1 &  2), but a shift from a density of Ik to 5k persons per sq mile (which is 1.5 vs. 7.8
                                            persons per acre) means an increase of 750 mi/yr/vehicle (which is about 7.5% of annual VMT). Such
                                            differences, and their practical  significance (or lack thereof) should be discussed in the reports.

                                            [LBNL] The vehicle mile traveled weights NHTSA assigns to each vehicle in the induced exposure dataset are
                                            based on the average odometer reading by vehicle make, model, and year from a database of odometer
                                            readings provided by Polk. Many of these odometer readings were obtained through state-run emissions
                                            inspection/maintenance programs, which tend to operate in urban areas of states with poor air quality. In a
                                            previous analysis LBNL (LBNL 2011a) found that vehicles registered in rural counties (<200 population per
                                            square mile) are driven between 2% and 8% more miles than vehicles registered in urban counties (>200
                                            population per square mile), depending on vehicle type (full size vans registered in urban counties are driven 3%
                                                                                                                                      176

-------
                                           more miles than those registered in rural counties; however this difference is not statistically significant).
                                           Therefore the Polk database of odometer readings is likely skewed towards vehicles driven in urban areas,
                                           which tend to be driven fewer miles than comparable vehicles registered in rural areas.

                                           An additional limitation of the VMT weights developed by NHTSA is that they do not account for the reduction
                                           in miles driven in response to higher gas prices and the economic recession in 2008, as noted in Section 5.3 of
                                           LBNL's Final Phase 1 report. However, despite these limitations, the average VMT weights NHTSA has
                                           developed are an improvement over the averages used in the 2003 analysis.
                                           [Van Auken] The inducted-exposure data set provided by NHTSA is based on the non-culpable vehicles in two-
                                           vehicle crashes.  See the comments in Table 1 on the limitations of this data. In addition, there are also many
                                           differences in the coding variables and values used by the different states, which tend to make the receding to
                                           a common data set imprecise. [Note: This peer review does not address the Wenzel (2011b) companion
                                           report, which examines the risks per police-reported crash. See Ref 4 for comments on the companion report.]

                                           [LBNL] As discussed above, NHTSA has developed vehicle miles traveled weighting factors for induced exposure
                                           cases based on stopped, rather than non-culpable, vehicles. LBNL summarized the effect this alternative
                                           method of exposure has on the estimated effect of mass or footprint reduction on risk in Alternative 15 in Table
                                           ES.3, and Section 5.6, in the Final Phase 1 report.
Please comment on any apparent, unstated,
or implicit impact on estimated risks inherent
in the two different approaches, and any
related caveats or limitations. For example,
what are the strengths and weaknesses of the
two measures of vehicle exposure, miles of
vehicle traveled scaled up from crash data
from 13 states, and number of police-reported
crashes?
[Chen and Kockelman]  The use of non-culpable vehicles in two-vehicle crashes as a proxy for vehicles which
are "just there" may be distorting the overall distribution of vehicle models. VMT may differ between vehicles
that are more prone to  run-off-road accidents, at-fault two-car crash vehicles, and non-culpable vehicles.

[LBNL] As discussed above, NHTSA has developed vehicle miles traveled weighting factors for induced exposure
cases based on stopped, rather than non-culpable, vehicles. LBNL summarized the effect this alternative
method of exposure has on the estimated effect of mass or footprint reduction on risk in Alternative 15 in Table
ES.3, and Section 5.6, in the Final Phase 1 report.
                                           [Van Auken] [1] The number of fatal cases tends to be much less than the number of induced-exposure cases.
                                           Therefore the effective numbers of degrees-of-freedom in the statistical estimates tend to be limited by the
                                                                                                                                     177

-------

available number of fatal cases. For example, it would not be possible to estimate the effects of two variables
(e.g., just curb weight and footprint) if we had data for only one fatal case even if we had thousands of
induced-exposure cases. Therefore it is desirable to use data for the entire US in order to get a large sample of
fatal cases for the logistic regressions. This then requires the available induced-exposure data (i.e., from 13
states) to be "scaled up" the US level using the method described in Kahane (2003 and 2011). The result is the
best currently available estimate of vehicle exposure.
[2] There may be some concerns about the accuracy of the vehicle miles-traveled data because the difficulty
estimating the number of vehicle miles travelled at the make-model-year=state level of detail. [Note: This
peer review does not address the Wenzel (2011b) companion report, which examines the risks per police-
reported crash. See Ref 4 for comments on the companion report.]
ADDITIONAL COMMENTS:
178

-------
    5.  RECOMMENDATIONS
                                             COMMENTS
Please comment on whether the LBNL
assessment adequately addresses the NHTSA
2011 study and identifies the safety impact
from mass reduction. Are the analytic
methods and data used to assess the NHTSA
study, and estimate the relationship between
risk, mass, and footprint, appropriate? Is
casualty risk per crash a legitimate measure of
vehicle safety?  What other methods or data
could be  used to better predict the effect of
future vehicle designs on safety?
[Chen and Kockelman] While driver fatalities per crash seems a useful measure of vehicle design safety, and
examination of fatal crash rates is very valuable (using Polk- based exposure estimates), there are many
caveats to work of this type. As noted above: a primary concern remains a neglect of the notion that the type
of car is very much a proxy for driver type, and a vehicle's crash avoidance may have very little to do with
vehicle type. It has a lot to do with the person behind the wheel. Simply including gender and age variables
cannot account for important covariates such as education, risk aversion, driving ability, wealth, etc. In reality,
small cars may be less crash prone than Kahane's and Wenzel's results suggest, because they are driven by
lower-income, younger, less risk averse people driving in more crash prone settings (e.g., commercial strips
rather than pricey residential suburbs). Of course, as noted above, it is very difficult to control for all these
variables, and the contractor was asked to rely on the original data.  In reality, the best the report authors can
do with such data sets is to explain how all the other, relevant attributes may factor in (e.g., quality of driver
and typical driving settings), and how they can generate biased estimation (sometimes in either direction).
Discussion of relevant literature that looks more deeply at crash outcomes (e.g., Wang or Chen's papers,
mentioned above, allowing for heteroscedasticity and individual vehicle attributes, non-driver outcomes, etc.)
will also be useful.

[LBNL] As discussed above, additional sensitivity analyses were run using a control variable for household
income, based on California vehicle registration data in 2010 (see Section 5.4 in the Final Phase 1 report).  And
sensitivity analyses were also run using a control variable for initial vehicle purchase price as a proxy for quality
of vehicle design, using values in Folk's VIN decoding software (see Section 5.2 in the Final Phase 1 report).

[Van Auken] [1] The basic methodology described by Kahane (2011) seems appropriate; however some
results using this method and data are not well understood and need further diagnosis.

[2] The induced-exposure data set provided by NHTSA is based on the non-culpable vehicles in two-vehicle
crashes.  See the Table 1 comments on the limitations of this data. [Note: This peer review does not address
the Wenzel (2011b) companion report, which examines the risks per police-reported crash.]

[LBNL] As discussed above, NHTSA has developed vehicle miles traveled weighting factors for induced exposure
cases based on stopped, rather than non-culpable, vehicles.  LBNL summarized the effect this alternative
method of exposure has on the estimated effect of mass or footprint reduction on risk in Alternative 15 in  Table
ES.3, and Section 5.6, in the Final Phase 1 report.
                                                                                                                                      179

-------
Please comment on the overall adequacy of
LBNL's assessment of the 2011 NHTSA report
and its independent study of casualty risk for
predicting the effect of vehicle mass or
footprint reduction on safety. Provide any
recommended improvements that might
reasonably be adopted by the author to
improve the analysis.
[Chen and Kockelman] Overall, the study is a comprehensive assessment of the 2011 NHTSA report and
introduces interesting additional analyses to examine the relationship of vehicle mass and footprint reduction
on safety. However, as stated previously in the comments here, driver preference for specific car types
(including size and mass) is related to driver socioeconomic characteristics and driving behavior. As vehicle,
driver, and roadway environment characteristics all contribute to fatality risk, the effects of physical vehicle
changes such as mass or footprint reduction on safety should not  be overstated when the other two types of
characteristics are not  sufficiently accounted for.

[Farmer] Overall these are reasonably good studies.  The Phase I report does a very good job of assessing the
NHTSA report of fatality risk.

[Van Auken] The Wenzel (2011a) report provides a valuable supplement to the analysis and results in the
Kahane (2011) report.  [Note: This peer review does not address the Wenzel (2011b) companion report, which
examines the risks per  police-reported crash.]
ADDITIONAL COMMENTS:
[Van Auken] See attached tables 6 and 7 below.
                                                                                                                                    180

-------
                                  Table 6. Additional General Comments and Recommendations
                                                          Mike Van Auken
 Section
                                          COMMENTS AND RECOMMENDATIONS
           Use of R2 is confusing. Suggest using lower case "r" when referring to the sample correlation coefficient (Box, Hunter, Hunter, 1978,
           P. 61); or upper case R when referring to the regression coefficient of multiple determination (Draper and Smith, 1981, p. 90).

           [LBNL] Figures 3.1 and 3.2 and 4.1 through 4.5 report r and not R2 in the Final Phase 1 report. In addition, a new Table 3.1 with r and
           VIF, and a new Table 4.1 with r and R2, have been added.
All
In most cases the reported results are just estimates, but are not described as such. For example, "The effect of mass reduction on
heavier cars and CUVs and minivans are not statistically significant" on p. iii should say "The estimated effect of mass reduction on
heavier cars and CUVs and minivans are not statistically significant."

This distinction  is important when comparing results based on different models and assumptions because the different models and
assumptions do not change the effect itself, but rather the estimate of the effect. For example, the statement "The first sensitivity,
in dark purple, includes the weight variables in the regression model but excludes the footprint variable; this model tests the effect
of mass reduction while allowing footprint to vary with vehicle mass.  This sensitivity increases the risk from a 100-lb mass reduction
in cars (from 1.43% to 2.64% for lighter cars, and from 0.48% to  1.94% for heavier cars) and CUVs/minivans (from a 0.47% decrease
in risk to a 0.52% increase in  risk); however, there is no change in fatality risk in light-duty trucks" on page 15 is misleading.  It would
be better to state that "The first sensitivity, in dark purple, includes the weight variables in the regression model but excludes the
footprint variable; this model tests the estimated effect of mass reduction while allowing footprint to vary with vehicle mass. Th4&
GGnsitivity/?emoi//'ng the footprint variable from the regression model increases the estimated risk from a 100-lb mass reduction in
cars (from 1.43% to 2.64% for lighter cars, and from  0.48% to 1.94% for heavier cars) and  CUVs/minivans (from a 0.47% decrease in
risk to a 0.52% increase in risk); however, there is nothe change  in  the estimated fatality risk in light-duty trucks is very small and
not statistically significant."

This also applies to table and figure captions. For example, "Table  ES.l. Effect of mass and footprint reduction on fatality risk, under
alternative regression model specifications" should say "Table ES.l. Estimated effects of mass and footprint reduction on fatality
risk, under alternative regression model specifications."  "Figure 3.3 Effect of reduction in mass or footprint on US fatality risk per
VMT, by vehicle type: mass only, footprint only, and both" should say "Figure 3.3 Estimated effects of reduction in mass or footprint
on US fatality risk per VMT, by vehicle type: mass only, footprint only, and both."

Overall the word "effect" appears over 200 times in  this report with the "estimated" or other qualifier. In some cases this may be
                                                                                                                                181

-------
appropriate and in other cases it is not appropriate.  It is recommended that the author review each instance and revise as
appropriate.

[LBNL] The word "estimated" will be used extensively in  the final report.  In addition the following text was added in the Executive
Summary, Section 1, and Section 7:

Although the purpose of the NHTSA and LBNL reports is to estimate the effect of vehicle mass reduction on societal risk, this is not
exactly what the regression models are estimating.  Rather, they are estimating the recent historical relationship between mass and
risk, after accounting for most measurable differences between vehicles, drivers, and crash times and locations. In essence, the
regression models are comparing the risk of a 2600-lb Dodge Neon with that of a 2500-lb Honda Civic, after attempting to account
for all other differences between the two vehicles.  The models are not estimating the effect of literally removing 100 Ibsfrom the
Neon, leaving everything else unchanged.

In addition, the analyses are based on the relationship of vehicle mass and footprint on risk for recent vehicle designs (model year
2000 to 2007).  These relationships may or may not continue into the future as manufacturers utilize new vehicle designs and
incorporate new technologies, such as more extensive use of strong lightweight materials and specific safety technologies. Therefore,
throughout this report we use the phrase "the estimated effect of mass (or footprint) reduction on risk" as shorthand for "the
estimated change in risk as a function of its relationship to mass (or footprint) for vehicle models of recent design."
Figures 4.1 through 4.17 do not control for the effect of vehicle size (e.g. footprint), which has been shown to be correlated with
vehicle weight (e.g., Figure 3.1), and therefore these figures may be misleading. It is strongly suggested that the horizontal axis label
be changed to "Curb weight (Ibs) and corresponding changes in size," and/or a note such as the following be added to each figure:
"Note these results do not control for the effect of vehicle size on fatality risk.  Therefore the horizontal axis represents changes to
both vehicle weight and vehicle size."

[LBNL] The figures, as well as Tables 4.2 and 4.4, show the relationship between risk and vehicle mass, before and after accounting
for all modelled differences across vehicle models except mass and footprint, and are labelled as such. A new Table 4.3 has been
added to the final report that compares predicted risk after accounting for all modelled differences across vehicle models except mass
(but including footprint), and a new Table 4.5 shows the relationship between predicted risk and footprint after accounting for all
differences except footprint (but including mass); this increases the estimated risks in some cases, makes some statistically
significant, but does not appreciably reduce the range in risk by vehicle model.
The statistical significance of the linear trends in Figures 4.1 through 4.17 are not reported. It would be helpful if the confidence
                                                                                                                       182

-------
intervals or statistical significance of the linear trends were reported, either in addition to or instead of R2.
The confidence intervals for the estimated slopes should be added to the results in Tables 4.1 and 4.2.

[LBNL] Tables 4.2 through 4.5 in the Final Phase 1 report summarize the linear trends and statistical significance of Figures 4.1
through 4.17.
                                                                                                                        183

-------
                                 Table 7. Additional Specific Comments and Recommendations
                                                        Mike Van Auken
   Section
   Page
                                COMMENTS AND RECOMMENDATIONS
Executive
Summary, 7
   iii, 65
                             -,nd
2n paragraph refers to "our analysis," however the results are the same as the NHTSA analysis. The author
should clarify who or what "our analysis" refers to and how it relates to the NHTSA analysis. Perhaps the
statement "LBNL was able to reproduce the NHTSA analysis, which finds that..." would be more appropriate.

[LBNL] The suggested change has been made.
Executive
Summary
    IV
Last bullet - suggest changing the statement that "Logistic regression does not allow a statistic" to "Logistic
regression methods do not have a statistic."

[LBNL] The suggested change has been made.
Executive
Summary, 4
iv, v, 22, 66
Suggest changing "variance in risk" to variation in risk" throughout.

[LBNL] The suggested change has been made.
Executive
Summary, 7
  viii, 69
The numerical results for the NHTSA preferred model in Tables ES.l and 7.1 are slightly different than the results
reported in the NHTSA report.  For example 1.43%/0.48%/0.52%/-0.40%/-0.47% should be 1.44%/0.47%/0.52%/-
0.39%/-0.46%

[LBNL] We are not sure why LBNL's results are slightly different from NHTSA's; perhaps due to rounding
differences, or the factor LBNL used to convert log-odds ratios to probabilities, as mentioned in the Final Phase 1
report.
                  31-32
            Figures 4.12 through 4.14 have the results for small and heavy-duty pickups combined, which is inconsistent with
            the results in Table 4.1

            [LBNL] The figures in the Final Phase 1 report present the data for small and heavy-duty pickups separately.
                    35
            The R2 values in Table 4.1 are different than the values in Figures 4.6, 4.8, 4.9, 4.11.

            [LBNL] The R2 values in the bottom four rows of the table are incorrect; these have been corrected in the Final
            Phase 1 report.
                                                                                                                             184

-------
5.1
37
The subsection title should be "Alternative measures of exposure and outcome" because fatal crashes and
fatalities are measures of the crash outcome, not exposure.

[LBNL] The title has been changed to "Alternative measures of risk"
5.2
39
It would be helpful to list the 18 manufacturer dummy variables in a table.

[LBNL] The indicator variables for the vehicle manufacturers have been added to the final report.
5.2
39
It is unclear why Lexus, Acura, and Infinity are treated as separate manufacturers, but Cadillac and Lincoln are
not.

[LBNL] The results of two regressions have been included in the Final Phase 1 report: one including the five luxury
brands in their parent manufacturers, and one accounting for each of the five luxury brands (both in Figure 5.2).
Accounting for the five luxury brands substantially increases the estimated detrimental effect of car mass
reduction on risk, as shown in Section 5.3 in the Final Phase 1 report.
5.2
39
It is unclear why AM General is considered a Chrysler brand. The AM General Hummer was sold by GM
beginning with the 2001 model year.

[LBNL] AM General has been removed from the Chrysler brand and included in the Other category.
5.3
44
It would be helpful if the figures include error bars or shading to indicate the confidence intervals.
6.4
63
Table 6.3 - Suggest adding a note that the 72,316 total includes fatalities that are counted more than once in
crashes involving more than one vehicle type.

[LBNL] Table 6.3 is mis-labelled; it represents the total number of fatalities between 2004 and 2008 for model
year 2000 to 2007 light duty vehicles, not the average annual fatalities. The text in the paragraph above explains
that the number of fatalities in mulit-vehicle crashes is divided by the number of case vehicles, to avoid double-
counting of fatalities.  The title of Table 6.3 has been corrected in the Final Phase 1 report, and VMTfor each
vehicle type added.
                                                                                                                                185

-------
                       An Analysis of the Relationship between Casualty Risk Per Crash and Vehicle Mass and Footprint
                                                 for Model Year 2000-2007 Light-Duty Vehicles
                                                             (LBNL Phase 2 Report)
    1.  ASSUMPTIONS
                                             COMMENTS
Please comment on the validity of any
assumptions embedded in the LBNL
assessment analysis and the independent
casualty analysis that could affect the
projected relationship between vehicle
mass/footprint reductions and
fatality/casualty risk.  Examples might include
assumptions regarding whether recent
historical relationships between vehicle
weight, size, and safety will continue into the
future; potential future improvements in
vehicle technology and design may result in
compensatory safety benefits; and the annual
baseline fatality distribution.
[Chen and Kockelman] [1] The Phase 2 report serves as a complimentary document to the Phase 1 report by
isolating the effect of vehicle mass and footprint on crashworthiness. Whereas the Phase 1 report analyzes
fatality risk per estimated VMT, the Phase 2 report analyzes casualty risk per crash. The parallel structure of
the two reports makes it easy for the reader to compare the results of the two analyses.

[2] The binary logistic model employed in this analysis can only account for two injury outcome categories;
here it is used to distinguish crashes resulting in serious injury or death from all other crash outcomes. Thus,
the model does not account for the ordinal nature of injury severity and neglects the difference between a
serious injury and a death.

[LBNL] The LBNL Phase 2 report analyzes the relationships between vehicle mass, footprint, and casualties,
defined as fatalities plus serious or incapacitating injuries. The effects on other injury severities (i.e. minor
injuries) were not analyzed because of the differences in reporting of injury severity in  the thirteen states.

[3] The report states that  "a serious incapacitating injury can be just as traumatic to the victim and her family,
and costly from an economic perspective, as a fatality." While serious injuries are very costly to society (and
may have similar economic cost implications as deadly crashes), willingness to-pay estimates (which include
pain and suffering) price the cost of a fatality at almost 20 times the cost of  an incapacitating injury (NSC
2010). Thus, it is difficult to assess the economic cost of the estimates of increases in casualty risk per crash
without distinguishing whetherthat outcome is a serious injury or a death. This limitation of the model should
be addressed in the report.

[LBNL] The focus of the analysis is to estimate the relationship between changes in vehicle mass or footprint on
fatality or injury risk.  Estimating the economic cost of any estimated increase in risk is beyond the scope of the
analysis.
                                            [4] The logistic model also assumes error-term homoscedasticity and cannot account for increases and
                                                                                                                                      186

-------
decreases in the variation of injury outcomes due to vehicle and driver type, for example. Such limitations of
the model should be discussed.

[LBNL] As mentioned above, the effects on other injury severities (i.e. minor injuries) were not analyzed because
of the differences in reporting of injury severity in the thirteen states.

[Farmer] The report concludes "that much of the detrimental effect of mass or footprint reduction on risk can
be attributed to the tendency for mass or footprint reduction to increase crash frequency, rather than to
reduce vehicle crashworthiness (risk once a crash has occurred)." However, the interpretation of casualties
per crash as inversely proportional to crashworthiness ignores the possibility that injury severity also depends
upon the circumstances of the crash. Casualties per crash must be divided into casualties per severe crash and
severe crashes per crash, where a severe crash would be one involving more energy, e.g., high-speed or
rollover.  It could be that weight reduction increases casualties per severe crash (i.e., reduces
crashworthiness), but reduces the likelihood that a crash is severe.

[LBNL] The state crash data do not consistently provide a measure of crash severity, such as an accurate
estimate of travel speed, to change the measure of exposure from all police-reported crashes to all severe
police-reported crashes. Figure 2.10 in the Final Phase 2 report shows that using casualty, as opposed to all,
crashes as the measure of exposure does change the estimated detrimental effect of mass reduction on fatality
risk per crash

[Van Auken ] The basic assumptions, methodology, and data are primarily the same as in the Kahane (2011)
report, but have been extended to include serious injuries as well as fatalities, and  also address crash
involvement (i.e., fatalities and serious injuries per accident, and also accidents per VMT).. These include the
following:

1) The probability of a  crash fatality or serious injury is proportional to the  number of accidents (provided the
   crash conditions remain the same); and the probability of an accident is proportional  to the vehicle miles
   travelled (VMT).
2) The logarithm of probabilities of fatality or serious injury per accident, and accidents  per VMT for a given
   curb weight, footprint, and control variable values varies as a linear combination of the curb weight,
   footprint, and control variables within the domain of the data.
3) The logistic regression methods determine a maximum likelihood estimate of model coefficients.
4) It is assumed that the above  relationships remain constant in the recent past  (i.e., 2000-2007 model year
                                                                                           187

-------
                                               vehicles in the 2002-2008 calendar years), present, and near future (i.e., 2017-2025 model year vehicles).

                                           The first assumption that crash fatalities and serious injuries are proportional to the number of accidents
                                            provided the crash conditions remain the same seems self-evident (e.g., if two fatal crashes had exactly the
                                            same conditions, then the expected number of fatalities for the two crashes would be twice the value for just
                                            one of the crashes). The assumption that the number of accidents are proportional to VMT rather than the
                                            number of vehicle registration years (VRY) is also appropriate because accidents cannot occur if the vehicles
                                            are not driven on the  road (i.e., VMT = 0).  This assumption is qualified however because VMT is more difficult
                                            to measure than VRY  and therefore may be less accurate.  On the other hand the probability of a fatal crash or
                                            the number of fatalities in a crash may also depend on the vehicle occupancy.

                                           The second and third assumptions are  appropriate provided that it is recognized that it is essentially
                                            impossible with currently available knowledge and information to model all of the factors that could affect the
                                            probability of fatality  in a crash, and that the objective of the analysis is to identify overall trends versus
                                            vehicle weight and footprint. In general the probability of fatality depends on other many other factors which
                                            have not been modeled (e.g., driver behavior factors, vehicle design factors, roadway design factors, EMT
                                            factors), and these unmodeled factors are assumed to be uncorrelated with vehicle weight and footprint,
                                            and/or are represented by the other control variables. The latter assumption might or might not be valid.

                                           The fourth assumption is perhaps the weakest because it assumes that future vehicles will  have the same
                                           design characteristics  as past vehicles,  and that the characteristics of the vehicle population (e.g., collision
                                           partner weight, size, type) will also remain the same.
Please comment on any apparent unstated or
implicit assumptions and related caveats or
limitations.
[Chen and Kockelman] The role of driver behavior is briefly addressed in the report but not emphasized
sufficiently. Casualty risk is a combination of driver, vehicle, and roadway characteristics. Whereas vehicle
characteristics significantly influence crashworthiness, driver behavioral differences play a significant, if not
primary, role in determining crash frequency. Socioeconomic data such driver household income, size, and
education influence driver attitudes and driving environments. For example, Chen et al. (2010) found that
crash risk increases for those living in socioeconomically disadvantaged areas (including households more likely
to drive less expensive and older vehicles). Though such data is not typically available in state and national
crash databases, the importance of these driver and environmental characteristics on crash rates (per mile
driven) and casualty risk should be stressed in both reports. It is clearly very difficult to control for, but is a
major caveat to the NHTSA (& now LBNL) results. We expect that crash severity could be probably be lower for
many of the small cars and pickups if they were driven by those who tend to drive more expensive vehicles,
                                                                                                                                      188

-------
under the same settings (e.g., daytime, urban freeway). Thus, statements like "a 100-lb reduction in the mass
of lighter cars leads to a 1.84% increase in crash frequency" should be accompanied by an explanation of the
possibility of the mass variable accounting/proxying for effects of lower income households owning smaller
vehicles.

[LBNL] We agree that it would be preferable to include additional variables that account for driver behavior
rather than just driver age and gender. However, variables such as driver age, gender, or other socio-economic
characteristics probably have a less direct effect on risk than the actual driving skill or behavior of individual
drivers.  The PARS data includes information on whether alcohol or drug use, or speeding or reckless driving,
was a factor in the current crash, as well as the driver's record for the last three  years. NHTSA 2003 found that
including a "bad driver" rating variable comprised of 8 of these variables did not have an appreciable effect on
the relationship between vehicle mass and risk.  Although these variables are included in the PARS fatal crash
records, they are not included in the state crash databases, so a similar variable cannot be used in the
regression models. However, Alternative 12 in Tables ES.2 and 7.1, and the third column in Figure 5.14, in the
Final Phase 1 report indicates that excluding bad drivers (after also excluding reported driver alcohol or drug
use) results in additional estimated increases in fatality risk as vehicle mass decreases, and estimated decreases
in risk as vehicle footprint decreases. LBNL believes that driver behavior in the current crash and their recent
driving record, and not general socio-economic information, is the best available measure to account for the
effect driver skill and behavior has on risk in individual vehicles.

Nonetheless, LBNL explored the effect of adding a control variable for one additional driver socio-economic
characteristic, household income.  Unfortunately, crash data do not include household income of vehicle
drivers.  PARS includes the driver's zip code, from his or her license, which could be merged with US Census data
by zip code to obtain the average median income of households in a particular zip code. However, very few
states include driver zip code in their databases of police-reported crashes.

Kweon and Kockelman  (2003) merge police-reported crash data from GES with household demographic data,
including household income, from the National Household Travel Survey (NHTS), to examine crash rates per
annual VMTby driver age, gender, and vehicle type. Each dataset includes population weights to derive
national estimates of crashes and households, which the authors use to estimate national crash rates for driver
age, gender and vehicle type. However, there are several limitations with the NHTS data. First, the survey
includes only 25,000 national households, supplemented by an additional 124,000 households from "add-on"
state and municipal jurisdictions. The national weights provided in the NHTS most likely do not represent the
national distribution of vehicle registrations by make and model. Second, 30% of the households in the 2009
                                                                                           189

-------
NHTS reported income "over $100,000"; using the income bins provided in the NHTS to estimate household
income will likely understate the actual income of the households participating in the survey.

Chen el al (2010) makes use of an Australian survey of 20,000 young adult drivers that asked questions
regarding driving behavior.  The injury severity of 127 of these participants who were admitted to hospitals
after a vehicle crash was analyzed as a function of socio-economic status. The socio-economic status of the
drivers was based on average or median values from the zip code where they (or their parents, since most were
at University) lived.

Using 2010 California vehicle registration data, LBNL calculated the median household income by registered
owner's zip code, using 2000 census data, then took the average household income by vehicle make and model.
As discussed in the Final Phase 1 report, predicted US fatality risk is correlated with this measure of average
household income by vehicle model, with risk decreasing as income increases; the correlation is highest for cars
and CUVs, with an R2 of 0.60 (or a correlation coefficient r of 0.77) or higher, as shown in new Figure 4.20 and
the first column of new Table 4.7. However, while income tends to increase as mass increases, the correlation is
not strong (R2 is highest for 4-door cars, at 0.16, and CUVs, at 0.19, as shown in new Figure 4.21 and the middle
columns of new Table 4.7). As with predicted fatality risk, there is a wide range in the average income of
households owning vehicle models with similar mass. This analysis is summarized at the end of Section 4 of the
Final Phase 1 report.

Using average household income by vehicle model, rather than driver alcohol or drug use, or reckless driving, as
an alternative indicator of driver behavior substantially reduces the estimated detrimental effect of mass
reduction on risk in cars and CUVs/minivans, and has little effect on the relationship in light trucks.  This
analysis is summarized in Section 5.5 of the Final Phase 2 report.
[Van Auken]  [1] The induced-exposure data set provided by NHTSA is based on the "non-culpable" vehicle in
two-vehicle crashes. It is assumed that the dataset is a reprehensive sample of the driver and environmental
exposure factors for vehicle use.  However, since these cases include moving vehicles, some vehicle-driver-
environmental conditions may be under or over represented in this data depending on how they affect the
ability of a non-culpable vehicle to avoid a crash. Results in Ref (17) indicated that the estimated effect of
weight and size reduction are sensitive to whether the induced-exposure data are based on the Kahane (2003)
non-culpable vehicle definition of the Kahane (1997) stopped vehicle definition. Unfortunately it is not
                                                                                           190

-------
                                            currently possible to test this sensitivity with the NHTSA-provided induced-exposure data.

                                            [LBNL] NHTSA has developed vehicle miles traveled weighting factors for induced exposure cases based on
                                            stopped, rather than non-culpable, vehicles.  LBNL investigated what effect this alternative method of exposure
                                            has on the estimated effect of mass or footprint reduction on risk, in Alternative 15 in Table ES.3 and Section
                                            5.6 in the Final Phase 1 report.

                                            [2] It is also assumed that the accident data from the 13 or 16 states are representative of all US states.  Figure
                                            2.1 in Wenzel (2011b) provides a useful comparison of the distribution of fatalities in the US and 13 states by
                                            the nine different crash types.
ADDITIONAL COMMENTS:

[Chen and Kockelman] National Safety Council (2010) Estimating the Costs of Unintentional Injuries. Available online at
http://www.nsc.org/news_resources/injury_and_death_statistics/Pages/
EstimatingtheCostsof Unintentional Injuries

Chen, H.Y., Ivers, R.Q., Mariniuk, A.L.C., Boufous, S., Senserrick, T., Woodward, M., Stevenson, M. and Norton R. Socioeconomic status and risk of car crash
injury, independent of place of residence and driving exposure: Results from the DRIVE study. Journal of Epidemiology and Community Health 64(10), 2010, pp.
998-1003.

[Farmer] NHTSA's fatality analysis covered calendar years 2001-08, but the casualty analysis excludes 2008. Such exclusion is understandable given that 2008
data were at the time unavailable for a majority of the states (I think they are available now). However, 2008 was an unusual year and may have affected the
size and weight effect estimates. The footnote on p. 6 of the Phase II report states that an analysis including the available 2008 data will be summarized in
Appendix A.  I don't see Appendix A. Is an analysis planned including 2008 data?

[LBNL] The footnote is incorrect, there was no Appendix A in the final draft report.  For the Final Phase 2 report LBNL included the 12 states that have provided
crash data for 2008 (all but Wyoming).
                                                                                                                                      191

-------
    2.  CONTROL AND DEPENDENT
       VARIABLES USED IN THE REGRESSION
       MODELS
                                             COMMENTS
Please comment on the adequacy of control
and dependent variables used in the
assessment analysis and independent casualty
analysis, and recommend any alternative
control or dependent variables that are
available for possible inclusion in the analysis.
For example, what are the relative merits of
the main dependent variables used, fatality
risk per estimated VMT, and casualty risk per
police-reported crash?
[Chen and Kockelman] [1] As alluded to above, a primary concern is that the NHTSA analysis (& thus the
LBNL analyses) largely neglect the idea that vehicle type (make & model) is very much a proxy for driver type,
and a vehicle's crash frequency may have very little to do with physical vehicle characteristics. It has a lot to do
with the person behind the wheel, and gender and age simply aren't enough to control for such distinctions.
Education, risk aversion, ability, wealth, etc., are important covariates. But existing data sets are quite limiting
(though the MVOSS & FAR with 3-year driver violation history do offer some valuable insights, not discussed in
these reports). In reality, small cars may be less  crash prone than Kahane's & Wenzel's results suggest, because
they are driven by lower-income, younger, less risk averse people driving in more crash prone settings (e.g.,
commercial strips rather than pricey residential  suburbs). Such key caveats need thoughtful discussion. Four
relevant papers on  the topics of crash frequency and vehicle size-and-weight implications (by Knipling, Kweon
& Kockelman, Wang and Kockelman, and  Chen & Kockelman) have been sent to Tom Wenzel. These all include
useful literature reviews for further connections to useful findings for citation in the reports, as time allows the
contractor.

[LBNL] As discussed above, LBNL believes  that driver behavior in the current crash and their recent driving
record, and not general socio-economic information, is the best available measure to account for the effect
driver skill and behavior have on risk in individual vehicles. As described in Section 5.4 of the Final Phase 1
report, LBNL used these measures of driver behavior in one of its sensitivity analyses. Unfortunately, the states
do not consistently report this information in their crash databases.  LBNL examined the effect of including a
measure of driver household  income on risk in Section 5.5 in the Final Phase 2 report.

[2] Independent variables such as vehicle mass and footprint  may be accounting for effects of driver
socioeconomic factors as discussed in the Assumptions section. Furthermore, vehicle option variables such as
AWD and side curtain airbags may be  reflecting  the effects of driver environment (e.g., those living in areas
with icy winters opting for AWD) and attitude (e.g., more risk-averse drivers opting for side curtain airbags)
rather than the vehicle technology themselves. While extremely heavy and extremely large vehicles may have
significantly different handling and braking characteristics which influence crash frequency and casualty risk, it
is unlikely that given the same driver in the same environment, a small change in vehicle mass or footprint
would influence the driver's crash proneness.
                                                                                                                                      192

-------
                                            [LBNL]  We agree that it would be preferable to include other variables to control for individual driver behavior
                                            and skill; unfortunately such data are not available.

                                            [Farmer]  One needs to restrict control variables to those that are available and reliable. A problem when
                                            combining state databases is that the states often are not consistent as to the variables coded and the
                                            definitions of those variables.  This severely limits the list of possible control variables.

                                            [LBNL] The control variables used from the state crash databases are either derived from the VIN, or based on
                                            the driver age and gender, or the time or location of the crash. LBNL believes that these variables are
                                            consistently coded among the thirteen states.

                                            [Van Auken] [1] Reducing the total number of fatalities and serious injuries is desirable from a societal
                                            viewpoint. This includes both subject vehicle occupant and collision partner (e.g., other vehicle occupant,
                                            pedestrian) fatalities and serious injuries.

                                            [2] VMT is a good measure of accident exposure provided that it can be accurately determined.

                                            [3] The number of fatalities and serious injuries per accident is a measure of vehicle crashworthiness (i.e.,
                                            effect of a crash on the subject vehicle occupants) and crash compatibility (i.e., effect of a crash on the other
                                            vehicle occupants or vulnerable road users).  Subject vehicle occupant fatalities and serious injuries per
                                            accident are a measure of the subject vehicle crashworthiness. Collision partner fatalities and serious injuries
                                            per accident are a measure of vehicle crash compatibility.

                                            [4] The number of accidents per VMT is a measure of the crash avoidance  capabilities of a given vehicle.
What additional control variables, such as
vehicle make or model, might be included in
the regression models?
[Chen and Kockelman] [1] Vehicle height, a variable which may be more valuable than vehicle type for
similarly structured vehicles such as sedans, wagons, CUVs, and minivans, would be a valuable control variable.
In addition to a wider track, a lower center of gravity also increases vehicle stability, thereby reducing the risk
of rollover. Relevant  literature & findings exist, and should be cited.

[LBNL] The propensity for a vehicle to roll over increases as its center of gravity increases (and static stability
factor decreases), while risk in frontal crashes may decline as bumper height increases (and the degree  of
overlap with the bumper of a  crash partner increases). However, overall vehicle height is only a crude proxy for
these two other vehicle dimensions that are thought to correlate with safety, and therefore was not included in
                                                                                                                                        193

-------
                                            the regression models.
                                            [2] Other variables which have been found in past studies to influence fatality risk such as seat belt use,
                                            roadway geometry and division type are not included in this study.

                                            [LBNL] Seat belt use was not included as a control variable because it is notoriously under-reported for non-
                                            fatal injuries. Roadway geometry and roadway division were not included because they are not consistently
                                            reported or coded in the state databases of police-reported crashes.

                                            [3] To account for driver characteristics that contribute to casualty risk, socioeconomic variables such as
                                            household income, education, household size, etc. would be valuable additions. Unfortunately, both state and
                                            national crash databases typically do not include such information (outside of MVOSS). Such issues should be
                                            flagged for readers.

                                            [LBNL] As  discussed above, an additional sensitivity analysis was run using a control variable for household
                                            income, based on California vehicle registration data in 2010; see Section 5.5 in the Final Phase 2 report.
                                            [Farmer] I think that already there are too many control variables in the regression models. Instead I would
                                            consider defining different classifications of crash types. Table 2.2 of the Phase II report shows that the
                                            distribution of crash types for casualty crashes is very different from that for fatal crashes.

                                            [LBNL] It is possible that inclusion of too many control variables leads to non-intuitive results in certain cases.
                                            This problem might be resolved by removing control variables (other than vehicle weight or footprint) that are
                                            not statistically significant. NHTSA and LBNL ran a sensitivity where non-significant control variables were
                                            removed; see Section 5.8 in the Final Phase 2 report.
Please comment on any caveats or limitations
that these dependent variable or control
variables entail with respect to use of the
results as the basis for estimating the safety
effect of mass reduction.
[Chen and Kockelman] Please see above comment (in Assumptions section) regarding driver behavior and
environment.

[LBNL] As discussed above, an additional sensitivity analysis was run using a control variable for household
income, based on California vehicle registration data in 2010; see Section 5.5 in the Final Phase 2 report.
                                                                                                                                        194

-------
                                            [Farmer]  Model overspecification could be the reason for results that are non-intuitive, especially in the Phase
                                            II analyses of police-reported crashes. Control variables may be correlated with each other or with the size
                                            and weight variables. For example, Figure 2.9 of the Phase I report implies that torso side airbags increase
                                            fatality risk in CUVs.

                                            [LBNL] It is possible that inclusion of too many control variables leads to non-intuitive results in certain cases.
                                            This problem might be resolved by removing control variables (other than vehicle weight or footprint) that are
                                            not statistically significant.  The example cited suggests that torso side airbags increase fatality risk in
                                            CUVs/minivans by an estimated 0.7%, is a small effect, and not statistically-significant. LBNL ran a sensitivity
                                            where non-significant control variables were removed; see Section 5.8 in the Final Phase 2 report.
ADDITIONAL COMMENTS:

[Chen and Kockelman] Table 2.1 has many indicator variables labeled as "C" for continuous variable (such as ABS, ESC, AWD, DRVMALE, etc). These C's should
be removed.

[LBNL] These vehicle variables are continuous because, for some models, the VIN does not indicate whether a particular vehicle is equipped with that option or
not.  In these cases the fraction of that model that is equipped with the particular feature is used.  On the other hand, the DRVMALE variable is always a discrete
variable, and is be coded as such in  the table in the Final Phase 2 report.

[Farmer] The sensitivity results of Chapter 5 (Phase II) point out the extreme differences in results when changing the control variables.  For example, including
vehicle make changes the effect of  a 100-lb reduction in heavier cars from -0.91% to +0.55% (see Fig 5.3).

[Van Auken]  The underlying reasons for some of the estimated effects are unknown at this time, but presumably involve driver, vehicle, environment or
accident factors than have not been controlled for in the Kahane (2011) and Wenzel (2011b) analyses. See, for example, Refs 17 and 23.

[LBNL] The report notes that other vehicle, driver, or crash characteristics may account for the residual risk not  explained by the control variables included in the
regression models. As discussed above, additional sensitivity analyses were run using an alternative control variable for vehicle characteristics, initial vehicle
purchase price (using values in Folk's VIN decoding software), and a control variable for driver behavior, household income (based on California vehicle
registration data in 2010). The results of these sensitivities are shown in Sections 5.3 and 5.5 in the Final Phase 2 report.
                                                                                                                                        195

-------
    3.  METHODOLOGY AND STATISTICS
                                             COMMENTS
Please comment on the validity and
applicability of the methodology LBNL used in
assessing the NHTSA 2011 study and its
analysis of the relationship between mass,
footprint, and risks per police-reported crash.
[Chen and Kockelman] The Phase 2 report enhances the findings in the Phase 1 report by isolating the effect
of vehicle mass and footprint on crashworthiness. Like the Phase 1 report, this analysis goes into a fair amount
of detail and seeks to introduce additional analyses to better examine the relationship between mass,
footprint, and casualty risks. However, due to a lack of control for very specific vehicle differences (which vary
by make & sub-model), the exclusion of driver characteristics and crash setting details (which  cannot always be
controlled for, but are often correlated with vehicle type), the effects of downweighting vehicles and/or
shifting vehicle styles and sizes may be overestimated. Simply changing the vehicle mass or footprint on a risky
driver in a high-risk setting is unlikely to influence crash outcomes significantly.

[LBNL] As discussed above, additional sensitivity analyses were run using an alternative control variable for
vehicle characteristics, initial vehicle purchase price (using values in Folk's VIN decoding software), and an
alternative control variable for driver behavior, household income (based on California vehicle registration data
in 2010).  The results of these sensitivities are shown in Sections 5.3 and 5.5 in the Final Phase  2 report.

[Farmer] Figure 2.11 of the Phase II report implies that NHTSA's fitting of a separate regression model for each
of the 9 crash types was unnecessary, at least for the analysis of casualty risk per crash.  I don't recall seeing a
similar analysis for fatality risk per VMT. Is it possible to get essentially the same results as the NHTSA study
using a single regression model?

[LBNL] Figure 2.1 of the Final Phase 1 report has the same analysis (although the second and third columns are
reversed). NHTSA ran a separate regression model for each of nine crash types in order to reweight the
estimated coefficients to reflect an estimated reduction in the number of fatal rollovers and crashes with a
stationary object from widespread adoption of ESC in the vehicle fleet. Running a single  regression model
across all crash types obtains essentially the same result as running nine regression models.
                                            [Van Auken]  [1] The logistic regression methods seem to be appropriate. The confidence intervals are based
                                            on the logistic regression Wald Chi-Square statistic, which as Kahane (2003, 2011) has demonstrated does not
                                            include all sources of variation.  However, these confidence intervals are useful because they do provide some
                                            indication of the uncertainty in the results.

                                            [2] The two-stage results for the 13-state fatalities per crash and 13-state crashes per VMT, and the one-stage
                                                                                                                                        196

-------
                                            result for 13-state fatalities per VMT reported in Tables ES.l and 6.1 and Figure 2.7 were computed using
                                            independent logistic regressions. The differences between the two-stage results and the one-stage results for
                                            fatalities per crash could have been eliminated by using the "simultaneous three-way" logistic regression
                                            method described in DRI (2003). This method imposes the constraint that the combined two-stage estimated
                                            and the one-stage estimated are equal.
Please review other statistical methods LBNL
has used in the analysis, in addition to the
logistic regression methodology.  Examples
include the alternative approaches used by
LBNL to assess NHTSA interval estimation
results, and LBNL's linear regression analysis
of actual, predicted, and residual risk by
vehicle model.
[Chen and Kockelman] In the alternative measures of exposure, the author examines the effect of vehicle
manufacturer on fatality risk and treats the luxury models produced by Toyota, Honda, and Nissan as separate
manufacturers. However, domestic luxury brands (such as Cadillac & Lincoln) are categorized with their
nameplate manufacturers (GM and Ford), which appears inconsistent.

[LBNL] The analysis has been revised to include the luxury models Lexus, Acura, and Infiniti with their
nameplate manufacturers Toyota, Honda, and Nissan. An additional sensitivity was conducted using separate
indicator variables for the five luxury brands Lexus,  Acura, Infiniti, Cadillac, and Lincoln. The results of these
analyses are summarized in Section 5.3 in the Final Phase 2 report.

LBNL also examined the effect of replacing the vehicle make indicator variables with a single continuous
variable for vehicle initial purchase price, taken from the Polk VIN decoder.  Initial purchase price can be seen as
a proxy for the general quality of vehicle design; we expect that more expensive vehicles will have lower risk
than less expensive vehicles. This analysis is summarized in Section 5.5 in the Final Phase 2 report.
                                            [Van Auken]  [1] The correlations in Section 3 appear to be assessed using the Coefficient of Multiple
                                            Determination (R2) based on a linear fit to the data (e.g., the correlation between footprint versus curb weight
                                            in Figure 3.1 on p. 32). The linear regression model attributes the differences between the dependent variable
                                            (vertical axis) and the linear fit to the independent variable (horizontal axis) to random effects.  If there is no
                                            preference as to the choice of independent and dependent variables (e.g., footprint versus curb weight, or
                                            curb weight versus footprint), then the linear trend and R2 result would be different if the two variables were
                                            interchanged, and having two different yet equally valid results would be undesirable.

                                            [LBNL] The R2 values are the same if the two variables are interchanged.
                                                                                                                                       197

-------
                                            [2] If the variation in the data can be attributed to both variables (e.g., footprint and curb weight), then it
                                            would be better to report the square of the sample correlation coefficient r2, where r is computed according to
                                            Eqn (1).  The trend lines in these correlation figures should not be computed using a linear regression. Instead,
                                            the trend line should pass through the sample means (i.e., (x, y)), and have a slope equal to the ratio of the
                                            sample standard deviations in the data (i.e., sy/sx). Therefore, the reported correlation results do not depend
                                            on the ordering of the data variables.

                                            [LBNL] Again, the R2 values are the same if the two variables are interchanged.  The R2 values in Figures 3.1 and
                                            3.2, and  Figures 4.1 through 4.6, have been replaced with the correlation coefficient r, and a new Table 3.1 has
                                            been added to Section 3 that lists r and VIFfor curb weight and footprint by vehicle type, in the Final Phase 2
                                            report.
                                            Note this comments does not apply to linear trends indicated in Section 4, for which the Coefficient of Multiple
                                            Determination (R2) seems appropriate.

                                            [3] The Coefficient of Multiple Determination (R2) is frequently used in the Wenzel (2011b) report as an
                                            indicator of the statistical importance of a linear trend (e.g., lvalues in Tables 4.1 and 4.2 were compared to
                                            0.3). It would be better to report the standard error, confidence interval, and/or probability value as measures
                                            of the  statistical significance of a linear trend.

                                            [LBNL] The significance of the estimated relationships have been added to the tables in Section 4 of the Final
                                            Phase  2 report. However, the point of the section is to indicate that while the relationship may be statistically
                                            significant, on average (as indicated by the confidence level around the estimated effect), there remains a large
                                            range  in  risk for individual vehicle models with similar mass (as indicated by the relatively low R2 values).
Please comment on caveats or limitations of
using non-significant regression estimates to
project the safety impact of mass reduction.
[Chen and Kockelman] First, the t-statistics are not provided in the report which makes it difficult for the
reader to assess statistical significance of specific regression estimates (except where noted by the author).
Second, inclusion of a statistically insignificant variable can influence the estimates of coefficients associated
with related variables. Nevertheless, in general, it is best to keep insignificant estimates if one has a strong
defense for their role, since removing such variables (& thus their parameters) will shift the burden of
response to a correlated covariate's parameter, thus biasing the latter. We generally keep key covariates in a
model  up to a pvalue of 0.20 or 0.25 or so, especially in relatively small data sets (e.g., n < 1,000). Covariates
for which we have no strong basis can be removed for pvalues > 0.10.
                                                                                                                                         198

-------
[LBNL] The 95% confidence intervals are included in every figure in the document, to indicate whether the
estimated variable is statistically different from zero. LBNL ran a sensitivity where non-significant control
variables are removed; see Section 5.8 of the Final Phase 2 report.

[Farmer] Making projections from non-significant regression estimates is proper so long as the resulting
confidence intervals are constructed conservatively (to account for the accumulated imprecision). In that
sense, I prefer NHTSA's jackknife approach to the standard errors produced by SAS (see p. 13 of Phase II).

[LBNL] LBNL agrees that NHTSA's jack-knife method is a preferable method of estimating confidence intervals;
however, the jack-knife method obtains even larger estimates of uncertainty than using the standard errors
output from the regression models.

[Van Auken]  [1] Regression estimates are random numbers which have an unknown expected value and
variance, and  known sample value and standard error. If the sample value can be explained by a zero
expected value and known standard error then the  result is considered  not statistically significantly different
than zero and therefore the result is not considered to be statistically significant. However, If we can combine
this estimate with other estimates then the unknown expected values and variances can also be combined
using the same transformation, and the statistical significance of the combined result can be tested.
Therefore, depending on the sample values and inter-correlation, the combined result may be statistically
significant even if the individual estimates are not statistically significant.

For example, the results from each of the nine different crash types can be combined into an overall estimate
and the standard error calculated assuming that the results for each crash type are independent of each other.
Then the statistical significance of the combined effect can be determined.

[2] However, and Kahane (2011) points out there are two sources of uncertainty in the regression results. The
first is the PARS based sampling error which is uncorrelated across crash types because they are based on
different fatal cases (Kahane 2011, p. 77). The second is the state based induced-exposure sampling error
which is correlated across crash types because they are based on the same induced-exposure cases. Therefore
a confidence interval estimated using the "jackknife" method described by Kahane (2011) and accounting for
correlation of these two error sources would be more accurate than a simple estimate based on the Wald Chi-
Square statistic and assumed independence.

[LBNL] LBNL agrees that NHTSA's jack-knife method is a preferable method of estimating confidence intervals;
                                                                                          199

-------
                                            however, the jack-knife method obtains even larger estimates of uncertainty than using the standard errors
                                            output from the regression models.
How might the LBNL methodology be
strengthened to better represent future
vehicle designs and reduce multi-collinearity
between mass and footprint in the regression
analysis?
[Chen and Kockelman] Including more vehicle-specific characteristics (such as vehicle height and engine size)
reduces the analysis' dependence on vehicle type, since vehicle shapes and structures will continue to evolve.
There is also correlation with context (e.g., pickups are driven in more rural locations, with greater hazards
[like less lighting, higher speed, & few medians]). Disaggregate data are almost always best, to avoid ecological
fallacies & such.

[LBNL] As discussed above, overall vehicle height is only a crude proxy for two vehicle dimensions, center of
gravity and bumper height, that are thought to correlate with safety in rollover and two-vehicle crashes,
respectively, and therefore was not included in the regression models. Other vehicle attributes that might
affect risk are engine power-to-weight ratio, braking distance, and handling capabilities; LBNL may examine
the effect of these accounting for these vehicle attributes in future analyses.

LBNL's analysis does account for whether a vehicle is driven in a rural area, defined as a county in which the
population density is less than 250 residents per square mile of land area.
                                            [Van Auken]  [1] The effects of multi-collinearity can be mitigated by 1) obtaining more data, 2) pooling data
                                            from different crash type or vehicle types, or 3) reducing the number of regression variables. The first option
                                            would require more states (for serious injuries and police-reported accidents), calendar years and/or model
                                            years, which would involve  added newer data as it becomes available (or using older data). The second option
                                            might be to recombine the CUVs and minivans with truck based vans and adding a control variable to
                                            compensate for the differences in the vehicles types. The third option might involve removing statistically
                                            insignificant control variables or removing control variables that would not be expected to have an effect on
                                            the probability of crash or crash outcome (e.g., the side airbag variable is not included in pedestrian crashes
                                            because it is not expected to affect pedestrian fatality risk). The number of driver age control variables might
                                            be reduced from eight to three (as in the Kahane (1997) and DRI (2002-2005) studies). Finally,  a linear curb
                                            weight model instead of a two-piece linear model may help to better elucidate the general trend.

                                            [2] The Variance Inflation Factor (VIF) has been suggested as a measure of multi-collinearity in the Kahane
                                                                                                                                       200

-------

(2010 and 2011) reports, however this diagnostic metric does not account for differences in database size (i.e.,
Options 1 and 2 above). The Wenzel (2011b) report does not discuss the Variance Inflation Factor or report
any VIF results.
[LBNL] A table of VIF results, Table 3.1, has been added to the Final Phase 2 report.
ADDITIONAL COMMENTS:
201

-------
    4.   DATA SETS
                                             COMMENTS
Please comment on the validity and
applicability of the datasets used to project
changes in risk resulting from reduction in
vehicle mass. LBNL's casualty analysis used
police-reported crash data from 16 states,
while the 2011 NHTSA study used national
fatality data, combined with a subset of non-
culpable vehicles involved in two-vehicle
crashes from police-reported crash data from
13 states.
[Chen and Kockelman] [1] The Phase 2 report uses an unusually extensive data set of police-reported crash
data from 13 states which the author compares in detail to national data sets to illustrate similarities and
differences. The author is very thorough in addressing the difference in definitions of "serious" and
"incapacitating" injuries across different states and the effects of such inconsistency on the regression results.

[2] Since casualty risk in the report accounts for serious injuries but not minor injuries, the author should note
that police-reported injury levels may also be poor indicators of the actual or Modified Abbreviated Injury
Scale (MAIS) level, following medical evaluation. Farmer (2003) found that 41% of injuries reported by U.S.
police as incapacitating received MAIS ratings of "minor injury" by health care professionals using NASS
Crashworthiness Data System (CDS). Thus, the results of the estimated casualty risk increases and decreases
rely heavily on the assumption that police errors in reporting actual MAIS ratings are consistent across states.

[LBNL] Inconsistencies in injury reporting across states are another reason to include a control variable for each
state. The following text has been added to Section 2.3 of the Final Phase 2 report:

"Another type of bias is inaccurate reporting of injury outcomes by police officers at the scene of a crash. Using
detailed NASS CDS records,  in which a crash investigator tracks hospital records of victims in a small sample of
police-reported crashes, Farmer (2003) found that 41% of injuries that police responders coded as serious or
incapacitating received Modified Abbreviated Injury Scale (MAIS) ratings  of "minor  injury" by health  care
professionals.   The possibility that these injury  reporting errors are not consistent across states is another
reason to include a control variable for the state in which the crash  occurred."

[Farmer] A major limitation of the Phase II analysis is a bias that  may be due to the patterns of missing data.
In particular, the vehicle identification  number (VIN) is missing or mistyped for many crash records.  High-
severity crashes (especially fatal) are more likely to have detailed police investigation, so VINs (and other
variables) in these crashes may be more complete.  State crash files are therefore much less reliable than
PARS.

[LBNL] LBNL assumes that missing or erroneous VINs are equally  distributed among vehicle year, make and
model.  In its primary analysis LBNL used only vehicles whose model year reported in the crash database
matched the model year from the decoded VIN (as well as all vehicles in certain years in Washington where
model year was not recorded in the state database). In addition, in Section 5.7 of the Final Phase 2 report LBNL
                                                                                                                                        202

-------
corrected obviously erroneous VINs, by translating VIN position 10, shifting the VIN one position to the right
starting at position 9 (both based on the reported model year), and translating VIN position 8, the vehicle
engine code (based on a large database of known valid VINS, as described in Wenzel 2011a). Combined these
changes increased the number of available records by only 1%.

[Van Auken]  [1] The induced-exposure data set provided by NHTSA is based on the non-culpable vehicles in
two-vehicle crashes. See the comments in Table 1 on the limitations of this data.

[2] The use of property damage accident data and cases with serious injury from the 13 states seems
appropriate (with  the noted qualification that the different states may have different accident reporting
thresholds and injury reporting criteria). The concerns about the use of data for the 3 additional states
(Georgia, Illinois, and New Mexico) have also been noted.

[LBNL] Because of the differences in crash or injury reporting thresholds and injury severity reporting across
states, LBNL used  a control variable for each state in which the crash occurred.

[3] In addition, there are also many differences in the coding variables and values used by the different states,
which tend to make the receding to a common data set (either induced-exposure, police-reported accident, or
severe injury) imprecise.

[LBNL] The coding of the variable for type of crash may vary across states.  However, the remaining control
variables used from the state crash databases are either derived from the VIN, or based on the driver age and
gender, or the time or location of the crash.  LBNL believes that these variables are consistently coded among
the thirteen states.
                                                                                           203

-------
Please comment on any apparent, unstated,
or implicit impact on estimated risks inherent
in the two different approaches, and any
related caveats or limitations. For example,
what are the strengths and weaknesses of the
two measures of vehicle exposure, miles of
vehicle traveled scaled up from crash data
from 13 states, and number of police-reported
crashes?
[Chen and Kockelman] The Phase 1 analysis used non-culpable vehicles in two-vehicle crashes as a proxy for
induced exposure crashes. In contrast, Phase 2 analysis uses data from vehicles involved in one-car crashes and
the responsible vehicle in two-car crashes. The exclusion of the not-at-fault vehicle in two-car crashes may be
distorting the distribution of crash frequency and casualty risk across different vehicle makes and models if
crash-prone drivers are more likely to drive certain types of vehicles.

[LBNL] The Phase 1 analysis used a subset of non-culpable vehicles in two-vehicle crashes to assign vehicle,
driver, and crash characteristics to the total number of US  vehicle registrations and miles driven. The measure
of exposure in the Phase 1 analysis is national miles driven. The Phase 2 analysis used all non-fatality (or non-
casualty) crashes in the 13 states as the measure of exposure for estimating fatalities (or casualties) per crash;
the same VMTfrom the Phase 1 study was used to estimate crashes per VMT, and fatalities (or casualties) per
VMT.

[Farmer] The VMT weights provided by NHTSA were scaled to represent the entire US. Comments on pp. 9
and 18 of the Phase II report seem to acknowledge this deficiency, promising to adjust these to the 13 states in
the future. Was any adjustment made, such as multiplying the weights by the proportion of annual US VMT
accounted for by each of these states? The accuracy of the VMT weights is critical is we are to believe the
somewhat surprising results concerning crashes per VMT.

[LBNL] LBNL requested that NHTSA recalculate the VMT weights to represent the total number of vehicles
registered,  and the estimated total number of miles driven, in the thirteen states, rather than for the entire US.
NHTSA declined to provide LBNL  VMT weights for the 13 states.

In their previous analyses DRI similarly found that crash frequency (per VMT) increases as mass decreases. And
in Section 1.6 of its 2011 report NHTSA notes that small and light vehicles historically have had higher crash
and insurance claim frequency per vehicle mile traveled, despite their theoretical advantage in terms of
handling, braking, and accelerating.

[Van Auken] [1] The number of fatal or serious injury cases tends to be much  less than the number of
induced-exposure cases  (and the number of police-reported accidents). Therefore the effective numbers of
degrees-of-freedom in the statistical estimates tend to be  limited by the available number of fatal or serious
injury cases. For example, it would not be possible to estimate the effects of two variables (e.g., just curb
weight and footprint) if we had data for only one fatal of serious injury case even if we had thousands of
induced-exposure cases. Therefore it is desirable to use data for the entire US in order to get a large sample of
                                                                                                                                      204

-------
                                            fatal cases for the logistic regressions. This then requires the available induced-exposure data (i.e., from 13
                                            states) to be "scaled up" the US level using the method described in Kahane (2003 and 2011). The result is the
                                            best currently available estimate of vehicle exposure.

                                            [2] There may be some concerns about the accuracy of the vehicle miles-travelled data because the difficulty
                                            estimating the number of vehicle miles travelled at the make-model-year-state level of detail.

                                            [LBNL] RL Polk provided NHTSA a database of average odometer reading by vehicle year, make and model
                                            based on hundreds of thousands of odometer readings. Because many of these records came from state
                                            emission inspection and maintenance programs, it is likely that they are skewed towards vehicles driven in
                                            urban areas, which tend to be driven fewer miles than comparable vehicles registered in rural areas.

                                            An additional limitation of the  VMT weights developed by NHTSA is that they do not account for the reduction
                                            in miles driven in response to higher gas prices and the economic recession in 2008, as noted in Section 5.3 of
                                            LBNL's Phase 1 report.  Despite these limitations, the average VMT weights NHTSA has developed are an
                                            improvement over the averages used in the 2003 analysis.
ADDITIONAL COMMENTS:

[Chen and Kockelman] Farmer, C.M. Reliability of police-reported information for determining crash and injury severity. Traffic Injury Prevention 4(1), 2003,
pp.38-44.

[Farmer] Statements above Figure 2.7 in the Phase II report imply that the effects of weight reduction on crashes per VMT and fatalities per crash should add
up to the effect on fatalities per VMT. This is not the case. For example, a 1.43% increase in crashes per VMT and a 0.76% decrease in fatalities per crash
would imply a 2.16% decrease in fatalities per VMT (i.e., 1 - 0.9924/1.0143). The fact that the model on fatalities per VMT yields an estimated 1.08% increase
should be a cause for concern.  Either the VMT weights are inaccurate or the control variables have different effects on crash frequency and crashworthiness.

[LBNL] Table ES.l reports for lighter-than-average cars a 2.00% increase in crash frequency, a 0.54% decrease in fatalities per crash, and a 1.42% increase in
fatalities per VMT; in this case the first two estimates sum almost exactly to the third (2.00% - 0.54% = 1.46%). In other cases the first two estimates do not sum
to the third.  In its previous studies DRI solved the three equations, crashes per VMT, fatalities per crash, and fatalities per VMT, simultaneously, which forces
the estimated effects on fatalities per VMT to equal the sum of the estimated effects on crashes per VMT and fatalities per crash.
                                                                                                                                      205

-------
    5.  RECOMMENDATIONS
                                             COMMENTS
Please comment on whether the LBNL
assessment adequately addresses the NHTSA
2011 study and identifies the safety impact
from mass reduction. Are the analytic
methods and data used to assess the NHTSA
study, and estimate the relationship between
risk, mass, and footprint, appropriate? Is
casualty risk per crash a legitimate measure of
vehicle safety?  What other methods or data
could be  used to better predict the effect of
future vehicle designs on safety?
[Chen and Kockelman] As noted above, a primary concern remains a neglect of the notion that the type of car
is very much a proxy for driver type, and a vehicle's crash avoidance may have very little to do with vehicle
type. It has a lot to do with the person behind the wheel. Simply including gender and age variables cannot
account for important covariates such as education, risk aversion, driving ability,  wealth, etc. In reality, small
cars may be less crash prone than Kahane's and Wenzel's results suggest, because they are driven by
lower-income, younger, less risk averse people driving in more crash prone settings (e.g., commercial strips
rather than pricey residential suburbs). Alas, it is very difficult to control for all these variables, since they are
not readily available in data sets. In reality, the best the report authors can  do with such data sets is to explain
how all the other, relevant attributes  may factor in (e.g., quality of driver and typical driving settings),  and how
they can generate biased estimation (sometimes in either direction). Discussion of relevant literature that
looks more deeply at crash outcomes  (e.g., Wang or Chen's papers, mentioned above, allowing for
heteroscedasticity and individual vehicle attributes, non-driver outcomes, etc.) will also be useful.

[LBNL] As discussed above, additional sensitivity analyses were run using an alternative  control variable for
vehicle characteristics, initial vehicle purchase price (using values in Folk's VIN decoding software), and an
alternative control variable for driver behavior, household income (based on California vehicle registration data
in 2010). The results of these sensitivities are shown in Sections 5.3 and 5.5 in the Final Phase 2 report.

[Farmer] Casualty risk per crash does not  fully measure the effects of vehicle size and weight reductions on
society. Casualty risk per VMT best coincides with the NHTSA analysis of fatalities per VMT. The breakdown of
casualty risk per VMT into the crash frequency and crashworthiness components is of interest. However, the
surprising results reported here make everything suspect. For example, the Phase II report concludes that "the
detrimental effect of male drivers has to do with their higher tendency of getting into a serious crash rather
than their sensitivity to injury once a serious crash has occurred" (p. 24). A few pages later it concludes that
"male drivers have essentially no effect on crash frequency, but cause a statistically significant increase in
fatality risk once a crash occurs" (p. 28).

[LBNL] A summary discussing unexpected results has been added to the end of Section 2 in the Final Phase 2
report.

[Van Auken]  [1] The  basic methodology described by Kahane (2011) seems appropriate; and the extension by
Wenzel  (2011b) are also appropriate.  However some results using these methods and data are not well
                                                                                                                                       206

-------
                                            understood and need further diagnosis.

                                            [2] The induced-exposure data set provided by NHTSA is based on the non-culpable vehicles in two-vehicle
                                            crashes. See the Table 1 comments on the limitations of this data.

                                            [3] The state accident data files tend to have different database variable and coding definitions and criteria,
                                            which could confound the results.
Please comment on the overall adequacy of
LBNL's assessment of the 2011 NHTSA report
and its independent study of casualty risk for
predicting the effect of vehicle mass or
footprint reduction on safety.  Provide any
recommended improvements that might
reasonably be adopted by the author to
improve the analysis.
[Chen and Kockelman] Overall, the study is an enriching complementary document to the Phase 1 assessment
of the 2011 NHTSA report. The parallel structure of the two reports allows the reader to easily compare and
contrast the various additional analyses which examine the relationship of vehicle mass and footprint
reduction on safety. However, as stated previously in the comments here, driver preference for specific car
types (including size and mass) is related to driver socioeconomic characteristics and driving behavior. As
vehicle, driver, and roadway environment characteristics all contribute to fatality risk, the effects of physical
vehicle changes such as mass or footprint reduction on safety should not be overstated when the other two
types of characteristics are not sufficiently accounted for.

[LBNL] As discussed above, additional sensitivity analyses were run using an alternative control variable for
vehicle characteristics, initial vehicle purchase price (using values in Folk's VIN decoding software), and an
alternative control variable for driver behavior, household income (based on California vehicle registration data
in 2010).  The results of these sensitivities are shown  in Sections 5.3 and 5.5 in the Final Phase 2 report.
                                            [Farmer]  Overall these are reasonably good studies. The Phase I report does a very good job of assessing the
                                            NHTSA report of fatality risk.  However, the Phase II report should be more cautious in its conclusions
                                            concerning casualty risk.  The casualty analysis is based solely on police-reported data from 13 states, which:

                                               1.  May not be representative of the US as a whole.
                                               2.  Are inconsistent in the information given and the way in which it is coded.
                                               3.  Suffer from information that is missing, inaccurate, or unclear.

                                            [Van Auken] The Wenzel (2011b) report provides a valuable supplement to the analysis and results in the
                                            Kahane (2011) report.
ADDITIONAL COMMENTS:
                                                                                                                                      207

-------
[Farmer] Column G of Table 6.1 in the Phase II report provides the most appropriate comparison to results from the NHTSA report (Column A).  For both
fatalities and casualties per VMT, a 100-lb weight reduction is most harmful in lighter cars, less harmful in heavier cars and lighter light trucks, and slightly
beneficial in heavier light trucks, minivans, and crossovers.

[Van Auken] See attached tables 6 and 7 below.
                                                                                                                                     208

-------
                                  Table 6.  Additional General Comments and Recommendations
                                                         Mike Van Auken
   Section
                                        COMMENTS AND RECOMMENDATIONS
               Use of R2 is confusing. Suggest using lower case "r" when referring to the sample correlation coefficient (Box, Hunter, Hunter,
               1978, P. 61); or upper case R when referring to the regression coefficient of multiple determination (Draper and Smith, 1981, p.
               90).

               [LBNL] Figures 3.1 and 3.2, and 4.1 through 4.5, report r and not R2 in the Final Phase 2 report.  In addition, a new Table 3.1 with
               r and VIF, and a new Table 4.1 with r and R2, have been added.
All
In most cases the reported results are just estimates, but are not described as such. The word "effect" appears several hundred
times in this report with the "estimated" or other qualifier. In some cases this may be appropriate and in other cases it is not
appropriate.  It is recommended that the author review each instance and  revise as appropriate.

[LBNL] The word "estimated" will be used extensively in the final report.  In addition the following text will be included in the
Executive Summary, Section 1, and Section 6:

Although the purpose of the NHTSA and LBNL reports is to estimate the effect of vehicle mass reduction on societal risk, this is
not how the regression models should be interpreted. Rather, they are estimating the recent historical relationship between
mass and risk, after accounting for most measurable differences between vehicles, drivers, and crash times and locations. In
essence, the regression models are comparing the risk of a 2600-lb Dodge Neon with that of a 2500-lb Honda Civic, after
attempting to account for all other differences between the two vehicles. The models are not estimating the effect of literally
removing 100 Ibsfrom the Neon, leaving everything else unchanged.

In addition, the analyses are based on the relationship of vehicle mass and footprint on  risk for recent vehicle designs (model
year 2000 to 2007).  These relationships may or may not continue into the future as manufacturers utilize new  vehicle designs
and incorporate new technologies, such as more extensive use of strong lightweight materials and specific safety technologies.
Therefore, throughout this report we use the phrase "the estimated effect of mass (or footprint)  reduction on risk" as shorthand
for "the estimated change in risk as a function of its relationship to mass (or footprint) for vehicle models of recent design."
All
"Crashworthiness" in most instances should be changed to "crashworthiness and crash compatibility" because the fatalities
and/or serious injuries may either be in the subject vehicle (crashworthiness effect) or collision partner (crash compatibility
                                                                                                                               209

-------
effect).

[LBNL] Text has been changed
The statistical significance of the linear trends in Figures 4.1 through 4.9 are not reported.  It would be helpful if the confidence
intervals or statistical significance of the linear trends were reported, either in addition to or instead of R2.
The confidence intervals for the estimated slopes should be added to the results in Tables 4.1 and 4.2.

[LBNL] New Tables 4.1 through 4.5 summarize the linear trends and statistical significance of Figures 4.1 through 4.17 in the
Final Phase 2 report.
                                                                                                                   210

-------
                                 Table 7. Additional Specific Comments and Recommendations
                                                        Mike Van Auken
   Section
   Page
                                 COMMENTS AND RECOMMENDATIONS
Executive
Summary, 4
iv, v, 22, 66
Suggest changing "variance in risk" to "variation in risk" throughout.

[LBNL] The suggested change has been made.
Executive
Summary, 6
  vii, 63
The statement "In conclusion, casualty risk per crash is not necessarily a better metric than fatality risk per VMT
for evaluating the effect of mass or footprint reduction on risk; rather, it provides a different perspective in
assessing the benefits or drawbacks of mass and footprint reduction on safety in vehicles. However, it does
allow the separation of risk per VMT to be separated into its two components, crash frequency and risk per
crash" suggests that the casualty risk per crash metric was needed in order to assess the crash frequency and risk
per crash, which is incorrect. The DRI (2003-2012) methods have also estimated the effects of weight and size
on crash frequency (A/E) and risk per crash (F/E) in terms of fatalities.

[LBNL] Text has been changed.
5.3
    49
It would be helpful to list the 18 manufacturer dummy variables in a table.

[LBNL] The indicator variables for the vehicle manufacturers has been added to the final report.
5.3
  48-49
It is unclear why Lexus, Acura, and Infinity are treated as separate manufacturers, but Cadillac and Lincoln are
not.

[LBNL] The results of two regressions has been included in the final report: one including the five luxury brands in
their parent manufacturers, and one accounting for each of the five luxury brands. Accounting for the five luxury
brands has little effect on the estimated detrimental effect of car mass reduction on risk, as shown in Figure 5.3
in the Final Phase 2 report.
5.3
    49
It is unclear why AM General is considered a Chrysler brand.  The AM General Hummer was sold by GM
beginning with the 2003 model year.

[LBNL] AM General has been removed from the Chrysler brand and included in the Other category.
                                                                                                                              211

-------
212

-------
   Review of "An Analysis of the Relationship between Casualty Risk per Crash and Vehicle Mass and
                      Footprint for Model Year 2000-2007 Light-Duty Vehicles"

                                         David L Greene

                                        January 10, 2012
Summary

The Phase I and Phase II analyses by Tom Wenzel of LBNL have been executed diligently and consistently
in accord with the methods and data used in the original NHTSA analysis. The studies contain many
valuable, new insights.  The phase I study highlights the weakening relationship between vehicle mass
and highway fatalities.  This is not only seen in decreasing coefficient estimates but in the very large
number of results that are not statistically significant. When regressions were done separately by
footprint deciles, vehicle mass was statistically significantly positively related to fatalities only for light-
duty trucks in rollovers, there were almost as many cases in which mass was negatively related to
fatalities (9 vs. 13 out of 27) and there were more instances of statistically significant negative
relationships than positive relationships. Given that so many tests are being jointly conducted, it is quite
possible that when joint probabilities are considered, there is no significant relationship between mass
and fatalities (more on joint probabilities later).  Showing the weakness and inconsistency of these
results is an important contribution.

Another meaningful contribution of the phase I study, and one that deserves more emphasis, is a logical
inference from the following findings: 1) much of the variance in risk remains unexplained even by the
most complete models, 2) control variables explain 1 to 2 orders of magnitude more of the variance
than the variables of interest (mass and size), 3) when key control variables are removed or changed it
strongly influences the coefficients of mass and size.  These results have very important implications for
the robustness of the results and the likelihood that some or all of the apparently statistically significant
relationships are due to spurious correlations with omitted or imperfectly controlled factors. Noting
that exposure measures are control variables with constrained coefficients, the following observation
from the phase I study is especially perceptive.

   "Calculating risk as total fatalities per induced exposure crash, rather than per vehicle mile
   traveled, reverses the sign of mass reductions on risk in  cars and the lighter light trucks, with  mass
   reduction leading to a reduction in risk in all vehicle types."

Finally, the phase I report notes that if only the control variables are included in the regression and
not size or mass, the resulting residuals from the  regression are uncorrelated with size or mass.
Given these findings (as well as those of phase II) the conclusions that,

   "The 2011 NHTSA study, and this report, conclude that the effect of mass reduction while
   maintaining footprint on societal US fatality risk is small, and may be non-existent."
                                                                                             213

-------
should be revised with the following emendation, "... and probably non-existent."

[LBNL] Text will be revised as indicated in bold above.

Both studies, like the NHTSA analysis, have shortcomings in terms of interpreting the results and the
language used to describe the results, and acknowledging the limitations of the data and methodology.
The limitations are extensive.  The interpretation of the results of the LBNL studies commits two
important, related errors.  The first is to attribute inferred coefficients of mass and size as representing
only the effects of vehicle mass and size when, as the phase I and II study results indicate, there is a
virtual certainty that aliasing effects are present due to a combination of omitted variables, errors in
variables and correlations among variables.  Given that estimated driver and environmental factors
tend to have 1-2 orders of magnitude larger impacts on safety outcomes than vehicle factors, the almost
certain presence of aliasing effects must be explicitly acknowledged as severely limiting the ability to
draw inferences about the effects of vehicle attributes.  Second, the language used in interpreting
results fails to  acknowledge that the analysis does not address the effects of down-weighting or down-
sizing specific vehicles or vehicle designs, but instead relies on correlations between vehicle weight and
size in existing vehicle designs. In existing vehicles, weight and size are correlated with each other and
many other vehicle attributes (and driver and environmental attributes, as well). Thus, the study is not
actually measuring the effects of down-weighting via the material substitution and design changes likely
to occur as a consequence of fuel economy and emissions standards. An early example of the kind of
misleading language referred to here can be found on page iv.

   "For example, a 100-lb  reduction in the mass of lighter cars leads to a 1.84% increase in crash
   frequency (columns B), while mass reduction leads to a 0.76% decrease in the number of fatalities
   per crash (column C);"

This statement is misleading in that it implies causality rather than correlation, and it is additionally
misleading in that it implies that the inference applies to removing weight from specific vehicles.
Neither is correct. A better statement would be the following.

   "For example, vehicles in the lighter class that  are 100 Ibs. lighter are correlated with a 1.84%
   increase in  crash frequency...."

There are so many examples of this misleading language that it is not feasible to list them all. All
should be corrected, however. Failure to correct them could lead to serious misinterpretation of the
studies' findings.

[LBNL] The word "estimated" will be used extensively in the final report.  For example, the above
sentence will be rewritten as:

"For example,  the models estimate that 100-lb lower mass in lighter-than-average cars is associated with
a 2.00% increase in crash frequency (column B),  while lower mass is associated with a 0.54% decrease in
the number of fatalities per crash (column C);"
                                                                                            214

-------
In addition, the following text will be included in the Executive Summary, Section 1 and Section 6 of the
reports:

"Although the purpose of the NHTSA and LBNL reports is to estimate the effect of vehicle mass reduction
on societal risk, this is not exactly what the regression models are estimating. Rather, they are
estimating the recent historical relationship between mass and risk, after accounting for most
measurable differences between vehicles, drivers, and crash times and locations. In essence, the
regression models are comparing the risk of a 2600-lb Dodge Neon with that of a 2500-lb Honda Civic,
after attempting to account for all other differences between the two vehicles. The models are not
estimating the effect of literally removing 100 Ibsfrom the Neon, leaving everything else unchanged.

In addition, the analyses are based on the relationship of vehicle mass and footprint on risk for recent
vehicle designs (model year 2000 to 2007).  These relationships may or may not continue into the future
as manufacturers utilize new vehicle designs and incorporate new technologies, such as more extensive
use of strong lightweight materials and specific safety technologies.  Therefore, throughout this report
we use the phrase "the estimated effect of mass (or footprint) reduction on risk" as shorthand for "the
estimated change in risk as a function of its  relationship to mass (or footprint) for vehicle models of
recent design.""

Following in the footsteps of the seminal study by DRI, the NHTSA and LBNL studies contribute to the
literature in three important ways: 1) the LBNL and NHTSA studies recognize that the societal safety
perspective is the correct perspective to when assessing the impacts of fuel economy and emissions
regulations, 2) they recognize that vehicle dimensions and vehicle mass may have separate and
potentially different impacts on both the likelihood of a crash and the outcomes of the crash and, 3) the
LBNL phase two analysis makes an additional contribution by attempting to disentangle factors affecting
the likelihood of a crash and factors affecting the outcomes of a crash.

Speaking of the DRI study, I am puzzled about why there are no references cited in the phase I study and
only a handful all by Kahane and Wenzel, in the phase II study. This is perhaps due to the scope of work
defined for the two studies but there are highly relevant studies in the literature that could have been
cited, those by DRI foremost among them.  Making use of the insights from these studies would have
been helpful in interpreting the results of both phase I and phase II.

[LBNL] LBNL has summarized the 2003 DRI studies, as well as updated results published by DRI in 2011;
see Sections 2 and 3 in the Final Phase 2 report.

Lack of a Theory or Model of the Phenomenon

Both the  NHTSA and LBNL studies lack a rigorous theory of the process by which down-weighting at
constant  size or down-sizing at constant  mass affect societal safety either through crash avoidance or
crashworthiness. This is not a  trivial shortcoming because it affects the  ability to formulate hypotheses
and interpret results. Prior to  the dissenting report on safety  of the NRC 2002 CAFE report, the physics
of elastic collisions between objects was typically cited as the  underlying physical model.  That report
showed how taking the societal perspective renders that model inappropriate.  What remains appears
                                                                                           215

-------
to be far more complex, involving the quantity of kinetic energy, the ability of vehicle designs to absorb
that energy so as to minimize maximum deceleration rates, stability, maneuverability, safety
technologies, and more.

The consequences of the lack of a rigorous theory are that it is not known, a priori, what the signs of
coefficients are expected to be, let alone what their quantitative relationships should be. Hypotheses
must be formulated based on intuition and the interpretation of results is likewise ad hoc.  One
implication of this is that results that suggest that lower fatalities are associated with lower vehicle mass
have equal standing, a priori, with results that  indicate that higher fatalities are associated  with lower
vehicle mass, and similarly for vehicle size. There  are no surprising or unsurprising results,  in theory.

This also makes it difficult to develop a plan for statistically testing the model or theory and its
implications.  It would have been helpful to the reader to have been presented early on in the report
with such a plan of analysis.

[LBNL] Section 1.5 of NHTSA's 2011 report summarizes the hypothetical physical relationships between
vehicle mass, footprint, and societal fatality risk.  LBNL has added a section summarizing this discussion
at the end of Section 1 of the Final Phase 1 and Phase 2 reports.

On the Virtual Certainty of Aliasing

The LBNL report typically attributes causal effects to correlations between mass or size and safety. In
fact, most or all of the observed correlations are almost certainly affected by aliasing effects. There is
ample evidence for this inference in the results presented in the LBNL phase II report.

The coefficients of mass and size change in important ways when different model formulations are
estimated.  Removing and adding control variables changes the magnitudes and sometimes the signs of
the mass and size variables. This means that, at a minimum, the mass and size variables alias the effects
of the omitted control variables. The question is whether the aliasing is eliminated entirely by the
inclusion of the control variables available or whether some aliasing remains either because not all
relevant and correlated control variables have  been included or because the included control variables
are imperfect measures of the factors they are intended to represent.

The latter seems highly likely for the following  reasons. First, the overall explanatory power of the full
models (including control variables), as measured by their R2 is low. Most of the variance in casualties
and fatalities remains unexplained.  Second,  at least some of the important included control variables
are only crude approximations of the factors they are intended to represent.  For example, dummy
variables represent differences in state reporting practices, age and gender represent risky driving
behavior differences among owners of different sizes and masses of vehicles, the presence or absence
of a kind of safety equipment represents both  its performance and use in a particular vehicle, and
calendar year dummy variables represent unknown factors associated with the respective calendar
years. Such practices are common and their use is appropriate.  Third, the control variables generally
account for 1-2 orders of magnitude more variance in the casualty and fatality variables than do vehicle
weight and size. To recap, the amount of unexplained variability in the dependent variables is larger
                                                                                            216

-------
relative to the variance statistically explained by the most complete models. Control variables are
correlated with size and mass, and they account for 1-2 orders of magnitude more variability in the
dependent variables than the variables of interest, mass and size. Therefore, even small correlations of
size and mass with omitted variables or with errors (imprecision) of the control variables could easily
result in biased estimates for the effects of size and mass on the dependent variables.

[LBNL] The only control variables used in the NHTSA regression models that are correlated with size or
mass are the HD_PKP variable (r of 0.65 on OVERWTOO and r of 0.54 on FOOTPRNT) and the SUV
variable (r of 0.62 on FOOTPRNT). The control variable LBNL has added to the sensitivity analyses for
initial vehicle purchase price, PRICEOOO, also is correlated with mass and footprint. However it is possible
that other controls not included in the regressions may be correlated with mass or footprint.

Tom Wenzel is to be commended for providing the results that definitively demonstrate the three key
points made above. The above  is not a criticism of the analysis nor of the results, per se. It is a criticism
of their interpretation. In light of the above, the results should be interpreted in light of the virtual
certainty that many of the estimated coefficients are likely to be biased in ways that make their
interpretation highly uncertain.  The implication is that phrases such as "down-weighting or down-sizing
caused" to "mass (or size) and unobserved correlated factors are associated with..."

[LBNL] The following text will be added to the executive summary:

"It is unclear why lower vehicle  mass is associated with higher crash frequency, but lower risk per crash,
in the regression  models.   It is possible that including variables  that  more accurately account for
important differences among vehicles and driver behavior would reverse this relationship. On  the other
hand, it is also possible that over thirty years of improvements in vehicle design to achieve high crash test
ratings have enabled manufacturers  to use clever vehicle design to  mitigate the hypothetical safety
penalty of low mass vehicles."

"The large remaining unexplained variance in risk by vehicle model could be attributable to other
differences in vehicle design, or how drivers who select certain vehicles drive them. It is possible that
including variables that account for these factors in the regression models would change the estimated
relationship between mass or footprint and risk."

Additional text, discussing unexpected results, has been added to the end of Section 2 of the Final Phase
2 report.

On Joint Probabilities

The NHTSA and LBNL studies do not correctly interpret their results as joint statistical tests.  When
testing a hypothesis on, for example, 5 vehicle classes simultaneously, a result for one equation that
might be statistically significant  on  its own may not be statistically significant as one of five related tests.

Statistical analyses comprised of multiple regressions too often  overlook the fact that tests of statistical
significance designed for individual regressions may not apply in the case of multiple regressions. That is
the case here. NHTSA conducted 5 analyses to infer relationships between mass differences among
                                                                                             217

-------
vehicles holding footprint constant for 5 classes of cars.  The results showed one relationship out of five
was statistically significant. As table 1 illustrates, using a simple example, if one conducts 5 trials, each
with a 0.05 probability of given result, there is a 22.6% probability of finding at least one such result in
the five trials. Thus, the joint significance level of the overall result (1 statistically significant regression
out of 5) is 0.226, rather than 0.05.

Table 1.  Simplified  Illustration of the Joint Probability of Inferences in Multiple Regressions
# Significant
Regressions
0
1
2
3
4
5

Combinations
1
5
10
10
5
1

0.95
0.773780938
0.81450625
0.857375
0.9025
0.95
1



0.05
1
0.05
0.0025
0.000125
0.00000625
3.125E-07


Joint
Probability
77.37809%
20.36266%
2.14344%
0.11281%
0.00297%
0.00003%
100.0%
22.6%
So there is between a 1:4 and a 1:5 chance of getting one statistically significant result by pure chance.
In fact, the actual significance level of the results is more complicated to calculate, and probably a bit
smaller than 0.226. Thus, it is very appropriate for Dr. Kahane to add the qualifier "if any" to his
conclusions about the relationship between the societal highway fatalities and mass reduction, holding
footprint constant.  Had appropriate tests of joint statistical significance been used to evaluate the
results in the NHTSA and LBNL studies, the significance levels very likely would not meet accepted
criteria for statistical significance. This could change the conclusions of the studies from the inference
that mass is correlated with fatalities or casualties in some case but not others to the lack of statistically
significant evidence that mass is correlated with fatalities or injuries on the highway.  This is an
important difference.

[LBNL] The comment implies that the significance of the estimate on mass for one of five types of
vehicles, or for a handful of nine crash types, does not translate into whether the estimate of mass on
risk across all vehicle types, or across all crash types, is statistically significant.  For this reason we
compared the results from NHTSA's nine regressions for nine types of crashes, in which the estimated
coefficients were weighted by the number of fatalities in each type of crash, with results from a single
regression model for all crash types combined (without controlling for crash type). The second and third
columns in Figure 2.1 of the Final Phase  1 report present this comparison. The figure indicates that the
estimated effects of mass reduction on risk, and the statistical uncertainty of those estimates,  are
virtually identical, whether one takes the weighted average estimates from the nine regression models
by crash type or whether one runs a single regression model including all types of crashes.
                                                                                              218

-------
Figure 6.13 in the Final Phase 1 report similarly shows the estimated effect of mass reduction from nine
regression models, each including all vehicle types (with control variables for vehicle type) in each
regression model.  However, this analysis does not run a single regression model for all types of crashes.
We ran a single regression model, across nine crash types and three vehicle types, including control
variables for vehicle types but not for crash types.  The estimated effect of a 100-lb reduction in mass is a
0.15% (+/- 0.23%) increase in US fatality risk per VMT holding footprint constant, while the estimated
effect of a 1-squarefoot reduction in footprint is 0.43% (+/- 0.23%) increase in risk, holding mass
constant.

These results suggest that, across the entire vehicle fleet, the estimated effect of mass reduction on
fatality risk is very small, and not statistically significant,  even though mass is associated with risk for
certain vehicle types in certain types of crashes. However, we believe NHTSA is justified in examining the
relationship between mass and risk for certain classes of vehicles.

NHTSA found that there is a statistically-significant, albeit small, estimated increase in fatality risk as
mass decreases in one of five vehicle classes studied, lighter-than-average cars. Although it is possible
that this result was obtained by pure chance, as expected by joint probabilities or the multiple-
comparison fallacy, the fact that this result was obtained for the lightest cars, the vehicle class we expect
to be most sensitive to mass reduction, does not appear to be a purely random result.

Page-by-Page Comments

I will make page by page comments on the phase II study only, since that contains the overwhelming
share of original contributions and the key findings of the phase I study are recapitulated there.

p. iii    Paragraph 3. This would be a very good place to acknowledge the importance of driver behavior
       and environment on crash avoidance  especially.

p. iv    Para. 3.  This would be a good place to discuss probability inference in joint tests.

        Para 4. The statement about a 100-lb  reduction in the mass, etc., is a good example of
       misleading language.

p. v     Para. 1. Again,  it is misleading to say that mass reduction increases crash frequency, for reasons
       stated above.

        Para. 2. It is more accurate to describe the association of lower vehicle mass with casualty risk
       than the "effect of mass reduction on..." casualty risk.
        Para. 3. Would benefit greatly from joint probability inferences.

        Para. 4. As noted above, this shows how much more important the control variables are than
       the variables of interest.

        Para 5. Again, these are correlations not necessarily effects.
                                                                                             219

-------
p. vi    Para. 1. Again, mass reduction is misleading terminology and you do not know if it increases
       casualty risk or not, you know only a correlation. Why is this so important? It is the virtual
       certainty of spurious correlations, or aliasing, as noted above.

       Para. 4. (1st bullet) This is clear evidence of aliasing. Take variables out of the regression and the
       coefficients of interest change in important ways. Are there no important factors still missing?
       Are the variables included perfect measures of the factors of interest? Of course not. Thus,
       there must be remaining aliasing. How bad is it? We don't know.

       The third bullet shows the same effect with a different set of variables.

p. vii   Para. 4. No, your analysis does not indicate "...that much of the detrimental effect of mass or
       footprint reduction on risk can be attributed to the tendency for mass or footprint reduction to
       increase crash frequency."  Again, you have correlation, not causation and you have good
       reason to believe that what you are seeing is affected by spurious correlations.

       Para. 5. The "effect" is small, 1-2 orders of magnitude smaller than correlations with other
       control variables, and IS strongly affected by which variables are in the equation, as  stated on
       the previous page, and there is a great deal of unexplained variance. Please reconsider the
       meaning of these results in light of the comments above.

       Finally, as the last paragraph of the ES implies, it would be far better not to speak in  terms of
       "reducing" mass or size.  That is not what is happening in your data set.

p. 1    Para. 4. Risk per VMT includes the effects of how well vehicles are driven as well as  how well
       they can be driven. I think there is no chance that you have fully accounted for how well
       vehicles are driven.

       [LBNL] Including driver age does account for how well vehicles are driven, to some extent. In
       addition, LBNL ran a sensitivity where vehicles with "bad" drivers (i.e. crashes involving drug or
       alcohol use, or the driver was cited for speeding, another traffic violation, or had been cited in
       the previous three years) were excluded from the analysis.  Figure 5.14 of the Final Phase 1
       report shows that this sensitivity found that the estimated detrimental effect of mass reduction
       on risk per  VMT increased substantially for cars and lighter light trucks, and the estimated
       beneficial effect of mass reduction on risk decreased substantially for heavier light trucks and
       CUVs/minivans. Unfortunately, this detailed information on the driver is not available in the
       state crash data, so a similar analysis cannot be done to estimate the effect of removing bad
       drivers on crash frequency or on risk per crash. However, LBNL did run a sensitivity including
       average household income; see Section 5.5 in the Final Phase 2 report.

p. 2    Para. 2. Exposure  measures are explanatory variables whose elasticity is constrained to 1. That
       is, it is assumed that an increase in vehicle use of 1 vehicle mile produces a 1 unit  increase in the
       chance of a fatality (or casualty as the case may be). This is actually a maintained  hypothesis.  If
       this hypothesis is incorrect, it can bias the other coefficients in the equation. Thus, the  change
       from fatalities/VMT to fatalities/registration-year, to fatalities  per crash not only changes the
       meaning of the analysis, it also may bias coefficients in the event that the true relationship
       between fatalities and VMT is not an  elasticity of 1.
                                                                                             220

-------
       [LBNL] We expect that the regressions using a different measure of exposure or of risk would
       estimate a different relationship between vehicle mass and risk.

p. 3    Line 1. Please acknowledge that your "accounting for differences in driver characteristics, crash
       locations, and other vehicle attributes" is incomplete and that this could affect your inferences
       about size and mass.

       [LBNL] The phrase "...included in the NHTSA regression models" will be added to the end of this
       sentence.

p. 3    Para. 2. NHTSA's use of "non-culpable" vehicles involved in two vehicle crashes as an exposure
       measure raises its own  issues. How non-culpable was the non-culpable vehicle. Often this is a
       matter of degree, rather than black or white.  Driver behavior may also be involved. It seems to
       me this is just another potential measure of exposure that may or may not be better than any
       other measure and may introduce new sources of bias in the analysis.

       [LBNL] DRI suggested using stopped vehicles, rather than non-culpable vehicles, in two vehicle
       crashes as the measure of exposure. NHTSA and LBNL ran sensitivity analyses using this
       alternative measure of exposure; see Section 5.6 in the Final Phase 1 report.

p. 3    Para. 4. Induced exposure needs to be defined. What is it intended to mean? This needs to be
       explicit.

       Also, I am startled that there are no equations in these reports. Equations can provide an
       unequivocal explanation of the assumed relationships that cannot be adequately accomplished
       by words, in many cases. Why no equations?

       [LBNL] The method that NHTSA used to develop the vehicle registration and vehicle miles
       traveled weights using the non-culpable vehicles from the crash data from 13 states, national
       and state vehicle registration data from R.L. Polk, and average vehicle odometer reading by
       vehicle year, make, and model is complicated, and thoroughly described in Sections 2.3 through
       2.6 of the 2012 NHTSA final report.

p. 7    Para 3. CUVs and minivans are involved in fewer crashes with stationary objects than cars.
       Why?  Is  it the drivers, the vehicles, or the passengers?  How well can you control for such
       differences?  Not well.  What does this mean for your analysis?

p. 9    Para 1. Here an equation showing how the weighting was done would be very helpful.

       [LBNL] Please refer to Sections 2.3 through 2.6 of the 2012 NHTSA final report for a detailed
       description of how NHTSA created the VMT weights.

p. 9    Bullet 1. Excluding these vehicle types implies that the control variables in the model are not
       adequate to account  for whatever makes these vehicle types different from the vehicles
       included  in the analysis. First, this is an admission that the model is not adequate to explain the
       fatalities associated with these vehicles. Second, it is an admission that if they were included
       the coefficients on the variables of interest would likely be biased by spurious correlations.
       Clearly, it would not even be sufficient to include the vehicles along with a control variable (e.g.,
                                                                                           221

-------
       X = 1 if vehicle is a police car, 0 otherwise). This is yet another indication that the model suffers
       generally from omitted variables, errors in variables and correlation among right-hand side
       variables.

       [LBNL] LBNL agrees that excluding these vehicle types is a tacit admission by NHTSA that the
       control variables used in their regression models do not fully account for differences in vehicle
       types or the behavior of their drivers. However, LBNL ran a sensitivity in both Phase 1 and Phase
       2 draft final reports where these four vehicle types were included in the regression models, with
       a control variable for each vehicle type.  Figure 5.15 in the Final Phase 1 report indicates that
       including these vehicle types increases the estimated detrimental effect of mass reduction on US
       fatality risk per VMTfor lighter-than-average cars, but increases the estimated beneficial effect
       for heavier-than-average light trucks.  Figure 5.12 in the Final Phase 2 report shows that
       including these vehicle types has little effect on the estimated effect of mass reduction on
       casualty risk per crash.

p. 10   Line 3. Sentence does not make sense. Please correct.  [LBNL] The sentence fragment has been
       deleted.

p. 12   Sect. 2.3. Please provide an equation. [LBNL] The term "equation" has been replaced with the
       term "con version factor."

p. 13   Were the confidence intervals calculated using ex-l? Please state explicitly or, better, show an
       equation. [LBNL] Yes, the end of this paragraph states that the confidence intervals were
       calculated the same way.

       Para. 2.  Here another instance where you say "mass reduction  increases societal fatality risk"
       but you really are not entitled to say that. It is misleading. Also, the NHTSA CI's are larger, as
       they should be  in a joint test.

       Para. 3. These results require an underlying theory for interpretation. The  lack of one makes it
       seem like there is just no consistency in the results.

       [LBNL] A section summarizing the physical theory of vehicles and fatality risk has been added
       after Section 1 in both the Final Phase 1 and Phase 2 reports.

       Para. 4. The fact that the results for fatalities per crash differ substantially from fatality per VMT
       may be very important.  Taken at face value, it would imply that any negative effect of reduced
       mass is due to its effect  on crash avoidance (crash probability) rather than crashworthiness.
       This is where the lack of a theory is most troubling. Why would that be? Are lighter vehicles
       less easily controlled, etc.?  Or, as seems much more likely, is there a spurious correlation
       between mass and other omitted or imperfectly measured factors (including driver behavior)
       that lead to an increased probability of a crash? Consider, for example, driver age. Driver age  is
       related to crash involvement. Driver age is a control variable. But are all young drivers the
       same? Is it possible that young drivers more prone to risky behavior tend to drive lighter
       vehicles? If so, this could partly explain the result observed. Of course, this is just speculative,
       but the point is that correlations with imperfectly measured and omitted factors are highly likely
       to be present in the data and, if there, could easily affect the statistical inferences.
                                                                                             222

-------
       [LBNL] We expect that the regressions using a different measure of exposure would estimate a
       different relationship between vehicle mass and risk, although the differences could also be
       explained by other factors not included in the regression models.

p. 14   Here we see that changing the exposure measure influence the effect of mass and size on
       fatalities and casualties, which is more evidence that spurious correlations are likely to be
       biasing estimated coefficients for mass and size.

       [LBNL] Again, we expect that the regressions using a different measure of exposure would
       estimate a different relationship between vehicle mass and risk.  Although the differences could
       also be explained by other factors not included in the regression models.

       The bar graphs with confidence intervals are well done and convey a great deal  of information
       effectively.  The patterns of magnitude and statistical significance are difficult to interpret, partly
       because there is no explicit theory of what should happen and partly, perhaps, because the
       relationships are actually not real.

p. 16   Para. 1. Reduction in the mass of lighter cars increases crash frequency but reduces fatalities
       per crash. This is contradictory to the previously maintained theory that mass protects due to
       the physics of velocity changes in elastic collisions.  Indeed, there is no theoretical explanation
       for these results, only speculation.

       [LBNL] It is unclear why lower vehicle mass is associated with higher crash frequency, but lower
       risk per crash, in the regression models.  It is possible that including variables that more
       accurately account for important differences among vehicles and driver behavior would reverse
       this relationship.

       Adding vehicle purchase price substantially reduces the estimated increase in  crash frequency as
       vehicle mass decreases for all vehicle types; mass reduction is now estimated to slightly decrease
       crash frequency in the case of heavier-than-average cars, but is still estimated to increase crash
       frequency for the other four types of vehicles.  See Figure 4.10 in  the Final Phase 2 report.

       On the other hand, it is also possible that over thirty years of improvements in vehicle design to
       achieve high crash test ratings have enabled manufacturers to use clever vehicle design to
       mitigate the hypothetical safety penalty of low mass vehicles.

       Text to this effect has been added to the Final Phase 2 report.

p. 18   Para. 1. Here is a good example of such  casual speculation.

p. 19   Para. 1. Developing VMT weights for the 13 states is a good idea, given the effect of exposure
       measures on inferences. Still the results would not be definitive.

p. 20   Para. 2. "Mass reduction leads to a large reduction in  risk only in crashes with objects for heavier
       cars?" There is only one type of crash in which the simple physics of collisions leads to an
       unambiguous benefit for increased mass, and that is collisions with moveable or breakable
       objects.  This finding contradicts even that.
                                                                                             223

-------
       [LBNL] As summarized in the new Section 1.1 in the Final reports, additional mass may be
       beneficial if it enables the vehicle to knock down the object and continue moving, thereby
       reducing its delta V than if the vehicle was stopped by the object. In a previous study NHTSA
       estimated that the object is knocked down in about 25% of frontal collisions with stationary
       objects (Partyka, 1995). However, if the object is immovable, reducing vehicle mass would lower
       the kinetic energy of the crash, thereby reducing the amount of energy for the vehicle's structure
       to absorb, and likely reducing risk. Additional crush space or structural strength would increase
       the amount of crash energy the vehicle can absorb, tending to reduce risk to occupants.

       Figures 2.12 through 2.14, and Table 2.3, in the Final Phase 2 report indicates that lower mass is
       associated with a reduction in 13-state casualty risk per crash in crashes with  a stationary object,
       for four of the five vehicle classes, although  the estimated reduction is large enough to become
       statistically-significant only for heavier-than-average cars (3.75%) and  CUVs/minivans (2.60%).
       Note that in the same table lower footprint is associated with a significant 3.55% increase in
       casualty risk in car crashes with an object; smaller size substantially increases risk in crashes with
       objects for light trucks (0.99%) and CUVs/minivans (5.56%) as well.

       The relationship between lower mass and fatality risk per crash in crashes  with stationary
       objects is  even stronger (not shown in the Final Phase 2 report, but shown  in the table below); for
       fatality risk per crash, mass reduction is estimated to reduce fatality risk per crash in crashes
       with stationary objects in all five vehicle classes, and the estimated reductions are statistically
       significant for cars and the lighter light trucks.  As in casualty risk per crash, footprint reduction is
       estimated to increase risk in  all three types of vehicles.

       Estimated effect of mass or footprint reduction on risk per
       crash, in crashes with a stationary object

Variable
Weight




Footprint



Vehicle type
Cars < 3106
Cars > 3106
LTs < 4594
LTs > 4594
CUVs/minivans
Cars
LTs
CUVs/minivans
Casualty risk
per crash
-0.80%
-3.75%
-0.78%
0.04%
-2.60%
3.55%
0.99%
5.56%
Fatality risk
per crash
-3.96%
-5.14%
-3.98%
-0.81%
-5.77%
4.41%
2.11%
5.93%
       Para. 3. More speculation, this time about rollovers.

       [LBNL] A section summarizing the physical theory of vehicles and fatality risk has been added
       after Section 1 in both final reports. As noted in that section, reducing a vehicle's mass without
       changing its roof structure would reduce the force applied on the roof once a vehicle rolls over,
       and thus reducing risk.

p. 21   Para. 3. "Curiously,..." Curiouser and curiouser.

       Figure 2.15 printed without labels. Could  be my computer but the other graphs were fine.
                                                                                             224

-------
p. 24   Para. 2. This is probably a very important finding that needs further investigation and
       explanation. As figure 2.16 illustrates well, the correlations with mass and size are orders of
       magnitude smaller than the correlations with driver and environmental factors. This is why
       even small correlations with omitted or imperfectly measured control factors could be, are even
       likely to be, predominantly responsible for the estimated coefficients of mass and size.

p. 25   Para. 2. The results for minivans discussed here could be due to what is going on inside the
       vehicle as much or more than the vehicle itself.  How could these results be explained in terms
       of the vehicle itself. Figure 2.17 shows this again. The effects of calendar year dummies, which
       can only be considered rough approximations to unknown and various time-related factors,
       have much large effects than size or mass. Again in figure 2.20.

       [LBNL] Text discussing unexpected results such as this one will be added to the end of Section 2
       in the Final Phase 2 report.

p. 28   Para. 1. "Surprisingly,..."  How can side airbags, which deploy only in a crash, reduce crash
       frequency but not fatality risk in a crash? Only if the real effect is a reflection of who buys a
       CUV/minivan with side airbags and how and where they drive. There are more surprising
       inferences in paragraph 2 about male and female drivers.  Surprising relative to what theory?

       [LBNL] The coefficients for side airbags in cars have the expected effect; airbags have a small
       effect on crash frequency, but a large beneficial effect on risk per crash. The unexpected results
       for CUVs/minivans could be attributable  to the relatively small number of vehicles included in the
       analysis.  Text discussing  unexpected results such as this one will be added to the end of Section
       2 in the Final Phase 2 report.
p. 32   The problem here is not numerical multicollinearity (numerical difficulties inverting the cross-
       product matrix) but the more complex problem of correlations among right-hand side variables,
       omitted variables, errors in variables, and correlations of included variables with omitted and
       imperfectly measured variables.  This leads to biased estimates.

p. 36   Table 3.1 cries out for inference based on joint probabilities.  What is the probability of
       observing "success" in at least 3 out of 27 trials when the true probability of success is only 0.05.
       See discussion above. The probability is certainly much  higher than 0.05 and probably closer to
       0.5. The implication is that, taken together, these results do not show any statistically
       significant relationship between mass or size and risk per crash.  If there were a rigorous
       underlying theory, the interpretation might be different (patterns of significance could matter)
       but there is none. Again, good graphs on succeeding pages.

       [LBNL] We believe that the results in Table 3.1 (Table 3.2 in the Final Phase 2 report) speak for
       themselves, without having to calculate joint probabilities.  Casualty risk increases with
       decreasing mass in more than half of the footprint deciles in only 9 of the 27 vehicle/crash
       combinations, and the increase is statistically significant in fewer than half of the footprint
       deciles in each of these 9 cases.

p. 41   The statistical significance of such a relationship should  be the same whether bins are used or
       not. Is it?
                                                                                            225

-------
       [LBNL] Because the risk of a casualty is a binary outcome for an individual vehicle, binning is
       necessary, either by increments of weight, make and model, or some other factor, to present the
       relationship between risk per VMT and weight in a plot. The relationship between risk per VMT
       and curb weight is statistically significant when the values for each vehicle weight bin are
       weighted by the number of observations in each bin.

       The statistical significance of the relationship between casualty risk and weight bin has been
       included in a new Table 4.1  in the Final Phase 2 report.

p. 42   Para. 3.  R-squared is not the correct measure of statistical significance. Is the coefficient of
       weight significantly different from zero?

       [LBNL] A new Table 4.1 has been added to both final reports which includes the statistical
       significance of the relationship.  However, the R2 value is shown to indicate that, when the data
       are binned by vehicle weight the relationship appears stronger than when the data are plotted
       by vehicle make and model, in the first column of new Table 4.2. For example, the relationship
       between US fatality risk per VMT and weight for individual car models in new Table 4.2 in the
       Final Phase 1 report (4.0%) is statistically significant; however, the R2 is only 0.17. Therefore, on
       average the relationship is statistically significant, but there is a wide range in risk for individual
       car models.

p. 43   Para. 2. This is perhaps the key finding of the phase I and II analyses.  Control variables explain
       1-2 orders of magnitude more variance than size or mass. Still,  most of the variance remains
       unexplained and is uncorrelated with mass or size. It is very likely there is nothing going on
       here.

p. 47   Para. 2.  More evidence of correlation of mass and size with control variables and how changing
       definitions or excluding control  variables results in important changes in the coefficient of mass
       and size.  Such results are considered unstable.

       [LBNL] Section 5.1 in the Final Phase 2 report discusses the importance of including control
       variables for each of the thirteen states, to account for differences among the states in reporting
       rates for non-injury crashes.

p. 49   Para. 1. The rationale for the grouping of manufacturers is not  obvious. Can you explain it?

       [LBNL] The analysis has been revised to include the luxury models Lexus, Acura, and Infiniti with
       their nameplate manufacturers Toyota, Honda, and Nissan. An additional sensitivity was
       conducted using separate indicator variables for the five luxury  brands Lexus, Acura,  Infiniti,
       Cadillac, and Lincoln; see Section 5.3 in the Final Phase 2 report.

       Para. 2. Yet more evidence for the instability of the model and likelihood that variables still
       missing from the model, plus errors in measuring the included control variables are likely biasing
       the inferences. The results  described in this paragraph do not make sense to me. How can they
       be explained other than random results?
                                                                                             226

-------
p. 50   Para. 1. More casual interpretation of results.  OK, maybe, maybe not.  Same for paragraph 2.
       The economy faltered in 2008 but the big negative effect was in 2007. The downward trend
       started in 2004. This correlates neither with vehicle mass changes over time nor economic
       growth as measured by real GDP. Idle speculation.

p. 51   Para. 1. Good discussion of the gratuitous speculation by NHTSA about the meaning of the
       observed correlations. This is more a Rorschach test than statistical analysis.

p. 51   Para. 2. "We have no explanation for why..." More of this kind of honest appraisal is needed in
       studies like these.

p. 52   More results for which there is no explanation.

p. 53   Why the interaction between calendar year dummy variables and safety equipment? The
       presence of the safety equipment on a particular vehicle is established. What has calendar year
       (not model year) to do with it? Again, one suspects spurious correlations.

p. 55   One needs to think carefully about the reasons why vehicles would be excluded. It does not
       appear that NHTSA did that.  First line of first paragraph "used" rather than "sused". [LBNL]
       Text has been changed.

p. 57   Well reasoned.  It is interesting that NHTSA resisted including footprint  or size in previous
       analyses on the grounds of correlation with mass. These results show that assertion was
       groundless.

p. 59   Para. 3. Rather than say risk  per VMT accounts for two effects, it is better to say it includes or
       comprises two effects. [LBNL] Text has been changed. But this statement also ignores the
       important influences of drivers and environment and their potential correlations with other
       factors. Yes, it includes how well a vehicle can be driven, but more importantly it includes how
       well a vehicle IS driven. That is in there too and is very likely to be correlated with make, model,
       and other vehicle attributes.

p. 60   Para. 3. Here again, the conclusions are misstated. It is not a genuine "reduction" in mass, but
       an association with the mass of vehicles. And how does it "lead" to and increase in crash
       frequency? What is the theory or model that predicts this? Driver and  environment are very
       likely mixed up in these results to an unknown but likely substantial degree. So what can we
       really conclude? Not this. [LBNL] Text has been changed.

       As I read the conclusions and inferences I find myself asking, why?, why?, repeatedly without
       any sound explanations. Page 61, paragraph 3 contains more "surprising" results.  Surprising
       because they are contrary to theory? Surprising because they are contrary to  intuition?
       Surprising because they are random? To what can we attribute so many "surprising" results,
       and how many must there be before one concludes that the analysis is not revealing what we
       had hoped it would.

       Para. 4. Again, this cries out for joint statistical inference. Three statistically significant results
       out of 27 is probably nothing statistically significant at all.
                                                                                           227

-------
p. 62   Para. 1. What this shows, again, is that the coefficients of mass and size are strongly influenced
       by which control variables are included in the model and how they are defined. These results
       and their implications need explanation. The bottom line is that the effects of mass and size are
       likely to be (after the necessary joint significance calculations are done) not statistically
       significant, not consistent, and not robust.

p. 63   How do mass and footprint reduction (again, it's not really reduction in the sense of designing
       lighter vehicles to increase fuel economy or reduce GHG emissions, the issue at hand) increase
       crash frequency. What is the theory? I don't find a theory in either the NHTSA report or the
       phase I and phase II studies. Absent a theory, these results seem sufficiently unstable and
       inconsistent to be highly questionable as evidence of any relationship between mass or size and
       crashworthiness or crash avoidance. I think joint estimation of significance levels would provide
       additional support for this view.

       [LBNL] Section 1.5 of NHTSA's 2011 report summarizes the hypothetical physical relationships
       between vehicle mass, footprint, and societal fatality risk.  LBNL has added a section
       summarizing this discussion at the end of Section 1 in each of the final reports.

4.     References

       Chen, H.Y., Ivers, R.Q., Mariniuk, A.L.C., Boufous, S., Senserrick, T., Woodward, M., Stevenson,
       M. and Norton R. 2010. "Socioeconomic status and risk of car crash injury, independent of
       place of residence and driving exposure: Results from the DRIVE study." Journal of Epidemiology
       and Community Health 64(10), pp. 998-1003.

       Kweon, Y-J. and Kockelman, K.M. 2003. "Overall Injury Risk to Different Drivers: Combining
       Exposure, Frequency, and Severity Models."  Accident Analysis and Prevention 35(4), pp. 441-
       450.

       Wenzel, Tom P. 2011a. Assessment of NHTSA's Report "Relationships  Between Fatality Risk,
       Mass, and Footprint in Model  Year 2000-2007 Passenger Cars and LTVs". Preliminary report
       prepared for the Office of Energy Efficiency and Renewable Energy, US Department of Energy,
       Berkeley, California.  August.  LBNL-.

       Wenzel, Tom P. 2011b. An Analysis of the Relationship between Casualty Risk Per Crash  and
       Vehicle Mass and Footprint for Model Year 2000-2007 Light-Duty Vehicles.  Preliminary report
       prepared for the Office of Energy Efficiency and Renewable Energy, US Department of Energy,
       Berkeley, California.  December. LBNL-.

       Wenzel, Tom P. 2012a. Assessment of NHTSA's Report "Relationships Between Fatality Risk,
       Mass, and Footprint in Model  Year 2000-2007 Passenger Cars and LTVs". Final report prepared
       for the Office of Energy Efficiency and Renewable Energy, US Department of Energy,  Berkeley,
       California. August. LBNL-.

       Wenzel, Tom P. 2012b. An Analysis of the Relationship between Casualty Risk Per Crash  and
       Vehicle Mass and Footprint for Model Year 2000-2007 Light-Duty Vehicles.  Final report prepared
                                                                                           228

-------
for the Office of Energy Efficiency and Renewable Energy, US Department of Energy, Berkeley,
California. August. LBNL-.
                                                                                   229

-------