Peer Review of ALPHA
Full Vehicle Simulation Model

£%	United States
Environmental Protect
Agency

-------
Peer Review of ALPHA
Full Vehicle Simulation Model
Assessment and Standards Division
Office of Transportation and Air Quality
U.S. Environmental Protection Agency
Prepared for EPA by
ICF International
EPA Contract No. EP-C-12-011
Work Assignment No. 4-03
This technical report does not necessarily represent final EPA decisions or
positions. It is intended to present technical analysis of issues using data
that are currently available. The purpose in the release of such reports is to
facilitate the exchange of technical information and to inform the public of
technical developments.
NOTICE
4>EPA
United States
Environmental Protection
Agency
EPA-420-R-16-013
October 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Contents
1.	Introduction	1-1
2.	Selection of Peer Reviewers	2-1
3.	Peer Review Process	3-1
4.	Review Comments Grouped by Charge Letter Topic	4-1
Appendix A. Resumes and Conflict of Interest Statements	A-l
Appendix B. Charge Letter	B-l
Appendix C. Sujit Das Comments	C-l
Appendix D. Shawn Midlam-Mohler Comments	D-l
Appendix E. ICCT - John German (lead reviewer), Anup Bandivadekar, Oscar Delgado,
Francisco Posada Comments	E-l
ICF International	i	October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Acronyms and Abbreviations
Acronym / Abbreviation
Stands For
ALPHA
Advanced Light-Duty Powertrain and Hybrid Analysis model
EPA
U.S. Environmental Protection Agency
FTP
Federal Test Procedure
ICCT
The International Council on Clean Transportation
ICF
ICF International
LDV
Light-Duty Vehicle
OTAQ
Office of Transportation and Air Quality
SAE
Society for Automotive Engineers
TAR
Technical Assessment Report
WAM
Work Assignment Manager
ICF International
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Introduction
1. Introduction
As EPA's Office of Transportation and Air Quality develops its programs to control greenhouse gas (GHG)
emissions from light-duty highway vehicles, there is a need to evaluate the effectiveness of technologies
likely to be used to meet these standards. The Advanced Light-Duty Powertrain and Hybrid Analysis
(ALPHA) was created by EPA as an analysis tool to estimate the Greenhouse Gas (GHG) emissions from
Light-Duty (LD) vehicle sources. It is a physics-based, forward-looking, full vehicle simulator, which is
capable of simulating various vehicle types and powertrain technologies. The ALPHA model uses the
industry standard MathWorks software products MatLab, Simulink, and Stateflow version 2014a. The
entire model and all subsystems are unlocked for complete transparency and is scheduled to be
released to the public in 2016 along with the release of the 2017-2025 light-duty Technical Assessment
Report (TAR). Conducting a comprehensive peer review of the model is an important step in gaining
wide acceptance of this model by the light-duty automotive vehicle community.
This report details the peer review of the ALPHA Full Vehicle Simulation Model dated May 5th- 2016. A
number of independent subject matter experts were identified and the process managed to provide
reviews and comments on the methodologies used in the model. This peer review process was carried
out under EPA's peer review guidelines1.
This report is organized as follows:
¦	Chapter 2 details the selection of the peer reviewers
¦	Chapter 3 details the peer review process
¦	Chapter 4 shows comments grouped by charge question
¦	Appendix A provides resumes and conflict of interest statements for the six selected reviewers
¦	Appendix B provides the charge letter sent to the selected reviewers
¦	Appendix C, D and E provide the actual reviews submitted by the three selected reviewers/review
team
1 U.S. Environmental Protection Agency, Peer Review Handbook, 4th Edition with appendices. Prepared for the U.S. EPA by
Members of the Peer Review Advisory Group, for EPA's Science Policy Council, EPA/100/B-15/001. Available at
https://www.epa.gov/osa/peer-review-handbook-4th-edition-2015
ICF International
1-1
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Introduction
ICF International
1-2
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Selection of Peer Reviewers
2. Selection of Peer Reviewers
ICF International (ICF) compiled a list of 10 reviewers who would be capable of reviewing the ALPHA
model. ICF contacted these potential reviewers to determine their availability to participate and
obtained a CV. We also requested information about potential conflict of interest.
Based on the contacts and a qualitative analysis of the qualifications of each reviewer, ICF selected two
reviewers and a review team with four members from ICCT led by John German. The six reviewers
selected are listed in Table 2-1. Each had the necessary expertise, were available to review the report in
a timely manner and had no conflict of interest. All were agreed upon by the EPA WAM.
Table 2-1. Final Reviewers
Reviewer
Contact Information
Necessary
Expertise
Conflict of
Interest
Sujit Das
Oakridge National Laboratory
P: 865-946-1222
dass@ornl.gov
Yes
No
Shawn Midlam-Mohler
The Ohio State University
P: 614-247-8650
midlam-mohler.l@osu.edu
Yes
No
John German (Lead
reviewer for ICCT)
The International Council on Clean
Transportation
P: 202.534.1600
john@theicct.org
Yes
No
Francisco Posada
The International Council on Clean
Transportation
P: 202.534.1600
francisco@theicct.org
Yes
No
Oscar Delgado
The International Council on Clean
Transportation
P: 202.534.1600
oscar@theicct.org
Yes
No
Anup Bandivadekar
The International Council on Clean
Transportation
P: 202.534.1600
anup@theicct.org
Yes
No
Resumes and conflict of interest statements for the six reviewers can be found in Appendix A.
ICF International
2-1
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Selection of Peer Reviewers
ICF International
2-2
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Peer Review Process
3. Peer Review Process
Once the six reviewers had been decided upon and approved by the EPA WAM, a charge letter (see
Appendix B), the model files and supporting materials for the peer review were distributed via email. A
teleconference was held with the reviewers to answer any questions. EPA provided additional
documentation and published SAE papers that helped the reviewers understand the methods and data
used in the model. An additional teleconference was held between one of the reviewers and EPA to
address a technical issue that needed to be resolved to get the model to run on a specific computer.
Each reviewer provided a written peer review in a timely manner. These were sent to ICF and the
reviews were then forwarded by ICF directly to the EPA WAM.
ICF managed the peer review process to ensure that each peer reviewer had sufficient time to complete
their review of the data analysis by the deliverable date. A two business day extension of the timeline
was granted to two reviewers - one for an illness and another for a schedule conflict. ICF adhered to
the provisions of EPA's Peer Review Handbook guidelines to ensure that all segments of the peer review
conformed to EPA peer review policy.
ICF International
3-1
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Peer Review Process
ICF International
3-2
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Review Comments Grouped by Charge Letter Topic
4. Review Comments Grouped by Charge Letter Topic
In this section, review comments from the peer reviewers are grouped by charge letter topic. Full
comments (including those in addition to the charge topics) can be found in Appendix C for Sujit Das,
Appendix D for Shawn Midlam-Mohler and Appendix E for the ICCT team review. John German was
the lead reviewer for ICCT, supported by Anup Bandivadekar, Oscar Delgado and Francisco Posada.
The response of each reviewer by charge letter topic is shown below.
Topic 1: EPA's overall approach to the stated purpose of the model (demonstrate technology
effectiveness for various fuel economy improvement technologies) and whether the particular
attributes found in resulting model embodies that purpose.
Suiit Das
1.	The ALPHA model approach is a fairly simple forward-looking based on underlying physics used in
other similar commercial packages available today. It consists of a simplified structure of total five
modules, three for vehicle, and one each for engine and transmission. It is significantly more
sophisticated in terms of its capability of estimating the C02 and the resulting fuel consumption
than the original lumped parameter model used by EPA in the original analysis of light-duty vehicle
GHG emission standards. Concepts and methodologies implemented in a simplified
MatLab/Simulink framework are consistent with other similar currently available models, although
less complicated with the limited capability such as with the only C02 emissions estimates.
EPA Response: The ALPHA model does not replace the Lumped Parameter Model but rather is used
to inform its calibration. The ability to model criteria emissions is not a goal of ALPHA at this time.
This feature could be added in the future if required.
2.	The modular nature of modeling framework provides the flexibility in using a technology
specific from a list of available individually parametrized powertrain components collected by
engine and chassis dynamometer testing to examine the vehicle performance of user-defined
specific technology packages.
EPA Response: ALPHA uses MatLab Classes to define the most common powertrain objects to
provide clean documentation (through MatLab doc) and to provide a consistent methodology
for setting default parameter values when required.
3.	It is difficult to examine in detail the model approach due to a lack of detailed documentation.
However, several peer review papers ~10 among which include seven recently published at the
SAE 2016 Annual Congress related to benchmarking several types of engines and transmissions to
generate inputs for use in ALPHA model have been published.
EPA Response: Since ALPHA is a modeling tool used by in-house EPA experts, its release to outside
parties presents a different need to provide documentation that is sufficient to "observe and
ICF International
4-1
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Review Comments Grouped by Charge Letter Topic
review" the model. ALPHA is not intended to be a commercial product or supported for wide
external usage as a development tool. EPA has documentation necessary to observe the inputs,
modeling assumptions & behavior, and outputs of the model. The model itself is largely
straightforward and the MatLab Classes provide formatted documentation for most critical
components.
4.	The model is completely input data driven, which need to be collected by either engine or
chassis dynamometer testing by specific vehicle system technology case. The model application
is thereby limited to the extent of validated data availability. The overall model performance is
dictated by calibration of numerous technology-specific parameters used to determine final
vehicle fuel economy and C02 emissions for various vehicle drive cycles.
EPA Response; Yes, the model is data-driven and care must be taken to provide reasonable input
assumptions. However, not every input needs to come from specific test data. Data from the
literature or other modeling (such as GT-POWER) can also be used to create or modify existing
input data sets.
5.	A simplistic approach without any consideration of any aftertreatment is sufficient for C02
emissions estimation based on the actual fuel consumption.
EPA Response; A user of ALPHA must consider the underlying input data sources, including
the criteria emissions performance of the engine(s) used to generate the engine fueling
map(s). The model accounts for some extra fuel that is consumed during normal vehicle
operation (referred within the model as engine transient fuel penalties) in part to account for
operational concerns like drivability, NVH, and emission reduction strategies such as catalyst
oxygen management after decel fuel cutoff.
Shawn Midland-Mohler
1. No technical issues were found in the model that would impact its intended use. The main concern I
have is in the overall fidelity of the model. The model reviewed has a decent level of fidelity -
perhaps even greater than required for the intended use. An excellent example would be the use of
4-D maps of certain parameters in the transmission. The concern is on how one calibrates these
maps to be representative of future technology. If one has the component on a test bench, then it
is possible to extract these parameters, however, that is not the context in which this will be
applied. The components being evaluated don't yet exist in physical form in many cases. One will
be left to alter relatively complex non-physical models that are black-box models of complex
physical behavior. I am not sure if all of the submodels will lend themselves for that type of
activity. It can be done, but the question is what will the effort be to do so and how does one verify
the results. A simpler model may be more appropriate in some cases.
Alternately, calibrating the existing models from models of much higher fidelity may be an
option as well. To completely understand this one would need to go through a couple of test
cases to understand the overall process from a workflow perspective.
ICF International
4-2
June 15, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Review Comments Grouped by Charge Letter Topic
EPA Response: The ALPHA model is designed to handle high fidelity input data if available. This
high level of fidelity is typically available from component benchmarking. However, this level of
fidelity is not required: lower fidelity data can also be used as needed. In fact, one of the more
interesting features of the ALPHA model is its ability to adapt to data from varying sources with
differing parameterizations. For example, if transmission losses are parameterized by line
pressure and temperature versus, say, input torque only, then the model can reconfigure itself
to handle either situation.
ICCT
1.	The model in its current form will be capable of performing its intended purpose of modeling
technology benefits for most of the technologies that the agencies are considering. Moreover, the
inclusion of CVT technologies as part of the modeling efforts shows the commitment by EPA to
include all potential technology pathways to meet the targets. We do recommend that a table be
added in the documentation that informs the reader of what technologies ALPHA is capable of
modeling and what technologies are yet to be implemented in the model or the model is
incapable to simulate.
EPA Response; Documentation of the technical aspects of ALPHA is included in the Draft Technical
Assessment Report (TAR) and on the EPA ALPHA website
(https://www.epa.gov/otaq/climate/alpha.htm).
2.	Some specific model elements that could be improved to better reflect technologies in the
future that may be impacted as well by Tier 3/LEV III emission standards:
Cold start operation modeling: ALPHA does not simulate or model cold-start operation, instead
applying the adjustment factors derived by Ricardo for Ricardo's 2011 modeling for EPA. This
approach likely works fine for current, known technology, but it is likely inadequate for future
engines with fast warm-up strategies, especially considering the upcoming changes to emission
standards. There is also no ability to model other drive cycles with a cold start, as the adjustment
factors are specific to the FTP.
EPA Response; ALPHA includes the ability to alter the "adjustment factors," and EPA has
implemented smaller fuel penalty factors to predict the effects of fast warm-up strategies.
Although including a temperature model would theoretically result in higher fidelity modeling, it's
questionable whether the characteristics of future warm-up strategies would be known with
enough accuracy that a temperature model would give better final results than adjusting the
post-processed penalty factors as necessary. Using adjustment factors is an accepted modeling
practice when specifically applied to the FTP. Two-cycle (FTP and HWFET) C02 emissions are the
primary focus of the current version of ALPHA.
ICF International
4-3
June 15, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Review Comments Grouped by Charge Letter Topic
3.	Also, specifically relating to the correction factors for Bag 1, where the fuel consumed during this
period is increased by around 16-17%, how does ALPHA correct the energy audit to account for
this correction? Would that 16-17% extra be reflected on energy losses, thermal or mechanical?
The effect of the correction on the energy audit should be described.
EPA Response: ALPHA simulates operation of warm vehicles and the energy audit correctly
accounts for their energy usage. The adjustment factor used to predict the fuel consumption
during cold start operation is applied in post-processing and does not affect the energy audit,
which reflects energy flows calculated during the warm simulation only. We will clarify this in the
ALPHA model documentation.
4.	According to the documentation review, ALPHA'S stop/start modeling appears to be very
simplistic. Their description says, "Alpha contains a sub-model for 12 volt electrical start-stop
technology, which simulates shutting the engine off after vehicle has stopped moving for 0.1
second. During a simulation, the start-stop mode is disabled when the vehicle is assumed to be
operating cold such as during the first 100 sec of bag 1 of the FTP cycle." Potential limitations of
this approach to SS technology modeling are:
No ability to do stop/start during coasting or deceleration (sailing).
EPA Response; Sailing is not a technology currently being modeled in ALPHA. The capability to
simulate a true sailing mode may be added to a future version of ALPHA.
5.	The length of stop/start disablement after a cold start appears to be completely arbitrary. Note
that actually modeling cold start operation, instead of using a simple adjustment factor, would
fix this problem as well, although we recognize that this would require development of more
sophisticated modeling.
EPA Response; As the locations of the FTP hills are fixed, the length of stop-start disablement is
really only a decision of whether or not stop-start is active in between hills 1 and 2. Testing of
MY2011-2015 vehicles equipped with start-stop technology has shown that, for some vehicles,
start-stop technology is enabled between FTP hills 1 and 2, and for some it remains disabled.
Because we expect OEMs to continue to reduce engine warm-up times, ALPHA assumes for the
2020 and later timeframe that start-stop will be enabled during FTP hills 1 and 2. We will revisit
this assumption if vehicle test data indicates otherwise.
6.	Engine scaling to maintain vehicle performance. ICCT's comments regarding this approach have
two components, dealing with the technical aspects and the way it is incorporated into the
ALPHA model.
First, ALPHA'S approach to maintain the vehicle performance when reducing weight is to scale the
BSFC maps to increase fuel consumption while downsizing an engine, reflecting the increase in
heat losses due to higher cylinder surface area to volume ratio. Ricardo also used this method on
their previous modeling work. Although the idea is technically sound, the limitation that we
ICF International
4-4
June 15, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Review Comments Grouped by Charge Letter Topic
perceive with this approach is that it ignores the option of reducing the number of cylinders.
which would decrease the cylinder surface area to volume ratio. The option of using an engine
with fewer cylinders implies that the model would have to incorporate algorithms that can select
the appropriate engine for downsizing; this implies developing additional engine maps and criteria
for selecting the right one.
EPA Response; We are aware of this type of analysis for selecting engine size, and recognize that
ALPHA'S engine scaling could be enhanced to cover more options. This is an area of active study at
this time.
7.	Second, the documentation describes the development of a parametric analysis tool to compare
the different technology approaches to find the best technology option (SAE 2016-01-0910). In
this case, the ideal outcome would be to incorporate the ability to change number of cylinders,
i.e., engine maps, as part of the effort.
EPA Response; Altering the number of cylinders and/or adjusting cylinder displacement to
accomplish optimal engine scaling to maintain vehicle performance would affect the overall
efficiency of the engine. The parametric analysis is an internal quality control tool (as opposed to
an optimization tool) used to evaluate changes in engine efficiency after scaling has occurred. The
parametric analysis is not dependent on any particular method of resizing, the end number of
cylinders, or final displacement.
8.	Synergy between engines and transmissions. In the SAE paper describing ALPHAs performance,
the authors correctly note that engines and transmissions have some overlap in their benefits
and derive parametric estimates for the "synergy factor" (SAE 2016-01-0910). As shown in Table
7 of that paper, the synergy factor is negative for 6AT, 8AT, and future 8AT. But it is positive for
the future 8DCT - which basically defies theory. EPA should evaluate why there are positive
synergies between engines and future 8DCTs and, if this is not an error in the modeling, describe
in the next iteration of the supporting documentation how the synergy factor was determined
for future 8DCTs and why it has positive synergies.
EPA Response; You correctly detected an issue with that specific "synergy factor". As stated in
the SAE paper, one of the key purposes ofEPA's internal parametric analysis of the data matrix of
1080 unique vehicle packages in the SAE paper was to facilitate vetting of the complete dataset
for quality control problems. While we did not detect this issue before the SAE paper was
published, soon afterwards we determined that this positive value of the "synergy" factor was
due to an improper idle fueling rate contained in an early version of one of the input engine maps
used to generate results for the SAE paper. There was no error in ALPHA'S calculation method or
the software which uses the engine fuel consumption maps to determine the C02 values in the
paper. However, there was just a simple problem with the input data for one specific fuel
consumption map. This engine map has since been corrected.
ICF International
4-5
June 15, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Review Comments Grouped by Charge Letter Topic
Topic 2: The appropriateness and completeness of the contents of the overall model structure and its
individual systems, such as:
a.	The performance of the example component models, including the reviewer's assessment of the
underlying equations and/or physical principles coded into that component.
b.	The input and output structures and how they interface with the model to obtain the expected
result, i.e., fuel consumption over the given driving cycles.
Suiit Das
1.	With the availability of new engine, transmission, and operational control benchmarking data,
ALPHA model would be able to support the 2017-2025 light-duty GHG rule requiring a
comprehensive advanced technology review, known as the mid-term evaluation for the 2022-
2025 light-duty GHG emission standard. The model is flexible enough with the capability to
determine the effectiveness contributions from advanced technologies not considered during
the original Federal rulemaking.
EPA Response; Thank you for your comment.
2.	The model performance validation is a continuous process, which has been accomplished by
using the newly acquired in-depth vehicle, engine, and transmission benchmarking data from
more than 25 different types of conventional and hybrid vehicles 2013-2015. At any point,
model appropriateness and completeness will be dictated by the extent of benchmarking data
available for the model performance validation.
EPA Response: EPA agrees that the model is data-driven which is why we have utilized data
from so many different types of test vehicles.
3.	A complete listing of model limitations (e.g., sensitivity of electric power steering losses with
vehicle speed and a lack of dynamic temperature algorithm) is critical for any model validation.
However, a few of these limitations have been discussed in the recent SAE publications.
EPA Response; The current version of the ALPHA model was not designed to be a general
purpose vehicle simulation model. It was designed to accurately estimate C02 emissions over the
EPA City and Highway drive cycles from a variety of advanced technology vehicle packages.
Within that context, ALPHA has no "limitationsonly a lack of additional features that it does
not need to estimate C02 emissions over the city and highway cycles. ALPHA does not require
the additional fidelity or flexibility that would come from adding those features.
To specifically address the items mentioned above, ALPHA has adequate assumptions for
alternator loading to cover the electric power steering requirements. ALPHA also has an
algorithm to adjust ALPHA'S warm FTP results for cold start operation and has no other need to
dynamically simulate an engine's temperature. Using ALPHA'S existing capabilities we have
successfully validated ALPHA using test data from over 20 engines and vehicles.
4.	The primary gear selection routine, the ALPHAshift algorithm critical to the fuel economy and
C02 emissions, has recently been validated and updated (based on the recently published SAE
ICF International
4-6
June 15, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Review Comments Grouped by Charge Letter Topic
paper) to dynamically generate transmission shift logic from a set of user-defined parameters
and generate more realistic vehicle performance during simulation. Since the modelling
approach used is completely data driven, algorithms for new control strategies need to be
developed including its validation of tunable control parameters in order for reliable vehicle
performance estimation using ALPHA.
EPA Response; The ALPHAshift parameters are determined during a simulation pre-processing
calibration step. Basic shift speed limitations are provided within reasonable limits and
adjustments are made based on available engine capability in order to provide reasonable
performance and drivability. For example, downsized turbo engines shift at higher speeds than
naturally aspirated engines. Consequently, appropriate ALPHAshift parameters are selected
when simulating a vehicle with this type of engine. This pre-processing step is an input to the
model and was not included as part of the peer review.
5.	A recent SAE publication (SAE 2016-01-1142) reports an excellent agreement of fuel
consumption results from a comparative examination of advanced transmissions among studies
conducted by National Research Council, Argonne National Laboratory, and earlier EPA lumped
parameter model.
EPA Response; EPA thanks the reviewer for the comments on EPA publication SAE 2016-01-1142.
EPA has spent considerable time and resources investigating transmission losses and behavior
since transmissions are second only to engines in terms of influence on fuel economy for
conventional vehicles.
6.	It is a fairly simple transparent model which allows to examine both fuel economy and C02
emissions of alternative light-duty technology pathways. The model execution requires an expert
MatLab/Simulink user since no user-friendly interface currently exists. Although the model use as
indicated will be mainly by in-house EPA experts, but the model validation of its transparency
particularly when pertaining to the Federal rulemaking needs to be addressed. A specific
simulation runtime is significantly high, more than 10 mins. without providing any indication to
the user progress made so far. A fairly more complicated model such as Autonomie available even
with enhanced capabilities is significantly faster today.
EPA Response; We appreciate your comment that "r\o user-friendly interface currently exists" and
your acknowledgement ofEPA's intent that ALPHA will be mainly used by in-house EPA experts.
Indeed ALPHA was developed primarily to help EPA estimate C02 emissions from future vehicle
technology packages. As with any internal tool, EPA does not have the need for a "user-friendly
interface" like one that would normally accompany a commercial product which is available for
purchase and fully supported for wide external usage.
We recognize the need for sufficient documentation to be transparent about the model for review
by the public. EPA has provided documentation necessary to observe and review the model inputs,
modeling assumptions & behavior, and outputs of the model, which are sufficient for external
review and use by technical experts. Through the release of the Draft Technology Assessment
Report (TAR) in 2016, EPA has significantly increased transparency by providing more in-depth
ICF International
4-7
June 15, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Review Comments Grouped by Charge Letter Topic
modeling information than was released with the 2012 FRM. As shown in the table below, we have
expanded the transparency of the ALPHA modeling functions; ALPHA inputs, results and their use
to update EPA's Lumped Parameter Model (LPM); model source code; and finally benchmarking,
mapping and validation through the use of peer reviewed SAE papers. We plan to further increase
transparency over the next year through the release of enhanced ALPHA documentation and
additional SAE papers.
Description of ALPHA materials published with the Draft TAR
Explanation of the modeling functions
Modeling results used to calibrate EPA's Lumped Parameter Model (LPM)
Access to inputs that produced the modeling results used to calibrate LPM
Access to full model source code
SAE papers describing various benchmarking, mapping and validation (10 papers)
Your comment about the slowness of the model available to the Peer Reviewers is simply the result
of the fact that a Simulink model was provided for the review process so the internal structure of
the model could be examined easily. The compiled version of the model is executed at EPA and it
only takes a few seconds to run a simulation.
7. Although a model run requires comparatively a fewer number of consolidated input parameter
files, but a lack of proper model documentation (both as a standalone document and within the
MatLab files) makes harder for a better model structure understanding including any sensitivity
runs of any user-specified variables.
EPA Response: Since ALPHA is a modeling tool used by in-house EPA experts, its release to outside
parties presents a different need to provide documentation that is sufficient to "observe and
review" the model. ALPHA is not intended to be a commercial product or supported for wide
external usage as a development tool. EPA has documentation necessary to observe the inputs,
modeling assumptions & behavior, and outputs of the model. The model itself is largely
straightforward and the MatLab Classes provide formatted documentation for most critical
components.
ICF International
4-8
June 15, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Review Comments Grouped by Charge Letter Topic
Shawn Midland-Mohler
1.	Driver Submodel
The driver does a good job at manipulating the brake and accelerator pedals in a smooth and
natural way while matching the drive cycle for the vehicle and cycles included in the review. For
different vehicles and cycles, this may or may not be true. Just checking for the error in the
target and actual speed is not enough to check for this as it is possible to meet the desired
speed trace with rapidly oscillating (i.e. unnatural) manipulation of the accelerator and brake
signals. This can lead to poor quality results while driving the trace accurately. This is not a flaw
with the approach used - just that a diagnostic needs to be added to check for this type of
behavior to ensure that the data produced is accurate.
EPA Response: At EPA, as a pre-processing setup, the driver model setup is adapted to the test
weight and performance level of the target vehicle. As a result the driver model can adapt from
compact cars up to Class 8 tractor-trailers. In addition, SAE J2951 drive quality statistics are
calculated in post-processing for each phase of any drive cycle to determine if the driver model is
performing satisfactorily.
2.	Engine Submodel
There are many controls and calibrations imbedded in the engine plant model. This can be
problematic in some circumstances for model reuse and calibration. An example would be the
decel fuel cutoff strategy and the idle speed control strategy. At a minimum, clarity could be
improved by making control function and plant models visible and different.
EPA Response; Thank you for your comment. It might be possible in the future to update the engine
submodel in this way, which might be more like the transmission submodels where the overall
controls are separated from the plant.
For a model aimed at fuel economy predictions, there may be more fidelity then strictly necessary.
For instance, use of inertia in the engine model requires additional complexity for the
transmission/engine model for minimal gains in fuel economy prediction. The approach used is fine
but it may be more complex than necessary for the model goals. The use of so many inertias when
things like shift durations simply enforced via the initialization files seems odd - but not wrong from
a technical perspective.
EPA Response: It may be the case that a model of lower fidelity could achieve similar results.
However, reducing model fidelity can lead to modeling shortcuts that can cause as many problems
as they solve. For example, it would probably be possible to eliminate the closed loop idle speed
control without any effect on fuel consumption (and the model might run faster), but then unnatural
behaviors may result during the transition away from idle. Generally speaking, different
transmissions in different drivelines have similar shift times relative to the number of gears in the
transmission, regardless of the specifics of the powertrain involved, therefore shift times are
parametrized primarily by gear count and not component inertias.
ICF International
4-9
June 15, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Review Comments Grouped by Charge Letter Topic
I was not able to find any type of thermal model for the engine - which is well known to have
temperature effects for efficiency. Components like the 12 V battery have one - so it is inconsistent
to not include one for the engine.
EPA Response: Engine thermal behavior is handled as a fuel consumption adjustment in post-
processing as discussed previously. The battery model is parameterized by temperature simply
because detailed data was available. For a 12V battery, thermal response is not a critical
factorhowever the same battery model (with appropriate parameter data) is also used to
model electric vehicle batteries and can be used to investigate their response to temperature
variations.
There are several ad-hoc factors like the 'tip-in penalty' that appears in the engine fuel-flow
submodel. It is not clear on how one would calibrate this or what it is really meant to capture from a
physical standpoint. Fuel enrichment on tip is not something that is done in modern engine controls
- air prediction is good enough that in general there is only a minor amount of enrichment
happening. Another example would be the 'acceleration_penalty_squelch_gps' factor.
EPA Response: Perhaps the naming convention of this variable was somewhat misleading. The 'tip-
in penalty' refers to extra fuel consumed after operating in decel fuel cutoff as observed during
vehicle testing, it does not refer to a performance-based enrichment. These parameters are meant
to represent the fuel consumption of the engine that is not captured by a steady-state fuel
consumption map.
3. Automatic Transmission Submodel
The fidelity of the transmission model and the resulting number of calibration parameters is fairly
high given the goal of the model. There are many look up tables some of which have four dimensions.
It is not clear how one would calibrate these parameters given the context of use.
EPA Response; As mentioned previously, the model can adapt to different and unique input data
sets. This gives the model the flexibility to use very simplified or highly specified input data.
The shift logic appears to produce reasonable shift commands - there are no frequent shifts,
inappropriate skip shifts, etc. There is a great deal of logic in the 'automatic trans control' block.
Similar to previous comments, the question is how does one calibrate all of this for technology that
may not exist?
EPA Response; Conventional transmissions are relatively straightforward. The shift control has been
documented in detail in the paper SAE 2015-01-1142 " Development and Testing of an Automatic
Transmission Shift Schedule Algorithm for Vehicle Simulationand is based on observed behavior in
test vehicles. The shift parameters are also adjusted in pre-processing for a particular powertrain, as
discussed previously.
There are many thermal models present - only constant temperature is enabled. Transmission
temperature is important to fuel economy predictions and likely future vehicles will involve more
tightly integrated thermal systems.
ICF International
4-10
June 15, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Review Comments Grouped by Charge Letter Topic
EPA Response: Faster transmission warmup is accounted for as part of the fuel consumption
penalties applied to the FTP results to account for cold start operation. EPA has acquired
transmission thermal data during benchmarking but not all of this data is required for modeling two-
cycle fuel economy. The transmission is assumed to operate under fully warmed-up conditions (true
for the HWFET and the last two phases of a four-phase FTP) and an adjustment factor is applied to
increase the fuel consumption for phases 1 and 2 to account for the additional fuel used to warm up
both engine and transmission. This same approach is used for all transmission types.
4.	Continuously Variable Transmission Submodel
Control logic is much simpler in contrast to the automatic - this is a positive thing in terms of the
intended use.
EPA Response; Yes, the CVT has a simpler shift strategy, as documented in SAE 2016-01-1141
"Modeling of a Conventional Mid-Size Car with CVT Using ALPHA and Comparable Powertrain
Technologies".
There are many thermal models present - only constant temperature is enabled. Transmission
temperature is important to fuel economy predictions and likely future vehicles will involve more
tightly integrated thermal systems.
EPA Response; The same cold temperature adjustment factor described in the previous response on
this topic under automatic transmissions is used for all transmission types including CVTs.
5.	Dual Clutch Transmission Submodel
There are many thermal models present - only constant temperature is enabled. Transmission
temperature is important to fuel economy predictions and likely future vehicles will involve more
tightly integrated thermal systems.
EPA Response; The same cold temperature adjustment factor described in the previous response on
this topic under automatic transmissions is used for all transmission types including DCTs.
6.	12V System Model
The 12V battery model is one of the more complex models in the model. The alternator model is
some control logic and a single constant efficiency for current to torque based on speed - so quite
simple. This is a large mismatch in fidelity. Having a fairly elaborate 2nd order RC model for the
battery and then modeling the alternator so simply is not technically incorrect, but is something to
consider.
EPA Response: High fidelity data is used for the 12V battery model because it was available. EPA
has a battery test facility and is actively benchmarking several types of conventional 12V batteries
in order to provide accurate parameters to the ALPHA model. The alternator model can be
ICF International
4-11
June 15, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Review Comments Grouped by Charge Letter Topic
updated to include, for example, a full 2D efficiency map. However, at this time, the alternator
model remains simplified and the approach taken is consistent with the 2011 Ricardo modeling
work performed for the 2017-2015 FRM.
The electric/mechanical loads are all fairly simple map-based which matches the overall fidelity of
the model. They will, of course, need to be calibrated to somehow represent future vehicle
systems. How that will be done is an important item to consider for the future application of the
model.
EPA Response: At this time, future vehicles are modeled with a slightly improved alternator
efficiency relative to current alternators.
7.	Vehicle Sub Model - The vehicle driver model typical of model of this class.
EPA Response: Thank you for your comment.
8.	Transmission Input File - The input files are appropriate given the model - it is done in a pretty
typical way and should work well. As mentioned in the model sections, the input files contain a lot of
parameters - particularly the automatic transmission one. The main concern here is how one would
arrive at this large list of parameters given the intended use of the model.
EPA Response: As mentioned in a previous response, the data used for automatic transmissions was
obtained through laboratory benchmarking of several transmissions.
9.	Engine Input File
The input file contains a handful of unspecified parameters - not causing any issues but it is
strange to have them there if they are unused. It is not sure how something like fuel octane
number would be used in this type of model.
EPA Response: Fuel properties are stored in a simple database format, fuel Octane is simply one of
the fields and is automatically pulled in to the fuel properties object regardless of whether it is used
in the model. Extra information associated with fuel properties is important in understanding the
performance level of the engine and the conditions under which it was mapped or modeled.
Input file is appropriate given the model - done in a very typical way.
EPA Response: Thank you for your comment.
10.	Other Input Files - Pretty typical of what one would expect from the accompanying model.
EPA Response: Thank you for your comment.
11.	Output Structures
ICF International
4-12
June 15, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Review Comments Grouped by Charge Letter Topic
The output used a structure of structures and included a wide range of data (time-domain,
summary data, etc.) This could be adapted as necessary to bring in whatever data was deemed
necessary.
I may have missed it because it is hard to navigate through all of a nested set of structures, but I
did not see that the model input data was stored in the output structure. If not, then it should
definitely be included as well as the model version (which was included.) This is so that a model
result can be rerun if necessary using the same input parameters without having to track down the
initialization files that created the model input.
EPA Response: For a normal Simulink run, the input data is not saved as part of the output, it's
already present the in the MatLab workspace (which can be saved in its entirety, including the
outputs). However, when running the executable version of the model, a complete input data file is
saved and may be reviewed or reloaded later in conjunction with the output data file for that
particular simulation. In this way the initial conditions can be produced in order to replicate or
troubleshoot the simulation.
icq
1.	ICCT checked the underlying equations for the physical models and in general they seem
reasonable, although perhaps simplified (e.g. rotational dynamics simply assume an "equivalent
mass" to account for rotational inertia). We also evaluated the impacts of selected changes in
model inputs on the outputs and we are providing our views on overall model structure from a
perspective of the final user. We also considered the types of outputs that can be relevant for the
development and operation of the updated OMEGA model that is expected to be released along
with the release of this ALPHA model.
EPA Response: Converting inertias to equivalent mass is acceptable as it maintains conservation
of energy and simplifies the model.
2.	Test Runs
ICCT conducted a series of runs by changing a parameter at a time and observing the result in terms
of 2-cycle C02 emissions. Note that our simplified parametric test did not include constant
performance, due to the iterative modeling required to match performance, thus no changes were
made to the model engine size.
The results of the parametric test, summarized in Table 1, confirm the results of the parameter
estimates in the SAE paper - load reduction results in a constant gC02/mi reduction, regardless of
the baseline fuel consumption when no changes are made to engine size. Note that this was true for
mass, rolling resistance (RR), and aerodynamic drag (Cd) reductions - in every case the gC02/mi
reduction for the NA+AT5 vehicle, with 270.5 gC02/mi, was almost identical to the gC02/mi for the
ICF International
4-13
June 15, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Review Comments Grouped by Charge Letter Topic
EGRB24_TDS+8DCT+ALT vehicle, with 189.7 gC02/mi.2 This defies basic theory, as fuel consumption
(and C02) is generally proportional to vehicle load. In addition, ICCT has reduced load using both
FEV's and Ricardo's simulation models, and both modeled proportional reductions in gC02/mi, not a
constant g/mi reduction. This strongly suggests that the model has errors in the underlying
equations or coding with respect to all of the load reductions.
2-cycle ALPHA results
Emission results, gC02/mi
Reductions, gC02/mi
NA +AT5
NA + CVT+SS
EGRB24_TDS+
8DCT+ALT
NA +AT5
NA + CVT+SS
EGRB24_TDS+
8DCT+ALT
Baseline per model
270.5
250.1
189.7
-
-
-
Mass red 5%
265.4
244.9
184.6
5.1
5.2
5.1
Mass red 10%
260.8
239.2
179.9
9.8
10.9
9.8
Mass red 15%
256.1
234.6
174.3
14.4
15.5
15.4
RR red 10%
267.1
246.1
186.2
3.4
4.0
3.4
RR red 20%
263.1
242.6
182.2
7.4
7.4
7.4
Cd red 10%
266.3
245.3
185.4
4.3
4.8
4.3
Cd red 20%
261.9
241.5
181.1
8.6
8.6
8.6
EPA Response: The reviewer's conclusion and analysis shown in the chart above examines the effect of
applying a constant vehicle road load reduction to three different powertrains. Our understanding of this
comment is that the reviewer expected to find C02 g/mi reductions proportional (relatively equal to) to
the base C02 g/mi, rather than near constant reductions in C02.
Although in some circumstances the reviewer is correct that gC02/mi reduction is a function of the
baseline fuel consumption, this general principle holds best for vehicles with a range of road loads but
similar powertrain technology. On the other hand, the principle does not hold for vehicles with similar
vehicle road loads but a range of different powertrain technology. The text box following this response
contains a more technical discussion of the reasons behind this principle. The analysis in EPA's SAE
paper3 compared vehicles with similar vehicle road loads, but different powertrain technology.
Runs similar to those referenced by the reviewer were also completed by Argonne National Laboratory
using their Autonomie vehicle simulation tool.4 We used the Autonomie results to calculate the
2	The SAE paper reported mass reduction results after adjusting for performance and found C02 reductions varied more
proportionally with the baseline vehicle gC02/mi. Perhaps correcting for constant performance shifted the C02 reduction to
more of a constant percent reduction instead of a constant g/mi reduction, but we did not see this when reducing mass
without correcting for performance.
3	EPA's SAE paper: Kargul, J., Moskalik, A., Barba, D., Newman, K. et al., "Estimating GHG Reduction from Combinations of
Current Best-Available and Future Powertrain and Vehicle Technologies for a Midsized Car Using EPA's ALPHA Model," SAE
Technical Paper 2016-01-0910, 2016, doi:10.4271/2016-01-0910
4	The Autonomie results are publically available from the National Highway Traffic Administration, via their Midterm Evaluation
website located at: http://www.nhtsa.gov/Laws+&+Regulations/CAFE+-+Fuel+Economy/ld-cafe-midterm-evaluation-2022-25
ICF International
4-14
June 15, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Review Comments Grouped by Charge Letter Topic
incremental C02 reduction associated with a reduction in various road loads. The same trend of
approximately constant C02 reduction was observed (for example, the effect of reducing the aero loads
by 10% within both models, for a mid-sized car equipped with a range of conventional engine,
transmission, and other technologies, is shown in the figures below), with numbers roughly
corresponding to the ALPHA results, as well as with those in the table above in the reviewer's comment.
We believe that the effect of the road load change was correctly calculated using both models, and
should also be observed from any other well designed and physics-based vehicle simulation model.
Results from EPA's ALPHA Model
Similar Results from Autonomie
210 230 250
Bases/mile CD2e
ICF International
4-15
June 15, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Review Comments Grouped by Charge Letter Topic
as higher fuel requirement for
incremental torque).
Additional Technical Discussion for Previous EPA Response:
Generally in a high level analysis of a powertrain, the total fuel used at any operational point consist of some
constant fuel to cover parasitic overhead losses that are independent of road load/power, plus incremental fuel to
cover the power needs that are proportional to road load/power, but independent of the overhead losses. Thus,
when incremental fuel needs are roughly the same for two different engines, the incremental additional fuel saved
for an incremental decrease in road load remains constant (as does the incremental fuel consumed for an
incremental increase in road load).
A recent SAE paper does an excellent job of describing this for the curious. (Patrick Phlips, "Analytic Engine and
Transmission Models for Vehicle Fuel Consumption Estimation," SAE International Journal of Fuels and Lubricants,
8(2):2015, doi: 10.4271/2015-01-0981) The paper states that there is "historically fairly wide recognition that
internal combustion engine fuel use increases proportionally with output, with an offset related to engine losses
[overhead losses] and a slope related to indicated efficiency [marginal efficiency]." [bracketed words added]
Many technologies used to increase powertrain efficiency (engine or transmission friction reduction, as an obvious
example) decrease overhead losses rather than increase marginal efficiency. Some engine technologies (increasing
compression ratio, for example) do increase marginal efficiency; however, many of the most popular advanced
technologies - those reducing pumping work in engines - reduce the overhead losses at the expense of decreasing
marginal efficiency.
So, although it's not a hard and fast rule that marginal efficiency (and thus incremental C02 consumption) remain
the same across technology packages, it should not be surprising to find that this is indeed approximately the case.
For reductions in C02 to only be proportional to the base C02 emissions, as suggested by the reviewer, would
require advanced technologies to exclusively increase marginal efficiency with absolutely no effect on overhead
losses, which is certainly not the case.
As an example, the figure below illustrates the incremental fuel usage required to produce one additional Nm of
torque from the engine alone, for a NA PFI engine versus a turbo-downsized (TDS) 24 bar engine with EGR. In the
area most used during the two-cycle testing, the two engines have roughly equivalent incremental fuel
requirements, with the TDS engine actually requiring slightly more incremental fuel (likely due to the introduction
of EGR and consequent reduction in pumping losses). As can be inferred from the figure, an identical decrease in
road load for vehicles containing these two engines would lead to very similar reductions in fuel consumption.
Additional ng/shot fuel required for INm incremental torque
increase (average, lOOOrpm - 2000rpm)
4
3.5
3
2.5
	NA PFI engine
	TDS 24bar w/ EGR
2
1.5
1
0.5
0
0	20	40	60	80	100
Torque (Nm)
Note:
Generally, the TDS engine is
quite a bit more efficient than
the naturally aspirated engine,
but this higher efficiency is due
to the reduction in overhead
losses (not shown in the figure)
which more than offsets the
slight decrease in marginal
efficiency (shown in the figure
as higher fuel requirement for
incremental torque).
ICF International
4-16
June 15, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Review Comments Grouped by Charge Letter Topic
3. Accessories
According to the supporting documents, "power steering, air conditioning, fan and a generic load to
cover the remaining losses are observed. Each load can apply mechanical loads to the engine
crankshaft and/or electrical loads to the battery. Each load can be independently correlated to
model signals via dynamic lookup tables, and is calibrated to match test data."
Access to defining such losses was difficult to find and the structure of the inputs was not clearly
defined in the model. Allowing accessory power consumption to be user-defined inputs could
promote developments in technologies that reduce the power requirements of accessories such as
the alternator, air-conditioning compressor, power steering pump, or cooling fan. There are other
opportunities for engine accessories such as oil, coolant, and fuel pumps, but is not clear at this
point if all those savings are going to be captured by the engine mapping process. Accurate
accounting of the benefits of advanced accessories is extremely relevant to the implementation of
future off-cycle credits for GHG.
EPA Response: The quoted reference refers to the structure of the model, not necessarily what
was providedor what is used, for MTE modeling. These represent modeling flexibilities, not
requirements. At this time, the MTE modeling for current and future vehicles assumes a fixed
electrical load that represents the estimated expected total average load. Improvements to oil,
coolant, and fuel pumps would presumably be reflected in improved steady-state engine maps
since these components would be accounted for during engine dyno testing. Off-cycle credit
modeling is not considered by ALPHA at this time.
Topic 3: Use of good engineering judgment to ensure robust and expeditious program execution
Shawn Midland-Mohler
1.	See comments above regarding the possible higher than necessary model fidelity.
EPA Response: High model fidelity is a matter of flexibility, not a requirement. See previous
responses such as the one for Shawn Midland-Mohler Topic 1-Q1.
2.	The choice of a 0.01 sec step size should be validated by running a step size independence study.
If that step size is unnecessary then it would be possible to decrease the model run time.
EPA Response: EPA has run studies as suggested by the reviewer. The model may run slightly
faster, perhaps with 0.02 second steps, but larger steps may cause instability in, for example, the
engine idle speed control. See response for next comment as well.
3.	To run a HWFET, the model as delivered required 175.8 seconds to run. Enabling the 'accelerator'
option in Simulink took it down to 89.7 seconds including the time to build it - and only 32.9
ICF International
4-17
June 15, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Review Comments Grouped by Charge Letter Topic
seconds if the model did not need to be rebuilt for a run. The model developers are likely already
aware of this feature in Simulink - if not then they should familiarize themselves with it.
EPA Response: EPA staff are aware of this feature in Simulink. As mentioned previously, EPA uses a
fully compiled version of the model when running batch simulations that only takes a few seconds to
run.
4. In terms of robustness, I do have some concerns regarding the amount of control logic that is
imbedded at various places. The model fidelity requires some of this - but there are also a
number of things that are not contributing to fuel economy greatly (some were noted in the
submodel notes above.) These controls can lead to problems in certain applications if they are
not adapted accordingly.
EPA Response: The reviewer's point is well received. Since the basic underpinnings of the ALPHA
model are shared by the heavy-duty simulation model GEM and the heavy-duty GHG compliance
certification tool, the model has been specifically designed to adjust to a wide variety of target
vehicles from compact cars to Class 8 tractor-trailer vehicles with little or no intervention required
by the user. In other words, the controls are designed to scale with the application, either in pre-
processing or in the model itself.
icq
1. In our opinion, the best measure of engineering judgment and proper program execution is
obtaining good agreement between ALPHA simulations and actual testing data. The
documentation reviewed suggests that the errors over the FTP and highway drive cycles are often
within 3%, which are within the +/-3% test-to-test variability of chassis dynamometer testing.
EPA Response: We agree with the reviewer's input. As stated in earlier responses, EPA has spent
considerable time and effort to validate the model and its inputs. These efforts are documented
for example, in a number ofSAE papers that have been published in the last two years and will
continue to be published.
Topic 4: Clarity, completeness and accuracy of the output/results
Suiit Das
1. A simplified set of model outputs exists consisting of (a) Energy Audit Report; (b) SAE J2951 Drive
Quality Metrics; and (3) Fuel Consumption and Economy and C02 emissions by two specific drive
cycles. Although the specific details are not intense, but appropriate enough for the overall model
objective. In addition, sixty specific plots as a function of drive cycle time are available. Use of
MatLab/Simulink modeling software allows us to examine variables of common interest generated
in the MatLab workspace for each simulation. Unless an expert MatLab/Simulink, it is not intuitive
to track down the logical flow of summary final results from its initial parameter values used in
underlying equations. Complete model documentation would be helpful in this regard.
ICF International
4-18
June 15, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Review Comments Grouped by Charge Letter Topic
EPA Response: The large number of plots provided allow for one element of quality control and for
demonstration purposes and not a complete set of outputs. Many possibilities exist for data
analysis and ALPHA applies a "data class" methodology that is consistent when examining
benchmarking data as well as model outputs so that they may be directly compared, if desired, in a
common and documented name space.
As stated in our previous responses regarding documentation (Sujit Das' Topic 1-Q3), EPA has
provided model documentation that can be used by outside parties to observe and review the
model. We plan to continually look for ways to enhance the ALPHA documentation as
opportunities arise.
2. Model simulation results for the three transmission cases provided for the review were
appropriate as one would expect. Higher fuel economy and resulting lower C02 emissions were
obtained with more efficient transmission technology, maximum in the case of dual clutch
transmission (DCT). For the maximum efficient DCT technology, a simple sensitivity case was run
by increasing the vehicle chassis mass by 175 lbs. A 4.8% increase in vehicle chassis mass resulted
in a 2.1% decrease in fuel economy.
EPA Response: Note that the different powertrains (AT/CVT/DCT) also represented different
configurations in terms of stop/start and alternator regen. In this case the DCT included alternator
regen and the CVT included stop/start. This approach was intended to demonstrate a range of
ALPHA'S modeling capabilities for this peer review.
Shawn Midland-Mohler
1.	Without having validation data, there is no way to evaluate accuracy. It was also noted in the
peer review directive that we were looking mainly at model structure as it is not yet fully
calibrated.
EPA Response: ALPHA development/validation is ongoing and the simulations provided for the
peer review were for example purposes. ALPHA validation activities have utilized test data from
over 25 different vehicles, engines and transmissions. EPA's model validation efforts have been
documented in SAE papers and other forums. Please refer to the MTE website for additional
information: https://www3.epa.gov/otaq/climate/mte.htm
2.	The energy balance that is conducted is the first step in ensuring things are working well. More
would need to be done do validate the model.
EPA Response: We agree with the reviewers comment. Energy balance within ALPHA simulation
runs is definitely one of the measures we utilize to validate the model. Again, model validation
has been accomplished by using newly acquired in-depth vehicle, engine, and transmission
benchmarking data from more than 25 different types of conventional and hybrid vehicles 2013-
2015.
ICF International
4-19
June 15, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Review Comments Grouped by Charge Letter Topic
3. The model appears to have the major vehicle system models one would expect for fuel economy
prediction.
EPA Response: Thank you for your comment.
icq
1.	The report is very thorough, including a detailed energy audit and 50+ figures, which is
commendable. However, we recommend that a smaller "summary" report, only with the very key
parameters (fuel consumption, engine cycle efficiency, speed-trace following metrics) be
produced for easy tracking of multiple runs.
EPA Response: Thank you for your comment, we will consider such an approach for future versions
of ALPHA.
2.	The input and output structure of ALPHA was not finalized when released for peer review,
however the current version of the output structure were provided to give the reviewer a flavor of
the potential structure. The inclusion of performance metrics is highly commended, although we
suggest spelling out some metrics in the output file to facilitate troubleshooting and give the user
a better perspective.
EPA Response: Performance metrics are an important component of the MTE modeling effort to
enable powertrains to be properly sized while moving from one technology to another. We will
take your suggestion under advisement for future versions of ALPHA.
Topic 5: Any recommendations for specific improvements to the functioning or the quality of the
outputs of the model.
Suiit Das
1.	A detailed model documentation including a detailed listing of model variables definition is
necessary to satisfy the model objective of transparency.
EPA Response: Thank you for your comment. Documentation of model variables is an ongoing
component of our current ALPHA work as exemplified by using MatLab classes as much as possible
(instead of structures, for example) as they provide well formatted documentation sheets when
used as intended. As stated in earlier responses about documentation (Sujit Das' Topic 1-Q3), we
plan to release more detailed ALPHA documentation as appropriate to continue our commitment
to transparency.
2.	Timely availability of the validated vehicle system data for the potential technology pathways to
be considered for the EPA mid-term evaluation will be critical to achieve the model objective.
ICF International
4-20
June 15, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Review Comments Grouped by Charge Letter Topic
EPA Response: EPA has publically released the modeling assumptions and benchmarking data as
part of the MTE process.
3.	Any comparative analysis with the similar forward-looking, full vehicle computer simulation model
such as AUTONOMIE used by U.S. Department of Energy will be useful towards the model
validation.
EPA Response: Several comparative analyses were completed as part of the coordination efforts
between EPA and NHTSA during the ongoing MTE process. ALPHA results have been compared
with the original MY2017-2025 LD FRM Ricardo EASY5 results, as well as with results from
AUTONOMIE. When the models are viewed as a calculators, then providing the same inputs to the
calculators should provide the same outputs. In some cases there were minor differences between
simulation results due to specific model behaviors or implementations, but in effect these models
are very close in terms of computational results when run using the same input assumptions.
4.	It is critical that the level of accuracy of vehicle performance results obtained from a simplistic
model such as ALPHA be frequently demonstrated and documented to meet the stringent
requirements of any Federal regulation such as CAFE in this case.
EPA Response: We agree that it is important to document ALPHA, and for this reason EPA has
spent considerable time and effort producing publicly available ALPHA documentation in
publications such as the Draft Technical Assessment Report (TAR), benchmarking data, key
ALPHA input file descriptions, key ALPHA result outputs, and SAE papers and presentations.
However, it is also important to note that ALPHA is EPA's internal tool used to estimate future
C02 emissions, not a regulation compliance tool for either EPA's GHG or NHTSA's CAFE
regulations.
5.	It'd be good to add Aftertreatment component module in order to enhance ALPHA estimation
capability of several other types of pollutants beyond C02.
EPA Response: We agree this would be a good enhancement to ALPHA if there ever becomes a
need to estimate criteria pollutants other than C02. Because the light-duty GHG rules does not
require this, we have not included it in this version of the ALPHA model.
6.	It is recommended that to better simulate a vehicles cold-start operation, a study of bag fuel
consumption reported in EPA certification data would be useful for a better understanding of
the cold-temperature correction factor (currently used in ALPHA simulations) trend with new
model vehicles.
EPA Response: EPA has performed this examination using certification data and used the
results to determine the nominal correction factor used for bag one of the FTP. Further testing
with four-bag FTPs and warm vs. cold UDDS cycles was used to determine the nominal
correction factor used for bag two of the FTP.
ICF International
4-21
June 15, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Review Comments Grouped by Charge Letter Topic
icq
1.	It would be useful for the user have access to a list of all the relevant vehicle parameters that can
be modified and the corresponding file where those parameters are input. Some parameters are
intuitive, but some others may fall out of sight unless there is a set list. Moreover, EPA has
invested a great deal of time and resources making sure the physical models are representing
the physical elements, and a parameters list would help produce a better representation of the
system being modeled.
EPA Response: Currently, we use MatLab Classes to help document the available parameters for
the primary components. As stated in our response about documentation (Sujit Das' Topic 1-Q3),
we plan to continue to enhance ALPHA documentation in the future as appropriate.
2.	The way the model handles vehicle weight, tire rolling resistance, and Cd inputs may be improved.
As presented in the model, when changing vehicle mass, the user must change the vehicle mass
and separately change the rolling resistance by changing the A coefficient on the road load
equation. In the same way, changes in RR values have to be performed in the model by changing
the A coefficient. It would be better suited and less prone to errors to input these parameters
separately and program the ALPHA code to do the changes automatically. It would also be
desirable to provide flexibility to input either m, A, B, and C; or m, Cd, and RRc.
EPA Response: ALPHA supports both ABC and coefficient-based road loads. There are benefits and
liabilities to each approach. When modeling for the 2016 Draft TAR, the roadload corrections, as a
function of mass, for example, are handled in the processing steps prior to model execution.
3.	We were not able to find any information on how the model handles component weight
changes. Specifically, how does ALPHA handle changes in mass due to components with
different mass? For example, if an AT6 is being replaced with a DCT8 and that there are mass
changes associated with that. Does the delta in transmission mass change affect the vehicle
overall weight or road load parameter A? The model needs to at least inform the user that it
the user's responsibility to assess and incorporate any impacts of changes in components on
vehicle mass.
EPA Response: ALPHA does not track changes to component masses individually, only the total
vehicle weight is embodied in the test weight/inertia. Within ALPHA, for the draft Technology
Assessment Report (TAR), mass and roadload reductions are considered separately from
individual component specifications.
4.	Extra fuel usage for other vehicle operating requirements. The introduction of overhead
functions to compensate the steady-state based model for fuel used during transient conditions
is very useful. Our recommendation is to make those key adjustments available to the user on a
clearer way; such as a list of functions and parameters, or maybe developing a parameter list
that deals with such items.
ICF International
4-22
June 15, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Review Comments Grouped by Charge Letter Topic
EPA Response: We agree, and EPA staff are working on a peer reviewed technical paper on this
topic at this time. Transient operation penalties are also currently under active investigation.
See previous responses regarding user documentation.
ICF International
4-23
June 15, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Review Comments Grouped by Charge Letter Topic
ICF International
4-24
June 15, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Resumes and Conflict of Interest Statements
Appendix A. Resumes and Conflict of Interest
Statements
ICF International
A-l
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Appendix B. Charge Letter
ICF International
B-l
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Charge Letter
May 7th, 2016
Sujit Das
Oak Ridge National Laboratory
Energy and Transportation Science Division
12305 Fort West Drive
Knoxville, TN 37934
Subj ect: Peer Review of EPA ALPHA Model
Dear Mr. Das,
ICF International has been contracted by EPA to facilitate a peer review. In late April we corresponded
by email and you indicated your availability to participate as a paid reviewer of the EPA Advanced Light-
Duty Powertrain and Hybrid Analysis (ALPHA) model. You have been selected to participate on this
panel. ICF will compensate you up to $4,000 for your services. This charge letter provides you with a list
of directed questions for your review, the review schedule, and the materials we would like you to send
to us at the conclusion of the review. In addition, attached to the email is a Zip file with a copy of the
model that we would like you to review. You will need to rename the file with a zip extension to open it.
Charge Questions
The purpose of this peer review is to examine the structure, operation, and simulation results of the
ALPHA tool used by EPA to determine the effectiveness of various technologies via simulation. For this
review, no independent data analysis is required. EPA is looking for the reviewer's opinion of the
concepts and methodologies upon which the model relies and whether or not the model can be
expected to execute these algorithms correctly. This review concentrates on the implementation of
conventional powertrain (non-hybrid) vehicles. Model inputs such as engine and transmission maps are
not part of this review and are only included to provide the reviewer with a complete functioning model.
Feedback on the technical aspects of ALPHA rather than a critique of variable names or presentation of
results is preferred.
In your comments, please distinguish between recommendations for clearly defined improvements that
can be readily made, based on data or literature reasonably available to EPA, and improvements that
are more exploratory or dependent, which would be based on information not readily available to EPA.
Comments should be clear and detailed enough to EPA readers or other parties familiar with the report
to allow a thorough understanding of the comment's relevance to material provided for review.
Below are questions to define the scope of the review; we are not expecting individual responses to the
questions, but would like them to help guide your response.
General Questions and Issues to Consider
ICF International
B-2
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Charge Letter
1.	EPA's overall approach to the stated purpose of the model (demonstrate technology
effectiveness for various fuel economy improvement technologies) and whether the particular
attributes found in resulting model embodies that purpose.
2.	The appropriateness and completeness of the contents of the overall model structure and its
individual systems, such as:
a.	The performance of the example component models, including the reviewer's
assessment of the underlying equations and/or physical principles coded into that
component.
b.	The input and output structures and how they interface with the model to obtain the
expected result, i.e., fuel consumption over the given driving cycles.
3.	Use of good engineering judgment to ensure robust and expeditious program execution;
4.	Clarity, completeness and accuracy of the output/results; and
5.	Any recommendations for specific improvements to the functioning or the quality of the outputs
of the model.
EPA requests that the reviewers not release the peer review materials or their comments until the
Agency makes its ALPHA model and supporting documentation public. EPA will notify the reviewers
when this occurs.
Schedule
The schedule for this peer review is as follows:
¦	May 7th, 2016: Charge letter distributed to reviewers
¦	June 10th, 2016: Comment/review due via email to Laurence.0'Rourke@icfi.com
Materials
Upon completion of your review, you should submit your report under a cover letter that states 1) your
name, 2) the name and address of your organization, and 3) a statement of any real or perceived
conflict(s) of interest.
Should you have any questions or concerns, feel free to contact me via phone at 617-250-4226 or by
email. In addition, the EPA project manager for this effort is Jeff Cherry and he may be reached at 734-
214-4371 or cherry.jeff@epa.gov. For any questions about the review process itself, please contact Ruth
Schenk in EPA's Quality Office, National Vehicle and Fuel Emissions Laboratory at 734-214-4017 or
schenk.ruth@epa.gov.
Thanks for your participation!
Sincerely,
Larry O'Rourke
Manager, ICF International
Attachment: Alpha Peer Review WA 4-03.EPA
ICF International
B-3
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Sujit Das Comments
Appendix C. Sujit Das Comments
ICF International
C-l
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Sujit Das Comments
Sujit Das
12305 Fort West Drive, Knoxville,TN 37934
Knoxville, TN 37934, USA
(865)789-0299
dass@ornl.gov
June 9, 2016
Larry O'Rourke
Manager, ICF International
RE: Peer Review of Draft Report on EPA ALPHA Model
Dear Mr. O'Rourke:
Thank you for inviting me to conduct a peer review of the Draft Report on EPA ALPHA Model. I have
completed the review.
Enclosed with this letter is a summary of my review comments and recommendations. These comments
are made on the basis of the current state of science as I understand it. To the best of knowledge, I have
no real or perceived conflicts of interest in conducting this review.
Please feel free to contact me should you have any questions or need additional regarding this review.
Sincerely,
Sujit Das
Enclosure: A summary of review comments and recommendations
ICF International
C-2
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Sujit Das Comments
PEER REVIEW COMMENTS
EPA ALPHA MODEL
Sujit Das
Oak Ridge National Laboratory
12305 Fort West Drive, Knoxville, TN 37934
June 10, 2016
Peer Review of EPA ALPHA Model
A. APPROACH
1.	The ALPHA model approach is a fairly simple forward-looking based on underlying physics used in
other similar commercial packages available today. It consists of a simplified structure of total five
modules, three for vehicle, and one each for engine and transmission. It is significantly more
sophisticated in terms of its capability of estimating the C02 and the resulting fuel consumption than the
original lumped parameter model used by EPA in the original analysis of light-duty vehicle GHG emission
standards. Concepts and methodologies implemented in a simplified MatLab/Simulink framework are
consistent with other similar currently available models, although less complicated with the limited
capability such as with the only C02 emissions estimates.
2.	The modular nature of modeling framework provides the flexibility in using a technology specific from
a list of available individually parametrized powertrain components collected by engine and chassis
dynamometer testing to examine the vehicle performance of user-defined specific technology packages.
3.	It is difficult to examine in detail the model approach due to a lack of detailed documentation.
However, several peer review papers ~10 among which include seven recently published at the SAE
2016 Annual Congress related to benchmarking several types of engines and transmissions to generate
inputs for use in ALPHA model have been published.
4.	The model is completely input data driven, which need to be collected by either engine or chassis
dynamometer testing by specific vehicle system technology case. The model application is thereby
ICF International
C-3
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Sujit Das Comments
limited to the extent of validated data availability. The overall model performance is dictated by
calibration of numerous technology-specific parameters used to determine final vehicle fuel economy
and C02 emissions for various vehicle drive cycles.
5. A simplistic approach without any consideration of any aftertreatment is sufficient for C02 emissions
estimation based on the actual fuel consumption.
B.	APPROPRIATENESS AND COMPLETENESS
1.	With the availability of new engine, transmission, and operational control benchmarking data, ALPHA
model would be able to support the 2017-2025 light-duty GHG rule requiring a comprehensive advanced
technology review, known as the mid-term evaluation for the 2022-2025 light-duty GHG emission
standard. The model is flexible enough with the capability to determine the effectiveness contributions
from advanced technologies not considered during the original Federal rulemaking.
2.	The model performance validation is a continuous process, which has been accomplished by using the
newly acquired in-depth vehicle, engine, and transmission benchmarking data from more than 25
different types of conventional and hybrid vehicles 2013-2015. At any point, model appropriateness and
completeness will be dictated by the extent of benchmarking data available for the model performance
validation.
3.. A complete listing of model limitations (e.g., sensitivity of electric power steering losses with vehicle
speed and a lack of dynamic temperature algorithm) is critical for any model validation. However, a few
of these limitations have been discussed in the recent SAE publications.
4.	The primary gear selection routine, the ALPHAshift algorithm critical to the fuel economy and C02
emissions, has recently been validated and updated (based on the recently published SAE paper) to
dynamically generate transmission shift logic from a set of user-defined parameters and generate more
realistic vehicle performance during simulation. Since the modelling approach used is completely data
driven, algorithms for new control strategies need to be developed including its validation of tunable
control parameters in order for reliable vehicle performance estimation using ALPHA.
5.	A recent SAE publication (SAE 2016-01-1142) reports an excellent agreement of fuel consumption
results from a comparative examination of advanced transmissions among studies conducted by
National Research Council, Argonne National Laboratory, and earlier EPA lumped parameter model.
C.	MODEL STRCUTURE AND OUTPUTS/RESULTS
1. It is a fairly simple transparent model which allows to examine both fuel economy and C02 emissions
of alternative light-duty technology pathways. The model execution requires an expert MatLab/Simulink
user since no user-friendly interface currently exists. Although the model use as indicated will be mainly
by in-house EPA experts, but the model validation of its transparency particularly when pertaining to the
Federal rulemaking needs to be addressed. A specific simulation runtime is significantly high, more than
10 mins. without providing any indication to the user progress made so far. A fairly more complicated
model such as Autonomie available even with enhanced capabilities is significantly faster today.
ICF International
C-4
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Sujit Das Comments
2.	Although a model run requires comparatively a fewer number of consolidated input parameter files,
but a lack of proper model documentation (both as a standalone document and within the MatLab files)
makes harder for a better model structure understanding including any sensitivity runs of any user-
specified variables.
3.	A simplified set of model outputs exists consisting of (a) Energy Audit Report; (b) SAE J2951 Drive
Quality Metrics; and (3) Fuel Consumption and Economy and C02 emissions by two specific drive cycles.
Although the specific details are not intense, but appropriate enough for the overall model objective. In
addition, sixty specific plots as a function of drive cycle time are available. Use of MatLab/Simulink
modeling software allows to examine variables of common interest generated in the MatLab workspace
for each simulation. Unless an expert MatLab/Simulink, it is not intuitive to track down the logical flow
of summary final results from its initial parameter values used in underlying equations. A complete
model documentation would be helpful in this regard.
4.	Model simulation results for the three transmission cases provided for the review were appropriate as
one would expect. Higher fuel economy and resulting lower C02 emissions were obtained with more
efficient transmission technology, maximum in the case of dual clutch transmission (DCT). For the
maximum efficient DCT technology, a simple sensitivity case was run by increasing the vehicle chassis
mass by 175 lbs. A 4.8% increase in vehicle chassis mass resulted in a 2.1% decrease in fuel economy.
D. RECOMMENDATIONS
1.	A detailed model documentation including a detailed listing of model variables definition is necessary
to satisfy the model objective of transparency.
2.	Timely availability of the validated vehicle system data for the potential technology pathways to be
considered for the EPA mid-term evaluation will be critical to achieve the model objective.
3.	Any comparative analysis with the similar forward-looking, full vehicle computer simulation model
such as AUTONOMIE used by U.S. Department of Energy will be useful towards the model validation.
4.	It is critical that the level of accuracy of vehicle performance results obtained from a simplistic model
such as ALPHA be frequently demonstrated and documented to meet the stringent requirements of any
Federal regulation such as CAFE in this case.
5.	It'd be good to add Aftertreatment component module in order to enhance ALPHA estimation
capability of several other types of pollutants beyond C02.
6.	It is recommended that to better simulate a vehicle's cold-start operation, a study of bag fuel
consumption reported in EPA certification data would be useful for a better understanding of the cold-
temperature correction factor (currently used in ALPHA simulations) trend with new model vehicles.
ICF International
C-5
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Shawn Midlam-Mohler Comments
Appendix D. Shawn Midlam-Mohler Comments
ICF International
D-l
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Shawn Midlam-Mohler Comments
ICF International
D-2
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Shawn Midlam-Mohler Comments
Shawn Midlam-Mohler
3938 Norbrook Drive
Columbus Ohio 43220
Larry O'Rourke
ICF International, LLC
9300 Lee Highway
Fairfax, VA 22031
Dear Mr. O'Rourke,
Attached you find my comments regarding my peer review of the EPA Alpha model.
I am an employed as a professor by Ohio State University and my campus address is:
OSU Center for Automotive Research
Attn.: Shawn Midlam-Mohler
930 Kinnear Road
Columbus OH 43212
This work was done outside of my normal job duties as consulting, thus, any correspondence
regarding for this work should be sent to my home address:
Shawn Midlam-Mohler
3938 Norbrook Drive
Columbus OH 43212
I am not aware of any real or perceived conflicts of interest that would effect my performance on
this peer review.
I appreciate the opportunity to work with ICF and would welcome additional opportunities.
Sincerely,
Shawn Midlam-Mohler
ICF International
D-3
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Shawn Midlam-Mohler Comments
PEER REVIEW
EPA ALPHA Model
Conducted by: Shawn Midlam-Mohler
Submitted on: 6/13/2016
ICF International
D-4
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Shawn Midlam-Mohler Comments
Overall Summary:
No technical issues were found in the model that would impact its intended use. The main concern I have
is in the overall fidelity of the model. The model reviewed has a decent level of fidelity - perhaps even
greater than required for the intended use. An excellent example would be the use of 4-D maps of certain
parameters in the transmission. The concern is on how one calibrates these maps to be representative of
future technology. If one has the component on a test bench, then it is possible to extract these
parameters, however, that is not the context in which this will be applied. The components being
evaluated don't yet exist in physical form in many cases. One will be left to alter relatively complex non-
physical models that are black-box models of complex physical behavior. I am not sure if all of the
submodels will lend themselves for that type of activity. It can be done, but the question is what will the
effort be to do so and how does one verify the results. A simpler model may be more appropriate in some
cases. Alternately, calibrating the existing models from models of much higher fidelity may be an option
as well. To completely understand this one would need to go through a couple of test cases to understand
the overall process from a workflow perspective.
Driver Submodel:
•	The driver does a good job at manipulating the brake and accelerator pedals in a smooth and
natural way while matching the drive cycle for the vehicle and cycles included in the review. For
different vehicles and cycles, this may or may not be true. Just checking for the error in the target
and actual speed is not enough to check for this as it is possible to meet the desired speed trace
with rapidly oscillating (i.e. unnatural) manipulation of the accelerator and brake signals. This can
lead to poor quality results while driving the trace accurately. This is not a flaw with the approach
used - just that a diagnostic needs to be added to check for this type of behavior to ensure that
the data produced is accurate.
Engine Submodel:
•	There are many controls and calibrations imbedded in the engine plant model. This can be
problematic in some circumstances for model reuse and calibration. An example would be the
ICF International
D-5
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Shawn Midlam-Mohler Comments
decel fuel cutoff strategy and the idle speed control strategy. At a minimum, clarity could be
improved by making control function and plant models visible and different.
•	For a model aimed at fuel economy predictions, there may be more fidelity then strictly necessary.
For instance, use of inertia in the engine model requires additional complexity for the
transmission/engine model for minimal gains in fuel economy prediction. The approach used is
fine but it may be more complex then necessary for the model goals. The use of so many inertias
when things like shift durations simply enforced via the initialization files seems odd - but not
wrong from a technical perspective.
•	I was not able to find any type of thermal model for the engine - which are well known to have
temperature effects for efficiency. Components like the 12 V battery have one - so it is
inconsistent to not include one for the engine.
•	There are several ad-hoc factors like the 'tip-in penalty' that appears in the engine fuel-flow
submodel. It is not clear on how one would calibrate this or what it is really meant to capture
from a physical standpoint. Fuel enrichment on tip is not something that is done in modern engine
controls - air prediction is good enough that in general there is only a minor amount of
enrichment happening. Another example would be the 'acceleration_penalty_squelch_gps'
factor.
Automatic Transmission Submodel:
•	The fidelity of the transmission model and the resulting number of calibration parameters is fairly
high given the goal of the model. There are many look up tables some of which have four
dimensions. It is not clear how one would calibrate these parameters given the context of use.
•	The shift logic appears to produce reasonable shift commands - there are no frequent shifts,
inappropriate skip shifts, etc. There is a great deal of logic in the 'automatic trans control' block.
Similar to previous comments, the question is how does one calibrate all of this for technology
that may not exist?
•	There are many thermal models present - only constant temperature is enabled. Transmission
temperature is important to fuel economy predictions and likely future vehicles will involve more
tightly integrated thermal systems.
ICF International
D-6
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Shawn Midlam-Mohler Comments
Continuously Variable Transmission Submodel:
•	Control logic is much simpler in contrast to the automatic - this is a positive thing in terms of the
intended use
•	There are many thermal models present - only constant temperature is enabled. Transmission
temperature is important to fuel economy predictions and likely future vehicles will involve more
tightly integrated thermal systems.
Dual Clutch Transmission Submodel:
•	There are many thermal models present - only constant temperature is enabled. Transmission
temperature is important to fuel economy predictions and likely future vehicles will involve more
tightly integrated thermal systems.
12V System Model:
•	The 12V battery model is one of the more complex models in the model. The alternator model is
some control logic and a single constant efficiency for current to torque based on speed - so quite
simple. This is a large mismatch in fidelity. Having a fairly elaborate 2nd order RC model for the
battery and then modeling the alternator so simply is not technically incorrect, but is something
to consider.
•	The electric/mechanical loads are all fairly simple map-based which matches the overall fidelity
of the model. They will, of course, need to be calibrated to somehow represent future vehicle
systems. How that will be done is an important item to consider for the future application of the
model.
Vehicle Sub Model:
•	The vehicle driver model typical of model of this class.
Transmission Input File:
ICF International
D-7
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Shawn Midlam-Mohler Comments
•	The input files are appropriate given the model - it is done in a pretty typical way and should work
well.
•	As mentioned in the model sections, the input files contain a lot of parameters - particularly the
automatic transmission one. The main concern here is how one would arrive at this large list of
parameters given the intended use of the model.
Engine Input File:
•	The input file contains a handful of unspecified parameters - not causing any issues but it is
strange to have them there if they are unused. It is not sure how something like fuel octane
number would be used in this type of model.
•	Input file is appropriate given the model - done in a very typical way.
Other Input Files:
•	Pretty typical of what one would expect from the accompanying model.
Output Structures:
•	The output used a structure of structures and included a wide range of data (time-domain,
summary data, etc.) This could be adapted as necessary to bring in whatever data was deemed
necessary.
•	I may have missed it because it is hard to navigate through all of a nested set of structures, but I
did not see that the model input data was stored in the output structure. If not, then it should
definitely be included as well as the model version (which was included.) This is so that a model
result can be rerun if necessary using the same input parameters without having to track down
the initialization files that created the model input.
Program Execution and Robustness:
ICF International
D-8
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Shawn Midlam-Mohler Comments
•	See comments above regarding the possible higher than necessary model fidelity.
•	The choice of a 0.01 sec step size should be validated by running a step size independence study.
If that step size is unnecessary then it would be possible to decrease the model run time.
•	To run a HWFET, the model as delivered required 175.8 seconds to run. Enabling the 'accelerator'
option in Simulink took it down to 89.7 seconds including the time to build it - and only 32.9
seconds if the model did not need to be rebuilt for a run. The model developers are likely already
aware of this feature in Simulink - if not then they should familiarize themselves with it.
•	In terms of robustness, I do have some concerns regarding the amount of control logic that is
imbedded at various places. The model fidelity requires some of this - but there are also a number
of things that are not contributing to fuel economy greatly (some were noted in the submodel
notes above.) These controls can lead to problems in certain applications if they are not adapted
accordingly.
Accuracy, Clarity, and Completeness:
•	Without having validation data, there is no way to evaluate accuracy. It was also noted in the
peer review directive that we were looking mainly at model structure as it is not yet fully
calibrated.
•	The energy balance that is conducted is the first step in ensuring things are working well. More
would need to be done do validate the model.
•	The model appears to have the major vehicle system models one would expect for fuel economy
prediction.
ICF International
D-9
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Appendix E. ICCT-John German (lead reviewer),
Anup Bandivadekar, Oscar Delgado, Francisco
Posada Comments
ICF International
E-l
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Peer Review Comments of EPA's ALPHA model
Francisco Posada, Oscar Delgado, John German
The International Council on Clean Transportation
1225 I St NW Suite 900
john@theicct.org
INTRODUCTION
This document summarizes the findings of ICCT's review of the US EPA's the EPA Advanced Light-Duty
Powertrain and Hybrid Analysis (ALPHA) model and supporting documentation ("Chapter 5: Technology
Cost, Effectiveness, and Feasibility Assessment"). The tool will serve as one of the principal supports for
EPA's midterm review phase of the Light-Duty GHG 2022-2025 emissions regulations. The agencies
consider simulation modeling to be critical to assess the expected real-world performance of various
vehicle, engine, and transmission technologies. The main purpose of this review is to evaluate how well
the developed model can serve as a regulatory and compliance tool.
I. GENERAL IMPRESSIONS
After reviewing the MatLab/Simulink model and the accompanying report, our general impression is
that the ALPHA model tool constitutes a valuable development effort by EPA and a major step forward.
It is beyond the limitations of our review to determine if the ALPHA model is more accurate than the
preceding model developed by Ricardo, but the ALPHA model offers two major advantages. The first is
the transparency of the model. EPA intends to publicly release both the model and the data used for the
modeling, which is a huge improvement over the confidential Ricardo model. The second is the flexibility
to develop maps and algorithms for future technology and efficiency improvements and to model a
wide variety of scenarios.
The model architecture is clear and easy to follow and has incorporated some key features that enhance
its overall accuracy with respect to real world performance of technologies, and allows the model to
capture fuel consumption reductions from a broad range of technologies. In our opinion, the model will
be capable of performing its intended purpose of reflecting technology benefits for compliance
purposes of most of the technologies that the agencies are considering.
Some features are specially welcome, namely the ability of the model to incorporate user-defined
engine fueling maps and driveline parameters, the development of different transmission options
(especially CVTs and DCTs), and the enhanced transmission gear-shifting strategy. We are also
impressed by the testing effort that was done to validate the model and the fact that the model is
capturing the incorporated technologies in close agreement with engine and chassis dynamometer
testing, which is not an easy task.
The documentation available for this peer review appears to be at an early draft stage and needs to be
enhanced. There is an overall lack of detail on key technical features that are new in the model. Proper
ICF International
E-2
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
documentation needs to be provided. An interested reader would like to see a better description of such
features, how they were developed, and perhaps, more quantitative results.
Although we understand that most of the key technical design elements have been already peer-
reviewed and published in SAE Technical papers, some claims in the report need to be appropriately
quantified, and supported via appendices or any means that don't require direct access to the SAE
documents. Also, the quality of the report may be enhanced with an appendix containing a table with all
parameters and the respective file names (filename.m or equivalent) that contain each parameter. The
Figures in the documentation also need to be improved. Most of the inputs and outputs simply
reference "system bus" or "bus_out", instead of describing how each component is input or output to
other components.
II. RESPONSE TO CHARGE QUESTIONS
1. EPA's overall approach to the stated purpose of the model (demonstrate technology
effectiveness for various fuel economy improvement technologies) and whether the particular
attributes found in resulting model embodies that purpose.
The model in its current form will be capable of performing its intended purpose of modeling
technology benefits for most of the technologies that the agencies are considering. Moreover, the
inclusion of CVT technologies as part of the modeling efforts shows the commitment by EPA to
include all potential technology pathways to meet the targets. We do recommend that a table be
added in the documentation that informs the reader of what technologies ALPHA is capable of
modeling and what technologies are yet to be implemented in the model or the model is incapable to
simulate.
Some specific model elements that could be improved to better reflect technologies in the future
that may be impacted as well by Tier 3/LEV III emission standards:
1)	Cold start operation modeling: ALPHA does not simulate or model cold-start operation, instead
applying the adjustment factors derived by Ricardo for Ricardo's 2011 modeling for EPA. This
approach likely works fine for current, known technology, but it is likely inadequate for future
engines with fast warm-up strategies, especially considering the upcoming changes to emission
standards. There is also no ability to model other drive cycles with a cold start, as the adjustment
factors are specific to the FTP.
Also, specifically relating to the correction factors for Bag 1, where the fuel consumed during this
period is increased by around 16-17%, how does ALPHA correct the energy audit to account for this
correction? Would that 16-17% extra be reflected on energy losses, thermal or mechanical? The
effect of the correction on the energy audit should be described.
2)	According to the documentation review, ALPHA'S stop/start modeling appears to be very
simplistic. Their description says, "Alpha contains a sub-model for 12 volt electrical start-stop
technology, which simulates shutting the engine off after vehicle has stopped moving for 0.1 second.
During a simulation, the start-stop mode is disabled when the vehicle is assumed to be operating cold
ICF International
E-3
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
such as during the first 100 sec of bag 1 of the FTP cycle." Potential limitations of this approach to SS
technology modeling are:
•	No ability to do stop/start during coasting or deceleration (sailing).
•	The length of stop/start disablement after a cold start appears to be completely arbitrary.
Note that actually modeling cold start operation, instead of using a simple adjustment
factor, would fix this problem as well, although we recognize that this would require
development of more sophisticated modeling.
3)	Engine scaling to maintain vehicle performance. ICCT's comments regarding this approach have
two components, dealing with the technical aspects and the way it is incorporated into the ALPHA
model.
First, ALPHA'S approach to maintain the vehicle performance when reducing weight is to scale the
BSFC maps to increase fuel consumption while downsizing an engine, reflecting the increase in heat
losses due to higher cylinder surface area to volume ratio. Ricardo also used this method on their
previous modeling work. Although the idea is technically sound, the limitation that we perceive with
this approach is that it ignores the option of reducing the number of cylinders, which would decrease
the cylinder surface area to volume ratio. The option of using an engine with fewer cylinders implies
that the model would have to incorporate algorithms that can select the appropriate engine for
downsizing; this implies developing additional engine maps and criteria for selecting the right one.
Second, the documentation describes the development of a parametric analysis tool to compare the
different technology approaches to find the best technology option (SAE 2016-01-0910). In this case,
the ideal outcome would be to incorporate the ability to change number of cylinders, i.e., engine
maps, as part of the effort.
4)	Synergy between engines and transmissions. In the SAE paper describing ALPHAs performance,
the authors correctly note that engines and transmissions have some overlap in their benefits and
derive parametric estimates for the "synergy factor" (SAE 2016-01-0910). As shown in Table 7 of that
paper, the synergy factor is negative for 6AT, 8AT, and future 8AT. But it is positive for the future
8DCT - which basically defies theory. EPA should evaluate why there are positive synergies between
engines and future 8DCTs and, if this is not an error in the modeling, describe in the next iteration of
the supporting documentation how the synergy factor was determined for future 8DCTs and why it
has positive synergies.
2. The appropriateness and completeness of the contents of the overall model structure and its
individual systems, such as:
a.	The performance of the example component models, including the reviewer's
assessment of the underlying equations and/or physical principles coded into that
component.
b.	The input and output structures and how they interface with the model to obtain the
expected result, i.e., fuel consumption over the given driving cycles.
ICCT checked the underlying equations for the physical models and in general they seem reasonable,
although perhaps simplified (e.g. rotational dynamics simply assume an "equivalent mass" to account
ICF International
E-4
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
for rotational inertia). We also evaluated the impacts of selected changes in model inputs on the
outputs and we are providing our views on overall model structure from a perspective of the final
user. We also considered the types of outputs that can be relevant for the development and
operation of the updated OMEGA model that is expected to be released along with the release of this
ALPHA model.
Test Runs
ICCT conducted a series of runs by changing a parameter at a time and observing the result in terms
of 2-cycle C02 emissions. Note that our simplified parametric test did not include constant
performance, due to the iterative modeling required to match performance, thus no changes were
made to the model engine size.
The results of the parametric test, summarized in Table 1, confirm the results of the parameter
estimates in the SAE paper - load reduction results in a constant gC02/mi reduction, regardless of the
baseline fuel consumption when no changes are made to engine size. Note that this was true for
mass, rolling resistance (RR), and aerodynamic drag (Cd) reductions - in every case the gC02/mi
reduction for the NA+AT5 vehicle, with 270.5 gC02/mi, was almost identical to the gC02/mi for the
EGRB24_TDS+8DCT+ALT vehicle, with 189.7 gC02/mi.5 This defies basic theory, as fuel consumption
(and C02) is generally proportional to vehicle load. In addition, ICCT has reduced load using both
FEV's and Ricardo's simulation models, and both modeled proportional reductions in gC02/mi, not a
constant g/mi reduction. This strongly suggests that the model has errors in the underlying
equations or coding with respect to all of the load reductions.
2-cycle ALPHA
results
Emission results, gC02/mi
Reductions, gC02/mi
NA +AT5
NA +
CVT+SS
EGRB24_TDS
+
8DCT+ALT
NA +AT5
NA +
CVT+SS
EGRB24_TDS+
8DCT+ALT
Baseline per
model
270.5
250.1
189.7



Mass red 5%
265.4
244.9
184.6
5.1
5.2
5.1
Mass red 10%
260.8
239.2
179.9
9.8
10.9
9.8
Mass red 15%
256.1
234.6
174.3
14.4
15.5
15.4
RR red 10%
267.1
246.1
186.2
3.4
4.0
3.4
RR red 20%
263.1
242.6
182.2
7.4
7.4
7.4
Cd red 10%
266.3
245.3
185.4
4.3
4.8
4.3
Cd red 20%
261.9
241.5
181.1
8.6
8.6
8.6
5 The SAE paper reported mass reduction results after adjusting for performance and found C02
reductions varied more proportionally with the baseline vehicle gC02/mi. Perhaps correcting for
constant performance shifted the C02 reduction to more of a constant percent reduction instead of a
constant g/mi reduction, but we did not see this when reducing mass without correcting for
performance.
ICF International
E-5
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
Accessories
According to the supporting documents, "power steering, air conditioning, fan and a generic load to
cover the remaining losses are observed. Each load can apply mechanical loads to the engine
crankshaft and/or electrical loads to the battery. Each load can be independently correlated to
model signals via dynamic lookup tables, and is calibrated to match test data."
Access to defining such losses was difficult to find and the structure of the inputs was not clearly
defined in the model. Allowing accessory power consumption to be user-defined inputs could
promote developments in technologies that reduce the power requirements of accessories such as
the alternator, air-conditioning compressor, power steering pump, or cooling fan. There are other
opportunities for engine accessories such as oil, coolant, and fuel pumps, but is not clear at this point
if all those savings are going to be captured by the engine mapping process. Accurate accounting of
the benefits of advanced accessories is extremely relevant to the implementation of future off-cycle
credits for GHG.
3. Use of good engineering judgment to ensure robust and expeditious program execution.
In our opinion, the best measure of engineering judgment and proper program execution is obtaining
good agreement between ALPHA simulations and actual testing data. The documentation reviewed
suggests that the errors over the FTP and highway drive cycles are often within 3%, which are within the
+/-3%, test-to-test variability of chassis dynamometer testing.
4. Clarity, completeness and accuracy of the output/results.
The report is very thorough, including a detailed energy audit and 50+ figures, which is commendable.
However, we recommend that a smaller "summary" report, only with the very key parameters (fuel
consumption, engine cycle efficiency, speed-trace following metrics) be produced for easy tracking of
multiple runs.
The input and output structure of ALPHA was not finalized when released for peer review, however the
current version of the output structure were provided to give the reviewer a flavor of the potential
structure. The inclusion of performance metrics is highly commended, although we suggest spelling out
some metrics in the output file to facilitate troubleshooting and give the user a better perspective.
5. Any recommendations for specific improvements to the functioning or the quality of the outputs of
the model.
It would be useful for the user have access to a list of all the relevant vehicle parameters that can be
modified and the corresponding file where those parameters are input. Some parameters are
intuitive, but some others may fall out of sight unless there is a set list. Moreover, EPA has invested a
great deal of time and resources making sure the physical models are representing the physical
ICF International
E-6
October 17, 2016

-------
Peer Review of ALPHA Full Vehicle Simulation Model
elements, and a parameters list would help produce a better representation of the system being
modeled.
The way the model handles vehicle weight, tire rolling resistance, and Cd inputs may be improved. As
presented in the model, when changing vehicle mass, the user must change the vehicle mass and
separately change the rolling resistance by changing the A coefficient on the road load equation. In
the same way, changes in RR values have to be performed in the model by changing the A coefficient.
It would be better suited and less prone to errors to input these parameters separately and program
the ALPHA code to do the changes automatically. It would also be desirable to provide flexibility to
input either m, A, B, and C; or m, Cd, and RRc.
We were not able to find any information on how the model handles component weight changes.
Specifically, how does ALPHA handle changes in mass due to components with different mass? For
example, if an AT6 is being replaced with a DCT8 and that there are mass changes associated with
that. Does the delta in transmission mass change affect the vehicle overall weight or road load
parameter A? The model needs to at least inform the user that it the user's responsibility to assess
and incorporate any impacts of changes in components on vehicle mass.
Extra fuel usage for other vehicle operating requirements. The introduction of overhead functions
to compensate the steady-state based model for fuel used during transient conditions is very useful.
Our recommendation is to make those key adjustments available to the user on a clearer way; such
as a list of functions and parameters, or maybe developing a parameter list that deals with such
items.
ICF International
E-7
October 17, 2016

-------