Light-Duty Technology Cost Analysis
Pilot Study
Peer Review Report -
Response to Comments Document
United States
Environmental Protection
Agency
-------
Light-Duty Technology Cost Analysis
Pilot Study
Peer Review Report -
Response to Comments Document
Assessment and Standards Division
Office of Transportation and Air Quality
U.S. Environmental Protection Agency
v>EPA
NOTICE
This technical report does not necessarily represent final EPA decisions or
positions. It is intended to present technical analysis of issues using data
that are currently available. The purpose in the release of such reports is to
facilitate the exchange of technical information and to inform the public of
technical developments.
United States EPA-420-R-09-021
Environmental Protection _. , „„_
Agency December 2009
-------
Overview
This document provides responses to peer review comments on the FEV report, Light-
Duty Technology Cost Analysis (EPA 420-R-09-020). The peer review for the FEV
report was conducted by ICF International, and the documentation of their peer review
process and analysis of its findings have been published in a document titled, Light-Duty
Technology Cost Analysis Pilot Study - Peer Review Report.
The original text of the ICF report is used in this document for the purpose of directly
addressing each comment topic or comment theme with a response from EPA and FEV.
Thus, the EPA/FEV responses are incorporated into the body of the ICF comments
summary, and are identified by blue text and a response number (e.g. "Response #:...)
In the interest of clarity and context for the reader, the Introduction, Background of Peer
Review, and Organization of Peer Review Comments sections of the ICF report are
included in this document, but the large Appendices containing peer reviewer
qualifications and the original review submittals are not.
-------
Introduction
As the Environmental Protection Agency's (EPA) Office of Transportation and Air Quality
develops its programs to control greenhouse gas (GHG) emissions from light-duty highway
vehicles, there is a need to evaluate the costs of technologies likely to be used to meet these
standards. EPA contracted with FEV, Inc., to perform this cost analysis through tearing down
vehicles, engines, and components, both with and without these technologies, and evaluating,
part by part, the observed differences in size, weight, materials, machining steps, and other
cost-affecting parameters. Though complex and time-consuming, EPA believes this approach
has great potential for determining accurate technology costs, a goal that is of paramount
importance in the setting of appropriate GHG standards. EPA tasked ICF International (ICF), an
independent third-party contractor, to conduct a peer review of the cost analysis now in
progress to assure that this work incorporates the highest quality science.
Background of Peer Review
From August to October 2009, EPA arranged for ICF to conduct a peer review of the costing
methodologies used by FEV, Inc., and its subcontractor, as detailed in the draft report, "Light-
Duty Technology Cost Analysis Pilot Study" (FEV-07-069-103F, herein referred to as 'the
Report'). ICF coordinated the peer review, adhering to EPA's "Peer Review Handbook" (3rd
Edition).1
EPA's work assignment requesting the peer review required that peer reviewers be subject
matter experts who have a general familiarity with manufacturing cost estimating, automotive
design, or some knowledge of both.
Using these criteria, ICF developed a list of qualified candidates from the public, private, and
academic sectors. ICF compiled candidates from the following sources: (1) ICF experts in this
field with knowledge of relevant professional society membership, industry, academia, and other
organizations, and (2) suggestions from EPA.
ICF identified 19 qualified individuals as candidates to participate in the peer review. ICF sent
each of these individuals an introductory screening email to describe the needs of the peer
review and to gauge the candidate's interest and availability. Also, ICF attached to the email a
detailed conflict of interest (COI) disclosure. ICF asked all candidates to read the disclosure and
disclose any real or perceived COI or other matters that would create the appearance of a lack
of impartiality. ICF also asked candidates to provide an updated resume or curriculum vitae
(CV). Several candidate reviewers were unable to participate in the peer review due to previous
commitments, and several others did not respond. ICF reviewed the responses and COI
statements and evaluated the resumes/CVs of the interested and available individuals for
relevant experience and demonstrated expertise in the above areas, as demonstrated by
educational degrees attained, research and work experience, publications, awards, and
participation in relevant professional societies.
ICF reviewed the interested, available, and qualified candidates with the following concerns in
mind. As stated in the EPA's Peer Review Handbook, the group of selected peer reviewers
should be "sufficiently broad and diverse to fairly represent the relevant scientific and technical
perspectives and fields of knowledge; they should represent a balanced range of technically
legitimate points of view." As such, ICF selected peer reviewers to provide a complimentary
1 EPA/100/B-06/002
-------
balance of expertise covering the relevant economic and technical fields, including experts from
industry, not-for-profit, and academic backgrounds. ICF submitted a list of the proposed peer
reviewers to EPA, which EPA subsequently reviewed and approved.
ICF selected the following four individuals who agreed to participate in the peer review:
1. Dr. Dennis Assanis, University of Michigan
2. Mr. Sujit Das, Oakridge National Laboratory (UT-Battelle)
3. Ms. Laurie Harbour, Harbour Results, Inc.2
4. Mr. Wallace Wade, Ford Motor Company (retired)
In addition to the general COI screen mentioned above, prior to distribution of the peer review
materials, ICF asked the four selected peer reviewers to complete and sign a COI disclosure
form that addressed topics such as employment, investment interests and assets, property
interests, research funding, and various other relevant issues. Upon review of each form, ICF
determined that no peer reviewer had direct and substantial COI issues or appearance of a lack
of impartiality. In addition, the peer reviewers were instructed to disclose to ICF any potential
COI issues that may have arisen during their review, and no peer reviewer made such a
disclosure.
ICF provided reviewers with the following materials:
• The draft report by FEV, Inc., entitled, "Light-Duty Technology Cost Analysis Pilot
Study," dated September 3, 2009;
• Supporting spreadsheets detailing tear-down cost analyses; and
• A Peer Reviewer Charge to guide their evaluation.
Specific "charge" questions were not included in the Peer Reviewer Charge. Instead, EPA
provided peer reviewers with general guidelines for preparing their overall review, with particular
emphasis on the costing methodology and sources of information used in determining labor
rates, material prices, manufacturing burdens, and other key factors. In addition, EPA asked
each reviewer to distinguish between recommendations for clearly defined improvements that
can be readily made, based on data or literature reasonably available to EPA, and
improvements that are more exploratory or dependent, which would be based on information
not readily available to EPA.
The charge to peer reviewers can be found in Appendix A. The original, complete comments as
submitted by the peer reviewers are listed in Appendices B through E. The curricula vitae can
be found in Appendix F. A list of acronyms is provided in Appendix G.3
2 In responding to the peer review charge, Ms. Harbour consulted with two colleagues at Harbour Results, Inc.
(Harbour). As such, Ms. Harbour's comments in her submitted review (Appendix D) were written as coming from her
team at Harbour. However, in this summary, we use 'Harbour' when referring to the comprehensive set of comments
submitted by Ms. Harbour.
3 Acronyms are defined at first use throughout this report, including in the reviewers' raw comments. No other
editorial changes to the reviewers' comments have been made.
2
-------
Overview of Peer Reviewer Responses
I. Organization of Reviewer Comments
The reviews varied widely in size, scope, and content. This summary of comments reorganizes
the individual comments from all reviewers by theme and establishes common threads of
reasoning. ICF organized the comments from the four peer reviewers into two main themes and
sections of this report:
• Comments regarding methodologies and validation
• Comments regarding editorial content
Within each theme, relevant comments are presented, summarized, and attributed to their
author. Subcategories are identified where appropriate from each major comment theme. For
example, the methodological comments are broken down into comments generally regarding
either study scope, inputs, processes, or assumptions (pre-processing and processing
parameters) versus those regarding outputs or validation (post-processing parameters). Within
each subcategory, any clear consensus reached by the various reviewers is emphasized,
followed by any dissenting or unique comments.
Some comments addressed multiple items. For example, many comments referred to unclear
statements, which could be due to a flaw in the methodology and/or insufficient explanation. In
these cases, the comment is placed where it seems most appropriate. In all cases, comments
are attributed to the author by the name listed in bold in the numerated list above (e.g.,
Assanis).
Each reviewer organized and provided his or her comments in the following ways:
1. Assanis presented comments in a five-page text file. His comments are organized into
the following three sections: summary, organizational, and detailed technical comments.
2. Das presented his comments in a four-page text file and as a numerated list,
unorganized by theme or report section.
3. Harbour presented her comments as both an eleven-page summary text file and 94
additional comments within a spreadsheet. The spreadsheet generally contains
comments directed to a specific element of the Report and cited by page or section
number. The summary text file contains more general comments. In both cases, the
comments are grouped into one of the following seven "classifications": Technology,
Premise, Methodology, Protocol, Operational Definition, Statistical Validity, or
Presentation. A General Feedback section is also presented.
4. Wade presented detailed comments in the following three sections: an overall review,
specific opinions and recommendations, and discussion. The latter two sections focused
on eight primary opinions and recommendations for the Report. His eight primary
opinions were summarized in the second section, then repeated and expanded upon
with editorial comments in the third section. (Before summarizing, ICF merged the latter
two sections.)
In all cases, ICF mapped each individual reviewer comment to one of the two main themes. In
cases where the comment could apply to either theme, ICF made a judgment on which topic it
most closely represented and included it under that theme.
-------
II. Methodological Comments
a. Comments Regarding Study Pre-Processing and Processing Concepts
Comments Regarding Study Scope
All reviewers noted that the study was well focused and praised the use of case studies.
However, all reviewers also noted specific issues with the present case study, the potential
scaling methodology, and/or future case studies. Assanis, Das, and Harbour were specific about
this. Their comments follow.
Harbour noted that the study becomes complex so quickly that the user could lose sight of the
bigger issues and challenges posed by such a technology shift, although she appreciated the
use of the methodology presented to improve the accuracy and reliability of its economic impact
studies. Specifically, she noted that her clients have been historically at odds with EPA
forecasted impacts and costs and are hopeful that this study will alleviate some of these
concerns. However, she noted a significant difference between historical EPA forecasts and
industry's assessed costs. Harbour also expressed concern that the scope of the study will
dictate the level of assumptions and data required. She noted that the big questions, such as
whether the scope regards only replacing 2.4L with 1.6L engines or extrapolating these results
to other future engine technologies, will impact the study's scope.
Response 1: EPA agrees that it is difficult to forecast the future impact and cost of complex
technologies, but we are intentionally limiting the scope of the case studies to those
technologies we believe are candidates for mass-production, and so conducive to this type of
analysis. Because the study methodology uses a "ground up" approach, we do not intend to
extrapolate results to other technologies unconnected with ground-up, teardown-based work.
However, where costed components or subsystems are common to multiple technologies, we
believe it is fully appropriate to use cost data generated for these in one case to inform the
additional case studies (and in some cases, these components/subsystems are 'scaled' to
match the appropriate configuration needed for the case study). For the most part, scaling the
cost results to larger or smaller vehicle classes is being done separately by EPA using
methodologies documented in rulemaking analyses, and so is beyond the scope of the study.
The FEV report has been revised to clarify this. In some cases this technology scaling across
vehicle classes involves step changes that are less easily scaled. For example, scaling the 2.4L
to 1.6L downsized engine results to larger or smaller 14 engines can be fairly straightforward,
but scaling these results to the case where a V6 engine is downsized to an 14 engine involves
components not costed in the original teardown. In such cases, additional teardown and
component costing is warranted.
Assanis expressed specific concerns regarding the future technologies slated for study. He
noted that, of the powertrain technologies identified for evaluation, some are advanced versions
of current powertrains, such as downsized turbocharged gasoline direct injection (GDI),
advanced diesel, and advanced transmissions, while others, such as homogeneous charge
compression ignition (HCCI) and lean-burn GDI—while still predominantly mechanical
powertrains—will involve new challenges in electronics, calibration, and catalytic after treatment
that may not be addressed by the study in its current state. He also notes that
flexible/alternative fuel powertrain packages are omitted from the current scoping list. Finally,
regarding the future technology groups, he noted that the proposed technologies are quantized
into discrete bins, when in fact production hybrid versions are likely to be available soon, such
as GDI engines with operation split between the lean stratified and homogenous stoichiometric
-------
modes varying by required power. He noted that the current scope may thus need to be
modified to treat technology combinations, such as GDI/HCCI hybrids, and that the time
horizon, manufacturing, research and development (R&D) and calibration costs for some
technologies (such as for camless versus cam profile switching/ phasing and mechanical lift
control) will depend on their implementation.
Das stated that, while the bottom-up approach with detailed cost estimation based on teardowns
of vehicles implemented in the study is an improvement over past analyses, based on supplier
price quotes for key components, the scope may have limited success for new technologies. He
also noted that the scope is prohibitively expensive and time-consuming for regular use, unless
an appropriate scaling methodology is developed.
Response 2: The choice of technologies to be costed is not within the scope of the study, but
these comments are valuable as they concern how well the study methodology can be applied
to more advanced technologies with complex vehicle integration challenges and components
that may not yet be in mass-production. We agree that these technologies may well be more
challenging than the pilot study technology for this teardown-based costing approach. EPA
expects that the proper application of Indirect Cost Multipliers (ICMs) will address a portion of
these challenges, especially with respect to development and integration costs. The magnitude
of the ICM applied to the direct manufacturing cost is a function of the level of complexity for a
given technology; low, medium, and high (see Automobile Industry Retail Price Equivalent and
Indirect Cost Multipliers, EPA-420-R-09-003, February 2009).
Complex technologies which are new to the market, such as HCCI, will likely use a 'high
complexity' ICM to account for the cost of developing the necessary electronic controls. In
response to a number of peer review comments related to treatment of costs beyond the direct
manufacturing costs, we have added a section to the report describing the ICMs. We do not
believe that determining direct manufacturing costs for typical electronic (or aftertreatment)
components is inherently more difficult than for mechanical components, and study team
members have considerable experience with such costing. However, we do agree that costing
state-of-the-art components not yet in production may be a challenge in some upcoming case
studies and expect to justify and document any special procedures we develop for them as
need arises. See Response #1 for a discussion of scaling.
Comments Regarding Study Assumptions
Das, Harbour, and Wade all expressed concerns regarding production volume assumptions.
Specifically, Harbour challenged the use of 450,000 single unit volume, as opposed to five units
of annual volume 80,000 to 90,000, as the basis for estimations. She expressed concern that
this approach may significantly understate the total amount of amortized fixed costs in supplier
pricing. Further elaboration on the basis of her breakdown is given in her full comments in
Appendix D. Wade noted that the selection of vehicles that represent the base and new
technology cases for the teardown and cost analysis could have different production volumes
and that higher volumes for a new technology may result in different design and manufacturing
techniques, and thus impact costs. The assumed 20,000 units per year production volumes of
the new technology engine versus 840,000 for the comparator baseline could affect costs in the
present analysis and even more for future analyses, such as lithium-ion batteries. Das stated
that 450,000 units assumed for a robust market penetration rate may not be the same across
various technology types if numerous suppliers are involved simultaneously because the
present cost estimation is based on an individual tier-level supplier. Rather, varying annual
production volume should be included.
-------
Das, Harbour, and Wade expressed concerns regarding specific cost assumptions. Harbour
noted that, while a detailed "bottom-up" cost analysis is an essential part of the estimate of the
impact of technology changes on the marketplace, the Report does not adequately identify
economic and marketplace assumptions. Wade commented that different levels of maturity may
be represented in the new technology versus the base comparator technology, which could bias
the cost estimates. He also noted that no added vehicle costs were considered for the pilot
study new technologies, although new technology is likely to require new or different vehicle
installation hardware and the associated costs and should be included in the overall costs.
Finally, he stated that intellectual property, and the costs associated with licensing, patents,
know-how, and trade secrets, which is often an important part of new technology powertrains,
seems to be assumed to not affect costs and/or was not addressed in the methodology.
Similarly, Harbour noted that the Report needs to be explicit in assuming where current original
equipment manufacturers (OEM) and supplier manufacturing equipment and facility capabilities
are insufficient because, should demand for certain technologies exceed supply, OEMs will be
forced to pay a premium for these parts. She considered assuming sales below capacity for
modeling incremental costs to be specious. Das noted that, while consideration of maintenance
and end-of-life costs are given, the assumptions and approach regarding these costs are
insufficiently detailed and appear subjective. Finally, Das questioned the assumed 0.3%-0.7%
total manufacturing cost (TMC) for end-item scrap markup.
Response 3: Wth each case study, the team presumes that both the base and new
technologies are 'mature' - in that they are designed and developed to be produced at a level of
450,000 units per year. The report has been revised to better document the source of the
450,000 unit figure. This 'high-volume' assumption was chosen to reflect mass-market adoption
of the technologies. The study is not attempting to forecast costs for individual manufacturers
and models with limited sales. It should be noted too that many of the supplier-made
components costed are used in a number of vehicles and engines so that further economies of
scale may be available, though again we would expect these to be minor at such high relative
volumes.
We agree that the selection of vehicles and systems for teardown can have a bearing on the
results, whereby a selected model in lower volume production may have higher piece-part costs
than one that is more mature or produced in higher volumes. We believe that this effect is
adequately dealt with in the study methodology, especially in the steps that employ Munro
LeanDesign and material cost reduction analyses. Thus, if a part associated with a new
technology is currently manufactured using a 'low-volume' process, a high-volume process (with
similar functional performance) is substituted. We note too that, to the extent this dynamic is not
completely dealt with, it would tend toward overestimating technology costs, as it is more likely
that new technology teardowns will involve less mature teardown candidates than the base ("old
technology") vehicles or systems. Cost analysis in a low-volume scenario would be complicated
by the fact that, typically, the base ("old") technology teardowns are of mature products. So that
these would have to somehow be "unlearned" to a less-mature state of design and
manufacturing to make the right comparisons.
Vehicle installation costs for new technology are considered in the study through the
identification of assembly processes and related equipment. This would include any new or
unusual steps or hardware. For this case study, we considered, but did not identify, any vehicle-
side components or costs outside of the powertrain due to the addition of turbocharging and
direct injection technologies. For technologies where there will likely be additional costs due to
licensing fees or the use of intellectual property, those costs will be researched and included.
-------
Because the pilot case study did not identify any new maintenance or end-of-life costs, this
section of the report is not very detailed. Because such costs are highly technology-specific,
their derivation will be described in future case studies as needed. The 0.3%-0.7% end-item
scrap rate is an assumption that provides a default starting point, and is adjusted as the team
identifies generic processes requiring different rates (e.g., 5% for sand casting).
Assanis noted that, given the recent turmoil in the North American automotive industry (both
OEMs and suppliers), the assumed structure of labor wages and rates, based on historical rates
(e.g., fringe rates based on a 2006/2007 average United Auto Workers (UAW) OEM wage and
labor rate) may no longer be appropriate. Also, concerning the assumed labor mix and
associated costs, Wade stated that the decisions to use either OEM or suppliers for
manufacturing specific components is not clearly explained but appears to be based on
conventional practice and/or input from experts. However, since OEM labor rates are
significantly higher than supplier wage rates, these assumptions can significantly affect overall
costs and should be clearly explained.
Response 4: The unprecedented industry turmoil in recent years makes it difficult to adequately
forecast labor rates. For the pilot case study we have chosen to deal with this via a sensitivity
analysis included in the report. It finds that a 20% drop in average labor rates would cause a 3%
drop in the pilot case study final result. We may update the labor database for subsequent case
studies to reflect clear labor trends. Regarding the choice of OEM or supplier for component
manufacturing, for the most part we have chosen to assume multi-tiered suppliers remain a key
part of the cost structure for high volume products. We note that the gap between supplier and
OEM labor and fringe rates appears to be closing over time, making this less of an issue in
costing forecasts. We note too that we have conservatively assumed that automotive
components are manufactured in North America, even where we know that they have been or
can be obtained offshore at lower cost.
Wade and Harbour questioned the assumed technology and part supply chain. Wade noted that
OEMs and suppliers are assumed to have the manufacturing facilities and equipment for the
new technologies, but components comprising the core of the new technology will probably be
incremental and require new manufacturing facilities and equipment. While the Report assumes
that OEMs and suppliers have manufacturing equipment and facilities capable of handling
required manufacturing processes and capacities, this assumption may not be valid in most
cases. He stated that while all manufacturing processes and operations are assumed to be
based on standard/mainstream industrial processes, new technology is likely to involve unique,
new, non-standard industrial processes, especially in the battery area, which do not seem to be
accounted for in the Report. He further noted that different internal design and manufacturing
practices may be represented in new technology if from different manufacturers, and thus rely
on different internal practices that are not captured in the present assumed methodology.
Harbour commented that, contrary to present assumptions, use of off-shore suppliers may not
by default reduce costs due to increases in freight costs, tariffs, unmeasured internal
supply/program management resources, and exchange rate changes. Wade noted that
warranty data on new and baseline technologies should have been examined to validate
assumptions on design methodologies and ensure comparable useful lives because high
warranty costs for the new technology would indicate inadequate designs, materials, or
processes. Finally, Harbour questioned the selection of projection years (2015 and 2020) and
whether assumed technologies will be leading edge then.
-------
Response 5: With a few possible exceptions such as advanced batteries (to be dealt with in
subsequent case studies), we believe that the facilities needed to build the costed components
and systems already exist, and that this is certainly so for the pilot case study. Any unique
manufacturing equipment necessary for production of a technology is accounted for in the
burden rate. Considering the high volumes we are assuming, we expect that any components
supplied from offshore at increased costs to the OEMs would be re-sourced to domestic
suppliers. The idea of examining warranty claims on candidate vehicles as a check on the
robustness of their design is an intriguing one, and we will consider it for subsequent case
studies to avoid tearing down and costing poorly-designed and poorly-built products. However,
we believe that the team's general knowledge of automotive products provides a similar
safeguard. The warranty costs of new technologies themselves are covered in the overall
costing methodology through the application of ICMs, outside the scope of this study. The years
2015 and 2020 were selected by EPA as target years consistent with its analysis needs to
support rulemaking and other work. It is recognized that emerging and shifting technology
trends may make some of these cost forecasts less valuable in time. This is an inherent risk in
performing any analysis for future years and does not negate the need to have the best
available results for policy decisions.
Comments Regarding Study Inputs
All reviewers provided specific comments addressing various study input parameters.
Regarding the Annual Adjustment Factor (AAF), Wade stated that a serious discrepancy
between Figure 6 and Appendix E must be resolved before the Report is published. Although
Figure 6 shows an AAF of -5% from 2008 to 2015, Appendix E lists an AAF of 0%, which will
affect results. Harbour also noted this parameter and claims that a 5% per year decline in
supplier labor rates is inappropriate given inflation, generally increasing labor rates in the
manufacturing sector, and the distinction between declining labor rates and declining labor
content due to process improvements. Harbour further noted that Design Profit® might assume
that the part will be manufactured in the leanest possible way in year one, which is inconsistent
with the assumed 5% reduction.
Response 6: In the report, an exaggerated AAF of "-5%" was included only to illustrate the
effect of a gross year-to-year decline in labor rates from 2008 through to 2015. The actual AAF
used in the labor rate database - which is then used in the calculation of the direct
manufacturing cost-was "0%".
Harbour commented that the manual input cost parameters are sound and the study does an
exceptional job accounting for part variability. She also said the logic applied to material
identification is sound, but added that consideration should be given to whether a mistaken
material selection would be financially material. She also recommended that the study add
classifications in Manufacturing Assumption and Quote Summary (MAQS) to manufacturing
operations as either lean or mass, and be prepared to sort results accordingly to explore
resulting cost differences, given that the industry could experience higher batch processing
costs than those estimated by the model.
Das and Harbour commented on the inputs accounting for markup. Harbour noted that
engineering, design and testing (ED&T), scrap, and selling, general, and administrative (SG&A)
mark-ups all appear arbitrary and should be justified and validated through a sensitivity analysis.
Das stated that it may be inaccurate to apply the same level of mark-up regardless of the
primary manufacturing process and equipment groups involved. Variations in company size,
part complexity, and manufacturing technology type and their impact on final mark-up rate
8
-------
should be included in the calculations. Finally, Das noted that, if the cost methodology is based
on Vyas et al. (2000), then the end-item scrap mark-up includes mainly the warranty cost, but is
used in this study as if it also includes in-process scrap. He stated that this should be grouped
with manufacturing overhead rate parameters to better reflect technology characteristics.
Das, Harbour, and Wade all commented on wage and salary inputs. Harbour noted that the
active rates (Figure 12) and rates (Figure 13) appear arbitrary and not derived. Das questioned
why the OEM fringe rate is more than three times higher than for tiered supplier manufacturing,
while Wade noted a discrepancy in the projected labor rates used for 2015 that should be
corrected.
Response 7: All production methods/processes for this study are assumed to be "mass" and
"lean" (batch processing costs are not considered within the scope of this study). Concerning
supplier-level ED&T and other mark-up rates, the study did attempt to delineate between the
size and complexity of the component as well as the size of the company. These mark-up rates
are based on the industry experience of FEV's cross-functional teams, and the various rates
were scaled to capture the upper and lower bounds of the team's mark-up estimates (e.g. a
small size, low complexity, Tier 2/3 supplier has the lower mark-up rates, whereas the large
size, high complexity, Tier 2/3 supplier has the higher mark-up rates). In response to comments
concerning the justification for these mark-up rates, Table 7-1 ("Tierl Published SG&A and
ED&T/R&D Rates for 2007 and 2008") has been added to Section 7.5.2 in the final report.
Comments Regarding Study Methodology
All reviewers commented favorably on the study's general methodology. Wade, in particular,
noted that FEV has developed and applied a detailed, objective methodology for determining
net incremental/assembly cost impact to the OEM for new technology powertrains with reduced
GHG potential. He said this methodology will both provide EPA with the capability to estimate
the cost of technology to meet regulations as well as act as a medium for EPA to interact with
OEMs and suppliers to review, compare, and rationalize their costs. However, he noted that
while the methodology ideally should yield accurate results, he found potential issues with the
methodology that may affect the accuracy of the results. In particular, he noted that during the
conference call it was indicated that three specific, important steps described in the
methodology (a marketplace crosscheck, accounting for design modifications for Material Cost
Reductions (MCR), and accounting for New Technology Advances (NTA)) were not being
followed, and that these should either be followed or removed from the Report. Each step is
detailed in Appendix E, Opinion #7 of this summary report. Das noted that the methodology
presented, using a standard quoting template used by the automotive industry for the MAQS, is
an excellent approach and deserving of credit. Harbour stated that the overwhelming majority of
methodologies employed in this study appear to be sound and, in particular, she approved of
the use of activity-based burden rates, machine-based burden rates, and AAF. She also
generally approved of the differential analysis logic, but questioned the statistical basis for
assuming differences in means between baseline and new technologies. She recommended
that the methodology be adjusted such that any proposed technology changes are classified
into one of the following four distinct categories of R&D investment and that a total cost versus
incremental cost model for these changes be developed: new product development, fuel
economy improvement and emissions, pure research, and plant energy and emissions.
Response 8: The report has been modified to better reflect the actual process applied to the
case study. It should be noted that in some cases the pilot case study did not identify a need to
conduct processes described in the report, but later case studies likely will. This is true for
-------
example for the material cost reduction (MCR) and new technology advances (NTA) processes,
as discussed in Response #15. See Response #21 for discussion of statistical analysis. The
R&D investment categories commented on are included in the ICMs; see Response #2.
Assanis noted that the topic is both an important one and a difficult one for arriving at a
universal methodology, but that it should be recognized that the proposed methodology focuses
more heavily on tracking component materials and manufacturing costs and less on system
integration issues and costs, which are typically captured through indirect cost multipliers. He
felt that for new powertrain technologies that are evolutionary versions of current technologies,
the methodology has the potential to produce fairly accurate estimates of final costs, but as
components and systems become different from current versions, both the manufacturing and
systems integration aspects of this methodology will be challenged.
Response 9: Again, we will rely on the ICMs to account for system integration indirect costs.
We believe that tying the magnitude of the ICM to the complexity of the technology - as well as
the timeframe that it will be implemented - is a valid approach for capturing system integration
costs. The upcoming teardowns for these more advanced technologies will provide valuable
information on the complexity of the systems involved and the appropriate ICMs to apply to
them.
Harbour and Wade questioned the methodology dealing with unit volume. Harbour questioned
whether the vehicle class summary was used to identify target vehicle platforms that would
likely adopt the new engine technology and set expected volumes. She found 450,000 units to
be unrealistically high. Similarly, Wade commented that the analysis assumes annual volume of
450,000 units, the basis of which should be provided, and which may be inconsistent with the
recent severe fluctuations in the automotive market. He requested that volume-sensitive costs
be provided and the methodology be made flexible to the impact of severe fluctuations in the
automotive market, which could possibly affect the accuracy of the cost estimates.
Response 10: See Response #3.
Wade, Harbour, and Das provided other specific comments regarding the process used. Wade
noted that production testing (e.g., on fuel rails) and end-of-line testing may be required for
some components or systems but is not addressed in the methodology. Harbour stated that the
process mapping is sound but manufacturing practices that are less than optimal should be
addressed by clearly distinguishing between batch and lean and possibly a sensitivity analysis
showing a mix of practices that are more "batch" oriented. Das noted that the linkage between
primary fabrication processes and secondary cost estimating modules is unclear. Finally, Wade
commented that the visual teardown inspection used cannot capture many detailed
specifications used in the manufacture of the part, such as tolerances, surface finishes, or
flatness requirements that could add additional cost to the parts. However, these processes do
not appear to be included in Design Profit.
Response 11: The need for additional end-of-line (EOL) testing is considered in the
methodology, and a note has been added in Section 9.2 of the final report to highlight this fact.
For example, costing of the turbocharger included the equipment and labor needed to perform
these tests. Wherever new or unique EOL tests are identified, they will be included in the
costing methodology. However, in this particular case study, we did not identify any unique EOL
tests that would be required at the engine or vehicle assembly plants. Batch processes are
assumed in the analysis where they are considered appropriate in a high-volume scenario. The
report has been rewritten to clarify the linkage between primary and secondary processes. The
10
-------
visual teardown inspection process is conducted in a careful meticulous fashion to ensure that
cost-affecting manufacturing steps are observed and recorded.
Harbour and Das provided opposing views on the general use of software in this methodology,
although both expressed concerns. Das noted that the cost estimation approach becomes
"fuzzy" after the Lean Design® software converts serial processing operations into mass
production operations. This process is not at all transparent, at times an ad-hoc procedure is
used, and it requires subjective judgments along the way. He also expressed concern that the
material usage "Ib" variable is automatically uploaded from Process Mapping instead of a user
input. However, Harbour commented that she has already experienced Design Profit technology
in prior projects and is confident in its valid integration of product design, manufacturing
processes, and associated demand, cycle time, labor, material, costs, and forecast data.
However, she expressed concern that Design Profit uses a static treatment of data, whereas
other simulation software dynamically models manufacturing behavior and may provide a better
understanding of cost variances over time. She also noted that Design Profit is costing on ideal
designs and manufacturing flows that may make costs appear lower.
Response 12: A section has been added to the report to provide additional information in
response to these comments.
Wade, Das, and Harbour all noted potential inconsistencies in the costing methodology.
Harbour stated that the cost database integration and automatic updates to "active Rates" is
exceptional, and that the logic used for shipping costs, investment cost as part of manufacturing
overhead, product development costs, and tooling cost allocation are sound. However, she said
it is unclear how/if unique investment costs are included or referenced and how forecasting of
future prices are extrapolated from limited input. Wade commented that the Comparison Bill of
Materials (CBOM) chart includes a provision for accounting for design modifications for material
cost reduction (MCR), but noted that during the conference call it was indicated that this was not
done. Das expressed uncertainty about whether the Munro & Associates costing software also
is used for secondary processes, since mapping of these processes using Design Profit
software is indicated in the text but not in the costing procedure. Further, he noted that it is
unclear how tooling and investment assumptions are incorporated into the part cost estimation,
and how tooling is different and estimated as a part of the indirect cost (1C) multiplier of the
OEM and as manufacturing overhead. Finally, he noted that, while the inclusion of MCR and
NTA assumptions in the CBOM is an excellent option that should be used more frequently to
project future cost reductions with technology maturation, it is unclear how these have been
implemented into the overall cost estimation procedure.
Response 13: Supplier investment costs are indeed accounted for in the analysis; however,
tooling is assumed to be owned by the OEM and included in the ICMs. The report has been
modified to clarify this. We agree that forecasting future material prices is a difficult undertaking,
as evidenced by experience among forecasting experts over the years. Forecasting based
simply on historical trends is, in our view, a reasonable approach. See Response #15 for a
discussion of MCRs and NTAs. Munro & Associates software is used for secondary processes;
the report wording has been clarified.
All reviewers provided comments on the methodology used for markup costs. Assanis noted
that the calibration of multi-variable, high degree of freedom systems can require more
resources and time than their direct manufacturing costs, although the availability of historical
cost databases for relevant components of advanced technology vehicles is very limited.
Because of this, the related R&D and calibration issues would not be captured well by currently
11
-------
used 1C multipliers. This will become more significant with other technologies, such as electric
vehicles. All other reviewers found the need for more clear application of the 1C multiplier report.
Harbour noted that many of the premises set forth in the 1C multipliers report apply here and
should be incorporated. She also noted that active rates and mark-ups appear arbitrary and not
derived. However, she saw no major flaws in the 1C multiplier logic, the logic for ED&T mark-up,
end-item scrap mark-up, SG&A mark-up, Tier 2 and 3 supplier mark-ups, and profit mark-up.
However for the profit mark-up, she said it is unclear where proprietary or unknown materials
are used as well as the value of the impacted component in achieving more desirable
emissions. Das noted that 1C has not been applied consistently, particularly as the 1C relates to
a Tier 1 vendor. This is noted repeatedly where an OEM has been used as the supplier, but no
indirect cost multipliers (ICM) are incorporated. He found that the concept of 1C multiplier is
mentioned several times in the document, but no specific applications of the actual data are
found. More of his specific comments regarding this are included in Appendix C. Similarly,
Wade stated that the recent ICM study should be applied to capture the overall price for new
technology because the present methodology generally is designed to determine incremental,
direct manufacturing costs for new versus baseline technologies.
Response 14: The appropriateness of ICMs to characterize certain OEM costs is discussed in
Response #2. A new section has been added to the report to clarify the relationship between
the study results and ICMs. We believe that the indirect costs at the vendor level are best
captured as a fixed rate, as the high volume assumption of the approach should drive vendor
costs for ER&D to level typically found in today's manufacturing environment.
Wade and Harbour provided additional, specific comments on the methodologies used for
technology analysis. Wade stated that the CBOM chart shows potential accounting for NTA -
which could provide alternative, advanced technology ideas as a potential substitute for some
existing hardware being evaluated - but that on the conference call it was indicated that this
was not done. He further noted that differences between components contained in both the new
technology and comparator baseline but not directly related to the technology may need to be
analyzed to determine the proper handling of these differences. Wade commented further that
although the report states that "...application of new technologies in five vehicle size classes is
considered," no details are provided and that the methodology would require, for each of the five
vehicle size classes, a detailed analysis to account for significant change in the engine
configuration for larger engine sizes. He also concluded that the analysis of the port fuel
injection (PFI) powertrain control module (PCM) cost is insufficient (see detailed comments in
Appendix E). Harbour questioned how an engine with new or proprietary materials and physical
designs can have no change in maintenance or end of life costs. More significantly, Harbour
stated that industry may push back on the general assumption that a product technology is
mature, or is something that would have been invested in any case, especially if EPA
regulations mandate significant emission reductions over currently planned R&D efforts. She
noted that a survey of industry to validate the maturity of various technologies and estimate the
amount of R&D should be built into the suppliers' price.
Response 15: The New Technology Advances (NTA) and Material Cost Reduction (MCR)
sections are provided in the CBOM template to facilitate consideration of these factors. Potential
MCRs and NTAs were both identified and evaluated for the pilot case study, however neither
was found to be of great significance to the incremental cost. These fields may find more utility
in subsequent case studies with less developed technologies. We do not believe it appropriate
to rely too heavily on this aspect of the analysis because it moves the methodology away from
its foundation in physical teardowns and down more conjectural pathways. We believe their
primary use will be in situations for which the teardown processes or designs clearly evidence
12
-------
room for improvement. Although MCRs and NTAs can be identified for both base and new
technology costing, they are more likely to show up in the new technology work and so will
generally tend to overestimate costs to the extent they are not fully pursued. See Response #1
for a discussion of scaling across vehicle classes.
Regarding component materials and suppliers, Wade and Das found the methodology
somewhat lacking, while Harbour generally was pleased with the approach. Wade stated that
the method for determining the required material weights for the manufacturing processes,
which would be higher than the finished part weights found in the teardown process, was not
explained in adequate detail. In particular, material cost appears to be derived by multiplying the
component part weight by the material cost per weight, which ignores metal removal or casting
waste. He noted that further explanation of the methodology to determine material weights and
the overall size of the part in bar or rod form should be explained. Wade also commented that
the use of the cross-functional review team is not a rigorous method for determining material
used in a specific part from a teardown. Instead, a more reliable method, such as conducting an
analysis of the actual material from the part, should be used. Das found that subjective expert
judgment is necessary for the material selection process for the unmarked materials. Harbour
stated that sound logic is used for commodity-based purchased parts, primary equipment
groupings in Table 2, unknown proprietary material compositions, and in the use of marketing
firms to trend future prices as compared to similar material trends. However, she noted that
consideration should be given to why proprietary materials are typically used to offset other
lifecycle costs when proprietary materials typically increase start-up and validation costs. She
also commented that the material identification methodology is sound and demonstrates real
industry knowledge.
Response 16: Material waste is relevant manufacturing processes is indeed accounted for
costed, using typical industry figures for these processes. The report has been modified to
clarify this. The cross functional team has been able to identify material with a high level of
confidence for the great majority of parts with a substantial impact on overall cost. We agree
that more sophisticated materials analyses may be appropriate in future case studies if
potentially high-impact parts cannot be clearly identified.
Both Das and Harbour commented that the methodology treating packaging cost is adequate.
Harbour stated that sound logic was used for packaging selection and costs. Das noted that the
attention paid in the estimation of packaging cost is good, but possibly not worth the
consideration because its contribution to the net incremental unit part cost may not be as
significant as compared to other cost categories.
Response 17: We agree that the impact of packaging cost is small, but in the interest of having
a thorough analysis, we chose to include them.
Harbour, Das, and Wade all commented on the methodological treatment of wages. Harbour
found that there was insufficient information to determine how the analysis forecasts the union
versus non-union labor mix for future dates and recommended a sensitivity analysis on this
parameter. She found that the logic for calculating and applying indirect labor ratio is adequate,
as is the logic for calculating the labor rate, although it was unclear how the median wage rate is
used and how it impacts the Report findings. Das agreed that the labor rate estimation approach
is sound and based on a reliable public data source. However, Wade noted that using labor
rates based on the described labor mix may result in significant errors in labor costs from
applying a mixed labor rate to a fully unionized supplier or, conversely, a non-unionized
supplier. He said further justification is needed, especially following the severe restructuring in
13
-------
the automotive industry in 2008 and 2009. Wade further noted that the 2006/2007 fringe rate
burden data for OEMs may be obsolete as a result of the many recent changes in the
automotive industry and should be updated.
Response 18: The labor wage data, published by the Bureau of Labor Statistics, does not
differentiate between union and non-unionized labor rates within the primary North American
Industry Classification System. As such the mean average labor wages, for all standard
occupations referenced within the analysis, are calculated averages based on both union and
non-unionized labor. To understand the impact of labor rates on a new technology configuration,
the rates within the database can be adjusted, automatically updating throughout the various
cost analysis worksheets. We may update the database for later case studies with new
information gleaned from labor rate trends, but for the pilot case study we have chosen to rely
on a sensitivity analysis in the Report to show how results change with lower labor rates
For OEM labor rates, both base wage and fringe values were calculated using UAW published
data from 2006/2007. Effective 2007, new UAW hourly labor contract agreements were
established substantially reducing the entry level base wage and fringe contributions. Over the
long term the expectation is that as new laborers enter the workforce, under the new entry level
agreements, the average labor rate will begin to decline. In the near-term (i.e.2008/2009) these
reductions are estimated to have minimal impact as most North American OEMs are currently
downsizing their workforce to help counteract marketplace conditions.
Regarding other factors that may be considered along with the cited results, Harbour stated that
other design features, such as aerodynamics, rolling resistance, type of fuel, and weight also
work in combination with the engine technology to achieve emission levels, and should be
considered along with engine technology. She also found it unclear whether vehicle
classification is useful in determining the potential application of the 1.6L engine and its
competitive alternatives. Wade noted that the comparison of fuel economy for the new
technology and comparator baseline vehicle is flawed because the new technology is applied to
a significantly lighter vehicle. Additionally, he noted that the methodology does not appear to
account for the costs required to reduce emissions of the new technology to the level of the
baseline, including costs required to ensure comparable certification emission levels for the new
technology.
Response 19: Aerodynamic and other vehicle improvements which reduce engine load could
be handled in a separate case study. We feel however that mixing them into a cost analysis for
new engine or transmission technology would make it harder to separate cost figures for
technologies that do not have to be bundled to make them individually viable. Although fuel
consumption (as well as the corresponding GHG emissions reduction) is the primary focus of
the technologies selected to be costed, the cost analysis does not actually concern itself with
the change in fuel consumption. The cost of changes to emissions controls necessitated by
adoption of a new technology are indeed part of the cost analysis methodology, but were
assessed to be unaffected in the pilot case study.
b. Comments Regarding Study Outputs and Validation
Comments Regarding Study Outputs
Only Harbour provided comments regarding the grouping and analysis of study outputs. She
presented three primary recommendations. First, she recommended that various parts identified
in the CBOMs be classified by their known or projected impact on emissions so industrial and
14
-------
consumer costs can be attributed to performance capabilities, as expected for a value
engineering study. Second, she recommended that a list of torque variables be developed and
ranked by their financial impact and their impact on lowering emissions. These are parameters
that can greatly influence the financial materiality or discernable improvements in emission
levels, such as engineered items and union labor content. Third, she recommended that a list of
uncertainty factors be developed, that uncertainty analysis be used to validate other proxies of
prices, and that results be indexed to the price models.
Response 20: We are assuming equivalent emissions performance for the base and new
technologies, other than for CO2. As discussed in Response #19, we are not actually assessing
the CO2 reduction potential in this cost study. For case studies in which a unique emission
control component is required (e.g. urea-SCR, or a LNT), those components will be included in
the analysis. EPA typically addresses the emissions effectiveness of new technologies in the
Regulatory Impact Analysis. See Response #21 for a discussion of the challenge of conducting
uncertainty analysis and the alternative use of sensitivity analysis to inform this area of analysis.
Comments Regarding Study Validation
Wade and Assanis requested that minor validation steps be taken on the study's results, but
Harbour requested a more significant review and validation of the study. Assanis requested a
sensitivity assessment of the results to the assumption of an average volume of 450,000 units.
Wade noted that the methodology described subjecting a part with high or unexpected cost
results to a marketplace crosscheck for validation, but that this had not been performed. He
stated that this is a critical step for validation of the results, especially for the four high cost
items. Harbour noted that a statistical expert review needs to be added as a final step to
validate that the resultant costs are indeed statistically different for materials, labor, and
combined into direct manufacturing costs. She said, in all cases, data used for comparison
needs to be described by at least a mean and a standard deviation to establish if proposed
average costs of the new engine are within the price variations assumptions of the baseline
engine. An assessment also needs to be made of whether future prices will have the same
variation between the baseline and new engines. Harbour also requested a sensitivity analysis
on the different classifications discussed in Table 3 and to include changes in arbitrary
weighting factors. If the regression analysis in Figure 9 is of price averages, Harbour noted that
a statistical test of homoscedasticity is needed to validate the weightings. Harbour also called
for an uncertainty analysis for ill-defined or unknown materials, manufacturing, and procurement
practices that materially affect cost outcomes in addition to the sensitivity analysis. She said the
uncertainty analysis should validate other proxies of prices, such as raw material cost trends
and exchange rate fluctuations, then subsequently index them to the price models used. Finally,
Harbour stated that uncertainty factors should be aligned to torque variables and ranked to their
impact on lowering emissions.
Response 21: See Response #3 for a discussion of the basis for the high-volume assumption.
The team has discussed performing sensitivity analysis for production volumes in response to
the peer review comments and has concluded that this would be a significant undertaking that
would delay or preclude some further case studies; however, it may be a valuable follow up
study in the future. The report has been revised to more clearly describe the part-by-part
validation that does occur. This takes place whenever a preliminary cost for a part appears
unusually high or low. In such cases the team consults industry contacts, supplier pricing, or
written reference material, and rechecks the analysis for possible errors. Given's the team's
sizeable knowledge base in automotive design and manufacturing, this validation process,
although somewhat informal, is a robust one.
15
-------
Because automobile systems are complex, analyzing their cost from the ground up requires a
very large number of inputs. A properly done statistical/uncertainty analysis on these inputs
would be a very difficult task, even for a single case study. Instead we have chosen to perform
sensitivity analyses, varying key parameters such as labor rates, to provide some measure of
bounding for the cost figures. We agree that this does not establish absolute ranges for the cost
results, but feel it provides a sensible middle approach to establishing a degree of confidence in
the results.
III. Editorial Comments
a. Comments on Organization and Pagination
Generally, the reviewers considered the Report presentation to be adequate. Harbour stated
that the flow is logical.
Assanis noted that page numbering is confusing. Das noted that in Section 5.0, "material costs
include a combination of raw materials, material processing scrap, and commodity purchased
parts," but material cost is discussed in Section 6 and material processing scrap has been
included under "End-Item Scrap" based on the methodology described in the document - he
said this organization should be improved. Harbour noted that parts should be classified as
repairable or replaceable and this should be included in the organization, since this affects the
consumer's cost of ownership.
Response 22: The report page numbering has been revised. A complete evaluation of the
consumer cost of ownership is beyond the scope of this study. However, new required
maintenance resulting from the use of new technologies is included; see Response#3. This
would not extend to identification of repairable or replaceable parts unless they are part of the
new scheduled maintenance.
Wade and Assanis presented two specific organizational comments. Wade noted that to
evaluate the validity of the overall methodology, information needs to be added regarding the
software and databases with respect to 1) what the software does, 2) how embedded data in
the software or the databases were developed, and 3) how the software or databases have
been validated. Assanis stated that the Report relies on presenting the proposed methodology
in tandem with the case study, but references a large number of spreadsheets and databases in
appendices that obfuscates the presentation of the general methodology in the body of the
Report. Although he approved of including in the Report a comprehensive case study, he said
the presentation of specific examples is too lengthy while other sections, particularly those
where key methodological assumptions are presented, are too laconic. He stated the draft
generally needs to be reorganized and provided a suggested structure, which is presented in his
full comments (Appendix B).
Response 23: In Section 4 of the report ("Costing Methodology - Teardown, Process Mapping
and Costing"), significant updates were added to better explain the overall costing process.
Updates include additional information on how the software and spreadsheets work, their
function within the analysis, position within the costing process, and how they are validated.
Additional details on how Manufacturing Overhead rates are created and validated was also
updated in Section 7.4, Manufacturing Overhead Database, of the report.
16
-------
Many study assumption details for the pilot case study are referenced throughout the report to
help explain the methodology developed for the overall costing analysis process. An update was
made to Section 13 (Results for Case Study #0101) of the report to consolidate some of these
key case study details, in one location, supporting the quantitative results. We agree that
presenting details of the case study in the report body and in the Appendices adds a large
volume of case-specific information in a report that has as its primary goal the description of a
general methodology. We felt, however, that this approach helps the reader to understand a
methodology that might otherwise appear too abstract. Unfortunately, the shear complexity of
this type of costing does not lend itself to succinct, easy to lay out examples. We believe this
approach and the report structure, which treats this complex material by proceeding from high-
level to detailed-level discussion, provides the best way to present this material.
b. Comments on Grammar and Wording
Reviewers provided the following typographical edits:
• Pages 1-3: there is a typo in the Flowchart. "Cross Fucntion" should be "Cross Function."
(Assanis)
• Pages 4-5, 4th paragraph: "To accomplish this" needs to be followed by a comma.
(Assanis)
• Page 5.2, the first line: "...are involved..." has an extra period. (Assanis)
• Pages 6-7: the last line "Annual Adjustment Factors" should be corrected to "Annual
Adjustment Factor." (Wade)
• Page 31: the last line:"... in the cast study analysis." should be changed to "... in the cost
study analysis." (Wade)
• Page 35, Section 6.2.3: line 6: "...Metalprices, estainlesssteel..." should be changed to
"...metal prices, stainless steel..." (Wade)
• Page 36, Section 6.2.4, paragraph 2, line 1 and Figure 9: "Figure 9 illustrates the power
curve..." should be changed to "Figure 9 illustrates the price curve..." (Wade)
• Page 45, line 1: "...establish the reference the baseline..." should be changed to
"...establish the reference baseline..." (Wade)
• Page 52, line 2: "...same level of mark-up is applied regardless the..." should be
changed to "...same level of mark-up is applied regardless of the..." (Wade)
• Page 58, last paragraph, line 4:"... manufacturing costs are capture for the..." should be
changed to "...manufacturing costs are captured for the..." (Wade)
• Page 58, last paragraph, next to the last line: "Alternatively in single a MAQS..." should
be changed to "Alternatively in a single MAQS..." (Wade)
• Page 61, first paragraph under Section 9.3, line 4:"... interactions tacking place..." should
be changed to "...interactions taking place..." (Wade)
Reviewers provided the following suggestions to provide clarity and consistency within the
Report:
• Page 4.5, 3rd paragraph: "Most all of the processes" needs rewriting. (Assanis)
• Pages 4-5: the factoring methodology used to deal with part variability needs more
description. (Assanis)
• Page A-0, Section 3.1: The statement that no new technology was considered between
a naturally-aspirated 14 and a downsized turbocharged engine is confusing. (Assanis)
• Page 13: The convention adopted in this report of referring to a "quote" as the
"analytically-determined cost of a part or assembly, not a price provided by a supplier", is
17
-------
• The authors should better define and be prepared to defend the "judicious scaling"
protocol used. (Harbour)
• Modules, top level components, end-item components, parts, are introduced as new
terms in the system hierarchy, but could be better written to use generic terminology.
(Harbour)
• The Report uses mixed and confusing definitions of system/subsystem/high level
modules/modules/top level components/assemblies/components/end-item
components/piece parts/parts hierarchies. The author often uses multiple terms to
describe the same part. Generally, the hierarchy needs to be defined and consistently
used throughout analysis. (Harbour)
• The subsystem, sub-subsystem, assembly, subassembly, component are all well stated,
but the Report does not read this consistently. Also, it is unclear if levels change as data
is entered in Design Profit. (Harbour)
• The term "high level modules" is introduced as part of the Design Profit nomenclature.
(Harbour)
• While there is an excellent distinction between MCR and NTA, a better operational
definition is needed. (Harbour)
• The distinction between Lean Design and Design Profit needs to be clarified. (Harbour)
• The statement "annual consumption rate which impacts whether high or low volume
prices are used" is unclear regarding whether a nominal or mean rate is also used (or
just high or low?). (Harbour)
• A definition of financial materiality is needed. (Harbour)
• The authors should clarify what method is used for the investment cost depreciation over
the production volume in the estimation of manufacturing overhead rate. (Das)
• The authors should clarify whether there are known variances in the active rate and
if/how these are accounted for in determining either statistical differences and/or
financial materiality. (Harbour)
• While the work assignment focuses on the determination of incremental direct
manufacturing cost, a solid definition of direct manufacturing costs should be added.
(Das)
• The five vehicle classes considered should be mapped to the 13 EPA vehicle classes
{49 CFR part 523}. (Das)
• The variable OEM/T1 Classification in Appendix G should be uniformly added to the
subsystem worksheets. (Das)
• A glossary of terms should be provided. (Wade)
• Each of the appendices should be listed in the Table of Contents (Wade)
• An operational definition of high and low volumes; high impact purchased parts, low
impact purchased parts and commodity parts; standard/mainstream industrial practices;
and how parts were deemed to be excluded should be added. (Harbour)
• The authors should clarify whom or what validates NTA's "fitness for use." (Harbour)
• A representative formula should be given that describes the use of input factors,
following the first paragraph of Page 4-6. (Assanis)
• An explanation of "composition analysis plus surcharge" should be provided. (Wade)
• For ease of cross-referencing, the numbering used for the subsystems throughout the
analysis should be used on the summary chart showing the incremental costs. (Wade)
18
-------
• In numerous cases, instead of citing a published reference, recourse is made to a
consultation between EPA and FEV, or feedback from a subject matter expert without
presentation of ranges and sensitivities considered, and a discussion of where the
estimate selected for this study fell within the range. (Assanis)
• Clarification is needed on the protocol employed to determine if differences in emission
benefits and cost factors, for base versus new components, are worthy of industry
investment and consumer investment. (Harbour)
• While "Component level quotes" are logically classified as full quote, modification quote,
and differential quote, operational definitions should be added for each with clear
examples. (Harbour)
• The Report should be explicit in where current OEM and Supplier manufacturing
equipment and facility capabilities are insufficient. (Harbour)
Response 24: The report has been revised in response to these comments and a number of
definitions have been added, or clarified. A 'Glossary of Terms' section has been added to the
Appendix to assist the reader in understanding the study.
Most of the comments from Harbour have been addressed in previous responses. Concerning
Das' comment on expanding the 5 vehicle classes to cover all 13 EPA classes, per EPA's
direction, the study was limited to only five vehicle classes to limit the cost and complexity of the
study. EPA staff retains the responsibility to map the study results to the larger number of
classes analyzed for its rulemaking activity. This mapping is outside the scope of the study.
c. Recommended Content to be Added
In addition to the specific itemized changes listed above, reviewers made numerous comments
regarding the need for additional material in some sections.
Assanis noted that, in general, there should be more discussion of the results presented in
figures, tables, worksheets, and more critical assessment of results, rather than just
summarizing the results in a table or figure, and more discussion of uncertainties present in the
analysis along with specific findings. For example, in Section 5, page A-0, a section on 1C
multipliers is needed to summarize key findings of the companion EPA study, along with a
discussion of areas where improvement to that methodology needs to be made to deal with the
R&D and calibration costs of some of the new technologies (especially high complexity,
disruptive electrical technologies).
Response 25: Additional details were added throughout the Report in response to this
comment. Significant content was added to Section 4 (Process Methodology - Teardown,
Process Mapping and Costing), as previously discussed in Response #23. Section 6, Indirect
OEM Costs, was added to provide additional details on the Indirect Cost (1C) multiplier. Also
Section 10 was added, explaining the marketplace cross-checking processes employed as part
of the overall costing methodology. In addition to these major additions, several smaller
clarifications were added to many of the sections, providing additional details on tables, figures
and worksheets. These include changes in the following sections of the report: Executive
Summary, Material Database (Section 7.2), Manufacturing Overhead Database (Section 7.4),
Mark-up Database (Section 7.5) and Results for Case Study #0101 (Section 13).
Regarding use of databases, Das noted that annual consumption rates and source references
to the materials database are not listed in Figures 7A and 7B and that the material price needs
19
-------
to be referenced to a specific production volume with indications of the volume sensitivity factor.
Although the electronic database contained some of the information, he said the report text
should be clarified. Wade noted that the databases for material costs, labor rates,
manufacturing overhead rates, mark-up rates, and packaging costs provide the foundation of
the cost analysis, but were not adequately addressed in the Report and were not provided for
review with the original package. Furthermore, he noted that descriptions of how the proprietary
software and databases were validated, which would be required to determine the accuracy of
the cost estimates, are not included. Wade also state that while parts considered a "buy" and
"surrogate costs are pulled from an applicable database" there is inadequate description of this
"applicable database," including how it was developed and how it was validated. Further, Wade
noted that Appendix G states that a "Purchase Part Database" is used to provide values for
commodity purchased parts, but it is unclear if and how this database differs from the
"applicable database." Generally, he said the methodology makes extensive use of proprietary
software and databases, which are not available for peer review. Wthout detailed information
regarding these proprietary tools, especially Lean Design, Design Profit, "An applicable
database" for surrogate costs, and the "Purchase Part Database," the accuracy of the cost
estimates cannot be determined. The Report needs to clarify how these four software and
databases are related.
Response 26: Material prices in the study were based on a volume of 450,000 units, and
'sensitivity' to price fluctuations can be evaluated by adjusting the prices up/down in the
databases. The commodity purchase part prices come from a variety of sources as defined in
Section 7.2.6, Purchase Part - Commodity Parts, of the report. The commodity purchase part
costs are gathered from numerous sources and grouped together in several different
spreadsheets; collectively, this group of spreadsheets is referred to as "purchase part
databases". Unlike the primary costing databases (e.g. material, labor, burden, mark-up,
packaging) which are developed from publicly available data, the commodity purchase part
databases are generated from information which is classified as confidential business
information.
Reviewers also provided comments regarding Design Profit and Lean Design. Assanis noted
that its description is too superficial, even considering Figure 2. The key assumptions and
algorithms of the methodology need to be presented, and a discussion of the mapping process
needs to be included in Figure 2. Wade noted that more information is needed on the details of
Design Profit, especially how it determines the times required for each step of the process, how
the software results were validated, and to clarify how the Lean Design input to the MAQS
differs from the Design Profit input into the MAQS. Das stated that the manufacturing
assumptions section, positioned to the right of the quote summary section in MAQS, is where
the additional assumptions and calculations are made to convert the serial processing
operations from Lean Design into mass production operations, but inadequate information is
given regarding the underlying assumptions and calculations. Harbour stated that having a
consistent hierarchy and part naming between spreadsheets, Design Profit, and the Report
would add value. Assanis noted that several critical assumptions of the methodology are not
presented in the Report, but instead are left to reference the proprietary software Design Profit.
While recognizing the need to protect proprietary intellectual property, he said the basic
assumptions, math, and algorithms should be described in the Report to give the reader a basis
for judgment without giving away sensitive cost figures, etc. Harbour noted that a more detailed
diagram is needed to show the flow of information into and out of the MAQS worksheet,
especially regarding mark-up levels shown in Figure 14. Finally, Harbour commented that a data
architecture diagram indicating which secondary cost estimating modules, Munro cost
databases, etc., are used and how they feed into Design Profit, would be useful to put forth
20
-------
premises, arguments, and conclusions for future EPA regulations. She also said all data
sources should be validated statistically with hypothesis stated regarding differences between
base and new technology.
Response 27: As mentioned in Response #23, Section 4 ("Costing Methodology - Teardown,
Process Mapping and Costing") had significant updates made to improve the transparency of
the costing process. In particular, two new subsections - Costing Methodology Fundamentals
and Serial and Parallel Operations and Processes - have been added to Section 4 of the final
report. The Costing Methodology subsection improves the explanation of the two primary steps
involved in the costing process; including the connection to Design Profit® software. The Serial
and Parallel Operations and Processes subsection describes how and why serial operations,
capture in the process flow charts, are converted into mass production processes. In addition to
the new subsections added to Section 4, several updates were made to existing subsections,
providing additional details on how the Design Profit® software works. See Response #21 for a
discussion of statistical validation of input data.
All reviewers requested clarifying information regarding overhead and labor costs. For example,
Wade noted that "the Applied Burden Rate generally results in labor costs that are many times
higher than the direct labor costs," however the derivation and validation of the values used in
the equation for calculating Applied Burden Rate were not clearly described in the Report. Since
the Applied Burden Rate has a major influence on the overall labor costs, he said the lack of
adequate detail regarding the values used to calculate the Applied Burden Rate is a concern
and could potentially affect the accuracy of the cost estimate. Harbour noted that acquiring
Manufacturing Overhead Data is the most nebulous part of the Report and that more
explanation of sources is needed to support numbers. Das stated that estimation of
manufacturing overhead rates is one of the critical elements to the overall cost estimation
process - it is dependent on many parameters but the database neither indicates the estimation
procedure nor its underlying assumptions. Pages 6-16, Section 6.4.3, indicates that a template
has been developed to better organize and access these various data sources, but no such
example was available in the documentation. Finally, Harbour noted that Figure 11 is unclear
how the median wage rate is used other than to provide an informal method to determine
skewness and how this parameter impacts the Report findings.
Response 28: The discussion of how burden rates are calculated has been substantially
revised to more clearly describe the process, including a detailed description of the burden rate
template.
Regarding the scaling methodology for other engine sizes, Assanis noted that the Report states
that judicious scaling of the tear-down cost results is assumed to adequately represent other
vehicles. While the scaling process is a critical part of the methodology, he said this is not
adequately discussed. Other than the scaling assumptions documented in quote assumptions of
the CBOM, a summary needs to be included.
Response 29: See Response #1. The final report has been revised to clarify that scaling of the
cost results scaling is primarily done by EPA, outside the methodology of this study.
Wade noted that the make-up of the cross-functional team that reviews all of the data from the
high-level teardown is critical, but that the qualifications and experience of the cross-functional
team members, which would be expected to affect the quality of the reviews, are not provided.
21
-------
He said this should be added because the qualifications and experience of the team could
possibly affect the accuracy of overall costs.
Response 30: The average experience and areas of expertise of the cross functional team
members has been added to Section 1.2 ("Process Methodology and Key Supporting
Documents") of the final report.
Das, Harbour, and Wade all requested that additional information be added to the report
regarding costing. Harbour noted that the Report should explicitly state the volume basis for the
net incremental costs and whether the incremental costs are statistically significant at any
volume or financially material (to the producer and/or to the consumer) at the proposed volume.
Wade commented that to assess accuracy of cost estimates, an explanation of "total serial
process time" input data should be provided, along with clarification of the phrase "referenced to
mass production assumption calculations." Harbour requested that the Report provide the
statistical basis for the library of costing models, such as the source of data, number of data
points (samples), average, and standard deviation. Similarly, Das noted that the Report
mentions that Munro & Associates costing software is used to calculate the primary fabrication
parameters including part cost estimation, but insufficient documentation and references are
provided on the estimation methodology. Also, he questioned whether this follows the same
cost model principle outlined elsewhere in the report, including the manufacturing overhead
database for investment cost information. Wade noted that the assignment of a low or high
annual consumption rate affects pricing for a material, yet this is not reflected in Figure 7. He
requested that the text of the Report state that the application of either low or high annual
consumption rate pricing for a specific material be found in Appendix E and that confirmation of
the pricing with supplier quotes be added and/or made apparent in the Appendix. He stated that
this is viable since, at least for high annual consumption rate materials, pricing should have
been confirmed with supplier quotes. Harbour stated that, for the most likely incremental cost
drivers, presenting the assumptions used around differences to potential OEMs is critical. She
said the Brief Explanation of Differences column needs to be classified by attribute and type of
difference, such as weight and material.
Finally, regarding a specific calculation, Block Turbo Shaft Support - Cooling & Lub, Wade
noted that the calculation includes four manual and four automatic stations to derive a parallel
process multiplier of eight. Since this implies that all eight stations have labor associated with
them, he requested that an explanation of why an "automatic" station needs to have a labor
charge provided.
Response 31: The report has been revised to more clearly state the volume basis assumption
(450,000 units); see also Response #3. See Response #21 for a discussion of statistical
validation of input data. A statement in Section 7.2 ("Material Database") describes how the
material pricing used for the pilot case study (#0101) was considered to be "high consumption"
(or high volume). In reference to the Block Turbo Shaft Support - Cooling & Lub. process
captured in the Turbo Assembly Manufacturing Assumption and Quote Summary (MAQS)
worksheet, all eight stations do not have labor associated with them. Both overhead and labor,
have independent rates and quantities (i.e. quantity of people and machines) captured in the
MAQS worksheets. In this particular example, eight stations are assigned an average
manufacturing overhead of $30.00/station; total burden rate for the assembly line equals
$240/hour. There are four manual stations requiring 4 operators at a rate of $35.51/hour each;
total labor rate for the assembly line equals $142.04/hour. Wthin the MAQS worksheets, only
the total burden rate is displayed as an "Applied Burden Rate" since multiple considerations are
sometimes required to establish this rate. The Labor rate, being a much more straightforward
22
-------
calculation, only displays the base rate and quantity of laborers. Calculation of the total labor
rate is captured within the formula calculating the labor cost/part.
d. Comments on the Appendices
All reviewers provided comments on appendix materials. Many of the methodological or specific
clarifications are addressed in other sections, as appropriate, but comments referring principally
to editorial comments or omissions in the material are included here.
Assanis wondered how, due to size and format limitations, certain database files will be made
available in hard copy form. Wade recommended that the appendices be re-labeled so that the
designations E.1, F.1, etc. are clearly associated with the respective topic. Harbour, however,
commented that the appendices are generally exceptional and indicate that information is
derived from resources that know the industry.
More specifically, Das stated that the following omissions in the appendices should be
addressed: Quote Assumptions, Component Specification Assumptions, Component
Manufacturing Assumptions, and the Potential Component Suppliers in the CBOM for the
included example. Also, he found the list of variables listed in Appendix F, Table F.1 (currently
labeled Table E.1) to be inconsistent for the example case shown in Appendix G. Further, he
noted that none of the outputs in Appendix G indicate the year (current vs. future) on which the
cost analysis is based and that Appendix F.1 manufacturing process input parameters listed
under "Project Process Requirements" disagree with those used in the actual example shown in
Appendix G.
Response 32: The following note was added to Section 9.3 of the report addressing different
revisions of MAQS worksheets used during the first pilot case study:
"Note, revisions were made to the MAQS worksheet template as the cost study
analysis progressed. As such, the information shown in appendix F. 1, MAQS
Worksheet Task Reference Guide and accompanying MAQS worksheet template, will
exist at a higher revision level than a few of the case study MAQS worksheets found in
appendix G. 1. Only components which required the latest revision level MAQS
template features were updated during the first cost analysis."
In Section 13 ("Results for Case Study #0101"), a summary table of the cost analysis
assumptions has been added to the final report. Within this summary table, the year in which
the cost analysis is based upon is defined. In addition, the MAQS worksheet template has been
revised for future studies to include the production year upon which the cost estimates are
based.
23
------- |