Cost Reduction through Learning in
Manufacturing Industries and in the
Manufacture of Mobile Sources
Final Report and Peer Review Report
£% United States
Environmental Protect
Agency
-------
Cost Reduction through Learning in
Manufacturing Industries and in the
Manufacture of Mobile Sources
Final Report and Peer Review Report
Assessment and Standards Division
Office of Transportation and Air Quality
U.S. Environmental Protection Agency
Sections Prepared for EPA by
IFC
EPA Contract No. EP-C-12-011
Work Assignment No. 3-09, September 2016
and
RTI International
EPA Contract No. EP-C-11-045
Work Assignment No. 4-14, May 2016
4>EPA
United States EPA-420-R-16-018
Environmental Protection .. .
Agency November 2016
-------
Table of Contents
Executive Summary
Part I. Cost Reduction through Learning in Manufacturing Industries and in the
Manufacture of Mobile Sources, prepared for EPA by ICF (EPA Contract EP-C-12-011, Work
Assignment 3-09, September 2016)
1. Introduction
2. Selection of Subject Matter Expert and Identification of Relevant Learning-Related
Studies
3. Review of Learning Curves and Progress Ratios and a Summary of Results and
Recommendations
4. Review of Learning Curve Literature by Topic
Sources of Learning Variation
Knowledge Persistence and Depreciation
Knowledge Transfer and Spillovers
Location of Organization Knowledge
Application of the Learning Curve
5. Response to Peer Reviewer Comments Related to the Analysis
References
Appendices
Part II. Cost Reduction through Learning in Manufacturing Industries and in the
Manufacture of Mobile Sources Peer Review, prepared for EPA by RTI International (EPA
Contract EP-C-11-045, Work Assignment No. 4-14, May 2016)
1. Introduction
2. Peer Review Process
3. Summary of Findings
References
Appendices
i
-------
Executive Summary
Since the late 1990s, EPA's Office of Transportation and Air Quality (OTAQ) has
included a learning effect in our estimates of the costs of regulatory packages. In its most basic
formulation, the learning curve reflects the simple idea that the more a person does something,
the better the person can do it. This idea can also be applied to organizations: "as organizations
gain operating experience, organizational performance improves, albeit at a decreasing rate"
(Lapre and Nembhard 2010, p. 3). When applied to OTAQ's economic analyses, this means that
the cost of applying emission control technology decreases as the production volume of
compliant engines and equipment increases.
The application of the learning effect to organizations has been studied by academia and
industry for more than 60 years, and is well-known and well-accepted. In addition, there is more
and more research that seeks to quantify the learning effect for individual industries. This
research ranges from analysis of specific companies based on confidential plant-level data to
broad, industry sector studies based on national economic census data.
A brief summary of the way OTAQ has incorporated learning into our cost analyses is set
out below. To improve and validate our cost analysis methodology, OTAQ engaged ICF,
assisted by a Subject Matter Expert, Dr. Linda Argote of Carnegie Mellon University, to
examine recent empirical research on the learning effect (defined as the relationship between the
volume of production and unit costs) for manufacturing generally and the mobile source industry
in particular.1 The study has three goals:
• Provide a definitive, up-to-date, reliable, single source of information
demonstrating the occurrence of learning in general and in the mobile source
industry specifically;
• Gather into a single compendium study recent empirical research on industrial
learning in the mobile source sector for use in future OTAQ costs analyses; and
• Using the information drawn from the empirical studies, provide an estimated
summary effect of learning in mobile source industries.
The ICF study, "Cost Reduction through Learning in Manufacturing Industries and in the
Manufacture of Mobile Sources," is contained in Part I of this report. Section 2 of the ICF study
describes how ICF and the Subject Matter Expert identified the 55 published articles that form
the reference list for the study. All of these articles confirm the existence of the learning effect,
and none of them suggest that learning does not occur in organizations. Twenty-nine of the
articles address learning effects in the manufacturing sector generally; 8 of these were selected
1 OTAQ originally requested ICF to provide estimates of the learning effect separately for each of the
specific mobile source sectors (e.g., original equipment auto makers, parts suppliers to those auto makers, loose
engine manufacturers, large truck manufacturers, and nonroad equipment manufacturers) for which studies are
found that address those specific sectors. However, the literature did not support the development of unique
estimates and therefore only one progress ratio for the mobile source sector was estimated.
1
-------
for in-depth review. Twenty-six of the articles address learning effects in the mobile source
sector specifically; 10 were selected for in-depth review and the other 16 received a cursory
review. The appendix to this Executive Summary lists the 55 articles that form the basis of the
ICF study.
Section 3 of the ICF study describes the economic theory behind learning curves and
progress ratios and provides a summary table of the key findings of the articles selected for in-
depth review2; the articles are reviewed in Section 4. Most importantly, Section 3 also provides
an estimated mobile source progress ratio on the basis the results reported in 5 of the mobile
source articles (see Table 2 of the ICF study), using a weighted mean approach. The
recommended mobile source progress ratio is 84.3 percent, with a 95 percent confidence interval
of 83.9 percent to 84.8 percent.
The ICF study was peer reviewed pursuant to EPA's Science Policy Council Peer Review
Handbook, 3rd edition {Peer Review Handbook).3 In their general comments, the peer reviewers
were very supportive of the study:
• I find the report to be comprehensive, and I believe it does a good job of
characterizing the rates of learning typically found in transportation equipment
manufacturing plants. ... [T]he EPA report offers a more in-depth view of the
literature on industrial learning that is most relevant to the mobile source sector.
Overall, I find the report to be a well-executed document that is likely to be
helpful in providing a basis for incorporating forecasts of learning into EPA and
other government rulemaking. (Lieberman)
• On balance, the study is a very fine review of the literature on learning by doing in
general, but especially with regard to its manifestation in manufacturing operations
during the past few decades. The report is notably comprehensive within this scope,
makes sensible topical categorizations in its discussion of the literature's findings,
and is clearly written. ... In sum, it is my opinion that the report does achieve the
intended goal of being a definitive, reliable, single source of information
demonstrating the occurrence of learning in general and in the mobile source industry
specifically. (Syverson)
• The overall presentation and organization of the Report is generally clear. However,
there are some specific areas that require greater clarity. These are described below.
(B al asubr amani an)
2 There are 21 articles included in this table: 18 articles selected for in-depth review as well as three others that were
deemed important.
3 These guidelines can be found at http://www.epa.gov/peerreview/. Further, the Office of Management and
Budget's (OMB's) Information Quality Bulletin for Peer Review and Preamble (found in the EPA's Peer Review
Handbook, Appendix B) contains provisions for conducting peer reviews across federal agencies and may serve as
an overview of EPA's peer review process and principles. The results of the peer review of this study are included
in the Appendix to this report.
2
-------
Part II of this report contains information about the peer review, which was performed for
OTAQ by RTI International, and all of the peer review comments. The ICF responses to the
peer review comments are set out in Section 5 of the ICF study. It should be noted that while the
peer reviewers commented on the methodology used to estimate the recommended mobile source
progress ratio, those comments did not lead the Subject Matter Expert to change that
methodology or revise that estimate.
Learning Effect in OTAQ's Cost Analyses
EPA's Office of Transportation and Air Quality (OTAQ) has included learning effects in
its cost estimates for its rulemaking packages beginning with its 1997 rule adopting emission
standards for Model Year 2004 heavy-duty engines (62 FR 54694, October 21, 1997).4 Table 1
provides information on many of these rules and how they incorporated learning.
As explained in the 1997 heavy-duty rule, "[rjesearch in the costs of manufacturing has
consistently shown that as manufacturers gain experience in production, they are able to apply
innovations to simplify machining and assembly operations, use lower cost materials, and reduce
the number or complexity of component parts" (62 FR 54711, October 21, 1997). To
incorporate this principle, OTAQ used a learning curve algorithm that applied a learning factor
of 20 percent (80 percent progress ratio) for each doubling of cumulative production volume.
This approach was simplified by using a time-based learning progression rather than a pure
production volume progression (i.e., after a specified number of years of production it was
assumed that cumulative production volumes would have doubled and, therefore, costs would be
reduced by 20 percent). This approach of reducing costs in discrete steps, with a varying number
of steps depending on the novelty of the relevant technology, was used through the 2008 Small
SI rule (also called the Bond Rule, 73 FR 59034, October 8, 2008).
Beginning with the first light-duty greenhouse gas rule (EPA420-R-10-109, April 2010),
OTAQ began to apply a more nuanced approach to incorporate learning effects in cost analyses,
in which the rate of learning and therefore the level of cost reduction due to learning depends on
where on the learning curve a technology's learning progression is. In this approach, the steep-
portion learning algorithm applies to those technologies considered to be newer technologies
likely to experience rapid cost reductions through manufacturer learning and the flat-portion
learning algorithm applies to those technologies considered to be mature technologies likely to
experience minor cost reductions through manufacturer learning.5 Costs for newer technologies,
4 In 1977, a contractor commissioned by EPA developed "estimates of the retail price equivalent or "sticker price"
for a variety of automotive exhaust emission control related components/systems," which included a learning
component. Learning was estimated based on prices from U.S. and European sources for varying quantities of
specific components. Based on those prices, a progress ratio of 91.4 percent was estimated. EPA 1980, Cost
Estimates for Emission Control Related Components/Systems and Cost Methodology Description, Heavy Duty
Trucks, EPA-460-3-80-001, February 1980; see also EPA 460/3-78-002 (report date December 1977).
5 Initially, OTAQ distinguished between "volume-based" learning (steep portion of the learning curve) and "time-
based" learning (flat portion of the learning curve); see EPA 420-4-10-901, April 2010, p. 3-18. However, as noted
in the Heavy-Duty GHG rule, OTAQ quickly recognized that
3
-------
said to be on the "steep" portion of the learning curve, were reduced by 20 percent at discrete
intervals; later, and for mature technologies said to be on the "flat" portion of the learning curve,
costs were reduced at a decreasing percentage (three, then two, then one percent) and at longer
intervals.
In its 2014 report, the National Research Council of the National Academy of Sciences
noted that "[although technological change is certain, its direction, magnitude, and impacts on
cost are difficult to predict. For most components, manufacturing costs tend to decrease with
increased production volumes and with the accumulation of experience. However, there are no
exact methods for predicting future rates of learning by doing or technological progress" (NAS
2014, p 245, 250). The authors note that EPA uses an unconventional approach for learning, as a
function of time rather than volume. Their recommendation 7.2 states:
The Agencies should make clear the terminology associated with learning and should
assess whether and how volume-based learning might be better incorporated into their
cost estimates, especially for low volume technologies. The Agencies should also
continue to conduct and review empirical evidence for the cost reductions that occur in
the automobile industry with volume, especially for large-volume technologies that will
be relied on to meet the CAFE/GHG standards. NAS 2014, p. 259-60.
To ensure that the learning effects incorporated in OTAQ's cost estimates are based on a
comprehensive survey of the literature, OTAQ engaged ICF, with the assistance of a Subject
Matter Expert (Dr. Linda Argote of Carnegie Mellon University), to develop a single
compendium study on industrial learning in the mobile source sector. This report contains the
results of that study.
.. .all learning is, in fact, volume-based learning, the level of cost reductions depend only on where on the
learning curve a technology's learning progression is. We distinguish the flat portion of the curve from the
steep portion of the curve to indicate the level of learning taking place in the years following
implementation of the technology." (EPA 420-R-l 1-901, August 2011, p. 2-9)
More recently, in the Light-Duty GHG rule, EPA explained
... we have updated our terminology in an effort to clarify that we consider there to be one learning
effect—learning by doing—which results in cost reductions occurring with every doubling of production.
In the past, we have referred to volume-based and time-based learning. Our terms were meant only to
denote where on the volume learning curve a certain technology was—"volume-based learning" meant the
steep portion of the curve where learning effects are greatest, while "time-based learning" meant the flatter
portion of the curve where learning effects are less pronounced. Unfortunately, our terminology led some to
believe that we were implementing two completely different types of learning—one based on volume of
production and the other based on time in production. Our new terminology—steep portion of the curve
and flat portion of curve—is simply meant to make more clear that there is one learning curve and some
technologies can be considered to be on the steep portion while others are well into the flatter portion of the
curve. (EPA 420-R-12-901, August 2012, p. 3-23)
4
-------
Table 1 - OTAQ Rules Incorporating Learning Effects in Cost Analyses, 1997-2008
Rule
Federal Register
Citation
Technologies
Learning Progress Ratio
New or Mature Technology
1997 Heavy-duty
MY2004 Highway
Rule
62 FR 54694
(10/21/97)
Fuel system changes; EGR
- One 20% learning curve
reduction
- Applied in Year 3
Rule met via changes to existing
technology
RIAEPA420-R-97-011, September
1997
1998 Nonroad Diesel
Tier 2 & 3
63 FR 56968
(10/23/98)
Fuel system changes; EGR
- Two 20% learning curve
reductions
- Applied in Years 3 and 6
Rule met via application of emission
controls to the sector for the first time
RIAEPA420-R-98-016; August 1998
1999 Marine Diesel
Rule
64 FR 73300
(12/29/99)
Fuel system changes; EGR
- Two 20% learning curve
reductions
- Applied in Years 3 and 6
Rule met via application of emission
controls to the sector for the first time
RIA EPA420-R-99-026; November
1999
2000 Tier 2 Light-duty
Highway Rule
65 FR 6698
(2/10/00)
catalyst; secondary air injection,
fuel control, exhaust system
changes, combustion chamber
changes, EGR
- One 20% learning curve
reduction
- Applied in Year 3
Rule met via changes to existing
technology
RIA EPA420-R-99-023; December
1999
2000 Tech Review of
HD2004 Rule
65 FR 59896
(10/6/00)
Fuel system changes; EGR
- One 20% learning curve
reduction
- Applied in Year 3
Rule met via changes to existing
technology
RIA: EPA420-R-00-010; July 2000
2001 Heavy-duty
MY2007 Highway
Rule
66 FR 5002
(1/18/01)
Afltertreatment systems including
in-exhaust reductant injectors,
catalyst components
- Two 20% learning curve
reductions
- Applied in Years 3 and 5
Rule met via new technology
RIA: EPA420-R-00-026; December
2000
2002 Nonroad Large
SI and Recreational
Engines Final Rule
67 FR 68242
(11/8/02)
Recalibration, fuel system upgrades;
improved combustion and
aftercooling
- Two 20% learning curve
reductions
- Applied in Years 3 and 6
Rule met via application of emission
controls to the sector for the first time
RIA: EPA420-R-02-022; September
2002
5
-------
Rule
Federal Register
Citation
Technologies
Learning Progress Ratio
New or Mature Technology
2004 Nonroad Tier 4
Rule
69 FR 38958
(6/29/04)
Aftertreatment systems including
in-exhaust reductant injectors,
catalyst components; EGR
- One 20% learning curve
reduction
- Applied in Year 3
- Starting point was the Year 3
HD2007 Rule
Rule met via HD2007 technology
applied to nonroad engines for first
time
RIA: EPA420-R-04-007; May 2004
2008 LocoMarine
Rule
73 FR 37096
(6/30/08)
Aftertreatment systems including
in-exhaust reductant injectors,
catalyst components
- One 20% learning curve
reduction
- Applied in Year 3
- Starting point was the Year 3
NRT4 Rule
Rule met via HD2007/NRT4
technology applied to LocoMarine
engines for first time
RIA: EPA420-R-08-001; May 2008
2008 Nonroad Small
SI Rule (Bond Rule)
73 FR 59034
(10/8/08)
Catalyst, combustion chamber
changes, improved fuel systems
- One 20% learning curve
reduction
- Applied in Year 6
Rule met via changes to existing
technology
RIA: EPA420-R08-014; September
2008
2010 Light-duty GHG
Rule
75 FR 25324
(5/7/10)
Fuel consumption reducing
powertrain and vehicle technologies
along with vehicle electrification
technologies
Technology dependent -
technologies were placed on the
steep or flat portion of the
typical learning progression; as
in above analyses, time was used
as a proxy for production
volumes
Joint TSD: EPA420-R-10-901; April
2010
2011 Heavy-duty
GHG rule
76 FR 57106
(9/15/11)
Fuel consumption reducing
powertrain and vehicle technologies
along with vehicle electrification
technologies
Technology dependent -
technologies were placed on the
steep or flat portion of the
typical learning progression; as
in above analyses, time was used
as a proxy for production
volumes
RIA: EPA420-R-11 -901; August
2011
6
-------
Rule
Federal Register
Citation
Technologies
Learning Progress Ratio
New or Mature Technology
2012 Light-duty GHG
rule
77 FR 62624
(10/15/12)
Fuel consumption reducing
powertrain and vehicle technologies
along with vehicle electrification
technologies
Technology dependent -
technologies were placed on the
steep or flat portion of the
typical learning progression; as
in above analyses, time was used
as a proxy for production
volumes
RIA: EPA420-R-12-016; August,
2012; Joint TSD: EPA420-R-12-901;
August 2012
2014 Tier 3 Light-duty
Highway Rule
79 FR 23414
(4/28/2014)
catalyst; secondary air injection,
fuel control, exhaust system
changes, combustion chamber
changes, EGR
Technology dependent -
technologies were placed on the
steep or flat portion of the
typical learning progression; as
in above analyses, time was used
as a proxy for production
volumes
RIA: EPA420-R-14-005; February
2014
2015 Heavy-duty
GHG proposed rule
80 FR 40138
(7/13/2015)
Fuel consumption reducing
powertrain and vehicle technologies
along with vehicle electrification
technologies
Technology dependent -
technologies were placed on the
steep or flat portion of the
typical learning progression; as
in above analyses, time was used
as a proxy for production
volumes
RIA: EPA420-D -15-002; June 2015
7
-------
Appendix: List of 55 Articles that Form the Basis of the ICF Study
1. Learning in Manufacturing in General - Articles selected for detailed review (8 articles)
Argote, Linda, Dennis Epple. 1990. Learning curves in manufacturing. Science, 247(4945), 920-
924. http://www.sciencemag.org/content/247/4945/920.short
Argote, Linda. 2013. Organizational learning: Creating, retaining and transferring knowledge.
Springer. (Chapters 1. 3 and 6)
https://facultv.fuqua.duke.edu/~charlesw/s591/willstuff/oldstuff/PhD 2008 2009 LongStrat/Rea
dings/Class07 Learning/OrganOOl.pdf
Bahk, B., & Gort, M. 1993. Decomposing learning by doing in new plants. Journal of Political
Economy, 101, 561-582.
http://www.istor.org/discover/10.2307/2138739?uid=3739936&uid=2&uid=4&uid=3739256&si
d=21103414588873.
Balasubramanian, Natarajan, and Marvin B. Lieberman. 2010. Industry learning environments
and the heterogeneity of firm performance. Strategic Management Journal 31.4, 390-412.
http://www.anderson.ucla.edu/faculty/marvin.lieberman/docs/Balasubramanian-Lieberman-SMJ-
April2010.pdf
Dutton, John, Annie Thomas. 1984. Treating Progress Functions as a Managerial Opportunity.
The Academy of Management Review, 9(2), 235-247.
http://www.istor.org/discover/10.2307/258437?uid=3739936&uid=2&uid=4&uid=3739256&sid
=21103550037613
Lapre, Michael A, Ingrid M Nembhard. 2010. Inside the Organizational Learning Curve:
Understanding the Organizational Learning Process. Foundations and Trends in Technology,
Information and Operations Management, 4(1), 1-103.
http://resource.owen.vanderbilt.edu/facultvadmin/data/research/2265full.pdf.
Macher, J.T., & Mowery, D. C. 2003. Managing learning by doing: An empirical study in semi-
conductor manufacturing. Journal of Product Innovation Management, 20(5), 391-410.
http://faculty.msb.edu/itm4/Papers/ipim.2003.pdf
Rubin, Edward S., et al. 2004. Learning curves for environmental technology and their
importance for climate policy analysis. Energy 29, 1551-1559.
http://energv.lbl.gov/staff/tavlor/pdfs/rubin-tavlor-energy-2004.pdf
8
-------
2. Learning in Manufacturing in General - Articles not selected for review (21 articles)
Adler, P. S., & Clark, K. B. 1990. Behind the learning curve: A sketch of the learning process.
Management Science, 37, 267-281. http://www-
bcf.usc.edu/~padler/research/Behind%20the%20Learning%20Curve-l.pdf
Balasubramanian, Natarajan, and Marvin B. Lieberman. 2011. Learning-by-doing and market
structure. The Journal of Industrial Economics 59.2, 177-198.
http://164.67.163.139/Documents/areas/fac/policv/learning marketstructure.pdf
Day, George S., and David B. Montgomery. 1983. Diagnosing the experience curve. The Journal
of Marketing, 44-58. https://gsbapps.stanford.edu/researchpapers/library/RP641.pdf
Ghemawat, P. 1984. Building strategy on the experience curve. Harvard Business Review, 63,
143-149. http://hbr.org/product/building-strategv-on-the-experience-curve/an/85206-PDF-ENG
Heim, J. 1992. Manufacturing systems: foundations of world-class practice. Committee on
Foundations of Manufacturing, National Academy of Engineering.
http://www.nap.edu/catalog.php7record id=l867
Hendel, I. and Spiegel, Y. 2014. Small steps for workers, a giant leap for productivity.
American Economic Journal: Applied Economics. 6.1,73-90. January 2014.
https://www.aeaweb.org/articles?id=10.1257/app.6.1.73
Irwin, Douglas, Peter Klenow. 1994. Learning-by-Doing Spillovers in the Semiconductor
Industry. The Journal of Political Economy, 102(6), 1200-1227.
http://www.istor.org/discover/10.2307/2138784?uid=3739936&uid=2&uid=4&uid=3739256&si
d=21103414588873
Jarmin, R.S. 1994. Learning by doing and competition in the early rayon industry. The Rand
Journal of Economics, 25, 441-454.
http://www.istor.org/discover/10.2307/2555771?uid=3739936&uid=2&uid=4&uid=3739256&si
d=21103550037613
Klenow, Peter J. 1998. Learning curves and the cyclical behavior of manufacturing industries.
Review of Economic Dynamics 1.2, 531-550. http://www.klenow.com/REDLC.pdf
Laitner, John A'Skip, and Alan H. Sanstad. 2004. Learning-by-doing on both the demand and
the supply sides: implications for electric utility investments in a Heuristic model. International
Journal of Energy Technology and Policy 2.1, 142-152.
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.202.8902&rep=repl&tvpe=pdf
Lapre, Michael A, Amit Shankar Mukherjee, Luk Van Wassenhove. 2000. Behind the Learning
Curve: Linking Learning Activities to Waste Reduction. Management Science, 46(5), 597-611.
http://www.istor.org/discover/10.2307/2661461?uid=3739936&uid=2&uid=4&uid=3739256&si
d=21103550037613
9
-------
Lieberman, Marvin B. 1984. The learning curve and pricing in the chemical processing
industries. The RAND Journal of Economics 15.2, 213-228.
http://www.anderson.ucla.edu/facultv/marvin.lieberman/publications/LC-Randl984.pdf
Lieberman, Marvin B. 1987. Market growth, economies of scale, and plant size in the chemical
processing industries. The Journal of Industrial Economics, 175-191.
http://www.anderson.ucla.edu/faculty/marvin.lieberman/publications/PlantSize-JIE1987.pdf
Lieberman, Marvin B. 1987. Patents, learning by doing, and market structure in the chemical
processing industries. International Journal of Industrial Organization 5.3, 257-276.
http://www.sciencedirect.com/science/article/pii/S0167718787800Q97
Lieberman, Marvin B. 1987. The learning curve, diffusion, and competitive strategy. Strategic
management journal 8.5, 441-452.
http://www.anderson.ucla.edu/facultv/marvin.lieberman/publications/LC-Strategy-SMJ1987.pdf
Lieberman, Marvin B. 1989. The learning curve, technology barriers to entry, and competitive
survival in the chemical processing industries. Strategic Management Journal 10.5, 431-447.
http://www.anderson.ucla.edu/facultv/marvin.lieberman/publications/LC-Survival-SMJ1989.pdf
Nagy, Bela, et al. 2013. Statistical basis for predicting technological progress. PloS one 8.2,
e52669. http://www.plosone.ore/article/info%3Adoi%2F10.1371%2Fiournal.pone.0052669
Sinclair, G., Klepper, S., & Cohen, W. 2000. What's experience got to do with it? Sources of
cost reduction in a large specialty chemicals producer. Management Science, 46, 28-45.
http://pubsonline.informs.org/doi/abs/10.1287/mnsc.46.1.28.15133
Suarez, Fernando, Michael Cusumano, Charles Fine. 1996. An Empirical Study of
Manufacturing Flexibility in Printed Circuit Board Assembly. Operations Research, 44(1), 223-
240.
http://people.bu.edu/suarezf/Fernando Suarez Website/Publications files/1996 An%20Empiric
al%20Studv%20of%20Manuf%20Flexibilitv Suarez Cusumano Fine OR.pdf
Syverson, C. (2011). What determines productivity? Journal of Economic Literature, 49 (2),
326-365. (See especially sections 3.4 on learning by doing and 4.1 on productivity spillovers).
http://home.uchicago.edu/~svverson/productivitvsurvev.pdf
Yelle, Louis E. 1979. The learning curve: Historical review and comprehensive survey. Decision
Sciences 10.2, 302-328. http://tuvalu.santafe.edu/~bn/reading group/Yelle.pdf
10
-------
3. Learning in Mobile Source Manufacturing Sectors - Articles selected for detailed review
(10 articles)
Agrawal, A., Muthlingam, S. 2015. Does organization forgetting affect vendor quality
performance? An empirical investigation. Manufacturing & Service Operations Management,
Articles in Advance, pp. 1-18. April 2015.
http://pubsonline.informs.org/doi/abs/10.1287/msom.2015.0522
Benkard, C Lanier. 2000. Learning and Forgetting: The Dynamics of Aircraft Production. The
American Economic Review, 90(4), 1034-1054.
http://www.econ.yale.edu/~lanierb/research/Learning and Forgetting AER.pdf
Bernstein, Paul. 1988. The Learning Curve At Volvo. Columbia Journal Of World Business
23.4. 87-95. Business Source Complete. Web. 25 Nov. 2013.
Epple, Dennis, Linda Argote, and Rukmini Devadas. 1991. Organizational learning curves: A
method for investigating intra-plant transfer of knowledge acquired through learning by doing.
Organization Science 2.1, 58-70.
http://pubsonline.informs.org/doi/abs/10.1287/orsc.2.1.58?iournalCode=orsc
Epple, Dennis, Linda Argote, Kenneth Murphy. 1996. An empirical investigation of the
microstructure of knowledge acquisition and transfer through learning by doing. Operations
Research, 44, 77-86.
http://www.istor.org/discover/10.2307/171906?uid=3739936&uid=2&uid=4&uid=3739256&sid
=21103490584923
Gopal, A, M Goyal, S Netessine, M Reindorp. 2013. The Impact of New Product Introduction on
Plant Productivity in the North American Automotive Industry. Management Science, 59(10),
2217-2236. http://www.rhsmith.umd.edu/facultv/agopal/AutoLaunch%20MS%20Final.pdf
Lee, Jaegul, et al. 2010. Forcing technological change: A case of automobile emissions control
technology development in the US. Technovation 30.4, 249-264.
http ://www. sciencedirect.com/science/article/pii/SO 166497209001746
Levitt, Steven D., John A. List, and Chad Syverson. 2012. Toward an Understanding of Learning
by Doing: Evidence from an Automobile Assembly Plant. No. wl8017. National Bureau of
Economic Research, http ://www.nber.org/papers/wl 8017
Nykvist, B., Nilsson, M. 2015. Rapidly falling costs of battery packs for electric vehicles.
2015. Nature Climate Change, 5, 329-332.
http://www.nature.com/nclimate/iournal/v5/n4/full/nclimate2564.html
Shinoda, Yukio, et al. 2009. Evaluation of the plug-in hybrid electric vehicle considering
learning curve on battery and power generation best mix. IEEJ Transactions on Power and
Energy 129, 84-91.
http://onlinelibrarv.wiley.com/doi/10.1002/eei.21098/abstract;isessionid=FF83CDDA7D582392
635F6CB5F4F4FCCA.f03t01?deniedAccessCustomisedMessage=&userIsAuthenticated=false
11
-------
4. Learning in Mobile Source Manufacturing Sectors - Articles selected for cursory review
(16 articles)
Alchian, Arm en. 1963. Reliability of Progress Curves in Airframe Production. Econometrica:
Journal of the Econometric Society, 31, 679-693.
http://www.caps.am/materials/diaspora%20writings/Reliabilitv%20of%20progress%20curves.pd
f
Argote, Linda, Sara Beckman, Dennis Epple. 1990. The persistence and transfer of learning in
industrial settings. Management Science, 36(2), 140-154.
http://wpweb2.tepper.cmu.edu/facultvadmin/upload/ppaper 632795255914 2661452.pdf
Bailey, M.N., Farrell, D., Greenberg, E., Henrich, J.-D., Jinjo, N. Jolles, M. & Remes, J. 2005.
Increasing global competition and labor productivity: Lessons learned from the US automotive
industry. Paper presented at the Federal Reserve Bank of San Francisco conference,
"Productivity Growth: Causes and Consequences." Nov 18-19, 2005.
http://www.mckinsev.com/industries/automotive-and-assemblv/our-insights/increasing-global-
competiti on-and-1 ab or-producti vitv
Fisher, Marshall L., and Christopher D. Ittner. 1999. The impact of product variety on
automobile assembly operations: Empirical evidence and simulation analysis. Management
science 45.6, 771-786. http://pubsonline.informs.Org/doi/abs/10.1287/mnsc.45.6.771
Fisher, Marshall, Kamalini Ramdas, Karl Ulrich. 1999. Component Sharing in the Management
of Product Variety: A Study of Automotive Braking Systems. Management Science, 45(3), 297-
315. http://opim.wharton.upenn.edu/~ulrich/downloads/parts.pdf
Haunschild,p.R., Rhee, M. 2004. The role of volition in organization learning: The case of
automotive product recalls. Management Science, 50.11, 1545-1560. 2004.
http://pubsonline.informs.org/doi/abs/10.1287/mnsc.1040.0219
Jaber, M. Y., S. K. Goyal, and M. Imran. 2008. Economic production quantity model for items
with imperfect quality subject to learning effects. International Journal of Production Economics
115.1, 143-150. http://www.sciencedirect.com/science/article/pii/S09255273080Q1527
Kim, Hhyung, Hae Lim Seo. 2009. Depreciation and transfer of knowledge: an empirical
exploration of a shipbuilding process. International Journal of Production Research, 47(7), 1857-
1876. http://www.tandfonline.eom/doi/abs/10.1080/00207540701499481#.UvaC4fldVHA
Levin, D.Z. 2000. Organizational learning and the transfer of knowledge: An investigation of
quality improvement. Organization science, 11.5,630-647.
http://www.levin.rutgers.edu/research/learning-curve-paper.pdf
MacDuffie, John Paul, Kannan Sethuraman, and Marshall L. Fisher. 1996. Product variety and
manufacturing performance: evidence from the international automotive assembly plant study.
Management Science 42.3, 350-369.
http://pubsonline.informs.Org/doi/abs/10.1287/mnsc.42.3.350
12
-------
Randall, Taylor, Karl Ulrich. 2001. Product Variety, Supply Chain Structure, and Firm
Performance: Analysis of theU. S. Bicycle Industry. Management Science, 47(12),1588-1604.
http://www.ktulrich.eom/uploads/6/l/7/l/6171812/bike-supplvchains.pdf
Rapping, Leonard. 1965. Learning and World War II Production Functions. The Review of
Economics and Statistics, 47(1), 81-86.
http://www.istor.org/discover/10.2307/1924126?uid=3739936&uid=2&uid=4&uid=3739256&si
d=21103 551891913
Thompson, Peter. 2007. How Much Did the Liberty Shipbuilders Forget? Management Science,
53(6), 908-918.
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.98.2440&rep=repl&type=pdf
Thompson, Peter. 2001. How much did the Liberty Shipbuilders learn: New evidence for an old
case study. Journal of Political Economy, 109(1), 103-137.
http://tuvalu.santafe.edu/~bn/reading group/Thompson.pdf
Thompson, Peter, Rebecca Achee Thornton. 2001. Learning by experience and learning from
others: An exploration of learning and spillovers in wartime shipbuilding. American Economic
Review, 91(5), 1350-1368.
http://www.istor.org/discover/10.2307/2677929?uid=3739936&uid=2&uid=4&uid=3739256&si
d=21103 551891913
Tsuchiya, Haruki, and Osamu Kobayashi. 2004. Mass production cost of PEM fuel cell by
learning curve. International Journal of Hydrogen Energy 29.10, 985-990.
http://www.sciencedirect.com/science/article/pii/S03603199030Q3136
13
-------
Cost Reduction through
Learning in
Manufacturing Industries
and in the Manufacture
of Mobile Sources
September 30, 2016
Prepared for
U.S. Environmental Protection Agency
Office of Transportation and Air Quality
Assessment and Standards Division
2000 Traverwood Drive
Ann Arbor, Ml 48105
Prepared by
ICF
9300 Lee Highway
Fairfax, VA 22031
-------
-------
Contents
1. Introduction 1
2. Selection of Subject Matter Expert and Identification of Relevant Learning-Related
Studies 3
3. Review of Learning Curves and Progress Ratios and a Summary of Results and
Recommendations 7
3.1. What are Learning Curves? 7
3.2. What are Progress Ratios? 10
3.3. Summary of Literature Review 12
3.4. Discussion of Mobile Source Results and Recommendations 19
4. Review of Learning Curve Literature by Topic 23
4.1. Sources of Learning Variation 23
4.1.1. Dutton & Thomas, 1984 23
4.1.2. Argote & Epple, 1990 24
4.1.3. Macher & Mowery, 2003 25
4.1.4. Balasubramanian & Lieberman, 2010 25
4.1.5. Lapre & Nembhard, 2010 26
4.1.6. Conclusion 28
4.2. Knowledge Persistence and Depreciation 28
4.2.1. Epple, Argote, & Murphy, 1996 29
4.2.2. Benkard, 2000 30
4.2.3. Argote, 2013 30
4.2.4. Gopal, Goyal, Netessine, & Reindorp, 2013 32
4.2.5. Agrawal & Muthulingam, 2015 32
4.2.6. Conclusion 33
4.3. Knowledge Transfer and Spillovers 35
4.3.1. Epple, Argote, & Devadas, 1991 35
4.3.2. Benkard, 2000 36
4.3.3. Levitt, List, & Syverson, 2013 36
4.3.4. Conclusion 37
4.4. Location of Organizational Knowledge 37
4.4.1. Epple, Argote, & Devadas, 1991 38
4.4.2. Levitt, List, & Syverson, 2013 38
4.4.3. Agrawal & Muthulingam, 2015 39
4.4.4. Bahk & Gort, 1993 39
4.4.5. Conclusion 40
4.5. Application of the Learning Curve 40
4.5.1. Bernstein, 1988 40
4.5.2. Rubin, Taylor, Yeh, & Hounshell, 2004 41
4.5.3. Shinoda, Tanaka, Akisawa, & Kashiwagi, 2009 41
4.5.4. Lee, Veloso, Hounshell, & Rubin, 2010 42
4.5.5. Nykvist & Nilsson, 2015 42
4.5.6. Conclusion 43
si
'ICF
September 30, 2016
-------
5. Responses to Peer Reviewer Comments Related to the Analysis 45
6. References 49
Appendix A. Method of Estimating Impacts of Learning 53
Appendix B. Summaries of Articles that Received a Detailed Review 59
Appendix C. Summaries of Articles Related to the Mobile Source Sector that Received a
Cursory Review 129
Appendix D. Responses to Peer Review Comments 153
Appendix E. Peer Review Report 179
Figures
Figure 1. Learning Curve for the Truck Plant 8
Figure 2. Logarithm of Direct Labor Hours per Vehicle versus Logarithm of Cumulative
Hours 9
Figure 3. Distribution of Progress Ratios Observed in 22 Field Studies (N=108) 12
TaMes
Table 1. Summary of Progress Ratios in Sample 15
Table 2. Confidence Intervals of Progress Ratios from Selected Studies 20
Table 3. Summary of Depreciation Parameter Estimates 33
sl_
'icf iv September 30,2016
-------
Acronyms and Abbreviations
BEV
Battery Electric Vehicles
CARB
California Air Resource Board
C02
Carbon Dioxide
EPA
United States Environmental Protection Agency
DLA
Deliberate Learning Activity
FGD
Flue Gas Desulfurization
HR
Human Resource
IT
Information Technology
NOx
Nitrogen Oxides
OTAQ
Office of Transportation and Air Quality
PHEV
Plug-In Hybrid Electric Vehicle
R&D
Research and Development
SCR
Selective Catalytic Reduction
SIC
Standard Industrial Classification
SME
Subject Matter Expert
SO2
Sulfur Dioxide
sl_
'ICF
V
September 30, 2016
-------
'ICF vi September 30,2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
1. Iliriitr@dl[mctloim
Since the late 1990s, EPA's Office of Transportation and Air Quality (OTAQ) has included a learning effect
when estimating the costs of regulatory packages. Specifically, technology costs—for technologies
added to mobile sources to allow for compliance with new emissions standards—are estimated to
decrease in the years following first implementation. This decrease in technology costs, either due to
the volume of production or to time, is considered to be due to learning (i.e., the "learning effect").
We use the term "learning effect" to refer to the relationship between the volume of production (i.e.,
cumulative output) and unit costs. Cumulative output is a measure of experience gained in production.
Just as individuals have been found to benefit from their experience, groups and organizations have also
been found to benefit from the experience they acquire. Learning can reflect efficiencies gained in
production processes, improvements in tooling and in the design of the manufactured components,
increased proficiency of individual employees, and improvements in the organization's structure or
some combination of these factors. This learning effect has been studied by academia and industry for
more than 60 years. Many studies are available that examine the learning effect, or aspects of it; the
vast majority of these studies conclude that cost reductions through learning do, in fact, occur. Other
studies assume that cost reductions will occur based on the body of evidence suggesting that they do
and incorporate learning effects into their analysis, as EPA does in its cost analyses.
The relationship between experience and performance has been documented in both laboratory and
field studies. Laboratory studies are high in internal validity and enable one to establish causality while
field studies are high in external validity and enable one to estimate the effects of variables in realistic
conditions (Croson, Anand, & Agarwal, 2007). For purposes of the current project, we focus our review
on field studies of the relationship between cumulative output and unit costs. The evidence of an effect
of experience on performance in laboratory studies (Argote, Insko, Yovetich, & Romero, 1995; Guetkow
& Simon, 1955) increases our confidence that experience has a causal effect on performance indicators,
such as unit costs.
While there is little doubt that this learning effect occurs, the learning estimates used by OTAQ in its
recent cost analyses are based on somewhat dated studies that are not specific to the mobile source
sector. Therefore, EPA tasked ICF with a work assignment1 that would involve conducting an assessment
of learning covering most notably the automotive industry (both original equipment manufacturers and
Tier 1 suppliers). In addition to studies of learning for the light-duty vehicle sector and automotive parts
suppliers, the scope of the learning assessment would cover other on-road mobile source industries,
such as manufacturing of loose engines (i.e., those built for installation in large highway trucks and/or
non-road equipment), manufacturing of large vocational and line-haul trucks, and manufacturing of
large non-road equipment. This work would provide a definitive, up-to-date, reliable, single source of
information evaluating the occurrence of learning in the mobile source industries. It would also
summarize empirical estimates of the learning effect separately for each of the specific mobile source
industries (e.g., original equipment auto makers, parts suppliers to those auto makers, loose engine
1 EPA Contract EP-C-12-011 Work Assignment 3-09.
'ICF
1
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
manufacturers, large truck manufacturers, and non-road equipment manufacturers) for which studies
are found that address those specific sectors. Finally, using that information, the study would provide an
estimate of learning effects for each of the separate mobile source industries for which published data
exists.
As explained in more detail in Section 2, the literature did not support the development of unique
estimates for the separate mobile source industries because very few studies have been published on
learning in mobile source industries outside of the automotive industry. This could be due to the
confidential nature of the data that would be necessary to conduct such a study. Such data are typically
viewed as proprietary and are not publically available. It would be very difficult to obtain permission to
combine such proprietary data with those from other firms and competitors for the purpose of such a
study. For this reason, this report provides EPA with a single learning rate for the whole mobile source
sector at the organizational (i.e., plant) level, rather than for specific mobile source industries.
Although the literature did not support the development of unique estimates for separate mobile
source industries, it did support the development of estimates of the rate of organizational learning in
the mobile source sector generally. Therefore, this report aims to meet three objectives: (1) to be a
definitive, up-to-date, reliable, single source of information demonstrating the occurrence of learning in
general and in the mobile source industry specifically; (2) to develop a single compendium study on
industrial learning in the mobile source sector that could be considered for use in future OTAQ costs
analyses; and (3) to develop a summary effect of learning based on cumulative output in mobile source
industries. By developing a summary effect, we mean that we will aggregate learning rates—more
specifically, progress ratios—found in relevant articles to come up with a single mobile source progress
ratio for EPA to consider for use in future OTAQ cost analyses.
This report provides an assessment of learning, both generally and as it relates to the mobile source
industry through a review of 18 published studies on learning curves. In Section 2, we describe the
methodology used to identify studies that form the basis of the analysis. Section 3 contains a summary
of the analysis and recommended progress ratio for the mobile source industry. Section 4 contains
detailed summaries of the 18 studies reviewed by topic. There are 4 appendices to this report. Appendix
A describes two methods that can be used for estimating the impacts of learning. Appendix B provides
notes on the 18 articles that received a detailed review and are discussed in Section 4. Appendix C
provides notes on the 16 articles that apply to the mobile source sector and received a cursory review.
Appendix D contains responses to comments that were provided in a separate peer review process
undertaken by EPA.
\l_
'ICF
2
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
2. Selection of Subject IVIafiw bxpest arid Identification of
Relevant; l.earriirig-Relatecl Studies
EPA engaged ICF to perform an assessment of learning as it relates to manufacturing sectors generally
and mobile source industries specifically. The assessment would consist of a literature review of studies
of learning in mobile source industries and would identify empirical estimates of learning from those
studies, as well as studies of learning in general manufacturing to provide background and context for
the literature review. The goals of the assessment are to develop (1) a definitive, up-to-date, reliable,
single source of information demonstrating the occurrence of learning in general and in the mobile
source industry specifically; (2) a single compendium study on industrial learning in the mobile source
sector that could be considered for use in future OTAQ costs analyses; and (3) an estimated summary
effect of learning in mobile source industries.
Because of the specialized nature of this project, EPA requested ICF seek the assistance of a subject
matter expert (SME). To identify the SME, ICF searched university websites to find academic researchers
who had published extensively in the field of manufacturing learning curves as it related to automotive
and mobile source equipment industries. This resulted in eight possible candidates who included: (1) Dr.
Linda Argote of Carnegie Mellon University; (2) Jamie McCarthy of the Boston Consulting Group; (3) Dr.
Pete Klenow of Stanford University; (4) Dr. Edward S. Rubin of Carnegie Mellon University; (5) Dr.
George Day of the University of Pennsylvania; (6) Dr. Birger Wernerfelt of the MIT Sloan School of
Management; (7) Dr. David B. Montgomery of Stanford University; and (8) Dr. Marvin Lieberman of the
University of California, Los Angeles. Dr. Argote expressed an interest in this project and was selected
because of her expertise and extensive publications in the area of automotive manufacturing learning
curves.
At the same time, ICF conducted a preliminary literature search to identify articles that examine learning
curves in the manufacturing sectors generally and in mobile source manufacturing specifically. Initially,
this list was developed by researching various academic journals in economics such as The Journal of
Industrial Economics, the Journal of Economic Literature, The American Economic Review, the Journal of
Political Economy, and The RAND Journal of Economics as well as in management, such as Management
Science, Organization Science, and the International Journal of Production Research. Next, articles
published by the eight SME candidates were added to the list. Priority was given to articles published
since 1990 and those related to mobile source industry manufacturing. ICF obtained the selected articles
and examined the reference list of each article for additional articles on manufacturing learning curves
in general and in mobile source manufacturing specifically. The reference lists of the additional articles
were also reviewed and articles were further added to the list. The literature review includes studies
spanning many years, but whenever possible ICF attempted to capture studies published since 1990 in
order to identify recent estimates of learning rates. Lapre and Nembhard (2010) provided a strong
review of the recent literature that was also culled for additional sources.
The initial list was sent to the SME, Dr. Argote, for review. Based upon her extensive knowledge on the
subject matter and the literature she reviewed for the second edition of her book, Organizational
'ICF
3
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Learning: Creating, Retaining and Transferring Knowledge (Argote, 2013), she added 11 studies relevant
to learning curves in manufacturing sectors generally and 16 studies for mobile source industries
specifically.
Dr. Argote provided ICF with the revised list of studies. After consultation with ICF, a list with the
combined search results was provided to EPA for review and approval. The reference list contained 26
studies related to manufacturing sectors generally and 23 studies for mobile source industries
specifically. EPA added several articles to the list based upon their extensive research on the subject.
Based on EPA's feedback, the reference list was divided into four sections: studies of learning curves in
general and studies of learning curves in the mobile source sector, with those two sets of studies
selected for either a detailed review or a cursory review.
The list was further refined as new articles were published. For example, EPA identified an article
published online in Nature on March 23, 2015 (Nykvist & Nilsson, 2015) and Dr. Argote identified an
article forthcoming in Manufacturing & Service Operations Management (Agrawal & Muthulingam,
2015) that were relevant. After discussion with EPA, both articles were added to the reference list.
Furthermore, Dr. Argote consulted other learning curve SMEs about potential studies and thereby
identified two additional studies. After discussions with EPA, one of these studies was added to the
reference list (Levin, 2000). Dr. Argote concluded that this selection of published materials would be
sufficient to support the development of robust observations about learning in mobile source
industries.
Peer reviewers of this report identified three additional articles that should be considered
(Balasubramanian & Lieberman, 2011; Haunschild & Rhee, 2004; and Hendel & Spiegel, 2014). After
discussion with EPA, these articles were added to the reference list as well.
The final reference list consists of eight articles of learning curves in general, selected for a detailed
review. The 21 other articles of learning curves in general were selected for a cursory review. Ten
articles of learning curves in the mobile source sector were selected for a detailed review. The additional
16 articles of learning curves in the mobile source sector were selected for a cursory review. The
overarching criterion for selecting articles for a detailed review was to focus on articles that provided
empirical estimates of learning rates that could be used in future cost estimates for the mobile source
sector. Thus, we focused on studies containing empirical estimates of learning in contemporary
production environments. For example, six of the studies in the reference list for mobile source
manufacturing were based on analyses of Liberty ship production during World War II (Rapping, 1965;
Argote, Beckman, & Epple, 1990; Kim & Seo, 2009; Thompson, 2007; Thompson, 2001; and Thornton &
Thompson, 2001). Although five of these studies were published since 1990, their empirical estimates of
learning were based on historical data from a unique context and thus, are less useful for our purposes
than estimates based on more contemporary data.
There is general agreement across this literature that learning occurs in organizations in general and in
mobile source industries in particular (see Argote, 2013; Balasubramanian & Lieberman, 2010; Dutton
& Thomas, 1984; and Lapre & Nembhard, 2010, for reviews). Thus, the articles in the reference list
\l_
'ICF
4
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
provide strong evidence that learning occurs in firms in the mobile source industry.2
Although the literature supports the development of estimates of the rate of organizational learning in
the mobile source sector, the literature does not support the development of different estimates for
separate mobile source industries.3 Multiple studies of organizational learning have been conducted at
plants in the automotive industry; however, fewer studies have been done on learning in other mobile
source industries. This may be due to the confidential nature of the data that would be necessary to
conduct such a study. Such data are typically viewed as proprietary and are not publically available. It
would be very difficult to obtain permission to combine such proprietary data with those from other
firms and competitors for the purpose of a study of learning for a particular mobile source industry. As
a result, there is not enough information in published studies to support the development of different
rates of learning for separate mobile source industries. However, this may not be important for EPA's
work and there is good reason to believe that a rate of learning estimated at the mobile source
industry level may be applied to the separate sub-industries. Based on a review of the literature on
learning effects across different organizational contexts, Argote (2013) concluded that the biggest
difference in learning rates was between manufacturing and service sectors, with organizations in the
manufacturing sector learning at a faster rate than those in the service sector. In addition, an earlier
review focused on learning curves in manufacturing industries did not find evidence of industry
effects (Dutton & Thomas, 1984). Based on the available evidence to date and because all of the
mobile source industries are in the manufacturing sector, it would be reasonable to use the same
learning rate for different mobile source industries.
2 One of the peer reviewers stated that overall approach to the literature (i.e., identifying studies of learning-by-doing in the
mobile source sector, reviewing them for relevance to the study's goals, and identifying a shorter list of relevant articles)
appears reasonable. The peer reviewer stated that the list of topics included in Section 4 of the report and the coverage of
those topics appears broadly reasonable. See Appendix D for the full comments.
3 One of the peer reviewers stated that the overall conclusion that learning-by-doing occurs in the mobile source sector is well-
founded and largely indisputable. See Appendix D for the full comments.
\l_
'ICF
5
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
'ICF 6 September 30,2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
3, Review of Learning Curves and Progress Ratios and a
Summary of Results arid Recommendations
As discussed in the Introduction, this report aims to meet three objectives: (1) to be a definitive, up-to-
date, reliable, single source of information demonstrating the occurrence of learning in general and in
the mobile source industry specifically; (2) to develop a single compendium study on industrial learning
in the mobile source sector that could be considered for use in future OTAQ costs analyses; and (3) to
estimate a summary effect of learning based on cumulative output in mobile source industries. In
Section 3.1 and 3.2, we provide background information about learning curves and progress ratios,
respectively. In Section 3.3, we provide a summary of the 18 studies on learning in general and in the
mobile source sector specifically that we included in our literature review. In Section 3.4, we discuss the
results of our review and, on the basis of that review provide an estimated progress ratio for the mobile
source industry.
3.1„ What are Learning Curves?
A learning curve represents a fundamental relationship: as a person or organization does more of
something, it gets better at doing it. More specifically, "as organizations produce more of a product, the
unit cost of production typically decreases at a decreasing rate" (Argote, 2013, p. 1). Research in
organizational and manufacturing learning builds on research in psychology, where it was demonstrated
that error rates and time to complete tasks decrease with experience (Argote, 2013).
Learning is an important source of productivity improvements in organizations. Organizations that are
able to learn more from experience enjoy greater productivity and greater prospects of survival than
their counterparts that are less adept at learning (Argote & Ingram, 2000; Baum & Ingram, 1998).
Estimates of learning are used in many applications in organizations, including forecasting production,
purchasing, making delivery commitments, monitoring performance, determining manufacturing
strategy, pricing, and deciding about whether to enter a new market.
Although individuals are the mechanism through which organizations learn, organizational learning
involves more than learning by individuals. In order for learning to be considered organizational, it
should be embedded in a supra-individual repository, such as a routine or process, a database, a
template, or a tool or technology. Thus, organizational learning can be embedded in individual
employees, including managers and engineers as well as direct production workers, in tools and
technologies, and in routines and processes.
Figure 1 shows an example of a learning curve based on data from the start of production of a new
model at a truck plant. Cumulative output, the cumulative number of trucks produced, is plotted on the
horizontal axis. The labor hours required to assemble each truck is plotted on the vertical axis. The figure
illustrates the classic learning curve: labor hours per vehicle decrease at a decreasing rate as experience
is gained in production. While many researchers have focused on labor costs, others have included
additional costs, such as material costs (e.g., Balasubramanian & Lieberman, 2010; Darr, Argote & Epple,
1995) and found that these measures also evidence learning.
'ICF
7
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Intuitively, there is more to learn at the beginning of production. Employees have to learn their
individual tasks and how to coordinate their tasks with others' tasks. Routines are developed. The layout
is improved and tools are modified to improve their performance. Hence, the learning at the beginning
of a production program is steeper than learning later in the production program, where it takes longer
to double cumulative output.
*
Note: Reprinted from Epple, Argote, & Murphy (1996)
Figure 1. Learning Curve for the Truck Plant
The conventional form of a learning curve is a power function:
(Eq. 1)
Where:
Cumulative number of units produced by an organization (i.e., experience
gained) by date t
Vi
Costs required to produce an additional unit at date t
a
Costs required to produce the first unit
b
Parameter that measures the rate unit costs change as cumulative output
increases. If learning occurs, b<0.
t
Time subscript
'ICF
8
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
While learning curves are typically expressed as the relationship between costs per unit and production
volume, other dependent measures have been used including the amount of time it takes to produce a
unit of output, defects per unit, or accidents per unit. The particular dependent measure used depends
on the researcher's purpose. Our focus in this report is on unit costs.
Equation 1 can be rewritten in logarithmic form:
In (yt) = a + b ln(xt) (Eq. 2)
Figure 2 shows the same relationship depicted in Figure 1 in logarithmic form. As can be seen from
Figure 2, when the data are plotted using a log-log scale, the relationship is closer to a straight line.
Note: Reprinted from Epple et al. (1996).
Figure 2. Logarithm of Direct Labor Hours per Vehicle versus Logarithm of Cumulative Hours
The cumulative number of units produced (also referred to as cumulative output or cumulative volume)
measures how much experience the organization has acquired in production. The measure is computed
by adding the number of units produced from the start of production through the end of the previous
time period. If unit costs change as a function of experience, other factors equal, then learning has
occurred.
Other variables that are likely to affect the outcome variable can be added to the equation in order to
control for explanations alternative to learning, such as economies of scale. In addition, one can
investigate whether the rate of learning slows down or plateaus by including a quadratic term for the
cumulative output variable. Including a quadratic function for the experience variable and evaluating it
at values less than the value at which the function reaches a minimum, approximates a function with a
'ICF
9
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
positive asymptote, and thus allows one to investigate whether the rate of learning slows down in
logarithmic form.
Debate has occurred about whether cumulative output or the amount of time that an organization has
produced a product is the better measure of experience to use in investigations of learning. Several
studies that have included both cumulative output and time found that cumulative output was
significant but time was not (Rapping, 1965; Lieberman, 1984). Other studies that have included both
time and cumulative output reported that both were significant but that the magnitude of the
regression coefficient on the cumulative output variable was greater than that on the time variable
(Argote, Epple, Rao, & Murphy, 1997 as cited in Argote, 20134; Bahk & Gort, 1993). Benkard (2000)
included both cumulative output and time as well, and although the fit improved when the time variable
was included, the sign of the time variable was negative; hence, the model was rejected. Levitt, List, and
Syverson (2013) found that the time trend was small in magnitude and only marginally significant when
included in a model with cumulative output. Yet, Levin (2000) found that time was a more important
source of improvement in the quality of cars than cumulative output because the significance of the
cumulative output variable disappeared once year-of-production variables were taken into account. On
balance, researchers have concluded that learning is more related to production activity, as measured
by cumulative output, than to the passage of time.
As described in more detail in the following sections, researchers have also attempted to unpack the
relationship between cumulative output and cost by investigating factors such as organizational
forgetting and knowledge transfer or spillover (i.e., learning from the experience of other organizational
units). Equations 1 and 2 can be generalized to investigate these issues. Results of investigating these
issues are summarized in our literature review in Section 4.
3„?,o What are Progress Ratios?
Organizations often characterize their learning rates in terms of a progress ratio, p, which describes how
the outcome variable changes when cumulative output doubles. For example, the interpretation of an
80% progress ratio is that for every doubling of cumulative output, the outcome variable (e.g., costs per
unit in Equation 1) declines to 80% of its previous value. An 80% progress ratio means that costs decline
by 20%. Thus, lower progress ratios imply faster learning because costs are declining at a faster rate.
A progress ratio, p, can be computed from the learning rate, b, as follows:
yx = Unit cost after producing x1 units
y2 = Unit cost after producing 2x1
Yi= a xi
y2 = a (2Xl)b
4 We referenced the description in the Argote (2013) book because the Argote et al. (1997) article has not been published due
to its use of proprietary information.
'ICF
10
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
P= — =2b (Eq. 3)
Y1
Conversely, the learning rate, b, can be computed from the progress ratio, p:
ln(p) = b In(2)
I n(p)
(Eq4)
In a seminal study often cited in the industrial and manufacturing learning literature, Dutton and
Thomas (1984) examined progress ratios estimated from 108 production programs that covered
manufacturing processes in several industries as reported in 22 field studies (see Section 4.1.1, below).
These authors used only progress ratios that were estimated using either unit costs or average costs as
the outcome variable and cumulative volume as the independent variable and excluded studies
estimating industry-wide estimates. The authors constructed a histogram, reproduced in Figure 3,
illustrating their results. Several conclusions can be drawn from the histogram. First, the rate of learning
varies across organizations. Second, although the rate of learning varies, all but 1 of the 108 production
programs improved with experience. Third, the mode of the progress ratios was between 81% and 82%.
This implies that for every doubling in cumulative output, unit costs decrease to 81% or 82% of their
former value.
\l_
'ICF
11
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
10-
>-
u
z
W
Z3
o
UJ
U.
5 "
« 2 3 i S
^ ft
r- I
? 3 $ s a
at
PROGRESS RATIO
Note: Reprinted from Dutton and Thomas (1984).
Figure 3. Distribution of Progress Ratios Observed in 22 Field Studies (N=108)
3.3. Summary of literature Review
An objective of this report was to provide a definitive, up-to-date, reliable, single source of information
demonstrating the occurrence of learning in general and in the mobile source industry specifically and to
develop a single compendium study on industrial learning in the mobile source sector that could be
considered for use in future OTAQ cost analyses. In addition to providing background and context about
learning, we were particularly interested in identifying empirical estimates of progress ratios in the
literature.
In total, we reviewed 55 articles related to learning, of which 18 articles were reviewed in detail mainly
because they contained empirical estimates of learning in contemporary production environments (see
Section 2). The 18 articles cover several industries in the mobile source sector (e.g., cars, electric
vehicles, trucks, aircraft, and wartime ships) and some outside the mobile source sector (e.g., fast food,
electric power plants, and the manufacturing sector in general). The research method varied among
these articles. Most research was quantitative; however, some authors made valuable insights using
qualitative methods. Similar to Figure 2 above, we found that the rate of learning varies, albeit not as
'ICF
12
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
much as shown in Figure 3. The estimated progress ratios found in the 18 articles ranged from 70% to
98%. These 18 articles represent a range of research conducted over the last 30 years and provide
strong evidence that learning occurs in the mobile source industry and in general.
The articles we reviewed support the claim that learning is a major source of productivity improvements
in organizations. In addition, learning is a source of competitive advantage for firms (Balasubramanian &
Lieberman, 2010). Firms that are able to learn from experience and transfer the knowledge they acquire
throughout their establishments are more productive and more likely to survive than their counterparts
that are less adept at organizational learning. Thus, organizational learning is of great importance to
managers as well as to policy makers. Learning enables organizations to be more productive and
competitive. An understanding of learning enables organizations to perform a host of activities more
effectively, including planning, budgeting, production scheduling, making delivery commitments, and
monitoring performance.
Learning occurs through individuals in organizations. Not only direct production workers but also
managers, engineers, and support staff learn as an organization gains experience in production.
Individuals become better at their particular jobs and also better at coordinating their tasks with those
of other employees. Improvements are discovered in the technology (both hardware and software) and
layout of the plant. Routines and processes are modified to become more efficient and the structure of
the organization is fine-tuned to enable more effective problem solving. Thus, knowledge acquired by
learning by doing in organizations is embedded in individual employees and in the organization's
technology, routines, and structure.
Table 1 presents a summary of 21 articles of which 18 are included with detailed review in this study,
and are subsequently described in the next sections and are summarized in Appendix B. Table 1 provides
the following information:
• Column 1 - Article citation: Column 1 provides the authors' names and the years of publication.
It also lists the sections in which the article is discussed within this report. Several articles (i.e.,
Argote et al., 1990; Argote et al., 1997 as cited in Argote, 2013; and Darr, Argote, & Epple, 1995)
were not selected for detailed review, but were described in articles that were reviewed in
detail or received a cursory review; hence, these articles are not discussed in the sections below.
• Column 2 - Type of Analysis (Qualitative vs. quantitative): The progress ratios presented include
only those from studies that analyzed original data. Therefore, progress ratios are not featured
for the few studies that are solely qualitative reviews or thought pieces (which are highlighted in
the table). If the qualitative analysis mentioned progress ratios that were estimated in other
studies, those progress ratios are listed in the table under their original authors' names. These
studies include Argote et al. (1990); Argote et al. (1997) as cited in Argote (2013); and Darr et al.
(1995).
\l_
'ICF
13
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
• Column 3 - Type of Data (Primary vs. secondary): Column 3 lists whether each study was based
on primary, or secondary data5. For example, the progress ratios in Nykvist and Nilsson's (2015)
study were estimated using estimates from other studies rather than primary data.
• Column 4 and 5 - Type of Industry: Column 4 lists the industry that was the focus of the article
and Column 5 describes whether the industry belongs to the mobile source sector.
• Column 6 - Type of Outcome Variable: Column 6 lists the outcome variable used to estimate the
progress ratio. Many studies use unit costs or a related variable; the number of units produced
which can be expressed in terms of unit costs. Several studies used other outcome variables
such as shipments (see Bahk & Gort, 1993), real value added (see BahK & Gort, 1993;
Balasubramanian & Lieberman, 2010), and price (see Shinoda, Tanaka, Akisawa, & Kashiwagi,
2009). These others measures are not appropriate for the goals of this study: Output measured
by shipments would not be a good measure of productivity if firms keep output in inventory
before shipping. Output measured in terms of dollar value as well as measures of value added
are based on revenues, which are affected by many factors besides learning in manufacturing;
and prices are affected by external conditions. As one of our reviewers noted, using any
measure that embodies price is likely to confound supply-side learning (our focus) with demand-
side changes that might be unrelated to learning. We focus on studies using unit costs, the
number of units produced, or defects per unit because these variables are the most closely
related to costs in mobile source manufacturing. For studies using the number of units produced
as the dependent variable (Argote et al., 1997; Epple et al., 1991; Epple et al., 1996), the models
were re-estimated with costs per unit as the dependent variable.
• Column 7 and 8 - Type of Progress Ratio (Cumulative output i/s best fit): For each study, we list
the progress ratio based on a model using only cumulative output and the progress ratio based
on the model with the best fit according to the adjusted R2 value presented in the study. The
models that use only cumulative output are more comparable across studies and to previous
reviews, such as the Dutton and Thomas (1984) review described earlier in Section 3.2.
Researchers had different goals in the various studies so they included different variables in
their models in addition to cumulative output, depending on the purpose and empirical context
of the study. Many studies had the goal of dissecting the relationship between cumulative
output and cost into different components. Because our goal is to develop a reliable estimate of
the effect of cumulative output, we focus on models that include only cumulative output as a
predictor.
5 We consider primary data to be data collected by a study's researcher directly and secondary data to be data collected by or
produced by a different study.
\l_
'ICF
September 30, 2016
14
-------
Cost Reduction through Learning in
Table 1.
Manufacturing Industries and in the Manufacture of Mobile Sources
Summary of Progress Ratios in Sample
;t
Agrawal & Muthulingam
(2015)
Quantitative
Primary
Car manufacturer
vendors
Y
Defect rate
N/Aa
N/A
See Sections 4.2.5 and 4.4.3
below
Argote (2013)
Qualitative
Secondary
N/A
N
N/A
N/A
N/A
See Section 4.2.3 below
Argote, Beckman, & Epple
(1990)
Quantitative
Secondary
Wartime ships
Y
Current output (i.e.,
tonnage of ships
produced per month)
74%
97%
Argote & Epple (1990)
Qualitative
Secondary
N/A
N
N/A
N/A
N/A
See Section 4.1.2 below
Argote, Epple, Rao, &
Murphy (1997) as cited in
Argote (2013)
Quantitative
Primary
Trucks
Y
Current output
86%b
83%
Bahk & Gort (1993)
Quantitative
Primary
A pool of 15 industries
N
Output measured by
shipments
97%
95%
See Section 4.4.4 below
Motor vehicle parts,
accessories
Y
98%
98%
A pool of 41 industries
N
95%
95%
\i_
'ICF
15
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Balasubramanian &
Quantitative
Primary
U.S manufacturing
N
Current period real
84%
86%
Lieberman (2010)
sector
value added (i.e., real
See Section 4.1.4 below
Motor vehicles and
equipment
Y
revenues less real
materials)
88%
88%
Aircraft and parts
Y
90%
90%
Ship and boat building
Y
93%
93%
and repairing
Railroad equipment
Y
91%
91%
Motorcycles bicycles
Y
89%
89%
and parts
Misc. transportation
Y
94%
94%
equipment
Benkard (2000)
Quantitative
Primary
Aircraft (commercial)
Y
Labor input per unit
82%
65%
See Sections 4.2.2 and 4.3.2
below
Bernstein (1988)
Qualitative
N/A
Automobiles
Y
N/A
N/A
N/A
See Section 4.5.1 below
Darr, Argote, & Epple
Quantitative
Primary
Fast food industry
N
N/A
93%
93%
(1995)
\i_
'ICF
16
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Dutton & Thomas (1984) Qualitative
See Section 4.1.1 below
Secondary
A variety of industries
(e.g., electronics,
machine tools,
papermaking, aircraft,
steel, and
automobiles)
N/A
N/A
N/A
Epple, Argote, & Devadas
(1991)
See Sections 4.3.1 and 4.4.1
below
Quantitative
Primary Trucks
Output during week t
87%'
o/b
35%
Epple, Argote, & Murphy
(1996)
See Section 4.2.1 below
Quantitative
Primary Trucks
Output during week t
86%
o/b
66%
Gopal, Goyal, Netessine,
& Reindorp (2013)
See Section 4.2.4 below
Quantitative
Primary
N/A
N/A
N/A
N/A
Lapre & Nembhard (2010)
See Section 4.1.5 below
Qualitative
Secondary
Manufacturing and
service industries
N/A
N/A
N/A
Lee, Veloso, Hounshell, &
Rubin (2010)
See Section 4.5.4 below
Qualitative and Primary Automobiles;
Quantitative automobile emission
control technologies;
specifically, non-
catalyst components
Cost of non-catalyst
components
93%
93%
\i
'ICF
17
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Levitt, List, & Syverson
Quantitative
Primary Automobiles
Average defect for the
(2013) week
See Sections 4.3.3 and 4.4.2
below
Macher & Mowery (2003) Quantitative Primary Semiconductors N Defect density (i.e., the N/Aa N/A
number of fatal defects
See Section 4.1.3 below per centimeter squared)
Nykvist & Nilsson (2015) Quantitative Secondary Battery electric Y Cost data 91% 91%
vehicles (industry-
See Section 4.5.5 below wide)
Rubin, Taylor, Yeh, & Quantitative Unclear Electric power plants; N Cost to produce the/th 89% 89%
Hounshell (2004) FGD systems unit
See Section 4.5.2 below Electric power plants; N Cost to produce the/th 88% 88%
SCR systems unit
Shinoda, Tanaka, Quantitative Primary Plug-in hybrid electric Y Battery unit price 70% 70%
Akisawa, & Kashiwagi vehicles
(2009)
See Section 4.5.3 below
Notes:
a. The authors did not estimate learning using the power function; hence, their learning rates could not be converted to progress ratios as described in the text.
b. The papers did not provide estimates of costs per unit as a function of cumulative output. Because the SME was a coauthor on these papers, she was able to estimate the
learning rate when the dependent measure was costs per unit and the predictor was cumulative output.
\i_
'ICF
18
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
3.4. Discussion of Mobile* Source iiesults and Recommendations
The report's third goal was to develop a summary effect of learning based on cumulative output in
mobile source industries. We used several criteria to determine the summary effect of learning on costs
in mobile source industries. We relied on studies that were primary analyses of data from firms in the
mobile source sector whose methods were quantitative and statistically sound. We did not include
studies that based their estimates on only a very small number of observations (e.g., Lee et al., 2010;
Nykvist and Nilsson, 2015; and Rubin et al., 2004). Because the focus of our analysis is on manufacturing
costs, we included studies that used unit costs or variables closely related to costs, such as the number
of units produced or defects per unit, as the dependent variable. Finally, while many studies analyzed
data from the production of ships during World War II, we did not use these estimates because of the
studies' unique empirical context (e.g., exceptionally high motivation due to the need to build ships for
the war effort and coordination across firms by the U.S. Maritime Commission).
Note that in its regulatory packages, OTAQ has accounted for learning when estimating the technology
costs for technologies added to mobile sources to allow for compliance with new emission standards.
Our extensive search of the learning curve literature indicates that the literature has not focused on
learning at the individual emission technology level (e.g., learning with respect to the manufacture of
catalytic converters, evaporative control canisters, oxygen sensor, etc.). Instead, published studies
typically examine learning at the final assembly stage of transportation equipment. Because
organizations in the mobile source sector use the same type of labor and processes, assembly at the
final vehicle assembly stage is substantially similar to assembly at the subcomponent level (e.g.,
automobile component assembly). As noted previously, the biggest and most reliable difference in
learning rates was found between the manufacturing and service sectors (Argote, 2013). All firms in the
mobile source industry are in the manufacturing sector. Based on the available evidence as well as the
similarity of firms in the mobile source industry, they would not be expected to differ dramatically in
their learning rates. Thus, we use findings from the final assembly stage to develop recommendations
about learning effects in mobile source industries.
In Table 2, five studies' progress ratios are reproduced (i.e., Argote et al., 1997 reported in Argote, 2013;
Benkard, 2000; Epple, Argote, & Devadas, 1991; Epple et al., 1996; and Levitt et al., 2013). These five
studies met the criteria described in the previous paragraphs and thus form the basis for our
recommendation about learning effects and progress ratios in the mobile source sector.
sl_
'ICF
19
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Table 2. Confidence Intervals of Progress Ratios from Selected Studies3
Author
(Publication Date)
Industry
Progress Ratio
(Cumulative
Output
Approach)
Confidence
Interval
Learning
Coefficient
Standard
Error of
Coefficient
(1)
(2)
(4)
(5)
(6)
(7)
Argote, Epple, Rao,
& Murphy (1997)
as cited in Argote
(2013)
Trucks
86%
(85%, 87%)
-0.221
0.007
See Appendix B
below for a detailed
summary (p. 64)
Benkard (2000)
See Appendix B
below for a detailed
summary (p. 81)
Aircraft
(commercial)
82%
(80%, 84%)
-0.290
0.020
Epple, Argote, &
Devadas (1991)
Trucks
87%
(85%, 90%)
-0.197
0.021
See Appendix B
below for a detailed
summary (p. 90)
Epple, Argote, &
Murphy (1996)
Trucks
86%
(85%, 86%)
-0.226
0.007
See Appendix B
below for a detailed
summary (p. 93)
Levitt, List, &
Syverson (2013)
Automobiles
82%
(81%, 83%)
-0.289
0.007
See Appendix B
below for a detailed
summary (p. 110)
Note:
a. To facilitate comparison across studies, models in studies using output as the dependent variable (Argote et
al., 1997; Epple et al., 1991; Epple et al., 1996), were re-estimated with labor costs per unit as the dependent
variable.
As can be seen from Column 4 in Table 2, estimated progress ratios are very similar: 82% at an
automotive plant (Levitt et al., 2013), 82% at an aircraft assembly plant (Benkard, 2000), 86% at two
different light-duty truck plants (Argote, et al., 1997 as cited in Argote, 2013; Epple et al., 1996)6, and
6 The papers did not provide estimates of the learning rate with just cumulative output as a predictor. Because the SME was a
coauthor on both papers, she was able to compute the learning rate when just cumulative output was included.
\i/
'ICF
20
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
87% at a third light-duty truck plant (Epple et al., 1991). Thus, the estimated progress ratios fall in a
narrow range between 82% and 87%. Based on the learning rates and standard errors provided in the
papers, 95% confidence intervals around these estimates of progress ratios were calculated and are
presented in Table 2.
The estimated progress ratios from these five mobile source studies based on cumulative output are
similar to—but slightly higher than—the Dutton and Thomas (1984) results, which are also based on
cumulative output and where the most frequently observed progress ratio was between 81% and 82%.
For the models just including cumulative output, two production programs in the mobile source
industries had progress ratios of 82%, two had progress ratios of 86% and one had a progress ratio of
87%.
The progress ratios from the best-fitting models (see Column 8 in Table 1) were significantly different for
three of the five studies: 35% at a truck plant (Epple et al., 1991), 64% at an aircraft producer (Benkard,
2000), and 66% at a different truck assembly plant (Epple et al., 1996). The best-fit approach yielded
similar results to the cumulative output approach for two of the studies: 80% at an automotive plant
(Levitt et al., 2013) and 83% at a third truck plant (Argote et al., 1997). These differences can be
explained by the fact that the best-fitting models have more explanatory variables than just cumulative
output, and the additional explanatory variables that were included differ from one study to another,
depending on the goals of the research.
In order to arrive at an estimate of the average progress ratio, we computed a weighted mean, where
the weight assigned to the estimate in each study was the inverse of the study's variance (see
Borenstein, Hedges, Higgins, & Rothstein, 2009). For estimation purposes, we used the coefficients of
cumulative output and their standard errors. These are also shown in Table 2. The weight assigned to
the estimated coefficient from a study is the inverse of the estimated variance of that coefficient. The
weighted mean from a set of studies is obtained by: (1) calculating the sum of the product of the
weights times the estimated coefficients, and (2) dividing the result in (1) by the sum of the weights. The
estimated standard error of the weighted mean is the inverse of the square root of the sum of the
weights (Borenstein, et al., pages 65-66). This approach gives less weight to studies with higher standard
errors.7 Thus, the Benkard (2000) and the Epple et al. (1991) studies receive less weight than the other
three studies.
The mean learning rate is estimated to be -0.245, with a standard error of 0.0039.8 Thus, the lower
bound for a 95% confidence interval for the learning rate is -0.253; the upper bound is -0.238.9 These
estimates translate into a mean progress ratio of 84.3%. The confidence interval around this number
7 One of the peer reviewers commented that the methodology used for estimating the weighted-average progress ratio from
the five studies is broadly reasonable. Given the report's objects, it appeared reasonable to focus only on studies that
examine unit costs, to exclude studies that use a different measure of performance, and to exclude studies of learning-by-
doing in shipbuilding during World War II due to the uniqueness of the context. See Appendix D for the full comments.
8 We estimated the standard error as the square root of the inverse of the sum of the weights.
9 The lower bound of the confidence interval (-0.253) is calculated as the mean (-0.245) minus the margin of error (0.008). The
margin of error is the product of the standard error (0.0039) and the critical value according to a t-distribution (1.96). The
upper bound (-0.238) is calculated as the mean plus the margin of error.
\i_
'ICF
21
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
ranges from 83.9% to 84.8%, suggesting that one can be reasonably confident that the progress ratio
falls in this interval. Thus, the summary effect of the progress ratio in mobile source industries is 84%.
Our estimate of the summary effect is based on the standard approach used in meta-analysis for
combining information across studies (Borenstein et al. 2009). Lieberman, one of our reviewers,
concluded that our estimates are substantially in line with the learning rates Balasubramanian and
Lieberman (2010) found for the mobile source sector (see Balasubramanin & Lieberman, 2011, for
similar results using more fine-grained data).
Further information regarding results and the analysis can be found in Section 5 which contains
responses to peer review comments directed at the analysis and results.
\i_
'ICF
22
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
4. linvSow of learning Curvo literature by lopic
As described in Section 2, ICF and the SME identified 18 studies related to learning in general and
learning in the mobile source sector that would be most relevant to the goals our report which include
being a definitive, up-to-date, reliable, single source of information demonstrating the occurrence of
learning in general and in the mobile source industry specifically. In this section, we provide an overview
of these 18 studies. The summaries are organized by topic, with respect to explanations of the variation
in learning rates that was so clearly illustrated by the Dutton and Thomas (1984) histogram set out in
Section 3.2, above. In addition, in their review of the literature on learning rates, Dutton and Thomas
concluded that there is often more variation across organizations producing the same product than
across organizations producing different products. Argote and Epple (1990) illustrated this variation by
depicting learning curves from three truck plants that differed significantly in their rates of learning.
Similarly, Chew, Bresnahan, and Clark (1990) found dramatic performance differences across plants in
the same firm that produced the same or similar products. These findings underscore that learning is
not automatic and is not determined by the product but rather depends on conditions at the
organization that enable or hinder learning.10 These conditions are now discussed.
The 18 articles examined four aspects of learning variation: sources of that variation (Section 4.1), the
persistence and depreciation of organizational knowledge (Section 4.2), knowledge transfers and
spillovers (Section 4.3), and the location of organizational knowledge (Section 4.4). The last set of
articles provides qualitative descriptions of how learning curves can be applied (Section 4.5).
4.1. Sources of Learning Variation
4.1.1. Dutton & Thomas, 1984
Dutton and Thomas (1984) investigated whether future progress ratios could be predicted and how the
rate of improvement could be managed by identifying which factors cause progress. The authors
performed a secondary analysis of over 200 studies in a variety of industries such as electronics,
machine tools, papermaking, and automobiles, drawn from 50 years of literature.11
The authors found that the rate of improvement was not fixed and could be influenced by managerial
policy decisions. The authors identified four categories of factors related to progress: (1) technological
progress in capital goods, (2) the Horndal-plant effect, (3) local system characteristics, and (4) scale
effects. Technological progress in capital goods describes progress caused by cumulative investments
and improvements in capital equipment. The Horndal-plant effect describes progress that results from
direct learning (i.e., workers' improvement in performing a task); indirect labor learning (e.g., adaptation
10 One of the peer reviewers agreed with the report's interpretation of the literature that heterogeneity in learning rates could
be larger across organizations, even within an industry, than across industries and stated that this was an important point to
highlight. See Appendix D for the full comments.
11 The histogram presented in Section 3.2 features progress ratios from 108 studies, which were estimated in 22 field studies.
Of the 200 studies, the histogram features only progress ratios from 108 studies that were quantitative, estimated
organizational-level progress ratios, used unit cost or average cost as the outcome variable, and used cumulative volume as
the independent variable.
si_
'ICF
23
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
of tooling and process changes made by staff or managers); and other cost-reducing measures such as
scheduling, inventory management, quality control, and wage incentives. The local system
characteristics category includes progress that results from an industry's or firm's operating system
characteristics such as the degree of mechanization, the ratio of assembly to machining work, and the
length of cycle times. Finally, the scale effects category includes progress that results from increases in
the scale of operation.
While three of the categories of factors related to progress (i.e., technological change, labor learning,
and organizational characteristics) continue to be regarded as important predictors of learning, the
fourth category, economies of scale, is now seen as a variable that is distinct from learning and should
be controlled for in empirical analyses. Economies of scale is the relationship between current inputs
and current outputs while learning is the relationship between cumulative experience and current
output.
4.1.2. Argote Si (ripple, 1990
Argote and Epple (1990) also identified potential factors that could affect organizational learning curves.
Similar to Dutton and Thomas (1984), they performed a qualitative analysis of empirical studies focused
on organizational learning curves. The authors identified factors that explain variation observed in
organizational learning rates, especially organizational forgetting or knowledge depreciation and
knowledge transfer.
Knowledge depreciation can be evident following an interruption in production due to factors such as
strikes and input shortages when unit costs are higher than they were before the interruption (see
Section 3.2). Knowledge acquired from learning can depreciate for reasons such as workers forgetting
how to perform tasks, changes in the product or production processes making knowledge obsolete,
workers being replaced by less experienced workers, records being lost, or routines being disrupted.
Additionally, employee turnover can influence rates of learning and forgetting—the extent to which it is
able to do so depends on organizational characteristics. For instance, turnover is more likely to have an
impact in organizations where jobs are not standardized and procedures do not exist for transmitting
knowledge to new employees (Argote, 2013).
Knowledge transfer can also affect the learning rate (see Section 3.3). Knowledge transfer is the process
through which one unit is affected by the experience of another. For example, knowledge transfer can
occur across products, across shifts within a manufacturing facility, or across sister plants that are part
of the same firm. A variety of mechanisms including communication, training, technology, routines, and
personnel movement enable transfer. Organizations that are able to transfer knowledge effectively are
more productive and have lower unit costs than their counterparts that are less adept at knowledge
transfer. Through knowledge transfer, an organization leverages knowledge gained by one unit of the
organization for the benefit of other units.
sl_
'ICF
24
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Finally, the authors point out that when estimating learning rates, one should control for factors
alternative to learning that affect the learning rate. For example, not controlling for economies of scale
can result in an overestimation of the learning rate (see also Balasubramanian & Lieberman, 2010).
4.1.3. Macher & Mowery, 2003
Macher and Mowery (2003) studied learning in the semiconductor industry related to the production of
silicon wafers. The researchers examined how a manufacturer's performance (i.e., its learning rate) was
influenced by human resource (HR) and organizational practices such as: teams for problem solving and
intra-firm knowledge transfer, the use of information technology (IT), and workflow and production
scheduling systems. Macher and Mowery conducted a quantitative regression analysis based on data
from 36 wafer fabrication facilities from U.S., European, and Asian semiconductor firms.
The HR and organizational practices that improved performance included implementing problem-solving
teams, policies that collocated production and key personnel, and the use of information handling
automation and data analysis capabilities. The results showed that introducing HR and organizational
practices initially negatively influenced performance, but the rate of improvement increased as
production expanded. Interestingly, not all of the HR and organizational practices examined resulted in
improved manufacturing performance. The authors concluded that those practices that did so improved
performance by facilitating the organization's internal use of tacit knowledge. Nonaka, Toyama, and
Bossier (2000) described tacit knowledge as "informal and hard-to-pin down know-how, crafts, and
skills" and as "mental models, such as schemata, paradigms, perspectives, beliefs, and viewpoints" (p.
494).
Although the Macher and Mowery (2003) study was not conducted in the mobile source sector, its
implications can still be useful. Results indicate that managers can actively implement strategies to
facilitate learning.
4.1.4. Balasubramanian & Lieberman, 2010
Balasubramanian and Lieberman (2010) estimated the learning rate of over 100 industries in the
manufacturing sector using the U.S. Census Bureau's Longitudinal Research Database and Compustat
data from 1973 to 2000. By performing regression analyses using plant-level data, the authors also
tested whether the learning rate was higher in industries with greater complexity (i.e., industries with
higher capital, research and development (R&D), or advertising intensity) and whether the
heterogeneity of firm performance was higher in industries with faster learning rates.
The results showed that organizations learned faster within industries that had greater capital-labor
ratios as well as greater R&D and advertising intensity. These industries displayed productivity that was
initially low but rose steeply with experience. Thus, Balasubramanian and Lieberman's (2010) article
sheds light on several of the characteristics that explain variation in learning rates (see Dutton &
Thomas, 1984).
sl_
'ICF
25
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Lieberman, one of our reviewers, concluded that our estimates are substantially in line with the learning
rates Balasubramanian and Lieberman (2010) found for the mobile source sector (see Balasubramanin &
Lieberman, 2011, for similar results using more fine-grained data). We should note that the learning
rates in the Balasubramanian and Lieberman (2010) study, and the additional study by the same authors
(Balasubramanian and Lieberman, 2011) were estimated using revenues less materials costs as the
outcome variable, rather than unit cost, which is the focus of our analysis. Thus, we did not include their
results to develop our summary effect. It is reassuring that approaches using different methods and data
yield results consistent with ours.
4.1.5, Lapre & Membhard, 2010
Lapre and Nembhard (2010) performed a secondary analysis of empirical studies related to
organizational learning in manufacturing and service industries to determine why organizational
learning rates vary. The authors distinguished between learning from experience and deliberate learning
(i.e., "planned activities of managers and staff conducted with the explicit intent of acquiring, creating,
and implementing new knowledge" (p. 41)), and found that both were important mechanisms for
learning. Additionally, task and organizational characteristics were found to influence the learning rate.
Lemming from Experience
Based on their review of the literature, the authors suggested that the impact of experience on an
organization's learning rate can depend on whether the experience (1) was homogenous or diversified,
(2) resulted in success or failure, and (3) occurred at the individual, team, or organizational level.
The authors did not find a consensus in the studies examined as to whether more homogeneous tasks,
more diversified tasks, or tasks in the middle of the spectrum fostered a faster learning rate.
Homogenous experience with the same specialized task gives individuals the opportunity to better
understand a specific task and become more proficient at it; however, constantly repeating a task can
lead to stagnation in the learning rate. Performing diverse tasks allows individuals to understand the
bigger picture, but it can be costly to switch between tasks. Several of the studies showed that the best
performance is observed when tasks are similar to each other. That is, performing similar tasks
(moderate task heterogeneity) resulted in better performance than performing identical (low task
heterogeneity) or different (high task heterogeneity) tasks.
The studies examined showed that although organizations learn from both successful and failed
experience, they tend to learn more from failures. Once an organization experiences success, it is likely
to reinforce past tactics and become more risk averse in an effort to preserve the status quo. Following
a failure, an organization is more likely to critically review its past tactics and innovate new ways to
improve its performance. The way an organization responds depends on four factors: (1) the nature of
the success or failure, (2) the level of each experience and the presence of other experiences, (3) the
aspiration level and (4) the context. The authors suggest there are four reasons for paying more
attention to failures than success. First, outcomes with various causes are more complex to analyze than
outcomes with a clear cause and therefore organizations tend to devote more resources to
sl_
'ICF
26
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
understanding and addressing them. Secondly, an organization is more likely to learn from either
success or failure when the outcome surpasses a certain threshold. In addition, an organization will
learn more from a success if it has had a related failure in the past. Thirdly, an organization is likely to
learn from its own experiences if it succeeds but is more likely to learn from other organizations'
experience if it fails. Finally, an organization is less likely to learn from failure if their competitors are
also failing or if they have an historical investment in a strategy.
The studies examined by these authors suggest that experience results in learning at every level of the
organization: individual, team, and overall organization. At the individual level, individuals develop skills
and knowledge; at the team level, individuals learn how to coordinate and use each team member's skill
the most efficiently; and at the organizational level, individuals learn from the knowledge accumulated
by others. Reagans, Argote, and Brooks (2005) found that experience at the team and organizational
level had a positive relationship with performance while individual experience had a U-shaped
relationship with performance. At very low levels of experience, increases in experience hurt
performance, while at high levels, increases in experience improved performance. At very low levels of
experience, individuals might not apply the knowledge gained from previous experiences correctly, but
this rectifies itself as the individuals accumulate more experience.
Deliberate Learning
With respect to deliberate learning, Lapre and Nembhard (2010) found that variations in the learning
rate depend on: (1) the types of deliberate learning activities (DLAs) used and (2) contextual differences.
Types of DLAs include activities such as training, experiments, and quality management programs.
Overall, it appears that learning rates are faster in organizations that use more types of DLAs than those
that use fewer. Learning rates are also faster in organizations that use DLAs that contribute to their
know-how and know-why.
In terms of contextual differences, a DLA's impact on the organizational learning rate depends on who is
involved, their level of investment, where and when the DLA occurs, and why it has been pursued. Lapre
and Nembhard (2010) found that a DLA impacts the learning rate the most when individuals at all levels
of the organization (i.e., management, team leaders, and staff) actively support the chosen DLA, when
the DLA is used in multiple locations within the organization, when there is enough time available to
reflect on the knowledge gained from the DLA, and when the intention of the DLA is to improve quality
rather than efficiency.
Task-based Learning
Task and organizational characteristics have also been found to affect the learning rate. Task
characteristics focus on the knowledge required to complete a task with characteristics such as
complexity, observability, and causal ambiguity. Tacitness was the task characteristic most focused on in
this type of research. As explained above in Section 4.1.3, tacit knowledge is know-how that is difficult
to articulate, while explicit knowledge is formalized and easily articulated. The authors found that the
variation in the learning rates is related to the proportion of tacit-to-explicit knowledge in a task. Tasks
sl_
'ICF
27
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
with a higher proportion of tacit knowledge tend to have different learning rates due to the difficulty of
learning tasks with little guidance, while tasks with higher proportions of explicit knowledge tend to
have similar learning rates. In addition, knowledge gained from tasks that have complex, unproven, or
causally ambiguous characteristics is also more difficult to transfer than knowledge from tasks that are
less complex and better understood.
Organization Level Learning
Organizational characteristics include elements such as the internal structure (i.e., vertically structured
organizations versus less interdependent organizations), organizational capacity, staffing, and
expectations and incentives. The impact of internal structure on the learning rate depends on the
business environment. In stable environments, vertically integrated organizations learn at a faster rate
than less interdependent organizations; however, the opposite is true in volatile environments. Two
types of organizational capacity have been found to increase learning rates. Organizations with more
resource-based capacity (e.g., organizations that have more slack time) learn at faster rates because
staff have more resources that assist in learning. Organizations with more absorptive capacity—the
"ability to recognize the value of new, external information, assimilate it, and apply it to commercial
ends" (Cohen & Levinthal, 1990, p. 128)—are able to learn at faster rates because these organizations
are better equipped to use new knowledge based on knowledge gained in their past experiences.
Related to staffing, organizations with a higher percentage of temporary workers and more diverse
teams tend to learn faster than those with a lower percentage of temporary workers and less diverse
teams because these organizations can innovate better. Finally, organizations tend to base their
incentive structures around their expectations of future performance. Because organizations have
different expectations and subsequently have different incentive structures, workers engage in different
types of learning activities (e.g., R&D); hence, different learning rates result.
4.1.6. Conclusion
Learning rates are not fixed and these five articles highlight several causes for variation in learning
rates—several of which can be influenced by managerial policy decisions. Sources of variation include,
but are not limited to, technological improvements, organizational practices, organizational
characteristics, and the type of learning in which an organization engages.12
4.2. Knowledge Persistence and Depreciation
The conventional learning curve model shown in Equation 1 assumes that knowledge gained from
learning by doing is cumulative and persists indefinitely over time. More recent research suggests that
knowledge acquired from learning might not persist indefinitely in organizations (Argote et al., 1990;
Darr et al., 1995). Instead, knowledge could depreciate due to factors such as turnover, interruptions in
12 ICF also reviewed a study by Laitner and Stanstad (2004) who investigated the relationship between demand-side learning
(i.e., "learning by using") and the cost of energy technologies. The authors found that demand-side learning could affect
costs estimates and concluded that researchers should include learning on both the demand and supply side in their models
to avoid biased results and forecasts.
si_
'ICF
28
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
production that lead to individuals forgetting how to perform tasks, disruptions in routines or changes in
products and processes that render previous knowledge obsolete. Thus, while individual forgetting
could contribute to knowledge depreciation in organizations, knowledge depreciation is caused by more
factors than just individual forgetting.
The presence of knowledge depreciation does not negate the presence of learning curves. Just as
individuals can continue to learn while they forget some material, organizations can also continue to
learn while their stock of knowledge might depreciate.
Knowledge depreciation has typically been assessed by determining whether recent production
experience is more important than earlier production experience in predicting current unit costs. The
extent of knowledge depreciation is measured by estimating a parameter that determines the geometric
weight past output receives in predicting current performance. If the parameter does not differ from
one, there is no evidence of depreciation. A parameter less than one provides evidence of depreciation
because it implies that past output receives less weight than recent output. That is, current performance
is more driven by knowledge acquired recently than by knowledge acquired in the more distant past.
The following five studies on knowledge depreciation were examined, each of which pertain to the
mobile source sector.1314
4.2.1. Epple, Argote, & Murphy, 1996
Epple et al. (1996) analyzed intra-plant knowledge transfer and knowledge depreciation in an
automotive assembly plant that operated for 2 years with one-shift operation before adding an
additional shift. This plant, a sister plant to the plant studied in Epple et al. (1991), used a different
technology and introduced the second shift much later than the plant initially studied. In addition, more
fine-grained data were available for this plant. Using 12 months of daily data related to the plant's one-
shift operation and 15 months of data following their switch to two-shift operation, the authors
investigated whether and how knowledge transferred between the first and second shift following the
introduction of the second shift as well as between the two shifts during two-shift operation and
whether knowledge acquired through learning by doing depreciated over time.
This study showed that knowledge depreciated in this mobile source plant. The estimated depreciation
parameter, based on daily data was approximately ranged from .979 to .988, which implies that 0.6% -
5.5 % of the knowledge available at the beginning on one year would be available at the beginning of
13 In order to estimate depreciation, researchers typically estimate a parameter that is the geometric weight that past output
receives in predicting current production. This depreciation parameter represents the percentage of the knowledge stock
acquired in one period that would carry over to the following period. Thus, the depreciation parameter can be thought of as
providing an indicator of how much knowledge is retained from one period to the next. For example, a parameter estimated
to be .98 would imply that 98% of the knowledge acquired in the previous period carried over to the current period. The
period chosen for each study depended on the frequency of the data available (e.g., daily, weekly, or monthly). To facilitate
comparisons of the estimated depreciation parameters between the studies, we converted all of the estimated depreciation
parameters to an annual basis in the report. We describe the estimated depreciation parameters in the footnotes and in
Column 3 of Table 3.
14 A peer reviewer stated that Section 4.2 is a good characterization of studies about learning that have considered knowledge
depreciation. See Appendix D for full comments.
si_
'ICF
29
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
the next. Of course, because the organization continued production, it generated new knowledge.15
Rates of learning remained significant (see Table 1) when depreciation was taken into account.
4.2.2. Benka"/ ;X)0
Benkard (2000) analyzed two groups of similar commercial aircraft models that consisted of four
Lockheed L-1011 TriStar models and tested whether learning by doing, knowledge spillovers (see Section
4.3.2), or knowledge depreciation occurred during the production of the 250 units of aircraft between
1970 and 1984. Benkard conducted an empirical analysis by generalizing the traditional learning curve to
allow for knowledge spillovers and knowledge depreciation.
Using monthly data, Benkard estimated annual depreciation parameters to be between .55 and .61,
which implies that 55%-61% of the firm's experience that existed at the beginning of the year was
available at the end of the year.16 Benkard (2000) concluded that this was a relatively high rate of
depreciation, which he attributed to characteristics of the aircraft industry which include low production
rates, high labor turnover, and displacement rights which allow employees to request a higher position
if one becomes available and can cause employee movement within the firm. Other industries in the
mobile source sector that do not share these characteristics could experience less knowledge
depreciation. Argote (2013) pointed out that Benkard's results showed that knowledge depreciation
occurred despite incomplete knowledge transfers across products, which implies that knowledge
depreciation was not solely caused by product changes that made previously gained knowledge
obsolete.
4.2.3. Argote, 2013
Argote (2013) performed a secondary analysis of empirical studies on mobile source industries such as
aircraft, ships, and automobiles as well as unrelated industries such as fast food franchises to determine
whether organizational knowledge gained through learning by doing persisted or depreciated over time,
the causes of knowledge depreciation, and whether turnover of key personnel affected organizational
performance.
First, Argote (2013) reviewed the Lockheed L-1011 TriStar aircraft case study. Benkard (2000) analyzed
the Lockheed data and found that knowledge depreciation occurred (see Section 4.3.2). Using monthly
data, Benkard estimated the annual depreciation parameter to be between .55 and .61.16 Argote also
reviewed the empirical study conducted by Argote et al. (1990) regarding the production of Liberty ships
during World War II, which found that knowledge depreciated rapidly with the annual depreciation
15 The Epple et al. (1996) study estimated a daily depreciation parameter that ranged from 0.979 to 0.988. As calculated in
Column 5 of Table 3, to convert the daily depreciation parameter to an annual basis, we raised the estimated parameter by
240 (i.e., the number of work days in a year). This finding indicates that, absent current production to replenish the
knowledge stock, approximately 0.6% - 5.5% of the knowledge available at the beginning of one year would be available at
the beginning of the next year.
16 The Benkard (2000) study estimated a monthly depreciation parameter that ranged from 0.952 to 0.960. As calculated in
Column 5 of Table 3, to convert the monthly depreciation parameter to an annual basis, we raised the estimated parameter
by 12 (i.e., the number of months in a year). This result indicates that approximately 55% to 61% of the knowledge available
at the beginning of one year would be available at the beginning of the next year.
si_
'ICF
30
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
parameter estimated to be between .01 and .14.17 Argote then reviewed a study by Argote et al. (1997)
conducted in an automobile assembly plant, which found less depreciation. Based on the monthly data,
an annual depreciation parameter was estimated to be .56, which implied that 56% of the knowledge
available at the beginning of one year would be available at the beginning of the next.18 Further, there
was evidence that organizational knowledge had both a permanent and a transitory component. The
permanent component was attributed to procedural knowledge that is embedded in an organization's
technologies or routines. For more discussion on the location of knowledge within an organization, see
Section 3.4. Finally, Argote examined a study by Darr et al. (1995) on fast food franchises. Darr et al.
estimated that only 0.001% to 0.01% of the knowledge stock available at the beginning of a year would
remain at the end of the year, which is a very rapid rate of depreciation.19 Along with the industry's high
turnover rate, this depreciation rate could be due to its low level of technological sophistication.
Argote (2013) noted that debate has occurred in the literature about how much depreciation occurred
in the production of Liberty ships during World War II. Argote et al. (1990) were the first to investigate
knowledge depreciation and reported rapid knowledge depreciation, which suggested that between 1
and 14 % of the knowledge available at the beginning of a year would be available one year later.
Thompson (2007) obtained additional data about Liberty ships from the National Archives and also
found evidence that knowledge depreciated, albeit at a slower rate than Argote et al. (1990). His
estimates suggested that between 49 % and 64 % of the knowledge available at the beginning of one
year would be available at the beginning of the next.20 Kim and Seo (2009) analyzed data from the
shipyard that produced the largest number of Liberty ships. Using a different model than Argote et al.,
they found a similar estimate of the depreciation parameter, which implied that approximately 3% of
the knowledge available at the start of one year would be available at the beginning of the next.21 Thus,
while all three studies of Liberty ship production found evidence of the depreciation, estimated amounts
are sensitive to model specifications and variables included.
17 The Argote et al. (1990) study estimated a monthly depreciation parameter that ranged from 0.70 to 0.85. As calculated in
Column 5 of Table 3, to convert the monthly depreciation parameter to an annual basis, we raised the estimated parameter
by 12 (i.e., the number of months in a year). This result implies that approximately 1%-14% of the knowledge available at the
beginning of a year would be available one year later.
18 The Argote et al. (1997) study as cited in Argote (2013) estimated a weekly depreciation parameter of 0.989. As calculated in
Column 5 of Table 3, to convert the weekly depreciation parameter to an annual basis, we raised the estimated parameter by
52 (i.e., the number of weeks in a year). This result implies that approximately 56% of the knowledge available at the
beginning of a year would be retained one year later.
19 The Darr et al. (1995) estimated a weekly depreciation parameter that ranged from 0.80 to 0.83. As calculated in Column 5
of Table 3, to convert the weekly depreciation parameter to an annual basis, we raised the estimated parameter by 52 (i.e.,
the number of weeks in a year). This results implies that only a negligible amount (approximately 0.01%) of knowledge
available at the beginning of a year would be retained one year later.
20 The Thompson (2007) study estimated a monthly depreciation parameter that ranged from 0.943 to 0.964. As calculated in
Column 5 of Table 3, to convert the monthly depreciation parameter to an annual basis, we raised the estimated parameter
by 12 (i.e., the number of months in a year). This result implies that approximately 49% to 64% of the knowledge available at
the beginning of a year would be available one year later.
21 The Kim and Seo (2009) study estimated a monthly depreciation parameter that ranged from 0.7379 to 0.7410. As calculated
in Column 5 of Table 3, to convert the monthly depreciation parameter to an annual basis, we raised the estimated
parameter by 12 (i.e., the number of months in a year). This result implies that approximately 3% of the knowledge available
at the beginning of the year would be retained one year later.
\i_
'ICF
31
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
4.2.4. Gopal, C>oyal, Netessine, & Reindorp, 2013
Gopal, Goyal, Netessine, and Reindorp (2013) analyzed product launches in the North American
automotive industry. Using the Harbour Reports, which contained production and launch data from
1999 to 2007 on 78 plants owned by former Daimler-Chrysler, Ford, GM, and Toyota, the authors
evaluated how product launches affected a plant's productivity, and how any decreases in productivity
resulting from the disruption caused by launches could be mitigated. The authors examined the 3 years
prior to each launch and evaluated three types of experiences: (1) platform experience, the number of
vehicles produced on the same platform as the launch product; (2) launch experience, the number of
launches at the plant; and (3) firm experience, the number of launches within the firm. The authors
tested whether the plant learned from these three types of experience and whether knowledge gained
from these types of experience persisted over time.
Gopal et al. (2013) found that knowledge acquired from platform experience and knowledge acquired
from past launch experience at the plant mitigated reductions in plant productivity during a new
product launch. Further, knowledge acquired from platform experience tended to persist for 3 years
while knowledge acquired through launch experience depreciated faster. The authors attributed the
difference in persistence between platform and launch experience to the fact that while platform
experience was consistently acquired over time, launches only occurred sporadically; hence, knowledge
gained by launches was likely not reinforced or ingrained in routines.
4.2.5. Agrawal & Muthulingam, 2015
Agrawal and Muthulingam (2015) analyzed data from 295 vendors of a large car manufacturer in Asia
with the aim of determining how knowledge depreciation affected the vendors' quality performance.
The authors distinguished between two types of learning, learning by doing (autonomous learning) and
quality improvement initiatives (induced learning). The authors analyzed data on 2,732 quality
improvement initiatives implemented by the vendors between 2006 and 2009 using regression. To
discern which factors influenced the rate of knowledge depreciation, the authors further examined the
type of initiative and where the knowledge was located within an organization.
The authors found that knowledge depreciation affected quality gains obtained from learning by doing
and quality improvement initiatives. Specifically, 16% and 13% of quality gains from learning by doing
and quality improvement initiatives depreciated every year, respectively. These depreciation rates are
lower than those observed in several other studies, which Agrawal and Muthulingam (2015) attributed
to the low turnover rate during the study period and to the outcome variable used. Instead of unit costs,
the authors used the defect rate, a measure of quality, which can be easier to document and track than
cost measures. Additionally, quality problems are salient and are often addressed, which can contribute
to a higher retention of knowledge.
The authors identified whether each of the 2,732 quality improvement initiatives primarily focused on
(1) quality assurance, (2) process improvement, or (3) design quality. Quality gains from quality
assurance initiatives did not depreciate; however, quality gains from process improvement initiatives
sl_
'ICF
32
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
depreciated by more than 14% per year (the authors did not analyze if quality gains from design quality
initiatives depreciated, as the organizational learning estimates related to these initiatives were not
significant). Agrawal and Muthulingam (2015) attributed these results to differences in how the
initiatives addressed problems. Quality assurance initiatives often included solving the problem directly
and making changes to test equipment. Hence, knowledge became embedded in technology. However,
process improvement initiatives did not always solve the problem. The authors then evaluated whether
the rate of depreciation depended on where the knowledge was embedded within the organization (see
Section 4.4.3). They examined three locations: technology, routines, and organizational members (i.e.,
workers). The results showed that knowledge depreciated faster when it was embedded in individuals
(26%), followed by routines (14%), and technology (9%).
4.2.6. Conclusion
Column 3 of Table 3 presents the depreciation parameter estimates found in 10 articles of the 18
articles that received a detailed review and the 15 articles related to the mobile source sector that
received a cursory review. Column 5 presents the percentage of the knowledge stock held at the
beginning of the year that would survive to the end of the year, if the knowledge stock were not
replenished by production. It is important to note that most organizations continue production and thus
replenish their knowledge stock. Estimated values of the depreciation parameter indicate how much
knowledge is retained from one period to the next. Note that these estimates depend on specification
of the model and the variables used.
Table 3. Summary of Depreciation Parameter Estimates
Author
(Publication Date)
Industry
Depreciation
Parameter Estimates
Data Frequency
Percent of Knowledge
Remaining from One
Year Ago
(1)
(2)
(3)
(4)
(5)
Agrawal &
Automobiles -
.9852-9866
Monthly
84%-85%
Muthulingam
Autonomous learning
(2015)
(=.985212)-
(=.986612)
Automobiles -
,9752-.9994a
Monthly
74%-99%
Induced Learning
(=.975212)-
(=.999412)
Argote, Beckman,
Liberty ships
.70-85
Monthly
1%-14%
Epple (1990)
(=.7012)-(=.8512)
Argote, Epple, Rao,
Automobiles
.989
Weekly
56%
& Murphy (1997)
as cited in Argote
(=.98952)
(2013)
\i/
'ICF
33
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Author
(Publication Date)
Industry
Depreciation
Parameter Estimates
Data Frequency
Percent of Knowledge
Remaining from One
Year Ago
Benkard (2000)
Aircraft (commercial)
.952-960
Monthly
55%-61%
(=.95212)-(=.96012)
Darr, Argote, &
Epple (1995)
Fast food franchise
m
00
f
o
00
Weekly
0.001%-0.01%
(=.8052)-(=.8352)
Epple, Argote, &
Devadas (1991)
Trucks
,99b
Weekly
59%
(=.9952)
Epple, Argote, &
Murphy (1996)
Automobiles
.979-988
Daily
0.6%-5.5%
(=.979240)-(=.988240)c
Gopal, Goyal,
Netessine, &
Reindorp (2013)
Trucks
N/A
Kim & Seo (2009)
Liberty Ships
.7379-7410
Monthly
2.6%-2.7%
(=.737912)-
(=.741012)
Levitt, List, &
Syverson (2013)
Automobiles
,927-.965d
Weekly
2%-16%
(=.92 752)-
(=.96 552)
Thompson (2007)
Liberty Ships
.943-964
Monthly
49%-64%
(=.94312)-
(=.96412)
Notes:
a. These values include depreciation parameters estimated from induced learning in general; learning from quality
assurance, process improvement, and design quality initiatives; and learning from technology, routines, and
operator solutions.
b. This depreciation parameter was not significantly different from 1—the case of no depreciation.
c. There are 20 work days in a month and 240 work days in a year.
d. The .927 value is the implied weekly depreciation parameter based on daily data which are compounded over a 5-
day production week.
Current thinking on knowledge depreciation has focused on understanding the causes of depreciation
(e.g., see the Agrawal & Muthulingam, 2015). Researchers acknowledge that the extent of depreciation
can vary and they aim to understand the causes of the variation. On balance, knowledge appears to
depreciate most rapidly in organizations where there is high turnover (see Darr et al., 1995), when rates
of production are uneven or interrupted (see Benkard, 2000; Gopal et al., 2013), and when knowledge is
embedded primarily in individuals rather than in routines or technology (see Argote, 2013; Agrawal &
Muthulingam, 2015). Because organizations in mobile source industries tend to produce at a relatively
even rate, embed a significant portion of the knowledge in technology and routines and do not
\i/
'ICF
34
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
experience turnover rates near the rates that Darr et al. reported in their study of fast food franchises,
we do not expect knowledge depreciation to be large in mobile source industries.
It is important to note that learning continues to occur even though some of the knowledge acquired via
learning by doing might depreciate. Just as individuals can forget some things while they continue to
learn others, learning and "forgetting" (i.e., knowledge deprecation) can co-occur in organizations.
Studies finding evidence of deprecation reviewed above continue to find evidence of learning.
4.3. Knowledge Transfer and Spillovers
Knowledge transfer is the process through which one organizational unit is affected by or learns from
the experience of another unit. For example, a second shift introduced at a manufacturing plant might
benefit from or learn from experience acquired on the first shift (see Epple et al., 1991) or the
manufacture of a new model of a product might benefit from experience acquired producing the initial
model (Benkard, 2000). Knowledge transfer has been studied within and between plants. The concept of
knowledge spillover is identical to the concept of knowledge transfer. Economists tend to use the term
"spillover" while management researchers generally use the term "transfer." Knowing whether intra-
plant transfers occur is useful in identifying sources of learning within firms. Furthermore, analyzing
intra-plant transfers allows researchers to determine where knowledge is embedded within
organizations. For examples of studies doing so, refer to Section 3.4. Three studies focused on
knowledge transfer in mobile source industries.22,23 (Note, that these studies do not address all of the
components of knowledge transfer (e.g., inter-firm spillover) because distinguishing the separate
components of learning is not an objective of this report.)
4.3.1. Epple, Argote, & Devadas, 1931
Epple et al. (1991) analyzed intra-plant knowledge transfer between shifts in a North American truck
plant. The plant operated with one shift for 19 weeks and then added a second shift. Specifically, the
authors assessed the knowledge transfer that occurred between the first and second shift when the
second shift was introduced as well as the ongoing transfer between the first (day) and second (night)
shifts during two-shift operation. Eighty weeks of data were analyzed from the period after the second
shift was introduced. The authors extended the conventional learning curve model by allowing for
knowledge depreciation, a changing learning rate, and intra-plant knowledge transfer.
Significant but incomplete transfer of knowledge occurred from the first to the second shift when it was
introduced. Results indicate that 69% of the knowledge acquired during the period of operating with
one shift transferred to the period of operating with two shifts. Once both shifts were operating, about
half (56%) of knowledge acquired on one shift transferred to the other shift. The authors compared their
22ICF also reviewed a study by Thornton and Thompson (2001) who analyzed knowledge spillovers across shipyards in the
production of Liberty ships produced during World War II. The authors found that knowledge spillovers had a significant
impact on increasing productivity.
23 One of the peer reviewers commented that Section 4.3 is effective in describing research findings relating to knowledge
transfer across organizational units (e.g., additional shifts and new models) within a given firm. See Appendix D for full
comments.
si_
'ICF
35
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
results to those of Argote et al. (1990) who investigated knowledge transfers between shipyards
producing ships during World War II. Similar to the results of Epple et al. about the introduction of a
second shift, Argote et al. found that when shipyards began production, they benefited from the
experience of shipyards with earlier start dates. In contrast to the Epple et al. results on cross-shift
transfer, Argote et al. did not find evidence of ongoing knowledge transfer between shipyards once they
were in operation.
4.3.2. Benkard, 2IMMJ
Benkard (2000) analyzed the extent of knowledge spillover or transfer during the production of two
groups of similar commercial airline models (specifically, four Lockheed L-1011 TriStar models) between
1970 and 1984. Benkard's empirical analysis generalized the traditional learning curve by allowing for
organizational forgetting and knowledge spillover. His analysis took into account that experience gained
from working on one group of airline models might differ from experience gained from working on the
second group.
Benkard found that when production was switched to a new model, approximately 70% of the
knowledge transferred. Hence, there was considerable but incomplete knowledge transfer to a new
model. The author interpreted 70% as being relatively low given that the two groups of models were
similar and produced at the same plant. The results showing incomplete knowledge transfers led
Benkard to conclude that (1) introducing a new model can cause production costs to increase and (2)
producing multiple models simultaneously can cause production costs to be higher than if only one
model were produced.
Caution should be used when comparing the amount of knowledge transfer in the aerospace industry to
other mobile source industries due to the nature of commercial aircraft production, which involves
labor-intensive production processes, low annual output, high entry costs and imperfect competition.
Other industries that do not share these characteristics might exhibit different patterns related to the
extent of knowledge spillover.
4.33. Hewitt, List, & Syverson, 2013
Levitt et al. (2013) analyzed learning and knowledge spillovers at an automobile assembly plant. The
authors used production data, absenteeism records, and warranty claims to conduct quantitative
analyses to estimate learning's impact on defect rates. Similar to Epple et al. (1991), the authors
analyzed knowledge transfer across shifts in an automotive plant. Similar to Benkard (2000), Levitt et al.
analyzed knowledge transfer across different product models. Unlike Epple et al. and Benkard, Levitt et
al. analyzed quality improvements (i.e., reductions in the average defect rate) rather than unit costs as
their outcome variable.
While analyzing knowledge transfers between the first and second shifts, Levitt et al. (2013) found
evidence of knowledge transfer: from the outset, the defect rates in the second shift were lower than
they were during the first shift. On average, the defect rates in the second shift were 5% to 10% lower
sl_
'ICF
36
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
than those in the first shift. Furthermore, defect rates in the first shift increased during the second
shift's ramp-up period.
The authors also tested for knowledge transfer between three models. Defects in Model 1 increased
during the ramp-up period of Models 2 and 3; however, defects in Model 2 were not significantly related
to the ramp-up period of Model 3. The authors concluded that because Model 3 was a specialized
version of Model 1, more resources were taken away from Model l's production for problem solving
during Model 3's ramp-up period. The results of Levitt et al. (2013) on cross product transfer are
consistent with Benkard's (2000), who found that the addition of a new model can negatively impact the
production of existing models.
Finally, Levitt et al. (2013) found spillovers between cars produced sequentially on an assembly line. A
defect on a car significantly increased the likelihood of defects on the next 15 cars, although the
magnitude of the defects decreased the further the cars are from each other.
43JL Conclusion
These three studies found evidence of knowledge transfers between shifts and product models. Epple et
al. (1991) found that 69% of the knowledge acquired during the period of operating with one shift at a
truck plant transferred to the period of operating with two shifts. Once both shifts were operating, 56%
of knowledge acquired on one shift transferred to the other shift. Benkard (2000) found that
approximately 70% of knowledge transferred when production of a commercial aircraft was switched to
a new model. Levitt et al. (2013) found evidence of positive knowledge transfers at an automobile plant
between the first and second shifts and evidence of negative transfer when new models were
introduced (i.e., the addition of two new models harmed production of the initial model).
4A Location off Oirga iritafcfional Knowledge
Understanding the learning process requires, along with topics discussed in other sections, a
comprehension of where knowledge is embedded in organizations. Knowledge can be embedded in
individual employees, in tools and physical capital, or in routines and procedures for accomplishing
tasks. Knowing where knowledge is embedded can assist managers in choosing a production strategy
that would increase production rates and thereby decrease per unit costs. There is agreement across
the following three articles, which found that organizational knowledge resides in multiple locations. In
addition, Agrawal and Muthulingam (2015) found that organizational knowledge is not equally retained
within each location.24 Finally, the fourth article by Bahk and Gort (1993) found that the conventional
learning curve could be expanded upon by decomposing learning by doing into different types of
learning.
24 The study conducted by Epple et al. (1996) also analyzes the location of knowledge (see Section 4.2.1). The study is not
summarized here, but its results are consistent with the three articles presented in this section.
si_
'ICF
37
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
4.4.1. Epple, Argote, & Devadas, 1991
Using 2 years of weekly data from a North American truck plant, Epple et al. (1991) analyzed what
proportion of knowledge acquired during 19 weeks of one-shift operation was carried forward as the
plant transitioned to two-shift operation as well as what proportion of knowledge was transferred
between the day and night shifts during two-shift operation. Workers on both shifts used the same
tooling and physical capital. Workers on the second shift were new to the organization, having recently
been hired to work on the second shift.
The results indicated considerable but incomplete knowledge transfers between one- and two-shift
operations as well as between day and night shifts (see Section 4.3.1). Because the same equipment and
physical facilities were used on both shifts, the authors attributed a significant amount of the knowledge
transfer to knowledge being embedded in the organization's technology, which includes plant layout,
equipment, and computer software. That is, as the first shift gained experience in production it made
improvements in the tooling and technology. Because the second shift used the same technology as the
first shift, it benefited from knowledge embedded in the technology by the first shift.
4.4.2,, Levitt, List, & Syverson, 2013
Similar to Epple et al. (1991), Levitt et al. (2013) aimed to discern the location of organizational
knowledge by analyzing knowledge transfer between shifts at an automotive assembly plant. Using one
year of production, absenteeism, and warranty claims data from an automotive assembly plant that
transitioned from one- to two-shift production, the authors evaluated the relationship between
production experience and defect rates and between absenteeism and defect rates. While the Epple et
al. study analyzed knowledge transfers at a plant producing one vehicle model, Levitt et al. examined
transfers at a plant producing three models (see Section 4.3.3).
Similar to the results of Epple et al. (1991), Levitt et al.'s (2013) results indicate that a significant amount
of organizational knowledge was embedded in the broader organization or physical capital, rather than
in the workers. The following findings led them to this conclusion: (1) experience gained during first-shift
operation appeared to be fully incorporated in the second-shift operation, despite the fact that new
workers were employed on the second shift;25 (2) workers were not able to fully transfer their
production knowledge from one model to new models; (3) the distribution of defects among stations
was similar between day and night shifts, although the workers were different; and (4) although
absenteeism varied significantly over the analysis period, the defect rate only experienced minor
changes.
We should note that due to the nature of their data set, Levitt et al. (2013) focused on how learning
affects defect rates, unlike Epple et al.'s (1991) study, which focused on how learning affects unit costs.
25 Epple et al. (1996) found a similar result while studying knowledge transfers at an automotive assembly plant and drew a
similar conclusion that knowledge appears to be embodied in the broader organization rather than the human capital of
workers.
si_
'ICF
38
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Analyzing learning related to quality measures could have implications for learning related to unit costs.
Correcting defects identified at the plant typically causes costs to increase.
4.4.3. Agrawal & Muthulingam, 2015
Agrawal and Muthulingam (2015) evaluated how organizational learning and knowledge depreciation
affect the quality performance of car manufacturer vendors. The authors focus on quality improvement
initiatives, which they refer to as "induced learning." The authors used data from a large automotive
manufacturer in Asia, which included 2,732 quality improvements initiatives implemented by the car
manufacturer's 295 vendors between 2006 and 2009. The authors categorized the initiatives as focused
primarily on technology (e.g., new equipment), routines, or operators (e.g., training). The authors
conducted an empirical analysis, using regression to analyze the relationship between the stock of
induced knowledge related to quality improvement projects with technology, routines, or operator
solutions and the defect rate.
The authors found that the rate of knowledge depreciation depends on where knowledge is located.
Knowledge embedded in operators depreciates faster than knowledge embedded in organizational
routines or technology. Annually, 9%, 14%, and 26% of knowledge embedded in technology,
organizational routines, and operators depreciated, respectively.
Similar to Levitt et al. (2013), this study analyzes the defect rate instead of unit costs. This study differs
from Epple et al. (1991) and Levitt et al. because it differentiates between learning by doing and induced
learning.
4.4.4. Bahk & Gort, 1993
Bahk and Gort (1993) analyzed the magnitude of firm-specific learning by doing and aimed to
decompose learning by doing into three elements: organizational learning, capital learning, and labor
learning. The authors also investigated the length of time over which learning accumulated. The authors
evaluated new plants in multiple industries using a 15- and 41-industry pool of samples from U.S. Census
Bureau data that spanned from 1973 to 1986. The authors aimed to distinguish the relationship
between learning by doing and the outcome variable (i.e., shipments or value added) from the
relationship between labor accumulation, human capital, physical capital, and embodied technical
change (i.e., change that is reflected in labor or capital inputs) and the outcome variable.
Bahk and Gort (1993) found that learning by doing significantly increased output. The authors concluded
that industry-wide learning was related to embodied technical change and physical capital. The authors
also found that the rate at which learning by doing declined varied by the type of learning.
Organizational learning continued for 10 years following the birth of a plant while capital learning
continued for only 5 to 6 years following the birth. Labor learning could not be measured with their
data.
The authors used shipments and value added as outcome variables in this analysis. Shipments may not
be a good measure of productivity because firms often store output in inventory prior to shipping it. The
sl_
'ICF
39
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
authors acknowledge that value added was a relatively weak outcome variable because it contained
measurement errors. Both shipments and value added appear to be measured in dollar value (see page
580), which raises an additional concern. Such measures would embody price and therefore are likely to
confound supply-side learning with demand-side changes that might not be related to learning.
4.4.5. Conclusion
These four articles have found evidence that organizational knowledge resides in multiple locations.
Epple et al. (1991) and Levitt et al. (2013) found that a significant amount of organizational knowledge
was embedded in the broader organization or physical capital such as its technology. Agrawal and
Muthulingam (2015) found that knowledge embedded in operators depreciates faster than knowledge
embedded in organizational routines or technology. Bahk and Gort (1993) found that embodied
technical change and physical capital drove industry-wide learning and that organizational learning
continued longer than capital learning.
4.5. Application of the 1,earning Curwe
The final five studies reviewed for this report provide examples of how learning rates are being used to
evaluate learning in mobile source and other industries. Bernstein (1988) described how learning was
used in an organization's automotive plant to reduce costs by reducing absenteeism and turnover.
Studies such as Rubin, Taylor, Yeh, and Hounshell (2004), Shinoda et al. (2009), and Nykvist and Nilsson
(2015) examined the learning rate with the aim of forecasting the future cost of technologies to
determine when the technology would be cheap enough to be competitive on the market. Other studies
such as Rubin et al. (2004) and Lee, Veloso, Hounshell, and Rubin (2010) analyzed how learning is
affected by government regulations.
4.5.1. Bernstein, 1988
Bernstein (1988) performed a case study on Volvo's use of long-term organizational development
programs in its Swedish automotive plants. In the 1960s and 1970s, during a "Spontaneous Trial Period,"
Volvo allowed trial plants to add to their socio-technical knowledge stock by experimenting with various
solutions to issues such as high absenteeism and turnover. During the "Socio-Technical Strategy Period,"
using feedback from employees at the trial plants, Volvo implemented practices that were tailored to
specific problems, such as giving newly created teams supervisory and quality control responsibilities,
providing monetary incentives for staff to learn new skills, and creating workplaces with low supervisor-
to-worker ratios. In the 1970s and 1980s, Volvo also created organizational development programs that
stressed communication and worker involvement.
Volvo experienced success that was evident in the reduction of their high absenteeism and turnover
rates over the period of the study. According to Bernstein (1988), the organization's trial-and-error
method helped the organization to move down the learning as learning occurred at all levels within the
organization. Volvo found solutions to issues with their labor force and their focus on communication
sl_
'ICF
40
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
helped them spread the knowledge gained between plants. The SME pointed out that these socio-
technical changes, however, were dismantled in the 1990s (see Adler & Cole, 1993; Berggren, 1994).
4.5.2. Rubin, Taylor, Yell, & Hounshell, 2004
Rubin et al. (2004) described the common practice of using exogenous or arbitrary rates of change in
cost or efficiency over time in energy economic models that study global climate change and carbon
management options. The authors aimed to produce more accurate estimates that reflected how costs
change in response to government actions or policies. The authors focused on two environmental
technologies used in electric power plants. Data from 5 years were used to estimate the learning rate of
flue gas desulfurization (FGD) systems (i.e., 1976, 1980, 1982, 1990, and 1995), which control sulfur
dioxide (S02) emissions, and selective catalytic reduction (SCR) systems (i.e., 1983, 1989, 1993, 1995,
and 1996), which control mono-nitrogen oxide (NOx) emissions.
The authors found that FGD systems exhibited a progress ratio of 89%, which implies that for each
doubling of installed FGD capacity, capital cost would decrease by 11%. The progress ratio and learning
rate for SCR systems was similar with a progress ratio of 88% and a learning rate of 12%.
Rubin et al. (2004) did not provide much information about the source of the data, which makes it
difficult to replicate the analysis or to determine its reliability. Additionally, the data only consisted of
five data points spanning from 1976 to 1995 for FGD systems and from 1983 to 1996 for SCR systems.
Yet, most of the capacity was added after 1980 and 1989 for FGD and SCR systems, respectively. If the
model excluded the outliers and the regressions were repeated using only the four data points after
1980 and 1989 for FGD and SCR systems, respectively, the learning curve would have indicated learning
occurred at a faster rate.
4.5.3. Shinoda, loinaka, Akisawa, & Kashiwagi, 2009
Shinoda et al. (2009) used a model that incorporated learning to predict scenarios of how widely used
plug-in hybrid electric vehicles (PHEVs) would be over the period between 2010 and 2030. The authors
aimed to find a scenario that minimized the total cost in the passenger car sector and power supply
sector. PHEVs are another example of technologies that have benefits (i.e., they reduce carbon dioxide
(C02) emissions), but the battery costs are currently too expensive to be competitive.
The authors found that for PHEVs to be competitive in the market by 2030, the battery cost must drop
to approximately 132,000¥ by 2015 if the batteries were not replaced and approximately 125,000¥ if
they were. The authors estimated that if the price dropped to 100,000¥/kWh by 2010, PHEVs could
comprise over 60% of the new vehicles bought in Japan in 2030.
Unlike other articles examined, Shinoda et al. (2009) used price as the outcome variable in their learning
estimates instead of unit costs. Price is affected by firm strategy and market conditions. For example, a
firm might price its product below unit costs to attempt to gain market share. Thus, price would not be a
good indicator of unit costs.
sl_
'ICF
41
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
4„hiA, (let;, Veloso, Hounshell, & Rubin, 2010
Lee et al.'s (2010) study focused on whether learning occurred during the technological development
caused by "technology-forcing regulations" in the automotive industry. Technology-forcing regulations
set performance standards, which require organizations to develop or improve technology to meet
them. The authors combined data from 1970 to 1998 from the U.S. Patent and Trademark Office,
technical papers published by the Society of Automotive Engineers, and cost data on automobile
emissions control devices from the EPA and California Air Resource Board (CARB) with qualitative
sources such as interviews with experts.
By analyzing trends in patents and papers published, the authors concluded that the level of innovation
from automakers and suppliers increased when technology-forcing regulations went into effect. During
periods of stricter regulations, which the authors claimed caused uncertainty in the industry26, they
found that automakers dominated architectural innovation, while suppliers dominated component
innovation. However, despite the innovation, Lee et al. (2010) did not find that learning occurred after
1984. The authors suggested that any cost reductions due to learning could have been cancelled out by
increases in the cost of precious metal catalysts. Cost of precious metals fluctuated and increased
radically during that period. The authors then examined non-catalyst components, which were not
affected by the cost of precious metals, and estimated that during 1984 and 1990, learning occurred
with a progress ratio of 93%. However, much of the innovation during that period involved catalyst
improvements together with fuel regulations. The seven data points which were used by these authors
make their finding a rough estimate at best as described by the authors.
A further note is that while technology-forcing regulations were used in the 1970s and 1980s, by the
mid-1990s, EPA and CARB started working with automobile manufacturers in developing standards. This
led to even tighter standards, which the industry could accomplish and provided environmental benefits
for the regulatory agencies. Furthermore, there is a concern that the number of patents and papers
would measure the relationship between cost and technological change rather than learning.
4.5.5. Nykvist & Nilsson, 2015
Nykvist and Nilsson (2015) estimated the current costs of Li-ion battery packs for battery electric
vehicles (BEVs) and forecasted future costs to determine if the battery packs would be cheap enough for
BEVs to become competitive with internal combustion vehicles. The authors analyzed over 80 cost
estimates from peer-reviewed articles; grey literature (i.e., work that is not formally published);
estimates from agencies, consultants, industry analysts, and leading BEV manufacturers; and news
reports from 2007 to 2014.
26 Both CARB and EPA do Regulatory Impact Analyses that provide costs and potential technologies to use to meet any
proposed standards. Uncertainty only lies in calibration of engine systems to work with the new technologies.
si_
'ICF
42
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
The authors estimated that the current learning rate was 9% for the overall industry and 6% for the
market-leading manufacturers by regressing cost data on cumulative output.27 From 2007 to 2014,
industry-wide average costs declined by 14% annually, while average costs for the market leaders
declined by 8% annually. The authors expected BEV battery costs to continue declining 8% annually in
the future. At this rate, the battery pack would not be cheap enough for BEVs to be competitive by
2030. However, Nykvist and Nilsson (2015) noted that the forecasted 8% rate was made under the
assumption that there would be no breakthroughs in technology for similar batteries and that with the
public's continued support of BEVs, manufacturers would continue to produce the batteries and take
advantage of economies of scale.28
Nykvist and Nilsson (2015) listed several areas of concern with their quantitative analysis that
surrounded their results with uncertainty. These issues included variance in costs, variance in the types
of batteries analyzed, incentives by industry to overestimate costs or subsidize production, and the
sparse availability of data.
4.5.6. Conclusion
These five studies provide examples for how learning curve research is being applied to real-world issues
in the mobile source sector. Applications range from observing companies, such as Volvo, move along
the learning curve to predicting the costs of future technologies.
27 One commenter on this report noted that the Nykvist and Nilsson learning rate of 9% would result in a 91%
progress ratio, and suggested that it would be informative to consider possible sources of this large discrepancy
in learning rates between Li-ion battery manufacturing and transportation equipment final assembly. See
Appendix D with respect to the response to this comment.
28 In EPA's Final Rulemaking for 2017-2025 Light-Duty Vehicle Greenhouse Gas Emission Standards and Corporate Average Fuel
Economy Standards, EPA applied a learning curve to battery pack development. Like Nykvist and Nilsson (2015), EPA's
models projected that the cost of producing battery packs would experience a sharp decline in the initial years of
development (i.e., the research phase) and would later experience a slower decline along the flat portion of the learning
curve.
si_
'ICF
43
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
\i/
'ICF
44
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
5. Ilesponses to Itewiewer Comments Kelated to the
Analysis
This report has undergone peer review. While the peer reviewers found the literature review of the 18
selected studies in Section 4 to be comprehensive, they suggested 3 additional articles for consideration.
We added two of these to our reviews; the third was unrelated to our work. The peer reviewers also
raised a number of questions, the most significant of which are discussed here with regard to the shape
of the learning function, supply- vs. demand-side effects; learning effects and economies of scale; and
net learning effects. A summary of all of the peer reviewers' comments and our responses can be found
in Appendix D.
First, a peer reviewer questioned whether the logarithmic learning curve should be estimated with an
initially "steep" portion followed by a "flat" portion or whether it should be estimated with a constant
slope over time (see Comment #19 under "Literature Review - General" in Appendix D). In response to
this comment, we reviewed our five selected articles and found that only one of the articles included a
model with a quadratic term, which allows one to investigate whether the rate of learning slows down
in logarithmic form. Epple, Argote, and Devadas (1991) found that the quadratic term was significant,
which suggested that there was a diminution in the learning rate in their empirical context. Because only
one of the five studies investigated whether the rate of learning slowed down, we did not see a basis for
departing from the standard model used in the literature (see Equation 1). It is important to note that
the power function shown in Equation 1 has the property that it is steeper in the earlier part of the
curve than in the later part. Although the rate of learning is constrained to be the same in Equation 1 for
different levels of cumulative output, it takes longer for cumulative output to double later (e.g., going
from 100, 000 to 200,000 units) than earlier (e.g., going from 1,000 to 2,000 units) in the production
program. Hence the learning curve is flatter later in the production program (see Figure 1).
Second, another peer reviewer noted that studies using price as a variable are likely to confound supply-
side learning effects with demand-side changes that could be unrelated to the learning process. The
peer reviewer pointed out that this concern applies to using shipments as an outcome variable because
shipments are reported in real dollar values, thereby raising the supply-versus-demand conundrum. The
reviewer argues that this concern was not always made clear in the report (see Comment #21 under
"Literature Review - General" in Appendix D). We agree with the peer reviewer and we added this issue
as an additional concern when using shipments as an outcome variable. We did not use studies with
shipments as an outcome variable when estimating our recommended value.
Third, a peer reviewer commented that there is a distinction between learning curves and economies of
scale and that the report provides no guidance on how to perform a cost analysis forecast that
incorporates learning and economies of scale as separate elements. The reviewer argued that several
studies (e.g., Lieberman, 1984) have shown that when controls for economies of scale are omitted from
the analysis, the estimated progress ratio includes the effects of both learning and scale economies. We
re-examined the studies and found that adding a separate parameter for economies of scale normally
improves the statistical fit but, as the reviewer points out, the improvement is seldom dramatic, and
sl_
'ICF
45
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
most studies have found scale economies to be less important than the learning effect. Moreover, if the
data sample is small, collinearity between the learning and scale parameters can reduce the accuracy
with which each is estimated. The reviewer further noted that an implication is that if the analyst or
policy maker is able to apply only a single cost driver for forecasting purposes, application of a learning
curve or progress ratio to forecasted cumulative output may provide the best projection of future costs
(see Comment #26 under "Literature Review - Sources of Learning Variation (Section 4.1)" in Appendix
D).
In response to the comment, there are two ways to investigate the effect of scale economies and
learning. One way is to include both current output (i.e., scale) and cumulative output up to the previous
period (i.e., not including the current period learning) as predictors (e.g., see Darr, Argote & Epple,
1995). Another way is to estimate production functions with measures of labor and capital and
investigate if there are economies of scale as indicated by coefficients greater than one. Due to the
difficulty of getting fine-grained measures, especially of capital, few researchers are able to follow the
latter approach. Further, as the reviewer notes, collinearity between the learning and scale effects can
reduce the accuracy with which each is estimated. As the reviewer notes, most studies that include scale
economies have found scale economies to be less important than learning (i.e., cumulative output).
Further, as the reviewer notes, "One implication is that if the analyst or policy maker is able to apply
only a single cost driver for forecasting purposes, application of a learning curve or progress ratio to
forecasted cumulative output may provide the best projection of future costs."
Fourth, a peer reviewer commented that the progress ratio estimated from the five selected studies are
not based upon the total cost of production and that the report should be clear about the need to
consider cost reduction of the component parts as well as the learning curve in the final assembly plant.
We point out that studies that have had measures of other costs find that they also follow a learning
curve. For example, Darr, Argote & Epple (1995) found that total costs, which included material as well
as labor, followed a learning curve. Similarly, Balasubramanian and Lieberman (2010) included material
costs in their measure, which exhibited a learning effect.
Finally, a peer reviewer commented that one objective of this report is to identify the expected pace at
which mobile source manufacturing productivity should improve with production experience. Therefore,
we should be attempting to identify a net effect of learning and depreciation rather than the gross
learning rate. While it may not be possible to derive a bottom-line net learning rate parameter that is as
comparable and applicable as the gross parameter the study reports now, the reviewer argued that we
should discuss the net-versus-gross distinction and how it might matter when applying the findings of
the report to practical settings (see Comment #30 under "Literature Review - Knowledge Persistence
and Depreciation (Section 4.2)" in Appendix D).
We appreciate the reviewer's comment about the net-versus-gross distinction. The investigation of
depreciation is a newer area than the investigation of learning. We identified ten studies that estimated
the rate of depreciation (see Table 3). If one eliminates the studies analyzing data on the production of
Liberty ships during World War II, the number drops to seven. Estimating the rate of depreciation
requires considerable data in order to disentangle the rate of depreciation from the rate of learning and
\i_
'ICF
46
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
other effects such as calendar time, with which it is likely to be correlated. For example, Levitt, List and
Syverson (2013) found evidence of knowledge depreciation but also concluded (see page 657):
"explicitly modeling the forgetting process does not substantially improve the ability of the power law
specification to fit the data, particularly relative to simply controlling for a time trend."
As we noted in our report and the reviewer ratified, mobile source manufacturing has several properties
(e.g., relatively even rates of production, learning embedded in routines and technologies, modest
amounts of worker turnover) that are likely to lead to low levels of knowledge depreciation. Comparing
rates of depreciation found in three different empirical contexts, Argote (2013, p. 80) concluded that the
rate of depreciation found in a truck assembly plant was less than the rate of depreciation found in
World War II shipyards, and both were less than the rate of deprecation found in fast food franchises.
This pattern can be seen in Table 3: The fastest depreciation (and correspondingly the least retention)
was found in the study of fast food franchises (Darr, Argote & Epple, 1995), followed by studies of World
War II shipyards (Argote, Beckman & Epple, 1990; Kim & Seo, 2009).
As noted previously, the rate of depreciation is not likely to be high in modern mobile source industries.
And it can be difficult to disentangle the effect of depreciation from other effects, such as time, with
which it is likely to be correlated. Further, our goal in the report was not to provide estimates of the
various subcomponents of learning but rather to provide an overall summary effect. For these reasons,
our summary learning effect is based on cumulative output without considering depreciation.
\i_
'ICF
47
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
'ICF 48 September 30,2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
6. filefareinces
Adler, p. S., & Cole, R. (1993). Designed for learning: A tale of two auto plants. Sloan Management
Review, 34(3), 85-94.
Agrawal, A., & Muthulingam, S. (2015). Does organizational forgetting affect vendor quality
performance? An empirical investigation. Manufacturing & Service Operations Management,
Articles in Advance, 1-18.
Argote, L. (2013). Organizational forgetting. In Organizational learning: Creating, retaining and
transferring knowledge (2nd ed., pp. 57-84). Springer.
Argote, L., Beckman, S. L., Epple, D. (1990). The persistence and transfer of learning in industrial settings.
Management Science, 36(2), 140-154.
Argote, L., & Epple, D. (1990). Learning curves in manufacturing. Science, 247(4945), 920-924.
Argote, L., Epple, D., Rao, R. D., & Murphy, K. (1997). The acquisition and depreciation of knowledge in a
manufacturing organization: Turnover and plant productivity. Working paper, Graduate School
of Industrial Administration, Carnegie Mellon University.
Argote, L., Insko, C. A., Yovetich, N., & Romero, A. A. (1995). Group learning curves: The effects of
turnover and task complexity on group performance. Journal of Applied Social Psychology, 25,
512-529.
Bahk, B. H., & Gort, M. (1993). Decomposing learning by doing in new plants. Journal of Political
Economy, 101(4), 561-583.
Balasubramanian, N., & Lieberman, M. B. (2010). Industry learning environments and the heterogeneity
of firm performance. Strategic Management Journal, 31(4), 390-412.
Balasubramanian, N., & Lieberman, M.B. (2011). Learning-by-doing and market structure. The Journal of
Industrial Economics, 59(2), 177-198.
Benkard, C. L. (2000). Learning and forgetting: The dynamics of aircraft production. The American
Economic Review, 90(4), 1034-1054.
Berggren, C. (1994). Point/counterpoint: NUMMI vs. Uddevalla. Sloan Management Review, 35(2), 37-
45.
Bernstein, p. (1988). The learning curve at Volvo. Columbia Journal of World Business, 23(4), 87-95.
Borenstein, M., Hedges, L. V., Higgins, J. p. T., & Rothstein, H. R. (2009). Introduction to Meta-Analysis.
Chichester, United Kingdom: John Wiley & Sons.
sl_
'ICF
49
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Chew, W. B., Bresnahan, T. F., & Clark, K. B. (1990). Measurement, coordination and learning in a multi-
plant network. In R. S. Kaplan (Ed.), Measures for manufacturing excellence (pp. 129-162).
Boston: Harvard Business Press.
Cohen, W. M. & Levinthal, D. A. (1990). Absorptive capacity: A new perspective on learning and
innovation. Administrative Science Quarterly, 35(1), 128-152.
Croson, R., Anand, J., & Agarwal, R. (2007). Using experiments in corporate strategy research. European
Management Review, 4(3), 173-181.
Darr, E., Argote, L., & Epple, D. (1995). The acquisition, transfer and depreciation of knowledge in service
organizations: Productivity in franchises. Management Science, 41(11), 1750-1762.
Dutton, J. M., & Thomas, A. (1984). Treating progress functions as a managerial opportunity. The
Academy of Management Review, 9(2), 235-247.
Epple, D., Argote, L., & Devadas, R. (1991). Organizational learning curves: A method for investigating
intra-plant transfer of knowledge acquired through learning by doing. Organization Science,
2(1), 58-70.
Epple, D., Argote, L., & Murphy, K. (1996). An empirical investigation of the microstructure of knowledge
acquisition and transfer through learning by doing. Operations Research, 44(1), 77-86.
Gopal, A., Goyal, M., Netessine, S., & Reindorp, M. (2013). The impact of new product introduction on
plant productivity in the North American automotive industry. Management Science, 59(10),
2217-2236.
Guetzkow, H., & Simon, H. A. (1955). The impact of certain communication nets upon organization and
performance in task-oriented groups. Management Science, 1, 233-250.
Haunschild, P.R., & Rhee, M. (2004). The Role of volition in organizational learning: the case of
automotive product recalls. Management Science, 50(11), 1545-1560.
Hendel, I., & Spiegel, Y. (2014). Small Step for Workers, a giant leap for productivity. American
Economic Journal: Applied Economics, 6(1), 73-90.
Kim, I., & Seo, H. L. (2009). Depreciation and transfer of knowledge: an empirical exploration of a
shipbuilding process. International Journal of Production Research, 47(7), 1857-1876.
Laitner, J. A. S., & Sanstad, A. H. (2004). Learning-by-doing on both the demand and the supply sides:
Implications for electric utility investments in a Heuristic model. International Journal of Energy
Technology and Policy, 2(1), 142-152.
Lapre, M. A., & Nembhard, I. M. (2010). Inside the organizational learning curve: Understanding the
organizational learning process. Foundations and Trends in Technology, Information and
Operations Management, 4(1), 1-103.
\i_
'ICF
50
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Lee, J., Veloso, F. M, Hounshell, D. A., & Rubin, E. S. (2010). Forcing technological change: A case of
automobile emissions control technology development in the US. Technovation, 30(4), 249-264.
Levin, D. Z. (2000). Organizational learning and the transfer of knowledge: An investigation of quality
improvement. Organization Science, 11(6), 630-647.
Levitt, S. D., List, J. A., & Syverson, C. (2013). Toward an understanding of learning by doing: Evidence
from an automobile assembly plant. Journal of Political Economy, 121 (4), 643-681.
Lieberman, M. B. (1984). The learning curve and pricing in the chemical processing industries. The
RAND Journal of Economics, 15(2), 213-228.
Macher, J. T., & Mowery, D. C. (2003). "Managing" learning by doing: An empirical study in
semiconductor manufacturing. The Journal of Product Innovation Management, 20(5), 391-410.
Nonaka, I., Toyama, R., & Byosiere, p. (2000). A theory of organizational knowledge creation:
Understanding the dynamic process of creating knowledge. In M. Dierkes, A. Antal, J. Child, & I.
Nonaka (Eds.), Handbook of Organizational Learning and Knowledge (pp. 491-517). Oxford:
Oxford University Press.
Nykvist, B., & Nilsson, M. (2015). Rapidly falling costs of battery packs for electric vehicles. Nature
Climate Change, 5, 329-332.
Rapping, L. (1965). Learning and World War II production functions. The Review of Economics and
Statistics, 47(1), 81-86.
Reagans, R., Argote, L., & Brooks, D. (2005). Individual experience and experience working together:
Predicting learning rates from knowing who knows what and knowing how to work together.
Management Science, 51(6), 869-881.
Rubin, E. S., Taylor, M. R., Yeh, S., & Hounshell, D. A. (2004). Learning curves for environmental
technology and their importance for climate policy analysis. Energy, 29(9), 1551-1559.
Shinoda, Y., Tanaka, H., Akisawa, A., & Kashiwagi, T. (2009). Evaluation of the plug-in hybrid electric
vehicle considering learning curve on battery and power generation best mix. IEEJ Transactions
on Power and Energy, 129(1), 84-91.
Thompson, p. (2001). How much did the Liberty shipbuilders learn? New evidence for an old case
study. Journal of Political Economy, 109(1), 103-137.
Thompson, p. (2007). How much did the Liberty shipbuilders forget? Management Science, 53(6), 908-
918.
Thornton, R. A., & Thompson, p. (2001). Learning from experience and learning from others: An
exploration of learning and spillovers in wartime shipbuilding. American Economic Review, 91(5),
1350-1368.
\i_
'ICF
51
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
U.S. Environmental Protection Agency (EPA). (2015). EPA Science and Technology Policy Council Peer
Review Handbook (4th edition). EPA Peer Review Advisory Group (Ed.). EPA/100/B-15/001.
Retrieved September 2016, from http://www.epa.gov/osa/peer-review-handbook-4th-edition-
2015
U.S. Environmental Protection Agency (EPA). (2016). July 2016 Draft Technical Assessment Report
regarding the midterm evaluation of light-duty vehicle greenhouse gas emission standards and
Corporate Average Fuel Economy standards for model years 2022-2025. EPA-420-D-16-900.
Retrieved September 2016, from
https://www3.epa.gov/otaq/climate/documents/mte/420dl6901.pdf
\i_
'ICF
52
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Appendix A. Method of Estimating Impacts of Learning
The goals of this report are (1) to be a definitive, up-to-date, reliable, single source of information
demonstrating the occurrence of learning in general and in the mobile source industry specifically; (2) to
develop a single compendium study on industrial learning in the mobile source sector that could be
considered for use in future OTAQ costs analyses; and (3) to develop a summary effect of learning based
on cumulative output in mobile source industries
This section begins with a discussion of how learning rates and progress ratios are calculated. The
section then develops two methods for estimating the impacts of learning and discusses when one
method would be preferable to the other. These approaches could be used by OTAQ in future cost
analyses. To demonstrate the approaches, we use the summary effect of learning described in Section
3.4 in a hypothetical example. Because the methods rely on Equations 1-4 in Section 3.1 and 3.2, those
sections are repeated here for ease of referral.
Calculating Learning Rates and Progress Ratios
The conventional form of a learning curve is a power function:
yt = a xtb_! (Eq. 1)
Where:
xt = Cumulative number of units produced by an organization (i.e., experience
gained) by date t
yt = Costs required to produce an additional unit at date t
a = Costs required to produce the first unit
b = Parameter that measures the rate unit costs change as cumulative output
increases. If learning occurs, b<0.
t = Time subscript
While learning curves are typically expressed as the relationship between costs per unit and production
volume, other dependent measures have been used including the amount of time it takes to produce a
unit of output, defects per unit, or accidents per unit. The particular dependent measure used depends
on the researcher's purpose. Our focus in this report is on unit costs.
Equation 1 can be rewritten in logarithmic form:
\i_
'ICF
53
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
ln(yt) = a + b ln(xt) (Eq. 2)
In actual organizational settings, learning is expected to be more complicated than the simple form
expressed in Equation 1, and costs are affected by more than just production volume. Indeed,
researchers have examined myriad other factors that can also affect learning, such as organizational
forgetting and knowledge transfer or spillover (i.e., learning from the experience of other organizational
units). Equation 1 can be generalized to investigate these issues.
Organizations often characterize their learning rates in terms of a progress ratio, p, which describes how
the outcome variable changes when cumulative output doubles. For example, the interpretation of an
80% progress ratio is that for every doubling of cumulative output, the outcome variable (e.g., costs per
unit in Equation 1) declines to 80% of its previous value. An 80% progress ratio means that costs decline
by 20%. Thus, lower progress ratios imply faster learning because costs are declining at a faster rate.
A progress ratio, p, can be computed from the learning rate, b, as follows:
y1 = Unit cost after producing x1 units
y2 = Unit cost after producing 2x1
y^axi
y2= a(2xi)b
P= — =2b (Eq. 3)
yx
Conversely, the learning rate, b, can be computed from the progress ratio, p:
ln(p) = b In(2)
I n(p)
(Eq4)
Method for Estimating Future Costs Incorporating Learning
In order to estimate future costs based on learning, we extend the framework in Equation 1 to
accommodate multiple organizations as follows. Let:
Xj t = Cumulative number of units produced by organization i through date t.
N = Number of organizations producing the product
Xt=2jJXj t = Cumulative number of units produced in the industry by date t
\i_
'ICF
54
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
A conservative approach for estimation purposes is to assume symmetric production across
organizations so that production by each organization by date t is the same across organizations:
X
xi,t= xt = 77 (Eq-5)
This approach is conservative in two respects. First, it assumes no transfer of knowledge across
organizations. If transfer across organizations occurs, costs would decline more rapidly. Second, the
approach assumes symmetry. Industry costs would decline more rapidly if production were
asymmetrically distributed across organizations, absent diseconomies of scale, than if symmetrically
distributed. This approach is conservative in the sense that it would underestimate the amount of
learning if knowledge transfer occurs or if production were distributed unevenly across organizations.
With production per organization, xt, defined as in Equation 5, cost for production of the next unit as
given by Equation 1 applies both at organizational and industry levels.
Two methods are described for estimating future costs from the above equations. Here is the notation
used for both methods:
yt+1 = Costs required to produce a unit at time t+1
yt = Costs required to produce a unit at time t
a = Costs required to produce the first unit
qt+1 = Number of units forecast to be produced in year (t+1)
xt = Cumulative number of units produced through period t
xt+1 = Cumulative number of units produced through period t+1 = xt + qt+1
b = A parameter measuring rate unit costs change as cumulative output increases
-the learning rate
Xt = Cumulative number of units produced by industry
N = Number of organizations producing the product
Both approaches require information about the learning rate, cumulative output (xt), and forecasts of
the number of units to be produced by the industry in the coming period (qt+i) as well as the number of
organizations involved in production.
The learning rate (b) can be calculated from the progress ratio (p) according to Equation 4. Based on our
review of the literature, we expect an 84% progress ratio in mobile source industries. If the progress
ratio (p) is 84%, the learning rate (b) would equal -.25. Information about cumulative output and
\i_
'ICF
55
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
forecasts of the number of units to be produced by the industry in the coming period and the number of
organizations involved in production can be obtained from industry sources and trade associations.
Method 1
The first method requires knowledge of yt, the unit cost of production at time t, but does not require
knowledge of a, the costs required to produce the first unit.
We use Equation 1 to estimate the cost of production at a future point in time, yt+i. From Equation 1:
yt= a xj?
yt+i=a xt+i
Note that yt+i is defined for the subsequent time period and not for when cumulative output doubles as
in Equation 3. If we form a ratio of these two equations, the a terms cancel:
Yw = /W|b
yt \ xt /
Rearranging terms, we solve for unit cost in the coming period, yt+i:
To illustrate this approach, assume that the following values of parameters were determined based on
the literature and trade association data:
qt+1 = 30,000 units to be produced in the coming year
xt = 100,000 cumulative units produced as of time (t)
xt+1 = 130,000 cumulative units produced as of time (t+1)
b = -.25
yt = 68
Inserting these values into Equation 6, we calculate:
yt+1=(130,000/100,000)-25(68)=63.7
That is, the unit cost of production at time (t+1) would equal $63.70. Thus, for an 84% progress ratio
(which corresponds to a learning rate of -.25) and for the values of parameters noted above, the unit
costs of production would decline from $68.00 in one period to $63.70 in a subsequent period.
\i_
'ICF
56
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Method 2
In contrast to Method 1 which requires an estimate of the current cost, yt, but does not require an
estimate of the initial cost of production, a, Method 2 requires an estimate of the initial cost, a, but not
of current costs, yt. For Method 2, we compute costs according to Equation 1:
yt+i= axt+i
To illustrate the approach, we introduce the estimate of a = 1,200 and assume xt+i = 130,000 and b= -
.25, as in Method 1. Inserting these values into Equation 1, we obtain:29
yt+1=(l,200)(130,000) •25 =63.2
Thus, the unit cost of production in year (t+1) would be $63.20. This is a dramatic decrease from the
initial value of $1,200. The intuition behind the dramatic decrease is that cumulative output would have
doubled very many times from the start of production to the current period (e.g., from 1 to 2 units, from
2 to 4, from 4 to 8, 8 to 16 and so on).
Which method to use would depend on whether one had more confidence in estimates of current costs
or of the initial cost. If one had more confidence in estimates of current costs than the initial cost,
Method 1 would be preferable to Method 2. Conversely, if one had more confidence in estimates of
initial costs than current costs, Method 2 would be preferable. In addition, if a product is just going into
production, Method 2 would be appropriate.
Both methods have the advantage of applying to organizations (and industries) that are mature as well
in early stages. The power function that underlies the learning curve has the property that the rate of
learning is the same for each doubling of cumulative output. It would take longer for cumulative output
to double in mature industries than in nascent industries but the effect of the doubling would be the
same. For example, going from producing 100,000 to 200,000 units would typically take longer than
going from 100 to 200 units. The rate of improvement in both cases, however, would be the same.
Both methods require forecasts of the number of units that will be produced in a future time period. In
most instances, such forecasts would be more readily obtained than forecasts of when cumulative
output will double. In addition, firms are often interested in forecasting their costs at a future point in
time. If one had access to good forecasts of when cumulative output would double and estimates of
current costs, one could compute the costs when cumulative output doubled, y2, from Equation 3.
29 These numbers were chosen for illustrative purposes. If we had both the number required by Method
1 and the number required by Method 2 from the same firm, results would be consistent across the two
methods.
\i_
'ICF
57
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
\i/
'ICF
58
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Appendix B. Summaries of Articles that Received a Detailed
Review
Contents
Agrawal, A. & Muthlingam, S 60
Argote, L 64
Argote, L. & Epple, D 68
Bahk, B. H. & Gort, M 70
Balasubramanian, N. & Liberman, M.B 76
Benkard, C. L 81
Bernstein, P 85
Dutton, J. M., & Thomas, A 87
Epple, D., Argote, L., & Devadas, R 90
Epple, D., Argote, L., & Murphy, K 93
Gopal, A., Goyal, M., Netessine, S., & Reindorp, M 96
Lapre, M. A., & Nembhard, I. M 99
Lee, J., Veloso, F. M., Hounshell, D. A., & Rubin, E. S 106
Levitt, S. D., List, J. A., &Syverson, C 110
Macher, J. T., & Mowery, D. C 116
Nykvist, B. & Nilsson, M 121
Rubin, E. S., Taylor, M. R., Yeh, S., & Hounshell, D. A 124
Shinoda, Y., Tanaka, H., Akisawa, A., & Kashiwagi, T 126
sl_
'ICF
59
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Agrawal, A. & Muthlingam, S.
Article
Does organizational forgetting affect vendor quality performance? An empirical
investigation
Publication
Manufacturing & Service Operations Management, Articles in Advance, pp. 1-18
Date
2015
Industry
Car manufacturer vendors
examined
Research
• How does organizational learning and organizational forgetting affect vendor
question(s)
quality performance?
• What factors influence the impact of such learning and depreciation?
Type of
Organizational forgetting; Two mechanisms of organizational learning: (1) learning-by-
learning
doing (autonomous learning) and (2) quality improvement initiatives (induced
examined
learning); Location of knowledge
Data sources
• Actual data from an unidentified large automotive manufacturer in Asia (for
confidentiality reasons)
• Interviews with senior managers and engineers of the manufacturer and its
suppliers
Data size
• 2,732 quality improvement initiatives implemented by the car manufacturer's 295
vendors
• 43 semi-structured interviews
Data years
2006-2009
Data
• The defect rate is calculated as the number of defective parts per million received
adjustment
divided by total parts supplied multiplied by 10.
• Lagged cumulative production experience is the lagged number of units (in
hundred thousands) supplied by the vendor.
Methodology
The authors estimate Eq. 4 to assess the impact of organizational learning.
(4 )ln(Yit) = &i + /3pPj(t_ i) + yqQi(t-i) + 1 ^ iVi +
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Where,
AKi(M) - stock of autonomous knowledge in the prior period
IKi(t-i) — stock of induced knowledge in the prior period
The authors estimate Eq. 6 to evaluate the impact for quality improvement initiatives.
The authors estimate Eq. 7 to include organizational forgetting.
(7) ln(Yit) = at + f3pAK+ y5KrSj(t_1) + yRKRi^t^ + yDKDift-i) + +
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
• Accounting for organizational forgetting:
o Improvement in quality performance driven by quality assurance initiatives
does not depreciate over time.
o Process improvement depreciation is estimated at .9872. Gains obtained
from doing process improvement projects depreciate by 14.32% every year
(=l-.987212, since the .9872 represents the monthly depreciation parameter).
To evaluate the impact of where quality knowledge gets embedded. (Eqs. 8 & 9):
• Without accounting for organizational forgetting:
o Lagged cumulative technology, routines, and operator solutions contribute to
organizational learning.
• Accounting for organizational forgetting:
o Estimates of organizational forgetting for lagged cumulative technology
solutions (.9923), for lagged cumulated routines solutions (.9873), and for
lagged cumulative operators solutions (.9752) are significant. Hence, quality
gains obtained from quality improvement initiatives that focus on
technology, routines, and operators depreciate by 8.86%, 14.22%, and
26.02% per year, respectively.
Assessment To assess the impact of organizational learning and forgetting:
• The results are significant.
• Quality gains obtained from organizational learning are substantial even after
accounting for the impact of organizational forgetting.
• Induced learning provided nearly a 2.5 times larger annual net defect reduction
than autonomous learning.
• The annual depreciation of quality gains, which ranged from 13% to 16%, was
lower than depreciation rates estimated in other studies. The authors attribute
this to two factors:
o Quality performance is often better documented and tracked from the
outset of production than are measures of productivity and cost.
o There was negligible turnover of Supplier Improvement Unit engineers
during the analysis period.
To evaluate the impact for different types of quality improvement initiatives:
• Without accounting for organizational forgetting:
o The estimates of organizational learning are significant only for quality
assurance and process improvement initiatives.
• Accounting for organizational forgetting:
o The estimate of organizational forgetting in quality assurance is not
significant.
o The authors do not make inferences about organizational forgetting for
design quality, as the relevant organizational learning estimates are not
significant.
To evaluate the impact of where quality knowledge gets embedded:
• All estimates are significantly different from one.
Conclusions • Organizational forgetting affects quality gains obtained from learning-by-doing
(autonomous learning) and quality improvement initiatives (induced learning).
\i_
'ICF
62
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
o 16% of quality gains from learning-by-doing and 13% of quality gains from
induced learning depreciate every year.
• The impact of organizational forgetting differs across the types of quality
improvement initiatives.
o Quality gains from process improvement initiatives depreciate by more than
14% per year.
o Quality gains from quality assurance initiatives do not depreciate,
o Significant organizational learning estimates for design quality initiatives
were not observed.
• The impact of organizational forgetting depends on where quality knowledge was
embedded.
o Depreciation is lower for knowledge embedded in technology (9%) than for
knowledge embedded in organizational routines (14%) or organizational
members (26%).
• The results suggest the need for continued attention to sustain and enhance
quality performance in supply chains.
Future • Consider costs incurred by vendors to implement quality improvement initiatives,
research • Observe solutions that were not implemented.
• Investigate whether all modes of organizational forgetting identified by de Holan
and Phillips (2004) (i.e., dissipation, degradation, purging, and suspension) are
relevant in the quality domain.
Other notes Definition of terms used in the article:
• Learning-bv-doing/autonomous learning - improving quality by performing the
same task repeatedly
• Quality improvement initiatives/induced learning - undertaking conscious actions
to improve quality
• Quality assurance initiatives - its principal focus is introduction or modification of
vendor inspection procedures
• Process improvement initiatives - its principal focus is changes or modifications to
vendor production processes
• Design quality initiatives - its principal focus is changes or modifications to the
design of the components manufactured by vendors
• Technology solution initiatives - address quality issues by introducing new
equipment, modifications to existing equipment, changes to materials, or changes
in design
• Routine solution initiatives - focus on changes to repetitive patterns of work or
introduced new repetitive activity
• Operator solution initiatives - address quality issues primarily by developing or
improving operator skills via training and monitoring
This study did inform EPA's learning rate estimate. The study is related to the mobile
source sector and the methodology used to determine progress ratios was consistent
with other studies that measured learning and forgetting in terms of improvements in
quality performance (i.e., the defect rate)—not unit costs.
Themes Organizational learning, Organizational knowledge depreciation, Disaggregation of
learning and knowledge depreciation, Location of knowledge (e.g., embedded in
technology)
Applicability
of results
\i_
'ICF
63
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Argote, L.
Article
Chapter 3: Organizational forgetting
Publication
Organizational learning: Creating, retaining and transferring knowledge. Springer.
Date
2013
Industry
Secondary analysis of studies on aircraft, ship, and automotive production as well as
examined
fast food franchises.
Research
• Does organizational knowledge acquired through learning by doing persist
question(s)
through time or does it depreciate?
• Why might knowledge depreciate?
• Could the departure of key people affect organizational performance?
Type of
Organizational learning by doing; Organizational forgetting
learning
examined
Data sources
Results from three production programs were summarized: results from the Lockheed
L-011 TriStar aircraft study as reported in Argote and Epple (1990) and Benkard (2000);
results from the production of Liberty ships during World War II as reported in Rapping
(1965) and Argote, Beckman, and Epple (1990); and results from a study of fast food
franchises as reported in Darr, Argote, and Epple (1995).
In addition, new results were presented from study of a North American truck plant
(Argote, Epple, Murphy & Rao, 1997). The plant is unionized with about 3,000
employees and has extremely advanced technology.
Data size
Unspecified
Data years
The Lockheed L-011 TriStar aircraft study: 1972-1981
The shipyard study: 1941-1943
The automotive study: Weekly data over a 2-year period from the start of production
at the plant. Exact years unspecified.
The franchise study: Weekly data. Exact years unspecified
Data
None
adjustment
Methodology The Lockheed L-011 TriStar aircraft study:
• Argote and Epple (1990) pieced together data on production from publically
available data (e.g., newspapers, trade publications, annual reports), showed that
data did not fit the classic learning curve, which assumes knowledge is cumulative,
suggested depreciation occurred, and discussed factors that could have
contributed to depreciation.
• Benkard (2000) obtained detailed data from Lockheed and determined empirically
that a model that allows knowledge to depreciate explained the data better than
the conventional model that assumes that knowledge is cumulative and persists
through time.
The shipyard study:
• Rapping (1965) had convincingly demonstrated that learning occurred in the
production of Liberty ships during World War II. His study advanced the state of
the art at the time by controlling for economies of scale and finding strong
evidence of learning when variables measuring economies of scale were included
in the statistical models.
\i_
'ICF
64
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
• Argote, Beckman and Epple (1990) built on Rapping's work by investigating
whether knowledge depreciated over time and whether knowledge transferred
across the different shipyards.
• Estimated production functions in which output produced in a given period
depended on the inputs of labor, capital, organizational experience, and other
variables
(3.1) ln(qit) = a0 + (E-|2«iA) + aln(Hit) + /3 In(Wit) + yln^u^) + S'Zit + uit
Where, Kit = AK^ + qit
qit - tonnage (in thousands) produced in yard i in month t
D, - dummy variables for each shipyard (to control for unmeasured yard-
specific factors)
Hit - labor hours (in hundreds) in yard i in month t
Wit - shipways used in yard i in month t
Kit - knowledge acquired in yard i through month t
X - depreciation parameter (A<1 implies depreciation)
• The authors tested alternative models which compared cumulative output and
time.
The automotive study:
• Estimated a production function
The franchise study:
• Unspecified
Statistical Not specified
methods used
Results The Lockheed L-011 TriStar aircraft study:
• Possible factors for why unit costs rose with increasing experience:
o The program was plagued by shortages of personnel and parts, strikes,
deregulation, and high fuel prices,
o Lockheed attempted to increase production dramatically in the late 1970s
and hired many workers without previous experience in aircraft construction
and without high school diplomas,
o Competitors had a larger experience base from which to learn and improve.
• Benkard (2000) confirmed that knowledge depreciation occurred in his empirical
study.
The shipyard study:
• Organizational learning occurred. With each doubling of the cumulative number
of ships produced, the unit cost of production declined to 74% of its former value.
• There appears to be a rapid rate of knowledge depreciation. The estimated rate
ranged from .70 to .85, which implies that from a stock of knowledge available at
the beginning of a year, only 1.4% (=.7012) to 14.2% (=.8512) would remain 1 year
later.
• The coefficient on the calendar time variable was negative, which indicated that
\i_
'ICF
65
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
the passage of time did not explain productivity gains. When a more general
translog specification of the production function was used, the coefficient was
smaller in magnitude and statistically insignificant.
The automotive study:
• When the depreciation parameter was constrained (i.e., the conventional learning
curve model):
o Provided strong evidence of learning at the plant (i.e., production increased
significantly with rising cumulative output)
o There were constant returns to labor hours and output increased
proportionately with the number of shifts worked.
• When the depreciation parameter was not constrained:
o Monthly depreciation parameter = .989
• When a time explanatory variable was added:
o There was evidence that the plant became more productive over time,
o The experience variable remained highly significant,
o The estimated value of the depreciation parameter decreased.
• When analyzing the relationship between personnel movement into the plant and
productivity:
o Found an inverted-U relationship between the number of new hires moving
into the plant and the plant's productivity (increases in productivity were
observed up to 38 people, l%-2% of the workforce, per week)
o Turnover of high-performing employees appeared to negatively affect the
organization's productivity; turnover of low-performing employees might
have improved the organization's productivity, but the variable of the
number of employees discharged for poor performance was not consistently
significant.
o Turnover of the third group whose reason for leaving was not performance
related, was not significantly related to productivity,
o The rate of learning did not change in the production environment (i.e., the
quadratic form the learning variable was not significant,
o Progress ratio: 83%. Each doubling of cumulative output at the plant led to a
17% reduction in unit cost.
The franchise study:
• The estimated weekly depreciation parameter ranged from .80 to .83 (Darr et al.,
1995). This implies that roughly half of the knowledge stock available at the
beginning of a month would remain at the end of the month.
Assessment The Lockheed L-011 TriStar aircraft study:
• The classic learning curve model that assumes knowledge is cumulative is too
simplistic to capture the dynamics of organizational learning.
The shipyard study:
• The authors repeated the study using a translog specification and the results
reinforced the results regarding knowledge depreciation.
• When input effects and economies-of-scale effects are controlled for, strong
evidence of learning and knowledge depreciation remain.
The automotive study:
\i_
'ICF
66
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Conclusions
Future
research
Other notes
Applicability
of results
Themes
• The evidence of learning is strong.
• Movement of new employees into the plant at moderate levels appears to help
productivity.
• Turnover of high-performing employees appears to hurt productivity.
• There appears to be a relatively permanent component to organizational memory
that does not evidence depreciation (i.e., knowledge embedded in the
organization's procedures and routines).
• There appears to be a more transitory component of organizational memory that
experiences a faster depreciation rate, which could be declarative knowledge (i.e.,
knowledge of facts).
The franchise study:
• The estimated rate of depreciation was the most rapid found in the literature.
• Knowledge acquired through learning by doing depreciates.
• Recent experience is a more important predictor of current productivity than
experience in the distant past.
• Possible causes of knowledge depreciation:
o Products or processes change and thereby render old knowledge obsolete,
o Organizational records are lost or become difficult to access,
o Member turnover
o Uneven rates of production, which can lead to forgetting by individuals
• Knowledge depreciation seems to depend on an organization's technological
sophistication (knowledge embedded in technology may be more resistant to
depreciation than in other repositories) and the extent of labor turnover (high
levels make it difficult to retain knowledge).
• Why do depreciation rates vary?
• What is the role of labor turnover in knowledge depreciation?
• Under what conditions does knowledge depreciate in organizations and what
factors affect the rate of depreciation?
The authors investigated different types of turnover: (1) Promotion - turnover of high-
performing employees who left the plant because they were promoted; (2) Discharge -
turnover of employees who were discharged for poor performance; and (3) All other
reasons employees departed that were not a function of performance (e.g., retired,
deceased, quit).
This study did not inform EPA's learning rate estimate because it is a secondary
analysis of other studies related to learning by doing and it does not estimate any
progress ratios based on original data.
Organizational learning by doing, Organizational forgetting, Determinants of
organizational forgetting, Knowledge depreciation
\i_
'ICF
67
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Argote, L.,
& Epple, D.
Article
Learning curves in manufacturing
Publication
Science, Vol. 247, No. 4945, pp. 920-924
Date
1990
Industry
A thought piece on studies from several disciplines; focuses on manufacturing
examined
Research
• Why do some organizations show rapid rates of learning and why do others fail to
question(s)
learn?
• Identify factors affecting organizational learning curves.
Type of
Organizational learning by doing; Organizational forgetting; Knowledge transfer
learning
examined
Data sources
A selection of empirical studies of organizational learning curves in manufacturing
(focused on organizations or work groups)
Data size
Unspecified
Data years
Unspecified
Data
None
adjustment
Methodology
Qualitative summation of previous literature
Statistical
None
methods used
Results The studies reviewed suggest organizational learning rates vary for the following
reasons:
• Organizational forgetting
o Unit costs are often higher than level achieved before interruptions such as
strikes, material shortages, and fluctuations in product demand.
o Knowledge acquired through learning by doing depreciates for reasons such
as: individuals forget how to perform tasks; individuals are replaced by
others with less experience through turnover; changes in products or
processes that make previously acquired knowledge obsolete; organizational
records or routines are lost or become difficult to access.
• Employee turnover
o It matters more in organizations where jobs are not standardized and
procedures do not exist for transmitting knowledge to new members.
o Turnover of managers and technical support staff (e.g., engineers) matter
more than turnover of direct production workers.
• Transfer of knowledge across products and across organizations
o Transfers across organizations might occur through personnel movement,
communication, participation in meeting and conferences, training,
improved supplies, modifications in technology, or reverse-engineering of
products.
• Incomplete transfer within organizations
• Economies of scale
o Estimating the rate of learning without controlling for the changing scale of
operation can result in an overestimation.
\i_
'ICF
68
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
N/A
• The knowledge about which factors affect organizational learning curves can be
used to improve manufacturing performance.
• Organizations vary considerably in the rate at which they learn and identify factors
responsible for the variation.
• Issues that need to be considered during selection of functional form:
o Choice of variables, which varies according to the production process being
studied
o Specification of the properties of random factors affecting the production
process
o Appropriate method of estimating the parameters of interest
Future N/A
research
Other notes • Organizational learning curves focus on the performance of entire organizations
or organizational subunits in contrast to the performance of individuals.
• There is often more variation across organizations or organizational units
producing the same product than within organizations producing different
products.
Applicability This study did not inform EPA's learning rate estimate because it is a secondary
of results analysis of other studies related to learning by doing and it does not estimate any
progress ratios based on original data.
Themes Sources of variation in learning rates, Organizational forgetting
Assessment
Conclusions
\i_
'ICF
69
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Bahk, B.-H., & Gort, M.
Article
Decomposing learning by doing in new plants
Publication
Journal of Political Economy, Vol. 101, No. 4, pp. 561-583
Date
1993
Industry
New plants in 15 manufacturing industries which include: bottled and canned soft
examined
drinks; sawmills, planning mills; mobile homes; corrugated, solid fiber boxes;
commercial printing, lithographic; industrial gases; paints and allied products;
petroleum refining; metal cans; fabricated structural metal; electronic computing
equipment; refrigeration, heating equipment; radio, TV communication equipment;
semiconductors, related devices; and motor vehicle parts, accessories.
41 industries were used as a robustness test. Refer to Appendix Table A1 for the
complete list.
• What is the magnitude of firm-specific learning by doing (in the context of a
production function that distinguishes the effects of such learning from the
accumulation of labor, general human capital, physical capital, and embodied
technical change)?
• Over which time intervals do the three elements of firm-specific learning by doing
(i.e., organizational learning, capital learning, and manual (labor) task learning)
accumulate?
Type of Firm-specific learning by doing (Note, this concept differs from the typical concept of
learning learning by doing. See the "Other notes" section.)
examined
Data sources U.S. Bureau of the Census, Longitudinal Research Database
Data size A set of time-series and cross-section data
• The 15-industry sample
o Consists of 1,281 plants born 1973 or later; Excludes plants born 1983 or
later because not enough time had passed to capture the learning effects
o 7,064 observations in the time-series and cross-section pool from 1973-
1986. The data were predominantly cross-sectional.
• The 41-industry sample
o Consists of 2,150 plants born 1973 or later
o Consists of the 15 industries and those with too few plants to carry out the
analysis at the industry level. Each industry had at least 16 plants.
Data years 1973-1986. The average length of a panel was between 6 and 7 years.
Data Capital:
adjustment • Variable was lagged half a year.
• The authors added the capitalized value of the changes in rentals of fixed assets
to the cumulative total of gross capital expenditure.
• Used a capital expenditure deflator for the year preceding the plant's birth for
plants that had initial capital stock that preceded their birth.
Output
• Output was proxied by data for shipments and value added, each deflated by an
appropriate deflator for the relevant 4-digit industry.
Methodology Instead of using a progress function, which defines learning by doing as the change in
unit costs over time, the authors view learning by doing as a productivity-enhancing
Research
question(s)
\i_
'ICF
70
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
factor in a conventional production function. The authors introduce separate
arguments in the production function for embodied input-augmenting technical
change (labor, human capital, physical capital, and vintage). The authors proxy firm-
specific learning by doing using cumulative output per employee (or per unit of
physical capital), and by time elapsed since the organization's birth.
(6) log(Yjt) = fa+/32 log(Lit) + fa log(Wit) + fa log(Kit) + fa logO^t) + faVit +
fat + Uit
Where,
Y - output measured by shipments (or measured by value added)
L - "pure" labor measured by the number of employees
W - human capital measured by the average wage rate
K - gross stock of physical capital
X - index of accumulated experience
V - weighted average vintage of the capital stock with ascending values for
more recent vintage
t - chronological time in years
/'- plant
The authors used the following equation to decompose learning by doing into its
principal elements (with the exception of manual/labor learning):
(7) log(Yit) = fa + fa log(L;t) + fa log(VKjt) + fa log(Kit) + uit
Where all of the variables are the same as in Eq. 6 with the exception of t, which is the
amount of time elapsed from the birth of a plant.
Learning is now captured by shifts in the (B's across successive t's. Note, that the
authors used the 15-industry sample, which they tested twice. First, the test had 399
plants (assumed 8 consecutive years of operation). The second test had 237 plants
(assumed 10 consecutive years of operation). The dependent variable, output, is
measured by shipments.
Statistical Regression using (pooled) time-series and cross-section data
methods used
Results From the pooled 15-industry sample:
• Increases in output attributed to industry-wide learning by doing (i.e., increases in
the knowledge stock) are uniquely related to embodied technical change of
physical capital (and perhaps human capital, but this was not tested).
• Using the following proxies for firm-specific learning by doing:
o Cumulative gross output since birth: A 1% increase results approximately in a
0.03% increase in output (Models i-iii).
o Cumulative gross output since birth divided by the average number of
employees at the plant (i.e., cumulative output per unit of labor input): A 1%
increase results in a 0.079% of increase in output (Model iv).
o Number of years from the birth of the plant: Each additional year results in
1.2% rise in output per year.
\i_
'ICF
71
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
• Embodied technical change of capital is associated with a 2.5%-3.5% change in
output for each 1-year change in average vintage.
• The elasticity of output with respect to "pure" labor was roughly the same as that
with respect to human capital.
From the industry-specific sample of the 15 industries:
• Confirms the results from the pooled data
• The coefficients for the key inputs showed considerable variability between
industries.
• In the "Motor Vehicle Parts, Accessories" industry (SIC 3714), when firm-specific
learning by doing is proxied by cumulative gross output since birth, a 1% increase
results in a 0.025%-increase in output, which is equivalent to a progress ratio of
0.98.
From the pooled 41-industry sample:
• The results were similar when the dependent variable, output measured by
shipments, was interchanged with output measured by value added, but the fit
was better with the former proxy.
• Estimated coefficients are similar to those in the 15-industry samples.
• Using the following proxies for firm-specific learning by doing:
o Cumulative gross output divided by the 1982 book value of gross physical
capital at the plant (i.e., cumulative output per unit of capital input): A 1%
increase results in approximately a 0.08%-increase in output as measured by
shipments (Model vi).
o Cumulative gross output divided by the 1982 number of employees at the
plant (i.e., cumulative output per unit of labor input): A 1% increase results in
approximately a 0.149%-increase in output as measured by shipments
(Model vii).
From distinguishing between the elements of firm-specific learning by doing:
• Capital learning continues until the 5th or 6th year after the birth of a plant.
Initially, the productivity of capital varies greatly across plants.
• Organizational learning is reflected in the coefficients of "pure" labor and human
capital.
o There appears to be a steady rise in the elasticity of output with respect to
labor input that continues through at least the 10th year after a plant's birth
(with the exception of the first 2 years which is likely due to the distorting
effect caused by unequal rates of capital learning),
o The effect of human capital is more erratic. Using a 3-year moving average,
there appears to be rise in the elasticity of output with respect to human
capital from the 4th to 8th year (with the exception of the first 2 years which is
likely due to the distorting effect caused by unequal rates of capital
learning).
• Overall, productivity continues to rise for a considerable number of years after a
plant's birth.
\i_
'ICF
72
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Assessment From the pooled 15-industry sample:
• The results are significant. However, the high R2 values may be due to the mostly
cross-sectional data and the large difference in plant sizes.
• Model iv, uses cumulative output per unit of labor input—an independent
variable that has been standardized to avoid simply capturing plant scale.
From the pooled 41-industry sample:
• The estimated coefficient on cumulative output per unit of capital is higher than
for the cumulative output per unit of labor.
• There is a higher estimated coefficient for learning when output is measured by
shipments than when it is measured by value added. The most plausible
explanation lies in measurement errors associated with deriving value added.
Specifically, in the measurement of costs of materials and from inconsistencies
over time in the valuation of semi-finished and finished product inventories.
From distinguishing between the elements of firm-specific learning by doing:
• The R2 values rise as the time since birth elapses. This indicates that the
consistency of the relationship between inputs and output rises with learning.
• At first, the productivity of capital varies greatly across plants; likely because
capital goods are not initially installed in balanced systems.
Conclusions • Industry-wide learning appears to be uniquely related to embodied technical
change of physical capital. But once physical capital is accounted for, industry-
wide learning is no longer a significant explanatory variable.
• Firm-specific learning is a significant explanatory variable.
• Organizational learning appears to continue over a period of 10 years following a
plant's birth.
• Capital learning continues for 5 to 6 years following a plant's birth.
• Hence, new entrants incur costs that established organizations no longer face.
Future Include the possibility for interplant learning spillovers
research
\i_
'ICF
73
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Other notes • The authors separate learning into two forms of knowledge and skill
accumulation. The first form consists of accumulation that requires an investment
(e.g., hiring, training programs, R&D expenditures). The second form, learning by
doing, is a by-product (or joint product) of production of goods and services.
• Learning by doing costs less than knowledge acquired under the first form (gives
older firms an advantage over new entrants).
• Returns to general human capital are reflected in the wage rate. Firm-specific
learning by doing is not captured by labor and enters into the firm's stock of
organizational capital.
• According to the authors, firm-specific learning by doing is an aspect of
disembodied technical change (i.e., in that it is reflected in neither the labor nor
the capital inputs but rather explains differences across firms or plants in the
productivity of the same levels and types of inputs)
• A plant was deemed "new" if there were no records for it prior to 1972.
• Definitions of terms used in the article:
o Manual task/Labor learning
¦ The routinization of tasks and adaptation to tasks that are peculiar to
individual plants/firms. (Does not capture the acquisition of general
skills through experience.)
¦ This should be reflected in the productivity of the labor input, but the
data used were not suitable for capturing this effect. The data used did
not effectively distinguish between organizational and manual learning.
o Capital learning
¦ Increases in knowledge about the characteristics of given physical
capital (e.g., engineering information that accumulates through
experience on the tolerances to which parts are machined, on the use of
special tools and devices, and on improvement in plant layouts, and the
routing and handling of materials, the true capacity of equipment, on
required maintenance, how to avoid breakdowns).
¦ This is reflected mainly in the productivity of the capital input.
o Organization learning
¦ The matching of individuals and tasks based on knowledge derived from
experience of the capacity/limitations of employees, the accumulation
of interdependent knowledge about production possess by team
members (not portable by any one team member), the development of
interactions among employees, and managerial learning reflected in
improved scheduling and coordination among departments and in the
selection of external suppliers.
Applicability This study did not inform EPA's learning rate estimate because it estimates progress
of results ratios for industries, one of which (i.e., Motor vehicle parts, accessories) is related to
the mobile source sector. Note that this study uses shipments as a dependent variable
when estimating the progress ratio, which may not be a good measure of productivity
because firms often keep output in inventory before shipping it. The other dependent
measure used was value added, which is problematic for our purposes because
measures such as value added that embody price can confound supply-side learning
with demand-side changes that are unrelated to learning. The problem of confounding
supply- and demand-side learning might also apply to their shipment variable because
\i_
'ICF
74
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
it appears to be expressed in dollar values rather than number of shipments: "Output
was proxied alternatively by data for shipments and for value added, each deflated by
an appropriate deflator for the relevant four-digit industry (p. 580)."
Themes Estimation of the learning rate, Persistence of firm-specific learning by doing,
Disaggregation of learning's elements
\i_
'ICF
75
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Balasubramanian, N., & Lieberman, M. B.
Article
Industry learning environments and the heterogeneity of firm performance
Publication
Strategic Management Journal, Vol. 31, No. 4, pp. 390-412
Date
2010
Industry
U.S. manufacturing sector
examined
Research
What is the rate of learning (overall and by industry)?
question(s)
Two hypotheses are established:
• HI: The rate of learning by doing, as measured by the slope of the learning curve,
will be higher in industries with greater complexity.
• H2: The heterogeneity of firm performance will be greater in industries with
higher rates of learning.
Type of
Learning from direct operating experience (i.e., learning by doing)
learning
examined
Data sources
• The U.S. Census Bureau - to estimate the industry learning rate.
• Compustat - to estimate the cross-sectional variation in business performance
within an industry, after applying the industry learning rates estimated using U.S.
Census Bureau data to Compustat's firm data.
Data size
The U.S. Census Bureau; The Longitudinal Research Database
• Combines data with a link from:
o Census of Manufacturing
¦ Plant-level data on all U.S. manufacturing plants with at least one
employee (over 55,000 plants over 1973-2000)
o Annual Survey of Manufactures
¦ Data from a sample of U.S. manufacturing establishments
¦ Place considerable weight on large plants and plants belonging to multi-
plant firms
¦ Every year, a sample of new entrants is added
• Data is subject to access restrictions and disclosure constraints (e.g., no data can
identify or relate to a single firm or plant)
• Contains over 4 million plant-year observations from 1963-2001
• Sample selection criteria:
o Eliminated all plants established before 1973 or after 1997
¦ 1973 is the first year of the Annual Survey of Manufactures (ASM);
therefore, it is not possible to "reliably obtain the entry year for plants
that first appear in the 1963, 1967, or 1972 censuses" (p. 397).
¦ In 1997, the U.S. Census Bureau switched from the standard industrial
classification code (SIC) to the North American Industry Classification
System (NAICS). Plants established after 1997 were omitted to
"minimize errors from industry misclassifications" (p. 397).
o Excluded all subsequent observations for a plant if the gap between
consecutive survey years is longer than 2 years
o Removed all plants that have a primary industry specialization ratio (i.e., the
output share of the primary 4-digit standard industrial classification (SIC)
industry in the case of a multiproduct plant) of less than 75%
\i_
'ICF
76
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
o Dropped outlier plants that are in the top 0.5 percentile of capital-labor
ratio or of growth in the number of employees, shipments, or capital
expenditure
• 182,603 plant-year observations
Com pu stat
• Firm data from firms that have a strictly positive total asset value
• 1,523 industry-year observations
Data years 1973-2000
Data The authors adjusted the Compustat data by aggregating firm-year level data to
adjustment industry-year level:
• For each firm-year observation, the authors compute:
o Tobin's q (the ratio of market value of assets to book value of assets)
o profitability (the ratio of operating profits before depreciation to total assets)
• Eliminate all outlying observations in the top and bottom 1% in terms of firms' q
or profitability
• These data on firm performance are aggregated to obtain the dispersion in firms'
q and profitability for each 3-digit SIC industry in each year.
Methodology • Used the information-theoretic model (Jovanovic & Nyarko, 1995)
(5 )p =
N - the number of process stages
<7w - the noise arising at each stage
o Describes three complexities
¦ N - The greater the number of tasks that any production activity
requires, the greater the number of decisions involved, and the higher
the complexity.
¦ o„- The variance of 0; the uncertainty surrounding the optimal way to
perform the activity
¦ w - The importance of transitory disturbances. Decision makers can
glean more useful information from each production run in contexts
when there are low levels of disturbances than when there are high
levels.
The authors note that the traditional method for measuring learning by doing
requires cost and production data that might not be widely available. The authors
use the approach of Bahk and Gort (1993), which replaces the variable, unit costs,
traditionally used as the dependent variable in learning curve models with the
variable, current period real value added, measured as real revenues minus real
material expenses.
It is an extension of the Cobb-Douglas production function (capital, labor, and
operating experience are considered inputs).
\Pjfv..
(8)Yijt =
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
v - plant-specific term
plant, industry, and year, respectively
To estimate the importance of learning in Eq. 9, the authors used OLS to estimate the
logarithmic version of Eq. 8.
(9) Yijt ~ ajt ajk-ijt Pjhjt jxi]t £ijt
To formally test HI, (Unit of analysis - plant year)
(10) yijf (ijt + akijt + filijf + "I" ^-i jt ^2^jt-^ijt~^~^-s^jt-^ijt
A^AjtXijt + £; jt
C - industry capital intensity (capital stock 4- employment)
W - industry wages
R - industry R&D intensity (R&D expenditure 4 sales)
A - industry advertising intensity (advertising expenditure 4 sales)
To test H2, (Unit of analysis - industry year)
(11) njt = at + + C1 Rjt + c2Ajt+c3Cjt + C4Sjt + C5Njt + C6Pjt + £ijt
% - 90th-10th percentile range of firm performance, either firm's q or firm's
profitability, in industry j during year t
Aj - estimated industry learning intensity
R - industry R&D intensity (R&D expenditure 4 sales)
A - industry advertising intensity (advertising expenditure 4 sales)
C - industry capital intensity (total assets 4 sales)
P - average industry profitability (operating profits 4 total assets)
N -the number of firms in an industry
S - industry size (total industry sales)
Statistical • To test the importance of learning, the authors used OLS regression,
methods used • To test HI, the authors used OLS to estimate Eq. 10 with plant fixed effects and
instrumental variable specification as robustness checks.
Results Estimated the importance of learning (Eq. 9, Models 1-4)
• Model 1 - The production function did not estimate learning by doing.
• Model 2 - Added prior experience
o Learning coefficient is 0.26, which implies a progress ratio of 0.84 (i.e., a
19.7% gain in productivity for every doubling of cumulative output)
• Model 3 - Included 9,967 4-digit SIC industry-year dummies to control for all
productivity improvements in each industry
o Learning coefficient is 0.23, which implies a progress ratio of 0.85 (i.e., a
17.3% gain in productivity for every doubling of cumulative output)
• Model 4 (actually 117 different models for each SIC industry with 50+ plants) -
Controlled for 3-digit SIC industry-wide productivity improvements
o There is significant variation in learning intensities across industries (just
above 0 to almost 0.6)
o The average learning intensity is 0.22, which implies a progress ratio of 0.86
(i.e., a 16.5% gain in productivity for every doubling of cumulative output).
\i_
'ICF
78
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
To test HI (Eq. 10, Models 5-10)
• Models 5 and 6 - Used a larger sample, omitted industry R&D and advertising
intensity terms. Model 5 included year indicators. Model 6 included industry-year
dummies.
o The learning coefficient is higher in industries with greater capital intensity.
o The interaction effect of industry wages on prior experience becomes
insignificant once industry-year effects are controlled for.
• Model 7 - Used smaller sample, used industry R&D and advertising intensity
terms
o The learning coefficient is significantly higher in industries with higher
capital-labor ratios, as well as with greater R&D and advertising intensities.
• Model 8 - Repeated Model 7, but assumed capital and labor coefficients were not
fixed
o The results are not substantially different from Model 7.
• Models 9 and 10 - Included plant fixed effects as robustness checks. Model 10
included direct terms.
o In Model 9, the direction and significance persist.
o In Model 10, the significance of interaction terms increases considerably and
the direct terms are negative.
o When adding once-lagged instrumental variables, economic substance and
significance were similar to Models 7 and 8.
To test H2 (Eq. 11, Models 11-12)
• Model 11 - Used the range of firm profitability as the dependent variable and the
industry estimated learning coefficients
o The coefficient on industry learning intensity is 0.926; the difference in
relative profitability between the best performers (top 10%) and the worst
performers (bottom 10%) is considerably greater in industries with high
learning.
o The coefficient on industry learning is positive and significant.
• Model 12 - Same as Model 11, but used the range of firm q as the dependent
variable
o Similar results as Model 11
Robustness Checks:
• Survivor bias, sample selection, R&D investments, measurement errors in capital,
choice of production function form, and industry life cycle effects are not driving
heterogeneity in learning rates.
Assessment • The industry learning rate displays considerable heterogeneity across industries
and it is positively correlated with the industry capital-labor ratio, R&D intensity,
and advertising intensity.
• Models 9 and 10 suggest that in industries with high capital, R&D, or advertising
intensity, plant productivity is initially low but rises steeply with experience.
• Industry learning intensity has a robust relationship with firm performance.
Specifically, the cross-sectional variation in business performance within an
industry, as measured by the interpercentile range (10th-90th) of firm q and firm
profitability, is much greater in industries with higher learning intensities.
\i_
'ICF
79
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Conclusions
•
•
Learning intensity is an important characteristic of the industry environment that
should be considered in studies of firm and industry performance.
Industry learning intensity may explain competitive heterogeneity.
Future
•
Heterogeneity in products and learning rates within industries
research
•
Mechanisms of learning (e.g., training, engineering activities, routines)
•
The variation in the meaning and context of organizational learning across and
within industries
•
Other forms of learning (e.g., knowledge transfer or spillovers - learning from
others)
•
Organizational forgetting
•
The mechanisms that explain the link between learning intensity and
heterogeneity of firm performance
•
How variations in learning rates affect firm behavior
•
How variations in the knowledge acquisition processes across industries affect the
observed heterogeneity
Other notes The authors note that the model ignores a fourth dimension of complexity, the degree
of interaction among the tasks. Interactions can greatly increase system complexity.
This study did not inform EPA's learning rate estimate because the authors used real
value added (i.e., revenues minus material expenses) as the dependent variable.
Revenues are affected by many factors (e.g., sales, the economic climate) besides
manufacturing costs.
Themes Estimation of learning rate, Sources of variation in learning rates
Applicability
of results
\i_
'ICF
80
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Benkard,
C. L.
Article
Learning and forgetting: The dynamics of aircraft production
Publication
The American Economic Review, Vol. 90, No. 4, pp. 1034-1054
Date
September 2000
Industry
Commercial aircraft
examined
Research
Past empirical studies document learning-by-doing. The author tests this by applying it
question(s)
to commercial aircraft production. The author also tests the impacts of organizational
forgetting and incomplete spillover of production expertise from one generation of
production to the next.
Type of
Learning-by-doing; Knowledge depreciation; Knowledge spillovers
learning
examined
Data sources
Lockheed data made available to the author; L-1011 TriStar aircraft production
Data size
• 250 aircraft units produced during the production run (12 observations were
removed because complete data for all levels of production were not available;
hence, the author analyzed 238 aircraft units);
• Data set includes labor requirements for each aircraft unit produced (i.e., direct
man-hours)
Data years
1970-1984
Data
None
adjustment
Methodology The author modified the traditional learning curve specification by redefining
experience to reflect organizational forgetting.
The author used the Leontif production function: factors of production used in fixed
proportions, no substitutability between factors. In this sector, labor and engines are
the biggest inputs and neither can be substituted; capital stock is constant over time.
The author also tested the suitability of using a Cobb-Douglas production function by
adding input prices to the model: proxies price of oil (demand shifter) and wages and
price of aluminum (cost shifters)
(4) InLi = lnA(K) + 9ln(Ei) + y0ln(S{) + st
Where,
L - labor
K - capital (is fixed)
0 - learning rate (learning = l-2e)
E - experience (i.e., cumulative past output)
y - within period returns to production
S - line speed
Experience is cumulative past output (the traditional learning model):
(5)Ei = Ei-t + 1 with Et = 1
\i_
'ICF
81
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Incorporate forgetting and spillover (the general learning model):
Elt if i is type-1, -100,-200
(7 )Ei =
-500, t
if i is type -500
where
(8)£i,t — + Ri,t-i + ^500,t-i an<^ ^1,1 ~ 1
(9)E$00,t = 3E5oo,t-l + 9500,£-l + tyi.t-i anc' E\
500,1
= 1
Where,
6 - experience depreciation parameter
X - experience spillover parameter
Statistical
methods used
Results
• OLS cannot be used on the production function (Eq. 4) because experience and
line speed are correlated with productivity shocks to labor. Therefore, each model
used the following methods:
o The traditional learning model uses two-stage least squares (2SLS) (Eq. 5)
o The two general learning models use nonlinear estimators.
¦ Nonlinear estimator: Generalized Method of Moments model with a
conditional moment restriction described by Hansen (1982)
• Two variables were instrumented (i.e., line speed and experience). Instruments
are present and lagged demand and cost shifters. Various lags included.
o Demand shifters: GDP, price of oil, and time trend
o Cost shifters: world aluminum price and US manufacturing wages
Traditional learning hypothesis (Eq. 5; Regressions 1-5):
• The model works better for Units 1-112 than for 1-238, with a learning rate of
30% and 18%, respectively.
• Adjustments to the model to account for line speed, time, and changes in labor
costs do not improve the explanatory power of the model.
• Although including a calendar time variable, along with production experience,
improved the fit and the standard error on the time variable, the sign of the
coefficient indicated that technological change is negative, so the model was
rejected.
Production function specification (Eq. 5; Regressions 6-8):
• Adding wages and prices does not improve the fit of the model.
• The coefficient on wages is positive, which is unlikely.
• Added a "scope" variable to account for two models, which improved the fit of
the original model.
Forgetting and spillover (Eqs. 7-9; Regressions 9-10):
• The model has a good fit.
• The learning rate is 36%.
• The monthly depreciation parameter is .96, which implies 61% (=.9612) of the
firm's experience existing at the beginning of a year survives to the end of a year.
• The coefficient measuring forgetting is estimated extremely precisely and is
significantly different from one in all cases; thereby, strongly rejecting the
hypothesis of no forgetting.
• Adding in incomplete spillovers improves the fit.
• The spillover parameter estimates that approximately 70% of the knowledge
\i_
'ICF
82
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
spilled over from one model to another (perhaps due to task overlap between the
two models).
The results were tested against alternative specification including wages and prices and
the results were not significantly different.
Assessment
Conclusions
Future
research
Other notes
The discrepancy between Units 1-112 and 1-238 is caused by the fact that the
firm's experience is not being fully retained over time. This becomes apparent
only when the production rates are uneven and new models are introduced.
Adding forgetting and incomplete spillovers into the general learning model,
explains both halves of the data.
Depreciation is high (61% of firm's stock of experience existing at the beginning of
a year survives to the next year). This could be an artifact of labor (e.g., low
aircraft production rates, high turnover, job bumping resulting from
"displacement rights").
Estimated learning rate: 35%-40%. These "are much higher than those estimated
under the traditional learning hypothesis. The reason for this is that learning is no
longer relative to cumulative production, but is not relative to accumulated
experience, which is constantly depreciating. [... ] The new learning rate implies
that if experience were doubled, then labor requirements would fall by 35-40
percent" (p. 1049).
The hypothesis of complete spillovers is rejected.
Impacts of prices and diseconomies of scope are rejected. "[...] as a result of
incomplete spillovers, the decision to bring out a new [...] model can involve a
significant setback in learning, and an associated large and immediate increase in
variable costs. [...] [I]t becomes evident that introducing new models is a costly
endeavor, even within an existing aircraft program" (p.1051).
Researchers need to include organizational forgetting in an assessment of
production.
Forgetting may not be important to all industries where learning takes place.
Aircraft and ship markets are peculiar in that the products are labor intensive,
learning is thought to be important at the individual worker level, and there is
high turnover.
There are incomplete spillovers of production expertise when switching to the
production of a new model.
The number of models can have great impact on variable production costs.
Test impacts of forgetting on other industries.
Identify conditions under which forgetting occurs (e.g., high turnover and layoffs).
Test whether the experience depreciation rate is under a firm's control (e.g.,
avoiding layoffs, priority to workers that have been laid off).
This market was chosen because the dynamics of production are complex and
marginal costs of aircraft production do not always decrease over time. Note,
previous studies concentrated on military, not commercial, aircraft, and thus are
not subject to impacts of market forces.
Definitions of terms used in article:
o Organization forgetting: a firm's stock of production experience depreciates
overtime. Implication of forgetting: recent production is more important
\i_
'ICF
83
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
than more-distant past production in determining a firm's current efficiency,
o Experience spillovers: Whether experience spills over across firms or
products is a function of how specific the firm's production experience is. If
the skills required to build one model transfer to another model, then the
firm would experience a setback in learning and higher production costs for
the new aircraft program.
• The author analyzed three counterfactual production schedules and found the
optimal production run looks much like the actual one, which closely matched
schedule deliveries.
• A stochastic version of this model was estimated and yielded almost identical
results to the deterministic version.
Applicability This study is did inform EPA's learning rate estimate because it is related to the mobile
of results source sector, it is based on primary data, and it uses labor input per unit as a
dependent variable.
Themes Estimation of learning rate, Knowledge depreciation, Knowledge spillovers
\i_
'ICF
84
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Bernstein, P.
Article
The learning curve at Volvo
Publication
Columbia Journal of World Business, Vol. 23, No. 4, pp. 87-95
Date
1988
Industry
Automotive industry
examined
Research
• What might managers learn from the Volvo experience?
question(s)
• How did Volvo use a long-term organizational development (OD) program to meet
the requirements of the auto market and its employees?
Type of
Management techniques (e.g., an increased level of employee involvement, modest
learning
use of new technology, and diffusing strategies between plants) aimed at reducing
examined
absenteeism
Data sources
Description of plant operations in Sweden from the author's point of view
Data size
N/A
Data years
Mid-1960s to 1970s
Data
None
adjustment
Methodology
The author performed a case study of Volvo automotive plants to explain the evolution
of how managers responded to absenteeism and retention problems at their plants
during the 1960s and 1970s.
Statistical
No statistical methods used
methods used
Results
• The "Spontaneous Trial Period" allowed plants to add to Volvo's socio-technical
knowledge stock related to plant practices aimed at meeting the non-material
needs of its workers to reduce absenteeism.
• After conducting over 1,000 interviews with employees at the Torslanda plant,
which participated in the Spontaneous Trial Period, Volvo was able to take their
opinions into account when devising new practices at new plants.
• During the "Socio-Technical Strategy Period," Volvo employed solutions that were
tailored to problems at each plant such as creating teams and handing over
supervisory and quality control responsibilities to them, providing monetary
incentives for learning new skills, creating low supervisor to worker ratios, and
using team leaders and craftsmen to teach and integrate newcomers.
• Volvo came up with OD programs (e.g., the Match Project, Full Rulle, and Dialog)
to create a system-wide focus on the growth and development of its entire
workforce. These OD programs are described in more detail in the "Other notes"
section.
Assessment
N/A
\i_
'ICF
85
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Conclusions
Future
research
Learning at all levels was the key to Volvo's success.
Pragmatic trial and error and the diffusion of successful practices became a
hallmark of the new system.
Throughout the Spontaneous Trial Period and the Socio-Technical Strategy Period,
Volvo "moved down the learning curve as its managers built on earlier success
and reduced errors" (p. 87), which resulted in improved productivity, better
quality, and lessened absenteeism.
Based on Volvo's experience, the author suggested practices managers should
consider to improve productivity and the quality of their products as well as to
reduce absenteeism (e.g., diffusing industrial knowledge between plants;
communicating corporate values and objectives clearly and consistently; and
investing in the workforce's education and skill development).
N/A
Other notes Definitions of terms used in the article:
• Spontaneous Trial Period - First Phase; Individual managers initiated work
improvement projects in different plants without each other's knowledge or
coordination by the central administration.
• Socio-Technical Strategy Period - Second Phase; Volvo took what it learned
during the Spontaneous Trial Period and spread its knowledge throughout the
Volvo system.
• The Match Project-This OD program concentrated on organizational
objectives which included improving communication about responsibilities,
schedules, and objectives as well as providing new employees with good
training.
• Full Rulle - This OD program was a company-wide effort to create a common
leadership philosophy and style. Among other things, it sought to empower
and improve the skills of employees and team leaders while advocating for
labor-management cooperation.
• Dialog - This OD program emphasized the need to create dialogue to support
change.
Applicability This article did not inform EPA's learning rate estimate. It discusses management
of results techniques related to managing the workforce and it does not estimate the
relationship between cumulative output and cost.
Themes Diffusion of knowledge gained through learning, Application of the learning curve
\i_
'ICF
86
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Dutton, J.
M., & Thomas, A.
Article
Treating progress functions as a managerial opportunity
Publication
The Academy of Management Review, Vol. 9, No. 2, pp. 235-247
Date
1984
Industry
Secondary analysis of data from studies in a variety of industries, including electronics,
examined
machine tools, EDP system components, papermaking, aircraft, steel, apparel, and
automobiles. The literature is drawn from industrial engineering, economics, and
management.
Research
• Can future progress rates be predicted?
question(s)
• What factors cause progress?
• How can the rate of improvement be managed?
Type of
Sources of variation in the learning rate
learning
examined
Data sources
More than 200 empirical and theoretical studies of progress functions in industrial
engineering, economics, and management from 50 years of literature
Data size
Unspecified
Data years
Unspecified
Data
None
adjustment
Methodology
Qualitative analysis of previous literature
Statistical
The authors constructed a frequency distribution of progress ratios obtained from a
methods used
sample of 108 studies of manufacturing processes in industries such as electronics,
machine tools, EDP system components, papermaking, aircraft, steel, apparel, and
automobiles to test the variability of progress rates. No industry-level experience curve
studies or studies showing price declines were included.
\i_
'ICF
87
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Results • The progress ratio, 81%-82%, has the highest frequency. Generally, reported
progress ratios range from 55% to 108%.
• The progress ratio is neither fixed nor automatic. It is often an outcome of
managerial policy decisions regarding production, marketing, and joint decisions.
• Not only do recorded progress rates vary across industries, processes, and
products, they also differ for similar process and products. Progress rates have
even varied widely for subsequent runs of the same product in the same plant.
• In any given industry, firms' progress functions, as well as progress rates, vary
widely. This variation extends not only across firms at a given time, but also within
firms over time.
• From their analysis, the authors found that four main categories of factors caused
progress:
o Effects of technological change
¦ Cumulative investments and improvements in capital equipment
explain a significant part of the variation in progress rates in similar
processes and facilities.
o Horndal (labor learning) effects
¦ Progress is brought about by direct and indirect labor learning.
¦ Progress can be attributed to adaptation efforts by labor and technical
personnel and to other autonomous cost-reducing effects of sustained
production of a good.
o Local industry and firm characteristics
¦ The progress curve is affected by local operating system characteristics
(e.g., the degree of mechanization, the ratio of assembly to machining,
the length of cycle times, continuous vs. batch process).
o Scale effects
¦ Scale can contribute to progress effects, but how this occurs is not fully
understood.
¦ Findings regarding the effects of the rate of output on the progress
curve remain mixed and contradictory.
• The four causal factors (or combinations of them) explain observed progress in
varying degrees.
• Because most causal factors of progress functions cut across organizational
subunit lines, intraorganizational relations may influence progress effects.
Assessment N/A
Conclusions • Due to the variation in the frequency distribution, caution is needed in estimating
future progress rates.
• The progress principle is of limited use in a firm's strategic planning because its
underlying dynamics are not well understood.
• To induce progress from variability (i.e., progress functions that are not subject to
the same known sources of variation over space and time), managers need to
document evidence for specific sources of progress variation accessible to the
firm's influence.
• Progress in the form of continuous cost improvements may occur autonomously
or be induced.
• Managers who wish to use the progress curve need to identify and take advantage
of static and dynamic opportunities (there are short-run and long-run dynamic
\i_
'ICF
88
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
opportunities).
Future • An analysis that combines progress functions and organizational behavior
research variables to capture the interdependence among causes of progress, which cut
across firms' hierarchical, subunit, and organization-environment boundaries
Other notes Definitions of terms used in the article:
• The progress principle - a firm can expect continuous improvement in its input-
output productivity ratios as a consequence of a growing knowledge stock (or the
cost input per unit declines at a uniform rate with cumulative production).
• Experience - a means for firms gaining knowledge
• Progress - a result of firms gaining knowledge
• Induced learning
o Requires investment, induction, or resources made available that are not
present in the current operating situation
o Affected by proximate causes
• Autonomous learning
o Automatic improvements that result from sustained production over long
periods
o Due to distant causes
o More systematic and predictable given a set of system characteristics
• Exogenous learning
o Progress usually results from information and benefits acquired from
external sources (e.g., suppliers, customers, competitors, and government).
• Endogenous learning
o Attributable to employee learning within a firm as manifested by technical
changes, direct-labor learning, and smoothing production flow
Applicability This study did not inform EPA's learning rate estimate because it is a secondary
of results analysis of other studies related to learning by doing and it does not estimate any
progress ratios based on original data.
Themes Estimated learning rates, Sources of variation in learning rates
\i_
'ICF
89
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Epple, D.,
Argote, L., & Devadas, R.
Article
Organizational learning curves: A method for investigating intra-plant transfer of
knowledge acquired through learning by doing
Publication
Organization Science, Vol. 2, No. 1, pp. 58-70 (Special Issue: Organizational Learning:
Papers in Honor of (and by) James G. March)
Date
1991
Industry
North American truck plant producing a single vehicle
examined
Research
• How can a conventional learning curve model be generalized to investigate factors
question(s)
responsible for the variations in organizational learning rates?
• Investigate three aspects of knowledge transfers acquired from learning:
o Carry forward of knowledge when the plant makes the transition from 1-
shift-a-day operation to two shifts per day
o Transfer across shifts after 2-shift-a-day operation is underway
o Transfer across time or the persistence of knowledge
Type of
Learning by doing; Knowledge transfer across shifts and across time; Knowledge
learning
depreciation, Location of knowledge
examined
Data sources
• Data from an actual truck plant. Operated with one shift for several months, then
switched to 2-shift operation. The plant is unionized.
• Weekly data for a period of 19 weeks under 1-shift operation and 80 weeks under
2-shift operation
Data size
• Weekly data beginning at the start of production for a period of 19 weeks of
operation with one shift and 80 weeks of operation with two shifts
• Deleted five observations that were not representative of normal operating
conditions from the sample
Data years
1980s, exact years not specified; 99 weeks of weekly data
Data
None
adjustment
Methodology
Linear estimation; adjust the model to capture many aspects of learning; test the
model as it is adjusted.
The authors started with Eq. 5 to estimate the coefficient related to the progress ratio.
(5) In (^) = a + yln(Qt-i) +
Where,
qt - output during week t
It - hours worked during week t
Qm- cumulative output at the end of the previous week
y - the coefficient related to the progress ratio; the percentage by which
average labor hours per unit fall with a doubling of cumulative output
The authors generalized Eq. 5 by capturing returns to increasing labor hours.
(6) ln(qt) = a + aln(Zt) + Yln(Qt-i) + et
Because diminishing returns to labor could be more pronounced with an increase in
\i_
'ICF
90
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
hours per shift rather than with an increase in shifts per week, the authors modify Eq. 6
as follows:
(7) ln(qt) = a + a\n(ht) + /3 ln(nt) + yln(Qt_i) + et
Where,
ht - hours per shift
nt - shifts per week (Note, lt = ht x nt)
The authors generalized Eq. 7 by capturing knowledge depreciation.
(9) ln(qt) = a + aln(ht) + p ln(nt) + Yln(JltsllAt-s~1qs) + et
Where,
X - depreciation parameter (A<1 implies a less than complete carry forward of
knowledge to the next period)
The authors generalized Eq. 9 by capturing the changing rate of learning as the
knowledge stock grows.
(10) ln(qt) = a + aln(ht) + p ln(nt) + yZn(/Tt_i) + SUntf^)]2 + et
Where,
Kt-i-the knowledge stock
The authors generalized Eq. 10 by capturing intra-plant transfers of knowledge (i.e.,
incomplete carry forward of knowledge and incomplete transfer across shifts).
(12a) In (y) = a + aln(ht) + pin (y) + ylnKt_t + SQnKf^)2 + et
Statistical
methods used
Results
Where accumulated knowledge stock is:
Kt = AKt_t + (1 + 9) ®
Where
0 - amount of transfer (1 is a full transfer; less than 1 is an incomplete transfer,
0 is no transfer)
The variables are halved to account for the special case in which the two shifts are
treated symmetrically. This is necessary because the data are not disaggregated by
shift.
• Regression with first-order autocorrelation of the residuals - Models 1-3 (Eqs. 5-
7)
• Maximum likelihood - Model 4 (Eq. 9)
• Model 1 (Eq. 5) shows strong evidence of learning (learning parameter is 0.15
(Std. Error = 0.02).
• Model 2 (Eq. 6) shows that diminishing returns to labor were not apparent.
• Model 4 (Eq. 9) shows that knowledge acquired through learning depreciates.
(Accounting for knowledge depreciation in the model changes the coefficient
related to the progress ratio).
• Model 5 (Eq. 10) shows that the rate of knowledge acquisition declines as the
knowledge stock increases.
\i_
'ICF
91
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
• Model 6 (Eq. 12a) shows that learning by doing yields large productivity gains as
production progresses and knowledge is accumulated, but that the rate of
knowledge accumulation declines as the stock of knowledge grows.
o A 2.9 fold increase in output per week would have occurred between the first
week and the same week 1 year later (a 190% growth in productivity).
(Learning parameter was 1.5)
o 69% of knowledge acquired during the period of 1-shift-a-day operation is
carried forward to the period of 2-shift-per-day operation,
o 56% of knowledge acquired on one shift is transferred to the other once both
shifts are in operation,
o 60% of the knowledge stock at the beginning of a year would remain at the
end of the year, if the stock were not replenished by continuing production.
(However, not significantly different from the case with no depreciation)
o The more general formulation of the learning curve yields a very substantial
improvement in the fit to the data.
Assessment For Models 1 through 6, the authors conclude that the results "fit the data quite well,
the algebraic signs of the coefficients are all as anticipated, and the magnitudes of the
coefficients are all quite reasonable" (p. 67).
Conclusions • A significant part of accumulated knowledge becomes embodied in the
organization's technology. The results provide evidence, however, against the
hypothesis that knowledge becomes completely embodied in the technology (e.g.,
tooling, programming, and assembly line layout and balancing) because the
transfer of knowledge over time and across shifts was not complete despite using
the same production facilities.
• A substantial proportion of knowledge carried forward from 1-shift to 2-shift
operation.
• Develop strategies for assessing the relative importance of training to develop
individual skills, managerial skills, and/or a network of coordination and
communication among members of the workforce.
• Further illuminate the nature of the learning process.
• Research the extent to which knowledge can be shared within production
facilities.
Other notes N/A
Applicability This article did inform EPA's learning rate estimate because it is related to the mobile
of results source sector, it uses primary data, and uses output as a dependent variable.
Themes Generalizing the conventional learning curve, Factors responsible for organizational
learning, Sources of variation in learning rates, Knowledge transfer, Knowledge
depreciation, Location of knowledge within an organization (e.g., embedded in
technology), Automotive industry
Future
research
\i_
'ICF
92
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Epple, D.,
Argote, L., & Murphy, K.
Article
An empirical investigation of the microstructure of knowledge acquisition and
transfer through learning by doing
Publication
Operations Research, Vol. 44, No. 1, pp. 77-86
Date
1996
Industry
Automotive assembly plant
examined
Research
• How does knowledge transfer across shifts?
question
• Does knowledge acquired through learning by doing on one shift transfer to a
second shift? If so, what amount?
• Does the rate of knowledge acquisition differ by shift?
• How much transfer of knowledge occurs across shifts when they are both in
operation?
• Does knowledge acquired through learning by doing accumulate or depreciate
over time?
• Is knowledge embedded in an organization's technology (e.g., tooling,
programming, assembly line layout and balancing)?
Type of
Learning by doing; Knowledge transfer across shifts; Knowledge depreciation; Location
learning
of knowledge
examined
Data sources
• Data from an actual automotive assembly plant
• The plant operated with one shift for about 2 years, then switched to 2-shift
operation.
• The plant is unionized with about 1,000 direct labor employees working on a
typical shift.
Data size
Daily data for each shift for 12 months prior to the introduction of 2-shift operation
(244 observations) and 15 months afterwards (326 observations for Shift 1 and 329
observations for Shift 2): N=899.
Data years
Not specified; prior to 1996
Data
None
adjustment
Methodology
Log-linear approach to allow for the use of linear estimation
Production function:
(1) ln(qit) = p0 + pHln(flit) + PLln{Lit) + + Pt* + %
Where,
q - number of vehicles produced
H - total direct labor hours
L - line hours of operation
K - stock of knowledge
/'- shift; 0 is single-shift operation; 1 is Shift 1; 2 is Shift 2
t- date
The model of knowledge acquisition, retention, and transfer (from Epple, Argote, &
Devadas, 1991):
\i_
'ICF
93
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
During the period of 1-shift operation):
I?) K =1° for ^= 0
ot U^ot-i + Rot for 1 Oi, then additional
increments are carried forward in subsequent periods.
Statistical Maximum likelihood approach; Tobit estimation procedure is used because the
methods used dependent variable is truncated (i.e., maximum line speed, which determines the
maximum number of vehicles that can be produced).
Four versions of the model were estimated to sequentially add the impacts of
knowledge carry-forward, knowledge acquired by vehicle by shift, and time. The fully
estimated model's results are directionally as expected. Different versions were
estimated to address questions with the results of the previous estimations.
Results • All coefficients are statistically significant and in the correct direction.
• The daily parameter measuring forgetting is .98; hence, the monthly rate of
depreciation is approximately 67%. (There are 20 working days in a month; ,9820=
0.67)
\i_
'ICF
94
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Assessment
Conclusions
Future
research
Other notes
Applicability
of results
Themes
All knowledge acquired during the period of 1-shift operation was carried forward
to the period of 2-shift operation,
o Carrying the knowledge to the day shift occurred instantly, and carrying the
knowledge to the night shift was slower, yet complete after 2 weeks of the 2-
shift operation.
Knowledge depreciates (.98, for a monthly rate of approximately 67%). Note there
is also productivity growth associated with time, and this component does not
depreciate.
The rate of knowledge acquisition per unit for 2-shift operation is about half as
large as the rate for 1-shift operation. This could be due to the fact that there are
less indirect labor hours (e.g., engineering and R&D) in the second shift.
The authors developed "an intuitively plausible and appealing picture of the
learning process" (p.84) and tested the model on actual data.
Every coefficient is of the predicted sign and falls within the predicted bounds.
Knowledge acquired during the period of 1-shift operation carried forward to both
shifts of the 2-shift regime.
The rate of carry forward was somewhat slower for the second than the first shift,
but was rapid in both cases.
The rapid and almost complete carry forward of knowledge from the first shift to
the second when it was introduced, with only technology and structure being
constant for both, suggests that knowledge acquired during the period of 1-shift
operation was embedded in the organization's structure or technology.
The learning rate per unit of output during the 2-shift regime was roughly half that
during the 1-shift regime. Most of the learning occurred on the first shift, and
most of that knowledge was transferred to the second shift. Reduced managerial
and industrial engineering attention on the second shift suggests why the reduced
learning rate per unit of output on the second shift occurred.
N/A
• The authors made a few adjustments to the model which did not affect the
results. That is, in addition to removing the largest outliers, the authors squared
the log of knowledge to allow for the possibility that depreciation is the result of
decreases in the incremental benefits of knowledge. The coefficient of the squared
knowledge variable was negligible in magnitude and statistically insignificant.
• There is an interesting discussion on learning at the sector level that suggests
there is no cross-industry learning and learning occurs on the level of the
production facility. While the idea that organizations learn from each other has
been explored (Levitt & March, 1988; Huber, 1999), the following authors note
that cross-industry learning is difficult to measure: Zimmerman (1982); Joskow and
Rose (1985); Darr, Argote, and Epple (in press); Argote, Beckman, and Epple
(1990).
This study did inform EPA's learning rate estimate because it is related to the mobile
source sector, it uses primary data, and it uses output as a dependent variable.
Knowledge depreciation, Location of an organization's knowledge (e.g., embedded in
technology), Automotive industry
\i_
'ICF
95
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Gopal, A., Goyal, M., Netessine, S., & Reindorp, M.
Article
The impact of new product introduction in plant productivity in the North American
automotive industry
Publication
Management Science, Vol. 59, No. 10, pp. 2217-2236
Date
2013
Industry
examined
North American automotive industry
Research
question
What are the impacts of new product development on a plant's productivity?
Five hypotheses are tested:
• HI: Plants that are involved in the new product launch exhibit lower productivity
than plants that are not.
• H2: Plants that deploy product flexibility in the body shop show smaller declines in
productivity from a product launch compared to plants that do not.
o If the number of platforms produced > the number of production lines, the
plant is body-shop flexible,
o The higher the ratio, the more flexible the body shop,
o 13 of 84 plants were body-shop flexible.
• H3: Plants with prior experience in manufacturing a different product - but on the
same platform as the launch product - show smaller declines in productivity from
a product launch.
• H4: Plants with more experience at launching products in the past show smaller
declines in productivity from a new product launch.
• H5: Plants that have peers within the same firm with experience in launching new
products show smaller declines in productivity from a new product launch.
Two objectives:
• Ascertain whether a plant hosting a launch suffers from a decline in productivity
(HI).
• Identify factors that can mitigate the loss in productivity (H2-5).
Does learning persist over time?
Type of
learning
examined
This article does not directly focus on learning that results from production. It focuses
on how past experiences can mitigate decreases in productivity resulting from product
launches.
Three measures of experience:
• Platform experience: number of vehicles (which differ from the type of vehicle
currently being launched) produced in the 3 previous years
• Launch experience: number of launches at the plant in the 3 previous years
• Firm experience: number of launches in the specific control set of plants in the 3
previous years
Data sources
• Harbour Reports (a survey of all North American automotive manufacturing
plants)
• Ward's Automotive (data such as monthly production at each plant and monthly
sales)
\i_
'ICF
96
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Data size Harbour Reports include:
• 78 plants owned by four firms (i.e., former Daimler-Chrysler, Ford, GM, and
Toyota)
• 408 plant observations
o 88 product launches at 50 distinct plants (34 plants had more than one
launch)
o Four observations were removed because the plants did not exist before the
launch; hence, there were 84 total plant-year launch events (and 320
observations of non-launch events)
Data years 1999-2007
Data Control variables:
adjustment • Sales variance - changes in productivity due to demand variance
• (Prior) Utilization - output 4- annual capacity
• Late model - plants with models near end of life-cycle
• Product type-types of vehicles
• Company - capture any firm-specific fixed effects
• Year - capture any year-specific fixed effects
Methodology • The method is similar to an event study (i.e., the authors estimate the loss of
productivity at launch plants compared to non-launch plants.)
• The dependent variable is productivity change (i.e., the relative change in
productivity at each plant from the preceding year)
o A positive value reflects the percentage productivity decline during the
launch.
o Average productivity change: 9.52% for launch plants; -5.24% for non-launch
plants
Statistical HI: Two approaches
methods used • Series of matched sample methodologies:
o "obvious" managerial constraints
o propensity scoring methods,
o nearest-neighbor bias-corrected matching estimators
o nearest-neighbor matching estimators with trajectories
• Instrumental variable approach; Probit model to estimate probability of being a
launch site
• Identify best match and compute mean productivity change for each group; do for
single lag and trend
• OLS regression; Heckman sample selection methodology
H2-H5: OLS regression; Heckman sample selection methodology; panel data
To assess if learning persists over time:
• Re-estimate the OLS model of productivity by explicitly disaggregating these
variables by year
Results HI: All of the methods to assess impacts of product launch show a decrease in
productivity. Range is 12.5% to 15.9% for matched samples. For regression, the
estimated coefficient is 11.97% (treatment effects) and 14.84% (random effects)
\i_
'ICF
97
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Findings related to mitigating increases in HPV:
• H2: Product flexibility in the body shop: -10.82%
• H3: Producing using the launch platform in the past: -2.057%
• H4: Launching products in the past: -3.133%
• H5: Firm experience in launches: No support
Both methods (OLS and panel data) support these findings; panel data findings slightly
lower (-7.38%, -1.953%, and -2.311%, respectively)
With respect to persistence of learning over time, some forms of learning persist
across 3 years (e.g., prior platform experience); others fade more quickly (prior launch
event experience).
Assessment N/A
Conclusions Product launches cost money by reducing plant productivity by 12%-15% ($42-$53
million). One could reframe these results as showing that there is transfer from
previous products to the new product. That is, when a new product is launched, it, not
surprisingly, causes a downward blip in productivity of 12%-15% but much of the
knowledge transfers to the new product.
Steps can be taken to mitigate the decrease in productivity due to launches: plants
with experience in manufacturing similar products, experience in product launches,
and with flexible body shops do better.
• Each unit of platform experience (100,000 launch platforms produced within the
past 3 years) yields 2.1% savings in productivity.
• Flexibility yields 10.8% savings in productivity.
Implied savings of $15.5 million at an average plant from one standard deviation of
improvement
Knowledge acquired through production on the launch-platform is 'sticky,' while the
knowledge acquired through launching products in the past tends to depreciate faster.
Thus, formal efforts to internalize and ingrain the knowledge acquired through product
launches can further increase the efficacy of launches.
Future
• Re-do with a more detailed data set (e.g., monthly productivity data). Look at
research
networks of plants.
• Study supplier relationships.
• Look at impacts of product architecture.
• Look at different types of firms (e.g., those producing similar products, those
which are geographically close) to see if launch experience is significant.
Other notes
Definitions of terms used in the article:
Launch: introduction of an all-new vehicle or maior product change (e.g., new
sheet metal or new exterior on a vehicle)
Applicability
This study did not inform EPA's learning rate estimate because it does not estimate any
of results
progress ratios.
Themes
Length of knowledge persistence, Automotive industry
\i_
'ICF
98
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Lapre, M. A., & Nembhard, I. M.
Article
Inside the organizational learning curve: Understanding the organizational learning
process
Publication
Foundations and Trends in Technology, Information, and Operations Management, Vol.
4, No. 1, pp. 1-103
Date
2010
Industry
Secondary analysis of studies from several disciplines, both manufacturing and services
examined
Research
Why do significant differences in learning rates exist across organizations?
question(s)
Type of
Organizational learning; Typology
learning
examined
Data sources
None
Data size
None
Data years
None
Data
None
adjustment
Methodology
The authors gather previous research and assemble it into a systematically organized
body of knowledge on organization learning.
Statistical
None
methods used
Results Chapter 1: Introduction
Common elements in the definition of org. learning:
• The focus must be on the org. level not the individual level.
• Enhancing knowledge and understanding within the organization
• The purpose is to facilitate changes in actions to produce better org. performance.
• It is an ongoing process that occurs throughout an organization's lifetime.
Levels of learning
• Learning is an iterative, multi-level process in organizations.
• Knowledge and practices move from the individual to groups and teams to org.
levels.
• Learning at the org. level shapes how individuals and groups act and what they
learn.
Chapter 2: Organizational Learning Curves
Three ways to measure experience:
• Cumulative volume
• Calendar time elapsed since the start of operation
• Maximum proven capacity to date
Four ways to measure performance: (1) unit time, (2) unit cost, (3) quality, and (4) total
factor productivity
\i_
'ICF
99
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Chapter 3: Behind the Learning Curve: Understanding Variation in Learning Rates
Frameworks for Understanding the Variation in Learning Curves:
• Levy (1965): Autonomous vs. Induced Learning
o Identified three types of firm learning:
¦ Planned/Induced learning - results from firms applying techniques
designed to increase the rate of output (i.e., reduce production costs)
¦ Random/Exogenous learning - results when a firm acquires information
unexpectedly from its environment (e.g., suppliers, government/trade
publications, competitors)
¦ Autonomous learning - results from employees' on-the-job learning or
training
• Dutton and Thomas (1984): Autonomous vs. Induced and Endogenous vs.
Exogenous
o Learning-type dimension
¦ Induced learning - requires investment, induction, or resources that are
not currently present
¦ Autonomous learning - automatic improvements that result from
sustained production
o Origin dimension
¦ Endogenous learning - employee learning within a firm
¦ Exogenous learning - results from information and benefits acquired
from external sources (e.g., suppliers, customers, competitors, and
government)
• Bohn (1994): Inside the learning curve
o Variation in learning rates may be due to organizations differing in:
¦ The amount/nature of experience and deliberate learning activities
(DLAs) and the ability to learn from them
¦ The ability to translate learning into better org. knowledge
¦ The ability to change behavior in response to better org. knowledge
¦ The ability to obtain better org. performance as a result of changed
behavior
Variation Derived from Experience:
• Each framework agrees that experience is a core mechanism for org. learning;
although all experience is not the same. Scholars have focused on three attributes
of experience:
o Specialized vs. diversified experience
¦ Emerging studies found that a good balance between specialized
experience and widely diversified or generalized experience maximizes
learning; there is a U-shape relationship between exposure to variety
and performance (more variety is better, but only up to a point).
• Specialization - one can get a deeper understanding of an area
and easier transferability of knowledge, but repetition can lead to
stagnation
• Diversity - can stimulate new ideas and foster a more complex
understanding, but it can be difficult to integrate and apply
\i_
'ICF
100
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
knowledge across experiences
o Success vs failure experience
¦ Organizations respond differently to the experience of success/failure
¦ Org. learning can be facilitated by both success and failure.
¦ Kim et al. (2009) found that organizations must accumulate a certain
amount of the same experience (success or failure) before org.
performance will improve as a result of learning.
¦ Results are mixed on whether success or failure is more advantageous.
¦ Research indicates whether and how an organization responds to and
learns from successes/failures depends on a variety of factors: (1) the
nature of the success/failure, (2) the level of each experience/the
presence of other experiences, (3) the level of aspiration, and (4) the
context.
o Individual vs. team vs. org. experience
¦ Each level of experience has been theorized to provide learning and
performance benefits.
• With increased cumulative individual experience comes individual
proficiency through knowledge/skill development.
• With cumulative team experience comes better coordination and
teamwork as individuals learn who knows what, who is best at
performing each task, and how to trust each other.
• With cumulative org. experience, staff learn from the knowledge
accumulated by others.
¦ Reagens et al. (2005) found that team and org. experience had a
consistently positive relationship with performance while individual
experience had a U-shape relationship with performance (i.e., at low
levels, increases hurt performance, at high levels, increases improved
performance)
Variation Derived from Deliberate Learning:
• Types and amount of DLAs (e.g., training sessions, experiments, and quality
management programs)
o Faster learners use more DLAs that generate know-how and know-why, and
they focus on learn-how.
• Contextual differences
o The greatest positive impact occurs when all org. participants support
deliberate learning, when the use of DLAs occurs across multiple locations
with time for reflection and with the purpose of quality improvement.
• Macro-factors
o Task characteristics
¦ Org. theory suggests that the proportion of tacit-to-explicit knowledge
in a task explains a significant percentage of the variation in
improvement rates for organizations learning to perform the same task.
o Org. characteristics
¦ Argote et al. (2003) classified factors into three categories; those that
affect the motivation, ability, and opportunity to learn
\i_
'ICF
101
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Chapter 4: Relative Effectiveness of Experience vs. Deliberate Learning as Sources of
Learning
The Path to Optimal Learning: Experience or Deliberate Learning?
• The relative effectiveness of DLAs and experience may depend on:
o The Stage of Production
¦ When organizations are early in production, deliberate learning benefits
the organization more than experience.
¦ The relative benefit changes as the production process matures,
o The Stage of Knowledge
¦ Research found that learning from experience is more effective when
knowledge is under-developed, while deliberate learning is more
effective when knowledge is well-developed.
o Task Characteristics (Zollo & Winter, 2002)
¦ Deliberative learning would benefit tasks with high economic
importance; a larger scope, involving multiple groups/departments; low
frequency; high heterogeneity; or a high degree of causal ambiguity
Chapter 5: Moving from Learning to Performance: Steps Inside the Learning Curve
• To improve performance, organizations must go through three steps "inside the
learning curve"
o Develop better org. knowledge
o Step 1 motivates changes in behavior
o Step 2 contributes to improved cost and quality performance
From Learning to Better Organizational Knowledge
• Lapre et al. (2000) and Choo et al. (2007) provide evidence that learning is
associated with knowledge creation.
• However, not all learning leads to better org. knowledge and performance.
From Better Organizational Knowledge to Changed Behaviors
• Mukherjee et al. (1998) showed conceptual and operational learning altered
improvement project teams' ability to change behavior.
• Tucker et al. (2007) showed that learning results in the ability to spur behavioral
change and actual behavioral change.
From Changed Behavior to Organizational Performance
• Nembhard and Tucker (2010) found that learning activities can facilitate the
interdisciplinary collaboration, which is needed for performance improvement
over time. They offer three explanations: Interdisciplinary collaborators (1) make
better decisions, (2) have improved coordination, and (3) are skilled at detecting
and learning from errors.
Challenges to Advancing
• Organizations can experience difficulty progressing from learning to improved
performance due to at least four sets of factors:
\i_
'ICF
102
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
o Psychological and sociological factors
¦ Low psychological safety in organizations stifles the willingness to
engage in learning and discourages individuals from interpersonal risk-
taking out of fear of negative consequences.
¦ Other psychological factors limit learning - e.g., aversion to change and
failure.
¦ Sociological factors
• Social pressure - conformity creates a bias towards the leader's
perspective
• Traditional conflict management strategies - tend to force the
majority view on the other party
• Competency traps - tend to under-react by making the flawed
assumption that current routines are preferable to alternatives
o Cognitive factors/Learning capacity
¦ An organization's capacity for learning is a function of its resource and
absorptive capacity.
¦ Organizations with limited resource and/or absorptive capacity are
likely to learn less at a slower pace.
¦ Knowledge depreciation/forgetting limits the knowledge stock of
learning.
• Rates vary across settings and depend on calculation methods,
o Complexity
¦ Several complexities impede org. learning such as:
• Detail complexity - the presence of too many variables makes it
difficult to comprehend a problem in its entirety
• Dynamic complexity - when distance and time make cause-and-
effect difficult to establish
• Incomplete technological knowledge - lack of understanding the
effects of a process' input variables on output
o Multi-level process
¦ Learning's effectiveness is susceptible to factors at multiple levels
• Individual - their knowledge/experiences can facilitate or hinder
learning
• Group - interpersonal dynamics and group norms
• Org. - organizational structure/design
¦ Four core challenges to moving to the next level: (1) role-constrained
learning, (2) audience learning, (3) superstitious learning, and (4)
learning under ambiguity
Assessment N/A
Conclusions • Evidence consistently documents the org. learning curve phenomenon.
• Evidence shows there is variation in learning rates.
• The accumulation of experience at all levels of the organization enhances the
learning rate.
• Not all experiences are equally beneficial (e.g., failures seem to accelerate
learning more than success).
• The benefit of any experience depends on other factors (e.g., the nature of
experience, the level of other experiences, the aspirational level, and the context).
\i_
'ICF
103
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
• There is little consensus on whether experience or deliberate learning influence
productivity more.
• Knowledge learned from experience depreciates.
Future The authors provide suggestions for future research throughout the book. Below, we
research are including those suggestions that were summarized in Chapter 6, which was
specifically devoted to future research.
Chapter 6: The Next Frontiers in Organizational Learning Curve Research
Knowledge Creation
• At what stages of causal and control knowledge can an organization expect to
make more than merely incremental improvements?
• Do breakthrough improvements require balanced climbing of the stages?
• What is the impact of climbing the stages of knowledge for primary variables vs
secondary variables?
Learning by Experimentation
• What experimentation strategies allow an organization to climb the stages of
knowledge faster?
Development of the Learning Organization
• How does an organization become skilled at creating, acquiring, and transferring
knowledge, and at modifying its behavior to reflect new knowledge and insights?
• Are teams and training sufficient to transform an organization into a learning
organization?
• Identify different underlying mechanisms that govern the development of building
blocks (besides a supportive learning environment, the presence of learning
processes and practices, and leader behavior that provides reinforcement of
learning.)
• How do organizations manage and store their knowledge?
• How does an organization effectively use knowledge reservoirs for org. learning?
• How does an organization sustain learning?
Learning Curves for Other Measures of Organizational Performance
• Study additional dimensions of operational performance (besides cost, quality,
and lead-time) such as supply chain management and sustainable operations
management
• Will too much experience eventually be detrimental?
• Is there a way to avoid the competency trap of focusing on exploitation at the
expense of exploration?
• What org. learning efforts can re-ignite improvement after experiencing a reversal
in performance?
• Can the reversal effect be avoided by accumulating related experience?
• Study dependent variables such as customer satisfaction, customer retention,
repeat purchase, customer loyalty, and lifetime value of the customer.
Learning to Improve Multiple Measures of Performance
• Study the learning efforts behind performance improvement paths.
• Can operating experience and DLAs simultaneously drive improvement for
multiple measures of org. performance, or do different performance measures
require different learning variables?
• How would learning effects differ across different pairs of performance measures?
\i_
'ICF
104
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
• What learning activities can prevent trade-offs between operating priorities?
Longitudinal Data Envelopment Analysis: Competitive Learning
• Which firms consistently perform on the efficient frontier? Which do not and
why?
• What types of experiences and DLAs foster competitive learning?
• What learning activities can prevent a firm from falling off of the efficient
frontier?
Other notes
N/A
Applicability
While this book is a comprehensive review of the status of the learning curve field, this
to Results
study did not inform EPA's learning rate estimate because it is a secondary analysis of
other studies related to learning by doing and it does not estimate any progress ratios
based on original data.
Themes
Specification of the learning curve, Sources of variation in learning rates, The learning
process, Barriers to learning
\i_
'ICF
105
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Lee, J., Veloso, F. M., Hounshell, D. A., & Rubin, E. S.
Article
Forcing technological change: A case of automobile emissions control technology
development in the US
Publication
Technovation, Vol. 30, No. 4, pp. 249-264
Date
2010
Industry
examined
Automobile industry; automobile emission control technologies
Research
question(s)
• How do firms manage and organize their R&D processes concerning automobile
emissions control technologies amid the uncertainties resulting from the issuance
of new regulations?
• Did government actions, from merely threatening to impose regulations to the
actual imposition of increasingly stringent ones, actually influence the innovative
activities of automakers and their suppliers?
• If so, where does the technology come from?
• Who are the key contributors?
• How does "learning" take place during the process of technological development
under technology-forcing regulatory regimes?
Type of
learning
examined
Learning by doing
Data sources
• U.S. Patent and Trademark Office (USPTO) data set
• Technical papers published by the Society of Automotive Engineers (SAE) special
(SP) series publications
• Cost data set for automobile emissions control devices compiled from two main
sources: (1) the EPA (1990) and the California Air Resource Board (1996)
• Interviews with industry experts involved in the development of automobile
emissions control technology
Data size
• 2,253 automotive emissions control-related patents
• 701 SAE technical papers
Data years
1970-1998
Data
adjustment
• The authors generated the relevant patent set using the USPTO data set using
abstract-based and class-based keyword searches. Duplicate or irrelevant patents
were removed.
• The authors generated the SAE technical paper data set by screening the
relevancy of the articles.
• The authors adjusted costs data to constant 2000 dollars.
Methodology
Combination of quantitative and qualitative methods (i.e., interviews with experts).
Inventive activities: Timing of technology introductions and patenting trend
• Mapped a series of the onset of automotive emissions control regulations and
corresponding levels of stringencies for major pollutants against the introduction
of critical new technologies
• Contrasted time-series with the magnitudes of patenting activities with the same
series of stringency levels for the major pollutants
• Regressed the onset of technology-forcing regulations on the level of innovation
\i_
'ICF
106
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Sources and the locus of innovation
• Associated entities developing patents and technical papers
Knowledge management and task uncertainty
• The authors classified patents as either architectural or component innovation
and defined three periods of either certainty or uncertainty between 1970 and
1998. The authors then estimated the share of patents held by automakers and
suppliers by innovation type and period of certainty/uncertainty.
• Performed a Probit estimation using component innovation as the dependent
variable
Learning by doing
• The authors graphed the estimated average cost of catalysts per vehicle over the
period of 1972-1994.
• The authors regressed the cumulative number of emission control devices
installed on the normalized cost of emission control devices.
• Estimated the progress ratio for learning related to non-catalyst components
Statistical The authors developed a statistical model based on the Probit estimation approach
methods used using component innovation as the dependent variable.
Results Inventive activities: Timing of technology introductions and patenting trend
• The authors found that the automotive industry launched new emissions control
technologies whenever increasingly more stringent regulatory standards phased-
in. The timing of their findings did not always match regulation changes.
• An increase in stringency appears to lead to an increase in patenting activity.
• The authors found a positive and significant relationship between the onset of
technology-forcing regulations and the level of innovation. Again, findings did not
always match regulation changes.
• Automakers and suppliers were the major players in the development of
automobile emissions technologies, accounting for more than 93% of patents and
73% of technical papers.
Sources and the locus of innovation
• Suppliers were the main locus of innovation prior to 1975 (before the introduction
of the first-generation catalytic converter). Automakers became the principal
locus of innovation thereafter, which implies they became active as "system
integrators" in product development. They possessed knowledge of all of the
system's aspects.
Knowledge management and task uncertainty
• Automakers and suppliers dominated architectural and component innovation,
respectively, throughout each period.
• Component innovations by automakers and architectural innovations by
component suppliers increased with the imposition of more stringent regulatory
standards during the 1970s and 1990s, both periods of uncertainty. This suggests
that suppliers and automakers tend to engage in architectural and component
innovation, respectively, amid task uncertainties.
\i_
'ICF
107
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Learning by doing
• The authors believe the overall cost of emission control devices did not change
after 1984 because potential cost reductions due to learning could have been
cancelled out by the increases in the cost of precious metal catalysts. However,
precious metal costs varied significantly during this period.
• The authors estimated that reductions in the cost of non-catalyst components due
to learning took place with a progress ratio of 0.93 between 1984 and 1990. These
components were not affected by fluctuations in the price of precious metals. This
finding was based upon seven data points and the authors suggest this is a rough
estimate of progress.
Assessment • The rough estimate of the progress ratio of 0.93 is somewhat slower than the
average progress ratio of 0.81 found in manufacturing by Dutton and Thomas,
1984.
o Much of the innovation during this period was due to catalyst formulations,
which were not taken into account in the analysis because of the large
variation in precious metal costs,
o The estimate is based on a very limited number of points (seven points),
o While technology -forcing regulations occurred during the 1970s and 1980s,
by the mid-1990s, regulatory agencies began working with manufacturers to
develop new emission standards that could be accomplished by industry and
represented a step reduction in emissions.
Conclusions • High regulatory standards under the technology-forcing regulation played an
important role in forcing technological innovations and determining subsequent
direction of technological change. This method has now changed to a more
collaborative approach with industry.
• Component suppliers were important sources of innovation in the 1970s, but over
the course of technological evolution, automakers gradually emerged as the locus
of innovation.
• Firms strategically manage architectural and component knowledge in the
presence of uncertainties about their technological capacity to meet new auto
emissions control standards.
• The rough progress ratio estimated in this work was based upon limited data and
did not take into account catalyst formulation changes which were the main
factor in reducing emissions to meet new standards.
• The authors claim great period of uncertainty when new regulations were passed.
EPA and other regulatory agencies develop regulatory impact analyses while
formulating regulations, which provide information on technologies to meet
standards and provide expected costs.
Future • Test whether there is a relationship between stringent regulatory pressures and
research the competitiveness of regulated firms by incorporating international trade
dimensions with regulatory pressure and patenting activities of regulated firms.
• Evaluate innovative capabilities of supplier networks.
• Understanding the structural relationships, changes in the key players, and their
linkages with exogenous environmental events should provide a clearer picture of
the forces that drive technological evolution and the success of government
regulations in stimulating innovation.
Other notes
Definitions of terms used in the article:
\i_
'ICF
108
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
• Technology-forcing regulations - regulations that mandate firms to meet
performance standards that go beyond the existing technical capabilities of the
industry or to adopt specific technologies that have not been fully developed
(Jaffe et al., 2002)
• System integrators - firms that integrate and coordinate the internally developed
and externally produced works of suppliers (Robertson & Langlois, 1995; Brusoni
et al., 2001)
• Architectural innovation - embodies knowledge on how components are linked
• Component innovation - embodies knowledge on components
• Period of uncertainty - (1970-1981; 1990-1998) during the presence of
regulatory pressure
• Period of certainty - (1982-1989) during the absence of regulatory pressure
Applicability This study did not inform EPA's learning rate estimate because its data on learning
of results were limited. The progress ratio was estimated using only seven data points and the
dependent variable used to estimate the progress ratio was not described in the
article. There is also a concern that the number of patents and papers is not a valid
measure for learning and is more a measure of technological change. In addition, the
estimation of the progress ratio did not take into account the interaction between
regulations, regulatory intent, and politics. For these reasons, the progress ratio
calculated here may not be a good indicator of learning.
Themes Automobile industry, Regulation's role in learning
\i_
'ICF
109
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Levitt, S. D., List, J. A., & Syverson, C.
Article
Toward an understanding of learning by doing: Evidence from an automobile
assembly plant
Publication
Journal of Political Economy, Vol. 121, No. 4, pp. 643-681
Date
2013
Industry
Automobile (assembly plant of an auto producer)
examined
Research
• What is the rate of learning by doing?
question(s)
• What are the processes by which improvements occur?
Type of
Learning by doing; Knowledge transfer or spillover; Location of knowledge
learning
examined
Data sources
• Production data from an assembly plant of a major auto producer collected by
Factory Information System (FIS) proprietary software
• Daily records of absent employees from an administrative database
• Warranty claims made on the cars produced
Data size
The data cover the production of 200,000 cars (include three model variants).
Data years
• One year, which is not specified for proprietary reasons. The August to December
period is labeled Year 1. The January to July period is labeled Year 2.
• Model 2 was introduced 17 weeks after the start of the analysis period. Model 3
was introduced 13 weeks after the start of Model 2.
Data • The authors removed the small number of prototype vehicles produced at the
adjustment start of the analysis period from the sample. These cars were used for training
and to find major difficulties in the production process; therefore, they featured
high defect rates.
• For consistency, when describing the introduction dates of Models 2 and 3, the
authors impose a threshold of 100 cars per week being produced for the cars'
production data to be included in the sample.
• The authors segmented the production process by benchmark operations. They
apportion the car to a production week by segment. The sum of complete and
partial cars produced within a period equals the number of cars produced per
period.
• The authors exclude any weekend operations.
Methodology To estimate overall learning patterns:
• Use the basic specification:
In (St) = ln04) + /?ln(£"t)
Where,
St - productivity at time t (average defect rate)
Et - production experience up to that point (i.e., cumulative production)
(B - learning parameter
• Add a time trend to the basic specification.
• Replace the time trend with the following experience term to allow for
organizational forgetting:
Et = + Rt-1)
\i_
'ICF
no
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Where,
6 - the retention parameter
Em- experience at the start of the prior period
qt-i- production in the prior period
Supplementary evidence from quality audits:
• Compare FIS data to an independent production defect measure (the quality
audits on randomly selected cars) and data on warranty claims.
To explore the mechanisms driving learning by doing:
Adding a Second Shift:
• Estimate shift-specific learning rates, where the logged average error rates on a
shift are regressed on the log of cumulative production from that shift.
• Estimate ramp-up spillovers by including a dummy variable for the second-shift
ramp-up period to the first-shift-specific learning regression.
Introducing Additional Product Variants:
• Estimate model-specific learning by doing rates by regressing the logged average
error rates on a shift on experience (cumulative production of the specific model
variant).
To test for station-level patterns:
Distribution of defects:
• Measure the skewness of defect rates across production stations and test for
intertemporal changes in this skewness.
Persistence:
• Investigate the correlation of station-level error rates across shifts by grouping all
stations by their quintile within the shift-specific defect rate distribution during a
particular week and compare a given station's quintiles across the first and
second shifts that week.
To test defect spillovers across cars:
• Regress the defect count of a given car on the defect counts for each of the 25
cars that preceded it along the assembly line. Control for day fixed effects. The
model is done separately for three different production periods (i.e., early in the
model year, middle of the year, and year's end)
To test absences and the role of worker-embodied learning by doing:
• Disaggregate production and absentee data to test if absenteeism and defect
rates are correlated.
• Use the "forgetting specification" and allow the rate at which the knowledge
stock depreciates to vary with the fraction of workers who are absent.
• Compute defect and absences by department-shift-day cells and combine the
data to create a panel. Regress the log of defect rates on the log of employee
absences, controlling for department-shift and day fixed effects.
\i_
'ICF
111
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
To test implications for warranty payments:
• Regress warranty payments for a particular car on the number of defects that
occurred during assembly. Include week-of-production fixed effects.
Statistical Regression with fixed effects
methods used
Results To test overall learning patterns:
• Using the basic model specification, the estimated learning rate was -0.284
(s.e.=0.008) and -0.301 (s.e.=0.006) using weekly and daily data, respectively; a
doubling of cumulative output led defect rates to fall by 17.9% (2 ° 284=0.821) and
18.8% (2"0301=0.812) using weekly and daily data, respectively.
• Adding the time trend did not change the estimated coefficients much.
• When allowing for forgetting, the estimated retention rate was .965 (weekly data)
and .985 (daily data, which was .927 when compounded over a 5-day production
week). Approximately 3% to 7% of the plant's production experience stock was
lost every week.
To explore the driving mechanisms of learning by doing:
Adding a Second Shift:
• The estimated learning rates were smaller for the second shift than the first shift.
Using weekly data, the estimated learning rate was -0.318 (s.e.=0.011) and -0.148
(s.e.=0.010) for the first and second shift, respectively. A doubling of cumulative
output led defect rates to fall by 19.8% (2 ° 318=0.802) and 9.7% (2 ° 148=0.903) for
the first and second shift, respectively.
• The estimated coefficient on second shift ramp-up dummy variable was 0.151
(s.e.=0.058); indicating that first-shift defects were roughly 15% higher during the
weeks the second shift was ramping up.
Introducing Additional Product Variants:
• The estimated learning rate was -0.331, -0.188, and -0.214 using weekly data for
Model 1, 2, and 3, respectively. (A doubling of cumulative output led defect rates
to fall by 20.5%, 12.2%, and 13.8%, respectively). The estimated learning rate was
-0.355, -0.204, and -0.236 using daily data for Model 1, 2, and 3, respectively. (A
doubling of cumulative output led defect rates to fall by 21.8%, 13.1%, and 15.1%,
respectively).
• The ramp ups for Model 2 and 3 led to an 8% and 30% increase in defects rates in
Model 1 production, respectively (using daily or weekly data).
To test station-level patterns:
Distribution of defects:
• Defect rates were highly skewed across stations.
• All quantiles featured considerable reductions in relative defects early in the
production period that decelerated over time.
Persistence:
• Defect rates at the station level were quite persistent. Still, the share of defects
accounted for by top-quintile stations shrunk between the first and second
period.
\i_
'ICF
112
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
• The persistence of error rates was evident from the transition matrix. Persistence
was greatest at the very top and bottom of the distribution.
• The correlation across shifts indicated that defect rates were persistent across
shifts within a week. Again, the persistence was greatest for stations at the
distribution's tails.
To test defect spillovers across cars:
• Spillovers clearly existed as defects on one car raised the likelihood of defects on
the cars that came later in line. Defects on one car had statistically significant
spillovers on at least the next 15 cars.
• The magnitude of the spillovers fell as the distance between cars grew.
• The economic size of these spillovers was nontrivial.
To test absences and the role of worker-embodied learning by doing:
• The estimated coefficient was 0.156 (s.e.=0.029), which implied a one-standard
deviation increase in absences raises defect rates by about l/7th of a standard
deviation.
To test implications for warranty payments:
• There was a positive relationship between defects on a car and the amount of
warranty payments the company made on it.
Assessment To test overall learning patterns:
• The daily and weekly specifications fit the data very well (high R2 values).
• The results with the time trend suggested quality improvement is related to
production activity rather than to the passage of time.
• Explicitly modeling forgetting does not substantially improve the ability of the
model to fit the data relative to controlling for a time trend.
• Evidence from independent quality control audits supports the authors' findings.
To explore the driving mechanisms of learning by doing:
Adding a second shift:
• The estimated learning rate was significant for the first and second shifts using
weekly data, but significant for only the first shift using daily data.
• The patterns indicate that the efficiency and quality gains from the first period
seem to be fully incorporated into second shift production immediately, despite
having new workers. This suggests learning is embodied in the broader
organization rather than in human capital.
• The estimated coefficient on the second shift ramp-up variable was only
significant using daily data.
Introducing Additional Product Variants:
• The estimated learning rate for Model 1 was similar to those found in the overall
sample. The estimated rates for Models 2 and 3 were smaller.
• The ramp up periods of subsequent models had a significant relationship with
Model l's defect rates but not Model 2's. This may be because solving problems
that arise as Model 3's production begins detracts more resources from
production of the most similar model, Model 1.
\i_
'ICF
113
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
• Knowledge stocks are not accumulated simply by workers producing any type of
car. Learning depends not only on workers becoming acclimated to working
together, but also on the similarity of the products being produced. Workers who
have acquired experience producing one model cannot fully transfer the
knowledge to the production of other models. This also suggests learned
knowledge is not simply contained in human capital, but may be embodied in
physical or organizational capital.
To test station-level patterns:
Distribution of defects:
• Aggregate learning by doing reflects a proportional tightening of the entire
station-level defect rate distribution, with all quantiles experiencing similar
percentage declines in defect rates.
• The results further indicate that an important component of learned production
knowledge appears to be tied to the particular capital of the station, the
organizational capital managing that station across shifts, or temporal
fluctuations in the quality of parts being used as inputs.
To test defect spillovers across cars:
• It did not appear that a decrease in spillover effects over time explained the
observed learning by doing patterns.
Learning by doing is an important factor in the production process, particularly for
the first few months of production after the initial ramp up.
Efficiency/quality gains seem to be immediately fully incorporated into the
second-shift production, despite having completely new workers.
Introducing a new model variant into production causes productivity setbacks for
those already in production.
The distribution of defect rates across the assembly processes is highly skewed.
Station-specific defect rates persist over time and across shifts.
Defects spill over to the cars following on the assembly line (albeit at a declining
magnitude). These effects do not decline over the year.
Worker absenteeism is related to defect rates, both directly and through the rate
at which acquired learning-by-doing knowledge is retained. But the impact is
economically small.
Defects per vehicle are related to warranty payouts by the firm.
Several findings suggest that productivity gains from learning are embodied in the
broader organization rather being retained within the human capital of workers
(supports the findings by Epple et al. (1996)).
Future Extend their research to other production operations,
research
Other notes Definitions of terms used in the article:
• Ramp-up period - the first 3 weeks of second-shift production, the time it took
second-shift output to rise to the level of the first.
• Department - a major portion of an assembly line's operations
Applicability This study did inform EPA's learning rate estimate because it is related to the mobile
of results source sector, uses primary data, and estimates the progress ratio based on average
defects.
Conclusions •
\l_
'ICF
114
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Themes Estimation of learning rate, Determinants of learning by doing, Location of knowledge
(e.g., embedded in technology)
\i_
'ICF 115 September 30,2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Macher, J. T., & Mowery, D. C.
Article
"Managing" learning by doing: An empirical study in semiconductor manufacturing
Publication
The Journal of Product Innovation Management, Vol. 20, No. 5, pp. 391-410
Date
2003
Industry
Semiconductor industry, wafers of silicon
examined
Research
question(s)
Type of
learning
examined
Data sources
• How does semiconductor manufacturers' use of teams for problem solving and
intrafirm knowledge transfer influence performance?
• How does the level of internal adoption of information technology (IT) influence
performance?
• How does more extensive and effective workflow and production scheduling
systems influence performance?
Organizational-based learning by doing; Problem solving for production improvement;
Location of knowledge
Competitive Semiconductor Manufacturing Program
Data size Data from 36 wafer fabrication facilities of U.S., European, Japanese, Korean, and
Taiwanese semiconductor firms operating domestically and offshore
Data years Multiple years (not specified); some firms have only 2 years of data
Data • The wafer size variable was normalized to the industry standard,
adjustment • Unequal weights were assigned to the Materials Handling variable to reflect that
interbay materials handling automation is more complex and potentially more
valuable to performance improvement.
• Conveyor systems receive a higher weight because industry experts considered
them to be more important than systems that load/unload production lots.
• The variable, Cycle Time per Layer, was normalized according to the number of
mask layers required by the device to not penalize product types with larger
areas.
• The variable, Cumulative Volume, is scaled to represent units of 1,000 wafer
starts.
Methodology The authors estimated the following equation:
(10) Pt = Y + e-[fi1- CVt + HRP ¦ + a2 ¦ CVt) + OP ¦ (a3 + a4 ¦ CVt)] ¦
[Po ~ Y ~ (foLW + fcWS + (34CR + p5ML)] + p2LW + p3WS + /34CR +
PsML
Where,
Pt - defect density or cycle time parameter
CVt - cumulative volume; the sum of wafer starts from the initial observation
to the current period (scaled to represent units of 1,000 wafer starts)
HRP - the knowledge gained by implementing a particular human resource
(HR) practice in the fab. These practices include:
• Team Diversity - the degree to which both direct (i.e., operators and
technicians) and indirect (i.e., engineers and supervisors) personnel are
involved in problem-solving activities within the manufacturing facility
• Team Number - measures the diversity of problem-solving team types
\i_
'ICF
116
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
operating in the facility
• Colocation - measures whether manufacturing engineers are
transferred to development and/or development engineers are
transferred to manufacturing
OP - the knowledge gained by implementing a particular organizational
practice in the fab. These practices include:
• Material Handling - the extent of use of automated material handling
in critical functions
• Information Handling - the extent and use of automated information
handling
• Database Analysis - the extent of use of integrated database analysis
of production performance and problem solving
• Scheduling - the extent of use of production scheduling systems
Static Variables
LW - linewidth of the manufacturing process; a measure of technological
sophistication
WS - normalized dimension of wafers manufactured
CR - the maximum clean room grade that exists in the fab; a measure of the
number of particles per cubic foot in the fabrication facility
ML - number of mask layers used in the process; a proxy for the total number
of steps in the process
Statistical All models are estimated using a nonlinear maximum likelihood estimator using a first-
methods used order (AR1) correction for serial correlation. Fixed effects for each manufacturing
facility are included in the estimation.
Results Using Cycle Time as the dependent variable; examines the effect on the speed of
production (each model also includes the static variables):
• Model 1 (CV) - As production increases, cycle time performance improves.
• Model 2 (CV, Team Diversity, Team Number, Colocation, and their respective
interaction terms)
o Cumulative volume shows a positive and significant relationship with cycle
time performance.
o Team Diversity, Team Number, and Colocation appear to have a significant
direct effect on cycle time performance. They initially shift the cycle time
learning curve up, implying some cycle time penalty associated with their
use.
o Team Number and Colocation appear to have a significant indirect effect on
cycle time performance. Their use accelerates performance in the face of
new processes.
• Model 3 (CV, Material Handling, Information Handling, Data Analysis, Scheduling,
and their respective interaction terms)
o Material Handling, Information Handling, Data Analysis, and Scheduling
appear to have a significant direct effect on cycle time performance.
¦ Material Handling shifts the cycle time learning curve down, initially
improving cycle time.
¦ Information Handling, Data Analysis, and Scheduling initially shift the
cycle time learning curve up, implying some cycle time penalty
associated with their use.
\i_
'ICF
117
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
o Material Handling, Information Handling, Data Analysis, and Scheduling
appear to have a significant indirect effect on cycle time performance.
¦ Material Handling produces slow rates of cycle time improvement as
production volumes grow.
¦ Information Handling, Data Analysis, and Scheduling increase the rate of
cycle time improvement as production volumes grow.
• Model 4 (combines HR and technological practices)
o All of the HR and technological practice variables appear to have a significant
direct effect on cycle time performance.
¦ Team Diversity and Material Handling initially shift the cycle time
learning curve down, improving cycle time.
¦ Team Number, Colocation, Information Handling, Data Analysis, and
Scheduling initially shift the cycle time learning curve up, implying some
cycle time penalty associated with their use.
o Colocation, Material Handling, and Information Handling appear to have a
significant indirect effect on cycle time performance.
¦ Material Handling produces slow rates of cycle time improvement as
production volumes grow.
¦ Information Handling, Data Analysis, and Scheduling increase the rate of
cycle time improvement as production volumes grow.
Using Die Yield (Defect Density) as the dependent variable; examines the effect on the
rate of learning (each model also includes the static variables):
• Model 1 (CV) - Yield performance improves as cumulative volume increases.
• Model 2 (CV, Team Diversity, Team Number, Colocation, and their respective
interaction terms)
o Cumulative volume shows a positive and significant relationship with cycle
time performance.
o Team Number and Colocation appear to have a significant direct effect on
yield performance. They initially shift the yield improvement learning curve
up, implying some yield penalty associated with their use.
o Team Number and Colocation appear to have a significant indirect effect on
yield performance. Their use accelerates performance in the face of new
processes.
• Model 3 (CV, Material Handling, Information Handling, Data Analysis, Scheduling,
and their respective interaction terms)
o Material Handling, Information Handling, Data Analysis, and Scheduling
appear to have a significant direct effect on yield performance. They initially
shift the yield improvement learning curve up, implying some yield penalty
associated with their use.
o Material Handling and Information Handling appear to have a significant
indirect effect on yield performance.
¦ Material Handling produces slow rates of yield improvement as
production volumes grow.
¦ Information Handling accelerates performance in the face of new
processes.
• Model 4 (combines HR and technological practices) - Colocation and Information
\i_
'ICF
118
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Handling are still significant,
o Team Number, Colocation, Material Handling, Information Handling, and
Data Analysis appear to have a significant direct effect on yield performance.
They initially shift the yield learning curve up, implying some yield penalty
associated with their use.
o Material Handling and Information Handling appear to have a significant
indirect effect on yield performance.
¦ Material Handling produces slow rates of yield improvement as
production volumes grow.
¦ Information Handling accelerates performance in the face of new
processes.
Assessment • Each successive model is a statistically significant improvement in comparison to
the initial model and produces largely consistent results.
• The authors found the lack of influence of an integrated data analysis capability
on yield improvement surprising given how important data analysis is viewed in
yield management. The authors stated it may reflect that the levels of investment
in this type of IT are less important than the details of its organization and
deployment within the fab, aspects of IT investment that their measures captured
imperfectly.
• The authors were not surprised that automated scheduling systems of yield lacked
influence because production scheduling affects production volumes and queues,
which are more important for cycle time than yield improvement.
Conclusions • The results obtained generally show similar effects from the implementation for
HR and organizational practices on yield and cycle time performance.
• The introduction of several of the practices initially have negative influence on
manufacturing performance at low production volumes, but tend to increase the
rate of improvement as production volumes expand.
• Manufacturers that implement more types of problem-solving teams and policies
that collocate production and development engineers and other key personnel
appear to learn faster by making better use of tacit knowledge typically "locked
up" within individual engineers or operators.
• Firms with superior information handling automation and data analysis
capabilities can improve yield or cycle time faster. These practices support higher
levels of codification of otherwise tacit knowledge, which facilitates internal
dissemination and accelerates firm-wide learning.
• There is little or no evidence of significant benefits from other practices, nor do
these activities affect all dimensions of performance equally.
Future
research
Other notes
Applicability
of results
N/A
Definitions of terms used in the article:
• Die yield - the proportion of die on a successfully processed wafer that pass
functionality tests (measure of quality)
• Cycle time - the time required to manufacture a semiconductor device (shorter
cycle times allow plants to boost output or adjust more quickly to changes)
This study did not inform EPA's learning rate estimate because it does not use the
power form to estimate learning; hence, progress ratios cannot be estimated using
their results.
\i_
'ICF
119
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Themes Determinants of learning, Location of knowledge (e.g., embedded in technology)
\i_
'ICF
120
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Nykvist, B. & Nilsson, M.
Article
Rapidly falling costs of battery packs for electric vehicles
Publication
Nature Climate Change
Date
2015
Industry
Battery electric vehicles (BEVs)
examined
Research
What are the current and predicted costs of battery packs?
question(s)
Type of
Learning in general; Forecasting costs
learning
examined
Data sources
The authors specify that the cost estimates "are from peer reviewed papers in
international scientific journals; the most cited grey literature, including estimates by
agencies, consultancy and industry analysts; news items of individual accounts from
industry representatives and experts; and finally, some further novel estimates for
leading BEV manufacturers" (p. 329).
Data size
The authors collected 85 cost estimates to be assessed for historical costs and 37 for
future forecast costs.
Data years
2007-2014
Data
• The authors collected data and eliminated cross referrals and duplicative data
adjustment
points. They also excluded data that did not specify the method used.
• For all data, costs ranges (if given) were converted to the arithmetic mean of the
highest and lowest data points in the range.
• Historical costs were inflation adjusted to US$2014 using data from the Bureau of
Labor Statistics.
• Currencies were converted using historical exchange rates from the US Federal
Reserve.
• Cumulative battery pack volumes were assessed by combining several sources in
press releases for car manufacturers, data provided by actors following the
industry, and data found in individual reports.
Methodology
• The authors performed a secondary analysis of over 80 estimates reported by
other analyses between 2007 and 2014 to systematically trace the cost of Li-ion
battery packs for BEV manufacturers.
• The authors estimated the annual percent change in average costs between 2007
and 2014.
• Learning rates were estimated by regression of log cost data on log cumulative
output data using four data points modelled from this paper for 2011-2014
separately for industry as a whole, the market-leading manufacturers, and the net
of both groups (excluding market leaders).
Statistical
• Data were fitted with log regression and 95% confidence intervals, derived with a
methods used
two-tailed £-test
Results
• Average costs for the industry as a whole declined by 14% annually and average
costs for market-leading manufacturers declined by 8% annually over 2007 to
2014.
• The estimated cost range in 2014 was $410/kWh and $300/kWh for industry and
market-leading manufacturers, respectively.
\i_
'ICF
121
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
• The estimated learning rate was 9% for industry as a whole and 6% for market-
leading actors. This implies the cost reduction following a cumulative doubling of
production falls between 6% and 9%.
• Costs in 2014 were probably already below average projected costs for 2020.
• The cost estimates for the whole industry and market-leading cost manufacturers
are estimated to converge in 2017-2018 at around $230/kWh.
Assessment • All estimated declines in costs are significant.
• The learning rate is in line with earlier studies on vehicle battery technology.
• Cost data contained too much uncertainty to estimate learning rates directly.
Modeled data from this paper was used instead, which gave highly significant
results, but the underlying uncertainty in cost data must be taken into account
when interpreting the results.
• All of the results come with large uncertainties.
• Sparse data makes statistical testing difficult. Learning rates were initially
calculated by regressing log cost data and log cumulative capacity data; however,
because the cost data's uncertainty was too high (i.e., the R2-value was less than
0.1), the authors used modelled data from this paper, which consisted of only four
data points, for the period 2011-2014.
• The estimated current cost range in 2014 is two to four times lower than
suggested in many recent peer-reviewed papers.
• Industry may have incentive to overestimate costs to avoid revealing actual costs
or conversely, that they subsidize battery packs to gain market shares.
• The price range is widened as cost estimates are based on many cell chemistry
varieties.
• The estimated cost when industry and market-leading manufacturers converge
($230/kWh) is lower than estimates in peer-review literature, but on par with
other estimates (McKinsey Quarterly, 2012).
Conclusions • The literature reveals that costs of battery packs are decreasing, but with large
uncertainties on past, current, and future costs of the dominating Li-ion
technology.
• Industry-wide cost estimates declined approximately 14% annually between 2007
and 2014. To some degree, this represents a correction of earlier, overestimated
costs.
• The costs of battery packs used by BEV manufacturers are lower and declined by
8% annually between 2007 and 2014. This decline likely represents the probable
future cost improvement for Li-ion battery packs in BEVs.
• The learning rate is 6% (for market-leading actors) and 9% (industry wide).
Future Future research efforts modeling scenarios for energy and transport transitions need
research to take these lower cost estimates into account.
Other notes Possible explanations for the steep decline in industry-wide cost estimates:
• The inclusion of data on market-leading actors
• Cumulative global sales of BEVs are doubling annually, and learning rates for the
constituent Li-ion cells have been estimated to be 16%-17% (Gerssen-Gondelack
& Faaij, 2012).
• Improvements made to input material cost and economies of scale
• The period since 2007 represents the earliest stage of sales growth for BEVs. The
estimates thus reflect a wide range of Li-ion battery variants at initially low
\i_
'ICF
122
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
production volumes as well as necessarily immature battery pack production
techniques among BEV manufacturers. A rapidly developing and restructuring
industry in its early phase could yield high learning rates at pack level.
Applicability
of results
Themes
This study did not inform EPA's learning rate estimate because the study used
secondary data and estimates were based on only four data points.
Estimating learning rates, Use of learning curves in forecasting
\i_
'ICF
123
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Rubin, E. S.
,, Taylor, M. R., Yeh, S., & Hounshell, D. A.
Article
Learning curves for environmental technology and their importance for climate
policy analysis
Publication
Energy, Vol. 29, No. 9, pp. 1551-1559
Date
2004
Industry
Electric power plants; flue gas desulfurization (FGD) and selective catalytic reduction
examined
(SCR) systems to control S02 and NOx
Research
• How did the deployment and cost of these environmental technologies change
question(s)
over time?
• How were these changes and technological innovations related to government
actions and policies?
Type of
Learning generally
learning
examined
Data sources
Data from a 2001 PhD thesis by Taylor, based on a series of studies performed by the
same organizations over a period of years using a consistent set of design premises.
Data size
Data size not provided
Data years
Data years not specified, although results are provided for 5 data years for each system
(i.e., FGD systems: 1976, 1980, 1982, 1990, and 1995; SCR systems: 1983, 1989, 1993,
1995, and 1996)
Data
Costs are adjusted to a common basis for a standardized 500 MW power plant burning
adjustment
3.5% S coal with wet limestone FGD systems achieving 90% S02 removal.
Methodology
Regress cost on cumulative production using the following functional form; log-linear
format to allow linear regression.
Learning curve: yt = ax[b
Where,
y - cost to produce a unit
x - cumulative production
learning rate = l-2"b
progress ratio = 2"b
In this approach, cumulative production or capacity is a surrogate for total
accumulated knowledge gained from many different activities whose individual
contributions cannot be readily discerned or modeled. The model includes both
benefits from "learning by doing" and R&D investments that produce new knowledge
and new generations of technology. The authors asserted that it would be ideal to
distinguish R&D impacts, but data limitations prevent this. The model also precludes
the impacts of government regulations.
Statistical
Statistical methods used not specified
methods used
Results
S02: yi = a - 1.45x(r°'17; learning rate: 11%; progress ratio: 89%
NOx: yi = a - 1.28x(r°18; learning rate: 12%; progress ratio: 88%
\i_
'ICF
124
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Assessment
S02:
• The importance of R&D programs for process improvements may be significant.
• Increased competition among vendors may have an impact on costs.
• Uncertainties include the functional form (i.e., the assumption of a constant
learning rate).
NOx:
• There were significant improvements in catalyst manufacturing methods as well
as increased competition, although there was no significant change in the price of
precious metals.
Conclusions
Learning rates and progress rates are similar for both pollutants and are similar to
other estimates for a wide range of market-based technologies.
Future
research
• Explore the impact of different policy scenarios.
• Longer time horizon
Other notes
N/A
Applicability
to results
This study did not inform EPA's learning rate estimate because it is based on a limited
number of data points (i.e., five data points each for FGD and SCR systems).
Themes
Learning rate estimations, Environmental technologies
\i_
'ICF
125
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Shinoda, Y.
, Tanaka, H., Akisawa, A., & Kashiwagi, T.
Article
Evaluation of the plug-in hybrid electric vehicle considering learning curve on battery
and power generation best mix
Publication
IEEJ Transactions on Power and Energy, Vol. 129, No. 1, pp. 84-91
Date
2009
Industry
Plug-in hybrid electric vehicles (PHEVs)
examined
Research
• Given different scenarios: How widely used will PHEVs be in the future? How
question(s)
much will the introduction of PHEVs reduce C02 emissions? Will there be serious
effects on the power supply system?
• Demonstrate an ideal scenario for PHEV introduction that minimizes the total cost
in the passenger car sector and the power supply sector.
• Estimate beneficial effects, including the reduction of C02 emissions.
Type of
Learning in general; Forecasting costs
learning
examined
Data sources
• Survey from METI Clean Diesel Passenger Car Study Council (2004)
• MLIT Road Traffic Census (1999)
• ANRE Power Development Outline (2006)
• Comparative Cost of Power Generators by Model Calculation, ANRE Advisory
Committee for Energy (2004)
• Kaino, K. Power generation mix models and cost comparison. (2003)
• Iwafune, Y. Comprehensive evaluation of C02 emission countermeasures in
private sector, doctoral thesis (2000).
• TENPES: Directory of Thermal and Nuclear Power Generation Facilities (2005)
• Oda, T., Akisawa, A., & Kashiwagi, T. Method to estimate long-term change of heat
and electric power daily load curves in Japan. IEEJ Trans PE (2005)
• AIRIA: Number of Vehicles by Year of Registration, 1970-2006
• NPA Police White Paper: Number of Licensed Drivers by Age and Sex, 1970-2004
• Website of IPSS: Population Projection for Japan (2006)
• METI: Recommendations for the Future of Next-Generation Vehicle Batteries
(2006)
Data size
Unspecified
Data years
2010-2035 (based on projections)
Data
None
adjustment
Methodology
The authors extend the linear model they previously developed (2008) which
integrates power supply and passenger car models. The model is extended by allowing
for renewals of car types and power sources as well as cost reductions due to the
battery learning effect.
The authors set up an objective function to be minimized, which is defined as the sum
of the fixed and variable costs throughout the period. The objective function involves
costs in the passenger car and power supply sectors.
In the power supply sector, the authors consider the possible construction of new
\i_
'ICF
126
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
nuclear power plants, integrated coal gasification combined cycle plants, advanced
LNG combined cycle plants, oil-fired thermal power plants, and pumped storage plants.
Fixed costs include repair, labor, and initial construction costs.
In the passenger car sector, the authors consider three car sizes (normal, small, and
mini) and 11 categories differing by annual mileage. They consider four car types
(gasoline vehicles, diesel vehicles, and gasoline hybrid electric vehicles (GHEVs), and
PHEVs). The fixed cost is the initial cost divided by the age of service. For GHEVs and
PHEVs, the battery cost is taken into account in the case of battery replacement.
The authors incorporate the learning curve as follows. A certain battery cost is set as
the initial value in each of the 5-year intervals and the integrated model is optimized.
Then the battery cost is found from the cumulative battery quantity and the learning
curve. The integrated model is optimized again using the new battery cost. This
procedure is repeated until the difference in battery cost before and after optimization
drops below some small value.
Statistical The authors use a multiyear extension of the linear integrated model of the power
methods used supply sector and passenger car sector and incorporating the learning effect of
batteries by iterative calculations. The model minimizes the objective function, which is
defined as the sum of fixed and variable costs throughout the period.
Results The authors estimated the progress ratio to be 70% based on a regression analysis of
actual data (using cumulative production and price).
Assessment N/A
Conclusions • For PHEVs to be accepted in 2030 as a standard passenger car type, the battery
type in the first 5-year interval must be about 132,000¥ if batteries are not
replaced, and about 125,000¥ if batteries are replaced.
• If the official target of 100,000 ¥/kWh for the battery price in 2010 is achieved,
the share of PHEVs among all new cars in Japan can exceed 60% in 2030 in the
case of no-replacement. In this case, there is hardly any effect on the power
supply construction schedule, but charging power requires an increase in power
output of 2.3%.
• Total C02 emissions in the passenger car and power supply sectors in 2030 can be
reduced by about 100 Mt due to PHEV acceptance, even under limitations on the
construction of nuclear power plants.
Future
Extend the evaluation by considering other factors such as financial subsidies and C02
research
constraints.
Other notes
N/A
Applicability
This study did not inform EPA's learning rate estimate because it the authors used
to results
price as a dependent variable, which affected by market dynamics; hence, it is often
out of the organization's control and is affected by many other variables.
Themes
Estimation of the progress ratio, Application of the learning curve
\i_
'ICF
127
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
\i/
'ICF
128
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Appendix C. Summaries of Articles Related to the Mobile
Source Sector that Received a Cursory Review
Contents
Alchian, A 130
Argote, L., Beckman, S. L., & Epple, D 132
Bailey, M. N., Farrell, D., Greenberg, E., Henrich, J.-D., Jinjo, N., Jolles, M., &
Remes, J. (McKinsey Global Institute) 133
Fisher, M. L., & Ittner, C. D 135
Fisher, M., Ramdas, K., & Ulrich, K 137
Haunschild, p. R., & Rhee, M 139
Jaber, M. Y., Goyal, S. K., & Imran, M 141
Kim, I. & Seo, H. L 142
Levin, D. Z 143
MacDuffie, J. P., Sethuraman, K. & Fisher, M. L 145
Randall, T., & Ulrich, K 147
Rapping, L 148
Thompson, P 149
Thompson, P 150
Thornton, R. A., & Thompson, P 151
Tsuchiya, H. & Kobayashi, 0 152
sl_
'ICF
129
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Alchian, A.
Article
Reliability of progress curves in airframe production
Publication
Econometrica: Journal of the Econometric Society, Vol. 31, No. 4, pp. 679-693
Date
1963
Industry
examined
Airframe production
Research
question(s)
• How long do labor costs decrease as the number of items produced increases?
• Can learning be represented by a linear function on a double-log scale?
• Does the reduction in labor costs fall at the same rate for different airframe
manufacturing facilities?
• How reliably can one predict marginal and total labor requirements for a
particular production facility from an industry average progress curve derived
from the experience of all airframe manufacturers?
• How reliably can a curve fitted to the experience of all bomber (fighter)
production predict labor requirements for a specific type of bomber (fighter)
produced in a particular facility?
• How reliable is a single manufacturing plant's own early experience for
predicting its later requirements for producing a particular type of airframe?
• What are the consequences of the margins of error involved in these
estimating methods?
Type of learning Learning by doing
examined
Methodology; Quantitative. The author estimated the average error of prediction that would
Quantitative or occur if learning curves were fitted to the past performance of a facility in order to
Qualitative? predict the facility's future requirements. The statistical methods included: (1) a
visual examination of graphs, (2) analysis of variance tests, (3) tests whether the
samples from each category (i.e., bombers, trainers, and fighters) are from
populations with equal slopes, and (4) fitting specific progress curves to past
performance of a facility.
Results • Based on a visual examination of the graphs in the Source Book of World War
II Basic Data; Airframe Industry, Vol. I, there is no evidence of any cessation in
the decline in labor costs as the number of items produced increased, but the
author stated he could not determine whether the decline would stop for a
substantially larger number of items produced.
• A linear function on a double-log scale is appropriate for a progress curve.
• The progress curve slope or height is not the same for all model-facility
combinations (MFCs). The relationships differ in slope and height even among
the various facilities producing the same general type of airframe. Hence,
individual MFCs do not have the same progress functions.
• Using an industry-wide average progress curve, the absolute differences
between predicted and actual values average 25% of the actual.
• Using a general airframe-type progress curve, the weighted average of the
errors was 25% (i.e., the ratio of the difference between predicted and actual
values to the actual). Hence, there is no significant difference between the
\i_
'ICF
130
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
average learning coefficients by airframe type.
• The average margin of error using a build-up progress curve is about 22%.
Conclusions Before making decisions based on costs from predictions formed using historical
data, researchers should investigate the range of uncertainty in the prediction.
Themes Application of learning curves, Specification of the learning curve
\i_
'ICF
131
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Argote, L.,
Beckman, S. L., & Epple, D.
Article
The persistence and transfer of learning in industrial settings
Publication
Management Science, Vol. 36, No. 2, pp. 140-154.
Date
1990
Industry
U.S. wartime ship production
examined
Research
• Does learning persist within organizations?
question(s)
• Does learning transfer across organizations?
Type of learning
Organizational learning by doing; Knowledge depreciation; Knowledge transfers
examined
across organizations
Methodology;
Quantitative. The authors used monthly data from the production of Liberty ships
Quantitative or
during WWII to estimate knowledge depreciation and knowledge transfers across
Qualitative?
shipyard by regressing production functions using maximum likelihood. The authors
used tonnage as the outcome variable and cumulative output as one of the
independent variables. The authors also used a calendar time separate the
relationship between cost and technical progress associated with the passage of
time from those associated with increasing cumulative output.
Results
• The monthly depreciation parameter ranges from .70 to .85, which implies
that from a stock of knowledge available at the beginning of a year, only 1% to
14% of the stock would remain at the end of the year (=.7012) and (=.8512).
• When a calendar time variable was introduced to the model, its negative
coefficient indicates that the passage of time is not responsible for
productivity improvements in shipbuilding.
• There is no evidence of learning transfers.
• Shipyards with later start dates were more productive than yards with early
state dates.
• Yards benefited from production at other yards up to their begin date.
Conclusions
• There was evidence that knowledge acquired from learning by doing
depreciated: recent output was a more important predictor of current
production than output from the more distant past.
• There was evidence that learning transfers across organizations: organizations
beginning production later were more productive than those with earlier start
dates. That is, knowledge from the shipyards that began production early
benefited those with later start dates.
• Once organizations begin production, however, they did not appear to benefit
from learning in other organizations.
Themes Knowledge depreciation, Knowledge transfers across organizations
\i_
'ICF
132
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Bailey, M. N., Farrell, D., Green berg, E., Henrich, J.-D., Jinjo, N., Jolles,
M., & Remes, J. (McKinsey Global Institute)
Article
Increasing global competition and labor productivity: Lessons from the US
automotive industry
Publication
Paper presented at the Federal Reserve Bank of San Francisco conference,
"Productivity Growth: Causes and Consequences" (Nov. 18-19th, 2005)
Date
2005
Industry
US automotive manufacturing; specifically, production of new vehicles in the US,
examined
including parts assembly
Research
• How did the Big Three US original equipment manufacturers (OEMs) (i.e.,
question(s)
GM, Ford, and Chrysler) respond to the changed competitive environment?
• How did the Big Three overcome barriers to compete or fail to do so?
• How did the Big Three's introduction of process and product innovations
drive productivity growth?
• How has regulation directly impacted on measured productivity and how has
it influenced the competitive dynamics.
• How does global competition change domestic sector dynamics and
productivity growth?
• How quickly do these changes occur and what factors determine the speed
of adjustment?
• What is the impact on stakeholders?
• What can policy makers and companies elsewhere learn from the US auto
sector experience?
Type of learning
Unspecified, but described learning in general. This study focuses on how increasing
examined
global competition leads to productivity growth
Methodology;
The authors used actual data to derive the relative contribution of OEMs and parts
Quantitative or
to productivity growth (Total productivity growth is the sum of the contributions).
Qualitative?
In a case study, the authors attributed the increases in productivity growth to
specific actions taken by the OEMs.
Results
The authors found that nearly 45% of the productivity increase was driven by the
Big Three's adoption of improved process technology; 25% came from the shift to
new products with higher value-added per hour worked; and 30% came from
increased features and quality in existing products, more efficient producers, and
process efficiency improvements that have arisen from changes in product mix.
The authors also found that each of the three phases in the evolution of a specific
innovation had a different impact on productivity. The initial phase, which covers
the initial development and introduction of the innovation, had a low impact on
industry productivity. The second phase, adoption and learning, had a moderate
impact on industry productivity. The third phase, penetration, when innovations
become widely adopted within companies, and across the industry, drove
significant changes in productivity.
Conclusions Global competition forced the Big Three to raise labor productivity between 1987
and 2002 by introducing and adopting process and product innovations as well as
\i_
'ICF
133
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
by improving overall vehicle quality.
Themes Decomposing sources of productivity growth, determinants of productivity growth,
industry level, regulation's impact on productivity
\i_
'ICF
134
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Fisher, M.
L., & Ittner, C. D.
Article
The impact of product variety on automobile assembly operations: Empirical
evidence and simulation analysis
Publication
Management Science, Vol. 45, No. 6, pp. 771-786
Date
1999
Industry
Automobile assembly plants
examined
Research
• Which dimensions of product variety affect measures of manufacturing
question(s)
performance such as labor productivity, rework, and inventory?
• What is the magnitude of productivity losses due to product variety?
• Which types of labor are most affected by product variety?
• What are the specific mechanisms through which variety impacts
productivity?
• What is the ability of option bundling and the provision of direct labor slack in
work stations with high product mix variability to minimize the adverse effects
of increased product variety?
Type of learning Although learning is not directly mentioned, this article relates to the debate
examined described in Lapre & Nembhard (2010) regarding whether learning and
performance are improved more through specialized or diversified experience.
Quantitative. Using three data sets (i.e., monthly data, daily data, and cross-
sectional data for work stations) from a GM assembly plant, the authors use
regression to examine the impact of product variety on plant performance. In
addition, the authors perform a simulation analysis of a more general automotive
assembly line to test the impact of option variability on direct labor productivity.
Results • Empirical analyses:
o Using monthly data:
¦ Greater option variety adversely impacts overhead
productivity but not direct labor hours per car.
¦ Paid direct labor hours do not vary significantly with the
number of options.
¦ Greater option variety increases overhead requirements.
¦ Increases in major rework account for the lower labor
productivity in months with higher option variability.
o Using daily data:
¦ The strongest determinant of direct labor hours per car is
downtime in the body shop.
¦ Found no relation between paid direct labor hours and option
content or option variability.
o Using cross-sectional work station data:
¦ Workstations with more variability in option-related work
content have more slack resources to compensate for this
variation.
• Simulation analysis:
o The impact of option variety can be greatly reduced by buffering the
assembly line and bundling options.
Conclusions The authors found that option variability (i.e., the standard deviation in the number
Methodology;
Quantitative or
Qualitative?
\i_
'ICF
135
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
of the eight key options per car in a given month) has a significantly greater
negative impact on productivity than option content (i.e., the average number of
options per car). The authors also found that option variability increases overhead
hours, rework, inventory, and the excess labor capacity assigned to a work station.
But option variability does not significantly impact direct labor hours when labor
slack is provided. The level of option variety has an insignificant impact on direct
labor once the assembly line has been optimally buffered against process time
variability with excess capacity. Bundling options can reduce the amount of buffer
capacity required and random variation is more pernicious to productivity than
product variety.
Themes Automobile industry, Specialized vs. Diversified experience
\i_
'ICF
136
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Fisher, M.,
Ramdas, K., & Ulrich, K.
Article
Component sharing in the management of product variety: A study of automotive
breaking systems
Publication
Management Science, Vol. 45, No. 3, pp. 297-315
Date
1999
Industry
Manufacturers of automotive breaking systems
examined
Research
• What are the key drivers and trade-offs of component-sharing decisions?
question(s)
• How much variation exists in actual component-sharing practice?
• How can this variation be explained?
Type of learning
The authors listed learning as one of the key drivers for component sharing because
examined
the quality and performance of a shared component may be higher than that of a
component designed and produced for unique applications because learning is
associated with increased volume.
Methodology;
Quantitative. The authors identified key costs related to component sharing and
Quantitative or
developed an optimization model to predict the ideal component sharing practice.
Qualitative?
Using the results from the optimization model, they formulated hypotheses about
industrial practice and tested them using data from the automobile industry.
Results
• HI: Front brakes variety is increasing in JRWV, a composite variable based on
the range of weights and total remaining sales volume of all models in the
product line of the manufacturer.
o The number of brake rotors increases with the composite variable.
• H2: Front brakes variety is a decreasing function of the variability in model
volumes.
o The number of brake rotors decreases as the variation in volume
across different models increases.
• H3: U.S. firms exhibit a greater amount of front brakes sharing than do the
Japanese firms in the study.
o Japanese companies share components less than the U.S. companies.
• H4: Front brakes variety is an increasing function of product line variety.
o There is a positive relationship between the number of products and
the number of different brake rotors, which supports the hypothesis
that the number of different components is driven by the number of
different products.
Conclusions • Component sharing is practiced in the industry according to an economic logic
consistent with the authors' analytic model.
• Their results are consistent with the theory that for a given total product
volume, 'lumpiness" in the distribution of this volume gives rise to the
possibility of opportunistically assigning unique rotors to the models with high
volumes, while sharing components across the models with low volumes.
• Japanese firms share less than U.S. firms for three possible reasons: (1) fixed
costs of creating a new rotor may be lower for Japanese firms, (2) Japanese
firms invoked heavyweight project organizations for product development
more frequently than U.S. firms over the course of the study and therefore
\i_
'ICF
137
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
may lack cross-project coordination mechanisms, and (3) Japanese firms had
higher design quality than U.S. firms, which is strengthened by the
optimization of unique components.
• The number of different components is driven by the number of different
products. This may stem from (1) the tendency to design new products from
scratch and (2) the tendency toward more autonomous project teams.
Themes Learning as a cost driver for component sharing
\i_
'ICF
138
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Haunschild, p. R., & Rhee, M.
Article
The role of volition in organizational learning: The case of automotive product
recalls
Publication
Management Science, Vol. 50, No. 11, pp. 1545-1560
Date
2004
Industry
Automakers
examined
Research
• What is the role of volition in organizational learning?
question(s)
• Do firms learn better in response to internal procedures or external
mandates?
Type of learning
Organizational learning
examined
Methodology;
Quantitative. Using National Highway and Traffic Safety Administration data on
Quantitative or
auto makers' recalls, the authors use regression to examine the impact of
Qualitative?
cumulative production on subsequent recalls. In addition, they examine the impact
of cumulative recall experience on subsequent recalls. Cumulative recall experience
was separated into the number of voluntary and involuntary recalls. As an
alternative, the author used the relative proportion of voluntary to involuntary
recalls. Further, the authors test whether involuntary recalls cause shallow
responses. The authors then add measures of generalism and specialism to the
model to see if learning was affected differently due to this structural characteristic.
The models included control variables (e.g., auto maker age, organization size,
dummy variables to capture the effects of different presidential administrations,
level of industry competition, and time trends).
Results
• Do auto makers learn from experience to reduce recalls?
o Production experience reduces subsequent recalls.
• Is subsequent recall performance affected more by voluntary or involuntary
recalls?
o Prior voluntary recalls reduce subsequent involuntary recalls, but there
is no effect of prior involuntary recalls on subsequent involuntary
recalls.
o The higher the proportion of voluntary recalls, the lower the
subsequent involuntary recalls,
o Prior recalls (both voluntary and involuntary) increase subsequent
voluntary recalls.
o There is no effect of proportion of voluntary recalls on the subsequent
voluntary recall rate,
o Learning from involuntary recalls may be shallower and less likely to
penetrate the organization or be stored in organizational memory.
• Do generalists and specialists learn differently from involuntary and voluntary
recalls?
o Generalists do not have higher involuntary recall rates than specialists,
o Generalists with a high proportion of voluntary recalls reduce their
subsequent involuntary recall rates more than specialists,
o Generalists learn more from voluntary recalls than specialists when the
\i_
'ICF
139
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
learning target is reducing subsequent involuntary recalls,
o Generalists have more severe voluntary recalls than specialists,
o Generalists and specialists do not learn differently from voluntary
recalls.
Conclusions Volition is an important determinant of the rate and effectiveness of learning
because voluntary recalls result in more learning than mandated recalls. This is
partly due to involuntary recalls resulting in shallower learning processes. The effect
of volition differs for generalist and specialist auto makers.
Themes Determinants of variation in learning rates, Automotive industry
\i_
'ICF
140
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Jaber, M. Y.,
Goyal, S. K., & Imran, M.
Article
Economic production quantity model for items with imperfect quality subject to
learning effects
Publication
International Journal of Production Economics, Vol. 115, No. 1, pp. 143-150
Date
2008
Industry
Automotive industry
examined
Research
How are the conclusions drawn from the economic order quantity (EOQ) model
question(s)
extended by Salameh and Jaber (2000) affected by learning?
Type of learning
Learning in general
examined
Methodology;
Using data from an automotive manufacturer, the authors used an EOQ model from
Quantitative or
inventory literature, which allows managers to compute their order quantities, that
Qualitative?
was extended by Salameh and Jaber (2000) using the assumption that each lot size
shipment contains a random fraction of imperfect quality items with a known
probability distribution. The authors further extended this model by assuming the
percentage of defective items in a shipment reduces in conformance with a learning
curve (as was observed in practice). The authors created two mathematical models
that optimize profit functions. The first assumes an infinite planning horizon, while
the second assumes a finite one. The authors then applied parameters used by
Salameh & Jaber in numerical examples.
Results
• The results of the model, which assumed an infinite planning horizon, suggest
that the number of defective units, the shipment size, and cost reduces as
learning increases.
• The results of the model, which assumed a finite planning horizon, suggest
that as learning becomes faster, one should order larger lots less frequently.
Conclusions
The authors found that the typical learning curve laid out by Wright (1936) cannot
be viewed as the universal learning curve. The authors state that in practice, an S-
shaped curve may be more appropriate. The S-shaped curve consists of three
phases. The first phase features slow improvement as workers get acquainted. The
most improvement occurs during the second phase. The third phase is the leveling
of the curve. Wright's model would be good for situations with a short first phase.
Themes
Specification of the learning curve (power vs. exponential), Application of the
learning curve, Automotive industry
\l_
'ICF
141
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Kim, 1. & Seo,
H. L.
Article
Depreciation and transfer of knowledge: An empirical exploration of a
shipbuilding process
Publication
International Journal of Production Research, Vol. 47, No. 7, pp. 1857-1876
Date
2009
Industry
Shipbuilding
examined
Research
The authors examined a learning curve model that overcomes the restrictions of
question(s)
period-based depreciation models and measures learning, transfers, and knowledge
depreciation in one integrated framework that is governed by different rules (i.e.,
learning depends on cumulative units produced while knowledge depreciation
depends on (1) the amount of knowledge accumulated and (2) the elapsed time
between when knowledge is acquired and when it is used).
Type of learning
Learning by doing (direct learning); Learning from others (indirect learning or
examined
knowledge transfer), Knowledge depreciation at the organizational level
Methodology;
Qualitative and quantitative. The authors evaluate learning curve using log-linear,
Quantitative or
replacement, and accumulation models in the literature and proposed a new
Qualitative?
learning curve model that captures the acquisition of knowledge and its
depreciation according to their distinction rules. They test their model using the
WWII Liberty ship production dataset.
Results
• The learning rate for the most general model ranges from 0.3197 and 1.5822,
which corresponds with a progress ratio that ranges from 33%-80%.
• The monthly forgetting rate is approximately 26% in all three models, that is
only 74% of knowledge available at the beginning of a month would remain by
the end of the month.
• Production cycle time could be reduced by 38.6%-46.5% through direct
learning.
• Production cycle time could be further reduced by 14.1%-18.7% through
indirect learning.
Conclusions
The authors found that learning by doing is the major source of productivity
growth. Indirect learning's potential contribution to productivity is about 40% of
direct learning's contribution. They also found that knowledge depreciates rapidly
(only 74% of knowledge available at the beginning of a month would remain by the
end of the month). Hence, they conclude that knowledge depreciation and indirect
learning should be included in learning curve model specifications aiming to
estimate production rates and costs.
Themes
Estimated learning rates, Knowledge transfers; Knowledge depreciation, Learning
curve specification
\i_
'ICF
142
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Levin, D. Z.
Article
Organizational learning and the transfer of knowledge: An investigation of quality
improvement
Publication
Organization Science, Vol. 11, No. 6, pp. 630-647
Date
2000
Industry
Automotive industry
examined
Research
• Does a learning curve for quality exist? If so, what form does it take? What factors
question(s)
influence it?
• What happens during new product introduction, before the learning curve starts?
Is there improvement in the "starting points" of learning curves?
• Which type of learning, annual learning curve improvements or annual starting
point improvements, has a greater impact?
Type of
Learning in general; Knowledge transfers
learning
examined
Methodology;
Quantitative Quantitative. The author used panel data on automobile reliability factors (largely
or Qualitative? based on surveys) to estimate learning using ordinary least squares (OLS) regression
with fixed effects. The author used repair rate as the outcome variable and cumulative
output as one of the independent variables. The author then sequentially added the
following independent variables: car model output; a time dummy; and sibling, cousin,
division, firm, and Big Three output. In addition, the author tested for knowledge
depreciation. Because the model's cumulative production output was insignificant, the
author removed that variable and expanded the analysis using combinations of the
following variables: fixed effects, a time dummy variable, a ceiling effect dummy
variable, a year of production variable, interaction terms for the year of production
with Ford and Chrysler dummy variables, a debut year variable, and control variables.
Results HI: Learning curve
• The estimated slope of the learning curve is -0.128 and -0.093 in the third and
sixth year, respectively. A doubling of cumulative output led repair rates to fall by
8.5% (2 ° 128=0.915) and 6.2% (2 0093=0.938) in the third and sixth year,
respectively.
• On average, when a manufacturer has previously produced a lot of cars of a
given model, that model's repair rate is lower.
H2 and H2-ALT: Learning over Time
• Once the author controlled for the average model's repair rate generally
improving each year during its production life, the extent of a manufacturer's
production experience for a particular model appeared to make no difference.
• A year in a model's production life best predicts a model's ultimate repair rate.
• There was no evidence of knowledge depreciation.
• The estimated coefficients for the years of production seem to indicate a gradual
reduction in repair rates for each subsequent year of a model's production life.
• There is some evidence of a ceiling effect.
\i_
'ICF
143
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
H3: Transfer of Production-Based Knowledge
• The data provide no evidence for the transfer of production-based knowledge on
product quality, as measured by repair rates.
• The study found no benefit to a model's repair rate from any measure of
cumulative production experience, not for the model, its siblings, cousins,
division, or firm.
H4: "Debut-Year" Learning
• The later a model begins its production life, the lower its baseline repair rate is.
H5: Debut-Year Learning Versus the Learning Curve
• An extra year of "debut-year" learning leads to better repair rates than does an
extra year of incremental learning during a model's production life. A model's
debut is an enhanced learning event.
Stable learning curves are not limited to the cost or efficiency domain. They can
include quality learning curves.
Some learning curves appear to be more a function of time than a function of
cumulative experience.
Improvements to the starting point of some learning curves, when a product is
first introduced, are even more important than improvements made during
subsequent production.
The results suggest know-how brought in from outsiders does not accumulate as
a function of their production experience, but the results also show that outside
knowledge accumulates with the passage of time. Thus, manufacturers probably
share quality-related knowledge across product families, divisions, and firms.
Hence, there is knowledge transfer.
Themes Learning in general, Knowledge transfers, Timing of learning, Determinants of variation
in learning rates, Automotive industry
Conclusions •
\i_
'ICF
144
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
MacDuffie, J. P., Sethuraman, K. & Fisher, M. L.
Article
Product variety and manufacturing performance: Evidence from the international
automotive assembly plant study
Publication
Management Science, Vol. 42, No. 3, pp. 350-369
Date
1996
Industry
Automotive assembly plant
examined
Research
• What is the effect of increased product variety on manufacturing
question(s)
performance?
• What are the consequence of different product strategies held by Japanese
and US auto manufacturers? (Japanese manufacturers offer more distinct
models, but fewer possible option combinations than U.S. manufacturers.)
• What are the ways in which companies and plants attempt to minimize the
impact of complexity on manufacturing performance?
Type of learning
The impact of plant characteristics and management practices on performance
examined
Methodology;
Quantitative. The authors use multiple product complexity measures derived from
Quantitative or
the International Assembly Plant Study (i.e., model mix complexity, parts
Qualitative?
complexity, option content, and option variability), the production organization
index (i.e., use of buffers, work systems, and human resource management
policies), and control variables (i.e., automation, plant scale, and product design
age) to test the impact of product variety on total labor productivity and quality
using regression analysis.
Results
• The relationship between product complexity measures and productivity:
o Model mix complexity had no statistical significant explanatory power
with respect to productivity.
o Parts complexity, option content, and option variability are
statistically significant.
o Parts complexity and option content had the expected positive signs,
but the coefficient for option variability is negative, which was
unexpected.
• When the authors introduce each individual variable related to product
complexity is introduced into the regression equation instead of the overall
index:
o The production organization index had a strong, statistically significant
impact on productivity in that the more lean a plant was, the more
productive it was.
o The option content measure is no longer significant,
o Lean production policies had little impact on parts complexity.
• When examining the three component indices of the production organization
index (i.e., the use of buffers, work system, and HR management policies)
o The use of buffers index is not statistically significant,
o The work systems and HR management policies are statistically
significant.
Conclusions Most of the product complexity measures did not have a negative impact on labor
productivity or quality. Interestingly, option content had a negative relationship
\i_
'ICF 145 September 30,2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
with productivity, while option variability had a positive relationship. However,
parts complexity did have a persistent negative impact on productivity.
The authors found support that management policies, in operations and human
resource areas, can facilitate the absorption of higher levels of product variety. This
implies that lean production plants (i.e., plants that use ongoing problem-solving
processes on the shop floor and make incremental improvements) are capable of
handling higher levels of product variety with less adverse effect on total labor
productivity than traditional mass production plants (i.e., plants that use extra
inventories or repair space to protect against potential disruptions).
Themes Sources of variations in productivity, Automotive industry
\i_
'ICF
146
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Randall, T., & Ulrich, K.
Article
Product variety, supply chain structure, and firm performance: Analysis of the U.S.
bicycle industry
Publication
Management Science, Vol. 47, No. 12, pp. 1588-1604.
Date
2001
Industry
U.S. bicycle industry
examined
Research
• How does product variety relate to supply chain structure?
question(s)
• How does matching product variety to supply chain structure affect firm
performance?
Type of learning
The development of the authors' hypotheses are based, in part, on two
examined
assumptions related to learning: (1) economies of scale result, in part, through
labor efficiency gains through learning and (2) that product variety exacerbates
production costs when efficiency gains from learning are delayed as resources
alternate focus among multiple products.
Methodology;
Quantitative. Using data from the U.S. bicycle industry (i.e., a buyer's guide and a
Quantitative or
survey on the supply chain structure), the authors tested their first hypothesis using
Qualitative?
ANOVA to compare the mean level of variety across different structural options and
they tested their second hypothesis using ordinary least-squares (OLS) regression.
Results
• Hypothesis 1 tested whether firms using scale-efficient production processes
will have higher levels of production-dominant variety that firms using scale-
inefficient process and whether firms with plants located within target
markets will have higher levels of market mediation-dominant variety than
firms located away from the target market.
o Production-dominant variety is positively associated with scale-
efficient/distant production
o Market-mediation dominant variety is positively associated with scale-
inefficient/local production
• Hypothesis 2 tested whether firms matching production-dominant variety
with scale-efficient production and mediation-dominant variety with local
production outperform firms which fail to make such matches.
o Firm performance is positively associated with correctly matching
supply chain strategies to product variety strategy.
Conclusions
Firms which match their supply chain structure to the type of product variety they
offer outperform firms which fail to match such choices.
Themes
Application of learning, Determinants of variation in learning rates
\i_
'ICF
147
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Rapping, L.
Article
Learning and World War II Production Functions
Publication
The Review of Economics and Statistics, Vol. 47, No. 1, pp. 81-86
Date
1965
Industry
U.S. wartime shipbuilding
examined
Research
What is the role of organizational learning resulting from accumulated production
question(s)
experience?
Type of learning
Learning at the organizational level. (Individual learning is not measured.)
examined
Methodology;
Quantitative. Using data from 15 shipyards, the author estimated the parameters of
Quantitative or
a production function with standard least squares using the log of annual rate of
Qualitative?
physical output as the dependent variable and the log of the annual rate of physical
labor and capital inputs as independent variables. The author tests variations of the
model by adding time variables (i.e., calendar and yard time), cumulated output
variables (i.e., using three varying definitions), and a variable for annual rate of
output of ship types other than Liberties.
Results The author found increasing returns to proportionate changes in labor and capital
inputs. Neither a time variable nor a cumulated output variable could explain away
this finding. Evidence also showed that cumulated output could account for
productivity increases, which the author attributed to learning or adaption. The
author noted the effect of cumulated output on productivity is sensitive to the
definition of cumulated output.
When using calendar time and yard time as the independent variable, each
doubling of time is accompanied by a 23% and 28% increase in the rate of output,
respectively.
When using only cumulated output as the independent variable, each doubling of
time is accompanied by an ll%-29% increase in the rate of output depending on
which measure of cumulated output was used.
When controlling for time, the author found each doubling of cumulated output is
accompanied by a 12%-34% increase in the rate of output (depending on which
measure of cumulative output was used). Thus, estimated progress ratios ranged
from 66% to 88%. The results suggested the cumulated output had a relationship
with productivity independent of other variables correlated with time.
Conclusions The author finds evidence of learning while controlling for time, using various
definitions of cumulated output, and controlling for economies of scale. The paper
advanced the state-of-the -art at the time by controlling for economies of scale.
Evidence of learning was found when economies of scale were controlled.
Themes Estimated learning rate
\i_
'ICF
148
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Thompson, P.
Article
How much did the Liberty shipbuilders learn? New evidence for an old case study
Publication
Journal of Political Economy, Vol. 109, No. 1, pp. 103-137
Date
2001
Industry
U.S. wartime ship production
examined
Research
Do previous learning by doing studies on Liberty ships suffer from omitted variable
question(s)
bias (specifically capital investment and quality changes)?
Type of learning
Learning by doing
examined
Methodology;
Quantitative. Equipped with new data on capital investment from the National
Quantitative or
Archives, Thompson expanded on the work done by Rapping (1965) and Argote,
Qualitative?
Beckman, & Epple (1990). Using seemingly unrelated regression estimation, the
author estimated a temporal production function which incorporated a measure of
all physical capital structures and non-structures whereas the production functions
used by Rapping and Argote et al. only used a subset of structures. The outcome
variable was monthly deliveries per yard and the independent variables used for
experience were either cumulative output or cumulative labor hours. Thompson
estimated the learning rate while controlling for capital investments and quality and
compared his results with those from Rapping and Argote et al.
Results
• Using cumulative output as the independent variable, the authors estimated
the following learning coefficients:
o Rapping: 0.110, which corresponds to a progress ratio of 93%
o Argote et al.: 0.44, which corresponds to a progress ratio of 74%
o Thompson: 0.263-0.493, which correspond to a progress ratio of 71%-
83%
• Using cumulative employment as the independent variable, Thompson
estimates a learning coefficient of 0.208-0.359, which corresponds to a
progress ratio of 77%-87%.
Conclusions Two omissions from previous research led to overestimation of learning rates:
investment in physical capital and variations in product quality. Capital deepening
was more extensive than assumed and part of the increase in productivity came at
the expense of quality, which accounted for 50% and 5% of the increase in labor
productivity, respectively.
Themes Specification of the learning curve, Estimated learning rate
\i_
'ICF
149
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Thompson, P.
i
Article
How much did the Liberty shipbuilders forget?
Publication
Management Science, Vol. 53, No. 6, pp. 908-918
Date
2007
Industry
U.S. wartime ship production
examined
Research
• What is the rate of organizational forgetting of U.S. wartime ship production?
question(s)
• Do unobserved changes in a firm's product mix produce spurious evidence for
organizational forgetting?
• Is the estimated rate of organizational forgetting sensitive to assumptions
made about the learning process?
• Does labor turnover influence productivity?
Type of learning
Organizational forgetting
examined
Methodology;
Quantitative. Using data from the National Archives, the author expanded on the
Quantitative or
quantitative analysis done by Argote, Beckman, & Epple (1990) using regression
Qualitative?
analysis to estimate the depreciation parameter using a (1) a loglinear learning-
forgetting model, (2) accumulation model, and (3) a replacement model. The author
also tested whether labor turnover is correlated with productivity by adding the
recorded rates of labor hiring and separation as a level effect.
Results
• The monthly depreciation rate ranged from 5.8% to 8.4%. When correcting for
the product mix, the monthly depreciation rate ranged from 3.6% to 4.2%.
This implies that 49% to 64% of the knowledge stock at the beginning year
would remain at the end of the year. (=(1-0.058)12) to (=(1-0.084)12)
• When testing for the effect of labor turnover, the author found that increased
labor turnover either has no effect on productivity or raises it.
Conclusions The estimated rate of organizational forgetting was less than in previous studies
analyzing the same data. Argote et al. (1990) estimated a 25% monthly rate of
knowledge depreciation. The author's estimates ranged from 3.6% to 5.7%. The
author found that controlling adequately for changes in a firm's product mix has
significant effects on the estimated rate of organizational forgetting, but the
estimated rate of forgetting was only moderately sensitive to the specification of
the learning curve. In addition, the author found that labor turnover was largely
unrelated to productivity changes. When labor turnover was included in the model,
organizational forgetting did not appear to occur.
Themes Knowledge depreciation, Estimated organizational learning rate, Specification of the
learning curve
\i_
'ICF
150
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Thornton, R.
A., & Thompson, P.
Article
Learning from experience and learning from others: An exploration of learning
and spillovers in wartime shipbuilding
Publication
American Economic Review, Vol. 91, No. 5, pp. 1350-1368
Date
2001
Industry
U.S. wartime shipbuilding
examined
Research
What is the importance of leaning spillovers? Specifically, are learning spillovers
question(s)
sufficiently large for on-the-job learning to be a plausible source of long-run
growth? Are external spillovers larger enough for the suboptimality of production
to be a cause for concern?
Type of learning
Learning spillovers
examined
Methodology;
Quantitative. Using an expanded dataset (including non-Liberty ships and
Quantitative or
shipyards), the authors estimated spillovers by fitting a parametric and a semi-
Qualitative?
parametric production function using ordinary least-squares (OLS) estimation. The
dependent variable is the realized labor requirement. The independent variables
include capital stock, total labor hours, calendar date on which the keel was laid, a
vector of experience consisting of four elements to capture learning and spillover
effects.
Results
• Cross-yard spillovers within the same product design were almost as important
as own-yard learning effects. The potential effect of cross-yard spillovers within
the same product design is estimated to have been about 88% of the potential
effect of own-yard experience.
• The reduction in the unit labor requirements obtained from an extra unit of
experience in the same product design is 5 times as great as the increase
obtained from an extra unit of experience on prior designs.
• The maximum contribution of learning spillovers across yards is found to be
106% of the combined contribution of the two types of within=yard learning.
Conclusions
The authors found that learning spillovers, across products and across yards, were a
significant source of productivity growth. The spillover effects may have been more
important than conventional learning effects. In addition, the size of learning
externalities across yards was small. Together these findings suggest that spillovers
help firms grow, but market failures induced by learning externalities are modest.
In terms of the conventional learning effect, the authors found that the learning
effect is positively sloped and concave. It exhibits rapid rates of learning at early
stages and strong negative effects at higher levels of experience.
Themes
Learning spillovers, Learning externalities
\l_
'ICF
151
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Tsuchiya, H. & Kobayashi, O.
Article
Mass production cost of PEM fuel cell by learning curve
Publication
International Journal of Hydrogen Energy, Vol. 29, No. 10, pp. 985-990
Date
2004
Industry
Proton exchange membrane (PEM) fuel cells for automobiles
examined
Research
Is it possible to reduce the cost of PEM fuel cells through learning?
question(s)
Type of learning
Learning in general
examined
Methodology;
Quantitative (simulation using assigned progress ratios). The authors use a learning
Quantitative or
curve to estimate the future cost reduction in fuel cell stacks due to mass
Qualitative?
production. Using predicted values, the authors constructed nine scenarios with
combinations of power density improvement (three scenarios) and cost reduction
speed (three scenarios). For each of the three cost reduction speed scenarios, the
authors assigned a different progress ratio. Using the nine scenarios, the authors
estimated the resulting cost of the overall fuel cell stack and its components.
Results The following table presents nine scenarios of fuel cell stack costs based on
combinations of power density improvement and cost reduction speed. The
progress ratios for power density improvement and cost reduction speed are in
parentheses.
Scenario (Progress
Ratio)
High Power
Density
(94.5%)
Medium Power
Density (96%)
Low Power
Density
(97.5%)
Rapid (78%)
$88/kW
$103/kW
$121/kW
Moderate (82%)
$143/kW
$167/kW
$196/kW
Slow (88%)
$285/kW
$334/kW
$392/kW
Conclusions The authors estimated that by 2020, it would be possible to reduce a fuel cell stack
cost enough to be comparable to the cost of the internal combustion engine (which
is used today) if it was mass produced. In addition, the authors found an
improvement in power density would be essential to decreasing the overall stack
cost because it would decrease the resource use of other materials per unit power
output.
Themes Application of the learning curve
\i_
'ICF
152
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Appendix D. Responses to Peer Review Comments
This report has undergone peer review according to the guidelines set forth in EPA's Peer Review
Handbook (U.S. EPA, 2015). The peer review was independently managed by RTI International (RTI). RTI
selected the following three peer reviewers: Dr. Natarajan Balasubramanian of Syracuse University; Dr.
Marvin Lieberman of the University of California, Los Angeles; and Dr. Chad Syverson of the University of
Chicago.
Appendix D contains the summaries of the peer review comments and responses to each comment.
Most comments were addressed directly in the report. For these comments, we will indicate the section
where the updates can be found. If no changes were made to the report based on the peer review
feedback, we explain our reasoning for not doing so. EPA retained the original contractor, ICF, and the
SME, Dr. Argote, who developed this report to prepare responses to certain comments.
We categorized summaries of the comments into the following 11 groups organized by topic area: (1)
the report in general, (2) background and summary, (3) the recommended progress ratio, (4) the
literature review in general, (5) the literature review on sources of learning variation, (6) the literature
review on knowledge persistence and depreciation, (7) the literature review on knowledge transfer and
spillovers, (8) the literature review on the location of knowledge, (9) the literature review on the
specification and aggregation of learning, (10) the literature review on the application of the learning
curve, and (11) typographical errors and other minor corrections.
General Comments
Peer Reviewer
Peer Reviewer Comment
Response
1
Lieberman
The report is comprehensive, and does a good
job of characterizing the rates of learning
typically found in transportation equipment
manufacturing plants. Compared with
Argote's (2013) book or any individual
research study, this report offers a more in-
depth view of the literature on industrial
learning that is most relevant to the mobile
source sector. Overall, the report is a well-
executed document that is likely to be helpful
in providing a basis for incorporating forecasts
of learning into EPA and other government
rulemakings.
Despite these strengths, the report has a
number of limitations that should be
acknowledged more clearly. There are several
areas where improvements can be made in
the document.
See the "Summary and
Background" section.
2
Syverson
On balance, the study is a very fine review of
the literature on learning by doing in general,
See the "Summary and
Background" section.
\i^
'ICF
153
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Peer Reviewer Peer Reviewer Comment Response
especially with regard to its manifestation in
manufacturing operations during the past few
decades. The report is notably comprehensive
within this scope, makes sensible topical
categorizations in its discussion of the
literature's findings, and is clearly written. The
report achieves the intended goal of being a
definitive, reliable, single source of
information demonstrating the occurrence of
learning in general and in the mobile source
industry specifically.
3
Balasubramanian
The overall presentation and organization of
the report is generally clear. However, there
are some specific areas that require greater
clarity.
See the "Summary and
Background" section.
Background and Summary
Peer Reviewer
Peer Reviewer Comment
Response
4
Balasubramanian
The report appears to have multiple objectives
that are stated in several places. In addition,
there is at least one aspect that is provided in the
report but not mentioned as an objective (i.e.,
the methods of forecasting in Appendix A). 1
recommend a short subsection that explicitly
states the objective(s) in one location. In
addition, 1 recommend that the document refer
to these objectives consistently throughout the
document. For instance, Objectives 3 and 4
above are similar but it is not clear what the
difference between a "reliable" and a "best"
estimate is. It may be more appropriate to
choose one of them, and use that consistently.
Also, note that the term "best estimate" has a
generally accepted econometric definition as the
estimate with the lowest variance among a set of
estimates. Hence, it may be prudent to avoid
using that term or to clarify its meaning as used
in this report.
See updated discussion about the
report's objectives in Section 1,
"Introduction." We refer back to
these objectives throughout the
report.
In addition, we replaced the term
"best" or "reliable" estimate with
"summary effect" following the
example of Borenstein et al. (2009).
5
Balasubramanian
The two paragraphs in Section 3.2 beginning
"Learning is a major source of...." do not directly
relate to the discussion in Section 3.2, "What are
Progress Ratios?" and appear out of place. 1
recommend that they be moved to the Section
3.3, "Summary of Literature Review."
This discussion has been moved to
Section 3, "Summary of Literature
Review."
\i/
'ICF
154
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
#
Peer Reviewer
Peer Reviewer Comment
Response
6
Balasubramanian
The report does not seem to provide a clear
summary of the literature review. The summary
in the "Summary and Background" section of the
report focuses almost entirely on the estimation
of the average progress ratio, which is only a
small part of the review.
The summary in Section 3.3 is only a table with
no additional explanation. 1 recommend that a
more descriptive summary of the literature
review be included. Among others, 1 suggest that
the summary highlight the variation observed in
the rates of learning-by-doing (currently
discussed in Section 4).
See the "Summary and Background"
section and Section 3.3, "Summary
of Literature Review."
7
Balasubramanian
Section 1 of the report (paragraph 4) states, "It
will also summarize empirical estimates of the
learning effect separately for each of the specific
mobile source industries (e.g., original
equipment auto makers, parts suppliers to those
auto makers, loose engine manufacturers, large
truck manufacturers, and nonroad equipment
manufacturers) for which studies are found that
address those specific sectors." This break-down
by industry is not provided in the report. The
report provides only one estimate for the entire
sector. Hence, this statement should be
corrected or placed in a different context (e.g.,
the original intent of the study was to summarize
empirical estimates separately...).
See Section 1, "Introduction."
8
Balasubramanian
Section 2 of the report provides two reasons for
not providing a break-down of progress ratios by
industry. The first is the lack of studies in many
of the individual industries and the second is the
greater within-industry variation in rates of
learning-by-doing as compared to inter-industry
variation in those rates. While the first has merit,
the second is not a valid reason for not providing
a break-down by industry. It raises the question
of why studies from outside the mobile source
sector should not be used for estimating the
"best" or "reliable" progress ratio for the mobile
source sector. In my opinion, since there is
significant variation across industries (albeit less
than the within-industry variation) in the average
progress ratios (e.g., see provided progress ratios
or Dutton & Thomas, 1984), it is appropriate to
consider using industry-specific estimates, if and
when such estimates become available. In
general, it will be more informative to use the
means of two sub-groups than the mean of the
group as a whole.
See Section 2, "Selection of Subject
Matter Expert and Identification of
Relevant Learning-Related Studies."
sl^
'ICF
155
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Recommended Progress Ratio
Peer Reviewer
Peer Reviewer Comment
Response
9
Lieberman
1 agree that the weighted average progress ratio
across the five selected studies is 84.3%.
Moreover, based on my experience and my
reading of the broader literature on learning
curves, this is not an unreasonable figure for
manufacturing cost projections and forecasting
in the mobile source sector (at least for plants of
the type surveyed by the five studies).
However, the claim that there is a 95%
confidence interval of 83.9% to 84.8% is
misleading. That statement of the confidence
interval overstates the precision of the estimate.
[The method used would be appropriate if there
were some underlying, universal rate of learning
in the mobile source sector. That is unlikely; the
data show there is variation in the rate of
learning.] Rather than taking the (weighted)
average value of 84.3% across the five studies, if
one chose to be more conservative, a reasonable
choice would be to use the smallest rate of
learning in the sample, that is, the progress ratio
of 87%. In any case, the estimates from these
five studies all lie in a fairly close range.
Depending on the purpose at hand, one could
justify using 84.3%, or 87%.
See Section 3.4, "Discussion of
Mobile Source Results and
Recommendations."
\i/
'ICF
156
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Peer Reviewer Peer Reviewer Comment Response
10
Lieberman
All five of the plants that are studied in this
sample are engaged in final assembly of
transportation equipment (trucks, automobiles
and airplanes). Thus, the progress ratio
estimates are indicative of plants of this type,
that is, assembly plants for relatively complex
mechanical products made on a production line.
The estimates may not be suitable for plants
producing other types of products or plants
using other types of processes.
For example, the article by Nykvist and Nilsson
(2015) that surveyed dozens of studies on
learning in the production of Li-ion battery
packs, found a learning rate of only 9% for the
overall industry and 6% for the leading
manufacturers. This is a much lower learning
rate than the 84.3% progress ratio observed on
average across the five selected studies.
See discussion in Section 3. 4,
"Discussion of Mobile Source Results
and Recommendations."
For a response to the comment
about Nykvist and Nilsson (2015),
please see Comment #36 under
"Literature Review-Application of
the Learning Curve (Previously
Section 4.6)" below.
11
Lieberman
A further deficiency in the report is the failure to
point out that the progress ratio estimates in the
five selected studies are not based upon the
total cost of production. All five studies in the
final sample focus on assembly plants for
transportation equipment. None of the studies
utilizes data on the total costs per unit of output
in these plants. Rather, four of the studies focus
on labor costs and labor productivity in the
assembly plant (vehicles produced per labor
hour, or labor hours per aircraft), and one study
focuses on defect rates.
An 84.3% progress ratio based on labor cost
reflects a 15.7% savings in labor cost per unit for
each doubling of cumulative output. It does not
imply a 15.7% savings in total cost per unit for
each doubling.
Thus, any forecast of reduction in total unit cost
depends on (1) the progress ratio multiplied by
the growth in cumulative output (number of
"doublings") in the assembly plant, as well as (2)
the progress ratio and change in cumulative
volume applicable to the production of the
component parts. The report should be clear
about this need to consider cost reduction of the
component parts as well as the learning curve in
the final assembly plant. If a new vehicle model
is produced with new component parts, the
rates of cost reduction for parts production and
final assembly are likely to largely coincide (so
See discussion in Section 5,
"Responses to Peer Reviewer
Comments Related to the Analysis."
sl^
'ICF
157
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Peer Reviewer Peer Reviewer Comment Response
that a single progress ratio can be used), but this
need not be the case.
Similarly, the report is unclear and misleading in
describing the nature of the cost analysis in the
five representative studies. The "unit costs"
analyzed in the five studies are essentially labor
costs, or unit costs of final assembly, per se. The
studies do not tell us the extent to which the
total cost per unit, including the cost of the
component parts, followed a similar progress
ratio.
12
Syverson
1 have a couple of comments about the standard
error of the "meta-estimate" calculated in the
report. First, it would be helpful if the report
offered a brief explanation of how this standard
error is calculated from the literature's values
cited in Table 2. While the point estimate of-
0.245 is described as an inverse-variance-
weighted average of the five point estimates, the
standard error is left unexplained. If the
calculation is complex, it need not be spelled out
line-for-line; a short description of the
calculation's intuition would be enough.
Second, and more substantively, is the possibility
that the standard errors across the five studies in
Table 2 vary for reasons besides just sample size
differences. There are, after all, some basic
differences across the studies (e.g., industry and
outcome measure). In some ways—and the
report notes this—the fact that despite these
differences their estimates are all markedly
similar might suggest inferring that any
heterogeneity across the studies is more or less
orthogonal to the learning rate. On the other
hand, it is not practically possible to statistically
reject heterogeneous parameters with respect
to covariates such as industry and outcome
measures, with only five observations. As with
the gross-versus-net distinction discussed above,
1 do not know if there is any straightforward way
to quantitatively address this issue, but it strikes
me as something worth discussing a bit more in
the report.
See Section 3.4, "Discussion of
Mobile Source Results and
Recommendations."
13
Balasubramanian
The overall conclusion that learning-by-doing
occurs in the mobile source sector is well-
founded and largely indisputable.
We added this comment in a
footnote in Section 1,
"Introduction."
Sl^
'ICF
158
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
1 iPi
Peer Reviewer
Peer Reviewer Comment
Response
14
Balasubramanian
The methodology for estimating the weighted-
average progress ratio from the five studies is
broadly reasonable. In particular, the following
executive decisions related to estimating the
average progress ratio appear reasonable given
the objectives of the report:
a. Focusing only on studies that examine
unit costs and excluding studies that use
other measures of performance
b. Excluding studies of learning-by-doing in
shipbuilding during the Second World
War due to the uniqueness of the
context
We added this comment to a
footnote in Section 3.4, "Discussion
of Mobile Source Results and
Recommendations."
15
Balasubramanian
The report uses a "fixed-effects" model to
combine estimates from different studies (the
weight is the inverse of the variance). However,
it is not clear that all studies used the same
method to computing standard errors. For
instance, some studies may have computed
heteroscedasticity-robust or clustered standard
errors, which would typically be larger than
studies that assume homoscedasticity. If that is
indeed the case, taking a simple inverse would
not be accurate, and presenting one or more
alternative estimates in addition to this "fixed
effects" estimate (e.g., a simple average) may
provide a more complete picture. An additional
rule can be applied if one of these estimates has
to be chosen (e.g., the most conservative).
The methodology for estimating the standard
error of the average progress ratio is not explicit
in the report. A sentence or two describing this
should be added in Section 3.4.
Though the estimate of the weighted-average
progress ratio is broadly reasonable, the
discussion about the uncertainty associated with
learning-by-doing is quite sparse. Such a
discussion is important for a full understanding
of the weighted-average progress ratio. The
standard error of the weighted-average progress
ratio is likelv to be small, as currently stated in
the Report. However, that small standard error
does not reflect the true variation in the
progress ratios across organizations and
contexts, which is likely to be significantly larger.
Also, some important aspects of the studies
need highlighting to provide readers a better
understanding of their context (which could be
possibly different from today's context or other
See discussions in Section 3.4,
"Discussion of Mobile Source Results
and Recommendations."
We reviewed Benkard (2000) and
Levitt et al. (2013) to see how they
calculated standard errors. Both
authors controlled for
heteroscedasticity, but only Benkard
controlled for autocorrelation and
serial correlation.
sl^
'ICF
159
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Peer Reviewer Peer Reviewer Comment Response
contexts in the mobile source sector). Hence,
providing a prominent contextual discussion in
the "Summary and Background" section of the
report and in Section 3.4 covering the following
aspects is recommended:
a. There is significant variation and
uncertainty in the rates of learning-by-
doing depending on many factors, and
that learning-by-doing is not automatic as
discussed in Section 4.
b. The specific empirical context of the five
studies, viz. the production of a new car
model, as well as the dates of these
studies (where available).
These aspects are currently discussed in
different places in the report but it is important
that a summarized version of these points be
located close to discussions of the weighted-
average progress ratio.
16
Balasubramanian
The report aims to get a "best" or "reliable"
estimate of the "effect" of learning-by-doing (or
cumulative output) on costs. The term "effect"
has a causal connotation. However, it is not clear
that all five studies used econometric techniques
to causally estimate the effect of learning-by-
doing. If so, it may be more appropriate to
characterize the estimated weighted-average
progress ratio as the association between unit
costs and cumulative output, rather than as the
effect of learning on costs. This approach is also
consistent with the decision to focus on models
that include only cumulative output as a
predictor instead of using a more complete
model that includes other factors. This decision
implies that the effect of other factors is not
isolated from the effect of cumulative output,
when estimating the weighted-average progress
ratio.
We maintained our use of the term
"learning effect." We added a
discussion to the introduction
explaining how we define "learning
effect" and how it is used in the
report. We also added a discussion
about how difficult it is to prove
causation and how it can be done
with controlled laboratory
experiments.
See Section 1, "Introduction."
We did, however, replace any terms
that infer causation from our
summaries of the 18 articles in the
report.
\i^
'ICF
160
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
17
Peer Reviewer
Balasubramanian
Peer Reviewer Comment
As discussed in the report, cumulative output
can be correlated with many other factors (e.g.,
economies of scale). Also, the estimated
weighted-average progress ratio in the report
uses models that include only cumulative output
as a predictor. Hence, forecasting the impact of
learning-by-doing alone based on that ratio is
not possible in the absence of information on
the other factors. However, this does not render
the forecasting exercise provided in Appendix A
meaningless. It still measures the likely change in
unit costs due to a change in cumulative output,
which could be due to learning-by-doing or due
to other factors. Recognizing this assumption
implicit in these methods is important, especially
when applying these methods.
Response
We elaborate our discussion in
Section 3.4, "Discussion of Mobile
Source Results and
Recommendations."
Literature Review - General
18
Peer Reviewer
Lieberman
Peer Reviewer Comment
Given that the final recommendations in the
report are based almost exclusively upon the
five selected studies, it is useful for a reader to
be able to review a detailed summary of these
studies. Four of the studies are summarized in
Appendix B. However, the (truck plant) study by
Argote, Epple, Rao, and Murphy (1997) does
not seem to be included in Appendix B. I
recommend that a summary of this study be
added to the appendix.
Moreover, it might be helpful to add some
additional information to Table 2, which very
briefly summarizes the five selected studies.
This information might include the dependent
variable. While this can be determined from
Table 1, it is awkward for a reader to have to
search and scan between these sections. Table
2 might also indicate the pages in the appendix
where the summary of each study can be
found.
Response
We did not review the working paper
from Argote, Epple, Rao, & Murphy
(1997) because it was not able to be
published due the proprietary nature
of the data. The progress ratio used
in Table 2 was taken from Argote's
(2013) description of the Argote et
al. (1997) study. We clarified the
citations to show this.
In addition, we added a column to
Table 2 that indicates the outcome
variable used by the author(s). We
referred readers to the detailed
summaries in Appendix B under the
list of authors and publication date in
Column 1.
\i/
'ICF
161
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
* Peer Reviewer Peer Reviewer Comment Response
19
Lieberman
In general, 1 find the literature review to be
comprehensive and informative.
One specification issue that is left hanging in
the report is whether the learning curve should
be estimated with an initially "steep" portion
followed by a "flat" portion (once the data have
been transformed into logarithms). This
specification issue is raised on the last page of
the "Summary and Background" section;
however, there is no specific follow-up in the
report. (Virtually all of the presentation in the
report is consistent with a single learning curve
that does not change slope over time.) This
issue of whether the slope of the learning curve
is constant or diminishing should be discussed,
and ideally, resolved in the report.
See discussion in Section 5,
"Responses to Peer Reviewer
Comments Related to the Analysis."
20
Syverson
The only recent paper on learning by doing in
manufacturing that 1 did not see discussed in
this study is Hendel and Spiegel (American
Economic Journal: Applied Economics, Jan.
2014). That said, the paper's setting is not in
mobile source manufacturing, and it is a
judgment call whether the paper warrants any
more attention than a cursory review for the
purposes of this study.
We gave this article about learning in
general a cursory review. This article
did not receive a general review for
two main reasons. For one, the study
was not related to the mobile source
sector. Secondly, the authors include
a time trend variable in every model
that included a cumulative output
variable. Including both variables in
the same models could have
introduced multicollinearity into the
results. This could potentially explain
why both variables were
insignificant.
21
Syverson
There are several points in the report where
contrasts are made between measures of the
outcome variable in learning by doing
estimation. The report rightly points out (e.g.,
page 13) that using price or any metric that
embodies price is likely to confound supply-side
learning effects with demand-side changes that
could be unrelated to the learning process.
For example, this concern applies to value
added. However, it applies equally to shipments
as an outcome variable. The report holds out
shipments as problematic because they include
any inventory accumulation or de-
accumulation, and that is true, but shipments
are also reported in real dollar values, raising
the supply-versus-demand conundrum. This
fact was not always made clear in the text. For
example, when shipments are mentioned on
page 13, only the inventory issue is raised, and
See Section 3.3, "Summary of
Literature Review."
sl^
'ICF
162
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Peer Reviewer
Peer Reviewer Comment
Respon
moreover the output measure of Bahk and Gort
(1993) is described as "the number of
shipments." Perhaps I am just interpreting the
wording differently than the sense in which it
was meant, but this sounds like a quantity of
units of a good rather than a dollar value.
22
Syverson
I completely agree with the study's
interpretation of the literature that
heterogeneity in learning rates could well be
large across organizations, even within an
industry, than across industries. This is a very
useful point to make.
We added this comment to a
footnote in Section 4, "Review of
Learning Curve Literature by Topic."
23
Balasubramanian
The overall approach to the review—identifying
studies of learning-by-doing in the mobile
source sector, reviewing them for relevance to
the goals of the study and identifying a shorter
list of relevant studies for more detailed
review—appears reasonable. The list of topics
included in the review and the coverage of
those topics appear broadly reasonable.
We added this comment to a
footnote in Section 2, "Selection of
Subject Matter Expert and
Identification of Relevant Learning-
Related Studies."
24
Balasubramanian
The set of articles related to progress ratio
estimation in the mobile source sector and
included for review appears to be reasonably
comprehensive. A search for articles on
learning-by-doing in the mobile source sector
on Google Scholar did not yield any new
substantively-contributory articles on this
subject. A possible, but not necessary, addition
is Balasubramanian and Lieberman (2011). The
article itself is not relevant, but the Online
Appendix to this article contains estimates of
new-plant learning-by-doing using different
methods for several industries, at a more fine-
grained level (at the SIC-4 level) than
Balasubramanian and Lieberman (2010).
We would like to thank the
commenter for the additional data
from his 2010 article. Because the
dependent variable used in the study
(i.e., real value added) was not
appropriate for the goals of this
study (See discussion in Section 3.3)
and the range of progress ratios
provided was large, the additional
data will not be used to inform our
estimate.
After conducting a cursory review of
the 2011 article, we agreed that it
was not relevant to the goals of this
study.
25
Balasubramanian
Based on a broader search of articles on
learning-by-doing, an article (Haunschild and
Rhee, 2004) may potentially add some insights
in Section 4.1, but not including it will not
detract substantively from the findings of the
Report.
We gave this article a cursory review
and added a summary to Appendix C,
"Summaries of Articles Related to
the Mobile Source Sector that
Received a Cursory Review."
sl^
'ICF
163
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Literature Review - Sources of Learning Variation (Section 4.1)
#
Peer Reviewer
Peer Reviewer Comment
Response
26
Lieberman
The report provides no guidance on how to
perform a cost analysis forecast that incorporates
learning and economies of scale as separate
elements. Perhaps the text should be more
explicit about this, although the last paragraph of
Section 3.3 ("Column 6 - type of outcome
variable") makes it clear that the report is focused
on using only cumulative output as a predictor.
When controls for economies of scale are omitted
from the analysis, the estimated progress ratio
includes the effects of both learning and scale
economies. This has been shown in a number of
studies (e.g., my 1984 article on chemical
products). Adding a separate parameter for
economies of scale normally improves the
statistical fit, but the improvement is seldom
dramatic, and most studies have found scale
economies to be less important than the learning
effect. Moreover, if the data sample is small,
colinearity between the learning and scale
parameters can reduce the accuracy with which
each is estimated. One implication is that if the
analvst or policv maker is able to applv onlv a
single cost driver for forecasting purposes,
application of a learning curve or progress ratio to
forecasted cumulative output mav provide the
best proiection of future costs.
See discussion in Section 5,
"Responses to Peer Reviewer
Comments Related to the Analysis."
27
Lieberman
1 am puzzled that the findings from the
Balasubramanian and Lieberman (2010) article
are heavily discounted because the learning rate
"was estimated using revenues less materials
costs (i.e., value added) as the outcome variable,
rather than unit cost." None of the five studies
selected as representative of the mobile source
sector actually utilize data on unit cost. Four of
the studies use data that correspond to value
added in final assembly, omitting materials costs.
Thus, the dependent variable in the article from
Balasubramanian and Lieberman is not so
different from those of the selected studies.
(However, Balasubramanian and Lieberman
estimate a learning rate over the life of the
manufacturing plant, rather than over the life of a
new product within the plant.)
1 provided data to show that the average learning
rates by 4-digit SIC code for the mobile source
sector are substantially in line with those in the
summary section of the report.
Lieberman's 2010 study with
Balasubramanian used real value
added (which is based on real
revenue) rather than costs as their
dependent variable. Thus, their
dependent measure confounds
demand-side issues with supply-side
issues. See discussion in Section
4.1.4 "Balasubramanian &
Lieberman, 2010."
\i^
'ICF
164
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Peer Reviewer Peer Reviewer Comment Response
28
Balasubramanian
Consider Haunschild and Rhee (2004).
We gave this article a cursory review
and added a summary to Appendix
C, "Summaries of Articles Related to
the Mobile Source Sector that
Received a Cursory Review."
Literature Review - Knowledge Persistence and Depreciation (Section
4.2)
#
Peer Reviewer
Comment
Response
29
Lieberman
This section does a good job of characterizing
studies of the learning effect that have
considered knowledge depreciation.
One confusing element in this section is that
some of the depreciation rates are monthly and
others are annual. On pages 27 and 28, for
example, the text might clarify that Benkard and
Argote's estimates are monthly rates of
depreciation (although the figures are converted
to an annual basis in Table 3).
We added this comment to a
footnote in Section 4.2, "Knowledge
Persistence and Depreciation."
We presented the annual
depreciation rates in the text. We
added a footnote to each converted
depreciation parameter, which
would (1) explain that we are
showing the converted value(s), (2)
show the original value(s) used in
the article, and (3) refer the reader
to Table 3, where we show the
conversion.
30
Syverson
To the extent that one objective of the study is
to identify the expected pace at which mobile
source manufacturing productivity should
improve with production experience, though, it
seems to me that what matters in the end is the
net effect of learning and depreciation rather
than the gross learning rate. 1 recognize the
gross-versus-net distinction might not be easy to
quantitatively reconcile. Therefore it might not
be possible to derive a bottom-line net learning
rate parameter that is as comparable and
applicable as the gross parameter the study
reports now. However, it does seem prudent to
at least discuss the net-versus-gross distinction
and how it might matter when applying the
findings of the report to practical settings. 1
realize that the study argues that mobile source
manufacturing has several properties
(production typically is conducted at an even
rate, learning is often embedded in technology
and routines, and the sector experiences
relatively modest worker turnover) that make it
likely that depreciation would tend to be on the
low end of estimates in the literature. This does
not seem unreasonable. However, arguing that
See discussion in Section 5,
"Responses to Peer Reviewer
Comments Related to the Analysis."
\i/
'ICF
165
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
#
Peer Reviewer
Comment
Response
these effects are likely to be smaller than usual
does not necessarily imply depreciation is likely
to be zero. Again, there might not be any easy
practical alternative here in terms of
quantitative reports, but it is worth discussing
the issue.
Literature Review - Knowledge Transfer and Spillovers (Section 4.3)
#
Peer Reviewer
Peer Reviewer Comment
Response 1
31
Lieberman
This section is effective in describing research
findings relating to knowledge transfer across
organizational units (e.g., additional shifts, new
models) within a given firm. However, the
section ignores the existing literature on
knowledge transfer and spillovers across firms
(except for very brief mention in Footnote 5).
This literature on inter-firm spillover of learning
is fairly extensive, although the evidence is
based mostly on studies using data outside the
mobile source sector.
In the introduction of Section 4.3, we
clarified that distinguishing
components of learning was not an
objective of our report; therefore,
the studies do not cover all
components of knowledge transfer.
In addition, we added this comment
to a footnote in Section 4.3.
Literature Review - Location of Organizational Knowledge (Section
4.4)
#
Peer Reviewer
Peer Reviewer Comment
Response 1
32
Lieberman
This section is informative and well done. 1 think
it would be helpful to provide some of this
material earlier in the report—specifically, to
make it clear that learning and knowledge can be
embedded in people, in organizational routines,
or in technology/physical capital.
We discuss that learning and
knowledge can be embedded in
people, routines, and technology in
the third paragraph of the
introduction to Section 3, "Summary
of Results and Recommendations."
We also added a discussion in our
summary of the 18 articles in Section
3.3.
\i/
'ICF
166
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Literature Review - The Specification and Aggregation of Learning
(Previously Section 4.5)
33
Peer Reviewer
Lieberman
Peer Reviewer Comm
"
Section 4.5 does not truly serve a standalone
function; rather, it seems to be a placeholder to
summarize three studies that were otherwise
hard to classify. Perhaps the section should take
a broader perspective, summing up many of the
conclusions of the previous sections that relate
to the specification and aggregation of learning.
Respon
We agreed that Section 4.5 did not
serve an important function.
Therefore, we reviewed the three
articles, Bahk and Gort (1993),
Laitner and Sanstad (2004) and Levin
(2000), to decide whether they were
correctly categorized. We decided to
move Bahk and Gort to Section 4.4
and to re-categorize the Laitner and
Sanstad and Levin articles to those
that receive a cursory review.
The Bahk and Gort (1993) article
focuses on disaggregating learning
into organizational learning, capital
learning, and labor learning. We
moved the discussion of this article
to Section 4.4, "Location of
Organizational Learning."
We moved the Levin (2000) article to
Appendix C and removed the
discussion from the body of the
report. We retained the discussion
about whether time is an important
source of improvement in the quality
of cars (as opposed to cumulative
output).
We moved Laitner and Sanstad
(2004) to the category of articles
that received a cursory review.
Because the article dealt with
learning and general, it is not
featured in Appendix C. We re-
categorized this article mainly
because it is unrelated to the mobile
source sector, it is based on
projections rather than actual data,
and it is not focused on learning
from the producer's point of view.
We added a footnote to Section 4.1,
"Sources of Learning Variation" to
point out that learning from the
consumer's point of view is another
interesting type of learning.
We moved Levin (2000) to Appendix
\i/_
'ICF 167 September 30,2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
#
Peer Reviewer
Peer Reviewer Comment
Response
C and removed the discussion from
the body of the report. We retained
the discussion about whether time is
an important source of improvement
in the quality of cars (as opposed to
cumulative output).
34
Syverson
Considering adding Hendel and Spiegel
(American Economic Journal: Applied Economics,
Jan. 2014)
We gave this article a cursory
review; however, it will not be
included in Appendix C because is
not related to the mobile source
sector.
In the response to peer reviewers,
we will explain why we choose not
to give the article a detailed review
(e.g., it may not be representative
because it is a single for that
produces a single product, they may
be on the flat portion of the learning
curve, small increases in learning
may be offset by small increases in
forgetting, learning may be
embedded in technology, learning
may be insignificant because a time
trend was included).
35
Syverson
1 struggled to understand how the work of
Laitner and Sanstad (2004) fit into the
discussion. 1 realize that there might be learning
about products among consumers, but it wasn't
exactly clear to me from the description of their
paper how this would influence supply-side
learning. My best guess of the story is that
demand-side learning affects the equilibrium
quantity of a product, and that can change how
quickly experience is accumulated on the supply
side. If that is correct, though, then it is less clear
to me that one would necessarily want to purge
demand-side influences from learning
estimation, as asserted in the price-as-an-
outcome issue discussed above. Is there a
fundamental difference between that point and
the Laitner and Sanstad (2004) analysis?
We re-categorized the Laitner and
Sanstad article to group of articles
receiving a cursory review. Because
the article deals with learning in
general, we do not feature the
article in Appendix C.
The article does not explain how
demand-side learning reduces costs.
However, we would argue that our
decision to discount articles that us
price as an outcome variable is still
valid. If Syverson's guess is accurate,
price may be viable as an outcome
variable if one could control for
things such as firm strategy and
market conditions. However, data
related to a firm's strategy would
likely not be available and therefore,
it would be difficult to parse out the
relationship between price and
learning on the demand or supply
side.
sl/i
'ICF
168
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Literature Review-Application of the Learning Curve (Previously
Section 4.6)
*1
36
Lieberman
The studies summarized in the section are quite
diverse. Nevertheless, it seems appropriate to
have a concluding section to consider these
studies.
It is striking that Nykvist and Nilsson's (2015)
survey found learning rates for production of
automotive Li-ion battery packs to be
substantially smaller than the 84.3% progress
ratio that the EPA report proposes for cost
forecasting in the mobile source sector. It would
be informative to consider possible sources of
this large discrepancy in learning rates between
Li-ion battery manufacturing and transportation
equipment final assembly.
We agree that Nykvist and Nilsson
estimated a learning curve rate of
9% for the Li-ion battery industry
generally and 6% for the market-
leading manufacturers. However, we
disagree that this is evidence of a
large discrepancy in learning rates
between Li-ion battery
manufacturing and transportation
equipment final assembly. The
authors note that those learning
rates are estimated using data from
2007 to 2014. However, they also
note that while industry-wide
average costs declined by about 14%
annually from 2007 to 2014, costs
are expected to decline 8% annually
in the future ("Hence, we believe
that the 8% annual cost decline for
market-leading actors is more likely
to represent the probable future
cost improvement for Li-ion battery
packs in BEV."). When this projected
cost decrease is taken into account,
the results are not dissimilar to the
84.3% progress ratio estimated in
this report. Specifically, in early
years, the classic progress ratio-
based cost reductions reflected in
the 84.3% progress ratio result in
more rapid cost declines, but those
declines flatten out due to the
logarithmic nature of the
calculations. An 8% annual rate of
cost reduction results in less rapid
cost declines, but those declines
remain at 8% per year going forward
such that, after 11 years, costs are
actually lower using the 8% annual
rate of decline. As explained in
Section 3.1 of this report, with
respect to the form of the learning
curve, the preponderance of studies
support a logarithmic relationship
over a linear relationship. This
suggests that, while the two rates
are similar in this case, the
logarithmic relationship is more
\i^
'ICF
169
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
appropriate for use in cost
estimations. See attachment D1 to
this appendix for the discussion of
the Nykvist and Nilsson study that
was included in EPA's July 2016 Draft
Technical Assessment Report
Typographic Errors and Other Minor Corrections
Peer Reviewer
Peer Reviewer Comment
Response
37
Lieberman
In the title of Summary Table 1, "Progress
Rations" should be Progress Ratios."
Update to body of EPA Report (by
EPA)
38
Lieberman
Summary and Background, page 3. In the middle
paragraph, "for each doubling of production
volume" should be "for each doubling of
cumulative production volume."
In the sentence that follows, "it was assumed
that production volumes would have doubled"
should be "it was assumed that cumulative
production volumes would have doubled".
Update to body of EPA Report (by
EPA)
39
Lieberman/
Syverson
Page 19. "In error! Reference source not found"
is a typographical error. From the context it
appears to be a reference to Table 2.
We updated the link to 'Table 2".
40
Syverson
There is a missing closed parenthesis in the first
sentence of EPA summary.
Update to body of EPA Report (by
EPA)
41
Syverson
On page 49 in the appendix, the "review of the
literature" progress ratio is cited as 83%, but the
estimate given in the main body of the review is
84%.
We updated the calculations in
Appendix A to 84% to be consistent
with our recommendation.
42
Syverson
The Levitt, List, and Syverson study is cited as
being published in both 2012 and 2013 in
different locations.
We replaced the study's publication
dates with 2013.
43
Syverson
Also, on page 38, Levitt, List, and Syverson are
described as studying the repair rate as an
outcome variable rather than the defect rate.
This discussion was removed from
the report. However, with the
statement, "Levin's results contrast
with the findings of Levitt et al.
(2013) who also examined quality
learning curves and found that
cumulative output was a better
predictor of the outcome variable,
the repair rate, than time," we
intended to convey that Levin used
repair rate as the outcome
variable—not Levitt et al.
\i^
'ICF
170
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Attachment Dl. Excerpt from EPA's Draft Technical Assessment
Report: Midterm Evaluation of Light-Duty Vehicle Greenhouse
Gas Emission Standards and Corporate Average Fuel Economy
Standards for Model Years 2022-2025.
The following is an excerpt from EPA's July 2016 Draft Technical Assessment Report, which is provided in
connection with a response to a peer reviewer's comments regarding the estimated learning rates in an
article by Nykvist and Nilsson (2015).
\i_
'ICF
171
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Technology Cost, Effectiveness, and Lead-Time Assessment
historical average). The researchers also suggested that the primary difficulty imposed by such
fluctuations would be felt by cell manufacturers in. maintaining profit margins, rather than by
vehicle manufacturers or consumers.
5.2.4.4.9 Evaluation of 2012 FRM Batten- Cost Projections
In the 2012 FILM, the agencies adopted a bottom-up, bill-of-materials approach to projecting
the future DMCof xEV batteries by using the AN I, BatPaC battery cost model.1 J7 As discussed
in the Technical Support Document (TSD),}6 accompanying the 2012 FRM. battery pack costs
projected by this model were shown to compare favorably with cost projections provided by
suppliers and OEMs that were interviewed during development of the rule. In the 2015 NA5
report (Finding 4.4, p. 443), the committee found that "the battery cost estimates used by the
agencies are broadly accurate," providing further support for the use of this model.
At the time of the FRM, few public sources were available to further validate these
projections. Since that time, several sources have emerged that provide additional information
on the evolution of battery costs since the FRM and potential future trends.
In 2015, a peer-reviewed journal article (Nykvi stand Nilsson, 2015) appeared that provides a
comprehensive review of over 80 public sources of battery cost projections for BE Vs.112 Based
on a statistical analysis of these estimates, it was shown that industry cost estimates for lithium-
ion batteries for BEVs have declined 14 percent annually between 2007 and 2014, and that pack
costs applicable to leading BEV manufacturers have followed a cost reduction curve of about 8
percent per year, with a learning rate of between 6 percent and 9 percent. The authors concluded
that the battery costs experienced by market leading OEMs are significantly lower than
previously predicted, and that battery costs may be expected to continue declining.
Figure 5.37 compares the full population of cost estimates reviewed by Nykvist and Nilsson
to the battery pack cost projections of the 2012 FRM analysis. Because BatPaC does not
produce cost estimates for multiple years, the 2012 FRM analysis applied a learning curve to
generate costs for the years 2017 through 2025, with BatPaC output costs assigned to the year
2025. The learning-adjusted FRM costs shown in the figure include those for PITEV40, EV75.
EV100 and EVI50, which ha\e relatively large capacities similar to those likely included in the
review. The plot shows that the battery costs projected in the 2012 FRM fit well with the
reviewed estimates, and lie on a similar cost reduction curve.
\i/
'ICF
5-122
172
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Technology Cost, Effectiveness, and Lead-Time Assessment
1600
1400
1200
TJ
LOGO
"O
300
«- 600
*
0.
1 400
200
~
~
~
o Bo
°bT d
Dn
~ ~
Pi I" 1
HC
~
j n C a ¦ *
BdBS i..g
~ II u "-1 I * *
DnS~ 'II
n 1
• • * *
¦ • i ¦
III!
L 1
i
~
2ECO
a Nylcviji^NI&son
• 2D12 FRM
2005
2013
2015 202D
Year
2025
2030 2035
l-'iljure 5,37 Comparison of 2*112 l"RM Projected Battery Cost Per kWh to Estimates Reviewed by Nykvist &
Nilsson
Cost estimates and projections are most useful when they can be validated by comparison to
actual costs. Unfortunately, information about actual battery costs paid by manufacturers for
production vehicles is rarely disclosed publicly. 1 lowever, in October 2015, General Motors
publicly commented on its battery costs for the Chevy Bolt EV, providing an opportunity to
evaluate the FRM projections of REV battery costs.
At the General Motors Global Business Conference on Oct. I, General Motors described to an
investor audience its current and projected cost per kWh (on a cell basis) for battery cells for the
Chevy Bolt EV. Citing partnership with cell manufacturer LG Chern, Executive Vice President
of Global Product Development Mark Reuss stated, "When we launch the Bolt, we will have a
cost per kWh of $145, and eventually we will get our cost down to about S100. We believe we
will have the lowest cell cost with much less capital and volume dependency."' ' An
accompanying chart shows the SI45 cost continuing to 2019. dropping to SI 20 per kWh in 2020
and to $100 per kWh in 2022.
It is important to note that the costs described above are cell-level costs and not pack-level
costs. To compare them to the pack-level costs projected by the agencies requires converting
them to that basis using an appropriate methodology. Also, although the context of the
announcement suggests that the costs are comparable to a direct manufacturing cost, their exact
basis is unknown. Although these factors introduce some uncertainty in comparing the
announced costs to the FRM projections, a qualified comparison is possible.
Several sources exist that suggest a cost conversion factor from cell-level costs to pack-level
costs for lithium-ion batteriesThese are summarized in Table 5.6. Most of
these sources suggest a conversion factor of about 1.25 to 1.4 may be appropriate.
Table 5.6 also shows two estimates derived from the ANL BatPaC model for a liquid-cooled
BEV-sized pack at a production volume of 50,000 to 100,000. Outputs from this model suggest
5-123
\i_
'ICF
173
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Technology Cost, Effectiveness, and Lead-Time Assessment
that the ratio of pack-level cost to cell-level cost for the pack format modeled by BatPaC may
range from about 1.5 for a 16 kWh pack to about 1.3 for a 32 kWh pack, and continuing to
decrease tor larger pack capacities.
Table 5,6 Examples orConversion Factors for Cell Costs to Pack Costs
Source
Low
High
Kalhammer et al,wo
1,24
1,4
Element Energym
1,6
1.85
Konekamp248
1.29"6
USABCU!
Tataria/l,opezvc
l.2Bw
Keller'45
1.2"
BatPaC, 16 kWh
1.5
BatPaC, 32 kWh
1,3
On the basis of the BatPaC-derived ratios of 1,3 to 1,5, the 2015-2019 cell-level figure of
$145 per kWh would translate to approximately S I 90 to $220 per kWh on a pack level. The
future projections of $120 and $100 per cell kWh in 2020 and 2022 would translate to
approximately SI 56-$ 180 perk Wh and $130-S150 per kWh at the pack level, respectively.
On this pack-converted basis the GM ceil costs agree well with the BatPaC cost projections
that the 2012 FRM analysis applied to 2025. Table 5.7 summarizes the estimated pack-level
equivalents of the cell costs disclosed by GM and compares them to the EV150 pack-level
HntPaC output costs of the FRM analysis. The pack-converted GM projection for 2020, at $156-
$180 per kWh, compares well to the FRM BatPaC output costs for EVI501'1' for 2025, which
ranged from $160 to $175 per kWh (at 450,000 units annual volume). The pack-con verted GM
projection for 2022 at $ 130-$ 150 per kWh is significantly lower than the agencies' projection for
2025. This suggests that the 2012 FRM cost projections, at least for EV150, may have been
quite conservative.
Table 5.7 Comparison ofGM/LGCtem Pack-Convert C til Costs to FRM EVI5Q Pack Cost
Pack CostfkWb (2015$)
Source of Estimate
fear Applicable
Low
High
EV150 in FRM
2025
$160
$175
GM/IG Global Business Conference
2015-2019
$190
$220
2020
$156
$180
2022
$130
$150
Figure 5.38 compares the pack-converted GM costs to the year-by-year learning-adjusted
costs used in the 2012 FRM for Small, Standard, and Large Car EVI50. It can be seen that the
88 Cell cost - 620 Euros* 16 modules = 9.920 Euros; pack cost. = 12,800Euros; 12,800/9.920 = ! .29.
2016" (slide 6).
n The Chevy Boll is anticipated to offer a 200-mile driving range, potentially comparable to the real-world 150-mile
range of the EV ISO thai ihe agencies modeled in the FRM.
5-124
si/
'ICF
174
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Technology Cost, Effectiveness., unci Lead-Time Assessment
range of the pack-con verted GM costs is lower than the costs predicted by the 2012 FRM
analysis.
400
350
vv
T5 300
1-250
-=¦ 200
150
K
%
f
100
50
1
1
I !
\
[
i
VI
I
L
I il
1
* FRM
-A—SM/LG low
-B-GMAGHgh
2015 2016 2017 201G 2019 2020 2021 2022 2023 2024 2025 2026
Year
Figure 5.38 Comparison of Estimated GM/LG Pack-Level Costs to 2012 FRM Estimates tor EVI5Q
At the time of the FRM, the agencies' battery cost estimates appeared to be lower than costs
being reported by many suppliers and OEMs at the time, and also lower than some independent
estimates said to be applicable to the time frame of the rule. The agencies chose to place
confidence in the peer-review a I \\l. BatPnC model due to its rigorous, bottom-up approach to
battery pack costing, and the expertise of leading battery research scientists that contributed to its
development. The comparisons described above suggest that this approach was effective and
may in fact have been conservative not only with respect to characterizing the pace of reductions
in battery cost that have taken place in the time since the FRM but also to projecting future costs
for the 2020-2025 time frame. Up to and including the development of this Draft TAR analysis,
the agencies have continued to invest significant resources into understanding developments and
emerging trends in battery technologies so that these critically important projections of xEV
battery cost may be as reliable as possible.
While other public examples of battery costs to manufacturers remain elusive, several
suppliers and manufacturers have made battery-re la ted product announcements since the FRM.
Some of these include information suggestive of battery costs or pricing. Some manufacturers
have published pricing for battery replacement parts or upgrades available to authorized service
providers. Others have offered different options, such as battery size or purchase method, the
relative pricing of which may suggest a relationship to battery cost. Finally, stand-alone non-
automotive Li-ion battery packs are beginning to become available to end users and their pricing
may be informative. While the agencies recognize that the pricing of these early-stage product
offerings may be subsidized by their manufacturers for competitive and marketing reasons, these
announcements may still be relevant to understanding the evolution of battery pack costs as these
products increase their presence in the market.
5-125
\\/_
'ICF
175
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Technology Cost, Effectiveness, and Lead-Time Assessment
In 2013-2014, Tesla Motors offered the Model S in two battery pack sizes, 60 kWh and 85
kWh, at retail prices of around $69,900 and $79,900, respectively. Assuming no content
difference betw een the two versions, the retail price differential would suggest a battery cost of
$10,000 / 25 kWh S400/kWh, An alternate analysis presented by Nykvist et al.344 subtracts the
estimated value of added content found in the 85 kWh version (Supercharger, premium tires, and
associated markup), resulting in a net price difference of $8,500 or $340 per kWh.
In July 2014, Nissan announced the replacement cost of a 24-kWh batter)' for the Nissan Leaf
at $5499 with core return, which amounts to about $229'kWh net. Although Nissan requires
return of the original battery (core), a $1000 credit is then applied for the core, suggesting a full
retail price of $6499, or $271 /kWh.34%Mt'*H7 Later the same month, Nissan followed up by
pointing out that the quoted price is in fact subsidized by Nissan, although they declined to report
the amount of subsidy or the actual manufacturing cost.u-s Nissan does not allow purchase of the
battery except as a Leaf battery replacement.
In 2015, an independent vendor of OEM parts listed the 2011 Chevy Volt battery pack at
$10,208 list price, discounted to $7,228, with no mention of core exchange. Assuming a 16 kWh
capacity, these prices would value the battery at $63S/kWh and $452/kWh, respecti vely.
Although the product was listed and priced by the vendor, it was on restriction from ordering for
reasons that remain unci ear.34Mj5n
In January 2015, it was reported that the MSRP for a BMW i3 battery pack module was listed
at $1.805.89, each module being 2.7 kWh (21.6 kWh total divided by 8 modules). This module
price would equate to $669/kWh. A specific dealer was reported to be offering the module at a
price of $1715.60, or $635/kWh.JSI
In September 2015, Tesla announced the price for a range-increasing battery pack upgrade for
the Tesla Roadster at $29,000, including installation and logistics. Tesla indicated that the
quoted price is meant to be equal to Tesla's expected cost in providing the pack, and disclaimed
any intention to make a profit. Tesla also indicated that the price per k Wh is higher than for a
Model S batter)- due to the low volume production expected for the Roadster upgrade pack (only
approximately 2,500 Roadsters were produced). T esla did not list the kWh capacity of the
upgrade pack, but describes it as having approximately 40 percent more energy capacity than the
original Roadster pack, which is commonly listed as 56 kWh. This suggests that Tesla's cost for
low volume production of this pack is around $29,000/(56* 1.4} = $370 per kWh.352 In October
2015, Tesla further announced that the Roadster upgrade packs would be provided through a
partnership with LG Chem.J5J This suggests that the price of the pack may not reflect
anticipated savings from the Panasonic-Tesla "Gigafactory" partnership.
In August 2013, the Smart ED was offered with a 17.6 kWh battery, with the option to either
purchase the battery with the car, or lease it separately. The vehicle price was $5,010 lower
without the batter)'when the battery was leased at a price of $80/mo. Ifthe $5,010 differential
was taken to represent the incremental cost of the battery, it would value the batter)' at
$285/kWh. Of course, the present value of the lease payments would also contribute value to the
transaction, and it is possible that marketing considerations could also be represented in the
pricing*4,35"56
In September 2015, Nissan announced pricing in the UK for the 2016 Nissan Leaf.Ina press
release from Nissan, equivalent versions of the Leaf having a 30 kWh pack instead of a 24 kWh
5-126
\i/
'ICF
176
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Technology Cost, Effectiveness, and Lead-Time Assessment
pack were priced at a difference of 1,600 British pounds. This would amount to approximately
267 British pounds per k Wh, or U.S. $411 per kWh (assuming an exchange rate of 1.54 U.S.
dollars per pound). It should be noted, however, that although the two versions of the pack
appear to be designed to install into the same footprint and volume, any cost comparison is
potentially complicated by differences in chemistry and construction of the two versions.5'7
In 2014, Tesla Motors began construction of a so-called "Gigafhctory" in Nevada in
partnership with Panasonic. This factors' is commonly cited by Tesla as enabling a potential 30
percent reduction in battery pack costs from the levels Tesla currently pays. According to one
analysis,Js!1 Tesla's current cost is estimated at about $274 per kWh. A 30 percent reduction on
that figure would bring costs to about $192 per kWh.
In April 2015, Tesla announced a home battery pack product called Powerwall, pricing a 7
kWh version at $3,000 ($428/kWh) and a 10 kWh version at $3,500 ($350/kWh). Although
designed for stationary home use, the pack design bears similarities to automotive packs, being
liquid-cooled and using similar chemistries. The 7 kWh version employs NML' chemistry
similar to many production BEVs, while the 10 kWh version employs the NIC'A chemistry like
the Tesla Model S. Tesla also announced a similar product called Powerpaek for commercial
use. Powerpaek was said to be priced at $25,000 for 100 kWh capacity, or $250/kWh. These
products are expected to take advantage of much of the cell output of the Gigafactory, suggesting
that these products may be priced in anticipation of the cost reductions it is expected to achieve.
Table 5.8 summarizes the estimated cost or pricing information derived from the foregoing
examples.
Table 5.8 Summary of PuMkhcd Evidence of Buttery Pack Cost and Pricing
Pack Cost or Price
per kWh
Source of Evidence
Year Applicable
High
Low
Tesla Model 5 60 kWh vs 85 kWh comparison
2013-2014
$340
$400
Nissan 24 kWh replacement pricing
2015
$229
$271
Vendor pricing for 2011 Volt pack
2015
$432
$638
Dealer pricing for BMW i3 module
2015
$635
$668
Tesla Roadster upgrade pricing
2015
$370
Smart ED lease vs buy pricing
2013
$285
Nissan UK price differential 30 kWhvs24 kWh
2015
$411
Tesla Lik Research estimate
2014
$274
Tesla Lux Research estimate modified byGigafactory
2017
$192
Tesla Powerwall
2015-2016
$350 | $428
Tesla Powerpaek
2015-2016
$250
It is important to remember that the figures derived from these examples should be interpreted
with caution. The agencies' cost projections represent direct manufacturing costs and not retail
pricing. Also, as previously noted, retail pricing of these early-stage product offerings may be
subsidized by their manufacturers and may reflect competitive and marketing considerations that
further obscure their true manufacturing cost. Furthermore, some of the estimates arc derived
from full-product comparisons that may or may not accurately represent the battery portion of
the comparison. It should also be noted that the examples presented here represent current
pricing, while the FRM applies its BatPaC cost projections to the year 2025.
5-127
\i/
'ICF
177
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Technology Cost, Effectiveness, and Lead-Time Assessment
On the other hand, the existence of these examples shows that the industry has progressed
considerably since the FRM, when such examples were almost entirely unknown. The
identification and packaging of specific battery products tor upgrade, replacement or standalone
use is a significant development and suggests that the industry is continuing to gain in maturity
and is growing along multiple paths. The establishment of MSRPs for many of these products
also suggests that manufacturers are beginning to gain confidence in their understanding of the
cost structure of battery products. The examples and estimates derived from this analysis, even
if approximate, can serve to ground the various cost estimates and projections that have
previously been the primary source of battery costing information (and will continue to play an
important role going forward).
5.Z4.5 Fuel Cell Electric Vehicles
5.2.4.5.1 Intmductkm to FCEVs
Fuel Cell Electric Vehicles (FCT.Vs) are another potential technology option lor
implement ins? electrified drive to achieve zero tailpipe emissions, like the BHV technology
presented in Section 5.2.4.3.5. Like BMVs, FCT.Vs use electricity to turn electric motors onboard
the vehicle that provide the motive power for driving. However, unlike a BEV, the FCEV also
produces this power onboard. It achieves this by harnessing the energy produced in an
electrochemical reaction that combines hydrogen and oxygen to form water. This process occurs
within the lucl cell itself, a device that shares a basic structure w ith batteries; namely, it consists
primarily of an anode, a dividing electrolyte, and a cathode. Hydrogen from an onboard tank
enters the fuel cell's anode and is separated into its constituent electron and proton. The electron
is directed to an external circuit, where it ultimately provides power to the electric motors driving
the wheels. The proton is transferred across the fuel cell's electrolyte membrane to the cathode,
where it combines with oxygen from air entering the cathode and electrons returning from the
external circuit to form water. Thus, the basic reaction in the fuel cell is II: + K»0: —*11:0, with
usable electric power (and some amount of heat) produced in the process.
State and national policies have increasingly adopted the perspective that FCEV and BEV
technologies w ill be complementary vehicle technologies that will likely both be needed in order
to achieve long-tenri CilICi reduction goals. Well-to-wheel (311(3 emissions for FCEVs and BE Vs
vary depending on the method of production lor their various fuels (electricity lor BEVs and
hydrogen for FCEVs), but both technologies hold promise for significant reduction below
current and projected future ICE vehicle (311(3 emission rates (see Chapter l>, Infrastructure
Assessment for a more complete presentation of(311(3 emissions from hydrogen production).
Hydrogen energy storage, the conversion of electrical energy into hydrogen gas through the
process of electrolysis, has recently gained significant attention for its potential to enable
increased renewable penetration in the electric grid, thus potentially playing a significant role in
decarbonizing multiple industries in the full US energy system. Although there is potential for
FCEVs to play a significant role in reducing C311(3 emissions, the technology is still relatively
new (the first mass-produced vehicles entered the market in 2014) and costs have historically
been higher than other options. For this reason, FCEVs were not included m the projections of
the future vehicle licet in the 2012 FRM.
The 2010 Technical Assessment Report (TAR) covered developments and state-of-the-art
technology lor the FCEV at the time. Since then, researchers anil developers in government,
academia, and industry have continued to advance the technology's performance capability and
5-128
\i/
'ICF
178
September 30, 2016
-------
Cost Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources
Appendix E. Peer Review Report
\i_
'ICF
179
September 30, 2016
-------
RTI Project Number
0213244.004.014
Cost Reduction through Learning in
Manufacturing Industries and in the
Manufacture of Mobile Sources
Peer Review
Peer Review Report
May 2016
Prepared for
U.S. Environmental Protection Agency
Office of Transportation and Air Quality
Office of Air and Radiation (US EPA OAR/OTAQ)
2000 Traverwood Dr.
Ann Arbor, Ml 48105
Prepared by
RTI International
3040 E. Cornwallis Road
Research Triangle Park, NC 27709
SJRTI
INTERNATIONAL
-------
CONTENTS
Section Page
1 Introduction 1-1
2 Peer Review Process 2-1
3 Summary of Findings 3-1
3.1 Overview of the Peer Reviewer Comments 3-1
3.2 Clarity of the Presentation 3-2
3.3 Overall Approach and Methodology 3-2
3.4 Appropriateness of the Studies Included and Other Inputs 3-3
3.5 Data Analyses Conducted 3-3
3.6 Appropriateness of the Conclusions 3-4
References R-l
Appendixes
A Conflict of Interest Analysis and Bias Questionnaire A-1
B Peer Reviewer Resumes B-l
C Peer Review Panel Charge C-l
D Peer Reviewer Questions and Answers on the Charge D-1
E Peer Reviewer Reports E-l
i
-------
SECTION 1
INTRODUCTION
The Office of Transportation and Air Quality (OTAQ) within the U.S. Environmental
Protection Agency (EPA) has requested a peer review of "Cost Reduction through Learning in
Manufacturing Industries and in the Manufacture of Mobile Sources" developed for EPA by ICF
International. The purpose of the study is to develop a single compendium study on industrial
learning in general and the mobile source sector specifically that the Agency can use as the basis
for accounting for learning effects in the cost analyses developed for regulatory and other
actions. The study provides an assessment of manufacturing learning through analysis of
published studies and literature and, using that information, estimates a progress ratio (learning
rate) for the mobile source sector.
RTI International (RTI), an independent contractor, was contracted by OTAQ to facilitate
a peer review of the study. The peer review was carried out based on the EPA Science Policy
Council Peer Review Handbook, 4th Edition (U.S. EPA, 2015; henceforth referred to as the Peer
Review Handbook). The peer review was conducted to ensure that the learning study can be
considered a definitive, reliable, single source of information demonstrating the occurrence of
learning in general and in the mobile source industry specifically. Three recognized experts in
learning effects were engaged to review the learning study and provide feedback on: (1) clarity
of the presentation, (2) overall approach and methodology, (3) appropriateness of the studies
included and other inputs, (4) data analyses conducted, and (5) appropriateness of the
conclusions.
This report includes a description of the peer review process, a summary of the peer
review reports, and the individual peer reviewer reports. In addition, all materials provided to the
peer reviewers to support the review, such as the panel charge and the technical work product, as
well as peer reviewer resumes and a conflict-of-interest (COI) disclosure form, are provided in
the appendices.
1-1
-------
SECTION 2
PEER REVIEW PROCESS
In December 2015, EPA's OTAQ requested that RTI facilitate a peer review of the report
Cost Reduction through Learning in Manufacturing Industries and In the Manufacture of Mobile
Sources. RTI managed the peer review independently and according to guidelines set forth in
EPA's Peer Review Handbook (U.S. EPA, 2015). RTI initiated the process of identifying and
selecting three peer reviewers in January 2016 and completed the peer review process in April
2016.
To identify qualified candidates for consideration, RTI identified 12 candidates based on
recommendations from EPA; a literature review; and an online resources investigation. Qualified
candidates were those with knowledge of learning effects and expertise in mobile sources and
manufacturing sectors. Per instructions from EPA, RTI aimed to select three reviewers from the
candidate pool based on all of the following criteria:
¦ Their expertise, knowledge, and experience;
¦ Their adherence to the COI guidance in the EPA's Peer Review Handbook (U.S.,
EPA, 2015); and,
¦ The diversity of their relevant scientific and technical perspectives.
Three candidates were highlighted based on recommendations from subject matter
experts and those with relevant expertise. RTI contacted these candidates to ascertain their
availability and potential COI. Each candidate completed a COI disclosure form to identify any
and all real or perceived COI or bias, including funding sources, employment, public statements,
and other areas of potential conflict, in accordance with EPA's Peer Review Handbook (U.S.
EPA, 2015). A template of the COI disclosure form completed by the candidates is included in
Appendix A. RTI staff supporting the peer review also underwent a COI investigation to
corroborate the independence and a lack of bias across all components of the peer review.
Based on the candidates' availability and qualifications, the information provided in the
completed COI disclosure forms, and an independent COI investigation conducted by RTI staff,
RTI selected the following three candidates:
2-1
-------
¦ Marvin Lieberman, Ph.D., University of California, Los Angeles (UCLA) Anderson
School of Management
¦ Natarajan Balasubramanian, Ph.D., Whitman School of Management, Syracuse
University
¦ Chad Syverson, Ph.D., University of Chicago Booth School of Business.
All three selected peer reviewers reported no COI on the disclosure form and were
identified to be in compliance with EPA's Peer Review Handbook (U.S. EPA, 2015). EPA
reviewed and approved the list of candidates selected by RTI as appropriate choices from the
candidate pool. Copies of the selected candidate resumes are included in Appendix B of this
report.
RTI provided the peer reviewers with the following materials to guide the evaluations:
¦ EPA-developed Peer Review Charge (see Appendix C)
¦ Technical Work Product Cost Reduction through Learning In Manufacturing
Industries and In the Manufacture of Mobile Sources (hereafter referred to as the
EPA Report)
The peer reviewers met with EPA once by conference call in March 2016 to give peer
reviewers the opportunity to ask questions about the context of the study. Peer reviewer
questions and answers regarding the charge are included in Appendix D.
RTI received the review reports and cover letters that stated the reviewer's name, the
name and address of the reviewer's organization, the documents that were received and reviewed
by the reviewer, and a statement of any real or perceived COI from each of the reviewers, and
forwarded the reports to EPA by the requested dates. The review reports included the responses
to the charge questions and any additional comments or recommendations. The cover letters and
the review reports are included in Appendix E of this report.
Peer reviewers were provided with an honorarium of $4000 to compensate for their
effort. The following sections provide the findings of the peer review.
2-2
-------
SECTION 3
SUMMARY OF FINDINGS
This section provides a summary of the comments received from the three reviewers:
Marvin Lieberman (UCLA), Natarajan Balasubramanian (Syracuse University), and Chad
Syverson (University of Chicago). The charge directed peer reviewers to evaluate the EPA
Report using the following five criteria: (1) clarity of the presentation, (2) overall approach and
methodology, (3) appropriateness of the studies included and other inputs, (4) data analyses
conducted, and (5) appropriateness of the conclusions. The remaining summary of comments
have been organized into sections according to these criteria, and other comments have been
included as the final section. Please see Appendix E for the complete reports from each peer
reviewer.
3.1 Overview of the Peer Reviewer Comments
Overall, the reviewers found the EPA Report to be well-executed, with a reasonable
approach, inputs, and conclusions. Comments received on the overall report include the
following:
¦ "The overall conclusion that learning-by-doing occurs in the mobile source sector is
well-founded and largely indisputable" (Dr. Balasubramanian)
¦ "On balance, the study is a very fine review of the literature on learning by doing in
general, but especially with regard to its manifestation in manufacturing operations
during the past few decades... .The report does achieve the intended goal of being a
definitive, reliable, single source of information demonstrating the occurrence of
learning in general and in the mobile source industry specifically" (Dr. Syverson)
¦ "I find the report to be comprehensive, and I believe it does a good job of
characterizing the rates of learning typically found in transportation equipment
manufacturing plants" (Dr. Lieberman).
Dr. Lieberman added that "Dr. Linda Argote of Carnegie Mellon University, the Subject
Matter Expert for the report, is widely regarded as the world expert on industrial learning curves,
having published numerous research studies in this topic area and a major book."
Reviewer comments included technical suggestions, such as recommendations to
improve methods transparency, as well as requests for clarification, three additional studies for
consideration, and a few clerical edits.
3-1
-------
3.2 Clarity of the Presentation
The reviewers felt the overall presentation and organization of the report was generally
clear and easy to follow. Comments provided from the reviewers to further improve clarity
included the following:
¦ Dr. Balasubramanian recommended that the report explicitly state the objectives of
the report and elaborate the summary of the literature review. In addition, Dr.
Balasubramanian suggested replacing the term "best estimate" to avoid confusion
with the econometric definition.
¦ Dr. Syverson requested additional context around the Laitner and Sanstad (2004)
analysis and how it might influence supply-side learning.
¦ Dr. Lieberman recommended that the organizational knowledge discussed in Section
9 would better inform the reader at an earlier location in the report. He also noted an
inconsistency among the reported temporal basis of the depreciation rates in Section
4.4 as a source of confusion.
3.3 Overall Approach and Methodology
Comments regarding the approach and methodology were generally positive; however,
additional clarification was recommended regarding the approaches and assumptions made in the
report. All reviewers provided comments on the method of estimating the progress ratios and
suggested that the report clarify the standard error calculation methods. Drs. Lieberman and
Balasubramanian discussed various methods that might be used to compute the progress ratio
and variations that may occur across industries, but stated that the estimated ratio is justified and
reasonable.
Dr. Balasubramanian recommended a more detailed summary of the literature review to
accompany the table in Section 3.3 of the report. Dr. Balasubramanian also suggested a detailed
discussion about the report's use of cumulative output as a predictor and the variation and
uncertainty associated with learning-by-doing, including the potential effect of other factors.
Furthermore, the peer reviewer recommended greater context regarding the five highlighted
studies in the discussion of the weighted-average progress ratio to improve the transparency of
the approach. Additionally, Dr. Balasubramanian stated that the overview of the report endeavors
to provide an analysis of the learning effect by industry but ultimately provides one estimate for
the entire sector. He suggested that using the means of the subgroups rather than the mean of the
group as a whole may be useful if and when estimates become available.
Dr. Lieberman suggested that the average value is useful for forecasting purposes, but
cautioned against any implication that the estimated progress ratio is a precise and universal
3-2
-------
standard due to variation across products, plants, and processes in the sector. Dr. Lieberman also
suggested that the report recognize the emphasis on cumulative output as a predictor.
Dr. Syverson stated that the consistency across progress ratio estimates is striking, but
differences across industry and outcome measures cannot be ruled out with only five studies. The
reviewer recommended further study and discussion where possible. Additionally, Dr. Syverson
recommended further inquiry into the net versus gross rate of learning and depreciation to help
determine whether there is variation in when and how to apply a learning effect.
3.4 Appropriateness of the Studies Included and Other Inputs
All three peer reviewers stated that the literature review is comprehensive. Each peer
reviewer identified an additional paper or article that may add insight to the literature review
provided in the report, but the peer reviewers stated that their exclusion would not detract from
the findings of the report. The studies recommended for consideration are: Hendel and Spiegel
(American Economic Journal: Applied Economics, January 2014), Balasubramanian and
Lieberman (The Journal of Industrial Economics, 2011) and Haunschild and Rhee (Management
Science, November 2004).
3.5 Data Analyses Conducted
All three peer reviewers found data analyses reasonable and appropriate for the objectives
of the study. Dr. Lieberman stated that the report surveyed "a substantial amount of literature"
and that it characterizes the literature well. Similarly, Dr. Syverson commented that the report
"does an excellent job of sorting through the large research literature to focus on studies that are
most germane to its mission."
Reviewers posed a few comments on the elements considered within the studies and
across the sector. For example, Dr. Lieberman stated that the studies are not based on the total
costs of production; therefore, forecasts will need to consider cost reduction of parts. He
recommended further information of the nature of the studies' cost analysis would be helpful to
improve the transparency of the approach proposed in the report. Dr. Lieberman also suggested
that the report highlight the difficulties of incorporating learning and economies of scale as
separate elements, and discuss the slope of the learning curve.
Dr. Balasubramanian requested clarification as to whether all five studies used
econometric techniques to causally estimate the effect of learning-by-doing. He recommended
that the report "characterize the estimated weighted-average progress ratio as the association
between unit costs and cumulative output, rather than as the effect of learning on costs."
3-3
-------
Furthermore, the reviewer also noted that it is not clear that all studies used the same method to
compute standard error.
Finally, Dr. Syverson stated that the report should clarify the discussion on shipment
inventory.
3.6 Appropriateness of the Conclusions
The peer reviewers unanimously support the conclusions reached in the study. Dr.
Syverson specifically commented on the study's interpretation that heterogeneity in learning
rates may be large within and across organizations and industries, and concurred that it is a
critical aspect to highlight. Dr. Lieberman listed several strengths of the report, and stated that it
"is likely to be helpful in providing a basis for incorporating forecasts of learning into EPA and
other government rulemaking." Dr. Balasubramanian remarked that the conclusion is "well-
founded and largely indisputable."
It should be noted that while Dr. Lieberman suggested that, with respect to the estimated
progress ratio, a more conservative approach would be to use the smallest learning rate of the
sample of five studies (87%), he also agreed that because the five studies used to estimate the
mobile source progress ratio are in the same range, "[djepending on the purpose at hand, one
could justify using 84.3% or 87%, in my opinion."
3-4
-------
REFERENCES
Balasubramanian, N. and M. B. Lieberman. (2011). Learning-by-doing and market structure.
The Journal of Industrial Economics. 54(2): 177-198
Haunschild, P.R. and M. Rhee. (2004). The role of volition in organizational learning: the case of
automotive product recalls. Management Science. 50, 11: 1545-1560.
Hendel, I. and Y. Spiegel. (2014). Small steps for workers, a giant leap for productivity.
American Economic Journal: Applied Economics. (5(1): 73-90
U.S. Environmental Protection Agency (EPA). (2015). EPA Science Policy Council Peer Review
Handbook, 4th Edition. Prepared for the U.S. Environmental Protection Agency Science
and Technology Policy Council under the direction of the EPA Peer Review Advisory
Group, October 2015. EPA/100/B-15/001. https://www.epa.gov/osa/peer-review-
handbook-4th-edition-2015.
R-l
-------
APPENDIX A
CONFLICT OF INTEREST ANALYSIS AND BIAS QUESTIONNAIRE
A-l
-------
Conflict of Interest Analysis and Bias Disclosure Form
Instructions:
This disclosure form has been developed in accordance with EPA's Peer Review
Handbook, 4th Edition (2015). The questions help identify any conflicts of interest and
other concerns regarding each candidate reviewer's ability to independently evaluate the
compendium study on industrial learning in the mobile source sector, developed by ICF
International. The compendium, entitled "Cost Reduction through Learning in
Manufacturing Industries and in the Manufacture of Mobile Sources" (referred to as
"subject topic" on the following page), is intended to be used by EPA to ensure that the
learning impacts in EPA's cost estimates are based on a comprehensive survey of the
literature and focused on learning effects in the mobile sources sector.
Please answer Yes or No in response to each question to the best of your knowledge and
belief. If you answer Yes to any of the questions, please provide a detailed explanation
on a separate sheet of paper.
Answering Yes to any of the questions will not necessarily result in disqualification, but a
record of any conflicts of interest is necessary to ensure that the peer review is composed
of an unbiased group of peer reviewers. RTI International will include the responses as
part of the published peer review record.
It is expected that the candidate make a reasonable effort to obtain the answers to each
question. For example, if you are unsure whether you or a relevant associated party (e.g.,
spouse, dependent, significant other) has a relevant connection to the peer review subject,
a reasonable effort such as calling or emailing to obtain the necessary information should
be made.
By signing the attached form you certify that:
1. You have fully and to the best of your ability completed this disclosure form,
2. You will update your disclosure form promptly by contacting the RTI
International peer review facilitator if relevant circumstances change,
3. You are not currently arranging new professional relationships with, or obtaining
new financial holdings in, an entity (related to the peer review subject) which is
not yet reported, and
4. This signature page, based on information you have provided, and your CV may
be made public for review and comment.
-------
U.S. Environmental Protection Agency
Conflict of Interest Inquiry
You have been requested by EPA to serve as a Peer Reviewer for the compendium study "Cost
Reduction through Learning in Manufacturing Industries and in the Manufacture of Mobile
Sources" (referred to below as "subject topic"), and your involvement in certain activities could pose a
conflict of interest or create the appearance of a loss of impartiality in your review. Although your
involvement in these activities is not necessarily grounds for exclusion from the peer review,
affiliations or activities that could potentially lead to conflicts of interest are included in the table.
Please complete the table and sign the certification below. If you have any questions, contact
jrichkus@rti.org at your earliest convenience to discuss any potential conflict of interest issues.
Conflict of Interest Analysis
YES
NO
a. To the best of your knowledge and belief, is there any connection between the subject topic
and any of your and/or your spouse's compensated or uncompensated employment,
including government service, during the past 24 months?
b. To the best of your knowledge and belief, is there any connection between the subject topic
and any of your and/or your spouse's research support and project funding, including from
any government source, during the past 24 months?
c. To the best of your knowledge and belief, is there any connection between the subject topic
and any consulting by you and/or your spouse, during the past 24 months?
d. To the best of your knowledge and belief, is there any connection between the subject topic
and any expert witness activity by you and/or your spouse, during the past 24 months?
e. To the best of your knowledge and belief, have you, your spouse, or dependent child, held
in the past 24 months, any financial holdings (excluding well-diversified mutual funds and
holdings, with a value less than $15,000) with any connection to the subject topic?
f. Have you made any public statements or taken positions on or closely related to the subject
topic under review?
g. Have you had previous involvement with the development of the document (or review
materials) you have been asked to review?
h. To the best of your knowledge and belief, is there any other information that might
reasonably raise a question about an actual or potential personal conflict of interest or bias?
i. To the best of your knowledge and belief, is there any financial benefit that might be
gained by you or your spouse as a result of the outcome of this review?
CERTIFICATION
I hereby certify that I have read the above statements and, to the best of my knowledge and belief, no
conflict of interest exists that may diminish my capacity to provide an impartial, technically sound,
objective review of the subject matter or otherwise result in a biased opinion.
(Name - please print)
(Signature)
(Date)
EPA Peer Review Handbook: Conflict of Interest Memoranda for ISI
J-6
-------
APPENDIX B
PEER REVIEWER RESUMES
B-l
-------
Natarajan Balasubramanian
Associate Professor (Strategy)
Whitman School of Management
Syracuse University
721 University Ave, Syracuse, NY 13244. Email: nabalasu@syr.edu
Current Position
Associate Professor, Whitman School of Management, 2013-
Prior Related Employment
Assistant Professor, Whitman School of Management, (2009-2013)
Assistant Professor (Strategy), College of Business Administration, Florida International
University, Miami (2007-2009)
Adjunct Lecturer, Stephen M Ross School of Business, University of Michigan, Ann Arbor,
Michigan (Mar 2006-Apr 2006)
Teaching and Research Assistant, UCLA Anderson School of Management, Los Angeles,
California (2002-2007)
Prior Corporate Employment
Manager, Andersen, Saudi Arabia (2002)
Customer Relationship Manager, Infosys, USA (2001-02)
Senior Consultant, Arthur Andersen, India (1996-2000)
Education
• PhD (Management), UCLA Anderson School of Management (2007)
(Dissertation Committee Chair: Marvin Lieberman)
• M.A. (Economics), UCLA (2005)
• PGDM (MBA), Indian Institute of Management, Bangalore, India (1996)
• B. Tech (BS) in Chemical Engineering, Indian Institute of Technology, Madras, India (1994)
Published Works
• Starr E., Balasubramanian N. and Sakakibara, M., 2014. Enforcing Covenants Not to Compete: The
Life-Cycle Impact on New Firms, in Academy of Management Proceedings, 2014:1 13238;
doi: 10.5465/AMBPP.2014.216
• Lee J. and Balasubramanian N., 2013. Who leads whom? Technological leadership in
nanotechnology: Evidence from patent data, in Restoring America's Global Competitiveness through
Innovation, Eds. Ben Kedia and Subash Jain., Edward Elgar Publishing.
• Ramanathan, S., Balasubramanian, N. and Krishnadas, R, 2013. Is the Macroeconomic Environment
during Infancy a Risk Factor for Adolescent Behavioral Problems? JAMA Psychiatry, 70(2):218-25.
-------
• Balasubramanian N., 2011. New Plant Venture Performance Differences Among Incumbent,
Diversifying, and Entrepreneurial Firms: The Impact of Industry Learning Intensity. Management
Science. 57(3): 549-565
• Balasubramanian N. and Lieberman M., 2011. Learning by Doing and Market Structure. Journal of
Industrial Economics. 59(2): 177-198 (Lead Article)
• Balasubramanian N. and Sivadasan J., 2010. What happens when firms patent? New evidence from
US Census Data. Review of Economics and Statistics. 93(1) 126:146
• Balasubramanian N. and Lieberman M., 2010. Industry Learning Environments and the
Heterogeneity of Firm Performance, Strategic Management Journal, 31(4), pp. 390-412.
o Also as Balasubramanian N. and Lieberman M., Industry Learning Environments and the
Heterogeneity of Firm Performance, In K. Mark Weaver (Ed.), Proceedings of the Sixty-Fifth
Annual Meeting of the Academy of Management (CD), ISSN 1543-8643.
• Balasubramanian N. and Sivadasan J., 2009. Capital Resalability, Productivity Dispersion and
Market Structure, Review of Economics and Statistics, 91(3), pp. 547-557.
• Balasubramanian N. and Lee J., 2008. Firm Age and Innovation, Industrial and Corporate Change
17(5), pp. 1019-1047.
o Also as Balasubramanian N. and Lee J., Firm Age and Innovation, In George T. Solomon
(Ed.), Proceedings of the Sixty-Sixth Annual Meeting of the Academy of Management
(CD), ISSN 1543-8643.
• Chacar A, Balasubramanian N. and Vissa B., Does it pay to be a Business Group Member? 2008
Proceedings of the Academy Of International Business-SE (USA)
Works under Revision
• Starr E., Balasubramanian N. and Sakakibara, M. Enforcing Covenants Not to Compete: The Life-
Cycle Impact on New Firms (Management Science, 2nd Round R&R)
o Also as Working Paper CES 14-27, Center for Economic Studies, U.S. Census Bureau
• Balasubramanian N., Lee, J., and Sivadasan J. Deadlines, Work Flows, Task Sorting, and Work
Quality (Management Science, 2nd Round R&R)
• Garcia, R., Balasubramanian N. and Lieberman M. Measuring Value Creation and Appropriation
in Firms: Application of the VCA Model (Strategic Management Journal, 1st Round R&R)
• Lieberman M., Balasubramanian N. and Garcia, R. Toward a Dynamic Notion of Value Creation
and Appropriation in Firms: The Concept and Measurement of Economic Gain (Strategic
Management Journal, 2nd Round R&R)
Working Papers
• Balasubramanian, N., Chang J. W., Sakakibara, M., Sivadasan J. and Starr E. Locked in? Noncompete
Enforceability and the Mobility and Earnings of High Tech and High Earnings Workers
• Sakakibara, M. and Balasubramanian N. Human Capital of Spinoffs. Working Paper CES 15-06,
Center for Economic Studies, U.S. Census Bureau
-------
• Sakakibara, M. and Balasubramanian N. Spinout Formation: Do Opportunities and Constraints
Benefit High Human Capital Founders? Working Paper CES 15-07, Center for Economic Studies, U.S.
Census Bureau
• Sakakibara, M. and Balasubramanian N. Incidence and Performance of Spinoffs: A Cross-Industry
Analysis.
• Balasubramanian N., and Deb, P. Learning by Doing and Capital Structure
• Balasubramanian N., Dharwadkar, R., and Sivadasan J. Firm Growth and Governance: Running to
Stand Still?
• Dharwadkar, R., Balasubramanian N., and Suh, S. Managerial Insulation and Research and
Development Investments: An Empirical Examination.
Competitive Awards and Grants
• Kauffman Junior Faculty Fellow in Entrepreneurship Research, 2012
• Whitman Research Fellow, Whitman School of Management, 2014-2015
• Guttag Junior Faculty Award, Whitman School of Management, 2012
• Finalist, Outstanding Dissertation Award Competition, BPS Division, Academy of Management, 2008
• Entrepreneurship Research Grant, Winter 2008 (Joint with Jeongsik Lee, Georgia Inst, of Tech.)
• UCLA Dissertation Year Fellowship (2006-2007)
• California Census Research Data Center Dissertation Fellowship (2005-2007)
• Gladys Byram Fellowship (2002-2006)
External Service Activities
• Member, Editorial Board, Journal of Management (2014-)
• Member, Research Committee, BPS Division, Academy of Management (2015-)
• Ad hoc reviewer: National Science Foundation, Kauffman Foundation, Management Science,
Strategic Management Journal, Organization Science, Journal of Management Studies, Strategic
Entrepreneurship Journal, Journal of Industrial Economics, Journal of Labor Economics, Journal of
Economics & Management Strategy, BE Journal of Economic Analysis & Policy, Research Policy,
Journal of Business Venturing, European Financial Management
-------
Internal Service Assignments
• Department Representative, Doctoral Board, Whitman School of Management (2014-)
• Member, Promotion & Tenure Committee, Whitman School of Management (2014-2015)
• Department Representative, Undergraduate Board, Whitman School of Management (2009-2014)
• Co-director, Department Speaker Series (2011-13)
• Conducted several "How to Prepare for a Case" Sessions in the 1st Year MBA Orientations
• Judge for Whitman Annual Case Competitions
• Member, Journal List Committee, College of Business, Florida International University, (2007-09)
Presentations
• Florida International University, January 2016 (Invited)
• Strategic Science Mini Conference, Philadelphia, November 2015 (Invited)
• Whitman School of Management, February and April 2015
• Kauffman Emerging Scholars Conference, Kansas City, October 2014 (Invited)
• Annual Meeting of the Academy of Management, Philadelphia, August 2014 (Peer-reviewed)
• Goizueta School of Business, Emory University, November 2012 (Invited)
• Annual Meeting of the Academy of Management, Boston, August 2012 (Peer-reviewed)
• Krannert School of Management, Purdue University, October 2011 (Invited)
• Stern School of Business, NYU, August 2011 (Invited)
• Annual Meeting of the Academy of Management, Chicago, August 2009 (Peer-reviewed)
• Annual Meeting of the Academy of Management, Anaheim, August 2008 (Peer-reviewed)
• Annual Meeting of the American Economic Association, 2008 (Peer-reviewed)
• Annual Meeting of the Academy of Management, Atlanta, August 2006 (Peer-reviewed)
• Atlanta Competitive Advantage Conference, June 2006 (Peer-reviewed)
• The Evolution of Ideas in Innovation and Entrepreneurship: A Conference to Honor Michael Gort's
Contributions, University of Washington, St Louis, June 2006 (Invited)
• Annual Meeting of the Academy of Management, Honolulu, August 2005 (Peer-reviewed)
• Annual Meeting of the Western Economic Association, San Francisco, July 2005 (Peer-reviewed)
• Business and its Social Environment (BASE) Conference, Kellogg School of Business, June 2005
(Peer-reviewed)
• Innovation Workshop, Anderson School of Management, UCLA, May 2005
• Consortium for Competition and Co-operation (CCC), UC Berkeley, April 2005
Teaching Experience
• Whitman School of Management, Syracuse University.
-------
Courses: SHR 247; SHR 447; MBC 618; MBC 619; MBC 645 (Strategic Management for Undergraduates
and Graduates)
• College of Business Administration, Florida International University, Miami (2007-2009)
Courses: Strategic Management (for Undergraduates and Graduates)
• Stephen M Ross School of Business, University of Michigan, Ann Arbor, Winter 2006.
Course: Competitive Tactics and Competition Policy, MBA Elective.
• Teaching Assistant, University of California, Los Angeles, 2002-2006.
Courses: Business Strategy and Negotiations Analysis
Teaching Awards
• Best Professor, Downtown MBA Program, Florida International University (April 2008)
Professional Memberships
• Member, Academy of Management
• Member, American Economic Association
-------
MARVIN B. LIEBERMAN
Office Address: Home Address:
UCLA Anderson School of Management 180 Acacia Lane
Gold Hall, Room B-415 Newbury Park, CA 91320
Los Angeles, CA 90095-1481
(310) 206-7665
E-mail: marvin ,lieberman@anderson .ucla.edu
Education
Ph.D. Harvard University
A.B. Harvard University
Business Economics 1982
Economics 1976
Dissertation
Title: The Learning Curve, Pricing, and Market Structure in the Chemical Processing Industries
Committee: Richard E. Caves, Michael E. Porter, A. Michael Spence
Academic Positions
2001 - present: Professor, UCLA Anderson School of Management
1990 - 2001: Associate Professor, Anderson Graduate School of Management, UCLA
1989 - 1990: National Fellow, Hoover Institution
1982 - 1989: Assistant Professor of Business Policy, Graduate School of Business, Stanford
University
1979 - 1981: Teaching Fellow (Introductory Economics), Harvard University
Academic Honors
Strategic Management Society Fellow
TMS Distinguished Speaker, Fall 2009 INFORMS Conference
1996 Best Paper Prize, Strategic Management Journal
Hoover National Fellowship, 1989-90
Shigeo Shingo Prize for Manufacturing Excellence, 1989
Harvard Business School Division of Research Thesis Fellowship, 1981
Browder Thompson Best Paper Award (IEEE), 1979
National Science Foundation Fellowship, 1976-80
-------
Journal Articles
"Entry, Exit and the Potential for Resource Redeployment," (with Gwendolyn Lee and Tim Folta).
Forthcoming, Strategic Management Journal.
Best paper award finalist, BPS Division, 2010 Academy of Management Meeting.
"Production Frontier Methodologies and Efficiency as a Performance Measure in Strategic
Management Research," (with C.M. Chen and Magali Delmas). Strategic Management
Journal, Vol. 36, January 2015, pp. 19-36.
"Conundra and Progress: Research on Entry Order and Performance," (with David Montgomery). Long
Range Planning, Special Issue on Entry Timing Strategies, Vol. 46, August 2013, pp. 313-324.
"Learning by Doing and Market Structure," (with Natarajan Balasubramanian). Journal of Industrial
Economics, Vol. 59, No. 2, June 2011, pp. 177-198.
"Internal and External Influences on Adoption Decisions in Multi-Unit Firms: The Moderating Effect
of Experience," (with Daniel Simon). Strategic Organization, Vol. 8, No. 2, May 2010, pp.
132-154.
"Industry Learning Environments and the Heterogeneity of Firm Performance," (with Natarajan
Balasubramanian). Strategic Management Journal, Vol. 31, No. 4, April 2010, pp. 390-412.
"Acquisition vs. Internal Development as Modes of Market Entry," (with Gwendolyn Lee). Strategic
Management Journal, Vol. 31, No. 2, February 2010, pp. 140-158.
"How to Measure Company Productivity using Value-added: A Focus on Pohang Steel (POSCO)"
(with Jina Kang). Asia Pacific Journal of Management., Vol. 25, No. 2, June 2008, pp. 209-
224.
"Why Do Firms Imitate Each Other?" (with Shigeru Asaba). Academy of Management Review, Vol.
31, No. 2, April 2006, pp. 366-385.
Reprinted in Competitive Strategy, C. Maritan and M. Peteraf, eds., Edward Elgar, 2011.
"Assessing the Resource Base of Japanese and U.S. Auto Producers: A Stochastic Frontier Production
Function Approach," (with Rajeev Dhawan), Management Science, Vol. 51, No. 7, July 2005,
pp. 1060-1075.
"The Birth of Capabilities: Market Entry and the Importance of Pre-History," (with Constance Helfat),
Industrial and Corporate Change, Vol. 11, No. 4, August 2002, pp. 725-760.
Reprinted in Competitive Strategy, C. Maritan and M. Peteraf, eds., Edward Elgar, 2011.
"The Magnesium Industry in Transition," Review of Industrial Organization, Vol. 19, June 2001, pp.
71-79.
"Inventory Reduction and Productivity Growth: Linkages in the Japanese Automotive Industry," (with
Lieven Demeester), Management Science, Vol. 45, No. 4, April 1999.
"The Empirical Determinants of Inventory Levels in High-Volume Manufacturing," (with Susan
Helper and Lieven Demeester), Production and Operations Management, Vol. 8, No. 1, Spring
1999, pp. 44-55.
-------
"Comparative Productivity of Japanese and US Steel Producers, 1958-1993," (with Douglas Johnson),
Japan and the World Economy, Vol. 11, No. 1, January 1999, pp. 1-27.
"First-Mover (Dis)Advantages: Retrospective and Link with Resource-Based View," (with David
Montgomery), Strategic Management Journal, Vol. 19, No. 12, December 1998, pp. 1111-
1125.
"Patent Trends in Steelmaking Technologies," (with Aya Chacar), Iron and Steel Engineer, Vol. 75,
No. 8, August 1998, pp. 72-73.
"Inventory Reduction and Productivity Growth: A Comparison of the Japanese and US Automotive
Sectors," (with Shigeru Asaba). Managerial and Decision Economics, Special issue on
Japanese Technology Management, Vol. 18, No. 2, March 1997, pp. 73-85.
"Determinants of Vertical Integration: An Empirical Test "Journal of Industrial Economics, Special
Issue on Vertical Relationships, Vol. 39, No. 5, September 1991.
"Exit from Declining Industries: ' Shakeout' or ' Stakeout'?" Rand Journal, Vol. 21, No. 4, Winter
1990.
Reprinted in Applied Industrial Economics, L. Phlips, ed., Cambridge University Press, 1998.
"Firm-Level Productivity and Management Influence: A Comparison of U.S. and Japanese Automobile
Producers," (with Lawrence Lau and Mark Williams). Management Science, Vol. 36, No. 10,
October 1990.
"The Learning Curve, Technology Barriers to Entry, and Competitive Survival in the Chemical
Processing Industries." Strategic Management Journal, Nol. 10, No. 5, September-October
1989.
Reprinted in Innovation, Evolution of Industry, and Economic Growth, D.B. Audretsch and S.
Klepper, eds., Edward Elgar, 1999.
"Capacity Utilization: Theoretical Models and Empirical Tests." European Journal of Operational
Research, Vol. 40, No. 2, May 1989.
"First-Mover Advantages," (with David Montgomery). Strategic Management Journal, Vol. 9,
Summer 1988.
Recipient of 1996 SMJ Best Paper Prize (awarded for articles more than five years old with
significant impact on the field of strategic management).
Reprinted in Readings in Marketing Strategy, V. J. Cook, J. Larreche and E. C. Strong, eds.,
Scientific Press, Redwood City, 1989.
Reprinted in Strategic Management, (volume of the International Library of Critical Writings
on Business and Management), J. Birkenshaw, ed., Edward Elgar, London, 2003.
Reprinted in Competitive Strategy, C. Maritan and M. Peteraf, eds., Edward Elgar, 2011.
"Post-Entry Investment and Market Structure in the Chemical Processing Industries," Rand Journal of
Economics, Vol. 18, No. 4, Winter 1987.
-------
"Market Growth, Economies of Scale, and Plant Size in the Chemical Processing Industries," Journal
of Industrial Economics, Vol. 36, No. 2, December 1987.
"The Learning Curve, Diffusion, and Competitive Strategy," Strategic Management Journal, Vol. 8,
No. 5, September-October 1987.
Reprinted in Competitive Strategy, C. Maritan and M. Peteraf, eds., Edward Elgar, 2011.
"Patents, Learning by Doing, and Market Structure in the Chemical Processing Industries,"
International Journal of Industrial Organization, Vol. 5, No. 3, September 1987.
"Strategies for Capacity Expansion," Sloan Management Review, Vol. 28, No. 4, Summer 1987.
"Excess Capacity as a Barrier to Entry: An Empirical Appraisal," Journal of Industrial Economics,
Vol. 35, No. 4, June 1987.
Reprinted in The Empirical Renaissance in Industrial Economics, T. Bresnahan and R.
Schmalensee, eds., Basil Blackwell, London, 1987.
"Investment and Coordination in Oligopolistic Industries," (with Richard Gilbert), Rand Journal of
Economics, Vol. 18, No. 1, Spring 1987.
"The Learning Curve and Pricing in the Chemical Processing Industries," Rand Journal of Economics,
Vol. 15, No. 2, Summer, 1984.
"A Literature Citation Study of Science-Technology Coupling in Electronics," Proceedings of the
IEEE, Vol. 66, No. 1, January 1978.
Browder Thompson prize for best research paper by an author under 30 published in an IEEE
journal during 1978.
Conference Proceedings (Published)
"An Extension of the VCA Model To Estimate Stakeholder Value Appropriation" (with Roberto
Garcia), Best Papers Proceedings, Academy of Management, 2012.
"Relatedness and Market Exit," (with Gwendolyn Lee and Tim Folta), Best Papers Proceedings,
Academy of Management, 2010. BPS Division Best Paper Award finalist.
"Acquisition vs. Internal Development as Entry Modes for New Business Development: The Dynamics
of Firm-Market Relevance," (with Gwendolyn Lee), Best Papers Proceedings, Academy of
Management, 2007.
"Industry Learning Environments and the Heterogeneity of Firm Performance," (with Natarajan
Balasubramanian), Best Papers Proceedings, Academy of Management, 2006.
"Why Do Firms Behave Similarly? A Study on New Product Introduction in the Japanese Soft-drink
Industry," (with Shigeru Asaba), Best Papers Proceedings, Academy of Management, 1999.
"Determinants of Vertical Integration: An Empirical Test," Best Papers Proceedings, Academy of
Management, 1991.
-------
Book Chapters
"Business Model Innovation and Replication: Implications for the Measurement of Productivity," (with
Natarajan Balasubramanian, Joan Enric Ricart and Roberto Garcia Castro). Forthcoming as
Chapter 10 of Productivity and Business Strategy, Oxford University Press.
"First Mover/ Pioneer Strategies," (with David Montgomery), in Venkatesh Shankar and Gregory S.
Carpenter, eds., Handbook of Marketing Strategy, (Edward Elgar, 2012).
"Business Imitation," (with Shigeru Asaba), in Charles Wankel, ed., Handbook of 21st Century
Management (Sage, 2007).
"Organizing for Technological Innovation in the U.S. Pharmaceutical Industry," (with Aya Chacar), in
Joel Baum and Olav Sorenson, eds., Geography and Strategy (Advances in Strategic
Management, Volume 20, Elsevier, 2003).
"Dow Chemical and the Magnesium Industry," in D. I. Rosenbaum, ed., Market Dominance: How
Firms Gain, Hold or Lose It and the Impact on Economic Performance, Praeger, Westport, CT,
1998, pp. 69-87.
"Distribution of Returns Among Stakeholders: Method and Application to US and Japanese Auto
Companies," (with Aya Chacar), in H. Thomas and D. O'Neal, eds., Strategic Discovery:
Competing in New Arenas, Wiley, 1997, pp. 299-313.
"Strategy of Market Entry: Pioneer or Follow?" (with David Montgomery). In Handbook of Business
Strategy, H. E. Glass, ed., Warren, Gorham & Lamont, 1991.
"Inventory Reduction and Productivity Growth: A Study of Japanese Automobile Producers." In
Manufacturing Strategy, J. E. Ettlie, M. C. Burstein and A. Feigenbaum, eds., Kluwer
Academic Publishers, 1990.
"Learning, Productivity, and US-Japan Industrial Competitiveness." In Managing International
Manufacturing, K. Ferdows, ed., North Holland, 1989.
"Estimating the Benefits to Society from Integrated Circuit Innovations: the Case of MOS Dynamic
RAM's," in R. Wilson et al., Innovation, Competition and Government Policy in the
Semiconductor Industry, Lexington Books, 1981, pp. 122-31.
Other Publications
"First Mover Advantages," in Palgrave Encyclopedia of Strategic Management, 2013.
"Just-in-Time," in Palgrave Encyclopedia of Strategic Management, 2013.
"Cost," in Palgrave Encyclopedia of Strategic Management, 2013.
"Sunk Cost," in Palgrave Encyclopedia of Strategic Management, 2013.
"Cost Leadership," in Palgrave Encyclopedia of Strategic Management, 2014.
"The Revitalization of US Manufacturing," Institute for International Economic Studies, Tokyo, Japan,
Seminar Series 9706, June 1997.
-------
Papers Under Review and Work in Progress
Market Entry
"Who Imitates Whom? A Study on New Product Introductions in the Japanese Soft-drink Industry,"
(with Shigeu Asaba), January 2011. (Presented at USC, UCLA, Academy of Management
annual meeting (2012), Industry Studies Association Conference (2013).)
"Did First-Mover Advantage Survive the Dot-Com Crash?" December 2007. (Presented at CMU,
Emory, Maryland, NYU, Wharton, UC Berkeley, UCLA, University of Illinois, and the
Stanford and Utah Strategy Conferences.)
Economic Value Creation
"Measuring Value Creation and Appropriation in Firms: The VCA Model" (with Natarajan
Balasubramanian and Roberto Garcia). Under 3rd round review, Strategic Management
Journal.
"Toward a Dynamic Notion of Value Creation and Appropriation in Firms: The Concept and
Measurement of Economic Gain," (with Natarajan Balasubramanian and Roberto Garcia).
Under 2nd round review, Strategic Management Journal.
Cases and Teaching Notes
Lectures on "Creation and Distribution of Economic Value"
The Magnesium Industry in 1964 (A), S-BP-231A
The Magnesium Industry 1964-1974 (B), S-BP-231B
The Magnesium Industry 1974-1982 (C), S-BP-231C
Magnesium Industry Teaching Note
Learning Curve Computer Exercise
Teaching Note on the Learning Curve Computer Exercise
Note on Production Economics: Cost Structures and Process Types
Courses Taught
Business Strategy
Industry Structure and Competitive Strategy
Market Entry Strategy
Entrepreneurial Perspectives on Biotechnology
Strategies for Internet Business
Production/Operations Management
Introductory Economics
Invited Presentations
Carnegie-Mellon University
Columbia University
Dartmouth (Tuck)
Duke University
Emory University
Florida International University
Harvard University
INSEAD
Institute for International Economic Studies
Kobe University
London Business School
Massachusetts Institute of Technology
New York University
Northwestern University
Peking University
Southern Methodist University
University of British Columbia
University of California, Berkeley
-------
University of California, Irvine
University of California, San Diego
University of Chicago
University of Illinois, Champaign-Urbana
University of Maryland
University of Michigan
University of Minnesota
University of Pennsylvania (Wharton)
University of Pittsburgh
Professional Societies
Academy of Management
American Economic Association
University of Rochester
University of Southern California
University of Texas at Austin
University of Toronto
University of Washington
University of Wisconsin, Madison
U.S. Department of Justice
U.S. Federal Trade Commission
Washington University at St. Louis
Strategic Management Society (Fellow)
Industry Studies Association
Editorial
Strategic Management Journal (Editorial Board)
Production and Operations Management (Senior Editor)
Revised: February 2016
-------
CHAD SYVERSON
Curriculum Vitae
(January 2016)
University of Chicago Booth School of Business
5807 S. Woodlawn Ave.
Chicago, IL 60637
Phone: (773) 702-7815
chad.swerson@chicagobooth.edu
svverson@uchicago.edu
ACADEMIC POSITIONS
J. Baum Harris Professor of Economics', University of Chicago Booth School of Business, 2013-
Present
Professor of Economics', University of Chicago Booth School of Business, 2008-2013
(Charles M. Harper Faculty Fellow, 2012-2013)
Associate Professor (with tenure)', Department of Economics, University of Chicago, 2007-2008
Associate Professor, Department of Economics, University of Chicago, 2006-2007
Assistant Professor, Department of Economics, University of Chicago, 2001-2006
OTHER PROFESSIONAL APPOINTMENTS
Editor, RAND Journal of Economics, 2013-Present
Editor, Journal of Industrial Economics, 2013-2014
Associate Editor, Management Science, 2011-Present
Associate Editor, Journal of Economic Perspectives, 2012-2015
Associate Editor, Journal of Economics & Management Strategy, 2010-2014
Associate Editor, RAND Journal of Economics, 2007-2013
Associate Editor, Journal of Industrial Economics, 2005-2013
Editorial Board Member, B.E. Journals in Economic Analysis and Policy, 2005-2013
Research Associate', National Bureau of Economic Research (Productivity, Industrial
Organization, and Environmental and Energy Economics Programs), 2003-Present
Visiting Scholar, Federal Reserve Bank of Minneapolis, 2003, 2004, 2005
EDUCATION
Ph.D. Economics, University of Maryland, 2001
M.A. Economics, University of Maryland, 1998
B.S. Mechanical Engineering, University of North Dakota, 1996
B.A. Economics, University ofNorth Dakota, 1996
PUBLICATIONS
"Healthcare Exceptionalism? Performance and Allocation in the U.S. Healthcare Sector"
American Economic Review (with Amitabh Chandra, Amy Finkelstein, and Adam
Sacarny), forthcoming.
-------
"Productivity Dispersion in Medicine and Manufacturing" (with Amitabh Chandra, Amy
Finkelstein, and Adam Sacarny) American Economic Review Papers and Proceedings,
forthcoming.
Microeconomics, 2nd Ed. (with Austan Goolsbee and Steve Levitt), Worth, 2016.
"The Slow Growth of New Plants: Learning about Demand." (with Lucia Foster and John
Haltiwanger.) Economica, 83(329), (January 2016), 91-129.
"Geographic Variation in Rosiglitazone use Surrounding FDA Warnings in the Department of
Veterans Affairs" (with Vishal Ahuja, Min-Woong Sohn, John R. Birge, Elly Budiman-
Mak, Nicholas Emanuele, Jennifer M. Cooper, and Elbert S. Huang), Journal of
Managed Care & Specialty Pharmacy, 21(12), (December 2015), 1214-34.
"The Ongoing Evolution of US Retail: A Format Tug-of-War" (with Ali Hortacsu). Journal of
Economic Perspectives, 29(4), (Fall 2015), 89-112.
"Competition in the Audit Market: Policy Implications" (with Joseph Gerakos), Journal of
Accounting Research, 53(4), (September 2015), 725-775.
"Acquisitions, Productivity, and Profitability: Evidence from the Japanese Cotton Spinning
Industry" (with Serguey Braguinsky, Atsushi Ohyama, and Tetsuji Okazaki), American
Economic Review, 105(7), (July 2015), 2086-2119.
"Vertical Integration and Input Flows" (with Enghin Atalay and Ali Hortacsu). American
Economic Review, 104(4), (April 2014), 1120-48.
Review of Vaclav Smil's "Made in the USA: The Rise and Retreat of American Manufacturing."
Journal of Economic Literature, 52(3), (September 2014), 872-3.
"The Importance of Measuring Dispersion in Firm-Level Outcomes." in IZA World of Labor,
May 2014.
"Toward an Understanding of Learning by Doing: Evidence from an Automobile Plant" (with
Steve Levitt and John A. List), Journal of Political Economy, 121(4), (August 2013),
643-681.
"Indirect Costs of Financial Distress in Durable Goods Industries: The Case of Auto
Manufacturers" (with Ali Hortacsu. Gregor Matvos, and Sriram Venkataraman), Review
of Financial Studies, 26(5), (May 2013), 1248-1290.
Microeconomics. (with Austan Goolsbee and Steve Levitt), Worth, 2013.
"Will History Repeat Itself? Comments on 'Is the Information Technology Revolution Over?'"
International Productivity Monitor, 25, (Spring 2013), 37-40.
"Online vs. Offline Competition" (with Ethan Lieber), in Oxford Handbook of the Digital
Economy, Martin Peitz and Joel Waldfogel (eds.), 2012, 189-223.
"What Determines Productivity?" Journal of Economic Literature, 49(2), (June 2011), 326-65.
2
-------
"Is an Automaker's Road to Bankruptcy Paved with Customers' Beliefs?" (with Ali Hortacsu.
Gregor Matvos, Chaehee Shin, and Sriram Venkataraman), American Economic Review
Papers and Proceedings, 101(3), (May 2011), 93-7.
"Network Structure of Production." (with Enghin Atalay, Ali Hortacsu and Jimmy Roberts)
Proceedings of the National Academy of Sciences, 108(13), (March 29, 2011), 5199-
5202.
"Technological Change and the Growing Inequality in Managerial Compensation." (with Hanno
Lustig and Stijn Van Nieuwerburgh) Journal of Financial Economics, 99(3), (March
2011), 601-27.
"E-commerce and the Market Structure of Retail Industries." (with Maris Goldmanis, Ali
Hortacsu, and Onsel Emrq.) Economic Journal, 120(545), (June 2010), 651-82.
"How Do Incumbents Respond to the Threat of Entry? Evidence from the Major Airlines." (with
Austan Goolsbee.) Quarterly Journal of Economics, 123(4), (November 2008), 1611-33.
"Market Distortions when Agents are Better Informed: The Value of Information in Real Estate
Transactions." (with Steve Levitt.) Review of Economics and Statistics 90(4), (November
2008), 599-611.
"Reallocation, Firm Turnover, and Efficiency: Selection on Productivity or Profitability?" (with
Lucia Foster and John Haltiwanger.) American Economic Review 98(1), (March 2008),
394-425.
"Markets: Ready-Mixed Concrete." Journal of Economic Perspectives 22(1), (Winter 2008), 217-
33.
"Antitrust Implications of Home Seller Outcomes when using Flat-Fee Real Estate Agents." (with
Steve Levitt.) Brookings-Wharton Papers on Urban Affairs, 2008, 47-93.
"Prices, Spatial Competition, and Heterogeneous Producers: An Empirical Test." Journal of
Industrial Economics 55(2), (June 2007), 197-222.
"Cementing Relationships: Vertical Integration, Foreclosure, Productivity, and Prices." (with Ali
Hortacsu.) Journal of Political Economy 115(2), (April 2007), 250-301.
"Market Structure and Productivity: A Concrete Example." Journal of Political Economy 112(6),
(Dec. 2004), 1181-1222.
"Product Differentiation, Search Costs, and the Welfare Effects of Entry: A Case Study of S&P
500 Index Funds." (with Ali Hortacsu) Quarterly Journal of Economics 119(4), (May
2004), 403-456.
"Product Substitutability and Productivity Dispersion." Review of Economics and Statistics 86(2),
(May 2004), 534-550.
"Leak Testing of a Raised Face Weld Neck Flange." (with George Bibel, Dion Weinberger, and
Steven Dockter), Welding Research Council Bulletin, (January 2002).
3
-------
SUBMITTED PAPERS/WORKING PAPERS
"Competition and Regulation of Advertising: Evidence from Privatized Social Security in
Mexico." (with Justine Hastings and Ali Hortacsu)
"The Effects of Environmental Regulation on the Competitiveness of U.S. Manufacturing" (with
Michael Greenstone and John A. List)
"Once and Done: Leveraging Behavioral Economics to Increase Charitable Contributions" (with
Amee Kamdar, Steve Levitt, and John A. List)
RESEARCH PROJECTS AND AWARDS
PI, "Empirical Studies of Production in Plants and Firms." NSF Award SES-0820307, 2008-
2010.
PI, "Empirical Research on Vertical Integration and Productivity." NSF Award SES-0519062,
2005-2007.
"Patterns of Firm Expansion." (with Ali Hortacsu) Chicago Census Research Data Center, Project
CH-00573, 2007-2012.
"E-commerce and the Market Structure of Retail Industries." (with Ali Hortacsu). NET Institute,
2005.
Co-PI (with Ali Hortacsu). "Entry, Competition, and Welfare in the Mutual Fund Industry." NSF
Award SES-0242031. 2003-2005.
"How Do Incumbents Respond to the Threat of Entry on Their Networks: Evidence from the
Major Airlines." (with Austan Goolsbee). NET Institute, 2004.
"Technology Dispersion and Persistence within Industries." Olin Foundation Grant, University of
Chicago, 2005.
"Studies in Productivity." Olin Foundation Grant, University of Chicago, 2004.
"Market Structure and Competition in the Mutual Fund Industry." (with Ali Hortacsu.) Olin
Foundation Grant, University of Chicago, 2002.
"Productivity Heterogeneity and Market Segmentation." Chicago Census Research Data Center,
Project CH-00276, 2003-2005.
"Vertical Integration." (with Ali Hortacsu) Chicago Census Research Data Center, Project CH-
00277, 2003-2005.
"Economics of Production." Social Sciences Divisional Research Grant, University of Chicago.
2002-2003.
"Using Market Segmentation to Identify Plant-Specific Instruments." Center for Economic
Studies, U.S. Census Bureau, Project W099-21, 1999-2001.
4
-------
OTHER RESEARCH
Output Market Segmentation, Heterogeneity, and Productivity
Ph.D. Dissertation, University of Maryland, 2001. Profs. John Haltiwanger and John Shea, Co-
Chairs
"Production Function Estimation with Plant-Level Data: Productivity Proxies or Instrumental
Variables?" 2000.
"Healthcare Exceptionalism? Productivity and Allocation in the U.S. Healthcare Sector" NBER
Working Paper 19200 (with Amitabh Chandra, Amy Finkelstein, and Adam Sacarny)
INVITED PRESENTATIONS
Aarhus (keynote), American Enterprise Institute, Arizona, ASSA Annual Meeting (multiple),
Banco de Portugal, Bank of Canada, BEA (2), BLS, Boston University, Brookings Institution,
CAED 2008, 2010 (keynote) and 2012, CEA 2012 Meetings (keynote), Carnegie Mellon, CEPR
IO Meetings 2009 (keynote), Clemson, Colorado, Columbia GSB (3), Competition Bureau of
Canada, Dartmouth (Tuck), Department of Justice Antitrust Division, Drexel (2), Duke (3),
EARIE (keynote), ECB (keynote), ESEM, FRB Chicago, FRB Cleveland, FRB Kansas City,
FRB Minneapolis (3), FRB Philadelphia, Fed Board of Governors (2), FTC, Georgetown,
Harvard (Econ. [4] and HBS [2]), Illinois, Illinois-Chicago, Indiana, Iowa, Iowa State (2),
Kentucky, LSE (3), Mannheim, Michigan (Econ. [2] and Ross), Michigan State, Minnesota (2),
MIT (Econ. [3] and Sloan [3]), National Bank of Belgium (keynote), NBER IO and Productivity
meetings (multiple), NBER Summer Institute (multiple), NET Institute Conference 2005-2006,
North Dakota (2), Northwestern (Kellogg GSB [2]), Notre Dame, NYU (Econ. and Stern [2]),
OECD (2), Penn (Wharton), Penn State, Pittsburgh, Princeton (2), Purdue (2), Rice, Rochester,
SED 2004 and 2010, SITE 2003 and 2007-2008, Stanford (Econ and GSB [2]), Texas A&M,
Toronto (Rotman [2]), UBC (Sauder [2]), UC-Berkeley (Haas [4]), UC-Irvine, UCLA (3), UCSD
(2), UNSW, Utah Winter Business Economics Conference (2), University of Washington,
Washington Univ.-St. Louis (Olin), Wisconsin, Yale (3), World Bank, ZEW ICT Workshop 2012
(keynote).
PROFESSIONAL SERVICE
National Academy of Sciences Panel on Reengineering the Census Bureau's Annual Economic
Surveys, 2015-.
National Science Foundation Site Visit Review Panel, 2015.
National Academy of Engineering Making Value for America Committee, 2013-2015
National Academy of Engineering Committee on Manufacturing, Design, and Innovation, 2012
Graduate Business Council Faculty-Student Committee, Booth School of Business, 2011-
Census Scientific Advisory Committee (AEA representative), 2009-2012
Chicago Census Research Data Center Advisory Board, 2002-Pres (Chair 2008-).
University Benefits Committee, Univ. of Chicago, 2005-08
Graduate Economics Computer Lab Faculty Advisor, Univ. of Chicago Dept of Econ., 2002-08
Social Sciences Divisional Research Grant Evaluation Committee, Univ. of Chicago, 2003-04.
Faculty Advisory Committee to Social Science Computing, Univ. of Chicago, 2004-08
Committee to Review the Economics Department Chair, Univ. of Maryland, 1999
Co-Chair, Economics Graduate Student Association, Univ. of Maryland, 1997-98
Chapter President, Omicron Delta Epsilon (Economics Honorary), 1995-96
5
-------
President, Univ. of North Dakota Engineers' Student Council, 1993-94
OTHER PROFESSIONAL EXPERIENCE
Research Assistant, Assistant to Prof. John Haltiwanger; University of Maryland, U.S. Census
Bureau, andNBER, 1998-2000
Mechanical Engineer Co-op; Loral Defense Systems, Eagan, MN, and Unisys Corporation,
Roseville, MN, 1993-95
Assistant Football Coach, Red River High School; Grand Forks, ND, 1991-93
PROFESSIONAL AFFILIATIONS
Member, American Economic Association
SECURITY CLEARANCE
Special Sworn Status, U.S. Census Bureau
PUBLIC SERVICE
Elected Member of Local School Council, Keller Regional Gifted Center, Chicago Public
Schools, 2012-Present; Council Chair, 2014-Present
HONORS AND AWARDS
Distinguished Visiting Scholar, Drexel University School of Economics, 2015
Young Alumni Achievement Award, University of North Dakota Alumni Association, 2013
Charles M. Harper Faculty Fellow, Booth School of Business 2012-13
Excellence in Refereeing Award, American Economic Review, 2011-14
Brookings Dissertation Fellowship, 2000-01
University of Maryland Graduate Fellowship, 1996-98
North Dakota Society of Professional Engineers Outstanding Student Award, 1993-96
Graduated Summa Cum Laude, University of North Dakota
Alexis Diakoff Mechanical Engineering Scholarship, University of North Dakota
Bohlman Economics Scholarship, University of North Dakota
Presidential Honor Roll, University of North Dakota, 1991-96
Commencement Grand Marshal, University of North Dakota, 1993
One of 141 National Presidential Scholars named by the White House Commission on
Presidential Scholars, and White House guest of President George H. W. Bush, 1991
6
-------
APPENDIX C
PEER REVIEW PANEL CHARGE
SUBJECT: Charge questions for Peer Review of "Cost Reduction through Learning in
Manufacturing Industries and in the Manufacture of Mobile Sources"
Thank you for agreeing to review the enclosed report, "Cost Reduction through Learning in
Manufacturing Industries and in the Manufacture of Mobile Sources."
EPA undertook this study to improve our cost estimates for our mobile source rulemakings and,
specifically, to provide clarity about the effects of industrial learning. We are submitting this
document to you for a peer review of the methodology, and the validity and assumptions that go
into it.
EPA has provided direction and charge questions for this review and these are included below. A
teleconference call will also be arranged so that EPA can respond to questions from individual
reviewers on the material that was provided for review. The completed review reports are to be
furnished to RTI by April 15, 2016.
Elements to be addressed in the Charge to the Reviewers of the Report on "Cost Reduction
through Learning in Manufacturing Industries and in the Manufacture of Mobile Sources"
The study is intended to be a definitive, reliable, single source of information demonstrating the
occurrence of learning in general and in the mobile source industry specifically. It consists of a
literature review of studies of learning in mobile source industries, most notably the automotive
industry (both original equipment manufacturers and tier 1 suppliers); identifies and summarizes
empirical estimates of learning from those studies; develops a methodology to estimate the
impacts of learning in the mobile source sectors using the quantitative estimates obtained from
the literature review; and develops a best estimate for learning in the mobile source sector.
We request that your review primarily focus on: (1) clarity of the presentation, (2) the overall
approach and methodology, (3) appropriateness of the studies included and other inputs, (4) the
data analyses conducted, and (5) appropriateness of the conclusions. For this review, no
independent data analysis is required, nor is it required that you duplicate the results. In your
comments, you should distinguish between recommendations for clearly defined improvements
that can be readily made based on data reasonably available to EPA, versus improvements that
are more exploratory or dependent on data not available to EPA. The comments should be
sufficiently detailed to allow a thorough understanding by EPA or other parties familiar with the
work.
Your comments should be provided as an enclosure to a cover letter that clearly states your
name, the name and address of your organization, what material was reviewed, a summary of
your expertise and qualifications, and a statement that you have no real or perceived conflicts of
interest. Please also enclose an email with your comments in MS Word, or a format that can be
imported into MS Word. The comments should be sent in care of Jennifer Richkus to the Email:
jrichkus@rti.org.
C-l
-------
EPA will make the report and your comments available to the public, and we may submit the
report and your comments to public dockets that support future rulemakings and studies. We
would appreciate you not providing the peer review materials or your comments to anyone else
until EPA makes them public. We would also like to receive the results of this review in the
shortest time frame possible, preferably within six weeks of your receipt of this request. If you
have any questions about what is required in order to complete this review, or if you find you
need additional background material, please contact RTI contact by phone (202-974-7831) or e-
mail [jrichkus@rti.org]. If you have any questions about the EPA peer review process itself,
please direct them to Ms. Ruth Schenk of EPA by phone (734-214-4017) or e-mail
[schenk.ruth@epa.gov]
We estimated 40 hours of review time for this peer review. In your cover letter please indicate
the number of hours spent on the review; spending fewer or more hours than our estimate will
not affect the fee paid for this work, but will help us improve our future budget estimates.
C-2
-------
APPENDIX D
PEER REVIEWER QUESTIONS AND ANSWERS ON THE CHARGE
Q. Please provide formal definitions for mobile sources and the mobile source industry.
A. "Mobile sources" include cars and light trucks, heavy trucks and buses, nonroad engines,
equipment, and vehicles. More specifically:
• On-road vehicles and engines
o Cars & Light Trucks
o Heavy Trucks, Buses & Engines
o Motorcycles
• Nonroad engines, equipment and vehicles
o Aircraft
o Diesel boats and ships
o Gasoline boats & personal watercraft
o Nonroad diesel equipment (including excavators and other construction
equipment, farm tractors and other agricultural equipment, heavy forklifts,
airport ground service equipment, and utility equipment such as
generators, pumps, and compressors)
o Nonroad gasoline equipment (forklifts, generators & compressors)
o Small gasoline equipment (lawn & garden)
o Locomotives
o Snowmobiles, dirt bikes & ATVs
For purposes of the report, EPA defined "mobile source industry" as original equipment
auto makers, parts suppliers to those auto makers, loose engine manufacturers, large truck
manufacturers, and nonroad equipment manufacturers.
Q. Is the intended task to review the report in the context of its applicability to rulemaking
by the EPA?
A. The peer review is intended to review the reasonableness and the comprehensiveness
provided by the report, however EPA has asked that peer reviewers look critically at
Section 3.4 and comment on whether the recommendations are reasonable given the
information provided in the report.
D-l
-------
APPENDIX E
PEER REVIEWER REPORTS
E-l
-------
April 14, 2016
To,
Jennifer Richkus
Research Environmental Scientist
RTI International
Sub: Review of "Cost Reduction through Learning in Manufacturing Industries and in the
Manufacture of Mobile Sources"
Dear Jennifer:
Please find attached my review of the above-mentioned document. I provide the other required
details below.
Materials Reviewed: The review was based on the following materials received via email on
March 2, 2016: (i) "Cost Reduction through Learning In Manufacturing Industries and In the
Manufacture of Mobile Sources" (ii) Charge Letter. Please refer to the section "Scope of the
Review" in my review report for further details.
Summary of Related Expertise and Qualifications: I have a PhD in Management from the
Anderson School at the University of California, Los Angeles. My dissertation was on the topic
of learning-by-doing, and examined learning-by-doing and its competitive implications in the
manufacturing sector. Based on this work, I have published three peer-reviewed articles in the
following journals: Strategic Management Journal, Management Science and Journal of
Industrial Economics. I have also reviewed manuscripts on this topic for leading journals in the
field of management.
Statement Regarding Conflict of Interest: I do not have any real or perceived conflict of
interest with respect to the document I reviewed. I was not involved in writing that document
nor have I made any public statements about it.
Estimated Hours of Work: ~30 hours
Disclaimer: The opinions, comments and statements made in this review are my own and do
not reflect my employer's views.
Sincerely,
Natarajan Balasubramanian, PhD
Associate Professor of Management
Whitman School of Management, Syracuse University
721 University Ave Rm 522
Syracuse, NY, 13244
-------
Review of "Cost Reduction through Learning in Manufacturing Industries and in the
Manufacture of Mobile Sources"
Natarajan Balasubramanian, Ph.D.
April 14, 2016
-------
SCOPE OF THE REVIEW
This review was completed in response to a review request by RTI International made via email
on Feb 9, 2016. The materials for review, a report titled "Cost Reduction through Learning In
Manufacturing Industries and In the Manufacture of Mobile Sources" ('the Report') by the
United States Environmental Protection Agency, numbered EPA-420-R-16-XXX and the Charge
Letter were received on March 2, 2016 via email. The aforementioned report contains within it, a
study ('the ICF Report'), dated Sep 15, 2015, prepared by ICF International.
The scope of this review was based on the Charge Letter and additional responses to questions
asked by reviewers via email and during a conference call on March 23, 2016 with personnel
from RTI International and the United States Environmental Protection Agency ('EPA'). In line
with these instructions, this review focuses on the following aspects, specifically with respect to
the stated objectives of the Report:
1) Reasonableness and comprehensiveness
2) Clarity of the presentation including the organization
3) Suitability of the overall approach and methodology, and the data analyses conducted
4) Appropriateness of the studies included and other inputs, and the
5) Appropriateness of the conclusions and in particular, the recommendations made in
Section 3.4 of the ICF Report.
No independent data analysis was conducted. Further, no attempt was made to duplicate the
results stated in the Report. The review was based only on the material provided in the Report.
Unless stated otherwise, no external material including any original books and articles
summarized in the Report was used during the review process. Hence, this review cannot
comment on the accuracy of those original books and articles. In accordance with the
instructions, the Report was not reviewed in the context of its applicability to rulemaking by the
EPA.
STATED OBJECTIVES OF THE REPORT
The Report appears to have four stated objectives.
Objective 1: To be "a definitive, reliable, single source of information demonstrating the
occurrence of learning in general and in the mobile source industry specifically" (Charge Letter
and Section 2 of the ICF Report).
Objective 2: "[T]he goal of this work assignment is to develop a single compendium study on
industrial learning in the mobile source sector that can be relied on as the basis for this effect
(italics added) in future cost analyses."(Section 1 of the ICF Report) "This effect" is explained
further in "[w]hile there is little doubt that this learning effect occurs, the learning estimates
used by OTAQ [Office of Transportation and Air Quality] in its cost analyses are based on
1 of 9
-------
somewhat dated studies that are not specific to the mobile source sector." (Section 1 of the ICF
Report)
Objective 3: "[T]o determine the best estimate of the effect of learning on costs in mobile source
industries." (Section of the ICF Report)
Objective 4: "[T]o develop a reliable estimate of the effect of cumulative output" (Section 3.3 of
the ICF Report)
SUMMARY OF THE REPORT
The Report contains two sections: summary and background and ICF Report. There is also a
placeholder for an Appendix with peer review comments. The summary and background
summarizes the findings of the ICF Report and provides a background about the EPA's need
for the ICF Report.
The ICF Report reviews 53 studies on learning-by-doing, of which 20 are reviewed in detail.
Subsequent to a description of how the subject matter expert and the relevant studies were
chosen (Section 2), the ICF Report briefly discusses the concepts of learning curves and progress
ratios (Sections 3.1-3.2) and summarizes the literature review (Table 1).
Section 3.4 presents an estimate of the average progress ratio for the mobile source sector (84.3%
with a 95% CI: 83.9%-84.8%). This average is computed as the weighted average progress ratio
from regression estimates of unit costs on cumulative output based on 5 studies in the mobile
source sector. The inverse of the variance in the study is assigned as the weight.
It is well known that the rate of learning-by-doing varies across organizations and contexts.
Section 4 discusses some aspects of such variation based on the literature review: (a) sources of
such variation (b) depreciation of knowledge accumulated from learning-by-doing (c) the
location of organizational knowledge (d) extensions of the conventional learning curves. This
section also presents some examples of how learning curves have been applied.
Appendix A presents two methods for forecasting the change in unit costs due to learning-by-
doing and Appendix B provides a summary of the reviewed articles.
REVIEW COMMENTS AND RECOMMENDATIONS
Unless otherwise stated, the recommendations can be readily made based on data reasonably
available to the EPA.
Presentation and Organization
1. The overall presentation and organization of the Report is generally clear. However, there
are some specific areas that require greater clarity. These are described below.
2 of 9
-------
2. The Report and ICF Report appear to have multiple objectives that are stated in several
places. In addition, there is at least one aspect that is provided in the Report but not
mentioned as an objective (methods of forecasting in Appendix A). So, I recommend a short
subsection somewhere that explicitly states the objective(s) in one location. In addition, I
recommend that the document refer to these objectives consistently throughout the
document. For instance, Objectives 3 and 4 above are similar but it is not clear what the
difference between a "reliable" and a "best" estimate is. It may be more appropriate to
choose one of them, and use that consistently. Also, note that the term "best estimate" has a
generally accepted econometric definition as the estimate with the lowest variance among a
set of estimates. Hence, it may be prudent to avoid using that term or clarify its meaning as
used in this Report.
3. The two paragraphs beginning "Learning is a major source of...." (p.11-12 of the ICF
Report) do not directly relate to "what are progress ratios" and appear out of place in that
subsection. I recommend that they be moved to the next subsection on "Summary of
Literature Review". Also see point 4 below.
4. The Report and the ICF Report do not seem to provide a clear summary of the literature
review. The summary in the "Summary and Background" section of the Report focuses
almost entirely on the estimation of the average progress ratio, which is only a small part of
the review. The summary in the ICF Report (Section 3.3) is only a table with no additional
explanation. I recommend that a more descriptive summary of the literature review be
included. Among others, I suggest that the summary highlight the variation observed in the
rates of learning-by-doing (currently discussed in Section 4 of the ICF Report). Also see
point 14 below.
5. Section 1 of the ICF Report (paragraph 4) states "It will also summarize empirical estimates
of the learning effect separately for each of the specific mobile source industries (e.g.,
original equipment auto makers, parts suppliers to those auto makers, loose engine
manufacturers, large truck manufacturers, and nonroad equipment manufacturers) for
which studies are found that address those specific sectors." This break-down by industry is
not provided in the Report. The Report provides only one estimate for the entire sector.
Hence, this statement should be corrected or placed in a different context (e.g., the original
intent of the study was to summarize empirical estimates separately...). Also see point 15.
6. There is a minor typographical error on p.19 (ICF Report) at the beginning of the second
paragraph.
Review Approach, Comprehensiveness and Appropriateness of Studies Included
7. The overall approach to the review—identifying studies of learning-by-doing in the mobile
source sector, reviewing them for relevance to the goals of the study and identifying a
3 of 9
-------
shorter list of relevant studies for more detailed review—appears reasonable. The list of
topics included in the review and the coverage of those topics appear broadly reasonable.
8. The set of articles related to progress ratio estimation in the mobile source sector and
included for review appears to be reasonably comprehensive. A search for articles on
learning-by-doing in the mobile source sector on Google Scholar did not yield any new
substantively-contributory articles on this subject. A possible, but not necessary, addition is
Balasubramanian and Lieberman, 2011. The article itself is not relevant but the Online
Appendix to this article contains estimates of new-plant learning-by-doing using different
methods for several industries, at a more fine-grained level (at the SIC-4 level) than
Balasubramanian and Lieberman, 2010.1 attach the relevant portions of this article, with
learning rates translated into progress ratios, as Appendix I.
9. Based on a broader search of articles on learning-by-doing, an article (Haunschild and Rhee,
2004) may potentially add some insights in Section 4.1, but not including it will not detract
substantively from the findings of the Report. I have included the abstract in Appendix II.
Methodology and Conclusions Related to Estimation of Average Progress Ratio
10. The overall conclusion that learning-by-doing occurs in the mobile source sector is well-
founded and largely indisputable.
11. The methodology for estimating the weighted-average progress ratio from 5 studies is
broadly reasonable. In particular, the following executive decisions related to estimating the
average progress ratio appear reasonable given the objectives of the Report:
a. Focusing only on studies that examine unit costs and excluding studies that use
other measures of performance
b. Excluding studies of learning-by-doing in shipbuilding during the Second World
War due to the uniqueness of the context
12. The Report uses a "fixed-effects" model to combine estimates from different studies (the
weight is the inverse of the variance). However, it is not clear that all studies used the same
method to computing standard errors. For instance, some studies may have computed
heteroscedasticity-robust or clustered standard errors, which would typically be larger than
studies that assume homoscedasticity. If that is indeed the case, taking a simple inverse
would not be accurate, and presenting one or more alternative estimates in addition to this
"fixed effects" estimate (e.g., a simple average) may provide a more complete picture. An
additional rule can be applied if one of these estimates has to be chosen (e.g., the most
conservative).
13. The methodology for estimating the standard error of the average progress ratio is not
explicit in the Report. A sentence or two describing this should be added in Section 3.4.
14. Though the estimate of the weighted-average progress ratio is broadly reasonable, the
discussion about the uncertainty associated with learning-by-doing is quite sparse. Such a
4 of 9
-------
discussion is important for a full understanding of the weighted-average progress ratio. The
standard error of the weighted-average progress ratio is likely to be small, as currently
stated in the Report. However, that small standard error does not reflect the true variation
in the progress ratios across organizations and contexts, which is likely to be significantly
larger. Also, some important aspects of the studies need highlighting to provide readers a
better understanding of their context (which could be possibly different from today's
context or other contexts in the mobile source sector). Hence, providing a prominent
contextual discussion in the Summary and Background section of the Report and in Section
3.4 of the ICF Report covering the following aspects is recommended:
a. There is significant variation and uncertainty in the rates of learning-by-doing
depending on many factors, and that learning-by-doing is not automatic as
discussed in Section 4 of the ICF Report
b. The specific empirical context of the 5 studies, viz. the production of a new car
model, as well as the dates of these studies (where available).
These aspects are currently discussed in different places in the Report but it is important
that a summarized version of these points be located close to discussions of the weighted-
average progress ratio.
15. Section 2 of the ICF Report (p.4-5) provides two reasons for not providing a break-down of
progress ratios by industry (see Appendix III for the list of industries in the mobile source
sector). The first is the lack of studies in many of the individual industries and the second is
the greater within-industry variation in rates of learning-by-doing as compared to inter-
industry variation in those rates. Of these reasons, the first has merit. However, the second
is not a valid reason for not providing a break-down by industry. It raises the question of
why studies from outside the mobile source sector should not be used for estimating the
"best" or "reliable" progress ratio for the mobile source sector. In my opinion, since there is
significant variation across industries (albeit less than the within-industry variation) in the
average progress ratios (e.g., see Appendix I to this review or Dutton & Thomas, 1984 cited
in the Report), it is appropriate to consider using industry-specific estimates, if and when
such estimates become available. In general, it will be more informative to use the means of two
sub-groups than the mean of the group as a whole.
16. The Report aims to get a "best" or "reliable" estimate of the "effect" of learning-by-doing (or
cumulative output) on costs. The term "effect" has a causal connotation. However, it is not
clear that all five studies used econometric techniques to causally estimate the effect of
learning-by-doing. If so, it may be more appropriate to characterize the estimated weighted-
average progress ratio as the association between unit costs and cumulative output, rather
than as the effect of learning on costs. This approach is also consistent with the decision to
focus on models that include only cumulative output as a predictor instead of using a more
5 of 9
-------
complete model that includes other factors. This decision implies that the effect of other
factors is not isolated from the effect of cumulative output, when estimating the weighted-
average progress ratio.
Methodology Related to Forecasting the Impact of Learning
17. Point 16 above also relates to the discussion in Appendix A. As discussed in the Report,
cumulative output can be correlated with many other factors (e.g., economies of scale). Also,
the estimated weighted-average progress ratio in Report uses models that include only
cumulative output as a predictor. Hence, forecasting the impact of learning-by-doing alone
based on that ratio is not possible in the absence of information on the other factors.
However, this does not render the forecasting exercise provided in Appendix A
meaningless. It still measures the likely change in unit costs due to a change in cumulative
output, which could be due to learning-by-doing or due to other factors. Recognizing this
assumption implicit in these methods is important, especially when applying these
methods.
*******
6 of 9
-------
APPENDIX I: PROGRESS RATIO ESTIMATES FROM BALASUBRAMANIAN AND
LIEBERMAN, 2011
SIC and Description
Estimate (Olley-
Pakes Method)
3537 Industrial Trucks, Tractors, Trailers & Stackers
85%
3711 Motor Vehicles & Passenger Car Bodies
89%
3713 Truck & Bus Bodies
82%
3714 Motor Vehicle Parts & Accessories
81%
3715 Truck Trailers
89%
3716 Motor Homes s
80%
3721 Aircraft
86%
3724 Aircraft Engines & Engine Parts
81%
3728 Aircraft Parts & Auxiliary Equipment, NEC
75%
3731 Ship Building & Repairing
90%
3732 Boat Building & Repairing
85%
3743 Railroad Equipment
84%
3751 Motorcycles, Bicycles & Parts
74%
3792 Travel Trailers and Campers
91%
3799 Transportation Equipment, NEC
85%
Source: Balasubramanian N. and Lieberman M., 2011. Learning by Doing and Market Structure.
Journal of Industrial Economics. 59(2): 177-198, Online Appendix, Appendix III.
7 of 9
-------
APPENDIX II: POTENTIALLY USEFUL RECENT STUDIES OF LEARNING
A. The Role of Volition in Organizational Learning: The Case of Automotive Product Recalls.
Haunschild, Pamela R; Rhee, Mooweon. Management Science 50.11 (Nov 2004): 1545-1560.
What is the role of volition in organizational learning? Do firms learn better in response to
internal procedures or external mandates? Existing literature provides conflicting answers to
this question, with some theories suggesting that volition is important for learning because
autonomy increases commitment and problem analyses, whereas external mandates tend to
produce defensive reactions that are not coupled to the organization in any useful way. Yet,
other theories suggest that mandate is important for learning because external pressures act as
jolts that help overcome organizational inertia, resulting in deep exploration of problems to
prevent future surprises. We investigate this issue in the context of automakers learning from
voluntary versus involuntary product recalls. Using data on all recalls experienced by
automakers that sold passenger cars in the United States during the 1966-1999 period, we follow
the learning - curve tradition in investigating the effects of voluntary and involuntary recalls on
subsequent recall rates. We find that voluntary recalls result in more learning than mandated
recalls when learning is measured as a reduction in subsequent involuntary recalls. This effect is
at least partly because of shallower learning processes that result from involuntary recalls. The
results of this study suggest an important, yet understudied, determinant of the rate and
effectiveness of learning - volition. The results also add to our knowledge of the different
learning processes of generalist and specialist organizations.
8 of 9
-------
APPENDIX III: DEFINITION OF MOBILE SOURCE MANUFACTURING SECTOR
It includes:
On-road vehicles and engines
Cars & Light Trucks
Heavy Trucks, Buses & Engines
Motorcycles
Nonroad engines, equipment and vehicles
Aircraft
Diesel boats and ships
Gasoline boats & personal watercraft
Nonroad diesel equipment (including excavators and other construction equipment, farm
tractors and other agricultural equipment, heavy forklifts, airport ground service equipment,
and utility equipment such as generators, pumps, and compressors)
Nonroad gasoline equipment (forklifts, generators & compressors)
Small gasoline equipment (lawn & garden)
Locomotives
Snowmobiles, dirt bikes & ATVs
9 of 9
-------
UCLAAnderson
Marvin Lieberma n
Professor of Strategy
Tel (310) 206-7665 Fax (310) 206-3337
marvin.lieberman@anderson.ucla.edu
SCHOOL Of MANAGE E N T
http://www.anderson.ucla.edu/faculty/marvin.lieberman/
April 14, 2016
To Whom It May Concern:
I am a professor in the UCLA Anderson School of Management in Los Angeles,
California. I was asked to review the report, "Cost Reduction through Learning In
Manufacturing Industries and in the Manufacture of Mobile Sources," prepared for the
U.S. Environmental Protection Agency. My comments on the report are attached to this
letter.
My qualifications and expertise to review the EPA report are as follows. I have
contributed to the literature on industrial learning curves and have followed that
literature over many years. My PhD dissertation ("The Learning Curve, Pricing and
Market Structure in the Chemical Processing Industries," Harvard University, 1982)
focused on the nature and implications of learning in manufacturing industries. Two of
my journal articles on industrial learning are described in the EPA report. In addition to
my academic contributions in this area, I served as consultant to a number of
companies in the chemical, energy and electronic sectors in the 1980s and 1990s,
performing studies that used the learning curve concept to forecast manufacturing
costs. I also served as subject matter expert to the RAND Corporation in preparing a
report that surveyed the literature on learning curves in the energy sector (similar to
the present report which focuses on the mobile source sector).
I have no real or perceived conflicts of interest in reviewing the EPA report. Other than
the current review, I have not performed any consulting work relating to learning
curves in more than 15 years. I have no connection to the EPA or to companies in the
mobile source sector.
I spent a total of 27 hours to complete this task.
Sincerely yours,
Marvin Lieberman
110 Westwood Plaza, Suite B415, Gold Mall
Box 951481, Los Angeles, California 90095-1481
-------
Review of "Cost Reduction through Learning In Manufacturing Industries and
in the Manufacture of Mobile Sources," a report prepared for the Assessment
and Standards Division, Office of Transportation and Air Quality, U.S.
Environmental Protection Agency.
I have been asked to review this report for the EPA, with a focus on: 1) clarity of the
presentation, 2) the overall approach and methodology, 3) appropriateness of the
studies included and other inputs, 4) the data analyses conducted, and 5)
appropriateness of the conclusions.
The EPA report "is intended to be a definitive, reliable, single source of information
demonstrating the occurrence of learning in general and in the mobile source
industry specifically. It consists of a literature review of studies of learning in mobile
source industries, most notably the automotive industry (both original equipment
manufacturers and tier 1 suppliers); identifies and summarizes empirical estimates
of learning from those studies; develops a methodology to estimate the impacts of
learning in the mobile source sectors using the quantitative estimates obtained from
the literature review; and develops a best estimate for learning in the mobile source
sector."
Thus, the report aims to provide "a single compendium study on industrial learning
in the mobile source sector" (p. 1) which can serve as a source document for the EPA
and other organizations. It is my understanding that beyond compiling evidence on
the prevalence of the learning curve phenomenon across a range of mobile source
manufacturing environments, a key objective is to identify a representative learning
rate or "progress ratio" in the mobile source sector that could be incorporated into
future cost analyses and rulemaking by the EPA.
I find the report to be comprehensive, and I believe it does a good job of
characterizing the rates of learning typically found in transportation equipment
manufacturing plants. Dr. Linda Argote of Carnegie Mellon University, the Subject
Matter Expert for the report, is widely regarded as the world expert on industrial
learning curves, having published numerous research studies in this topic area and
a major book, Organizational Learning (now in second edition), which provides a
critical summary and guide to findings in the literature. Compared with this book or
any individual research study, the EPA report offers a more in-depth view of the
literature on industrial learning that is most relevant to the mobile source sector.
Overall, I find the report to be a well-executed document that is likely to be helpful
in providing a basis for incorporating forecasts of learning into EPA and other
government rulemaking.
Despite these strengths of the report, I believe it has a number of limitations that
should be (more clearly) acknowledged. I also see several areas where
improvements can be made in the document.
-------
In my initial comments (points #1 through #4 below), I focus on the "summary"
sections of the report, which are likely to be the most widely read material. (This
summary material appears in two places: at the beginning of the initial "Summary
and Background" section as well as pages 19 and 20 of the report.)
(1) While the report surveys a substantial amount of literature, the summary is
based upon five representative studies of manufacturing assembly plants that were
"used as the basis to estimate the progress ratio for the mobile source sector." These
five studies include one on the manufacture of commercial aircraft, three on the
manufacture of trucks, and one on passenger cars. Averaging across these studies,
the bottom line estimate from the report is that: "the recommended progress ratio is
84.3 percent, with a 95% confidence interval of 83.9 percent to 84.8 percent."
I agree that the weighted average progress ratio across the five selected studies is
84.3 percent. Moreover, based on my experience and my reading of the broader
literature on learning curves, this is not an unreasonable figure for manufacturing
cost projections and forecasting in the mobile source sector (at least for plants of the
type surveyed by the five studies - see points #2 and #3 below).
However, the claim that there is "a 95% confidence interval of 83.9 percent to 84.8
percent" is misleading, in my opinion. That statement of the confidence interval
overstates the precision of the estimate. Let me explain.
The method used in the report to compute the confidence interval is appropriate if
there were some underlying, universal rate of learning in the mobile source sector.
In this case the individual studies provide independent estimates of this "true" rate
of learning, with the specific values obtained by each study being subject to random
error. As we add more studies, the errors wash out, and the mean of the estimates
converges on the true universal value. Under these assumptions, it would be
appropriate to use the standard errors of the individual studies to determine the
precision of the mean in denoting the "true" rate of learning (as is done in the report
to establish the "confidence interval").
However, it seems very unlikely that there is a single, universal rate of learning in
the mobile source sector. Even among the five studies included in the final sample,
there is some evidence that the learning curve is steeper in aircraft and automobile
manufacturing (both of which show progress ratios of 82% in the studies) than in
truck manufacturing (which show progress ratios of 86% or 87%). Thus, the
progress ratio does seem to show variation across products in the sector. Moreover,
studies described elsewhere in the report make it clear that the rate of learning is
subject to managerial influence. Such variation across manufacturing environments
does not imply that it is inappropriate to use an average value (e.g., a progress ratio
of 84.3%) for forecasting purposes. But it does mean that we should not regard the
84.3% progress ratio as some kind of precise and universal standard in the sector.
-------
Rather than taking the (weighted) average value of 84.3% across the five studies, if
one chose to be more conservative, a reasonable choice would be to use the smallest
rate of learning in the sample, i.e., the progress ratio of 87%. In any case, the
estimates from these five studies all lie in a fairly close range. Depending on the
purpose at hand, one could justify using 84.3%, or 87%, in my opinion.
(2) All five of the plants that are studied in this sample are engaged in final assembly
of transportation equipment (trucks, automobiles and airplanes). Thus, the progress
ratio estimates are indicative of plants of this type, i.e., assembly plants for relatively
complex mechanical products made on a production line. The estimates may not be
suitable for plants producing other types of products or plants using other types of
processes. For example, the article by Nykist and Nilsson (2015) cited on page 41,
which surveyed dozens of studies on learning in the production of Li-ion battery
packs, found a learning rate of only 9% for the overall industry and 6% for the
leading manufacturers. (Presumably, these figures correspond to progress ratios of
91% and 94%.) This is a much lower learning rate than the 84.3% progress ratio
observed on average across the five selected studies.
(3) A further deficiency in the presentation of this material (in the summary and
recommendations section as well as the broader report, e.g., Section 3.3., bullet #5)
is the failure to point out that the progress ratio estimates in the five selected
studies are not based upon the total cost of production. As noted above, all five
studies in the final sample focus on assembly plants for transportation equipment.
None of the studies utilizes data on the total costs per unit of output in these plants.
Rather, four of the studies focus on labor costs and labor productivity in the
assembly plant (vehicles produced per labor hour, or labor hours per aircraft), and
one study focuses on defect rates.
An 84.3% progress ratio based on labor cost reflects a 15.7% savings in labor cost
per unit for each doubling of cumulative output. It does not imply a 15.7% savings in
total cost per unit for each doubling. Consider a truck assembly plant where 80% of
the final cost of a truck is the cost of purchased components. In this case, the
estimated 84.3% progress ratio applies only to the value added at the plant, i.e., the
20% of total cost above that of the component parts. (Over time, these proportions
will change slightly as the amount of assembly labor input declines.) If the
component parts used in the truck are conventional parts that have long been made
in high-volume and require little or no redesign for the vehicle being produced, one
would see little cost reduction for the parts over time due to learning. In this case, a
learning curve applied to data on total costs per truck would show a much smaller
rate of cost reduction than the 84.3% progress ratio, which applies only to the final
assembly labor. (Note that if total employment in the plant does not change as
output increases over time, this progress ratio also applies to the per-unit cost of
property, plant and equipment at the assembly plant).
Thus, any forecast of reduction in total unit cost depends on (1) the progress ratio
multiplied by the growth in cumulative output (number of "doublings") in the
-------
assembly plant, as well as (2) the progress ratio and change in cumulative volume
applicable to the production of the component parts. In my opinion, the report
should be clear about this need to consider cost reduction of the component parts as
well as the learning curve in the final assembly plant. If a new vehicle model is
produced with new component parts, the rates of cost reduction for parts
production and final assembly are likely to largely coincide (so that a single
progress ratio can be used), but this need not be the case.
Similarly, the report is unclear (and, I think, misleading) in describing the nature of
the cost analysis in the five representative studies. The first paragraph of the
"Results and Recommendations" section on page 19 states: "Because the focus of our
analysis is on manufacturing costs, we included studies that used unit costs or
variables closely related to costs, such as the number of units produced or defects
per unit, as the dependent variable." As I have indicated above, the "unit costs"
analyzed in the five studies are essentially labor costs, or unit costs of final
assembly, perse. The studies do not tell us the extent to which the total cost per unit,
including the cost of the component parts, followed a similar progress ratio.
(4) Given that the final recommendations in the report are based almost exclusively
upon the five selected studies, it is useful for a reader to be able to review a detailed
summary of these studies. Four of the studies are summarized in Appendix B.
However, the (truck plant) study by Argote, Epple, Rao and Murphy (1997), does not
seem to be included in Appendix B. I recommend that a summary of this study be
added the appendix.
Moreover, it might be helpful to add some additional information to Table 2, which
very briefly summarizes the five selected studies. This information might include
the dependent variable. (Alternatively, the report could point out in the text that the
Levitt et al. (2013) study is different from the others in that it uses defect rates as
the dependent variable. While this can be determined from Table 1 and the
discussion of Levitt et al. (2013) later in the report, it is awkward for a reader to
have to search and scan between these various sections.) Table 2 might also
indicate the pages in the appendix where the summary of each study can be found.
Points 5 through 12 below apply to the literature review, which covers six topic
areas in subsections 4.1 through 4.6 of the report. In general, I find the literature
review to be comprehensive and informative. I comment on each subsection
separately.
Section 4.1. Sources of learning variation
(5) One issue raised in this section is the distinction between the learning curve and
economies of scale. Various studies distinguish between the two concepts and
provide estimates on the magnitude of each effect. (At the bottom of page 22 this
-------
distinction is called "a new development and learning curve theory." However,
studies have made the separation between learning and scale economies for
decades now, so I would not call it a "new development".)
I agree that when possible, the two effects should be estimated separately. However,
the typical progress ratios described in the report incorporate the impact of
economies of scale within the overall learning effect. Thus, the report provides no
guidance on how to perform a cost analysis forecast that incorporates learning and
economies of scale as separate elements. Perhaps the text should be more explicit
about this, although the last paragraph of section 3.3 ("Column 6 - type of outcome
variable") makes it clear that the report is focused on using only cumulative output
as a predictor.
When controls for economies of scale are omitted from the analysis, the estimated
progress ratio includes the effects of both learning and scale economies. This has
been shown in a number of studies (including my 1984 article on chemical
products). Adding a separate parameter for economies of scale normally improves
the statistical fit, but the improvement is seldom dramatic, and most studies have
found scale economies to be less important than the learning effect. Moreover, if the
data sample is small, colinearity between the learning and scale parameters can
reduce the accuracy with which each is estimated. One implication is that if the
analyst or policy maker is able to apply only a single cost driver for forecasting
purposes, application of a learning curve or progress ratio to forecasted cumulative
output may provide the best projection of future costs.
(6) I am puzzled that the findings in my study with Balasubramanian (2010) are
heavily discounted because the learning rate "was estimated using revenues less
materials costs (i.e., value added) as the outcome variable, rather than unit cost." As
indicated in point #3 above, none of the five studies selected as representative of the
mobile source sector actually utilize data on unit cost. Rather, four of the studies use
data that correspond to value added in final assembly, omitting materials costs.
Thus, the dependent variable in Balasubramanian and Lieberman (2010) is not so
different from that of the selected studies. (However, Balasubramanian and
Lieberman estimate a learning rate over the life the manufacturing plant, rather
than over the life a new product within the plant.)
Balasubramanian and Lieberman (2010) develop estimates of learning rates by SIC
code, including many industries in the mobile source sector. The findings show that
learning rates differ significantly across industries and sectors. The estimated
progress ratios identified by Balasubramanian and Lieberman (2010) for industries
in the mobile source sector, by three-digit and four-digit SIC code, are as follows:
-------
Estimated Progress
Ratios:
OP
ACF
371 MOTOR VEHICLES AND MOTOR VEHICLE EQUIPMENT
80%
79%
372 AIRCRAFT AND PARTS
81%
87%
373 SHIP AND BOAT BUILDING AND REPAIRING
89%
93%
374 RAILROAD EQUIPMENT
79%
89%
AVERAGE:
82%
87%
3711 MOTOR VEHICLES AND PASSENGER CAR BODIES
89%
86%
3713 TRUCK AND BUS BODIES
82%
90%
3714 MOTOR VEHICLE PARTS AND ACCESSORIES
81%
78%
3715 TRUCK TRAILERS
89%
90%
3716 MOTOR HOMES
79%
76%
3721 AIRCRAFT
86%
93%
3724 AIRCRAFT ENGINES AND ENGINE PARTS
81%
89%
3728 AIRCRAFT PARTS AND AUXILIARY EQUIPMENT, N.E.C.
75%
83%
3731 SHIP BUILDING AND REPAIRING
90%
98%
3732 BOAT BUILDING AND REPAIRING
85%
89%
3743 RAILROAD EQUIPMENT
83%
87%
AVERAGE: 84% 87%
Estimates in the two columns above are based on two different procedures (OP and
ACF) for correcting potential endogeneity in the data.
I show these estimated progress ratios, not to have them included in the EPA report,
but rather to indicate that the average learning rates for industries in the mobile
source sector, as estimated by Balasubramanian and Lieberman (2010), are
substantially in line with those in the summary section of the EPA report.
[In the attached Appendix, I give more detail on these progress ratios and their
derivation from the industry-specific learning rate estimates reported by
Balasubramanian and Lieberman (2010).]
Section 4.2. Knowledge persistence and depreciation
(7) This section does a good job of characterizing studies of the learning effect that
have considered knowledge depreciation. The differences in the estimated
depreciation parameter across the various studies are striking. Perhaps the best
-------
explanation for these large differences is provided by the summary of Agrawal and
Muthulingam (2015), which appears in section 4.4 (as well as partly in section 4.2):
"the rate of knowledge depreciation depends on where knowledge is located...."
One confusing element in this section is that some of the depreciation rates are
monthly and others are annual. On pages 27 and 28, for example, the text might
clarify that Benkard and Argote's estimates are monthly rates of depreciation
(although the figures are converted to an annual basis in table 3).
Section 4.3. Knowledge transfer and spillovers
(8) This section is effective in describing research findings relating to knowledge
transfer across organizational units (additional shifts, new models, etc.) within a
given firm. However, the section ignores the existing literature on knowledge
transfer and spillovers across firms (except for very brief mention in footnote 5).
This literature on inter-firm spillover of learning is fairly extensive, although the
evidence is based mostly on studies using data outside the mobile source sector.
Section 4.4. Location of organizational knowledge
(9) This section is informative and well done. Indeed, I think it would be helpful to
provide some of this material earlier in the report - specifically, to make it clear that
learning and knowledge can be embedded in people, in organizational routines, or
in technology/physical capital. The fact that accumulated knowledge can be
embedded in these three ways - which are each quite different - is fundamental to
gaining an understanding of the differences found across studies with respect to
knowledge depreciation, knowledge transfer, and (potentially) industry-specific
rates of learning, which are discussed in the previous sections of the report.
Although this decomposition of the location of organizational knowledge has only
recently been fully documented in studies of learning, it has been generally
recognized for some time.
Section 4.5. The specification and aggregation of learning
(10) This seems to be a residual section in the report; many key issues relating to
the specification and aggregation of learning have already been discussed in
previous sections. Thus, Section 4.5 does not truly serve a standalone function;
rather, it seems to be a placeholder to summarize three studies that were otherwise
hard to classify. Perhaps the section should take a broader perspective, summing up
many of the conclusions of the previous sections that relate to the specification and
aggregation of learning.
(11) One specification issue that is left hanging in the report is whether the learning
curve should be estimated with an initially "steep" portion followed by a "flat"
-------
portion (once the data have been transformed into logarithms). This specification
issue is raised on the last page of the Summary and Background section; however,
there is no specific follow-up in the report. (Virtually all of the presentation in the
report is consistent with a single learning curve that does not change slope over
time.) This issue of whether the slope of the learning curve is constant or
diminishing should be discussed, and ideally, resolved in the report.
Section 4.6. Application of the learning curve
(12) The studies summarized in the section are quite diverse. Nevertheless, it seems
appropriate to have a concluding section to consider these studies.
As noted in point #2 above, it is striking that Nykist and Nilsson's (2015) survey
found learning rates for production of automotive Li-ion battery packs to be
substantially smaller than the 84.3% progress ratio that the EPA report proposes for
cost forecasting in the mobile source sector. It would be informative to consider
possible sources of this large discrepancy in learning rates between Li-ion battery
manufacturing and transportation equipment final assembly.
Typographic errors and other minor corrections
In the title of Summary Table 1, "Progress Rations" should be Progress Ratios".
Summary and Background, page 3. In the middle paragraph, "for each doubling of
production volume" should be "for each doubling of cumulative production volume".
In the sentence that follows, "it was assumed that production volumes would have
doubled" should be "it was assumed that cumulative production volumes would
have doubled".
Page 19. "In error! Reference source not found" is a typographical error.
Respectfully submitted,
Marvin Lieberman
Los Angeles, California
April 14, 2016
-------
APPENDIX
Estimated Progress Ratios for Mobile Source industries by SIC Code, Based ori Balasubramanian and Lieberman (2010)
Ol<
OP
ACF
Estimated Progress Ratios:
Coeff,
Sid Error
Coeff.
Std Error
Cwff.
Std Error
OlS
OP
ACF
371 MOTOR VEHICLES AND MOTOR VEHICLE EQUIPMENT
371
0.191
tnj
0.323
(0.05)
0,332
(0.04)
88%
80%
79%
372 AIRCRAFT AND PARTS
372
0.157
(0.02)
0.298
(0.05)
0.194
(0.09)
90%
81%
87%
373 SHIP AND BOAT BUILDING AND REPAIRING
373
0.108
(0.02)
0.167
(0.04)
0.104
(0.04)
93%
89%
93%
374 RAILROAD EQUIPMENT
374
0.144
(0.05)
0.335
(0.14)
0.172
(0,10)
91%
79%
89%
AVERAGE: 90% 82% 87%
3711 MOTOR VEHICLES AND PASSENGER CAR BODIES
3711
0.127
(0.03)
0.174
(0.07)
0.215
(0.07)
92%
89%
86%
3713 TRUCK AND BUS BODIES
3713
0.192
(0.02)
0.289
(0.05)
0,152
(0.07)
88%
82%
90%
3714 MOTOR VEHICLE PARTS AND ACCESSORIES
3714
0.177
(0.01)
0.301
(0.03)
0.351
(0.03)
88%
81%
78%
3715 TRUCK TRAILERS
3715
0.098
(0.03)
0.169
(0.05)
0.149
(0.06)
93%
89%
90%
3716 MOTOR HOMES
3716
0.166
(0.04)
0.333
(0.09)
0.395
(0.16)
89%
79%
76%
3721 AIRCRAFT
3721
0.055
(0.05)
0.214
(0.12)
0.098
(0.08)
96%
86%
93%
3724 AIRCRAFT ENGINES AND ENGINE PARTS
3724
0.159
(0.03)
0.305
(0.09)
0.170
(0.11)
90%
81%
89%
3728 AIRCRAFT PARTS AND AUXILIARY EQUIPMENT, N.E.C.
3728
0.188
(0.02)
0.424
(0.06)
0.269
(0.08)
88%
75%
83%
3731 SHIP BUILDING AND REPAIRING
3731
o.as4
(0.02)
0.157
(0.05)
0.023
(0.04)
96%
90%
98%
3732 BOAT BUILDING AND REPAIRING
3732
0.134
{0.02)
0.234
(0.04)
0.166
(0.04)
91%
85%
89%
3743 RAILROAD EQUIPMENT
3743
0.138
(0.04)
0.264
(0.09)
0.205
(0.08)
91%
83%
87%
AVERAGE: 91% 84% 87%
-------
CHICAGO BOOTH H
The University of Chicago Booth School of Business
Chad Syverson
J. Baum Harris Professor of Economics
5807 South Woodlawn Avenue
Chicagp, Illinois 60637
Telephone: (773) 702-7815
chad.syverson@chicagobooth.edu
April 7, 2016
Jennifer Richkus, Research Environmental Scientist
RTI International
Dear Ms. Richkus:
Please find enclosed my peer review of the report "Cost Reduction through Learning in
Manufacturing Industries and in the Manufacture of Mobile Sources."
By way of background, I am the J. Baum Harris Professor of Economics at the University of
Chicago Booth School of Business. I obtained a PhD in Economics from the University of
Maryland in 2001 and have been on the faculty at the University of Chicago since that time.
Regarding my specific qualifications and expertise as a peer reviewer for this report, I have
conducted extensive research and published multiple peer-reviewed articles on the
productivity of plants, companies, and industries within the manufacturing sector. I also
coauthored a peer-reviewed study investigating the qualitative and quantitative nature of
learning by doing in the automobile industry. Indeed, this study is reviewed in the report
and one of the five used to derive the "bottom line" learning rate discussed therein.
I have read the report thoroughly and offer in the enclosed review my opinion of how well
it achieves its intended goal of being, as stated in my charge letter, "a definitive, reliable,
single source of information demonstrating the occurrence of learning in general and in the
mobile source industry specifically." My impressions were primarily formed based on the
following aspects of the report: 1) clarity of the presentation, 2] the overall approach and
methodology, 3) appropriateness of the studies included and other inputs, 4) the data
analyses conducted, and 5) appropriateness of the conclusions.
As was also requested in the charge letter, I am informing you that I spent an estimated 20-
25 hours of total work time reading the report, examining various supplementary
materials, and preparing this review.
-------
CHICAGO BOOTH H
The University of Chicago Booth School of Business
Sincerely,
T
Chad Syverson
Enclosure
-------
Review of "Cost Reduction through Learning in Manufacturing Industries and in the
Manufacture of Mobile Sources"
Purpose and Charge
I have been asked to provide peer review of the report "Cost Reduction through Learning in
Manufacturing Industries and in the Manufacture of Mobile Sources," which has been
prepared by ICF International for the U.S. Environmental Protection Agency. My charge is
to determine how well the report achieves its intended goal of being "a definitive, reliable,
single source of information demonstrating the occurrence of learning in general and in the
mobile source industry specifically." I conducted this evaluation with five general criteria in
mind: 1) clarity of the presentation, 2) the overall approach and methodology, 3)
appropriateness of the studies included and other inputs, 4) the data analyses conducted,
and 5) appropriateness of the conclusions.
Overall Assessment
On balance, the study is a very fine review of the literature on learning by doing in general,
but especially with regard to its manifestation in manufacturing operations during the past
few decades. The report is notably comprehensive within this scope, makes sensible topical
categorizations in its discussion of the literature's findings, and is clearly written.
The report does an excellent job of sorting through the large research literature to focus on
studies that are most germane to its mission. First, its classification of works into parts that
receive more cursory reviews as opposed to more comprehensive ones is very sensible;
there is no paper in the former category that I think clearly belongs in the latter. Second,
the five particular studies from which the report extracts its "meta-estimate" of the
learning rate for mobile source manufacturing also appear to be well chosen based on the
report's quantitative objective. The meta-estimate—a progress ratio of 84% (each doubling
of cumulative experience reduces productivity, e.g. unit costs, by 16%)—strikes me as
quite plausible, though I have some questions about the associated reported precision that
I detail below.
I expect that this study will serve EPA's needs well, as best as I understand those needs.
The report is also comprehensive and detailed enough to be able to serve as an academic
resource. It will function nicely both as a map of the literature for researchers seeking to
learn more and as a teaching guide for related coursework.
1
-------
In sum, it is my opinion that the report does achieve the intended goal of being a definitive,
reliable, single source of information demonstrating the occurrence of learning in general
and in the mobile source industry specifically.
Specific Comments, Questions, and Suggestions
• The report extensively discusses "forgetting," the depreciation of the experience
stock, at least in terms of the contribution of that experience to productivity gains.
This discussion is appropriate, as several empirical studies have found evidence of
simultaneous learning and forgetting. Knowledge depreciation appears to be part of
reality in many production settings.
The review of the empirical estimates of forgetting rates is conducted in isolation
from the review of learning rates estimates. To the extent that one objective of the
study is to identify the expected pace at which mobile source manufacturing
productivity should improve with production experience, though, it seems to me
that what matters in the end is the net effect of learning and depreciation rather
than the gross learning rate.
I recognize the gross-versus-net distinction might not be easy to quantitatively
reconcile. The particular ways depreciation is parameterized in the literature does
vary across papers, clouding the mapping from the estimated depreciation rates to
the expected value in mobile source manufacturing. It is definitely more complicated
than comparing the standard log-cost, log-experience bivariate gross learning rate
regressions as the report does now. (Though of course even in that case things are
not perfectly comparable across settings.)
Therefore it might not be possible to derive a bottom-line net learning rate
parameter that is as comparable and applicable as the gross parameter the study
reports now. However, it does seem prudent to at least discuss the net-versus-gross
distinction and how it might matter when applying the findings of the report to
practical settings.
I realize that the study argues that mobile source manufacturing has several
properties (production typically is conducted at an even rate, learning is often
embedded in technology and routines, and the sector experiences relatively modest
worker turnover) that make it likely that depreciation would tend to be on the low
end of estimates in the literature. This does not seem unreasonable. However,
2
-------
arguing that these effects are likely to be smaller than usual does not necessarily
imply depreciation is likely to be zero. Again, there might not be any easy practical
alternative here in terms of quantitative reports, but it is worth discussing the issue.
• The only recent paper on learning by doing in manufacturing that I know about but
did not see discussed in this study is Hendel and Spiegel (American Economic
Journal: Applied Economics, Jan. 2014).
Unlike many other papers in the literature, it breaks down an overall cost-reduction
trend into components explained by investment and an incentive plan while also
identifying residual gains likely achieved through traditional learning channels.
Perhaps this paper is applicable to Section 4.5 due to its attempt to distinguish
among types of learning (or more accurately, distinguish other time-trend
productivity drivers from learning).
That said, the paper's setting is not in mobile source manufacturing, and I think it is
still a judgment call whether the paper warrants any more attention than a cursory
review for the purposes of this study. In any case, I thought I would mention it.
• There are several points in the report where contrasts are made between measures
of the outcome variable in learning by doing estimation. The report rightly points
out (e.g., page 13, though see my comment on demand further below) that using
price or any metric that embodies price is likely to confound supply-side learning
effects with demand-side changes that could be unrelated to the learning process.
This concern applies to value added, for example. However, it applies equally to
shipments as an outcome variable. The report holds out shipments as problematic
because they include any inventory accumulation or de-accumulation, and that is
true, but shipments are also reported in real dollar values, raising the supply-
versus-demand conundrum. This fact was not always made clear in the text. For
example, when shipments are mentioned on page 13, only the inventory issue is
raised, and moreover the output measure of Bahk and Gort (1993) is described as
"the number of shipments." Perhaps I am just interpreting the wording differently
than the sense in which it was meant, but this sounds like a quantity of units of a
good rather than a dollar value.
3
-------
• I have a couple of comments about the standard error of the "meta-estimate"
calculated in the report. First, it would be helpful if the report offered a brief
explanation of how this standard error is calculated from the literature's values
cited in Table 2. While the point estimate of -0.245 is described as an inverse-
variance-weighted average of the five point estimates, the standard error is left
unexplained. If the calculation is complex, it need not be spelled out line-for-line; a
short description of the calculation's intuition would be enough. (I made my own
quick guess at a calculation: I assumed the only variation in the standard errors
across the five papers was due to sample sizes and then calculated the total effective
sample and hence the implied standard error across the five papers. That came out
to the same 0.0039 reported in the study. Maybe I was lucky. In any case, some sort
of guidance for the reader would be useful.)
Second, and more substantively, is the possibility that the standard errors across the
five studies in Table 2 vary for reasons besides just sample size differences. There
are, after all, some basic differences across the studies: industry, outcome measure,
etc. In some ways—and the report notes this—the fact that despite these differences
their estimates are all markedly similar might suggest inferring that any
heterogeneity across the studies is more or less orthogonal to the learning rate. On
the other hand, it is not practically possible to statistically reject heterogeneous
parameters with respect to covariates like industry, outcome measures, etc., with
only five observations. As with the gross-versus-net distinction discussed above, I
do not know if there is any straightforward way to quantitatively address this issue,
but again it strikes me as something worth discussing a bit more in the report.
• I struggled to understand how the work of Laitner and Sanstad (2004) fit into the
discussion. I realize that there might be learning about products among consumers,
but it wasn't exactly clear to me from the description of their paper how this would
influence supply-side learning. My best guess of the story is that demand-side
learning affects the equilibrium quantity of a product, and that can change how
quickly experience is accumulated on the supply side. If that is correct, though, then
it is less clear to me that one would necessarily want to purge demand-side
influences from learning estimation, as asserted in the price-as-an-outcome issue
discussed above. Is there a fundamental difference between that point and the
Laitner and Sanstad (2004) analysis?
4
-------
• I completely agree with the study's interpretation of the literature that
heterogeneity in learning rates could well be large across organizations, even within
an industry, than across industries. This is a very useful point to make.
Typos and Minor Edits
• There is a missing closed parenthesis in the first sentence of EPA summary.
• There is what looks to be a LaTex citation error on page 19. From the context it
appears to be a reference to Table 2.
• On page 49 in the appendix, the "review of the literature" progress ratio is cited as
83%, but the estimate given in the main body of the review is 84%.
• The Levitt, List, and Syverson study is cited as being published in both 2012 and
2013 in different locations. Also, on page 38, Levitt, List, and Syverson is described
as studying the repair rate as an outcome variable rather than the defect rate.
5
------- |