Peer Review of EPA's Response Surface
Equation Report
Final Report
United States
Environmental Protection
tl	Agency

-------
Peer Review of EPA's Response Surface
Equation Report
Final Report
Assessment and Standards Division
Office of Transportation and Air Quality
U.S. Environmental Protection Agency
Prepared for EPA by
RTI International
EPA Contract No. EP-C-16-021
NOTICE
This technical report does not necessarily represent final EPA decisions or
positions. It is intended to present technical analysis of issues using data
that are currently available. The purpose in the release of such reports is to
facilitate the exchange of technical information and to inform the public of
technical developments.
United States
Environmental Protection
tl	Agency
EPA-420-R-18-006
May 2018

-------
Table of Contents
1	Introduction	1
2	Selection of Peer Reviewers	2
3	Peer-Review Process	2
4	Review Comments Grouped by Charge Letter Topic	3
Appendix A. Peer Reviewers' Resumes	11
Appendix B. Charge Letter	84
Appendix C. Sanya Carley Comments	88
Appendix D. Sujit Das Comments	98
Appendix E. Doug Montgomery Comments	104
Appendix F. Response Surface Report	109
List of Tables
2-1. Selected Peer Reviewers	2
List of Figures
4-1. Histogram of Residuals	3
4-2. Plot of Actual Versus Predicted Response	4
4-3. Summary of Fit and Analysis of Variance for the RSM Model	5
4-4. RSM Model Parameters Estimates	5
4-5. PRESS Statistic	6
RTI International
Peer Review of EPA's RSE Report

-------
1 Introduction
The U.S. Environmental Protection Agency's (EPA's) Office of Transportation and Air Quality has
developed a statistical approach to access results from the Advanced Light-Duty Powertrain and Hybrid
Analysis (ALPHA) model. To demonstrate the credibility of the methodology and gain acceptance in the
light-duty automotive community, EPA contracted with RTI International to support an independent
peer review.
The ALPHA model is a full vehicle simulation model that is used to assess the effectiveness of different
technology packages in vehicles. Effectiveness values from ALPHA act as robust inputs to the
Optimization Model for Reducing Emissions of Greenhouse Gases from Automobiles (OMEGA) and to
the overall rulemaking process.
Because operating the ALPHA model in real time to conduct full vehicle simulations is cost and time
prohibitive, EPA developed a method of deriving the necessary effectiveness values using an industry
standard statistical methodology known as a Response Surface Model (RSM). An RSM is used to
computationally synthesize a large set of simulation outputs to derive response surface equations
(RSEs). The derived RSEs can then be used in place of running the ALPHA model in real time for
determining the effectiveness of vehicle technologies.
The peer review was conducted in a manner that is consistent with the guidance in EPA's Peer Review
Handbook (4th edition).
This report is organized as follows:
¦	Section 2 details the selection of peer reviewers.
¦	Section 3 describes the peer-review process.
¦	Section 4 groups review comments by charge letter topic.
¦	Appendix A provides peer reviewers' resumes.
¦	Appendix B provides a copy of the charge letter sent to reviewers.
¦	Appendices C, D, and E provide exact copies of the reviews submitted by the peer-review panel.
RTI International
Peer Review of EPA's RSE Report
1

-------
2 Selection of Peer Reviewers
RTI compiled a list of 10 reviewer candidates who had the necessary expertise to make a contribution to
this review. RTI contacted each candidate to inquire about their interest, availability, and any potential
conflicts of interest with the topic.
Table 2-1 lists the final panel of reviewers. Based on availability and the need to comprehensively cover
the topic, RTI selected three peer reviewers. EPA approved all three chosen reviewers. Appendix A
contains resumes for each reviewer.
Table 2-1. Selected Peer Reviewers
Reviewer
Affiliation
Expertise?
Conflict of
Interest?
Sanya Carley
Indiana University-Bloomington
School of Public and Environmental Affairs
Yes
No
Sujit Das
Oak Ridge National Laboratory
Yes
No
Doug Montgomery
Arizona State University
Yes
No
3 Peer-Review Process
Upon completing the peer-reviewer selection process, RTI distributed a charge letter (Appendix B) and
review documentation to each reviewer. The charge letter contained instructions for each peer reviewer
with respect to the review schedule and the general topics to be addressed in their review.
Documentation provided by EPA was sufficient for the reviewers to reproduce the RSEs from the EPA
report and test the robustness of the results.
Reviewers were given 3 weeks to write their review report. RTI coordinated a kick-off conference call
and a mid-review conference call to ensure that reviewers had every resource they required to conduct
a full and comprehensive review of the report. During the review period, reviewers had regular access to
both RTI and EPA to ask questions about the RSE report or the peer-review process. All correspondence
between a reviewer and EPA was shared with all the review panel members to ensure that everyone
had the same information for their review.
At the end of the 3-week period, each reviewer submitted a written report to RTI, and these reports are
reproduced in Appendices C, D, and E. RTI adhered to the provisions of EPA's Peer Review Handbook
guidelines to ensure that the peer-review process followed EPA policy.
RTI International
Peer Review of EPA's RSE Report
2

-------
4 Review Comments Grouped by Charge Letter Topic
The following section compiles the feedback from peer reviewers by charge letter topic. With
the exception of grouping by topic, the comments have not been altered or paraphrased in any
way.
TOPIC 1: EPA's overall approach to applying response surface modelling to accessing ALPHA model
results and whether the resulting response surface equations provide accurate and robust inputs for
the OMEGA model.
Sanya Carlev
1.	There are a variety of performance metrics that one could use to assess response surface equation
accuracy and adequacy. For this review, I evaluated the size of the residuals, the percent error, and
the distribution of the residuals.
2.	These statistics confirm that the predicted values have excellent accuracy. The average residual is
0.0013 and the average percent error is -0.0004 percent. All combinations of vehicle type and
powertrain perform similarly. The combination that has the highest residual is the High
Power/Weight 2014 Atkinson.
3.	I also plotted the residuals to see if they fit a normal distribution, as suggested by Bezerra et al.
(2008). Figure 4-1 presents a histogram of all residuals across the 8,257 model runs. The distribution
appears normal. I also looked at the histograms for all vehicle types, powertrain technologies, and
vehicle type-powertrain combinations separately (not shown here). These plots provide no cause for
concern.
Figure 4-1. Histogram of Residuals
CM -
LO
"55
C ,	

-------
Suiit Das
1. A comparison of C02 results between RSE and ALPHA has confirmed the validity of the data transfer
between these two models thereby proving the accuracy of the technical application of response
surface modeling. A total of 21 results (only 2020_TURB24 was available for LPW_LRL vehicle) out of
total 24 vehicle types were examined for the RSM validation. Residuals were found to be between a
narrow range of -1.0 and 1.0 gC02/mile in all cases. The line slope of the plot of results of ALPHA and
RSE was also found to be 45° and thus has ensured the validity of data transfer between them. In
addition, as the physics behind the Mass, Aero, and Roll are quite linear in reality, and so C02
emission impacts of any values between the range of these parameters were also found to be
reasonable using the RSE results.
Doug Montgomery
I selected a subset of the 24 models for further investigation. I loaded the experimental designs for these
models into JMP PRO V 13 and performed my own RSM analysis, fitting the standard second-order
model. The results for one of these RSM model from spreadsheet HPW 1026 2017a tab 2014 GDI are
discussed below. This is typical of the results I obtained for all models that I investigated.
Figure 4-2. Plot of Actual Versus Predicted Response
d Actual by Predicted Plot

360

340

320
6

<

-j
o
300
u

<

X
280
Cl
<


260

240
240 260 280 300 320 340 360
ALPHA C02 Predicted RMSE=0.2477 RSq=1.00
PValue<.0001
The points in this plot lie almost exactly along a straight line, indicating excellent agreement
between the simulation model output and the predicted value from the second-order RSM
model
RTI International	Peer Review of EPA's RSE Report

-------
Figure 4-3. Summary of Fit and Analysis of Variance for the RSM Model
A Summary of Fit
RSquare	0.999925
RSquare Adj	0.999921
Root Mean Square Error	0.247653
Mean of Response	285.8269
Observations (or Sum Wgts)	351
Analysis of Variance


Sum of


Source
DF
Squares
Mean Square
F Ratio
Model
14
27340737
19529.1
318414.8
Error
336
20.61
0.061332
Prob > F
C Total
350
273427.97

<.0001*
The R2 statistic for the model exceeds 0.99, indicating that most of the variability in the sample
data (in excess of 99%) is explained by the RSM model. Also, the Readjusted statistic is also in
excess or 0.99. Readjusted is a reflection of potential overfitting; that is including terms not
really important in the model just to inflate the ordinary R2. When these two statistics are in
close agreement as they are here there is likely to be no substantial issue with overfitting. The
analysis of variance indicates that the model contains at least one statistically significant term.
Figure 4-4. RSM Model Parameters Estimates
Parameter Estimates
Term
Estimate
Std Error
t Ratio
Prob>|t|
Intercept
382.68894
0.075154
5092.1
<.0001*
Mass
-201.7124
0.190516
-1059
<.0001*
Aero
-52.06903
0,190476
-2734
<.0001*
Roll
-53.30226
0.189323
-281.5
<.0001*
Trans
-18,23885
0.012694
-1437
<.0001*
(Mass-0.100 8 5)*(Mass-0.10085)
8,0 1 52892
3.191378
2.51
0.0125*
(Mass-0.10085)*(Aero-0,09715)
-10,63205
2.735951
-3,89
0.0001*
(Aero- 0.09715)*( Aero-0.09715)
-5.158863
3.177543
-1.62
0,1054
(Mass-0. 10085)*[Roll-0.09886)
50.490674
2.751456
18.35
<.0001*
(Aero- 0.0 9 715)*( Rol 1 - 0.09886)
-6.814333
2.723004
-2.50
0.0128*
(Rol 1 - 0.0 9 8 S 6) *(Rol 1 - 0.09886)
-3,497919
3.169082
-1.10
0.2705
(Mass-0,1008 5)*(T rans-3.68091)
23,883683
0.149944
159,28
<.0001*
(Aero-0.0 9715)*(Tra n s- 3.68091)
0,4606774
0.150692
3,06
0.0024*
(Rol 1 ¦- 0.0 9 8 8 6)*(Tra n s-3.68091)
0,626202
0,149753
4,18
<.0001*
(Tra n s-3.6 8 0 9 l)*(Tra n s-3.68091)
0,5810956
0.014437
40,25
<.0001*
The second -order model contains 15 parameters; an intercept, four main effects, six 2-factor
interactions, and four quadratic terms. The parameter estimates display indicates that all but
two of these terms are statistically significant at the 0.05 level. However, in RSM we usually
think that it's the order of the model that is most important so we often do not remove non-
significant terms from the model unless there are many of them. That is not the case here.
1. The PRESS Statistic
In model validation it is important that the model both fit the sample and that it provide good
predictions of new data. The PRESS (Prediction Error Sum of Squares) statistic, reported below, is a
standard one-sample-at-a-time cross-validation used to assess potential prediction performance.
RTI International	Peer Review of EPA's RSE Report	5

-------
Figure 4-5. PRESS Statistic
^ Press
Press Press RMSE
22.448639323 0.2528957
Notice that the PRESS statistic is very similar to the residual sum of squares from the analysis of
variance. An /?2-like prediction error statistic can be computed from PRESS simply by replacing
the residual sum of squares in the equation for R2 by PRESS. This gives:
Rl	^.£^§1-^2
™on TotalSS 273428
We would expect the RSM model to explain in excess of 99% of the variability in data produced
by the simulation model. This is excellent validation of potential prediction performance.
2. Summary of Conclusions
I conclude that the RSM approach has produced statistical metamodels that are an excellent
alternative to the ALPHA simulation model. So long as they are used to interpolate over the ranges
of the four factor used in their construction I expect that they will be excellent alternatives to the
ALPHA simulation procedure.
TOPIC 2: Reasonableness of any assumptions, implicit or explicit, contained in EPA's execution of the
methodology.
Sanva Carlev
1. After a thorough review of the report and supporting documentation, my general impression is that
response surface statistical methods are an appropriate and efficient approach to generate data
needed to populate the OMEGA model. The RSM is an analysis tool that is increasingly accepted in
engineering and other disciplines, and subjected to rigorous peer review. An analysis of the model
performance in this specific case also leads me to believe that the RSM approach is highly accurate,
and capable of generating results that match the significantly more time-intensive ALPHA
simulations.
Doug Montgomery
1. I investigated the adequacy of the RSM models by first analyzing the residuals from these models in
the spreadsheets that were provided. I constructed normal probability plots of the residuals and
plots of the residuals versus the predicted response. These plots investigate the normality of the
response variable and the equality of variance assumption, both of which are standard RSM
assumptions. The normality assumption is of only moderate importance since the underlying
statistical methodology is robust to all but severe departures from normality. A few of the normal
probability plots exhibited very small potential departures from normality but nothing severe
RTI International	Peer Review of EPA's RSE Report	6

-------
enough to call model validity into question.
2.	The equal variance assumption is more important, and moderate to large departures from this
assumption may require remedial measure such as the use of variance-stabilizing transformation.
Similarly [to the normal probability plots], some of the plots of residuals versus the predicted
response exhibited a non-random pattern, but none of the patterns were serious enough to
question the equal variance assumptions.
3.	It is also worth noting that the model residuals are extremely small as all models provide extremely
good fits to the data obtained from the simulation model.
TOPIC 3: Clarity, completeness and accuracy of the technical application of response surface
modelling.
Suiit Das
1.	Response surface methodology (RSM) explores the relationships between several explanatory
variables and one or more response variables. A sequence of designed experiments (DOE) was used,
i.e., the main idea of RSM to obtain an optimal response. A DOE used in this case was based on an
automated process that is configured to produce a complete set of ALPHA results for all
combinations of engines, transmissions, roadloads, and vehicle types to be used in the OMEGA
analysis. It is a relatively easy statistical model to estimate and apply, even when little is known
about the process. It maximizes the production of a special substance by optimization of operational
factors. A factorial experiment or a fractional factorial design generally used to estimate RSE process
has generated as series of equations from a complete set of ALPHA data for each vehicle type and
powertrain model. A second-degree RSE polynomial model was developed for each 24 vehicle cases
based on a combination of 6 vehicle types and 4 powertrain types in the present analysis.
2.	Overall, the quality of RSE methodology appears to be reasonable for the four independent variables
considered. The validity of this methodology need to reexamined if it is expanded to a higher
number of independent variables in the future.
TOPIC 4: Any recommendations for specific improvements to the functioning or the quality of the
methodology.
Sanya Carlev
1. Design of Experiments: A future extension of model validation could be an assessment of the RSM
output with actual testing data. One should assume that the results would be similar to the
estimates of comparison between ALPHA and RSM, however, since the EPA's previous work found
that ALPHA estimates were within the margin of 3% error as compared to actual vehicle
performance testing.
RTI International
Peer Review of EPA's RSE Report
7

-------
2. Transparency: As stated in the report, one of the benefits of the RSM is "increased transparency
regarding synthesis of ALPHA simulation into OMEGA modeling". It is not entirely clear to me how
the use of RSM will increase transparency. But I strongly encourage and support full transparency of
modeling inputs, outputs, processes, and supporting information.
EPA response: EPA's mention of transparency refers to comments received from stakeholders
discussing the challenge to understand portions of the Lumped Parameter Model that was previously
used to determine the effectiveness of vehicle technologies. In response, a full matrix of ALPHA
model runs along with the industry standard RSE methodology completely replaces the Lumped
Parameter Model providing a straightforward method for stakeholders to evaluate.
Suiit Das
1.	Section 6. Baseline Vehicle Adaptation needs further details in terms of the necessary process steps
for adjusting the effectiveness of a baseline vehicle to match the ALPHA model. The adjustment
approach for the baseline vehicle adaptation is an interesting one as it allows ~ 50 alternative
options to consider in a baseline vehicle.
EPA response: Since the writing of the report, the ALPHA model parameters were expanded
eliminating the need for the Baseline Vehicle Adaptation described in section 6. This section of the
report has also been deleted as this process is no longer applicable.
2.	Since the RSE final output is C02 emissions provided to the OMEGA model with the technology
alternatives necessary to produce the most cost-effective path for compliance, a short discussion of
it will be useful for unfamiliar users.
EPA response: One of the preprocessing steps for the OMEGA model is to produce approximately 50
technology improvement options for each vehicle in the current baseline fleet. The OMEGA model
iterates through the technology options for all vehicles in a manufacturers fleet until compliance is
achieved.
3.	A description of three different transmission types considered and denoted by numerals (i.e., 2, 4, &
5) would be useful. An appropriate justification needs to be included why other two types, i.e., 1 and
3 were not considered for the RSM DOE analysis.
EPA response: Transmissions 2, 4, and 5 represent three actual transmissions that EPA benchmarked
for efficiency. For successful use in the RSMthe increase in efficiency needs to be as linear as
possible. Randomly assigning the numbers 1, 2, and 3 for example would have resulted in a very
nonlinear response and not suitable for the RSM. The efficiencies for the three transmissions plotted
against the numbers 2, 4, and 5 were quite linear in this case and was chosen for simplicity. A future
case with a different mix of transmissions may require more resolution either by using decimal points
or larger numbers to find the proper number to represent a particular transmission for a linear
RTI International
Peer Review of EPA's RSE Report
8

-------
response.
4.	It is unclear why the assumed vehicle mass reduction value is not actually reflected in the ALPHA
spreadsheets provided, e.g., for 2020 TURB24 vehicle, 3109.15 lbs and 2961.3 lbs Test Weight have
been assumed for a mass reduction of 5% and 10%, respectively, for a baseline vehicle Test Weight
of 3257 lbs? Similar level of difference was found in all 21 different vehicle type/powertrain
considered for RSM.
EPA response: The mass reduction is calculated from the curb weight of the vehicle, not the test
weight The test weight adds 300 lbs. to the curb weight. For this example, 3257 lbs. test weight -
300 lbs. = 2957 lbs. curb weight. 10% of2957 lbs. = 295.7 lbs. Subtracting 10% of the curb weight
from the test weight (3257 lbs. - 295.7 lbs.) results in a final test weight of 2961.3 lbs.
5.	The draft report mentions about six vehicle types in OMEGA analysis and four powertrain categories
in the ALPHA. It is unclear about the consistency in the number of vehicle types and powertrain
categories between these two tools and thereby to what extent does the current RSM cover the
overall analysis scope of the OMEGA technology options?
EPA response: At the time of this report, there were 6 vehicle types and 4 powertrain types resulting
in 24 RSEs. The 6 spreadsheets provided represented the complete set of ALPHA runs for each
vehicle type. The 4 individual tabs in each spreadsheet filtered the specific set of runs for each of the
4 powertrain types resulting in 24 combinations and a 1 to 1 correlation between the ALPHA DOE
and the RSM.
6.	In spite of the fact that there are four independent variables, i.e., mass reduction, aerodynamic drag
reduction, rolling resistance reduction, and transmission type have been used for the development
of RSE equations, but +50 ALPHA data variables have been included in the several vehicle
spreadsheets provided. It'd be good to provide the description of each of the ALPHA variables for an
understanding of impacts of the four major dependent variables considered.
EPA response: The ALPHA variables define each powertrain type and are held constant for each DOE
generated. The descriptions of the individual ALPHA data variables are beyond the scope of this
review.
7. As the RSE "Effectiveness" implementation is expanded beyond the currently limited six vehicles,
four powertrains, and three transmission type options provided, the user-friendliness in terms of
RTI International
Peer Review of EPA's RSE Report
9

-------
inputs should be kept in mind. Using the current framework provided as an example, it is difficult for
a novice user to perform a quick analysis. Specifically, a discussion on the "Baseline Vehicle
Adaptation" procedure needs to be included in the documentation, when all original LPM
technology options are also available for RSM for the baseline vehicle adaptation. Some
Comments/Warning should be included if the results are invalid for transmission cases 1 & 3 as is
the case now. The inputs for Vehicle Type, Model, and Transmission in Column A should be
interlinked with the corresponding numeric value in Column B on this worksheet.
EPA response: As stated above, the Baseline Vehicle Adaptation is no longer used in the RSM
process. The RSM tool in the form presented is designed as part of an automated process and not for
manual input at this release.
8.	It'd be useful for the EPA draft report completeness to provide some background information on the
models and tools used in EPA's light-duty Greenhouse Gas (GHG) rulemakings for unfamiliar
audience.
EPA response: The report mentions the tools for historical context without extensive detail as this is
beyond the scope of this review. Details for the previous tools used can be found here:
https://www.epa.gov/regulations-emissions-vehicles-and-engines/midterm-evaluation-light-duty-
vehicle-greenhouse-gas
9.	Not sure whether any model validation was done in terms of using the model to predict the
response for one or more combinations of design factors that were not used to build the RSM
models? What agreements between the two results were found for such a validation?
EPA response: As stated earlier, the road load factors included in the RSM are linear and predictable.
Many additional ALPHA runs were performed to verify that the ALPHA model and the RSM remain
stable for these intermediate values.
RTI International
Peer Review of EPA's RSE Report
10

-------
Appendix A. Peer Reviewers7 Resumes
RTI International
Peer Review of EPA's RSE Report
11

-------
Sanya Carley
School of Public and Environmental Affairs
Indiana University, Room 353, 1315 East Tenth Street, Bloomington, IN 47405
(812) 856-0920; scarley@indiana.edu
Professional Appointments
Associate Professor, School of Public and Environmental Affairs, Indiana University
Chair, Policy Analysis and Public Finance Faculty Group
Assistant Professor, School of Public and Environmental Affairs, Indiana University
Professional Affiliations
Research Fellow, Center for Organization Research and Design
Research Member, The Richard G. Lugar Center for Renewable Energy
Member, Scholars Strategy Network
Brain Trust Member, IronOak
Educati ON
University of North Carolina at Chapel Hill. Ph.D. Public Policy, 2010.
Dissertation Committee: Richard Andrews (Chair), Doug Crawford-Brown, Gary Henry, Richard
Newell, Tim Johnson
University of Wisconsin-Madison. M.S. Urban and Regional Planning, Masters Certificate, Energy Analysis
and Policy, 2006.
Swarthmore College. B.A. Economics, B.A. Sustainable Development, 2003.
Areas of Research
Energy Policy, Electricity Markets, Transportation Industry, Energy-based Economic Development, Policy
Instruments, Electric Vehicles, Distributed Generation
Consulting and Work Experience
Consultant, Institute for International Business, Indiana University, Bloomington, IN. (2013)
Consultant, Environmental Protection Agency, Conflict Prevention and Resolution Center, Washington D.C.
(2010)
Consultant, Research Triangle Institute International, Center for Technology Applications, Research Triangle
Park, NC. (2009 -2010)
Consultant, ARCeconomics, SC. (2007 - 2010)
Consultant, The Nicholas Institute for Environmental Policy Solutions, Durham, NC. (2008)
Graduate Fellow, Center for Sustainable Energy, Environment, and Economic Development, Chapel Hill,
NC. (2006 -2010)
Energy Program Specialist, Wisconsin Public Utility Institute, Madison, WI. (2005 - 2006)
Consultant, World Bank Group, Development Economic Research of the Public Sector, Washington D.C.
(2003 - 2006)
2014 - present
2016 - present
2010 -2014
2016	- present
2013 - present
2017	- present
2016 - present
RTI International
Peer Review of EPA's RSE Report
12

-------
Books
Carley, S., Lawrence, S. 2014. Energy-based Economic Development: How clean energy can drive
development and stimulate economic growth. Springer: New York.
Reviews of the Book:
1.	Sharma, K. R., Wilson, E. 2016. Book Review of "Energy-based Economic Development: How clean
energy can drive development and stimulate economic growth." Journal of Policy Analysis and
Management 35(3): 728-731.
2.	Ghadimi, H. 2017. Book review: Energy-based Economic Development: How clean energy can drive
development and stimulate economic growth. Economic Development Quarterly 31(1): 92-96.
Peer or Editor Reviewed Publicati ons
* Denotes student co-author at time of writing
Carley, S., Evans, T. P., Konisky, D. M. 2018. Adaptation, culture, and the energy transition in American
coal country. Energy Research & Social Science.
Nicholson-Crotty, S., Carley, S. 2017. Policy Learning in the Context of State Energy Policy. Forthcoming,
State Politics and Policy Quarterly.
Carley, S., Baldwin, E.*, MacLean, L. M., Brass, J. N. 2017. Global Expansion of Renewable Energy
Generation: An Analysis of Policy Instruments. Environmental and Resource Economics 68(2): 397-440.
- Winner of the 2014 Best Paper Award for Research in Comparative Policy Analysis, honored by the
Association of Public Policy Analysis and Management and the International Comparative Policy
Analysis Forum.
Carley, S., Nicholson-Crotty, S., Miller, C.* 2017. Adoption, Reinvention, and Amendment of Renewable
Portfolio Standards in the American States. Journal of Public Policy 37(4): 1-28.
Baldwin, E.*, Carley, S., Brass, J. N., MacLean, L. M. 2017. Global renewable energy policy: A comparative
analysis of countries by economic development status. Journal of Comparative Policy Analysis 19(3): 277-
298.
Davies, L. L., Carley, S. 2017. Emerging shadows in national solar policy? Nevada's Net Metering
Transition in Context. The Electricity Journal.
Krause, R., Lane, B., Carley, S., Sperl J.*, Graham, J. 2016. Assessing the Demand for Electric Vehicles
under Future Cost and Technological Scenarios. International Journal of Sustainable Transportation 10(8):
742-751.
Clark-Sutton, K.*, Siddiki, S., Carley, S., Wanner, C.*, Rupp, J., Graham, J.D. 2016. Plug-in electric vehicle
readiness: Rating cities in the United States. The Electricity Journal 29(1): 30-40.
Carley, S. 2016. Energy programs of the American Recovery and Reinvestment Act of 2009. Review of
Policy Research 33(2): 201-223.
Carley, S. 2016. The American Recovery and Reinvestment Act of 2009: What have we learned? Review of
Policy Research 33(2): 119-123.
Zirogiannis, N., Alcorn, J.*, Rupp, J., Carley, S., Graham, J. 2016. State regulation of unconventional gas
development in the U.S.: An empirical evaluation. Energy Research and Social Science 11:142-154.
RTI International
Peer Review of EPA's RSE Report
13

-------
Nicholson-Crotty, S., Carley, S. 2016. Effectiveness, Implementation Capacity, and Policy Diffusion: Or,
"Can We Make that Work for Us?" State Politics and Policy Quarterly 16(1), 78-97.
Paydar, N.*, Schenk, O., Alcorn, J.*, Bowers, A., Carley, S., Rupp, J., Graham, J.D. 2015. The Effect of
Community Reinvestment Funds on Local Acceptance of Unconventional Gas Development. Economics of
Energy & Environmental Policy 15(1): 1-26.
Esposito, D.*, Rupp, J., Carley, S. 2015. Interaction of risks associated with natural gas and renewable based
electricity. The Electricity Journal 28(8): 69-84.
Siddiki, S., Dumortier, J., Curley, C., Carley, S., Krause, R. 2015. Regulating for Innovation and Technology
Adoption: The Case of Plug-In Vehicles. Review of Policy Research 32(6): 649-674.
Warren, D.*, Wendling, Z.*, Bower-Bir, J.*, Fields, H.*, Richards, K., Carley, S., Rubin, B. 2015.
Estimating State and Sub-State Economic Effects of a Carbon Dioxide Tax Policy: An Application of a New
Multi-Region Energy-Economy Econometric Model. Regional Science, Policy and Practice 7(3): 119-139.
MacLean, L., Brass, J., Carley, S., El-Arini, A.*, Breen, S.* 2015. Democracy and the distribution ofNGOs
promoting renewable energy in Africa. Journal of Development Studies 51(6): 725-742.
Dumortier, J., Siddiki, S., Carley, S., Cisney, J.*, Krause, R., Lane, B., Rupp, J., Graham, J. 2015. Effects of
providing total cost of ownership information on consumers' intent to purchase a hybrid or plug-in electric
vehicle. Transportation Research Part A: Policy and Practice 72: 71-86.
Carley, S., Nicholson-Crotty, S., Fisher, E.* 2015. Capacity, Guidance, and the Implementation of the
American Recovery and Reinvestment Act. Public Administration Review 75(1): 113-125.
Carley, S., Hyman, M.* 2014. The American Recovery and Reinvestment Act: Lessons from Energy
Program Implementation Efforts. State and Local Government Review 46(2): 140-147.
Baldwin, E.*, Brass, J., Carley, S., MacLean, L. 2014. Issues of scale in distributed generation electrification
for rural development. WIRES: Energy and Environment.
Warren, D.*, Carley, S., Krause, R., Rupp, J., Graham, J. 2014. Predictors of attitudes toward carbon capture
and storage using data on world views and CCS-specific attitudes. Science and Public Policy.
Krause, R., Carley, S., Warren, D.*, Rupp, J., Graham, J. 2014. Not Under My Backyard: Geographic
proximity and public acceptance of CCS facilities. Risk Analysis 34(3): 529-540.
Wendling, Z. A.*, Attari, S. Z., Carley, S., Krause, R. M., Warren, D.*, Rupp, J., Graham, J. D. 2013. On the
importance of strengthening moderate beliefs in climate science to foster support for immediate action.
Sustainability 5(12): 5153-5170.
Krause, R., Carley, S., Lane, B., Graham, J. 2013. Perception and Reality: Public Knowledge of Plug-in
Electric Vehicles. Energy Policy 63: 443-440.
Lane, B., Messer, N.*, Hartman, D.*, Carley, S., Krause, R., Graham, J. 2013. Government promotion of the
electric car: Risk management or industrial policy? European Journal of Risk Regulation 2: 227-245.
RTI International
Peer Review of EPA's RSE Report
14

-------
Carley, S., Krause, R., Lane, B. Graham, J. 2013. Intent to purchase a plug-in electric vehicle: A survey of
early impressions in large US cites. Transportation Research Part D: Transport and Environment 18: 39-45.
Brass, J., Carley, S., MacLean, L., Baldwin, E.* 2012. Power for development: An analysis of on-the-ground
experiences of distributed generation in the developing world. Annual Review of Environment and Resources
37: 107-136.
Carley, S., Browne, T.* 2012. Innovative US Energy Policy: A review of states' policy experiences. WIREs:
Energy and Environment 00: 1-19.
Carley, S., Miller, C.* 2012. Regulatory stringency and policy adoption: Reassessment of renewable
portfolio standards. Policy Studies Journal 40(4): 730-756.
Carley, S., Krause, R., Warren, D.*, Rupp, J., Graham, J. 2012. Early public impressions of terrestrial CCS
in a coal-intensive state. Environmental Science & Technology 46: 7086-7093.
Gaul, C.*, Carley, S. 2012. Solar set asides and renewable energy certificates: Early lessons from North
Carolina's experience with its Renewable Portfolio Standard. Energy Policy 48: 460-469.
Carley, S., Andrews, R. L. 2012. Creating a sustainable U.S. electricity sector: The question of scale. Policy
Sciences 45(2): 97-121.
Carley, S. 2012. Energy demand-side management: New perspectives for a new era. Journal of Policy
Analysis and Management 31(1): 6-32.
Carley, S., Brown, A., Lawrence, S. 2012. Economic development and energy: From fad to a sustainable
discipline? Economic Development Quarterly 26(2): 111-123.
Carley, S. 2012. National clean energy standards: Experience from the states. Review of Policy Research
29(2): 301-307. Originally printed in SPEA Insights, July 2011.
Carley, S. 2011. Decarbonization of the U.S. electricity sector: Are state energy policy portfolios the
solution? Energy Economics 33(5): 1004-1023.
Carley, S. 2011. Normative dimensions of sustainable energy policy. Ethics, Policy & Environment 14(2):
211-229.
Carley, S. 2011. The era of state energy policy innovation: A review of policy instruments. Review of Policy
Research 28(3): 265-294.
Carley, S., Lawrence, S., Brown, A., Nourafshan, A.*, Benami, E.* 2011. Energy-Based Economic
Development. Renewable and Sustainable Energy Reviews 15(1): 282-295.
Carley, S. 2010. Historical analysis of U.S. electricity markets: Reassessing carbon lock-in. Energy Policy
39(2): 720-732.
Carley, S. 2009. Distributed generation: An empirical analysis of primary motivators. Energy Policy 37(5):
1648-1659.
Carley, S. 2009. State renewable energy electricity policies: An empirical evaluation of effectiveness. Energy
Policy 37(8): 3071-3081.
RTI International
Peer Review of EPA's RSE Report
15

-------
Law Journal Publications
Carley, S., Messer, N.* Graham, J. 2012. Innovation in the Auto Industry: The Role of the U.S.
Environmental Protection Agency. Duke Environmental Law and Policy Forum 21: 367-399.
Carleyolsen, S. 2006. Tangled in the Wires: An Assessment of the Existing U.S. Renewable Energy Legal
Framework. Natural Resources Journal 46 (3): 759-792.
Book Reviews
Carley, S., Graff, M.* 2017. Review of "Climate and Clean Energy Policy: State Institutions and Economic
Implications." American Review of Public Administration. Forthcoming.
Peer-Reviewed Policy and BusinessReports
Carley, S., Duncan, D., Graham, J. D., Siddiki, S., Zirogiannis, N., 2017. "A Macroeconomic Study of
Federal and State Auto Regulations with Recommendations for Analysts, Regulators, and Legislators."
Carley, S., Davies, L. 2016. "Nevada's Net Energy Metering Experience: The Making of a Policy Eclipse?"
Brookings Institution Report.
Carley, S., Duncan, D., Esposito, D.*, Graham, J. D., Siddiki, S., Zirogiannis, N., 2016. "Rethinking Auto
Fuel Economy: Technical and Policy Suggestions forthe 2016-17 Midterm Reviews."
Carley, S., Jasinowski, J., Glassley, G.*, Strahan, P.,* Attari, S., Shackelford, S. October 2014. "Success
Paths to Sustainable Manufacturing."
School of Public and Environmental Affairs, 2011. "Plug-in Electric Vehicles: A Practical Plan for
Progress." The report of an expert panel [Contributing author].
Policy Reports and White Papers
Indiana University Public Policy Institute, February 2012. "An environmentally sound energy policy: One
key to Indiana's economic future." Policy brief prepared for Indiana policymakers by the Indiana Policy
Choices Energy and Environment Commission [Commission member and contributing author].
Carley, S., Hyman, M.* The "Grand Experiment:" An early review of energy-related American Recovery
and Reinvestment Act Efforts. PERI Working Paper Series Report 338.
Carley, S., Desai, S., Bazilian, M., Kammen, D. 2012. Energy-based economic development: Prioritizing
opportunities for developing countries. FEEM Working Paper 25.2012.
Baldwin, L.*, Carley, S., Gardner, W.*, June 2011. "Demand-side Management and Energy Efficiency in
Indiana: A Comparison of Policy Instruments." Policy brief prepared for the Indiana Utility Regulatory
Commission.
The Nicholas Institute, 2009. "An Evaluation of Utah's Greenhouse Gas Reduction Options." Technical
policy report prepared forthe state of Utah. [Contributing researcher],
Carleyolsen, S., Voss, S., 2006. "Recommendations forthe Governor's Taskforce on a Wisconsin
Bioindustry Strategy." White Paper prepared for the Wisconsin's Bioindustry Consortium Taskforce.
RTI International
Peer Review of EPA's RSE Report
16

-------
Carleyolsen, S., Rude, J., Jenkins, A., 2006. "IGCC: A Cost-Benefit Analysis." White Paper prepared for the
Wisconsin Public Service Commission and the Wisconsin IGCC Governor's Taskforce.
Carleyolsen, S., Meyer, T., Scott, I., Rude, J., 2005. "Estimating Economic Value of Jefferson County Parks,
Trails, and Open Space." White Paper given to the Jefferson County Board of Supervisors. Jefferson County,
WI.
Media Publications
Carley, S. October 2017. Op-Ed: Mandates help motorists, economy in the long run. Printed in McClatchy
papers.
Carley, S., Konisky, D. March 2017. Op-Ed: Changes to Indiana's Solar Policy Misguided. The Herald-
Times (as well as numerous other outlets).
Jasinowski, J., Carley, S. 2014. Op-Ed: Sustainable Manufacturing Makes Cents. Manufacturing Leadership
Journal.
Carley, S., Hyman, M. January 12, 2012. Op-Ed: '"Green energy' is the best route to profitable public
investment." Printed in McClatchy papers, including the Miami Herald, Kansas City Star, and the
Sacramento Bee (Also printed in 37 other U.S. news outlets).
Other Publicati ons
Carley, S. January 2017. How states are grappling with solar panels, net energy metering, and the evolving
electric utility industry. Scholars Strategy Network Brief.
Graham, J. D., Cisney, J.*, Carley, S., Rupp, J. 2014. No time for pessimism about electric cars. Issues in
Science & Technology.
Carley, 2014. Response to Pollin, R. 2014. A Clean Energy Program for the United States. Boston Review.
July/August Issue.
Carley, S. 2012. Electric vehicles: Public acceptance, infrastructure and policy. USAEE Dialogue 20(3).
Graham, J., Carley, S., Messer, N.*, Hartman, D.* February, 2011. Plug-in Electric Vehicles: A Practical
Plan for Progress. SPEA Insights.
Carley, S. May, 2011. National clean energy standards: Experience from the states. SPEA Insights.
Selected Works in Progress
Siddiki, S., Carley, S., Zirogiannis, N., Duncan, D., Graham, J. Does dynamic federalism yield compatible
policies? A study of federal and state vehicle standards. Revise and Resubmit Status at Publius: The Journal
of Federalism.
Lane, B., Carley, S., Siddiki, S., Dumortier, J., Clark-Sutton, K.*, Krause, R., Graham, J. D. All electric
vehicles are not the same: An intent to purchase analysis accounting for heterogeneity among vehicle types.
Revise and Resubmit Status at Transportation Research Part D: Transport and Environment.
Wendling, Z.*, Warren, D.*, Rubin, B., Carley, S., Richards, K. An Energy-Economy Econometric Model
for Conducting State-Level Energy Policy Analysis. Revise and Resubmit Status at Energy Policy.
RTI International
Peer Review of EPA's RSE Report
17

-------
Carley, S., Zirogiannis, N., Duncan, D., Siddiki, S., Graham, J. D. An analysis of the macroeconomic effects
of 2017-2025 federal fuel economy and greenhouse gas emissions standards. Manuscript under review.
Carley, S., Nicholson-Crotty, S. Energy Policy Learning and Information Channels in the American States.
Manuscript under review.
Carley, S., Yahng, L. Willingness to pay for sustainable beer. Manuscript under review.
Carley, S., Davies, L., Spence, D., Zirogiannis, N. Renewable Portfolio Standards, Renewable Energy
Markets, and the Importance of Policy Design. Manuscript under review.
Ross, J., Carley, S. Efficient Siting of Nuisance Facilities Under Regulatory and Fiscal Decentralization:
Empirical Evidence from the Effect of Political Borders on Wind Farms Location. Manuscript under review.
Duncan, D., Zirogiannis, N., Carley, S., Siddiki, S., Graham, J. D. The effect of CAFE standards on vehicle
sales projections: A total cost of ownership approach. Manuscript under review.
Carley, S., Evans, T., Konisky, D. Vulnerability and the U.S. Energy Transition. Manuscript under review.
Jenn, A., Hardman, S., Carley, S., Zirogiannis, N., Duncan, D., Graham, J. D. Cost implications for
automakers' compliance with emission standards from Zero Emissions Vehicle mandate. Working paper.
Baldwin, E., Carley, S., Nicholson-Crotty, S. The global diffusion of renewable energy policies. Working
paper.
Zirogiannis, N., Carley, S., Duncan, D., Siddiki, S., Graham, J. D. Overcoming the shortcomings of U.S.
plug-in electric vehicle policies. Working paper.
Siddiki, S., Carley, S., Zirogiannis, N., Duncan, D., Graham, J. D. Policy compatibility by design: The case
of U.S. vehicle emissions standards. Working paper.
Alcorn, J., Carley, S. Exploring Renewable Energy Certificate market dynamics: What role do markets play
in renewable energy growth and development? Working paper.
Conference Proceedings, Papers, and Posters (co-author presented work not listed)
"A Macroeconomic Study of Federal and State Auto Regulations with Recommendations for Analysts,
Regulators, and Legislators." Paper presented at the Annual Conference, U.S. Association of Energy
Economics, Houston, TX, November, 2017.
"A Macroeconomic Study of Federal and State Auto Regulations with Recommendations for Analysts,
Regulators, and Legislators." Paper presented at the 39th Annual Research Conference, Association for
Public Policy Analysis and Management, Chicago, IL, November, 2017.
"The Global Diffusion of Renewable Energy Policies." Paper presented at the 38th Annual Research
Conference, Association for Public Policy Analysis and Management, Washington, D.C., November, 2016.
"Exploring renewable energy certificate market dynamics: What role do markets play in renewable energy
growth and development?" Paper presented at the International Association of Energy Economics
Conference, Bergen, Norway, June, 2016.
RTI International
Peer Review of EPA's RSE Report
18

-------
"Policy Learning in the Context of State Energy Policy." Paper presented at the 74th annual Midwest Political
Science Association Conference, Chicago, IL, April, 2016.
"Exploring renewable energy certificate market dynamics: What role do markets play in renewable energy
growth and development?" Paper presented at the 37th Annual Research Conference, Association for Public
Policy Analysis and Management, Miami, FL, November, 2015.
"The Electric Vehicle Attitude-Behavior Gap: Moving Beyond the Early Adopters." Paper presented at the
34th Annual Conference, U.S. Association of Energy Economics, Pittsburgh, PA, October, 2015.
"Global Renewable Energy Generation: An Analysis of Renewable Energy Drivers Across Gross National
Income Categories." Paper presented at the 36th Annual Research Conference, Association for Public Policy
Analysis and Management, Albuquerque, NM, November, 2014.
"Global expansion of renewable energy generation: An evaluation of policy instruments." Paper presented at
the 35th Annual Research Conference, Association for Public Policy Analysis and Management, Washington,
D.C., November, 2013.
"Social learning and policy diffusion: adoption, reinvention, and amendment of the renewable portfolio
standard." Paper presented at the Energy systems in Transition Conference, Karlsruhe, Germany, October,
2013.
"Global expansion of renewable energy generation: An evaluation of policy instruments." Paper presented at
the 32nd U.S. Association of Energy Economists/International Association for Energy Economists
Conference, Anchorage, AK, July, 2013.
"Global expansion of renewable energy generation: An evaluation of policy instruments." Paper presented at
the annual Transatlantic Policy Consortium, The Hague, Netherlands. May 2013.
"Intent to purchase a plug-in electric vehicle: A survey of early impressions in large U.S. cites." Conference
proceeding presented at the 31st U.S. Association of Energy Economists/International Association for Energy
Economists Conference, Austin, TX. November, 2012.
"Power for development: An analysis of on-the-ground experiences of distributed generation in the
developing world." Paper presented at the 33rd Annual Research Conference, Association for Public Policy
Analysis and Management, Washington, D.C., November, 2011.
"NGOs and collaborative energy service provision in developing countries." Paper presented at the American
Political Science Association Conference, Seattle, WA, September 4,2011.
"Energy-based Economic Development: From Fad to Sustainable Discipline?" Paper presented at the
Seventh International Conference on Environmental, Cultural, Economic and Social Sustainability,
Hamilton, New Zealand, January 7, 2011.
"Demand-side management: New perspectives for a new era." Paper presented at the 29th U.S. Association of
Energy Economists/International Association for Energy Economists Conference, Calgary, October, 2010.
"Demand-side management: New perspectives for a new era." Paper presented at the SPEA-Speyer
Workshop, Bloomington, IN, November, 2010.
RTI International
Peer Review of EPA's RSE Report
19

-------
"State energy policy instruments: Lessons learned from the era of state energy innovation policy." Paper
presented at the 32nd Annual Research Conference, Association for Public Policy Analysis and Management,
Boston, November, 2010.
"Decarbonization of the U.S. electricity sector: Are state energy policy portfolios the solution?" Poster
presented at the Solar Energy Research Center's Conference, Solar Fuels and Energy Storage: The Unmet
Needs, Chapel Hill, NC, January, 2010.
"Decarbonization of the U.S. electricity sector: Are state energy policy portfolios the solution?" Paper
presented at the 31st Annual Research Conference, Association for Public Policy Analysis and Management,
Washington, D.C., November, 2009.
"State renewable energy electricity policy: An empirical evaluation of effectiveness." Paper presented at the
30th Annual Research Conference, Association for Public Policy Analysis and Management, Los Angeles,
CA, November 6, 2008.
"Evaluating the Effectiveness of State Renewable Energy Policies." Poster presented at the RTEC
Sustainable Energy Symposium, Raleigh, NC, 2007.
"Tracking Social Capital Indicators using Geographic Information Systems." Presentation at the Upper
Midwest Regional Planning Conference, MN, 2005. Received Best Student Presentation award.
Invited Talks, Lectures, Webinars, or Panel Presentations
2017: Innovation, Property Rights, and the Structures of Energy, Property and Environment Research
Center (PERC), Bozeman, MT; Environmental Protection Agency, Ann Arbor; Environmental
Protection Agency, Washington D.C.; Electricity Dialogue, Northwestern University; Association of
Public Policy Analysis and Management Webinar, Washington D.C.; Workshop on Durability and
Adaptability in Energy Policy, Resources for the Future; Earth and Mineral Sciences Energy
Institute, Pennsylvania State University
2016: 2016 Austin Electricity Conference, University of Texas; U.S. Association of Energy Economics,
dual plenary session on Transportation; South Carolina Journal of International Law and Business
Symposium; University of Texas at Austin, Regional Challenges and Opportunities in Energy
Transformations Workshop.
2015: Panel on National Science Foundation funding, Indiana University; U.S. Association of Energy
Economics, session on Energy Economics Education; Workshop on Manufacturing and Public
Policy; Mini University, Indiana University; Richard G. Lugar Center for Renewable Energy;
University of Utah, S.J. Quinney College of Law, 20th Annual Stegner Symposium; University of
North Carolina at Chapel Hill, Odum Institute.
2014: Kelley School of Business, Indiana University; Martin School of Public Policy and Administration,
University of Kentucky; Ford School of Public Policy, University of Michigan.
2013: ARPA-E; Centre for Energy Economics and Policy, ETH Zurich; Global Mini-Conference, Indiana
University; Energy Student Leaders Association, Indiana University; Energy and Climate Seminar
Series, Georgetown University; International Public Affairs Association, Indiana University; 13th
Annual Association of SPEA Ph.D. Students Conference, Indiana University; Kelley School of
Business Renaissance Week, Indiana University.
2012: Center for Local, State, and Urban Policy, University of Michigan; School of Public and
Environmental Affairs Dean's Council Meeting, Indiana University; Kelley School of Business
Renaissance Week, Indiana University; Policy Lecture Series, UNC-Chapel Hill Department of
Public Policy.
2011: Mini University, Indiana University; Ph.D. Student Research Seminar, School of Public and
Environmental Affairs, Indiana University;
RTI International
Peer Review of EPA's RSE Report
20

-------
2010 Ph.D. Student Research Seminar, School of Public and Environmental Affairs, Indiana University;
Ph.D. Student Research Seminar, School of Public and Environmental Affairs, Indiana University;
University Research Day, University of North Carolina at Chapel Hill; Carolina Institute for the
Environment Board of Visitors, University of North Carolina at Chapel Hill.
2006: Wisconsin's Bioindustry Consortium Taskforce, Madison, WI.
2005: Jefferson County Board of Supervisors, Jefferson County, WI.
Panel Chair or Moderator:
2017: Environmental Politics & Governance conference; Association for Public Policy Analysis and
Management conference; U.S. Association of Energy Economics conference
2016: Midwest Political Science Association conference; U.S. Association of Energy Economics annual
conference; Association for Public Policy Analysis and Management conference
2015: U.S. Association of Energy Economics conference; Association for Public Policy Analysis and
Management conference
2014: Association for Public Policy Analysis and Management conference
2013: Association for Public Policy Analysis and Management conference; International Public Affairs
Association conference; U.S. Association of Energy Economics conference
2011: U.S. Association of Energy Economics conference; Association for Public Policy Analysis and
Management conference
2010: Association for Public Policy Analysis and Management conference
2006: Wisconsin Public Utility Institute conference
Guest Seminar Presentations at Indiana University
2016: SPEA, Experience Day; SPEA, V680 Research Design
2015: SPEA, Experience Day; SPEA, E574 Energy Markets and Analysis; SPEA, V680 Research Design;
Statistics, S690 Statistical Consulting
2014: SPEA, V680 Research Design
2013: SPEA, V680 Research Design; SPEA, Experience Day; Kelley School of Business, L302
Sustainability Law & Policy.
2012: SPEA, Experience Day.
2011: SPEA, V680 Research Design; SPEA, V669 Economic Development; SPEA, V160 National and
International Policy; SPEA, Experience Day.
2010: SPEA, Advanced Math Camp
Grants
"Toward the Diffusion of Sustainable Technologies: The Case of Electric Vehicles" Co-PI with Sean
Nicholson-Crotty and Saba Siddiki. National Science Foundation. $184,996. 2016-2018.
"The Siting of Energy Infrastructure: Public Perceptions and Public Finance Impacts" Co-PI with David
Konisky and Steven Ansolabehere. Alfred P. Sloan Foundation. $259,900. 2016-2018.
"The U.S. Energy and Climate Transition: Aggregated Impacts of Policy on Vulnerable Populations" PI with
Co-PIs Tom Evans and David Konisky. Indiana University Collaborative Research Grant. Office of the Vice
Provost of Research, Indiana University. $63,437. 2016-2017.
"Consumer Willingness to Pay for Sustainability: The Case of the Brewing Industry" PI. Office of the Vice
Provost of Research Award for Research Methods and Collaboration, Indiana University. $4,942. 2016.
RTI International
Peer Review of EPA's RSE Report
21

-------
"Study of the macro-economic impact of the light-duty vehicle corporate average fuel economy, greenhouse
gas and zero-emission vehicle standards: Phases II and III" Co-PI with John Graham, Denvil Duncan, Saba
Siddiki, andNikos Zirogiannis. Alliance for Automobile Manufacturers. $590,000. 2016-2017.
"Study of the macro-economic impact of the light-duty vehicle corporate average fuel economy, greenhouse
gas and zero-emission vehicle standards: Phase I" Co-PI with John Graham, Denvil Duncan, and Saba
Siddiki. Alliance for Automobile Manufacturers. $202,723. 2015-2016.
"Informing Energy Policy Choices in Indiana using an Econometric and Technology Model." PI with Barry
Rubin. Faculty Research Support Program, Indiana University. $72,341. 2012-2013.
"Power for Development: Sustaining Small-Scale Electricity Implementation in Africa." PI with Jennifer
Brass and Lauren MacLean. Faculty Research Support Program, Indiana University. $74,484. 2012-2013.
"Exploratory Study of Risks, Benefits, and Costs of DEF and Alternatives." PI with John Graham. Navistar.
$89,509. 2011-2012.
"NGO Involvement in Sustainable Energy Programs for International Development." PI with Jennifer N.
Brass. Mitsui Environment Fund, Mitsui & Co., Ltd. $59,706. 2011-2012.
"Collaborative Provision of Low-Carbon Distributed Energy in Developing Countries." PI with Jennifer N.
Brass. Sustainability Research Development Grant, Indiana University Office of Sustainability. $15,000.
2011-2012.
"Energy-based Economic Development." Co-PI with Adrienne Brown (PI) and Sara Lawrence (PI). RTI
International R&D Grant, RTI International. $63,000. 2009-2010.
Conference Travel Grant. GPSF Travel Award, University of North Carolina at Chapel Hill. $400. 2009
Conference Travel Grant Department of Public Policy, University of North Carolina at Chapel Hill. $600.
2008.
Grant awarded for travel to Ghana, West Africa, to establish an environmental study abroad program for an
East coast consortium of colleges. Environmental Studies Grant, Swarthmore College. $10,500. 2001.
Honors and Awards
Campus Catalyst Excellence in Teaching Award, Indiana University Office of Sustainability, 2017.
George I. Treyz Award for Excellence in Economic Analysis, Best Paper Award, Regional Economic
Modeling, Inc. 2017.
Most Personable Faculty Award, Student Choice Award, School of Public and Environmental Affairs,
Indiana University. 2016.
Outstanding Junior Faculty Award, Office of the Vice Provost for Faculty and Academic Affairs and the
Office of the Vice Provost for Research, Indiana University-Bloomington. $14,500 research grant.
2013.
IU Trustees Teaching Award, Indiana University-Bloomington. 2012.
Spot Award, Research Triangle Institute International, RTP, NC. 2009.
Progress Energy Fellow, University of North Carolina at Chapel Hill. 2006-2010.
Future Faculty Fellowship, University of North Carolina at Chapel Hill. 2008.
American Planning Association Best Student Presentation Winner, Upper Midwest American Planning
Association Conference. 2005.
Morris Udall Scholar, Swarthmore College. 2002.
RTI International
Peer Review of EPA's RSE Report
22

-------
Phillip Barley Memorial Scholar, Swarthmore College. 2002.
Selected Publicity and Media Mentions
Forward Kentucky, November 13, 2017. "Did environmental rules kill mining? For coal country, that's
yesterday's debate."
Science Daily, October 27, 2017. "Efforts to revive coal industry unlikely to work, may slow job growth."
(Similar story printed in Science Newsline: Nature & Earth, Common Dreams, IWW Environmental
Unionism Caucus, and the Indiana Daily Student)
Greenwire, August 2017. "EPA gathers consumer data as it rethinks GHG standards."
CNBC, Washington Times, IU Newsroom, March 2017. "IU research shows mileage regulations bring long-
term benefits but short-term economy lag." (Reprinted in over 150 other media outlets).
Indianapolis Star, March 2017. "Solar energy in crossroads in Indiana" (Reprinted in 24 other sources).
Inside EVs, March 2016. "U.S. Cities Ranked for Plug-in Electric Car Readiness—Portland takes Top Spot."
Similar news reports appear in Autocarr, Fleet Management Weekly, and Greener Ideal.
Herald Times, February 19, 2016. "IU Researchers Urge Review of Fuel Economy Standards."
WalletHub, July, 2015. "2015 Most & Least Energy-Expensive States." (Statements quoted in three
subsequent news outlets)
CQ Researcher, April 2015. "Sustainability."
Indiana University Press, November 13, 2013. "Survey: Most Americans unaware of financial advantages of
owning an electric car" (Reprinted by four other media outlets across the country).
Society for Risk Analysis, Press Release, October 30, 2013. "Residents weigh global benefits and local risks
in views of climate change measures." (Reprinted by 289 other media outlets across the country).
Freakonomics, July 24, 2013. "How Politicians Plug Electric Cars."
Indiana University Press, July 17, 2013. "Economy edges out environment for governments plugging electric
vehicles" (A variant of the article was also published in Domestic Fuel, Green Autoblog, Blog and
Batter Chargers, EV World, Environmental Leader, Earth Techling, and The Green Optimistic).
Indiana Daily Student, April 4, 2013. "Awards granted to outstanding junior faculty."
Inside Higher Ed and WAMC, Northeast Radio, Academic Minute, March 14, 2013. "Dr. Sanya Carley,
Indiana University—Consumer Attitude and Electric Cars."
CBS, January 7, 2013. "American Drivers Not Interested in Electric Cars."
International New York Times, International Herald Tribune, January 7, 2013. "Will 2013 be the Year of the
Electric Car?"
New York Times, December 26, 2012. "Car Buyers Lack Interest in Electric Cars, Study Says."
Indianapolis Business Journal, January 2, 2013. "Report: Plug-in vehicles slow to spark interest in Indy."
Indiana University Press, December 27, 2012. "IU Study: Consumer intent to purchase electric vehicles is
low, varies by city." (Reprinted in a variety of other online outlets).
Indiana University Press, September 18, 2012. "Indiana University Study: Support for Carbon Capture is
extensive but not strong." (Reprinted in Science Daily, among a variety of other online outlets).
WTIU "Weekly Special," September 15, 2011. "Early Adopters."
WTIU News, July 19, 2011. "Next-Generation Electric Vehicle Appears in Bloomington."
SPEA Podcast, May, 2011. "The Future of Electric Cars."
Indiana University Press, May 23, 2011. "Journal article examines the effectiveness of state-level energy
policies." (Reprinted in Indiana Ag Connection, UtilityProducts.com, Newswise, Indiana Valuation).
AOL Autos, February 23, 2011. "Are Obama's Million EV's Just Science Fiction?"
Kokomo Tribune, September 9, 2010. "19% rate hike coming: Duke Energy plans increase to help pay for
$2.88B coal plant."
Indiana University Press, August 20, 2010. "Energy-based Economic Development: A Fad of Here to Stay?"
(Reprinted in News Blaze, Newswise, Renewable Energy Sources, World.org)
RTI International
Peer Review of EPA's RSE Report
23

-------
Teaching Experience
V674: Energy Economics and Policy (Graduate level)
School of Public and Environmental Affairs, Indiana University.
Spring 2011, Spring 2012, Spring 2013, Spring 2014, Spring 2015, Spring 2016
V600: Capstone (Graduate level)
School of Public and Environmental Affairs, Indiana University.
Spring 2016, Spring 2018
V550: Energy Policy Seminar (Masters and Ph.D. level)
School of Public and Environmental Affairs, Indiana University.
Fall 2015, Fall 2017
V450: Research Design (Undergraduate level)
School of Public and Environmental Affairs, Indiana University.
Fall 2015, Fall 2017
E5 74: Energy Analysis and Markets (Graduate level)
School of Public and Environmental Affairs, Indiana University.
Fall 2010, Fall 2011, Fall 2013
V680: Research Design (Ph.D. level)
Co-instructor of a team-led course, School of Public and Environmental Affairs, Indiana University.
Fall 2011, Fall 2013, Fall 2014, Fall 2015, Fall 2016
E555: Energy Resources and Policy (Undergraduate level)
Teaching Fellow, The Department of Public Policy and the Institute for the Environment, University
of North Carolina at Chapel Hill.
Spring 2009
E190H: Honors Freshman Seminar on Energy and Society (Undergraduate level)
Co-instructor, Institute for the Environment, University of North Carolina at Chapel Hill.
Spring 2009
Professional Service
Referee and Reviewer Service:
American Journal of Political Science
Climate Policy
Ecological Economics
Economic Development Quarterly
Energies
Energy Economics
Energy Policy
Energy Journal
Energy Research & Social Science
Environmental and Resource Economics
Environmental Practice
Environmental Science & Technology
Ethics, Policy & Environment
Evaluation Review
Geography
Global Environmental Change
International Journal of Business and Economics
IEEE Transactions on Power Systems
J of the Assoc of Envir and Res Economists
J of Environmental Economics and Management
J of Geography and Regional Planning
J of Policy Analysis and Management
J of Policy History
J of Politics
J of Public Admin Research and Theory
National Science Foundation
Nature Energy
Nature Climate Change
PLOS One
Policy Sciences
Policy & Society
Policy Studies Journal
Public Administration Review
Publius: The Journal of Federalism
Regulation & Governance
Review of Policy Research
Renewable and Sustainable Energy Reviews
Springer Publishing
SPEA Insights
State and Local Government Review
Sustainability: Science, Practice & Policy
Transportation Letters
RTI International
Peer Review of EPA's RSE Report
24

-------
Transportation Research Part D: Transport and
Environment
Utilities Policy
Professional, National, and State Service:
Managing Editor, Journal of Policy Analysis and Management, 2017-present.
Peer Review Committee Member, EPA Response Surface Methodology, 2017.
VP for Academic Affairs, U.S. Association of Energy Economics, 2018.
Secretary/Treasurer, U.S. Association of Energy Economics, 2015-2016.
Editorial Board, Energy Research and Social Science, 2017-2019.
Editorial Board, Public Administration Review, 2015-2017.
Executive Committee, Richard G. Lugar Center for Renewable Energy, 2017-present.
Editorial Board, State and Local Government Review, 2014-2016.
Conference student paper judge, U.S. Association of Energy Economics, 2014, 2015, 2017.
Conference poster judge, Association of Public Policy Analysis and Management, 2016, 2017.
Conference poster judge, U.S. Association of Energy Economics, 2016, 2017.
Guest editor, Special issue on the American Recovery and Reinvestment Act of 2009, Review of Policy
Research, 2015-2016.
Finance Committee, Chair, U.S. Association of Energy Economics, 2015-2016.
Chair, Natural Resource Security, Energy, and Environmental Policy Conference Program Committee,
Association of Public Policy Analysis and Management, 2015-2016.
Committee Member, Natural Resource Security, Energy, and Environmental Policy Conference Program
Committee, Association of Public Policy Analysis and Management, 2013-2016.
Conference Program Committee, U.S. Association of Energy Economics, 2015-2016.
Council Member, U.S. Association of Energy Economics, 2014.
Presidential Advisor, U.S. Association of Energy Economics, 2013.
Academic Affiliate, National Renewable Energy Laboratory, 2013.
Member, Policy Choices Energy and Environment Commission, IU Public Policy Institute, 2011-2012.
University Service:
Member, Bloomington Faculty Council Research Affairs Committee, 2015-2016.
Co-Chair, Energy and the Built Environment, Indiana University, 2013-2016.
Advisory Committee, Workshop in Methods, 2013-2016.
Member, Academic Initiatives Working Group, Indiana University, 2011-2013.
Steering committee, Student Summit on a Green Economy, Indiana University, 2010.
School Service:
Chair, Policy Analysis and Public Finance Faculty Group, 2016-present.
Promotion and Tenure Committee, Indiana University Northwest, 2017.
MPA Curriculum Committee, 2017-2018.
Chair, Policy Committee, School for Public and Environmental Affairs, Indiana University, 2015-2016.
Ph.D. Public Affairs Program Committee, School for Public and Environmental Affairs, Indiana
University, 2015-2016.
Chair, Environmental Policy Search Committee, School for Public and Environmental Affairs, Indiana
University, 2014-2015.
Policy Committee, School for Public and Environmental Affairs, Indiana University, 2014-2016.
Budgetary Affairs Committee, School for Public and Environmental Affairs, Indiana University, 2014-
2016.
Faculty Advisor, Energy Student Leaders Association, Indiana University, 2012-present.
MPA Selection Committee, School for Public and Environmental Affairs, Indiana University, 2013,
2014.
RTI International
Peer Review of EPA's RSE Report
25

-------
Sanya Carley
Peer Reviewer Resume
Last Updated 7/19/17
Environmental Policy Ph.D. Exam Committee, School for Public and Environmental Affairs, Indiana
University, 2013-2014.
Faculty Hiring Committee, Industrial Ecology and Life-Cycle Assessment, School for Public and
Environmental Affairs, Indiana University, 2011-2012.
Faculty Hiring Committee, MPA Program Director, School for Public and Environmental Affairs,
Indiana University, 2011-2012.
Member, Hiring Priorities Committee, Policy Analysis and Public Finance faculty group, School for
Public and Environmental Affairs, Indiana University, 2011-2012.
Faculty Hiring Committee, Energy Policy, School for Public and Environmental Affairs, Indiana
University, 2010-2011.
Committee Member, Energy Concentration, School for Public and Environmental Affairs, Indiana
University, 2010-2011.
Ph.D. Dissertation Committee Member:
Adam Abelkop, 2017; Yu Zhang, 2017; Jose Iracheta, 2017; Dave Warren, 2017; Sojin Jang,
2017; Jessica Alcorn, 2016; Zach Wendling, 2016; Elizabeth Baldwin, 2015; Shuang Zhao,
2015.
Ph.D. Program Committee Member, School of Public and Environmental Affairs, Indiana University:
Arthur Ku, 2017; Michelle Lee, 2016; Yu Zhang, 2014; Jessica Alcorn, 2014; Ben Inskeep,
2013; Naveed Paydar, 2013; Chris Miller, 2012; Elizabeth Baldwin, 2012; Dave Warren, 2011;
Zach Wendling, 2011; Shuang Zhao, 2010.
Honors Undergraduate or Graduate Thesis Committee Member:
Damon Smith, Indiana University, 2015; Chip Gaul, Department of Public Policy, University of
North Carolina-Chapel Hill, 2011; Elinor Benami, Department of Economics, University of
North Carolina-Chapel Hill, 2010; Rachel Escobar, Depart, of International Studies, University
of North Carolina-Chapel Hill, 2009; Jessie Prentice-Dunn, Depart, of Public Policy, University
of North Carolina-Chapel Hill, 2007.
Junior Faculty Hiring Committee, University of North Carolina-Chapel Hill, Department of Public
Policy, 2008.
Student representative, University of North Carolina-Chapel Hill, Department of Public Policy, 2007-
2008.
Facilitator, Environmental Studies in Ghana, University of Ghana-Legon and Swarthmore College, 2001.
Professional Membership
Association for Public Policy Analysis & Management (APPAM)
Association of Collegiate Schools of Planning (ACSP)
Brewers Association (BA)
Midwest Political Science Association (MPSA)
International Association of Energy Economists (IAEE)
United States Association for Energy Economics (USAEE)
RTI International
Peer Review of EPA's RSE Report
26

-------
Sujit Das
Peer Reviewer Resume
Last Updated 1/2016
VITA
SUJIT DAS
12305 Fort West Drive
Knoxville, Tennessee 37934
(865)789-0299
Email: Dass@ornl.gov
EDUCATION
MBA
Management Science and Computer Science, University of Tennessee 1984
MS
Metallurgical Engineering, University of Tennessee, 1982
B. Tech	Metallurgical Engineering, Indian Institute of Technology, Kharagpur, India, 1979.
Ranked 2nd in class with Honors.
PROFE SSIONAU EXPERIENCE
Sr. Research Staff Member, Energy and Transportation Science Division, Oak Ridge National
Laboratory, December 1984-present.
Program manager of the cost modeling of lightweight materials and clean energy manufacturing programs
forthe U.S. Department of Energy. Develop, manage and lead projects for the DOE Office of Vehicle
Technologies and Advanced Manufacturing Office. Responsible for a total annual budget of more than
$75OK consistently over the past several years and managing a team of 1-6 people per project depending
on the project type. Develop cost models of advanced materials and transportation technologies and
decision-making tools for several resource markets. Provide market assessments of energy efficient
technologies including environmental implications for both domestic and international markets.
Developed expertise in several multi-disciplinary research areas including:
~	Life Cycle Assessment of Aluminum Intensive Vehicles for the Aluminum Association
~	Next generation materials with energy/emissions reduction potential in the U.S. industry for
DOE Advanced Manufacturing Office
Manufacturing process modeling of high temperature stationary fuel cell systems in the 350-
400 kW power range for DOE Fuel Cell Technologies Program
Life cycle modeling of alternative lightweight engine design options forthe DOE Propulsion
Materials Program
~	Market potential and infrastructure assessment of ethanol and hydrogen as alternative
transportation fuels
Cost modeling and life cycle analysis of advanced vehicles and lightweight materials
Technologies for DOE Office of Vehicle Technologies
~	Material technology assessments related to Partnership for A New Generation of Vehicles
(PNGV)/Freedom Cooperative Automotive Research (FreedomCAR)
Potential of renewable energy technologies in rural Bangladesh
~	Biomass refinery analysis
Economic analysis of advanced power electronics, electric motors, and intelligent
transportation systems
RTI International	Peer Review of EPA's RSE Report	27

-------
Sujit Das
Peer Reviewer Resume
Last Updated 1/2016
Energy efficiency of distribution transformers
Cost of alternative fuels
Forecasting of petroleum and uranium supplies
~	Estimation of flood-stage economic damages
~	The economic viability of plastics and automobile recycling
Environmental implications of privatization of the power sector in India
~	Market assessments of energy efficient technologies such as home refrigerators in India
~	Inspection and Maintenance of two-wheeler vehicles in India
Assessment of uranium resources
Visiting Fellow, Tata Energy Research Institute (TERI), New Delhi, India, October 1992-June 1993.
Developed a comprehensive, computerized, and PC-based Energy-Economic-Environment database for
TERI -- the first of its kind in India and provided technical support in their ongoing energy and economic
modeling activities.
Research Assistant, Energy and Economic Analysis Section, Oak Ridge National Laboratory,
September 1982-December 1984.
Documented and evaluated several EIA, DOE maintained computers models, i.e., Headwater Benefit
Energy Gains Model and the Petroleum Allocation Model. Developed a computer software "BIOCUT"
for Economic Evaluation Model for Wood Energy Plantations.
LIST OF PUBLICATIONS
BOOK/CHAPTERS PUBLISHED
Two book chapters published in "Advanced Composite Materials for Automotive Applications:
Structural Integrity and Crash worthiness." Edited by Ahmed Elmarakbi, Univ. of
Sunderland, UK and published by Wiley & Sons (Aug.' 13)
Chapter 3: Low Cost Carbon Fibre for Automotive Applications (Part 1: Low Cost Carbon Fibre
Development);
Chapter 17: Low Cost Carbon Fibre for Automotive Applications (Part 2: Applications,
Performance and Cost Reduction Models)
"Recycling and Life Cycle Issues for Lightweight Vehicles," A Book Chapter in Materials. Design and
Manufacturing for Lightweight Vehicles, edited by P.K. Mallick, Woodhead Publishing Limited,
pp. 309-330, 2010
"Material Use in Automobiles." A Book Chapter in Encyclopedia of Energy, published by Elsevier Inc.,
Vol. 3, pp. 859-869, 2004.
"Plastic Wastes: Management. Control. Recycling, and Disposal." Noyes Data Corporation, NJ (Co-
Authored with U.S. Environmental Protection Agency and T. R. Curlee), 1991.
RTI International
Peer Review of EPA's RSE Report
28

-------
Sujit Das
Peer Reviewer Resume
Last Updated 1/2016
SELECTED REFERRED ARTICLES/PRESENTATIONS (Out of 60+ articles)
"Cost of Ownership and Well-to-Wheels Carbon Emissions/Oil Use of Alternative Fuels and
Advanced Light-Duty Vehicle Technologies," Energy for Sustainable Development.
17(2013), pp. 626-641
Served as one of the expert reviewers for the following three recent U.S. DOT/U.S. EPA reports
Mass Reduction for Light-Duty Vehicles for Model Years 2017-2025, EDAG/The George
Washington University Report, Apr. 2012
Light-Duty Technology Cost Analysis Pilot Study, FEV Draft Report, Sept. 3, 2009
An Assessment of Mass Reduction Opportunities for a 2017-2020 Model Year Vehicle Program,
Lotus Engineering Inc., Mar. 2010
"Lightweighting Opportunities in the Global Automotive Industry," invited presentation at the
2011 International Automotive Lightweight Materials Development Forum, held in
Chongqing, China, on Mar. 24-25,' 11.(Also at the 12th IUMRS International Conference
on Advanced Materials, held in Qingdao, China on Sept. 22-28, 2013)
"Importance of Economic Viability Assessment of Automotive Lightweight Materials" invited
presentation at the 3rd Annual Advanced Lightweight Materials for Vehicles conference
held on Aug. 11-12, '10, Detroit, MI.
"Analysis of Fuel Ethanol Transportation Activity and Potential Distribution Constraints,"
Transportation Research Record: Journal of the Transportation Research Board, No.
2168, Transportation Research Board of the National Academies, Washington, DC, 2010,
pp. 136-145.
"Reducing GHG Emissions in the United States' Transportation Sector" Energy for Sustainable
Development. 15 (2011) 117-136, May 11.
"Life Cycle Assessment of Carbon Fiber-Reinforced Polymer Composites," Intl. Journal of Life Cycle
Assessment. Volume 16, Issue 3, pp. 268-282, 2011
"Battle Green," an interview article published in American Metal Market, Oct. 2010, pp. 36-40.
"Shedding Pounds On a Magnesium Diet," Automotive Engg. International. Apr. 6, 2010, pp. 34-36,
interview article by Steven Ashley.
"Analysis of Fuel Ethanol Transportation Activity and Potential Distribution Constraints,"
Transportation Research Record: Journal of the Transportation Research Board. No. 2168,
Transportation Research Board of the National Academies, Washington, DC, 2010, pp. 136-145.
"Low-Carbon Fuel Standard - Status and analytic issues," Energy Policy, vol. 38, No.l, Jan. 2010,
pp. 580-591.
"Importance of Economic Viability Assessment of Automotive Lightweight Materials," invited
presentation at the 3rd Annual Advanced Lightweight Materials for Vehicles," held in Detroit, MI
on Aug. 11-12, 2010.
RTI International
Peer Review of EPA's RSE Report
29

-------
Sujit Das
Peer Reviewer Resume
Last Updated 1/2016
"A Comparative Life Cycle Assessment of Magnesium Front End Parts," SAE Paper No. 2010-01-0275.
Society of Automotive Engineers, Warrendale, PA.
"Primary Magnesium Production Costs for Automotive Applications," Journal of Metals. Vol. 60, No. 11,
2008, pp. 51-58.
"A Systems Approach to Life Cycle Truck Cost Estimation," SAE Paper No. 2006-01-3562. Society of
Automotive Engineers, Warrendale, PA.
"Automotive Lightweighting Materials Benefit Evaluation," ORNL/TM-2006/545, Oak Ridge National
Laboratory, Oak Ridge, TN, Nov. 2006
"Lightweight Opportunities for Fuel Cell Vehicles," SAE Paper No. 2005-01-0007. Society of
Automotive Engineers, Warrendale, PA.
"A Comparative Assessment of Alternative Powertrains and Body-in-White Materials for Advanced
Technology Vehicles," SAE Paper No. 2004-01-0573. Society of Automotive Engineers,
Warrendale, PA.
"Back To Basics? The Viability of Recycling Plastics by Tertiary Approaches," Working Paper #5,
Program on Solid Waste Policy, School of Forestry and Environmental Studies, Yale University,
New Haven, CT, September 1996. (with T. R. Curlee)
AWARDS & PROFESSIONAL ACTIVITIES
Awarded 2004 Journal of Metals Best Paper by the Mineral, Metals, and Materials Society (TMS)
Chair of Society of Automotive Engineering (SAE) Sustainable Program Development Committee
(2013-2014)
Member of Transportation Research Board (TRB) Committees (2008- Present)
Transportation Economics
Alternative Transportation Fuels and Technologies
Invited Speaker on the Life Cycle Assessment of Materials by Beijing University of Technology, China
Conference Session Organizers for SAE and TRB
Peer Reviewers for Several Energy and Environmental Related Journals
Past peer reviewers for the EPA and NHTSA draft reports on the vehicle mass reduction and cost analysis
of light-, medium-, and heavy-duty vehicles including:
(i)	2014 EPA Light-Duty Pickup Truck
(ii)	2015 NHTSA Costs of Medium- and Heavy-Duty Vehicle Fuel Efficiency Emission
Reduction Technologies for MY 2019-2022
(iii)	2016 NHTSA Mass Reduction for Light-Duty Vehicles for MY 2017-2025
RTI International
Peer Review of EPA's RSE Report
30

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
ARIZONA STATE UNIVERSITY
Industrial Engineering
DOUGLAS C. MONTGOMERY
Regents' Professor of Industrial Engineering and Statistics
ASU Foundation Professor of Engineering
EDUCATION AND EXPERIENCE
Degrees
Ph.D.	Virginia Polytechnic Institute, 1969
M.S.I.E.	Virginia Polytechnic Institute, 1967
B.S.I.E.	Virginia Polytechnic Institute, 1965
Academic Experience
1988 - PresentRegents' Professor of Industrial Engineering and ASU Foundation Professor of
Engineering, School of Computing, Informatics and Decision Systems Engineering (Program in Industrial
Engineering), Arizona State University.
1984 - 1988	John M. Fluke Distinguished Professor of Engineering, Director of
Industrial Engineering, Professor of Mechanical Engineering, Department of Mechanical Engineering,
University of Washington.
1978 - 1984 Professor, School of Industrial and Systems Engineering, Georgia Institute of Technology.
1972 - 1978	Associate Professor, School of Industrial and Systems Engineering, Georgia
Institute of Technology.
1969 - 1972	Assistant Professor, School of Industrial and Systems Engineering, Georgia
Institute of Technology.
1967 - 1969	Instructor, Department of Industrial Engineering and Operations Research,
Virginia Polytechnic Institute.
Industrial Experience
1966	Manufacturing/Development Engineer, Eli Lilly, Inc. Creative Packaging
Division.
1963 - 1964	Process Engineer, Union Carbide Corporation.
Professional Interests
Engineering statistics, including design and analysis of experiments, statistical methods for process
monitoring, control, and optimization, and the analysis of time-oriented data. The application of statistics to
industrial problems, including engineering design, product and process development, and manufacturing.
Consulting Experience
Extensive consulting assignments involving projects with over 100 organizations. General area of
professional experience focused on engineering applications of statistics and operations research methods.
Projects have involved design of experiments and response surface methods, implementation of statistical
process control, process development including characterization and optimization, time series analysis and
the design of forecasting systems, empirical model building, and the design and analysis of physical
distribution systems. Specific industry experience includes semiconductors and electronics, medical
devices, biotechnology, consumer products, chemical and process industries, aerospace, and the service
industries. Some consulting clients include Pfizer, Procter and Gamble, Intel, Motorola, AT&T, Boeing,
IBM, The Coca-Cola Company, Lucent Technologies, Dial Corporation, Dow Chemicals, Amoco, Georgia-
Pacific, Monsanto Chemicals, Hercules, Alcoa, and Eli Lilly.
RTI International
Peer Review of EPA's RSE Report
31

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
HONORS AND AWARDS
1.	Spring 2016 Outstanding Professor Award, given by the Vice Rector of Online Education Programs,
Technologico de Monterrey, Mexico.
2.	Spring 2015 Outstanding Professor Award, given by the Vice Rector of Online Education Programs,
Technologico de Monterrey, Mexico.
3.	2015 ASU President's Award for Innovation, as Member, Vietnam Higher Engineering Education
Alliance Program (HEEAP)
4.	Honorary Member, American Society for Quality (at time of election, the 25111 Honorary Member)
5.	Fellow, American Statistical Association
6.	Fellow, Royal Statistical Society
7.	Fellow, Institute of Industrial and Systems Engineers
8.	Elected Member, International Statistical Institute
9.	Academician, International Academy for Quality
10.	ASQ Reliability Division 2013 Award for Best Reliability Paper in Quality Engineering for the
article "Experiments for Reliability Achievement"
11.	Distinguished Service Medal, 2013. Given by the American Society for Quality.
12.	Best paper in 2012 award from the Emerald press, for the paper, "Deploying Lean Six Sigma in a
Global Enterprise - Project Identification", by B. Duarte, D.C. Montgomery, J. Fowler and J.
Konopka, published in the International Journal of Lean Six Sigma, Vol. 3, No. 3, pp. 187-205.
13.	Greenfield Medal, 2010. Given by the Royal Statistical Society, for "Contributions to the effective
application of statistical methods, particularly process monitoring and optimization, quality
improvement and design and analysis of experiments, and for his influential and accessible
expository work."
14.	American Statistical Association, 2010 Excellence in Continuing Education Course Recognition
award for the course "Modern Design and Analysis of Experiments" presented at the 2009 JSM.
15.	Arizona Society of Professional Engineers (Engineer's Week, 2010), Engineering Lifetime
Achievement Award, 2010
16.	George Box Medal, 2008. Given by the European Network for Business and Industrial Statistics
(ENBIS) for lifetime contributions to the development and application of statistical methods in
European business and industry.
17.	Deming Lecture Award, American Statistical Association, presented at the Joint Statistical
Meetings, Salt Lake City, 31 July, 2007. The presentation given accompanying the award was
entitled "A Modern Framework for Enterprise Excellence".
18.	Conference Honoree, Quality and Productivity Research Conference (American Statistical
Association), Santa Fe, New Mexico, 4-6 June, 2007.
19.	Testimonial Award, American Society for Quality, 2007, for distinguished service as Chair of the
Shewhart Medal Committee, 2005-2007.
20.	Lloyd S. Nelson Award, 2005. Given by the Statistics Division of the American Society for Quality
for the Journal of Quality Technology paper having the greatest impact for professional
practitioners.
21.	ASU Outstanding Doctoral Mentor Award, 2004.
RTI International
Peer Review of EPA's RSE Report
32

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
22.	Shewell Award, 2001. Given by the Chemical and Process Industries Division of the ASQC for the
best technical paper at the ASQC/ASA Fall Technical Conference, 2000.
23.	Shewhart Medal, 1997. Awarded by the American Society for Quality Control for Outstanding
Technical Leadership in the Field of Modern Quality Control.
24.	William G. Hunter Award, 1996. Given by the Statistics Division of the American Society for
Quality Control. This award is given for excellence in technical innovation and in the integration
of statistics with other disciplines.
25.	Brumbaugh Award, 1994. Given by the American Society for Quality Control for the best paper in
a journal of the Society.
26.	Shewell Award, 1993. Given by the Chemical and Process Industries Division of the ASQC for the
best technical paper at the ASQC/ASA Fall Technical Conference, 1992.
27.	Ellis R. Ott Award, 1992. Given by the Ellis R. Ott Foundation for the best paper on quality
engineering during a two-year period.
28.	Inagural W.L. Gore Lecture in Management Science, The Alfred Lerner College of Business and
Economics, The University of Delaware, "Design of Experiments: New Methods and How to Use
Them in Design, Development and Decision-making", 16 March 2011.
29.	Invited Keynote Address, "Innovation, Statistics and Quality Technology", Forth International
Conference on Lean Six Sigma, Glasgow, Scotland, 26-27 March, 2012.
30.	Invited Keynote Address, "The Industrial Engineer and the Quality Improvement Sciences: Have
We Missed an Opportunity?", 8th Israeli Industrial Engineering Research Conference, Beer Sheva,
Israel, May 1994
31.	W. J. Youden Memorial Address, "A Perspective on Models and the Quality Sciences: Some
Challenges and Future Directions", presented at the 42nd Annual ASQC/ASA Fall Technical
Conference October 1998.
32.	Inyong Ham Distinguished Lecturer, Department of Industrial and Manufacturing Engineering,
Pennsylvania State University, "Statistical Methods for Process Robustness Studies", November
11, 1999.
33.	Invited Keynote Address, "Experimental Design for Process and Product Design and Development"
Royal Statistical Society, Glasgow Scotland, 11 September 1998.
34.	Invited Keynote Address, "The Future of Industrial Statistics", South African Statistical Association
Annual Meeting, University of the Witswaterstrand, Johannesburg, South Africa, 10 November
2000.
35.	Invited Keynote Address, "Some Opportunities and Challenges for Industrial Statisticians",
Industrial Statistics in Action 2000, conference at the University of Newcastle-Upon-Tyne, United
Kingdom, 8-10 September, 2000.
36.	Isobel Loutit Invited Plenary Address on Business and Industrial Statistics, "The Modern Practice
of Statistics in Business and Industry", 33rd Annual Meeting of the Statistical Society of Canada,
Halifax, NS, 8-11 June 2003. This was the inaugural Isobel Loutit Address.
37.	Invited Keynote Address, "Statistics and Statisticians in Today's Business World", Royal Statistical
Society Conference on Business Improvement through Statistical Thinking, 21-22 April 2004,
Coventry, UK.
38.	Invited Keynote Address, "Statistics and the Transformation of Science, Business and Industry",
5th Annual ENBIS Conference, University of Newcastle, Newcastle-Upon-Tine, UK, 14-16
September, 2005.
39.	Invited Keynote Address, "The Modern Practice of Statistics in Business and Industry" Swiss
Statistics Meeting, Lucerne, Switzerland, 14-16 November, 2007.
RTI International	Peer Review of EPA's RSE Report	33

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
40.	Invited Keynote Address, "Modern Experimental Design and its Impact on Design for Six Sigma",
Third International Conference on Six Sigma, Edinburgh, Scotland, 15-16 December, 2008.
41.	Testimonial Award from the Board of Directors of the American Society for Quality Control, 2000,
for Leadership and Distinguished Service as Chair of the Brumbaugh Award Committee from 1996-
2000.
42.	Testimonial Award from the Board of Directors of the American Society for Quality Control, 1998,
for Leadership and Distinguished Service as Editor of the Journal of Quality Technology, 1994-
1997.
43.	Pritsker Award - Annual Teaching Award, Department of Industrial and Management Systems
Engineering, Arizona State University, 1997
44.	University Distinguished Visitor, University of Manitoba, Fall, 1994.
45.	Distinguished Alumnus, Department of Industrial and Systems Engineering, Virginia Tech
(Awarded 1994).
46.	College of Engineering and Applied Sciences, Arizona State University, Teaching Excellence
Award (Graduate), 1994.
47.	Pritsker Award - Annual Teaching Award, Department of Industrial and Management Systems
Engineering, Arizona State University 1994.
48.	Anderson Teaching Award, Department of Industrial and Management Systems Engineering,
Arizona State University, 1993.
49.	Pritsker Award - Annual Teaching Award, Department of Industrial and Management Systems
Engineering, Arizona State University, 1992.
50.	Engineer of the Year, 1987, Puget Sound Engineering Council.
51.	Industrial Engineer of the Year, 1986, Puget Sound Region Institute of Industrial Engineers.
52.	Alpha Pi Mu/AIIE Outstanding Teacher Award, School of ISyE, Georgia Tech, 1976-1977.
53.	Listed in Who's Who in the American South and Southwest, American Men and Women of Science,
Who's Who in Engineering.
54.	Phi Kappa Phi
55.	Sigma Xi
56.	Alpha Pi Mu
57.	Mu Rho Sigma (Honorary Member, Va. Tech Chapter 1995)
Professional Activities
1.	Editor, Quality & Reliability Engineering International, 2000-present
2.	Editor, Journal of Quality Technology, 1994-1997
3.	Member, Technical Advisory Board, United States Golf Association, 1997-2007
4.	American Society for Quality, Honorary Member
a.	Chair, Shewhart Medal Committee, 2004
b.	Chair, Brumbaugh Award Committee, 1997
c.	Member, Brumbaugh Award Committee, 1993 - 2000
d.	Member, Shewhart Medal Committee, 1997 - present
e.	Chair, Statistics Division, 1981-1982
f.	Statistics Technical Committee, 1976-1979
g.	Publications Management Board, 1977-1980, 1994-1997, 2000-present
h.	Secretary, PBM, 1979-1980
RTI International
Peer Review of EPA's RSE Report
34

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
5.	American Statistical Association, Fellow
a.	Founding member, Committee on Statistics in Quality and Productivity
b.	Advisory Board, Section on Physical and Engineering Sciences, 1981-1983.
6.	Royal Statistical Society, Fellow
7.	The Institute of Industrial and Systems Engineers, Fellow
a.	Region IV Chair, Production Planning and Control Division, 1971-1972
b.	Research and Publications Chair, Production Planning and Control Division, 1972-1973
c.	Director-Elect, Production Planning and Control Division, 1972-1973
d.	National Director, Production Planning and Control Division, 1973-1974
e.	Research Chair, Production Planning and Control Division, 1975-1976
f.	Advisory Board Member, Production Planning and Control Division, 1975-1976
g.	Program Chair, Quality Control and Reliability Division, 1975-1976
h.	Region IV Chair, Operations Research Division, 1974-1978
8.	Elected Member, International Statistical Institute
9.	National Academy of Science, Committee on Applied and Theoretical Statistics, Panel on Research
Needs in Statistical Quality Control, 1982-1984
10.	Advisory Editor in Engineering, John Wiley & Sons, Inc., 1979-1983
11.	Department Editor, Applied Probability and Statistics, HE Transactions, 1980-1986.
12.	Book Review Editor, Journal of Quality Technology, 1980-1982
13.	Associate Editor, Naval Research Logistics Quarterly, 1982-1988
14.	Associate Editor, Journal of Forecasting, 1981-1983
15.	Associate Editor, Revue Francaise d'Automatique, d'Informatique et de Recherche Operationnelle, 1980-
1991
16.	Department Editor, Quality & Reliability Engineering, HE Transactionsa 1992-1994
17.	Editorial Board Member, Journal of Quality Technology, 1980-1982, 1987-present
18.	Editorial Board Member, Quality and Reliability Engineering International, 1994-present
19.	Editorial Board Member, Quality Engineering, 1997- present
20.	Editorial Board Member, Journal of Applied Statistics, 2000 - present
21.	Editorial Board Member, International Journal of Experimental Design and Process Optimization, 2009-
present
22.	Advisory Editor, Quality Technology and Quantitative Management, 2005-present
23.	Editorial Board Member, International Journal of Production Research, 1997- 2009
24.	Editorial Board Member, Total Quality Management, 2000-present
25.	Advisory Editor, Journal of Probability and Statistical Science, 2002-present
26.	Associate Editor, Naval Research Logistics, 2003-2010.
27.	Referee for various journals, including: Technometries, Operations Research, Management Science, HE
Transactions, Operational Research Quarterly, Naval Research Logistics Quarterly, Journal of the Royal
Statistical Society, Computational Statistics and Data Analysis, The American Statistician, American
Institute of Chemical Engineers Transactions, Communications in Statistics, Journal of Statistical
Computation and Simulation, The Engineering Economist, Computers in Industrial Engineering,
Transportation Research, Journal of the American Statistical Association, and IEEE Transactions on
Semiconductor Manufacturing.
28.	Technical manuscript reviewer for John Wiley and Sons, McGraw-Hill, Holden-Day, Marcel Dekker and
Duxbury.
29.	Research Proposal Reviewer for the National Science Foundation, various divisions, 1987-present
30.	Chair and organizer of technical paper sessions at several conferences of IIE, ASQ, ORSA/TIMS, and
the Winter Simulation Conference, 1970-1985
RTI International
Peer Review of EPA's RSE Report
35

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
31.	Co-chair for the 12th Annual Quality and Productivity Research Conference, Co-sponsored by ASU and
Motorola Semiconductor Products, May 17-19, 1995
32.	Co-chair for the 2002 Industrial Engineering Research Conference, Orlando, FL, May 2002
33.	Chair for the 18th Annual Quality and Productivity Research Conference, Tempe, AZ, June, 2002
34.	Invited short course presenter at the Joint Statistical Meetings, the ASQ/ASA Fall Technical Conference,
the U.S. Army Design of Experiments Conference, and the Army Conference on Applied Statistics.
35.	Speaker for numerous local chapters of IIE, ASA, and ASQ, 1969-present
36.	Participant in several seminar programs for international visitors to Georgia Tech, 1969-1984
SERVICE ACTIVITIES
1. Arizona State University
Chair Search Committee, Industrial Engineering, 1993-94
Department Personnel Committee, 1988-1997, Chair 1996-97
Department Faculty Recruiting Committee, 1989-1993, 1996-97
Department Graduate Committee, 1996-present, Chair 1996-2001
Engineering College Personnel Committee, 1998-present
Engineering College Core Curriculum Committee, 1989-1991
Engineering College Graduate Committee, 1989-1993
Engineering College Bylaws Committee, 1995-1997
University Council on Research and Creative Activities, 1995-1998
Ira A. Fulton School of Engineering Personnel Committee, 1997-2005
Chair Search Committee, Industrial Engineering, 2003-04
SCIDSE Personnel Committee, 2011-2012
2.	Georgia Institute of Technology
Advisory Committee, School of ISyE, 1980-82
Chair, Graduate Committee, School of ISyE, 1982-83
M.S. Comprehensive Exam Committee, School of ISyE, 1977-1978, 1982-1983
Undergraduate Curriculum Committee, School of ISyE, 1975-76
Chair, M.S. Comprehensive Exam Committee, School of ISyE, 1974-1975
Research Evaluation Committee, School of ISyE, 1973-1974, 1974-1975
Computer Engineering Committee, College of Engineering, 1972
Chair, Probability and Statistics Interest Group, 1971-1973
Chair, Computers and Simulation Interest Group, 1971-1972
Computer Coordinator, School of ISyE, 1970-1971
Graduate Committee, School of ISyE, 1970-1973, 1977-1979
3.	Professional Development Courses Taught (Georgia Institute of Technology)
Design of Production-Inventory Systems
Design of Experiments
Materials Handling
Simulation Techniques (academic administrator, 1973)
Industrial Engineering Review (P.E. Exam)
Traffic Engineering
Statistical Methods
Statistical Design and Analysis
Design and Analysis of Experiments (academic administrator, 1978-1984)
Applied Regression Analysis (academic administrator, 1978-1984)
Sampling Methods and Statistical Analysis in Power Systems Load Research (co-administrator, 1982-
1984)
4. Professional Development Courses Taught (University of Washington)
RTI International	Peer Review of EPA's RSE Report
36

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
Applied Regression Analysis (academic administrator)
Design and Analysis of Experiments (academic administrator)
Statistical Process Control (academic administrator)
Process Optimization & Response Surfaces (academic administrator)
5. Professional Development Courses Taught (Arizona State University)
Instructor in ASU Master Black Belt Certification Program
Training the Trainer in Experimental Design
Introduction to Design of Experiments (also presented over NTU)
Developed on-line courses in design of experiments, regression analysis, and six sigma methods, certificate
program in six sigma methods/industrial statistics, participated in numerous global outreach programs to
organizations in the US and abroad
INTERNATIONAL ACTIVITIES AND CONFERENCE PARTICIPATION
1.	Academic Program Reviewer, National University of Singapore, Department of Industrial and Systems
Engineering, 2014, 2009.
2.	Invited keynote address, "Innovation, Statistics and Quality Technology", Fourth International
Conference on Lean Six Sigma, Glasgow, Scotland, 26-27 March, 2012.
3.	Invited speaker, "Generating and Assessing Exact G-optimal Designs", lassie Newton Institute for
Mathematical Sciences, Cambridge, Design and Analysis of Experiments Workshop, 30 August - 2
September, 2011.
4.	Invited keynote Address, "Modern Experimental Design and its Impact on Design for Six Sigma", Third
International Conference on Six Sigma, Edinburgh, Scotland, 15-16 December, 2008.
5.	Invited Keynote Address, "The Modern Practice of Statistics in Business and Industry" Swiss Statistics
Meeting, Lucerne, Switzerland, 14-16 November, 2007.
6.	Invited Keynote Address, "Statistics and the Transformation of Science, Business and Industry", 5th
Annual ENBIS Conference, University of Newcastle, Newcastle-Upon-Tine, UK, 14-16 September,
2005.
7.	Invited Keynote Address, Royal Statistical Society Conference on Business Improvement through
Statistical Thinking, 21-22 April 2004, Coventry, UK.
8.	Isobel Loutit Invited Plenary Address on Business and Industrial Statistics, 33rd Annual Meeting of the
Statistical Society of Canada, Halifax, NS, 8-11 June 2003. This was the inaugural Isobel Loutit Address.
9.	Invited Speaker, 6th International Conference on Teaching Statistics, Cape Town South Africa, July 2002.
10.	Invited Keynote Address, South African Statistical Association Annual Meeting, University of the
Witswaterstrand, Johannesburg, South Africa, November 2000.
11.	Invited Keynote Address, Industrial Statistics in Action 2000 Conference at the University of Newcastle-
Upon-Tyne, United Kingdom, September 2000.
12.	Invited Keynote Speaker, Royal Statistical Society, Glasgow, Scotland, September 1998.
13.	Invited Speaker, Congress of the International Federation of Nonlinear Analysts, Athens, Greece, July
1996.
14.	Invited Keynote Speaker, 8th Israeli Industrial Engineering Conference, Beer Sheva, Israel, 1994
15.	Invited Keynote Speaker, International Quality Forum, University of Texas El Paso, El Paso, Texas,
1992.
RTI International	Peer Review of EPA's RSE Report	37

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
16.	Invited Speaker, International Industrial Engineering Research Symposium, Monterey Institute of
Technology, Monterey, Mexico 1991.
17.	Chairman, Regression Methodology Session, International Forecasting Symposium, Quebec, May 1981.
18.	Invited Speaker, International Symposium on Industrial Engineering, University of Regiomontana,
Monterrey, Mexico, September 1980.
19.	Co-Chairman and organizer, Forecasting Session; 24th International TIMS Conference, Honolulu,
Hawaii, June 1979.
20.	Consultant to Coca-Cola Export Corporation (includes: Coca-Cola Europe, Coca-Cola Latin America),
1974-1984.
21.	Invited Speaker, Joint Meeting of Japan Operations Research Society, Japan Industrial Management
Association, and Kansai Institute for Information Systems, Osaka, Japan, August 1977.
22.	Invited Speaker, 6th Management Science Colloquium, Osaka University, Japan, August 1977.
23.	Invited Speaker, 23rd International Management Science Conference, Athens, Greece, July 1977.
24.	Invited Speaker, 2nd Interamerican Conference on Systems and Information Engineering, Mexico City,
November 1974.
25.	Visiting Professor of Engineering, Monterrey Institute of Technology, Monterrey, Mexico, spring quarter
1972; co-sponsored by the organization of American States.
FUNDED RESEARCH
1.	Design Rules for Vertical Paper-based Immuno-Diagnostic System, Defense Threat Reduction Agency,
joint with the University of Arizona School of Medicine and the University of Utah-Reno School of
Medicine, 2016-2017, co-principal investigator, $188,500.
2.	Science of Test, Department of Defense, 2014-2018, co-principal investigator, $485,000.
3.	Science of Test, Department of Defense, 2011-2014, co-principal investigator, $385,000.
4.	Collaborative Research: Efficient Experimentation for Product and Process Reliability Improvement,
NSF, 2009-2012, co-principal investigator, $348,000.
5.	Web-Based Active Learning Modules for Teaching Statistical Quality Control, NSF, 2009-2011, co-
principal investigator, $245,000.
6.	Advanced Techniques in Design of Experiments for Computational and Physical Multivariate
Experiments, NASA, 2008, principal investigator, $50,000.
7.	Collaborative Research: Hierarchical Modeling of Yield and Defectivity to Improve Factory Operations,
NSF/SRC/ISMT, 2004-2007, co-principal investigator, $300,000.
8.	Collaborative Research: Monitoring Product and Product Quality Profiles, NSF, 2004-2005, co-principal
investigator, $100,000.
9.	Generalized Linear Model-Based Process Control for Multivariate Measurements, National Science
Foundation, 1999-2003, co-principal investigator, $211,000.
10.	NSF IUERC in Quality & Reliability Engineering, 1997-2001, co-director and co-principal investigator.
ASU share of annual center funding was approximately $150,000.
11.	Research in Industrial Statistics (projects/graduate student research) sponsored by various organizations
including Chrysler Electronics, Kellogg Corporation, SGS Thompson, and the Dial Corporation, 1996-
1998, co-principal investigator, $231,000.
RTI International
Peer Review of EPA's RSE Report
38

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
12.	Funding for Editorial Operation of JQT, ASQC, 1994-1997, principal investigator, $315,000.
13.	Graduate Education in Engineering for Worn en and Minorities, National Science Foundation, 1992-1997,
co-principal investigator, $850,000.
14.	Process Control Methodology for the Hall Process, Alcoa, 1991, principal investigator, $65,000.
15.	Implementation of Design of Experiments, Allied-Signal Aerospace, 1989, principal investigator,
$40,000.
16.	Statistical Methods for Quality and Process Improvement, IBM Corporation, 1985-1988, principal
investigator, $450,000.
17.	Process Control and Optimization Studies for Substrate Manufacturing, IBM Corporation, 1985, principal
investigator, $67,000.
18.	Quality and Process Control in the Factory of the Future, Boeing Electronics Company, 1986, principal
investigator, $25,000.
19.	Statistical Modeling and Analysis in Quality Assurance, Office of Naval Research, 1979-1985, principal
investigator, $320,000.
20.	Determination of International Differences in Reported Fire Losses: Update and Extensions, National
Fire Data Center, U.S. fire Administration, 1981, co-principal investigator, $50,000.
21.	Factor Screening Designs in Computer Simulation Experiments, Office of Naval Research, 1978,
principal investigator, $25,000.
22.	Studies in Support of the Application of Statistical Methodology to the Design and Evaluation of
Operational Tests, Department of the Army, Harry Diamond Laboratories, 1977, co-principal
investigator, $45,000.
23.	Studies in Support of the Application of Statistical Theory and Methodology to the Design and Evaluation
of Operational Tests, Department of the Army, Harry Diamond Laboratories, 1976, co-principal
investigator, $56,000.
24.	Research Support on Method-Model Development, U.S. Army Material Systems Agency, 1976, co-
principal investigator, $15,000.
25.	Operational Testing of Complex Command and Control Systems, Department of the Army, Harry
Diamond Laboratories, 1974-1975, principal investigator, $15,000.
PUBLICATIONS
Textbooks
1.	Montgomery, D. C. (2017), Design and Analysis of Experiments, 9th edition, Wiley, Hoboken, NJ. (1st
edition, 1976, 2nd edition, 1984, 3rd edition, 1991, 4th edition, 1997, 5th edition, 2001, 6th edition, 2005,
7th edition, 2009, 8fe edition, 2012)
2.	Myers, R. H., Montgomery, D.C. and Anderson-Cook, C.M. (2016), Response Surface Methodology:
Process and Product Optimization Using Designed Experiments, 4th edition, John Wiley & Sons, New
York (Probability and Statistics Series; 1st edition, 1995, 2nd edition, 2002, 3rd edition, 2009).
3.	Montgomery, D. C., Peck, E. A., and Vining, G. G. (2012), Introduction to Linear Regression Analysis,
5th edition, John Wiley & Sons, New York. (Probability and Statistics Series; 1st edition, 1983, 2nd
edition, 1992, 3rd edition, 2001, 4th edition, 2006).
4.	Montgomery, D. C. (2013), Introduction to Statistical Quality Control, 7th edition, Wiley, Hoboken, NJ.
(1st edition, 1985, 2nd edition, 1991, 3rd edition 1996, 4th edition 2001, 5th edition, 2005, 6th edition,
2009).
RTI International
Peer Review of EPA's RSE Report
39

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
5.	Montgomery, D.C., Jennings, C.L. and Kulahci, M. (2015), Introduction to Forecasting and Time Series
Analysis, 2nd edition, Wiley (Series in Probability and Statistics, 1st edition 2009), Hoboken, NJ.
6.	Montgomery, D. C. and Runger, G.C. (2014), Applied Statistics and Probability for Engineers, 6th
edition, John Wiley & Sons, New York (1st edition, 1994, 2nd edition, 1999, 3rd edition, 2003, 4th edition,
2006, 5th edition, 2011).
7.	Montgomery, D.C., Jennings, C.L. and Pfund, M.E. (2011), Managing, Controlling and Improving
Quality, Wiley, Hoboken NJ.
8.	Kowalski, S.M. and Montgomery, D.C. (2011), Minitab Companion to Design and Analysis of
Experiments, 7th edition, Wiley, Hoboken, NJ.
9.	Montgomery, D. C., Runger, G. C. and Hubele, N. F. (2011), Engineering Statistics, 5th edition, John
Wiley & Sons, New York (1st edition, 1998, 2nd edition 2001, 3rd edition 2004, 4th edition 2007).
10.	Myers, R. H., Montgomery, D. C., Vining, G. G. and Robinson, T.J. (2010), Generalized Linear Models
with Applications in Engineering and the Sciences 2nd edition, John Wiley & Sons, New York (Probability
and Statistics Series; 1 edition 2002).
11.	Hines, W. W., Montgomery, D.C., Goldsman, D. M. and Borror, C. M. (2003), Probability and Statistics
in Engineering, 4th edition, John Wiley & Sons, New York (1st edition 1972, 2nd edition 1980, 3rd edition
1990).
12.	Montgomery, D. C., Johnson, L. A. and Gardiner, J.S. (1990), Forecasting and Time Series Analysis, 2nd
edition, McGraw-Hill, New York. (1st edition 1976).
13.	Johnson, L. A. and Montgomery, D.C, (1974), Operations Research in Production Planning, Scheduling,
and Inventory Control, John Wiley & Sons, New York.
Research Books and Edited Volumes
1.	Coleman, S., Greenfield, T., Stewardson, D., and Montgomery, D. C. (editors) (2008), Statistical Practice
in Business and Industry, Wiley, Hoboken, NJ.
2.	Burdick, R. K., Borror, C. M., and Montgomery, D. C. (2005), Design and Analysis of Gauge R&R
Studies: Making Decisions with Confidence Intervals inRandom and Mixed ANOVA Models, ASA-SIAM
Series on Statistics and Applied Probability, SIAM, Philadelphia, PA, and ASA, Alexandria, VA.
3.	Calado, V. and Montgomery, D.C. (2003), Planejamendo de Experimentos Usando o Statistica, E-Papers
Servifjos Editorials Ltda., Rio de Janeiro, Brazil.
4.	Beichelt, F. E. and Montgomery, D.C. (editors) (2003), Wahrscheinlichkeitstheorie, Stochastische
Prozesse, Mathematische Statistik, B. G. Teubner Verlag, Weisbaden.
5.	Keats, J. B. and Montgomery, D.C. (editors) (1996), Statistical Applications in Process Control, Marcel
Dekker, New York.
6.	Keats, J. B. and Montgomery, D.C. (editors) (1991), Statistical Process Control in Manufacturing,
Marcel Dekker, New York.
7.	George, M. L., Gooch, J. and Montgomery, D.C. (1986), America Can Compete, SMU Press, Dallas, TX.
Refereed Journal Publications
267. Stone, B.B., Montgomery, D.C., Silvestrini, R.T. and Jones B. (2017), "No-confounding designs
with 24 runs for 7 - 12 factors", International Journal of Experimental Design and Process
Optimization, Vol. 5, No. 3, pp. 151-171.
RTI International
Peer Review of EPA's RSE Report
40

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
266. Montgomery, D.C. and Borror, C.M. (2017), "Systems for modern quality and business
improvement", Quality Technology and Quantitative Management, Vol. 14, No. 4, pp. 343-352.
265. Jones, B. and Montgomery, D.C. (2017), "Partial Replication of Small Two-Level Factorial
Designs", Quality Engineering, Vol. 29, No. 3, pp. 190-195.
264. Jones, B., Schoen, E.D., and Montgomery, D.C. (2016), "A Comparison of Two-level Designs to
Estimate All Main Effects and Two-Factor Interactions", Quality Engineering, Vol. 28, No. 4, pp.
369-380.
263. Mancenido, M., Pan, R., and Montgomery, D.C. (2016),"Analysis of Subjective Ordinal
Responses in Mixture Experiments", Journal of Quality Technology1', Vol. 48, No. 2, pp. 196-
208.
262. Lu, Y., Steptoe, M., Burke, S., Wang, H., Tsai, J.Y., Davulcu, H., Montgomery, D., Corman, S.R.,
and Maciejewski, R. (2016), "Exploring Evolving Media Discourse Through Event Cueing",
IEEE Transactions on Visualization and Computer Graphics, Vol 22(1), pp.220-229.
261. Kennedy, K., Silvestrini, R.T., Montgomery, D.C., and Jones, B. (2015), "Prediction Variance
Properties of Bridge Designs", International Journal of Experimental Design and Process
Optimisation", Vol. 4, pp. 234-255.
260. Krishnamoorthy, A., Montgomery, D.C., Jones, B., and Borror, C.M. (2015), "Analyzing No-
confounding Designs using the Dantzig Selector", International Journal of Experimental Design
and Process Optimisation", Vol. 4, pp. 183-205
259. Jones, B., Silvestrini, R.T., Montgomery, D.C. and Steinberg, D.M. (2015), "Bridge Designs for
Modeling Systems with Low Noise", Technometrics, Vol. 57, No. 2, pp. 155-163
258. Montgomery, D.C. (2015), "Discussion of 'The Case Against Normal Probability Plots of Effects'
by R.V. Lenth", Journal of Quality Technology, Vol. 47, No. 2, pp. 105-106.
257. Jones, B., Shinde, S.M., and Montgomery, D.C. (2015), "Alternatives to Resolution III Regular
Fractional Factorial Designs for 9-14 Factors in 16 Runs", Applied Stochastic Models in Business
and Industry, Vol. 31, pp. 50-58.
256. Krueger, D.C. and Montgomery, D.C. (2014), "Modeling and Analyzing Semiconductor Yield
with Generalized Linear Mixed Models", Applied Stochastic Models in Business and Industry,
Vol. 30, No. 6, pp. 691 -707.
255. Shinde, S.M., Montgomery, D.C., and Jones, B. (2014), "Projection Properties of No-
Confounding Designs for Six, Seven, and Eight Factors in 16 Runs", International Journal of
Experimental Design and Process Optimization, Vol. 4, No. 1, pp. 1-26.
254. Park, Y.J., Pan, R., Borror, C.M., Montgomery, D.C., and Lee, G.B. (2014), "Simultaneous
Improvement of Energy Efficiency and Product Quality in PCB Lamination Process",
International Journal of Precision Engineering and Manufacturing - Green Technology, Vol. 1,
No. 3, pp. 247-256.
RTI International
Peer Review of EPA's RSE Report
41

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
253. Stone, B. B., Montgomery, D.C., Hassler, E., and Silvestrini, R.T. (2014), "An Expected Cost
Methodology for Screening Design Selection", Quality Engineering, Vol. 26, No. 2, pp. 139-153.
252. Adibi, A., Montgomery, D.C. and Borror, C.M. (2014), "Phase II Monitoring of Linear Profiles
using a P-value Approach", International Journal of Quality Engineering and Technology,
Special Issue on Monitoring and Control, Vol 4, No. 3, p. 97- 106.
251. Laungrungrong, B., Borror, C.M., and Montgomery, D.C. (2014), "A One-Sided MEWMA
Control Chart for Poisson-Distributed Data", International Journal of Data Analysis Techniques
and Strategies, Vol. 6, No. 1, pp.15-42.
250 Adibi, A., Borror, C.M. and Montgomery, D.C. (2014), "A P-value Approach for Phase II
Monitoring of multivariate Profiles", International Journal of Quality Engineering and
Technology, Special Issue on Monitoring and Control, Vol 4, No. 3, p. 133-143.
249. Montgomery, D.C. (2014), "Stu Hunter's Contributions to Experimental Design and Quality
Engineering", Quality Engineering, Vol. 26, No.l, pp. 5-15.
248. Woodall, W.H and Montgomery, D.C. (2014), "Some Current Trends in the Theory and Application
of Statistical Process Monitoring", Journal of Quality Technology, Vol. 46, No. 1, pp. 78-94.
247. Silvestrini, R.T., Montgomery, D.C. and Jones, B. (2013), "Comparing Computer Experiments for
the Gaussian Process Model Using Integrated Prediction Variance", Quality Engineering, Vol. 25,
No. 2, pp. 164-174.
246. Rigdon, S.E., Englert, B. R., Lawson, I.A., Borror, C.M., Montgomery, D.C. and Pan, R. (2013),
"Experiments for Reliability Achievement", Quality Engineering, Vol. 25, No. 1, p. 54-72.
245. Abelson, R., Lane, J.K., Rodriguez, R., Johnston, P., Angjeli, E., Ousler, G., and Montgomery,
D.C. (2012), "A single-center study evaluating the effect of the controlled adverse environment
(CAEsm) model on tear film stability," Clinical Ophthalmology, Vol. 6, pp. 1865-1872.
244. Duarte, B., Montgomery, D.C., Fowler, J., and Konopka, J. (2012), "Deploying Lean Six Sigma in
a Global Enterprise - Project Identification", International Journal of Lean Six Sigma, Vol. 3, No.
3, pp. 187-205.
243. Capehart, E.R., Keha, A.B., Kulahci, M., and Montgomery, D.C. (2012), "Generating Blocked
Fractional Factorial Split-Plot Designs using Integer Programming", International Journal of
Experimental Design and Process Optimization, Vol. 3, No. 2, pp. 111-132.
242. Rodriguez-Sifuentes, M., Montgomery, D.C., and Borror, C.M. (2012), "Prediction Variance
Performance of Combined Array Designs", International Journal of Experimental Design and
Process Optimization, Vol. 3, No. 1, pp. 1-32.
241. Abelson, R., Lane, K.J., Rodriguez, J., Johnston, P., Angjeli, E., Ousler, G., and Montgomery,
D.C. (2012), "Validation and Verification of the OPI 2.0 System", Clinical Ophthalmology, Vol.
6, pp. 613-622.
240. Anderson-Cook, C.M., Lu, L., Clark, G., DeHart, S.P., Hoerl, R., Jones, B., MacKay, J.,
Montgomery, D.C., Parker, P.A., Simpson, J.R., Snee, R.D., Steiner, S.H., VanMullekom, J.,
RTI International
Peer Review of EPA's RSE Report
42

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
Vining, G.G., and Wilson, A.G. (2012), "Statistical Engineering - Forming the Foundations",
Quality Engineering, Vol. 24, No. 2, pp. 110-132.
239. Anderson-Cook, C.M., Lu, L., Clark, G., DeHart, S.P., Hoerl, R., Jones, B., MacKay, J.,
Montgomery, D.C., Parker, P.A., Simpson, J.R., Snee, R.D., Steiner, S.H., VanMullekom, J.,
Vining, G.G., and Wilson, A.G. (2012), "Statistical Engineering - Roles for Statisticians and the
Path Forward", Quality Engineering,Vol. 24, No. 2, pp. 110-132.
238. Antony, J., Bhuller, A.S., Kumar, M., Mendibil, K., and Montgomery, D.C. (2012), "Application
of Six Sigma DMAIC Methodology in a Transactional Environment", International Journal of
Quality and Reliability Management, Vol. 29, no. 1, pp. 31 - 53.
237. Cho, T.-Y., Montgomery, D.C. and Borror, C.M. (2012),"A Case Study Involving Mixture-
Process Variable Experiments within a Split-Plot Structure", Quality Engineering, Vol. 24, No. 1,
pp. 80-93.
236. Johnson, R.T., Hutto, G.T., Simpson, J.R. and Montgomery, D.C. (2012), "Designed Experiments
for the Defense Community", Quality Engineering, Vol. 24, No. 1, pp. 60-79
235. Antony, J., Coleman, S., Montgomery, D.C., Anderson, M.J., and Silverstrini, R.T. (2011),
"Design of Experiments for Non-manufacturing Processes: Benefits, Challenges and Some
Examples", Journal ofEngineering Manufacture, Vol. 225, No. 11, pp. 2088-2095.
234. Broyles, J.R., Cochran, J.K., and Montgomery, D.C. (2011), "A Markov Decision Process to
Dynamically Match Hospital Inpatient Staffing to Demand, HE Transactions on Healthcare
Systems Engineering, Vol. 1, No. 2, pp. 116-130
233. Kumar, N., Mastrangelo, C. and Montgomery, D.C. (2011), "Hierarchial Modeling using
Generalized Linear Models", Quality and Reliability Engineering International, Vol. 27, No. 6,
pp.835-842.
232. Borror, C.M., Beechy, T., Shunk, D., Gish, M., and Montgomery, D.C. (2011), "TASER's
Roadmap to Quality", The Quality Management Forum, Vol. 37, No. 3, pp. 13-18.
231. Abelson, R., Lane, K.J., Angjeli, E., Johnston, P., Ousler, G., and Montgomery, D.C. (2011),
"Measurement of Ocular Surface Protection Under Natural Blink Conditions", Clinical
Ophthalmology, Vol. 5, pp. 1349-1357.
230. Laungrungrong, B., Borror, C.M. and Montgomery, D.C. (2011), "EWMA Control Charts for
Multivariate Poisson-Distributed Data", International Journal of Quality Engineering and
Technology, Vol. 2, No. 3, pp. 185-211.
229. Monroe, E.M., Pan, R., Anderson-Cook, C.M., Montgomery, D.C. and Borror, C.M. (2011), "A
Generalized Linear Model Approach to Designing Accelerated Life Test Experiments", Quality
and Reliability Engineering International, Vol. 27, No. 4, pp. 595-607.
228. Johnson, R.T., Montgomery, D.C. and Jones, B.A. (2011), "An Expository Paper on Optimal
Design", Quality Engineering, Vol. 23, No. 3, pp. 276-301.
RTI International
Peer Review of EPA's RSE Report
43

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
227. Cho, T.-Y., Borror, C.M. and Montgomery, D.C. (2011), "Mixture-Process Variable Experiments
Including Control and Noise Variables Within a Split-Plot Structure", International Journal of
Quality Engineering and Technology, Vol. 2, No. 1, pp. 1-28.
226. Krueger, D.C., Montgomery, D.C. and Mastrangelo, C.M. (2011), "Application of Generalized
Linear Models to Predict Semiconductor Yield Using Defect Metrology Data", IEEE
Transactions on Semiconductor Manufacturing, Vol. 24, No. 1, pp. 44-58.
225. Capehart, S.R., Keha, A., Kulahci, M. and Montgomery, D.C. (2011), "Designing Fractional
Factorial Split-plot Experiments Using Integer Programming", International Journal of
Experimental Design and Process Optimisation, Vol. 2, pp. 34-57.
224. Johnson, R.T., Montgomery, D.C. and Jones, B. (2011), "An Empirical Study of the Prediction
Performance of Space-filling Designs", International Journal of Experimental Design and
Process Optimisation, Vol. 2, pp. 1-18.
223. Shinde, S.M., Orozco, C., Brengues, M., Lenigk, R., Montgomery, D.C. and Zenhausern, F.
(2011), "Optimization of a Microfluidic Mixing Process for Gene Expression-Based Bio-
Dosimetry", Quality Engineering, Vol 23, pp. 59-70.
222. Broyles, J. R., Cochran, J. K. and Montgomery, D. C. (2010), "A Statistical Markov Chain
Approximation of Transient Hospital Inpatient Inventory", European Journal of Operational
Research, Vol. 207, No. 3, pp. 1645-1657.
221. Johnson, R.T. and Montgomery, D.C. (2010), "Designing Experiments for Nonlinear Models -
An Introduction", Quality and Reliability Engineering International, Vol. 26, No. 5, pp. 431-441.
220. Chen, J.Y., Pfund, M.E., Fowler, J.W., Montgomery, D.C. and Callarman, T.E. (2010), "Robust
Scaling Parameters for Composite Dispatching Rules", HE Transactions, Vol. 42, No. 11, pp.
842-853.
219. Jones, B. and Montgomery, D.C. (2010), "Alternatives to Resolution IV Screening Designs in 16
Runs", International Journal of Experimental Design and Process Optimisation, Vol. 1, No. 4,
pp. 285-295.
218. Gupta, S., Kulahci, M., Montgomery, D.C. and Borror, C.M (2010), "Analysis of Signal-
Response Systems using Generalized Linear Mixed Models", Quality and Reliability Engineering
International, Vol. 26, No. 4, pp. 375-385.
217. Montgomery, D.C. (2010), "A Modern Framework for Achieving Enterprise Excellence",
International Journal of Lean Six Sigma, Vol. 1, No. 1, pp. 56-65.
216. Laungrungrong , B., Mobasher, B., Montgomery, D.C. and Borror, C.M. (2010), "Hybrid Control
Charts for Active Control and Monitoring of Concrete Strength", Journal of Materials in Civil
Engineering, Vol. 2, No. 1 (January), pp. 77-87.
215. Monroe, E.M., Pan, R., Anderson-Cook, C.M., Montgomery, D.C. and Borror, C.M. (2010),
"Sensitivity Analysis of Optimal Designs for Accelerated life Testing", Journal of Quality
Technology, Vol. 42, No. 2, pp. 121-135.
214. Johnson, R.T., Montgomery, D.C., Jones, B. and Parker, P. A. (2010), "Comparing Computer
Experiments for Fitting High-Order Polynomial Models", Journal of Quality Technology, Vol. 42,
No. 1, pp. 86-102.
RTI International
Peer Review of EPA's RSE Report
44

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
213. Rodriguez, M., Jones, B., Borror, C.M. and Montgomery, D.C. (2010), "Generating and
Assessing Exact G-Optimal Designs", Journal of Quality Technology, Vol. 42, No. 1, pp. 3-29.
212. Hoskins, D.S., Colbourn, C.J. and Montgomery, D.C. (2009), "D-optimal Designs with
Interaction Coverage", Journal of Statistical Theory and Practice, Vol. 3, No. 4, pp. 817-830.
211. Cho, T-Y., Borror, C.M. and Montgomery, D.C. (2009) "Graphical Evaluation of Mixture-Process
Variable Designs Within a Split-Plot Structure', International Journal of Quality Engineering and
Technology, Vol. 1, No. 1, pp. 2-26.
210. Rodriguez, M., Montgomery, D.C. and Borror, C.M. (2009), "Generating Experimental Designs
Involving Control and Noise Variables using Genetic Algorithms", Quality and Reliability
Engineering International", Vol. 25, No. 8, pp. 1045-1065.
209. Chung, P. J., Goldfarb, H.B., Montgomery, D.C. and Borror, C.M. (2009), "Optimal Designs for
Mixture-Process Experiments Involving Continuous and Categorical Noise Variables", Quality
Technology and Quantitative Management, Vol. 6, No. 4, pp. 451-470.
208. Li, J., Liang, L., Borror, C.M., Anderson-Cook, C.M. and Montgomery, D.C. (2009), "Graphical
Summaries to Compare Prediction Variance Performance for Variations of the Central Composite
Design for 6 to 10 Factors", Quality Technology and Quantitative Management, Vol. 6, No. 4, pp.
433-449.
207. Johnson, R.T. and Montgomery, D.C. (2009), "Choice of Second-Order Response Surface Designs
for Logistic and Poisson Regression Models", International Journal of Experimental Design and
Process Optimization, Vol. 1, No. 1, pp. 2-23.
206. Montgomery, D.C. (2009), "A Conversation with Stu Hunter", Quality Engineering, Vol. 21, No.
3, pp. 233-240.
205. Almimi, A.A., Kulahci, M. and Montgomery, D.C. (2009), "Checking the Adequacy of Fit of
Models from Split-Plot Designs", Journal of Quality Technology, Vol. 41, No. 3, pp. 272-284.
204. Johnson, R.T., Parker, P.A., Montgomery, D.C., Cutler, A.D., Danehy, P.M. and Rhew, R.D.
(2009), "Design Strategies for the Response Surface Models for the Study of Supersonic
Combustion", Quality and Reliability Engineering International, Vol. 25, pp. 365-377.
203. Anderson-Cook, C.M., Borror, C.M. and Montgomery, D.C. (2009), "Response Surface Design
Evaluation and Comparison" (with discussion), Journal of Statistical Planning and Inference, Vol.
139, pp. 629-674.
202. Montgomery, D. C. and Woodall, W.H. (2008), "An Overview of Six Sigma", International
Statistical Review, Vol. 76, No. 3, pp. 329-346.
201. Steinberg, D.M., Bisgaard, S., Doganaksoy, N., Fisher, N., Gunter, B., Hahn, G., Keller-McNulty,
S., Kettenring, J., Meeker, W.G., Montgomery, D.C. and Wu, C.F.J. (2008), "The Future of
Industrial Statistics: A Panel Discussion", Technometrics, Vol. 50, No. 2, pp. 103 - 127.
200. Kumar, M., Antony, J., Madu, C.N., Montgomery, D.C., and Park, S.H. (2008), "Common Myths
of Six Sigma Demystified", International Journal of Quality & Reliability Management, Vol. 25
No. 8, pp. 878-895.
199. Boushell, T.G, Fowler, J.W., Keha, A., Knutson, K., and Montgomery, D.C. (2008), "Evaluation
of Heuristics for a Class-constrained Lot-to-Order Matching Problem in Semiconductor
Manufacturing", International Journal of Production Research, Vol. 46, No. 12, pp. 4143-4166.
RTI International	Peer Review of EPA's RSE Report	45

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
198. Almimi, A. A., Kulahci, M., and Montgomery, D.C. (2008), "Follow-Up Designs to Resolve
Confounding in Split-Plot Experiments", Journal of Quality Technology, Vol. 40, No. 2, pp. 154-
166.
197. Montgomery, D. C. (2008), "Discussion of 'Must a Process Be in Statistical Control Before
Conducting Designed Experiments?'", by Soren Bisgaard, Quality Engineering, Vol. 20, pp. 165-
168.
196. Elias, R.J., Montgomery, D.C., Low, S.A. and Kulahci, M. (2008), "Demand Signal Modeling: A
Short-range Panel Forecasting Algorithm for Semiconductor Firm Device-level Demand",
European Journal of Industrial Engineering, Vol. 2, No. 3, pp. 253-278.
195. Almimi, A. A., Kulahci, M, and Montgomery, D.C. (2008), "Estimation of Missing Observations in
Two-level Split-plot Designs", Quality and Reliability Engineering International, Vol. 24, pp. 127-
152.
194. Montgomery, D. C. (2008), "Discussion of 'An Overview of the Shainin™ System for Quality
Improvement'", by Stefan H. Steiner, R. JockMacKay, and John S. Ramberg, Quality
Engineering, Vol. 20, pp. 36-37.
193. Jearkpaporn, D., Borror, C.M., Runger, G.C., and Montgomery, D.C. (2007), "Process Monitoring
for Mean Shifts for Multiple Stage Processes", International Journal of Production Research,
Vol. 45, No. 23, pp. 5547-5570.
192. Lawson, C. and Montgomery, D.C. (2007), "A Logistic Regression Modeling Approach to
Business Opportunity Assessment", International Journal of Six Sigma and Competitive
Advantage, Vol. 3, No. 2, pp. 120-136.
191. Perry, L.A., Montgomery, D.C., and Fowler, J.W. (2007), "A Partition Experimental Design for a
Sequential Process with a Large Number of Variables", Quality and Reliability Engineering
International, Vol. 23, pp. 555-564
190. Chung, P. J., Goldfarb, H. B., and Montgomery, D. C. (2007), "Optimal Designs for Mixture-
Process Experiments with Control and Noise Factors", Journal of Quality Technology, Vol. 39,
No. 3, pp. 179-190.
189. Holcomb, D.R., Montgomery, D. C., and Carlyle, W.M. (2007), "The use of Supersaturated
Experiments in Turbine Engine Development", Quality Engineering, Vol. 19, No. 1, pp. 17-27.
188. Lawson, C. and Montgomery, D. C. (2006), "Logistic Regression Analysis of Customer
Satisfaction Data", Quality and Reliability Engineering International, Vol. 22, No. 8, pp. 971-
984.
187. Kumar, N., Kennedy, K., Gildersleeve, K., Abelson, R., Mastrangelo, C. M., and Montgomery, D.
C. (2006), "A Review of Yield Modeling Techniques for Semiconductor Manufacturing",
International Journal of Production Research, Vol. 44, No. 23, 5019-5036.
186. Kowalski, S. M., Vining, G. G., Montgomery, D. C., and Borror, C. M. (2006), "Modifying a
Central Composite Design to Model the Process Mean and Variance when there are Hard-to-
Change Factors", Journal of the Royal Statistical Society C (Applied Statistics), Vol. 55, Part 5,
pp. 615-630.
185. Skinner, K. R., Runger, G. C., and Montgomery, D. C. (2006), "Process Monitoring for Multiple
Count Data Using a Deleted-)7 Statistic", Quality Technology and Quantitative Management, Vol.
2, No. 3, pp. 247-262.
184. Vadde, K. K., Syrotiuk, V. R., and Montgomery, D. C. (2006), "Optimizing Protocol Interaction
using Response Surface Methodology," IEEE Transactions on Mobile Computing, Vol. 5, No. 6,
pp. 627-639.
RTI International
Peer Review of EPA's RSE Report
46

-------
Douglas Montgomery	Peer Reviewer Resume	Last Updated 9/5/17
183. Gupta, S., Montgomery, D. C., and Woodall, W. H. (2006), "Performance Evaluation of Two
Methods for Online Monitoring of Linear Calibration Profiles", International Journal of
Production Research, Vol. 44, No. 10, pp. 1927-1942.
182. Elias, R.J., Montgomery, D.C. and Kulahci, M. (2006), "An Overview of Short-Term Statiatical
Forecasting Methods", International Journal of Management Science and Engineering
Management, Vol. 1, No. 1, pp. 17-36.
181. Park, Y.-J., Montgomery, D.C., Fowler, J. W. and Borror, C.M. (2006), "Cost-constrained G-
efficient Response Surface Designs for Cuboidal Regions", Quality and Reliability Engineering
International, Vol. 22, No. 2, pp. 121-139.
180. Robinson, T. J., Wulff, S. S., Montgomery, D. C. and Khuri, A. I. (2006), "Robust Parameter
Design using Generalized Linear Models", Journal of Quality Technology, Vol. 38, No. 1, pp. 65-
75.
179. Burdick, R. K., Park, Y.-J., Montgomery, D. C. and Borror, C. M. (2005), "Confidence Intervals
for Misclassification Rates in a Gauge R&R Study", Journal of Quality Technology, Vol. 37, No.
4, pp. 294-303.
178. Houston, D., Ferreira, S. and Montgomery, D. C. (2005), "Using Unreplicated 2k'p Designs for
Characterizing Moderately Dimensioned Deterministic Computer Models", Quality and
Reliability Engineering International, Vol. 21, No. 8, pp. 809-824.
177. Park, Y.-J., Richardson, D. E., Montgomery, D. C., Ozol-Godfrey, A., Borror, C. M. and
Anderson-Cook, C. M. (2005), "Prediction Variance Properties of Second-Order Designs for
Cuboidal Regions", Journal of Quality Technology, Vol. 37, No. 4, pp. 253-266.
176. Ozol-Godfrey, A., Anderson-Cook, C. M. and Montgomery, D. C. (2005), "Fraction of Design
Space Plots for Examining Model Robustness", Journal of Quality Technology, Vol. 37, No. 3,
pp. 223-235.
175. Mason, S.J., Fowler, J.W., Carlyle, W.M. and Montgomery, D.C. (2005), "Heuristics for
Minimizing Total Weighted Tardiness in Complex Job Shops", International Journal of
Production Research, Vol. 43, No. 10, pp. 1943-1963.
174. Montgomery, D.C., Burdick, R.K., Lawson, C.A., Molnau, W.E., Zenzen, F., Jennings, C.L.,
Shah, H.K., Sebert, D.M., Bowser, M.D. andHolcomb, D.R. (2005), "A University-Based Six-
Sigma Program", Quality and Reliability Engineering International, Vol. 21, No. 3, pp. 243-248.
173. Vining, G. G., Kowalski, S. M. and Montgomery, D. C. (2005), "Response Surface Designs
Within a Split-Plot Structure", Journal of Quality Technology, Vol. 37, No. 2, pp. 115-129.
172. Jearkpaporn, D., Montgomery, D. C., Runger, G. C. and Borror, C. M. (2005), "Model-Based
Process Monitoring using Robust Generalized Linear Models", International Journal of
Production Research, Vol. 43, No. 7, pp. 1337-1354.
171. Montgomery, D. C., Myers, R. H., Carter, W. H. Jr. and Vining, G. G. (2005), "The Hierarchy
Principal in Designed Industrial Experiments", Quality and Reliability Engineering International,
Vol. 21, No. 2, pp. 197-201.
170. Suryanarayanan, S., Montgomery, D. C., and Heydt, G. T. (2005), "Considerations for
Implementing Tag Schedules in Transmission Circuits", IEEE Transactions on Power Systems,
Vol. 20, No. l,pp. 523-524.
169. Kowalski, S. M., Borror, C. M., and Montgomery, D. C. (2005), "A Modified Path of Steepest
Ascent for Split-Plot Experiments", Journal of Quality Technology, Vol. 37, No. 1, pp. 75-83.
168. Goldfarb, H. B., Borror, C. M., Montgomery, D. C., and Anderson-Cook, C. M. (2005), "Using
Genetic Algorithms to Generate Mixture-Process Experimental Designs Involving Control and
Noise Variables", Journal of Quality Technology, Vol. 37, No. 1, pp. 60-74.
RTI International
Peer Review of EPA's RSE Report
47

-------
Douglas Montgomery	Peer Reviewer Resume	Last Updated 9/5/17
167. Skinner, K. M., Montgomery, D. C., and Runger, G. C. (2004), "Generalized Linear Model-Based
Control Charts for Discrete Semiconductor Data", Quality and Reliability Engineering
International, Vol. 20, pp. 777-786.
166. Anderson-Cook, C. M., Goldfarb, H. B., Borror, C. M., Montgomery, D. C., Canter, K. K., and
Twist, J. N. (2004), "Mixture and Mixture-Process Variable Experiments for Pharmaceutical
Applications", Pharmaceutical Statistics, Vol. 3, pp. 247-260.
165. Drain, D., Carlyle, W. M., Montgomery, D. C., Borror, C. M., and Anderson-Cook, C. M. (2004),
"A Genetic Algorithm Hybrid for Constructing Optimal Response Surface Designs", Quality and
Reliability Engineering International, Vol. 20, pp. 637-650.
164. Schaefer, L. A., Montgomery, D. C., and Wolfe, P. M. (2004), "A Flow Metamodel for Delivery
Agents Over a Spatial Area", HE Transactions, Vol. 36, pp. 1055-1065.
163. Woodall, W. H., Spitzner, D. J., Montgomery, D. C., and Gupta, S. (2004), "Using Control Charts
to Monitor Process and Product Profiles", Journal of Quality Technology, Vol. 36, No. 3, pp. 309-
320.
162. Heredia-Langner, A., Montgomery, D. C., Carlyle, W. M., and Borror, C. M. (2004), "Model-
Robust Optimal Designs: A Genetic Algorithm Approach", Journal of Quality Technology, Vol.
36, No. 3, pp. 263-279.
161. Goldfarb, H. B., Borror, C. M., Montgomery, D. C., and Anderson-Cook, C. M. (2004),
"Evaluating Mixture-Process Designs with Control and Noise Variables", Journal of Quality
Technology, Vol. 36, No. 3, pp. 245-262.
160. Robinson, T. J., Myers, R. H., and Montgomery, D. C. (2004), "Analysis Considerations in
Industrial Split-Plot Experiments with Non-Normal Responses", Journal of Quality Technology,
Vol. 36, No. 2, pp. 180-192.
159. Goldfarb, H. B., Anderson-Cook, C. M., Borror, C. M., and Montgomery, D. C. (2004), "Fraction
of Design Space Plots to Assess the Prediction Capability of Mixture and Mixture-Process
Designs", Journal of Quality Technology Vol. 36, No. 2, pp. 169-179.
158. Shah, H. K., Montgomery, D. C., and Carlyle, W. M. (2004), "Response Surface Modeling and
Optimization in Multiresponse Experiments using Seemingly Unrelated Regressions", Quality
Engineering, Vol. 16, No. 3, pp. 387-397.
157. Myers, R. H., Montgomery, D. C., Vining, G. G., Borror, C. M., and Kowalski, S. M. (2004),
"Response Surface Methodology: A Retrospective and Literature Survey", Journal of Quality
Technology, Vol. 36, No. 1, pp. 53-77.
156. Goldfarb, H. B., Borror, C. M., Montgomery, D. C., and Anderson-Cook, C. M. (2004), "Three-
Dimensional Variance Dispersion Graphs for Mixture-Process Experiments", Journal of Quality
Technology, Vol. 36, No. 1, pp. 109-124.
155. Jearkpaporn, D., Montgomery, D. C., Runger, G. C., and Borror, C. M. (2003), "Process
Monitoring for Correlated Gamma Distributed Data Using Generalized Linear Model Based
Control Charts", Quality and Reliability Engineering International Vol. 19, No. 6, pp. 477-491.
154. Burdick, R. K., Borror, C. M., and Montgomery, D. C. (2003), "A Review of Methods for
Measurement Systems Capability Analysis", Journal of Quality Technology, Vol. 35, No. 4, pp.
342-354.
153. Goldfarb, H. B., Borror, C. M., and Montgomery, D. C. (2003), "Mixture-Process Variable
Experiments with Noise Variables", Journal of Quality Technology, Vol. 35, No. 4, pp. 393-405.
152. Borror, C. M., Keats, J. B., and Montgomery, D. C. (2003), "Robustness of the Time Between
Events CUSUM", International Journal of Production Research, Vol. 41, No. a5, pp. 3435-3444.
RTI International
Peer Review of EPA's RSE Report
48

-------
Douglas Montgomery	Peer Reviewer Resume	Last Updated 9/5/17
151. Wisnowski, J.W., Simpson, J.R., Montgomery, D.C., and Runger, G. C. (2003). "Resampling
Methods for Variable Selection in Robust Regression," Computational Statistics and Data
Analysis, Vol. 43, No. 3, pp. 341-355.
150. Skinner, K. R., Montgomery, D. C., and Runger, G. C. (2003), "Process Monitoring for Multiple
Count Data Using Generalized Linear Model Based Control Charts", International Journal of
Production Research, Vol. 41, No. 6, pp. 1167-1180.
149. Holcomb, D. R., Montgomery, D. C., and Carlyle, W. M. (2003), "Analysis of Supersaturated
Designs", Journal of Quality Technology, Vol. 35, No. 1, pp. 13-27.
148. Heredia-Langner, A., Carlyle, W. M., Montgomery, D. C., Borror, C. M., and Runger, G. C.
(2003), "Genetic Algorithms for the Construction of D-Optimal Designs", Journal of Quality
Technology, Vol. 35, No. 1, pp. 28-46.
147. Lanning, J. W., Montgomery, D. C., and Runger, G. C. (2002-03), "Monitoring a Multiple Stream
Filling Operation Using Fractional Samples", Quality Engineering, Vol. 15, No. 2, pp. 183-195.
146. Skinner, K. R., Montgomery, D. C., Runger, G. C., Fowler, J. W., McCarville, D. R., Rhoads, T.
R., and Stanley, J. D. (2002), "Multivariate Statistical Methods for Modeling and Analysis of
Wafer Probe Test Data", IEEE Transactions on Semiconductor Manufacturing, Vol. 15, No. 4,
pp. 523-530.
145. Perry, L. A., Montgomery, D. C., and Fowler, J. W. (2002), "Partition Experimental Designs for
Sequential Processes: Part II - Second-Order Models", Quality and Reliability Engineering
International, Vol. 18, No. 5, pp. 373-382.
144. Somerville, S. E., Montgomery, D. C., and Runger, G. C. (2002), "Filtering and Smoothing
Methods for Mixed Particle Count Distributions", International Journal of Production Research,
Vol. 40, No. 13, pp. 2991-3013.
143. Wisnowski, J. W., Simpson, J. R., and Montgomery, D. C. (2002) "An Improved Compound
Estimator for Robust Regression," Communications in Statistics: Simulation and Computation,
Vol. 31, No. 4, pp. 653-772.
142. Montgomery, D. C., Loredo, E. N., Jearkpaporn, D., and Testik, M. C. (2002), "Experimental
Designs for Constrained Regions", Quality Engineering, Vol. 14, No. 4, pp. 587-601.
141. Heredia-Langner, A., Montgomery, D. C., and Carlyle, W. M. (2002), "Solving a Multistage
Partial Inspection Problem using Genetic Algorithms", International Journal of Production
Research, Vol. 40, No. 8, pp. 1923-1940.
140. Wisnowski, J. W., Simpson, J. R., and Montgomery, D. C. (2002), "A Performance Study for
Multivariate Location and Shape Estimators", Quality and Reliability Engineering International,
Vol. 18, No. 2, pp. 117-129.
139. Borror, C. M., Heredia-Langner, A., and Montgomery, D. C. (2002), "Generalized Linear Models
in the Analysis of Industrial Experiments", Journal of Propagations in Probability and Statistics
{InternationalEdition), Vol. 2, No. 2, pp. 127-144.
138. Fowler, J. W., Phojanamongkolkij, N., Cochran, J. K., and Montgomery, D. C. (2002), "Optimal
Batching in a Wafer Fabrication Facility Using a Multi-product G/G/c Model with Batch
Processing", International Journal of Production Research, Vol. 40, No. 2, pp. 275-292.
137. Borror, C. M., Montgomery, D. C., and Myers, R. H. (2002), "Evaluation of Statistical Designs
for Experiments Involving Noise Variables", Journal of Quality Technology, Vol. 34, No. 1, pp.
54-70.
136. Canter, K. G., Kennedy, D. J., Keats, J. B., Montgomery, D. C., and Carlyle, W. M. (2002),
"Screening Stochastic Environmental Life Cycle Assessment Inventory Models", International
Journal of Life Cycle Assessment, Vol. 7, No. 1, pp. 18-26.
RTI International
Peer Review of EPA's RSE Report
49

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
135. Montgomery, D. C. (2001), "The Future of Industrial Statistics", Orion, Vol. 16, No. 1, pp. 1-21.
134. Perry, L. A., Montgomery, D. C., and Fowler, J. W. (2001), "Partition Experimental Designs for
Sequential Processes: Part I - First-Order Models", Quality and Reliability Engineering
International, Vol. 17, No. 6, pp. 429-438.
133. Houston, D. X., Ferreira, S., Collofello, J. S., Montgomery, D. C., Mackulak, G. T., and Shunk,
D. L (2001), "Behavorial Characterization: Finding and Using the Influential Factors in Software
Process Simulation Models", Journal of Systems and Software, Vol. 59, pp. 259-270.
132. Molnau, W. E., Runger, G. C., Montgomery, D. C., Skinner, K. R., Loredo, E. N., and Prabhu, S.
S. (2001), "A Program for ARL Calculation for Multivariate EWMA Control Charts", Journal of
Quality Technology, Vol. 33, No. 4, pp. 515-521.
131. Montgomery, D. C., Lawson, C., Molnau, W. E., and Elias, R. (2001), Invited discussion of "Six
Sigma Black Belts: What Do They Need to Know?", by R. W. Hoerl, Journal of Quality
Technology, Vol. 33, No. 4, pp. 407-409.
130. Montgomery, D. C. and Borror, C. M. (2001), Invited discussion of "Factor Screening and
Response Surface Exploration", by S.-W. Cheng and C. F. J. Wu, Statistica Sinica, Vol. 11, No. 3,
pp. 591-595.
129. Lewis, S. L., Montgomery, D. C. and Myers, R. H. (2001), "Confidence Interval Coverage for
Designed Experiments Analyzed with Generalized Linear Models", Journal of Quality
Technology, Vol. 33, No. 3, pp. 279-292.
128. Lewis, S. L., Montgomery, D. C. and Myers, R. H. (2001), "Examples of Designed Experiments
with Nonnormal Responses", Journal of Quality Technology, Vol. 33, No. 3, pp. 265-278.
127. Heredia-Langner, A., Montgomery, D. C., Runger, G. C., Borror, C. M., and Post, R. I. (2001),
"Performance of Customized Control Charts to Detect Process Disturbances", Quality and
Reliability Engineering International, Vol. 17, No. 3, pp. 205-218.
126. Wisnowski, J. W., Montgomery, D. C., and Simpson, J. R. (2001), "A Comparative Analysis of
Multiple Outlier Detection Procedures in the Linear Regression Model", Computational Statistics
and Data Analysis, Vol. 36, No. 3, pp 351-382.
125. Montgomery, D. C. (2001), "Opportunities and Challenges for Industrial Statisticians", Journal of
Applied Statistics, Vol. 28, Nos. 3&4, pp. 427-439.
124. Molnau, W. E., Montgomery, D. C., and Runger, G. C. (2001), "Statistically Constrained
Economic Design of the Multivariate Exponentially Weighted Moving Average Control Chart",
Quality and Reliability Engineering International, Vol. 17, No. 1, pp. 39-49.
123. Montgomery, D. C., Keats, J. B., Yatskievitch, M., and Messina, W. S. (2000), "Integrating
Statistical Process Monitoring with Feedforward Control", Quality and Reliability Engineering
International, Vol. 16, No. 6, pp. 515-525.
122. Woodall, W. H. and Montgomery, D. C. (2000-01), "Using Ranges to Estimate Variability",
Quality Engineering, Vol. 13, No. 2, pp. 211-217.
121. O'Neill, J. C., Borror, C. M., Eastman, P. Y., Fradkin, D. G., James, M. P., Marks, A. P., and
Montgomery, D. C. (2000), "Optimal Assignment of Samples to Treatments for Robust Design",
Quality and Reliability Engineering International, Vol. 16, No. 5, pp. 417-421.
120. Nelson, B. J., Montgomery, D. C., Elias, R. J., and Maass, E. (2000), "A Comparison of Several
Design Augmentation Strategies", Quality and Reliability Engineering International, Vol. 16, No.
5, pp. 435-449.
119. Heredia-Langner, A., Loredo, E. N., Montgomery, D. C., and Griffin, A. H. (2000), "Optimization
of a Bonded Leads Process using Statistically Designed Experiments", Robotics and Computer
Integrated Manufacturing, Vol. 16, No. 5, pp. 377-382.
RTI International	Peer Review of EPA's RSE Report	50

-------
Douglas Montgomery	Peer Reviewer Resume	Last Updated 9/5/17
118. Zimmer, L. S., Montgomery, D. C., and Runger, G. C. (2000), "Guidelines for the Application of
Adaptive Control Charting Schemes", International Journal of Production Research, Vol. 38, No.
9, pp. 1977-1992.
117. Borror, C. M. and Montgomery, D. C. (2000), "Mixed Resolution Designs as Alternatives to
Taguchi Inner/Outer Array Designs for Robust Design Problems", Quality and Reliability
Engineering International, Vol. 16, No. 2, pp. 117-127.
116. Carlyle, W. M., Montgomery, D. C., and Runger, G. C. (2000), "Optimization Problems and
Methods in Quality Control and Improvement" (with Discussion), Journal of Quality Technology,
Vol. 32, No. l,pp. 1-31.
115. Montgomery, D. C., Keats, J. B., Perry, L. A., Thompson, J. R., and Messina, W. S. (2000),
"Using Statistically Designed Experiments for Process Development and Improvement: An
Application in Electronics Manufacturing", Robotics and Computer Integrated Manufacturing,
Vol. 16, No. l,pp. 55-63.
114. Lewis, S. L., Montgomery, D. C., and Myers, R. H. (1999-2000), "The Analysis of Designed
Experiments with Nonnormal Responses", Quality Engineering, Vol. 12, No. 2, pp. 225-244.
113. Wisnowski, J. W., Runger, G. C., and Montgomery, D. C. (1999-2000), "Analyzing Data From
Designed Experiments: A Regression Tree Approach", Quality Engineering, Vol. 12, No. 2, pp.
185-198.
112. Woodall, W.H. and Montgomery, D. C. (1999), "Research Issues and Ideas in Statistical Process
Control", Journal of Quality Technology, Vol. 31, No. 4, pp. 376-386.
111. Silknitter, K. O., Wisnowski, J. W., and Montgomery, D. C. (1999), "The Analysis of Covariance:
A Useful Technique for Analyzing Quality Improvement Experiments", Quality and Reliability
Engineering International, Vol. 15, No. 4, pp. 303-316.
110. Runger, G. C., Keats, J. B., Montgomery, D. C., and Scranton, R. D. (1999), "Improving the
Performance of the Multivariate EWMA Control Chart", Quality and Reliability Engineering
International, Vol. 15, No. 3, pp. 161-166.
109. Borror, C. M., Montgomery, D. C., and Runger, G. C. (1999), "Robustness to Normality of the
EWMA Control Chart", Journal of Quality Technology, Vol. 31, No. 3, pp. 309-316.
108. Montgomery, D. C. (1999), "Experimental Design for Product and Process Design and
Development" (with commentary), Journal of the Royal Statistical Society Series D (The
Statistician), Vol. 48, Part 2, pp. 159-177.
107. Catlin, A. E., Bauer, K. W., Jr., Mykytka, E. W., and Montgomery, D. C. (1999), "System
Comparison Procedures for Automatic Target Recognition Systems", Naval Research Logistics,
Vol.46, No. 4, pp. 357-371.
106. Hauck, D. J., Runger, G. C., and Montgomery, D. C. (1999), "Multivariate Statistical Process
Monitoring and Diagnosis with Grouped Regression-Adjusted Variables", Communications in
Statistics: Simulation and Computation, Vol. 28, No. 2, pp. 309-328.
105. Montgomery, D. C. (1999), "Some Comments on Future Directions in RSM", Invited discussion
of papers by G. E. P. Box and R. H. Myers, Journal of Quality Technology, Vol. 31, No. 1, pp.
45-46.
104. Simpson, J. R. and Montgomery, D. C. (1998), "A Performance-Based Assessment of Robust
Regression Methods", Communications in Statistics: Simulation and Computation, Vol. 27, No. 4,
pp. 1031-1049.
103. Simpson, J. R. and Montgomery, D. C. (1998), "The Development and Evaluation of Alternative
Generalized M-estimation Techniques," Communications in Statistics: Simulation and
Computation, Vol. 27, No. 4, pp. 999-1018.
RTI International
Peer Review of EPA's RSE Report
51

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
102. Stanley, J. D., McCarville, D. R., and Montgomery, D. C. (1998), "A Time Series Approach for
Compensating Errors in Complex Gauge Systems", Quality and Reliability Engineering
International, Vol. 14, No. 4, pp. 273-280.
101. Sebert, D. M., Montgomery, D. C., and Rollier, D. A. (1998), "A Clustering Algorithm for
Identifying Multiple Outliers in Linear Regression", Computational Statistics and Data Analysis,
Vol. 27, pp. 461-484.
100. Simpson, J.R., and Montgomery, D.C. (1998), "A Robust Regression Technique Using
Compound Estimation", Naval Research Logistics, Vol. 45, No. 2, pp. 125-139.
99. Zimmer, L. S., Montgomery, D. C., and Runger, G.C. (1998), "Evaluation of a Three-State
Adaptive Sample Size X-Bar Control Chart", International Journal of Production Research, Vol.
36, No. 3, pp. 733-743.
98. Borror, C. M., Montgomery, D. C., and Runger, G. C. (1997), "Confidence Intervals on Variance
Components from Gauge Capability Studies", Quality and Reliability Engineering International,
Vol. 13, No. 6, pp. 361-369.
97. Kennedy, D. J., Montgomery, D. C., Rollier, D. A. and Keats, J. B. (1997), "Assessing Input Data
Uncertainty in Life Cycle Assessment Inventory Models", International Journal of Life Cycle
Assessment, Vol. 2, No. 4, pp. 229-239.
96. Bowles, M.L., and Montgomery, D.C. (1997-98), "How to Formulate the Ultimate Margarita: A
Tutorial on Experiments with Mixtures", Quality Engineering, Vol. 10, No. 2, pp. 239-253.
95. Montgomery, D. C., Borror, C. M., and Stanley, J. D. (1997-98), "Some Cautions in the Use of
Plackett-Burman Designs", Quality Engineering, Vol. 10, No. 2, pp. 371-381.
94. Andere-Rendon, J., Montgomery, D. C., and Rollier, D. A. (1997), "Design of Mixture
Experiments Using Bayesian D-Optimality", Journal of Quality Technology, Vol. 29, No. 4, pp.
451-463.
93. Schaub, D.A., Chu, K.-R., and Montgomery, D.C.(1997), "Optimizing Stereolithography
Throughput", Journal of Manufacturing Systems, Vol. 16, No. 4, pp. 290-303.
92. Myers, R. H., and Montgomery, D. C. (1997), "A Tutorial on Generalized Linear Models",
Journal of Quality Technology, Vol. 29, No. 3, pp. 274-291.
91. Runger, G.C., and Montgomery, D.C. (1997), "Multivariate and Univariate Process Control:
Geometry and Shift Directions", Quality and Reliability Engineering International, Vol. 13, No.
3, pp. 153-158.
90. Prabhu, S. S., Runger, G. C., and D. C. Montgomery (1997), "Selection of the Subgroup Size and
Sampling Interval for a CUSUM Control Chart", HE Transactions, Vol. 29, No. 6, pp. 451-457.
89. Schaub, D. A., and Montgomery, D. C. (1997), "Using Experimental Design to Optimize the
Stereolithography Process", Quality Engineering, Vol. 9, No. 4, pp. 575-585.
88. Montgomery, D. C., Keats, J. B., Fowler, J. W., Runger, G. C., and Rajavelu, G. (1997),
"Statistical Monitoring Techniques for Contamination Data", Journal of the Institute of
Environmental Sciences, Vol. XL, No. 2, pp. 23-30.
87. Prabhu, S. S., Montgomery, D. C., and G. C. Runger (1997), "Economic-Statistical Design of an
Adaptive x Control Chart", International Journal of Production Economics, Vol. 49, pp. 1-15.
86. Lawson, C., Keats, J. B., and Montgomery, D. C. (1997), "Comparison of Robust and Least
Squares Regression in Computer-Generated Probability Plots", IEEE Transactions on Reliability,
Vol. 46, No. l,pp. 108-115.
85. Montgomery, D. C., and Woodall, W. H. (1997) (editors), "A Discussion on Statistically-Based
Process Monitoring and Control", Journal of Quality Technology, Vol. 29, No. 2, pp. 121-162.
RTI International
Peer Review of EPA's RSE Report
52

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
84. Simpson, J.R., and Montgomery, D. C. (1996), "A Biased-Robust Regression Technique for the
Combined Outlier-Multicollinearity Problem", Journal of Statistical Computation and Simulation,
Vol. 56, pp. 1-22.
83. Kennedy, D. J., Montgomery, D. C., and Quay, B. H. (1996), "Stochastic Environmental Life
Cycle Assessment Modeling", International Journal of Life Cycle Assessment, Vol. 1, No. 4, pp.
199-207.
82. McCarville, D. R., and D. C. Montgomery (1996), "Optimal Guard Bands for Gauges in Series,"
Quality Engineering, Vol. 9, No. 2, pp. 167-177.
81. Somerville, S. E. and D. C. Montgomery (1996), "Process Capability Indices and Nonnormal
Distributions," Quality Engineering, Vol. 9, No. 2, pp. 305-316.
80. Rhoads, T. R., D. C. Montgomery, and C. M. Mastrangelo (1996), "A Fast Initial Response
Scheme for the Exponentially Weighted Moving Average Control Chart," Quality Engineering,
Vol.9, No. 2, pp. 317-327.
79. Del Castillo, E., Grayson, J. M., Montgomery, D. C., and G. C. Runger (1996), "A Review of
Statistical Process Control Techniques for Short-Run Manufacturing Systems," Communications
in Statistics-Theory and Methods, Vol. 25, No. 11, pp. 305-316.
78. Runger, G. C., Alt, F. B., and D. C. Montgomery (1996), "Controlling Multiple Stream Processes
with Principal Components," International Journal of Production Research, Vol. 34, No. 11, pp.
2991-2999.
77. Keats, J.B., Montgomery, D.C., Runger, G.C., and W.S. Messina (1996), "Feedback Control and
Statistical Process Monitoring", International Journal of Reliability, Quality and Safety
Engineering, Vol. 3, No. 3, pp. 231-241.
76. Montgomery, D. C. and G. C. Runger (1996), "Foldovers of 2k~p Resolution IV Designs," Journal
of Quality Technology, Vol. 28, No. 4, pp. 446-450.
75. Runger, G. C., Alt, F. A., and D. C. Montgomery (1996), "Contributors to a Multivariate
Statistical Process Control Signal", Communications in Statistics - Theory and Methods, Vol. 25,
No. 10, pp. 2203-2213.
74. Shumate, D. A. and D. C. Montgomery (1996), "Development of a TiW Plasma Etch Process
Using a Mixture Experiment and Response Surface Methodology," IEEE Transactions on
Semiconductor Manufacturing, Vol. 9, No. 3, pp. 335-343.
73. Mastrangelo, C. M., Runger, G. C. and D. C. Montgomery (1996) "Statistical Process Monitoring
with Principal Components," Quality and Reliability Engineering International, Vol. 12, No. 3,
pp. 203-210.
72. Cornell, J. A. and D. C. Montgomery (1996), "Interaction Models as Alternatives to Low-Order
Polynomials, " Journal of Quality Technology, Vol. 28, No. 2, pp. 163-176.
71. Del Castillo, E., Montgomery, D. C., and D. R. McCarville (1996), "Modified Desirability
Functions for Multiple Response Optimization," Journal of Quality Technology, Vol. 28, No. 3,
pp. 337-345.
70. Cornell, J. A. and D. C. Montgomery (1996), "Fitting Models to Data: Interaction Versus
Polynomial? Your Choice!", Communications in Statistics-Theory andMethods, Vol. 25, No. 11,
pp. 2531-2555.
69. Scranton, R., G. C. Runger, J. B. Keats and D. C. Montgomery (1996), "Efficient Shift Detection
Using Multivariate EWMA Control Charts and Principal Components," Quality and Reliability
Engineering International, Vol. 12, No. 3, pp. 165-172.
68. Del Castillo, E., P. Mackin, and D. C. Montgomery (1996), "Multiple Criteria Optimal Design of
Control Charts," HE Transactions, Vol. 28, No. 6, pp. 467-474.
RTI International	Peer Review of EPA's RSE Report	53

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
67. Del Castillo, E. and D. C. Montgomery (1996), "A General Model for the Optimal Economic
Design of Charts Used to Control Short or Long Run Processes," HE Transactions, Vol. 28, No.
3, pp. 193-201.
66. Lowry, C. A., and D. C. Montgomery (1995), "A Review of Multivariate Control Charts," HE
Transactions, Vol. 27, No. 6, pp. 800-810.
65. Grayson, J. M., G. C. Runger, and D. C. Montgomery (1995-96), "Average Run Length
Performance of the (/-Chart with Control Limits Based on the Average Sample Size," Quality
Engineering, Vol. 8, No. 1, pp. 117-128.
64. Annadi, H.P., J. B. Keats, G. C. Runger, and D. C. Montgomery (1995), "An Adaptive Sample
Size CUSUM Control Chart," International Journal of Production Research, Vol. 33, No. 6, pp.
1605-1616.
63. Torng, J. C-C., J. K. Cochran, D. C. Montgomery, and F. P. Lawrence (1995), "Implementing
Statistically Constrained Economic EWMA Control Charts," Journal of Quality Technology, Vol.
27, No. 3, pp. 257-264.
62. Montgomery, D. C., J. C-C. Torng, J. K. Cochran, and F. P. Lawrence (1995), "Statistically
Constrained Economic Design of the EWMA Control Chart," Journal of Quality Technology,
Vol. 27, No. 3, pp. 250-256.
61. Mastrangelo, C. M. and D. C. Montgomery (1995), "SPC with Correlated Observations for the
Chemical and Process Industries," Quality and Reliability Engineering International, Vol. 11, No.
2, pp. 79-89.
60. Del Castillo, E. and D. C. Montgomery (1995), "A Kalman Filtering Process Control Scheme with
an Application to Semiconductor Short-Run Manufacturing," Quality and Reliability Engineering
International, Vol. 11, No. 2.
59. Prabhu, S. S., D. C. Montgomery, and G. C. Runger (1995), "A Design Tool to Evaluate Average
Time to Signal Properties of Adaptive Control X-Bar Charts," Journal of Quality Technology,
Vol. 27, No. l,pp. 74-83.
58. Heinsman, J. A. and D. C. Montgomery (1995), "Optimization of a Household Product
Formulation Using a Mixture Experiment," Quality Engineering, Vol. 7, No. 3, pp. 583-600.
57. Torng, James C., D. C. Montgomery and J. K. Cochran (1994), "Economic Design of the EWMA
Control Chart," Economic Quality Control, Vol. 9, No. 1, pp. 3-23.
56. Prabhu, S. S., D. C. Montgomery, and G. C. Runger (1994), "A Combined Adaptive Sample Size
and Adaptive Sampling Interval Control Scheme," Journal of Quality Technology, Vol. 26, No. 3,
pp. 164-176.
55. Del Castillo, E. and D. C. Montgomery (1994), "Short-Run Statistical Process Control: Q-Chart
Enhancements and Alternatives," Quality and Reliability Engineering International, Vol. 10, No.
2, pp. 87-97.
54. Montgomery, D. C., J. B. Keats, G. C. Runger, and W. S. Messina (1994), "Integrating Statistical
Process Control and Engineering Process Control," Journal of Quality Technology, Vol. 26, No.
2, pp. 79-87.
53. Montgomery, D. C. and S. R. Voth (1994), "Multicollinearity and Leverage in Mixture
Experiments," Journal of Quality Technology, Vol. 26, No. 2, pp. 96-108.
52. Del Castillo, E. and D. C. Montgomery (1993), "Optimal Design of Control Charts for Finite -
Length Production Runs," Economic Quality Control, Vol. 8, No. 4, pp. 225-240.
51. Del Castillo, E. and D. C. Montgomery (1993), "A Nonlinear Programming Solution to the Dual
Response Problem," Journal of Quality Technology, Vol. 25, No. 3, pp. 199-204.
RTI International
Peer Review of EPA's RSE Report
54

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
50. Montgomery, D. C. and G. C. Runger (1993), "Gage Capability and Designed Experiments, Part
II: Experimental Design Models and Variance Component Estimation," Quality Engineering, Vol.
6, No. 2, pp. 289-305.
49. Montgomery, D. C. and G. C. Runger (1993), "Gage Capability and Designed Experiments, Part
I: Basic Methods," Quality Engineering, Vol. 6, No. 1, pp. 115-135.
48. Runger, G. C. and D. C. Montgomery (1993), "Adaptive Sampling Enhancements for Shewhart
Control Charts," HE Transactions, Vol. 25, No. 3, pp. 41-51.
47. Montgomery, D. C. and D. J. Friedman (1993), "Prediction Using Regression Models with
Multicollinear Predictor Variables," HE Transactions, Vol. 25, No. 3, pp. 73-84.
46. Coleman, D. E. and D. C. Montgomery (1993), "A Systematic Approach to Planning for a
Designed Industrial Experiment," (with discussion), Technometrics, Vol. 35, No. 1, pp. 1-27.
45. Montgomery, D. C. (1992), "The Use of Statistical Process Control and Design of Experiments in
Product and Process Development," HE Transactions, Vol. 24, No. 5, pp. 4-17.
44. Chua, M. K. and D. C. Montgomery (1992), "Investigation and Characterization of a Control
Scheme for Multivariate Quality Control," Quality and Reliability Engineering International, Vol.
8, no. 1, pp. 37-44.
43. Hubele, N. F., D. C. Montgomery and W. H. Chin (1991-1992), "An Application of Statistical
Process Control in Jet-Turbine Engine Component Manufacturing," Quality Engineering, Vol. 4,
No. 2, pp. 197-210.
42. Chua, M. K. and D. C. Montgomery (1991), "A Multivariate Quality Control Scheme,"
International Journal of Quality and Reliability Management, Vol. 8, No. 6, pp. 29-46.
41. Montgomery, D. C. and C. M. Mastrangelo (1991), "Some Statistical Process Control Methods for
Autocorrelated Data," (with discussion), Journal of Quality Technology, Vol. 23, No. 3, pp. 179-
193.
40. Yourstone, S. A. and D. C. Montgomery (1991), "Detection of Process Upsets - Sample
Autocorrelation Control Chart and Group Autocorrelation Control Chart Applications," Quality
and Reliability Engineering International, Vol. 7, pp. 133-140.
39. Montgomery, D. C. (1990-91), "Using Fractional Factorial Designs for Robust Process
Development," Quality Engineering, Vol. 3, No. 2, pp. 193-205.
38. Yourstone, S. A. and D. C. Montgomery (1989), "A Time-Series Approach to Discrete Real-Time
Process Quality Control," Quality and Reliability Engineering International, Vol. 5, No. 4, pp.
309-317.
37. Montgomery, D. C. (1988), "Experimental Design for Product and Process Optimization," SAE
Transactions, Fall Volume, pp. 145-155.
36. Gardiner, J. S., Friedman, D. J. and D. C. Montgomery (1987), "A Note on the Average Run
Length of Cumulative Sum Control Charts for Count Data," Quality and Reliability Engineering
International, Vol. 3, No.l, pp. 53-55.
35. Gardiner, J. S. and D. C. Montgomery (1987), "Using Statistical Control Charts for Software
Quality Control," Quality and Reliability Engineering International, Vol. 3, No. 1, pp. 15-20.
34. Montgomery, D. C. and R. H. Storer (1986), "Economic Models and Process Quality Control,"
Quality and Reliability Engineering International, Vol. 2, No. 4, pp. 221-228.
33. Montgomery, D. C., Panagos, M., and R. G. Heikes (1985), "Economic Design of x Control
Charts for Two Manufacturing Process Models," Naval Research Logistics Quarterly, Vol. 32.
No. 4, pp. 631-647.
RTI International
Peer Review of EPA's RSE Report
55

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
32. Montgomery, D. C. (1985), "The Effect of Non-normality on Variables Sampling Plans," Naval
Research Logistics Quarterly, Vol. 32, No. 1, pp. 34-41.
31. Friedman, D. J. and D. C. Montgomery (1985), "Evaluation of the Predictive Performance of
Biased Regression Estimators," Journal of Forecasting, Vol. 4, No. 2, pp. 153-163.
30. Askin, R. G. and D. C. Montgomery (1984), "An Analysis of Constrained Robust Regression
Estimators," Naval Research Logistics Quarterly, Vol. 31, No. 2, pp. 283-296.
29. Montgomery, D. C. and L. Greene (1983), "Validation of Computer Simulation Models of Missile
Systems," Journal of Spacecraft and Rockets, Vol. 20, No. 3, pp. 272-278.
28. Montgomery, D. C. (1982), "Economic Design of an x Control Chart," Journal of Quality
Technology, Vol. 14, No. 2, 99. 40-43.
27. Saniga, E. W. and D. C. Montgomery (1981), "Economical Quality Control Policies for a Single-
Cause System," AIIE Transactions, Vol. 13, No. 3, pp. 258-264.
26. Montgomery, D. C. and R. G. Askin (1981), "Problems of Nonnormality and Multicollinearity for
Forecasting Methods Based on Least Squares," AIIE Transactions, Vol. 13, No. 2, pp. 102-15.
25. Montgomery, D. C. and G. Weatherby (1980), "Modeling and Forecasting Time Series Using
Transfer Function and Intervention Methods," AIIE Transactions, Vol. 12, No. 4, pp. 289-307.
24. Askin, R. G. and D. C. Montgomery (1980), "Augmented Robust Estimators," Technometrics,
Vol.22, No. 3, pp. 333-341.
23. Montgomery, D. C., E. W. Martin and E. A. Peck (1980), "Interior Analysis of the Observations
in Multiple Linear Regression," Journal of Quality Technology, Vol. 12, No. 3, pp. 163-172.
22. Montgomery, D. C. and R. G. Conard (1980), "Comparison of Simulation and Flight-Test Data
for Missile Systems," Simulation, Vol. 31, pp. 63-72.
21. Montgomery, D. C. (1980), "The Economic Design of Control Charts: A Review and Literature
Survey," Journal of Quality Technology, Vol. 12, No. 2, pp. 75-87.
20. Stewart, R. D., D. C. Montgomery and R. G. Heikes (1978), "Choice of Double Sampling Plans
Based on Prior Distributions and Costs," AIIE Transactions, Vol. 10, No. 1, pp. 19-30.
19. Montgomery, D. C. and V. M. Bettencourt, Jr. (1977), "Multiple Response Surface Methods in
Computer Simulation," Simulation, Vol. 29, No. 4, pp. 113-121.
18. Montgomery, D. C. (1977), "Discussion of'Studies in a Simulated Job Shop'" by S. Eilon and I.
Chowdhury, Invited Discussion, Proceedings of the Institution of Mechanical Engineers, London,
Vol. 189, pp. 185-186.
17. Montgomery, D. C. and L. E. Contreras (1977), "A Note on Forecasting with Adaptive Filtering,"
Operational Research Quarterly, Vol. 28, No. 1, pp. 87-91.
16. Montgomery, D. C. and R. G. Heikes (1976), "Process Failure Mechanisms and Optimal Fraction
Defective Control Charts," AIIE Transactions, Vol. 8, No. 4, pp. 467-472.
15. Heikes, R. G., D. C. Montgomery, and R. L. Rardin (1976), "Using Common Random Numbers
in Simulation Experiments—An Approach to Statistical Analysis," Simulation, Vol. 27, No. 3, pp.
81-85.
14. Montgomery, D. C. and D. M. Evans, Jr. (1975), "Second-Order Response Surface Designs in
Computer Simulation", Simulation, Vol. 26, pp. 169-178.
13. Montgomery, D. C., R. G. Heikes and J. F. Mance (1975), "Economic Design of Fraction
Defective Control Charts " Management Science, Vol. 21, No. 11, pp. 1272-1284
12. Heikes, R. G., D. C. Montgomery, and J. Young (1974), "Alternative Process Models in the
Economic Design of T2 Control Charts," AIIE Transactions, Vol. 6, No. 1, pp. 55-61.
RTI International
Peer Review of EPA's RSE Report
56

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
11. Horwitz, J. S. and D. C. Montgomery (1974), "A Computer Simulation Model of a Rubella
Epidemic," Computers in Biology and Medicine, Vol. 4, No. 2, pp. 189-198.
10. Montgomery, D. C., M. S. Bazaraa and A. J. Keswani (1973), "Inventory Models with a Mixture
of Backorders and Lost Sales," Naval_Research Logistics Quarterly, Vol. 20, No. 2, pp.255-263.
9. Montgomery, D. C. and P. J. Klatt (1972), "Economic Design of 72 Control Charts to Maintain
Current Control of a Process," Management Science, Vol. 19, No. 1, pp. 76-89.
8. Montgomery, D. C., J. J. Talavage and C. J. Mullen (1972), "A Response Surface Approach to
Improving Traffic Signal Settings in a Street Network," Transportation Research, Vol. 6, pp. 69-
80.
7. Sipper, D. and D. C. Montgomery (1972), "Probability Zones in Stochastic Project Networks,"
Journal of Systems Management, Vol. 23, No. 8, pp. 36-42.
6. Montgomery, D. C. and P. J. Klatt (1972), "Minimum Cost Multivariate Quality Control Tests,"
AIIE Transactions, Vol. 4, No. 2, pp. 103-110 This paper also appeared in Proceedings of the
AIIE Annual Conference, Anaheim, California, June 1972.
5. Ghare, P. M., D. C. Montgomery and W. C. Turner (1971), "Optimal Interdiction Policy for a
Flow Network" Naval Research Logistics Quarterly, Vol. 18, No. 1, pp. 37-45.
4. Montgomery, D. C. (1970), "Adaptive Control of Exponential Smoothing Parameters by
Evolutionary Operation," AIIE Transactions, Vol. 2, No. 3, pp. 268-269.
3. Dickey, J. W. and D. C. Montgomery (1970) "A Simulation-Search Technique: An Example
Application for Left-Turn Phasing," Transportation Research, Vol. 4, pp. 339-347.
2. Montgomery, D. C. (1969), "An Application of Statistical Forecasting Techniques in an Inventory
Control Policy," Production and Inventory Management, Vol. 10, No. 1, pp. 66-73.
1. Montgomery, D. C. (1968), "An Introduction to Short-Term Forecasting," The Journal of
Industrial Engineering, Vol. 11, No. 10, pp. 500-504. This paper was also selected for inclusion
in Mathematical Models in Marketing, edited by Robert G. Murdick, International Textbook Co.
(1971).
Chapters in Books
18. Hassler, E., Montgomery, D.C. and Silvestrini, R.T. (2015), "Bayesian D-Optimal Design Issues
for Binomial Generalized Linear Model Screening Designs", in Frontiers in Statistical Quality
Control If edited by S. Knoth and W. Schmid, Springer AG, Switzerland, pp. 337-353.
17. Johnson, R.T., Montgomery, D.C. and Kennedy, K.S. (2012), "Hybrid Space-Filling Designs for
Computer Experiments", in Frontiers in Statistical Quality Control 10, edited by H.-J. Lenz, W.
Schmid and P.-T. Wilrich, Springer, New York, pp. 287-301.
16. Englert, B.R., Rigdon, S.E., Borror, C.M., Montgomery, D.C., and Pan, R. (2012), "Optimal
Designs for Multifactor Experiments for Exponentially Distributed Lifetimes", in Frontiers in
Statistical Quality Control 10, edited by H.-J. Lenz, W. Schmid and P.-T. Wilrich, Springer, New
York, pp. 303-317.
15. Goldfarb, H. B. and Montgomery, D. C. (2006), "Graphical Methods for Comparing Response
Surface Designs for Experiments with Mixture Components", Chapter 14 in Response Surface
Methodology and Related Topics, edited by Andre I. Khuri, World Scientific Publishing Co.,
Singapore.
RTI International
Peer Review of EPA's RSE Report
57

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
14. Montgomery, D. C. and Jennings, C. L. (2006), "An Overview of Industrial Screening
Experiments", Chapter 1 in Screening: Methods for Experimentation in Industry, Drug Discovery,
and Genetics, Angela Dean and Susan Lewis, Editors, Springer, NY, pp. 1-20.
13. Montgomery, D. C. and Jennings, C. L. (2003), "Statistische Methoden in der Prozesskontrolle",
in Wahrscheinlichkeitstheorie, Stochastische Prozesse, Mathematische Statistik, F.E. Beichelt and
D. C. Montgomery, Editors, B. G. Teubner Verlag, Weisbaden.
12. Mastrangelo, C. M. and Montgomery, D. C. (1999), "Process Monitoring with Autocorrelated
Data", in Statistical Process Monitoring and Optimization, edited by G. G. Vining and Sung H.
Park, pp. 139-160, Marcel Dekker, Inc., New York.
11. Montgomery, D. C., Peck, E.A., and Simpson, J.R.(1998), "Multicollinearity and Biased
Estimation in Regression", in Handbook of Statistical Methods for Engineers and Physical
Scientists, 2nd edition, edited by H.M. Wadsworth, Jr., McGraw-Hill, pp. 16.1-16.27.
10. Mastrangelo, C. M. and D. C. Montgomery (1996), "Time Series Analysis," Encyclopedia of
Operations Research and Management Science, Kluwer Academic Publishers, Norwell, MA.
9. McCarville, D. R. and D. C. Montgomery (1996), "Optimizing Defect Levels and Losses from
Gage Errors," in Statistical Applications in Process Control, edited by J. B. Keats and D. C.
Montgomery, Marcel Dekker, New York.
8. Montgomery, D. C. and G. C. Runger (1996), "Experimental Design Models with Random
Components," in Statistical Applications in Process Control, edited by J. B. Keats and D. C.
Montgomery, Marcel Dekker, New York.
7. Montgomery, D. C. (1996), "Some Practical Guidelines for Designing an Industrial Experiment,"
in Statistical Applications in Process Control, edited by J. B. Keats and D. C. Montgomery,
Marcel Dekker, New York.
6. Messina, W. S., Montgomery, D. C., Keats, J. B. and G. C. Runger (1996), "Strategies for
Statistical Monitoring of Integral Control for the Continuous Process Industries," in Statistical
Applications in Process Control, edited by J. B. Keats and D. C. Montgomery, Marcel Dekker,
New York.
5. Montgomery, D. C. (1991), "Analyzing Location and Dispersion Effects from Designed
Experiments: Same Examples," Statistical Process Control in Manufacturing, J. B. Keats and D.
C. Montgomery, editors, Marcel Dekker, New York.
4. Montgomery, D. C. and E. A. Peck (1990), "Multicollinearity in Regression," Handbook of
Statistical Methods for Engineers, McGraw-Hill Book Co., H.M. Wadsworth, Jr., Editor,
McGraw-Hill, New York.
3. Montgomery, D. C. and D. J. Friedman (1989), "Statistical Process Control in a Computer-
Integrated Manufacturing Environment," Statistical Process Control in Automated Manufacturing,
J. B. Keats and N. F. Hubele, editors, Marcel Dekker, New York.
2. Johnson, L. A. and D. C. Montgomery (1979), "Forecasting with Exponential Smoothing and
Related Methods," Forecasting, Vol. 12, TIMS Studies in the Management Sciences, North-
Holland, Amsterdam.
1. Johnson, L. A. and D. C. Montgomery (1978), "Production and Inventory Control," The
Encyclopedia of Computer Science and Technology, Vol. 10, Marcel Dekker, New York.
Papers in Conference Proceedings
RTI International
Peer Review of EPA's RSE Report
58

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
37. Montgomery, D.C. (2014), "The Contribution of Six Sigma to the Development of Statistical
Thinking in the Workplace", Proceedings of the 9th International Conference on Teaching
Statistics, Flagstaff, Arizona, July, the International Statistics Institute, pp. 1-5.
36. Shaukat, K., Montgomery, D.C. and Syrotiuk, V.R. (2011), "Adaptive Overhead Reduction via
MEWMA Control Charts", Proceedings of the 14th ACM International Conference on Modeling,
Analysis and Simulation of Wireless and Mobile Systems (MSWiM'll), Miami, Florida, U.S.A.,
October 31-November 4, pp. 205-212.
35. Johnson, R.T., Montgomery, D.C., Jones, B., and Fowler, J.W. (2008), "Comparing Designs for
Computer Simulation Experiments," Proceedings of the Winter Simulation Conference, pp. 463-
470, Miami, FL, Dec. 7-10.
34. Chatlani, V., Tylavsky, D. J., Montgomery, D.C., and Dyer, M, (2007), "Statistical Properties of
Diversity Factors for Probabilistic Loading of Distribution Transformers," 2007 North American
Power Symposium, September, pp. 581-587.
33. Lin, Y.K., Pfund, M.E., Fowler, J.W., and Montgomery, D.C. (2006), "Classification of Parallel
Machine Environments under Various Correlation Structures", 36th International Conference on
Computers and Industrial Engineering, Taipei, Taiwan, R.O.C., June 20-23, pp. 1253-1261
32. Hoskins, D.S., Colbourn, C.J., and Montgomery, D.C. (2005), "Software Performance Testing
using Covering Arrays, Efficient Screening Designs with Categorical Factors", Proceedings of the
5th International Workshop on Software Performance, WOPS 05, pp. 131-136.
31. Montgomery, D. C., Borror, C. M., and Lewis, S. L. (1997), "Analysis of Designed Experiments
using S AS PROC GENMOD", Proceedings of the Western Users of SAS Software, Universal City,
CA, 22-24 October.
30. Rhoads, T. R. and D. C. Montgomery (1996), "Process Monitoring with Principal Components and
Partial Least Squares," Proceedings of the Industrial Engineering Research Conference,
Minneapolis, MN.
29. Montgomery, D. C., Keats, J. B., and G. Rajavelu (1996), "Statistical Monitoring Techniques for
Contamination Data," Institute of Environmental Sciences-Proceedings, Orlando, Fl.
28. Montgomery, D. C. and M. L. Bowles (1995), "Multiple Response Optimization Methods,"
Proceedings of the Annual Conference on Applied Statistics, Atlantic City, NJ.
27. Myers, R. H. and D. C. Montgomery (1994), "Robust Design and Response Surface
Methodology," Proceedings of the 50th Conference on Applied Statistics, Atlantic City, NJ.
26. Montgomery, D. C. (1994), "Strategies for Integrating Statistical Process Control and Engineering
Process Control", Proceedings of the Rutgers Conference on Computer Integrated Manufacturing
in the Process Industries, Piscataway, NJ.
25. Mastrangelo, C. M., and D. C. Montgomery (1994), "Shift Detection Properties of Moving-
Centerline EWMA Control Schemes", Proceedings of the Industrial Engineering Research
Conference, Atlanta, GA.
24. Montgomery, D. C., and J. E. Taggart (1993), "Selection of a Second Order Response Surface
Design," Proceedings of the SAS Users Group International, New York, NY.
23. Montgomery, D. C., C. M. Mastrangelo, and C. A. Lowry (1993), "Statistical Process Monitoring
for Dynamic Systems," Proceedings of the Industrial Engineering Research Conference, San
Francisco, CA.
22. Lowry, C. A. and D. C. Montgomery (1992), "Multivariate Quality Control: Review and
Enhancement," Proceedings of the Institute for Decision Sciences Conference, San Francisco, CA..
21. Montgomery, D. C. (1992), "Some Problems in Computer-aided Design of Experiments,"
Proceedings of the SAS Users Group International, Honolulu, HI.
RTI International	Peer Review of EPA's RSE Report	59

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
20. Montgomery, D. C. and C. M. Mastrangelo (1991), "Basic Properties of the EWMA," Proceedings
of the Arizona State University Conference on Statistical Control and Design, Tempe, AZ.
19. Montgomery, D. C. (1991), "Experimental Design in Engineering Design and Development,"
Proceedings of the SAS Users Group International, New Orleans, LA.
18. Montgomery, D. C. (1989), "Rational Subgroups and Control Charts," Proceedings of the Arizona
State University Conference on Statistical Control and Design, Tempe, AZ.
17. Montgomery, D. C., Gardiner, J. S., and B. A. Pizzano (1987), "Statistical Process Control
Methods for Detecting Small Process Shifts," Frontiers in Statistical Quality Control, H. J. Lenz et
al., editors, Physica-Verlag, Heidelberg.
16. Montgomery, D. C. (1984), "Biased Estimation and Robust Regression," Proceedings of the
American Association of Physicists in Medicine Midyear Topical Symposium, Mobile, AL.
15. Montgomery, D. C. (1983), "Statistical Consulting: Some Comments on Training of Statisticians,"
Proceedings of the SAS Users Group International, New Orleans, LA.
14. Montgomery, D. C., Heikes, R. G., and M. R. Scheffler (1981), "Probability Models for the
Occurrence of Defects," Frontiers in Statistical Quality Control, H. J. Lenz, G. B. Wetherill and
P.Th. Wilrich, Editors, Physica-Verlag, Vienna.
13. Montgomery, D. C. (1981), "Cost Based Acceptance Sampling Plans and Process Control
Schemes," Proceedings of the AIIEFall Conference, Washington, D. C.
12. Montgomery, D. C. and E. A. Peck (1980), "The Multicollinearity Problem in Regression,"
Proceedings of the Southeast Decision Sciences Conference, Orlando, FL.
11. Montgomery, D. C. and G. Weatherby (1979), "Factor Screening Methods in Computer
Simulation," Proceedings of the Winter Simulation Conference, San Diego, CA.
10. Simms, E. D. and D. C. Montgomery (1977), "The Use of Discriminant Analysis for Risk
Assessment in Operational Testing," Proceedings of the 16th Annual U.S. Army Operations
Research Symposium, Ft. Lee, VA.
9. Russ, S. W., Jr., D. C. Montgomery, and H. M. Wadsworth, Jr., (1977) "A Cost Optimal Approach
to Selection of Experimental Designs for Operational Testing Under Conditions of Constrained
Sample Size," Proceedings of the 16th Annual U.S. Army Operations Research Symposium, Ft.
Lee, VA.
8. Friese, W. F., Jr., and D. C. Montgomery (1977), "A Cost-Optimal Approach to Selecting a
Fractional Factorial Design," Proceedings of the 16th Annual U.S. Army Operations Research
Symposium, Ft. Lee, VA.
7. Brown, E. L. and D. C. Montgomery (1975), "An Application of Network Simulation to
Operational Testing and Evaluation," Proceedings of the 14th Annual U. S. Army Operations
Research Symposium, Ft. Lee, VA., November.
6. Montgomery, D. C., J. F. Mance and R. G. Heikes (1974), "An Economic Model of the Fraction
Defective Control Chart with Multiple Assignable Causes," Transactions of the ASQC, Boston,
MA.
5. Johnson, L. A. and D. C. Montgomery (1974), "On Dynamic Production Planning Models,"
Proceedings of the Southeast IDS Meeting, New Orleans, LA.
4. Marsh, J. D. and D. C. Montgomery (1973), "Optimal Procedures for Scheduling Jobs with
Sequence Dependent Changeover Times on Parallel Processors," Proceedings of the AIIE Annual
Conference, Chicago, IL.
3. Montgomery, D. C. and H. M. Wadsworth (1972), "Some Techniques for Multivariate Quality
Control Applications," Transactions of the ASQC, Washington, D.C.
RTI International
Peer Review of EPA's RSE Report
60

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
2. Ghare, P. M. and D. C. Montgomery (1970), "Flow Management in Transportation Networks,"
Proceedings of the Fifth International Conference on Operations Research, Venice, Italy, June
1969, Tavistock Publishers, Ltd..
1. Montgomery, D. C. (1971), "Stochastic Capacity Decision Models for Production Facilities,"
Proceedings of the AIIE Annual Conference, Boston, MA.
Papers and Presentations at Meetings
193. Hill, R. R., Ahner, D, Dillard, D. and Montgomery, D. C. (2017), "Examining Potential
Reductions in Wind Tunnel Testing Data Requirements", Presented at the Quality and
Productivity Research Conference, Storrs, CT, June, 2017.
192. Hassler, E., Montgomery, D.C., and Silvestrini, R. (2016), "Design of Experiments for
Generalized Linear Models with Random Blocks", Invited presentation at the Quality and
Productivity Research Conference, Tempe, AZ 13-16 June, 2016.
191. Burke, S.E., Anderson-Cook, C.M., Borror, C.M., and Montgomery, D.C., (2016), "A Layered
Pareto Front Approach to Search for the Top N Subpopulations in a Stockpile", Invited
presentation at the Quality and Productivity Research Conference, Tempe, AZ 13-16 June,
2016.
190. Montgomery, D.C. (2016), "Modern Experimental Design or The Flight of The Phoenix",
Invited plenary address at the JMP Discovery Summit Europe, Amsterdam, 15-17 March
2016.
189. Mancenido, M.V., Montgomery, D.C., and Pan, R. (2015), "Performance of Standard Mixture
Designs in Modeling Ordinal Responses", presentation at the 2015 INFORMS annual meeting,
Philadelphia.
188. Montgomery, D.C. (2015), "Design of Experiments: A Key to Successful Innovation", Invited
presentation at the 59fe Annual Fall Technical Conference, Houston, TX. 8-9 October.
187. Burke, S.E., Montgomery, D.C., Borror, C.M., and Silvestrini, R.T. (2015), "Optimal Designs
for Dual Response Systems", Invited presentation at the 59th Annual Fall Technical
Conference, Houston, TX. 8-9 October.
186. Weese, M.L., Montgomery, D.C. and Ramsey, P.J. (2015), "Analysis Strategies for Definitive
Screening Designs", Invited presentation at the 59th Annual Fall Technical Conference, Houston,
TX. 8-9 October.
185. Montgomery, D.C. (2015), "Modern Experimental Design or The Flight of The Phoenix",
Invited plenary address at the JMP Discovery Summit, San Diego, CA, 14-17 September 2015.
184. Ramsey, P., Weese, M., and Montgomery, D.C. (2015), "Model Selection Strategies for
Definitive Screening Designs Using JMP Pro and R", Invited presentation at the JMP
Discovery Summit, San Diego, CA, 14-17 September 2015.
183. Montgomery, D.C. (2015), "Teaching Design of Experiments to Engineers and Scientists",
invited presentation at the Design and Analysis of Experiments Conference 2015, Cary NC, 4-
6 March 2015.
RTI International
Peer Review of EPA's RSE Report
61

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
182. Mancendito, M., Montgomery, D.C., and Pan, R. (2014), "Modeling Ordered Categorial
Responses in Mixture Expoeriments", Invited presentation at the 58th Annual Fall Technical
Conference, Richmond, Va., 2-3 October.
181. Stone, B.B., Montgomery, D.C., Silvestrini, R., and Jones, B. (2014), "No Confounding
Designs of 20 and 24 Runs: Alternatives to Resolution IV Screening Designs", Invited
presentation at the 58th Annual Fall Technical Conference, Richmond, Va., 2-3 October.
180. Montgomery, D.C. (2014), "The Contribution of Six Sigma to the Development of Statistical
Thinking in the Workplace", Invited presentation at the 9th International Conference on
Teaching Statistics, Flagstaff, AZ, 13-18 July.
179. Montgomery, D.C. (2014), "Innovation, Six Sigma and Quality Technology", Invited
presentation at the 2014 Joint Research Conference, Seattle, Washington, 24-26 June.
178. Shinde, S.M., Montgomery, D.C. and Jones, B. (2013), "Projection Properties of No-
Confounding Designs for Six, Seven, and Eight Factors in Sixteen Runs", invited presentation
at the 57th annual Fall Technical Conference, San Antonio, Texas, 18 October 2013.
177. Montgomery, D.C. (2013), "Stu Hunter's Contributions to Statistics and Quality Engineering",
invited presentation at the 57th annual Fall Technical Conference, San Antonio, Texas, 17
October 2013.
176. Jones, B. and Montgomery, D.C. (2013), invited short course, "Recent Developments in
Design of Experiments", presented at the 57th annual Fall Technical Conference, San Antonio,
Texas, 16 October 2013.
175. Jones, B. and Montgomery, D.C. (2013), "Stu Hunter's Contributions to Statistics and Quality
Engineering", invited presentation in the Technometrics session, Joint Statistical Meetings,
Montreal, Canada, August 2013.
173. Montgomery, D.C. (2013), "Stu Hunter's Contributions to Statistics and Quality Engineering",
invited presentation at the conference honoring Stu Hunter's 90th birthday, Amsterdam, March
2013.
172. Chen, Y., Montgomery, D.C., Fowler, J., and Pfund, M. (2013), "Using Regression Splines to
Parameterize Composite Dispatching Rules", presented at the 43rd International Confernce on
Computers and Industrial Engineering, 16-18 October, 2013, Hong Kong.
171. Hassler, E., Montgomery, D.C., and Silvestrini, R. (2013), "Bayesian D-Optimal Design Issues
for Generalized Linear Models", invited presentation at the 11th Workshop on Intellegent
Statistical Quality Control, Sydney, Australia, 20-23 August, 2013.
170. Rigdon, S., Pan, R., Montgomery, D.C. and Borror, C.M. (2012), "Design of Experiments for
Reliability Improvement", invited presentation at the 56th Annual Fall Technical Conference,
St. Louis, MO, 3-4 October, 2012.
169. Timmer, D., Gonzalez, Montgomery, D.C. and Borror, C.M. (2012), "DOE Education
Strategies", invited presentation at the 56th Annual Fall Technical Conference, St. Louis, MO,
3-4 October, 2012.
168. Montgomery, D.C. (2012), "Methods and Applications of Generalized Linear Models", one-
day short course presented at the 56111 Annual Fall Technical Conference, St. Louis, MO, 3
October, 2012.
RTI International
Peer Review of EPA's RSE Report
62

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
167. Montgomery, D.C. (2012), "Experiments with Physical and Resource Constraints", Invited
presentation at the 2012 Joint Statistical Meetings, 28 July - 2 August, San Diego, CA.
166. Montgomery, D.C. (2012), "Innovation, Statistics and Quality Technology", Invited keynote
address presented at the Fourth International Conference on Lean Six Sigma, Glasgow,
Scotland, 26-27 March, 2012
165. Timmer, D., Gonzalez, M., Borror, C. and Montgomery, D.C. (2011), "Web-Based Active
Learning Laboratories for Teaching Control Charts", presented at the 55th Annual Fall
Technical Conference, Kansas City, October 13-14, 2011.
164. Krueger, D. and Montgomery, D.C. (2011). "Integrating CART and Generalized Linear
Models for Improving process Understanding", presented at the 55th Annual Fall Technical
Conference, Kansas City, October 13-14, 2011.
163. Montgomery, D.C. (2011), "Generating and Assessing Exact G-optimal Designs" , Invited
presentation at the lassie Newton Institute for Mathematical Sciences, Cambridge, Design and
Analysis of Experiments Workshop, 30 August - 2 September, 2011.
162. Montgomery, D.C. (2011), "Design of Experiments: New Methods and How to Use Them in
Design, Development and Decision-making", Inaugural W.L. Gore lecture at the Alfred Lerner
College of Business and Economics, The University of Delaware, 16 March 2011.
161. Shinde, S. and Montgomery, D.C. (2010), "Analysis Methods for Non-regular Fractional
Factorial Designs, presented at the INFORMS Annual Meeting, Austin Texas, 7-10
November.
160. Monroe, E., Pan, R., Montgomery, D.C., Borror, C.M., and Anderson-Cook, C.M. (2010),
"Sensitivity Analysis of Optimal Designs for Accelerated Life Testing", Journal of Quality
Technology invited paper session, INFORMS Annual Meeting, Austin Texas, 7-10
November.
159. Jones, B. and Montgomery, D.C. (2010), "Workshop on Modern Experimental Design
Methods", presented at the 2010 Army Conference on Applied Statistics" Cary, NC, 18-19
October.
157. Johnson, R.T., Montgomery, D.C., Jones, B., and Parker, P.A. (2010), "Comparing Computer
Experiments for Fitting High-Order Polynomial Models", invited presentation at the Journal of
Quality Technology Session at the 54th Annual Fall Technical Conference, Birmingham,
Alabama, 7-8 October.
156. Fish, B.R., Rigdon, S.E., Borror, C.M., Montgomery, D.C. and Pan, R. (2010), "Optimal
Designs for Multifactor Life Testing Experiments", invited presentation at the 10th
International Workshop on Intelligent Statistical Quality Control, Seattle, WA 18-20 August.
155. Johnson, R.T., Montgomery, D.C. and Kennedy, K.S. (2010), "Hybrid Space-Filling Designs
for Computer Experiments", invited presentation at the 10th International Workshop on
Intelligent Statistical Quality Control, Seattle, WA 18-20 August.
154. Rigdon, S.E., Montgomery, D.C., Pan, R., and Borror, C.M. (2010), "Optimal Design for
Multi-Factor Life-Testing Experiments", presented at the Joint Statistical Meetings,
Vancouver BC, Canada, 31 July-5 August.
RTI International
Peer Review of EPA's RSE Report
63

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
153. Montgomery, D.C. (2010), "A New Framework for Teaching Design of Experiments", Invited
Panel Discussion on Future Developments in Experimental Design at the Joint Statistical
Meetings, Vancouver BC, Canada, 31 July-5 August.
152. Laungrungrong, B., Borror, C.M., and Montgomery, D.C. (2009), "Multivariate Poisson-
Distributed Control Charts", 53rd Annual Fall Technical Conference, Indianapolis, 7-10
October.
151. Capehart, S.R., Kulahci, M, Keha, A., and Montgomery, D.C. (2009), "Designing Fractional
Factorial Split-Plot Experiments using Integer Programming", 53rd Annual Fall Technical
Conference, Indianapolis, 7-10 October.
150. Montgomery, D.C. (2009), "Critical Components of a Quality and Reliability Engineering
Graduate Program", presentation and panel discussion at the INFORMS annual meeting, San
Diego CA, 11-14 October.
149. Montgomery, D.C. (2009),"Panel Discussion: Information and Messages from Editors of QSR
Journals", presented at the INFORMS annual meeting, San Diego CA, 11-14 October.
148. Montgomery, D.C. (2009), "Generating and Assessing Exact G-Optimal Designs: (Is it worth
it?)", invited presentation at the Joint Statistical Meetings, 3 August, Washington, DC.
147. Montgomery, D.C. (2009), "Modern Experimental Design methods and Their Impact on
Business and Industry", invited tutorial session at the INFORMS Regional Conference, 24 April,
Tempe AZ.
146. Jones, B. Johnson, R.T., and Montgomery, D.C. (2009), "Comparing Space Filling Designs for
Gaussian Process Models", presented at the INFORMS Regional Conference, 24 April, Tempe
AZ.
145. Broyles, J.R., Cochran, J.K., and Montgomery, D.C. (2009), "A Markov Decision Process for
Hospital Inpatient Staffing", presented at the INFORMS Regional Conference, 24 April, Tempe
AZ.
144. Montgomery, D.C. (2008), "Modern Experimental Design and its Impact on Design for Six
Sigma", Invited keynote Address at the Third International Conference on Six Sigma,
Edinburgh, Scotland, 15-16 December, 2008.
143. Johnson, R.T., Montgomery, D.C., and Jones, B. (2008), "Comparing Designs used for Fitting
Gaussian process Models", invited presentation at the 52nd Annual Fall Technical Conference,
Phoenix, AZ, 9-10 October, 2008.
142. Krueger, D. and Montgomery, D.C. (2008), "Semiconductor Yield Modeling using Generalized
Linear Models", invited presentation at the 52nd Annual Fall Technical Conference, Phoenix,
AZ, 9-10 October, 2008.
141. Montgomery, D. C. (2008), "Some Experiences with Designing Experiments", Friday
Luncheon Address, 52nd Annual Fall Technical Conference, Phoenix, AZ, 9-10 October, 2008.
140. Montgomery, D.C. (2008), "Statistical Design Techniques for Robust Design", invited
presentation at the European Conference on Design of Experiments, Antwerp, Belgium,
January, 2008.
139. Montgomery, D.C. (2007), "A Modern Framework for Enterprise Excellence", Deming
Lecture, presented at the Joint Statistical Meetings, Salt Lake City, 31 July.
RTI International
Peer Review of EPA's RSE Report
64

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
138. Montgomery, D.C. (2007), "Statistics and Science, Business and Industry", Invited keynote
presentation at the JMP Users' Conference, Cary NC, 13 June 2007.
137. Montgomery, D. C. (2007), "Teaching DOX: Some Adventures and Lessons Learned", Invited
Plenary Presentation at the ASA Quality and Productivity Research Conference, Santa Fe,
New Mexico, 4 June 2007.
136. Montgomery, D. C. (2006), "Logistic Regression", invited short course given at the 50th
Annual ASA/ASQ Fall Technical Conference, Columbus OH, October, 2006
135. Montgomery, D. C. (2006), "Comparison and Evaluation of Designs", invited presentation at
the Joint Statistical Meetings, Seattle, WA, August, 2006.
134. Montgomery, D. C. (2006), "The Impact of Statistics on Science, Business and Industry",
invited Keynote Address, meeting of the International Statistical Institute, Lima, Peru, January,
2006.
133. Montgomery, D. C. (2005), "Some Trends in Six-Sigma Education", invited presentation at
the 49th Annual ASA/ASQ Fall Technical Conference, St. Louis MO, 20-21 October, 2005.
132. Lawson, C. A. and Montgomery, D. C. (2005), "Business Process Characterization using
Categorical Data Models", invited presentation at the 49th Annual ASA/ASQ Fall Technical
Conference, St. Louis MO, 20-21 October, 2005.
131. Montgomery, D. C. (2005), "Criteria for Designing Experiments: Some Practical
Considerations", invited presentation at the Los Alamos National Labs Design and Analysis of
Experiments Conference, Santa Fe, NM,. 11-14 October, 2005.
130. Montgomery, D. C. and Goldfarb, H. B. (2005),"Graphical Methods for the Evaluation of
Mixture and Mixture-Process Designs", invited presentation in the 50th anniversary of mixture
experiments session, Joint Statistical Meetings, Minneapolis, MN, 7-11 August, 2005.
129. Montgomery, D. C. (2005), "Statistics and the Transformation of Science, Business and
Industry", invited keynote presentation at the 5th Annual ENBIS Conference, University of
Newcastle, Newcastle-Upon-Tine, UK, 14-16 September, 2005.
128. Park, You-Jin, Richardson, D. E., Borror, C. M., Anderson-Cook, C. M., and Montgomery, D.
C. (2004), "Prediction Variance Properties of Second-Order Response Surface Designs for
Cuboidal Regions", invited paper presented at the 48th ASA/ASQ Fall Technical Conference,
Roanoke, VA, 14-15 October.
127. Robinson, T. J., Wulff, S. S., Montgomery, D. C., and Kurhi, A. I (2004), "A Response
Surface Approach to Robust Parameter Design using Generalized Linear Models", invited
paper presented at the 48th ASA/ASQ Fall Technical Conference, Roanoke, VA, 14-15
October.
126. Heredia-Langner, A., Montgomery, D. C., Carlyle, W. M., and Borror, C. M. (2004), "Model-
Robust Optimal Designs: A Genetic Algorithm Approach", invited paper presented in the
Journal of Quality Session at the 48th ASA/ASQ Fall Technical Conference, Roanoke, VA, 14-
15 October.
125. Chung, J., Goldfarb, H. B., and Montgomery, D. C. (2004), "Statistical Designs for Mixture-
Process Variable Experiments with Control and Noise Variables", invited paper presented at
the 48th ASA/ASQ Fall Technical Conference, Roanoke, VA, 14-15 October.
124. Holcomb, D. R., Montgomery, D. C., and Lurponglukana, N. (2004), "A Bootstrap Method for
Determining Active factors in Unreplicated Factorial Designs", invited paper presented at the
48th ASA/ASQ Fall Technical Conference, Roanoke, VA, 14-15 October.
RTI International
Peer Review of EPA's RSE Report
65

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
123. Woodall, W. H., Spitzner, D., Montgomery, D. C., and Gupta, S. (2004), "Using Control
Charts to Monitor Product and Process Quality Profiles", invited presentation at the Joint
Statistical Meetings, Toronto, CA 8-12 August.
122. Kowalski, K. M., Vining, G. G., Montgomery, D. C. and Borror, C. M. (2004), Modeling the
Process Mean and Variance from a CCD", contributed paper presentation at the Joint
Statistical Meetings, Toronto, CA 8-12 August.
121. Montgomery, D. C., Jearkpaporn, D., Runger, G. C., and Borror, C. M. (2004), "Monitoring
Mean Shifts for Multistage Processes using Generalized Linear Models", invited presentation
at the Joint Statistical Meetings, Toronto, CA 8-12 August.
120. Montgomery, D. C. (2004), "Six Sigma: New Directions for DOX", invited presentation at the
2004 Quality and Productivity Research Conference, 19-21 May, Durham, NC.
119. Montgomery, D. C. (2004), "Designing Experiments: Some Adventures and Lessons
Learned", invited seminar at the Industrial Statistics Research Center, University of
Newcastle-Upon-Tyne, 23 April, Newcastle-Upon-Tyne, UK.
118. Montgomery, D. C. (2004), "Statistics and Statisticians in Today's Business World", keynote
address, Royal Statistical Society Conference on Business Improvement Through Statistical
Thinking, 21-22 April, Coventry, UK.
117. Goldfarb, H. B., Anderson-Cook, C. M., Borror, C. M., and Montgomery, D. C. (2003),
"Graphical Methods to Assess the Prediction Capability of Mixture and Mixture-Process
Designs", invited presentation at the 47th Annual ASA/ASQ Fall Technical Conference, El
Paso, TX, 16-17 October 2003.
116. Drain, D. C., Borror, C. M., Montgomery, D. C., and Anderson-Cook, C. M. (2003), "The
Effect of Correlated Noise Variables on Designed Experiments", invited presentation at the
47th Annual ASA/ASQ Fall Technical Conference, El Paso, TX, 16-17 October 2003.
115. Burdick, R. K, Borror, C. M., and Montgomery, D. C. (2003), "A Review of Methods for
Measurement Systems Capability Analysis", invited presentation at the Journal of Quality
Technology session, 47th Annual ASA/ASQ Fall Technical Conference, El Paso, TX, 16-17
October.
114. Kowalski, S., Vining, G.G., Montgomery, D.C., and Borror, C.M. (2003), Modifying a Central
Composite Design to Model the Mean and Variance Within a Split-Plot Structure", invited
presentation at the Joint Statistical Meetings, San Francisco, CA 3-7 August 2003.
113. Robinson, T. J., Myers, R. H., and Montgomery, D. C. (2003), "Analysis Considerations in
Industrial Split-Plot Experiments when the Responses are Non-normal", invited presentation at
the Joint Statistical Meetings, San Francisco, CA, 3-7 August 2003.
112. Montgomery, D.C., Burdick, R. K, Sebert, D.M., Shah, H. K, Molnau, W., Lawson, C.,
Zenzen, F., and Holcomb, D. R. (2003), "Teaching Six-Sigma Concepts in a University
Setting", invited presentation at the Joint Statistical Meetings, San Francisco, CA 3-7 August
2003.
111. Drain. D. C., Montgomery, D. C., and Borror, C.M. (2003), "The Application of Hybrid
Heuristic Optimization in Design of Experiments", invited presentation at the Joint Statistical
Meetings, San Francisco, CA 3-7 August 2003.
110. Montgomery, D. C. (2003), "The Modern Practice of Statistics in Business and Industry", the
Isobel Loutit Invited Plenary Address on Business and Industrial Statistics, 33rd Annual
Meeting of the Statistical Society of Canada, Halifax, NS, 8-11 June 2003.
109. Jearkpaporn, D., Eastman, S. A., Gonzalez-Altamirano, G., Holcomb, D. R., Heredia-Langner,
A., Borror, C. M., and Montgomery, D. C. (2003), "Using Supersaturated Designed
RTI International
Peer Review of EPA's RSE Report
66

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
Experiments for Factor Screening and Robustness Analysis in the Design of a Semiconductor
Clock Circuit", contributed paper presented at the 2003 ASA Quality and Productivity
Research Conference, Yorktown Heights, NY, 21-23 May 2003.
108. Anderson-Cook, C. M., Ozol, A., Myers, R. H., and Montgomery, D. C. (2003), "Fraction of
Design Space Plots for Generalized Linear Models", invited paper presented at the 2003 ASA
Quality and Productivity Research Conference, Yorktown Heights, NY, 21-23 May 2003.
107. Goldfarb, H. B., Borror, C. M., Montgomery, D. C., and Anderson-Cook, C. M. (2003),
"Graphical Methods to Assess the Prediction Capability of Mixture and Mixture-Process
Designs", invited paper presented at the 2003 ASA Quality and Productivity Research
Conference, Yorktown Heights, NY, 21-23 May 2003.
106. Montgomery, D. C. (2003), "Research Needs in Experimental Design", Invited Presentation at
the Journal Editors' Session, INFORMS National Meeting, San Jose, CA, 18-20 November
2002.
105. Janakirim, M. and Montgomery, D.C. (2002), "Integrating Engineering Process Control and
Statistical Process Control for Effective APC for Semiconductor Processes", Invited
Presentation at the ASQ/ASAFall Technical Conference, Valley Forge, PA, 17-18 October
2002.
104. Jearkpaporn, D., Montgomery, D. C., Runger, G. C., and Borror, C. M. (2002), "Process
Monitoring for Correlated Gamma Distributed Variables using GLM Based Control Charts",
Invited Presentation at the ASQ/ASA Fall Technical Conference, Valley Forge, PA, 17-18
October 2002.
103. Montgomery, D. C. (2002), "Education of Future (Industrial) Statistical Consultants", Invited
Presentation at the Joint Statistical Meetings, New York, 11-15 August 2002.
102. Kowalski, S., Borror, C. M., and Montgomery, D. C. (2002), "The Path of Steepest Ascent in
Split-Plot Experiments", Contributed Presentation at the Joint Statistical Meetings, New York,
11-15 August 2002.
101. Montgomery, D. C. (2002), "Teaching Experimental Design to Engineers: Some Experiences
and Advice", Invited Presentation at the 6th International Conference on Teaching Statistics,
Cape Town, South Africa, 7-12 July 2002.
100. Fowler, J. A. and Montgomery, D. C. (2002), "The Future of the IERC", Presentation at the
Industrial Engineering Research Conference, Orlando, Florida, 18-19 May 2002.
99. Montgomery, D. C. (2002), "Some Thoughts About Research", Invited Presentation at the
First IIE Doctoral Colloquium, Industrial Engineering Research Conference, Orlando, Florida,
18-19 May 2002.
98. Montgomery, D. C. (2002), "A Retrospective on Response Surface Methodology", invited
presentation at the Virginia Tech Conference on RSM in Honor of Professor Raymond H.
Myers, Blacksburg, Virginia, 19-20 April 2002.
97. Wisnowski, J. W., Runger, G. C., and Montgomery, D. C. (2001), "Enhanced Analysis of
Factorial Designs with Regression Trees", Invited Presentation at the 45th Annual Fall
Technical Conference, 18-19 October, Toronto, Canada.
96. Skinner, K. R., Runger, G. C., and Montgomery, D. C. (2001), Multivariate Control Charts for
Discrete Data", Invited Presentation at the 45th Annual Fall Technical Conference, 18-19
October, Toronto, Canada.
95. Montgomery, D. C. (2001), Invited Panelist for the Session "The 50th Anniversary of Response
Surface Methodology", 45th Annual Fall Technical Conference, 18-19 October, Toronto,
Canada.
RTI International
Peer Review of EPA's RSE Report
67

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
94. Holcomb, D. R., Jr. and Montgomery, D. C. (2001), "Some Difficulties in Analyzing Plackett-
Burman Design with Interactions", Invited Presentation at the 45th Annual Fall Technical
Conference, 18-19 October, Toronto, Canada.
93. Heredia-Langner, A., Carlyle, W. M., and Montgomery, D. C. (2001), "Genetic Algorithms for
the Construction of D-Efficient Designs", Invited Presentation at the 45th Annual Fall
Technical Conference, 18-19 October, Toronto, Canada.
92. Rejavelu, G., Montgomery, D. C., and Vining, G. G. (2001), "Graphical Design Evaluation
Techniques for Constrained Mixture Experiments", Invited Presentation at the 45th Annual
ASQ/ASAFall Technical Conference, 18-19 October, Toronto, Canada.
91. Heredia-Langner, A., Carlyle, W. M., and Montgomery, D. C. (2001), "Model-Robust Optimal
Designs Using Genetic Algorithms", Invited Presentation at the Joint Statistical Meetings, 5-9
August, Atlanta GA.
90. Myers, R. H. and Montgomery, D. C. (2001), "Analysis of Designed Experiments using
GLMs", Invited presentation at the Joint Statistical Meetings, Atlanta GA, 5-9 August.
89. Myers, R. H. and Montgomery, D. C. (2001), "Generalized Linear Models and Response
Surface Methods", Invited presentation at the ASA Quality and Productivity Research
Conference, Austin TX, 22-25 May.
88. Holcomb, D. R., Montgomery, D. C., and Carlyle, W. M. (2000), "Supersaturated Designs in
Product Design and Development", Invited presentation at the 44th Annual ASQC/ASA Fall
Technical Conference, Minneapolis, MN, 12-13 October, 2000.
87. Somerville, S. E., Montgomery, D. C., and Runger, G. C. (2000), "Filtering and Smoothing
Methods for Mixed Particle Count Distributions", Invited presentation at the 44th Annual
ASQ/ASAFall Technical Conference, Minneapolis, MN, 12-13 October, 2000.
86. Wisnowski, J. W., Simpson, J. R., Montgomery, D. C., and Runger, G. C. (2000), "Regressor
Variable Selection for Contaminated Data Sets", Invited presentation at the 44th Annual
ASQ/ASAFall Technical Conference, Minneapolis, MN, 12-13 October, 2000.
85. Montgomery, D. C., Loredo, E. N., Jearkpaporn, D., and Testik, M. C. (2000), "Experimental
Designs for Constrained Regions", Invited presentation at the 44fe Annual ASQ/ASA Fall
Technical Conference, Minneapolis, MN, 12-13 October, 2000.
84. Montgomery, D. C. (2000), "Some Opportunities and Challenges for Industrial Statisticians"
(Invited Keynote Address), Industrial Statistics in Action 2000, conference at the University of
Newcastle-Upon-Tyne, United Kingdom, 8-10 September, 2000.
83. Lewis, S. M., Montgomery, D. C. and Myers, R. H. (1999), "The Analysis of Designed
Experiments using Generalized Linear Models", Invited presentation at the 43rd Annual Fall
Technical Conference, Houston Texas, October 14-15.
82. Carlyle, W. M., Montgomery, D. C. and Runger, G. C. (1999), "Optimization Problems and
Methods in Quality Control and Improvement", Journal of Quality Technology session -
invited presentation at the 43rd Annual Fall Technical Conference, Houston Texas, October 14-
15.
81. Borror, C. M., Keats, J. B. and Montgomery, D. C. (1999), "Control Charts for Low Rates of
Process Nonconformance", Invited presentation at the 43rd Annual Fall Technical Conference,
Houston Texas, October 14-15.
80. Lanning, J., Montgomery, D. C. and Runger, G. C. (1999), "Adaptive Methods for Monitoring
Fractionally Sampled Multiple Stream Processes", Invited presentation at the 43rd Annual Fall
Technical Conference, Houston Texas, October 14-15.
RTI International
Peer Review of EPA's RSE Report
68

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
79. Wisnowski, J. W., Montgomery, D. C. and Simpson, J. R. (1999), "A Comparative Analysis of
Multiple Outlier Detection Procedures in the Linear Regression Model", Invited presentation
at the 43rd Annual Fall Technical Conference, Houston Texas, October 14-15.
78. Montgomery, D. C. (1999), "Statistical Methods for Process Robustness Studies", Inyong Ham
Distinguished Lecture, Department of Industrial and Manufacturing Engineering, Pennsylvania
State University, November 11.
77. Montgomery, D. C. (1999), "Statistical Methods for Achieving Six-Sigma Results", Invited
presentation at the Pharmaceutical and Medical Device Industries Conference on Six-Sigma,
Institute of International Research, Philadelphia, PA, September 23-24.
76. Vining, G. G., Kowalski, S. L. and Montgomery, D. C. (1999), "Hard-to-Change Design
Variables in a Response Surface Setting", Invited presentation at the Joint Statistical Meetings,
Baltimore, MD, August 8-12
75. Montgomery, D. C. (1999), "Multiple Response Optimization Methods", Invited presentation
at the Joint Statistical Meetings, Baltimore, MD, August 8-12.
74. Montgomery, D. C. (1998), "A Perspective on Models and the Quality Sciences: Some
Challenges and Future Directions", W. J. Youden Memorial Address presented at the 42nd
Annual ASQ/ASA Fall Technical Conference, 22-23 October, Corning, NY.
73. Zimmer, L. S., Montgomery, D. C., and Runger, G. C. (1998), "Some Guidelines for the
Application of Adaptive Control Charts", Invited presentation at the 42nd Annual ASQC/ASA
Fall Technical Conference, 22-23 October, Corning, NY.
72. Borror, C. M., Montgomery, D. C., and Myers, R. H. (1998), "Optimal Design Strategies for
Experiments Involving Noise Variables", Invited presentation at the 42nd Annual ASQC/ASA
Fall Technical Conference, 22-23 October, Corning, NY.
71. Montgomery, D. C. (1998), "Some Challenges and Opportunities for Industrial Statisticians"
Invited presentation at a Panel Discussion on Emerging Issues and Directions in Quality
Improvement, INFORMS, Seattle Washington, 27 October.
70. Montgomery, D. C. (1998), "Designed Experiments for Product and Process Development:
Some Examples", Invited presentation at INFORMS, Seattle Washington, 27 October.
69. Montgomery, D. C. (1998), "Experimental Design for Process and Product Design and
Development" Invited Keynote Address, Royal Statistical Society, Glasgow Scotland, 11
September.
68. Montgomery, D. C. and Vining, G. G. (1998), "Methods and Applications of Generalized
Linear Models", Invited Short Course presented for the Section on Engineering and Physical
Sciences, Joint Statistical Meetings, Dallas, TX, 11 August.
67. Montgomery, D. C. (1998), "Some Challenges for Industrial Statisticians", Invited
Presentation at the Joint Statistical Meetings, Dallas, TX, 10 August.
66. Montgomery, D. C. (1997), "Generalized Linear Models and Designed Experiments", Invited
plenary presentation at the Applied Probability and Statistics Day, Johns Hopkins University
Applied Physics Laboratory, 18 October, Laurel, MD.
65. Sebert, D. M., Montgomery, D. C., and D. A. Rollier (1997), "Identifying Multiple Outliers
and Influential Subsets in Linear Regression: A Clustering Approach", presented at the 41st
Annual ASQ/ASA Fall Technical Conference, 16-17 October, Baltimore, MD.
64. Montgomery, D. C. (1997), "Some Aspects of Generalized Linear Models for Designed
Experiments", Plenary Address, Nineteenth Annual Midwest Biopharmaceutical Statistics
Workshop, Ball State University, Muncie, Indiana.
RTI International
Peer Review of EPA's RSE Report
69

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
63. Montgomery, D. C. (1997), "Response Surface Methodology", invited tutorial presented at the
Southern California American Statistical Association Applied Statistics Workshop, Long
Beach, California.
62. Montgomery, D. C., and G. C. Runger (1996), "Multivariate Control Charts and Process
Monitoring", invited short course at the ASQ/ASA Fall Technical Conference, Scottsdale, AZ,
(sponsored by the Statistics Division of ASQ).
61. Montgomery, D. C. (1996), "Multiple Response Optimization Methods," invited presentation
at the 2nd Congress of the International Federation of Nonlinear Analysts, Athens, Greece.
60. Montgomery, D. C. (1995), "Response Surface Methods and Designs," invited short course at
the ASQC/ASA Fall Technical Conference, St. Louis, MO, (sponsored by the Statistic
Division of ASQC).
59. Montgomery, D. C. (1994), "Regression Analysis," invited short course at the ASQC/ASA
Fall Technical Conference, Birmingham, Alabama, (sponsored by the Statistics Division of
ASQC).
58. Montgomery, D. C. (1994), "Design of Experiments," invited short course at the 40th U.S.
Army Design of Experiments Conference, U.S. Military Academy, West Point, NY.
57. Montgomery, D. C. (1994), "Statistical Process Control for the Process Industries," invited
short course at the Joint Statistical Meetings, Toronto, Canada, (sponsored by the Quality and
Productivity Section of ASA).
56. Mastrangelo, C. M., and D. C. Montgomery (1994), "Shift Detection Properties of Moving-
Centerline EMWA Control Schemes", presented at the IIE Research Conference, Atlanta, GA.
55. Montgomery, D. C. (1994), "The Industrial Engineer and the Quality Improvement Sciences:
Have We Missed an Opportunity?", invited Keynote Address at the 8th Israeli Industrial
Engineering Conference, Beer Sheva, ISRAEL.
54. Montgomery, D. C. (1994), "Strategies for Integrating Statistical Process Control and
Engineering Process Control", presented at the Conference on Computer Integrated
Manufacturing in the Process Industries, Rutgers University.
53. Montgomery, D. C. (1993), "Planning, Conducting, and Analyzing Industrial Experiments,"
invited presentation at the 49th Annual Conference on Applied Statistics, Atlantic City, NJ.
52. Montgomery, D. C. (1993), "Solutions for Customer-Driven Quality Problems with Design of
Experiments", invited tutorial at the Fall ORSA/TIMS Conference.
51. J. B. Keats, D. C. Montgomery, G. C. Runger, and W. S. Messina (1993), "Strategies for
Integrating Statistical Process Control with Feedback (PID) Controllers", presented at the
ASQ/ASA Fall Technical Conference, Rochester, NY.
50. Del Castillo, E. and D. C. Montgomery (1993), "Methods for Finite-Horizon Process Control:
"Q" Charts and Alternative Techniques", presented at the ASQ/ASA Fall Technical
Conference, Rochester, NY.
49. Montgomery, D. C. and J. A. Heinsman (1993), "Optimization of Product Formulation Using
Mixture Experiments," invited presentation at the ORSA/TIMS Conference, Chicago, IL.
48. Montgomery, D. C., C. M. Mastrangelo and C. A. Lowry (1993), "Statistical Process
Monitoring for Aluminum Smelting," invited presentation at the 10th Annual Quality and
Productivity Research Conference, Knoxville, TN.
47. Montgomery, D. C., C. M. Mastrangelo and C. A. Lowry (1992), 'Statistical Process
Monitoring for Dynamic Systems," invited presentation at the IIE Research Conference, Los
Angeles, CA.
RTI International
Peer Review of EPA's RSE Report
70

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
46. Montgomery, D. C. and J. E. Taggart (1993), "Selection of a Second-Order Response Surface
Design," invited presentation at the SAS Users Group International Conference, New York.
45. Coleman, D. E. and D. C. Montgomery (1992), "A Systematic Approach to Planning for a
Designed Industrial Experiment," invited paper, Technometrics Session, ASQ/ASA Fall
Technical Conference, Philadelphia, PA.
44. Mastrangelo, C. M. and D. C. Montgomery (1992), "Characterization of a Moving Centerline
EWMA Control Chart," invited presentation at the ASQCASAFall Technical Conference,
Philadelphia, PA.
43. Montgomery, D. C. and S. R. Voth (1991), "Some Practical Aspects of Designing Mixture
Experiments." Invited presentation at the ASQ/ASA Fall Technical Conference, Lexington,
KY.
42. Montgomery, D. C. and C. M. Mastrangelo (1990), "Statistical Process Control Methods for
Autocorrelated Data." Invited paper, JQT Session, ASQ/ASA Fall Technical Conference,
Richmond, VA.
41. Montgomery, D. C. (1984), "Economic Models and Statistical Process control," invited
presentation at the Joint Statistical Meetings, Philadelphia, PA.
40. Montgomery, D. C. (1984), "Design of Experiments in Development and Manufacturing
Engineering," invited presentation at the 3rd Annual IBM Corporate Quality Conference,
Austin, TX.
39. Montgomery, D. C. (1984), "Improving Quality and Productivity in Manufacturing with
Design of Experiments," invited presentation at the 10th Annual IBM Design of Experiments
Conference, Lexington, Kentucky.
38. Montgomery, D. C. (1984), "Sampling Procedures for Monitoring Service Contracts," invited
paper given at the Spring ORSA Meeting, San Francisco, Calif.
37. Montgomery, D. C. and F. D. Baker (1983), "Statistical Modeling of Soybean Growth,"
Workshop on Crop Simulation, University of Illinois, Urbana-Champaign, Illinois.
36. Montgomery, D. C. (1983), "The Effect of Nonnormality on Acceptance Sampling Plans for
Variables," presented at a Meeting of the National Academy of Science, Washington, D.C.
35. Montgomery, D. C. and D. J. Friedman (1982), "An Evaluation of Biased Estimators for
Prediction," invited paper given at the Joint Statistical Meetings, Cincinnati, Ohio.
34. Montgomery, D. C. (1982), "Some Hazards of Using Regression Analysis as a Statistical Tool
for Load Research," invited paper given at the AEIC Load Research Conference, Atlanta,
Georgia.
33. Montgomery, D. C. (1981), "Cost Based Acceptance Sampling Plans and Process Control
Schemes," invited paper presented at the AIIE Fall Technical Conference, Washington, D. C.,
also in Conference Proceedings.
32. Montgomery, D. C. (1981), "Regression Analysis - Some Aspects of its Use in Load
Research," invited paper given at the AEIC Load Research Conference, Atlanta, Georgia.
31. Montgomery, D. C. and E. A. Peck (1981), "The Multicollinearity Problem in Regression,"
invited tutorial session at the Southeast Institute for Decision Sciences Meeting, Orlando,
Florida, February 1980; also in Conference Proceedings.
30. Montgomery, D. C. and G. Weatherby (1979), "Factor Screening Methods in Computer
Simulation," presented at the Winter Simulation Conference, San Diego, Calif., also in
Conference Proceedings.
RTI International
Peer Review of EPA's RSE Report
71

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
29. Montgomery, D. C. (1979), "Methods for Combining Forecasts," presented at the 24th
International TIMS Conference, Honolulu, Hawaii.
28. Johnson, L. A. and D. C. Montgomery (1979), "Forecasting Methods in Production and
Operations Management," invited paper presented at the 24th International TIMS Conference,
Honolulu, Hawaii.
27. Simms, E. D. and D. C. Montgomery (1977), "The Use of Discriminant Analysis for Risk
Assessment in Operational Testing," Presented at the 16th Annual U.S. Army Operations
Research Symposium, Ft. Lee, Virginia.
26. Russ, S. W., Jr., D. C. Montgomery, and H. M. Wadsworth, Jr. (1977), "A Cost Optimal
Approach to Selection of Experimental Designs for Operational Testing Under Conditions of
Constrained Sample Size," Presented at the 16th Annual U.S. Army Operations Research
Symposium, Ft. Lee, Virginia.
25. Friese, W. F., Jr., and D. C. Montgomery (1977), "A Cost-Optimal Approach to Selecting a
Fractional Factorial Design," presented at the 16th Annual U.S. Army Operations Research
Symposium, Ft. Lee, Virginia, also in Conference Proceedings.
24. Montgomery, D. C. (1977), "Procedures for Optimizing and Integrating Production and
Distribution Operations," invited paper presented at the 6th Management Science Colloquium,
Osaka University, Osaka, Japan.
23. Johnson, L. A. and D. C. Montgomery (1977), "Forecasting with Prediction Limits," invited
paper presented at the 23rd International TIMS Conference, Athens, Greece.
22. Montgomery, D. C. and V. M. Bettencourt, Jr. (1976), "A Review of Multiple Response
Surface Methods in Computer Simulation," invited paper presented at the Fall ORSA/TIMS
National Meeting, Miami, Florida.
21. Brown, E. L. and D. C. Montgomery (1975), "An Application of Network Simulation to
Operational Testing and Evaluations," presented at the 14th Annual U.S. Army Operations
Research Symposium, Ft. Lee, Virginia, also in Conference Proceedings.
20. Johnson, L. A. and D. C. Montgomery (1975), "Forecasting and Time Series Analysis,"
seminar presented at the 3rd Annual AIIE Fall Systems Engineering Conference, Las Vegas.
19. Johnson, L. A. and D. C. Montgomery (1975), "Planning Lot Size Production for Inventory,"
invited paper presented at the Fall ORSA/TIMS National Meeting, Las Vegas.
18. Gearing, D. V., R. G. Heikes and D. C. Montgomery (1975), "Development of an Economic
Model of Moving Average Control Charts," presented at the Fall 1975 ORSA/TIMS National
Meeting, Las Vegas.
17. Montgomery, D. C., R. G. Heikes, and Y. G. Yap (1975), "A Comparison of Two Adaptive
Forecasting Systems," presented at the 47th National ORSA Meeting, Chicago, Illinois.
16. Cummings, J. M., B. B. McCra, D. C. Montgomery and R. G. Heikes (1974), "Repairing
Response Surface Designs to Minimize Bias," presented at the 46th National ORSA Meeting,
San Juan, Puerto Rico.
15. Montgomery, D. C. (1974), "Experimental Design Techniques for Computer Simulation,"
invited paper at the Second Interamerican Conference on Information and Systems
Engineering, Mexico City.
14. Marsh, J. D. and D. C. Montgomery (1974), "Scheduling Jobs with Sequence Dependent Setup
Times on Parallel Machines," presented at the 45th National ORSA Meeting, Boston,
Massachusetts.
RTI International
Peer Review of EPA's RSE Report
72

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
13. Johnson, L. A. and D. C. Montgomery (1974), "On Dynamic Production Planning Models,"
invited paper presented at the Distinguished Scholars Seminar, Southeast Institute of Decision
Sciences Meeting, New Orleans, Louisiana, also in Conference Proceedings.
12. Montgomery, D. C. and C. K. Hudson (1973), "Use of Equiradial Designs in Response Surface
Methodology," presented at the 44th National ORSA Meeting, San Diego, California.
11. Heikes, R. G., D. C. Montgomery and J. Young (1973), "Alternate Process Models in the
Economic Design of T2 Control Charts," presented at the 44th National ORSA Meeting, San
Diego, California, subsequently published in AIIE Transactions.
10. Alt, F. B., J. J. Goode, D. C. Montgomery and H. M. Wadsworth (1973), "Variable Control
Charts for Multivariate Data," invited paper presented at the American Statistical Association
Meeting, New York.
9. Marsh, J. D and D. C. Montgomery (1973), "Optimal Procedures for Scheduling Jobs with
Sequence-Dependent Changeover Times on Parallel Processors," invited paper presented at the
AIIE Annual Conference, Chicago, Illinois.
8. Montgomery, D. C. and D. M. Evans (1972), "Second Order Response Surface Designs in
Digital Simulation," invited paper presented at the 41st National ORSA Meeting, New
Orleans, Louisiana, a revised version of this paper was subsequently published in Simulation.
7. Montgomery, D. C. and P. J. Klatt (1972), "Minimum Cost Multivariate Quality Control
Tests," invited paper presented at the AIIE Annual Conference, Anaheim, California, also in
Conference Proceedings and subsequently published in AIIE Transactions.
6. Montgomery, D. C. and H. M. Wadsworth (1972), "Some Techniques for Multivariate Quality
Control Applications," invited paper presented at the American Society for Quality Control
Annual Conference, Washington, D.C., also in Conference Proceedings.
5. Montgomery, D. C. (1971), "Stochastic Capacity Decision Models for Production Facilities,"
invited paper presented at the AIIE Annual Conference, Boston, Massachusetts, also in
Conference Proceedings.
4. Montgomery, D. C. (1970), "Expectations of Young Engineers from Their Employers and
Professional Societies," invited paper presented at the 13th International Meeting of APICS,
Cincinnati, Ohio.
3. Fabrycky, W. J., V. Chachra and D. C. Montgomery (1970), "A Simulation Study of Three
Classes of Job-Shop Sequencing Rules," invited paper presented at the 13th International
Meeting of APICS, Cincinnati, Ohio.
2. Ghare, P. M. and D. C. Montgomery (1969), "Flow Management in Transportation Networks,"
invited paper presented at the 5th International Conference on Operations Research, Venice,
Italy, also in Conference Proceedings.
1. Montgomery, D. C. (1970), "Evolutionary Operation and Machine Center Capacity Control in
Job-Shop Systems," contributed paper presented at the 11th American Meeting of TIMS, Los
Angeles, California.
Other Publications
59. Montgomery, D.C. and Anderson-Cook, C.M. (2016), "In Memory of Connie M. Borror",
Obituary in Quality Engineering, Vol. 28, No. 3, pp. 247-248.
58. Montgomery, D.C. (2016), "Why Do Lean Six Sigma Projects Sometimes Fail?", editorial in
Quality and Reliability Engineering International, Vol. 32, No. 4, pp. 1279.
RTI International
Peer Review of EPA's RSE Report
73

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
57. Montgomery, D.C. (2016), "Collecting Data", editorial in Quality and Reliability Engineering
International, Vol. 32, No. 2, pp. 333.
56. Montgomery, D.C. (2015), "Show Me the Money", editorial in Quality and Reliability
Engineering International, Vol. 31, No. 8, pp. 1303.
55. Montgomery, D.C. (2015), "Robert Vincent (Bob) Hogg", editorial in Quality and Reliability
Engineering International, Vol. 31, No. 4, pp. 555.
54. Montgomery, D.C. (2015), "A.V. Feigenbaum", editorial in Quality and Reliability Engineering
International, Vol. 31, No. 2, pp. 163.
53. Montgomery, D.C. (2014), "Big Data and the Quality Profession", editorial in Quality and
Reliability Engineering International, Vol. 30, No. 4, pp. 447.
52. Montgomery, D.C. (2014), "Lean Six Sigma and Promoting Innovation", editorial in Quality and
Reliability Engineering International, Vol. 30, No. 1, pp. 1.
51. Montgomery, D.C. (2013), "Lean Six Sigma and Quality Management", editorial in Quality and
Reliability Engineering International, Vol. 29, No. 7, pp. 935.
50. Montgomery, D.C. (2013), "2013: The International Year of Statistics", editorial in Quality and
Reliability Engineering International, Vol. 29, No. 3, pp. 305.
49. Montgomery, D.C. (2013), "The Quality, Reliability and Statistical Engineering Profession in the
21st Century", editorial in Quality and Reliability Engineering International, Vol. 29, No. 1, pp.
1.
48. Montgomery, D.C. (2012), "Giants of Quality - W. Edwards Deming", editorial in Quality and
Reliability Engineering International, Vol. 28, No. 3, pp. 247-248.
47. Montgomery, D.C. (2011), "Giants of Quality - Walter Shewhart", editorial in Quality and
Reliability Engineering International, Vol. 27, No. 8, pp. 979.
46. Montgomery, D.C. (2011), "Innovation and Quality Technology", editorial in Quality and
Reliability Engineering International, Vol. 27, No. 6, pp. 733-734.
45. Montgomery, D.C. (2011), The Principles of Testing", The ITEA Journal, invited editorial, Vol
32, No. 3, pp. 231-234.
44. Montgomery, D.C. (2010), "The 25th Anniversary Volume of Quality and Reliability Engineering
International', editorial in Quality and Reliability Engineering International, Vol. 26, No. 1, pp.
1-2.
43. Montgomery, D.C. (2009), "Computer Modelling", editorial in Quality and Reliability
Engineering International, Vol. 25, No. 6, pp. 645.
42. Montgomery, D.C. (2009), "It's a Great Time to be a Statistician", editorial in Quality and
Reliability Engineering International, Vol. 25, No. 4, pp. 379-380.
41. Tiwari, M.K., Antony, J., and Montgomery, D. C. (2008), "Editorial Note for the Special Issue on
Effective Decision Support to Implement Lean and Six Sigma Methodologies in the
Manufacturing and Service Sectors", International Journal of Production Research, Vol. 46, No.
23, pp. 6563-6566.
40. Montgomery, D.C. (2008), "Applications of Design of Experiments in Engineering", editorial in
Quality and Reliability Engineering International, Vol. 24, pp. 501-502.
39. Montgomery, D.C. (2008), "Does Six Sigma Stifle Innovation?", editorial in Quality and
Reliability Engineering International, Vol. 24, pp. 249.
RTI International
Peer Review of EPA's RSE Report
74

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
38. Montgomery, D.C. (2008), "A Retrospective on Volume 23 of Quality and Reliability
Engineering International', editorial in Quality and Reliability Engineering International, Vol.
24, pp. 1-2.
37. Montgomery, D.C. (2007), "SPC Research - Current Trends", editorial in Quality and
Reliability Engineering International, Vol. 23, pp. 515-516.
36. Montgomery, D. C. (2006), "Designed Experiments in Process Improvement", editorial in
Quality and Reliability Engineering International, Vol. 22, No. 8, pp. 863-864.
35. Montgomery, D. C. (2006), "Analyzing and Improving Measurement Systems: A Key to
Effective Decision-Making", editorial in Quality and Reliability Engineering International, Vol.
22, No. 3, pp. 237-238.
34. Montgomery, D. C. and Brombacher, A.C. (2006), "Carol J. Feltz and David Newton", editorial
in Quality and Reliability Engineering International, Vol. 22, No. 2, pp. i.
33. Brombacher, A. C. and Montgomery, D. C. (2005), "News from Newcastle: Product Quality
from a Customer Perspective", editorial in Quality and Reliability Engineering International,
Vol. 21, No. 8, pp. iii.
32. Montgomery, D. C. (2005), "Generation III Six Sigma", editorial in Quality and Reliability
Engineering International, Vol. 21, No. 6, pp. iii-iv.
31. Montgomery, D. C. (2005), "Changing of the Guard", editorial in Quality and Reliability
Engineering International, Vol. 21, No. 1, pp. iii.
30. Montgomery, D. C. (2004), "Selecting the Right Improvement Projects", editorial in Quality
and Reliability Engineering International, Vol. 20, No. 7, pp. iii-iv.
29. Montgomery, D. C. (2004), "Improving Business Performance: Project-by-Project", editorial in
Quality and Reliability Engineering International, Vol. 20, No. 4, pp. iii.
28. Montgomery, D. C. (2003), "Corporate Ethics and Quality", editorial in Quality and Reliability
Engineering International, Vol. 19, No. 6, pp. iii-iv.
27. Montgomery, D. C. (2003), "Quality Improvement and Economic Growth", editorial in Quality
and Reliability Engineering International, Vol. 19, No. 3, pp. iii.
26. Montgomery, D. C. (2003), review: The Mahalanobis-Taguchi Strategy, G. Taguchi and R.
Jugulum, in: Journal of Quality Technology, Vol. 35, No. 2.
25. Montgomery, D. C. (2003), "Education for Industrial Statisticians", editorial in Quality and
Reliability Engineering International, Vol. 19, No. 1.
24. Montgomery, D. C. (2002), "Changing Roles for the Industrial Statistician", editorial in Quality
and Reliability Engineering International, Vol. 18, No. 5.
23. Montgomery, D. C. (2002), "Research in Industrial Statistics - Part II", editorial in Quality and
Reliability Engineering International, Vol. 18, No. 2.
22. Montgomery, D. C. (2001), "Research in Industrial Statistics - Part I", editorial in Quality and
Reliability Engineering International, Vol. 17, No. 6.
21. Montgomery, D. C. (2001), "Beyond Six-Sigma", editorial in Quality and Reliability
Engineering International, Vol. 17, No. 4.
20. Montgomery, D. C. (2001), "Some Thoughts on ISO/QS Registration", editorial in Quality and
Reliability Engineering International, Vol. 17, No. 1.
19. Borror, C. M., Montgomery, D. C., and Runger, G. C. (2000), "Statistical Experimental Design
- Some Recent Advances and Applications", editorial in Quality and Reliability Engineering
International, Vol. 16, No. 5.
RTI International
Peer Review of EPA's RSE Report
75

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
18. Montgomery, D. C. (2000), "The Present State of Industrial Statistics", editorial in Quality and
Reliability Engineering International, Vol. 16, No. 4.
17. Montgomery, D. C. (2000), "A Meeting for Industrial Statisticians", editorial in Quality and
Reliability Engineering International, Vol. 16, No. 2.
16. Montgomery, D. C. (1988), "Experimental Design and Product and Process Development,"
Manufacturing Engineering, Vol. 101, No. 3, September.
15. Heikes, R. G. and D. C. Montgomery (1981), "Productivity is Enhanced by Statistical Quality
Control," Industrial Engineering, Vol. 13, No. 3.
14. Montgomery, D. C. (1981), review: Dynamic Regression: Theory and Algorithms, M. H.
Pearson and L. J. Slater, in: Journal of Quality Technology, Vol. 13, No. 1.
13. Montgomery, D. C. (1980), review: Practical Experiences with Modeling and Forecasting
Time Series, G. M. Jenkins, In: Journal of Quality Technology, Vol. 12, No. 1.
12. Montgomery, D. C. (1977), review: Statistical Methods for Digital Computers, K. Enslein, A.
Ralstan and H. S. Wolf, eds., in: TIMS Interfaces.
11. Montgomery, D. C. (1977), review: Fundamentals of Finite Mathematics, R. L. Childress, in:
Interfaces, Vol. 8, No. 1.
10. Montgomery, D. C. (1975), review: Engineering Mathematics, A. C. Bajapi, L. R. Mustoe and
D. Walker, In: Industrial Engineering, Vol. 7, No. 1.
9. Montgomery, D. C. (1975), review: Industrial Systems: Planning, Analysis, and Control,
David D. Bedworth, in: Industrial Engineering, Vol. 7, No. 1.
8. Johnson, L. A. and D. C. Montgomery (1973), review: An Introduction to Production and
Inventory Control, and Production and Inventory Control: Theory and Practice, R. W. Van
Ness and W. Monhemius, in: Industrial Engineering, Vol. 5, No. 12.
7. Montgomery, D. C. (1970), review: An Illustrated Guide to Linear Programming, S.I. Gass, in:
Industrial Engineering, Vol. 12, No. 8.
6. Montgomery, D. C. (1970), review: Theory of Games and Strategies, Levin and DesJardins, in:
Industrial Engineering, Vol. 2, No. 7.
5. Montgomery, D. C. (1969), review: Queuing Theory, J. A. Panico, in: Industrial Engineering,
Vol. 1.
4. Montgomery, D. C. (1968), review: Fundamentals of Operations Research, Ackoff and Sasieni,
The Journal of Industrial Engineering, Vol. 19, No. 7.
3. Montgomery, D. C. (1968), review: Operations Research and the Design of Management
Information Systems, J. F. Pierce, ed., The Journal of Industrial Engineering, Vol. 19, No. 4.
2. Montgomery, D. C. (1971), "Simulation Predicts Product Behavior," Machine Design.
1. Montgomery, D. C. and W. L. Berry (Editors) (1974), Production Planning and Control:
Concepts, Techniques and Systems, Production Planning and Control Division Monograph No.
1, American Institute of Industrial Engineers.
RESEARCH STUDENTS SUPERVISED
Ph.D. Dissertations
68. Sarah E. Burke, "Optimal Design of Experiments for Dual-Response Systems"
RTI International
Peer Review of EPA's RSE Report
76

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
67. Michelle V. Mancenido, "Categorical Responses in Mixture Experiments", co-advisor with Rong
Pan
66. Edgar Hassler, "Bayesian D-Optimal Design Issues and Optimal Design Construction for
Generalized Linear Models with Random Effects", co-advisor with Rachel Silvestrini
65. Azadeh Adibi, "A P-Value Approach for Phase II Profile Monitoring", co-advisor with Connie M.
Borror.
64. Brian B. Stone, "No-Confounding Designs of 20 and 24 Runs for Screening Experiments and a
Design Selection Methodology", co-advisor with Rachel Silvistrini.
63. Kathryn S. Kennedy, "Bridging the Gap Between Space-Filling Designs and Optimal Designs:
Designs for Computer Experiments", co-advisor with Rachel Silvistrini.
62. Shilpa M. Shinde, "Projection Properties and Analysis Methods for Six-to-Fourteen Factor No-
Confounding Designs in 16 Runs".
61. Richard B. Abelson, "The Development of a Validated Clinically Meaningful Endpoint for the
Evaluation of Tear Film Stability as a Measure of Ocular Surface Protection in the Diagnosis and
Treatment of Dry Eye Disease".
60. Joseph M. Juarez, "Accelerated Life Testing of Electronic Circuit Boards with Applications in
Lead-Free Design".
59. Brett Duarte, "An Analytical Approach to Lean Six Sigma Deployment Strategies: Project
Identification and Prioritization", co-advisor with John Fowler.
58. Dana C. Krueger, "Semiconductor Yield Modeling using Generalized Linear Models".
57. Busaba Laungrungrong, "Multivariate Charts for Multivariate Poisson-Distributed Data".
56. Shilpa Gupta, "Profile Monitoring - Control Chart Schemes for Monitoring Linear and Low-Order
Polynomial Profiles".
55. Tae-Yeon Cho, "Mixture-Process Variable Design Experiments with Control and Noise Variables
within a Split-Plot Structure".
54. James R. Broyles, "Markovian Model of Patient Throughput in Hospitals: A Regression and
Decision Process Approach", co-advisor with Jeff Cochran.
53. Eric M. Monroe, "Optimal Experimental Designs for Accelerated Life Tests with Censoring and
Constraints", co-advisor with Rong Pan.
52. Rachel T. Johnson, "Experimental Designs for Computer Experiments", co-advisor with John
Fowler.
51. Capehart, S. R., "Designing Fractional Factorial Split-Plot Experiments Using Integer
Programming", co-advisor with Murat Kulahci.
50. Myrta Rodriguez, "Evaluation and Construction of Optimal Experimental Designs for Fitting
Response Surface Models", co-advisor with Connie Borror.
49. Ashraf Almimi, "Split-Plot Designs: Follow-Up Experiments, Missing Observations, and Model
Adequacy Checking", co-advisor with Murat Kulahci.
48. Russell Elias, "Demand Model Management: A Model-Based Expert System for the Forecasting of
Semiconductor Product".
47. Peter J. Chung, "Mixture-Process Experiments with Continuous and Categorical Noise Variables".
46. Cathy Lawson, "Business Process Characterization using Categorical Data Models".
45. You-Jin Park, "Application of Genetic Algorithms in Response Surface Optimization Problems".
RTI International
Peer Review of EPA's RSE Report
77

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
44. Duangporn Jearkpaporn, "Multivariate Process Monitoring using Generalized Linear Models", co-
advisor with G. C. Runger.
43. David C. Drain, "Response Surface Methods for Experiments Involving Correlated Noise
Variables".
42. Heidi B. Goldfarb, "Mixture-Process Experiments with Control and Noise Variables".
41. Katina R. Skinner, "Multivariate Process Control for Discrete Data", co- advisor with G. C. Runger.
40. Alejandro Heredia-Langner, "Genetic Algorithms in Quality Control Problems", co- advisor with
W. M. Carlyle.
39. Elvira N. Loredo, "Annual Electrical Peak Load Forecasting Methods with Measures of Prediction
Error".
38. Mani Janakiram, "Statistical and Engineering Process Control Integration Strategies for
Constrained Controllers".
37. Harendra Shah, "Impact of Correlated Responses on the Desirability Function", co- advisor with
W. M. Carlyle.
36. Geetha Rajavelu, "Graphical Design Evaluation Techniques for Constrained Mixture
Experiments".
35. Lisa Custer, "Augmenting Experimental Designs: Approaches and Comparisons".
34. Kelly G. Canter, "Screening Methods for Life Cycle Inventory Models", co-advisor with W. M.
Carlyle.
33. Don R. Holcomb, Jr., "Supersaturated Experiments for use in Product Development", co-advisor
with W. M. Carlyle.
32. Leonard A. Perry, "Partition Experimental Designs for Sequential Processes".
31. Teri Reed Rhoads, "An Investigation of Strategies for Multivariate Monitoring of Continuous
Processes".
30. Daniel R. McCarville, "Test Guard Band Probability Models and Strategies for Optimization".
29. Wisnowski, James W., "Multiple Outliers in Regression: Detection Methods, Robust Estimation,
and Variable Selection", co-advisor with G. C. Runger.
28. Steven E. Somerville, "Filtering and Monitoring Methods for Univariate and Multiple Stream
Processes", co-advisor with G. C. Runger.
27. Jeffrey W. Lanning, "Methods for Monitoring Fractionally Sampled Multiple Stream Processes",
co-advisor with G. C. Runger.
26. Connie M. Borror, "Response Surface Methods for Experiments Involving Noise Variables".
25. Sharon L. Lewis, "Analysis of Designed Experiments using Generalized Linear Models".
24. Wade E. Molnau, "Economic Statistically Constrained Design of the Multivariate Exponentially
Weighted Moving Average Control Chart", co-advisor with G. C. Runger.
23. Lora S. Zimmer, "Contributions to Adaptive Control Charts", co-advisor with G. C. Runger.
22- Daryl J. Hauck, "Extensions to Regression Adjustment Techniques in Multivariate Process
Monitoring", co-advisor with G. C. Runger.
21. Dale J. Kennedy, "Development and Analysis of Stochastic Environmental Life Cycle Assessment
Inventory Modeling", co-advisor with D. A. Rollier.
20. David M. Sebert, "A Clustering Approach to Finding Multiple Outliers in Linear Regression".
RTI International
Peer Review of EPA's RSE Report
78

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
19. Carole Shlaes, "Use of Chance-Constrained Programming Techniques to Determine Optimal
Insurance Deductible Levels", co-advisor with J. B. Keats.
18. James R. Simpson, "New Methods and Comparative Evaluations for Robust and Biased-Robust
Regression Estimation".
17. Jose Andere-Rendon, "Computer-Aided Robust Design of Mixture Experiments Based on
Bayesian D-Optimality," co-advisor with D. A. Rollier.
16. Diane Schaub, "Experimental Design Strategy: An Application to Rapid Prototyping of Aerospace
Parts".
15. Sharad S. Prabhu, "Adaptive Sample Size and Adaptive Sampling Interval Schemes for an X-Bar
Control Chart".
14. Christina M. Mastrangelo, "Statistical Process Monitoring for Autocorrelated Data".
13. Enrique Del Castillo, "Some Models and Methods for Statistical Process Control in Short-Run
Manufacturing Systems".
12. William S. Messina, "Strategies for the Integration of Engineering Process Control and Statistical
Process Control," co-advisor with J. B. Keats.
11. James C. C. Torng, "The Economic Statistical Design of Variables Control Charts with an
Application to Exponentially Weighted Moving Average Charts," co-advisor with J. K. Cochran.
10. D. Y. Kim, "Economic Statistical Design and Analysis for the Poisson CUSUM Control Chart,"
co-advisor with J. B. Keats.
9. M. K. Chua, "A Control Scheme for Multivariate Quality Control".
8. John S. Gardiner, "Statistical Process Control Methods for Detecting Small Process Shifts with
Applications to Integrated Circuit Manufacturing".
7. Steven A. Yourstone, "Real-Time Process Quality Control in a Computer-Integrated
Manufacturing Environment".
6. David J. Friedman, "An Evaluation of Biased Estimators in Prediction".
5. Rickey A. Kolb, "Robustness of Current Methodologies for the Analysis of Contingency Tables
with Respect to Small Expected Cell Values".
4. Ginner Weatherby, "Aggregation, Disaggregation and Combination of Forecasts".
3. Joseph D. Marsh, Jr., "Scheduling Parallel Processors".
2. Michael P. Deisenroth, "On Simulation Methodology in Vehicular Traffic Flow".
1. Ronald G. Askin, "The Combination of Biased and Robust Estimation Techniques in Multiple
Regression Models".
M.S. Theses and MS Statistics Projects
48. Archana Krishnamoorthy, "Analysis of No-Confounding Designs using the Dantzig Selector."
47. Jeanne Huddleston, "Harm During Hospitalizations for Heart Failure: Adverse Events as a
Reliability Measure of Hospital Policies and Procedures", co-advisor John Fowler.
46. Dean S. Hoskins, "D-Optimal Designs with Interaction Coverage", MS Statistics Project.
45. Shilpa Madhavan Shinde, "Statistical Analysis and Optimization of a Microfluidic Mixing Process
Based on the Bubble-Induced Acoustic Microstreaming Principle for a Gene Expression Assay."
RTI International
Peer Review of EPA's RSE Report
79

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
44. Andrea M. Archer, "Bootstrap Confidence Regions on the Optimum Point of a Quadratic
Response Surface", MS Statistics Project.
43. Jiahong Li, "Comparisons of Prediction Variance Properties of Six-to Ten-Factor CCD and CCD
Min Res V Response Surface Designs", MS Statistics Project.
42. Nuttha Lurponglukana, "Using a Bootstrap Method to Determine Active Factors in an
Unreplicated Factorial Experiment".
41. Amy K. Volpe, "Fitting Response Surface Models with Mixed Effects using Generalized Linear
Mixed Models", MS Statistics Project.
40. Diane E. Richardson, "Variance Dispersion Studies of Second Order Response Surface Designs in
Cuboidal Regions", MS Statistics Project.
39. Russell J. Elias, "Demand Signal Modeling: A Model-Based Approach to the Forecasting of
Future Product Demand", co - advisor J. B. Keats.
38. Duangporn Jearkpaporn, "Using Half-Normal Plots to Identify Important Factors in Screening
Experiments Analyzed with the Generalized Linear Model".
37. Sharlyn R. Stocker, "Measurement Capability with Wear Out", co-advisor, George C. Runger.
36. John A. Druyor, "An Investigation of Linear Screening and Prediction Accuracy in Constrained
Mixture Experiments".
35. Pamela A. Okamoto, "Parameter Estimation from Hazard Plots using Robust Regression
Techniques", co-advisor with J. B. Keats.
34. M. J. Yatskievych, "Integrating Statistical Process Control with Feedforward Control," co-advisor
with J. B. Keats.
33. Richard D. Scranton, "Enhancements to the Multivariate Exponentially Weighted Moving
Average Control Chart," co - advisor J. B. Keats.
32. Steven E. Somerville, "Process Capability Ratios and Non-normal Distributions".
31. Daniel R. McCarville, "Defect Levels and Losses due to Gauge Error".
30. James E. Taggart, "A Comparison of Some Second-Order Response Surface Designs Based on
Rotability and Prediction Variance".
29. Nora B. Peterschmidt, "Comparison of Regression Estimates in Mixture Experiments".
28. Sheila R. Voth, "Leverage, Multicollinearity and Bias in Response Surface Designs for Mixtures".
27. Robert Gilby, "A Wald Sequential Ratio Test, Modified to Ignore Small Shifts in the Process
Variable".
26. Christina M. Mastrangelo, "Statistical Process Control Methods for Autocorrelated Data".
25. Deborah Garner, "Regression Diagnostics for Influence with Many Influential Cases".
24. Timothy G. Fields, "Nonlinear Programming Techniques for the Multiple Response Problem".
23. Beth Quay, "Investigation of Methods for Determining Influential Data Points in Regression
Analysis".
22. Ronda K. Martin, "Interactive Regression Diagnostics".
21. Margaret Panagos, "Economic Design of the x Chart for Two Process Models".
20. Richard Matteson, "Minimum Bias Estimation of the Slope of a Response Surface".
19. Joseph F. Mance, "Economic Design of Fraction Defective Control Charts to Maintain Current
Control of a Process".
18. Phillip J. Klatt, "Design of Control Charts for the Mean Vector of a Multivariate Normal Process".
RTI International	Peer Review of EPA's RSE Report	80

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
17. A. K. Keswani, "Single Item Inventory Models for Back Orders and Lost Sales".
16. Mari Krista Jones, "A Comparison of Direct Smoothing with Short Term Forecasting Techniques
for Periodic Data".
15. James M. Jerkins, "Some Algorithms for Nonlinear Regression".
14. William F. Friese, "A Cost Optimal Approach to Selecting a Fractional Factorial Design".
13. Claude K. Hudson, "Use of the Class of Equiradial Designs as Second Order Response Surface
Designs".
12. David E. Ferguson, "The Development of an Adaptive Prediction and Control System".
11. Daniel M. Evans, Jr., "The Use of Second Order Response Surface Designs in Digital Simulation".
10. Joseph M. Cummings, "Repairing Response Surface Experiments to Minimize Bias".
9. Geneveive M. Cruz, "A Statistical Approach to the Combination of Forecasts".
8. Philip V. Coyle, "An Adaptation of Bayesian Statistical Methods to the Determination of Optimal
Sample Sizes for Operational Testing".
7. Elwyn L. Brown, "An Application of Simulation Networking Techniques in Operational Test
Design and Evaluation".
6. Jimmie K. Boles, "A GPSS II Simulation of an Air Defense Problem".
5. Joan S. Horwitz, "A Mathematical Model of an Epidemic Process".
4. Richard Harris, "Economic Design of T2 Control Charts for Multivariate, Multi-State Processes".
3. Timothy G. Fields, "Nonlinear Programming Techniques for the Multiple Response Problem".
2. Robert M. Baker, "An Application of Bayesian Statistical Methods in the Determination of
Sample Size for Operational Testing in the U.S. Army".
1. Frank B. Alt, "Some Aspects of Multivariate Statistical Control Charts".
RTI International
Peer Review of EPA's RSE Report
81

-------
Douglas Montgomery
Peer Reviewer Resume
Last Updated 9/5/17
DOUGLAS C. MONTGOMERY
Biographical Data
Dr. Douglas C. Montgomery is Regents' Professor and the ASU Foundation Professor of Engineering at
Arizona State University. He held the John M. Fluke Distinguished Chair in Engineering, was the Director
of Industrial Engineering and Professor of Mechanical Engineering at the University of Washington in
Seattle. He was a Professor of Industrial & Systems Engineering at the Georgia Institute of Technology.
He holds BSIE, MS and Ph.D. degrees from Virginia Polytechnic Institute.
Professor Montgomery's professional interests are in industrial statistics, including design of experiments,
quality control, applications of linear models, and time series analysis and forecasting. He also has interests
in operations research and statistical methods applied to modeling and analyzing manufacturing systems.
He has lectured extensively throughout the Americas, Europe and the Far East. He was a Visiting
Professor of Engineering at the Monterey Institute of Technology in Monterey, Mexico, and a University
Distinguished Visitor at the University of Manitoba. Professor Montgomery has conducted basic research
in empirical stochastic modeling, process control, and design of experiments. The Department of Defense,
the Office of Naval Research, the National Science Foundation, the United States Army, and private
industry have sponsored his research. He has supervised 66 doctoral dissertations and more than 40 MS
theses and MS Statistics projects.
Professor Montgomery is an author of thirteen textbooks that have appeared in over 35 English editions and
numerous foreign languages, including Design and Analysis of Experiments, 8th edition (2012),
Introduction to Statistical Quality Control, n edition (2012), Generalized Linear Models, 2nd edition (2010,
with R. H. Myers, G. G. Vining and T.J. Robinson), Engineering Statistics, 5th edition (2011, with G. C.
Runger and N. F. Hubele), Applied Statistics and Probability for Engineers, 5th edition (2011, with G.C.
Runger), Introduction to Linear Regression Analysis, 5th edition (2012, with E. A. Peck and G. G. Vining),
and Response Surface Methodology, 3rd edition (2009, with R. H. Myers and C.M. Anderson-Cook). He has
edited or coauthored seven other research books or edited volumes. His research papers have appeared in
many journals, including the Journal of Quality Technology, Technometrics, Management Science, Naval
Research Logistics Quarterly, HE Transactions, Journal of the Royal Statistical Society, Communications
in Statistics, IEEE Transactions on Reliability, Quality and Reliability Engineering International, IEEE
Transactions on Semiconductor Manufacturing, Quality Engineering, Operational Research Quarterly,
International Journal of Production Research, Journal of Spacecraft and Rockets, and Transportation
Research. He is a past Editor of the Journal of Quality Technology and one of the current chief editors of
Quality and Reliability Engineering International. He has served as the Applied Probability and Statistics
Department Editor and as the Quality and Reliability Engineering Department Editor for HE Transactions.
He is a member of the Editorial Board of the Journal of Quality Technology, the Journal of Applied
Statistics, Quality Engineering, the Journal of Probability and Statistical Science, and Quality Technology
and Quantitative Management.
Professor Montgomery's industrial experience includes engineering assignments with Union Carbide
Corporation and Eli Lilly and Company. He also has extensive consulting experience with many national
and international organizations.
Professor Montgomery is an Honorary Member of the American Society for Quality, a Fellow of the
American Statistical Association, a Fellow of the Royal Statistical Society, a Fellow of the Institute of
Industrial Engineers, an Elected Member of the International Statistical Institute, and an Elected
Academician of the International Academy for Quality. He has held several national offices in ASQ, ASA,
and IIE. He is a member of the honorary societies Phi Kappa Phi, Sigma Xi, Mu Sigma Rho, and Alpha
Pi Mu. He received the Deming Lecture Award from the American Statistical Association. He is a
recipient of the Shewhart Medal, the Distinguished Service Medal, the William G. Hunter Award, the
Brumbaugh Award, the Lloyd S. Nelson Award, and two Shewell Awards from the American Society for
Quality, the George Box Medal from ENBIS, the Greenfield Medal from the Royal Statistical society, and
the Ellis R. Ott Award. He has also received several outstanding teaching awards, including the Arizona
RTI International
Peer Review of EPA's RSE Report
82

-------
Douglas Montgomery	Peer Reviewer Resume	Last Updated 9/5/17
State University Engineering College Graduate Teaching Excellence Award in 1994. He was named an
ASU Outstanding Doctoral Mentor in 2004.
RTI International
Peer Review of EPA's RSE Report
83

-------
Appendix B. Charge Letter
RTI International
Peer Review of EPA's RSE Report
84

-------
3040 E. Cornwallis Road • PO Box12194 • Research Triangle Park, NC 27709-2194 • USA
Telephone+1.919.541.6000 • Fax+1.919.541.5985 • www.rti.org
October 30, 2017
Dr. Sanya Carley
School of Public and Environmental Affairs
Indiana University
1315 E. 10th Street
Bloomington, IN 47405
SUBJECT: Peer Review of EPA" s Response Surface Equation Report
Dear Dr. Carley,
RTI International has been contracted by EPA to facilitate a peer review of their Response Surface
Methodology Report. You have been selected to participate on this panel and your conflict of interest
evaluation is complete. RTI will compensate you $3,000 for your services.
This charge letter contains specific questions to guide you in your review, the review schedule, and details
about the materials we would like you to send us by the end of the three-week review period.
Additionally, you should receive the peer review materials in the same email that delivered this letter.
Background
EPA's Office of Transportation and Air Quality has developed a statistical approach to access results
from the Advanced Light-Duty Powertrain and Hybrid Analysis (ALPHA) model. To demonstrate the
credibility of the methodology and gain acceptance in the light-duty automotive community, EPA has
decided to initiate an independent peer review.
The ALPHA model is a full vehicle simulation model which is used to assess the effectiveness of
different technology packages in vehicles. Effectiveness values from ALPHA act as robust inputs to the
Optimization Model for Reducing Emissions of Greenhouse Gases from Automobiles (OMEGA) as well
as the overall rulemaking process.
Because operating the ALPHA model in real time to conduct full vehicle simulations is cost- and time-
prohibitive, EPA has developed a method of deriving the necessary effectiveness values using a statistical
methodology known as a Response Surface Model (RSM). An RSM is used to computationally
synthesize a large set of simulation outputs to derive response surface equations (RSE). The derived RSEs
can then be used in place of running the ALPHA model in real time.
HRTI
INTERNATIONAL
RTI International
Peer Review of EPA's RSE Report
85

-------
Charge Questions
For this review, EPA is looking for the reviewer's opinion of the use of response surface modeling to
access the results of the ALPHA model for use in the OMEGA model. The ALPHA and OMEGA models
themselves are not part of the review and no independent data analysis is required.
EPA would like you to consider the questions below to help define the scope of the review. You are not
expected to respond to the questions individually; instead, they should be considered a guide for your
response.
General Questions and Issues to Consider
1.	EPA's overall approach to applying response surface modelling to accessing ALPHA model
results and whether the resulting response surface equations provide accurate and robust inputs
for the OMEGA model.
2.	Reasonableness of any assumptions, implicit or explicit, contained in EPA's execution of the
methodology.
3.	Clarity, completeness and accuracy of the technical application of response surface modelling;
and
4.	Any recommendations for specific improvements to the functioning or the quality of the
methodology.
In your review, please identify any recommendations that would improve the methodology, clearly
distinguishing between specific improvements that can be readily made using available data and
literature, and improvements that are more theoretical or exploratory, which would rely on data or
literature not readily available to the EPA. Comments should be detailed enough that EPA readers or
others familiar with the report can understand the comments' relevance to the Response Surface Equation
Report.
Schedule
The schedule for this peer review is as follows:
•	October 30th, 2017: Charge letter distributed to reviewers
•	October 31st, 2017: Peer Review Kick-Off Call
•	Date TBD: Mid-review conference call
•	November 22nd, 2017: Comment/review due via email to Kyle Clark-Sutton at kcs@rti.org.
Materials
Upon completion of your review, you should submit your report under a cover letter that states:
1) Your name
RTI International
Peer Review of EPA's RSE Report
86

-------
2)	The name and address of your organization
3)	A statement of any real or perceived conflict(s) of interest.
Should you have any questions or concerns, feel free to contact me via phone at 919-541-5874 or by
email. In addition, the EPA project manager for this effort is Jeff Cherry and he may be reached at 734-
214-4371 orcherry.jeff@epa.gov.
For any questions about the review process itself, please contact Ruth Schenk in EPA's Quality Office,
National Vehicle and Fuel Emissions Laboratory at 734-214-4017 or schenk.ruth@epa.gov.
Thanks for your participation!
Sincerely,
Kyle Clark-Sutton
Research Economist, RTI International
(919)541-5784
kcs@rti.org
RTI International
Peer Review of EPA's RSE Report
87

-------
Appendix C. Sanya Carley Comments
RTI International
Peer Review of EPA's RSE Report
88

-------
November 16, 2017
Kyle Clark-Sutton
Research Economist
RTI International
RE: Peer Review of EPA's Response Surface Equation Report
Dear Mr. Clark-Sutton,
Thank you for the invitation to conduct a peer review of the EPA's Response Surface Equation
Report. I am an associate professor in the School of Public and Environmental Affairs at Indiana
University. My work address is presented below.
Please find a summary of my review enclosed in this submission package. These comments and
recommendations are based on my understanding of Response Surface Methodology, cost-
effectiveness of different vehicle technology packages, the use of input parameters and the
operation of the OMEGA model, and U.S. fuel economy and greenhouse gas emissions standards
for light-duty vehicles between 2017 and 2025.
To the best of my knowledge, I have no real or perceived conflicts of interest in conducting this
review. I have conducted research with colleagues on the macroeconomic implications of U.S.
fuel economy and greenhouse gas emissions standards. As part of this effort, my colleagues and I
recreated the EPA's OMEGA model and used it to generate estimates of vehicle prices. This
research was funded by the Alliance of Automobile Manufacturers but the work was conducted
independently of the funding organization. I disclosed this potential conflict of interest to the
EPA when they were in the process of seeking peer reviewers and it was determined at the time
to not be a conflict.
Please do not hesitate to contact me with further questions about my review, or if there are other
questions that I could address that would assist with the process.
Sincerely,
Sanya Carley
Associate Professor
Chair, Policy Analysis and Public Finance
School of Public and Environmental Affairs
Indiana University
1315 E. 10th St., Bloomington, IN 47408, Room 353
812-856-0920
scarlev@indiana.edu
Enclosure: A summary of review comments and recommendations
RTI International
Peer Review of EPA's RSE Report
89

-------
PEER REVIEW
EPA RESPONSE SURFACE METHODOLOGY REPORT
Sanya Carley
School of Public and Environmental Affairs
Indiana University
November 16, 2017
I. SUMMARY
This document summarizes my review of the "EPA Report on the Implementation of Response
Surface Methods to Reproduce ALPHA Modeling Results in the OMEGA Model Preprocess"
and all supporting modeling outputs provided in the review package. The proposed response
surface method (RSM) will be used to replicate simulation modeling in a manageable time
frame, and generate technology effectiveness estimates to be used in the Environmental
Protection Agencies' (EPA) Optimization Model for reducing Emissions of Greenhouse gases
from Automobiles (OMEGA). The OMEGA model, in turn, is an optimization model used to
generate light-duty vehicle technology cost estimates that comply with emissions standards, as
used for the Final Rulemaking for the 2017-2025 greenhouse gas emissions standards (OMEGA
vl.4.1) and more recently the Technical Assistance Report of 2022-2025 standards (OMEGA
vl.4.56).
After a thorough review of the report and supporting documentation, my general impression is
that response surface statistical methods are an appropriate and efficient approach to generate
data needed to populate the OMEGA model. The RSM is an analysis tool that is increasingly
accepted in engineering and other disciplines, and subjected to rigorous peer review. An analysis
of the model performance in this specific case also leads me to believe that the RSM approach is
highly accurate, and capable of generating results that match the significantly more time-
intensive ALPHA simulations.
RTI International
Peer Review of EPA's RSE Report
90

-------
II. OVERVIEW OF APPROACH
The general approach to the use of RSM is as follows. The modeler uses the ALPHA model to
evaluate all combinations of engines, transmissions, road loads, and vehicle types to produce a
design of experiments. The design of experiments is then entered into the response surface
equation modeling program, a standard statistical modeling software program, JMP from SAS.
The response surface equations are then generated using this statistical program. Using four
different input parameters—mass reduction, aero drag reduction, rolling resistance reduction,
and transmission type—and the response surface equations, the modeler then compares the
results between the RSM outputs and the design of experiments from the ALPHA simulations,
and assesses the quality of the fitted model as well as the accuracy of the model to predict the
experimental results. The output of the RSM is converted into a spreadsheet of vehicle
effectiveness to be used in the OMEGA model, as designed to match the former spreadsheets
used with the lumped parameter model.
All of these steps are clearly described, and in greater detail, in the "EPA Report on the
Implementation of Response Surface Methods to Reproduce ALPHA Modeling Results in the
OMEGA Model Preprocess" report. This process, as outlined, is appropriate and matches
standard procedures.
III. APPROPRIATENESS OF THE APPROACH
A. RSM as an Accepted Approach
RSM is a set of mathematical and statistical techniques that allows one to fit a
polynomial model to data. RSM can account for several different independent variables
(also referred to as factors or operating parameters), that can vary at the same time over a
set of experimental runs. RSM can be used to develop the functional relationship between
an outcome of interest (or a "response") and several different independent variables, so as
to simultaneously optimize the values of these variables. The errors in RSM are assumed
to be random.
Response Surface Methodology was first introduced by Box and Wilson in 1951 (Box
and Wilson, 1951). Since its inception, but particularly beginning it in the early 1970s, it
has been applied to a range of complex topics, such as automobiles and impact load
conditions (e.g., Avelle et al. 2002), water desalination (e.g., Boubakri et al., 2014), and
food industry processes (see Yolmeh and Jafari, 2017 for a comprehensive review of this
literature), among many other topics.
Figure 1 graphs the number of different types of studies that have used RSM since it was
first introduced, according to a search within the Scopus database. This graph
demonstrates that engineering studies are the most common applications of the method,
followed closely by biochemistry and agricultural/biological sciences. Figure 2 shows the
same data but over time, between the 1961 and 2016. This figure demonstrates that the
majority of studies that used RSM in the earliest years of the methodology were
immunology and microbiology studies. The use of the methodology has grown
RTI International
Peer Review of EPA's RSE Report
91

-------
significantly over time within various chemistry disciplines as well as, more recently,
engineering disciplines.
Figure 1. Number of Published RSM Articles by General Category of Study, 1951-2017
Economics, Econometrics and Finance
1


Psychology
¦


Nursing
-


Dentistry
-


Neuroscience
-


Business, Management and Accounting
-


Earth and Planetary Sciences



Pharmacology, Toxicology and Pharmaceutics



Computer Science



Physics and Astronomy



Environmental Science



Agricultural and Biological Sciences



Biochemistry, Genetics and Molecular Biology



Engineering




0
2000
4000 6000 8000 10000 12000 14000



Number of Published Articles
Figure 2. Published RSM Articles over Time, by General Category of Study, 1961-2016
7000
I Engineering
I Chemistry
I Biochemistry, Genetics and Molecular
Biology
I Chemical Engineering
I Agricultural and Biological Sciences
I Medicine
Environmental Science
I Materials Science
I Physics and Astronomy
I Immunology and Microbiology
*H^-|^OrOlDCT>(NL/100*-l^-|^OrOlDCT>(NL/l
ioioioi^i^i^i^ooooooct>ct>ct>oooo(T>(T>(T>(T>(T>(T>(T>(T>(T>000000
t—It—It—It—It—It—It—It—It—It—It—It—It—IC\IC\IC\IC\IC\I(N
Year
RTI International
Peer Review of EPA's RSE Report
92

-------
B.	Timing and Practical Considerations
The execution time of the RSM is similar to that associated with the former lumped
parameter model, and significantly faster than running the ALPHA model. This modeling
efficiency allows for real-time input parameter generation for the OMEGA model, and
also allows the EPA—and others that choose to replicate EPA results—to use standard
computing equipment. These conditions also have implications for the EPA budget, since
running the RSM will not require additional financial resources.
C.	Response Surface Modeling Process
To perform the RSM analysis, the modelers use a standard software package for this
purpose, the JMP Classical Response Surface Design Model. This software is highly
flexible, able to generate equations with the appropriate functional form, and assess
which independent variables should be included.
IV. VALIDATION OF THE MODELING OUTPUTS
A. Validation of Modeling Results
a.	Model Performance
A standard test that is used to evaluate the performance of the model is a
goodness-of-fit estimate. The EPA has confirmed that they obtained sufficiently
high R2 values for all of their model runs.
b.	Comparison of Output to ALPHA Modeling Output
There are a variety of performance metrics that one could use to assess response
surface equation accuracy and adequacy. For this review, I evaluated the size of
the residuals, the percent error, and the distribution of the residuals.
Table 1 displays the first two performance metrics, along with additional detail
about the model runs. The first column designates the vehicle type, of which there
are six. The second column designates the powertrain category, of which there are
four. The combination of six vehicle types of four powertrain categories results in
24 different categories of model runs. The observation count is the number of
ALPHA runs with all allowable combinations of the independent variables that
stay within parameter bounds. The next four columns provide statistics for the
residual (the difference between the dependent variable of the response surface
equation and the ALPHA simulation). The final column displays the percent
error, or the deviation between the experimental ALPHA values and the predicted
RSM values for a determined set of conditions.
These statistics confirm that the predicted values have excellent accuracy. The
average residual is 0.0013 and the average percent error is -0.0004 percent. All
combinations of vehicle type and powertrain perform similarly. The combination
that has the highest residual is the High Power/Weight 2014 Atkinson.
RTI International
Peer Review of EPA's RSE Report
93

-------
Table C-1. RSM Performance Metrics
Vehicle Type
Powertrain
Category
Obs
Average
Residual
Residual
Standard
Deviation
Residual
Min
Residual
Max
% Error
Low Power/Weight ~ Low
2014 GDI
375
-2.5E-10
0.19
-0.51
0.60
-2.4E-08
Road Load







Low Power/Weight ~ Low
2014 Atkinson
371
6.5E-03
0.20
-0.51
0.41
-2.9E-05
Road Load







Low Power/Weight ~ Low
2020 Atkinson
375
2.3E-10
0.18
-0.52
0.40
-3.1E-08
Road Load







Low Power/Weight ~ Low
2020 24 Bar Turbo
375
3.9E-10
0.18
-0.65
0.50
7.7E-08
Road Load







Medium Power/Weight ~
2014 GDI
375
6.1E-11
0.14
-0.40
0.50
4.8E-08
Low Road Load







Medium Power/Weight ~
2014 Atkinson
375
5.7E-10
0.26
-0.57
0.69
-3.7E-07
Low Road Load







Medium Power/Weight ~
2020 Atkinson
375
3.1E-11
0.18
-0.43
0.48
-4.0E-07
Low Road Load







Medium Power/Weight ~
2020 24 Bar Turbo
375
5.0E-11
0.28
-0.73
0.55
-8.9E-07
Low Road Load







High Power/Weight
2014 GDI
351
-1.1E-02
0.25
-0.58
0.57
4.1E-05
High Power/Weight
2014 Atkinson
313
0.42
0.35
-0.69
0.70
-1.8E-04
High Power/Weight
2020 Atkinson
340
8.1E-04
0.32
-0.65
0.64
5.8E-07
High Power/Weight
2020 24 Bar Turbo
375
3.0E-11
0.23
-0.69
0.68
3.3E-07
Lower Power/Weight ~
2014 GDI
325
2.5E-10
0.17
-0.49
0.59
1.2E-08
High Road Load







Lower Power/Weight ~
2014 Atkinson
274
2.5E-10
0.17
-0.38
0.41
7.3E-08
High Road Load







Lower Power/Weight ~
2020 Atkinson
285
-1.6E-12
0.11
-0.29
0.33
4.7E-08
High Road Load







Lower Power/Weight ~
2020 24 Bar Turbo
375
5.5E-11
0.15
-0.43
0.35
4.2E-07
High Road Load







Medium Power/Weight ~
2014 GDI
332
9.7E-05
0.21
-0.52
0.50
2.5E-06
High Road Load







Medium Power/Weight ~
2014 Atkinson
280
-3.4E-11
0.23
-0.58
0.65
-2.9E-08
High Road Load







Medium Power/Weight ~
2020 Atkinson
282
1.1E-10
0.21
-0.60
0.63
2.2E-08
High Road Load







Medium Power/Weight ~
2020 24 Bar Turbo
363
-9.9E-03
0.28
-0.59
0.65
4.7E-05
High Road Load







Truck
2014 GDI
357
-3.5E-03
0.27
-0.58
0.58
7.4E-06
Truck
2014 Atkinson
315
-1.4E-02
0.38
-0.77
0.78
6.2E-05
Truck
2020 Atkinson
336
-1.2E-03
0.38
-0.77
0.86
1.4E-05
Truck
2020 24 Bar Turbo
358
2.2E-02
0.36
-0.74
0.78
-6.8E-05
All vehicle types
All powertrain
8257
1.3E-03
0.25
-0.77
0.86
-4.1E-06

categories






RTI International
Peer Review of EPA's RSE Report
94

-------
I also plotted the residuals to see if they fit a normal distribution, as suggested by
Bezerra et al. (2008). Figure 3 presents a histogram of all residuals across the 8,257
model runs. The distribution appears normal. I also looked at the histograms for all
vehicle types, powertrain technologies, and vehicle type-powertrain combinations
separately (not shown here). These plots provide no cause for concern.
Figure 3. Histogram of Residuals
CM
in
(/)
c _
CD
Q
lO
	1	
0
Residual
—r~
.5
-.5
B. The Design of Experiments
The RSM output is compared to the ALPHA modeling output, which assumes that the
ALPHA output (the design of experiments) is accurate. I have no reason to believe that
this is cause for concern, however, since the ALPHA model has already gone through
thorough rigorous peer review (see
https://nepis.epa. gov/Exe/ZvPdf.cgi?Dockev=P100PUKT.pdf). The peer reviewers found
the ALPHA model to be highly reliable and accurate.
V. RECOMMENDATIONS
A. Design of Experiments
A future extension of model validation could be an assessment of the RSM output with
actual testing data. One should assume that the results would be similar to the estimates
of comparison between ALPHA and RSM, however, since the EPA's previous work
RTI International
Peer Review of EPA's RSE Report
95

-------
found that ALPHA estimates were within the margin of 3% error as compared to actual
vehicle performance testing.
B. Transparency
As stated in the report, one of the benefits of the RSM is "increased transparency regarding
synthesis of ALPHA simulation into OMEGA modeling". It is not entirely clear to me how the
use of RSM will increase transparency. But I strongly encourage and support full transparency of
modeling inputs, outputs, processes, and supporting information.
RTI International
Peer Review of EPA's RSE Report
96

-------
References
Avalle, M., Chiandussi, G., Belingardi, G. 2002. Design optimization by response surface
methodology: application to crashworthiness design of vehicle structures. Structural and
Multidisciplinary Optimization 24, 325-332.
Bezerra, M.A., Santelli, R.E., Oliveira, E.P., Villar, L.S., Escaleira, L.A. 2008. Response surface
methodology (RSM) as a tool for optimization in analytical chemistry. Talanta 76, 965-977.
Boubakri, A., Hafiane, A., Bouguecha, S.A.T. 2014. Application of response surface
methodology for modeling and optimization of membrane distillation desalination process.
Journal of Industrial and Engineering Chemistry 20, 3163-3169.
Box, G.E., Wilson, K. 1951. On the experimental attainment of optimum conditions. Journal of
the Royal Statistical Society. Series B (StatisticalMethodology) 13(1), 1-45.
Yolmeh, M., Jafari, S.M. 2017. Applications of response surface methodology in the food
industry processes. FoodBioprocess Technology 10, 413-433.
RTI International
Peer Review of EPA's RSE Report
97

-------
HRTI
INTERNATIONAL
3040 E. Cornwallis Road • POBox12194 • Research Triangle Park, NC 27709-2194 • USA
Telephone+1.919.541.6000 • Fax+1.919.541.5985 • www.rti.org
Appendix D. Sujit Das Comments
RTI International
Peer Review of EPA's RSE Report
98

-------
SujitDas
12305 Fort West Drive, Knoxville, TN 37934
Knoxville, TN 37934, USA
(865)789-0299 dass@ornl.gov
November 17, 2018
Kyle Clark-Sutton
Research Economist, RTI International
3040 Cornwallis Road
PO Box 12194, Research Triangle Park, NC 27709
RE: Peer Review of EPA's Response Surface Equation Report
Dear Mr. Clark-Sutton:
Thank you for inviting me to conduct a peer review of EPA's Response Surface Equation Report.
I have completed the review.
Enclosed with this letter is a summary of my review comments and recommendations. These
comments are made on the basis of the current state of science as I understand it. To the best of
knowledge, I have no real or perceived conflicts of interest in conducting this review.
Please feel free to contact me should you have any questions or need additional regarding this
review.
Sincerely,
SujitDas
Enclosure: A summary of review comments and recommendations
RTI International
Peer Review of EPA's RSE Report
99

-------
PEER REVIEW COMMENTS
EPA RESPONSE SURFACE MODEL (RSM)
Sujit Das
Oak Ridge National Laboratory
12305 Fort West Drive, Knoxville, TN 37934
November 22, 2017
RTI International
Peer Review of EPA's RSE Report
100

-------
Peer Review of EPA Response Surface Model (RSM)
1.	The industry standard statistical software JMP from SAS has the design of experiments and
design generation capabilities besides being interactive and visual and thereby justifies its use
towards the development of Response Surface Equations. Unlike SAS (which is command-
driven), JMP has a graphical user interface to explore data visually. JMP is the tool of choice for
scientists, engineers and other data explorers in almost every industry and government sector. It
combines dynamic data visualization with powerful statistics, in memory and on the desktop.
Interactive and visual, JMP reveals insights that raw tables of numbers or static graphs tend to
hide.
2.	RSM approach has demonstrated clearly an effective use of the large scale simulations from
the already validated full vehicle simulation model ALPHA. It definitely serves the intended
appropriate and accurate means of assessing technology packages by means of the efficient
transposition of full-vehicle simulation results into OMEGA inputs. RSM design concept is very
similar to the design of LPM which was used for each Light-duty Greenhouse Gas rulemaking,
from the 2009 FRM through the 2016 Proposed Determination. The RSEs allows any ALPHA
run to be derived at a similar speed as the current spreadsheet LPM. A similar user-friendly and
execution time LPM front end used for RPM is definitely an advantage, but it needs to be
customized for RPM which is limited to only a combination of few technology options (i.e., for
specific vehicle type and powertrain model with user-specific inputs for mass reduction, aero
drag reduction, rolling resistance reduction, and transmission type) compared to LPM, for a large
number of RPM users.
3.	Response surface methodology (RSM) explores the relationships between several explanatory
variables and one or more response variables. A sequence of designed experiments (DOE) was
used, i.e., the main idea of RSM to obtain an optimal response. A DOE used in this case was
based on an automated process that is configured to produce a complete set of ALPHA results
for all combinations of engines, transmissions, roadloads, and vehicle types to be used in the
OMEGA analysis. It is a relatively easy statistical model to estimate and apply, even when little
is known about the process. It maximizes the production of a special substance by optimization
of operational factors. A factorial experiment or a fractional factorial design generally used to
RTI International
Peer Review of EPA's RSE Report
101

-------
estimate RSE process has generated as series of equations from a complete set of ALPHA data
for each vehicle type and powertrain model. A second-degree RSE polynomial model was
developed for each 24 vehicle cases based on a combination of 6 vehicle types and 4 powertrain
types in the present analysis.
4.	A comparison of CO2 results between RSE and ALPHA has confirmed the validity of the data
transfer between these two models thereby proving the accuracy of the technical application of
response surface modeling. A total of 21 results (only 2020 TURB24 was available for
LPW LRL vehicle) out of total 24 vehicle types were examined for the RSM validation.
Residuals were found to be between a narrow range of-1.0 and 1.0 gCCh/mile in all cases. The
line slope of the plot of results of ALPHA and RSE was also found to be 45° and thus has
ensured the validity of data transfer between them. In addition, as the physics behind the Mass,
Aero, and Roll are quite linear in reality, and so CO2 emission impacts of any values between the
range of these parameters were also found to be reasonable using the RSE results.
5.	Section 6. Baseline Vehicle Adaptation needs further details in terms of the necessary process
steps for adjusting the effectiveness of a baseline vehicle to match the ALPHA model. The
adjustment approach for the baseline vehicle adaptation is an interesting one as it allows ~ 50
alternative options to consider in a baseline vehicle.
6.	Since the RSE final output is CO2 emissions provided to the OMEGA model with the
technology alternatives necessary to produce the most cost-effective path for compliance, a short
discussion of it will be useful for unfamiliar users.
7.	A description of three different transmission types considered and denoted by numerals (i.e.,
2, 4, & 5) would be useful. An appropriate justification needs to be included why other two
types, i.e., 1 and 3 were not considered for the RSM DOE analysis.
8.	It is unclear why the assumed vehicle mass reduction value is not actually reflected in the
ALPHA spreadsheets provided, e.g., for 2020 TURB24 vehicle, 3109.15 lbs and 2961.3 lbs Test
Weight have been assumed for a mass reduction of 5% and 10%, respectively, for a baseline
vehicle Test Weight of 3257 lbs? Similar level of difference was found in all 21 different vehicle
type/powertrain considered for RSM.
RTI International
Peer Review of EPA's RSE Report
102

-------
9.	The draft report mentions about six vehicle types in OMEGA analysis and four powertrain
categories in the ALPHA. It is unclear about the consistency in the number of vehicle types and
powertrain categories between these two tools and thereby to what extent does the current RSM
cover the overall analysis scope of the OMEGA technology options?
10.	In spite of the fact that there are four independent variables, i.e., mass reduction,
aerodynamic drag reduction, rolling resistance reduction, and transmission type have been used
for the development of RSE equations, but +50 ALPHA data variables have been included in the
several vehicle spreadsheets provided. It'd be good to provide the description of each of the
ALPHA variables for an understanding of impacts of the four major dependent variables
considered.
11.	As the RSE "Effectiveness" implementation is expanded beyond the currently limited six
vehicles, four powertrains, and three transmission type options provided, the user-friendliness in
terms of inputs should be kept in mind. Using the current framework provided as an example, it
is difficult for a novice user to perform a quick analysis. Specifically, a discussion on the
"Baseline Vehicle Adaptation" procedure needs to be included in the documentation, when all
original LPM technology options are also available for RSM for the baseline vehicle adaptation.
Some Comments/Warning should be included if the results are invalid for transmission cases 1 &
3 as is the case now. The inputs for Vehicle Type, Model, and Transmission in Column A should
be interlinked with the corresponding numeric value in Column B on this worksheet.
12.	It'd be useful for the EPA draft report completeness to provide some background information
on the models and tools used in EPA's light-duty Greenhouse Gas (GHG) rulemakings for
unfamiliar audience.
13.	Not sure whether any model validation was done in terms of using the model to predict the
response for one or more combinations of design factors that were not used to build the RSM
models? What agreements between the two results were found for such a validation?
14.	Overall, the quality of RSE methodology appears to be reasonable for the four independent
variables considered. The validity of this methodology need to reexamined if it is expanded to a
higher number of independent variables in the future.
RTI International
Peer Review of EPA's RSE Report
103

-------
HRTI
INTERNATIONAL
3040 E. Cornwallis Road • POBox12194 • Research Triangle Park, NC 27709-2194 • USA
Telephone+1.919.541.6000 • Fax+1.919.541.5985 • www.rti.org
Appendix E. Doug Montgomery Comments
RTI International
Peer Review of EPA's RSE Report
104

-------
Douglas C. Montgomery
3841 East Talowa Street
Phoenix AZ 85044
480.496.8872
30 November 2017
Kyle Clark-Sutton
Research Economist, RTI International
3040 Cornwallis Road
PO Box 12194, Research Triangle Park, NC 27709
RE: Peer Review of EPA's Response Surface Equation Report
Dear Mr. Clark-Sutton:
Thank you for asking me to participate in a peer review of EPA's Response Surface Equation
Report. I have completed the requested review.
Previously I have sent you a summary of my review comments and recommendations. These
comments are made on the basis of the current state of science as I understand it. To the best of
my knowledge, I have no real or perceived conflicts of interest in conducting this review.
Please feel free to contact me should you have any questions or need additional regarding this
review.
Sincerely,
1 _ >•' ¦ ¦ 	
Douglas C. Montgomery
RTI International
Peer Review of EPA's RSE Report
105

-------
ALPHA is the acronym for the Advanced Light-Duty Powertrain and Hybrid Analysis full vehicle
simulation model developed to study greenhouse gas emissions from vehicle internal combustion engines.
This is a validated model that has been shown to provide accurate prediction of emissions for various
combinations of engines and vehicle types. However, running the ALPHA model is very time-consuming.
In situations like this a standard industry practice is to replace the computer model with a statistical model
that can be executed more quickly but which has comparable accuracy in prediction. Such a statistical
model is usually called a metamodel.
A widely used approach to creating the statistical metamodel is to conduct a designed experiment on the
computer model investigating factors of interest to the analysts and then fit the model to the data resulting
from the experiment. Response surface methodology (RSM) is a standard technique for this purpose.
This report is a review of the RSM models produced for the ALPHA simulation model. There are 24
models representing a range of powertrain and vehicle types. For each RSM model I was furnished with
a spreadsheet that contained the designed experiment that was performed on that configuration of the
ALPHA model, along with the observed responses, the predicted responses from the RSM model, and the
residuals. For these experiments the inputs factors for the design include Mass Reduction, Aero Drag
Reduction, Rolling Resistance Reduction, and Transmission type. The experiments used were variations
of standard factorial designs. Response surface models were fit to the experimental results to produce the
spreadsheet outputs that I was given.
I investigated the adequacy of the RSM models by first analyzing the residuals from these models in the
spreadsheets that were provided. I constructed normal probability plots of the residuals and plots of the
residuals versus the predicted response. These plots investigate the normality of the response variable
and the equality of variance assumption, both of which are standard RSM assumptions. The normality
assumption is of only moderate importance since the underlying statistical methodology is robust to all
but severe departures from normality. The equal variance assumption is more important, and moderate to
large departures from this assumption may require remedial measure such as the use of variance-
stabilizing transformation. A few of the normal probability plots exhibited very small potential
departures from normality but nothing severe enough to call model validity into question. Similarly,
some of the plots of residuals versus the predicted response exhibited a non-random pattern, but none of
the patterns were serious enough to question the equal variance assumptions. It is also worth noting that
the model residuals ae extremely small as all models provide extremely good fits to the data obtained
from the simulation model.
I selected a subset of the 24 models for further investigation. I loaded the experimental designs for these
models into JMP PRO V 13 and performed my own RSM analysis, fitting the standard second-order
model. The results for one of these RSM model from spreadsheet HPW 1026 2017a tab 2014 GDI are
discussed below. This is typical of the results I obtained for all models that I investigated.
RTI International
Peer Review of EPA's RSE Report
106

-------
Plot of actual versus predicted response:
^ Actual by Predicted Plot
2 320
; -o.
240 260 280 300 320 340 360
ALPHA C02 Predicted RMSE=0.2477 RSq=1.00
PValue<.0001
The points in this plot lie almost exactly along a straight line, indicating excellent agreement between
the simulation model output and the predicted value from the second-order RSM model.
Summary of fit and analysis of variance for the RSM model:
zi Summary of Fit
RSquare	0.999925
RSquare Adj	0.999921
Root M ea n Sq u a re E rror	0.247653
Mean of Response	2 85.8269
Observations (or Sum Wgts)	351
i Analysis of Variance


Sum of


Source
DF
Squares
Mean Square
F Ratio
Model
14
273407.37
19529.1
318414.8
Error
336
20.61
0.061332
Prob > F
C. Total
350
273427.97

<•0001*
The ft2 statistic for the model exceeds 0.99, indicating that most of the variability in the sample data (in
excess of 99%) is explained by the RSM model. Also, the Readjusted statistic is also in excess or 0.99.
Readjusted is a reflection of potential overfitting; that is including terms not really important in the
model just to inflate the ordinary ft2. When these two statistics are in close agreement as they are here
there is likely to be no substantial issue with overfitting. The analysis of variance indicates that the
model contains at least one statistically significant term.
RSM Model Parameters Estimates:
Parameter Estimates
Term
Estimate
Std Error
t Ratio
Prob>|t|
Intercept
382.68894
0.075154
5092.1
<,0001*
Mass
-201.7124
0.190516
-1059
<.0001*
Aero
-52.06903
0.190476
-2734
<.0001*
Roll
-53.30226
0.189323
-281.5
<,0001*
Trans
-18.23885
0.012694
-1437
<,0001*
(Mass-0.10085)*(Mass-0,10085)
8,0152892
3.191378
2.51
0,0125*
(Mass-0.1008 5)*[Aero-0,09715)
-10,63205
2.735951
-3.89
0,0001*
(Aero-Q .09715)*[Aero-0,09715)
-5.158863
3.177543
-1.62
0.1054
(Mass-0,10085)*(Roll-0,09886)
50.490674
2,751456
18.35
<.0001*
(Aero-0.09715)*(Rol 1 -0,09886)
-6.814333
2,723004
-2.50
0.0128*
(Rol 1 -0.09 8 8 6)*[Rol 1-0.09886)
-3.497919
3.169082
-1.10
0.2705
(Mass-0.10085)*(Trans-3,68091)
23.883683
0.149944
159.28
<.0001*
(AerQ-0.09715)*{Trens-3,68091)
0.4606774
0.150692
3.06
0.0024*
(Roll-0.09886)*(Trans-3.68091)
0.626202
0.149753
4.18
<.0001*
(Trans-3,68091)*(Trans-3,68091)
0.5810956
0.014437
40.25
<.0001*
RTI International
Peer Review of EPA's RSE Report
107

-------
The second -order model contains 15 parameters; an intercept, four main effects, six 2-factor
interactions, and four quadratic terms. The parameter estimates display indicates that all but two of
these terms are statistically significant at the 0.05 level. However, in RSM we usually think that it's the
order of the model that is most important so we often do not remove non-significant terms from the
model unless there are many of them. That is not the case here.
The PRESS Statistic
In model validation it is important that the model both fit the sample and that it provide good
predictions of new data. The PRESS (Prediction Error Sum of Squares) statistic, reported below, is a
standard one-sample-at-a-time cross-validation used to assess potential prediction performance.
A Press
Pre® Press RMSE
22.448639323 0.2528957
Notice that the PRESS statistic is very similar to the residual sum of squares from the analysis of
variance. An /?2-like prediction error statistic can be computed from PRESS simply by replacing the
residual sum of squares in the equation for R2 by PRESS. This gives:
2 _t_PRESS_ 22449
Prediction 1	1	KJ.yyyy
TotalSS 273428
We would expect the RSM model to explain in excess of 99% of the variability in data produced by the
simulation model. This is excellent validation of potential prediction performance.
Summary of Conclusions
I conclude that the RSM approach has produced statistical metamodels that are an excellent alternative
to the APLHA simulation model. So long as they are used to interpolate over the ranges of the four
factor used in their construction I expect that they will be excellent alternatives to the ALPHA simulation
procedure.
RTI International
Peer Review of EPA's RSE Report
108

-------
Appendix F. Response Surface Report
RTI International
Peer Review of EPA's RSE Report
109

-------
EPA Report on the Implementation of Response
Surface Statistical Methods to Reproduce
ALPHA Modeling Results in the OMEGA
Model Preprocess
J. Cherry
March 2017
RTI International
Peer Review of EPA's RSE Report
110

-------
1 Background:
For the Light-Duty (LD) Greenhouse Gas (GHG) rulemakings created by the Environmental
Protection Agency (EPA) including the MY 2012-2016 and MY 2017-2025 Final Rules, estimates for the
effectiveness of vehicle technologies have played an important role as a robust input into the overall
rulemaking analysis process and as input to EPA's Optimization Model for reducing Emissions of
Greenhouse gases from Automobiles (OMEGA).
For each Light-duty Greenhouse Gas rulemaking, from the 2009 FRM through the 2016 Proposed
Determination, EPA has applied a combination of full-vehicle simulation modeling and a Lumped
Parameter Model (LPM). The LPM methodology has been continuously developed, refined, and
calibrated throughout each of these rulemakings to reflect the latest technology developments and
comments received regarding the application of the LPM. The National Academy of Sciences (NAS)
reviewed the application of the LPM in their 2011 and 2015 reports on technologies available for
reducing fuel consumption and found the LPM to be robust and to accurately predict the overall
effectiveness of combinations of technology. While EPA continues to believe that the LPM is an
appropriate and accurate means of assessing technology packages, the efficient transposition of full-
vehicle simulation results into OMEGA inputs has historically required many hours of manual
calibration, that has not been well understood by our stakeholders.
In response to comments received from stakeholders and in an effort to reduce the manual
interpretation and calibration of the LPM, EPA is considering replacing the LPM with an industry
standard statistical methodology. This methodology, commonly known as a Response Surface Model
(RSM), computationally is able to synthesize large numbers of simulations and distill the outputs into an
equation which represents the effectiveness of technology packages. This latest process to reproduce these
technology effectiveness estimates in real time for OMEGA is the subject of this report. First, some
history of the process for reference.
1.1 History:
One method for determining the effectiveness value for a vehicle technology package required for an
OMEGA analysis would be to run a validated full vehicle simulation. In practice, robust full vehicle
simulations require a considerable set of data and a finite amount of time to execute for each simulation.
During atypical analysis cycle, many thousands of simulations are performed. For example,
preprocessing data for an OMEGA run requires approximately one million technology package results
that would require several days to execute on a modern computer. This situation along with the lack of a
complete set of engine maps, transmission maps, and other validated data required for such simulations
during the analysis for the MY 2012-2016 Final Rule required an alternative solution.
In response to this need, EPA combined an extensive library of full vehicle simulation data, test data,
and public literature to create the Lumped Parameter Model (LPM). Historically, the LPM has been
implemented as a spreadsheet method to provide vehicle technology package effectiveness values in the
preprocessing phase for the OMEGA model. The LPM was originally based on the techniques for
combining (lumping) various vehicle technologies into their various loss categories as detailed in a SAE
paper by General Motors (2002-01-0628). This lumping process results in a first-principles energy
RTI International
Peer Review of EPA's RSE Report
111

-------
balance accounting for the various synergies and di synergies as the technologies are merged to reduce
double counting and missed efficiencies. The result is a final effectiveness value to represent the changed
efficiency of the vehicle as the result of the additional (or subtracted) technologies. An example of these
loss categories is shown in Figure 6 and the original Excel version of the LPM is shown in Figure 7.
Fuel
energy
100%

Exhaust
33%
Cooling
29%
Thermodynamic
losses
Total
energy
losses
Mechanical
power

Friction
losses
33%
Air drag 5%

Engine
11.5%
Transmission 5%
Rolling
resist 11.5%
Brakes 5%
Air drag 5%

Energy
used to
move the
car 21.5%
Figure 6 - Example loss categories in a light duty vehicle
RTI International
Peer Review of EPA's RSE Report
112

-------
4
EPA Staff Deliberative Materia Is-Do Not Quote or Cite
Vehicle Energy Effects Estimator
Vehicle type: Standard Car
Family
Description: Technology picklist
Package: Z

Indicated Energy
Heat


Brake Energy
Engine Friction
Lost To


V ehicle
Road Loads
Parasitics
Gearbox,

Exhaust &


Mass
Drag Tires

T.C.

Coolant


Inertia
Aero
Rolling
Access
Trans
Friction Pumping
IndEff
Second

Load
Load
Load
Losses
Losses
Losses Losses
Losses
Law
Baseline % of fuel
13.0%
4.0%
4.0%
1.8%
4.2%
6.6% 4.4%
32.0%
30.0%
Reduction
0%
16%
8%
64%
33%
16% 75%


% of original fuel
13.0%
3.4%
3.7%
0.8%
3.3%
5.6% 1.1%
31.8%
| 30% |
Check
100.0%

Indicated
Mech
Brake
Drivetrain
Fuel
Road

Efficiency
Efficiency
Efficiency
Efficiency
Efficiency
Loads
Baseline
38.0%
71.1%
27.0%
77.8%
21.0%
100.0%
New
38.2%
82.5%
31.5%
87.2%
27.5%
95.4%
Current Results
72.9%	Fuel Consumption
27.1%	FC Reduction
37.2%	FE Improvement
N/A	Diesel FC Reduction
Original friction/brake ratio
Based on PMEP/IMEP »»
(GM study)
PMEP
Losses
Brake
Efficiency
11%
27%
4

Independent


User Picklist


Technology
FC Estimate
Loss Category
Implementation into estimator
Include? (0/1)
Gross FC Red

Aero Drag Reduction
3.0%
Aero
16% aero (cars), 10.5% aero (trucks)
1
3.0%

Rolling Resistance Reduction
1.5%
Rolling
8% rolling
1
1.5%

Low Fric Lubes
0.5%
Friction
2% friction
1
0.5%

EF Reduction
2.0%
Friction
8.5% friction
1
2.0%

ICP
2.0%
Pumping
12% pumping, 38.2% IE, -2% fric
0
0.0%

DCP
3.0% total VVT
Pumping
18.5% pumping, 38.2% IE, -2% fric
0
0.0%
Pick one
CCP
3.0% total VVT
Pumping
18.5% pumping, 38.2% IE, -2% fric
1
3.0%

Deac
6.0%
Pumping, friction
39% pumping
0
0.0%

DVVL
4.0%
Pumping
30% pumping, -3% friction
1
4.0%
Pick one
CVVL
5.0%
Pumping
37% pumping, -3% friction
0
0.0%
Camless
10.0%
Pumping
76% pumping, -5% friction
0
0.0%

GDI
1.5%
IndEff
38.6% Ind Eff
0
0.0%

Turbo/Dnsize
6.0%
Pumping
39% pumping
0
0.0%

5-spd
2.5%
Trans, pumping
22% pumping, -5% trans
0
0.0%
Pick one or
CVT
6.0%
Trans, pumping
46% pumping, -5% trans
0
0.0%
6-spd
ASL
1.5%
Pumping
9.5% pumping
1
1.5%

AggTC Lockup
0.5%
Trans
2.5% trans
1
0.5%

6-spd auto
5.5%
Trans, pumping
42% pumping, -5% trans
1
5.5%
Or #44/45
AMT
6.5%
Trans
35% trans (increment)
1
6.5%

42V S-S
7.5%
F, P, A
13% friction, 19% pumping, 38% access
1
7.5%

12V acc + Imp alt
1.5%
Access
18% access
0
0.0%
Or #53
EPS
1.5%
Access
18% access
1
1.5%

42V acc + imp alt
3.0%
Access
36% access
1
3.0%
Or #51
HCCI dual-mode
11.0%
Ind. Eff, pumping
41% IE, 25% pumping
0
0.0%

GDI (lean)
10.5%
Ind. Eff, pumping
40% IE, 38% pumping
0
0.0%

Diesel - LNT
30.0% over gas
Ind Eff, pumping
48% IE, 85% pumping, -13% friction
0
0.0%
Pick one
Diesel - SCR
30.0% over gas
Ind Eff, pumping
46% IE, 80% pumping, -13% friction
0
0.0%

Opt. E25
8.5%
Ind. Eff, pumping
39% IE, 40% pumping
0
0.0%

Figure 7 - Original LPM Spreadsheet
The LPM user interface consisted of selecting a vehicle type and various technology combinations
(see arrows in Figure 7) to calculate the final result in percent. In practice the spreadsheet was used
twice, first to calculate a percentage effectiveness improvement using the technologies on a baseline
vehicle, and second to calculate a percentage effectiveness improvement with additional technologies
applied to the same baseline vehicle. This process was automated to provide approximately fifty
improvements for each baseline vehicle as input to the OMEGA model.
The LPM was validated for present day vehicles and technologies by comparing the results to test
data from various EPA databases and a contract with Ricardo Inc. provided another set of simulation
results for validating present and future technologies. For the regulatory activities associated with the
RTI International
Peer Review of EPA's RSE Report
113

-------
Light-duty Greenhouse Gas Midterm Evaluation (MTE), the LPM has also been calibrated with results
from EPA's full-vehicle simulation model, ALPHA.
2 Development of ALPHA:
After the completion of the MY 2017-2025 LD Final Rule, EPA began an extensive project to
benchmark a wide variety of engines, transmissions, and vehicles to create the Advanced Light-Duty
Powertrain and Hybrid Analysis (ALPHA) full vehicle simulation model based on the existing GEM
model used for heavy duty compliance purposes. The intent of this project was for the ALPHA model to
be fully functional, validated, and peer reviewed for use during the MY 2017-2025 LD Final Rule
Midterm Evaluation (MTE) process.
As the MTE progressed, the ALPHA model matured and was capable of providing most of the
technology package effectiveness values needed for the OMEGA analysis. With the ALPHA model
results being applied widely across EPA's analyses, the LPM quickly became less a model and more a
repeater of ALPHA model results. For the Proposed Determination phase of the MTE, EPA recognized
that a more efficient and less complex method could be developed to access ALPHA results directly. In
addition, EPA now has the capability to perform large scale simulation using ALPHA and the application
of these large-scale simulation results requires a more streamlined and less manual process. EPA
considered several alternatives for its future analyses with respect to the application of simulation results.
Figure 8 illustrates some possible methods to accomplish this task.
Possible
Alternatives
Maintain
LPM
Run ALPHA
in Real-Time
Create H Create Response
ALPHA H Surface Equations
Matrix Mfrom ALPHA Results
Figure 8 - Possible alternatives to access ALPHA results
Discussion for each alternative:
•	Maintain LPM:
o The LPM was originally designed to generate values - not to match several models
simultaneously resulting in a significant increase in complexity and maintenance. While
EPA feels that the LPM continues to be a robust and accurate tool, the calibration and
maintenance of the tool is manually intensive.
•	Run ALPHA in Real-Time:
o Running ALPHA in real-time would be the ideal solution, however, as discussed earlier,
the execution time would be prohibitive.
•	Create full ALPHA Matrix of results:
RTI International
Peer Review of EPA's RSE Report
114

-------
o The execution time would be prohibitive initially unless significant budget and resources
are dedicated for access to a complex cloud or supercomputing system. This process
would have to be repeated for any additions or changes to the ALPHA model and would
impact sensitivity analyses depending on scenario.
• Create Response Surface Equations (RSEs) from ALPHA results:
o The RSEs allow any ALPHA run to be derived at a similar speed as the current
spreadsheet LPM.
o Requires no programming or calibration - A simple check sheet verifies RSE alignment
with ALPHA results.
o An overnight batch job producing several thousand ALPHA results is sufficient to create
a set of RSEs.
o This method was chosen and is described in the following sections.
3 New Method for OMEGA Preprocess:
Given the above discussion along with the ALPHA model now capable of providing most of the
needed effectiveness data, the LPM is no longer required and can be replaced with industry standard
Response Surface Equations. This technique allows any combination of the ALPHA full vehicle
simulations to be accessed in real time to assemble the necessary effectiveness data for the OMEGA
Model.
This process begins by instructing the ALPHA model to execute a Design Of Experiments (DOE) to
provide the necessary inputs to the RSE. The DOE used for this task is an automated process that is
configured to produce a complete set of ALPHA results for all combinations of engines, transmissions,
roadloads, and vehicle types to be used in the OMEGA analysis. The DOE generates thousands of
modeling results to populate the statistical RSE generation tool. EPA adheres to a "Performance Neutral"
methodology for all rulemaking simulation work as described in Chapter 2 of the Technical Support
Document (TSD) of the Proposed Determination (PD) phase of the MTE1. Several ALPHA runs are
executed per table point and the run closest to the same performance as the base vehicle is selected to
ensure the DOE is populated with "Performance Neutral" results. For this example, the inputs to the RSE
include Mass Reduction, Aero Drag Reduction, Rolling Resistance Reduction, and Transmission type. A
small sample of the 21,000+ ALPHA results used to generate the RSE for vehicle type MPW_LRL is
shown in Table 2.
1 https://nepis.epa.gov/Exe/Zy PDF. cgi?Dockey=P100Q3L4.pdf
RTI International	Peer Review of EPA's RSE Report
115

-------
Mass
Reduction
Aero
Reduction
Rolling
Reduction
Transmission
Type
C02
g/mi
5%
10%
15%
4
215.7
5%
10%
20%
4
213.4
5%
15%
0%
4
219.9
5%
15%
5%
4
217.9
5%
15%
10%
4
215.6
5%
15%
15%
4
213.3
5%
15%
20%
4
211.1
5%
20%
0%
4
217.6
5%
20%
5%
4
215.4
5%
20%
10%
4
213.2
5%
20%
15%
4
210.9
5%
20%
20%
4
208.8
0%
0%
0%
5
222.8
0%
0%
5%
5
220.4
0%
0%
10%
5
218.3
0%
0%
15%
5
215.9
0%
0%
20%
5
213.7
0%
5%
0%
5
220.5
0%
5%
5%
5
218.1
0%
5%
10%
5
215.9
0%
5%
15%
5
213.8
0%
5%
20%
5
211.3
0%
10%
0%
5
218.2
0%
10%
5%
5
215.9
0%
10%
10%
5
213.7
0%
10%
15%
5
211.4
0%
10%
20%
5
209.0
0%
15%
0%
5
216.0
0%
15%
5%
5
213.8
0%
15%
10%
5
211.3
0%
15%
15%
5
208.8
0%
15%
20%
5
207.0
0%
20%
0%
5
213.5
0%
20%
5%
5
211.2
0%
20%
10%
5
208.9
0%
20%
15%
5
206.6
0%
20%
20%
5
204.6
Table 2 - Small sample of ALPHA results for vehicle type MPW LRL
RTI International
Peer Review of EPA's RSE Report
116

-------
4 RSE Generation:
The next phase of the RSE process generates a series of equations from a complete set of ALPHA
data for each vehicle type and powertrain model. The sets of data used in this process are shown in
Figure 11. EPA used the industry standard statistical software JMP from SAS2 to create the response
surface equations. The complete table of ALPHA results for a particular vehicle type and powertrain
model is entered and an example Response Surface Equation result is shown in Figure 9.
220.667785899048+-14.9927683931428*((Mass-0.1)/0.1)+-4.89195006285714*((Aero-0.1)/0.1)+-
4.37358584114285*((Roll-0.1)/0.1)+-18.64483848*((Trans-3)/2)+((((Mass-0. l)/0.1)*(Aero-0.1))/0.1)*-
0.0912061119999999+((((Mass-0. l)/0. l)*(Roll-0. l))/0.1)*0.405110426666666+((((Aero-0. l)/0. l)*(Roll-
0.1))/0.1)*0.039134533333334 l+((((Mass-0.1)/0.1)*(Trans-3))/2)* 1.52171338742856+((((Aero-
0.1)/0.1)*(Trans-3))/2)*0.17252074057143+((((Roll-0.1)/0.1)*(Trans-3))/2)*0.22308929142857+((((Mass-
0. l)/0. l)*(Mass-0. l))/0. l)*-0.158016769523809+((((Aero-0. l)/0. l)*(Aero-
0.1))/0.1)*0.0321151047619073+((((Roll-0.1)/0.1)*(Roll-0.1))/0.1)*0.00802461333332157+((((Trans-
3)/2)*(Trans-3))/2)*-l.66503986133335
Figure 9 - Example RSE for Vehicle Type = MPW LRL and Model = 2014 GDI
Throughout this process, the ALPHA results are compared to the RSE results as shown in Figure 10
to ensure the validity of the data transfer from ALPHA and RSE equation implementation. An added
benefit of this comparison is the verification that the ALPHA model results are smooth and predictable as
expected.
2 https://www.jmp.com/en_us/software/data-analysis-software.html
RTI International	Peer Review of EPA's RSE Report
117

-------
260
250
240
_ 230
0)
220
bO
r\j
8 210
LU
OO
DC
200
190
180
170
i:
Alpha vs RSE C02 Values

















#






























































•*









70 180 190 200 210 220 230 240 250 260
ALPHA C02 (g/mile)
Figure 10 - ALPHA vs RSE C02 values
5 Practical Implementation:
The practical implementation of the RSE method uses a similar effectiveness spreadsheet and user
interface format as before replacing the LPM with methods described in this document. This similar
format and user interface avoided disruptive modifications to the existing OMEGA preprocess and
continues the practicality of a visual tool for verification purposes and transparency for stakeholders. The
new effectiveness tool has been reduced to a single spreadsheet tab labeled "Effectiveness" as the LPM
and associated equations no longer exist.
5.1 RSE Layout
The current OMEGA analysis consists of six vehicle types based on power to weight ratio and road
loads:
•	Low Power/Weight - Low Road Load (Typical Small Car)
•	Medium Power/Weight - Low Road Load (Typical Standard Car)
•	High Power/Weight (Typical Large Car)
•	Low Power/Weight - High Road Load (Typical Small SUV)
•	Medium Power/Weight - High Road Load (Typical Large SUV)
•	Truck (Typical Full Size Pickup)
RTI International
Peer Review of EPA's RSE Report
118

-------
The current ALPHA model consists of four powertrain categories:
2014 GDI (2014 GDI)
2014 Atkinson (2014_ATK2)
2020 Atkinson (2020_ATK2)
2020 24 Bar Turbo (2020_TURB24)
The six vehicle types combined with the four model categories result in twenty-four RSEs as shown
in Figure 11.
0)
~o
o
<
X
~_
_l
<
LPW_
.LRL
2014.
.GDI
LPW_LRL
2014 ATK2
LPW LRL
2020 ATK2
LPW_LRL
2020 TURB24
Vehicle Type
MPW_
.LRL
2014.
.GDI
MPW_LRL
2014 ATK2
MPW_LRL
2020 ATK2
MPW_LRL
2020 TURB24
HPW
2014 GDI
HPW
2014 ATK2
HPW
2020 ATK2
HPW
2020 TURB24
LPW.
HRL
2014.
_GDI
LPW_HRL
2020 TURB24
MPW.
.HRL
2014.
.GDI
LPW.
_l
a:
x,
2014.
ATK2
LPW.
.HRL
2020.
ATK2


MPW_HRL
2014 ATK2
MPWJHRL
2020 ATK2
MPWJHRL
2020 TURB24
Truck
2014 GDI
Truck
2014 ATK2
Truck
2020 ATK2
Truck
2020 TURB24
Figure 11 - RSE Layout
5.2 Process Summary
A summary of the process is shown in Figure 7. Simply stated, the inputs are applied to the
selected RSE and the corresponding ALPHA C02 value is generated.
Mass %
Aero %
Roll %
Trans #
MPW_LRL
2014 ATK2
ALPHA C02
Figure 12 - RSE Implementation Summary
RTI International
Peer Review of EPA's RSE Report
119

-------
5.3 Example Case
An example case of the "Effectiveness" RSE in shown in Figure 13. The Vehicle Type selection and
Model selection determine the appropriate RSE to be used. In this example the RSE equation in Figure 9
(MPW LRL 2014 GDI) has been selected with the inputs:
•	Mass Reduction = 5%
•	Aero Drag reduction =15%
•	Rolling Resistance reduction = 5%
•	Transmission = TRX21 (4)
THE RESULTING VALUE FROM THE RSE IS CALCULATED AS SHOWN IN FIGURE 13 CLOSELY
MATCHING THE ALPHA VALUE FROM THE RSE INPUT TABLE IN TABLE 3.
EPA Staff Deliberative Materials—Dd Not Quote or Cite











Vehicle Type










MPW LRL 2




















Model










2014 GDI 1




















Transmission










TRX21 4





























Current C02 Results
//
Exemplar Vehicle Characteristics




Vehicle Type
C02
//
Rated Power
190
hp




LPW LRL
201.2 |
J
Rated Torque
191
ft-lb




MPW LRL
217.9 1

ETW
3626
lb




HPW
2S&.0

SOmph RL
11.4
hp




LPW HRL
241.6

P/W
0.052399
hp/lb




MPW HRL
307.6

Null C02
296.31 OS
g/mi




Truck
341.2









RS Result
217.9









Final Result
217.9









Figure 13 - Example Case
Table 3 - Example ALPHA Result from RSE Input Table
Mass
Reduction
Aero
Reduction
Rolling
Reduction
Transmission
Type
CO 2
g/mi
5%
10%
15%
4
215.7
5%
10%
20%
4
213.4
5%
15%
0%
4
219.9
5%
15%
5%
4
217.9
5%
15%
10%
4
215.6
RTI International
Peer Review of EPA's RSE Report
120

-------
Comparisons of execution time has shown the RSE method is similar to the LPM including the
overhead of the automation system. The added benefits include:
•	Elimination of LPM programming and calibration
•	Increased transparency regarding synthesis of ALPHA simulation into OMEGA modeling
•	Real-Time extraction of ALPHA results with the ability to quickly represent the latest available
benchmarking and simulation data in greenhouse gas analyses.
•	Ability for stakeholders to readily reproduce the RSEs based on ALPHA simulations and/or their
own large-scale simulation results.
•	Vast speed improvement over executing ALPHA in Real-Time allowing the OMEGA analysis to
run on standard EPA computing equipment without additional resources or budget.
RTI International
Peer Review of EPA's RSE Report
121

-------
6 Conclusion
EPA's Lumped Parameter Model has been a robust tool for estimating the effectiveness of light-duty
vehicle technology packages to reduce greenhouse gas emissions. While EPA continues to believe that
the continued application of the LPM would provide accurate assessments, we also recognize that the
required manual calibration of the LPM and the associated interpretation of ALPHA full-vehicle
simulation could be improved. EPA considered several alternatives in considering the future
development or replacement of the LPM. EPA has found the most efficient approach is to replace the
LPM with statistically derived Response Surface Equations. EPA believes this change in methodology
will allow the agency to more readily access large-scale simulation results, improve the robustness of the
analyses, and improve transparency in the OMEGA process.
RTI International
Peer Review of EPA's RSE Report
122

-------