vvEPA
United States
Environmental
Protection Agency
EPA Science Advisory
Board Staff Office (1400F)
Washington DC
EPA-SAB-COU NCIL-ADV-04-004
May 2004
www.epa.gov/sab
REVIEW OF THE REVISED
ANALYTICAL PLAN FOR
ERA'S SECOND
PROSPECTIVE ANALYSIS -
BENEFITS AND COSTS OF
THE CLEAN AIR ACT 1990-
2020
An Advisory by a
Special Panel of the
Advisory Council on Clean Air
Compliance Analysis
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON D.C. 20460
OFFICE OF THE ADMINISTRATOR
SCIENCE ADVISORY BOARD
May 20, 2004
EPA-SAB-COUNCIL-ADV-04-004
The Honorable Michael O. Leavitt
Administrator
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, NW
Washington, DC 20460
Subject: Review of the Draft Analytical Plan for EPA's Second Prospective
Analysis - Benefits and Costs of the Clean Air Act, 1990-2020: An
Advisory by the Advisory Council for Clean Air Compliance Analysis
Dear Administrator Leavitt:
Congress established the US EPA Advisory Council for Clean Air Compliance Analysis
Council (the Council) to review the data and methodologies to be used for the 812 Analyses and
make recommendations on their use. Section 812 of the Clean Air Act Amendments of 1990
also requires the Council to review the findings made in reports developed under Section 812,
and "make recommendations to the Administrator concerning the validity and utility of such
findings." A Special Panel of the Council presents in this document a review of the Agency's
Draft Analytical Plan for EPA's Second Prospective Analysis -Benefits and Costs of the Clean
Air Act, 1990-2020.
The Draft Analytical Plan reflects the Agency's design for the Second Prospective "812
Analysis." The series of Section 812 reports produced by the Agency are the flagship examples
of benefit-cost analysis of environmental regulation in the U.S. These analyses have assisted the
Agency in developing methods used in quantifying benefits and costs for rules issued by EPA
pursuant to the 1990 amendments to the Clean Air Act. Those benefits have been recognized by
OMB as constituting the majority of quantified benefits attributable to federal regulation over the
ten-year period, October 1, 1992 to September 30, 2002. (OMB 2003 Report, Informing
Regulatory Decisions: 2003 Report to Congress on the Costs and Benefits of Federal
Regulations and Unfunded Mandates on State, Local, and Tribal Entities).
-------
The 812 Analyses were initially mandated as ongoing biennial reports to Congress. The
Council understands that the 1995 Reports Elimination and Sunset Act removed the requirement
for the Agency to report to Congress. However, the Council strongly advocates that the Agency
continue to conduct these important benefit-cost assessments as Clean Air regulations continue
to evolve. These analyses provide a rigorous example for other regulatory impact assessments
and serve an important educational role for the Agency. Information requirements identified in
the 812 Analysis stimulate important research both inside and outside the Agency.
The Council emphasizes that the 812 Analyses are not merely a perfunctory accounting
exercise, but an ambitious and difficult enterprise that pushes the Agency to the frontiers of
science in many different disciplines. To an extent unmatched in almost any other benefit-cost
assessment, these analyses require the creative synthesis of knowledge across many interrelated
fields—from engineering to atmospheric chemistry to meteorology to epidemiology and
ecosystems science to toxicology to economics and a number of other specialties.
A significant portion of the value of the 812 Analyses lies in the extent to which they can
shape future regulations and legislation. Their role is not limited merely to assessment of the
1990 Clean Air Act. For example, the Agency learns much from the 812 Analyses that can
guide strategic planning for the programs of the Office of Air and Radiation.
In this report, the Council emphasizes the notion of a "Section 812 Learning Laboratory,"
as well as several technical points that deserve the Administrator's attention. These include
scenario development, mortality risk valuation (which is both important and controversial), the
role of Quality Adjusted Life Years (QALYs) in assessment of the benefits of implementing the
Clean Air Act, uncertainty analysis and characterization, computable general equilibrium (CGE)
modeling for capturing indirect costs and benefits, and approaches to discounting. Highlights for
these topics and others are presented in our Executive Summary.
The Council received 37 formal charge questions in May 2003 from the Agency
concerning technical questions related to data and methodologies identified in the Analytical
Plan for possible use in the Second Prospective Analysis. This Council report addresses
overarching questions concerning the analytical framework for the analysis and detailed
questions related to economic analysis. This report supplements previous reports provided by
the Council's subcommittees on emissions estimation and health effects analysis issues raised by
the Analytical Plan. A third subcommittee, the Ecological Effects Subcommittee (EES) has just
been constituted. Its perspective and advice will be available for future advice.
-------
We appreciate the opportunity to review the Analytical Plan and to provide you with
advice on the design of the Agency's approach so that the resulting study will have the most
validity and utility for the Agency and Congress. The Council would be pleased to expand on
any of the findings described in this report and we look forward to your response.
Sincerely,
/Signed/
Dr. Trudy Ann Cameron, Chair
Advisory Council on Clean Air
Compliance Analysis
-------
Advisory Council on Clean Air Compliance Analysis
Special Council Panel for the Review of the Third 812 Analysis
CHAIR
Dr. Trudy Ann Cameron, University of Oregon, Eugene, OR
MEMBERS
Dr. David T. Allen, University of Texas, Austin, TX
Ms. Lauraine Chestnut, Stratus Consulting Inc, Boulder, CO
Dr. Lawrence Goulder, Stanford University, Stanford, CA
Dr. James Hammitt, Harvard University, Boston, MA
Dr. F. Reed Johnson, Research Triangle Institute, Research Triangle Park, NC
Dr. Charles Kolstad, University of California, Santa Barbara, CA
Dr. Lester B. Lave, Carnegie Mellon University, Pittsburgh, PA
Dr. Virginia McConnell, Resources for the Future, Washington, DC
Dr. Bart Ostro, California Office of Environmental Health Hazard Assessment (OEHHA),
Oakland, CA
Dr. V. Kerry Smith, North Carolina State University, Raleigh, NC
OTHER SAB MEMBERS
Dr. Dale Hattis, Clark University, Worcester, MA
CONSULTANTS
Dr. John Evans, Harvard University, Portsmouth, NH
Dr. D. Warner North, NorthWorks Inc, Belmont, CA
Dr. Thomas S. Wallsten, University of Maryland, College Park, MD
SCIENCE ADVISORY BOARD STAFF
Dr. Angela Nugent, Washington, DC
-------
NOTICE
This report has been written as part of the activities of the EPA Advisory Council on
Clean Air Compliance Analysis (Council), a public advisory group providing extramural
scientific information and advice to the Administrator and other officials of the Environmental
Protection Agency. The Council is structured to provide balanced, expert assessment of
scientific matters related to problems facing the Agency. This report has not been reviewed for
approval by the Agency and, hence, the contents of this report do not necessarily represent the
views and policies of the Environmental Protection Agency, nor of other agencies in the
Executive Branch of the Federal government, nor does mention of trade names of commercial
products constitute a recommendation for use. Reports of the Council are posted on the EPA
website at http://www.epa.gov/sab.
-------
Table of Contents
1. EXECUTIVE SUMMARY 1
2. INTRODUCTION 5
2.1. BACKGROUND 5
2.2. PROCESS FOR DEVELOPING THIS ADVISORY 6
3. PROJECT GOALS AND ANALYTICAL SEQUENCE 7
3.1. CHARGE QUESTION 1 7
3.2. SUMMARY OF COUNCIL RESPONSE 7
3.3. SECTION 812 ANALYSIS AS A LEARNING LABORATORY 8
3.4. DlSAGGREGATION 9
3.5. AIR Toxics 10
3.6. NON-HEALTH BENEFITS 11
3.7. UNCERTAINTY 11
4. SCENARIO DEVELOPMENT AND ALTERNATIVE PATHWAYS 12
4.1. AGENCY CHARGE QUESTIONS 12
4.2. SUMMARY OF COUNCIL RESPONSE 12
4.3. BENCHMARKING AND SENSITIVITY ANALYSIS 13
4.4. CONSISTENCY: ECONOMIC ACTIVITY AND INCOMES 13
4.5. ARTIFICIALITY OF SCENARIOS 14
4.6. TRAJECTORIES AFTER 2000: PREVENTING DETERIORATION 16
4.7. THE MOVING TARGET PROBLEM 16
4.8. TREATMENT OF NAAQS COMPLIANCE 17
5. COST ESTIMATES 18
5.1. CHARGE QUESTION 7 18
5.2. SUMMARY OF COUNCIL RESPONSE 18
5.3. ECONOMETRIC MODELS AND COSTS 19
5.4. DIRECT COSTS VERSUS BROADER DEFINITIONS OF COSTS 20
5.5. VALIDATION AGAINST REALIZED HISTORICAL COSTS 20
5.6. LEARNING 21
5.7. IPM VERSUS HAIKU MODELS FOR COST ESTIMATES 23
5.8. UNCERTAIN FUTURE ENERGY DEMAND CONDITIONS 24
5.9. COMPETING RISKS DUE TO HIGHER ENERGY PRICES 24
5.10. MISCELLANEOUS 25
6. COMPUTABLE GENERAL EQUILIBRIUM MODELING 27
6.1. CHARGE QUESTION 8 27
6.2. SUMMARY OF COUNCIL RESPONSE 27
6.3. COSTS OUTSIDE THE REGULATED MARKET 28
6.4. JUST EX POST COST SPILLOVERS? OR EMISSIONS PROJECTIONS TOO? 28
6.5. COMPETING CGE MODELS 29
6.6. PRINCIPLES FOR CGE MODEL SELECTION 31
6.7. THE TAX-INTERACTION EFFECT 32
6.8. TENSION BETWEEN CGE, ECONOMETRIC MODELS 34
7. DISCOUNTING 36
7.1. CHARGE QUESTION 9 36
7.2. SUMMARY OF COUNCIL RESPONSE 36
7.3. THEORY 37
7.4. THE SOCIAL DISCOUNT RATE AND FIRMS'OPPORTUNITY COSTS OF CAPITAL 38
7.5. IMPORTANCE OF APPLYING A RANGE OF VALUES FOR THE SOCIAL DISCOUNT RATE 39
11
-------
8. ECOLOGICAL EFFECTS ASSESSMENT AND VALUATION 40
8.1. AGENCY CHARGE QUESTIONS RELATED TO ECOLOGICAL EFFECTS ASSESSMENT AND VALUATION
40
8.2. SUMMARY OF COUNCIL RESPONSE 40
8.3. EMPHASIZING VERIFIABLE CONNECTIONS 41
8.4. VALUING STATISTICAL ECOSYSTEMS? 41
8.5. USING AVAILABLE QUANTITATIVE INFORMATION 42
8.6. INTEGRATION BETWEEN CONCEPTUAL BASIS AND CASE STUDIES 42
8.7. INADVISABILITY OF USING PLACEHOLDER VALUES 43
8.8. AWAITING INSIGHTS FROM EES AND THE SAB's C-VPESS 43
8.9. AGENCY PLANS FOR CONDUCTING AN ECOLOGICAL BENEFITS CASE STUDY 43
8.10. PLANS FOR A HEDONIC PROPERTY STUDY 45
9. ECONOMIC VALUATION - PLANS 46
9.1. CHARGE QUESTION 21 46
9.2. SUMMARY OF COUNCIL RESPONSE 46
9.3. DISTRIBUTIONAL EFFECTS 46
9.4. WORKER PRODUCTIVITY 46
9.5. MISCELLANEOUS WELFARE EFFECTS (VISIBILITY AND SOILING/MATERIALS DAMAGE) 47
10. USE OF VSL META-ANALYSES 49
10.1. AGENCY CHARGE QUESTIONS RELATED TO USE OF VSL META-ANALYSIS 49
10.2. SUMMARY OF COUNCIL RESPONSE 50
10.3. EXPERT JUDGMENT - VSLs 51
10.4. ADJUSTING FOR LATENCIES, INCOME GROWTH? 52
10.5. AVAILABLE META-ANALYSES 54
10.6. INTERPRETING CV MEASURES AS OPPOSED TO WAGE-RISK MEASURES 55
10.7. EMERGING CONSIDERATIONS 56
10.8. WHICH META-ANALYSES TO USE 57
10.9. UNPUBLISHED META-ANALYSES? 57
11. QALY-BASED COST EFFECTIVENESS 59
11.1. CHARGE QUESTION 24: 59
11.2. SUMMARY OF COUNCIL RESPONSE: 59
11.3. CHALLENGES AND LIMITATIONS OF CEA 60
11.4. QALYs AS A MEASURE OF EFFECTIVENESS 62
11.5. SUMMARY: 63
12. MORBIDITY EFFECTS 64
12.1. CHARGE QUESTION 25 64
12.2. SUMMARY OF COUNCIL RESPONSE: 64
12.3. GENERAL POINTS 64
12.4. ACUTE RESPIRATORY ILLNESSES AND SYMPTOMS 65
12.5. ASTHMA EXACERBATIONS 66
12.6. NON-FATAL HEART ATTACK 67
12.7. CHRONIC BRONCHITIS 67
13. UNCERTAINTY ANALYSIS - PLANS 68
13.1. CHARGE QUESTIONS CONCERNING UNCERTAINTY ADDRESSED IN THIS REPORT 68
13.2. SUMMARY OF COUNCIL RESPONSE TO CHARGE QUESTION 26 68
13.3. DETAILED COMMENTS RELATED TO CHARGE QUESTION 26 69
13.4. SUMMARY OF COUNCIL RESPONSE TO CHARGE QUESTION 27 CONCERNING THE COMPLIANCE COST PILOT
71
13.5. GENERAL DISCUSSION 72
13.6. SENSITIVITY OR INFLUENCE ANALYSIS 72
in
-------
13.7. OTHER SOURCES OF COST UNCERTAINTY 72
13.8. COMPLIANCE AND ENFORCEMENT ASSUMPTIONS AND CONSISTENCY REQUIREMENTS 73
14. DATA QUALITY AND INTERMEDIATE DATA PRODUCTS 74
14.1. CHARGE QUESTION 32 74
14.2. SUMMARY OF COUNCIL RESPONSE 74
14.3. GENERAL ADVICE 75
14.4. REFINEMENTS OF INPUT DATA 76
14.5. POTENTIAL FOR A LEARNING LABORATORY APPROACH 78
14.6. ITEMIZED LIMITATIONS IN DATA REVIEW 79
14.7. CONSISTENCY CHECKS 79
14.8. UNDERSTANDING SOURCES OF DIFFERENCES 81
14.9. INTERMEDIATE OUTCOMES AND CONSISTENCY CHECKING 81
14.10. ADDITIONAL SPECIFIC RECOMMENDATIONS 81
15. RESULTS AGGREGATION AND REPORTING 83
15.1. CHARGE QUESTION 33 83
15.2. SUMMARY OF COUNCIL RESPONSE 83
15.3. GENERAL OBSERVATIONS 83
15.4. PRIMARY RESULTS 84
15.5. FUTURE FORECASTS AND PRESENT VALUE CALCULATIONS 85
15.6. DlSAGGREGATION 86
REFERENCES 89
APPENDIX A: LIST OF SAB REVIEW CHARGE QUESTIONS AND RELATED CHAPTERS IN THE
AGENCY DRAFT ANALYTICAL PLAN AS RECEIVED FROM EPA ON JULY 3,2003 96
APPENDIX B: LIST OF ACRONYMS 107
APPENDIX C: ADDITIONAL DISCUSSION CONCERNING COSTS AND LEARNING 109
APPENDIX D: ADDITIONAL BIBLIOGRAPHIES Ill
D.I. VALUE OF TIME Ill
D.2. MATERIALS DAMAGE 113
APPENDIX E: ADDITIONAL DISCUSSION CONCERNING THE USE OF VSLS 114
E.I. VSLs vs. MICROMORTS 114
E.2. PROPORTIONALITY 117
E.3. HETEROGENEITY: CONTEXT-DEPENDENT WTP 117
E.4. PROBLEMS WITH META-ANALYSES 118
E.6. WHAT TO DO IN THE NEAR TERM 119
APPENDIX F: SPECIFIC RESERVATIONS ABOUT THE USE OF QALYS IN THE CONTEXT OF THE
SECTION 812 ANALYSES 121
F.I. CONSUMER SOVEREIGNTY AND REPRESENTATIVENESS: 121
F.2. ORDINALITY VERSUS CARDINALITY 121
F.3. HETEROGENEITY IN HEALTH STATES 122
F.4. ECONOMIC BENEFIT ANALYSIS USING QALYs? 122
APPENDIX G: BIOSKETCHES OF MEMBERS OF THE SPECIAL COUNCIL PANEL FOR THE
REVIEW OF THE THIRD 812 ANALYSIS 124
IV
-------
1. EXECUTIVE SUMMARY
EPA requested that the Council provide detailed advice on 37 technical questions related
to the planned Second Prospective Analysis. Overall, the Agency's general approach to this
major benefit-cost analysis has become much more mature and complex with this third
undertaking. The Council's response to each charge question begins with a set of bulleted points
that highlight the key issues in the discussion. Here, the Council summarizes the most important
recommendations for strengthening the Agency's plans for conducting the 812 Analysis. The
points are ordered roughly in terms of the Council Special Panel's sense of the importance of the
topic.
The first three issues highlighted below—the "Learning Laboratory," uncertainty, and
issues of integration and consistency—are pervasive. Related to them, several other issues have
importance of special note: (1) discounting; (2) the indirect costs revealed by Computable
General Equilibrium models; (3) the Value of a Statistical Life; and (4) development of methods
for assessing benefits associated with ecological effects and regulation of air toxics. These
controversial issues have posed challenges in past 812 Analyses and will likely reappear in the
course of future benefit-cost analyses by the Agency. They will continue to demand the
Agency's close attention.
The Learning Laboratory. The series of 812 Analyses, if they are to incorporate the state
of the art in relevant disciplines, must involve auxiliary activities that can be collected under an
umbrella that might be termed the "812 Learning Laboratory." The Council advises the Agency
to develop a public and expert process to review new data and methods for upcoming 812
Analyses carefully and to evaluate the rationale for incorporating new data and methods in
subsequent analyses. When warranted, these approaches can then be moved into the mainstream
for the next 812 Analysis, replacing less suitable data or methods used in previous studies.
Candidates for the Learning Laboratory process include broadly cross-cutting issues that will
have implications not just for the 812 Analyses, but for many other benefit-cost analyses
conducted at the Agency and elsewhere, including a number of the issues itemized directly
below.
Uncertainty. The Council applauds the Agency's intentions to incorporate much more
recognition of uncertainty in the Second Prospective Analysis than was present in the First
Prospective Analysis. In the Second Prospective Analysis, the Agency intends to address the
pervasiveness of uncertainty in both its cost and its benefit estimates. Those elements that are
both highly uncertain and have the potential to change the results significantly should be the
focus of sensitivity analyses. The results of these sensitivity analyses should be presented in
close proximity to the central estimates in summary tables of Clean Air Act Amendment
(CAAA) impacts. Sensitivity/uncertainty analysis needs to be an iterative process to identify and
assess the significance of key uncertainties in each step of the assessment. As a practical matter,
only a selected set of the most influential uncertainties should be quantitatively followed all the
way through to the final results. The Council advises the Agency to develop its uncertainty
analyses with reference to the recommendations in reports of the National Research Council
-------
(2002) and OMB (2003). It also advises the Agency to utilize the list of "key uncertainties" from
the First Prospective Analysis as a framework.
In the Executive Summary of the planned Prospective Analysis and in the body of the
text itself, the Agency should report its best central estimate as the "base case." Alternative
cases should be associated with likelihoods of these cases and any provision of a "low"
alternative estimate should be balanced by a corresponding "high" alternative estimate. Pivotal
assumptions should be clearly identified and the need for additional research on these issues
should be emphasized.
Issues of Integration, Consistency, and Validation. The 812 Analyses have become a
more complex modeling enterprise, and public and OMB scrutiny has increased with respect to
federal efforts that use models as the basis for developing policy tools. Thus, the Council Special
Panel emphasizes the importance of choosing consistent and compatible modeling assumptions
across all components of the analysis. Especially important issues arise in this regard in the
areas of discounting and computable general equilibrium (CGE) analysis. The Council also
advises the Agency to consider approaches for assuring data quality and providing intermediate
information about analytical results that will improve the quality of the overall analysis and
increase the transparency of the benefit-cost exercise, while not resulting in substantial costs to
the Agency.
Discounting. The Prospective Analysis will derive discounted values of the projected
benefits and costs resulting from Clean Air Act emissions reductions for selected future years.
Such discounting should be performed using a "social discount rate" throughout the analysis,
unless the Agency wants to show discounted private costs as perceived by an individual firm.
The Council commends the Agency drawing attention to the challenges and uncertainties
associated with the choice of social discount rate. The Council urges the Agency to employ a
range of values - perhaps between 3 and 7 percent, with a central case of 5 percent - for the
social discount rate in its assessments.
Indirect Costs and Use of Computable General Equilibrium Models. Incorporation of
indirect "spillover" costs of air quality regulations is important and these costs should continue to
receive close attention. CGE models have the capability to reveal indirect costs and other
consequences of air quality regulations that spill over into unregulated sectors, not just to better
estimate the direct costs of regulation on regulated sectors. The current Analytical Plan
describes CGE methods only for "post-processing" and relegates them to secondary status
compared to engineering estimates of compliance costs. Ideally, general equilibrium modeling
should enjoy similar status to direct cost calculations, even though each of the main CGE models
proposed for use in the 812 Analysis has some limitations. CGE models and econometric models
for costs are not competing methods, but complementary methods. Indirect costs should be
defined and itemized more clearly in the Analytical Plan. Ongoing comparisons of the predicted
and actual costs of air quality regulations will be important to the evolution of the ongoing
Section 812 Analyses.
Value of Human Health Risk Reductions Associated with Reductions in Air Pollution.
Ideally, uncertainty analysis with respect to Value of a Statistical Life (VSL) assumptions
-------
requires information about the distribution of VSL estimates corresponding to risks and
populations that are similar to those relevant for the CAAA. The marginal distribution of all
empirical VSL estimates derived across all contexts is unlikely to be appropriate for this purpose,
as is any arbitrary convenient assumption about distributional shape. Unfortunately, there are
very few, if any, values in the literature that are derived in a context that is a sufficiently close
match for this policy context. Instead, the playing field is occupied by unexplained large
differences in VSL estimates, even those derived in very similar (e.g. workplace) contexts.
The Panel does not wish to encourage the strategy, pursued in the first analytical
blueprint, of excluding a variety of VSL studies on fairly arbitrary criteria because the are
"unsuitable." Resolution of this issue awaits the findings of further comprehensive meta-
analyses. While the results of different meta-analyses continue to come in, the Council might
lean toward recommending reliance on the Viscusi-Aldy estimates of VSLs based on U.S.
studies. However, these are limited to wage-risk studies and it is probably premature to conclude
that the Viscusi-Aldy analysis provides the last word. The Agency should not rely exclusively
on the Kochi et al. meta-analysis, which has not yet been peer-reviewed and published.
The Council understands the Agency's interest in conducting cost-effectiveness analysis
since this is being required by OMB in addition to benefit-cost analysis for major regulations.
The Council has had difficulty, however, in coming to full agreement about the appropriateness
of Quality Adjusted Life Years (QALYs) for use in this context. The limitations of the measure
have led some members to want to recommend against using it at all, but others are more
comfortable endorsing exploratory efforts to apply the measure, even though they also
acknowledge the same limitations. The deliberations of the Institute of Medicine's Committee to
Evaluate Measures of Health Benefits for Environmental, Health, and Safety Regulations can be
expected to be of considerable value in resolving some of the Council's concerns. In addition,
the Council wants to emphasize that there are important limitations of any cost-effectiveness
analysis for a regulatory program as broad as the Clean Air Act Amendments, because there are
many other classes of benefits besides human health benefits to be taken into consideration.
While cost-effectiveness analyses do not belong in the main 812 Analysis, because the latter is
defined as a benefit-cost analysis, the Council recognizes that the Agency may wish to develop
alternative cost-effectiveness analyses and these are appropriate for consideration with the
"Learning Laboratory."
Concerning morbidity, the Agency should continue to use Willingness-to-Pay (WTP)
estimates for morbidity values, rather than cost-of-illness (COI) estimates, should these be
available. Where WTP is unavailable, COI estimates can be used as placeholders, awaiting
further research, provided suitable caveats are included in the analysis. The Dickie and Ulery
study is a valuable addition to the repertoire of empirical results concerning WTP for acute
respiratory illnesses and symptoms, although it is not so superior as to supercede all earlier
studies.
Ecological Effects. Human health risk reductions may be the most substantial benefit
from the CAAA, but they are not the only important benefit. Benefits to ecosystems and other
welfare benefits such as visibility are likely to be substantial and are still receiving limited
attention. The Council nevertheless recognizes substantial challenges in quantitative assessment
-------
of these benefits. The greater heterogeneity in ecosystems services makes it even more difficult
to produce estimates of the benefits from their protection than for the protection of human health.
Ecological effects to be valued must be limited to those effects for which there is a defensible,
rather than just speculative, link between air emissions and service flows. The Council strongly
objects to using inappropriate or unsupported placeholder values in the absence of better
information.
The advice of the new Council Ecological Effects Subcommittee (EES) may be able to
stimulate more progress in the analytical work in this area, as well as the development of greater
expertise on this issue than is presently available. The Council also notes that the Science
Advisory Board (SAB) Committee on Valuing the Protection of Ecological Systems and
Services (C-VPESS) will be providing advice generally to the Agency on this topic. The
Council plans to follow the progress of this new Committee closely for insights helpful to the
812 process.
Hazardous Air Pollutants. Appropriate methods for measuring the benefits of reducing
hazardous air pollutants continue to present a challenge for the 812 Analysis. Great uncertainty
about the character and magnitude of health effects at ambient exposure levels will continue to
hamper valuation efforts, but the potential importance of this category of benefit necessitates
continued careful attention to this task.
-------
2. INTRODUCTION
2.1. Background
The purpose of this Advisory is to continue the Council's advice to the Agency in
developing the third in a series of statutorily mandated comprehensive analyses of the total costs
and total benefits of programs implemented pursuant to the CAAA. Section 812 of the CAAA of
1990 requires the EPA periodically to assess the effects of the 1990 CAA on the "public health,
economy and the environment of the United States" and to report the findings and results of the
assessments to Congress. Section 812 also established the Council and gave it the following
mission: "to review the data and methodology used to develop the 812 Analysis and to advise the
EPA Administrator concerning the utility and relevance of the Study." EPA has, to date,
completed two assessments and received the advice of the Council on them: The Benefits and
Costs of the Clean Air Act: 1970 to 1990 (published 1997) and The Benefits and Costs of the
Clean Air Act, 1990 to 2010 (published 1999).
In this document, a special panel of the Council provides a review of the May 12, 2003
Analytical Plan for the study, and revisions to that plan dated July 8, 2003. The Analytical Plan
is more formally titled Benefits and Costs of the Clean Air Act 1990-2020: Revised Analytical
Plan for EPA's Second Prospective Analysis. The Analytical Plan reflects earlier advice that the
Council provided in September 2001 in its earlier Advisory concerning a draft version of the
Analytical Plan (EPA-SAB-COUNCIL-ADV-01-004).
In the course of this review of the 2003 Analytical Plan, the Council has reviewed the
Agency's major goals, objectives, methodologies, and analytical choices for the Section 812
Analysis before the analysis will be implemented. In its review of the Analytical Plan, the
Council and its panel and subcommittees were guided by the charge questions as identified in the
CAAA of 1990.J
a. Are the input data used for each component of the analysis sufficiently valid and
reliable for the intended analytical purpose?
b. Are the models, and the methodologies they employ, used for each component of
the analysis sufficiently valid and reliable for the intended analytical purpose?
c. If the answer to either of the two questions above is negative, what specific
alternative assumptions, data or methodologies does the Council recommend the
Agency consider using for the second Prospective Analysis?
The Agency provided the Council with additional detailed charge questions for its
consideration. These detailed charge questions were initially provided to the Council in May
Specifically, subsection (g) of CAA §312 (as amended by §812 of the amendments) states: (g) The Council shall — (1) review
the data to be used for any analysis required under this section and make recommendations to the Administrator on the use of
such data, (2) review the methodology used to analyze such data and make recommendations to the Administrator on the use of
such methodology; and (3) prior to issuance of a report required under subsection (d) or (e), review the findings of such report,
and make recommendations to the Administrator concerning the validity and utility of such findings."
-------
2003 and were then revised and resubmitted in July 2003. The final set of 37 charge questions is
included in Appendix A. Appendix A also indicates charge questions that have been addressed
in detail by the Council's Air Quality Modeling Subcommittee (AQMS) and Health Effects
Subcommittee (HES) and documented in their two reports, which have been reviewed and
finalized by the Council.2
The Council envisions that its new Ecosystems Effects Subcommittee (EES) will provide
additional expertise to assist the Council in responding fully to Charge Questions 18-20
concerning ecological assessment and valuation, for which only provisional, limited responses
are given in this report.
2.2. Process for Developing this Advisory
To address the charge questions identified by the Agency regarding the Analytical Plan,
the SAB Staff Office, with the advice of the Council Chair, formed a Special Council Panel for
the Review of the Third 812 Analysis to provide the Council with additional expertise in the
areas of expert elicitation, uncertainty analysis and statistical and subjective probability. The
Staff Office also issued a call for new membership on the Council's AQMS and its HES.
The Council Special Panel held a public teleconference on May 28, 2003 to plan its
approach for providing advice. Those members participating in the teleconference voted to
cancel a planned face-to-face meeting during June 11-13, 2003, pending more information about
those portions of the Analytical Plan that were to be revised. The majority of these revisions
were completed and submitted to the Council on July 8, 2003. The Council held one
teleconference on July 11, 2003 and another on July 15, 2003, where a subset of the charge
questions considered most urgent by the Agency were addressed. Those charge questions were
1, 2, 3, 7, 8, and 9. Teleconferences on September 23, 2003 and September 24, 2003 continued
this discussion and also addressed charge questions 32 and 33. A teleconference on October 23,
2003 reviewed the draft report on discussion to that point. Discussion of question 1 (Project
Goals and Analytical Sequence), question 3 (Alternative Pathways) and question 9 (Discounting)
raised the need for additional information from the Agency, so discussion was deferred to
November 5-6, 2003 when the first face-to-face meeting of the Panel was held in Washington,
D.C. Subsequent teleconferences were held on December 19, 2003, December 22, 2003, and
March 18, 2004.
In addition to the advice provided in this document, the Council's AQMS has met to
address issues concerning the Agency's plans for estimating emissions and the HES has met to
address the Agency's plan to assess health effects. The advice developed by these Council
Subcommittees is provided in separate reports.
2 The Advisory on Plans for Emissions Estimation Presented in the May 12, 2003 Analytical Plan: An Advisory by
the Air Quality Modeling Subcommittee of the Advisory Council for Clean Air Compliance Analysis (EPA-SAB-
COUNCIL-ADV-04-001), and the Advisory on Plans for Health Effects Analysis in the Analytical Plan for EPA 's
Second Prospective Analysis - Benefits and Costs of the Clean Air Act, 1990-2020: An Advisory by the Health
Effects Subcommittee of the Advisory Council for Clean Air Compliance Analysis (EPA-SAB-COUNCIL-ADV-04-
002).
-------
3. PROJECT GOALS AND ANALYTICAL SEQUENCE
3.1. Charge Question 1
Does the Council support the study goals, general analytical framework, disaggregation
plan, analytical sequence, and general analytical refinements defined in chapter 1? If there are
particular elements of these plans which the Council does not support, are there alternatives the
Council recommends?
3.2. Summary of Council Response
• The series of 812 Analyses, if they are to incorporate the state of the art in relevant
disciplines, must involve auxiliary activities that can be collected under an umbrella
that might be termed the "812 Learning Laboratory." Of course, the main policy
analysis in each cycle must be based upon fully vetted methods and data. However,
the expectation of changes and improvements in methods should be institutionalized
by an ongoing process of formal evaluation of proposed enhancements. As
enhancements are carefully reviewed and the reasons for them thoroughly
understood, they can be moved into the next main policy analysis, replacing inferior
approaches used in previous studies. Candidates for the Learning Laboratory process
include broadly cross-cutting issues that will have implications not just for the 812
Analyses, but for many other benefit-cost analyses conducted at the Agency and
elsewhere.
• Disaggregation is a very desirable strategy which should be pursued to the extent that
analytical resources permit, subject to the constraints imposed by nonlinearities and
general equilibrium effects. The Council supports the Agency's plans to report costs
and benefits disaggregated by major economic sectors as an important addition for the
Second Prospective Analysis.
• Air toxics remain an important issue in the 812 Analysis. The benzene case study is a
good start, but much more work is still necessary. Case studies on a few selected
Hazardous Air Pollutants (HAPs) are merely a beginning.
• Human health risk reductions may be the most substantial benefit from the CAA, but
they are not the only important benefit. Benefits to ecosystems and other welfare
benefits such as visibility are likely to be substantial and are still receiving limited
attention. The Council nevertheless recognizes substantial challenges in quantitative
assessment of these benefits.
• Chapter 1 of the 812 Analysis should address the pervasiveness of uncertainty in cost
and benefit estimates, but then identify the methods the Agency will use to identify
the most important areas of uncertainty. Those elements that are both highly uncertain
and have the potential to significantly change the results should be the focus of
-------
sensitivity analyses. The results of these sensitivity analyses should be presented in
close proximity to the central estimates in summary tables of CAAA impacts.
Sensitivity/uncertainty analysis needs to be an iterative process to identify and assess
the significance of key uncertainties in each step of the assessment. Only a selected
set of the most influential uncertainties should be quantitatively followed all the way
through to the final results.
3.3. Section 812 Analysis as a Learning Laboratory
The Council emphasizes that the Agency's Prospective Analyses address important
policy questions with a very broad audience. As a result, these analyses attract significant public
attention. This status poses challenges for the Agency's efforts to innovate and reflect new
research insights on a continuous basis. Any recommendations to modify existing
methodologies to take advantage of the most up-to-date insights from the relevant literature may
be viewed with suspicion by different groups of stakeholders if their interests are affected by
these methodological changes. To protect the Agency's credibility, there is a need to balance
innovation in methods against the appearance of manipulation of results to achieve some implicit
predefined objective.
These concerns seem to require that the long-term analysis protocol for the Prospective
Analyses distinguish three separate classes of Agency activities:
a. "Policy Evaluations" - Analyses included as part of the formal 812 Analyses.
These activities are based on established and fully vetted methods, even if the
inputs are somewhat less than ideal (e.g. they may be identified as resorting to the
best available approximations for some needed measurements).
b. "Satellite or Experimental Evaluations" - These activities use proposed methods
and new techniques that have not yet been fully vetted. The models currently
used in Policy Evaluations, such as those included in 812 Analyses, may embody
some assumptions that deserve examination either on the basis of new data, or a
priori on the basis of theory. The need for improved models may be readily
acknowledged, and exploratory Satellite/Experimental Evaluations will address
this need.3
c. "Formal Review and Discussion" - These activities will precede the development
of Satellite/Experimental Evaluations. The Agency needs to make a commitment
to involve the research community in discussions that assess possible new
methods through workshops or conferences, detailed and comprehensive reviews
of unofficial analyses, and evaluation of their implications in working papers and
3 In these evaluative activities, the Agency would parallel the Bureau of Economic Analysis (BEA) satellite accounts
for the national income and product accounts, or the provisional or unofficial price indexes developed by Bureau of
Labor Statistics (BLS). In each of these analogous classes, the research staff of the relevant agency develops and
publishes results designated as exploratory. These exploratory results are carefully documented and are intended for
general review and criticism. However, they would not be used for policy making or included in 812 Analysis at
this stage.
-------
published articles. For example, this approach has been taken in price index
development at the BLS.
All three classes of activities should probably be ongoing, all the time. This formal
process would institutionalize the recognition that methodological innovations over time are a
natural and expected part of progress on this front. This process would also emphasize that
changes in methodology require full disclosure and discussion of the implications of new
methods - both their strengths and their weaknesses. The disclosure and discussion process is
not simply a matter of refereed publication followed by Agency adoption of new methods.
Instead, it is one of attaining broad public understanding of the assumptions involved in different
approaches and acceptance of the reasons for changes in methodologies.
At present, this tiered approach to methodological innovation is not an established
component of the Agency's research in support of policy, although there have been occasional
instances. The Council Special Panel recommends that this component be given serious
consideration. It is only through a commitment to internal but widely circulated public efforts to
review, evaluate and understand new methods that the Agency can promote necessary analytical
innovations, yet avoid the appearance of strategic manipulation of the process.
Additional discussion of the Learning Laboratory may be found in Section 14.5 of this
Advisory Report
3.4. Disaggregation
The Council commends the Agency's willingness to disaggregate, something that the
Council has recommended for some time. In an ideal world, the disaggregation would be at the
level of individual regulatory decisions so that the Agency, Congress, and society would know
whether each regulation should be tightened or loosened. Effort toward disaggregation to the
level of individual sectors is an important step. The next steps beyond sectoral disaggregation
might be limited regulation-by-regulation disaggregation and/or some cautious region-by-region
disaggregation (although this is likely to be more feasible for selected benefits than for costs).
There remain some important constraints on the task of disaggregation. The Council
understands that it is often impossible to separate the benefits or costs of abating one pollutant
versus another. Analytical resource constraints must also be accommodated. The Council also
warns that the benefits and/or the costs associated with different sectors, regulations, or regions
may not be additively separable because of nonlinearity or interaction effects among the
disaggregated entities. In addition, general-equilibrium adjustments may shift incidence among
sectors and regions. These complications make the process of disaggregating benefits and costs
more difficult. However, decision makers often are interested in sectoral and regional effects.
Providing disaggregated estimates wherever possible will increase the usefulness of the analysis
in policy making.
The Council suggests that the Agency consider disaggregating by region or program on a
case-by-case basis, where costs are significant or other policy needs are well articulated, and then
evaluating the result.
-------
3.5. Air Toxics
The plan to address the particular benefits and costs of abating toxics is a step forward
and the Council endorses this effort. Although the proposed case study on benzene will be very
helpful, the assessment of air toxics for 812 Analyses should not be expected to stop there. For
example, Congress mandated maximum achievable control technology (MACT) for a list of
chemicals, but the chemicals on this list were not identified by any rigorous systematic analysis.
This mandate has imposed substantial costs on the economy without any formal assessment of
either its benefits or its costs.
The Agency is entering a period when it must examine the residual risk after MACT to
determine whether more stringent regulations are required in some cases. One role of the
Section 812 Analyses is to explore new methods relevant to the assessment of environmental
management strategies. This is a good reason for the Second Prospective Analysis to address the
task of benefit-cost analysis with respect to the control of air toxics. The Agency is likely to find
that MACT is supported by benefit-cost analysis for some chemicals and not supported for
others. These insights will be important to the Administrator, to Congress, and to society more
generally. While some environmental laws are implemented with a safety standard in mind,
rather than an efficiency standard, environmental laws should be implemented so as to be
consistent with the best available scientific information.
The benzene study was recommended in the last round of Council advice primarily
because of the relatively greater availability of data on this HAP. It would be useful to have the
Agency propose some other target examples for case studies. Whether these can actually be
pursued in the context of the Second Prospective Analysis is questionable, but assessment of
HAPs should be a priority among longer-term assessment tasks facing the Agency.
As a starting point for future analyses, perhaps the Agency should pick at least one
chemical that is likely to have regulatory benefits exceed costs, and at least one chemical that
will have costs exceed benefits. This would constitute a useful demonstration exercise that could
reveal what resources are required for this type of air toxics analysis. Alternatively, some
argument can be made that it would be preferable to see a more representative sample of HAPs
being analyzed, for example, those from relatively small sources, such as perchlorethylene from
dry cleaning establishments, or chromate from plating operations. These tend to be from isolated
sources, rather than major sectors, and to be common in urban areas.
Are case studies really useful in the formal benefit-cost analysis of the Section 812
Analysis? Perhaps not directly, but the Council advocates these exercises as part of "progress
toward a goal," rather than suggesting that they represent any intermediate or final input to the
current benefit-cost analysis. More complete and more formal analysis of air toxics is certainly
needed as the Section 812 analytical process matures. As in the case of certain aspects of the
calculation of non-market economic benefits, the air toxics tasks fall into the category of
methods development, or contributions to the evolution of a body of knowledge—efforts that are
relevant to the ongoing Section 812 analytical activity. Fostering valuable new research is a
tangential goal of the 812 process.
10
-------
3.6. Non-health benefits
Mortality risk reduction benefits are about 90% of total benefits quantified in the
previous Section 812 Analyses. But it is likely to be implausible to most people (and most
members of Congress) that non-mortality health benefits are small, or that benefits other than
human health benefits are tiny or immeasurable. The Analytical Plan touches on visibility as a
non-health effect. More contentious, and potentially very important, are the benefits from
protection of the natural environment (ecosystems) stemming from the CAA.
In the first round of advice from the Council to the Agency concerning the Second
Prospective Analysis (EPA-SAB-COUNCIL-ADV-01-004), the Council emphasized that the
Costanza et al. (1998) method was an inappropriate way to approach the task of ecosystem
benefits estimation. However, the Agency cannot ignore this category of benefits or continue
simply to characterize their valuation as intractable. Certainly the planned case study is too little.
Delays in bringing online the new subcommittee of the Council, the EES, and the SAB C-
VPESS may lead to corresponding delays in any advice that can be provided to the Agency
concerning the challenges presented by valuation needs in this area. Nevertheless, the
importance of this category of benefits should be recognized in the Prospective Analysis.
3.7. Uncertainty
Uncertainty will be addressed much more comprehensively in the Council's discussion of
Chapter 9 of the Analytical Plan. However, with respect to the overview of the Agency's goals
in Chapter 1, it would be helpful to see more attention to the pervasiveness of the problem of
uncertainty, especially where linearity assumptions are crucial and tenuous. Uncertainty analysis
is something that needs to be ongoing throughout the assessment process. Informed judgments
need to be made about what might be the key sources of uncertainty, and the potential
consequences of this uncertainty, in each step of the assessment.
However, this does not mean that every alternative model and alternative assumption
needs to be tracked all the way through the assessment to the bottom line. The Council does not
wish to lead the Agency down an intractable path of including so many alternative models and
alternative assumptions that the assessment loses its focus and coherence. For example, it is
vitally important that the electric utility cost analysis involve some assessment of how sensitive
the cost results are to different assumptions about the future price of natural gas and general
economic growth, and some discussion of this exploration should be reported in the Second
Prospective Analysis. Those elements that are both highly uncertain and have the potential to
change the results significantly should be the focus of sensitivity analyses. The results of these
sensitivity analyses should be presented in close proximity to the central estimates in summary
tables of CAAA impacts.
11
-------
4. SCENARIO DEVELOPMENT AND ALTERNATIVE PATHWAYS
4.1. Agency Charge Questions
Charge Question 2: Does the Council support the choices for analytical scenarios defined
in Chapter 2? Are there alternative or additional scenarios the Council recommends EPA
consider for inclusion in the analysis?
Charge Question 3: Does the Council support the alternative compliance pathway
estimation and comparison methodology described in chapter 2, including the specification of
alternative compliance pathways which may not reflect precisely constant emissions or air
quality outcomes between scenarios due (primarily) to the non-continuous nature and interaction
effects of emission control options?
4.2. Summary of Council Response
• Agency Charge Question 3 was made largely obsolete by revisions in the Analytical
Plan that were made clear to the Council at its November 4-5, 2003 meeting and thus
this Council report does not address the question.
• The evolving baseline assumptions for the 812 Analysis need to be carefully
benchmarked against realized values of key forecasts from previous editions of the
analysis, and sensitivity analysis with respect to key assumptions will be important.
• Care must be taken to ensure that key assumptions affecting different components of
the overall 812 Analysis (discount rates, income growth projections, substitutability)
are consistent across all the models used in the analysis.
• The "with CAAA" and "without CAAA" scenarios are neither observable nor likely
to materialize exactly as described. They are artificial constructs. However, they
should at least be internally consistent.
• The Agency should make it very clear to the audience for the 812 Analysis to what
extent the post-2000 benefits of the CAAA are expected to stem from the prevention
of deterioration in air quality versus absolute improvements from 1990 conditions.
• The evolutionary nature of regulations pursuant to the CAAA means that it is difficult
to forecast future benefits and costs based solely on knowledge of the shape of current
regulations. The Agency needs to be clearer about how feedback and regulatory
evolution will be modeled.
• Finally, the Council applauds the Agency's transition to short turn-around air-quality
models that will enhance opportunities for sensitivity analyses.
12
-------
4.3. Benchmarking and sensitivity analysis
First, the Council recommends changing the description of the different scenarios from
"pre-CAAA and post-CAAA" to "with CAAA and without CAAA." This simple change will
eliminate confusion between differences over time and counterfactual differences over
alternative scenarios, which is the intended distinction.
To evaluate the implications of the proposed update of the 1990 Baseline Emissions
assumptions, it would be helpful to have an explicit comparison of how the proposed update to
the 1990 baseline differs from the earlier 1990 baseline. The Second Prospective Analysis
should compare the ambient pollution concentrations implied by the 1990 baseline used in the
First Prospective Analysis versus the new baseline, and each ambient concentration should be
compared with the 1990 actual monitored values for each pollutant. This could be done for
targeted metropolitan areas (e.g., the Los Angeles air basin).
The description in the First Prospective Analysis suggests that a scaling factor was used
to adjust the projected ambient air quality in 2000 and 2010. This scaling factor was apparently
derived by taking the ratio of modeled target year to modeled base year and applying this ratio to
scale base year concentrations (whether monitored directly or estimated using Voronoi Neighbor
Averaging) to get the projected target year concentration. This type of benchmarking, of
backcasted simulations to actual observed outcomes in 1990 and 2000, should be possible in the
Second Prospective Analysis. It would help policy-makers understand the sensitivity of the
results from air quality models to changes in the emissions profiles used in the analysis.
4.4. Consistency: economic activity and incomes
At the time the analysis was done for the First Prospective Analysis, expectations for
economic activity were completely different than the realities experienced between 1999 and
2003. There is no discussion of how the recent slowdown in economic activity is being
incorporated into the projections for 2000, 2010, and 2020. There must be some discussion of
this linkage. A component of the uncertainty analysis will have to consider the status of the
aggregate economy, including any assumptions about when there may be a return to a more
robust growth pattern. Otherwise, the exercise might seem foolish.
There should be some explicit discussion of the connections between assumptions about
economic activity at the aggregate level and the corresponding assumptions about household
income growth that underlie the benefit measures. These assumptions should be consistent
throughout the analysis. The Agency needs to make its "central case" economic assumptions
clear, although the Council notes that there will continue to be considerable uncertainty about the
nature of the relationship between economic activity and emission rates. Even a well-defined
central case assumption about future levels of economic activity will not lead to an unambiguous
forecast about pollutant emissions.
There is a need for sensitivity analysis concerning any assumptions about the baseline
level of overall macroeconomic growth. However, the need to understand uncertainty about
baseline growth rates for the economy as a whole is distinct from the need to understand the
uncertainty about any differences in growth rates across individual sectors of the economy. It is
13
-------
possible that assessments of the behavior of particular sectors are excessively dependent upon
the predictions of just a small set of models. These models are, in general, rather highly
aggregated and have been developed for different purposes than those for which they are being
used in the Second Prospective Analysis. The Agency should use alternative models and solicit
expert judgment on these issues, perhaps via a workshop. Rather than starting with the
predictions of these models, it is important to step back and evaluate each model's assumptions
and the sensitivity of its predictions to these assumptions.
Consistency is also an important issue in several other places in the Analytical Plan. For
example, there is some discussion of meta-analysis with respect to the value of a statistical life to
be used in the analysis. In the context of this discussion, there is mention of the prospect of
making adjustments to VSL estimates to account for differences in income levels of the original
study populations. How do these proposed income adjustments correspond to the income
changes that are part of the general equilibrium consequences of the effects of air quality
regulations on costs of production and therefore upon factor demands?
Finally, the underlying assumptions of different types of models used in the Analysis
must be compatible. Most procedures for benefits assessment based on revealed preferences of
individuals hinge crucially upon non-separability between pollution levels and observable
behaviors. It is highly inconsistent to require non-separability in support of the valuation portion
of the analysis that supports the benefits estimates, yet to preclude it in the general equilibrium
assessment of cost estimates. How are the insights from Williams (2002, 2003) concerning
health effects and optimal environmental policy to be incorporated as adjustments? Will there be
scenarios to test the sensitivity of the cost estimates to these adjustments?
4.5. Artificiality of scenarios
Scenarios are being developed for the Second Prospective Analysis for 1990, 2000, 2010,
etc. Obviously, some of the analysis needs to be done well before the point in time when the
outcome levels for all activities in all periods are known. The First Prospective Analysis was
done in 1997. At that time, the scenario data for 1990 was presumably based on actual levels of
economic activity and actual emissions. In 1997, however, the scenario for 2000 could not have
been based on realized levels of economic activity or emissions. There will have been a number
of important variables intended to capture the consequences of the CAAA by 2000 that would
have needed to be forecasted.
From the perspective of 2004, how well do the 1997 ex ante levels (assumed for the year
2000 for these "with CAAA" values of the variables) compare to the levels actually realized,
now that the data for 2000 are available? If what was observed when the actual data for 2000
became available was different from what was assumed in 1997 for the "with CAAA" scenario,
what were the reasons and what were the differences? The Agency needs to be concerned with
level of economic activity and with the levels of emissions resulting from that level of economic
activity. If there are any important "lessons" from the 1997 analysis, what do they imply for the
Second Prospective Analysis, in terms of accuracy in forecasting the level and mix of economic
activity with and without the CAAA regulations in place?
14
-------
In forecasting future conditions under the "with CAAA" and "without CAAA" scenarios,
a number of concerns may be relevant. For example, some non-attainment areas will remain out
of attainment. It is also difficult to fully anticipate all of the general equilibrium consequences of
the CAAA regulations. Looking into the future, both the baseline and the control cases are based
on hypothetical scenarios defined to meet the specific mandates of the CAAA. All of these
scenarios involve some necessary simplification, so that neither the baseline nor the control
scenarios is intended to be an exactingly accurate forecast of future real conditions. Conceding
the need to simplify, however, it is still not clear from the description of the different scenarios
how a couple of important issues are to be addressed:
a. If firms are currently minimizing costs, increased emission controls imply higher
costs and, under the assumptions of most CGE models, higher prices. These price
increases will change the distribution of economic activities by sector and the
resulting levels of emissions from each sector. How are these general equilibrium
consequences of emissions controls to be handled? Shouldn't there be
comparisons that allow uncertainties in aggregate economic activity and technical
change to be described, especially as one attempts to forecast activity levels and
emissions further into the future (e.g., beyond 2010)?
b. What is the nature of the feedback loop to measure changes in household incomes
in response to these policies? At a minimum, one should be able to deal with
Hazilla-Kopp, Jorgenson-Wilcoxen type computations of the effects of policy on
their measures of costs. The price vectors derived from these models include
wages and returns to capital, so it should be possible to evaluate the implied
changes in household incomes. This type of interconnectedness is very relevant
to the process of scenario development. It is not clear in the Analytical Plan
whether there are inconsistencies across components in the different assumptions
about how economic activity affects the outcomes.
It is a daunting task to accommodate fully all of the general equilibrium interactions in
the economy that will ensue from environmental regulations with the scope and impact of the
CAAA. The abilities of researchers to build sufficiently complex models are still evolving. The
Agency, however, should stay focused on the fundamental importance of the fact that the level
and mix of activity in the US economy is a function of CAAA implementation. One cannot hold
fixed the level and mix of economic activities, independent of the regulatory regime. Thus, it is
not relevant to consider "with CAAA" and "without CAAA" scenarios that do not reflect the
endogeneity of economic activity. For smaller and more local regulatory interventions, it might
be a reasonable approximation to assume that the level and mix of economic activities would not
be affected by the presence or absence of the regulation, but this assumption almost certainly
cannot be made for the CAAA.
In an extreme example, imagine that clean air regulations mandated the installment of
equipment that was expensive to both purchase and operate. But suppose that this equipment
was completely ineffective at reducing air emissions of pollutants. The pollution control
equipment itself would contribute nothing to the reduction of emissions. However, by affecting
marginal and fixed costs and output prices, and therefore altering the output and shut-down
15
-------
decisions of firms and the incomes of factor owners, these regulations would have a measurable
effect on total emissions.
The description of the proposed analysis could be enhanced if the Agency could provide
a clearer specification of its plans in terms of selecting the levels and mixes of economic
activities under the different regulatory scenarios. The issue of the level and mix of economic
activity needs to be presented separately from the discussion of aggregate emissions. If only
emissions are presented, one cannot benchmark the baseline and control scenarios in terms of
what they imply for the levels of economic activity.
4.6. Trajectories after 2000: preventing deterioration
The Council now understands that the shapes of the time profiles in Exhibit 2-1 of the
draft Analytical Plan are not factual, and that the diagram is merely a schematic designed to
identify the different reference periods. However, the "without-CAAA" and "with-CAAA"
trajectories in this diagram, if at all realistic, suggest to readers that for 2010 and 2020, the
benefits of the CAAA may result to a significant degree from how high emissions would have
risen without it. It will be important to communicate to policy makers that a significant share of
the benefits that the Second Prospective Analysis is likely to identify for 2010 and 2020 stem
from the prevention of air quality deterioration that would otherwise have occurred.
4.7. The moving target problem
The inventory of new regulations and changes since the first Prospective Analysis (pages
2-9 and 2-10) highlights that the Clean Air Act was designed to be an evolving regulatory
process [e.g., with periodic reviews of the National Ambient Air Quality Standards (NAAQS)].
This adaptive evolution allows for adjustments and/or additions to the arsenal of regulations and
emission control strategies in response to new scientific or engineering knowledge and
technological innovations.
Some previous regulations have precipitated technological innovations (e.g. as with
automobile emission controls) that have allowed the achievement of greater emissions
reductions, at lower costs, than were originally expected. At the same time, most standards have
been held the same or tightened due to new information that some of the human health and
environmental effects of air pollution are worse than originally thought. All this means that
assessing the future costs and benefits of the CAAA is like trying to hit a moving target. There is
no remedy for this, but it remains a limitation of the entire assessment exercise that should be
emphasized to policy-makers.
The NAAQS are a complication in forecasting scenarios for the Section 812 Analysis.
Are the emission controls currently in place and those expected to come on line in the future,
under the CAAA, going to be sufficient to meet the NAAQS? If not, then more emissions limits
or control requirements will presumably have to be implemented. These modifications will be
driven (or constrained) by NAAQS attainment schedules and SIP schedules.
The discussion on page 1-3 of the Analytical Plan seems to imply that there will be some
mechanism in the analytical process to periodically assess progress toward meeting the NAAQS
16
-------
under a particular scenario. If the growth in emissions is larger than anticipated, this assessment
could potentially trigger feedback in the form of additional emissions reductions requirements
(with their associated costs and benefits). However, it is not as clear in Chapter 2 of the
Analytical Plan that this feedback will be incorporated.
One of the most important scenarios may be the "additional controls" scenario (i.e. going
beyond current CAAA requirements). This scenario is likely to be more relevant than the
alternative pathways scenarios initially suggested in the current Plan. It is listed as a scenario in
the current Plan, but little detail is provided (Chapter 2). This scenario seems important because
it may stimulate discussion about what the alternatives may be for different emissions source
categories, and may suggest least-cost directions for future policy.
4.8. Treatment of NAAQS Compliance
At the November 5th meeting of the Council, Mr. James Neumann of Industrial
Economics presented new information on the planned treatment of NAAQS compliance in the
construction of the post-1990 control scenarios. The bullets on the relevant slide said:
"The 1997 revisions to the Particulate Matter (PM) and Ozone National Ambient
Air Quality Standards (NAAQS) will not be included in the Post-CAAA scenario
because of the uncertainty associated with the continuing development of
implementation plans. The Agency intends to use the 'beyond-the-CAAA'
federal-level control scenarios to inform development of the implementation plans
for 1997 NAAQS revisions. This approach will help the Agency determine the
air quality shortfalls in individual non-attainment areas to comply with the
NAAQS revisions."
The Council recognizes the computational convenience of the baseline of no-additional -
PM/Ozone NAAQS compliance measures. Presenting intermediate results on this basis can be
seen as part of measures the Agency is taking to increase the transparency of its calculations.
However, the Council is very concerned that this incomplete NAAQS compliance
baseline does not correctly represent the full actual legal requirements of the 1990 CAAA. The
Council urges the Agency to calculate and present its final results for the post-CAAA scenario in
terms of full likely implementation of the post-CAAA requirements. Because the details of what
will be needed for this "full implementation" are not fully defined at present, the Council urges
the Agency to consider a range of plausible implementation scenarios to bracket the likely range
of PM and ozone NAAQS compliance pathways. Utilizing this bracketed range as the baseline,
some effects of the "beyond-the-CAAA" federal level control scenarios may then be seen in part
as displacing the need for some of the higher-cost NAAQS compliance measures and in part as
achieving PM and ozone control beyond that formally required for NAAQS compliance.
17
-------
5. COST ESTIMATES
5.1. Charge Question 7
Does the Council support the plans for estimating, evaluating, and reporting compliance
costs described in chapter 4? If there are particular elements of these plans which the Council
does not support, are there alternative data or methods the Council recommends?
5.2. Summary of Council Response
The Council generally supports the Agency's plans and makes several important
recommendations to improve the Agency's approach.
• Econometric models for abatement costs are limited by their incomplete coverage but
they can sometimes offer insights not available from engineering estimates of
compliance costs, in particular, with respect to the impacts of abatement activity on
total factor productivity. Econometric models are one important source of the
stylized facts about economic relationships that are used to calibrate CGE models.
• Indirect costs on regulated industries should be defined and itemized more clearly.
Direct abatement costs are the focus of the cost analysis in the Analytical Plan, but in
some cases, indirect costs on these same industries or in other markets could be very
important. If the range of possible indirect costs, including productivity effects,
process changes, and spillover effects in other markets are identified for major
regulations, those likely to be most significant can be measured.
• Comparison of the predicted and actual costs of air quality regulations will be
important to the evolution of the ongoing Section 812 Analyses.
• Assumptions about the effect of learning on abatement costs need to be carefully
thought-out and supported by the literature in this area. A distinction can be made
between learning and technological changes in many cases. Both learning and
technological change effects are likely to be heterogeneous across sectors and
processes. The Agency should employ the best information currently available about
learning effects, limit the use of speculative estimates, and clearly identify additional
research needs in qualifying the approach used in the current analysis. It will be
appropriate to tailor the level of detail to the significance of the sector. For example,
it will be important to evaluate carefully how the Agency plans to handle learning for
the electrical generating unit (EGU) sector and for the mobile source sector.
• The Integrated Planning Model (IPM) appears to be a reasonable choice for modeling
emissions and costs from the EGU sector. However, if policies in certain regions
prevent efficient pricing, or if emissions allowances in some scenarios are not
grandfathered, there will be a need to adjust the results. The Council also advises the
Agency to explain more clearly the way the IPM model handles the changes in prices
in the energy sector and their effects on the demand for electricity.
18
-------
• Future conditions in energy markets may have strong implications for realized
abatement costs. Sensitivity of the benefit-cost results to alternative assumptions
about energy markets may be an important dimension of the 812 Analysis.
• Other concerns with respect to abatement costs include some caveats about
comparisons with the Pollution Abatement and Control Expenditures (PACE) data,
the need for consistency in discounting assumptions, some questions about the use of
ControlNet, the NAAQS and PACE data, and the relative cost of abatement via
market-based instruments versus command and control.
5.3. Econometric models and costs
Econometric models allow the researcher, in principle, to address indirect effects and
behavioral responses to changes in regulations. These models can be used to 1) suggest the
magnitude of additional costs beyond direct pollution abatement expenditures, and 2) provide
parameters and functions for use in CGE models.
The econometric methods section in the Analytical Plan looks at several different cost
studies of specific industries that have tried to isolate the full incremental costs to these
industries from abatement activities. The Agency's current method for estimating industry costs
focuses on the direct cost of abatement equipment required by the regulations. The value of
these econometric studies is that they can suggest the magnitude of the additional costs (or
savings) to firms as a result of the direct abatement expenditures. Hence, they suggest whether
these indirect effects are important enough that the Agency should worry about capturing them in
the 812 Analyses.
One type of indirect cost stems from the impacts of abatement activity on total factor
productivity. Barbera and McConnell (1990) find some evidence of reductions in total factor
productivity in five industries as a result of abatement equipment, but the magnitude of the effect
is relatively small. Gray and Shadbegian (1994) and Joshi, Lave, Shih and McMichael (1997)
also find evidence of effects on total factor productivity. The estimated effects are relatively
large for the steel industry.
The other industry study described in Chapter 4 of the analytical plan is that by
Morgenstern, Pizer and Shih (2001). This study examines the extent to which a dollar of
abatement expenditure can be expected to result in more or less than $1 of expenditure on other
non-environmental factors of production in four polluting industries (i.e. are direct abatement
expenditures strongly complementary with other inputs, such as specialized labor?). They do not
find strong evidence that direct abatement expenditures either over or under-estimate the total
costs associated with controls. If anything, there is some indication that abatement expenditures
may overstate full costs for some industries.
On net, there is mixed evidence about whether estimating abatement costs by just
calculating direct abatement expenditures through engineering cost functions will result in under-
or over-estimates of costs in individual industries. At a minimum, the Council advises the
19
-------
Agency to review the evidence from this literature and make a judgment about whether to do any
adjustment to the forecast of future costs on the basis of the empirical evidence.
The limitations of econometric cost estimation raised on page 4-7 of the Analytical Plan
apply with equal force to engineering estimates of future compliance costs, because similar
assumptions must be made about factor prices, levels of output produced, and so on. These
estimates must be made just as far into the future for engineering cost models as for econometric
models. Thus, it is difficult to argue that the described limitations are a particular disadvantage
for econometric cost forecasting models as opposed to other types of cost forecasting models.
Because these types of assumptions must also be made for CGE modeling, how will these
separate estimates be reconciled? This issue is not well explained in the Analytical Plan.
In areas where new control technology is needed or costs are highly uncertain,
econometric techniques are not a good substitute for uncertainty analysis, since such techniques
rely on observed choices by firms. When no empirical data exist concerning new technologies,
expert judgment may be the only available source for information about likely costs.
5.4. Direct costs versus broader definitions of costs
In the Analytical Plan for the Second Prospective Analysis, the major thrust of the effort
to estimate costs is still to forecast the direct abatement costs associated with the CAAA.
However, the Analytical Plan does make a number of attempts at capturing broader, more
complete estimates of costs. But indirect costs, in the context of the Analytical Plan, are not
presently defined very clearly. Whatever the Agency has in mind when it refers to "indirect
costs," it needs to be spelled out explicitly. It is important to identify what these more-complete
measures of cost include and how different they might be from narrowly defined engineering
cost estimates.
Some of the relevant indirect costs include costs borne within industries, but other costs
stem from productivity effects. Econometric studies can shed some light on how important these
additional costs might be. Other relevant indirect costs stem from process changes. Treatment
of the effect of learning on costs is addressed in detail below.
Other indirect costs stem from price changes and their effects on consumer behavior in
the goods market and in the labor market. Regulations change prices, which can change
behavior. For example, in emissions inspection and maintenance (I/M) programs, significant
emissions-related repair costs appear to be inducing some drivers to sell their vehicles outside of
the I/M area. Both out-of-area vehicle sales and early scrappage as a result of these programs
have both costs and benefits beyond the usual direct effects measured for the program. [See
evidence from the Colorado I/M program, ENVIRON (2003)].
5.5. Validation against realized historical costs
Earlier comments by the Council have emphasized that it is important to try to validate
the assumptions underlying key scenarios in the 812 Analysis. A major refinement in the Second
Prospective Analysis will be to enhance validation of the cost forecasts by comparison with
20
-------
historical data and with the results from models which are alternatives to those used in the
analysis. This task is very important and the Council enthusiastically commends the Agency's
attempts to do more of this. Earlier ex ante cost (and emissions reductions) forecasts should be
compared, where possible, with ex post measurement of these costs in subsequent Prospective
Analyses.
CAAA regulations are in many cases designed to encourage innovations and
technological advancement to reduce emissions at lower costs. Market based regulations are
explicitly designed to do so, but other regulations, such as automobile emission limits, have also
reduced emissions at lower costs. The Council notes that the CAA has reduced emissions at
lower costs than were originally expected. Comparisons with ex post costs are not just a matter
of validating previous forecasts, but are also an indication of the effectiveness of the CAA and a
potentially important part of the story concerning the costs and benefits of the CAA.
Of course, it will be important to assess whether technologies or processes have changed
compared to expectations when the ex ante forecasts were made. Ex post assessments of the
success of prior cost forecasts must be made for the same regulatory program as was assumed in
the ex ante prediction exercise, and the same baseline must be used. The predictive model in
general may perform well if it is run using the right assumptions, even though it predicts less
well if the forecasted determinants of its predictions are less accurate. Predicting the future is
never an easy task.
5.6. Learning
The discussion of learning in the Analytical Plan could be enhanced by a careful
distinction between learning and technological change. There can be a tendency to confound
learning and technological change. Learning can be interpreted as those improvements in
productivity and associated cost reductions that are derived from a firm's growing experience
with a new technology. Overall, the impact of technological change may be hard to separate
from subsequent learning effects, but the impact of technological change arises directly from the
introduction of new technology itself, such as new equipment or new software. Some
technological innovations will require little or no associated learning to show their full effect on
productivity. Others will require considerable learning.
It is not clear whether the Agency proposes to account for measured "learning curves" in
the sense of the observed empirical relationships between declines in unit costs with increases in
cumulative output produced using a given technique or process. (See Argote and Epple, 1990).
Most analyses of learning curves have examined empirical relationships. To the committee's
knowledge, the only effort to frame learning curves in an economic context was by Auerswald et
al. (2000).
The Council is concerned that the Agency is oversimplifying the default 80% rule for
learning effects. The influence of "learning" on compliance costs received much emphasis in the
document, but the 80% rule for all sectors for a doubling of cumulative production is a gross
oversimplification, even though it is an improvement over entirely failing to acknowledge the
effect of the learning process on costs. It is hard to come up with a better suggestion than the
80% rule, but there has been growing experience with compliance costs over the last three
21
-------
decades and it will be important to do the analysis that will allow the rule to be refined. For
example, there is likely to be great variance across sectors in the extent to which "learning" can
be assumed to decrease compliance costs over time.
A comment was made during the Council's deliberations that the RFF HAIKU model
accommodates learning via assumptions about technological change and the Argonne AMIGA
model accommodates learning through adjustments to hurdle rates for new technology adoption.
Neither of these statements were carefully explained or developed. A review and evaluation of
the specific learning assumptions in each framework requires careful specification of exactly
what is being represented in each model.
The Agency should consider the econometrics of doubling outputs and the empirical
evidence about scale economies. The sophistication of these models varies widely across
applications. Some models consider a pure learning effect in the form of technical change, while
others consider differences in the scale of production and changes in the mix of inputs. It is not
even clear that a pure "learning effect" can be empirically isolated.
Peretto and Smith (2001) conducted a 48-study meta-analysis of the effects of learning on
compliance costs. This meta-analysis focused only on energy industries. A PDF file for a recent
final report to the U.S. Department of Energy has been provided to the Agency. In that report,
pp. 20-25 and Tables 2-9 summarize the database and a preliminary analysis that was conducted
for all learning curve studies that the authors could identify, including published and unpublished
research.
As the tables in Peretto and Smith document, a diverse set of industries is covered.
Unfortunately, none of the studies in the meta-analysis adopted a framework that would be
consistent with conventional neoclassical models. While the work of Peretto and Smith remains
at an early stage for a meta-analysis, the tables certainly document a simple inventory of what is
known. The evidence one can glean from these tables is unfortunately at odds with published
literature that claims there is empirical support for the 80% rule.
The preliminary results of the Peretto and Smith meta-analysis can thus be characterized
as "pretty grim." One would like to identify a range of alternative values by sector for learning
effects, but the extant studies vary greatly in terms of their quality. The central tendency of the
magnitude of estimated learning effects suggested by the meta-analysis depends on choices
related to quality control. The distinction between learning via changes in process versus
learning related to "management technique" matters, especially in the service sector.
As research into learning effects matures, uncertainty analysis needs to be incorporated to
insulate the bottom line from any vulnerability to this problem. There will be deviations from
the 80% rule for cost savings. These are likely to differ not just across industries or sectors, but
across processes (for example, taking nitrogen oxides (NOx) out of coal and gas combustion).
These cost savings may be an important issue, but capturing them may require corrections all the
way down to the process level, not just to the industry level.
22
-------
The "learning rule" for costs will be refined and tailored to different contexts with the
emergence of additional credible research. Until then, the Agency cannot afford to pursue the
same level of detail everywhere, since identifying process- and sector-specific estimates will be
very labor-intensive. It would seem most appropriate to tailor the level of detail to the
significance of the sector. For example, it will be important to evaluate carefully how the
Agency plans to handle learning for the EGU sector. The Agency should employ the best
information currently available about learning effects, limit the use of speculative estimates, and
clearly identify additional research needs in qualifying the approach used in the current analysis.
Appendix C contains additional detail on costs and learning.
5.7. IPM versus HAIKU models for cost estimates
The Draft Analytical Plan states that the IPM will be used for utility cost estimates. The
IPM model is national in scope, but involves 26 modeling regions for the United States power
market. In many of these regions there is, and will continue to be, fairly stringent economic
regulation of the utility sector. Any model that assumes efficient markets may not adequately
capture what is going on. Thus, a capability to do some analysis of EGU environmental
regulation at the regional level will continue to be important. However, while regional impacts
are certainly policy-relevant, the Council re-affirms its concerns about the general equilibrium
consequences of regulation and the difficulty of distinguishing regional effects because of cost
spillovers via product, labor, and capital markets.
Some researchers who work with utility sector models emphasize the need for any such
model to have a well-developed demand side. When prices go up, there must be some feedback
effect upon demand. If demand is exogenous and serves as an input to the model, it is not clear
how changes in electricity prices or alternative scenarios about the costs and prices are built into
the model. An understanding of this is also important to determine whether this part of the
assessment is consistent with assumptions made throughout the analysis about energy prices and
elasticities. Sufficient information to allow a comparison of IPM with HAIKU or other models
would also be helpful to the Council in developing an understanding the advantages and
disadvantages of the IPM model relative to other alternatives.
For future analyses, the Agency should consider a more detailed comparison of the IPM
model with other utility-sector models in terms of methods, assumptions and results. This would
provide important information about the advantages and disadvantages of the IPM model and
would aid in understanding whether its results are consistent with other assumptions made
throughout the analysis.
The IPM model does appear to take account of utility purchase and sale of emission
allowances. The initial allocation of those allowances can be very important for the outcome in
terms of the final allocation of control responsibility and the resulting costs of control, especially
if allowance markets are thin or if unequal market power rests in the hands of some traders. The
IPM model assumes that allowances are to be grandfathered based on allocations allowed by the
CAAA. It would be helpful to know whether the model might allow for alternative assumptions
in order to examine the importance of this assumption.
23
-------
5.8. Uncertain future energy demand conditions
Relative prices of natural gas and assumptions about their future trajectories will be very
important to the forecasting of future costs of the CAAA. The Analytical Plan is not clear about
how assumptions about natural gas prices will be made and supported. These assumptions have
direct implications for the calculated costs of the CAAA. If the price of natural gas, a cleaner
fuel, is much higher than initial estimates, then more of other dirtier fuels will be substituted and
more air quality controls will be needed. Future natural gas prices are a major source of
uncertainty in cost forecasts. Sensitivity analysis with respect to different assumptions about
these prices will likely be an important part of the uncertainty section of the Second Prospective
Analysis.
It will also be important for the Agency to be clear about how demand is determined for
the electricity produced by EGUs, and how these demands are regionalized in the models used
for cost estimation. Will energy demand models be integrated with the CGE model? In general,
fuel prices, energy demand conditions, the competitiveness of different regional (energy)
markets, and technical progress assumptions are key ingredients in the forecasting of costs for
the utility sector.
5.9. Competing risks due to higher energy prices
The Council's report must acknowledge that one Council Special Panel member has
drawn attention to the suggestion that the Agency's benefit-cost analysis should not ignore the
impact upon health, including both mortality and morbidity for adults and children, from
increased energy costs due to air quality regulations (specifically, higher electricity prices). The
low-income elderly appear to be especially vulnerable to higher energy costs. This subgroup
also appears to be at high health risk for PM exposure. There was a question as to whether it is
relevant to compare the direct health risk to the elderly from PM with the indirect health risks
stemming from higher energy prices operating through, for example, lesser ability to pay for air
conditioning during heat waves or adequate heating during severely cold weather.
It could also be argued that the Agency should consider the health impact of increased
prices from air pollution emission controls in other sectors of the economy, such as
transportation. There are tradeoffs between fuel economy (and its air quality effects) and vehicle
weight (and its safety implications) that may be equally important in determining competing
risks to be considered in formulating air quality regulations. These tradeoffs are considered in
the literature on "risk-risk analysis." Other considerations are related to the "richer is safer"
literature (also called "health-health analysis," where risks are mediated through changes in
disposable incomes). There is also a literature that tries to quantify how regulatory (or other)
costs can simultaneously reduce health for some populations, in addition to improving it for
others, in ways that might not be fully anticipated. For example, regulation may also reduce
vehicle miles traveled and thereby reduce the risk of highway accident deaths.
The "health-health" approach is useful in policy comparison settings where one looks
only at the beneficial health effects of an intervention and ignores the costs. The Council notes
that this approach is not as useful, however, in the context of the 812 Analyses, where both
health effects and costs are explicitly considered. Such a benefits-only approach would be a new
24
-------
strategy. Since benefit-cost analysis accounts for the costs directly, there is a risk of double
counting when the analysis includes both costs and foregone benefits. By foregone benefits is
meant the specific goods, such as better health that people give up when they incur regulatory
costs, through the richer-is-safer pathway. If the adverse health consequences of higher prices
are to be considered for inclusion in the 812 Analysis, there will need to be a careful justification
for why these costs are not captured directly by the decreases in incomes that are already likely
to be part of the explicit costs. This can happen, in principle, when there are externalities
involved, but the literature on the existence of such externalities is insufficiently developed.
There is also a risk when undertaking a piecemeal accounting of selected general equilibrium
effects without considering others. Some secondary effects will be harmful to health, but others
will be beneficial. If it is appropriate to address some secondary effects, it is appropriate to
consider all of them.
A further difficulty in the richer-is-safer literature is that the empirical estimates are
difficult because of the problem of sorting out causality. Income and health are likely to be
jointly endogenous. Higher income is likely to promote health, but health may also promote
income, and additional factors may contribute to both. The most useful papers in the richer-is-
safer literature probably include Chapman and Hariharan (1994, 1996), Keeney (1990, 1997),
Lindahl (2002), Lutter, Morrall, and Viscusi (1999), Ruhm (2000, 2003), Smith (1999), and
Viscusi (1994).
5.10. Miscellaneous
Problems with Pollution Abatement Cost and Expenditures (PACE) Survey data
comparisons must be acknowledged. Some of the problems with the PACE data on costs of air
pollution control for utilities (identified on page 4-5 of the Analytical Plan) will also afflict direct
engineering cost estimates. Neither approach to the calculation of control costs includes process
changes or integration of abatement with other firm activities, nor do they include insurance
costs. It is important to determine how previous cost forecasts might not be expected to match
realized reported PACE costs. Has the Agency determined whether there are any other unique or
specialized opportunities to examine data on actual costs or expenditures on air pollution control
by electric utilities besides the PACE data? If so, it will be important to take advantage of any
reasonable opportunity to validate cost assumptions.
Consistency in interest rate assumptions is another consideration. Throughout the 812
Analysis, there is a need to enforce consistency in key assumptions. For example, is the interest
rate being used to annualize costs consistent across sectors and models, and consistent with the
discount rates being used to compare benefits across different time periods? A 5% interest rate is
used in the cost analysis. The plan is to convert fixed capital costs to a real capital cost and then
to annualize using this interest rate. If 5% is used here, it should also be used elsewhere in the
analysis when the same types of time tradeoffs are at stake. The Council revisits the
discount/interest issue in more detail in the sections devoted to charge questions about
Discounting and about Results Aggregation and Reporting.
In general, there needs to be more explanation of how ControlNet will be used to develop
costs of alternative scenarios. Under certain of the scenarios that will be developed (for
example, in the "alternative pathways" proposed in the initial version of the Analytical Plan or
25
-------
some revision to those), sectors will require either more or fewer controls depending on the
assumptions of the scenario. How are these reallocations of abatement responsibility to be
implemented with the ControlNet model? There are many options for control. How is it decided
which controls will be used? Even under command and control regulations, there can be various
possible ways of achieving goals. How will forecasts be generated concerning how firms will
choose between different compliance strategies?
The model used to evaluate some of the scenarios will need to allow for the impacts of
changing factor prices. Does ControlNet allow for changes in factor prices? Page 4-6 of the
Analytical Plan says it does, but the document is not clear about how. Is it necessary to make
specific assumptions about a variety of elasticities, for example? Does ControlNet allow process
changes to be built into cost scenarios for alternative pathways (top of page 4-11)? How?
Market Based Incentives (MBI) may be lower-cost solutions than command and control.
In an interesting paper on costs of pollution control, Harrington, Morgenstern and Nelson (2000)
found that MBI as pollution control policies have tended to have both lower costs and greater
emissions reductions than predicted. This implies that regulations that allow market based
solutions should be treated differently in terms of cost estimates. Is this being accounted for in
the analysis?
26
-------
6. COMPUTABLE GENERAL EQUILIBRIUM MODELING
6.1. Charge Question 8
EPA seeks advice from the Council concerning the choice of Computable General
Equilibrium (CGE) model which EPA intends to use as a post-processor to gauge the general
equilibrium effects of the various control scenarios. In the first 812 Analysis -the retrospective-
EPA used the Jorgenson/Wilcoxen model to gauge the general equilibrium effects of returning to
the economy the reported compliance expenditures which formed the basis of the retrospective
study direct cost estimates. This model has since been refined in many ways, and EPA considers
both the Jorgenson/Wilcoxen/Ho and AMIGA to be acceptable tools. Although a final decision
on model choice can be deferred until later in the analysis, EPA has tentative plans to use the
AMIGA model because of its greater sectoral disaggregation, better industrial sector matching
with CAA-affected industries, richer representation of relevant production and consumption
technologies, and better model validation opportunities due to its use of open code. However,
AMIGA is limited given its inability to deal with dynamics over time. Does the Council support
the current, tentative plan to use the AMIGA model for this purpose? If not, are there alternative
model choices or selection criteria the Council recommends?
6.2. Summary of Council Response
• The choice of a CGE model should be moved up in the analytical sequence, since
CGE models can illuminate the likely emissions consequences of regulations as well
as identify indirect costs or spillovers.
• Incorporation of spillover costs of air quality regulations is important and these costs
should continue to receive close attention.
• CGE models have the capability to reveal spillovers of air quality regulations into
unregulated sectors, not just to better estimate the direct costs of regulation on
regulated sectors. The current Analytical Plan describes CGE methods for "post-
processing," using estimates of direct cost estimates as inputs. However, this tends to
leave an impression of relegating them to a secondary status. General equilibrium
modeling should enjoy similar status to direct cost calculations and should not be
subordinate to them.
• Each of the main CGE models which are proposed for use in the 812 Analysis has
some limitations. The Jorgenson-Ho-Wilcoxen (JHW) model has a longer track
record and has been more extensively reviewed. The extent of substitutability in the
AMIGA model represents a cause for concern to the Council.
• The AMIGA model needs to be revisited by the Council after the Agency can provide
a fuller characterization of its assumptions and can compare and contrast its elements
with other available models, including the new EMPAX CGE model currently under
development. The issue of substitution is especially important. The current
description, which seems to limit substitution to own-price elasticities, is inadequate.
27
-------
The Council needs a specific detailed comparison of the structural elements in the
AMIGA model versus the EMPAX model versus the more-established JHW CGE
model.
• The Council advocates a serious effort to accommodate the consequences of possible
tax interactions in the 812 Analysis. Considerable sensitivity analysis is indicated,
however, since simple formulas for the magnitudes of tax interactions for regulations
imposed on particular sectors have not yet been identified.
• CGE models and econometric models for costs are not competing methods, but
complementary methods. Econometric results, where available and appropriate, are
generally more desirable than expert judgment for calibrating the parameters of CGE
models. However, where no econometric estimates exist for key parameters, expert
judgment is essential.
6.3. Costs outside the regulated market
Theory and empirical work suggest that some of the most important costs of
environmental regulations are manifested outside of the regulated market. The structure of
substitution implied by the specification of production and preference functions as well as the
characterization of intermediate goods in these models will affect how important the model
implies these effects will be. In some circumstances these secondary impacts may be of greater
magnitude than the impacts in the targeted sector or industry. Thus it seems important for the
Agency to consider these impacts in its assessment. The Council commends the Agency for its
commitment to addressing these impacts.
6.4. Just ex post cost spillovers? Or emissions projections too?
It is not clear in the Analytical Plan how the engineering cost estimates will be linked to
CGE models. As a rule, the engineering studies used to estimate compliance costs distinguish: a)
fixed or investment-related costs required for new equipment (or retrofitting of specific add-on
technologies) to be added to existing plant and equipment; and b) increased operating costs.
CGE models usually characterize production activities with a composite of neoclassical
production (or cost) functions and input requirement functions (or input-output materials). These
are often defined at levels of aggregation that do not match the detail used to develop the
engineering cost estimates. As a result, some linkage must be developed. This implies
adjustments to input measures, input price measures, parameters or technical coefficients. The
relationship between CGE cost measures and engineering-based compliance cost measures will
be affected by the nature of the assumptions made in these types of reconciliations.
The Analytical Plan needs to be clear about whether: a) CGE modeling will be done after
the main direct-cost analysis, as an additional step with the sole objective of producing more-
comprehensive estimates of overall costs by capturing cost spillovers into other sectors or b)
CGE models will also be used early in the analysis to help clarify emissions projections by
recognizing possible interactions among regulated industries and outside these industries.
28
-------
The existing text of the Analytical Plan suggests that the CGE modeling would serve
largely as a check on the direct cost estimates from the engineering and sector studies. This
suggests that the CGE analysis largely covers the same impacts as the other models, and it
implies a subordinate role for the CGE modeling. This characterization does not to convey the
main purpose or significance of the CGE modeling enterprise.
While CGE models can indeed give information on the direct costs, they are especially
important in capturing indirect cost-impacts that cannot be considered by the other analyses. For
such impacts, there seems to be no substitute for CGE models. Thus, the discussion of the
purpose of CGE analysis should be modified.
CGE models can gauge how regulations indirectly affect demand and supply conditions
in related sectors. These changes can influence emissions levels as well as economic costs.
These general equilibrium impacts on emissions can be important. The Analytical Plan
emphasizes the use of CGE models on the cost side, but the impact on emissions is potentially
important as well. These indirect impacts on emissions should be explored.
6.5. Competing CGE models
The Jorgenson-Ho-Wilcoxen (JHW) model has many antecedents in the literature, has
continually improved over the years, and has a long history of peer review. While it is not
perfect, it does capture a number of processes that are crucial to our understanding of the
responses of the economy to air quality regulation. The most important virtues of the JHW
model are:
a. attention to margins of substitution among factors, inputs, and goods that seem
most important a priori,
b. a serious empirical (econometric) basis for most of its parameters,
c. careful modeling of saving behavior, capital demands and technological change,
d. significant degree of sectoral disaggregation, and
e. incorporation of pre-existing distortionary taxes. (The significance of this last
feature is discussed below.)
Like all models, however, this model also has some acknowledged limitations. These include:
a. an overly optimistic specification of the sectoral mobility of capital (it is assumed
to be perfectly mobile),
b. excessively elastic savings behavior, and
c. the absence of explicit modeling of natural resource stocks (such as the
exhaustibility of domestic petroleum stocks) and associated extraction-cost
implications.
However, for the purpose of gauging the general equilibrium cost impacts, this model is, overall,
probably a good choice. One attractive feature of the JHW model is that it has been extensively
peer-reviewed and is "about as good as it gets" among the class of thoroughly vetted models.
29
-------
It will be important to explain further the choice of CGE model, even if it is to be used
only for "post-processing" tasks. It is the Council's opinion that the criteria for choice of a CGE
model should consider all of the features just listed for the JHW model, and possibly more. As
CGE development continues, researchers will become more aware of the implications of other
simplifying assumptions incorporated in existing models.
However, beyond just the JHW model, the Analytical Plan also refers to a newer
contender, the AMIGA model, as a possible vehicle for CGE analysis, and the Agency is now
also apparently considering the EMPAX model. The EMPAX model is still under development
but may be available and vetted soon enough to consider for the Second Prospective Analysis.
As of the present point in this review process, few members of the Council are sufficiently
familiar with the details of the AMIGA model and have no specific information about the
proposed structure for the EMPAX model.
It is important for Agency staff to provide briefing materials so that the Council is able to
review these models carefully during the evaluation process before making any suggestions
about their relative suitability. The Agency has provided some limited review materials.
However, the Council wishes to make it clear that it is the Agency's responsibility, not the
Council's, to inventory the properties of each competing model and make arguments for why one
might be preferred over the others. For subsequent phases of the review process, the Agency
may have time to build such an analysis, which would serve to justify the Agency's planned
selection to a broader audience than just the Council.
In contrast to the JHW model, the AMIGA model has no track record in peer-reviewed
journals. It is a "new entrant." There is one paper forthcoming. It will be necessary for the
Agency to examine the model very closely to compensate for the lack of peer review, and/or to
wait until some external independent peer review has taken place. It will be important to assess
the relationship between current conditions and the prediction of the AMIGA or EMPAX models
based on earlier conditions, to see how well these alternative models can predict realized
historical outcomes. This needs to be done to reinforce our confidence in how well the
alternative CGE models might perform in predicting future developments.
On pages 4-23, the document describes a number of what are described as "minor
concerns" about the AMIGA model. The last is described as follows: ".. .for consumption of
goods other than transportation and housing-related services, the model's implicit assumption of
zero substitutability may not be supported empirically" (emphasis added). The Analytical Plan
does not contain sufficient information about the AMIGA model for the reader to understand this
comment. If it implies that the AMIGA model assumes that all commodities except housing and
transportation are consumed in fixed proportions, then this is a very restrictive assumption.
During the October 23, 2003 teleconference of the Council Special Panel, the Council
was provided with additional information about AMIGA indicating that the model does feature
substitutability in that it embodies price elasticities for all goods and services relevant to
households, and there is labor, capital and energy substitutability among producers. However,
despite the presence of own-price elasticities in these models, the Council remains concerned
about how the model's specification constrains the extent of cross-price elasticities.
30
-------
The "deadweight losses" due to taxation occur because these taxes drive a wedge
between buyer's gross prices and the seller's net prices of a variety of goods. If demands for
some goods are unresponsive to the prices of other goods, quantities traded of these goods will
not change when these other goods are taxed and the analysis may not be able to capture these
deadweight losses fully. It may be the case, however, that the description of this aspect of the
model in the Analytical Plan is just prone to misinterpretation.
The Council wishes to emphasize that use of the AMIGA model, if it does indeed
embody limited substitutability assumptions, would be inconsistent with the objective of a CGE
analysis. That objective is to reflect inter-sectoral substitution effects of the costs that arise from
environmental policies. If AMIGA is limited in terms of cross-price elasticities, a choice to use
AMIGA by the Agency would reduce the standing of the CGE analysis in relationship to other
cost analyses.
6.6. Principles for CGE model selection
The Council strongly supports the Agency's plans to coordinate a workshop concerning
the array of CGE models available for Agency use. The insights to be drawn from such a
Workshop will be helpful to the Council's future deliberations as well. In the Council
teleconference of December 22, 2003, the suggestion was put forward that the Council could be
of assistance to the Agency by beginning to formulate an outline of appropriate criteria for CGE
model selection—a "statement of principles." The inventory of included and excluded features
for existing models such as the JHW model (outlined in the last section) might provide a
reasonable starting point. A good CGE model should be characterized, among other things, by:
a. attention to margins of substitution among inputs and among outputs,
b. a serious empirical basis for as many parameters as possible,
c. careful modeling of saving behavior, capital demands and technological change,
including relevant elasticities
d. a significant degree of sectoral disaggregation,
e. incorporation of pre-existing distortionary taxes,
f. reasonable assumptions about the degree of sectoral mobility of capital,
g. explicit modeling of the status of natural resource stocks and associated
extraction-cost implications.
In discussions with Agency staff in the March 18, 2004 teleconference, the Agency asked for
specific guidance with respect to major options in selecting and using CGE models in current
and future Prospective Analyses. The options being considered by the Agency include:
Option 1. A single post-processing CGE run that reveals some of the first-order general
equilibrium considerations on the cost and benefits sides of the exercise, but with no up-
front integration and reconciliation of CGE and sectoral models, except to ensure
consistent input assumptions such as population growth;
Option 2. Moving the CGE effort up to the front of the analysis, but only integrating it
with cost-side considerations, in order to capture indirect costs to the extent possible;
31
-------
Option 3. Like Option 2, but also including some exogenous estimates of how the
CAAA affects labor productivity and availability in order to capture some of the indirect
benefits of air quality regulations
Option 4. Run the entire analytical sequence with feedback into the CGE model. This
approach would involve a full run of the analytic sequence as described in the revised
analytical plan (to obtain a first approximation of the scenario-specific values for both
cost and benefit effects for the with- and without-CAAA scenarios), followed by a second
set of full runs (including supplemental and disaggregated runs) starting with a
reconfigured CGE.
The Council recognizes the currently overwhelming burden that the ideal approach,
summarized as Option 4, would place upon the Agency. Option 1 would be the easiest and
quickest approach, but would provide too little integration, in the view of the Council. Option 2
would represent some progress over this strategy, and Option 3 would be preferred.
The best current strategy for the Agency would be to incorporate experimental
applications using more-integrated CGE models into the "Learning Laboratory" dimension of the
research program in support of future Prospective Analyses. The best short-term implementation
will need to reflect the opportunity costs of resources that could otherwise be used to enhance
other aspects of the main analysis for the current Section 812 Analysis. The best long-term
strategy will be to continue to explore means whereby advancing computing technologies and
increasingly sophisticated CGE models can be exploited to allow greater and greater integration
of CGE calculations into the main benefit-cost assessment.
6.7. The tax-interaction effect
Two years ago, in its preliminary review of the Draft Analytical Plan, the Council was
disappointed about the Agency's treatment of the tax interaction effect. The literature indicates
that the tax interaction effect is not just a second-order effect, but a first-order effect and
therefore needs greater status in the analysis. The Council endorses the Agency's commitment
to attend to this effect in the Second Prospective Analysis, because the effect is important and
stems from the impact of environmental regulations on relative prices. In particular, to the extent
that regulations raise costs and lead to higher output prices, they raise the prices of goods in
general. This effectively lowers the real returns to factors of production (e.g., the real wage). To
the extent that pre-existing taxes have already reduced factor supplies below the efficient level,
the further reduction in factor returns stemming from higher goods prices produces a first-order
efficiency loss. This is the tax-interaction effect. In several studies, this effect involves a greater
cost than the direct cost or compliance cost in the regulated market [see: Bovenberg and Goulder
(1997); Fullerton and Metcalf (2002); Goulder et al. (1997); Goulder and Robertson (2003);
Parry (1995); Parry et al. (1999) and Schob (1997)].
The Council notes, however, that the Revised Analytical Plan's characterization of the
tax-interaction effect has some problems. The Plan correctly points out that there is uncertainty
surrounding the magnitude and sign of the tax-interaction effect. However, it incorrectly
concludes from this that the central case estimates should assume that this effect is zero. It is
more appropriate to use a best estimate of the mean of the tax-interaction effect.
32
-------
Both theoretical and empirical studies consistently indicate that, in realistic settings, the
tax-interaction effect involves a positive cost. Moreover, for environmental regulations that do
not raise revenue - for example, performance standards, technology mandates, or freely allocated
emissions permits - there is no "revenue-recycling effect" to offset the tax-interaction effect.
For these regulations, if the required emissions reduction is a small percent of baseline
emissions, the tax-interaction effect can be several times larger than the direct costs.
The tax-interaction effect will be smaller to the extent that the regulated commodity is an
especially strong complement to leisure. However, even in this case this effect will generally
imply an extra cost rather than a reduction in cost. The regulated commodity would have to be
an extremely strong leisure complement to switch the sign of the tax-interaction effect.
The Committee endorses a balanced approach to CGE modeling, so that indirect benefits
as well as indirect costs are considered. There may also be a benefits-side tax-interaction effect.
The general equilibrium effects of compliance costs are critical and so also may be the general
equilibrium effects of beneficial health changes. Abatement of air pollution by the CAAA is
intended to create positive health effects. It is just as important that the analysis include the
general equilibrium consequences of improved health status on labor availability and
productivity and therefore on the costs of labor and the costs of health care. Morbidity certainly
has indirect effects on productivity that need to be recognized. The general health consequences
of changes in the ambient levels of pollutants need to be considered, not just mortality.
The impact of regulations on labor productivity and the associated "benefit-side" tax-
interaction effect is indeed an important issue and has been analyzed specifically by Williams
(2002, 2003). This beneficial effect offsets the adverse tax-interaction effect described in the
previous section. However, Williams's work indicates that, in general, this offset is not likely to
be large enough to entirely undo the adverse tax-interaction effect. Thus it seems appropriate to
assume in the central case that the tax-interaction effect does raise costs.
On page 4-26, the Analytical Plan suggests that: "Improvements in CGE models that the
Agency is considering for this analysis have made it possible to account for tax interaction
effects more precisely." The Council assumes that this comment pertains only to indirect effects
on the cost side of the analysis, not the benefits. Part of the tax interaction effect can be
addressed in CGE models, but no existing CGE model will capture all of it. At a minimum, the
Williams (2002, 2003) adjustments for the productivity-enhancing consequences of health
improvements due to environmental regulations need to be considered.
There are in fact a number of citations concerning the health benefits of emissions
controls for labor productivity and their spillovers into less-regulated sectors. The Council is
aware of several papers on this topic. Some of these papers (e.g., Espinosa and Smith, 1995)
demonstrate how non-separability between pollutants and private goods, a prerequisite for such
beneficial spillovers, can be incorporated into CGE models.
Two of the already-published papers in this literature are Espinosa and Smith (1995) and
Smith and Espinosa (1996).4 These papers use an updated version of the Harrison-Rutherford-
' The fifth one is a conceptual paper Schwartz and Repetto (2000)
33
-------
Wooton model that includes measures of particulate matter, sulfur dioxides, and nitrogen oxides
as non-separable influences on consumer preferences. The model includes eleven regions and
six goods and three factors in each region. International trade and transboundary pollution are
included. There is a simple air diffusion model between the different countries in Europe. The
model relies on the concentration response functions presented in Desvousges, Johnson, and
Banzhaf (1998) and uses estimates of willingness to pay that are adjusted for each country. A
newer paper that addresses the tax interaction effects, Espinosa and Smith (2000) is under review
for publication.
The tax interaction effect should be an explicit dimension of the presentation of costs.
The precise methods for including tax interaction considerations in the Second Prospective
Analysis are not adequately described in the current Analytical Plan. The Council could be more
confident in its advice on this matter if the Analytical Plan included more specific details on
these issues, including a description of how engineering cost estimates will be linked to the CGE
models for the analysis of tax interaction effects.
It should be noted that the Analytical Plan's suggestion of a 25-35% increase in costs due
to the tax interaction effect in the current document may be a result of miscommunication in, or
misinterpretation of, the earlier Council review of the Draft Analytical Plan. The indirect cost
consequences of the tax interaction effect can differ by orders of magnitude and can be vastly
larger when regulations actually result in little abatement and when there is no revenue recycling.
For the sulfur dioxide emissions covered by Title IV, it may be appropriate to make the
assumption of a 25-30% increase in costs, but such an assumption is unlikely to be universally
appropriate.
The question thus remains as to how large a cost impact the Agency might assume for tax
interactions. The Agency could address this issue two ways. First, it can employ its
commissioned CGE model or models to evaluate the costs of specific regulations. The tax-
interaction effect should be embodied in the aggregate cost impacts obtained from such models.
Second, the Agency should consult results from other, prior CGE studies of particular
regulations. This second step will be useful as a cross-check on the results from the Agency's
commissioned model or models. Moreover, this second step may be necessary to obtain general
equilibrium cost estimates in some instances, since there will surely be some particular
regulations that the commissioned model or models cannot capture.
Given the uncertainties surrounding the magnitude of the tax-interaction effect and of
cost-impacts in general, it is very important that the Agency require considerable sensitivity
analysis in its CGE assessments. Past applications of the JHW model have tended to skimp on
sensitivity analysis.
6.8. Tension between CGE, econometric models
The Analytical Plan rejects econometric methods for developing cost estimates but
accepts CGE models. This sort of top-down approach in the cost calculations, embracing CGE
models, is puzzling. The Council feels that both types of models should be informative. Their
implications should be convergent and a plurality of methods is desirable. However, it is
34
-------
possible that the implications of the different approaches will not be convergent. If this is the
case, then there is a clear need for more basic research to resolve the conflicts.
Are CGE models sufficiently comprehensive? Some members of the Council have
voiced a concern about whether even the largest CGE models are large enough. These are based
on empirical studies of individual industries, but more coverage is certainly needed. There is not
presently enough coverage by empirical studies to permit reliance on econometric models
exclusively. CGE models are calibrated on a selection of empirical results and researchers can
then rely upon plausible assumptions, informed by expert opinion, to fill in for missing
information.
There could, however, be more use of engineering and expert judgment when empirical
results from econometric models are absent. The analysis could proceed based on expert
judgments, using an engineering "bottom-up" strategy. For example, assumptions about the
availability of natural gas will be critical to forecasts. Even the experts do not know enough
about the determinants of availability of natural gas to base the modeling assumptions on
existing empirical results, so the analysis may need to rely more heavily on engineering expert
judgment.
35
-------
7. DISCOUNTING
7.1. Charge Question 9
In the two previous 812 Analyses, the primary cost estimates reflected use of a 5 percent
real discount rate, which an earlier Council endorsed as a reasonable compromise between a 3
percent real rate considered by EPA to be an appropriate estimate of the consumption rate of
interest or rate of social time preference and a 7 percent rate, OMB's estimate of the opportunity
cost of capital. Limited sensitivity testing was also conducted in the previous 812 Analyses by
substituting 3 and 7 percent rates to annualize the benefit and cost streams. EPA's new
Economics Guidelines (peer-reviewed by the SAB EEAC) call for using both a 3 and a 7 percent
rate. A recent draft of new OMB economic guidelines suggests providing results based on both
3 and 7 percent discount rates, while also acknowledging the need for further efforts to refine
analytical policies for discounting methods and rates. EPA plans on following both sets of
Guideline documents by using both 3 and 7 percent in our core analyses. It is true that this will
require presentation of two sets of results - one based on each rate. This may not be necessary
given the expected insensitivity of the overall results to the discount rate assumption. Does the
Council support this approach? If not, are there alternative rates, discounting concepts, methods,
or results presentation approaches the Council recommends?
7.2. Summary of Council Response
• The Prospective Analysis is concerned with arriving at discounted values of the
benefits and costs that may extend into the future for Clean Air Act emissions
reductions in selected years. Such discounting should be performed using a "social
discount rate." The Council commends the Agency's having drawn attention to the
challenges and uncertainties associated with the choice of social discount rate.
• The Council urges the Agency to employ a range of values - perhaps between 3 and 7
percent - for the social discount rate in its assessments. Given the difficulties of
pinning down the "right" social discount rate, it is important to apply these alternative
values and examine the robustness of results to the alternative values. While the
Council supports using a "low" (3 percent) and "high" value (7 percent), it
emphasizes the importance of using a central value as well. This will offer a
"central" case and facilitate interpretation of the Agency's estimates. It is important
to employ a central value in the main analysis. In addition, the sensitivity analysis
should include this central value as well as "low" and "high" values for the social
discount rate.
• The benefit-cost calculations in the Prospective Analysis are social benefits and costs.
To calculate such benefits and costs, the social rate of discount should be applied.
This holds even for calculating the present discounted (social) value of firms'
compliance costs. On the other hand, if one wants to indicate what the costs are, as
perceived by the firm, it is appropriate to employ the firm's own opportunity cost of
capital. This provides information on the cost impact to the firm in question, but does
not represent the overall cost to society. It is important to emphasize that such
36
-------
calculations should not be used to calculate the overall (social) costs or benefits from
the Clean Air Act.
7.3. Theory
The Prospective Analysis is concerned with arriving at discounted values of the benefits
and costs from the Clean Air Act. Such discounting should be performed using a "social
discount rate," which is the rate used to translate future consumption flows into equivalent
current flows. (This is different from a "utility discount rate," which converts future utilities into
equivalent utilities in the present.)
When costs and benefits are not identically distributed over time, the discount rate
assumptions in the analysis will be important. Under these conditions, different discount rates
will yield differences in the relative magnitudes of discounted benefits and discounted costs (as
well as differences in absolute magnitudes). The Council commends the Agency's having drawn
attention to the challenges and uncertainties associated with the choice of social discount rate.
The theoretical literature offers two alternative approaches for determining a social
discount rate. The "demand-side" approach [articulated, for example, by Arrow et al. (1996)],
defines the social discount rate as the sum of a pure social rate of time preference and an
adjustment term reflecting future changes in the marginal utility of consumption (future goods
may be worth less at the margin as people get richer). Even if one assumes a value of zero for
the first term, declining marginal utility of consumption can yield a positive second term and
thus a positive value for this social discount rate.
An alternative approach is the "supply-side" approach, which has been articulated, for
example, by Lind (1982) and Diamond and Mirrlees (1971). This approach defines the social
discount rate as the shadow price of capital, which in turn is the real-world trade-off between
present and future consumption implied by the marginal productivity of capital. This shadow
price is related to market interest rates.
Neither approach dominates the other. Under the demand-side approach, the social
discount rate is inherently a subjective concept: it depends on the value of the pure social rate of
time preference, a parameter that cannot be established empirically. (In contrast, an individual's
pure time preference rate can be gauged empirically.) Under the supply side approach, the social
discount rate has a closer tie to observable phenomena - market interest rates (as representing the
shadow value of capital). An attraction of the supply-side approach is that if the social rate of
discount is equated to the shadow value of capital, then a policy that withstands the benefit-cost
test using that discount rate will offer the potential for a Pareto improvement. Although this
feature has some appeal, it can be argued that the ethically appropriate social discount rate need
not equal the shadow price of capital. Defenders of the demand-side approach argue that
intergenerational equity may call for a social discount rate different from the actual rate of
exchange between current and future consumption implied by the shadow price of capital.
These theoretical considerations imply that, in practice, one cannot pinpoint the "correct"
social discount rate. Neither of the two approaches can identify a social discount rate with
precision. Under the demand-side approach, the rate depends importantly on the social rate of
37
-------
time preference, but analysts offer differing views as to the best value for this parameter.
[Ramsey (1928) argued that it should be zero; Solow (1974) and Arrow et al. (1996) suggest
higher values.] Moreover, one's view of the appropriate value can differ depending on the
context of the choice. The choice context includes the time horizon over which the discounting
is to occur, the sizes of the benefits and costs at stake, and a number of sociodemographic
factors. See also Warner and Fleeter (2001) and Harrison et al. (2002).
Under the supply-side approach, the rate (in principle) is given by the shadow price of
capital, but in practice this shadow price cannot be measured with precision. As discussed by
Lind (1990) and Freeman (1992), the shadow price or social opportunity cost of capital depends
on the extent to which a public project crowds out private investment or private consumption. If
the gross of tax rate of return to an investment is rg and the after-tax or net return to household
savers is rn, then the social opportunity cost will be a weighted average of these two returns, with
the weights reflecting the relative amounts of investment crowding-out and consumption
crowding-out. Pinpointing the social opportunity cost is impossible because the crowding-out
proportions cannot be determined precisely. Moreover, real-world complications in capital
markets imply that the Lind-Freeman formula will not perfectly describe the shadow price of
capital. These complications include restrictions on capital flows, externalities associated with
investments, and the inability to pool risks perfectly. The 7% rate advocated by the Office of
Management and Budget is one plausible estimate of the social discount rate that stems from the
supply-side approach. But estimates of this shadow price vary significantly. Typical estimates
are in the range of 4-10 percent.
7.4. The Social Discount Rate and Firms' Opportunity Costs of Capital
In general, the social discount rate will not coincide with a given firm's opportunity cost
of capital. This is the case even when one applies the supply-side approach and identifies the
social discount rate with the society's shadow price of capital. (Society's shadow price - or the
opportunity cost of investment in terms of future consumption - need not equal a given firm's
opportunity cost of capital. On the other hand, if the firm has access to fluid capital markets, its
opportunity cost might approximate the social opportunity cost of capital.)
Even as the Agency has acknowledged, the strong theoretical basis for relying on social
discount rates and for using a consistent discount rate throughout the analysis, the Agency has
explained to the Council that the linear programming-based IPM model is configured to predict
the private profit-maximizing decisions of firms with respect to capital investments. These
individual firms' behavioral responses will be dictated by their own opportunity cost of capital,
which can differ from the social discount rate. The IPM is designed to predict what firms are
likely to do, rather than what they should do, if they were being managed by a social planner.
There is no need to over-ride firm-specific private discount rates if the purpose of the analysis is
to estimate costs to the firm. However, if costs generated from the model are used to
characterize the overall costs of abatement, then the analysis should use the social rate of
discount.
There is apparently some possibility that it may be feasible to manipulate the structure of
the IPM model to allow an intervention into the capital investment outcomes for firms, arraying
38
-------
these temporally and applying to them the social discount rate. To determine the feasibility of
this approach, further analysis will be needed.
This analysis should address the difficulty in choosing between or in integrating these
two perspectives for measuring the present value of net benefits from regulations and measuring
the annual benefits and costs of regulations. If one adopts the social rate of discount perspective
the capital costs incurred by firms would not be annualized at the private discount rate. Capital
costs would be included in the present value calculation in the year when they were incurred and
discounted along with operating costs in that year using the social rate. Annualizing these capital
costs at the social rate would not reflect firms' private costs of the relevant regulations.
7.5. Importance of Applying a Range of Values for the Social Discount Rate
Thus, assessments of the "right" social discount rate vary both because there are two
alternative approaches and because each approach can yield a range of values. Under these
circumstances it is appropriate and crucial for the Agency to employ a range of values for the
social discount rate in its benefit and cost assessments. The demand-side approach often leads to
values in the range of 1-4 percent. The supply side approach generally leads to somewhat higher
values. Based on these considerations, the Council urges the Agency to employ a range of
values - perhaps between 3 and 7 percent - for the social discount rate in its assessments. Given
the difficulties of pinning down the "right" social discount rate, it is important to apply these
alternative values and examine the robustness of results to the alternative values.
While the Council supports using a "low" (3 percent) and "high" value (7 percent), it
emphasizes the importance of using a central value as well. This will offer a "central" case and
facilitate interpretation of the Agency's estimates. It is important to employ a central value in
the main analysis. In addition, the sensitivity analysis should include this central value as well as
"low" and "high" values for the social discount rate.
The sensitivity of the conclusions to different discount rates and different assumptions
about time profiles needs to be featured prominently. The Council addresses this issue further in
its discussion of the material in Chapter 11 of the Revised Analytical Plan.
39
-------
8. ECOLOGICAL EFFECTS ASSESSMENT AND VALUATION
8.1. Agency Charge Questions Related to Ecological Effects Assessment And Valuation
Charge Question 18. Does the Council support the plans described in chapter 7 for: (a)
qualitative characterization of the ecological effects of Clean Air Act-related air pollutants, (b)
an expanded literature review, and (c) a quantitative, ecosystem-level case study of ecological
service flow benefits? If there are particular elements of these plans which the Council does not
support, are there alternative data or methods the Council recommends?
Charge Question 19. Initial plans described in chapter 7 reflect a preliminary EPA
decision to base the ecological benefits case study on Waquoit Bay in Massachusetts. Does the
Council support these plans? If the Council does not support these specific plans, are there
alternative case study designs the Council recommends?
Charge Question 20. Does the Council support the plan for a feasibility analysis for a
hedonic property study for valuing the effects of nitrogen deposit!on/eutrophication effects in the
Chesapeake Bay region, with the idea that these results might complement the Waquoit Bay
analysis?
8.2. Summary of Council Response
The Council did not include experts in ecological sciences in the development of this
report, because it awaited the formation of its new EES (Ecological Effects Subcommittee) to
help address issues specifically related to assessment of ecological effects linked to
implementation of the CAA. The Council plans to receive a draft report from the EES related to
the ecological assessment components of charge question and then to review and approve such a
report to the Agency as the final installment of the Council's advice on the draft Analytical Plan.
The Council is proceeding to provide advice to the Agency on aspects of the Charge
Questions tractable at this time, with the caveat that future advice will follow. A summary of the
current advice follows.
• Ecological effects to be valued must be limited to those effects for which there is a
defensible, rather than just speculative, link between air emissions and service flows.
The Council strongly objects to using inappropriate or unsupported placeholder
values in the absence of better information.
• The greater heterogeneity in ecosystems services makes it even more difficult to
produce estimates of the benefits from their protection than for the protection of
human health. The input of the Council's new EES may be able to stimulate the
development of greater expertise on this issue than is presently available. The SAB's
new C-VPESS, whose work has just begun, may also provide advice for the Council
to consider, as C-VPESS provides advice to the Agency generally.
• There is a clear need for a better conceptual basis for valuation of ecological effects,
which would also permit the proposed case studies to be integrated as components of
a larger model. Ongoing attention to new literature will be important.
40
-------
8.3. Emphasizing Verifiable Connections
In the First Prospective Analysis, the Agency identified a limited number of ecological
impacts that were amenable to quantitative analysis because there existed a defensible link
between changes in air emissions and a corresponding service flow for which there are peer-
reviewed money values. However, the only monetized benefits, based on displaced treatment
costs, were not reported in the primary central benefit estimates because there are few effects for
which a defensible link exists between changes in air emissions and a corresponding service flow
evaluated in peer-reviewed valuation studies. There has been little increase in the inventory of
available value estimates in the intervening four years since the First Prospective Analysis, so the
Agency proposes to use the same approach for the second Prospective Analysis.
8.4. Valuing Statistical Ecosystems?
The Council's earlier efforts to render greater parallels between the way researchers think
about valuing human health and valuing ecosystem health speculated that it might be possible to
think about "statistical ecosystems" the same way one thinks about "statistical lives" in the sense
that most environmental stressors do not wipe out entire ecosystems with certainty (analogous to
killing individual people with certainty). Instead, they compromise the viability of a wide
variety of ecosystems to some degree, resulting in the collapse of some fraction of these systems,
although the identity of these particular systems cannot be identified ex ante. (This is analogous
to compromising the health of many different people, resulting in the deaths of a few people,
although these individuals cannot be identified ex ante).
However, the Council now recognizes the importance of heterogeneity across human
health risks in arriving at monetary valuation estimates, as well as the likelihood that these
problems can only be more complicated when ecosystems are being considered, rather than
human health. Ecosystems are vastly more heterogeneous than humans. The number of
dimensions across which the willingness to pay function for risk reductions for ecosystems may
vary is likely to be much greater than the number of relevant dimensions for human heath risk
reductions. The Council now has reservations about attempting to push the "statistical
ecosystems" analogy in conceptualizing techniques for determining ecosystem benefits.
Although the language did originate from previous Council deliberations, the Council
encourages the Agency to drop the "value of a statistical ecosystem" term. The term implies that
it is possible to elicit reliably the public's preferences for reducing risks to ecosystems. While
the possibility of obtaining such values for hypothetical risk reductions is an interesting research
question, such an approach may be a distraction from the task of removing the primary
impediments to improved value estimates. As the Agency acknowledges elsewhere, these
impediments include poor understanding of concentration-response functions for ecological
resources and poor understanding of linkages between physical effects and service flows. In
addition, it has proven challenging to describe changes in ecological service flows in terms that
are meaningful to the public. Finally, research on valuing health risks, which are far more
tangible to most survey respondents, has encountered difficulties in eliciting reliable estimates
for small changes in relatively small baseline risks.
41
-------
8.5. Using Available Quantitative Information
The Agency's plans to qualitatively characterize the ecological effects of the Clean Air
Act-related air pollutants is thorough and appropriately focused on a broad characterization of
ecosystem services. However, more could be done to make use of quantitative information that
is available. Although it must be acknowledged that neither the available data nor the available
analytical tools are sufficiently developed to provide a comprehensive quantitative assessment of
the ecological benefits of the CAAA, there is some quantitative information available for some
components of such an assessment that can help to characterize the nature of the progress
expected as a result of the CAAA. The Agency included this type of information in the first
Prospective Analysis. The Agency should continue to do so and perhaps increase its prominence
in the report. This information includes:
a. Air quality models can provide quantitative estimates of expected reductions in
acid deposition (sulfate and nitrate), nitrogen deposition, and ambient ozone
concentrations, which are the primary air pollutants of concern for ecological
effects. Some emissions and/or deposition data may also be available for
important hazardous air pollutants (HAPs), such as mercury. This information
can be presented spatially on maps to illustrate the scope of the improvements that
can be expected.
b. Even though quantitative dose-response estimation may not be feasible at this
time, some quantitative measures of effects of air pollution on ecosystems are
available. These include:
1. the extent of acidification in lakes and streams and the implications for
reductions in some aquatic species,
2. the locations and sizes of estuaries with degraded quality because of
eutrophication and other effects of excess nitrogen and the implications
such as lost habitat for spawning, and
3. locations where forests show evidence of pollution-related stress, etc., and
implications for forest health and diversity.
The analysis should provide some nation-wide characterization of the actual extent of
identified ecological effects along with a description of their implications. It should also provide
information about the expected reductions in pollutant exposures associated with these effects
that may be attained due to the CAAA. These two classes of information will help provide some
context for the more detailed case study proposed for examining the benefits of reducing excess
nitrogen in one estuary. They will also begin to support a link between the current conceptual
discussion of ecosystem services and the likely quantitative social benefits of the CAAA. This
framework will also place in some context the few specific benefits that have already been
approximately quantified, such as improved recreational fishing in the Adirondacks and
increased yields for commercial forests.
8.6. Integration between Conceptual Basis and Case Studies
The Analytical Plan would benefit from a better connection between the discussion of a
conceptual basis for valuing ecosystem services and the proposed case studies described in the
42
-------
document. In general, there should be a more serious attempt to connect the developments in
literature on ecosystems and the strategies being developed by the Agency. For example, the
Agency should begin to pursue some of the ideas contained in Sanchirico and Wilen (2001),
Finnoff and Tschirhart (2003), and Smith (2003).
8.7. Inadvisability of Using Placeholder Values
The revised Analytical Plan acknowledges the disagreements among Council members
reviewing the initial Analytical Plan for the Second Prospective Analysis. The main point here is
that regardless of the validity of the Costanza et al. (1988) estimate of the total value of the
world's ecosystems (which was advocated by a minority of Council members as a starting point
for a placeholder value for ecosystem benefits), a total value for an ecosystem does not
communicate useful information about the value of avoiding different types of incremental
quality-degrading effects of air pollution at levels relevant to the CAAA.
The Council is sympathetic to the concerns that leaving the ecological benefits
incompletely quantified may leave the perhaps erroneous impression that they are unimportant.
However, the Council deems it prudent for the Agency to reject using a placeholder value
because it introduces purely speculative values that provide little guidance for resolving
persistent uncertainties. Furthermore, the use of speculative values could undermine the
credibility of the analysis as a whole.
8.8. Awaiting Insights from EES and the SAB's C-VPESS
While the Council would like to be able to offer some clear resolution on the issue of
ecosystem valuation, the state of the science in this area is at present insufficiently developed to
allow anyone to be conclusive. The Council expects that its new EES will provide needed
scientific advice in the future on how to characterize and quantify ecological effects of
implementation of the CAAA. The Council expects to receive a draft report containing advice
related to the ecological assessment components of Charge Questions 18 through 20 from the
EES in the future and to complete the Council response to those charge questions at that time.
In addition, the Council notes that the separate SAB's C-VPESS has been charged with
providing advice to the Agency generally on how to improve knowledge, methodologies,
practice, and research. The results of its work, just begun, should prove useful to inform future
812 Analyses and will be of interest to the Council.
8.9. Agency Plans for Conducting an Ecological Benefits Case Study
Based on the information provided to the Council and the current perspective of Council
Special Panel members (who did not include ecological science among their expertise set) the
Council believes that if the case studies involve relatively modest opportunity costs, they will
provide some data of interest to the Section 812 process, but the findings will by no means be
generalizable. Advice of the new EES will be valuable on this issue.
Pursuant to prior Council advice, the Agency proposes to conduct a prototype case study
of a specific site. The Agency has solicited the Council's views on selection of one of two
43
-------
possible sites: Waquoit Bay in Massachusetts and the Chesapeake Bay. The Agency suggests
several criteria for selecting an appropriate site. It is not clear how the Agency may have
weighted these criteria in comparing the relative advantages of the two sites. The following table
suggests some possible qualitative evaluations based on the Agency's site descriptions.
Comparison of Qualitative Site Evaluation Ratings
Criterion
1 . Well-documented impacts to a particular
ecosystem function or service
2. a. Quantifiable ecological endpoints
2.b. Quantifiable economic endpoints
3 . Available monetary values for at least some
endpoints
4. Take advantage of existing EPA initiatives
to maximize use of available resources, avoid
redundant research, and demonstrate multiple
applications of ongoing projects
Waquoit Bay
Good
Very Good
Good
Good
Good
Chesapeake Bay
Fair
Good
Very Good
Good
Very Good
Chesapeake Bay is weakest in the area of criterion 1—documented impacts to functions or
services. Chesapeake Bay is a very large and complicated ecosystem that is challenging to
model. In contrast, Waquoit Bay is a small, almost laboratory-sized system. However, the size
and complexity of the Chesapeake Bay provides opportunities for quantifying more endpoints,
including potential impacts on commercially important species and property values.
Oddly, the Agency mentions only in passing that Chesapeake Bay is more representative
of the estuaries affected by air pollution emissions and that Waquoit Bay provides little
opportunity for potential benefits transfers. Nevertheless, the Agency indicates its intention to
use Waquoit Bay for the primary case study because there are available dose-response models
for ecological indicators. Chesapeake Bay will be used only for a property value study. If the
Agency's primary goal is to demonstrate "current deficiencies in our knowledge about both the
physical effects of air quality on ecological services and the value to society of these effects,"
then the atypical availability of dose-response models for Waquoit Bay may argue against that
choice. Chesapeake Bay appears to provide a far richer opportunity to conduct a prototype study
in a realistic setting.
The discussion of the economic valuation component of the Waquoit Bay study is
inadequate. It does not use the "direct use," "indirect use," and "non-use" approach the Agency
has used elsewhere. There should be a more detailed articulation of how the ecosystem services
in question are connected to valuation methods, as well as a discussion of what is being left out.
In general, there seems to be no strong sentiment among Council Special Panel members
to recommend modifying the Agency's proposed strategy. There is some concern that the
proposed case studies seem like a fairly weak response to a very serious data problem. For
example, it might be difficult to detect the relatively small incremental effects of air pollution on
44
-------
water quality on property values in the Chesapeake Bay region. Some members were mildly
supportive of taking advantage of the relatively abundant data concerning Waquoit Bay, even if
this particular resource is not particularly representative.
The Council plans to work with the newly formed EES in developing further advice
related to this charge question.
8.10. Plans for a Hedonic Property Study
The Agency should begin to develop an infrastructure for combining different sources of
information about demand for ecosystem services. The emerging literature on preference
calibration holds promise for integrating hedonic property value estimates with travel cost
demand estimates and other related evidence about demand for these types of non-market goods
as a function of environmental quality.
In the proposed Chesapeake Bay property value application, the same specification of
ecosystem services and their explicit connection to what can be "valued" with hedonic property
value needs to be described. The Council asks how this analysis relates to recreational fishing
considerations and points out that the Agency has not noted the overlap discussed by McConnell
(1990) and Parsons (1991).
This would seem to be an opportunity for a preference calibration exercise (Smith et al.,
2002) combining the Leggett and Bockstael (2000) hedonic study with the extensive travel cost
recreational demand work.
As with the Waquoit Bay application, the discussion is too vague to offer specific
guidance. There needs to be a detailed description of services, approaches used for valuation and
discussion of how the phenomena that can be measured relate to the ecosystem services provided
by this resource.
45
-------
9. ECONOMIC VALUATION - PLANS
9.1. Charge Question 21
Does the Council support the plans described in chapter 8 for economic valuation of
changes in outcomes between the scenarios? If there are particular elements of these plans which
the Council does not support, are there alternative data or methods the Council recommends?
9.2. Summary of Council Response
• There are a number of additional resources that the Agency can consider in
developing estimates of a variety of non-mortality benefits of the CAAA.
• Charge questions 22-25 deal specifically with plans for evaluating health outcomes,
which are the most important of the endpoints listed in Chapter 8. This generic
charge question apparently relates primarily to non-health, distributional and
ecological effects.
9.3. Distributional Effects
The Agency's plans for identifying distributional impacts are somewhat cryptic. The
Analytical Plan simply states that the Agency will assess distributional consequences across age,
income, and racial sub-populations using Census county-level data for the year 2000. In light of
the Agency's (and earlier Council) concerns about their ability to disaggregate costs and benefits
geographically, it seems odd they are not concerned about disaggregating even further by sub-
population. It is indeed possible to measure benefits to different sociodemographic groups in
physical terms and report unmonetized benefits by beneficiary group. However, while some
valuation models report the effect of income, there is very little known about age-specific and
race-specific preferences for environmental services.
9.4. Worker Productivity
The Agency plans to follow the same approach to worker productivity as they did in the
first assessment. They will use the study by Crocker and Horst (1981) on the effect of ozone
concentrations on worker productivity. As it does for other endpoints involving productivity
losses and the value of time, the Agency will use mean or median wage rate. However, the
relevant outcomes are impacts on marginal product and the marginal value of time in a given
activity. Average wage rates are, at best, crude proxies for the average marginal product.
Averages may either overstate or understate marginal values.
Here and elsewhere, the Agency treats the value of time far too simplistically.
Economists have studied market and nonmarket time values extensively over the last 25 years in
areas such as labor, transportation, and recreation economics. The Agency should evaluate
empirical alternatives to using market wage rates to value time. Where the Agency is
46
-------
constrained to use wage rates for pragmatic reasons, they should evaluate the likely direction of
bias and incorporate that assessment in the uncertainty analysis.
For specialized references on the Value of Time, see Appendix C, which contains a
bibliography.
9.5. Miscellaneous Welfare Effects (Visibility and Soiling/Materials Damage)
Visibility. There are some published visibility valuation studies available. Some
evaluation of the visibility benefits for eastern and western parks based on the meta-analysis in
Smith and Osborne (1996) seems warranted. This meta-analysis offers the Agency an
opportunity to adjust statistically for the different approaches used to estimate visibility benefits
across different studies. The more-recent Beron et al. (2001) residential hedonic property value
(HPV) analysis of the housing-price effects of visibility changes should also be considered.
The Agency proposes combining the estimates from Chestnut and Rowe (1990b,c) with
the preference-calibration approach to benefits transfer for valuing changes in visibility at
national parks. The preference-calibration approach is preferred to previous ad hoc transfer
methods. Nevertheless, like any transfer method, it is constrained by the quality and relevance of
the original study estimates and the data available to support specification of calibration
parameters. While the Agency is currently sponsoring a major visibility study, the complete
results will not be available in time for this assessment. In the meantime, the Agency's only
recourse is to report appropriate error bounds for existing estimates.
Quantified benefits from the improvement of visibility in the Second Prospective
Analysis are limited to recreational visibility benefits in the primary estimates. The Agency
indicated that the main residential visibility study at its disposal had been judged to be too old to
use. There is now additional research that is more recent (e.g., Beron, Murdoch and Thayer,
2001). As much as any other category, visibility benefits have figured large in empirical air
quality benefits estimates from hedonic property value models. The Agency should review the
available studies, revisiting the older ones and adding the newer ones, and develop an approach
for including residential visibility values in the primary estimates. There is no doubt that such
benefits exist and the available studies, both contingent valuation and hedonic property value,
provide a substantial amount of information about the likely magnitude of these benefits.
Additional effort on this front can help reduce errors in benefits calculations stemming from
omitted categories of benefits.
It is possible, independent of the Beron, Murdoch and Thayer (2001) paper, to consider
evaluating stated preference studies concerning residential visibility. Beyond residential
visibility, the recreational visibility studies are also rather old, dating back to 1990, and detailed
literature reviews and attempts to reconcile differences in results have not been updated recently
(e.g., Chestnut and Rowe, 1990a). The Electrical Power Research Institute (EPRI) is sponsoring
a visibility study conducted by Dr. Anne Smith of Charles River Associates. The Agency should
establish contact with this research team and remain abreast of its work.
An important issue that needs to be addressed in a quantitative assessment of residential
visibility values from both the contingent valuation and the hedonic property value studies is that
47
-------
visual air quality is inextricably associated, in terms of people's perceptions, with their concern
about potential health effects. Points on this issue include:
a. CV studies found that some subjects could not ignore their concerns about
potential health effects when answering questions about visibility. Some
approach to separating these values is needed. Results showed visibility
aesthetics were 20% to 40% of value for air quality changes as a whole in
residential areas.
b. Responses to contingent valuation (CV) questions for public goods, such as air
quality, may include altruistic values for other households as well as for the
respondent. But this is an issue with all CV studies for public goods and should
not be a reason to completely ignore the study results.
c. Hedonic property value studies, even when using an objective measure of visual
air quality, can be expected to yield results that reflect values for the aesthetics of
air quality as well as concerns about health effects. The Council suggests that the
Agency consider the possibility of using marginal WTP estimates for a few cities
(LA, Chicago, and others) where recent hedonic studies are available for
comparative evaluation with health effects (see Taylor and Smith, 2000). Doing
so would be approximately consistent with implicit logic of preference
calibration, but would be simpler to implement.
The CV and hedonic studies each have strengths and weaknesses, but considered together
they likely provide enough information for a quantitative assessment with some acceptable
amount of uncertainty.
Materials Damage. The Agency cites obsolete estimates from the 1970's and plans to
monetize soiling damages with new estimates of the demand for cleaning products and services.
This approach has problems similar to using cost-of-illness estimates to value health. Costs are
not the same as benefits. In this case, cleaning expenditures neglect aesthetic losses. The
Agency seems unaware of several more recent studies that have updated the initial "Mathtech"
study. For example, Harrison et al. (1993) obtained updated estimates from Mathtech.
In addition to soiling damages, air pollution can corrode metals and other materials,
leading to potential productivity losses and damage to structures and historic monuments. Most
of these effects are not included in the demand for cleaning products and services. Acres
International Limited (1991) estimated replacement costs for some of these damages. As in
other areas, the Agency should provide appropriate caveats and discuss reasons that estimates are
likely to understate materials damage benefits.
Appendix C includes a separate bibliography on the subject of Materials Damage.
Recreational Fishing: Forestry: The Agency plans to use an updated version of
Montgomery and Needleman's random-utility model for New York state recreational angling
values. Is it is possible to extend the geographic coverage beyond the Adirondack region?
48
-------
10. USE OF VSL META-ANALYSES
10.1. Agency Charge Questions Related to Use of VSL Meta-Analysis
Charge Question 22: EPA's current analytic blueprint calls for an expert-judgment
project on VSL determination that would produce a probability distribution over the range of
possible VSL values for use in the 812 project. EPA is not sure how much priority to give to this
project. A much simpler alternative would be for EPA to specify a plausible range of VSL
values. One option would be to use a range bounded by $1 million (based roughly on the lower
bound of the interquartile range from the Mrozek-Taylor meta-analysis) and $10 million (based
roughly on the upper bound of the interquartile range of the Viscusi-Aldy meta-analysis). This
range would match that reflected in EPA's sensitivity analysis of the alternative benefit estimate
for the off-road diesel rulemaking. The range would then be characterized using a normal, half-
cosine, uniform or triangular distribution over that range of VSL values. EPA would then ask
this Committee to review this distribution. This approach could be done relatively quickly,
based on the reviews and meta-analyses commissioned to date, and would allow a formal
probability analysis to proceed, without suggesting that the Agency is trying to bring more
precision to this issue than is warranted by the available science.
Charge Question 23: Pursuant to SAB Council advice from the review of the first draft
analytical blueprint, EPA reviewed a number of meta-analyses -either completed or underway-
developed to provide estimates for the value of statistical life (VSL) to be applied in the current
study. EPA plans to consult with the Council (and coordinate this consultation with the EEAC)
on how best to incorporate information from the Kochi et al (2002) meta-analysis, other
published meta-analyses (Mrozek and Taylor and Viscusi and Aldy), and recent published
research to develop estimates of VSL for use in this study. In addition, EPA plans to implement
two particular adjustments to the core VSL values: discounting of lagged effects and longitudinal
adjustment to reflect changes in aggregate income. Does the Council support these plans,
including the specific plans for the adjustments described in chapter 8? If the Council does not
support these plans, are there alternative data or methods the Council recommends?
Charge Question 31: EPA plans to work with the Council and the EEAC to develop
revised guidance on appropriate VSL measures. We hope to include the Kochi et al (2002) meta-
analysis, other recent meta-analysis, recent publications, and the 3 literature reviews sponsored
by EPA. (A separate charge question pertaining to this element of EPA's VSL plan is presented
below). In addition, EPA plans to conduct a follow-on meta-regression analysis of the existing
VSL literature to provide insight into the systematic impacts of study design attributes, risk
characteristics, and population attributes on the mean and variance of VSL. Does the Council
support the plans described in chapter 9 for conducting this meta-regression analysis? If the
Council does not support this analysis or any particular aspect of its design, are there alternative
approaches which the Council recommends for quantifying the impact of study design attributes,
risk characteristics, and population attributes on the mean and variance of VSL?
49
-------
Charge Question 37: Does the Council support including the Kochi et al. (2002) meta-
analysis as part of a larger data base of studies to derive an estimate for the value of avoided
premature mortality attributable to air pollution? Are there additional data, models, or studies the
Council recommends? Does the SAB think that EPA should include Kochi et al. 2003 if not
accepted for publication in a peer reviewed journal by the time the final 812 report is completed?
10.2. Summary of Council Response
The Council has combined the responses to charge questions 22, 23, 31, and 37 and has
provided additional discussion concerning the use of VSLs in Appendix B of this Council
Report. Major summary points appear below.
• Since the Panel's initial receipt of the Analytical Plan, the plan for an expert-
judgment project on VSLs has been dropped from the blueprint. The expert
elicitation exercise is no longer an active portion of this charge question.
• Uncertainty analysis with respect to VSL values requires information about the
distribution of VSL estimates. The univariate distribution of all empirical VSL point
estimates derived across all contexts is unlikely to be appropriate for this purpose, as
is any arbitrary convenient distributional shape for this univariate distribution.
• Discounting of effects when there is a latency period is advisable, but the literature on
discount rates for future financial outcomes and future health states is not clear on
whether straightforward discounting using an exponential model and a common rate
will be appropriate. Sensitivity analysis and caveats are recommended.
• Adjustments for future changes in aggregate income levels are being based on very
limited empirical evidence and should be considered placeholder efforts at present. It
would be preferable in the future if these adjustments were made in the context of a
formal model of preferences and the relevant elasticities. Placeholder efforts should
be clearly identified as such and accompanied by strong caveats. The First
Prospective Analysis included (in an Appendix) WTP values for human health
adjusted for real income growth. This type of analysis may be a candidate for the
recommended "Learning Laboratory" or preliminary analyses discussed earlier.
• The Panel recommends a primary focus, at this juncture, on the Viscusi-Aldy
estimates based on U.S. studies, although work in the direction of the Kochi et al.
analysis is encouraged. Ultimately, mean and variance estimates of the VSL
measures for the Prospective Analysis should be based on the conditional
expectations from a model that includes empirically relevant variables for the risk
context and relevant characteristics of the population affected by the CAAA.
Developing such a model should be a priority for future research and meta-analysis
efforts.
• It is certainly reasonable to expect that the Second Prospective Analysis would
consider insights derived from the other VSL meta-analyses (e.g., Mrozek and
50
-------
Taylor, and Kochi et al.). The Council recommends that, to the extent VSL measures
are developed as conditional expectations from a meta-analysis, they should rely
primarily on published peer review studies. However, as the Council's general
comments on approaches to methodological innovation imply, the meta-analyses that
best serve Agency needs will not always be published in the usual academic outlets.
In cases where the analysis is not published, we would recommend an independent
peer review be undertaken that considers the specific elements of the intended use.
• Continual evolution of the relevant literatures justifies development by the Agency of
a more formal laboratory phase for evaluation of potential methodological
innovations. A "satellite benefit-cost analysis," based on updated methodologies,
could serve as a forum for evaluation of new methods before these innovations are
formally and widely adopted by the Agency for the Section 812 Analyses and other
analyses. This is a suitable activity for the "Learning Laboratory."
10.3. Expert Judgment - VSLs
The Agency desires to bound the range of plausible VSL values and define a distribution
from which to select a central value and use in the uncertainty analysis. The proposed range of
$1 million to $10 million is a reasonable placeholder, given the state of the knowledge about
empirical values in different contexts. The Council Special Panel understands that some
distribution is needed from which to draw alternative point values of the VSL for simulations of
the effect of uncertainty about VSL values. However, the Council Special Panel does not agree
with arbitrary assignment of some convenient distribution (e.g., normal, half-cosine, uniform or
triangular) for the range of values. The choice of distribution needs to be empirically supported
as much as possible. Why not compare Mrozek-Taylor versus Viscusi-Aldy meta-analyses,
using the latter's re-estimates with a sample consisting of one observation per study? The
Agency could use these estimates to derive a mean and variance of the relevant conditional
distribution from that model "configured" for the policy analysis. For now, this probably means
excluding studies that are clearly out of bounds in terms of the policy context, such as wage-risk
studies that look at only one occupation (e.g., police).
The Council strongly advises that the issue of context be given high priority as the
Agency pursues more meta-analyses that include more than just wage-risk studies (including
future revisions of the Kochi et al. meta-analysis). The ideal VSL distribution to employ would
be the conditional distribution of VSL values, derived for contexts that most closely match the
risks and affected populations relevant to the CAAA. This VSL does not necessarily lie in the
middle of the overall marginal distribution of empirical VSL estimates across the entirety of the
broad range of contexts examined in the literature. Before this conditional policy-relevant
distribution can be defined, however, empirical work needs to determine what aspects of context
matter to individuals in valuing mortality risk reductions. Estimates in the literature should not
be excluded if the contexts in which they are derived differ in ways that do not significantly
affect the VSL results. This issue is discussed further in subsequent sections of this chapter.
51
-------
10.4. Adjusting for latencies, income growth?
Latency in health effects, as well as cessation lags, means that a comprehensive
assessment of mortality risk reduction benefits must take into account individual discounting. In
discounting individual health effects, there remains an important question as to whether the usual
convenient exponential form of discounting is an appropriate assumption, given the numerous
empirical anomalies related to discounting behavior. There are also unresolved questions about
the difference in discount rates concerning future health, as opposed to future financial status.
While the Council concurs that future benefits need to be discounted, there is no consensus in the
literature concerning how to do this. As a practical matter, pending additional research, the
Agency should adopt discounting assumptions that are consistent with the rest of the Analytical
Plan and include sensitivity analysis and caveats.
The Council has reviewed the SAB report, An SAB Report on EPA's White Paper Valuing
the Benefits of Fatal Cancer Risk Reduction (EPA-SAB-EEAC-00-013), as well as the
background documents of the recommendations for adjusting willingness to pay estimates for
reductions in health effects and mortality risks to reflect changes in real income (i.e., memoranda
to Jim DeMocker from Kleckner and Neumann dated June 3, 1999 and September 30, 2000).
The Council agrees with the general principle that the willingness to pay to reduce mortality
risks is likely to increase with growth in real income. The same increase should be assumed for
the WTP for serious nonfatal health effects. However, our primary concern with the Agency's
proposal to include an adjustment for real income growth in the primary estimates for the Second
Prospective Study is the weakness of the available empirical evidence that can be used to
determine what this adjustment should be. Three factors underlie our concerns:
a. The meta-analyses cited as a basis for the estimates for income elasticities do not
have measures of income to develop estimates of the income elasticity. Both
Mrozek and Taylor (2002) and Viscusi and Aldy (2003) clearly note they do not
have income measures. Mrozek and Taylor use hourly earnings as a "proxy" for
income in a relationship (i.e., one specification for their meta-analysis summary
functions) describing the slopes of hourly earnings equations with respect to job
risk. Viscusi and Aldy use annual earnings in this same role. Both summary
functions could then be interpreted as simply reflecting non-linearities in the
estimates of this slope (an ex ante marginal rate of substitution between wages
and risk, not a WTP).
b. The other evidence cited in these memos is primarily from non-U.S. studies [e.g.,
Persson et al. (1995), Kristrom and Riera (1996), Miller and Guria (1991), Miller
(2000), Jones-Lee et al. (1985)] where the context may well be quite different and
imply tradeoffs that are not relevant to the U.S. situation. The Council
recommended against using wage risk hedonic studies conducted outside the U.S.
in meta analysis summaries of the VSL for similar reasons. Furthermore, many of
the income elasticity estimates are derived from comparing WTP estimates from
different countries. There may be many reasons other than income differences
that may explain why these WTP estimates differ across countries The Council's
concern over this issue is compounded because cross-country comparisons are the
only empirical evidence supporting the high adjustment factor of 1.0 for the
52
-------
values for mortality risk. Most of the estimates from within the U.S. are less than
0.5. In addition, several of the studies relate to changes in aspects of
environmental quality that are not related to health or mortality risk.
c. The Agency's logic for assessment of benefits uses unit values per case of
relevant morbidity to develop annual estimates of the aggregate benefits from
specific policies affecting that health outcome. The concentration-response
functions provide estimates of the changes in cases associated with the proposed
policy. These unit values are derived transforming results that are based on WTP
(or marginal WTP) measures. To account for the effects of income on these unit
benefit measures properly, both estimates of the income elasticity of WTP and a
basis for evaluating how income influences other factors that contribute to the unit
value measure are needed. In some cases this will be straightforward. In others it
will not.
Palmquist (2003), for example, has demonstrated that when weak
complementarity and Willig's (1978) restriction are used to recover a measure of
the economic benefits of a quality change, then one can assume that the income
elasticity of the marginal WTP for the change in environmental quality is equal to
the income elasticity of demand for the private good serving as the weak
complement. This same condition can be used to imply the unit benefit of a
change in quality does not change with income. It is important to acknowledge
that this is a restriction that affects the interpretation of available estimates. It
does not mean that WTP per case does not change. Rather, it means that to
develop estimates of the WTP, analysts assumed that WTP per case did not
change, so those studies would not provide information that would allow anyone
to judge the responsiveness of the WTP per unit to income.
Given the limitations and uncertainties in the available empirical evidence, the Council
does not support the use of the proposed adjustments for aggregate income growth as part of the
primary analysis. It is appropriate to continue to include this as a sensitivity analysis and to
continue to look for stronger empirical evidence from which to derive adjustment factors. The
Council realizes that this advice differs from the somewhat stronger endorsement of this
adjustment that was given in previous Council recommendations (EPA Advisory Council on
Clean Air Compliance Analysis, 2001). However, after taking a close look at the available
literature and proposed interpretations of the available evidence, the Council concludes that
moving these adjustments into the realm of "primary estimates" is premature. The case of cancer
valuation may have some special circumstances because the issue involves a discounting because
of the latency period as well as an adjustment for real income growth, both of which have limited
empirical support for specific numbers.
Any income adjustments in the present analysis fall within the category of satellite or
exploratory analyses that may be developed as supplementary to the primary analysis as one of
the activities of the proposed Learning Laboratory. As such, they would be intended to stimulate
discussion and review, rather than constituting a primary component of an analysis intended to
be used in evaluating a policy. In any provisional analysis, it may be possible to place bounds on
53
-------
the likely errors that would accompany simple approximations to likely income effects. If an
adjustment of this type is considered essential even at this stage in the analytical process, the
Agency should be especially prudent in qualifying it and present the results in a format that is as
transparent as possible. This would include explaining in detail how any income adjustments
have been accomplished and why they are deemed to be necessary.
It is worth emphasizing that as soon as the Agency begins to manipulate VSL estimates
to reflect anticipated changes in real incomes, it opens the door to arguments that VSLs should
also be adjusted for other long-run changes. These might include other changes in budget
constraints, such as alterations to the relative prices of medical care. Or, they could include
shifts in typical indicators of preferences, such as trends in the sociodemographic mix in the
population (e.g., changes in the age distribution).
The Agency should also be aware that if monetary values for health risk reductions are to
be adjusted for income growth, so should be all of the other demand-based benefit measurements
entertained in the Section 812 Analyses. It may be difficult to defend making income-growth
adjustments only to one component of the benefits algebra.
In the longer term, consideration should be given to obtaining income-based adjustments
to VSLs (or even other types of adjustments) through preference calibration techniques. These
methods hold promise for generating forecasts that are consistent with the relevant elasticities
(see Smith, Pattanayak, and Van Houtven, 2003).
10.5. Available meta-analyses
Three meta-analyses were discussed in the Agency's evaluation of summary measures for
the available VSL estimates (Mrozek and Taylor, 2002, Viscusi and Aldy, 2003, and Kochi,
Hubbell, and Kramer, 2003). The three meta-analyses differ in several key respects, including:
a. The number of observations included from each study;
b. The format of the observations (e.g., actual estimates, use of group means, and
other transformations of the primary estimates);
c. The sample composition - U.S. studies, international, revealed and stated
preference;
d. The set of independent variables used for controls (e.g., inclusion of industry
effects);
e. Bayesian means versus regression summaries;
f Published versus unpublished summaries.
The background for the charge questions tends to focus attention on the selection of a
single meta-analysis as the basis for developing the primary VSL estimate of reductions in
mortality risk for the next Prospective Analysis. However, the charge questions explicitly refer
to the "systematic impacts of study design attributes, risk characteristics, and population
attributes on the mean and variance of VSL." The earlier meta-analysis strategies tended to miss
the opportunity to combine the insights from all studies to influence how summary measures are
constructed and used. The Council recommends that future meta-analyses, including revisions to
the Kochi et al. analysis, give priority to examination of how values for mortality risk reductions
54
-------
may vary systematically with study design attributes, risk characteristics, and population
attributes. Insights gained should be used to guide selection of VSL results for use in specific
policy analysis applications.
Equally important, the sensitivity of VSL estimates from meta-summary equations to the
sample composition (i.e., which studies are included) and to the controls used (i.e., which study
features are explicitly modeled) suggests that it would be prudent to use the resulting lessons
from this research in at least three ways:
a. If one meta-analysis, such as Viscusi and Aldy (2003) is selected, evaluate the
sensitivity of the conditional expectation to the baseline risk and other control
variables selected in measuring the conditional prediction.
b. Evaluate the variance in the conditional prediction as a function of the values for
the independent variables included in the model in relation to the mean values for
these variables for the sample used to estimate the model.
c. Consider the effects of inclusion or exclusion of independent variables or
observations on the coefficient estimate for the risk measure. The data sets used
in these studies are generally available for attempts at replication, so this type of
comparison can be readily undertaken and would permit evaluation of the
sensitivity of the VSL estimate to assumptions made, based on the available
literature.
In general, it does not seem prudent to extend the sample to include studies for labor
markets outside the U.S. The terms of employment, information about safety conditions, fringe
benefits (e.g., health insurance), etc., are likely to be so different that one could not be sure that
differences attributed to income or risk levels were in fact due to these variables.
10.6. Interpreting CV measures as opposed to wage-risk measures
One advantage asserted for the Kochi et al. study is the inclusion of CV evidence
concerning VSLs. However, there is an important issue that has not been adequately discussed
when CV results are included with revealed preference wage-risk results concerning VSLs.
Calculating a VSL point estimate from CV results implicitly accepts a proportionality
assumption between ex ante WTP and the risk change.
The proper theoretical interpretation of the CV measures is as an ex ante option price for
a risk change. If OP denotes the value for a risk reduction from PO to PI (with PI < PO), and the
P's designate the probability of death before and after the risk reduction, theory implies:
Equation 1: OP = f (PO, PI , and other variables)
The comma between PO and PI implies that linear proportionality in (PO - PI) is an
approximation, not a feature implied by theory. Thus, to rewrite equation (1) as equation (2)
below, where the option price associated with a risk reduction is proportional to the size of the
risk reduction (as well as being a function of a number of other variables) and then to
55
-------
approximate VSL as in equation (3) by normalizing upon a 1.00 risk change, adds additional
untested assumptions.
Equation 2: <>P = (P0-P,)- ^(other variables)
OP
VSL x -, - r = Bother variables)
Equations: (P° ~P"
A meta-analysis that includes CV studies to expand the range of risk changes (or the
types of risks considered) will accomplish this objective. However, it also changes the summary
measure from an ex ante marginal rate of substitution to a linear approximation. Unfortunately,
this added condition makes it difficult to evaluate whether the resulting differences in summary
results between CV and wage-risk studies should be attributed to these additional assumptions
implicitly added to the model or to the expansion in the range or types of risks.
Nevertheless, the Council recognizes that CV-based studies offer unique opportunities to
examine the empirical influence of many additional factors on the resulting estimates of VSLs.
Despite the potential difficulty in rendering their findings compatible with those from revealed
preference wage-risk studies, CV studies have the potential to make important contributions to
our understanding of how consumers value risk reductions and it is important to take advantage
of these opportunities.
10.7. Emerging considerations
As recent unpublished research by Cameron and DeShazo seems to suggest, the terms
identified in equations (1), (2), and (3) above, and other things, may well be very important to
the ex ante option price measured for the risk change. This research is presently available only
as early reports from a detailed CV study. Nonetheless, it reaffirms the notion that it may be
important to evaluate the sensitivity of the conditional expectation of the VSL to the conditioning
variables used in its construction.
The Council's discussion also supported efforts to refocus attention on incremental WTP
for an incremental risk change, rather than the traditional, but potentially confusing construct that
is a VSL. The Panel's discussion urged the Agency to consider including an introduction to the
concept the Agency is using as a benefit measure, its likely link to the conditions of daily living
and illness preceding death, as well as to any latency and temporal issues associated with
exposure and increased risk of death.
The Panel recognizes that the current state of research makes it unlikely that empirical
measures can immediately be developed that reflect all of these concerns. Nonetheless, the
discussion led to a consensus that the Panel should urge Agency staff to consider careful
qualification and sensitivity analysis for the measure used to monetize mortality risk reductions.
56
-------
10.8. Which meta-analyses to use
In general, the Council Special Panel recommends that the Kochi et al. meta-analysis
should not be given any particular prominence among the alternative meta-analyses used for
determining one appropriate measure to use for the VSL. There are several reasons:
a. The Kochi study is still unpublished. While it can sometimes be difficult to
publish further meta-analyses when others are already in the literature, the
Agency should not rely disproportionately on the Kochi study before it has been
thoroughly peer-reviewed. The standards for peer-review obviously differ across
journals and even across reviewers, but reliable peer-review can also be
accomplished outside of the journal publication process. Both Mrozek and Taylor
(2001) and Viscusi and Aldy (2003), however, have already appeared in the peer-
reviewed literature.
b. There are problems in the derivation of the variance of the VSL estimates. Some
appear to be typographical errors. The researchers apparently faced some
problems in terms of unobserved (or unreported) covariances among parameter
estimates. However, it might be possible to derive estimates of variance in mean
annual wage from the current population survey (CPS) or other sources, and use
this information to fill in some of the blanks. It is not clear whether one should
use a predicted wage or an actual mean wage. Overall, this is a careful study but,
like all meta-analyses, it needs to address the potential impact of some of its key
assumptions on the results of the analysis before it is possible to assess their
importance.
c. The use of author-specific means of VSL (p. H-12 to H-13) is troublesome if the
different estimates have been derived from different samples.
If called upon to recommend just a single meta-analysis at this point, the Council Panel
would recommend a primary focus on the Viscusi-Aldy estimates based on U.S. wage studies.
However, as the 812 process evolves over time, the Council has recommended a commitment to
satellite or provisional analysis to test new methods in a policy relevant format. This would
assure that the Agency did not miss opportunities to incorporate insights from new research as it
emerges. It would also signal a commitment to understanding the full implications of
methodology change before it was adopted as the "Agency Practice."
Ultimately, variance estimates for the VSL measures predicted for a risk context and an
affected population similar to those relevant to the CAAA should be based on the variance in the
conditional expectation from a model that includes empirically relevant variables for risk context
and population characteristics. Developing such a model should be a priority for future research
and meta-analysis efforts.
10.9. Unpublished meta-analyses?
The Council was asked explicitly to address the question of unpublished meta-analyses.
In general, the Council believes a peer-reviewed study will have greater professional credibility
57
-------
than one that has not met this standard. The Panel has some reservations about basing an
analysis with the gravity of the Second Prospective Analysis on unpublished research, but has
even greater reservations about using entirely non-peer-reviewed research. Each of the available
meta-analytic studies has different advantages and shortcomings so that no single study should
be the sole basis for information about the distribution to be used for the VSL in the Second
Prospective Analysis.
This is another reason for creating an ongoing commitment by the Agency to engage in
activities that serve as laboratories for methodological developments. Based on innovations in
the literature, new methods and new meta-analyses will continue to be developed and applied to
policy issues. First, they should be used for evaluative purposes. Results designated as
explicitly as "exploratory" can be disseminated in Agency working papers to evaluate the
implications of new proposals for analysis. This process serves a role that parallels the peer
review process. However, it is more focused and relevant to Agency needs because the
appropriate policy context is being considered. These satellite benefit cost analyses could then
provide a forum for exchange and evaluation of new methods before they are formally adopted
for specific analyses that would be submitted as the Agency's official evaluation of a proposed
regulation.
58
-------
11. QALY-BASED COST EFFECTIVENESS
11.1. Charge Question 24:
For the 812 Report, EPA has decided to perform a cost-effectiveness analysis of the Clean
Air Act provisions using quality-adjusted life years as the measure of effectiveness. This is the
standard approach used in medicine and public health and this type of analysis has previously
been recommended by the SAB. Moreover, the recent NAS Report (2002) on benefits analysis
discussed how this method could be applied to the health gains from air pollution control.
a. Do you agree that QALYs are the most appropriate measure of effectiveness for
this type of analysis? Would you suggest any alternative measures to replace or supplement the
QALY measure? (This question relates to effectiveness measures, not monetary benefit measures
as used in benefit-cost analysis).
b. OMB has suggested that EPA plan a workshop with clinicians, social scientists,
decision analysts and economists to examine how the specific diseases and health effects in the
812 Report should be handled with respect to longevity impact and health-related preference.
Participants would have knowledge of the relevant clinical conditions, the related health
preference studies, and the stated-preference literature in economics. The recent RFF conference
has laid the groundwork for this type of workshop. Is there a superior approach to making sure
that the CEAQALY project is executed in a technically competent fashion and that the details of
the work receive in-depth technical input in addition to the broad oversight provided by this
Committee?
c. Does the Council support the specific plans for QALY-based cost-effectiveness
described in the current draft blueprint? If the Council does not support specific elements of
these plans, are the alternative data, methods, or results presentation approaches which the
Council recommends?
11.2. Summary of Council Response:
The Council understands the Agency's interest in conducting a cost-effectiveness
analysis since this is being required by OMB in addition to benefit-cost analysis for major
regulations. Some cost-effectiveness analysis for the Section 812 Analysis has also been
suggested in previous Council recommendations (EPA Advisory Council on Clean Air
Compliance Analysis, 2001). In this Advisory, the Council cautions the Agency to proceed
carefully in this regard and keep the primary focus on the benefit-cost analysis.
• This Council has had difficulty coming to full agreement about recommendations
regarding the appropriateness of QALYs for use in this context. The limitations
of the measure have led some members to want to recommend against using it at
59
-------
all. Other members acknowledge the limitations, but are more comfortable
endorsing exploratory efforts by the Agency to apply the measure in a cost-
effectiveness analysis.
• There are important limitations of any cost-effectiveness analysis that need to be
recognized. Focusing exclusively on human health effects relegates the other
benefits of the CAAA to the sidelines. There are also other problems with respect
to the selection of an effectiveness measure for reductions in human health risks
(e.g., QALY).
• The Council's reservations about QALYs stem primarily from concerns about
QALY weights on health state attributes being inconsistent with the utility-
theoretic models that underlie benefit-cost analysis unless excessively strong
assumptions are made. All members agree that there should be no attempt to
develop utility-based monetary valuations for QALYs (such as WTP per QALY)
as these are conceptually inconsistent approaches.
• The deliberations of the Institute of Medicine's Committee to Evaluate Measures
of Health Benefits for Environmental, Health, and Safety Regulations can be
expected to be of considerable value in resolving some of the Council's concerns.
This study was requested by the Office of Information and Regulatory Affairs of
the Office of Management and Budget and is supported by a consortium of
federal agencies that are responsible for assessing and reducing environmental,
occupational, and consumer risks to health and safety. The committee's report
will not be available until late in 2005. The Council advises that the Agency
forestall any efforts to conduct cost-effectiveness analysis using QALYs until that
report is available.
11.3. Challenges and limitations of CEA
Cost-effectiveness analysis (CEA) calculates costs per unit of effectiveness. A metric of
effectiveness therefore needs to be defined that reflects the expected outcomes of the program.
Benefit-cost analysis (BCA) estimates net benefits, which is an indication of how much better off
society as a whole is likely to be if the program is implemented. In BCA both costs and benefits
are defined in terms of changes in well-being or utility, and both are quantified in monetary
units. In BCA, analysts' measurements of benefits are grounded conceptually in individual
preferences.
Although the conceptual basis for valuation of benefits in BCA is clear, the empirical
implementation is fraught with difficulties and limitations, especially when the primary effects of
a program are non-market goods and services, such as protection of human life and health and
quality of the natural environment. CEA, therefore, has some appeal because it avoids the need
to determine how much better off individuals are with the program. It simply measures the
effect in some selected metric, such as numbers of acres restored, number of deaths prevented,
number of accidents prevented, etc. The calculation of the cost per unit of effect is helpful in
determining which of several programs, designed to achieve the same goal, is most cost-
60
-------
effective. However, it does not inform about whether the program is worthwhile, i.e., whether
the value of the benefit of the program exceeds the costs. CEA also says nothing about how to
allocate resources among programs that achieve different effects (e.g., saving trees or saving
people).
The Council concedes that CEA is widely used in other public-health domains and that
some users of the Second Prospective Analysis will wish to compare the cost-effectiveness of the
CAAA as a form of public health policy with the cost-effectiveness of other health programs.
CEA comparisons may be a reasonable way to approach alternative medical treatments where all
of the benefits of each alternative treatment accrue as changes in health status. For the CAAA,
however, a strategy that attributes all of the costs of the policy only to the increases in health
status does not provide a valid comparison, regardless of the health measure used (QALY, lives
saved, life-years saved, etc.). There are other non-human-health benefits associated with the
CAAA (e.g.., ecosystem benefits). Furthermore, since the costs of the policy are joint costs that
cannot be attributed separately to the different classes of benefits from the CAAA, there is no
way to apportion these costs to arrive at a cost just for the health changes produced.
Apportioning these costs is essential before any meaningful cost-effectiveness comparison can
be attempted between the CAAA and private medical interventions as alternative means of
improving human health. Researchers have invested heavily in the fine-tuning of standardized
cardinal physical measures for human health improvements, but these measures cannot capture
the broader benefits of clean air policies.5
The proposed remedy for this problem is to calculate net costs by subtracting all the non-
health benefits that have been monetized in the benefits assessment. Such a procedure, however,
remains a less than satisfactory solution when there are many potential non-health benefits that
are poorly measured or not quantified at all. Some Council members find this approach
troubling because it mixes benefit-cost analysis with cost-effectiveness. The Council
acknowledges elsewhere in this report that the task of monetizing ecosystem benefits is a
particularly difficult one. In general, when policy costs are non-separable and additional benefits
cannot readily be monetized, it is extremely difficult to arrive at a cost that applies only to the
health outcomes produced.
Separability in preferences is also a pervasive concern in cost-effectiveness analysis.
Some of the important nuances in the QALY-WTP discussion hinge upon the extent to which
health affects the marginal utility of income or wealth. The possibility that marginal utility of
income depends on health means that WTP for health, environmental quality, or anything else
may depend on health. This implies that one should account for the effect of population
heterogeneity in health states when estimating WTP.6
5 One Council member points out that omitted non-health benefits of clean air policies are also a qualification
affecting formal benefit-cost analyses, so this problem is not exclusive to QALY analysis. The two methods merely
handle this problem differently.
6 Different members of the Council express different degrees of concern about the consequences of assuming
separability between health and income.
61
-------
11.4. QALYs as a Measure of Effectiveness
The Council acknowledges that it has previously recommended that QALYs be
considered as candidate measures for "units of physical benefit" for the human health benefits of
air quality improvements. Nevertheless, the composition of the Council has changed over time
and the opinions of some of its members have been influenced by new information. Some of this
information was provided in a special conference hosted by Resources for the Future entitled
"Valuing Health Outcomes: An Assessment of Approaches" which took place in Washington,
DC, on February 12-14, 2003. The subject matter of the conference was "the conceptual and
empirical bases for alternative health-benefit measures, the ways in which such measures are
used and could be used in policymaking, and whether the choice of measure would actually
make a difference in policy outcomes." In attendance were diverse groups of "experts,
government officials, and stakeholders," and the tenor of much of the discussion concerned the
relative appropriateness of cost-per-QALY measures versus WTP measures.
It is likely that the Second Prospective Analysis will provide sufficient detail about
benefits and costs that some audiences will be tempted to make cost-effectiveness calculations
even if the Agency does not provide them. However, in view of the standards to which the
Council has held other dimensions of the Section 812 Analysis, QALY-based analyses should be
subjected to comparable scrutiny. The usual applications for QALY-based cost-utility
comparisons involve only well-defined human-health benefits. The Clean Air Act and its
amendments do not fit neatly into this framework. Members of the Council have articulated a
number of additional specific reservations about the use of QALYs in the context of the Section
812 Analyses. These reservations concern consumer sovereignty and representativeness,
ordinality versus cardinality, heterogeneity in health states, and the notion of willingness to pay
for a QALY. Details about these concerns appear in Appendix F.
The Council would prefer to present the Agency with an unambiguous conclusion on the
QALY cost-effectiveness matter. However, after several rounds of discussion on the topic,
spanning several meetings, the Council has been unable to reach a unified view. The Council
agrees that the jury is still out on whether QALY cost-effectiveness measures can be successfully
adapted, in the future, to reflect both sufficiently general consumer preferences and the full array
of non-human-health benefits also stemming from air quality improvements. Some Council
members note that there is likely to be strong demand for QALY measures; others are firm in
their convictions that the Agency should not be pressured by the wide acceptance of what they
believe to be incorrect practices into using them in the Second Prospective Analysis.
The Council thus supports the Agency's plan to do a benefit-cost analysis as the main
analysis and to treat any cost-effectiveness analysis as an ancillary calculation. In the current
round, QALY-based methods should, at best, be included among the various methods and
procedures to be considered for the "Learning Laboratory" where possible future enhancement
are explored, tested and vetted by experts in all relevant fields.
62
-------
11.5. Summary:
The Agency seems obliged, in complying with OMB guidance, to consider cost-
effectiveness measures. The current Council, however, would prefer that the Agency not
interpret this mandate as specifically requiring that this cost-effectiveness analysis take the form
of explicit cost-per-QALY assessments. While QALYs may capture the majority of benefits
from private individual medical treatments such as surgeries or medications, QALYs are not able
to fold in all of the diverse benefits of a public good like clean air. Clean air may produce
substantial human health benefits, but it may also provide substantial benefits to ecosystems. In
general, it is not possible to accurately attribute shared costs to different categories of benefits.
Only with an assumption of complete separability among costs and benefits across human health
and other benefits can non-health benefits be treated as cost offsets and netted out of the cost-
effectiveness calculations. Assessment of this separability assumption is a task for the Learning
Laboratory the Council is advising the Agency to develop to support the Section 812 Analyses.
In cases such as the Section 812 Analysis, it may currently be possible to go no further
than describing the costs and listing the array of known, estimated, and speculative physical
benefits from the Clean Air Act and its amendments. QALYs could of course be entertained as
one category of these physical benefits, but it should be made clear that overly simple cost-per-
QALY calculations will be biased upward for the Section 812 Analysis, relative to alternative,
exclusively health-enhancing, programs with which stakeholders may wish to make
comparisons. If separability could indeed be assumed and if the monetary value of the non-
health benefits is first subtracted from costs, then the cost-effectiveness ratio is biased upward if
and only if the monetary benefits of the non-health effects are underestimated.7 Without
separability, it may not even be possible to sign the direction of the resulting bias.
The Council advises the Agency to determine whether this type of accounting, with costs
and an enumeration of all classes of physical benefits (perhaps including, but not limited to
QALYs) would satisfy the OMB requirement for cost-effectiveness analysis. However, the core
of the Second Prospective Analysis should concentrate on using generally accepted and
thoroughly vetted benefit-cost methodologies, as the proposed main analysis currently does. The
Council does not endorse any substantial effort to calculate QALYs or benefits in the form of
WTP per QALY as part of the main analytic agenda for current Section 812 assessment. The
Council recommends that the Agency reserve judgment on this matter at least until the Institute
of Medicine report becomes available in late 2005. The mandate to conduct some type of cost-
effectiveness analysis suggests that the Agency devote attention to alternative strategies for
meeting this mandate. However, the Agency should explore candidate methods under the
category of Learning Laboratory activities, rather pursuing such analyses on an equal footing
with the main benefit-cost analysis. In general, cost-effectiveness analyses should be presented
as "alternative" analyses even when (or if) they are mainstreamed into future Prospective
Analyses.
7 In this situation, however, the net benefits in any benefit-cost analysis will also be biased in a way that makes the
CAAA look less favorable.
63
-------
12. MORBIDITY EFFECTS
12.1. Charge Question 25
EPA plans to use updated unit values for a number of morbidity effects, as described in
chapter 8. Of particular note, EPA plans to rely on a study by Dickie and Ulery (2002) to provide
heretofore unavailable estimates of parental WTP to avoid respiratory symptoms in their
children. This study is not yet published and has limitations concerning response rate and sample
representativeness; however, EPA expects the study to be published prior to completion of the
economic valuation phase of this analysis. Does the Council support the application of unit
values from this study, contingent on its acceptance for publication in a peer-reviewed journal? If
the Council does not support reliance on this study, are there other data or methods for valuation
of respiratory symptoms in children which the Council recommends?
12.2. Summary of Council Response:
• The Agency should continue to use WTP estimates for morbidity values, rather than
cost-of-illness (COI) estimates, should these be available. Where WTP is
unavailable, COI estimates can be used as placeholders, awaiting further research,
provided these decisions offer suitable caveats.
• The Dickie and Ulery study is a valuable addition to the repertoire of empirical results
concerning WTP for acute respiratory illnesses and symptoms, although it is not so
superior as to supercede all earlier studies.
• Values for "bad asthma days" might be approximated by transfer of results for
respiratory-related minor restricted activity days, pending the development of updated
results on this topic.
• The Analysis could still benefit from new estimates of WTP to reduce the risk of non-
fatal heart attacks. Current COI estimates assuming average lost earnings over 5
years do not comport entirely with all evidence in the literature concerning
employment and earnings effects.
• Where mortality valuations subsume pre-mortality morbidity, the Agency should be
careful to avoid double-counting. Where values for the two health states, morbidity
and lost life-years, can be separated, both should be counted.
12.3. General Points
The primary challenge for the Agency in determining monetary values for morbidity
health effects is to match the valuation exercise as closely as possible to the definition of the
health effect in the studies being used as the basis for the relevant concentration-response
function. The Agency has done a good job with this in applying the available literature and
making appropriate adjustments when possible, such as for the average severity for chronic
bronchitis cases. The Council cautions that the closeness of the match should continue to be
64
-------
taken into account as new health effects and economic valuation studies become available.
Improvements in matches may be possible as new studies emerge.
The Council recommends that, in general, all available valuation studies that pass
reasonable quality and applicability standards should be considered when developing a range of
values for a particular morbidity category. Most studies have limitations but these vary for
different studies. Considering the results from all available studies provides a more reliable basis
for valuation and a more realistic picture of the uncertainty in the estimates. It may be
appropriate to give some studies more weight than others based on their various strengths and
weaknesses and relevance for a given health effect.
The Agency should continue to use WTP estimates when these are available, rather than
COI estimates. However, it is useful to compare available WTP estimates to available COI
estimates, as the Agency is doing for some morbidity categories such as chronic bronchitis,
because this may help provide a general sense of credibility for the WTP estimates that are based
on survey elicitation or revealed preference estimation approaches. However, it is important to
recognize that the COI estimates are not appropriate alternative estimates to be substituted for
WTP estimates because they do not reflect the preferred concept of valuation.
It is nevertheless appropriate to use COI estimates when WTP estimates are not available,
such as the Agency proposes to do for non-fatal heart attacks. It is reasonable to presume that
this strategy typically understates WTP values. However, it is important to keep in mind that an
individual's WTP to prevent an illness may not fully reflect the costs covered by insurance. This
could result in a situation where a COI value may exceed an individual's WTP when medical
costs are substantial and are covered to a significant extent by health insurance.
12.4. Acute respiratory illnesses and symptoms
Dickie and Ulery (2002) is a good addition to the WTP literature for acute respiratory
illnesses and should be included in the set of studies used as the basis for the values for these
health effects. However, it is not so superior that it should supercede all previous studies. It
should simply be added to the pool of studies available for valuing acute respiratory illness or
symptoms in adults.
The Council urges some caution in interpreting the Dickie and Ulery results in the
context of previous morbidity studies. The estimates are based on an unrepresentative
convenience sample of Mississippi households that are more educated and have higher incomes
than the general population. There are also some concerns about response rates. In addition, the
authors employ a repeated CV elicitation format. This format has not been subject to the validity
testing of more conventional formats. When the problem involves eliciting tradeoffs among
multiple symptoms, durations, and costs, stated-choice conjoint analysis is an alternative with
better-known theoretical and empirical properties.
Dickie and Ulery provide information on WTP values for preventing acute respiratory
illness in children that has not been available from previous studies. The results suggest that
parents value the prevention of acute respiratory illness in their children at about twice the value
they place on the same prevention for themselves. The estimates of WTP values for preventing
65
-------
illnesses in children from this study are appropriate to use for comparable pollution-related
health effects. The ratio of values for adults to those for children is appropriate to use when only
adult values are available. It would also be appropriate to compare adult values for the same
illnesses from other studies, adjusted using this ratio, to the results from Dickie and Ulery for
children.
Dickie and Ulery's Table 7 reports results from other WTP studies. Overall, the Dickie
and Ulery results suggest that the current Agency values for respiratory illnesses, especially for
children, are probably too low. This table also raises questions about the estimates selected for
use in the previous Prospective Analysis. Those numbers are generally lower than the numbers
shown in the Dickie and Ulery table although they appear to be based on a similar set of studies.
These apparent differences in the interpretation of the previous literature need to be reconciled.
It would also be useful to take a look at the results of Johnson et al. (2000). Although
this study was done in Canada it was a nicely designed choice format approach for valuation of
short-term respiratory and cardiovascular symptoms of varying severities. Given the limited
number of U.S. studies, the uncertainties about differences in preferences between the U.S. and
Canada may be acceptable given the additional information the study provides. An important
concern with the Canadian study is that the health care type payment vehicle may be affected by
the availability in Canada of a public health care system. One Council member (who is also an
author of the Johnson et al. (2000) study) noted that all health care costs are not covered by the
Canadian health care system. This is similar to the situation in the United States where many
people have health insurance, but some out-of-pocket expenses are still incurred.
12.5. Asthma exacerbations
The HES has recommended that asthma exacerbations be added back into the base case
estimates, so some economic valuation of these will be needed. (EPA Council, 2004). The
Agency stopped using the estimates of WTP for preventing a "bad asthma day" (Rowe and
Chestnut, 1985) because of concerns about matching the definition of a bad asthma day to the
epidemiology results used to calculate asthma exacerbations. The endpoint was defined in the
original study to reflect the heterogeneity in the severity of asthma symptoms in a particular
panel of asthma patients.
However, the challenges of matching available valuation estimates to the epidemiology
evidence is an issue for all of the acute respiratory illnesses or symptoms. Rather than exclude a
study because of these transfer uncertainty issues, it may be preferable to consider all the
available valuation studies on respiratory symptoms such as coughing, wheezing or shortness of
breath for those with diagnosed asthma and the general population.
As a whole, these studies suggest a reasonable range of WTP values for these types of
symptoms. Preventing asthma exacerbations can be presumed to be at least as valuable as
preventing similar symptoms in the general population. The HES has noted that asthma
exacerbations are likely to result in some level of activity restriction. Thus, even if a specific
value for preventing asthma exacerbations is uncertain given available information, it may be
reasonable to presume that preventing an asthma exacerbation is at least as valuable as
preventing a respiratory-related minor restricted activity day.
66
-------
12.6. Non-fatal heart attack
Lacking a WTP estimate for reducing the risk of having a non-fatal heart attack, the
Agency is basing a valuation for this effect on a COI estimate. This will likely understate the
total welfare effect, as acknowledged by the Agency. It is reasonable to presume hospitalization
for a non-fatal heart attack, and the 5-year medical costs seem appropriate as there is often
significant follow-up treatment after an initial heart attack. However, it does remain somewhat
uncertain whether air pollution exposure causes a heart attack that would not have otherwise
occurred, or merely causes it to occur earlier than it otherwise would have. This cannot be
determined based on the available epidemiology results for this health effect. It remains an
important research question whether air pollution is a factor contributing to the development of
the underlying coronary heart disease (as it has been associated with onset of some chronic
respiratory diseases). However, a heart attack does cause damage that might not have otherwise
occurred until much later, if at all, so it is appropriate to include follow-up costs linked to the
heart attack.
Cropper and Krupnick (1990) is cited as the source of estimates on lost earnings resulting
from non-fatal heart attack. This study provides results of a unique analysis that may not be
available elsewhere in which labor force participation and reduced earnings for those who
remain employed, are both estimated for several chronic health conditions. The data used for
this analysis, however, are fairly dated as they are drawn from a Social Security survey on
disabilities conducted in 1978.
Results from Krupnick and Cropper show a decline in earnings through age 65 for those
who experience a first heart attack between age 45 and 54, but no significant loss in earnings for
those aged 55 and older, or for those under age 45. This is not consistent with the assumption
used in the proposed estimates which is that everyone suffers the average earnings lost for 5
years only. Wages can be updated to current levels. However, if treatments for heart attack have
changed significantly since 1978, then estimated effects on employment and earning may be out-
of-date.
12.7. Chronic Bronchitis
Charge Question 15 asks whether premature mortality implications of morbidity
endpoints should be added. The HES recommendation is that mortality risks from chronic
conditions caused by air pollution exposure should be presumed to be captured in the prospective
cohort studies. The HES has recommended against alternative estimates that totally exclude the
prospective cohort mortality risk studies. Thus, adding mortality risks associated with chronic
conditions that have been linked to pollution exposures in other studies would potentially result
in double counting mortality risks. Consistent with this interpretation, the valuations for the
chronic illnesses should not include value for any associated increase in mortality risk.
The results in Viscusi et al. (1991) provide the basis for the chronic bronchitis valuation
estimates. Respondents to this survey were not told anything about changes in life expectancy
associated with the condition so there is no reason to expect their responses to reflect any
significant concern for this.
67
-------
13. UNCERTAINTY ANALYSIS - PLANS
13.1. Charge Questions Concerning Uncertainty Addressed in this Report
Charge Question 26. Does the Council support the plans described in chapter 9 for
estimating and reporting uncertainty associated with the benefit and cost estimates developed for
this study? If there are particular elements of these plans which the Council does not support, are
there alternative data, models, or methods the Council recommends?
Charge Question 27. Does the Council support the plans described in chapter 9 for the
pilot project to develop probability-based estimates for uncertainty in the compliance cost
estimates? If the Council does not support this pilot project, or any particular aspect of its design,
are there alternative approaches to quantifying uncertainty in cost estimates for this analysis
which the Council recommends?
13.2. Summary of Council Response to Charge Question 26
• The Revised Analytical Plan sets ambitious goals for improved treatment of
uncertainty. However, due to the lack of detail in Chapter 9, the Council Panel has
had some difficulty in evaluating the proposed actions implementing those plans.
• The Revised Analytical Plan does offer specific proposals for analysis of uncertainty
in components of the benefit-cost analysis. The Council Panel endorses two of these
(i.e., the plan for a pilot study of expert judgment on dose-response for particulate
matter and the ozone mortality meta analysis) with small reservations noted below,
but has reservations about others.
• The Council Panel's larger criticism of the Revised Analytic Plan is that it offers little
insight about either how these specific components were chosen as the focus of more
detailed analysis or about how information from these component analyses will be
combined to yield useful information about the overall level of uncertainty in the
analysis of net benefits of air pollution control, the major contributors to that
uncertainty, or of the priorities for research to reduce such uncertainties.
• The Second Prospective Analysis should address the pervasiveness of uncertainty in
cost and benefit estimates. Those elements that are both highly uncertain and have
the potential to have a significant impact on the results should be the focus of
sensitivity analyses. Sensitivity/uncertainty analysis needs to be an iterative process
to identify and assess the significance of key uncertainties in each step of the
assessment. Only a selected set of the most influential uncertainties should be
quantitatively followed all the way through to the final results.
• The Council advises the Agency to develop its uncertainty analyses with reference to
the recommendations in reports of the National Research Council (2002) and OMB
(2003). It also advises the Agency to use the list of "key uncertainties" from the first
Prospective Analysis as a framework.
68
-------
13.3. Detailed Comments Related to Charge Question 26
The Revised Analytical Plan sets ambitious goals for improved treatment of uncertainty.
However, due to the lack of detail in Chapter 9, the Council Panel has had some difficulty in
evaluating the proposed actions implementing those plans.
The Agency proposes to follow the guidance in the National Research Council (2002)
and in the September 2001 Council report, which recommended that "parameter uncertainty and
as many types of model uncertainty as possible, be treated within a probabilistic framework"
(page 9-4). Chapter 9, however, is relatively brief. It provides mainly broad discussion, with
little additional specific content on how uncertainty analysis will be accomplished.
The Plan discusses utilization of an expert in the field of uncertainty analysis and
developing a lexicon and taxonomy. The Council agrees that it is important to have a common
language and agreed-upon methods for analysis of uncertainty. However, the Council believes
that the NAS (2002) and Council (2001) reports, and various standard references cited in these
and other reports such as OMB (2003), already provide the Agency with a workable taxonomy
and a basis upon which to implement uncertainty analysis.
The Agency has suggested uncertainty analysis projects in four specific areas:
a. A pilot project to use expert judgment to better characterize the current state of
knowledge about the concentration-response function for PM induced mortality;
b. A meta-analysis of ozone mortality concentration response coefficients;
c. An attempt to characterize better the uncertainty in estimating the changes in air
pollution concentrations likely to result from emissions reductions; and
d. An investigation of uncertainty in estimates of air pollution control costs.
Based on briefings received at its November 5-6, 2003 meeting, the Council also
understands that the Agency no longer intends to undertake a study of the uncertainty in
estimates of the VSL, an additional area that was also discussed in the draft Analytical Plan.
The Council advises the Agency to develop the uncertainty analyses plans listed above
with reference to the recommendations in the above-mentioned reports. It also advises the
Agency to use the list of "key uncertainties" from the first Prospective Analysis as a framework.
The Council and its subcommittees have considered three of the four8 specific proposed
efforts for addressing uncertainty and have provided more-detailed comments on each of them
elsewhere (either in this report or in the supporting HES report). Our comments about each plan
are summarized below:
a. PM Expert Judgment Pilot Project - The Council generally supports the use of
expert judgment to inform policy analysis; commends the Agency for moving in
this direction; understands their hesitancy to move too quickly; supports the pilot
sPlans for this fourth project will be addressed by the Council's Air Quality Modeling Subcommittee when the
Agency has more details about the choice of models and the modeling protocols that would be employed.
69
-------
study; questions whether it is advantageous to use the results of the pilot study in
support of a major regulatory initiative; advises that the project be subjected to a
careful peer review; and urges the Agency to invest adequate resources, time and
managerial attention to further development of this approach so that it can be used
to inform the Third Prospective Review of the Clean Air Act. [See Advisory
Council on Clean Air Compliance Analysis, (2004) especially the HES Response
to Charge Question 29, for further detail.]
b. Ozone Mortality Meta-analysis - While a meta-analysis of ozone mortality data
may be useful, the Council does not regard the plan for uncertainty analysis on
ozone as adequate. [See Advisory Council on Clean Air Compliance Analysis,
(2004) especially the HES Response to Charge Question 30, for further detail.]
c. Control Cost Uncertainty Analysis — As discussed in more detail in sections 13.4
- 13.10, the Council believes that the focus of this project on uncertainty in
engineering cost-estimates is poorly founded and recommends greater attention to
issues such as:
1) what is left out or not counted in the cost estimates (welfare effects, process
and productivity changes);
2) uncertainty about the introduction and penetration of new technologies (e.g.,
penetration of alternative fuel vehicles);
3) economic changes (energy prices, aggregate economic activity), and
4) the extent of learning in different industries - in future efforts in this area.
See the Council response to Charge Question 27 in this report for further
detail.
Uncertainty analysis is vital to the integrity of the Prospective Analysis. Thus, the
Council Special Panel also recommends that the Agency take the following steps to strengthen
its overall approach: a) provide an explicit description or justification of the rationale underlying
the identification of these areas as the critical targets for improved characterization of
uncertainty; b) develop a strategy for using the results from these specific projects to better
characterize the extent of the uncertainty in estimates of the net benefits expected from the CAA;
and c) provide sufficient detail about the specific plans for the projects listed above to permit a
constructive critical review of the Agency's plans. The Council sees this area as a priority for
the Agency and for the advice it will provide to strengthen the 812 process.
In Chapter 9 the Agency mentions that it plans to develop an approach that "will involve
EPA experts working together to identify the major sources of uncertainty in (emissions and air
quality modeling) and then working with a combination of off-line tools and formal and informal
elicitation processes to develop a representation of uncertainty in emissions and, perhaps, key air
chemistry calculations that can be used in downstream analysis" (page 9-7). Such an "alternative
approach" to traditional deterministic benefit-cost analysis seems like an excellent objective for
the Agency, consistent with the recommendations of NAS (2002) and the September 2001
Council report. The Council Panel is not aware of detailed plans to develop this "alternative
70
-------
approach." Without further detail it is difficult for the Council to offer constructive criticism of
these plans.
During the period since the Analytical Plan and the charge questions were initially
presented to the Council, many of the activities described in Chapter 9 have been initiated and
the PM expert judgment pilot project has been completed.
The Council suggests that the Agency may wish to develop more detailed plans for its
uncertainty analysis for review by the Council in 2004, after the pilot project on PM mortality
has been completed. The Council recommends that the Agency again review the guidance and
references cited in the 2002 National Research Council report (especially chapter 5), the
September 2001 Council report, and the 2003 OMB report.
An important goal for the Second Prospective 812 Report should be the identification of
the most important uncertainties associated with the costs and benefits of air pollution, so that the
Agency can more effectively target research and improved analytical methods to reduce
uncertainties and improve the characterization of remaining uncertainties in subsequent 812
Analyses of the costs and benefits of air pollution. The Council believes that more emphasis
should be placed on identifying key uncertainties and associated research priorities.
While the Council recognizes the evolutionary nature of the Agency's development and
use of methodologies for uncertainty analysis, it is unfortunate that the text of Chapter 9 does not
contain more specific plans for identifying which are the most important factors underlying cost
and benefit uncertainties and for developing appropriate methodological approaches to
characterize such uncertainties. Uncertainty analysis should be carried out as an iterative
process, using initial characterizations of uncertainty to guide subsequent efforts to characterize
important uncertainties more precisely using available data and expert judgment.
The Council suggests that the list of "key uncertainties" from the First Prospective
Analysis (Table 9-1) could play a larger and more important role in developing the approach to
characterizing uncertainties in costs and benefits (and consequent decisions about the most
valuable allocation of scarce analytical resources). The Council hopes the guidance from its
current reports and further interaction between Agency staff and the Council in 2004, can lead to
an improved plan for characterizing these uncertainties in the most effective way for the Second
Prospective Analysis, given the constraints under which the Agency must carry out the Second
Prospective Analysis.
13.4. Summary of Council Response to Charge Question 27 Concerning the Compliance
Cost Pilot
• Just including uncertainty in engineering costs is an important improvement over the
First Prospective Analysis. The Council advises the Agency to explore uncertainty in
more than just engineering cost estimates. Other sources of cost uncertainty will also
be important and should not be neglected.
71
-------
13.5. General Discussion
The pilot project on costs described in Chapter 9 is the Agency's major new effort for
examining uncertainty with respect to costs. The proposed analysis will make an effort to
identify the key parameters of existing cost models and then attempt to quantify uncertainty
around these (primarily engineering) cost parameters. The Council sees this as a reasonable
initial approach to examining uncertainty on the cost side, especially if the cost variation is a
reflection of learning and/or technological progress that will likely occur over the 20 year
horizon of the analysis. However, the nature of the uncertainty being measured is not completely
clear from the description. In general, the Council would like to urge the Agency to be as
transparent as possible about the types of uncertainty in costs and how each is treated in the
analysis.
An exclusive focus on quantifying engineering control costs would be likely to understate
overall cost uncertainties. However, starting with uncertainties in engineering compliance costs
is natural because engineering estimates of capital and operating costs are certainly the most
visible types of costs that are directly attributable to regulatory compliance. And the very fact
that there has been little effort in the past to assess uncertainties in these probably warrants some
effort, particularly in the light of: a) the enormous effort that is going into quantification of
uncertainty on the benefits side, and b) the evidence that for certain regulatory actions the costs
have been overestimated, especially in the case of market based policies such as the sulfur
dioxide rules of the 1990 CAAA.
13.6. Sensitivity or Influence Analysis
The plan is to perform a type of sensitivity or influence analysis to determine which
parameters of the various cost models (e.g., IPM and ControlNet) have the greatest effect on
overall cost estimates. These parameters could include, for example, the coefficient on the cost
of Selective Non-Catalytic Reduction (SCNR) capital or the prices of certain precious metals
used for catalysts. The Council sees this as a reasonable way to identify the key parameters
driving costs within the cost models being used in the analysis. However, there can also be
model uncertainty - the models may not reflect how the regulations will be implemented over
time.
13.7. Other Sources of Cost Uncertainty
Although the engineering costs are a reasonable place to start looking at cost
uncertainties, the Council strongly urges the Agency to delineate all areas of cost uncertainty and
explore others in this analysis. It seems likely that considerable additional uncertainty in costs
pertains to what is left out or not counted in the cost estimates (tax interaction effects, process
and productivity changes), uncertainty about the introduction and penetration of new
technologies (e.g., penetration of alternative fuel vehicles), economic changes (energy prices,
aggregate economic activity), and the extent of learning in different industries. Some of these
may be included in the scenarios, such as the influence of uncertainty in future energy prices, but
others could be considered for future uncertainty efforts
72
-------
Indirect Costs. Another area that could be explored is the magnitude of indirect costs.
Direct environmental control costs are measured or calculated, but productivity effects, process
changes, etc,, are not included as part of these costs. There are empirical studies of these effects
that could be drawn upon to calculate distributions. For example, the non-environmental costs
increase by some expected amount as a result of the requirement to abate in an affected industry
(e.g., Morgenstern, Pizer and Shih (2001), Barbera and McConnell (1990), and others). It
should be noted that purchasing new capital equipment, which may sometimes occur as part of
modernization efforts stimulated by compliance requirements, may have positive as well as
negative influences on productivity.
Learning Assumptions. Learning effects have been documented in manufacturing. The
manufacture of more units is associated with reduced unit costs at a predictable rate as
efficiencies are realized in utilizing available equipment. Designs can also be modernized in the
light of practical operating experience. It is well worth assessing the body of experience
concerning how the increasingly widespread use of particular types of pollution control
equipment is associated with reductions in unit capital and operating costs.
One area of promise for uncertainty analysis is to allow some uncertainly around the
learning assumptions discussed in Chapter 4. There are a few empirical studies, as well as the
possibility of eliciting expert judgment about learning for different industries or processes. The
study distributed by the Agency, "Assessing the Impact of Progress and Learning Curves on
Clean Air Act Compliance Costs," (Manson et al., 2002) provides a literature review and
summary of the issue. This study suggests three reasons why costs may change over time:
learning by doing over time, innovation and technological change, and cost-reducing changes in
regulatory design.
The study focuses only on the first of these and shows some of the empirical analyses that
have been done to estimate such learning effects for scrubbers and nitrogen oxide source
reductions. In chapter 4, the draft Analytical Plan seems to be assuming an 80% rule for this
type of learning for many industries. Some quantitative uncertainty analysis around this rule,
including sensitivity analysis concerning how long the learning process persists over time, could
be done for the industries where learning is anticipated.
13.8. Compliance and Enforcement Assumptions and Consistency Requirements
In general, the costs and emissions reductions components of the uncertainty analysis
must be consistent. There is another common "80% rule" concerning practical rates of
compliance with environmental regulations that should not be confused with a similar rule
concerning learning and productivity effects. This incomplete compliance reduces costs, but is
also associated with a corresponding 20% reduction in likely benefits that would be achieved
with full compliance with the implemented rules. The cost and emissions reduction assumptions
must be consistent. To the extent that uncertainty in costs reflects uncertainty in what controls
are used or in how effectively they are used, emissions will also be affected. Compliance
assumptions are worth assessing in more detail and are well worth including as part of an overall
uncertainty analysis.
73
-------
14. DATA QUALITY AND INTERMEDIATE DATA PRODUCTS
14.1. Charge Question 32
Does the Council support the plans described in chapter 10 for evaluating the quality of
data inputs and analytical outputs associated with this study, including the planned publication of
intermediate data products and comparison of intermediate and final results with other data or
estimates? If the Council does not support these plans, are there alternative approaches,
intermediate data products, data or model comparisons, or other data quality criteria the Council
recommends? Please consider EPA's Information Quality Guidelines in this regard.
14.2. Summary of Council Response
• The validation exercises described in Chapter 10 of the Draft Plan are necessary and
appropriate, but a number of pitfalls, limitations and qualifications are noted.
• The revised Analytical Plan, by itself, is insufficiently clear about what it envisions as
"meta-data" for public dissemination. It is not necessarily raw data, but pre-
processed data that can be used to replicate intermediate results. The Agency needs
clearer guidelines concerning the type and scope of information that will be made
public during the course of the analysis and what will be provided only when the
analysis is complete.
• Preliminary release of raw data, intermediate data, intermediate models, and other
analytical components will certainly improve the transparency of the benefit-cost
exercise, but may result in substantial costs to the Agency. The Council supports
contemporaneous release along with the final Analysis (or even ex post release of
intermediate data and models) as a tool to inform future Prospective Analyses, but not
necessarily the current analysis.
• In considering the future of the Section 812 analytical process and the sharing of
intermediate data and models with outside researchers, the Agency may wish to
consider more fully some alternative mechanisms for engaging third-party researchers
in validation exercises. Peer review of requests for data or models, focused calls for
external activity, and collaboration or other formalized interactions with external
researchers might be considered. These activities may be appropriate to consider as
part of the Learning Laboratory effort discussed in this Advisory.
• The outlined activities in the Intermediate Data Products section are, in many cases,
simply too terse to permit thorough evaluation by the Council. Some examples of
useful intermediate and related data might have been suggested.
It is difficult to evaluate the Agency's plans for Intermediate Data Products with
respect to Scenario Development because the range of proposed scenarios seems still
to be evolving.
74
-------
• Obviously, consistency checking is important throughout the Analysis, not just ex
post. It is also important for the Analytical Plan to be clearer about what is to be
compared in consistency checks and how big a difference would be enough to worry
about.
• Before comparing the intermediate results of the Second Prospective Analysis with
other sources of similar information, it will be important that there be some
theoretical basis for expecting similarities. Comparisons based on the out-of-sample
extensions of models estimated in very different contexts should be subjected to
particular scrutiny.
• Along with a careful accounting of differences between the Second Prospective
Analysis and other analyses, there must be an effort to understand the most likely
sources of any differences.
• The Agency may have the resources or the authority to assemble intermediate data
that would also be valuable to other researchers but is not presently generally
available. In the process of encouraging external consistency checking, the Agency
could create public goods of great value to the external research community.
• In future Prospective Analyses, consistency checks might be expanded to include
assessments of the degree of correspondence between model predictions and other
major sources of data about economic activity, emissions profiles, and estimates of
health and ecosystem benefits.
14.3. General Advice
The Agency plans to rely upon two methods for enhancing data quality: a) publishing
detailed model outputs to expose the data to scrutiny by third parties (Intermediate Data
Products); and b) comparing certain "produced data" (e.g., model output) with counterpart real
data (Consistency Checks).
These are both good ideas and will clearly strengthen the findings of the Second
Prospective Analysis. Given the constraints faced by the Agency in meeting the schedule for
Section 812 Analyses, the Council supports these two methods. Over the longer term, however,
and looking toward future Analyses, a relevant question is whether the planned validation
exercises will continue to be sufficient. In the Council's view, these current strategies constitute
an appropriate approach to validation under time and resource constraints, but more could
potentially be done in each of these two categories in future Analyses.
The discussion that follows reflects the thoughts of Council members concerning the
general task of "validation." The Council recognizes that the term validation means something
very specific to the Agency. The Council uses the term in this report in the more general sense.
The Council does not intend that the Agency should immediately comply with all of these
suggestions. Instead, the Council's intent is to provide some reflections on the Agency's current
75
-------
strategy and where it might lead (as information technologies evolve and if sufficient resources
could be made available).
With respect to the first of the two validation approaches (i.e., publishing detailed model
outputs, termed Intermediate Data Products), many third parties will be interested in more than
just model output. One reasonable objective is to enhance confidence in the main results by
validating the computations used in various modeling components. For instance, to ascertain
whether a CGE model is producing reliable results, validation involves examining far more than
just the outputs. One needs to "look under the hood." Third parties will be interested not only in
data inputs, but in the algorithms used in intermediate calculations. For instance, abatement cost
curves may be important inputs into a cost model and their assumed or estimated nature will be
of significant relevance to validation exercises. The Council suggests that the Agency keep in
mind the broader research value of making available to outside researchers, where possible, not
just the data articulated in Figure 10-1, but the key intermediate data used in the sequence of
models and the algorithms used to process it.
The second of the two approaches: consistency checks-comparing produced data with
counterpart real data—is a great idea a priori. However, this endeavor is limited by the
availability of appropriate real data. In the case of direct costs and CGE results, it is suggested
that comparisons will be made with the PACE data. Although this is a lofty goal, it is unclear
exactly how this will be accomplished. The devil is in the details. How will data on
expenditures specifically for pollution control be compared to abatement costs under a
counterfactual scenario, let alone the data for total economic costs? In principle, this is a
worthwhile undertaking, but the Council strongly encourages that these proposed methods be
fleshed out in greater detail.
14.4. Refinements of Input Data
The Council focused its discussions of intermediate data products on scenario
development, direct cost estimation, economic valuation of benefits, and computable general
equilibrium results. It also discussed advice from the HES and the AQMS.
The Council supports the Agency's plan to make available through its web site the
intermediate information and data products produced in the course of the 812 Analysis. The
BENMAP system appears to be an invaluable tool for both generation and widespread
understanding of the analysis and its results. In particular, it will enhance understanding of the
assumptions used in constructing the aggregates of results, as well as the consequences of
alternative aggregation approaches and assumptions.
It may be helpful for the Agency to perform some other consistency checks on the air
quality from emissions and predicted population exposures in the form of calculations of
regional or national "intake fractions" (ratios of total population aggregate intake to aggregate
emissions) for pollutants that are not thought to result from secondary reactions in the
atmosphere. Finally, some comparison of predicted and observed levels of monitored pollutants
should be possible, at least for the year 2000.
76
-------
One missing element of the discussion is a plan to utilize the results of these "consistency
checks" to derive useful feedback for both the main effect estimations and the various parts of
the uncertainty analysis.
As an example on the emissions side, one important type of input into the assessment of
emissions uncertainties can be the amount of change (and the reasons for change) between older
and newer estimates of particular emissions from particular classes of sources for recent past
years. For example, one can compare previous year-2000 emissions estimates and more recent
estimates for the same or a comparable year. The following steps might be suggested for
analyzing the implications of such revisions:
a. Assess and document the changes. The material presented in Exhibit 8 (of
Chapter 2) of the Draft Analytical Plan and the accompanying text is a good start
on this process.
b. Try to understand the reasons for the changes; and what they imply about the
likely uncertainty in the revised estimates.
c. Assess the degree of "surprise" (i.e., where possible, compare the extent of each
change with the prior belief about the uncertainty in the estimate).
Historically, even in fields with well-established procedures for estimating uncertainties
(such as measurements of elementary particle masses by physicists), it is found that traditional
statistical procedures for estimating standard errors, etc., systematically understate actual
uncertainties as later calculated by comparing improved measurements with older measurements
and previously estimated uncertainties. For some examples, see Shlyakhter and Kammen
(1992), Shlyakhter (1994a, 1994b) and Hattis and Burmaster (1994).
These surprises occur because traditional statistical uncertainty estimation approaches
tend to be based solely on random sampling-error uncertainties in the data, neglecting what
frequently turn out to be appreciable systematic or calibration errors [see Shlyakhter (1994a,
1994b)]. Developing fair estimates of uncertainties for the CAAA benefit and cost projections
will require analysts to have inputs that can be interpreted in terms of both types of uncertainty.
Systematic evaluation of the extent and reasons for changes in successive sets of emissions
estimates will be a start toward providing invaluable inputs to the overall uncertainty analysis.
As an example on the health side, there is an opportunity to document the history of
changing estimates of the overall magnitude of the particle-related mortality problem, as indexed
by successively more refined measures of particle exposure—from smoke shade to total
suspended particulate to sulfate, to PM10 and now PM2.5. From the magnitude and the trends
indicated from these comparisons, experts could perhaps be led to adjust/expand their current
uncertainty estimates in the light of plausible opportunities for refining our risk assessments
further in the next decade or two—e.g., effects of still-smaller sized particles, improved
dosimeters based on particle mass deposited in specific respiratory locations, particle surface
area or particle number metrics, and particles from higher versus lower potency sources, etc.
Another suggestion is that although the text of the Analytical Plan refers to data controls,
there is considerable value in having clearly stated data quality objectives and a specific
77
-------
comprehensive data quality assurance (QA) protocol. These objectives should be derived from
the context of the 812 Analysis and should guide the design and presentation of the intermediate
data products to best serve the needs of specific audiences for the data. There are probably two
broad types of users whose differing needs should be kept in mind: a) policy and staff advisors
whose main goal may be to just understand the basis of the 812 Analysis and its conclusions, and
also b) highly sophisticated analysts who wish to do their own professional evaluations of
specific risk and benefit issues based on some of the data generated by Agency and its 812
Analysis contractors. With the needs of these two groups in mind, the disclosure and ready
availability of the intermediate data products should greatly enhance the value of the 812
Analysis for both public and private sector decision-makers.
14.5. Potential for a Learning Laboratory Approach
The Council believes that the Agency's interest in involving outside researchers in the
analysis is admirable as a guiding principle for future Prospective Analyses. The Council
considers the Learning Laboratory approach described in Section 3.3 as a productive avenue to
pursue for improving data quality by involving outside researchers in the review of important
intermediate data products.
In regard to the Learning Laboratory, the Council notes an analogy to the Stanford
Energy Modeling Forum. As discussed in Section 3.3 of this report, the Council notes that the
ongoing Section 812 Prospective Analyses represent a potentially valuable laboratory for
understanding the methods used for constructing a comprehensive benefit-cost of environmental
regulation. While it is probably not feasible for the Second Prospective Analysis, the Agency
might begin to plan for a process for evaluating the constituent models and for learning from
these evaluations. A possible approach, broached by the Council in 2001, is to examine formally
several models that purport to address the same issue. This is how the Stanford Energy
Modeling Forum (EMF) compares different models. The Agency could target key databases or
key modeling steps with specific analytical issues in mind and invite internal and external
researchers to address these issues using competing approaches.
One approach to the external validation process might be to use the project's web site to
pose specific problems and proposed solutions. Where appropriate, data and preliminary
analysis related to a particular problem could be provided to encourage involvement and
suggestions from outside experts. It might be constructive to explore the feasibility of engaging
outside researchers specifically to address mission-critical research questions. This could be
accomplished by inviting requests for original data and access to non-proprietary models so that
these outside researchers can coordinate their own, possibly regional, analytical interest with the
Agency's need for different types of validation exercises. It may be appropriate that these
requests be peer-reviewed to ensure that the costs to the Agency of compliance with such
requests represents an appropriate use of scarce Agency resources. There might be specific
opportunities for these outside researchers to identify the types of data to which they would most
like to gain access. An Agency workshop might be a suitable vehicle to bring together Agency
modeling needs and researchers with expertise in the relevant area.
The Agency's comparative advantage in assembling key data from diverse sources could
facilitate third-party research by making these data available. For example, one Council member
78
-------
has indicated that it would be desirable to provide some mechanism for requesting the data
developed in the detailed runs of air diffusion models for selected areas, such as the South Coast
Air Basin in California. This would allow researchers who are working with regional models
that have the spatial resolution to accommodate these data the opportunity to use them.
External research on issues relevant to the Second Prospective Analysis would also be
aided by availability of morbidity and mortality data at a level of spatial resolution finer than the
county-level information available in the Compressed Mortality Files from the National Center
for Health Statistics. For example, deaths from potentially air-pollution-related causes on a five-
kilometer grid scale would be greatly valuable, but individual researchers have difficulty gaining
access to this type of information.
14.6. Itemized limitations in data review
Members of the Council feel that there are some limitations in the current plans for data
review:
a. The benefits analysis information as outlined briefly in Chapter 10, page 10-2, is
inadequate. Results are described as being produced at the state level and by
pollutant-endpoint combination. The outline identifies "some of the uncertainties
inherent in projections of state-level results ten or twenty years into the future" as
the focus of likely meta-data validation exercises.
b. Detailed input information and assumptions embodied in the CGE analysis are
essential to evaluating the outputs of that analysis.
c. The Council will defer to the HES in evaluating the Agency's approach to
morbidity and mortality estimates. However, the Council encourages the Agency
to stay on top of any emerging or future opportunities to assemble health statistics
on related (actual) health conditions that might be associated with morbidity or
mortality rates due to air quality. Various prospective cohort studies may be a
valuable resource in determining disease incidence and there is a great need to
assemble all available health status databases and panels to identify the incidence
of different diseases for areas that are particularly polluted. Given the expense of
assembling these databases, the Agency should look for opportunities to make
those already assembled available for additional research and analysis.
14.7. Consistency Checks
Chapter 10 also outlines the Agency's plans for internal consistency checks. This
summary appears to treat consistency checking as something that happens after models have
been constructed and populated with the necessary parameters. In fact, calibration is a necessary
and integral feature of model development. Given the numerous assumptions and simplifications
required to build models, it is always necessary to check model performance against known,
observed values and make necessary adjustments to improve accuracy. The Council hopes that
ongoing consistency checking is standard practice in the Section 812 Analyses.
79
-------
What is to be compared in making consistency checks? Comparing one model's
predictions with another model's predictions, rather than with observational data, is more
problematic. Different models use different inputs and employ different analytical structures.
Thus it often is unclear whether prediction differences are a result of differences in the input data
or differences in the models themselves. (The Agency refers to differences in scenarios and
differences in modeling approach.) Sometimes it is possible to use one model's data with
another model's structure and vice versa to isolate the cause of the discrepancy.
Inevitably, researchers will have to cope with the question of how to resolve
inconsistencies. It often is unclear how big the inconsistencies have to be to raise concerns,
given inherent modeling uncertainties and measurement error in the data. How much of a
discrepancy is a big discrepancy? The public problem-solving procedure facilitated by publicly
available data might be useful in developing a professional consensus about how to resolve or
explain discrepancies.
The Council notes that there is actually only a modest possibility of doing consistency
checks. The Agency must keep in mind that only one of the "with" and "without" scenarios can
actually be observed. Scenarios involving recent years (e.g., 2000) allow us to observe what
happened under the "with" case. In the future, both "with" and "without" become projections.
Existing surveys such as the PACE refer to regulations that were imposed, not regulations that
are projected to be imposed. Thus, even the PACE data do not support ceterisparibus
comparisons. It is particularly difficult to do plausibility checks when two different projections
are being compared, since either projection could be questionable. In the usual context for
comparison in benefit-cost analyses, known is either a baseline or a change. That is, in the
retrospective study, one knew actual conditions and projected what happened if the Agency did
nothing further to regulate beyond 1970. In the Prospective Analyses, both the baseline and the
regulated cases are projected. Thus, there is not a known reference or baseline.
Using models to project expected quantities out-of-sample, when non-overlapping data
has been used to estimate each model, can be risky. For example, transfer of models from US
cities to Mexico City predicted so many deaths from air pollution that the number would have
amounted to between one-third and one-half of all deaths in that city, a prediction that is
implausible. The challenge lies in how to extrapolate the results of studies outside their ranges.
Linear extrapolation is clearly not reliable. Nonlinear estimation may offer improvements, but
any outside forecasting needs to be subjected to plausibility tests.
The Agency mentions several specific consistency checks. In particular, they plan to
compare BenMAP model predictions to actual incidence data. The model predicts changes
based on regulatory changes relative to the baseline scenario. The Agency notes the
inconsistency of trying to compare marginal changes with absolute levels for 2000, but suggests
no strategy for checking BenMAP predictions against observational data. Ideally, one would
look for a natural experiment where exposures changed, then replicate this change in exposure in
the context of the Section 812 models to check predicted marginal changes from these models
against observed marginal changes in the natural experiment.
80
-------
The Agency's statement about economic valuation consistency checks is similarly
ambiguous. They suggest comparing unit WTP estimates with COI values. Again, these
generally are not congruent measures. Depending on how WTP is obtained, it may only measure
pain and suffering, or it may include some components of lost productivity and cost of treatment.
Estimated COI values often include only a relatively easily observed subset of the components of
the social cost of illness. Moreover, COI estimates often rely on average wage and treatment
costs rather than marginal values. Thus the problem of comparing marginal changes with
observed averages may crop up in this context, as well.
14.8. Understanding sources of differences
A full understanding of the sources of differences in the costs and benefits between the
First and Second Prospective Analyses is critical for interpreting the results of the Second
Prospective Analysis. The Agency appears to be considering a number of possible ways to make
those comparisons. Comparison of outcomes at the most disaggregated levels is important. An
Appendix is suggested on p. 10-4 of the revised Analytical Plan. At what level of detail would
the comparison of results be provided in this Appendix?
Because this Prospective Analysis will be undertaking more disaggregated analyses, with
results by source category and even by provision in some cases, there may be possibilities to
compare the results, particularly for the 2000 time frame, to other studies that have been done.
Are the results consistent with those from other studies? There could be some attempt to suggest
what might give rise to the differences.
14.9. Intermediate outcomes and consistency checking
Any component of the Section 812 Prospective Analyses that leads up to the calculation
of final costs and benefits is an "intermediate product" of the analysis. Many of these
intermediate products summarize relationships that are used to reach the eventual benefit and
cost calculations. These estimated or assumed relationships afford many opportunities for
benchmarking the analysis against other studies or against real data. For example, there may be
future opportunities to examine the incidence of lung disease by industrial sector for workers, or
lung disease against census tracts or zip codes for place of residence. Morbidity information is
naturally more difficult to pin down than mortality, since most illnesses are not reportable,
whereas the causes of death are. However, assembling whatever information is available on
morbidity stemming from air-quality-related disease could be extremely valuable. Public
perceptions of air-quality-related health risks will influence the perceived benefits of air quality
management and thus individual WTP the costs incurred due to regulation.
14.10. Additional specific recommendations
If not for the current analysis, then potentially for future analyses, the Council suggests
that some of the following activities might be considered as candidates for addition to the
Agency's consistency-checking regimen:
a. There does not appear to be a plan to make public the economic projections
underlying the emissions estimates and to reference these emissions estimates to
81
-------
actual levels of economic activity in sectoral, regional, or aggregate terms. Levels
of economic activity are critically important determinants of emissions and it will
be important for these assumptions to be scrutinized as the Agency moves into
producing subsequent Prospective Analyses.
b. Results at the state level and by pollutant-endpoint combination should be
matched to other economic data at the same spatial resolution to offer future
opportunities for cross checks. For example, there should be adequate
consideration of Census economic information on household income.
c. There might be comparisons of the assumptions about future economic activity
embodied in the Second Prospective Analysis to actual levels of economic
activity by sector and region in actual years covered and with independent
national projects. For example, this task could employ regional Federal Reserve
Bank statistics and forecasts, or forecasts prepared by other federal sources.
d. The analysis might include more-explicit consideration of time profiles of
concentrations prior to 2000 (actual ambient readings) in comparison to the levels
and time profiles projected for future policy effects.
e. There might be more attention in future analyses to the morbidity states that may
precede mortality outcomes. What do the available epidemiological results
suggest for the incidence of new serious lung and heart conditions? Whether or
not these can be proven to be related to air quality, they can influence public
perceptions concerning the urgency of air quality management.
f. The analysis might be accompanied by comparison of benefits estimates to
household income and to WTP estimates for air quality improvements from
current hedonic or random utility models for specific areas. This practice has
historical precedents and can be used as a gauge of plausibility for the benefits
estimates incorporated in the analysis.
82
-------
15. RESULTS AGGREGATION AND REPORTING
15.1. Charge Question 33
Does the Council support the plans described in Chapter 11 for the aggregation and
presentation of analytical results from this study? If the Council does not support these plans,
are there alternative approaches, aggregation methods, results presentation techniques, or other
tools the Council recommends?
15.2. Summary of Council Response
• Reporting of central and alternative cases should be associated with likelihoods of these
cases and any provision of a "low" alternative estimate should be balanced by a
corresponding "high" alternative estimate. Pivotal assumptions should be clearly
identified and the need for additional research on these issues should be emphasized.
• The Council urges the Agency to dispense with benefit-cost ratios and focus attention
on net benefits estimates as the appropriate summary measure in benefit-cost analysis.
• The Council understands the Agency's current reluctance to take the somewhat heroic
steps necessary to process the time profiles of benefits and costs into net present value
(NPV) estimates. However, the Council urges the Agency to persist in its efforts
toward this important goal in planning for future Analyses. In the meantime, the
Agency must more clearly explain its rationale for annualizing costs but not
calculating present discounted values of net benefits.
• As problematic as disaggregation may be, the Agency should anticipate strong demand
for this type of information by policy-makers and stakeholders.
• There is insufficient information in Chapter 11 to permit a thorough review of the
Agency's plans to disaggregate net benefits by sector.
• Spatial disaggregation is problematic, in general, because of all the connections among
markets that give rise to general equilibrium consequences from the regulation of any
one plant or industry. The Agency is advised to proceed very cautiously in terms of
spatial disaggregation and only in special cases.
• A more through explanation of the inadvisability of further disaggregation by title of
the CAAA would help readers understand why no such further disaggregation is
planned.
15.3. General Observations
83
-------
The Council's discussion of this Charge Question was separated rather artificially into a
segment on costs and a separate segment on benefits. In this write-up, elements of the discussion
that are relevant to both topics have been combined.
The Council notes that the strategy of reporting a "primary" estimate and an "alternative"
can be misleading to the public if the alternative estimate combines conservative assumptions on
several dimensions and results in a "low" estimate of net benefits. At the very least, if a "low"
alternative is offered, so should be a "high" alternative, so readers are not left with the
impression that the "true" case is half-way between the primary estimate and the low alternative.
Providing only a low alternative invites biased inferences. Computational challenges preclude a
full continuous distribution for the range of possible outcomes, for which standard confidence
intervals could be constructed. However, information about the full distribution of possible
results should be a goal to which the Agency aspires.
If the Agency continues to present sensitivity analyses concerning alternative scenarios, it
is essential to associate with each of these alternatives some sense of their relative likelihood.
Failure to do so encourages readers to employ a uniform distribution, which is almost certainly
inappropriate.
Even at the intermediate data level, there should be more effort to explain how
probability weights will be used to combine alternative point estimates of the magnitudes of key
relationships. For example, with the ozone/mortality association, suppose there are three
credible estimates. If all three estimates are close, then their average could be used. But what if
one estimate is very different? The Second Prospective Analysis central case will presumably
use the "best estimate" of this relationship. How will that value be determined?
In reporting its main results, the Council encourages the Agency to give particular
prominence to the key assumptions and methodological choices that may be driving the results.
Clear identification of these pivotal aspects of the analysis will emphasize the need for additional
research on these topics and help focus the research community upon finding solutions.
15.4. Primary Results
The revised Draft Analytical Plan proposes some changes relative to procedures used in
the first Prospective Analysis. For example, the Agency acknowledges previous SAB comments
about reporting benefit-cost (B/C) ratios. They plan to report B/C ratios in this study, but de-
emphasize them relative to net-benefit estimates. The role of "appropriate explanation" is
important to help readers avoid well-known problems with using B/C ratios for decision making.
However, the Council does not favor ANY use of benefit-cost ratios. This concept does
not have a consistent economic interpretation. Consequently, these ratios do not offer new
information. If there is a concern that some portion of the constituency for the analysis will be
more comfortable thinking in terms of benefit-cost ratios, the calculated benefit-cost ratio should
be no more prominent than being mentioned in a footnote. The Agency should take a lead in
shifting the emphasis to net benefits information, as opposed to benefit-cost ratios. If benefit-
cost ratios are introduced at all, they should be qualified carefully.
84
-------
It is true that any policy or project with positive net benefits will also have a benefit-cost
ratio greater than one, if both benefits and costs were known with certainty. However, in ranking
projects with net benefits greater than zero (or less than zero) the net benefits and benefit-cost
criteria can give conflicting rankings. Also, given greater attention to uncertainty, the net
benefits approach has much to recommend it. The variance of a difference in two random
variables is generally easier to calculate than the distribution of a ratio of two random variables.
An emphasis on benefit-cost ratios would require consideration of how the variance in the ratio
of two random variables (uncertain benefits over uncertain costs) was derived. There are
approaches (e.g., Goodman and Hartley (1958), Goodman (1960, 1962), and Bohrnstedt and
Goldberger, 1969) but this seems to add needless complexity.
15.5. Future forecasts and present value calculations
In the Second Prospective Analysis, the cumulative or present discounted value of costs,
benefits, and net benefits will not be presented. The reason given in the Draft Analytical Plan is
that the time paths of costs and benefits are not linear. An example provided is which there may
be high up-front costs, with benefits in later years Analogous problems can afflict benefits
estimates, since multi-period chronic health effects must also be taken into account.
Part of this problem is dealt with, implicitly, in the so-called "annual" estimates. For
example, the annual costs in each reported year (2000, 2010, and 2020) are average annual costs.
If there are up-front capital costs, these are annualized (capitalized forward using an assumed
interest rate) to get the annual estimates for the target years. The Council accepts the Agency's
plans, for the Second Prospective Analysis, not to report cumulative estimates in the form of
present discounted values, but recommends that the nature of the annual estimates should be
made clearer and they should be called "forecasted average annualized costs and benefits."
The Analytical Plan states that changing the discount rate will have little effect on the
results, because no net present value estimates are calculated. However, changing the discount
rate does affect the annualized results in various ways, including the cost estimates if capital
costs have been capitalized forwards to produce estimates of average annual costs. The Plan
should be clearer about the specific interest rates used to annualize the costs of firms (where
private rates may sometimes influence individual firms' predicted behavior but social rates
should in general be used for collective decision-making), as opposed to the appropriate social
discount rates needed to compute the present value of net benefits.
Some members of the Council agree with the proposal to delete discussion of the
approximate present value of net benefits given the current quality of the components available
to calculate it. The practices that will be used to estimate the time profiles of costs and benefits
(in particular, the lack of good techniques for interpolation between discrete forecasting years)
make these time profiles difficult to rely upon. Further effort to calculate present values would
not really be justified on the basis of the underlying quality of these time profiles. Any present
value calculations would exaggerate the precision with which these time profiles can be
calculated.
85
-------
Nevertheless, other members of the Council express considerable unease about the fact
that present discounted net benefits are, in principle, the criterion upon which judgments are
based (prior to the introduction of distributional considerations). When benefits and costs are
distributed unevenly over time, it is necessary to determine whether overall present discounted
net benefits are positive. By neglecting NPV calculations, the Analysis does not provide what is
needed to inform policy-makers.
The Council is troubled by the Agency's explanation that it has decided not to provide
annual interpolations of net-benefit estimates between target years because of the difficulty of
quantifying uncertainties related to interpolation. Different strategies for interpolation could be
used and the sensitivity of the NPV calculations to these differences could be assessed. If the
Agency reports carefully upon the methods used to fill in the intervening years (latency of
benefits, durability of costs), then the resulting NPV calculations would be suitably qualified.
The Agency explained to the Council that the exorbitant data requirements for air quality
modeling for the intervening years in the main forecasts were the rate-determining factor in
filling in trajectories of costs and benefits for intervening years over the forecasting horizon.
However, there would seem to be some prospect of improving upon simple linear interpolation
by taking advantage of the richness of emissions trends. The Council urges the Agency to
continue to grapple with possible alternative techniques for interpolating the disparate time
patterns of benefits and costs and working towards plausible NPV results in future Prospective
Analyses.
15.6. Disaggregation
Chapter 11 of the revised Analytical Plan is advertised to concern "Results Aggregation
and Reporting," although its subject matter could more informatively be termed "Results
Disaggregation and Reporting." The central issue is the extent to which costs and/or benefits
should be disaggregated spatially (e.g., by state), by CAAA Title, or by sector.
The Agency notes some potential problems with sectoral and spatial disaggregation,
attributed to factors such as nonlinearities, jointness, and incidence dispersion through related
markets. These problems can result in subadditivity or superadditivity when aggregating up
from component estimates or disaggregating down from total estimates. However, because
sectoral and geographic incidence is of considerable interest to policy makers, it may be
necessary to plan for adding evaluation of alternative (at least partial) disaggregation schemes to
the already long list of sensitivity and uncertainty analyses that this study, or perhaps future
Prospective Analyses, will require.
Any attempts at sectoral decomposition of benefits and costs must be compared and
reconciled with sectoral analyses from the CGE models to be used in this enterprise.
Explanations for any anticipated or realized discrepancies between sectoral and aggregated
analyses should be clarified. The current description refers to "non-linearities" as the source of
potential discrepancies, but this explanation needs to be clearer. In the discussion of sectoral
reporting, it is not clear what sectoral breakdown will be used.
86
-------
The Council, in its previous review, argued strongly against spatial disaggregation of the
costs of the CAAA. The general equilibrium consequences of air quality interventions are
propagated widely throughout the economy, acting as they do through goods markets, labor
markets, and capital markets. In its 2001 review, due to these issues of incidence, the Council
advised against spatial disaggregation of costs. The Analytical Plan adopts that suggestion with
a nicely phrased argument and explanation.
However, some types of air quality regulations that affect only local or regional air
quality, rather than broader areas, may have sufficiently localized benefits that it is reasonable to
address spatially disaggregated benefits estimates. Stratospheric ozone concentrations or the
effect of carbon emissions on world climate clearly do not fall into this category. Spatial
disaggregation of benefits should be contemplated only when the Agency has access to spatially
delineated projections for ambient concentrations of pollution. This could offer opportunity for
local or regional estimates of benefits derived from hedonic property value and hedonic wage
studies.
Although there are many regulations for which it makes no sense to spatially
disaggregate costs, for the general equilibrium reasons already mentioned, there may still be a
few exceptions. It must be acknowledged that there will occasionally be vocal demands for
spatial disaggregation by policy makers. It may be important for the Agency to anticipate
demands that it examine costs and benefits by geographical area for some provisions of the
CAAA, for some sources, but only where costs and benefits are sufficiently localized for the
exercise to be meaningful.
For example, additional local controls to meet NAAQS may have costs and benefits that
are borne primarily, although not entirely, within the region. Certain future policies may make
sense in some regions and not in others. State-by-state costs and benefits probably will not
capture the right geographic areas, but it seems important to consider regional disaggregation for
some cases.
Even judicious spatial disaggregation of benefits is not without potential complications,
however. The example in the Plan of the geographic dispersion of cost incidence from power
plant emission-control investments in Indiana may also apply to benefits in a general-equilibrium
analysis. Improved health that enhances worker productivity may benefit a firm's shareholders
and customers in distant locations. The Agency's example of how to allocate visibility benefits
accruing to visitors to a national park is a good illustration of where problems may arise. The
physical improvement occurs at the national park, but the beneficiaries are park visitors who live
elsewhere. Should their benefits be associated with the location of the park, or the location of
their residence? In many cases, geographic disaggregation will involve arbitrary judgments that
may be difficult to defend. Fortunately, these are rather minor examples. By far the largest
share of measured total benefits from the CAAA appears to stem from human health
improvements that can be captured fairly reliably at the census tract level.
The Council also urged previously that the Agency should pursue disaggregating costs by
Title. Although this is not explicitly treated in the text of Chapter 11, Table 11-2 suggests that
costs will be aggregated over Titles I through IV. The Council would a priori prefer more
87
-------
disaggregation by Title and suggests that the Plan present reasons why this is not possible or
desirable. The 2001 Council review of the first Draft Analytical Plan clarified some of the
reasons for limiting disaggregation by title, but too few of these reasons appear in the revised
Draft Analytical Plan.
The Analytical Plan focuses on monetized benefits and costs. Chapter 11 does not
describe any planned reporting of cost-effectiveness measures in the Second Prospective
Analysis. The First Prospective Analysis provided some auxiliary cost-per-life-saved measures.
Given that the results from the Second Prospective Analysis are to be calculated and reported on
a more disaggregated basis, there may be some cases where these cost-effectiveness estimates
can be provided and would be helpful to the constituency's understanding of the effects of the
CAAA. The Council acknowledges, however, that when policies provide benefits that are
broader than simply improvements in human health, cost-per-life-saved measures can be
misleading (e.g., when there may be substantial ecosystem benefits). This issue received
attention in the earlier section on QALY-based cost effectiveness.
-------
REFERENCES
Argote, L, and D. Epple. (1990) "Learning curves in manufacturing," Science 247(4945): 920-
924.
Arrow, K. I, W. R. Cline, K.-G. Maler, M. Munasinghe,R. Squitieri, and J. E. Stiglitz (1996)
"Intertemporal Equity, Discounting, and Economic Efficiency." In J. P. Bruce, H. Lee,
and E. R. Haites (eds.) Climate Change 1995: Economic and Social Dimensions of
Climate Change. Cambridge, U.K., New York and Melbourne: Cambridge University
Press, Chapter 4, pp. 125-44.
Auerswald, P, S. Kauffman, J. Lobo, and K. Shell (2000) "The production recipes approach to
modeling technological innovation: An application to learning by doing" Journal of
Economic Dynamics and Control 24(3): 389-450.
Barbera, AJ. and V.D. McConnell (1990) "The Impact of Environmental Regulations on
Industry Productivity: Direct and Indirect Effects" Journal of Environmental Economics
and Management 18: 50-65.
Bohrnstedt, George W. and Goldberger, Arthur S. (1969) "On the exact covariance of products
of random variables," Journal of the American Statistical Association 64: 1439-1442.
Beron, K, J. Murdoch, and M. Thayer (2001) "The benefits of visibility improvement: New
evidence from the Los Angeles Metropolitan Area," Journal of Real Estate Finance and
Economics 22(2-3): 319-337.
Bovenberg, A. Lans, and Lawrence H. Goulder. (1997) "Costs of Environmentally Motivated
Taxes in the Presence of Other Taxes: General Equilibrium Analyses," National Tax
Journal 50 (1)
Chapman, K.S. and Hariharan, G. (1994) "Controlling for Causality in the Link from Income to
Mortality," Journal of Risk and Uncertainty 8(1): 85-94.
Chapman, K.S. and Hariharan, G. (1994) "Controlling for Causality in the Link from Income to
Mortality," (VOL 8, PG 85, 1994, correction) Journal of Risk and Uncertainty 8(3): 320-
320.
Chapman, K.S. and Hariharan, G. (1996) "Do Poor People Have a Stronger Relationship
between Income and Mortality than the Rich? Implications of Panel Data for Health-
Health Analysis," Journal of Risk and Uncertainty 12: 51-63.
Chestnut, L.G., and R.D. Rowe. (1990a) "Economic Valuation of Change in Visibility: A State
of the Science Assessment for NAPAP," Acidic Deposition: State of the Science and
Technology Report 27; Brown, G.M Jr., Callaway, J.M., Eds.; National Acid
Precipitation Program: Washington DC.
Chestnut, L.G., and R.D. Rowe. (1990b) "New National Park Visibility Value Estimates." In
Visibility and Fine Particles, Transactions ofanAWMA/EPA International Specialty
Conference, C.V. Mathai, ed. Air and Waste Management Association, Pittsburgh.
89
-------
Chestnut, L.G. and R.D. Rowe. (1990c) "Preservation Values for Visibility Protection at the
National Parks." Cooperative Agreement #CR-813-686, U.S. Environmental Protection
Agency: Research Triangle, North Carolina.
Clancy, L., Goodman, P., Sinclair, H., Dockery, D.W. (2002) "Effect of air pollution control on
death rates in Dublin, Ireland: An intervention study." Lancet 360: 1210-1214.
Clemen, R.T. (1996) Making Hard Decisions: An Introduction to Decision Analysis. Duxbury
Press.
Costanza R, d'Arge R, de Groot R, Farber S, Grasso M, Hannon B, Limburg K, Naeem S,
O'Neill RV, Paruelo J, Raskin RG, Sutton P, van den Belt M. (1998) "The value of the
world's ecosystem services and natural capital" (Reprinted from Nature, vol 387, pg 253,
1997) Ecological Economics 25(1): 3-15.
Cropper, M.L., and AJ. Krupnick (1990) "The Social Costs of Chronic Heart and Lung
Disease." (2000) in Valuing Environmental Benefits: Selected Essays of Maureen
Cropper, Maureen Cropper, ed. (Edward Elgar; United Kingdom).
Desvousges, W. H., F.R. Johnson, andH.S. Banzhaf (1998) Environmental Policy Analysis with
Limited Information Cheltenham, UK: Edward Elgar.
Diamond, P. A., and J. A. Mirrlees (1971) "Optimal Taxation and Public Production I:
Production Efficiency. American Economic Review 61: 8-27.
Dickie, M., and V.L. Ulery (2002) "Parental Altruism and the Value of Child Health: Are Kids
Worth More Than Parents?" Forthcoming, Journal of Environmental Economics and
Management.
Diewert, W.E. (1993) Essays in Index Number Theory. Elsevier Science.
ENVIRON International Corporation (2003) Audit of the Colorado AIR Program, June, Novato,
CA, (www.state.co.us/gov dir/audit dir/2004/2004 perf/1509.pdf)
EPA Advisory Council on Clean Air Compliance Analysis (2001) Review of the Draft
Analytical Plan for EPA's Second Prospective Analysis - Benefits and Costs of the Clean
Air Act, 1990-2020: An Advisory by the Advisory Council for Clean Air Compliance
Analysis. EPA-SAB-COUNCIL-ADV-01-004.
EPA Advisory Council on Clean Air Compliance Analysis (2004a) "Advisory on Plans for
Emissions Estimation Presented in the May 12, 2003 Analytical Plan: An Advisory by
the Air Quality Modeling Subcommittee of the Advisory Council for Clean Air
Compliance Analysis," EPA-SAB-COUNCIL-04-001)
EPA Advisory Council on Clean Air Compliance Analysis (2004b) "Advisory on Plans for
Health Effects Analysis in the Analytical Plan for EPA's Second Prospective Analysis -
Benefits and Costs of the Clean Air Act, 1990-2020: An Advisory by the Health Effects
Subcommittee of the Advisory Council for Clean Air Compliance Analysis," EPA-SAB-
COUNCIL-ADV-04-002.
EPA Science Advisory Board (2000) An SAB Report on EPA's White Paper Valuing the
Benefits of Fatal Cancer Risk Reduction. EPA-SAB-EEAC-00-013.
90
-------
Espinosa, J.A. and V. K. Smith (1995) "Measuring the Environmental Consequences of Trade
Policy: A Non-Market CGE Analysis," American Journal of Agricultural Economics
77(3): 772-775.
Espinosa, J.A. and V.K. Smith (2002) "Environmental Levies with Non-Separable Damages:
Taxing Air Pollution in the EU Economies," CEnREP Working Paper September.
Finnoff, D., and J. Tschirhart (2003) "Protecting an endangered species while harvesting its prey
in a general equilibrium ecosystem model," Land Economics 79(2): 160-180.
Freeman, A.M. Ill (1992) The measurement of environmental and resource values: theory and
methods. Washington, D.C.: Resources for the Future.
Friedman, M.S., K.E. Powell, L. Hutwagner, L.M. Graham, and W.G. Teague (2001) "Impact of
Changes in Transportation and Commuting Behaviors During the 1996 Summer Olympic
Games in Atlanta on Air Quality and Childhood Asthma." Journal of the American
Medical Association 285(7): 897-905.
Fullerton, Don and Gilbert Metcalf (2002). "Cap and Trade Policies in the Presence of Monopoly
and Distortionary Taxation," Resource and Energy Economics 24:327-347.
Gold, M.R., Seigel, J.D., Russell, L.B. and Weinstein, M.C. (1996) Cost Effectiveness in Health
and Medicine., New York: Oxford University Press.
Goodman, Leo A. (1960) "On the exact variance of products," Journal of the American
Statistical Association 55: 708-713 (Correction: Volume 57: 917).
Goodman, Leo A. and Hartley, H.O. (1958) "The precision of unbiased ratio-type products,"
Journal of the American Statistical Association 53: 491 - 508 (Corrections: Volume 58:
1162; Volume 64; 1700)
Goodman, Leo A. (1962) "The variance of the product of k random variables," Journal of the
American Statistical Association 57: 54-60.
Goulder, L. H., I. W. H. Parry, and D. Burtraw. (1997) "Revenue-Raising vs. Other Approaches
to Environmental Protection: The Critical Significance of Pre-Existing Tax Distortions,"
RAND Journal of Economics Winter.
Goulder, L. H., and Roberton C. Williams III. (2003) "The Substantial Bias from Ignoring
General Equilibrium Effects in Estimating Excess Burden, and a Practical Solution,"
Journal of Political Economy., August.
Gray, W.B. and RJ. Shadbegian. (1994) "Pollution Abatement Costs, Regulation, and Plant-
Level Productivity," Center for Economic Studies discussion paper, U.S. Department of
Commerce.
Gyrd-Hansen, D. (2003) "Willingness to pay for a QALY." Health Economics 12: 1049-1060.
Hammitt, J. K. (2002) "QALYs versus WTP," Risk Analysis 22 (5): 985-1001.
Hammitt, J.K. (2000a) "Evaluating Contingent Valuation of Environmental Health Risks: The
Proportionality Test," Association of Environmental and Resource Economists
Newsletter 20(1): 14-19.
Hammitt, J.K (2000b) "Evaluating Risk Communication: In Search of a Gold Standard," 15-19
in M.P. Cottam, D.W. Harvey, R.P. Pape, and J. Tait (eds.), Foresight and Precaution
91
-------
(Proceedings of ESREL 2000, SARS and SRA-Europe Annual Conference), A.A.
Balkema, Rotterdam.
Hammitt, J.K., Graham, J.D. (1999) "Willingness to pay for health protection: Inadequate
sensitivity to probability?" Journal of Risk and Uncertainty 18(1): 33-62.
Harrington, W., R.D. Morgenstern, and P. Nelson (2000) "On the Accuracy of Regulatory Cost
Estimates," Journal of Policy Analysis and Management 19(2): 297-322.
Harrison, G. W., M. I. Lau, and M. B. Williams (2002) "Estimating individual discount rates in
Denmark: Afield experiment" American Economic Review 92: 1606-1617.
Hattis, D. and Burmaster, D. E. (1994) "Assessment of Variability and Uncertainty Distributions
for Practical Risk Analyses" Risk Analysis 14: 713-730.
Hedley, A.J., Wong, C.M., Thach, T.Q., Ma, S., Lam, T.H., Anderson, H.R (2002)
"Cardiorespiratory and all-cause mortality after restrictions on sulphur content of fuel in
Hong Kong: An intervention study." Lancet 360: 1646-1652.
Hirth, R. A., et al. (2000) "Willingness to pay for a quality-adjusted life year: In search of a
standard." Medical Decision Making 20: 332-342.
Howard, R.A. (1984) "On Fates Comparable to Death," Management Science 30: 407-42.
Howard, Ronald A. and James E. Matheson (1984) "Influence diagrams," In Howard, R. and
Matheson, J., editors, Readings on the Principles and Applications of Decision Analysis,
volume II: 721-762. Strategic Decision Group, Menlo Park, CA.
Johnson, F.R., M.R. Banzhaf, and W.H. Desvousges (2000) "Willingness to Pay for Improved
Respiratory and Cardiovascular Health: A Multiple-Format Stated-Preference Approach."
Health Economics 9 295-317.
Johnson, F.R., J. Mauskopf, and A.B. Hauber (2004) "Willingness to Pay, Willingness to Wait,
and Super QALYs: Narrowing the Gap Between Ordinal and Cardinal Health Utility
Measures," RTI-HS Working Paper., Research Triangle Institute.
Jones-Lee, M.W., M. Hammerton, and P.R. Philips (1985) "The Value of Safety: Results of a
National Sample Survey," The Economic Journal 95: 49-72
Joshi, S., L. Lave, S. Shih, and F. McMichael (1997) "Impact of Environmental Regulations on
the U.S. Steel Industry," Carnegie Mellon University mimeograph.
Keeney, R.L. and D. von Winterfeldt (1986) "Why Indirect Health Risks of Regulations Should
be Examined," Interfaces 16(6): 13-27.
Keeney, R.L. (1990) "Mortality Risks Due to Economic Expenditures," Risk Analysis 10(1) 147-
159.
Keeney, R.L. (1994) "Mortality Risks Induced by the Costs of Regulations," Journal of Risk and
Uncertainty 8(1): 95-110.
Keeney, R.L. (1997) "Estimating Fatalities Induced by the Economic Costs of Regulations,"
Journal of Risk and Uncertainty 14(1): 5-23.
92
-------
Klein, D.E. and R.L. Keeney (2002) "Mortality Reductions from the Use of Low-Cost Coal-
Fueled Power: An Analytical Framework" report available at
http://www.ceednet.org/kkhealth/download/Mortality%20Reductions.pdf
Klose, T. (2003) "A utility-theoretic model for QALYs and willingness to pay." Health
Economics 12: 17-31.
Kristrom, B. and P. Riera (1996) "Is the Income Elasticity of Environmental Improvements Less
Than One?," Journal of Environmental and Resource Economics 7: 45-55.
Leggett, C. G., and N. E. Bockstael (2000) "Evidence of the effects of water quality on
residential land prices," Journal of Environmental Economics and Management 39(2):
121-144.
Lind, R. C. (1982) "A Primer on the Major Issues Relating to the Discount Rate for Evaluating
National Energy Options," in R. C. Lind, et al., Discounting for Time and Risk in Energy
Policy. Washington, D.C.: Resources for the Future/Johns Hopkins University Press.
Lind, R.C. (1990) "Reassessing the Government's Discount Rate Policy in Light of New Theory
and Data in a World Economy with a High Degree of Capital Mobility," Journal of
Environmental Economics and Management 18: S-8 - S-25.
Lindahl, M. (2002) "Estimating the Effect of Income on Health and Mortality Using Lottery
Prizes as Exogenous Source of Variation in Income," 1ZA Discussion Paper No. 442,
IZA (Institute for the Study of Labor), Bonn, Germany.
Lutter, R., Morrall, J.F. Ill, and Viscusi, W.K. (1999) "The Cost-per-Life-Saved Cutoff in
Safety -Enhancing Regulations," Economic Inquiry 37(4): 599-608.
Manson, CJ. M.B. Nelson, and I.E. Neumann (2002) "Assessing the Impact of Progress and
Learning Curves on Clean Air Act Compliance Costs" manuscript, Industrial Economics,
Incorporated, Cambridge, MA (July 12) 29 pp.
McConnell, K. E. (1990) "Double Counting in Hedonic and Travel Cost Models," Land
Economics 66(2): 121-127.
Miller, T. R. (2000) "Variations between countries in values of statistical life," Journal of
Transport Economics and Policy, 34: 169-188.
Miller, Ted and Jagadish Guria (1991) The Value of Statistical Life in New Zealand: Market
Research on Road Safety. Report for the Land Transport Division.
Morgan, M. Granger and Max Henri on (1990) Uncertainty: A Guide to Dealing with Uncertainty
in Quantitative Risk and Policy Analysis. Cambridge University Press, Cambridge.
Morgenstern, R.D., W.A. Pizer and J.-S. Shih (2001) "The Cost of Environmental Protection,"
Review of Economics and Statistics 83(4): 732-738.
Mrozek, J. R., and L. O. Taylor (2002) "What determines the value of life? A meta-analysis,"
Journal of Policy Analysis and Management 21(2): 253-270.
National Research Council (2002) Estimating the Public Health Benefits of Proposed Air
Pollution Regulations, National Academy of Science, Board on Environmental Studies
and Toxicology. National Academy Press, Washington, DC.
93
-------
National Research Council (1994) Science and Judgment in Risk Assessment, National Academy
of Sciences, Commission on Life Sciences, National Academies Press.
Office of Management and Budget (2003) Informing Regulatory Decisions: 2003 Report to
Congress on the Costs and Benefits of Federal Regulations and Unfunded Mandates on
State, Local, and Tribal Entities, Office of Information and Regulatory Affairs.
Parry, Ian W. H. (1995) "Pollution Taxes and Revenue Recycling," Journal of Environmental
Economics and Management 29, S64-S77.
Parry, Ian W. H, R. C. Williams III, and L. Goulder. (1999) "When Can CO2 Abatement
Policies Increase Welfare? The Fundamental Role of Pre-Existing Factor Market
Distortions," Journal of Environmental Economics and Management 37:52-84.
Parsons, G. R. (1991) "A Note on Choice of Residential Location in Travel Cost Demand
Models," Land Economics 67(3): 360-364.
Persson, U., A. Lugner Norinder, and M. Svensson. (1995) "Valuing the Benefits of Reducing
the Risk of Non-Fatal Roads Injuries: The Swedish Experience," in Schwab Christe, N G,
and Soguel, N C (eds) Contingent Valuation, Transport Safety and the Value of Life,
Boston: Kluwer, 63-83.
Peretto, P, and V.K. Smith (2001) "Carbon Policy and Technical Change: Market Structure,
Increasing Returns, and Secondary Benefits," DOE Final Report, Grant number DE-
FG02-97ER62504.
Pope, C.A III (1989) "Respiratory disease associated with community air pollution and a steel
mill, Utah Valley." American Journal of Public Health 79: 623-628.
Pope, C.A. Ill (1991) "Respiratory hospital admissions associated with PM10 pollution in Utah,
Salt Lake, and Cache Valleys," Archives of Environmental Health 46: 90-97.
Ramsey, Frank (1928) "A Mathematical Theory of Saving," Economic Journal, 38: pp 543-559.
Rowe, R.D., and L.G. Chestnut (1985) "Oxidants and Asthmatics in Los Angeles: A Benefits
Analysis." In: S.D. Lee, ed., Evaluation of the Scientific Basis for Ozone/Oxidants
Standards. Pittsburgh, PA: Air Pollution Control Association.
Ruhm, CJ. (2000) "Are Recessions Good for Your Health?" Quarterly Journal of Economics
115(2): 617-650.
Ruhm, CJ. (2003) "Good times make you sick," Journal of Health Economics, 22(4): 637-658.
Sanchirico, J. N., and J. E. Wilen (2001) "A bioeconomic model of marine reserve creation,"
Journal of Environmental Economics and Management, 42(3).
Schob, Ronnie. (1997) "Environmental Taxes and Pre-Existing Distortions: The Normalization
Trap," International Tax and Public Finance 4:167-176
Schwartz J, and R. Repetto (2000) "Nonseparable utility and the double dividend debate:
Reconsidering the tax-interaction effect," Environmental and Resource Economics 15(2):
149-157.
Shlyakhter, A. L, and Kammen, D. M. (1992) "Sea-Level Rise or Fall?" Nature 253:25.
94
-------
Shlyakhter, A. (1994a) "An Improved Framework for Uncertainty Analysis: Accounting for
Unsuspected Errors," Risk Analysis 14: 441-447.
Shlyakhter, A. (1994b) "Uncertainty Estimates in Scientific Models: Lessons from Trends in
Physical Measurements, Population and Energy Projections," In Uncertainty Modeling
and Analysis: Theory and Application, B. M. Ayyub and M. M. Gupta, Eds., Elsevier
Science, B. V., pp. 477-496.
Smith, J.P. (1999) "Healthy Bodies and Thick Wallets: The Direct Relations Between Health
and Economic Status,'" Journal of Economic Perspectives 13(2): 145-166.
Smith, M. D. (2003) "The economics of conserving wildlife and natural areas," Australian
Journal of Agricultural and Resource Economics 47(3): 418-420.
Smith, V.K., and J.A. Espinosa (1996) "Environmental and Trade Policies: Some
Methodological Lessons," Environment and Development Economics (February);
reprinted in A.A. Batabyal and H. Beladi, editors, The Economics of International Trade
and the Environment (New York: Lewis Publishers, 2001).
Smith, V.K., and Osborne, L.L. (1996) "Do Contingent Valuation Estimates Pass a "Scope"
Test? A Meta analysis." Journal of Environmental Economics and Management 31:
287-301.
Smith, V. K., S. K. Pattanayak, and G. L. Van Houtven (2003) "VSL reconsidered: what do
labor supply estimates reveal about risk preferences?" Economics Letters 80(2): 147-
153.
Smith, V. K., G. Van Houtven, and S. K. Pattanayak (2002) "Benefit transfer via preference
calibration: "Prudential algebra" for policy," Land Economics 78(1): 132-152.
Solow, R. M. (1974) "The Economics of Resources or the Resources of Economics." American
Economic Review 64: 1-14.
Taylor, L.O., and V.K. Smith. (2000) "Environmental amenities as a source of market power,"
Land Economics 76(4): 550-568.
Viscusi, W.K. (1994) "Mortality Effects of Regulatory Costs and Policy Evaluation Criteria,"
RAND Journal of Economics 25( 1): 94-109.
Viscusi, W. K., and J. E. Aldy (2003) "The value of a statistical life: A critical review of market
estimates throughout the world," Journal of Risk and Uncertainty 27(1): 5-76.
Viscusi, W.K., W.A. Magat, and J. Huber (1991) "Pricing Environmental Health Risks: Survey
Assessments of Risk-Risk and Risk-Dollar Trade-Offs for Chronic Bronchitis." Journal
of Environmental Economics and Management 21:32-51.
Warner, J. T., and S. Fleeter (2001) "The personal discount rate: Evidence from military
downsizing programs." American Economic Review 91: 33-53.
Williams, R.C., III (2003) "Health Effects and Optimal Environmental Taxes," Journal of Public
Economics 87(2): 322-335.
Williams, R.C., III (2002) "Environmental Tax Interactions when Pollution Affects Health and
Productivity," Journal of Environmental Economics and Management 44(2): 261-270.
95
-------
APPENDIX A: LIST OF SAB REVIEW CHARGE QUESTIONS AND
RELATED CHAPTERS IN THE AGENCY DRAFT ANALYTICAL PLAN
AS RECEIVED FROM EPA ON JULY 3, 2003
Chapter 1: Project Goals and Analytical Sequence
1. Does the Council support the study goals, general analytical framework,
disaggregation plan, analytical sequence, and general analytical refinements defined in
chapter 1? If there are particular elements of these plans which the Council does not
support, are there alternatives the Council recommends?
Chapter 2: Scenario Development
2. Does the Council support the choices for analytical scenarios defined in chapter
2? Are there alternative or additional scenarios the Council recommends EPA consider
for inclusion in the analysis?
3. Does the Council support the alternative compliance pathway estimation and
comparison methodology described in chapter 2, including the specification of alternative
compliance pathways which may not reflect precisely constant emissions or air quality
outcomes between scenarios due (primarily) to the non-continuous nature and interaction
effects of emission control options?
Chapter 3: Emissions Estimation (Addressed by the Air Quality Modeling Subcommittee Report,
EPA Advisory Council on Clean Air Compliance Analysis, 2004a))
4. Does the Council support the plans for estimating, evaluating, and reporting
emissions changes as defined in chapter 3? If there are particular elements of these plans
which the Council does not support, are there alternative data or methods the Council
recommends?
5. Chapter 3 of the analytical plan describes several alternative approaches
considered by EPA for estimating non-EGU emissions growth rates. These options reflect
different relative emphasis between two conflicting analytical objectives: (1) extensive
refinement of the geographically differentiated, source-specific economic activity growth
estimates embedded in EGAS 4.0, and (2) maintaining the current project schedule and
budget. EPA plans to use "approach #4", a compromise option which targets the most
important source categories for potential refinement. Does the Council support the initial
plan to use "approach #4"? If the Council does not support the use of approach #4, are
there other approaches -including either the approaches described in chapter 3 or others
identified by the Council- which the Council suggests EPA consider?
96
-------
6. Some state-supplied emissions data incorporated in the 1999 National Emissions
Inventory (NET) -the core emissions inventory for this analysis- incorporate different
emissions factors from those used in MOBILE6, the mobile source emissions model EPA
plans to use for estimating emissions changes between scenarios. Of particular
importance, some of the emissions factors embedded in California's EMFAC model may
be significantly different from factors used in MOBILE6. EPA considered three options
for estimating emissions changes in California, which are described in chapter 3. EPA
plans to implement option #3 based on the belief that the emission factors embedded by
California in its EMFAC model may be more accurate for their particular state than the
factors incorporated in MOBILE6. Does the Council support the plan to implement
option #3? If the Council does not support the adoption of option #3, are there other
options -including either the options described in chapter 3 or others identified by the
Council- which the Council suggests EPA consider?
Chapter 4: Cost Estimates
7. Does the Council support the plans for estimating, evaluating, and reporting
compliance costs described in chapter 4? If there are particular elements of these plans
which the Council does not support, are there alternative data or methods the Council
recommends?
8. EPA seeks advice from the Council concerning the choice of Computable General
Equilibrium (CGE) model which EPA intends to use as a post-processor to gauge the
general equilibrium effects of the various control scenarios. In the first 812 Analysis -the
retrospective- EPA used the Jorgenson/Wilcoxen model to gauge the general equilibrium
effects of returning to the economy the reported compliance expenditures which formed
the basis of the retrospective study direct cost estimates. This model has since been
refined in many ways, and EPA considers both the Jorgenson/Wilcoxen/Ho and AMIGA
to be acceptable tools. Although a final decision on model choice can be deferred until
later in the analysis, EPA has tentative plans to use the AMIGA model because of its
greater sectoral disaggregation, better industrial sector matching with CAA-affected
industries, richer representation of relevant production and consumption technologies,
and better model validation opportunities due to its use of open code. However, AMIGA
is limited given its inability to deal with dynamics over time. Does the Council support
the current, tentative plan to use the AMIGA model for this purpose? If not, are there
alternative model choices or selection criteria the Council recommends?
9. In the two previous 812 Analyses, the primary cost estimates reflected use of a 5
percent real discount rate, which an earlier Council endorsed as a reasonable compromise
between a 3 percent real rate considered by EPA to be an appropriate estimate of the
consumption rate of interest or rate of social time preference and a 7 percent rate, OMB's
estimate of the opportunity cost of capital. Limited sensitivity testing was also conducted
in the previous 812 Analyses by substituting 3 and 7 percent rates to annualize the benefit
and cost streams. EPA's new Economics Guidelines (peer-reviewed by the SAB EEAC)
call for using both a 3 and a 7 percent rate. A recent draft of new OMB economic
guidelines suggests providing results based on both 3 and 7 percent discount rates, while
97
-------
also acknowledging the need for further efforts to refine analytical policies for
discounting methods and rates. EPA plans on following both sets of Guideline documents
by using both 3 and 7 percent in our core analyses. It is true that this will require
presentation of two sets of results - one based on each rate. This may not be necessary
given the expected insensitivity of the overall results to the discount rate assumption.
Does the Council support this approach? If not, are there alternative rates, discounting
concepts, methods, or results presentation approaches the Council recommends?
Chapter 5: Air Quality Modeling
10. Does the Council support the plans described in chapter 5 for estimating,
evaluating, and reporting air quality changes associated with the analytical scenarios? If
there are particular elements of these plans which the Council does not support, are there
alternative data, models, or methods the Council recommends? (To be addressed by the
Air Quality Modeling Subcommittee when the Agency has more details about the choice
of models and the modeling protocols that would be employed. )
Chapter 6: Human Health Effects Estimation (Addressed by the Health Effects Subcommittee,
EPA Advisory Council on Clean Air Compliance Analysis, 2004b)
11. Does the Council support the plans described in chapter 6 for estimating,
evaluating, and reporting changes in health effect outcomes between scenarios? If there
are particular elements of these plans which the Council does not support, are there
alternative data or methods the Council recommends?
12. EPA seeks advice from the Council regarding the technical and scientific merits
of incorporating several new or revised endpoint treatments in the current analysis. These
health effect endpoints include:
a. Premature mortality from particulate matter in adults 30 and over, PM (Krewski
et al., 2000);
b. A PM premature mortality supplemental calculation for adults 30 and over using
the Pope 2002 ACS follow-up study with regional controls;
c. Hospital admissions for all cardiovascular causes in adults 20-64, PM
(Moolgavkar et al., 2000);
d. ER visits for asthma in children 0-18, PM (Norris et al., 1999);
e. Non-fatal heart attacks, adults over 30, PM (Peters et al., 2001);
f School loss days, Ozone (Gilliland et al., 2001; Chen et al., 2000);
g. Hospital admissions for all respiratory causes in children under 2, Ozone (Burnett
etal., 2001); and,
h. Revised sources for concentration-response functions for hospital admission for
pneumonia, COPD, and total cardiovascular: Samet et al., 2000 (a PM10 study),
to Lippmann et al., 2000 and Moolgavkar, 2000 (PM2.5 studies).
13. EPA seeks advice from the Council regarding the merits of applying updated data
for baseline health effect incidences, prevalence rates, and other population
98
-------
characteristics as described in chapter 6. These updated incidence/prevalence data
include:
a. Updated county-level mortality rates (all-cause, non-accidental, cardiopulmonary,
lung cancer, COPD) from 1994-1996 to 1996-1998 using the CDC Wonder
Database;
b. Updated hospitalization rates from 1994 to 1999 and switched from national rates
to regional rates using 1999 National Hospital Discharge Survey results;
c. Developed regional emergency room visit rates using results of the 2000 National
Hospital Ambulatory Medical Care Survey;
d. Updated prevalence of asthma and chronic bronchitis to 1999 using results of the
National Health Interview Survey (HIS), as reported by the American Lung
Association (ALA), 2002;
e. Developed non-fatal heart attack incidence rates based on National Hospital
Discharge Survey results;
f Updated the national acute bronchitis incidence rate using HIS data as reported in
ALA, 2002, Table 11;
g. Updated the work loss days rate using the 1996 HIS data, as reported in Adams, et
al. 1999, Table 41;
h. Developed school absence rates using data from the National Center for
Education Statistics and the 1996 HIS, as reported in Adams, et al., 1999, Table
46.
1. Developed baseline incidence rates for respiratory symptoms in asthmatics, based
on epidemiological studies (Ostro et al. 2001; Vedal et al. 1998; Yu et al; 2000;
McConnell et al., 1999; Pope et al., 1991).
14. EPA plans to initiate an expert elicitation process to develop a probability-based
method for estimating changes in incidence of PM-related premature mortality. Plans for
this expert elicitation are described in chapter 9 of this blueprint, and a separate charge
question below requests advice from the Council pertaining to the merits of the design of
this expert elicitation. EPA recognizes, however, the possibility that this expert elicitation
process may not be fully successful and/or may not be completed in time to support the
current 812 Analysis. Therefore, in order to facilitate effective planning and execution of
the early analytical steps which provide inputs to the concentration-response calculations,
EPA seeks advice from the Council regarding the scientific merits of alternative methods
for estimating the incidences of PM-related premature mortality, including advice
pertaining to the most scientifically defensible choices for the following specific factors:
a. Use of cohort mortality studies, daily mortality studies, or some combination of
the two types of studies
b. Selection of specific studies for estimating long-term and/or short-term mortality
effects
c. Methods for addressing -either quantitatively or qualitatively- uncertain factors
associated with the relevant concentration-response function(s), including:
i. Shape of the PM mortality C-R function (e.g., existence of a threshold),
99
-------
ii. PM causality,
iii. PM component relative toxicity, and
iv. PM mortality effect cessation lag structure
v. Cause of death and underlying health conditions for individuals dying
prematurely due to chronic and/or short term exposures to particulate
matter
vi. The use of ambient measures of exposure for estimating chronic health
effects, given recent research reviewed in the NAS (2002) report that
questions the implications of using ambient measures in cohort studies
15. EPA estimates of benefit from particulate control may underestimate the impact
of nonfatal cardiopulmonary events on premature mortality and life expectancy. For the
base analyses, which rely on cohort evidence, the limited follow-up periods for the
cohorts may not fully capture the impacts of nonfatal cardiovascular events on premature
mortality later in life. For the alternative analyses -including cost-effectiveness analyses-
which rely more on acute studies and life-expectancy loss, the years of life are estimated
only for fatal events. Yet nonfatal events such as myocardial infarction reduce a person's
life expectancy by a substantial percentage.
a. Do you agree that EPA, in the 812 Analyses, should adjust benefit estimates to
account for the mortality effects of non-fatal cardiovascular and respiratory
events?
b. What medical studies and mathematical models of disease might be useful to
review or use if EPA moves in this direction?
c. When the nonfatal events are valued in economic terms, should EPA assume that
the published unit values for morbidity already account for the life-expectancy
loss or should an explicit effort be made to monetize the resulting longevity
losses?
16. In recent EPA rulemakings, EPA's "base estimate" of benefit from PM control has
been based on cohort epidemiological studies that characterize the chronic effects of
pollution exposure on premature death as well as capturing a fraction of acute premature
mortality effects. If these chronic effects occur only after repeated, long-term exposures,
there could be a substantial latency period and associated cessation lag. As such, a proper
benefits analysis must consider any time delay between reductions in exposure and
reductions in mortality rates. For the acute effects, such as those considered in EPA's
alternative benefit analyses, the delays between elevated exposure and death are short
(less than two months), and thus time-preference adjustments are not necessary.
a. In the previous 812 Analysis and in recent rulemakings, EPA assumed a weighted
5-year time course of benefits in which 25% of the PM-related mortality benefits
were assumed to occur in the first and second year, and 16.7% were assumed to
occur in each of the remaining 3 years. Although this procedure was endorsed by
SAB, the recent NAS report (2002) found "little justification" for a 5-year time
course and recommended that a range of assumptions be made with associated
100
-------
probabilities for their plausibility. Do you agree with the NAS report that EPA
should no longer use the deterministic, 5-year time course?
b. One alternative EPA is considering is to use a range of lag structures from 0 to
20-30 years, with the latter mentioned by NAS in reference to the Nyberg et al
PM lung cancer study, with 10 or 15 years selected as the mid-point value until
more definitive information becomes available. If this simple approach is used,
should it be applied to the entire mortality association characterized in the cohort
studies, or only to the difference between the larger mortality effect characterized
in the cohort studies and the somewhat smaller effect found in the time series
studies of acute exposure? Should judgmental probabilities be applied to different
lags, as suggested by NAS?
c. Another option under consideration is to construct a 3-parameter Weibull
probability distribution for the population mean duration of the PM mortality
cessation lag. The Weibull distribution is commonly used to represent
probabilities based on expert judgment, with the 3-parameter version allowing the
shaping of the probability density function to match expected low, most likely,
and expected high values. EPA is still considering appropriate values for the low,
most likely, and expected high values -and therefore for the Weibull shape and
location parameters- and EPA is interested in any advice the Council wishes to
provide pertaining to the merits of this approach and/or reasonable values for the
probability distribution.
17. In support of Clear Skies and several recent rule makings the Agency has
presented an Alternative Estimate of benefits as well as the Base Estimate. EPA
developed the Alternative Estimate as an interim approach until the Agency completes a
formal probabilistic analysis of benefits. NAS (2002) reinforced the need for a
probabilistic analysis. The Alternative Estimate is not intended as a substitute method and
needs to be considered in conjunction with the Base Estimate. Presentation of Base and
Alternative estimates in the 812 Report may not be necessary if the probability analysis
planned for the 812 Report is successful. While the Base Estimate assumes that acute and
chronic mortality effects are causally related to pollution exposure, the Alternative
Estimate assumes only acute effects occur or that any chronic effects are smaller in size
than assumed in the Base Estimate. The Council's advice is sought on the following
matters:
a. It has been noted by some particle scientists that the size of estimates based on
time series studies that incorporate a distributed lag model, accounting for effects
of 30 to 60 days after elevated exposure, may be similar in size to some
interpretations of the results from the cohort studies. Does the Council agree that
it is a reasonable alternative to use an estimate of the concentration-response
function consistent with this view? If the Council agrees with the assumption, can
it suggest an improved approach for use in an Alternative Estimate? The agency
also seeks advice on appropriate bounds for a sensitivity analysis of the mortality
estimate to be used in support of the Alternative Estimate.
b. An assumption that a specific proportion of the PM-related premature mortality
incidences are incurred by people with pre-existing Chronic Obstructive
101
-------
Pulmonary Disease (COPD) and that these incidences are associated with a loss of
six months of life, regardless of age at death. If these values are not valid, what
values would be more appropriate? Do you recommend a sensitivity analysis of 1
to 14 years (with the latter based on standard life tables), as included in the draft
regulatory impact analysis of the proposed nonroad diesel rule?
An assumption that the non-COPD incidences of PM-related premature mortality
are associated with a loss of five years of life, regardless of age at death. If these
values are not valid, what values would be more appropriate? Do you recommend
a sensitivity analysis of 1 to 14 years (with the latter based on standard life
tables), as included in the draft regulatory impact analysis of the proposed
Nonroad diesel rule?
Additional quantified and/or monetized effects are those presented as sensitivity
analyses to the primary estimates or in addition to the primary estimates, but not
included in the primary estimate of total monetized benefits. While no causal
mechanism has been identified for chronic asthma and ozone exposure, there is
suggestive epidemiological evidence.
i. Two studies suggest a statistical association between ozone and
new onset asthma for two specific groups: children who spend a lot of
time exercising outdoors and non-smoking men. We seek SAB comment
on our approach to quantifying new onset asthma in the sensitivity
analyses.
ii. Premature mortality associated with ozone is not currently
separately included in the primary analysis because the epidemiological
evidence is not consistent. We seek SAB comment on our approach to
quantifying ozone mortality in the sensitivity analyses.
iii. Does the Council agree that there is enough data to support a
separate set of health impacts assessment for asthmatics? If so, does the
approach proposed by the Agency address the uncertainty in the literature?
Chapter 7: Ecological Effects
18. Does the Council support the plans described in chapter 7 for (a) qualitative
characterization of the ecological effects of Clean Air Act-related air pollutants, (b) an
expanded literature review, and (c) a quantitative, ecosystem-level case study of
ecological service flow benefits? If there are particular elements of these plans which the
Council does not support, are there alternative data or methods the Council recommends?
19. Initial plans described in chapter 7 reflect a preliminary EPA decision to base the
ecological benefits case study on Waquoit Bay in Massachusetts. Does the Council
support these plans? If the Council does not support these specific plans, are there
alternative case study designs the Council recommends?
20. Does the Council support the plan for a feasibility analysis for a hedonic property
study for valuing the effects of nitrogen deposit!on/eutrophication effects in the
102
-------
Chesapeake Bay region, with the idea that these results might complement the
Waquoit Bay analysis?
Chapter 8: Economic Valuation
21. Does the Council support the plans described in chapter 8 for economic valuation
of changes in outcomes between the scenarios? If there are particular elements of these
plans which the Council does not support, are there alternative data or methods the
Council recommends?
22. EPA's current analytic blueprint calls for an expert-judgment project on VSL
determination that would produce a probability distribution over the range of possible
VSL values for use in the 812 project. EPA is not sure how much priority to give to this
project. A much simpler alternative would be for EPA to specify a plausible range of
VSL values. One option would be to use a range bounded by $1 million (based roughly
on the lower bound of the interquartile range from the Mrozek-Taylor meta-analysis) and
$10 million (based roughly on the upper bound of the interquartile range of the Viscusi-
Aldy meta-analysis. This range would match that reflected in EPA's sensitivity analysis
of the alternative benefit estimate for the off-road diesel rulemaking. The range would
then be characterized using a normal, half-cosine, uniform or triangular distribution over
that range of VSL values. EPA would then ask this Committee to review this distribution.
This approach could be done relatively quickly, based on the reviews and meta-analyses
commissioned to date, and would allow a formal probability analysis to proceed, without
suggesting that the Agency is trying to bring more precision to this issue than is
warranted by the available science.
23. Pursuant to SAB Council advice from the review of the first draft analytical
blueprint, EPA reviewed a number of meta-analyses -either completed or underway-
developed to provide estimates for the value of statistical life (VSL) to be applied in the
current study. EPA plans to consult with the Council (and coordinate this consultation
with the EEAC) on how best to incorporate information from the Kochi et al (2002)
meta-analysis, other published meta-analyses [Mrozek and Taylor and Viscusi and Aldy],
and recent published research to develop estimates of VSL for use in this study. In
addition, EPA plans to implement two particular adjustments to the core VSL values:
discounting of lagged effects and longitudinal adjustment to reflect changes in aggregate
income. Does the Council support these plans, including the specific plans for the
adjustments described in chapter 8? If the Council does not support these plans, are there
alternative data or methods the Council recommends?
24. For the 812 Report, EPA has decided to perform a cost-effectiveness analysis of
the Clean Air Act provisions using quality-adjusted life years as the measure of
effectiveness. This is the standard approach used in medicine and public health and this
type of analysis has previously been recommended by the SAB. Moreover, the recent
NAS Report (2002) on benefits analysis discussed how this method could be applied to
the health gains from air pollution control.
103
-------
a. Do you agree that QALYs are the most appropriate measure of effectiveness for
this type of analysis? Would you suggest any alternative measures to replace or
supplement the QALY measure? (This question relates to effectiveness measures,
not monetary benefit measures as used in benefit-cost analysis).
b. OMB has suggested that EPA plan a workshop with clinicians, social scientists,
decision analysts and economists to examine how the specific diseases and health
effects in the 812 Report should be handled with respect to longevity impact and
health-related preference. Participants would have knowledge of the relevant
clinical conditions, the related health preference studies, and the stated-preference
literature in economics. The recent RFF conference has laid the groundwork for
this type of workshop. Is there a superior approach to making sure that the
CEAQALY project is executed in a technically competent fashion and that the
details of the work receive in-depth technical input in addition to the broad
oversight provided by this Committee?
c. Does the Council support the specific plans for QALY-based cost-effectiveness
described in the current draft blueprint? If the Council does not support specific
elements of these plans, are the alternative data, methods, or results presentation
approaches which the Council recommends?
25. EPA plans to use updated unit values for a number of morbidity effects, as
described in chapter 8. Of particular note, EPA plans to rely on a study by Dickie and
Ulery (2002) to provide heretofore unavailable estimates of parental willingness to pay to
avoid respiratory symptoms in their children. This study is not yet published and has
limitations concerning response rate and sample representativeness; however, EPA
expects the study to be published prior to completion of the economic valuation phase of
this analysis. Does the Council support the application of unit values from this study,
contingent on its acceptance for publication in a peer-reviewed journal? If the Council
does not support reliance on this study, are there other data or methods for valuation of
respiratory symptoms in children which the Council recommends?
Chapter 9: Uncertainty Analysis
26. Does the Council support the plans described in chapter 9 for estimating and
reporting uncertainty associated with the benefit and cost estimates developed for this
study? If there are particular elements of these plans which the Council does not support,
are there alternative data, models, or methods the Council recommends?
27. Does the Council support the plans described in chapter 9 for the pilot project to
develop probability-based estimates for uncertainty in the compliance cost estimates? If
the Council does not support this pilot project, or any particular aspect of its design, are
there alternative approaches to quantifying uncertainty in cost estimates for this analysis
which the Council recommends?
28. Does the Council support the plans described in chapter 9 for the pilot project to
develop probability-based estimates for uncertainty in the emissions and air quality
104
-------
modeling estimates? If the Council does not support this pilot project, or any particular
aspect of its design, are there alternative approaches to quantifying uncertainty in
emissions and/or air quality concentration estimates for this analysis which the Council
recommends? (To be addressed by the Air Quality Modeling Subcommittee when the
Agency has more details about the choice of models and the modeling protocols that
would be employed. )
29. Does the Council support the plans described in chapter 9 for the expert elicitation
pilot project to develop a probability-based PM2.5 C-R function for premature mortality,
including in particular the elicitation process design? If the Council does not support the
expert elicitation pilot project, or any particular aspect of its design, are there alternative
approaches the Council recommends for estimating PM-related mortality benefits for this
analysis, including in particular a probabilistic distribution for the C-R function to reflect
uncertainty in the overall C-R function and/or its components?
30. EPA plans to develop estimates of an independent mortality effect associated with
ozone, as described in chapter 9. Does the Council support the use of the most recent
literature on the relationship between short-term ozone exposure and daily death rates,
specifically that portion of the literature describing models which control for potential
confounding by PM2.5? Does the Council agree with the use of that literature as the basis
for deriving quantified estimates of an independent mortality impact associated with
ozone, especially in scenarios where short-term PM2.5 mortality estimates are used as the
basis for quantifying PM mortality related benefits? Does the Council support the plans
described in chapter 9 for the pilot project to use this literature to develop estimates of the
ozone related premature mortality C-R function using the three alternative meta-analytic
approaches? If the Council does not support this pilot project, or any particular aspect of
its design, are there alternative approaches to quantifying ozone-related premature
mortality which the Council recommends?
31. EPA plans to work with the Council and the EEAC to develop revised guidance
on appropriate VSL measures. We hope to include the Kochi et al (2002) meta-analysis,
other recent meta-analysis, recent publications, and the 3 literature reviews sponsored by
EPA.(a separate charge question pertaining to this element of EPA's VSL plan is
presented below). In addition, EPA plans to conduct a follow-on meta-regression analysis
of the existing VSL literature to provide insight into the systematic impacts of study
design attributes, risk characteristics, and population attributes on the mean and variance
of VSL. Does the Council support the plans described in chapter 9 for conducting this
meta-regression analysis? If the Council does not support this analysis or any particular
aspect of its design, are there alternative approaches which the Council recommends for
quantifying the impact of study design attributes, risk characteristics, and population
attributes on the mean and variance of VSL?
Chapter 10: Data Quality and Intermediate Data Products
32. Does the Council support the plans described in chapter 10 for evaluating the
quality of data inputs and analytical outputs associated with this study, including the
105
-------
planned publication of intermediate data products and comparison of intermediate and
final results with other data or estimates? If the Council does not support these plans, are
there alternative approaches, intermediate data products, data or model comparisons, or
other data quality criteria the Council recommends? Please consider EPA's Information
Quality Guidelines in this regard.
Chapter 11: Results Aggregation and Reporting
33. Does the Council support the plans described in Chapter 11 for the aggregation
and presentation of analytical results from this study? If the Council does not support
these plans, are there alternative approaches, aggregation methods, results presentation
techniques, or other tools the Council recommends?
Appendix D: Stratospheric Ozone Analysis
34. Does the Council support the plans describe in Appendix D for updating the
estimated costs and benefits of Title VI programs? If the Council does not support these
plans, are there alternative data, models, or methods the Council recommends?
Appendix E: Air Toxics Case Study
35. Does the Council support the plans described in Appendix E for the benzene case
study, including the planned specific data, models, and methods, and the ways in which
these elements have been integrated? If the Council does not support these plans, are
there alternative data, models, or methods the Council recommends?
36. A cessation lag for benzene-induced leukemia is difficult to estimate and model
precisely due to data limitations, and EPA plans to incorporate a five-year cessation lag
as an approximation based on available data on the latency period of leukemia and on the
exposure lags used in risk models for the Pliofilm cohort (Crump, 1994 and Silver et al.,
2002). Does the SAB support adoption of this assumed cessation lag? If the Council does
not support the assumed five-year cessation lag, are there alternative lag structures or
approaches the Council recommends? (Addressed by the Health Effects Subcommittee,
EPA Advisory Council on Clean Air Compliance Analysis, 2004b)
Appendix H: Meta-analysis of VSL
37. Does the Council support including the Kochi et al. (2002) meta-analysis as part
of a the larger data base of studies to derive an estimate for the value of avoided
premature mortality attributable to air pollution? Are there additional data, models, or
studies the Council recommends? Does the SAB think that EPA should include Kochi et
al. 2003 if not accepted for publication in a peer reviewed journal by the time the final
812 report is completed?
106
-------
APPENDIX B: LIST OF ACRONYMS
AQMS - Air Quality Modeling Subcommittee
CAA - Clean Air Act
CAAA - Clean Air Act Amendment
B/C - Benefit-Cost
BCA - Benefit-cost Analysis
BLS - Bureau of Labor Statistics
CEA - Cost-effectiveness Analysis
CGE - Computable General Equilibrium
COI-Cost of Illness
Council - The Advisory Council for Clean Air Compliance Analysis
C-VPESS - SAB Committee on Valuing the Protection of Ecological Systems and Services
EES - Ecological Effects Subcommittee
EGU - Electrical Generating Unit
Electrical Power Research Institute - EPRI
HAP - Hazardous Air Pollutant
HES - Health Effects Subcommittee
I/M - Inspection and Maintenance
IPM - Integrated Planning Model
JHW - Jorgenson-Ho-Wilcoxen
MACT - Maximum Achievable Control Technology
MBI - Market Based Incentives
NAAQS - National Ambient Air Quality and Standards
107
-------
NOX - Nitrogen Oxides
NPV - Net Present Value
PACE - Pollution Abatement and Control Expenditures
PM - Particulate Matter
QALYs- Quality-Adjusted Life-Years
VSL - Value of Statistical Life
VOC - Volatile Organic Compounds
VSLY - Value of a Statistical Life-Year
WTP - Willingness-to-Pay
108
-------
APPENDIX C: ADDITIONAL DISCUSSION CONCERNING COSTS AND
LEARNING
The assortment of published models that yield markedly different point estimates for
learning effects are frequently inconsistent with neoclassical economics in terms of the use of
factor inputs. To be deemed admissible, it would also be desirable for a study to meet higher
standards in terms of accounting for technical change.
For cost-savings due to learning, there is a potentially very important question of whether
firms enjoy advantages, or suffer penalties, for early implementation of technologies. Being a
"first mover" may limit opportunities for learning from the experiences of other firms.
It is not clear that cumulative output is the sole, or best, indicator of learning effects on
the eventual costs of abatement activities. The time horizon over which cost reductions due to
learning will be exhausted is also not clear. Costs just a few months out may differ substantially
from the cost levels that can be attained in the long-term steady-state, even when cumulative
production is identical. Eighteen months out, costs can be a little lower, or a lot lower, than the
level to which they may fall with early learning.
Process versus industry-specific. It should be emphasized in the 812 Analysis that the
80% rule of thumb for learning effects is a gross oversimplification. For example, the effect of
learning on compliance costs is more likely to be process-specific, rather than industry specific.
Thus it may be inappropriate just to make different assumptions across industries. Instead, the
correct "representative" learning effect may depend upon the mix of processes used in each
industry.
Desirability/attainability of one number for learning. Despite the preliminary results of
the meta-analysis and the absence of any real weight-of-the-evidence conclusions concerning
learning effects, it would still be helpful to come up with a best estimate to use for assumptions
about cost reductions from experience with compliance technologies. It would be easiest if it
were safe to assume a single "learning effect" in the form of an unbiased estimate, neither too
high nor too low. However, the effect of learning on costs is likely to display considerable
systematic heterogeneity across pollutants and technologies. There is unlikely to be a single
"one-size-fits-all" number that is satisfactory for all contexts.
Is it preferable to make an inaccurate adjustment for learning (e.g., when it is not known
whether the adjustment should be 10% or 20%) rather than make no adjustment at all, which is
known definitely to be incorrect (i.e., there need to be some downward adjustment to costs as a
result of learning, but the appropriate magnitude of this adjustment is unclear)? The question of
just how much must be known before the Agency is warranted in making a quantitative
adjustment permeates many aspects of the Analytical Plan, not just the learning issue, and merits
more thought and discussion. In principle, what is desired is the best unbiased estimate, but
where is the threshold of empirical evidence needed to decide upon the appropriate magnitude of
that quantitative adjustment?
109
-------
For example, in its review of the Draft Analytical Plan, two years ago, a majority on the
Council agreed that there was insufficient evidence to support using for ecosystem benefits a
particular percentage of the Costanza et al. (1998) estimates of total value of the earth's
ecosystems. This conclusion was reached in part because there was not sufficient evidence to
determine the appropriate percentage of these ecosystems values that would have been lost or
reduced without the CAAA.
The Council feels it would be inappropriate to endorse adjustments that have minimal
empirical verification as to their specific quantitative values. The cumulative effect of too many
such adjustments puts the entire assessment process at risk of losing objective credibility and
becoming more a product of subjectivity and political negotiation. The Council encourages the
Agency to explore the likely consequences of adjustments that are within the realm of
possibility, but not to build in any specific unsupported value for specific adjustments.
110
-------
APPENDIX D: ADDITIONAL BIBLIOGRAPHIES
D.I. Value of Time
Altonji, J. and Paxson, C. (1988) "Labor supply preferences, hours constraints, and hours-wage
trade-offs," Journal of Labor Economics 6(2): 254-276.
Bates, J. (1987) "Measuring travel time values with a discrete choice model: a note," Economic
Journal 97: 493-498.
Becker, G. (1965) "A theory of the allocation of time," Economic Journal! 5: 493-517.
Ben-Akiva , Moshe, Denis Bolduc, and Mark, Bradley (1994) "Estimation of Travel Choice
Models with Randomly Distributed Values of Time". Transportation Research Record
(#1413).
Ben-Akiva , Moshe, and Lerman, S. (1985) Discrete choice analysis: Theory and Application to
Travel Demand, MIT Press, Cambridge, MA.
Blayac, T. and A. Causse (2001) "Value of travel time: A theoretical legitimization of some
nonlinear representative utility in discrete choice models," Transportation Research 35B
(4): 391-400.
Becker, G. S. (1965) "A theory of the allocation of time," Economic Journal 75 (299): 493-517.
Beesley, M.E. (1965) "The value of time spent in traveling: Some new evidence," Economica
pp. 174-185.
Bishop, R.C. and Heberlein, T.A. (1980) "Simulated Markets, Hypothetical Markets and Travel
Cost Analysis: Alternative Methods of Estimating Outdoor Recreation Demand,"
Working Paper a!87.
Bockstael, N., Strand, I. and Hanemann, W. (1987a) "Time and the recreational demand model,"
American Journal of Agricultural Economics 69 (2): 293-302..
de Donnea, F.X. (1972) "Consumer behavior, transport mode choice and value of time: Some
micro-economic models," Regional and Urban Economics 14 : 355-382.
DeSerpa, A. C. (1971) "A theory of the economics of time," Economic Journal Dec, pp. 828-
846.
Fernandez, R. (1994) "Race, space and job accessibility: Evidence from a plant relocation,"
Economic Geography 70 (4): 390-416..
Feather, P. and Shaw, W.D.(2000) "The Demand for Leisure Time in the Presence of
Constrained Work Hours." Economic Inquiry 38 (4): 651-62.
Feather, P., and Shaw, W.D. (1999). "Estimating the cost of leisure time for recreation demand
models." Journal of Environmental Economics and Management 38 (1): 49-65.
Gunn, H. (2001) "Spatial and temporal transferability of relationships between travel demand,
trip cost and travel time," Transportation Research 37 E (2-3), 163-189.
Ill
-------
Hensher, D.A. (1997) "Behavioral value of travel time savings in personal and commercial," in:
Greene, D.L., D.W. Jones, and M.A. Delucchi (Ed.) The Full Costs and Benefits of
Transportation, 245-279, Springer.
Hensher, D.A. (2001) "Measurement of valuation of travel time savings," Journal of Transport
Economics and Policy 35 (1), 71-98.
Ham, J.C. (1982) "Estimation of a labour supply model with censoring due to unemployment
and underemployment," Review of Economic Studies. 49 (3) : 335-354 .
Heckman, JJ. (1974) "Shadow prices, market wages, and labor supply," Econometrica 42 (4):
679-694.
Johnson, M.B. (1966) "Travel time and the price of leisure," Western Economic Journal 4 (2):
135-145.
Jara-Diaz, S. (1998) "Time and income in travel choice: towards a microeconomic activity
framework." In Theoretical Foundations of Travel Choice Modeling, T. Garling, T. Laitia
y K. Westin, eds. Pergamon, 51-73.
Larson, D. (1993) "Separability and the shadow value of leisure time," American Journal of
Agricultural Economics 75 (3): 572-577'.
Macurdy, T., Green, D. and Paarsch, H.(1990) "Assessing empirical approaches for analyzing
taxes and labor supply," Journal of Human Resources 25(3) : 415-490.
McConnell, K. 1992. "On-site time in the demand for recreation," American Journal of
Agricultural Economics 74(4): 918-925.
Moffitt, R. (1990) "The estimation of a joint wage hours labor supply model," Journal of Labor
Economics 2 (4) : 550-566.
Moffitt, R. (1982) "The Tobit model, hours of work and institution constraints," Review of
Economics and Statistics 64 (3): 510-515..
Mossin, J. and Bronfenbrenner, M. (1967) "The shorter work week and labor supply," Southern
Economic Journal 34 (3): 322-331.
Smith, V. K., Desvousges, W. H. and McGivney, M. P. (1983) "The opportunity cost of travel
time in recreational demand models," Land Economics 59 (3): 259-277.
Tummers, M. and Woittiez, I. (1991) "A simultaneous wage and labor supply model with
hours restrictions," Journal of Human Resources 26(3): 393-423.
van Soest, A. (1995) "Structural models of family labor supply," Journal of Human Resources,
30(1): 63-88.
Wardman, M. (1998) "The value of travel time: A review of British evidence," Journal of
Transport Economics and Policy 32(3), 285-316.
Yen, S. (1993) "Working wives and food away from home: The Box-Cox double hurdle model,"
American Journal of Agricultural Economics 75 (4): 884-895.
Zabel, J. (1993) "The relationship between hours of work and labor force participation in four
models of labor supply behavior," Journal of Labor Economics 11 (2): 387-416.
112
-------
D.2. Materials Damage
Acres International Limited (1991) "The Effects and Social Costs of Fossil-Fired Generating
Station Emissions on Structural Materials" (Update), Niagara Falls, Ontario, prepared for
Ontario Hydro.
Courant, P.N. and R.C. Porter (1981), "Averting expenditure and the cost of pollution," Journal
of Environmental Economics and Management 8(4), 321-329.
Harford, J.D. (1984) "Averting behavior and the benefits of reduced soiling," Journal of
Environmental Economics and Management 11, 296-302.
Harrison, D., A.L. Nichols, S.L. Bittenbender and M.L. Berkman (1993) "External costs of
electric utility resource selection in Nevada," report to Nevada Power Company, March,
Cambridge, MA: National Economic Research Associates.
Manuel, E.H., R.L. Horst, K.M. Brennan, W.N. Lanen, M.C. Duff and J.K. Tapiero (1982)
Benefits Analysis of Alternative Secondary National Ambient Air Quality Standards for
Sulphur Dioxide and Total Suspended Particulates, report prepared for the US
Environmental Protection Agency, Princeton, NJ: Mathtech, Inc.
TRC Environmental Consultants, Inc. (1984) "Damage Cost Models for Pollution Effects on
Materials, report prepared for Environmental Sciences Research Lab," East Hartford, CT:
TRC Environmental Consultants, Inc.
Watson, W.D. and J.A. Jaksch (1982) "Air pollution: Household soiling and consumer welfare
losses," Journal of Environmental Economics and Management 9, 248-262.
113
-------
APPENDIX E: ADDITIONAL DISCUSSION CONCERNING THE USE OF VSLs
This appendix covers material that can be classified as "experimental" or "methods
development." It emphasizes some shortcomings of existing practices with respect to VSLs.
The Agency is advised to anticipate changes in the state of the art in human health benefits
valuation that may be appropriate to incorporate in future 812 Analyses as these updated
approaches are vetted and as the justification for them becomes more widely understood.
The Council first wishes to highlight persistent conceptual problems stemming from the
use of "the VSL." Normalizing WTP to a 1.00 risk reduction is arbitrary and has proven to be
confusing to non-specialists and therefore open to being used in a strategically misleading
fashion. As a device for combining WTP estimates based on different risk changes, any arbitrary
normalization is equally appropriate and a more policy-relevant risk change would be preferable
for normalization, even if this necessitates a change in traditions.
That WTP should be close to proportional to the size of the risk change has theoretical
support and would be enormously convenient. However, empirical tests of this theory are very
difficult, with hedonic wage data and contingent valuation studies tending to produce results at
odds with this assumption. More information on this important aspect of VSL implementation
would be valuable.
WTP for risk reductions should be presumed to be heterogeneous across risks and
individuals, unless demonstrated otherwise. It is important that the proposed meta-analyses are
designed to recognize this.
Existing meta-analyses have tended to maintain the hypothesis that there exists a single
immutable VSL (or a simple VSL function that depends mostly on income levels). The early
Agency posture suggested that this unknown VSL merely needed to be revealed by somehow
combining VSL estimates from different studies.
The studies that form the raw material for meta-analysis may be compromised to varying
degrees by their subjects having had incomplete information about risk. Credible meta-analyses
should address these problems as well.
The Agency should proceed cautiously in adopting the results of existing or new meta-
analyses as the basis for some assumed distribution for the WTP that will be appropriate for the
Second Prospective Analysis. The contexts of the constituent studies may not adequately match
the policy context where the WTP is needed.
E.I. VSLs vs. Micromorts
The concept of the value of a statistical life has unnecessarily impeded clear
communication with risk managers about the public's value for small changes in health risks.
However, the Council acknowledges that it is not in the Agency's best interest to attempt to take
114
-------
the lead by proposing fundamental changes in the way economists traditionally have thought
about valuing mortality risks. Such initiatives properly comes from the academic community.
However, the Council wishes to draw the Agency's attention to ideas and approaches that are
likely to develop in the literature over the next few years. Even without adopting a substantially
different perspective on mortality risk valuation, the Agency can report mortality values in ways
that are less susceptible to misinterpretation by non-experts in the constituency for the Section
812 reports. Specifically, the Agency should exercise more precision in describing and
qualifying the measures of mortality risk reduction it currently uses. Whenever the concept of a
VSL is introduced, the Agency should identify the VSL explicitly as a normalization relative to a
particular baseline risk. The corresponding range of untransformed WTP estimates for the
policy-relevant range of risk changes should be provided for comparison.
VSL is defined as the marginal rate of substitution (MRS), namely the (local) difference
in income that will leave an individual equally well off in the face of a difference in mortality
risk. It is well recognized in the literature that this MRS depends on baseline risk, income, and
may well depend on other characteristics of the risk and the individual. The units in which this
MRS is described are arbitrary (e.g., dollars per pound, pennies per ton, etc.). By focusing on
"the Value of a Statistical Life," we have arbitrarily adopted as our units "dollars per 1.00 risk
change."
The population WTP for a specified risk reduction is defined as the sum of individuals'
WTP for the individual risk reductions. For example, if a policy change reduces fatality risk this
year by Ar for everyone in a population of size N, the population WTP for this change can be
calculated as vN, where v is the population average WTP for a Ar reduction in the chance of
dying this year. This same population value is often described as the product of the average VSL
and the expected number of "lives saved" by the risk reduction. Using the normalization of
dollars per 1.0 risk change, VSL is defined as v / Ar, and "lives saved" is equal to the expected
number of deaths averted this year, i.e., N Ar.
While this alternative formulation, in terms of the average VSL and the number of "lives
saved," is mathematically equivalent to the population WTP (i.e., the product of the average
WTP and the population size), it is potentially misleading. It suggests that the value of each "life
saved" is equal to the average VSL, and that one only needs to know the expected number of
"lives saved" in order to calculate population WTP. In addition to other factors, VSL is likely to
depend on the size of the individual risk reduction Ar, and so the population WTP for a change
that "saves one life" may depend on whether the change reduces many people's risk by a small
amount or reduces a small number of people's risk by a large amount.
The arbitrary choices made with respect to the normalization of VSLs unnecessarily court
objections from non-specialists who confuse "The Value of a Statistical Life" (the economists'
technical term for an extrapolated linear approximation to a marginal measure) with "The Value
of Life" in the sense of some measure of the intrinsic value of one human life with certainty.
Long ago, Howard (1984) proposed the term "micromort," meaning the value of a one-in-a-
million risk reduction, which would translate into one one-millionth of our usual $5-6 million
VSL, or just 5 to 6 dollars. This metric would be less misleading than the VSL, but
unfortunately it has never achieved currency. There is no imperative to choose a 1.00 risk
115
-------
change as the intervening metric for scaling. Scaling all estimates to the risk change relevant for
some specific policy is just as valid, and would lead to the identical mathematical result for
aggregate WTP for a risk reduction policy.
VSL-2
VSL-1
There are other potential
concerns about empirical measures of WTP
WTP for risk reductions. Suppose that
we are trying to combine the
information about WTP for risk
reductions from five different studies,
each involving one particular
(different) risk reduction, rl through
r5, as in the figure. (With any luck,
there will be standard errors on the
underlying WTP estimates, as shown,
so there will be corresponding standard
errors on the resulting individual
studies' estimates of VSLs, although
these are not depicted in the diagram.)
If we use the WTP and risk
information from each study to impute
the associated VSL for a 1.00 risk
change, the numbers may vary widely,
as shown. It is these different VSL
estimates that most meta-analyses seek
to "average" according to formulas of
different complexity and
sophistication. By taking some type of
average of the five separate VSLs, we
can infer an average WTP for risk reductions that controls for the different risks across studies.
However, if the true WTP function tracks along the dashed line, and if the policy context
concerns a risk change that is, say, slightly larger than r5, then the WTP that would be inferred
from the average VSL would be an inappropriate estimate.
The individual WTP point values depicted in the diagram may also differ because of
other types of heterogeneity across the contexts wherein they were derived. In that case, it would
of course be inappropriate to average these results, even after normalization to a common 1.00
risk change.
VSLs are based on empirical data concerning choices in the neighborhood of very small
risks and small risk differences. Outside of this domain, we can really say nothing about WTP
for much larger risks and risk changes. The implicit extrapolation to a 1.00 risk change that
produces a VSL is understood by specialists to be purely a convenient device to control for
variations in the sizes of risk reductions across the studies that yield these estimates.
Unfortunately, this is often not understood as such by non-specialists.
study risk
changes
1.00
risk change
116
-------
E.2. Proportionality
The VSL can be viewed simply as a strategy for getting around the fact that WTP from
different studies corresponds to different sized risk changes. It would be inappropriate to
average the individual WTP estimates without acknowledging that they apply to different risk
changes. The issue of proportionality of estimated WTP for risk reduction and magnitudes of
these risk reductions has been raised previously (e.g., Hammitt and Graham, 1999). Certainly, if
we wish to maintain the hypothesis that there exists a single one-size-fits-all VSL that is the
same for all possible risk reductions, then the estimated WTP for different risk reductions ought
to be proportional to the sizes of the risk reductions in question. This constitutes a requirement
for a very specific type of "scope test." However, not all empirical estimates of WTP functions
produce parameters that are consistent with this requirement. Some studies show negligible
effects of risk changes on WTP. Such a result is clearly problematic for valuing mortality risks.
However, other studies reveal estimates that suggest that WTP is not strictly proportional to the
size of the risk change.
Stated-preference (e.g., contingent valuation) studies almost invariably show that WTP is
an increasing but concave function of risk reduction. Revealed-preference studies (e.g., hedonic
wage studies) typically do not tell us anything about how WTP depends on the magnitude of the
risk change because we model workers as choosing jobs from a continuous set of jobs that differ
in wage and risk, and typically do not have information on what jobs (and risks) an individual
rejects.
For example, compensating-wage-differential estimates are based on fitting a regression
model to data on individual workers' wages, occupational fatality risks, and other variables such
as education and job experience that influence wages. This regression estimates how wages vary
with occupational fatality risk, holding other factors constant. Each worker is assumed to prefer
the job he holds to other jobs that are potentially available to him, which are characterized by the
regression. Setting the independent variables equal to the worker's characteristics, the regression
is interpreted as describing how the set of jobs available to him differ in wage and risk.
Many of the studies that yield WTP estimates do so for only a single common risk
difference for all subjects, so there is too little information in any single study to assess the effect
of the size of the risk change on WTP. Some sort of preference calibration exercise would be
necessary in order to combine all of the available estimates.
E.3. Heterogeneity: Context-dependent WTP
Many practitioners seem to lose sight of the subtlety that the VSL is not a physical
constant, like the constant of gravitation (6.673 ± 0.003) x 10-8 cm3gm-ls-2 , or the mass of a
hydrogen atom (1.67339 ± 0.0031) x 10-24 g. Instead, VSL is an artifact of human preferences.
It is based on willingness to pay for risk reduction, which depends on the marginal (dis)utility of
risk and on the marginal utility of income. While it may be possible to identify some regularities
across types of people in these two marginal utilities, it is conceivable that they are essentially
unique to each person. Therefore, so can be the corresponding VSL.
117
-------
The contexts for empirical studies concerning risk tradeoffs differ in many more ways
besides just the risk change they consider. The types of risk and the characteristics of the
individuals experiencing these risks can also lead to heterogeneity in WTP. If the policy context
is not "in the middle" of the range of study contexts, then it can be potentially very misleading to
assume that the "average VSL" implied by the range of available studies is a good measure of
WTP to reduce the specific risk in the specific affected population for the policy under
consideration.
The Council agrees that it is important to look at how estimated VSLs depend on
characteristics of the individual (e.g., age, life expectancy), characteristics of the risk (e.g.,
latency, accompanying morbidity, voluntariness), and any other relevant factors. To the extent
that WTP may not be a precisely proportional function of the size of the risk change, it will also
be important to look more closely at the relationship between WTP estimates for different
studies, concerning different specified risk changes, and to assess whether the proportionality
assumption is generally tenable.
E.4. Problems with Meta-analyses
The meta-analysis in the Kochi paper, like many other meta-analyses, is premised on the
assumption that there is a simple VSL relationship that is merely revealed with different degrees
of bias and noise by different studies. At best, unfortunately, the underlying construct is
probably a complex VSL function. This function has many, many arguments. VSL is either
known or strongly suspected to depend on the nature of the risk (severity, latency, voluntariness,
etc.) and on the attributes of the individual who is considering this risk (age, gender, health
status, etc.). VSL is also likely to depend upon the manner in which the demand information
behind it is elicited (from self-selected employment decisions, housing choices, stated preference
surveys, etc.). If only this last source of heterogeneity existed, we might be confident that
techniques for pooling VSL estimates across studies would be a sensible exercise.
Unfortunately, we can be fairly confident that there is fundamental heterogeneity in preferences
with respect to risk, so that there is no reason, a priori, to expect that any summary statistic
across studies corresponds to any single underlying "true" VSL.
The distribution of VSLs to be "averaged" in a meta-analysis is an artifact of the range of
contexts (types of risks and affected populations) analyzed in the list of studies contributing to
the meta-analysis. If this distribution of contexts does not correspond to the context pertinent to
the environmental policy in question, then the "meta-analysis VSL" may have little to do with
people's willingness to pay the costs of this policy.
E.5. WTP and Incomplete Information
118
-------
It is important to recognize two explanations for why people's empirical decisions about
mortality risk may differ from conventional theory: a) the individuals may be ill-informed or
may make mistakes (e.g., cognitive errors), and b) the theory may be oversimplified or wrong. It
is likely that most people would like to make decisions in a way that optimizes their risk
reduction spending (i.e., equal marginal spending per unit risk reduction) across various domains
(e.g., housing, employment choices). However, they do not do so in practice because of
information limitations and well-known errors in decision making about risk.
Some published research has made an attempt to sort out which of the factors that lead to
differences between perceived risk and simple theory are simply cognitive errors (e.g.,
susceptibility to framing effects), and which are attributes of preferences potentially meriting
normative recognition (e.g., distribution of benefits and risks of activity; such as voluntariness)
(see Hammitt, 2000b).
In general, economists are inclined to defer to "consumer sovereignty" in measuring the
types of tradeoffs people are willing to make. In the event of misinformation or cognitive
problems, however, good policy should probably over-ride consumer errors where possible and
simulate what would have been consumers' WTP under similar conditions, but with complete
and accurate information.
E.6. What to do in the near term
The Agency needs to verify that the distribution of risk reductions over which each meta-
analysis has been estimated, and the context for these reductions, at least corresponds to the
types of risk reductions relevant to the Clean Air Act and its amendments. The Panel continues
to support meta-analyses of willingness to pay for risk reductions, but discourages the Agency
from leaving the impression that it is searching for a single one-size-fits-all VSL. Instead, it
should be a maintained hypothesis that heterogeneity matters. Heterogeneity should be ignored
only if it can be shown to be inconsequential. The benefits from mortality (and morbidity) risk
reduction attributed to a particular policy should be commensurate with the size and nature of the
risk reduction and with the attributes of the affected populations.
It seems worth speculating that researchers' habit of talking in terms of conventional
VSLs has much to do with the recent public relations problems concerning the "senior death
discount." This different VSL for seniors was embodied in the alternative net benefits
calculations associated with some recent analyses by the Agency. The public backlash to this
differential seems to have been attributable almost entirely to the use of the VSL concept, which
led the public to think that the issue at stake is the "value of a senior." In reality, the issue at
stake is much closer to "how much money should we as a society pay for small risk reductions
for seniors, and should it be the same as the amount paid for the same benefits for middle-aged
individuals and children." In particular, it is worth questioning whether we should, as a society,
oblige people through regulations to pay more for health risk reductions than they would choose
to pay if they were to buy these risk reductions privately, themselves. It is essential to steer the
press and the public towards the legitimacy of individual preferences and the corresponding
demands (consumer sovereignty), rather than sticking with the arbitrary unit choice that
expresses a marginal rate of substitution between risk changes and income as the "value of life."
119
-------
The word "value" is assumed by non-economists to be something intrinsic. Demand for risk
reductions is not intrinsic and immutable, independent of context. It is subjective and individual,
and measured differences in this demand across subpopulations and risk contexts should be
honored wherever they are verifiable and based on complete information about those risks.
If WTP for small risk reductions can be shown to be approximately proportional to the
size of these risk reductions over the relevant domain of the WTP function, the Panel believes it
would be less inflammatory to present the marginal rate of substitution expression in terms of
risk changes of a size that are pertinent to policy choices. The Panel recommends that the
Agency consider converting VSL estimates into units with a less potentially misleading
denominator (micromorts, millimorts, picomorts, etc.) and presenting these estimates in tandem
with ordinary VSL estimates, if not in lieu of them.
120
-------
APPENDIX F: SPECIFIC RESERVATIONS ABOUT THE USE OF QALYS IN THE
CONTEXT OF THE SECTION 812 ANALYSES
Members of the Council have articulated a number of specific reservations about the use
of QALYs in the context of the Section 812 Analyses. These reservations concern consumer
sovereignty and representativeness, ordinality versus cardinality, and heterogeneity in health
states. Details about these concerns follow.
F.I. Consumer sovereignty and representativeness:
Much progress has been made over the last dozen years in rendering QALY weights
more fully representative of general population preferences, but some of the assumptions they
require still trouble economists. There is no basis in economics for QALY weights based solely
on the opinions of experts. Consumer sovereignty is a hallmark of the economic framework for
benefit-cost analysis. The weights on different health states—used in the aggregation of a vector
of health state characteristics into a one-dimensional index of well-being—should be based on the
tradeoffs that a representative sample of consumers is willing to make between those states.
State-of-the-art QALY-weight estimates used to convert a bundle of health-state
attributes into a one-dimensional index now tend to be determined ex ante with respect to the
degraded health states in question, by random samples from the population of consumers, so
there is a greater expectation that these weights are representative. Departures from this strategy
are sometimes justified as approximations, but acknowledged to be conceptually inferior. The
Agency, if it elects to use QALYs in future cost-effectiveness calculations, should insist upon
weights that are based on general public/consumer preferences, rather than experts' opinions, and
that these weights reflect ex ante rather than ex post tradeoffs. This was a recommendation of
the Panel on Cost-Effectiveness (see Gold et al. 1996). Some members of the Council are
concerned, however, that there do not yet exist sufficient numbers of general-population
estimates of QALY weights for the Agency to be confident in any estimates it might use.
F.2. Ordinality versus cardinality.
Economics is clear that tradeoffs with respect to health need not necessarily be expressed
in terms of the marginal utility derived from a health attribute divided by the marginal utility
derived from money (which is the manipulation that produces a WTP estimate for changes in
health states). Any numeraire will do. The choice of a monetary unit is merely convenient.
WTP measures the rate of substitution between some change in a health state (or lottery over
health states) and income, where income is a measure of the consumption of "all other goods and
services." In contrast, QALY weights measure the rate of substitution between a change in
health state and length of life, so length of life is the numeraire. To this point, then, the marginal
rates of substitution in both the WTP and QALY approaches require only ordinality in
preferences. The subtle difference, however, is that while empirical QALY studies typically
elicit ordinal utility scales, they give the scale a real zero (i.e. death), which gives the scale ratio
properties,
121
-------
In the case of QALYs, the shift to a cardinal interpretation seems to come about in one of
two ways. The first is when practitioners want to add QALYs across people. This creates a need
to interpret QALYs as measuring interpersonally-comparable utilities, so that sums of QALYs
across people can remain consistent with utilitarian welfare. Many QALY practitioners clearly
treat QALYs as cardinal by adding utility across time and across individuals. Second, if
practitioners want to evaluate uncertain health risks by calculating expected QALYs, it seems
necessary to assume that QALYs reflect a von Neumann-Morgenstern utility function, which is
necessary for the expected value to be a meaningful summary of utility under uncertainty. (It is
of course also necessary to assume that expected utility theory is consistent with human
behavior). QALYs are derived from von Neumann-Morgenstern utility under uncertainty.
However, in practice, QALY calculations violate the postulates of expected-utility theory by
treating an ex ante interval utility scale as if it were an ex post ratio utility scale.
In the case of WTP calculations, it is not necessary to rely on direct interpersonal utility
comparisons, so it is likewise unnecessary to think about WTP as a cardinal utility measure.
WTP can be summed across individuals because the Kaldor-Hicks compensation principle
provides for this to be a way to identify potential Pareto improvements. In contrast, any analogy
to the idea of Pareto improvements is harder to apply to the QALY story, since the idea of
winners actually compensating losers by handing over some of their net improvements in health
seems like it would be impossible, even in principle. The analogy to the Kaldor-Hicks intuition
would still suggest that the net health gains of winners should exceed the net health losses to
losers. Across many simultaneous health-improvement policies with different distributions of
winners and losers, if net gains across all programs exceed net losses across all programs, society
as a whole would be better off in terms of health.
F.3. Heterogeneity in health states
QALY practitioners have focused on heterogeneity in health states and the desire for a
one-dimensional index of health that controls for this heterogeneity. WTP researchers have
emphasized utility-theoretic strategies in support of benefit-cost analysis, but early empirical
estimates did not distinguish between health states beyond just "alive" versus "dead." The latest
generations of empirical WTP analyses now incorporate information about disease types, age
differences, latencies in effects, comorbidity, and other types of heterogeneity. Most economists
would agree that the ideal approaches to both benefit-cost analysis and cost-effectiveness
analysis should include both adequate recognition of heterogeneity in health effects and a utility-
theoretic framework. QALY approaches are relatively strong on the first count, but lacking on
the second count. WTP approaches are strong on the second count, and gaining rapidly in terms
of the first.
F.4. Economic benefit analysis using QALYs?
A cul-de-sac in the QALY-WTP literature attempts to bring the medical decision-making
and economic approaches to efficiency questions somewhat closer together. Some QALY
researchers have considered the demand for the improved health states offered by different
policies, not just the costs of these improvements. Instead of just cost-effectiveness analysis,
something approaching a full benefit-cost analysis can be sought. See Hirth, et al. (2000),
122
-------
Hammitt (2002), Klose (2003), and Gyrd-Hansen (2003). Even in this endeavor, though, the
standardization of health units embodied in a QALY still tends to raise objections from
economists. One QALY-based WTP method has two steps: a) model QALYs as a function of a
wide array of health state attributes and calculate the non-economic cardinal QALY index for a
specified bundle of health attributes, then b) determine WTP for a QALY with the assumption
that each QALY has equal value. Some studies have also tried (incorrectly) to derive WTP per
QALY using VSL, or to regress WTP estimates on QALY estimates from the same sample.
However, there has been little sustained interest in using such estimates to evaluate health
outcomes.
In conducting a WTP analysis, it is reasonable to question whether an intervening QALY
step is even necessary. The economic approach is presently evolving to model WTP directly as a
function of heterogeneous health state attributes—in one step. This approach models WTP to
avoid a future health state as a function of the vector of attributes of that health state, allowing
inferences about the marginal WTP for distinct health state attributes, holding other attributes
constant. Forcing WTP to fit a QALY model seems to place unnecessary and perhaps
undesirable constraints on WTP. There is no reason why an individual would have to place the
same monetary value on every QALY. This implies linearity of WTP with respect to changes in
life expectancy, an assumption that does not appear to be supported empirically (e.g., Krupnick
et al., 2002).
123
-------
APPENDIX G: BIOSKETCHES OF MEMBERS OF THE SPECIAL
COUNCIL PANEL FOR THE REVIEW OF THE THIRD 812
ANALYSIS
Dr. Trudy Ann Cameron (Council Chair)
Dr. Trudy Ann Cameron is the Raymond F. Mikesell Professor of Environmental and
Resource Economics at the University of Oregon. She holds a Ph.D. in Economics from
Princeton University (*82), and was a member of the faculty in Economics at UCLA for
seventeen years before moving to UO in January of 2002. She has served as a member of the
board of directors, as well as vice-president, of the Association of Environmental and Resource
Economics, and as an associate editor for the Journal of Environmental Economics and
Management and the American Journal of Agricultural Economics. For the EPA's Science
Advisory Board, she has served on the Environmental Economics Advisory Committee and the
Economics and Assessment Working Group of the Children's Health Protection Advisory
Committee, and she now chairs the Advisory Council for Clean Air Compliance Analysis. Dr.
Cameron's research concentrates on the methodology of non-market resource valuation, with
special emphasis on econometric techniques for the analysis of stated preference survey data.
Her recent projects have included a study of popular support (i.e., willingness to pay) for climate
change mitigation programs (funded by the National Science Foundation). A current project,
begun at UCLA with former colleague JR DeShazo, uses stated preference survey methods to
elicit household choices that reveal willingness to pay to avoid illness, injury, and death. The
"value of a statistical life" is a key ingredient in the benefit-cost analysis of many environmental,
health, and safety regulations, and this project seeks to more clearly identify how the context of
such choices influences the public's willingness to pay for such policies.
Dr. David Allen
Dr. David Allen is the Gertz Professor of Chemical Engineering and the Director of the
Center for Energy and Environmental Resources at the University of Texas at Austin. His
research interests lie in environmental reaction engineering, particularly issues related to air
quality and pollution prevention. He is the author of four books and over 125 papers in these
areas. The quality of his research has been recognized by the National Science Foundation
(through the Presidential Young Investigator Award), the AT&T Foundation (through an
Industrial Ecology Fellowship) and the American Institute of Chemical Engineers (through the
Cecil Award for contributions to environmental engineering). Dr. Allen was a lead investigator
in one of the largest and most successful air quality studies ever undertaken: the Texas Air
Quality Study (www.utexas.edu/research/ceer/texaqs). His current research is focused on using
the results from that study to provide a sound scientific basis for air quality management in
Texas. In addition, Dr. Allen is actively involved in developing Green Engineering educational
materials for the chemical engineering curriculum. His most recent effort is a textbook on design
of chemical processes and products, jointly developed with the U.S. EPA. Dr. Allen received his
B S. degree in Chemical Engineering, with distinction, from Cornell University in 1979. His
M.S. and Ph.D. degrees in Chemical Engineering were awarded by the California Institute of
Technology in 1981 and 1983. He has held visiting faculty appointments at the California
124
-------
Institute of Technology, the University of California, Santa Barbara, and the Department of
Energy.
Ms. Lauraine G. Chestnut
Ms. Lauraine G. Chestnut, Managing Economist at Stratus Consulting Inc., is an
economist who specializes in the quantification and monetary valuation of human health and
environmental effects associated with air pollutants. She has 20 years of experience with Stratus
Consulting and its predecessors working for clients including the U.S. Environmental Protection
Agency, California Air Resources Board, Environment Canada, World Bank, and Asian
Development Bank, quantifying the damages of air pollution, including human health effects,
visibility aesthetics, materials damages, and crop damage. She has conducted original economic
and survey research to estimate the value to the public of protecting human health and visibility
aesthetics from the effects of air pollution. She has developed quantification models to estimate
the health benefits of reductions in air pollutants that have been used to assess the benefits of
provisions of the Clean Air Act in the U.S., proposed Canadian air quality standards, air quality
standards in Bangkok, and elsewhere. Ms. Chestnut has published articles related to this work in
Land Economics, Environmental Research, Journal of the Air and Waste Management
Association, and Journal of Policy Analysis and Management, and as chapters in the following
titled books: Valuing Cultural Heritage, Air Pollution and Health, and Air Pollution's Toll on
Forests and Crops. Ms. Chestnut managed an epidemiology and economic study of the health
effects of particulate air pollution in Bangkok, working closely with the Thai Pollution Control
Department, the School of Public Health at Chulalongkorn University, and the World Bank. Ms.
Chestnut co-authored publications on the Bangkok studies in the Journal of the Air and Waste
Management Association, Environmental Health Perspectives, American Journal of Agricultural
Economics, Journal of Exposure Analysis and Environmental Epidemiology. Ms. Chestnut
received a B.A. in economics from Earlham College, Richmond, Indiana, in 1975, and an M.A.
in economics from the University of Colorado, Boulder, in 1981. She is a member of the
Association of Environmental and Resource Economists and of the Air and Waste Management
Association.
Dr. John Evans
Dr. Evans is Senior Lecturer in Environmental Science at Harvard School of Public
Health, where he serves as co-director of the Program in Environmental Science and Risk
Management. He holds a B.S.E. (Industrial Engineering) and a M.S. (Water Resources
Management) from the University of Michigan and earned his S.M. and Sc.D. in Environmental
Health Sciences at Harvard. Dr. Evans has worked in the field of risk analysis for over twenty
years and has emphasized the importance of characterizing uncertainty in estimates of health
risks in his research. He has experience in uncertainty analysis and has conducted several studies
using formally elicited expert judgment to describe uncertainty in environmental health risks. His
recent work has examined the role of decision and value of information analysis in setting
priorities for environmental research. Dr. Evans has been a member of the Society for Risk
Analysis since it was founded; has served as the Chair of the New England Chapter, and as both
a member of the Editorial Board of the SRA's journal Risk Analysis and as an area editor of Risk
Analysis. He was a member of the NAS Committee on Estimating the Health Benefits of Air
125
-------
Pollution Regulations and also served on the EPA Science Advisory Board (Drinking Water
Committee). Dr. Evans' current research funding comes largely (over 90%) from the
Government of Kuwait. In the past his work has been funded by a number of sources, including
the US EPA Office for Research and Development, the Mexican Government (through
subcontracts with MIT), several corporations and individuals (through contracts with and/or gifts
to the Harvard Center for Risk Analysis), Health Canada, and the US Nuclear Regulatory
Commission.
Dr. Lawrence H. Goulder
Dr. Lawrence H. Goulder is the Shuzo Nishihara Professor in Environmental and
Resource Economics at Stanford University. He is also a Senior Fellow of Stanford's Institute
for International Studies and Institute for Economic Policy Research, a Research Associate at the
National Bureau of Economic Research, and a University Fellow of Resources for the Future.
He is a member of the EPA's Science Advisory Board's Environmental Economics Advisory
Committee. Dr. Goulder's research examines the environmental and economic impacts of U.S.
and international environmental policies. He has focused on policies to reduce emissions of
"greenhouse gases" that contribute to climate change, and on "green tax reform," revamping the
tax system to introduce taxes on pollution and reduce taxes on labor effort or investment. His
analyses of environmental policies often employ a general equilibrium analytical framework that
integrates the economy and the environment and links the activities of government, industry, and
households. His work considers both the aggregate benefits and costs of various policies as well
as the distribution of policy impacts across industries, income groups, and generations. Some of
his work is interdisciplinary, involving collaborations with climatologists and biologists. Dr.
Goulder graduated from Harvard College with an A.B. in philosophy in 1973. He obtained a
master's degree in musical composition from the Ecole Normale de Musique de Paris in 1975
and earned a Ph.D. in economics from Stanford in 1982.
Dr. James K. Hammitt
James K. Hammitt is Associate Professor of Economics and Decision Sciences and
Director of the program in Environmental Science and Risk Management at the Harvard School
of Public Health. His teaching and research concern the development and application of
quantitative methods—including benefit-cost, decision, and risk analysis—to health and
environmental policy in both industrialized and developing countries. Research interests include
the management of long-term environmental issues such as global climate change and
stratospheric-ozone depletion, the evaluation of corollary benefits and countervailing risks
associated with risk-control measures, and the characterization of social preferences over health
and environmental risks using revealed-preference and contingent-valuation methods. Professor
Hammitt is a member of the National Academy of Sciences Committee on Implications of
Dioxin in the Food Supply, the American Statistical Association Committee on Energy Statistics
(the Advisory Committee to the US Energy Information Administration), and the National
Science Foundation panel for Decision, Risk and Management Science. He holds degrees in
Applied Mathematics (A.B., Sc.M.) and Public Policy (M.P.P., Ph.D.) from Harvard University.
Previously, he was Senior Mathematician at the RAND Corporation and on the faculty of the
RAND Graduate School of Policy Studies.
126
-------
Dr. Dale Hattis
Dr. Dale Hattis is Research Professor with the Center for Technology Environment and
Development (CENTED) of the George Perkins Marsh Institute at Clark University. For the past
twenty-seven years he has been engaged in the development and application of methodology to
assess the health ecological and economic impacts of regulatory actions. His work has focused
on the development of methodology to incorporate interindividual variability data and
quantitative mechanistic information into risk assessments for both cancer and non-cancer
endpoints. Specific studies have included quantitative risk assessments for hearing disability in
relation to noise exposure, renal effects of cadmium reproductive effects of ethoxyethanol,
neurological effects of methyl mercury and acrylamide, and chronic lung function impairment
from coal dust four pharmacokinetic-based risk assessments for carcinogens (for
perchloroethylene ethylene oxide butadiene and diesel parti culates), an analysis of uncertainties
in pharmacokinetic modeling for perchloroethylene, and an analysis of differences among
species in processes related to carcinogenesis. He has recently been appointed as a member of
the Environmental Health Committee of the EPA Science Advisory Board and for several years
he has served as a member of the Food Quality Protection Act Science Review Board. Currently
he is also serving as a member of the National Research Council Committee on Estimating the
Health-Risk-Reduction Benefits of Proposed Air Pollution Regulations. The primary source of
his recent cooperative agreement support is the U.S. Environmental Protection Agency and
specifically the Office of Research and Development's National Center for Environmental
Assessment. This research includes: (1) Age related differences in susceptibility to
carcinogenesis; towards a quantitative analysis of empirical data. Instrument number (Term:
April 2002-Sept 2003); (2) Methods for evaluating human interindividual variability regarding
susceptibility to particulates (Term Sept 98~September 2002); and (3) also funding from the
State of Connecticut to work on Child/Adult differences in pharmacokinetic parameters, as a
subcontractor as part of a cooperative agreement. He has been a councilor and is a Fellow of the
Society for Risk Analysis and serves on the editorial board of its journal Risk Analysis. He
holds a Ph.D. in Genetics from Stanford University and a B.A. in biochemistry from the
University of California at Berkeley.
127
-------
Dr. F. Reed Johnson
Dr. F. Reed Johnson is Principal Economist at Research Triangle Institute. He was
recently named as one of the first four RTI Fellows. He has served on the economics faculties of
Illinois State University, Simon Fraser University, the Stockholm School of Economics, the
University of Stockholm, Linkoping University, and the U.S. Naval Academy. He currently is
Adjunct Professor of Public Policy at the University of North Carolina at Chapel Hill. He is also
a member of RTI's Scientific Advisory Council. From 1994 to 2001 he was Vice President for
Research and Development at Triangle Economic Research. He previously worked as an
economist in the Office of Policy Analysis, U.S. Department of the Interior, and in the Office of
Policy, Planning, and Evaluation, U.S. Environmental Protection Agency.
Dr. Johnson received his B.A. degree in economics from Occidental College in 1970 and
his Ph.D. degree in economics from the State University of New York, Stony Brook in 1974. He
has been awarded a Brookings Economic Policy Fellowship and two Fulbright-Hayes
scholarships to Sweden. As a staff member in the U.S. Environmental Protection Agency's
environmental economics research program during the 1980s, Dr. Johnson helped pioneer
development of basic nonmarket valuation techniques. These techniques are now widely used
for benefit-cost analysis in health and environmental economics. He has designed and analyzed
numerous surveys for measuring willingness to pay for health-risk reduction and improved
environmental quality. His current research includes developing improved conjoint analysis
methods for quantifying patient and physician preferences for health-care interventions and
health risks.
Dr. Charles Kohtad
Charles Kolstad is the Donald Bren Professor of Environmental Economics and Policy at
the University of California, Santa Barbara, where he is jointly appointed in the Department of
Economics and the Bren School of Environmental Science and Management. Most of Prof.
Kolstad's research has been in the area of regulation, particularly environmental regulation.
Recently, he has also done work in environmental valuation theory. He is particularly interested
in the role of information in environmental decision-making and regulation. Currently he has a
major research project on the role of uncertainty and learning in controlling the precursors of
climate change. His past work in energy markets has focused on coal and electricity markets,
including the effect of air pollution regulation on these markets. Prof. Kolstad is the editor of
Resource and Energy Economics, has been an Associate Editor of the Journal of Environmental
Economics & Management (JEEM), and is currently on the editorial board of Land Economics
and JEEM. Dr. Kolstad is the president of the Association of Environmental and Resource
Economists(AERE). He has also served on AERE's Board of Directors. With over 100
publications, he has published in a variety of journals including the American Economic Review,
Journal of Political Economy, Review of Economic Studies, Review of Economics and Statistics,
Land Economics and The Journal of Environmental Economics and Management (JEEM). He
received his Ph.D. from Stanford (1982), his M.A. from Rochester and his B.S. from Bates
College.
128
-------
Dr. Lester B. Lave
Dr. Lester B. Lave is University Professor and Higgins Professor of Economics at
Carnegie Mellon University, with appointments in the Business School, Engineering School, and
the Public Policy School. Reed College granted him a B.A. and Harvard University a Ph.D. in
economics. His research has focused on health, safety, and environmental issues, from the effect
of air pollution on mortality to estimating the benefits and costs of automobile safety standards,
risk analysis of carcinogenic chemicals, testing the carcinogenicity of chemicals, valuing natural
resources and global climate change. As a Senior Fellow at the Brookings Institution from 1978-
1982, he investigated a variety of regulatory and risk analysis issues. Lave has served as a
consultant to a large number of federal and state agencies, as well as corporations. He was
elected to the Institute of Medicine of the National Academy of Sciences, is a past president of
the Society for Risk Analysis, and has served on many committees of the National Academy of
sciences, AAAS, American Medical Association, and Office of Technology Assessment. Dr.
Lave is the director of the Carnegie Mellon University university-wide Green Design Initiative
(Practical Pollution Prevention). This program is focused on using pollution prevention and
sustainable development to boost economic development. The program has partnerships with
leading companies to address these issues and design produces and processes for the
environment. Although it is only four years old, the program has already received extensive
support from IBM, the National Science Foundation, then Department of Energy, the
Environmental Protection Agency, Texaco, the American Plastics Council, AT&T, Xerox, NCR,
General Motors, Ford, Chrysler, Union Carbide, Alco, and other industrial Companies. Lave is
also a principal in the Carnegie Mellon Global Change Center sponsored by NSF.
Dr. Virginia McConnell
Dr. Virginia D. McConnell is currently Senior Fellow at Resources for the Future and
Professor of Economics at the Baltimore Campus of the University of Maryland (UMBC). She
is currently a member of several EPA Advisory Committees, including the EPA Clean Air Act
Advisory Committee, Subcommittee on Mobile Sources Technical Review, and the Chesapeake
Bay Program Advisory Committee, Air Subcommittee. She recently served on a National
Academy of Sciences Panel, Board on Environmental Studies and Toxicology, to evaluate
vehicle emission inspection programs. In the past, she worked with the President's Commission
on Environmental Quality, and was awarded a Gilbert White Fellowship at Resources for the
Future. She received a B.A. in Economics from Smith College in 1969 and Ph.D. in Economics
from the University of Maryland in 1978. Her research interests are in the general area of air
pollution and urban transportation, and more recently on the link between urban growth,
transport and the environment. She has just completed work on a review article on 'Vehicles and
the Environment' for the International Yearbook of Environmental and Resource Economics.
Her published work has focused on evaluation of policies and policy design for the reduction of
vehicle pollution; analysis of the productivity effects of environmental regulations; the effect of
environmental regulations on firm location; and transport externalities and urban structure. In
addition, she is currently studying the role of economic incentive policies for achieving goals of
more efficient urban growth.
129
-------
Dr. D. Warner North
Dr. D. Warner North is president and principal scientist of North Works, Inc., a consulting
firm in Belmont, California, and consulting professor in the Department of Management Science
and Engineering at Stanford University. Over the past thirty years Dr. North has carried out
applications of decision analysis, risk analysis, and benefit-cost analysis for electric utilities in
the US and Mexico, for the petroleum and chemical industries, and for US government agencies
with responsibility for energy and environmental protection. He has served as a member and
consultant to the Science Advisory Board of the US Environmental Protection Agency since
1978, and as a Presidentially-appointed member of the US Nuclear Waste Technical Review
Board (1989-1994). Dr. North is a co-author of many reports dealing with environmental risk
for the National Research Council of the National Academy of Sciences, including "Risk
Assessment in the Federal Government: Managing the Process" (1983), "Improving Risk
Communication" (1989),"Science and Judgment in Risk Assessment" (1994), and
"Understanding Risk: Informing Decisions in a Democratic Society" (1996). Dr. North was a
member of the Board on Radioactive Waste Management of the National Research Council from
1995 until 1999. He was the chair for the steering and advisory committees for the International
Workshop on the Disposition of High-Level Radioactive Waste, held November 4-5, 1999, and
leading to the National Research Council report, "Disposition of High-Level Waste and Spent
Nuclear Fuel: The Continuing Societal and Technical Challenges," published in June 2001. Dr.
North is a past president (1991-92) of the international Society for Risk Analysis, a recipient of
the Frank P. Ramsey Medal from the Decision Analysis Society in 1997 for lifetime
contributions to the field of decision analysis, and the 1999 recipient of the Outstanding Risk
Practitioner Award from the Society for Risk Analysis. Dr. North received his Ph.D. in
operations research from Stanford University and his B.S. in physics from Yale University.
Dr. Bart Ostro
Bart Ostro, Ph.D., is currently the Chief of the Air Pollution Epidemiology Unit, Office
of Environmental Health Hazard Assessment, California Environmental Protection Agency. His
primarily responsibilities are to formulate the Agency's recommendations for state ambient air
quality standards and to investigate the potential health effects of criteria air pollutants. His
previous research on mortality and morbidity effects of air pollution, has contributed to the
determination of federal and state air pollution standards for ozone and particulate matter. Dr.
Ostro was also a co-author of the EPA regulatory impact analysis that was a basis for the federal
ban of lead in gasoline. Dr. Ostro has served as a consultant with several federal and
international institutions including the World Health Organization and the World Bank, and with
several foreign governments including Mexico, Indonesia, Italy, the European Union, Thailand,
and Chile. H e currently serves on the National Academy of Sciences' Committee on Estimating
the Health Risk Reduction Benefits of Proposed Air Pollution Regulations, and is on the
Scientific Oversight Committee for ATHENA (Air Pollution Health Effects in Europe and North
America) for the Health Effects Institute. Dr. Ostro received a Ph.D. in Economics from Brown
University and a Certification in Environmental Epidemiology from the State of California. He
has published over 60 articles on air pollution epidemiology and environmental economics in
peer reviewed journals. His current research interests involve conducting epidemologic studies
on the mortality and morbidity effects of criteria air pollutants, examining the health effects of
130
-------
traffic, and quantifying the health benefits and associated uncertainties related to air pollution
control.
Dr. V. Kerry Smith
Dr. V. Kerry Smith is University Distinguished Professor and Director, Center for
Environmental and Resource Economic Policy in the Department of Agricultural and Resource
Economics at North Carolina State University, and he is a University Fellow in the Quality of the
Environment Division of Resources for the Future. Since October 2000 he has been a member
of the Advisory Council on Clean Air Compliance Analysis of the U.S. Environmental
Protection Agency's Science Advisory Board, and in 2001 he was a member of the Arsenic Rule
Benefits Review Panel of EPA's SAB. Dr. Smith received his AB in Economics from Rutgers
University in 1966 and his Ph.D. in Economics there in 1970. He presented the Federick V.
Waugh Lecture for the American Agricultural Economics Association in 1992, and at the 2002
AAEA annual meeting he was named an association fellow, the association's most prestigious
honor. In addition to the AAEA , he is a member of the American Economic Association, the
Southern Economic Association, the Association of Environmental and Resource Economists,
and numerous other professional associations. He has held editorial positions with the Journal of
Environmental Economics and Management, Land Economics, Review of Economics and
Statistics, and other professional journals. His research interests include non-market valuation of
environmental resources, role of public information in promoting private risk mitigation,
environmental policy and induced technical change, non-point source pollution and nutrient
policy.
Dr. Thomas Wallsten
Dr. Thomas S. Wallsten is a professor in the Department of Psychology and in the
Program in Cognitive Science and Neuroscience. He received his Ph.D. from the University of
Pennsylvania in 1969, did a postdoctoral fellowship at the University of Michigan in 1970, and
then joined the faculty at the University of North Carolina, Chapel Hill. He was professor of
psychology and director of the Cognitive Science program when he left UNC-CH in 2000. Over
the past years he was a visiting professor or visiting scholar at the University of Chicago, Duke
University, Haifa University in Israel, and University of Oldenburg in Germany. He is a
mathematical and cognitive psychologist with expertise in subjective probability, judgment,
choice, decision behavior, and related areas of decision science and cognitive psychology. His
current research focuses on subjective probability encoding and representation, communication
of opinion, and human information processing under uncertainty. This research has been
supported over the past 30 years primarily by grants from the National Science Foundation
(NSF), with occasional additional support from other agencies. Current grants are from NSF and
the Air Force Office of Scientific Research. Among his advisory roles, he was editor of the
Journal of Mathematical Psychology from 1990-1994, associate editor of Psychometrika from
1984-1988, associate editor of the Journal of Experimental Psychology: Learning, Memory, and
Cognition from 2000-2003, and on numerous editorial boards. He served in various advisory
roles for NSF: During 1995-1997 on the grant review panel for Methodology, Measurement, and
Statistics Program in the Division of Social, Behavioral, and Economic Research; in 2000 as a
member of the Committee of Visitors for Social, Behavioral, and Economic Sciences
131
-------
Directorate; in 2003 as a member of the Committee of Visitors for the Behavioral and Cognitive
Sciences Directorate; in 1998 on an ad hoc NSF-EPA grant review panel. In 2002, he was a grant
review panel member for the Cognition and Student Learning Program of the Department of
Education Office of Educational Research and Improvement.
132
------- |