United States
Environmental Protection
Agency
Office of
Research and Development
Washington DC 20460
EPA-600/7-81-001
January 1981
Office of Environmental Engineering and Technology
Review of the
Department of Energy's
and
Solar Energy Programs
A Report to the
President and Congress
Prepared by the Environmental
Protection Agency under
direction of Section 11 of the
Federal Nonnuclear Energy
Research and Development Act
(Public Law 93-577)
-------
-------
-------
UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON, D.C. 20460
THE ADMINISTRATOR
LETTER OF TRANSMITTAL
January 1981
I am pleased to submit herewith, to the President of the United
States, the Speaker of the House of Representatives, and the
President of the Senate, the Environmental Protection Agency's Report
to the President and Congress in accordance with the mandate
contained in Section 11 of the Federal Nonnuclear Energy Research and
Development Act (Public Law 93-577).
The Act requires the Agency to conduct a "continuing analysis of
the effect of the application of nonnuclear energy technologies to
evaluate: (a) the adequacy of attention to energy conservation
methods, and (b) the adequacy of attention to environmental
protection and the environmental consequences of the application of
energy technologies."
This Report sets forth the Agency findings concerning the
adequacy of attention by the Department of Energy to energy
conservation. "Adequacy of attention" to energy conservation for the
1980 Section 11 program was measured in three ways: adequacy of the
resource allocation process, adequacy of implementation plans and
management processes and adequacy o-f evaluative information for
deci sion-maki ng.
We have sought to define both the scope and context of these
issues. The Report reflects the full range of opinions available to
the Agency, incorporating the ideas and thoughts of interested
individuals obtained through public hearings mandated by the Act.
I trust that the Report proves fully satisfactory. If such is
not the case, or if there are any additional questions or comments,
please contact meฎ
as M. Costle
-------
Table of Contents
Page
Executive Summary 1
Introduction 7
The Section 11 Mandate 9
1980 Section 11 Program Focus 9
1980 Section 11 Program Activities 10
Conclusions and Recommendations 11
Definition of Adequacy of Attention 13
Resource Allocation Process 13
Implementation and Management 14
Evaluative Information 14
The Role of Government in the Energy Market 14
Background: The Nature of the Energy Marketplace 14
Government's Role in Conservation 15
Assessment of the Resource Allocation Process 17
Background 17
End-Use Sector Framework 18
Analytical Tools 19
The Least-Cost Criterion 19
The Oil Import Premium 20
Assessment of Implementation and Management 22
Background 22
Legislative History 22
Organizational History 22
Characteristics of Conservation and Solar Programs 24
Integration of Conservation and Solar Programs 24
Coordination of Information Programs 25
-------
Page
Appropriate Management of State and Local Programs 28
Partnership 28
Need for Comprehensive Review 31
Program Specific Issues 31
Weatherization Assistance Program 32
Residential Conservation Service 34
Institutional Buildings Conservation Program 35
Appropriate Management of Research and Development 36
Innovation 36
Procurement and Financial Assistance 38
Field Activities 39
Assessment of Program Evaluation 41
Background ; 41
Need for Evaluation 41
Current DOE Evaluation Activities 42
Uses of Evaluation 43
Evaluation Policy 44
Institutional Barriers to Implementation 47
Methodological Barriers to Implementation 48
Appendices 51
References inside back cover
IV
-------
Executive Summary
-------
Executive Summary
Background
The Federal Nonnuclear Energy Research and Development
Act of 1974 was enacted to support a broad range of energy
technologies. In creating this program, Congress demon-
strated its concern that conservation and environmental
protection receive adequate attention in Federal energy re-
search and development. To ensure that this intent was
carried out, it included a requirement in the Act (Section 11,
Public Law 93-577) for "a continuing analysis of the effect of
the application of nonnuclear energy technologies to evalu-
ate:
(1) The adequacy of attention to energy conservation
methods; and
(2) The adequacy of attention to environmental protection
and the environmental consequences of the application of
energy technologies."
In carrying out the requirements of Section 11, during the
past two years the Environmental Protection Agency (EPA)
has focused on the environmental aspects of the mandate. In
1980, the Section 11 review shifted its emphasis to assess the
adequacy of attention to energy conservation. Because of the
close legislative, organizational, and programmatic ties, solar
energy was also included in the review, though to a lesser
degree.
Energy conservation can have several distinct meanings
and a variety of Federal programs have been developed to
correspond to them. Conservation can be interpreted as:
Increased energy efficiency such as using a car with better
mileage;
Minor lifestyle changes, such as joining a car pool; or
Curtailment, such as reducing driving.
In the past, conservation has been most often equated with
curtailment or sacrifice"freezing in the dark." However, as
the costs of energy supplies rise, EPA found that conservation
is being viewed increasingly as a positive supplement to
conventional supplies by all sectors of society. Many indi-
viduals, industrial corporations, and utilities are beginning to
perceive investments in conservation and renewables as
cost-effective alternatives to paying the rising prices of en-
ergy. For example, Southern California Edison recently de-
cided to postpone construction of two new coal plants and to
try to meet increased demand through conservation and
renewable resources.
In the 1980 review, EPA examined the Department of
Energy's conservation activities from three perspectives:
Adequacy of the resource allocation process;
Adequacy of implementation and management processes;
and
Adequacy of evaluative information on program effective-
ness.
We also looked at the context within which DOE operates,
including the role of the Federal government in the energy
market, and the legislative and organizational history of
conservation and solar programs. Some of the issues included
in the 1980 program arose from concerns raised during our
1979 review. These include the need for resource allocation to
be based on systematic comparisons of conservation and
supply options, and -the inappropriateness of certain DOE-
wide management systems for conservation and solar pro-
grams.
This Report is the result of EPA analysis combined with
an extensive program of public participation. In addition to a
public hearing, the review included five meetings on specific
topics. Over 150 people attended these meetings represent-
ing state and local governments, industry, labor, research
organizations, and public interest groups. Congressional
committee staff and DOE representatives also participated in
these meetings. Appendix A contains summaries of these
five meetings and the National Hearing.
-------
Major Conclusions
and Recommendations
The adequacy of attention to conservation must be examined
in the context of overall Federal energy policy and the role of
the Federal government in the energy market.
Energy is produced and consumed in a market that is
heavily dependent on imported oil and that is characterized
by a number of imperfections such as prices which do not
reflect total costs of energy production and subsidies of
certain fuels. Further, lack of information and other barriers
hamper consumer decision making on energy use.
In order to overcome these imperfections and ensure that
the rate at which the private sector invests in energy conser-
vation is consistent with the national interest, there is a need
for a continuing Federal involvement in providing consumers
with information and, as appropriate, financial and tax'incen-
tives which will attempt to compensate for imperfections,
thereby allowing renewable resource and conservation in-
vestments to compete fairly in the marketplace.
Resource Allocation Process
In the past, DOE has emphasized increasing the supply of
energy without a thorough examination of the needs that give
rise to the demand for energy. DOE should analyze alternative
ways of meeting the energy service needs of end-use sectors
by systematically comparing supply enhancement with de-
mand reduction options. Use of an "end-use" framework
would enable DOE to examine the various sectors that ulti-
mately use energy, the needs of consumers in those sectors
and the services that can satisfy those needs. This analytic
framework would also sharpen DOE's current analyses of fuel
supplies because it asks directly: what fuels, and how much
or how little fuel for what specific services7
DOE should take the following steps to integrate an
end-use framework into its planning and budget system and
future evaluation efforts:
The Energy Information Administration should calculate
the current and projected energy service needs in all sec-
tors;
DOE's Office of Policy and Evaluation should explore the
complete range of alternatives available to meet end-use
sector service needs and lay out those alternatives;
Analyses of major budget issues should contain an ap-
praisal of how the issues affect end-use sectors' energy
services; and
The Third National Energy Plan should address end-use
sector needs directly, with an even-handed treatment of
how supply enhancement and demand reduction options
can contribute to meeting them.
Within the end-use framework, DOE needs analytical
tools to guide its cross-technology comparisons. The least-
cost criterion and oil import premium should be applied to
DOE's programs as part of the budgeting process.
The "least-cost criterion" calls for allocating resources to
support the development and adoption of those conservation
or supply options that will provide energy services at the
lowest economic, social, and environmental cost. The oil
import premium is a tool that enables analysts to estimate the
strategic, economic, and social cost to the nation of depend-
ence on foreign oil. When the premium is applied, it adds
another dimension to cost-effectiveness projections and ena-
bles analysts to take into account the value of reducing oil
imports beyond the market price per barrel.
The least-cost approach provides a useful mechanism for
arraying and systematically comparing conservation and
supply options. Therefore, it should be applied to the resource
allocation process wherever possible. This analytical tool
should be applied in addition to the other criteria used in
resource allocation (e.g., the private sector's capability to
undertake the task and the need for Federal intervention).
Specifically DOE should'
Conduct a study to identify the costs and benefits of
alternative strategies, and
Develop a data base which includes environmental effects
to enable DOE to apply the least-cost criterion and oil
import premium at a minimum to proposed investments in
conservation and supply technologies at the commercial
development stage.
Implementation and Management
The Section 11 review identified two factors that have con-
tributed to the overall implementation and management
problems of DOE's conservation and solar programs: lack of
coordination and integration of related programs and
inappropriate department-wide management systems.
Both of these elements grew out of the legislative and organi-
zational history of Federal involvement in conservation and
solar energy, and out of the characteristics of conservation
and solar programs that distinguish them from other energy
programs.
4
-------
Conservation and Solar Program Integration. Rapid
and piecemeal growth, combined with the decentralized na-
ture of solar and conservation programs, have contributed to a
lack of integration and coordination of conservation and solar
programs. DOE has taken a significant step by combining
conservation and solar programs under one Assistant Secre-
tary. However, further integration of these programs can and
should be accelerated without further reorganization within
the Office of Conservation and Solar Energy (CSE).
The focus of efforts to improve the integration of conser-
vation and solar programs should be on tasks that involve the
joint participation of programs offices in comparable areas
(e.g., buildings). Cooperative projects should be expanded
over a two to three year period until a de facto integration by
end-use sector has been accomplished.
Information Programs. Federal information dissemination
programs present an example of the need for coordination of
separate programs and integration of conservation and solar
efforts. DOE should attempt to create an integrated conserva-
tion and solar information network within the context of
DOE's overall information program, and should specify that
DOE information programs include both conservation and
solar information Further, there is currently overlap and con-
fusion among various information programs. DOE needs to
define explicitly the roles of each of the major institutions
involved in information dissemination.
DOE has recently taken several steps to alleviate this
problem: a CSE Information Steering Committee has been
formed, and oversight of information and outreach activities
in CSE has been assigned to a newly established office under
the Deputy Assistant Secretary for Field Operations and
International Programs.
State and Local Grant Programs. The role of the Federal
government in bringing about immediate energy conserva-
tion improvements is unlike any other function that Federal
energy programs have served in the past, because of the
highly decentralized energy conservation process and the
interdependence of Federal, state and local governments.
DOE should assess the strengths and weaknesses of the
Federal, state and local levels in energy conservation and
design strategies which build upon the particular strengths of
each member of this partnership.
making toward national objectives. A decision should be
made during the review process about whether this allocation
of resources offers a sensible balancing of conservation and
equity considerations.
Research, Development and Applications. DOE's man-
agement systems are based largely on the experience of its
predecessors the Atomic Energy Commission, the Energy
Research and Development Administration, and the Federal
Energy Administration. These agencies dealt primarily with
large scale, centralized technologies, and so some of their
management approaches and procedures have been inap-
propriate for conservation and solar programs. There has been
some recognition in DOE and the Congress for different proc-
esses. For example, the Approriate Technology Grant Pro-
gram was created to encourage innovation among small,
independent research organizations and inventors. There-
fore, it has a simplified application process and a more rapid
award process.
If DOE is to make a serious commitment to conservation
and solar innovation, the Appropriate Technology (AT) pro-
gram should receive increased emphasis. DOE should also
develop systematic mechanisms for bringing the results of
the Appropriate Technology and Energy-Related Inventions
programs to the attention of DOE program managers and for
channeling unsolicited proposals to these programs. Finally,
DOE should consider establishing specialized solicitations in
the AT program and should make technical assistance avail-
able to grantees.
DOE's procurement system presents particularly difficult
problems for conservation and solar R&D. Alternatives to this
system such as the use of cooperative agreements are avail-
able and should be employed more regularly.
Finally, DOE inherited a field structure of national labora-
tories from its predecessor agencies. DOE should develop an
explicit policy of the use of these mutiprogram laboratories in
the conservation effort. DOE has supplemented this network
and added the Solar Energy Research Institute and the Re-
gional Solar Energy Centers. SERI and the RSEC's should be
given more flexibility in the conduct of their operations.
Evaluation*
DOE is taking steps to improve the existing state and
local programs, although concerns remain in a number of
specific areas. It should also step back and review whether
the state and local programs taken as a whole are adequate to
the task of encouraging conservation. The impending Sunset
Review should examine comprehensively the state and local
programs, including the relative resource allocation among
programs and how the grant programs fit into national energy
"onservation priorities. The review should examine the pro-
rams not only on a case-by-case basis but more importantly
lust assess the overall progress which the programs are
Adequate information on factors affecting energy use, the
achievements of energy conservation programs and the per-
formance of specific DOE projects has not been available to
* Because the term "evaluation" is often used loosely to describe any formal or informal
assessment of actual or anticipated results of a program, rt is important to provide a
working definition For purposes of thrs discussion, evaluation is viewed as a formal,
systematic study of the actual processes or results of a program Evaluation is a
retrospective or historrcal activity, examrnrng what has occurred or is occurring It is
therefore clearly different from projectrons, forecasts, and other studres of proposed or
hypothetical programs
-------
Congress, DOE officials and the public. Because of this lack of
information, it has been difficult for Congress and DOE to
decide how much emphasis to place on conservation pro-
grams relative to supply enhancement programs, and which
particular conservation programs deserve strongest support.
Although program evaluation will not provide all of the an-
swers to these critical questions, it will make a major contri-
bution. Therefore, DOE should take the steps necessary to
evaluate systematically its major conservation and supply
programs and incorporate this information into its ongoing
management processes.
The Office of Conservation and Solar Energy has begun
work on developing a comprehensive plan for evaluating its
programs. In addition, evaluations of a number of conserva-
tion and solar programs are currently planned or in progress.
Evaluation is time-consuming, expensive and difficult to
implement in many instances. The Section 11 review found
virtually unanimous agreement, however, that it was essen-
tial. Therefore, EPA strongly recommends greatly increased
use of evaluation. However, we have tried not to paint an
overly optimistic picture. Instead, EPA has tried to develop
balanced recommendations, which cite the benefits of evalu-
ation and also raise the practical considerations that must be
addressed if evaluation efforts are to be successful.
If DOE is to produce consistent information on program
effectiveness, department-wide policy should be established
regarding types of programs to be evaluated and criteria for
program selection, department-wide information require-
ments, timing, roles and responsibilities of various offices,
and funding mechanisms. All offices with a stake in the
evaluation process should be involved in developing this
policy, including state and local agencies and utilities.
Successful evaluation requires an environment that is
receptive to new information and open to change. To foster
this type of environment, top DOE management must dem-
onstrate their commitment to using evaluation results and
must develop incentives for good evaluation at all levels of the
department such as:
ซ Rewarding program managers for conducting high-quality
evaluations and acting upon their results;
Establishing evaluation as a high priority activity within
the Department; and
Providing opportunities for users of evaluative information
to learn more about evaluation.
Two disincentives to evaluation are inflexible legislative
requirements that make it difficult for managers to change
programs in response to evaluation results, and nationwide
implementation of programs (such as information dissemina-
tion, standards, or state grants) without pilots or demon-
strations. Changes in these two areas would encourage
greater use of evaluation.
Finally, evaluations should focus not only on program
impact but should also examine program processes.
Evaluators should employ both quantitative and qualitative
techniques in these assessments, and tailor evaluation de-
signs to individual programs.
The 1981 Section 11 Program
This Report is part of an ongoing process. The 1981 Section 11
review will also focus on conservation and solar energy pro-
grams. In part, it will monitor the response to the 1980 pro-
gram's conclusions and recommendations.
There are opportunities in the near future wherein DOE
can demonstrate if any of the changes suggested in this
Report are useful. These opportunities are cited in various
parts of the Report and include planning and budgeting
documents (required in the Planning, Programming and
Budgeting System), the third National Energy Plan (due in
April/May 1981), and DOE's response to the Sunset Provisions
(Title X) of the DOE Organization Act (due in January 1982).
This Report is organized to allow the reader to review
individual sections without loss of continuity. One cross-
cutting theme that appears throughout the Report is that
DOE knows much about energy supply but not enough about
energy use. This departmental bias could be corrected with-
out significant organizational changes within DOE.
-------
Introduction
-------
Introduction
The Section 11 Mandate
The Federal Nonnuclear Energy Research and Development
Act (Public Law 93-577) was passed in December 1974 as part
of the national response to the effects of the 1973 Arab oil
embargo. The legislation created a comprehensive national
research, development, and demonstration (RD&D) program
for nonnuclear energy technologies, with total Federal in-
vestments of at least $20 billion over a 10-year period. Further,
it required the development of the " technological
capabilities to support the broadest range of energy policy
options through conservation and the use of domestic re-
sources by socially and environmentally acceptable means."
To ensure that this intent is reflected in the nonnuclear RD&D
program, Section 11 of Public Law 93-577 directs an annual
review of
... the adequacy of attention to energy conserva-
tion methods and environmental protection ...
and the environmental consequences of the appli-
cation of technologies.
The Office of Environmental Engineering and Technol-
ogy within the Environmental Protection Agency's (EPA)
Office of Research and Development has been assigned
responsibility for the review and is charged with conducting
annual public hearings and preparing this Report to the
President and Congress.
1980 Section 11 Program Focus
For the past two years, the Section 11 review emphasized the
environmental aspects of the mandate. In 1980, the review
has concentrated on energy conservation issues. It also has
included solar energy technologies because of their organi-
zational, legislative, and programmatic links with conserva-
tion.
Conservation can have several meanings and take sev-
eral distinct forms.
The first meaning for conservation is energy efficiency.
This signifies achieving the same result or obtaining the
identical service by using less energy. In this instance energy
efficiency would mean driving to work in an automobile that
has a higher mileage rating.
A second meaning for conservation is life style adjust-
ments that do not necessarily require any reduction in
activities. An example of "conservation by adjustment"
would be switching from single passenger driving to joining a
car pool, van-pool, or using mass transit.
Finally, it can signify using less energy by curtailing
various activities. An obvious example of "conservation as
curtailment" is the plea to reduce driving in order to use less
gasoline.
Most Federal conservation programs are aimed at using
energy more efficiently or saving energy with modest altera-
tions in lifestyle. As a rule, they do not involve curtailment or
sacrifice. Further, in recognition of the fact that economic
consequences of increased prices will be particularly severe
for low income consumers, some Federal programs are aimed
at subsidizing conservation measures for these groups to
alleviate the rising cost of energy.
One of the purposes of the Department of Energy, accord-
ing to its Organizational Act, is "to create and implement a
comprehensive energy conservation strategy that will receive
the highest priority in the national energy program" (em-
phasis added). The 1980 Section 11 review has assessed the
adequacy of attention to conservation and solar energy pro-
grams in three general areas: adequacy of the resource
allocation process to reflect the potential contributions of
Federal conservation programs, adequacy of implementation
plans and management processes, and adequacy of informa-
tion on program effectiveness.
EPA has not attempted to review comprehensively all of
DOE's Conservation and Solar programs. Instead, we have
-------
focused on several areas in which concerns have been raised
in the 1979 Section 11 Program, and other recent studies, or
in our early discussions with program participants. Thus, our
review of resource allocation focused on the process
that is used to allocate resources among program areas,
not on the adequacy of the budget for Conservation,
Solar, and other programs. In implementation and man-
agement, we examined several major state grant programs,
major information programs, and several issues relating to
R&D innovation, procurement, and field activities. In as-
sessing the adequacy of information on program effective-
ness, the review emphasized program evaluation.
Although the review has focused on the Department of
Energy, it has also considered the context within which DOE
operates, including Congress' and DOE's reasons and justifi-
cation for Federal roles in the energy market, the legislative
framework for Federal activity and the historical development
of the Federal energy bureaucracy.
EPA has interpreted the Section 11 mandate not as a
license simply to criticize but as an opportunity to develop
practical recommendations for resolving identified issues.
Although the primary purpose of this Report is to assess the
adequacy of attention to conservation, wherever possible we
have tried to develop suggestions for improvement. EPA
has involved DOE in the Section 11 process from its initial
stages, and hopes that DOE staff have found the process
informative and useful.
The conclusions and recommendations in this Report
emanate in large measure from an extensive public participa-
tion process. Almost every major constituency affected by
DOE programs was invited to appear at one or more of the
public meetings held this year. Participants in the Section 11
process included representatives from: more than thirty state
and local government agencies; Congressional Committee
staff, public utilities, private utilities and public utility com-
missions; trade associations; labor unions; independent re-
search institutes; Solar Energy Research Institute; Regional
Solar Energy Centers; National Labs; national, state and local
public interest groups; appropriate technology community,
and energy policy institutes.*
and solar issues, and discussions of these issues with repre-
sentatives of a variety of organizations, interests, and areas of
expertise.
The Section 11 mandate requires that an annual public
hearing be held. But EPA believes that a formal hearing does
not provide an adequate opportunity for public participation
in the review. Therefore, the National Hearing was sup-
plemented by five meetings on specific issues. These meet-
ings addressed Energy Policy Analysis (Durham, North
Carolina, June 29-30), Program Evaluation (San Francisco,
California, July 8-9), State and Local Assistance Programs
(Minneapolis, Minnesota, July 17-18, and Portland, Oregon,
July 29-30), and Research, Development and Application
(Denver, Colorado, July 24-25).
Following these meetings, EPA prepared a Background
Document for the National Hearing. This document pre-
sented the results of the review to date, and issues and
preliminary recommendations. A notice of the availability of
the document was published and public comment was solic-
ited; EPA received comments from a number of individuals
and organizations. Hearing witnesses were also invited to
discuss these tentative conclusions.
The National Hearing was held September 24-25,1980 in
Washington, D.C. Thirty witnesses testified during two days
of sessions. The Hearing panel included EPA officials, the
DOE Assistant Secretary for Conservation and Solar Energy
and three of his Deputy Assistant Secretaries, the Deputy
Assistant Secretary for Conservation and Solar Policy, Con-
gressional staff, an Office of Management and Budget official,
and representatives of conservation, solar, and environmental
organizations. It also included noted experts on each of the
topics considered. A transcript of the Hearing will be pub-
lished separately.,**
This final Report to the President and Congress draws
upon public comment, Hearing testimony, and material from
the Background Document. Where tentative conclusions
were supported by Hearing witnesses and follow-up research,
portions of the Background Document appear in this Report
verbatim.
1980 Section 11 Program Activities
The 1980 Section 11 activities have had two components:
background research and analysis of selected conservation
The second chapter of the report presents our conclu-
sions and recommendations. It outlines the criteria we used
and it presents our assessment of the adequacy of attention
to conservation and solar energy. Two Appendices present
information to help clarify our conclusions. Appendix A pres-
ents a summary of public participation activities, including
the National Hearing and the five public meetings. It also
includes a list of participants in these sessions. Appendix B
presents an overview of illustrative evaluations of conserva-
tion and solar programs.
* A list of participants is included in Appendix A
"This transcript may be obtained by requesting a copy of the: Section 11 National
Hearing Transcript. EPA 600/9-80-060, December 1980, from the Office of Research
and Development Publications, Center for Environmental Research Information, U S.
EPA, Cincinnati, Ohio 45268
10
-------
Conclusions and Recommendations
-------
Definition of
Adequacy of Attention
Although by including Section 11 in the Federal Nonnuclear
Energy R&D Act Congress demonstrated its concern that
conservation receive "adequate attention" in nonnuclear
R&D, it did not provide a specific definition of "adequacy of
attention."
"Adequacy of attention" to energy conservation for the
1980 Section 11 program was measured in three ways:
Adequacy of the resource allocation process;
Adequacy of implementation plans and management
processes; and
Adequacy of evaluative information for decision mak-
ing.
These issues can only be examined against the current
Federal roles in energy development from both a supply
expansion and demand reduction perspective. Some ques-
tions that have been raised relative to this Federal role are:
given the characteristics of the energy marketplace, is there a
need for Federal involvement to ensure adequacy of attention
to energy conservation? What is the appropriate Federal role?
Adoption of energy conservation measures in the United
States has been hampered more by market imperfections and
institutional barriers than by lack of technological expertise.
Some of these impediments are the result of Federal policies.
Before presenting EPAs major conclusions and recommenda-
tions, this chapter examines the current Federal role within
the energy marketplace, and how Federal actions have fur-
thered or impeded energy conservation.
Understanding this role will provide a context against
which the three criteria for "adequacy of attention" in the
1980 Section 11 program can be measured. For each of these
criteria, relatively straightforward questions have been de-
veloped for assessing DOE's performance. These questions
guided the 1980 review.
Resource Allocation Process
contribution of conservation been adequately consid-
ered in DOE's resource allocation process? What analyt-
ical tools can be utilized to help facilitate comparison and
ensure that an optimal balance is achieved?
A series of recent studies* has helped to establish the
contribution that conservation, supplemented by renewable
resources, can make to alleviating our near term energy di-
lemma. Oil import reductions in the last year provide evi-
dence that conservation has begun to work.** Two other
studies still in preparation provide evidence that conservation
has even greater potential than previously imagined.*** Yet
DOE does not appear to have taken the potential for energy
conservation into account adequately in formulating its
budget.
The section on the Resource Allocation Process exam-
ines how DOE's Planning, Programming and Budgeting Sys-
tem (PPBS) is utilized to make resource allocation decisions. It
discusses how the adequacy of attention to conservation
could be improved if an end-use approach were incorporated
into PPBS. The term "end-use approach" is a method for
examining alternatives for supplying an energy service, such
as heating or cooling of buildings, mechanical transportation
and others. An-end use methodology can be important since
it examines the advantages and disadvantages of both non-
fuel (conservation) and fuel based options. This section then
examines two analytical toolsthe "least -cost" method and
the oil import premium that can facilitate more rational
allocation of resources based upon systematic comparisons
among all conservation and supply alternatives
Given the technical capability of energy conservation to
reduce oil imports, improve industrial productivity, and con-
tribute to environmental and social goals, has the potential
*See for example Hans H Landsberg, Chairman, et al , Energy. The Next Twenty
years, Report by a study group sponsored by the Ford Foundation and adminis-
tered by Resources for the Future (Cambridge, Mass Balmger Publishing Com-
pany, 1979), National Research Council, Energy m Transition 1985-2010, Final
Report of the Committee on Nuclear and Alternative Energy Systems, National
Academy of Sciences (San Francisco WH Freeman and Co , 1979), Robert
Stobaugh and Daniel Yergm, Eds , Energy Future, Report of the Energy Project at
the Harvard Business School (New York Random House, 1979), Roger Sant et al ,
The Least Cost Energy Strategy, The Energy Productivity Center, Mellon Institute
(Pittsburgh Carnegie-Mellon University Press, 1979), Domestic Policy Review
Panel, The Domestic Policy Review of Solar Energy, A Response Memorandum to
the President of the United States (U S Department of Energy, February 1979)
TID-22834
'"Dr Thomas Stelson, Assistant Secretary for Conservation and Solar Energy noted
in recent testimony that "In the last 4 weeks, oil imports were down 37% over the
previous year" He asserted that much of this reduction was due to conservation
Testimony of Dr Thomas Stelson, Oversight Hearings on the Department of
Energy Conservation and Solar Energy Programs, Committee on Science and
Technology, Subcommittee on Energy Development and Applications, U S House
of Representatives, September 9, 1980, Unpublished Committee Transcript
:*The Solar Energy Research Institute's Solar/Conservation Protect, and Conserva-
tion and Solar Strategy Both documents are scheduled for publication in early
1981
13
-------
Implementation & Management
Considering the history of DOE programs, the decentralized
nature of the programs and the relative resources available,
how effectively has DOE implemented and managed its
energy conservation programs?
Energy conservation programs have evolved in a
piecemeal fashion, largely as responses to supply disruptions
and price escalations. In each year since the 1973 Arab oil
embargo, Congress has passed major legislation, or the Exec-
utive branch has taken major initiatives, that affected DOE
programs. Initially, the Implementation and Management
Section explores the impact that this piecemeal growth of
Federal programs has had on DOE programs. It examines the
major areas in which a lack of integration and coordination
among programs has caused significant problems for DOE
and suggests approaches for improving the operation of some
major programs.
The Section next traces the management implications of
the fact that conservation and solar programs are significantly
different from other energy programs. All of DOE's other
programs are concerned with the behavior of a relatively small
number of large highly organized institutions: energy utilities,
power marketing agencies, suppliers of steam generators,
and integrated energy companies. Solar and conservation
programs, by contrast, depend predominantly on the actions
of millions of individuals and small businesses, and thousands
of state and local jurisdictions. Further, DOE's other programs
focus primarily on overcoming technological barriers. In con-
servation and solar programs, while some technical R&D is
needed, there is also an unprecedented need to deal with the
numerous institutional and communications barriers these
programs face. This has affected particularly the state and
local grant programs. Compared to R&D programs, which are
closer to DOE's historical experience, successful state and
local programs depend on DOE's development of a new
management philosophy and the use of a broader range of
Federal instruments. Several new approaches, as well as
suggested program improvements for both R&D and near
term programs, are discussed.
Evaluative Information
The last Section looks at evaluation of energy conserva-
tion and renewable resource programs. Because the effec-
tiveness of these activities should be compared with supply
technologies, an evaluation strategy is needed that encom-
passes all major technologies. The discussion reviews various
uses of program evaluation, describes DOE evaluation efforts,
and analyzes additional steps DOE should take to support
evaluation. Finally, it explores ways of overcoming some of the
major institutional and methodological barriers to evaluation.
The Role of Government
Background: The Nature of the Energy Marketplace
An assessment of the Department of Energy's "adequacy of
attention" to conservation and solar energy must begin with
an understanding of Federal involvement in the energy mar-
ketplace.
During the September 24, 1980, Section 11 National
Hearings, participants expressed two views of the energy
marketplace which appear to be contradictory. Steven C.
Carhart, Assistant Director of the Mellon Institute's Office of
Integrative Analysis, stated: "Energy is a commodity which
is produced, used, and traded predominantly in the private
sector, and as such, it is most appropriate to look at it and treat
it as a commodity." Eugene Frankel, a staff member of the
House Committee on Science and Technology, emphasized
that energy is not a "commodity like any other commodity...
it's a very special kind of commodity, in part because it is a
natural resource and in part because it has so many health,
safety, and environmental side effects, in part because ... of
the concentrated nature of the energy industry."
Both of these views are critical to understanding the role
of government in the energy market. On the one hand, there
is a market in which energy is produced, priced, distributed,
and consumed. On the other hand, there are some charac-
teristics that make the energy commodity different from other
commodities.
During the Section 11 program, EPA attempted to de-
lineate those characteristics of the energy market that have
given impetus to various government activities. These in-
clude :
* Most of our energy is sold through a petroleum-based,
substantially regulated market which is in transition to less
regulation and greater use of coal and renewable resources.
Considering the importance of the choices for both resource
allocation and management, does DOE have adequate
information for decision making?
ซ A critical part of the U.S. energy supply oil imports is
concentrated in the hands of potentially unstable nations;
the level of the nation's oil imports about 40% of all
14
-------
petroleum consumed in the U.S. creates unacceptable
economic and social strains and strategic vulnerability.
Energy pricesfor both domestic supplies and for imports
do not reflect the total costs of energy production and
consumption.
Because some fossil fuels, such as petroleum and natural
gas, and electricity are not priced at their replacement
value and because energy prices do not reflect the eco-
nomic, social, and environmental costs of energy produc-
tion and consumption, the market in the absence of gov-
ernment intervention undervalues the benefits of conser-
vation and renewable resource technologies.
The Federal government has heavily subsidized energy
production and supply A recent study indicates that these
subsidies have amounted to more than $200 billion over the
past 25 years. * Since the Arab oil embargo, these subsidies
have increased substantially.
The next Section discusses how these current market
characteristics, the demands of the transition to a less regu-
lated, less petroleum-based market, and the security and
welfare interests of the United States have contributed to
shaping the Federal role.
Government's Role in Conservation
Certain conservation investments may be the most cost-
effective way to deal with the strains and vulnerability of
existing forms of energy supplies. It reduces the need for oil
imports; relieves pressure on oil prices; increases U.S. eco-
nomic efficiency; extends the U.S. supply of fossil fuels; and
reduces pollution. Residential, commercial, and industrial
energy consumers have increasingly made investments in
conservation and solar technologies. As a response to rising
energy prices and with each increase in OPEC prices, these
investments become even more cost-effective. Even so, in-
vestments in conservation and solar technologies are not
taking place at a rate consistent with the nation's economic or
strategic welfare for a number of reasons. For example, some
energy consumers do not have sufficient information about
the benefits of conservation investments or about the range of
options available to promote energy efficiency.
Federal energy policy has evolved as a series of efforts to
respond to market imperfections and international crisis. The
Federal role appears to be in transition from an emphasis on
managing the marketplace to complementing market forces
and addressing those issues the market does not. Five ele-
ments that characterize this changing Federal role are:
'Bruce Cone, An Analysis of Federal Incentives Used to Stimulate Energy Produc-
tion, Battelle Pacific Northwest Laboratory (Rrchlands, Washington, February
1980), PNL 24-10, Revision 2
1. To respond to the problems of strategic vulnerability
posed by the control of oil imports by OPEC countries, the
government has attempted to: adopt policies to encourage oil
import reduction; provide market incentives to encourage the
development of alternative energy sources; diversify the
sources of oil supply; provide a supply buffer to smooth the
effects of disruption; and plan for disruption and the ensuing
need to allocate supplies to essential uses.
2. Since energy prices do not reflect the total costs of
production or consumption, the government has attempted
to encourage greater efficiency in the production and con-
sumption of resources. Policies consistent with this function
include: encouraging state utility regulatory agencies to re-
quire the adoption of marginal cost pricing of electricity; and
requiring that U.S. energy producers and users bear the costs
of environmental degradation associated with the production
and consumption of energy.
3. Since conservation is undervalued in the marketplace
and therefore is an incompletely tapped source of energy, the
government has attempted to encourage consumers to invest
in conservation at a rate which is in the nation's security and
economic interests. For example the government has pro-
vided information about the benefits of conservation; re-
duced market barriers to these investments where they exist;
provided financial and tax incentives to encourage the rapid
application of conservation techniques; and provided finan-
cial assistance to ensure that low income individuals and
marginal businesses can avail themselves of the benefits of
conservation investments.
4. The private sector can devote only a portion of its
funds to research, development, and high risk demonstration
projects. Also, the activities the private sector chooses to
engage in may not be the activities required by the nation
overall for its long-term health and security Therefore, the
government has sponsored research and development and
encouraged the private sector to invest in the commercial
development of alternative fuels and renewable resources.
5. While the distribution of the nation's income is not
strictly an energy problem, income redistribution caused by
rising energy prices is the result of both Federal policies and
the workings of the energy marketplace. Therefore, govern-
ment policy has tried to deal with the equity problems that
result from higher energy prices, through programs such as
the Weathenzation Assistance Program, and the conserva-
tion grant program of the new Solar and Energy Conservation
Bank.
Despite differences of opinion on the ultimate ability of
the energy market to resolve energy problems, virtually all
Section 11 participants maintained that there was a need for
an active Federal role now and in the foreseeable future
As former DOE Assistant Secretary for Policy and Evalu-
ation Alvin L. Aim stressed during the Section 11 National
Hearings," it will take decades to create a true energy market"
15
-------
and meanwhile "market forces alone will not achieve an
optimum conservation level." Therefore, participants in the
1980 Section 11 program emphasized that because the market
has undervalued and continues to undervalue conservation
and renewable technologies, the government should pursue
vigorously programs to promote conservation and renewable
resources. These programs should focus on providing con-
sumers with information and, as appropriate, financial and
tax incentives which will attempt to compensate for imper-
fections, thereby allowing renewable resource and conserva-
tion investments to compete fairly in the marketplace.
Since market forces will not achieve an optimum level of
conservation, Federal actions in each of these areas appear
necessary to reflect the national interest in conservation.
However, the existence of a Federal policy or activity in each
of these areas does not ensure "adequacy of attention." The
next sections examine three aspects of adequate attention:
the method by which DOE allocates resources to energy
conservation, the management and implementation of
selected DOE conservation and solar energy programs and
the availability of evaluative information for decision making.
16
-------
Assessment of the
Resource Allocation Process
Overview
Given the technical capability of energy conservation to reduce oil imports,
improve industrial productivity, and contribute to environmental and social
goals, has the potential contribution of conservation been adequately consid-
ered in DOE's resource allocation process? What analytical tools can be utilized
to facilitate comparisons of supply and conservation technologies?
Since its creation, DOE has focused on increasing fuel supplies rather than
on analyzing the needs that give rise to energy use (such as comfortable houses,
personal mobility and the like) and comparing alternatives for meeting those
needs.
DOE's Planning, Programming, and Budgeting System should provide a
forum for systematic comparisons among supply enhancement and demand
reduction alternatives to achieve an appropriate balance. This section explores
the process DOE uses to allocate resources and how an end-use orientation and
two analytical techniquesthe least cost method and an oil import premium
would facilitate cross-technology comparisons within this process.
Background
This past year, DOE adopted a Planning, Programming, and
Budgeting System (PPBS) to help allocate departmental re-
sources. The purpose of PPBS is to provide DOE policymakers
with the information needed to make informed decisions
about the direction of programs and to make trade-offs be-
tween programs.
The PPBS establishes a formal schedule of documents to
be used in setting funding levels for various programs, and
dates upon which those decisions are to be made. Two
characteristics of PPBS design are especially important to our
assessment of DOE's conservation and solar programs.
First, PPBS emphasizes the use of analysis and, in par-
ticular, the estimation of the benefits and costs of programs;
that permits divergent programs to be compared in terms of
the same units. Second, PPBS can highlight trade-offs be-
tween programs, and it can make trade-off decisions the
responsibility of high-level administrators.
DOE has made progress during this PPBS implementa-
tion year in estimating the benefits and costs of Federal
energy programs and in recognizing the need for ongoing and
strengthened program evaluation activities. However, many
participants in the 1980 Section 11 program stressed that DOE
could improve its ability to make resource allocation decisions
if better means were available within PPBS to reveal the
contributions of programs aimed at oil import reduction and
their relative effectiveness in meeting energy consumers'
needs at reasonable costs.
It appears that the PPBS is superior to DOE's earlier
systems to allocate resources. DOE's management has de-
voted a great deal of time and effort to developing and imple-
menting PPBS, and it is a significant improvement. In EPA's
view, however, it has not incorporated the systematic,
explicit comparisons of alternatives that would ensure
adequacy of attention to conservation and an optimal
balance among supply and conservation programs.
Three analytic aids explored by the Section 11 review
an end-use sector framework, a least-cost method and an oil
import premiumcould improve DOE's ability to make these
17
-------
comparisons. All three concepts can be used directly in the
PPBS, and have actually been used to a limited extent already.
However, a more concerted application of these concepts
within the current PPBS system would not only improve the
quality of the analysis used in the system, but would also help
ensure that DOE gives "adequate attention" to conservation
and solar programs. The remainder of this section discusses
how the resource allocation process can be improved it
does not discuss the adequacy of specific budget levels.
End-Use Sector Framework
In the past, DOE has emphasized increasing the supply
of energy without sufficient examination of the exact
nature of the needs that give rise to the demand for
energy in various end-use sectors. DOE should incorpo-
rate an analysis of alternative ways of meeting the
energy service needs of end-use sectors into its PPBS
process and its analyses for the third National Energy
Plan (NEP-III).
While DOE, since its creation in 1977, has legitimately
concerned itself with ensuring that the nation has depend-
able fuel supplies, it has not explored comprehensively the
nature of consumers' energy needs. In other words, DOE has
been preoccupied with the supply of energy to the exclusion
of a thorough examination of the needs that give rise to the
demand for energy. DOE has not fully analyzed the services
(like mechanical motion or artificial lighting) that fulfill those
needs, nor the alternative ways by which such services can
be provided. It has not collected the data on which these
analyses must rest. It has not evaluated its programs in terms
of their effects on energy services. Finally it does not allocate
its Department-wide resources according to an overall strat-
egy that takes full account of the needs of various end-use
sectors.
EPA believes that if greater consideration were given to
consumers' energy needs within each sector and the alterna-
tive ways by which energy services could be provided, the
benefits of conservation and other environmentally advan-
tageous approaches would become more apparent. It might
turn out, upon examination, that a vast array of needs could
be better satisfied by improving energy efficiency instead of
expanding its supply. Therefore, in this year's Section 11
program, EPA considered ways to broaden DOE's policy and
programmatic focus and increase its attention to energy
needs.
Participants in the Section 11 program suggested that
DOE should conduct a portion of its analyses within an
"end-use sector" framework.* This analytical framework
requires: (1) a division of the economy into the various sectors
that ultimately make use of energy; and (2) an orientation of
analyses towards the needs of consumers in those sectors and
the services that can satisfy those needs.
DOE already has some experience with an end-use
orientation. Its conservation and solar programs have been
reorganized along end-use lines and it has attempted to build
program strategies in these areas around end-use sector
needs. However, if the framework is to have the desired effect
of promoting explicit cross-technology and cross-fuel com-
parisons, then DOE must implement the framework on a
Department-wide basisnot as an organizational structure,
but as a tool of analysis within PPBS.
An end-use analytical framework, integrated with DOE's
planning and budgeting system, should contain the following
elements:
' A detailed specification of the end-use sectors. Currently,
the major sectors are assumed to be transportation,
utilities, commercial and residential buildings, and indus-
try. However, the subdivisions and boundaries of these
sectors have not been well distinguished;
The identification of consumers' energy needs within each
sector;
The exploration of various ways of meeting those needs,
including alternative fuels and non-fuels techniques;
The calculation (using varying methodologies) of the so-
cial, economic, and environmental costs and benefits of
meeting those needs, including an assessment of how
each alternative impacts on oil import reduction; and
The use of analytic tools (such as the least-cost criterion
and oil import premium described in the next section) to
compare the relative cost-effectiveness of alternative ac-
tions, their relative contributions to oil import reduction,
and the efficiency of the system by which they would be
delivered.
The advantages of such a framework are many. First, it
would facilitate a fair hearing for conservation and solar
energy because DOE policy analysts and management could
explicitly examine whether conservation and solar options
exist to meet end-use needs and how those options compare
to the employment of conventional fuel sources. The end-use
framework would also sharpen DOE's current analyses of fuel
supplies because it asks directly: what fuels and how mucji
fuel for what services? The framework could, moreover, en-
courage DOE to evaluate more thoroughly the environmental
' In the 1979 Section 11 program, the advantages of an end-use orientation in setting
priorities among programs was discussed. This approach was contrasted with a
reliance on econometric models to predict energy demand.
18
-------
consequences of alternative supply technologies and their
implications for consumer costs. Finally the framework re-
quires that DOE deepen its understanding of energy consum-
ers' decision making and the factors that potentially influence
energy demand. Without this kind of understanding, DOE will
continue to find it difficult to design elective market incen-
tive programs to encourage desirable energy investments.
The implementation of this analytical framework would
be complex but ultimately feasible. DOE should take the
following steps to integrate the end-use framework into the
planning and budgeting system and future evaluation efforts:
The Energy Information Administration (EIA), working in
conjunction with DOE's program areas (Resource Applica-
tions, Fossil Energy, Solar, Conservation, and Nuclear)
should continue to define the end-use sectors. Further, EIA
should calculate the current and projected energy
service needs in all sectors. This goes beyond the
activities EIA currently conducts because it requires de-
tailed information about energy demand and behavior on a
sector-by-sector basis.
In conjunction with the program offices, DOE's Office of
Policy and Evaluation should explore the range of
alternatives available to meet end-use sector service
needs and lay out those alternatives. This should be
followed by a comprehensive study of the economic, social,
and environmental costs of those alternatives. A critical
component of the study would be to assess the adequacy
and dependability of fuel supplies needed under each al-
ternative .
DOE's Office of Policy and Evaluation should develop
guidelines for the FY-83 to FY-87 budget process to
help program areas analyze program and budget plans in
light of end-use sector energy needs. The Office of Policy
and Evaluation should detail these guidelines in its Policy,
Programming and Fiscal Guidance Document.
The Office of Policy and Evaluation's Secretarial decision
memoranda on the major budget issues should con-
tain an appraisal of how the issues affect end-use
sectors' energy services in addition to how the issues
affect fuel supplies.
The Third National Energy Plan (NEP-m), currently in
preparation, should address end-use sector needs di-
rectly across-the-board. It should explain how these needs
will be met by the programs in the plan and what the role of
conservation and renewable resources will be in meeting
them.
Finally, EPA recommends that the Congressional Budget
Office and other Congressional analytical staffs also examine
proposed energy incentives and ongoing programs in terms
of their effects on energy consumers' service needs as well as
their contributions to fuel supplies.
Analytical Tools
Within the end-use framework, DOE needs analytical
tools to guide cross-technology comparisons. The
least-cost criterion and oil import premium should be
applied to DOE's programs as part of the budgeting
process.
This section now turns to an examination of two analytic
tools that can be used within an end-use framework to guide
DOE's resource allocation process: first, the least-cost energy
criterion, which would help DOE set priorities among cost-
effective supply expansion and demand reduction energy
programs; and second, the oil import premium, which would
allow DOE to measure the cost-effectiveness of energy in-
vestments against their potential contributions to reducing
U.S. oil import dependence.
The Least-Cost Criterion
Participants in the Section 11 program suggested that DOE
use a "least-cost" criterion to help set priorities among alter-
native programs. This criterion calls for allocating resources
to support the development and adoption of those con-
servation or energy supply options that will provide
energy services at the lowest economic, social, and
environmental cost. Its use within PPBS and the end-use
sector framework can help DOE determine which proposed
programs are most deserving of Federal support, and gauge
the relative cost-effectiveness of current programs. Instead of
examining programs primarily from the perspective of techno-
logical feasibility, this approach would also require careful
consideration of the eventual usefulness and cost-effec-
tiveness of the products of the programs. This approach
would be applied in addition to other criteria, such as the
appropriate Federal role.
It is important to define clearly the concept of least cost.
By "least-cost" we mean least total cost to the nation, includ-
ing the environmental and social costs of energy supply en-
hancement. These factors have tended not to be considered.
It should be emphasized that the least-cost criterion is not the
same as the simple recommendation to choose the least
expensive energy service or supply option. Instead, it requires
DOE to calculate the external costs as well as the economic
resource costs of various alternatives for providing energy
services, and then to give priority to those whose total costs
will be least. The advantage of the least-cost criterion in this
19
-------
respect is that it requires that assumptions about energy
service needs, acceptable means of meeting needs, and costs
be made explicit.
EPA also distinguishes the least-cost criterion as an
analytic tool from the "least-cost strategy" developed by the
Mellon Center for Energy Productivity. During the Section 11
hearings, representatives of the Mellon Institute argued that
national energy policy should be directed towards one over-
riding objective: the provision of needed energy services to
consumers at the lowest possible cost. They argued that by
following this approach, the various objectives now guiding
energy policy the reduction of oil imports in particular
could be achieved more efficiently if they were pursued
directly. The translation of the Mellon theoretical model into
practice at DOE, however, could require more changes in the
way policy is made than are achievable in the short run.
Neither DOE nor the Congress has made much use of a
least-cost approach in shaping policy, proposing new pro-
grams, or evaluating ongoing programs.* One reason is that
least-cost is not the only acceptable criterion for allocating
resources, and other criteria have been given greater empha-
sis. As DOE recently told the House Committee on Science
and Technology: "A high benefit-cost ratio does not auto-
matically dictate government action or major Federally
funded research and development where the scale of required
research, low private incentives, great uncertainties, or crit-
ical national security interests dictate that private sector
actions will be inadequate."
Furthermore, given the lack of existing data about con-
sumer behavior, the effects of Federal programs, and the
social and environmental impacts of untested commercial-
scale energy supply programs, it is difficult for DOE and the
Congress to identify, quantify and project all the relevant
economic, social, and environmental costs of proposed ac-
tions.
EPA recognizes that application of the least-cost criter-
ion is no simple matter. Nevertheless, this approach pro-
vides a useful mechanism for arraying and systemati-
cally comparing demand reduction and supply en-
hancement options, so EPA advocates that it should be
applied along with other factors to resource allocation
decisions wherever possible. Specifically, DOE should:
Conduct a study to identify the costs and benefits of
alternative strategies for reducing oil imports, ensuring
adequate energy supplies, and meeting the current and
projected needs of energy consumers as part of NEP-HL
This study should identify the gap between the least-cost
and existing strategies for meeting DOE objectives. Where
significant gaps exist, DOE should review whether there
are compelling reasons for maintaining existing strategies
and, if appropriate, define new policies and programs for
meeting consumers' energy service needs.
Develop a data base, through EIA and program office
studies, which will enable DOE at a minimurn to apply
the least-cost criterion to proposed investments in
conservation and energy supply technologies at the
commercial development stage. DOE should empha-
size the development of data on the environmental
effects of energy supply technologies, and apply that
data in determining the least-cost alternative.
Increase efforts to apply the least-cost criterion when
evaluating specific budget and program plans within
PPBS. The Office of Policy and Evaluation should provide
program offices with guidance on cost and benefit estima-
tion, with particular emphasis on how to account for
environmental and equity impacts.
The Oil Import Premium
A major thrust of U.S. energy policy is the reduction of oil
imports because reliance upon imported oil imposes high
strategic, economic, and social costs upon the nation. The
"oil import premium" represents an attempt to estimate those
costs. It is, strictly speaking, the cost in excess of market
price of importing an additional barrel of oil. It places an upper
limit on how much the U.S. should be willing to pay, over and
above the world oil price, to reduce oil imports. It therefore
provides a measure of cost-effectiveness against which indi-
vidual programs can be assessed.
EPA recommends use of the premium as an analytic
tool. If applied in the resource allocation process in conjunc-
tion with the least-cost criterion, the premium would provide
a cut-off point for determining the value the government
should place on oil import reduction programs. As less expen-
sive options are exhausted, the premium provides guidance
on the cost-effectiveness of programs. It would lead to more
careful analysis of programs where per barrel cost approaches
or exceeds the world oil price plus the premium. For example,
in assessing the cost-effectiveness of a residential audit and
retrofit program, the cost of the program (measured in terms of
dollars per barrel of oil saved) would be compared to the world
price of oil plus the oil import premium. If the program's cost
exceeds the bound set by the premium, then it should be
scrutinized to determine whether it should be continued. A
program that is not cost-effective on these grounds may be
justified for other reasons, such as equity goals, but this
justification must be made explicit.
"DDEs current programming and fiscal guidance mandates that the "lowest cost
options should be exhausted frrst," but this does not reflect the comprehensive
concept of "least-cost" recommended here (U.S Department of Energy, Draft
Policy, Programming and Fiscal Guidance for FY-82-86, January 1980, p 15.)
*"The premium concept has also been suggested in conjunction with a tax on
imported oil. EPA has not included this potential use of the premium in the Section
11 review.
20
-------
DOE recognizes the oil import premium is a valuable tool
for judging whether oil import reduction programs are cost-
effective. As stated in the DOE Draft Policy, Programming,
and Fiscal Guidance for FY 1982-1986, "A single measure
of the benefits of reducing imports should be applied consis-
tently over all programs." However, because premium estima-
tion work was still in a preliminary stage during this PPBS
implementation year, DOE did not explicitly use the premium
tool in this year's planning and budget process.
Analysts, both inside and outside of DOE, have been
working to refine the premium concept for the past several
years. The current state of the art in estimating the premium
was discussed at two conferences held in October 1980: the
Conference on the Oil Import Premium, sponsored by DOE;
and the Second Annual North American Meeting of the
International Association of Energy Economists. *
There are difficulties in calculating the premium. The
calculations are contingent upon a number of issues that
cannot be resolved conclusively, including: the reaction of
OPEC nations to the import reduction policies adopted by the
U.S. and other nations; and the basic economic parameters
used in these calculations, such as growth rates for the
economy, and discount rates. However, in spite of the varying
assumptions made, independent estimates of the oil import
premium fall in a narrow range, with the premium applicable
to long run import reduction ranging generally from $4 to $15
per barrel (with some estimates more than twice that).
In addition to quantification problems, there are a num-
ber of other difficulties in applying the premium concept to
aid program analysis. Relating Federal programs in basic
research to reduced oil imports will be difficult. The uncer-
tainties inherent in long term research and development
projects are generally very large and estimates of the ex-
pected benefits from such projects are usually reported as a
wide range. Once research projects reach the stages of
demonstration and commercial development, however, their
uncertainty is reduced and the premium becomes a revealing
aid to policy making. The premium, therefore, could be
applied to the appraisal of synthetic fuels production and
demonstrations of innovative enhanced oil recovery methods.
Another problem is that the benefits of some programs
are difficult to calculate because there is not enough informa-
tion to evaluate their impacts. For example, there is insuffi-
cient information to estimate the effect of solar technology tax
credits on the use of oil. Often, it is difficult to assess the effect
of a program because it is almost impossible to tell what
would have happened in its absence. In general, DOE can
profitably use the premium to analyze programs only when
the benefits of those programs can be estimated with reason-
able precision. Therefore, the use of the premium could
increase substantially as program evaluation improves.
To conclude: As Alvin L. Aim stated during the Section
11 National Hearings, "We still have some distance to travel
before an analytically sound and generally accepted import
premium is available. Even when we can agree on such a
measure, its application in the real world is complex." Al-
though the methodology for calculating the premium is still
evolving towards greater sophistication and soundness, DOE
should apply the premium concept to the extent feasi-
ble as part of its preparation for the next fiscal year's
budget and NEP-III. Tb apply the premium, DOE will have to
increase its efforts to develop the data needed to calculate
with reasonable precision the benefits and costs of conserva-
tion, renewable resources, alternative coal-based fuels, and
conventional fuels. Accomplishing this objective will require
a higher priority for program evaluation.
*At these meetings, in addition to discussion of the oil import premium, there was
also consideration of a "supply disruption premium" and a "stock pile premium."
EPA has not included the two latter concepts in its recommendations
21
-------
Assessment of
Implementation and Management
Overview
Considering the history of DOE programs, the nature of the task, and the
relative resources available, how effectively has DOE implemented and man-
aged its energy conservation and solar programs?
The Office of Conservation and Solar Energy (CSE) has had difficulty
adapting a management system designed to promote centralized supply
technologies to decentralized conservation and solar processes. The inappro-
priateness of some aspects of DOE's department-wide management systems
has affected both R&D and state/local grant programs. Conservation and solar
R&D could benefit by increased emphasis on innovation, and more flexibility in
procurement operations. To maximize the effectiveness of the grant programs,
DOE needs a new management approach which is built upon a partnership
among Federal, state, and local level agencies.
CSE's management and implementation problems have been compounded
by the piecemeal growth of conservation and solar programs. Increased coordi-
nation and program integration would improve CSE programs as a whole.
Background
There have been significant achievements in DOE's conser-
vation and solar energy programs; at the same time the
Section 11 review identified two factors that have contributed
to overall implementation and management problems of
DOE's conservation and solar programs: lack of coordina-
tion and integration of related programs and inappropri-
ate department-wide management systems. Both of
these elements grew out of the legislative and organizational
history of Federal involvement in conservation and solar
energy and out of the characteristics of conservation and
solar programs that distinguish them from other energy pro-
grams.
Legislative History
multiplied rapidly and budget outlays to support their ac-
tivities have increased substantially. Much of this growth has
resulted from Congressional actions rather than Executive
Branch initiatives. In addition to the regular agency authori-
zation and appropriation bills Congress has enacted, major
pieces of conservation and/or solar legislation have been
passed in each of the last seven years. Most of this legislation
created at least one new program (see Figure 1).
In many respects, this proliferation of programs repre-
sents unplanned growth. By and large, these new initiatives
were 'created as a response to a series of crises, and taken
as a whole, they do not necessarily provide a coherent
framework for achieving national energy goals.
In some areas, inadequate attention has been devoted to
integrating and coordinating these disparate programs. Effi-
ciency of operation and effectiveness of programs would be
enhanced if conservation and solar were more closely inte-
grated, and related programs, such as information dissemina-
tion efforts, were better coordinated.
Organizational History
From FY-74 to FY-80, growth in the Federal conservation and
solar effort has been significant. Programs in these areas have
22
In 1977, energy activities from ERDA, FEA, and six other
agencies were combined into a cabinet-level Department of
-------
Figure 1
MAJOR FEDERAL LEGISLATIVE INITIATIVES
AFFECTING CONSERVATION AND SOLAR ENERGY
Year
Legislation
Description
1974
Energy Reorganization Act
Federal Nonnuclear R&D Act
Solar Energy RD&D Act
Solar Heating and Cooling
Demonstration Act
Split functions of the AEC between the Nuclear Regulatory
Agency and ERDA.
Public Law 93-438
Established ERDA and called for a $20 billion, decade-long
effort in nonnuclear energy R&D.
Public Law 93-577
Authorized a broad program of renewable energy RD&D .. .
established the Solar Energy Research Institute . ..
established the Solar Energy Information Data Bank.
Public Law 93-473
Mandated a demonstration program for residential and
commercial active solar applications . .. established the
National Solar Heating and Cooling Information Center.
Public Law 93-409
1975
Energy Policy and Conservation Act
Established fuel efficiency standards for automobiles and
energy efficiency standards for appliances . . . authorized
for State Energy Conservation Program . . . established
programs for energy efficiency in Federal agencies.
Public Law 93-163
1976
Energy Conservation and Production Act
Mandated the development of Building Energy Performance
Standards . . . established Weatherization Assistance
Program . . . amended the State Energy Conservation
Program.
Public Law 99-385
1977
DOE Organization Act
Established conservation as a major function within the new
DOE, consolidating conservation programs from other
Federal agencies.
Public Law 95-91
1978
National Energy Conservation Policy Act
National Energy Tax Act
Public Utility Regulatory Policy Act
Photovoltaic RD&D Act
Authorized the Residential Conservation Service . . .
established the Institutional Buildings Program . . .
established programs for solar and conservation
applications in Federal buildings.
Public Law 95-619
Created income tax credits for investments in business and
residential conservation and renewable energy.
Public Law 95-618
Required utilities to establish buyback rates for power
generated by customers, including power from renewable
sources.
Public Law 95-617
Set goals and authorized funds for an aggressive, decade-
long photovoltaics RD&D program.
Public Law 96-223
1980
Crude Oil Windfall Profits Tax Act
Energy Security Act
Increased residential and business tax credit for solar
energy.
Public Law 95-590
Mandated stepped-up programs to demonstrate and
commercialize biomass energy technologies . . . established
the Solar and Conservation Banks .. . expanded the RCS
to include commercial and multifamily buildings . , .
authorized grants for state training programs for energy
auditors... expanding tax credits.
Public Law 96-294
23
-------
Energy. The new department had nine major functions, in-
cluding atomic energy defense activities; commercial appli-
cations of nuclear power; basic nuclear science research;
regulation of interstate sale and transmission of electricity
and natural gas; regulation of petroleum allocation and price
controls; power marketing; fossil energy development; en-
ergy conservation; and renewable energy. It is important to
note that each of these activities, except conservation and
solar, had already developed into a substantial program before
incorporation into DOE.
When DOE was created, it established internal man-
agement systems and procedures to govern all of its programs
that were adapted largely from its predecessors. In many
ways, however, the AEC/ERDA systems have turned out to
be inappropriate for conservation and solar programs. This
chapter assesses how these procedures have affected state
grant programs and certain aspects of conservation and solar
R&D.
Characteristics of Conservation
and Solar Programs
general, and specific problems that have arisen in individual
programs. Under lack of integration and coordination .we
need for:
Greater integration and coordination of conservation, and
solar programs overall; and
Coordination of information dissemination programs.
Under inappropriate department-wide management ap-
proaches we discuss the need for:
A new management philosophy that recognizes the crucial
role of states and local agencies in conservation and solar
development and implementation; and
Alternative management systems for conservation and
solar R&D to more effectively encourage innovation and
respond "to the characteristics of conservation and solar
energy.
Integration of Conservation
and Solar Programs
Conservation and solar programs exhibit several characteris-
tics that make them quite different from other energy pro-
grams DOE supports. These include:
user. While fossil and nuclear programs can be technically
successful if the hardware produced by a small number of
manufacturers and architect and engineering firms is pur-
chased by a relatively small number of users, the success of
most conservation and solar programs depends on the
actions of numerous individuals and local communities.
diversity. Conservation and solar programs encompass an
enormous diversity of processes, products, methods, etc.,
whereas most DOE programs include only a few potential
approaches to given technical problems.
scale. Most conservation and solar technologies are of
much smaller scale than other DOE programs.
regionalism. The potential application of specific conser-
vation and solar techniques varies by region. Few other
DOE programs exhibit this characteristic.
These characteristics have important implications for
the implementation and management of DOE's conservation
and solar programs which are discussed later in this chapter.
This section discusses how the two overall problems
cited earlier the lack of integration and coordination and
inappropriate department-wide management approaches
have affected DOE's adequacy of attention to implementation
and management of conservation and solar programs in
Rapid and piecemeal growth, combined with the decen-
tralized nature of solar and conservation programs,
have led to a lack of integration and coordination. DOE
has taken a significant step by combining conservation
and solar programs under one assistant secretary.
Meaningful integration of these programs can and
should be accelerated without further reorganization
within CSE.
Conservation and solar energy are closely related on a
number of levels. The similarities they exhibit and their close
operational ties provide many reasons for coordinating con-
servation and solar programs.
First, both are primarily decentralized approaches. For
conservation and solar energy to contribute significantly to
national energy needs, millions of individuals will have to
make decisions to use energy more efficiently or shift to
renewables.
Second, many solar and conservation technologies are
integrally related. For example, in building applications, it is
inefficient to install an expensive solar heating system on a
poorly weatherized building. Similarly, most new residences
can use less energy by incorporating passive solar design as
well as standard energy conserving features.
24
-------
- Third, energy conservation and solar activities will re-
quire similar infrastructures, particularly in the residen-
tial/commercial sector. For example, the building industry
will have the primary responsibility for making new homes
and office buildings more energy efficient, and for incorporat-
ing passive solar features in new construction.
Finally, both conservation and solar will reduce the de-
mand for conventional energy, or at least reduce the rate of
growth in energy use. Consequently utilities will need to plan
for integrating these new sources into their plans to install
new central generating capacity.
Until DOE was created there were few ties between the
Federal government's conservation and solar programs. In
ERDA, there was one Assistant Administrator for conserva-
tion and a separate Assistant Administrator for solar, geo-
thermal, and fusion energy. ERDAs premise was that solar
energy was a new energy source, and as such, it was housed
with other "production technologies," rather than with con-
servation.
The split between the two was only partially alleviated
when the Department of Energy was created in October, 1977.
Initially, DOE was organized primarily by functional
categories that corresponded to stages of technology devel-
opment: basic research, technology development, and com-
mercialization. All conservation activities were deemed to be
ready for commercialization, so these technologies, along
with solar heating and cooling, industrial process heat and
solar commercialization activities, were assigned to an As-
sistant Secretary for Conservation and Solar Applications. All
other solar technologies wind, biomass, photovoltaics,
solar thermal and ocean thermal were assigned to the
Assistant Secretary for Energy Technology, along with nuclear
fission, magnetic fusion, geothermal, synthetic fuel and coal
technologies. As most solar technologies approached com-
mercial viability it was expected that they would be trans-
ferred to Conservation and Solar Applications or an Assistant
Secretary for Resource Applications.
The most recent reorganization combined solar pro-
grams with conservation under an Assistant Secretary for
Conservation and Solar Energy. Solar technologies were
grouped by end-use applications: buildings, industry, and
utilities, as were conservation technologies.
Although conservation and solar are now organized
under the same Assistant Secretary, the separate parallel
structures for each have resulted in a continued lack of
adequate programmatic integration. The intended coordina-
tion between programs in these two branches is, for the most
part, incomplete; contact occurring between corresponding
offices is spotty. This lack of program integration means that
solar and conservation offices set separate goals, pursue
different activities, and fail to take the impacts and possible
benefits of the other technologies into account sufficiently. In
the residential buildings area, for example, the impact of
conservation technologies on home heating demand sub-
stantially affects the size and type of solar systems that are
being developed to provide space heat.
DOE has taken some recent steps in the right direction.
These include the ongoing SERI "Solar and Conservation
Project." This study is examining how 20% of U.S. energy
needs can be supplied from renewable sources by 2000 in the
context of enhanced energy efficiency.
There are significant steps, short of reorganization, that
CSE can take to improve the integration of conservation and
solar programs. The focus of these efforts should be on
increasing the joint participation of program offices in
comparable end-use areas (e.g., buildings). Cooperative
projects should be expanded over a two- to three-year
period, until a de facto integration by end-use sector has
been accomplished, at least for the R&D programs.
An example of how this might work is given in the
Report of the House Committee on Science and Technology
accompanying the proposed FY-81 DOE authorization bill.
The Committee recommended adding 2.5 million to the
administration request for developing and testing pas-
sive/hybrid commercial buildings; added $0.5 million for an
analysis of the relation ship between conservation and solar
within buildings, continuing the work begun in this area by
the solar group at Los Alamos Scientific Laboratory; and
directed that a new Hybrid Building activity be created in the
Market Test and Applications area. These additions, $5 mil-
lion of existing activities and $2.5 million from conservation
programs, would be transferred to this new activity. The
Committee also directed that DOE prepare a plan to fully
integrate solar and conservation programs, particularly in the
buildings sector. The recommendations in the Science Com-
mittee report contain useful ideas for improving the ties
between solar and conservation programs.
Coordination of
Information Programs
There is currently overlap and confusion among various
separately-created information programs. DOE needs
to explicitly define the roles of each of the major institu-
tions involved in information dissemination, and inte-
grate and coordinate solar conservation informa-
tion activities.
One of the principal market imperfections identified in
earlier sections of this Report is lack of information needed by
consumers to make energy-related decisions. Individual
25
-------
homeowners and small businesses are at a particular disad-
vantage, especially when compared with industry. Most large
companies have corporate energy managers, large account-
ing staffs, and well-developed criteria for making econom-
ically rational decisions. Homeowners and small businesses,
on the other hand, are frequently unaware of the economic
advantages of energy saving investments. Most households
have little experience with rate-of-return computations or
payback calculations, much less lifecycle costing procedures.
But useful economic data is only one type of information that
should be available to homeowners and small-business
people. They also need to know where they can find knowl-
edgeable contractors, architects or builders. If they do the
work themselves, they need access to materials, equipment
and suppliers. Finally they need to be aware of the warranties
or guarantees that are offered for products and services.
Congress and the Federal government have recognized
the importance of information to the widespread adoption of
cost-effective conservation and solar measures. At least 13
separate Federal agencies and departments currently con-
duct outreach and dissemination activities. These informa-
tion components of programs have developed piecemeal,
however, rather than from a comprehensive Federal plan. For
example, five major programs that contain information serv-
ices were authorized in five separate pieces of legislation (see
Figure 2 for descriptions of these programs):
Although there have been efforts to avoid overlap and
duplication as these programs were created, problems have
arisen in their implementation. Because these information
programs were developed piecemeal, the roles of the principal
institutions have not been carefully defined. Some of these
roles have been assigned by legislation; others have been
added incrementally by DOE; still others have been devel-
oped by the organizations themselves. In addition to causing
overlap and duplication, the lack of explicit demarcation of
organizational mission can lead to "turf" struggles, diverting
resources from the substantive mission of each organization.
Moreover, it can cause confusion within DOE, Congress and
the general public, which can undermine support for informa-
tion programs.
DOE has recently taken a number of constructive steps
to alleviate this problem. In June, 1980, the Assistant Secre-
tary for Conservation and Solar Energy chartered a CS Infor-
mation Steering Committee "to ensure that Conservation and
Solar Energy information activities are effectively managed
and that program resources are appropriately used." In No-
vember, 1980, DOE formally approved establishment of an
office, reporting to the Deputy Assistant Secretary for Field
Operations and International Programs for "developing
policies and providing oversight and management of infor-
mation dissemination and outreach activities within CS."
National Solar Heating and Cooling Information Center
(NSHCIC), from the Solar Heating and Cooling Demonstra-
tion Act of 1974;
The Solar Energy Research Information Data Bank (SEIDB),
authorized by the Solar Energy Research, Development
and Demonstration Act of 1974;
The Energy Extension Service (EES), created by the EES
Act of 1977;
The information program of Residential Conservation Serv-
ice (RCS) created by the National Energy Conservation
Policy (NECPA) of 1978; and
The promotional program Solar and Energy Conservation
Bank, authorized by the Energy Security Act of 1980
(located in the Department of Housing and Urban Devel-
opment).
In addition to these information activities established
through legislative mandate, DOE operates a series of other
solar and conservation programs with information compo-
nents. These include information activities of the four Re-
gional Solar Energy Centers (RSECs), and the President's
Clearinghouse on Community Energy Efficiency. Many of
DOE's program offices also disseminate information on their
programs in response to consumer inquiries.
With the large number of institutions involved in infor-
mation dissemination, the need for coordination is obvious. In
recognition of this need, Title IV of the Energy Security Act'
requires "the Secretary of Energy to coordinate solar and
conservation information dissemination activities funded by.
DOE, including a summary of how DOE services are coordi-;
nated with the services of other agencies." DOE's report in.
response to the congressional mandate will be completed
early in 1981.
DOE can utilize this report to explicitly define the
roles and responsibilities of each of the major institu-
tions, so that each organization has a distinct role.
In its efforts to clarify roles and responsibilities, DOE
should specify that programs include both conservation
and solar information. The lack of coordination between
solar and conservation information programs reflects the
general lack of integration of solar and conservation programs
which was discussed in the previous section. Solar and
conservation are clearly complementary, yet information pro-
grams do not reflect this close relationship. For example,
NSHCIC has only recently received permission to give con-
sumers information about home weatherization, and conser-
vation programs such as the RCS are including solar only
because of pressure from solar activist groups to include solar
in legislation and regulations.
26
-------
Figure 2
MAJOR FEDERAL CONSERVATION
AND SOLAR INFORMATION PROGRAMS
Established by Legislation
Program
Description
National Solar
Heating and Cooling
Information Center
(NSHCIC)
Public Law
93-409
1974
NSHCIC operates a series of nationwide toll-free hotlines to respond to individual consumer
requests for information about active and passive solar heating systems. In its four years of
operation, NSHCIC has reponded to more than 500,000 inquiries and mailed more than seven
million pieces of literature. An FY-80 budget of $4 million funds information research activities,
workshops, and traveling exhibits in addition to the "Hotline." NSHCIC is operated by the Franklin
Research Corporation for the Department of Housing and Urban Development. Funds are supplied
to HUD by DOE through an interagency agreement. Technically, legislative authority for NSHCIC
ends in May, 1981 when the Heating and Cooling Demonstration Act expires.
Solar Energy
Research Information
Data Bank
(SEIDB)
Public Law
93-473
1974
The SEIDB is operated by the Solar Energy Institute (SERI) in Golden, Colorado. Initially, the
SEIDB was designed to supply technical data on solar energy, with users accessing the system
through remote computer terminals. However, the SEIDB has recently expanded to an information
service paralleling NSHCIC. Two toll-free hotlines have been established at SERI, as adjuncts to
the data bank. One supplies information on alcohol fuels; the other answers consumer questions on
all solar technologies other than active and passive heating and cooling.
Energy
Extension
Service
(EES)
Public Law
96-39
1977
Established with an explicit mandate not to duplicate other information programs, the EES provides
services to individuals and small establishments, in order to encourage conservation and conversion
to renewables..
Residential
Conservation
Service
(RCS)
Public Law
95-619
1978
The RCS will operate through gas and'electric utilities to provide three principal services: home
energy audits; information about contractors who will install energy conserving improvements or
solar systems; and arranging for financing of energy conservation measures. The recently passed
Energy Security Act will expand the scope of the RCS to small commercial establishments.
27
-------
Appropriate Management of
State and Local Programs
Partnership
The role of the Federal government in bringing about
immediate energy conservation improvements is unlike
any other function that DOE has served in the past,
because of the highly decentralized energy conserva-
tion process and the interdependence of Federal, state,
and local governments. DOE should assess the
strengths and weaknesses of each level, and should not
attempt to perform activities that are more appropriate
to state or local governments.
Federal policy to encourage the adoption of conservation
and solar requires a set of incentives and efforts much differ-
ent from those used to support other energy technologies.
Because of the distinguishing characteristics of conservation
and solar that were discussed earlier, particularly the decen-
tralized nature of their use and application, state and local
governments must play a large role in partnership with the
Federal government in implementing these programs. Two
major reasons stand out for strong state and local involve-
ment.
First, the diversity of local opportunities for achieving
conservation and solar necessitates a similar diversity of local
strategies for taking advantage of these opportunities. No
single national strategy can fit the needs and opportunities of
all, or even most, communities. Energy consumption pat-
terns, sources of energy waste, climate factors, available
resources and capabilities, and building types all vary greatly
among communities. Given this diversity, finding the most
cost-effective areas for conservation and structuring effective
approaches to them is a task best accomplished at the state
and local levels.
Second, a major factor in motivating people to invest in
conservation and solar is outreach that concentrates on one-
on-one contact between energy consumers and energy pro-
fessionals. The human resources for such outreach activities
will have to come from and be organized at the local level.
Local governments, by virtue of their connections with local
businesses, unions, academia and citizens groups, are in the
best position to persuade these groups to join in conservation
efforts. Successful conservation efforts so far point convinc-
ingly to the importance of local governments in mobilizing
and coordinating community resources toward conservation
goals.
To achieve thesf goal! as rapidly as possible, the Presi-
dent and Congress have identified energy conservation as a
priority issue and have enacted a series of programs over the
past five years designed to involve state and local agencies in
bringing about immediate improvements in the efficient use
of energy across the country. The two largest of these pro-
grams are the institutional Buildings Grants Program, in-
tended to upgrade the energy efficiency of schools, hospitals
and other public buildings, and the Weatherization Assist-
ance Program, which targets the residences of low-income
families for weatherization. Other programs include the State
Energy Conservation Program, to build the capacity for en-
ergy conservation management in state energy offices, the
Energy Extension Service, to provide education, information
and direct personalized assistance to individual energy users,
and the Residential Conservation Service, to provide informa-
tion and assistance on weatherization to all1 United States
homes.by 1985. With the exception of the Weatherization
Program, all of these programs were designed to be im-
plemented through the state energy offices.
While the debate surrounding the unsuccessful efforts to
pass the Energy Management Partnership Act underscored
the interdependency of all three levels of. government upon
one another, the basic issue of what role each level of govern-
ment should play in this partnership has not been satisfactor-
ily resolved. There is a pressing need for DOE to carefully
assess the strengths and weaknesses of each level of
government and then to allow a full measure of respon-
sibility for each level based upon its capacities to con-
tribute to energy conservation. Tb illustrate the need for
clear role definition, Figure 3 shows the large number of actors
involved in weatherizing a home under the Residential Con-
servation Service. This figure demonstrates the potential for
confusion about roles that could produce inefficient use of
resources, time delays, and frustration for all involved.
Tb avoid these problems and to best take advantage of
the strengths of each agency will require a unique degree of
cooperation among all levels of government and clearly de--
lineated roles and responsibilities. EPA's discussions during
the 1980 Section 11 review with people involved in these
programs from Federal, state and local agencies indicated
that there was some consensus on the activities each level
was best equipped to perform. For example:
The Federal government should establish national
program goals and set guidelines and milestones to
achieve those goals. It should provide a sense of mission for
conservation programs and the rationale behind goals, so
that states and communities have a sense of how a particu-
lar program helps the country move forward. Other func-
tions which DOE is best equipped to fulfill include
continuing to provide state-of-the-art information
about energy conservation technology and manage-
ment techniques, technical assistance and training
for state and local staffs, and to fund for state and
local governments to plan for and implement conser-
vation measures.
28
-------
Figure 3.
RESIDENTIAL AUDIT PROCESS
DOE Headquarters
provides general policy guidance
approves state plans
monitors state performance
carries out stand-by authority
where necessary
DOE Regional Offices
shares information on successful
efforts among states
Homeowner
requests energy audit
decides whether to invest in
recommended measures
arranges financing and
installation through utility or
independent contractor
Contractors and Utility
Subcontractors
supply and install conservation
and renewable resource measures
comply with DOE standards for
installation practices
Manufacturers of Materials and
Hardware
comply with DOE standards for
materials
Financing Institutions
provide financing for
homeowners investing in
recommended measures
State Government
develops state plan for operation
of the RCS
carries out listing and consumer
protection functions
monitors utilities
assures quality of installations
Utilities
implement state plans; energy
audits; arranging services;
financing and installing through
sub-contractors; inspections
recommends conservation or
solar measures based on audit
trains energy auditors
29
-------
The states, as the bridge between DOE and local com-
munities, are best situated to coordinate energy ac-
tivities statewide, develop emergency planning pro-
grams and energy-efficient transportation policies,
set an example by making energy conservation im-
provements throughout the operation of their own
facilities and programs* and provide technical assist-
ance to local governments.
Cities and counties have the most direct contact with
individual energy users and should design community
strategies that draw upon local resources and
capabilities. They can initate local efforts which respond
to local conditions and build local constituencies to support
and help carry out these efforts.
In order to strengthen its ability to manage the state and
local programs, DOE recently made a number of changes in
the Office of State and Local Programs. The position of the
director was upgraded to the level of Deputy Assistant Secre-
tary, placing him on a par with the Deputy Assistant Secretary
for Conservation. DOE also established a new office of Grants
Management and Technical Assistance to consolidate state
grant applications and provide coordinated technical assist-
ance where there is overlap among programs. In addition,
DOE has recently taken steps to consolidate and simplify
administrative procedures for several programs that now
have overlapping requirements.
While these changes clearly mark a significant step in
the right direction, it is crucial that the remaining legislative
and administrative barriers to forging a strong partnership
between Federal and state levels be removed. For example, in
the past, highly specific program regulations which permit-
ted little flexibility to adapt to state and local energy situations
have created problems. Participants in the Section 11 work-
shops cited the Weatherization Assistance Program and the
Residential Conservation Service as programs which were
saddled with far too detailed and prescriptive regulations. But
to be effective, new programs must be designed to fit within
existing conservation efforts, codes and other local restric-
tions, contractor capability and availability, and other ele-
ments of the community environment. Thus, the specific
priorities and activities of each program should be developed
by the local communities and state energy officials. As one
Section 11 participant urged:
"DOE should provide overall policy guidance and
technical assistance but allow state and local pro-
grams flexibility as determined by local need... The
implementation of policy by detailed regulation,
meticulous specification, and inflexible program
requirements puts more efforts into organization
and paperwork than into actual implementation of
energy conservation measures... Federal and state
agencies should be oriented toward facilitating
(program implementation) rather than rigorously
dictating and controlling local efforts." (Paul Tutino,
Energy Coordinator, City of Oakland, CA)
One factor that complicates definition of roles and build-
ing a DOE-state-local partnership is the varying levels of
capability of state and local energy offices. DOE's lack of
experience with supporting energy conservation is mirrored
in many state and local agencies. There are still many com-
munities that have no energy management capability,
whereas others are equipped with staff but no plans, and only
a few have trained staffs implementing multi-year plans. As a
result, an important role for DOE is to assist state and local
agencies to increase their skills and capabilities.
Major components of building state and local
capabilities are providing information, technical assistance,
and training. DOE should focus more attention on identifying
successful projects and invest greater effort in sharing them
with other communities. Section 11 participants said that
DOE does not successfully market its successes; disaster
stories win circulation far more rapidly than notable accom:
plishments. Yet the constituencies of the state and local
programs state and local energy officials are hungry for
information and case studies that will help them improve
their fledgling conservation programs.
The Weatherization Assistance Program, which relies
heavily on CETA labor for Weatherization crews, is an exam-
ple of the type of program in which training assistance is
particularly important. As Weatherization techniques and
materials improve, DOE has a role to play in upgrading the
skills and capabilities of local crews through training assist-
ance. Such assistance from the Federal level can best provide
for transferring state-of-the-art techniques between states.
The President's Clearinghouse for Community Energy
Efficiency offers an example of an effective program that
combines information-sharing and training. Each month
since the Clearinghouse's creation in 1979, a group of local
energy officials has spent four weeks at DOE assisting other
energy managers from across the country who call in on the
telephone hotline for help. These visiting local officials are
fully briefed during their stay in Washington by the various
program managers within DOE and also meet with conserva-
tion staff from other agencies such as HUD .Local officials who
have completed this program report having a much better
grasp of how DOE operates and seem to have greater success
after their Clearinghouse experience in dealing with DOE
than before. The Clearinghouse should be continued and
expanded to include a wider range of state and local offi-
cials. *
A final opportunity for building up a more effective
partnership among the three levels of government is to build
up the role of the DOE Regional Offices in the grant
programs. This would not involve adding a layer of bureauc-
racy to program management. In general, the Regional Of-
* Although the Clearinghouse provides a useful experience for local officials, the
users people who telephone with problems or questions are not altogether
satisfied with the quality of service provided, perhaps because the local officials
are always on the steeply ascending portion of the learning curve.
30
-------
flees currently provide assistance to state energy offices and
monitor the progress'of the grant applications through the
administrative process. However, if equipped with appropri-
ate expertise, the Regional Office staff could take an aggres-
sive role in building information networks among states,
provide marketing expertise and facilitating training pro-
grams.
While DOE has made progress in many of these areas,
much more work needs to be done to assure that implementa-
tion of the conservation programs is built upon a true, work-
ing partnership. The recent changes of the State and Local
Program Office are promising but will require continued effort
to assure that administrative problems do not hold back its
forward progress.
Need for Comprehensive Review
The impending sunset review* of DOE creates a useful
opportunity to conduct a comprehensive review of the
state and local programs. The review should exmaine
the relative resource allocation among programs, and
how the grant programs fit into national energy conser-
vation priorities.
The state grant programs discussed in this section were
developed independently in response to crises and to meet
the needs of specific constituencies. These programs have
made a major contribution to furthering energy conservation
in the country, but they do not represent a coherent,
carefully-planned policy. The Department of Energy Organiza-
tion Act Sunset Provisions require DOE to submit a com-
prehensive program review to Congress by January 15,1982.
The review offers a good opportunity to assess the cumulative
impact of all of DOE's conservation delivery programs and to
match program accomplishments with national energy con-
servation objectives.
Although the Sunset Provisions call for a program-by-
program review, the most important question is, given the
resources available and national conservation objectives,
what is the most effective way of connecting the two? The
review should examine the programs not only on a
case-by-case basis, but more importantly, must assess
the overall progess which the programs are making
toward national objectives.
There are a number of additional questions that should
be asked in the course of the Sunset Review, foremost of
which is the issue of the dual objectives contained in the
Congressionally-mandated conservation grant programs.
Some of these programs, such as the Weathefization Assist-
ance Program, incorporate both energy conservation and so-
cial welfare goals. A decision should be made during the
review process about whether there is a sensible balancing of
conservation and equity objectives.
The review process should include an assessment
of the feasibility of the DOE program goals. DOE should
set out reasonable expectations based upon an assessment of
the rate of progress at which the state and local governments
are proceeding. In certain cases DOE might conclude that
achievement of program goals on current timetables is im-
practical because of the lack of existing institutional infra-
structure and might then recommend to Congress a revision
of the plans to extend deadlines or revise the program.
The need for this comprehensive review is underscored
by an observation from the General Accounting Office's Re-
port, Energy Conservation: An Expanding Program
Needing More Direction (July 1980):
"The Department has yet to develop a comprehen-
sive plan which details how the nation can be
moved to greater energy efficiency... What is miss-
ing is an explanation of how separate DOE pro-
grams will reinforce or complement each other, and
what overall contribution is expected to be made by
the combination of all programs and activities."
Program Specific Issues
The state and local assistance programs have been
hampered by a combination of legislative requirements
and administrative problems. These impediments
should be removed as quickly as possible.
DOE's state and local assistance programs have the po-
tential for substantial contributions to national energy con-
servation and attendant reductions in our dependence on
foreign oil supplies. These programs are aimed primarily at
increasing the energy efficiency of buildings, particularly
existing residential'and commercial buildings, where the
substantial potential for energy savings has been widely doc-
umented. **
EPA examined three of DOE's state and local assistance
programs the Weatherization Assistance Program (WAP),
the Institutional Buildings Conservation Program (ICP), and
the Residential Conservation Service (RCS) as part of its
'The Sunset Provisions (Title X of the DOE Organization Act of 1977, Public Law
95-91) require a thorough review of all DOE programs, to be submitted to Congress
in January, 1982. Requirements of this review are discussed more thoroughly m the
section on Program Evaluation.
* A recent OTA study noted that building retrofit in the residential sector could save
the equivalent of 1.9 million barrels of oil per day Office of Technology Assessment,
Conservation and Solar Energy Programs of the Department of Energy A Critique,
June 1980, GPO Stock No. 052-003-00757-6.
31
-------
assessment of DOE's adequacy of attention to conservation.
These particular programs were chosen because they ac-
count for a significant percentage of DOE's conservation
budget.
During the course of the 1980 review, DOE has made
changes in these programs, some of which appear to repre-
sent significant improvements. EPA has attempted to reflect
these changes and the most current information available in
this Report.
A major problem with assessing DOE's state and local
assistance programs has been the lack of objective data with
which to judge the programs' success. Each of the programs
examined by EPA has significant problems, but in many
cases, it is not possible at present to know precisely the extent
to which these problems impede attainment of program
goals. DOE is in the process of conducting evaluative studies
of WAP, has initiated similar efforts of ICP, and is incorporating
ongoing evaluation activities into the RCS. Since the results
of these efforts were not available during the course of the
1980 Review, though, EPA relied more heavily on the conclu-
sions developed in its workshops to assess DOE s programs. *
In the process of examining issues surrounding these
programs, EPA often found very different points of view which
could not be satisfactorily reconciled given the lack of objec-
tive data. For instance, the issue of per house spending limits
in WAP was seen differently by DOE staff and the state and
local program managers attending the workshops. While the
workshop participants argued for thorough weatherizations
and the need for more flexibility in the per house spending
limits to allow such thoroughness, DOE staff emphasized the
need to reach as many homes as possible, and pointed out the
tradeoff between higher per house spending and number of
households which can be assisted in a given year. Better data
on energy savings achieved by WAP may point to a better
balance between these two strategies, but at present firm
conclusions cannot be drawn.
The program-specific discussions which follow attempt
to strike a balance between differing points of view where
issues are difficult to resolve. For each program, we describe
past weaknesses and problems, relate current DOE efforts to
improve the programs, and briefly outline areas of continuing
concern along with recommendations where appropriate.
Figure 4 provides basic information for each of the programs
discussed.
Program Specific Issues
Weatherization Assistance Program
The Weatherization Assistance Program (WAP) is DOE's
largest conservation grant program, providing subsidies for
* EPA held two workshops during this year's Review which focused entirely on the
state and local assistance programs. The workshops represent one of the first
coordinated efforts to bnng together state and local program managers from 30
states to discuss their strengths and weaknesses
the Weatherization of low-income households, particularly
those of the elderly and handicapped. First established as a
Federal program in 1974 within the Community Services
Administration and subsequently transferred to DOE, WAP
aims at increasing energy efficiency among those households
which are least able to afford the costs of Weatherization. DOE
makes grants to the states, which then distribute funds to
local governments and local action agencies to perform the
Weatherization activities. Historically the program has relied
heavily on Comprehensive Employment Training Act (CETA)
labor for Weatherization crews because of statutory require-
ments limiting expenditures for labor-costs. Recent statutory
and regulatory changes have made possible increased use of
contracted private labor.
WAP, in the past, has experienced difficulties which
resulted in low productivity and expenditure levels. A major-
ity of states were not meeting their goals, due to a combina^
tion of management problems, inflexible regulations, and the
unavailability of CETA labor. Prompted by criticism that it
was not demonstrating an adequate commitment to the pro-
gram, DOE took several steps to improve the program, begin-
ning in September, 1979, with the development of an Inter-
agency Action Plan that coordinated DOE, Department of
Labor and CSA efforts to make sufficient CETA labor available
for weatherizations.
In January, 1980, DOE established a Special Project Office
(SPO) for WAP, reporting directly to DOE's Undersecretary.
The SPO undertook activities to remedy the program's
weaknesses, including revising the program regulations in
several critical areas, initiating a new reporting system and
strengthening program management. In April, 1980, a new
program director was named and WAP was returned to CSE.
DOE's efforts over the last year have resulted in substan-
tially increased productivity and expenditure levels. In fiscal
year 1979, 94,000 homes were weatherized with expenditures
of $40 million, while in FY 80, 265,000 homes were weath-
erized and $182 million spent. DOE has made significant
progress in correcting WAPs past problems; however, there
remain areas of concern that require attention.
Program Continuity. WAP's goal of alleviating fuel costs for
the nation's poor necessitates as rapid progress as possible
toward weatherizing all low-income houses. As of September,
1980, though, the program had reached only 450,000 of the
12.6 million estimated eligible households. As an increasing
number of states expand and improve their programs, fears
were expressed in the workshops that some states will
exhaust their 1981 grants well before the end of that fiscal year,
and that the lapse will damage their programs. Continuity is
important in order that states and local operators have
adequate incentive to maintain effective programs.
Program Flexibility. The wide variation in local weath-
erization needs, conditions, and resources makes it essential
to allow states and local program operators wide latitude in
32
-------
Figure 4.
DOE ADMINISTERED STATE AND
LOCAL ASSISTANCE PROGRAMS
Weatherization
Assistance
Program
Residential
Conservation
Service
Institution
Buildings
Conservation
Program
Goal
To weatherize low-income
homes (no specific goals)
To reduce energy consumption in
existing U.S. houses by providing
information and other services to
homeowners
To offer preliminary audits to
90% of qualified buildings and to
fund actual weatherization in
some of them, over 3-year
period.
Progress
to date
450,000 homes completed as of
September, 1980
Utilities to begin offering
program, Spring 1981.
First grant cycle complete,
second underway
Enabling
Law
Energy Conservation
and Production Act (ECPA),
1976
National Energy Conservation
Policy Act (NECPA), 1977.
National Energy Conservation
Policy Act (NECPA), 1977.
FY 1981
Funding
Level
$182 million
$5 million
$181 million (a matching grant
program)
Number of
existing
buildings
in this
category
12.6 million
80 million
500,000
Approximate
energy
use
Unavailable
12.4 quads/year"
2.8 quads**
Implemented
by
State offices of 'economic
opportunity, local governments,
local community action
agencies and other non-
profits
Electric and gas utilities,
coordinated by state energy
offices
Schools, hospitals, public care
buildings, local governments
Principal
services
offered
Funding for staff and materials to
carryout weatherization
(insulating attics, caulking, etc.)
Energy audits, financing and
installation of conservation
retrofit measures.
Building audits and
implementation of conservation
measures.
*1977 residential consumption for space heating, cooling, and hot water.
Source: Energy Information Administration
'Source: FY 80 Program Summary Document for Conservation, DOE
33
-------
structuring their weatherization programs. DOE regulations
for WAP remain too prescriptive in some provisions, and do
not take the need for local flexibility into account.
Program Specific Issues
Residential Conservation Service
For instance, the present limit on'per-house ex-
penditures ($1,000; $1,600 if contracted labor is used) is
regarded by many state and local managers as inadequate for
completely weatherizing many houses. They emphasize the
importance of thorough weatherizations, particularly given
the significant fixed costs of weatherizing a house that are
incurred regardless of the completeness of the job. Higher
per-house expenditure limits would substantially benefit
many houseowners receiving weatherizations, as would
some flexibility for the limit, such that local operators could
vary per-house spending so long as the average for a group of
houses remained within the limit. However, this would intro-
duce a tradeoff between thoroughness of weatherizations and
number of households reached, since increasing the per
house expenditures would reduce the number of homes WAP
can reach. As more data on the program's performance
becomes available, the per house limit should be reevaluated
in light of the concerns expressed by state and local mana-
gers.
In recognition of both the substantial energy savings possible
in the residential sector and the unique capacity of utilities to
help bring about these savings, Congress adopted the RCS to
expedite residential weatherization in the United States. The
RCS requires utilities to offer energy audits to all residential
and small commercial customers, accompanied by follow-up
assistance in implementing the recommendations arising
from the audit. States carry the primary responsibility for
program management.
Support for the RCS goals is widespread, as the two
Section 11 workshops with state andlocal government energy
managers revealed. There are even a few cases where states
initiated their own energy audit programs prior to the RCS,
and also where utilities initiated conservation programs of
their own. A broad consensus was expressed in the Section 11
workshops that enlarging the utilities' energy delivery ca-
pacity to include the delivery of conservation is appropriate in
reducing the country's energy use.
Substantive Issues. In addition to these administrative
problems, workshop participants identified two substantive
issues related to the long-term effectiveness of WAP: rental
housing and need for coordination between WAP and the Low
Income Energy Assistance Program (LEAP).
Over 50 percent of low-income households rent their
dwellings, yet WAP has had only limited success in reaching
the rental sector. The original legislation creating WAP con-
tains a provision requiring that landlords of rental units
guarantee that the major portion of weatherization benefits
will accrue to the occupants. Many observers feel this has
discouraged landlords from signing the required agreement
which would permit their low-income tenants to participate
in WAP. Most participants in the Minneapolis and Portland
workshops agreed that WAP should not be relied exclusively
on to reach the low-income rental sector; they suggested that
other incentives, such as an increased tax credit for landlord
investments in conservation, might be more effective.
It would be useful for WAP to better coordinate with
LEAP, which provides fuel subsidies to low-income house-
holds. LEEAP payments, though vitally important to many
poor households, do nothing to increase energy efficiency and
represent an indefinite drain on the nation's resources. LEEAP
assistance could be reduced by more closely coordinating the
weatherization program with LIEAP so that households
receiving energy assistance would be targeted for priority
weatherization. In Missouri, for instance, low-income house-
holds must apply for weatherization in order to qualify for
LIEAP assistance. DOE should examine this approach and
other ways of coordinating weatherization with LEEAP.
However, there is also widespread doubt about the
feasibility of the program and the ability of the program to
accomplish its goals. Section 11 participants predicted that
without increased staffing for all levels of the RCSDOE, the
states and utilities the program will not succeed. In many
cases, states and utilities do not have the capability to
implement and manage the program. The states are charged
with substantial responsibility for implementation of RCS.
Among the activities for which the states are responsible are
the investigation and enforcement of utilities' compliance;
development of consumer complaint procedures; listing of all
suppliers, lenders and installers willing to participate in the
RCS; record-keeping of utilities' progress, as well as reporting
requirements to DOE. Participants at the Minneapolis and
Portland workshops expressed strong doubts about the. feasi-
bility of the RCS without direct financial aid to enable the
states to carry out RCS duties. It was suggested that Con-
gress may not be aware of the considerable effort required of
the states and non-regulated utilities to implement the RCS.
Program Flexibility, Participants generally agreed that the
RCS is being implemented under a highly detailed and
prescriptive set of regulations, many of which were necessi-
tated by the similarly detailed language of the authorizing
legislation. As a result, the states have only limited flexibility
in designing the plans which will guide implementation of
the RCS. The regulations, by requiring certain services and
prohibiting others, substantially restrict the range of strate-
gies the states and utilities can employ to accomplish the
goals of the program. For instance, the regulations prohibit
utilities from giving away free conservation measures during
home energy audits, even though this service might some-
times benefit both utility and customer as well as provide an
34
-------
effective encouragement for further conservation invest-
ments by the homeowners.
On the other hand, DOE believes that it has exercised
great care to provide states with ample flexibility in designing
their RCS plans, and that states' or utilities' difficulties with
the RCS arise from their reluctance to take steps that DOE
feels are necessary to an effective program.
Utility Credibility. For homeowners to request and pay for
energy audits, the utilities must be seen as a credible source
of information on home energy conservation. Several
provisions of the present regulations, however, work against
utility credibility. For instance, workshop participants felt that
the required state-approved lists of contractors and financial
institutions will not work as intended by Congress, and may
damage the credibility of utilities distributing them by con-
taining inaccurate or misleading information. They recom-
mended that the requirement for listing be dropped from the
regulations, and suggested that more useful information
could be provided through consumer guidance booklets
covering conservation financing and contracting. Other im-
portant aspects of RCS credibility that DOE should ensure:
program announcement requirements that allow utilities to
mail their announcements over a broad enough period of time
to avoid building up a backlog of audit requests that they
cannot promptly satisfy, and assistance to states and utilities
in building staffs of well-trained, effective energy auditors,
skilled at working with homeowners on conservation tech-
niques as well as assessing the best strategies for each home. *
Program Specific Issues Institutional Buildings
Conservation Program
The second largest conservation grant program at DOE, the
ICP is targeted at upgrading the energy efficiency of the
country's schools, hospitals, local government and public care
buildings. As the most ambitious conservation program di-
rected at non-government public buildings, the ICP has
pioneered in the development of a public energy manage-
ment infrastructure and in educating state and local officials,
many of whom would otherwise have postponed taking
conservation actions without Federal leadership and support.
However, the problems encountered in the implementa-
tion of the ICP illustrate the serious consequences of an
ambitious timetable, coupled with delays in authorizing the
program** and lags in revising the original timetable to reflect
those delays. When the program was initiated, there was little
opportunity for adequate planning and development, inclu-
sion of participants in the planning process, and the resolu-
tion of key policy issues. Outside Washington, the impact of
'In this case, ICP was contained in the larger, complicated National Energy
Conservation Policy Act (Public Law 95-619).
"Support for auditor training was included in the Energy Security Act
the deficiencies was keenly felt. Had the necessary energy
management infrastructure already been in place at the state
and local levels, it might have been possible for some states to
conduct feasibility studies, design programs, and submit
applications without serious delay. However, because the ICP
was a ground-breaking effort, they generally lacked the nec-
essary skills to plan for and implement the program. The ICP
goals as a result proved unattainable and many participating
states, institutions and local governments found the ICP to be
an attractive but frustrating program. The problems associ-
ated with the ICP cogently illustrate the consequences of
forcing a program to implementation before DOE, State, and
local agencies have developed the capacity to manage that
program.
Program Pacing. During its first years, the ICP experienced
a series of problems related to program pacing and the
constraints of the Congressional timetable. Because of late
passage of the FY 80 Appropriations Bill, timetable changes
and late announcements of funding availability, the states
and local institutions had insufficient time in many instances
to adequately prepare grant applications, plan local matching
funds, and perform other tasks. Participants at the Min-
neapolis and Portland workshops felt that many localities and
institutions were discouraged from participating in ICP at
least partly by DOE's unrealistic time requirements. DOE's
requirements however, resulted from the short time-frame
provided for ICP in the authorizing legislation, and the late-
ness of the Appropriations Bill. The workshop participants
urged that Congress be made fully aware of the pacing
problems it created in the ICP.
DOE is now moving to full year grant cycles for ICP,
which should ease the time constraints on states and local
institutions. It is important that participating institutions
have at least 90 days from the time of receiving the grant
application forms to the due date for their submission, in
order that they have adequate time to prepare the applica-
tions. States also need sufficient time to prepare their grant
management plans, to notify and assist the local institutions,
and to involve them in the planning process.
Program Continuity. The original ICP authorizing legisla-
tion provided for funding through FY-80, with the intention
that three years would be sufficient to achieve the program's
objectives. In view of the problems experienced during the
first two grant cycles and the time required to make the
program operate smoothly, some revision of the original ex-
pectations for the ICP are needed. The consensus of the
workshop participants involved with the ICP was that the
program has only recently overcome many of the problems
that had constrained it. With most of the "starting-up" prob-
lems resolved, these participants felt that the ICP can be
expected to accomplish significant results if continued for a
few more years.
Program continuity is also critical to the local institutions
participating in ICP. Since participating institutions have to
come up with matching funds for the ICP grants, they need to
35
-------
plan for the giants well in advance of their availability.
Schools, hospitals and local governments typically have to
identify matching funds for ICP grants during the budget
process that precedes each fiscal year, which means that ICP
grants should be announced at least a year ahead of the grant
cycle. To allow local institutions to plan ahead for participa-
tion in ICP, DOE should announce the grant cycles a year
ahead, removing much of the uncertainty that currently
hampers participation in the program.
Undue Expense for Small Grant Applfcants. It appears
that the complexity and expense of applying for ICP grants
have discouraged some institutions seeking small grants
from participating in the program. The same application
forms and reporting procedures are used for institutions
seeking $500,000 or $500 grants, even though the adminis-
trative cost for institutions applying for and reporting on small
grants can exceed the amount of the grant. DOE could act to
ease the paperwork burdens on states and local institutions in
several ways: by providing simpler grant applications forms,
reduced reporting requirements and simpler audit procedures
for small grant applicants.
Energy Conservation Measure Funds for Local Gov-
ernment. As currently authorized, ICP provides funds to
units of local government and public care centers for energy
audits and technical assistance, but not for actually imple-
menting or installing the chosen conservation measures.
Schools and hospitals, on the other hand, receive funds for all
three phases. Although local governments can obtain funds
for conservation retrofits through other Federal sources such
as the Economic Development Administration, unavailability
of ICP funds for conservation measures may have discour-
aged local government participation in the program, since
many governments have apparently been reluctant to apply
only for energy audit and technical assistance funds.
DOE should promote an awareness among local gov-
ernments of the benefits of the ICP energy audit process.
Given the enormity of the retrofit task facing local govern-
ments, much if not most of the needed funds will have to
come from local sources. ICP energy audit assistance, while
not funding retrofits, can be valuable to local governments
interested in establishing a systematic, ordered approach to
conservation. DOE needs to "sell" the energy audit process
aggressively as part of its activity to stimulate local initiative
in conservation.
Appropriate Management of
Research and Development
Innovation
low priority. Programs managed, by the Office of Inven-
tions and Small Scale Technology should be given in-
creased emphasis and have closer ties with the conser-
vation and solar programs.
The Section 11 review defined innovation broadly to
include the creation, design, production, first use and diffu-
sion of a new technological product, process, or system." The,
innovation process is not limited to technological break-
throughs in the research phase. Innovation can also.occur in
systems for adoption of the technology, such as production,
marketing, information dissemination, training, and other
activities.
In the last few years there has been increased interest in
innovation, spurred in large measure by an apparent decline
in industrial productivity. The interest in innovation is appar-
ent in both the Executive Branch and in Congress. For
example, President Carter initiated a Domestic Policy Review
of Industrial Innovation, under the aegis of the Department of
Commerce, which was completed in October, 1979. Congres-
sional attention to innovation is reflected in the Stevenson-
Wydler Technology Innovation Act of 1980 which would
create joint industry-university centers to develop the R&D
base for selected technologies. Other legislation relating to
patent reform, small business exports and trade policy is
currently pending before both Houses.
Energy is a key factor in industrial production, especially
as energy prices have increased dramatically in recent years.
A recent report by the National Academy of Sciences Com-
mittee on Nuclear.and Alternative Energy Sources concluded
that increases in efficiency may be prerequisite for continued
growth in the gross national product. Substantial im-
provements in energy efficiency depend on a number of
factors including energy prices, information systems and the
growth of a conservation infrastructure. However, such im-
provements will also be influenced by a willingness to inno-
vate in the private sector combined with government policies
that foster innovation.
The character of the innovation process is one factor that
differentiates conservation and solar R&D from R&D pro-
grams related to centralized energy sources. In the latter,
attention is concentrated on developing a few alternatives to
one basic concept such as gas-cooled reactors replacing
water-cooled reactors or a thorium-based fuel cycle as an
alternative to the uranium fuel cycle. By contrast, innovation
in the conservation and solar area involves the development
of a wide range of new products, processes and technologies.
It encompasses the application of these ideas in each of the
end-use sectors for different regions. Innovations can also
involve the combination of technologies in different ways:
solar systems with heat pumps, wood stoves with solar, wind
or photovoltaics for powering electric cars, and so on.
Considering the importance of innovation in conserva-
tion and solar programs, DOE appears to be giving it a
'SEWSolar Conservation Project: Task Force on Innovation, June, 1980; draft.
36
-------
Most of the research and development for centralized
technologies can be conducted by a relatively small number
of large firms and national laboratories. By its very nature,
solar and conservation R&D requires a considerable amount
of "prospecting" for new ideas and approaches in many areas.
Many of these will be either technically unsound or commer-
cially unsuccessful. But a small number of successful innova-
tions perhaps one or two percent of all tried could
dramatically transform energy use patterns.
The Section 11 analysis focused on two aspects of-DOE
efforts to promote innovation in conservation and solar ac-
tivities.
First, the Section 11 program examined management prac-
tices that affect the overall "climate of innovation1' at DOE.
These include procurement policy, management of field
activities and relationships with state and local govern-
ments. Recommendations which would lead to im-
provements in these areas and a more favorable "climate of
innovation" are discussed in other sections of this Report.
Second, EPA has reviewed the programs specifically de-
signed to promote conservation and solar innovation, and
the results of this review are presented in this section.
Inventions and innovation in conservation and solar
research and development occurs within DOE's program
offices and within several other specialized offices. There are
two programs managed by CSE's Office of Small Scale Tech-
nology the Appropriate Technology Small Grants Program
and the Energy Related Inventions Programand one within
the Office of Energy Research the Advanced Technology
Projects Office (ATPO). (ATPO manages a small number of
projects in all areas of energy, not just conservation and solar.)
The Office of Inventions and Small Scale Technology
(OISST) was originally proposed in the National Energy Plan
to "fund small, innovative research and development proj-
ects." The Appropriate Technology (AT) Grants Program was
mandated by the ERDA Authorization Act of 1977. It was
designed to reach individuals and institutions that did not
have the resources to get a favorable response from DOE's
regular procurement system. Grants were limited to no more
than $50,000, and DOE was instructed to develop a simple
solicitation that could make awards in a short period of time.
The program began as a pilot in FY-78 and the first coordi-
nated national solicitation was issued in February, 1980.
Program funding has increased from $8 million in FY-79 to $12
million both in FY-80, and in FY-81.
The Energy-Related Inventions (El) Program is con-
ducted by DOE with the assistance of the National Bureau of
Standards. Its purpose is to assist independent inventors and
small businesses in developing and commercializing inven-
tions which show significant promise for energy conservation
or providing new sources of energy. The assistance usually
provided is a one-time direct grant to the inventortypically
$50,000-$80,000but may take other forms, such as inven-
tion testing services at government labs, market analysis
studies performed for the inventor, and occasionally access to
contracts elsewhere in DOE. As of December, 1979, 61 inven-
tors had been assisted through the program; total awards
were slightly more than $4 million.
Although neither the AT nor El programs have been in
operation for more than three years, they are serving as a
source of new ideas, and they provide access to Federal funds
for individuals and organizations who have traditionally been
excluded from the complex, time-consuming procurement
system. During the FY-79, $340 million worth of grant applica-
tions were received for $8 million in available funds. About
18% of these proposals were rated good to exellent by techni-
cal reviewers. State "peer review" committees concluded
that 36% of all proposals were worthy of being funded.
The effectiveness of both programs could be improved
through:
Better integration with other CSE programs; and
Modifications to the programs themselves.
One problem with the AT and El programs is their sep-
aration from other R&D efforts. DOE should develop sys-
tematic mechanisms for bringing the results of either
program to the attention of the DOE program managers,
and for channeling unsolicited proposals received by
other program offices to OISST. One factor that may affect
communication and coordination is the fact that OISST re-
ports to the Deputy Assistant Secretary for State and Local
Programs rather than the solar or conservation R&D branch.
Because the AT grant money is administered regionally and
apportioned to the states, there is some justification for this
organizational arrangement. However, more than two-thirds
of the FY-79 AT grant funds were awarded for renewable
energy projects, and clearly there is a need to coordinate
closely with the solar office. (Although it is sensible to col-
locate the AT grants and El programs, the states and regions
have no substantial involvement in the El program.)
Further, there has been no coordination between OISST
and the Advanced Technology Projects Office. ATPO is re-
sponsible for intensive management of a small number of
selected high-technology research projects, and OISST is
responsible for funding a relatively large number of low-
technology projects. However, both offices are concerned
with the adoption of innovative projects in the marketplace.
They should be aware of each other's programs and strate-
gies.
Secondly, there are a number of program-specific im-
provements that could be made in the AT program. The
existing AT solicitation could be modified to allow a more
intensive focus on a particular technology or approach. Cur-
rently, applicants can submit a proposal on any topic that fits
37
-------
the broad criteria in the program announcement, which state
that the projects should be oriented toward local needs, be
labor intensive, environmentally benign and utilize local re-
sources. DOE should consider establishing specialized
solicitations. These solicitations should be developed in
cooperation with the relevant conservation or solar program
office to emphasize solutions to particular problems. The
specialized solicitation should be developed after a series of
outreach activities to tap the opinions of relevant constituen-
cies on priorities. These solicitations would supplement the
current program announcement, not substitute for it.
The AT program could also benefit substantially
from the experience of other innovation-related ac-
tivities conducted by the Federal government. Perhaps
the most useful model is, the Small Business Innovation
Research Program (SBIR) administered by the National Sci-
ence Foundation. The SBIR program is designed to encour-
age a systematic progression from concept development
through marketing. In particular, as a means of ensuring
commercial adoption of products of research, SBIR proposals
that demonstrate a commitment for follow-up funding from
the private sector or other sources, receive extra considera-
tion as a point of merit in the evaluation process. SBIR also
gives much larger awards averaging $200,000 for projects
that progress beyond the concept development state. For the
AT program to provide similar support and incentives to its
grantees, Congressional action would be required to raise the
limits on grants. However, no legislative changes would be
required if the equivalent of SBIR funding were supplied by
the solar or conservation program offices, possibly in combi-
nation with the AT Office. This presents an excellent target of
opportunity for coordination of the AT program with other
offices in CSE.
The AT Program should also incorporate some of the
approaches of the Energy Related Inventions Program. The El
Program is able to supply technical assistance as well as
support to prospective inventors. The AT Program should also
make technical assistance available to grantees.
Procurement and Financial Assistance
DOE is taking some steps to adapt its procurement and
assistance relationships to conservation and solar ac^
tivities. Continued actions are needed to ensure that
there are significantly reduced delays in making finan-
cial awards.
Many of the conservation and solar programs involve a
financial relationship between DOE and the private sector,
state and local governments and individuals. Awards are
made through a variety of financial instruments, including
contracts, grants, cooperative agreements, and loan guaran-
tees. The efficiency and effectiveness of DOE's financial
processes can have a significant impact on the adequacy of
attention to energy conservation:
In EPA's Section 11 workshops, participants cited pro-
curement as a major problem in both state and local programs
and R&D programs. In addition, DOE's procurement policies
have been singled out for attention in a General Accounting
Office report, an Office of Technology Assessment study, and
recent testimony by CSE officials. *
One problem noted by most of these sources is the
extensive delays in processing procurement requests. These
delays can slow the attainment of program goals, make it
difficult for program offices to expend appropriated funds
wisely, and cause severe cash flow problems for prospective
contractors, particularly small businesses. Other problems
that have been mentioned are a lack of receptiveness to
unsolicited proposals, excessive costs incurred in both writ-
ing proposals and complying with contractual requirements,
and too great a reliance on large corporations and National
Laboratories.
EPA looked at ways in which DOE's procurement system
is in many ways inappropriate for conservation and. solar
Some Section 11 participants felt that it may not be
possible to foster innovative conservation and solar research
within the Department of Energy. For example:
"The present efforts in DOE represent a new
low in innovation as far as I am concerned. The
amount of effort that is expended on truly new
innovations is now at an all time low... The program
planning by essentially non-innovative risk-adverse
type people simply... leads to the situation in which
there is no room for innovators." (Dr. Jerry Plunkett,
Managing Director, Montana Energy and MHD Re-
search and Development Institute.)
The possibility of transferring innovation related pro-
grams to the private sector should be examined.
'For example, recent testimony of Dr Thomas Stelson, DOE Assistant Secretary for
Conservation and Solar Energy noted that: "Procurement is another critical area
with us. We have more small programs in conservation and solar energy. That is,
we have lots of little programs. We are dealing with many unsophisticated con-
tractors and grantees. We, for example, have exceeded our minority business goal,
jnd so on. Now, these small and unsophisticated contractors need a lot of help to
relate to the Federal government. If we had more personnel, I think that we could
get our procurement process to be very much improved. Currently, it takes four to
seven months to process the procurement out to them So, we have lots of
operating difficulties like that."
Testimony of Dr. Thomas Stelson, Oversight Hearings on the Department of
Energy Conservation and Solar Energy Programs, Committee on Science and
Technology, Subcommittee on Energy Development and Applications, U. S. House
of Representatives, September 9, 1980, Unpublished Committee Transcript.
A recent OTA report draws the following conclusion about the DOE procurement
and contracting system' "The substantial delays and bureaucratic complications
that characterize the current DOE procurement process threaten the viability of
even the best conceived and most competently planned initiations." Office of
Technology Assessment, Conservation and Solar Energy Programs of the Depart-
ment of Energy: A Critique, June 1980, p. 23, GPO Stock No, 052-003-00757-6.
38
-------
R&D. As discussed in an earlier section, when the Depart-
ment of Energy was established in 1977, most of its adminis-
trative procedures were adapted from its principal predeces-
sor, the Energy Research and Development Administration,
the successor to the Atomic Energy Commission.
The AEC/ERDA procurement process was modeled after
the Major Systems Acquisition approach, also utilized in the
Department of Defense and the National Aeronautic and
Space Administration.* It was designed to guide the devel-
opment of large scale equipment or systems for use by the
government itself. Under this process, Federal program man-
agers define the steps that must be taken in producing a final
product, as well as the criteria that product should meet. The
Major Systems Acquisition process assumes that since the
government knows precisely what it wants in terms of prod-
uct performance, it is best able to achieve its goals by
defining and monitoring how the product is developed. This
may be workable when the government wants to obtain a
piece of equipment. However, the Major Systems Acquisition
approach is less useful when the Federal agency can neither
precisely define the product, nor how it is to be produced. The
process may be deficient if the ultimate goal is to foster
innovation and the commercialization of new technologies in
the marketplace.
An alternative approach, which permits greater flexibil-
ity, is the use of assistance relationships, as defined under the
Federal Grant and Cooperative Agreement Act of 1977 (Public
Law 95-224). These provide a means for sharing information
and responsibility with non-Federal participants while retain-
ing for the Federal government the degree of control neces-
sary to achieve policy objectives. In assistance relationships,
the Federal program manager does not have to define the
precise specifications for the final product. Instead, the man-
ager identifies the overall desired result, and contractors are
free to apply their own expertise during contract execution, a
task that is simplified by the formal agreement with the
government or the definition of what the "product" should be.
If the provisions of the Act could be applied, the respective
Federal and contractor roles could be defined in terms of
encouraging marketplace innovation rather than procure-
ment of products for government use.
DOE's major experience with assistance relationships
has been in the state and local programs area. The solar
program has also utilized assistance modes from time to time.
For example, Texas Instruments and DOE entered into a
cooperative agreement to develop TI's residential photovol-
taic fuel cell combination. Similarly, the Solarex Corporation
received a DOE grant to pursue its process for the production
of lower cost silicon photovoltaic cells.
In March, 1979, after the passage of the National Energy
Conservation Policy Act (NECPA), DOE promulgated its final
rule on Assistance Regulations (Federal Register, Vol. 44, No.
47, March 8,1979). These regulations provide guidance on the
use of grants and cooperative agreements in situations when
a financial award is clearly in the assistance mode..
,' '
At this point DOE does not have a department-wide
policy for defining the circumstances under which program
managers can utilize assistance relationships as an alterna-
tive to procurement instruments. However, DOE currently
has under development a set of guidelines that should help to
systematize the process of making financial awards. Utilizing
this Program Principal Purpose Determination (PPPD), pro-
gram managers may be able to more readily distinguish
between procurement and assistance relationships. The
PPPD will probably have a more substantial impact on CSE
than on the rest of DOE. Therefore, Congress needs to closely
monitor the implementation of PPPD guidance to determine if
it resolves some of these problems.
Field Activities
DOE needs to devise a consistent policy for the use of
the multiprogram national laboratories in conservation
R&D. There is scope for combining conservation and
solar activities and for giving SERI and the RSECs
greater flexibility in their day-to-day operations.
DOE inherited a well-developed field structure for R&D
composed of a nationwide network of national laboratories.
With the addition of fossil, conservation and renewable re-
source activities to its mission, DOE has modified the field
structure with the establishment of the Energy Research
Centers for fossil activities, the Solar Energy Research Insti-
tute and the Regional Solar Energy Centers. However, there
has been no lead center created for energy conservation.
There is disagreement over what an appropriate field
structure for conservation would be. National Laboratories
have amassed a wealth of scientific and technical talent.
However, technical breakthroughs alone are not sufficient for
accelerating the use of energy conserving improvements and
solar systems. Certain institutional barriers such as zoning
restrictions, consumer assurance, building codes, access to
capital, and utility interface can hinder adoption of conserva-
tion and solar to a far greater extent than they can impede
other technologies.
Because of the talent and resources within the National
Labs, it is natural to assume that they can play a major role in
all new areas. For example, GAO** has recommended that
the National Labs play an increased role in nonnuclear energy
"The 1979 Section 11 Report (EPA 600/9-80-008, January, 1980) discussed the Major
Systems Acquisitions Process as it affected environmental assessments in tech-
nology development. See pp. 7-8, 17-18.
* General Accounting Office, The Multiprogram Laboratories: A National Resource
for Nonnuclear Research, Development and Demonstration, May 1978 EMD
78-62.
39
-------
technologies. The report suggested realignment of the eight
multiprogram laboratories under a separate office in DOE to
facilitate greater involvement in nonnuclear R&D. (Currently
the multiprogram laboratories report to the office in DOE that
is responsible for the principal activity of a particular lab.)
However, participants in the EPA Section 11 workshops
had a very different point of view from GAO. They expressed
the belief that DOE's prior commitment to maintaining the
National Laboratories stifles innovation by making it difficult
for small business, individuals and private sector labs to
compete for Federal funds on an equitable basis. It was even
suggested that DOE consider phasing the National Labs out
of solar and conservation work.
The following steps should enable DOE to take better
advantage of the resources in the National Labs, SERI, and
the RSEC's.
First, DOE should develop an explicit policy on the
use of the multiprogram laboratories in the conserva-
tion effort. This will not be an easy task since the labs report
to different offices within DOE. Only SERI and the RSEC's are
under the direct control of the Office of Conservation and Solar
Energy. Currently, individual laboratories, as well as SERI and
the RSEC's, develop both annual and multiyear plans for all
activities, including conservation. However, there does not
appear to be an overall DOE plan for distributing conservation
activities among various field operations. Decisions appear to
be made on an ad hoc basis, and each program office decides
how much of its work should be contracted to the National
Labs. This approach may yield excellent technical results; the
energy conservation groups at Oak Ridge and the Lawrence
Berkeley Laboratory are clear examples. However, other ob-
jectives can easily be lost sight of, including involvement of
small business, a proper balance between R&D and commer-
cialization, and encouragement of a regional and local em-
phasis for conservation activities.
Second, establishing a field structure to coordinate
both conservation and solar should be considered. The
current arrangement is a reflection of the overall lack of
integration between conservation and solar in DOE. Careful
consideration ought to be given to gradually expanding the
role of both SERI and the RSEC's into conservation. Perhaps
the most sensible approach would be to make SERI and/or the
RSEC's principal contractors for efforts that involve both re-
newables and energy conservation.
Finally, in some areas, SERI and the RSEC's should
be given more flexibility in the conduct of their day-to-
day operations. For example, SERI's FY-80 operating budget
was more than $120 million. Yet only $2 million was available
for discretionary activities. Both DOE and SERI are con-
strained by requirements in authorization and appropriation
bills. However, SERI's mission as a lead center for solar R&D
might be hindered if it does not have more flexibility to pursue
promising technical applications.
40
-------
Assessment of Program Evaluation
Overview
Considering the importance of choices for both resource allocation and man-
agement, does DOE have adequate information for decision making?
Members of Congress, the General Accounting Office, DOE officials, and
Section 11 participants have all stated clearly that adequate information for
energy policy formulation, resource allocation, program management, and ac-
countability has not been available. Program evaluation* could make a substan-
tial contribution to improving the information base by providing data on actual
program performance, and DOE should take the steps required to ensure that
this evaluative information will be produced. These steps include establishing
department-wide policy on the conduct of evaluation, creating incentives to
encourage good evaluations, and devoting resources to developing needed
research technologies.
Program evaluation is particularly important for conservation programs
because their effects are more difficult to document than those of many supply
programs, and less is understood about the factors that affect energy use.
Although evaluation of these programs is not easy, several high quality studies
in the past prove that it can be done. DOE's Office of Conservation and Solar
Energy is currently developing a comprehensive plan for evaluating its pro-
grams.
Background
specific policies has made it difficult for Congress and DOE to
chart a course which is most likely to achieve energy policy
goals. For just as it is important not to waste money on
programs that are not cost-effective, it is equally important
not to terminate a promising program before its effects are
known.
Need for Evaluation
"Fenagle's Law:
The information we have is not what we want. The
information we want is not what we need. And the
information we need is not available." (Eric Hirst,
Oak Ridge National Laboratory).
Both Congress and DOE have become increasingly con-
cerned about understanding and documenting the results of
energy programs. In many cases, however, lack of information
on program outcomes and the contribution of programs to
It is clear that serious attention is now being focused on
evaluative questions regarding energy programs. The need
for additional information on program costs, benefits, and
performance has been recognized by a number of sources,
including the Office of Technology Assessment, the General
Accounting Office, the Congress, observers of DOE pro-
grams, and DOE itself.
"Because the term "evaluation" is often used loosely to describe any formal or
informal assessment of actual or anticipated results of a program, it is important to
provide a working definition. For purposes of this discussion, evaluation is viewed
as a formal, systematic study of the actual processes or results of a program
Evaluation is a retrospective or historical activity, examining what has occurred or
is occurring It is therefore clearly different from projections, forecasts, and other
studies of proposed or hypothetical programs.
41
-------
Participants in the 1980 Section 11 program argued
strongly for the importance of more and better evaluation of
DOE's programs. Whether they were discussing policy
analysis, information dissemination programs, or state and
local assistance programs, the need for evaluation and the
information it produces was raised. In discussions focused
specifically on evaluation, the message was clear DOE
must systematically evaluate the performance and ef-
fects of its programs, and the results of these evalua-
tions should be used at all levels of the Department to
improve decision making.
Although the 1980 Section 11 program emphasized con-
servation and solar programs, participants stressed the
need for evaluation to be applied to the entire range of
DOE supply enhancement and demand reduction pro-
grams. In fact, one participant summed up the views of many
others when he stated:
"If the OS programs are more critically evaluated
than other energy programs, they may be put at a
disadvantage. Evaluation historically has tended to
be more negative than positive. If conservation and
solar programs are thoroughly evaluated and the
nuclear, coal, oil shale, etc. programs are not, con-
servation and solar will be at a disadvantage in
competing within a constrained program budget.
Proponents of competing energy programs will be
less vulnerable to criticism and also better armed to
criticize CSE programs." (Lewis Perleman, Jet
Propulsion Laboratory.)
Many of the examples in this chapter are drawn from conser-
vation and solar programs, but it must be stressed that
evaluation can only contribute to ensuring adequacy of atten-
tion to conservation and solar if all DOE programs are thor-
oughly evaluated. Therefore, the recommendations presented
in this Chapter are not directed at the Office of Conservation
and Solar Energy but are intended for DOE as a whole.
Current DOE Evaluation Activities
Many DOE officials recognize the need for more informa-
tion based on program evaluation. Within Conservation and
Solar Energy (CSE), several evaluative activities are under-
way. Many of the program offices are conducting or planning
studies of specific programs. CSE reports that evaluations are
"in process" for the following programs and projects: Schools
and Hospitals, Energy Conversion and Utilization, Appropri-
ate Technology, Weatherization, Comprehensive Community
Energy Management, Solar Heating and Cooling, and Solar
Applications for Buildings. In addition, there are plans to
evaluate the Residential Conservation Service, Appliance
Standards, State Energy Conservation Programs, Energy Ex-
tension Service, Energy-Related Inventions, and Solar In-
ternational (SOLERAS). Staff within the Office of the Assist-
ant Secretary for Policy and Evaluation provide informal guid-
ance and assistance in these efforts.
CSE's Office of Policy, Planning, and Evaluation is cur-
rently developing an Evaluation Plan for all CSE programs.
This effort has included a survey and inventory of current and
past evaluation activities, identification of the CSE Assistant
Secretary's information needs, development of evaluative
criteria, and definition of roles for implementing evaluations.
It is too early to comment on these activities; however, CSE is
to be commended for taking the initiative to develop this plan
in the absence of specific guidance or requirements from
higher DOE management. To some extent, the Office of Con-
servation and Solar Energy represents a microcosm of the
overall variation among programs at DOE since CSE includes
all of these program types. Thus, insofar as CSE can overcome
comparability problems in its Evaluation Plan, it will demon-
strate the feasibility of using evaluative information for cross-
technology comparisons.
DOE is also beginning to work on its response to the
"Sunset Provisions" (Title X) of DOE's Organization Act (Pub-
lic Law 95-91). This section mandates a comprehensive re-
view of each DOE program to be submitted to Congress no
later than January 15, 1982. The review must include:
An identification of each program's objectives;
An assessment of the degree to which the original objec-
tives of each program have been achieved in terms of
performance,.impact, or accomplishments;
A statement of the number and types of beneficiaries or
persons served by each program; and
An assessment of the effect of each program on the
national economy, health and safety.
Preparation of the Report win be the joint responsiblity of
DOE's Chief Financial Officer and the Assistant Secretary for
Policy and Evaluation.
An in-depth review of all of the areas included in the
Sunset Provisions is an enormous and expensive undertak-
ing. It is not clear that DOE's approach to this requirement
accurately reflects its importance. Considering the Sunset
Review's potential visibility and the vigorous debates it
is likely to engender, DOE does not appear to be devot-
ing the necessary time or effort to it. Conservation pro-
grams in particular may have difficulty documenting their
results because of the lack of available evaluative information.
Currently, DOE appears to be emphasizing avoidance of
overlap with their Annual Report, and may not be giving
adequate attention to requirements for consideration of alter-
natives to each program and steps that would be necessary to
phase out a program. Guidance to the program offices for
preparing material for the Sunset report is not scheduled to be
42
-------
available until February. This leaves very little time to assem-
ble necessary documentation.
Much of this section is devoted to examining the prob-
lems involved in designing and implementing evaluations. It
is important to stress, however, that these problems are not
insurmountable, and that DOE has conducted high quality
evaluations in the past which have contributed to decision
making. Section 11 participants cited DOE's evaluation of the
Energy Extension Service* as an example of the value and
feasibility of evaluating DOE's conservation and solar pro-
grams. This study demonstrated that many methodological
problems can be overcome and that evaluation can provide
useful insights for decision makers. The findings of this study
supplied essential information for Congress' deliberation on
whether to expand the program nationwide, and guided the
development of the Program Planning Manual, which advises
states on how to set up their programs.
Uses of Evaluation
Adequate information on the effects of energy pro-
grams and the performance of specific DOE projects has
not been available to Congress, DOE officials or the
public. DOE should systematically evaluate its major
programs and incorporate this information into its ongo-
ing management processes.
As those who have experience in designing and con-
ducting evaluations are aware, evaluation can serve a variety
of purposes, and no single study can serve them all. These
purposes include:
Basic Knowledge: it can contribute to an improved data
base for policy formulation.
Resource Allocation: it can enable decision makers to
use more precise analytical tools in the resource allocation
process to compare the relative cost-effectiveness of vari-
ous programs.
Program Management: it can assist program managers
in improving the efficiency and effectiveness of their pro-
grams.
Accountability: it can permit Congress, the public, and
the press to scrutinize programs.
Although these areas are not as clear cut and distinct in
practice as they are on paper, they highlight the wide variety
"Examples of other evaluations of conservation and solar programs conducted by
DOE, other Federal agencies, and state agencies are outlined m Appendix B.
of areas in which evaluation can be useful. Participants in the
Section 11 program felt that it was important to emphasize
that evaluation does not only contribute to "go-no-go" deci-
sions,. Therefore, each of these potential uses of evaluation are
discussed briefly below.
Basic Knowledge. DOE already understands a great
deal about how energy is produced, but it needs to
understand a great deal more about how energy is
used. Currently, policy makers do not have an adequate
understanding of issues such as what will actually cause
changes in energy consumption price increases, informa-
tion and promotion campaigns, tax incentives and how
lasting certain behavioral changes are. Although enhancing
basic knowledge is rarely a primary purpose of an evaluation
study, accumulation of a number of evaluations will in
fact contribute significantly to DOE's understanding of
energy problems and potential solutions. For example,
evaluations of outreach and information programs can pro-
vide a better understanding of consumer behavior and how
conservation programs interact with the market to affect
consumption; evaluations of demonstration programs can
lead to more accurate characterization of market forces and
barriers to adoption of technologies or products. Knowledge
of this type will reduce some of the uncertainty that now
surrounds policy making, especially in terms of what level of
energy savings can reasonably be expected in the future from
various types of Federal activities, and can enable program
planners to design programs with greater likelihoods of suc-
cess.
For evaluation to produce this type of information,
studies cannot just measure the effects of programs. They
must also identify the reasons those effects occurred. From
this perspective, it should be noted that an "unsuccessful"
program can make as great a contribution to basic knowledge
as a "success."
Resource Allocation. In DOE's resource allocation process,
the Planning, Programming, and Budgeting System (PPBS)
evaluation can contribute data to support comparisons
of options and selection of the most advantageous pro-
gram alternatives. This type of analysis requires evaluation
information on the entire range of supply and conservation
program options. Projections of anticipated costs and
benefits of both supply and conservation programs can
be improved significantly as more is learned about ac-
tual cost and performance. This information is particularly
important to conservation programs, first since the effects of
these programs are more difficult to document, and second
because the program results are influenced by many
variables, such as consumer behavior, that are not currently
understood.
Ideally, cross-cutting measures of effectiveness would be
used in these comparisons. Cost per barrel of oil equivalent
and cost of energy service delivered have been suggested as
criteria. It has also been suggested that unintended environ-
mental, health, safety, and economic side effects also be
43
-------
included in these comparisons. Evaluative data can lead to
increasing refinement and accuracy of these measures. The
oil import premium concept and least-cost model, discussed
in an earlier Section, are examples of analytical devices that
depend to some extent on evaluative data.
Program Management. Of the various purposes evalua-
tion can serve, providing feedback to managers to improve
programs recerved the strongest support from participants rn
the Section 11 process. It is perhaps the most readily achieva-
ble objective for evaluation since these studies do not neces-
sarily get enmeshed in problems of measuring impact and
attributing causality. A key objective of this type of evalu-
ation is to provide timely feedback on aspects of a pro-
gram that are amenable to change. A major emphasis is
identifying problems and recommending solutions rather
than measuring ultimate impact. For example, an evaluation
could assess the appropriateness of the methodology of a
research program, the quality of materials and dissemination
procedures" in an information program, or the number and
types of homes reached in a weatherization program It could
identify problems encountered by state energy offices in in-
terpreting regulations or local agencies in complying with
schedules and reporting requirements. In short, evaluation
can provide program managers with information
needed to make "mid-course corrections" and maximize
program efficiency and effectiveness.
This type of evaluation can also be extremely valuable in
designing new programs. For example, the new Residential
Conservation Service should build upon the lessons learned
in previous utility conservation programs, such as those
conducted by the Tennessee Valley Authority and Pacific Gas
and Electric. TVA found in one audit program, for example,
that it was not difficult to generate interest in the program
through an outreach effort, but that it was far more difficult to
meet this demand than expected. As a result, they were
unable to perform as many audits per month as anticipated
and built up a substantial backlog of audit requests. TVAs
experience should help RCS estimate how many audits it can
perform and how it can organize to perform those audits most
efficiently.
Accountability. Although evaluations can provide useful
information for oversight and monitoring purposes, evalua-
tion that is used to provide accountability can be the most
threatening to a program manager for obvious reasons.
Whenever the results of an evaluation are used in a public
forum such as a Congressional hearing or newspaper article,
they can easily be misused by a program's opponents or
proponents. Few evaluations have clear, unambiguous con-
clusions, and all are subject to numerous caveats about their
findings Yet in public debates, their conclusions are often
oversimplified and quoted out of context. Therefore, this is a
particularly touchy use of evaluation.
As DOE officials respond to internal and external pres-
sures to "do more evaluation," they must carefully sort out
what they expect from evaluation and how it will fit into
existing management systems. Above all, expectations for
the potential contributions of evaluation should be
realistic. In other fields, such as education and human
services, large expensive studies were launched involving
methodologically sophisticated research on program impact.
But more was expected of evaluation than it could deliver
because the data collected were not always as good as
anticipated; this type of research is inherently conservative,
tending to err toward a finding of no significant effects; and
finally, political and bureaucratic opposition to evaluation was
generally underestimated. Therefore, in many cases, evalua-
tions were not able to detect the positive results anticipated
from popular programs, and in other cases were unable to
authoritatively attribute changes in the affected population to
the program under study. As a result, many decision makers
became disillusioned with evaluation and its ability to con-
tribute to decisions.
The remainder of this Section discusses specific steps
that DOE should take to ensure that evaluations are per-
formed and used. It begins by suggesting elements of a
department-wide policy that would guide evaluation ac-
tivities. Then, it suggests ways of addressing some of the
institutional and methodological barriers that may inhibit
implementation of this policy. Throughout this section, EPA
presents recommendations which are tempered by an
awareness of the limits of evaluation. This does not represent
equivocation on the value of evaluation we believe that
evaluation is crucial for developing the information Congress
and DOE needs. However, we are equally firm in our belief
that evaluation should not be oversold.
Evaluation Policy
If DOE is to produce consistent information on program
effectiveness, departmental policy should be estab-
lished regarding types of programs to be evaluated,
department-wide information requirements, timing,
roles and responsibilities of various offices, and funding
mechanisms. All offices with a stake in the evaluation
process should be involved in developing this policy.
Although the need for evaluation has been recognized by
many within and outside DOE, overall guidance on conduct-
ing evaluations has not been provided to program managers
and planning offices, nor has a funding mechanism been
established. Because of the organizational and methodologi-
cal barriers to conducting evaluations, many managers have
been reluctant to initiate a major evaluation effort without
such direction.
As is clear from the experience of other Federal agencies,
developing an agency evaluation strategy and methodologies
44
-------
to support it are not simple tasks. Convincing managers and
decision makers at all levels that evaluation can be a useful
tool is perhaps even more difficult. Thus, it is critical that
those who will implement evaluations and those who
will use their results be involved in developing this
policy. As an example of this type of involvement, CSE's
Office of Policy, Planning and Evaluation has formed an
Advisory Committee to assist in developing its Evaluation
Plan. This group is composed of representatives of each CSE
Deputy Assistant Secretary, and advises on issues such as
evaluation criteria; requirements for evaluation methodology
research and development; priorities and schedules for im-
plementing the Evaluation Plan; and requirements for evalua-
tion, guidelines, data sources and evaluation support.
Several topics should be addressed in a departmen-
tal policy on evaluation. These include criteria for setting
priorities among programs to be evaluated, information to be
produced, timing, definition of responsibilities, and funding.
Some type of evaluation may be required of all programs,
but certain programs may merit greater emphasis than
others. Thus, the procedures and criteria used to set
priorities among programs to be evaluated are critical.
Discussions with Section 11 participants and DOE officials
suggested a number of criteriathe Section 11 workshop in
Menlo Park developed a list of approximately twenty potential
elements. The following items seem to be most useful.
Is there a statutory requirement for evaluation? Even in
these cases where evaluation is required though, decisions
must be made on the type of evaluation to be conducted
and the level of effort to be expended.
Is the program significant in terms of high costs, visibility,
potential energy savings, or potential impact on the envi-
ronment, safety/health, or the economy? Is the program a
candidate for expansion or greatly increased funding?
Can the results of an evaluation influence decisions regard-
ing the program? Evaluations should be tied to program
and project decision schedules, and programs should be
selected for evaluation so that information will be available
when decisions are being made. Particularly high priority
should be given to evaluating programs about which there
is controversy.
Can the evaluation be done? A great deal of time and
money can be wasted on programs with unclear goals,
uncooperative personnel, or large methodological barriers.
These problems should be solved before an evaluation is
attempted.
Is the program's performance marginal or suspect? There
should be a system of "red flags" indicators that a
program may be in troublethat could suggest the need
for an evaluation. These could include a series of missed
milestones, unexpected costs, or erosion of industry inter-
est.
Is there a great deal of uncertainty about the program? New
programs and pilot programs should receive priority
Is the program typical or can it yield information that will
contribute to decisions on important issues? When infor-
mation can be generalized to other programs or is needed
for a specific purpose, programs should be selected to
produce that information. DOE policy makers should try to
anticipate major issues that are likely to arise and plan
studies to generate information that will be needed in
these debates. For example, the role of state and local
government is likely to be an important issue in coming
years, as is defining cost-effective approaches to equity
questions.
A second element of an evaluation policy should address
the type of information to be produced and the evalua-
tive criteria to be applied.-Criteria will vary by the purpose
of the evaluation and by type of program. For example, evalu-
ation to support resource allocation would be geared toward
facilitating comparisons of program alternatives and selec-
tion of the best ones. For this purpose, comparable data on
program costs and benefits would be most useful. A DOE
evaluation policy should describe the uses to which evalua-
tive information will be put department-wide, the specific
data that will be required, and when the data will be needed.
The more precisely these requirements are described, the
more likely compliance will be. These elements should then
be incorporated into guidance and instructions for PPBS, the
Major System Acquisition process, and other appropriate
management and reporting systems.
An evaluation strategy should also address the
issue of timing. It is generally agreed that the best approach
to evaluation is to build it into programs from the beginning,
thus allowing for the collection of adequate baseline data and
design of programs to facilitate evaluation. A key question
regarding timing, however, is when to begin measurement
and reporting. Evaluations of program outcomes should not
be forced too early. For example, all programs have start-up
costs, and costs of a program will often decline and then
stabilize after the "bugs" have been eliminated. Further, the
impacts of information programs and new standards or regu-
lations are not felt immediately and premature measurement
of results will underestimate their ultimate effects. Perhaps
most difficult are R&D programs for which impacts may not
be observable for several years.*
However, if evaluation results are to be useful, they must
be available when decisions are being made. In general,
studies that attempt to apply experimental or quasi-
experimental designs require the longest time, and some
decisions about the program will occur before their final
TWo examples illustrate the delays in observable effects of R&D A study of DOD
R&D funded between 1945 and 1963 found that there was usually a 5 to 10 year
delay before R&D investment paid off Another study prepared for the National
Science Foundation concluded that the delay in R&D payoff could be as long as 30
years From John Salasm, et al . The Evaluation of Federal Research Pro-
grams, MITRE Technical Report MTR-SOW 129. June 1980, page 30.
45
-------
conclusions are reached. In most studies, though, evaluators
can provide some feedback fairly early, although data is not
likely to be as comprehensive or as thoroughly validated as
the final product. Case studies and descriptive data usually
can be produced quickly and can be useful in many decisions.
The evaluation policy should also clearly define re-
sponsibility for initiating, overseeing and implementing
evaluations. Currently, there is inadequate delineation of the
evaluation responsibilities of the Assistant ^Secretary of Policy
and Evaluation, program offices, and staff offices (such as
Conservation and Solar Energy's Office of Policy, Planning,
and Evaluation).
There are many different organizational arrangements
that can be employed and none is inherently better than the
others. Responsibility can be delegated to program offices or
the cross-cutting staff support offices, or it can be shared. One
study of Federal evaluation policy suggested that"... respon-
sibility for evaluation should be placed at a level appropriate to
the decisions it is designed to assist..." and "... evaluations
should be directed by persons not having a great deal to gain
or lose from the outcome. Thus ... major responsibilties for
evaluating projects and alternative strategies within the pro-
gram should rest with program managers, but responsibility
for evaluating the worth of an entire program should be placed
above the program level." * The study goes on to recommend
that where states and local governments are responsible for
program implementation, they should be provided funds and
technical assistance to conduct evaluative studies. Partici-
pants in a symposium on Federal program evaluation noted
that "the threatening nature of evaluation may be the most
important obstacle to its effective use... To overcome this, the
panel believes in the importance of including 'victims' in all
phases of evaluation projects from pre-design and planning
through execution and product packaging." **
Participants in the Section 11 program were divided in
their opinion. Some suggested that evaluations intended
primarily to support resource allocation be conducted by the
Office of the Assistant Secretary for Policy and Evaluation or
the CSE Office of Policy Planning and Evaluation, while
studies intended primarily to support program management
should be the responsibility of the appropriate program office.
Other participants felt that the role of the Office of the Assist-
ant Secretary for Policy and Evaluation should be limited to
technical assistance and oversight, but that evaluations
should be actually conducted by program offices. (The latter
more closely corresponds to the current arrangement, and to
the Evaluation Plan being prepared by Conservation and Solar
Energy)
Eleanor Chebmskv An Analysis of the Proceedings of a Symposium on the
Use of Evaluation by Federal Agencies. Vol II, MITRE Report M77-.39, July
19" D .'
The important point is that various offices understand
what their responsibilities are and have the capability to carry
them out. This implies that appropriate offices have staff
available who can perform both contract management (as-
suming outside contractors perform at least a portion of the
work) and technical oversight activities. Additional slots, staff
training or technical assistance may be required to develop
these capabilities.
In addition to defining responsibility within DOE, there
was a great deal of consensus among Section 11 participants
on the need to involve state and local agencies in evalua-
tion efforts since a large portion of DOE's conservation pro-
grams are implemented and managed by these agencies.
Further, variations in implementation strategies and methods
provide an excellent opportunity for collecting comparative
data. For states to play a role in evaluation, they win need
technical assistance. A participant from a state noted for its
efforts in evaluation suggested that DOE:
" .. provide technical assistance to the states by
developing model evaluation components, com-
puter software, and an evaluation handbook; im-
prove its understanding of how states implement
Federal conservation programs and, thus, what will
appropriate evaluation requirements be (1) state
Federal staff exchanges and (2) creation of an evalu-
ation advisory committee ... ; conduct basis re-
search on conservation; and provide states with
financial resources to develop baseline end-use
data and to meet evaluation requirements." (Keith
Kozloff, Minnesota Energy Agency.)
DOE should also involve utilities in its evaluation
efforts for several reasons. First, utilities are a valuable,
largely untapped source of data. Second, the utilities are
increasingly involved in implementing their own and
Federally-sponsored conservation and renewables programs
and, therefore, have a stake in ensuring program effective-
ness and management efficiency. Finally utilities have a
stake in evaluation as it can contribute information for the
difficult decisions they face regarding investments in new
capacity versus investments in reducing demand. For exam-
ple, the recent decision by Southern California Edison to try to
meet increased demand through a combination of renewable
technologies and conservation instead of new coal projects
demonstrates the persuasiveness of information. In this case,
independent reports by the Environmental Defense Fund and
the Department of Interior showed the cost-effectiveness of
conservation and renewables when compared to the coal
plants. Although other considerations certainly affected the
decision, many believe that these reports played an important
role. DOE's evaluation activities should be designed to assist
the utilities as appropriate and to obtain useful data from
them.
Finally, an evaluation policy should address the
issue of funding. Evaluation should be viewed as an invest-
46
-------
ment in information that win improve DOE's management
and decision making capabilities. Evaluation is expensive,
but as one Section 11 participant stressed, "we can't afford not
to do it." The money for evaluation can come from various
sources: normal budget requests, discretionary funds, a spe-
cial agency-wide "tax'' designed to support evaluation. Some
Federal agencies require that a specific percentage of all
projects be set aside for evaluation, which is one option for
DOE to consider. Another option is for a department-wide
evaluation budget, with particular evaluation studies
negotiated in that framework.
The Office of Technology Assessment* recommended
that five to ten percent of total program funds be set aside for
evaluation. In testimony at the Section 11 National Hearing, a
witness with evaluation experience in a number of Federal
agencies estimated that six percent of program costs may be
necessary for start-up of evaluation activities, and that one
and a half percent would be needed after program operations
have stabilized.
Institutional Barriers
to Implementation
Successful evaluation requires an environment that is
receptive to new information and open to change. To
foster this type of environment, top DOE management
must demonstrate their commitment to using evalua-
tion results and develop incentives for good evaluation
at all levels of the department.
Adoption of evaluation as a management tool
within DOE will require more than establishment of a
policy. Although a policy may be successful in obtaining
compliance with its paper requirements, it cannot ensure that
evaluation results actually are used in decision making. Parti-
cipants in the Sectional! program emphasized that this will
require a serious commitment from senior officials, and
evidence that their commitment has led to action. They
must be willing to ask difficult questions about the worth of
programs, and act on the answers. This does not imply, of
course, a one-to-one correspondence between evaluative
results and actions taken, because many other factors must
be considered in decision making; it does mean, however,
that decisions may require more detailed justification when
they appear to run counter to evaluative information.
There is a great deal of institutional resistance within any
organization to evaluation and use of its results. When ap-
proached improperly, evaluation is viewed as a threat or a
punishment, and obtaining accurate data can become virtu-
ally impossible. This is particularly relevant for DOE, because
it has been under fire since its inception. As a Section 11
participant noted, "criticizing DOE for our energy problems
has become a national pastime only slightly less ubiquitous
than baseball." Thus, it will be critically important for DOE
managers to build in incentives to establish support at
all levels of the organization. Several types of incentives
should be developed.
First, program managers should be rewarded for
conducting high quality evaluations and acting upon
their results. This does not mean that managers should only
be rewarded for positive evaluations; on the contrary, greater
incentives should be provided for routinely identifying prob-
lems within programs and projects, and attempting to resolve
them. Evaluative studies should be designed to complement
this incentive structure. Instead of the "gotcha!" approach
which some evaluators enjoy, program managers should be
involved at all stages of the study, and should receive regular
feedback from the researchers. Evaluation can be sold to
program managers as a means of staying one step ahead of
criticism.
Another type of incentive regards the status of evalua-
tion and those who do it. One Section 11 participant noted
that "there is a bit of a tendency to talk about evaluation as if it
just 'happened,' without our having to pay much attention to
who was doing the work." He went on to stress the im-
portance of "establishing evaluation as a high-status occupa-
tion within the energy business, in order to attract to it some
of the very best talent, not just the left-overs from more
'important' work." In addition to raising the status and
caliber of evaluators, users of evaluative information
should be more knowledgeable, thus making them a
more competent and supportive audience. Specific ac-
tions which DOE should consider to promote this type of
incentive are in-service training and support for other forms of
education; establishing published professional literature on
energy program evaluation; supporting workshops and con-
ferences among energy program evaluators within and out-
side DOE (such as state and local agency staff, utility program
evaluators, etc.), and between evaluators of energy programs
and evaluators from other fields. **
One incentive for all levels of management would
be removal of two barriers to use of evaluation. First, it
could be made easier for programs to be changed in
response to evaluations. In some instances, there may be
fewer requirements and more discretion over program design
or operation so that improvements can be made. Inflexible
* Office of Technology Assessment, Conservation and Solar Energy Programs of
the Department of Energy: A Critique, GPO No. 052-003-00757-6, p. 21
'For example, during the Section 11 program, thirty-five experts in energy program
evaluation and evaluation of health, welfare and education programs met for a day
and a half to discuss evaluation of conservation and solar programs and lessons to be
learned from other fields; participants felt strongly that this was an extremely useful
meeting and urged DOE to conduct follow-up activities
47
-------
program requirements discourage use of evaluation. For
example, some of the problems discovered by formal and
informal evaluations of DOE's state and local assistance pro-
grams cannot be readily corrected because they arise from
legislative requirements. A second change is greater use
of pilot programs and small scale demonstrations. Prior
to full scale implementation of a nationwide program, it
should be tested and carefully evaluated. This would result in
many problems being averted. The phase-in of the Energy
Extension Service provides a model for this approach.
Methodological Barriers
to Implementation
savings are not always measurable, especially in the time
frame required.
Establishing Causality. Classical research designs em-
ploy experimental methods with control groups to clearly
establish cause-and-effect relationships among indepen-
dent and dependent variables. In the real world, however, it
is virtually impossible to create such a "laboratory" situa-
tion and various techniques are substituted for the ideal.
The generic issues concerning causality can (and do) fill
many research textbooks. Problems specific to conserva-
tion and solar programs include the difficulties in control-
ling for external variables, regulations that limit flexibility to
introduce planned variations into programs, and inability
to randomly select program participants or assign them to
participant and control groups. Lack of an adequate data
base regarding energy consumption generally and in vari-
ous end-use sectors exacerbates this situation.
Evaluations should focus not only on program impact
but should also examine program processes, and
evaluators should employ both quantitative and qual-
itative techniques in these assessments.
Evaluation is often viewed only as an assessment of
impact. In energy program evaluation, this perception may be
narrowed even further when users expect evaluation results
to be reduced to a single figure such as cost per barrel of oil
equivalent. But to enable evaluation to fulfil an expanded role
within DOE, it should focus not only on program impact
but should also examine program processes. This will
provide information both on program results and the reasons
for them. Decision makers need to understand why a program
worked or did not work, and which program elements were
responsible for success or failure. If a program works, others
will want to imitate it, and they need to know what features to
replicate. For example, DOE's evaluation of the Energy Exten-
sion Service pilot programs focused on which programs
worked best and why. It was able to identify the types of
services that were most effective and the target audiences
that found these services most useful.
For many conservation programs, evaluation of results is
a very difficult task. Whereas a supply program can point to a
demonstration plant capable of producing a given quantity of
fuel per day as an indicator of success, conservation programs
must attempt to document energy savings. Serious methodo-
logical barriers inhibit efforts to define and measure energy
savings, and to authoritatively attribute these savings to a
particular program. Examples of these barriers include the
following:
Defining Measurable Outcomes. Defining outcomes to
be measured requires agreement on a clear set of program
goals and objectives. But goals are often not described in
measurable terms. For most conservation programs, "en-
ergy savings" is posited as the desired result But energy
Collecting Valid, Reliable Data. Few mechanisms exist
for gathering information on actual behavioral changes and
changes in consumption or efficiency. Most studies rely on
self-reported data, which is notoriously unreliable. Re-
spondents tend to give answers they believe are desired by
the questioner, which can seriously skew the data. It is
difficult to estimate and correct for this bias.
Interpreting the Data. Because so many factors can
affect energy consumption and so little is understood about
them, it is sometimes difficult to interpret evaluative re-
sults. In residential audit programs, for example, installa-
tion of conservation measures does not always result in
saved energy. This can occur because of increases in the
number of people in the house or because residents were
able to increase their comfort level (e.g., turning up the
thermostat) or increase their activity level (e.g., open up
unused rooms) for the same price. Without additional
understanding of consumer behavior, interpreting even a
relatively clear indicator, like energy savings, becomes
difficult.
Tb design successful evaluations which will address
these barriers and respond to identified information needs,
studies must be tailored to specific programsno single
methodology can be applied to all programs. DOE supports a
wide variety of programs, such as basic research, information
dissemination, communication and marketing, grants,
standards and regulations, each of which involves different
activities and objectives. Evaluations must be designed to
address each program type's goals and components. For
example, data collection techniques for a basic research
program on properties of materials or the physics and chemis-
try of an industrial process would differ from data collection
techniques for an informational program, such as the Energy
Extension Service or the Low-Cost/No-Cost Project. Similarly,
regulatory programs and standards, such as Building Effi-
ciency Performance Standards and grants programs, such as
Weatherization Assistance, wiU differ. Therefore, although
48
-------
guidelines and policies for evaluative information and
minimum criteria can be articulated, flexibility must be re-
tained so that evaluation designs can accurately assess these
specific programs.
The barriers discussed for DOE programs suggest that
quantitative studies of impact should be augmented by
examination of processes and non-quantitative meas-
urement techniques. Quantitative methods exist for
measuring some program impacts and should be used
whenever possible. However, this information can be further
enhanced through measurement of intermediate results (pro-
gram outputs, such as numbers of people contacted) and case
studies. Case studies are particularly useful in helping to lay
the groundwork for testing hypotheses, as a preliminary
activity to more rigorous evaluation, and as a means for
providing guidance and constructive recommendations to
managers. "Put another way, evidence based on statistical
analysis of desirable project characteristics is not understood
or trusted by program managers. Short case studies which
contain essential elements of success give program manag-
ers much more information and more evidence that the
contractor's understanding is deeper and does not reflect
what they view as simple statistical manipulations." *
As a corollary to the above, alternatives to traditional
experimental designs should be explored. Evaluation
methodology is itself a potential target for R&D activities.
Effort should be focused on developing credible, feasible
research design and measurement techniques, and data
collection methods. This developmental work could draw
extensively from recent research in other fields, and the peer
review mechanisms currently in place.
Finally, more comprehensive data on energy use and
energy using systems should be developed. It is difficult
to interpret the results of evaluations without adequate in-
formation about the environment within which programs
operate. Compiling such data bases on consumption and
capital stock in each-end use sector is, however, expensive
and time-consuming. Further, these data require periodic
updating if they are to be useful. The Energy Information
Administration (EIA) is currently working on improving and
expanding data available through a National Interim Energy
Consumption Survey covering all four end-use sectors, and
specific studies within each sector. Several Section 11 partic-
ipants suggested that EIA should draw more extensively on
utility data as well. EIA could become a valuable partner in
evaluation as well as data collection activities by providing
data needed to describe the context within which programs
operate. For example, the evaluators of the Weatherization
Assistance Program, currently in a planning stage, worked
with EIA to insert several questions into the Residential
Energy Consumption Survey. This will save money for the
evaluation while not increasing EIA's costs substantially and
will provide very useful data. Evaluators should carefully con-
sider ways they can tap this valuable resource. EIA could also
assist evaluation efforts by incorporating results of evaluative
studies into its data base, comparing and reconciling the
conclusions about individual program impacts.
In addition to organization by end-use sector, EIA's data
base activities could also be geared toward the needs of state
and local agencies. A significant portion of the planning and
management of conservation programs is performed by state
energy offices, local agencies and utilities. These state and
local agencies frequently do not have the capability to build
and maintain adequate data files. Section 11 participants
emphasized that this could contribute significantly to en-
hancing their ability to ensure program efficiency and effec-
tiveness.
'Eleanor Chelimsky, An Analysis of the Proceedings of a Symposium on the
Use of Evaluation by Federal Agencies, Vol. II, MITRE Report M77-39, July
1977, p 22
49
-------
Appendices
-------
A. Summaries of
Public Participation
National
Hearing Summary
The Section 11 National Hearing was held in Washington, D.C., on
September 24 and 25,1980. The Hearing panels included the Assist-
ant Secretary and three Deputy Assistant Secretaries from the DOE
Office of Conservation and Solar Energy; the Deputy Assistant Sec-
retary for conservation from the DOE Office of Policy and Evaluation;
representatives from the Environmental Protection Agency and the
Office of Management and Budget; Congressional committee staff;
the Congressional Office of Technology Assessment, state govern-
ments, solar and conservation interest groups, utilities, and industry
Thirty witnesses presented testimony, and five additional partici-
pants subsequently submitted written testimony. Witnesses repre-
sented a broad spectrum of interests and backgrounds, including
industry, environmental organizations, solar and conservation public
interest groups, research organizations, universities and research
laboratories, and state and local governments. Many of the partici-
pants had attended one of the Section 11 workshops and meetings
held earlier this year.
Prior to this Hearing, witnesses were sent materials discussing
the focus of this year's Section 11 activities and the major issues that
emerged from the workshops. Most of the witnesses addressed one
or more of these issues in their testimony. This Appendix sum-
marizes comments from the National Hearing, and is divided into
four sections that correspond to the four sessions of the Hearing
Policy, Evaluation, State and Local Programs, and Research, Devel-
opment, and Applications. The Appendix is not intended as a
comprehensive record of all the issues addressed in the witnesses'
testimonies; for the full record, a complete transcript of the Hearing
has been published.
Policy Analysis
The first Hearing session dealt with conservation and solar policy at
DOE, and the ways that policy translates into actual programs of
research and development, commercialization, or other applica-
tions.* The witnesses all expressed concern over the gap between
the ambitious policy goals established for conservation and solar and
the current level of support these technologies receive in DOE's
programs. If DOE's stated goals for these technologies are to be met,
the witnesses agreed, support for conservation and solar will need to
increase substantially.
Need for Conservation and Solar. There was general agreement
among the witnesses about the need for expanded use of conserva-
tion and solar. Some saw conservation and solar as the quickest,
least expensive means of reducing our dependence on foreign oil and
improving national security as well as the economy. Other witnesses
stressed the unacceptable environmental consequences of con-
tinued supply expansion. All witnesses saw substantial opportuni-
ties for investments in conservation and solar that are economically
preferable to fossil or nuclear technologies, and it was repeatedly
emphasized that large economic savings are available through con-
servation and solar at today's energy prices. For example, Henry
Kelly head of the Analysis and Applications Directorate at the Solar
Energy Research Institute (SERI), outlined the results of recent re-
search conducted at SERI and Lawrence Berkeley Laboratory into
the potential for energy conservation in buildings. It will be techni-
cally feasible and economical, he said, to reduce energy demand in
the buildings sector by a factor of two over the next 20 years without
constraining building construction or decreasing comfort levels. He
contended that the impact of a coherent program to save energy in
buildings could be as large as any national program to supply energy
from other sources.
There were differences in opinion among the witnesses about
the barriers to and incentives for conservation. Some witnesses held
that conservation decisions are made primarily in response to higher
prices. Other witnesses, while acknowledging the importance of the
price incentive, pointed out that many barriers exist that reduce or
prevent energy consumers from responding to price alone, such as
the lack of skilled energy auditors, inadequate incentives for utilities
to invest in conservation, and a lack of consumer awareness con-
cerning economic investments in conservation.
Role of Government. All the witnesses agreed that the Federal
government should play a role in encouraging conservation and
solar; as with other issues, however, there were differing views on
what that role should consist of
Alvin Aim, from the John F. Kennedy School of Government at
Harvard University, and other witnesses considered the govern-
ment's primary function in energy policy to be enhancing competi-
tion and the operation of the free market for all forms of energy,
including conservation. He perceived a need for government in-
volvement in conservation beyond supporting competition, though,
pointing out the numerous market imperfections that constrain
conservation. He also argued for providing subsidies to conservation
"This section includes the testimony of some witnesses who participated in the
Research, Development, and Applications session, as their remarks on DOE's program
balance complement those of the Policy Analysis session witnesses.
53
-------
to offset the hidden costs of imported oil to our national security,
costs that the market does not pass on to consumers. Steven
Carhart, from the Mellon Institute in Arlington, Virginia, said that
government programs to develop and commercialize technologies
should complement private efforts in the same areas, focusing on
activities that the private sector is not willing to pursue on its own
because of high risk or low return on investment
"The implication of this is that under non-emergency con-
ditions the role of the Federal government will be primarily
to maintain and enhance the competitiveness of that mar-
ket, to remedy market failures as they are identified, and to
deal with the externalities related to energy" (Steven
Carhart, Mellon Institute.)
Other witnesses had less confidence in the ability of the free
market to adequately address energy problems. David Brower, of
Friends of the Earth, discussed the market's effect on three critically
important issues:
"The danger of atomic war, a policy for using the world's
natural resources, and nationalism. ... Energy is deeply
involved in every one of the three. And the free market,
which seems to have been looked longingly to (in the
Section 11 workshops) has a perfect record of exacerbating
each of these three problems.'' (David Brower, Friends of
the Earth.)
Instead, Brower said that we need a "new pair of glasses" that
will enable us to see the energy problems from a more global perspec-
tive that includes resource depletion, environmental degradation,
and overall quality of life.
Most witnesses agreed that DOE's role in conservation and solar
energy should encompass a wide range of activities, including:
research and development on new technologies, use of the tax
system to structure incentives for conservation and solar; experi-
ments in new institutional mechanisms for delivering conservation
and solar technologies; grants to states and localities aimed at
building local capacity, and dissemination of information on solar
and conservation.
Resource Allocation. Several witnesses focused their testimony
on the need for DOE to base its resource allocation decisions on more
rational, systematic comparisons between supply and conservation
programs. There was consensus that cross technology comparisons
do not play a significant role in DOE's decisions, and in the absence
of such comparisons, conservation and solar programs remain un-
derfunded relative to the contribution they could make to our energy
needs.
This lack of systematic comparison ran counter to what many of
the witnesses felt should be the basis of DOE's policy Carhart
pointed to the Mellon Institute's Least Cost Energy Strategy as a
method of cross technology comparison. He described the least cost
approach as a process which identifies society's end-use energy
needs heated buildings, mobility, etc and then identified the
mix of technologies which meets those needs at the minimum cost.
An important element of the analysis, he stated, is the inclusion of
hidden costs such as environmental impacts in the overall cost of
each technology. The analysis thus reveals the least cost strategies in
social terms, not merely economic terms.
Ah/in Aim saw the need for an analytic tool based on the cost of
imported oil to use in comparing and selecting energy programs. He
explained the concept of the oil "premium" price a price some-
where above the market price which would incorporate the external
costs to our national security of importing oiland recommended
using that price as a cost yardstick against which to measure
proposed energy programs. The premium price would represent the
value that society places on reducing oil imports; used as a policy
tool, it would ensure that any government program aimed at reduc-
ing imports would have to show a lower cost-per-barrel than the
world price of oil plus the premium.
Joel Darmstadter, from Resources for the Future in Washington
D.C., expressed concern that DOE has become preoccupied with
quantified energy targets for the various energy technologies, often
set in isolation from other technologies. Programmatic goal-setting,
he argued, has to be based on cross technology comparisons and
cost-benefit analyses, or else the targets remain arbitrary. In addi-
tion, he argued that the target-setting process needs to allow for
adjustments in targets over time to n't changing economic and
technological conditions.
Other witnesses stressed that cross technology comparisons
should involve more than quantitative cost-benefit analyses of com-
peting energy technologies. Janice Hamrin, from the California En-
ergy Commission, spoke of the need for comparisons and allocation
decisions to include qualitative considerations.
"If energy is not an end in itself but rather a means to an
end, that end being the human quality of life, then the
allocation of resources for the DOE should be based on
some cntena that include a comparison of the cost-effec-
tiveness of meeting specific end-use needs." (Janice Ham-
rin, California Energy Commission.)
Evaluation
The second Hearing session focused on the role of program evalua-
tion in conservation and solar programs. The witnesses agreed on
the need for an ongoing evaluation effort within DOE's conservation
and solar programs With increasing fiscal conservation in Congress
and the likelihood of slower growth for the conservation and solar
budgets, one witness stated, there will be closer scrutiny of the
various programs for their cost-effectiveness and results. Evaluation
can play an important role in budget and other resource allocation
decisions. Evaluations which examine program processes as well as
results were seen as valuable tools for program managers in directing
their programs most effectively. Another witness cited the role of
program evaluation in demonstrating the credibility of solar and
conservation as alternatives to nonrenewable supply technologies.
In general, conservation and solar programs have received little
or no evaluative attention Eric Hirst, from Oak Ridge National
Laboratory, commented that this lack of effort has stemmed to a large
degree from the relative newness of the programs, along with the
crisis atmosphere that pervades most offices at DOE.
Much of the witnesses' testimony focused on barriers that have
limited the role evaluation plays in DOE's conservation and solar
programs. Four of the barriers given most attention in the testimony
are briefly described here:
54
-------
Lack of Consensus on Program Purposes. Witnesses agreed
that a prerequisite for successful evaluation is the clear identification
of program purposes and goals by which program effectiveness can
be judged. In many conservation and solar programs, though, these
purposes have been ambiguous, obscure, or controversial.
"During the 'apple pie' phase of the past several years,
conservation and-solar programs enjoyed a very broad
political constituency. But a broad political base is neces-
sarily comprised of many groups with unrelated or incom-
patible political objectives. The very existence of such a
variegated coalition makes 'explicit definition of program
purposes difficult and politically hazardous." (Louis Perel-
man, Jet Propulsion Laboratory.)
As a means to overcome this barrier, Joseph Nay, from the Perform-
ance Development Institute in Washington, D.C., described a proc-
ess for developing consensus among various stakeholders in a pro-
gram.. During the initial phases of program evaluation, he explained,
evaluators attempt to identify the range of program expectations
held by the stakeholders. At the same time, the evaluators collect
data on the program's performance and effects, developing a picture
of what the program is actually doing. Working with the program
staff, Nay said, the evaluators compare expectations against each
other and against the actual program results in a continual process
that allows program managers to better understand the program,
and how to adjust expectations or make changes to the program so
that expectations and results become closer.
"There is a fascination in seeing the rhetorics of expecta-
tion compared more and more closely with actual proc-
esses and outcomes and then related through common
frameworks to other programs. This generates interest
from most stakeholders involved and often leads to conver-
gent opinions rather than divergent arguments." (Joe Nay,
Performance Development Institute.)
Acceptance by Program Staff. Several witnesses addressed the
problem of obtaining acceptance and participation in evaluation by
program managers and staff who often see evaluation as a threat to
the program and its budget. Eva Baker, from the Center for the Study
of Evaluation of UCLA, emphasized the importance of educating
program staff in the techniques and uses of evaluation, pointing out
that many, if not most, DOE staff have not been involved previously
with programs where evaluation was a standard practice. She also
suggested that the evaluations be structured to assist managers in
making program improvements, rather than being aimed primarily
at higher level decisions on allocating resources between programs,
so as to minimize the threat that evaluation results will be used
against the program, while maximizing the usefulness of the evalua-
tion results to the program manager.
Keith Kozloff, from the Minnesota Energy Agency, emphasized
the need for involving the program staff in every phase of an evalua-
tion so that evaluation becomes an integral element of the program
instead of a task imposed on the program from outside. The process
of matching stakeholder expectations with actual results, described
by Nay earlier, is also intended to promote staff acceptance and
participation in evaluation.
Lack of Commitment from Top Management. All witnesses
identified the lack of top DOE management commitment to evalua-
tion as a prime barrier to its use in conservation and solar programs.
Given overcommitted program staffs and high turnover rates among
program managers, there is little incentive for managers to perform
evaluations. Without direction and commitment of sufficient re-
sources to address these problems from upper management, the
witnesses agreed, program evaluation will continue to receive a low
priority in most conservation and solar offices.
Evaluability Problems. One barrier to evaluation that all witnes-
ses spoke of was that of methodological problems. Many activities in
conservation and solar such as R&D programs pose substantial
problems to evaluators trying to assess program results. Several
witnesses observed that measuring the impact of government
spending in conservation is particularly difficult, due to the simulta-
neous influence of rising energy prices and other economic factors
on energy consumption decisions. The witnesses pointed out that
evaluation of conservation programs is hampered by our current poor
understanding of what motivates people to conserve energy. They
emphasized the need for developing better disaggregated data on
energy consumption trends as well as research into the actual sav-
ings to be expected from various conservation measures.
In addition to discussing barriers to evaluation, witnesses ad-
dressed criteria which should guide selection of programs to be
evaluated. Most witnesses felt that not all programs in conservation
and solar should receive formal evaluation. Several criteria were
suggested for choosing programs to be evaluated, including pro-
gram size, controversy, willingness of staff to participate construc-
tively, and feasibility of measuring program results. One witness felt
that managerial flexibility should also be considered: where mana-
gers are constrained by restrictive program legislation, evaluation
findings may not be translated into improved program structures and
processes.
Witnesses involved in state and local conservation programs
voiced concerns for evaluation efforts being carried out at those
levels. There was agreement that DOE has an important role to fill in
assisting states, localities, and utilities in evaluating their conserva-
tion and solar activities. Barbara Barkovich, from the California
Public Utility Commission, spoke of the need for better evaluation
techniques that the utilities can apply to their conservation pro-
grams. Keith Kozloff felt that DOE should be helping states to build
evaluative capabilities by developing model evaluation techniques,
performing basic research into conservation motivation, and provid-
ing funds for collecting state level energy consumption data.
While calling for stronger evaluation efforts within conservation
and solar, several witnesses cautioned that evaluation should not be
"oversold." Given the inherent difficulties of attaining clear results
when assessing many conservation and solar programs, it was felt
that moderate expectations should accompany program evaluation.
One witness pointed out the limits of evaluation:
"For the most part, evaluation will not always deliver clear
and unequivocal data; sometimes reality does not come so
packaged. Evaluation will not, by itself, save or kill pro-
grams." (Eva Baker, Center for the Study of Evaluation,
UCLA.)
State and Local Programs
In the third Hearing session, witnesses from state and local govern-
ments, utilities, and public interest groups testified on issues and
55
-------
problems related to DOE's state and local assistance programs in
conservation and solar.
State and Local Roles. Several witnesses discussed the roles of
state and local governments in energy conservation. The major
theme that emerged was the need for building capacity of local
governments to implement conservation programs.
Ann Cline, an energy planner for the city of Richmond, Indiana,
explained that state level outreach programs such as the State
Energy Conservation Program (SECP) and the Energy Extension
Service (EES) succeed in proportion to the ability of local com-
munities to use that help. Local governments, she said, have advan-
tages of formal and informal local connections that make them better
suited than state governments to involving people in conservation
programs. However, most local governments lack cohesive, pur-
poseful organization with which to approach energy problems. Cline
argued that the problems states have had with DOE in these pro-
grams would be less consequential if stronger partnerships existed
between states and localities. She believed that the Energy Man-
agement Partnership Act could do more to further this partnership
and assure success of state programs than any other foreseeable
initiatives of the next few years
Alec Wisch, head of the energy division of the Allegheny
County (Pennsylvania) Department of Planning, criticized the lack of
knowledge he perceived among DOE staff concerning state and local
government processes. He felt that DOE should be reviewing other
Federal programs to learn from their successes at building local
capacity and encouraging efficient local programs. Wisch argued
that DOE should deal directly with local governments as much as
possible, rather than sending locally-targeted grant monies through
the states. He felt that state involvement in grant distribution adds
only further expense and an additional layer of bureaucracy to many
programs. Acknowledging the impracticality of DOE's directly deal-
ing with all local governments, he suggested that at least the larger
energy-consuming counties and cities be supported directly point-
ing out that Allegheny County, a heavy industrial area, uses more
energy than 23 states.
Lee Callaway from Pacific Gas and Electric, discussed the
importance of the Federal role in facilitating information sharing
among localities, states, and other local conservation providers such
as utilities. He saw much potential for reducing redundant efforts
through information sharing, as well as providing knowledge and
techniques to organizations that lack that ability or motivation to
devise their own approaches to conservation programs.
Related to the issue of local capacity building was that of public
participation in conservation and solar programs. Barbara Brown,
from the Florida Solar Coalition, testified on the importance of public
understanding and cooperation to the success of conservation and
solar programs, particularly since these technologies depend so
much on consumer acceptance She contended that DOE and state
governments do not adequately acknowledge the role that public
interest groups play as vehicles for public concerns in conservation
and solar, nor do they adequately involve these interest groups in
program planning and implementation.
Residential Conservation Service (RCS). The RCS is intended
to reach 60 million American households with energy audits and
other assistance designed to encourage the purchase and use of
conservation measures, including renewable energy sources
Utilities -s-l} r!sv fhe key role in implementing RCS. States have the
responsibility of developing plans by which the utilities will design
and manage their programs, and DOE oversees and approves the
state plans, as well as providing management assistance to the
states and utilities. Under the present time table, most utilities will
begin their services in spring 1981.
Robin Calhoun, of HDR Associates, devoted her testimony to
the RCS program, detailing the weaknesses she has observed in the
legislation and the regulations being developed to guide the pro-
gram. Her comments reflected the opinions of many workshop par-
tfcipants and covered three broad issues that she considered particu-
larly important to the success of the RCS: program funding, policy,
and implementation.
Regarding program funding, Calhoun saw the current $5 million
appropriated for 1981 as reflecting a lack of commitment on the part of
Congress to conservation. Observing the lack of Federal assistance
to states and utilities for the RCS, she stated that the program will fail
to meet its goals unless substantially greater sources of Federal funds
are made available. Conservation, she stated,
". .is not simply a word, a belief, or a commitment. It is a
reality a complex, dispersed domestic energy resource
which is available at a price. That price is not $5 million for
a program such as the RCS." (Robin Calhoun, HDRAssoci-
ates.)
Calhoun also criticized Congress for having loaded the legisla-
tion authorizing RCS with excessively detailed program require-
ments. She felt that the states and utilities are in the best position to
decide on program methods and strategies, and that Congress
should limit its role to setting overall policy and program goals, with
responsibility for the attainment of those goals entrusted to DOE. For
example, the required listing service, she contended, is unnecessary
and a potential disincentive for contractors who would be providing
services to homeowners. Instead the program should be focusing on
incentives for participation, such as conservation measures during
the course of an audit, which provide an immediate benefit to the
customer. Calhoun recommended that Congress hold oversight hear-
ings immediately to examine those provisions which are overly
restrictive, and act as soon as possible to simplify the requirements.
Weatherization Assistance Program (WAP). DOE's Weath-
erization Assistance Program currently exists as a $200 million per
year grant program that provides subsidies for the weathenzation of
low-income households. DOE makes grants to the states, which
then distribute the funds to local governments and non-profit organi-
zations. Three witnesses at the National Hearings commented on
WAP' Timothy Wilson, director of the Maine Division of Community
Services; Lee Callaway, director of conservation programs at Pacific
Gas and Electric, in San Francisco; and Samuel Sperry from the
Seattle Energy Office.
One issue addressed by Sperry and Callaway was the need for
coordination between WAP and other residential conservation pro-
grams, such as the Residential Conservation Service at the national
level, and local initiatives such as Seattle's Weatherization Plan. The
witnesses emphasized the importance of program coordination in
order to minimize duplication of services and redundant efforts. Both
witnesses urged DOE to encourage this type of local cooperation
primarily by building more flexibility into the WAP (and RCS) regu-
lations, so that local and state governments can structure the pro-
gram delivery mechanisms in locally effective ways.
56
-------
As another means of improving WAP, Wilson recommended that
DOE make more use of its authority to reallocate funds from gtates
with lagging weatherization efforts to states needing additional
funds. Reallocation, he stated, can be-an important tool in motivating
low-performance states to improve their programs, while at the same
time increasing the effectiveness of WAP funds by rewarding the
states that can best use them.
Both Wilson and Callaway called for a balance between WAP
and the Low Income Energy Assistance Program (LIEAP), a $1.8
billion grant program which subsidizes low-income fuel bills. Both
witnesses recognized the need for LIEAP assistance, yet argued that
investing those same funds in weatherization would be a better
investment in energy security.
Institutional Buildings Grants Program (IBGP). Authorized by
Congress at $900 million over three years, the IBGP provides match-
ing grants to schools, hospitals, and units of local government for
energy audits and conservation retrofits. Local institutions apply for
the grants, which are made available on a 50 percent matching basis
by DOE through their state's energy office.
Two witnesses commented specifically on the IBGP: Robert
Pauls, energy coordinator for Carbondale, Illinois, and John McMil-
lan, energy coordinator for Hennipin County, Minnesota. Both wit-
nesses viewed the program as useful and worth continuing with
certain modifications. McMillan applauded IBGP for having pro-
vided the basic ingredient for a systematic evaluation of institutional
buildings, as well' as a method for prioritizing conservation im-
provements. He saw IBGP as an incentive particularly for small
governments and institutions which would probably not be conduct-
ing energy audits and keeping energy consumption records without
Federal funds.
According to McMillan, one of the program's major weaknesses
is the lack of assured funding for future years. The IBGP is currently
funded on an annual basis, which is causing problems for local
governments, since they need to budget their matching funds well
in advance of the fiscal year during which they will be spent. He
recommended that Congress authorize funds for IBGP on a multi-
year basis.
Both witnesses were concerned by what they saw as the un-
necessarily complex and expensive procedures for applying for IBGP
grants, particularly for small units of local government. McMillan felt
that the data requirements of grant application were inappropriate to
the size of most IBGP grants, many of which are under $5,000. Pauls
cited examples of local governments which had declined to apply for
a grant based on their assessment of the expense and time that
would be involved in the application process. As a remedy for this
potential disincentive to participation, it was recommended that
grant application and reporting procedures be simplified for small
grant applications.
Research, Development,
and Applications
In the afternoon session of the September 25th Hearings, seven
witnesses testified on a variety of topics related to how DOE carries
out its research and development activities in solar and conserva-
tion. These topics, which included information dissemination,
DOE's role in innovation, and program balance, had been previously
discussed at the Section 11 Workshop held in Denver on July 24 and
25, which several of the Hearing witnesses attended. Since much of
the testimony concerning the program balance topic is closely re-
lated and complementary to the issues addressed by the Policy
Session witnesses, program balance testimony is summarized in the
earlier section.
Information Dissemination. Joan Habib, from the Franklin Re-
search Center, which manages the National Solar Heating and Cool-
ing Information Center (NSHCIC), devoted her testimony to the
recommendations on information dissemination developed by the
work group she had participated in at the Section 11 Workshop in
Denver, Colorado (July 24 and 25). The recommendation dealt
primarily with ways to coordinate the many programs and organi-
zations that disseminate conservation and solar information. The
work group had agreed that the variety of existing information
programs creates confusion among information seekers, especially
since distinctions in function between the programs are unclear or
nonexistent.
Habib felt it critical to streamline and coordinate responsibilities
of these programs to provide a more efficient, comprehensible infor-
mation network. She also emphasized the need for combining solar
and conservation information in all programs, pointing out that the
technologies and principles involved in both areas are inextricably
linked, and therefore logically dealt with as a whole. As an approach
to coordinating and integrating the information programs, Habib
outlined the work group's recommendation for dividing information
functions among the major programs and organizations. NSHCIC,
they recommended, should become the primary entry point in the
solar and conservation network for inquiries from the general public,
providing coordination and referral services as well as a broad range
of general information. The Solar Energy Information Data Bank,
managed by SERI, should serve as the main source of technical
information on both conservation and solar. The Regional Solar
Energy Centers, another major actor in solar information dissemina-
tion, should focus their services on information appropriate to their
regions, particularly information relating to technology marketing.
Habib summed up the work group's consensus:
"I would suggest that if our goal is to promote solar energy
and conservation in a cost effective manner, the existing
organizations, streamlined, coordinated, and with en-
hanced conservation input, are the best framework on
which to build." (Joan Habib, Franklin Research Center.)
Marvin Yarosh, from the Florida Solar Energy Center, addressed the
issue of information adequacy m current government efforts to
promote solar energy. The present policy of encouraging the use of
solar, he said, is proceeding without valid information on the quality
or performance of commercial solar technologies a situation he
feared could damage the government's credibility among consum-
ers. Yarosh questioned the wisdom of aggressively promoting a
technology such as solai without knowing how commercially in-
stalled systems are performing.
"If the government is actively engaged, as it is, in en-
couraging solar adoption, then it incurs some degree of
responsibility towards the likely impact on those consum-
ers that follow government advice and commit to solar.
Fragmentary information which is available suggests that
conventional wisdom concerning solar system perform-
ance may be seriously in error." (Marvin Yarosh, Florida
Solar Energy Center.)
57
-------
Yarosh said he believed that most problems with commercially
installed solar systems are correctable with the various tools gov-
ernment has availableperformance standards, licensing programs
for manufacturers and installers, etc. but that first information is
needed to determine where the problems are. He urged expanded
efforts at monitoring solar systems, emphasizing the importance of
monitoring private, commercially installed systems rather than the
government demonstration systems which have been the primary
object of monitoring activities to date. Most demonstration systems,
he observed, are not characteristic of current equipment and prac-
tices in the solar energy industry.
Innovation. Jerry Plunkett, head of the Montana Energy Research
and Development Institute, devoted the majority of his testimony to
the need for innovation in the energy field and to DOE's role in
supporting innovation. Other witnesses also commented on innova-
tion issues. Plunkett contended that DOE, for a variety of reasons,
has not been committed to pursuing radical innovation in solar and
conservation. He argued that the need for new approaches to energy
supply and use are particularly acute now, but instead, DOE has
tended to fund low-risk, low-innovation technologies. This is due, he
said, to factors such as conservative attitudes on the part of program
managers, and detailed, inflexible program plans which leave little
room for receptive responses to unsolicited ideas.
Plunkett was particularly critical of DOE's involvement with
individual inventors and small businesses. Citing the 1972 Sharply
Report, which showed that at least 50 percent of innovations having
a major impact on the economy came from individuals or small
businesses, Plunkett pointed out that the most of DOE's R&D money
goes to large firms. He expressed concern that small businesses and
individuals are not adequately involved in DOE's R&D, and stated
that their participation should be actively solicited if innovation is a
goal of DOE.
One of the major barriers to small business involvement in DOE,
Plunkett continued, is the procurement system, which discourages
small business participation in several ways, including slow de-
cisionmaking on contract awards and delays in contract payments.
As an alternative, Plunkett recommended that DOE fund a number of
private innovation centers, which would promote energy-related
innovation by assisting small businesses and rnvento'rs to bring their
inventions to the market. As an example of how these innovatio^
centers could work, he pointed to the Center for Innovation locateu
in Butte, Montana, an independent organization funded by five
western states to support local businesses in a wide variety of
activities aimed at commercializing new inventions
Workshops and
Meeting Summaries*
Section 11 of the Federal Nonnuclear Energy Research and Develop-
ment Act of 1974 requires that EPA hold public hearings as part of its
annual review of DOE's research and development activities.
Recognizing the importance of meaningful public participation in
the Section 11 process, EPA chose to hold a series of five public
meetings in addition to the National Hearings for this year's review.
These meetings, which were attended by approximately 150 people
representing state, local and Federal government, research organi-
zations, public interest groups, industry, universities and other orga-
nizations, have played an important role in the Section 11 review.
Participants in each meeting were asked to discuss specific
aspects of DOE's conservation and solar programs and decision
making processes, identifying their strengths and weaknesses and
then developing recommendations for their improvement. The re-
sults of these meetings have served as the basis for EPA's conclu-
sions and recommendations to the President and Congress in this
year's Section 11 Final Report.
This Appendix contains summaries of each of the five meet-
ings. The Durham meeting examined how DOE sets policy and goals
for the conservation and solar programs. The Menlo Park meeting
explored the uses and need for program evaluation in DOE's solar and
conservation efforts. Two of the workshops, in Minneapolis and
Portland, focused on DOE's conservation and solar programs which
are administered by state and local governments. Finally, in the
Denver workshop, participants looked at a set of issues related to
DOE's conduct of solar/conservation research and development.
Solar and
Conservation Policy
The first 1980 Section 11 meeting, held in Durham, North Carolina on
June 29 and 30, was organized to discuss the Department of Energy's
policy for solar and conservation, and the means by which that policy
is translated into programs and activity. Major issues explored at the
meeting included the lack of cross technology comparisons in DOE's
resource allocation process, the role of government in supporting
solar and conservation, and the relation of conservation and solar
policy to national energy policy. It also examined several analytical
tools which could aid in policy analysis. Twenty-five participants
attended the meeting, representing the Department of Energy, uni-
versities, research organizations, public interest groups, utilities, the
Environmental Protection Agency, Congressional committee staff,
and industry.
Least-Cost Energy Strategy. The first session began with a brief
talk by Dennis Bakke, of the Mellon Institute's Center for Energy
Productivity, on the Least Cost Energy Strategy In the group discus-
sion that followed Bakke's talk, participants considered the least cost
approach and its potential application to DOE's resource allocation
process. The least-cost approach, as described by Bakke, examines
energy needs in terms of the services provided to people at the
end-use point: heating and cooling for buildings, illumination, au-
tomotive motion, etc The least-cost approach involves analyzing the
various alternative sources of energy that could provide a given
end-use service to determine the least expensive source for that
service, taking external costs such as environmental degradation
into account. In the work done by the Mellon Institute, conservation
is a key element of the least-cost approach, as it represents some of
the cheapest approaches to meeting end-use needs that are avail-
able.
'More complete summaries of these meetings are included in the Background Docu-
ment for the National Hearing, published by EPA in September 1980
58
-------
The participants generally approved of the least-cost strategy,
though several expressed concern that it does not necessarily pro-
vide DOE with the criteria needed to choose between various energy
programs. There was also criticism that the least-cost strategies
developed by Mellon do not adequately incorporate all the important
external costs of energy technologies.
Role of Government/Oil Import Premium. The second session
was' opened with a talk with Alvin Aim from the Kennedy School of
Government at Harvard University, who spoke on the role of the1
Federal government in energy and need for a rational framework for
selecting government programs based on cross technology compari-
sons. He emphasized that Government policy should serve to com-
plement the energy market, not to direct it. Until a freely-operating
energy market can be achieved, he acknowledged that the Govern-
ment had additional roles in removing impediments to the free
market, and mitigating some of its negative effects.
Considerable discussion was generated by Aim's suggestion of
calculating a "premium" price on oil a price somewhere above
market price that would incorporate the external costs to our na-
tional security associated with importing oiland using that as a
cost yardstick against which to measure proposed energy programs.
Cross technology comparisons using the oil premium measure, it
was suggested, could permit conservation and solar policy to be
directed without relying on specfic quantitative goals for conserva-
tion or solar. It was also expected that cross technology comparisons
would demonstrate the economic attractiveness of conservation and
solar, and result in increased emphasis on both in DOE's budget.
Application of Analytical Tools. The third session of the meeting
was devoted to small work group discussions on the application of
conservation and solar policy to each of four energy end-use sectors:
residential, commercial, industrial, and transportation. In each work
group, participants attempted to apply the least-cost approach to
choosing programs in the given end-use sector. The problems of
applying the cost-benefit methodologies to various government ac-
tivitiesresearch and development, regulation, commercialization,
and so on were examined as part of this task. Findings and
conclusions of the work groups were reported to the whole meeting
for discussion and comment during the fourth session.
Conservation and
Solar Program Evaluation
Defining Evaluation. In the first session of the meeting the group
focused on clarifying and reaching consensus on a working defini-
tion that differentiates evaluation from other forms of data collection
and analysis. The first conclusion reached by the group was that
evaluation is a retrospective activity and should be distinguished
from projections and forecasts.
Participants agreed that evaluation should not be narrowly
defined to include only program impacts. Instead, it can apply to
studies of results and processes. In fact, participants felt that for
most purposes, quantitative measurement of impact alone is in-
adequatethat a useful study must also provide insight into why
certain outcomes occurred.
The Role of Evaluation. Following this initial discussion, partici-
pants broke into small groups to consider the past, present, and
potential future role of evaluation of DOE. There was general agree-
ment that DOE does not currently use evaluation to support these
processes and that the role of evaluation should be expanded. Partic-
ipants stressed, however, that all evaluations must be done carefully
that DOE must be sensitive to the limits of evaluation, its cost, and
the potential for its misuse.
Scope of Evaluation. In the second session, participants met in
small groups to discuss (1) what types of programs should be evalu-
ated, and (2) how programs and projects should be selected. Some
participants felt that all programs should receive some type of evalu-
ation and that DOE should obtain information on the results of all of
its investments. Other participants argued that it was impractical to
attempt evaluation of all programs, and that in some instances
evaluation was a waste of time and money.
This session demonstrated clearly the problems involved in
setting criteria for selecting which programs to evaluate. Groups
suggested numerous separate elements that could be employed, but
there was fairly wide agreement on a few factors that should be used
in setting priorities. These included: existence of a requirement for
evaluation, such as a Congressional mandate; feasibility and likeli-
hood of obtaining information; size or importance of program; and
relevance to decisions.
Selected Implementation Issues. In the third session, partici-
pants met in small groups to discuss specific methodological or
operational issues that had been identified during previous sessions.
Topics for these groups included:
EPA's panel on conservation and solar program evaluation met on
July 8 and 9,1980, at the Vallombrosa Center in Menlo Park, Califor-
nia. This meeting was designed to assess the adequacy of DOE's
current evaluation activities and to develop constructive recom-
mendations for improvement. Recognition of the need for more
evaluation appears to be developing in DOE, and this meeting was
designed to complement DOE's efforts to enhance its evaluation
capabilities.
Participants were selected to represent several areas of exper-
tise. Most of the 38 participants had experience in designing and
implementing evaluations of conservation of solar programs for the
Federal government in energy and need for a rational framework
tion, several evaluation methodologists were invited who were famil-
iar with the development and current state-of-the-art of evaluation in
other fields, such as education and human services.
Determining causality and the use of control groups: The group
felt that a key factor in establishing causality is the degree to
which an evaluation is able to include measures of intermediate
outcomes as well as measures of ultimate outcomes, such as
energy savings.
Useful and effective indicators: The group stressed that it was
important to distinguish "consumption" from "efficiency," the
latter requiring a link between energy inputs and products or
service outputs. The group also noted that indicators have limited
usefulness unless they reflect the value of energy conserved in
terms of location, time, and quality of energy.
Evaluation of R&D programs: The group believed that it is impor-
tant to learn as much as possible from all programs, but they
agreed that basic R&D presents particularly thorny problems for
e valuators.
59
-------
Responsibility for evaluation within DOE: The group felt that roles
should be defined as follows: program managers should conduct
evaluations; the Office of Policy, Planning, and Evaluation with
Conservation and Solar should have an evaluation unit, with a
clearly defined purpose; and the Assistant Secretary for Policy and
Evaluation should have staff committed to evaluation, and should
primarily be responsible for technical assistance in evaluation.
Selling Evaluation. In the final session, participants stressed the
need to "sell" evaluation within DOE since it now lacks broad
support. Obvious barriers exist to building support for evaluation at
DOE: budget constraints, insufficient staff and staff turnover, per-
ceptions of evaluation as a threat, and time spent "firefighting." Tb
overcome these barriers, participants believed that several steps are
crucial. First, departmentwide policy on evaluation must be estab-
lished. Second, responsibility for evaluation must be clearly as-
signed, and people must be personally accountable for planning and
conducting evaluative studies. Third, methodologies should be de-
veloped to promote the credibility of evaluative study results. Finally
an "evaluation ethic" must be fostered at all levels, which views
evaluation as a learning process, not a punitive one.
State and Local
Assistance Programs (I)
The first Section 11 Workshop on state and local programs was held in
Minneapolis, Minnesota, on July 17 and 18,1980, at the Bloomington
Marriott Hotel. The workshop focused on state and local energy
programs funded and managed by the Department of Energy's Office
of Conservation and Solar Energy. Specifically the workshop exam-
ined five predominately conservation-oriented programs: the
Weatherization Assistance Program (WAP), the Institutional Build-
ings Grants Program (IBGP), the Residential Conservation Service
(RCS), the State Energy Conservation Program (SECP), and the
Energy Extension Service (EES). In addition, attention was given to
the proposed Energy Management Partnership Act (EMPA), which
would combine the EES and SECP
Along with two observers from the Department of Energy, the
group was composed of 29 representatives of city county and state
governments, public interest groups, electric utilities, community
action agencies, and private consulting firms. The participants
spent most of the time in four small work groups. Each group
considered one specific program, with the exception of the group
which examined together the SECP, EES, and the proposed EMPA
legislation. While in the small groups, the participants focused on
identifying strengths and weaknesses of the existing programs and
developing suggestions for their improvement.
Residential Conservation Service (RCS). The Residential Con-
servation Service group viewed the RCS program as a worthwhile,
well-intended effort that has a number of potentially serious flaws
Most of the discussion session was spent identifying these flaws and
developing recommendations for their remedy.
One of the serious program flaws identified by the discussion
group was the lack of specific program goals and evaluative mecha-
nisms which would motivate the states and utilities to run effective
programs. They felt that RCS program goals, rather than specifying
only the percentage of homes to be offered the audit service, should
specify perhaps a goal of actual energy savings (based on utility
energy sales) to be achieved through the program.
A major criticism voiced during the group discussion con-
cerned state program funding. The participants were unanimous in
believing that giving the states responsibility for developing RCS
plans without providing Federal assistance for developing and ad-
ministering the plans was a major flaw in the program structure. The
strong recommendation was that Congress authorize funds for state
RCS planning and administration. Along the same line, the group
recommended that each state be required to perform periodic evalu-
ations of their RCS programs, in order to judge the success of audit
services in motivating energy conservation.
Another group concern related to the possible exclusion of
low-cost/no-cost measures from the audits. The group recom-
mended that DOE repeal the prohibition on recommending unspeci-
fied measures, and perhaps require discussion of low-cost/no-cost
measures as part of the audit.
Institutional Buildings Grants Program (IBGP). The consen-
sus of the Institutional Buildings Grants Program group was that the
IBGP, though plagued with problems during its first year of opera-
tion, has provided some real benefits and should be continued with
various improvements.
One of the primary criticisms of the IBGP brought out in the
discussion concerned program pacing. All the participants consid-
ered the program's first two grant cycles to have been rushed, with
little or no advance notice of upcoming requirements and deadlines.
The group recommended that future grant cycles cover a year, to
give the states adequate time to prepare their plans, to notify and
assist the local institutions, and to allow participating institutions
time to coordinate the Federal assistance with their own capital
improvement programs.
Another concern was the uncertain future of the program.
Without the continuity of a new multi-year funding authorization,
some participants felt that local participation will suffer. They felt
that without reauthorization for at least another two or three years,
program success and effectiveness would be limited, at best.
The participants representing local governments expressed
their frustration that the technical assistance they needed for under-
standing and particpating in the program has been inadequate in
the past, from both the Federal and state levels, and often too late
when available to be useful. The group was also concerned about the
complexity of applying for and reporting on small IBGP grants. It was
recommended that DOE provide simpler, less detailed grant applica-
tion forms and reduced reporting requirements for small grant appli-
cants.
Weatherization Assistance Program (WAP). DOE program
management was a main theme of the Weatherization Assistance
Program discussion There was consensus among the work group
members that management at all levels of the program Federal,
state and local program operators has been generally weak.
Another recurring theme of the discussion was the importance of
program flexibility in effective program planning and operation at the
state and local levels Participants pointed out repeatedly that the
wide variations in local Weatherization needs, conditions, and re-
sources make it essential to allow states and local operators wide
latitude in structuring their programs.
60
-------
DOE was criticized for having concentrated too much de-
cision making authority in its Headquarters office. All participants
felt that the role of DOE's regional offices in the Weatherization
Program should be expanded to allow for more frequent, .closer
contact between the states and DOE. To spur states whose weath-
erization programs have lagged, the group recommended that DOE
make greater use of its authority to reallocate weatherization funds
from states not using their full grants to states whose programs could
use more money.
A prime concern of the work group was program funding. The
participants agreed that WAP should proceed as rapidly as feasible
toward the goal of reaching all eligible low-income households.
Group consensus was that there are enough areas in the country now
capable of greatly expanding their programs to warrant substantial
budget increases for WAR
State Energy Conservation Program (SECP)/Energy Exten-
sion Service (EES). One of the discussion's general themes in-
volved the group perception that DOE lacks understanding of the
ways in which state and local governments operate. Participants felt
that without sufficient attention to state and local processes, DOE is
bound to continue imposing programs on state and local govern-
ments at those levels that are difficult to implement and often do not
promote energy conservation.
The group agreed that communication channels between state,
local and Federal levels need to be improved, both horizontally and
vertically. Localities and states need better mechanisms to exchange
conservation program information among themselves, and DOE
needs the capacity to learn from state and local experiences in
developing its conservation programs. A specific recommendation
was made that the President's Clearinghouse on Community Energy
Efficiency be continued and expanded to meet the needs of a wider
range of state and local governments and citizens' groups.
Participants expressed concern over the multiplication of new
government units and programs with the implementation of new
conservation initiatives, and recommended that EMPA contain
provisions to encourage the use of existing governmental mecha-
nisms in providing conservation support to the states and localities.
The group argued that state energy offices be given a larger role in
reviewing and coordinating local conservation programs, largely
from a concern that DOE's regional offices are not equipped to
adequately perform these functions.
Regarding program funding, there was consensus that Con-
gress should set funding levels appropriate to meeting the program
goals; that is, more ambitious programs such as SECP and RCS
should receive considerably more funds than they are now allocated.
Research and Development
and Application
The fourth Section 11 meeting was held in Denver, Colorado on July
24 and 25, at the Denver Marina Hotel. The workshop examined a
variety of issues related to DOE's conduct of research, development,
and application programs involving solar and conservation
technologies.
Like the Minneapolis Workshop, the Denver session was or-
ganized so that participants spent most of their time in small work
groups. Rather than focusing on specific DOE programs, however,
the work groups at the Denver Workshop each considered a different
aspect of DOE's research, development and application pro cess. One
work group examined conservation and solar program balance is-
sues; another explored innovation and how DOE's programs promote
or stifle it; and the third group examined DOE's information dissemi-
nation activities. The workshop was attended by 34 participants,
along with one observer from the Department of Energy. The group
was composed of representatives from state and local governments,
private research institutes, state and regional solar energy centers,
the Solar Energy Research Institute (SERI), universities, public inter-
est groups, and Congressional committees.
Program Balance. The program balance group examined the bal-
ance of Federal funds currently supporting solar and conservation
programs, both within and outside of DOE. They explored various
objectives and criteria which guide Federal investments in conser-
vation and solar, and developed recommendations to DOE for im-
proving these criteria.
Early in the discussion, several participants stressed the need
for DOE's funding of solar and conservation to reflect a commitment
to attaining the goals established for these technologies in national
energy policy. The work group strongly criticized DOE for not having
taken these goals seriously in providing for the solar and conserva-
tion programs. Participants saw a persistent bias in both DOE and
Congress against energy technologies with significant near-term
potential, including passive solar and building conservation
technologies, as opposed to longer-term supply technologies, many
of which receive heavier funding. The group was unanimous in
emphasizing the critical need for promoting near-term technologies,
particularly conservation.
The group considered how DOE could increase the effective-
ness of its current conservation and solar spending, and developed
the recommendation that more funds be targeted at states, localities
and private industries which have committed their own funds to
conservation or solar efforts. The participants saw cost sharing
projects and programs as having greater chances for success than
those funded by DOE alone.
The group agreed that cost-effectiveness must become the
primary criterion for DOE's program funding decisions. The partici-
pants urged that no DOE expenditures be made on energy supply
technologies without an explicit comparison to the cost-effective-
ness of conservation alternatives, stating that DOE's broad mission
should be to support the development of the least expensive forms of
energy, within the bounds of environmental and other necessary
considerations.
Innovation. The innovation group's consensus was that for a num-
ber of reasons DOE rarely has been successful in supporting innova-
tion in its solar and conservation programs. Several of the reasons
involve the difficulties small firms and individual inventors often
have participating in DOE programs. There was consensus that
effective efforts on DOE's part to support innovation in solar and
conservation must include the innovative small firms and inventors.
Other barriers to innovation identified by the group were: the
excessive management burdens placed on DOE program managers;
DOE's emphasis on long-term program planning, which leaves little
opportunity for funding higher-risk innovative technologies; DOE's
extensive use of the National Laboratories in research and develop-
ment work; and DOE's procurement process, which discourages
61
-------
and, m some cases, prevents small firms from participating in solar
and conservation work.
The group discussed DOE's Office of Small Scale Technology
(OSST), which contains the Appropriate Technology Grants Program
and the Energy-Related Inventions Program. There was agreement
among the group members that the two programs are worthwhile,
but underfunded and poorly integrated with DOE's other solar and
conservation programs. They recommended that the total OSST
funding be increased to at least 1% of the total DOE budget.
The group also explored possible alternatives to direct govern-
ment involvement in technology development. One of these alterna-
tives which received considerable discussion was Federal support of
private organizations, such as the Center for Innovation in Butte,
Montana, which promotes energy-related innovation by assisting
small businesses and inventors in bringing their inventions to the
market. Several members of the group expressed their opinion that
government-funded private innovation centers could take over and
expand the innovative function of DOE's OSST without the difficul-
ties they saw in large DOE-directed programs.
Information Dissemination. The main theme of the information
dissemination work group's discussion was the need for better coor-
dination of the solar and conservation information activities of the
numerous DOE programs and other Federal agencies. The group
agreed that the various solar and conservation information programs
need more sharply defined, distinct roles in order to reduce duplica-
tion of services and enable information users to better understand
the array of information services available. They made specific
recommendations of coordinated roles that the major organizations
NSHCIC, SEIDB and the RSEC's should serve. There was
consensus among the group members that all primary information
organizations need to offer both solar and conservation information,
since solar and conservation technologies are so closely related.
The group discussed the problems associated with maintaining
quality control over energy information sent out by DOE. There was
general consensus that the actual extent of inaccuracy and incon-
sistency is unknown, and that better mechanisms for assessing
informational quality need to be set up within each information
program. The work group spent much time discussing the critical
need for evaluation in building effective energy information pro-
grams. Participants emphasized the importance of well-designed
user surveys in information program evaluations.
Participants agreed that solar/conservation information pro-
grams need to adopt more aggressive advertising and marketing
attitudes, rather than merely providing information to requestors. It
was observed that appropriate solar and conservation technologies
vary considerably from region to region, so that information market-
ing efforts need to be targeted for specific areas and localities.
State and Local Assistance Programs (II)
The last Section 11 workshop was held in Portland, Oregon, on July
29 and 30, at the Portland Hilton Hotel The workshop, like the
Minneapolis workshop, focused on DOE's state and local conserva-
tion programs. In addition to four observers from the Department of
Energy, the group's 61 participants included representatives from
state, county and local government, community action agencies,
utilities, Congressional committees, public interest groups, private
industry, and other institutions involved in state and local conserva-
tion programs. The Portland workshop was organized in the same
way as the Minneapolis workshop, with most of the participants'
time spent in small work groups, each of which examined a different
DOE program. The work groups focused on identifying strengths
and weaknesses of DOE's programs and developing suggestions for
improving these programs.
Residential Conservation Service (RCS). The work group con-
sidered the RCS programs goals laudable, but found many problems
with the program's structure and funding which weaken its chances
for success. There was strong consensus among the participants
that the current $5 million funding level is inadequate to the task of
reaching 60 million homes. They criticized Congress for not taking
the program seriously enough to provide for funding in the RCS
authorizing legislation, and called for expanded funding to satisfy
state and utility needs, as well as those of DOE.
A second theme of the group discussion was the excessive level
of detail and direction in program legislation and regulations. Local
flexibility was regarded by all as an essential ingredient in program
success, and there was considerable concern that the program
regulations as they now exist will severely hamper the states and
utilities in fashioning effective RCS programs The proper role, all
agreed, is for Congress to set the overall policy goals and objectives of
the program, and for the states and utilities implementing the pro-
gram to develop the locally appropriate methods for achieving those
program goals
The work group spent considerable time exploring the problems
of motivating homeowners to take part in RCS, and identified key
improvements that are needed in the program to ensure its credibil-
ity among homeowners. There was consensus among the group that
the program should focus more on providing incentives for participa-
tion, and contain less emphasis and regulatory detail on the "hand-
holding" aspects of the arranging/financing/installing services to be
provided by utilities. A majority of the group participants felt that
state listing of financing institutions and contractors/suppliers will
not work as intended by Congress, and recommended that its re-
quirement be dropped from the legislation.
Institutional Buildings Grants Program (IBGP). The Institu-
tional Buildings Grants Program group generally agreed that the
IBGP has achieved a fair measure of success, though the program
was plagued by serious problems in its first two years.
Program continuity was an important theme of the discussion.
Given the problems experienced during the program's first two years
and the time spent resolving some of those problems, the partici-
pants felt strongly that the IBGP needs to continue for at least a few
more years. The group recommended that Congress reauthorize and
fund the IBGP for at least another three years.
Another concern raised in the discussion was the issue of state
flexibility m apportioning funds. The group considered the present
regulations too restrictive in several ways, causing funds to go
unspent. There was consensus among the group that the excessive
degree of detail in the IBGP authorizing legislation was the source of
much of the regulatory restrictiveness.
The participants expressed a general concern over the lack of
effective communication between DOE and the state energy offices
62
-------
involved in the IBGP. This lack of communication, all agreed, had
caused considerable delays in the identification and resolution of
problems within the program. They felt that it is essential to establish
some formal structure for periodic, ongoing talks between DOE
headquarters, its regional offices and the state offices.
Weatherization Assistance Program (WAP). The WAP work
group explored a variety of topics related to improving the program
and coordinating it with other conservation programs, particularly
the RCS, which also provide services to low-income homeowners.
The group participants generally considered WAP to have improved
significantly in the last year, but they identified some program areas
of concern that need attention by DOE and Congress.
The work group participants were unanimous in recommend-
ing expanded funding for WAP, citing the relatively slow progress
toward weatherizing the nation's low-income dwellings being made
with the current $200 million per year program. They pointed out the
large imbalance between the Home Energy Assistance Program and
WAP budgets, and asked that DOE provide for better coordination
and balance between the programs.
Another issue covered by the group was that of coordinating
WAP with the RCS and locally initiated weatherization programs.
There was strong consensus that the services of these programs
need to be carefully coordinated to maximize their benefit and avoid
redundant, wasted efforts. Rather than attempt to dictate local
cooperative arrangements for these programs through Federal regu-
lation, which several participants felt could not work, the group
recommended that DOE do two things: (1) ensure that its program
regulations for RCS and WAP do not constrain local coordinative
efforts, and (2) encourage local initiative in this area through non-
regulatory means such as providing technical assistance and infor-
mation exchange among communities.
State Energy Conservation Program (SECP)/Energy Exten-
sion Service (EES). Two groups were organized to discuss SECP
and EES, and they brought different perspectives to their discus-
sions, due to their composition: Group A contained a majority of
state level energy officials, while Group B contained more local level
officials.
Group A: The participants were of the general opinion that the
SECP and EES programs have been, on the whole, beneficial in
promoting state and local conservation efforts. The program
weaknesses they identified involved such issues as the potential
overlap of various programs, the lack of adequate program evalua-
tion, and the confusing flow of energy information from DOE.
There was consensus among the participants that Federal fund-
ing for state and local programs needs to be more appropriate to the
program goals: that is, Federally established programs with ambiti-
ous goals such as SECP need a commensurate level of Federal
'assistance to enable them to achieve the goals. The issue of how
states should allocate local funds among commmunities was dis-
cussed in the work group, with most of the participants agreeing
that a consistent formula balancing entitlement and competitive
funds needs to be developed.
The group discussed the need for better communication among
local energy offices, to enable the exchange of ideas and program
successes. They saw an important role for DOE in this process as a
facilitator of local information exchange, providing travel money and
organizing meetings for local energy officials.
Group B: The main theme that emerged from the second SECP/
EES work group's discussion was the importance of energy conser-
vation efforts organized at the community level. The participants
were convinced that conservation programs, whether originating at
the Federal, state, or local level, must involve and tie into local
institutions and governments in order to be effective. They felt that
both Congress and DOE give too little attention and support to the
role of localities in energy conservation, and expressed the hope that
this lack is beginning to be corrected through development of the
EMPA legislation.
Given the limited Federal conservation funds that currently are
designated for local use, the group recommended that DOE target
the funds toward building community level institutions which will
promote conservation. Regarding the EES, the group recommended
that the program should adopt a more aggressive approach to mar-
keting energy information. They considered the present program's
"passive" approach to have had little effect on energy conservation.
Another major recommendation that emerged from the group
discussion was that DOE establish a network of training centers for
local energy managers, structured to provide education in the vari-
ous skills needed to deal effectively with state and local Federal
government and run conservation programs at the local level.
63
-------
participants
EPA Section 11 Meeting on Conservation and
Solar Energy Policy Analysis
Duke University, Durham, North Carolina
June 29 and 30,1980
Alvin L. Aim
Harvard University,
Cambridge, Massachusetts
Dennis Bakke
Mellon Institute,
Arlington, Virginia
James Bishop
Burson-Marsteller,
Washington, D.C.
Colin Blaydon
Duke University,
Durham, North Carolina
Clark Bullard
University of Illinois,
Urbana, Illinois
Melvin H. Chiogioji
DOE, Conservation and Solar
Energy,
Washington, D.C.
Norman Dean
National Wildlife Federation,
Washington, D.C.
Gerald Decker
Kaiser Aluminum and Chemical
Corporation,
Oakland, California
James Drewry
Gas Research Institute,
Chicago, Illinois
Carl Gawell
Solar Energy Research Institute,
Golden, Colorado
Michael Gillette
DOE, Office of Policy and
Evaluation,
Washington, D.C.
Thomas Glennon
Rand Corporation,
Washington, D.C.
Paul Greiner
Edison Electric Institute,
Washington, D.C.
James Harding
Friends of the Earth,
San Francisco, California
Robert Hemphill
Tennessee Valley Authority,
Chattanooga, Tennessee
Robert Koger
North Carolina Public Utility
Commission,
Raleigh, North Carolina
Wesley Magat
Duke University,
Durham, North Carolina
John Milhone
DOE, Conservation and Solar
Energy,
Washington, D.C.
William Morrill
Mathematica,
Princeton, New Jersey
J. Michael Power
DOE, Conservation and Solar
Energy,
Washington, D.C.
James Quinn
DOE, Conservation and Solar
Energy,
Washington, D.C.
James Spensley
House Subcommittee on Energy
Development and Applications,
Washington, D.C.
James Stukel
University of Illinois
Urbana, Illinois
David Tunderman
EPA, Office of Planning
and Evaluation,
Washington, D.C.
64
-------
participants
EPA Section 11 Meeting on Conservation and
Solar Energy Program Evaluation
Menlo Park, California
July Sand 9,1980
Peter Back
DOE, Conservation and Solar
Energy
Washington, D.C.
Eva L. Baker
Center for the Study of Evaluation
Los Angeles, California
Barbara Barkovich
California Public Utilities
Commission,
San Francisco, California
Lynda Carlson
Energy Information Administration,
Washington, D.C.
Brad Chase
State Office of Policy and
Management,
Hartford, Connecticut
Lynn Collins
Alliance to Save Energy,
Washington, D.C.
Elizabeth Cooper
Regional Solar Energy Center,
Portland, Oregon
John Cooper
Pacific Gas and Electric,
San Francisco, California
Peter deLeon
Science Applications, Incorporated,
Englewood, Colorado
Warren Fujimoto
Pacific Gas and Electric,
San Francisco, California
Seymore Goldstone
California Energy Commission,
Sacramento, California
Bruce Hannon
University of Illinois,
Urbana, Illinois
James Harding
Friends of the Earth,
San Francisco, California
Jeffrey Harris
Lawrence Berkeley Laboratory,
Berkeley, California
David Harrje
Princeton University,
Princeton, New Jersey
Eric Hirst
Oak Ridge National Laboratory,
Oak Ridge, Tennessee
Bruce Hutton
University of Denver,
Denver, Colorado
Keith Kozloff
Minnesota Energy Agency,
St. Paul, Minnesota
Ron Larson
Solar Energy Research Institute,
Golden, Colorado
Mark Levine
Lawrence Berkeley Laboratory,
Berkeley, California
John Moore
Academy for Contemporary
Problems,
Columbus, Ohio
Joe Nay
Performance Development Institute,
Washington, D.C.
David Nichols
Senate Governmental Affairs
Committee,
Washington, D.C.
Lewis Perleman
Jet Propulsion Laboratory,
Pasadena, California
Robert Plunkett
DOE, Conservation and Solar
Energy,
Washington, D.C.
Michael Power
DOE, Conservation and Solar
Energy,
Washington, D.C.
Sam Stewart
Sam Stewart and Associates,
St. Paul, Minnesota
Robert St. Pierre
Abt Associates,
Cambridge, Massachusetts
Tom Swanson
Tennessee Valley Authority,
Chattanooga, Tennessee
Grant Thompson
Conservation Foundation,
Washington, D.C.
Catherine Waddell
California Public Utilities
Commission,
San Francisco, California
Keith J. Ward
Auburn University,
Auburn, Alabama
Robert E. Webber
Michigan Energy Administration,
Lansing, Michigan
Carl York
Lawrence Berkeley Laboratory,
Berkeley, California
-------
participants
EPA Section 11 Meeting on State and
Local Energy Conservation Programs
Minneapolis St. Paul, Minnesota
July 17 and 18,1980
Robert Anderson
Consultant,
Chanhassen, Minnesota
Kathleen Birmingham
Comprehensive Community Energy
Management Program,
Detroit, Michigan
W. Kim Boas
Department of Central Services,
Dayton, Ohio
Barbara Brown
Governor of Florida's
Energy Office
Tallahassee, Florida
Ann Cline
Comprehensive Community Energy
Management Program,
Richmond, Indiana
Kenneth Paul Dobbs
Barton, Ashman & Associates,
Evanston, Illinois
Marion S. Feeney
USDA, Community Energy Conservation,
Washington, D.C.
Phillip Gillespie
Schenectady Community Action Program,
Schenectady, New York
Gary Grufman
League of Minnesota Cities,
St. Paul, Minnesota
Jeff Hammarlund
DOE, Conservation and Solar Energy,
Washington, D.C.
James R. Hays
Energy Coordinator,
Toledo, Ohio
Lex Hester
Florida State Energy Office,
Tallahassee, Florida
Judy Karon
Ramsey County Energy Coordinator,
St. Paul, Minnesota
June Kenfield
New York State Energy Office,
Albany, New York
Fred Klein
City of Milwaukee,
Milwaukee, Wisconsin
Al Lessik
Minnesota Energy Agency,
St. Paul, Minnesota
Paul F. Levy
Arkansas Department of Energy,
Little Rock, Arkansas
John G. McMillan
Hennepin County Energy Coordinator
Minneapolis, Minnesota
Robert Pauls
Energy Coordinator,
Carbondale, Illinois
Margaret Post
Minnesota Energy Agency,
St. Paul, Minnesota
Bill Rolland
AHP Energy Associates,
Minneapolis, Minnesota
Pat Rose
DOE, Conservation and Solar Energy,
Washington, D.C.
William Schott
Base and Electric Power Cooperative,
Bismark, North Dakota
J. Randy Shick
Illinois Solar Resource Advisory Council,
Springfield, Illinois
Paul Richard Thomas
Ohio Cooperative Extension Service,
Columbus, Ohio
Timothy P. Wilson
Maine Division of Community Services,
Augusta, Maine
Alec Wisch
Energy Division,
Pittsburgh, Pennsylvania
Ron Wise
State of Missouri
Jefferson City, Missouri
66
-------
participants-
EPA Section 11 Meeting on DOE Research,
Development and Application
Denver, Colorado
July 24 and 25,1980
Lynn Collins
Alliance To Save Energy,
Washington, D.C.
Sam Doctors
University of Pittsburgh,
Pittsburgh, Pennsylvania
Gene Frankel
House Committee on Science and
Technology,
Washington, D.C.
Scott Grace
EPA,
Denver, Colorado
Joan Habib
Franklin Research Corporation,
Washington, D.C.
Blair Hamilton
Memphremagog Group,
Newport, Vermont
Jeff Hammarlund
DOE, Conservation and Solar
Energy,
Washington, D.C.
Janice Hamrin
California Energy Commission,
Sacramento, California
Margie Harris
Western SUN,
Portland, Oregon
Harold Liebowitz
Center for Renewable Resources,
Washington, D.C.
John Lowry
National Center for Appropriate
Technology,
Butte, Montana
David Miller
New Mexico Solar Energy Institute,
Las Cruces, New Mexico
Steve Nelson
Northeast Solar Energy Center,
Boston, Massachusetts
Web Otis
Golden Gate Energy Center,
San Francisco, California
Jerry Plunkett
Montana Energy and MHD
Research and Development
Institute,
Butte, Montana
David Roessner
Solar Energy Research Institute,
Golden, Colorado
Carl Strojan
Solar Energy Research Institute,
Golden, Colorado
Rafael Ubico
Solar Energy Research Institute,
Golden, Colorado
Lynn Wray
Energy Conservation Operation and
Maintenance,
Grand Lake, Colorado
Jeff Witwer
SRI International
Menlo Park, California
Marvin Yarosh
Florida Solar Energy Center,
Cape Canaveral, Florida
Michael Yokell
Resource Management
Consultants, Incorporated,
Boulder, Colorado
67
-------
participants-
EPA Section 11 Meeting on State and
Local Energy Conservation Programs
Portland, Oregon
July 29 and 30,1980
Mary Anderson
Oregon Department of Energy,
Salem, Oregon
W.S. Baker
Oregon State University,
Hillsboro, Oregon
Mike Barrett
House Subcommittee on Energy
and Power,
Washington, D.C.
Bernie Birnbaum
Citizens for Solar Washington,
Seattle, Washington
Michael Bull
King County Energy Coordinator,
Seattle, Washington
John Burge
City of Kansas City, Missouri
Robin C. Calhoun
HDR, Incorporated, Energy and
Environmental Programs,
Seattle, Washington
Lee Callaway
Pacific Gas and Electric,
San Francisco, California
Mel Chiogioji
DOE, Conservation and Solar
Energy,
Washington, D.C.
Clay Dunlop
Washington State Energy Office,
Olympia, Washington
Nicholas Fedoruk
DOE, Conservation and Solar
Energy,
Washington, D.C.
Leonard Gassner
Oil Heat Institute of Oregon,
Portland, Oregon
Mark Ginsberg
Arizona Energy Office,
Planning and Economic
Development,
Phoenix, Arizona
Mona Green
Public Utility Commission,
Salem, Oregon
Donald Grimm
Pacific Power and Light,
Portland, Oregon
Glenn O. Harding
Oil Heat Institute of Oregon,
Portland, Oregon
Karen Hastings
Colorado Energy Office,
Grand Junction, Colorado
Karen Heidel
City Manager's Office,
Tucson, Arizona
Dan Hitchcock
Pacific Power and Light,
Portland, Oregon
Duane Keeran
Texas Energy and Natural
Resources Advisory Council,
Austin, Texas
Veronica Kun
Senate Committee on Energy and
Natural Resources,
Washington, D.C.
Larry Landry
Arizona Office of Planning and
Economic Development,
Phoenix, Arizona
Ronald Lehr
Colorado Energy Conservation
Office,
Denver, Colorado
Andrew Lewis
Benton-Franklin Community Action
Committee,
Pasco, Washington
Jim Liles
Boulder Country Architect,
Boulder, Colorado
Melissa McClaren-Lighty
Environmental Management/
General Services Agency,
San Jose, California
Nancy H. McKay
League of Oregon Cities,
Salem, Oregon
Sandra Monje
DOE, Conservation and Solar
Energy,
Washington, D.C.
Gregory Page
City Manager's Office,
Eugene, Oregon
Leon Peterson
Utah Energy Office,
Salt Lake City, Utah
Theodor Rauh
California Energy Commission,
Sacramento, California
Paul Raver
Resources Program Coordinator,
Santa Ana, California
Jean Reeder
Energy Conservation Center of the
Eugene, Oregon Electric Board,
Eugene, Oregon
Donald Rocks
Department of Administrative
Services,
Multnomah County, Oregon
Pat Rose
DOE, Conservation and Solar
Energy,
Washington, D.C.
Sam Sadler
Oregon Appropriate Technology,
Eugene, Oregon
Sam Sperry
City Energy Office,
Seattle, Washington
Dennis J. Sykes
California Energy Extension
Service,
Sacramento, California
Darshan S. Teji
Department of Facilities and
Maintenance,
Phoenix, Arizona
Paul Tutino
General Office of Services,
Oakland, California
Phillip Welker
Idaho Office of Energy,
Boise, Idaho
Ray Wiley
Oregon Energy Conservation
Board,
Eugene, Oregon
Lynn Youngbar
Portland SUN,
Portland, Oregon
-------
B. Conservation and
Solar Program Evaluations
Exhibit 1 presents an overview of a sample of past conserva-
tion and solar program evaluations It includes a brief descrip-
tion of the program, the purpose of the evaluation, and the
evaluation methodology (including the major criteria used to
measure program success). It also indicates the organization
that sponsored the study Finally in cases where information
was available, the chart describes how the results of the
evaluation were used and problems encountered in the de-
sign or implementation of the study
Exhibit 2 presents information on selected current DOE
evaluations of conservation and solar programs as of June
1980. Many of the studies are still in early planning and design
stages, however, and little information is available on them.
-------
Exhibit 1 Examples of Conservation and Solar Program Evaluation
Title
Program
Description
Purpose
of Evaluation
Sponsor
Methodology
Major
Criteria
Use
Problems
Encountered
Energy Extension Service
Pilot Program
state grant program
legislatively mandated
to provide outreach informa-
tion and technical assist-
ance to small energy users
to assess impacts and
costs of the program
to determine which service
delivery mechanisms and
target audiences were most
cost-effective
to meet legislative require-
ments for evaluation
DOE/CSE/Office of
Government Conservation
Programs
compared similar partici-
pant and non-participant
groups
ป telephone survey of actions
taken and planned, at-
titudes, and perceived
usefulness of prpgram
services
collected activity data on
costs, number of clients
served
energy savings
provided justification for ex-
panding pilot to national
program
identification of adequate
control group
reliance on self-reported ac-
tions taken
no independent measure-
ment of energy savings
estimates based on re-
ported actions
Low-Cost, No-Cost
Project Evaluation
multimedia marketing effort
to motivate residential con-
sumers in 6 New England
states to make energy con-
serving actions
combined mass mailing
with TV advertising
to determine the effective-
ness of the program to
motivate New England
households to adopt low-
cost or no-cost conservation
actions
DOE
a telephone survey com-
pared a sample of 1200
New England households
with a control group of 600
in New York
day-after recall tests of 100
households regarding tele-
vision advertising
use of television diaries and
follow-up questionnaires
energy savings
dollar savings
final report not released
could provide guidance to
future energy conservation
marketing efforts
identification of adequate
control group
reliance on self-reported ac-
tions taken
no independent measure-
ment of energy savings
estimates based on re-
ported actions
State Energy
Conservation Program
state grant program
legislatively mandated
combined mandatory and
voluntary conservation
measures
to evaluate operating effec-
tiveness of activities and
ability to achieve legislative
objectives
to recommend program
improvements
DOE/CSE/Office of
Government Conservation
Programs
9 sample states included
field visits and interviews
with Regional Office, state
staff
interviews with DOE Head-
quarters staff
program effectiveness and
goal achievement
guidance to development of
other grant programs
changes in instructions &
guidance to states
additional technical
assistance to states
timing too early to assess
actual impact
State Energy
Conservation Program
and Weatherization
Assistance Program (1977
and 1978)
SECP see adjoining
column
weatherization assists
low-income people to
weatherize their homes
ซ state-level administration
to meet legislative require-
ment for annual report on
activities of programs
to provide guidance for
developing and improving
administration and
effectiveness
GAO
case study approach
field visits to 4 DOE Re-
gional Offices and 24 states
review of state plans and
policies
interviews with state and
federal program staff
management effectiveness
compliance with
requirements
projected energy savings
guidance on technical as-
sistance needed by states
timing too early to assess
actual impact and start-up
problem not resolved
70
-------
Exhibit 1 Examples of Conservation and Solar Program Evaluation (continued)
Title
Program
Description
Purpose
of Evaluation
Sponsor
Methodology
Major
Criteria
Use
Problems
Encountered
Project Conserve
2 pilot sites (Topeka, Kan-
sas and Danbury, CT)
computerized energy audit
announced through utility
mailing
to determine consumer re-
sponse and energy savings
FEA
mail questionnaire to par-
ticipants on actions taken
and satisfaction
gathered use data from fuel
bills
energy savings
have followed up by
monitoring response rates
(requests for audits)
high response rates have
been used to justify con-
tinuation of program
changes in fuel use could
not be directly tied to ac-
tions taken in response to
programs
President's Domestic Policy
Review of Solar Energy
nationwide assessment of
solar programs
public participation through
regional hearings
to respond to Presidential
directive
to assess all Federal solar
programs as to whether
they constitute optimal
policy
Interagency Committee
chaired by DOE
panel reports
public testimony
document reviews
program effectiveness
one of several bases for
recommendations for Pres-
idential initiatives
evaluation started late in
process
difficult to develop consen-
sus on program assess-
ments
Residential Solar
Demonstration Program
legislatively mandated
to demonstrate practical
use of solar heating in 3
years, solar heating and
cooling in 5 years
to develop performance
criteria
to describe experiences of
those involved in market
acceptance of solar houses
to explore trends in charac-
teristics, opinions, and ac-
tions of participants
HDD/Office of Policy Devel-
opment and Research
marketing study
developed descriptive pro-
files of actors and institu-
tions involved
personal interviews with
887 participants and 640 in
comparison groups
market acceptance
small sample size
sample not randomly
selected so analysis and
applicability of findings is
limited
Federal Solar Heating and
Cooling Demonstration
Program
Residential Buildings
Commercial Buildings
legislatively mandated
to demonstrate practical
use of solar heating in 3
years, solar heating and
cooling in 5 years
to develop performance
criteria
to measure success in
meeting program goals
GAO
examined 20 operating
demonstrations in 7 states
examined economic viability
and technical soundness
ซ reviewed management
procedures
used interview, reviewed
documents
goal achievement
71
-------
Exhibit 1 Examples of Conservation and Solar Program Evaluation (continued)
Title
Program
Description
Purpose
of Evaluation
Sponsor
Methodology
Major
Criteria
Use
Problems
Encountered
Tennessee Valley
Authority's Home Insulation
Program
provides free audits and
no-interest loans for weath-
erizing residences
to assess program impact
during program's first 18
months
TVA
pretest post test design,
participants served as their
own controls
home energy surveys of
housing characteristics and
conservation actions
data on actual energy
consumption taken from
monthly bills
energy savings
TVA reformulated forecasts
of impact, reemphasized
efforts to reach all
residential consumers
TVA expanded data
collection on program
participants
lack of data on individual
participant characteristics
lack of control group to
estimate actions that would
have been taken in
absence of program
Boiler Operator Workshop
Evaluation
2-day training program on
energy conserving boiler
adjustments
to assess cost effectiveness
of workshops for boiler op-
erators in state-owned
buildings
State of Minnesota
surveys of workshop
participants, one
immediately following and
one 24 months later
projection of fuel savings
based on estimates
associated with boiler
adjustments
energy savings
dollar savings
justified program
reliance on self-reported
actions taken
no independent measure of
energy savings estimates
based on reported actions
Evaluation of Telephone
Energy Conservation
Hotlines
2 telephone hotlines run by
the state energy office and
the state's largest utility
to assess attitudinal and
behavioral effects of serv-
ices on callers
State of Minnesota
randomly selected
participants (178) and
control group (169)
mail survey
examined census-type
characteristics,
weatherization actions
performed, participant
attitudes, energy savings
collected fuel use data on
both groups
energy savings
response rates and
incomplete questionnaires
obtaining reliable fuel use
data, particularly for liquid
fuel
Weatherization Program
assists low-income people
to weatherize their homes
to assess energy savings
State of Minnesota
examined 20 operating
demonstrations in 7 states
examined economic viability
and technical soundness
reviewed management
procedures
used interview, reviewed
documents
goal achievement
72
-------
Exhibit 1 Examples of Conservation and Solar Program Evaluation (continued)
Title
Program
Description
Purpose
of Evaluation
Sponsor
Methodology
Major
Criteria
Use
Problems
Encountered
Analysis of Cost
Effectiveness of
Appropriate Technology
Programs
advises and assists Gov-
ernor and state agencies
provides services to local
government and community
organizations
to fulfill state legislative
mandate
to assess costs and bene-
fits of office and its pro-
grams
State of California
separate direct and indirect
benefits
data on programs derived
from projections and esti-
mates based on other
studies, life-cycle cost
analyses
estimated dollar savings
fulfill reporting requirement
no independent, direct
measure of energy savings
Federal Energy
Management Program
(several reports)
legislatively mandated
to implement mandatory
and voluntary conservation
measures
to assess progress of
measures to conserve en-
ergy in Federal buildings
and transportation systems
GAO
review whether available
products are being used
review of plans and policies
cost benefit
compliance
Federal Energy
Management Program
legislatively mandated
to implement mandatory
and voluntary conservation
measures
to assess progress of
measures to conserve en-
ergy in Federal buildings
and transportation systems
House Committee on Gov-
ernment Operations
expert testimony
review of documents
compliance
progam effectiveness
Twin Rivers Project
examined requirements for
effective retrofit strategy,
role of the resident
sought to clarify role of field
experiments
NSF
ERDA/DOE
a group of nearly identical
townhouses was weath-
erized and studied
energy savings
73
-------
Exhibit 1 Examples of Conservation and Solar Program Evaluation (continued)
Title
Program
Description
Purpose
of Evaluation
Sponsor
Methodology
Major
Criteria
Use
Problems
Encountered
Residential Energy
Utilization Analysis
(Concord Pilot Study)
energy consumption
analyses by trained inspec-
tors were compared with
do-it-yourself analysis by
residents
to determine if either type of
audit affected energy use
to determine which type
was more effective
Pacific Gas and Electric
2 participant and one
control group
market research study of
demographic, attitudinal,
and behavioral differences
between participants
statistical analysis of parti-
cipant energy consumption
using actual data where
available
energy savings
cost
Energy Conservation Center
Evaluation
provided information in re-
sponse to consumer ques-
tions about insulation, solar
equipment, financing, etc.
to assess users' reactions
to Center
to determine what actions
were taken
Pacific Gas and Electric
telephone interviews with
100 randomly selected resi-
dential customers who had
called the Center
client satisfaction
Electrical Rate
Demonstration Project
examined the potential of
various feedback mecha-
nisms to reduce energy
consumption
to assess effectiveness of
different forms and fre-
quencies of feedback
University of Washington
samples of consumers and
control groups participated
in experiments with differing
rate structures, rebates,
and varying feedback
mechanisms
collected fuel use data
energy savings
74
-------
Exhibit 2 Current DOE Conservation and Solar Evaluation
Title
Program
Description
Purpose
of Evaluation
Sponsor
Methodology
Major
Criteria
Use
Problems
Encountered
Institutional Buildings
Grants Program
provides matching grants
for energy audits, technical
assistance, and conserva-
tion measures for schools,
hospitals, local government
buildings, public care
institutions
to collect information on
program operations prior to
design of full-scale
evaluation
DOE/CSE
objectives for the program
have been clearly defined
and performance measures
have been developed
interviews with people in
target institutions and state
offices
review of documents
produced (e.g., audit
reports)
will be basis for full-scale
impact evaluation (intended
use)
Weatherization Assistance
Program
assists low-income house-
holds to weatherize homes
to save energy
provides employment
to assess effectiveness in
meeting goals
to assess impact on energy
use
DOE/P&E
currently in design stage
plan full-scale evaluation in
October 1980
design activities cleaning
up data, pilot study in
Minnesota to refine
collection techniques for
before and after data,
designing questionnaire
will be basis for full-scale
impact evaluation (intended
use)
delays
Appropriate Technology
Small Grants Program
supports concept develop-
ment, project development,
project demonstration of
"appropriate technologies"
to develop findings and
recommendations for ad-
ministrative improvements
in program
to assess program success
in achieving goals
DOE/CSE/OISST
"evaluability assessment"
focus on management
issues
interviews with
Headquarters personnel,
Program managers,
Regional office staff, state
staff, reviewers of
applications, applicants,
public interest organizations
management effectiveness
measurement of program
impact
75
-------
Exhibit 2 Current DOE Conservation and Solar Evaluation (continued)
Title
Program
Description
Purpose of
Evaluation
Sponsor
Methodology
Major
Criteria
Use
Problems
Encountered
Comprehensive Community
Energy Management Pro-
gram
2 year pilot in 17
communities
provides local communities
with materials and method-
ology for energy planning
to examine the effective-
ness of the program's ana-
lytical approach and
process
to recommend modifica-
tions/
improvements to program
DOE/CSE
for each phase of planning,
process documents
produced are reviewed and
participants are interviewed
policy relevance of
analytical approach
effectiveness of process
interim reports can assist
other communities doing
energy planning
planning phases are
designed as sequential in
operation planning is not
linear
timing difficult to measure
effects immediately
Residential Conservation
Service
legislatively mandated
encourages energy
conservation in homes
through audits, lists of
suppliers, loans
operates through utilities
DOE/P&E and CSE/BCS
currently beginning work on
an evaluation plan and
design due in October 1980
Solar Heating and Cooling
Demonstration Program
4 year multi-state program
involving residential, com-
mercial, industrial users
to learn lessons which may
be useful for subsequent
solar demonstrations
to provide input into future
solar policy initiatives
DOE
just beginning
direct interviews with a
geographically dispersed
sample of participants
State Energy Conservation
Program
state grant program
legislatively mandated
combines mandatory and
voluntary conservation
measures
to assess impact of
program
DOE/CSE/Office of State
and Local Programs
states take lead on
collecting data
DOE provides technical
assistance and validates a
sample of state data
energy savings
76
-------
References
Reports
Other Materials
Department of Energy. Domestic Policy Review Panel, The Domestic
Policy Review of Solar Energy. A Response Memorandum to the
President of the United States. TTD-22834, February 1979.
Department of Energy Draft Policy Programming and Fiscal Guid-
ance, FY1982-1986, January 30, 1980.
Department of Energy Energy Conservation Program Summary
Document, FY 1981, February 1980.
Environmental Protection Agency. 2979 Section 11 Report to the
President and Congress, EPA 600/9-80-008. January 1980.
Environmental Protection Agency. 1980 Section 11 National Hearing
Transcript, EPA 600/9-80-060. December 1980.
General Accounting Office. Energy Conservation: An Expanding
Program Needing More Direction. EMD-80-82. July 24, 1980.
General Accounting Office. Hie Multiprogram Laboratories: A Na-
tional Resource for Nonnuclear Research, Development and Demon-
stration. EMD-78-62. May 1978.
National Academy of Sciences, National Research Council. Energy
in Transition: 1985-2010, Final Report of the Committee on Nuclear
and Alternative Energy Systems, San Francisco: W. H. Freeman and
Co., 1979.
Office of Technology Assessment. Conservation and Solar Energy
Programs of the Department of Energy: A Critique, GPO Stock No.
052-003-00757-6. June 1980.
Chelimsky Eleanor, An Analysis of the Proceedings of a Symposium
on the Use of Evaluation by Federal Agencies, Vol. II, MITRE Report
M77-39, July 1977.
Cone, Bruce, An Analysis of Federal Incentives Used to Stimulate
Energy Production, Battelle Pacific Northwest Laboratory, Richlands,
Washington, PNL 24-10, Revision 2. February 1980.
Landsberg, Hans H., et al, Energy, the Next Twenty Years (Spon-
sored by the Ford Foundation and Administered by Resources for the
Future), Cambridge, Mass.: Ballinger Publishing Company, 1979.
Salasin, John, et al., The Evaluation of Federal Research Programs,
MITRE Technical Report MTR-80W 129, June 1980.
Sant, Roger, et al., The Least Cost Energy Strategy, The Energy
Productivity Center, Mellon Institute, Pittsburgh: Carnegie-Mellon
University Press, 1979.
Schurr, Sam H., et al., Energy in America's Future. The Choices
Before Us, Johns Hopkins University Press for Resources for the
Future, 1979.
Stobaugh, Robert, and Daniel Yergin, Eds., Energy Future, New York:
Random House, 1979.
Wholey, Josephs., et al., Federal Evaluation Policy, Washington, DC:
The Urban Institute, 1976.
------- |