r/Bft
May 2009
EVALUATION OF ERA'S
TEMPORALLY INTEGRATED
MONITORING OF
ECOSYSTEMS (TIME) AND
LONG-TERM MONITORING
(LTM) PROGRAMS
Promoting Environmental Results
4
Through Evaluation
-------
Acknowledgements
This evaluation was conducted by Industrial Economics, Incorporated (lEc) and Ross &
Associates Environmental Consulting, Ltd. (Ross & Associates) for EPA's Office of Policy,
Economics, and Innovation under Contract EP-W-07-028. An Evaluation Team guided the effort
consisting of David LaRoche, Jerry Kurtzweg, Michael Hadrick, and Michele McKeever of EPA's
Office of Air and Radiation; Matt Keene of EPA's Office of Policy, Economics, and Innovation; and
Nancy Tosta, Jennifer Major, Shawna McGarry, and Tim Larson of Ross & Associates. Matt
Keene also served as the technical program evaluation advisor. Keith Sargent, EPA National
Center for Environmental Economics; Jay Messer, EPA Office of Research and Development; and
Kent Thornton, FTN Associates also provided important historical context and input regarding
analysis of economic data. We would also like to thank TIME/LTM program principal
investigators who agreed to be interviewed for this evaluation.
This report was developed under the Program Evaluation Competition, sponsored by EPA's
Office of Policy, Economics and Innovation. To access copies of this or other EPA program
evaluations, please go to EPA's Evaluation Support Division's website at
http://www.epa.gov/evaluate
-------
TABLE OF CONTENTS
ACRONYMS i
EXECUTIVE SUMMARY ii
CHAPTER 1: INTRODUCTION 1
I. PURPOSE/OBJECTIVES OF THE PROGRAM EVALUATION 1
II. EVALUATION QUESTIONS 1
III. OVERVIEW OF TIME/LTM PROGRAM 2
IV. STRUCTURE OF THE REPORT 2
CHAPTER 2: METHODOLOGY FOR THE TIME/LTM PROGRAM EVALUATION 3
I. EVALUATION DESIGN 3
II. TIME/LTM PROGRAM LOGIC MODEL 3
III. STEPS FOR CONDUCTING THE EVALUATION 5
IV. QUALITY ASSURANCE PROCEDURES 7
CHAPTER 3: TIME/LTM EVALUATION FINDINGS 8
QUESTION 1: WHAT ISTHE PURPOSE OF TIME/LTM PROGRAMS? 9
QUESTION 2: WHAT ARE THE KEY CHARACTERISTICS OF THE TIME/LTM PROGRAM? 13
QUESTIONS: WHAT ARE THE USES OF TIME/LTM DATA? 19
QUESTION 4: WHAT IS THE RELATIONSHIP OF TIME/LTM TO OTHER ECOLOGICAL
MONITORING PROGRAMS? 27
QUESTION 5: WHAT ARE THE COSTS ASSOCIATED WITH THE TIME/LTM PROGRAM? 32
QUESTION 6: HOW IS THE TIME/LTM PROGRAM CURRENTLY ADMINISTERED
AND MANAGED? 36
QUESTION 7: WHAT OPPORTUNITIES EXIST TO IMPROVE THE TIME/LTM PROGRAM? 38
CHAPTER 4: CONCLUSIONS AND RECOMMENDATIONS 41
APPENDIX A: HISTORY AND CONTEXT FOR TIME/LTM A-l
APPENDIX B: TIME/LTM PROGRAM EVALUATION METHODOLOGY B-l
APPENDIX C: PUBLICATIONS DRAWING ON TIME/LTM DATA, 1985-2007 C-l
APPENDIX D: OVERVIEW OF ENVIRONMENTAL MONITORING PROGRAMS D-l
-------
EXHIBITS
Exhibit 2-1. TIME/LTM Logic Model Components 3
Exhibit 2-2. TIME/LTM Logic Model 4
Exhibit 2-3. TIME/LTM Evaluation Methodological Approach 7
Exhibit 3-1. Regions Identified as Acid-Sensitive in the Northern and Eastern United States 10
Exhibit 3-2. Locations of TIME/LTM Sites Designated "Current" in ORD Database 14
Exhibit 3-3. Years of Record for TIME/LTM Sites 15
Exhibit 3-4. Characteristics of TIME/LTM Sites 16
Exhibit 3-5. Major Variables Measured at Current TIME/LTM Sites 1
Exhibit 3-6. Logic Model of TIME/LTM Program 21
Exhibit 3-7. Trends in Lake and Stream Sulfate Ion Concentrations at LTM sites, 1990-2006 24
Exhibit 3-8. LTM Sites Added or Removed Based on First and Last Observations at Each Site 29
Exhibit 3-9. Examples of Environmental Monitoring Programs 30
Exhibit 3-10. Terrestrial, Air, and Aquatic Monitoring Programs (Federal) 31
Exhibit 3-11. EPA funding for TIME/LTM program, 1993-2009 33
Exhibit 3-12. TIME/LTM Resource Allocation by Region 34
-------
ACRONYMS
ALSC Adirondack Lakes Survey Corporation
ANC Acid neutralizing capacity
ARP Acid Rain Program
CAAA Clean Air Act Amendments of 1990
CAMD EPA Clean Air Markets Division
CASTNET Clean Air Status and Trends Network
DOC Dissolved organic carbon
EMAP Environmental Monitoring and Assessment Program
ERP Episodic Response Project
LTM Long-term Monitoring
NADP National Atmospheric Deposition Program
NAPAP National Acid Precipitation Assessment Program
NPS National Park Service
NSWS National Surface Water Survey
NYSERDA New York State Energy Research and Development Authority
OAR EPA Office of Air and Radiation
OPEI EPA Office of Policy, Economics, and Innovation
ORD EPA Office of Research and Development
OWOW EPA Office of Wetlands, Oceans, and Watersheds
TIME Temporally Integrated Monitoring of Ecosystems
TMDL Total Maximum Daily Load
USGS US Geological Survey
-------
EXECUTIVE SUMMARY
An evaluation of the Temporally Integrated Monitoring of Ecosystems and Long Term
Monitoring (TIME/LTM) program was selected as one of five program evaluations in Fiscal Year
(FY) 2009 under EPA's Office of Policy, Economics and Innovation (OPEI) 2008 Program
Evaluation Competition. This evaluation includes an assessment of program design,
implementation, costs, and other factors to determine program effectiveness.
TIME/LTM collects data to examine trends in surface water chemistry in response to changing
air emissions and acid deposition. The program is currently managed by the EPA Office of
Research and Development (ORD) which intends to discontinue funding for the program in FY
2010. The EPA Office of Air and Radiation (OAR) is likely to assume responsibility for the
program and is considering needed changes and options for management. The evaluation was
conducted between November 2008 and May 2009 based on approximately 25 formal
interviews conducted with principal investigators working under cooperative agreements with
EPA to collect TIME/LTM data, EPA managers, and other interested parties. Background
research was conducted and literature reviewed.
The key findings of this evaluation were developed in response to several questions posed to
interviewees, addressing program objectives, characteristics, uses, relationship to other
monitoring efforts, costs, administration, and potential improvements. Summary findings are:
The objectives of TIME/LTM have changed overtime
Current objectives are to measure patterns and trends in acidity of freshwater
ecosystems and establish a long-term record of ecological conditions
The TIME/LTM program design has changed over the years to focus on sites with long
term data in the Eastern United States
Similar chemical data are collected from current TIME/LTM sites but at different
frequencies for different purposes
TIME/LTM data are used for a broad range of purposes, but by relatively few people.
TIME/LTM data are used to better understand patterns of and trends in acidification in
freshwater ecosystems
TIME/LTM data are used for reporting on the effectiveness of national and international
programs to reduce acid deposition
TIME/LTM data have contributed to policy development, implementation, and
enforcement
TIME/LTM data are used to contribute knowledge and understanding of
interrelationships between acidification and other ecological conditions
TIME/LTM evolved from a variety of environmental monitoring programs and continues
to evolve as sites are added and deleted
TIME/LTM is one monitoring effort among many, but is not integrated with others
Program costs, in sum, have generally remained the same but EPA funding has declined
-------
Overall program costs and costs per site sampled and analyzed are difficult to ascertain
Specific details on how cooperative agreements were established and are currently
managed across sites are not clear
TIME/LTM program data and documentation are not easily accessible
Opportunities for cooperators to interact have been limited
Cooperators believe that TIME/LTM data are collected and analyzed efficiently, but
offered suggestions for future consideration
Cooperators suggest additions to the types of data collected
Expansion of sites into other geographic regions is of interest to some interviewees
Based on these findings, several conclusions were drawn. Overall TIME/LTM has been used for
many purposes and appears to have met the original objectives of providing a long term data
record and contributing to understanding the effectiveness of the Clean Air Act, but these
objectives are currently poorly documented. While the long term data record is a valuable
resource, access to the data are limited and publically available data documentation is non-
existent. The years of staff experience invested in TIME/LTM represent a valuable scientific
resource and this experience coupled with the long-term data record could be better used to
shape future aspects of the program. The transfer of the program from ORD provides an
opportunity to establish clearer management roles and responsibilities, but requires
identification of stable funding. TIME/LTM is nearly invisible to most scientists other than those
directly involved. TIME/LTM data frequently appear to be used in scientific publications
addressing acid precipitation, but they are not always acknowledged as "TIME/LTM" and are
selectively used or merged with and augmented by other lake and stream acidification
measurements. TIME/LTM is a relatively small monitoring effort that could possibly be
integrated with other monitoring efforts such as the emerging water quality monitoring efforts
in the EPA Office of Water.
Several recommendations result from this evaluation as follows:
Clarify the critical scientific question about acidification that needs addressing
Provide a forum for scientists and experts with knowledge of acidification and
TIME/LTM to discuss how best to collect data to address the critical question
Examine monitoring methodologies, including frequency and parameters of data
collection
Explore options to link TIME/LTM to other long-term monitoring programs
Provide funding to continue collection of at least a sub-sample of TIME/LTM data as
program adjustments are made
Continue to improve access to TIME/LTM data, publications, and tools
Establish clear institutional roles and responsibilities for monitoring and data
management
Appendices provide supporting information on the history of TIME/LTM, details of the program
evaluation methodology, TIME/LTM publications, and an overview of other ecological
monitoring programs.
-------
%*sj%»»t*sj%»»t*sj%»»t*sj%»^^
CHAPTER 1: INTRODUCTION
%*Sj%»»t*Sj%»»t*Sj%»»t*Sj^^
This program evaluation investigates the history, operations, costs, products, and perceptions of
the Temporally Integrated Monitoring of the Environment (TIME) and Long Term Monitoring
(LTM) programs to assess program effectiveness. TIME/LTM is currently managed as an
integrated monitoring program to examine trends in surface water chemistry in response to
changing air emissions and acid deposition. The evaluation was conducted between November
2008 and May 2009 based on interviews, background research, and discussions with numerous
parties involved in TIME/LTM in some capacity. This report outlines the methodology used in the
evaluation, summarizes the findings, and offers recommendations for possible future actions.
An evaluation of TIME/LTM was selected as one of five program evaluations in FY 2009 under
EPA's Office of Policy, Economics and Innovation (OPEI) 2008 Program Evaluation Competition.
The evaluation entails an assessment of program design, implementation, costs, and other
factors to determine TIME/LTM program effectiveness, long-term sustainability, and
contributions to knowledge of ecological conditions affected by acid deposition. Assessment of
these programmatic aspects of this environmental monitoring effort may also help to identify
and develop performance measures for both TIME/LTM and other ecological monitoring
programs, to improve overall relevance for environmental monitoring programs.
EPA's Office of Research and Development (ORD) is interested in discontinuing its funding and
management of the TIME/LTM program. The Office of Air and Radiation (OAR) is a potential
recipient for these responsibilities. OAR requested funds from OPEI to conduct this evaluation,
in part to address opportunities for improved program effectiveness during and after this
transition. The intended audience for the report and related products is both OPEI and OAR, but
also includes other agencies that partner with EPA to collect and utilize ecological monitoring
data, such as the National Park Service (NPS) and U.S. Geological Survey (USGS). OAR plans to
use the results of the evaluation to assess the extent to which the program is meeting its
objectives and identify opportunities for program improvement. OPEI may use the results and
learning from this evaluation to inform planning, management and evaluations of other
environmental monitoring efforts nationwide.
n, ,: " ".,'".' ' '
The evaluation is designed to answer the following seven questions:
1. What is the purpose of the TIME/LTM program?
2. What are the key characteristics of the TIME/LTM program?
3. Who uses TIME/LTM data and for what purposes (e.g., basic research, policy
development)?
4. What is the relationship of TIME/LTM to other ecological monitoring programs?
-------
5. What are the costs associated with TIME/LTM?
6. How is TIME/LTM administered and managed?
7. What opportunities exist to improve TIME/LTM?
The TIME/LTM program has evolved from a variety of monitoring activities, originally started
under the National Acid Precipitation Assessment Program (NAPAP)1 in the early 1980s.
Appendix A provides a history of the evolution of TIME/LTM from initial concerns about air
pollution and acid precipitation. Additional details are included in the findings of this report.
TIME/LTM currently supports data collection on chemical conditions in water bodies in the
Northeast and Mid-Atlantic states to provide scientists and policy-makers data on patterns and
trends in acidification. The principal investigators (cooperators) and other staff involved in the
TIME/LTM program use EPA funding to collect and analyze data from lakes and streams in
various regions susceptible to acidification. As described in more detail in Chapter 3, water
samples are collected on a varied schedule, ranging from several times per month at LTM sites
to once per year at TIME sites. The key chemical variables analyzed in each sample include the
major acid anions (sulfate and nitrate), base cations (calcium and magnesium), pH and ANC,
aluminum, and dissolved organic carbon. The data collected by the cooperators are compiled by
ORD and made available for interpretation by EPA and other researchers. Numerous
publications and reports addressing acid precipitation and ecological conditions have been
generated based on TIME/LTM data (Appendix C).
The remainder of this document is organized into three chapters and a series of appendices.
Appendix A provides background information on the history of TIME/LTM. Chapter 2 provides
an overview of the methodology used in this evaluation. The entire methodology, including a
list of interviewees, is provided in Appendix B. Chapter 3 describes the major findings based on
the questions used in the interviews and background research conducted by the evaluators.
Chapter 4 presents the conclusions resulting from the evaluation findings and the
recommendations to OAR for improving the TIME/LTM program. Appendix C lists the numerous
publications that have drawn on TIME/LTM data. Appendix D describes other monitoring
programs potentially related to TIME/LTM.
1 NAPAP is a federal interagency program, originally created under the 1980 Acid Precipitation Act to conduct acid
rain research and report findings to Congress for ten years. The research by NAPAP significantly contributed to the
establishment of Title IV Clean Air Act Amendments of 1990 under which the Acid Rain Program was created. NAPAP
member agencies include EPA, US Dept of Energy, US Dept of Agriculture, US Dept of the Interior, the National
Aeronautics and Space Administration, and the National Oceanic and Atmospheric Administration
-------
CHAPTER 2: METHODOLOGY FOR THE TIME/LTM PROGRAM
EVALUATION
A Core Evaluation Team was established that consists of the contractor (Industrial Economics,
Inc. and Ross & Associates Environmental Consulting, Ltd), managers and senior staff from OAR,
and the evaluation lead from OPEI's National Center for Environmental Innovation, Evaluation
Support Division. A Steering Committee was also established to provide broad input over the
course of the evaluation. The Steering Committee was comprised of representatives from OAR,
ORD, and other federal agencies that conduct ecological monitoring, including NPS and USGS.
The Core Evaluation Team, with input from the Steering Committee, developed the evaluation
questions referenced in Chapter 1. The following sections provide an overview of the key
methodological components used to conduct this program evaluation. The entire methodology
is included in Appendix B.
II, - :. - , ,, .: ...'.:.
A logic model is a graphical representation of the relationships among program inputs, outputs,
and outcomes (Exhibit 2-1). A logic model helps to elucidate the components, participants, and
processes that affect a program and provides a key means to understand interactions and
dependencies that are critical to the success of a program evaluation.
Inputs: basic resources of funds, staffing, and knowledge dedicated to the program
Activities: specific processes or results of the inputs needed to achieve program goals
Outputs: immediate products that result from activities and often used to measure short-term
progress
Customers: groups and individuals targeted by TIME/LTM funding and associated activities and
outputs
Short-Term Outcomes: immediate uses of TIME/LTM data linked to outputs
Intermediate Outcomes: changes in knowledge and understanding based on use of TIME/LTM
data
Long-Term Outcomes: changes in behavior based on TIME/LTM data; the overarching goals of
the program
Exhibit 2-1. TIME/LTM Logic Model Components
Exhibit 2-2 depicts a high-level logic model of the TIME/LTM process as understood by the Core
Evaluation Team prior to initiation of the evaluation. As the Team gained understanding
throughout this evaluation, a more detailed logic model was developed, particularly focused on
understanding the uses of TIME/LTM and is shown in Chapter 3 under evaluation Question #3.
-------
Resources
Activities
Outputs
Customers
Short-Term
Outcomes
Intermediate
Long-Term
Data on
environmental
conditions of
waterbodiesin
acid sensitive
regions
Exhibit 2-2. TIME/LTM Logic Model
-------
III.
Four major steps were taken to conduct this evaluation, utilizing a number of primary and
secondary sources of information. These steps include: (1) identification and review of relevant
documentation and literature; (2) collection of information from interviews; (3) analysis of data
from documentation and interviews; and (4) preparation of the final evaluation report.
1. Identification and Review of Relevant Documentation
To describe the purpose, objectives, and general program characteristics of TIME/LTM, a
literature search was conducted, based on program publications provided by OAR and OPEI, as
well as other publicly accessible documents. Key data sources for this evaluation included:
Research articles and other TIME/LTM-based publications from peer-reviewed scientific
journals (1985-2007), compiled by ORD. The articles include a range of topics such as
analysis of trends in surface water chemistry in specific geographic regions, applications
of various models for detecting and predicting changes in analyte levels, and case
studies on the effects of acid deposition on aquatic ecosystems. (See Appendix C)
NAPAP reports, publicly accessible online. The most recent report (2005) is based on
2002 air emissions data, and uses quantitative and qualitative indicators to assess the
effectiveness of the cap and trade approach to reduce emissions, improve air quality
and reduce acid deposition while minimizing compliance costs. NAPAP also identifies
emerging areas of acid rain research and long-term environmental monitoring.
Pages of earlier NAPAP documents, as faxed and scanned from researchers' personal
libraries (few are available electronically or from public libraries).
Cooperative agreements, interagency agreements, and research proposals.
Recent review articles and other relevant publications found through basic online
literature search, using Google Scholar and University of Washington Library search
engines. Key search terms included TIME, LTM, ecological monitoring, acid rain, and
ecosystem acidification (Note: these searches yielded few new publications beyond the
original 105 publications compiled by ORD, but the ones found provided useful
information).
2. Collection of Information from Interviews
To identify specific uses of TIME/LTM data and the types of policy and research questions being
answered, telephone interviews were conducted with a number of stakeholders from various
program perspectives. The current cooperators from each of the six TIME/LTM regions were
interviewed; they comprise the majority of TIME/LTM data collectors and users and publish
regularly on the status and trends in surface water chemistry of acid-sensitive lakes and streams
across the network. ORD contractors, funded through on-site research support contracts at both
TIME/LTM laboratories (Corvallis and Cincinnati), were interviewed based on their experience
with TIME/LTM data collection and analysis. Representatives from OAR, ORD, and other federal
agencies and nongovernmental organizations identified by EPA were also interviewed to gather
-------
information on current and potential data uses, management and administration, and the
relationship of TIME/LTM to other ecological monitoring systems. The Methodology in Appendix
B provides a brief description of stakeholders selected for interviews and reasons for their
selection, as well as the interview guide with detailed interview questions and sub-questions.
3. of
This evaluation was primarily based on a content analysis of data collected from document and
literature review and interviews. The evaluation utilized qualitative interview and analysis
methods. From comprehensive notes taken during interviews, responses were broadly
categorized and summarized into common themes by stakeholder group. The framework for
organizing information consisted of a series of summary documents to catalog themes and
corresponding passages, generating support for findings, conclusions, and recommendations.
Preliminary findings and recommendations were reviewed by the Core Evaluation Team.
Exhibit 2-3 below depicts the general data collection methods and sources that were used to
answer each of the seven major question areas in this evaluation.
Evaluation Questions
Data Collection Method
Data Source(s)
(1) What is the purpose of
the TIME/LTM programs?
(2) What are the key
characteristics of the
programs?
Document review and
literature search
Interviews
Document review
Interviews
TIME/LTM bibliography
ORD
jOAR
TIME/LTM bibliography
ORD
OAR
TIME/LTM cooperators
(3) Who uses TIME/LTM
data and for what
purposes (e.g., basic
research, policy
development)?
Document review and
literature search
Interviews
TIME/LTM bibliography, EPA Acid Rain
Progress Reports, NAPAP annual
summary
Logic model for TIME/LTM program
ORD
OAR
TIME/LTM cooperators and program
managers
NPS, USGS, CEBC, Data Basin, etc.
(4) What is the
relationship of TIME/LTM
to other ecological
monitoring programs?
Literature search
Interviews
Publicly available documents online
ORD, NPS, USGS, OW, etc.
(5) What are the costs
associated with
TIME/LTM?
Document review
Interviews
Cooperative/interagency agreements,
contracts
ORD
TIME/LTM cooperators and program
managers
-------
Evaluation Questions
administered and
managed?
(7) What opportunities
exist to improve
TIME/LTM?
Data Collection Method
Interviews
Analysis, development
of findings and
recommendations
Data Source(s)
ORD
Information collected from 1-6
Discussions with Evaluation Team
Steering Committee
and
Exhibit 2-3. TIME/LTM Evaluation Methodological Approach
4.
This report constitutes the final evaluation report, which has been prepared in accordance with
EPA guidelines. Members of the Core Evaluation Team provided review and input on a draft of
this report. Cooperators and other interviewees were also given the opportunity to review the
findings and provide input on technical accuracy and completeness, prior to finalization of this
document.
This evaluation required a quality assurance review of the analysis of qualitative information
gathered through interviews. See the full Quality Assurance Project Plan for this analysis in
methodology included in Appendix B. The following measures were taken to ensure consistency
in the qualitative research process conducted for this evaluation.
1. Interview questions were emailed to the evaluation team and interview participants and
followed the interview guide to ensure consistency in the way questions were asked
during discussions with TIME/LTM interview participants.
2. At each interview, at least one staff person in addition to the person leading the
interview was present to record notes.
3. Interview notes were compiled into summary documents, with responses to specific
interview questions grouped together to facilitate their analysis and characterization.
4. For quality assurance, EPA staff, the Core Evaluation Team, and interviewees had
opportunities to review the report for technical accuracy and completeness.
-------
CHAPTER 3: TIME/LTM EVALUATION FINDINGS
This chapter presents findings from the evaluation of TIME/LTM, organized around each of the
seven evaluation questions. Based on the interviews and background research, findings are
summarized below. Details are described in the remainder of this chapter.
1. What is the purpose of the TIME/LTM program?
A. The objectives of TIME/LTM have changed over time
B. Current objectives are to measure patterns and trends in acidity of freshwater
ecosystems and establish a long-term record of ecological conditions
2. What are the key characteristics of the TIME/LTM program?
A. The TIME/LTM program design has changed over the years to focus on sites with
long term data in the Eastern United States
B. Similar chemical data are collected from current TIME/LTM sites but at different
frequencies for different purposes
3. Who uses TIME/LTM data and for what purposes (e.g., basic research, policy
development)?
A. TIME/LTM data are used for a broad range of purposes, but by relatively few
people.
B. TIME and LTM data are used to better understand patterns of and trends in
acidification in freshwater ecosystems
C. TIME/LTM data are used for reporting on the effectiveness of national and
international programs to reduce acid deposition
D. TIME/LTM data have contributed to policy development, implementation, and
enforcement
E. TIME/LTM data are used to contribute knowledge and understanding of
interrelationships between acidification and other ecological conditions
4. What is the relationship of TIME/LTM to other ecological monitoring programs?
A. TIME/LTM evolved from a variety of environmental monitoring programs and
continues to evolve as sites are added and deleted
B. TIME/LTM is one monitoring effort among many, but is not integrated with others
5. What are the costs associated with TIME/LTM?
A. Program costs, in sum, have generally remained the same but EPA funding has
declined
B. Overall program costs and costs per site sampled and analyzed are difficult to
ascertain
6. How is TIME/LTM administered and managed?
A. Specific details on how cooperative agreements were established and are currently
managed across sites are not clear
B. TIME/LTM program data and documentation are not easily accessible
C. Opportunities for cooperators to interact have been limited
7. What opportunities exist to improve TIME/LTM?
-------
A. Cooperators believe that TIME/LTM data are collected and analyzed efficiently, but
offered suggestions for future consideration
B. Cooperators suggest additions to the types of data collected
C. Expansion of sites into other geographic regions is of interest to some Cooperators
FINDING 1A. THE OBJECTIVES OF TIME/LTM HAVE CHANGED OVER TIME
The objectives of LTM and of TIME/LTM have evolved since the inception of the programs. The
primary current objective of TIME/LTM as a single program is to measure changing levels of
acidification occurring in Northeastern and Mid-Atlantic watersheds as a means to assess the
effectiveness of Title IV of the 1990 Clean Air Act Amendments (CAAA). A second purpose of
the program, although not explicitly documented, is to maintain an established record of data
collection for ecological research and modeling. TIME and LTM began as separate programs at
different times, each with different but related purposes.
In 1982-83, under NAPAP, long term monitoring was initiated at a collection of sites previously
managed by several universities and federal and state agencies. The goals of the long-term
monitoring were initially to "detect and measure deposition related trends in the chemistry of
low acid neutralizing capacity (ANC) surface waters, and to compare the response of these
waters over geographic gradients of sulfate and hydrogen ion deposition as well as among major
different geographic regions receiving comparable deposition."2 (Exhibit 3-1)
2 Newell, Powers, and Christie. Analysis of Data from Long-Term Monitoring of Lakes. 1987. EPA/600/4-87/014, U.S.
Environmental Protection Agency, Washington, D.C.
-------
.
Exhibit 3-1. Regions Identified as Acid-Sensitive in the Northern and Eastern United States
Source: US EPA, Office of Research and Development. Response of Surface Water Chemistry to the Clean
Air Act Amendments of 1990 (2003)
The 1986 NAPAP Annual Report describes the objectives of the "long-term monitoring effort"
conducted from 1982-1986 as "(1) detect and measure long-term trends in the chemistry of
surface waters with low ANC; and (2) compare chemical trends in these surface waters over
gradients of acidic deposition and in different geographic areas that receive similar levels of
deposition."3 The sites comprising long-term monitoring, however, were not ideal for
quantifying regional trends. Many of the sites were chosen based on availability of past
sampling data and were not representative of specific regional conditions. Additionally, quality
control protocols were not standardized among the study regions.
Between 1984 and 1986, the National Surface Water Survey (NSWS) was initiated under NAPAP,
based on probability surveys, to document the status and extent of chronic acidification within
acid sensitive regions throughout the United States. Using statistical techniques, the NSWS
sampled a total of 2075 lakes in the East and 752 lakes in the West that allowed estimation of
chemical conditions in 28,300 lakes and 56,000 stream reaches in all major acid-sensitive
regions. The Adirondack Mountains in New York had the largest proportion of acidic surface
waters (14%) in the NSWS.4 The NSWS consisted of multiple phases that assessed not only the
Herrick, Charles N. 1986. Annual Report, National Acid Precipitation Assessment Program. Office of the Director of
Research, Washington, DC. 163pp.
4 Stoddard, J.L., Kahl, J.S., et al. 2003. Response of Surface Water Chemistry to the Clean Air Ace Amendments of
1990. EPA 620/R-03/001.
10
-------
extent of acidification, but examined periodicity of acid runoff in a subset of the original NSWS
sites, based on quarterly sampling during one year.5
The results of the NSWS were used to consider design changes to the long-term monitoring
sites, both to understand how well those sites represented changes in the regional population of
surface waters and to determine measurement parameters for chemical variables (e.g., how
much change must occur to indicate a trend). The 1986 NAPAP Annual Report indicates that
the NSWS analyses were to be used to design the Long-Term Monitoring Project (LTMP).6 The
goal of the LTMP was to detect changes or trends in the biologically relevant chemistry of
surface waters.
LTM was redesigned in 1987 with a goal "to detect and measure trends in the chemistry of low
acid-neutralizing capacity surface waters over gradients of hydrogen ion and sulfate deposition
and in different geographic regions receiving comparable deposition."7 The LTM redesign
selected lakes and streams in clusters, across sulfate and hydrogen ion depositional gradients, in
different geographic regions of the United States. Sites for which data already existed as a part
of other programs were preferentially chosen in an effort to extend the period of record for the
program, and various sites were added to the LTM program at different times. For example,
research in Pennsylvania watersheds began in 1988 as part of EPA's Episodic Response Project
(ERP), which focused on stream chemistry during runoff events and the effects on aquatic biota.
When ERP ended in 1991, the watersheds were adopted into the LTM program (as part of the
Appalachian Plateau region) and the focus and sampling schedule changed to determine the
effects on stream chemistry that were due to changes in atmospheric deposition. The new
design allowed LTM to answer questions about the changes in individual freshwater systems
related to seasonal chemistry and episodic acidification and to identify trends related to these
changes overtime.
TIME was planned in the late 1980s and implemented in 1991.8 In 1990 the goals of TIME were
identified as:
Provide regional early warning signals of surface water acidification or recovery
Provide ongoing assessment of regional patterns or trends in surface water acidification
or recovery
Assess the extent to which observed spatial and temporal patterns in surface water
chemistry correspond with model forecasts
5 NAPAP Annual Report, 1986
6 This appears to be the first reference to a formal program and shortly thereafter LTMP was referred to as LTM.
7 Newell, Powers, and Christie. Analysis of Data from Long-Term Monitoring of Lakes. 1987. EPA/600/4-87/014, U.S.
Environmental Protection Agency, Washington, D.C.
8 While not explicitly mentioned by name, TIME, among other programs, was authorized in Section 7403(c)(2) of the
Clean Air Act. The statute calls for the EPA Administrator to conduct a research program that includes "establishment
of a national network to monitor, collect, and compile data with quantification of certainty in the status and trends of
air emissions, deposition, air quality surface water quality, forest condition, and visibility impairment, and to ensure
the comparability of air quality data collected in different States and obtained from different nations."
11
-------
Assess relationships between patterns in surface water chemistry and patterns in
atmospheric deposition.9
The program was developed as a special study within EPA's Environmental Monitoring and
Assessment Program (EMAP) to track in more detail the trends in acid relevant chemistry of
particular classes of sensitive lakes in the northeast and streams in the mid-Appalachians. TIME
was designed to enhance LTM by giving unbiased regional trend information through repeated
probability surveys of surface water populations in acid-sensitive regions of the Eastern United
States. As it was first described in 198710, the TIME program was to be a coordinated long-term
monitoring effort that would obviate many of the criticisms associated with environmental
monitoring systems, including those related to design and data comparability. Selected TIME
sites were to include a small number of trend sites in each region, with spatially extensive,
regional sites selected from a statistically distributed population. Many TIME sites were
selected based on sampling done as part of the NSWS and other NAPAP-based assessments.
Sites were selected to represent the regions shown in Exhibit 3-1. Some TIME sites were
statistically representative of these regions and may change overtime. Other TIME sites were
based on longevity of record and have continued to be sampled.
TIME/LTM as a program does not represent a true probability sample as envisioned under EMAP
principles. The TIME/LTM program monitors both probability and site-specific trends and
seasonal variation from deliberately selected sites. With the exception of Florida, where the
majority of lake acidity is due to natural organic acidity, the regions include the vast majority of
acidic surface waters in the U.S. The TIME/LTM program design has changed over the years,
primarily by elimination of dozens of LTM sites. All sites were eliminated in Colorado and the
Upper Midwest (Michigan, Minnesota, and Wisconsin), as were almost two dozen sites in
Vermont (see discussion and graphic in Finding #4). According to ORD, sites were dropped
primarily because of funding cuts.
FINDING IB. CURRENT OBJECTIVES ARE TO MEASURE PATTERNS AND TRENDS IN ACIDITY OF
FRESHWATER ECOSYSTEMS AND ESTABLISH A LONG-TERM RECORD OF ECOLOGICAL
CONDITIONS.
There was consistent agreement among cooperators and officials from EPA and other federal
and nonfederal agencies that the primary purpose of TIME/LTM is to measure patterns and
trends in the acidity of freshwater ecosystems to determine the effectiveness of policy
regulating atmospheric deposition. A second objective, if less specified or measureable, was
often reported by interviewees as being of equal importance: TIME/LTM provides an established
long term record of ecological conditions that can be used to track trends and develop
hypotheses on ecological processes. Neither of these current objectives is consistently
described in current documentation and publications.
9 Aquatic Effects Research Program (AERP) Status, EPA, April 1990. EPA/600/M-90/001
10 Thornton, Payne, Ford, and Landers. The Concept of TIME. 1987. U.S. Environmental Protection Agency, Corvallis
Environmental Research Laboratory, Corvallis, OR.
12
-------
In the ten-year period after the enactment of the 1990 CAAA, marked decreases in emissions of
sulfur dioxide and, to a lesser extent, nitrogen oxide occurred. During this same time frame
variability in climate increased and in many of the watersheds where TIME/LTM sites are found,
forests matured. Atmospheric deposition and land use, vegetative cover, and soils are all known
to influence the acid-base chemistry of surface water. At the TIME/LTM sites, decreased levels
of sulfate and acidity were observed in three of five TIME/LTM regions. A number of
cooperators interviewed expressed the view that TIME/LTM data are now providing policy-
makers information on which to base decisions, but the program is only just beginning to realize
its potential to increase understanding of complex ecological processes in forested watersheds.
Characteristics of TIME/LTM sites can be described from many perspectives. The following
findings consider spatial and temporal characteristics and nature of the data collected at each
site.
FINDING 2A. THE TIME/LTM PROGRAM DESIGN HAS CHANGED OVER THE YEARS TO FOCUS
ON SITES WITH LONG TERM DATA IN THE EASTERN UNITED STATES
The TIME/LTM program currently consists of fewer sites than when it began in the early 1980s.
There are now approximately 286 lakes and streams11, collaboratively sampled and analyzed by
state agencies, academic institutions, EPA, and other federal agencies. The program continues
to collect a core set of chemical variables, with only minor modifications to the original
TIME/LTM Quality Assurance/Quality Control protocol published in 1991.
As described under the TIME/LTM program objectives, the current TIME/LTM program began as
a collection of cooperative efforts between EPA and several universities and federal and state
agencies. Many sites were eliminated due to funding cuts, including all sites in Colorado and the
Upper Midwest (Michigan, Minnesota, and Wisconsin), and almost two dozen sites in Vermont
(see discussion and graphic in Finding #4).12 The current program structure is a reflection of its
evolution from a patchwork of monitoring sites to a cohesive network. Exhibit 3-2 shows
locations of current TIME/LTM sites.
11 This estimate is based on data provided by cooperators and ORD for 2009, and at least six sites are sampled as both
TIME and LTM. The ORD Database reports 303 current sites, more than cooperators identify. The number of sites
can change year to year based on several factors, including inability to access sites due to property restrictions and
weather events, and lack of funding.
12 The Vermont region originally provided data on 36 LTM lakes. This number was subsequently reduced to 25 and
then further reduced to 12, within approximately the last 10 years. According to the principal investigator in Vermont,
the sampling scheme for the 12 lakes is more intensive, requiring about 80 samples per year with weekly monitoring
of 7 lake outlets during spring runoff.
13
-------
LTM Lakes
9 LTM Streams
TIME Lakes
9 TIME Streams
Exhibit 3-2. Locations of TIME/LTM Sites Designated "Current" in ORD Database13
Source Map: Google Earth, Source Data: ORD
The TIME/LTM sites represent a long term record of data. For LTM sampling, the average years of data
collection for sites is 19 years and for TIME sites it is 11 years. As can be seen in Exhibit 3-3, for current
sites within the ORD Database, there are significant numbers of TIME sites with more than ten years of
data and LTM sites with more than 16 years of data.
This mapping is based on latitude/longitude values provided for 303 sites in the ORD Database. These sites to not
match the sites shown on the OAR/CAMD Web site: http://www.epa.gov/airmarkets/assessments/activesites.html
14
-------
10
Years of Data
11-14
10-15
16-20
Years of Data
21-25
>25
Exhibit 3-3. Years of Record for TIME/LTM Sites
Source: ORD Database
FINDING 2B. SIMILAR CHEMICAL DATA ARE COLLECTED FROM CURRENT TIME/LTM SITES BUT
AT DIFFERENT FREQUENCIES FOR DIFFERENT PURPOSES.
The same chemical variable data are collected from all current TIME and LTM sites in each
region, with the exceptions of stream flow and speciated aluminum. Exhibit 3-4 provides a
summary of the sites by region, sampling frequency, and variables sampled. At each site, data
on the following chemical variables are collected at least once per year: acid neutralizing
capacity, aluminum, calcium, chloride, magnesium, nitrate, potassium, sodium, sulfate, and
other variables including dissolved organic carbon and inorganic carbon, silicon, color, and
conductivity.
15
-------
Exhibit 3-4. Characteristics of TIME/LTM Sites
TIME Lakes**
Ad iron docks (New York)
Maine
- Massachusetts
- Maine
- New Hampshire
- Rhode Island
- Vermont
Total TIME Lakes
TIME Streams**
Northern Appalachians
- Pennsylvania
- West Virginia
Ridge / Blue Ridge
- Maryland
- Pennsylvania
-Virginia
-West Virginia
Total TIME Streams
Total TIME Lakes and
Streams
LTM Lakes
Adirondacks (New York)
Maine
Vermont
Total LTM Lakes
LTM Streams
Appalachians
- Pennsylvania!
Catskills
-New York ±
Virginia Intensive ±
Virginia Extensive (Trout
Streams)
Total LTM Streams
Total LTM Lakes and
Streams
TOTAL TIME/LTM SITES
Sites
43
8
5
14
1
1
72
22
14
1
3
14
4
58
130
52
16
12
80
5
4
3
64
76
156
286
Collection Major ions
interval* collected
Summer/Fall y
X
Summer
Summer
Summer
Summer
Summer
X
Spring
Spring
X
Spring
Spring
Spring
Spring
Monthly y
Quarterly y
Quarterly y
Monthly y
Monthly/Episodes y
Weekly/Episodes y
Quarterly y
Total Al" Al" Limited Al"
Collected Speciated Speciation
XX"
XX"
XX"
XX"
XX"
X " X
XX"
X " X
XX"
X
X
* Samples are collected once per specified season/interval
** All TIME sites are monitored annually
± Stream flow data are collected from these sites
16
-------
Exhibit 3-5 describes several of the
key variables collected at current
sites and their importance to
ecological monitoring in
undisturbed forested watersheds.
Additional measurements, while
not required forTIME/LTM, were
established earlier by NSWS and
continue to be collected by some
cooperators. These measurements
include titrated acidity, dissolved
inorganic carbon, iron, manganese,
silicates, and total phosphorus.
Stream flow data and speciated
aluminum are collected from some,
but not all, current TIME and LTM
sites. Stream flow data are
collected at 12 LTM streams in the
Catskills, Northern Appalachians,
and Virginia. Flow data enhances
data interpretation because water
can be much more acidic during
times of heavy flow, such as spring
runoff or after large storms. When
measuring patterns and trends
associated with episodic events, as
LTM sampling frequencies are
designed to do, the ability to
control for flow as a confounding
factor affecting acidity is important.
Aluminum data, on the other hand,
are collected from all current TIME
and LTM lakes and streams, but the
extent to which they are speciated
into their various forms of dissolved compounds, including the toxic monomeric form, varies
across the network. Total aluminum is collected and reported for all TIME sites and about half of
the LTM sites. According to ORD, total aluminum as an individual measure is less important than
various aluminum species as determinants of freshwater integrity.
Exhibit 3-5. Major Variables Measured at Current
TIME/LTM Sites
Acid neutralizing capacity (ANC) - Widely used to measure
the acidity of surface water, ANC is the amount of base in
water that allows for the determination of the amount of
acid required to make it acidic. The greater the ANC, the
more acid is required to acidify it.
Aluminum (Al~) - Inorganic Al- reacts with proteins and is
toxic to fish and other aquatic animals. Al increases with
increasing acidity and can be an indication that a watershed
has been affected by atmospheric deposition.
Calcium (Ca2+) - Watershed soils contain exchangeable Ca2+
ion. Soils in which Ca2+ has been depleted by atmospheric
acids have a limited ability to neutralize acids, which can
then travel to and increase the acidity of surface waters.
Dissolved organic carbon (DOC)- A mixture of organic
compounds that are the major source of natural acidity in
freshwater. Increasing DOC can be an indication of
ecosystem recovery.
Nitrate (NO3~) - Negatively charged compound (anion) of
nitric acid. Nitrate is used to measure the saturation of
nitrogen in ecosystems that can result from atmospheric
deposition.
pH - The concentration of hydrogen ion (H+) measured on a
logarithmic scale. As pH decreases, water's acidity
increases. Lakes with pHs below 6.0 (7.0 is neutral) are less
diverse and lack many aquatic plant and animal groups.
Sulfate (SO4) - Anion of sulfuric acid, a major acid supplied
to ecosystems by acid deposition. Sulfate, as with nitrate, is
usually found in low concentrations in undisturbed waters.
17
-------
Sampling frequencies vary by TIME and LTM study sites for different purposes. All TIME sites are
sampled annuallyin fall for lakes and spring for streams.14 Because TIME sites were chosen to
statistically represent various populations of lakes, annual samples are sufficient to assess
trends in the number of acidic lakes and streams within regions. LTM sites, on the other hand,
are not statistically representative of other lake and stream populations. To answer specific
questions about acid-sensitive ecosystems, such as factors related to nitrogen and sulfur cycling,
sampling is conducted on an annual, quarterly, or monthly basis. To capture data during
episodic events such as spring runoff, large storms, or droughts, sampling may occur more or
less frequently. In Vermont, for example, to capture an entire runoff event, researchers sample
one particular lake every two days, beginning in March and throughout the spring, using an Isco
automatic water sampler. The value of having a combination of TIME and LTM sites within a
region is important to understanding what TIME may not be capturing in seasonal conditions.
The holding time - the time between sample collection and sample analysis - varies for each
chemical variable and is a significant factor affecting the timing and scheduling for cooperators
and their field workers. For example, pH must be measured on site, nitrate can be measured up
to seven days after collection, sulfate up to 28 days, and aluminum up to six months.
Generally the samples are collected by two-person teams who travel by foot, truck, boat, ski,
and for particularly remote sites, helicopter. The two-person team records lake height and
temperature, recent shore activities or other watershed disturbances, and whether there has
been recent precipitation before collecting the water samples. Collection may be done by hand
using a plastic Van Dorn sampling device or, at lakes sampled by boat or helicopter, a device
such as the Kemmerer sampler. Samples are typically filtered and aliquoted for anion and cation
analyses as soon as possible after collection, then iced or refrigerated for transportation back to
the lab for analysis.
While each region had established protocols for data collection and analysis prior to initiation of
a formal LTM program, a standard protocol, the Long Term Monitoring Quality Assurance Plan,
was developed in 1985 that built upon existing protocols and that used for the NSWS. The ORD
LTM protocol describes the general analytic methods to be used by each cooperator. In reality,
however the cooperators use a range of different instruments and cite various different
references for the methods they employ. The need for and development of a single, updated
QA/QC protocol for use by all cooperators was raised by several interviewees as an important
issue to discuss as a group.
According to the ORD LTM protocol, cooperators are required to perform tests of data quality,
using estimates of precision and accuracy and results of blank analyses, and report them in the
quality assurance reports sent to ORD Corvallis Laboratory along with the data for analysis. To
monitor laboratory performance overtime, laboratories collecting LTM have participated in the
14 Water tends to be less acid when water levels are lowest. Because TIME sites are sampled in late summer or early
fall when water levels are lowest, acidity patterns and trends must be interpreted cautiously as they are potentially
higher at other times of the year.
18
-------
Canada Centre for Inland Waters Long Range Transport of Airborne Pollutants Interlaboratory
Comparability Studies, but current requirements to participate in these studies as part of the
TIME/LTM network are unclear.15
TIME and LTM data have been used by a limited number of scientists and policy analysts at the
national, regional, and state levels for a range of purposes, including increasing basic
understanding of acidification in freshwater ecosystems; reporting on program effectiveness;
policy development, implementation, and enforcement; and applied research and modeling.
The primary data users are program cooperators who collect the data and ORD and OAR. The
logic model (Exhibit 3-6) provides a more refined depiction of the TIME/LTM program (see
Exhibit 3-6). This refined view shows with enhanced clarity, the flow of information from
collection through use. The detailed logic model reflects information collected over the course
of the evaluation and builds upon the basic model shown in Chapter 2. Specific components of
the logic model, including outputs, customers, and outcomes, are detailed in the findings that
follow. The "TIME/LTM Monitoring" Stage represents the set up and management of the
monitoring program, as well as actual data collection. The "Post-Monitoring" Stage reflects
actions taken and use of the data (e.g., research) once data have been processed and are
available for distribution.
FINDING 3A. TIME/LTM DATA ARE USED FOR A BROAD RANGE OF PURPOSES, BUT BY
RELATIVELY FEW PEOPLE.
The majority of publications, such as peer-reviewed journal articles and assessments of EPA
programs and policy, are generated by a small community of scientists and analysts already
familiar with TIME/LTM and with connections to program data stewards. As detailed in the
following sections, program cooperators and officials from ORD and OAR are the primary data
users and have managed to produce over one hundred articles and reports during the past
twenty years; but because data, metadata, and other relevant program documentation have
not been publicly available, new and potential users must contact ORD or a cooperator to access
data. Additional details on data access and documentation can be found in this chapter under
Question #6.
FINDING 3B. TIME/LTM DATA ARE USED TO BETTER UNDERSTAND PATTERNS OF AND TRENDS
IN ACIDIFICATION IN FRESHWATER ECOSYSTEMS
Research and monitoring data collected for the TIME/LTM program has furthered the
understanding of acidification in freshwater ecosystems since the first chemical surveys were
conducted from about 1950-1979. By the 1980s, based on a number of studies in the
15 Some cooperators participate in the National Water Research Institute's annual auditing program with
approximately 60 freshwater research laboratories in Canada and the United States. The audit includes evaluation of
the mean and standard deviation of specific chemical parameters tested; results are included in the annual data
reports submitted to ORD.
19
-------
Adirondacks, it was known that acid deposition from the combustion of fossil fuels had affected
a number of lakes in the Northeastern United States, but the extent of regional deposition - the
number of affected lakes and streams, the amount, and whether acidity was naturally occurring
or anthropogenic - had not yet been described. According to a research summary prepared by
the Adirondack Lakes Survey Corporation (ALSC), initial long term monitoring in 1983 was
focused on measuring the level of acidification, the extent to which it might be caused by acid
rain, and developing estimates of reductions in acid emissions that would be required to correct
it.16 Based on the current TIME/LTM program's design and ability to monitor both probability
and site-specific trends and seasonal variation from deliberately selected sites, TIME/LTM has
since allowed scientists to answer questions about the number of chronically acidic lakes in the
Northeastern United States, how the number of episodically acidic lakes have changed through
time, and more specifically, how nitrogen and sulfur cycle through ecosystems.
16 Adirondack Lake Survey Corporation. Acid Rain and the Adirondacks: A Research Summary. 2005. Ray Brook, New
York.
20
-------
Outcomes
Intermediate Long-Term
Data on
biological and
other physica
conditions at
TIME/LTM sites
STAGE 1:
TIME/LTM
MONITORING
Academic, state,
and federal funds
TIME/LTM water
data
Other soil, water,
foliar data
Air monitoring
data (e.g.,
CASTNET,
NADP)
awareness of
environmental
conditions and
trends
1
r-»
Chang
environ
protec
policie
Environmental baselines
established
programs
STAGE 2:
POST
MONITORING
Exhibit 3-6. Logic Model of TIME/LTM Program
21
-------
In the 2008 Report on the Environment/7 EPA used TIME/LTM to derive an indicator to help
answer questions about the trends in outdoor air quality and effects on human health and the
environment. The indicator used data collected from TIME/LTM to correlate decreasing levels of
air pollution to general improvements in watershed integrity. Specifically, ANC data collected
during 1992-2005 were used to report patterns of increasing recovery from acidification in the
Adirondack Mountains, New England (Maine, New Hampshire, and Vermont), and the relatively
unchanged proportion of chronically acidic streams in the Ridge and Blue Ridge provinces of
Virginia. Trend assessments beyond the borders of Northeastern U.S. watersheds and
comparisons between North America and Europe have also been conducted using TIME/LTM
data.18 More recently, scientists have used TIME/LTM data to assess chemical variables other
than the primary ones used to describe freshwater acidification trends (sulfate and nitrate)
including dissolved organic carbon (DOC).
FINDING 3C. TIME/LTM DATA ARE USED FOR REPORTING ON THE EFFECTIVENESS OF
NATIONAL AND INTERNATIONAL PROGRAMS TO REDUCE ACID DEPOSITION
TIME/LTM data, together with data from other EPA programs such as NADP and CASTNET, are
used to help determine the effectiveness of the Acid Rain Program (ARP) as measured by
specified program performance measures. Specifically, TIME/LTM data are used to measure the
percent change in number of chronically acidic water bodies in acid-sensitive regions. The ARP,
in turn, is one of several contributing programs in EPA's Performance and Accountability
Reports, required by the Government Performance and Results Act, which help EPA to report on
its progress in meeting its strategic goal related to clean air.19
In its 2005 Report to Congress, NAPAP used TIME/LTM data from 1990-2000 to describe trends
in recovery among each of the acid-sensitive regions and concluded that many previously acidic
lakes and streams were recovering due to reductions in acid deposition - primarily sulfate
deposition - mandated by Title IV of the 1990 CAAA. The report also noted that not all areas
were making a recovery, and even those with increasing ANC and decreasing sulfate and nitrate,
key indicators of chemical recovery, may still be acidic enough to damage sensitive fish and
other aquatic life.
Under Title IX of the 1990 CAA, NAPAP was reauthorized to continue research on the ecological
effects of acid deposition and to provide Congress with periodic reports known as integrated
assessments, the first of which was released in 1996. The major goal of the integrated
See the full 2008 Report on the Environment at http://www.epa.gov/lndicators/index.htm.
18 Stoddard, et al. Regional trends in aquatic recovery from acidification in North America and Europe. Nature. 1999;
401:575-578.
19 Other programs contributing to the clean air goal in EPA's Fiscal Year 2008 Performance and Accountability Report
include AirNow, Air Toxics, Clean Air Allowance Trading Programs, Clean Air Research, National Ambient Air Quality
Standards Development and Implementation, Mobile Sources, New Source Review, Regional Haze, Indoor Air Quality,
Stratospheric Ozone Layer Protection Program, Radiation Programs, and Voluntary Climate Programs. See the full
report at http://www.epa.gov/ocfo/par/2008par/index.htm.
22
-------
assessments, of which aquatic ecosystems are one critical element, is to provide "structured,
technical information in a format that enables decision makers to evaluate the effectiveness of
current public policy and that provides a sound science base for future policy decisions."
Officials from OAR and ORD report that a key accomplishment of the 2005 Report to Congress,
the most recent integrated assessment, was the ability to ascertain that a market-based
approach to controlling air emissions through a cap and trade system is effective. Specifically,
they cite the TIME/LTM program as an example of a research-based effort with a robust
outcomes-based design.
The data on freshwater acid-sensitive ecosystems used in the 2005 NAPAP Report to Congress
were drawn from an earlier publication from ORD, Response of Surface Water Chemistry to the
Clean Air Amendments of 1990. A key recommendation from this report states "the
effectiveness of current or future amendments to the CAA can best be determined by
monitoring the response of subpopulations of sensitive surface waters through time. Long-term
records provide the benchmark for understanding trends in ecological responses. The reviewers
of this report strongly urged the authors to recommend continuation of the long-term research
program upon which this report is based and the addition of biological monitoring to begin
documenting potential biotic recovery."
TIME/LTM data are also used extensively in EPA's annual ARP Progress Reports.20 Published
since the first phase of implementation of the ARP in 1995, the reports are used to update the
public on the ARP and related programs, including emissions reductions, compliance, and
environmental results. In the 2007 Report, the most recent publication, EPA cites TIME/LTM as
"essential for tracking the ecological response to ARP emissions reductions." Using levels of
sulfate, nitrate, and ANC as indicators of recovery for the 1990-2006 period, the report
describes decreasing trends of sulfate in most regions, generally flat or increasing trends of
nitrate concentrations, and generally increasing trends in ANC, suggesting changes in
acidification not entirely commensurate with implementation of the ARP. Exhibit 3-7 below
shows the trends for sulfate concentrations, based on LTM sites with complete records for the
time period represented.
20 From 1995-1999 EPA produced ARP Compliance Reports, but beginning in 2000 it stopped producing Compliance
Reports and only released Progress Reports. See also http://www.epa.gov/airmarket/progress/progress-reports.html.
Accessed 3/20/09.
23
-------
Increasing trend
O Increasing non-significant trend
O Decreasing non-significant trend
0 Decreasing trend
Exhibit 3-7. Trends in Lake and Stream Sulfate Ion Concentrations at LTM sites, 1990-2006
Source: EPA, 2008
TIME/LTM data have also been used in conjunction with data from the Canada Acid Rain
Program under the US-Canada Air Quality Agreement, created in 1991 to reduce the impact of
transboundary air pollution. In its 2008 Progress Report, the US-Canada Air Quality Committee
used data from TIME/LTM to develop estimates of critical loads using the Steady-State Water
Chemistry model to determine the combined deposition load of sulfur and nitrogen to which
lakes could be exposed without significant harmful effects (as measured by a threshold level of
ANC). The report found that the proportion of lakes receiving acid deposition greater than their
estimated critical loads has improved overtime, but consistent with NAPAP's 2005 Report to
Congress, concluded that acid-sensitive ecosystems might still be at risk of acidification at
current deposition levels.21
FINDING 3D. TIME/LTM DATA HAVE CONTRIBUTED TO POLICY DEVELOPMENT,
IMPLEMENTATION, AND ENFORCEMENT.
Managers with the National Park Service (NPS) Air Resources Division use TIME/LTM data in
conjunction with air quality monitoring in Shenandoah National Park, as several sites are located
within the park's boundaries. Of the 86 CASTNET monitoring stations across the United States
(primarily in the east), 25 are operated by NPS in cooperation with EPA, one of which is located
in Shenandoah NP. NPS officials report the importance of TIME/LTM data in understanding
watershed response to changes in deposition, particularly when estimating the time needed for
ecosystem recovery in areas adversely affected by acid rain. Notably, scientists have learned
that the lag time between emissions of acid rain precursors and watershed recovery (as
United States-Canada Air Quality Agreement, 2008 Progress Report
http://www.epa.gov/airmarkt/progsregs/usca/docs/2008report.pdf Accessed March 20, 2009.
24
-------
measured by increased ANC and other indicators of recovery) in Shenandoah could take up to
one hundred years.22
In addition to using the data to directly measure trends in surface water chemistry in
conjunction with dry deposition data from CASTNET and wet deposition data from NADP, the
NPS and others have used TIME/LTM data to develop estimates of critical loads, or measures
that determine how much pollutant an ecosystem can tolerate before it becomes degraded.
Unlike a market-based cap-and-trade approach to controlling pollution emissions, critical loads
focus on an ecosystem's capacity to withstand acidic deposition. Language about the use of
critical loads as an assessment tool for ecological response and recovery is not included in the
CAA, however in 2005, EPA included a provision in its Nitrogen Dioxide Increment Rule that
individual states may propose the use of critical load information as part of their air quality
management approach. In 2006 the Critical Loads Ad Hoc Committee was formed under NADP
to provide a venue for discussion regarding the science and application of critical loads for
atmospheric deposition in the United States. The NPS and US Forest Service (USFS) have since
developed recommendations for using critical loads as a tool for managing federal lands. While
the NPS has no regulatory authority over sources of pollution, they do have a consultative role
in the regulatory process.23 Land managers at NPS and other federal agencies have used
TIME/LTM data to develop critical load estimates for setting resource protection and restoration
goals on federal lands.24
States in which TIME/LTM data are collected have also adopted the critical load approach to
resource management. Under Section 303(d) of the Federal Clean Water Act25, states are
required to identify and publish a list of waterways (lakes, ponds, rivers, and streams) that are
"water quality impaired," meaning they do not meet EPA's water quality standards as described
in regulation. In Vermont, LTM data are used to develop that state's Total Maximum Daily Load
(TMDL) for the waterways on the impaired list. A TMDL is similar to a critical load in that it is a
calculation of the maximum amount of a pollutant that a body of water can receive and still
meet water quality standards. To estimate the TMDLs for 37 Vermont lakes, the Vermont
Agency of Natural Resources implemented the Steady State Water Chemistry model using LTM
chemical variable data.26 Following Vermont's example, New Hampshire adopted the critical
load TMDL format for its assessments; and the New York State Department of Environmental
Conservation also uses available LTM data and modeling for assessing water quality changes.
22 Research has shown that acid deposition causes the progressive loss of biologically available calcium and
magnesium, a process known as base cation depletion, which limits some watersheds' ability to effectively neutralize
acid rain.
23 Mandated by the Clean Air Act (42 USC 7470[2] and 42 USC 7475[d][2]), the Wilderness Act (16 USC 1131-1136),
the NPS Organic Act of 1916 (16 USC 1-4), the National Wildlife Refuge System Improvement Act of 1997 (PL 105-57),
and the National Forest Management Act (16 USC 1600-1614).
24 Porter, et al. Protecting Resources on Federal Lands: Implications of Critical Loads for Atmospheric Deposition of
Nitrogen and Sulfur. BioScience. 2005; 55:603-612.
25 33 U.S.C. §1251 et seq. (1972)
26 Vermont 2008 303(d) List of Waters http://www.anr.state.vt.us/dec/waterq/planning/docs/pl_2008.303d_Final.pdf
Accessed March 23, 2009.
25
-------
TIME/LTM data have also been used to enforce CAA regulations. In 1999, Connecticut,
Maryland, Massachusetts, New Hampshire, New Jersey, New York, Vermont, and Rhode Island
successfully brought litigation against the American Electric Power Corporation, the nation's
largest operator of coal-fired power plants, for violating the CAA by emitting nitrogen oxides and
sulfur dioxides at levels known to contribute to the formation of ozone and acid rain.27 Two
other major settlements were brought by the US Department of Justice and EPA against
Midwestern power companies. Researchers were able to use 20 years of stream monitoring
data in Virginia and the central Appalachian region to demonstrate that the companies had
contributed significantly to water quality degradation in the state's native trout streams. Both
companies were required to install pollution control equipment required by the CAA, and to
conduct other measures to decrease air pollution.
FINDING 3E. TIME/LTM DATA ARE USED TO CONTRIBUTE KNOWLEDGE AND UNDERSTANDING
OF INTERRELATIONSHIPS BETWEEN ACIDIFICATION AND OTHER ECOLOGICAL CONDITIONS
In some states where TIME/LTM data are collected, agencies and other organization have
partnered to use the data for a range of multidisciplinary purposes. For example, LTM data
collected by the Vermont Agency of Natural Resources is shared among the University of
Vermont and USFS through the Vermont Monitoring Cooperative. Through this partnership, the
Agency for Natural Resources carried out a number of studies on mercury, using datasets from
LTM and other datasets to show that increases in levels of mercury were due to increased
hydrogen ion (i.e., increased acidity), and that the trend was occurring on a regional level.
Mercury studies have also been carried out in New York using LTM data, by way of data
collection conducted by the ALSC and others with funding from the New York State Energy
Research and Development Authority (NYSERDA), including a statewide mercury survey on lake
fish and associated chemistry; an assessment of DOC in wetlands; and mercury deposition in
lake sediments.28 Cooperators interviewed reported the value of having historically important
records that span twenty or more years to make such assessments.
Cooperators and other scientists have used TIME/LTM data to answer research questions about
acidification that indirectly relate to atmospheric deposition, yet provide contextual information
needed to better understand the effects of air pollution in forested watersheds. For example,
defoliation caused by insect infestation affects a forest's ability to process and utilize nitrogen
that is added to the system from nitrogen fixation or atmospheric deposition. Before gypsy
27 From the settlement, after eight years since AEP was first brought to trial in 1999, Vermont will receive five
installments, each for approximately $360,000. In future years, the money will be used to fund programs that
improve Vermont's air quality, public health and the environment. Under a joint project with Vermont, New York has
agreed to dedicate $500,000 of its 2009 share of the settlement to be spent on a fish and habitat restoration project
for Lake Champlain. See also
http://governor.vermont.gov/tools/index.php?topic=GovPressReleases&id=3245&v=Article. Accessed Feb. 9, 2009.
28 NYSERDA. (2008). Strategic Monitoring of Mercury in New York State Fish.
http://www.nyserda.org/programs/Environment/EMEP/Final%20%20Report%20revised%2008-27-08.pdf; NYSERDA.
(2004). Effects of Atmospheric Deposition of Sulfur, Nitrogen, and Mercury on Adirondack Ecosystems. Accessed May
3, 2009.
26
-------
moths were found on long term monitoring sites in Virginia in 1984, nitrogen levels had been
steadily low, but increased dramatically during moth infestation. Nitrogen levels returned to
near pre-infestation levels after the moth infestation subsided and forests began to recover.
Cooperators report that without the long term data to provide a baseline for nitrogen levels in
the area, and depending on the timing of the sampling (before, during, or after infestation), it
would have been difficult to determine the direction of the trend and contributing factors.
Cross-sectional sampling at a time of particularly high or low nitrogen levels would not have
provided a basis for trend description or pattern identification. Similarly, comparisons of
chloride levels in undisturbed TIME/LTM lakes with urbanized lakes have highlighted the
importance of factoring in anthropogenically introduced chloride when assessing surface water
recovery from acid rain; a salt-affected watershed will alter base cation geochemistry and
complicate assessment of the ecological effect of atmospheric deposition.29 Using USGS Water
Center funding, scientists in Pennsylvania have conducted a survey of mercury in a number of
LTM and other headwater brook trout streams in 2008. Research has also shown that an
indicator of acidification recovery may hinder the return of biota to some lakes and streams.
Using LTM data from several Maine sites, researchers concluded that increasing dissolved
organic carbon, generally believed to be a sign of ecosystem recovery, may actually offset the
return of salmon in certain classes of streams.30
Researchers use TIME/LTM data to develop and test new models and methods that in turn help
improve the science used to make decisions around policy for air emissions. Interviewees from
OAR discussed plans to use TIME/LTM data for PnET-BCG31 modeling, which requires the data
for model inputs and model calibration. PnET-BCG is able to simulate the response of soil and
surface waters in northern forest ecosystems to different types of disturbances, including acid
deposition. The models generate critical load estimates that are used to make predictions about
response under various emission scenarios. Cooperators and other researchers have used
TIME/LTM data for model development and application for more than 20 years. Appendix C
provides a comprehensive bibliography of TIME and LTM publications.
Ecological monitoring is of critical importance to understanding the effects of stressors and
pollutants in the environment. Ecological monitoring, like human health monitoring and
assessment, provides a means to examine the outcomes or results of activities that impact the
environment and potentially actions or policies that mitigate the activities. Researchers suggest
29 Rosfjord, Webster, Kahl, et. al. Anthropogenically Driven Changes in Chloride Complicate Interpretation of Base
Cation Trends in Lakes Recovering from Acidic Deposition. 2007. Environmental Science and Technology; 41:7688-
7693.
30 Kahl and Johnson. Streamwater chemistry and Recovery of Maine Atlantic Salmon. March 2004. Water Quality Fact
Sheet. Center for Environmental and Watershed Research, University of Maine, Orono, ME.
31 PnET-BGC is an integrated biogeochemical model developed to simulate forest and aquatic ecosystems that allows
simultaneous simulation of major element cycles in forest and interconnected aquatic ecosystems.
http://www.ecs.syr.edu/faculty/driscoll/personal/PnET%20BGC.asp
27
-------
that the success of ecological monitoring programs such as TIME/LTM are dependent on several
factors, including: (1) addressing a clearly articulated question, (2) using consistent and accepted
methods to produce high quality data, (3) providing a data management systems that ensures
long-term data accessibility and optimizes data use, and (4) integrating the monitoring effort
into programs that foster continual examination and use of the data.32 Many interviewees
acknowledged that TIME/LTM, while unique in addressing the ecological effects of acid
precipitation, is one of an array of monitoring efforts examining acid precipitation specifically
and ecological conditions generally. Interviewees mentioned water quality monitoring
programs for acidification in some of their states that augment information collected as part of
TIME/LTM.33 The EPA Office of Water in the last several years launched both lake and stream
surveys. These surveys are based on a random sample design as described by EMAP. Several
interviewees noted the need to consider approaches for integrating TIME/LTM and other
national monitoring programs for both cost savings and development of more comprehensive
environmental understanding, but none were aware of any current efforts to do so.
FINDING 4A. TIME/LTM EVOLVED FROM A VARIETY OF ENVIRONMENTAL MONITORING
PROGRAMS AND CONTINUES TO EVOLVE AS SITES ARE ADDED AND DROPPED.
As described in Finding 1A and Appendix A, LTM evolved under NAPAP from many water quality
monitoring efforts that existed in the early 1980s. LTM sites in many cases were already being
sampled as part of state, university, or federal efforts to understand the extent of acid
precipitation. A selection of sites was brought under the umbrella of a Long Term Monitoring
Program (LTMP) originally in 1983, with quality assurance protocols later developed for
sampling by the EPA/ORD Laboratory in Corvallis, OR. These sites were frequently selected
because they had a record of measurements. They did not represent a probabilistic sample and
findings from these sites could not be extrapolated to describe broader regional conditions.
Studies done under the NSWS provided the foundation to conceptualize an integrated approach
to the needed monitoring, based on use of sites drawn from past monitoring efforts, sites
representative of various regions, and sites that could indicate seasonal variability, etc. The
1986 NAPAP Annual Report states the following: "The design of LTMP is mult/tiered, with each
tier representing an increasing degree of detail as needed to assess trends in regional scale
aquatic conditions.... The base of the tier represents broad-scale data collected infrequently on a
subpopulation for which their relationship to the total regional population is known. Seasonal
variability studies could be conducted on a subset of systems selected from those represented by
the base tier. Successive tiers represent successively smaller subpopulations and increased
complexity and intensity of the studies. In most cases existing Task Group [Aquatic Effects Task
Group] water studies would be included in these tiers. At each level, the relationship of the
subpopulation to the resource base established at the base tier is known; hence the results of the
special studies, for example, rates of mineral weathering or artificial acidification of watersheds
32 Lovett, G.M., et al. 2007. Who Needs Environmental Monitoring? Front Ecol Environ; 5(5): 253-260.
33 An example of this is described in "Results from the 2003-2005 Western Adirondack Stream Survey," Lawrence, GB,
et al. November 2008.
28
-------
would be applicable to regions." The AETG envisioned the potential to integrate various types
of monitoring and data from different monitoring programs to describe more completely the
effects of acidification. More details on the evolution of these efforts and the integration of
TIME sites with LTM are described in Finding #1A.
Over the last 25 years the collection of sites comprising LTM has changed. As can be seen in
Exhibit 3-8, many sites were added in 1987 as a result of the analyses done based on the NSWS.
Other sites were added in 1992, perhaps as a function of integration with TIME (although Exhibit
3-8 does not depict TIME sites). Sites have been dropped over the years as shown by the red
bars in the graphic. Sites from 2002-2005 are still considered "current" in the ORD database,
but data have not yet been processed.
E
m
u
N 10
I
1
Sites shown after 2002
have notbeen"dropped/
butdata forapprox 23
^
sites are not yetavailable
in the ORD Database.
Added
t.
Dropped
. . . .
'I II 'I
~T
1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005
Exhibit 3-8. LTM Sites Added or Removed Based on First and Last Observations at Each Site
Source: ORD Database
TIME sites represent both the probabilistic sites established as part of EMAP and sites that had
been measured as part of other surveys and were considered important to incorporate.
29
-------
FINDING 4B: TIME/LTM IS ONE MONITORING EFFORT AMONG MANY, BUT IS NOT DIRECTLY
INTEGRATED WITH OTHERS
Numerous environmental monitoring programs exist across the U.S., managed by many
different federal agencies. TIME/LTM is focused on measurements specifically representing lake
and stream water chemistry to understand acidification. Other monitoring programs assess
both water chemistry and other measures of environmental conditions that may play a role in
acidification. Exhibit 3-9 depicts various existing monitoring programs and the medium of focus.
Exhibit 3-10 provides some additional detail on each of these programs and the agencies with
management responsibilities. Appendix D describes of the programs.
Photochemistry
Chemical Transformations
Cloud Processes
Agriculture
* V
Fire
*At
.Vertical
IP *
Industry r..9..,\; NADP/NTN CASTNET '
_ Transportation*' j.'AM?,j 7;-- -; - - -:- . bry 1. _ IMPROVE J
WetDeposiWon Deposition Visibility
^ SLAMS/NAMS »National Parks
~"1 M^l -L^S^!!:^:
iN H NAWQA ! ^ Forest Productivity j
i ,-
; PI A ;
ater Quality
Ecosystems
1FEMAP
Soils | NRI ji FIA j
Ground Water
i National Lakes Assessment i
-3 ^K
,"i A Aquatic
^^^ Ecosystem
i WSA i
I Human Health
Agricultural
Resources
i NRI !
Sources Transport/Transformation Removal Effects i Monitoring Program j
Exhibit 3-9. Examples of Environmental Monitoring Programs
Source: Adapted from "The Role of Monitoring Networks in the Management of the Nation's Air Quality,"
National Science and Technology Council, CENR, 1999)
Name
Vital Signs Monitoring
Forest Inventory and Assessment
(FIA)
Environmental Monitoring and
Assessment Program (EMAP)
Long Term Ecological Research (LTER)
National Resources Inventory (NRI)
Air Quality Monitoring:
National Air Monitoring Stations
(NAMS)
State and Local Air Monitoring
Lead Agency
NPS
USFS
EPA
NSF
NRCS
EPA
States/Local
States/Local
Number of Sites
32 inventory and monitoring
networks over 270 park units
125,000
12,600
26
800,000
1080
~4000
Date Begun
2007
1930
1988
1979
1956
1979
1979
30
-------
Name
Stations (SLAMS)
Photochemical Assessment
Monitoring Stations (PAMS)
Special Purpose Monitoring
Stations (SPMS)
Interagency Monitoring of Protected
Visual Environments (IMPROVE)
Clean Air Status and Trends Network
(CASTNET)
National Atmospheric Deposition
Program (NADP)/National Trends
Network (NTN)
National Water Quality Assessment
(NAWQA)
National Lakes Assessment
Wadeable Streams Assessment (WSA)
Hydrologic Benchmark Network
TIME/LTM
Lead Agency
States/Local
States/Local
NPS
EPA
Interagency
uses
EPA
EPA
uses
EPA
Number of Sites
57+
156
86
250
42 River Basins
1200 Lakes
1392
15-58 (varies over time)
303 (ORD Database)
Date Begun
1994
??
1985
1991
1978
1991
2007
2000
1963
1983
Exhibit 3-10. Terrestrial, Air, and Aquatic Monitoring Programs (Federal)
Interviewees acknowledged and recognized that they and the agencies they work with
frequently support a variety of monitoring networks. They also mentioned that some
environmental variables (e.g., soils and vegetation) that could be helpful in understanding
transport and behavior of sulfates and nitrates are not extensively monitored in the watersheds
supporting TIME/LTM sites. EPA's Office of Water noted that the initiation of the Wadeable
Streams Assessment (WSA) and the National Lakes Assessment (NLA) may provide opportunities
to work closely with TIME/LTM sites. Both the WSA and the NLA are built on statistical designs
defined by EMAP. Many of these monitoring efforts have detailed and comprehensive Web
sites describing the history and methodologies of the programs and means to access data. The
IMPROVE Web site (http://vista.cira.colostate.edu/improve/ ) is an example of an interactive
site supporting interactive use of tools and publications based on the monitoring activities.
Interviewees also recognized that integrating and using data from multiple monitoring efforts to
understand environmental conditions is challenging due to factors such as:
variability in questions being addressed and statistical requirements to adequately
sample populations
temporal and spatial characteristics of sampling
variability in parameters measured
variability in site characteristics of interest (e.g., undisturbed, prone to acidification,
accessible to road)
Despite these challenges, interviewees indicated strong interest in exploring opportunities for
coordination, integration, and sharing information across monitoring networks. This support
aligns with recent recommendations made by The Nature Conservancy and the Gary Institute of
Ecosystem Studies in their publication: "Threats From Above." The report discusses the impacts
31
-------
of air pollution on ecosystems and biodiversity in the Eastern United States and points out that
"Many important monitoring programs exist in the U.S., but there is currently no comprehensive
integrated network to measure atmospheric deposition, soil and surface water concentrations of
pollutants and biological effects."34 The Report makes recommendations to continue to support
existing monitoring efforts, but also develop a more comprehensive, integrated effort to ensure
that the information needed to evaluate critical loads for sulfur, nitrogen, and mercury and to
track responses of ecosystems to these loads is made available. The Report strongly
recommends expansion of air pollution monitoring to create a comprehensive national program
as soon as possible, with expanded funding to address multiple pollutants.
TIME/LTM support from EPA has been on a declining trend for most of the program's duration
as a single integrated network, while operational costs have not significantly changed. Cost
comparisons by region are difficult for a number of reasons, including the pooling of EPA and
non-EPA resources and differential costs associated with specific sites, but there was general
agreement among interviewees that data collection and analysis are conducted relatively cost-
efficiently and approaches to improve efficiency are likely to be few at current EPA funding
levels.
FINDING 5A. PROGRAM COSTS, IN SUM, HAVE GENERALLY REMAINED THE SAME BUT EPA
FUNDING HAS DECLINED.
Since EPA began investing in long-term monitoring in the early 1980s, ORD has provided the
majority of funding required for program operation, as well as data quality assurance and
overall database management. Over time, ORD funding levels for TIME/LTM have generally
declined from a high of $1.2 million in 1993 to a low of $720,000 in 2009, as shown in Exhibit 3-
II.35 At the same time, according to cooperators, costs for program operation have remained
the same or in some cases increased.
34 Lovett, G.M., and T.H. Tear. 2008. Threats from Above: Air Pollution Impacts on Ecosystems and Biological Diversity
in the Eastern United States. The Nature Conservancy and the Gary Institute of Ecosystem Studies.
35 NAPAP 1986-88 Operating Research Plan (pp Vl-2) documents budget amounts for the LTM program at $700,00 in
1986 and $638,700 in 1987.
32
-------
$1,400,000
51,200,000
51,000,000
$£00,000
$600,000
$400,000
$200,000
Exhibit 3-11. EPA fund ing for TIME/LTM program, 1993-2009.
Source: ORD
In addition to an overall funding decline, the uncertainty of funding availability from year to year
has presented challenges to cooperators. Several cooperators interviewed reported that the
constant hunt for funding has detracted from effective program planning and operation,
including potential staff hiring and training graduate students. During certain years of the
program's history, funding shortages have resulted in fewer sites sampled and in some cases
incomplete data for a given region or group of sites. Funding shortages do not, however, appear
to have significantly affected generation and use of program data. In contrast, during some
years, ORD has provided supplemental funding to cooperators based on funds remaining at
year-end. For example, in fiscal year 2009, supplemental funding raised available funding from
$720,000 to approximately $975,000. Additionally, funds from other federal and nonfederal
sources must also be taken into consideration when estimating total TIME/LTM program costs.
The following section provides additional cost details.
FINDING 5B: OVERALL PROGRAM COSTS AND COSTS PER SITE SAMPLED AND ANALYZED ARE
DIFFICULT TO ASCERTAIN
Several aspects of TIME/LTM funding complicate analysis of the actual costs required to operate
and maintain the program as currently designed. A comparison of costs across regions must
take into account the unique mix of site characteristics and cooperator responsibilities in each
region. Also, other sources, including federal and state agencies and nongovernmental
organizations, contribute funding and in-kind resources to varying degrees. Exhibit 3-12 provides
a general description of total program resources reported for TIME/LTM activities in fiscal year
2009; the data were compiled from interviews with ORD, cooperators, and analysis of
cooperative agreements. The amount ORD allocates to each cooperator generally reflects cost
requirements based on the combination of factors detailed in the following subsections and
assumes some amount of resources contributed by state agencies and other organizations.
Based on the data provided by cooperators and ORD, it is not meaningful to estimate a per site
cost.
33
-------
The Office of Water mentioned that for cost allocation purposes, they estimate costs for the
WSA at approximately $6000 per site visit and $250 for data analysis per site. These sampling
protocols differ from TIME/LTM sample collection and analysis.
Sites
Type of
Site
Sites
Funding
ORD Annual
Funding
Other
Funding
Adirondacks
TIME1
LTM3
TOTAL
43
52
95
$225,400
$0
$225,400
$24,700
NR
$24,700
TOTAL
$250,100
$250,100
Costs
Personnel
Travel
Analysis,
supplies
Other5
TOTAL
$225,400
NR
$0
NR
$14,440
NR
$10,260
NR
$250,100
Catskills
TIME
LTM3
TOTAL
0
4
4
$79,000
$79,000
NR
$0
$79,000
$79,000
$29,535
$3,688
$24,633
$21,144
$79,000
Vermont
TIME
LTM2
TOTAL
0
12
12
$120,346
$6,334
$120,346 |$6,334
$126,680
$126,680
$68,414
$4,745
$27,695
$25,826
$126,680
Maine
TIME1
LTM2
TOTAL
29
16
45
NR
NR
$210,000
NR
NR
$0
$210,000
NR
NR
NR
NR
NR
NR
NR
NR
Pennsylvania
TIME
LTM3
TOTAL
0
5
5
$110,000
$110,000
NR
$0
$110,000
$110,000
$62,000
$7,600
$19,000
$21,400
$110,000
Virginia
TIME
LTM2'4
TOTAL
0
67
67
$125,000
$125,000
$50,000
$50,000
$175,000
$175,000
$107,773
$5,000
$4,840
$57,387
$175,000
Mid-Atlantic
TIME1
LTM
TOTAL
50
0
50
TOTAL 278
Exhibit 3-12. TIME/LTM 1
Source: ORD, Cooperators
$106,000
$106,000
$0
$o~
$106,000
$106,000
$975,746 $81,034 $1,056,780
Resource Allocation by Region
$493,122
$21,033
$90,608
$136,017
$106,000
$1,056,780
Estimated Sampling Frequency
'Annual
2 Quarterly
3 Monthly
Weekly/Episodic
Other
5 Includes fringe benefits, contractual costs
NR = not reported
34
-------
Cooperators pool ORD and non-ORD funding to support program operations
With the exception of the Adirondack region, most funding to support TIME/LTM data collection
and analysis has come from ORD. In the Adirondack region, the majority of the TIME/LTM
program is supported by funds from NYSERDA and the New York State Department of
Environmental Conservation (NYSDEC). The NYSDEC provides in-kind support for the annual
collection of samples from 43 TIME lakes and funds are pooled with those from ORD.36 But for
collection of monthly samples from 52 LTM lakes in the Adirondacks, NYSERDA and NYSDEC
provide 100 percent of the funding and supply the complete LTM dataset to ORD at no cost. In
other regions where data are collected by university researchers, graduate students, and staff,
or researchers affiliated with state agencies, smaller percentages of total program costs are
provided and combined with ORD funding to support TIME/LTM data collection and analysis.
Variation in site type and sampling frequency complicate regional cost comparisons
As described under the program characteristics for Question #2 and illustrated in Exhibit 3-3,
TIME/LTM sites are sampled at different frequencies, although the chemical parameters
measured at each are generally the same. Fiscal burden can largely be driven by sampling
frequency. Several LTM streams, for example, are sampled monthly, in contrast with TIME lakes
that are typically sampled on an annual basis. Some cooperators also collect samples from
particular lakes or streams during episodic events such as spring runoff or floods, requiring
sampling on a weekly or biweekly basis. Each time a site is sampled, regardless of the method
or equipment used to collect water samples (e.g., by hand or by use of an automatic sampler),
samples must be physically collected, transported, and processed by field workers according to
the chemical holding times defined in QA/QC protocols. The location of a lake or stream also
plays a role in the expense to sample it. Because TIME/LTM sites are generally remote and in
undisturbed watersheds, they can be difficult to access. A region with mostly remote sites
requiring aviation for sample collection would probably incur higher costs; but these costs may
still be less than sampling a stream 12 times or more per year that can be reached by truck or
hiking a short distance.
Federal and state agencies contribute funding for collection of data from TIME/LTM sites for
other program purposes
At least two regions are supported in part by funds from state and federal agencies that use
selected TIME/LTM sites as part of other ecological monitoring efforts. The NPS contributed
$50,000 in fiscal year 2009 for the continuation of integrated surface water monitoring in the
Ridge and Blue Ridge Provinces of Virginia, including watersheds in Shenandoah National Park.
In the Catskills Mountains, the New York City Department of Environmental Protection (NYCDEP)
conducts water quality monitoring in three LTM streams that overlap with sites used for
monitoring New York City's drinking water supply. NYCDEP funds pay for field workers' salaries
and sample analysis (amount not specified).
36 The ALSC, a non-profit corporation established in 1983 to gather baseline environmental monitoring data, has
responsibility for TIME/LTM field and laboratory work. See also http://www.adirondacklakessurvey.org/
35
-------
Some cooperators analyze samples collected by other cooperators
Some cooperators collect samples but send them to other cooperators for analysis. For
example, all TIME lake samples, approximately 75 in 2009, are analyzed by the principal
investigator in Maine.
In-kind contributions are important assets
At least four regions in the network rely on the work of volunteers, graduate students, and other
staff who are not paid with TIME/LTM program funds for sample collection and analysis. For
some regions, significant data are collected for the program at minimal cost. For example,
TIME/LTM data are collected as part of monitoring efforts in Shenandoah National Park, known
as the Shenandoah Watershed Study-Virginia Trout Stream Sensitivity Study.37 Sample
collection and logistical support on about 60 streams is provided by state and federal resource
managers, as well as trained volunteers associated with Trout Unlimited. Similarly, in the
Adirondacks, in-kind support is provided by the NYSDEC Division of Air Resources, which
includes office and storage facilities, helicopter and vehicle support, personnel, laboratory, and
computer equipment. Work done in conjunction with universities is also carried out in part by
unpaid graduate students and volunteers who in turn train new students and volunteers.
In each region the principal investigator or cooperator is generally responsible for sampling,
analysis, quality assurance and quality control procedures, and data validation for their sites.
They report these data to EPA ORD in Corvallis, OR where they are quality assured before being
added to the overall database. ORD also analyzes the samples from the 58 TIME streams in the
Ridge, Blue Ridge, and Northern Appalachian regions. The ORD staff in Corvallis is responsible
for data management, final reporting of TIME/LTM data, and general TIME/LTM coordination
across the network. According to ORD, most of these tasks are accomplished through the
dedication of one staff person for 10 percent of his time. Minimal resources are available for
overall management and administration of TIME/LTM, meaning that activities not directly
related to data collection are difficult to accomplish such as managing and providing access to
the data, planning, strategy development, or collaborating with other monitoring efforts.
Additionally, ORD has indicated its intent to withdraw program support, creating a funding and
management vacuum for the program.
FINDING 6A. SPECIFIC DETAILS ON HOW COOPERATIVE AGREEMENTS WERE ESTABLISHED
AND ARE CURRENTLY MANAGED ACROSS SITES ARE NOT CLEAR.
Several interviewees commented that the cooperative agreement process might be simplified
and made more transparent. EPA uses cooperative agreements to provide benefits to
cooperators as much as or more than to EPA. Agreements exist for different time frames for
37 See also http://swas.evsc.virginia.edu/ accessed April 22, 2009.
36
-------
various cooperators and varying amounts, with little clarity of how costs were determined. EPA
and cooperators are very aware that in-kind contributions are made by the cooperators (as
described in Question #5), with cases such as the Adirondacks, where funds other than EPA's are
primarily responsible for ensuring TIME/LTM data collection. Some cooperators would like to
better understand how funds are allocated. Additionally, ORD would like to optimize the
process of managing cooperative agreements.
FINDING 6B. TIME/LTM PROGRAM DATA AND DOCUMENTATION ARE NOT EASILY
ACCESSIBLE.
The primary TIME/LTM data users are the program cooperators who collect the data and EPA,
specifically ORD and OAR (see Question #3 for detailed discussion of data use). In general, all
interviewees for this report acknowledged the significant amount of work by ORD over the last
two decades to manage the data, involving careful observation and data validation before the
data can be used for reporting, often on a constrained budget. For most of the last two
decades, ORD has made the data available by responding to specific requests from cooperators,
other researchers, and/or EPA. The data have not been available for download on the Internet.
Detailed metadata describing the TIME/LTM data are also not available. A few cooperators have
made data from their study sites publicly available on university or state agency Web sites.
More recently, OAR/CAMD has made program-wide data accessible on EPA's Web site. There is
currently a "gentlemen's agreement" to not publicly release data until three years after
collection (unless it's an EPA publication). While it is unclear whether this is documented in
existing agreements, this is a policy that all cooperators agreed should be revisited. All
interviewees agreed that data should be more easily accessible. Additionally, data are not
current. Cooperators indicated that while, for the most part, they provide the data to ORD in a
timely manner, there can be a three to five year lag time in the data being available.
In the course of conducting this evaluation, it was extremely difficult to track and document the
history, decisions, and goals of the program given the lack of documentation on the program by
EPA. Creating the history and descriptions of the program in this evaluation required extensive
searching and requests of individuals to peruse personal libraries to fax pages from documents.
The lack of documentation about decisions such as adding or dropping sites makes it extremely
difficult to develop a complete and accurate picture of the number of sites, their history, and
reasons for inclusion or elimination. In many cases, cooperators provided numbers for sites
sampled and timing of samples that differed from numbers shown in the ORD Database, making
it difficult to reconcile actual sampling and costs for the program. Some cooperators expressed
a desire to better understand decisions such as dropping or adding sites for the surveys.
FINDING 6C. OPPORTUNITIES FOR COOPERATORS TO INTERACT HAVE BEEN LIMITED.
All primary stakeholders interviewed for this report strongly supported regular opportunities to
meet and share ideas and concerns about TIME/LTM, but cooperators in particular identified a
number of potential benefits to be realized in doing so. This could take the form of an annual
conference or workshop that would allow program stakeholders to share current work and
37
-------
better understand patterns and trends on a regional level, something that is currently
accomplished by reading journal articles often years after data are collected. The ability to
share ideas for standardizing and improving data quality and quality control procedures was also
cited as a need and program benefit. Alternatively, a steering committee or leadership task
force could provide a forum for collective thinking and strategic planning to fully leverage
TIME/LTM and help set direction. Cooperators expressed an interest in establishing more
effective means for "give and take" dialog to address issues and explore new ideas. Many other
environmental monitoring programs have conducted periodic reviews to assess objectives,
protocols, and assumptions about monitoring. TIME/LTM has not conducted such an
assessment in nearly two decades.
Additionally, many other environmental monitoring efforts are built on a collaboration model of
contributions from multiple parties. ORD has funded and managed TIME/LTM for decades.
Opportunities to discuss how other entities, such as states, other federal agencies, or other EPA
programs, might engage more directly, provide leadership, expertise, and funds, could help
ensure the long term viability of TIME/LTM.
OAR/CAMD is organizing a workshop for TIME/LTM cooperators in early June 2009.
Cooperators expressed keen interest in participating in this event.
Interviewees identified several areas for program improvement, including changes to the
sampling design and parameters measured, a need for improved access to data, and more
opportunities for data collector and user interactions. These latter two were discussed under
the previous question.
FINDING 7A. COOPERATORS BELIEVE THAT TIME/LTM DATA ARE COLLECTED AND ANALYZED
EFFICIENTLY, BUT OFFERED SUGGESTIONS FOR FUTURE CONSIDERATION
Cooperators reported that the program has operated efficiently and produced a significant
amount of data with a relatively small and often unreliable budget. Interviewees were not able
to identify many ways in which efficiency could be improved or costs reduced based on current
levels of funding. Some suggestions were offered by cooperators and interviewees as possible
approaches to improve efficiency as follows:
Develop a matrix mapping each individual site and the frequency of sampling; in some cases
it may be possible to sample less frequently (e.g., quarterly versus monthly) without
compromising dataset integrity or representativeness.
Contracting with a single collection entity is a possibility, although cooperators noted the
program would lose the expertise that has been established over two decades by having the
same group of principal investigators collect data, with intimate local site knowledge.
Similarly, new methods for data collection, based on remote sensing or other more
advanced technologies could conceivably be adapted to TIME/LTM, but are not likely to be
38
-------
less expensive; furthermore, details on site characteristics and condition currently captured
in field workers' notes would go unrecorded.
FINDING 7B. COOPERATORS SUGGEST ADDITIONS TO THE TYPES OF DATA COLLECTED.
Cooperators and EPA staff acknowledged that the addition of data collected or analyzed would
be challenging unless the funding to support it was made available. Nonetheless, they made a
number of suggestions that would improve the breadth and depth of the TIME/LTM dataset for
potentially small incremental costs. A suggestion made by nearly all cooperators was the
collection of biological data. Biological data, taken together with water chemistry, would
provide the strongest indication of freshwater integrity and the ability to support life. A number
of cooperators currently collect (or have collected in the past when funding has allowed) data
on fish and macroinvertebrate populations from TIME/LTM sites, and report that the data can
be collected within the existing program with little additional effort for the wealth of
information it can provide about the effects of acid rain on freshwater biota. For example,
researchers in Vermont collected data from the mid-1980s on fish and macroinvertebrate
populations from 29 of the original lakes selected from long term monitoring, as well as
plankton data from acidic lakes during from the early 1980s and 1990s. Some of these data
have been analyzed and published (e.g., fish and macroinvertebrate), but the plankton data
have not. Data on aquatic biota and native fishes continue to be collected from several
Pennsylvania LTM sites, but not as formal EPA-funded work; and the Adirondack Effects
Assessment Programa separate but related EPA-funded program currently collects annual
plankton data from 28 LTM lakes, but has yet to publish any results.
Data on stream flow were also cited as an important variable to understanding acidification.
Acidity can vary widely during transient events including large storms or spring runoff. During
high streamflow, the pH or ANC of a stream or lake decreases to a value significantly below its
baseline. At sites where flow data are collected, scientists can better interpret pH and ANC
values in the dataset and tease out confounding variables. Currently, flow is measured at 12
LTM sites in the TIME/LTM program, but cooperators report they would like to see it used in the
analysis and interpretation of TIME/LTM data in reports where the data are used for national
assessments of the CAA or elsewhere. Researchers also cite the importance of flow data for
interpreting trends during droughts and floods that may be related to climate change.
Detailed documentation of vegetation and soil conditions in each watershed of the network was
reported by some EPA interviewees as an area for program improvement. Field workers
collecting samples are required to document a fair amount of detail, based on field notes from
TIME stream assessments in the northern Appalachians we reviewed, including watershed
activities and disturbances, reach characteristics, water clarity and character, and general
assessments of area wildlife and vegetation diversity; yet OAR officials noted that greater detail
is needed for site characterizations as these additional inputs are helpful for interpretation of
the long-term trend data and enhance data comparability across the network. In addition to
more thorough site depictions, updated or verified latitudinal and longitudinal coordinates were
also cited as needed improvements. For example, coordinates in the current database should
39
-------
be at the centroid of the lake or at sampling locations, some however are at parking areas or
access points to the lake.
A number of cooperators suggested the addition of mercury, another atmospheric deposition
component, would be a beneficial variable to include as part of theTIME/LTM program. While
mercury is not an atmospheric acid, it originates from many of the same sources as sulfuric and
nitric acids and accumulates in watersheds where it is converted to its toxic derivative
methylmercury. According to research conducted in the Adirondacks, mercury deposition rates
are currently the highest they have ever been, and since mercury emissions are not controlled
by the CAA, mercury deposition is expected to continue. Some TIME/LTM researchers collect
mercury data at a number of sites in New Hampshire, New York, and Vermont, but suggested a
more uniform approach to collecting it would be beneficial. It is unclear what level of resources
would be required to support mercury data collection across the TIME/LTM network, and some
have noted the technically demanding aspect of mercury sample collection.
FINDING 7C. EXPANSION OF SITES INTO OTHER GEOGRAPHIC REGIONS IS OF INTEREST TO
SOME COOPERATORS.
The addition of TIME and LTM sites to the network has been reported by some cooperators as a
needed improvement to the program, but OAR officials emphasized the importance of increased
precision and additional study variables over adding more sites (see discussion under 7A).
Cooperators in support of adding sites acknowledged the value of having focused on Eastern
watersheds for most of the programs' existence, due to demonstrated acid sensitivity in those
regions, making them an excellent cohort to study relatively small changes in chemistry over
time; but given increased interest in climate change, the addition of sites could provide a more
complete picture of freshwater acid-base chemistry across a nationwide gradient of systems not
limited by current TIME/LTM site inclusion criteria. But aside from an expansion of current
TIME/LTM program objectives to address emerging concerns such as climate change, NAPAP
supported the idea of site expansion in 2005 by reporting that additional sites in acid-sensitive
regions of the Southeast, Midwest, and West would make more complete assessments possible.
40
-------
CHAPTER 4: CONCLUSIONS AND RECOMMENDATIONS
This evaluation describes the development of the TIME/LTM program from a collection of
preexisting water quality/aquatic monitoring programs with differing but related objectives and
protocols to the current integrated program that is primarily used to assess ecosystem response
to reduced air emissions mandated by the 1990 CAAA. Over the last two decades, the program
has undergone a number of design modifications resulting in expansion of sites within some
regions and elimination of sites altogether in others. Despite these changes and shifts,
TIME/LTM has established a laudable long-term record for use by both policy makers and
researchers to contribute to understanding the effectiveness of air pollution policies and the
science of acidification in forested watersheds. The following general conclusions can be made:
Overall TIME/LTM has served a valuable purpose and appears to have met the original
objectives of providing a long term data record and contributing to understanding the
effectiveness of the Clean Air Act.
The long term data record is a valuable resource, but access to these data are very
limited and metadata are non-existent.
The years of staff experience invested in TIME/LTM represent a valuable scientific
resource, as does the long-term data record. Both of these assets could be used to
shape future aspects of the program such as determining specific data needed to better
understand acidification processes, frequency of data collection, priority sites for
measurements, etc.
Business is not as usual for TIME/LTM. The transfer of the program from ORD brings
challenges and opportunities. It is not always clear who makes decisions for TIME/LTM
or how those decisions are or will be made (e.g., sites dropped), or sources of funding.
While numerous publications have been generated using TIME/LTM data, the
objectives, scope, funding, and aspects of management of the program are poorly
documented. The transition provides an opportunity to reassess numerous aspects of
the program, including identification of more stable funding options.
TIME/LTM is nearly invisible to most scientists other than those directly involved. While
data appear to be frequently incorporated in scientific publications, they are not always
acknowledged as "TIME/LTM" and appear to be selectively used or merged with and
augmented by other lake and stream acidification measurements.
In light of the expanded interest in and development of long term environmental
monitoring programs, including the recent launch of stream and lake surveys by the
Office of Water, it is appropriate to consider how a relatively small effort such as
TIME/LTM might be effectively integrated with other monitoring programs.
Based on these conclusions the following recommendations are offered. The June 2009
TIME/LTM Cooperator Workshop can provide a logical forum to initiate discussion on several of
these recommendations.
41
-------
Recommendation 1: Clearly articulate and document the scientific question of
current interest relative to acidification of fresh water lakes and streams.
It is clear that TIME/LTM has contributed to the science and policy of fresh water acidification.
It is less clear what the specific question(s) are that TIME/LTM is currently addressing and
whether the most critical questions about acidification are being addressed by TIME/LTM. An
assessment of TIME/LTM's objectives has not been done in nearly twenty years. Based on
many of the findings and conclusions in this evaluation, as well as increasing interest in data
collection for other major issues such as climate change and interest in understanding the
effectiveness of EPA policies and programs (e.g., acid rain), the time is opportune for an
examination and validation of the critical questions relative to acidification.
Recommendation 2: Establish a forum for interaction and discussion among
the appropriate individuals with expertise to identify the right question (s)
and knowledge of data needed to answer the question.
Original efforts to consider acid precipitation as a national issue were benefited by the
formation and operation of NAPAP that brought together scientists and policy makers from
diverse disciplines. While acid precipitation is just one of several air quality concerns today, it
continues to be a significant issue in several geographic regions. As TIME/LTM management
responsibility is shifted within EPA, an opportunity exists for stakeholders, cooperators, EPA
scientists, and other relevant parties to engage and determine needed and effective paths
forward. A structured forum to promote such engagement is needed.
Recommendation 3: Based on the critical question and data gathered over the
history of TIME/LTM, examine the methodologies and protocols to affirm
approaches to ensure relevance of data collected to the question being
addressed.
TIME/LTM data have been gathered and analyzed over significant time frames. These data are
not only valuable for understanding specific trends in the environmental conditions being
monitored, but also provide information on requirements for monitoring - how frequently must
sites be visited (e.g., seasonally, annually, biennially), what parameters must be measured at
each visit, how many sites require measurement, etc. Clarifying the critical scientific question
should then allow an analysis of existing data to determine optimal protocols for continued
sampling. Examination of individual site records may be step in this process. This type of
assessment has not been done for TIME/LTM in many years. In the late 1980s/early 1990s when
TIME was established to supplement LTM, the theory and needs for specific data were
examined. An adaptive approach to data collection is beneficial to continually refine and
improve both data collection relevance and cost. Some cooperators suggested that changes
could be made to the benefit of TIME/LTM.
Recommendation 4: Explore other long term monitoring options and how
TIME/LTM can leverage or benefit them
Detailed analyses of all other monitoring programs was not conducted as part of this evaluation,
but based on interviews and the continued evolution of water quality monitoring programs,
42
-------
opportunities to integrate TIME/LTM sites into larger surveys appear to exist. The EPA Office of
Water and USGS both expressed interest in exploring these opportunities. Cooperators also
suggested that these opportunities may exist. Additionally, given current challenges of
overarching issues such as climate change, examination of ways that site data such as collected
in TIME/LTM might contribute to understanding ecological effects would be timely. Again,
however, TIME/LTM should be considered as a potential component of larger and better funded
efforts rather than a stand-alone monitoring program.
Recommendation 5: While engaging in the above activities, identify short
term funding to continue data collection at some level to ensure the integrity
of the long-term TIME/LTM data record.
OAR, OW, and ORD should consider approaches to identify funding to ensure that the value of
the TIME/LTM data record is preserved. This may mean funding to continue to collect data on a
subset of existing sites or less frequently on all sites than is currently supported. The approach
in the interim may not be the best long-term solution, but can help TIME/LTM remain functional
until needed longer term protocols are known and more stable funding identified.
Recommendation 6: Continue to develop the OAR/CAMD Web site to include
not only TIME/LTM data sets in usable formats, but also literature and
documentation.
The efforts of OAR/CAMD to provide a Web site that gives access to data and other resources is
an important evolution in TIME/LTM data management. This effort should continue and be
enhanced by adding comprehensive program information, such as peer-reviewed literature,
studies, and related publications from all contributing sources. Additional information on each
TIME/LTM site and metadata for data records would also be valuable, although potentially
beyond scope without additional funding. Improved access and usability of TIME/LTM data and
research can increase program visibility and are important to preservation of the decades of
data.
Recommendation 7: Based on the outcome of the above recommendations,
determine optimal institutional arrangements for program oversight and data
management.
Consideration of institutional arrangements for TIME/LTM should be based on addressing the
above recommendations. Depending on decisions to refocus the scientific question and evolve
the program with other monitoring efforts, various options may be considered. Ideally, an
infrastructure that supports a collaborative approach across science and policy interests, with
opportunities for partner engagement can evolve. A single office such as a program in OAR may
optimally manage data, but other offices, such as OW may best oversee data collection. ORD
should continue to play a role in determining research needs and monitoring design.
Alternatively, interagency approaches such as supports the NADP could be considered. Another
option is a model such as the National Water Quality Monitoring Council
(http://acwi.gov/monitoring/). These alternatives should be explored with consideration to
approaches to ensure stable funding to support the needs of long-term monitoring.
43
-------
APPENDICES
44
-------
Page Intentionally Blank
45
-------
APPENDIX A: HISTORY AND CONTEXT FOR TIME/LTM
Various issues, efforts, programs, and interests have intersected to contribute to the current
state of TIME/LTM. TIME/LTM crosses several disciplinary lines of interest, including acid
precipitation, acid deposition, ecological conditions, surface water quality, science and policy
interaction, and inter-agency interactions. These multi-disciplinary perspectives were a
foundation of the National Acid Precipitation Assessment Program (NAPAP) that spawned
TIME/LTM. The following timeline provides a brief overview of the events and activities that
brought TIME/LTM to where it is today.
TIME/LTM Timeline
1950s
1960s
1955: The Air Pollution Control Act of 1955 was passed as the first federal legislation
involving air pollution (California had passed the first state air pollution law in 1947).
The 1955 Act mandated that federal research programs investigate the health and
welfare effects of air pollution. This was recognition that air pollutants may have effects
on health and the environment.
1960: The Hubbard Brook (New Hampshire) Ecosystem Study (HBES) was initiated as a
small watershed approach to study element flux and cycling (Exhibit A-l).
1963: The US Forest Service and Dartmouth College establish a cooperative agreement
and the National Science Foundation provides funding for Hubbard Brook studies.
These efforts have had continuous funding ever since.
1963: The Clean Air Act of 1963 was the first federal legislation regarding air pollution
control. It established a federal program within the U.S. Public Health Service and
authorized research into techniques for monitoring and controlling air pollution.
1967: Acid Rain becomes a policy issue based on Swedish scientists calling attention to
the phenomenon at the United Nation's Conference on the Human Environment.
Norway was the first European country to initiate a national program of research
1967: The Air Quality Act was enacted to expand federal government activities,
including enforcement proceedings in areas subject to interstate air pollution transport.
For the first time extensive ambient monitoring studies and stationary source
inspections were conducted by the federal government. Air pollutant mission
inventories, ambient monitoring, and control techniques were authorized.
A-l
-------
State Factors
Tims
Topography
Parent material
Climate
Biota
Ecosystem Functions
& Services
Forest production
"Water quality
"Water quantity
Trace gas fluxes
Soil C storage
Biodiversity maintenance
Landscape diversity
Extreme climate events
Irruptions of native biota
(e.g. pathogens, insect outbreaks, big herbivores)
Geologic events
Lightning fires
j ANTHROPOGENIC
Human-accelerated climate change
Introduction of exotic species
Land use disturbance (e.g. clear-cutting,, agricultu
ir pollution/ atmospheric depositior
Ecosystem
M anagem ent
Exhibit A-l: Initial Concepts of Ecosystem Studies
Source: http://www.hubbardbrook. org/overview/historical_perspective. htm
1970s
1970: The Clean Air Act was enacted, authorizing the development of comprehensive
federal and state regulations to limit emissions from stationary (industrial) sources and
mobile sources of air pollutants. Four major regulatory programs affecting stationary
sources were initiated: the National Ambient Air Quality Standards (NAAQS), State
Implementation Plans (SIPs), New Source Performance Standards (NSPS), and National
Emission Standards for Hazardous Air Pollutants (NESHAPs).
1970: The Environmental Protection Agency was created by Executive Order from
President Nixon, with a key responsibility of implementing the Clean Air Act.
Early 1970s: US scientists assert that lakes in the Adirondack Park and other pristine
areas of upper New York, Vermont, and New Hampshire were suffering from the effects
of "acid rain"
Mid -1970s: Sulfur deposition peaks at Hubbard Brook based on ongoing surface water
quality surveys
1975: The New York State Department of Health and the New York State Department of
Environmental Conservation sample the chemistry of 57 remote lakes in the Adirondack
Mountains.
1977: Major amendments added to the Clean Air Act to address the Prevention of
Significant Deterioration (PSD) of air quality in areas attaining the NAAQS. National goals
were established for carbon monoxide, ozone, particulate matter, oxides of nitrogen,
oxides of sulfur, and lead.
1978: Initiation of the National Atmospheric Deposition Program/National Trends
Network (NADP/NTN) as a nationwide network of precipitation monitoring sites (22
sites in 1978).
A-2
-------
1980s
1980: Vermont Department of Environmental Conservation began monitoring the
chemistry of lakes which later become monitoring in cooperation with EPA through its
LTM Program in 1983.
1980: The Acid Precipitation Act of 1980 (Title VII of the Energy Security Act of 1980, P.L
96-294) established an Interagency Task Force on Acid Precipitation. The Task Force
implemented the National Acid Precipitation Assessment Program (NAPAP) to do the
following:
o Identify the causes and sources of acid precipitation
o Evaluate the environmental, social, and economic effects of acid precipitation, and
(based on the results of the research program) authorize action to the extent
necessary and practicable to (A) limit or eliminate the identified sources of acid
precipitation and (B) remedy or otherwise ameliorate the harmful effects that may
result from acid precipitation.
198238-8339: The LTM Program (LTMP) was established as part of NAPAP by
incorporating sites already under study or planned in the Adirondacks, Vermont, Maine,
Upper Midwest, Rocky Mountains, and Catskill Mountains. The objectives, as listed in
NAPAP 198440 (these changed somewhat in subsequent years) were to:
o "detect and measure long-term chemistry trends of surface waters with low
alkalinity and
o compare the response of low alkalinity waters over a geographic gradient of
hydrogen ion and sulfate deposition"
1983: The Aquatic Effects Task Group (EPA, USGS, USFWS, the Tennessee Valley
Authority, and various state agencies and universities) under NAPAP formed the Aquatic
Effects Research Program (AERP). The AERP was EPA's primary contribution to NAPAP
and examined the effects of acid precipitation on aquatic ecosystems.
1984: As part of the LTM -127 lake and stream sites were monitored in 11 states. The
data were stored in the Acidic Deposition Assessment Data Network (ADDNET) at the
Department of Energy Oak Ridge National Laboratory. Preliminary analyses were
expected to establish baseline conditions of alkalinity.
1984: Also as part of the LTM - studies of the effects of storm and snowmelt events
were initiated in the Catskills, SW PA, the Southern Blue Ridge Province, and the
Ouachita Mountains of Arkansas. Studies were continued on aluminum in acid lakes.
1984-85- The National Surface Water Survey (NSWS) was funded under NAPAP to
address the question of "how many lakes are affected by acidification?" A total of 2075
lakes were selected for sampling in the Eastern US and 752 lakes in the Western US in
1984 and 1985 respectively.
38 Newell, Powers, and Christie. Analysis of Data from Long-Term Monitoring of Lakes. 1987. EPA/600/4-87/014, U.S.
Environmental Protection Agency, Washington, D.C.
39 Aquatic Effects Research Program (AERP) Status, EPA, April 1990. EPA/600/M-90/001
40 National Acid Precipitation Program Annual Report, 1984
A-3
-------
1985: As part of the LTM, 117 lakes and 23 streams in the mountainous West, Upper
Midwest, New England, New York, the Appalachians, and Piedmont Plateau were
monitored. Additionally, the USGS monitored watersheds in Washington, North Dakota,
Wisconsin, and the Appalachians and cooperated with the NPS in studies in Rocky
Mountain NP and Sequoia NP. The NPS conducts watershed scale research in Isle
Royale NP and Olympic NP. The objectives of these efforts were to:
o "detect long term trends in the chemistry and surface waters with low acid
neutralizing capacity, and
o compare the response of low acid neutralizing capacity waters over a geographic
gradient of hydrogen ion and sulfate deposition."41
1986: EPA establishes the National Dry Deposition Network and CASTNET
1987: The Concept of TIME paper is published stating the following: "The purpose of
the TIME project is to design and implement, through NAPAP, a coordinated long-term
monitoring effort that will obviate many of the criticisms associated with environmental
monitoring programs."
1988-91: EPA Episodic Response Project (ERP) was conducted to determine the nature
of episodic changes in stream chemistry and how they affect aquatic biota, especially
fish, in 13 streams in Pennsylvania Appalachians, and Catskills and Adirondacks of New
York. The effort documented for first time in US that episodic stream acidification can
lead to fish mortality.
1989: TIME and LTM projects were transferred from the AERP to the EPA
Environmental Monitoring and Assessment Program (EMAP).
1990: Congress passes Title IV of the Clean Air Act Amendments (Acid Deposition
Control). This Act calls for major reductions of sulfur and dioxides and nitrogen oxides,
the pollutants that cause acid rain. The Act established the Acid Rain Program, which
authorized EPA to limit emissions of these pollutants.
1991: NAPAP releases its Integrated Assessment Report and is reauthorized by Congress
as an open-ended program to continue acid rain research, set targets for emissions
reductions, and make periodic assessments on the effectiveness of these measures. A
total of $530M was expended over the decade of NAPAP work (1980-1990).
1991: TIME program began sampling Northeast lakes
1993: TIME program begins sampling Mid-Atlantic streams
1995: CAAA Title IV Phase I implemented
1995: The Mercury Deposition Network (MDN)was formed to collect weekly samples of
precipitation to monitor the amount of mercury in precipitation on a regional basis;
1997: NAPAP began to operate under the auspices of the Committee on Environment
and Natural Resources (CENR) of the National Science and Technology Council. NAPAP's
goal continued to be providing credible technical findings on acid deposition and its
41 NAPAP Annual Report 1985
A-4
1990s
-------
2000s
effects to inform the public decision-making process. To ensure that this goal was met,
NAPAP coordinated its activities through the Air Quality Research Subcommittee of
CENR.
2000: CAAA Title IV Phase II implemented
2003: ORD issues Response of Surface Water Chemistry to the Clean Air Act
Amendments of 1990, which showed large decreases in sulfate and base cations,
smaller decreases in nitrates, widespread increases in ANC, and almost no changes in
hydrogen ion, concluding that a market-based approach to pollution control is effective.
2005: NAPAP Report to Congress describes small but significant increases in ANC of
surface water in the Upper Midwest, Adirondacks, and northern Appalachians, but
smaller increases in New England (Maine and Vermont).
2007: The decision was made to redefine the scope of NAPAP in advance of the next
report. Parts of previous NAPAP reports essentially duplicate what is already covered in
annual progress reports issued by the Clean Air Markets Division of the Office of
Atmospheric Programs of EPA. These EPA progress reports include annual data on
emissions, air quality and deposition, market indicators (e.g. allowance prices), and
health benefits, as well as information on the status of acid-sensitive lakes and streams
as a result of implementation of Title IV. Future plans call for EPA to continue to issue
these annual reports as a means of reporting progress on new legislation such as the
Clean Air Interstate Rule (CAIR) and Clean Air Visibility Rule (CAVR). In light of these
ongoing EPA reports, a decision was made that future NAPAP reports should focus on
providing an integrated assessment of the effects of acid precipitation on sensitive
ecosystems.
2008: The EPA Office of Research and Development determines that they will stop
supporting the TIME/LTM effort and makes plans to transfer the program to OAR.
2008: OAR requests funding from OPEI to conduct a performance evaluation on
TIME/LTM
2009: TIME/LTM cooperators and EPA conduct a workshop to discuss the future of
TIME/LTM, including the results of the performance evaluation
A-5
-------
APPENDIX B: TIME/LTM PROGRAM EVALUATION METHODOLOGY
Evaluation of EPA's Temporally Integrated Monitoring
of Ecosystems (TIME) and Long-Term Monitoring
(LTM) Programs: Evaluation Methodology
B-l
-------
Table of Contents
Introduction and Purpose B-3
Evaluation Team and Steering Committee B-3
Background B-3
Evaluation Questions B-4
Logic Models B-7
Steps for Conducting the Evaluation B-9
Identify and Review Relevant Documentation B-9
Conduct Interviews with Key Stakeholders B-10
Data Management and Analysis B-13
Prepare Final Evaluation Report B-13
Program Evaluation Timeline B-13
Appendix M-l: Initial Contact Emails Sent to Interviewees B-15
Appendix M-2: Interview Guide B-17
Appendix M-3: Interviewees for TIME/LTM Program Evaluation B-22
B-2
-------
Introduction and Purpose
This document describes the methodology to be used to evaluate the effectiveness of the
Temporally Integrated Monitoring of Ecosystems (TIME) and Long-Term Monitoring (LTM)
programs. The TIME/LTM programs were established to measure changes in ecological
conditions in response to changing air emissions and acid deposition. An evaluation of
TIME/LTM was selected as one of five program evaluations in FY 2009 under EPA's Office of
Policy, Economics and Innovation (OPEI) 2008 Program Evaluation Competition. The evaluation
entails an assessment of program design, implementation, costs, and other factors to determine
TIME/LTM program effectiveness, long-term sustainability, and contributions to knowledge of
ecological conditions affected by acid deposition. Assessment of these programmatic aspects of
this environmental monitoring effort may also help to identify and develop performance
measures for both TIME/LTM and other ecological monitoring programs, to improve monitoring
relevance for environmental protection programs.
EPA's Office of Research and Development (ORD) is transferring the responsibilities for funding
and managing the TIME/LTM programs to the Office of Air and Radiation (OAR). This is expected
to occur by the end of fiscal year 2009. OAR requested funds from OPEI to conduct this
evaluation, in part to address opportunities for improved program effectiveness during and
after this transition. The intended audience for the report and related products is both OPEI and
OAR, but also includes other agencies that partner with EPA to collect and utilize ecological
monitoring data, such as the National Park Service and U.S. Geological Survey. OAR plans to use
the results of the evaluation to assess the extent to which the program is meeting its objectives
and identify opportunities for program improvement. OPEI may use the results to help
structure other evaluations to improve environmental monitoring efforts nationwide.
Evaluation Team and Steering Committee
An Evaluation Team was established that consists of the contractor (Industrial Economics, Inc.
and Ross & Associates Environmental Consulting, Ltd), managers and senior staff from OAR, and
the evaluation lead from OPEI's National Center for Environmental Innovation, Evaluation
Support Division. A Steering Committee was also established to provide broad input over the
course of the evaluation. It is comprised of representatives from OAR, ORD, and other federal
agencies that conduct ecological monitoring, including the National Park Service and U.S.
Geological Survey (USGS).
Background
LTM was initiated in 1983 as part of the National Acid Precipitation Assessment Program
(NAPAP) and specifically focused on sampling sites prone to acid deposition in the Rocky
Mountains, Adirondacks, Catskills, the upper Midwest, Maine, and Vermont. In 1990, Title IV of
the Clean Air Act Amendments (CAAA) set target reductions for sulfur and nitrogen emissions
B-3
-------
from electric utilities to help reduce the acidity of atmospheric deposition. One purpose of such
reductions was to protect ecosystems from acidifying deposition of nitrogen and sulfur. TIME
was established to provide a more random sample of water bodies in the Northeast and Mid-
Atlantic regions to better understand the percentage affected by acidification and changes over
time. Since then the focus of both programs has been on acid-prone water bodies, with
chemical data collected such as acid neutralizing capacity (ANC), pH, sulfates, nitrates, various
cations, some toxic metals, dissolved organic carbon, conductivity, and color.
Data collected as part of TIME/LTM have contributed to documentation of declines in surface
water sulfates in most regions (except Ridge/Blue Ridge Provinces). Acid neutralizing capacity
(ANC), a key indicator of recovery, has been measured as increasing in three of the regions
(Adirondacks, Northern Appalachian Plateau and Upper Midwest). TIME/LTM data have also
documented increased amounts of Dissolved Organic Carbon (DOC). These trends, being seen in
many areas around the world, are considered by scientists to be a function of processes
associated with recovery from acidification or a potential ecological response to climate change.
Other monitoring programs such as the Clean Air Status and Trends Network (CASTNET) and the
National Atmospheric Deposition Program/National Trends Network (NADP/NTN) measure
precipitation chemistry and dry deposition, respectively, providing status and trends of
pollutants being released to the environment. EPA, a primary data user, integrates TIME/LTM
data collected by a network of cooperators from various state agencies, academic institutions,
and other federal agencies with data from CASTNET and NADP to help determine whether
changes in atmospheric emissions affect ecological response.
Evaluation Questions
The following questions were included in the work assignment to form the basis for preliminary
discussions with the core evaluation team and Steering Committee:
Why were the TIME/LTM programs established? Do their program objectives remain
valid? Have the objectives changed over time?
What data/information do TIME/LTM collect, and how do the TIME/LTM data contribute
to, overlap with, and/or enhance the understanding and relevance of other monitoring
efforts?
How are TIME/LTM data accessed? Who uses them and for what purposes (e.g., ecological
research, environmental modeling, program performance measurement)?
What changes, if any (e.g., data parameters, data accessibility, geographic coverage,
timeliness), are needed in TIME/LTM to make the monitoring data more useful to EPA and
other users of the data? Are environmental conditions changing (e.g., climate change)
and if so, how do they affect the relevance and usefulness of TIME/LTM and any changes
that should be made to TIME/LTM?
B-4
-------
Are there now more cost-effective and efficient methods (e.g., new technologies) for
assessing the impacts or obtaining data comparable to or better than those currently
provided by TIME/LTM? What are they?
What is the most effective means within EPA/OAR to administer TIME/LTM? Is the current
approach used by ORD - grants, interagency agreements, and direct funding to EPA labs -
the most effective or are there alternatives such as one contract for monitoring?
The Steering Committee and the evaluation team used these original questions to develop a set
of refined evaluation questions. The refined evaluation questions encompass a number of
programmatic and administrative areas and can be answered with available data (see the
section on Steps for Conducting the Evaluation for description of data sources), including
program objectives, data use and access, cost-effectiveness, and administrative approaches; and
give rise to a number of sub-questions that will be asked of selected interviewees. Details on
interview questions and sub-questions and the individuals selected to address them can be
found in the following section and the interview guide in appendix M-2.
The refined evaluation questions below reflect consideration of program context in relation to
other ecological monitoring programs and also deconstruct TIME/LTM program aspects (data
collection, use, costs, etc.) into sufficient detail for analysis of their contributions to overall
program effectiveness.
Refined Evaluation Questions
1. What is the purpose of the TIME/LTM programs?
a. What are the TIME/LTM objectives?
b. Have the objectives changed over time?
2. What are the key characteristics of the TIME/LTM programs?
3. Who uses TIME/LTM data and for what purposes (e.g., basic research, policy
development)?
4. What is the relationship of TIME/LTM to other ecological monitoring programs?
5. What are the costs associated with TIME/LTM?
a. Are there more cost-effective approaches to data collection and analysis?
b. What other resources does TIME/LTM help to leverage?
6. How are TIME/LTM administered and managed?
7. What opportunities exist to improve TIME/LTM?
1. What is the purpose of the TIME/LTM programs? la. What are the programs' objectives?
Ib. Have program objectives changed over time?
The purpose of this evaluation question is to gain an understanding of TIME/LTM objectives as
initially established, determine whether and in what ways they have evolved over time, and
whether the program objectives remain valid. For example, new questions about emerging
environmental issues, such as climate change, as well as an increasing emphasis on performance
management may affect the relevance and use of the programs' long-term record of
B-5
-------
acidification in freshwater ecosystems. This question will establish context for information
gathered to address evaluation question 7.
2. What are the key characteristics of TIME/LTM?
Information gathered from this evaluation question will be used to describe the major
TIME/LTM program components including contractual agreements and requirements, the
type(s) and number of sites monitored, and sampling frequencies for the data collected.
Information collected from this question complements evaluation question 5 (program costs) by
(1) illustrating the full spectrum of TIME/LTM data collected, on which objective discussion of
program costs can be based, and (2) identifying program strengths and/or weakness made
apparent by assessing regions separately and as part of a monitoring network .
3. Who uses TIME/LTM data and for what purposes (e.g., basic research, policy
development)?
This evaluation question will be used to describe the current and potential users of TIME/LTM
data and the purposes for which they are used. Beyond basic monitoring and its application to
national policy, TIME/LTM may have other uses at the global, regional, state, and local level.
This evaluation question also probes users' access to data; ways in which access could be
expanded; benefits and challenges associated with expanding TIME/LTM data access to a
broader audience.
4. What is the relationship of TIME/LTM to other ecological monitoring programs?
This evaluation question examines the role of TIME/LTM in relation to other ecological
monitoring programs, such as CASTNET and NADP; as part of EMAP and broader ecological
monitoring; and at the regional and state levels.
5. What are the costs associated with TIME/LTM? 5a. Are there more cost-effective
approaches to data collection and analysis? 5b. What other resources does TIME/LTM help to
leverage?
The purpose of this evaluation question is to explore the EPA and non-EPA resources required to
maintain program operations, and perspectives on other approaches to data collection and/or
analysis that could represent cost savings to the program.
6. How are TIME/LTM administered and managed?
A thorough understanding of current TIME/LTM program administration and management is
needed to help identify opportunities for program improvement in these areas, as EPA considers
the transfer of the program from ORD to OAR.
7. What opportunities exist to improve TIME/LTM?
The purpose of this evaluation question is to tie together multiple aspects of program
effectiveness (e.g., meeting objectives, increasing program visibility through data access) and
gather perspectives from a range of stakeholders on various opportunities EPA can consider for
overall program improvement.
B-6
-------
Logic Models
A logic model is a graphical representation of the relationships among program inputs, outputs,
and outcomes (Exhibit 1). A logic model helps to elucidate the components, participants, and
processes that affect a program and provides a key means to understand interactions and
dependencies that are critical to the success of a program evaluation.
Resources: basic inputs of funds, staffing, and knowledge dedicated to the program
Activities: specific processes or results of the inputs needed to achieve program goals
Outputs: immediate products that result from activities and often used to measure short-term
progress
Customers: groups and individuals targeted by TIME/LTM funding and associated activities and
outputs
Short-Term Outcomes: immediate uses of TIME/changes data linked to outputs
Intermediate Outcomes: changes in knowledge and understanding based on use of TIME/LTM
data
Long-Term Outcomes: changes in behavior based on TIME/LTM data; the overarching goals of
the program
Exhibit 1. TIME/LTM Program Logic Model Components
Logic models can contain varying levels of detail, ranging from simple depictions of flows of
information to more detailed nuances of processes and interactions. Exhibit 2 depicts a high-
level model to show the general interactions among TIME/LTM monitoring efforts, other
monitoring data, and expected outcomes. It captures the various customers (e.g., researchers,
scientists, and policy-makers) who have contributed to and utilized TIME/LTM data for over two
decades. A more detailed logic model will be developed as interviews are conducted and the
program evaluation proceeds, to show specific aspects of funding and resources that EPA
commits to TIME/LTM, including how resources are used, by whom, and for what purposes, as
well as how TIME/LTM results are used.
B-7
-------
EXHIBIT 2: HIGH-LEVEL TIME/LTM PROGRAM EVALUATION LOGIC MODEL
Outcomes
Resources
Activities
Outputs
Data Quality
Procedures
l
Data Collection,
Analysis, and
Management
t
Data on
environmental
conditions of
waterbodiesin
acid sensitive
regions
Changes in
EPA/OAR
programs for
managing acid
deposition
B-8
-------
Steps for Conducting the Evaluation
Four major steps are planned to conduct this evaluation, utilizing a number of primary and
secondary sources of information. These steps include: (1) identifying and reviewing relevant
documentation and literature; (2) collecting information from interviews; (3) analysis of data
from documentation and interviews; and (4) preparation of the final evaluation report. Table A
below depicts the basic design of the evaluation research methodology, followed by discussion
of each step in detail.
Table A. TIME/LTM Evaluation
Evaluation Questions |_Dat<
(1) What is the purpose of
the TIME/LTM programs?
(2) What are the key
characteristics of the
programs?
(3) Who uses TIME/LTM data
and for what purposes (e.g.,
basic research, policy
development)?
(4) What is the relationship
of TIME/LTM to other
ecological monitoring
programs? |
(5) What are the costs
associated with TIME/LTM?
(6) How are TIME/LTM
administered and managed?
(7) What opportunities exist
to i m p rove Tl M E/LTM ?
Methodology
Document review and
literature search
Interviews
Document review
Interviews
Document review and
literature search
Interviews
Literature search
Interviews
Document review
Interviews
Document review
Interviews
Analysis, development of
findings and
recommendations
j Data Source(s)
1 TIME/LTM bibliography
ORD-Stoddard
J OAR- Haeuber
1 TIME/LTM bibliography
ORD -Stoddard
OAR -Haeuber
J TIME/LTM cooperators
1 TIME/LTM bibliography, EPA Acid Rain Progress
Reports, NAPAP annual summary
Logic model for TIME/LTM program
ORD -Stoddard, Linthurst
OAR -Haeuber
TIME/LTM cooperators and program managers
I NPS, USGS, CEBC, Data Basin, etc.
Publicly available documents online
ORD, NPS, USGS, CEBC, Data Basin, etc.
Cooperative/interagency agreements, contracts
ORD-Stoddard
J TIME/LTM cooperators and program managers
1 TIME/LTM bibliography
ORD - Stoddard, Washburn, Teichman,
Linthurst
Information collected from 1-6
Discussions with Evaluation Team and Steering
1 Committee
To describe the purpose, objectives, and general program characteristics of TIME/LTM, Ross &
Associates will conduct a literature search and review program publications provided by OAR
and OPEI, and other publicly accessible documents. Key data sources at this stage of the
evaluation include:
B-9
-------
Research articles and other TIME/LTM-based publications from peer-reviewed scientific
journals (1984-2007), provided by OAR and OPEL The primary focus of each article
ranges from trends in surface water chemistry in specific geographic regions,
applications of various models for detecting and predicting changes in analyte levels,
and case studies on the effects of acid deposition on aquatic ecosystems.
NAPAP reports, publicly accessible online. The most recent report (2005) is based on
2002 air emissions data, and uses quantitative and qualitative indicators to assess the
effectiveness of the cap and trade approach to reduce emissions, improve air quality
and reduce acid deposition while minimizing compliance costs. NAPAP also identifies
emerging areas of acid rain research and long-term environmental monitoring.
Data quality reports and annual summaries reported by TIME/LTM cooperators as a
requirement of cooperative agreements with EPA.
Cooperative agreements, interagency agreements, and research proposals.
Recent review articles and other relevant publications found through basic online
literature search, using Google Scholar and University of Washington Library search
engines. Key search terms included TIME/LTM, ecological monitoring, acid rain, and
ecosystem acidification (Note: these searches yielded very few articles outside of the
105 publications sent by OAR and OPEI).
A preliminary analysis of the literature suggests that the following components are critical to the
success of ecological monitoring programs such as TIME/LTM: (1) Clearly articulated program
objectives; (2) maintenance of high quality data through careful documentation of data
collected, collection methods, and measurements; (3) data management systems that are
compatible with long-term data accessibility and optimize use; (4) flexibility/capacity to adapt to
changing research questions and new technologies without compromising core measures of
target analytes or conditions; and (5) sufficient funding base for continuity of data collection,
data management, data use, and publication. These components strongly align with the
evaluation questions and provide fertile ground for the development of detailed interview
questions and sub-questions shown in appendix M-2.
To identify specific uses of TIME/LTM data and the types of policy and research questions they
answer Ross & Associates will conduct telephone interviews (approximately one hour each) with
a number of stakeholders from various program perspectives. The current principal investigators
from each of the six TIME/LTM regions will be interviewed because they comprise the majority
of TIME/LTM data collectors and users and publish regularly on the status and trends in surface
water chemistry of acid-sensitive lakes and streams across the network. ORD contractors,
funded through on-site research support contracts at both TIME/LTM laboratories (Corvallis and
Cincinnati), will be interviewed based on their experience with TIME/LTM data collection and
analysis. Representatives from OAR, ORD, and other federal agencies and nongovernmental
organizations identified by EPA will also be interviewed to gather information on current and
potential data uses, management and administration, and the relationship of TIME/LTM to
other ecological monitoring systems. Table C in appendix M-2 provides a brief description of
stakeholders selected for interviews and reasons for their selection. The focus of each interview
B-10
-------
will vary depending on the extent of interviewees' involvement with TIME/LTM and perspectives
sought (see Tables B-D). Interviews will directly follow the questions in the interview guides;
however, interviewees who skip ahead to other topics/questions will be allowed to continue as
long as all questions are answered during the course of the interview. This approach was chosen
over other types of qualitative interviewing techniques to allow the systematic collection of data
while giving the interviewers flexibility to probe for in-depth responses when necessary. Ross &
Associates will facilitate the interviews to ensure they remain focused and do not exceed the
allocated interview time. Appendix M-2 is the interview guide with detailed interview questions
and sub-questions. Appendix M-3 lists the interviewees selected for this evaluation and the
rationale for selecting them.
TABLE B. Information Gathered in Interviews
Evaluation Question
1. What is the purpose of the TIME/LTM program? .
2. What are the key characteristics of the program?
with TIME/LTM Cooperators and Contractors
Information Sought to Help Answer Evaluation Question
What are the program's goals and objectives? Have they
been consistently communicated across the network
(and documented)?
How have goals changed over time and why?
Perspectives on the validity of the research questions
TIME/LTM currently addresses.
How were TIME/LTM contracts initially established?
What are cooperators required to do or provide to EPA
as contractual requirements?
What data are collected by the program, and what
methods are used for data collection?
3. Who uses TIME/LTM data and for what purposes
(e.g., basic research, policy development)?
What other TIME/LTM data users may exist that
currently do not have access (internal to EPA, external)?
Based on type of users identified, perspectives on ways
in which data/data access can be improved to meet
needs.
5. What are the costs associated with TIME/LTM?
Proportion of funding received from ORD versus other
resources needed to maintain data collection efforts.
Adequacy of funding to regularly carry out data collection
and analysis
Approaches taken by cooperators to leverage other
resources and resulting program benefits (if any).
Ability to implement new technologies or methods for
collecting data.
Overall value of TIME/LTM data for the funding required
to sustain the program.
6. How are TIME/LTM administered and managed?
How data are managed and stored; the extent to which
metadata is captured.
Are there ways to optimize data accessibility and use?
Perspectives on necessity of data hold-back times
Perspectives on ways in which OAR could improve
management/administration of TIME/LTM
7. What opportunities exist to improve TIME/LTM?
Perspectives on various approaches OAR should consider
to improve program effectiveness across a range of
areas (e.g., program design, costs, data access and use).
B-ll
-------
TABLE C. Information Gathered in Interviews with EPA (OAR, ORD, and OW)
I Evaluation Question Information Sought to Help Answer Evaluation Question
1. What is the purpose of the TIME/LTM programs? What are the program's goals and objectives? Are they
consistently communicated across the network (and
documented)?
How have goals changed overtime and why?
Perspectives on the validity of the research questions
TIME/LTM currently addresses.
L>>>>>>>>>>^
3. Who uses TIME/LTM data and for what purposes What other TIME/LTM data users may exist that
(e.g., basic research, policy development)? currently do not have access (internal to EPA, external)?
Based on type of users identified, perspectives on ways
in which data/data access can be improved to meet
needs.
L.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.^^^
4. What is the relationship of TIME/LTM to other The role of TIME/LTM in relation to other EPA ecological
ecological monitoring programs? monitoring programs or aspects thereof (e.g., CASTNET,
NADP, National Aquatic Resource Surveys); how
I I TIME/LTM data are integrated with other program data.
5. What are the costs associated with TIME/LTM? Proportion of funding received from ORD versus other
resources needed to maintain data collection efforts.
EPA perspective on adequacy of funding to regularly
carry out data collection and analysis
Options to improve cost-effectiveness.
Overall value of TIME/LTM data for the funding required
I [ to sustain the program.
6. How are TIME/LTM administered and managed? How data are managed and stored; the extent to which
metadata is captured.
Are there ways to optimize data accessibility and use?
Perspectives on necessity of data hold-back times
Perspectives on ways in which OAR could improve
management/administration of TIME/LTM
7. What opportunities exist to improve TIME/LTM? Perspectives on divisions within OAR that may inherit
TIME/LTM. How will the program be managed, by
whom, and to what extent will ORD continue to be
involved?
Perspectives on various approaches OAR should consider
to improve program effectiveness across a range of
I I areas (e.g., program design, costs, data access and use).
TABLE D. Information Gathered in Interviews with Officials from Other Federal Agencies and
Nongovernmental Organizations
Evaluation Question Information Sought to Help Answer Evaluation Question
3. Who uses TIME/LTM data and for what purposes What other TIME/LTM data users may exist that
(e.g., basic research, policy development)? currently do not have access (internal to EPA,
external)?
Based on type of user, perspectives on ways in which
I I data/data access can be improved to meet needs.
4. What is the relationship of TIME/LTM to other Information on non-EPA ecological monitoring programs
ecological monitoring programs? (including international programs) and what can be
learned from these efforts and applied to TIME/LTM,
with respect to acid deposition.
7. What opportunities exist to improve TIME/LTM? External perspective on benefits and challenges of
expanding data accessibility.
B-12
-------
Approaches for improving data access and using data to
improve program effectiveness and inform decisions
around policy making.
Content analysis of data collected from document and literature review and interviews will be
the general analytic approach used for the evaluation, based on a number of factors. The
evaluation is qualitative and descriptive in nature; different sets of interview questions will be
asked of each stakeholder group resulting in varied responses and perspectives within and
across each group; and interpretation of testimonial data is likely to require a degree of
subjectivity for theme development that is not easily supported by a complex database or
development of quantitative indicators for statistical analysis. Analysis of information will begin
with the first interviews. From comprehensive notes taken during interviews, Ross & Associates
will begin to broadly categorize and summarize responses into common themes using an
appropriate coding scheme that facilitates identification of patterns. The framework for
organizing coded information will consist of a simple spreadsheet or summary document to
catalog themes and corresponding coded passages, providing a record to support findings,
conclusions, and recommendations. As themes and common issues are developed, the core
evaluation team and Steering Committee will meet via teleconference to discuss preliminary
findings, gather input, and identify areas for follow-up. Ross & Associates will develop a report
outline based on preliminary findings and guidance from the core evaluation team and Steering
Committee, from which the find evaluation report will be drafted.
Ross & Associates will synthesize the information collected through background research,
document and literature review, and interviews into a final report that addresses the evaluation
questions and supports fully developed recommendations for approaches to improve the
effectiveness of the TIME/LTM program. The Evaluation Team and Steering Committee will
guide report development and provide direction on areas of focus during periodic conference
calls. For quality assurance, EPA staff, the Steering Committee, and others that provided
information through interviews will have opportunities to review the report for technical
accuracy and completeness. In addition to the evaluation report, a fact sheet and materials for
an oral presentation will be developed as directed by EPA.
Program Evaluation Timeline
The program evaluation will be conducted from November 2008 to May 2009. Table E below
provides a general timeline for collection and analysis of information and preparation of
evaluation products.
B-13
-------
Table E. TIME/LTM Program Evaluation Timeline
Timeframe Activities
Nov. 10-Dec. 12
$ Finalize evaluation questions
$ Develop bibliography database
$ Identify interviewees
$ Continue background research
$ Draft logic models
Dec. 15-Jan. 9
$ Develop program effectiveness criteria
$ Draft evaluation methodology
$ Finalize logic models
$ Complete background research
$ Schedule and begin conducting interviews
Jan. 12-Feb. 20 $ Finalize evaluation methodology
$ Complete remaining interviews
$ Begin to synthesize information from background research and interviews
$ Develop draft report outline
$ Hold teleconference with core evaluation team and Steering Committee to discuss
I information gathered, analytic approaches, and need for follow-up interviews
Feb. 23 - Mar. 13 $ Refine outline based on findings from data collection and input from core EPA team
$ Begin developing initial draft of report
Mar. 16-Apr. 10 $ Schedule and conduct follow-up interviews as necessary
$ Complete draft report
$ Submit draft report to core evaluation team and Steering Committee for review and to
others as needed for technical review
Apr. 13 - May5
$ Refine draft report, solicit and incorporate final comments
$ Develop fact sheet and briefing and/or presentation materials as needed
$ Present report findings as requested
Note -A TIME/LTM Cooperator's Workshop has been scheduled for June 3-4, 2009 at Penn State
University.
B-14
-------
Appendix M-l: Initial Contact Emails Sent to
Interviewees
Message to TIME/LTM Principal Investigators and Cooperators
From: Laroche.David@epamail.epa.gov
Sent: Fri 12/12/2008
Subject: Interview Phase of the TIME/LTM Program Evaluation
Hi Everyone,
Some of you may know, and others not, of a study that EPA is undertaking to assess EPA's
Temporally Integrated Monitoring of Ecosystems and Long Term Monitoring (TIME/LTM)
programs. As you may also know, the administration of the TIME/LTM programs may be
transferred, within EPA, from the Office of Research and Development (ORD), where these
programs have been managed since their inception, to the Office of Air and Radiation (OAR),
one of the principal users of the data generated under the programs. OAR is taking this
opportunity to take a closer look at the programs, understand their value to some OAR core
programs, and assess whether they might be improved or augmented to provide a better
assessment of the impact of reduced deposition on ecosystem conditions.
Planning for this study has been underway for some time and data collection is to begin at the
start of calendar year 2009. We are alerting you to this activity because you will be contacted to
provide information about the programs from your perspective (e.g., how you collect data and
how you use it), as well as any other insights to the programs you want to provide. EPA has
hired Ross & Associates (a contractor) to assist with the project and they will lead the data
collection effort. To give you a better sense of how this project is organized, we have attached a
copy of the project proposal, which includes a series of questions that will provide structure for
the data collection effort.
Our plan is complete the project in early spring 2009 and provide the results to stakeholders in
April. However, in the meantime, we welcome your questions or comments at any time during
the process. Please feel free to call with questions, comments, or suggestions at the number
provided below.
David R. LaRoche
Senior Advisor
U.S. EPA
Office of Air & Radiation 6102-A
Washington, DC 20460
Phone: (202)564-3926
Fax: (202) 564-1327
B-15
-------
Message to TIME/LTM Steering Committee
From: Laroche.David@epamail.epa.gov
Sent: Fri 12/12/2008
Subject: Interview Phase of the TIME/LTM Program Evaluation
Hi Everyone,
I am writing to give you a heads up about the next phase in our program evaluation process. As
you may recall, we have been in the organizing and planning phase of the program evaluation
for the last couple of months. In the next few weeks, our contractor (Ross & Associates of
Seattle, WA) will begin the interview phase of the project and will be contacting you to set up a
time to talk by phone. These conversations will be focused primarily on collecting background
information about the TIME/LTM programs, other ecological monitoring programs you may be
aware of, and other information you may be aware of that would be helpful in our assessment
of the programs.
Once again, thank you for agreeing to help us out and don't hesitate to give me a call if you have
any questions.
David R. LaRoche
Senior Advisor
U.S. EPA
Office of Air & Radiation 6102-A
Washington, DC 20460
Phone: (202)564-3926
Fax: (202) 564-1327
B-16
-------
Appendix M-2: Interview Guide
[Introductions] Thank you for taking the time to talk with us today. We (Ross & Associates) are
working under contract with EPA to help them assess the Temporally Integrated Monitoring of
Ecosystems and Long Term Monitoring (TIME/LTM) programs. As you may know, the
administration of the TIME/LTM programs may be transferred within EPA from the Office of
Research and Development (ORD), where these programs have been managed since their
inception, to the Office of Air and Radiation (OAR). OAR is taking this opportunity to take a
closer look at the programs, understand their value to some OAR core programs, and assess
whether they might be improved or augmented to provide a better assessment of the impact of
reduced deposition on ecosystem conditions.
[For interviewees less familiar with TIME/LTM, the programs collect surface water chemistry
data that is used to assess the ecological response of acid-sensitive watersheds most impacted
by acidic deposition. Both programs are operated cooperatively among state agencies, academic
institutions, EPA, and other federal agencies].
Your perspective on TIME/LTM is critical to this effort. Upon completion of the data collection
and analysis phase of the evaluation, we will compile the results and our conclusions in a report
to EPA, which will be available for you to review. We anticipate that the results of this
evaluation will help EPA better understand the TIME/LTM programs and identify opportunities
for improvement. Finally, we will maintain the confidentiality of your responses to the interview
questions; any data obtained through this interview will be analyzed and reported in aggregate
with other interview data without individual attribution.
Together with information gathered from principal investigators, cooperators, EPA staff, and
others knowledgeable of ecological monitoring programs, we will draft a report to be completed
in early May 2009.
Do you have any questions to ask before we start the interview?
To begin with, please provide us with some background information on your involvement with
TIME/LTM (including length of time with the program, if applicable).
Interview Questions
Evaluation Question 1: What is the purpose of the TIME/LTM program? la. What are the program's
objectives? Ib. Have program objectives changed overtime?
The first set of questions relates to the TIME/LTM programs' goals and objectives.
1. To your knowledge, why was TIME/LTM originally established? [If the interviewee was
involved with monitoring efforts prior to the establishment of LTM or TIME/LTM, ask
them to describe the objectives of that program and any overlap with LTM or
TIME/LTM].
B-17
-------
2. What is your understanding of the programs' current stated objectives?
3. How have program objectives changed over time and why?
4. Would you say that, given the current design of TIME/LTM, it is meeting its objectives?
[If not, why not?]
Evaluation Question 2: What are the key characteristics of the TIME/LTM programs?
The next three questions are about the design of the TIME/LTM programs and the data
collected.
5. To your knowledge, how was the TIME/LTM contract initially established with your
facility? [If the interviewee has no knowledge of the contract origin, ask to follow-up at
a later time when such information can be made available].
6. [For principal investigators and other cooperators] What types (TIME or LTM) of sites
and how many of each do you monitor?
7. What methods do you use to collect data? Do you also do any of the data analysis, or is
it all sent to the Corvallis laboratory?
8. What data do you collect for TIME/LTM? Are there data on other chemical (or physical)
variables you collect in addition to the primary ones (e.g., N, S, ANC, DOC)? [Ask the
interviewee to point to particular publications they have authored in this arena, if
applicable].
Evaluation Question 3: Who uses TIME/LTM data and for what purposes (e.g., basic research, policy
development)?
The next five questions will help us understand the ways in which TIME/LTM data are used
and, potentially, the ways in which certain program aspects such as data access and use,
could be modified to meet current and potential users' needs.
9. To your knowledge, whoboth internal and external to EPAuses the data generated
by TIME/LTM and what do they use the data for (e.g., to support scientific
investigations, evaluate ARP, develop policies)?
10. [For each type of user identified] In what ways can the data be easily improved to meet
current users' needs?
B-18
-------
11. Do you know of other potential users of the data? Are there programs in OAR or the
EPA not currently using the data collected through TIME/LTM that have a need for the
data?
12. To what extent have the TIME/LTM data been used to support the development of new
or modification of existing policy (at the federal, state or local levels)? [If applicable, ask
for documentation].
13. From your perspective, how does TIME/LTM contribute to assessing and understanding
the effectiveness of the Acid Rain Program?
Evaluation Question 4: What is the relationship of TIME/LTM to other ecological monitoring
programs?
The next set of questions pertains to TIME/LTM in relation to other ecological monitoring
efforts at the national, regional, state and local levels.
14. To your knowledge, what is the relationship between the data and information that the
TIME/LTM programs collect and other monitoring efforts/programs? Please tell us about
any state or local monitoring efforts that relate to or otherwise leverage those of
TIME/LTM.
15. Are there surface water chemistry or acidification data collection efforts in other
government (federal or state) agencies you know of that could complement the
TIME/LTM data or broaden the EPA's understanding of pollution deposition? (e.g.,
broaden the understanding over wider geography and/or broaden the understanding of
acidification).
16. To your knowledge, what other ecological monitoring systems (government and
nongovernment?) collect data that would be helpful in expanding/deepening our
understanding of the impact of reduced acidic deposition on ecosystem conditions?
Evaluation Question 5: What are the costs associated with TIME/LTM? 5a. Are there more cost-
effective approaches to data collection and analysis? 5b. What other resources does TIME/LTM help
to leverage?
The following four questions concern TIME/LTM funding sources (from EPA and other
sources), activities supported, and the advantages and disadvantages of current funding
model as compared to other options (e.g., using a contractor).
B-19
-------
17. How much funding does EPA provide to support TIME/LTM and how have these costs
changed over time? How are costs likely to change going forward?
18. What are other the resources (funding or otherwise) and/or in-kind contributions that
TIME/LTM helps to leverage (e.g., PI dollars)?
19. To your knowledge, are there more cost-effective and efficient methods (e.g., new
technologies) for collecting comparable data? [In either case, ask for specific examples
that supports their response]
20. What are the options for contracting with data collectors and for lab analyses that
would cost-effectively gather appropriate information that might serve multiple
program requirements? Are there particular pros and cons to using contractors to
collect and analyze TIME/LTM data versus using the university/state agency model
currently in place?
Evaluation Question 6: How are TIME/LTM administered and managed?
This set of questions asks about the way in which TIME/LTM is currently administered and
managed in EPA's ORD, including the way in which data are housed and accessed and the extent
to which these and other aspects can be improved if the programs migrate to EPA's OAR.
21. Where are the data located? How are the data managed?
22. To what extent are there metadata? How are metadata managed, accessed, searched,
etc?
23. How do users of the TIME/LTM data access the data, and within what time frames are
they given access to the data?
24. Based on your experiences, what other approaches to allocating TIME/LTM resources
should be considered?
Evaluation Question 7: What opportunities exist to improve TIME/LTM?
The last set of questions seeks to tie much of the preceding information together and identify key
factors of the TIME/LTM programs that may contribute to overall program improvement.
25. From your perspective, to what extent are TIME/LTM program objectives being met?
26. How can data access be improved? How will expanding data access to a broader
audience enhance the programs' effectiveness?
B-20
-------
27. Who should/will establish objectives for the programs and on what basis? Does OAR
have different objectives for the programs than ORD and if so, what are they?
28. Based on OAR objectives for TIME/LTM, are there more effective means to administer
TIME/LTM (e.g., as is currently done by ORD - grants, interagency agreements, and
direct funding to EPA labs; one contract)?
29. How should/will OAR manage and change the programs to optimize the usability of the
data being collected?
30. Based on your experiences, what suggestions would you make to improve the
effectiveness of the TIME/LTM programs?
B-21
-------
Appendix M-3: Interviewees for TIME/LTM Program
Evaluation
TIME/LTM Monitoring Network- Principal Investigators
Karen Roy
Michael McHale
Jim Kellogg
Bill McDowell
Steve Kahl
David DeWalle
John Karish
Rick Webb
Adirondack Lakes Survey Corporation
USGS District Office (Catskills)
Vermont Agency of Natural Resources
University of New Hampshire
Plymouth State University (Maine)
Penn State University
NPS
University of Virginia (Blueridge/VA)
TIME/LTM cooperators and contractors collect
and use the data and form the monitoring
network itself. As principal investigators, they
publish regularly on the status and trends in
surface water chemistry of acid-sensitive lakes
and streams across the network. In addition to
monitoring, their research contributes to
understanding broader aspects of the effects of
pollution on ecosystem health. Principal
investigators and cooperators receive a
significant portion of funding to support
TIME/LTM data collection and analysis from
ORD, but may also leverage other sources of
funding to keep region-specific programs viable.
EPA/OAR -Potential Program Managers
Rick Haeuber
Michael Kolian
Jerry Kurtzweg
OAR, Clean Air Markets Division
OAR, Clean Air Markets Division
EPA OAR, OPMO
As OAR prepares to inherit TIME/LTM from ORD,
perspectives on the management of the
programs and capacity for OAR to effectively
manage, administer, and address current
program challenges are critical.
EPA/ORD - Current Program Managers and Historians
John Stoddard
Michael Moeykens
Dave Peck
Ed Washburn
Kevin Teichman
Rick Linthurst
Western Ecology Division
Work assignment manager (sample
collection) Cincinnati, OH laboratory
Work assignment manager (sample
analysis), Corvallis, OR laboratory
Branch Chief, Office of Science Policy
Director, Office of Science Policy
National Program Director for Ecology
ORD has provided most of the funding for
TIME/LTM since the programs' inception. ORD
has been responsible for overall quality
assurance, database management, and
verification and validation of data, ensuring that
data are comparable across regions and through
time. ORD officials have thorough knowledge of
TIME/LTM's history.
Monitoring Experts (various agencies)
Susan Holdsworth
MarkNilles
Other Agencies and C
Chris Shaver
Tosha Comendant
EPA, Office of Water (OWOW) Branch
Chief, Monitoring
Office of Water Quality, USGS
OWOW conducts EPA's National Aquatic
Resource Surveys, statistically-representative
surveys of U.S. aquatic resources designed to
identify national priorities and evaluate the
effectiveness of pollution control actions. USGS
studies effects of acidification on undeveloped
watersheds. NPS and USGS, as part of the NADP,
coordinate efforts to better understand the
effects of acid deposition on ecosystem health.
trganizations - Potential TIME/LTM Users
Director, Air Resources Division,
National Park Service
Conservation Biology Institute
NPS and USGS, as part of the NADP, coordinate
efforts to better understand the effects of acid
deposition on ecosystem health. NPS uses
TIME/LTM to assess water quality in Shenandoah
National Park.
Dr. Comendant developed Data Basin, an
B-22
-------
innovative tool that provides access to data
about complex ecological, political, and economic
systems, and fosters collaboration between
policymakers, conservationists, activists, and
scientists by facilitating the exchange of
environmental information.
Christine Negra
Environmental Reporting Program,
Heinz Center
The Environmental Reporting Program publishes
The State of the Nation's Ecosystems: Measuring
the Lands, Waters, and Living Resources of the
United States.
B-23
-------
APPENDIX B Continued: TIME/LTM Program Quality
Assurance Plan
Quality Assurance Plan for EPA Contract 68-W-07-028,
Work Assignment B-28
Supporting EPA's Program Evaluation on Temporally
Integrated Monitoring of Ecosystems and Long-Term
Monitoring (TIME/LTM) Programs
B-24
-------
Purpose of the Evaluation:
An evaluation of TIME/LTM was selected as one of five program evaluations in FY 2009 under
EPA's Office of Policy, Economics and Innovation (OPEI) Evaluation Support Division (ESD) 2008
Program Evaluation Competition.42 The evaluation entails an assessment of program design,
implementation, costs, and other factors to determine TIME/LTM program effectiveness, long-
term sustainability, and contributions to knowledge of ecological conditions affected by acid
deposition. Assessment of these programmatic aspects of this environmental monitoring effort
may also help to identify and develop performance measures for both TIME/LTM and other
ecological monitoring programs, to improve monitoring relevance for environmental protection
programs. Details of the evaluation approach can be found in the "Evaluation Methodology" for
this project.
Design:
Ross & Associates designed its data collection and analysis approach in consultation with
OPEI/ESD and OAR. These individuals comprise a core evaluation team. The approach is
primarily based on use of publications and interviews of individuals involved in the conduct of
TIME/LTM or in the use of TIME/LTM data. A logic model to show relationships among inputs,
activities, outputs, customers, and outcomes was developed, as were interview questions. Both
the model and evaluation questions are outlined in detail in the "Evaluation Methodology." The
general approach to analysis will be qualitative, based on responses to the questions and
information that can be derived from review of publications and reports. Content analysis will
begin with the first interviews. The core evaluation team and others who provided information
through interviews will have opportunities to review the final summary of data for technical
accuracy and completeness.
Rationale:
This evaluation does not lend itself to a quantitative analysis given the nature of the program.
Data Sources:
Key data sources to be used in the evaluation include:
Primary Data Sources:
o Interviews with TIME/LTM principal investigators and contractors, stakeholders
from OAR and ORD, and stakeholders from other federal agencies (e.g., U.S.
Geological Survey, National Park Service).
Secondary Data Sources:
42 ESD's mission is to enable its partners to more effectively conduct program evaluations and analyses that inform
management decisions, enhance organizational learning, promote innovation and foster environmental results. As
part of the effort to encourage the effective use of program evaluations throughout the Agency, ESD promotes
program evaluation through a Program Evaluation Competition (PEC). The competition is part of an ongoing, long-
term effort to build capacity in EPA headquarters and regional offices to evaluate activities and to improve measures
of program performance. For the 2008 PEC, evaluation of the TIME/LTM Programs was chosen for support.
B-25
-------
o Articles and other publications from peer-reviewed scientific journals (1984-
2007) based on the use of TIME/LTM data. These reports are derived from:
ORD, OAR, and OPEI
Online literature searches
State and agency websites citing TIME/LTM as primary data sources
References provided by interviewees
o Reports from the National Acid Precipitation Assessment Program
o Cooperative and interagency agreements and research proposals
Consistency:
A structured interview approach is described in the "Evaluation Methodology" with different
questions for different interviewees, depending on expertise and involvement with TIME/LTM.
From comprehensive notes taken during interviews, Ross & Associates will categorize and
summarize responses to identify patterns. Given the relatively small number of individuals, all
with significant variation in experience with TIME/LTM (e.g., EPA personnel overseeing contracts
or laboratory analysis, EPA program managers using data, academics collecting data, scientists
from academic and federal agencies interpreting data) every attempt will be made to validate
statements and information provided. New information derived in later interviews may be
corroborated with earlier interviewees or with individuals with more years of program
experience to ensure that results are accurately recorded.
Data Limitations:
This evaluation is based on analysis of qualitative information and will therefore be limited by
accurate characterization and summarization of data collected be from the different stakeholder
groups (e.g., data collectors, data consumers, others).
Audience:
The final evaluation report will be useful to a variety of stakeholders, including: OPEI, ORD, and
OAR program managers and administrators; TIME/LTM principal investigators and their staff,
and data consumers from other federal and state agencies.
EPA Office: OPEI
EPA Project Leaders: Matt Keene, OPEI; David LaRoche, OAR
EPA Quality Manager: Matt Keene, OPEI
Ross & Associates Evaluators: Nancy Tosta, Jennifer Major
Ross & Associates Quality Manager: Tim Larson
B-26
-------
APPENDIX C: PUBLICATIONS DRAWING ON TIME/LTM DATA,
1985-2007
1. Aber, J. D., C. L Goodale, S. V. Ollinger, M.-L Smith, A. H. Magil, M. E. Martin, R. A.
Hallett, and J. L. Stoddard. 2003. Is nitrogen deposition altering the nitrogen status of
Northeastern forests? BioScience 53:375-389.
2. Baker, L. A., P. R. Kaufmann, A. T. Herlihy, and J. M. Eilers. 1990. Current Status of
Surface Water Acid-Base Chemistry. NAPAP State-of-Science/Technology Report No. 9,
National Acid Precipitation Assessment Program, Washington, DC.
3. Barker, J. L, and E. C. Witt. 1990. Effects of acidic precipitation on the water quality of
streams in the Laurel Hill area, Somerset County, Pennsylvania, 1983-1986. Water
Resources Investigations Report 89-4113.
4. Burns, D. A., M. R. McHale, C. T. Driscoll, and K. M. Roy. 2006. Response of surface water
chemistry to reduced levels of acid precipitation: comparison of trends in two regions of
New York, USA. Hydrological Processes 20:1611-1627.
5. Campbell, J. L, J. W. Hornbeck, M. J. Mitchell, M. B. Adams, M. S. Castro, C. T. Driscoll, J.
S. Kahl, J. N. Kochenderfer, G. E. Likens, J. A. Lynch, P. S. Murdoch, S. J. Nelson, and J. B.
Shanley. 2004. Input-output budgets of inorganic nitrogen for 24 forest watersheds in
the northeastern United States: A review. Water Air and Soil Pollution 151:373-396.
6. Canham, C. D., M. L. Pace, M. M. Papaik, A. G. B. Primack, K. M. Roy, R. J. Maranger, R. P.
Curran, and D. M. Spada. 2004. A spatially explicit watershed-scale analysis of dissolved
organic carbon in Adirondack lakes. Ecological Applications 14: 839-854.
7. Chen, L., and C. T. Driscoll. 2004. An evaluation of processes regulating spatial and
temporal patterns in lake sulfate in the Adirondack region of New York. Global
Biogeochemical Cycles 18:GB3024.
8. Chen, L., and C. T. Driscoll. 2005. Regional assessment of the response of the acid-base
status of lake watersheds in the Adirondack Region of New York to changes in
atmospheric deposition using PnET-BGC. Environmental Science and Technology 39:787-
794.
9. Chen, L. M., and C. T. Driscoll. 2004. Modeling the response of soil and surface waters in
the Adirondack and Catskill regions of NewYork to changes in atmospheric deposition
and historical land disturbance. Atmospheric Environment 38:4099-4109.
10. Chen, L. M., C. T. Driscoll, S. Gbondo-Tugbawa, M. J. Mitchell, and P. S. Murdoch. 2004.
The application of an integrated biogeochemical model (PnET-BGC) to five forested
watersheds in the Adirondack and Catskill regions of New York. Hydrologic Processes
18:2631-2650.
11. Civerolo, K., E. Brankov, S. Trivikrama Rao, K. Roy, P. Lewis, and P. Galvin. 2003. Analysis
of ambient, precipitation-weighted, and lake sulfate concentrations in the Adirondack
region of New York. Environmental Pollution 123:337-345.
C-l
-------
12. Cook, R. B., and H. I. Jager. 1991. Upper Midwest. Pages 421-466 in D. F. Charles, editor.
Acidic Deposition and Aquatic Ecosystems: Regional Case Studies. Springer-Verlag, New
York.
13. Davies, T. D., M. Tranter, P. J. Wigington, K. N. Eshleman, N. E. Peters, J. Van Sickle, D. R.
DeWalle, and P. S. Murdoch. 1999. Prediction of episodic acidification in North-eastern
USA: an empirical mechanistic approach. Hydrological Processes 13:1181-1195.
14. Deviney, F. A., K. C. Rice, and G. M. Hornberger. 2006. Time series and recurrence
interval models to predict the vulnerability of streams to episodic acidification in
Shenandoah National Park, Virginia. Water Resources Research 42:1-14.
15. DeWalle, D. R., A. R. Buda, J. A. Eismeier, W. E. Sharpe, B. R. Swistock, P. L Craig, and M.
O'Driscoll. 2005. Nitrogen cycling on five headwater forested catchments in Mid-
Appalachians of Pennsylvania. Pages 29-36 in L. Heathwaite, B. Webb, D. Rosenberry, D.
Weaver, and M. Hayash, editors. Dynamics and Biogeochemistry of River Corridors and
Wetlands. International Association of Hydrological Sciences.
16. DeWalle, D. R., and B. R. Swistock. 1994. Causes of episodic acidification in five
Pennsylvania streams on the northern Appalachian Plateau. Water Resources Research
30:1955-1963.
17. DeWalle, D. R., B. R. Swistock, and W. E. Sharpe. 1995. Episodic flow-duration analysis: a
method of assessing toxic exposure of brook trout (Salvelinus fontinalis) to episodic
increases in aluminum. Canadian Journal of Fisheries and Aquatic Science 52:816-827.
18. Driscoll, C. T., K. M. Driscoll, M. J. Mitchell, and D. J. Raynal. 2003. Effects of acidic
deposition on forest and aquatic ecosystems in New York State. Environmental Pollution
123:327-336.
19. Driscoll, C. T., K. M. Driscoll, K. M. Roy, and J. Dukett. 2007. Changes in the chemistry of
lakes in the Adirondack region of New York following declines in acidic deposition.
Applied Geochemistry 22:1181-1188.
20. Driscoll, C. T., K. M. Driscoll, K. M. Roy, and M. J. Mitchell. 2003. Chemical response of
lakes in the Adirondack region of New York to declines in acidic deposition.
Environmental Science and Technology 37:2036-2042.
21. Driscoll, C. T., G. B. Lawrence, A. J. Bulger, T. J. Butler, C. S. Cronan, C. Eager, K. F.
Lambert, G. E. Likens, J. L. Stoddard, and K. C. Weathers. 2001. Acid Rain Revisited:
advances in scientific understanding since the passage of the 1970 and 1990 Clean Air
Act Amendments. Science Links Publication Vol. 1, no. 1, Hubbard Brook Research
Foundation.
22. Driscoll, C. T., G. B. Lawrence, A. J. Bulger, T. J. Butler, C. S. Cronan, C. Eager, K. F.
Lambert, G. E. Likens, J. L. Stoddard, and K. C. Weathers. 2001. Acidic deposition in the
northeastern United States: Sources and inputs, ecosystem effects and management
strategies. BioScience 51:180-198.
23. Driscoll, C. T., G. B. Lawrence, A. J. Bulger, T. J. Butler, C. S. Cronan, C. Eager, K. F.
Lambert, G. E. Likens, J. L. Stoddard, and K. C. Weathers. 2004. Acidic deposition in the
northeastern United States: Sources and inputs, ecosystem effects and management
strategies. Pages 159-190 in]. M. Gunn, R. J. Steedman, and R. A. Ryder, editors. Boreal
Shield Watersheds: Lake Trout Ecosystems in a Changing Environment. Lewis Publishers,
Boca Raton, FL.
C-2
-------
24. Driscoll, C. T., G. E. Likens, and M. R. Church. 1998. Recovery of surface waters in the
northeastern U.S. from decreases in atmospheric deposition of sulfur. Water Air and Soil
Pollution 105:319-329.
25. Driscoll, C. T., and R. M. Newton. 1985. Chemical characteristics of Adirondack lakes.
Environmental Science and Technology 19:1018-1024.
26. Driscoll, C. T., R. M. Newton, C. P. Gubala, J. P. Baker, and S. W. Christensen. 1991.
Adirondack Mountains. Pages 133-202 in D. F. Charles, editor. Acidic Deposition and
Aquatic Ecosystems: Regional Case Studies. Springer-Verlag, New York, NY.
27. Driscoll, C. T., K. M. Postek, W. Kretser, and D. J. Raynal. 1995. Long-term trends in the
chemistry of precipitation and lake water in the Adirondack region of New York, USA.
Water Air and Soil Pollution 85:583-588.
28. Driscoll, C. T., and G. D. Schafran. 1984. Short-term changes in the base neutralizing
capacity of an acid Adirondack lake, New York. Nature 310: 308-310.
29. Driscoll, C. T., and R. Van Dreason. 1993. Seasonal and long-term temporal patterns in
the chemistry of Adirondack lakes. Water Air and Soil Pollution 67:319-344.
30. Dupont, J., T. A. Clair, C. Gagnon, D. S. Jeffries, J. S. Kahl, S. J. Nelson, and J. M.
Peckenham. 2005. Estimation of critical loads of acidity for lakes in northeastern United
States and Eastern Canada. Environmental Monitoring and Assessment 109:275-291.
31. Evans, C., T. D. Davies, and P. S. Murdoch. 1999. Component flow processes at four
streams in the Catskill Mountains, New York, analysed using episodic
concentration/discharge relationships. Hydrological Processes 13:563-575.
32. Fenn, M. E., M. A. Poth, J. D. Aber, J. S. Baron, B. T. Bormann, D. W. Johnson, A. D.
Lemly, S. G. McNulty, D. F. Ryan, and R. Stottlemyer. 1998. Nitrogen excess in North
American ecosystems: predisposing factors, ecosystem responses, and management
strategies. Ecological Applications 8:706-733.
33. Ford, J., J. L. Stoddard, and C. F. Powers. 1993. Perspectives in environmental
monitoring: an introduction to the U.S. EPA Long-Term Monitoring (LTM) project. Water
Air and Soil Pollution 67:247-255.
34. Goodale, C. L., J. D. Aber, P. M. Vitousek, and W. H. McDowell. 2005. Long-term
decreases in stream nitrate: Succesional causes unlikely; Possible links to DOC?
Ecosystems 8:334-337.
35. Herlihy, A. T., P. R. Kaufmann, J. L. Stoddard, K. N. Eshleman, and A. J. Bulger. 1997.
Effects of Acidic Deposition on Aquatic Resources in the Southern Appalachians with a
Special Focus on Class I Wilderness Areas. Southern Appalachian Mountain Initiative
(SAMI).
36. Ito, M., M. J. Mitchell, C. T. Driscoll, and K. M. Roy. 2005. Factors affecting acid
neutralizing capacity in the Adirondack region of New York: A solute mass balance
approach. Environmental Science and Technology 39:4076-4081.
37. Ito, M., M. J. Mitchell, C. T. Driscoll, and K. M. Roy. 2005. Nitrogen input-output budgets
for lake-containing watersheds in the Adirondack region of New York. Biogeochemistry
72:283-314.
38. Kahl, J. S., S. J. Nelson, J. L. Stoddard, S. A. Norton, and T. A. Haines. 2004. Lakewater
chemistry at Acadia National Park, Maine, in response to declining acidic deposition.
Pages 314-321 in D. Harmon, B. M. Kilgore, and G. E. Vietzke, editors. Response to
C-3
-------
Declining Acidic Deposition, in Protecting our Diverse Heritage: The Role of Parks,
Protected Areas, and Cultural Sites. Proceedings of the George Wright Society/National
Park Service Joint Conference, April 14-18, 2003, San Diego, CA. The George Wright
Society, Hancock, Ml.
39. Kahl, J. S., S. A. Norton, C. A. Cronan, I. J. Fernandez, L C. Bacon, and T. A. Haines. 1991.
Maine. Pages 203-235 in D. F. Charles, editor. Acidic Deposition and Aquatic Ecosystems:
Regional Case Studies. Springer-Verlag, New York, NY.
40. Kahl, J. S., S. A. Norton, T. A. Haines, and R. B. Davis. 1993. Recent trends in the acid-
base status of surface waters in Maine, U.S.A. Water Air and Soil Pollution 67:281-300.
41. Kahl, J. S., J. L. Stoddard, R. Haueber, S. G. Paulsen, R. Birnbaum, F. A. Deviney, J. R.
Webb, D. R. DeWalle, W. Sharpe, C. T. Driscoll, A. T. Herlihy, J. H. Kellogg, P. S. Murdoch,
K. Roy, K. E. Webster, and N. S. Urquhart. 2004. Have U.S. surface waters responded to
the Clean Air Act Amendments. Environmental Science and Technology 38:485A-490A.
42. Lawrence, G. B., M. B. David, G. M. Lovett, P. S. Murdoch, D. A. Burns, J. L. Stoddard, B.
P. Baldigo, J. H. Porter, and A. W. Thompson. 1999. Soil calcium status and the response
of stream chemistry to changing acidic deposition rates. Ecological Applications 9:1059-
1072.
43. Liikewille, A., J. Jeffries, M. Johannessen, G. Raddum, J. Stoddard, and T. Traaen. 1997.
The Nine Year Report: Acidification of Surface Water in Europe and North America.
Long-term Developments (1980s and 1990s), Report O-86001. O-86001, Norwegian
Institute for Water Research, Oslo, Norway.
44. Mitchell, M. J., C. T. Driscoll, J. S. Kahl, G. E. Likens, P. S. Murdoch, and L. H. Pardo. 1996.
Climatic control of nitrate loss from forested watersheds in the northeast United States.
Environmental Science and Technology 30:2609-2612.
45. Mitchell, M. J., P. J. McHale, S. Inamdar, and D. J. Raynal. 2001. Role of within-lake
processes and hydrobiogeochemical changes over 16 years in a watershed in the
Adirondack Mountains of New York State, USA. Hydrological Processes 15:1951-1965.
46. Monteith, D. T., J. L. Stoddard, C. D. Evans, H. A. de Wit, M. Forsius, T. H0gasen, D. S.
Jeffries, A. Wilander, B. L. Skjelkvale, J. Vuorenmaa, J. Kopacek, and J. Vesely. 2007.
Dissolved organic carbon trends resulting from changes in atmospheric deposition
chemistry. Nature 450:537-540.
47. Moore, M. V., M. L. Pace, J. R. Mather, P. S. Murdoch, R. W. Howarth, C. L. Folt, C. Y.
Chen, H. F. Hemond, P. A. Flebbe, and C. T. Driscoll. 1997. Potential effects of climate
change on freshwater ecosystems of the New England/Mid-Atlantic Region. Hydrological
Processes 11:925-947.
48. Morrison, M. 1991. Quality Assurance Plan for the Long-Term Monitoring Project. Pages
1.1-B.l in U.S. Environmental Protection Agency, editor. Data User's Guide to the United
States Environmental Protection Agency's Long-Term Monitoring Project: Quality
Assurance Plan and Data Dictionary. U.S. Environmental Protection Agency, Corvallis,
OR.
49. Murdoch, P. S. 1988. Chemical Budgets and Stream-Chemistry Dynamics at Biscuit
Brook, Catskill Mountains, New York, 1984-85. Water Resources Investigations Report
88-4035.
C-4
-------
50. Murdoch, P. S., J. S. Baron, and T. L Miller. 2000. Potential effects of climate chance on
surface-water quality in North America. Journal of the American Water Resources
Association 36:347-366.
51. Murdoch, P. S., D. A. Burns, and G. B. Lawrence. 1998. Relation of climate change to the
acidification of surface waters by nitrogen deposition. Environmental Science and
Technology 32:1642-1647.
52. Murdoch, P. S., and J. B. Shanley. 2006. Detection of water-quality trends at high,
median, and low-flow in a Catskill Mountain stream, New York, USA, through a new
statistical method. Water Resources Research 42:8407-8417.
53. Murdoch, P. S., and J. B. Shanley. 2006. Flow-specific trends in river-water quality
resulting from the effects of the clean air act in three mesoscale, forested river basins in
the northeastern United States through 2002. Environmental Monitoring and
Assessment 120:1-25.
54. Murdoch, P. S., and J. L. Stoddard. 1992. The role of nitrate in the acidification of
streams in the Catskill Mountain of New York. Water Resources Research 28:2707-2720.
55. Newell, A. D. 1987. Predicting spring lake chemistry from fall samples. Pages 353-363 in
R. Perry, R. M. Harrison, J. N. B. Bell, and J. N. Lester, editors. Acid Rain: Scientific and
Technical Advances. Selper Ltd., London.
56. Newell, A. D. 1993. Inter-regional comparison of patterns and trends in surface water
acidification across the United States. Water Air and Soil Pollution 67:257-280.
57. Newell, A. D., D. J. Blick, and R. C. Hjort. 1993. Testing for trends when there are changes
in methods. Water Air and Soil Pollution 67:457-468.
58. Newell, A. D., and M. L. Morrison. 1993. Use of overlap studies to evaluate method
changes in water chemistry protocols. Water Air and Soil Pollution 67:433-456.
59. Newell, A. D., C. F. Powers, and S. J. Christie. 1987. Analysis of Data from Long-term
Monitoring of Lakes. U.S. Environmental Protection Agency, Corvallis, OR.
60. Newell, A. D., and B. L. Skjelkvale. 1997. Acidification trends in surface waters in the
International Program on Acidification of Rivers and lakes. Water Air and Soil Pollution
93:27-57.
61. Rice, K. C., J. G. Chanat, G. M. Hornberger, and J. R. Webb. 2004. Interpretation of
concentration-discharge patterns in acid-neutralizing capacity during storm flow in three
small, forested catchments in Shenandoah National Park, Virginia. Water Resources
Research 40:5301-5310.
62. Schaefer, D. A., and C. T. Driscoll. 1992. Reply. Water Resources Research 28:2875-2878.
63. Schaefer, D. A., and C. T. Driscoll. 1993. Identifying sources of snowmelt acidification
with a watershed mixing model. Water Air and Soil Pollution 67:345-365.
64. Schaefer, D. A., C. T. Driscoll, R. S. Van Dreason, and C. P. Yatsko. 1990. The episodic
acidification of Adirondack Lakes during snowmelt. Water Resources Research 26:1639-
2099.
65. Schafran, G. C., and C. T. Driscoll. 1987. Comparison of terrestrial and hypolimnetic
sediment generation of acid neutralizing capacity for an acidic Adirondack lake.
Environmental Science and Technology 21:988-993.
66. Schafran, G. C., and C. T. Driscoll. 1993. Flow path-composition relationships for
groundwater entering an acidic lake. Water Resources Research 29:145-154.
C-5
-------
67. Simonin, H. A., J. R. Colquhoun, E. A. Paul, J. Symula, and H. J. Dean. 2005. Have
Adirondack stream fish populations changed in response to decreases in sulfate
deposition? Transactions of the American Fisheries Society 134:338-345.
68. Skjelkvale, B. L, editor. 2004. The 15-year report: Assessment and monitoring of surface
waters in Europe and North America; acidification and recovery, dynamic modelling and
heavy metals. Norwegian Institute for Water Research, Oslo.
69. Skjelkvale, B. L, A. D. Newell, G. Raddum, M. Johannessen, H. Hovind, T. Tjomsland, and
B. M. Wathne. 1994. The six year report: Acidification of surface waters in Europe and
North America. Dose/response relationships and long-term trends, Report 3041-94.
Norwegian Institute for Water Research, Oslo, Norway.
70. Skjelkvale, B. L, J. L. Stoddard, and T. Andersen. 2001. Trends in surface water
acidification in Europe and North America (1989-1998). Water Air and Soil Pollution
130:787-792.
71. Skjelkvale, B. L, J. L. Stoddard, D. Jeffries, K. T0rseth, T. H0gasen, J. Bowman, J. Mannio,
D. Monteith, R. Mosello, M. Rogora, D. Rzychon, J. Vesely, J. Wieting, A. Wilander, and
A. Worsztynowicz. 2005. Regional scale evidence for improvements in surface water
chemistry 1990-2001. Environmental Pollution 137:165-176.
72. Skjelkvale, B. L, J. L. Stoddard, D. S. Jeffries, K. T0rseth, T. H0gasen, J. Bowman, I. Licsko,
I. Lyulko, J. Mannio, D. Monteith, R. Mosello, M. Rogora, D. Rzychon, A. Srybny, R.
Talkop, J. Vesely, J. Wieting, A. Wilander, and A. Worstztynowicz. 2003. Trends in
surface water chemistry 1990-2001.in B. L. Skjelkvale, editor. The 15-Year Report:
Assessment and Monitoring of Surface Waters in Europe and North America;
Acidification and Recovery, Dynamic Modeling and Heavy Metals. Norwegian Institute
for Water Research, Oslo, Norway.
73. Stoddard, J. L. 1990. Plan for Converting the NAPAP Aquatic Effects Long-Term
Monitoring (LTM) Project to the Temporally Integrated Monitoring of Ecosystems (TIME)
Project. Internatl Report, U.S. Environmental Protection Agency, Corvallis, OR.
74. Stoddard, J. L. 1994. Long-term changes in watershed retention of nitrogen: its causes
and aquatic consequences. Pages 223-284 in L. A. Baker, editor. Environmental
Chemistry of Lakes and Reservoirs. American Chemical Society, Washington, DC.
75. Stoddard, J. L. 2004. Use of ecological regions in aquatic assessments of ecological
condition. Environmental Management 34:S61-S70.
76. Stoddard, J. L., C. T. Driscoll, J. S. Kahl, and J. Kellogg. 1998. A regional analysis of lake
acidification trends for the northeastern U.S., 1982-1994. Environmental Monitoring
and Assessment 51:399-413.
77. Stoddard, J. L, C. T. Driscoll, S. Kahl, and J. Kellogg. 1998. Can site-specific trends be
extrapolated to a region? An acidification example for the Northeast. Ecological
Applications 8:288-299.
78. Stoddard, J. L, D. S. Jeffries, A. Lukewille, T. A. Clair, P. J. Dillon, C. T. Driscoll, M. Forsius,
M. Johannessen, J. S. Kahl, J. H. Kellogg, A. Kemp, J. Mannio, D. Monteith, P. S. Murdoch,
S. Patrick, A. Rebsdorf, B. L. Skjelkvale, M. Stainton, T. Traaen, H. van Dam, K. E.
Webster, J. Wieting, and A. Wilander. 1999. Regional trends in aquatic recovery from
acidification in North America and Europe. Nature 401:575-578.
C-6
-------
79. Stoddard, J. L, J. S. Kahl, F. A. Deviney, D. R. DeWalle, C. T. Driscoll, A. T. Herlihy, J. H.
Kellogg, P. S. Murdoch, J. R. Webb, and K. E. Webster. 2003. Response of surface water
chemistry to the Clean Air Act Amendments of 1990. EPA/620/R-03/001, U.S.
Environmental Protection Agency, Washington, DC.
80. Stoddard, J. L., and J. H. Kellogg. 1993. Trends and patterns in lake acidification in the
state of Vermont: evidence from the Long-Term Monitoring project. Water Air and Soil
Pollution 67:301-317.
81. Stoddard, J. L, and P. S. Murdoch. 1991. Catskill Mountains. Pages 237-271 in D. F.
Charles, editor. Acidic Deposition and Aquatic Ecosystems: Regional Case Studies.
Springer-Verlag, New York, NY.
82. Stoddard, J. L, A. D. Newell, N. S. Urquhart, and D. Kugler. 1996. The TIME project
design: II. Detection of regional acidification trends. Water Resources Research
32:2529-2538.
83. Stoddard, J. L., and T. S. Traaen. 1995. The stages of nitrogen saturation: Classification
of catchments included in "ICP on waters". Pages 69-76 in Mapping and modelling of
critical loads for nitrogen - a workshop report. U.K. Department of the Environment,
Grange-over-Sands, Cumbria, U.K.
84. Stoddard, J. L, T. S. Traaen, B. L. Skjelkavale, and M. Johannessen. 2001. Assessment of
nitrogen leaching at ICP-Waters sites (Europe and North America). Water Air and Soil
Pollution 130:781-786.
85. Sullivan, T. J. 1990. Historical Changes in Surface Water Acid-Base Chemistry in
Response to Acidic Deposition. NAPAP State of Science and Technology Report 11,
National Acid Precipitation Assessment Program.
86. Traaen, T. S., and J. L. Stoddard. 1995. An Assessment of Nitrogen Leaching from
Watersheds included in ICP on Waters. Report No. 86001, Norwegian Institute for Water
Research.
87. Turk, J. T., D. H. Campbell, and N. E. Spahr. 1993. Use of chemistry and stable sulfur
isotopes to determine sources of trends in sulfate of Colorado lakes. Water Air and Soil
Pollution 67:415-431.
88. Turk, J. T., and N. E. Spahr. 1991. Controls on the chemistry of Rocky Mountain lakes.
Pages 471-501 in D. F. Charles, editor. Acidic Deposition and Aquatic Ecosystems:
Regional Case Studies. Springer-Verlag, New York.
89. U.S. Environmental Protection Agency, editor. 1991. Data User's Guide to the United
States Environmental Protection Agency's Long-Term Monitoring Project: Quality
Assurance Plan and Data Dictionary. U.S. Environmental Protection Agency, Corvallis,
OR.
90. U.S. Environmental Protection Agency. 1993. Air Quality Criteria for Oxides of Nitrogen.
U.S. Environmental Protection Agency, Washington, D.C.
91. Webb, J. R., B. J. Cosby, F. A. Deviney, J. N. Galloway, S. W. Maben, and A. J. Bulger.
2004. Are brook trout streams in western Virginia and Shenandoah National Park
recovering from acidification? . Environmental Science and Technology 38:4091-4096.
92. Webb, R., J. Cosby, K. Eshleman, and J. Galloway. 1995. Change in the acid-base status of
Appalachian Mountain catchments following forest defoliation by the Gypsy Moth.
Water Air and Soil Pollution 85:535-540.
C-7
-------
93. Webster, K. E., P. L Brezonik, and B. J. Holdhusen. 1993. Temporal trends in low
alkalinity lakes of the Upper Midwest (1983-1989). Water Air and Soil Pollution 67:397-
414.
94. Webster, K. E., and P. O. Brezonik. 1995. Climate confounds detection of chemical
trends related to acid deposition in Upper Midwest lakes in the USA. Water Air and Soil
Pollution 85:1575-1580.
95. Webster, K. E., A. D. Newell, L. A. Baker, and P. L. Brezonik. 1990. Climatically induced
rapid acidification of a softwater seepage lake. Nature 347:374-376.
96. Wentz, D. A., W. J. Rose, and K. E. Webster. 1995. Long-term hydrologic and
biogeochemical responses of a soft water seepage lake in north central Wisconsin.
Water Resources Research 31:199-212.
97. Wigington, P. J. 1999. Episodic acidification: causes, occurrence and significance to
aquatic resources. Pages Chapter 1, pg. 1-5 in The Effects of Acidic Deposition on
Aquatic Ecosystems in Pennsylvania. 1998 PA Acidic deposition Conference.
Environmental Resources Research Institute, University Park, PA.
98. Wigington, P. J., J. P. Baker, D. R. DeWalle, W. A. Kretser, P. S. Murdoch, H. A. Simonin, J.
Van Sickle, M. K. McDowell, D. V. Peck, and W. R. Barchet. 1993. Episodic Acidification of
Streams in the Northeastern United States: Chemical and Biological Results of the
Episodic Response Project. U.S. Environmental Protection Agency, Washington, DC.
99. Wigington, P. J., J. P. Baker, D. R. DeWalle, W. A. Kretser, P. S. Murdoch, H. A. Simonin, J.
Van Sickle, M. K. McDowell, D. V. Peck, and W. R. Barchet. 1996. Episodic acidification of
small streams in the northeastern United States: Episodic Response Project. Ecological
Applications 6:374-388.
100. Wigington, P. J., T. D. Davies, M. Tranter, and K. Eshleman. 1990. Episodic Acidification
of Surface Waters due to Acidic Deposition. State of Science/Technology Report No. 12,
National Acid Precipitation Assessment Program, Washington, DC.
101. Wigington, P. J., T. D. Davies, M. Tranter, and K. N. Eshleman. 1992. Comparison of
episodic acidification in Canada, Europe and the United States. Environmental Pollution
78:29-35.
102. Willard, K. W. J., D. R. DeWalle, and P. J. Edwards. 2005. Influence of bedrock geology
and tree species composition on stream nitrate concentrations in mid-Appalachian
forested watersheds. Water Air and Soil Pollution 160:55-76.
103. Williard, K. W. J., D. R. DeWalle, and P. J. Edwards. 2005. Influence of bedrock geology
and tree species composition on stream nitrate concentrations in mid-Appalachian
forested watersheds. Water Air and Soil Pollution 160:55-76.
104. Young, T. C. 1991. A method to assess lake responsiveness to future acid inputs using
recent synoptic water column chemistry. Water Resources Research 27:317-326.
105. Young, T. C., and J. L. Stoddard. 1996. The TIME project design: I. Classification of
Northeast lakes using a combination of geographic, hydrogeochemical, and multivariate
techniques. Water Resources Research 32:2517-2528.
C-8
-------
APPENDIX D: OVERVIEW OF ENVIRONMENTAL MONITORING
PROGRAMS
Summary details on a wide variety of national monitoring and site-specific monitoring activities
can be found at: http://www.epa.gov/monitor/programs/index2.html and
http://www.epa.gov/monitor/programs/index3.html. Summary descriptions of several
programs are included below.
TERRESTRIAL MONITORING
Vital Signs Monitoring - National Park Service: - The National Park Service has initiated a long-
term ecological monitoring program, known as "Vital Signs Monitoring", to provide the
minimum infrastructure to allow more than 270 national park system units to identify and
implement long-term monitoring of their highest-priority measurements of resource condition.
The NPS has used the term "vital signs monitoring" to refer to a relatively small set of physical,
chemical, and biological elements and processes of park ecosystems that represent the overall
health or condition of park resources, known or hypothesized effects of stressors, or elements
that have important human values.43 While this is listed under the "terrestrial monitoring"
category, NPS is also measuring various water quality components, including temperature,
specific conductance, pH, dissolved oxygen, quantity, flow and discharge. See:
http://acwi.gov/monitoring/conference/2008/presentations/sessionC/C4A_Long.pdf. The
overall Vital Signs Monitoring effort is described at:
http://science.nature.nps.gov/im/monitor/index.cfm
Forest Inventory and Analysis (FIA) - US Forest Service: The FIA classifies land into forest and
non-forest and examines fragmentation, urbanization, and distance variables. This is done
using aerial photography/satellite data. Field sample locations are then distributed across the
landscape with approximately one sample location (FIA plot) every 6,000 acres. Forested sample
locations are visited by field crews who collect a variety of forest ecosystem data. There are
approximately 125,000 sampled plots. Non forest locations are also visited as necessary to
quantify rates of land use change. Forest health as part of the Forest Health Monitoring (FHM)
Program is measured on a subset of field plots (approx 1/96,000 acres or 8000 plots).
Parameters measured include plant species diversity, leaf area index, tree regeneration, lichen,
mortality, air pollution, soils, etc44. More details are available at:
http://fia.fs.fed.us/library/fact-sheets/
S. G. Fancy, J. E. Gross, S. L. Carter; Monitoring the Condition of Natural Resources in US National Parks,
Environmental Monitoring and Assessment (2009) Vol. 151, pages 161-174.
44 US EPA, Clean Air Markets Division, How to Measure the Effects of Acid Deposition: A Framework for Ecological
Assessments, June 2001. EPA 430-R-01-005
D-l
-------
Environmental Monitoring and Assessment Program (EMAP) - US EPA: The Environmental
Monitoring and Assessment Program (EMAP) is a research program to develop the tools
necessary to monitor and assess the status and trends of national ecological resources. EMAP's
goal is to develop the scientific understanding for translating environmental monitoring data
from multiple spatial and temporal scales into assessments of ecological condition and forecasts
of the future risks to the sustainability of our natural resources. EMAP's research supports the
National Environmental Monitoring Initiative of the CENR. The objectives are to advance the
science of ecological monitoring and ecological risk assessment, guide national monitoring with
improved scientific understanding of ecosystem integrity and dynamics, and demonstrate the
CENR framework through large regional projects. EMAP has defined a component structure.
See: http://www.epa.gov/emap/html/components/index.html and
http://www.epa.gov/nheerl/arm/
Long Term Ecological Research (LTER) - NSF: Twenty-six research sites currently comprise the
LTER Network (having grown from 6 sites in 1980). These include a diverse array of ecosystem
types spanning broad ranges of environmental conditions and human domination of the
landscape. Each site develops individual research programs in five core areas: pattern and
control of primary production; spatial and temporal distribution of populations selected to
represent trophic structure; pattern and control of organic matter accumulation in surface
layers and sediments; patterns of inorganic inputs and movements of nutrients through soils,
groundwater and surface waters; and patterns and frequency of site disturbances. See:
http://www.lternet.edu/
National Resources Inventory (NRI) - USDA Natural Resources Conservation Service (NRCS):
The National Resources Inventory (NRI), originally initiated in 1956, is a statistical survey of
natural resource conditions and trends on non-Federal land in the United States non-Federal
land includes privately owned lands, tribal and trust lands, and lands controlled by state and
local governments. Data collected include visibility/fine particulates; sediment load; soil
texture, chemistry, toxicity, mineralogy, climate, structure, strength, erodability; vegetation
growth rate/above-ground biomass, species/cover/range. The NRI provides nationally
consistent statistical data on how lands are used and on changes in the lands for the period
1982 - 2003. The NRI is a longitudinal sample survey based on scientific statistical principles and
procedures. From 1982-1997, the NRI was conducted on a five-year cycle on 800,000 sample
sites. It is now sampled annually at slightly less than 25 percent of the same sample sites. The
same sample sites have been studied since 1982, with the same data collection protocols.
http://www.nrcs.usda.gov/technical/NRI/
AIR MONITORING
Many national air quality and deposition monitoring networks have evolved to provide scientists
and policymakers with data on the fate and transport of regional sources of emissions. Much of
the information in these networks is used to contribute to understanding the effectiveness of air
pollution control strategies.
D-2
-------
Air Quality Monitoring - US EPA: The EPA's ambient air quality monitoring program is carried
out by State and local agencies and consists of three major categories of monitoring stations:
State and Local Air Monitoring Stations (SLAMS) - The SLAMS consist of a network of ~
4,000 monitoring stations whose size and distribution is largely determined by the
needs of State and local air pollution control agencies to meet their respective State
implementation plan (SIP) requirements.
National Air Monitoring Stations (NAMS) - The NAMS (1,080 stations) are a subset of
the SLAMS network with emphasis being given to urban and multi-source areas. In
effect, they are key sites under SLAMS, with emphasis on areas of maximum
concentrations and high population density
Special Purpose Monitoring Stations (SPMS) - The SPMS are established for special
studies needed by the State and local agencies to support State implementation plans
and other air program activities. The SPMS supplement the fixed monitoring network as
circumstances require and resources permit.
Additionally, a fourth category of a monitoring station, the Photochemical Assessment
Monitoring Stations (PAMS), which measures ozone precursors (approximately 60 volatile
hydrocarbons and carbonyl) has been required by the 1990 Amendments to the Clean Air Act. A
PAMS network is required in each ozone nonattainment area designated serious, severe, or
extreme. The required networks will have from two to five sites, depending on the population of
the area. See: http://www.epa.gov/air/oaqps/qa/monprog.html
Interagency Monitoring of Protected Visual Environments (IMPROVE) - Interagency: IMPROVE
is a cooperative measurement program overseen by an interagency-intergovernmental steering
committee. The monitoring was established in 1985 to aid creation of federal and state
implementation plans for protection of visibility in Class I areas (156 parks and wilderness areas)
as stipulated under the 1977 amendments to the Clean Air Act. The objectives of IMPROVE are
to: establish current visibility and aerosol conditions in mandatory Class I areas, identify
chemical species and emission sources responsible for existing man-made visibility impairment,
document long-term trends for assessing progress toward the national visibility goal, and
provide regional haze monitoring. The effort is supported by a Web site that provides access to
data, publications, documentation, etc: http://vista.cira.colostate.edu/improve/ (example of
interagency effort)
Clean Air Status and Trends Network (CASTNET) - US EPA: CASTNET is a regional long-term
environmental monitoring program administered and operated by EPA's Clean Air Markets
Division (CAMD). Developed from the existing National Dry Deposition Network (NDDN),
CASTNET was established in 1991 under the Clean Air Act Amendments. The regional monitoring
network was formed to assess trends in acidic deposition due to emission reduction regulations,
such as the Acid Rain Program (ARP) and NOx Budget Trading Program (NBP). CASTNET has since
become the nation's primary monitoring network for measuring concentrations of air pollutants
involved in acidic deposition affecting regional ecosystems and rural ambient ozone levels.
CASTNET provides data needed to assess and report on geographic patterns and long-term
D-3
-------
temporal trends in ambient air pollution and dry atmospheric deposition. CASTNET can also be
used to track changes in measurements associated with climate change (such as temperature
and precipitation). Presently there are a total of 86 operational CASTNET sites located in or near
rural areas and sensitive ecosystems collecting data on ambient levels of pollutants where urban
influences are minimal. As part of an interagency agreement, the National Park Service (NPS)
sponsors 27 sites which are located in national parks and other Class-l areas designated as
deserving special protection from air pollution. CASTNET data support various modeling efforts,
including PNIT and Magic and CMAQ models. The current CASTNET budget is approximately $3.9
million. See: http://www.epa.gov/castnet/docs/CASTNET_factsheet_2007.pdf
The National Atmospheric Deposition Program/National Trends Network (NADP/NTN) -
Interagency: is a nationwide network of over 250 precipitation monitoring sites. The network is
a cooperative effort among USGS, USDA, EPA, and other governmental and private entities. The
NADP/NTN started in 1978 with a network of 22 stations. Sites currently span the US, Puerto
Rico, and the Virgin Islands. The purpose of the network is to collect data on the chemistry of
precipitation for monitoring geographical and temporal long-term trends. The sites measure
atmospheric deposition dissolved in cloud droplets and deposited during rain and other forms of
precipitation (wet deposition). Data are collected weekly and analyzed for hydrogen (acidity as
pH), sulfate, nitrate, ammonium, chloride, and base cations (such as calcium, magnesium,
potassium and sodium). The current budget for NADP is approximately $5.5 million annually.
The NADP has expanded sampling to two additional networks. The Mercury Deposition Network
http://nadp.isws.illinois.edu/mdn/), currently with over 90 sites, was formed in 1995 and
analyzes weekly precipitation samples for mercury. The Atmospheric Integrated Research
Monitoring Network (AIRMoN), was formed to study precipitation chemistry trends with greater
temporal resolution. Precipitation samples are collected daily from a network of seven sites and
analyzed for the same constituents as the NADP/NTN samples. See:
http://nadp.isws.illinois.edu/. The Atmospheric Integrated Research Monitoring Network
(AirMON) managed by NOAA is a subset of NADP. AirMON dry-data is managed under CASTNET.
AIRMoN-wet precipitation sampling maintains rigorous sampling procedures, developed in
conjunction with NADP, that ensure that AIRMoN precipitation chemistry and wet deposition
estimates are timely and accurate. Currently, AIRMoN consists of several collocated operational
research establishments (CORE sites) throughout the eastern U.S. The daily precipitation
sampling protocol facilitates quantification of deposition estimates for several species, including
ammonium, which is an important consideration to the role of atmospheric deposition in coastal
eutrophication. The CORE site operated near NOAA/ARL/ATDD is located in the Walker Branch
Watershed, a long-term environmental research area. The watershed is forest ecosystem typical
of Southern Blue Ridge Mountains. See: http://www.atdd.noaa.gov/?q=node/46.
WATER MONITORING
National Water-Quality Assessment Program (NAWQA) - USGS: The USGS implemented the
National Water-Quality Assessment (NAWQA) Program in 1991 to develop long-term consistent
and comparable information on streams, rivers, ground water, and aquatic systems in support
of national, regional, State, and local information needs and decisions related to water-quality
management and policy. The NAWQA program was designed to address the following questions:
D-4
-------
What is the condition of the Nation's streams, rivers, and groundwaters?, How are these
conditions changing overtime?, How do natural features and human activities affect these
conditions, and where are those effects most pronounced? From 1991 to 2001, NAWQA
examined 51 river basins and aquifers to establish a national baseline. More than 1000 reports
were developed describing water quality conditions, including pesticides, nutrients, volatile
organic compounds, etc. From 2001-2012 the focus is to build on the data from the previous
decade in 42 units looking at a continuation of the work just outlined, including aquatic ecology,
to examine national priority topics such as agricultural chemicals, urbanization, mercury, and
public supply wells. See: http://water.usgs.gov/nawqa/
National Lakes Assessment - US EPA: During summer 2007, States, Tribes, and U.S. EPA
collaboratively sampled over 1,200 lakes across the country in pursuit of a National Lakes
Assessment (NLA). Completion of the NLA marks the first comprehensive, nationwide
evaluation of lakes since the National Eutrophication Survey of the 1970s. The NLA is set apart
from other national-scale lake assessment initiatives in that it is statistically designed and
focuses on multiple indicators of water quality, ecological integrity, and recreational suitability.
See: http://www.epa.gov/owow/lakes/lakessurvey/.
Wadeable Streams Assessment (WSA) - US EPA: The Wadeable Streams Assessment (WSA) is a
statistically-valid survey of the biological condition of small streams throughout the U.S. EPA and
states conducted the assessment in 2000-2004. Approximately, 1,392 sites were randomly
selected based on protocols established as part of EMAP to represent the condition of all
streams in regions that share similar ecological characteristics. Wadeable streams were chosen
for study because they are a critical natural resource and a well-established set of methods for
monitoring them exist. Data were collected on chemical (phosphorus, nitrogen, acidity, and
salinity) and physical properties (streambed sediments, in-stream fish habitat, riparian vegetative
cover, and riparian disturbance) as well as benthic macroinvertebrates. The protocols allow for
national comparability. The WSA establishes a national baseline for future studies. The WSA
also provides funding and expertise that will enhance each state's ability to monitor and assess
the quality of its waters in the future. The annual budget for WSA is approximately $8 million.
See: http://www.epa.gov/owow/streamsurvey/
Hydrologic Benchmark Network (HBN) - USGS: The Hydrologic Benchmark Network (HBN) was
established in 1963 to provide long-term measurements of streamflow and water quality in
areas that are minimally affected by human activities. These data are used to study long-term
trends in surface water flow and water chemistry and as a benchmark against which to compare
changes in flow and chemistry in developed watersheds. At its peak the network consisted of 58
drainage basins in 39 States. Changes in funding and land use within the watersheds reduced
the number of stations and samples collected by HBN. In the mid-1990s, the USGS conducted a
complete review of the network, and selected 5 eastern stations to conduct a pilot study to
assess the optimum sampling strategy for assessing long-and short-term trends. In 2003, the
USGS re-established a 15-station water-quality and 36-station discharge monitoring network
with a new design that allows tracking of trends in water quality at a range of river flow
conditions. Additional stations are anticipated to be added to the network as funding allows.
D-5
-------
See: http://ny.cf.er.usgs.gov/hbn/. One of the HBN sites - the Neversink River in NY is also
sampled as an LTM site by the USGS. See:
http://ny.cf.er.usgs.gov/hbn/siteinfo.cfm?ID=Neversink%20River
D-6
------- |