r/EPA United States Environmental Protection Agency Office of Water Washington, DC 20460 EPA822-R-02-015 March 2002 METHODS FOR EVALUATING WETLAND CONDITION #4 Study Design for Monitoring Wetlands ------- ------- United States Environmental Office of Water EPA 822-R-02-015 Protection Agency Washington, DC 20460 March 2002 METHODS FOR EVALUATING WETLAND CONDITION #4 Study Design for Monitoring Wetlands Errata sheet Principal Contributor Warnell School of Forest Resources, University of Georgia Amanda K. Parker Prepared jointly by: The U.S. Environmental Protection Agency Health and Ecological Criteria Division (Office of Science and Technology) and Wetlands Division (Office of Wetlands, Oceans, and Watersheds) ------- NOTICE The material in this document has been subjected to U.S. Environmental Protection Agency (EPA) technical review and has been approved for publication as an EPA document. The information contained herein is offered to the reader as a review of the "state of the science" concerning wetland bioassessment and nutrient enrichment and is not intended to be prescriptive guidance or firm advice. Mention of trade names, products or services does not convey, and should not be interpreted as conveying official EPA approval, endorsement, or recommendation. APPROPRIATE CITATION U.S. EPA. 2002. Methods for Evaluating Wetland Condition: Study Design for Monitoring Wetlands. Office of Water, U.S. Environmental Protection Agency, Washington, DC. EPA-822-R-02-015. This entire document can be downloaded from the following U.S. EPA websites: http ://www.epa.gov/ost/standards http://www.epa.gov/owow/wetlands/bawwg ------- r/EPA United States Environmental Office of Water EPA 822-R-02-015 Protection Agency Washington, DC 20460 March 2002 METHODS FOR EVALUATING WETLAND CONDITION #4 Study Design for Monitoring Wetlands Principal Contributor Amanda K. Parker Warnell School of Forest Resources, University of Georgia Prepared jointly by: The U.S. Environmental Protection Agency Health and Ecological Criteria Division (Office of Science and Technology) and Wetlands Division (Office of Wetlands, Oceans, and Watersheds) ------- ------- NOTICE The material in this document has been subjected to U.S. Environmental Protection Agency (EPA) technical review and has been approved for publication as an EPA document. The information contained herein is offered to the reader as a review of the "state of the science" concerning wetland bioassessment and nutrient enrichment and is not intended to be prescriptive guidance or firm advice. The views expressed by the author are his or her own and do not necessarily reflect those of EPA. Mention of trade names, products or services does not convey, and should not be interpreted as conveying official EPA approval, endorsement, or recommendation. APPROPRIATE CITATION Parker, Amanda K. 2001. Methods for Evaluating Wetland Condition: Study Design for Monitoring Wetlands..EPA 822-R-02-015. U.S. Environmental Protection Agency; Office of Water; Washington, DC. This entire document can be downloaded from the following U.S. EPA websites: http://www.epa.gov/ost/standards http://www.epa.gov/owow/wetlands/bawwg 11 ------- ------- CONTENTS FOREWORD iv LIST OF "METHODS FOR EVALUATING WETLAND CONDITION" MODULES : v SUMMARY 1 PURPOSE 1 INTRODUCTION 1 CONSIDERATIONS FOR SAMPLING DESIGN 2 SAMPLING PROTOCOI 6 SUGGESTED WEBSITES 13 REFERENCES 14 i LIST OF TABLES TABLE 1: COMPARISON OF STRATIFIED RANDOM, TARGETED, AND BACI SAMPLING DESIGNS 7 111 ------- ------- FOREWORD In 1999, the U.S. Environmental Protection Agency (EPA) began work on this series of reports entitled Methods for Evaluating Wetland Condition. The purpose of these reports is to help States and Tribes develop methods to evaluate (1) the overall ecological condition of wetlands using biological assessments and (2) nutrient enrichment of wetlands, which is one of the primary stressors damaging wetlands in many parts of the country. This information is intended to serve as a starting point for States and Tribes to eventually establish biological and nutrient water quality criteria specifically refined for wetlands. This purpose was to be accomplished by providing a series of "state of the science" modules concerning wetland bioassessment as well as the nutrient enrichment of wetlands. The individual module format was used instead of one large publication to facilitate the addition of other reports as wetland science progresses and wetlands are further incorporated into water quality programs. Also, this modular approach allows EPA to revise reports without having to reprint them all. A list of the inaugural set of 20 modules can be found at the end of this section. This series of reports is the product of a collaborative effort between EPA's Health and Ecological Criteria Division of the Office of Science and Technology (OST) and the Wetlands Division of the Office of Wetlands, Oceans and Watersheds (OWOW). The reports were initiated with the support and oversight of Thomas J. Danielson (OWOW), Amanda K. Parker and Susan K. Jackson (OST), and seen to completion by Douglas G. Hoskins (OWOW) and Ifeyinwa F. Davis (OST). EPA relied heavily on the input, recommendations, and energy of several panels of experts, which unfortunately have too many members to list individually: • Biological Assessment of Wetlands Workgroup • Wetlands Nutrient Criteria Workgroup More information about biological and nutrient criteria is available at the following EPA website: http://www.epa.gov/ost/standards More information about wetland biological assessments is available at the following EPA website: http://www.epa.gov/owow/wetlands/bawwg IV ------- ------- LIST OF "METHODS FOR EVALUATING WETLAND CONDITION" MODULES MODULE # MODULE TITLE 1 INTRODUCTION TO WETLAND BIOLOGICAL ASSESSMENT 2 INTRODUCTION TO WETLAND NUTRIENT ASSESSMENT 3 THE STATE OF WETLAND SCIENCE 4 STUDY DESIGN FOR MONITORING WETLANDS 5 ADMINISTRATIVE FRAMEWORK FOR THE IMPLEMENTATION OF A WETLAND BIOASSESSMENT PROGRAM 6 DEVELOPING METRICS AND INDEXES OF BIOLOGICAL INTEGRITY 7 WETLANDS CLASSIFICATION 8 VOLUNTEERS AND WETLAND BIOMONITORING 9 DEVELOPING AN INVERTEBRATE INDEX OF BIOLOGICAL INTEGRITY FOR WETLANDS 10 USING VEGETATION TO ASSESS ENVIRONMENTAL CONDITIONS IN WETLANDS 11 USING ALGAE TO ASSESS ENVIRONMENTAL CONDITIONS IN WETLANDS 1 2 USING AMPHIBIANS IN BlOASSESSMENTS OF WETLANDS 13 BIOLOGICAL ASSESSMENT METHODS FOR BIRDS 14 WETLAND BIOASSESSMENT CASE STUDIES 15 BIOASSESSMENT METHODS FOR FISH 16 VEGETATION-BASED INDICATORS OF WETLAND NUTRIENT ENRICHMENT 17 LAND-USE CHARACTERIZATION FOR NUTRIENT AND SEDIMENT RISK ASSESSMENT 1 8 BlOGEOCHEMICAL INDICATORS 19 NUTRIENT LOAD ESTIMATION 2O SUSTAINABLE NUTRIENT LOADING v ------- ------- SUMMARY Otate and Tribal monitoring programs kJ should be designed to assess wetland condi- tion with statistical rigor while maximizing available management resources. The three study designs described in this module—stratified random sam- pling, targeted/tiered approach, and before/after, control/impact (BACI)—allow for collection of a significant amount of information for statistical analy- ses with relatively minimal effort. The sampling de- sign selected for a monitoring program will depend on the management question being asked. Sam- pling efforts should be designed to collect informa- tion that will answer management questions in a way that will allow robust statistical analysis. In addi- tion, site selection, characterization of reference sites or systems, and identification of appropriate index periods are all of particular concern when selecting an appropriate sampling design. Careful selection of sampling design will allow the best use of finan- cial resources and will result in the collection of high- quality data for evaluation of the wetland resources of a State or Tribe. Examples of different sampling designs currently in use for State and Tribal wet- land monitoring are described in the Case Study (Bioassessment) module and on http:// www.epa.gov/owow/wetlands/bawwg/case.htnil. PURPOSE rhe purpose of this module is to provide tech- nical guidance information on designing effective sampling programs for State and Tribal wet- land water quality monitoring. INTRODUCTION T/rTetlands are included as waters of the United W States in the Federal regulations (40 CFR 122.2,40 CFR230.3, and40 CFR232.2) imple- menting the Clean Water Act [Section 502(7)]. Wetlands are important waterbodies; they can pro- vide many functions that are beneficial to the local landscape, for example, water storage, water qual- ity improvement, and wildlife habitat. Wetlands are also valuable as ecosystems in their own right, pro- viding carbon storage, biogeochemical transforma- tions, and aquifer recharge (Mitsch and Gosselink 1993). However, few States or Tribes (only six States and Tribes reported attainment of designated uses in the "National Water Quality Inventory 1996 Report to Congress") include wetland monitoring in their routine water quality monitoring programs (U.S. EPA 1996, 1998). Yet, wetland chemical and biological water quality monitoring data are scarce to nonexistent in many States and most Fed- eral databases. The need for water quality moni- toring data on wetlands is obvious; the vast major- ity of data about wetlands are collected in relation to dredge and fill permitting. Indeed, only 4% of the nation's wetlands were surveyed and only 11 States and Tribes reported information concerning wetland designated use attainment in the "National Water Quality Inventory 1998 Report to Congress" (U.S. EPA 1998). This module is intended to pro- vide guidance to State and Tribal water quality man- agers on designing wetland monitoring programs to be included as a part of their routine water quality sampling. ; . Most States and Tribes will need to begin wet- land monitoring programs to collect water quality and biological data (U.S. EPA 1990). The best monitoring programs are designed to assess wet- land condition with statistical rigor while maximiz- ing available management resources. At the broadest level, monitoring should include the following: • Detecting and characterizing the ambient condi- tion of existing wetlands • Describing whether wetland condition is improv- ing, degrading, or staying the same • Defining seasonal patterns in wetland conditions • Identifying thresholds for system stressors, that is, how much the system can be disturbed with- ------- out causing unacceptable changes in wetland sys- tem quality or degradation of beneficial uses. Water quality monitoring programs at the State and Tribal level are often poorly and inconsistently funded or are improperly designed and carried out, making it difficult to collect a sufficient number of samples over time and space to identify changes in system condition or to estimate average conditions with statistical rigor. Three approaches to study design for assessing water quality as well as bio- logical and ecological condition and for identifying degradation in wetlands are described in this mod- ule. Specific issues to consider in designing moni- toring programs for wetland systems are also dis- cussed in this module. The study designs presented can be tailored to fit the specific goals of monitoring programs. The assemblage-specific modules will discuss detailed sampling considerations. The three approaches described—stratified ran- dom sampling; targeted/tiered approach; and BACI—present study designs that allow one to obtain a significant amount of information with rela- tively minimal effort. Stratified random sampling begins with a large-scale random monitoring design that is reduced as the wetland system or habitat conditions are characterized. This approach is used to find the mean condition of each wetland class, or type, in a specific region. Stratified random sam- pling design is frequently used for new large-scale monitoring programs at the State and Federal level (e.g., the Environmental Monitoring and Assess- ment Program, the Regional Environmental Moni- toring and Assessment Program, and State pro- grams in Maine, Montana, and Wisconsin). The tiered or targeted approach to monitoring begins with coarse screening and proceeds to more de- tailed monitoring protocols as impaired and high- risk systems are identified for further investigation. Targeted sampling design provides atriage approach to identifying wetland systems in need of restora- tion, protection, and intensive management. Sev- eral State pilot proj ects use this method or a modi- fication of this method for wetland assessment (e.g., Florida, Ohio, Oregon, and Minnesota). The syn- optic approach described in Kentula et al. (1993) uses a modified targeted sampling design. The BACI design and its modifications are frequently used to assess the success of restoration efforts or other management experiments. BACI design al- lows for comparisons in similar systems over time to determine the rate of change in relation to the management activity, for example, to assess the suc- cess of a wetland hydrologic restoration. Detenbeck et al. (1996) used BACI design for monitoring water quality of wetlands in the Minne- apolis/St. Paul, Minnesota, metro area. Monitoring programs should be designed to an- swer questions such as how, when, where, and at what levels do unacceptable wetland conditions occur? These questions are interrelated, and a well- designed monitoring program can contribute to an- swering them. Sampling design is dependent on the management question being asked. Sampling efforts should be designed to collect information that will answer the management question. For example, stratified random sampling might be good for ambi- ent monitoring programs, BACI for evaluating res- toration, and targeted sampling for developing an Index of Biological Integrity or nutrient criteria thresholds, hi fact, some State programs will likely need to use a combination of approaches. CONSIDERATIONS FOR SAMPLING DESIGN DESCRIBING THE MANAGEMENT QUESTION Clearly defining the question being asked (identi- fying the hypothesis) encourages the use of appro- priate statistical analyses, reduces the occurrence of false-positive (Type I) errors, and increases the efficient use of management resources (Suter 1993, Leibowitz et al. 1992, Kentula et al. 1993). Begin- ------- ning a study or monitoring program with carefully defined questions and objectives helps to identify the statistical analyses most appropriate for the study and reduces the chance that statistical assumptions will be violated. Management resources are opti- mized because resources are directed at monitor- ing that is most likely to answer management ques- tions. In addition, defining the specific hypotheses to be tested, carefully selecting reference sites, and identifying the most useful sampling interval can help reduce the uncertainty associated with the results of any sampling design as well as further conserve management resources (Kentula et al. 1993). SITE SELECTION Site selection is arguably the most important task in developing a monitoring program (Kentula et al. 1993). Site selection for a monitoring program is based on the need to sample a relatively large num- ber of wetlands to establish the range of wetland quality in a specific regional setting. Protecting or improving the quality of a wetland system often de- pends on the ability of the monitoring program to identify dose-response relationships, for example, the relationship of nutrient concentration (dose) to periphyton abundance (response). Dose-response relationships can be identified using large sample sizes and systems that span the gradient (low to high) of wetland quality. All ranges of responses should be observed along the dosing gradient from low levels to high levels of human disturbance. In addition, wetland monitoring frequently requires an analysis of both watershed/landscape characteris- tics and wetland-specific characteristics (Kentula et al. 1993, Leibowitz et al. 1992). Therefore, wetland sampling sites should be selected on the basis of land use in the region so that watersheds range from minimally impaired with few expected stressors to high levels of development (e.g., agri- culture, forestry, or urban) with multiple expected stressors (see Module 17: Land-Use Character- ization for Nutrient and Sediment Risk Assessment). Establishing dose-response relationships may be confounded due to time lags between stressor oc- currence and biological or functional response. The duration of the time lag between stressor and re- sponse depends on many factors, including the type of stressor, climate, and system hydrology. These factors should be considered when selecting wet- land sites to establish the range of wetland quality within a region. The synoptic approach described in Leibowitz et al. (1992) provides a method of rapid assessment of wetlands at the regional and watershed levels that can help identify the range of wetland quality within a region. Leibowitz et al. (1992) recommend an initial assessment for site selection based on cur- rent knowledge of watershed- and landscape-level features; modification of such an assessment can be made as more data are collected. Assessing watershed characteristics through the use of aerial photography and geographical information systems linked to natural resource and land-use databases can aid in identifying reference and degraded sys- tems (see Module 17: Land-Use Characterization for Nutrient and Sediment Risk Assessment) (Johnston etal. 1988,1990,Gwinetal. 1999,Palik et al. 2000). Some examples of watershed char- acteristics that can be evaluated using aerial pho- tography and geographical information systems in- clude land use, land cover (including riparian veg- etation), soil, bedrock, hydrography, and infrastruc- ture (e.g., roads and railroads). IDENTIFYING AND CHARACTERIZING REFERENCE WETLANDS The term "reference" in this module refers to those systems that are least impaired by anthropogenic effects. This term can be confusing because of the different meanings that are currently in use in differ- ent classification methods (particularly in hydrogeomorphic wetland classification). A dis- cussion of the term "reference" and its multiple meanings is provided in Module 7: Wetlands Classification. ------- Watersheds with little or no development that re- ceive minimal anthropogenic inputs could potentially contain wetlands that would serve as minimally im- paired reference sites. Watersheds with a high per- centage of the drainage basin occupied by urban areas, agricultural land, and altered hydrology are likely to contain wetlands that are impaired or could potentially be considered "at risk" for developing problems. Wetland loss in the landscape should also be considered when assessing watershed char- acteristics for reference wetland identification. Biodiversity can become impoverished due to wet- land fragmentation or decreases in regional wet- land density, even in the absence of site-specific land-use activities. Reference wetlands may be more difficult to locate if fragmentation of wetland habitats is significant, and they may no longer rep- resent the biodiversity of minimally disturbed wet- lands in the region. The continued high rate of wet- land loss in most States and Tribes requires that multiple reference sites be selected to ensure some consistency in reference sites for multiple-year sam- pling programs (Leibowitz et al. 1992, Kentula et al. 1993). Once the watershed level has been con- sidered, a more site-specific investigation can be initiated to better assess wetland condition. Potential reference wetlands should be charac- terized to allow for the identification of appropriate reference wetland systems. Appropriate reference sites will have similar soils, vegetation, hydrologic regimes, and landscape settings to other wetlands in the region (Adamus 1992, Leibowitz etal. 1992, Kentulaetal. 1992,Detenbecketal. 1996). Clas- sification of wetlands, as discussed in Module 7: Wetlands Classification, will aid in identifying ap- propriate reference wetlands for specific regions and wetland classes. Wetland classification should be supplemented with information on hydroperiod and flood frequency to ensure that the selected refer- ence wetlands are truly representative of wetlands in the region, class, or subclass of interest. Refer- ence wetlands may not be available for all wetland classes. In this case, data from systems that are as close as possible to the assumed unimpaired state of wetlands in the wetland class of interest should be sought from within the same geologic province. Development of a conceptual reference may be nec- essary if appropriate reference sites cannot be found in the local region or geologic province. Techniques for defining a conceptual reference are discussed at length hi Harris et al. (1995), Trexler (1995), and Toth etal. (1995). Reference wetlands should be selected on the basis of low levels of human alteration in their watersheds (Leibowitz et al. 1992, Kentula et al. 1993, U.S. EPA 2000). Selecting reference wetlands usually involves the assessment of land use within water- sheds as well as visits to individual wetland systems to ground-truth expected land use and check for unsuspected impacts. Ground-truthing visits to ref- erence wetlands are crucial for the identification of ecological impairment that may not be apparent from land-use and local habitat conditions. Again, a suf- ficient sample size is important to characterize the range of conditions that can be expected in the least. impacted systems of the region (Detenbeck et al. 1996). Reference wetlands should be identified for each ecoregion or geological province in the State or Tribal lands and then characterized with respect to ecological integrity. A minimum of three low- impact reference systems is recommended for each wetland class for statistical analyses. However, power analysis can be performed to determine the degree of replication necessary to detect an impact to the systems being investigated (Detenbeck et al. 1996, Urquhart et al. 1998; see also http://www. mpl-pwrc.usgs.gov/powcase/index.html). Highest priority should be given to identifying reference sys- tems for those wetland classes considered to be at the greatest risk from anthropogenic stress. WHEN TO SAMPLE Sampling may be targeted to the period when problems are most likely to occur—the index pe- riod. The appropriate index period will be defined by what the investigator is trying to investigate and ------- by what taxonomic assemblage or parameters are being used for that investigation (Barbour et al. 1999). For example, increased nutrient concen- trations and sedimentation from nonpoint sources may occur following periods of high runoff during the spring and fall, whereas point sources of nutri- ent pollutants may cause plankton blooms and/or increased water and soil nutrient concentrations in wetland pools during times of low rainfall. Hence, different index periods may be needed for nonpoint source and point source nutrients. Each taxonomic assemblage studied will also have an appropriate index period—usually in the growing season (see assemblage methods in the Minnesota case study: http://www.epa.gov/owow/wetlands/bawwg/case/ mnl .html). The index period window may be early in the growing season for amphibians and algae. Other assemblages, such as vegetation and birds, may require a different sampling window for the index period (see the assemblage-specific modules for recommendations). Once wetland condition has been characterized, one-time annual sampling dur- ing the appropriate index period may be adequate for multiple-year monitoring of indicators of nutri- ent status, designated use, and biotic integrity. How- ever, criteria and ecological indicator development may require more frequent sampling to define con- ditions that relate to the stressor or the impact of interest (Karr and Chu 1999, Stevenson 1996, 1997). Ideally, water quality monitoring programs pro- duce long-term data sets compiled over multiple years to capture the natural, seasonal, and year-to- year variations in biological communities and waterbody constituent concentrations (Tate 1990, Doddsetal. 1997,McCormicketal. 1999). Mul- tiple-year data sets can be analyzed with statistical rigor to identify the effects of seasonality and vari- able hydrology. Once the pattern of natural varia- tion has been described, the data can be analyzed to determine the ecological state of the waterbody. Long-term data sets have also been important in influencing management decisions about wetlands, most notably in the Everglades, where long-term data sets have induced Federal, State, and Tribal actions for conservation and restoration of the larg- est wetland system in the United States (see Davis and Ogden 1994, Redfield 1999, 2000, 2001; 1994 Everglades Forever Act, Florida Statute §373.4592). In spite of the documented value of long-term data sets, there is a tendency to intensively study a waterbody for 1 year before and 1 year after treat- ment. A more cost-effective approach would be to measure only the indices most directly related to the stressor of interest (i.e., those parameters or indicators that provide the best information to an- swer the specific management question), but to double or triple the monitoring period. Two or more years of data are often needed to identify the ef- fects of years with extreme climatic or hydrologic conditions. Comparisons over time between refer- ence and at-risk or degraded systems can help de- scribe biological response and annual patterns in the presence of changing climatic conditions. Long- term data sets can also help describe regional trends. Flooding or drought may significantly affect wet- land biological communities and the concentration of water column and soil constituents. Effects of uncommon climatic events can be characterized to discern the overall effect of management actions (e.g., nutrient reduction and water diversion) if sev- eral years of data are available to identify the long- term trends. At the very minimum,-2 years of data before and 2 after specific management actions, but preferably 3 of more each, are recommended to evaluate the cost-effectiveness of management ac- tions with some degree of certainty (U.S. EPA 2000). If funds are limited, restricting sampling fre- quency and/or numbers of indices analyzed should be considered to preserve a long-term data set. Reducing sampling frequency or numbers of pa- rameters measured will allow for effectiveness of management approaches to be assessed against the high annual variability that is common in most wet- land systems. Wetlands witiihighhydrologicalvaria- ------- tion from year to year may require more years of sampling before and after mitigation procedures to identify the effects of the natural hydrologic vari- ability (Kadlec and Knight 1996). Using the B ACI study design may also provide substantial benefitfor determining the effectiveness of management activities. Tracking both reference (control) and impacted systems within a region over tune will help determine the rate and direction of change in monitored systems regardless of system variability. The B ACI design may require more fre- quent monitoring of reference systems than is sug- gested for random or targeted designs, but it would allow useful comparison of system change regard- less of variability (Detenbecketal. 1996). Characterizing precision of estimates Estimates of dose-response relationships, nutri- ent and biological conditions in reference systems, and wetland conditions in a region are based on sampling, hence precision must be assessed. Pre- cision is defined as "measure of the degree of agree- ment among the replicate analyses of a sample, usu- ally expressed as the standard deviation" (Eaton et al. 2000). Determining precision of measurements for one-time assessments from single samples in a wetland is often necessary. The variation associ- ated with one-time assessments from single samples can often be determined by resampling a specific number of wetlands during the survey. Measure- ment variation among replicate samples can then be used to establish the expected variation for one- time assessment of single samples. Resampling does not establish the precision of the assessment pro- cess; it identifies the precision of an individual mea- surement (Kentula et al. 1993). Resampling fre- quency is often conducted for one wetland site in every block of 10 sites. However, investigators should adhere to the objectives of resampling (of- ten considered an essential element of quality as- sessment/quality control) to evaluate the variation in a one-time assessment from a single sample. The larger the sample size, the better (smaller) will be the estimate of that variation. Often, more than 1 in 10 samples need to be replicated in monitoring pro- grams to provide a reliable estimate of measure- ment precision (Barbouretal. 1999). SAMPLING PROTOCOL APPROACHES TO SAMPLING DESIGN Three approaches to sampling design—stratified random, targeted design, and B ACI—have advan- tages and disadvantages that under different circum- stances warrant the choice of one approach over the other (Table 1). The decision as to the best approach for sample design in a new monitoring program must be made by the water quality resource manager or management team after carefully con- sidering different approaches. Justification of a dose- response relationship is confounded by lack of ran- domization and replication, and must be considered in choosing a sampling design for a monitoring pro- gram. Direct identification of a cause-response re- lationship is not possible in observational (monitor- ing) studies. However, inferences of causality can be argued if appropriate information is collected. Beyers (1998) describes assembly rules for causal arguments that can be used to infer causality for stressors of interest. The number of sites to be sampled and the sampling frequency are determined by the type of sampling design chosen by the re- source managers. Power analyses can be used to help make this decision by estimating the number of sites to be sampled and replicates needed to pro- duce a statistically significant result. The U.S. Geo- logical Survey provides an excellent website that can assist the resource manager in determining the number of sites and replicates needed to produce the desired analytical power for a particular sam- pling design: http://www.mpl-pwrc.usgs.gov/ powcase/howhtml. 6 ------- TABLE 1: COMPARISON OF STRATIFIED RANDOM, TARGETED, AND BACI SAMPLING DESIGNS STRATIFIED RANDOM TARGETED BACI Random selection of wetland systems from entire population within a region. This design requires minimal prior knowledge of wetlands within the sample population for stratification. This design may require more resources (time and money) to randomly sample wetland classes, because more wetlands may need to be sampled. System characterization for a class of wetlands is more statistically robust. Rare wetlands may be under- represented or absent from the sampled wedands. This design is potentially best for regional characterization of wetland classes, especially water quality conditions are not known. Targeted selection of wedands based on problematic (wedand systems known to have problems) and reference wetlands. This design requires prior knowledge of wetlands within the sample population. This design utilizes fewer resources because only targeted systems are sampled. System characterization for a class of wedands is less statistically robust, although characterization of a targeted wedand may be statistically robust. This design may miss important wetland systems if they are not selected for the targeted investigation. This design is potentially best for site-specific and watershed-specific criteria development when water quality conditions for the wedand of interest are known. Selection of wetlands based on a known impact. This design requires knowledge of a specific impact to be analyzed. This design utilizes fewer resources because only wedands with known impacts and associated control systems are sampled. Characterization of the investigated systems is statistically robust. The information gained in this type of investigation is not transferable to wetland systems not included in the study. This design is potentially best for monitoring restoration or creation of wedands and systems that have specific known stressors. STRATIFIED RANDOM SAMPLING DESIGN Probabilistic sampling—a sampling process in which randomness is a requisite (Hayek 1994)— can be used to characterize the status of water qual- ity conditions and biotic integrity in a region's wet- land systems. This type of sampling design is used to describe the average conditions of a wetland population, identify the variability among sampled wetlands, and to help determine the range of wet- land system conditions in a region. However, the data collected from a probabilistic random sample design will generally be characteristic of the domi- nant class of wetland in the region, and rare wet- lands may be underrepresented or absent from the probabilistically sampled wetlands. Additional sam- pling sites may need to be added to include the com- plete range of wetland conditions and classes in the region. Probabilistic designs are often modified by strati- fication (such as classification). Stratification, or stratified random sampling, is a type of probability sampling in which atarget population is divided into relatively homogenous groups (strata) before sam- pling based on factors that influence variability in that population (Hayek 1994). Analysis of vari- ance can be used to identify statistically different parameter means among the sampling strata. The strata are the analysis of variance treatments (Poole 7 ------- 1972). The result of collecting and assessing water quality and biotic responses with a stratified ran- dom sample is, presumably, an unbiased estimate of the descriptive statistics (e.g., means, variances, modes, and quartiles) of all wetlands in a stratum. Stratification by wetland size and class provides more information about different types of wetlands within a region. Sample statistics from random se- lection alone would be most characteristic of the dominant wetland class in a region if the population of wetlands is not stratified. Many State 305b and watershed monitoring pro- grams use stratified random sampling designs, for example, Maine, Montana, and Wisconsin pilot projects use this design. Details of these monitor- ing designs can be found in the Module 14: Case Studies (Bioassessment) and at http:// wvwv.epa.gov/owow/wetlands/bawwg/index.html. Stratification is based on identifying wetland sys- tems in a region (or watershed) and then selecting an appropriate sample of systems from the defined population. The determination of an appropriate sample population depends on the management questions being asked. A sample population of iso- lated depressional wetlands could be identified as a single stratum, but investigations of these wetlands would notprovide any information on riparian wet- lands in the same region. If the goal of the monitor- ing program is to identify wetland condition for all wetland classes within a region, then a sample population of wetlands should be randomly selected from all wetlands within each class. In practice, most State and Tribal programs stratify random populations by size, wetland class (see Module 7: Wetlands Classification), and landscape character- istics or location (see http://www.epa.gov/owow/ wetlands/bawwg/case/me.html, http://www. epa.gov/owow/wetlands/bawwg/case/wa.html, and http://www.epa.gov/owow/wetlands/bawwg/case/ \\ilJitrnl). Once the wetlands for each stratum have been selected, the sample population is often modified by deleting systems that are too close to other wet- lands to be different, thereby reducing redundant collection efforts. For example, the Environmental Monitoring and Assessment Program limits redun- dant collection efforts by applying a regular (hex- agonal) grid'to a map of the area. Sampling sites are chosen by randomly selecting grid cells and ran- domly sampling wetland resources within the cho- sen grid cells (Paulsen et al. 1991). Estimatesof ecological conditions from these kinds of modified . probabilistic sampling designs can be used to char- acterize the water quality conditions and biological integrity of wetland systems in a region and, over time, to distinguish trends in ecological condition within a region (see http://www.epa.gov/owow/wet- lands/bawwg/case/mtdev.html and http:// www.epa.gov/owow/wetlands/bawwg/case/ flLhtml). TARGETED DESIGN A targeted approach to sampling design may be more appropriate when resources are limited. Tar- geted sampling is a specialized case of random stratified sampling. The approach described here involves defining a gradient of impairment. Once the gradient has been defined and systems have been placed in categories of impairment, investigators focus the most effort on identifying and characteriz- ing wetland systems or sites likely to be impacted by anthropogenic stressors and on relatively undis- turbed wetland systems or sites (see "Identifying and Characterizing Reference Wetlands") that can serve as regional, subregional, or watershed ex- amples of natural biological integrity. The Florida Department of Environmental Protection uses a tar- geted sampling design for developing thresholds of impairment with macroinvertebrates (http:// www.epa.gov/owow/wetlands/bawwg/case/ fl2.html). Choosing sampling stations that best al- low the comparison of ecological integrity at refer- ence wetland sites of known condition can conserve financial resources. A sampling design that tests specific hypotheses (e.g., the study by the Florida 8 ------- Department of Environmental Protection tested the effect of elevated water column phosphorus on macroinvertebrate species richness) can generally be analyzed with statistical rigor and can conserve resources by answering specific questions. Fur- thermore, the identification of systems with prob- lems and reference conditions eliminates the need for selecting a random sample of the population for monitoring. Targeted sampling assumes some knowledge of the systems sampled. Systems with evidence of degradation are compared with reference systems that are similar in physical structure (i.e., in the same class of wetlands). Targeted sampling requires that the wetlands be characterized by a gradient of im- pairment. Wetland systems should be placed along a continuum from reference to most impacted. An impaired or degraded wetland is simply a system in which anthropogenic impacts exceed acceptable levels or interfere with beneficial uses. Compari- son of the monitoring data with the data collected from reference wetlands will allow characterization of the sampled systems. Wetlands identified as "at risk" should be evaluated through a sampling pro- gram to characterize the degree of degradation. Once characterized, the wetlands should be placed in categories such as the following: • Degraded wetlands—wetlands in which the level of anthropogenic perturbance interferes with des- ignated uses • High-risk wetlands—wetlands in which anthro- pogenic stress is high but does not significantly impair designated uses (Tnhigh-risk systems, im- pairment is prevented by one or a few factors that could be changed by human actions, although characteristics of ecological integrity are already marginal.) • Low-risk wetlands—wetlands in whichmany fac- tors prevent impairment, stressors are maintained below problem levels, and/or no development is contemplated that would change these conditions • Reference wetlands—wetlands in which the eco- logical characteristics most closely represent the pristine or minimally impaired condition. Once wetland systems have been classified on the basis of their physical structure (see Module 7: Wetlands Classification) and placed into the cat- egories previously defined, specific wetlands need to be selected for monitoring. At this point, ran- domness is introduced; wetlands should be ran- domly selected within each class and risk category for monitoring. An excellent example of categoriz- ing wetlands in this manner is given in the Ohio En- vironmental Protection Agency's case study at http://www.epa.gov/owow/wetlands/bawwg/case/ ohl.html. It used the Ohio Rapid Assessment Method to categorize wetlands by degree of im- pairment. The Minnesota Pollution Control Agency also used a targeted design for monitoring wetlands (see http://www.epa.gov/owow/wetlands/bawwg/ case/mnl .html). It used the best professional judg- ment of local resource managers to identify refer- ence sites as well as sites with known impairment from identified stressors (e.g., agriculture and stormwater runoff). Monitoring efforts are oftenprioritizedto best utilize limited resources. For example, case study investi- gators in Oregon chose not to monitor depressional wetlands because of funding constraints; they fur- ther tested the degree of independence of selected sites (and thus the need to monitor all of those sites) by using cluster analysis and other statistical tests (seehttp://www.epa.gov/owow/wetlands^awwg/ case/or.html). Frequency of monitoring is deter- mined by the management question being asked and the intensity of monitoring necessary to collect enough information to answer the question. In ad- dition, monitoring should identify the watershed-level activities that are likely to result in ecological deg- radation of wetland systems (Suter et al. 1993). Targeted sampling design involves monitoring iden- tified degraded systems and comparable reference systems most intensively. Low-risk systems are 9 ------- monitored less frequently (after initial identification), unless changes in the watershed indicate an in- creased risk of degradation. Activities surrounding impaired wetland systems may be used to help identity which actions nega- tively affect wetlands, and therefore may initiate more intensive monitoring of at-risk wetlands. Moni- toring should focus on factors likely to identify eco- logical degradation and anthropogenic stress and on any actions that might alter those factors. State/ Tribal water quality agencies should encourage adoption of local watershed protection plans to minimize ecological degradation of natural wetland systems. Development plans in a watershed should be evaluated to identify potential future stressors. Changes in point sources can be monitored through the National Pollutant Discharge Elimination Sys- tem permit program (U.S. EPA 2000). Changes in nonpoint sources can be evaluated through the iden- tification and tracking of wetland loss and/or deg- radation, increased residential development, in- creased tree harvesting, and shifts to more inten- sive agriculture with greater fertilizer use or increases in livestock numbers. Local planning agencies should be informed of the risk of increased anthro- pogenic stress and encouraged to guide develop- ment accordingly. Ecological degradation often gradually increases as a result of many growing sources of anthropogenic stress. Therefore, fre- quent monitoring is warranted for high-risk wetlands if sufficient resources remain after meeting the needs of degraded wetlands. Whenever development plans appear likely to alter factors that maintain eco- logical integrity in a high-risk wetland (e.g., veg- etated buffer zones), monitoring should be initiated at a higher sampling frequency in order to enhance the understanding of baseline conditions (U.S. EPA 2000). BACI SURVEY DESIGN An ideal impact survey has several features: The type of impact, time of impact, and place of occur- rence should be known in advance; the impact should not have occurred yet; and control areas should be available (Green 1979). The first feature allows surveys to be efficiently planned to account for the probable change in the environment. The second feature allows a baseline study to be estab- lished and to be extended as needed. The third feature allows the surveyor to distinguish between temporal effects unrelated to the impact and changes related to the impact. In practice, however, ad- vance knowledge of specific impacts is rare and the ideal impact survey is rarely conducted. BACI designs modified to monitor impacts during or after their occurrence can still provide information, but there is an increase in the uncertainty associated with the results, and the likelihood of finding a sta- tistically significant change caused by the impact is much less probable. Power analyses of after-only studies were conducted by Osenberg et al. (1994). They determined that because of the time constraints of most studies, relatively few of the population and chemical/physical parameters could provide ad- equate analytical power. They suggest expending a greater effort on monitoring individual-based pa- rameters (e.g., body size and recruitment density) in addition to population and chemical/physical pa- rameters for environmental impact assessments (Osenberg et al. 1994). Defining the study objec- tives, and identifying the specific hypotheses being tested, greatly increases the certainty of the results. In addition, other aspects of survey design are de- pendent on the study objectives: the sampling inter- val, the length of time the survey is conducted (i.e., sampling for acute vs. chronic effects), and the sta- tistical analyses appropriate for analyzing the data (Suterl993). The best interval for sampling is determined by the objectives of the study (Kentulaetal. 1993). If the objective is to detect changes in trends (e.g., regular monitoring for detection of changes in wa- ter quality or biotic integrity), regularly spaced in- tervals are preferred because the analysis is easier. However, if the objective is to assess differences ------- before and after impact, then samples at random time points are advantageous. Random sample in- tervals reduce the likelihood that cyclic differences unforeseen by the sampler will influence the size of the difference before and after the impact. For ex- ample, surveys taken every summer for several years before and after a clear-cut may show little differ- ence in system quality; however, differences may exist that can be detected only in the winter and, therefore, they may go undetected if sampling oc- curs only during summer. The simplest impact survey design involves taking a single survey before and after the impact event (Green 1979). This type of design has the obvious pitfall that there may be no relationship between the observed event and the changes in the response variable—the change may be entirely coincidental. This pitfall is addressed in B ACI design by com- paring before and after impact data with data col- lected from a similar control system nearby. Data are collected before and after a potential distur- bance in two areas (treatment and a control), with measurements on biological and environmental vari- ables in all combinations of time and area (Green 1979). For example, consider a study in which the investigators want to identify the effects of clear- cutting on wetland systems. In the simplest B ACI design, two wetlands would be sampled. One wet- land would be adjacent to the clear-cut (the treat- ment wetland); the other wetland would be adja- cent to a control site that is not clear-cut. The con- trol site should have characteristics (i.e., soil, veg- etation, structure, and functions) similar to the treat- ment wetland and should be exposed to climate and weather similar to the first wetland. Both wetlands are sampled at the same time points before and af- ter the clear-cut occurs. This design is technically known as an area-by-time factorial design. Evi- dence of an impact is found by comparing the con- trol site samples (before and after) with the treat- ment site before and after samples. Area-by-time factorial design allows for both natural wetland-to- wetland variation and coincidental time effects. If the clear-cut has had no effect, then the change in system quality between the two time points should be the same. If the clear-cut has had an effect, then the change in system quality between the two time points should be different. There are some potential problems with B ACI design. First, because the control and impact sites are not randomly assigned, observed differences between sites may be related solely to some other factor that differs between the two sites. One could argue that it is unfair to ascribe the effect to the impact (Hurlbert 1984, Underwood 1991). How- ever, as pointed out by Stewart-Oaten et al. (1986), the survey is concerned about a particular impact in a particular place, not about the average of several impacts when the survey is replicated in many dif- ferent locations. Consequently, it may be possible to detect a difference between these two specific sites. However, if there are no randomized repli- cate treatments, the results of the study cannot be generalized to similar events at different wetlands. However, the likelihood that the differences between sites are due to factors other than the impact can be reduced by monitoring several control sites (Underwood 1991). If one assumes that the varia- tion in the (before and after) measurements of mul- tiple control sites is the same as the variation among potentially impacted sites, and that the variability over time between the control sites is not corre- lated, one can estimate the likelihood that the im- pact caused the observed difference at the impacted site, given the observed variability in the control sites. That is, several control wetlands could be moni- tored at the same time points as the single impact wetland. If the observed difference in the impact wetland is much different than could be expected based on the multiple control wetlands, the event is said to have caused an impact. The lack of ran- domization is less of a concern when several con- trol sites are monitored, because the multiple con- trol sites provide some information about potential effects of other factors. 1 1 ------- The second and more serious concern with the simple before and after design with a single sam- pling point before and after the impact is that it fails to recognize that natural fluctuations may occur in the characteristic of interest that are unrelated to any impact (Hurlbert 1984, Stewart-Oaten 1986). Single samples before and after impact would be sufficient to detect the effects of the impact if no natural fluctuations occurred over time. However, if the population also has natural fluctuations over and above the long-term average, then it is impos- sible to distinguish between cases in which no ef- fect occurs from cases in which an impact does occur. Consequently, measured differences in sys- tem quality may be artifacts of the sampling dates, and natural fluctuations may obscure differences or lead one to believe differences are present when they are not. The simple BACI design was extended by Stewart-Oaten et al. (1986) by pairing surveys at several selected time points before and after the impact to help resolve the issue of pseudoreplication (Hurlbert 1984). This modification of the BACI design is referred to as the BACI-paired series (PS) design. The selected sites are measured at the same time points. The rationale behind this paired design is that repeated sampling before the impact gives an indication of the pattern of differences of poten- tial change between the two sites. BACI-PS study design provides information on the mean difference in the wetland system quality before and after im- pact and on the natural variability of the system qual- ity measurements. An effect is detected if the changes in the mean difference are large relative to natural variability. Considerations for sampling at either random and regularly spaced intervals also apply here. BACI-PS study design also has potential pitfalls. As with all studies, numerous assumptions need to be made during the analysis (Stewart-Oaten et al. 1992, Smith etal. 1993). The primary assumption for BACI-PS design is that the responses over time are independent of each other. A lack of indepen- dence over time tends to produce false-positive (Type I) errors, which may lead a manager to de- clare that an effect has occurred when, in fact, none has. Formal time series analysis methods or re- peated measures analysis may be necessary (Rasmussen et al. 1993) to eliminate Type I errors. (The analysis of time series is easiest with regularly spaced sampling points.) In addition, the differ- ence in mean level between control and impact sites is assumed to be constant over time in the absence of an impact effect. The effect of the impact is as- sumed to change the arithmetic difference. In the clear-cut example given previously, the difference in mean system quality between the two sites is as- sumed to be constant over time. That is, mean sys- tem quality measurements may fluctuate over time, but both sites are assumed to fluctuate in the same manner simultaneously, thereby maintaining the same average arithmetic difference. This assumption is violated if the response variable at the control site is a constant multiple of the response variable at the impact site. For example, suppose that the read- ings of water quality at two sites at the first time point were 200 vs. 100, which has an arithmetic difference of 100, and at the second time point were 20 vs. 10, which has an arithmetic difference of 10, but both pairs are in a 2:1 ratio at both time points. The remedy is simple: A logarithmic transform of the raw data converts a multiplicative difference into a constant arithmetic difference on the logarithmic scale. This is a common problem when system qual- ity measurements are concentrations (e.g., pH). Smith et al. (1993) pointed out that this may not solve the issue of pseudoreplication. Trends are common in most natural populations, but BACI design assumes that trends are not present in the populations or that the control and impact sites have the same trends, so that differences between the sites are identified as associated with the impact, not with differences in trends of natural populations (Smith etal. 1993). Violation of the BACI assump- tions may invalidate conclusions drawn from the data. Enough data must be collected before the impact to identify the trends in the communities of ------- each sampling site if the B ACI assumptions are to be met. Clearly defining the objectives of the study and identifying a statistically testable model of the relationships the investigator is studying can help resolve these issues (Suter 1993). Underwood (1991) also considered two varia- tions on the B ACI-PS design. First, it may not be possible to sample both sites simultaneously for tech- nical or logistical reasons. Underwood (1991) dis- cussed a modification in which sampling is done at different times in each site before and after impact (i.e., sampling times are no longer paired), but notes that this modification cannot detect changes that occurred in the two sites before the impact. For example, differences in system quality may show a gradual change over time in the paired design be- fore impact. Without paired sampling, it would be difficult to detect this change, hi addition, sampling only a single control site still has the problem iden- tified previously, that is, it is not known whether observed differences in the impact and the control sites are site specific. Again, Underwood (1991) suggests that multiple control sites should be moni- tored. The variability in the difference between each control site and the impact site provides informa- tion on transferability of the impact effects to other sites (i.e., it either refutes or supports the site speci- ficity of the impact and associated system response). The designs described are suitable for detecting long-term, or chronic, effects in the mean level of the variable of interest. However, the impact may have a short-term, or acute, effect, or it may change the variability in response (e.g., seasonal changes become more pronounced) in some cases. The sampling schedule can be modified to occur at two temporal scales (enhanced BACI-PS design) that encompass both acute and chronic effects (Underwood 1991). The modified temporal de- sign introduces randomization by randomly choos- ing sampling occasions in two periods (before and after) in the control or impacted sites. The two temporal scales (sampling periods vs. sampling oc- - casions) allow the detection of a change in mean and in yariability after impact. For example, groups of surveys could be conducted every year, with five surveys one week apart randomly located within each group. The analysis of such a design is pre- sented in Underwood (1991). Again, multiple con- trol sites should be used to confound the argument that detected differences are specific to the sampled site. BACI-PS design is also useful when there are multiple objectives. For example, the objective for one variable may be to detect a change in trend. The pairing of sample points on a long time scale leads to efficient detection of trend changes. The objectives for another variable may be to detect differences in the mean level. A short time scale surveys randomly located in time and space are ef- ficient for detecting differences in the mean level. The September 2000 issue of the Journal of Agri- cultural, Biological, and Environmental Statis-. tics discusses many of the advantages and disad- vantages of the B ACI design and provides several examples of appropriate statistical analyses for evalu- ation of BACI studies. SUGGESTED WEBSITES 1 http://ebook.stat.ucla.edu/calculators/ powercalc/ 2 http://www.math.sfu.ca/stats/Courses/Stat- 650/Notes/Handouts/nodel .html 3 http://www.mpl-pwrc.usgs.gov/powcase/ index-html 4 http://www.salnionweb.org/salmonweb/pubs/ pacnwfin.htnil 5 http://trochim.human.comeU.edu/tutorial/flynn/ multivarJilm 6 http://wvvw.ttitts.edu/~gdaUal/STUDY.HTM 7 http://www.umass.edu/tei/mwwp/studydes.html 13 ------- REFERENCES Adamus PR. 1992. Choices in Monitoring Wetlands. In: McKenzie DH, Hyatt DE, McDonald VJ (eds). Ecologi- cal Indicators. New York: Elsevier Applied Science, pp. 571-592. BarbourMT,GerritsenJ,SnyderBD(eds). 1999. Rapid Bioassessment Protocols for Use in Wadeable Streams and Rivers: Periphyton, Benthic Macroinvertebrates, and Fish, 2nd ed. U.S. Environmental Protection Agency, Washington, DC. EPA 841-B-99-002. Beyers DW. 1998. Causal inference in environmental impact studies. J North Am Benthol Soc 17:367-373. Davis SM, Ogden JC (eds). 1994. Everglades: The Ecosystem and Its Restoration. Delray Beach, FL: St. Lucie Press. Detenbeck NE, Taylor DL, Lima A. 1996. Spatial and temporal variability in wetland water quality in the Minneapolis/St. Paul, MN, metropolitan area. Environ Monitor Assess 40:11-40. Dodds WK, Smith VH, Zander B. 1997. Developing nutrient targets to control benthic chlorophyll levels in streams: a case study of the Clark Fork River. Water Res31:1738-1750. Eaton AD, Clesceri LC, Greenberg AE (eds). 2000. Standard Methods for Examination of Water and Wastewater, 19thed. Washington, DC: American Public Health Association. Green RH. 1979. Sampling Design and Statistical Methods for Environmental Biologists. New York: Wiley. Gvvin SE, KentulaME, Shaffer PW. 1999. Evaluating the effects of wetland regulation through hydrogeomorphic classification and landscape profiles. Wetlands 19(3):477-489. Harris SC, Martin TH, Cummins KW. 1995. A model for aquatic invertebrate response to Kissimmee River restoration. Restoration Ecol 3:181 -194. Hayek LC. 1994. Research Design for Quantitative Amphibian Studies. In: Measuring and Monitoring Biological Diversity, Standard Methods for Amphib- ians. Washington, DC: Smithsonian Institution Press. HurlbertSH. 1984. Pseudo-replication and the design of ecological field experiments. Ecol Monogr 52:187-211. Johnston CA, Detenbeck NE, Bonde JP, Niemi GJ. 1988. Geographic information systems for cumulative impact assessment. PhotogrEngin Remote Sens 54:1609- 1615. Johnston CA, Detenbeck NE, Niemi GJ. 1990. The cumulative effect of wetlands on stream water quality and quantity: A landscape approach. Biogeochemistry 10:105-141. Kadlec RH, Knight RL. 1996. Treatment Wetlands. Boca Raton, FL: Lewis Publishers. Karr JR, Chu EW. 1999. Restoring Life in Running Waters: Better Biological Monitoring. Washington, DC: Island Press. Kentula ME, Brooks RP, Gwin SE, Holland CC, Sherman AD, Sifheos JC. 1993. An Approach to Improving Decision Making in Wetland Restoration and Creation. Boca Raton, FL: CK Smoley. Leibowitz SG, Abbruzzese A, Adamus PR, Hughes LE, Irish JT. 1992. A Synoptic Approach to Cumulative Impact Assessment: A Proposed Methodology. Environmental Research Laboratory, U.S. Environmen- tal Protection Agency, Corvallis, OR. EPA/600/R-92/ 167. McCormick P, Newman S, Miao S, Reddy R, Gawlick D, FitzC, Fontaine T,MarleyD. 1999. Ecological Needs of the Everglades. In: Redfield G (ed). Everglades Interim Report. South Florida Water Management District, West Palm Beach, FL. Mitsch WJ, Gosselink JG. 1993. Wetlands, 2nd ed. New York: Van Nostrand Reinhold. Osenberg CW, Schmitt RJ, Holbrook SJ, Abu-Saba KE, Flegal R. 1994. Detection of environmental impacts natural variability, effect size, power analysis. Ecol Appl4:16-30. Palik AJ, Goebel CP, Kirkman KL, West L. 2000. Using landscape hierarchies to guide restoration of disturbed ecosystems. Ecol Appl 10(1):189-202. 14 ------- Paulsen SG, Larsen DP, Kaufinann PR, Whittier TR, Baker JR, Peck DV, McGue J, Hughes RM, McMullen D, Stevens D, Stoddard JL, Lazorchak J, Kinney W, Selle AR, Hjort R. 1991. EMAP-Surface Waters Monitoring and Research Strategy: Fiscal year 1991. Office of Research and Development, U.S. Environ- mental Protection Agency, Washington, DC. EPA/600/ 3-91/022. Poole RW. 1972. An Introduction to Quantitative Ecology. New York: McGraw-Hill. Rasmussen PW, Heisey DM, Nordheim EV, Frost TM. 1993. Time Series Intervention Analysis: Unreplicated Large-scale Experiments. In: Scheiner SM, Gurevitch J (eds). Design and Analysis of Ecological Experiments. New York: Chapman and Hall. RedfieldG (ed). 1999. Everglades Interim Report. South Florida Water Management District, West Palm Beach, FL. RedfieldG (ed). 2000. EVerglades Consolidated Report. South Florida Water Management District, West Palm Beach, FL. Redfield G (ed). 200,1. Everglades Consolidated Report. South Florida Water Management District, West Palm Beach, FL. Smith EP, Orvos D, Cairns J, Jr. 1993. Impact assessment using before-after control-impact (BACI) models: Concerns and comments. Can J Fish AquatSci 50:627- 637. Stevenson RJ. 1996. An Introduction to Algal Ecology in Freshwater Benthic Habitats. In: Stevenson RJ, Bothwell M, Lowe RL (eds). Algal Ecology: Freshwa- ter Benthic Ecosystems. San Diego, CA: Academic Press, pp. 3-33. Stevenson RJ. 1997. Scale dependent determinants and consequences of benthic algal heterogeneity. J North AmBentholSoc 16(l):248-262. Stewart-Oaten A. 1986. Assessing local impacts: progress and some problems. Oceans '86 Conference Record3:964-973. Stewart-JOaten A, Bence JR, Osenberg CW. 1992. Assessing effects of unreplicated perturbations: No simple.solutions. Ecology 73:1396-1404. Stewart-Oaten A, Murdoch WW, Parker KR. 1986. Environmental impact assessment: "Pseudoreplication" in time? Ecology 67:929-940. SuterGW. 1993. Ecological Risk Assessment. Boca Raton, FL: Lewis Publishers. Tate CM. 1990. Patterns and controls of nitrogen in tallgrass prairie streams. Ecology 71:2007-2018. Toth LA, Arrington DA, Brady MA, Muszick DA. 1995. Conceptual evaluation of factors potentially affecting restoration of habitat structure within the channelized . Kissimmee River ecosystem. Restoration Ecol 3:160- 180. Trexler JC. 1995. Restoration ofthe Kissimmee River—A conceptual model of past and present fish communi- ties and its consequences for evaluating restoration success. Restoration Ecol 3:195-210. Underwood A J. 1991. Beyond BACI: Experimental designs for detecting human environmental impacts on temporal variations in natural populations. Austral J Marine Freshw Res 42;569-587. UrquhartNS, Paulsen SO, Larsen DP. 1998. Monitoring for policy-relevant regional trends overtime. Ecol Appl8(2):246-257. U.S. Environmental Protection Agency (EPA). 1990. Water Quality Standards for Wetlands: National Guidance. Office of Water, Washington, DC. EPA 440/ 5-90-011. U.S. EPA. 1996. National Water Quality Inventory 1996 Report to Congress. Office of Water, Washington, DC. EPA841-R-97-008. U.S. EPA. 1998. National Water Quality Inventory 1998 Report to Congress. Office of Water, Washington, DC. EPA841-S-00-001. U.S. EPA. 2000. Nutrient Criteria Technical Guidance Manual: Rivers and Streams. Office of Water, Wash- ington, DC. EPA 822-B-00-002. ------- ------- |