United States Environmental Protection Agency Office of Water Washington, B.C. 20460 841-F-9*3-009 September 1993 wEPA Paired Watershed Study Design INTRODUCTION The purpose of this fact sheet is to describe the paired watershed approach for conducting nonpoint source (NFS) water quality studies. The basic approach requires a minimum of two watersheds - control and treatment - and two periods of study - calibration and treatment. The control watershed accounts for year-to-year or seasonal climate variations, and the management practices remain the same during the study. The treatment watershed has a change in management at some point during the study. During the calibration period, the two watersheds are treated identically and paired water quality data are collected (Table 1). Such paired data could be annual means or totals, or for shorter studies (<5 yr), the observations could be seasonal, monthly, weekly, or event-based. During the treatment period, one watershed is treated with a best management practice (BMP) while the control watershed remains in the original management (Table 1). The treated watershed should be selected randomly by such means as a coin toss. The reverse of this schedule is possible for certain BMPs; the treatment period could precede the calibration period. For example, the study could begin with two watersheds in two different treatments, such as "BMP" and "no BMP". Later both watersheds could be managed identically to calibrate them. Since no calibration exists before the treatment occurs, this reversed design is considered risky. Table 1. Schedule of BMP implementation. Period Watershed Control Treated Calibration Treatment no BMP no BMP no BMP BMP The basis of the paired watershed approach is that there is a quantifiable relationship between paired water quality data for the two watersheds, and that this relationship is valid until a major change is made in one of the watersheds. At that time, a new relationship will exist. This basis does not require that the quality of runoff be statistically the same for the two watersheds; but rather that the relationship between paired observations of water quality remains the same over time except for the influence of the BMP. Often, in fact, the analysis of paired observations indicates that the water quality is different between the paired watersheds. This difference further substantiates the need to use a paired watershed approach because the technique does not assume that the two watersheds are the same; it does assume that the two watersheds respond in a predictable manner together. EXAMPLE To illustrate the paired watershed approach, data taken from a study in Vermont will be used. The purpose of the study was to compare changes in field runoff (cm) due to conversion of conventional tillage to conservation tillage. ------- Selection of Watersheds 1. Watersheds should be similar in size, slope* location, soils, and land cover. 2. Watersheds should be small enough to obtain uniform treatment over the entire watershed. 3. Watershed outlets should have a stable channel and cross section for discharge monitoring, and should not leak at the outlet. 4. Each watershed should be in the same land cover for a number of years prior to the study so that they are at a steady-state.; Advantages , , 1. Climate and hydrologic differences over years are statistically controlled. 2. Can attribute water quality changes to a treatment. 3. Control watershed eliminates need to measure all components causing change. 4. Watersheds need not be identical, 5. Study can be completed in shorter time frame than trend studies. 6, Cause-effect relationships can be indicated, Disadvantages 1. Response to treatment likely to be gradual over tiros which influences the variance. 2. Study vulnerable to catastrophes such as hurricanes. 3. Shortened calibration may result in serially correlated data. 4. Variances between time periods may not be equal due to drastic treatment. 5. Minimal change in the control watershed is permitted. 6. Requires similar watersheds in close proximity. The west watershed was the control and was 1.46 hectares (ha) in area. The east watershed was the treatment field and was 1.10 ha. Conventional tillage was moldboard plow whereas conservation tillage was a single disk harrow. The calibration period was one year during which 49 paired observations of storm runoff were "made. The treatment period was three years during which 114 paired observations of runoff were made.. Data were log-transformed to approach normality based upon the Wilks-Shapiro (W) statistic. The equality of variances between periods was tested using the F- test. Residual plots were examined to check for independence of errors. The statistical package SAS® was used for all analyses. CALIBRATION The relationship between watersheds during the calibration period is described by a simple linear regression (Figure 1) ------- between the paired observations, taking the form: treated{ = b0 b^controfy + e (1) where treated and control represent flow, water quality concentration, or mass values for the appropriate watershed, b0 and bl are regression coefficients representing the regression intercept and slope, respectively, and e is the residual error. Three important questions must be answered prior to shifting from the calibration period t® the treatment period: a) is there a significant relationship between the paired watersheds for all parameters.of interest, b) has the calibration period continued for a sufficient length of time, and c) are the residual errors about the regression smaller than the expected BMP effect? Regression significance. The significance of the relationship between paired observations is tested using analysis of variance (ANOVA). The test assumes that the regression residuals: are normally distributed, have equal variances between treatments, and are independent. Hand calculations to test for the significance of the relationship are shown in Snedecor and Cochran (1980, p. 157) (Table 2). The values for Table 2 are calculated from: (2) n P2 sx -- n n - 2- (3) (4) (5) Also,] the regression coefficients and coefficient of determination are determined from: (6) - b0 -7 - (7) (8) Table 2. Analysis of variance for linear regression. Source regression residual total Degrees of freedom 1 n-2 n-1 Sum of squares 4 Mean squares------- In order to perform the calculations by hand, initially calculate: SXb SYj, SXjYj, EXr2, £Yj2,.X , Y. The^mean squares (MS) are determined by dividing the sum of squares by the degrees of freedom (df). For the example above, the following was calculated by hand: SX; = -123.403, SYj = -180.704, SXjYj = 533.553, EX;2 = 381.713, SY;2 = 814.847, _X= -2.518 (10"x =0.003041 cm), and 7= -3.688 (10Y = 0.000205 cm). Therefore, 'Sj = 148.441, S = 78.463, S? = 70.933, and Sj= 1.312. Using SAS, the appropriate program is listed below. This program was used to generate Table 3. Table 3. Analysis of variance for regression of treatment watershed runoff on control watershed runoff. Source df MS model error total 1 47 48 86.79 66.17 1.31 0.0001 has been taken to detect that difference, from: (9) SAS PC Program data flow; title 'Total Flow (cm)1; infile 'fhame.dat*; input flowl flow2; logflowl=loglQ(flowl); Iogflow2=logl0(flow2); Proc reg; Model Iogffow2~logflowl ; /PCLM; runj The resulting F statistic for this example would indicate that the regression relationship adequately explains a significant amount (p< 0.001) of the variation in paired flow data. Calibration duration. The ratio between the residual variance (mean squares) for the regression and the smallest worthwhile difference (d) is used to determine if a sufficient, sample where 5^ is the estimated residual variance about the regression, d2 is the square of the smallest worthwhile difference, nj and n2 are the numbers of observations in the calibration and treatment periods (HJ = n2 for this calculation because n2 is not known yet), and F is the table value (p=0.05) for the variance ratio at 1 and nx + n2 - 3 df. The difference (d) is selected based on experience and would vary with project expectations. If the left side of the equation is greater than the right side of the equation, then there are an insufficient number of samples taken to detect the difference. For the example, Sy was 1.312 (from Table 3), i^ = n2 was 49, and F was 3.94. A ten percent change from the mean was considered a worthwhile_ difference; therefore, d = 0.10 * X = 0.10 * log 0.003041 cm and sj/d2 = 20.7. The right side of Equation (9) = 6.0; since 20.7 is greater than 6.0, there------- •d H ID fa g15^ to 10'- id |