United States Office of Water EPA 816-R-01-016
Environmental Protection (4606) March 2001
Agency
Study Design for Evaluating
Impacts to Underground
Sources of Drinking Water by
Hydraulic Fracturing of
Coalbed Methane Reservoirs
-------
Final Study Design for Evaluating of Impacts
to Underground Sources of Drinking Water
by Hydraulic Fracturing of Coalbed Methane
Reservoirs
1.0 INTRODUCTION
1.1 Study Background
Coalbed gas production through wells began in the 1970's as a safety measure in underground
coal mines to reduce the explosion hazard posed by methane (Elder and Deul, 1974). In 1980,
the U.S. Congress enacted a tax credit for non-conventional fuels production, including coalbed
methane, as part of the Crude Oil Windfall Profit Act. In 1984, there were fewer than 100
coalbed wells in the U.S. By 1990, however, with the anticipated expiration of the tax credit
almost 8,000 coalbed wells had been drilled nationwide (Pashin and Hinkle, 1997). In 1996,
coalbed methane production in 12 states totaled about 1,252 billion cubic feet, accounting for
approximately seven percent of U.S. gas production (U.S. Department of Energy, 1999).
According to the U.S. Department of Energy, natural gas demand is expected to increase at least
45% in the next 20 years (U.S. Department of Energy, 1999a). The rate of coalbed methane
production is also expected to increase in response to the growing demand.
Although the use of natural gas has many environmental benefits over traditional energy sources,
concerns have been raised regarding the environmental impacts of coalbed methane production.
Coalbed methane production in certain areas has led to alleged ground water depletion and
produced water discharge issues. Citizens, state agencies, producers, and the regional EPA
offices in those areas are working in concert to understand and mitigate potential problems.
Separate from ground water depletion and water discharge issues are allegations that hydraulic
fracturing of coalbed methane wells has affected the quality of ground water. State oil and gas
agencies receiving citizen complaints have stated that, based on their investigations, hydraulic
fracturing did not contribute to water quality degradation.
The State of Alabama recently adopted hydraulic fracturing regulations into its Underground
Injection Control (UIC) program in response to an 11th Circuit Court of Appeals (hereafter, "the
Court") decision (LEAF v. EPA, 118F.3d 1467). Prior to the Court's decision, EPA had not
considered hydraulic fracturing as underground injection because it did not regard production
well stimulation as an activity subject to regulation under SDWA's UIC program. However, the
Court held that the injection of fluids for the purpose of hydraulic fracturing constitutes
underground injection as defined in the Safe Drinking Water Act (SDWA); that all underground
injection must be regulated; and that hydraulic fracturing of coalbed methane wells in Alabama
was not regulated under Alabama's UIC program. Stakeholders are interested as to whether or
not and to what extent hydraulic fracturing in coalbed methane could be a national issue.
-------
Given that coalbed methane production will likely increase and given that EPA has received
reports from citizens in different states alleging hydraulic fracturing resulted in ground water
degradation, EPA believes further investigation is necessary to evaluate the potential risks.
1.2 Purpose
The purpose of the fact-finding study described herein is to assist EPA in determining if
hydraulic fracturing of coalbed methane wells poses a threat to underground sources of drinking
water. EPA intends to complete this study before making regulatory or policy decisions
regarding hydraulic fracturing.
In July, 2000 EPA published a Federal Register notice (Volume 65, Number 143, [Page 45774-
45775]) requesting comment on a conceptual study design in order to receive stakeholder input
on how an EPA study should be structured. EPA received more than 80 sets of comments from
industry, state oil and gas agencies, environmental groups and individual citizens. A summary of
comments can be viewed on EPA's Web site. Hard copies can be ordered by contacting EPA's
Water Resource Center by phone at (202) 260-7786, or by e-mail to center.water-
resource@epa.gov or by conventional mail to EPA Water Resource Center, RC-4100, Ariel Rios
Building, 1200 Pennsylvania Avenue, N.W., Washington, D.C. 20460.
1.2 What is Hydraulic Fracturing?
Hydraulic fracturing is the initiation and propagation of a fracture in a rock matrix by means of
hydraulic pressure. The hydraulic fracturing process uses very high pressures to initiate a fracture
at a specific depth in the coal formation. The length and height of this fracture is dependent upon
a number of geological properties of the rock in which the fracture is initiated and the rock types
that bound that interval. Due to the loading forces contributed by the rock above the injection
zone, hydraulic fractures that are initiated at depths greater than 1,500 feet below the surface are
typically vertical fractures. Hydraulic fracturing operations initiated at shallower depths may
create horizontal fractures under ideal situations. Rather than a branched network of major and
minor fractures, hydraulic fracturing creates fractures with vertical, two-dimensional fan-shaped,
planar features, having two "lobes" centered on the wellbore perforations (Diamond, 1987;
Morales et al, 1990).
One or more stimulation fluids is used in each hydraulic fracturing event. The composition of
these fluids include water, hydrochloric acid, "slick" water (which includes a viscosity reducer),
gels, and nitrogen foam (Palmer et al, 1993). Included with most stimulation fluids are
"proppants" which are sand grains or glass, plastic or ceramic beads that serve to hold the
hydraulic fracture open after the initiating pressure is released. The most common proppant used
in coalbed methane treatments is sand. The amount of proppant injected for each fracture
treatment ranges from 10,000 to 120,000 pounds (Holditch et al, 1989; Palmer et al, 1991a,
1993). Fracture widths in the formation vary from 0.5 inches to closed (i.e., no proppant used),
depending on the distance from the wellbore and the efficiency of the proppant displacement into
the fracture (Palmer and Sparks, 1990; Palmer et al, 1993; Steidl, 1993).
1.3 Summary of EPA's Study Approach
-------
EPA will perform an investigation of the potential impacts on underground sources of drinking
water (USDWs) from hydraulic fracturing of coalbed methane wells. Given the enormous
variation in geology among and within coalbed basins in the U.S., any evaluation of potential
impacts at a national level will necessarily be broad and general in scope. In order to best utilize
resources in investigating this issue, EPA will conduct the study in three phases, narrowing its
focus from general to more specific as needed. At the completion of each phase, EPA will decide
if findings warrant continuation into the next phase
Phase I of the study will be a limited-scope assessment designed to enable the Agency to
determine if hydraulic fracturing of coalbed methane wells clearly poses little or no threat to
USDWs, or if the practice may pose a threat and require further investigation. In Phase I, EPA
will
• Gather existing information to review hydraulic fracturing processes, practices, and
settings;
• Request public comment to identify incidents that have not been reported to EPA; and,
• Review reported incidents of ground water contamination and any follow-up actions or
investigations by other parties (State or local agencies, industry, academia, etc.). Phase I
will include a review of the results of the GSA study described above. Once the
information gathered through the Phase I effort is analyzed, EPA will make a
determination regarding whether further investigation is needed.
The decision to continue with Phase II will be based on the results of the Phase I study and
consideration of other priorities within the Office of Ground Water and Drinking Water. EPA
will not continue into Phase II of the study if the investigation finds no hazardous constituents
are used in fracturing fluids, hydraulic fracturing does not increase the hydraulic communication
between previously isolated formations, and reported incidents of water quality degradation can
be attributed to other, more plausible causes.
If OGWDW moves to Phase II, it would likely include site investigation(s) of proposed physical
or chemical mechanisms that potentially lead to USDW endangerment and an estimate of
national risk. If OGWDW believes Phase II findings indicate significant risk exists, EPA would
move to Phase III of the study. Phase III would consist of an evaluation of existing regulations
and a policy determination. EPA will not draft a detailed approach for conducting Phases II and
III unless and until the Agency makes decisions to continue the study into each subsequent
phase.
EPA will take several steps to involve the public during the course of the study. Those steps
include publishing Federal Register Notices requesting comments on study plans and draft work
products; providing periodic updates for stakeholders in the form of written communication; and
maintaining a web site where stakeholders can view the project documents and provide
information to EPA.
-------
In addition to efforts to review hydraulic fracturing at a national level, EPA has jointly funded a
site-specific study, which will be conducted by the Geological Survey of Alabama (GSA). This
study will attempt to address a concern that is central to contamination and drawdown issues -
the degree to which flow is confined within coal beds in coalbed methane fields (GSA, June,
2000). Additional information is available on the GSA's web site at
http://www.gsa.state.al.us/gsa/3DFracpage/3Dfracstudy.htm.
2.0 PHASE I STUDY OBJECTIVES - MECHANISMS THAT MAY IMPACT
USDWs
The first step in investigating the potential for hydraulic fracturing to endanger USDWs is to
define mechanisms by which endangerment could occur. EPA defined two hypothetical
mechanisms by which hydraulic fracturing of coalbed methane wells could potentially impact
USDWs:
1. The intentional direct injection of fracturing fluids into a USDW; and,
2. Creation of a hydraulic communication between the target coalbed formation and
adjacent USDWs.
The objective of the Phase I study is to consider these two mechanisms, based on existing
literature and data, when evaluating whether hydraulic fracturing endangers USDWs.
Fluids can be injected into a USDW (mechanism #1 above) directly or indirectly depending on
the location of the coalbed relative to a USDW. In many coalbed-methane producing regions, the
target coalbeds occur within USDWs, and the fracturing process injects stimulation fluids
directly into the USDWs (see Figure 1). In other production regions, target coalbeds are adjacent
to the USDWs that exist either higher or lower in the geologic section.
Once the fluids are injected, local geologic conditions may interfere with their complete
recovery. This may result in fracturing fluids being "stranded" in a USDW. If hazardous
materials are included in the stimulation fluids, they could potentially endanger a USDW and
any drinking water supplies relying on the USDW.
The second hypothetical mechanism is the creation of hydraulic communication, or a pathway,
between the target coalbed and an adjacent USDW that may cause associated water quality and
quantity impacts within the USDW, separate from the injection of fluids as described above.
Hydraulic fracturing in coalbed methane formations typically creates fractures that are taller than
they are wide (Morales et al, 1990; Zuber, 1990; Hoi ditch et al, 1989; Palmer et al, 1991, 199 la,
1993), and the potential exists for fractures to extend through the stratigraphic layers that
separate coalbeds and USDWs. Some stakeholders have theorized that either stimulation fluids
containing hazardous materials or more saline ground water may enter a USDW through newly-
formed pathways created from hydraulic fracturing (see Figure 2) and water quality may be
affected from this hydraulic communication between coalbeds and adjacent USDWs.
It will be difficult to properly evaluate the potential for hydraulic fracturing to create hydraulic
communication between coalbeds and adjacent stratigraphic layers as a Phase I study element. In
-------
order to make a defensible determination that hydraulic fracturing has impacted an aquifer, it
would be necessary to have site-specific data concerning:
• Water quantity and quality conditions in a USDW (or a well) both before and after a
fracturing event;
• Location, dimensions, and conductivity of fractures created during the coalbed
stimulation event;
• Measured changes in ground water flow between the USDW and coalbeds or other
aquifers; and
• Impacts of other, unrelated, hydrologic and water quality processes that could also be
affecting the USDW.
The extent to which these data are available to properly support an analysis of national scope is
unknown, but the effort necessary to collect and evaluate them fully is beyond the scope of Phase
I. EPA believes the first mechanism described in this section, fluid injection directly into a
USDW, presents the more plausible scenario of the two endangerment scenarios presented.
However, the data that are collected as part of Phase I will be analyzed to determine if
information exists to draw conclusions regarding the impacts of creating a hydraulic connection
between a coalbed and a USDW, and if so, this information will be included in the Phase I
report.
3.0 STUDY COMPONENTS
EPA will research the topic areas described below to evaluate the impacts of hydraulic fracturing
on USDWs. Information will be collected regarding the geology and hydrogeology of the
coalbed methane production regions (Section 3.1), the processes used to hydraulically fracture
coalbed methane production wells (Section 3.2), the fluids used in the fracturing process (Section
3.3) and the techniques that may be used to monitor the effects of fracturing (Section 3.4). EPA
will also evaluate water supply incidents possibly related to hydraulic fracturing of coalbed
methane production wells (Section 3.5). EPA will rely on currently available literature and data
as the primary source of information for Phase I efforts.
3.1 Hydrogeology of Coalbed Methane Production Regions
The study will evaluate, on a regional basis, the hydrogeologic setting of each coalbed methane
production region. Local hydrogeologic conditions vary in each coalbed methane production
region in the country, and, as a result, the potential impacts of the hydraulic fracturing process
may also vary.
The basic geology of each region will be evaluated, including the location and extent of the
coalbed formations. The study will also identify the depth and hydrologic relationships between
USDWs and coalbed methane wells by identifying where coalbed formations are located within
or directly adjacent to currently mapped USDWs. This information will be used to evaluate if
hydraulic fracturing of methane production wells could result in the injection of fluids into
USDWs as described in Section 2.0 above.
3.2 Processes Used to Hydraulically Fracture Coalbed Methane Formations
-------
Information will be obtained to describe the basic processes used in coalbed methane production,
including well construction information, water and methane withdrawal data. The ways in which
methane production practices vary by region will be evaluated and described.
If coalbed formations are identified in proximity to mapped USDWs as determined in the
hydrogeologic analysis (Section 3.1), the potential to create a hydraulic communication between
the target coalbeds into the USDW will be evaluated considering:
• The depth of separation between coalbed formations and USDWs;
• The petro-physical characteristics and susceptibility (e.g., the in-situ vertical stress
distribution) within intervening rock layers to fracturing; and
• The general characteristics and dimensions of fracturing techniques utilized in the
particular region or field from readily available data.
3.3 Hydraulic Fracturing Fluids and Proppants
EPA will collect the types of information listed below to address the potential effects of injecting
fracturing fluids directly into a USDW:
• The composition of hydraulic fracturing fluids, including potentially hazardous materials;
• The types of proppant materials injected with the hydraulic fracturing fluids;
• Quantities of materials injected;
• Data on the extent of fractures, specifically in coalbed formations; and
• Recovery of fluids during methane production following a fracturing event, and the
corresponding loss of stranded fluids in the rock matrix.
Because practices may vary from one region to another based on the nature of the coal seams
being produced, the study will describe the variation in fluids used for hydraulic fracturing
throughout the country.
3.4 Techniques Used to Monitor and Model the Hydraulic Fracturing Process
The project team will document how the hydraulic fracturing process and its effects are modeled
and monitored. The study will evaluate what types of instruments are used to document a
fracturing event, and what types of models are used to predict the extent of fracturing. A review
of modeling and monitoring information will help document how accurately the extent of
fracturing in coalbed formations can be predicted, and how useful these data may be in
evaluating the issues associated with this study.
3.5 Reports of Alleged Contamination Cases
In 1998, the Ground Water Protection Council conducted a survey of state oil and gas agencies
to determine if those agencies were aware of any cases in which hydraulic fracturing (CBM)
adversely impacted water wells. Of those 25 states that responded to the survey, thirteen state oil
and gas agencies reported coalbed methane wells. Each of those thirteen State agencies reported
they were not aware of any case in which hydraulic fracturing impacted water wells (Ground
Water Protection Council, 1998). In an effort to be thorough, EPA believes it should offer other
-------
drinking water agencies and the public at large an opportunity to provide information to EPA on
any impacts to ground water believed to be associated with hydraulic fracturing. EPA intends to
offer this opportunity through a request for public comment.
Reports of alleged ground water impacts due to hydraulic fracturing of coalbed methane wells
that are received through the request for public comment will be reviewed in conjunction with
the appropriate State agency. EPA will evaluate reports based on State agency responses and
available data to determine if sufficient information exists to understand the source of water
quality issues, or if additional data should be collected. The most pertinent information to
evaluate in these reviews will be:
• basic construction details of allegedly impacted wells,
• the timing of the alleged impact relative to fracturing events at nearby methane
production wells;
• nature of the alleged water quality problems;
• baseline and subsequent water quality data; previous industrial activity in the area of
concern; and
• climate data.
To the extent possible, based on the available incident data, EPA will evaluate where or when
USDW degradation has occurred specifically from hydraulic fracturing of coalbed methane
production wells.
4.0 DATA COLLECTION
The study of potential impacts associated with hydraulic fracturing will be based on existing
information on hydraulic fracturing as utilized in coalbed methane production. The ability to
evaluate the study objectives is based on the amount of information available from existing
literature sources, academic institutions, industry organizations, federal, state and local agencies,
and individuals' reports.
Data collection for the study will follow a four-tiered approach. First, EPA will review existing
industry, academic, and government agency resources and collect pertinent information. Second,
contacts will be made with State and federal agencies involved in coalbed methane production.
These will include the U.S. Geological Survey, the U.S. Department of Energy, State water
supply agencies, State oil and gas boards, and the Groundwater Protection Council. Third, EPA
will contact State agencies to discuss the history and nature of complaints received from citizens
and any follow-up conducted in relation to those complaints. Fourth, to the extent possible within
this project, visits will be made to targeted coalbed methane production areas to collect site-
specific data on methane production and hydraulic fracturing practices. Depending on the
information available from State agencies, EPA may review locations and construction details of
subject water wells, timing and locations of fracturing events, and other data detailing conditions
that may have contributed to problems in water wells.
Quality Assurance (QA) procedures for data collection are included in Appendix A, Section 3.1.
Section 3.2 of the QA protocol addresses procedures for data logging.
-------
5.0 DATA ANALYSIS
Once data have been collected, EPA will evaluate the information to determine if hydraulic
fracturing of coalbed methane wells clearly poses little or no threat to USDWs, or if the practice
may pose a threat and require further investigation. EPA will decide if Phase II is warranted
based on Phase I results, resources available to EPA to conduct further investigation, priorities of
the Agency at that time, and whether the scope of the issue is national or localized.
6.0 STAKEHOLDER INVOLVEMENT
Maximizing stakeholder input to the study process is a priority of EPA. To that end, efforts will
be made to identify all stakeholders to the greatest extent possible. Stakeholder input has been
solicited on this study methodology and will be solicited on the draft summary report. Postings
on relevant web sites, mailings, public meetings, public notices, and other means, will be used to
involve industry, environmental groups, citizen groups, academia, drinking water agencies, and
other governmental and non-governmental agencies. EPA will obtain input from experts
(industry, academia, and government agencies) on the issues of coalbed methane production and
hydraulic fracturing, in order to peer-review data sources, analytical approaches, and
conclusions. The review process will include:
• EPA internal review conducted by Offices outside the Office of Ground Water and
Drinking Water;
• Inter-agency review requested from Department of Energy, U. S. Geologic Survey,
Bureau of Land Management, and appropriate state agencies;
• Independent scientific peer review conducted by an EPA-convened panel; and
• Stakeholder review.
Further discussion of the quality assurance aspects of the review process is included in Section
4.0 of the QA protocol. This review process will result in an official record of comments on
study methods and results, and a determination of whether the study conclusions can be
supported by the data.
7.0 REPORTING
Reporting of study methods, data sources, data analysis, and conclusions will be made in a clear,
accurate, and complete manner. The body of technical information considered during the study
will be recorded and available for review by stakeholders. In addition, the manner for evaluating,
analyzing, and drawing conclusions from the collected data will be communicated in order that
reviewers may observe and validate methods. Limitations on where a conclusion may apply, and
alternative interpretations of data, will be documented. Overall, the conclusions that address the
objectives of the study will be supported by the results of peer-reviewed data evaluation and
analysis. EPA will post materials to the internet periodically to inform stakeholders of progress
on the study. The stakeholders will have an additional opportunity for input when the draft study
is released. Notice of availability will be made through mailings to individual stakeholders.
8.0 SCHEDULE AND DELIVERABLES
-------
EPA estimates the study as described herein will be completed by February 2002. EPA expects
to complete the literature search by April 2001; a request for public comment in May, 2001; and
follow-up of reported incidents in August, 2001. A draft report is expected to be completed and
available for comment three months after data collection.
Deliverables for Phase I of the study will include a study methodology and draft and final
reports.
9.0 REFERENCES
Diamond, W.P. 1987. Characterization of fracture geometry and roof penetrations associated
with stimulation treatments in coalbeds, Proceedings of the 1987 Coalbed Methane Symposium,
U. of Alabama, p. 243.
Elder, C. and M. Deul. 1974. Degasification of the Mary Lee coalbed near Oak Grove, Jefferson
County, Alabama, by vertical borehole in advance of mining, U.S. Bur. Of Mines Report 7968.
Holditch, S., J. Ely and R. Carter. 1989. Development of a coal seam fracture design manual,
Proceedings 1989 Coalbed Methane Symposium, Tuscaloosa, Alabama, pp. 299-320.
Hansen, C. 2000. Interstate Oil and Gas Conservation Commission August 15, 2000 Comments
in Response to FRL6839-8, Oklahoma City, OK, pp. 1-3.
Geological Survey of Alabama. 2000. Grant Application for 3-D Characterization of Natural and
Induced Fractures in Coalbed Methane Reservoirs of the Black Warrior Basin in Alabama,
Tuscaloosa, AL.
Ground Water Protection Council. 1998. Survey Results on Inventory and Extent of Hydraulic
Fracturing in Coalbed Methane Wells in the Producing States, Oklahoma, City, OK.
Morales, R., J. McLennan, A. Jones andR. Schraufnagel. 1990. Classification of treating
pressures in coal fracturing, Proceedings of the 31st U.S. Symposium on Rock Mechanics, 31,
pp. 687-694.
Palmer, I, N. King and D. Sparks. 1993. The character of coal fracture treatments in the Oak
Grove Field, Black Warrior Basin, In Situ, J. of Coal Research, v. 17(3), pp. 273-309.
Palmer, I, N. King and D. Sparks. 1991. The character of coal fracture treatments in Oak Grove
Field, Black Warrior Basin, SPE Paper 22914, Proceedings, 1991 Society of Petroleum
Engineering Annual Technical Conference and Exhibition, pp. 277-286.
Palmer, I, R. Fryar, K. Tumino and R. Puri. 1991a. Comparison between gel-fracture and water-
fracture stimulations in the Black Warrior Basin, Proceedings 1991 Coalbed Methane
Symposium, University of Alabama, pp. 233-242.
-------
Palmer, I. and D. Sparks. 1990. Measurement of induced fractures by downhole TV camera in
coalbeds of the Black Warrior Basin, Soc. Of Petroleum Engineers Paper 20660, Proceedings,
1990 Soc. Of Petroleum Engineers Annual Technical Conference and Exhibition, pp. 445-458.
Pashin, J. and F. Hinkle. 1997. Coalbed methane in Alabama, Geol. Surv. Of Alabama, Circular
192, 71pp.
Steidl, P. 1993. Evaluation of induced fractures intercepted by mining, Proceedings 1993
Coalbed Methane Symposium, University of Alabama, pp.675-686.
U.S. Department of Energy. 1999. U.S. Crude Oil, Natural Gas, Natural Gas Liquids Reserves
1999 Annual Report, p. 34.
U.S. Department of Energy. 1999a. Environmental Benefits of Advanced Oil and Gas
Exploration and Production Technology, Office of Fossil Energy, p 8.
Zuber, M., V. Kuuskraa and W. Sawyer. 1990. Optimizing well spacing and hydraulic fracture
design for economic recovery of coalbed methane, SPE Formation Evaluation, v.5(l):98-102.
Zuber, M. 1997. Production characteristics and reservoir analysis of coalbed methane reservoirs,
International Journal of Coal Geology (Appalachian Coalbed Methane), v.38(1-2).
APPENDIX A
QUALITY ASSURANCE PROTOCOL
The current strategic plan of the U.S. Environmental Protection Agency (EPA), published in
1997 in response to the Government Performance and Results Act, acknowledges that
environmental protection efforts need to be "based on the best available scientific information,"
and "sound science." Although this goal sounds straightforward, achieving it in practice is
complicated and challenging. However, the credibility of the resulting policy decision depends,
to a large extent, on the strength of the scientific evidence on which it is based. This Quality
Assurance Process for data collection describes the procedures to be used for a systematic and
well-documented approach to realizing this goal for the Hydraulic Fracturing Study Design and
Methodology.
It is imperative that this quality assurance process produces a set of data and scientific findings
that are sound and conclusions that are supported by the data. "Sound science can be described as
organized investigations and observations conducted by qualified personnel using documented
methods and leading to verifiable results and conclusions (SETAC, 1999)." In order to ensure
that findings are sound, the following quality assurance questions will be addressed for all
sources of data:
• What was the purpose of the study?
-------
• Whose data is it?
• What is their source?
• Are the data reliable?
• Is the interpretation biased?
This quality assurance process establishes a set of guidelines and general approaches to assess
available data and information in a clear, consistent, and explicit manner. Data collection and
review according to this process will make conclusions more transparent and thus more readily
understood and communicable to stakeholders. In this study, the quality assurance process will
proceed even when the data and scientific information is insufficient. Areas lacking data will be
recorded and discussion will ensue on the role of scientific uncertainty in the quality assurance
process.
The objectives of the systematic expert review of data and information are transparency,
avoidance of bias, validity, replicability, and comprehensiveness. Following a protocol for
review of data and information, by everyone involved in data collection and evaluation, can
ensure a common understanding of the task and adherence to a systematic approach. The
components of this quality assurance protocol are as follows:
1. Specification of the hypotheses to be addressed;
2. Justification of the expertise represented in the expert investigators team;
3. Specification of the methods to be used for identification of relevant studies, assessment
of evidence of the individual studies, and interpretation of the entire body of available
evidence (WHO, 2000);
4. Review process; and
5. Communication of findings.
Revisions of the protocol may be necessary as new aspects of the task emerge during the study
development process. The components of the protocol are addressed in further detail below.
1.0 SPECIFICATION OF THE HYPOTHESES/OBJECTIVES
Specification of the hypotheses to be addressed is included in Section 2.0 of the Study
Methodology. This quality assurance process will ensure that all participants in the study
understand the scope and purpose of the study. Definition of the scope and purpose of the study
in Section 2.0 of the Study Methodology is the basis for the choices made concerning data
collection and methods for evaluation.
2.0 QUALIFIED INVESTIGATORS
To provide authoritative assessments of data and information, it is important to rely on expert
investigators to evaluate the evidence, draw conclusions on the existence of actual and/or
potential hazard, and estimate the magnitude of the associated risk. The team of expert
investigators, which will evaluate the evidence associated with this study, possesses the
following qualifications:
-------
Formal training and/or on-the-job experience;
Knowledge of the subject and the body of technical information pertaining to it;
Experience in scientific review of technical data and information;
Ability to use descriptive and analytical tools appropriately;
Ability to design studies to test hypotheses;
Experience in risk assessment of environmental factors;
Ability to communicate results accurately to decision-makers and stakeholders;
Experience coordinating multiple tasks and disciplines to ensure timely and accurate
delivery of study components; and
• Representation of a broad range of abilities.
3.0 DOCUMENTED METHODS
Processes and methods used to collect and evaluate the data and information must be clear,
explicit, and based on valid practice. It is important to adhere to a rigorous and thorough
approach to the processes of data collection (Section 3.1), data logging (Section 3.2), evaluation
of data (Section 3.3), data analysis (Section 3.4), and use of data in drawing conclusions (Section
3.5).
3.1 Data Collection
Comprehensive identification of relevant studies will be achieved by use of the following
strategy:
Involvement of qualified researchers and trained investigators;
Adherence to an explicit search plan including identification of key words;
Considered effort to include all available studies;
Search of bibliographic databases and journals;
Inclusion of abstracts and unpublished data;
Contact with authors of published data for further information;
Contact with applicable agencies and associations;
Contact with industry representatives;
Contact with environmental group representatives;
Other stakeholder contacts; and
• Other.
3.2 Data Logging
Verification of data and information will be achieved by adherence to a rigid data collection
protocol involving explicit written documentation of where, when, and from whom particular
data and information is obtained, including information obtained from personal or telephone
contacts. All sources of data and information, both verbal and written will be recorded in ink in
bound notebooks.
All reports, records, etc., potentially providing data for the study, will initially be included in the
inventory (data log) of relevant information. Strict maintenance of the data log will address the
necessity for producing a record of the comprehensiveness of the study base of information.
-------
3.3 Data Evaluation
Subsequent to the data logging process, those reports potentially providing useful information
will undergo a selection process to evaluate quality of the information and usefulness to the
study. Systematic evaluation of the validity of individual studies, data, and information will
therefore include assessment of the following:
Source of the data and information;
Qualitative review of the literature;
Qualitative review of data and information collected;
Scientific strength of the data and information;
Geographical, geological, geochemical, spatial, and temporal relevance;
Relevance to determining baseline conditions;
Validity of extrapolation to the scope of the Study;
Characteristics of associations, plausibility, alternative explanations;
Consistency and specificity of the results;
Scientific uncertainties, limitations, and confounding variables; and
Other.
A scale or rating of the data and information with respect to a level of proof required to support
conclusions is specifically not proposed as part of this quality assurance process. Establishing a
specific level of scientific evidence required to justify a subsequent conclusion would generate
significant controversy. Instead, expert judgment will be used to evaluate and weigh available
data and information.
3.4 Data Analysis
A variety of technical methods and tools will be utilized to sort through the pertinent information
and decipher the meaning of the data. These data analysis methods may include:
Quantitative review of selected data and information collected;
Tabulating valid data and information;
Constructing geologic cross sections;
Evaluating current and historical site operations;
Examining adjacent land uses;
Review of consistencies between studies and information;
Review of sources of discrepancies between studies and information;
Evaluation of heterogeneity among studies;
Development of inclusion and exclusion criteria for data and information used for
hypothesis testing; and,
• Other.
All assumptions will be explicitly documented, the basis for use of any models explained, lack of
evidence noted, and scientific uncertainties described as precisely as possible.
3.5 Drawing Conclusions
-------
Drawing conclusions from evaluated and appropriately analyzed and summarized data and
information will involve expert judgment as to whether observations are consistent with the
study hypotheses/objectives (presented in Section 2.0 of the Study Methodology) or whether
some alternative is suggested. The expert investigators will draw upon all evaluated and
appropriately summarized data and information, however, no checklist or formula will be applied
to arrive at conclusions. Instead, critical scientific reasoning and judgment will be used to draw
conclusions. The process of scientific reasoning and judgment will be made explicit by
describing and documenting how expert investigators:
Assessed completeness of data and information;
Accounted for lack of evidence and limitations, and impacts on the conclusions;
Qualified and quantified the degree of scientific uncertainty;
Assessed and accounted for bias in original data and/or information;
Weighed particular information;
Used applicable guidelines and rationales;
Used any ranges of estimates to arrive at conclusions, where appropriate;
Incorporated assumptions into assessments and accounted for the implications of those
assumptions in their conclusions; and
• Other.
Conclusions will be drawn within the boundaries of the data and the scope of the study. Lack or
absence of evidence will be addressed. The relative strength or weakness of available
information to support conclusions, limitations on where a conclusion may apply, and alternative
interpretations of data, will be recognized and discussed. Any qualification on the use of the data
and factors that contribute to uncertainty will be conveyed.
4.0 REVIEW PROCESS
The quality assurance review process will provide a means to examine if the results and
conclusions are verifiable. The review process will result in a determination of whether the
conclusions are directly supported by the data or evidence and can be independently validated by
others. This quality assurance review process will be hierarchical and will include four review
levels:
• EPA internal review,
• Inter-agency review,
• Independent scientific peer review, and
• Stakeholder review.
Internal review will be accomplished by using an in-house staff member, skilled in quality
assurance and other EPA offices involved with coalbed methane or hydraulic fracturing. Other
federal agencies will be asked to review products including the USGS, the BLM and the DOE.
Those agencies may also participate as partners with EPA in conducting the study. Thus, this
level of peer-review will not constitute an independent peer review. EPA will assemble a peer
review panel consisting of experts in hydraulic fracturing or associated subjects. Finally, EPA
will accept comments from stakeholders. Work products will be available on the internet, or
-------
stakeholder may request hard copies by contacting EPA via electronic mail, telephone, or in
writing.
Supporting documentation and data will be easily available for in-depth consideration by other
reviewers. Candid and critical comments will be sought to assist the investigators in making the
study as sound as possible and to ensure that the study meets Agency standards for objectivity,
evidence, and responsiveness to the study charge. Reviewer comments and objections will be
preserved and made a part of the record for the study. Reasons for proceeding or not proceeding
with the study will be clearly explained.
5.0 COMMUNICATION OF FINDINGS
This quality assurance process will be reflected in the communication of scientific findings in a
clear, accurate, and complete manner to interested parties. Investigators will communicate to
Agency and stakeholders:
• The body of technical information that was considered;
• The manner for evaluating, and drawing conclusions from, collected data and
information; and
• Conclusions that address the hypotheses/objectives and are supported by the results of
data evaluation and analysis.
The use of presentation tools such as charts, diagrams, and computer-generated displays will be
based on sufficient, valid, and defensible data.
6.0 REFERENCES
Breslin, K., Airing on the Side of Caution or Pulling Standards Out of Thin Air? Environmental
Health Perspectives, 108(4), April 2000. http://ehpnetl.niehs.nih.gov/members/2000/108-
4/spheres.html <•••< "'-- '-'' *.
Clay, R., Still Moving Toward Environmental Justice, Environmental Health Perspectives,
107(6), June 1999. http://ehpnetl.niehs.nih.gov/members/1999/107-6/spheres.html -^'T^ '•>• » •'
Evaluation and Use of Epidemiological Evidence for Environmental Health Risk Assessment:
WHO Guideline Document. Environmental Health Perspectives, 108(10), October 2000. World
Health Organization European Centre for Environment and Health, Bilthoven Division, A. van
Leeuwenhoeklaan 9, Bilthoven, The Netherlands.
http://ehpnetl.niehs.nih.gov/members/2000/108p997-1002kryzanowski/abstracts.html. Manuel,
J., Truth in Numbers, Environmental Health Perspectives, 108(8), August 2000.
http://ehpnetl.niehs.nih.gov/docs/2000/108-8/niehsnews.html r-•|Tr«"-"*; j"1"-"
Sound Science Technical Issue Paper. Society of Environmental Toxicology and Chemistry
(SET AC), SET AC Press, Pensacola, FL, 1999. http://www.setac.org/sstip.html
-------
Strengthening Science at the U.S. Environmental Protection Agency: Research-Management and
Peer-Review Practices. National Academy Press, Washington, D.C. (2000).
http://www.nap.edu/catalog/9882.html ™""iT ni^'-o^i'
------- |