EPA-SAB-RAC-LTR-94-006
January 28, 1994
Honorable Carol M. Browner
Administrator
U.S. Environmental Protection Agency
401 M St., SW
Washington DC 20460
Re: ORIA's Radon Measurement
Protocol Evaluation Study
Dear Ms. Browner:
At the request of the Office of Radiation and Indoor Air (ORIA), the Radiation Advisory
Committee (RAC, hereafter referred to as "the Committee") of the Science Advisory Board
(SAB) has reviewed the study design for the Radon Measurement Protocol Evaluation Study
(also referred to as "the Protocol") at its meeting on July 20, 1993. An initial discussion of this
proposed study was held with ORIA staff at the RAC meeting in November 1992, where a
number of suggestions were made by the Committee. Following that meeting, ORIA staff
assembled the draft protocol evaluation study, which is the subject of this review. The Committee
completed its review of this topic at its meeting of October 28 and 29, 1993.
Overall, the Committee is pleased that many of the concerns and questions raised in the
November 1992 discussion have been addressed and that the resulting study design appears to be
robust. However, we do have substantive concerns about the design and execution of the study
and the subsequent analyses which should be addressed by the Agency. These are related in detail
later in this letter report, along with our specific response to the charge from the Agency. This
letter report addresses the charge from the Agency (See memorandum dated June 18, 1993 from
Ms. Margo T. Oge, Director of ORIA to Dr. Donald G. Barnes, Director SAB entitled "Review
of the Study Design for the Protocol Evaluation Study,") and provides additional comments on
the proposed study.
The issue of measurement methods and interpretive procedures for determining radon
concentrations and hence mitigation decisions has come before the Committee several times. The
Committee previously reviewed an analysis of the correlation between short-term and long-term
radon measurements conducted by ORIA (then ORP) [EPA-SAB-RAC-92-008: Report of
Correlation of Short- and Long-term Test Results for Indoor Radon, Dec. 9, 1991]. More
recently, the Committee reviewed the issue in conjunction with its reviews of the Citizen's Guide
to Radon and the Homebuyer's and Seller's Guide to Radon. In each case, the Committee
stated that long-term testing for radon offered the best basis for determining the average
concentration in an individual house and for evaluating the indoor radon concentration relative to
-------
the Agency's 4 pCi/L action guideline. The Committee did recognize, however, that in some
circumstances alternative testing approaches may be necessary under the pressure of time, such as
in a real- estate transaction. One such alternative was finding a means to shift the actual radon
testing away from the real-estate transaction time period. It now appears that several private
companies are offering "radon warranties" in selected markets that provide for longer term
testing, after the transfer of title upon completion of sale, followed by any necessary mitigation.
Another alternative presented to the Committee as part of the Homebuyer's and Sellers
Guide to Radon discussion was several short-term testing options, including making either two
simultaneous or two consecutive short-term measurements or making a single radon measurement
of at least two days duration using continuously-recording instrumentation (often referred to as a
continuous radon monitor). At the time of these discussions, the Agency did not have sufficient
data for the Committee to make any recommendations regarding which of these options was
preferred. The proposed study is directed toward providing some of these data.
The Committee wishes to stress that this Protocol, as a prospective study, has the
opportunity to avoid a number of problems before collecting a large amount of data before having
a clear analytical method in mind. This is a major design flaw in many other retrospective
analyses and meta-analyses where the data have already been collected. The data analysis and
statistical techniques are linked to the study design and should therefore be made very explicit
(See page 5 discussion on SAB Recommendation #3).
In the charge to the Committee, ORIA asked that three questions be addressed. The
Committee addresses each of these below and provides additional discussion and
recommendations for the study. This review and comments are based on an EPA draft document
entitled "Draft Radon Measurement Protocol Evaluation Study: Design Report," RTI/5485-
16/03D, dated July 8, 1993, and a briefing from ORIA staff entitled "Briefing for the SAB/RAC
on the Draft Design of the Radon Measurement Protocol Evaluation Study," dated July 20, 1993.
The first question in the charge is as follows:
1) Does the study accumulate data pertinent to further investigating the use of
short-term measurements for making mitigation decisions?
The proposed study will acquire data from both simultaneous and consecutive
measurements of radon concentrations using several short-term test devices — 3 month-long
measurements with alpha track detectors (ATD), 2-day measurements with open-face canisters
filled with activated charcoal, 7-day measurements with diffusion-limited canisters filled with
activated charcoal, and 14-day-long measurements with passive electret ion chambers. These
results will then be compared with 12-month-long measurements with ATDs located on each level
(floor) of the housing units tested. Each short-term measurement device will be tested in pairs
with tests conducted in each of the four seasons. The short- term testing devices will be placed in
the basement (where such exists) and in the first floor living area.
Overall, the study design will gather data that will permit pertinent comparisons of results
2
-------
from the different measurement methods with the (long- term) annual average. Because the study
population is those houses that participated in the National Residential Radon Survey (NRRS),
the collection of additional long-term data will permit the Agency to examine the degree of
temporal variability in the long term-average radon concentration. Since there are very few data
enabling such comparisons, the Committee believes this will be a very useful result of this study.
The ultimate value of the data obtained will be enhanced by having as complete a characterization
of the homes as possible, including any mitigation procedures that have been carried out.
Based on review of the document and briefing materials, the Committee has the following
recommendation regarding pertinent data:
Recommendation #1: The Committee recommends that NRRS houses with
mitigation systems installed not be enrolled in the present study.
Some of the houses in the NRRS may have installed mitigation systems in the time since
completion of that survey. The Committee believes that although long-term measurement of
post-mitigation concentrations would be interesting, such measurements do not supply data useful
for the objectives of this study. Although the numbers of such mitigated dwellings may not be
very large, their inclusion in the study sample would both confound the data comparisons and use
limited resources that could be better utilized in making measurements in houses relevant to the
study objectives.
The second question in the charge is as follows:
2) Are the Data Quality Objectives (DQOs) reasonable?
The DQOs for the study are:
a) Detect false positive and false negative errors with probability of at least 0.80, if
the error rates are 0.03 or higher;
b) Have a 95 percent upper confidence bound no higher than 0.05 for each study cell,
if no false positive or false negative errors are observed for that particular cell;
c) Have the power of at least 0.80 for detecting a difference between a false positive
or false negative error rate of 0.05 and an error rate of 0.15.
The Committee believes that these DQOs are reasonable for use in a study designed to
compare the relative effectiveness of different techniques for measuring radon levels in the
neighborhood of 4 pCi/L.
The third question in the charge is as follows:
3) Does the study design seem reasonable to achieve the Data Quality
-------
Objectives (DQOs)?
The study design is reasonable on the premise that the smallest design cell will yield 70
responses. As the study protocol document states, this sample cell will not meet the third DQO,
although the design does maximize the power for this one cell. The Committee does not believe
that the failure to meet this DQO for only one of the cells is an important limitation to the study
design. However, the ability for this design cell to meet the first two DQOs depends critically on
whether the projected 70 percent response rate is achieved. The tables presented in the draft
study document addressing these two DQOs (shown as Exhibits 2.3 and 2.4) do not extend below
a sample size of 70. However at a sample size of 70, both these DQOs are just met. Thus the
Committee is concerned that, should the response rate fall below that projected, meeting one or
both of these DQOs will be more difficult. In all other design cells, the sample size is sufficient to
meet the DQOs.
Recommendation #2: The Committee recommends that the Agency prepare a
contingency plan for how to handle the design cell with the smallest sample size and
update the Committee on its planning.
The design cell with the smallest sample size does not meet the third DQO. Furthermore,
participation may not meet expectations, and therefore this design cell may not meet the other
DQOs. The Committee, therefore recommends that the Agency prepare a contingency plan for
how to handle this design cell and update the Committee on its planning. The Committee
suggests that one possibility would be to augment the NRRS housing set with additional samples,
thus ensuring that the sample size will be sufficient to meet the DQOs. Another possibility would
be to combine the data from this design cell with data from other cells. This would then limit the
usefulness of maintaining this design cell as a separate part of the analysis.
In addition to the questions asked in the charge to the Committee, the Committee has
three other concerns and recommendations. These are discussed in turn, below.
1. The draft study document is incomplete in that it does not discuss the statistical
analyses proposed for the data resulting from the study; in particular the comparisons to be made
among the different testing devices, methods, locations and seasons. The Committee is concerned
that no specific questions are presented in the very brief Data Analysis and Reporting section of
the Protocol. Five features of the Protocol are highlighted on page 30 of that section and several
possible questions are presented for each feature. No mention is made of how the data analysis
pertinent to each question will be conducted. For example, feature #1 appears to require a
multiple comparisons t-test if the issues addressed relate to concentrations measured, but would
require a non-parametric rank test if all the radon measurements are compared to each other,
based on "correct mitigation decision" results. This is a crucial decision in the study design, that
is, the test chosen depends upon the objectives of the comparison.
Recommendation #3: The Committee recommends that, as the Agency completes its
planning for the evaluation study, ORIA discuss with the Committee the hypotheses
-------
and/or specific questions to be addressed in the study.
Specifically, the Committee recommends that ORIA discuss with the Committee the
following:
a) how do the analyses of the data collected in the Protocol address the study objectives, as
presented in the July 20 briefing:
"examine the relationship of short-term to year-long measurements to assess the frequency
of correct mitigation decisions"
"estimate a national average for the frequency of correct and incorrect mitigation decisions
for selected short-term measurement strategies and devices"?
b) what are the hypotheses and/or specific questions to be addressed by the study and what
statistical methods will be used in answering those questions?
In order to address in full the third element of the charge, the Committee needs to evaluate
the data analysis methods, the questions to be addressed, and the statistical methods proposed for
use in the Protocol. As drafted, the Committee sees the risk of the Agency collecting a large
amount of data before having a clear analytical method in mind. This is a major design flaw in
many retrospective analyses and meta-analyses where the data have already been collected; this
Protocol, as a prospective study, has the opportunity to avoid these problems. The data analysis
and statistical techniques are linked to the study design and should therefore be made very
explicit.
2. The Protocol document is not clear on how the final results are to be presented, and in
particular, the Committee is concerned that it appears that the metric for comparison among the
results is 'mitigation decision-making' rather than a direct comparison of measured concentrations.
Given the existing debate about both the statistical accuracy and precision inherent in the
measurements themselves and in the comparisons among different devices and protocols, the
Committee feels that the results of this study could help inform this debate.
Recommendation #4: In order to evaluate the results of this study across all
seasons, testing devices, methods and device locations, the Committee recommends
the results of each measurement should be presented in units of concentration, with
their attendant uncertainties (both precision and accuracy) and the raw data
preserved in such a format.
This will permit a straightforward evaluation of the accuracy, reproducibility and adequacy
of the different testing methods. This should be done, in particular, to gain a better understanding
of the uncertainty and variability of short-term vs. year-long measurements. These results can
then be analyzed with respect to false positives or false negatives and translated into various
mitigation decision scenarios.
-------
3. In the briefing presented to the Committee in July, 1993, ORIA staff noted that the
distribution of house substructure types within the United States (U.S.) follows a pattern that is
similar to the climatic distribution across the U.S., that is, the fraction of houses with basements is
highest in the northern tier of states and the upper midwest and lowest in the southern,
southwestern, and pacific coast states. The climate, as measured by heating/cooling degree days,
follows a similar pattern, with the northern tier states having the largest heating/cooling degree
days and the southern and southwestern states having the lowest heating/cooling degree days.
The presentation noted that the influence of the distribution of basement/non-basement houses on
radon concentrations might serve as a surrogate for the climate effects. The Committee is
concerned that, although there appears to be a similar pattern in these two parameters, the use of
basement/non-basement data as surrogate for the behavior of the data in different climate zones
does not appear to be well-founded. The radon entry rates and concentration dynamics across
housing substructure types can be quite different and season-to-season differences may magnify
these effects, which could confound attempts to examine climate zone effects. The draft study
design does not discuss this issue, which should be addressed if such an analysis is to be
conducted.
Recommendation #5: The Committee recommends that the Agency not examine the
effect of climate, based on the present study design.
It does not appear to the Committee that the climate zone effects can be determined by
use of a surrogate analysis of the data obtained by house substructure type, even though there
appears to be some geographical overlap between the two parameters. If the Agency has a
method for such a surrogate analysis that is robust, the Committee could review this in the course
of being updated on the techniques which ORIA plans to use for the statistical analysis
(recommendation #4).
In addition to the five recommendations just discussed, the Committee offers three further
comments for the Agency's consideration.
1. The questions of spatial and temporal variability of radon concentrations within a home
are important issues, particularly with respect to comparisons of short-term testing results with
annual average concentrations. These concentration dynamics can differ from house to house,
and may differ depending upon whether time-weighted average concentrations are above 2 or 3
pCi/L. At these concentrations and above, advective transport of soil-gas becomes increasingly
important. In the study design, the Agency proposes to divide the prospective test houses (those
that participated in the NRRS) into four major groups - whether the lowest level is a basement or
not and whether the long-term radon concentration measured on level 1 is above or below 4
pCi/L. The draft document is ambiguous regarding the definition of 'level 1', that is, whether it is
meant to refer to the basement, if it is present, or the first floor of the house.
The Committee believes that the Agency should consider apportioning the study houses
on the basis of whether the first floor radon concentration is above or below 4 pCi/L, rather than
using the 'lowest level' as this division criterion. In addition to the issue of potential differences in
-------
radon concentration dynamics, using the first floor concentration as a division point provides a
uniform basis for this decision across all houses in the study. All houses have a 'first floor'; in
addition, this is often the level most frequently occupied by the inhabitants. Use of this criterion
will ensure that all first floor radon concentrations for the houses in these cells will be greater than
or equal to 4 pCi/L, while using the basement concentrations as the selection criterion does not
provide this assurance (because the ratios of basement to first floor concentrations are often in the
range of 1.5 to more than 3). The Committee recognizes that this could alter the way in which
the Agency then distributes the different short-term measurement devices for housing units falling
into the greater than or equal to 4 pCi/L category; however this redistribution can be done with
no loss in the power of the study or in the DQOs.
2. The Agency has proposed to include 2-day-long measurements made with open-face
charcoal canisters as one of the four short-term measurement devices used in the study. A fairly
extensive literature on comparisons of measurements made using the 2-day charcoal canisters with
those measurements conducted over longer integration time periods exists, both published in
journals and in the 'gray' literature. It is not clear to the Committee that the proposed study will
significantly enhance or change the current understanding about these comparisons. Although not
a formal recommendation of the Committee, the Agency might find that because this study is
resource limited, the 'design cell' occupied by this particular measurement method might be better
allocated to another short-term testing method for which the comparison data set is less extensive.
3. Finally, the Agency has stated that some measurements will be conducted with
continuous radon monitors, although these are not formally part of the study [apparently] due to
the limited number of such monitors available and the labor intensity of their use. Both the
Agency and the Committee are well aware of the current 'discussions' among various interested
parties regarding the use of such devices and that the Homebuyer's and Sellers Guide to
Radon acknowledges their use as a short-term measurement technique. Although the Committee
is not suggesting that these devices be included as part of the present study, it encourages the
Agency to consider whether there should be a small sub-study to utilize a number of these devices
in a very limited number of design cells in order to provide a statistically-valid comparison of the
results obtained from the continuous monitors with those resulting from the passive integrating
detectors.
The Committee is pleased to have the opportunity to review and comment on the Agency's
proposed design for the Radon Measurement Protocol Evaluation Study, and particularly looks
forward to the Agency's response to the items addressed in this letter report.
Sincerely,
/signed/ /signed/
Dr. Genevieve M. Matanoski, Chair Dr. James E. Watson, Jr., Chair
Executive Committee Radiation Advisory Committee
Science Advisory Board Science Advisory Board
-------
-------
NOTICE
This report has been written as a part of the activities of the Science Advisory Board
(SAB), a public advisory group providing extramural scientific information and advice to the
Administrator and other officials of the Environmental Protection Agency (EPA). The Board is
structured to provide a balanced, expert assessment of scientific matters related to problems
facing the Agency. This report has not been reviewed for approval by the Agency; hence, the
comments of this report do not necessarily represent the views and policies of the EPA or of other
Federal agencies. Any mention of trade names or commercial products does not constitute
endorsement or recommendation for use.
This letter report was prepared while Dr. Genevieve M. Matanoski was Chair of the
SAB's Radiation Advisory Committee (RAC), and Dr. Raymond C. Loehr was Chair of the SAB's
Executive Committee. Subsequently, Ms. Carol M. Browner, Administrator of the EPA, selected
Dr. Matanoski as Chair of the SAB's Executive Committee and Dr. James E. Watson, Jr., as Chair
of the SAB's RAC.
-------
SCIENCE ADVISORY BOARD
RADIATION ADVISORY COMMITTEE
Chair
Dr. Genevieve M. Matanoski, M.D., Professor of Epidemiology, The Johns Hopkins University,
School of Hygiene and Public Health, Department of Epidemiology, Baltimore, MD
Members
Dr. Stephen L. Brown, Principal, R2C2 (Risks of Radiation and Chemical Compounds),
Oakland, CA
Dr. June Fabryka-Martin, Hydrogeologist, Isotope and Nuclear Chemistry Division, Los
Alamos National Laboratory, Los Alamos, NM
Dr. Ricardo Gonzalez, Associate Professor, University of Puerto Rico School of Medicine, San
Juan, PR
Dr. F. Owen Hoffman, President, SENES Oak Ridge, Inc., Center for Risk Analysis, Oak Ridge,
TN
Dr. Arjun Makhijani, President, Institute for Energy and Environmental Research, Takoma
Park, MD
Dr. Oddvar F. Nygaard, Professor Emeritus, Division of Radiation Biology, Department of
Radiology, School of Medicine, Case Western Reserve University, Cleveland, OH
Dr. Richard G. Sextro, Staff Scientist, Indoor Environment Program, Lawrence Berkeley
Laboratory, Berkeley, CA
Dr. James E. Watson, Jr., Professor, Department of Environmental Sciences and Engineering,
University of North Carolina at Chapel Hill, NC
Science Advisory Board Staff
Dr. K. Jack Kooyoomjian, Designated Federal Official, U.S. EPA, Science Advisory Board
(1400F), 401 M Street, S.W., Washington, D.C. 20460
Mrs. Diana L. Pozun, Secretary
Dr. Donald G. Barnes, Staff Director
11
-------
Ill
-------
ABSTRACT
The Radiation Advisory Committee (RAC, also "the Committee") of the Science Advisory
Board (SAB) reviewed, at its meeting in July 1993, the Agency's Office of Radiation and Indoor
Air (ORIA) study design for the Radon Measurement Evaluation Protocol Study. Overall, the
Committee is pleased that many of the concerns and questions raised in an earlier November 1992
discussion have been addressed and that the resulting study design appears to be robust and seems
reasonable to achieve the stated Data Quality Objectives (DQOs).
This letter report addresses the specific elements in the charge from the Agency to the
Committee, and provides recommendations for improvements to and additional comments on the
proposed study. These recommendations include 1) exclusion from the study those houses which
have had radon mitigation systems installed; 2) contingency plans for the study design cell with
the smallest sample size; 3) formulation and discussion of the specific study hypotheses to be
analyzed statistically; 4) presentation of results as direct comparisons of concentration
measurements, including measurement/analysis uncertainties; and 5) not to use the present study
design to evaluate the effect of climate.
In addition, the report offers comments on 1) whether to use basement or first-floor radon
concentration measurements of 4 pCi/L for disaggregating the selected study houses; 2) the
desirability of using other short-term measurement devices in place of the planned use of 2-day
open-face charcoal canisters; and 3) the use of continuous radon measurements in some of the
study homes.
Key Words: Radon, Radon Measurement Protocol, long-term and short-term radon testing, alpha
track detector, activated charcoal, electret ion chamber
IV
-------
DISTRIBUTION LIST
Deputy Administrator
Assistant Administrators
EPA Regional Administrators
EPA Laboratory Directors
Deputy Assistant Administrator for Air and Radiation:
Director, Office of Radiation and Indoor Air
Director, Office of Radiation Programs
Director, Center for Environmental Research Information (CERI)
EPA Headquarter Libraries
EPA Regional Libraries
EPA Laboratory Libraries
-------
VI
-------
F :\USER\JKOO YOOMVRNMPROF .LRR
VII
------- |