US Environmental Protection Agency Science Advisory Board Meeting to Review the Draft Report on Agency Draft Entitled “Multi-Agency Radiation Survey and Assessment of Materials and Equipment (MARSAME) Manual,” Draft Report for Comment, December 2006 Manual,” Draft Report for Comment, December 2006


Summary Minutes
US Environmental Protection Agency Science Advisory Board
Meeting to Review the Draft Report on Agency Draft entitled "Multi-
Agency Radiation Survey and Assessment of Materials and Equipment
(MARSAME) Manual," Draft Report for Comment, December 2006

Public Teleconference Meeting
May 29, 2008
1:30 pm - 2:30 pm (Eastern Time)

Meeting Location: Via Telephone Only

Purpose of the Meeting: The Meeting was held to allow for the Chartered SAB to review and approve
the subject draft report. The meeting agenda is in Attachment A. The list of SAB participants is below
and in Attachment B.

Meeting Participants:

Members Participating in the Meeting:

Dr. M. Granger Morgan, Chair
Dr. Virginia Dale
Dr. David Dzombak
Dr. Steve Heeringa
Dr. Bernd Kahn
Dr. George Lambert
Dr. Mike McFarland
Dr. Steve Roberts
Dr. Kathy Segerson
Dr. Thomas Theis

Members of the SAB Staff Office:

Dr. Vanessa Vu,

Mr. Thomas Miller
Dr. JackKooyoomjian
Dr. Angela Nugent

Members of EPA and the Public:

Ms. Mary Clarke, EPA Office of Air and Radiation
Ms. Katherine Reed
Mr. David Alperts, U.S. Army
Mr. Rah Bhat, U.S. Air Force

Dr. Thomas Burke
Dr. Ken Dickson
Dr. James Hammitt
Dr. Rogene Henderson
Dr. Cathy Kling
Dr. Jill Lipoti
Dr. Judith Meyer
Dr. James Sanders
Dr. Deborah Swackhamer
Dr. Valerie Thomas

l

-------
MEETING SUMMARY

Thursday, May 29, 2008

This meeting was announced in the Federal Register on May 7, 2008 (FR 73 25695) (see Attachment C).

1. Convene the Meeting

Dr. Angela Nugent, Acting, SAB Designated Federal Officer, convened the meeting as an official
meeting of the Chartered Science Advisory Board and noted that procedures of the meeting complied
with requirements of the FACA and EPA policy for expert advisory committees. Dr. Nugent noted that
no members of the public had provided written comments for SAB consideration on this topic nor had
any person asked for time on the agenda to make a public statement for Board consideration.

Dr. M. Granger Morgan, Chair, US EPA Science Advisory Board, and the SAB Members present,
then carried out the agenda as summarized below.

2. Status of Upcoming SAB Meetings

Dr. Granger Morgan summarized the status of following SAB Meetings that are scheduled
for the remainder of Calendar Year 2008:

a) July 28 SAB Meeting: Dr. Morgan noted that the July 28-29, 2008 face to face meeting had
been reconfigured into a July 28, 2008 teleconference for the chartered SAB (1:00 p.m. - 5:00
p.m.). The topics for discussion for that meeting are completion of the Board's initial
recommendations on EPA/ORD's Strategic Research Directions and completion of the
Board's advisory on Environmental Disasters. Edits of the two documents will be provided to
members prior to the meeting.

ACTION: Staff is to organize the meeting and assist in getting the background materials
ready for the Board. Staff will also handle logistics for the call.

b) EPEC Ecological Research MYP Quality Review: Dr. Judith Meyer noted that the Ecological
Processes and Effects Committee would like to have a quality review of their draft report on
EPA's Ecological Research Program Multi-Year Plan during the chartered SAB's July
conference call.

ACTION: Dr. Morgan asked the SAB Staff Office to explore the feasibility of this scheduling
issue.

c) Committee for the Valuation of Ecosystem Services (CYPESS) Report Quality Review: As
an additional update, Dr. Morgan noted that the chartered Board would be sent the ecological
valuation draft report in early August and that Board would hold a quality review
teleconference in September 2008.

ACTION: Staff is to organize the review teleconference and provide the background

-------
materials to Board members.

d) October Seminar Meeting: Dr. Morgan noted that plans were underway for a seminar meeting
in October to look at two important emerging science issues as part of the Board's focus on
broader strategic advice. The meeting would be followed by a half-day advisory meeting for
the chartered Board. Dr. Nugent noted that the focus would be on biofuels and epigenomic
research.

ACTION: Dr. Morgan asked that Dr. Nugent circulate the draft agenda to Board Members
within the next few weeks and firm up the tentative date, October 27-28, 2008.

3. Report on Agency Draft entitled "Multi-Agency Radiation Survey and Assessment of Materials
and Equipment (MARSAME) Manual." Draft Report for Comment December 2006 - An SAB
Quality Review

Dr. Morgan then directed the conduct of the MARSAME Quality Review (see draft report at
Attachment D). At the Chair's request, Dr. Bernd Kahn gave summary of the MARSAME report.
He noted that:

a) MARSAME is one piece of a four-piece set of Federal guidance (EPA, DOD, DOE, and
NRC) manuals on radiation monitoring. This mature effort has been ongoing for about 15
years and SAB RAC has been involved for much of that time as an expert review panel.
Many of the RAC's recommendations have been integrated into the series of guidance
manuals over the years. The manuals are much used by the involved community. Thus,
the recommendations in the current RAC MARSAME Panel's advisory are not calling for
large changes to the draft guidance.

b) Much of the RAC Panel's focus was on issues that it was most interested in and where its
expertise is concentrated.

c) The Panel recommended that statistical guidance be collected into a separate chapter and
that it be enhanced according to recommendations in an Appendix of its own report.

d) The Panel also recommended that removable surface contamination be accorded as much
consideration in the manual as that which is not removable and that which is "volumetric."

e) The Panel recommended that reference be made to regulatory levels that will help users
identify applicable limits in setting clean up levels.

f) The Panel recommended that the "case studies" in the manual be labeled as illustrative
because they were not actual cases but instead scenarios that were in essence "invented."

g) The need for training and education in the manual's use was recommended.

Dr. Kahn noted that written SAB Member comments had been received (see Attachment E) and
that he had considered, and responded to, all comments received as of May 23 and that the several
comments received from Board Members since that time were being responded to (see
Attachment F).

ACTION: The DFO was instructed to distribute the updated compilation of member
comments on the draft report, along with Dr. Kahn's responses to the Board Members as
soon as possible. Members should let Dr. Kahn know if they believe the responses are not

-------
sufficient to resolve their concerns.

Dr. Morgan asked for the Lead Reviewers to highlight any concerns they had with the
draft report. All Lead Reviewers referred to their written comments for more details and
highlighted only several points from those comments.

a) Dr. Dzombak stated that the report met EPA's charge though some issues might not
have been addressed as directly as he would wish. He also noted that the methods of
training that were being recommended in the letter and executive summary were not
clear.

b) Dr. McFarland found that committee provided comprehensive and specific
recommendations for how MARSAME could be improved. Appendix A is
noteworthy in supporting data quality objectives concept, could have been enhanced
by recommendations about how type A and type B errors could be identified

c) Dr. Thomas Theis thought that RAC did fine job of reviewing MARSAME. He
suggested a possible reorganization of the report that would put more details in
Appendices and thus improve the document's readability.

d) Dr. Thomas Burke thought that the committee did great j ob of reviewing report and
clarifying report. Dr. Burke thought the agencies' draft MARSAME report language
could be clarified. He suggested that the RAC draft report identify some fundamental
issues of clarity. As a former state official, he found the language of the manual
difficult for all audiences to operationalize. Key recommendations of RAC report are
on target. Conclusions are clear and supported.

Dr. Morgan asked other Board Members if they had things to highlight from their written
comments. Dr. Judy Meyer requested that RAC specify that illustrative examples are "invented,"
different from case studies. Dr. Henderson asked that the term "volumetric contamination" be better
defined. Dr. Morgan asked if the intended audience was radiation experts with knowledge of statistics
and health physics. He also asked that the letter to the administrator reflect the key insights of the
MARSAME review more clearly than the current draft. Dr. Kahn's oral summary at this meeting was
suggested as a better summary for the letter.

Dr. Granger Morgan asked for a motion in regard to the draft report. The motion for action was
that the Board conditionally approve the draft report, conditional that all four lead reviewers have second
chance to vet the report with the right to reopen discussions if they believe the revisions do not
adequately address the Board's comments during this quality review. The motion was moved and
seconded and in a subsequent vote all members supported the motion. Thus, the draft RAC Panel report
was conditionally approved.

Dr. Morgan thanked the SAB and Agency participants for their attention and assistance at the
meeting.

-------
The Designated Federal Officer adjourned the meeting at 2:20 p.m.
Respectfully Submitted:

/ Signed /	/ Signed /

Dr. Angela Nugent

Designated Federal Officer, Acting

US EPA Science Advisory Board

Certified as True:

/ Signed/

Mr. Thomas O. Miller
Designated Federal Officer
US EPA Science Advisory Board

Dr. M. Granger Morgan

Chair, EPA Science Advisory Board

Attachments:

A

Meeting Agenda

B

FR Announcement

C

Meeting Roster

D

Draft MARSAME Report

E

Compilation of SAB Comments

F

Compilation of Chair's Response to SAB Comments

5

-------
US Environmental Protection Agency
EPA Science Advisory Board
Agenda
Public Teleconference Meeting
1:30 pm - 3:00 pm (Eastern Time)
May 29, 2008

Meeting Location:
Via Telephone

Thursday, May 29, 2008

1:30 p.m. Convene the Teleconference Call

Mr. Thomas Miller,
Designated Federal
Officer, US EPA SAB

1:35 p.m. Chair's Welcome and Summary of Important Upcoming Dr. Granger Morgan,
2008 SAB events	Chair, US EPA SAB

-Chair's Welcome
- Today's Agenda

-Discussion of Upcoming SAB Events

—	Status of the October, 2008 SAB Meeting

—	Status of the CVPESS Project on Ecosystem

Services

—	Status of the SAB July 28 Meeting

2:00 p.m. Quality Review of SAB/RAC MARSAME Review Panel's Dr. Granger Morgan,

Draft Report on EPA's "Multi- Asency Radiation Survey
and Assessment of Materials and Equipment (MARSAME)
Manual" of December 2006

Review Panel - SAB Radiation Advisory Committee
MARSAME Panel
Chair - Dr. Bernd Kahn

Lead Reviewers - Drs. Dzombak, McFarland, Theis, and
Burke.

Chair SAB
Dr. Bernd Kahn
Chair, SAB Radiation
Advisory Committee
The Board

3:00 p.m.

Adjourn the Meeting

The Designated Federal
Officer

-------
Federal Register/Vol. 73, No. 89/Wednesday, May 7, 2008/Notices

25695

ENVIRONMENTAL PROTECTION
AGENCY

[FRL-8563-8]

EPA Science Advisory Board;
Notification of a Public Teleconference
Meeting of the Chartered Science
Advisory Board

AGENCY: Environmental Protection

Agency.

ACTION: Notice.

SUMMARY: The Environmental Protection
Agency (EPA) Science Advisory Board
(SAB) Staff Office announces a public
teleconference meeting of the Chartered
EPA Science Advisory Board to review
a draft report from the SAB's Radiation
Advisory Committee Augmented for the
review of the draft Multi-Agency
Radiation Survey and Assessment of
Materials and Equipment (MARSAME)
Manual.

DATES: The SAB will hold the public
teleconference on May 29, 2008. The
teleconference will be held from 1:30
p.m. to 3 p.m. (Eastern Time).
ADDRESSES: The meeting will be
conducted by telephone conference
only.

FOR FURTHER INFORMATION CONTACT: Any

member of the public wishing to obtain
general information concerning this
public teleconference or meeting should
contact Mr. Thomas O. Miller,
Designated Federal Officer (DFO), EPA
Science Advisory Board (1400F), U.S.
Environmental Protection Agency, 1200
Pennsylvania Avenue, NW.,

Washington, DC 20460; via telephone/
voice mail: (202) 343-9982; fax: (202)
233-0643; or e-mail at:
miller.tom@epa.gov. General
information concerning the EPA Science
Advisory Board can be found on the
SAB Web Site at: http://www.epa.govJ
sab.

SUPPLEMENTARY INFORMATION: The SAB

was established by 42 U.S.C. 4365 to
provide independent scientific and
technical advice to the Administrator on
the technical basis for Agency positions
and regulations. The SAB is a Federal
Advisory Committee chartered under
the Federal Advisory Committee Act
(FACA), as amended, 5 U.S.C., App. The
SAB will comply with the provisions of
FACA and all appropriate SAB Staff
Office procedural policies. Pursuant to
the Federal Advisory Committee Act,
Public Law 92-463, notice is hereby
given that the EPA SAB will hold a
public teleconference meeting to
conduct a quality review the SAB
Panel's draft Report on EPA's Draft
Entitled "Multi-Agency Radiation

Survey and Assessment of Materials and
Equipment (MARSAME) Manual," of
December 2006.

Background: The EPA SAB Radiation
Advisory Committee (RAC), augmented
with additional experts, reviewed the
"Multi-Agency Radiation Survey and
Assessment of Materials and Equipment
(MARSAME) Manual," Draft Report for
Comment, December 2006. A multi-
agency work group with participation
by staff from the U.S. Department of
Energy, U.S. Nuclear Regulatory
Commission, U.S. Department of
Defense and U.S. EPA prepared the
manual. The multi-agency work group
has been active since 1995 and prepares
radiological guidance documents. The
draft MARSAME manual complements
MARSSIM (a surficial soils radiation
survey manual) by providing a process
for surveying potentially radioactive
material and equipment (M&E). It
provides guidance to determine whether
M&E are sufficiently free of
radionuclide contamination to be
admitted to or removed from a site.
Additional information on this review
can be obtained on the EPA SAB Web
Site at: http://yosemite.epa.gov/sab/
sabpeople.nsf/

Web CommitteesSubcommittees/
Radiation%20Advisory%20Committee
and in the Federal Register at 72 FR
11356-11358 on the Web at: http://
www.epa.gov/fedrgstr/EPA-SAB/2007/
March/Day- 13/sab4562.htm.

The purpose of this upcoming
teleconference is for the Chartered SAB
to conduct a quality review of the draft
Panel report.

Availability of Materials: The draft
agenda and other materials will be
posted on the SAB Web Site at http://
www.epa.gov/sab prior to the meeting.
For questions and information
concerning the Agency's draft document
on this topic please contact Dr. Mary E.
Clark of the U.S. EPA, ORIA by
telephone at (202) 343-9348, fax at (202)
243-2395, or e-mail at:
clark.marye@epa .gov.

Procedures for Providing Public Input:
Interested members of the public may
submit relevant written or oral
information for the Chartered SAB's
consideration during this quality review
meeting. Oral Statements: In general,
individuals or groups requesting an oral
presentation at a public SAB
teleconference will be limited to three
minutes per speaker, with no more than
a total of one-half hour for all speakers.
At face-to-face meetings, presentations
will be limited to five minutes, with no
more than a total of one hour for all
speakers. To be placed on the public
speaker list, interested parties should
contact Mr. Thomas O. Miller, DFO, in

writing (preferably via e-mail), by May
21, 2008, at the contact information
noted above. Written Statements:
Written statements should be received
in the SAB Staff Office by May 21, 2008,
so that the information may be made
available to the SAB for their
consideration prior to the teleconference
meeting. Written statements should be
supplied to the DFO via e-mail to
miller.tom@epa.gov (acceptable file
format: Adobe Acrobat PDF,
WordPerfect, MS Word, MS PowerPoint,
or Rich Text files in IBM-PC/Windows
98/2000/XP format).

Accessibility: For information on
access or services for individuals with
disabilities, please contact Mr. Thomas
O. Miller at (202) 343-9982 or
miller.tom@epa.gov. To request
accommodation of a disability, please
contact Mr. Miller preferably at least ten
days prior to the meeting, to give EPA
as much time as possible to process
your request.

Dated: April 30, 2008.

Anthony F. Maciorowski,

Deputy Director, EPA Science Advisory Board
Staff Office.

[FR Doc. E8-10138 Filed 5-6-08; 8:45 am]
BILLING CODE 6560-50-P

ENVIRONMENTAL PROTECTION
AGENCY

[FRL-8563-4]

Meeting of the Total Coliform Rule
Distribution System Advisory
Committee—Notice of Public Meeting

AGENCY: Environmental Protection
Agency (EPA).

ACTION: Notice.

SUMMARY: Under section 10(a)(2) of the
Federal Advisory Committee Act, the
United States Environmental Protection
Agency (EPA) is giving notice of a
meeting of the Total Coliform Rule
Distribution System Advisory
Committee (TCRDSAC). The purpose of
this meeting is to discuss the Total
Coliform Rule (TCR) revision and
information about distribution systems
issues that may impact water quality.

The TCRDSAC advises and makes
recommendations to the Agency on
revisions to the TCR, and on what
information should be collected,
research conducted, and/or risk
management strategies evaluated to
better inform distribution system
contaminant occurrence and associated
public health risks.

Topics to be discussed in the meeting
include options for revising the Total
Coliform Rule, for example, rule

-------
U.S. Environmental Protection Agency
Science Advisory Board
BOARD

May 29, 2008

CHAIR

Dr. M. Granger Morgan, Lord Chair Professor in Engineering; Professor and Department Head,
Department of Engineering and Public Policy, Carnegie Mellon University, Pittsburgh, PA

SAB MEMBERS

Dr. Thomas Burke, Professor and Co-Director Risk Sciences and Public Policy Institute,
Bloomberg School of Public Health The Johns Hopkins University, Baltimore, MD

Dr. Virginia Dale, Corporate Fellow, Environmental Sciences Division, Oak Ridge National
Laboratory, Oak Ridge, TN

Dr. Kenneth Dickson, Professor, Institute of Applied Sciences, University of North
Texas, Denton, TX

Dr. David Dzombak, Professor, Department of Civil and Environmental Engineering, Carnegie
Mellon University, Pittsburgh, PA

*Dr. Baruch Fischhoff, Howard Heinz University Professor, Department of Social and Decision
Sciences, Department of Engineering and Public Policy, Carnegie Mellon University, Pittsburgh,
PA

*Dr. James Galloway, Professor, Department of Environmental Sciences, University of
Virginia, Charlottesville, VA

Dr. James K. Hammitt, Professor of Economics and Decision Sciences, Harvard Center for Risk
Analysis, Harvard University, Boston, MA

Dr. Rogene Henderson, Scientist Emeritus, Lovelace Respiratory Research Institute,
Albuquerque, NM

**Dr. Bernd Kahn, Professor Emeritus and Director, Environmental Resources Center, School
of Nuclear Engineering and Health Physics, Georgia Institute of Technology, Atlanta, GA

*Dr. Agnes Kane, Professor and Chair, Department of Pathology and Laboratory Medicine,
Brown University, Providence, RI

*Dr. Meryl Karol, Professor Emerita, Graduate School of Public Health, University of
Pittsburgh, Pittsburgh, PA

Dr. Catherine Kling, Professor, Department of Economics, Iowa State University, Ames, IA

-------
Dr. George Lambert, Associate Professor of Pediatrics, Director, Center for Childhood
Neurotoxicology, Robert Wood Johnson Medical School-UMDNJ, Belle Mead, NJ

**Dr. Jill Lipoti, Director, Division of Environmental Safety and Health, New Jersey
Department of Environmental Protection, Trenton, NJ

Dr. Michael J. McFarland, Associate Professor, Department of Civil and Environmental
Engineering, Utah State University, Logan, UT

Dr. Judith L. Meyer, Distinguished Research Professor Emeritus, Institute of Ecology,

University of Georgia, Lopez Island, WA

Dr. Jana Milford, Associate Professor, Department of Mechanical Engineering, University of
Colorado, Boulder, CO

*Dr. Rebecca Parkin, Professor and Associate Dean, Environmental and Occupational Health,
School of Public Health and Health Services, The George Washington University Medical
Center, Washington, DC

Dr. Stephen M. Roberts, Professor, Department of Physiological Sciences, Director, Center for
Environmental and Human Toxicology, University of Florida, Gainesville, FL

*Dr. Joan B. Rose, Professor and Homer Nowlin Chair for Water Research, Department of
Fisheries and Wildlife, Michigan State University

Dr. James Sanders, Director, Skidaway Institute of Oceanography, University of Georgia,
Savannah, GA

Dr. Jerald Schnoor, Allen S. Henry Chair Professor, Department of Civil and Environmental
Engineering, Co-Director, Center for Global and Regional Environmental Research, University of
Iowa, Iowa City, IA

Dr. Kathleen Segerson, Professor, Department of Economics, University of Connecticut, Storrs,
CT

*Dr. Kerry Smith, W.P. Carey Professor of Economics, Dept. of Economics, Carey Scl of
Business, Arizona State University, Tempe, AZ

Dr. Deborah Swackhamer, Interim Director and Professor, Institute on the Environment,
University of Minnesota, St. Paul, MN

Dr. Thomas L. Theis, Director, Institute for Environmental Science and Policy, University of
Illinois at Chicago, Chicago, IL

Dr. Valerie Thomas, Anderson Interface Associate Professor, School of Industrial and Systems
Engineering, Georgia Institute of Technology, Atlanta, GA

-------
LIAISONS

Dr. Steven Heeringa, (FIFRA SAP), Research Scientist and Director, Statistical Design
Group, Institute for Social Research (ISR), University of Michigan, Ann Arbor, MI

SCIENCE ADVISORY BOARD STAFF

Mr. Thomas Miller, Designated Federal Officer, 1200 Pennsylvania Avenue, NW
1400F, Washington, DC, 20460, Phone: 202-343-9982, Fax: 202-233-0643,
(miller.tom@epa.gov)

*Written comments only provided, was not on the call.
**Member of the RAC MARSAME Panel

-------
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

SAB Draft Report dated April 24, 2008 Quality Review Draft for Panel Review - Do Not Cite or Quote. This review draft
is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by the
Science Advisory Board's Charter Board, and does not represent EPA policy.

UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
WASHINGTON D.C. 20460

OFFICE OF THE ADMINISTRATOR
SCIENCE ADVISORY BOARD

— Date to be Inserted —

EPA-SAB-08-XXX

The Honorable Stephen L. Johnson
Administrator

U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, N.W.

Washington, DC 20460

Subject: Re Report on Agency Draft entitled "Multi-Agency Radiation Sun'ey and
Assessment of Materials and Equipment (MARSAME) Manual," Draft Report for
Comment, December 2006

Dear Administrator Johnson:

The Radiation Advisory Committee (RAC) Multi-Agency Radiation Survey and
Assessment of Materials and Equipment (MARSAME) Manual Review Panel of the Science
Advisory Board has completed its review of "Multi-Agency Radiation Sun'ey and Assessment
of Materials and Equipment (MARSAME)Manual," Draft Report for Comment, December
2006. The Draft Manual was prepared by a multi-agency work group with participation by staff
from US DOE, US NRC, US DoD and US EPA. The multi-agency work group has been active
since 1995, for some periods with representation from additional agencies, to prepare a series of
radiological guidance documents, of which this is the third. The preceding documents are
entitled "Multi-Agency Radiation Survey and Site Investigation Manual (MARSSIM) " and
"Multi-Agency Radiological Laboratory Analytical Protocols (MARLAP) Manual." Both
manuals underwent this review process. Preparation of at least one more manual is planned for
sub-surface radiation surveys.

The MARSAME manual complements MARSSIM (a surficial soils radiation survey
manual) by providing a process for surveying potentially radioactive material and equipment
(M&E). It is a detailed document that provides guidance to determine whether M&E are
sufficiently free of radionuclide contamination to be admitted to or removed from a site. Its
chapters address the components of a survey plan: initial assessment, input needed for decision
making, survey design, survey implementation, and reaching a disposition decision. The manual
begins with a road map to help the user navigate the manual, includes a chapter with illustrative
examples, and collects pertinent information in seven appendices. Much of its presentation is
based on the contents of MARSSIM and MARLAP because M&E surveys often are related to
site investigations and utilize laboratory analyses; however, an M&E survey may stand alone.

-------
1

SAB Draft Report dated April 24, 2008 - Quality Review Draft for Panel Review - Do Not Cite or Quote. This review
draft is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by
the Science Advisory Board's Charter Board, and does not represent EPA policy.

The Review Panel found the MARSAME manual to be an admirable cooperative and
competently written effort by staff from the several agencies to provide guidance in an important
endeavor. The Panel expects the manual to be as widely applied as the two earlier radiological
guidance manuals, and to contribute significantly to radiation protection for the US population.
To assist this endeavor, the Panel presents 37 Recommendations and a Statistical Analysis
Appendix in the enclosed review.

The main Panel recommendations are:

• Provide training and add an Appendix to assist important users who are not the radiation
protection specialists addressed in the MARSAME manual, such as project managers, in
utilizing the manual without having to assimilate the lengthy MARS SIM and MARLAP
documents.

• Collect detailed guidance for statistical analysis, experimental design, and hypothesis testing
in a separate chapter and enhance this guidance in accord with comments in the Appendix to
this review.

• Re-label as 'illustrative examples' what are described as 'case studies' and, to provide greater
benefit to the reader, enhance the content of these illustrative studies so that they more
closely approach that of case studies.

• Tabulate or make reference in MARSAME to all known regulations and guidance for
meeting M&E action levels, with a mechanism for updating them.

• Give as much consideration to surveys for radioactive contamination that is removable from
the surface or that is volumetric as currently is given in this manual to undifferentiated
surface contamination.

• Present the alternative forms of M&E surveys in sufficient detail to give the reader a wide
choice of options, from no further action needed through minor survey efforts to a major
survey that applies the full contents of the MARSAME manual. Include non-linear processes
such as iterative M&E release efforts embodied in decontamination or storage for decay.

Other Panel recommendations concern refinements and improvements in content and
presentation.

In summary, the SAB finds the reviewed MARSAME Manual draft to be a potentially
useful document for ORIA/EPA as well as other Federal and State agencies in providing
guidance to control transfer of material and equipment that may be contaminated with
radionuclides. The MARSAME Panel of RAC appreciates the opportunity to review this draft
manual and hopes that the recommendations provided will enable EPA and cooperating agencies
to issue effective guidance for radiological surveys of material and equipment. We look forward
to your response.

Sincerely,

Dr. M. Granger Morgan, Chair
Chair, Science Advisory Board

Dr. Bernd Kahn, Chair
Chair, MARSAME Panel
Radiation Advisory Committee

-------
1

2

3

4

5

6

7

8

9

10

11

12

13

SAB Draft Report dated April 24, 2008 - Quality Review Draft for Panel Review - Do Not Cite or Quote. This review
draft is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by
the Science Advisory Board's Charter Board, and does not represent EPA policy.

NOTICE

This report has been written as part of the activities of the EPA Science Advisory Board
(SAB), a public advisory group providing extramural scientific information and advice to the
Administrator and other officials of the Environmental Protection Agency. The SAB is
structured to provide balanced, expert assessment of scientific matters related to problems facing
the Agency. This report has not been reviewed for approval by the Agency and, hence, the
contents of this advisory do not necessarily represent the views and policies of the
Environmental Protection Agency, nor of other agencies in the Executive Branch of the Federal
government, nor does mention of trade names of commercial products constitute a
recommendation for use. Reports and advisories of the SAB are posted on the EPA website at
http://www.epa. gov/ sab.

i

-------
SAB Draft Report dated April 24, 2008 - Quality Review Draft for Panel Review - Do Not Cite or Quote. This review
draft is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by
the Science Advisory Board's Charter Board, and does not represent EPA policy.

U.S. Environmental Protection Agency

Science Advisory Board
Radiation Advisory Committee (RAC)

Multi-Agency Radiation Survey and Assessment of Materials and
Equipment (MARSAME) Manual Review Panel

CHAIR:

Dr. Bernd Kahn, Professor Emeritus, Nuclear and Radiological Engineering Program, and
Director, Environmental Radiation Center, GTRI, Georgia Institute of Technology, Atlanta, GA

PAST CHAIR:

Dr. Jill Lipoti, Director, Division of Environmental Safety and Health, New Jersey Department
of Environmental Protection, Trenton, NJ

RAC MEMBERS:

Dr. Thomas B. Borak, Professor, Department of Environmental and Radiological Health
Sciences, Colorado State University, Fort Collins, CO

Dr. Antone L. Brooks, Professor, Radiation Toxicology, Washington State University Tri-
Cities, Richland, WA

Dr. Faith G. Davis, Senior Associate Dean, Professor of Epidemiology, Division of
Epidemiology and Biostatistics, School of Public Health, University of Illinois at Chicago,
Chicago, IL

Dr. Brian Dodd, Consultant, Las Vegas, NV
Dr. Shirley A. Fry, Consultant, Indianapolis, IN

Dr. William C. Griffith, Associate Director, Institute for Risk Analysis and Risk
Communication, Department of Environmental and Occupational Health Sciences, University of
Washington, Seattle, WA

Dr. Jonathan M. Links, Professor, Department of Environmental Health Sciences, Bloomberg
School of Public Health, Johns Hopkins University, Baltimore, MD

Mr. Bruce A. Napier, Staff Scientist, Radiological Science & Engineering Group, Pacific
Northwest National Laboratory, Richland, WA1

Mr. Napier was unable to attend the face-to-face meeting of October 29-31, 2007 and the closure
conference call of March 10, 2008.

-------
SAB Draft Report dated April 24, 2008 - Quality Review Draft for Panel Review - Do Not Cite or Quote. This review
draft is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by
the Science Advisory Board's Charter Board, and does not represent EPA policy.

Dr. Daniel O. Stram, Professor, Department of Preventive Medicine, Division of Biostatistics
and Genetic Epidemiology, Keck School of Medicine, University of Southern California, Los
Angeles, CA

Dr. Richard J. Vetter, Head, Radiation Safety Program, Mayo Clinic, Rochester, MN
CONSULTANTS:

Mr. Bruce W. Church, President, BWC Enterprises, Inc., Hurricane, UT
Mr. Kenneth Duvall, Environmental Scientist/Consultant, Washington, D.C.

Dr. Janet A. Johnson, Consultant, Carbondale, CO 81623

Dr. Paul J. Merges, President, Environment & Radiation Specialists, Inc., Loudonville, N.Y.
SCIENCE ADVISORY BOARD STAFF

Dr. K. Jack Kooyoomjian, Designated Federal Officer, 1200 Pennsylvania Avenue, NW,
Washington, DC, 20460-0001, Phone: 202-343-9984, Fax: 202-233-0643, or 0645
(koovoomiian.iack@epa.gov) Messenger/Physical Delivery Address: 1025 F Street, NW, Room
3606, Mail Code 1400F

in

-------
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

SAB Draft Report dated April 24, 2008 Quality Review Draft for Panel Review - Do Not Cite or Quote. This review draft
is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by the
Science Advisory Board's Charter Board, and does not represent EPA policy.

U.S. Environmental Protection Agency
Science Advisory Board
BOARD
April 21, 2008

CHAIR

Dr. M. Granger Morgan, Lord Chair Professor in Engineering, Department of Engineering and
Public Policy, Carnegie Mellon University, Pittsburgh, PA

VICE CHAIR

SAB MEMBERS

Dr. Gregory Biddinger, Coordinator, Natural Land Management Programs, Toxicicology and
Environmental Sciences, ExxonMobil Biomedical Sciences, Inc, Houston, TX

Dr. Thomas Burke, Professor, Department of Health Policy and Management, Johns Hopkins
Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD

Dr. James Bus, Director of External Technology, Toxicology and Environmental Research and
Consulting, The Dow Chemical Company, Midland, MI

Dr. Deborah Cory-Slechta, Professor, Department of Environmental Medicine, School of
Medicine and Dentistry, University of Rochester, Rochester, NY

Dr. Maureen L. Cropper, Professor, Department of Economics, University of Maryland,
College Park, MD

Dr. Virginia Dale, Corporate Fellow, Environmental Sciences Division, Oak Ridge National
Laboratory, Oak Ridge, TN

Dr. Kenneth Dickson, Regents Professor, Department of Biological Sciences, University of
North Texas, Aubrey, TX

Dr. David A. Dzombak, Walter J. Blenko Sr. Professor of Environmental Engineering,
Department of Civil and Environmental Engineering, College of Engineering, Carnegie Mellon
University, Pittsburgh, PA

Dr. Baruch Fischhoff, Howard Heinz University Professor, Department of Social and Decision
Sciences, Department of Engineering and Public Policy, Carnegie Mellon University, Pittsburgh,
PA

Dr. James Galloway, Professor, Department of Environmental Sciences, University of Virginia,
Charlottesville, VA

Dr. James K. Hammitt, Professor, Center for Risk Analysis, Harvard University, Boston, MA

iv

-------
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

SAB Draft Report dated April 24, 2008 - Quality Review Draft for Panel Review - Do Not Cite or Quote. This review
draft is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by
the Science Advisory Board's Charter Board, and does not represent EPA policy.

Dr. Rogene Henderson, Senior Scientist Emeritus, Lovelace Respiratory Research Institute,
Albuquerque, NM

Dr. James H. Johnson, Professor and Dean, College of Engineering, Architecture & Computer
Sciences, Howard University, Washington, DC

Dr. Bernd Kahn, Professor Emeritus, Nuclear and Radiological Engineering Program, and
Director, Environmental Radiation Center, GTRI, Georgia Institute of Technology, Atlanta, GA

Dr. Agnes Kane, Professor and Chair, Department of Pathology and Laboratory Medicine,
Brown University, Providence, RI

Dr. Meryl Karol, Professor Emerita, Graduate School of Public Health, University of
Pittsburgh, Pittsburgh, PA

Dr. Catherine Kling, Professor, Department of Economics, Iowa State University, Ames, IA

Dr. George Lambert, Associate Professor of Pediatrics, Director, Center for Childhood
Neurotoxicology, Robert Wood Johnson Medical School-UMDNJ, Belle Mead, NJ

Dr. Jill Lipoti, Director, Division of Environmental Safety and Health, New Jersey Department
of Environmental Protection, Trenton, NJ

Dr. Michael J. McFarland, Associate Professor, Department of Civil and Environmental
Engineering, Utah State University, Logan, UT

Dr. Judith L. Meyer, Distinguished Research Professor Emeritus, Institute of Ecology,
University of Georgia, Athens, GA

Dr. Jana Milford, Associate Professor, Department of Mechanical Engineering, University of
Colorado, Boulder, CO

Dr. Rebecca Parkin, Professor and Associate Dean, Environmental and Occupational Health,
School of Public Health and Health Services, The George Washington University Medical
Center, Washington, DC

Mr. David Rejeski, Director, Foresight and Governance Project, Woodrow Wilson International
Center for Scholars, Washington, DC

Dr. Stephen M. Roberts, Professor, Department of Physiological Sciences, Director, Center for
Environmental and Human Toxicology, University of Florida, Gainesville, FL

Dr. Joan B. Rose, Professor and Homer Nowlin Chair for Water Research, Department of
Fisheries and Wildlife, Michigan State University, East Lansing, MI

v

-------
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

SAB Draft Report dated April 24, 2008 - Quality Review Draft for Panel Review - Do Not Cite or Quote. This review
draft is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by
the Science Advisory Board's Charter Board, and does not represent EPA policy.

Dr. James Sanders, Director and Professor, Skidaway Institute of Oceanography, Savannah,

GA

Dr. Jerald Schnoor, Allen S. Henry Chair Professor, Department of Civil and Environmental
Engineering, Co-Director, Center for Global and Regional Environmental Research, University
of Iowa, Iowa City, IA

Dr. Kathleen Segerson, Professor, Department of Economics, University of Connecticut, Storrs,
CT

Dr. Kristin Shrader-Frechette, O'Neil Professor of Philosophy, Department of Biological
Sciences and Philosophy Department, University of Notre Dame, Notre Dame, IN

Dr. V. Kerry Smith, W.P. Carey Professor of Economics , Department of Economics , W.P
Carey School of Business, Arizona State University, Tempe, AZ

Dr. Deborah Swackhamer, Interim Director and Professor, Institute on the Environment,
University of Minnesota, St. Paul, MN

Dr. Thomas L. Theis, Director, Institute for Environmental Science and Policy, University of
Illinois at Chicago, Chicago, IL

Dr. Valerie Thomas, Anderson Interface Associate Professor, School of Industrial and Systems
Engineering, Georgia Institute of Technology, Atlanta, GA

Dr. Barton H. (Buzz) Thompson, Jr., Robert E. Paradise Professor of Natural Resources Law
at the Stanford Law School and Director, Woods Institute for the Environment Director, Stanford
University, Stanford, CA

Dr. Robert Twiss, Professor Emeritus, University of California-Berkeley, Ross, CA

Dr. Lauren Zeise, Chief, Reproductive and Cancer Hazard Assessment Branch, Office of
Environmental Health Hazard Assessment, California Environmental Protection Agency,
Oakland, CA

SCIENCE ADVISORY BOARD STAFF

Mr. Thomas Miller, Designated Federal Officer, 1200 Pennsylvania Avenue, NW
1400F, Washington, DC, Phone: 202-343-9982, Fax: 202-233-0643

vi

-------
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

SAB Draft Report dated April 24, 2008 - Quality Review Draft for Panel Review - Do Not Cite or Quote. This review
draft is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by
the Science Advisory Board's Charter Board, and does not represent EPA policy.

TABLE OF CONTENTS

1.	EXECUTIVE SUMMARY	1

2.	INTRODUCTION	4

2.1	Background	4

2.2	Review Process and Acknowledgement	5

2.3	EPA Charge to the Panel	6

3.	RESPONSE TO THE STATISTICS ELEMENTS OF THE CHARGE QUESTIONS	8

4.	RESPONSE TO CHARGE QUESTION 1: PROVIDING AN APPROACH FOR PLANNING,

CONDUCTING, EVALUATING AND DOCUMENTING ENVIRONMENTAL
RADIOLOGICAL SURVEYS TO DETERMINE THE APPROPRIATE DISPOSITION
FOR MATERIALS AND EQUIPMENT	9

4.1	Charge Question #1	9

4.2	Charge Question # la	10

4.3	Charge Question # lb	 11

4.4	Charge Question # lc	11

4.5	Charge Question # Id	12

5.	RESPONSE TO CHARGE QUESTION 2: COMMENTS ON THE STATISTICAL METHODOLOGY

CONSIDERED IN MARSAME	14

5.1	Charge Question #2	14

5.2	Charge Question # 2a	14

5.3	Charge Question # 2b	15

5.4	Charge Question #2c	18

6.	RESPONSE TO CHARGE QUESTION 3: RECOMMENDATIONS PERTAINING TO THE

MARSAME ROADMAP AND APPENDICES	19

7.	RECOMMENDATIONS BEYOND THE CHARGE	22

REFERENCES CITED	23

APPENDIX A - STATISTICAL ANALYSIS - AN INTRODUCTION TO EXPERIMENTAL DESIGN AND
HYPOTHESIS TESTING WITH SPECIFIC COMMENTS ON STATISTICS	26

A-l An Introduction to Experimental Design and Hypothesis Testing	26

A-2 Specific Comments 	29

APPENDIX B -ACRONYMS AND ABBREVIATIONS	38

APPENDIX C - MARSAME TYPOS AND CORRECTIONS	40

vii

-------
SAB Draft Report dated April 24, 2008 - Quality Review Draft for Panel Review - Do Not Cite or Quote. This review
draft is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by
the Science Advisory Board's Charter Board, and does not represent EPA policy.

1

2	TEXT FIGURES

3	FIGURE 1 - Re-Worked Figure 6.3 from MARSAME Manual for Interpretation of Survey Results for Scan-

4	Only and In Situ Surveys 	16

5	FIGURE 2 - Re-Worked Figure 6.4 from MARSAME Manual for Interpretation of Results for MARSSIM-

6	Type Surveys 	17

7	FIGURE 3 - The MARSAME Process 	20

8

9	APPENDIX A FIGURES

10

11	FIGURE A-l SCENARIO A 	30

12	FIGURE A-2 SCENARIO B 	30

13	FIGURE A-3 Comparison of Sc 	34

14	FIGURE A-4 Comparison of Sc for longer background counting period	34

15	FIGURE A-5 Sc for a briefer background counting period 	35

16	FIGURE A-6 1-p as a function of % excess count above background 	35

17

viii

-------
1

1. EXECUTIVE SUMMARY

The Radiation Advisory Committee (RAC) of the Science Advisory Board (SAB) has
completed its review of the Agency's draft document entitled "Multi-Agency Radiation Survey
and Assessment of Materials and Equipment (MARSAME) Manual," Draft Report for
Comment, December 2006 (U.S. EPA. 2006; see also the MARSAME Hotlink at
http://www.marsame.org). The MARSAME manual presents a framework for planning,
implementing, and assessing radiological surveys of material and equipment (M&E).
MARSAME supplements the Multi-Agency Radiation Survey and Site Investigation Manual
(MARS SIM; see also the MARS SIM Hotlink at http://epa.gov/radiation/marssim/index.htmn.
and refer to information provided in the Multi-Agency Radiological Laboratory Analytical
Protocols (MARLAP) Manual. The MARLAP Hotlink is
http: //epa. gov/radi ati on/marl ap/index. html.

All manuals were prepared collaboratively by a multi-agency work group comprising
staff members of several pertinent Federal agencies. The three documents, taken together,
describe radiological survey programs in great detail and address recommendations to competent
radiation protection professionals and managers for performing such surveys. The manuals are
designed to enable effective comparisons of survey measurements of radionuclide concentrations
to regulations or guides for accepting or rejecting approval of a program or process. Vocabulary
and techniques in MARSAME are carried forward from MARSSIM and MARLAP.

The MARSAME manual complements MARSSIM (a surface-soil radiation survey
manual) by providing a process for surveying potentially radioactive M&E that may be in nature,
commerce, or use when considered for receipt or disposition. It presents an overview of the
various aspects of initial assessment, decision inputs, survey design, survey implementation, and
assessment of results. Important activities such as hypothesis testing and statistical analysis of
measurement reliability are described in considerable detail. A number of illustrative examples,
incorrectly termed "case studies," are presented. A road map assists the reader in moving among
chapters. Useful information is collected in appendices.

This review of the MARSAME Manual by the EPA-SAB Radiation Advisory Committee
(RAC) MARSAME Manual Review Panel was requested by the EPA Office of Radiation and
Indoor Air (ORIA). It is based on reading the MARSAME Draft Report for Comment (December
2006), presentations by MARSAME multi-agency work group members at the meeting on
October 29-31, 2007, and discussions in a series of teleconference meetings held October 9,
2007, December 21, 2007, and March 10, 2007. The review responds to the set of charge
questions posed by ORIA, but also refers to other technical items. (NOTE: Add a statement
regarding the Quality Review meeting here when it is complete. KJK)

The Panel recognizes the magnitude of the effort by the multi-agency work group and the
value of its product. The Panel recommends modifications to only a small fraction of this
product. Panel recommendations can be summarized in the following broad categories:

-------
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

SAB Draft Report dated April 24, 2008 - Quality Review Draft for Panel Review - Do Not Cite or Quote. This review
draft is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by
the Science Advisory Board's Charter Board, and does not represent EPA policy.

•	MARSAME guidance is suitable for experienced radiation protection and surveillance
staff, but managers must be given special training and information directed to them in the
manual so that they do not need to assimilate the lengthy MARS SIM and MARLAP
documents. (1-3, 3-2, 3-4, C-4)2

•	Specialized guidance for applying statistical tools for data analysis, experimental design,
and hypothesis testing should be separated from the otherwise pervasively non-
quantitative guidance for the convenience of the general audience and for acceptance by
specialists. This guidance should be in a separate chapter, enhanced in accord with
comments in the Appendix to this review, (lb-3, lc-1, 2a-l, 2a-2, 2c-1, 3-6)2

•	Label as 'illustrative examples' what are now incorrectly entitled 'case studies' and
enhance their contents to assure realism, (ld-1, ld-2, ld-3, ld-4, 2c-2, 2c-3)2

•	Known regulations and guidance for meeting M&E action levels in MARSAME must be
tabulated or cited by reference, with a mechanism for updating them, (lb-1, 3-5)2

•	As much consideration must be given to surveys for radioactive contamination that is
removable from the surface and that is volumetric as is given currently to undifferentiated
surface contamination in order to distinguish among the three categories for radiation
protection, (lb-2, 2b-3, 2b-4)2

•	The various alternatives for M&E surveys should be described in sufficient detail in
sufficient detail to provide a wide choice of options, from no further action needed
through minor survey efforts to a major survey that applies the full contents of the
MARSAME manual. The options should include non-linear processes such as iterative
M&E release efforts embodied in decontamination or storage for decay. (1-1, 1-2, lc-2,
C-3)2

•	Other recommendations are intended to improve the usefulness of various portions of the
MARSAME manual, (la-1, la-2, l-a-3, 2a-3, 2b-1, 2b-2, 3-1, 3-3, 3-7, 3-8, C-l, C-2)2

The multi-agency work group clearly has devoted considerable effort to describing the
statistical tools. This is important because acceptance of survey measurements depends on their
reliability near the action level (AL). Meeting this requirement can only be demonstrated in a
statistical framework; for example, the discrimination level (DL) must be below the AL in
Scenario A, where the DL is defined to the satisfaction of the surveyor and the regulator in terms
of the values for allowable type I error a and the allowable type II error p.

Because of the importance of clarity in the mathematical support structure, a sub-group of
the Panel has prepared a guide to those topics in MARSAME that is collected in Appendix A to
this review. This guide is devoted to matters such as survey design, the gray region, the DL, the

The parenthetical numbers identify responses to the charge questions.

2

-------
SAB Draft Report dated April 24, 2008 - Quality Review Draft for Panel Review - Do Not Cite or Quote. This review
draft is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by
the Science Advisory Board's Charter Board, and does not represent EPA policy.

1	test significance levels a and P, and hypothesis testing for Scenario A and Scenario B. The guide

2	is intended to present to the multi-agency work group the Panel's view of (1) making this

3	approach readily accessible to persons only generally familiar with statistical analysis, and (2)

4	gaining acceptance by those who are knowledgeable on this topic.

5

6

3

-------
1

2. INTRODUCTION

2.1 Background

The MARSAME Manual (U.S. EPA. 2006b.) was designed to guide a radiation
protection professional through all aspects of radiological surveys of M&E prior to intended
receipt or appropriate disposition. It is written sufficiently broadly to pertain to all types of
M&E. Cited as examples are metals, concrete, tools, trash, equipment, furniture, containers of
material, and piping, among others. The presented alternative outcomes are release or
interdiction, i.e., acceptance or rejection of M&E transfer.

The draft document for comment was prepared collaboratively by staff working together
from the following Federal agencies: US EPA, US NRC, US DOE, and US DoD. It is part of a
continuing and technically significant effort that began with writing MARSSIM (U.S. EPA. 2000
and 2001.) continued with MARLAP (U.S. EPA. 2004.), and anticipates preparation of at least
one other manual after MARSAME for sub-surface radiation surveys and characterization. The
methodology and associated vocabulary in MARSAME follow those of the preceding manuals,
although a few aspects of MARSAME are distinct. Notably, MARSAME may be connected to
MARSSIM and MARLAP as part of a site survey, or stand by itself in considering the transfer of
M&E to or from a site.

Survey guidance in the MARSAME manual and its predecessors is based on the Data
Quality Objectives (DQO) process to design the best survey with regard to disposition option,
action level (AL), and M&E type. The Data Life Cycle (DLC) supports DQO by carrying
suitable information through the planning, implementation, assessment, and decision stages of
the program. The data are collected, evaluated, and applied in terms of Measurement Quality
Objectives (MQO) established with statistical concepts of data uncertainty and Minimum
Quantifiable Concentrations (MQC). The sensitivity of measurements is defined in terms of the
discrimination limit (DL), which is is attained by selecting suitable radionuclide detectors and
conditions of sampling and measurement. The measurement results must be acceptable relative
to action levels and significance levels specified in regulations or guidance.

The MARSAME document is structured as follows, shown with the relevant charge
question (CQ) number:

Acronyms and Abbreviations
Symbols, Nomenclature, and Notations
Conversion factors
Road Map (CQ 3)

Chapter 1, Introduction and overview (CQ 1)

Chapter 2, Initial assessment of M&E (CQ la)

Chapter 3, Identify inputs for the decision (CQ lb)

Chapter 4, Survey design (CQ lc)

Chapter 5, Implementation of disposition surveys (CQ 2a)

Chapter 6, Assess the results of the disposition survey (CQ 2b)

-------
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

SAB Draft Report dated April 24, 2008 - Quality Review Draft for Panel Review - Do Not Cite or Quote. This review
draft is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by
the Science Advisory Board's Charter Board, and does not represent EPA policy.

Chapter 7, Case studies (CQ Id and 2c)

7 Appendices (CQ 3)

References

Glossary

Response to the charge questions was the primary purpose of the RAC MARSAME
Review Panel and is addressed first. The Panel also considered a few related topics, commented
in detail on the MARSAME discussion of statistical and operational aspects, and suggested
minor corrections.

2.2 Review Process and Acknowledgement

The U.S. EPA's Office of Radiation and Indoor Air (ORIA), on behalf of the Federal
Agencies participating in the development of the draft MARSAME Manual, requested the SAB
to provide advice on the draft document entitled "Multi-Agency Radiation Survey and
Assessment of Materials and Equipment (MARSAME) Manual, " Draft Report for Comment,
December 2006 (U.S. EPA. 2006b.; also numbered as NUREG-1575, Supp. 1; EPA 402-R-06-
002; and DOE/EH-707). MARSAME is a supplement to the 'Multi-Agency Radiation Survey
and Site Investigation Manual" (MARSSIM; U.S. EPA. 2000 and 2001; also numbered as
NUREG-1575, rev. 1; EPA 402-R-970-016, Rev. 1; and DOE/EH-0624, Rev. 1). The SAB Staff
Office announced this advisory activity and requested nominations for technical experts to
augment the SAB's Radiation Advisory Committee (RAC) in the Federal Register (72 FR
11356; March 13,2007).

MARSAME was developed collaboratively by the Multi-Agency Work Group (60 FR
12555; March 7, 1995) and provides technical information on approaches for planning,
conducting, evaluating, and documenting radiological surveys to determine proper disposition of
materials and equipment (M&E). The techniques, methodologies, and principles that form the
basis of this manual were developed to be consistent with current Federal limits, guidelines, and
procedures.

The SAB RAC MARSAME Review Panel met in an initial public teleconference meeting
on Tuesday, October 9, 2007. The meeting was intended to introduce the subject and discuss the
charge to the Panel, determine if the review and background materials provided were adequate to
respond to the charge questions directed to the MARSAME Review Panel, and agree on charge
assignments for the Panelists. A public meeting was scheduled on Monday, October 29 through
Wednesday, October 31, 2007, to receive presentations by the Multi-Agency Work Group staff,
consider the charge questions, and draft a report in response to the charge questions pertaining to
the draft MARSAME manual. The Panel reviewed the first public draft report dated December
17, 2007, in a December 21, 2007, public conference call. The second public draft report, dated
February 27, 2008, was reviewed in the March 10, 2008, public conference call. The April 24,

2008 Quality Review Draft was provided to the SAB Charter Board for their review. (	

Continue with chronology to the Quality Review draft by the SAB Charter Board.	KJK)

5

-------
1

2.3 EPA Charge to the Panel

The EPA's Science Advisory Board (SAB) previously conducted the scientific peer
reviews of the companion multi-agency documents MARSSIM (U.S. EPA/SAB. 1997.; EPA-
SAB-RAC-97-008, dated September 30, 1997) and MARLAP (U.S. EPA/SAB. 2003b.; EPA-
SAB-RAC-03-009, dated June 10, 2003). The Federal agencies participating in those peer
reviews considered the process used by the SAB to be beneficial in assuring the accuracy and
usability of the final manuals. Subsequently, two consultations took place for MARSAME (U.S.
EPA/SAB. 2003a.; EPA-SAB-RAC-CON-03-002, dated February 27, 2003, and U.S. EPA/SAB.
2004.; EPA-SAB-RAC-CON-04-001, dated February 9, 2004). These are now being followed
by a request from EPA ORIA on behalf of the four participating Federal agencies that the SAB
conduct this formal technical peer review of the draft MARSAME manual.

The following charge questions were posed to the SAB RAC's MARSAME Review
Panel (U.S. EPA. 2007b):

1) The objective of the draft MARSAME is to provide an approach for planning, conducting,
evaluating, and documenting environmental radiological surveys to determine the appropriate
disposition for materials and equipment with a reasonable potential to contain radionuclide
concentration(s) or radioactivity above background. Please comment on the technical
acceptability of this approach and discuss how well the document accomplishes this objective.
In particular, please

a) Discuss the adequacy of the initial assessment process as provided in MARSAME
Chapter 2, including the new concept of sentinel measurement (a biased measurement
performed at a key location to provide information specific to the objectives of the Initial
Assessment).

b) Discuss the clarity of the guidance on developing decision rules, as provided in
MARSAME Chapter 3.

c) Discuss the adequacy of the survey design process, especially the clarity of new
guidance on using Scenario B, and the acceptability of new scan-only and in-situ survey
designs, as detailed in MARSAME Chapter 4.

d) Discuss the usefulness of the case studies in illustrating new concepts and guidance, as
provided in MARSAME Chapter 7.

2) The draft MARSAME, as a supplement to MARSSIM, adapts and adds to the statistical
approaches of both MARSSIM and MARLAP for application to radiological surveys of materials
and equipment. Please comment on the technical acceptability of the statistical methodology
considered in MARSAME and note whether there are terminology or application assumptions
that may cause confusion among the three documents. In particular, please

a) Discuss the adequacy of the procedures outlined for determining measurement
uncertainty, detectability, and quantifiability, as described in MARSAME Chapter 5.

-------
1

2

3

4

5

6

7

8

9

10

11

12

SAB Draft Report dated April 24, 2008 - Quality Review Draft for Panel Review - Do Not Cite or Quote. This review
draft is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by
the Science Advisory Board's Charter Board, and does not represent EPA policy.

b)	Discuss the adequacy of the data assessment process, especially new assessment
procedures associated with scan-only and in-situ survey designs, and the clarity of the
information provided in Figures 6.3 and 6.4, as detailed in MARSAME Chapter 6.

c)	Discuss the usefulness of the case studies in illustrating the calculation of
measurement uncertainty, detectability, and quantifiability, as provided in MARSAME
Chapter 7.

3) The draft MARSAME includes a preliminary section entitled Roadmap as well as seven
appendices. The goal of the Roadmap is to assist the MARSAME user in assimilating the
information in MARSAME and determining where important decisions need to be made on a
project-specific basis. MARSAME also contains appendices providing additional information
on the specific topics. Does the SAB have recommendations regarding the usefulness of these
materials?

7

-------
1

3. RESPONSE TO THE STATISTICS ELEMENTS OF THE CHARGE

QUESTIONS

Detailed discussions of statistical analysis related to experimental design and hypothesis
testing permeate the otherwise non-mathematical guidance for M&E surveys. The Panel
response and comments specifically addressed to statistical analysis are compiled in Appendix A
rather than scattering them throughout this review. Appendix A consists of an introduction that
describes the view of the Panel, followed by specific reviewer responses based on these reviews.
All related responses to individual charge questions, notably for charge questions lb, lc, and 2a,
are referred to Appendix A.

The Panel recommends that topics presented in Appendix A be included as a separate
chapter that appears early in the MARSAME manual. This will serve to consolidate many of the
important statistical concepts that now are now scattered throughout several chapters. The first-
time user will then become familiar with statistical considerations that are the backbone for the
MARSAME process.

MARSAME contains many suggested equations for designing and interpreting survey
procedures (e.g., Tables 5.1, 5.2). The equations are derived from sound statistical principles,
but can lead to incorrect conclusions if the underlying assumptions in the derivations are not
satisfied. Not every equation needs to be derived in detail, but the assumptions and sampling
requirements to implement specific equations should be thoroughly documented.

Classical hypothesis testing procedures require specification of a null hypothesis and
values of a and P that quantify boundaries for type I and type II errors. The selection of these
values provides a measure of tolerance for uncertainty and assurance that the ultimate goals
relating to risk are satisfied. The existing discussion in Chapter 4 is too vague to provide
guidance on how these values should be selected and should either be more specific or — if this
is considered to be beyond the scope of MARSAME - refer to sources of detailed guidance on
the selection of a and p.

In the design of disposition surveys, the manual discusses determination of measurement
uncertainty, detectability, and quantifiability in terms of MQO requirements. These MQO values
should be organized and presented for individual types, such as in situ, scan only, or MARS SIM.

-------
1

4. RESPONSE TO CHARGE QUESTION 1: PROVIDING AN APPROACH
FOR PLANNING, CONDUCTING, EVALUATING AND DOCUMENTING
ENVIRONMENTAL RADIOLOGICAL SURVEYS TO DETERMINE THE
APPROPRIATE DISPOSITION FOR MATERIALS AND EQUIPMENT

4.1 Charge Question 1: The objective of the draft MARSAME is to provide an approach for
planning, conducting, evaluating, and documenting environmental radiological surveys to
determine the appropriate disposition for materials and equipment with a reasonable potential
to contain radionuclide concentration(s) or radioactivity above background Please comment
on the technical acceptability of this approach and discuss how well the document
accomplishes this objective.

The MARSAME manual impresses the Panel as an excellent technical document for
guiding an M&E survey. Regarding CQ 1, the Panel recommends greater detail in describing the
"alternate approaches or modification" for applying MARSAME, as discussed in Chapter 1, lines
50 - 56. For example, the option of decontaminating the M&E as part of the process when
considering alternate actions appears to be missing. The Panel also recommends making the
manual more accessible to interested non-specialists, notably project managers and other
decision makers. Such non-specialists generally are not included in the intended "technical
audience having knowledge of health physics and an understanding of statistics," with further
capabilities described in Chapter 1, lines 187 - 194. The following itemized recommendations
elaborate on these points.

RECOMMENDATION 1-1: Create a sub-section for the discussion that begins in Chapter 1,
line 49, to present clearly the concept of simple alternatives to what may appear to the reader to
be a major undertaking. Also, in lines 103-111 further define 'release' vs. 'interdiction' to
clarify the distinction between the terms. Follow these paragraphs with sufficient detail and
references to later chapters to assure the reader that when M&E is reasonably expected to have
little or no radioactive contamination, it can be processed without excessive effort under the
MARSAME system. One approach identified subsequently is applying standard operating
procedures (SOP's). Categorization as non-impacted or as class 3 M&E based on historical data
also can lead to an appropriately simple process.

RECOMMENDATION 1-2: Insert a sub-section in Chapter 1 and in appropriate subsequent
chapters to consider various degrees of M&E decontamination as part of the available options
associated with a MARSAME survey. Storage for radioactive decay can be an option for
decontamination.

RECOMMENDATION 1-3: Insert a paragraph after Chapter 1, line 196, to address use by
persons less skilled professionally than defined in a preceding paragraph. Reference to
Appendices B, C, and D, would be helpful for such persons. Adding an appendix that includes
portions of the MARS SIM Roadmap and Chapters 1 and 2 could provide suitable background
information without requiring that all of MARS SIM be read. Presentation of training courses for

-------
1

managers and other generalists with responsibility for MARSAME radiation surveys would be
most helpful.

4.2 Charge Question la: Discuss the adequacy of the initial assessment process as provided
in MARSAME Chapter 2, including the new concept of sentinel measurement (a biased
measurement performed at a key location to provide information specific to the objectives of
the Initial Assessment).

The initial assessment (IA) process is useful as described. That many measurements
made throughout the MARSAME process could be biased should be obvious to the radiation
protection and survey professional. Additional information sources cited below could be helpful.

Sentinel measurements, as described for the IA process of MARSAME have been widely
applied, although not necessarily designated by that name. They are rational and useful for
obtaining an IA of the type and magnitude of radioactive contaminants although they may not
have been randomly selected and, hence, are biased by definition. These measurements and their
applicability and limitations are well described in the document, and their use is clear. In fact,
wider application appears practical.

RECOMMENDATION la-1: Add to the information sources in Chapter 2, lines 104 - 115,
the files (inspection reports, incident analyses, and compliance history) maintained by currently
and formerly involved regulatory agencies. Discussion with agency staffs, especially their
inspectors, also could be fruitful.

RECOMMENDATION la-2: The listing of complexity attributes in Table 2.1 could include
Toxic Substances Control Act (TSCA) materials and hazardous waste.

RECOMMENDATION la-3: In Chapter 1, lines 253 - 259, MARSAME should recognize
that sentinel measurements are important because they may represent the entire historical record
available for IA. Moreover, the measurements may have been so well planned that considering
them "limited data" is misleading without a clear definition of terms. Sentinel measurements are
particularly useful to evaluate assumptions based on process knowledge. In Chapter 2, lines 277
- 280, design of a preliminary survey for radioactive contaminants to fill knowledge gaps often
depends on the availability of data from sentinel measurements. In some instances, the physical
shape of the M&E may limit further survey to sentinel measurements. On the other hand, the
MARSAME Manual draft, line 258, is correct in stating that sentinel measurements should not
be used alone to justify categorization of M&E as non-impacted, especially when geometric or
non-homogeneity limitations in radiation detection are suspected.

-------
1

4.3 Charge Question lb: Discuss the clarity of the guidance on developing decision rules, as
provided in MARSAME Chapter 3.

This chapter, devoted to developing decision rules, is very useful. The decision rules are
admirably clear. The Panel has the following recommendations concerning (1) distinction
among surface removable, surface fixed, and volumetric radioactive contamination; (2)
presentation of regulations and guidance that address these contaminant forms; and (3) the
mathematically complex aspects of measurement method uncertainty, detection capability, and
quantification capability. With regard to the latter, Chapter 3, lines 567 - 622 takes the
MARSAME presentation from broad guidance to specific statistical tutorial, which raises
difficulties for some general readers and questions for some professionals.

RECOMMENDATION lb-1: The regulations or guidance for radionuclide clearance that
define the action levels (AL) discussed in Chapter 3, lines 118 - 120, and listed in Appendix E
should be sufficiently inclusive to apply to the usual M&E handled by users with regard to both
non-fixed ( removable) surface contamination and volumetric contamination. Tabulate or cite all
other known pertinent regulations and guides for this purpose. To the non-fixed surface
contamination regulations included in Table E.2 by DOE and Table E.3 by NRC, add the
Department of Transportation regulation, 49CFR173.443 (U.S. DOT. 49CFR173.443.), and
guides by states such as New Jersey (State of New Jersey. 2007.) and Nevada (State of Nevada.
2001.). Include guidance for volumetric contamination clearance, summarized in Table 5.1 of
NCRP (2002) from reports of national and international standard-setting groups.

RECOMMENDATION lb-2: Information that guides decisions for radioactively
contaminated M&E, listed in Chapter 3, lines 141 - 147, should include measurements of
removable vs. fixed surface contamination to match the distinctions specified in Tables E.2 and
E.3. Insert sub-sections that discuss the implications of planning for and responding to
measurement of removable vs. fixed and surface us. volumetric radioactive contamination and
the subsequent disposition of M&E according to this categorization (see also
RECOMMENDATIONS 2b-3 and ld-3 for discussion of removable radioactive contaminants).

RECOMMENDATION lb-3: Maintain the more general tone of MARSAME throughout
Chapter 3 while moving detailed discussions of statistical aspects to a separate chapter (see also
RECOMMENDATIONS lc-1 and 2a-l). This approach could remove concerns such as why the
Minimum Detectable Concentration (MDC) is recommended for the Measurement Quality
Objective (MQO) in Chapter 3, lines 593 - 597, instead of the Measurement Quality Uncertainty
(MQC), and how item #1 differs from item #3 on lines 609 - 617.

4.4 Charge Question lc: Discuss the adequacy of the survey design process, especially the
clarity of new guidance on using Scenario B. and the acceptability of new scan-only and in-
situ survey designs, as detailed in MARSAME Chapter 4.

With the exception of Section 4.2, Statistical Decision Making, Chapter 4 is easily
understood by the general reader. Classification of M&E is an effective and helpful process.

-------
1

The Disposition Survey Design and Documentation sections are well prepared. Further
discussion would help in addressing problems associated with complex geometric or non-
homogeneous distributions of the radioactive contamination relative to the detector. These are of
particular interest when using scanning or in situ detection methods, and could be demonstrated
effectively in the illustrative example concerning rubble disposal of Section 7.3.

Regarding statistical decision making, the concepts of hypothesis testing and uncertainty
per se are readily understood. However, the aspects of uncertainty with default significance
levels and the resulting gray area and discrimination limits (DL) leading to minimum
quantifiable concentrations (MQC) are not so readily assimilated. Extensive consideration of the
statistical approach is attached to this review as Appendix A.

RECOMMENDATION lc-1: Consider maintaining the same level of generalized guidance
that pervades most of MARSAME in brief sub-sections that address statistical matters. Collect
the mathematical discussion in a separate chapter, as proposed above. Chapter 19, Measurement
Statistics, in MARLAP should serve as example. The separation will serve both the specialist in
statistics, who will appreciate the exposition in the newly added chapter, and readers with less
training in statistics who can follow the general import of the MARSAME approach in the
existing chapters.

RECOMMENDATION lc-2: The MARSAME manual has emphasized disposition options
that, after identification and segregation, lead directly to the disposition survey. Conditioning of
the M&E, such as vacuuming, wiping down, chemical etching, and other forms of
decontamination should be encouraged for meeting disposition options (see also
RECOMMENDATION 1-2). Preliminary measurements are useful for this purpose. The
MARSAME should provide more detail on these approaches and encourage them as an As Low
As Reasonably Achievable (ALARA) policy.

4.5 Charge Question Id: Discuss the usefulness of the case studies in illustrating new
concepts and guidance, as provided in MARSAME Chapter 7.

Case studies can be immensely beneficial for clarifying the MARSAME process and
guiding the user, but members of the multi-agency work group informed the Panel that Chapter 7
does not contain case studies but rather invented illustrative examples. The latter usually are not
as instructive as case studies because they lack the element of reality, but can be helpful if
created carefully to represent actual situations.

RECOMMENDATION ld-1: Delete or replace the example for Standard Operating Procedure
(SOP) use in Section 7.2. Given the good discussion in Section 3.10 for improving an SOP
within the MARSAME framework, the example of applying SOP's at a nuclear power station
appears to contribute little.

RECOMMENDATION ld-2: The example in Section 7.3 of mineral processing of concrete
rubble is instructive, but the reader should be informed that many more measurement results than
those listed in Table 7.3 are obtained under actual conditions and must be evaluated before
making decisions. The radionuclide concentrations reported in Chapter 7, lines 213 - 214,

-------
1

should be confirmed as typical values or replaced by such values, because readers may apply
them as default values. For the same reason, the AL taken from U.S. Nuclear Regulatory
Commission (NUREG-1640;U.S. NRC. 2003.) should be identified as a specific selection, not a
general limit. Inserting boxes with interpretive comments would help the reader to understand
the process used for illustration and the logic leading to the decisions.

RECOMMENDATION ld-3: Insert an introductory statement to place in context the sheer
length of the 21-page example devoted in Section 7.4 to a simple baseline survey of a rented
front loader, to avoid discouraging the reader from applying it. The introduction should explain
that these details are needed to describe the survey process, but that the actual work is brief. This
survey provides an opportunity to present the benefit of sentinel measurements and the
comparison of removable with fixed surface contamination. An actual case history undoubtedly
would show these and also contain a table of survey measurements.

RECOMMENDATION ld-4: Include in each of the illustrative example headings a statement
that they are demonstrating the MARSAME process.

-------
1

5. RESPONSE TO CHARGE QUESTION 2: COMMENTS ON THE
STATISTICAL METHODOLOGY CONSIDERED IN MARSAME

5.1 Charge Question # 2: The draft MARSAME, as a supplement to MARSSIM, adapts and
adds to the statistical approaches of both MARSSIM and MARLAP for application to
radiological surveys of materials and equipment. Please comment on the technical
acceptability of the statistical methodology considered in MARSAME and note whether there
are terminology or application assumptions that may cause confusion among the three
documents.

MARSAME contains tables and text that carefully compare the three documents and
identify consistencies and differences. To Panel members familiar with the three documents,
application of the statistical methodology in MARSAME appears to match that used in
MARSSIM and MARLAP to the extent observable over the wide range of applications.

A shift appears to have occurred from use of the Data Quality Objective (DQO)
terminology of MARSSIM to the Measurement Quality Objective (MQO) of MARSAME, but
the principle is comprehensible. Clearly, MARSAME has close connections to MARSSIM in
surveys of M&E at MARSSIM sites. The manual also addresses M&E that is to be moved onto
or from a site for various reasons, including - - but not necessarily - - processing and surveying
the site subject to MARSSIM.

5.2 Charge Question # 2a: Discuss the adequacy of the procedures outlined for determining
measurement uncertainty, detectability, and quantifiability, as described in MARSAME,
Chapter 5.

The presentation for determining uncertainty, detectability, and quantifiability in Chapter
5, as well as aspects of this discussion in Chapters 4 and 6, follows the well-developed path in
MARSSIM and MARLAP and is essential to the disposition survey planner. The Panel believes
that correct application by the user requires (1) previous reading of MARSSIM and MARLAP,
and (2) the expertise and knowledge specified in Chapter 1, lines 189 - 194.

RECOMMENDATION 2a-l: Enable the reader to understand the topics in Chapter 5 more
clearly by separating the entire mathematically detailed statistical exposition in a chapter that
could be entitled "Review of Experimental Design and Hypothesis Testing." Appendix G can be
included in this chapter. The chapter can be placed before Chapter 4. All sections currently in
Chapters 4-6 that discuss generalized aspects of these topics, including measurement
uncertainty, detectability, and quantifiability, can be kept in place; reference should be made to
the technical discussions, equations, and tables in the new chapter.

RECOMMENDATION 2a-2: Consider the comments made in Appendix A concerning the
topics of experimental design, hypothesis testing, and the statistical aspects of uncertainty in
preparing the separate chapter suggested above.

-------
1

RECOMMENDATION 2a-3: Organize a summary or guide that focuses on the procedures for
setting MQOs and for determining uncertainty, MDC, and MQC. The ability to set
Measurement Quality Objectives (MQOs) is an important element of the MARSAME process,
but the discussion involving the implementation of MQOs in the design of the three survey types
may confuse the reader. Aspects of implementation are immersed in details defining,
explaining, and deriving theoretical concepts. Move the discussion on setting MQOs, in Sections
5.5 thru 5.9, to Chapter 4 on Survey Design.

5.3 Charge Question # 2b: Discuss the adequacy of the data assessment process, especially
new assessment procedures associated with scan-only and in-situ survey designs, and the
clarity of the information provided in Figures 6.3 and 6.4.

The data assessment process is carefully presented and thoroughly explored. The advice
is pertinent and the examples are helpful.

The Panel discusses statistical considerations in Appendix A. The information presented
in Figures 6.3 and 6.4 is clear (See Figures 1 and 2, below), but minor changes, shown in the
following two revised Figures are proposed.

The Panel noted above the importance of distinguishing among contamination that is (1)
removable on the surface, (2) fixed to the surface, or (3) volumetric in all MARSAME chapters.
Smear surveys (wipe tests) are an integral part of an M&E survey because of the potential
radiation dose from removable radionuclides that can spread from M&E surfaces and be inhaled
and ingested. Removable surface contamination is included in DOE regulations in Table E.2 and
NRC regulations in Table E.3, as well as DOT regulations and International Atomic Energy
Agency (IAEA) guidance. Multi-agency working group members expressed reluctance about
including in MARSAME a survey technique that they consider to be poorly reproducible for
defining the removable radionuclide amount per area. The Panel response is that insufficiently
discussing wipe tests is unrealistic and misleading. Each type of measurement has its own
uncertainty. A reasonable approach is to begin with the instruction in 49CFR173.443 (U.S. DOT
49CFR173.443) on "wiping an area of 300 cm3 ... with an absorbent material ... using moderate
pressure" and that "sufficient measurements shall be taken in the most appropriate locations to
yield a representative assessment" and then provide guidance on defining and controlling
variability.

RECOMMENDATION 2b-l: In Fig. 6.3 (See Figure 1 below, which reworks Fig. 6.3), clarify
the distinction of a MARSSIM-type survey by moving "Start" to immediately above the decision
point "Is the Survey Design Scan-only or In situT and then connecting this to an inserted
decision diamond "Is the AL equal to zero or background?". A "yes" leads to "Requires scenario
B ..." and a "no" leads to "Disposition Decision Based on Mean

RECOMMENDATION 2b-2: In Fig. 6.4 (See Figure 2 below, which reworks Fig. 6.4), for a
more consistent presentation, insert a decision diamond after both "Perform the Sign Test" and
"Perform the WRS Test" that says "Scenario A," followed by a "yes" or "no" leading to the two
"Scenario A" and "Scenario B" branches at both locations.

-------
SAB Draft Report dated April 24, 2008 - Quality Review Draft for Panel Review - Do Not Cite or Quote. This review
draft is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by
the Science Advisory Board's Charter Board, and does not represent EPA policy.

Assess the Results of the Disposition Survey





Requires Scenario B
LBGR = AL
Scan MDC * UBGR





1



Individual Results Must
be Recorded









Record Individual Scan
Results If Required





! M&E Meet the \
Disposition Criterion j

M&E Meet the
v Disposition Criterion

M&E Meet the \
\ Disposition Criterion J

M&E Do Not Meet the
Disposition Criterion

Consult a
Statistician

Figure 6.3 Interpretation of Survey Results for Scan-Only and In Situ Surveys

Figure 1 - Re-Worked Figure 6.3 from MARSAME Manual for Interpretation of Survey Results
for Scan-Only and In-Situ Surveys

16

-------
SAB Draft Report dated April 24, 2008 - Quality Review Draft for Panel Review - Do Not Cite or Quote. This review
draft is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by
the Science Advisory Board's Charter Board, and does not represent EPA policy.

704

705	Figure 6.4 Interpretation of Results for M A RSSI M-Type Surveys

MARSAME	63-6	December 2006

Figure 2 - Reworked Figure 6.4 from MARSAME Manual for Interpretation of Results for
MARSSIM-Type Surveys

17

-------
1

RECOMMENDATION 2b-3: To counteract the discomfort of Multi-agency working group
members with the qualitative aspect of wipe tests, the MARSAME manual could recommend
evaluations of the removable radionuclide fraction measured by wipe test for the surveyed M&E.
These evaluations can include, for example, sequential smears at a given location at the M&E, or
smears at adjoining locations performed with different material and pressure, by different
persons, and for different radionuclides. Refer to State of Nevada (2001), State of New Jersey
(2007), for a description of the process, to Rademacher and Hubbell (2008) pp. 10, 16 for an
application to radiological monitoring, and U.S. EPA (2007a) for more general applications of
the wipe test.

RECOMMENDATION 2b-4: Insert sub-sections in all chapters to address implementation and
assessment of survey processes to distinguish between surface and volumetric contamination
(i.e., measurement after surface cleaning or observing the effect of counting geometry) and
between removable and fixed surface contamination (i.e., wipe test results compared to total
surface activity). These types of contamination are described in Chapter 1, lines 127 - 152, but
their implications should be considered throughout the MARSAME manual. Concerns in
measuring volumetric contamination include characterizing non-uniformly distributed
radionuclides and quantifying radionuclides that emit no gamma rays.

5.4 Charge Question # 2c: Discuss the usefulness of the case studies in illustrating the
calculation of measurement uncertainty, detectability, and quantifiability as provided in
MARSAME chapter 7.

As stated in the response to CQ Id, case studies are invaluable in guiding the user
through complex operations. The illustrative examples given instead of case studies in
MARSAME lack the realistic data accumulation that permits estimation of uncertainty.
Excessively detailed derivations of equations for calculation are shown in Chapter 7, lines 579 -
628, 658 - 665, 682 - 689, and 1133 -1150. For discussions related to uncertainty, refer to
Appendix A.

RECOMMENDATION 2c-l: Move the detailed derivations, including partial derivatives,
identified above to the newly added separate chapter recommended for discussion of
experimental design and hypothesis testing.

RECOMMENDATION 2c-2: Use illustrative examples to demonstrate any MARSAME
guidance that the Multi-agency Working Group considers difficult to follow. These may include
approximating uncertainty (see Chapter 5), distinctions such as interdiction vs. release, and
applying scenarios A vs. B.

RECOMMENDATION 2c-3: Use Sections 7.4 and 7.5 to illustrate the benefit of wipe tests for
determining removable radioactive surface contaminants. Experience suggests that the
contaminant usually is in this form on M&E such as earth-moving equipment.

-------
1

6. RESPONSE TO CHARGE QUESTION 3: RECOMMENDATIONS
PERTAINING TO THE MARSAME ROADMAP AND APPENDICES

Charge Question 3: The draft MARSAME includes a preliminary section entitled Roadmap
as well as seven appendices. The goal of the Roadmap is to assist the MARSAME user in
assimilating the information in MARSAME and determining where important decisions need
to be made on a project-specific basis. MARSAME also contains appendices providing
additional information on the specific topics. Does the SAB have recommendations regarding
the usefulness of these materials?

The Roadmap is crucial in guiding the reader through a document as complex as
MARSAME. The appendices are useful in various ways, such as providing information
compilations and statistical tables, and avoiding the need to seek this information in MARSSIM
and MARLAP. Also necessary to the reader are the acronyms and abbreviations; symbols,
nomenclature, and notations; and glossary. The following Recommendations are intended to
enhance their use.

RECOMMENDATION 3-1: Roadmap Figure 1 connects the MARSAME chapters in terms of
the Data Life Cycle. Consider establishing an analogous connection with Roadmap Figures 2, 3,
5, 6, 7, and 8. At present, the only Roadmap figures connected to each other are Fig. 2, 3, and 4,
and 7 with 8.

RECOMMENDATION 3-2: Consider assisting project managers by highlighting major
operational decision points in the roadmaps.

RECOMMENDATION 3-3: The roadmap should ensure that the primary components of the
process are identified, their relationship to one another is depicted, and the boundaries of
application are well-defined, in accord with the DQO process. Figure 3 provided below
illustrates this suggestion.

-------
SAB Draft Report dated April 24, 2008 - Quality Review Draft for Panel Review - Do Not Cite or Quote. This review
draft is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by
the Science Advisory Board's Charter Board, and does not represent EPA policy.

PLAN

Categorization

Initial Assessment

Preliminary

Survevs

Decision Rule

CONDUCT

Design Disposition
Survev

Disposition

Survey

ASSESS

Verification

&

Validation

Evaluate Results

1

2

3

4

5

DECIDE

Decision

FIGURE 3 - The MARSAME Process

20

-------
SAB Draft Report dated April 24, 2008 - Quality Review Draft for Panel Review - Do Not Cite or Quote. This review
draft is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by
the Science Advisory Board's Charter Board, and does not represent EPA policy.

1

2

3	RECOMMENDATION 3-4: Indicate in the body of the text that Appendices B, C, and D are

4	useful overviews of the environmental radiation background, sources of radionuclides, and

5	radiation detection instruments, respectively, for managers and generalists; they may be too

6	general for the experienced health physicist to whom the manual is addressed.

7

8	RECOMMENDATION 3-5: Insert a table with AL guidance for volumetric radionuclide

9	contamination in Appendix E (see RECOMMENDATION lb-1).

10

11	RECOMMENDATION 3-6: Either move Appendix G into the new chapter on experimental

12	design and hypothesis testing or indicate its relation to that new chapter.

13

14	RECOMMENDATION 3-7: Move the Glossary to the front to join the tables of acronyms and

15	of symbols.

16

17	RECOMMENDATION 3-8: Expand the definition of 'Interdiction' in the glossary to clarify

18	its application to receiving or disposing of M&E.

19

20

21

22

23

24

25

26

27

28

29

30

31

32

21

-------
1

7. RECOMMENDATIONS BEYOND THE CHARGE

RECOMMENDATION C-l: In Chapter 3, discuss decisions leading to selecting the degree of
confidence, embedded in the choice of significance level a and P values. Selection may be a
matter of the acceptable uncertainty specified by the agency that sets the action level.

RECOMMENDATION C-2: In Chapter 2, discuss the impact of survey cost and needed skills,
instruments, and time on the MARSAME effort. Brief projects obviously need different designs
than lengthy ones. Discuss requirement and program for data retention, especially in long
projects and when contractors are replaced.

RECOMMENDATION C-3: In Chapter 6, discuss the options to be considered and pursued
when the plan proposed initially for M&E transfer is rejected because of the observed
contaminant levels.

RECOMMENDATION C-4: Provide an additional Appendix that summarizes topics in
MARS SIM and MARLAP that are important to the MARSAME manual but are insufficiently
described in it, or at least give page references to the earlier documents. Such topics may include
aspects of quality assurance (e.g., validation and verification of results); data reliability affected
by sample dimensions, measurement frequency, and detector characteristics. Consider also the
effect of non-random variability in measurement (e.g., fluctuating geometry or monitor
movement rate).

-------
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

SAB Draft Report dated April 24, 2008 - Quality Review Draft for Panel Review - Do Not Cite or Quote. This review
draft is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by
the Science Advisory Board's Charter Board, and does not represent EPA policy.

REFERENCES CITED

(Alphabetical and date sequenced listing of Author Last name, First name, Middle Initial, Title,
Date, etc. To be finalized following Quality Review by SAB Charter Board — KJK).

Federal Register Notice Citations:

FR, Vol. 60, No. , March 7, 1995, p. 12555

FR, Vol. 72, No. , March 13, 2007, p. 11356

FR, Vol. 72, No. 184, September 24, 2007, pp. 54255 - 54257.

FR, Vol. 72, NO.	, Date, pp.	-	..(Charter Board Mtg. announcement to be added)

NCRP. 2002. Managing Potentially Radioactive Scrap Metal, Report #141, National Council on
Radiation Protection and Measurements, Bethesda, MD 20814.

Rademacher, Steven E. and Joshua L. Hubbell. 2008. Boeing Michigan Aeronautical Research
Center (BOMARC) Missle Shelters and Bunkers Scoping Survey Report. Vols. 1 &2, United
States Air Force Institute for Operational Health, Surveillance Directorate, Radiation
Surveillance Division, Brooks City-Base, TX, IOH-SD-BR-SR-2008-000_, February 2008

State of Nevada. 2001. Division of Environmental Protection Dry Wipe Sampling, Oct. 16,
2001 (7 pages) http: //ndep. nv. gov/fall on/dry. pdf

State of New Jersey. 2007. Radiological Emergency Response Plan. SOP 361 Wipe Sampling,
Rev. 9, May 2007

U.S. DOT. 49CFR173.443. Package and Vehicle Contamination Limits. Minimum Required
Packaging for Class 7 (Radioactive) Materials

U.S. EPA. 2000 and 2001. "Multi-Agency Radiation Survey and Site Investigation Manual'
(MARSSIM). NUREG-1575, rev. 1; EPA 402-R-970-016, Rev. 1, DOE/EH - 0624, Rev. 1,
August 2000 and June 2001 update

U.S. EPA. 2006. "Guidance on Systematic Planning Using the Data Quality Objectives
Process, EPA QA/G4," EPA/240/B-06/001, February 2006

U.S. EPA. 2006b. "Multi-Agency Radiation Survey and Assessment of Materials and
Equipment Manual (MARSAME), Draft Report for Comment," NUREG-1575, Supp. 1; EPA
402-R-06-002; DOE/EH-707, December 2006

U.S. EPA. 2007a. "A Literature Review of Wipe Sampling Methods for Chemical Warfare
Agents and Toxic Industrial Chemicals," EPA/600/R-07/004, January 2007
(http://www.epa.gov/NHSRC/pubs/reportWipe042407.pdf)

23

-------
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

SAB Draft Report dated April 24, 2008 - Quality Review Draft for Panel Review - Do Not Cite or Quote. This review
draft is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by
the Science Advisory Board's Charter Board, and does not represent EPA policy.

U.S. EPA. 2007b. Memo from Elizabeth A. Cotsworth, Director, Office of Radiation and Indoor
Air (ORIA) to Vanessa Vu, Director, Science Advisory Board Staff Office, and entitled

"Review of the Draft Multi-Agency Radiation Survey and Assessment of Materials and
Equipment Manual, " October 23, 2007

U.S. EPA/SAB. 1997. "An SAB Report: Review of the Multi-Agency Radiation Survey and
Site Investigation Manual (MARSSIM)," Prepared by the Radiation Advisory Committee
(RAC) of 5he Science Advisory Board , EPA-SAB-RAC-97-008, Sept. 30, 1997

U.S. EPA SAB. 2002. "Panel Formation Process: Immediate Steps to Improve Policies and
Procedures: An SAB Commentary, "EPA-SAB-EC-COM-02-003, May 17, 2002.

U.S. EPA. 2004. "Multi-Agency Radiological Laboratory Analytical Protocols (MARLAP)
Manual," NUREG-1576; EPA 402-B-04-001A; NTIS PB2004-10521, July 2004

U.S. EPA/SAB. 2003a. "Consultation on Multi-Agency Radiation Site Survey Investigation
Manual (MARSSIM) Supplements for Materials and Equipment (MARSAME): A Science
Advisory Board Notification of a Consultation," EPA-SAB-RAC-CON-03-002, February 27,
2003

U.S. EPA/SAB. 2003b. "Multi-Agency Radiological Laboratory Analytical Protocols
(MARLAP) Manual: An SAB Review of the MARLAP Manual and Appendices by the
MARLAP Review Panel of the Radiation Advisory Committee (RAC) of the U.S. EPA Science
Advisory Board (SAB)," EPA-SAB-RAC-03-009, June 10, 2003

U.S. EPA/SAB. 2004. "Second Consultation on Multi-Agency Radiation Site Survey
Investigation Manual (MARSSIM) Supplements for Materials & Equipment (MARSAME): A
Science Advisory Board Notification of a Consultation," EPA-SAB-RAC-CON-04-001,

February 9, 2004

U.S. NRC. 1997. "A Nonparametric Statistical Methodology for the Design and Analysis of
Final Status Decommissioning Survey," Draft Report for Comment, Washington, DC, Nuclear
Regulatory Commission (NRC) NUREG-1505, August, 1995.

U.S. NRC. 2003. Radiological Assessment for Clearance of Materials from Nuclear
Facilities," Nuclear Regulatory Commission (NRC), Office of Nuclear Regulatory Research,
Washington DC, NUREG-1640, Vols. 1-4, June 2003.

24

-------
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

SAB Draft Report dated April 24, 2008 - Quality Review Draft for Panel Review - Do Not Cite or Quote. This review
draft is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by
the Science Advisory Board's Charter Board, and does not represent EPA policy.

Web-based Citations and Hotlinks

MARS SIM: http://epa.gov/radiation/marssim/index.html
MARLAP: http: //epa. gov/radi ati on/marl ap/index. html

MARSAME: http://www.marsame.org

DRY WIPE SAMPLING PROTOCOL FOR THE STATE OF NEVADA:

State of Nevada. 2001. Division of Environmental Protection Dry Wipe Sampling, Oct. 16,
2001 (7 pages) http: //ndep. nv. gov/fall on/dry. pdf

25

-------
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

SAB Draft Report dated April 24, 2008 - Quality Review Draft for Panel Review - Do Not Cite or Quote. This review
draft is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by
the Science Advisory Board's Charter Board, and does not represent EPA policy.

APPENDIX A - STATISTICAL ANALYSIS - AN INTRODUCTION TO
EXPERIMENTAL DESIGN AND HYPOTHESIS TESTING AND
SPECIFIC COMMENTS ON STATISTICS

A-l An Introduction to Experimental Design and Hypothesis Testing:

The general problem of designing a survey of the sort described in the MARSAME document
involves the following issues:

(1)	Understanding the error properties of the measurement instrument and how they can be
manipulated (by changing counting times or performing repeated measurements of the
same radionuclide quantity, for example). Generally the measurement error can be well
characterized by its standard deviation om. This value may be a constant (all
measurements have the same standard deviation) or it may vary with radiation level (as in
the behavior of an idealized radiation counter);

(2)	Understanding the distribution of radionuclides in the population of equipment or
material that is to be measured. This distribution can often be well characterized by a
standard deviation as which we may call the sampling standard distribution;

(3)	Deciding upon the number of samples, N, from the distribution of radionuclide
concentration that will be used in the detection problem;

(4)	Specifying the null and alternative hypotheses to be examined; the symbol A represents
the quantity of excess radionuclides equal to the difference between the null and the
alternative hypothesis values;

(5)	Specifying values of a and P that quantify acceptable limits for type I and type II errors;

(6)	Determining, with fixed A and a, the power 1 - P to reject the null hypothesis in favor or
the alternative.

From a statistical standpoint, designing an experiment means finding values of the
sample size N and the detectable difference A that will control type I error and power, given the
instrument's measurement error properties and the sampling radionuclide concentration
distribution.

In MARSAME, the null and alternative hypotheses generally concern the true difference
in radionuclide levels between a potentially contaminated material or piece of equipment and the
appropriate background reference. In Scenario A, the null hypothesis is that the M&E is at least
as radioactive (over background) as some number called AL (the action level), and the
alternative is that the true exess radionuclide level is less than AL. In Scenario B the null
hypothesis is that the M&E is at the action level (which usually equals the background in
scenario B) and the alternative hypothesis is that the M&E is over the AL. The MARSAME
manual should note the interplay between a and 1 - p. For a fixed study design, power can be

26

-------
1

defined only in terms of a since power is the probability of rejecting the null hypothesis at a
given a.

When a single measurement is taken, the variance of that measurement will equal
Cm + qo. Again, some or
all of the measurement error variance may still remain.

For most situations in MARSAME, the null hypothesis concerns the difference between
background levels and the level of contamination of the M&E. Table 5.1 (in the current
document) gives some special formulae used when counts in time follow a Poisson distribution
(so that the variability of the counts of both background and the item of interest depends on
counting time and radiation level). In general, the variance of the difference between sampled
radioactivity and the estimate of background will require special investigation as a part of the
survey design.

For simplicity, it is useful to denote the standard deviation of measurement minus
background as a, which refers to the standard deviation of the estimate (often termed the
standard error) obtained from the entire measurement method (involving either single readings,
multiple readings, scans of some or all of the material, etc). This a can be a relatively
complicated function of the underlying measurement and sampling variability (which must
include the uncertainties in the estimate of background) that may require careful study to
quantify properly.

-------
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

SAB Draft Report dated April 24, 2008 - Quality Review Draft for Panel Review - Do Not Cite or Quote. This review
draft is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by
the Science Advisory Board's Charter Board, and does not represent EPA policy.

Once a is determined, the power, 1-J3, of a study will depend upon two other parameters,
(1) the type I error rate a and (2) the size of the assumed true difference A. If the standard error
of the estimate, a, is the same for all radiation levels being measured, then the ratio A/a
determines power for a given value of a (otherwise a more complicated expression is used, as in
Table 5.1 of MARSAME). For known a, we may specify the "detectable difference A by fixing
both the type I error a and the power 1-|3 and solving for A. In the MARSAME manual, this
detectable difference A is called the width of the "gray region." (Differences less than this A are
only detectable with power less than the required 1-|3 and hence are "gray.") If the action level,
AL, is defined to be the upper bound of the "gray region," then the lower bound (AL- detectable
difference A) is called the "discrimination limit" (DL). Note that implicitly the detectable
difference A and the DL depend upon the power, type I error rate, and the standard error of the
estimated a. One of the confusing aspects of the MARSAME manual is that the DL is introduced
long before the concept of power or type I error.

The two scenarios (A and B) considered in the report both assume that the null
hypothesis is at the action level, but differ in the direction of the alternative hypothesis and
generally in the value of AL. Under scenario A, the alternative hypothesis is that the radiation
level is less than the action level (which is the upper limit above background to be allowed)
whereas under scenario B the alternative hypothesis is that the radiation level is greater than the
action level (which is typically set to background). Under scenario A the M&E is only deemed
to be safe for release if the null hypothesis is rejected, whereas under scenario B the M&E is
safe for release if the null hypothesis is not rejected.

If under scenario A, for example, the true value of the radionuclide level (or level above
background) is less than or equal to DL then the survey will have power 1-|3 to reject the null
hypothesis that the true value is equal to the AL with type I error a. Under scenario B, if the
value of true contamination-background is greater than the detectable difference A, then the
study will again have power 1-|3 to reject this null hypothesis at type I error rate a. Assuming
that the standard error of the estimate a, does not depend upon the radiation levels being
measured, the formula for the "detectable" A, given a, a and power 1-|3 is

Detectable difference A = (Zx_p + Z, u)a	(1)

Where Zx_p and Zx_a are the corresponding critical regions for the standard normal random

variable. A somewhat more complicated formulae for A is needed when a is not independent of
radiation level as in Table 5.1; however, formulae (1) gives a useful (conservative)
approximation to the detectable difference if we choose a to be at its maximum likely value for
either the null or alternative hypothesis.

In general, use of equation (1) for the detectable difference A requires that the estimate of
contamination (measurement - background) be approximately normally distributed. For
radiation counters with long count times and large values of N (when there is sampling
variability as well as measurement variability), this assumption is usually quite appropriate.
Because the width of A (for fixed power and type I error) depends on a, it is important that an
instrument or measurement technique (and sampling fraction for spatially distributed
contamination) is selected which is sensitive enough (provides small enough a) so that the

28

-------
1

detectable A meets requirements (for example so that the DL is not set to be too small in
Scenario A, or that the upper range of the gray region is not set too high above background in
Scenario B).

In some situations (non-normal distributions, short count times), the detectable A will be
larger than described in equation (1) and more specialized statistical analysis may be needed.
Such techniques as segregation according to likely level of contamination may improve the
accuracy of equation (1), as will longer count times.

Hypothesis testing (accepting or rejecting the null hypothesis) involves comparing an
estimate of contamination level to a "critical value" (termed Sc in the manual) which allows us to
decide whether the observed estimate is consistent with the null value (at a certain type I error
level) after taking account of the variability (i.e., a) of the measurement. For Scenario A, this
value is equal to Sc = AL - Zi_a a, and for Scenario B it is Sc = AL + Zi_a a. By definition,
power is the probability, as computed under the alternative hypothesis, of rejecting the null
hypothesis; that is, the probability that the observed estimate is less than (for scenario A) or
greater than (for scenario B) the critical value Sc.

If the normality of the estimate is in doubt, then other approaches to hypothesis testing
may be needed. For example, while for long count times the Poisson distribution can be
approximated as normal for the purpose of hypothesis testing, for short count times, specialized
formulae (see section 5.7.1) may be needed to give a better approximation to the distribution of
(measured-baseline) for an idealized radiation counter.

A-2 Specific Comments:

Section 3.8.1 describes "Measurement Method Uncertainty" but in somewhat more vague
terms than above. The intent of this section could be better understood in reference to the
suggested introduction to experimental design and hypothesis testing.

All of Section 4 would be more comprehensible if it consistently referred back to the
suggested introduction to experimental design and hypothesis testing.

Section 4.4.1.2 gives a recommendation for how much of an impacted material should be
scanned: it is not clear to what the a value now refers (eq 4-1). This appears to be the
measurement error standard deviation oM rather than the total standard deviation of the
measurement method (measurement method uncertainty). Presumably, this is giving a
recommendation that will keep the total measurement method uncertainty bounded for a given
level of measurement error (om).

The statistical concepts described earlier MARSAME are illustrated for the first time in
Figures. 4.2 and 4.3. It is unfortunate that even though the concepts shown in the figures all
relate to net radioactivity, they are termed "level," "value" or "limit." This could cause
misinterpretation by someone who is preparing to establish a survey design. An expansion of

-------
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

SAB Draft Report dated April 24, 2008 - Quality Review Draft for Panel Review - Do Not Cite or Quote. This review
draft is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by
the Science Advisory Board's Charter Board, and does not represent EPA policy.

these figures to include several additional parameters with some supplemental text would be
helpful.

Recommendations for scenario A and B are presented in Figs. A-l and A-2. These
embellished Figures with some additional text should also eliminate the need to repeat this
information in Chapter 5, as in Figs. 5.2, 5.3, 5.4.

Scenario A
(H0: Net Activity £ Action Level)

Figure A-l. Scenario A.

Scenario B
(H0: Net Activity < Action Level)

Figure A-2. Scenario B

As mentioned above, the Action Level (AL) for net excess radioactivity is used in
defining the null hypothesis. However, the decision on accepting the null hypothesis is not based
on the numerical value of net radioactivity at the AL. Rather, each sample is compared with the
Critical Value shown in the Figures. This insures that the probability for rejecting the

30

-------
1

null hypothesis, when it is true, will not exceed a. The Discrimination Limit (DL) is the net
radioactivity in the sample where the probability of accepting the null hypothesis, when it is
false, is (3 (i.e. the power for rejecting the null hypothesis is 1-|3). The Gray area is the region of
net radioactivity in the sample where the statistical power to reject the null hypothesis, when it is
false, is less than 1-|3.

Application of Measurement Quality Objectives (MQOs) discussed in Section 5.5 is an
operational aspect of the MARSAME process. MQOs are part of the Data Quality Objectives
process (DQOs) used as a platform for both the MARSSIM and MARSAME process. Use of
MQOs was not incorporated into the MARSSIM process, so it maintains a unique role to
MARSAME. The application of MQOs is fairly new to Decommissioning planning. It was
employed in MARLAP in 2004 for laboratory-based measurements and now has been extended
to field measurements in MARSAME. The Guide to the Expression of Uncertainty in
Measurement (GUM), which forms the basis for much of the conceptual and statistical
framework of MQOs, was published by the International Standards Organization (ISO) and the
National Institute of Standards and Technology (NIST) in 1995 and 1994, respectively. The
topic of MQOs may be unfamiliar to many users of the MARSAME. For this reason, it is
important to provide a sound basis for the operational and statistical aspects of its use.

Some SAB MARSAME Review Panel comments, in the text and in this Appendix,
specifically address the theoretical foundations of the underlying statistical assumptions used in
the mathematical relationships and equations. Other panel comments address the application of
MQOs from an operational standpoint. The identification of MQOs for certain types of
measurement cases and survey designs may be confusing to readers unfamiliar to MQO
applications. Considerable detail in the manual is provided on defining, explaining, and deriving
the relevant theoretical concepts. The writers of the MARSAME manual should ensure that
operational information on the implementation of MQOs is not too deeply embedded within the
theoretical discussion. More distinction should be placed on information applicable to
identifying performance characteristics, setting MQOs, and selecting appropriate measurement
methods. Effective use of the manual relies on the reader to be able to apply MQOs to their
specific measurement problem.

A summary or guide, that organizes the measurement uncertainty, detectability, and
quantifiability requirements for each of the three types of MARSAME surveys, including In-
Situ, Scan-only, and MARSSIM-type, would be beneficial to the user. The guide would collect
information on the selection of MQOs, which may be scattered throughout the chapter, into one
coherent presentation for ready reference. The guide would be useful for designing MARSAME
disposition surveys, training activities and for reference when regulators evaluate the
measurement requirements of disposition survey plans.

The presentation of statistical formulations and derivations can be quite detailed and
extensive and, if not properly balanced with the operational aspects of the guidance, may detract
from the clear presentation of the guidance to the target audience. It is important to recognize
that the manual is written for those directing and implementing the process, interpreting results,
and making decisions. The operational aspects of the guidance address this broad audience
directly, however, there is an audience concerned with the scientific and technical soundness of

-------
1

the procedures and the rigor for which the process is founded. An appropriate balance between
the presentation of the operational aspects and the statistical foundations of the guidance is
paramount.

The intent of Section 5.5 would be made clearer as dealing with the factors that impact
the measurement error uncertainty a as described in more general terms in the suggested review
of experimental design and hypothesis testing. Apparently, however, Om (the standard deviation
of a single measurement not taking into account spatial distribution of materials or the variability
of the background) is being confused with the overall a (total measurement method uncertainty
taking these factors into account). It is A / a, not A / oM, that determines the overall power of the
experiment. The document should clearly differentiate these two a 's.

Section 5.5.1 lines 289-293 seems to be confusing Om with as. It is as that, generally
speaking, can be decreased by improving scan coverage (not aMif this includes "shared" error
terms such as the "variance of measured efficiency"). The new terminology uMr apparently
refers either to an estimate of the measurement error uncertainty Om or to overall a but this is not
made clear in this section (and the requirement that uMr < cr/3 makes no sense if as can be
reduced to 0 by improving scan coverage).

The comments on line 302-303 seem to require that uMr estimates the overall a.

Example 2 is confusing because the requirement that uMr be a factor of 10 times smaller than A
seems to assume that uMr is an estimate of om rather than the overall uncertainty a (this would be
a very stringent requirement indeed). Here one needs to focus not just on aM but rather on the
total variability including as. If os can be reduced to zero by scanning all of a material why is
such a stringent requirement made on oM?

Line 360 introduces new and not clearly defined uncertainties (uc and (Pmr). Example 5 is
unclear, and needs to be tied to some general design or hypothesis testing principles - it just
comes out of thin air as it stands.

Section 5.6 is a good description of addressing measurement uncertainty Om in certain
special cases. One thing that could be clarified is that om now refers to the error in measurement
- background rather than just the error in the measurement itself. At other points in the manual,
aM seems to refer rather to the variance of just the measurement.

All determinations of excess radioactivity are based on the difference between a sample
with an unknown amount of radioactivity, and an appropriate control that may contain
radioactivity not related to the source of contamination. MARSAME does not provide very
much information on how to characterize properly the "background" radiation contained in
controls or "reference samples."

Tables 5.1 and 5.2 list equations to determine critical values, Sc. A sample is considered
to contain radioactivity in excess of the control if the "net" result is greater than the Sc. The
value of Sc is based on the probability that the net result of a sample with no excess radioactivity
that will exceed Sc, is equal to a (i.e.false positive). This is, in effect, an example of Scenario B

-------
1

described in Chapter 4. This is expanded in Table 5.2 to the minimum detectable value, Sd. It is
the smallest value of net radioactivity, MDC, that will yield an observed measurement greater
than Sc with a statistical power of 1-|3. That is, the probability that a sample containing exactly
the MDC will be less than Sc is (3 (i.e. false negative).

The equations in Tables 5.1 and 5.2 are used throughout MARSAME as examples for
estimating critical values Sc and MDC. These equations are based on the Poisson assumption for
counting statistics and distribution of the difference between two random numbers that are
Poison distributed. In effect, this implies that an independent measurement of a control is paired
with each measurement of a sample. Sc is based on the distribution of two random numbers
selected from the same distribution of background.

Although the equations are correct, it is not common to measure a control for every
sample of unknown contamination. This process of comparing paired samples is rare.

Generally, an estimate of background radioactivity is established, and subtracted from every
sample to estimate the "net" count.

Tables 5.1 and 5.2 are used throughout MARSAME without any reference to any
assumptions that were used to derive the equations. There could be serious implications in
decisions relating to the presence of radioactivity using Sc and hypothesis testing using MDC as
the DL. On the other hand, for most cases these equations might be satisfactory. It will be
important for the MARSAME manual to clarify this, and to provide more details on how to
measure and characterize "background" in controls that are used to determine "net" activity.

Some examples are shown below. For this case, equations 5.1.1 (Currie) and 5.1.3
(Stapleton) were used to compute Sc. A Monte Carlo model was used to estimate Sc for paired
samples from the true background distribution (MC) and also for a constant background, equal to
the true mean, that was subtracted from a random sample of background (MCB). For these
cases, a = |3 = 0.05. Fig. A-3 is for the case where the sample time ts and the background time tb
are equal and yield a mean count of 200. The abscissa is normalized to the value of Sc obtained
from the Currie equation.

This illustrates that Sc obtained from 5.1.1 does indeed come from a distribution of paired
samples which is simulated in MC. However the value for Sc obtained by subtracting a constant
value equivalent to the mean value of background, MCB, is actually about 30% lower than Sc
from the equations.

Fig. A-4 is for the case where the sample time ts is 5 and the background time tb is 50.
For this case, the background is estimated with greater precision because tb is large. With a
constant background to estimate background, the value of Sc is similar to that obtained from the
equations in Table 5.1.; however both MCB and the Currie equation yield a value of Sc that is
somewhat lower that that obtained from paired samples (MC) by Monte Carlo simulation.

-------
SAB Draft Report dated April 24, 2008 - Quality Review Draft for Panel Review - Do Not Cite or Quote. This review
draft is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by
the Science Advisory Board's Charter Board, and does not represent EPA policy.

Rb = 10 ts= 20, tb = 20



120.0



110.0





c



3

100.0

o



0)



>

90.0

re



0)



£&

80.0

0



if)





70.0



60.0

o

o

Currie (5.5.1) Stapleton (5.1.3)

a

MC

n.

MCB

Fig. A-3. Comparison of Se

Rb = 10 ts= 5, tb = 50



120.0



110.0





£



3

100.0

o



0)



>

90.0

to



0)



£

80.0

o



(/>





70.0



60.0

o



Currie (5.5.1) Stapleton (5.1.3)



MC



MCB

Fig. A-4. Comparison of Sc for longer background counting period

34

-------
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

SAB Draft Report dated April 24, 2008 - Quality Review Draft for Panel Review - Do Not Cite or Quote. This review
draft is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by
the Science Advisory Board's Charter Board, and does not represent EPA policy.

Rb = 10 ts= 20, tb = 10

110.0

t/>

"E

g 100.0

o

> 90.0

O

o

a

a 80.0
W

70.0
60.0

Currie (5.5.1) Stapleton (5.1.3)

-o-

Fig. A-5. Sc for a briefer background counting period

Fig. A-6. J -p as function of % excess count above background

Fig. A-5 is for the case where ts is twice the value of tb. Values obtained for Sc using the
Currie equation are close to the value from the Monte Carlo simulation for paired samples, but
the estimate of Sc using constant value of background is low by about 40%.

Fig. A-6 shows an example of the statistical power, l-(3, as a function of the increasing
amounts of radioactivity above background. The blue curve (the curve starting on the ordinate at
a statistical power, l-(3, of 0.05) represents the simulation for paired samples and the red curve
(the curve starting at the origin) represents the simulation when a constant value of background
is subtracted from the sample to form the net value. Without excess radioactivity, (3 for the
paired samples is 0.05 and (3 = 0.01 when background is a constant. The two curves are identical
when the excess radioactivity corresponds to Sc and therefore (3 = 0.5. The vertical line
corresponds to the value of M DC obtained from equation 5.2.1. Note that the MDC, (1 -(3) =
0.95, obtained from the simulation with constant value for background is smaller than when
using the assumption of paired samples.

35

-------
1

MARLAP provides additional modifications to estimating Sc when the Poisson
approximation may not be satisfied. However, it is not clear that the concerns relating to the
process of measuring controls or reference materials have been eliminated.

Many equations have been suggested for designing and interpreting survey procedures in
MARSAME. The equations are derived from sound statistical principles. They can lead to
incorrect conclusions if the underlying assumptions in the derivations are not satisfied. The
Panel does not recommend that each equation be derived in detail, but suggests that the
assumptions and sampling requirements needed to properly implement equations be documented
in MARSAME.

Section 5.8, Determining Measurement Quantifiability is a complicated way of saying
that a must be small enough (and hence A /a large enough) for the measurement method to have
good power to reject the null hypothesis that the level of radioactivity is at the AL for a
reasonable A (width of the gray region). It also must give a reasonably narrow confidence limit
for the estimated value, i.e. where the width of the confidence limit is small compared to the
value of the AL.

One complication that is explicitly dealt with in the definition of the MQC is that the
measurement method uncertainty, i.e. a, generally will depend upon the (unknown) true level of
radioactivity itself - for example a perfect counter has Poisson variance equal to its mean. Thus
the MDC is just the value, yO, of the radioactivity level for which the ratio, k=y0/a, is large (the
manual recommends k=10). If yO is small relative to the action limit (between 10-50 percent of
the AL is recommended), then it is clear that (1) the detectable A will be small with respect to the
action limit (i.e. the DL will be close to the AL) and (2) confidence limits around an estimated
value of radioactivity will be narrow relative to the value of the AL. Saying this clearly
improves the intelligibility of this section.

Section 5.8.1 would be more intelligible if it first noted that it is giving a computation of
the MDC, yO, for a fixed k by a formulae for a that takes account of several factors which are
combined into this one a. These factors are the length of the reading time for the source, the
length of reading time for the background, the true value of the background reading, and an
estimate of the variance of a "shared" measurement error term, i.e. the measured efficiency of the
monitor.

Section 6.2.1 has some confusing aspects: as described earlier, the gray region is defined
in terms of the power and type I error of the test with a measurement method of total standard
deviation a. Sentences like "Clearly MDCs must be capable of detecting radionuclide
concentrations or levels of radioactivity at or below the upper bound of the gray region" seem
tautological if the gray region is defined in terms of detection ability; specifically in terms of
power, type 1 error, and a.

Lines 215-224 of Section 6.2.3 confuse by the statements about how individual
measurement results can be utilized for scan-only measurements. The statement that "if

-------
1

disposition decisions will be made based on the mean of the logged data, an upper confidence
level for the mean is calculated and compared to the UBGR," must be interpreted carefully. If
one did a standard test such as Wilcoxan or t-test) one would ignore any uncertainty component
resulting from variability in the measurement process (i.e. measurement error shared by all
measurements that constitute the scan). Only if aM has no shared components (or if they are
very small) would it make sense to do a standard statistical test of the observed data alone.
Specifically, the sample standard deviation would underestimate the true measurement standard
deviation a if a shared uncertainty (such as errors in the estimate of counting efficiency) is
incorporated in oM.

The recommendation (line 60) that for MARSSIM type surveys the sample standard
deviation can be used to generate a power curve also implicitly assumes that no shared
measurement error components exist. But this contradicts the conclusion of line 223-224 that
"Measuring 100% of the M&E accounts for spatial variability but there is still an uncertainty
component resulting from variability in the measurement process." In fact, all the discussion of
selecting and performing a statistical test, and drawing conclusions in the rest of Section 6 seems
to be implicitly assuming that there are no shared errors from measurement to measurement. Is
this the intention? Was this what was being meant by the (confusing) discussion in 5.5.1 lines
289-293? For example, even if all measurements are less than the action level, this might not
really be enough information to conclude that the M&E meet the disposition criterion.

Suppose all measurements are only somewhat less than the action level but it is also
known that the counting efficiency was not well estimated. Ignoring the uncertainty in the
counting efficiency could lead to the wrong conclusion in this case, if the uncertainty in the
counting efficiency is indeed "shared error" over all the measurements. In many places in this
document, errors in counting efficiency or other apparently shared measurement errors are
mentioned (as on line 223-224), but this issue seems to be ignored in most of Section 6. If the
manual assumes that such shared errors are small enough to be ignored, this should be stated
explicitly. (See also footnote 4 on page 6-17).

One possible resolution is to assume that the measurement of background has exactly the
same "shared" uncertainties (counter efficiencies, etc.) as does the measurement of the
radioactivity level in the M&E. In this case, the shared uncertainties will be subtracted when the
background is subtracted from the level measured in the M&E. If this is meant, then it should be
stated clearly (and this should be highlighted in the any initial "review of experimental design
and hypothesis testing" when discussing the various components included in a).

-------
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

SAB Draft Report dated April 24, 2008 - Quality Review Draft for Panel Review - Do Not Cite or Quote. This review
draft is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by
the Science Advisory Board's Charter Board, and does not represent EPA policy.

APPENDIX B -ACRONYMS AND ABBREVIATIONS

A	Scenario A for hypothesis testing

AL	Action Limit (or Level)

ALARA	As Low As Reasonably Achievable

a	Maximum acceptable probability for Type I error rate (alpha)

AM	Arithmetic Mean

|3	Maximum acceptable probability for Type II error rate (Beta)

B	Scenario B for hypothesis testing

1-P	Numerical value of the statistical power to reject the null hypothesis when it is
true

CFR	Code of Federal Regulations

CON	Consultation

CQ	Charge Question (CQ1, CQ 2, CQ3)

A	Difference (Alternative - Null hypothesis), also the Detectable Difference

DFO	Designated Federal Officer

DL	Discrimination Limit (also Discrimination Level)

DLC	Data Life Cycle

DoD	Department of Defense (U.S. DoD)

DOE	Department of Energy (U.S. DOE)

DOT	Department of Transportation (U.S. DOT)

DQO	Data Quality Obj ective

EH	Environmental Safety and Health (U.S. DOE/EH)

EPA	Environmental Protection Agency (U.S. EPA)

FR	Federal Register

GUM	Guide to the Expression of Uncertainty in Measurement

H0	Null Hypothesis

IA	Initial Assessment

IAEA	International Atomic Energy Agency

ISO	International Standards Organization

k	Coverage Factor for Uncertainty

LBGR	Lower Bound of the Gray Region

MARLAP	Multi-Agency Laboratory Analytical Protocols (Manual)

MARSAME Multi-Agency Radiation Survey and Assessment of Materials and Equipment
(Manual)

MARS SIM

Multi-Agency Survey and Site Investigation Manual

M&E

Materials and Equipment

MC

True Background Distribution

MCE

Random Sample of Background

MDC

Minimum Detectable Concentration

MQC

Measurement Quality Uncertainty (also Minimum Quantifiable Concentrations)

MQO

Measurement Quality Objective(s)

N

The Sample Size (N measurements, for instance)

NCRP

National Council on Radiation Protection and Measurements

NHSRC

National Homeland Security Research Center

38

-------
1

NIST National Institute of Standards and Technology

NRC Nuclear Regulatory Commission (U.S. NRC)

NUREG NRC NUclear REGulatorv Guide (U. S. NRC)

OAR Office of Air and Radiation (U.S. EPA/OAR)

ORIA Office of Radiation and Indoor Air (U.S. EPA/OAR/ORIA)

PAG Protective Action Guide

pdf Portable Document Format

q critical value for statistical tests

QA Quality Assurance

QC Quality Control

QA/QC Quality Assurance/Quality Control

Rb Mean Background Count Rate

RAC Radiation Advisory Committee (U.S. EPA/SAB/RAC)

rev Revision

SAB Science Advisory Board (U.S. EPA/SAB)

a Standard deviation

Om Standard Deviation of Measurement Error

as Standard Deviation of Sampling Distribution

Sc Critical Value

SI International System of Units (from NIST, as defined by the General Conference

of Weights & Measures in 1960)

SOP Standard Operating Procedure

(I>nir The relative upper bound of the estimated measurement method uncertainty ]lmr,

ts Background Time

ts Sample Time

TSCA Toxic Substances Control Act

Type I Type I error is rejecting the null hypothesis when it is true

Type II Type II error is failing to reject the null hypothesis when it is false

fimr Estimated Measurement Method Uncertainty

cp Uncertainty (e.g., (Pmr)

UBGR Upper Bound of the Gray Region

UCL Upper Control Limit

US United States

Wr Adjusted Reference Measurement (WRS test)

Ws Sum of the Ranks of the Sample Measurements (WRS test)

WRS Wilcoxon Rank Sum Test

yO Estimate of Zero Order Output Quantity; also Minimum Detectible Concentration

Z Critical Regions (e.g., Z i. a, or Z i. p, that is, quantile of the standard normal
distribution)

-------
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

SAB Draft Report dated April 24, 2008 - Quality Review Draft for Panel Review - Do Not Cite or Quote. This review
draft is a work in progress, does not reflect consensus advice or recommendations, has not been reviewed or approved by
the Science Advisory Board's Charter Board, and does not represent EPA policy.

APPENDIX C -MARSAME TYPOS AND CORRECTIONS

(NOTE: Can be kept here, but original intent was to have this moved from the report to a memo
from the RAC MARSAME Review Panel DFO to the Multi-Agency Work Group via the ORIA
Staff Office — KJK)

xxix line 504 power?

522 delete one (
xxxi 561 delete one )

567 delete one (

xxxiv	671 Technetium (sp.)

xxxv	676 delete (duplicates 675)

1-3	80 change "activity concentrations" to "area activity" or leave as is but change

"Bq/m2" to "Bq/m3" and add "and area activity (Bq/m2)

3-9	194 non-radionuclide-specific (insert dash)

4-5	Figure 4.1a replace second "Large" by "Much Larger"

Figure 4b. replace second "Small" by "Equally Small or Smaller"

5-21	523 value in denominator should be 0.4176 (see line 527)

527 plus should be behind square root of 87

5-53	1148 delete 2nd period

6-6	142 insert "to" behind "likely"

6-11 280 insert "that" behind "determine"

6-13 329 insert "that" behind "demonstrate"

6-23	474 and 482 critical value in symbols table is not in italics (italicized k is coverage

factor)

7-10	210 Tl-208 should be beta/gamma, not just beta, with gamma-ray energy in next
column

B-6 151 maximize, not minimize
D-9 219 what does "varies" mean?

D-36 849 for LS spectrometer, insert (alpha) on first line of column 2 and (gamma) for the

HPGE and Nal detectors
F-l	26 delete (FRER)

End of Document

40

-------
June 2, 2008

Compilation of Comments on MARSAME
LEAD REVIEWERS:

1) Dr. David Dzombak:

"Report on Agency Draft entitled 'Multi-Agency Radiation Survey and
Assessment of Materials and Equipment (MARSAME) Manual,' Draft Report for
Comment, December 2006"

(a) Are the original charge questions to the SAB Panel adequately addressed in
the draft report?

The SAB Radiation Advisory Committee (RAC) MARSAME Review Panel has
addressed all of the charge questions. Each of the charge questions appears to be
addressed in sufficient depth, and specific recommendations have been developed
for each of the charge questions and sub-questions.

Several of the charge questions ask the RAC to give their evaluation of the
adequacy or acceptability of some approach or method. In some cases, the RAC
does not answer the basic question of adequacy or acceptability directly, and in
my view an explicit statement should be provided so that the view of the RAC is
clear. Specific instances where this issue arises are as follows:

(i)	Charge Question 2, p. 14, lines 6-7: "Please comment on the technical
acceptability of the statistical methodology considered in MARSAME
The response begins on line 11 of page 14 with an observation about the
relationship of the statistical methodology in MARSAME to that in
MARSSSIM and MARLAP. In my view, the response should begin with
a general statement indicating the RAC's view of the technical
acceptability of the statistical methodology.

(ii)	Charge Question 2a, p. 14, lines 23-24: "Discuss the adequacy of the
procedures outlined for determining measurement uncertainty,
detectability, and quantifiability, as described in MARSAME ..." The
response begins on line 27 with a statement about the similarity in
MARSAME procedures with those in MARS SIM and MARLAP. In my
view, the response should begin with a general statement indicating the
RAC's view of the adequacy of the procedures for determining
measurement uncertainty, detectability, and quantifiability

(iii)	Charge Question 2b, p. 15, lines 9-10: "Discuss the adequacy of the
data assessment process, especially the new assessment procedures ..."
The response begins on line 13 with a statement indicating that "the data
assessment process is carefully presented and thoroughly explored." In my
view, the response should begin with a general statement indicating the
RAC's view of the adequacy of the data assessment process.

-------
(b)	Is the draft report clear and logical?

The organization of the draft report by the SAB RAC MARSAME Review Panel
follows the charge questions directly and is easy to follow. Detailed comments
and suggestions for improving the statistical treatment in the MARSAME report
are provided in an appendix which serves to keep the main body of the report
concise and focused.

There are some specific portions of the text where some revision would improve
clarity. These are listed below.

(i)	The views of the RAC in response to charge questions about the
adequacy and acceptability of some particular approaches and methods
need to be stated more clearly, as discussed in my comments under
question (a) above.

(ii)	Letter to the Administrator.

-	p.2, line 9: The recommendation to "provide training" needs to be
explained a little more here. It is unclear as to whether a training guide is
being requested for inclusion in the report, or whether separate training
outside of the report is being discussed.

-	p.2, lines 17-18: There is a logic problem in the latter part of this
sentence. The first part of the sentence recommends that "illustrative
examples" be used rather than "case studies", while in the latter part of the
same sentence it is recommended to enhance the "illustrative studies so
that they more closely approach that of case studies." This is confusing. I
recommend replacing "illustrative studies" with "illustrative examples",
and "case studies" with "real situations".

-	p.2, line 22: Use of the term "volumetric" to refer to contamination in
the bulk medium requires a little more explanation. "Volumetric
contamination" is an imprecise term of jargon.

(iii)	Executive Summary, p.2, line 3: It is unclear as to whether a training
guide is being requested for inclusion in the report, or whether separate
training outside of the report is being discussed.

(c)	Are the conclusions drawn, and/or recommendations made, supported by the
information in the body of the draft SAB report?

The conclusions drawn and recommendations made are supported by the
information in the body of the draft report. My only recommendation in regard to
this question is that some of the conclusions of the RAC need to be stated more
clearly. Please see my comments under question (a) above.

Miscellaneous typos and cleanup needed:

(i)	Letter to the Administrator, line 11: delete "Re" after "Subject:"

(ii)	Executive Summary, p.l, lines 38-39: additional text needed

(iii)	Executive Summary, p.2, line 23: delete extra "in sufficient detail"

(iv)	p.5, lines 40-41: additional text needed

-------
(v)	p. 15, lines 20-21: insert "in all MARSAME chapters" after
"distinguishing" in line 20, and remove "in all MARSAME chapters"
from line 21

(vi)	p. 15, line 31: the units should be cm2, not cm3

(vii)	p.21, line 8: spell out Action Level in place of AL

2) Dr. Michael McFarland:

The SAB review panel (Panel) is commended for providing a clear and
concise scientific evaluation of the Multi-Agency Radiation Survey and
Assessment of Materials and Equipment (MARSAME) Manual. The Panel
is applauded for highlighting the need of MARSAME to provide improved
clarity and guidance in its description and use of statistical approaches for
conducting radiation surveys and data assessments as well as its support
for the use of sentinel measurements for initial radiation assessments. In
addition to the following section, which summarizes specific responses to
the quality review charge questions, a number of editorial corrections are
provided at the end of the review comments.

a)	Are the original charge questions to the SAB panel adequately
addressed in the draft report?

The Panel has systematically and comprehensively addressed each
of the original charge questions posed by the Agency. Within each
response, the Panel provides multiple technical recommendations
rich in detail for Agency consideration including specific
recommendations for improving the clarity of a number of report
figures.

b)	Is the draft report is clear and logical?

On the whole, the draft report is clear and unambiguous. The Panel
provides comprehensive descriptions of its salient findings as well
as detailed recommendations for improving the framework and
content of MARSAME.

c)	The conclusions drawn and/or recommendations made are
supported by information in the body of the draft SAB report.

In general, all of the conclusions and recommendations are
supported by information contained in the body of the report. The
cover letter and executive summary provide a succinct yet
compelling summary of the Panel's findings and recommendations
all of which are corroborated by strong scientific and/or technical
arguments that are fully described within the report.

-------
Editorial Issues

Page 1 - Executive Summary Line 17: It isn't entirely clear what
the Panel means by the phrase "competent radiation protection
professionals". Are competent radiation protection professionals
those who are licensed, certified and/or trained in an "accredited"
program? Clarifying the language would make this statement
unambiguous.

Page 2 - Executive Summary Lines 23-24: The phrase "in
sufficient detail" is repeated twice in the same sentence.

Page 22 (Line 6-7) - The report states "Selection (of the value of
a and/or (3) may be a matter of acceptable uncertainty specified by
the agency the sets the action level." In this report and all others
that have espoused the use of the data quality objectives (DQO)
process, the issue of how to establish the tolerable probability
limits of committing Type I and Type II errors arises. Although it
was not an explicit request made within the original charge,
identification of the factors that should be considered by any
agency in establishing the tolerable Type I and Type II error rates,
e.g., practical consequences of committing a Type I and/or Type II
error, assessment/monitoring budget etc., is vital if the Panel
supports the DQO approach to decision-making. If the Panel
anticipates addressing this issue as part of its overall
recommendation for a separate chapter on statistical analysis,
experimental design and hypothesis testing, that should be clearly
stated somewhere in the report.

Page 30 - Appendix A Line 16 - The word "insures" should be
changed to "ensures".

Page 33 - Appendix A Lines 28-29: The example states that a =
|3 = 0.05. Again, it is unclear whether the tolerable probability
limits of committing Type I and Type II errors have been
established by policy, costs or an evaluation of the consequences
associated with committing a specific type of error. Highlighting
the scientific, regulatory and/or policy considerations that resulted
in the established error rates would be helpful in guiding future
users of MARSAME.

Page 37 - Appendix A Lines 17 -19: A series of questions are
posed by the Panel to the Agency. It isn't clear whether the Panel
is expecting the Agency to respond to these questions or if the
questions are merely rhetorical.

-------
3) Dr. Tom Burke:

General comments:

Overall the review panel has been a very good job addressing the questions. They
may however have been a bit too polite concerning the clarity of the MARSAME
draft. I find the draft itself to be a very difficult read and am concerned that it may
not adequately address the information needs of the users. Perhaps this is
reflected in the panel's first recommendation to "provide training". The report is
in essence a very complex cookbook with varying degrees of detail on methods
that range from very specific quantitative approaches to very general qualitative
approaches to selecting action levels. The MARSAME is very strong on process
but does not clearly present the underlying environmental protection and public
health goals.

(a)	Are the original charge questions to the SAB Panel adequately addressed in
the draft report?

The MARSAME Review Panel has addressed all of the charge questions.

(b)	Is the draft report clear and logical?

Project report is clear and logical and the recommendations will greatly improve
the clarity of the MARSAME document.

(c)	Are the conclusions drawn, and/or recommendations made, supported by the
information in the body of the draft SAB report?

The conclusions and recommendations are well supported throughout the body
of the draft SAB report. However there is one conclusion that may be overly
optimistic: page 11 line 5 question lb "The decision rules are admirably clear".
This conclusion is not consistent with the multiple recommendations for
clarifying this chapter.

Other comments:

The MARSAME draft needs to more clearly present the target audience of users,
and the intended or actual applications of the process. What are the most likely
uses of the approach? Might there be future broader applications in the event of
natural disasters or accidental or intentional contamination scenarios?

I strongly agree with the recommendations for strengthening the case studies by
including actual application of the roadmap and decision rules. This should
include plans for evaluation of the adequacy of the approach under real-world
conditions.

-------
4) Thomas Theis:

I have read the RAC review of the Marsame report, as well as the report itself. In
general I think the RAC did an admirable job of answering the charge and
providing useful comments to the multi-agency group that put the report together.

Reports such as Marsame have a difficult job in trying to explain the intricacies,
details, and basis for guidance involving complex topics such as radiation safety.
This is further confounded by the twin desires to write a report that is readable to
more than one level of user (e.g. practitioner, theoretician, manager, etc.), and be
as technically complete as possible. Marsame appears to contain all, or most, of
the information needed to provide expert guidance for radiation assessments of
M&E. Yet the RAC suggests, and for the most part I agree, that some
clarifications are needed, some reorganization desirable, and in some instances
different emphasis needed in parts of the report.

Three of the recommendations (lb-3, lc-1, and 2a-l) suggest the addition of a
new chapter in which mathematical and statistical details are segregated from
other chapters in the report, with Appendix G being incorporated into the main
text of this new chapter. These recommendations are clearly motivated by the
desire to provide a more readable report, parts of which could then be read by
different user groups as appropriate. I faced much the same problem as editor of a
journal that tried to serve both the academic and practitioner communities, in
general satisfying neither (one common critique: too many equations; its opposite,
also common, not enough theoretical development). Marsame isn't a collection of
scholarly articles, so the parallel shouldn't be taken too far, but it does suggest the
value of deciding who the main users are so that the "readability" of the report can
be tailored, and appendices used in their appropriate, and necessary, supporting
role. Thus an alternate approach to that proposed by the RAC, which I would
favor, is to place as much background material as possible in the appendices,
relieving the main text of the burden of supplying too much information and
improving the readability in its totality (a third alternative, as I think about it,
might be to place as much detail and information as possible in the body of the
report, minimizing appendices, but then providing a "guide to readers" on which
chapters to read depending on one's function—but this, I think, would be much too
cumbersome).

OTHER MEMBERS:

1) Dr. Rebecca Parkin:

a)	The original charge questions are adequately addressed.

b)	The report is clearly organized and well-written. I agree with the panel that the
statistical methods should be presented in a separate section of MARSAME.

There are several calls for clarity about assumptions, contexts, methods and
terminology; all of which should improve the practical value and use of the report.

-------
The panel's logic supporting its points is almost always described well. One
section of the draft that didn't flow adequately for me was on p. 15, lines 2-7.
There seem to be some gaps in logic or assumptions about reader understanding
here.

c) The conclusions drawn and recommendations are in large part supported by the
text provided.

Minor edits noted:

p. 2, line 23: Delete "in sufficient detail" as it is repeated here

p. 16-17:1 suggest adding color to highlight the changes. Color would aid the

reader in finding the changes more rapidly.

p. 20: What are the black lines with blue boxes indicating? If they only mark the
separation between phases, I don't think they add much to this figure. I actually
think they distract the reader, and suggest that they be removed.

2) Dr. Valerie Thomas:

a)	Yes, the original charge questions were adequately addressed in the report.

b)	the draft report is logical and for the most part clear (although some of the
detailed recommendations are, understandably, hard to follow without in-depth
knowledge of the Marsame report). Three suggestions for clarification are given
below:

i)	In the overall recommendation that as much attention be given to
contamination that is removable and volumetric (Exec Summ p. 2, lines
18-21) it would be helpful to clarify that the types considered are
removable versus non-removable and surface versus volumetric; the
Executive Summary statement is a bit hard to grasp because it refers to
removable, volumetric and undifferentiated contamination.

ii)	Recommendation 3-3 and Figure 3 are not clear. Specifically, what are
EPA staff supposed to do with Figure 3? Is it meant to be inserted into the
Marsame report to help clarify the structure? Or is it a template for
reorganization? Also, what are the three oar-shaped levers in Figure 3
meant to signify?

iii)	The appendix is welcome in that it provides a more comprehensive
discussion that is normally contained in the "responses to charge
questions" format. However, in contrast to the main body of the report
which very clearly identifies recommendations that are referenced in the
Executive Summary, the suggestions made in the Appendix are not shown
as Recommendations and are not cross referenced with the Executive
Summary or the main body of the report. The statistical recommendations
could be made more clearly with introduction of explicit recommendations
in the Appendix, and cross-referencing in the main body of the report.

-------
c)	The recommendations are well supported by information in the body of
the SAB report. The panel is to be commended for the cross-referencing of
the overall recommendations in the Executive Summary with the detailed
recommendations in the body of the report.

3)	Dr. Baruch Fischhoff:

I will be unavailable during the May 29 meeting. However, I did read through
the report quickly and did not see any obvious problems with it (although this
is far from my area of expertise). It seems clearly written, conscientious, and
responsive to the charge.

4)	Dr. Agnes Kane:

I find the draft report to be acceptable.

5)	Dr. Meryl Karol:

a)	The charge questions were adequately addressed in the draft report.

b)	The draft report makes some excellent recommendations, such as organizing
guidance for statistical analyses, design, and hypothesis testing into a separate
chapter, and in an Appendix.

c)	A number of recommendations would gain clarity if their sentence structure
were less complex. For example,

i)	Cover letter, (p.2) lines 26-27 are unclear.

ii)	Executive Summary, (p.2), lines 23-26 could be clarified by
forming 2 (or more) sentences.

iii)	Introduction, (p.4), lines 8-9 are unclear.

iv)	Recommendation lc-1 (p. 12) is unclear.

v)	Recommendation ld-3 (p. 13) lines 7-10.

d)	Appendix A, section A-l would benefit from use of sub-headings, such as:
Null hypothesis

Alternative hypotheses
Experimental power, etc
Variance

e)	Consider inclusion of a list of Definitions

Null hypothesis
Discrimination limit
Variance, etc.

6)	Dr. Jerald Schnoor

I have read the 50 page report from the Radiation Advisory Committee (RAC) of
the SAB reviewing the Agency Draft MARSAME Manual, and I find it to be an
excellent report. It is well written and well organized from the cover letter to the
Table of Contents, and from the Executive Summary through the numbered
sections of the report, which respond to each charge question. RAC makes its
recommendations clearly and persuasively. I especially like the Appendix on

-------
Statistical Analysis (which I have saved and intend to use in my water quality
class).

I agree with the points made on page 8 of the report (lines 24-30) regarding the
importance of stating the null hypothesis and alpha and beta values for Type I and
Type II errors. I also agree with the revisions to Figures 6.3 and 6.4 in the
MARSAME report depicted in the RAC review as Figures 1 and 2.

In summary, I find the report to be exemplary and concur fully with its findings.
As stated by the RAC in its report, the MARSAME manual is " an admirable ...
and competently written effort by staff from several agencies to provide guidance
in an important endeavor".

7) Dr. Steve Heeringa:

A few comments on the draft report on the "MARSAME Manual" follow.

I did not read or study the MARSAME Manual that is the basis for the Panel's
report in detail. Therefore, my comments pertain more to the overall clarity and
organization of the Panel's report than to the accuracy of the specific statements it
contains. In general, the Panel's responses to the charge questions are clear and in
many cases highly specific when pointing out changes or additions that it feels are
needed. I have just a few suggestions and edits that I would like to note.

a)	Introductory letter, Page 2, Lines 1-3: The intent is clear but this might
be rephrased. This reader stumbled a bit in reading this the first time. It
might be broken in two simpler sentences—one acknowledging the
collaborative effort and a second commenting on the writing quality.

b)	Page iii) Strike the ZIP from Dr. Johnson's listing.

c)	Pages 16 and 17 - Including the revised Figures (as opposed to simply
describing the changes in text) is very useful.

d)	Appendix A (General) - The Panel has collected its response to Charge
questions related to the statistical presentation in the Manual in Appendix
A. Given the breadth of the comments, I believe this is an effective
strategy for communicating several messages:

i)	The manual must be more consistent in its discussion of
decision criteria for hypothesis tests, variability in measurement
and sampling variability, and a number of related statistical
concepts.

ii)	The importance of not masking the operational implications of
the statistical decision-making process in the mathematics of
defining the decision regions and rules.

iii)	Specific recommendations (A-2) for reordering some content
correcting/clarifying terminology.

-------
e) Appendix A (Specific)

i)	Page 26, Line 19 "sampling standard distribution" should
probably be "sampling standard deviation"

ii)	Page 26, Line 30 "in favor of the alternative".

iii)	The definition of the statistical decision rules for Scenario A
and B is touched on at several points in the Appendix. Reading
Lines 38-44 on Page 26, it was not clear to me why the null
hypothesis in A is that the measure exceeds the AL threshold and
in B the null is at the AL (indicated to be near background) and the
alternative is that the AL is exceeded. The Panel discusses this
issue on Page 26 and Page 28. In fact, the italicized note on Lines
21-22 of Page 28 confirmed my hunch. At a minimum, I suggest
this sentence be brought forward to Page 26 when the decision
rules for A and B are first discussed. My preference though would
be to suggest that for Scenario A, the Manual present the null for A
as "safe for release if the null is not rejected" just as in Scenario B.
I believe this would make the decision rule for both Scenarios
consistent with standard practice as I know it and would require
only Figure A-2 to illustrate the decision principles. Even in one
sided tests of the type considered here, the concepts of Type I and
Type II error are often difficult for practitioners to visualize.
Flopping the alternative distribution from right to left in the two
scenarios—while not necessarily incorrect—just increases the risk of
confusion.

These are the only comments that I have. My appreciation to the Panel for the
work that they put into this review.

Dr. James Galloway:

I have read the MARSAME Review Panel's report and find that the panel has
more than adequately addressed the SAB charge in a clear and logical manner.
Further, the conclusions/recommendations are supported by the report itself. The
only suggestion for a change is in the letter to the Administrator, page 2, line 32.
Specifically, the use of the phrase 'potentially useful document' implies that it will
only be useful if something is done. Does the panel mean that that 'something' is
to follow their recommendations, or is it something else? In either case, a
clarification would probably be helpful."

Dr. James Sanders:

Charge questions adequately addressed?

The draft report addresses the charge questions in clear, tightly focused fashion.
The review panel should be commended for their attention to each question, and
for also identifying several other areas that, if considered, would result in a more
valuable report. In particular, the suggestion to pull the discussions of statistical

-------
analyses into a separate chapter will improve readability, and the panel's desire to
ensure a report that will serve management personnel as well as professionals will
be of value overall.

Report clear and logical?

The panel has written a short report that is logical and easy to understand. Their
attention to statistical issues, while more detailed, is still clear to the outside
reader.

Support for conclusions and recommendations?

The panel's recommendations all flow from either the MARSAME draft itself, or
from the panel's deliberations. It is possible for the reader to follow the panel's
arguments, and to find support for their recommendations. In addition, the
general conclusion that the MARSAME draft will be a valuable addition is also
supported.

Other comments:

One typo on p.l, 1 37. I believe that the panel meant March 10, 2008.

10)	Dr. James Hammitt:

Dr. Hammit responded in the affirmative on the report and did not provide
additional comment.

11)	Dr. Steve Roberts:

General comments:

The SAB MARSAME Panel Report is constructive, well-written, and well
thought out. Responses to the charge questions are on-target, and each of the
charge questions is adequately addressed. The organization of the report is
logical, listing each charge question and the Panel's response. Recommendations
are clear and linked logically to the charge questions. Overall, this is an excellent
report that should be quite useful to the Agency.

I do, however, have a few suggestions for improvement. As often happens with
these reports, the body of the report is quite lucid, but some of that clarity gets lost
in preparing a condensed Executive Summary. Even more is lost trying to
construct a concise cover letter to the Administrator. In the case of this report, the
Executive Summary is actually pretty good, but the summary of the main Panel
recommendations in the letter to Administrator Johnson has been diminished to
the point of being cryptic. Some are almost incomprehensible without having
read the report first (e.g., the fifth bullet), and for others the summary description
is not entirely faithful to the actual recommendation in the report (e.g., the third
bullet). In my opinion, the cover letter should not try to summarize the report in

-------
200 words, but should instead highlight a few key points that the Administrator
should know (e.g., in this case, 1) what the MARSAME manual is; 2) that the
MARSAME manual we reviewed is quite good; 3) we have some suggestions to
make it even better — with maybe an example or two; and 4) this is part of a really
useful program that the Agency should continue to support).

Picky details:

Cover letter

P2,L9: delete "important" Are there unimportant users?

P2,L29-30: Actually, all of the recommendations concern refinements and
improvements in content and presentation. Do we mean to say "Other Panel
recommendations concern additional refinements ..."?

Executive Summary:

Obviously, notes to "KJK" need to be deleted.

P2,L23: "in sufficient detail" is repeated

Main Report

Delete notes to "KJK"

12) Dr. Virginia Dale

The original change question to SAB Panel were adequately addressed
The draft report is clear and logical (with minor exceptions noted below)
The conclusions drawn are supported by the report (with wording exceptions
noted below). Appendix A seems particularly useful to the Agency. The
recommendations are clearly designed to assist in the clarity and implementation
of the report.

The SAB report needs to have an editor go over it to eliminate several wording,
punctuation and grammatical errors.

Minor wording points:

Letter:

Page 2, line 2- replace "to provide" with "that provides"

Page 2, line 4 - eliminate comma

Page 2, lines 17-18 - eliminate "so that they more closer approach that of case
studies" for this leaves the reader wondering what the differences are been
illustrative examples and case studies. Also, it is not clear what "enhance the
content "means here or "assure their realism" means in the executive summary.
Page 2, line 31 - Modify sentence so that it begins "After the SAB's
recommendations are implemented, the MARSAME Manual draft should be a
useful document... "for otherwise it sounds rather negative.

General comment - The letter spends too much space explaining what
MARSAME. It would helpful if the background be presented in less space so that
the important points beginning on page 2, line 8 can occur earlier in the letter.

-------
Report

Page 1, line 15 - use "comprised of' instead of "comprising"

Page 1, line 24 - replace "be in nature" with "occur under natural conditions"

Page 2, line 13 - not clear what or "to assure realism" means.

Page 2 -1 do not think it is useful to refer to the numbered sections where the

charge questions are answered.

Page 2, line 23-24 - "in sufficient detail" is repeated. [Clearly the report needs to
have an editor go over it to eliminate such problems].

Page 2, lines 30-31 - This could be a bit more specific.

Page 3, lines 1 - Scenarios A and B have not been defined, so this section needs
rewording.

Page 3 - It seems that the executive summary should end with a sentence stating
the overall value and importance of the report.

Page 4, Lines 32 to next page -1 do not see the value of giving the table of
contents of the report.

Page 5, Lines 30-41 - Too many details on meetings are included here.

13)	Dr. Kerry Smith:

I looked quickly at the report on the MARSAME Manual; the review looks fine
and does address the primary comments. My quick review raised only one
question —what are the provisions for periodic updates as technology changes —
shouldn't we build in a process for review and updates in fields with rapidly
changing technologies? This may be there and I missed it —it is my only question

14)	Dr. Kathy Segerson:

I have reviewed the draft report of the MARSAME Manual review panel. This is
way outside my area of expertise but here is my reaction.

In my view, the draft report is generally very responsive to the charge questions.
The only possible exception is lc. The Agency asked for advice on the
acceptability of new scan-only and in-situ survey designs. While the draft report
refers to these methods on p. 12, line 4,1 did not see any discussion of the
acceptability of these methods. Perhaps it is there, but, if so, it wasn't clear to me.

The draft report is clear and logical, and the conclusions and recommendations
are well-supported. The committee has done a very good job of considering how
the manual will be used and by whom, and making recommendations for
improving its usefulness (particularly for less technical managers).

My only other comment relates to the letter to the Administrator. The letter
describes the MARSAME manual as "an admirable cooperative and competently
written effort" that will be a "potentially useful document for ORIA/EPA..." Yet
the opening statement of the actual review (p. 9, lines 13-14) describes the manual
as "an excellent technical document for guiding an M&E survey." At first glance,
this appears to be a much stronger statement than those included in the letter. But

-------
perhaps the general point the review panel wants to make is that, while the
manual provides excellent technical guidance on conducting the survey, it would
be more useful for the overall process (which includes but is not limited to the
M&E survey itself) if it were revised along the lines suggested in the panel's
recommendations. If this is, in fact, their point, then I would suggest revising the
letter to the administrator (and the executive summary) to make this clearer.

15)	Dr. JanaMilford:

I have reviewed the MARSAME review panel report and find that the panel
adequately addressed the charge questions, the report is clear and logical, and
conclusions are well supported. I particularly appreciated the organization used in
the report, with the comments on experimental design and statistical analysis
pulled out separately. I would recommend promoting the appendix containing
those comments to a full-fledged report chapter, since labeling them as an
appendix may suggest they are of secondary importance, which is not the case.

On p. 15, line 31 the units of area are given as cmA3 — is this a typo?

On p. 35, lines 14 - 16, the report refers to blue and red curves in the figure, which
is inappropriate since the report is not printed in color.

16)	Dr. Rogene Henderson:

I have reviewed the charge questions and the report of the SAB subcommittee on
the MARSAME manual. I did not review Appendix A in detail. I found the
report to be clear and logical as well as responsive to the original charge
questions. The conclusions and recommendations were supported by information
in the text.

17)	M. Granger Morgan:

The report needs some editing. English is often not the best

On page 9 the report says that MARSAME is intended for a "technical audience
having knowledge of health physics and an understanding of statistics." If this is
a direct quote, then it strikes me that all the committee's arguments that there
needs to be discussions for semi-technical users etc. should be made more in the
form of recommendations and less as firm conclusions.

The report says on page 9 that this is "an excellent technical document for guiding
an M&E survey." SO, why does the letter say the document is "a potentially
useful document"?

In the letter the first recommendation has been generalized from the text that
appears in the report on the top of page 2 of the report. The letter should adopt
the same text as on page 2 of the report.

-------
In that text the word must should read should (page 2 of the report lines 15 and
18). This is advice. SAB has no veto power.

The third recommendation in the letter is scrambled. It says they should not be
called case studies and then says they should be rewritten to be more like case
studies.

Why does line 13 on page 2 of the letter start with the word "collect"? Should it
say "develop"?

The text on lines 12-16 on page 8 of the report might be modified to go in the
letter (perhaps in place of some of the existing more detailed suggestions there).

Line 22 page 8 "documented" should read "identified and explained"

Page 13 line 7 drop the word "sheer"

Page 14 lines 13-14. Why "appears.. .to the extent observable" Seems
unnecessarily conditional. Do you have a reason to think it does not?

Page 14 "comprehensible" seems stilted.

18) Dr. Deborah Swackhamer:

Quality Review of SAB RAC Review of Draft of "Multi-Agency Radiation
Survey and Assessment of Materials and Equipment (MARSAME) Manual"

(a)	Are the original charge questions to the SAB Panel adequately addressed in
the draft report? The SAB Radiation Advisory Committee (RAC) MARSAME
Review Panel has addressed all of the charge questions very thoroughly.

(b)	Is the draft report clear and logical? The report is very clear and logical, and
well-organized.

(c)	Are the conclusions drawn, and/or recommendations made, supported by
the information in the body of the draft SAB report? The recommendations are
very well supported by the SAB draft report.

(Very Minor) Editorial comments:

Letter to Administrator: p 2 line 9 (also pl2 line 2, and elsewhere) - how would
this manual be able to provide "training"?

PI 8 line 28-29 - the use of the is inconsistent

Throughout: Why did the RAC decide to label their non-charge question
recommendations with "C"?

-------
Corrections to Report on MARSAME based on SAB Quality Review

p.l, 1.11, letter: Delete 'Re'.

p.l, 1.19, letter: Insert 'MARSAME. The' after 'review of.

p.l, 1.21, letter: Delete '. The Draft Manual' after 'December 2006'; the two changes in lines 19
and 21 are intended for easier reading.

p.2,1. 9, letter: Insert: 'to support use of the Manual.' after 'Provide training'; delete 'and' and
capitalize 'Add'; delete 'important'.

p.2,1.17, letter: Replace 'studies' with 'examples'.

p.2,1.18, letter: Replace 'case studies' with 'real situations'.

p.2,1.20, letter: Replace 'them' with 'the references'.

p.2,1.22, letter: Replace 'volumetric' with 'dispersed throughout the material'; delete
'undifferentiated'.

p.2,1.23, letter: Insert 'that may be either fixed or removable' after 'contamination'.

p.2,1.26-27, letter: Delete 'non-linear processes such as'; replace 'embodied in' with 'such as'.

p.2,1.29, letter: Insert 'additional' after 'concern'.

p.2,1.30, letter: Insert',including the concept that MARSAME be updated periodically',
p.2,1.32, letter: Replace 'to be a potentially' with 'will be a'.

p.2,1.43, letter: Delete 'Chair' after 'Morgan' and after 'Kahn'; repeated on next line,
p.iii, 1.8: Delete Zip code after Dr. Janet A. Johnson.

p.l, 1.17-18: Replace 'competent radiation protection professionals and managers' with '
"technical audiences having knowledge of health physics and statistics" '; note that the phrase is
within quotation marks.

p.l, 1.37: Replace '2007' with '2008' after 'March 10,'.
p.l, 1.38-39: Fill blanks and remove note.

p.2,1.2: Insert: 'and professionals with only limited knowledge of health physics or statistics'
after 'managers'.

p.2,1.16: Replace 'them' with 'the references'.

10

-------
p.2,1.19-20: Replace 'volumetric' with 'dispersed throughout the material volume'; delete
'undifferentiated' and after 'surface contamination' insert '(fixed plus removable)'.

p.2,1.23: Delete 'in sufficient detail', which is repeated.

p.2,1.26-27: Delete 'non-linear processes such as'; replace 'embodied in' with 'such as'.

p.2,1.31: Insert', including updating it periodically' after 'manual' and insert ',3-3' after C-2'.

p.4,1.8-9: Delete 'The presented alternate outcomes are', capitalize 'Release', and add at end of
sentence 'are the alternate outcomes of the survey.'

p.5,1.37: Insert comma after 'report'.

p.5,1.40-41: Fill blanks and remove note.

p.8,1.34: Insert 'survey types' after 'MARSSIM'.

p.9,1.21: Insert 'also' after 'capabilities'.

p.11,1.17: Insert '(distributed throughout the material)' after 'volumetric'.

p. 12,1.13: Replace first sentence with 'In the organization of MARSAME, instead of the current
mixture of general guidance about surveillance with detailed presentations of statistical matters,
retain in each chapter only a brief and less detailed discussion of statistics.'

p. 12,1.26: Insert 'manual' after 'MARSAME'.

p.13,1.2-3: Insert 'a' before 'U.S. Nuclear Regulatory Commission' and 'document' after it.
p. 13,1.9: Replace 'The introduction' with 'This statement'.

p. 14,1.15: Insert new paragraph, 'The statistical methodology applied in MARSAME is
acceptable and not confusing when all three documents are read. Application of comments in
Appendix A to this report and consolidation of the mathematical aspects of MARSAME in a
single chapter as recommended below should enhance use of MARSAME.'.

p. 14,1.30: Insert 'the outlined procedures are adequate, but that' after 'believes that'.

p. 15,1.1-7: Move the last sentence to the front of the paragraph.

p.15,1.13: Insert 'appropriate,' after "process is'.

p. 15,1.20-21: Move 'in all MARSAME chapters' to follow 'distinguishing' on previous line,
p. 15,1.31: Replace 'cm cubed' with 'cm squared', i.e., replace the 3 with a 2 in the exponent.

-------
p. 18,1.6-7: Replace comma with 'and' after 'Nevada (2001)'; delete comma after '(2007)'.
p.18,1.8: Insert 'to' before 'U.S. EPA'.

p. 19,1.29: Replace 'illustrates this suggestion' with 'could be used in the MARSAME Roadmap
to illustrate application of the DQO process in the MARSAME Manual.'; also add the following
sentence, 'Realize also that the DQO process is iterative, so that, as in the case of MARSSIM,
the MARSAME program should have the potential to improve and update the manual.'.

p.20, Figure 3: Replace each of the 3 black horizontal lines that has paddles to the right with a
plain horizontal line that crosses the vertical arrows.

p.21,1. 8: Replace 'AL' with 'action level'

p.22,1.4: Insert 'in the recommended separate chapter on statistics any' after 'discuss',
p.26,1.19: Replace 'distribution' with 'deviation',
p.26,1.30: Replace 'or' with 'of.

p.26,1.44: Insert '(These scenarios are further discussed below)' after 'is over the AL.'.

p.27,1.2: Insert '(Note that terminology used here follows the MARSAME Glossary and list of
Symbols, Nomenclature, and Notations.)' after 'given a.'.

p.29,1.32: Replace 'Section' with 'Chapter'.

p.30,1.16: Replace 'insures' with 'ensures'.

p.33,1.41: Delete period after 'Table 5.1'

p.37,1.18-19: Replace the two sentences 'Was this ... lines 289-293?' with 'If this was the
intention, clarify the confusing discussion in Section 5.5.1, lines 289-293.'

BK May 22, 2008

Corrections Addendum to Report on MARSAME based on SAB Quality Review

p.2,1.2, letter: Replace 'to provide;' with 'that provides',
p.2,1.4, letter: Delete comma after 'manuals'.

p.35,1.14: Delete 'blue curve (the' and delete closing parenthesis on line 15 after '0.05'.

-------
p.35,1.15-16: Delete 'red curve (the' and delete closing parenthesis after 'origin'; both deletions
are needed when report is in black and white.

BK May 23, 2008

Corrections Addendum 2 to Report on MARSAME based on SAB Quality Review

p.2,1.9, letter: Insert 'courses' after 'training',
p.2,1.13, letter: Replace 'Collect' with 'Combine',
p.2,1.15: Replace 'must' with 'should'.

p.8,1.22: Replace 'documented' with 'identified and explained',
p.11,1.6: Delete'admirably',
p.13,1.7: Delete 'sheer'.

p. 14,1.14: Delete "to the extent observable over the wide range of applications'.

p.14,1.18: Replace 'comprehensible' with 'understandable'.

p. 18,1.28: Replace short dash with long dash between '1133' and '1150'.

BK June 2, 2008

40

-------