Intergovernmental Data Quality Task Force Facilitator's Guide for How to Plan Proje Uniform Federal P Assurance Proje (UFP QAPP) Training Workshop Version 1 October 2011 ------- Facilitator's Guide Overview This Facilitator's Guide provides you with the information and insights you need to facilitate the How to Plan Projects Using the Uniform Federal Policy for Quality Assurance Project Plan (UFP QAPP) training workshop. The workshop is intended to take approximately one-half day. The ideal number of participants is from six to twelve. This will allow for plenty of participant involvement. The purpose of the workshop is provide students with knowledge needed to conduct a successful scoping meeting as a part of the Systematic Planning Process (SPP) and then to record the results in a QAPP consistent with the UFP. The roles and responsibilities of the various team members are explored throughout the workshop. The target audience for the workshop includes all personnel who could potentially serve on a hazardous waste site cleanup project team, including Remedial Project Managers; Environmental Protection Agency (EPA) and state regulators; technical support personnel (both government and contractor personnel), e.g., Quality Assurance Specialists, Risk Assessors, Hydrogeologists, Geologists, Biologists, Chemists, Statisticians, Modelers, Lawyers, Health-Physicists, Community Relations Specialists, Information Technology Specialists; On-scene Coordinators; Project Officers and Contracting Officers Representatives (CORs). Participants will get more out of the workshop if they are familiar with the Systematic Planning Process prior to participating in the workshop. This workshop is not an instructor-driven program where your role is to provide lectures and serve as a subject matter expert. In fact, it is the opposite. The workshop's success hinges on the participants finding out for themselves that they have the knowledge and skills necessary to successfully plan projects using the UFP QAPP. Your role as the facilitator is to challenge the participants to use their knowledge, experiences, and insights throughout the workshop and to share them with their fellow participants. The workshop is based upon the Socratic method of learning: you will pose questions of the participants to stimulate critical thinking. This guide provides both the questions and a set of typical answers. However, the answers are not intended to be definitive or all inclusive. Your participants are likely to develop some which are not listed in this guide. Their answers are likely to be as correct as those included here. The DVD for this training package contains several video scenes. Prior to each scene, you will set-up the scene for the participants (example scripts of words for your use or adaptation are provided in this guide in italics). After showing each scene, you will pose questions of the participants. Your challenge is to keep the participants actively involved in all the discussion sessions so they can determine the answers on their own and learn from each other. During the participant discussions, you may find it useful to re-play the scenes: the scenes are short but they have been carefully crafted to emphasize the teaching points. Participants are provided with a Participant's Guide for use throughout the workshop. Included in the Participant's Guide are the How to Plan Projects Using UFP QAPP Facilitator's Guide 2 Version 1 Oct 2011 ------- Pre-Meeting Package (with draft QAPP worksheets) and the completed QAPP worksheets relevant to the project site. The focus of the workshop is the process that is being illustrated in the videos. The purpose of the workshop is not to solve the environmental issues in the hypothetical site being discussed. If the discussion starts to veer toward technical critique of the hypothetical site, the facilitator needs to redirect the discussion back toward the systematic planning process and the role of the scoping meeting in that process. These workshop materials are intended to be a starting place for your workshop. Feel free to tailor the materials to meet the training needs of your participants. Workshop scripts are intended as examples; adapt them as you see fit. These workshop materials were developed by the Intergovernmental Data Quality Task Force Training Subcommittee. How to Plan Projects Using UFP QAPP Facilitator's Guide 3 Version 1 Oct 2011 ------- General Preparation 1) Ideally, arrange the room in a U-Shape so all participants can see and interact with each other. 2) Ensure all participants will be able to see and hear the DVD. 3) Have an easel chart and marking pens available to record any unanswered questions/comments. 4) Be prepared to relate your personal experiences and knowledge. This will enhance each teaching point. 5) Enjoy yourself. Learning is more memorable when it is fun. Classroom Materials Checklist 1) Computer (equipped with a DVD drive and capable of playing video files), speakers and projector. 2) DVD entitled How to Plan Projects Using the Uniform Federal Policy (UFP) for Quality Assurance Project Plans (QAPP) [DVD contains the Facilitator's Guide, the Participant's Guide, PowerPoint slides, video files, and reference documents 3) Easel chart and marking pens 4) Facilitator's Guide 5) Participant's Guide (one for each participant) 6) Recommended: Hardcopies of Intergovernmental Data Quality Task Force, Uniform Federal Policy for Quality Assurance Project Plans, Evaluating, Assessing, and Documenting Environmental Data Collection and Use Programs Part 1: UFP-QAPP Manual, EPA-5050-B-04-900A, DTIC ADA 427785, March 2005 and Part 2B, Quality Assurance/Quality Control Compendium: Minimum QA/QC Activities EPA-505-B-04- 900B, DTIC ADA 426957. (Note: these documents are included on the workshop DVD in the reference documents folder.) How to Plan Projects Using UFP QAPP Facilitator's Guide 4 Version 1 Oct 2011 ------- Workshop Agenda The times listed below are approximate and should be used only as a basic guideline. It is more important to allow the participants to fully discuss the questions posed in the workshop rather than to stick to a set schedule. Breaks should be given at regular intervals, such as ten minute breaks every 50 minutes. facilitator Pre-Video Discussion Video 1 .ength Participant facilitator Question Answer Discussion Time facilitator Summary lireak Workshop Introduction* 30 Min Scene 1. Pie-Meeting between Remedial Project Manager and Contractor 5 min 5 min 15 Min 5 min/10 min Scene 2. Scoping Meeting. Problem Definition 5 min 7 min 15 Min 5 min Scene 3. Scoping Meeting Decision Statements and Sampling Design 5 min 8 min 15 Min 5 min/10 min Scene 4. Scoping Meeting Quality Control Samples 5 min 4 min 15 Min 5 min Scene 5. Scoping Meeting Data I sahilily 5 min 4 min 15 Min 5 min Workshop Conclusion 5 Min * An optional presentation on RPM Roles and Responsibilities may be conducted here. This would extend the Workshop Introduction time from 30 to 50 minutes (approximately) Total Estimated Time: 3-4 hours How to Plan Projects Using UFP QAPP Facilitator's Guide 5 Version 1 Oct 2011 ------- Workshop Introduction Example script: Welcome to the "How to Plan Projects Using the Uniform Federal Policy for Quality Assurance Project Plan Training Workshop. " This workshop is designed to demonstrate the how to conduct a successful scoping session as a part of the Systematic Planning Process as described in the UFP-QAPP manual and how to record the results in of the session in the QAPP worksheets. The focus on this training is how to implement the UFP QAPP manual. Other training has been developed that goes through "what" the UFP QAPP is. This training addresses "how. " The backbone of the Systematic Planning Process detailed in the UFP QAPP is getting all the stakeholders to work together in the planning phase. What makes UFP QAPP successful is that decision points are agreed upon. In this process all project team members are working together for common goal ofprotecting human health and the environmental. This workshop uses videos and participant discussion. For this to be successful I need you to use your knowledge, experiences, and insights throughout the workshop and to share them with your fellow participants. To start us off, here is a video to give you a bit of background on why the UFP QAPP is important: /Show video clip from "The Manager's Role in Assuring Quality Data: Overview of the Uniform Federal Policy for Quality Assurance Project Plans "] As you can see from this video lack ofplanning and proper documentation can result in unnecessary delays and expenditures. The data collected could be of limited use. In this extreme example, the project has no clear focus and thus no end. Welcome and Introduction Go through Introductory PowerPoint Slides (see speaker notes on each slide). An outline of these slides is included in the Participant's Guide. Optional Presentation Included in the workshop materials is an optional presentation on Remedial Project Manager Roles (RPM) and Responsibilities. Throughout the rest of the workshop, roles and responsibilities of the various team members are demonstrated and discussed in an indirect way. Depending upon your participants' experience, you may wish to have a more direct discussion of the roles and responsibilities of RPMs and others involved in the hazardous waste site cleanup process. An outline of the slides is included in the Participant's Guide as Appendix B. How to Plan Projects Using UFP QAPP Facilitator's Guide 6 Version 1 Oct 2011 ------- Scene 1: Pre-Meeting between Remedial Project Manager and Support Contractor Pre-Video Sample Script: Now we will begin Scene 1 of our scoping meeting. This is a pre-meeting between the contractor and the Remedial Project Manager (RPM). One important thing to note is that you should not concentrate on the example being discussed. The focus of the training isn 7 to solve the environmental issues detailed in the video; it is to demonstrate the process and the roles and responsibilities of each of the team members. Please turn to the section of the Participant Guide for Scene 1 and read over the teaching points. Teaching Points: 1. Never go into a scoping meeting with a blank sheet of paper; identify meeting objectives and your criteria for a successful meeting. Ensure the Lead Agency RPM and the Support Contractor have a mutual understanding of meeting objectives and the contractor's responsibilities. 2. Develop a meeting agenda. 3. Distribute packages to the meeting attendees well in advance of the meeting so they can be prepared for the meeting. Clearly identify any issues so participants can be prepared to discuss and reach consensus during the meeting. 4. Focus the scoping meeting on the sticky issues that you need to reach consensus on. 5. The completed QAPP worksheets for a project document the consensus decisions that were reached during the scoping meeting(s). Video: Play Scene 1 Post Video Questions to Pose to Participants: • What was the purpose of this meeting and why was it important? • Could the meeting have been conducted differently? • How did the RPM ensure he and the contractor were on the same page? • What information would be critical to provide to the regulators before the scoping meeting? Participant Guide Instructions: • Request participants to turn to the section for Scene 1. o Find the Pre-Meeting Package. These are the materials that were emailed to the scoping meeting participants prior to the scoping meeting. The package contains the scoping meeting agenda and relevant draft QAPP worksheets. Facilitator Summary & Conclusions: • The Facilitator should reiterate and expound upon the teaching points. • Sample Summary Script: o In preparation for the scoping meeting, the lead agency RPM needs to identify and anticipate controversies that might arise. How to Plan Projects Using UFP QAPP Version 1 Facilitator's Guide 7 Oct 2011 ------- o Have proposed solutions for problems ready (do not just ask the regulators to come up with a solution), o In advance, identify acceptable alternatives for potential issues/disputes, o Be open minded and encourage participation from other team members; although the lead organization may come into the planning process with a draft plan, systematic planning and the final decisions on the path forward should be a team- based effort. o Be prepared to discuss what happens when there is a stalemate on controversial issues. Keep all discussions professional. A functional team needs to have reached agreement on how disagreements will be resolved, o Keep in mind that at the scoping meeting you need to obtain the information that is needed to complete the QAPP worksheets—but don't try to fill out the worksheets during the scoping meeting, o Planning saves time and money in the long run. A well-planned, efficiently run scoping meeting minimizes the cost of the scoping meeting by saving everyone's time. "Schoolhouse Answers" to Questions: • What was the purpose of this meeting and why was it important? RPMand contractor planned out the scoping meeting. They discussed how the meeting would proceed. They came to a mutual understanding of the contractor's responsibilities. They anticipated potential problems that might occur. It was important because it sets the stage for the project team meeting enabling it to be run effectively and efficiently. Before the meeting with the whole project team it is important to provide the team members with the worksheets that contain information about site background, previous meeting minutes, and any draft competed worksheets. • Could the meeting have been conducted differently? This meeting could take place via teleconference or email, but it is critical that the RPM ensures that he and the contractor are on the same page and have the same goals for the meeting. • How did the RPM ensure he and the contractor were on the same page? He met with the contractor and articulated his goals (to get out into the field before the weather turned bad). He emphasized how crucial meeting success was. He clearly delineated the contractor's responsibilities (send out agenda andpre-meetingpackage, during meeting make presentation). They discussed what they already have consensus on and what they need to reach consensus on during the meeting to enable field work to proceed. • What information would be critical to provide to the regulators before the scoping meeting? It is important for all of the meeting participants to know what agenda and the scope of the meeting so the meeting can stay focused (and not go off on a tangent). Provide draft worksheets so the regulators can be prepared to discuss the plans and issues. How to Plan Projects Using UFP QAPP Facilitator's Guide 8 Version 1 Oct 2011 ------- Scene 2: Scoping Meeting—Site History and Problem Definition Pre-Video Sample Script: In this Act the actual scoping meeting begins. This is a highly functional team demonstrating how meetings should be done. The contractor makes a short presentation about the site. Just for your knowledge, this project team has already come to consensus on many issues. From the scenario you will notice that the State regulator is new to the team and is inexperienced. The State has participated in the past but now there has been personnel turnover. This characterization was created to help demonstrate some of the teaching points. Please turn to the section of the Participant Guide for Scene 2 and read over the teaching points. Teaching Points: 1. The proper players, empowered decision makers, need to come to the meeting. In some cases, the lead agency RPM is the leader and the decision maker (however, in some partnering situations all decisions are consensus decisions) 2. Determine the focus of the particular scoping meeting. Keep the meeting from straying to tangents or other issues beyond the scope of the meeting. 3. Agreements on site history and problem definition are critical before starting field work. 4. Document agreements in the QAPP worksheets. 5. The problem definition must be comprehensive and cover all aspects of problem; the more complete the conceptual site model (CSM) the better. 6. Identify where there are data gaps in the CSM. These data gaps may need to be filled by collecting field sampling data. Post Video Questions to Pose to Participants: • What happened in this scene? • What are some techniques to use if group can't reach consensus on the problem statement? • What would you do if during the meeting you found that the scope of the meeting was too large to handle? • What if your regulators are unable or unwilling to meet? What are some alternative methods for accomplishing the goals of this scoping meeting? Summary/Conclusion: • The Facilitator should reiterate and expound upon the teaching points brought up prior to the act. • Sample Summary Script: Agreements made during the scoping meeting are documented in the worksheets. "Schoolhouse Answers" to Questions: • What happened in this scene? Everyone introduced themselves. A cordial and professional tone for the meeting was set. A presentation was made about the site to ensure all How to Plan Projects Using UFP QAPP Facilitator's Guide 9 Version 1 Oct 2011 ------- have a common understanding of the conceptual site model of the site. The aspects of the site that the team has already reach consensus on were reviewed and the consensus was reinforced. Regulators were able to discuss their concerns and have then resolved (e.g., the other sub-areas, areas 5 and 6, were not being forgotten, they would be addressed later, but the scope of this meeting did not include those two areas). The RPMwas clearly in charge of the meeting. He focused this part of the meeting on the agenda item of obtaining consensus on the problem definition. • What are some techniques to use if group can't reach consensus on an issue? Table the issue for now and then come back to it. Bring in a professional facilitator. Elevate the issue for resolution by higher authority. • What would you do if during the meeting you found that the scope of the meeting was too large to handle? Chop the issues down into smaller issues. Move forward on those issues on which you can reach consensus. Identify those aspects that are sticking points and address them separately. • What if your regulators are unable or unwilling to meet? What are some alternative methods for accomplishing the goals of this scoping meeting? Write issues to be addressed on paper and then informally discuss then to get regulators input o Put all comments receivedfrom a regulator in a table with an explanation for each on how they were addressed. Then get consensus from regular on each table entry. (As opposed to sending a revised document and trying to get consensus from the regulator (they would have to try to figure out how each of their comments was addressed). How to Plan Projects Using UFP QAPP Facilitator's Guide 10 Version 1 Oct 2011 ------- Scene 3: Scoping Meeting—Decision Statements and Sampling Design Pre-Video Sample Script: Facilitator: In this act we continue with the scoping meeting and discuss decision statements and sampling design. In this example the term "decision statement" is equivalent to "if/then" statement. Please turn to the section of the Participant Guide for Scene 3 and read over the teaching points. Teaching Points: 1. Decision (If/then) statements and Project Quality Objectives (PQO's) need to be clear (not vague), match the problem definition, and solve the problem. 2. All players have to agree that they will abide by the if/then statements—these are the levels that will be used and the criteria that will be applied. 3. The various team members have different points of view which must be accommodated 4. Differences of opinion will occur. Keep the discussion professional and unemotional/personal. 5. A defensible (scientifically valid) sampling design is needed. 6. Once the sampling design is developed, the team needs to back and check against the if/then statements to ensure comprehensiveness. 7. Need to include documentation of the sampling design rationale in the QAPP to explain why this sampling design was selected. Post Video Questions to Pose to Participants: • What happened in this scene? • How did the RPM keep the meeting from being derailed since the needed technical experts weren't in the room? • What do you do if a regulator wants a whole laundry list of analyses and yet many of the contaminants on the list are not of concern at your site? • The selenium issue was presented here to illustrate that there may be situations where laboratory methods may not be sensitive enough to meet regulatory limits. What would need to be done in such a circumstance? • It has been said that having a scoping meeting with the regulators "showing all your cards." What do you think? Summary/Conclusion by Course Facilitator: • The Facilitator should reiterate and expound upon the teaching points brought up prior to the act. • Sample Summary Script: Ideally, everybody comes to the table with their concerns and then in the meeting you reach consensus. The representatives at the meeting may have gotten input from experts from their own offices (e.g., chemists, hydrogeologists). A common problem in previous projects: the laboratory people were left out of the communication of this portion of the project. If this happens, it can cause problems later on in the project. How to Plan Projects Using UFP QAPP Facilitator's Guide 11 Version 1 Oct 2011 ------- • For a simple site, one recommended way of achieving consensus on decision statements (including the If/Then statements) is to get agreement on a flowchart during the scoping meeting. Then after the meeting, you can craft the language for the If/Then statements. • The problem definition may need to be refined after discussion and consensus on the decision statements. UFP QAPP development is an iterative process. • Notice that the RPM gave in on one point, but defended another (not removing all of the soil if any one sample exceeds). These sessions are give and take. • During the meeting, you don 7 need to address all items on each of the worksheets—just need to address data gaps/ issues. Notice that nowhere in the video are they trying to fill out a worksheet. Rather the information neededfor the worksheets is discussed. Care must be taken to ensure all appropriate worksheets are updated when a decision is made that changes the design of the project. "Schoolhouse Answers" to Questions: • What happened in this scene? This part of the meeting focused on the proposed decision statements. They used a flowchart to graphically depict the decision statements. They ensured they had consensus on each box of the flowchart. When they did not have consensus they discussed the issues, provided explanations which were accepted or they came to a compromise. When changes to the draft sampling design were agreed to, the RPM reminded the contractor to modify all impacted UFP QAPP worksheets. They heard everyone out. The RPM was willing to listen to alternative approaches and they reached a compromised that all could live with. When they reached impasses they developed methodologies to deal with the issues (e.g., set up chemist conference call). • How did the RPM keep the meeting from being derailed since the needed technical experts weren't in the room? Rather than letting the meeting get derailed because experts weren 7 in the room, defer decision until after the experts have conferred. If possible, schedule a meeting/phone call with the experts during the scoping session to ensure it happens in a timely manner. • What do you do if a regulator wants a whole laundry list of analyses and yet many of the contaminants on the list are not of concern at your site? The project team needs to work together to determine what the actual contaminants of concern are and specify them in the QAPP. Avoid "laundry list" of contaminants. Identify any contaminants that the laboratory can 7 reach the required level and what the team will do to evaluate the data. • The selenium issue was presented here to illustrate that there may be situations where laboratory methods may not be sensitive enough to meet regulatory limits. What would need to be done in such a circumstance? If there are no method modifications that can be done to address this issue, the QAPP should document what action will be taken if the results from the analysis are below the laboratory's capability. • It has been said that having a scoping meeting with the regulators "showing all your cards." What do you think? [open discussion] How to Plan Projects Using UFP QAPP Facilitator's Guide 12 Version 1 Oct 2011 ------- Scene 4: Scoping Meeting—Quality Control Samples Pre-Video Sample Script: Facilitator: In this act we continue with the scoping meeting. Before moving forward, the team recaps there decisions up until now and then discusses quality control (QC) samples and the use of the QA/QC Compendium. Please turn to the section of the Participant Guide for Scene 4 and read over the teaching points. Teaching Points: 1. The QA/QC Compendium document is a tool to use for determining QC samples. 2. The project team needs decide whether the QC criteria will meet their project goals. Post Video Questions to Pose to Participants: • What happened in Scene 4? • What decisions were made? • What can you tell me about the QA/QC compendium? • Who else would be good to have in the room during a discussion of QC samples? Summary/Conclusion by Facilitator: • The Facilitator should reiterate and expound upon the teaching points brought up prior to the act. • Sample Summary Script: o During the scoping meeting, it is important to discuss what will happen in the event of QC failures. o Ultimately, project data must be of sufficient quality for decision making: data must be of known and documented quality, appropriate for its intended use. QC samples provide information for evaluating the quality of the data "Schoolhouse Answers" to Questions: • What happened in Scene 4? Before moving on, the RPM wanted to make sure there was consensus that the agreed upon sampling design will allow them to evaluate the decision statements. However, one regulator needed to get her concerns regarding QC samples resolvedfirst. • What decisions were made? Agreed to a set of QC samples for each of the sub areas. • What can you tell me about the QA/QC compendium? [open discussion] • Who else would be good to have in the room during a discussion of QC samples? The project chemist. How to Plan Projects Using UFP QAPP Facilitator's Guide 13 Version 1 Oct 2011 ------- Scene 5: Scoping Meeting—Data Usability Pre-Video Sample Script: Facilitator: In this scene the team discusses how the data will be evaluated after the samples have been collected and analyzed. This is the last scene. Please turn to the section of the Participant Guide for Scene 5 and read over the teaching points. Teaching Points: 1. Before data is used to make decisions, the credibility of the data must be established. There are various methods for evaluating the credibility of data; e. g., field audits, data review (validation, verification). 2. The overall goal is that data must be of sufficient quality for decision making: data must be of known and documented quality, appropriate for its intended use. The UFP QAPP documentation accomplishes this. 3. If there is insufficient information to fulfill the requirements for a UFP QAPP, then the necessary information to know whether the data is useable to support the decision may not be present. 4. Evaluations need to be conducted throughout the process to ensure that the data obtained is useable and credible. Post Video Questions to Pose to Participants: • What happened in this scene? • What decisions were reached? • How did they address data usability? • What techniques/criteria have you used to discuss data usability? Participant Guide Instructions: • Find the post scoping meeting QAPP Worksheets. Note the differences between the draft worksheets that were part of the pre-meeting package. Summary/Conclusion by Facilitator: • The Facilitator should reiterate and expound upon the teaching points brought up prior to the act. • Sample Summary Script: o Correct analytical methods must be selected and agreed upon by the team o Sometimes, during a scoping meeting, not all technical experts are present. Need to find ways to deal with this situation if it arises, o The planning phase of the project is where you need to reach agreement on how the team will reach consensus on the usability of the data, o Data usability is a team decision; although the project chemist may prepare the usability report, it should be presented to the project team for discussion and acceptance. How to Plan Projects Using UFP QAPP Facilitator's Guide 14 Version 1 Oct 2011 ------- o This project team opted to have the contractor prepare a usability report and send out for review. Some teams reconvene to review the usability of the data together. "Schoolhouse Answers" to Questions: • What happened in this scene? The RPM had accomplished all of his goals for the meeting but rather than just stop there, he asked if anyone had anything else they wanted to bring up. Concerned with data review and validation was brought up and discussion ensued. • What decisions were reached? All agreed that third party validation was not necessary for this project. Internal validation, which was consistent with what had occurred on the project in the past, was agreed would be satisfactory. They agreed to do 100% validation. The regulators agreed to a timeline for document review. Commitment was made to get meeting minutes distributed. • How did they address data usability? After data validation, a data usability report will be generated and distributed to the team so all can be in consensus regarding the usability of the data. • What techniques/criteria have you used to discuss data usability? [open discussion] How to Plan Projects Using UFP QAPP Facilitator's Guide 15 Version 1 Oct 2011 ------- Workshop Summary Facilitator: The Uniform Federal Policy for Quality Assurance Project Plans outlines a method using a team based, Systematic Planning Process (SPP) to create and document project plans. The use of this team based approach focuses on addressing the generation of environmental data of known, documented and agreed upon quality, before data collection begins. The UFP QAPP was developed to provide a single, consensus-based framework and approach for consistently implementing the project-specific requirements of ANSFASQ E4 across participating Federal agencies; thereby streamlining processes for both preparing and reviewing QAPPs. As you saw in this training each team member has specific roles and responsibilities which are critical to the successful implementation of the SSP. Thank very much for taking this training. If you have any further questions about the UFP QAPP please visit the EPA 'a Federal Facilities Restoration and Reuse Office (FFRRO) at: http://www.epa. gov/fedfac/ How to Plan Projects Using UFP QAPP Facilitator's Guide 16 Version 1 Oct 2011 ------- |