Highway airfax, Virginia . 22031-1207 703/934-3000 ICF INCORPORATED GLNPO CONTINUOUS QUALITY IMPROVEMENT RETREAT Agenda Chicago Hilton & Towers Thursday & Friday Chicago, Illinois September 14 & 15, 1989 Dav 1 9:00 - 9:15 Why CQI at GLNPO? 9:15 - 9:30 Review Agenda 9:30 - 11:00 The CQI Approach? 11:00 - 11:15 Break 11:15 - 12:30 Overview of CQI Tools 12:30 - 1:45 Lunch 1:45 - 3:30 CQI Tools Continued 3:30 - 3:45 Break 3:45 - 5:00 CQI Tools Continued 5:00 - Meeting with Project Team Leaders Dav 2 9:00 - 10:00 CQI Tools: Review 10:00 - 2:30 Workgroups Meet to Begin CQI Projects (with lunch) 2:30 - 2:45 Break 2:45 - 4:00 Briefings from Workgroups 4:00 - 4:30 Where Do We Go from Here? 4:30 - 4:45 Wrap-up ------- 1 THE COl APPROACH ------- THE BIG THREE TECHNOLOGY ft: s :fl MONEY POWER I - . — -I , 0 0 - ------- C C - ------- FORD MOTOR COMPANY: “QUALITY IS JOB 1” — u!1 0 -u 0 ------- MOTOROLA: “ZERO DEFECTS IN EVERYTHING WE Do” C I-a - C ------- QUALITY IMPROVEMENT TECHNIQUES Li W. EDWARDS DEMING Li JOSEPH M. JURAN u KAORU ISHIKAWA Li PHILIP B. CROSBY Li A. V. FEIGENBAUM — C j C C U I ------- Higher Q U a I i t y U.S. JAPAN Lower ‘ - I 0 ’ 50s 60s 70s Years 80s — 1 7, C a -C a C ------- MALCOLM C. BALDRIGE NATIONAL QUALITY AWARD CRITERIA D LEADERSHIP U INFORMATION AND ANALYSIS i PLANNING U HUMAN RESOURCE UTILIZATION U QUALITY ASSURANCE u QUALITY RESULTS — c i CUSTOMER SATISFACTION ------- 0MB QUALITY IMPROVEMENT PROTOTYPE AWARD RECIPIENTS EQUAL EMPLOYMENT OPPORTUNITY COMMISSION (1988) D NAVAL AVIATION DEPOT, CHERRY POINT, NC (1988) U FEDERAL TAX DEPOSIT SYSTEM, IRS (1988) U ONE-STEP ACCOUNT SERVICE, SAN FRANCISCO DISTRICT OFFICE, IRS (1988) u FRESNO SERVICE CENTER, IRS (1989) C) 0 I ;-’ 0 ------- 0MB QUALITY IMPROVEMENT PROTOTYPE AWARD RECIPIENTS (CONTINUED) u KANSAS CITY MEDICAL CENTER, VETERANS ADMINISTRATION (1989) U LEWIS RESEARCH CENTER, NASA (1989) c i NAVAL PUBLICATIONS AND FORMS CENTER (1989) U NORFOLK NAVAL SHIPYARD (1989) U OGDEN SERVICE CENTER, IRS (1989) — 0 0 ------- Continuous Quality Improvement at EPA eEPA 2 C C 0 C ------- THE PUBLIC SECTOR HAS STARTED TO: U Focus ON THE CONTINUOUS IMPROVEMENT OF ITS PRODUCTS AND SERVICES Li IMPROVE QUALITY TO IMPROVE PRODUCTIVITY U INVOLVE EVERYONE IN THE ORGANIZATION IN MAKING CHANGE HAPPEN — 2 C I- . ’ C ------- F ) CONTINUOUS QUALITY IMPROVEMENT ci Focus ON THE CUSTOMER i RELY ON THE EXPERTS U IMPROVE PRODUCTIVITY BY IMPROVING QUALITY u CONTINUOUSLY IMPROVE YOUR WORK PROCESSES ------- CONTINUOUS QUALITY IMPROVEMENT Focus ON THE CUSTOMER u RELY ON THE EXPERTS Li IMPROVE PRODUCTIVITY BY IMPROVING QUALITY u CONTINUOUSLY IMPROVE YOUR WORK PROCESSES I.I C -3 CT ------- WHO ARE YOUR CUSTOMERS? u WHOSE NEEDS MUST YOU SATISFY? Li WHO USES YOUR PRODUCTS? — : , I - ’ IC ------- CUSTOMERS WHO ARE THEY? INTERNAL ,rrP _f1 4Jj’ 7F- Ip _ I - 1 ’t - f5:’ C 5 ::_ - / j e - - ,ç i L ‘ I EXTERNAL 4 1 Id 1 A t: (t7 ) I — IC U ’ ___________________________________ _________________________________ -c _______________________ ______________________ 1 ------- SUPPLIERS WHO ARE THEY? INTERNAL P - J? &). EXTERNAL 4C ’ 0t)’)e-t j a’ ------- SUPPLIERS INPUT GLNPO’s OPERATION Supplies your inputs Products that you receive What do you do to convert inputs to outputs? (value added) t - 2 1,’I- p ° P i rI * I — -c -S. ------- [ :1::> * * SUPPLIERS INPUT GLNPO’s OPERATION DIRECT OUTPUT CUSTOMERS Supplies your Inputs Products that you receive What do you do to convert Inputs to outputs? (value added) Products/Services that you produce Receive your products/services e ip P i-) ’ 4’c M — 1 4 ii L 1 4- 7 C ’ - ) — .!, -c C ------- I * * SUPPLIERS INPUT GLNPO’s OPERATION DIRECT OUTPUT CUSTOMERS Supplies your Inputs Products that you receive What do you do to convert Inputs to outputs? (value added) Products/Services that you produce Receive your products/services Labs State Environmental Agency Lab analyses Remedial Action Plan Develop remediation plan Review document Report Review of RAP State Environmental Agencies IJC ------- CHANGE U IN THE WAY WORK GETS DONE U IN ATTITUDES U IN PROCEDURES U THAT YOU INITIATE 0 ------- CONTINUOUS QUALITY IMPROVEMENT Focus ON THE CUSTOMER RELY ON THE EXPERTS U IMPROVE PRODUCTIVITY BY IMPROVING QUALITY u CONTINUOUSLY IMPROVE YOUR WORK PROCESSES ------- AMNESTY — rb.) - ------- CONTINUOUS QUALITY IMPROVEMENT U Focus ON THE CUSTOMER u RELY ON THE EXPERTS I IMPROVE PRODUCTIVITY BY IMPROVING QUALITY u CONTINUOUSLY IMPROVE YOUR WORK PROCESSES ------- As Marketing Requested It C - ------- As Sales Ordered It I;- U I I - - : i C ------- As Engineering Designed It 0 ’ — C ------- As Production Manufactured It .T. - ------- As Plant Installed It I ; - . -v C ------- What Customer Wanted! Quality: The extent to which a product or service meets customer requirements and fit for use. is — 2 C -v C - ------- ICF INCORPORATED Point source impact zones exist in the vicinity of some point source discharges. Pending the achievement of the virtual elimination of persistent toxic ‘ substances, the size of such zones shall be reduced to the maximum extent r possible by the best available technology so as to limit the effects of toxic 1 substances in the vicinity of these discharges. rhese zones shall not be acutely toxic to aquatic species, nor shall their recognition be considered a substitute for I adequate treatment or control of discharges at their sources. 5 1-30 ------- TYPES OF WASTE U MATERIAL u TIME OF PEOPLE LI CAPITAL U LOST EFFECTIVENESS ------- TYPES.OF WASTE Material Time of People ? ie: 1 . &?f/IW Capital Lost Effectiveness p N.) ------- TYPES OF WASTE Material Time of People - Vendor/supplier quality - Price - Missing supplies - Excess or unused products - Work and re-work - Pace - Products not needed - Duplication of effort Capital - Rental assests - Office equipment not fully utilized or not accessable Lost Effectiveness - Delivery time - “Down” time (waiting) - Equipment failure - Underused space - Environment: social and physical - Inappropriate products - Unmet commitments — •1 C ------- I ;-’ TYPES OF WASTE IN CONGRESSIONALS PROCESS Material Time of People - Computer availability for typist - Misplaced disks - Management, staff chief, clerical and staff time (on-disk) - Multiple reviews Capital - Data collection - Equipment failures Lost Effectiveness - Retyping and typos - Access to needed information - Management changes ------- U I VARIATION: POINTS TO STEPS IN THE PROCESS WHERE IMPROVEMENT IS POSSIBLE. ------- ‘-I TYPES OF WORK - i _j-_ p • 9i — . U VALUE-ADDED WORK “ u NECESSARY, BUT NOT VALUE-ADDED u RE-WORK U UNNECESSARY WORK Li NOT WORKING ------- IMPROVING THE WAY YOU WORK U INCREASE VALUE-ADDED WORK U MINIMIZE “NECESSARY BUT NOT VALUE-ADDED” WORK u REDUCE REWORK, UNNECESSARY WORK, AND NOT WORKING — - ‘0 —4 I> I c ------- CONTINUOUS QUALITY IMPROVEMENT c Focus ON THE CUSTOMER U RELY ON THE EXPERTS U IMPROVE PRODUCTIVITY BY IMPROVING QUALITY 11 CONTINUOUSLY IMPROVE YOUR WORK PROCESSESf I — 0 0 tTl ------- CONTINUOUSLY: U Focus ON CUSTOMER U RELY ON EXPERTS LI IMPROVE PRODUCTIVITY BY IMPROVING QUALITY ------- 2 COITOOLS ------- ICF INCORPORATED The Case of Fish Kills in Lake Hernos Background: - Lake Hemos is one of the largest lakes in North America. - Four states border the lake; all are large states with substantial industrial, commercial and agricultural development. - Commercial and sport fishing are major businesses on the lake, but both have been threatened by pollution and contaminated fish. - EPA’s Hemos Program Office (HPO) is charged with responsibility for planning and for developing remedial programs for cleaning up the Lake. - Each of the four states that borders the lake has implemented programs to clean up the lake; their regulatory programs, all modeled after EPA regulations, are very similar. HPO is also responsible for assisting in coordinating these state programs. - Projections done three years ago by lake experts suggested that, given the states’ programs and other factors, the number of fish kills in the lake should decline substantially over that period; however, as the chart on the following page shows, actual fish kills have not declined as rapidly as projected. - Although this slight decline represents some progress, the management of HPO believes that greater progress is both necessary and possible. - Wally Pike is a Senior Analyst with HPO who is familiar with the concepts and tools of Continuous Quality Improvement (CQI). - He has been asked by his management to examine this disturbing trend that has been observed. 2-1 ------- Number of Fish Kills In Lake Hemos (1986-1988) Reporting Period (Quarterly) C ,) 0 1 c i) .0 E z 25 20 15 10 5 0 1/86 2/86 3/86 4/86 1/87 2/87 3/87 4/87 1/88 2/88 3/88 4/88 0 -v 0 ------- ICF INCORPORATED • Our analyst, Wally Pike, wishes to attack this problem using the tools of Continuous Quality Improvement. • His first analytic step is to call together a group of coworkers to brainstorm about the possible causes of fish kills. • To structure this brainstorming activity, he organizes it as a ‘Iishbone” session. • After a brief presentation on preparing “fishbone” diagrams, we will divide into breakout groups to construct a fishbone diagram on the question: What are the causes of fish kills in Lake Hemos? 2-3 ------- 2.1 Fishbone Diagrams ------- ICF INCORPORATED TALKING POINTS FOR FISHBONE DIAGRAM j’. r .b ” t- 1,1 ‘J , WHAT IS A FISHBONE DIAGRAM? • It is a diagram that displays causes and effects, or the reasons behind a particular problem or event. 2. WHAT CAN A FISHBONE DIAGRAM DO FOR YOU? • The fishbone diagram provides a method of structuring or organizing a brainstorming session. • It can help to direct and stimulate a group’s thinking about a problem. • It can help to depersonalize the analysis of the causes of a problem. 3. WHEN DO YOU USE A FISHBONE IN PROBLEM SOLVING? • When trying to get at the causes of a problem. • The fishbone is often used at the outset of a particular project. During initial meetings, a fishbone might be constructed to display the full dimensions of the problem, prior to narrowing the focus of the project. 4. WHAT TYPE OF DATA IS APPROPRIATE FOR A FISHBONE DIAGRAM? • No specific type of data is necessary to construct a f,shbone; only the experience and judgment of the members of the group. • However, it is often worthwhile to receive input from people with different perspectives when constructing a fishbone diagram. This will make the fishbone more comprehensive in identifying causes of a problem or event. 2-4 ------- ICF INCORPORATED 5. HOW DO YOU CONSTRUCT A FISHBONE DIAGRAM? • The first step in constructing a fishbone diagram is identifying the problem or event you want to analyze. • The problem or event will serve as the “fish head,” while the “bones” of the fish each signify a reason or cause for the problem or the event. • The fishbone may be divided up into 3 or 4 major categories of causes. This usually helps to organize the group’s thinking. • For each of these categories, more specific examples of causes are identified. Each of these causes can have their own causes, which in turn, of course, can have even more causes. • In constructing the fishbone, there are no wrong answers. The purpose is to get as many ideas on the table as possible. Don’t get sidetracked by long discussions over whether a particular cause belongs or not, or where in the diagram it should go. • After getting most of the group’s ideas down on paper, some refinement of the structure and wording may be worthwhile. Caution: The point of diminishing returns is reached quickly. 2-5 ------- WHY ARE CONGRESSIONALS OFTEN LATE? Staffing Delay In deleaation Delegated to wrong staffer / H Procedures Congressionals> of ten 7” 77 EPA Division Office provides tech. info late Extension needed but not requested Formal preparation Incorrect TvnIn ------- WHY ARE LMPS DIFFICULT TO DEFINE? Infinite need - Lack of_priority/ // / Resources s, are IMPS difficult to — C ) C ------- OBSTACLES TO REFLECTING TECHNICAL FINDINGS IN POLICY DECISIONS P Cs N) co — : 2 C ‘C ------- ICF INCORPORATED MEANWHILE, BACK AT LAKE HEMOS * *** ** ******* ****** ** * * * *** ***** ******* * 2-9 ------- ICF INCORPORATED Construct a Eishbone diagram on the question: What are the causes of fish kills in Lake Hemos? Blank shells have been provided for your use. 2-10 ------- WHAT ARE THE CAUSES OF FISH KILLS IN LAKE HEMOS? r%) ‘-I — C., C C ------- ICF INCORPORATED • Armed with an idea of the possible dimensions of the problem obtained from the flshbone session, Wally wants to narrow the focus of his analysis. • The scientific issues involved are fair!y complex, there is disagreement among experts, and getting a reasonably conclusive scientific answer to the question of “cause” could require years of research. • As a means of focusing his efforts, he asks a group of experts to assess the fishbone analysis and to rank order what they believe to be the four most likely causes of recent fish kills. • Wally wishes to combine these experts’ rankings and display them graphically. • To do this, he uses the data to construct a Pareto chart. • After a brief presentation on constructing Pareto charts, we will divide into breakout groups to construct a Pareto chart: Causes of fish kills. 2-12 ------- 2.2 Pareto Charts ------- ICF INCORPORATED TALKING POINTS FOR PARETO CHART WHAT IS A PARETO CHART? • A pareto chart is a graph that ranks factors in descending order of frequency, duration, or importance. • The key point is that pareto charts rank the relative significance of various events. 2. WHAT CAN A PARETO CHART DO FOR YOU? • Pareto charts point out the most significant elements of your problem. • Pareto charts are good tools to help you to avoid spending 90 percent of your time on 10 percent of the problem. 3. WHEN DO YOU USE PARETO CHARTS IN PROBLEM SOLVING? • Pareto charts help you decide what you should focus on and are useful in setting project priorities. They indicate where you may need to collect additional data or conduct more analyses. 4. WHAT TYPE OF DATA IS APPROPRIATE FOR PARETO CHARTS? • Pareto charts can be constructed from various types of data. -- Those based on experts’ rankings. -- Those based on empirical or quantitative data. • Look for data that tell you: -- how often; -- how long; or -- how important. 2-13 ------- ICF IINCOR PORATED 5. HOW DO YOU CONSTRUCT A PARETO CHART? • Select an activity, process, or other event of concern. • Select the variable elements you wish to rank and represent those on the horizontal axis. • Select the scale and range of measurements for those elements. Represent the scale on the vertical axis. • Columns should be placed in descending order of height, from left to right. 2-14 ------- REQUIRED SRS INPUT By Workplan Objective S&M ENVSTU SRCE STDS GLWQA OFF EFF EPA COORD Work Plan Objectives 6 5 4 3 2 1 C,) I- a, > 0 a) N D a) E I- 0 L u 0TH Others ------- REASONS GRANT AWARD DOCUMENTS ARE RETURNED BY FMB (FY 1988) Application Incomplete Budget Typos and Budget Special Other lncorrectty Inaccuracies Period Conditions Computed Ended Missing C a) cr 0 > C) C a) 0 a) LL 50 45 40 35 30 25 20 15 10 5 0 22 8 C C Reason For Return ------- ICF INCORPORATED TOTAL PHOSPHORUS IN THE GREAT LAKES (1983 - 1985)* VOLUME WEIGHTED MEAN (VWM) CONCENTRATIONS (uG/L) LAKE 1983. 1984 1985. ERIE 9 8 5 HURON 10 10 11 MICHIGAN 6 6 5 ONTARIO 9 8 8 SUPERIOR 7 7 4 * DATA FROM 1981 REPORT ON GREAT LAKES QUALITY , PAGE 4.2-8, MARCH 1989 2-17 ------- TOTAL PHOSPHORUS IN THE GREAT LAKES (1985) 0 F ) - I - . 0 0 ------- TOTAL PHOSPHORUS IN THE GREAT LAKES (1985) 20 18 16 14 >— . 12 U) C D.2 10 wa) o 8 C 0 0 > 4 C N ) 0 Huron Superior 0 Ontario Erie Michigan ------- ICF INCOR PORATED MEANWHILE, BACK AT LAKE HEMOS 2-20 ------- ICF INCORPORATED EXPERTS’ RATINGS OF LIKELY CAUSES OF FISH KH1 S PERMIT VIOLATIONS 22 OXYGEN DEPLETION 16 BACTERIA VIRUS 9 ACIDIFICATION 9 Using the data presented in the table above, construct a pareto on the likely causes of fish kills. Blank shells have been provided for your use. 2-21 ------- CAUSES OF FISH KILLS (Expert Ranking) ------- CAUSES OF FISH KILLS (Expert Ranking) Causes of Kills ci C) C c t 0 0 E a) > a) 25 20 15 10 5 0 9 F’ ) L ) Permit Violations Oxygen Depletion BacteriaNirus Acidification C ------- ICF INCORPORATED • The likelihood that permit violations are at the root of the problem suggests examining permit violations. However, a simple comparison of the overall numbers of permit violations over the past three years indicates that permit violations have been declining over the past year and a half. • Wally then turns to the possibility that the fish kills are resulting from contaminants released to the lakes through undetected violations. He therefore decides to focus on the compliance inspection process. • To gain a better understanding of the compliance inspection process, he decides to develop a flowchart of the compliance inspection processes used in the four states bordering Lake Hemos. • In order to construct the flowchart, he conducts telephone interviews with the heads of the inspection divisions of the four states’ environmental agencies. • He learns that the four states have compliance inspection processes that involve the same general steps, probably because all the states’ inspection programs are based on U.S. EPA guidance for NPDES inspections. • After a brief presentation on constructing flowcharts, we will divide into breakout groups to construct a flowchart: The permit compliance inspection process. 2-24 ------- 2.3 Flowcharts ------- ICF INCORPOR ’ 1 TED TALKING POINTS FOR FLOW CHARTS 1. WHAT IS A FLOW CHART? • A flow chart is a step-by-step representation of the sequence of events involved in a process: a Every step in the process is represented by a symbol (for example, square or triangle), each signifying a different type of event (for example, an action or a decision). The actual event is described inside the symbol. 2. WHAT CAN A FLOW CHART DO FOR YOU? • A flow chart breaks down a process to its component parts. By doing this it serves several important functions: — It is a tool that helps you understand what the real work is. — It gives you the ability to determine what’s really going on. Often, the perception of what takes place in a process is far different from reality, and different people can have different understandings of the same process. Constructing a flow chart can get you closer to the reality. — It can help you to compare differences in processes that serve similar purposes. — It can help identify who and what is involved in the process. — It can focus attention on the time required to perform various steps in the process. — It can help identify areas where the process can be improved. 2-25 ------- 1CF INCOR PORATED 3. WHEN DO YOU USE A FLOW CHART IN PROBLEM SOLVING? • When you want to understand, in detail, how the work gets done. • It may be the first step in many projects because the flow chart educates those involved and it targets those areas where improvement (and further analysis) may be needed. 4. WHAT TYPE OF DATA IS APPROPRIATE FOR A FLOW CHARV? • A detailed description of the process is necessary. This description can be written or oral. • The sequence of events in the process must be specified. • The players involved and the time required for each step may be important information. • There are many different types of symbols that can be used when constructing a flow chart. 5. HOW DO YOU CONSTRUCT A FLOW CHART? • Identify the process you want to describe. • Identify all steps involved in the process. It is often desirable to consult several people who play different roles in the process, and compare their descriptions. • Determine the sequence of events. • Determine the nature of each step (for example, a decision, an action, or a delay). • Plot the steps sequentially, representing each step with the appropriate symbol. • If appropriate to the problem at hand, identify who performs each step, and how long (on average if necessary) each step takes. 2.26 ------- ICE INCORPOR’\TED FLOW CHART SYMBOLS OPERATION I Conducting a site investigation • Writing a document DECISION (Yes/No Question) • Is the site an emergency? • Is the work assignment over 500 hours? CONTROL/INSPECTION • Proofreading a document • Checking travel requests 2-27 ------- ICF I 1 CORPORATED FLOW CHART SYMBOLS (Continued) DELAY • No computers available - waiting to type document • Memo sits on desk waiting to be signed MOVEMENT/TRANSFER • Mailing a document • Transfering a file FILE • File document • Copy computer disk > 2-28 ------- HOW GLNPO RESPONDS TO A CONGRESSIONAL REQUEST F ) ‘ .0 — .! C C > -3 ------- Great Lakes National Program Office Process for Evaluating Project Proposals for FIscal Year 1990 I Call for Proposals ssibia, F.. ig (GLNPq: • WW i EPA • To O ar r.a..j a Sato A s...u • . •-r-- w . • T,vw,to C . J i.iI Pn a Pfu wl t to GLNPO (pI os—s)• J U N E J U L Y A U G U S T S E P T. GLCC Tlniallna 1 FISCal Year IS I S — 2S i.. SeM Oul Propsa Pro os — 31si—Ci Ol r _ .,.. PPqIa — ion - Piopsa r - U*s ..d flsu. byGL - - Ra, i — 3Ito-flsi.dP.--- J L The Gnat Lakes Coordinating Committee (GLCC) provides advice to GLNPO on workplan dsv.iepm and proje seie uo1L M.mbsnsh of the GLCC is composed of representatives of 1 1w foIewie EPA offices: GLNPO (Chair); Regions II. Ill, and V (Air, Environmental S encss , Waste, Piwmsi and Management, and Water Divisions); the Off ios of International A ivities; and the Office of Waler. Tiw committee reviews GLNPO and other EPA office wortiplans and evaluates project pmposaie wlldrt support GLNPO Great Lakes initiativ.s. Proposals are reviewed for technical merit and priority. I ( Strawman Ranking i v- s eM vr 3a.i.aye ,1 “- , li GLNPO) — Preparation For Proj.ct Review Meeting • GL fl F u Pr - eMito Se. Rw • Sil to ls Put4.ia flst .... Usai i GL r .. ( —.- d by GLM sat u ar is . ...jLL , s . sre. o”r— le ( Projed Review Me.thig with I the COn rU.* fl!.. .. E I Pn4.,lã P( • Co m .u.e $A u,s Rs ..ant Rviwiq of Pm .ls r Final Ranking I Pi S.d Upon Its GL s n&......uLJ.s V — 1411— Q. l ls F si*i tMs i ( Funding Process Begins 0 C T. ) 2-30 ------- ICF INCORPORATED THE OFFICE DIRECTOR GOES TO WASHINGTON The Director of the Hemos Program Office must travel frequently from the program’s office on the shores of Lake Hemos to EPA Headquarters on the shores of the Potomac. When the need for such travel arises, the Director must first decide whether to drive to O’Hare airport, park, then walk to the gate, or to walk from home (in Lincoln Park) to the gate at Midway airport. In either event, the walking distance is about the same. Regardless of which airport the director chooses, the first step is to contact the reservation department of the airline to reserve a seat: center, aisle, or window. Normally, all of the reservation specialists are busy when you call, and you have the opportunity to listen to music for a while -- things like “101 Strings plays Aaron Copland’s Fanfare for the Common Man. ” When the ticket arrives, the Director has learned that it is important to check to see whether the dates and destinations for the flight are as requested. If the Director chooses to fly out of O’Hare, then two hours before flight time, the Director calls for a cab to come to the office immediately. An hour before flight time, the cab arrives. Upon arrival at the airport, the Director pays the cab driver, collects the receipt, and files it for later. The director then checks the nearest monitor to see which gate to run for. When passing through the tunnel to B Concourse, the Director is faced with a difficult decision: run down the main walkway, or run down the moving walkway. Running down the moving walkway is clearly faster, but there is the constant risk that a family of eight with ten pieces of luggage and a stroller will somehow appear on the moving walkway in front of you. Upon arrival at the gate, the Director checks in at the desk, receives a boarding pass, proceeds to board the plane, and waits about an hour for takeoff. If the Director chooses to fly out of Midway, the director walks to the airport, proceeds to the gate, receives a boarding pass, boards the plane, and waits about fifteen minutes for takeoff. 2-31 ------- THE DIRECTOR GOES TO WASHINGTON I .. :% F ’) — - 0 J. - ------- ICF INCORPORATED MEANWHILE, BACK AT LAKE HEMOS 2-33 ------- ICE INCORPORATED Using the information provided in the telephone interview summaries in the four pages that follow, construct flowcharts of the permit compliance inspection process in each of the four states. Bank shells have been provided for your use. 2-34 ------- ICF INCOR PORATED PERMIT COMPLIANCE INSPECTION PROCESS State 1 State 1 conducts on-site inspections br the FIPOES permit program as a regular enforcement activity. They may be scheduled in advance or unannounced. The first step undertaken by the inspector is to conduct background research of the facility: the permit that is currently held, any changes in the statu: of the permit, records of previous inspections and history of violations, and any other factors which may indicate the existence of a problem. Using this information, the inspector prepares an inspection plan of specific issues that need to be addressed during the visit. Once at the facility, the inspector identifies him/herself and states the intention of the visit. Refused entry is noted in the inspection report, and further steps are taken by the inspector to obtain access. The first review that takes place is a records verification and analysis. The permit information on file at the facility is verified, records are examined to assure completeness, and the recordkeeping process is reviewed to determine that all significant information is being tracked correctly. The remaining steps involve the analysis of the facility’s wastewater monitoring equipment. The sampling procedures for pollution discharge detection are examined, as well as sampling and lab analysis equipment. If the inspector feels that an independent sample analysis is necessary, then an effluent sample can be taken. Flow measurement equipment and procedures are then examined. After completing the inspection, the inspector prepares a report outlining the findings and recommending any further enforcement action which may need to be taken. The inspection report is reviewed by the inspector’s supervisor, revised if necessary, and then sent to the Office Director, who is responsible for the final decision on follow-up activities. 2-35 ------- ICF INCORPORATED PERMIT COMPLIANCE INSPECTION PROCESS State 2 On-site inspections for the NPDES permit program are regular enforcement activities of the program in State 2, with each facility scheduled for a visit about once every two years. Inspections may be scheduled with the site owner in advance or they may be unannounced. The first step undertaken by the inspector is to conduct background research of the facility: its permit, records of previous inspections and history of violations, and other factors which may indicate the existence of a problem. An inspection plan of specific issues that need to be addressed during the visit is prepared. Once at the facility, the inspector identifies him/herself and states the intention of the visit at an opening conference with the site owner or the owner’s representative. Refused entry is noted in the inspection report, and further steps are taken by the inspector to obtain access. The first review that takes place is a records check. The permit information on file at the facility is verified, records are examined to assure completeness, and the recordkeeping process is reviewed to determine that all significant information is being tracked correctly. The remaining steps involve the analysis of the facility’s wastewater monitoring equipment. The sampling procedures for pollution discharge detection are examined, as well as sampling and lab analysis equipment. If the inspector feels that an independent sample analysis is necessary, then a sample can be taken. A review is also done of flow measurement equipment and procedures. After completing the inspection, the inspector conducts a closing conference to complete the visit. A report outlining the findings and recommending any further enforcement action which may need to be taken is prepared. The inspection report is reviewed by senior inspectors and the inspecting supervisor, revised if necessary, and then sent to the Office Director, who makes the final decision on follow-up activities. 2-36 ------- ICF INCORPORATED PERMIT COMPLIANCE INSPECTION PROCESS State 3 On-site inspections in State 3 for the NPDES permit program are regular enforcement actMties of the program, but a number of inspections also take place in response to complaints from citizens, employees, or local officials. They are generally unannounced. The first action is a background check of the facility. The inspector examines the permit that is currently held, any changes in its status, records of previous inspections and history of violations, and any other factors which may indicate the existence of a problem. Using this information, an inspection plan is prepared detailing the specific issues that need to be addressed. Once at the facility, the inspector informs the owner of the site the intention of the visit. If an inspection was instigated by a complaint, the source is not revealed. Refused entry is noted in the inspection report, and further steps are taken by the inspector to obtain access. The first examination that takes place is a records review. The permit information on file at the facility is verified, records are examined to assure completeness, and the recordkeeping process is reviewed to determine that all significant information is being tracked correctly. Other steps involve analyzing wastewater monitoring equipment at the site. Sampling procedures for pollution discharge detection are examined, as well as sampling and lab analysis equipment. Samples of effluent are taken as needed, and a similar process is undergone for flow measurement equipment and procedures. After completing the inspection, the inspector prepares a report outlining the findings and recommending any further enforcement action which may need to be taken. The inspection report is reviewed by the inspector’s supervisor and then sent to the office. 2-37 ------- ICF INCORPORATED PERMIT COMPLIANCE INSPECTION PROCESS State 4 Due to staffing limitations and an overwhelming caseload, State 4 conducts most on-site NPDES permit inspections in response to complaints. These inspections are unannounced. Upon receMng a complaint file, the inspector reviews the background of the facility to determine possible causes of the problem and obtain a better understanding of the regulatory history of the site. Using this information, the inspector prepares an inspection plan of specific issues that need to be addressed during the visit. Once at the facility, the inspector identifies him/herself and states the intention of the visit. Refused entry is noted in the inspection report, and further steps are taken by the inspector to obtain access. The first review that takes place is a records verification and analysis. The permit information on file at the facility is verified, records are examined to assure completeness, and the recordkeeping process is reviewed to determine that all significant information is being tracked correctly. The remaining steps involve the analysis of the facility’s wastewater monitoring equipment. The sampling procedures for pollution discharge detection are examined, as well as sampling and lab analysis equipment. If the inspector feels that an independent sample analysis is necessary, then a sample of effluent is be taken. Finally flow measurement equipment and procedures are examined. After completing the inspection, the inspector prepares a report outlining the findings and recommending any further enforcement action which may need to be taken. The inspection report is reviewed by the inspector’s supervisor, revised if necessary, and then sent to the Office Director, who is responsible for the final decision on follow-up activities. 2-38 ------- PERMIT COMPLIANCE INSPECTION PROCESS 0 C ------- PERMIT COMPLIANCE INSPECTION PROCESS 0 ------- PERMIT COMPLIANCE INSPECTION PROCESS Inspector ‘researches faclllt background and prepares Inspection plan inspector verifies permit informai through records examination Inspector analyzes flow measurement techniques Inspector (analyzes effluent sampling procedures N) C -V C ------- ICF INCOR PORATED • Wally realizes, from his conversation with the heads of the state inspection divisions, that although the inspection processes are generally similar, they differ in the frequency with which various steps in the process are conducted. • To obtain a better understanding of the nature and extent of these differences, he is able to obtain from the four state agencies data from the reports prepared by the states’ inspectors after each site visit. • To highlight the differences these data represent, he employs a bar chart. • After a brief presentation on constructing bar charts and histograms, we will divide into breakout groups to construct a bar chart: Frequency of conducting major steps in the compliance inspection process. 2-42 ------- ICE INCORPORATED TYPES OF PERMiT INSPECTIONS CONDUCTED IN 1988 STATE 1 itdu 2 Reeords Thcwthendnn 8 Effluent Sampling 10 ASyZA flaw Menent tedunques f’tèta. \ > TDDM PERMW INSPRCNONS 2 1 \ STATE 2 hng Reeatds Ewnlnation 12 JFSeia ampl *g Measurement atber 3 tOTAL IRMIT 1NSPECUO1 3 STATE 4 Reecds Pxamlnation U 2 Ana1yze Flow Measurement Tcdtmques 1 ‘Other 2 TOTAL IERMEE ThSPECTIOtC 18 4+ STATE 3 2-43 ------- 2.4 Histograms/Bar Charts ------- ICF INCORPORATED TALKING POINTS FOR HISTOGRAMS AND BAR CHARTS 1. WHAT IS A HISTOGRAM OR BAR CHART? • A type of chart that shows how frequently different events, factors, or values occur. 2. WHAT CAN A HISTOGRAM OR BAR CHART DO FOR YOU? • Shows how frequently one event or measurement occurs compared to others. • Shows a range of events or measurements. • Provides information that a single statistic such as an average may hide. 3. WHEN DO YOU USE HISTOGRAMS OR BAR CHARTS IN PROBLEM SOLVING? • To summarize data obtained from a number of incidents or events. • To identify variation in process that single measures, such as averages, may hide. 4. WHAT TYPE OF DATA IS APPROPRIATE FOR HISTOGRAMS OR BAR CHARTS? • A histogram is used with numerical data that can be grouped into intervals. • A bar chart is used with data that can be represented as categories. 2-44 ------- ICF I CORPOR\TED 5. HOW DO YOU CONSTRUCT A HISTOGRAM OR BAR CHART? • Select a variable. • Collect data: obtain existing data or set up an experiment. • Establish intervals or categories, then go through data and group incidents by interval or category. • Count number of incidents or cases in each category or interval (Counts can be converted into percents if desired). • Place the categories or intervals on the horizontal axis and the number of incidents (or percents) on the vertical axis. 2-45 ------- STDs SACE F1EQUIRED GLNPO STAFF INPUT By Workplan Objective SEDS EPA NAT OFF COORD EFF OUTR INDs other S&M ENV NUTR CONGR REQs EEO GLWQA INTN STU GLNPO Workplan Objectives SRS RPS EPS 6 5 4 (1) 1 . 04) > 3 0 2 I 0 C . ’ ., C -3 ------- STATUS/SCHEDULE FOR SUBMISSION GREAT LAKES AOC REMEDIAL ACTION PLANS (Stage 1) Anticipated Actual 0 P;.) Ohio - .4 Illinois Minnesota Wisconsin Michigan 20 18 16 14 12 10 8 6 4 2 V a) E -o U) (1) 0 0 ci -o E z Indiana New York ------- ICF INCORPORATED PCB CONCENTRATIONS IN COHO SALMON IN THE GREAT LAKES IN 1986 LAKE PCB CONCENTRATIONS ERIE 0.48 HURON 0.72 MICHIGAN 0.58 ONTARIo 2.21 SUPERIOR < 0.10 * SALMON TISSUE SAMPLES WERE TAKEN FROM MULTIPLE SITES IN EACH LAKE -- CONCENTRATIONS ARE MEANS OF ALL SAMPLES. 2 -48 ------- PCB CONCENTRATIONS IN COHO SALMON IN THE GREAT LAKES IN 1986 — C) C C ------- PCB CONCENTRATIONS IN COHO SALMON IN THE GREAT LAKES IN 1986 0 . 0. (1) C 0 4.4 C a) () 0 0 m 0 0 2.25 2.00 1.75 1.50 1.25 1.00 0.75 0.50 0.25 0 0.58 Erie Huron Michigan Ontario Superior 3.00 2.75 2.50 0.1 / A C ) C C ------- ICF INCORPORATED MEANWHILE, BACK AT LAKE HEMOS ** * * * * *** ******** ***** ************* **** * 2-51 ------- ICF INCOR PORATED TYPES OP PERMIT INSPECTIONS CONDUCTED IN 1988 STATE 1 STATE 2 Analyze Sarnpihg Analyze Sampling Procedures 2 Procedures 3 Records ExaminatIon S Records nSnatiou 12 Ethent Sana Th*g ‘10 Effluent Sampl i ng 8 Analyze Plow Measurement Analyze Flow Measurement techniques ‘ 4 Techniques 3 Other ’ ‘ .3 Other’ ‘ 3 TOtAL ‘FERMIt TOTAL PERMIT INSPECTIONS 27 INSPECITOIC + 29 STATE 3 STATE 4 na1ywkuqdhg ‘ — Sampling “ Procedures ‘ i ‘P rocedures 2 ‘Retort E tmin4oI 9 Records EzatninMion ii ’ Ewnt Sampling 2 Effluent Sampling 2 Analyze Plow Measurement Analyze Flow Measurement’ techniques 5 Techniques I Otir ‘ Other. 2 TOtAI PERMIT TOTAL PSRMIT INSPEC TIONS , 20 INSPECTIONS 1 5 Using the data presented in the table above, construct a bar chart for each State on the types of permit inspections and a bar chart that compares the differences between States. Blank shells have been provided for your use. 2-52 ------- FREQUENCY OF PERMIT INSPECTION ACTIVITIES State 1 (1988) — C F.,, - C ------- FREQUENCY OF PERMIT INSPECTION ACTIVITIES State 2 (1988) — r’) - 0 > ------- FREQUENCY OF PERMIT INSPECTION ACTIVITIES State 3 (1988) -v 0 U ’ ------- FREQUENCY OF PERMIT INSPECTION ACTIVITIES State 4 (1988) 2 C I. ) - C C I I ------- STATE COMPARISON OF PERMIT INSPECTIONS (1988) 2 C F’., ,JI C tTl ------- FREQUENCY OF PERMIT INSPECTION ACTIVITIES State 1 (1988) Analyze Sampling Procedures I Effluent Sampling 20 18 16 a, 0 C a, 1 . 0 () 0 0 >‘ 0 a, ci IL 14 12 10 8 6 4 2 0 2 Li N ) L u Record Examination Analyze Flow Other Measurement Techniques C Inspection Steps ------- FREQUENCY OF PERMIT INSPECTION ACTIVITIES State 2 (1988) Analyze Sampling Procedures Record Effluent Sampling Analyze Flow Measurement Examination Techniques 20 18 16 14 12 10 8 6 4 2 a) C.) C w C.) C.) 0 0 >‘ C.) C C) 0 ci U- 12 0 Other C C Inspection Steps ------- FREQUENCY OF PERMIT INSPECTION ACTIVITIES State 3 (1988) Analyze Sampling Procedures Effluent Sampling Examination Measurement Techniques a, U C I- U C-) 0 0 > C) C a, 0 ci U- 20 18 16 14 12 10 8 6 4 2 0 9 0 ’ 0 2 Record Analyze Flow Other C Inspection Steps ------- FREQUENCY OF PERM!T INSPECTION ACTIVITIES State 4 (1988) Procedures Examination 20 18 16 w C) C ci) I- 0 C-) 0 0 >% C) w a- c i) I- LL 14 12 10 8 6 4 2 0 F ) Analyze Sampling Record Effluent Sampling 1 2 Analyze Flow Measurement Techniques Other 2 C C Inspection Steps ------- STATE COMPARISON OF PERMIT INSPECTIONS (1988) Procedures Examination o r ’) Effluent Sampling Analyze Flow Measurement Techniques a a L 20 18 16 14 12 10 8 6 4 G) C-) C U) 1 I— 0 C-) 0 0 > 0 C w a- U) I- U- State 1 State 2 State 3 State 4 Analyze Sampling 333 Records 1 21 Other .!, 0 0 -1 T1 Inspection Steps ------- ICF INCOR PORATED • Having now focused his efforts more clearly, Wally undertakes a project to improve the inspection process in the four states, focusing especially on the two that were doing little effluent sampling. • Wally convenes a project team to conduct a project planning process. • After a brief presentation on project planning, we will divide into breakout groups to complete a plan for an improvement project to improve the compliance inspection process. 2-63 ------- 2.5 Systematic Project Managetheñt ------- ICF INCORPORATED TALKING POINTS FOR PROJECT MANAGEMENT WHAT IS SYSTEMATIC PROJECT MANAGEMENV? • Systematic project management is a planning process used to apply the tools and techniques of continuous improvement to actual projects managed by GLNPO staff. • It lays out a series of steps to develop the tasks, schedule, assignments, and budget for a quality improvement project. 2. WHAT CAN SYSTEMATIC PROJECT MANAGEMENT DO FOR YOU? • At the start of a new assignment, it can help identify the concrete steps needed to begin and complete a project. • Systematic project management can help project the time and resources needed to complete a project. - A project plan summary can be used to brief management on a project’s costs, schedules, and expected deliverables or outputs. 3. WHEN DO YOU USE SYSTEMATIC PROJECT MANAGEMENT IN PROBLEM SOLVING? • When you are at the stage in solving a problem that you have identified a specific set of questions that should be addressed or tasks that should be performed. • When the questions you seek to address or tasks that must be performed will require a significant outlay of funds, staff, time, or management attention. • When important work has been stalled, needs greater focus, or requires an infusion of attention and energy. 2-64 ------- ICF INCORI’OR TED 4. WHAT TYPES OF INFORMATION IS NEEDED FOR SYSTEMATIC PROJECT MANAGEMENT? A description of the problem being addressed and the tasks we anticipate undertaking. - Where possible, the focus of our project has been identified through the use of the techniques of statistical process control: an analysis of the way we presently perform work. • Information on staff and contractor availability and costs, equipment and other than personnel costs, and management review procedures. 5. HOW DO YOU CONSTRUCT A PROJECT MANAGEMENT PLAN? • Complete the following worksheets: 1. Problem parameters : Typically, the result of a brainstorming session designed to identify the elements of a problem (e.g., through the development of a Rfishboneh diagram by a project team). 2. Problem definition : On which aspect of the problem (identified through brainstorming) will the quality improvement project seek to focus? What is the performance we are seeking to improve? Typically, this is based upon an analysis of variation in certain levels of performance, and selection of specific aspects of work targeted for improvement. 3. Assignment Summary : What is the assignment’s objective(s)? Who is the customer? Who will complete the assignment? When is it due? What is the expected output? 4. Task Breakdown : What are the specific, discrete tasks that must be performed to complete the project? 5. Schedule Planning : What are the start and end dates for each task and subtask? What are the key outputs or deliverables associated with each task? 6. Deliverable Summary : What are the steps and interim products necessary to produce each deliverable? 2-65 ------- ICF INCORPORATED 7. RevIew SteDs for Each Product : Who is likely to be affected by this output? Who is the customer? Who has an interest in the development of this project? Who are the required reviewers, both internal to GLNPO and external? 8. Resource Estimation by Task : How much staff time, contractor time, and other expenditures will be required to complete each task? 9. Resource Estimation by Project : A compilation of task estimates. 2-66 ------- ICF INCORPORATED I. PROBLEM PARAMETERS 1. What is the general problem that has been identified? 2. Without judging their importance or even correctness, list all the factors that your group identified as causes of the problem: B. D. E. F. 0. H. I. J. K. L M. N. 0. P. 0. R. S. T. 2-67 ------- ICF INCORPORATED I. PROBLEM PARAMEFERS (continued) 3. Of the problem elements listed in worksheet I, use data, or best professional judgement (vote by secret ballot if necessary) to identify the five most important factors contributing to the problem: A B. C. D. E. 2-68 ------- ICF INCOR PORATED H. PROBLEM DEFINITION 1. Briefly describe the process by which you identified the best target for improvement. 2. What is the specific performance we will be tiying to improve? 3. What is the definition of success for this project? 4. Who are the customers for this improvement? External: Internal: 2-69 ------- ICF INCORPORATED I lL ASSIGNMENT SUMMARY 1. Assignment Title: 2. Assignment Description: 3. Assignment Objective(s): 4. Audience for this Project: 5. Date Assigned: 6. Date Due: 7. Final Product Due: 8. Contractor Assigned: 9. EPA Staff Assigned: 2-70 ------- ICF INCORPORATED IV. TASK BREAKDOWN Tasks: Task 1.0: Task 2 O: Task 3.0: Task 4.0: Task 5.0: 2-7]. ------- ICF INCOR PORATED V. SCHEDULE PLANNING Start Date End Date Task 1.0: Subtask 1.1: _____________________________ __________ __________ Subtask 1.2: ___________________________ Subtask 1.3: _____________________________ Subtask 1.4: _______________________________ Subtask 1.5: _____________________________ Key Deliverables for This Task: 1. _________________________________________ ______________ ______________ 2. ___________________________________ ____________ ____________ 3. ___________________________________ ____________ ____________ 2-72 ------- ICF INCORPORATED V. SCHEDULE PLANNING Start Date End Date Task 2.0: ___________________________________ __________ __________ Subtask 2.1: ____________________________ __________ __________ Subtask 2.2: ___________________________ Subtask 2.3: _____________________________ Subtask 2.4: _____________________________ Subtask 2.5: _____________________________ Key Deliverables for This Task: 1. _________________________________________ _______________ ______________ 2. ______________________________ ___________ __________ 3. _________________________________ ____________ ____________ 2-73 ------- IC r INCORPORATED V. SCHEDULE PLANNING Start Date End Date Task 3.0: ___________________________________ __________ __________ Subtask 3.1: _____________________________ __________ __________ Subtask 3.2: _____________________________ Subtask 3.3: _____________________________ Subtask 3.4: _____________________________ Subtask 3.5: _____________________________ Key Deliverables for This Task: 1. _______________________________________ 2. _______________________________ ___________ ___________ 3. _______________________________ ___________ ___________ 2-74 ------- ICF INCORPORATED V. SCHEDULE PLANNING Start Date End Date Task 4.0: ___________________________________ __________ __________ Subtask 4.1: _____________________________ __________ __________ Subtask 4.2: _____________________________ Subtask 4.3: _____________________________ Subtask 4.4: ___________________________ Subtask 4.5: _____________________________ Key Deliverables for This Task: 1. _________________________________________ _______________ ______________ 2. _________________________________ ____________ ____________ 3. ___________________________________ ____________ ____________ 2-75 ------- ICF INCORPORATED V. SCHEDULE PLANNING Start Date End Date Task 5.0: ___________________________________ __________ __________ Subtask 5.1: ____________________________ __________ __________ Subtask 5.2: ____________________________ Subtask 5.3: _____________________________ Subtask 5.4: _____________________________ Subtask 5.5: _____________________________ Key Deliverables for This Task: 1. ____________________________________________ _______________ ________________ 2. _________________________________ ___________ ____________ 3. _________________________________ ___________ ____________ 2-76 ------- ICF INCORPORATED VI. DELIVERABLE SUMMARY Key Deliverable Interim Products/Steps Due Dates a) b) C) d) 2. a) b) c) d) 3. a) b) C) d) 2-77 ------- ICF INCOR PORATED VI. DELIVERABLE SUMMARY Key Deliverable Interim Products/Steps Due Dates 4. a) b) C) d) 5. a) b) c) d) 6. a) b) C) d) 2-78 ------- ICF INCORPORATED VI. DELIVERABLE SUMMARY Key Deliverable Interim Products/Steps Due Dates 7. a) b) c) d) 8. a) b) c) d) 9. a) b) c) d) 2-79 ------- ICF INCORPORATED VI. DELiVERABLE SUMMARY Key Deliverable Interim Products teps Due Dates 10. a) b) C) d) 11. a) b) c) d) 12. a) b) c) d) 2-80 ------- ICF INCOR PORATED VI. DELIVERABLE SUMMARY Key Deliverable Interim Products/Steps Due Dates 13. a) b) C) d) 14. a) b) C) d) 15. a) b) C) d) 2-81 ------- ICF INCORPORATED VII. REVIEW STEPS FOR EACH PRODUCT Product: Reviewers: Within GLNPO 1. 2. 3. Within the Office of the National Program Manager 1. 2. 3. Within Headquarters 1. 2. 3. Within EPA Regions (Identify Regions): 1. 2. 3. External (e.g., 0MB, White House, Canadian Federal Government, Canadian Provincial Government, States, I.J.C., Congress, news media, public, interest groups) 1. 2. 3. 2-82 ------- ICF lr CORPORATED VII. REVIEW STEPS FOR EACH PRODUCT Product: Reviewers: Within GLNPO 1. 2. 3. Within the Office of the National Program Manager 1. 2. 3. Within Headquarters 1. 2. 3. Within EPA ReQlons (Identify Regions): 1. 2. 3. External (e.g., 0MB, White House, Canadian Federal Government, Canadian Provincial Government, States, LJ.C., Congress, news media, public, interest groups) 1. 2. 3 2-83 ------- ICF INCORPORATED V I I. REVIEW STEPS FOR EACH PRODUCT Product: Reviewers: Within GLNPO 1. 2. 3. Within the Office of the National Program Manager 1. 2. 3. Within Headquarters 1. 2. 3. Within EPA Regions (Identify Regions): 1. 2. 3. External (e.g., 0MB, White House, Canadian Federal Government, Canadian Provincial Government, States. I.J.C., Congress, news media, public, InteresL groups) 1. 2. 3. 2-84 ------- ICF INCORPORATED VII. REVIEW STEPS FOR EACH PRODUCF Product: Reviewers: Within GLNPO 1. 2. 3. Within the Office of the National Program Manager 1. 2. 3. Within Headquarters 1. 2. 3. Within EPA Regions (Identify Regions): 1. 2. 3. External (e.g., 0MB, White House, Canadian Federal Government, Canadian Provincial Government, States, l.J.C., Congress, news media, public, interest groups) 1. 2 2-85 ------- ICF INCORPORATED VII. REVIEW si rS FOR EACH PRODUCT Product: ___________________________________ Reviewers: Within GLNPO 1. 2. 3. Within the Office of the National ProRram Mana2er 1. 2. 3. Within Headquarters 1. 2. 3. Within EPA Regions (Identify Regions): 1. 2. 3 External (e.g., 0MB, White House, Canadian Federal Government, Canadian Provincial Government, States, I.J.C., Congress, news media, public, interest groups) 1. 2. 3. 2-86 ------- ICE INCOR PORATED VU. REVIEW STEPS FOR EACH PRODUCT Product: Reviewers: Within GLNPO 1. 2. 3. Within the Office of the National Program Manager 1. 2. 3. Within Headquarters 1. 2. 3. Within EPA Regions (Identify Regions): 1. 2. 3. External (e.g., 0MB, White House, Canadian Federal Government, Canadian Provincial Government, States, LJ.C., Congress, news media, public, interest groups) 1. 2. 3. 2-87 ------- ICF INCORPORATED VII. REVIEW STEPS FOR EACH PRODUCT Product: Reviewers: Within GLNPO 1. 2. 3. Within the Office of the National Program Manager 1. 2. 3. Within Headquarters 1. 2. 3. Within EPA Regions (Identify Regions): 1. 2. 3. External (e.g., 0MB, White House, Canadian Federal Government, Canadian Provincial Government, States, I.J.C., Congress, news media, public, interest groups) 1. 2. 3. 2-88 ------- ICF INCORPORATED VII. REVIEW STEPS FOR EACH PRODUCI’ Product: Reviewers: Within GLNPO 1. 2. 3. Within the Office of the National Program Manager 1. 2. 3. Within Headquarters 1. 2. 3. Within EPA Regions (Identify Regions): 1. 2. 3. External (e.g., 0MB, White House, Canadian Federal Government, Canadian Provincial Government, States, 1.J.C., Congress, news media, public, interest groups) 1. 2 3. 2-89 ------- ICF INCORPORATED V I I. REVIEW STEPS FOR EACH PRODUCT Product: Reviewers: Within GLNPO 1. 2. 3. Within the Office of the National Proaram Manager 1. 2. 3. Within Headquarters 1. 2. 3. Within EPA Regions (Identify Regions): 1. 2. 3. External (e.g, 0MB, White House, Canadian Federal Government, Canadian Provincial Government, States. I.J.C., Congress, news media, public, interest groups) 1. 2. 3. 2-90 ------- ICF INCOR PORATED VII. REVIEW STEPS FOR EACH PRODUCE Product: Reviewers: Within GLNPO 1. 2. 3. Within the Office of the National Program Manager 1. 2. 3. Within Headquarters 1. 2. 3. Within EPA Regions (Identify Regions): 1. 2. 3. External (e.g., 0MB, White House, Canadian Federal Government, Canadian Provincial Government, States, I.J.C., Congress, news media, public, interest groups) 1. 2. 3. 2-9]. ------- ICF INCOR PORATED VII. REVIEW STEPS FOR EACH PRODUCT Product: Reviewers: Within GLNPO 1. 2. 3. Within the Office of the National Program Manager 1. 2. 3. Within Headquarters 1. 2. 3. Within EPA Regions (Identify Regions): 1. 2. 3. External (e.g., 0MB, White House, Canadian Federal Government, Canadian Provincial Government, States, I.J.C., Congress, news media, public, interest groups) 1. 2. 3. 2-92 ------- ICF INCOR PORATED VII. REVIEW smrS FOR EACH PRODUCT Product: Reviewers: Within GLNPO 1. 2. 3. Within the Office of the National Program Manager 1. 2. 3. Within Headuuarters 1. 2. 3. Within EPA Regions (Identify Regions): 1. 2. 3. External (e.g., 0MB, White House, Canadian Federal Government, Canadian Provincial Government, States, LJ.C., Congress, news media, public, interest groups) 1. 2. 3. 2-93 ------- ICF INCORPORATED VU. REVIEW STEPS FOR EACH PRODUCT Product: Reviewers: Within GLNPO 1. 2. 3. Within the Office of the National Program Manager 1. 2. 3. Within Headquarters 1. 2. 3. Within EPA Regions (Identify Regions): 1. 2. 3. External (e.g., 0MB, White House, Canadian Federal Government, Canadian Provincial Government, States, LJ.C., Congress, news media, public, interest groups) 1. 2. 3. 2-94 ------- ICF II\CORPORATED V I I. REVIEW STEPS FOR EACH PRODUCE Product: Reviewers: Within GLNPO 1. 2. 3. Within the Office of the National Program Mana2er 1. 2. 3. Within Headquarters 1. 2. 3. Within EPA ReQions (Identify Regions): 1. 2. 3. External (e.g., 0MB, White House, Canadian Federal Government, Canadian Provincial Government, States, I.J.C., Congress, news media, public, interest groups) 1. 2 3. 2-95 ------- ICF 1\CORPOR\TED VII. REVIEW STEPS FOR EACH PRODUCI Product; Reviewers: Within GLNPO 1. 2. 3. Within the Office of the National Program Manager 1. 2. 3. Within Headquarters 1. 2. 3. Within EPA Regions (Identify Regions): 1. 2. 3. External (e.g., 0MB, White House, Canadian Federal Government, Canadian Provincial Government, States, I.J.C., Congress, news media, public, interest groups) 1. 2. 3. 2-96 ------- VIII. RESOURCE ESTIMATION: TASKS Elapsed time is the number of days, weeks, or months between the start and end dates. In estimating work days (either EPA or contractor stall) it may help to think of percentages of time multiplied by the number of days of work on this task. Also factor in the roles and time needed of different EPA staff. “Other Costs” refers to any travel, printing, or equipment costs. Refer to the Estimated Price List (included in Reference Materials) for estimates of Other Costs. Task 1.0: Product(s) Elapsed EPA Staff Contractor Other Subtask (if any) Time Work Days Work Days Costs 1.1 12 13 1.4 Totals for Task 1.0 Staff Work Days: Contractor Work Days: Othcr Costs: r ) - 0 C ------- VIII. RESOURCE ESTIMATION: TASKS (Continued) Elapsed timc is the number of days, weeks, or months between the start and end dates. In estimating work days (either EPA or coniractor staff) it may help to think of percentages of time multiplied by the number of days of work on this task. Also factor in the roles and timc needed of different EPA staff. “Other costs” refers to any travel, printing, or equipment costs. Refer to the Estimated Price List (included in Rcfcrcncc Materials) for estimates of Other Costs. Task 2.0: Product(s) Elapsed EPA Staff Contractor Other Subtask (if any) Time Work Days Work Days Costs 21 22 23 24 Totals for Task 2.0 Staff Work Days: Contractor Work Days: Other Cosis: 1 . ) ‘ .0 — - ------- VIII. RESOURCE ESTIMATION: TASKS (Continued) Elapsed lime is the number of days, weeks, or months between thc start and end dates. In estimating work days (either EPA or contractor stafF) ii may help to think of percentages of time multiplied by the number of days of work on this task. Also factor in the roles and time needed of different EPA staff. Othcr Costs refers to any travel, printing, or equipment costs. Refer to the Estimated Price List (included in Reference Materials) for csiimatcs 01 Other Costs. Task 3.0: Product(s) Elapsed EPA Staff Contractor Other Subtask (if any) Time Work Days Work Days Costs 3.1 32 33 3.4 Totals for Task 3.0 Staff Work Days: Contractor Work Days: Other Costs: i c ------- VIII. RESOURCE ESTIMATION: TASKS (Continued) Elapscd time is the number of days, weeks, or months between the start and end dates. In estimating work days (either E A or contractor stall) it may help to think of percentages of time multiplied by the number of days of work on this task. Also factor in the roles and timc nccdcd of different EPA staff. “Other Costs” refers to any travel, printing, or equipment costs. Refer to the Estimated Price List (included in Reference Materials) for c timatcs of Other Costs. Task 4.0: Product(s) Elapsed EPA Staff Contractor Qther Subtask (if any) Time Work Days Work Days Costs 41 42 43 44 Totals for Task 4.0 Staff Work Days: Contractor Work Days: Other Costs: — Ie I I T ------- VIII. RESOURCE ESTIMATION: TASKS (Continued) Elapsed time is the number of days, weeks, or months between the start and end dates. In estimating work days (either EPA or contractor staff) it may help to think of percentages of time multiplied by the number of days of work on this task. Also factor in the roles and time needed of different EPA staff. MOther Costs refers to any travel, printing, or equipment costs. Refer to the Estimated Price List (included in Reference Materials) for estimates of Other Costs. Task 5.0: Product(s) Elapsed EPA Staff Contractor Other Subtask (if any) Time Work Days Work Days Costs 51 5.2 5.3 54 Totals for Task 5.0 Staff Work Days: Contractor Work Days: Othcr Costs: I— .! - 0 - ------- IX. RESOURCE ESTIMATION: PROJECT LEVEL Project: EPA Staff Contractor Other Task Work Days Work Days Costs 1 0: 2.0: 3.0: 4 0. 50: TOTALS: — : ------- ICF Ir’SCORPORATED I. PROBLEM PARAMETERS What is the general problem that has been identified? A. W7zy is travel funding always tight in the fourth quarter? 2. Without judging their importance or even correctness, list all the factors that your group identified as causes of the problem: A. Inflation in travel costs. B. Emeigency trips. C. Response to management fire drills. D. Poor trac!dng of travel expenditures. E. Inadequate travel budget for the mission of the office. F. Too much unnecessary travel - too many people on the same trips - too many PR nips 3. 0! the problem elements listed in worksheet I, use data, or best professional judgement (vote by secret ballot if necessary) to identify the most important factors contributing to the problem: A Inadequate travel budget. B. Poor tracking of travel expenditures. 2-103 ------- ICF INCORPORATED II . PROBLEM DEFINITION 1. Briefly 4escribe the process by which you identified the best target for improvement. Focus on travel expense tracking since we cannot get more money, but we might be able to plan our spending better by keeping track of expenditures. 2. What is the specific performance we will be trying to improve? Tracking of travel spending. 3. What is the definition of success for this project? A practicaL low-cost method for tracking travel spending. 4. Who are the customers for this improvement? Edema!: The National Flrignzm Manager, EPA’: Budget Office Internal: Carol, the Staff Chiefs 2-104 ------- ICE INCORPORATED III. ASSIGNMENT SUMMARY 1. Assignment Title: Improving tracldng of travel expenditures. 2. Assignment Description: Develop a simple system for tracldng travel fiintL . 3. Assignment Objective(s): 1) Identify process for projecting travel spending identifying travel plans, and recording travel spending. 2) Provide weekly reports on travel expenditures. 4. Date Assigned: 9/15/89 5. Date Due: 10/30/89 6. Final Product Due: 10/30/89 7. Contractor Assigned: Arthur 0. Little 8. EPA Staff Assigned: Riley 2-105 ------- ICF INCORPORATED W. TASK BREAKDOWN Tasks: Task 10: Describe prescnt tracking system Task 2.0: Identify possible improvements Task 3.0: Develop project tracking system Task 4.0: Pilot test system Task 5.0: Train staff Task 6.0: Implement 2-106 ------- ICE INCOR PORATED V. SCHEDULE PLANNING Start Date End Date Task 1.0: Descnbe Present System 9/15 9/26 Subtask 1.1: Flowchan .system 9/15 9/20 Subtask 1.2: Check flowchwt with 9/20 9/22 staff Subtask 1.3: Revise and finalize 9/25 9/26 fiowchwt Subtask 1.4: Develop test descnp- 9/25 9/26 tion of present system Key Deliverables for This Task: 1. Flowchan 9/15 9/24 2. Test description 9/23 9/24 2-107 ------- id INCOR PORATED V. SCHEDULE PLANNING Start Date End Date Task 2.0: Identify Possible Improvements 9/26 10/5 Subtask 2.1: Discuss fiowchwt with 9/26 9/28 staff from offices with better systems Subtask 2.2: Develop list of possible 9/29 10/2 changes Subtask 2.3: Discuss changes with 10/3 10/4 GLNPO staff Subtask 2.4: Finalize unprovenient 10/4 10/5 list Key Deliverables for This Task: 1. List of possible 9/29 10/5 improvements 2-108 ------- ICF INCOR PORATED V. SCHEDULE PLANNING Start Date End Date Task 3.0: Develop Revited System 10/4 10/11 Subtask 3.1: Draft new flow 10/4 10/5 diagram of system Subtask 3.2: Draft new SOPs for 10/5 10/6 tracking penditwes Subtask 3.3: Develop program for 10/6 10/10 GPC-based system Subtaslc 3.4: Test and modijy 10/10 10/11 program Key Deliverables for This Task: 1. Flow diagra n 10/4 10/5 2. New tracking SOPs memo 10/5 10/6 3. New tracldngprogrwn 10/6 10/11 2-109 ------- ICF INCORPORATED V. SCHEDULE PlANNING Start Date End Date Task 4.0: Pilot Test System Subtask4.1: Load ailfy 90* 10/11 10/13 data into new system Subtask 4.2: Produce report for 10113 1 0/13 the 1st 2 weeks of October Subtask 4.3: Provide reports to 10113 10/13 management and obtain comments Subtask 4.4: Revise reports on 10116 10/16 software as needed Key Deliverables for This Task: 1. 1st report tracking 10113 10113 spending * This would be expenditures from 10/1-10113. 2-110 ------- ICF INCORPORATED V. SCHEDULE PLANNING Start Date End Date Task 5.0: Train Staff Subtask 5.1: Develop guide to 10/12 10/13 tracldng system (explaining SOPs) Subtask 5.2: Hold briefings on 10/18 10/19 the new system Key Deliverables for This Task: 1. Guide to using the system 10/12 10/13 2-111 ------- ICF INCORPORATED V. SCHEDULE PLANNING Start Date End Date Task 6.0: Implement system 10/20 Subtask 6.1: Operate system and 10/20 produce weekly repons Subtask 6.2 Circulate repo?ts Weekly Key Deliverables for This Tasic 1. Weekly repons Weekly 2-112 ------- ICF INCOR PORATED VI. DELIVERABLE SUMMARY Key Deliverable Interim Products/Steps Due Dates 1. Flowchart a) Draft flowchart 9/20 b) FsnuJ flowchart 9/24 2. List of possible improvements a) b) Draft list . Final list 10/2 10/15 3. New flowchart a) Draft 10/S and new SOPs b) Final 10/6 4. New tracking a) Pilot test 10/10 software b) Final program 10/11 5. 1st report 10/13 6. Weekly reports Weekly 2-113 ------- ICF INCORPORATED VII. REVIEW STEPS FOR EACH PRODUCT Product: _________ 1-4 Reviewers: Within GLNPO 1. Staff Chiefs 2. Deputy Director 3. Director Within the Office of the National Program Manager 1. 2. 3. Within Headquarters 1. 2. 3. Within EPA Regions (Identify Regions): 1. 2. 3. External (e.g., 0MB, White House, Canadian Federal Government, Canadian Provincial Government, States, IJ.C., Congress, news media, public, interest groups) 1. 2. 3. 2-114 ------- ICF INCORPORATED VII. REVIEW STEPS FOR EACH PRODUCT Product: 1st repost of the sew tracking rystem and su equent repo,ts Reviewers: Within GLNPO 1. Staff Chiefs 2. Deputy Director 3. Director Within the Office of the National Pronram Manager 1. National F -ogirun Manager 2. 3. Within Headquarters 1. Budget Office 2. 3. Within EPA Regions (Identify Regions): 1. 2. 3. External (e.g., 0MB, White House, Canadian Federal Government, Canadian Provincial Government, States, IJ.C., Cons ress, news media, public, interest groups) 1. 2. 3. 2-115 ------- VIII. RESOURCE ESTIMATION: TASKS (Continued) Elapsed time is the number of days, weeks, or months between the start and end dates. In estimating work days (either EPA or contractor staff) ft may help to think of percentages of time multiplied by the number of days of work on this task. Also factor in the roles and time needed of different EPA staff. “Other Costs” refers to any travel, printing, or equipment costs. Refer to the Estimated Price List (included in Reference Materials) for estimates of Other Costs. Task 1.0: Describe Present System Product(s) Elapsed EPA Staff Contractor Other Subtask (if any) Time Work Days Work Days Costs 1.1 Flowchart system Flowchart 6 days 1.0 2.0 1.2 Check with staff 3 days 1.0 .5 1.3 Revise and finalize chart Flowchart 2 days .5 1.0 1.4 Develop text and description Test of system 2 days .1 1.0 $100 - Printing 2.6 4.5 $100 Totals for Task 1.0 Staff Work Days: 2.6 Contractor Work Days: 4.5 le Other Costs: S100 12 Ir iO I c 10 I 1 I> I I ’: ’ ------- VIII. RESOURCE ESTIMATION: TASKS (Continued) Elapsed time is the number of days, weeks, or months between the start and end dates. In estimating work days (either EPA or contractor stall) it may help to think of percentages of time multiplied by the number of days of work on this task. Also factor in the roles and time needed of different EPA staff. “Other Costs” refers to any travel, printing, or equipment costs. Refer to the Estimated Price List (included in Reference Materials) for estimates of Other Costs. Task 2.0: Identify Possible Improvements Product(s) Elapsed EPA Staff Contractor Other Subtask (if any) Time Work Days Work Days Costs 2.1 Discuss flow externally 3 days 1.5 3.0 2.2 List of changes-draft Improvement list 4 days .5 1.5 2.3 Discuss internally 2 days 1.0 1.0 2.4 Finalize improvement list Improvement final 2 days 1.0 1.0 4.0 6.5 Totals for Task 2.0 Staff Work Days: 4.0 Contractor Work Days: 6.5 Other Costs: 0 0 0 ------- VIII. RESOURCE ESTIMATION: TASKS (Continued) Elapsed time is the number of days, weeks, or months between the start and end dates. In estimating work days (either EPA or contractor staff) it may help to think of percentages of time multiplied by the number of days of work on this task. Also factor in the roles and time needed of different EPA staff. “Other Costs” refers to any travel, printing, or equipment costs. Refer to the Estimated Price List (included in Reference Materials) for estimates of Other Costs. Task 3.0: Develop Reused Expenditure Tracking System Product(s) Elapsed EPA Staff Contractor Other Subtask (if any) Time Work Days Work Days Costs 3.1 Draft new flow Flow dia am 2 dayt .5 LO $100 - Printing 3.2 Draft new SOPs Memo 2 days .5 1.5 3.3 Develop software M J’ P7vg7wn 3 days .2 3.0 $500 - Computer cost 3.4 Test and modify 2 days .2 3.0 1.4 8.5 $600 Totals for Task 3.0 Staff Work Days: 1.4 Contractor Work Days: 8.5 Other Costs: S600 C C > ------- VIII. RESOURCE ESTIMATION: TASKS (Continued) Elapsed time is the number of days, weeks, or months between the start and end dates. In estimating work days (either EPA or contractor staff) it may help to think of percentages of time multiplied by the number of days of work on this task. Also factor in the roles and time needed of different EPA staff. Other Costsu refers to any travel, printing, or equipment costs. Refer to the Estimated Price List (included in Reference Materials) for estimates of Other Costs. Task 4.0: Pilot Test System Product(s) Elapsed EPA Staff Contractor Other Subtask (if any) Time Work Days Work Days Costs 4.1 Load data 3 days .5 .5 4.2 Produce 1st report Report I day .0 .2 4.3 Circulate reportc for comment 1 day .5 .1 4.4 Revise report software Software revising I day .2 .5 $500 - computer cost L2 L3 $500 Totals for Task 4.0 Staff Work Days: 1.2 Contractor Work Days: 1.3 Other Costs: $500 C C ‘0 - ------- VIII. RESOURCE ESTIMATION: TASKS (Continued) Elapsed time is the number of days, weeks, or months between the start and end dates. In estimating work days (either EPA or contractor staff) it may help to think of percentages of time multiplied by the number of days of work on this task. Also factor in the roles and time needed of different EPA staff. “Other Costs” refers to any travel, printing, or equipment costs. Refer to the Estimated Price List (included in Reference Materials) for estimates of Other Costs. Task 5.0: Train Staff Product(s) Elapsed EPA Staff Contractor Other Subtask (if any) Time Work Days Work Days Costs 5.1 Develop guide to Training 2 dayr .2 .8 100 - system guide Printing 5.2 Hold briefings Briefings 2 days 2.0 3.0 $100 - Viewgraphs 2.2 3.8 $300 Totals for Task 5.0 Staff Work Days: 2.2 Contractor Work Days: 3.8 Other Costs: $300 0 0 ------- VIII. RESOURCE ESTIMATION: TASKS (Continued) Elapsed time is the number of days, weeks, or months between the start and end dates. In estimating work days (either EPA or contractor staff) it may help to think of percentages of time multiplied by the number of days of work on this task. Also factor in the roles and time needed of different EPA staff. “Other Costs” refers to any travel, printing, or equipment costs. Refer to the Estimated Price List (included in Reference Materials) for estimates of Other Costs. Task 60 Implement System Product(s) Elapsed EPA Staff Contractor Other Subtask (if any) Time Work Days Work Days Costs 6.1 Produce weekly Reports 52 weeks .1 per week .5 per week $30 per reports week 5.2 per year 26 per year $1560 per year (Punting) 6.2 Circulate reports .05 per week 2.6 per year 7.8 26 $1,560 Totals for Task 6.0 Staff Work Days: 7.8 Contractor Work Days: 26 Other Costs: $1,560 — C -v I -I ------- ICF INCORPORATED IX. RESOURCE ESTIMATION: PROJECT LEVEL Project: Travel Expenditure Tracldng System EPA Staff Contractor Other Work Days Work Days Costs 1.0: Describe present system 2.6 4.5 $ 100 2.0: Identify improvements 4.0 6.5 0 3.0: Develop revised system 1.4 &5 600 4.0: Pilot test 1.2 1.3 500 5.0: Train staff 2.2 3.8 300 6.0: Implement (one year) 1.560 TOTALS: 19.2 50.6 $3,060 2-122 ------- ICE INCOR PORATED * * * * * * ****** *** ****** ********* ******* * * *** * * * MEANWHILE, BACK AT LAKE HEMOS 2-123 ------- ICE’ INCORPORATED INSTRUCTIONS FOR WALLY PIKE’S SYSTEMATIZED PROJECT MANAGEMENT EXERCISE Wally and his project team have already begun the process of planning the pruject to improve permit compliance inspections. They have utilized fishbones, paretos, flowcharts and bar charts and focused on the specific performance to be improved. They have already: 1) Identified the problem parameters 2) Defined the problem 3) Summarized the assignment 4) Completed a task breakdown 5) Scheduled the project 6) Summarized the deliverables Each group should now complete the remaining worksheets: 7) Review steps (already partially filled Out) 8) Resource estimates: tasks (partially filled out) 9) Resource estimates: Project 2-124 ------- ICF INCORPORATED I. PROBLEM PARAMETERS What is the general problem that has been identified? A. Compliance inspections do not always detect violators. 2. Without judging their importance or even correctness, list all the factors that your group identified as causes of the problem: A. Insufficient inspection staff B. Corporations hiding violations. C. Inadequate procedures for checldng records. 1). inadequate effluent monitoring equipment. E. Inconsistent effluent monitonng procedures. F. No effluent monitoring. 3. Of the problem elements listed in worksheet I, use data, or best professional judgement (vote by secret ballot if necessary) to identify the most important factors contributing to the problem: A. Inconsistent effluent monitoring procedures. B No effluent monitoring. 2-125 ------- ICF INCORPORATED II. PROBLEM DEFINITION 1. Briefly describe the process by which you identified the best target for improvement. Fzshbone on fishkills; Pareto on significant causes; flowchart of permit compliance inspection process; bar chart of rime involved in each step of the inspection process. 2. What is the specific performance we will be trying to improve? Detecting permit violations. 3. What is the definition of success for this project? Fewer undetected permit violations. Fishkill reductions that approach projections. 4. Who are the customers for this improvement? External: Congress, the Presidcnr, the media, the Administrator, A.A. Water, The National Program Manager, the public, EPA ’s Budget Office Internal: Carol, the Staff Chiefs 2-126 ------- ICF INCORPORATED Ill. ASSIGNMENT SUMMARY 1 Assignment Tit1e: Improving Pennit Compliance Inspection. 2. Assignment Description: Developing a handbook and training workshop for permit compliance inspectors. 3. Assignment Objective(s): 1) Reducing undetected violations; and 2) Increasing effluent monuonng. 4. Date Assigned: 9/15/89 5. Date Due: 12/13/89 6. Contractor Assigned: Indiana University 7. EPA Staff Assigned: Wally Pike (leader) and Project Team 2- 127 ------- ICF INCORPOR ITED IV. TASK BREAKDOWN Tasks: Task 1.0: Convene workgroup of state inspectors. Task 2.0: Develop inspection procedures handbook Task 3.0: Develop workshop to train inspectors, Task 4.0: Pilot test workshop on new procedures. Task 5.0: Implement new procedures. 2-128 ------- ICF INCORPORATED V. SCHEDULE PLANNING Start Date End Date Task 1.0: Convene Workgroup Subtask 1.1: Identify potential participants 9115 9118 Subtask 1.2: Obtain clearance from management 9118 9/20 Subtask 1.3: Secure travel ftcnds and arrange travel 9118 9122 Subtask 1.4: Develop reference package for workgroup sessions 9115 9/23 Subtask 1.5: Hold meetings 10/2 10/20 Key Deliverables for This Task: 1. List of Participants 9/18 2. Memos Inviting Participants 9/22 3. Reference Package 9/23 2-129 ------- ICF INCORPORATED V. SCHEDULE PLANNING Start Date End Date Task 2 (F Develop Inspecriei Handbook Subtask 2.1: Review workgroup notes 10/2 10/27 Subtask 2.2: Develop handi)ook outline and abstract 10/25 10/30 Subtask 2.3: Draft Handbook 10/31 11/10 Subtask 2.4: Circulate for comments and revise 11/10 11/27 Subtask 2.5: Finalize, print Key Deliverables for This Task: 1. Outline and Abstract 10/30 2. Draft Handbook 11/1 0 3. Final Handbook 12/15 2-130 ------- ICF INCOR PORATED V. SCHEDULE PL4NNING Stan Date End Date Task 3.0: Develop Inspector Workshop 10/25 11114 Subtask 3.1: Develop Course Outline 10/25 10126 Subtask 3.2: Develop Course Materials 10/26 11/3 Subtask 3.3: Review and revise course mazenals 11/3 11110 Subtask 3.4: Workshop Logistics 1 0/25 11/3 Subtask 3.5: Conduct workshop 11113 11/19 Key Deliverables for This Task: 1. Course outline 1 0/26 2. Course workbook 11/1 0 2-131 ------- ICF INCORPORATED V. SCHEDULE PLANNING Start Date End Date Task 4.0: Pilot Test New Procedures 10/2 11/27 Subtask 4.1: Identify 10 facilities in four slates for test 10/2 10/27 Subtask 4.2: Provide Handbook to participants 11/10 11/10 Subtask 4.3: Conduct workshop on procedures 11/13 11/14 Subtask 4.4: Conduct inspectort 11115 11/22 Subtask 4.5: Evaluate and revise handbook procedures and workshops 11/22 11/27 Key Deliverables for This Task: 1. List of inspection targets 10/27 2. Inspection reports 11/22 2-132 ------- ICF INCORPORATED V. SCHEDULE PLANNING Start Date End Date Task 5.0: Implement New Procedures 12/15 Subtask 5.1: Schedule and hold additional workshops Quarterly Subtask 5.2: Circulate handbook 12/15 As needed Subtask 5.3: Revise handbook As needed Subtask 5.4: Evaluate inspector peifonnance Periodically Subtask 5.5: Continue to track fish/all data Key Deliverables for This Task: None 2-133 ------- ICF INCOR PORATED VI. DELWERABLE SUMMARY Key Deliverable Interim Products teps Due Dates 1. List of a) Identify 9118 workgroup participants b) Obtain clearances from management 9120 2. Memos seeking participants a) Draft b) Obtain management approval 9/19 9/20 C) d) 3. Reference package for workgroup a) Draft b) Obtain management approval 9/20 9/22 C) Duphc ate 9/24 d) 4. Handbook a) Outline and abstract 10/30 b) Draft 11/10 c) Review and revise 11/2 7 d) P inz 12/15 5. Course workbook a) Outline 10110 b) Draft 11/3 C) Review and revise 11/10 d) Duplicate 11/15 2-134 ------- ICF INCORPORATED VI. DELIVERABLE SUMMARY (continued) Key Deliverable Interim Products/Steps Due Dates 6. List of inspection a) Draft 10/10 targets b) Review and approve 10/27 C) d) 7. State inspection a) Conduct inspections 11/22 reports b) Draft and file reports 11/22 C) d) 2-135 ------- ICF Ir CORPORATED VII. REVIEW STEPS FOR EACH PRODUCT Product 1: List of Workshop Participants Reviewers: Within GLNPO 1. 2. p’-’ - j 3. Within the Office of the National Program Manager 1 2 3. Within Headquarters 1. 2. 3. Within EPA Regions (Identify Regions). 1. 2 3. External (e.g., 0MB, White House, Canadian Federal Government, Canadian Provincial Government, States, I.J.C., Congress, news media, public, interest groups) 1. 2. 3. 2-136 ------- ICF lNCORPOR TED VU. REVIEW STEPS FOR EACH PRODUCT Produ 2: Memos S kzi g Participants Reviewers: Within GLNPO . -c ç .f 2 rc’I 3 r-tc k cy Within the Office of the National Program Manager 1. 2. 3. Within Headquarters 1 2. 3 Within EPA Regions (Identify Regions): 1. 2. 3. External (e.g., 0MB, White House, Canadian Federal Government, Canadian Provincial Government, States, LLC., Congress, news media, public, interest groups) 1 e. Pr r i. + 1 1 rw4trr 5 2 Jn s e. cA 3. 2-137 ------- ICE I CORPOR TED VII. REVIEW STEPS FOR EACH PRODUCT Product 3: Reference Package for Workshop Reviewers: Within GLNPO 1. -ç 2. 3. Within the Office of the National Pronram Manager 1. 4-. 3. Within Headquarters 1. 2. 3. Within EPA Regions (Identify Regions): 1. 2 3. External (e.g., 0MB, White House, Canadian Federal Government, Canadian Provincial Government, States, LIC., Congress, news media, public, interest groups) 1. 2. 3. 2-138 ------- ICF I!\COR PORATED VII. REVIEW STEPS FOR EACH PRODUCT Product 4: Handbook Reviewers: Within GLNPO 1. s- -c c 2. 3• Within the Office of the National Program Manaper 1. 2. 3. Within Headquarters 1. 2. 3. Within EPA Re2ions (Identify Regions): 1. 2. 3. External (e.g., 0MB, White House, Canadian Federal Government, Canadian Provincial Government, States, I.J.C., Congress, news media, public, interest groups) 1. 5-t- -t e_ Pr rc r1 J 2. ck e- 5 3. 2-139 ------- ICF I CORPOR TED VU. REVIEW STEPS FOR EACH PRODUCT Product 5: Course Workbook Reviewers: Within GLNPO 1. 2. 3. Within the Office of the National Program Manager 1. 2. 3 Within Headquarters 1. 2. 3. Within EPA Regions (Identify Regions): 1. 2. 3. External (e.g., 0MB, White House, Canadian Federal Government, Canadian Provincial Government, States, LJ.C., Congress, news media, public, interest groups) 1. 2. 3. 2-140 ------- ICF I CORPOR TED VII. REVIEW STEPS FOR EACH PRODUCT Product: 6 LIST OF INSPECTION TARGETS (No Review for Product 7) Reviewers: Within GLNPO i. .& crEE Ck CE 2. — 9 1 3 Within the Office of the National Program Manager 1. NL± CY & ç YO-4 1°- i --v 2 3. Within Headquarters 1. 2. 3. Within EPA Regions (Identify Regions): 1. 2. 3. External (e.g.. OMR. White House, Canadian Federal Government, Canadian Provincial Government. States, 1.3 C., Congress, news media, public, interest groups) i. S+cde. Pytr rc rec fry 2. Ir- -y\ ..5 e—cth t 2-140 A ------- VIII. RESOURCE ESTIMATION: TASKS Elapsed time is the number of days, weeks, or months between the start and end dates. In estimating work days (either EPA or contractor stair) it may hclp to think of percentages of time multiplied by the number of days of work on this task. Also factor in the roles and time needed of different EPA staff. “Other Costs” refers to any travel, printing, or equipment costs. Rcfer to the Estimated Price List (included In Reference Materials) for estimates of Other Costs. Task 1.0: Convene Workgroup Subtask /V’ Product(s) (if any) EPA Staff Contractor Other Work Days Work Days Costs 1.1 1.2 1.3 Identify Potential Participants Obtain Management Approval Secure & A ange Travel Ltzto.( i,wJw 4 I. 0 .3 / 0 0 O 0 0 0 P 0 1.4 Develop Reference Packages i c.Q. L Po 4 .L?_ 7 3.0 / 0. o 0 1.5 Hold Meetings / 15 i,. o 3.o 0 Totals for Task 1.0 Staff Work Days: 11.3 Contractor Work Days: 13.0 Othcr Costs: #‘g s oo 11.3 ‘3 ) Elapsed Time “3 ‘ -I — 0 0 -I ------- VIII. RESOURCE ESTIMATION: TASKS (Continued) Elapsed time is the number of days, weeks, or months between the Start and end dates. In estimating work days (either EPA or contractor staff) it may help to think of percentages of time multiplied by the number of days of work on this task. Also factor in the roles and time needed of different EPA staff. “Other Costs” refers to any travel, printing, or equipment costs. Refer to the Estimated Price List (included in Reference Materials) for estimates of Other Costs. F Product(s) Elapsed EPA Staff (if any) Time Work Days Other Costs 2.1 2.2 2.3 Develop Workgroup Notes Outline and Abstract 1 Qj k. Draft I1a4IprmJL / 4. 24 , “ ‘T 2. r ?, o 2. ô / C C) 2.4 2.5 Circulate/RevLce Finalize and Print F 4 I . — . 0 — A 0 0 ‘2. 5 01) ! I Totals for Task 2.0 Staff Work Days: 7. 5• Contractor Wk Days: i g’. Olher Costs: I ,O l7oo - Task 2.0: Develop Inspection Handbook Subtask Work Days r 3 I - ’ •! C ) 0 0 ------- VIII. RESOURCE ESTIMATION: TASKS (ContInued) Elapsed lime is the number of days, weeks, or months between the start and end dates. In estimating work days (either EPA or contractor staff) it may help to think of percentages of time multiplied by the number of days of work on this task. Also factor in the roles and time needed of different EPA staff. “Other Costs” refers to any travel, printing, or equipment costs. Refer to the Estimated Price List (included in Reference Materials) for estimates of Other Costs. F 1 — Task 3.0: Develop Inspector Workshop Product(s) Elapsed EPA Staff Contractor Other Subtask (if any) Time Work Days Work Days Costs 3.1 Outline 2_2. 3 Printing 3.2 Develop Materials bt), LL .,,k /• t’ zoo 3.3 Review and Revise F 1 41 . 14/,LAJ.V1’IL ‘ I S ” 3.4 Logistics ‘ “ 3.5 Conduct Workshop o 1) “Is — Totals for Task 3.0 Staff Work Days: . 0 Contractor Work Days: d S• — Other Costs:/.. 7 i Ir 10 I. . ) 1 0 Iu 10 L I I I> I- It ’, I ------- VIII. RESOURCE ESTIMATION: TASKS (Continued) Elapsed time is the number of days, weeks, or months between the start and end dates. In estimating work days (either EPA or contractor stafi) it may help to think of percentages of time multiplied by the number of days of work on this task. Also factor in the roles and time needed of different EPA staff. “Other Costs” refers to any travel, printing, or equipment COStS. Refer to the Estimated Price List (included in Reference Materials) for estimates of Other Costs. — (P.2—132) Task 4.0: lot Test New ocedures Product(s) Elapsed EPA Staff Contractor Other Subtask (if any) Time Work Days Work Days Costs 4.1 Identify Targets L t .at .f Ta.t .J- .i 2/ 0 4.2 Circulate Handbook I DP .Ø (. 0 4.3 Conduct Workthop (Cri.’t . 7L.DI.. . s) 4.4 Conduct Inspections ..Th 1 Ji iL ‘ I. 0 - 0 4.5 Evaluate and Revise 2. 0 - cost Totals for Task 4.0 Staff Work Days: q. Contractor Work Days: q / — Other Costs: o 0 C -3 ------- I .- . VIII. RESOURCE ESTIMATION: TASKS (Continued) Elapsed time is the number of days, weeks, or months between the start and end dates. In estimating work days (either EPA or contractor stall) it may help to think of percentages of time multiplied by the number of days of work on this task. Also factor in the roles and time needed of different EPA staff. “Other Costs” refers to any travel, printing, or equipment costs. Refer to the Estimated Price List (included in Reference Materials) for estimates of Other Costs. F, w,J L - c Task 5.0: Implement New Procedures Product(s) Elapsed EPA Staff Contractor Other Subtask (if any) Time Work Days Work Days Costs 5.1 Schedule and Hold Workshops s”. ô 000 5.2 Circulate Handbook l 0 5.3 Revise Handbook . , 0 5.4 Evaluate Inspection 2 .0 q. 5.5 Track Fi.shldlLr Totals for Task 5.0 Staff Work Days: I 3 . 0 Contractor Work Days: , 0 Other Costs: ( ) 00 ------- ICF INCORPORATED IX. RESOURCE ESTIMATION: PROJECT LEVEL £*.?) Project: EPA Staff Contractor Other Task Work Days Work Days Costs 1.0: (p. z-i’ii) 13.0 s—0 f . 2.0: ( . - 1’(2 ) 7 r I8 0 1?OO ! 3.0: (P 2 - . i J, 0 4.0: (P. 2. —i . a/ j 5.0: (/2 _1L1 ) / .O TOTALS: 2-146 ------- ICF INCOR PORATED • In order to evaluate the effectiveness of the improvement project, Wally collects data over the next year on both compliance inspections and fish kills. • As the bar chart on the following page shows, the two states that had been doing little effluent sampling before the project are now doing such sampling at about the same frequency as the other two states. 2-147 ------- STATE COMPARISON OF PERMIT INSPECTIONS (1989) Analyze Sampling Procedures Analyze Flow Other Measurement Techniques 20 18 16 14 12 10 8 6 4 2 0 G) C -) C c i ) 1 C-) C) 0 0 >, 0 C c i) a- w LL State 1 State 2 State 3 State 4 U 8 L 2 Records Effluent Sampflng Examination C C > Inspection Steps ------- ICF INCORPORATED • To effectively present the data on fish kills, he decides to construct a run chart that compares fish kills before and after the improvement project. • After a brief presentation on run charts, we will divide into breakout groups to construct a run chart: Number of fish kills in Lake Hemos, 1986 - 1989. 2-149 ------- 2.6 Run Charts ------- ICF INCOR PORATED TALKING POINTS FOR RUN CHART WHAT IS A RUN CHART? • A plot of measurements taken at regular intervals of time. 2. WHAT CAN RUN CHARTS DO FOR YOU? • Illustrate trends . • Identify the degree of variation of performance levels over time. • Identify short term abnormalities or long term changes. • A run chart can be used to compare the progress of two or more different activities over time. 3. WHAT TYPE OF DATA IS APPROPRIATE FOR RUN CHARTS? • Data that can be measured over time. • Look for data that tell you: -- how often an activity takes place, -- how much of a product is produced, -- how long a process takes, — how many errors are made. 4. WHEN DO YOU USE RUN CHARTS IN PROBLEM SOLVING? • Use it early in the problem solving process when you are trying to understand the level of performance or to identify variation in performance. • Use it after implementing a program to improve a process, to monitor changes, and to compare results before and after the improvement effort. 2-150 ------- ICF INCORPORATED 5. HOW DO YOU CONSTRUCT A RUN CHART? • Select a variable: an activity, process, or other event of concern. • Select an appropriate interval of time for recording the measurement of the activity, process or event. • Define the time intervals on the horizontal axis, and the range of values of the variable on the vertical axis. • Plot the value for the variable for each time interval. 2-151 ------- PCB CONCENTRATIONS IN COHO SALMON IN LAKE MICHIGAN (1980-1986) 2 1.75 1.5 0 0 co 1.25 C 0 C a) () C 0 0 0 0.5 0.25 0 I I 1980 1981 1982 1983 1984 1985 1986 nodata Lfl Tj I Reporting Period ------- INCOMING PHONE CALLS PER HALF HOUR (Monday and Friday) 8:30 9:30 10:30 11:30 12:30PM 9:00 10:00 11:00 12:00 1:00 1:30 2:30 3:30 4:30 5:30 2:00 3:00 4:00 5:00 6:00 Time Period (1/2 hour ending at time shown) I -o w > ci C., ci C l ) C L I C) 30 25 20 15 10 5 0 Monday Friday --s-- / I , A A A / / / 4 7:30AM 8:00 Lfl 1., A I I I I I I I I I I I I I I I I I I I I 1 — C ) 0 C > ------- ICF INCORPORATED GRANTS AWARDED BY MONTH (FY 1988)* MONIR RANI MONTh GRANI AWARDED AWARDED 10/87 0 4/88 9 11/87 0 5/88 7 12/87 2 6/88 11 1/88 5 7/88 18 2/88 6 8/88 23 3/88 7 9/88 51 * DATA FROM GLNPO FY 1988 “B” MONEY TRACKING SYSTEM. 2-154 ------- GRANTS AWARDED BY MONTH (FY 1988) — .!, 0 rs) U I UI ------- GRANTS AWARDED BY MONTH (FY 1988) 55 50 45 4o V I- C I 10/87 11/87 12187 1/88 2/88 3/88 Month U I a’ 4/88 5/88 6/88 7/88 8/88 9/88 0 0 - ------- ICF INCORPORATED MEANWHILE, BACK AT LAKE HEMOS 2-157 ------- ICF INCORPORATED NUMBER OF FISH KILLS IN LAKE HEMOS (1986 - 1989) QtrIYr # Fish Kills Qtr/Yr # Fish Kills 1/86 20 1/88 15 2/86 18 2/88 14 3/86 17 3/88 14 4/86 17 4/88 13 1/87 18 1/89 12 2/87 17 2/89 10 3/87 16 3/89 8 4/87 15 4/89 6 Using the data in the table above, construct a run chart of fish kills. Blank shells are provided. 2-158 ------- NUMBER OF FISH KILLS IN LAKE HEMOS (1986-1989) — 0 F ) p-a 0 U I ‘ 0 ------- ICF INCORPORATED CONTINUOUS QUALITY IMPROVEMENT Focus on the customer Rely on the experts Improve productivity by improving quality Continuously improve your work processes 2-161 ------- 3 CQI TOOLS: REVIEW ------- ICF INCORPORATED THE PROCESS Identify a Problem Describe the Dimensions of the Problem Analyze the Causes Narrow the Focus of Attention Develop Improvement Project Monitor Effectiveness 3-1 ------- ICF INCORPORATED THE C&SE OF FISH KILLS IN THE OS t.A S: WALLY PIKE’ S QUEST TO REDUCE FISH KILLS Data has been collected on Memos’ rate of fish kills. While kills are being reduced, reductions are not meeting expectations. QUESTION q/3:WHAT ARE THE COMPLIANCE? STEPS INVOLVED IN INSPECTING FOR PERMIT METHOD: FLOW DIAGRAM OF PERMIT INSPECTION PROCESS. QUESTION #4:HOW FREQUENTLY ARE THE PRINCIPAL TYPES OP INSPECTION ACTIVITIES CONDUCTED? NETHOD j fl: STUDY OF INSPECTION ACTIVITIES IN EAGH OF THE FOUR STATES. #2: VISUAL DISPLAY OF DATA ON INSPECTION ACTIVITIES WITH FIVE BAR CHARTS QUESTION #5:WHY DOES THE L EVEL OF INSPECTION ACTIVITY VARY BY STATE? METHOD: INFORMED GUESS COLLEAGUES IN CONFIRMED THE FIELD BY DISCUSSIONS WITH QUESTION #6:WHAT CAN BE DONE TO IMPROVE INSPECTION PERFORMANCE? METHOD: PROJECT PLAN FOR IMPROVING INSPECTION PROCESS. 3-2 QUESTION #1: METHOD: WHAT ARE THE CAUSES OF FISH KILLS? FISHBONE QUESTION #2: WHAT ARE THE MOST SIGNIFICANT CAUSES OF FISH KILLS? METHOD #1: BEST PROFESSIONAL JUDGEMENT SURVEY #2: A PARETO OF THE EXPERT SURVEY DATA ON CAUSES OF FISH KILLS QUESTION j/7:IF THE INSPECTION PROCESS IS IMPROVED, WILL FISH KILLS BE REDUCED? METHOD #1: ANNUAL REPORTS ON INSPECTION ACTIVITIES #2: RUN CHART DISPLAYING 1989 DATA ON FISH KILLS ------- 4 COIPROJECTS ------- (TMU)09. 1 4 • S NU.LO FROM ICF IHC PROJECT: Develop means to improve communication and information sharing within the office. TEAM: Paul Bercram Team Leader ------- 1(1 l\ RII I \I GLNPO CQI PROJECTS The Project Leaders Role In Meetings: • Review project or problem, and expectation for meeting. • Discuss ground rules (participation, conversational courtesy, “Amnesty,” schedule). • Assign facilitator (may be project leader). • Assign timekeeper (may be same as facilitator). • Assign recorder (should be facilitator or timekeeper). • Assign person to present briefing(s) (may be project leader). Between Meetings: • Serve as contact point between the group and the rest of the organization, including clerical services. • Serve as keeper of the group’s records. • Prepare agendas for meetings. • Set meeting logistics (time, location, notification) 4-1 ------- 1(1- I\I 1k’l ’CPR \I I GLNPO COl PROJECTS The GrouD’s Role In Today’s Meeting: • Apply problem-solving techniques to the group’s project. (Use fishboning, flow charting, Pareto charts, systematic project management, or whatever methods seem appropriate.) • Summarize results of your efforts. Prepare simple overheads that include: - Tools/Approaches you used - Major conclusions/insights thus far - Problems you encountered - Next steps - (What more could your group or another group do? What should management consider doing?) Following Today’s Meeting: • Do what you can to make yourself available for Team meetings; keep schedules that you agree to. • Play specialized roles (facilitator, timekeeper, recorder) in Team meetings as requested. • Be a constructive meeting participant: participate actively, but don’t monopolize the discussion; exercise “conversational courtesy”; grant “amnesty” to fellow team members to encourage frank discussion. • Complete between-meeting assignments you commit to. 4-2 ------- ESTIMATED PRICE LIST (Government Rates) A. RoundtriD Travel Costs (S From Chicago: Plane* Per Diem Buffalo $344 $ 76 Cleveland $154 $ 93 Detroit $ 64 $100 Duluth $231 $ 70 New York City $276 $141 Philadelphia $232 $110 Washington, D.C. $268 $121 *Prices assume no advanced booking and unrestricted fares. B. Printing]Artwork : 1. Brochure (4 pages, 8-1/2 x 5-1/2; black ink only; 60 lb. paper; includes typesetting) -- 100 printed copies: $190 + $14 per photo -- 1,000 printed copies: $220 + $14 per photo 2. Poster (3’ x 2’ on 1/4” posterboard; black ink only; simple picture and minimal text) - - 1 poster, with EPA-supplied artwork: $100 -- 1 poster, with artwork part of the purchase: $150 ------- 1989 January SMT 123 8 9 10 15 16 17 22 23 24 29 30 31 1989 WTFS 4567 1112 13 14 18 19 20 21 25 26 2728 1989 TFS 456 1112 13 18 19 20 25 26 27 1989 WTFS 1234 8 9 10 11 15 16 17 18 22 23 24 25 1989 TFS 123 4 5 6 7 8 9 10 1112 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 1989 TWTFS 1234 6 7 8 9 10 11 13 14 15 16 17 18 20 21 22 23 24 25 27 28 29 30 31 1989 TWTFS 1 45678 11 12 13 14 15 18 19 20 21 22 25 26 27 28 29 November SMTWT 12 56789 12 13 14 15 16 19 20 21 22 23 26 27 28 29 30 2 9 16 23 30 1989 TWTFS I 345678 10 1112 13 14 15 17 18 19 20 21 22 24 25 26 2728 29 1989 FS 12 456789 1112 13 14 15 16 18 19 20 21 22 23 25 26 27 28 29 30 February SMT 567 12 13 14 19 20 21 26 27 28 March SM April 5 12 19 26 May SMTW 123 7 8 9 10 14 15 16 17 21 2223 24 28 29 30 31 June SMTW September SMTW July SM 23 9 10 16 17 23 24 30 31 1989 TFS 12 6789 13 14 15 16 20 21 22 23 27 28 29 30 345 10 11 12 17 18 19 24 25 26 August SMTWTFS 12345 6 7 8 9.10 11 12 13 14 15 16 25 26 20 21 October SMT 123 8 9 10 15 16 17 22 23 24 29 30 31 1989 WTFS 4567 1112 13 14 18 19 20 21 25 26 27 28 1989 FS 34 10 11 17 18 24 25 December SMTWT 10 17 24 31 ------- |