3/21/23, 2:57 PM Environmental Protection Agency https://archive.epa.gov/publicinvolvement/web/html/1711ss03.html Last updated on 2/20/2016 1% Public Involvement You are here: EPA Home Collection Request Office of Policy OSEM Historical Information Public Involvement Information Information Collection Request Note: EPA no longer updates this information, but it may be useful as a reference or resource. September 8, 1999 INFORMATION COLLECTION REQUEST for CUSTOMER SATISFACTION SURVEYS (ICR Number 1711.03) EPA The United States Environmental Protection Agency Table of Contents https://archive.epa.gov/publicinvolvement/web/html/1711ss03.html 1/41 ------- 3/21/23, 2:57 PM Title Environmental Protection Agency Page 1. Identification of the Information Collection 1 1(a) Title of the Information Collection 1 1(b) Short Characterization/Abstract 1 2. Need for and Use of the Collection 1 2(a) Need/Authority for the Collection 1 2(b) Practical Utility/Users of the Data 3 3. Non-duplication. Consultations, and Other Collection Criteria 5 3(a) Non-duplication 5 3(b) Public Notice Required Prior to ICR Submission to OMB 5 3(c) Consultations 5 3(d) Effects of Less Frequent Collection 6 3(e) General Guidelines 6 3(f) Confidentiality 6 3(g) Sensitive Questions 6 4. The Respondents and The Information Requested 6 4(a) Respondents/SIC Codes 6 4(b) Information Requested 7 (I) Data items, including record keeping requirements 7 (II) Respondent Activities 8 5. The Information Collected--Aaencv Activities. Collection Methodology, and Information Management 5(a) Agency Activities 5(b) Collection Methodology and Management 5(c) Small Entity Flexibility 5(d) Collection Schedule 6. Estimating the Burden and Cost of the Collection 6(a) Estimating Respondent Burden 6(b) Estimating Respondent Costs (I) Estimating Labor Costs (II) Estimating Capital and Operations and Maintenance Costs (III) Capital/Start-up vs. Operations and Maintenance (O&M) Costs (IV) Annualizing Capital Costs 6(c) Estimating Agency Burden and Cost 6(d) Estimating the Respondent Universe and Total Burden and Costs 6(e) Bottom Line Burden Hours and Cost Tables (I) Respondent Tally (II)The Agency Tally (III) Variations in the Annual Bottom Line (IV) Reasons for Change in Burden (V) Burden Statement 12 12 12 16 16 16 16 17 17 17 17 17 17 18 21 21 21 22 23 24 List of Figures Figure 1- Customer Groups Surveyed Figure 2 - Focus for Improvement Figure 3 - Past Use of Survey Instruments Figure 4 - Planned Use of Surveys FY 1998 - 2000 Figure 5 - Uses. Respondents & EPA Costs 7 8 13 15 24 List of Tables Table 4-2 - Survey Use 1995 - 1998 Table 5-1 - Planned Use of Surveys 2000 - 2002 Table 5-2 - Planned Survey Use Averages 7 10 15 16 https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 2/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency Table 6-1 - Planned Survey Activities 25 - 38 Table 6-la - Response Time Summary 38 Table 6-2 - Agency Burden/Cost for Telephone Surveys 19 Table 6-3 - Agency Burden/Cost for Mail Surveys 19 Table 6-4 - Agency Burden/Cost for Customer Feedback Forms/Internet Screens 20 Table 6-5 - Agency Burden/Cost for Focus Groups 20 Table 6-6 - Aggregate Agency Table for Annual Burden/Cost 21 Table 6-7 - Respondent Universe. Total Burden and Costs 21 Table 6-8 - Estimated Agency Costs during FY 2000 22 Table 6-9 - Estimated Agency Costs during FY 2001 22 Table 6-10 - Estimated Agency Costs during FY 2002 23 Attachments Exhibit 1 - Sample Focus Group Questions Exhibit 2 - Feedback Mechanisms a) - Sample Comment Card b) - Sample Evaluation Form c) - Sample Internet Feedback Screen Exhibit 3 - Sample Mail Surveys Exhibit 4 - Sample Telephone Survey U.S. ENVIRONMENTAL PROTECTION AGENCY VOLUNTARY CUSTOMER SATISFACTION SURVEYS TO IMPLEMENT EXECUTIVE ORDER (E.O.) 12862 1. Identification of the Information Collection 1(a) Title of the Information Collection: Voluntary Customer Satisfaction Surveys to Implement Executive Order 12862. ICR Number 1711.03. 1(b) Short Characterization/Abstract In accordance with E.O. 12862, the Environmental Protection Agency is seeking to renew its generic clearance (OMB Control No. 2090-0019, expiring 10/31/99) for a period of three years from Office of Management and Budget (OMB) to conduct two types of customer satisfaction surveys: "qualitative" surveys for identifying customer perceptions for expectations through focus groups or laboratory evaluations;"and""quantitative" surveys for establishing general attitudes of EPA customers through a statistical sampling of customers. Customers, as described in E.O. 12862, are considered to be "...an individual or entity who is directly served by a department of an Agency." https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 3/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency By seeking renewal of the generic clearance for customer surveys, EPA will have the flexibility to gather the views of our customers to better determine the extent to which our services satisfy their needs or need to be improved. The generic clearance will speed the review and approval of customer surveys that solicit opinions from EPA customers on a voluntary basis, and do not involve "fact-finding" for the purposes of regulatory development or enforcement. EPA sponsoring organizations seeking approval to conduct a customer survey will continue to submit their survey instruments with a brief description through the EPA's Regulatory Information Division (RID) to OMB. OMB will continue to review submissions for compliance with the Paperwork Reduction Act on an expedited schedule. EPA will continue to provide OMB an annual summary of surveys conducted in accordance with OMB's Resource Manual for Customer Surveys (dated October 1993). The EPA estimates that a combination of customer satisfaction surveys (mail, telephone, feedback forms and Internet) and focus group studies will request voluntary responses from approximately 206,221 respondents for an estimated burden of 46,084 hours over the three-year period - 73,512 respondents and 17,521 hours in FY 2000; 63,937 respondents and 14,034 hours in FY 2001, and 68,772 respondents and 14,529 hours during FY 2002, for an average of 13.4 minutes per respondent overall. 2. Need for and Use of the Collection 2a. Need/Authority for the Collection Executive Order 12862, dated September 11, 1993, calls upon agencies to take the following actions: (a) identify the customers who are, or should be, served by the agency; (b) survey customers to determine the kind and quality of services they want and their level of satisfaction with existing services; (c) post service standards and measure results against them; (d) benchmark customer service performance against the best in business; (e) survey front-line employees on barriers to, and ideas for, matching the best in business; (f) provide customers with choices in both the sources of service and the means of delivery; (g) make information, services, and complaint systems easily accessible, and (h) provide means to address customer complaints. A March 1995 Presidential memo called upon federal agencies to enhance their customer service improvement efforts. A March 1998 Presidential memo underscored the continuing need to improve customer service and directed agencies to provide expanded opportunities for customers to communicate their needs and expectations through "Conversations with America." These conversations include both formal and informal surveys. Finally, the Governmental Performance and Results Act of 1993 requires that agencies gather and use customer feedback. https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 4/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency OMB's Resource Manual for Customer Surveys (dated October 1993), outlines the steps an Agency must take to obtain a generic clearance for Customer Satisfaction Surveys, and provides guidance on obtaining quality survey results. Under this guidance, the EPA developed its second generic information collection request to permit continued implementation of the Executive Order for customer satisfaction surveys in 1997. During the two years of the renewal EPA has worked cooperatively with OMB to clear approximately 30 survey instruments. To fulfill its broad mandate of protecting human health and the environment, the EPA provides a wide variety of voluntary public services ranging from information clearinghouses to educational programs and emergency hot lines. Corresponding to this broad range of services is a diverse universe of EPA customers, loosely defined by E.O. 12862 as "...an individual or entity who is directly served by a department or agency." EPA expands this definition to include customers who could have, but chose not to participate in an EPA service function, such as persons who were provided the opportunity but did not comment on a permit, participate in a community meeting, join a partnership program, etc. Learning perceptions of our services from those who select not to use them may also assist the Agency in its service reinvention efforts. As we redesign our processes and practices, we will be asking customers who used our current services what, from their perspectives, would be the most useful improvements. Because Agency services and customers are so diverse, the Agency is requesting a generic clearance that will maximize flexibility in the methods used to fulfill the requirements for the Executive Order and expedite OMB review and clearance process of customer satisfaction surveys. EPA maintains a central repository of surveys submitted to OMB in the Regulatory Information Division. In addition, the Customer Service Program (CSP) developed a summary of the surveys and collected the analytical reports produced. CSP staff shares the findings, analysis and "success stories" following the conduct of surveys so this information can benefit those planning future surveys. The CSP "survey central" continues to be a resource to individuals considering the development of customer satisfaction measurement programs within their organizations. 2(b) Practical Utility/Uses of the Data The Executive Order stresses the need to engage top-level management at the earliest stages of the survey development to ensure the information collected provides actionable advice for managers. To this end, EPA implemented a standards and a Customer Service Plan (Putting Customers First EPA 230-B-95-004, September 1995) as required under Section 3 of E.O. 12862. Standards form the basis for performance measurement systems to determine our success at reaching customers, and provide the necessary framework for a management role in the development and use of the survey results. Information gathered from these surveys will continue to assist EPA to build and validate measurement systems. Survey results may be used to identify: 1) service needs and expectations of EPA customers; 2) strengths and weaknesses of EPA services; 3) ideas or suggestions for improvement of EPA services from its customers; 4) barriers to achieving customer service standards; and https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 5/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency 5) needed changes to customer service standards. While the information will not be used for regulatory development, the EPA anticipates that the results of customer surveys could lead to reallocations of resources, revisions in certain Agency processes and policies, and development of guidance related to EPA's customer services. Ultimately, these changes could result in improvements in services the Agency provides to the public, and in turn, the public perception of the Agency. During 1998, the CSP purchased survey software and a scanner, and since then has trained numerous individuals across the agency to use these resources to simplify satisfaction measurement work. The CSP staff members provide training in the use of the software, assist individuals to prepare survey instruments using it, and guide them in using the scanner and reporting features. To ensure proper design of EPA customer feedback and customer satisfaction measurement activities, increase the use and application of customer feedback, and build internal capacity to carry out these activities, the Customer Service Program (CSP) coordinated development of "Hearing the Voice of the Customer - Customer Feedback and Customer Satisfaction Measurement Guidelines." The "Feedback Guidelines" were first published in November 1998. In February 1999 the CSP provided training in the use of the Guidelines to a group of 23 Feedback Advisors from across the agency. These individuals are now working within their offices and regions to help others plan, construct, conduct, analyze and act on findings from feedback activities. Additional workshops will occur in 1999 and 2000. In May 1999, a revised version of the document was published, taking into account the requests of Feedback Advisors for specific additional information they believed would be helpful to them and their clients. The document is available on the Internet (https://www.epa.gov/customerservice/guide.htm), and is being used by individuals in other federal and state agencies to guide their feedback efforts. A five-stage model for feedback: Plan, Construct, Conduct, Analyze and Act, is the foundation for the Guidelines. The document focuses major attention on the planning phase, with the object being to prevent duplication and poor design, and to eliminate survey work that will not result in actions that can benefit customers and the agency. A long series of detailed questions supplement the Guidelines to further assist the Feedback Advisors and others. The Guidelines and questions are not our only resources. To help ensure that feedback information used in an appropriate fashion, EPA programs are encouraged to develop surveys consistent with OMB's Resource Manual for Customer Surveys. EPA's Survey Management Handbook and to take advantage of survey development training such as that offered by the Joint Program in Survey Methodology (JPSM). The EPA Customer Service will continue to facilitate sharing of information gathered from customer satisfaction surveys, and explore ways to aid programs in survey development. As a result of past survey feedback, sponsors have taken actions. A few examples follow. First, the New Chemicals Program's survey results were used to enlist customers in a cooperative effort to establish a set of Customer Service Standards which were published in 1997. As a direct result of this highly successful set of https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 6/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency cooperative activities, the New Chemical Program developed an active Customer Outreach program consisting of a New Chemicals Home Page on the Internet which features frequent programmatic news items and policy updates, a seminar program designed to improved customer understanding of the program, and an effort to engage industry clients in an expanded set of pollution prevention and innovative technology initiatives. In addition, the program developed a limited "Transaction-based Survey" to help EPA staff evaluate their performance in meeting established customer service standards. In response to the Dockets Survey (Air, Water, RCRA), the Air Docket installed a TDD telephone for the hearing impaired, and has increased the number of electronically available documents. The Water Docket continues to electronically streamline, receiving comments on Federal Register Notices of Proposed and Final Rules and other items, and making the Index of Rules available through the Water website. The Office of Solid Waste (OSW) Resource Conservation & Recovery Act (RCRA) Information Center took the recommendations from 1995 Docket Survey very seriously. Because respondents to the survey recommended that the Docket implement universal electronic text search and retrieval of its materials and direct access to the materials via the Internet, the following actions were taken: Converted RCRA Information Center's largest dockets to electronic format so that it can be made keyword and full text searchable on the Internet. Studied existing work flow and business processes in the Office of Solid Waste & Emergency Response (OSWER) Dockets. Implemented recommendations from the study included improving processes and providing the public with access to all docket materials via the Internet. Participating in the Agency Docket Common Indexing Study. (The survey results directly prompted this entire effort.) Accepting public comment via the Internet at the Docket. Developing all publications in both print and electronic format for posting on Internet is now required in OSW policy. Requiring all docket materials generated by EPA to be provided in hard copy and Internet ready format. Because Docket users recommended preparation of a pamphlet describing the Docket services, OSW revised its pamphlet describing docket services and translated it into Spanish to increase public access to OSW information. Because customers complained about poor photocopy quality, the RCRA Docket added a copier for the public. Based on customers' requests that Docket materials remain available in paper form, the RCRA Docket is committed to maintaining paper copies of the materials for public use. 3. Non-duplication, Consultations, and Other Collection Criteria 3(a) Non-duplication Customer satisfaction surveys developed by an EPA service provider will evaluate the perceptions of customers for that specific service. Therefore, the information collected will not overlap with other customer satisfaction https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 7/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency surveys. Every effort will be made to channel all customer related surveys through this ICR. Further, every effort will continue to be made to prevent the misuse of this ICR for program effectiveness surveys. 3(b) Public Notice Required Prior to ICR Submission to OMB EPA conformed to the requirement for public notice by publishing a preliminary and final Federal Register Notice concerning our intent under this ICR and requesting comment. 3 (c) Consultations This ICR was prepared in consultation with representatives from nearly every region and program within the Agency to project survey needs through the year 2002. This feedback was used to develop the estimates described in Figures Three and Four, and to prepare the estimates of respondent burden (See Section 6.) and the Estimated Respondent Burden Table (Table 6-1). Individuals who responded had opportunities to review and comment on the application and the burden projections. In addition, one telephone comment/question was received from the public in response to the first Federal Register Notice. That comment/question was "What a great idea to ask us how you are doing, and how likely is it that my company (en electric utility) will be surveyed?" 3(d) Effects of Less Frequent Collection This information collection could not be conducted less frequently. EPA will gage customer reactions to and perceptions of services and products the Agency now provides in order to improve them. Programs will not survey all customers, nor will each program survey every year. There will be sufficient time between surveys to allow the actions taken in response to customer comments to show results. There are no technical or legal obstacles to reducing the burden. 3 (e) General Guidelines This ICR complies with OMB's general guidelines for the collection of information. 3(f) Confidentiality Not applicable https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 8/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency 3(g) Sensitive Questions No sensitive data will be collected. 4. The Respondents and the Information Requested 4(a) Respondents/SIC Codes The Executive Order describes Customers as "...an individual or entity who is directly served by a department or agency." The EPA, by the very nature of its mandate, serves very large and diverse groups that receive or are in some way affected by EPA services. Figure One displays information about past EPA customer surveys under this general clearance. It provides an overview of EPA customer groups targeted for customer satisfaction surveys. Because several customer groups use the same services, a survey may reach more than one of the designated customer categories. (The code standard industrial code (SIC) for "General Public" is 99.) Past use of this ICR is arrayed in Table 4-1. It shows the annual number of surveys, respondents and burden hours for the years 1995 through 1998, and the averaged numbers. 1 https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 9/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency Customer Groups Surveyed J Citizens I States/Other Government . Business /Industry Table 4-1 ICR Use Year # Surveys Respondents Burden Hours 1995 20 16,735 4,395.5 1996 12 12,144 3,476.5 1997 16 16,275 3,234.0 1998 22 16,279 1,478.4 Average 17.5 15,358.25 3,146.1 4(b) Information Requested (I) Data items, including record keeping requirements The Agency will maintain records of the surveys sent to OMB in the RID. Offices sponsoring the surveys will retain files of the surveys, responses and analysis. Since customer satisfaction surveys seek to gauge public opinions on Agency services, the surveys have not and will not involve respondents in extensive searching of existing sources, or reformatting information to submit to the Agency. The Agency does not anticipate anv public record keeping activities under this ICR. (II) Respondent Activities https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 10/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency Figure Two describes the Agency's past experience with respect to the focus of customer service surveys under the customer service ICR. EPA customer service surveys have focused on services (hot lines, dockets, clearinghouse, websites), products (technical assistance, documents, information, training, workshops) and processes (grants, inspections, registrations, permitting). The surveys conducted under this clearance may be divided into two major types, "quantitative" and qualitative. Figure 2 Focus for Improvement Respondent activities related too "quantitative" are dependent on the survey method: Mail surveys and Customer Feedback Forms (including comment cards and evaluation forms)(Ł). Both may involve the following activities: - Read instructions; - Search data sources; - Complete questionnaire; - Mail questionnaire. Telephone Surveys - Listen to instructions; - Answer questions (oral response) EPA expects to continue its use of these surveys. Respondent activities related too "qualitative" may include: https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 11/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency Focus Groups or Interviews Listening to group instructions Participating in discussions; Completing any forms or materials provided at the group session. EPA expects to use focus groups for evaluating various aspects of its programs, to assist in improving and testing of outreach materials and websites, and to explore new aspects of service delivery. Training/Education/Outreach products and services. Evaluations of EPA training, outreach products and educational programs will be conducted through a variety of methods. The EPA distributes a broad array of materials to the public such as public affairs materials, videos, brochures and fact sheets, software, manuals, guidance material, reports, etc. It also hold many meetings, workshops and training sessions. Corresponding to this diverse set of products is a need to make extensive use of a variety of methods to evaluate customer satisfaction. Feedback forms in publications, focus groups, mail and telephone surveys are used. In addition, when publications are available on the Internet, the Agency is using short on-line surveys to solicit customer input. Offices are also asking for feedback on the usefulness of their websites. Many of these evaluation activities can use feedback forms to be completed by attendees after an EPA- sponsored event, by users of documents, software or websites. Focus groups may also be used extensively for pre-testing EPA training materials (videos, brochures, etc.) prior to their dissemination to the public. Mail or telephone surveys may be used to help EPA identify a need for changes in training/educational programs, outreach products or services to assure their usefulness to a specific audience. Hot lines/PICs/clearinghouses. Hotline evaluations will be conducted on selected samples of hotline users. By their very nature, hotline customers will most often be surveyed by telephone. However, more complex surveys may require face-to-face interviews, focus sessions, or mail questionnaires. In addition, comment cards will be used periodically when information packets are mailed by hotline, Public Information Center (PIC) or clearinghouse staff. Miscellaneous Service Related Activities. The EPA has a broad network consisting of its headquarters and regional offices, laboratories, and field offices that will conduct customer surveys on outreach and other services that they may provide. It is anticipated that most of the mail and telephone surveys will be conducted under this "miscellaneous" category. To reduce respondent burden, EPA expects to continue expanding the use of Internet feedback screens and comment cards to provide increased opportunity for customers to comment on attributes of our services. Only a few offices each year will be developing lengthy questionnaires. Since there is a comment button on all EPA Internet sites, the Agency is receiving and will continue to receive informal feedback and questions that are purely voluntary and not solicited specifically through sets of Agency https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 12/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency questions of nine or more individuals outside the Federal government. We plan to continue to manage and act upon such customer information, particularly to improve EPA's on-line information service on Internet. In addition, during the summer of 1999 and at some time in 2000, the Agency will participate in the Government wide Customer Satisfaction Survey required by the Vice President in January 1999, and being managed by GSA under OMB Clearance # 3090-0271, expiring December 31, 1999. For EPA the customer segment is Reference Librarians using the Internet to access environmental information using the Agency's on line information service. We expect that the Agency will want to do follow up survey work to explore specific aspects of its on line services and steps needed to improve them. Table 4-2 arrays the total number of surveys, the number of respondents and the burden hours for the years 1995 through 1998. Through June 1999, six additional surveys have been cleared through OMB. Table 4-2 Survey Use 1995 - 1998 Year # Surveys Respondents Burden Hours 1995 20 16,735 4,395.5 1996 12 12,144 3,476.5 1997 16 16,275 3,234.0 1998 22 16,279 1,478.4 Average 17.5 15,358.25 3,146.1 OMB's Resource Manual for Customer Surveys (dated October 1993) and other relevant guidance documents state that the generic clearance shall be used for "strictly voluntary collections of opinion information from clients that have experience with the program that is the subject of each data collection" and precludes this option for use: by regulatory agencies to survey regulated entities^2)-. in any situation where a respondent may perceive that a response will result in risks to his interests through potential penalties or loss of benefits; for collecting factual information (other than simple identifying information, where needed); or for collecting data from the general public/3) The Terms of Clearance provided to EPA for its two-year renewal of the ICR stated: https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 13/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency "This ICR is approved under 5 CFR 1320 for two (2) years, with the following specified Terms of Clearance. Over the past three years, there has apparently been some confusion as to the applicable uses for this generic ICR clearance. Perhaps as many as one in four of the draft surveys submitted to OMB so far in 1997 under this generic clearance have not been surveys addressing customer satisfaction, and are not consistent with OMB's Terms of Clearance. These surveys have instead dealt primarily with fact-finding for the purpose of enforcement, and are thus inconsistent with OMB's 1994 Terms of Clearance. The generic ICR is approved to allow the expedited OMB clearance of EPA customer satisfaction surveys that are simple, straight-forward, and narrowly focused to: 1. current or former customers of EPA products or services 2. the level of satisfaction with an actual service or product provided by EPA that they have utilized; and, 3. their recommendations for improving said EPA product or service. The first two of these elements are necessary for each customer satisfaction survey approved under this generic ICR. The third element is appropriate so long as the focus of the survey is on elements one and two. Surveys that target these elements and are submitted to OMB in accordance with the ICR will be reviewed by OMB within 10 working days. For example, the following types of surveys are NOT covered by these Terms of Clearance: potential customers of an EPA product or service. For example, a survey of the general population as customers of EPA Clean Air programs represents too broad a population category. In addition, surveys whose focus is asking various stakeholder groups about the direction an EPA regulatory program should take will not be approved. Data gathering an industry or population for the purposes of fact finding." And: "OMB is relying in large part on EPA's internal review and quality control to develop useful customer information. This generic ICR approval does not and is not intended to cover all types of surveys that EPA may wish to do relating to customer satisfaction -- only the narrow range of surveys - discussed above." With these Terms of Clearance as one factor in budget justification, EPA's Customer Service Program (CSP)developed the document "Hearing the Voice of the Customer - Customer Feedback and Customer Satisfaction Measurement Guidelines" and supported training in their use. (See discussion of these efforts on pages 3 and 4.) In addition, copies of a fact sheet clearly stating the restrictions on the use of this clearance were widely distributed and made available on the CSP web page. Further, when assisting headquarters and regional staffs in using the CSP survey software and in preparing their packages for OMB clearance, CSP staff made it clear that OMB would not approve some of the surveys proposed. As a result, numerous surveys were redesigned to fit the requirements, plans for survey work were canceled, or offices developed separate ICRs. 5. The Information Collected Agency Activities, Collection Methodology, and Information Management 5(a) Agency Activities. Agency activities associated with the collection of information include: Developing survey design, assembling data sources (mailing lists, etc.) and pretesting questionnaire; Internal EPA review and approval of questionnaire; https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 14/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency Disseminating questionnaire to respondents; Gathering information from respondents; Answering respondent questions, follow-up; Reviewing data; Recording submissions and analyzing results; Preparing findings; Storing and maintaining results Making results public via annual reports and Internet. We do not account for the work of implementing and tracking actions taken as a result of customer feedback. 5(b) Collection Methodology and Information Management. Figure 3 displays EPA's use of different types of customer satisfaction surveys requiring OMB review and approval from 1995 through June 1999. Included are surveys conducted by EPA program offices, regions and headquarters elements. The Customer Service Program staff will continue to encourage survey sponsors to develop instruments using the twelve step process outlined in OMB's Resource Manual for Customer Satisfaction Surveys (dated October 1993). The following internal review process, independent of the originating program office, will continue: Figure 3 Past Use of Survey Instruments 35 30 25 20 15 10 5 0 | | Mail | | Feedback/Comment/Evaluation Forms I | Telephone | | Web Site Surveys ~ Focus Groups -Prior to initiating the survey, sponsoring programs must seek final approval from OMB. -To obtain approval, sponsoring programs must submit a clearance package consisting of a memorandum from the program or office director and a copy of the survey instrument through RID to OMB. The memorandum must address the following o Survey title, identification of survey originator (Office, point of contact/phone number,); https://archive.epa.gov/publicinvolvement/web/html/1711ss03.html 15/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency o Description and intended purpose of the survey as it relates to EPA customers; o Methodology and use of anticipated results; o Collection schedule, follow-up plans; o Costs and burden to the Agency and respondents, and the number of respondents. - The memorandum will vary in length and detail, depending on the complexity of the survey. RID staff, experienced with the requirements of the Paperwork Reduction Act (PRA), will review each submission to ensure that it meets the requirements of the PRA and any conditions of the generic approval, and may reject any proposed customer survey that does not meet the criteria outlined in Section 3(b). - Statistical methods will not be used for the majority of the collections covered under this generic clearance. However, when a collection does use statistical methods to select a sample, answers to questions 1 through 5 in this section of the OMB guidelines for preparing supporting statements will be provided for that specific survey at the time the survey instrument is sent to OMB for clearance. If statistical design or methodological issues arise, the program shall solicit Agency statistical experts through EPA's Center for Environmental Statistics or program office statisticians to make any final determinations as to the statistical validity of the customer survey prior to OMB submittal. - RID will submit surveys and attached materials to OMB for any expedited review and determination. On an annual basis, the EPA shall submit a summary of the surveys cleared under the generic clearance to OMB. The summary shall include the survey title, sponsoring office, number of respondents and estimated burden hours. - Sponsoring organizations within the EPA should maintain records according to each survey schedule. In general, survey results should be maintained for three years or until after follow-up activities have been completed. - All offices will provide copies of their approved surveys, analytical reports and follow-up actions taken based on survey results to customer service. - The customer service staff will share results and success stories with other offices and provide feedback to RID on overall survey results. This base of experiences/lessons-learned could be useful in establishing model surveys for developing customer measurement programs within the EPA and other Agencies. Figure 4 depicts the planned use of survey instruments during the next three years. Table 5 -1 summarizes also displays the totals of the planned surveys for 2000 - 2002 Averages for the survey types were computed based on four basic types: feedback (to include comment cards, feedback and evaluation forms, and web based questionnaires); mail surveys, telephone surveys/short interviews and focus groups/long interviews. These are displayed in Table 5-2. https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 16/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency 2000 2001 2002 | | Feedback/Cornrnent/Evaluation Form ~ Mail Survey I I Telephone Survey | | Web-based I | Focus Groups Table 5-1 Planned Use of Surveys 2000 - 2002 Survey Type 2000 2001 2002 Total F eedback/Com-ment/Evaluation F orm 52 54 55 161 Mail 29 28 24 81 Telephone* 24 12 15 51 Web based 20 17 18 55 Focus Group* 126.6 groups 11 instruments 119.6 groups 2 new instruments 117.6 groups no new instruments 363.8 groupsl3 instruments Total 251.6 230.6 229.6 711.8 * Short interviews are counted as telephone surveys; long interviews as focus groups in Figure 4 and in cost tables in Section 6. Figure 4 Planned Use of Survey Instruments 2000 - 2002 https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 17/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency Past Use of Survey Instruments 35 30 25 20 15 10 5 0 - | | Mail | | Feedback/Cornrnent/Evaluation Forms I | Telephone | | Web Site Surveys n Focus Groups Table 5-2 Planned Survey Averages Survey Type Average Uses/year F eedback/Web-based 72 Mail Surveys 27 Telephone 17 Focus Groups 121.2 Instruments 237.2 5 (c) Small Entity Flexibility. Not applicable. 5 (d) Collection Schedule. This will be dependent upon the needs of each originator of a survey. Schedules for customer surveys will be documented in the package submitted to RID for review. 6. Estimating the Burden and the Cost of the Collection 6(a) Estimating Respondent Burden. https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 18/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency The estimate (Table 6-1) was based on the survey plans of EPA programs. Recent feedback from EPA programs suggests continuing interest in using customer satisfaction surveys as part of the overall long-term strategy of these organizations. The EPA estimates 17,521 hours of respondent burden on the part of 73,512 individuals in FY 2000; 14,034 hours from 63,937 respondents in FY 2001, and 15,054 hours of burden from 68,772 respondents in FY 2002. The EPA program staff planning to use this generic clearance know that burden should be as low as possible in keeping with the Paperwork Reduction Act. Survey designs will be simple, convenient, easy to respond to, and clear in content and purpose. Few long surveys will be designed; most surveys will be of limited scope and require only a short time to complete. Many comments card/feedback forms will be used, and programs will attempt to gather more customer satisfaction information through the use of the Internet. However, several major projects are planned: the Center for Environmental Information and Statistics plans to interview 4,000 individuals for 30 minutes per person in FY 2000, will hold 200 focus groups of 15 people for 3 hours over the three-year period, and expects 5,000 responses annually to its Internet surveys; the Design for the Environment program plans for 50,000 respondents over three years to its annual mail surveys. Figure 4 (page 15) displays the types and number of uses for six types of survey instruments. If programs succeed in their expanded use of Internet for customer satisfaction surveys, burden will be reduced below the level currently projected. Further reductions may be achieved by eliminating some surveys through sharing of results of completed surveys across the Agency. 6(b) Estimating Respondent Costs I Labor Costs See Table 6 -1. Since the respondents represent such a diverse group, wage estimates were based on the July 1999 U.S. Bureau of Statistics weekly earnings of wage and salary workers using the median earnings estimate for middle income families. Weekly earnings are $543; this computes to $13.575/hour. There is no need for "developing, acquiring, or utilizing technology and systems for the purpose of collecting, validating or verifying information," "....disclosing and providing information," "adjusting the existing ways to comply with any previously applicable instructions or requirements," "training personnel to be able to respond to a collection of information," "searching data sources," nor a need for the respondents to keep records. Burden activities include only a few steps: reviewing instructions, responding, and sending (e-mail or mail) responses when the surveys are not performed in person or over the telephone. Table 6-1 displays the annual burden estimates for respondents and total estimated respondent costs. The average burden estimate was derived by dividing the total hours for years one through three (46,084) and dividing by the total respondents (206, 221). The average hours per response is 0.23 hours, or 13.4 minutes per respondent. The average cost per response over the three-year period is estimated($543/40 hours = 13.575/hour/60; ,22/hours X $13,575 = $3.03. II Capital and Operations and Maintenance Costs Not applicable. https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 19/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency III Capital/Start-up vs. Operating and Maintenance (O & M) Costs Not applicable. IV Annualizing Capital Costs Not applicable. 6 (c) Estimating Agency Burden and Cost. Tables 6-2 through 6-6 provide the annual estimates for agency burden associated with developing, disseminating customer surveys and analyzing the results. Wage estimates were divided into three categories of labor: Management (GS-15), Technical (GS-13), and Clerical (GS-7)iŁ2 Rates used in 1997 were increased by 10%. (See Figure 4 and Table 5 -1 for yearly plan totals by survey instrument type.) The Customer Service Program and Regulatory Information staffs will be sharing information and survey instruments across the Agency. Feedback Advisors will also use this information to assist people. Costs per instrument should be reduced as Agency staff members become more experienced with feedback through Internet, and with developing and analyzing surveys of other types. However, since these cost reductions cannot be accurately estimated, aggregate annual costs that follow in Table 6-6 do not reflect these cost reductions to the Agency, nor do they reflect the still unknown burdens EPA is as yet unable to predict for Internet instrument development. Based on the actual use of the ICR during 1994 - 1998 and 1999 to date, the Agency is not likely to issue every survey now planned; some may be recognized as duplicative during internal clearance. Development costs may be reduced if several organizations use the same instrument, or if results of one group's survey can be applied to one or more units within the Agency. Further reductions may be realized as more people use CSP survey software and scanning equipment to perform routine analysis. In the tables presenting the cost estimates, when an instrument is to be used many times (200 focus groups, many uses of a feedback card), development costs will not be reflected for each use. Costs for activities after the survey will be reflected for each use. 6(d) Estimating the Respondent Universe and Total Burden and Costs Burden Table 6-1 provides information on each survey by instrument type, specific issuing office, number of respondents expected, burden per response and burden hours requested per survey. Table 6-7 summarizes the total burden and costs for respondents, and the Agency. Activities have been grouped to reflect the various types of surveys and the total respondents expected for each instrument type. In all cases, the activities performed remain only the time required to read, respond and transmit the survey instruments. Burden estimates were calculated using the median weekly earnings of the nation's 97.6 https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 20/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency million full-time wage and salary workers, $543 in the second quarter of 1999. The Bureau of Labor Statistics of the U.S. Department of Labor reported this figure on July 20, 1999, in its news release "Usual Weekly Earnings of Wage and Salary Workers: Second Quarter 1999." This computes to $13.58 ($13,575) per hour. Table 6-2. Agency Burden/Cost for Telephone Surveys/Short Interviews Activities Manager @ $43 Burden Technical @ $30 Hours Clerical @ $19 Total Hrs. Total Cost Developing survey l._ 4( 20.0 61 $ 1,644.50 Obtaining EPA approval 1.0 4.0 1.0 6.0 182.00 Gathering information 0.0 60.0 20.0 80.0 2,180.00 Reviewing data; follow-ups 0.0 16.0 8.0 24.0 632.00 Analyzing results 2.0 80.0 0.0 82.0 2,486.00 Storing and maintaining results 0.0 4.0 5.0 9.0 215.00 Preparing survey findings 1.0 80.0 8.0 89.0 2,595.00 Totals hours 5.5 $236.50 2S 62.0 3' $9,934.50 Category costs $8,520.00 $1,178.00 9,934.50 Table 6-3. Agency Burden/Cost for Mail Surveys Activities Manager @ $43 Burden Technical @ $30 Hours Clerical @$19 Total Hours Total Cost Developing survey 1. 80.0 8.0 8' $2,616.50 Obtaining EPA approval 1.0 4.0 1.0 6.0 182.00 Gathering information 0.0 40.0 16.0 56.0 1,504.00 Reviewing data 0.0 8.0 8.0 16.0 392.00 https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 21/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency Analyzing results 2.0 40.0 0.0 42.0 1,286.00 Storing and maintaining 0.0 2.0 3.0 5.0 117.00 results 40.0 Preparing survey findings 5.0 8.0 53.0 1,567.00 Totals hours 9.. 214.0 44.0 2( $7,664.50 Category costs $408.50 $6,420.00 $836.00 $7,664.50 Table 6-4. Agency Burden/Cost for Customer Feedback Forms/Internet Screens* Activities Manager @ $43 Burden Technical @ $30 Hours Clerical @ $19 Total Hours Total Cost Developing feedback instruments 1. 20.0 2.0 2: $681.00 Obtaining EPA approval 1.0 4.0 1.0 6.0 182.00 Gathering information 0.0 20.0 16.0 36.0 904.00 Reviewing data 0.0 8.0 8.0 16.0 392.00 Analyzing results 2.0 20.0 0.0 22.0 686.00 Storing and maintaining results 0.0 2.0 3.0 5.0 117.00 Preparing survey findings 2.0 20.0 8.0 30.0 838.00 Total hours Category costs 6.. $258.00 94.0 $2,820.00 38.0 $722.00 13*.11 $3800.00 $3,800.00 * Feedback forms, comment cards, evaluation forms and web-based surveys are grouped into this one category. These costs are used below, though once a card, form or web question list is completed, costs for all but the first use are greatly reduced. Table 6-5. Agency Burden/Cost for Focus Groups/ Longer Interviews* https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 22/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency Activities Manager @ $43 Burden Technical @ $30 Hours Clerical @ $19 Total Hours Total Cost Developing Focus Sessions 1.5 40.0 40.0 8: $2,024.50 Obtaining EPA approval 1.0 4.0 2.0 7.0 201.00 Conducting Focus Groups 0.0 8.0 8.0 16.0 392.00 Reviewing data 0.0 4.0 4.0 8.0 196.00 Analyzing results 1.0 20.0 0.0 21.0 643.00 Storing and maintaining result 0.0 2.0 3.0 5.0 117.00 Preparing findings (129) 3.0 20.0 8.0 31.0 881.00 Totals hours 6.5 98.0 65.0 1< $4,454.50 Category costs $279.50 $2,940.00 $1,235.00 $4,454.50 * Longer interviews are included with focus groups. Table 6-6. Aggregate Agency Table for Annual Burden/Cost Survey Collection Type Annual # of Collections (Avg) Annual Hours/Survey Annual Cost Annual Hours Survey Type Annual Cost Survey Type Telephone V 351.5 $ 9,934.50 5,975.50 $ 168,886.50 7,664.50 Mail 27 267.5 7,222.50 206,941.50 Feedback (cards, electronic & evaluation forms) 3,800.00 Focus Groups* 4,454.50 ($2,225.50 72 138.0 and 2,229.00) 9,936.00 273,600.00 121 (4.3 newperyear) 169.5 (88.5 for instrument 10,181.55 9,569.65 development & approval; 81.0) (380.55; 9,801) 269,709.00 Totals 2: $938,276.30 * See 6(e) 2 for explanation https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 23/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency Table 6-7 Respondent Universe, Total Burden and Costs Survey Type 3 years Surveys Respondents (Thousands) Burden Hours/Survey Total Hours Total Cost M;iil 8: 7f .2 18,283.85 $248,203.26 Telephone 5: 1< .3 6,719.' 91,216.33 Feedback cards, evalua-tions + Web- based 216 104 .09 9,390.50 127,476.03 Focus Groups 3( 7 1. 13,267.30 180,103.59 Totals 7: 2( 46,084.10 $646,999.21 6(e) Bottom Line Burden Hours and Cost Tables I Respondent Tally See Table 6-1. II The Agency Tally Tables 6-8 through 6-11 provide the Agency Tally estimates. The figures do not portray the real situation. Many surveys will be used more than once; therefore development and approval costs should be counted only once per instrument. This is particularly important when calculating the agency costs for focus groups because nearly fifty percent of their cost is in these two activities (development and approval of the instrument). Since there are as many as 200 uses of the same instrument, cost were calculated, based on the number of instruments for those two activities, and for the total uses for the remaining activities. Most other multiple use instruments have only a few uses, and the reduced costs to the agency were not computed. Total EPA tally for the three-year period is $ 2,713,391.50. Table 6-8 Estimated Agency Costs during FY 2000 Survey Collection Type Number of Annual Hours/Survey Annual Cost Annual Hours Collections Survey Type Annual Cost Survey Type https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 24/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency Telephone 24 3i... $ 7,664.50 8,436.0 $ 183,948.00 Mail 29 267.5 7,757.5 222,270.50 Feedback (cards, electronic & 3,800.00 evaluation forms) Focus Groups* 72 127 (ll instruments) 138.0 169.5 (88.5 for instrument development & approval; 81.0) 4,454.50 ($2,225.50 and 2,229.00) 9,936.0 11,260.5 (973.5; 10,287) 273,600.00 24,480.50 283,083.00 Totals 2i $ 987,382.00 Table 6-9 Estimated Agency Costs during FY 2001 Survey Collection Type Number of Collections Annual Hours/Survey Annual Cost Annual Hours Survey Type Annual Cost Survey Type Telephone Mail i: 28 351.5 267.5 $ 9,934.50 7,664.50 4,218.1 7,490.0 $ 119,214.00 214,606.00 Feedback (cards, electronic & evaluation forms) Focus Groups & Interviews 71 138.0 3,800.00 4,454.50 ($2,225.50 and 2,229.00) 9,798.0 269,800.00 120 (2 new instruments) 169.5 (88.5 for instrument development & approval; 81.0) 9,897.0 (177; 9720) 4,451.00 267,480.00 Totals 2': $ 875,551.00 III Variations in the Annual Bottom Line Many organizations have chosen FY 2000 as the only time they plan surveys during the next three years, so the costs are higher for FY 2000. In addition, the majority of Focus group development costs occur in FY 2000. https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 25/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency IV Reasons for Change in Burden Within the Agency, many organizations, including several of the ten Regional Offices and laboratories, have decided to do customer satisfaction surveys during the next three years. They will be trying many techniques at different times, and with varying numbers of respondents and burden per respondent. Burden varies very little considering the variety of techniques and frequencies of use over the three years. A few surveys will be done every two years. This pattern has repeated, so unless additional organizations determine the need to survey, the burden is likely to drop every second year under EPA's Customer Satisfaction generic clearances. Overall, the number of hours will continue to drop as more organizations use web-based surveys and feedback options, rather than longer and more formal survey instruments. The overall number of surveys planned for the 1997 clearance renewal grew dramatically over the number planned under the first generic ICR because the Agency's Customer Service Program publicized the advantages of the generic clearance, the requirements of the Executive Order, and GPRA to establish regular feedback from customers. The CSP also underscored the value of such feedback to overall reinvention efforts. However, reality has now settled in, and many organization realize that they were very ambitious in the plans they submitted for the 1997 application. Clearly, Table 6-10 Estimated Agency Costs during FY 2002 Survey Collection Type Number of Collections Annual Hours/Survey i Annual Cost Annual Hours Survey Type Annual Cost Survey Type Telephone i: 3;... $ 9,934.50 7,664.50 5,272.: $ 149,017.50 Mail 24 267.5 6,420.0 163,248.00 Feedback (cards, electronic & evaluation forms) 3,800.00 Focus Groups & 73 138.0 10,074.0 277,400.00 Interviews 4,454.50 ($2,225.50 and 117 (no new instruments) 169.5 (88.5 for instrument development & approval; 81.0) 2,229.00) 9477.0 260,793.00 Totals 2; $ 850,458.50 EPA staff have planned fewer surveys and fewer uses of their survey instruments, except in the area of focus groups. Repeating the process of 1997, the CSP director invited all Regions and Headquarters offices to work https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 26/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency with her to plan for the next three years of customer satisfaction survey work by participating in the development of this ICR application. The burden hours from year to year are: 2000 - 17,521; 2001 - 14,034 and 2002 - 14,529. Many organizations have chosen FY 2000 as the only time they plan surveys during the next three years. Therefore, the first year has the highest burden hours. V Burden Statement The following statement applies overall to the planned surveys for the next three years: Public reporting burden for this collection of information is estimated to average eleven (11) minutes per response, including the time for reviewing instructions, gathering information, and completing and reviewing the collection of information. Send comments on the Agency's need for this information, the accuracy of the provided burden estimates, and any suggestions for reducing the burden, including the use of automated collection techniques to the Director, OPPE Regulatory Information Division, United States Environmental Protection Agency (Mail Code 2137), 401 M Street, SW, Washington, D.C. 20460; and to the Office of Information & Regulatory Affairs, Office of Management & Budget, 725 17th Street, NW, Washington, DC 20503, Attention: Desk Officer for EPA. Include the EPA ICR number and the OMB control number in any correspondence. 5 https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 27/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency Uses, Respondents & EPA Costs Uses Respondents EPA Costs Mail ~ Phone Feedback Ś Focus Group Thousands EPA CUSTOMER SATISFACTION SURVEYS INFORMATION COLLECTION REQUEST BURDEN TABLE 2000 - 2002 The following table is provided to EPA customers both separately and as an attachment to the ICR submittal to the Office of Management & Budget. https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 28/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency Burden Calculation Hours 2000 2001 2002 Office of Solid Waste & Emergency Response Interns Proaram Overall Feedback form 10 people @ 15 minutes - 150 minutes 2.5 2.5 2.5 Interns orientation Feedback form 20 people @ 7 minutes; 140 minutes 2.33 2.33 2.33 Chemical Emeraencv Plannina & Prevention CCEPPO") Website - online survey 10 minutes/ # persons unknown estimate 50/month; 6,000 minutes 100.0 100.0 100.0 Publications - mail survev 15 documents. 100 resDondents each CI.5001: 10 minutes/survey; 15,000 minutes 250.0 250.0 250.0 Risk Manaaement Plans Software phone survey; 15 minutes/person 100 respondents; 1500 minutes 25 Local Government Reimbursement Proqram mail survey/follow-up telephone contact 100.0 100.0 100.0 with no more than 200 respondents; 30 minutes combined response time; 6,000 minutes Technoloav Innovation Comment card with no more than 200 respondents @ 15 minutes; 3,000 minutes 50.0 50.0 50.0 Jobs Throuah Recvclina Evaluation form for 100 respondents @ 2 minutes/person; 200 minutes 3.33 3.33 3.33 Recvclina Economic Information Proiect Mail Survey to maximum of 200 respondents @ 15 minutes/person; 3,000 minutes https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 29/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency 50.0 50.0 50.0 Hotline telephone survey 300 respondents @ 10 minutes/ person; 3,000 minutes 50.0 50.0 50.0 RCRA Docket mail survey with no more than 300 respondents & 10 minutes each 3,000 minutes 50.0 50.0 50.0 EPA Dockets online survey of no more than 500 respondents @ 10 minute each; 5,000 minutes 83.3 83.3 83.3 Office of Solid Waste website online survey; 1,000 respondents @ 10 minutes; 10,000 minutes 166.6 166.6 166.6 Regions/states trainings/meetings and conferences evaluation form to be used no more than 10 times per year by 30 persons each time at 10 minutes/person; 3000 minutes 50.0 50.0 50.0 OSW Outreach Products feedback cards no more than 10/year; each with 100 respondents @ 5 minutes each; 5,000 minutes 83.3 83.3 83.3 Underaround Storaqe Tanks 3 evaluation forms/comment cards; 200 respondents each; 5 minutes/respondent; 3,000 minutes 50.0 50.0 50.0 Office of Ground Water & Drinking Water Capacity Development Program 2 focus groups of 30 people each; 6 hour/group; 2 telephone surveys of 100 people each; 20 minute; 400 minutes 360 66.6 Consumer Confidence Reports Six Focus Groups of 10 people each; 60 minutes/person; 3600 minutes 60 Publications written for consumers https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 30/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency On-line evaluation for 2 web documents 100 on-line reader respondents/document; 5 minutes/respondent; 2,000 minutes Written survey for mail recipients 100 recipients each of two written documents respondents; 5 minutes/ respondent; 2,000 minutes 16.6 16.6 Safe Drinking Water Hotline Telephone Interview of 100 caller respondents; 5 minutes/caller; 500 minutes 8.3 Safewater Web Site On-line survey of 500 respondents; 3 minutes/respondents; 1500 minutes 25.0 Safewater Web Site and SDWIS data in Envirofacts One-on-One Interviews of 20 people; 1 hour per person 20.0 Source Water Assessment Program Mail survey; 500 respondents; 10 minutes/ respondent; 5,000 minutes 83.3 Source Water Assessment Program Telephone survey; 800 respondents; 10 minutes/respondent; 8,000 minutes 133.3 Source Water Assessment Program Focus groups; 5 groups, 10 people each; 60 minutes/ respondent 50 Source Water Protection Training Sessions Mail survey; 50 respondents; 10 minutes/ respondent; 500 minutes 8.3 Underground Injection Control Video and Print Materials for Class V Telephone Survey; 500 respondents; 20 minutes/ respondent; 1,000 minutes Mail Survey 175 respondents; 20 minutes/respondent; 3500 minutes 16.6 58.3 Office of Research & Development 4 Focus group activities/year - 100 for 2 hours 200.0 200.0 200.0 https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 31/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency - 75 twice annually for 2 hours 150.0 150.0 150.0 - 100 people for 2 hours 200.0 200.0 200.0 Mail Surveys - two per year as follows 100 respondents; 30 minutes/respondent; 3,000 minutes 300 respondents; 30 minutes/respondent; 50 50 50 9,000 minutes 150 150 150 Feedback/Comment Cards - seven per year (number of respondents depends on the # of requests) 60 seconds/ person to "comment" Rough estimate of 100/card (700 minutes) 11.6 11.6 11.6 Internet/ Feedback Screens 2000 - 5; 2001 - 10; 2002 - 15; (number of respondents depends upon number accessing web sites) 10 seconds/person; Rough estimate of 200/screen 2.7 5.5 8.3 Environmental Monitoring & Public Access for Community tracking (EMPACT) telephone interviews; 200 respondents/year; 15 minutes/respondent 50 50 50 Center for Environmental Information and Statistics Pilot Test for face to face survey 500 respondents; 1.5 hours/respondent 750 Face to face survey of 4,000 respondents; .5 hours/respondent 2,000 Pilot test for telephone survey 300 respondents; 30 minutes/respondent 150 Telephone survey of 4,000 respondents; 30 minutes/respondent 2,000 Mail surveys 1 - 1,000 respondents; 18 minutes/respondent; 18,000 minutes 300 https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 32/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency 2. - 1,000 respondents; 18 minutes/respondent; 18,000 minutes 300 Internet questionnaire 5 uses in 3 years 5,000 respondents/year; 10 minutes/ respondent; 50,000 minutes 833.3 833.3 833.3 Focus groups 200 in 3 years; 15 people per group; 3 hours/respondent; 1,000/yr 3,000 3,000 3,000 Evaluation forms for public meetings 15 uses over 3 years; 1,000 respondents/year; 15 minutes/respondent 250 250 250 Evaluation forms for publications 10 uses over 3 years; 500 respondents/form; 5 minutes/ respondent 125 166.6 125 Permitting - Region 2 Mail surveys (Three different surveys will be used: for permittees, the public, and states.) Permittees: 50 respondents/use; 15 minutes/respondent; 750 minutes 12.5 12.5 Public: 100 respondents/use; 15 minutes/respondent; 1,500 minutes States: 3 respondents; 15 minutes/ respondent; 45 minutes 25 .75 25 .75 25 .75 Comment cards Permittees: 50 respondent/use; 10 minutes/respondent; 500 minutes Public: 50 respondents/use; 10 minutes/respondent; 500 minutes 8.3 8.3 8.3 8.3 8.3 Office of Pollution Prevention, Pesticides & Toxics Desian for the Environment CDFE") Evaluation cards; 1,000 respondents; 5 minutes/respondent; 5000 minutes 83.3 83.3 83.3 DFE Garment & Textile Care surveys by mail; total over 3 years 50,000 respondents @15 minutes each; 16,667 people annually 4,166.7 4,166.7 4,166.7 Solid Waste https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 33/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency Superfund Telephone survey; 400 respondents; 10 minutes/respondent; 4,000 minutes 66.6 66.6 Brownfields-local govt mail survey; ) 200 respondents; 10 minutes/respondent; 2,000 minutes 33.3 33.3 DFE training and meetings evaluation form; 400 respondents annually; 10 minutes/respondent 66.6 66.6 66.6 Burden Calculation Hours 2000 2001 2002 Office of Water - Wastewater Management 6 Mail surveys, 2 /year; 1,000 respondents each; 10 minutes/respondent; 20,000 minutes 333.3 333.3 333.3 4 conference evaluation surveys; 50 respondent/survey; 10 minutes/ respondent; 66+ respondents/year; 660 minutes/year 11.1 11.1 11.1 Office of Air Programs - Atmospheric Pollution Hotline Two Telephone Surveys 300 respondents each; 10 minutes per call; 3,000 minutes 100 100 100 Distribution Center Two Telephone Surveys 300 respondents each; 10 minutes per call; 3,000 minutes 50 50 Enerav Star Buildinas Mail Survey; 150 respondents; 5 minutes/survey; 750 minutes 12.5 One Phone or fax survey; 20 respondents; 15 minutes/respondent; 300 minutes 5.0 One mail survey; 250 respondents; 15 minutes/respondent; 3,750 minutes 62.5 Two phone surveys; 100 respondents each; 20 minutes/call; 2,000 minutes 33.3 33.3 Enerav Star Label for Buildinas One Mail Survey 200 respondents; 10 minutes/survey; 2,000 minutes 33.3 https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 34/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency Burden Calculation Hours 2000 2001 2002 Two Telephone Surveys; 150 respondents each; 10 minutes/call; 1500 minutes/survey 25 25 Enerav Star Buildinas Website Three On-line or e-mailed Surveys; 30 respondents for each survey; 10 minutes/survey; 300 minutes/survey 5 Enerav Star Solution 3 Mail Surveys; 50 responding each time; 20 minutes/survey; 1,000 minutes/survey 16.6 16.6 16.6 Methane Enerav Branch (1) One Mail Survey 100 respondents; 15 minutes; 1500 minutes 16.6 New and Existina Homes Proaram 15 Focus Groups of 10 people; 45 minutes/respondent; 5/year 37.5 37.5 37.5 (60) Focus Groups of 20 people; 10 minutes/respondent; 100 minutes/group; 20/year 33.3 33.3 33.3 (60) Focus Groups of 20 people; 1 hour/respondent; 20/year; 8,000 minutes 133.3 133.3 133.3 Office of Air Proarams StratosDheric Protection Division 20 Hotline mail survey; 240 respondents; 5 minutes/respondent; 1,200 minutes Feedback card; 100 respondents/year; 5 minutes/respondent; 500 minutes 8.3 8.3 8.3 Environmental Information Service Center 12.5 12.5 12.5 Telephone survey; 150 respondents; 5 minutes/respondent; 750 minutes Burden Calculation Hours 2000 2001 2002 Office of PartnershiDS & Reaulatorv Assistance Evaluation form; 20 uses; 30 6.6 6.6 6.6 respondents each; 2 minutes; 400 minutes/year Mail survey; 10 uses (3,4,4 uses); 10 respondents/use; 10 minutes; 10,000/3 years 5.0 6.6 5.0 https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 35/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency Feedback card; 5 uses (1, 2, 1 uses); 100 respondents/use; 1 minute/response; 500 minutes/3 years 1.66 3.33 3.33 Office of Administration & Resource Management National Service Center for Environmental Publications annual mail survey; 2,000 respondents; 5 minutes/respondent; 10,000 minutes/3 years 166.6 166.6 166.6 Region 3 Annual report evaluation card; 200 respondents/year; 5 minutes/response; 16.6 16.6 16.6 Annual interview survey @ Philadelphia Flower Show; 500 respondents/year; 10 minutes/respondent; 5,000 minutes 83.3 83.3 83.3 1999 Focus Groups follow-up; 20 Interviews with reporters, educators, small businesses, environmental groups and librarians; 60 minutes/respondent 20.0 Interviews of Congressional staff and top state officials; 10 per year; 1 hour/respondent 10.0 10.0 10.0 25 Branch level mail customer surveys; 100 respondents/survey; 10 minutes/respondent; 25,000 minutes/3 years 138.8 138.8 138.8 Burden Calculation Hours 2000 2001 2002 ChesaDeake Bav Proqram Publications evaluation card; 5/year; 100/card; 10 minutes/respondent; 5,000 minutes 83.3 83.3 83.3 Region 4-20 surveys total External Affairs Correspondence feedback; 100 respondents; 5 minutes/respondent; 500 minutes 8.3 Congressional correspondence 50 respondents; 5 minutes/respondent; 250 minutes 4.16 4.16 Media mail survey; 100 respondents; 10 minutes/respondent; 1,000 minutes 16.6 Environmental groups telephone survey; 25 respondents; 15 minutes/respondent; 375 minutes https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 36/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency 6.25 Regional Customer Service Hotline; 200 respondents; 3 minutes/respondent; 600 minutes 10.0 10.0 Policy & Manaaement FOIA feedback card; 200 respondents; 5 minutes/respondent; 1,000 minutes 16.6 16.6 Personnel applicants feedback; 50 respondents; 5 minutes/respondent; 250 minutes 4.16 Personnel applicants internet feedback; 30 respondents; 5 minutes/respondent; 150 minutes 2.5 Environmental Justice Focus group; 25 respondents; 2-3 hours/respondent 75.0 Burden Calculation Hours 2000 2001 2002 Air. Pesticides & Toxics Mamt Division Pesticide Users Feedback card: 50 respondents; 5 minutes/respondent; 250 minutes 4.16 Air-local govt, mail survey; 200 respondents; 10 minutes/respondent; 2,000 minutes 33.3 33.3 Energy Star Partners Internet feedback; 50 respondents; 5 minutes/respondent; 250 minutes 4.16 4.16 Environmental Accountabilitv Community Based Environmental Protection Focus Group; 25 respondents; 2 hours/respondent 50.0 Environmental Justice Mail survey; 50 respondents; 10 minutes/respondent; 500 minutes 8.3 Project XL Partners Electronic survey; 20 respondents; 5 minutes/respondent; 100 minutes 1.6 Water Surface Water - Permit Applicants Electronic Survey; 50 respondents; 5 minutes/respondent; 250 minutes 4.16 https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 37/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency Wetlands/Coastal Mail survey to public; 30 respondents; 10 minutes/respondent; 300 minutes 5.0 Ground/Drinking Water Municipal Facilities Feedback; 200 respondents; 5 minutes/respondent; 1,000 minutes 16.6 Burden Calculation Hours 2000 2001 2002 Small business environmental assistance products & services mail survey; 500 respondents; 10 minutes 83.3 Small business environmental assistance products & services web site and feedback card survey; 600/year on web; 3 minutes/respondent; for cards , 1000/year 5 minutes. 30 83.3 30 83.3 30 83.3 Office of Enforcement & Compliance Assistance Website surveys 1/year; 500 respondents; 2 minutes/respondent 16.6 16.6 16.6 Mail surveys 1/year; 400 respondents 15 minutes 100 100 100 Telephone survey; 250 respondents 15 minutes 62.5 Feedback cards/evaluation forms 4/year 200/card; 5 minutes/respondent 66.6 66.6 66.6 State/Tribal and Local Grants Process Group Comment cards; 400 respondents; 5 minutes/respondent; 2,000 minutes 33.3 33.3 Region 1 Comment card; 5,000 respondents; 5 minutes/ respondent; 25,000 minutes 416.6 Burden Calculation Hours 2000 2001 2002 Region 6 Feedback cards and evaluation forms; 4/year; 2,000 respondents/year; 7 minutes/respondent; 14,000 minutes/year 700 700 700 https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 38/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency Region 8 Office of Communication & Public Involvement meetina evaluation forms; 20/year; 16.6 16.6 16.6 10 respondents/form; 5 minutes; 1000 minutes Feedback card for FOIA & publication requestors/controlled correspondence respondents; 100 recipients/year; 5 minutes/respondent; 500 minutes 8.3 8.3 8.3 Environmental Information Service Center 12.5 12.5 Telephone surveys; 150 respondents; 5 minutes/respondent; 750 minutes Permittina Evaluation form; 30 respondents; 2 minutes; 60 minutes 1.0 1.0 1.0 Mail survey; 10 uses (3, 4, 3); 10 respondents; 10 minutes; 300, 400, 300 minutes Feedback cards; 5 uses (1, 2, 2); 100 respondents 1 minute; 100, 200, 200 minutes 5.0 1.66 6.67 3.33 5.0 3.33 Region 9 Web site survey; 1,000 respondents at 1 minute; 1,000 minutes 16.6 16.6 16.6 Two Internet focus groups; 10 participants each; 2-hours/ participant 40 40 40 Feedback card for Library and Public Information Center; 3 minutes/respondent; 1,000 respondents; 3,000 minutes 50 50 50 Feedback card general information phone requestors; 3 minutes/respondent; 1,000 respondents; 3,000 minutes. 50 50 50 Feedback card for grant and/or contract recipients; 3 minutes/respondent; 1000 respondents; 3,000 minutes 50 50 50 Internet Site feedback page; 2 minutes/respondent, 1000 respondents; 2,000 minutes 33.3 33.3 33.3 Table 6 -la Response Time Summary https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 39/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency Year Respondents Burden Hours Average Response Time 2000 73,512 17,521 14.1 minutes 2001 63,937 14,034 13.1 minutes 2002 68,772 14,529 12.7 minutes Total 206,221 46,084 13.4 minutes EXHIBITS For copies of Exhibits, please contact Patricia Bonner USEPA _ Customer Service Program 401 M Street SW Washington, DC 20460 Call 202-260-0059 Fax 202-260-4968 e-mail bonner.Datricia@eDa.aov 1. Customer feedback forms/comment cards/evaluation forms are considered to be short, 5 to 15 question forms that typically accompany, and seek feedback for a specific service (such as a training course, or "over the counter" service) or product (such as a manual, software, etc). Internet (web based) surveys also fit into this https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 40/41 ------- 3/21/23, 2:57 PM Environmental Protection Agency category. Mail surveys may involve more extensive questionnaires and may require more rigorous statistical sampling methodology to evaluate a certain group or groups' perceptions about a service the Agency offers. 2. EPA interprets this to preclude any EPA purposes of regulatory development or enforcement. 3. EPA interprets this to mean random sampling of the general public in a "market research" mode. 4. For customer feedback forms and short questionnaires, a one page memorandum should be sufficient. Mail or telephone surveys making use of statistical sampling must include the statistician's name/phone, and a brief description of the statistical aspects of the survey, such as the statistical approach, population coverage, survey design, precision requirement, and pretests/pilot tests. 5. Hourly wages estimates were made using the 1997 figure plus 10%. https://archive.epa.gov/pubiicinvolvement/web/html/1711ss03.html 41/41 ------- |